Compare commits

..
Author SHA1 Message Date
Eelco Dolstra 3c1f09ca49 Bump version 2023-02-27 18:43:40 +01:00
Théophane Hufschmitt f12b1584c0 Merge pull request #7877 from NixOS/backport-7856-to-2.12-maintenance
[Backport 2.12-maintenance] Wait with making /etc unwritable until after build env setup
2023-02-22 06:52:10 +01:00
Yorick van Pelt d13640f3d7 Wait with making /etc unwritable until after build env setup
This fixes /etc/nsswitch.conf

(cherry picked from commit bbba49b3e4)
2023-02-21 19:22:31 +00:00
Robert Hensing 8639f41069 Merge pull request #7855 from NixOS/backport-7831-to-2.12-maintenance
[Backport 2.12-maintenance] Backport 7616 to 2.11 maintenance
2023-02-17 15:05:38 +01:00
Robert Hensing 48b407c23e NarInfoDiskCache: Also test id consistency with updated fields
And clarify test

(cherry picked from commit 19b495a48a)
(cherry picked from commit 26378d285b)
2023-02-17 13:09:46 +00:00
Robert Hensing 112246045f NarInfoDiskCache: Keep BinaryCache.id stable and improve test
Fixes #3898

The entire `BinaryCaches` row used to get replaced after it became
stale according to the `timestamp` column. In a concurrent scenario,
this leads to foreign key conflicts as different instances of the
in-process `state.caches` cache now differ, with the consequence that
the older process still tries to use the `id` number of the old record.

Furthermore, this phenomenon appears to have caused the cache for
actual narinfos to be erased about every week, while the default
ttl for narinfos was supposed to be 30 days.

(cherry picked from commit fb94d5cabd)
(cherry picked from commit 8449b3cac3)
2023-02-17 13:09:46 +00:00
Robert Hensing 6d6a836b31 NarInfoDiskCache: Prepare reproducer for #3898
(cherry picked from commit 2ceece3ef3)
(cherry picked from commit cabf1d9b40)
2023-02-17 13:09:46 +00:00
Robert Hensing f6192b8c23 NarInfoDiskCacheImpl: Make dbPath a parameter
This allows testing with a clean database.

(cherry picked from commit 79f62d2dda)
(cherry picked from commit 4972085195)
2023-02-17 13:09:46 +00:00
Robert Hensing 3b5a341c8c NarInfoDiskCache: Rename cacheExists -> upToDateCacheExists
This is slightly more accurate considering that an outdated record
may exist in the persistent cache. Possibly-outdated records are
quite relevant as they may be foreign keys to more recent information
that we want to keep, but we will not return them here.

(cherry picked from commit 29f0b196f4)
(cherry picked from commit 0bda735c57)
2023-02-17 13:09:46 +00:00
Robert Hensing 26dc468d8a sqlite.cc: Add SQL tracing
Set environment variable NIX_DEBUG_SQLITE_TRACES=1 to log all sql statements.

(cherry picked from commit 8a0ef5d58e)
(cherry picked from commit 54ac8fa919)
2023-02-17 13:09:46 +00:00
Eelco Dolstra b1d7aba400 Merge pull request #7836 from NixOS/backport-7830-to-2.12-maintenance
[Backport 2.12-maintenance] Don't allow writing to /etc
2023-02-14 16:53:01 +01:00
Yorick van Pelt e9ea93797a Make /etc writability conditional on uid-range feature
(cherry picked from commit 49fd72a903)
2023-02-14 15:50:25 +00:00
Yorick van Pelt c0dadbc803 container test: make /etc writable
(cherry picked from commit ad1f61c39b)
2023-02-14 15:50:25 +00:00
Yorick van Pelt 28897300ab Don't allow writing to /etc
(cherry picked from commit db41f74af3)
2023-02-14 15:50:25 +00:00
Robert Hensing 5ce482b329 Merge pull request #7618 from NixOS/backport-6645-to-2.12-maintenance
[Backport 2.12-maintenance] systemd,launchd match nixos open files limit
2023-01-18 12:41:56 +01:00
Artturin d2c82c681a systemd,launchd match nixos open files limit
it was bumped in https://github.com/NixOS/nixpkgs/pull/176558

(cherry picked from commit 2320a2f93e)
2023-01-17 19:23:04 +00:00
TIAN Yuanhao d6c9069454 Avoid poly_user_note_set twice
f06f810 incorrectly introduces a boolean flip, resulting in a senseless
poly_user_note_set even though the user comment has been set correctly.

(cherry picked from commit 09830ab829)
2023-01-10 20:32:52 +01:00
Naïm Favier 21b1a097d6 Allow disabling build users by unsetting build-users-group
Unsetting `build-users-group` (without `auto-allocate-uids` enabled)
gives the following error:

```
src/libstore/lock.cc:25: static std::unique_ptr<nix::UserLock> nix::SimpleUserLock::acquire(): Assertion `settings.buildUsersGroup != ""' failed.
```

Fix the logic in `useBuildUsers` and document the default value
for `build-users-group`.

(cherry picked from commit 1f3c0a3c1d)
2023-01-10 20:32:15 +01:00
Eelco Dolstra 4e6c3c356d Fix a crash in DerivedPath::Built::toJSON() with impure derivations
The use of 'nullptr' here didn't result in a null JSON value, but in a
nullptr being cast to a string, which aborts.

(cherry picked from commit 0687e16c4a)
2023-01-10 20:31:40 +01:00
Steven Shaw e3e1093e8b Fix error message
(cherry picked from commit 84b0893725)
2023-01-10 20:31:01 +01:00
Théophane Hufschmitt c0d64cafda Fix why-depends for CA derivations (again)
This has the same goal as b13fd4c58e81b2b2b0d72caa5ce80de861622610,but
achieves it in a different way in order to not break
`nix why-depends --derivation`.

(cherry picked from commit 8cac451fce)
2023-01-10 20:30:27 +01:00
Théophane Hufschmitt a8f78a5e6d Increase the test coverage of why-depends
- Test with `--derivation`
- Actually test with ca-derivations (was suuposedly done, but not
  activated because of a missing line in `local.mk`)

(cherry picked from commit 6a90ef072c)
2023-01-10 20:30:23 +01:00
Théophane Hufschmitt e381bfafbf Revert "Fix why-depends for CA derivations"
This reverts commit b13fd4c58e.

(cherry picked from commit 105d74eb81)
2023-01-10 20:30:17 +01:00
Alexandre Thomas 934ddd4e46 Fix Nix installation on older versions of fish
The `fish_add_path` function is only available for fish 3.2.0 or newer,
and not on older versions.
This commit adds an alternative way to update the PATH when
`fish_add_path` does not exist.

(cherry picked from commit 49e058f1cf)
2023-01-10 20:30:03 +01:00
Eelco Dolstra 65f15ffe3c Fix deadlock between auto-GC and addTempRoot()
Previously addTempRoot() acquired the LocalStore state lock and waited
for the garbage collector to reply. If the garbage collector is in the
same process (as it the case with auto-GC), this would deadlock as
soon as the garbage collector thread needs the LocalStore state lock.

So now addTempRoot() uses separate Syncs for the state that it
needs. As long at the auto-GC thread doesn't call addTempRoot() (which
it shouldn't), it shouldn't deadlock.

Fixes #3224.

(cherry picked from commit 28d5b5cd45)
2023-01-10 20:15:07 +01:00
Eelco Dolstra 6487d49a16 Move creation of the temp roots file into its own function
This also moves the file handle into its own Sync object so we're not
holding the _state while acquiring the file lock. There was no real
deadlock risk here since locking a newly created file cannot block,
but it's still a bit nicer.

(cherry picked from commit 224b56f10e)
2023-01-10 20:15:02 +01:00
Eelco Dolstra 2d4f6e6a4e On macOS with auto-uid-allocation and sandboxing, use the correct gid
macOS doesn't have user namespacing, so the gid of the builder needs
to be nixbld. The logic got "has sandboxing enabled" confused with
"has user namespaces".

Fixes #7529.

(cherry picked from commit 4e84b532ed)
2023-01-10 20:14:36 +01:00
Eelco Dolstra dff7d605c6 Bump version 2022-12-06 17:24:56 +01:00
Eelco Dolstra ef800f1e73 Mark official release 2022-12-06 14:50:25 +01:00
Eelco Dolstra eece14dce4 Merge pull request #7410 from edolstra/release-notes
Branch 2.12 release notes
2022-12-06 14:30:09 +01:00
Eelco Dolstra 5b4b2eefa1 Release notes 2022-12-06 13:55:09 +01:00
Eelco Dolstra 54906bc93c Merge pull request #7382 from fricklerhandwerk/doc-automatic-uid
move documentation on `auto-allocate-uids` to options docs
2022-12-06 11:31:34 +01:00
Eelco Dolstra 484578d3f9 Tweak option descriptions 2022-12-06 10:30:36 +01:00
Théophane Hufschmitt bfcf30f0ab Merge pull request #7390 from Radvendii/fix-6383
check the store for input before failing (hopefully fix #6700)
2022-12-05 18:04:41 +01:00
Eelco Dolstra a21d9523b9 Merge pull request #7401 from Mic92/cgroup-typo
getMaxCPU: fix cgroup path
2022-12-05 17:53:05 +01:00
Eelco Dolstra a4642ca73e Merge pull request #7392 from edolstra/revert-7251
Revert "docs: drop shell prompt character for consistency"
2022-12-05 17:51:13 +01:00
Taeer Bar-Yam e4f9f3bf24 check the store for input before failing (hopefully fix #6700) 2022-12-05 11:27:47 -05:00
Eelco Dolstra 14c3d76048 Merge pull request #7329 from fricklerhandwerk/maintainers
add maintainers' handbook
2022-12-05 17:21:36 +01:00
Eelco Dolstra 647e60b356 maintainers/README.md: Use CET 2022-12-05 16:35:25 +01:00
Eelco Dolstra 67d76cb8df Fix typo 2022-12-05 16:34:39 +01:00
Théophane Hufschmitt 3b05124682 Merge pull request #7277 from fricklerhandwerk/codeowners
add CODEOWNERS
2022-12-05 14:58:50 +01:00
Jörg Thalheim cccd57c022 getMaxCPU: fix cgroup path
Given this typo I am not sure if it has been tested.
2022-12-04 18:22:12 +01:00
Eelco Dolstra b4b13382f4 Merge pull request #7387 from fricklerhandwerk/upvote-convention
encourage adding 👍 to express interest
2022-12-02 20:13:24 +01:00
Eelco Dolstra c582150360 Merge pull request #7394 from edolstra/fix-7268
Lower verbosity of exceptions in getMaxCPU()
2022-12-02 17:02:32 +01:00
Eelco Dolstra 3a8341f57e Merge pull request #7395 from obsidiansystems/evaluating-to-calling
Change "while evaluating <fun>" to "while *calling*" in trace
2022-12-02 15:53:57 +01:00
John Ericson 19c5394971 Change "while evaluating <fun>" to "while *calling*" in trace
The old way was not correct.

Here is an example:
```
 $ nix-instantiate --eval --expr 'let x = a: throw "asdf"; in x 1' --show-trace
error: asdf

       … while evaluating 'x'

       at «string»:1:9:

            1| let x = a: throw "asdf"; in x 1
             |         ^

       … from call site

       at «string»:1:29:

            1| let x = a: throw "asdf"; in x 1
             |                             ^
```
and yet also:
```
 $ nix-instantiate --eval --expr 'let x = a: throw "asdf"; in x' --show-trace
<LAMBDA>
```

Here is the thing: in both cases we are evaluating `x`!

Nix is a higher-order languages, and functions are a sort of value. When
we write `x = a: ...`, `a: ...` is the expression that `x` is being
defined to be, and that is already a value. Therefore, we should *never*
get an trace that says "while evaluating `x`", because evaluating `a:
...` is *trival* and nothing happens during it!

What is actually happening here is we are applying `x` and evaluating
its *body* with arguments substituted for parameters. I think the
simplest way to say is just "while *calling* `x`", and so that is what I
changed it to.
2022-12-02 09:14:12 -05:00
Eelco Dolstra fa99ef6a87 getMaxCPU(): Lower verbosity level for ignored exceptions
Fixes #7268.
2022-12-02 15:03:40 +01:00
Valentin Gagarin 0ce5742bec Merge pull request #7381 from aakropotkin/doc-ltoa-conflicts
doc: listToAttrs: document repeated keys
2022-12-02 14:19:34 +01:00
Eelco Dolstra 1e6a5d1ff6 Clean up cgroup handling in getMaxCPU()
Also, don't assume in LocalDerivationGoal that cgroups are mounted on
/sys/fs/cgroup.
2022-12-02 12:59:13 +01:00
Eelco Dolstra 1211e59a03 Move cgroup.{cc,hh} to libutil 2022-12-02 12:38:03 +01:00
Eelco Dolstra ac06c8d248 Revert "docs: drop shell prompt character for consistency"
This reverts commit c068cce107 because
it makes the docs *less* consistent.
2022-12-02 11:56:12 +01:00
Valentin Gagarin ca42068bdc Merge pull request #7251 from RelationalAI-oss/sp-docs-uninstall-prompt
docs: drop shell prompt character for consistency
2022-12-01 19:24:09 +01:00
Valentin Gagarin 5b95412328 encourage adding 👍 to express interest
decided on the @NixOS/documentation-team, see NixOS/nix.dev#359 for more information
2022-12-01 19:04:04 +01:00
Alex Ameen ef524013aa doc: listToAttrs: add extra whitespace 2022-12-01 10:32:45 -06:00
Eelco Dolstra 7bba4b343b Merge pull request #7386 from edolstra/fix-json
Really fix 'nix store make-content-addressed --json'
2022-12-01 17:05:18 +01:00
Eelco Dolstra f1e1ba9fe0 Really fix 'nix store make-content-addressed --json'
https://hydra.nixos.org/log/mcgypcf9vj4n8vdmw7lj3l05c899v73w-nix-2.12.0pre20221201_16b03f0-x86_64-unknown-linux-musl.drv
2022-12-01 16:29:09 +01:00
Alex Ameen ec18b7d09b doc: listToAttrs: fix line wrapping 2022-11-30 23:21:09 -06:00
Alex Ameen ad46726546 doc: listToAttrs: document repeated keys 2022-11-30 22:53:41 -06:00
Valentin Gagarin 0ea62670ed move documentation on auto-allocate-uids to options docs
this is where it belongs and can be found together with the other
options.
2022-12-01 04:40:02 +01:00
Valentin Gagarin 16b03f03af Merge pull request #7375 from NixOS/formatting-nitpick
add missing newline, for consistent formatting
2022-12-01 01:58:49 +01:00
Valentin Gagarin 3f881e3378 add missing newline, for consistent formatting 2022-11-30 23:40:53 +01:00
Eelco Dolstra f91dc023f2 Merge pull request #7372 from edolstra/fix-json
nix store make-content-addressed: Fix JSON construction
2022-11-30 14:25:15 +01:00
Eelco Dolstra 0b092bd87f nix store make-content-addressed: Fix JSON construction
Fixes

  error: [json.exception.type_error.301] cannot create object from initializer list

in tests/fetchClosure.sh.
2022-11-30 13:46:33 +01:00
Théophane Hufschmitt 0596bdf3a9 Merge pull request #7342 from fricklerhandwerk/refactor-generate-builtins
refactor rendering documentation of builtins
2022-11-29 15:55:43 +01:00
Eelco Dolstra fbc53e97ed Merge pull request #3600 from NixOS/auto-uid-allocation
Automatic UID allocation
2022-11-29 14:01:42 +01:00
Eelco Dolstra 4f762e2b02 Restore ownership of / for non-uid-range builds 2022-11-29 13:10:53 +01:00
Eelco Dolstra f904f6a66f Merge pull request #7365 from NixOS/fix-6979-nobody-user-in-docker
Add nobody user/group to Nix docker image
2022-11-29 11:43:31 +01:00
Eelco Dolstra af8a32143b Merge pull request #7358 from ncfavier/repl-exit-newline
repl: print a newline on ctrl-D
2022-11-29 11:31:43 +01:00
Eelco Dolstra 4dda67260a Merge pull request #7363 from NixOS/dependabot/github_actions/zeebe-io/backport-action-0.0.9
Bump zeebe-io/backport-action from 0.0.8 to 0.0.9
2022-11-29 11:30:17 +01:00
Rok Garbas 46a6be28be Add nobody user/group to Nix docker image 2022-11-29 10:01:46 +00:00
dependabot[bot] dc61e1028d Bump zeebe-io/backport-action from 0.0.8 to 0.0.9
Bumps [zeebe-io/backport-action](https://github.com/zeebe-io/backport-action) from 0.0.8 to 0.0.9.
- [Release notes](https://github.com/zeebe-io/backport-action/releases)
- [Commits](https://github.com/zeebe-io/backport-action/compare/v0.0.8...v0.0.9)

---
updated-dependencies:
- dependency-name: zeebe-io/backport-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-11-28 22:00:44 +00:00
Eelco Dolstra 7dd3e1fec4 Add example 2022-11-28 22:04:51 +01:00
Eelco Dolstra 67bcb99700 Add a setting for enabling cgroups 2022-11-28 21:54:02 +01:00
Eelco Dolstra ff12d1c1a1 Check that auto-allocated UIDs don't clash with existing accounts 2022-11-28 20:49:17 +01:00
Eelco Dolstra dbf78a7ada Merge pull request #7313 from yorickvP/nlohmann-everywhere
Replace src/libutil/json.cc with nlohmann
2022-11-28 15:03:48 +01:00
Naïm FavierandEelco Dolstra 9b35cc716b use logger->cout
in order to avoid potential problems with the progress bar

Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2022-11-28 14:59:06 +01:00
Naïm Favier 04ec157517 repl: print a newline on ctrl-D 2022-11-28 10:38:23 +01:00
Eelco Dolstra fc14585610 Fix evaluation 2022-11-27 18:58:21 +01:00
Eelco Dolstra f1b5c6876b Add tests for auto-uid-allocation, uid-range and cgroups 2022-11-27 16:38:34 +01:00
Eelco Dolstra cc308ee93d Merge pull request #7353 from edolstra/gc-shutdown
Fix random client failures during GC server shutdown
2022-11-27 13:30:26 +01:00
Eelco Dolstra 5b798f6cae Fix random client failures during GC server shutdown
We need to close the GC server socket before shutting down the active
GC client connections, otherwise a client may (re)connect and get
ECONNRESET. But also handle ECONNRESET for resilience.

Fixes random failures like

  GC socket disconnected
  connecting to '/tmp/nix-shell.y07M0H/nix-test/default/var/nix/gc-socket/socket'
  sending GC root '/tmp/nix-shell.y07M0H/nix-test/default/store/kb5yzija0f1x5xkqkgclrdzldxj6nnc6-non-blocking'
  reading GC root from client: error: unexpected EOF reading a line
  1 store paths deleted, 0.00 MiB freed
  error: reading from file: Connection reset by peer

in gc-non-blocking.sh.
2022-11-27 12:57:18 +01:00
Eelco Dolstra 534332c8a0 Merge pull request #7350 from edolstra/remove-strndup
Don't use GC_STRNDUP
2022-11-26 08:50:54 +01:00
Eelco Dolstra 0b4c4d7434 Don't use GC_STRNDUP
It calls strlen() on the input (rather than simply copying at most
`size` bytes), which can fail if the input is not zero-terminated and
is inefficient in any case.

Fixes #7347.
2022-11-25 22:30:56 +01:00
Valentin Gagarin d6318e1638 refactor rendering documentation of builtins
as in [1], make the document structure visible, like in a template

[1]: 4655563470
2022-11-24 14:15:43 +01:00
Valentin Gagarin 341a807444 the point is setting a default reviewer, not notifications 2022-11-24 11:33:59 +01:00
Théophane Hufschmitt bc9692a6b7 Merge pull request #7337 from Radvendii/why-depends-ca
Fix why-depends for CA derivations
2022-11-23 20:16:14 +01:00
Taeer Bar-Yam bd8571a5c3 add explanation and test 2022-11-23 12:06:47 -05:00
Taeer Bar-Yam b13fd4c58e Fix why-depends for CA derivations
why-depends assumed that we knew the output path of the second argument.
For CA derivations, we might not know until it's built. One way to solve
this would be to build the second installable to get the output path.

In this case we don't need to, though. If the first installable (A)
depends on the second (B), then getting the store path of A will
necessitate having the store path B. The contrapositive is, if the store
path of B is not known (i.e. it's a CA derivation which hasn't been
built), then A does not depend on B.
2022-11-23 11:39:50 -05:00
Eelco Dolstra 2aa3f2e810 Include UID in hex 2022-11-23 17:07:59 +01:00
Eelco Dolstra 989fc8a8b9 Add release notes 2022-11-23 15:24:50 +01:00
Eelco Dolstra 6292d5616e Merge remote-tracking branch 'origin/master' into auto-uid-allocation 2022-11-23 11:16:09 +01:00
Eelco Dolstra 05d0892443 Merge pull request #7328 from edolstra/nix-build-stats
nix build --json: Include build statistics
2022-11-22 14:41:15 +01:00
Eelco Dolstra 96a9511a9c Merge pull request #7333 from fricklerhandwerk/language-overview
fix error in language overview
2022-11-22 11:18:53 +01:00
Valentin Gagarin 52f0c80917 fix error in language overview
it is not possible to antiquote numbers.
2022-11-22 10:36:21 +01:00
Eelco Dolstra 3d23b9d032 SimpleUserLock::getSupplementaryGIDs(): Filter out main gid
This avoids having the user's gid in the supplementary group list as
well.
2022-11-22 10:26:17 +01:00
Eelco Dolstra b37c2d84b6 Always call setgroups()
We shouldn't skip this if the supplementary group list is empty,
because then the sandbox won't drop the supplementary groups of the
parent (like "root").
2022-11-22 10:26:17 +01:00
Eelco Dolstra 02c02ee7c3 Merge pull request #6456 from amjoseph-nixpkgs/seccomp-mips
local-derivation-goal.cc: enable seccomp filters for mips{32,64}
2022-11-21 23:03:00 +01:00
Eelco DolstraandLinus Heckemann c776dfbb35 Use hex for startId
Co-authored-by: Linus Heckemann <git@sphalerite.org>
2022-11-21 18:46:55 +01:00
Valentin Gagarin 44dc5c6c13 reword 2022-11-21 15:13:19 +01:00
a4af966d5d add maintainers' handbook
write down the process we have been developing and following so far.

Co-Authored-By: Théophane Hufschmitt <theophane.hufschmitt@tweag.io>
Co-Authored-By: John Ericson <John.Ericson@Obsidian.Systems>
2022-11-21 14:35:01 +01:00
Eelco Dolstra 9d17ce07e8 AutoUserLock: If sandboxing is disabled, use the build users group
We have to use a gid that has write access to the Nix store.
2022-11-21 12:55:49 +01:00
Eelco Dolstra f0baa5c128 nix build --json: Include build statistics
Example:

  # nix build -L --extra-experimental-features cgroups --impure --expr 'with import <nixpkgs> {}; runCommand "foo" {} "dd if=/dev/urandom bs=1M count=1024 | md5sum; mkdir $out"' --json
  [
    {
      "cpuSystem": 1.911431,
      "cpuUser": 1.214249,
      "drvPath": "/nix/store/xzdqz67xba18hljhycp0hwfigzrs2z69-foo.drv",
      "outputs": {
        "out": "/nix/store/rh9mc9l2gkpq8kn2sgzndr6ll7ffjh6l-foo"
      },
      "startTime": 1669024076,
      "stopTime": 1669024079
    }
  ]
2022-11-21 12:06:01 +01:00
Eelco Dolstra e7a5b76844 Rename derivedPathsWithHintsToJSON -> builtPathsToJSON 2022-11-21 11:56:20 +01:00
Eelco Dolstra 82d5cf2a76 Fix macOS build 2022-11-21 11:45:41 +01:00
Eelco Dolstra 653b32a78f Merge remote-tracking branch 'origin/master' into auto-uid-allocation 2022-11-21 11:33:23 +01:00
Eelco Dolstra ec45f4b82e Fix indentation 2022-11-21 11:12:45 +01:00
Eelco Dolstra 300753d594 nix build --json: Include build statistics
Example:

  # nix build -L --extra-experimental-features cgroups --impure --expr 'with import <nixpkgs> {}; runCommand "foo" {} "dd if=/dev/urandom bs=1M count=1024 | md5sum; mkdir $out"' --json
  [
    {
      "cpuSystem": 1.911431,
      "cpuUser": 1.214249,
      "drvPath": "/nix/store/xzdqz67xba18hljhycp0hwfigzrs2z69-foo.drv",
      "outputs": {
        "out": "/nix/store/rh9mc9l2gkpq8kn2sgzndr6ll7ffjh6l-foo"
      },
      "startTime": 1669024076,
      "stopTime": 1669024079
    }
  ]
2022-11-21 10:49:01 +01:00
Eelco Dolstra f538ee4342 Rename derivedPathsWithHintsToJSON -> builtPathsToJSON 2022-11-21 09:38:08 +01:00
Eelco Dolstra e6b71f84a0 Use cgroup.kill to quickly kill cgroups 2022-11-18 16:59:36 +01:00
Eelco Dolstra fa68eb367e Get CPU stats from the cgroup 2022-11-18 13:40:59 +01:00
Eelco Dolstra 20f66c6889 Indentation 2022-11-18 13:40:48 +01:00
Eelco Dolstra 128910ba23 Separate cgroup support from auto-uid-allocation
The new experimental feature 'cgroups' enables the use of cgroups for
all builds. This allows better containment and enables setting
resource limits and getting some build stats.
2022-11-18 10:39:28 +01:00
Eelco Dolstra f1ab082ac4 createTempDir(): Use std::atomic 2022-11-18 09:37:11 +01:00
Eelco Dolstra f423d4425f Fix segfault in unprivileged mode 2022-11-17 11:56:45 +01:00
Yorick van Pelt 09f00dd4d0 Replace src/libutil/json.cc with nlohmann json generation 2022-11-16 16:50:50 +01:00
Théophane Hufschmitt 62960f3291 Merge pull request #7134 from yorickvP/disable-dbg-on-complete
Temporarily disable the debugger during completion evaluation
2022-11-16 11:28:40 +01:00
Théophane Hufschmitt 60dea270d0 Swallow the error in a more idiomatic way 2022-11-16 10:34:32 +01:00
Théophane Hufschmitt 4bf70b74a7 Merge pull request #7294 from tobim/support-aws-sdk-1.10
libstore: link to aws-crt-cpp
2022-11-15 16:51:09 +01:00
Théophane Hufschmitt 3ade5f5d60 Merge pull request #7283 from hercules-ci/issue-6572
Fix #6572 `requires non-existent output`
2022-11-15 16:24:24 +01:00
Théophane Hufschmitt daf1423a4a Merge pull request #7260 from ncfavier/readFile-scan-references
Restrict `readFile` context to references that appear in the string
2022-11-15 16:22:28 +01:00
Robert Hensing c279ddb18c tests: Reproduce #6572 2022-11-14 18:03:29 +01:00
Robert Hensing 7e162c69fe derivation-goal: Fix requires non-existing output error
It occurred when a output of the dependency was already available,
so it didn't need rebuilding and didn't get added to the
inputDrvOutputs.
This process-related info wasn't suitable for the purpose of finding
the actual input paths for the builder. It is better to do this in
absolute terms by querying the store.
2022-11-14 17:52:55 +01:00
Théophane Hufschmitt cb39e9a99e Test that the result of readFile gets ref-scanned 2022-11-14 16:13:26 +01:00
Eelco Dolstra 0efc314d4d Merge pull request #7295 from Et7f3/pkg-config-lowdown
build: use pkg-config for lowdown
2022-11-14 16:12:09 +01:00
Théophane Hufschmitt 6bf8736517 Add release-notes for the context-restriction in readFile 2022-11-14 15:03:53 +01:00
Théophane Hufschmitt 8b4352d79b Merge remote-tracking branch 'nixos/master' into readFile-scan-references 2022-11-14 15:00:05 +01:00
Et7f3 efadeee8fd build: use pkg-config for lowdown 2022-11-12 23:04:58 +01:00
Tobias Mayer 07f2cb1e8f libstore: link to aws-crt-cpp
This change is needed to support aws-sdk-cpp 1.10 and newer.

I opted not to make this dependent on the sdk version because
the crt dependency has been in the interface of the older
sdk as well, and it was only coincidence that libstore didn't
make use of any privately defined symbols directly.
2022-11-12 14:34:23 +01:00
Théophane Hufschmitt 302ddee749 Merge pull request #7279 from fricklerhandwerk/uninstall
add removing users to uninstall instructions
2022-11-11 14:39:07 +01:00
Valentin Gagarin 2af036e5a3 remove stray comma 2022-11-11 14:01:13 +01:00
Valentin Gagarin bb279257b3 Merge pull request #7229 from Mic92/ci
Auto-assign reviewers by file
2022-11-11 13:39:46 +01:00
Naïm Favier e7ed9ae0c7 Restrict readFile context to references that appear in the string
When calling `builtins.readFile` on a store path, the references of that
path are currently added to the resulting string's context.

This change makes those references the *possible* context of the string,
but filters them to keep only the references whose hash actually appears
in the string, similarly to what is done for determining the runtime
references of a path.
2022-11-11 13:04:34 +01:00
Théophane Hufschmitt 9550b1d519 Merge pull request #7258 from ncfavier/fix-eval-error-fmt
Fix printing of eval errors with two format placeholders
2022-11-10 18:41:16 +01:00
Eelco Dolstra 6c6eff8ac4 Remove the SystemdCgroup feature 2022-11-10 17:24:12 +01:00
Théophane Hufschmitt f225f43076 Merge pull request #7006 from fricklerhandwerk/redirects
manual: generalize anchor redirects
2022-11-09 11:23:26 +01:00
Valentin Gagarin d8781c4fc5 add removing users to uninstall instructions 2022-11-09 01:11:47 +01:00
Valentin Gagarin ffca3e34cb Merge pull request #6906 from fricklerhandwerk/language-overview
add syntax overview from NixOS manual
2022-11-09 00:52:53 +01:00
Valentin Gagarin daedaa197d fix typos in comments 2022-11-09 00:49:34 +01:00
Valentin Gagarin 97a9a8925a add CODEOWNERS 2022-11-08 23:34:58 +01:00
Eelco Dolstra 05d258667d Fix build on macOS 2022-11-08 08:00:29 -08:00
Eelco Dolstra 2fde7e0108 Split auto UID allocation from cgroups
Cgroups are now only used for derivations that require the uid-range
range feature. This allows auto UID allocation even on systems that
don't have cgroups (like macOS).

Also, make things work on modern systems that use cgroups v2 (where
there is a single hierarchy and no "systemd" controller).
2022-11-08 16:03:42 +01:00
Théophane Hufschmitt 37358d0bcf Merge pull request #7257 from Artturin/ca-referencesremove
tests/impure-derivations.sh: remove unknown experimental feature 'ca-…
2022-11-08 09:37:38 +01:00
Eelco Dolstra b378876894 Merge pull request #7264 from patricksjackson/remote-brackets
build-remote: Add brackets to error message
2022-11-07 16:59:29 +01:00
Eelco Dolstra 995f5f2e41 Merge pull request #7267 from fricklerhandwerk/build-task
manual: build action -> build task
2022-11-07 16:58:47 +01:00
Eelco Dolstra 4d2244c757 Merge pull request #7252 from yorickvP/fix-7175
tarfile: set directory mode to at least 0500, don't extract fflags and perms
2022-11-07 16:56:41 +01:00
Valentin Gagarin f7ab93b068 manual: build action -> build task
after discussing this with multiple people, I'm convinced that "build
task" is more precise: a derivation is not an action, but inert until it
is built. also it's easier to pronounce.

proposal: use "build task" for the generic concept "description of how
to derive new files from the contents of existing files". then it will
be easier to distinguish what we mean by "derivation" (a specific data
structure and Nix language value type) and "store derivation" (a
serialisation of a derivation into a file in the Nix store).
2022-11-06 13:28:18 +01:00
Patrick Jackson 907f52c337 build-remote: Add brackets to error message 2022-11-04 10:49:44 -07:00
Eelco Dolstra 40911d7dec Remove stray tab 2022-11-04 13:30:35 +01:00
Eelco Dolstra 6259fd7ea6 Fix indentation 2022-11-04 13:29:39 +01:00
Naïm Favier dad859ba0f Fix printing of eval errors with two format placeholders 2022-11-04 12:41:38 +01:00
Artturin 8e7bbc3c35 tests/impure-derivations.sh: remove unknown experimental feature 'ca-references'
ca-references was stabilized in d589a6aa8a
2022-11-03 21:53:11 +02:00
Eelco Dolstra 499e99d099 Merge pull request #7218 from Gabriella439/headless
Add `--yes` option for Nix installation script
2022-11-03 18:12:43 +01:00
Eelco Dolstra b95faccf03 Merge remote-tracking branch 'origin/master' into auto-uid-allocation 2022-11-03 17:43:40 +01:00
Yorick van Pelt 16f1720fd2 SourceExprCommand: swallow EvalError, add tests for this
Completing things that would error would print an ugly error in
the middle of your command line. Avoid printing this error.
2022-11-03 10:11:28 +01:00
Eelco Dolstra dd1970c233 Merge pull request #7246 from NixOS/dependabot/github_actions/cachix/cachix-action-12
Bump cachix/cachix-action from 11 to 12
2022-11-02 15:35:23 +01:00
Yorick van Pelt 444af85552 Temporarily disable the debugger during completion evaluation
readline is not re-entrant, so entering the debugger from the
completioncallback results in an eventual segfault.

The workaround is to temporarily disable the debugger when searching
for possible completions.
2022-11-02 13:02:24 +01:00
Yorick van Pelt 34ea0e2e7b tarfile: set directory mode to at least 0500, don't extract fflags
We don't need SGID, or any ACL's. We also want to keep every dir +rx.
2022-11-01 16:01:38 +01:00
Sagar Patel c068cce107 docs: drop shell prompt character for consistency
In addition to consistency, the fancy "Copy to clipboard" button on the
website will copy the prompt character. Retaining the prompt character
would mean having to edit each command after pasting in the terminal.
2022-11-01 00:09:00 -04:00
dependabot[bot] 06a6a7959e Bump cachix/cachix-action from 11 to 12
Bumps [cachix/cachix-action](https://github.com/cachix/cachix-action) from 11 to 12.
- [Release notes](https://github.com/cachix/cachix-action/releases)
- [Commits](https://github.com/cachix/cachix-action/compare/v11...v12)

---
updated-dependencies:
- dependency-name: cachix/cachix-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-10-31 22:08:02 +00:00
Eelco Dolstra 0edba4cc1e Merge pull request #7231 from jfroche/fix/savedArgv-access
Move savedArgv into libmain
2022-10-31 16:35:08 +01:00
Théophane Hufschmitt 90ed041677 Merge pull request #7039 from Mic92/libfetchers-variant
libfetchers: avoid api.github.com ratelimit if no github token is set
2022-10-31 11:48:51 +01:00
Jörg Thalheim e00761af73 Also test github flakes when access tokens are provided 2022-10-30 07:10:58 +01:00
Sandro Jäckel 9f1dd0df5b Update test after api.github.com ratelimit avoidance 2022-10-29 21:51:29 +02:00
Jean-François Roche cd86eeb693 Move savedArgv into libmain
`savedArgv` is not accessible by plugins when defined in main binary.
Moving it into one of the nix lib fix the problem.
2022-10-28 12:19:37 +02:00
Jörg Thalheim 48f840cff8 assign fricklerhandwerk for documentation releated PRs 2022-10-27 18:13:24 +02:00
Eelco Dolstra b7e8a3bf4c Merge pull request #7217 from NixOS/restore-nix-build-remote
Fix `nix __build-remote`
2022-10-27 13:33:51 +02:00
Théophane Hufschmitt f8d0193383 Pass the right argv when calling the build hook
Call it as `['nix', '__build-remote', ... ]` rather than the previous
`["__build-remote", "nix __build-remote", ... ]` which seemed to have
been most likely unintended
2022-10-27 11:53:04 +02:00
Eelco Dolstra 9323d139b0 Merge pull request #7174 from agbrooks/git-tag-bug
Prevent fetchGit from using incorrect cached rev for different refs
2022-10-26 16:48:48 +02:00
Théophane Hufschmitt 74cc24f4cf Merge pull request #7191 from jherland/antiquoted-paths
Explain how Nix handles antiquotation of paths
2022-10-26 12:42:59 +02:00
Théophane Hufschmitt 9bff7e8ee2 Fix nix __build-remote
Because of a wrong index, `nix __build-remote` wasn't working.

Fix the index to restore the command (and the build hook).
2022-10-26 11:53:46 +02:00
Théophane Hufschmitt bf2e6bcda3 Merge pull request #7223 from yorickvP/fix-defaultApp
nix run: fix "'defaultApp.x86_64-linux' should have type 'derivation'"
2022-10-26 11:42:59 +02:00
Yorick van Pelt aff6d10934 nix run: fix "'defaultApp.x86_64-linux' should have type 'derivation'" 2022-10-26 10:05:27 +02:00
Gabriella Gonzalez a71e3172af Add --yes option for Nix installation script 2022-10-25 16:43:00 -07:00
Théophane Hufschmitt 899878f77a Merge pull request #7206 from akiekintveld/master
Defer to SSH config files for ForwardAgent option
2022-10-25 20:26:52 +02:00
Eelco Dolstra fed1700754 Merge pull request #7207 from NixOS/flake-type
Mark flakes with ._type = "flake".
2022-10-25 17:31:23 +02:00
Théophane Hufschmitt c7414d48f2 Merge pull request #7190 from patricksjackson/fix-defaults
Fix default values in the manual
2022-10-25 16:48:59 +02:00
Eelco DolstraandRobert Hensing da2c61637b Use _type
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2022-10-25 16:48:31 +02:00
Théophane Hufschmitt b9290a0a5b Merge pull request #7196 from sternenseemann/reenable-eval-okay-eq
tests/lang: re-enable eval-okay-eq.nix tests
2022-10-25 16:41:37 +02:00
Théophane Hufschmitt b154070ab0 Merge pull request #7212 from wentasah/profile-help
Improve --profile description
2022-10-25 10:38:35 +02:00
Michal Sojka a9a868fe6a Improve --profile description
The description of the --profile option talks about the "update" operation.
This is probably meant for operations such as "nix profile install", but the
same option is reused in other subcommands, which do not update the profile,
such as "nix profile {list,history,diff-closures}".

We update the description to make sense in both contexts.
2022-10-24 08:49:46 +02:00
Shea Levy 334fa81d08 Mark flakes with .type = "flake".
Fixes #7186
2022-10-23 06:54:11 -04:00
Austin Kiekintveld 8e7804273c Defer to SSH config files for ForwardAgent option
Currently, Nix passes `-a` when it runs commands on a remote machine via
SSH, which disables agent forwarding. This causes issues when the
`ForwardAgent` option is set in SSH config files, as the command line
operation always overrides those.

In particular, this causes issues if the command being run is `sudo`
and the remote machine is configured with the equivalent of NixOS's
`security.pam.enableSSHAgentAuth` option. Not allowing SSH agent
forwarding can cause authentication to fail unexpectedly.

This can currently be worked around by setting `NIX_SSHOPTS="-A"`, but
we should defer to the options in the SSH config files to be least
surprising for users.
2022-10-22 19:51:22 -05:00
Patrick Jackson b44df13701 manual: update options generation code to use default values 2022-10-20 12:05:04 -07:00
Domen Kožar b3d2a05c59 Merge pull request #7185 from NixOS/dependabot/github_actions/cachix/cachix-action-11
Bump cachix/cachix-action from 10 to 11
2022-10-20 07:44:00 +02:00
sternenseemann c63646b693 tests/lang: re-enable eval-okay-eq.nix tests
This is a really old test case (which was originally written before the
proper Nix syntax). The tested deep comparison behavior was implemented
and reverted soon after due to performance problems, but it has been
restored in today's Nix again (thanks to the derivation comparison
optimization, presumably).
2022-10-19 18:39:26 +02:00
Johan HerlandandValentin Gagarin 1ec8810687 Explain how Nix handles antiquotation of paths
Specifically, explain why Nix does not _re_evaluate paths during a
`nix repl` session. This is a thing that bit me while playing around
with paths and antiquotation in `nix repl` while reading the Nix
language tutorial at https://nix.dev/tutorials/nix-language.

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2022-10-19 11:50:58 +02:00
Eelco Dolstra 95331cb9c9 Merge pull request #7183 from n8henrie/patch-1
Fix typo -- dashes not underscores
2022-10-19 11:21:52 +02:00
dependabot[bot] 0b64bac65c Bump cachix/cachix-action from 10 to 11
Bumps [cachix/cachix-action](https://github.com/cachix/cachix-action) from 10 to 11.
- [Release notes](https://github.com/cachix/cachix-action/releases)
- [Commits](https://github.com/cachix/cachix-action/compare/v10...v11)

---
updated-dependencies:
- dependency-name: cachix/cachix-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-10-19 08:50:25 +00:00
Eelco Dolstra ef77897f1d Merge pull request #7184 from NixOS/dependabot/github_actions/cachix/install-nix-action-18
Bump cachix/install-nix-action from 17 to 18
2022-10-19 10:49:45 +02:00
Eelco Dolstra 4bc70b8216 Merge pull request #7187 from NixOS/path-from-hash-part
Add command 'nix store path-from-hash-part'
2022-10-19 10:48:26 +02:00
Patrick Jackson 11b5cf9561 fix manual defaults 2022-10-18 15:09:17 -07:00
Eelco Dolstra e136d57f26 Implement BinaryCacheStore::queryPathFromHashPart() 2022-10-18 17:48:09 +02:00
Eelco Dolstra 61f89e954a Add command 'nix store path-from-hash-part'
This exposes the Store::queryPathFromHashPart() interface in the CLI.
2022-10-18 16:51:12 +02:00
Théophane Hufschmitt a324e9a5c8 Merge pull request #7132 from stelcodes/fix-fish-manpath
Fix fish shell MANPATH creation
2022-10-18 09:50:05 +02:00
dependabot[bot] dc20395de4 Bump cachix/install-nix-action from 17 to 18
Bumps [cachix/install-nix-action](https://github.com/cachix/install-nix-action) from 17 to 18.
- [Release notes](https://github.com/cachix/install-nix-action/releases)
- [Commits](https://github.com/cachix/install-nix-action/compare/v17...v18)

---
updated-dependencies:
- dependency-name: cachix/install-nix-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-10-17 22:01:48 +00:00
Nathan Henrie af9c9504ca Fix typo -- dashes not underscores 2022-10-17 14:15:32 -06:00
Andrew Brooks a259084c50 Fix #7146
When fetching a non-local git repo by ref (and no rev), don't consider unrelated
cached revs for the same repository.
2022-10-14 18:04:47 -05:00
Andrew Brooks c7059c9b24 Add test for issue 7146 2022-10-14 17:27:13 -05:00
Eelco Dolstra 3093bd3a85 Merge pull request #7168 from NixOS/rosetta-test
Improve Rosetta detection
2022-10-14 17:35:57 +02:00
Eelco Dolstra 9f7877abac Merge pull request #7172 from hercules-ci/libmain-extraStackOverflowHandler
libmain: Make stack overflow handler configurable
2022-10-14 17:12:45 +02:00
Robert Hensing ab4eb39386 libmain: Make the entire stack overflow handler pluggable 2022-10-14 12:53:46 +02:00
Robert Hensing 0d75675787 libmain: Add extraStackOverflowHandler 2022-10-14 12:53:07 +02:00
Eelco Dolstra 285277a61a Remove useless debug statements
We haven't parsed the '-v' command line flags yet when this code executes,
so we can't actually get debug output here.
2022-10-14 00:35:33 -07:00
Eelco Dolstra ddd5503950 Use /usr/bin/true 2022-10-14 00:34:31 -07:00
Eelco Dolstra 8196d4f4e9 Merge pull request #7169 from edolstra/fix-execve-error-msg
Fix error display if execve() in the builder fails
2022-10-14 09:29:31 +02:00
Eelco Dolstra 0359d6d123 Fix error display if execve() in the builder fails
After we've send "\2\n" to the parent, we can't send a serialized
exception anymore. It will show up garbled like

  $ nix-build --store /tmp/nix --expr 'derivation { name = "foo"; system = "x86_64-linux"; builder = "/foo/bar"; }'
  this derivation will be built:
    /nix/store/xmdip0z5x1zqpp6gnxld3vqng7zbpapp-foo.drv
  building '/nix/store/xmdip0z5x1zqpp6gnxld3vqng7zbpapp-foo.drv'...

  ErrorErrorEexecuting '/foo/bar': No such file or directory
  error: builder for '/nix/store/xmdip0z5x1zqpp6gnxld3vqng7zbpapp-foo.drv' failed with exit code 1
2022-10-13 21:35:16 +02:00
Eelco Dolstra 96eb5ef156 Improve Rosetta detection
Turns out that one of those *.plist files can exist even if Rosetta is
not installed. So let's just try to run an x86_64-darwin binary
directly.
2022-10-13 11:46:16 -07:00
Eelco Dolstra 59a304a9a8 Fix clang warnings 2022-10-13 11:25:49 -07:00
Eelco Dolstra 27ed3d0458 Merge pull request #7140 from crawford/chroot-store
Make warning about chroot store location more accurate
2022-10-13 16:35:31 +02:00
Steam Deck User a86916eb72 Make warning about chroot store location more accurate
While trying to use an alternate directory for my Nix installation, I
noticed that nix's output didn't reflect the updated state
directory. This patch corrects that and now prints the warning before
attempting to create the directory (if the directory creation fails,
it wouldn't have been obvious why nix was attempting to create the
directory in the first place).

With this patch, I now get the following warning:

    warning: '/home/deck/.var/app/org.nixos.nix/var/nix' does not
    exist, so Nix will use '/home/deck/.local/share/nix/root' as a
    chroot store
2022-10-12 12:12:12 -07:00
Eelco Dolstra a6239eb570 Merge pull request #7163 from edolstra/misc-category
Move some options into a misc category
2022-10-12 16:10:30 +02:00
Eelco Dolstra eba610956b Move some options into a misc category
This unclutters the per-command options a bit by moving out some
global options.
2022-10-12 15:09:17 +02:00
Eelco Dolstra ae2d330455 Merge branch 'print-common-flags' of github.com:Hoverbear/nix 2022-10-12 13:43:11 +02:00
Eelco Dolstra f3193edd87 Merge pull request #7149 from amjoseph-nixpkgs/pr/intersectAttrs/values
parseDrvName: remove doc/impl discrepancy, add test covering the gap
2022-10-12 10:00:39 +02:00
Adam Joseph 5e24863d5a tests/lang/eval-okay-versions.nix: add test for previous commit
This commit adds a test covering the discrepancy between parseDrvName's
implementation and documentation (the discrepancy was eliminated in the previous
commit).
2022-10-08 17:23:03 -07:00
Adam Joseph 7ef71cd21f src/libexpr/primops.cc: parseDrvName: make documentation follow implementation
The documentation for `parseDrvName` does not agree with the implementation when
the derivation name contains a dash which is followed by something that is
neither a letter nor a digit.  This commit corrects the documentation to agree
with the implementation.
2022-10-08 17:21:34 -07:00
Ana Hobden 069409d167 Print common flags in --help 2022-10-07 09:07:22 -07:00
Valentin Gagarin ac0fb38e8a Merge pull request #6652 from abathur/doc_install_tests 2022-10-05 17:45:42 +02:00
Travis A. EverettandValentin Gagarin e1418430ac Apply suggestions from code review
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2022-10-05 09:52:53 -05:00
Valentin Gagarin 927234cfb2 Merge pull request #6870 from amjoseph-nixpkgs/pr/doc/explain-local-remote-binary-substituter 2022-10-05 09:01:42 +02:00
Stel Abrego 34e3fa3a4a Fix fish shell MANPATH creation
Previously the MANPATH was set even if MANPATH was empty beforehand
which resulted in a MANPATH of only ~/.nix-profile/share/man which
omitted the default man page directory (commonly /opt/local/share/man)
from man page results.
2022-10-03 14:37:28 -07:00
Théophane Hufschmitt 3ae9467d57 Merge pull request #6969 from fricklerhandwerk/refactor-generate-manpage
refactor rendering command documentation to markdown
2022-10-03 15:50:17 +02:00
Jörg Thalheim d9abce4ad4 libfetchers: avoid api.github.com ratelimit if no github token is set
If we don't have any github token, we won't be able to fetch private
repos, but we are also more likely to run into API limits since
we don't have a token. To mitigate this only ever use the github api
if we actually have a token.
2022-10-01 10:38:09 +02:00
Eelco Dolstra 89ca75c9f9 Merge pull request #7080 from squalus/nar-close-file
archive: check close errors when extracting nars
2022-09-30 12:58:28 +02:00
Eelco Dolstra 16e088c5db Merge pull request #7108 from rapenne-s/wording
nix-copy-closure: improve wording
2022-09-30 12:54:25 +02:00
Valentin Gagarin d8bef7358f bring back lost newline 2022-09-30 01:43:57 +02:00
Valentin Gagarin 70eea97742 use more self-explanatory names 2022-09-30 01:43:57 +02:00
Valentin Gagarin 61188cb820 move final template to the top
this allows readers to enter the code starting with what is visible from
the outside, instead of working themselves up from purely technical details.
2022-09-30 01:43:57 +02:00
Valentin Gagarin 0e0f1832de remove superfluous let-in pair 2022-09-30 01:43:57 +02:00
Valentin Gagarin a85df04fcb refactor showOptions
it was quite a pain to manipulate strings just with `builtins`
2022-09-30 01:43:53 +02:00
Valentin Gagarin 4655563470 refactor rendering command documentation to markdown
idea:
- make document structure visible, like in a template
- order functions by descending abstraction
- avoid nested let bindings
2022-09-30 01:42:46 +02:00
Valentin Gagarin 6b56bb4a79 use indented strings where appropriate 2022-09-30 01:42:46 +02:00
Théophane Hufschmitt 82ef3c470b Merge pull request #7114 from rapenne-s/reenable_test
tests/build-dry: re-enable some test
2022-09-29 16:33:12 +02:00
Solene Rapenne 62d53bc8a4 tests/build-dry: re-enable some test 2022-09-28 15:54:45 +02:00
Solene Rapenne 847cd49909 nix-copy-closure: improve wording 2022-09-28 01:05:46 +02:00
Eelco Dolstra 050fcd391b Merge pull request #7105 from SquidDev/hotfix/fish-return
Use exit instead of return in /etc/profile.d/nix-daemon.fish
2022-09-27 15:02:29 +02:00
Jonathan Coates 0259702219 Use exit instead of return in fish profile
Older versions of Fish (such as those bundled with Ubuntu LTS 22.04) do
not support return outside of functions. We need to use the equivalent
exit instead.
2022-09-27 12:59:37 +01:00
Eelco Dolstra ace9f261da Merge pull request #7103 from lheckemann/remove-add-root
nix-build: remove unused --add-root arg
2022-09-27 11:37:09 +02:00
Linus Heckemann 19f3ecd830 nix-build: remove unused --add-root arg
Fixes #1982
2022-09-27 00:07:42 +02:00
Eelco Dolstra dfc3a9afc3 Merge pull request #7043 from edolstra/installer-test
Add an installer test
2022-09-26 17:34:23 +02:00
Eelco Dolstra 71e364c9c7 Quote URLs 2022-09-26 16:57:06 +02:00
Eelco Dolstra eceaf1997c Remove FIXME 2022-09-26 16:54:31 +02:00
Eelco Dolstra 6806f20157 Merge pull request #7089 from NixOS/mention-accept-flake-config-in-warning
Mention `--accept-flake-config` in the related warning
2022-09-26 11:13:02 +02:00
Eelco Dolstra e25a36cdac Merge pull request #7093 from amjoseph-nixpkgs/pr/intersectAttrs/values
src/libexpr/primops.cc: correct definition for intersectAttrs
2022-09-26 11:11:53 +02:00
Adam Joseph dc205c75a2 src/libexpr/primops.cc: correct definition for intersectAttrs
The current definition of `intersectAttrs` is incorrect:

> Return a set consisting of the attributes in the set e2 that also exist in the
> set e1.

Recall that (Nix manual, section 5.1):

> An attribute set is a collection of name-value-pairs (called attributes)

According to the existing description of `intersectAttrs`, the following should
evaluate to the empty set, since no key-value *pair* (i.e. attribute) exists in
both sets:

```
builtins.intersectAttrs { x=3; } {x="foo";}
```

And yet:

```
nix-repl> builtins.intersectAttrs { x=3; } {x="foo";}
{ x = "foo"; }
```

Clearly the intent here was for the *names* of the resulting attribute set to be
the intersection of the *names* of the two arguments, and for the values of the
resulting attribute set to be the values from the second argument.

This commit corrects the definition, making it match the implementation and intent.
2022-09-24 23:51:09 -07:00
Théophane Hufschmitt 371013c08d Merge pull request #7010 from edolstra/ignore-shell
nix develop: Ignore stdenv's $SHELL
2022-09-24 12:55:53 +02:00
Théophane Hufschmitt 3eb185d6d1 Merge pull request #7046 from mkenigs/fix-root-SHELL
dockerImage: fix root shell
2022-09-24 12:53:46 +02:00
Théophane Hufschmitt db29ddd113 Merge pull request #7078 from obsidiansystems/trustworthy-signature
"valid signature" -> "signature by a trusted key"
2022-09-24 12:52:35 +02:00
Théophane Hufschmitt 74276cb354 Merge pull request #7079 from matthewbauer/allow-untrusted-settings
Allow pass max-silent-time and build-poll-interval to daemon untrusted
2022-09-24 12:47:49 +02:00
Théophane Hufschmitt 6a64820cb4 Merge pull request #7082 from rapenne-s/ccache
add ccacheStdenv
2022-09-24 12:45:27 +02:00
Théophane Hufschmitt 1a5d094be7 Mention --accept-flake-config in the related warning
Make sure that people who run Nix in non-interactive mode (and so don't have the possibility to interactively accept the individual flake configuration settings) are aware of this flag.

Fix #7086
2022-09-24 12:11:26 +02:00
Eelco Dolstra cacfc32ee2 Merge pull request #7056 from hiroqn/fix-uname
Fix `uname -m` return value for armv6l/armv7l
2022-09-24 11:13:18 +02:00
Eelco Dolstra c13007f012 Merge pull request #7059 from NixOS/remove-useless-ca-file-message
Remove a useless debug message in filetransfer.cc
2022-09-24 10:39:23 +02:00
60e23c8bae Apply suggestions from code review
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
Co-authored-by: Rune K. Svendsen <runesvend@gmail.com>
2022-09-23 13:57:57 -04:00
Solène Rapenne e04b38f789 add ccacheStdenv
when using ccache, rebuild time has been measured
89% faster while not slowing the speed of cold builds
2022-09-23 11:21:19 +02:00
squalus 223f8dace0 archive: check close errors when extracting nars 2022-09-22 12:50:32 -07:00
Matthew Bauer 6e049ae607 Allow pass max-silent-time and build-poll-interval to daemon untrusted
These settings seem harmless, they control the same polling
functionality that timeout does, but with different behavior. Should
be safe for untrusted users to pass in.
2022-09-22 13:59:16 -05:00
John Ericson a2a8cb10ac Dodge "trusted" vs "trustworthy" by being explicit
Hopefully this is best!
2022-09-22 14:37:52 -04:00
John Ericson 752f967c0f "valid signature" -> "trustworthy signature"
I just had a colleague get confused by the previous phrase for good
reason. "valid" sounds like an *objective* criterion, e.g. and *invalid
signature* would be one that would be trusted by no one, e.g. because it
misformatted or something.

What is actually going is that there might be a signature which is
perfectly valid to *someone else*, but not to the user, because they
don't trust the corresponding public key. This is a *subjective*
criterion, because it depends on the arbitrary and personal choice of
which public keys to trust.

I therefore think "trustworthy" is a better adjective to use. Whether
something is worthy of trust is clearly subjective, and then "trust"
within that word nicely evokes `trusted-public-keys` and friends.
2022-09-22 10:49:31 -04:00
Théophane Hufschmitt f704c2720f Merge pull request #7070 from alyssais/repl-history
nix repl: warn if creating dir for history fails
2022-09-21 13:50:49 +02:00
Alyssa Ross d234d01f01 nix repl: warn if creating dir for history fails
The history is not critical to the functionality of nix repl, so it's
enough to warn here, rather than refuse to start if the directory Nix
thinks the history should live in can't be created.
2022-09-20 11:01:40 +00:00
Théophane Hufschmitt 9dadb5481c Merge pull request #7065 from squalus/schema-corrupt
Improve durability of schema version file writes
2022-09-20 10:55:13 +02:00
hiroqn ca22936853 fix uname -m return value for armv6l/armv7l 2022-09-20 15:36:36 +09:00
squalus 1b595026e1 Improve durability of schema version file writes
- call close explicitly in writeFile to prevent the close exception
  from being ignored
- fsync after writing schema file to flush data to disk
- fsync schema file parent to flush metadata to disk

https://github.com/NixOS/nix/issues/7064
2022-09-19 20:13:30 -07:00
Théophane Hufschmitt 9d860f3467 Merge pull request #7060 from fricklerhandwerk/issue-templates
issue template: fill 'about' field
2022-09-19 10:53:13 +02:00
Valentin Gagarin 4b11c22386 issue template: fill 'about' field
according to [GitHub documentation] some fields are required.
`about` is not listed, but it probably is required.

[GitHub documentation]: https://docs.github.com/en/communities/using-templates-to-encourage-useful-issues-and-pull-requests/syntax-for-issue-forms
2022-09-19 10:05:29 +02:00
Théophane Hufschmitt 0f977bf91e Remove a useless debug message in filetransfer.cc
Remove the `verify TLS: Nix CA file = 'blah'` message that Nix used to print when fetching anything as it's both useless (`libcurl` prints the same info in its logs) and misleading (gives the impression that a new TLS connection is being established which might not be the case because of multiplexing. See #7011 )
2022-09-19 08:42:43 +02:00
Travis A. Everett 84bdb0e3ad address review feedback
Mainly:
- Try to triangulate between narrative that framed this as
  a new/easy process and the need for a reference that will
  not quickly grow stale.
- Fix a ~continuity issue where the text was talking about
  "your Cachix cache" before saying that you'd need to make
  a Cachix cache to enable the installer tests.
- Adopt suggestion on titling, and nest subtitles in the
  installer test section.
2022-09-18 13:02:42 -05:00
Travis A. EverettandValentin Gagarin 4bd52bf6c4 Apply suggestions from code review
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2022-09-17 13:20:11 -05:00
Eelco Dolstra a38a55babe Merge pull request #7052 from amjoseph-nixpkgs/pr/fetch/impure
libexpr/fetchurl.nix: allow __impure fetch
2022-09-16 13:50:50 +02:00
Adam Joseph ad5b09423a release-notes/rl-next.md: note new argument to fetchurl.nix 2022-09-16 01:59:24 -07:00
Adam Joseph fb985f855c fetchurl.nix: change other use of __impure 2022-09-16 01:54:24 -07:00
Adam JosephandEelco Dolstra 673fd21b7c Update src/libexpr/fetchurl.nix
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2022-09-16 08:51:14 +00:00
Théophane Hufschmitt 41f9b74320 Merge pull request #7050 from fricklerhandwerk/issue-templates
add issue template for missing or incorrect documentation
2022-09-16 10:26:18 +02:00
Adam Joseph b3550d9179 libexpr/fetchurl.nix: allow __impure fetch
This commit adds an optional `__impure` parameter to fetchurl.nix, which allows
the caller to use `libfetcher`'s fetcher in an impure derivation.  This allows
nixpkgs' patch-normalizing fetcher (fetchpatch) to be rewritten to use nix's
internal fetchurl, thereby eliminating the awkward "you can't use fetchpatch
here" banners scattered all over the place.

See also: https://github.com/NixOS/nixpkgs/pull/188587
2022-09-16 00:48:01 -07:00
Valentin Gagarin 875a99eaa4 fix markdown rendering quirk
markdown would interpret parentheses as belonging to the first link without escaping.
2022-09-16 09:41:27 +02:00
Valentin Gagarin dc8c0b173c Update doc/manual/src/contributing/hacking.md 2022-09-16 09:36:55 +02:00
Valentin Gagarin 1ae974120a Update doc/manual/src/contributing/hacking.md 2022-09-16 09:36:37 +02:00
Valentin Gagarin 0a4bd9fe88 Update doc/manual/src/contributing/hacking.md 2022-09-16 09:36:30 +02:00
Valentin Gagarin 47fa1087c8 Update doc/manual/src/contributing/hacking.md 2022-09-16 09:36:20 +02:00
Valentin Gagarin 84fb036062 add issue template for missing or incorrect documentation
this allows anyone to create labelled issues for easy filtering.
2022-09-15 18:16:17 +02:00
Eelco Dolstra 0d4bf9c4d8 Fix evaluation 2022-09-15 15:56:46 +02:00
Eelco Dolstra 3dd313a7c2 Add RHEL 9 2022-09-15 15:50:52 +02:00
Eelco Dolstra ba04b5b1d7 Disable Ubuntu 14.04 2022-09-15 14:51:44 +02:00
Eelco Dolstra a96ad2ab25 Add RHEL 7/8 2022-09-15 14:51:10 +02:00
Eelco Dolstra 503f31e2a0 Use libvirt image 2022-09-15 13:28:03 +02:00
Eelco Dolstra ef714aa8a5 Remove pre-release Ubuntu 22.10 2022-09-15 13:25:26 +02:00
Eelco Dolstra 5c8cdb9b60 Add Ubuntu 22.04 LTS 2022-09-15 13:24:17 +02:00
Eelco Dolstra 29aaec1e59 Make cross product of images and tests 2022-09-15 13:15:26 +02:00
Eelco Dolstra fe958a682d Test building 2022-09-15 11:42:10 +02:00
Matthew Kenigsberg 02af02854d dockerImage: fix root shell
Currently root's shell is set to a path that does not exist; this change
sets it to the correct path to bash
2022-09-14 16:20:58 -06:00
Eelco Dolstra cc6e312315 Get Ubuntu 22.10 to work 2022-09-14 19:44:41 +02:00
Eelco Dolstra 906c947ee8 Enable daemon installation test on Fedora 2022-09-14 18:55:45 +02:00
Eelco Dolstra 0a8e666dd6 Add Fedora 36 2022-09-14 18:40:16 +02:00
Eelco Dolstra fae3b4fe8a Add an installer test
This runs the installer in a QEMU VM. Unlike the old installer test
that ran inside a declaratively built RedHat/Debian image, this uses
an image from Vagrant.
2022-09-14 15:41:11 +02:00
Eelco Dolstra 88a45d6149 Merge pull request #7014 from DeterminateSystems/graham/ds-327-fish-support-for-the-nix-installer
Add Fish suport to the Nix installer
2022-09-14 14:46:26 +02:00
Théophane Hufschmitt 88646eea29 Merge pull request #7025 from fricklerhandwerk/issue-templates
issue template: add feature label
2022-09-14 11:00:48 +02:00
Ana Hobden 7194c87dce Add installer_test matrix for shells
Signed-off-by: Ana Hobden <operator@hoverbear.org>
2022-09-13 12:57:29 -04:00
Ana HobdenandGraham Christensen 8ebdbeb257 Add fish suport to installer
Before this patch, installing Nix using the Fish shell did not
work because Fish wasn't configured to add Nix to the PATH. Some
options in #1512 offered workarounds, but they typically involve
extra plugins or packages.

This patch adds native, out-of-the-box support for the Fish shell.

Note that Fish supports a `conf.d` directory, which is intended
for exactly use cases like this: software projects distributing
shell snippets. This patch takes advantage of it. The installer
doesn't append any Nix loader behavior to any Fish config file.
Because of that, the uninstall process is smooth and a reinstall
obliterates the existing nix.fish files that we place instead of
bothering the user with a backup / manual removal.

Both single-user and multi-user cases are covered. It has been
tested on Ubuntu, and a Mac with MacPorts, homebrew, and the
Fish installer pkg.

Closes #1512

Co-authored-by: Graham Christensen <graham@grahamc.com>
2022-09-13 12:56:16 -04:00
Eelco Dolstra b8f334bd85 Merge pull request #7034 from edolstra/trim-descriptions
Trim option descriptions
2022-09-13 17:40:22 +02:00
Eelco Dolstra d365cced4f Trim option descriptions
This removes unintended blank lines in Markdown when the description
is a multiline string literal.
2022-09-13 16:58:32 +02:00
Eelco Dolstra b56906aae0 Merge pull request #7032 from edolstra/stop-progress-bar
RunPager: Stop the progress bar
2022-09-13 16:57:30 +02:00
Eelco Dolstra c6ff33ff5c RunPager: Stop the progress bar
In particular, the progress bar was interfering with 'less' rendering
in '--help' (e.g. run 'nix --help' and hit '/' to search).
2022-09-13 15:29:13 +02:00
Théophane Hufschmitt 0f64bf445a Merge pull request #6994 from agbrooks/master
Prevent tempdir from being GC-ed before addToStoreFromDump completes
2022-09-13 09:23:16 +02:00
Andrew Brooks 565d888e0f Address PR feedback on #6694 2022-09-12 11:33:23 -05:00
Valentin Gagarin 7852609999 issue template: add feature label
each change should be an improvement, a label for that is redundant.
2022-09-12 11:27:25 +02:00
Eelco Dolstra aa4ca8b7e4 Merge pull request #4918 from roberth/installer-pc
installer: Jeeze -> Oh no
2022-09-09 16:49:59 +02:00
Valentin Gagarin dceb4f9dbb Merge pull request #6825 from rapenne-s/nix_shell_example
nix shell: document how to invoke multiple commands from the command …
2022-09-09 13:35:23 +02:00
Valentin Gagarin 1d295e5fe7 Merge pull request #6934 from fricklerhandwerk/revert-architecture
Revert #6420 "Document what Nix *is*" so we can start over with smaller change sets
2022-09-09 13:31:05 +02:00
Valentin Gagarin 8dd5ba2f47 more precise variable types 2022-09-09 09:54:24 +02:00
Théophane Hufschmitt c8e8eea95e Merge pull request #7008 from jonringer/add-command-docs
Docs: Add nix develop --command entry
2022-09-08 13:34:40 +02:00
Valentin Gagarin 548c904d40 optimize performance
remove loops and function calls, modify arrays in place

this makes the whole thing harder to read, and probably only
marginally faster.
2022-09-08 11:58:43 +02:00
Andrew Brooks 84fe75a12a Keep created temp dirs inside store, but protect from GC
Implements the approach suggested by feedback on PR #6994, where
tempdir paths are created in the store (now with an exclusive lock).

As part of this work, the currently-broken and unused
`createTempDirInStore` function is updated to create an exclusive lock
on the temp directory in the store.

The GC now makes a non-blocking attempt to lock any store directories
that "look like" the temp directories created by this function, and if
it can't acquire one, ignores the directory.
2022-09-06 17:48:00 -05:00
Eelco Dolstra ece12a97d9 lockfile -> lock file for consistency 2022-09-06 19:20:31 +02:00
Eelco Dolstra 27be54ca53 nix develop: Ignore stdenv's $SHELL
Stdenv sets this to a bash that doesn't have readline/completion
support, so running 'nix (develop|shell)' inside a 'nix develop' gives
you a crippled shell. So let's just ignore the derivation's $SHELL.

This could break interactive use of build phases that use $SHELL, but
they appear to be fairly rare.
2022-09-06 18:27:39 +02:00
Eelco Dolstra 9600109879 Merge pull request #6995 from matthewbauer/disable-sa-restart-on-macos
Disable SA_RESTART for some signals on macOS
2022-09-06 17:36:31 +02:00
Jonathan Ringer 6ce2e96c88 Docs: Add nix develop --command entry
Add example of nix develop being used to execuate a series of script
commands. This is common when doing things like CI/CD, and should be
represented in the official documentation.

Also useful for people looking for the 'nix develop' equivalent of
'nix-shell --run'.

Related:
 - https://github.com/NixOS/nix/issues/6908
 - https://github.com/NixOS/nix/issues/6908#issuecomment-1229266853
2022-09-06 08:18:29 -07:00
Matthew Bauer 77d3d3d18d Remove some signals 2022-09-06 09:47:53 -05:00
Eelco Dolstra 7f31c08224 Merge pull request #7002 from tweag/reset-timestamps-in-installer
Installer: Reset the timestamps in the tarball
2022-09-06 13:19:56 +02:00
Valentin Gagarin 59be1e500a generalize anchor redirects
renaming section headers and changing manually set `id`s will break URLs
in the wild.

this change allows keeping track of all changes to ensure backwards
compatibility.
2022-09-06 00:49:16 +02:00
Théophane Hufschmitt 089f584173 Merge pull request #6980 from virusdave/patch-1
Set `HOME` var to `root`'s home when running `nix-store` as `root`
2022-09-05 16:21:05 +02:00
Théophane Hufschmitt 3fca5f6c69 Installer: Reset the timestamps in the tarball
Otherwise it isn't reproducible.

Fix https://github.com/NixOS/nix/issues/7001
2022-09-05 14:44:01 +02:00
Théophane Hufschmitt df5159e7d3 Merge pull request #6993 from matthewbauer/dont-read-derivation-in-goal
Don’t readDerivation if impure derivations feature is disabled
2022-09-05 11:20:58 +02:00
Matthew Bauer a47b5476e1 Add more signals 2022-09-03 16:06:33 -05:00
Matthew Bauer 102434e4cb Disable SA_RESTART for some signals on macOS
Disables the SA_RESTART behavior on macOS which causes:

> Restarting of pending calls is requested by setting the SA_RESTART bit
> in sa_flags. The affected system calls include read(2), write(2),
> sendto(2), recvfrom(2), sendmsg(2) and recvmsg(2) on a communications
> channel or a slow device (such as a terminal, but not a regular file)
> and during a wait(2) or ioctl(2).

From: https://man.openbsd.org/sigaction#SA_RESTART

This being set on macOS caused a bug where read() calls to the daemon
socket were blocking after a SIGINT was received. As a result,
checkInterrupt was never reached even though the signal was received
by the signal handler thread.

On Linux, SA_RESTART is disabled by default. This probably effects
other BSDs but I don’t have the ability to test it there right now.
2022-09-03 00:27:16 -05:00
Andrew Brooks 1f041ac54f Prevent tempdir from being GC-ed before addToStoreFromDump has renamed it
This fixes issue 6823 by placing the tempdir used in LocalStore::addToStoreFromDump
outside the Nix store, where automatic GC is no longer a concern.
2022-09-02 18:32:35 -05:00
Eelco Dolstra 0c7f213c87 Merge pull request #6990 from matthewbauer/no-space-in-zsh
Don’t add a space after attrs completion in zsh
2022-09-02 19:01:52 +02:00
Matthew Bauer 4894e567fb Don’t readDerivation if impure derivations feature is disabled
readDerivation is pretty slow, and while it may not be significant for
some use cases, on things like ghc-nix where we have thousands of
derivations is really slows things down.

So, this just doesn’t do the impure derivation check if the impure
derivation experimental feature is disabled. Perhaps we could cache
the result of isPure() and keep the check, but this is a quick fix to
for the slowdown introduced with impure derivations features in 2.8.0.
2022-09-02 11:46:34 -05:00
Matthew Bauer dd3cd1a167 Apply suggestions from code review 2022-09-02 10:50:02 -05:00
Eelco Dolstra a9af12e658 Merge pull request #6987 from matthewbauer/update-boehmgc-coroutine-sp-fallback-for-darwin
Update boehmgc-coroutine-sp-fallback.diff for darwin
2022-09-02 13:22:55 +02:00
Matthew Bauer bd63ae7e18 Don’t add a space after attrs completion in zsh
This matches the behavior of bash. We don’t want to add a space after
completion on attrs. Uses -S.

Switches to new compadd style comppletions instead of _describe.
Shouldn’t have any negative issues from what I can tell.
2022-09-01 23:16:55 -05:00
Adam Joseph e90f2fcfc7 glossary: add entry for chroot store (used 11 times in nix) 2022-09-01 18:28:05 -07:00
Adam Joseph e6f5352e71 #binary-cache -> #gloss-binary-cache 2022-09-01 18:27:00 -07:00
Adam Joseph 1ab913467e linkify mention of other options 2022-09-01 18:03:35 -07:00
Adam Joseph 59dc8346ca move substituter signature-checking conditions to configuration file documentation 2022-09-01 17:51:56 -07:00
Adam Joseph f6c750e8b2 glossary: remove extraneous comma 2022-09-01 17:48:34 -07:00
Adam Joseph 887e922be2 glossary: outputHash, not __outputHash 2022-09-01 17:47:13 -07:00
Adam Joseph d5e064d816 glossary: fix broken link 2022-09-01 17:46:31 -07:00
Adam Joseph 57f12df5e4 input-addressed store object: include FODOs 2022-09-01 15:09:10 -07:00
Adam Joseph 8139bbe2ba implement https://github.com/NixOS/nix/pull/6870#pullrequestreview-1093700220 2022-09-01 15:01:07 -07:00
Adam Joseph def4fb9a0f __outputHash: add link 2022-09-01 14:47:33 -07:00
Adam Joseph 1b2b8c39fd fix link to language/advanced-attributes.md#adv-attr-contentAddressed 2022-09-01 14:47:17 -07:00
Adam Joseph 0a98d564b3 glossary: resolve FIXME in #gloss-fixed-output-derivation 2022-09-01 14:41:27 -07:00
Adam Joseph 1f56b5d772 doc/manual: un-inline definitions from substitute 2022-09-01 14:13:21 -07:00
Adam Joseph 41153f30bd glossary: substituter: merge output-addressed cases 2022-09-01 14:13:21 -07:00
Adam JosephandJohn Ericson 9cb8412143 Update doc/manual/src/glossary.md
Co-authored-by: John Ericson <git@JohnEricson.me>
2022-09-01 20:41:04 +00:00
Adam JosephandJohn Ericson 2812682ebe Update doc/manual/src/glossary.md
Co-authored-by: John Ericson <git@JohnEricson.me>
2022-09-01 20:40:39 +00:00
Adam Joseph 520587b9a0 glossary: local store: clarify 2022-09-01 13:39:48 -07:00
Matthew Bauer f4d7208e23 Update boehmgc-coroutine-sp-fallback.diff for darwin
The darwin_stop_world implementation is slightly different. sp goes to
altstack_lo instead of lo in this case. Assuming that is an
implementation detail.

But the fix is the same, when we detect alstack_lo outside of the
expected stack range, we reset it to hi - stack_limit.

Here stack_limit is calculated with pthread_get_stacksize_np since
that is the BSD equivalent to pthread_attr_getstacksize.
2022-09-01 11:48:50 -05:00
Eelco Dolstra 4823067247 Merge pull request #6983 from edolstra/revert-6621
Revert "Merge pull request #6621 from Kha/nested-follows"
2022-09-01 16:07:17 +02:00
Eelco Dolstra a8b3d777fb Revert "Merge pull request #6621 from Kha/nested-follows"
This reverts commit c530cda345, reversing
changes made to 4adcdff5c1.
2022-09-01 15:26:19 +02:00
Théophane Hufschmitt c530cda345 Merge pull request #6621 from Kha/nested-follows
Fix nested flake input overrides
2022-09-01 12:04:00 +02:00
Rok Garbas 4adcdff5c1 Merge pull request #6973 from Enzime/skip-docker-push
Only push Docker image when Docker secrets are set
2022-08-31 23:52:37 +02:00
Dave Nicponski a2b7baa42f Set HOME var to root's home when running nix-store as root
A [recent-ish change](https://github.com/NixOS/nix/pull/6676) logs a warning when a potentially counterintuitive situation happens.

This now causes the multi-user installer to [emit a warning](https://github.com/NixOS/nixpkgs/issues/189043) when it's doing
the "seed the Nix database" step via a low-level `nix-store --load-db` invocation.

`nix-store` functionality implementations don't actually use profiles or channels or homedir as far as i can tell.  So why are we 
hitting this code at all?  

Well, the current command approach for functionality here builds a [fat `nix` binary](https://github.com/NixOS/nix/blob/master/src/nix/local.mk#L23-L26) which has _all_ the functionality of
previous individual binaries (nix-env, nix-store, etc) bundled in, then [uses the invocation name](https://github.com/NixOS/nix/blob/master/src/nix/main.cc#L274-L277) to select the
set of commands to expose.  `nix` itself has this behavior, even when just trying to parse the (sub)command and arguments:

```
dave @ davembp2
$ nix
error: no subcommand specified
Try 'nix --help' for more information.

dave @ davembp2
$ sudo nix
warning: $HOME ('/Users/dave') is not owned by you, falling back to the one defined in the 'passwd' file
error: no subcommand specified
Try 'nix --help' for more information.

dave @ davembp2
$ HOME=~root sudo nix
error: no subcommand specified
Try 'nix --help' for more information.
```

This behavior can also be seen pretty easily with an arbitrary `nix-store` invocation:
```
dave @ davembp2 
$ nix-store --realize

dave @ davembp2 
$ sudo nix-store --realize  # what installer is doing now
warning: $HOME ('/Users/dave') is not owned by you, falling back to the one defined in the 'passwd' file

dave @ davembp2
$ sudo HOME=~root nix-store --realize  # what this PR effectively does

dave @ davembp2
$ 
```
2022-08-31 17:25:26 -04:00
Valentin Gagarin f35b3aa47b do not use unwarranted pkgs in example 2022-08-31 09:16:37 +02:00
Michael Hoang 85248543b5 Only push Docker image when Docker secrets are set 2022-08-29 23:59:57 +10:00
Eelco Dolstra ddb82ffda9 Merge pull request #6972 from sashashura/patch-1
GitHub Workflows security hardening
2022-08-29 09:29:51 +02:00
Eelco Dolstra 1f2b12b30e Merge pull request #6968 from fricklerhandwerk/man-no-weblinks
do not render relative links in help pages
2022-08-29 08:46:35 +02:00
Alex 90ba893329 Update backport.yml 2022-08-28 21:59:29 +01:00
Sebastian Ullrich 6f65c11780 Fix overlapping flake follows 2022-08-28 11:50:25 +02:00
Sebastian Ullrich 2b9d381301 Fix nested flake input overrides 2022-08-28 11:47:25 +02:00
Valentin Gagarin e7dcacbd7c do not render relative links in help pages
this simplifies the setup a lot, and avoids weird looking `./file.md`
links showing up.

it also does not show regular URLs any more. currently the command
reference only has few of them, and not showing them in the offline
documentation is hopefully not a big deal.

instead of building more special-case solutions, clumsily preprocessing
the input, or issuing verbal rules on dealing with URLs, should better
be solved sustainably by not rendering relative links in `lowdown`:

https://github.com/kristapsdz/lowdown/issues/105
2022-08-26 17:10:41 +02:00
Eelco Dolstra 7918adbb62 Merge pull request #6954 from winterqt/darwin-sandbox-trustd
fix(libstore): allow access to trustd on macOS
2022-08-26 11:45:00 +02:00
Eelco Dolstra 0b25446f2e Merge pull request #6961 from edolstra/bump-version
Bump version
2022-08-25 12:30:09 +02:00
Eelco Dolstra 57cf36f81e Bump version 2022-08-25 11:50:14 +02:00
Winter 8e5659423e fix(libstore): allow access to trustd on macOS 2022-08-24 13:09:44 -04:00
Valentin Gagarin d8e54d19f7 Revert "Merge pull request #6420 from nix-community/doc-what-is-nix"
This reverts commit 81e101345f, reversing
changes made to 7d1280bbaf.
2022-08-22 12:52:15 +02:00
Valentin Gagarin 71e9c28694 reword introduction to overview 2022-08-15 11:15:03 +02:00
Valentin Gagarin bc315326fa fix whitespace to please markdown
keep some indentation to ease source readability
2022-08-15 11:13:05 +02:00
Valentin Gagarin 6ba8d6dc82 add more examples on string interpolation 2022-08-15 11:06:40 +02:00
Valentin Gagarin c209e6e108 add more list examples 2022-08-15 10:18:56 +02:00
Valentin Gagarin 0378531bf2 add curried function 2022-08-15 10:18:56 +02:00
Valentin Gagarin 2e4704ca93 add second @ pattern example 2022-08-15 10:18:56 +02:00
Valentin Gagarin 5c25bdee50 add null 2022-08-15 10:18:56 +02:00
Valentin Gagarin 391fd10b12 add home path 2022-08-15 10:18:56 +02:00
Valentin Gagarin 587ae9ada5 add search path 2022-08-15 10:18:56 +02:00
Valentin Gagarin 21438acc70 add absolute path 2022-08-15 10:18:56 +02:00
Valentin Gagarin e6f7c180de add floating point number 2022-08-15 10:18:56 +02:00
Valentin Gagarin 292cab039d add multi-line string 2022-08-15 10:18:56 +02:00
Valentin Gagarin f165a8ae08 flarify relative path semantics 2022-08-15 10:18:56 +02:00
Valentin Gagarin 43188d3d18 make hash and version distinguishable as placeholder 2022-08-15 10:18:56 +02:00
Valentin Gagarin 90836397d3 remove stale section link 2022-08-15 10:18:56 +02:00
Valentin Gagarin 3378a3bce8 add syntax overview from NixOS manual
taken verbatim to keep track of required corrections.

made it an HTML table to more easily change structure and keep diffs
minimal.
2022-08-15 10:18:56 +02:00
Adam JosephandValentin Gagarin bac1e1bf8c Update doc/manual/src/glossary.md
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2022-08-08 17:52:31 +00:00
Travis A. EverettandValentin Gagarin 9b7bd2dd1f Apply suggestions from code review
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2022-08-08 10:04:27 -05:00
Travis A. EverettandValentin Gagarin b5d85f0922 Apply suggestions from code review
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2022-08-05 13:49:18 -05:00
Adam Joseph 1d3b92e80c move package-management/terminology into glossary.md 2022-08-05 10:39:43 -07:00
Adam Joseph 4de95f7f56 gesture at explanation of why binary caches exist 2022-08-05 10:33:48 -07:00
Adam JosephandAttila Gulyas d5506aa712 Update doc/manual/src/package-management/terminology.md
Co-authored-by: Attila Gulyas <toraritte@gmail.com>
2022-08-05 17:19:52 +00:00
Adam JosephandAttila Gulyas 66a93a76b9 Update doc/manual/src/package-management/terminology.md
Co-authored-by: Attila Gulyas <toraritte@gmail.com>
2022-08-05 17:15:37 +00:00
Adam Joseph 2eb74c918d derivations do not need to be signed 2022-08-05 10:13:41 -07:00
Adam Joseph aae771cad2 !implement https://github.com/NixOS/nix/pull/6870#discussion_r938912244 2022-08-05 10:12:46 -07:00
Adam Joseph 1b97f3872e !fixup: transposed characters 2022-08-04 14:22:14 -07:00
Adam Joseph 62674659ed !fixup capitalize Nix 2022-08-04 14:21:17 -07:00
Adam Joseph 8f44d24c52 !fixup whitespace 2022-08-04 14:19:25 -07:00
Adam Joseph 56d4fc194b fourth trust condition: FODs 2022-08-04 14:14:24 -07:00
Adam Joseph a5be5e0120 doc/manual: define {local,remote} store, binary cache, substituter
Nix veterans intuitively know what the following terms mean.  They are
used in several places in the nix documentation, but never defined:

- local store
- remote store
- binary cache
- substituter

In particular, I found the last two terms to be confusingly similar.
Let's give definitions for them.
2022-08-04 14:07:06 -07:00
Solène RapenneandEelco Dolstra 92bae33ca5 nix shell: example shouldn't use an absolute path for the shell
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2022-07-21 14:25:07 +02:00
Solène Rapenne 64404220f5 nix shell: document how to invoke multiple commands from the command line 2022-07-20 14:54:46 +02:00
Travis A. Everett edfcc8256e doc: add install test info to hacking.md 2022-06-11 13:30:51 -05:00
Adam Joseph 49119072e7 local-derivation-goal.cc: seccomp filters for MIPS secondary arch/abi
A mips64el Linux MIPS kernel can execute userspace code using any of
three ABIs:

  mips64el-linux-*abin64
  mips64el-linux-*abin32
    mipsel-linux-*

The first of these is the native 64-bit ABI, and the only ABI with
64-bit pointers; this is sometimes called "n64".  The last of these is
the old legacy 32-bit ABI, whose binaries can execute natively on
32-bit MIPS hardware; this is sometimes called "o32".

The second ABI, "n32" is essentially the 64-bit ABI with 32-bit
pointers and address space.  Hardware 64-bit integer/floating
arithmetic is still allowed, as well as the much larger mips64
register set and more-efficient calling convention.

Let's enable seccomp filters for all of these.  Likewise for big
endian (mips64-linux-*).
2022-04-27 21:39:26 -07:00
Théophane Hufschmitt 47dec825c5 Merge pull request #6181 from obsidiansystems/auto-uid-allocation
Auto uid allocation -- update with latest master
2022-03-01 13:58:17 +01:00
John Ericson dc92b01885 Merge remote-tracking branch 'upstream/master' into auto-uid-allocation 2022-02-28 23:54:20 +00:00
Eelco Dolstra 79152e307e Merge pull request #5212 from mkenigs/auto-uid-allocation
Merge master into #3600
2021-09-15 20:33:44 +02:00
Matthew Kenigsberg 3b82c1a5fe Merge remote-tracking branch 'upstream/master' into auto-uid-allocation 2021-09-15 11:58:06 -05:00
Robert Hensing e30d1daf26 installer: Jeeze -> Oh no
Even if it doesn't offend me or probably most people,
this word can be considered mildly blasphemous.
2021-06-16 19:14:16 +02:00
John Ericson e023c985d5 Merge remote-tracking branch 'upstream/master' into auto-uid-allocation 2021-04-27 19:06:58 +00:00
John Ericson d3cfc14e3a Merge remote-tracking branch 'upstream/master' into auto-uid-allocation 2020-10-17 19:31:13 +00:00
John Ericson 801e6d96d8 Merge commit 'd334fd48824b41b57e267cd2926fa9619b7718e3' into auto-uid-allocation 2020-10-17 19:31:06 +00:00
John Ericson 2c9a8e7421 Merge commit '97ffc1e0139e124b7e36b5d1a62b90300f231118' into auto-uid-allocation 2020-10-17 19:20:18 +00:00
John Ericson 81c5f754a7 Merge commit '6aa64627c8e431c3b187f7bb44c943d06e39b929' into auto-uid-allocation 2020-10-17 19:12:59 +00:00
John Ericson 2546c63373 Merge commit 'f66bbd8c7bb1472facf8917e58e3cd4f6ddfa1b5' into auto-uid-allocation 2020-10-17 17:25:17 +00:00
Eelco Dolstra 7349f257da Only mount /sys in uid-range builds
Maybe this should be a separate system feature... /sys exposes a lot
of impure info about the host system.
2020-07-06 13:50:33 +02:00
Eelco Dolstra 8c4cce553c Fix macOS build 2020-07-06 13:50:33 +02:00
Eelco Dolstra ba50c3efa3 Add "uid-range" and "systemd-cgroup" system features
"uid-range" provides 65536 UIDs to a build and runs the build as root
in its user namespace. "systemd-cgroup" allows the build to mount the
systemd cgroup controller (needed for running systemd-nspawn and NixOS
containers).

Also, add a configuration option "auto-allocate-uids" which is needed
to enable these features, and some experimental feature gates.

So to enable support for containers you need the following in
nix.conf:

  experimental-features = auto-allocate-uids systemd-cgroup
  auto-allocate-uids = true
  system-features = uid-range systemd-cgroup
2020-07-06 13:50:33 +02:00
Eelco Dolstra 570c443f56 Simplify cgroup creation 2020-07-06 13:50:33 +02:00
Eelco Dolstra 7bdcf43b40 Destroy the cgroup prior to building 2020-07-06 13:50:33 +02:00
Eelco Dolstra ca2f64bcda Reduce # of UIDs per build to 65536
2^18 was overkill. The idea was to enable multiple containers to run
inside a build. However, those containers can use the same UID range -
we don't really care about perfect isolation between containers inside
a build.
2020-07-06 13:50:33 +02:00
Eelco Dolstra f5fa3de759 Run builds in their own cgroup
Also, run builds in a cgroup namespace (ensuring /proc/self/cgroup
doesn't leak information about the outside world) and mount /sys. This
enables running systemd-nspawn and thus NixOS containers in a Nix
build.
2020-07-06 13:50:33 +02:00
Eelco Dolstra c3e0a68c7e canonicalisePathMetaData(): Support a UID range 2020-07-06 13:50:33 +02:00
Eelco Dolstra 836573a9a2 Dynamically allocate UIDs
Rather than rely on a nixbld group, we now allocate UIDs/GIDs
dynamically starting at a configurable ID (872415232 by default).

Also, we allocate 2^18 UIDs and GIDs per build, and run the build as
root in its UID namespace. (This should not be the default since it
breaks some builds. We probably should enable this conditional on a
requiredSystemFeature.) The goal is to be able to run (NixOS)
containers in a build. However, this will also require some cgroup
initialisation.

The 2^18 UIDs/GIDs is intended to provide enough ID space to run
multiple containers per build, e.g. for distributed NixOS tests.
2020-07-06 13:50:33 +02:00
164 changed files with 3848 additions and 2397 deletions

No files matched your search

+15
View File
@@ -0,0 +1,15 @@
# Pull requests concerning the listed files will automatically invite the respective maintainers as reviewers.
# This file is not used for denoting any kind of ownership, but is merely a tool for handling notifications.
#
# Merge permissions are required for maintaining an entry in this file.
# For documentation on this mechanism, see https://help.github.com/articles/about-codeowners/
# Default reviewers if nothing else matches
* @edolstra @thufschmitt
# This file
.github/CODEOWNERS @edolstra
# Public documentation
/doc @fricklerhandwerk
*.md @fricklerhandwerk
+4
View File
@@ -30,3 +30,7 @@ A clear and concise description of what you expected to happen.
**Additional context**
Add any other context about the problem here.
**Priorities**
Add :+1: to [issues you find important](https://github.com/NixOS/nix/issues?q=is%3Aissue+is%3Aopen+sort%3Areactions-%2B1-desc).
+5 -1
View File
@@ -2,7 +2,7 @@
name: Feature request
about: Suggest an idea for this project
title: ''
labels: improvement
labels: feature
assignees: ''
---
@@ -18,3 +18,7 @@ A clear and concise description of any alternative solutions or features you've
**Additional context**
Add any other context or screenshots about the feature request here.
**Priorities**
Add :+1: to [issues you find important](https://github.com/NixOS/nix/issues?q=is%3Aissue+is%3Aopen+sort%3Areactions-%2B1-desc).
@@ -0,0 +1,31 @@
---
name: Missing or incorrect documentation
about: Help us improve the reference manual
title: ''
labels: documentation
assignees: ''
---
## Problem
<!-- describe your problem -->
## Checklist
<!-- make sure this issue is not redundant or obsolete -->
- [ ] checked [latest Nix manual] \([source])
- [ ] checked [open documentation issues and pull requests] for possible duplicates
[latest Nix manual]: https://nixos.org/manual/nix/unstable/
[source]: https://github.com/NixOS/nix/tree/master/doc/manual/src
[open documentation issues and pull requests]: https://github.com/NixOS/nix/labels/documentation
## Proposal
<!-- propose a solution -->
## Priorities
Add :+1: to [issues you find important](https://github.com/NixOS/nix/issues?q=is%3Aissue+is%3Aopen+sort%3Areactions-%2B1-desc).
@@ -5,3 +5,7 @@ Please include relevant [release notes](https://github.com/NixOS/nix/blob/master
**Testing**
If this issue is a regression or something that should block release, please consider including a test either in the [testsuite](https://github.com/NixOS/nix/tree/master/tests) or as a [hydraJob]( https://github.com/NixOS/nix/blob/master/flake.nix#L396) so that it can be part of the [automatic checks](https://hydra.nixos.org/jobset/nix/master).
**Priorities**
Add :+1: to [pull requests you find important](https://github.com/NixOS/nix/pulls?q=is%3Aopen+sort%3Areactions-%2B1-desc).
+5
View File
@@ -0,0 +1,5 @@
---
# This files is used by https://github.com/marketplace/actions/auto-assign-reviewer-by-files
# to assign maintainers
"doc/**/*":
- fricklerhandwerk
+12
View File
@@ -0,0 +1,12 @@
name: "Auto Assign"
on:
- pull_request
jobs:
assign_reviewer:
runs-on: ubuntu-latest
steps:
- uses: shufo/auto-assign-reviewer-by-files@v1.1.4
with:
config: ".github/assign-by-files.yml"
token: ${{ secrets.GITHUB_TOKEN }}
+7 -1
View File
@@ -2,9 +2,15 @@ name: Backport
on:
pull_request_target:
types: [closed, labeled]
permissions:
contents: read
jobs:
backport:
name: Backport Pull Request
permissions:
# for zeebe-io/backport-action
contents: write
pull-requests: write
if: github.repository_owner == 'NixOS' && github.event.pull_request.merged == true && (github.event_name != 'labeled' || startsWith('backport', github.event.label.name))
runs-on: ubuntu-latest
steps:
@@ -15,7 +21,7 @@ jobs:
fetch-depth: 0
- name: Create backport PRs
# should be kept in sync with `version`
uses: zeebe-io/backport-action@v0.0.8
uses: zeebe-io/backport-action@v0.0.9
with:
# Config README: https://github.com/zeebe-io/backport-action#backport-action
github_token: ${{ secrets.GITHUB_TOKEN }}
+26 -21
View File
@@ -9,7 +9,7 @@ permissions: read-all
jobs:
tests:
needs: [check_cachix]
needs: [check_secrets]
strategy:
matrix:
os: [ubuntu-latest, macos-latest]
@@ -19,33 +19,37 @@ jobs:
- uses: actions/checkout@v3
with:
fetch-depth: 0
- uses: cachix/install-nix-action@v17
- uses: cachix/install-nix-action@v18
- run: echo CACHIX_NAME="$(echo $GITHUB_REPOSITORY-install-tests | tr "[A-Z]/" "[a-z]-")" >> $GITHUB_ENV
- uses: cachix/cachix-action@v10
if: needs.check_cachix.outputs.secret == 'true'
- uses: cachix/cachix-action@v12
if: needs.check_secrets.outputs.cachix == 'true'
with:
name: '${{ env.CACHIX_NAME }}'
signingKey: '${{ secrets.CACHIX_SIGNING_KEY }}'
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
- run: nix --experimental-features 'nix-command flakes' flake check -L
check_cachix:
check_secrets:
permissions:
contents: none
name: Cachix secret present for installer tests
name: Check Cachix and Docker secrets present for installer tests
runs-on: ubuntu-latest
outputs:
secret: ${{ steps.secret.outputs.secret }}
cachix: ${{ steps.secret.outputs.cachix }}
docker: ${{ steps.secret.outputs.docker }}
steps:
- name: Check for Cachix secret
- name: Check for secrets
id: secret
env:
_CACHIX_SECRETS: ${{ secrets.CACHIX_SIGNING_KEY }}${{ secrets.CACHIX_AUTH_TOKEN }}
run: echo "::set-output name=secret::${{ env._CACHIX_SECRETS != '' }}"
_DOCKER_SECRETS: ${{ secrets.DOCKERHUB_USERNAME }}${{ secrets.DOCKERHUB_TOKEN }}
run: |
echo "::set-output name=cachix::${{ env._CACHIX_SECRETS != '' }}"
echo "::set-output name=docker::${{ env._DOCKER_SECRETS != '' }}"
installer:
needs: [tests, check_cachix]
if: github.event_name == 'push' && needs.check_cachix.outputs.secret == 'true'
needs: [tests, check_secrets]
if: github.event_name == 'push' && needs.check_secrets.outputs.cachix == 'true'
runs-on: ubuntu-latest
outputs:
installerURL: ${{ steps.prepare-installer.outputs.installerURL }}
@@ -54,8 +58,8 @@ jobs:
with:
fetch-depth: 0
- run: echo CACHIX_NAME="$(echo $GITHUB_REPOSITORY-install-tests | tr "[A-Z]/" "[a-z]-")" >> $GITHUB_ENV
- uses: cachix/install-nix-action@v17
- uses: cachix/cachix-action@v10
- uses: cachix/install-nix-action@v18
- uses: cachix/cachix-action@v12
with:
name: '${{ env.CACHIX_NAME }}'
signingKey: '${{ secrets.CACHIX_SIGNING_KEY }}'
@@ -64,8 +68,8 @@ jobs:
run: scripts/prepare-installer-for-github-actions
installer_test:
needs: [installer, check_cachix]
if: github.event_name == 'push' && needs.check_cachix.outputs.secret == 'true'
needs: [installer, check_secrets]
if: github.event_name == 'push' && needs.check_secrets.outputs.cachix == 'true'
strategy:
matrix:
os: [ubuntu-latest, macos-latest]
@@ -73,7 +77,7 @@ jobs:
steps:
- uses: actions/checkout@v3
- run: echo CACHIX_NAME="$(echo $GITHUB_REPOSITORY-install-tests | tr "[A-Z]/" "[a-z]-")" >> $GITHUB_ENV
- uses: cachix/install-nix-action@v17
- uses: cachix/install-nix-action@v18
with:
install_url: '${{needs.installer.outputs.installerURL}}'
install_options: "--tarball-url-prefix https://${{ env.CACHIX_NAME }}.cachix.org/serve"
@@ -87,21 +91,22 @@ jobs:
- run: exec fish -c "nix-instantiate -E 'builtins.currentTime' --eval"
docker_push_image:
needs: [check_cachix, tests]
needs: [check_secrets, tests]
if: >-
github.event_name == 'push' &&
github.ref_name == 'master' &&
needs.check_cachix.outputs.secret == 'true'
needs.check_secrets.outputs.cachix == 'true' &&
needs.check_secrets.outputs.docker == 'true'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0
- uses: cachix/install-nix-action@v17
- uses: cachix/install-nix-action@v18
- run: echo CACHIX_NAME="$(echo $GITHUB_REPOSITORY-install-tests | tr "[A-Z]/" "[a-z]-")" >> $GITHUB_ENV
- run: echo NIX_VERSION="$(nix --experimental-features 'nix-command flakes' eval .\#default.version | tr -d \")" >> $GITHUB_ENV
- uses: cachix/cachix-action@v10
if: needs.check_cachix.outputs.secret == 'true'
- uses: cachix/cachix-action@v12
if: needs.check_secrets.outputs.cachix == 'true'
with:
name: '${{ env.CACHIX_NAME }}'
signingKey: '${{ secrets.CACHIX_SIGNING_KEY }}'
+1 -1
View File
@@ -1 +1 @@
2.11.2
2.12.2
+32
View File
@@ -1,3 +1,35 @@
diff --git a/darwin_stop_world.c b/darwin_stop_world.c
index 3dbaa3fb..36a1d1f7 100644
--- a/darwin_stop_world.c
+++ b/darwin_stop_world.c
@@ -352,6 +352,7 @@ GC_INNER void GC_push_all_stacks(void)
int nthreads = 0;
word total_size = 0;
mach_msg_type_number_t listcount = (mach_msg_type_number_t)THREAD_TABLE_SZ;
+ size_t stack_limit;
if (!EXPECT(GC_thr_initialized, TRUE))
GC_thr_init();
@@ -407,6 +408,19 @@ GC_INNER void GC_push_all_stacks(void)
GC_push_all_stack_sections(lo, hi, p->traced_stack_sect);
}
if (altstack_lo) {
+ // When a thread goes into a coroutine, we lose its original sp until
+ // control flow returns to the thread.
+ // While in the coroutine, the sp points outside the thread stack,
+ // so we can detect this and push the entire thread stack instead,
+ // as an approximation.
+ // We assume that the coroutine has similarly added its entire stack.
+ // This could be made accurate by cooperating with the application
+ // via new functions and/or callbacks.
+ stack_limit = pthread_get_stacksize_np(p->id);
+ if (altstack_lo >= altstack_hi || altstack_lo < altstack_hi - stack_limit) { // sp outside stack
+ altstack_lo = altstack_hi - stack_limit;
+ }
+
total_size += altstack_hi - altstack_lo;
GC_push_all_stack(altstack_lo, altstack_hi);
}
diff --git a/pthread_stop_world.c b/pthread_stop_world.c
index 4b2c429..1fb4c52 100644
--- a/pthread_stop_world.c
+18 -14
View File
@@ -1,16 +1,20 @@
with builtins;
with import ./utils.nix;
builtinsDump:
let
showBuiltin = name:
let
inherit (builtinsDump.${name}) doc args;
in
''
<dt id="builtins-${name}">
<a href="#builtins-${name}"><code>${name} ${listArgs args}</code></a>
</dt>
<dd>
builtins:
${doc}
</dd>
'';
listArgs = args: builtins.concatStringsSep " " (map (s: "<var>${s}</var>") args);
in
with builtins; concatStringsSep "\n" (map showBuiltin (attrNames builtinsDump))
concatStrings (map
(name:
let builtin = builtins.${name}; in
"<dt id=\"builtins-${name}\"><a href=\"#builtins-${name}\"><code>${name} "
+ concatStringsSep " " (map (s: "<var>${s}</var>") builtin.args)
+ "</code></a></dt>"
+ "<dd>\n\n"
+ builtin.doc
+ "\n\n</dd>"
)
(attrNames builtins))
+102 -86
View File
@@ -1,99 +1,115 @@
{ command, renderLinks ? false }:
{ toplevel }:
with builtins;
with import ./utils.nix;
let
showCommand =
{ command, def, filename }:
''
**Warning**: This program is **experimental** and its interface is subject to change.
''
+ "# Name\n\n"
+ "`${command}` - ${def.description}\n\n"
+ "# Synopsis\n\n"
+ showSynopsis { inherit command; args = def.args; }
+ (if def.commands or {} != {}
then
let
categories = sort (x: y: x.id < y.id) (unique (map (cmd: cmd.category) (attrValues def.commands)));
listCommands = cmds:
concatStrings (map (name:
"* "
+ (if renderLinks
then "[`${command} ${name}`](./${appendName filename name}.md)"
else "`${command} ${name}`")
+ " - ${cmds.${name}.description}\n")
(attrNames cmds));
in
"where *subcommand* is one of the following:\n\n"
# FIXME: group by category
+ (if length categories > 1
then
concatStrings (map
(cat:
"**${toString cat.description}:**\n\n"
+ listCommands (filterAttrs (n: v: v.category == cat) def.commands)
+ "\n"
) categories)
+ "\n"
else
listCommands def.commands
+ "\n")
else "")
+ (if def ? doc
then def.doc + "\n\n"
else "")
+ (let s = showOptions def.flags; in
if s != ""
then "# Options\n\n${s}"
else "")
;
showCommand = { command, details, filename, toplevel }:
let
result = ''
> **Warning** \
> This program is **experimental** and its interface is subject to change.
# Name
`${command}` - ${details.description}
# Synopsis
${showSynopsis command details.args}
${maybeSubcommands}
${maybeDocumentation}
${maybeOptions}
'';
showSynopsis = command: args:
let
showArgument = arg: "*${arg.label}*" + (if arg ? arity then "" else "...");
arguments = concatStringsSep " " (map showArgument args);
in ''
`${command}` [*option*...] ${arguments}
'';
maybeSubcommands = if details ? commands && details.commands != {}
then ''
where *subcommand* is one of the following:
${subcommands}
''
else "";
subcommands = if length categories > 1
then listCategories
else listSubcommands details.commands;
categories = sort (x: y: x.id < y.id) (unique (map (cmd: cmd.category) (attrValues details.commands)));
listCategories = concatStrings (map showCategory categories);
showCategory = cat: ''
**${toString cat.description}:**
${listSubcommands (filterAttrs (n: v: v.category == cat) details.commands)}
'';
listSubcommands = cmds: concatStrings (attrValues (mapAttrs showSubcommand cmds));
showSubcommand = name: subcmd: ''
* [`${command} ${name}`](./${appendName filename name}.md) - ${subcmd.description}
'';
maybeDocumentation = if details ? doc then details.doc else "";
maybeOptions = if details.flags == {} then "" else ''
# Options
${showOptions details.flags toplevel.flags}
'';
showOptions = options: commonOptions:
let
allOptions = options // commonOptions;
showCategory = cat: ''
${if cat != "" then "**${cat}:**" else ""}
${listOptions (filterAttrs (n: v: v.category == cat) allOptions)}
'';
listOptions = opts: concatStringsSep "\n" (attrValues (mapAttrs showOption opts));
showOption = name: option:
let
shortName = if option ? shortName then "/ `-${option.shortName}`" else "";
labels = if option ? labels then (concatStringsSep " " (map (s: "*${s}*") option.labels)) else "";
in trim ''
- `--${name}` ${shortName} ${labels}
${option.description}
'';
categories = sort builtins.lessThan (unique (map (cmd: cmd.category) (attrValues allOptions)));
in concatStrings (map showCategory categories);
in squash result;
appendName = filename: name: (if filename == "nix" then "nix3" else filename) + "-" + name;
showOptions = flags:
processCommand = { command, details, filename, toplevel }:
let
categories = sort builtins.lessThan (unique (map (cmd: cmd.category) (attrValues flags)));
in
concatStrings (map
(cat:
(if cat != ""
then "**${cat}:**\n\n"
else "")
+ concatStrings
(map (longName:
let
flag = flags.${longName};
in
" - `--${longName}`"
+ (if flag ? shortName then " / `-${flag.shortName}`" else "")
+ (if flag ? labels then " " + (concatStringsSep " " (map (s: "*${s}*") flag.labels)) else "")
+ " \n"
+ " " + flag.description + "\n\n"
) (attrNames (filterAttrs (n: v: v.category == cat) flags))))
categories);
cmd = {
inherit command;
name = filename + ".md";
value = showCommand { inherit command details filename toplevel; };
};
subcommand = subCmd: processCommand {
command = command + " " + subCmd;
details = details.commands.${subCmd};
filename = appendName filename subCmd;
inherit toplevel;
};
in [ cmd ] ++ concatMap subcommand (attrNames details.commands or {});
showSynopsis =
{ command, args }:
"`${command}` [*option*...] ${concatStringsSep " "
(map (arg: "*${arg.label}*" + (if arg ? arity then "" else "...")) args)}\n\n";
parsedToplevel = builtins.fromJSON toplevel;
manpages = processCommand {
command = "nix";
details = parsedToplevel;
filename = "nix";
toplevel = parsedToplevel;
};
processCommand = { command, def, filename }:
[ { name = filename + ".md"; value = showCommand { inherit command def filename; }; inherit command; } ]
++ concatMap
(name: processCommand {
filename = appendName filename name;
command = command + " " + name;
def = def.commands.${name};
})
(attrNames def.commands or {});
tableOfContents = let
showEntry = page:
" - [${page.command}](command-ref/new-cli/${page.name})";
in concatStringsSep "\n" (map showEntry manpages) + "\n";
in
let
manpages = processCommand { filename = "nix"; command = "nix"; def = builtins.fromJSON command; };
summary = concatStrings (map (manpage: " - [${manpage.command}](command-ref/new-cli/${manpage.name})\n") manpages);
in
(listToAttrs manpages) // { "SUMMARY.md" = summary; }
in (listToAttrs manpages) // { "SUMMARY.md" = tableOfContents; }
+5 -5
View File
@@ -11,16 +11,16 @@ concatStrings (map
+ concatStrings (map (s: " ${s}\n") (splitLines option.description)) + "\n\n"
+ (if option.documentDefault
then " **Default:** " + (
if option.value == "" || option.value == []
if option.defaultValue == "" || option.defaultValue == []
then "*empty*"
else if isBool option.value
then (if option.value then "`true`" else "`false`")
else if isBool option.defaultValue
then (if option.defaultValue then "`true`" else "`false`")
else
# n.b. a StringMap value type is specified as a string, but
# this shows the value type. The empty stringmap is "null" in
# JSON, but that converts to "{ }" here.
(if isAttrs option.value then "`\"\"`"
else "`" + toString option.value + "`")) + "\n\n"
(if isAttrs option.defaultValue then "`\"\"`"
else "`" + toString option.defaultValue + "`")) + "\n\n"
else " **Default:** *machine-specific*\n")
+ (if option.aliases != []
then " **Deprecated alias:** " + (concatStringsSep ", " (map (s: "`${s}`") option.aliases)) + "\n\n"
+2 -2
View File
@@ -50,7 +50,7 @@ $(d)/src/SUMMARY.md: $(d)/src/SUMMARY.md.in $(d)/src/command-ref/new-cli
$(d)/src/command-ref/new-cli: $(d)/nix.json $(d)/generate-manpage.nix $(bindir)/nix
@rm -rf $@
$(trace-gen) $(nix-eval) --write-to $@ --expr 'import doc/manual/generate-manpage.nix { command = builtins.readFile $<; renderLinks = true; }'
$(trace-gen) $(nix-eval) --write-to $@ --expr 'import doc/manual/generate-manpage.nix { toplevel = builtins.readFile $<; }'
$(d)/src/command-ref/conf-file.md: $(d)/conf-file.json $(d)/generate-options.nix $(d)/src/command-ref/conf-file-prefix.md $(bindir)/nix
@cat doc/manual/src/command-ref/conf-file-prefix.md > $@.tmp
@@ -96,7 +96,7 @@ doc/manual/generated/man1/nix3-manpages: $(d)/src/command-ref/new-cli
if [[ $$name = SUMMARY ]]; then continue; fi; \
printf "Title: %s\n\n" "$$name" > $$tmpFile; \
cat $$i >> $$tmpFile; \
lowdown -sT man -M section=1 $$tmpFile -o $(DESTDIR)$$(dirname $@)/$$name.1; \
lowdown -sT man --nroff-nolinks -M section=1 $$tmpFile -o $(DESTDIR)$$(dirname $@)/$$name.1; \
rm $$tmpFile; \
done
@touch $@
+421 -327
View File
@@ -1,330 +1,424 @@
// Redirects from old DocBook manual.
var redirects = {
"#part-advanced-topics": "advanced-topics/advanced-topics.html",
"#chap-tuning-cores-and-jobs": "advanced-topics/cores-vs-jobs.html",
"#chap-diff-hook": "advanced-topics/diff-hook.html",
"#check-dirs-are-unregistered": "advanced-topics/diff-hook.html#check-dirs-are-unregistered",
"#chap-distributed-builds": "advanced-topics/distributed-builds.html",
"#chap-post-build-hook": "advanced-topics/post-build-hook.html",
"#chap-post-build-hook-caveats": "advanced-topics/post-build-hook.html#implementation-caveats",
"#part-command-ref": "command-ref/command-ref.html",
"#conf-allow-import-from-derivation": "command-ref/conf-file.html#conf-allow-import-from-derivation",
"#conf-allow-new-privileges": "command-ref/conf-file.html#conf-allow-new-privileges",
"#conf-allowed-uris": "command-ref/conf-file.html#conf-allowed-uris",
"#conf-allowed-users": "command-ref/conf-file.html#conf-allowed-users",
"#conf-auto-optimise-store": "command-ref/conf-file.html#conf-auto-optimise-store",
"#conf-binary-cache-public-keys": "command-ref/conf-file.html#conf-binary-cache-public-keys",
"#conf-binary-caches": "command-ref/conf-file.html#conf-binary-caches",
"#conf-build-compress-log": "command-ref/conf-file.html#conf-build-compress-log",
"#conf-build-cores": "command-ref/conf-file.html#conf-build-cores",
"#conf-build-extra-chroot-dirs": "command-ref/conf-file.html#conf-build-extra-chroot-dirs",
"#conf-build-extra-sandbox-paths": "command-ref/conf-file.html#conf-build-extra-sandbox-paths",
"#conf-build-fallback": "command-ref/conf-file.html#conf-build-fallback",
"#conf-build-max-jobs": "command-ref/conf-file.html#conf-build-max-jobs",
"#conf-build-max-log-size": "command-ref/conf-file.html#conf-build-max-log-size",
"#conf-build-max-silent-time": "command-ref/conf-file.html#conf-build-max-silent-time",
"#conf-build-repeat": "command-ref/conf-file.html#conf-build-repeat",
"#conf-build-timeout": "command-ref/conf-file.html#conf-build-timeout",
"#conf-build-use-chroot": "command-ref/conf-file.html#conf-build-use-chroot",
"#conf-build-use-sandbox": "command-ref/conf-file.html#conf-build-use-sandbox",
"#conf-build-use-substitutes": "command-ref/conf-file.html#conf-build-use-substitutes",
"#conf-build-users-group": "command-ref/conf-file.html#conf-build-users-group",
"#conf-builders": "command-ref/conf-file.html#conf-builders",
"#conf-builders-use-substitutes": "command-ref/conf-file.html#conf-builders-use-substitutes",
"#conf-compress-build-log": "command-ref/conf-file.html#conf-compress-build-log",
"#conf-connect-timeout": "command-ref/conf-file.html#conf-connect-timeout",
"#conf-cores": "command-ref/conf-file.html#conf-cores",
"#conf-diff-hook": "command-ref/conf-file.html#conf-diff-hook",
"#conf-enforce-determinism": "command-ref/conf-file.html#conf-enforce-determinism",
"#conf-env-keep-derivations": "command-ref/conf-file.html#conf-env-keep-derivations",
"#conf-extra-binary-caches": "command-ref/conf-file.html#conf-extra-binary-caches",
"#conf-extra-platforms": "command-ref/conf-file.html#conf-extra-platforms",
"#conf-extra-sandbox-paths": "command-ref/conf-file.html#conf-extra-sandbox-paths",
"#conf-extra-substituters": "command-ref/conf-file.html#conf-extra-substituters",
"#conf-fallback": "command-ref/conf-file.html#conf-fallback",
"#conf-fsync-metadata": "command-ref/conf-file.html#conf-fsync-metadata",
"#conf-gc-keep-derivations": "command-ref/conf-file.html#conf-gc-keep-derivations",
"#conf-gc-keep-outputs": "command-ref/conf-file.html#conf-gc-keep-outputs",
"#conf-hashed-mirrors": "command-ref/conf-file.html#conf-hashed-mirrors",
"#conf-http-connections": "command-ref/conf-file.html#conf-http-connections",
"#conf-keep-build-log": "command-ref/conf-file.html#conf-keep-build-log",
"#conf-keep-derivations": "command-ref/conf-file.html#conf-keep-derivations",
"#conf-keep-env-derivations": "command-ref/conf-file.html#conf-keep-env-derivations",
"#conf-keep-outputs": "command-ref/conf-file.html#conf-keep-outputs",
"#conf-max-build-log-size": "command-ref/conf-file.html#conf-max-build-log-size",
"#conf-max-free": "command-ref/conf-file.html#conf-max-free",
"#conf-max-jobs": "command-ref/conf-file.html#conf-max-jobs",
"#conf-max-silent-time": "command-ref/conf-file.html#conf-max-silent-time",
"#conf-min-free": "command-ref/conf-file.html#conf-min-free",
"#conf-narinfo-cache-negative-ttl": "command-ref/conf-file.html#conf-narinfo-cache-negative-ttl",
"#conf-narinfo-cache-positive-ttl": "command-ref/conf-file.html#conf-narinfo-cache-positive-ttl",
"#conf-netrc-file": "command-ref/conf-file.html#conf-netrc-file",
"#conf-plugin-files": "command-ref/conf-file.html#conf-plugin-files",
"#conf-post-build-hook": "command-ref/conf-file.html#conf-post-build-hook",
"#conf-pre-build-hook": "command-ref/conf-file.html#conf-pre-build-hook",
"#conf-repeat": "command-ref/conf-file.html#conf-repeat",
"#conf-require-sigs": "command-ref/conf-file.html#conf-require-sigs",
"#conf-restrict-eval": "command-ref/conf-file.html#conf-restrict-eval",
"#conf-run-diff-hook": "command-ref/conf-file.html#conf-run-diff-hook",
"#conf-sandbox": "command-ref/conf-file.html#conf-sandbox",
"#conf-sandbox-dev-shm-size": "command-ref/conf-file.html#conf-sandbox-dev-shm-size",
"#conf-sandbox-paths": "command-ref/conf-file.html#conf-sandbox-paths",
"#conf-secret-key-files": "command-ref/conf-file.html#conf-secret-key-files",
"#conf-show-trace": "command-ref/conf-file.html#conf-show-trace",
"#conf-stalled-download-timeout": "command-ref/conf-file.html#conf-stalled-download-timeout",
"#conf-substitute": "command-ref/conf-file.html#conf-substitute",
"#conf-substituters": "command-ref/conf-file.html#conf-substituters",
"#conf-system": "command-ref/conf-file.html#conf-system",
"#conf-system-features": "command-ref/conf-file.html#conf-system-features",
"#conf-tarball-ttl": "command-ref/conf-file.html#conf-tarball-ttl",
"#conf-timeout": "command-ref/conf-file.html#conf-timeout",
"#conf-trace-function-calls": "command-ref/conf-file.html#conf-trace-function-calls",
"#conf-trusted-binary-caches": "command-ref/conf-file.html#conf-trusted-binary-caches",
"#conf-trusted-public-keys": "command-ref/conf-file.html#conf-trusted-public-keys",
"#conf-trusted-substituters": "command-ref/conf-file.html#conf-trusted-substituters",
"#conf-trusted-users": "command-ref/conf-file.html#conf-trusted-users",
"#extra-sandbox-paths": "command-ref/conf-file.html#extra-sandbox-paths",
"#sec-conf-file": "command-ref/conf-file.html",
"#env-NIX_PATH": "command-ref/env-common.html#env-NIX_PATH",
"#env-common": "command-ref/env-common.html",
"#envar-remote": "command-ref/env-common.html#env-NIX_REMOTE",
"#sec-common-env": "command-ref/env-common.html",
"#ch-files": "command-ref/files.html",
"#ch-main-commands": "command-ref/main-commands.html",
"#opt-out-link": "command-ref/nix-build.html#opt-out-link",
"#sec-nix-build": "command-ref/nix-build.html",
"#sec-nix-channel": "command-ref/nix-channel.html",
"#sec-nix-collect-garbage": "command-ref/nix-collect-garbage.html",
"#sec-nix-copy-closure": "command-ref/nix-copy-closure.html",
"#sec-nix-daemon": "command-ref/nix-daemon.html",
"#refsec-nix-env-install-examples": "command-ref/nix-env.html#examples",
"#rsec-nix-env-install": "command-ref/nix-env.html#operation---install",
"#rsec-nix-env-set": "command-ref/nix-env.html#operation---set",
"#rsec-nix-env-set-flag": "command-ref/nix-env.html#operation---set-flag",
"#rsec-nix-env-upgrade": "command-ref/nix-env.html#operation---upgrade",
"#sec-nix-env": "command-ref/nix-env.html",
"#ssec-version-comparisons": "command-ref/nix-env.html#versions",
"#sec-nix-hash": "command-ref/nix-hash.html",
"#sec-nix-instantiate": "command-ref/nix-instantiate.html",
"#sec-nix-prefetch-url": "command-ref/nix-prefetch-url.html",
"#sec-nix-shell": "command-ref/nix-shell.html",
"#ssec-nix-shell-shebang": "command-ref/nix-shell.html#use-as-a--interpreter",
"#nixref-queries": "command-ref/nix-store.html#queries",
"#opt-add-root": "command-ref/nix-store.html#opt-add-root",
"#refsec-nix-store-dump": "command-ref/nix-store.html#operation---dump",
"#refsec-nix-store-export": "command-ref/nix-store.html#operation---export",
"#refsec-nix-store-import": "command-ref/nix-store.html#operation---import",
"#refsec-nix-store-query": "command-ref/nix-store.html#operation---query",
"#refsec-nix-store-verify": "command-ref/nix-store.html#operation---verify",
"#rsec-nix-store-gc": "command-ref/nix-store.html#operation---gc",
"#rsec-nix-store-generate-binary-cache-key": "command-ref/nix-store.html#operation---generate-binary-cache-key",
"#rsec-nix-store-realise": "command-ref/nix-store.html#operation---realise",
"#rsec-nix-store-serve": "command-ref/nix-store.html#operation---serve",
"#sec-nix-store": "command-ref/nix-store.html",
"#opt-I": "command-ref/opt-common.html#opt-I",
"#opt-attr": "command-ref/opt-common.html#opt-attr",
"#opt-common": "command-ref/opt-common.html",
"#opt-cores": "command-ref/opt-common.html#opt-cores",
"#opt-log-format": "command-ref/opt-common.html#opt-log-format",
"#opt-max-jobs": "command-ref/opt-common.html#opt-max-jobs",
"#opt-max-silent-time": "command-ref/opt-common.html#opt-max-silent-time",
"#opt-timeout": "command-ref/opt-common.html#opt-timeout",
"#sec-common-options": "command-ref/opt-common.html",
"#ch-utilities": "command-ref/utilities.html",
"#chap-hacking": "contributing/hacking.html",
"#adv-attr-allowSubstitutes": "language/advanced-attributes.html#adv-attr-allowSubstitutes",
"#adv-attr-allowedReferences": "language/advanced-attributes.html#adv-attr-allowedReferences",
"#adv-attr-allowedRequisites": "language/advanced-attributes.html#adv-attr-allowedRequisites",
"#adv-attr-disallowedReferences": "language/advanced-attributes.html#adv-attr-disallowedReferences",
"#adv-attr-disallowedRequisites": "language/advanced-attributes.html#adv-attr-disallowedRequisites",
"#adv-attr-exportReferencesGraph": "language/advanced-attributes.html#adv-attr-exportReferencesGraph",
"#adv-attr-impureEnvVars": "language/advanced-attributes.html#adv-attr-impureEnvVars",
"#adv-attr-outputHash": "language/advanced-attributes.html#adv-attr-outputHash",
"#adv-attr-outputHashAlgo": "language/advanced-attributes.html#adv-attr-outputHashAlgo",
"#adv-attr-outputHashMode": "language/advanced-attributes.html#adv-attr-outputHashMode",
"#adv-attr-passAsFile": "language/advanced-attributes.html#adv-attr-passAsFile",
"#adv-attr-preferLocalBuild": "language/advanced-attributes.html#adv-attr-preferLocalBuild",
"#fixed-output-drvs": "language/advanced-attributes.html#adv-attr-outputHash",
"#sec-advanced-attributes": "language/advanced-attributes.html",
"#builtin-abort": "language/builtins.html#builtins-abort",
"#builtin-add": "language/builtins.html#builtins-add",
"#builtin-all": "language/builtins.html#builtins-all",
"#builtin-any": "language/builtins.html#builtins-any",
"#builtin-attrNames": "language/builtins.html#builtins-attrNames",
"#builtin-attrValues": "language/builtins.html#builtins-attrValues",
"#builtin-baseNameOf": "language/builtins.html#builtins-baseNameOf",
"#builtin-bitAnd": "language/builtins.html#builtins-bitAnd",
"#builtin-bitOr": "language/builtins.html#builtins-bitOr",
"#builtin-bitXor": "language/builtins.html#builtins-bitXor",
"#builtin-builtins": "language/builtins.html#builtins-builtins",
"#builtin-compareVersions": "language/builtins.html#builtins-compareVersions",
"#builtin-concatLists": "language/builtins.html#builtins-concatLists",
"#builtin-concatStringsSep": "language/builtins.html#builtins-concatStringsSep",
"#builtin-currentSystem": "language/builtins.html#builtins-currentSystem",
"#builtin-deepSeq": "language/builtins.html#builtins-deepSeq",
"#builtin-derivation": "language/builtins.html#builtins-derivation",
"#builtin-dirOf": "language/builtins.html#builtins-dirOf",
"#builtin-div": "language/builtins.html#builtins-div",
"#builtin-elem": "language/builtins.html#builtins-elem",
"#builtin-elemAt": "language/builtins.html#builtins-elemAt",
"#builtin-fetchGit": "language/builtins.html#builtins-fetchGit",
"#builtin-fetchTarball": "language/builtins.html#builtins-fetchTarball",
"#builtin-fetchurl": "language/builtins.html#builtins-fetchurl",
"#builtin-filterSource": "language/builtins.html#builtins-filterSource",
"#builtin-foldl-prime": "language/builtins.html#builtins-foldl-prime",
"#builtin-fromJSON": "language/builtins.html#builtins-fromJSON",
"#builtin-functionArgs": "language/builtins.html#builtins-functionArgs",
"#builtin-genList": "language/builtins.html#builtins-genList",
"#builtin-getAttr": "language/builtins.html#builtins-getAttr",
"#builtin-getEnv": "language/builtins.html#builtins-getEnv",
"#builtin-hasAttr": "language/builtins.html#builtins-hasAttr",
"#builtin-hashFile": "language/builtins.html#builtins-hashFile",
"#builtin-hashString": "language/builtins.html#builtins-hashString",
"#builtin-head": "language/builtins.html#builtins-head",
"#builtin-import": "language/builtins.html#builtins-import",
"#builtin-intersectAttrs": "language/builtins.html#builtins-intersectAttrs",
"#builtin-isAttrs": "language/builtins.html#builtins-isAttrs",
"#builtin-isBool": "language/builtins.html#builtins-isBool",
"#builtin-isFloat": "language/builtins.html#builtins-isFloat",
"#builtin-isFunction": "language/builtins.html#builtins-isFunction",
"#builtin-isInt": "language/builtins.html#builtins-isInt",
"#builtin-isList": "language/builtins.html#builtins-isList",
"#builtin-isNull": "language/builtins.html#builtins-isNull",
"#builtin-isString": "language/builtins.html#builtins-isString",
"#builtin-length": "language/builtins.html#builtins-length",
"#builtin-lessThan": "language/builtins.html#builtins-lessThan",
"#builtin-listToAttrs": "language/builtins.html#builtins-listToAttrs",
"#builtin-map": "language/builtins.html#builtins-map",
"#builtin-match": "language/builtins.html#builtins-match",
"#builtin-mul": "language/builtins.html#builtins-mul",
"#builtin-parseDrvName": "language/builtins.html#builtins-parseDrvName",
"#builtin-path": "language/builtins.html#builtins-path",
"#builtin-pathExists": "language/builtins.html#builtins-pathExists",
"#builtin-placeholder": "language/builtins.html#builtins-placeholder",
"#builtin-readDir": "language/builtins.html#builtins-readDir",
"#builtin-readFile": "language/builtins.html#builtins-readFile",
"#builtin-removeAttrs": "language/builtins.html#builtins-removeAttrs",
"#builtin-replaceStrings": "language/builtins.html#builtins-replaceStrings",
"#builtin-seq": "language/builtins.html#builtins-seq",
"#builtin-sort": "language/builtins.html#builtins-sort",
"#builtin-split": "language/builtins.html#builtins-split",
"#builtin-splitVersion": "language/builtins.html#builtins-splitVersion",
"#builtin-stringLength": "language/builtins.html#builtins-stringLength",
"#builtin-sub": "language/builtins.html#builtins-sub",
"#builtin-substring": "language/builtins.html#builtins-substring",
"#builtin-tail": "language/builtins.html#builtins-tail",
"#builtin-throw": "language/builtins.html#builtins-throw",
"#builtin-toFile": "language/builtins.html#builtins-toFile",
"#builtin-toJSON": "language/builtins.html#builtins-toJSON",
"#builtin-toPath": "language/builtins.html#builtins-toPath",
"#builtin-toString": "language/builtins.html#builtins-toString",
"#builtin-toXML": "language/builtins.html#builtins-toXML",
"#builtin-trace": "language/builtins.html#builtins-trace",
"#builtin-tryEval": "language/builtins.html#builtins-tryEval",
"#builtin-typeOf": "language/builtins.html#builtins-typeOf",
"#ssec-builtins": "language/builtins.html",
"#attr-system": "language/derivations.html#attr-system",
"#ssec-derivation": "language/derivations.html",
"#ch-expression-language": "language/index.html",
"#sec-constructs": "language/constructs.html",
"#sect-let-language": "language/constructs.html#let-language",
"#ss-functions": "language/constructs.html#functions",
"#sec-language-operators": "language/operators.html",
"#table-operators": "language/operators.html",
"#ssec-values": "language/values.html",
"#gloss-closure": "glossary.html#gloss-closure",
"#gloss-derivation": "glossary.html#gloss-derivation",
"#gloss-deriver": "glossary.html#gloss-deriver",
"#gloss-nar": "glossary.html#gloss-nar",
"#gloss-output-path": "glossary.html#gloss-output-path",
"#gloss-profile": "glossary.html#gloss-profile",
"#gloss-reachable": "glossary.html#gloss-reachable",
"#gloss-reference": "glossary.html#gloss-reference",
"#gloss-substitute": "glossary.html#gloss-substitute",
"#gloss-user-env": "glossary.html#gloss-user-env",
"#gloss-validity": "glossary.html#gloss-validity",
"#part-glossary": "glossary.html",
"#sec-building-source": "installation/building-source.html",
"#ch-env-variables": "installation/env-variables.html",
"#sec-installer-proxy-settings": "installation/env-variables.html#proxy-environment-variables",
"#sec-nix-ssl-cert-file": "installation/env-variables.html#nix_ssl_cert_file",
"#sec-nix-ssl-cert-file-with-nix-daemon-and-macos": "installation/env-variables.html#nix_ssl_cert_file-with-macos-and-the-nix-daemon",
"#chap-installation": "installation/installation.html",
"#ch-installing-binary": "installation/installing-binary.html",
"#sect-macos-installation": "installation/installing-binary.html#macos-installation",
"#sect-macos-installation-change-store-prefix": "installation/installing-binary.html#macos-installation",
"#sect-macos-installation-encrypted-volume": "installation/installing-binary.html#macos-installation",
"#sect-macos-installation-recommended-notes": "installation/installing-binary.html#macos-installation",
"#sect-macos-installation-symlink": "installation/installing-binary.html#macos-installation",
"#sect-multi-user-installation": "installation/installing-binary.html#multi-user-installation",
"#sect-nix-install-binary-tarball": "installation/installing-binary.html#installing-from-a-binary-tarball",
"#sect-nix-install-pinned-version-url": "installation/installing-binary.html#installing-a-pinned-nix-version-from-a-url",
"#sect-single-user-installation": "installation/installing-binary.html#single-user-installation",
"#ch-installing-source": "installation/installing-source.html",
"#ssec-multi-user": "installation/multi-user.html",
"#ch-nix-security": "installation/nix-security.html",
"#sec-obtaining-source": "installation/obtaining-source.html",
"#sec-prerequisites-source": "installation/prerequisites-source.html",
"#sec-single-user": "installation/single-user.html",
"#ch-supported-platforms": "installation/supported-platforms.html",
"#ch-upgrading-nix": "installation/upgrading.html",
"#ch-about-nix": "introduction.html",
"#chap-introduction": "introduction.html",
"#ch-basic-package-mgmt": "package-management/basic-package-mgmt.html",
"#ssec-binary-cache-substituter": "package-management/binary-cache-substituter.html",
"#sec-channels": "package-management/channels.html",
"#ssec-copy-closure": "package-management/copy-closure.html",
"#sec-garbage-collection": "package-management/garbage-collection.html",
"#ssec-gc-roots": "package-management/garbage-collector-roots.html",
"#chap-package-management": "package-management/package-management.html",
"#sec-profiles": "package-management/profiles.html",
"#ssec-s3-substituter": "package-management/s3-substituter.html",
"#ssec-s3-substituter-anonymous-reads": "package-management/s3-substituter.html#anonymous-reads-to-your-s3-compatible-binary-cache",
"#ssec-s3-substituter-authenticated-reads": "package-management/s3-substituter.html#authenticated-reads-to-your-s3-binary-cache",
"#ssec-s3-substituter-authenticated-writes": "package-management/s3-substituter.html#authenticated-writes-to-your-s3-compatible-binary-cache",
"#sec-sharing-packages": "package-management/sharing-packages.html",
"#ssec-ssh-substituter": "package-management/ssh-substituter.html",
"#chap-quick-start": "quick-start.html",
"#sec-relnotes": "release-notes/release-notes.html",
"#ch-relnotes-0.10.1": "release-notes/rl-0.10.1.html",
"#ch-relnotes-0.10": "release-notes/rl-0.10.html",
"#ssec-relnotes-0.11": "release-notes/rl-0.11.html",
"#ssec-relnotes-0.12": "release-notes/rl-0.12.html",
"#ssec-relnotes-0.13": "release-notes/rl-0.13.html",
"#ssec-relnotes-0.14": "release-notes/rl-0.14.html",
"#ssec-relnotes-0.15": "release-notes/rl-0.15.html",
"#ssec-relnotes-0.16": "release-notes/rl-0.16.html",
"#ch-relnotes-0.5": "release-notes/rl-0.5.html",
"#ch-relnotes-0.6": "release-notes/rl-0.6.html",
"#ch-relnotes-0.7": "release-notes/rl-0.7.html",
"#ch-relnotes-0.8.1": "release-notes/rl-0.8.1.html",
"#ch-relnotes-0.8": "release-notes/rl-0.8.html",
"#ch-relnotes-0.9.1": "release-notes/rl-0.9.1.html",
"#ch-relnotes-0.9.2": "release-notes/rl-0.9.2.html",
"#ch-relnotes-0.9": "release-notes/rl-0.9.html",
"#ssec-relnotes-1.0": "release-notes/rl-1.0.html",
"#ssec-relnotes-1.1": "release-notes/rl-1.1.html",
"#ssec-relnotes-1.10": "release-notes/rl-1.10.html",
"#ssec-relnotes-1.11.10": "release-notes/rl-1.11.10.html",
"#ssec-relnotes-1.11": "release-notes/rl-1.11.html",
"#ssec-relnotes-1.2": "release-notes/rl-1.2.html",
"#ssec-relnotes-1.3": "release-notes/rl-1.3.html",
"#ssec-relnotes-1.4": "release-notes/rl-1.4.html",
"#ssec-relnotes-1.5.1": "release-notes/rl-1.5.1.html",
"#ssec-relnotes-1.5.2": "release-notes/rl-1.5.2.html",
"#ssec-relnotes-1.5": "release-notes/rl-1.5.html",
"#ssec-relnotes-1.6.1": "release-notes/rl-1.6.1.html",
"#ssec-relnotes-1.6.0": "release-notes/rl-1.6.html",
"#ssec-relnotes-1.7": "release-notes/rl-1.7.html",
"#ssec-relnotes-1.8": "release-notes/rl-1.8.html",
"#ssec-relnotes-1.9": "release-notes/rl-1.9.html",
"#ssec-relnotes-2.0": "release-notes/rl-2.0.html",
"#ssec-relnotes-2.1": "release-notes/rl-2.1.html",
"#ssec-relnotes-2.2": "release-notes/rl-2.2.html",
"#ssec-relnotes-2.3": "release-notes/rl-2.3.html"
// redirect rules for anchors ensure backwards compatibility of URLs.
// this must be done on the client side, as web servers do not see the anchor part of the URL.
// redirections are declared as follows:
// each entry has as its key a path matching the requested URL path, relative to the mdBook document root.
//
// IMPORTANT: it must specify the full path with file name and suffix
//
// each entry is itself a set of key-value pairs, where
// - keys are anchors on the matched path.
// - values are redirection targets relative to the current path.
const redirects = {
"index.html": {
"part-advanced-topics": "advanced-topics/advanced-topics.html",
"chap-tuning-cores-and-jobs": "advanced-topics/cores-vs-jobs.html",
"chap-diff-hook": "advanced-topics/diff-hook.html",
"check-dirs-are-unregistered": "advanced-topics/diff-hook.html#check-dirs-are-unregistered",
"chap-distributed-builds": "advanced-topics/distributed-builds.html",
"chap-post-build-hook": "advanced-topics/post-build-hook.html",
"chap-post-build-hook-caveats": "advanced-topics/post-build-hook.html#implementation-caveats",
"part-command-ref": "command-ref/command-ref.html",
"conf-allow-import-from-derivation": "command-ref/conf-file.html#conf-allow-import-from-derivation",
"conf-allow-new-privileges": "command-ref/conf-file.html#conf-allow-new-privileges",
"conf-allowed-uris": "command-ref/conf-file.html#conf-allowed-uris",
"conf-allowed-users": "command-ref/conf-file.html#conf-allowed-users",
"conf-auto-optimise-store": "command-ref/conf-file.html#conf-auto-optimise-store",
"conf-binary-cache-public-keys": "command-ref/conf-file.html#conf-binary-cache-public-keys",
"conf-binary-caches": "command-ref/conf-file.html#conf-binary-caches",
"conf-build-compress-log": "command-ref/conf-file.html#conf-build-compress-log",
"conf-build-cores": "command-ref/conf-file.html#conf-build-cores",
"conf-build-extra-chroot-dirs": "command-ref/conf-file.html#conf-build-extra-chroot-dirs",
"conf-build-extra-sandbox-paths": "command-ref/conf-file.html#conf-build-extra-sandbox-paths",
"conf-build-fallback": "command-ref/conf-file.html#conf-build-fallback",
"conf-build-max-jobs": "command-ref/conf-file.html#conf-build-max-jobs",
"conf-build-max-log-size": "command-ref/conf-file.html#conf-build-max-log-size",
"conf-build-max-silent-time": "command-ref/conf-file.html#conf-build-max-silent-time",
"conf-build-repeat": "command-ref/conf-file.html#conf-build-repeat",
"conf-build-timeout": "command-ref/conf-file.html#conf-build-timeout",
"conf-build-use-chroot": "command-ref/conf-file.html#conf-build-use-chroot",
"conf-build-use-sandbox": "command-ref/conf-file.html#conf-build-use-sandbox",
"conf-build-use-substitutes": "command-ref/conf-file.html#conf-build-use-substitutes",
"conf-build-users-group": "command-ref/conf-file.html#conf-build-users-group",
"conf-builders": "command-ref/conf-file.html#conf-builders",
"conf-builders-use-substitutes": "command-ref/conf-file.html#conf-builders-use-substitutes",
"conf-compress-build-log": "command-ref/conf-file.html#conf-compress-build-log",
"conf-connect-timeout": "command-ref/conf-file.html#conf-connect-timeout",
"conf-cores": "command-ref/conf-file.html#conf-cores",
"conf-diff-hook": "command-ref/conf-file.html#conf-diff-hook",
"conf-enforce-determinism": "command-ref/conf-file.html#conf-enforce-determinism",
"conf-env-keep-derivations": "command-ref/conf-file.html#conf-env-keep-derivations",
"conf-extra-binary-caches": "command-ref/conf-file.html#conf-extra-binary-caches",
"conf-extra-platforms": "command-ref/conf-file.html#conf-extra-platforms",
"conf-extra-sandbox-paths": "command-ref/conf-file.html#conf-extra-sandbox-paths",
"conf-extra-substituters": "command-ref/conf-file.html#conf-extra-substituters",
"conf-fallback": "command-ref/conf-file.html#conf-fallback",
"conf-fsync-metadata": "command-ref/conf-file.html#conf-fsync-metadata",
"conf-gc-keep-derivations": "command-ref/conf-file.html#conf-gc-keep-derivations",
"conf-gc-keep-outputs": "command-ref/conf-file.html#conf-gc-keep-outputs",
"conf-hashed-mirrors": "command-ref/conf-file.html#conf-hashed-mirrors",
"conf-http-connections": "command-ref/conf-file.html#conf-http-connections",
"conf-keep-build-log": "command-ref/conf-file.html#conf-keep-build-log",
"conf-keep-derivations": "command-ref/conf-file.html#conf-keep-derivations",
"conf-keep-env-derivations": "command-ref/conf-file.html#conf-keep-env-derivations",
"conf-keep-outputs": "command-ref/conf-file.html#conf-keep-outputs",
"conf-max-build-log-size": "command-ref/conf-file.html#conf-max-build-log-size",
"conf-max-free": "command-ref/conf-file.html#conf-max-free",
"conf-max-jobs": "command-ref/conf-file.html#conf-max-jobs",
"conf-max-silent-time": "command-ref/conf-file.html#conf-max-silent-time",
"conf-min-free": "command-ref/conf-file.html#conf-min-free",
"conf-narinfo-cache-negative-ttl": "command-ref/conf-file.html#conf-narinfo-cache-negative-ttl",
"conf-narinfo-cache-positive-ttl": "command-ref/conf-file.html#conf-narinfo-cache-positive-ttl",
"conf-netrc-file": "command-ref/conf-file.html#conf-netrc-file",
"conf-plugin-files": "command-ref/conf-file.html#conf-plugin-files",
"conf-post-build-hook": "command-ref/conf-file.html#conf-post-build-hook",
"conf-pre-build-hook": "command-ref/conf-file.html#conf-pre-build-hook",
"conf-repeat": "command-ref/conf-file.html#conf-repeat",
"conf-require-sigs": "command-ref/conf-file.html#conf-require-sigs",
"conf-restrict-eval": "command-ref/conf-file.html#conf-restrict-eval",
"conf-run-diff-hook": "command-ref/conf-file.html#conf-run-diff-hook",
"conf-sandbox": "command-ref/conf-file.html#conf-sandbox",
"conf-sandbox-dev-shm-size": "command-ref/conf-file.html#conf-sandbox-dev-shm-size",
"conf-sandbox-paths": "command-ref/conf-file.html#conf-sandbox-paths",
"conf-secret-key-files": "command-ref/conf-file.html#conf-secret-key-files",
"conf-show-trace": "command-ref/conf-file.html#conf-show-trace",
"conf-stalled-download-timeout": "command-ref/conf-file.html#conf-stalled-download-timeout",
"conf-substitute": "command-ref/conf-file.html#conf-substitute",
"conf-substituters": "command-ref/conf-file.html#conf-substituters",
"conf-system": "command-ref/conf-file.html#conf-system",
"conf-system-features": "command-ref/conf-file.html#conf-system-features",
"conf-tarball-ttl": "command-ref/conf-file.html#conf-tarball-ttl",
"conf-timeout": "command-ref/conf-file.html#conf-timeout",
"conf-trace-function-calls": "command-ref/conf-file.html#conf-trace-function-calls",
"conf-trusted-binary-caches": "command-ref/conf-file.html#conf-trusted-binary-caches",
"conf-trusted-public-keys": "command-ref/conf-file.html#conf-trusted-public-keys",
"conf-trusted-substituters": "command-ref/conf-file.html#conf-trusted-substituters",
"conf-trusted-users": "command-ref/conf-file.html#conf-trusted-users",
"extra-sandbox-paths": "command-ref/conf-file.html#extra-sandbox-paths",
"sec-conf-file": "command-ref/conf-file.html",
"env-NIX_PATH": "command-ref/env-common.html#env-NIX_PATH",
"env-common": "command-ref/env-common.html",
"envar-remote": "command-ref/env-common.html#env-NIX_REMOTE",
"sec-common-env": "command-ref/env-common.html",
"ch-files": "command-ref/files.html",
"ch-main-commands": "command-ref/main-commands.html",
"opt-out-link": "command-ref/nix-build.html#opt-out-link",
"sec-nix-build": "command-ref/nix-build.html",
"sec-nix-channel": "command-ref/nix-channel.html",
"sec-nix-collect-garbage": "command-ref/nix-collect-garbage.html",
"sec-nix-copy-closure": "command-ref/nix-copy-closure.html",
"sec-nix-daemon": "command-ref/nix-daemon.html",
"refsec-nix-env-install-examples": "command-ref/nix-env.html#examples",
"rsec-nix-env-install": "command-ref/nix-env.html#operation---install",
"rsec-nix-env-set": "command-ref/nix-env.html#operation---set",
"rsec-nix-env-set-flag": "command-ref/nix-env.html#operation---set-flag",
"rsec-nix-env-upgrade": "command-ref/nix-env.html#operation---upgrade",
"sec-nix-env": "command-ref/nix-env.html",
"ssec-version-comparisons": "command-ref/nix-env.html#versions",
"sec-nix-hash": "command-ref/nix-hash.html",
"sec-nix-instantiate": "command-ref/nix-instantiate.html",
"sec-nix-prefetch-url": "command-ref/nix-prefetch-url.html",
"sec-nix-shell": "command-ref/nix-shell.html",
"ssec-nix-shell-shebang": "command-ref/nix-shell.html#use-as-a--interpreter",
"nixref-queries": "command-ref/nix-store.html#queries",
"opt-add-root": "command-ref/nix-store.html#opt-add-root",
"refsec-nix-store-dump": "command-ref/nix-store.html#operation---dump",
"refsec-nix-store-export": "command-ref/nix-store.html#operation---export",
"refsec-nix-store-import": "command-ref/nix-store.html#operation---import",
"refsec-nix-store-query": "command-ref/nix-store.html#operation---query",
"refsec-nix-store-verify": "command-ref/nix-store.html#operation---verify",
"rsec-nix-store-gc": "command-ref/nix-store.html#operation---gc",
"rsec-nix-store-generate-binary-cache-key": "command-ref/nix-store.html#operation---generate-binary-cache-key",
"rsec-nix-store-realise": "command-ref/nix-store.html#operation---realise",
"rsec-nix-store-serve": "command-ref/nix-store.html#operation---serve",
"sec-nix-store": "command-ref/nix-store.html",
"opt-I": "command-ref/opt-common.html#opt-I",
"opt-attr": "command-ref/opt-common.html#opt-attr",
"opt-common": "command-ref/opt-common.html",
"opt-cores": "command-ref/opt-common.html#opt-cores",
"opt-log-format": "command-ref/opt-common.html#opt-log-format",
"opt-max-jobs": "command-ref/opt-common.html#opt-max-jobs",
"opt-max-silent-time": "command-ref/opt-common.html#opt-max-silent-time",
"opt-timeout": "command-ref/opt-common.html#opt-timeout",
"sec-common-options": "command-ref/opt-common.html",
"ch-utilities": "command-ref/utilities.html",
"chap-hacking": "contributing/hacking.html",
"adv-attr-allowSubstitutes": "language/advanced-attributes.html#adv-attr-allowSubstitutes",
"adv-attr-allowedReferences": "language/advanced-attributes.html#adv-attr-allowedReferences",
"adv-attr-allowedRequisites": "language/advanced-attributes.html#adv-attr-allowedRequisites",
"adv-attr-disallowedReferences": "language/advanced-attributes.html#adv-attr-disallowedReferences",
"adv-attr-disallowedRequisites": "language/advanced-attributes.html#adv-attr-disallowedRequisites",
"adv-attr-exportReferencesGraph": "language/advanced-attributes.html#adv-attr-exportReferencesGraph",
"adv-attr-impureEnvVars": "language/advanced-attributes.html#adv-attr-impureEnvVars",
"adv-attr-outputHash": "language/advanced-attributes.html#adv-attr-outputHash",
"adv-attr-outputHashAlgo": "language/advanced-attributes.html#adv-attr-outputHashAlgo",
"adv-attr-outputHashMode": "language/advanced-attributes.html#adv-attr-outputHashMode",
"adv-attr-passAsFile": "language/advanced-attributes.html#adv-attr-passAsFile",
"adv-attr-preferLocalBuild": "language/advanced-attributes.html#adv-attr-preferLocalBuild",
"fixed-output-drvs": "language/advanced-attributes.html#adv-attr-outputHash",
"sec-advanced-attributes": "language/advanced-attributes.html",
"builtin-abort": "language/builtins.html#builtins-abort",
"builtin-add": "language/builtins.html#builtins-add",
"builtin-all": "language/builtins.html#builtins-all",
"builtin-any": "language/builtins.html#builtins-any",
"builtin-attrNames": "language/builtins.html#builtins-attrNames",
"builtin-attrValues": "language/builtins.html#builtins-attrValues",
"builtin-baseNameOf": "language/builtins.html#builtins-baseNameOf",
"builtin-bitAnd": "language/builtins.html#builtins-bitAnd",
"builtin-bitOr": "language/builtins.html#builtins-bitOr",
"builtin-bitXor": "language/builtins.html#builtins-bitXor",
"builtin-builtins": "language/builtins.html#builtins-builtins",
"builtin-compareVersions": "language/builtins.html#builtins-compareVersions",
"builtin-concatLists": "language/builtins.html#builtins-concatLists",
"builtin-concatStringsSep": "language/builtins.html#builtins-concatStringsSep",
"builtin-currentSystem": "language/builtins.html#builtins-currentSystem",
"builtin-deepSeq": "language/builtins.html#builtins-deepSeq",
"builtin-derivation": "language/builtins.html#builtins-derivation",
"builtin-dirOf": "language/builtins.html#builtins-dirOf",
"builtin-div": "language/builtins.html#builtins-div",
"builtin-elem": "language/builtins.html#builtins-elem",
"builtin-elemAt": "language/builtins.html#builtins-elemAt",
"builtin-fetchGit": "language/builtins.html#builtins-fetchGit",
"builtin-fetchTarball": "language/builtins.html#builtins-fetchTarball",
"builtin-fetchurl": "language/builtins.html#builtins-fetchurl",
"builtin-filterSource": "language/builtins.html#builtins-filterSource",
"builtin-foldl-prime": "language/builtins.html#builtins-foldl-prime",
"builtin-fromJSON": "language/builtins.html#builtins-fromJSON",
"builtin-functionArgs": "language/builtins.html#builtins-functionArgs",
"builtin-genList": "language/builtins.html#builtins-genList",
"builtin-getAttr": "language/builtins.html#builtins-getAttr",
"builtin-getEnv": "language/builtins.html#builtins-getEnv",
"builtin-hasAttr": "language/builtins.html#builtins-hasAttr",
"builtin-hashFile": "language/builtins.html#builtins-hashFile",
"builtin-hashString": "language/builtins.html#builtins-hashString",
"builtin-head": "language/builtins.html#builtins-head",
"builtin-import": "language/builtins.html#builtins-import",
"builtin-intersectAttrs": "language/builtins.html#builtins-intersectAttrs",
"builtin-isAttrs": "language/builtins.html#builtins-isAttrs",
"builtin-isBool": "language/builtins.html#builtins-isBool",
"builtin-isFloat": "language/builtins.html#builtins-isFloat",
"builtin-isFunction": "language/builtins.html#builtins-isFunction",
"builtin-isInt": "language/builtins.html#builtins-isInt",
"builtin-isList": "language/builtins.html#builtins-isList",
"builtin-isNull": "language/builtins.html#builtins-isNull",
"builtin-isString": "language/builtins.html#builtins-isString",
"builtin-length": "language/builtins.html#builtins-length",
"builtin-lessThan": "language/builtins.html#builtins-lessThan",
"builtin-listToAttrs": "language/builtins.html#builtins-listToAttrs",
"builtin-map": "language/builtins.html#builtins-map",
"builtin-match": "language/builtins.html#builtins-match",
"builtin-mul": "language/builtins.html#builtins-mul",
"builtin-parseDrvName": "language/builtins.html#builtins-parseDrvName",
"builtin-path": "language/builtins.html#builtins-path",
"builtin-pathExists": "language/builtins.html#builtins-pathExists",
"builtin-placeholder": "language/builtins.html#builtins-placeholder",
"builtin-readDir": "language/builtins.html#builtins-readDir",
"builtin-readFile": "language/builtins.html#builtins-readFile",
"builtin-removeAttrs": "language/builtins.html#builtins-removeAttrs",
"builtin-replaceStrings": "language/builtins.html#builtins-replaceStrings",
"builtin-seq": "language/builtins.html#builtins-seq",
"builtin-sort": "language/builtins.html#builtins-sort",
"builtin-split": "language/builtins.html#builtins-split",
"builtin-splitVersion": "language/builtins.html#builtins-splitVersion",
"builtin-stringLength": "language/builtins.html#builtins-stringLength",
"builtin-sub": "language/builtins.html#builtins-sub",
"builtin-substring": "language/builtins.html#builtins-substring",
"builtin-tail": "language/builtins.html#builtins-tail",
"builtin-throw": "language/builtins.html#builtins-throw",
"builtin-toFile": "language/builtins.html#builtins-toFile",
"builtin-toJSON": "language/builtins.html#builtins-toJSON",
"builtin-toPath": "language/builtins.html#builtins-toPath",
"builtin-toString": "language/builtins.html#builtins-toString",
"builtin-toXML": "language/builtins.html#builtins-toXML",
"builtin-trace": "language/builtins.html#builtins-trace",
"builtin-tryEval": "language/builtins.html#builtins-tryEval",
"builtin-typeOf": "language/builtins.html#builtins-typeOf",
"ssec-builtins": "language/builtins.html",
"attr-system": "language/derivations.html#attr-system",
"ssec-derivation": "language/derivations.html",
"ch-expression-language": "language/index.html",
"sec-constructs": "language/constructs.html",
"sect-let-language": "language/constructs.html#let-language",
"ss-functions": "language/constructs.html#functions",
"sec-language-operators": "language/operators.html",
"table-operators": "language/operators.html",
"ssec-values": "language/values.html",
"gloss-closure": "glossary.html#gloss-closure",
"gloss-derivation": "glossary.html#gloss-derivation",
"gloss-deriver": "glossary.html#gloss-deriver",
"gloss-nar": "glossary.html#gloss-nar",
"gloss-output-path": "glossary.html#gloss-output-path",
"gloss-profile": "glossary.html#gloss-profile",
"gloss-reachable": "glossary.html#gloss-reachable",
"gloss-reference": "glossary.html#gloss-reference",
"gloss-substitute": "glossary.html#gloss-substitute",
"gloss-user-env": "glossary.html#gloss-user-env",
"gloss-validity": "glossary.html#gloss-validity",
"part-glossary": "glossary.html",
"sec-building-source": "installation/building-source.html",
"ch-env-variables": "installation/env-variables.html",
"sec-installer-proxy-settings": "installation/env-variables.html#proxy-environment-variables",
"sec-nix-ssl-cert-file": "installation/env-variables.html#nix_ssl_cert_file",
"sec-nix-ssl-cert-file-with-nix-daemon-and-macos": "installation/env-variables.html#nix_ssl_cert_file-with-macos-and-the-nix-daemon",
"chap-installation": "installation/installation.html",
"ch-installing-binary": "installation/installing-binary.html",
"sect-macos-installation": "installation/installing-binary.html#macos-installation",
"sect-macos-installation-change-store-prefix": "installation/installing-binary.html#macos-installation",
"sect-macos-installation-encrypted-volume": "installation/installing-binary.html#macos-installation",
"sect-macos-installation-recommended-notes": "installation/installing-binary.html#macos-installation",
"sect-macos-installation-symlink": "installation/installing-binary.html#macos-installation",
"sect-multi-user-installation": "installation/installing-binary.html#multi-user-installation",
"sect-nix-install-binary-tarball": "installation/installing-binary.html#installing-from-a-binary-tarball",
"sect-nix-install-pinned-version-url": "installation/installing-binary.html#installing-a-pinned-nix-version-from-a-url",
"sect-single-user-installation": "installation/installing-binary.html#single-user-installation",
"ch-installing-source": "installation/installing-source.html",
"ssec-multi-user": "installation/multi-user.html",
"ch-nix-security": "installation/nix-security.html",
"sec-obtaining-source": "installation/obtaining-source.html",
"sec-prerequisites-source": "installation/prerequisites-source.html",
"sec-single-user": "installation/single-user.html",
"ch-supported-platforms": "installation/supported-platforms.html",
"ch-upgrading-nix": "installation/upgrading.html",
"ch-about-nix": "introduction.html",
"chap-introduction": "introduction.html",
"ch-basic-package-mgmt": "package-management/basic-package-mgmt.html",
"ssec-binary-cache-substituter": "package-management/binary-cache-substituter.html",
"sec-channels": "package-management/channels.html",
"ssec-copy-closure": "package-management/copy-closure.html",
"sec-garbage-collection": "package-management/garbage-collection.html",
"ssec-gc-roots": "package-management/garbage-collector-roots.html",
"chap-package-management": "package-management/package-management.html",
"sec-profiles": "package-management/profiles.html",
"ssec-s3-substituter": "package-management/s3-substituter.html",
"ssec-s3-substituter-anonymous-reads": "package-management/s3-substituter.html#anonymous-reads-to-your-s3-compatible-binary-cache",
"ssec-s3-substituter-authenticated-reads": "package-management/s3-substituter.html#authenticated-reads-to-your-s3-binary-cache",
"ssec-s3-substituter-authenticated-writes": "package-management/s3-substituter.html#authenticated-writes-to-your-s3-compatible-binary-cache",
"sec-sharing-packages": "package-management/sharing-packages.html",
"ssec-ssh-substituter": "package-management/ssh-substituter.html",
"chap-quick-start": "quick-start.html",
"sec-relnotes": "release-notes/release-notes.html",
"ch-relnotes-0.10.1": "release-notes/rl-0.10.1.html",
"ch-relnotes-0.10": "release-notes/rl-0.10.html",
"ssec-relnotes-0.11": "release-notes/rl-0.11.html",
"ssec-relnotes-0.12": "release-notes/rl-0.12.html",
"ssec-relnotes-0.13": "release-notes/rl-0.13.html",
"ssec-relnotes-0.14": "release-notes/rl-0.14.html",
"ssec-relnotes-0.15": "release-notes/rl-0.15.html",
"ssec-relnotes-0.16": "release-notes/rl-0.16.html",
"ch-relnotes-0.5": "release-notes/rl-0.5.html",
"ch-relnotes-0.6": "release-notes/rl-0.6.html",
"ch-relnotes-0.7": "release-notes/rl-0.7.html",
"ch-relnotes-0.8.1": "release-notes/rl-0.8.1.html",
"ch-relnotes-0.8": "release-notes/rl-0.8.html",
"ch-relnotes-0.9.1": "release-notes/rl-0.9.1.html",
"ch-relnotes-0.9.2": "release-notes/rl-0.9.2.html",
"ch-relnotes-0.9": "release-notes/rl-0.9.html",
"ssec-relnotes-1.0": "release-notes/rl-1.0.html",
"ssec-relnotes-1.1": "release-notes/rl-1.1.html",
"ssec-relnotes-1.10": "release-notes/rl-1.10.html",
"ssec-relnotes-1.11.10": "release-notes/rl-1.11.10.html",
"ssec-relnotes-1.11": "release-notes/rl-1.11.html",
"ssec-relnotes-1.2": "release-notes/rl-1.2.html",
"ssec-relnotes-1.3": "release-notes/rl-1.3.html",
"ssec-relnotes-1.4": "release-notes/rl-1.4.html",
"ssec-relnotes-1.5.1": "release-notes/rl-1.5.1.html",
"ssec-relnotes-1.5.2": "release-notes/rl-1.5.2.html",
"ssec-relnotes-1.5": "release-notes/rl-1.5.html",
"ssec-relnotes-1.6.1": "release-notes/rl-1.6.1.html",
"ssec-relnotes-1.6.0": "release-notes/rl-1.6.html",
"ssec-relnotes-1.7": "release-notes/rl-1.7.html",
"ssec-relnotes-1.8": "release-notes/rl-1.8.html",
"ssec-relnotes-1.9": "release-notes/rl-1.9.html",
"ssec-relnotes-2.0": "release-notes/rl-2.0.html",
"ssec-relnotes-2.1": "release-notes/rl-2.1.html",
"ssec-relnotes-2.2": "release-notes/rl-2.2.html",
"ssec-relnotes-2.3": "release-notes/rl-2.3.html"
},
"language/values.html": {
"simple-values": "#primitives",
"lists": "#list",
"strings": "#string",
"lists": "#list",
"attribute-sets": "#attribute-set"
}
};
var isRoot = (document.location.pathname.endsWith('/') || document.location.pathname.endsWith('/index.html')) && path_to_root === '';
if (isRoot && redirects[document.location.hash]) {
document.location.href = path_to_root + redirects[document.location.hash];
// the following code matches the current page's URL against the set of redirects.
//
// it is written to minimize the latency between page load and redirect.
// therefore we avoid function calls, copying data, and unnecessary loops.
// IMPORTANT: we use stateful array operations and their order matters!
//
// matching URLs is more involved than it should be:
//
// 1. `document.location.pathname` can have an arbitrary prefix.
//
// 2. `path_to_root` is set by mdBook. it consists only of `../`s and
// determines the depth of `<path>` relative to the prefix:
//
// `document.location.pathname`
// |------------------------------|
// /<prefix>/<path>/[<file>[.html]][#<anchor>]
// |----|
// `path_to_root` has same number of path segments
//
// source: https://phaiax.github.io/mdBook/format/theme/index-hbs.html#data
//
// 3. the following paths are equivalent:
//
// /foo/bar/
// /foo/bar/index.html
// /foo/bar/index
//
// 4. the following paths are also equivalent:
//
// /foo/bar/baz
// /foo/bar/baz.html
//
let segments = document.location.pathname.split('/');
let file = segments.pop();
// normalize file name
if (file === '') { file = "index.html"; }
else if (!file.endsWith('.html')) { file = file + '.html'; }
segments.push(file);
// use `path_to_root` to discern prefix from path.
const depth = path_to_root.split('/').length;
// remove segments containing prefix. the following works because
// 1. the original `document.location.pathname` is absolute,
// hence first element of `segments` is always empty.
// 2. last element of splitting `path_to_root` is also always empty.
// 3. last element of `segments` is the file name.
//
// visual example:
//
// '/foo/bar/baz.html'.split('/') -> [ '', 'foo', 'bar', 'baz.html' ]
// '../'.split('/') -> [ '..', '' ]
//
// the following operations will then result in
//
// path = 'bar/baz.html'
//
segments.splice(0, segments.length - depth);
const path = segments.join('/');
// anchor starts with the hash character (`#`),
// but our redirect declarations don't, so we strip it.
// example:
// document.location.hash -> '#foo'
// document.location.hash.substring(1) -> 'foo'
const anchor = document.location.hash.substring(1);
const redirect = redirects[path];
if (redirect) {
const target = redirect[anchor];
if (target) {
document.location.href = target;
}
}
+1 -8
View File
@@ -59,19 +59,12 @@
@manpages@
- [Files](command-ref/files.md)
- [nix.conf](command-ref/conf-file.md)
<!--
- [Architecture](architecture/architecture.md)
- [Store](architecture/store/store.md)
- [Closure](architecture/store/store/closure.md)
- [Build system terminology](architecture/store/store/build-system-terminology.md)
- [Store Path](architecture/store/path.md)
- [File System Object](architecture/store/fso.md)
-->
- [Glossary](glossary.md)
- [Contributing](contributing/contributing.md)
- [Hacking](contributing/hacking.md)
- [CLI guideline](contributing/cli-guideline.md)
- [Release Notes](release-notes/release-notes.md)
- [Release 2.12 (2022-12-06)](release-notes/rl-2.12.md)
- [Release 2.11 (2022-08-25)](release-notes/rl-2.11.md)
- [Release 2.10 (2022-07-11)](release-notes/rl-2.10.md)
- [Release 2.9 (2022-05-30)](release-notes/rl-2.9.md)
@@ -1,79 +0,0 @@
# Architecture
*(This chapter is unstable and a work in progress. Incoming links may rot.)*
This chapter describes how Nix works.
It should help users understand why Nix behaves as it does, and it should help developers understand how to modify Nix and how to write similar tools.
## Overview
Nix consists of [hierarchical layers][layer-architecture].
```
+-----------------------------------------------------------------+
| Nix |
| [ commmand line interface ]------, |
| | | |
| evaluates | |
| | manages |
| V | |
| [ configuration language ] | |
| | | |
| +-----------------------------|-------------------V-----------+ |
| | store evaluates to | |
| | | | |
| | referenced by V builds | |
| | [ build input ] ---> [ build plan ] ---> [ build result ] | |
| | | |
| +-------------------------------------------------------------+ |
+-----------------------------------------------------------------+
```
At the top is the [command line interface](../command-ref/command-ref.md), translating from invocations of Nix executables to interactions with the underlying layers.
Below that is the [Nix expression language](../expressions/expression-language.md), a [purely functional][purely-functional-programming] configuration language.
It is used to compose expressions which ultimately evaluate to self-contained *build plans*, used to derive *build results* from referenced *build inputs*.
The command line and Nix language are what users interact with most.
> **Note**
> The Nix language itself does not have a notion of *packages* or *configurations*.
> As far as we are concerned here, the inputs and results of a build plan are just data.
Underlying these is the [Nix store](./store/store.md), a mechanism to keep track of build plans, data, and references between them.
It can also execute build plans to produce new data.
A build plan is a series of *build tasks*.
Each build task has a special build input which is used as *build instructions*.
The result of a build task can be input to another build task.
```
+-----------------------------------------------------------------------------------------+
| store |
| ................................................. |
| : build plan : |
| : : |
| [ build input ]-----instructions-, : |
| : | : |
| : v : |
| [ build input ]----------->[ build task ]--instructions-, : |
| : | : |
| : | : |
| : v : |
| : [ build task ]----->[ build result ] |
| [ build input ]-----instructions-, ^ : |
| : | | : |
| : v | : |
| [ build input ]----------->[ build task ]---------------' : |
| : ^ : |
| : | : |
| [ build input ]------------------' : |
| : : |
| : : |
| :...............................................: |
| |
+-----------------------------------------------------------------------------------------+
```
[layer-architecture]: https://en.m.wikipedia.org/wiki/Multitier_architecture#Layers
[purely-functional-programming]: https://en.m.wikipedia.org/wiki/Purely_functional_programming
-69
View File
@@ -1,69 +0,0 @@
# File System Object
The Nix store uses a simple file system model for the data it holds in [store objects](store.md#store-object).
Every file system object is one of the following:
- File: an executable flag, and arbitrary data for contents
- Directory: mapping of names to child file system objects
- [Symbolic link][symlink]: may point anywhere.
We call a store object's outermost file system object the *root*.
data FileSystemObject
= File { isExecutable :: Bool, contents :: Bytes }
| Directory { entries :: Map FileName FileSystemObject }
| SymLink { target :: Path }
Examples:
- a directory with contents
/nix/store/<hash>-hello-2.10
├── bin
│   └── hello
└── share
├── info
│   └── hello.info
└── man
└── man1
└── hello.1.gz
- a directory with relative symlink and other contents
/nix/store/<hash>-go-1.16.9
├── bin -> share/go/bin
├── nix-support/
└── share/
- a directory with absolute symlink
/nix/store/d3k...-nodejs
└── nix_node -> /nix/store/f20...-nodejs-10.24.
A bare file or symlink can be a root file system object.
Examples:
/nix/store/<hash>-hello-2.10.tar.gz
/nix/store/4j5...-pkg-config-wrapper-0.29.2-doc -> /nix/store/i99...-pkg-config-0.29.2-doc
Symlinks pointing outside of their own root or to a store object without a matching reference are allowed, but might not function as intended.
Examples:
- an arbitrarily symlinked file may change or not exist at all
/nix/store/<hash>-foo
└── foo -> /home/foo
- if a symlink to a store path was not automatically created by Nix, it may be invalid or get invalidated when the store object is deleted
/nix/store/<hash>-bar
└── bar -> /nix/store/abc...-foo
Nix file system objects do not support [hard links][hardlink]:
each file system object which is not the root has exactly one parent and one name.
However, as store objects are immutable, an underlying file system can use hard links for optimization.
[symlink]: https://en.m.wikipedia.org/wiki/Symbolic_link
[hardlink]: https://en.m.wikipedia.org/wiki/Hard_link
-105
View File
@@ -1,105 +0,0 @@
# Store Path
Nix implements [references](store.md#reference) to [store objects](store.md#store-object) as *store paths*.
Store paths are pairs of
- a 20-byte [digest](#digest) for identification
- a symbolic name for people to read.
Example:
- digest: `b6gvzjyb2pg0kjfwrjmg1vfhh54ad73z`
- name: `firefox-33.1`
It is rendered to a file system path as the concatenation of
- [store directory](#store-directory)
- path-separator (`/`)
- [digest](#digest) rendered in a custom variant of [base-32](https://en.m.wikipedia.org/wiki/Base32) (20 arbitrary bytes become 32 ASCII characters)
- hyphen (`-`)
- name
Example:
/nix/store/b6gvzjyb2pg0kjfwrjmg1vfhh54ad73z-firefox-33.1
|--------| |------------------------------| |----------|
store directory digest name
## Store Directory
Every [store](./store.md) has a store directory.
If the store has a [file system representation](./store.md#files-and-processes), this directory contains the store’s [file system objects](#file-system-object), which can be addressed by [store paths](#store-path).
This means a store path is not just derived from the referenced store object itself, but depends on the store the store object is in.
> **Note**
> The store directory defaults to `/nix/store`, but is in principle arbitrary.
It is important which store a given store object belongs to:
Files in the store object can contain store paths, and processes may read these paths.
Nix can only guarantee [referential integrity](store/closure.md) if store paths do not cross store boundaries.
Therefore one can only copy store objects to a different store if
- the source and target stores' directories match
or
- the store object in question has no references, that is, contains no store paths.
One cannot copy a store object to a store with a different store directory.
Instead, it has to be rebuilt, together with all its dependencies.
It is in general not enough to replace the store directory string in file contents, as this may render executables unusable by invalidating their internal offsets or checksums.
# Digest
In a [store path](#store-path), the [digest][digest] is the output of a [cryptographic hash function][hash] of either all *inputs* involved in building the referenced store object or its actual *contents*.
Store objects are therefore said to be either [input-addressed](#input-addressing) or [content-addressed](#content-addressing).
> **Historical Note**
> The 20 byte restriction is because originally digests were [SHA-1][sha-1] hashes.
> Nix now uses [SHA-256][sha-256], and longer hashes are still reduced to 20 bytes for compatibility.
[digest]: https://en.m.wiktionary.org/wiki/digest#Noun
[hash]: https://en.m.wikipedia.org/wiki/Cryptographic_hash_function
[sha-1]: https://en.m.wikipedia.org/wiki/SHA-1
[sha-256]: https://en.m.wikipedia.org/wiki/SHA-256
### Reference scanning
When a new store object is built, Nix scans its file contents for store paths to construct its set of references.
The special format of a store path's [digest](#digest) allows reliably detecting it among arbitrary data.
Nix uses the [closure](store.md#closure) of build inputs to derive the list of allowed store paths, to avoid false positives.
This way, scanning files captures run time dependencies without the user having to declare them explicitly.
Doing it at build time and persisting references in the store object avoids repeating this time-consuming operation.
> **Note**
> In practice, it is sometimes still necessary for users to declare certain dependencies explicitly, if they are to be preserved in the build result's closure.
This depends on the specifics of the software to build and run.
>
> For example, Java programs are compressed after compilation, which obfuscates any store paths they may refer to and prevents Nix from automatically detecting them.
## Input Addressing
Input addressing means that the digest derives from how the store object was produced, namely its build inputs and build plan.
To compute the hash of a store object one needs a deterministic serialisation, i.e., a binary string representation which only changes if the store object changes.
Nix has a custom serialisation format called Nix Archive (NAR)
Store object references of this sort can *not* be validated from the content of the store object.
Rather, a cryptographic signature has to be used to indicate that someone is vouching for the store object really being produced from a build plan with that digest.
## Content Addressing
Content addressing means that the digest derives from the store object's contents, namely its file system objects and references.
If one knows content addressing was used, one can recalculate the reference and thus verify the store object.
Content addressing is currently only used for the special cases of source files and "fixed-output derivations", where the contents of a store object are known in advance.
Content addressing of build results is still an [experimental feature subject to some restrictions](https://github.com/tweag/rfcs/blob/cas-rfc/rfcs/0062-content-addressed-paths.md).
-151
View File
@@ -1,151 +0,0 @@
# Store
A Nix store is a collection of *store objects* with references between them.
It supports operations to manipulate that collection.
The following concept map is a graphical outline of this chapter.
Arrows indicate suggested reading order.
```
,--------------[ store ]----------------,
| | |
v v v
[ store object ] [ closure ]--, [ operations ]
| | | | | |
v | | v v |
[ files and processes ] | | [ garbage collection ] |
/ \ | | |
v v | v v
[ file system object ] [ store path ] | [ derivation ]--->[ building ]
| ^ | | |
v | v v |
[ digest ]----' [ reference scanning ]<------------'
/ \
v v
[ input addressing ] [ content addressing ]
```
## Store Object
A store object can hold
- arbitrary *data*
- *references* to other store objects.
Store objects can be build inputs, build results, or build tasks.
Store objects are [immutable][immutable-object]: once created, they do not change until they are deleted.
## Reference
A store object reference is an [opaque][opaque-data-type], [unique identifier][unique-identifier]:
The only way to obtain references is by adding or building store objects.
A reference will always point to exactly one store object.
## Operations
A Nix store can *add*, *retrieve*, and *delete* store objects.
[ data ]
|
V
[ store ] ---> add ----> [ store' ]
|
V
[ reference ]
<!-- -->
[ reference ]
|
V
[ store ] ---> get
|
V
[ store object ]
<!-- -->
[ reference ]
|
V
[ store ] --> delete --> [ store' ]
It can *perform builds*, that is, create new store objects by transforming build inputs into build outputs, using instructions from the build tasks.
[ reference ]
|
V
[ store ] --> build --(maybe)--> [ store' ]
|
V
[ reference ]
As it keeps track of references, it can [garbage-collect][garbage-collection] unused store objects.
[ store ] --> collect garbage --> [ store' ]
## Files and Processes
Nix maps between its store model and the [Unix paradigm][unix-paradigm] of [files and processes][file-descriptor], by encoding immutable store objects and opaque identifiers as file system primitives: files and directories, and paths.
That allows processes to resolve references contained in files and thus access the contents of store objects.
Store objects are therefore implemented as the pair of
- a [file system object](fso.md) for data
- a set of [store paths](path.md) for references.
[unix-paradigm]: https://en.m.wikipedia.org/wiki/Everything_is_a_file
[file-descriptor]: https://en.m.wikipedia.org/wiki/File_descriptor
The following diagram shows a radical simplification of how Nix interacts with the operating system:
It uses files as build inputs, and build outputs are files again.
On the operating system, files can be run as processes, which in turn operate on files.
A build function also amounts to an operating system process (not depicted).
```
+-----------------------------------------------------------------+
| Nix |
| [ commmand line interface ]------, |
| | | |
| evaluates | |
| | manages |
| V | |
| [ configuration language ] | |
| | | |
| +-----------------------------|-------------------V-----------+ |
| | store evaluates to | |
| | | | |
| | referenced by V builds | |
| | [ build input ] ---> [ build plan ] ---> [ build result ] | |
| | ^ | | |
| +---------|----------------------------------------|----------+ |
+-----------|----------------------------------------|------------+
| |
file system object store path
| |
+-----------|----------------------------------------|------------+
| operating system +------------+ | |
| '------------ | | <-----------' |
| | file | |
| ,-- | | <-, |
| | +------------+ | |
| execute as | | read, write, execute |
| | +------------+ | |
| '-> | process | --' |
| +------------+ |
+-----------------------------------------------------------------+
```
There exist different types of stores, which all follow this model.
Examples:
- store on the local file system
- remote store accessible via SSH
- binary cache store accessible via HTTP
To make store objects accessible to processes, stores ultimately have to expose store objects through the file system.
@@ -1,32 +0,0 @@
# A [Rosetta stone][rosetta-stone] for build system terminology
The Nix store's design is comparable to other build systems.
Usage of terms is, for historic reasons, not entirely consistent within the Nix ecosystem, and still subject to slow change.
The following translation table points out similarities and equivalent terms, to help clarify their meaning and inform consistent use in the future.
| generic build system | Nix | [Bazel][bazel] | [Build Systems à la Carte][bsalc] | programming language |
| -------------------------------- | ---------------- | -------------------------------------------------------------------- | --------------------------------- | ------------------------ |
| data (build input, build result) | store object | [artifact][bazel-artifact] | value | value |
| build instructions | builder | ([depends on action type][bazel-actions]) | function | function |
| build task | derivation | [action][bazel-action] | `Task` | [thunk][thunk] |
| build plan | derivation graph | [action graph][bazel-action-graph], [build graph][bazel-build-graph] | `Tasks` | [call graph][call-graph] |
| build | build | build | application of `Build` | evaluation |
| persistence layer | store | [action cache][bazel-action-cache] | `Store` | heap |
All of these systems share features of [declarative programming][declarative-programming] languages, a key insight first put forward by Eelco Dolstra et al. in [Imposing a Memory Management Discipline on Software Deployment][immdsd] (2004), elaborated in his PhD thesis [The Purely Functional Software Deployment Model][phd-thesis] (2006), and further refined by Andrey Mokhov et al. in [Build Systems à la Carte][bsalc] (2018).
[rosetta-stone]: https://en.m.wikipedia.org/wiki/Rosetta_Stone
[bazel]: https://bazel.build/start/bazel-intro
[bazel-artifact]: https://bazel.build/reference/glossary#artifact
[bazel-actions]: https://docs.bazel.build/versions/main/skylark/lib/actions.html
[bazel-action]: https://bazel.build/reference/glossary#action
[bazel-action-graph]: https://bazel.build/reference/glossary#action-graph
[bazel-build-graph]: https://bazel.build/reference/glossary#build-graph
[bazel-action-cache]: https://bazel.build/reference/glossary#action-cache
[thunk]: https://en.m.wikipedia.org/wiki/Thunk
[call-graph]: https://en.m.wikipedia.org/wiki/Call_graph
[declarative-programming]: https://en.m.wikipedia.org/wiki/Declarative_programming
[immdsd]: https://edolstra.github.io/pubs/immdsd-icse2004-final.pdf
[phd-thesis]: https://edolstra.github.io/pubs/phd-thesis.pdf
[bsalc]: https://www.microsoft.com/en-us/research/uploads/prod/2018/03/build-systems.pdf
@@ -1,29 +0,0 @@
# Closure
Nix stores ensure [referential integrity][referential-integrity]: for each store object in the store, all the store objects it references must also be in the store.
The set of all store objects reachable by following references from a given initial set of store objects is called a *closure*.
Adding, building, copying and deleting store objects must be done in a way that preserves referential integrity:
- A newly added store object cannot have references, unless it is a build task.
- Build results must only refer to store objects in the closure of the build inputs.
Building a store object will add appropriate references, according to the build task.
- Store objects being copied must refer to objects already in the destination store.
Recursive copying must either proceed in dependency order or be atomic.
- We can only safely delete store objects which are not reachable from any reference still in use.
<!-- more details in section on garbage collection, link to it once it exists -->
[referential-integrity]: https://en.m.wikipedia.org/wiki/Referential_integrity
[garbage-collection]: https://en.m.wikipedia.org/wiki/Garbage_collection_(computer_science)
[immutable-object]: https://en.m.wikipedia.org/wiki/Immutable_object
[opaque-data-type]: https://en.m.wikipedia.org/wiki/Opaque_data_type
[unique-identifier]: https://en.m.wikipedia.org/wiki/Unique_identifier
@@ -30,8 +30,8 @@ Since `nix-copy-closure` calls `ssh`, you may be asked to type in the
appropriate password or passphrase. In fact, you may be asked _twice_
because `nix-copy-closure` currently connects twice to the remote
machine, first to get the set of paths missing on the target machine,
and second to send the dump of those paths. If this bothers you, use
`ssh-agent`.
and second to send the dump of those paths. When using public key
authentication, you can avoid typing the passphrase with `ssh-agent`.
# Options
+2 -2
View File
@@ -8,6 +8,6 @@
# Description
The Nix daemon is necessary in multi-user Nix installations. It performs
build actions and other operations on the Nix store on behalf of
The Nix daemon is necessary in multi-user Nix installations. It runs
build tasks and other operations on the Nix store on behalf of
unprivileged users.
+1 -1
View File
@@ -71,7 +71,7 @@ paths. Realisation is a somewhat overloaded term:
outputs are already valid, in which case we are done
immediately. Otherwise, there may be [substitutes](../glossary.md)
that produce the outputs (e.g., by downloading them). Finally, the
outputs can be produced by performing the build action described
outputs can be produced by running the build task described
by the derivation.
- If the store path is not a derivation, realisation ensures that the
+74 -4
View File
@@ -42,7 +42,7 @@ $ nix develop
```
To get a shell with a different compilation environment (e.g. stdenv,
gccStdenv, clangStdenv, clang11Stdenv):
gccStdenv, clangStdenv, clang11Stdenv, ccacheStdenv):
```console
$ nix-shell -A devShells.x86_64-linux.clang11StdenvPackages
@@ -54,6 +54,9 @@ or if you have a flake-enabled nix:
$ nix develop .#clang11StdenvPackages
```
Note: you can use `ccacheStdenv` to drastically improve rebuild
time. By default, ccache keeps artifacts in `~/.cache/ccache/`.
To build Nix itself in this shell:
```console
@@ -83,9 +86,7 @@ by:
$ nix develop
```
## Testing
Nix comes with three different flavors of tests: unit, functional and integration.
## Running tests
### Unit-tests
@@ -108,3 +109,72 @@ These tests include everything that needs to interact with external services or
Because these tests are expensive and require more than what the standard github-actions setup provides, they only run on the master branch (on <https://hydra.nixos.org/jobset/nix/master>).
You can run them manually with `nix build .#hydraJobs.tests.{testName}` or `nix-build -A hydraJobs.tests.{testName}`
### Installer tests
After a one-time setup, the Nix repository's GitHub Actions continuous integration (CI) workflow can test the installer each time you push to a branch.
Creating a Cachix cache for your installer tests and adding its authorization token to GitHub enables [two installer-specific jobs in the CI workflow](https://github.com/NixOS/nix/blob/88a45d6149c0e304f6eb2efcc2d7a4d0d569f8af/.github/workflows/ci.yml#L50-L91):
- The `installer` job generates installers for the platforms below and uploads them to your Cachix cache:
- `x86_64-linux`
- `armv6l-linux`
- `armv7l-linux`
- `x86_64-darwin`
- The `installer_test` job (which runs on `ubuntu-latest` and `macos-latest`) will try to install Nix with the cached installer and run a trivial Nix command.
#### One-time setup
1. Have a GitHub account with a fork of the [Nix repository](https://github.com/NixOS/nix).
2. At cachix.org:
- Create or log in to an account.
- Create a Cachix cache using the format `<github-username>-nix-install-tests`.
- Navigate to the new cache > Settings > Auth Tokens.
- Generate a new Cachix auth token and copy the generated value.
3. At github.com:
- Navigate to your Nix fork > Settings > Secrets > Actions > New repository secret.
- Name the secret `CACHIX_AUTH_TOKEN`.
- Paste the copied value of the Cachix cache auth token.
#### Using the CI-generated installer for manual testing
After the CI run completes, you can check the output to extract the installer URL:
1. Click into the detailed view of the CI run.
2. Click into any `installer_test` run (the URL you're here to extract will be the same in all of them).
3. Click into the `Run cachix/install-nix-action@v...` step and click the detail triangle next to the first log line (it will also be `Run cachix/install-nix-action@v...`)
4. Copy the value of `install_url`
5. To generate an install command, plug this `install_url` and your GitHub username into this template:
```console
sh <(curl -L <install_url>) --tarball-url-prefix https://<github-username>-nix-install-tests.cachix.org/serve
```
<!-- #### Manually generating test installers
There's obviously a manual way to do this, and it's still the only way for
platforms that lack GA runners.
I did do this back in Fall 2020 (before the GA approach encouraged here). I'll
sketch what I recall in case it encourages someone to fill in detail, but: I
didn't know what I was doing at the time and had to fumble/ask around a lot--
so I don't want to uphold any of it as "right". It may have been dumb or
the _hard_ way from the getgo. Fundamentals may have changed since.
Here's the build command I used to do this on and for x86_64-darwin:
nix build --out-link /tmp/foo ".#checks.x86_64-darwin.binaryTarball"
I used the stable out-link to make it easier to script the next steps:
link=$(readlink /tmp/foo)
cp $link/*-darwin.tar.xz ~/somewheres
I've lost the last steps and am just going from memory:
From here, I think I had to extract and modify the `install` script to point
it at this tarball (which I scped to my own site, but it might make more sense
to just share them locally). I extracted this script once and then just
search/replaced in it for each new build.
The installer now supports a `--tarball-url-prefix` flag which _may_ have
solved this need?
-->
+55 -3
View File
@@ -1,16 +1,50 @@
# Glossary
- [derivation]{#gloss-derivation}\
A description of a build action. The result of a derivation is a
A description of a build task. The result of a derivation is a
store object. Derivations are typically specified in Nix expressions
using the [`derivation` primitive](language/derivations.md). These are
translated into low-level *store derivations* (implicitly by
`nix-env` and `nix-build`, or explicitly by `nix-instantiate`).
- [content-addressed derivation]{#gloss-content-addressed-derivation}\
A derivation which has the
[`__contentAddressed`](language/advanced-attributes.md#adv-attr-__contentAddressed)
attribute set to `true`.
- [fixed-output derivation]{#gloss-fixed-output-derivation}\
A derivation which includes the
[`outputHash`](language/advanced-attributes.md#adv-attr-outputHash) attribute.
- [store]{#gloss-store}\
The location in the file system where store objects live. Typically
`/nix/store`.
From the perspective of the location where Nix is
invoked, the Nix store can be referred to
as a "_local_" or a "_remote_" one:
+ A *local store* exists on the filesystem of
the machine where Nix is invoked. You can use other
local stores by passing the `--store` flag to the
`nix` command. Local stores can be used for building derivations.
+ A *remote store* exists anywhere other than the
local filesystem. One example is the `/nix/store`
directory on another machine, accessed via `ssh` or
served by the `nix-serve` Perl script.
- [chroot store]{#gloss-chroot-store}\
A local store whose canonical path is anything other than `/nix/store`.
- [binary cache]{#gloss-binary-cache}\
A *binary cache* is a Nix store which uses a different format: its
metadata and signatures are kept in `.narinfo` files rather than in a
Nix database. This different format simplifies serving store objects
over the network, but cannot host builds. Examples of binary caches
include S3 buckets and the [NixOS binary
cache](https://cache.nixos.org).
- [store path]{#gloss-store-path}\
The location in the file system of a store object, i.e., an
immediate child of the Nix store directory.
@@ -19,8 +53,21 @@
A file that is an immediate child of the Nix store directory. These
can be regular files, but also entire directory trees. Store objects
can be sources (objects copied from outside of the store),
derivation outputs (objects produced by running a build action), or
derivations (files describing a build action).
derivation outputs (objects produced by running a build task), or
derivations (files describing a build task).
- [input-addressed store object]{#gloss-input-addressed-store-object}\
A store object produced by building a
non-[content-addressed](#gloss-content-addressed-derivation),
non-[fixed-output](#gloss-fixed-output-derivation)
derivation.
- [output-addressed store object]{#gloss-output-addressed-store-object}\
A store object whose store path hashes its content. This
includes derivations, the outputs of
[content-addressed derivations](#gloss-content-addressed-derivation),
and the outputs of
[fixed-output derivations](#gloss-fixed-output-derivation).
- [substitute]{#gloss-substitute}\
A substitute is a command invocation stored in the Nix database that
@@ -29,6 +76,11 @@
store object by downloading a pre-built version of the store object
from some server.
- [substituter]{#gloss-substituter}\
A *substituter* is an additional store from which Nix will
copy store objects it doesn't have. For details, see the
[`substituters` option](command-ref/conf-file.html#conf-substituters).
- [purity]{#gloss-purity}\
The assumption that equal Nix derivations when run always produce
the same output. This cannot be guaranteed in general (e.g., a
@@ -88,10 +88,24 @@ extension. The installer will also create `/etc/profile.d/nix.sh`.
### Linux
```console
sudo rm -rf /etc/profile/nix.sh /etc/nix /nix ~root/.nix-profile ~root/.nix-defexpr ~root/.nix-channels ~/.nix-profile ~/.nix-defexpr ~/.nix-channels
Remove files created by Nix:
# If you are on Linux with systemd, you will need to run:
```console
sudo rm -rf /nix /etc/nix /etc/profile/nix.sh ~root/.nix-profile ~root/.nix-defexpr ~root/.nix-channels ~/.nix-profile ~/.nix-defexpr ~/.nix-channels
```
Remove build users and their group:
```console
for i in $(seq 30001 30032); do
sudo userdel $i
done
sudo groupdel 30000
```
If you are on Linux with systemd, remove the Nix daemon service:
```console
sudo systemctl stop nix-daemon.socket
sudo systemctl stop nix-daemon.service
sudo systemctl disable nix-daemon.socket
@@ -99,8 +113,13 @@ sudo systemctl disable nix-daemon.service
sudo systemctl daemon-reload
```
There may also be references to Nix in `/etc/profile`, `/etc/bashrc`,
and `/etc/zshrc` which you may remove.
There may also be references to Nix in
- `/etc/profile`
- `/etc/bashrc`
- `/etc/zshrc`
which you may remove.
### macOS
+1 -1
View File
@@ -104,7 +104,7 @@ a currently running program.
Packages are built from _Nix expressions_, which is a simple
functional language. A Nix expression describes everything that goes
into a package build action (a “derivation”): other packages, sources,
into a package build task (a “derivation”): other packages, sources,
the build script, environment variables for the build script, etc.
Nix tries very hard to ensure that Nix expressions are
_deterministic_: building a Nix expression twice should yield the same
+1 -1
View File
@@ -1,7 +1,7 @@
# Derivations
The most important built-in function is `derivation`, which is used to
describe a single derivation (a build action). It takes as input a set,
describe a single derivation (a build task). It takes as input a set,
the attributes of which specify the inputs of the build.
- There must be an attribute named [`system`]{#attr-system} whose value must be a
+548
View File
@@ -31,3 +31,551 @@ The Nix language is
Type errors are only detected when expressions are evaluated.
# Overview
This is an incomplete overview of language features, by example.
<table>
<tr>
<th>
Example
</th>
<th>
Description
</th>
</tr>
<tr>
<td>
*Basic values*
</td>
<td>
</td>
</tr>
<tr>
<td>
`"hello world"`
</td>
<td>
A string
</td>
</tr>
<tr>
<td>
```
''
multi
line
string
''
```
</td>
<td>
A multi-line string. Strips common prefixed whitespace. Evaluates to `"multi\n line\n string"`.
</td>
</tr>
<tr>
<td>
`"hello ${ { a = "world" }.a }"`
`"1 2 ${toString 3}"`
`"${pkgs.bash}/bin/sh"`
</td>
<td>
String interpolation (expands to `"hello world"`, `"1 2 3"`, `"/nix/store/<hash>-bash-<version>/bin/sh"`)
</td>
</tr>
<tr>
<td>
`true`, `false`
</td>
<td>
Booleans
</td>
</tr>
<tr>
<td>
`null`
</td>
<td>
Null value
</td>
</tr>
<tr>
<td>
`123`
</td>
<td>
An integer
</td>
</tr>
<tr>
<td>
`3.141`
</td>
<td>
A floating point number
</td>
</tr>
<tr>
<td>
`/etc`
</td>
<td>
An absolute path
</td>
</tr>
<tr>
<td>
`./foo.png`
</td>
<td>
A path relative to the file containing this Nix expression
</td>
</tr>
<tr>
<td>
`~/.config`
</td>
<td>
A home path. Evaluates to the `"<user's home directory>/.config"`.
</td>
</tr>
<tr>
<td>
<nixpkgs>
</td>
<td>
Search path. Value determined by [`$NIX_PATH` environment variable](../command-ref/env-common.md#env-NIX_PATH).
</td>
</tr>
<tr>
<td>
*Compound values*
</td>
<td>
</td>
</tr>
<tr>
<td>
`{ x = 1; y = 2; }`
</td>
<td>
A set with attributes named `x` and `y`
</td>
</tr>
<tr>
<td>
`{ foo.bar = 1; }`
</td>
<td>
A nested set, equivalent to `{ foo = { bar = 1; }; }`
</td>
</tr>
<tr>
<td>
`rec { x = "foo"; y = x + "bar"; }`
</td>
<td>
A recursive set, equivalent to `{ x = "foo"; y = "foobar"; }`
</td>
</tr>
<tr>
<td>
`[ "foo" "bar" "baz" ]`
`[ 1 2 3 ]`
`[ (f 1) { a = 1; b = 2; } [ "c" ] ]`
</td>
<td>
Lists with three elements.
</td>
</tr>
<tr>
<td>
*Operators*
</td>
<td>
</td>
</tr>
<tr>
<td>
`"foo" + "bar"`
</td>
<td>
String concatenation
</td>
</tr>
<tr>
<td>
`1 + 2`
</td>
<td>
Integer addition
</td>
</tr>
<tr>
<td>
`"foo" == "f" + "oo"`
</td>
<td>
Equality test (evaluates to `true`)
</td>
</tr>
<tr>
<td>
`"foo" != "bar"`
</td>
<td>
Inequality test (evaluates to `true`)
</td>
</tr>
<tr>
<td>
`!true`
</td>
<td>
Boolean negation
</td>
</tr>
<tr>
<td>
`{ x = 1; y = 2; }.x`
</td>
<td>
Attribute selection (evaluates to `1`)
</td>
</tr>
<tr>
<td>
`{ x = 1; y = 2; }.z or 3`
</td>
<td>
Attribute selection with default (evaluates to `3`)
</td>
</tr>
<tr>
<td>
`{ x = 1; y = 2; } // { z = 3; }`
</td>
<td>
Merge two sets (attributes in the right-hand set taking precedence)
</td>
</tr>
<tr>
<td>
*Control structures*
</td>
<td>
</td>
</tr>
<tr>
<td>
`if 1 + 1 == 2 then "yes!" else "no!"`
</td>
<td>
Conditional expression
</td>
</tr>
<tr>
<td>
`assert 1 + 1 == 2; "yes!"`
</td>
<td>
Assertion check (evaluates to `"yes!"`).
</td>
</tr>
<tr>
<td>
`let x = "foo"; y = "bar"; in x + y`
</td>
<td>
Variable definition
</td>
</tr>
<tr>
<td>
`with builtins; head [ 1 2 3 ]`
</td>
<td>
Add all attributes from the given set to the scope (evaluates to `1`)
</td>
</tr>
<tr>
<td>
*Functions (lambdas)*
</td>
<td>
</td>
</tr>
<tr>
<td>
`x: x + 1`
</td>
<td>
A function that expects an integer and returns it increased by 1
</td>
</tr>
<tr>
<td>
`x: y: x + y`
</td>
<td>
Curried function, equivalent to `x: (y: x + y)`. Can be used like a function that takes two arguments and returns their sum.
</td>
</tr>
<tr>
<td>
`(x: x + 1) 100`
</td>
<td>
A function call (evaluates to 101)
</td>
</tr>
<tr>
<td>
`let inc = x: x + 1; in inc (inc (inc 100))`
</td>
<td>
A function bound to a variable and subsequently called by name (evaluates to 103)
</td>
</tr>
<tr>
<td>
`{ x, y }: x + y`
</td>
<td>
A function that expects a set with required attributes `x` and `y` and concatenates them
</td>
</tr>
<tr>
<td>
`{ x, y ? "bar" }: x + y`
</td>
<td>
A function that expects a set with required attribute `x` and optional `y`, using `"bar"` as default value for `y`
</td>
</tr>
<tr>
<td>
`{ x, y, ... }: x + y`
</td>
<td>
A function that expects a set with required attributes `x` and `y` and ignores any other attributes
</td>
</tr>
<tr>
<td>
`{ x, y } @ args: x + y`
`args @ { x, y }: x + y`
</td>
<td>
A function that expects a set with required attributes `x` and `y`, and binds the whole set to `args`
</td>
</tr>
<tr>
<td>
*Built-in functions*
</td>
<td>
</td>
</tr>
<tr>
<td>
`import ./foo.nix`
</td>
<td>
Load and return Nix expression in given file
</td>
</tr>
<tr>
<td>
`map (x: x + x) [ 1 2 3 ]`
</td>
<td>
Apply a function to every element of a list (evaluates to `[ 2 4 6 ]`)
</td>
</tr>
</table>
+14
View File
@@ -150,6 +150,20 @@
recognized as a path. `a.${foo}/b.${bar}` is a syntactically valid division
operation. `./a.${foo}/b.${bar}` is a path.
When a path appears in an antiquotation, and is thus coerced into a string,
the path is first copied into the Nix store and the resulting string is
the Nix store path. For instance `"${./foo.txt}" will cause `foo.txt` in
the current directory to be copied into the Nix store and result in the
string `"/nix/store/<HASH>-foo.txt"`.
Note that the Nix language assumes that all input files will remain
_unchanged_ during the course of the Nix expression evaluation.
If you for example antiquote a file path during a `nix repl` session, and
then later in the same session, after having changed the file contents,
evaluate the antiquotation with the file path again, then Nix will still
return the first store path. It will _not_ reread the file contents to
produce a different Nix store path.
- <a id="type-boolean" href="#type-boolean">Boolean</a>
*Booleans* with values `true` and `false`.
+43
View File
@@ -0,0 +1,43 @@
# Release 2.12 (2022-12-06)
* On Linux, Nix can now run builds in a user namespace where they run
as root (UID 0) and have 65,536 UIDs available.
<!-- FIXME: move this to its own section about system features -->
This is primarily useful for running containers such as `systemd-nspawn`
inside a Nix build. For an example, see [`tests/systemd-nspawn/nix`][nspawn].
[nspawn]: https://github.com/NixOS/nix/blob/67bcb99700a0da1395fa063d7c6586740b304598/tests/systemd-nspawn.nix.
A build can enable this by setting the derivation attribute:
```
requiredSystemFeatures = [ "uid-range" ];
```
The `uid-range` [system feature] requires the [`auto-allocate-uids`]
setting to be enabled.
[system feature]: (../command-ref/conf-file.md#conf-system-features)
* Nix can now automatically pick UIDs for builds, removing the need to
create `nixbld*` user accounts. See [`auto-allocate-uids`].
[`auto-allocate-uids`]: (../command-ref/conf-file.md#conf-auto-allocate-uids)
* On Linux, Nix has experimental support for running builds inside a
cgroup. See
[`use-cgroups`](../command-ref/conf-file.md#conf-use-cgroups).
* `<nix/fetchurl.nix>` now accepts an additional argument `impure` which
defaults to `false`. If it is set to `true`, the `hash` and `sha256`
arguments will be ignored and the resulting derivation will have
`__impure` set to `true`, making it an impure derivation.
* If `builtins.readFile` is called on a file with context, then only
the parts of the context that appear in the content of the file are
retained. This avoids a lot of spurious errors where strings end up
having a context just because they are read from a store path
([#7260](https://github.com/NixOS/nix/pull/7260)).
* `nix build --json` now prints some statistics about top-level
derivations, such as CPU statistics when cgroups are enabled.
+26
View File
@@ -5,6 +5,32 @@ rec {
concatStrings = concatStringsSep "";
replaceStringsRec = from: to: string:
# recursively replace occurrences of `from` with `to` within `string`
# example:
# replaceStringRec "--" "-" "hello-----world"
# => "hello-world"
let
replaced = replaceStrings [ from ] [ to ] string;
in
if replaced == string then string else replaceStringsRec from to replaced;
squash = replaceStringsRec "\n\n\n" "\n\n";
trim = string:
# trim trailing spaces and squash non-leading spaces
let
trimLine = line:
let
# separate leading spaces from the rest
parts = split "(^ *)" line;
spaces = head (elemAt parts 1);
rest = elemAt parts 2;
# drop trailing spaces
body = head (split " *$" rest);
in spaces + replaceStringsRec " " " " body;
in concatStringsSep "\n" (map trimLine (splitLines string));
# FIXME: O(n^2)
unique = foldl' (acc: e: if elem e acc then acc else acc ++ [ e ]) [];
+13 -1
View File
@@ -33,9 +33,20 @@ let
root = {
uid = 0;
shell = "/bin/bash";
shell = "${pkgs.bashInteractive}/bin/bash";
home = "/root";
gid = 0;
groups = [ "root" ];
description = "System administrator";
};
nobody = {
uid = 65534;
shell = "${pkgs.shadow}/bin/nologin";
home = "/var/empty";
gid = 65534;
groups = [ "nobody" ];
description = "Unprivileged account (don't use!)";
};
} // lib.listToAttrs (
@@ -57,6 +68,7 @@ let
groups = {
root.gid = 0;
nixbld.gid = 30000;
nobody.gid = 65534;
};
userToPasswd = (
+13 -1
View File
@@ -23,7 +23,7 @@
crossSystems = [ "armv6l-linux" "armv7l-linux" ];
stdenvs = [ "gccStdenv" "clangStdenv" "clang11Stdenv" "stdenv" "libcxxStdenv" ];
stdenvs = [ "gccStdenv" "clangStdenv" "clang11Stdenv" "stdenv" "libcxxStdenv" "ccacheStdenv" ];
forAllSystems = f: nixpkgs.lib.genAttrs systems (system: f system);
forAllSystemsAndStdenvs = f: forAllSystems (system:
@@ -260,6 +260,7 @@
echo "file binary-dist $fn" >> $out/nix-support/hydra-build-products
tar cvfJ $fn \
--owner=0 --group=0 --mode=u+rw,uga+r \
--mtime='1970-01-01' \
--absolute-names \
--hard-dereference \
--transform "s,$TMPDIR/install,$dir/install," \
@@ -505,6 +506,12 @@
overlay = self.overlays.default;
});
tests.containers = (import ./tests/containers.nix rec {
system = "x86_64-linux";
inherit nixpkgs;
overlay = self.overlays.default;
});
tests.setuid = nixpkgs.lib.genAttrs
["i686-linux" "x86_64-linux"]
(system:
@@ -545,6 +552,11 @@
# againstLatestStable = testNixVersions pkgs pkgs.nix pkgs.nixStable;
} "touch $out");
installerTests = import ./tests/installer {
binaryTarballs = self.hydraJobs.binaryTarball;
inherit nixpkgsFor;
};
};
checks = forAllSystems (system: {
+79
View File
@@ -0,0 +1,79 @@
# Nix maintainers team
## Motivation
The goal of the team is to help other people to contribute to Nix.
## Members
- Eelco Dolstra (@edolstra) – Team lead
- Théophane Hufschmitt (@thufschmitt)
- Valentin Gagarin (@fricklerhandwerk)
- Thomas Bereknyei (@tomberek)
- Robert Hensing (@roberth)
## Meeting protocol
The team meets twice a week:
- Discussion meeting: [Fridays 13:00-14:00 CET](https://calendar.google.com/calendar/event?eid=MHNtOGVuNWtrZXNpZHR2bW1sM3QyN2ZjaGNfMjAyMjExMjVUMTIwMDAwWiBiOW81MmZvYnFqYWs4b3E4bGZraGczdDBxZ0Bn)
1. Triage issues and pull requests from the _No Status_ column (30 min)
2. Discuss issues and pull requests from the _To discuss_ column (30 min)
- Work meeting: [Mondays 13:00-15:00 CET](https://calendar.google.com/calendar/event?eid=NTM1MG1wNGJnOGpmOTZhYms3bTB1bnY5cWxfMjAyMjExMjFUMTIwMDAwWiBiOW81MmZvYnFqYWs4b3E4bGZraGczdDBxZ0Bn)
1. Code review on pull requests from _In review_.
2. Other chores and tasks.
Meeting notes are collected on a [collaborative scratchpad](https://pad.lassul.us/Cv7FpYx-Ri-4VjUykQOLAw), and published on Discourse under the [Nix category](https://discourse.nixos.org/c/dev/nix/50).
## Project board protocol
The team uses a [GitHub project board](https://github.com/orgs/NixOS/projects/19/views/1) for tracking its work.
Issues on the board progress through the following states:
- No Status
Team members can add pull requests or issues to discuss or review together.
During the discussion meeting, the team triages new items.
If there is disagreement on the general idea behind an issue or pull request, it is moved to _To discuss_, otherwise to _In review_.
- To discuss
Pull requests and issues that are important and controversial are discussed by the team during discussion meetings.
This may be where the merit of the change itself or the implementation strategy is contested by a team member.
- In review
Pull requests in this column are reviewed together during work meetings.
This is both for spreading implementation knowledge and for establishing common values in code reviews.
When the overall direction is agreed upon, even when further changes are required, the pull request is assigned to one team member.
- Assigned for merging
One team member is assigned to each of these pull requests.
They will communicate with the authors, and make the final approval once all remaining issues are addressed.
If more substantive issues arise, the assignee can move the pull request back to _To discuss_ to involve the team again.
The process is illustrated in the following diagram:
```mermaid
flowchart TD
discuss[To discuss]
review[To review]
New --> |Disagreement on idea| discuss
New & discuss --> |Consensus on idea| review
review --> |Consensus on implementation| Assigned
Assigned --> |Implementation issues arise| review
Assigned --> |Remaining issues fixed| Merged
```
+6 -5
View File
@@ -10,14 +10,15 @@ function _nix() {
local -a suggestions
declare -a suggestions
for suggestion in ${res:1}; do
# FIXME: This doesn't work properly if the suggestion word contains a `:`
# itself
suggestions+="${suggestion/ /:}"
suggestions+=("${suggestion%% *}")
done
local -a args
if [[ "$tpe" == filenames ]]; then
compadd -f
args+=('-f')
elif [[ "$tpe" == attrs ]]; then
args+=('-S' '')
fi
_describe 'nix' suggestions
compadd -J nix "${args[@]}" -a suggestions
}
_nix "$@"
+4 -4
View File
@@ -58,7 +58,7 @@ readonly EXTRACTED_NIX_PATH="$(dirname "$0")"
readonly ROOT_HOME=~root
if [ -t 0 ]; then
if [ -t 0 ] && [ -z "${NIX_INSTALLER_YES:-}" ]; then
readonly IS_HEADLESS='no'
else
readonly IS_HEADLESS='yes'
@@ -375,7 +375,7 @@ finish_fail() {
finish_cleanup
failure <<EOF
Jeeze, something went wrong. If you can take all the output and open
Oh no, something went wrong. If you can take all the output and open
an issue, we'd love to fix the problem so nobody else has this issue.
:(
@@ -578,7 +578,7 @@ EOF
# to extract _just_ the user's note, instead it is prefixed with
# some plist junk. This was causing the user note to always be set,
# even if there was no reason for it.
if ! poly_user_note_get "$username" | grep -q "Nix build user $coreid"; then
if poly_user_note_get "$username" | grep -q "Nix build user $coreid"; then
row " Note" "Nix build user $coreid"
else
poly_user_note_set "$username" "Nix build user $coreid"
@@ -823,7 +823,7 @@ EOF
fi
_sudo "to load data for the first time in to the Nix Database" \
"$NIX_INSTALLED_NIX/bin/nix-store" --load-db < ./.reginfo
HOME="$ROOT_HOME" "$NIX_INSTALLED_NIX/bin/nix-store" --load-db < ./.reginfo
echo " Just finished getting the nix database ready."
)
+5 -1
View File
@@ -71,6 +71,8 @@ while [ $# -gt 0 ]; do
# # intentional tail space
# ACTIONS="${ACTIONS}uninstall "
# ;;
--yes)
export NIX_INSTALLER_YES=1;;
--no-channel-add)
export NIX_INSTALLER_NO_CHANNEL_ADD=1;;
--daemon-user-count)
@@ -90,7 +92,7 @@ while [ $# -gt 0 ]; do
shift;;
*)
{
echo "Nix Installer [--daemon|--no-daemon] [--daemon-user-count INT] [--no-channel-add] [--no-modify-profile] [--nix-extra-conf-file FILE]"
echo "Nix Installer [--daemon|--no-daemon] [--daemon-user-count INT] [--yes] [--no-channel-add] [--no-modify-profile] [--nix-extra-conf-file FILE]"
echo "Choose installation method."
echo ""
@@ -104,6 +106,8 @@ while [ $# -gt 0 ]; do
echo " trivial to uninstall."
echo " (default)"
echo ""
echo " --yes: Run the script non-interactively, accepting all prompts."
echo ""
echo " --no-channel-add: Don't add any channels. nixpkgs-unstable is installed by default."
echo ""
echo " --no-modify-profile: Don't modify the user profile to automatically load nix."
+2 -2
View File
@@ -40,12 +40,12 @@ case "$(uname -s).$(uname -m)" in
path=@tarballPath_aarch64-linux@
system=aarch64-linux
;;
Linux.armv6l_linux)
Linux.armv6l)
hash=@tarballHash_armv6l-linux@
path=@tarballPath_armv6l-linux@
system=armv6l-linux
;;
Linux.armv7l_linux)
Linux.armv7l)
hash=@tarballHash_armv7l-linux@
path=@tarballPath_armv7l-linux@
system=armv7l-linux
+17 -3
View File
@@ -1,6 +1,18 @@
function add_path --argument-names new_path
if type -q fish_add_path
# fish 3.2.0 or newer
fish_add_path --prepend --global $new_path
else
# older versions of fish
if not contains $new_path $fish_user_paths
set --global fish_user_paths $new_path $fish_user_paths
end
end
end
# Only execute this file once per shell.
if test -n "$__ETC_PROFILE_NIX_SOURCED"
return
exit
end
set __ETC_PROFILE_NIX_SOURCED 1
@@ -31,5 +43,7 @@ else
end
end
fish_add_path --prepend --global "@localstatedir@/nix/profiles/default/bin"
fish_add_path --prepend --global "$HOME/.nix-profile/bin"
add_path "@localstatedir@/nix/profiles/default/bin"
add_path "$HOME/.nix-profile/bin"
functions -e add_path
+18 -2
View File
@@ -1,3 +1,15 @@
function add_path --argument-names new_path
if type -q fish_add_path
# fish 3.2.0 or newer
fish_add_path --prepend --global $new_path
else
# older versions of fish
if not contains $new_path $fish_user_paths
set --global fish_user_paths $new_path $fish_user_paths
end
end
end
if test -n "$HOME" && test -n "$USER"
# Set up the per-user profile.
@@ -28,8 +40,12 @@ if test -n "$HOME" && test -n "$USER"
# Only use MANPATH if it is already set. In general `man` will just simply
# pick up `.nix-profile/share/man` because is it close to `.nix-profile/bin`
# which is in the $PATH. For more info, run `manpath -d`.
set --export --prepend --path MANPATH "$NIX_LINK/share/man"
if set --query MANPATH
set --export --prepend --path MANPATH "$NIX_LINK/share/man"
end
fish_add_path --prepend --global "$NIX_LINK/bin"
add_path "$NIX_LINK/bin"
set --erase NIX_LINK
end
functions -e add_path
+2 -2
View File
@@ -186,12 +186,12 @@ static int main_build_remote(int argc, char * * argv)
// build the hint template.
std::string errorText =
"Failed to find a machine for remote build!\n"
"derivation: %s\nrequired (system, features): (%s, %s)";
"derivation: %s\nrequired (system, features): (%s, [%s])";
errorText += "\n%s available machines:";
errorText += "\n(systems, maxjobs, supportedFeatures, mandatoryFeatures)";
for (unsigned int i = 0; i < machines.size(); ++i)
errorText += "\n(%s, %s, %s, %s)";
errorText += "\n([%s], %s, [%s], [%s])";
// add the template values.
std::string drvstr;
+3 -2
View File
@@ -88,7 +88,8 @@ EvalCommand::EvalCommand()
{
addFlag({
.longName = "debugger",
.description = "start an interactive environment if evaluation fails",
.description = "Start an interactive environment if evaluation fails.",
.category = MixEvalArgs::category,
.handler = {&startReplOnEvalErrors, true},
});
}
@@ -225,7 +226,7 @@ MixProfile::MixProfile()
{
addFlag({
.longName = "profile",
.description = "The profile to update.",
.description = "The profile to operate on.",
.labels = {"path"},
.handler = {&profile},
.completer = completePath
-2
View File
@@ -13,8 +13,6 @@ namespace nix {
MixEvalArgs::MixEvalArgs()
{
auto category = "Common evaluation options";
addFlag({
.longName = "arg",
.description = "Pass the value *expr* as the argument *name* to Nix functions.",
+2
View File
@@ -10,6 +10,8 @@ class Bindings;
struct MixEvalArgs : virtual Args
{
static constexpr auto category = "Common evaluation options";
MixEvalArgs();
Bindings * getAutoArgs(EvalState & state);
+62 -58
View File
@@ -207,55 +207,59 @@ Strings SourceExprCommand::getDefaultFlakeAttrPathPrefixes()
void SourceExprCommand::completeInstallable(std::string_view prefix)
{
if (file) {
completionType = ctAttrs;
try {
if (file) {
completionType = ctAttrs;
evalSettings.pureEval = false;
auto state = getEvalState();
Expr *e = state->parseExprFromFile(
resolveExprPath(state->checkSourcePath(lookupFileArg(*state, *file)))
);
evalSettings.pureEval = false;
auto state = getEvalState();
Expr *e = state->parseExprFromFile(
resolveExprPath(state->checkSourcePath(lookupFileArg(*state, *file)))
);
Value root;
state->eval(e, root);
Value root;
state->eval(e, root);
auto autoArgs = getAutoArgs(*state);
auto autoArgs = getAutoArgs(*state);
std::string prefix_ = std::string(prefix);
auto sep = prefix_.rfind('.');
std::string searchWord;
if (sep != std::string::npos) {
searchWord = prefix_.substr(sep + 1, std::string::npos);
prefix_ = prefix_.substr(0, sep);
} else {
searchWord = prefix_;
prefix_ = "";
}
std::string prefix_ = std::string(prefix);
auto sep = prefix_.rfind('.');
std::string searchWord;
if (sep != std::string::npos) {
searchWord = prefix_.substr(sep + 1, std::string::npos);
prefix_ = prefix_.substr(0, sep);
} else {
searchWord = prefix_;
prefix_ = "";
}
auto [v, pos] = findAlongAttrPath(*state, prefix_, *autoArgs, root);
Value &v1(*v);
state->forceValue(v1, pos);
Value v2;
state->autoCallFunction(*autoArgs, v1, v2);
auto [v, pos] = findAlongAttrPath(*state, prefix_, *autoArgs, root);
Value &v1(*v);
state->forceValue(v1, pos);
Value v2;
state->autoCallFunction(*autoArgs, v1, v2);
if (v2.type() == nAttrs) {
for (auto & i : *v2.attrs) {
std::string name = state->symbols[i.name];
if (name.find(searchWord) == 0) {
if (prefix_ == "")
completions->add(name);
else
completions->add(prefix_ + "." + name);
if (v2.type() == nAttrs) {
for (auto & i : *v2.attrs) {
std::string name = state->symbols[i.name];
if (name.find(searchWord) == 0) {
if (prefix_ == "")
completions->add(name);
else
completions->add(prefix_ + "." + name);
}
}
}
} else {
completeFlakeRefWithFragment(
getEvalState(),
lockFlags,
getDefaultFlakeAttrPathPrefixes(),
getDefaultFlakeAttrPaths(),
prefix);
}
} else {
completeFlakeRefWithFragment(
getEvalState(),
lockFlags,
getDefaultFlakeAttrPathPrefixes(),
getDefaultFlakeAttrPaths(),
prefix);
} catch (EvalError&) {
// Don't want eval errors to mess-up with the completion engine, so let's just swallow them
}
}
@@ -840,20 +844,20 @@ std::shared_ptr<Installable> SourceExprCommand::parseInstallable(
return installables.front();
}
BuiltPaths Installable::build(
std::vector<BuiltPathWithResult> Installable::build(
ref<Store> evalStore,
ref<Store> store,
Realise mode,
const std::vector<std::shared_ptr<Installable>> & installables,
BuildMode bMode)
{
BuiltPaths res;
for (auto & [_, builtPath] : build2(evalStore, store, mode, installables, bMode))
res.push_back(builtPath);
std::vector<BuiltPathWithResult> res;
for (auto & [_, builtPathWithResult] : build2(evalStore, store, mode, installables, bMode))
res.push_back(builtPathWithResult);
return res;
}
std::vector<std::pair<std::shared_ptr<Installable>, BuiltPath>> Installable::build2(
std::vector<std::pair<std::shared_ptr<Installable>, BuiltPathWithResult>> Installable::build2(
ref<Store> evalStore,
ref<Store> store,
Realise mode,
@@ -873,7 +877,7 @@ std::vector<std::pair<std::shared_ptr<Installable>, BuiltPath>> Installable::bui
}
}
std::vector<std::pair<std::shared_ptr<Installable>, BuiltPath>> res;
std::vector<std::pair<std::shared_ptr<Installable>, BuiltPathWithResult>> res;
switch (mode) {
@@ -899,10 +903,7 @@ std::vector<std::pair<std::shared_ptr<Installable>, BuiltPath>> Installable::bui
DrvOutput outputId { *outputHash, output };
auto realisation = store->queryRealisation(outputId);
if (!realisation)
throw Error(
"cannot operate on an output of the "
"unbuilt derivation '%s'",
outputId.to_string());
throw MissingRealisation(outputId);
outputs.insert_or_assign(output, realisation->outPath);
} else {
// If ca-derivations isn't enabled, assume that
@@ -914,10 +915,10 @@ std::vector<std::pair<std::shared_ptr<Installable>, BuiltPath>> Installable::bui
output, *drvOutput->second);
}
}
res.push_back({installable, BuiltPath::Built { bfd.drvPath, outputs }});
res.push_back({installable, {.path = BuiltPath::Built { bfd.drvPath, outputs }}});
},
[&](const DerivedPath::Opaque & bo) {
res.push_back({installable, BuiltPath::Opaque { bo.path }});
res.push_back({installable, {.path = BuiltPath::Opaque { bo.path }}});
},
}, path.raw());
}
@@ -927,7 +928,7 @@ std::vector<std::pair<std::shared_ptr<Installable>, BuiltPath>> Installable::bui
case Realise::Outputs: {
if (settings.printMissing)
printMissing(store, pathsToBuild, lvlInfo);
printMissing(store, pathsToBuild, lvlInfo);
for (auto & buildResult : store->buildPathsWithResults(pathsToBuild, bMode, evalStore)) {
if (!buildResult.success())
@@ -939,10 +940,10 @@ std::vector<std::pair<std::shared_ptr<Installable>, BuiltPath>> Installable::bui
std::map<std::string, StorePath> outputs;
for (auto & path : buildResult.builtOutputs)
outputs.emplace(path.first.outputName, path.second.outPath);
res.push_back({installable, BuiltPath::Built { bfd.drvPath, outputs }});
res.push_back({installable, {.path = BuiltPath::Built { bfd.drvPath, outputs }, .result = buildResult}});
},
[&](const DerivedPath::Opaque & bo) {
res.push_back({installable, BuiltPath::Opaque { bo.path }});
res.push_back({installable, {.path = BuiltPath::Opaque { bo.path }, .result = buildResult}});
},
}, buildResult.path.raw());
}
@@ -965,9 +966,12 @@ BuiltPaths Installable::toBuiltPaths(
OperateOn operateOn,
const std::vector<std::shared_ptr<Installable>> & installables)
{
if (operateOn == OperateOn::Output)
return Installable::build(evalStore, store, mode, installables);
else {
if (operateOn == OperateOn::Output) {
BuiltPaths res;
for (auto & p : Installable::build(evalStore, store, mode, installables))
res.push_back(p.path);
return res;
} else {
if (mode == Realise::Nothing)
settings.readOnlyMode = true;
+9 -2
View File
@@ -7,6 +7,7 @@
#include "eval.hh"
#include "store-api.hh"
#include "flake/flake.hh"
#include "build-result.hh"
#include <optional>
@@ -51,6 +52,12 @@ enum class OperateOn {
Derivation
};
struct BuiltPathWithResult
{
BuiltPath path;
std::optional<BuildResult> result;
};
struct Installable
{
virtual ~Installable() { }
@@ -91,14 +98,14 @@ struct Installable
return FlakeRef::fromAttrs({{"type","indirect"}, {"id", "nixpkgs"}});
}
static BuiltPaths build(
static std::vector<BuiltPathWithResult> build(
ref<Store> evalStore,
ref<Store> store,
Realise mode,
const std::vector<std::shared_ptr<Installable>> & installables,
BuildMode bMode = bmNormal);
static std::vector<std::pair<std::shared_ptr<Installable>, BuiltPath>> build2(
static std::vector<std::pair<std::shared_ptr<Installable>, BuiltPathWithResult>> build2(
ref<Store> evalStore,
ref<Store> store,
Realise mode,
+1 -1
View File
@@ -8,7 +8,7 @@ libcmd_SOURCES := $(wildcard $(d)/*.cc)
libcmd_CXXFLAGS += -I src/libutil -I src/libstore -I src/libexpr -I src/libmain -I src/libfetchers -I src/nix
libcmd_LDFLAGS = $(EDITLINE_LIBS) -llowdown -pthread
libcmd_LDFLAGS = $(EDITLINE_LIBS) $(LOWDOWN_LIBS) -pthread
libcmd_LIBS = libstore libutil libexpr libmain libfetchers
+1 -1
View File
@@ -18,7 +18,7 @@ std::string renderMarkdownToTerminal(std::string_view markdown)
.hmargin = 0,
.vmargin = 0,
.feat = LOWDOWN_COMMONMARK | LOWDOWN_FENCED | LOWDOWN_DEFLIST | LOWDOWN_TABLES,
.oflags = 0,
.oflags = LOWDOWN_TERM_NOLINK,
};
auto doc = lowdown_doc_new(&opts);
+11 -2
View File
@@ -242,7 +242,11 @@ void NixRepl::mainLoop()
// Allow nix-repl specific settings in .inputrc
rl_readline_name = "nix-repl";
createDirs(dirOf(historyFile));
try {
createDirs(dirOf(historyFile));
} catch (SysError & e) {
logWarning(e.info());
}
#ifndef READLINE
el_hist_size = 1000;
#endif
@@ -266,6 +270,7 @@ void NixRepl::mainLoop()
// ctrl-D should exit the debugger.
state->debugStop = false;
state->debugQuit = true;
logger->cout("");
break;
}
try {
@@ -380,6 +385,10 @@ StringSet NixRepl::completePrefix(const std::string & prefix)
i++;
}
} else {
/* Temporarily disable the debugger, to avoid re-entering readline. */
auto debug_repl = state->debugRepl;
state->debugRepl = nullptr;
Finally restoreDebug([&]() { state->debugRepl = debug_repl; });
try {
/* This is an expression that should evaluate to an
attribute set. Evaluate it to get the names of the
@@ -1046,7 +1055,7 @@ struct CmdRepl : InstallablesCommand
evalSettings.pureEval = false;
}
void prepare()
void prepare() override
{
if (!settings.isExperimentalFeatureEnabled(Xp::ReplFlake) && !(file) && this->_installables.size() >= 1) {
warn("future versions of Nix will require using `--file` to load a file");
+81 -83
View File
@@ -7,7 +7,6 @@
#include "globals.hh"
#include "eval-inline.hh"
#include "filetransfer.hh"
#include "json.hh"
#include "function-trace.hh"
#include <algorithm>
@@ -21,6 +20,7 @@
#include <functional>
#include <sys/resource.h>
#include <nlohmann/json.hpp>
#if HAVE_BOEHMGC
@@ -35,6 +35,8 @@
#endif
using json = nlohmann::json;
namespace nix {
static char * allocString(size_t size)
@@ -69,15 +71,11 @@ static char * dupString(const char * s)
// empty string.
static const char * makeImmutableStringWithLen(const char * s, size_t size)
{
char * t;
if (size == 0)
return "";
#if HAVE_BOEHMGC
t = GC_STRNDUP(s, size);
#else
t = strndup(s, size);
#endif
if (!t) throw std::bad_alloc();
auto t = allocString(size + 1);
memcpy(t, s, size);
t[size] = 0;
return t;
}
@@ -904,7 +902,7 @@ void EvalState::throwEvalError(const char * s, const std::string & s2,
const std::string & s3)
{
debugThrowLastTrace(EvalError({
.msg = hintfmt(s, s2),
.msg = hintfmt(s, s2, s3),
.errPos = positions[noPos]
}));
}
@@ -913,7 +911,7 @@ void EvalState::throwEvalError(const PosIdx pos, const char * s, const std::stri
const std::string & s3)
{
debugThrowLastTrace(EvalError({
.msg = hintfmt(s, s2),
.msg = hintfmt(s, s2, s3),
.errPos = positions[pos]
}));
}
@@ -922,7 +920,7 @@ void EvalState::throwEvalError(const PosIdx pos, const char * s, const std::stri
const std::string & s3, Env & env, Expr & expr)
{
debugThrow(EvalError({
.msg = hintfmt(s, s2),
.msg = hintfmt(s, s2, s3),
.errPos = positions[pos]
}), env, expr);
}
@@ -1648,7 +1646,7 @@ void EvalState::callFunction(Value & fun, size_t nrArgs, Value * * args, Value &
auto dts = debugRepl
? makeDebugTraceStacker(
*this, *lambda.body, env2, positions[lambda.pos],
"while evaluating %s",
"while calling %s",
lambda.name
? concatStrings("'", symbols[lambda.name], "'")
: "anonymous lambda")
@@ -1657,7 +1655,7 @@ void EvalState::callFunction(Value & fun, size_t nrArgs, Value * * args, Value &
lambda.body->eval(*this, env2, vCur);
} catch (Error & e) {
if (loggerSettings.showTrace.get()) {
addErrorTrace(e, lambda.pos, "while evaluating %s",
addErrorTrace(e, lambda.pos, "while calling %s",
(lambda.name
? concatStrings("'", symbols[lambda.name], "'")
: "anonymous lambda"));
@@ -2441,97 +2439,97 @@ void EvalState::printStats()
std::fstream fs;
if (outPath != "-")
fs.open(outPath, std::fstream::out);
JSONObject topObj(outPath == "-" ? std::cerr : fs, true);
topObj.attr("cpuTime",cpuTime);
{
auto envs = topObj.object("envs");
envs.attr("number", nrEnvs);
envs.attr("elements", nrValuesInEnvs);
envs.attr("bytes", bEnvs);
}
{
auto lists = topObj.object("list");
lists.attr("elements", nrListElems);
lists.attr("bytes", bLists);
lists.attr("concats", nrListConcats);
}
{
auto values = topObj.object("values");
values.attr("number", nrValues);
values.attr("bytes", bValues);
}
{
auto syms = topObj.object("symbols");
syms.attr("number", symbols.size());
syms.attr("bytes", symbols.totalSize());
}
{
auto sets = topObj.object("sets");
sets.attr("number", nrAttrsets);
sets.attr("bytes", bAttrsets);
sets.attr("elements", nrAttrsInAttrsets);
}
{
auto sizes = topObj.object("sizes");
sizes.attr("Env", sizeof(Env));
sizes.attr("Value", sizeof(Value));
sizes.attr("Bindings", sizeof(Bindings));
sizes.attr("Attr", sizeof(Attr));
}
topObj.attr("nrOpUpdates", nrOpUpdates);
topObj.attr("nrOpUpdateValuesCopied", nrOpUpdateValuesCopied);
topObj.attr("nrThunks", nrThunks);
topObj.attr("nrAvoided", nrAvoided);
topObj.attr("nrLookups", nrLookups);
topObj.attr("nrPrimOpCalls", nrPrimOpCalls);
topObj.attr("nrFunctionCalls", nrFunctionCalls);
json topObj = json::object();
topObj["cpuTime"] = cpuTime;
topObj["envs"] = {
{"number", nrEnvs},
{"elements", nrValuesInEnvs},
{"bytes", bEnvs},
};
topObj["list"] = {
{"elements", nrListElems},
{"bytes", bLists},
{"concats", nrListConcats},
};
topObj["values"] = {
{"number", nrValues},
{"bytes", bValues},
};
topObj["symbols"] = {
{"number", symbols.size()},
{"bytes", symbols.totalSize()},
};
topObj["sets"] = {
{"number", nrAttrsets},
{"bytes", bAttrsets},
{"elements", nrAttrsInAttrsets},
};
topObj["sizes"] = {
{"Env", sizeof(Env)},
{"Value", sizeof(Value)},
{"Bindings", sizeof(Bindings)},
{"Attr", sizeof(Attr)},
};
topObj["nrOpUpdates"] = nrOpUpdates;
topObj["nrOpUpdateValuesCopied"] = nrOpUpdateValuesCopied;
topObj["nrThunks"] = nrThunks;
topObj["nrAvoided"] = nrAvoided;
topObj["nrLookups"] = nrLookups;
topObj["nrPrimOpCalls"] = nrPrimOpCalls;
topObj["nrFunctionCalls"] = nrFunctionCalls;
#if HAVE_BOEHMGC
{
auto gc = topObj.object("gc");
gc.attr("heapSize", heapSize);
gc.attr("totalBytes", totalBytes);
}
topObj["gc"] = {
{"heapSize", heapSize},
{"totalBytes", totalBytes},
};
#endif
if (countCalls) {
topObj["primops"] = primOpCalls;
{
auto obj = topObj.object("primops");
for (auto & i : primOpCalls)
obj.attr(i.first, i.second);
}
{
auto list = topObj.list("functions");
auto& list = topObj["functions"];
list = json::array();
for (auto & [fun, count] : functionCalls) {
auto obj = list.object();
json obj = json::object();
if (fun->name)
obj.attr("name", (std::string_view) symbols[fun->name]);
obj["name"] = (std::string_view) symbols[fun->name];
else
obj.attr("name", nullptr);
obj["name"] = nullptr;
if (auto pos = positions[fun->pos]) {
obj.attr("file", (std::string_view) pos.file);
obj.attr("line", pos.line);
obj.attr("column", pos.column);
obj["file"] = (std::string_view) pos.file;
obj["line"] = pos.line;
obj["column"] = pos.column;
}
obj.attr("count", count);
obj["count"] = count;
list.push_back(obj);
}
}
{
auto list = topObj.list("attributes");
auto list = topObj["attributes"];
list = json::array();
for (auto & i : attrSelects) {
auto obj = list.object();
json obj = json::object();
if (auto pos = positions[i.first]) {
obj.attr("file", (const std::string &) pos.file);
obj.attr("line", pos.line);
obj.attr("column", pos.column);
obj["file"] = (const std::string &) pos.file;
obj["line"] = pos.line;
obj["column"] = pos.column;
}
obj.attr("count", i.second);
obj["count"] = i.second;
list.push_back(obj);
}
}
}
if (getEnv("NIX_SHOW_SYMBOLS").value_or("0") != "0") {
auto list = topObj.list("symbols");
symbols.dump([&](const std::string & s) { list.elem(s); });
// XXX: overrides earlier assignment
topObj["symbols"] = json::array();
auto &list = topObj["symbols"];
symbols.dump([&](const std::string & s) { list.emplace_back(s); });
}
if (outPath == "-") {
std::cerr << topObj.dump(2) << std::endl;
} else {
fs << topObj.dump(2) << std::endl;
}
}
}
+5 -3
View File
@@ -12,13 +12,13 @@
, executable ? false
, unpack ? false
, name ? baseNameOf (toString url)
, impure ? false
}:
derivation {
derivation ({
builder = "builtin:fetchurl";
# New-style output content requirements.
inherit outputHashAlgo outputHash;
outputHashMode = if unpack || executable then "recursive" else "flat";
inherit name url executable unpack;
@@ -38,4 +38,6 @@ derivation {
# To make "nix-prefetch-url" work.
urls = [ url ];
}
} // (if impure
then { __impure = true; }
else { inherit outputHashAlgo outputHash; }))
+1 -1
View File
@@ -43,7 +43,7 @@ let
outputs = flake.outputs (inputs // { self = result; });
result = outputs // sourceInfo // { inherit inputs; inherit outputs; inherit sourceInfo; };
result = outputs // sourceInfo // { inherit inputs; inherit outputs; inherit sourceInfo; _type = "flake"; };
in
if node.flake or true then
assert builtins.isFunction flake.outputs;
+1 -1
View File
@@ -68,7 +68,7 @@ void ConfigFile::apply()
}
}
if (!trusted) {
warn("ignoring untrusted flake configuration setting '%s'", name);
warn("ignoring untrusted flake configuration setting '%s'.\nPass '%s' to trust it", name, "--accept-flake-config");
continue;
}
}
+2 -2
View File
@@ -483,12 +483,12 @@ LockedFlake lockFlake(
} else if (auto follows = std::get_if<1>(&i.second)) {
if (! trustLock) {
// It is possible that the flake has changed,
// so we must confirm all the follows that are in the lockfile are also in the flake.
// so we must confirm all the follows that are in the lock file are also in the flake.
auto overridePath(inputPath);
overridePath.push_back(i.first);
auto o = overrides.find(overridePath);
// If the override disappeared, we have to refetch the flake,
// since some of the inputs may not be present in the lockfile.
// since some of the inputs may not be present in the lock file.
if (o == overrides.end()) {
mustRefetch = true;
// There's no point populating the rest of the fake inputs,
+1 -1
View File
@@ -36,7 +36,7 @@ LockedNode::LockedNode(const nlohmann::json & json)
, isFlake(json.find("flake") != json.end() ? (bool) json["flake"] : true)
{
if (!lockedRef.input.isLocked())
throw Error("lockfile contains mutable lock '%s'",
throw Error("lock file contains mutable lock '%s'",
fetchers::attrsToJSON(lockedRef.input.toAttrs()));
}
+22 -12
View File
@@ -5,14 +5,15 @@
#include "globals.hh"
#include "json-to-value.hh"
#include "names.hh"
#include "references.hh"
#include "store-api.hh"
#include "util.hh"
#include "json.hh"
#include "value-to-json.hh"
#include "value-to-xml.hh"
#include "primops.hh"
#include <boost/container/small_vector.hpp>
#include <nlohmann/json.hpp>
#include <sys/types.h>
#include <sys/stat.h>
@@ -1010,6 +1011,7 @@ static void prim_second(EvalState & state, const PosIdx pos, Value * * args, Val
derivation. */
static void prim_derivationStrict(EvalState & state, const PosIdx pos, Value * * args, Value & v)
{
using nlohmann::json;
state.forceAttrs(*args[0], pos);
/* Figure out the name first (for stack backtraces). */
@@ -1031,11 +1033,10 @@ static void prim_derivationStrict(EvalState & state, const PosIdx pos, Value * *
}
/* Check whether attributes should be passed as a JSON file. */
std::ostringstream jsonBuf;
std::unique_ptr<JSONObject> jsonObject;
std::optional<json> jsonObject;
attr = args[0]->attrs->find(state.sStructuredAttrs);
if (attr != args[0]->attrs->end() && state.forceBool(*attr->value, pos))
jsonObject = std::make_unique<JSONObject>(jsonBuf);
jsonObject = json::object();
/* Check whether null attributes should be ignored. */
bool ignoreNulls = false;
@@ -1137,8 +1138,7 @@ static void prim_derivationStrict(EvalState & state, const PosIdx pos, Value * *
if (i->name == state.sStructuredAttrs) continue;
auto placeholder(jsonObject->placeholder(key));
printValueAsJSON(state, true, *i->value, pos, placeholder, context);
(*jsonObject)[key] = printValueAsJSON(state, true, *i->value, pos, context);
if (i->name == state.sBuilder)
drv.builder = state.forceString(*i->value, context, posDrvName);
@@ -1182,8 +1182,8 @@ static void prim_derivationStrict(EvalState & state, const PosIdx pos, Value * *
}
if (jsonObject) {
drv.env.emplace("__json", jsonObject->dump());
jsonObject.reset();
drv.env.emplace("__json", jsonBuf.str());
}
/* Everything in the context of the strings in the derivation
@@ -1542,6 +1542,10 @@ static void prim_readFile(EvalState & state, const PosIdx pos, Value * * args, V
refs = state.store->queryPathInfo(state.store->toStorePath(path).first)->references;
} catch (Error &) { // FIXME: should be InvalidPathError
}
// Re-scan references to filter down to just the ones that actually occur in the file.
auto refsSink = PathRefScanSink::fromPaths(refs);
refsSink << s;
refs = refsSink.getResultPaths();
}
auto context = state.store->printStorePathSet(refs);
v.mkString(s, context);
@@ -2416,12 +2420,18 @@ static RegisterPrimOp primop_listToAttrs({
Construct a set from a list specifying the names and values of each
attribute. Each element of the list should be a set consisting of a
string-valued attribute `name` specifying the name of the attribute,
and an attribute `value` specifying its value. Example:
and an attribute `value` specifying its value.
In case of duplicate occurrences of the same name, the first
takes precedence.
Example:
```nix
builtins.listToAttrs
[ { name = "foo"; value = 123; }
{ name = "bar"; value = 456; }
{ name = "bar"; value = 420; }
]
```
@@ -2454,8 +2464,8 @@ static RegisterPrimOp primop_intersectAttrs({
.name = "__intersectAttrs",
.args = {"e1", "e2"},
.doc = R"(
Return a set consisting of the attributes in the set *e2* that also
exist in the set *e1*.
Return a set consisting of the attributes in the set *e2* which have the
same name as some attribute in *e1*.
)",
.fun = prim_intersectAttrs,
});
@@ -3821,8 +3831,8 @@ static RegisterPrimOp primop_parseDrvName({
.args = {"s"},
.doc = R"(
Split the string *s* into a package name and version. The package
name is everything up to but not including the first dash followed
by a digit, and the version is everything following that dash. The
name is everything up to but not including the first dash not followed
by a letter, and the version is everything following that dash. The
result is returned in a set `{ name, version }`. Thus,
`builtins.parseDrvName "nix-0.12pre12876"` returns `{ name =
"nix"; version = "0.12pre12876"; }`.
+24 -26
View File
@@ -1,84 +1,82 @@
#include "value-to-json.hh"
#include "json.hh"
#include "eval-inline.hh"
#include "util.hh"
#include <cstdlib>
#include <iomanip>
#include <nlohmann/json.hpp>
namespace nix {
void printValueAsJSON(EvalState & state, bool strict,
Value & v, const PosIdx pos, JSONPlaceholder & out, PathSet & context, bool copyToStore)
using json = nlohmann::json;
json printValueAsJSON(EvalState & state, bool strict,
Value & v, const PosIdx pos, PathSet & context, bool copyToStore)
{
checkInterrupt();
if (strict) state.forceValue(v, pos);
json out;
switch (v.type()) {
case nInt:
out.write(v.integer);
out = v.integer;
break;
case nBool:
out.write(v.boolean);
out = v.boolean;
break;
case nString:
copyContext(v, context);
out.write(v.string.s);
out = v.string.s;
break;
case nPath:
if (copyToStore)
out.write(state.copyPathToStore(context, v.path));
out = state.copyPathToStore(context, v.path);
else
out.write(v.path);
out = v.path;
break;
case nNull:
out.write(nullptr);
break;
case nAttrs: {
auto maybeString = state.tryAttrsToString(pos, v, context, false, false);
if (maybeString) {
out.write(*maybeString);
out = *maybeString;
break;
}
auto i = v.attrs->find(state.sOutPath);
if (i == v.attrs->end()) {
auto obj(out.object());
out = json::object();
StringSet names;
for (auto & j : *v.attrs)
names.emplace(state.symbols[j.name]);
for (auto & j : names) {
Attr & a(*v.attrs->find(state.symbols.create(j)));
auto placeholder(obj.placeholder(j));
printValueAsJSON(state, strict, *a.value, a.pos, placeholder, context, copyToStore);
out[j] = printValueAsJSON(state, strict, *a.value, a.pos, context, copyToStore);
}
} else
printValueAsJSON(state, strict, *i->value, i->pos, out, context, copyToStore);
return printValueAsJSON(state, strict, *i->value, i->pos, context, copyToStore);
break;
}
case nList: {
auto list(out.list());
for (auto elem : v.listItems()) {
auto placeholder(list.placeholder());
printValueAsJSON(state, strict, *elem, pos, placeholder, context, copyToStore);
}
out = json::array();
for (auto elem : v.listItems())
out.push_back(printValueAsJSON(state, strict, *elem, pos, context, copyToStore));
break;
}
case nExternal:
v.external->printValueAsJSON(state, strict, out, context, copyToStore);
return v.external->printValueAsJSON(state, strict, context, copyToStore);
break;
case nFloat:
out.write(v.fpoint);
out = v.fpoint;
break;
case nThunk:
@@ -91,17 +89,17 @@ void printValueAsJSON(EvalState & state, bool strict,
state.debugThrowLastTrace(e);
throw e;
}
return out;
}
void printValueAsJSON(EvalState & state, bool strict,
Value & v, const PosIdx pos, std::ostream & str, PathSet & context, bool copyToStore)
{
JSONPlaceholder out(str);
printValueAsJSON(state, strict, v, pos, out, context, copyToStore);
str << printValueAsJSON(state, strict, v, pos, context, copyToStore);
}
void ExternalValueBase::printValueAsJSON(EvalState & state, bool strict,
JSONPlaceholder & out, PathSet & context, bool copyToStore) const
json ExternalValueBase::printValueAsJSON(EvalState & state, bool strict,
PathSet & context, bool copyToStore) const
{
state.debugThrowLastTrace(TypeError("cannot convert %1% to JSON", showType()));
}
+3 -4
View File
@@ -5,13 +5,12 @@
#include <string>
#include <map>
#include <nlohmann/json_fwd.hpp>
namespace nix {
class JSONPlaceholder;
void printValueAsJSON(EvalState & state, bool strict,
Value & v, const PosIdx pos, JSONPlaceholder & out, PathSet & context, bool copyToStore = true);
nlohmann::json printValueAsJSON(EvalState & state, bool strict,
Value & v, const PosIdx pos, PathSet & context, bool copyToStore = true);
void printValueAsJSON(EvalState & state, bool strict,
Value & v, const PosIdx pos, std::ostream & str, PathSet & context, bool copyToStore = true);
+3 -3
View File
@@ -7,6 +7,7 @@
#if HAVE_BOEHMGC
#include <gc/gc_allocator.h>
#endif
#include <nlohmann/json_fwd.hpp>
namespace nix {
@@ -62,7 +63,6 @@ class StorePath;
class Store;
class EvalState;
class XMLWriter;
class JSONPlaceholder;
typedef int64_t NixInt;
@@ -98,8 +98,8 @@ class ExternalValueBase
virtual bool operator ==(const ExternalValueBase & b) const;
/* Print the value as JSON. Defaults to unconvertable, i.e. throws an error */
virtual void printValueAsJSON(EvalState & state, bool strict,
JSONPlaceholder & out, PathSet & context, bool copyToStore = true) const;
virtual nlohmann::json printValueAsJSON(EvalState & state, bool strict,
PathSet & context, bool copyToStore = true) const;
/* Print the value as XML. Defaults to unevaluated */
virtual void printValueAsXML(EvalState & state, bool strict, bool location,
+4
View File
@@ -485,6 +485,10 @@ struct GitInputScheme : InputScheme
}
input.attrs.insert_or_assign("ref", *head);
unlockedAttrs.insert_or_assign("ref", *head);
} else {
if (!input.getRev()) {
unlockedAttrs.insert_or_assign("ref", input.getRef().value());
}
}
if (auto res = getCache()->lookup(store, unlockedAttrs)) {
+11 -8
View File
@@ -262,17 +262,20 @@ struct GitHubInputScheme : GitArchiveInputScheme
DownloadUrl getDownloadUrl(const Input & input) const override
{
// FIXME: use regular /archive URLs instead? api.github.com
// might have stricter rate limits.
auto host = maybeGetStrAttr(input.attrs, "host").value_or("github.com");
auto url = fmt(
host == "github.com"
? "https://api.%s/repos/%s/%s/tarball/%s"
: "https://%s/api/v3/repos/%s/%s/tarball/%s",
host, getStrAttr(input.attrs, "owner"), getStrAttr(input.attrs, "repo"),
Headers headers = makeHeadersWithAuthTokens(host);
// If we have no auth headers then we default to the public archive
// urls so we do not run into rate limits.
const auto urlFmt =
host != "github.com"
? "https://%s/api/v3/repos/%s/%s/tarball/%s"
: headers.empty()
? "https://%s/%s/%s/archive/%s.tar.gz"
: "https://api.%s/repos/%s/%s/tarball/%s";
const auto url = fmt(urlFmt, host, getStrAttr(input.attrs, "owner"), getStrAttr(input.attrs, "repo"),
input.getRev()->to_string(Base16, false));
Headers headers = makeHeadersWithAuthTokens(host);
return DownloadUrl { url, headers };
}
+1
View File
@@ -32,6 +32,7 @@ MixCommonArgs::MixCommonArgs(const std::string & programName)
addFlag({
.longName = "option",
.description = "Set the Nix configuration setting *name* to *value* (overriding `nix.conf`).",
.category = miscCategory,
.labels = {"name", "value"},
.handler = {[](std::string name, std::string value) {
try {
+1
View File
@@ -6,6 +6,7 @@ namespace nix {
//static constexpr auto commonArgsCategory = "Miscellaneous common options";
static constexpr auto loggingCategory = "Logging-related options";
static constexpr auto miscCategory = "Miscellaneous global options";
class MixCommonArgs : public virtual Args
{
+1 -1
View File
@@ -503,7 +503,7 @@ public:
return s[0];
}
virtual void setPrintBuildLogs(bool printBuildLogs)
void setPrintBuildLogs(bool printBuildLogs) override
{
this->printBuildLogs = printBuildLogs;
}
+17 -4
View File
@@ -33,6 +33,7 @@
namespace nix {
char * * savedArgv;
static bool gcWarning = true;
@@ -182,8 +183,9 @@ void initNix()
/* Reset SIGCHLD to its default. */
struct sigaction act;
sigemptyset(&act.sa_mask);
act.sa_handler = SIG_DFL;
act.sa_flags = 0;
act.sa_handler = SIG_DFL;
if (sigaction(SIGCHLD, &act, 0))
throw SysError("resetting SIGCHLD");
@@ -195,9 +197,20 @@ void initNix()
/* HACK: on darwin, we need can’t use sigprocmask with SIGWINCH.
* Instead, add a dummy sigaction handler, and signalHandlerThread
* can handle the rest. */
struct sigaction sa;
sa.sa_handler = sigHandler;
if (sigaction(SIGWINCH, &sa, 0)) throw SysError("handling SIGWINCH");
act.sa_handler = sigHandler;
if (sigaction(SIGWINCH, &act, 0)) throw SysError("handling SIGWINCH");
/* Disable SA_RESTART for interrupts, so that system calls on this thread
* error with EINTR like they do on Linux.
* Most signals on BSD systems default to SA_RESTART on, but Nix
* expects EINTR from syscalls to properly exit. */
act.sa_handler = SIG_DFL;
if (sigaction(SIGINT, &act, 0)) throw SysError("handling SIGINT");
if (sigaction(SIGTERM, &act, 0)) throw SysError("handling SIGTERM");
if (sigaction(SIGHUP, &act, 0)) throw SysError("handling SIGHUP");
if (sigaction(SIGPIPE, &act, 0)) throw SysError("handling SIGPIPE");
if (sigaction(SIGQUIT, &act, 0)) throw SysError("handling SIGQUIT");
if (sigaction(SIGTRAP, &act, 0)) throw SysError("handling SIGTRAP");
#endif
/* Register a SIGSEGV handler to detect stack overflows. */
+20
View File
@@ -113,5 +113,25 @@ struct PrintFreed
/* Install a SIGSEGV handler to detect stack overflows. */
void detectStackOverflow();
/* Pluggable behavior to run in case of a stack overflow.
Default value: defaultStackOverflowHandler.
This is called by the handler installed by detectStackOverflow().
This gives Nix library consumers a limit opportunity to report the error
condition. The handler should exit the process.
See defaultStackOverflowHandler() for a reference implementation.
NOTE: Use with diligence, because this runs in the signal handler, with very
limited stack space and a potentially a corrupted heap, all while the failed
thread is blocked indefinitely. All functions called must be reentrant. */
extern std::function<void(siginfo_t * info, void * ctx)> stackOverflowHandler;
/* The default, robust implementation of stackOverflowHandler.
Prints an error message directly to stderr using a syscall instead of the
logger. Exits the process immediately after. */
void defaultStackOverflowHandler(siginfo_t * info, void * ctx);
}
+9 -3
View File
@@ -1,4 +1,5 @@
#include "error.hh"
#include "shared.hh"
#include <cstring>
#include <cstddef>
@@ -29,9 +30,7 @@ static void sigsegvHandler(int signo, siginfo_t * info, void * ctx)
ptrdiff_t diff = (char *) info->si_addr - sp;
if (diff < 0) diff = -diff;
if (diff < 4096) {
char msg[] = "error: stack overflow (possible infinite recursion)\n";
[[gnu::unused]] auto res = write(2, msg, strlen(msg));
_exit(1); // maybe abort instead?
nix::stackOverflowHandler(info, ctx);
}
}
@@ -67,5 +66,12 @@ void detectStackOverflow()
#endif
}
std::function<void(siginfo_t * info, void * ctx)> stackOverflowHandler(defaultStackOverflowHandler);
void defaultStackOverflowHandler(siginfo_t * info, void * ctx) {
char msg[] = "error: stack overflow (possible infinite recursion)\n";
[[gnu::unused]] auto res = write(2, msg, strlen(msg));
_exit(1); // maybe abort instead?
}
}
+16 -13
View File
@@ -9,7 +9,6 @@
#include "remote-fs-accessor.hh"
#include "nar-info-disk-cache.hh"
#include "nar-accessor.hh"
#include "json.hh"
#include "thread-pool.hh"
#include "callback.hh"
@@ -194,19 +193,12 @@ ref<const ValidPathInfo> BinaryCacheStore::addToStoreCommon(
/* Optionally write a JSON file containing a listing of the
contents of the NAR. */
if (writeNARListing) {
std::ostringstream jsonOut;
nlohmann::json j = {
{"version", 1},
{"root", listNar(ref<FSAccessor>(narAccessor), "", true)},
};
{
JSONObject jsonRoot(jsonOut);
jsonRoot.attr("version", 1);
{
auto res = jsonRoot.placeholder("root");
listNar(res, ref<FSAccessor>(narAccessor), "", true);
}
}
upsertFile(std::string(info.path.hashPart()) + ".ls", jsonOut.str(), "application/json");
upsertFile(std::string(info.path.hashPart()) + ".ls", j.dump(), "application/json");
}
/* Optionally maintain an index of DWARF debug info files
@@ -331,6 +323,17 @@ bool BinaryCacheStore::isValidPathUncached(const StorePath & storePath)
return fileExists(narInfoFileFor(storePath));
}
std::optional<StorePath> BinaryCacheStore::queryPathFromHashPart(const std::string & hashPart)
{
auto pseudoPath = StorePath(hashPart + "-" + MissingName);
try {
auto info = queryPathInfo(pseudoPath);
return info->path;
} catch (InvalidPath &) {
return std::nullopt;
}
}
void BinaryCacheStore::narFromPath(const StorePath & storePath, Sink & sink)
{
auto info = queryPathInfo(storePath).cast<const NarInfo>();
+1 -2
View File
@@ -95,8 +95,7 @@ public:
void queryPathInfoUncached(const StorePath & path,
Callback<std::shared_ptr<const ValidPathInfo>> callback) noexcept override;
std::optional<StorePath> queryPathFromHashPart(const std::string & hashPart) override
{ unsupported("queryPathFromHashPart"); }
std::optional<StorePath> queryPathFromHashPart(const std::string & hashPart) override;
void addToStore(const ValidPathInfo & info, Source & narSource,
RepairFlag repair, CheckSigsFlag checkSigs) override;
+4 -1
View File
@@ -5,7 +5,7 @@
#include <string>
#include <chrono>
#include <optional>
namespace nix {
@@ -78,6 +78,9 @@ struct BuildResult
was repeated). */
time_t startTime = 0, stopTime = 0;
/* User and system CPU time the build took. */
std::optional<std::chrono::microseconds> cpuUser, cpuSystem;
bool success()
{
return status == Built || status == Substituted || status == AlreadyValid || status == ResolvesToAlreadyValid;
+17 -2
View File
@@ -7,7 +7,6 @@
#include "finally.hh"
#include "util.hh"
#include "archive.hh"
#include "json.hh"
#include "compression.hh"
#include "worker-protocol.hh"
#include "topo-sort.hh"
@@ -344,7 +343,7 @@ void DerivationGoal::gaveUpOnSubstitution()
for (auto & i : dynamic_cast<Derivation *>(drv.get())->inputDrvs) {
/* Ensure that pure, non-fixed-output derivations don't
depend on impure derivations. */
if (drv->type().isPure() && !drv->type().isFixed()) {
if (settings.isExperimentalFeatureEnabled(Xp::ImpureDerivations) && drv->type().isPure() && !drv->type().isFixed()) {
auto inputDrv = worker.evalStore.readDerivation(i.first);
if (!inputDrv.type().isPure())
throw Error("pure derivation '%s' depends on impure derivation '%s'",
@@ -502,6 +501,14 @@ void DerivationGoal::inputsRealised()
now-known results of dependencies. If so, we become a
stub goal aliasing that resolved derivation goal. */
std::optional attempt = fullDrv.tryResolve(worker.store, inputDrvOutputs);
if (!attempt) {
/* TODO (impure derivations-induced tech debt) (see below):
The above attempt should have found it, but because we manage
inputDrvOutputs statefully, sometimes it gets out of sync with
the real source of truth (store). So we query the store
directly if there's a problem. */
attempt = fullDrv.tryResolve(worker.store);
}
assert(attempt);
Derivation drvResolved { *std::move(attempt) };
@@ -887,6 +894,14 @@ void DerivationGoal::buildDone()
cleanupPostChildKill();
if (buildResult.cpuUser && buildResult.cpuSystem) {
debug("builder for '%s' terminated with status %d, user CPU %.3fs, system CPU %.3fs",
worker.store.printStorePath(drvPath),
status,
((double) buildResult.cpuUser->count()) / 1000000,
((double) buildResult.cpuSystem->count()) / 1000000);
}
bool diskFull = false;
try {
+1 -1
View File
@@ -16,11 +16,11 @@ HookInstance::HookInstance()
buildHookArgs.pop_front();
Strings args;
args.push_back(std::string(baseNameOf(buildHook)));
for (auto & arg : buildHookArgs)
args.push_back(arg);
args.push_back(std::string(baseNameOf(settings.buildHook.get())));
args.push_back(std::to_string(verbosity));
/* Create a pipe to get the output of the child. */
+217 -77
View File
@@ -8,13 +8,13 @@
#include "finally.hh"
#include "util.hh"
#include "archive.hh"
#include "json.hh"
#include "compression.hh"
#include "daemon.hh"
#include "worker-protocol.hh"
#include "topo-sort.hh"
#include "callback.hh"
#include "json-utils.hh"
#include "cgroup.hh"
#include <regex>
#include <queue>
@@ -56,6 +56,7 @@
#include <pwd.h>
#include <grp.h>
#include <iostream>
namespace nix {
@@ -129,26 +130,44 @@ void LocalDerivationGoal::killChild()
if (pid != -1) {
worker.childTerminated(this);
if (buildUser) {
/* If we're using a build user, then there is a tricky
race condition: if we kill the build user before the
child has done its setuid() to the build user uid, then
it won't be killed, and we'll potentially lock up in
pid.wait(). So also send a conventional kill to the
child. */
::kill(-pid, SIGKILL); /* ignore the result */
buildUser->kill();
pid.wait();
} else
pid.kill();
/* If we're using a build user, then there is a tricky race
condition: if we kill the build user before the child has
done its setuid() to the build user uid, then it won't be
killed, and we'll potentially lock up in pid.wait(). So
also send a conventional kill to the child. */
::kill(-pid, SIGKILL); /* ignore the result */
assert(pid == -1);
killSandbox(true);
pid.wait();
}
DerivationGoal::killChild();
}
void LocalDerivationGoal::killSandbox(bool getStats)
{
if (cgroup) {
#if __linux__
auto stats = destroyCgroup(*cgroup);
if (getStats) {
buildResult.cpuUser = stats.cpuUser;
buildResult.cpuSystem = stats.cpuSystem;
}
#else
abort();
#endif
}
else if (buildUser) {
auto uid = buildUser->getUID();
assert(uid != 0);
killUser(uid);
}
}
void LocalDerivationGoal::tryLocalBuild() {
unsigned int curBuilds = worker.getNrLocalBuilds();
if (curBuilds >= settings.maxBuildJobs) {
@@ -158,28 +177,46 @@ void LocalDerivationGoal::tryLocalBuild() {
return;
}
/* If `build-users-group' is not empty, then we have to build as
one of the members of that group. */
if (settings.buildUsersGroup != "" && getuid() == 0) {
#if defined(__linux__) || defined(__APPLE__)
if (!buildUser) buildUser = std::make_unique<UserLock>();
/* Are we doing a chroot build? */
{
auto noChroot = parsedDrv->getBoolAttr("__noChroot");
if (settings.sandboxMode == smEnabled) {
if (noChroot)
throw Error("derivation '%s' has '__noChroot' set, "
"but that's not allowed when 'sandbox' is 'true'", worker.store.printStorePath(drvPath));
#if __APPLE__
if (additionalSandboxProfile != "")
throw Error("derivation '%s' specifies a sandbox profile, "
"but this is only allowed when 'sandbox' is 'relaxed'", worker.store.printStorePath(drvPath));
#endif
useChroot = true;
}
else if (settings.sandboxMode == smDisabled)
useChroot = false;
else if (settings.sandboxMode == smRelaxed)
useChroot = derivationType.isSandboxed() && !noChroot;
}
if (buildUser->findFreeUser()) {
/* Make sure that no other processes are executing under this
uid. */
buildUser->kill();
} else {
auto & localStore = getLocalStore();
if (localStore.storeDir != localStore.realStoreDir.get()) {
#if __linux__
useChroot = true;
#else
throw Error("building using a diverted store is not supported on this platform");
#endif
}
if (useBuildUsers()) {
if (!buildUser)
buildUser = acquireUserLock(parsedDrv->useUidRange() ? 65536 : 1, useChroot);
if (!buildUser) {
if (!actLock)
actLock = std::make_unique<Activity>(*logger, lvlWarn, actBuildWaiting,
fmt("waiting for UID to build '%s'", yellowtxt(worker.store.printStorePath(drvPath))));
worker.waitForAWhile(shared_from_this());
return;
}
#else
/* Don't know how to block the creation of setuid/setgid
binaries on this platform. */
throw Error("build users are not supported on this platform for security reasons");
#endif
}
actLock.reset();
@@ -270,7 +307,7 @@ void LocalDerivationGoal::cleanupPostChildKill()
malicious user from leaving behind a process that keeps files
open and modifies them after they have been chown'ed to
root. */
if (buildUser) buildUser->kill();
killSandbox(true);
/* Terminate the recursive Nix daemon. */
stopDaemon();
@@ -363,6 +400,64 @@ static void linkOrCopy(const Path & from, const Path & to)
void LocalDerivationGoal::startBuilder()
{
if ((buildUser && buildUser->getUIDCount() != 1)
#if __linux__
|| settings.useCgroups
#endif
)
{
#if __linux__
settings.requireExperimentalFeature(Xp::Cgroups);
auto cgroupFS = getCgroupFS();
if (!cgroupFS)
throw Error("cannot determine the cgroups file system");
auto ourCgroups = getCgroups("/proc/self/cgroup");
auto ourCgroup = ourCgroups[""];
if (ourCgroup == "")
throw Error("cannot determine cgroup name from /proc/self/cgroup");
auto ourCgroupPath = canonPath(*cgroupFS + "/" + ourCgroup);
if (!pathExists(ourCgroupPath))
throw Error("expected cgroup directory '%s'", ourCgroupPath);
static std::atomic<unsigned int> counter{0};
cgroup = buildUser
? fmt("%s/nix-build-uid-%d", ourCgroupPath, buildUser->getUID())
: fmt("%s/nix-build-pid-%d-%d", ourCgroupPath, getpid(), counter++);
debug("using cgroup '%s'", *cgroup);
/* When using a build user, record the cgroup we used for that
user so that if we got interrupted previously, we can kill
any left-over cgroup first. */
if (buildUser) {
auto cgroupsDir = settings.nixStateDir + "/cgroups";
createDirs(cgroupsDir);
auto cgroupFile = fmt("%s/%d", cgroupsDir, buildUser->getUID());
if (pathExists(cgroupFile)) {
auto prevCgroup = readFile(cgroupFile);
destroyCgroup(prevCgroup);
}
writeFile(cgroupFile, *cgroup);
}
#else
throw Error("cgroups are not supported on this platform");
#endif
}
/* Make sure that no other processes are executing under the
sandbox uids. This must be done before any chownToBuilder()
calls. */
killSandbox(false);
/* Right platform? */
if (!parsedDrv->canBuildLocally(worker.store))
throw Error("a '%s' with features {%s} is required to build '%s', but I am a '%s' with features {%s}",
@@ -376,35 +471,6 @@ void LocalDerivationGoal::startBuilder()
additionalSandboxProfile = parsedDrv->getStringAttr("__sandboxProfile").value_or("");
#endif
/* Are we doing a chroot build? */
{
auto noChroot = parsedDrv->getBoolAttr("__noChroot");
if (settings.sandboxMode == smEnabled) {
if (noChroot)
throw Error("derivation '%s' has '__noChroot' set, "
"but that's not allowed when 'sandbox' is 'true'", worker.store.printStorePath(drvPath));
#if __APPLE__
if (additionalSandboxProfile != "")
throw Error("derivation '%s' specifies a sandbox profile, "
"but this is only allowed when 'sandbox' is 'relaxed'", worker.store.printStorePath(drvPath));
#endif
useChroot = true;
}
else if (settings.sandboxMode == smDisabled)
useChroot = false;
else if (settings.sandboxMode == smRelaxed)
useChroot = derivationType.isSandboxed() && !noChroot;
}
auto & localStore = getLocalStore();
if (localStore.storeDir != localStore.realStoreDir.get()) {
#if __linux__
useChroot = true;
#else
throw Error("building using a diverted store is not supported on this platform");
#endif
}
/* Create a temporary directory where the build will take
place. */
tmpDir = createTempDir("", "nix-build-" + std::string(drvPath.name()), false, false, 0700);
@@ -580,10 +646,11 @@ void LocalDerivationGoal::startBuilder()
printMsg(lvlChatty, format("setting up chroot environment in '%1%'") % chrootRootDir);
if (mkdir(chrootRootDir.c_str(), 0750) == -1)
// FIXME: make this 0700
if (mkdir(chrootRootDir.c_str(), buildUser && buildUser->getUIDCount() != 1 ? 0755 : 0750) == -1)
throw SysError("cannot create '%1%'", chrootRootDir);
if (buildUser && chown(chrootRootDir.c_str(), 0, buildUser->getGID()) == -1)
if (buildUser && chown(chrootRootDir.c_str(), buildUser->getUIDCount() != 1 ? buildUser->getUID() : 0, buildUser->getGID()) == -1)
throw SysError("cannot change ownership of '%1%'", chrootRootDir);
/* Create a writable /tmp in the chroot. Many builders need
@@ -597,6 +664,11 @@ void LocalDerivationGoal::startBuilder()
nobody account. The latter is kind of a hack to support
Samba-in-QEMU. */
createDirs(chrootRootDir + "/etc");
if (parsedDrv->useUidRange())
chownToBuilder(chrootRootDir + "/etc");
if (parsedDrv->useUidRange() && (!buildUser || buildUser->getUIDCount() < 65536))
throw Error("feature 'uid-range' requires the setting '%s' to be enabled", settings.autoAllocateUids.name);
/* Declare the build user's group so that programs get a consistent
view of the system (e.g., "id -gn"). */
@@ -647,12 +719,28 @@ void LocalDerivationGoal::startBuilder()
dirsInChroot.erase(worker.store.printStorePath(*i.second.second));
}
#elif __APPLE__
/* We don't really have any parent prep work to do (yet?)
All work happens in the child, instead. */
if (cgroup) {
if (mkdir(cgroup->c_str(), 0755) != 0)
throw SysError("creating cgroup '%s'", *cgroup);
chownToBuilder(*cgroup);
chownToBuilder(*cgroup + "/cgroup.procs");
chownToBuilder(*cgroup + "/cgroup.threads");
//chownToBuilder(*cgroup + "/cgroup.subtree_control");
}
#else
throw Error("sandboxing builds is not supported on this platform");
if (parsedDrv->useUidRange())
throw Error("feature 'uid-range' is not supported on this platform");
#if __APPLE__
/* We don't really have any parent prep work to do (yet?)
All work happens in the child, instead. */
#else
throw Error("sandboxing builds is not supported on this platform");
#endif
#endif
} else {
if (parsedDrv->useUidRange())
throw Error("feature 'uid-range' is only supported in sandboxed builds");
}
if (needsHashRewrite() && pathExists(homeDir))
@@ -913,14 +1001,16 @@ void LocalDerivationGoal::startBuilder()
the calling user (if build users are disabled). */
uid_t hostUid = buildUser ? buildUser->getUID() : getuid();
uid_t hostGid = buildUser ? buildUser->getGID() : getgid();
uid_t nrIds = buildUser ? buildUser->getUIDCount() : 1;
writeFile("/proc/" + std::to_string(pid) + "/uid_map",
fmt("%d %d 1", sandboxUid(), hostUid));
fmt("%d %d %d", sandboxUid(), hostUid, nrIds));
writeFile("/proc/" + std::to_string(pid) + "/setgroups", "deny");
if (!buildUser || buildUser->getUIDCount() == 1)
writeFile("/proc/" + std::to_string(pid) + "/setgroups", "deny");
writeFile("/proc/" + std::to_string(pid) + "/gid_map",
fmt("%d %d 1", sandboxGid(), hostGid));
fmt("%d %d %d", sandboxGid(), hostGid, nrIds));
} else {
debug("note: not using a user namespace");
if (!buildUser)
@@ -947,6 +1037,10 @@ void LocalDerivationGoal::startBuilder()
throw SysError("getting sandbox user namespace");
}
/* Move the child into its own cgroup. */
if (cgroup)
writeFile(*cgroup + "/cgroup.procs", fmt("%d", (pid_t) pid));
/* Signal the builder that we've updated its user namespace. */
writeFull(userNamespaceSync.writeSide.get(), "1");
@@ -1552,6 +1646,22 @@ void setupSeccomp()
seccomp_arch_add(ctx, SCMP_ARCH_ARM) != 0)
printError("unable to add ARM seccomp architecture; this may result in spurious build failures if running 32-bit ARM processes");
if (nativeSystem == "mips64-linux" &&
seccomp_arch_add(ctx, SCMP_ARCH_MIPS) != 0)
printError("unable to add mips seccomp architecture");
if (nativeSystem == "mips64-linux" &&
seccomp_arch_add(ctx, SCMP_ARCH_MIPS64N32) != 0)
printError("unable to add mips64-*abin32 seccomp architecture");
if (nativeSystem == "mips64el-linux" &&
seccomp_arch_add(ctx, SCMP_ARCH_MIPSEL) != 0)
printError("unable to add mipsel seccomp architecture");
if (nativeSystem == "mips64el-linux" &&
seccomp_arch_add(ctx, SCMP_ARCH_MIPSEL64N32) != 0)
printError("unable to add mips64el-*abin32 seccomp architecture");
/* Prevent builders from creating setuid/setgid binaries. */
for (int perm : { S_ISUID, S_ISGID }) {
if (seccomp_rule_add(ctx, SCMP_ACT_ERRNO(EPERM), SCMP_SYS(chmod), 1,
@@ -1594,6 +1704,8 @@ void LocalDerivationGoal::runChild()
/* Warning: in the child we should absolutely not make any SQLite
calls! */
bool sendException = true;
try { /* child */
commonChildInit(builderOut);
@@ -1761,6 +1873,13 @@ void LocalDerivationGoal::runChild()
if (mount("none", (chrootRootDir + "/proc").c_str(), "proc", 0, 0) == -1)
throw SysError("mounting /proc");
/* Mount sysfs on /sys. */
if (buildUser && buildUser->getUIDCount() != 1) {
createDirs(chrootRootDir + "/sys");
if (mount("none", (chrootRootDir + "/sys").c_str(), "sysfs", 0, 0) == -1)
throw SysError("mounting /sys");
}
/* Mount a new tmpfs on /dev/shm to ensure that whatever
the builder puts in /dev/shm is cleaned up automatically. */
if (pathExists("/dev/shm") && mount("none", (chrootRootDir + "/dev/shm").c_str(), "tmpfs", 0,
@@ -1790,6 +1909,10 @@ void LocalDerivationGoal::runChild()
}
}
/* Make /etc unwritable */
if (!parsedDrv->useUidRange())
chmod_(chrootRootDir + "/etc", 0555);
/* Unshare this mount namespace. This is necessary because
pivot_root() below changes the root of the mount
namespace. This means that the call to setns() in
@@ -1803,6 +1926,12 @@ void LocalDerivationGoal::runChild()
if (unshare(CLONE_NEWNS) == -1)
throw SysError("unsharing mount namespace");
/* Unshare the cgroup namespace. This means
/proc/self/cgroup will show the child's cgroup as '/'
rather than whatever it is in the parent. */
if (cgroup && unshare(CLONE_NEWCGROUP) == -1)
throw SysError("unsharing cgroup namespace");
/* Do the chroot(). */
if (chdir(chrootRootDir.c_str()) == -1)
throw SysError("cannot change directory to '%1%'", chrootRootDir);
@@ -1888,9 +2017,8 @@ void LocalDerivationGoal::runChild()
if (setUser && buildUser) {
/* Preserve supplementary groups of the build user, to allow
admins to specify groups such as "kvm". */
if (!buildUser->getSupplementaryGIDs().empty() &&
setgroups(buildUser->getSupplementaryGIDs().size(),
buildUser->getSupplementaryGIDs().data()) == -1)
auto gids = buildUser->getSupplementaryGIDs();
if (setgroups(gids.size(), gids.data()) == -1)
throw SysError("cannot set supplementary groups of build user");
if (setgid(buildUser->getGID()) == -1 ||
@@ -2050,6 +2178,8 @@ void LocalDerivationGoal::runChild()
/* Indicate that we managed to set up the build environment. */
writeFull(STDERR_FILENO, std::string("\2\n"));
sendException = false;
/* Execute the program. This should not return. */
if (drv->isBuiltin()) {
try {
@@ -2103,10 +2233,13 @@ void LocalDerivationGoal::runChild()
throw SysError("executing '%1%'", drv->builder);
} catch (Error & e) {
writeFull(STDERR_FILENO, "\1\n");
FdSink sink(STDERR_FILENO);
sink << e;
sink.flush();
if (sendException) {
writeFull(STDERR_FILENO, "\1\n");
FdSink sink(STDERR_FILENO);
sink << e;
sink.flush();
} else
std::cerr << e.msg();
_exit(1);
}
}
@@ -2214,7 +2347,10 @@ DrvOutputs LocalDerivationGoal::registerOutputs()
/* Canonicalise first. This ensures that the path we're
rewriting doesn't contain a hard link to /etc/shadow or
something like that. */
canonicalisePathMetaData(actualPath, buildUser ? buildUser->getUID() : -1, inodesSeen);
canonicalisePathMetaData(
actualPath,
buildUser ? std::optional(buildUser->getUIDRange()) : std::nullopt,
inodesSeen);
debug("scanning for references for output '%s' in temp location '%s'", outputName, actualPath);
@@ -2307,6 +2443,10 @@ DrvOutputs LocalDerivationGoal::registerOutputs()
sink.s = rewriteStrings(sink.s, outputRewrites);
StringSource source(sink.s);
restorePath(actualPath, source);
/* FIXME: set proper permissions in restorePath() so
we don't have to do another traversal. */
canonicalisePathMetaData(actualPath, {}, inodesSeen);
}
};
@@ -2469,7 +2609,7 @@ DrvOutputs LocalDerivationGoal::registerOutputs()
/* FIXME: set proper permissions in restorePath() so
we don't have to do another traversal. */
canonicalisePathMetaData(actualPath, -1, inodesSeen);
canonicalisePathMetaData(actualPath, {}, inodesSeen);
/* Calculate where we'll move the output files. In the checking case we
will leave leave them where they are, for now, rather than move to
+9 -2
View File
@@ -15,6 +15,9 @@ struct LocalDerivationGoal : public DerivationGoal
/* The process ID of the builder. */
Pid pid;
/* The cgroup of the builder, if any. */
std::optional<Path> cgroup;
/* The temporary directory. */
Path tmpDir;
@@ -92,8 +95,8 @@ struct LocalDerivationGoal : public DerivationGoal
result. */
std::map<Path, ValidPathInfo> prevInfos;
uid_t sandboxUid() { return usingUserNamespace ? 1000 : buildUser->getUID(); }
gid_t sandboxGid() { return usingUserNamespace ? 100 : buildUser->getGID(); }
uid_t sandboxUid() { return usingUserNamespace ? (!buildUser || buildUser->getUIDCount() == 1 ? 1000 : 0) : buildUser->getUID(); }
gid_t sandboxGid() { return usingUserNamespace ? (!buildUser || buildUser->getUIDCount() == 1 ? 100 : 0) : buildUser->getGID(); }
const static Path homeDir;
@@ -197,6 +200,10 @@ struct LocalDerivationGoal : public DerivationGoal
/* Forcibly kill the child process, if any. */
void killChild() override;
/* Kill any processes running under the build user UID or in the
cgroup of the build. */
void killSandbox(bool getStats);
/* Create alternative path calculated from but distinct from the
input, so we can avoid overwriting outputs (or other store paths)
that already exist. */
+1 -1
View File
@@ -95,7 +95,7 @@ static void createLinks(State & state, const Path & srcDir, const Path & dstDir,
throw Error(
"files '%1%' and '%2%' have the same priority %3%; "
"use 'nix-env --set-flag priority NUMBER INSTALLED_PKGNAME' "
"or type 'nix profile install --help' if using 'nix profile' to find out how"
"or type 'nix profile install --help' if using 'nix profile' to find out how "
"to change the priority of one of the conflicting packages"
" (0 being the highest priority)",
srcFile, readLink(dstFile), priority);
+2
View File
@@ -239,6 +239,8 @@ struct ClientSettings
else if (trusted
|| name == settings.buildTimeout.name
|| name == settings.buildRepeat.name
|| name == settings.maxSilentTime.name
|| name == settings.pollInterval.name
|| name == "connect-timeout"
|| (name == "builders" && value == ""))
settings.set(name, value);
+9 -23
View File
@@ -20,11 +20,12 @@ nlohmann::json DerivedPath::Built::toJSON(ref<Store> store) const {
// Fallback for the input-addressed derivation case: We expect to always be
// able to print the output paths, so let’s do it
const auto knownOutputs = store->queryPartialDerivationOutputMap(drvPath);
for (const auto& output : outputs) {
for (const auto & output : outputs) {
auto knownOutput = get(knownOutputs, output);
res["outputs"][output] = (knownOutput && *knownOutput)
? store->printStorePath(**knownOutput)
: nullptr;
if (knownOutput && *knownOutput)
res["outputs"][output] = store->printStorePath(**knownOutput);
else
res["outputs"][output] = nullptr;
}
return res;
}
@@ -53,28 +54,13 @@ StorePathSet BuiltPath::outPaths() const
);
}
template<typename T>
nlohmann::json stuffToJSON(const std::vector<T> & ts, ref<Store> store) {
auto res = nlohmann::json::array();
for (const T & t : ts) {
std::visit([&res, store](const auto & t) {
res.push_back(t.toJSON(store));
}, t.raw());
}
return res;
}
nlohmann::json derivedPathsWithHintsToJSON(const BuiltPaths & buildables, ref<Store> store)
{ return stuffToJSON<BuiltPath>(buildables, store); }
nlohmann::json derivedPathsToJSON(const DerivedPaths & paths, ref<Store> store)
{ return stuffToJSON<DerivedPath>(paths, store); }
std::string DerivedPath::Opaque::to_string(const Store & store) const {
std::string DerivedPath::Opaque::to_string(const Store & store) const
{
return store.printStorePath(path);
}
std::string DerivedPath::Built::to_string(const Store & store) const {
std::string DerivedPath::Built::to_string(const Store & store) const
{
return store.printStorePath(drvPath)
+ "!"
+ (outputs.empty() ? std::string { "*" } : concatStringsSep(",", outputs));
-3
View File
@@ -125,7 +125,4 @@ struct BuiltPath : _BuiltPathRaw {
typedef std::vector<DerivedPath> DerivedPaths;
typedef std::vector<BuiltPath> BuiltPaths;
nlohmann::json derivedPathsWithHintsToJSON(const BuiltPaths & buildables, ref<Store> store);
nlohmann::json derivedPathsToJSON(const DerivedPaths & , ref<Store> store);
}
-1
View File
@@ -322,7 +322,6 @@ struct curlFileTransfer : public FileTransfer
}
if (request.verifyTLS) {
debug("verify TLS: Nix CA file = '%s'", settings.caFile);
if (settings.caFile != "")
curl_easy_setopt(req, CURLOPT_CAINFO, settings.caFile.c_str());
} else {
+62 -36
View File
@@ -77,60 +77,73 @@ Path LocalFSStore::addPermRoot(const StorePath & storePath, const Path & _gcRoot
}
void LocalStore::addTempRoot(const StorePath & path)
void LocalStore::createTempRootsFile()
{
auto state(_state.lock());
auto fdTempRoots(_fdTempRoots.lock());
/* Create the temporary roots file for this process. */
if (!state->fdTempRoots) {
if (*fdTempRoots) return;
while (1) {
if (pathExists(fnTempRoots))
/* It *must* be stale, since there can be no two
processes with the same pid. */
unlink(fnTempRoots.c_str());
while (1) {
if (pathExists(fnTempRoots))
/* It *must* be stale, since there can be no two
processes with the same pid. */
unlink(fnTempRoots.c_str());
state->fdTempRoots = openLockFile(fnTempRoots, true);
*fdTempRoots = openLockFile(fnTempRoots, true);
debug("acquiring write lock on '%s'", fnTempRoots);
lockFile(state->fdTempRoots.get(), ltWrite, true);
debug("acquiring write lock on '%s'", fnTempRoots);
lockFile(fdTempRoots->get(), ltWrite, true);
/* Check whether the garbage collector didn't get in our
way. */
struct stat st;
if (fstat(state->fdTempRoots.get(), &st) == -1)
throw SysError("statting '%1%'", fnTempRoots);
if (st.st_size == 0) break;
/* Check whether the garbage collector didn't get in our
way. */
struct stat st;
if (fstat(fdTempRoots->get(), &st) == -1)
throw SysError("statting '%1%'", fnTempRoots);
if (st.st_size == 0) break;
/* The garbage collector deleted this file before we could
get a lock. (It won't delete the file after we get a
lock.) Try again. */
}
/* The garbage collector deleted this file before we could get
a lock. (It won't delete the file after we get a lock.)
Try again. */
}
}
void LocalStore::addTempRoot(const StorePath & path)
{
createTempRootsFile();
/* Open/create the global GC lock file. */
{
auto fdGCLock(_fdGCLock.lock());
if (!*fdGCLock)
*fdGCLock = openGCLock();
}
if (!state->fdGCLock)
state->fdGCLock = openGCLock();
restart:
FdLock gcLock(state->fdGCLock.get(), ltRead, false, "");
/* Try to acquire a shared global GC lock (non-blocking). This
only succeeds if the garbage collector is not currently
running. */
FdLock gcLock(_fdGCLock.lock()->get(), ltRead, false, "");
if (!gcLock.acquired) {
/* We couldn't get a shared global GC lock, so the garbage
collector is running. So we have to connect to the garbage
collector and inform it about our root. */
if (!state->fdRootsSocket) {
auto fdRootsSocket(_fdRootsSocket.lock());
if (!*fdRootsSocket) {
auto socketPath = stateDir.get() + gcSocketPath;
debug("connecting to '%s'", socketPath);
state->fdRootsSocket = createUnixDomainSocket();
*fdRootsSocket = createUnixDomainSocket();
try {
nix::connect(state->fdRootsSocket.get(), socketPath);
nix::connect(fdRootsSocket->get(), socketPath);
} catch (SysError & e) {
/* The garbage collector may have exited, so we need to
restart. */
if (e.errNo == ECONNREFUSED) {
debug("GC socket connection refused");
state->fdRootsSocket.close();
fdRootsSocket->close();
goto restart;
}
throw;
@@ -139,30 +152,31 @@ void LocalStore::addTempRoot(const StorePath & path)
try {
debug("sending GC root '%s'", printStorePath(path));
writeFull(state->fdRootsSocket.get(), printStorePath(path) + "\n", false);
writeFull(fdRootsSocket->get(), printStorePath(path) + "\n", false);
char c;
readFull(state->fdRootsSocket.get(), &c, 1);
readFull(fdRootsSocket->get(), &c, 1);
assert(c == '1');
debug("got ack for GC root '%s'", printStorePath(path));
} catch (SysError & e) {
/* The garbage collector may have exited, so we need to
restart. */
if (e.errNo == EPIPE) {
if (e.errNo == EPIPE || e.errNo == ECONNRESET) {
debug("GC socket disconnected");
state->fdRootsSocket.close();
fdRootsSocket->close();
goto restart;
}
throw;
} catch (EndOfFile & e) {
debug("GC socket disconnected");
state->fdRootsSocket.close();
fdRootsSocket->close();
goto restart;
}
}
/* Append the store path to the temporary roots file. */
/* Record the store path in the temporary roots file so it will be
seen by a future run of the garbage collector. */
auto s = printStorePath(path) + '\0';
writeFull(state->fdTempRoots.get(), s);
writeFull(_fdTempRoots.lock()->get(), s);
}
@@ -506,6 +520,7 @@ void LocalStore::collectGarbage(const GCOptions & options, GCResults & results)
Finally cleanup([&]() {
debug("GC roots server shutting down");
fdServer.close();
while (true) {
auto item = remove_begin(*connections.lock());
if (!item) break;
@@ -619,6 +634,17 @@ void LocalStore::collectGarbage(const GCOptions & options, GCResults & results)
Path path = storeDir + "/" + std::string(baseName);
Path realPath = realStoreDir + "/" + std::string(baseName);
/* There may be temp directories in the store that are still in use
by another process. We need to be sure that we can acquire an
exclusive lock before deleting them. */
if (baseName.find("tmp-", 0) == 0) {
AutoCloseFD tmpDirFd = open(realPath.c_str(), O_RDONLY | O_DIRECTORY);
if (tmpDirFd.get() == -1 || !lockFile(tmpDirFd.get(), ltWrite, false)) {
debug("skipping locked tempdir '%s'", realPath);
return;
}
}
printInfo("deleting '%1%'", path);
results.paths.insert(path);
+7 -7
View File
@@ -130,6 +130,10 @@ StringSet Settings::getDefaultSystemFeatures()
actually require anything special on the machines. */
StringSet features{"nixos-test", "benchmark", "big-parallel"};
#if __linux__
features.insert("uid-range");
#endif
#if __linux__
if (access("/dev/kvm", R_OK | W_OK) == 0)
features.insert("kvm");
@@ -154,13 +158,9 @@ StringSet Settings::getDefaultExtraPlatforms()
// machines. Note that we can’t force processes from executing
// x86_64 in aarch64 environments or vice versa since they can
// always exec with their own binary preferences.
if (pathExists("/Library/Apple/System/Library/LaunchDaemons/com.apple.oahd.plist") ||
pathExists("/System/Library/LaunchDaemons/com.apple.oahd.plist")) {
if (std::string{SYSTEM} == "x86_64-darwin")
extraPlatforms.insert("aarch64-darwin");
else if (std::string{SYSTEM} == "aarch64-darwin")
extraPlatforms.insert("x86_64-darwin");
}
if (std::string{SYSTEM} == "aarch64-darwin" &&
runProgram(RunOptions {.program = "arch", .args = {"-arch", "x86_64", "/usr/bin/true"}, .mergeStderrToStdout = true}).first == 0)
extraPlatforms.insert("x86_64-darwin");
#endif
return extraPlatforms;
+86 -3
View File
@@ -46,6 +46,14 @@ struct PluginFilesSetting : public BaseSetting<Paths>
void set(const std::string & str, bool append = false) override;
};
const uint32_t maxIdsPerBuild =
#if __linux__
1 << 16
#else
1
#endif
;
class Settings : public Config {
unsigned int getDefaultCores();
@@ -273,8 +281,69 @@ public:
`NIX_REMOTE` is empty, the uid under which the Nix daemon runs if
`NIX_REMOTE` is `daemon`). Obviously, this should not be used in
multi-user settings with untrusted users.
Defaults to `nixbld` when running as root, *empty* otherwise.
)",
{}, false};
Setting<bool> autoAllocateUids{this, false, "auto-allocate-uids",
R"(
Whether to select UIDs for builds automatically, instead of using the
users in `build-users-group`.
UIDs are allocated starting at 872415232 (0x34000000) on Linux and 56930 on macOS.
> **Warning**
> This is an experimental feature.
To enable it, add the following to [`nix.conf`](#):
```
extra-experimental-features = auto-allocate-uids
auto-allocate-uids = true
```
)"};
Setting<uint32_t> startId{this,
#if __linux__
0x34000000,
#else
56930,
#endif
"start-id",
"The first UID and GID to use for dynamic ID allocation."};
Setting<uint32_t> uidCount{this,
#if __linux__
maxIdsPerBuild * 128,
#else
128,
#endif
"id-count",
"The number of UIDs/GIDs to use for dynamic ID allocation."};
#if __linux__
Setting<bool> useCgroups{
this, false, "use-cgroups",
R"(
Whether to execute builds inside cgroups.
This is only supported on Linux.
Cgroups are required and enabled automatically for derivations
that require the `uid-range` system feature.
> **Warning**
> This is an experimental feature.
To enable it, add the following to [`nix.conf`](#):
```
extra-experimental-features = cgroups
use-cgroups = true
```
)"};
#endif
Setting<bool> impersonateLinux26{this, false, "impersonate-linux-26",
"Whether to impersonate a Linux 2.6 machine on newer kernels.",
{"build-impersonate-linux-26"}};
@@ -560,9 +629,15 @@ public:
R"(
If set to `true` (the default), any non-content-addressed path added
or copied to the Nix store (e.g. when substituting from a binary
cache) must have a valid signature, that is, be signed using one of
the keys listed in `trusted-public-keys` or `secret-key-files`. Set
to `false` to disable signature checking.
cache) must have a signature by a trusted key. A trusted key is one
listed in `trusted-public-keys`, or a public key counterpart to a
private key stored in a file listed in `secret-key-files`.
Set to `false` to disable signature checking and trust all
non-content-addressed paths unconditionally.
(Content-addressed paths are inherently trustworthy and thus
unaffected by this configuration option.)
)"};
Setting<StringSet> extraPlatforms{
@@ -613,6 +688,14 @@ public:
are tried based on their Priority value, which each substituter can set
independently. Lower value means higher priority.
The default is `https://cache.nixos.org`, with a Priority of 40.
Nix will copy a store path from a remote store only if one
of the following is true:
- the store object is signed by one of the [`trusted-public-keys`](#conf-trusted-public-keys)
- the substituter is in the [`trusted-substituters`](#conf-trusted-substituters) list
- the [`require-sigs`](#conf-require-sigs) option has been set to `false`
- the store object is [output-addressed](glossary.md#gloss-output-addressed-store-object)
)",
{"binary-caches"}};
+43 -28
View File
@@ -158,7 +158,7 @@ void migrateCASchema(SQLite& db, Path schemaPath, AutoCloseFD& lockFd)
txn.commit();
}
writeFile(schemaPath, fmt("%d", nixCASchemaVersion));
writeFile(schemaPath, fmt("%d", nixCASchemaVersion), 0666, true);
lockFile(lockFd.get(), ltRead, true);
}
}
@@ -281,7 +281,7 @@ LocalStore::LocalStore(const Params & params)
else if (curSchema == 0) { /* new store */
curSchema = nixSchemaVersion;
openDB(*state, true);
writeFile(schemaPath, (format("%1%") % nixSchemaVersion).str());
writeFile(schemaPath, (format("%1%") % nixSchemaVersion).str(), 0666, true);
}
else if (curSchema < nixSchemaVersion) {
@@ -329,7 +329,7 @@ LocalStore::LocalStore(const Params & params)
txn.commit();
}
writeFile(schemaPath, (format("%1%") % nixSchemaVersion).str());
writeFile(schemaPath, (format("%1%") % nixSchemaVersion).str(), 0666, true);
lockFile(globalLock.get(), ltRead, true);
}
@@ -439,9 +439,9 @@ LocalStore::~LocalStore()
}
try {
auto state(_state.lock());
if (state->fdTempRoots) {
state->fdTempRoots = -1;
auto fdTempRoots(_fdTempRoots.lock());
if (*fdTempRoots) {
*fdTempRoots = -1;
unlink(fnTempRoots.c_str());
}
} catch (...) {
@@ -583,7 +583,10 @@ void canonicaliseTimestampAndPermissions(const Path & path)
}
static void canonicalisePathMetaData_(const Path & path, uid_t fromUid, InodesSeen & inodesSeen)
static void canonicalisePathMetaData_(
const Path & path,
std::optional<std::pair<uid_t, uid_t>> uidRange,
InodesSeen & inodesSeen)
{
checkInterrupt();
@@ -630,7 +633,7 @@ static void canonicalisePathMetaData_(const Path & path, uid_t fromUid, InodesSe
However, ignore files that we chown'ed ourselves previously to
ensure that we don't fail on hard links within the same build
(i.e. "touch $out/foo; ln $out/foo $out/bar"). */
if (fromUid != (uid_t) -1 && st.st_uid != fromUid) {
if (uidRange && (st.st_uid < uidRange->first || st.st_uid > uidRange->second)) {
if (S_ISDIR(st.st_mode) || !inodesSeen.count(Inode(st.st_dev, st.st_ino)))
throw BuildError("invalid ownership on file '%1%'", path);
mode_t mode = st.st_mode & ~S_IFMT;
@@ -663,14 +666,17 @@ static void canonicalisePathMetaData_(const Path & path, uid_t fromUid, InodesSe
if (S_ISDIR(st.st_mode)) {
DirEntries entries = readDirectory(path);
for (auto & i : entries)
canonicalisePathMetaData_(path + "/" + i.name, fromUid, inodesSeen);
canonicalisePathMetaData_(path + "/" + i.name, uidRange, inodesSeen);
}
}
void canonicalisePathMetaData(const Path & path, uid_t fromUid, InodesSeen & inodesSeen)
void canonicalisePathMetaData(
const Path & path,
std::optional<std::pair<uid_t, uid_t>> uidRange,
InodesSeen & inodesSeen)
{
canonicalisePathMetaData_(path, fromUid, inodesSeen);
canonicalisePathMetaData_(path, uidRange, inodesSeen);
/* On platforms that don't have lchown(), the top-level path can't
be a symlink, since we can't change its ownership. */
@@ -683,10 +689,11 @@ void canonicalisePathMetaData(const Path & path, uid_t fromUid, InodesSeen & ino
}
void canonicalisePathMetaData(const Path & path, uid_t fromUid)
void canonicalisePathMetaData(const Path & path,
std::optional<std::pair<uid_t, uid_t>> uidRange)
{
InodesSeen inodesSeen;
canonicalisePathMetaData(path, fromUid, inodesSeen);
canonicalisePathMetaData(path, uidRange, inodesSeen);
}
@@ -751,7 +758,7 @@ void LocalStore::registerDrvOutput(const Realisation & info, CheckSigsFlag check
if (checkSigs == NoCheckSigs || !realisationIsUntrusted(info))
registerDrvOutput(info);
else
throw Error("cannot register realisation '%s' because it lacks a valid signature", info.outPath.to_string());
throw Error("cannot register realisation '%s' because it lacks a signature by a trusted key", info.outPath.to_string());
}
void LocalStore::registerDrvOutput(const Realisation & info)
@@ -1266,7 +1273,7 @@ void LocalStore::addToStore(const ValidPathInfo & info, Source & source,
RepairFlag repair, CheckSigsFlag checkSigs)
{
if (checkSigs && pathInfoIsUntrusted(info))
throw Error("cannot add path '%s' because it lacks a valid signature", printStorePath(info.path));
throw Error("cannot add path '%s' because it lacks a signature by a trusted key", printStorePath(info.path));
addTempRoot(info.path);
@@ -1331,7 +1338,7 @@ void LocalStore::addToStore(const ValidPathInfo & info, Source & source,
autoGC();
canonicalisePathMetaData(realPath, -1);
canonicalisePathMetaData(realPath, {});
optimisePath(realPath, repair); // FIXME: combine with hashPath()
@@ -1382,13 +1389,15 @@ StorePath LocalStore::addToStoreFromDump(Source & source0, std::string_view name
std::unique_ptr<AutoDelete> delTempDir;
Path tempPath;
Path tempDir;
AutoCloseFD tempDirFd;
if (!inMemory) {
/* Drain what we pulled so far, and then keep on pulling */
StringSource dumpSource { dump };
ChainSource bothSource { dumpSource, source };
auto tempDir = createTempDir(realStoreDir, "add");
std::tie(tempDir, tempDirFd) = createTempDirInStore();
delTempDir = std::make_unique<AutoDelete>(tempDir);
tempPath = tempDir + "/x";
@@ -1442,7 +1451,7 @@ StorePath LocalStore::addToStoreFromDump(Source & source0, std::string_view name
narHash = narSink.finish();
}
canonicalisePathMetaData(realPath, -1); // FIXME: merge into restorePath
canonicalisePathMetaData(realPath, {}); // FIXME: merge into restorePath
optimisePath(realPath, repair);
@@ -1484,7 +1493,7 @@ StorePath LocalStore::addTextToStore(
writeFile(realPath, s);
canonicalisePathMetaData(realPath, -1);
canonicalisePathMetaData(realPath, {});
StringSink sink;
dumpString(s, sink);
@@ -1507,18 +1516,24 @@ StorePath LocalStore::addTextToStore(
/* Create a temporary directory in the store that won't be
garbage-collected. */
Path LocalStore::createTempDirInStore()
garbage-collected until the returned FD is closed. */
std::pair<Path, AutoCloseFD> LocalStore::createTempDirInStore()
{
Path tmpDir;
Path tmpDirFn;
AutoCloseFD tmpDirFd;
bool lockedByUs = false;
do {
/* There is a slight possibility that `tmpDir' gets deleted by
the GC between createTempDir() and addTempRoot(), so repeat
until `tmpDir' exists. */
tmpDir = createTempDir(realStoreDir);
addTempRoot(parseStorePath(tmpDir));
} while (!pathExists(tmpDir));
return tmpDir;
the GC between createTempDir() and when we acquire a lock on it.
We'll repeat until 'tmpDir' exists and we've locked it. */
tmpDirFn = createTempDir(realStoreDir, "tmp");
tmpDirFd = open(tmpDirFn.c_str(), O_RDONLY | O_DIRECTORY);
if (tmpDirFd.get() < 0) {
continue;
}
lockedByUs = lockFile(tmpDirFd.get(), ltWrite, true);
} while (!pathExists(tmpDirFn) || !lockedByUs);
return {tmpDirFn, std::move(tmpDirFd)};
}
+28 -13
View File
@@ -59,15 +59,6 @@ private:
struct Stmts;
std::unique_ptr<Stmts> stmts;
/* The global GC lock */
AutoCloseFD fdGCLock;
/* The file to which we write our temporary roots. */
AutoCloseFD fdTempRoots;
/* Connection to the garbage collector. */
AutoCloseFD fdRootsSocket;
/* The last time we checked whether to do an auto-GC, or an
auto-GC finished. */
std::chrono::time_point<std::chrono::steady_clock> lastGCCheck;
@@ -156,6 +147,21 @@ public:
void addTempRoot(const StorePath & path) override;
private:
void createTempRootsFile();
/* The file to which we write our temporary roots. */
Sync<AutoCloseFD> _fdTempRoots;
/* The global GC lock. */
Sync<AutoCloseFD> _fdGCLock;
/* Connection to the garbage collector. */
Sync<AutoCloseFD> _fdRootsSocket;
public:
void addIndirectRoot(const Path & path) override;
private:
@@ -256,7 +262,7 @@ private:
void findRuntimeRoots(Roots & roots, bool censor);
Path createTempDirInStore();
std::pair<Path, AutoCloseFD> createTempDirInStore();
void checkDerivationOutputs(const StorePath & drvPath, const Derivation & drv);
@@ -310,9 +316,18 @@ typedef std::set<Inode> InodesSeen;
- the permissions are set of 444 or 555 (i.e., read-only with or
without execute permission; setuid bits etc. are cleared)
- the owner and group are set to the Nix user and group, if we're
running as root. */
void canonicalisePathMetaData(const Path & path, uid_t fromUid, InodesSeen & inodesSeen);
void canonicalisePathMetaData(const Path & path, uid_t fromUid);
running as root.
If uidRange is not empty, this function will throw an error if it
encounters files owned by a user outside of the closed interval
[uidRange->first, uidRange->second].
*/
void canonicalisePathMetaData(
const Path & path,
std::optional<std::pair<uid_t, uid_t>> uidRange,
InodesSeen & inodesSeen);
void canonicalisePathMetaData(
const Path & path,
std::optional<std::pair<uid_t, uid_t>> uidRange);
void canonicaliseTimestampAndPermissions(const Path & path);
+1 -1
View File
@@ -20,7 +20,7 @@ endif
$(foreach file,$(libstore_FILES),$(eval $(call install-data-in,$(d)/$(file),$(datadir)/nix/sandbox)))
ifeq ($(ENABLE_S3), 1)
libstore_LDFLAGS += -laws-cpp-sdk-transfer -laws-cpp-sdk-s3 -laws-cpp-sdk-core
libstore_LDFLAGS += -laws-cpp-sdk-transfer -laws-cpp-sdk-s3 -laws-cpp-sdk-core -laws-crt-cpp
endif
ifdef HOST_SOLARIS
+181 -85
View File
@@ -2,105 +2,201 @@
#include "globals.hh"
#include "pathlocks.hh"
#include <grp.h>
#include <pwd.h>
#include <fcntl.h>
#include <unistd.h>
#include <grp.h>
namespace nix {
UserLock::UserLock()
struct SimpleUserLock : UserLock
{
assert(settings.buildUsersGroup != "");
createDirs(settings.nixStateDir + "/userpool");
}
AutoCloseFD fdUserLock;
uid_t uid;
gid_t gid;
std::vector<gid_t> supplementaryGIDs;
bool UserLock::findFreeUser() {
if (enabled()) return true;
uid_t getUID() override { assert(uid); return uid; }
uid_t getUIDCount() override { return 1; }
gid_t getGID() override { assert(gid); return gid; }
/* Get the members of the build-users-group. */
struct group * gr = getgrnam(settings.buildUsersGroup.get().c_str());
if (!gr)
throw Error("the group '%1%' specified in 'build-users-group' does not exist",
settings.buildUsersGroup);
gid = gr->gr_gid;
std::vector<gid_t> getSupplementaryGIDs() override { return supplementaryGIDs; }
/* Copy the result of getgrnam. */
Strings users;
for (char * * p = gr->gr_mem; *p; ++p) {
debug("found build user '%1%'", *p);
users.push_back(*p);
}
static std::unique_ptr<UserLock> acquire()
{
assert(settings.buildUsersGroup != "");
createDirs(settings.nixStateDir + "/userpool");
if (users.empty())
throw Error("the build users group '%1%' has no members",
settings.buildUsersGroup);
/* Get the members of the build-users-group. */
struct group * gr = getgrnam(settings.buildUsersGroup.get().c_str());
if (!gr)
throw Error("the group '%s' specified in 'build-users-group' does not exist", settings.buildUsersGroup);
/* Find a user account that isn't currently in use for another
build. */
for (auto & i : users) {
debug("trying user '%1%'", i);
struct passwd * pw = getpwnam(i.c_str());
if (!pw)
throw Error("the user '%1%' in the group '%2%' does not exist",
i, settings.buildUsersGroup);
fnUserLock = (format("%1%/userpool/%2%") % settings.nixStateDir % pw->pw_uid).str();
AutoCloseFD fd = open(fnUserLock.c_str(), O_RDWR | O_CREAT | O_CLOEXEC, 0600);
if (!fd)
throw SysError("opening user lock '%1%'", fnUserLock);
if (lockFile(fd.get(), ltWrite, false)) {
fdUserLock = std::move(fd);
user = i;
uid = pw->pw_uid;
/* Sanity check... */
if (uid == getuid() || uid == geteuid())
throw Error("the Nix user should not be a member of '%1%'",
settings.buildUsersGroup);
#if __linux__
/* Get the list of supplementary groups of this build user. This
is usually either empty or contains a group such as "kvm". */
int ngroups = 32; // arbitrary initial guess
supplementaryGIDs.resize(ngroups);
int err = getgrouplist(pw->pw_name, pw->pw_gid, supplementaryGIDs.data(),
&ngroups);
// Our initial size of 32 wasn't sufficient, the correct size has
// been stored in ngroups, so we try again.
if (err == -1) {
supplementaryGIDs.resize(ngroups);
err = getgrouplist(pw->pw_name, pw->pw_gid, supplementaryGIDs.data(),
&ngroups);
}
// If it failed once more, then something must be broken.
if (err == -1)
throw Error("failed to get list of supplementary groups for '%1%'",
pw->pw_name);
// Finally, trim back the GID list to its real size
supplementaryGIDs.resize(ngroups);
#endif
isEnabled = true;
return true;
/* Copy the result of getgrnam. */
Strings users;
for (char * * p = gr->gr_mem; *p; ++p) {
debug("found build user '%s'", *p);
users.push_back(*p);
}
if (users.empty())
throw Error("the build users group '%s' has no members", settings.buildUsersGroup);
/* Find a user account that isn't currently in use for another
build. */
for (auto & i : users) {
debug("trying user '%s'", i);
struct passwd * pw = getpwnam(i.c_str());
if (!pw)
throw Error("the user '%s' in the group '%s' does not exist", i, settings.buildUsersGroup);
auto fnUserLock = fmt("%s/userpool/%s", settings.nixStateDir,pw->pw_uid);
AutoCloseFD fd = open(fnUserLock.c_str(), O_RDWR | O_CREAT | O_CLOEXEC, 0600);
if (!fd)
throw SysError("opening user lock '%s'", fnUserLock);
if (lockFile(fd.get(), ltWrite, false)) {
auto lock = std::make_unique<SimpleUserLock>();
lock->fdUserLock = std::move(fd);
lock->uid = pw->pw_uid;
lock->gid = gr->gr_gid;
/* Sanity check... */
if (lock->uid == getuid() || lock->uid == geteuid())
throw Error("the Nix user should not be a member of '%s'", settings.buildUsersGroup);
#if __linux__
/* Get the list of supplementary groups of this build
user. This is usually either empty or contains a
group such as "kvm". */
int ngroups = 32; // arbitrary initial guess
std::vector<gid_t> gids;
gids.resize(ngroups);
int err = getgrouplist(
pw->pw_name, pw->pw_gid,
gids.data(),
&ngroups);
/* Our initial size of 32 wasn't sufficient, the
correct size has been stored in ngroups, so we try
again. */
if (err == -1) {
gids.resize(ngroups);
err = getgrouplist(
pw->pw_name, pw->pw_gid,
gids.data(),
&ngroups);
}
// If it failed once more, then something must be broken.
if (err == -1)
throw Error("failed to get list of supplementary groups for '%s'", pw->pw_name);
// Finally, trim back the GID list to its real size.
for (auto i = 0; i < ngroups; i++)
if (gids[i] != lock->gid)
lock->supplementaryGIDs.push_back(gids[i]);
#endif
return lock;
}
}
return nullptr;
}
};
return false;
}
void UserLock::kill()
struct AutoUserLock : UserLock
{
killUser(uid);
AutoCloseFD fdUserLock;
uid_t firstUid = 0;
gid_t firstGid = 0;
uid_t nrIds = 1;
uid_t getUID() override { assert(firstUid); return firstUid; }
gid_t getUIDCount() override { return nrIds; }
gid_t getGID() override { assert(firstGid); return firstGid; }
std::vector<gid_t> getSupplementaryGIDs() override { return {}; }
static std::unique_ptr<UserLock> acquire(uid_t nrIds, bool useUserNamespace)
{
#if !defined(__linux__)
useUserNamespace = false;
#endif
settings.requireExperimentalFeature(Xp::AutoAllocateUids);
assert(settings.startId > 0);
assert(settings.uidCount % maxIdsPerBuild == 0);
assert((uint64_t) settings.startId + (uint64_t) settings.uidCount <= std::numeric_limits<uid_t>::max());
assert(nrIds <= maxIdsPerBuild);
createDirs(settings.nixStateDir + "/userpool2");
size_t nrSlots = settings.uidCount / maxIdsPerBuild;
for (size_t i = 0; i < nrSlots; i++) {
debug("trying user slot '%d'", i);
createDirs(settings.nixStateDir + "/userpool2");
auto fnUserLock = fmt("%s/userpool2/slot-%d", settings.nixStateDir, i);
AutoCloseFD fd = open(fnUserLock.c_str(), O_RDWR | O_CREAT | O_CLOEXEC, 0600);
if (!fd)
throw SysError("opening user lock '%s'", fnUserLock);
if (lockFile(fd.get(), ltWrite, false)) {
auto firstUid = settings.startId + i * maxIdsPerBuild;
auto pw = getpwuid(firstUid);
if (pw)
throw Error("auto-allocated UID %d clashes with existing user account '%s'", firstUid, pw->pw_name);
auto lock = std::make_unique<AutoUserLock>();
lock->fdUserLock = std::move(fd);
lock->firstUid = firstUid;
if (useUserNamespace)
lock->firstGid = firstUid;
else {
struct group * gr = getgrnam(settings.buildUsersGroup.get().c_str());
if (!gr)
throw Error("the group '%s' specified in 'build-users-group' does not exist", settings.buildUsersGroup);
lock->firstGid = gr->gr_gid;
}
lock->nrIds = nrIds;
return lock;
}
}
return nullptr;
}
};
std::unique_ptr<UserLock> acquireUserLock(uid_t nrIds, bool useUserNamespace)
{
if (settings.autoAllocateUids)
return AutoUserLock::acquire(nrIds, useUserNamespace);
else
return SimpleUserLock::acquire();
}
bool useBuildUsers()
{
#if __linux__
static bool b = (settings.buildUsersGroup != "" || settings.autoAllocateUids) && getuid() == 0;
return b;
#elif __APPLE__
static bool b = settings.buildUsersGroup != "" && getuid() == 0;
return b;
#else
return false;
#endif
}
}
+23 -22
View File
@@ -1,37 +1,38 @@
#pragma once
#include "sync.hh"
#include "types.hh"
#include "util.hh"
#include <optional>
#include <sys/types.h>
namespace nix {
class UserLock
struct UserLock
{
private:
Path fnUserLock;
AutoCloseFD fdUserLock;
virtual ~UserLock() { }
bool isEnabled = false;
std::string user;
uid_t uid = 0;
gid_t gid = 0;
std::vector<gid_t> supplementaryGIDs;
/* Get the first and last UID. */
std::pair<uid_t, uid_t> getUIDRange()
{
auto first = getUID();
return {first, first + getUIDCount() - 1};
}
public:
UserLock();
/* Get the first UID. */
virtual uid_t getUID() = 0;
void kill();
virtual uid_t getUIDCount() = 0;
std::string getUser() { return user; }
uid_t getUID() { assert(uid); return uid; }
uid_t getGID() { assert(gid); return gid; }
std::vector<gid_t> getSupplementaryGIDs() { return supplementaryGIDs; }
bool findFreeUser();
bool enabled() { return isEnabled; }
virtual gid_t getGID() = 0;
virtual std::vector<gid_t> getSupplementaryGIDs() = 0;
};
/* Acquire a user lock for a UID range of size `nrIds`. Note that this
may return nullptr if no user is available. */
std::unique_ptr<UserLock> acquireUserLock(uid_t nrIds, bool useUserNamespace);
bool useBuildUsers();
}
+18 -15
View File
@@ -1,6 +1,5 @@
#include "nar-accessor.hh"
#include "archive.hh"
#include "json.hh"
#include <map>
#include <stack>
@@ -75,6 +74,9 @@ struct NarAccessor : public FSAccessor
createMember(path, {FSAccessor::Type::tRegular, false, 0, 0});
}
void closeRegularFile() override
{ }
void isExecutable() override
{
parents.top()->isExecutable = true;
@@ -240,42 +242,43 @@ ref<FSAccessor> makeLazyNarAccessor(const std::string & listing,
return make_ref<NarAccessor>(listing, getNarBytes);
}
void listNar(JSONPlaceholder & res, ref<FSAccessor> accessor,
const Path & path, bool recurse)
using nlohmann::json;
json listNar(ref<FSAccessor> accessor, const Path & path, bool recurse)
{
auto st = accessor->stat(path);
auto obj = res.object();
json obj = json::object();
switch (st.type) {
case FSAccessor::Type::tRegular:
obj.attr("type", "regular");
obj.attr("size", st.fileSize);
obj["type"] = "regular";
obj["size"] = st.fileSize;
if (st.isExecutable)
obj.attr("executable", true);
obj["executable"] = true;
if (st.narOffset)
obj.attr("narOffset", st.narOffset);
obj["narOffset"] = st.narOffset;
break;
case FSAccessor::Type::tDirectory:
obj.attr("type", "directory");
obj["type"] = "directory";
{
auto res2 = obj.object("entries");
obj["entries"] = json::object();
json &res2 = obj["entries"];
for (auto & name : accessor->readDirectory(path)) {
if (recurse) {
auto res3 = res2.placeholder(name);
listNar(res3, accessor, path + "/" + name, true);
res2[name] = listNar(accessor, path + "/" + name, true);
} else
res2.object(name);
res2[name] = json::object();
}
}
break;
case FSAccessor::Type::tSymlink:
obj.attr("type", "symlink");
obj.attr("target", accessor->readLink(path));
obj["type"] = "symlink";
obj["target"] = accessor->readLink(path);
break;
default:
throw Error("path '%s' does not exist in NAR", path);
}
return obj;
}
}
+2 -4
View File
@@ -2,6 +2,7 @@
#include <functional>
#include <nlohmann/json_fwd.hpp>
#include "fs-accessor.hh"
namespace nix {
@@ -24,11 +25,8 @@ ref<FSAccessor> makeLazyNarAccessor(
const std::string & listing,
GetNarBytes getNarBytes);
class JSONPlaceholder;
/* Write a JSON representation of the contents of a NAR (except file
contents). */
void listNar(JSONPlaceholder & res, ref<FSAccessor> accessor,
const Path & path, bool recurse);
nlohmann::json listNar(ref<FSAccessor> accessor, const Path & path, bool recurse);
}
+11 -11
View File
@@ -2,7 +2,6 @@
#include <nlohmann/json.hpp>
#include <regex>
#include "json.hh"
namespace nix {
@@ -90,6 +89,7 @@ std::optional<Strings> ParsedDerivation::getStringsAttr(const std::string & name
StringSet ParsedDerivation::getRequiredSystemFeatures() const
{
// FIXME: cache this?
StringSet res;
for (auto & i : getStringsAttr("requiredSystemFeatures").value_or(Strings()))
res.insert(i);
@@ -125,6 +125,11 @@ bool ParsedDerivation::substitutesAllowed() const
return getBoolAttr("allowSubstitutes", true);
}
bool ParsedDerivation::useUidRange() const
{
return getRequiredSystemFeatures().count("uid-range");
}
static std::regex shVarName("[A-Za-z_][A-Za-z0-9_]*");
std::optional<nlohmann::json> ParsedDerivation::prepareStructuredAttrs(Store & store, const StorePathSet & inputPaths)
@@ -144,16 +149,11 @@ std::optional<nlohmann::json> ParsedDerivation::prepareStructuredAttrs(Store & s
auto e = json.find("exportReferencesGraph");
if (e != json.end() && e->is_object()) {
for (auto i = e->begin(); i != e->end(); ++i) {
std::ostringstream str;
{
JSONPlaceholder jsonRoot(str, true);
StorePathSet storePaths;
for (auto & p : *i)
storePaths.insert(store.parseStorePath(p.get<std::string>()));
store.pathInfoToJSON(jsonRoot,
store.exportReferences(storePaths, inputPaths), false, true);
}
json[i.key()] = nlohmann::json::parse(str.str()); // urgh
StorePathSet storePaths;
for (auto & p : *i)
storePaths.insert(store.parseStorePath(p.get<std::string>()));
json[i.key()] = store.pathInfoToJSON(
store.exportReferences(storePaths, inputPaths), false, true);
}
}
+2
View File
@@ -38,6 +38,8 @@ public:
bool substitutesAllowed() const;
bool useUidRange() const;
std::optional<nlohmann::json> prepareStructuredAttrs(Store & store, const StorePathSet & inputPaths);
};
+10
View File
@@ -93,4 +93,14 @@ struct RealisedPath {
GENERATE_CMP(RealisedPath, me->raw);
};
class MissingRealisation : public Error
{
public:
MissingRealisation(DrvOutput & outputId)
: Error( "cannot operate on an output of the "
"unbuilt derivation '%s'",
outputId.to_string())
{}
};
}
+37 -20
View File
@@ -67,6 +67,40 @@ void RefScanSink::operator () (std::string_view data)
}
PathRefScanSink::PathRefScanSink(StringSet && hashes, std::map<std::string, StorePath> && backMap)
: RefScanSink(std::move(hashes))
, backMap(std::move(backMap))
{ }
PathRefScanSink PathRefScanSink::fromPaths(const StorePathSet & refs)
{
StringSet hashes;
std::map<std::string, StorePath> backMap;
for (auto & i : refs) {
std::string hashPart(i.hashPart());
auto inserted = backMap.emplace(hashPart, i).second;
assert(inserted);
hashes.insert(hashPart);
}
return PathRefScanSink(std::move(hashes), std::move(backMap));
}
StorePathSet PathRefScanSink::getResultPaths()
{
/* Map the hashes found back to their store paths. */
StorePathSet found;
for (auto & i : getResult()) {
auto j = backMap.find(i);
assert(j != backMap.end());
found.insert(j->second);
}
return found;
}
std::pair<StorePathSet, HashResult> scanForReferences(
const std::string & path,
const StorePathSet & refs)
@@ -82,30 +116,13 @@ StorePathSet scanForReferences(
const Path & path,
const StorePathSet & refs)
{
StringSet hashes;
std::map<std::string, StorePath> backMap;
for (auto & i : refs) {
std::string hashPart(i.hashPart());
auto inserted = backMap.emplace(hashPart, i).second;
assert(inserted);
hashes.insert(hashPart);
}
PathRefScanSink refsSink = PathRefScanSink::fromPaths(refs);
TeeSink sink { refsSink, toTee };
/* Look for the hashes in the NAR dump of the path. */
RefScanSink refsSink(std::move(hashes));
TeeSink sink { refsSink, toTee };
dumpPath(path, sink);
/* Map the hashes found back to their store paths. */
StorePathSet found;
for (auto & i : refsSink.getResult()) {
auto j = backMap.find(i);
assert(j != backMap.end());
found.insert(j->second);
}
return found;
return refsSink.getResultPaths();
}
+13
View File
@@ -27,6 +27,19 @@ public:
void operator () (std::string_view data) override;
};
class PathRefScanSink : public RefScanSink
{
std::map<std::string, StorePath> backMap;
PathRefScanSink(StringSet && hashes, std::map<std::string, StorePath> && backMap);
public:
static PathRefScanSink fromPaths(const StorePathSet & refs);
StorePathSet getResultPaths();
};
struct RewritingSink : Sink
{
std::string from, to, prev;
+3 -5
View File
@@ -1,6 +1,6 @@
#include <nlohmann/json.hpp>
#include "remote-fs-accessor.hh"
#include "nar-accessor.hh"
#include "json.hh"
#include <sys/types.h>
#include <sys/stat.h>
@@ -38,10 +38,8 @@ ref<FSAccessor> RemoteFSAccessor::addToCache(std::string_view hashPart, std::str
if (cacheDir != "") {
try {
std::ostringstream str;
JSONPlaceholder jsonRoot(str);
listNar(jsonRoot, narAccessor, "", true);
writeFile(makeCacheFile(hashPart, "ls"), str.str());
nlohmann::json j = listNar(narAccessor, "", true);
writeFile(makeCacheFile(hashPart, "ls"), j.dump());
} catch (...) {
ignoreException();
}
+1 -4
View File
@@ -879,10 +879,7 @@ std::vector<BuildResult> RemoteStore::buildPathsWithResults(
auto realisation =
queryRealisation(outputId);
if (!realisation)
throw Error(
"cannot operate on an output of unbuilt "
"content-addressed derivation '%s'",
outputId.to_string());
throw MissingRealisation(outputId);
res.builtOutputs.emplace(realisation->id, *realisation);
} else {
// If ca-derivations isn't enabled, assume that
Loaded 100 of 164 files, more files were not shown because too many files have changed in this diff. Show more