Compare commits

...

1329 Commits

Author SHA1 Message Date
Eelco Dolstra
fcb4da808f Updates from the last Nix team meeting 2022-10-28 14:12:38 +02:00
Valentin Gagarin
7ecf3074a5 Update roadmap.md
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2022-10-11 17:15:35 +02:00
Eelco Dolstra
10fe0c360e Update roadmap.md 2022-10-11 15:42:19 +02:00
Théophane Hufschmitt
8e898ced0a Create a roadmap for Nix
First very rough draft of a roadmap as discussed in last week's Nix team meeting.

This is merely the output of a 45mins brainstorming session so far, so absolutely nothing is settled
2022-10-11 10:05:54 +02:00
Valentin Gagarin
ac0fb38e8a Merge pull request #6652 from abathur/doc_install_tests 2022-10-05 17:45:42 +02:00
Travis A. Everett
e1418430ac Apply suggestions from code review
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2022-10-05 09:52:53 -05:00
Valentin Gagarin
927234cfb2 Merge pull request #6870 from amjoseph-nixpkgs/pr/doc/explain-local-remote-binary-substituter 2022-10-05 09:01:42 +02:00
Théophane Hufschmitt
3ae9467d57 Merge pull request #6969 from fricklerhandwerk/refactor-generate-manpage
refactor rendering command documentation to markdown
2022-10-03 15:50:17 +02:00
Eelco Dolstra
89ca75c9f9 Merge pull request #7080 from squalus/nar-close-file
archive: check close errors when extracting nars
2022-09-30 12:58:28 +02:00
Eelco Dolstra
16e088c5db Merge pull request #7108 from rapenne-s/wording
nix-copy-closure: improve wording
2022-09-30 12:54:25 +02:00
Valentin Gagarin
d8bef7358f bring back lost newline 2022-09-30 01:43:57 +02:00
Valentin Gagarin
70eea97742 use more self-explanatory names 2022-09-30 01:43:57 +02:00
Valentin Gagarin
61188cb820 move final template to the top
this allows readers to enter the code starting with what is visible from
the outside, instead of working themselves up from purely technical details.
2022-09-30 01:43:57 +02:00
Valentin Gagarin
0e0f1832de remove superfluous let-in pair 2022-09-30 01:43:57 +02:00
Valentin Gagarin
a85df04fcb refactor showOptions
it was quite a pain to manipulate strings just with `builtins`
2022-09-30 01:43:53 +02:00
Valentin Gagarin
4655563470 refactor rendering command documentation to markdown
idea:
- make document structure visible, like in a template
- order functions by descending abstraction
- avoid nested let bindings
2022-09-30 01:42:46 +02:00
Valentin Gagarin
6b56bb4a79 use indented strings where appropriate 2022-09-30 01:42:46 +02:00
Théophane Hufschmitt
82ef3c470b Merge pull request #7114 from rapenne-s/reenable_test
tests/build-dry: re-enable some test
2022-09-29 16:33:12 +02:00
Solene Rapenne
62d53bc8a4 tests/build-dry: re-enable some test 2022-09-28 15:54:45 +02:00
Solene Rapenne
847cd49909 nix-copy-closure: improve wording 2022-09-28 01:05:46 +02:00
Eelco Dolstra
050fcd391b Merge pull request #7105 from SquidDev/hotfix/fish-return
Use exit instead of return in /etc/profile.d/nix-daemon.fish
2022-09-27 15:02:29 +02:00
Jonathan Coates
0259702219 Use exit instead of return in fish profile
Older versions of Fish (such as those bundled with Ubuntu LTS 22.04) do
not support return outside of functions. We need to use the equivalent
exit instead.
2022-09-27 12:59:37 +01:00
Eelco Dolstra
ace9f261da Merge pull request #7103 from lheckemann/remove-add-root
nix-build: remove unused --add-root arg
2022-09-27 11:37:09 +02:00
Linus Heckemann
19f3ecd830 nix-build: remove unused --add-root arg
Fixes #1982
2022-09-27 00:07:42 +02:00
Eelco Dolstra
dfc3a9afc3 Merge pull request #7043 from edolstra/installer-test
Add an installer test
2022-09-26 17:34:23 +02:00
Eelco Dolstra
71e364c9c7 Quote URLs 2022-09-26 16:57:06 +02:00
Eelco Dolstra
eceaf1997c Remove FIXME 2022-09-26 16:54:31 +02:00
Eelco Dolstra
6806f20157 Merge pull request #7089 from NixOS/mention-accept-flake-config-in-warning
Mention `--accept-flake-config` in the related warning
2022-09-26 11:13:02 +02:00
Eelco Dolstra
e25a36cdac Merge pull request #7093 from amjoseph-nixpkgs/pr/intersectAttrs/values
src/libexpr/primops.cc: correct definition for intersectAttrs
2022-09-26 11:11:53 +02:00
Adam Joseph
dc205c75a2 src/libexpr/primops.cc: correct definition for intersectAttrs
The current definition of `intersectAttrs` is incorrect:

> Return a set consisting of the attributes in the set e2 that also exist in the
> set e1.

Recall that (Nix manual, section 5.1):

> An attribute set is a collection of name-value-pairs (called attributes)

According to the existing description of `intersectAttrs`, the following should
evaluate to the empty set, since no key-value *pair* (i.e. attribute) exists in
both sets:

```
builtins.intersectAttrs { x=3; } {x="foo";}
```

And yet:

```
nix-repl> builtins.intersectAttrs { x=3; } {x="foo";}
{ x = "foo"; }
```

Clearly the intent here was for the *names* of the resulting attribute set to be
the intersection of the *names* of the two arguments, and for the values of the
resulting attribute set to be the values from the second argument.

This commit corrects the definition, making it match the implementation and intent.
2022-09-24 23:51:09 -07:00
Théophane Hufschmitt
371013c08d Merge pull request #7010 from edolstra/ignore-shell
nix develop: Ignore stdenv's $SHELL
2022-09-24 12:55:53 +02:00
Théophane Hufschmitt
3eb185d6d1 Merge pull request #7046 from mkenigs/fix-root-SHELL
dockerImage: fix root shell
2022-09-24 12:53:46 +02:00
Théophane Hufschmitt
db29ddd113 Merge pull request #7078 from obsidiansystems/trustworthy-signature
"valid signature" -> "signature by a trusted key"
2022-09-24 12:52:35 +02:00
Théophane Hufschmitt
74276cb354 Merge pull request #7079 from matthewbauer/allow-untrusted-settings
Allow pass max-silent-time and build-poll-interval to daemon untrusted
2022-09-24 12:47:49 +02:00
Théophane Hufschmitt
6a64820cb4 Merge pull request #7082 from rapenne-s/ccache
add ccacheStdenv
2022-09-24 12:45:27 +02:00
Théophane Hufschmitt
1a5d094be7 Mention --accept-flake-config in the related warning
Make sure that people who run Nix in non-interactive mode (and so don't have the possibility to interactively accept the individual flake configuration settings) are aware of this flag.

Fix #7086
2022-09-24 12:11:26 +02:00
Eelco Dolstra
cacfc32ee2 Merge pull request #7056 from hiroqn/fix-uname
Fix `uname -m` return value for armv6l/armv7l
2022-09-24 11:13:18 +02:00
Eelco Dolstra
c13007f012 Merge pull request #7059 from NixOS/remove-useless-ca-file-message
Remove a useless debug message in filetransfer.cc
2022-09-24 10:39:23 +02:00
John Ericson
60e23c8bae Apply suggestions from code review
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
Co-authored-by: Rune K. Svendsen <runesvend@gmail.com>
2022-09-23 13:57:57 -04:00
Solène Rapenne
e04b38f789 add ccacheStdenv
when using ccache, rebuild time has been measured
89% faster while not slowing the speed of cold builds
2022-09-23 11:21:19 +02:00
squalus
223f8dace0 archive: check close errors when extracting nars 2022-09-22 12:50:32 -07:00
Matthew Bauer
6e049ae607 Allow pass max-silent-time and build-poll-interval to daemon untrusted
These settings seem harmless, they control the same polling
functionality that timeout does, but with different behavior. Should
be safe for untrusted users to pass in.
2022-09-22 13:59:16 -05:00
John Ericson
a2a8cb10ac Dodge "trusted" vs "trustworthy" by being explicit
Hopefully this is best!
2022-09-22 14:37:52 -04:00
John Ericson
752f967c0f "valid signature" -> "trustworthy signature"
I just had a colleague get confused by the previous phrase for good
reason. "valid" sounds like an *objective* criterion, e.g. and *invalid
signature* would be one that would be trusted by no one, e.g. because it
misformatted or something.

What is actually going is that there might be a signature which is
perfectly valid to *someone else*, but not to the user, because they
don't trust the corresponding public key. This is a *subjective*
criterion, because it depends on the arbitrary and personal choice of
which public keys to trust.

I therefore think "trustworthy" is a better adjective to use. Whether
something is worthy of trust is clearly subjective, and then "trust"
within that word nicely evokes `trusted-public-keys` and friends.
2022-09-22 10:49:31 -04:00
Théophane Hufschmitt
f704c2720f Merge pull request #7070 from alyssais/repl-history
nix repl: warn if creating dir for history fails
2022-09-21 13:50:49 +02:00
Alyssa Ross
d234d01f01 nix repl: warn if creating dir for history fails
The history is not critical to the functionality of nix repl, so it's
enough to warn here, rather than refuse to start if the directory Nix
thinks the history should live in can't be created.
2022-09-20 11:01:40 +00:00
Théophane Hufschmitt
9dadb5481c Merge pull request #7065 from squalus/schema-corrupt
Improve durability of schema version file writes
2022-09-20 10:55:13 +02:00
hiroqn
ca22936853 fix uname -m return value for armv6l/armv7l 2022-09-20 15:36:36 +09:00
squalus
1b595026e1 Improve durability of schema version file writes
- call close explicitly in writeFile to prevent the close exception
  from being ignored
- fsync after writing schema file to flush data to disk
- fsync schema file parent to flush metadata to disk

https://github.com/NixOS/nix/issues/7064
2022-09-19 20:13:30 -07:00
Théophane Hufschmitt
9d860f3467 Merge pull request #7060 from fricklerhandwerk/issue-templates
issue template: fill 'about' field
2022-09-19 10:53:13 +02:00
Valentin Gagarin
4b11c22386 issue template: fill 'about' field
according to [GitHub documentation] some fields are required.
`about` is not listed, but it probably is required.

[GitHub documentation]: https://docs.github.com/en/communities/using-templates-to-encourage-useful-issues-and-pull-requests/syntax-for-issue-forms
2022-09-19 10:05:29 +02:00
Théophane Hufschmitt
0f977bf91e Remove a useless debug message in filetransfer.cc
Remove the `verify TLS: Nix CA file = 'blah'` message that Nix used to print when fetching anything as it's both useless (`libcurl` prints the same info in its logs) and misleading (gives the impression that a new TLS connection is being established which might not be the case because of multiplexing. See #7011 )
2022-09-19 08:42:43 +02:00
Travis A. Everett
84bdb0e3ad address review feedback
Mainly:
- Try to triangulate between narrative that framed this as
  a new/easy process and the need for a reference that will
  not quickly grow stale.
- Fix a ~continuity issue where the text was talking about
  "your Cachix cache" before saying that you'd need to make
  a Cachix cache to enable the installer tests.
- Adopt suggestion on titling, and nest subtitles in the
  installer test section.
2022-09-18 13:02:42 -05:00
Travis A. Everett
4bd52bf6c4 Apply suggestions from code review
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2022-09-17 13:20:11 -05:00
Eelco Dolstra
a38a55babe Merge pull request #7052 from amjoseph-nixpkgs/pr/fetch/impure
libexpr/fetchurl.nix: allow __impure fetch
2022-09-16 13:50:50 +02:00
Adam Joseph
ad5b09423a release-notes/rl-next.md: note new argument to fetchurl.nix 2022-09-16 01:59:24 -07:00
Adam Joseph
fb985f855c fetchurl.nix: change other use of __impure 2022-09-16 01:54:24 -07:00
Adam Joseph
673fd21b7c Update src/libexpr/fetchurl.nix
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2022-09-16 08:51:14 +00:00
Théophane Hufschmitt
41f9b74320 Merge pull request #7050 from fricklerhandwerk/issue-templates
add issue template for missing or incorrect documentation
2022-09-16 10:26:18 +02:00
Adam Joseph
b3550d9179 libexpr/fetchurl.nix: allow __impure fetch
This commit adds an optional `__impure` parameter to fetchurl.nix, which allows
the caller to use `libfetcher`'s fetcher in an impure derivation.  This allows
nixpkgs' patch-normalizing fetcher (fetchpatch) to be rewritten to use nix's
internal fetchurl, thereby eliminating the awkward "you can't use fetchpatch
here" banners scattered all over the place.

See also: https://github.com/NixOS/nixpkgs/pull/188587
2022-09-16 00:48:01 -07:00
Valentin Gagarin
875a99eaa4 fix markdown rendering quirk
markdown would interpret parentheses as belonging to the first link without escaping.
2022-09-16 09:41:27 +02:00
Valentin Gagarin
dc8c0b173c Update doc/manual/src/contributing/hacking.md 2022-09-16 09:36:55 +02:00
Valentin Gagarin
1ae974120a Update doc/manual/src/contributing/hacking.md 2022-09-16 09:36:37 +02:00
Valentin Gagarin
0a4bd9fe88 Update doc/manual/src/contributing/hacking.md 2022-09-16 09:36:30 +02:00
Valentin Gagarin
47fa1087c8 Update doc/manual/src/contributing/hacking.md 2022-09-16 09:36:20 +02:00
Valentin Gagarin
84fb036062 add issue template for missing or incorrect documentation
this allows anyone to create labelled issues for easy filtering.
2022-09-15 18:16:17 +02:00
Eelco Dolstra
0d4bf9c4d8 Fix evaluation 2022-09-15 15:56:46 +02:00
Eelco Dolstra
3dd313a7c2 Add RHEL 9 2022-09-15 15:50:52 +02:00
Eelco Dolstra
ba04b5b1d7 Disable Ubuntu 14.04 2022-09-15 14:51:44 +02:00
Eelco Dolstra
a96ad2ab25 Add RHEL 7/8 2022-09-15 14:51:10 +02:00
Eelco Dolstra
503f31e2a0 Use libvirt image 2022-09-15 13:28:03 +02:00
Eelco Dolstra
ef714aa8a5 Remove pre-release Ubuntu 22.10 2022-09-15 13:25:26 +02:00
Eelco Dolstra
5c8cdb9b60 Add Ubuntu 22.04 LTS 2022-09-15 13:24:17 +02:00
Eelco Dolstra
29aaec1e59 Make cross product of images and tests 2022-09-15 13:15:26 +02:00
Eelco Dolstra
fe958a682d Test building 2022-09-15 11:42:10 +02:00
Matthew Kenigsberg
02af02854d dockerImage: fix root shell
Currently root's shell is set to a path that does not exist; this change
sets it to the correct path to bash
2022-09-14 16:20:58 -06:00
Eelco Dolstra
cc6e312315 Get Ubuntu 22.10 to work 2022-09-14 19:44:41 +02:00
Eelco Dolstra
906c947ee8 Enable daemon installation test on Fedora 2022-09-14 18:55:45 +02:00
Eelco Dolstra
0a8e666dd6 Add Fedora 36 2022-09-14 18:40:16 +02:00
Eelco Dolstra
fae3b4fe8a Add an installer test
This runs the installer in a QEMU VM. Unlike the old installer test
that ran inside a declaratively built RedHat/Debian image, this uses
an image from Vagrant.
2022-09-14 15:41:11 +02:00
Eelco Dolstra
88a45d6149 Merge pull request #7014 from DeterminateSystems/graham/ds-327-fish-support-for-the-nix-installer
Add Fish suport to the Nix installer
2022-09-14 14:46:26 +02:00
Théophane Hufschmitt
88646eea29 Merge pull request #7025 from fricklerhandwerk/issue-templates
issue template: add feature label
2022-09-14 11:00:48 +02:00
Ana Hobden
7194c87dce Add installer_test matrix for shells
Signed-off-by: Ana Hobden <operator@hoverbear.org>
2022-09-13 12:57:29 -04:00
Ana Hobden
8ebdbeb257 Add fish suport to installer
Before this patch, installing Nix using the Fish shell did not
work because Fish wasn't configured to add Nix to the PATH. Some
options in #1512 offered workarounds, but they typically involve
extra plugins or packages.

This patch adds native, out-of-the-box support for the Fish shell.

Note that Fish supports a `conf.d` directory, which is intended
for exactly use cases like this: software projects distributing
shell snippets. This patch takes advantage of it. The installer
doesn't append any Nix loader behavior to any Fish config file.
Because of that, the uninstall process is smooth and a reinstall
obliterates the existing nix.fish files that we place instead of
bothering the user with a backup / manual removal.

Both single-user and multi-user cases are covered. It has been
tested on Ubuntu, and a Mac with MacPorts, homebrew, and the
Fish installer pkg.

Closes #1512

Co-authored-by: Graham Christensen <graham@grahamc.com>
2022-09-13 12:56:16 -04:00
Eelco Dolstra
b8f334bd85 Merge pull request #7034 from edolstra/trim-descriptions
Trim option descriptions
2022-09-13 17:40:22 +02:00
Eelco Dolstra
d365cced4f Trim option descriptions
This removes unintended blank lines in Markdown when the description
is a multiline string literal.
2022-09-13 16:58:32 +02:00
Eelco Dolstra
b56906aae0 Merge pull request #7032 from edolstra/stop-progress-bar
RunPager: Stop the progress bar
2022-09-13 16:57:30 +02:00
Eelco Dolstra
c6ff33ff5c RunPager: Stop the progress bar
In particular, the progress bar was interfering with 'less' rendering
in '--help' (e.g. run 'nix --help' and hit '/' to search).
2022-09-13 15:29:13 +02:00
Théophane Hufschmitt
0f64bf445a Merge pull request #6994 from agbrooks/master
Prevent tempdir from being GC-ed before addToStoreFromDump completes
2022-09-13 09:23:16 +02:00
Andrew Brooks
565d888e0f Address PR feedback on #6694 2022-09-12 11:33:23 -05:00
Valentin Gagarin
7852609999 issue template: add feature label
each change should be an improvement, a label for that is redundant.
2022-09-12 11:27:25 +02:00
Eelco Dolstra
aa4ca8b7e4 Merge pull request #4918 from roberth/installer-pc
installer: Jeeze -> Oh no
2022-09-09 16:49:59 +02:00
Valentin Gagarin
dceb4f9dbb Merge pull request #6825 from rapenne-s/nix_shell_example
nix shell: document how to invoke multiple commands from the command …
2022-09-09 13:35:23 +02:00
Valentin Gagarin
1d295e5fe7 Merge pull request #6934 from fricklerhandwerk/revert-architecture
Revert #6420 "Document what Nix *is*" so we can start over with smaller change sets
2022-09-09 13:31:05 +02:00
Théophane Hufschmitt
c8e8eea95e Merge pull request #7008 from jonringer/add-command-docs
Docs: Add nix develop --command entry
2022-09-08 13:34:40 +02:00
Andrew Brooks
84fe75a12a Keep created temp dirs inside store, but protect from GC
Implements the approach suggested by feedback on PR #6994, where
tempdir paths are created in the store (now with an exclusive lock).

As part of this work, the currently-broken and unused
`createTempDirInStore` function is updated to create an exclusive lock
on the temp directory in the store.

The GC now makes a non-blocking attempt to lock any store directories
that "look like" the temp directories created by this function, and if
it can't acquire one, ignores the directory.
2022-09-06 17:48:00 -05:00
Eelco Dolstra
ece12a97d9 lockfile -> lock file for consistency 2022-09-06 19:20:31 +02:00
Eelco Dolstra
27be54ca53 nix develop: Ignore stdenv's $SHELL
Stdenv sets this to a bash that doesn't have readline/completion
support, so running 'nix (develop|shell)' inside a 'nix develop' gives
you a crippled shell. So let's just ignore the derivation's $SHELL.

This could break interactive use of build phases that use $SHELL, but
they appear to be fairly rare.
2022-09-06 18:27:39 +02:00
Eelco Dolstra
9600109879 Merge pull request #6995 from matthewbauer/disable-sa-restart-on-macos
Disable SA_RESTART for some signals on macOS
2022-09-06 17:36:31 +02:00
Jonathan Ringer
6ce2e96c88 Docs: Add nix develop --command entry
Add example of nix develop being used to execuate a series of script
commands. This is common when doing things like CI/CD, and should be
represented in the official documentation.

Also useful for people looking for the 'nix develop' equivalent of
'nix-shell --run'.

Related:
 - https://github.com/NixOS/nix/issues/6908
 - https://github.com/NixOS/nix/issues/6908#issuecomment-1229266853
2022-09-06 08:18:29 -07:00
Matthew Bauer
77d3d3d18d Remove some signals 2022-09-06 09:47:53 -05:00
Eelco Dolstra
7f31c08224 Merge pull request #7002 from tweag/reset-timestamps-in-installer
Installer: Reset the timestamps in the tarball
2022-09-06 13:19:56 +02:00
Théophane Hufschmitt
089f584173 Merge pull request #6980 from virusdave/patch-1
Set `HOME` var to `root`'s home when running `nix-store` as `root`
2022-09-05 16:21:05 +02:00
Théophane Hufschmitt
3fca5f6c69 Installer: Reset the timestamps in the tarball
Otherwise it isn't reproducible.

Fix https://github.com/NixOS/nix/issues/7001
2022-09-05 14:44:01 +02:00
Théophane Hufschmitt
df5159e7d3 Merge pull request #6993 from matthewbauer/dont-read-derivation-in-goal
Don’t readDerivation if impure derivations feature is disabled
2022-09-05 11:20:58 +02:00
Matthew Bauer
a47b5476e1 Add more signals 2022-09-03 16:06:33 -05:00
Matthew Bauer
102434e4cb Disable SA_RESTART for some signals on macOS
Disables the SA_RESTART behavior on macOS which causes:

> Restarting of pending calls is requested by setting the SA_RESTART bit
> in sa_flags. The affected system calls include read(2), write(2),
> sendto(2), recvfrom(2), sendmsg(2) and recvmsg(2) on a communications
> channel or a slow device (such as a terminal, but not a regular file)
> and during a wait(2) or ioctl(2).

From: https://man.openbsd.org/sigaction#SA_RESTART

This being set on macOS caused a bug where read() calls to the daemon
socket were blocking after a SIGINT was received. As a result,
checkInterrupt was never reached even though the signal was received
by the signal handler thread.

On Linux, SA_RESTART is disabled by default. This probably effects
other BSDs but I don’t have the ability to test it there right now.
2022-09-03 00:27:16 -05:00
Andrew Brooks
1f041ac54f Prevent tempdir from being GC-ed before addToStoreFromDump has renamed it
This fixes issue 6823 by placing the tempdir used in LocalStore::addToStoreFromDump
outside the Nix store, where automatic GC is no longer a concern.
2022-09-02 18:32:35 -05:00
Eelco Dolstra
0c7f213c87 Merge pull request #6990 from matthewbauer/no-space-in-zsh
Don’t add a space after attrs completion in zsh
2022-09-02 19:01:52 +02:00
Matthew Bauer
4894e567fb Don’t readDerivation if impure derivations feature is disabled
readDerivation is pretty slow, and while it may not be significant for
some use cases, on things like ghc-nix where we have thousands of
derivations is really slows things down.

So, this just doesn’t do the impure derivation check if the impure
derivation experimental feature is disabled. Perhaps we could cache
the result of isPure() and keep the check, but this is a quick fix to
for the slowdown introduced with impure derivations features in 2.8.0.
2022-09-02 11:46:34 -05:00
Matthew Bauer
dd3cd1a167 Apply suggestions from code review 2022-09-02 10:50:02 -05:00
Eelco Dolstra
a9af12e658 Merge pull request #6987 from matthewbauer/update-boehmgc-coroutine-sp-fallback-for-darwin
Update boehmgc-coroutine-sp-fallback.diff for darwin
2022-09-02 13:22:55 +02:00
Matthew Bauer
bd63ae7e18 Don’t add a space after attrs completion in zsh
This matches the behavior of bash. We don’t want to add a space after
completion on attrs. Uses -S.

Switches to new compadd style comppletions instead of _describe.
Shouldn’t have any negative issues from what I can tell.
2022-09-01 23:16:55 -05:00
Adam Joseph
e90f2fcfc7 glossary: add entry for chroot store (used 11 times in nix) 2022-09-01 18:28:05 -07:00
Adam Joseph
e6f5352e71 #binary-cache -> #gloss-binary-cache 2022-09-01 18:27:00 -07:00
Adam Joseph
1ab913467e linkify mention of other options 2022-09-01 18:03:35 -07:00
Adam Joseph
59dc8346ca move substituter signature-checking conditions to configuration file documentation 2022-09-01 17:51:56 -07:00
Adam Joseph
f6c750e8b2 glossary: remove extraneous comma 2022-09-01 17:48:34 -07:00
Adam Joseph
887e922be2 glossary: outputHash, not __outputHash 2022-09-01 17:47:13 -07:00
Adam Joseph
d5e064d816 glossary: fix broken link 2022-09-01 17:46:31 -07:00
Adam Joseph
57f12df5e4 input-addressed store object: include FODOs 2022-09-01 15:09:10 -07:00
Adam Joseph
8139bbe2ba implement https://github.com/NixOS/nix/pull/6870#pullrequestreview-1093700220 2022-09-01 15:01:07 -07:00
Adam Joseph
def4fb9a0f __outputHash: add link 2022-09-01 14:47:33 -07:00
Adam Joseph
1b2b8c39fd fix link to language/advanced-attributes.md#adv-attr-contentAddressed 2022-09-01 14:47:17 -07:00
Adam Joseph
0a98d564b3 glossary: resolve FIXME in #gloss-fixed-output-derivation 2022-09-01 14:41:27 -07:00
Adam Joseph
1f56b5d772 doc/manual: un-inline definitions from substitute 2022-09-01 14:13:21 -07:00
Adam Joseph
41153f30bd glossary: substituter: merge output-addressed cases 2022-09-01 14:13:21 -07:00
Adam Joseph
9cb8412143 Update doc/manual/src/glossary.md
Co-authored-by: John Ericson <git@JohnEricson.me>
2022-09-01 20:41:04 +00:00
Adam Joseph
2812682ebe Update doc/manual/src/glossary.md
Co-authored-by: John Ericson <git@JohnEricson.me>
2022-09-01 20:40:39 +00:00
Adam Joseph
520587b9a0 glossary: local store: clarify 2022-09-01 13:39:48 -07:00
Matthew Bauer
f4d7208e23 Update boehmgc-coroutine-sp-fallback.diff for darwin
The darwin_stop_world implementation is slightly different. sp goes to
altstack_lo instead of lo in this case. Assuming that is an
implementation detail.

But the fix is the same, when we detect alstack_lo outside of the
expected stack range, we reset it to hi - stack_limit.

Here stack_limit is calculated with pthread_get_stacksize_np since
that is the BSD equivalent to pthread_attr_getstacksize.
2022-09-01 11:48:50 -05:00
Eelco Dolstra
4823067247 Merge pull request #6983 from edolstra/revert-6621
Revert "Merge pull request #6621 from Kha/nested-follows"
2022-09-01 16:07:17 +02:00
Eelco Dolstra
a8b3d777fb Revert "Merge pull request #6621 from Kha/nested-follows"
This reverts commit c530cda345, reversing
changes made to 4adcdff5c1.
2022-09-01 15:26:19 +02:00
Théophane Hufschmitt
c530cda345 Merge pull request #6621 from Kha/nested-follows
Fix nested flake input overrides
2022-09-01 12:04:00 +02:00
Rok Garbas
4adcdff5c1 Merge pull request #6973 from Enzime/skip-docker-push
Only push Docker image when Docker secrets are set
2022-08-31 23:52:37 +02:00
Dave Nicponski
a2b7baa42f Set HOME var to root's home when running nix-store as root
A [recent-ish change](https://github.com/NixOS/nix/pull/6676) logs a warning when a potentially counterintuitive situation happens.

This now causes the multi-user installer to [emit a warning](https://github.com/NixOS/nixpkgs/issues/189043) when it's doing
the "seed the Nix database" step via a low-level `nix-store --load-db` invocation.

`nix-store` functionality implementations don't actually use profiles or channels or homedir as far as i can tell.  So why are we 
hitting this code at all?  

Well, the current command approach for functionality here builds a [fat `nix` binary](https://github.com/NixOS/nix/blob/master/src/nix/local.mk#L23-L26) which has _all_ the functionality of
previous individual binaries (nix-env, nix-store, etc) bundled in, then [uses the invocation name](https://github.com/NixOS/nix/blob/master/src/nix/main.cc#L274-L277) to select the
set of commands to expose.  `nix` itself has this behavior, even when just trying to parse the (sub)command and arguments:

```
dave @ davembp2
$ nix
error: no subcommand specified
Try 'nix --help' for more information.

dave @ davembp2
$ sudo nix
warning: $HOME ('/Users/dave') is not owned by you, falling back to the one defined in the 'passwd' file
error: no subcommand specified
Try 'nix --help' for more information.

dave @ davembp2
$ HOME=~root sudo nix
error: no subcommand specified
Try 'nix --help' for more information.
```

This behavior can also be seen pretty easily with an arbitrary `nix-store` invocation:
```
dave @ davembp2 
$ nix-store --realize

dave @ davembp2 
$ sudo nix-store --realize  # what installer is doing now
warning: $HOME ('/Users/dave') is not owned by you, falling back to the one defined in the 'passwd' file

dave @ davembp2
$ sudo HOME=~root nix-store --realize  # what this PR effectively does

dave @ davembp2
$ 
```
2022-08-31 17:25:26 -04:00
Michael Hoang
85248543b5 Only push Docker image when Docker secrets are set 2022-08-29 23:59:57 +10:00
Eelco Dolstra
ddb82ffda9 Merge pull request #6972 from sashashura/patch-1
GitHub Workflows security hardening
2022-08-29 09:29:51 +02:00
Eelco Dolstra
1f2b12b30e Merge pull request #6968 from fricklerhandwerk/man-no-weblinks
do not render relative links in help pages
2022-08-29 08:46:35 +02:00
Alex
90ba893329 Update backport.yml 2022-08-28 21:59:29 +01:00
Sebastian Ullrich
6f65c11780 Fix overlapping flake follows 2022-08-28 11:50:25 +02:00
Sebastian Ullrich
2b9d381301 Fix nested flake input overrides 2022-08-28 11:47:25 +02:00
Valentin Gagarin
e7dcacbd7c do not render relative links in help pages
this simplifies the setup a lot, and avoids weird looking `./file.md`
links showing up.

it also does not show regular URLs any more. currently the command
reference only has few of them, and not showing them in the offline
documentation is hopefully not a big deal.

instead of building more special-case solutions, clumsily preprocessing
the input, or issuing verbal rules on dealing with URLs, should better
be solved sustainably by not rendering relative links in `lowdown`:

https://github.com/kristapsdz/lowdown/issues/105
2022-08-26 17:10:41 +02:00
Eelco Dolstra
7918adbb62 Merge pull request #6954 from winterqt/darwin-sandbox-trustd
fix(libstore): allow access to trustd on macOS
2022-08-26 11:45:00 +02:00
Eelco Dolstra
0b25446f2e Merge pull request #6961 from edolstra/bump-version
Bump version
2022-08-25 12:30:09 +02:00
Eelco Dolstra
57cf36f81e Bump version 2022-08-25 11:50:14 +02:00
Eelco Dolstra
c777adc8d6 Merge pull request #6959 from edolstra/prepare-release
Prepare release
2022-08-24 23:24:21 +02:00
Eelco Dolstra
d11339948d Merge pull request #6958 from edolstra/fix-progress-bar-flicker
Fix progress bar flicker with -L
2022-08-24 23:15:10 +02:00
Eelco Dolstra
b0488a29dc Branch 2.11 release notes 2022-08-24 22:44:58 +02:00
Eelco Dolstra
d046eb1463 Bump version 2022-08-24 22:42:34 +02:00
Eelco Dolstra
bb411e4ae1 Fix progress bar flicker with -L
This was caused by -L calling setLogFormat() again, which caused the
creation of a new progress bar without destroying the old one. So we
had two progress bars clobbering each other.

We should change 'logger' to be a smart pointer, but I'll do that in a
future PR.

Fixes #6931.
2022-08-24 22:36:40 +02:00
Eelco Dolstra
5b8a53fb49 Merge pull request #6956 from edolstra/fix-eval-cache
Fix evaluation cache
2022-08-24 21:59:06 +02:00
Eelco Dolstra
a17ce0a8a9 Fix evaluation cache
98e361ad4c introduced a regression where
previously stored attributes were replaced by placeholders. As a
result, a command like 'nix build nixpkgs#hello' had to be executed at
least twice to get caching.

This code does not seem necessary for suggestions to work.
2022-08-24 21:19:43 +02:00
Winter
8e5659423e fix(libstore): allow access to trustd on macOS 2022-08-24 13:09:44 -04:00
Eelco Dolstra
04e74f7c8b Merge pull request #6612 from NixOS/parallel-nix-copy
Make nix copy parallel again
2022-08-24 15:31:42 +02:00
Eelco Dolstra
56d97d4b4d Remove redundant Finally 2022-08-24 14:49:58 +02:00
Eelco Dolstra
8d906b1f3b Fix macOS build 2022-08-24 14:11:03 +02:00
Eelco Dolstra
a1e1ec3dad Merge pull request #6945 from jsoref/nix-setup-tweak
darwin-install: fix shell hint
2022-08-24 13:56:26 +02:00
Eelco Dolstra
6528c11024 Merge pull request #6946 from rickynils/fix_serve_protocol_version_check
Fix a misplaced parenthese in serve protocol check
2022-08-24 13:09:50 +02:00
Rickard Nilsson
c2d7456926 Fix a misplaced parenthese in serve protocol check
This issue made it impossible for clients using a serve protocol of
version <= 2.3 to use the `cmdBuildDerivation` command of servers using
a protocol of version >= 2.6. The faulty version check makes the server
send back build outputs that the client is not expecting.
2022-08-24 01:54:43 +02:00
Josh Soref
4d4f2d10e7 darwin-install: fix shell hint 2022-08-23 19:38:53 -04:00
Eelco Dolstra
4a0c4ca186 Merge pull request #6944 from edolstra/bash-vars
nix develop: Ignore some more bash special variables
2022-08-23 20:15:18 +02:00
Eelco Dolstra
0e54fab0dd Merge pull request #6909 from ncfavier/json-paths
JSON: print paths as strings without copying them to the store
2022-08-23 15:19:33 +02:00
Eelco Dolstra
db026103b1 nix develop: Ignore some more bash special variables
Fixes #6940.
2022-08-23 14:57:08 +02:00
Eelco Dolstra
b2b607a7a9 Merge pull request #6902 from centromere/optional-nixpkgs
docker.nix: Provide boolean for whether to bundle nixpkgs
2022-08-23 14:45:15 +02:00
Eelco Dolstra
db71f5646f Merge pull request #6887 from winterqt/specify-bashinteractive-outputs
nix-shell: specify which outputs from bashInteractive to build
2022-08-23 14:44:23 +02:00
Naïm Favier
ff0b5a778c Revert to copyStore = true for nix-instantiate and nix-env 2022-08-23 14:40:27 +02:00
Eelco Dolstra
74c088566b Merge pull request #6907 from fricklerhandwerk/language-properties
reword description of language properties
2022-08-23 14:37:53 +02:00
Eelco Dolstra
f0358ed465 Fix a hang in nix-copy-ssh.sh
This hang for some reason didn't trigger in the Nix build, but did
running 'make installcheck' interactively. What happened:

* Store::addMultipleToStore() calls a SinkToSource object to copy a
  path, which in turn calls LegacySSHStore::narFromPath(), which
  acquires a connection.

* The SinkToSource object is not destroyed after the last bytes has
  been read, so the coroutine's stack is still alive and its
  destructors are not run. So the connection is not released.

* Then when the next path is copied, because max-connections = 1,
  LegacySSHStore::narFromPath() hangs forever waiting for a connection
  to be released.

The fix is to make sure that the source object is destroyed when we're
done with it.
2022-08-23 14:19:53 +02:00
Eelco Dolstra
f865048332 Indentation 2022-08-22 15:32:53 +02:00
Eelco Dolstra
4c96761c2b Merge remote-tracking branch 'origin/master' into parallel-nix-copy 2022-08-22 15:29:10 +02:00
Eelco Dolstra
7c3ab57515 Merge pull request #6937 from edolstra/fix-repl
nix repl: Stop the progress bar
2022-08-22 15:10:01 +02:00
Naïm Favier
4c2ff4a0f4 JSON: add missing newlines
after `nix eval --json` and `nix-instantiate --eval --json`.
2022-08-22 15:07:52 +02:00
Naïm Favier
062e4fcdde JSON: print paths as strings without copying them to the store
Makes `printValueAsJSON` not copy paths to the store for `nix eval
--json`, `nix-instantiate --eval --json` and `nix-env --json`.

Fixes https://github.com/NixOS/nix/issues/5612
2022-08-22 15:01:35 +02:00
Eelco Dolstra
8d84634e26 Merge pull request #6926 from rapenne-s/download_limit
add a nix.conf option to set a download speed limit
2022-08-22 14:57:19 +02:00
Eelco Dolstra
590660c1c6 Merge pull request #6932 from jack1142/patch-1
Fix default profile path for root in nix profile documentation
2022-08-22 14:55:04 +02:00
Eelco Dolstra
0d2163c6dc nix repl: Stop the progress bar
The repl was broken since c3769c6846. In
general, the progress bar is incompatible with the repl.
2022-08-22 14:27:36 +02:00
Eelco Dolstra
9ddef9f0be Merge pull request #6923 from ncfavier/nix-env-incomplete-json
Avoid producing incomplete JSON
2022-08-22 14:15:31 +02:00
Eelco Dolstra
c21b1a7e67 Spelling 2022-08-22 14:14:14 +02:00
Valentin Gagarin
d8e54d19f7 Revert "Merge pull request #6420 from nix-community/doc-what-is-nix"
This reverts commit 81e101345f, reversing
changes made to 7d1280bbaf.
2022-08-22 12:52:15 +02:00
Eelco Dolstra
33aca20616 Merge pull request #6916 from edolstra/tmp-slash
Installer: Don't assume TMPDIR ends in a slash
2022-08-22 12:47:33 +02:00
Eelco Dolstra
380bf60c2e Merge pull request #2 from abathur/tmp-slash
install: only create $TMPDIR if missing
2022-08-22 12:07:38 +02:00
Solène Rapenne
caad87e6db Better documentation wording
Co-authored-by: Anderson Torres <torres.anderson.85@protonmail.com>
2022-08-20 18:21:36 +02:00
Jakub Kuczys
7d800909e9 Fix default profile path for root in nix profile documentation 2022-08-20 03:48:42 +02:00
Travis A. Everett
0bf52b73f4 install: only create TMPDIR if missing 2022-08-19 15:16:34 -05:00
Solene Rapenne
0d2bf7acf9 add a nix.conf option to set a download speed limit 2022-08-19 12:40:22 +02:00
Eelco Dolstra
d5e979ab87 Merge pull request #6925 from pennae/fod-register-memory
don't read outputs into memory for output rewriting
2022-08-19 12:22:34 +02:00
pennae
7d934f7880 don't read outputs into memory for output rewriting
RewritingSink can handle being fed input where a reference crosses a
chunk boundary. we don't need to load the whole source into memory, and
in fact *not* loading the whole source lets nix build FODs that do not
fit into memory (eg fetchurl'ing data files larger than system memory).
2022-08-19 11:26:26 +02:00
Naïm Favier
7535ee345d nix-env: don't output incomplete JSON 2022-08-19 01:00:10 +02:00
Naïm Favier
8188b1d0ab json: write null on abnormal placeholder destruction
Avoids leaving dangling attributes like

    {
        "foo":
    }

in case of exceptions.
2022-08-19 01:00:10 +02:00
Eelco Dolstra
823e1017d8 Ensure that $TMPDIR exists if defined 2022-08-17 21:47:01 +02:00
Eelco Dolstra
6547dcde2a Use plain mktemp
This fixes the case where $TMPDIR doesn't end in a slash.
2022-08-17 21:41:19 +02:00
Eelco Dolstra
7aa36ae796 Merge pull request #6917 from edolstra/no-force
Don't pass --force to 'git add'
2022-08-17 17:43:43 +02:00
Eelco Dolstra
c80a74b7d5 Don't pass --force to 'git add'
Fixes #5810.
2022-08-17 16:59:02 +02:00
Eelco Dolstra
84cc7ad77c Merge pull request #6913 from edolstra/lazy-trees-cherrypicks
lazy-trees cherrypicks
2022-08-17 12:20:19 +02:00
Eelco Dolstra
81e42e0d3f Fix onError 2022-08-17 11:36:58 +02:00
Eelco Dolstra
53e7b7e8ac Remove warnLargeDump()
This message was unhelpful (#1184) and probably misleading since
memory is O(1) in most cases now.
2022-08-17 11:32:01 +02:00
Eelco Dolstra
c3769c6846 ProgressBar: Delay before showing a new activity
Some activities are numerous but usually very short (e.g. copying a
source file to the store) which would cause a lot of flickering. So
only show activities that have been running for at least 10 ms.
2022-08-17 11:31:44 +02:00
Eelco Dolstra
e62160579f nix flake metadata: Don't show "Inputs" if there are no inputs 2022-08-17 11:31:34 +02:00
Eelco Dolstra
3d4489b623 Show when we're evaluating a flake 2022-08-17 11:31:27 +02:00
Valentin Gagarin
703b335c1d reword description of language properties
the list style is supposed to give more structure.
each property is explained as concisely as possible while trying not to
sound too fancy.
2022-08-15 11:34:39 +02:00
Eelco Dolstra
af4e8b00fb Merge pull request #6882 from DeterminateSystems/allow-multi-user-as-root
Allow installing multi-user as root on macOS and Linux
2022-08-12 10:19:03 +02:00
Théophane Hufschmitt
80a960940b Merge pull request #6896 from blaggacao/patch-1
docfix: bundlers
2022-08-11 10:15:55 +02:00
David Arnold
b58785ab83 docfix: bundlers 2022-08-10 18:49:29 -05:00
Théophane Hufschmitt
8f3fdef1e0 Merge pull request #6850 from NinjaTrappeur/nin/build-check
Fix Nix build --check flag
2022-08-10 17:44:06 +02:00
Graham Christensen
7bb1e913b3 Don't prompt about using sudo if we're already root 2022-08-10 09:37:08 -04:00
Graham Christensen
88a0f3b6ba Strip whitespace in installing-binary.md 2022-08-10 09:37:08 -04:00
Graham Christensen
64c3adbe1a install-multi-user: abstract is_root, is_os_linux, is_os_darwin 2022-08-10 09:36:45 -04:00
Alex Wied
0eb9946e1d docker.nix: Provide boolean for whether to bundle nixpkgs 2022-08-09 23:21:27 -04:00
Winter
5f37c5191a nix-shell: specify which outputs from bashInteractive to build 2022-08-09 22:54:22 -04:00
Adam Joseph
bac1e1bf8c Update doc/manual/src/glossary.md
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2022-08-08 17:52:31 +00:00
Travis A. Everett
9b7bd2dd1f Apply suggestions from code review
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2022-08-08 10:04:27 -05:00
Théophane Hufschmitt
73fde9eed0 Merge pull request #6280 from thufschmitt/fix-mv-in-different-filesystems
Fix mv in different filesystems
2022-08-08 16:48:17 +02:00
Théophane Hufschmitt
5192bb093a Merge remote-tracking branch 'upstream/master' into fix-mv-in-different-filesystems 2022-08-08 15:42:56 +02:00
Théophane Hufschmitt
f4a8426098 Remove the explicit c++fs linkage on darwin
Doesn't seem needed on a recent-enough clang anymore (and even seems to
break stuff)
2022-08-08 14:34:22 +02:00
Eelco Dolstra
3a09a32b27 Merge pull request #6881 from thomasjcf29/feature/selinux_check_properly
Only fail if selinux is in enforcing
2022-08-08 14:03:37 +02:00
Théophane Hufschmitt
380fff1049 Merge pull request #6880 from virusdave/dnicponski/scratch/swap_homedir_check_master
Do not spam logs if the owned-homedir check results in a noop
2022-08-08 13:19:49 +02:00
Tom Franklin
96b4339d86 Only fail if selinux is in enforcing 2022-08-08 00:42:18 +01:00
Dave Nicponski
cb6794a0d9 Do not spam logs if the owned-homedir check results in a noop 2022-08-07 10:13:11 -04:00
Travis A. Everett
b5d85f0922 Apply suggestions from code review
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2022-08-05 13:49:18 -05:00
Adam Joseph
1d3b92e80c move package-management/terminology into glossary.md 2022-08-05 10:39:43 -07:00
Adam Joseph
4de95f7f56 gesture at explanation of why binary caches exist 2022-08-05 10:33:48 -07:00
Adam Joseph
d5506aa712 Update doc/manual/src/package-management/terminology.md
Co-authored-by: Attila Gulyas <toraritte@gmail.com>
2022-08-05 17:19:52 +00:00
Adam Joseph
66a93a76b9 Update doc/manual/src/package-management/terminology.md
Co-authored-by: Attila Gulyas <toraritte@gmail.com>
2022-08-05 17:15:37 +00:00
Adam Joseph
2eb74c918d derivations do not need to be signed 2022-08-05 10:13:41 -07:00
Adam Joseph
aae771cad2 !implement https://github.com/NixOS/nix/pull/6870#discussion_r938912244 2022-08-05 10:12:46 -07:00
Eelco Dolstra
6776e65fd9 Merge pull request #6872 from edolstra/architecture-section
Comment out the architecture section
2022-08-05 11:55:25 +02:00
Eelco Dolstra
4eb566603a Comment out the architecture section
This needs more review and probably should be a separate book.
2022-08-05 11:01:40 +02:00
Adam Joseph
1b97f3872e !fixup: transposed characters 2022-08-04 14:22:14 -07:00
Adam Joseph
62674659ed !fixup capitalize Nix 2022-08-04 14:21:17 -07:00
Adam Joseph
8f44d24c52 !fixup whitespace 2022-08-04 14:19:25 -07:00
Adam Joseph
56d4fc194b fourth trust condition: FODs 2022-08-04 14:14:24 -07:00
Adam Joseph
a5be5e0120 doc/manual: define {local,remote} store, binary cache, substituter
Nix veterans intuitively know what the following terms mean.  They are
used in several places in the nix documentation, but never defined:

- local store
- remote store
- binary cache
- substituter

In particular, I found the last two terms to be confusingly similar.
Let's give definitions for them.
2022-08-04 14:07:06 -07:00
Théophane Hufschmitt
81e101345f Merge pull request #6420 from nix-community/doc-what-is-nix
Document what Nix *is*
2022-08-04 20:49:01 +02:00
John Ericson
39d32ac4c6 Add disclaimer that arch section is WIP and links may rot
So we can iterate without worrying so much.
2022-08-04 14:02:58 -04:00
John Ericson
6f6498f59c Remove header fragments which is not needd 2022-08-04 14:02:38 -04:00
John Ericson
016d7a8f78 Fix rosetta stone file name 2022-08-04 14:02:19 -04:00
John Ericson
b430a6743c Remove sections within from SUMMARY 2022-08-04 14:01:55 -04:00
John Ericson
b74a3f51c2 Fix gitignore 2022-08-04 14:01:27 -04:00
John Ericson
bc118854d0 Merge remote-tracking branch 'upstream/master' into doc-what-is-nix 2022-08-04 09:11:42 -04:00
Valentin Gagarin
b631742277 fix page rendering
apparently it is not possible to link to page anchors with `mdBook`[1]

[1]: https://github.com/rust-lang/mdBook/issues/167
2022-08-04 14:31:33 +02:00
Domen Kožar
7d1280bbaf Merge pull request #6678 from abathur/update_darwin_uninstall
doc: update darwin uninstall instructions
2022-08-04 14:09:09 +02:00
Valentin Gagarin
cc3a5f4ba2 use correct mdBook syntax for callouts 2022-08-04 14:04:14 +02:00
Valentin Gagarin
8cec32e7f5 fix directory tree renderings 2022-08-04 13:51:35 +02:00
Eelco Dolstra
6fa2e8cb6f Merge pull request #6805 from jfly/jfly/fix-link-to-hacking-doc
Fix link to hacking doc
2022-08-04 13:35:52 +02:00
Eelco Dolstra
c9b24f0c17 Merge pull request #6863 from fricklerhandwerk/remove-writing-expressions
manual: remove "Writing Nix Expressions" chapter
2022-08-04 13:25:34 +02:00
Domen Kožar
84a26882f8 Update doc/manual/src/installation/installing-binary.md
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2022-08-04 06:23:45 -05:00
Eelco Dolstra
bc5907ff94 Merge pull request #6862 from erikarvstedt/fix-flake-update-doc
docs/flake-update: fix example
2022-08-04 13:23:45 +02:00
Domen Kožar
3c2de2da01 Update doc/manual/src/installation/installing-binary.md
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2022-08-04 06:23:36 -05:00
Valentin Gagarin
3df1ee2ba5 clarify what explicitly declaring certain dependencies means 2022-08-04 12:37:48 +02:00
Valentin Gagarin
b7309ce624 move architecture to the end 2022-08-04 12:37:48 +02:00
Valentin Gagarin
00a7eaed67 add file system object to table of contents 2022-08-04 12:37:48 +02:00
Valentin Gagarin
db6faf44a9 clarify what store objects can be
the store of course makes a distinction, but that is not relevant here
2022-08-04 12:37:48 +02:00
Valentin Gagarin
0228eb8ae7 add Java example on manual dependency declaration 2022-08-04 12:37:48 +02:00
Valentin Gagarin
29c062537d hashes: truncate -> reduce, mention SHA-256
the longer SHA-256 hashes are not truncated, but in fact processed.

Co-authored-by: Thomas <twatson52@mac.com>
2022-08-04 12:37:48 +02:00
Valentin Gagarin
9cabba1fc3 mention hard links
Co-authored-by: Thomas <twatson52@mac.com>
2022-08-04 12:37:48 +02:00
Valentin Gagarin
bac86231ad use "build plan" consistently 2022-08-04 12:37:47 +02:00
Valentin Gagarin
5a5a95696f note customized base32
Nix omits E O U T characters for some reason.
2022-08-04 12:37:47 +02:00
Valentin Gagarin
de5dea45f7 use correct Nix entity
Nix itself does care a lot about what type of store object you have.
2022-08-04 12:37:47 +02:00
Valentin Gagarin
fe4c0b8d75 fix typo
Co-authored-by: Thomas <twatson52@mac.com>
2022-08-04 12:37:47 +02:00
Valentin Gagarin
def80d5777 add subsections to table of contents 2022-08-04 12:37:47 +02:00
Valentin Gagarin
c345345dee concept map: align hights
this has the weird but nice emergent property that terms at the same
height are roughly at the same level of abstraction.
2022-08-04 12:37:47 +02:00
Valentin Gagarin
a28d6878b2 concept map: put closure as it is in the chapter
this is not as compact any more, but it more closely resembles the
chapter structure, and clearly shows that the closure property is the
key idea on which most of Nix operates.
2022-08-04 12:37:47 +02:00
Valentin Gagarin
475a332025 make concept map even more compact 2022-08-04 12:37:47 +02:00
Valentin Gagarin
d8b2f9f772 make concept map more compact
invert arrows to/from derivation:
- we need closures to form derivations
- we need derivations to perform builds
2022-08-04 12:37:47 +02:00
Valentin Gagarin
348432f48f store: add concept map
this should help nativate the chapter by indicating which terms should
be known to understand a given concept.
2022-08-04 12:37:47 +02:00
Valentin Gagarin
ec43977553 store: match chapter introduction to outline
the various levels of detail should describe the same things.
2022-08-04 12:37:47 +02:00
Valentin Gagarin
7c656d90ca simplify description of diagram
try not to be too fancy, it's just for reading the diagram out loud.
2022-08-04 12:37:47 +02:00
Valentin Gagarin
631ca18ee6 reword notes on copying
clarify that we are copying between different stores. we have not
introduced that notion or why it would be interesting, but for now it
should be fine to keep it in context of the store directory.

we could move that later to a more detailed explanation of different
store types.
2022-08-04 12:37:47 +02:00
Valentin Gagarin
c10dcccccd make example a simple list
using JSON notation is unwarranted and not explained.
2022-08-04 12:37:47 +02:00
Valentin Gagarin
9c544813d7 paths -> path
use singular for terminology uniformly
2022-08-04 12:37:47 +02:00
Valentin Gagarin
1681f4e9f3 better explain reference scanning 2022-08-04 12:37:47 +02:00
Valentin Gagarin
fa7ad4593d explain store directory 2022-08-04 12:37:47 +02:00
Valentin Gagarin
f632816cba add explanation and examples of file system objects 2022-08-04 12:37:47 +02:00
Valentin Gagarin
84ddfbf8fd remove diagonal from operations diagram 2022-08-04 12:37:47 +02:00
Valentin Gagarin
93f721b0d2 remove draft on derivations 2022-08-04 12:37:47 +02:00
Valentin Gagarin
19d8a5d839 move first mention of file system object before diagram 2022-08-04 12:37:47 +02:00
Valentin Gagarin
a90fc62b61 make clear that file system is for processes
Co-authored-by: John Ericson <John.Ericson@Obsidian.Systems>
2022-08-04 12:37:47 +02:00
Valentin Gagarin
7993ba1f38 constrain garbage collection scope
garbage collection is now incremental, and may (in theory) never delete all unreferenced objects if it is slow enough.
2022-08-04 12:37:47 +02:00
Valentin Gagarin
195aa28ff7 references are added according to build task
at this level of abstraction we do not really care about build instructions or what they are, and also build instructions including their arguments really amount to the build task.
2022-08-04 12:37:47 +02:00
Valentin Gagarin
80de4a4804 operations diagram: store' to the right
this also looks more diverse, hopefully easier to distinguish

Co-authored-by: John Ericson <John.Ericson@Obsidian.Systems>
2022-08-04 12:37:47 +02:00
Valentin Gagarin
3794618cce add commas between output values 2022-08-04 12:37:47 +02:00
Valentin Gagarin
7b7e4c6340 use singular to match section heading 2022-08-04 12:37:47 +02:00
Valentin Gagarin
e63a768e21 use reference links for URLs 2022-08-04 12:37:47 +02:00
Valentin Gagarin
843288a451 add subsections for objects and references
group description of data instead of spreading it across the section.

that should help direct skimming. as it turns out, people do not
actually read any of that.
2022-08-04 12:37:47 +02:00
Valentin Gagarin
4341849193 move closure property to discussion references 2022-08-04 12:37:47 +02:00
Valentin Gagarin
445f753a82 replace pseudo code by diagrams
change prose description to visually resemble the data structure
2022-08-04 12:37:47 +02:00
Valentin Gagarin
db8703bcac use reference links for URLs 2022-08-04 12:37:47 +02:00
Valentin Gagarin
4adb6602bd clarify first sentence on store objects 2022-08-04 12:37:47 +02:00
Valentin Gagarin
4eb11d4592 fix grammar for clarity 2022-08-04 12:37:47 +02:00
Valentin Gagarin
b84f2bdfdd introduce mapping to Unix files and processes 2022-08-04 12:37:47 +02:00
Valentin Gagarin
207992a71d introduce store and store objects without file system details
this leaves open implementation details, especially about store paths
and file system objects, and allows explaining them together were it is
more appropriate. also leaves room to carefully introduce the key
insight behind Nix: applying results from programming language theory to
the operating system paradigm of files and processes.
2022-08-04 12:37:47 +02:00
Valentin Gagarin
e72a7874dc beautify rosetta table
while this may eventually introduce ugly diffs, the table will now
render readably on the terminal (e.g. for `man nix` or `nix --help`)
without further intervention.
2022-08-04 12:37:47 +02:00
Valentin Gagarin
b6b112b366 use reference links for URLs 2022-08-04 12:37:47 +02:00
Valentin Gagarin
d5eea66615 introduce build tasks
while it appears a bit much for the overview, this way we set the stage
for going directly into data types when describing the store, instead of
first having to say what build tasks are and how they relate to build
plans.
2022-08-04 12:37:47 +02:00
Valentin Gagarin
7c3bca1372 revert to build plans in top-level overview
do not introduce build tasks yet, that is the next level of detail.
2022-08-04 12:37:47 +02:00
Valentin Gagarin
4639b36b53 use reference links for URLs 2022-08-04 12:37:47 +02:00
Valentin Gagarin
2303f84a68 revert to "build plan" in overview diagram
this displays correct composition again. build inputs and build results
are not part of build plans in terms of data objects.

also this is a much less complicated setup. this will be the first
impression of architecture, and we want to get it right.
2022-08-04 12:37:47 +02:00
Valentin Gagarin
25926c5fc6 Nix store does not underly literally everything
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2022-08-04 12:37:47 +02:00
Valentin Gagarin
0e63b9bf88 add link from overview to store section
the overview should only link to the three main concepts presented. the
store is now fairly fleshed out. others can follow later.
2022-08-04 12:37:47 +02:00
Valentin Gagarin
ef81276cc1 architecture overview: add link to command line reference 2022-08-04 12:37:47 +02:00
Valentin Gagarin
68d26010f6 architecture overview: add link to Nix expression language reference
update summary title to match file contents
2022-08-04 12:37:47 +02:00
Valentin Gagarin
75ce324052 use singular for class names consistently 2022-08-04 12:37:47 +02:00
Valentin Gagarin
689b32a543 clarify relation of tasks and plans 2022-08-04 12:37:47 +02:00
Valentin Gagarin
2a8532fb61 build rule -> build task
closer to "build systems a la carte", satisfies all other complaints
2022-08-04 12:37:47 +02:00
Valentin Gagarin
902638c519 build step -> build rule
"step" sounds atomic, while "rule" hints at internal structure, which in
our case consists of mapping inputs to outputs using build instructions.
2022-08-04 12:37:47 +02:00
Valentin Gagarin
87523f01e3 match grammatical case to arrow direction 2022-08-04 12:37:47 +02:00
Valentin Gagarin
d3effd014b update architecture diagram 2022-08-04 12:37:47 +02:00
Valentin Gagarin
ad8c2ed7f0 remove incomplete section: input/content-addressing 2022-08-04 12:37:47 +02:00
Valentin Gagarin
3bd125ebbe remove incomplete section: nix archives 2022-08-04 12:37:47 +02:00
Valentin Gagarin
b18852eb3f remove incomplete section: content-addressed objects 2022-08-04 12:37:47 +02:00
Valentin Gagarin
7cec9ee361 remove incomplete section: relocatability 2022-08-04 12:37:47 +02:00
Valentin Gagarin
96876b1eae remove incomplete section: related work 2022-08-04 12:37:47 +02:00
Valentin Gagarin
1ba6d8fb1d remove incomplete section: building 2022-08-04 12:37:47 +02:00
John Ericson
3d8f2f5cc1 Fix manual TOC links 2022-08-04 12:37:47 +02:00
John Ericson
b01bb65d30 Fix rel path in doc 2022-08-04 12:37:47 +02:00
Valentin Gagarin
b5ca3d12b6 reword details on keeping closure property 2022-08-04 12:37:47 +02:00
Valentin Gagarin
e90586c0a4 add motivation for references 2022-08-04 12:37:47 +02:00
Valentin Gagarin
07d490fd89 stores can also delete objects 2022-08-04 12:37:47 +02:00
Valentin Gagarin
5fda995491 formalize file system objects
convention: describe every data type in prose, and illustrate with
a class diagram, and a textual representation of an abstract
data type.

right now we save ourselves the trouble of doing class diagrams, we can
add them later. but they are important.
2022-08-04 12:37:47 +02:00
Valentin Gagarin
fb2ec7e4ec reword section on references
use file Contents instead of Data, as that flows more naturally in the
prose.

simplify explanation of the idea behind scanning for store paths

remove references to unfinished sections.
2022-08-04 12:37:47 +02:00
Valentin Gagarin
90fc5b41a8 reword file system objects
- use singular for the "class"
- more consistency in type definition
- minor fixes in wording
2022-08-04 12:37:47 +02:00
Valentin Gagarin
e5e48593c8 move git comparison to related work
it should be pulled out of the branch before we go for merging
2022-08-04 12:37:47 +02:00
Valentin Gagarin
a145007a57 component -> store object, realisation -> build 2022-08-04 12:37:47 +02:00
Valentin Gagarin
40efe5b30b build instrcution: Task -> function 2022-08-04 12:37:47 +02:00
Valentin Gagarin
ca5ebf6382 revert build plan/step distinction, reorder rows 2022-08-04 12:37:47 +02:00
Valentin Gagarin
610ddf44aa reword introduction to rosetta stone, add links
attempt to explain used and documented terminology, as well as how
the declarative programming paradigm relates to building software.

in the future one could highlight encouraged terms to shape future
material into higher consistency.
2022-08-04 12:37:47 +02:00
Valentin Gagarin
5f96a0b4e8 associated operations are not collected
Co-authored-by: John Ericson <git@JohnEricson.me>
2022-08-04 12:37:47 +02:00
Valentin Gagarin
070c85499b fix grammar
Co-authored-by: John Ericson <git@JohnEricson.me>
2022-08-04 12:37:47 +02:00
Valentin Gagarin
7b5c00f67f add concrete store examples, reword note on file system 2022-08-04 12:37:47 +02:00
Valentin Gagarin
c8c1b705ad reword section on Nix store 2022-08-04 12:37:47 +02:00
Valentin Gagarin
39f01176a7 design -> architecture, add motivation
following ideas found in Architecture of Gazelle[1]

[1]: 56d35f8db0/Design.rst
2022-08-04 12:37:47 +02:00
Valentin Gagarin
d30033759a address Nix language consistently as configuration language 2022-08-04 12:37:46 +02:00
Valentin Gagarin
7598126391 remove separate meta-section, add architecture diagram
the diagram is a first approximation and only covers that same section.
of course there is much more going on, and other features should at some
point also be illustrated.

we also have to think about presentation format and technicalities
behind it. the manual has to render to `man`, but we may want something
more refined for web view.
2022-08-04 12:37:46 +02:00
Valentin Gagarin
34ea74c9ec reword introductory section
there should be a meta section for each chapter to give motivation of
the presented structure. the structure itself is visible from the table
of contents.
2022-08-04 12:37:46 +02:00
Valentin Gagarin
b387d80943 remove sentence for chapter transition
idea: sections could be read in different orders by linking them in
different ways (e.g. depth-first or breadth-first). adding hard-coded
transitions makes that confusing.
2022-08-04 12:37:46 +02:00
John Ericson
89a7c956ff Apply suggestions from code review
Co-authored-by: Valentin Gagarin <valentin@fricklerhandwerk.de>
2022-08-04 12:37:46 +02:00
Valentin Gagarin
51e6bed25e do not mention implementation details
Co-authored-by: John Ericson <git@JohnEricson.me>
2022-08-04 12:37:46 +02:00
Valentin Gagarin
23ee0b24f7 correctly use comma for nesting
Co-authored-by: John Ericson <git@JohnEricson.me>
2022-08-04 12:37:46 +02:00
Valentin Gagarin
804e8bd747 indicate sequence with "then"
Co-authored-by: John Ericson <git@JohnEricson.me>
2022-08-04 12:37:46 +02:00
Valentin Gagarin
327ccd3b07 only use generic build system terminology
we will use a translation table to introduce nix-specific terms
2022-08-04 12:37:46 +02:00
Valentin Gagarin
0eae4bfad1 reword overview with clear terminology
trying to capture alternative terms in one go here, mirroring everyday
use:

derivation - build plan
realise - execute build

there will be more of that sort.
2022-08-04 12:37:46 +02:00
John Ericson
0737094161 Add draft "Rosetta stone" by @fricklerhandwerk and stub commentary
The idea and most of the execution are @fricklerhandwerk's. I changed a
few things best I could based on @edolstra's corrections, and a Bazel
glossary.

Valentin Gagarin <valentin@fricklerhandwerk.de>
2022-08-04 12:37:46 +02:00
John Ericson
c86c1ec7e3 Make refernces sneak preview more concise 2022-08-04 12:37:46 +02:00
John Ericson
4e4bbd9e83 Improve store objects session more 2022-08-04 12:37:46 +02:00
John Ericson
e4eea5e84e Include abstract syntax based on the thesis for FSOs
See https://edolstra.github.io/pubs/phd-thesis.pdf, page 91.
2022-08-04 12:37:46 +02:00
John Ericson
b98dc3b19c store objects, better opining sentances 2022-08-04 12:37:46 +02:00
John Ericson
55b437b551 Improve store path section 2022-08-04 12:37:46 +02:00
John Ericson
b4df351880 Relocability -> relocation in store object title 2022-08-04 12:37:46 +02:00
John Ericson
5f4d2ac091 Improve store object section
In particular, Nix is *not* like Git, so that needs to be fixed.
2022-08-04 12:37:46 +02:00
John Ericson
1bbad62c7d doc: File system data -> file system object, to match Nix 2022-08-04 12:37:46 +02:00
John Ericson
838ba26fda Rename files after store entry -> store object rename 2022-08-04 12:37:46 +02:00
John Ericson
4e2d5ae202 doc: Store entry -> store object
This matches the terminology in Eelco's thesis.
2022-08-04 12:37:46 +02:00
John Ericson
e308602fdf Update doc/manual/src/design/store/drvs/drvs.md
Co-authored-by: Matthieu Coudron <teto@users.noreply.github.com>
2022-08-04 12:37:46 +02:00
John Ericson
cdb0bf3b65 Update doc/manual/src/design/overview.md
Co-authored-by: Valentin Gagarin <valentin@fricklerhandwerk.de>
2022-08-04 12:37:46 +02:00
John Ericson
75c5191a1f Update doc/manual/src/design/overview.md
Co-authored-by: Valentin Gagarin <valentin@fricklerhandwerk.de>
2022-08-04 12:37:46 +02:00
John Ericson
a04340f9a1 Update doc/manual/src/design/overview.md
Co-authored-by: Valentin Gagarin <valentin@fricklerhandwerk.de>
2022-08-04 12:37:46 +02:00
John Ericson
f5386d7059 Fix stub file's name 2022-08-04 12:37:46 +02:00
John Ericson
678d75baea Start on the derivations section 2022-08-04 12:37:46 +02:00
John Ericson
e3a0209a9e Move the bits on relocating store entires to the end
They are too advanced for up front.
2022-08-04 12:37:46 +02:00
John Ericson
a210504bc7 Apply suggestions from code review 2022-08-04 12:37:46 +02:00
John Ericson
e64633f98f Flesh out TOC 2022-08-04 12:37:46 +02:00
John Ericson
a2b3160f28 Briefly describe the digest of a store path 2022-08-04 12:37:46 +02:00
John Ericson
523359d133 WIP: Document the design of Nix
The current docs are all "how to do things" and no "what is Nix" or "why
are things the way they are".

I see lots of misconception on the wider internet, and I also think we
would benefit from a "living document" to answer some questions people
currently turn to the thesis for.

I think a new section of the manual can address all these issues.
2022-08-04 12:37:46 +02:00
Valentin Gagarin
499ed26508 manual: remove "Writing Nix Expressions" chapter
it is out of date, all over the place in level of detail, is really
about `nixpkgs`, and in general instructions should not be part of
a reference manual.

also:
- update redirects and internal links
- use "Nix language" consistently
2022-08-04 11:59:25 +02:00
Erik Arvstedt
4c8441be0a docs/flake-update: fix example 2022-08-04 09:45:30 +02:00
Erik Arvstedt
53833dfb40 libexpr/flake: remove FIXME
Line 593 checks that all overrides (i.e. all elements of
`lockFlags.inputOverrides`) are members of `overridesUsed`.
2022-08-04 09:45:29 +02:00
Erik Arvstedt
c9f446ede1 flakeref: fix comment 2022-08-04 09:45:28 +02:00
Eelco Dolstra
075bf6e556 Merge pull request #6861 from edolstra/fix-count-calls
Fix NIX_COUNT_CALLS=1
2022-08-03 18:26:01 +02:00
Eelco Dolstra
ccbd906c86 Fix NIX_COUNT_CALLS=1
Also, make the JSON writer support std::string_view.

Fixes #6857.
2022-08-03 17:46:51 +02:00
Théophane Hufschmitt
7d1ccd9105 Merge pull request #6846 from fricklerhandwerk/values
manual: use subheadings for primitive types
2022-08-03 14:50:41 +02:00
Théophane Hufschmitt
c55bea4204 Fix the html id of the list headers 2022-08-03 14:16:00 +02:00
Félix Baylac-Jacqué
b47b6a418d tests/check.sh: add nix3-build check test 2022-08-03 12:17:13 +02:00
Valentin Gagarin
ceed4d4142 encode primitive as list with anchors
to make it consistent with builtins and configuration options
2022-08-03 11:25:41 +02:00
Théophane Hufschmitt
d1cda07ce4 Don't use -load_all on darwin
That flag breaks `-lc++fs` (introducing a duplicate symbol for some
reason). Besides, it was apparently needed for bzip2, but we're not using bzip2
anymore.
2022-08-03 10:27:25 +02:00
Théophane Hufschmitt
1ba5b3e001 Make moveFile more atomic
Rather than directly copying the source to its dest, copy it first to a
temporary location, and eventually move that temporary.
That way, the move is at least atomic from the point-of-view of the destination
2022-08-03 10:27:25 +02:00
Théophane Hufschmitt
90f9680733 Only use renameFile where needed
In most places the fallback to copying isn’t needed and can actually be
bad, so we’d rather not transparently fallback
2022-08-03 10:27:25 +02:00
Théophane Hufschmitt
d71d9e9fbf moveFile -> renameFile
`move` tends to have this `mv` connotation of “I will copy it for you if
needs be”
2022-08-03 10:27:25 +02:00
Théophane Hufschmitt
a4f0fd633c Link against c++fs on darwin
Required by the old clang version
2022-08-03 10:27:25 +02:00
Théophane Hufschmitt
c5db1821a9 Re-implement the recursive directory copy
The recursive copy from the stl doesn’t exactly do what we need because
1. It doesn’t delete things as we go
2. It doesn’t keep the mtime, which change the nars

So re-implement it ourselves. A bit dull, but that way we have what we want
2022-08-03 10:27:25 +02:00
Théophane Hufschmitt
6f89fb6008 rename: Fallback to a copy if the filesystems mismatch
In `nix::rename`, if the call to `rename` fails with `EXDEV` (failure
because the source and the destination are in a different filesystems)
switch to copying and removing the source.

To avoid having to re-implement the copy manually, I switched the
function to use the c++17 `filesystem` library (which has a `copy`
function that should do what we want).

Fix #6262
2022-08-03 10:27:25 +02:00
Théophane Hufschmitt
c2de0a232c Create a wrapper around stdlib’s rename
Directly takes some c++ strings, and gently throws an exception on error
(rather than having to inline this logic everywhere)
2022-08-03 10:27:25 +02:00
Théophane Hufschmitt
8119390abc Move some fs-related functions to their own file
Unclutter `util.cc` a bit
2022-08-03 10:27:25 +02:00
Théophane Hufschmitt
780a479386 Merge pull request #6851 from K900/patch-1
doc/distributed-builds: don't use deprecated alias
2022-08-02 07:17:09 +02:00
K900
f675ba5331 doc/distributed-builds: don't use deprecated alias
`nix ping-store` -> `nix store ping`.
2022-08-01 13:50:35 +03:00
Félix Baylac-Jacqué
1467a98d4c derivation-goal.cc: remove bmCheck custom return branch on buildDone
Once a derivation goal has been completed, we check whether or not
this goal was meant to be repeated to check its output.

An early return branch was preventing the worker to reach that repeat
code branch, hence breaking the --check command (#2619).

It seems like this early return branch is an artifact of a passed
refactoring. As far as I can tell, buildDone's main branch also
cleanup the tmp directory before returning.
2022-08-01 11:39:19 +02:00
Jeremy Fleischman
297f6b5d56 Fix link to hacking doc
Right now,
https://hydra.nixos.org/job/nix/master/build.x86_64-linux/latest/download-by-type/doc/manual/contributing/hacking.html
redirects to
https://hydra.nixos.org/build/183877779/download/1/manual/contributing/hacking.html,
which gives me a "500 Internal Server Error". Not super useful =(

Feel free to ignore if someone's working to fix the 500 I was running
into.
2022-07-30 09:12:50 -07:00
Rok Garbas
e9178d7d4a Merge pull request #6844 from centromere/custom-nix-conf
docker.nix: Allow Nix configuration to be customized
2022-07-29 13:47:24 +02:00
Valentin Gagarin
27138f1ec6 manual: use singular in body, too
Co-authored-by: Cole Helbling <cole.e.helbling@outlook.com>
2022-07-28 23:30:07 +02:00
Valentin Gagarin
41a3b315fd manual: values -> data types 2022-07-28 17:25:25 +02:00
Valentin Gagarin
8f4fab8fab manual: use singular for headings 2022-07-28 17:25:18 +02:00
Valentin Gagarin
4ff48854b8 manual: simple values -> primitives
"simple" is a loaded term
2022-07-28 17:23:57 +02:00
Valentin Gagarin
3063e5b94c manual: use subheadings for primitive types
this gives us HTML anchors for each of them
2022-07-28 17:23:57 +02:00
Eelco Dolstra
86fcd4f692 Merge pull request #6845 from fricklerhandwerk/attrset
manual: set -> attribute set
2022-07-28 16:55:03 +02:00
Valentin Gagarin
85cdaebcd6 manual: set -> attribute set
reword description to have shorter sentences.
2022-07-28 16:10:24 +02:00
Valentin Gagarin
be4654c344 manual: fix section title in table of contents 2022-07-28 15:55:41 +02:00
Alex Wied
228028fc1a docker.nix: Allow Nix configuration to be customized 2022-07-28 03:36:39 -04:00
Théophane Hufschmitt
2805439335 Merge pull request #6814 from amjoseph-nixpkgs/pr/sandbox-error-messages
local-derivation-goal.cc: improve error messages when sandboxing fails
2022-07-22 13:27:52 +02:00
Théophane Hufschmitt
e10807cdbb Merge pull request #6813 from centromere/cgroup-cpu-detection
libstore/globals.cc: Automatically set cores based on cgroup CPU limit
2022-07-22 10:15:32 +02:00
Solène Rapenne
92bae33ca5 nix shell: example shouldn't use an absolute path for the shell
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2022-07-21 14:25:07 +02:00
Solène Rapenne
64404220f5 nix shell: document how to invoke multiple commands from the command line 2022-07-20 14:54:46 +02:00
Théophane Hufschmitt
7ed91d6c6a Merge branch 'master' into parallel-nix-copy 2022-07-20 10:05:34 +02:00
Alex Wied
722de8ddcc libstore/globals.cc: Move cgroup detection to libutil 2022-07-19 16:25:53 -04:00
Alex Wied
1af5d798a4 libstore/globals.cc: Automatically set cores based on cgroup CPU limit
By default, Nix sets the "cores" setting to the number of CPUs which are
physically present on the machine. If cgroups are used to limit the CPU
and memory consumption of a large Nix build, the OOM killer may be
invoked.

For example, consider a GitLab CI pipeline which builds a large software
package. The GitLab runner spawns a container whose CPU is limited to 4
cores and whose memory is limited to 16 GiB. If the underlying machine
has 64 cores, Nix will invoke the build with -j64. In many cases, that
level of parallelism will invoke the OOM killer and the build will
completely fail.

This change sets the default value of "cores" to be
ceil(cpu_quota / cpu_period), with a fallback to
std::thread::hardware_concurrency() if cgroups v2 is not detected.
2022-07-19 16:03:58 -04:00
Théophane Hufschmitt
56f6f3725f Don't ultimately trust the signed paths
Like the old implem did (and like you'd want it to be anyways)
2022-07-19 19:46:00 +02:00
Adam Joseph
36e1383b6b local-derivation-goal.cc: save global errno to the stack before performing tests which might clobber it 2022-07-19 03:53:20 -07:00
Adam Joseph
a9e75eca00 error.hh: add additional constructor with explicit errno argument 2022-07-19 03:49:33 -07:00
Adam Joseph
99fcc91f67 as requested by @thufschmitt https://github.com/NixOS/nix/pull/6814#discussion_r924275777 2022-07-19 03:33:12 -07:00
Adam Joseph
5f51539f88 change warn() to notice() 2022-07-19 03:30:52 -07:00
Théophane Hufschmitt
fbd0a6c6e2 Merge pull request #6784 from tweag/completion-test
Add some tests for the CLI completion
2022-07-18 20:32:14 +02:00
Eelco Dolstra
2584c151bd Merge pull request #6812 from lovesegfault/rosetta-paths
fix(libstore): allow Nix to access all Rosetta 2 paths on MacOS
2022-07-18 14:09:54 +02:00
Adam Joseph
c8c6203c2c local-derivation-goal.cc: detect unprivileged_userns_clone failure mode
The workaround for "Some distros patch Linux" mentioned in
local-derivation-goal.cc will not help in the `--option
sandbox-fallback false` case.  To provide the user more helpful
guidance on how to get the sandbox working, let's check to see if the
`/proc` node created by the aforementioned patch is present and
configured in a way that will cause us problems.  If so, give the user
a suggestion for how to troubleshoot the problem.
2022-07-17 01:27:22 -07:00
Adam Joseph
6fc56318bf local-derivation-goal.cc: add comment re: CLONE_NEWUSER
local-derivation-goal.cc contains a comment stating that "Some distros
patch Linux to not allow unprivileged user namespaces."  Let's give a
pointer to a common version of this patch for those who want more
details about this failure mode.
2022-07-17 01:23:32 -07:00
Adam Joseph
8d35f387dc local-derivation-goal.cc: warn if failing and /proc/self/ns/user missing
This commit causes nix to `warn()` if sandbox setup has failed and
`/proc/self/ns/user` does not exist.  This is usually a sign that the
kernel was compiled without `CONFIG_USER_NS=y`, which is required for
sandboxing.
2022-07-16 19:37:27 -07:00
Adam Joseph
90830b1074 local-derivation-goal.cc: warn if failing due to max_user_namespaces==0
This commit uses `warn()` to notify the user if sandbox setup fails
with errno==EPERM and /proc/sys/user/max_user_namespaces is missing or
zero, since that is at least part of the reason why sandbox setup
failed.

Note that `echo -n 0 > /proc/sys/user/max_user_namespaces` or
equivalent at boot time has been the recommended mitigation for
several Linux LPE vulnerabilities over the past few years.  Many users
have applied this mitigation and then forgotten that they have done
so.
2022-07-16 19:30:53 -07:00
Adam Joseph
8ea3a911aa local-derivation-goal.cc: improve error messages when sandboxing fails
The failure modes for nix's sandboxing setup are pretty complicated.
When nix is unable to set up the sandbox, let's provide more detail
about what went wrong.  Specifically:

* Make sure the error message includes the word "sandbox" so the user
  knows that the failure was related to sandboxing.

* If `--option sandbox-fallback false` was provided, and removing it
  would have allowed further attempts to make progress, let the user
  know.
2022-07-16 14:56:24 -07:00
Alex Wied
b88fb50e21 fix(libstore): allow Nix to access all Rosetta 2 paths on MacOS
Fixes: #5884
2022-07-15 12:10:56 -07:00
Eelco Dolstra
59764eb842 Merge pull request #6810 from jfly/jfly/do-not-assume-savedvars-exist
nix develop: do not assume that saved vars are set
2022-07-15 13:59:25 +02:00
Eelco Dolstra
0621e99414 Merge pull request #6811 from edolstra/fix-auto-chroot
Disable auto-chroot if $NIX_STATE_DIR is set
2022-07-15 13:11:08 +02:00
Eelco Dolstra
3bcd7a5474 Disable auto-chroot if $NIX_STATE_DIR is set
Issue #6732.
2022-07-15 12:32:29 +02:00
Jeremy Fleischman
04386f7d69 nix develop: do not assume that saved vars are set
This fixes https://github.com/NixOS/nix/issues/6809
2022-07-14 23:25:39 -07:00
Domen Kožar
de287964d5 Merge pull request #6807 from NixOS/curl-patch
curl: patch for netrc regression in Nix
2022-07-14 19:30:14 -05:00
Domen Kožar
99208bb8cc curl: patch for netrc regression in Nix 2022-07-14 17:45:02 -05:00
Eelco Dolstra
ca4d5bee09 Merge pull request #6804 from edolstra/fix-auto-chroot
Disable auto-chroot if $NIX_STORE_DIR is set
2022-07-14 18:24:08 +02:00
Eelco Dolstra
ff49c75502 Disable auto-chroot if $NIX_STORE_DIR is set
Fixes #6732.
2022-07-14 17:47:09 +02:00
Eelco Dolstra
73ff9b863c Merge pull request #6803 from edolstra/test-stack-trace
On test failures, print a bash stack trace
2022-07-14 15:56:06 +02:00
Eelco Dolstra
2532fee157 On test failures, print a bash stack trace
This makes it easier to identify what command failed. It looks like:

  follow-paths.sh: test failed at:
    main in follow-paths.sh:54
2022-07-14 15:07:19 +02:00
Eelco Dolstra
819615c7f4 Merge pull request #6802 from edolstra/split-flakes-tests
Split flakes tests
2022-07-14 09:15:58 +02:00
Eelco Dolstra
b15c4fdbde Split off 'nix flake check' tests 2022-07-13 21:01:16 +02:00
Eelco Dolstra
752158a8ef Move flake-searching.sh and make it less dependent on git 2022-07-13 20:55:17 +02:00
Eelco Dolstra
6ba45f81a8 Move flake-local-settings.sh 2022-07-13 20:51:28 +02:00
Eelco Dolstra
7abcafcfea Move the 'nix bundle' tests
Note: these were previously not actually called.
2022-07-13 20:49:07 +02:00
Eelco Dolstra
d16f1070f4 Split off following paths tests 2022-07-13 20:46:22 +02:00
Eelco Dolstra
a094259d35 Split off 'nix flake init' tests 2022-07-13 20:37:40 +02:00
Eelco Dolstra
f011c269c9 Split off the circular flake import tests 2022-07-13 20:37:32 +02:00
Eelco Dolstra
c591efafd3 Split off the Mercurial flake tests 2022-07-13 15:06:57 +02:00
Eelco Dolstra
420957e149 Move flakes tests to a subdirectory 2022-07-13 15:06:54 +02:00
Eelco Dolstra
e1153069bd Merge pull request #6797 from edolstra/overrides-check
Simplify the check for overrides on non-existent inputs
2022-07-13 14:45:07 +02:00
Eelco Dolstra
19190c2346 tests/flakes.sh: Make sure flake7 is clean
Cherry-picked from the lazy-trees branch, where we no longer write a
lock file if any of the inputs is dirty.
2022-07-13 13:46:33 +02:00
Eelco Dolstra
12df8885cc Simplify the check for overrides on non-existent inputs 2022-07-13 13:40:40 +02:00
Théophane Hufschmitt
438776cce7 Merge pull request #6794 from eltociear/patch-1
Fix typo in flake.cc
2022-07-13 10:55:25 +02:00
Théophane Hufschmitt
b052e7e71d Add some more completion tests
- Test another command than `build`
- Test with two input flakes
2022-07-13 10:31:17 +02:00
Théophane Hufschmitt
d34a333e2e Fix the “out of order” completion test
`--override-input` id snarky because it takes two arguments, so it
doesn't play well when completed in the middle of the CLI (since the
argument just after gets interpreted as its second argument). So use
`--update-input` instead
2022-07-13 10:25:28 +02:00
Ikko Ashimine
694a9dc282 Fix typo in flake.cc
non-existant -> non-existent
2022-07-13 01:10:32 +09:00
Eelco Dolstra
a9fab18a91 Merge pull request #6791 from edolstra/fix-installer
Fix --no-daemon installation
2022-07-12 17:00:38 +02:00
Théophane Hufschmitt
2dbd5ed0b4 Merge pull request #6663 from Ma27/follows-invalid-input
flakes: throw an error if `follows`-declaration for an input is invalid
2022-07-12 16:44:22 +02:00
Eelco Dolstra
c9d406ba04 Fix --no-daemon installation
It was accidentally triggering the auto-chroot code path because
/nix/var/nix didn't exist.

Fixes #6790.
2022-07-12 16:15:21 +02:00
Eelco Dolstra
f6a434c8a4 Fix debug message 2022-07-12 11:53:34 +02:00
Maximilian Bosch
1f771065f1 Move follows-check into its own function 2022-07-12 11:25:33 +02:00
Maximilian Bosch
411111a3bc Turn error for non-existant follows into a warning 2022-07-12 11:22:35 +02:00
Eelco Dolstra
6260040b9d Merge pull request #6781 from ryantm/stale
update stale bot per RFC 0124
2022-07-12 10:32:56 +02:00
Maximilian Bosch
c1c37f3200 flakes: throw an error if follows-declaration for an input is invalid
I recently got fairly confused why the following expression didn't have
any effect

    {
      description = "Foobar";
      inputs.sops-nix = {
        url = github:mic92/sops-nix;
        inputs.nixpkgs_22_05.follows = "nixpkgs";
      };
    }

until I found out that the input was called `nixpkgs-22_05` (please note
the dash vs. underscore).

IMHO it's not a good idea to not throw an error in that case and
probably leave end-users rather confused, so I implemented a small check
for that which basically checks whether `follows`-declaration from
overrides actually have corresponding inputs in the transitive flake.

In fact this was done by accident already in our own test-suite where
the removal of a `follows` was apparently forgotten[1].

Since the key of the `std::map` that holds the `overrides` is a vector
and we have to find the last element of each vector (i.e. the override)
this has to be done with a for loop in O(n) complexity with `n` being
the total amount of overrides (which shouldn't be that large though).

Please note that this doesn't work with nested expressions, i.e.

    inputs.fenix.inputs.nixpkgs.follows = "...";

which is a known problem[2].

For the expression demonstrated above, an error like this will be
thrown:

    error: sops-nix has a `follows'-declaration for a non-existant input nixpkgs_22_05!

[1] 2664a216e5
[2] https://github.com/NixOS/nix/issues/5790
2022-07-12 10:18:38 +02:00
Théophane Hufschmitt
21c443d4fd Test the tilde expansion for the flake completion
Also add a disabled test for when the `--override-input` flag comes
*before* the flake ref
2022-07-12 09:48:25 +02:00
Théophane Hufschmitt
07e14d3ef0 Harden the comparisons in the completion test
- Don't use `printf` for the expected result, but just use bash's `$' '`
  litteral strings
- Quote the `nix` call result
- Invert the order in the comparisons (just because it feels more
  natural)
2022-07-12 09:24:31 +02:00
Théophane Hufschmitt
260fb837de Fix the name of the completions test 2022-07-12 09:24:31 +02:00
Théophane Hufschmitt
32effccb51 Add some tests for the CLI completion 2022-07-12 09:24:31 +02:00
Théophane Hufschmitt
f071eb3702 Merge pull request #6693 from ncfavier/complete-flake-inputs
Improve shell completion of flake inputs
2022-07-12 09:12:00 +02:00
Eelco Dolstra
7e23039b7f Merge pull request #6786 from edolstra/fix-aarch64-static
Fix buildStatic.aarch64-linux
2022-07-11 21:42:53 +02:00
Eelco Dolstra
28e913c605 Branch 2.10 release notes 2022-07-11 21:10:23 +02:00
Eelco Dolstra
c364e0b098 Fix buildStatic.aarch64-linux
Commit 925b975224 accidentally enabled
use of the gold linker on aarch64-linux, which apparently doesn't work.

https://hydra.nixos.org/build/183500621
2022-07-11 20:57:29 +02:00
Eelco Dolstra
b6e90de0aa Merge pull request #6608 from bburdette/ignore-try
--ignore-try flag for debugger mode
2022-07-11 20:17:11 +02:00
Ben Burdette
cdcc349072 Merge branch 'master' into ignore-try 2022-07-11 11:29:22 -06:00
Ben Burdette
a3629ab0cc move ignore-try to EvalSettings 2022-07-11 10:47:09 -06:00
Ben Burdette
6ac8200ff5 use util.hh class instead of local 2022-07-11 10:21:40 -06:00
Ben Burdette
69ea265fd2 'tryEval' not 'try clause' 2022-07-11 10:21:12 -06:00
Naïm Favier
711b2e1f48 Fix flake input completion for InstallablesCommands
Defers completion of flake inputs until the whole command line is parsed
so that we know what flakes we need to complete the inputs of.

Previously, `nix build flake --update-input <Tab>` always behaved like
`nix build . --update-input <Tab>`.
2022-07-11 15:13:51 +02:00
Eelco Dolstra
545383208e Merge pull request #6777 from laalsaas/master
Fix logical and typographical errors in nix-env man page
2022-07-11 13:52:40 +02:00
Eelco Dolstra
517ce38dad Update release notes 2022-07-11 11:17:19 +02:00
Ryan Mulligan
5022a71305 update stale bot per RFC 0124 2022-07-10 16:50:49 -07:00
laalsaas
eac211a171 Fix logical and typographical errors in nix-env man page 2022-07-10 12:30:17 +02:00
Eelco Dolstra
f6316b49a0 Merge pull request #6768 from NixOS/add-rl-for-traceverbose
Add a release notes entry for #4914
2022-07-06 10:46:47 +02:00
Théophane Hufschmitt
6fa95c35c7 Add a release notes entry for #4914
cc @gytis-ivaskevicius
2022-07-06 06:46:54 +02:00
Théophane Hufschmitt
b0e18df994 Merge pull request #4914 from gytis-ivaskevicius/master
Add builtins.debug
2022-07-06 06:41:18 +02:00
Gytis Ivaskevicius
b2703c73a4 builtins.traceVerbose: Post rebase fixes 2022-07-05 19:56:39 +03:00
Gytis Ivaskevicius
ba1fe85b65 Add builtins.traceVerbose
Co-Authored-By: Silvan Mosberger <contact@infinisil.com>

Add builtins.traceVerbose tests
2022-07-05 19:44:26 +03:00
Théophane Hufschmitt
f653185107 Merge pull request #6762 from thufschmitt/restrict-ci-permissions
Restrict the permissions of the CI
2022-07-05 17:11:16 +02:00
Théophane Hufschmitt
541e10496a Fix the hydra_status CI job 2022-07-05 16:28:39 +02:00
Théophane Hufschmitt
70083218b3 Restrict the permissions of the CI 2022-07-05 16:01:20 +02:00
Théophane Hufschmitt
057b02fd25 Merge pull request #6746 from turrisxyz/Pinned-Dependencies-GitHub
chore: Set permissions for GitHub actions
2022-07-05 16:00:39 +02:00
Théophane Hufschmitt
d63cd77549 Merge pull request #6664 from Ma27/innixshell-backwards-compat
nix-shell: restore backwards-compat with old nixpkgs
2022-07-05 15:57:20 +02:00
Rok Garbas
8a3d34e974 Merge pull request #6749 from centromere/docker-extra-pkgs-layers
Allow specification of extra packages, maxLayers in Docker image
2022-07-05 12:42:49 +02:00
Alex Wied
07416a6005 Allow specification of extra packages, maxLayers in Docker image 2022-07-01 17:53:14 -04:00
naveen
d31c520f40 chore: Set permissions for GitHub actions
Restrict the GitHub token permissions only to the required ones; this way, even if the attackers will succeed in compromising your workflow, they won’t be able to do much.

- Included permissions for the action. https://github.com/ossf/scorecard/blob/main/docs/checks.md#token-permissions

https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#permissions

https://docs.github.com/en/actions/using-jobs/assigning-permissions-to-jobs

[Keeping your GitHub Actions and workflows secure Part 1: Preventing pwn requests](https://securitylab.github.com/research/github-actions-preventing-pwn-requests/)

Signed-off-by: naveen <172697+naveensrinivasan@users.noreply.github.com>
2022-07-01 00:29:30 +00:00
Travis A. Everett
649c9d9b4c doc: acknowledge post-rsync reality
Before #5150 the copy-to-store phase of the install was idempotent,
but the recursive cp isn't. This is probably baiting a few people
into trying corrective installs that will fail.
2022-06-30 13:31:04 -05:00
Théophane Hufschmitt
b7eb4ac169 Merge pull request #6699 from tennox/better-flake-new-error-message
flakes: apply templates partially on conflicts
2022-06-29 18:21:07 +02:00
Théophane Hufschmitt
4c8210095e Merge pull request #6233 from flox/nix-repl-flakes
Nix repl flakes
2022-06-29 17:59:22 +02:00
Théophane Hufschmitt
3b18058969 Merge pull request #6674 from virusdave/dnicponski/scratch/dont_capture_stdout_in_nix_repl_master
Don't capture `stdout` when launching subshells in `nix repl`
2022-06-29 17:54:41 +02:00
Théophane Hufschmitt
83f96e61a4 Add some test for nix flake init with conflicts 2022-06-29 17:08:03 +02:00
Eelco Dolstra
4248174e71 Merge pull request #6702 from ncfavier/develop-prepend-vars
nix develop: save XDG_DATA_DIRS for loadable completion
2022-06-29 14:49:31 +02:00
Théophane Hufschmitt
f8fea9075c Merge remote-tracking branch 'origin/master' into better-flake-new-error-message 2022-06-29 13:44:14 +02:00
Eelco Dolstra
7633764342 Merge pull request #6739 from edolstra/ignore-chroot-error
Don't fail if we can't create ~/.local/share/nix/root
2022-06-29 13:06:35 +02:00
Eelco Dolstra
6cab528461 Don't fail if we can't create ~/.local/share/nix/root
https://hydra.nixos.org/build/182135943
2022-06-29 12:16:51 +02:00
Eelco Dolstra
865af66112 Merge pull request #6738 from edolstra/quote
src/libexpr/tests/primops.cc: Quote Nix expressions
2022-06-29 12:16:15 +02:00
Eelco Dolstra
455177cbe0 src/libexpr/tests/primops.cc: Quote Nix expressions
Otherwise they don't survive reformatting, see the failure in
https://github.com/NixOS/nix/pull/6721.
2022-06-29 11:37:25 +02:00
Eelco Dolstra
3c1814423b Merge pull request #6737 from centromere/fix-typo
doc: Fix typo
2022-06-29 11:02:46 +02:00
Théophane Hufschmitt
56cf96a1b9 Merge pull request #6706 from lheckemann/cache-info-cache-invalidation
nar-info-disk-cache: refresh nix-cache-info weekly
2022-06-29 07:56:27 +02:00
Alex Wied
cd361b31fa doc: Fix typo 2022-06-28 22:43:37 -04:00
Eelco Dolstra
6cb41288ac Merge pull request #6727 from symphorien/shallow-doc
fetchGit: document `shallow` argument
2022-06-27 14:11:37 +02:00
Manu [tennox]
ae4c9ef8e2 #6699 flake init: fix trying to add unchanged file
After skipping because of being of identical content it tried to git add it.
2022-06-26 21:35:21 +01:00
Manu [tennox]
4374e3ec67 #6699 flake init: Apply suggestions of @thufschmitt 2022-06-26 18:12:30 +01:00
Manuel
58cbbdc5e7 Update src/nix/flake.cc
https://github.com/NixOS/nix/pull/6699#discussion_r904097147

Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2022-06-26 18:00:57 +01:00
Manuel
117baee1b7 Update src/nix/flake.cc
https://github.com/NixOS/nix/pull/6699#discussion_r904096906

Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2022-06-26 18:00:34 +01:00
Guillaume Girol
e8109cf405 fetchGit: document shallow argument 2022-06-26 12:00:00 +00:00
Eelco Dolstra
49c4a37efc Merge pull request #6717 from edolstra/move-flake-registry
Fetch flake-registry.json from channels.nixos.org
2022-06-25 21:17:57 +02:00
Eelco Dolstra
586fa707fc Merge pull request #6714 from edolstra/auto-chroot-store
Automatically use a chroot store if /nix doesn't exist
2022-06-25 00:03:35 +02:00
Eelco Dolstra
30d4aa5dd6 Only do the auto chroot store on Linux 2022-06-24 23:35:21 +02:00
Eelco Dolstra
749d914d10 Add reminder to remove nix-static testing hack
https://github.com/NixOS/nix/pull/6708#issuecomment-1165912951
2022-06-24 23:14:56 +02:00
Tom Bereknyei
f801d70ba7 tests: enable ca-derivations for simple.nix in repl tests 2022-06-24 11:17:29 -04:00
Théophane Hufschmitt
f06b6fa4fa Merge pull request #6698 from tweag/forbid-tilde-paths-in-pure-mode
Forbid the tilde expansion in pure eval mode
2022-06-24 10:02:40 +02:00
Dave Nicponski
c6f7726f48 Don't capture stdout when launching subshells in nix repl 2022-06-23 22:45:13 -04:00
Eelco Dolstra
2ef6785d8e Merge pull request #6716 from Mindavi/bugfix/cast
eval-cache: cast rowId to correct type
2022-06-24 00:28:38 +02:00
Eelco Dolstra
ae2a8c0737 Merge pull request #6715 from cole-h/drop-narinfo-system
libstore/nar-info: drop unused system field
2022-06-24 00:03:50 +02:00
Eelco Dolstra
4b6cc3da62 Fetch flake-registry.json from channels.nixos.org
Using fastly is slightly faster, provides some resilience due to a
high stale TTL, and allows some usage metrics.
2022-06-23 23:56:26 +02:00
Rick van Schijndel
2beb929753 eval-cache: cast rowId to correct type
Prevents errors when running with UBSan:

/nix/store/j5vhrywqmz1ixwhsmmjjxa85fpwryzh0-gcc-11.3.0/include/c++/11.3.0/bits/stl_pair.h:353:4: runtime error: load of value 229, which is not a valid value for type 'AttrType'
2022-06-23 21:11:08 +02:00
Linus Heckemann
8cf26385cd [fixup] handle cache expiration in sqlite query 2022-06-23 14:54:25 -04:00
Cole Helbling
561a258f1d libstore/nar-info: drop unused system field
This was unused everywhere (and even the official NixOS binary cache
did not produce .narinfo files containing a "System:" field).
2022-06-23 14:25:10 -04:00
Eelco Dolstra
1cb376d60e Fix typo
Co-authored-by: Cole Helbling <cole.e.helbling@outlook.com>
2022-06-23 17:18:22 +02:00
Eelco Dolstra
2a9fddc0b1 Automatically use a chroot store if /nix doesn't exist
Specifically, if we're not root and the daemon socket does not exist,
then we use ~/.local/share/nix/root as a chroot store. This enables
non-root users to download nix-static and have it work out of the box,
e.g.

  ubuntu@ip-10-13-1-146:~$ ~/nix run nixpkgs#hello
  warning: '/nix' does not exists, so Nix will use '/home/ubuntu/.local/share/nix/root' as a chroot store
  Hello, world!
2022-06-23 16:29:50 +02:00
Eelco Dolstra
3c57db1a0f Merge pull request #6710 from edolstra/embedded-sandbox-shell
Embed the sandbox shell into the statically linked 'nix' binary
2022-06-23 15:34:16 +02:00
Eelco Dolstra
0b2ea0023c Fix typo 2022-06-23 14:22:11 +02:00
Théophane Hufschmitt
027f6a735f Merge pull request #6673 from asymmetric/warn
libstore: improve warning message on missing sig
2022-06-23 06:42:48 +02:00
Eelco Dolstra
925b975224 Embed the sandbox shell into the statically linked 'nix' binary
With this, Nix will write a copy of the sandbox shell to /bin/sh in
the sandbox rather than bind-mounting it from the host filesystem.
This makes /bin/sh work out of the box with nix-static, i.e. you no
longer get

  /nix/store/qa36xhc5gpf42l3z1a8m1lysi40l9p7s-bootstrap-stage4-stdenv-linux/setup: ./configure: /bin/sh: bad interpreter: No such file or directory
2022-06-23 04:08:28 +02:00
Eelco Dolstra
eafa2721ca Merge pull request #6709 from edolstra/fix-static-build-hook
Fix build-remote in nix-static
2022-06-23 02:12:55 +02:00
Eelco Dolstra
1e55ee2961 getSelfExe(): Support macOS 2022-06-23 01:32:46 +02:00
Eelco Dolstra
184f4e40de Remove NIX_LIBEXEC_DIR 2022-06-23 01:32:46 +02:00
Eelco Dolstra
d3176ce076 Fix build-remote in nix-static
'build-remote' is now executed via /proc/self/exe so it always works.
2022-06-23 01:32:46 +02:00
Naïm Favier
155c57c171 nix develop: save XDG_DATA_DIRS for loadable completion 2022-06-23 01:11:33 +02:00
Eelco Dolstra
3c48c4b4f7 Merge pull request #6708 from edolstra/run-static-tests
Enable tests for nix-static
2022-06-23 00:11:23 +02:00
Eelco Dolstra
958b8d1e35 Merge pull request #6707 from edolstra/fix-incremental-static
Fix incremental static builds
2022-06-22 23:50:24 +02:00
Eelco Dolstra
8bbbb6e737 Enable/fix tests in nix-static
pkgsStatic is apparently considered a cross environment, so checkPhase
and installCheckPhase are disabled even when we ask for them.
2022-06-22 23:33:15 +02:00
Maximilian Bosch
e94aa1f647 tests/nix-shell: more meaningful testcase 2022-06-22 22:35:48 +02:00
Eelco Dolstra
696121fe1d Fix incremental static builds
$? refers to the object files that are newer, so the resulting file
would lack all the older object files.
2022-06-22 17:53:58 +02:00
Linus Heckemann
d533a88546 nar-info-disk-cache: refresh nix-cache-info weekly
This allows changes to nix-cache-info to be picked up by existing
clients. Previously, the only way for this to happen would be for
clients to delete binary-cache-v6.sqlite, which is quite awkward for
users.

On the other hand, updates to nix-cache-info should be pretty rare,
hence the choice of a fairly long TTL. Configurability is probably not
useful enough to warrant implementing it.
2022-06-22 10:49:18 -04:00
Eelco Dolstra
f6cf644e5f Style 2022-06-22 15:35:52 +02:00
Manu [tennox]
3a85fd077c #6542 Apply flake templates partially on conflicts
Will still exit with non-zero exit code and clearly prompt which files to merge:
 ```
nixx flake init -t github:numtide/devshell
wrote: /home/manu/dev/stuff/gopassbridge/.envrc
refusing to overwrite existing file '/home/manu/dev/stuff/gopassbridge/.gitignore'
-> merge manually with '/nix/store/ksmwhyghjwb4d9dw6hcpbvng1msdvjim-source/template/.gitignore'
wrote: /home/manu/dev/stuff/gopassbridge/devshell.toml
wrote: /home/manu/dev/stuff/gopassbridge/flake.nix
error: Encountered 1 conflicts - please merge manually
 ```
2022-06-21 17:37:02 +01:00
Eelco Dolstra
bbe7187db9 Merge pull request #6695 from afishhh/search-fix-exclude
Fix arity of `--exclude` flag in `nix search`
2022-06-21 16:12:04 +02:00
Théophane Hufschmitt
983efdbde4 Forbid the tilde expansion in pure eval mode
Fix #6684
2022-06-21 14:08:18 +02:00
Théophane Hufschmitt
16c6c6c963 Merge pull request #6676 from virusdave/dnicponski/scratch/swap_homedir_check_master
Verify this if `$HOME` exists, it is owned by current user in `getHome()`
2022-06-20 21:17:22 +02:00
Fishhh
df21173b70 Add another test for --exclude in nix search 2022-06-20 19:21:45 +02:00
Fishhh
4ade8a5f25 Fix arity of --exclude flag in nix search
Due to incorrectly using the Handler(vector<string>*) constructor the
`--exclude` flag would swallow all proceeding arguments instead of just
one.
2022-06-20 18:00:32 +02:00
Naïm Favier
d6d0e781bb Complete flake inputs for all given flakes
Allow `nix build flake1 flake2 --update-input <Tab>` to complete the
inputs of both flakes.

Also do tilde expansion so that `nix build ~/flake --update-input <Tab>`
works.
2022-06-20 04:17:13 +02:00
Dave Nicponski
ca2be509b9 Verify $HOME is owned by current user in getHome(), if it exists.
Useful because a default `sudo` on darwin doesn't clear `$HOME`, so things like `sudo nix-channel --list`
will surprisingly return the USER'S channels, rather than `root`'s.

Other counterintuitive outcomes can be seen in this PR description:
  https://github.com/NixOS/nix/pull/6622
2022-06-17 17:42:08 -04:00
Travis A. Everett
fe76b9f4b4 doc: fix some darwin uninstall gaps 2022-06-16 09:37:52 -05:00
Théophane Hufschmitt
7e301fd74e Merge pull request #6620 from maralorn/fix-print-missing
respect print-missing variable in new-style build command
2022-06-16 13:23:15 +02:00
Lorenzo Manacorda
475249db8a libstore: improve warning message on missing sig
Clarifies that the substitute will be ignored/skipped.
2022-06-15 17:34:28 +02:00
Tom Bereknyei
51268ceb79 Merge branch 'master' into nix-repl-flakes 2022-06-15 09:17:03 -04:00
tomberek
dae4a8a6c8 Apply suggestions from code review
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2022-06-15 09:02:36 -04:00
Théophane Hufschmitt
63df0fda94 Merge pull request #6615 from WhyNotHugo/man-improvements
Add disambiguation to man page
2022-06-15 10:41:52 +02:00
Eelco Dolstra
3720a4f6cf Merge pull request #6662 from tweag/stiky-non-flake-overrides
flake.cc: Make non-flake overrides sticky
2022-06-14 19:47:11 +02:00
Hugo Osvaldo Barrera
fd7f795750 Add disambiguation to man page
This should help future lost newcomers like myself understand where to
find the docs for both of these commands and how they differ.
2022-06-14 18:13:35 +02:00
Maximilian Bosch
98946e2d9c nix-shell: restore backwards-compat with old nixpkgs
Basically an attempt to resume fixing #5543 for a breakage introduced
earlier[1]. Basically, when evaluating an older `nixpkgs` with
`nix-shell` the following error occurs:

    λ ma27 [~] → nix-shell -I nixpkgs=channel:nixos-18.03 -p nix
    error: anonymous function at /nix/store/zakqwc529rb6xcj8pwixjsxscvlx9fbi-source/pkgs/top-level/default.nix:20:1 called with unexpected argument 'inNixShell'

           at /nix/store/zakqwc529rb6xcj8pwixjsxscvlx9fbi-source/pkgs/top-level/impure.nix:82:1:

               81|
               82| import ./. (builtins.removeAttrs args [ "system" "platform" ] // {
                 | ^
               83|   inherit config overlays crossSystem;

This is a problem because one of the main selling points of Nix is that
you can evaluate any old Nix expression and still get the same result
(which also means that it *still evaluates*). In fact we're deprecating,
but not removing a lot of stuff for that reason such as unquoted URLs[2]
or `builtins.toPath`. However this property was essentially thrown away
here.

The change is rather simple: check if `inNixShell` is specified in the
formals of an auto-called function. This means that

    { inNixShell ? false }:
    builtins.trace inNixShell
      (with import <nixpkgs> { }; makeShell { name = "foo"; })

will show `trace: true` while

    args@{ ... }:
    builtins.trace args.inNixShell
      (with import <nixpkgs> { }; makeShell { name = "foo"; })

will throw the following error:

    error: attribute 'inNixShell' missing

This is explicitly needed because the function in
`pkgs/top-level/impure.nix` of e.g. NixOS 18.03 has an ellipsis[3], but
passes the attribute-set on to another lambda with formals that doesn't
have an ellipsis anymore (hence the error from above). This was perhaps
a mistake, but we can't fix it anymore. This also means that there's
AFAICS no proper way to check if the attr-set that's passed to the Nix
code via `EvalState::autoCallFunction` is eventually passed to a lambda
with formals where `inNixShell` is missing.

However, this fix comes with a certain price. Essentially every
`shell.nix` that assumes `inNixShell` to be passed to the formals even
without explicitly specifying it would break with this[4]. However I think
that this is ugly, but preferable:

* Nix 2.3 was declared stable by NixOS up until recently (well, it still
  is as long as 21.11 is alive), so most people might not have even
  noticed that feature.

* We're talking about a way shorter time-span with this change being
  in the wild, so the fallout should be smaller IMHO.

[1] 9d612c393a
[2] https://github.com/NixOS/rfcs/pull/45#issuecomment-488232537
[3] https://github.com/NixOS/nixpkgs/blob/release-18.03/pkgs/top-level/impure.nix#L75
[4] See e.g. the second expression in this commit-message or the changes
    for `tests/ca/nix-shell.sh`.
2022-06-13 23:29:28 +02:00
Alexander Bantyev
d82a3dc70d flake.cc: Make non-flake overrides sticky
Overrides for inputs with flake=false were non-sticky, since they
changed the `original` in `flake.lock`. This fixes it, by using the same
locked original for both flake and non-flake inputs.
2022-06-13 20:49:16 +04:00
Eelco Dolstra
9f58df4c91 Merge pull request #6619 from Jonpez2/patch-1
Add security.csm to ignored-acls
2022-06-13 16:23:13 +02:00
Eelco Dolstra
798efdf10b Merge pull request #6027 from Ma27/pure-repl
repl: `--option pure-eval true` actually enables pure eval mode
2022-06-13 16:11:46 +02:00
Eelco Dolstra
2fef24f528 Merge pull request #6634 from lovesegfault/fix-getgrouplist
fix(libstore/lock): support users that belong to more than 10 groups
2022-06-13 15:45:09 +02:00
Eelco Dolstra
9a26a01475 Merge pull request #6639 from Artturin/checkse
install-multi-user: check if selinux is enabled and if it is then abort
2022-06-13 15:42:43 +02:00
Eelco Dolstra
95eccfd5db Merge pull request #6648 from andersk/missing-rethrow
Add missing rethrows in conditional exception handlers
2022-06-13 15:37:38 +02:00
Théophane Hufschmitt
03226aa053 Merge pull request #6651 from ncfavier/nix-build-stop-logger
nix-build: stop logger when appropriate
2022-06-13 10:43:14 +02:00
Théophane Hufschmitt
84c4666467 Merge pull request #6653 from Misterio77/readd-sourcehut-regex-matching
fix sourcehut brach/tag resolving regression
2022-06-13 08:57:34 +02:00
Gabriel Fontes
9f6b4639c2 fix sourcehut brach/tag resolving regression
nixos/nix#6290 introduced a regex pattern to account for tags when
resolving sourcehut refs. nixos/nix#4638 reafactored the code,
accidentally treating the pattern as a regular string, causing all
non-HEAD ref resolving to break.

This fixes the regression and adds more test cases to avoid future
breakage.
2022-06-11 16:52:20 -03:00
Travis A. Everett
edfcc8256e doc: add install test info to hacking.md 2022-06-11 13:30:51 -05:00
Naïm Favier
502d7d9092 nix-build: stop logger when appropriate
Reverts b944b588fa in `nix-build.cc`.
2022-06-11 15:17:35 +02:00
Théophane Hufschmitt
37fc4d73bb Merge pull request #6646 from YorikSar/file-impure-doc
Mention that -f implies --impure for eval in docs
2022-06-11 08:48:16 +02:00
Ben Burdette
49ff4ef637 remove unused parameter 2022-06-10 12:22:36 -06:00
Anders Kaseorg
754cd53faf Add missing rethrows in conditional exception handlers
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2022-06-10 10:52:41 -07:00
Yuriy Taraday
65d09fce22 Mention that -f implies --impure for eval in docs
Right now this is not mentioned anywhere and it is unexpected.
2022-06-10 19:16:17 +04:00
Théophane Hufschmitt
45ebaab665 Merge pull request #6643 from ncfavier/fix-git-again
libfetchers/git: add missing `--git-dir` flags
2022-06-10 14:33:32 +02:00
Théophane Hufschmitt
e557f86490 Merge pull request #6641 from NixOS/fix-docker-ci-job
Correctly get the nix version in the docker job
2022-06-10 13:36:30 +02:00
Naïm Favier
da8f8668ca libfetchers/git: add missing --git-dir flags 2022-06-10 12:57:13 +02:00
Théophane Hufschmitt
460117a238 Correctly get the nix version in the docker job
`defaultPackage` doesn't exist anymore, so we can't use it.
Instead just use the new CLI which should be more robust to these
changes

Fix #6640
2022-06-10 12:09:09 +02:00
Théophane Hufschmitt
17e54a602e Merge pull request #6637 from sidkshatriya/small-move-optimization-2
nix-env: A small std::move() optimization
2022-06-10 10:28:37 +02:00
Jonpez2
3f5f84258d Update docker.nix
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2022-06-10 09:17:28 +01:00
Artturin
bd3a17d00c install-multi-user: check if selinux is enabled and if it is then abort 2022-06-10 05:12:03 +03:00
Sidharth Kshatriya
7868405d58 nix-env: A small std::move() optimization
Avoids doing a O(n) copy of Strings i.e. std::list<std::string>
2022-06-09 20:01:01 +05:30
Eelco Dolstra
1bc17ae617 Merge pull request #6635 from sidkshatriya/small-move-optimization
nix-store: small std::move() optimization
2022-06-09 13:48:48 +02:00
Sidharth Kshatriya
3efea3d28a nix-store: small std::move() optimization 2022-06-09 16:25:26 +05:30
Bernardo Meurer
931930feb1 fix(libstore/lock): support users that belong to more than 10 groups
The manpage for `getgrouplist` says:

> If the number of groups of which user is a member is less than or
> equal to *ngroups, then the value *ngroups is returned.
>
> If the user is a member of more than *ngroups groups, then
> getgrouplist() returns -1.  In this case, the value returned in
> *ngroups can be used to resize the buffer passed to a further
> call getgrouplist().

In our original code, however, we allocated a list of size `10` and, if
`getgrouplist` returned `-1` threw an exception. In practice, this
caused the code to fail for any user belonging to more than 10 groups.

While unusual for single-user systems, large companies commonly have a
huge number of POSIX groups users belong to, causing this issue to crop
up and make multi-user Nix unusable in such settings.

The fix is relatively simple, when `getgrouplist` fails, it stores the
real number of GIDs in `ngroups`, so we must resize our list and retry.
Only then, if it errors once more, we can raise an exception.

This should be backported to, at least, 2.9.x.
2022-06-08 13:45:41 -04:00
Eelco Dolstra
12e86c0735 Merge pull request #6630 from edolstra/boost-dylib
Remove ${boost}/lib from the RPATH
2022-06-08 17:55:21 +02:00
Eelco Dolstra
7b968af930 Update docker.nix
Co-authored-by: Cole Helbling <cole.e.helbling@outlook.com>
2022-06-08 17:41:31 +02:00
Eelco Dolstra
4a3f217bde Remove ${boost}/lib from the RPATH 2022-06-08 15:51:35 +02:00
Théophane Hufschmitt
34d90fbe22 Mention the parallel copy in the release notes 2022-06-08 15:25:52 +02:00
Théophane Hufschmitt
480c2b6699 Rewrite the CA paths when moving them between store
Bring back the possibility to copy CA paths with no reference (like the
outputs of FO derivations or stuff imported at eval time) between stores
that have a different prefix.
2022-06-08 15:13:11 +02:00
Théophane Hufschmitt
cb0553ecd0 Restore the "low-latency" ssh copying 2022-06-08 14:03:46 +02:00
Théophane Hufschmitt
694b12052a Merge pull request #6624 from NixOS/new-flake-style
flake.nix: Convert to new naming convention
2022-06-08 13:34:09 +02:00
Eelco Dolstra
e88892dc21 Merge pull request #6627 from asymmetric/key-comment
Fix missing ` in key manual
2022-06-08 12:10:52 +02:00
Lorenzo Manacorda
814ddfa5f5 Fix missing ` in key manual 2022-06-08 11:46:50 +02:00
Jonpez2
a7d25d339d Add security.csm to the default ignore list 2022-06-08 09:32:14 +01:00
Eelco Dolstra
b2dea231cf Merge pull request #6618 from afishhh/search-exclude
Add `-e`/`--exclude` flag to `nix search`
2022-06-07 19:01:36 +02:00
Fishhh
9ae22b1fde Use grep -c instead of grep|wc -l in some nix search tests 2022-06-07 18:25:59 +02:00
Fishhh
0338cf5539 Add tests for --exclude flag in nix search 2022-06-07 18:25:52 +02:00
Fishhh
e009367c8d Remove redundant std::moves in calls to hiliteMatches 2022-06-07 18:25:52 +02:00
Fishhh
b42358b9be Add --exclude flag to nix search
If a package's attribute path, description or name contains matches for any of the
regexes specified via `-e` or `--exclude` that package is excluded from
the final output.
2022-06-07 18:25:48 +02:00
Eelco Dolstra
8977cdee6a Merge pull request #6598 from cole-h/update-flake-for-real
flake: update to 22.05
2022-06-07 15:13:16 +02:00
Eelco Dolstra
faf80fa920 Convert to new flake style
https://github.com/NixOS/nix/issues/5532
2022-06-07 14:36:23 +02:00
Eelco Dolstra
0f8754cd30 Fix 22.05 eval warnings 2022-06-07 14:27:10 +02:00
Eelco Dolstra
5a9d83aa59 Disable cross builds on platforms other than x86_64-linux
Needed because evaluation was broken on x86_64-darwin.
2022-06-07 14:27:06 +02:00
Eelco Dolstra
739c358c68 Merge pull request #6602 from cw789/patch-2
Explain exactly what nix-upgrade nix does
2022-06-07 13:08:56 +02:00
Malte Brandy
bf2f25e3d8 respect print-missing variable in new-style build command
Currently nix-build prints the "printMissing" information by default,
nix build doesn’t.
People generally don‘t notice this because the standard log-format of
nix build would not display the printMissing
output long enough to perceive the information.

This addresses https://github.com/NixOS/nix/issues/6561
2022-06-06 21:15:31 +02:00
Jonpez2
0cd560c95d Add security.csm to ignored-acls
The security.csm ACL is, as far as I know, never reasonable to remove, so let's add it to the ignore-list in the vanilla nix image.  This makes this image usable on GKE.
2022-06-06 16:56:42 +01:00
Fishhh
d137ceccef Fix incorrect comment in hiliteMatches 2022-06-05 20:30:18 +02:00
Théophane Hufschmitt
95f47c28fb Make nix copy parallel again
FILLME
2022-06-03 17:01:16 +02:00
Eelco Dolstra
1dd7253133 Merge pull request #6603 from abathur/installer_fix_vi_swapfile
darwin-install: work around existing vim swap files
2022-06-03 08:51:48 +02:00
Eelco Dolstra
2097c30b08 Merge pull request #6600 from asymmetric/hash-comment
schema.sql: add comment about hash being in base16
2022-06-03 08:51:07 +02:00
Eelco Dolstra
b2edca1def Merge pull request #6610 from edolstra/random-fixes
Random fixes/improvements from the lazy-trees branch
2022-06-03 08:50:16 +02:00
Tom Bereknyei
ffd41d1757 Merge branch 'master' into nix-repl-flakes 2022-06-02 16:58:35 -04:00
Eelco Dolstra
28e08822a3 Avoid unnecessary string copy 2022-06-02 21:26:28 +02:00
Eelco Dolstra
abb80cfa4c Add operator for concatenating strings and string_views 2022-06-02 21:26:21 +02:00
Eelco Dolstra
24b3a500a7 Typo 2022-06-02 21:21:10 +02:00
Eelco Dolstra
81a486c607 Shut up clang warnings 2022-06-02 21:19:54 +02:00
Eelco Dolstra
2868acb4a5 tests/flakes.sh: Fix some ignored breakage 2022-06-02 21:19:16 +02:00
Ben Burdette
8cf6ae8664 use Counter class to count tryEval levels 2022-06-02 12:29:38 -06:00
Ben Burdette
bc0d41e9ba print message with exceptions in a try clause 2022-06-02 12:17:28 -06:00
Ben Burdette
9151dbff88 ignore-try flag 2022-06-02 10:26:46 -06:00
Chris Wögi
51b3cc151e Explain exactly what nix-upgrade nix does 2022-06-02 15:05:27 +02:00
Théophane Hufschmitt
1892355766 Merge pull request #5813 from pennae/lto
enable LTO in optimized builds
2022-06-01 19:07:58 +02:00
Théophane Hufschmitt
a5f7b934bd Merge branch 'master' into lto 2022-06-01 17:55:03 +02:00
Travis A. Everett
505d6ee5e2 darwin-install: work around existing vim swapfiles
User on Matrix reported install problems which presented as
"vifs:editing error" which we traced back to vim griping about an
existing swap file. When opened interactively, it did this:

E325: ATTENTION
Found a swap file by the name "/etc/.fstab.swp"
owned by: root dated: Sön Apr 24 16:54:10 2022
file name: /private/etc/fstab
modified: YES
user name: root host name: MBP.local
process ID: 1698
While opening file "/etc/fstab"
dated: Sön Apr 24 16:56:27 2022
NEWER than swap file!
...
2022-06-01 09:41:00 -05:00
Lorenzo Manacorda
a9358a6097 schema.sql: add comment about hash being in base16 2022-06-01 14:59:57 +02:00
Maximilian Bosch
159b5815b5 repl: --option pure-eval true actually enables pure eval mode
To quote Eelco in #5867:

> Unfortunately we can't do
>
>     evalSettings.pureEval.setDefault(false);
>
> because then we have to do the same in main.cc (where
> pureEval is set to true), and that would allow pure-eval
> to be disabled globally from nix.conf.

Instead, a command should specify that it should be impure by
default. Then, `evalSettings.pureEval` will be set to `false;` unless
it's overridden by e.g. a CLI flag.

In that case it's IMHO OK to be (theoretically) able to override
`pure-eval` via `nix.conf` because it doesn't have an effect on commands
where `forceImpureByDefault` returns `false` (i.e. everything where pure
eval actually matters).

Closes #5867
2022-05-31 22:20:11 +02:00
Cole Helbling
dbf0d3a150 tests/nss-preload: move nix-fetch binding 2022-05-31 12:14:34 -07:00
Cole Helbling
bd0192d0bb flake: update to 22.05
The static build works now :)
2022-05-31 11:51:17 -07:00
Eelco Dolstra
078c80f750 Merge pull request #6582 from NixOS/debug-fetchgit-sigsev
Fix a segfault in the git fetcher
2022-05-31 17:17:37 +02:00
Eelco Dolstra
04a699b8a9 Typo 2022-05-31 10:38:03 +02:00
Eelco Dolstra
403201e011 Merge pull request #6537 from cole-h/update-flake
flake: use github: reference to nixpkgs
2022-05-31 08:30:42 +02:00
Eelco Dolstra
929ab5b195 Bump version 2022-05-30 21:11:20 +02:00
Eelco Dolstra
de13b44573 Branch 2.9 release notes 2022-05-30 20:42:55 +02:00
Eelco Dolstra
af23d38019 Merge pull request #6590 from edolstra/fix-noexcept
Fix noexcept violations
2022-05-30 17:45:44 +02:00
Eelco Dolstra
452dba510d Mark nix-perl as a Perl module
The call to perl.withPackages in nix-serve expects this.
2022-05-30 14:10:26 +02:00
Eelco Dolstra
a72e75e6b6 Merge pull request #6589 from edolstra/respect-output-specified
Respect the outputSpecified attribute
2022-05-30 13:40:07 +02:00
Eelco Dolstra
948515efb7 Set meta.platforms
'nix-serve' in nixpkgs expects the nix package to set this.
2022-05-30 13:35:28 +02:00
Eelco Dolstra
6378f0bb32 RemoteStore::queryRealisationUncached(): Fix potential noexcept violation 2022-05-30 13:27:13 +02:00
Eelco Dolstra
b8faa83742 HttpBinaryCacheStore::getFile(): Don't throw an exception
This violates the noexcept specification.

Fixes #6445.
2022-05-30 13:24:04 +02:00
Eelco Dolstra
8e8e9d8705 Respect the outputSpecified attribute
E.g. 'nix build nixpkgs#libxml2.dev' will build the 'dev' output.
2022-05-30 11:34:47 +02:00
Théophane Hufschmitt
027fd45230 Fix a segfault in the git fetcher
The git fetcher code used to dereference the (potentially empty) `ref`
input attribute. This was magically working, probably because the
compiler somehow outsmarted us, but is now blowing up with newer nixpkgs
versions.

Fix that by not trying to access this field while we don't know for sure
that it has been defined.

Fix #6554
2022-05-27 16:15:28 +02:00
Eelco Dolstra
ec07a70979 Style fix 2022-05-27 11:25:05 +02:00
Eelco Dolstra
d67a2ae52d Merge branch 'doc-redirects' of https://github.com/jtojnar/nix 2022-05-27 11:12:41 +02:00
Eelco Dolstra
b2d0bd4067 Merge pull request #6580 from trofi/fix-build-for-gcc-13
src/libutil/json.cc: add missing <cstdint> include for gcc-13
2022-05-26 21:07:47 +02:00
Sergei Trofimovich
b36d5172cb src/libutil/json.cc: add missing <cstdint> include for gcc-13
Without the change llvm build fails on this week's gcc-13 snapshot as:

    src/libutil/json.cc: In function 'void nix::toJSON(std::ostream&, const char*, const char*)':
    src/libutil/json.cc:33:22: error: 'uint16_t' was not declared in this scope
       33 |             put(hex[(uint16_t(*i) >> 12) & 0xf]);
          |                      ^~~~~~~~
    src/libutil/json.cc:5:1: note: 'uint16_t' is defined in header '<cstdint>'; did you forget to '#include <cstdint>'?
        4 | #include <cstring>
      +++ |+#include <cstdint>
        5 |
2022-05-26 18:37:15 +01:00
Jan Tojnar
26d1877d6e doc: Add redirects for the DocBook manual
There are still many links to the old manual on the web and
having them end up on the Introduction page is a bad user experience.
2022-05-26 18:17:21 +02:00
Jan Tojnar
a793863b97 doc: Manually insert some anchors 2022-05-26 18:17:21 +02:00
Jan Tojnar
7708a34a51 doc: Add anchors to long lists
Added using the following sed scripts:

- For command-ref/opt-common.md:

    s~- `(--?)([^`]+)`~- [`\1\2`]{#opt-\2}~g

- For expressions/builtin-constants.md:

    s~- `(builtins\.?)([^`]+)`~- [`\1\2`]{#builtins-\2}~g

- For expressions/advanced-attributes.md

    s~^  - `([^`]+)`~  - [`\1`]{#adv-attr-\1}~g

  and manually adjusted outputHashAlgo & outputHashMode.

- For glossary.md

    s~^  - (`([^`]+)`|(.+)) ?\\~  - [\1]{#gloss-\2\3}\\~g;
    s~(gloss-\w+) ~\1-~g

  and manually adjusted anchors for Nix expression, user environment, NAR, ∅ and ε.

- For command-ref/env-common.md

    s~^  - `([^`]+)`~  - [`\1`]{#env-\1}~g'
2022-05-26 18:17:21 +02:00
Jan Tojnar
3272afa17b doc: Port anchors preprocessor to jq script
Python is only pulled into the build closure by Mercurial, which might end up being removed.
Let’s port the script to jq, which is more likely to stay.
2022-05-26 18:17:21 +02:00
Jan Tojnar
4de84e095d doc: Introduce pre-processor for adding anchors to text
It is now possible to use the following syntax to insert anchors into the text:

    []{#anchor-name}

The anchor will allow linking to the location it is placed by appending #anchor-name to the URL.

Additionally, it is possible to create a link pointing to its own location by adding text between the square brackets:

    [`--add-root`]{#opt-add-root}
2022-05-26 17:54:15 +02:00
Eelco Dolstra
5bcd0f09b3 Merge pull request #6578 from edolstra/remove-perl
Remove unused Perl dependency
2022-05-26 15:53:26 +02:00
Eelco Dolstra
5b27040203 Remove unused Perl dependency 2022-05-26 14:49:17 +02:00
Eelco Dolstra
1e67582d75 Merge pull request #6576 from hercules-ci/daemon-increase-socket-backlog
Fix `Connection refused` on daemon socket
2022-05-26 12:43:45 +02:00
Eelco Dolstra
8f4548d401 Tweak IN_NIX_SHELL description 2022-05-26 12:40:01 +02:00
Eelco Dolstra
9acc770ce4 Remove pre-C++11 hackiness 2022-05-26 12:40:01 +02:00
Robert Hensing
c156155239 createUnixDomainSocket: listen(unix, 5 -> 100)
This solves the error

    error: cannot connect to socket at '/nix/var/nix/daemon-socket/socket': Connection refused

on build farm systems that are loaded but operating normally.

I've seen this happen on an M1 mac running a loaded hercules-ci-agent.
Hercules CI uses multiple worker processes, which may connect to
the Nix daemon around the same time. It's not unthinkable that
the Nix daemon listening process isn't scheduled until after 6
workers try to connect, especially on a system under load with
many workers.

Is the increase safe?

The number is the number of connections that the kernel will buffer
while the listening process hasn't `accept`-ed them yet.
It did not - and will not - restrict the total number of daemon
forks that a client can create.

History

The number 5 has remained unchanged since the introduction in
nix-worker with 0130ef88ea in 2006.
2022-05-26 11:24:04 +02:00
Eelco Dolstra
762fa2b2ff Merge pull request #6570 from edolstra/eof-in-string
Handle EOFs in string literals correctly
2022-05-26 08:13:39 +02:00
Eelco Dolstra
5f097fbcef Merge pull request #5416 from bburdette/debug-exploratory-PR
--debugger flag
2022-05-26 08:12:19 +02:00
Ben Burdette
9068d32e12 remove parens from repl help 2022-05-25 12:55:58 -06:00
Ben Burdette
9a5ea6c359 Merge branch 'master' into debug-exploratory-PR 2022-05-25 10:41:10 -06:00
Ben Burdette
6031a36208 add --debugger to rl-next list 2022-05-25 10:38:13 -06:00
Ben Burdette
b4c24a29c6 back to ref<EvalState> in NixRepl 2022-05-25 10:21:20 -06:00
Eelco Dolstra
27ebb97d0a Handle EOFs in string literals correctly
We can't return a STR token without setting a valid StringToken,
otherwise the parser will crash.

Fixes #6562.
2022-05-25 17:58:13 +02:00
Eelco Dolstra
d8398d33c9 Typo 2022-05-25 15:29:27 +02:00
Eelco Dolstra
89a8955e79 Merge remote-tracking branch 'origin/file-fetcher' 2022-05-25 15:15:16 +02:00
Eelco Dolstra
2f8a34cddc Fix warning 2022-05-25 15:05:39 +02:00
pennae
b092afe77d Merge branch 'master' into lto 2022-05-25 11:55:13 +00:00
Eelco Dolstra
91b7d5373a Style tweaks 2022-05-25 12:32:22 +02:00
Eelco Dolstra
09e55010f9 Merge pull request #6568 from yorickvP/check-busybox-standalone
configure.ac: check for sandbox-shell's FEATURE_SH_STANDALONE
2022-05-25 10:48:17 +02:00
Eelco Dolstra
a2911e2e4a Merge pull request #6565 from NixOS/dependabot/github_actions/zeebe-io/backport-action-0.0.8
Bump zeebe-io/backport-action from 0.0.7 to 0.0.8
2022-05-25 10:46:57 +02:00
Eelco Dolstra
40821f857d Merge pull request #6567 from Ma27/simplify-git-fix
libfetchers: drop `getGitDir` and hardcode `.git`
2022-05-25 10:46:42 +02:00
Yorick van Pelt
7e52472759 configure.ac: don't run sandbox-shell test when cross compiling 2022-05-24 17:00:27 +02:00
Yorick van Pelt
cbf60bec6f configure.ac: check for sandbox-shell's FEATURE_SH_STANDALONE
See also: https://bugs.archlinux.org/task/73998. Busybox's
FEATURE_SH_STANDALONE feature causes other busybox applets to
leak into the sandbox, where system() calls will start preferring
them over tools in $PATH. On arch, this even includes `ar`.

Let's check for this evil feature and disallow using this as a
sandbox shell.
2022-05-24 16:26:40 +02:00
Maximilian Bosch
b916c08feb libfetchers: drop getGitDir and hardcode .git
As discussed[1] this is most likely not desirable.

[1] https://github.com/NixOS/nix/pull/6440#issuecomment-1120876248
2022-05-24 14:20:48 +02:00
dependabot[bot]
d1c270431a Bump zeebe-io/backport-action from 0.0.7 to 0.0.8
Bumps [zeebe-io/backport-action](https://github.com/zeebe-io/backport-action) from 0.0.7 to 0.0.8.
- [Release notes](https://github.com/zeebe-io/backport-action/releases)
- [Commits](https://github.com/zeebe-io/backport-action/compare/v0.0.7...v0.0.8)

---
updated-dependencies:
- dependency-name: zeebe-io/backport-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-05-23 22:01:52 +00:00
Domen Kožar
4f98bc29ff Merge pull request #6490 from abathur/fix_macos_install_vim_again
darwin-install: fix break from bad vimrc
2022-05-23 21:33:42 +02:00
Ben Burdette
ba035f7dd0 comment 2022-05-23 10:13:47 -06:00
Théophane Hufschmitt
1e7c27cb91 Merge pull request #6560 from ncfavier/patch-1
typo: defaultApps → defaultApp
2022-05-23 17:08:06 +02:00
Théophane Hufschmitt
50ab1587bf Merge pull request #6532 from flox/show_warning
fix: alignment during flake show of legacyPackages
2022-05-23 09:30:26 +02:00
Ben Burdette
13d02af079 remove redundant 'debugMode' flag 2022-05-22 21:45:24 -06:00
Ben Burdette
7ccb2700c0 comments 2022-05-22 19:15:58 -06:00
Ben Burdette
5bc97fdfa6 change state derefs 2022-05-22 19:12:03 -06:00
Ben Burdette
34ffaa9f57 changning repl to use EvalState& instead of ref 2022-05-22 18:57:45 -06:00
Naïm Camille Favier
81a9bf0ad2 typo: defaultApps → defaultApp 2022-05-21 14:41:24 +02:00
Ben Burdette
982c8a959b remove special tryEval behavior 2022-05-20 12:45:36 -06:00
Ben Burdette
884d591787 debugRepl ftn pointer 2022-05-20 10:33:50 -06:00
tomberek
8c3939af14 Apply suggestions from code review
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2022-05-20 12:09:41 -04:00
Tom Bereknyei
7a04fb1c56 repl: add repl-flake experimental feature for gating 2022-05-20 08:20:00 -04:00
Tom Bereknyei
0053dab43f repl: fix tests to run on any testing store 2022-05-20 08:03:41 -04:00
Théophane Hufschmitt
938150472d Add some tests for the new REPL cli
- Test that without the XP feature things work as before
- Test that with or without the XP feature `--file file` works
- Test that with XP feature passing a flakeref works
- Test `:reload` with a flake
2022-05-20 07:42:06 -04:00
Tom Bereknyei
82c4af41e3 repl: clarify change and usage of <nixpkgs> 2022-05-20 01:49:49 -04:00
tomberek
542e36c6e7 Apply suggestions from code review
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2022-05-20 01:48:24 -04:00
Tom Bereknyei
db613a85fb repl: allow --file to always utilize autoargs 2022-05-20 01:35:06 -04:00
tomberek
7d7e00272a Apply suggestions from code review
Style fixes from @edolstra

Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2022-05-20 01:28:20 -04:00
Ben Burdette
0600df86b8 'debugMode' 2022-05-19 17:01:23 -06:00
Eelco Dolstra
5aeda91639 Merge pull request #6553 from Artturin/raisefilelimit
nix-daemon.service: sync LimitNOFILE with the nixos service
2022-05-19 22:12:11 +02:00
Ben Burdette
7ddef73d02 de-const evalState exceptions 2022-05-19 12:44:40 -06:00
Artturin
cebef6a250 nix-daemon.service: sync LimitNOFILE with the nixos service
5628480acd/nixos/modules/services/misc/nix-daemon.nix (L737)
Closes https://github.com/NixOS/nix/issues/6007
2022-05-19 21:16:07 +03:00
Ben Burdette
f9cdb6af8d Merge branch 'debug-exploratory-PR' into debuggerHook-eval-arg 2022-05-19 11:07:18 -06:00
Ben Burdette
357fb84dba use an expr->StaticEnv table in evalState 2022-05-19 10:48:10 -06:00
Tony Olagbaiye
5b8c1deb18 fetchTree: Allow fetching plain files
Add a new `file` fetcher type, which will fetch a plain file over
http(s), or from the local file.

Because plain `http(s)://` or `file://` urls can already correspond to
`tarball` inputs (if the path ends-up with a know archive extension),
the URL parsing logic is a bit convuluted in that:

- {http,https,file}:// urls will be interpreted as either a tarball or a
  file input, depending on the extensions of the path part (so
  `https://foo.com/bar` will be a `file` input and
  `https://foo.com/bar.tar.gz` as a `tarball` input)
- `file+{something}://` urls will be interpreted as `file` urls (with
  the `file+` part removed)
- `tarball+{something}://` urls will be interpreted as `tarball` urls (with
  the `tarball+` part removed)

Fix #3785

Co-Authored-By: Tony Olagbaiye <me@fron.io>
2022-05-19 18:24:49 +02:00
Tom Bereknyei
f21dec5bef repl: hide flake behavior behind flag and provide warning 2022-05-19 01:02:24 -04:00
Tom Bereknyei
e1f308a1ec repl: provide backward compat with legacy usage 2022-05-18 22:28:15 -04:00
Tom Bereknyei
7534798eed refactor: factor out getValue 2022-05-18 21:36:50 -04:00
tomberek
9f8c1183fa Apply suggestions from code review
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2022-05-18 21:21:58 -04:00
Tom Bereknyei
1ca3f6035d repl: update docs with installables 2022-05-18 21:21:38 -04:00
Tom Bereknyei
5640b52834 repl: use installables 2022-05-18 21:20:59 -04:00
Tom Bereknyei
81567a0962 repl: allow loading installables from CLI
repl: search installable with findAlongAttrPath
repl: refactor handling of args
repl: temp
2022-05-18 21:20:59 -04:00
Timothy DeHerrera
06d57ce759 nix repl: load flakes from cli args
If experimental feature "flakes" is enabled, args passed to `nix repl`
will now be considered flake refs and imported using the existing
`:load-flake` machinery.

In addition, `:load-flake` now supports loading flake fragments.
2022-05-18 21:20:59 -04:00
Théophane Hufschmitt
452ffe5464 Hint at the source file on conflict in flake new
Add a pointer to the source file (from the template) when `nix flake new` (or `init`) encounters an already existing file

Fix #6542
2022-05-18 16:46:13 +02:00
Théophane Hufschmitt
78dc64ec1e Merge pull request #6544 from ncfavier/getFlake-no-write-lock-file
Do not attempt to write a lock file in builtins.getFlake
2022-05-18 16:41:13 +02:00
Naïm Favier
169384abb2 Do not attempt to write a lock file in builtins.getFlake
Fixes https://github.com/NixOS/nix/issues/6541
2022-05-18 15:45:06 +02:00
Eelco Dolstra
51d13c43c1 Merge pull request #6543 from herberteuler/master
primop_match: fix example letter case in document
2022-05-18 10:56:11 +02:00
zhujun
b8e44dc62b primop_match: fix example letter case in document 2022-05-18 14:05:26 +08:00
Cole Helbling
685107c6c8 flake: use github: reference to nixpkgs
This allows flakes that don't override the Nixpkgs input and also have a
different flake registry.
2022-05-16 11:46:44 -07:00
Ben Burdette
667074b586 first whack at passing evalState as an arg to debuggerHook. 2022-05-16 09:20:51 -06:00
Théophane Hufschmitt
fdc97fc3f9 Merge pull request #6522 from elikoga/master
Add priority for nix profile install
2022-05-16 17:13:43 +02:00
Théophane Hufschmitt
43a2c13672 Make nix::eval_cache::int_t more idiomatic
Don’t explicitely give it a constructor, but use aggregate
initialization instead (also prevents having an implicit coertion, which
is probably good here)
2022-05-16 16:36:21 +02:00
Eli Kogan-Wang
e53349dd6e change priority conflict message 2022-05-16 16:16:06 +02:00
Eli Kogan-Wang
27d0f6747d resolve redundant priority passing, wrap NixInt in eval-cache variant 2022-05-16 15:17:35 +02:00
Eelco Dolstra
937a106100 Merge pull request #6517 from NixOS/document-the-libexpr-tests
Expand the testing section of the hacking docs
2022-05-16 13:19:03 +02:00
Eli Kogan-Wang
c81d24f1c7 Add int to eval-cache, bump eval cache schema version 2022-05-16 02:29:29 +02:00
elikoga
0cea59ab4b Merge branch 'NixOS:master' into master 2022-05-16 01:01:37 +02:00
Ben Burdette
86ba0a702c fix thunk issue 2022-05-15 12:05:51 -06:00
Ben Burdette
6faa56ea1f remove extra argument 2022-05-15 12:05:34 -06:00
Eli Kogan-Wang
be2b19041e Integrate review changes 2022-05-13 22:02:28 +02:00
Tom Bereknyei
8150b93968 fix: alignment during flake show of legacyPackages
Fixes:
https://github.com/NixOS/nix/issues/6240
https://github.com/NixOS/nix/issues/6045
2022-05-13 11:12:11 -04:00
Eelco Dolstra
bf89cd95a4 Merge pull request #6525 from J-Swift/feature/bash-prefix
Add `bash-prompt-prefix` option
2022-05-13 14:06:07 +02:00
Ben Burdette
2acdb90438 remove debug code 2022-05-12 14:20:45 -06:00
Ben Burdette
2d0d1ec99d remove debug code 2022-05-12 14:15:35 -06:00
Ben Burdette
4f48095c66 Merge branch 'debugThrow' into debug-exploratory-PR 2022-05-12 14:11:35 -06:00
Ben Burdette
1ea13084c9 template-ize debugThrow 2022-05-12 13:59:58 -06:00
Théophane Hufschmitt
65a913d29b Don’t recommend writing unit tests
As asked in <https://github.com/NixOS/nix/pull/6517#discussion_r869416905>
2022-05-12 12:02:31 +02:00
Eelco Dolstra
d354fc30b9 Merge branch 'fix-unrecognized-archive-format' of https://github.com/NobbZ/nix 2022-05-12 11:24:51 +02:00
Norbert Melzer
831e2743ea fix GitHub URL template 2022-05-12 00:56:39 +02:00
Théophane Hufschmitt
b944b588fa Merge pull request #6523 from ncfavier/stop-logger-legacy
Stop the logger properly in legacy commands
2022-05-11 16:53:57 +02:00
Naïm Favier
1461e6cdda Stop the logger properly in legacy commands
Ensures the logger is stopped on exit in legacy commands. Without this,
when using `nix-build --log-format bar` and stopping nix with CTRL+C,
the bar is not cleared from the screen.
2022-05-11 12:58:45 +02:00
Eli Kogan-Wang
aefc6c4f41 Add priority for nix profile install 2022-05-11 12:16:35 +02:00
Eelco Dolstra
54457382f9 Fix static build
https://hydra.nixos.org/build/176211267
2022-05-11 11:36:56 +02:00
Jimmy Reichley
584475acf9 Add documentation for bash-prompt-prefix 2022-05-10 16:55:25 -04:00
Jimmy Reichley
2998527b18 Allow setting bash-prompt-prefix nix develop configuration 2022-05-10 16:53:22 -04:00
Eelco Dolstra
eb957ad6d8 Merge pull request #6497 from danielfullmer/ghe-fetcher-url
Use correct URL for GitHub Enterprise
2022-05-10 19:23:22 +02:00
Eelco Dolstra
73b9b008f0 Merge pull request #6518 from edolstra/fix-nix-develop
nix develop: Find bin/bash in the bashInteractive outputs
2022-05-10 17:10:13 +02:00
Eelco Dolstra
a9cbc2857f nix develop: Find bin/bash in the bashInteractive outputs 2022-05-10 16:43:41 +02:00
Théophane Hufschmitt
7c75f1d52b Expand the testing section of the hacking docs
- Make it clear what the different kind of tests are, where they live
  and how they can be ran
- Ask people to primarily write unit tests
2022-05-10 13:35:23 +02:00
Eelco Dolstra
7062ebf5be Merge pull request #6510 from NixOS/dependabot/github_actions/docker/login-action-2
Bump docker/login-action from 1 to 2
2022-05-10 13:06:28 +02:00
dependabot[bot]
c060e93b3c Bump docker/login-action from 1 to 2
Bumps [docker/login-action](https://github.com/docker/login-action) from 1 to 2.
- [Release notes](https://github.com/docker/login-action/releases)
- [Commits](https://github.com/docker/login-action/compare/v1...v2)

---
updated-dependencies:
- dependency-name: docker/login-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-05-09 22:01:15 +00:00
Ben Burdette
7cd7c7c91a Merge branch 'master' into debug-exploratory-PR 2022-05-09 09:30:44 -06:00
Théophane Hufschmitt
3ec979fa90 Merge pull request #6505 from jtojnar/patch-3
libexpr: Fix manual link in error message
2022-05-09 11:09:32 +02:00
Jan Tojnar
59d9551c25 libexpr: Fix manual link in error message
It was changed to the old manual in 8895fa70a4
2022-05-08 18:59:00 +02:00
Daniel Fullmer
7a3d5b2ff0 Use correct URL for GitHub Enterprise
For GitHub Enterprise, the API is accessed through a slightly different
URL. See [1], where it says:

> Use http(s)://[hostname]/api/v3 to access the API for GitHub
> Enterprise Server.

Also tested working on a GHE instance.

[1] https://docs.github.com/en/enterprise-server@3.3/rest/guides/getting-started-with-the-rest-api
2022-05-06 13:13:11 -07:00
Andreas Rammhold
059ae7f6c4 Add unit tests for libexpr (#5377)
* libexpr: fix builtins.split example

The example was previously indicating that multiple whitespaces would be
collapsed into a single captured whitespace. That isn't true and was
likely a mistake when being documented initially.

* Fix segfault on unitilized list when printing value

Since lists are just chunks of memory the individual elements in the
list might be unitilized when a programming error happens within Nix.

In this case the values are null-initialized (at least with Boehm GC)
and we can avoid a nullptr deref when printing them.

I ran into this issue while ensuring that new expression tests would
show the actual value on an assertion failure.

This is unlikely to cause any runtime performance regressions as
printing values is not really in the hot path (unless the repl is the
primary use case).

* Add operator<< for ValueTypes

* Add libexpr tests

This introduces tests for libexpr that evalulate various trivial Nix
language expressions and primop invocations that should be good smoke
tests wheter or not the implementation is behaving as expected.
2022-05-06 18:05:27 +02:00
Ben Burdette
fc66f48812 debugError() 2022-05-06 09:09:49 -06:00
Ben Burdette
2c9fafdc9e trying debugThrow 2022-05-06 08:47:21 -06:00
Eelco Dolstra
b470218d9a renderMarkdownToTerminal(): Avoid line overflow
Lowdown doesn't respect '.cols' exactly (maybe because of the
whitespace in front of each line), so adjust .cols a bit.
2022-05-06 13:14:49 +02:00
Eelco Dolstra
33affa0a02 Merge pull request #6483 from NixOS/fix-sourcehut-ref-parsing
Fix the parsing of the sourcehut refs file
2022-05-06 12:24:55 +02:00
Ben Burdette
99d69ac23f fix repl bug 2022-05-05 21:23:03 -06:00
Ben Burdette
dea998b2f2 traceable_allocator 2022-05-05 20:26:10 -06:00
Ben Burdette
f400c5466d rename valmap 2022-05-05 15:43:23 -06:00
Ben Burdette
09fcfee925 don't print the 'break' argument 2022-05-05 15:34:59 -06:00
Ben Burdette
ce304d0154 rename debug commands to be more gdb-like; hide them except in debug mode 2022-05-05 15:24:57 -06:00
Ben Burdette
0ac121a094 Merge branch 'NixOS:master' into debug-exploratory-PR 2022-05-05 13:54:20 -06:00
Eelco Dolstra
58645a78ab builtins.break: Return argument when debugging is not enabled 2022-05-05 17:17:03 +02:00
Eelco Dolstra
dd8b91eebc Style fixes
In particular, use std::make_shared and enumerate(). Also renamed some
fields to fit naming conventions.
2022-05-05 17:17:03 +02:00
Eelco Dolstra
f4102de84b Merge pull request #6489 from alexshpilkin/fix-6488
nix: Add forgotten null check in AttrCursor::getListOfStrings()
2022-05-05 16:16:56 +02:00
Théophane Hufschmitt
4c5aa8520c Make sure that nix build works in --impure mode
Regression test for <https://github.com/NixOS/nix/issues/6488>
2022-05-05 14:53:59 +02:00
Eelco Dolstra
f2a15091c2 Merge pull request #6492 from thkoch2001/patch-2
Change json example to be original
2022-05-05 12:58:11 +02:00
thkoch2001
7388cd55bf Change json example to be original
Closes: #3391
2022-05-05 12:39:23 +03:00
Travis A. Everett
240124f7b1 darwin-install: fix break from bad vimrc
It looks like the `--noplugin` flag added in #5489 wasn't enough to
skirt this class of vim-init error, so this is swing 2 at a full fix.
Fixes #6462.
2022-05-04 19:26:03 -05:00
Alexander Shpilkin
b3ed32d0fd Add forgotten null check 2022-05-04 22:13:49 +03:00
Théophane Hufschmitt
e68676e6c8 Fix the parsing of the sourcehut refs file
Since a26be9f3b8, the same parser is used
to parse the result of sourcehut’s `HEAD` endpoint (coming from [git
dumb protocol]) and the output of `git ls-remote`. However, they are very
slightly different (the former doesn’t specify the current reference
since it’s implied to be `HEAD`).

Unify both, and make the parser a bit more robust and understandable (by
making it more typed and adding tests for it)

[git dumb protocol]: https://git-scm.com/book/en/v2/Git-Internals-Transfer-Protocols#_the_dumb_protocol
2022-05-04 14:38:59 +02:00
Eelco Dolstra
c98648bef0 Merge remote-tracking branch 'origin/master' into debug-exploratory-PR 2022-05-04 14:10:21 +02:00
Eelco Dolstra
470e27ce80 Merge pull request #6482 from edolstra/json-utils
Move json stuff out of util.cc
2022-05-04 11:58:20 +02:00
Eelco Dolstra
107613ad2b Fix compiler warning 2022-05-04 11:31:39 +02:00
Eelco Dolstra
3e87c8e62b Move json stuff out of util.cc 2022-05-04 11:22:06 +02:00
Alain Zscheile
1385b20078 Get rid of most .at calls (#6393)
Use one of `get` or `getOr` instead which will either return a null-pointer (with a nicer error message) or a default value when the key is missing.
2022-05-04 07:44:32 +02:00
Eelco Dolstra
9489b4b7ef Merge pull request #6449 from edolstra/outputs-spec
Allow selecting derivation outputs
2022-05-03 15:38:47 +02:00
Eelco Dolstra
a3c6c5b1c7 nix profile: Support overriding outputs 2022-05-03 15:00:34 +02:00
Eelco Dolstra
4a79cba511 Allow selecting derivation outputs using 'installable!outputs'
E.g. 'nixpkgs#glibc^dev,static' or 'nixpkgs#glibc^*'.
2022-05-03 13:43:52 +02:00
Eelco Dolstra
404c222444 Merge pull request #6426 from edolstra/respect-outputs-to-install
nix: Respect meta.outputsToInstall, and use all outputs by default
2022-05-03 13:43:22 +02:00
Théophane Hufschmitt
cae7473884 Merge pull request #6465 from layus/fix-test-crashes
tests: Distinguish crashes from expected failures
2022-05-03 11:11:37 +02:00
Guillaume Maudoux
275f8561eb tests/lang: Distinguish crashes from expected failures 2022-05-02 15:12:50 +02:00
Guillaume Maudoux
4845886bed Revert "tests: Distinguish crashes from expected failures"
This reverts commit 143b73f52d.
2022-05-02 14:22:00 +02:00
Eelco Dolstra
61289ceee3 Style fixes 2022-05-02 13:37:53 +02:00
Eelco Dolstra
564faa6b4e Merge pull request #6470 from Ma27/git-followup
libfetchers/git: fix every occasion of a permission error
2022-05-02 13:33:08 +02:00
Eelco Dolstra
a26be9f3b8 Merge pull request #4638 from orbekk/read_head
Resolve reference for remote git repositories (makes fetchGit work with non-'master' branch)
2022-05-02 13:31:30 +02:00
Eelco Dolstra
dde71899dd tests: Don't create tests/result
https://hydra.nixos.org/log/lns780srkka4dv7r69mn4zfy6fdij4yr-nix-2.9.0pre20220428_4bb111c.drv
2022-05-02 09:38:51 +02:00
Maximilian Bosch
1849e6a1f6 libfetchers/git: fix every occasion of a permission error
I'm afraid I missed a few problematic `git(1)`-calls while implementing
PR #6440, sorry for that! Upon investigating what went wrong, I realized
that I only tested against the "cached"-case by accident because my
git-checkout with my system's flake was apparently cached during my
debugging.

I managed to trigger the original issue again by running:

    $ git commit --allow-empty -m "tmp"
    $ sudo nixos-rebuild switch --flake .# -L --builders ''

Since `repoDir` points to the checkout that's potentially owned by
another user, I decided to add `--git-dir` to each call affecting
`repoDir`.

Since the `tmpDir` for the temporary submodule-checkout is created by
Nix itself, it doesn't seem to be an issue.

Sorry for that, it should be fine now.
2022-04-30 15:56:12 +02:00
Kjetil Orbekk
9bf296c970 Extract git reference parsing to a shared library
These utility functions can be shared between the git and github fetchers.
2022-04-29 18:46:21 -04:00
Kjetil Orbekk
c21afd684c Update nix flake documentation of ref handling
Update the documentation about how `ref` is resolved if it is not
specified.

Add a note about special handling of local workdirs with `git+file`.
2022-04-29 18:46:21 -04:00
Kjetil Orbekk
1203e48926 Store cached head in cached git repo
The previous head caching implementation stored two paths in the local
cache; one for the cached git repo and another textfile containing the
resolved HEAD ref. This commit instead stores the resolved HEAD by
setting the HEAD ref in the local cache appropriately.
2022-04-29 18:46:21 -04:00
Kjetil Orbekk
de54e1cd3f Refactor fetching of dirty workdir
Extract the handling of a local dirty workdir to a helper function.
2022-04-29 18:46:17 -04:00
Kjetil Orbekk
05a3fbac5a Test fetchGit with non-'master' remote repo 2022-04-29 18:42:31 -04:00
Kjetil Orbekk
401e60f289 Resolve reference for remote repository
Resolves the HEAD reference from the remote repository instead
of assuming "master".
2022-04-29 18:42:28 -04:00
Ben Burdette
c81ffa692e remove 'libnix' 2022-04-29 11:35:50 -06:00
Ben Burdette
c941803861 spacing 2022-04-29 11:27:38 -06:00
Ben Burdette
172a83d22a line endings 2022-04-29 11:24:54 -06:00
Ben Burdette
ca6cba8b81 fix 'suggestions' error 2022-04-29 10:51:10 -06:00
Ben Burdette
2a5632c70d incorporate PosIdx changes, symbol changes. 2022-04-29 10:02:17 -06:00
Guillaume Maudoux
143b73f52d tests: Distinguish crashes from expected failures 2022-04-29 11:14:08 +02:00
Ben Burdette
6e19947993 Merge branch 'master' into debug-merge-master 2022-04-28 12:32:57 -06:00
Eelco Dolstra
4bb111c8d4 Merge pull request #6461 from edolstra/libcxx
Fix libcxx build
2022-04-28 15:11:49 +02:00
Eelco Dolstra
70a30dbc00 Fix libcxx build
Fixes #6458.
2022-04-28 14:37:05 +02:00
Eelco Dolstra
cb5361628d Add libcxxStdenv devshell 2022-04-28 14:36:48 +02:00
Eelco Dolstra
d77d813017 Shut up clang warning 2022-04-28 14:24:17 +02:00
Eelco Dolstra
660835da69 Merge pull request #6459 from edolstra/fix-out-paths
Fix passing $OUT_PATHS to the post-build hook
2022-04-28 14:05:29 +02:00
Eelco Dolstra
4a9623b129 Fix passing $OUT_PATHS to the post-build hook
Fixes #6446.
2022-04-28 13:36:01 +02:00
Eelco Dolstra
35393dc2c6 Merge pull request #6444 from midchildan/feat/zsh-runhelp
feat: add integration with zsh's run-help
2022-04-26 21:26:33 +02:00
midchildan
68d51ae012 refactor: don't hardcode nix subcommands in run-help-nix 2022-04-27 01:56:10 +09:00
Eelco Dolstra
717298c749 Bump eval cache schema version 2022-04-26 17:17:51 +02:00
Eelco Dolstra
13d8400ac5 Remove obsolete FIXME 2022-04-26 17:17:27 +02:00
Eelco Dolstra
6e0a2b971b Add a test for outputsToInstall 2022-04-26 17:17:27 +02:00
Eelco Dolstra
1ddabe1a01 nix: Respect meta.outputsToInstall, and use all outputs by default
'nix profile install' will now install all outputs listed in the
package's meta.outputsToInstall attribute, or all outputs if that
attribute doesn't exist. This makes it behave consistently with
nix-env. Fixes #6385.

Furthermore, for consistency, all other 'nix' commands do this as
well. E.g. 'nix build' will build and symlink the outputs in
meta.outputsToInstall, defaulting to all outputs. Previously, it only
built/symlinked the first output. Note that this means that selecting
a specific output using attrpath selection (e.g. 'nix build
nixpkgs#libxml2.dev') no longer works. A subsequent PR will add a way
to specify the desired outputs explicitly.
2022-04-26 17:17:27 +02:00
Eelco Dolstra
a81622c21d Merge pull request #6447 from edolstra/eval-cache-symbols
EvalCache: Use Symbol in more places
2022-04-26 15:41:40 +02:00
Théophane Hufschmitt
fb5f13fb65 Merge pull request #6440 from Ma27/fix-nix-with-latest-git
libfetchers/git: hardcode `--git-dir`
2022-04-26 14:50:47 +02:00
Eelco Dolstra
b12c33510c EvalCache AttrKey: Use Symbol instead of std::string 2022-04-26 14:16:20 +02:00
Eelco Dolstra
474695975d EvalCache: Revert to using symbols in getAttr() 2022-04-26 14:01:42 +02:00
Eelco Dolstra
fab731a9d4 Don't pass Symbol by reference
Since Symbol is just an integer, passing it by const reference is
never advantageous.
2022-04-26 13:25:17 +02:00
Eelco Dolstra
29e52194a8 Merge branch 'symbolidx-symbol' of https://github.com/pennae/nix 2022-04-26 13:25:10 +02:00
pennae
d6d6bbd9ef Merge branch 'master' into lto 2022-04-25 14:02:37 +00:00
pennae
a385e51a08 rename SymbolIdx -> Symbol, Symbol -> SymbolStr
after #6218 `Symbol` no longer confers a uniqueness invariant on the
string it wraps, it is now possible to create multiple symbols that
compare equal but whose string contents have different addresses. this
guarantee is now only provided by `SymbolIdx`, leaving `Symbol` only as
a string wrapper that knows about the intricacies of how symbols need to
be formatted for output.

this change renames `SymbolIdx` to `Symbol` to restore the previous
semantics of `Symbol` to that name. we also keep the wrapper type and
rename it to `SymbolStr` instead of returning plain strings from lookups
into the symbol table because symbols are formatted for output in many
places. theoretically we do not need `SymbolStr`, only a function that
formats a string for output as a symbol, but having to wrap every symbol
that appears in a message into eg `formatSymbol()` is error-prone and
inconvient.
2022-04-25 15:37:01 +02:00
midchildan
0f7c7ab97b fix: typo make-content-addressable -> make-content-addressed 2022-04-25 02:34:59 +09:00
midchildan
c67d8876c3 feat: add integration with zsh's run-help 2022-04-25 02:21:41 +09:00
Maximilian Bosch
d1f5356311 libfetchers/git: fix for nixos-rebuild
The `--git-dir=` must be `.` in some cases (for cached repos that are
"bare" repos in `~/.cache/nix/gitv3`). With this fix we can add
`--git-dir` to each `git`-invokation needed for `nixos-rebuild`.
2022-04-24 18:14:24 +02:00
Maximilian Bosch
0256e5578e libfetchers/git: hardcode --git-dir
To demonstrate the problem:

* You need a `git` at 2.33.3 in your $PATH
* An expression like this in a git repository:

  ``` nix
  {
    outputs = { self, nixpkgs }: {
      packages.foo.x86_64-linux = with nixpkgs.legacyPackages.x86_64-linux;
        runCommand "snens" { } ''
          echo ${(builtins.fetchGit ./.).lastModifiedDate} > $out
        '';
    };
  }
  ```

Now, when instantiating the package via `builtins.getFlake`, it fails on
Nix 2.7 like this:

    $ nix-instantiate -E '(builtins.getFlake "'"$(pwd)"'").packages.foo.x86_64-linux'
    fatal: unsafe repository ('/nix/store/a7j3125km4h8l0p71q6ssfkxamfh5d61-source' is owned by someone else)
    To add an exception for this directory, call:

    	git config --global --add safe.directory /nix/store/a7j3125km4h8l0p71q6ssfkxamfh5d61-source
    error: program 'git' failed with exit code 128
    (use '--show-trace' to show detailed location information)

This breaks e.g. `nixops`-deployments using flakes with similar
expressions as shown above.

The cause for this is that `git(1)` tries to find the highest
`.git`-directory in the directory tree and if it finds a such a
directory, but with another owning user (root vs. the user who evaluates
the expression), it fails as above. This was changed recently to fix
CVE-2022-24765[1].

By explicitly specifying `--git-dir`, Git assumes to be in the top-level
directory and doesn't attempt to look for a `.git`-directory in the
parent directories and thus the code-path leading to said error is never
reached.

[1] https://lore.kernel.org/git/xmqqv8veb5i6.fsf@gitster.g/
2022-04-23 23:20:17 +02:00
Théophane Hufschmitt
7f814d6d9a Merge pull request #6259 from Artturin/nixbuildaddprintstorepaths
nix build: add --print-out-paths flag
2022-04-22 14:21:55 +02:00
Théophane Hufschmitt
197aa2fa96 Merge remote-tracking branch 'origin/master' into nixbuildaddprintstorepaths 2022-04-22 13:20:39 +02:00
Théophane Hufschmitt
92de62fe6f Merge pull request #6437 from NixOS/fix-darwin-build
Fix the darwin build
2022-04-22 13:19:53 +02:00
Théophane Hufschmitt
be28603dca Merge remote-tracking branch 'origin/master' into nixbuildaddprintstorepaths 2022-04-22 11:11:01 +02:00
Théophane Hufschmitt
7b889f31ea Fix the darwin build
Looks like the auto-merge is indeed quite broken and merges even when the CI fails
2022-04-22 10:56:56 +02:00
Théophane Hufschmitt
c4ffc8e2f8 Merge pull request #6218 from pennae/pos-symbol-tables
reduce the size of Attr from 3 pointers to 2 on 64 bit machines
2022-04-22 10:28:06 +02:00
Théophane Hufschmitt
484badfa09 Add some tests for ChunkedVector 2022-04-22 10:03:44 +02:00
Théophane Hufschmitt
7ca6fbc8ca Move ChunkedVector to its own header 2022-04-22 10:01:02 +02:00
Théophane Hufschmitt
35ca5fdf91 Merge pull request #6436 from flox/tofile_allow
fix: builtins.toFile adds path to allowedPaths
2022-04-22 08:50:54 +02:00
Tom Bereknyei
f25112d383 fix: builtins.toFile adds path to allowedPaths
The produced path is then allowed be imported or utilized elsewhere:
```
assert (43 == import (builtins.toFile "source" "43")); "good"
```

This will still fail on write-only stores.
2022-04-21 16:41:37 -04:00
pennae
8adaa6acb5 remove pos<T>
it's no longer needed now that positions aren't really pointers any
more.
2022-04-21 21:56:34 +02:00
pennae
8168a4cf4a shrink Attr by 8 bytes on 64bit machines
with position and symbol tables in place we can now shrink Attr by a full
pointer with some simple field reordering. since Attr is a very hot struct this
has substantial impact on memory use, decreasing GC allocations and heap size by
10-15% each. we also get a ~15% performance improvement due to reduced GC
loading.

pure parsing has taken a hit over the branch base because positions are now
slightly more expensive to create, but overall we get a noticeable improvement.

before (on memory-friendliness):

  Benchmark 1: nix search --no-eval-cache --offline ../nixpkgs hello
    Time (mean ± σ):      6.960 s ±  0.028 s    [User: 5.832 s, System: 0.897 s]
    Range (min … max):    6.886 s …  7.005 s    20 runs

  Benchmark 2: nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
    Time (mean ± σ):     328.1 ms ±   1.7 ms    [User: 295.8 ms, System: 32.2 ms]
    Range (min … max):   324.9 ms … 331.2 ms    20 runs

  Benchmark 3: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
    Time (mean ± σ):      2.688 s ±  0.029 s    [User: 2.365 s, System: 0.238 s]
    Range (min … max):    2.642 s …  2.742 s    20 runs

after:

  Benchmark 1: nix search --no-eval-cache --offline ../nixpkgs hello
    Time (mean ± σ):      6.902 s ±  0.039 s    [User: 5.844 s, System: 0.783 s]
    Range (min … max):    6.820 s …  6.956 s    20 runs

  Benchmark 2: nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
    Time (mean ± σ):     330.7 ms ±   2.2 ms    [User: 300.6 ms, System: 30.0 ms]
    Range (min … max):   327.5 ms … 334.5 ms    20 runs

  Benchmark 3: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
    Time (mean ± σ):      2.330 s ±  0.027 s    [User: 2.040 s, System: 0.234 s]
    Range (min … max):    2.272 s …  2.383 s    20 runs
2022-04-21 21:56:34 +02:00
pennae
8775be3393 store Symbols in a table as well, like positions
this slightly increases the amount of memory used for any given symbol, but this
increase is more than made up for if the symbol is referenced more than once in
the EvalState that holds it. on average every symbol should be referenced at
least twice (once to introduce a binding, once to use it), so we expect no
increase in memory on average.

symbol tables are limited to 2³² entries like position tables, and similar
arguments apply to why overflow is not likely: 2³² symbols would require as many
string instances (at 24 bytes each) and map entries (at 24 bytes or more each,
assuming that the map holds on average at most one item per bucket as the docs
say). a full symbol table would require at least 192GB of memory just for
symbols, which is well out of reach. (an ofborg eval of nixpks today creates
less than a million symbols!)
2022-04-21 21:56:31 +02:00
pennae
00a3280232 don't use Symbol in Pos to represent a path
PosTable deduplicates origin information, so using symbols for paths is no
longer necessary. moving away from path Symbols also reduces the usage of
symbols for things that are not keys in attribute sets, which will become
important in the future when we turn symbols into indices as well.
2022-04-21 21:46:10 +02:00
pennae
6526d1676b replace most Pos objects/ptrs with indexes into a position table
Pos objects are somewhat wasteful as they duplicate the origin file name and
input type for each object. on files that produce more than one Pos when parsed
this a sizeable waste of memory (one pointer per Pos). the same goes for
ptr<Pos> on 64 bit machines: parsing enough source to require 8 bytes to locate
a position would need at least 8GB of input and 64GB of expression memory. it's
not likely that we'll hit that any time soon, so we can use a uint32_t index to
locate positions instead.
2022-04-21 21:46:06 +02:00
pennae
34b72775cf make throw*Error member functions of EvalState
when we introduce position and symbol tables we'll need to do lookups to turn
indices into those tables into actual positions/symbols. having the error
functions as members of EvalState will avoid a lot of churn for adding lookups
into the tables for each caller.
2022-04-21 21:25:18 +02:00
pennae
39df15fb8e don't use full Pos for findPackageFilename/editorFor
only file and line of the returned position were ever used, it wasn't actually
used a position. as such we may as well use a path+int pair for only those two
values and remove a use of Pos that would not work well with a position table.
2022-04-21 21:25:18 +02:00
pennae
38de79fcf7 remove Bindings::need
a future commit will remove the ability to convert the symbol type used in
bindings to strings. since we only have two users we can inline the error check.
2022-04-21 21:25:18 +02:00
pennae
ff0fd91ed2 remove Symbol::empty
the only use of this function is to determine whether a lambda has a non-set
formal, but this use is arguably better served by Symbol::set and using a
non-Symbol instead of an empty symbol in the parser when no such formal is present.
2022-04-21 21:25:18 +02:00
pennae
90b5c0a1a6 turn primop names into strings
we don't *need* symbols here. the only advantage they have over strings is
making call-counting slightly faster, but that's a diagnostic feature and thus
needn't be optimized.

this also fixes a move bug that previously didn't show up: PrimOp structs were
accessed after being moved from, which technically invalidates them. previously
the names remained valid because Symbol copies on move, but strings are
invalidated. we now copy the entire primop struct instead of moving since primop
registration happen once and are not performance-sensitive.
2022-04-21 21:25:17 +02:00
Eelco Dolstra
3b9d31b88c Rename fmt test -> hilte 2022-04-21 13:00:50 +02:00
Eelco Dolstra
f1eee873ea Fix fmt test 2022-04-21 13:00:24 +02:00
Eelco Dolstra
dea76f581b Merge pull request #6433 from edolstra/hilite
Move hiliteMatches into a separate header
2022-04-21 12:21:54 +02:00
Eelco Dolstra
f05e1f6fbb Move hiliteMatches into a separate header
This is mostly so that we don't #include <regex> everywhere (which
adds quite a bit of compilation time).
2022-04-21 12:06:29 +02:00
Eelco Dolstra
684e679e07 Merge pull request #6416 from moduon/feat-openssh
feat: include openssh in docker image
2022-04-21 10:41:31 +02:00
Eelco Dolstra
ebad9213f7 Merge pull request #6431 from NixOS/unbreak-my-build
Make the default SQLiteError constructor public
2022-04-21 10:41:07 +02:00
Théophane Hufschmitt
d6efc07f85 Merge pull request #5479 from NixOS/selfref-ca
Fix the removal of ca-induced self-references
2022-04-21 10:30:22 +02:00
Théophane Hufschmitt
0155a3b3da Merge pull request #6324 from trofi/selfref-ca-index
ca: add sqlite index on `RealisationsRefs(realisationReference)`
2022-04-21 10:06:39 +02:00
regnat
86d7a11c6b Make sure to delete all the realisation refs
Deleting just one will only work in the test cases where I didn’t bother
creating too many of them :p
2022-04-21 10:06:39 +02:00
Sergei Trofimovich
975b0b52e7 ca: add sqlite index on RealisationsRefs(realisationReference)
Without the change any CA deletion triggers linear scan on large
RealisationsRefs table:

    sqlite>.eqp full
    sqlite> delete from RealisationsRefs where realisationReference IN ( select id from Realisations where outputPath = 1234567890 );
    QUERY PLAN
    |--SCAN RealisationsRefs
    `--LIST SUBQUERY 1
       `--SEARCH Realisations USING COVERING INDEX IndexRealisationsRefsOnOutputPath (outputPath=?)

With the change it gets turned into a lookup:

    sqlite> CREATE INDEX IndexRealisationsRefsRealisationReference on RealisationsRefs(realisationReference);
    sqlite> delete from RealisationsRefs where realisationReference IN ( select id from Realisations where outputPath = 1234567890 );
    QUERY PLAN
    |--SEARCH RealisationsRefs USING INDEX IndexRealisationsRefsRealisationReference (realisationReference=?)
    `--LIST SUBQUERY 1
       `--SEARCH Realisations USING COVERING INDEX IndexRealisationsRefsOnOutputPath (outputPath=?)
2022-04-21 10:06:39 +02:00
regnat
74d6782a6a Disable the selfref-gc test when the daemon is too old 2022-04-21 10:06:39 +02:00
Sergei Trofimovich
b6e59d7137 tests: remove 'ca-references' feature
The feature was ctabilized in d589a6aa8a.
2022-04-21 10:06:39 +02:00
regnat
92656da0b9 Fix the gc with indirect self-references via the realisations
If the derivation `foo` depends on `bar`, and they both have the same
output path (because they are CA derivations), then this output path
will depend both on the realisation of `foo` and of `bar`, which
themselves depend on each other.
This confuses SQLite which isn’t able to automatically solve this
diamond dependency scheme.

Help it by adding a trigger to delete all the references between the
relevant realisations.

Fix #5320
2022-04-21 10:06:39 +02:00
Sergei Trofimovich
6ada496311 nix: add (failing) selfreference test for multiple realizations
The test illustrates failure in issue #5320. Here derivation and
it's built input have identical CA sotre path. As a result we generate
extraneout reference to build input:

    $ make installcheck
    ...
    ran test tests/selfref-gc.sh... [PASS]
    ran test tests/ca/selfref-gc.sh... [FAIL]
    ...
        deleting '/tmp/.../tests/ca/selfref-gc/store/iqciq1mpg5hc7p6a52fp2bjxbyc9av0v-selfref-gc'
        deleting '/tmp/...tests/ca/selfref-gc/store/zh0kwpnirw3qbv6dl1ckr1y0kd5aw6ax-selfref-gc.drv'
        error: executing SQLite statement
          'delete from ValidPaths where path = '/tmp/.../tests/ca/selfref-gc/store/fsjq0k146r85lsh01l0icl30rnhv7z72-selfref-gc';':
            constraint failed (in '/tmp/.../tests/ca/selfref-gc/var/nix/db/db.sqlite')
2022-04-21 10:06:39 +02:00
Théophane Hufschmitt
e7d79c7861 Make the default SQLiteError constructor public
Otherwise the clang builds fail because the constructor of `SQLiteBusy`
inherits it, `SQLiteError::_throw` tries to call it, which fails.

Strangely, gcc works fine with it. Not sure what the correct behavior is
and who is buggy here, but either way, making it public is at the worst
a reasonable workaround
2022-04-21 09:40:55 +02:00
Eelco Dolstra
3db663a3e4 Merge pull request #6430 from NixOS/missing-realisation-error-message
Fix the error message in case of a missing realisation
2022-04-21 09:40:24 +02:00
Théophane Hufschmitt
445ddebde5 Fix the error message in case of a missing realisation
Don’t say that the derivation is CA as it might happen on a non-ca
derivation too.

Technically we could always recover _something_ for a purely
input-addressed derivation (like we already do when the `ca-derivations`
xp feature isn’t enabled), but it seems better to consistently fail −
the end-result wouldn’t really make sense anyways in most cases.
2022-04-21 09:27:16 +02:00
Théophane Hufschmitt
9345b4e9ca Merge pull request #3720 from obsidiansystems/fix-url-format
Avoid `fmt` when constructor already does it
2022-04-20 20:01:37 +02:00
John Ericson
f63b0f4540 Actually, solve this in a lighter-weight way
The templating is very superficial
2022-04-20 17:37:59 +00:00
John Ericson
05ec0beb40 Move templated functions to sqlite-impl.hh
This ensures that use-sites properly trigger new monomorphisations on
one hand, and on the other hand keeps the main `sqlite.hh` clean and
interface-only. I think that is good practice in general, but in this
situation in particular we do indeed have `sqlite.hh` users that don't
need the `throw_` function.
2022-04-20 16:57:06 +00:00
John Ericson
3c220442ff Merge remote-tracking branch 'upstream/master' into fix-url-format 2022-04-20 16:53:16 +00:00
Artturin
51cfea8bb0 nix build: add --print-out-paths flag
has the same functionality as default nix-build

$ nix-build . -A "bash" -A "bash.dev" -A "tinycc"
/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12
/nix/store/c49i1ggnr5cc8gxmk9xm0cn961z104dn-bash-5.1-p12-dev
/nix/store/dbapb08862ajgaax3621fz8hly9fdah3-tcc-0.9.27+date=2022-01-11

$ nix-build . -A "bash"
/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12

$ $HOME/nixgits/nix/result/bin/nix build "nixpkgs#bash" "nixpkgs#bash.dev" "nixpkgs#tinycc" --print-out-paths
/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12
/nix/store/c49i1ggnr5cc8gxmk9xm0cn961z104dn-bash-5.1-p12-dev
/nix/store/dbapb08862ajgaax3621fz8hly9fdah3-tcc-0.9.27+date=2022-01-11

$ $HOME/nixgits/nix/result/bin/nix build "nixpkgs#bash" --print-out-paths
/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12
2022-04-20 19:35:46 +03:00
Théophane Hufschmitt
2ffc5a4542 Merge pull request #6425 from yorickvP/fix-6424
Add custom to_json and from_json functions for ExperimentalFeature
2022-04-20 16:37:44 +02:00
Yorick van Pelt
ebf2fd76b1 Add custom to_json and from_json functions for ExperimentalFeature
nix show-config --json was serializing experimental features as ints.
nlohmann::json will automatically use these definitions to serialize
and deserialize ExperimentalFeatures.

Strictly, we don't use the from_json instance yet, it's provided for
completeness and hopefully future use.
2022-04-20 15:41:01 +02:00
Eelco Dolstra
cd0549a9cd Merge pull request #6419 from ckiee/repl-build-symlink
nix repl: make symlinks with the :bl command
2022-04-20 10:11:36 +02:00
ckie
0e2b01b14e nix repl: make symlinks with the :bl command
Requested by ppepino on the Matrix:
https://matrix.to/#/!KqkRjyTEzAGRiZFBYT:nixos.org/$Tb32BS3rVE2BSULAX4sPm0h6CDewX2hClOTGzTC7gwM?via=nixos.org&via=matrix.org&via=nixos.dev

This adds a new command, :bl, which works like :b but also creates
a GC root symlink to the various derivation outputs.

ckie@cookiemonster ~/git/nix -> ./outputs/out/bin/nix repl
Welcome to Nix 2.6.0. Type :? for help.

nix-repl> :l <nixpkgs>
Added 16118 variables.

nix-repl> :b runCommand "hello" {} "echo hi > $out"

This derivation produced the following outputs:
  ./repl-result-out -> /nix/store/kidqq2acdpi05c4a9mlbg2baikmzik44-hello
[1 built, 0.0 MiB DL]
ckie@cookiemonster ~/git/nix -> cat ./repl-result-out
hi
2022-04-20 00:20:29 +03:00
Eelco Dolstra
ee57f91413 Bump version 2022-04-19 21:48:17 +02:00
Eelco Dolstra
a3c843e635 Fix 'nix fmt' test 2022-04-19 21:47:13 +02:00
Eelco Dolstra
1cdad1074c Move rl-next.md to rl-2.8.md 2022-04-19 21:12:33 +02:00
Eelco Dolstra
c9e58aa5ff Require formatters to be packages
Because of 9b41239d8f, a formatter can
no longer be a package *or* an app. So let's require it to be a
package for now.
2022-04-19 20:48:13 +02:00
Eelco Dolstra
48a467f2b9 Merge branch 'issue-6075' of https://github.com/kamadorueda/nix 2022-04-19 20:21:32 +02:00
Eelco Dolstra
51712bf012 Merge pull request #6128 from ncfavier/fix-completion
Shell completion improvements
2022-04-19 13:45:33 +02:00
Eelco Dolstra
2016b7142a Fix compilation, style fixes 2022-04-19 13:41:32 +02:00
Eelco Dolstra
b529a41814 Merge branch 'make-flake-show-more-lenient-on-apps' of https://github.com/flox/nix 2022-04-19 13:41:24 +02:00
Eelco Dolstra
1b43d64648 Merge pull request #6415 from aakropotkin/doc.connect-timeout.default
doc: document nix.conf connect-timeout default
2022-04-19 12:27:30 +02:00
Eelco Dolstra
018db1c420 Merge pull request #6404 from edolstra/unify-flake-attr-eval
Make InstallableFlake::toValue() and toDerivation() behave consistently
2022-04-19 11:53:27 +02:00
Eelco Dolstra
cabd07d8b1 Merge pull request #6418 from rycee/doc/tgz-tarball
Add .tgz as tarball extension in documentation
2022-04-19 11:49:42 +02:00
John Ericson
75b62e5260 Avoid fmt when constructor already does it
There is a correctnes issue here, but #3724 will fix that. This is just
a cleanup for brevity's sake.
2022-04-19 01:44:11 +00:00
Robert Helgesson
8b659eacce Add .tgz as tarball extension in documentation
Support for the `tgz` shorthand was added in
52f5fa948a.
2022-04-18 17:14:15 +02:00
Alex Ameen
e5c934cd48 doc: rephrase connect-timeout help message
Co-authored-by: Cole Helbling <cole.e.helbling@outlook.com>
2022-04-17 18:17:37 -05:00
Alex Ameen
25c85f5a0e doc: document nix.conf connect-timeout default 2022-04-17 17:14:38 -05:00
Ben Burdette
93b8d31508 Merge branch 'master' into debug-exploratory-PR 2022-04-15 14:49:08 -06:00
Eelco Dolstra
b135de2b5f Merge pull request #6323 from erikarvstedt/eval-read-only
`nix eval`: Add option `--read-only`
2022-04-15 10:31:05 +02:00
Tom Bereknyei
9b41239d8f fix: ensure apps are apps and packages are packages 2022-04-14 23:57:52 -04:00
Eelco Dolstra
d6effddd3b Merge pull request #6387 from Uthar/fix
assert hash types for Git and Mercurial
2022-04-14 14:55:27 +02:00
Jairo Llopis
aa3927f0f1 feat: include openssh in docker image
When leveraging remote builders or cache in CI workloads, sometimes you need to configure nix to connect via SSH to a remote server.

It is the case for example when using nixbuild.net.

By including `openssh` package, CI should be able to reach remote builders when configured i.e. with environment variables.
2022-04-14 13:49:47 +01:00
Eelco Dolstra
d89840b103 Make InstallableFlake::toValue() and toDerivation() behave consistently
In particular, this means that 'nix eval` (which uses toValue()) no
longer auto-calls functions or functors (because
AttrCursor::findAlongAttrPath() doesn't).

Fixes #6152.

Also use ref<> in a few places, and don't return attrpaths from
getCursor() because cursors already have a getAttrPath() method.
2022-04-14 14:07:04 +02:00
Eelco Dolstra
0e58affd39 Merge pull request #6213 from NixOS/dependabot/github_actions/actions/checkout-3
Bump actions/checkout from 2 to 3
2022-04-13 14:36:04 +02:00
dependabot[bot]
dc9510c8d7 Bump actions/checkout from 2 to 3
Bumps [actions/checkout](https://github.com/actions/checkout) from 2 to 3.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v2...v3)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-04-13 12:10:29 +00:00
Eelco Dolstra
727cf888c2 Merge pull request #6401 from NixOS/dependabot/github_actions/cachix/install-nix-action-17
build(deps): bump cachix/install-nix-action from 16 to 17
2022-04-13 14:09:31 +02:00
Kasper Gałkowski
2769e43f61 assert hash types for Git and Mercurial 2022-04-12 21:13:14 +02:00
dependabot[bot]
5fc73c276b build(deps): bump cachix/install-nix-action from 16 to 17
Bumps [cachix/install-nix-action](https://github.com/cachix/install-nix-action) from 16 to 17.
- [Release notes](https://github.com/cachix/install-nix-action/releases)
- [Commits](https://github.com/cachix/install-nix-action/compare/v16...v17)

---
updated-dependencies:
- dependency-name: cachix/install-nix-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-04-11 22:01:20 +00:00
Eelco Dolstra
f7276bc948 Merge pull request #6392 from danpls/fix-actualUrl-mercurial
libfetchers: Fix assertion (Mercurial)
2022-04-11 11:16:04 +02:00
Eelco Dolstra
1f5d8e590e Merge pull request #6384 from mschwaig/confirm-multi-user-install-without-systemd
installer: ask for confirmation on multi-user install without systemd
2022-04-11 11:15:27 +02:00
Eelco Dolstra
092f6d2e7a Merge pull request #6380 from thufschmitt/fix-double-slahsh-in-uri
Allow empty path segments in urls
2022-04-11 11:15:14 +02:00
Eelco Dolstra
2311868aaa Merge pull request #6391 from danpls/replace-regex
libfetchers: Replace regex to clarify intent
2022-04-11 11:14:10 +02:00
Eelco Dolstra
734019ce56 Merge pull request #6397 from sebastianblunt/builderlogging
Log builder args and environment variables
2022-04-11 11:13:35 +02:00
Sebastian Blunt
63d9a81819 Log builder args and environment variables
Previously it only logged the builder's path, this changes it to log the
arguments at the same log level, and the environment variables at the
vomit level.

This helped me debug https://github.com/svanderburg/node2nix/issues/75
2022-04-10 21:10:37 -07:00
Ben Burdette
8b197c492e remove comma 2022-04-09 21:54:41 -06:00
Daniel Pauls
38125a47ab Test fetchMercurial with path containing a . segment 2022-04-09 23:39:00 +02:00
Daniel Pauls
d6b7529579 libfetchers: Fix assertion (Mercurial)
See commit 1e1cd6e7a for more information.
2022-04-09 19:10:23 +02:00
Daniel Pauls
770f7371f3 libfetchers: Replace regex to clarify intent 2022-04-09 17:00:14 +02:00
Ben Burdette
a61841ac41 don't use std::map merge 2022-04-09 07:45:23 -06:00
Ben Burdette
f5757a0804 revise command help 2022-04-08 16:34:20 -06:00
Ben Burdette
3aaf02839f trace stack, not call stack 2022-04-08 16:22:27 -06:00
Ben Burdette
31bcd55626 clean up makefiles 2022-04-08 15:53:24 -06:00
Ben Burdette
27d45f9eb3 minor cleanup 2022-04-08 15:46:12 -06:00
Ben Burdette
a86c2a8481 remove 'debugError', dead code 2022-04-08 13:30:18 -06:00
Ben Burdette
b8b8ec7101 move throw to preverve Error type; turn off debugger for tryEval 2022-04-08 12:34:27 -06:00
Théophane Hufschmitt
646af7325d Merge pull request #6376 from Uthar/master
don't assume that rev is a SHA1 hash
2022-04-08 17:56:27 +02:00
Théophane Hufschmitt
f3d3587ab3 Allow empty path segments in urls
Valid per https://datatracker.ietf.org/doc/html/rfc3986#section-3.3 (and
also somewhat frequently happening for local paths)
2022-04-08 16:09:49 +02:00
Eelco Dolstra
a52e369c07 Merge pull request #6382 from edolstra/remove-error-name
Remove unused "name" field from Error
2022-04-08 12:18:47 +02:00
Eelco Dolstra
c68963eaea Remove duplicate "error:" 2022-04-08 11:48:30 +02:00
Eelco Dolstra
8bd9ebf52c Error: Remove unused sname() method 2022-04-08 11:31:51 +02:00
Eelco Dolstra
168ef9f3ab Remove unused Error.name field 2022-04-08 11:31:51 +02:00
Martin Schwaighofer
4f29cf1a1d installer: ask for confirmation on multi-user install without systemd
On Linux a user can go through all the way through the multi-user install
and find out at the end that they now have to manually configure their
init system to launch the nix daemon.

I suspect that for a significant number of users this is not
what they wanted. They might prefer a single-user install.
Now they have to manually uninstall nix before they can
go through the single-user install.

This introduces a confirmation dialog before the install
in that specific situation to make sure that they want to proceed.

See also: https://github.com/NixOS/nix/issues/4999#issuecomment-1064188080
This closes #4999 but rejecting it and closing that issue anyways
would also be valid.
2022-04-08 11:23:54 +02:00
Ben Burdette
1a93ac8133 Merge remote-tracking branch 'upstream/master' into upstream-merge 2022-04-07 13:42:01 -06:00
Ben Burdette
d2ec9b4e15 in debugger mode, print the current error when another repl returns. 2022-04-07 12:09:47 -06:00
Ben Burdette
50b52d5110 remove debug code 2022-04-07 12:03:18 -06:00
Kasper Gałkowski
2c2fd4946f don't assume that rev is a SHA1 hash
This was a problem when writing a fetcher that uses e.g. sha256 hashes
for revisions. This doesn't actually do anything new, but allows for
creating such fetchers in the future (perhaps when support for Git's
SHA256 object format gains more popularity).
2022-04-07 19:49:47 +02:00
Ben Burdette
d29af88d58 newline before env 2022-04-07 11:17:57 -06:00
Ben Burdette
f37562187f free valmap on exit 2022-04-07 11:17:31 -06:00
Eelco Dolstra
8b1e328d5d Merge pull request #6348 from cole-h/fix-restoring-mount-namespace
libutil: Fix restoring mount namespace
2022-04-07 18:15:33 +02:00
Théophane Hufschmitt
b53e0a6aa0 Merge pull request #6374 from danpls/fix-actualUrl
libfetchers: Fix assertion
2022-04-07 17:57:24 +02:00
Théophane Hufschmitt
305d3a0ec3 Test fetchgit with path containing a . segment 2022-04-07 17:31:12 +02:00
Théophane Hufschmitt
e80dd0b600 Merge pull request #6375 from rehno-lindeque/nixosmodules-dot-default
Rename `nixosModule` to `nixosModules.default` consistent with other outputs
2022-04-07 13:01:19 +02:00
Ben Burdette
5cfd038bd8 show expr pos if DebugTrace one is noPos 2022-04-06 19:08:29 -06:00
Rehno Lindeque
5ff4c42608 Update release notes 2022-04-06 12:24:35 -04:00
Rehno Lindeque
b9c969a866 nix flake check: Warn about deprecated nixosModule output 2022-04-06 12:20:39 -04:00
Daniel Pauls
1e1cd6e7a9 libfetchers: Fix assertion
The filter expects all paths to have a prefix of the raw `actualUrl`, but
`Store::addToStore(...)` provides absolute canonicalized paths.
To fix this create an absolute and canonicalized path from the `actualUrl` and
use it instead.

Fixes #6195.
2022-04-06 17:33:23 +02:00
Eelco Dolstra
f01e33f283 Merge pull request #6372 from edolstra/curl-fail
Installer: Use curl --fail so we don't silently ignore download errors
2022-04-06 15:02:20 +02:00
Eelco Dolstra
a7b12c6bd9 curl: Use --fail to catch errors 2022-04-06 13:34:25 +02:00
Eelco Dolstra
36c1e3bc5f Merge pull request #6371 from edolstra/substitution-error-msg
Fix empty 'nix copy' error message
2022-04-06 13:17:01 +02:00
Eelco Dolstra
318936366d Fix empty 'nix copy' error message
This was caused by SubstitutionGoal not setting the errorMsg field in
its BuildResult. We now get a more descriptive message than in 2.7.0, e.g.

  error: path '/nix/store/13mh...' is required, but there is no substituter that can build it

instead of the misleading (since there was no build)

  error: build of '/nix/store/13mh...' failed

Fixes #6295.
2022-04-06 12:43:53 +02:00
Eelco Dolstra
fbeb8fd1b4 Merge pull request #6370 from edolstra/fetch-closure-query-params
fetchClosure: Don't allow URL query parameters
2022-04-06 12:22:39 +02:00
Eelco Dolstra
589f6f267b fetchClosure: Don't allow URL query parameters
Allowing this is a potential security hole, since it allows the user
to specify parameters like 'local-nar-cache'.
2022-04-06 11:52:51 +02:00
Eelco Dolstra
c0ad86f681 Merge pull request #6366 from danpls/base64-reserve
libutil: Reserve memory when en/decoding base64
2022-04-05 23:20:33 +02:00
Eelco Dolstra
f89fa29914 Merge pull request #6367 from danpls/fix-npos
tokenizeString: Fix semantic mistake
2022-04-05 23:19:35 +02:00
Daniel Pauls
513652d594 tokenizeString: Fix semantic mistake
`string_view::find_first_not_of(...)` and
`string_view::find_first_of(...)` return `string_view::npos` on error
not `string::npos`.
2022-04-05 22:33:03 +02:00
Daniel Pauls
1fa0393479 libutil: Reserve memory when en/decoding base64
The size of the output when encoding to and decoding from base64 is
(roughly) known so we can allocate it in advance to prevent
reallocation.
2022-04-05 21:30:50 +02:00
Eelco Dolstra
27b952a8a1 Merge pull request #6362 from thufschmitt/verbose-doctor
doctor: Always show the output
2022-04-05 17:33:10 +02:00
Eelco Dolstra
71c07ac0e7 Merge pull request #6363 from thufschmitt/definition-list-in-nix.conf-manual
Add anchors to the nix.conf options in the manual
2022-04-05 17:32:48 +02:00
Eelco Dolstra
5fe4fe823c Merge pull request #6365 from edolstra/update-nixpkgs
Update to latest Nixpkgs 21.05
2022-04-05 17:23:23 +02:00
Eelco Dolstra
8d6c937d6a flake.lock: Update
Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/82891b5e2c2359d7e58d08849e4c89511ab94234' (2021-09-28)
  → 'github:NixOS/nixpkgs/530a53dcbc9437363471167a5e4762c5fcfa34a1' (2022-02-19)
2022-04-05 16:41:40 +02:00
Eelco Dolstra
f98d76ff1a rl-2.7.md: Fix title 2022-04-05 14:13:26 +02:00
Théophane Hufschmitt
9a640afc1e doctor: Always show the output
Fix https://github.com/NixOS/nix/issues/6342
2022-04-05 14:04:01 +02:00
Théophane Hufschmitt
660c19eb49 manual: Add some anchor targets for the nix.conf options
For each `nix.conf` option, add an empty html node with a unique `id`
that can be used as an anchor target. Also make the name of the option
be a link to that target so that it’s easily discoverable.

We can’t rewrite the whole list as an html definition list like it’s
done for the builtins because these options also appear in a man page,
and the manpage renderer (lowdown) can’t render arbitrary html. But the
hack here allows to keep the manpage and have the links in the html
version.

Fix https://github.com/NixOS/nix/issues/5745
2022-04-05 13:50:42 +02:00
Eelco Dolstra
ec90fc4d1f Merge pull request #6360 from thufschmitt/flake-check-accept-welcomeText
Allow `welcomeText` when checking a flake template
2022-04-05 11:50:45 +02:00
Théophane Hufschmitt
5abe3f4aa6 Allow welcomeText when checking a flake template
Fix https://github.com/NixOS/nix/issues/6321
2022-04-05 11:03:43 +02:00
Cole Helbling
56009b2639 libutil: don't save cwd fd, use path instead
Saving the cwd fd didn't actually work well -- prior to this commit, the
following would happen:

    : ~/w/vc/nix ; doas outputs/out/bin/nix --experimental-features 'nix-command flakes' run nixpkgs#coreutils -- --coreutils-prog=pwd
    pwd: couldn't find directory entry in ‘../../../..’ with matching i-node
    : ~/w/vc/nix ; doas outputs/out/bin/nix --experimental-features 'nix-command flakes' develop -c pwd
    pwd: couldn't find directory entry in ‘../../../..’ with matching i-node
2022-04-04 10:28:08 -07:00
Cole Helbling
10b9c1b2b2 libutil: save cwd fd in restoreMountNamespace
This doesn't work very well (maybe I'm misunderstanding the desired
implementation):

    : ~/w/vc/nix ; doas outputs/out/bin/nix --experimental-features 'nix-command flakes' develop -c pwd
    pwd: couldn't find directory entry in ‘../../../..’ with matching i-node
2022-04-04 10:28:00 -07:00
Cole Helbling
f89b0f7846 libutil: try restoring the cwd from fdSavedCwd 2022-04-04 08:33:59 -07:00
Cole Helbling
e135d223f6 libutil: save fd to cwd instead of cwd itself 2022-04-04 08:32:45 -07:00
Cole Helbling
e5b70d47aa libutil: cleanup savedCwd logic
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2022-04-04 08:20:11 -07:00
Eelco Dolstra
a4a1de69dc Add missing #include 2022-04-04 16:49:39 +02:00
Eelco Dolstra
e496241413 Merge pull request #6350 from flox/lock_installable
fix(run): set applyNixConfig lockFlag
2022-04-04 10:50:17 +02:00
Tom Bereknyei
c1e2ce4515 fix(run): set applyNixConfig lockFlag 2022-04-01 23:41:34 -04:00
Cole Helbling
2a45cf54e4 libutil: Properly guard self-allocating getcwd on GNU
It's a GNU extension, as pointed out by pennae.
2022-04-01 12:20:34 -07:00
Cole Helbling
7f5caaa7c0 libutil: Don't use std::filesystem
Just in case making libutil depend on std::filesystem is unacceptable,
here is the non-filesystem approach.
2022-04-01 10:24:31 -07:00
aszlig
435848cef1 libutil: Fix restoring mount namespace
I regularly pass around simple scripts by using nix-shell as the script
interpreter, eg. like this:

    #!/usr/bin/env nix-shell
    #!nix-shell -p dd_rescue coreutils bash -i bash

While this works most of the time, I recently had one occasion where it
would not and the above would result in the following:

    $ sudo ./myscript.sh
    bash: ./myscript.sh: No such file or directory

Note the "sudo" here, because this error only occurs if we're root.

The reason for the latter is because running Nix as root means that we
can directly access the store, which makes sure we use a filesystem
namespace to make the store writable. XXX - REWORD!

So when stracing the process, I stumbled on the following sequence:

    openat(AT_FDCWD, "/proc/self/ns/mnt", O_RDONLY) = 3
    unshare(CLONE_NEWNS)                            = 0
    ... later ...
    getcwd("/the/real/cwd", 4096)                   = 14
    setns(3, CLONE_NEWNS)                           = 0
    getcwd("/", 4096)                               = 2

In the whole strace output there are no calls to chdir() whatsoever, so
I decided to look into the kernel source to see what else could change
directories and found this[1]:

    /* Update the pwd and root */
    set_fs_pwd(fs, &root);
    set_fs_root(fs, &root);

The set_fs_pwd() call is roughly equivalent to a chdir() syscall and
this is called when the setns() syscall is invoked[2].

[1]: b14ffae378/fs/namespace.c (L4659)
[2]: b14ffae378/kernel/nsproxy.c (L346)
2022-04-01 09:30:52 -07:00
Eelco Dolstra
bf4895961d Merge pull request #6344 from flox/profile_url_uri
profile!: consistent use of url/uri. create new version
2022-04-01 14:38:32 +02:00
Eelco Dolstra
c74eac9fde Merge pull request #6347 from edolstra/fix-output-hash-algo
Fix handling of outputHash when outputHashAlgo is not specified
2022-04-01 13:18:26 +02:00
Eelco Dolstra
fdfe737867 Fix handling of outputHash when outputHashAlgo is not specified
https://hydra.nixos.org/build/171351131
2022-04-01 12:40:49 +02:00
Théophane Hufschmitt
a24d0777b0 Merge pull request #6343 from Artturin/anothertypo
scripts/install-systemd-multi-user.sh: fix another typo
2022-04-01 11:44:58 +02:00
Artturin
7492030ed7 scripts/install-systemd-multi-user.sh: fix another typo 2022-03-31 22:14:53 +03:00
Eelco Dolstra
c9a29d0d92 Merge pull request #6227 from NixOS/impure-derivations-ng
Impure derivations
2022-03-31 19:58:35 +02:00
Ben Burdette
f41c18b221 comments 2022-03-31 09:39:18 -06:00
Eelco Dolstra
6377442c98 tests/impure-derivations.sh: Ensure that inputAddressed build fails 2022-03-31 17:38:15 +02:00
Ben Burdette
1096d17b65 show 'with' bindings as well as static 2022-03-31 09:37:36 -06:00
Eelco Dolstra
d63a5f5dd3 Update release notes 2022-03-31 17:33:06 +02:00
Eelco Dolstra
7537097284 Provide default values for outputHashAlgo and outputHashMode 2022-03-31 16:56:44 +02:00
Eelco Dolstra
a99af85a77 Fix macOS build 2022-03-31 16:39:18 +02:00
Eelco Dolstra
6051cc954b Rename 'pure' -> 'sandboxed' for consistency 2022-03-31 16:12:25 +02:00
Eelco Dolstra
e279fbb16a needsNetworkAccess() -> isSandboxed() 2022-03-31 16:06:40 +02:00
Eelco Dolstra
4e043c2f32 Document isPure() 2022-03-31 16:01:50 +02:00
Eelco Dolstra
d7fc33c842 Fix macOS build 2022-03-31 15:59:14 +02:00
Eelco Dolstra
162beb2595 Fix test 2022-03-31 13:43:20 +02:00
Eelco Dolstra
b2ae922747 tests/impure-derivations.sh: Restart daemon 2022-03-31 13:43:20 +02:00
Eelco Dolstra
18935e8b9f Support fixed-output derivations depending on impure derivations 2022-03-31 13:43:20 +02:00
Eelco Dolstra
5cd72598fe Add support for impure derivations
Impure derivations are derivations that can produce a different result
every time they're built. Example:

  stdenv.mkDerivation {
    name = "impure";
    __impure = true; # marks this derivation as impure
    outputHashAlgo = "sha256";
    outputHashMode = "recursive";
    buildCommand = "date > $out";
  };

Some important characteristics:

* This requires the 'impure-derivations' experimental feature.

* Impure derivations are not "cached". Thus, running "nix-build" on
  the example above multiple times will cause a rebuild every time.

* They are implemented similar to CA derivations, i.e. the output is
  moved to a content-addressed path in the store. The difference is
  that we don't register a realisation in the Nix database.

* Pure derivations are not allowed to depend on impure derivations. In
  the future fixed-output derivations will be allowed to depend on
  impure derivations, thus forming an "impurity barrier" in the
  dependency graph.

* When sandboxing is enabled, impure derivations can access the
  network in the same way as fixed-output derivations. In relaxed
  sandboxing mode, they can access the local filesystem.
2022-03-31 13:43:20 +02:00
Eelco Dolstra
0fe8849914 Merge pull request #6337 from danpls/fix-to-json-repl
libexpr: Throw the correct error in toJSON
2022-03-31 11:33:16 +02:00
Eelco Dolstra
28309352d9 replaceEnv(): Pass newEnv by reference 2022-03-31 10:39:53 +02:00
Eelco Dolstra
212623195c Merge pull request #6339 from flox/bundler_default
bundler: update default bundler to support new bundler API
2022-03-31 10:10:11 +02:00
Tom Bereknyei
50f9f335c9 profile!: consistent use of url/uri. create new version 2022-03-30 16:35:26 -04:00
Tom Bereknyei
d77823b502 bundler: update default bundler to support new bundler API 2022-03-30 16:10:42 -04:00
Daniel Pauls
629edd43ba libutil: Change return value of addTrace to void
The return value of BaseError::addTrace(...) is never used and
error-prone as subclasses calling it will return a BaseError instead of
the subclass.
This commit changes its return value to be void.
2022-03-30 18:37:32 +02:00
Daniel Pauls
fa83b865a2 libexpr: Throw the correct error in toJSON
BaseError::addTrace(...) returns a BaseError, but we want to
throw a TypeError instead.

Fixes #6336.
2022-03-30 15:50:13 +02:00
Eelco Dolstra
22522722a6 Merge pull request #6335 from thufschmitt/gitignore-stray-files
Gitignore or don’t create some build outputs
2022-03-30 14:49:15 +02:00
Théophane Hufschmitt
87f867ef62 Gitignore the generated systemd nix-daemon conf file 2022-03-30 11:43:08 +02:00
Théophane Hufschmitt
8dee15cd31 Don’t create a file in the worktree in the fetchPath test 2022-03-30 11:42:47 +02:00
Eelco Dolstra
ed581e51cc Merge pull request #6334 from NixOS/require-mounts-for-db
nix-daemon.service: require mounts for /nix/var/nix/db
2022-03-30 11:40:38 +02:00
Graham Christensen
3b26dd51ff nix-daemon.service: require mounts for /nix/var/nix/db
Users may want to mount a filesystem just for the Nix database, with
the filesystem's parameters specially tuned for sqlite. For example, on
ZFS you might set the recordsize to 64k after changing the database's
page size to 65536.
2022-03-29 21:05:57 -04:00
Ben Burdette
c0a567e196 remove const_cast 2022-03-29 16:44:47 -06:00
Eelco Dolstra
03be091e0a Merge pull request #6268 from thufschmitt/remove-the-variant-in-hashmodulo
Simplify the handling of the hash modulo
2022-03-29 20:26:47 +02:00
Théophane Hufschmitt
390269ed87 Simplify the handling of the hash modulo
Rather than having four different but very similar types of hashes, make
only one, with a tag indicating whether it corresponds to a regular of
deferred derivation.

This implies a slight logical change: The original Nix+multiple-outputs
model assumed only one hash-modulo per derivation. Adding
multiple-outputs CA derivations changed this as these have one
hash-modulo per output. This change is now treating each derivation as
having one hash modulo per output.
This obviously means that we internally loose the guaranty that
all the outputs of input-addressed derivations have the same hash
modulo. But it turns out that it doesn’t matter because there’s nothing
in the code taking advantage of that fact (and it probably shouldn’t
anyways).

The upside is that it is now much easier to work with these hashes, and
we can get rid of a lot of useless `std::visit{ overloaded`.

Co-authored-by: John Ericson <John.Ericson@Obsidian.Systems>
2022-03-29 18:17:35 +02:00
Ben Burdette
5ab7bdf0b1 load debug trace staticenv on 'show' 2022-03-28 15:28:59 -06:00
Ben Burdette
14f515544b debugTraceIndex 2022-03-28 12:09:21 -06:00
Théophane Hufschmitt
2d572a250f Merge pull request #6330 from edolstra/run-remote-store
nix {run,shell}: Print a better error message if the store is not local
2022-03-28 17:57:37 +02:00
Eelco Dolstra
a3f932db32 Merge pull request #6328 from edolstra/fix-nix-profile-install
nix profile install: Don't use queryDerivationOutputMap()
2022-03-28 17:25:27 +02:00
Eelco Dolstra
b266fd53dd nix {run,shell}: Print a better error message if the store is not local
Closes #6317
2022-03-28 14:58:38 +02:00
Eelco Dolstra
057f9ee190 nix profile install: Don't use queryDerivationOutputMap()
Instead get the outputs from Installable::build(). This will also
allow 'nix profile install' to support impure derivations.

Fixes #6286.
2022-03-28 14:23:39 +02:00
Théophane Hufschmitt
0b33c9f9c3 Merge pull request #6319 from Artturin/fixtypo
scripts/install-systemd-multi-user.sh: fix typo
2022-03-28 09:05:26 +02:00
Erik Arvstedt
16860a0328 nix eval: Add option read-only 2022-03-26 11:32:38 +01:00
Erik Arvstedt
679b3b32c9 Minor comment fix 2022-03-26 11:32:37 +01:00
Ben Burdette
1bec3fb337 add DebugTrace for error 2022-03-25 18:15:31 -06:00
Artturin
247d2cb661 scripts/install-systemd-multi-user.sh: fix typo
sytemd-tmpfiles -> systemd-tmpfiles
2022-03-26 00:58:19 +02:00
Eelco Dolstra
1844172dd1 Merge pull request #6314 from edolstra/experimental-primop
Only provide builtins is the corresponding experimental feature is enabled
2022-03-25 16:16:31 +01:00
Eelco Dolstra
16cf1e6089 Merge pull request #6311 from edolstra/return-wanted-paths
Make buildPathsWithResults() only return info on wanted outputs
2022-03-25 15:44:39 +01:00
Eelco Dolstra
fc35b11a7c Fix mismatched tag warning on clang 2022-03-25 15:22:22 +01:00
Eelco Dolstra
8c363eb3eb Document getFlake
Fixes #5523.
2022-03-25 14:19:55 +01:00
Eelco Dolstra
86b05ccd54 Only provide builtin.{getFlake,fetchClosure} is the corresponding experimental feature is enabled
This allows writing fallback code like

  if builtins ? fetchClosure then
    builtins.fetchClose { ... }
  else
    builtins.storePath ...
2022-03-25 14:04:18 +01:00
Eelco Dolstra
55bc524019 Merge pull request #6051 from Ma27/fix-empty-nix-log
`nix log` should also work if the log didn't provide any output
2022-03-25 10:35:10 +01:00
Eelco Dolstra
377782ecae Merge pull request #6292 from polykernel/permissive-spacing-repl
nix: allow whitespace characters before command in repl
2022-03-25 10:24:06 +01:00
Eelco Dolstra
50c229ad9a Use wantOutput
Co-authored-by: John Ericson <git@JohnEricson.me>
2022-03-25 08:02:49 +01:00
polykernel
cbcb69a39c nix: allow whitespace characters before command in repl
Before this change, processLine always uses the first character
as the start of the line. This cause whitespaces to matter at the
beginning of the line whereas it does not matter anywhere else.

This commit trims leading white spaces of the string line so that
subsequent operations can be performed on the string without explicitly
tracking starting and ending indices of the string.
2022-03-24 21:33:29 -04:00
Eelco Dolstra
187dc080a2 tests/build.sh: Test that 'nix build' only prints wanted outputs 2022-03-24 23:36:14 +01:00
Eelco Dolstra
540d7e33d8 Retry substitution after an incomplete closure only once
This avoids an infinite loop in the final test in
tests/binary-cache.sh. I think this was only not triggered previously
by accident (because we were clearing wantedOutputs in between).
2022-03-24 23:25:12 +01:00
Eelco Dolstra
fe5509df9a Only return wanted outputs 2022-03-24 23:24:48 +01:00
Eelco Dolstra
09796c0263 Random cleanup 2022-03-24 23:24:10 +01:00
Eelco Dolstra
175c78591b Random cleanup 2022-03-24 23:09:43 +01:00
Maximilian Bosch
d02e34ef06 Implement regression test for empty logs loaded via nix log 2022-03-24 22:31:52 +01:00
Maximilian Bosch
c85467a1b6 Revert "TarArchive: Small refactoring"
This reverts commit 50a35860ee.

With this change Nix fails to open bzip2 logfiles that were created from
builds with no stdout/stderr.
2022-03-24 22:30:46 +01:00
Eelco Dolstra
d9cfd853e5 Merge pull request #6302 from edolstra/fetch-closure
Add builtins.fetchClosure
2022-03-24 22:13:57 +01:00
Eelco Dolstra
f902f3c2cb Add experimental feature 'fetch-closure' 2022-03-24 21:33:33 +01:00
Eelco Dolstra
e5f7029ba4 nix store make-content-addressed: Support --from / --to 2022-03-24 21:33:33 +01:00
Eelco Dolstra
98658ae9d2 Document fetchClosure 2022-03-24 21:33:33 +01:00
Eelco Dolstra
28186b7044 Add a test for fetchClosure and 'nix store make-content-addressed' 2022-03-24 21:33:33 +01:00
Eelco Dolstra
4120930ac1 fetchClosure: Only allow some "safe" store types 2022-03-24 21:33:33 +01:00
Eelco Dolstra
7ffda0af6e fetchClosure: Skip makeContentAddressed() if toPath is already valid 2022-03-24 21:33:33 +01:00
Eelco Dolstra
5acaf13d35 Rename 'nix store make-content-addressable' to 'nix store make-content-addressed' 2022-03-24 21:33:33 +01:00
Eelco Dolstra
f18607549c Fix makeContentAddressed() on self-references
LocalStore::addToStore() since
79ae9e4558 expects a regular NAR hash,
rather than a NAR hash modulo self-references. Fixes #6300.

Also, makeContentAddressed() now rewrites the entire closure (so 'nix
store make-content-addressable' no longer needs '-r'). See #6301.
2022-03-24 21:33:33 +01:00
Eelco Dolstra
545c2d0d8c fetchClosure: Allow a path to be rewritten to CA on the fly
The advantage is that the resulting closure doesn't need to be signed,
so you don't need to configure any binary cache keys on the client.
2022-03-24 21:33:33 +01:00
Eelco Dolstra
7f6fe8ca1d Rename 2022-03-24 21:33:33 +01:00
Eelco Dolstra
41659418cf fetchClosure: Require a CA path in pure mode 2022-03-24 21:33:33 +01:00
Eelco Dolstra
f4bafc412f Add builtins.fetchClosure
This allows closures to be imported at evaluation time, without
requiring the user to configure substituters. E.g.

  builtins.fetchClosure {
    storePath = /nix/store/f89g6yi63m1ywfxj96whv5sxsm74w5ka-python3.9-sqlparse-0.4.2;
    from = "https://cache.ngi0.nixos.org";
  }
2022-03-24 21:33:33 +01:00
Eelco Dolstra
c9148f4ece Merge pull request #6285 from flokli/add-tmpfile
nix-daemon.conf.in: add tmpfiles file to create nix/daemon-socket directory
2022-03-24 21:24:53 +01:00
Eelco Dolstra
d67fe90375 Merge pull request #6305 from flox/genericClosure_doc
docs: genericClosure
2022-03-24 14:02:58 +01:00
Eelco Dolstra
97734fea1b Merge pull request #6308 from NixOS/consisten-use-of-url-uri-5872
Fix flake profile use of originalUrl vs. originalUri
2022-03-24 14:02:33 +01:00
Eelco Dolstra
721481e092 Merge pull request #6307 from edolstra/cp-symlinks
install-multi-user.sh: Preserve symlinks
2022-03-24 13:22:24 +01:00
Tom Bereknyei
0736f3651d docs: genericClosure 2022-03-24 08:03:59 -04:00
Eelco Dolstra
bb0c4b9f25 install-multi-user.sh: Preserve symlinks
We need to pass -P to ensure that symlinks are copied correctly. Fixes #6303.
2022-03-24 12:48:59 +01:00
Rok Garbas
4546a007a4 Fix flake profile use of originalUrl vs. originalUri
Fixes #5872
2022-03-24 12:28:38 +01:00
Eelco Dolstra
284cb0aad7 Merge pull request #6306 from trofi/add-lexer-locations
lexer: add error location to lexer errors
2022-03-24 10:07:40 +01:00
Sergei Trofimovich
9174d884d7 lexer: add error location to lexer errors
Before the change lexter errors did not report the location:

    $ nix build -f. mc
    error: path has a trailing slash
    (use '--show-trace' to show detailed location information)

Note that it's not clear what file generates the error.

After the change location is reported:

    $ src/nix/nix --extra-experimental-features nix-command build -f ~/nm mc
    error: path has a trailing slash

           at .../pkgs/development/libraries/glib/default.nix:54:18:

               53|   };
               54|   src = /tmp/foo/;
                 |                  ^
               55|
    (use '--show-trace' to show detailed location information)

Here we see both problematic file and the string itself.
2022-03-24 08:16:14 +00:00
Florian Klink
67af5f7eda scripts/install-systemd-multi-user.sh: install /etc/tmpfiles.d/nix-daemon.conf, too
While `create_directories()` from install-multi-user.sh seems to already
create parts of the directory structure, it's marked as deprecated, and
it won't hurt also copying over the tmpfiles config and have it execute
once.
2022-03-23 13:51:38 +01:00
Domen Kožar
98ce1a21b7 Merge pull request #6144 from toonn/doc-macos-uninstall
doc: Add detailed uninstall section for macOS
2022-03-23 08:53:48 +01:00
Eelco Dolstra
d5d4d98042 Merge pull request #6298 from kayhide/nix-edit-support-kakoune
nix edit: support kakoune
2022-03-22 16:56:35 +01:00
Hideaki Kawai
3b776cb0a7 nix edit: support kakoune 2022-03-22 23:18:02 +09:00
Eelco Dolstra
8434869632 Merge pull request #6296 from edolstra/fix-profile-manifest
Don't hide repeated values while generating manifest.nix
2022-03-22 14:09:25 +01:00
Eelco Dolstra
a0259a21a4 Don't hide repeated values while generating manifest.nix
Fixes #6243.
2022-03-22 13:18:56 +01:00
Eelco Dolstra
732296ddc0 printValue(): <REPEAT> -> «repeated»
This ensures that it doesn't get parsed as a valid Nix expression.
2022-03-22 13:00:27 +01:00
Eelco Dolstra
63f564ccf2 Merge pull request #6294 from edolstra/filter-manifest
buildProfile(): Ignore manifest.{nix,json}
2022-03-22 11:28:16 +01:00
Eelco Dolstra
0b42afe027 buildProfile(): Ignore manifest.{nix,json}
If a package installs a file named manifest.json, it caused nix-env to
consider the profile a new-style profile created by 'nix
profile'. Fixes #6032.
2022-03-22 10:48:02 +01:00
Eelco Dolstra
e4ff430866 Merge pull request #6237 from obsidiansystems/store-path-string-context
Decode string context straight to using StorePaths
2022-03-22 10:29:46 +01:00
Eelco Dolstra
7ed81701ee Merge pull request #6290 from Misterio77/fix-sourcehut-tags-refs
Fix sourcehut tags refs resolving
2022-03-21 11:23:56 +01:00
Gabriel Fontes
31544b93ff Fix sourcehut integration test
The new implementation relies on tab separting the hash and ref (this is how sourcehut does it). This fixes the integration test to use a tab instead of a space.
2022-03-19 11:38:45 -03:00
Gabriel Fontes
9720797f69 Don't partial match sourcehut refs 2022-03-19 11:04:04 -03:00
Gabriel Fontes
345a8ee0cb Fix sourcehut tag ref resolving 2022-03-19 10:56:13 -03:00
Théophane Hufschmitt
8ad485ea89 Merge pull request #6284 from obsidiansystems/generlized-derivation-type
Generalize DerivationType in preparation for impure derivations
2022-03-18 16:36:32 +01:00
John Ericson
4d6a3806d2 Decode string context straight to using StorePaths
I gather decoding happens on demand, so I hope don't think this should
have any perf implications one way or the other.
2022-03-18 15:36:11 +00:00
Florian Klink
d60f3cf6e9 nix-daemon.conf.in: add tmpfiles file to create nix/daemon-socket directory
nix-daemon.socket is used to socket-activate nix-daemon.service when
/nix/var/nix/daemon-socket/socket is accessed.

In container usecases, sometimes /nix/var/nix/daemon-socket is
bind-mounted read-only into the container.

In these cases, we want to skip starting nix-daemon.socket.

However, since systemd 250, `ConditionPathIsReadWrite` is also not met
if /nix/var/nix/daemon-socket doesn't exist at all. This means, a
regular NixOS system will skip starting nix-daemon.socket:

> [  237.187747] systemd[1]: Nix Daemon Socket was skipped because of a failed condition check (ConditionPathIsReadWrite=/nix/var/nix/daemon-socket).

To prevent this from happening, ship a tmpfiles file that'll cause the
directory to be created if it doesn't exist already.

In the case of NixOS, we can just add Nix to `systemd.tmpfiles.packages`
and have these files picked up automatically.
2022-03-18 16:14:09 +01:00
John Ericson
a544ed7684 Generalize DerivationType in preparation for impure derivations 2022-03-18 14:59:56 +00:00
John Ericson
049fae155a Avoid some pointless copying of drvs 2022-03-18 14:59:56 +00:00
John Ericson
8496be7def Use Deferred when building an input-addressed drv
Easier than using dummy path with input addressed.
2022-03-18 14:59:56 +00:00
Théophane Hufschmitt
e433e42c5a Merge pull request #6283 from obsidiansystems/outputs-less-nesting
Clean up `DerivationOutput`, and headers
2022-03-18 14:56:16 +01:00
John Ericson
197feed51d Clean up DerivationOutput, and headers
1. `DerivationOutput` now as the `std::variant` as a base class. And the
   variants are given hierarchical names under `DerivationOutput`.

   In 8e0d0689be @matthewbauer and I
   didn't know a better idiom, and so we made it a field. But this sort
   of "newtype" is anoying for literals downstream.

   Since then we leaned the base class, inherit the constructors trick,
   e.g. used in `DerivedPath`. Switching to use that makes this more
   ergonomic, and consistent.

2. `store-api.hh` and `derivations.hh` are now independent.

   In bcde5456cc I swapped the dependency,
   but I now know it is better to just keep on using incomplete types as
   much as possible for faster compilation and good separation of
   concerns.
2022-03-17 22:35:53 +00:00
Eelco Dolstra
6afc361798 Merge pull request #6277 from thufschmitt/ca/nix-build-dry-run
Fix `nix build --dry-run` with CA derivations
2022-03-17 22:23:15 +01:00
Eelco Dolstra
d8f69b9a4c Merge branch 'avoid-quadratic-gc' of https://github.com/trofi/nix 2022-03-17 21:35:42 +01:00
Sergei Trofimovich
d58453f72e gc: don't visit implicit referrers on garbage collection
Before the change garbage collector was not considering
`.drv` and outputs as alive even if configuration says otherwise.
As a result `nix store gc --dry-run` could visit (and parse)
`.drv` files multiple times (worst case it's quadratic).

It happens because `alive` set was populating only runtime closure
without regard for actual configuration. The change fixes it.

Benchmark: my system has about 139MB, 40K `.drv` files.

Performance before the change:

    $ time nix store gc --dry-run
    real    4m22,148s

Performance after the change:

    $ time nix store gc --dry-run
    real    0m14,178s
2022-03-17 18:47:29 +00:00
Théophane Hufschmitt
7117053457 Merge pull request #6270 from Artturin/stdineval
nix: allow using --file - to read from stdin
2022-03-17 14:35:01 +01:00
Théophane Hufschmitt
3fc4c612fb Fix nix build --dry-run with CA derivations
Don’t try and assume that we know the output paths when we’ve just built
with `--dry-run`. Instead make `--dry-run` follow a different code path
that won’t assume the knowledge of the output paths at all.

Fix #6275
2022-03-17 11:55:19 +01:00
Théophane Hufschmitt
a0b517de57 Merge pull request #6242 from ncfavier/print-output-names
nix-env: always print output names in JSON and XML
2022-03-17 10:55:22 +01:00
Théophane Hufschmitt
c20e07763d Add some tests for nix-env -q --json 2022-03-16 21:33:38 +01:00
Naïm Favier
8dcecc0738 nix-env: print a final newline after JSON 2022-03-16 21:26:19 +01:00
Naïm Favier
5736661922 nix-env: always print output names in JSON and XML
The current `--out-path` flag has two disadvantages when one is only
concerned with querying the names of outputs:
- it requires evaluating every output's `outPath`, which takes
  significantly more resources and runs into more failures
- it destroys the information of the order of outputs so we can't tell
  which one is the main output

This patch makes the output names always present (replacing paths with
`null` in JSON if `--out-path` isn't given), and adds an `outputName`
field.
2022-03-16 21:26:19 +01:00
Théophane Hufschmitt
fe1ad96135 Merge pull request #6267 from nomeata/patch-1
distributed-builds.md: fixing typo of the most minor sort
2022-03-16 21:18:14 +01:00
Artturin
4f8ad41d4e add tests for nix eval and nix-instantiate 2022-03-16 21:01:51 +02:00
Artturin
a5c969db49 nix: allow using --file - to read from stdin 2022-03-16 21:01:51 +02:00
Ben Burdette
88a54108eb formatting 2022-03-16 12:09:47 -06:00
Joachim Breitner
af013281c9 distributed-builds.md: fixing typo of the most minor sort 2022-03-16 12:53:38 +01:00
Eelco Dolstra
a53c1dc96d Merge pull request #6245 from trofi/fix-nix-store-gc-limit-auto-optimise-store
nix store gc: account for auto-optimised store
2022-03-16 09:19:57 +01:00
Eelco Dolstra
7b0eb9eaf2 Merge pull request #6264 from thufschmitt/fix-32bits-tests
Fix the tests on 32bits machines
2022-03-16 09:13:23 +01:00
Théophane Hufschmitt
3cea6f569e Fix the date in the comment of fetchPath’s test
Co-authored-by: pennae <82953136+pennae@users.noreply.github.com>
2022-03-16 08:56:01 +01:00
Théophane Hufschmitt
516a7ac4de Merge pull request #6229 from obsidiansystems/refactor-hash-modulo
Overhaul derivation hash modulo somewhat
2022-03-15 21:23:44 +01:00
regnat
2d5c43f210 Fix the tests on 32bits machines
year 2222 is too much for a 32 bit timestamp. So replace it by something
smaller
2022-03-15 21:05:01 +01:00
Théophane Hufschmitt
ba9e69cdcd Merge pull request #6119 from Ma27/path-mtime
libfetchers/path: set `lastModified` to path's mtime
2022-03-15 17:51:16 +01:00
Maximilian Bosch
975bade7f0 Implement simple test for path-fetcher setting a correct lastModifiedDate
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2022-03-15 12:55:32 +01:00
Maximilian Bosch
244baff2c7 libfetchers: remove obsolete filesystem #include 2022-03-15 12:32:11 +01:00
Maximilian Bosch
34e20c164c libfetchers/path: set lastModified to path's mtime
When importing e.g. a local `nixpkgs` in a flake to test a change like

    {
      inputs.nixpkgs.url = path:/home/ma27/Projects/nixpkgs;
      outputs = /* ... */
    }

then the input is missing a `lastModified`-field that's e.g. used in
`nixpkgs.lib.nixosSystem`. Due to the missing `lastMoified`-field, the
mtime is set to 19700101:

    result -> /nix/store/b7dg1lmmsill2rsgyv2w7b6cnmixkvc1-nixos-system-nixos-22.05.19700101.dirty

With this change, the `path`-fetcher now sets a `lastModified` attribute
to the `mtime` just like it's the case in the `tarball`-fetcher already.
When building NixOS systems with `nixpkgs` being a `path`-input and this
patch, the output-path now looks like this:

    result -> /nix/store/ld2qf9c1s98dxmiwcaq5vn9k5ylzrm1s-nixos-system-nixos-22.05.20220217.dirty
2022-03-15 12:32:11 +01:00
Ben Burdette
3dfab6e534 have only one debuggerHook declaration 2022-03-14 11:58:11 -06:00
Ben Burdette
eaecaaa00b more debug_throw coverage of EvalErrors 2022-03-14 11:39:53 -06:00
Eelco Dolstra
a61809722f Merge pull request #6244 from Artturin/nixenvbettermessage
nix-env: Add a suggestion for when there's a name collision in channels
2022-03-14 14:17:50 +01:00
Théophane Hufschmitt
0e86ebf461 Merge pull request #6220 from obsidiansystems/log-store
Factor out a `LogStore` interface
2022-03-14 10:51:01 +01:00
Eelco Dolstra
edfd94c486 Merge pull request #6248 from thkoch2001/patch-1
Add documentation= entry to systemd unit file
2022-03-14 10:34:18 +01:00
thkoch2001
e06b264f94 Add documentation= entry to systemd unit file
Closes: #6246
2022-03-13 18:42:26 +02:00
Sergei Trofimovich
6b1872312f nix store gc: account for auto-optimised store
Before the change on a system with `auto-optimise-store = true`:

    $ nix store gc --verbose --max 1

deleted all the paths instead of one path (we requested 1 byte limit).

It happens because every file in `auto-optimise-store = true` has at
least 2 links: file itself and a link in /nix/store/.links/ directory.

The change conservatively assumes that any file that has one (as before)
or two links (assume auto-potimise mode) will free space.

Co-authored-by: Sandro <sandro.jaeckel@gmail.com>
2022-03-13 07:24:48 +00:00
Artturin
cb1a76112e nix-env: Add a suggestion for when there's a name collision in channels
help new users find a solution to their problem

./result/bin/nix-env -qa hello
warning: name collision in input Nix expressions, skipping '/home/artturin/.nix-defexpr/channels_root/master'
suggestion: remove 'master' from either the root channels or the user channels
hello-2.12
hello-2.12
2022-03-13 00:24:46 +02:00
John Ericson
91adfb8894 Create some type aliases for string Contexts 2022-03-11 22:30:10 +00:00
John Ericson
0948b8e94d Reduce variants for derivation hash modulo
This changes was taken from dynamic derivation (#4628). It` somewhat
undoes the refactors I first did for floating CA derivations, as the
benefit of hindsight + requirements of dynamic derivations made me
reconsider some things.

They aren't to consequential, but I figured they might be good to land
first, before the more profound changes @thufschmitt has in the works.
2022-03-11 21:20:37 +00:00
Eelco Dolstra
d5322698a2 Merge pull request #6230 from obsidiansystems/nix-build-derived-path
Desugar `StorePathWithOutputs` in nix-build implementation
2022-03-11 22:19:56 +01:00
John Ericson
015d2ad507 Desugar StorePathWithOutputs in nix-build implementation
`DerivedPath` has replaced `StorePathWithOutputs` internally, so
shrinking the usage of `StorePathWithOutputs` to just the boundary is
good.
2022-03-11 20:50:52 +00:00
Kevin Amado
2191dab657 nix-fmt: add command 2022-03-11 10:00:19 -05:00
John Ericson
a03b1fd7f6 Deduplicate the Store downcasting with a template 2022-03-11 13:32:33 +00:00
John Ericson
678d1c2aa0 Factor out a LogStore interface
Continue progress on #5729.

Just as I hoped, this uncovered an issue: the daemon protocol is missing
a way to query build logs. This doesn't effect `unix://`, but does
effect `ssh://`. A FIXME is left for this, so we come back to it later.
2022-03-11 13:32:16 +00:00
John Ericson
89effe9d4a GcStore::resolve should print the URI 2022-03-11 13:27:38 +00:00
John Ericson
073e134de6 Rename requireGcStore to GcStore::require
I should have done this to begin with. This will be nicer once more
Store sub-interfaces exist too, to illustrate the pattern.
2022-03-11 13:27:38 +00:00
Eelco Dolstra
aee56e0f89 Merge remote-tracking branch 'origin/eval-suggestions' 2022-03-11 12:02:26 +01:00
Eelco Dolstra
31a392dfe2 Merge pull request #5865 from pennae/memory-friendliness
be more memory friendly
2022-03-11 11:52:39 +01:00
Eelco Dolstra
167766b65c Style 2022-03-11 11:19:21 +01:00
Théophane Hufschmitt
7ebd6f1093 Merge pull request #5758 from mschwaig/fix-git-workspace-dirty-detection
git fetcher: fix detection of dirty git workspaces
2022-03-11 09:35:01 +01:00
pennae
f2603e9c92 Merge branch 'master' into lto 2022-03-10 00:32:34 +00:00
Eelco Dolstra
4d98143914 BuildResult: Remove unused drvPath field 2022-03-09 20:31:50 +01:00
Eelco Dolstra
1c1a7074da Merge pull request #6221 from NixOS/build-paths-with-results
Add Store::buildPathsWithResults()
2022-03-09 14:37:14 +01:00
Eelco Dolstra
761242afa0 BuildResult: Use DerivedPath 2022-03-09 12:25:35 +01:00
pennae
4d629c4f7a add HAVE_BOEHMGC guards to batched allocation functions 2022-03-09 00:18:50 +01:00
pennae
8e2eaaaf69 make Finally more local
no need for function<> with c++17 deduction. this saves allocations and virtual
calls, but has the same semantics otherwise. not going through function has the
side effect of giving compilers more insight into the cleanup code, so we need a
few local warning disables.
2022-03-09 00:16:50 +01:00
pennae
47baa9d43c make Pos smaller
reduces peak hep memory use on eval of our test system from 264.4MB to 242.3MB,
possibly also a slight performance boost.

theoretically memory use could be cut down by another eight bytes per Pos on
average by turning it into a tuple containing an index into a global base
position table with row and column offsets, but that doesn't seem worth the
effort at this point.
2022-03-08 23:30:18 +01:00
pennae
c96460f352 force-inline a few much-used functions
these functions are called a whole lot, and they're all comparatively small.
always inlining them gives ~0.7% performance boost on eval.

before:

  Benchmark 1: nix flakes search --no-eval-cache --offline ../nixpkgs hello
    Time (mean ± σ):      6.935 s ±  0.052 s    [User: 5.852 s, System: 0.853 s]
    Range (min … max):    6.808 s …  7.026 s    20 runs

  Benchmark 2: nix flakes eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
    Time (mean ± σ):     329.8 ms ±   2.7 ms    [User: 299.0 ms, System: 30.8 ms]
    Range (min … max):   326.6 ms … 336.5 ms    20 runs

  Benchmark 3: nix flakes eval --raw --impure --file expr.nix
    Time (mean ± σ):      2.655 s ±  0.038 s    [User: 2.364 s, System: 0.220 s]
    Range (min … max):    2.574 s …  2.737 s    20 runs

after:

  Benchmark 1: nix flakes search --no-eval-cache --offline ../nixpkgs hello
    Time (mean ± σ):      6.912 s ±  0.036 s    [User: 5.823 s, System: 0.856 s]
    Range (min … max):    6.849 s …  6.980 s    20 runs

  Benchmark 2: nix flakes eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
    Time (mean ± σ):     325.1 ms ±   2.5 ms    [User: 293.2 ms, System: 31.8 ms]
    Range (min … max):   322.2 ms … 332.8 ms    20 runs

  Benchmark 3: nix flakes eval --raw --impure --file expr.nix
    Time (mean ± σ):      2.636 s ±  0.024 s    [User: 2.352 s, System: 0.226 s]
    Range (min … max):    2.574 s …  2.681 s    20 runs
2022-03-08 23:30:18 +01:00
pennae
60ed4e908a cache singleton Envs just like Values
vast majority of envs is this size.

before:

  Benchmark 1: nix flakes search --no-eval-cache --offline ../nixpkgs hello
    Time (mean ± σ):      6.946 s ±  0.041 s    [User: 5.875 s, System: 0.835 s]
    Range (min … max):    6.834 s …  7.005 s    20 runs

  Benchmark 2: nix flakes eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
    Time (mean ± σ):     330.3 ms ±   2.5 ms    [User: 299.2 ms, System: 30.9 ms]
    Range (min … max):   327.5 ms … 337.7 ms    20 runs

  Benchmark 3: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
    Time (mean ± σ):      2.671 s ±  0.035 s    [User: 2.370 s, System: 0.232 s]
    Range (min … max):    2.597 s …  2.749 s    20 runs

after:

  Benchmark 1: nix flakes search --no-eval-cache --offline ../nixpkgs hello
    Time (mean ± σ):      6.935 s ±  0.052 s    [User: 5.852 s, System: 0.853 s]
    Range (min … max):    6.808 s …  7.026 s    20 runs

  Benchmark 2: nix flakes eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
    Time (mean ± σ):     329.8 ms ±   2.7 ms    [User: 299.0 ms, System: 30.8 ms]
    Range (min … max):   326.6 ms … 336.5 ms    20 runs

  Benchmark 3: nix flakes eval --raw --impure --file expr.nix
    Time (mean ± σ):      2.655 s ±  0.038 s    [User: 2.364 s, System: 0.220 s]
    Range (min … max):    2.574 s …  2.737 s    20 runs
2022-03-08 23:30:18 +01:00
pennae
4b2b0d3a55 remove GC_PTR_STORE_AND_DIRTY
turns out it's only necessary for MANUAL_VDB, which nix doesn't use. omitting
them gives a slight performance improvement on eval.

before:

  Benchmark 1: nix flakes search --no-eval-cache --offline ../nixpkgs hello
    Time (mean ± σ):      6.988 s ±  0.061 s    [User: 5.935 s, System: 0.845 s]
    Range (min … max):    6.865 s …  7.075 s    20 runs

  Benchmark 2: nix flakes eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
    Time (mean ± σ):     332.6 ms ±   3.9 ms    [User: 299.6 ms, System: 32.9 ms]
    Range (min … max):   328.1 ms … 339.1 ms    20 runs

  Benchmark 3: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
    Time (mean ± σ):      2.681 s ±  0.049 s    [User: 2.382 s, System: 0.228 s]
    Range (min … max):    2.607 s …  2.776 s    20 runs

after:

  Benchmark 1: nix flakes search --no-eval-cache --offline ../nixpkgs hello
    Time (mean ± σ):      6.946 s ±  0.041 s    [User: 5.875 s, System: 0.835 s]
    Range (min … max):    6.834 s …  7.005 s    20 runs

  Benchmark 2: nix flakes eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
    Time (mean ± σ):     330.3 ms ±   2.5 ms    [User: 299.2 ms, System: 30.9 ms]
    Range (min … max):   327.5 ms … 337.7 ms    20 runs

  Benchmark 3: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
    Time (mean ± σ):      2.671 s ±  0.035 s    [User: 2.370 s, System: 0.232 s]
    Range (min … max):    2.597 s …  2.749 s    20 runs
2022-03-08 23:30:18 +01:00
Eelco Dolstra
a4604f1928 Add Store::buildPathsWithResults()
This function is like buildPaths(), except that it returns a vector of
BuildResults containing the exact statuses and output paths of each
derivation / substitution. This is convenient for functions like
Installable::build(), because they then don't need to do another
series of calls to get the outputs of CA derivations. It's also a
precondition to impure derivations, where we *can't* query the output
of those derivations since they're not stored in the Nix database.

Note that PathSubstitutionGoal can now also return a BuildStatus.
2022-03-08 19:56:34 +01:00
regnat
f6078e474d Also display some suggestions for invalid formal arguments
```console
$ nix eval --expr '({ foo ? 1 }: foo) { fob = 2; }'
error: anonymous function at (string):1:2 called with unexpected argument 'fob'

       at «string»:1:1:

            1| ({ foo ? 1 }: foo) { fob = 2; }
             | ^
       Did you mean foo?
```

Not that because Nix will first check for _missing_ arguments before
checking for extra arguments, `({ foo }: foo) { fob = 1; }` will
complain about the missing `foo` argument (rather than extra `fob`) and
so won’t display a suggestion.
2022-03-08 16:40:22 +01:00
regnat
33b7514035 Try and make the darwin build happy 2022-03-08 16:07:17 +01:00
regnat
0c6e46e349 Add some suggestions to the evaluator
Make the evaluator show some suggestions when trying to access an
invalid field from an attrset.

```console
$ nix eval --expr '{ foo = 1; }.foa'
error: attribute 'foa' missing

       at «string»:1:1:

            1| { foo = 1; }.foa
             | ^
       Did you mean foo?
```
2022-03-08 06:21:45 +01:00
Eelco Dolstra
92b8d4d886 Bump version 2022-03-07 20:12:35 +01:00
Eelco Dolstra
ad7c99ef20 Move rl-next.md to rl-2.7.md 2022-03-07 20:10:18 +01:00
Eelco Dolstra
504e3b2a8f Style 2022-03-07 20:07:43 +01:00
Eelco Dolstra
fa614fac7f Merge branch 'GuillaumeDesforges/issue6192' of https://github.com/GuillaumeDesforges/nix 2022-03-07 19:58:06 +01:00
Eelco Dolstra
0123b9aec8 Tweak release notes 2022-03-07 19:56:10 +01:00
Eelco Dolstra
30ddd37873 Merge branch 'cli-suggestions' of https://github.com/thufschmitt/nix 2022-03-07 19:47:45 +01:00
Guillaume Desforges
436c6e900f Explicit error in flake init/new when not in store 2022-03-07 17:57:52 +01:00
regnat
dcf4f77fac Merge or-suggestions.hh into suggestions.hh
No real need for keeping a separate header for such a simple class.

This requires changing a bit `OrSuggestions<T>::operator*` to not throw
an `Error` to prevent a cyclic dependency. But since this error is only
thrown on programmer error, we can replace the whole method by a direct
call to `std::get` which will raise its own assertion if needs be.
2022-03-07 17:49:02 +01:00
Eelco Dolstra
60a68a4fee Merge pull request #6173 from hercules-ci/fix-mkString-for-empty-string-view
Fix `mkString` for empty `string_view`
2022-03-07 17:10:41 +01:00
Robert Hensing
da260f579d dupStringWithLen -> makeImmutableString
Refactor the `size == 0` logic into a new helper function that
replaces dupStringWithLen.
The name had to change, because unlike a `dup`-function, it does
not always allocate a new string.
2022-03-07 16:09:12 +01:00
Robert Hensing
bbf55383e7 Value::mkPath: Avoid potential crash from null string_view 2022-03-07 16:09:12 +01:00
Robert Hensing
1b978596b5 Value::mkString: Avoid crash from null string_view 2022-03-07 16:09:12 +01:00
Naïm Favier
da7d8daa77 Add shell completion for --override-flake
Requires moving the MixEvalArgs class from libexpr to libcmd because
that's where completeFlakeRef is.
2022-03-07 12:01:54 +01:00
Naïm Favier
55c6906701 Perform tilde expansion when completing flake fragments
Allows completing `nix build ~/flake#<Tab>`.
We can implement expansion for `~user` later if needed.
Not using wordexp(3) since that expands way too much.
2022-03-07 12:01:54 +01:00
Naïm Favier
5461ff532d Make completeDir follow symlinks
Allows completing `nix why-depends /run/cur<Tab>` to /run/current-system
2022-03-07 12:01:54 +01:00
Naïm Favier
a6d7cd4183 Ensure the completion marker is not processed beyond completion
I was surprised to see an error mentioning ___COMPLETE___ when trying to
complete a flag argument that had no completer implemented
2022-03-07 12:01:54 +01:00
Naïm Favier
5f06a91bf7 Fix completion of nested attributes in completeInstallable
Without this, completing `nix eval -f file.nix foo.<Tab>` suggests `bar`
instead of `foo.bar`, which messes up the command
2022-03-07 12:01:54 +01:00
Naïm Favier
7ddcb39206 Add shell completion for --override-input 2022-03-07 12:01:54 +01:00
Naïm Favier
7f5cf87d56 Accept and discard fragments in getFlakeRefForCompletion
Otherwise trying to complete `nix build foo#bar --update-input <Tab>`
fails with "unexpected fragment"
2022-03-07 12:01:54 +01:00
Eelco Dolstra
c28e2b1b29 Tweak release notes 2022-03-07 11:30:40 +01:00
regnat
313bbc07a8 Implement operator<< for Suggestions
That way there’s no need to explicitely convert it to a string when
printing it
2022-03-07 10:09:10 +01:00
regnat
fd45d85b41 Move OrSuggestions to its own header
Prevents a recursive inclusion
2022-03-07 10:09:10 +01:00
regnat
b44cebd1fd Add a release-notes entry for the cli suggestions 2022-03-07 10:09:10 +01:00
regnat
91635206c0 Add some end-to-end tests for the suggestions 2022-03-07 10:09:10 +01:00
regnat
98e361ad4c Also display suggestions for the commands using the eval cache
Make `nix build .#nix-armv8l-linux` work for example
2022-03-07 10:09:10 +01:00
regnat
2405bbbb5e Add some tests for the suggestions 2022-03-07 10:09:10 +01:00
regnat
c0792b1546 Implement a suggestions mechanism
Each `Error` class now includes a set of suggestions, and these are printed by
the top-level handler.
2022-03-07 10:09:09 +01:00
Théophane Hufschmitt
a2ace54fe4 Merge pull request #6029 from Ma27/nix-log-ssh-ng
ssh-ng: also store build logs to make them accessible by `nix log`
2022-03-07 09:51:40 +01:00
Théophane Hufschmitt
860016bcbf Explain why the log tests are disabled for CA derivations 2022-03-07 09:15:34 +01:00
Théophane Hufschmitt
2051c0f409 Merge pull request #6208 from sielicki/fix-url-github-oauth
Point to new github oauth docs url
2022-03-07 08:54:50 +01:00
Nicholas Sielicki
314852a10e Point to new github oauth docs url
Previous URL was 404'ing.
2022-03-06 17:01:14 -06:00
Maximilian Bosch
697d1dac01 tests: grep for string in nix log for remote-builds 2022-03-04 16:58:27 +01:00
Eelco Dolstra
b09baf690b Merge pull request #6188 from obsidiansystems/store-gc-subclass
Factor out a `GcStore` interface
2022-03-03 20:53:10 +01:00
John Ericson
6636202356 Factor out a GcStore interface
Starts progress on #5729.

The idea is that we should not have these default methods throwing
"unimplemented". This is a small step in that direction.

I kept `addTempRoot` because it is a no-op, rather than failure. Also,
as a practical matter, it is called all over the place, while doing
other tasks, so the downcasting would be annoying.

Maybe in the future I could move the "real" `addTempRoot` to `GcStore`,
and the existing usecases use a `tryAddTempRoot` wrapper to downcast or
do nothing, but I wasn't sure whether that was a good idea so with a
bias to less churn I didn't do it yet.
2022-03-03 19:01:25 +00:00
regnat
417aaf4ff7 Correctly hijack the file:// uri scheme with _NIX_FORCE_HTTP
Setting the `_NIX_FORCE_HTTP` environment variable is supposed to force `file://` store urls to use the `HttpBinaryCacheStore` implementation rather than the `LocalBinaryCacheStore` one (very useful for testing).
However because of a name mismatch, the `LocalBinaryCacheStore` was still registering the `file` scheme when this variable was set, meaning that the actual store implementation picked up on `file://` uris was dependent on the registration order of the stores (itself dependent on the link order of the object files).

Fix this by making the `LocalBinaryCacheStore` gracefully not register the `file` uri scheme when the variable is set.
2022-03-03 17:47:50 +01:00
pennae
2799fe4cdb enable LTO in optimized builds
gives 2-5% performance improvement across a board of tests.
LTO is broken when using clang; some libs link fine while others crash
the linker with a segfault in the llvm linker plugin. 🙁
2022-03-03 17:47:49 +01:00
Eelco Dolstra
391f4fcabe Merge pull request #6201 from edolstra/print-value
printValue(): Don't show repeated values
2022-03-03 14:35:16 +01:00
Eelco Dolstra
e9c04c3351 Be more aggressive in hiding repeated values
We now memoize on Bindings / list element vectors rather than Values,
so that e.g. two Values that point to the same Bindings will be
printed only once.
2022-03-03 13:33:34 +01:00
Eelco Dolstra
ecff9d969a printValue(): Don't show repeated values
Fixes #6157.
2022-03-03 13:18:23 +01:00
Eelco Dolstra
6097790863 Fix segfault in headerCallback()
https://hydra.nixos.org/build/168594664
2022-03-03 11:11:16 +01:00
Eelco Dolstra
885d709393 Merge pull request #6198 from edolstra/coerce-store-path
Add EvalState::coerceToStorePath() helper
2022-03-03 10:02:51 +01:00
Eelco Dolstra
a7c835e9cb Use C++11-style initializer
Co-authored-by: John Ericson <git@JohnEricson.me>
2022-03-03 10:02:11 +01:00
Eelco Dolstra
b55d79728c Add EvalState::coerceToStorePath() helper
This is useful whenever we want to evaluate something to a store path
(e.g. in get-drvs.cc).

Extracted from the lazy-trees branch (where we can require that a
store path must come from a store source tree accessor).
2022-03-02 23:58:58 +01:00
Eelco Dolstra
3e3d0711d4 Merge pull request #6197 from edolstra/nix-profile-ca
nix profile: Support CA derivations
2022-03-02 23:48:52 +01:00
Eelco Dolstra
d4538034b7 nix profile test: Restart daemon
Fixes

  nix-daemon: src/libstore/sqlite.cc:97: nix::SQLiteStmt::Use::Use(nix::SQLiteStmt&): Assertion `stmt.stmt' failed.

which happens because the daemon doesn't properly handle the case
where ca-derivations isn't enabled at daemon startup.
2022-03-02 23:08:49 +01:00
Eelco Dolstra
3a3821bcd7 Remove obsolete todo 2022-03-02 22:22:55 +01:00
Eelco Dolstra
b0d65b3d11 Silence kill output 2022-03-02 22:22:23 +01:00
Eelco Dolstra
d2586188fe tests/common.sh.in: Add enableFeatures helper 2022-03-02 21:48:25 +01:00
Eelco Dolstra
5d208cbe41 mk/run_test.sh: Add missing backslash 2022-03-02 21:36:46 +01:00
Eelco Dolstra
f9375778ae nix profile: Add a test for non-flake packages 2022-03-02 20:56:40 +01:00
Eelco Dolstra
161f798aa1 nix profile: Support CA derivations 2022-03-02 20:38:51 +01:00
Eelco Dolstra
54888b92de Move installables-related operations 2022-03-02 19:19:51 +01:00
Eelco Dolstra
b2da2a22c6 Merge pull request #6194 from edolstra/nix-profile
Add basic tests for 'nix profile'
2022-03-02 16:07:00 +01:00
Eelco Dolstra
5850fd17b4 Add basic tests for 'nix profile'
Fixes #6193.
2022-03-02 14:40:46 +01:00
Eelco Dolstra
1aca6b9f17 Merge pull request #4345 from bjornfor/installer-configurable-uid-gid
installer: allow overriding nix user GID and UIDs
2022-03-02 12:38:52 +01:00
Eelco Dolstra
b39ef07414 Style 2022-03-02 11:46:15 +01:00
Eelco Dolstra
c10865a46e tests: Rename nix-profile.sh -> bash-profile.sh 2022-03-02 11:21:00 +01:00
Eelco Dolstra
010ffc31f8 Remove stray debug line 2022-03-02 11:20:32 +01:00
Eelco Dolstra
03df331c3f Merge pull request #6189 from obsidiansystems/build-result-header
Move `BuildResult` defintion to its own header
2022-03-02 08:47:51 +01:00
Eelco Dolstra
3c66839b9d Merge pull request #6190 from andersk/bel
filterANSIEscapes: Ignore BEL character
2022-03-02 08:46:19 +01:00
Anders Kaseorg
b5cd3e2d5c filterANSIEscapes: Ignore BEL character
GCC is not as good at music as it seems to think it is.  Fixes #4546.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2022-03-01 15:35:42 -08:00
John Ericson
e862833ec6 Move BuildResult defintion to its own header
Just like we did for `ValidPathInfo` in
d92d4f85a5.
2022-03-01 19:43:07 +00:00
Bjørn Forsman
f4d57aa490 installer: allow overriding nix user GID and UIDs
Needed to resolve conflict in case the default GID and UIDs are in use.
2022-03-01 19:17:41 +01:00
Eelco Dolstra
9087fe0760 Merge pull request #6187 from thufschmitt/allow-the-darwin-tests-to-be-flaky
Acknowledge that the macOS tests are flaky
2022-03-01 19:11:36 +01:00
regnat
c511134a94 Acknowledge that the macOS tests are flaky
Restart the tests (at most once) on `unexpected EOF` errors.

This is truly ugly, but might prevent half of the CI runs to fail
because of https://github.com/NixOS/nix/issues/3605
2022-03-01 15:25:05 +01:00
Eelco Dolstra
0cb5af5000 Merge pull request #6185 from hercules-ci/fetchTree-reuse-local-paths
fetchTree: Use isValidPath, add comment
2022-03-01 13:15:42 +01:00
Robert Hensing
b6deca7c0d fetchTree: Use isValidPath, add comment 2022-03-01 12:11:10 +01:00
Eelco Dolstra
d974d2ad59 fetch{url,Tarball}: Remove 'narHash' attribute
This was introduced in #6174. However fetch{url,Tarball} are legacy
and we shouldn't have an undocumented attribute that does the same
thing as one that already exists ('sha256').
2022-03-01 11:30:26 +01:00
Eelco Dolstra
9ab81a9d38 Merge pull request #6183 from obsidiansystems/sort-config
Move some stuff from `Settings` to a new `FetchSettings`.
2022-03-01 10:52:55 +01:00
Théophane Hufschmitt
983c991652 Merge pull request #6174 from hercules-ci/fetchTree-reuse-local-paths
fetchTree: Do not re-fetch paths already present + refactor
2022-03-01 10:29:32 +01:00
John Ericson
ea71da395f Move some stuff from Settings to a new FetchSettings.
Starting work on #5638

The exact boundary between `FetchSettings` and `EvalSettings` is not
clear to me, but that's fine. First lets clean out `libstore`, and then
worry about what, if anything, should be the separation between those
two.
2022-03-01 01:39:25 +00:00
Robert Hensing
ee019d0afc Add EvalState::allowAndSetStorePathString helper
This switches addPath from `printStorePath` to `toRealPath`.
2022-02-28 21:37:49 +01:00
Robert Hensing
158280d8e9 fetchTree: Do not re-fetch paths already present 2022-02-28 21:37:49 +01:00
Théophane Hufschmitt
1c985428c4 Merge pull request #6179 from NixOS/properly-start-the-daemon-in-tests
tests: Fix the start of the daemon
2022-02-28 19:23:50 +01:00
Maximilian Bosch
6a8f1b548f logging.hh: json.hpp -> json_fwd.hpp 2022-02-28 17:27:52 +01:00
Maximilian Bosch
102cb39086 libstore/build: add a few explanatory comments; simplify 2022-02-28 17:27:52 +01:00
Maximilian Bosch
287642f132 tests: implement test for nix log with ssh-ng:// builds
A few notes:

* The `echo hi` is needed to make sure that a file that can be read by
  `nix log` is properly created (i.e. some output is needed). This is
  known and to be fixed in #6051.
* We explicitly ignore the floating-CA case here: the `$out` of `input3`
  depends on `$out` of `input2`. This means that there are actually two
  derivations - I assume that this is because at eval time (i.e.
  `nix-instantiate -A`) the hash of `input2` isn't known yet and the
  other .drv is created as soon as `input2` was built. This is another
  issue on its own, so we ignore the case here explicitly.
2022-02-28 17:27:52 +01:00
Maximilian Bosch
cd92ea5885 libstore/derivation-goal: avoid double-parsing of JSON messages
To avoid that JSON messages are parsed twice in case of
remote builds with `ssh-ng://`, I split up the original
`handleJSONLogMessage` into three parts:

* `parseJSONMessage(const std::string&)` checks if it's a message in the
  form of `@nix {...}` and tries to parse it (and prints an error if the
  parsing fails).
* `handleJSONLogMessage(nlohmann::json&, ...)` reads the fields from the
  message and passes them to the logger.
* `handleJSONLogMessage(const std::string&, ...)` behaves as before, but
  uses the two functions mentioned above as implementation.

In case of `ssh-ng://`-logs the first two methods are invoked manually.
2022-02-28 17:27:52 +01:00
Maximilian Bosch
7a04839ea5 ssh-ng: also store build logs to make them accessible by nix log
Right now when building a derivation remotely via

    $ nix build -j0 -f . hello -L --builders 'ssh://builder'

it's possible later to read through the entire build-log by running
`nix log -f . hello`. This isn't possible however when using `ssh-ng`
rather than `ssh`.

The reason for that is that there are two different ways to transfer
logs in Nix through e.g. an SSH tunnel (that are used by `ssh`/`ssh-ng`
respectively):

* `ssh://` receives its logs from the fd pointing to `builderOut`. This
  is directly passed to the "log-sink" (and to the logger on each `\n`),
  hence `nix log` works here.
* `ssh-ng://` however expects JSON-like messages (i.e. `@nix {log data
  in here}`) and passes it directly to the logger without doing anything
  with the `logSink`. However it's certainly possible to extract
  log-lines from this format as these have their own message-type in the
  JSON payload (i.e. `resBuildLogLine`).

  This is basically what I changed in this patch: if the code-path for
  `builderOut` is not reached and a `logSink` is initialized, the
  message was successfully processed by the JSON logger (i.e. it's in
  the expected format) and the line is of the expected type (i.e.
  `resBuildLogLine`), the line will be written to the log-sink as well.

Closes #5079
2022-02-28 17:27:52 +01:00
regnat
b8f8aef9d3 tests: Fix the start of the daemon
- Make sure that it starts even without the `nix-command` xp feature
- Fail if it doesn’t manage to start

This fixes a 30s wait for every test in `init.sh` as the daemon couldn’t
start, but the code was just waiting 30s and continuing as if everything
was all right.
2022-02-28 17:00:31 +01:00
Eelco Dolstra
b592359c56 Merge pull request #6178 from edolstra/fix-darwin
Fix Darwin build
2022-02-28 16:13:00 +01:00
Eelco Dolstra
b91500a14e Fix clang warning 2022-02-28 15:48:26 +01:00
Eelco Dolstra
a949673a5b Fix Darwin build
Fixes #6169
2022-02-28 15:48:26 +01:00
Théophane Hufschmitt
c2720797d5 Merge pull request #6072 from Ma27/fix-nix-why-depends-non-precise
nix/why-depends: fix output when not using `--precise`
2022-02-28 11:46:50 +01:00
Domen Kožar
e85d565b5a Merge pull request #6123 from abathur/fix_6122
install-darwin: fix mount permission edge-case
2022-02-28 08:45:22 +00:00
toonn
947d4761b3 doc: Add removal of darwin-store LaunchDaemon
The uninstall instructions used to accidentally remove the nix-darwin
LaunchDaemon, this was dropped. However, the original intent was to
remove the Store volume mounting LaunchDaemon.
2022-02-26 14:16:35 +01:00
Eelco Dolstra
df552ff53e Remove std::string alias (for real this time)
Also use std::string_view in a few more places.
2022-02-25 16:13:02 +01:00
Eelco Dolstra
14b38d0887 xml-writer: Remove std aliases 2022-02-25 16:13:02 +01:00
Théophane Hufschmitt
fd4b693ca2 Merge pull request #6164 from kamadorueda/issue-5859
refactor: remove verbose-build from docs
2022-02-25 15:29:59 +01:00
Kevin Amado
b312d4d096 refactor: remove verbose-build from docs
- From what I see it is an implementation detail
  but is no longer configurable from the settings
2022-02-25 09:16:42 -05:00
toonn
2df23e2b3e doc: Drop nix-darwin service from macOS uninstall 2022-02-25 10:50:01 +01:00
toonn
064cad7e9f doc: Add macOS uninstall note about /nix
Clarify that `/nix` being present after the uninstall is normal and it
will only disappear after a reboot.

Co-authored-by: Travis A. Everett <travis.a.everett@gmail.com>
2022-02-25 10:32:45 +01:00
Eelco Dolstra
9d4697453f ExprAttrs::show(): Show attrs in sorted order
Issue #6160.
2022-02-25 09:21:04 +01:00
Eelco Dolstra
59683733f5 Merge pull request #6089 from edolstra/dot-default
Replace defaultBla.$system with bla.$system.default
2022-02-24 21:16:58 +01:00
Eelco Dolstra
cf7f98483a Merge pull request #6161 from edolstra/locked
libfetchers: Rename immutable -> locked
2022-02-24 19:01:33 +01:00
Eelco Dolstra
8518cebfce libfetchers: Rename immutable -> locked
This is more consistent with flake terminology.
2022-02-24 18:09:00 +01:00
Eelco Dolstra
3317866060 Merge pull request #6150 from GuillaumeDesforges/doc/builtins-function-anchor
Add anchor to builtin functions in HTML documentation
2022-02-24 17:54:48 +01:00
Eelco Dolstra
7ec244aec2 Merge pull request #6159 from NixOS/more-eager-daemon-polling-in-tests
testS: poll more eagerly for the daemon start/stop
2022-02-24 17:54:09 +01:00
Eelco Dolstra
5f88411af0 Merge pull request #6158 from NixOS/reorder-tests
Sort the tests by wall time
2022-02-24 17:53:48 +01:00
regnat
9c470cb969 testS: poll more eagerly for the daemon start/stop
Polling every 1 second means that even the simplest test takes at least
2 seconds. We can reasonably poll 1/10 of that to make things much
  quicker (esp. given that most of the time 0.1s is enough for the
  daemon to be started or stopped)
2022-02-24 15:00:23 +01:00
regnat
42766f8924 Sort the tests by wall time
The tests are scheduled in the order they appear, so running the long
ones first slightly improves the scheduling.

On my machine, this decreases the time of `make install` from 40s to 36s
2022-02-24 14:32:31 +01:00
Guillaume Desforges
cfbf9ee2ce Anchor with prefix 2022-02-24 13:50:50 +01:00
Guillaume Desforges
6462ee61c7 Anchor link for builtin functions in HTML doc 2022-02-23 15:07:01 +01:00
Guillaume Desforges
2cc645a91a Add html make target 2022-02-23 15:06:13 +01:00
Travis A. Everett
ba9a8c4b3d install-darwin: track mount permission edge-case fix
Same as 1fd127a068, but applied to a
code path (volume_pass_works -> verify_volume_pass) that the reporting
user didn't hit and wasn't able to trigger manually. I am not certain
but I suspect it will be easier to add prophylactically than to debug
if its absence causes trouble some day.
2022-02-22 12:44:15 -06:00
toonn
400d70a3a9 doc: Add detailed uninstall section for macOS
The multi-user installation on macOS, which is now the only option, has
gotten complicated enough that it discourages some users from checking
Nix out for fear of being left with a "dirty" system. Detailed
uninstallation instructions should make this less of an issue.
2022-02-22 16:28:24 +01:00
Eelco Dolstra
1a6548ca75 Update docs 2022-02-22 14:32:56 +01:00
Eelco Dolstra
38eea2c503 Update release notes 2022-02-22 14:23:39 +01:00
Eelco Dolstra
d4428d00ae nix flake check: Warn about deprecated flake outputs 2022-02-22 14:19:39 +01:00
Eelco Dolstra
162fbe31ff Replace defaultBla.$system with bla.$system.default
This also simplifies some InstallableFlake logic and fixes 'nix
bundle' parsing its installable twice.

Fixes #5532.
2022-02-22 11:47:41 +01:00
Martin Schwaighofer
53523c0ab8 git fetcher: set locale for rev-parse 2022-02-19 22:58:23 +01:00
Martin Schwaighofer
9504445cab git fetcher: distinguish errors more precisely 2022-02-19 22:58:23 +01:00
Martin Schwaighofer
c7e527b82b git fetcher: invoke diff instead of diff-index
diff-index operates on the view that git has of the working tree,
which might be outdated. The higher-level diff command does this
automatically. This change also adds handling for submodules.

fixes #4140

Alternative fixes would be invoking update-index before diff-index or
matching more closely what require_clean_work_tree from git-sh-setup.sh
does, but both those options make it more difficult to reason about
correctness.
2022-02-19 22:58:23 +01:00
Martin Schwaighofer
0bfa0cdea1 git fetcher: improve check for valid repository
The .git/refs/heads directory might be empty for a valid
usable git repository. This often happens in CI environments,
which might only fetch commits, not branches.
Therefore instead we let git itself check if HEAD points to
something that looks like a commit.

fixes #5302
2022-02-19 22:58:22 +01:00
Travis A. Everett
1fd127a068 install-darwin: fix mount permission edge-case
Fixes #6122, which reports a problem with trying to run the installer
under another user (probably: user is not the disk "owner" and thus
can't mount the volume).
2022-02-18 23:15:37 -06:00
Ben Burdette
3d94d3ba91 Expr refs instead of pointers 2022-02-15 15:46:45 -07:00
Ben Burdette
c9bc3735f6 quit repl from step mode 2022-02-15 09:49:25 -07:00
Ben Burdette
e761bf0601 make an 'info' level error on break 2022-02-14 14:04:34 -07:00
Ben Burdette
4cffb130e3 for primops, enter the debugger at the last DebugTrace in the stack 2022-02-11 14:14:25 -07:00
Ben Burdette
3ff5ac3586 update the eval-inline throw fns 2022-02-10 16:01:49 -07:00
Ben Burdette
bc67cb5ad1 remove fakeEnv stuff and instead use last context from the stack 2022-02-10 15:05:38 -07:00
Maximilian Bosch
5b586575ac nix/why-depends: fix output when not using --precise
On Nix 2.6 the output of `nix why-depends --all` seems to be somewhat
off:

    $ nix why-depends /nix/store/kn47hayxab8gc01jhr98dwyywbx561aq-nixos-system-roflmayr-21.11.20220207.6c202a9.drv /nix/store/srn5jbs1q30jpybdmxqrwskyny659qgc-nix-2.6.drv --derivation  --extra-experimental-features nix-command  --all
    /nix/store/kn47hayxab8gc01jhr98dwyywbx561aq-nixos-system-roflmayr-21.11.20220207.6c202a9.drv
        └───/nix/store/g8bpgfjhh5vxrdq0w6r6s64f9kkm9z6c-etc.drv
        │   └───/nix/store/hm0jmhp8shbf3cl846a685nv4f5cp3fy-nspawn-inst.drv
        | [...]
            └───/nix/store/2d6q3ygiim9ijl5d4h0qqx6vnjgxywyr-system-units.drv
                └───/nix/store/dil014y1b8qyjhhhf5fpaah5fzdf0bzs-unit-systemd-nspawn-hydra.service.drv
                    └───/nix/store/a9r72wwx8qrxyp7hjydyg0gsrwnn26zb-activate.drv
                        └───/nix/store/99hlc7i4gl77wq087lbhag4hkf3kvssj-nixos-system-hydra-21.11pre-git.drv

Please note that `[...]-system-units.drv` is supposed to be a direct
child of `[...]-etc.drv`.

The reason for that is that each new level printed by `printNode` is
four spaces off in comparison to `nix why-depends --precise` because the
recursive `printNode()` only prints the path and not the `tree*`-chars in
the case of `--precise` and in this format the path is four spaces further
indented, i.e. on a newline, but on the same level as the path's children, i.e.

    /nix/store/kn47hayxab8gc01jhr98dwyywbx561aq-nixos-system-roflmayr-21.11.20220207.6c202a9.drv
    └───/: …1-p8.drv",["out"]),("/nix/store/g8bpgfjhh5vxrdq0w6r6s64f9kkm9z6c-etc.drv",["out"]),("/nix/store/…
        → /nix/store/g8bpgfjhh5vxrdq0w6r6s64f9kkm9z6c-etc.drv

As you can see `[...]-etc.drv` is a direct child of the root, but four
spaces indented. This logic was directly applied to the code-path with
`precise=false` which resulted in `tree*` being printed four spaces too
deep.

In case of no `--precise`, `hits[hash]` is empty and the path itself
should be printed rather than hits using the same logic as for `hits[hash]`.

With this fix, the output looks correct now:

    /nix/store/kn47hayxab8gc01jhr98dwyywbx561aq-nixos-system-roflmayr-21.11.20220207.6c202a9.drv
    └───/nix/store/g8bpgfjhh5vxrdq0w6r6s64f9kkm9z6c-etc.drv
        ├───/nix/store/hm0jmhp8shbf3cl846a685nv4f5cp3fy-nspawn-inst.drv
        | [...]
        └───/nix/store/2d6q3ygiim9ijl5d4h0qqx6vnjgxywyr-system-units.drv
            └───/nix/store/dil014y1b8qyjhhhf5fpaah5fzdf0bzs-unit-systemd-nspawn-hydra.service.drv
                └───/nix/store/a9r72wwx8qrxyp7hjydyg0gsrwnn26zb-activate.drv
                    └───/nix/store/99hlc7i4gl77wq087lbhag4hkf3kvssj-nixos-system-hydra-21.11pre-git.drv
2022-02-10 14:26:00 +01:00
Ben Burdette
7954a18a48 link change 2022-02-04 17:40:06 -07:00
Ben Burdette
195db83148 a few merge fixes 2022-02-04 17:35:56 -07:00
Ben Burdette
dbe3fd3735 Merge branch 'master' into debug-step 2022-02-04 15:09:40 -07:00
Ben Burdette
3ddf864e1b print value in break 2022-02-04 14:50:25 -07:00
Ben Burdette
412d58f0bb break() primop; step and go debug commands 2022-02-03 13:15:21 -07:00
Ben Burdette
990bec78d3 clear screen and show top debug trace 2022-01-08 15:43:04 -07:00
Ben Burdette
a963674d88 optinoal error; compiles 2022-01-08 11:03:48 -07:00
Ben Burdette
c51b527c28 add env to DebugTrace 2022-01-07 16:37:44 -07:00
Ben Burdette
84aeb74377 revert value-add 2022-01-05 14:25:45 -07:00
Ben Burdette
bf8a065be0 add colors; remove headings 2022-01-05 12:28:31 -07:00
Ben Burdette
a4d8a799b7 tidy up debugtraces 2022-01-05 12:21:18 -07:00
Ben Burdette
1b6b33d43d filter out underscore names 2022-01-03 18:29:43 -07:00
Ben Burdette
c669108981 merge cleanup 2022-01-03 18:13:16 -07:00
Ben Burdette
a47de1ac37 Merge branch 'master' into debug-exploratory-PR 2022-01-03 16:08:28 -07:00
Ben Burdette
5954cbf3e9 more cleanup 2021-12-27 18:29:55 -07:00
Ben Burdette
4610e02d04 remove debug code 2021-12-27 18:12:46 -07:00
Ben Burdette
9760fa8661 add DebugTrace for the current error 2021-12-27 17:35:27 -07:00
Ben Burdette
6801a423fc :d env 2021-12-27 16:28:45 -07:00
Ben Burdette
2a66c120e6 by refernce for addVarToScope 2021-12-27 14:48:34 -07:00
Ben Burdette
ff82ba98b4 don't add builtins to extras, initEnv() in regular repl 2021-12-27 14:06:04 -07:00
Ben Burdette
d0d5890445 don't add underscore names to extras 2021-12-27 13:47:35 -07:00
Ben Burdette
e5eebda194 DebugTrace 2021-12-23 13:36:39 -07:00
Ben Burdette
deb1fd66e8 makeDebugTraceStacker 2021-12-23 09:08:41 -07:00
Ben Burdette
1bda6a01e1 indenting 2021-12-23 08:14:17 -07:00
Ben Burdette
bc20e54e00 stack traces basically working 2021-12-22 19:40:08 -07:00
Ben Burdette
b4a59a5eec DebugStackTracker class in one place 2021-12-22 15:38:49 -07:00
Ben Burdette
f317019edd :d error 2021-12-20 12:32:21 -07:00
Ben Burdette
c151a9b426 fix linking 2021-11-30 15:14:23 -07:00
Ben Burdette
e82aec4efc fix merge issues 2021-11-30 14:15:02 -07:00
Ben Burdette
64c4ba8f66 Merge branch 'master' into debug-merge 2021-11-25 08:53:59 -07:00
Ben Burdette
69e26c5c4b more cleanup 2021-11-25 08:23:07 -07:00
Ben Burdette
7e2a3db4eb cleanup 2021-11-09 13:14:49 -07:00
Ben Burdette
885f819922 remove dead code 2021-11-09 11:20:14 -07:00
Ben Burdette
fb8377547b more code cleanup 2021-10-22 14:49:58 -06:00
Ben Burdette
71da988d47 more debug removal 2021-10-22 14:34:50 -06:00
Ben Burdette
e54f17eb46 remove more debug code 2021-10-22 14:27:04 -06:00
Ben Burdette
cbc2f0fe31 remove dead code 2021-10-22 14:02:47 -06:00
Ben Burdette
383ab600ee show expr on error 2021-10-22 13:41:04 -06:00
Ben Burdette
427fb8d158 comment out debugs 2021-10-11 16:48:10 -06:00
Ben Burdette
98eb13691a print staticenv bindings 2021-10-11 16:32:43 -06:00
Ben Burdette
2ee1fa4afd add nullable Expr argument 2021-10-11 14:42:29 -06:00
Ben Burdette
aad27143c6 storing staticenv bindings 2021-10-02 13:47:36 -06:00
Ben Burdette
b9d08b98da ok was unconditoinally throwing on any With var 2021-09-23 13:02:39 -06:00
Ben Burdette
c07edb1932 staticenv should be With 2021-09-22 18:14:57 -06:00
Ben Burdette
c7e3d830c1 more debug stuff 2021-09-22 16:22:53 -06:00
Ben Burdette
037d53d9d9 turn off the stack usage thing 2021-09-17 16:58:54 -06:00
Ben Burdette
cd8c232b55 add cout debugging 2021-09-15 16:16:53 -06:00
Ben Burdette
1e04b2568d remove version.txt 2021-09-14 10:52:43 -06:00
Ben Burdette
2f90d92763 remove docs accidentally added to version control 2021-09-14 10:51:14 -06:00
Ben Burdette
21071bfdeb shared_ptr for StaticEnv 2021-09-14 10:49:22 -06:00
Ben Burdette
176911102c printEnvPosChain 2021-09-13 11:57:25 -06:00
Ben Burdette
310c689d31 remove more explicit valmap code 2021-08-25 13:18:27 -06:00
Ben Burdette
d8a977a22e adding all the value names from env.values[0] 2021-08-25 11:19:09 -06:00
Ben Burdette
bd3b5329f9 print env bindings 2021-08-24 16:32:54 -06:00
Ben Burdette
4b5f9b35f0 env to bindings 2021-08-18 21:25:26 -06:00
Ben Burdette
2272021536 more error fixes 2021-08-18 20:02:23 -06:00
Ben Burdette
e82cf13b1e switch to fakeenvs 2021-08-18 17:53:10 -06:00
Ben Burdette
b6eb38016b moving towards env in exceptions 2021-08-17 14:39:50 -06:00
Ben Burdette
030271184f trying env args; but unecessary? 2021-08-09 14:30:47 -06:00
Ben Burdette
89264d20e6 move valmap to hh; add to env 2021-08-06 11:09:27 -06:00
Robert Hensing
e30d1daf26 installer: Jeeze -> Oh no
Even if it doesn't offend me or probably most people,
this word can be considered mildly blasphemous.
2021-06-16 19:14:16 +02:00
Ben Burdette
edb5a28024 hintfmt for eye searing varnames 2021-06-11 18:55:40 -06:00
Ben Burdette
129dd760e6 mapEnvBindings 2021-06-11 18:55:15 -06:00
Ben Burdette
d22de1dd0c remove dead code 2021-06-09 15:38:08 -06:00
Ben Burdette
93ca9381da formatting; string arg for throwTypeError 2021-06-08 18:37:28 -06:00
Ben Burdette
ebf530d31e line endings 2021-06-08 18:17:58 -06:00
Ben Burdette
a8df239752 highlight the extra vars 2021-06-08 14:44:53 -06:00
Ben Burdette
ff2e72054f another throwTypeError form 2021-06-08 14:44:41 -06:00
Ben Burdette
99304334ca showType(fun) 2021-05-14 18:09:30 -06:00
Ben Burdette
644567cf7e clean up w LocalNoInline macro 2021-05-14 13:40:00 -06:00
Ben Burdette
17af7dc326 throwAssertionError, throwUndefinedError -> valmap-ized 2021-05-14 11:29:26 -06:00
Ben Burdette
d041dd874e throwEvalError form 4 2021-05-14 11:15:24 -06:00
Ben Burdette
ed74eaa07f throwEvalError form 3 2021-05-14 11:09:18 -06:00
Ben Burdette
989a4181a8 throwEvalError form 2 2021-05-14 11:06:20 -06:00
Ben Burdette
ab19d1685d throwEvalError; mapBindings 2021-05-13 16:00:48 -06:00
Ben Burdette
459bccc750 plain env pointer 2021-05-12 11:33:31 -06:00
Ben Burdette
0c2265da85 unique_ptr for valmap 2021-05-12 09:43:58 -06:00
Ben Burdette
e7847ad7a1 map1/2 for stack usage 2021-05-11 15:38:49 -06:00
Ben Burdette
a8fef9a6b1 throwTypeError with debugger/env 2021-05-10 18:36:57 -06:00
Ben Burdette
2dd61411af debugger on autoCallFunction error 2021-05-03 14:37:33 -06:00
Ben Burdette
f32c687f03 move repl.cc to libcmd for linkage 2021-04-28 15:50:11 -06:00
Ben Burdette
57c2dd5d85 fixes 2021-04-28 09:55:08 -06:00
Ben Burdette
b466d51952 Merge branch 'master' into debugger-merge 2021-04-27 12:31:01 -06:00
Eelco Dolstra
e486996cef Rename to --debugger
Co-authored-by: Domen Kožar <domen@dev.si>
2021-02-01 15:50:58 +01:00
Eelco Dolstra
e5662ba652 Add a flag to start the REPL on evaluation errors
This allows interactively inspecting the state of the evaluator at the
point of failure.

Example:

  $ nix eval path:///home/eelco/Dev/nix/flake2#modules.hello-closure._final --start-repl-on-eval-errors
  error: --- TypeError -------------------------------------------------------------------------------------------------------------------------------------------------------------------- nix
  at: (20:53) in file: /nix/store/4264z41dxfdiqr95svmpnxxxwhfplhy0-source/flake.nix

      19|
      20|           _final = builtins.foldl' (xs: mod: xs // (mod._module.config { config = _final; })) _defaults _allModules;
        |                                                     ^
      21|         };

  attempt to call something which is not a function but a set

  Starting REPL to allow you to inspect the current state of the evaluator.

  The following extra variables are in scope: arg, fun

  Welcome to Nix version 2.4. Type :? for help.

  nix-repl> fun
  error: --- EvalError -------------------------------------------------------------------------------------------------------------------------------------------------------------------- nix
  at: (150:28) in file: /nix/store/4264z41dxfdiqr95svmpnxxxwhfplhy0-source/flake.nix

     149|
     150|           tarballClosure = (module {
        |                            ^
     151|             extends = [ self.modules.derivation ];

  attribute 'derivation' missing

  nix-repl> :t fun
  a set

  nix-repl> builtins.attrNames fun
  [ "tarballClosure" ]

  nix-repl>
2020-08-05 21:26:17 +02:00
422 changed files with 14810 additions and 6468 deletions

View File

@@ -2,7 +2,7 @@
name: Feature request
about: Suggest an idea for this project
title: ''
labels: improvement
labels: feature
assignees: ''
---

View File

@@ -0,0 +1,28 @@
---
name: Missing or incorrect documentation
about: Help us improve the reference manual
title: ''
labels: documentation
assignees: ''
---
## Problem
<!-- describe your problem -->
## Checklist
<!-- make sure this issue is not redundant or obsolete -->
- [ ] checked [latest Nix manual] \([source])
- [ ] checked [open documentation issues and pull requests] for possible duplicates
[latest Nix manual]: https://nixos.org/manual/nix/unstable/
[source]: https://github.com/NixOS/nix/tree/master/doc/manual/src
[open documentation issues and pull requests]: https://github.com/NixOS/nix/labels/documentation
## Proposal
<!-- propose a solution -->

9
.github/stale.yml vendored
View File

@@ -1,10 +1,9 @@
# Configuration for probot-stale - https://github.com/probot/stale
daysUntilStale: 180
daysUntilClose: 365
daysUntilClose: false
exemptLabels:
- "critical"
- "never-stale"
staleLabel: "stale"
markComment: |
I marked this as stale due to inactivity. &rarr; [More info](https://github.com/NixOS/nix/blob/master/.github/STALE-BOT.md)
closeComment: |
I closed this issue due to inactivity. &rarr; [More info](https://github.com/NixOS/nix/blob/master/.github/STALE-BOT.md)
markComment: false
closeComment: false

View File

@@ -2,20 +2,26 @@ name: Backport
on:
pull_request_target:
types: [closed, labeled]
permissions:
contents: read
jobs:
backport:
name: Backport Pull Request
permissions:
# for zeebe-io/backport-action
contents: write
pull-requests: write
if: github.repository_owner == 'NixOS' && github.event.pull_request.merged == true && (github.event_name != 'labeled' || startsWith('backport', github.event.label.name))
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v3
with:
ref: ${{ github.event.pull_request.head.sha }}
# required to find all branches
fetch-depth: 0
- name: Create backport PRs
# should be kept in sync with `version`
uses: zeebe-io/backport-action@v0.0.7
uses: zeebe-io/backport-action@v0.0.8
with:
# Config README: https://github.com/zeebe-io/backport-action#backport-action
github_token: ${{ secrets.GITHUB_TOKEN }}

View File

@@ -4,53 +4,61 @@ on:
pull_request:
push:
permissions: read-all
jobs:
tests:
needs: [check_cachix]
needs: [check_secrets]
strategy:
matrix:
os: [ubuntu-latest, macos-latest]
runs-on: ${{ matrix.os }}
timeout-minutes: 60
steps:
- uses: actions/checkout@v2.4.0
- uses: actions/checkout@v3
with:
fetch-depth: 0
- uses: cachix/install-nix-action@v16
- uses: cachix/install-nix-action@v17
- run: echo CACHIX_NAME="$(echo $GITHUB_REPOSITORY-install-tests | tr "[A-Z]/" "[a-z]-")" >> $GITHUB_ENV
- uses: cachix/cachix-action@v10
if: needs.check_cachix.outputs.secret == 'true'
if: needs.check_secrets.outputs.cachix == 'true'
with:
name: '${{ env.CACHIX_NAME }}'
signingKey: '${{ secrets.CACHIX_SIGNING_KEY }}'
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
- run: nix --experimental-features 'nix-command flakes' flake check -L
check_cachix:
name: Cachix secret present for installer tests
check_secrets:
permissions:
contents: none
name: Check Cachix and Docker secrets present for installer tests
runs-on: ubuntu-latest
outputs:
secret: ${{ steps.secret.outputs.secret }}
cachix: ${{ steps.secret.outputs.cachix }}
docker: ${{ steps.secret.outputs.docker }}
steps:
- name: Check for Cachix secret
- name: Check for secrets
id: secret
env:
_CACHIX_SECRETS: ${{ secrets.CACHIX_SIGNING_KEY }}${{ secrets.CACHIX_AUTH_TOKEN }}
run: echo "::set-output name=secret::${{ env._CACHIX_SECRETS != '' }}"
_DOCKER_SECRETS: ${{ secrets.DOCKERHUB_USERNAME }}${{ secrets.DOCKERHUB_TOKEN }}
run: |
echo "::set-output name=cachix::${{ env._CACHIX_SECRETS != '' }}"
echo "::set-output name=docker::${{ env._DOCKER_SECRETS != '' }}"
installer:
needs: [tests, check_cachix]
if: github.event_name == 'push' && needs.check_cachix.outputs.secret == 'true'
needs: [tests, check_secrets]
if: github.event_name == 'push' && needs.check_secrets.outputs.cachix == 'true'
runs-on: ubuntu-latest
outputs:
installerURL: ${{ steps.prepare-installer.outputs.installerURL }}
steps:
- uses: actions/checkout@v2.4.0
- uses: actions/checkout@v3
with:
fetch-depth: 0
- run: echo CACHIX_NAME="$(echo $GITHUB_REPOSITORY-install-tests | tr "[A-Z]/" "[a-z]-")" >> $GITHUB_ENV
- uses: cachix/install-nix-action@v16
- uses: cachix/install-nix-action@v17
- uses: cachix/cachix-action@v10
with:
name: '${{ env.CACHIX_NAME }}'
@@ -60,37 +68,45 @@ jobs:
run: scripts/prepare-installer-for-github-actions
installer_test:
needs: [installer, check_cachix]
if: github.event_name == 'push' && needs.check_cachix.outputs.secret == 'true'
needs: [installer, check_secrets]
if: github.event_name == 'push' && needs.check_secrets.outputs.cachix == 'true'
strategy:
matrix:
os: [ubuntu-latest, macos-latest]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v2.4.0
- uses: actions/checkout@v3
- run: echo CACHIX_NAME="$(echo $GITHUB_REPOSITORY-install-tests | tr "[A-Z]/" "[a-z]-")" >> $GITHUB_ENV
- uses: cachix/install-nix-action@v16
- uses: cachix/install-nix-action@v17
with:
install_url: '${{needs.installer.outputs.installerURL}}'
install_options: "--tarball-url-prefix https://${{ env.CACHIX_NAME }}.cachix.org/serve"
- run: nix-instantiate -E 'builtins.currentTime' --eval
- run: sudo apt install fish zsh
if: matrix.os == 'ubuntu-latest'
- run: brew install fish
if: matrix.os == 'macos-latest'
- run: exec bash -c "nix-instantiate -E 'builtins.currentTime' --eval"
- run: exec sh -c "nix-instantiate -E 'builtins.currentTime' --eval"
- run: exec zsh -c "nix-instantiate -E 'builtins.currentTime' --eval"
- run: exec fish -c "nix-instantiate -E 'builtins.currentTime' --eval"
docker_push_image:
needs: [check_cachix, tests]
needs: [check_secrets, tests]
if: >-
github.event_name == 'push' &&
github.ref_name == 'master' &&
needs.check_cachix.outputs.secret == 'true'
needs.check_secrets.outputs.cachix == 'true' &&
needs.check_secrets.outputs.docker == 'true'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2.4.0
- uses: actions/checkout@v3
with:
fetch-depth: 0
- uses: cachix/install-nix-action@v16
- uses: cachix/install-nix-action@v17
- run: echo CACHIX_NAME="$(echo $GITHUB_REPOSITORY-install-tests | tr "[A-Z]/" "[a-z]-")" >> $GITHUB_ENV
- run: echo NIX_VERSION="$(nix-instantiate --eval -E '(import ./default.nix).defaultPackage.${builtins.currentSystem}.version' | tr -d \")" >> $GITHUB_ENV
- run: echo NIX_VERSION="$(nix --experimental-features 'nix-command flakes' eval .\#default.version | tr -d \")" >> $GITHUB_ENV
- uses: cachix/cachix-action@v10
if: needs.check_cachix.outputs.secret == 'true'
if: needs.check_secrets.outputs.cachix == 'true'
with:
name: '${{ env.CACHIX_NAME }}'
signingKey: '${{ secrets.CACHIX_SIGNING_KEY }}'
@@ -100,7 +116,7 @@ jobs:
- run: docker tag nix:$NIX_VERSION nixos/nix:$NIX_VERSION
- run: docker tag nix:$NIX_VERSION nixos/nix:master
- name: Login to Docker Hub
uses: docker/login-action@v1
uses: docker/login-action@v2
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}

View File

@@ -1,15 +1,19 @@
name: Hydra status
permissions: read-all
on:
schedule:
- cron: "12,42 * * * *"
workflow_dispatch:
jobs:
check_hydra_status:
name: Check Hydra status
if: github.repository_owner == 'NixOS'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2.4.0
- uses: actions/checkout@v3
with:
fetch-depth: 0
- run: bash scripts/check-hydra-status.sh

7
.gitignore vendored
View File

@@ -22,11 +22,13 @@ perl/Makefile.config
/doc/manual/src/SUMMARY.md
/doc/manual/src/command-ref/new-cli
/doc/manual/src/command-ref/conf-file.md
/doc/manual/src/expressions/builtins.md
/doc/manual/src/language/builtins.md
# /scripts/
/scripts/nix-profile.sh
/scripts/nix-profile-daemon.sh
/scripts/nix-profile.fish
/scripts/nix-profile-daemon.fish
# /src/libexpr/
/src/libexpr/lexer-tab.cc
@@ -35,6 +37,7 @@ perl/Makefile.config
/src/libexpr/parser-tab.hh
/src/libexpr/parser-tab.output
/src/libexpr/nix.tbl
/src/libexpr/tests/libexpr-tests
# /src/libstore/
*.gen.*
@@ -79,6 +82,7 @@ perl/Makefile.config
/tests/shell.drv
/tests/config.nix
/tests/ca/config.nix
/tests/repl-result-out
# /tests/lang/
/tests/lang/*.out
@@ -90,6 +94,7 @@ perl/Makefile.config
/misc/systemd/nix-daemon.service
/misc/systemd/nix-daemon.socket
/misc/systemd/nix-daemon.conf
/misc/upstart/nix-daemon.conf
/src/resolve-system-dependencies/resolve-system-dependencies

View File

@@ -1 +1 @@
2.7.0
2.12.0

View File

@@ -8,6 +8,7 @@ makefiles = \
src/libfetchers/local.mk \
src/libmain/local.mk \
src/libexpr/local.mk \
src/libexpr/tests/local.mk \
src/libcmd/local.mk \
src/nix/local.mk \
src/resolve-system-dependencies/local.mk \
@@ -27,7 +28,8 @@ makefiles = \
OPTIMIZE = 1
ifeq ($(OPTIMIZE), 1)
GLOBAL_CXXFLAGS += -O3
GLOBAL_CXXFLAGS += -O3 $(CXXLTO)
GLOBAL_LDFLAGS += $(CXXLTO)
else
GLOBAL_CXXFLAGS += -O0 -U_FORTIFY_SOURCE
endif

View File

@@ -1,4 +1,3 @@
HOST_OS = @host_os@
AR = @AR@
BDW_GC_LIBS = @BDW_GC_LIBS@
BOOST_LDFLAGS = @BOOST_LDFLAGS@
@@ -7,18 +6,20 @@ CC = @CC@
CFLAGS = @CFLAGS@
CXX = @CXX@
CXXFLAGS = @CXXFLAGS@
CXXLTO = @CXXLTO@
EDITLINE_LIBS = @EDITLINE_LIBS@
ENABLE_S3 = @ENABLE_S3@
GTEST_LIBS = @GTEST_LIBS@
HAVE_LIBCPUID = @HAVE_LIBCPUID@
HAVE_SECCOMP = @HAVE_SECCOMP@
HOST_OS = @host_os@
LDFLAGS = @LDFLAGS@
LIBARCHIVE_LIBS = @LIBARCHIVE_LIBS@
LIBBROTLI_LIBS = @LIBBROTLI_LIBS@
LIBCURL_LIBS = @LIBCURL_LIBS@
LIBSECCOMP_LIBS = @LIBSECCOMP_LIBS@
LOWDOWN_LIBS = @LOWDOWN_LIBS@
OPENSSL_LIBS = @OPENSSL_LIBS@
LIBSECCOMP_LIBS = @LIBSECCOMP_LIBS@
PACKAGE_NAME = @PACKAGE_NAME@
PACKAGE_VERSION = @PACKAGE_VERSION@
SHELL = @bash@
@@ -30,6 +31,7 @@ datadir = @datadir@
datarootdir = @datarootdir@
doc_generate = @doc_generate@
docdir = @docdir@
embedded_sandbox_shell = @embedded_sandbox_shell@
exec_prefix = @exec_prefix@
includedir = @includedir@
libdir = @libdir@

View File

@@ -20,7 +20,7 @@ Information on additional installation methods is available on the [Nix download
## Building And Developing
See our [Hacking guide](https://hydra.nixos.org/job/nix/master/build.x86_64-linux/latest/download-by-type/doc/manual/contributing/hacking.html) in our manual for instruction on how to
See our [Hacking guide](https://nixos.org/manual/nix/stable/contributing/hacking.html) in our manual for instruction on how to
build nix from source with nix-build or how to get a development environment.
## Additional Resources

View File

@@ -1,3 +1,35 @@
diff --git a/darwin_stop_world.c b/darwin_stop_world.c
index 3dbaa3fb..36a1d1f7 100644
--- a/darwin_stop_world.c
+++ b/darwin_stop_world.c
@@ -352,6 +352,7 @@ GC_INNER void GC_push_all_stacks(void)
int nthreads = 0;
word total_size = 0;
mach_msg_type_number_t listcount = (mach_msg_type_number_t)THREAD_TABLE_SZ;
+ size_t stack_limit;
if (!EXPECT(GC_thr_initialized, TRUE))
GC_thr_init();
@@ -407,6 +408,19 @@ GC_INNER void GC_push_all_stacks(void)
GC_push_all_stack_sections(lo, hi, p->traced_stack_sect);
}
if (altstack_lo) {
+ // When a thread goes into a coroutine, we lose its original sp until
+ // control flow returns to the thread.
+ // While in the coroutine, the sp points outside the thread stack,
+ // so we can detect this and push the entire thread stack instead,
+ // as an approximation.
+ // We assume that the coroutine has similarly added its entire stack.
+ // This could be made accurate by cooperating with the application
+ // via new functions and/or callbacks.
+ stack_limit = pthread_get_stacksize_np(p->id);
+ if (altstack_lo >= altstack_hi || altstack_lo < altstack_hi - stack_limit) { // sp outside stack
+ altstack_lo = altstack_hi - stack_limit;
+ }
+
total_size += altstack_hi - altstack_lo;
GC_push_all_stack(altstack_lo, altstack_hi);
}
diff --git a/pthread_stop_world.c b/pthread_stop_world.c
index 4b2c429..1fb4c52 100644
--- a/pthread_stop_world.c

View File

@@ -147,6 +147,20 @@ if test "x$GCC_ATOMIC_BUILTINS_NEED_LIBATOMIC" = xyes; then
LDFLAGS="-latomic $LDFLAGS"
fi
# LTO is currently broken with clang for unknown reasons; ld segfaults in the llvm plugin
AC_ARG_ENABLE(lto, AS_HELP_STRING([--enable-lto],[Enable LTO (only supported with GCC) [default=no]]),
lto=$enableval, lto=no)
if test "$lto" = yes; then
if $CXX --version | grep -q GCC; then
AC_SUBST(CXXLTO, [-flto=jobserver])
else
echo "error: LTO is only supported with GCC at the moment" >&2
exit 1
fi
else
AC_SUBST(CXXLTO, [""])
fi
PKG_PROG_PKG_CONFIG
AC_ARG_ENABLE(shared, AS_HELP_STRING([--enable-shared],[Build shared libraries for Nix [default=yes]]),
@@ -282,18 +296,28 @@ AC_CHECK_FUNCS([setresuid setreuid lchown])
AC_CHECK_FUNCS([strsignal posix_fallocate sysconf])
# This is needed if bzip2 is a static library, and the Nix libraries
# are dynamic.
case "${host_os}" in
darwin*)
LDFLAGS="-all_load $LDFLAGS"
;;
esac
AC_ARG_WITH(sandbox-shell, AS_HELP_STRING([--with-sandbox-shell=PATH],[path of a statically-linked shell to use as /bin/sh in sandboxes]),
sandbox_shell=$withval)
AC_SUBST(sandbox_shell)
if test ${cross_compiling:-no} = no && ! test -z ${sandbox_shell+x}; then
AC_MSG_CHECKING([whether sandbox-shell has the standalone feature])
# busybox shell sometimes allows executing other busybox applets,
# even if they are not in the path, breaking our sandbox
if PATH= $sandbox_shell -c "busybox" 2>&1 | grep -qv "not found"; then
AC_MSG_RESULT(enabled)
AC_MSG_ERROR([Please disable busybox FEATURE_SH_STANDALONE])
else
AC_MSG_RESULT(disabled)
fi
fi
AC_ARG_ENABLE(embedded-sandbox-shell, AS_HELP_STRING([--enable-embedded-sandbox-shell],[include the sandbox shell in the Nix binary [default=no]]),
embedded_sandbox_shell=$enableval, embedded_sandbox_shell=no)
AC_SUBST(embedded_sandbox_shell)
if test "$embedded_sandbox_shell" = yes; then
AC_DEFINE(HAVE_EMBEDDED_SANDBOX_SHELL, 1, [Include the sandbox shell in the Nix binary.])
fi
# Expand all variables in config.status.
test "$prefix" = NONE && prefix=$ac_default_prefix

31
doc/manual/anchors.jq Executable file
View File

@@ -0,0 +1,31 @@
"\\[\\]\\{#(?<anchor>[^\\}]+?)\\}" as $empty_anchor_regex |
"\\[(?<text>[^\\]]+?)\\]\\{#(?<anchor>[^\\}]+?)\\}" as $anchor_regex |
def transform_anchors_html:
. | gsub($empty_anchor_regex; "<a name=\"" + .anchor + "\"></a>")
| gsub($anchor_regex; "<a href=\"#" + .anchor + "\" id=\"" + .anchor + "\">" + .text + "</a>");
def transform_anchors_strip:
. | gsub($empty_anchor_regex; "")
| gsub($anchor_regex; .text);
def map_contents_recursively(transformer):
. + {
Chapter: (.Chapter + {
content: .Chapter.content | transformer,
sub_items: .Chapter.sub_items | map(map_contents_recursively(transformer)),
}),
};
def process_command:
.[0] as $context |
.[1] as $body |
$body + {
sections: $body.sections | map(map_contents_recursively(if $context.renderer == "html" then transform_anchors_html else transform_anchors_strip end)),
};
process_command

View File

@@ -1,2 +1,7 @@
[output.html]
additional-css = ["custom.css"]
additional-js = ["redirects.js"]
[preprocessor.anchors]
renderers = ["html"]
command = "jq --from-file doc/manual/anchors.jq"

View File

@@ -6,9 +6,9 @@ builtins:
concatStrings (map
(name:
let builtin = builtins.${name}; in
"<dt><code>${name} "
"<dt id=\"builtins-${name}\"><a href=\"#builtins-${name}\"><code>${name} "
+ concatStringsSep " " (map (s: "<var>${s}</var>") builtin.args)
+ "</code></dt>"
+ "</code></a></dt>"
+ "<dd>\n\n"
+ builtin.doc
+ "\n\n</dd>"

View File

@@ -1,99 +1,110 @@
{ command, renderLinks ? false }:
{ command }:
with builtins;
with import ./utils.nix;
let
showCommand =
{ command, def, filename }:
''
**Warning**: This program is **experimental** and its interface is subject to change.
''
+ "# Name\n\n"
+ "`${command}` - ${def.description}\n\n"
+ "# Synopsis\n\n"
+ showSynopsis { inherit command; args = def.args; }
+ (if def.commands or {} != {}
then
let
categories = sort (x: y: x.id < y.id) (unique (map (cmd: cmd.category) (attrValues def.commands)));
listCommands = cmds:
concatStrings (map (name:
"* "
+ (if renderLinks
then "[`${command} ${name}`](./${appendName filename name}.md)"
else "`${command} ${name}`")
+ " - ${cmds.${name}.description}\n")
(attrNames cmds));
in
"where *subcommand* is one of the following:\n\n"
# FIXME: group by category
+ (if length categories > 1
then
concatStrings (map
(cat:
"**${toString cat.description}:**\n\n"
+ listCommands (filterAttrs (n: v: v.category == cat) def.commands)
+ "\n"
) categories)
+ "\n"
else
listCommands def.commands
+ "\n")
else "")
+ (if def ? doc
then def.doc + "\n\n"
else "")
+ (let s = showOptions def.flags; in
if s != ""
then "# Options\n\n${s}"
else "")
;
showCommand = { command, details, filename }:
let
result = ''
> **Warning** \
> This program is **experimental** and its interface is subject to change.
# Name
`${command}` - ${details.description}
# Synopsis
${showSynopsis command details.args}
${maybeSubcommands}
${maybeDocumentation}
${maybeOptions}
'';
showSynopsis = command: args:
let
showArgument = arg: "*${arg.label}*" + (if arg ? arity then "" else "...");
arguments = concatStringsSep " " (map showArgument args);
in ''
`${command}` [*option*...] ${arguments}
'';
maybeSubcommands = if details ? commands && details.commands != {}
then ''
where *subcommand* is one of the following:
${subcommands}
''
else "";
subcommands = if length categories > 1
then listCategories
else listSubcommands details.commands;
categories = sort (x: y: x.id < y.id) (unique (map (cmd: cmd.category) (attrValues details.commands)));
listCategories = concatStrings (map showCategory categories);
showCategory = cat: ''
**${toString cat.description}:**
${listSubcommands (filterAttrs (n: v: v.category == cat) details.commands)}
'';
listSubcommands = cmds: concatStrings (attrValues (mapAttrs showSubcommand cmds));
showSubcommand = name: subcmd: ''
* [`${command} ${name}`](./${appendName filename name}.md) - ${subcmd.description}
'';
maybeDocumentation = if details ? doc then details.doc else "";
maybeOptions = if details.flags == {} then "" else ''
# Options
${showOptions details.flags}
'';
showOptions = options:
let
showCategory = cat: ''
${if cat != "" then "**${cat}:**" else ""}
${listOptions (filterAttrs (n: v: v.category == cat) options)}
'';
listOptions = opts: concatStringsSep "\n" (attrValues (mapAttrs showOption opts));
showOption = name: option:
let
shortName = if option ? shortName then "/ `-${option.shortName}`" else "";
labels = if option ? labels then (concatStringsSep " " (map (s: "*${s}*") option.labels)) else "";
in trim ''
- `--${name}` ${shortName} ${labels}
${option.description}
'';
categories = sort builtins.lessThan (unique (map (cmd: cmd.category) (attrValues options)));
in concatStrings (map showCategory categories);
in squash result;
appendName = filename: name: (if filename == "nix" then "nix3" else filename) + "-" + name;
showOptions = flags:
processCommand = { command, details, filename }:
let
categories = sort builtins.lessThan (unique (map (cmd: cmd.category) (attrValues flags)));
in
concatStrings (map
(cat:
(if cat != ""
then "**${cat}:**\n\n"
else "")
+ concatStrings
(map (longName:
let
flag = flags.${longName};
in
" - `--${longName}`"
+ (if flag ? shortName then " / `-${flag.shortName}`" else "")
+ (if flag ? labels then " " + (concatStringsSep " " (map (s: "*${s}*") flag.labels)) else "")
+ " \n"
+ " " + flag.description + "\n\n"
) (attrNames (filterAttrs (n: v: v.category == cat) flags))))
categories);
cmd = {
inherit command;
name = filename + ".md";
value = showCommand { inherit command details filename; };
};
subcommand = subCmd: processCommand {
command = command + " " + subCmd;
details = details.commands.${subCmd};
filename = appendName filename subCmd;
};
in [ cmd ] ++ concatMap subcommand (attrNames details.commands or {});
showSynopsis =
{ command, args }:
"`${command}` [*option*...] ${concatStringsSep " "
(map (arg: "*${arg.label}*" + (if arg ? arity then "" else "...")) args)}\n\n";
manpages = processCommand {
command = "nix";
details = builtins.fromJSON command;
filename = "nix";
};
processCommand = { command, def, filename }:
[ { name = filename + ".md"; value = showCommand { inherit command def filename; }; inherit command; } ]
++ concatMap
(name: processCommand {
filename = appendName filename name;
command = command + " " + name;
def = def.commands.${name};
})
(attrNames def.commands or {});
tableOfContents = let
showEntry = page:
" - [${page.command}](command-ref/new-cli/${page.name})";
in concatStringsSep "\n" (map showEntry manpages) + "\n";
in
let
manpages = processCommand { filename = "nix"; command = "nix"; def = builtins.fromJSON command; };
summary = concatStrings (map (manpage: " - [${manpage.command}](command-ref/new-cli/${manpage.name})\n") manpages);
in
(listToAttrs manpages) // { "SUMMARY.md" = summary; }
in (listToAttrs manpages) // { "SUMMARY.md" = tableOfContents; }

View File

@@ -6,7 +6,8 @@ options:
concatStrings (map
(name:
let option = options.${name}; in
" - `${name}` \n\n"
" - [`${name}`](#conf-${name})"
+ "<p id=\"conf-${name}\"></p>\n\n"
+ concatStrings (map (s: " ${s}\n") (splitLines option.description)) + "\n\n"
+ (if option.documentDefault
then " **Default:** " + (

View File

@@ -1,5 +1,9 @@
ifeq ($(doc_generate),yes)
MANUAL_SRCS := \
$(call rwildcard, $(d)/src, *.md) \
$(call rwildcard, $(d)/src, */*.md)
# Generate man pages.
man-pages := $(foreach n, \
nix-env.1 nix-build.1 nix-shell.1 nix-store.1 nix-instantiate.1 \
@@ -46,7 +50,7 @@ $(d)/src/SUMMARY.md: $(d)/src/SUMMARY.md.in $(d)/src/command-ref/new-cli
$(d)/src/command-ref/new-cli: $(d)/nix.json $(d)/generate-manpage.nix $(bindir)/nix
@rm -rf $@
$(trace-gen) $(nix-eval) --write-to $@ --expr 'import doc/manual/generate-manpage.nix { command = builtins.readFile $<; renderLinks = true; }'
$(trace-gen) $(nix-eval) --write-to $@ --expr 'import doc/manual/generate-manpage.nix { command = builtins.readFile $<; }'
$(d)/src/command-ref/conf-file.md: $(d)/conf-file.json $(d)/generate-options.nix $(d)/src/command-ref/conf-file-prefix.md $(bindir)/nix
@cat doc/manual/src/command-ref/conf-file-prefix.md > $@.tmp
@@ -61,10 +65,10 @@ $(d)/conf-file.json: $(bindir)/nix
$(trace-gen) $(dummy-env) $(bindir)/nix show-config --json --experimental-features nix-command > $@.tmp
@mv $@.tmp $@
$(d)/src/expressions/builtins.md: $(d)/builtins.json $(d)/generate-builtins.nix $(d)/src/expressions/builtins-prefix.md $(bindir)/nix
@cat doc/manual/src/expressions/builtins-prefix.md > $@.tmp
$(d)/src/language/builtins.md: $(d)/builtins.json $(d)/generate-builtins.nix $(d)/src/language/builtins-prefix.md $(bindir)/nix
@cat doc/manual/src/language/builtins-prefix.md > $@.tmp
$(trace-gen) $(nix-eval) --expr 'import doc/manual/generate-builtins.nix (builtins.fromJSON (builtins.readFile $<))' >> $@.tmp
@cat doc/manual/src/expressions/builtins-suffix.md >> $@.tmp
@cat doc/manual/src/language/builtins-suffix.md >> $@.tmp
@mv $@.tmp $@
$(d)/builtins.json: $(bindir)/nix
@@ -72,6 +76,7 @@ $(d)/builtins.json: $(bindir)/nix
@mv $@.tmp $@
# Generate the HTML manual.
html: $(docdir)/manual/index.html
install: $(docdir)/manual/index.html
# Generate 'nix' manpages.
@@ -91,12 +96,12 @@ doc/manual/generated/man1/nix3-manpages: $(d)/src/command-ref/new-cli
if [[ $$name = SUMMARY ]]; then continue; fi; \
printf "Title: %s\n\n" "$$name" > $$tmpFile; \
cat $$i >> $$tmpFile; \
lowdown -sT man -M section=1 $$tmpFile -o $(DESTDIR)$$(dirname $@)/$$name.1; \
lowdown -sT man --nroff-nolinks -M section=1 $$tmpFile -o $(DESTDIR)$$(dirname $@)/$$name.1; \
rm $$tmpFile; \
done
@touch $@
$(docdir)/manual/index.html: $(MANUAL_SRCS) $(d)/book.toml $(d)/custom.css $(d)/src/SUMMARY.md $(d)/src/command-ref/new-cli $(d)/src/command-ref/conf-file.md $(d)/src/expressions/builtins.md $(call rwildcard, $(d)/src, *.md)
$(docdir)/manual/index.html: $(MANUAL_SRCS) $(d)/book.toml $(d)/anchors.jq $(d)/custom.css $(d)/src/SUMMARY.md $(d)/src/command-ref/new-cli $(d)/src/command-ref/conf-file.md $(d)/src/language/builtins.md
$(trace-gen) RUST_LOG=warn mdbook build doc/manual -d $(DESTDIR)$(docdir)/manual
endif

330
doc/manual/redirects.js Normal file
View File

@@ -0,0 +1,330 @@
// Redirects from old DocBook manual.
var redirects = {
"#part-advanced-topics": "advanced-topics/advanced-topics.html",
"#chap-tuning-cores-and-jobs": "advanced-topics/cores-vs-jobs.html",
"#chap-diff-hook": "advanced-topics/diff-hook.html",
"#check-dirs-are-unregistered": "advanced-topics/diff-hook.html#check-dirs-are-unregistered",
"#chap-distributed-builds": "advanced-topics/distributed-builds.html",
"#chap-post-build-hook": "advanced-topics/post-build-hook.html",
"#chap-post-build-hook-caveats": "advanced-topics/post-build-hook.html#implementation-caveats",
"#part-command-ref": "command-ref/command-ref.html",
"#conf-allow-import-from-derivation": "command-ref/conf-file.html#conf-allow-import-from-derivation",
"#conf-allow-new-privileges": "command-ref/conf-file.html#conf-allow-new-privileges",
"#conf-allowed-uris": "command-ref/conf-file.html#conf-allowed-uris",
"#conf-allowed-users": "command-ref/conf-file.html#conf-allowed-users",
"#conf-auto-optimise-store": "command-ref/conf-file.html#conf-auto-optimise-store",
"#conf-binary-cache-public-keys": "command-ref/conf-file.html#conf-binary-cache-public-keys",
"#conf-binary-caches": "command-ref/conf-file.html#conf-binary-caches",
"#conf-build-compress-log": "command-ref/conf-file.html#conf-build-compress-log",
"#conf-build-cores": "command-ref/conf-file.html#conf-build-cores",
"#conf-build-extra-chroot-dirs": "command-ref/conf-file.html#conf-build-extra-chroot-dirs",
"#conf-build-extra-sandbox-paths": "command-ref/conf-file.html#conf-build-extra-sandbox-paths",
"#conf-build-fallback": "command-ref/conf-file.html#conf-build-fallback",
"#conf-build-max-jobs": "command-ref/conf-file.html#conf-build-max-jobs",
"#conf-build-max-log-size": "command-ref/conf-file.html#conf-build-max-log-size",
"#conf-build-max-silent-time": "command-ref/conf-file.html#conf-build-max-silent-time",
"#conf-build-repeat": "command-ref/conf-file.html#conf-build-repeat",
"#conf-build-timeout": "command-ref/conf-file.html#conf-build-timeout",
"#conf-build-use-chroot": "command-ref/conf-file.html#conf-build-use-chroot",
"#conf-build-use-sandbox": "command-ref/conf-file.html#conf-build-use-sandbox",
"#conf-build-use-substitutes": "command-ref/conf-file.html#conf-build-use-substitutes",
"#conf-build-users-group": "command-ref/conf-file.html#conf-build-users-group",
"#conf-builders": "command-ref/conf-file.html#conf-builders",
"#conf-builders-use-substitutes": "command-ref/conf-file.html#conf-builders-use-substitutes",
"#conf-compress-build-log": "command-ref/conf-file.html#conf-compress-build-log",
"#conf-connect-timeout": "command-ref/conf-file.html#conf-connect-timeout",
"#conf-cores": "command-ref/conf-file.html#conf-cores",
"#conf-diff-hook": "command-ref/conf-file.html#conf-diff-hook",
"#conf-enforce-determinism": "command-ref/conf-file.html#conf-enforce-determinism",
"#conf-env-keep-derivations": "command-ref/conf-file.html#conf-env-keep-derivations",
"#conf-extra-binary-caches": "command-ref/conf-file.html#conf-extra-binary-caches",
"#conf-extra-platforms": "command-ref/conf-file.html#conf-extra-platforms",
"#conf-extra-sandbox-paths": "command-ref/conf-file.html#conf-extra-sandbox-paths",
"#conf-extra-substituters": "command-ref/conf-file.html#conf-extra-substituters",
"#conf-fallback": "command-ref/conf-file.html#conf-fallback",
"#conf-fsync-metadata": "command-ref/conf-file.html#conf-fsync-metadata",
"#conf-gc-keep-derivations": "command-ref/conf-file.html#conf-gc-keep-derivations",
"#conf-gc-keep-outputs": "command-ref/conf-file.html#conf-gc-keep-outputs",
"#conf-hashed-mirrors": "command-ref/conf-file.html#conf-hashed-mirrors",
"#conf-http-connections": "command-ref/conf-file.html#conf-http-connections",
"#conf-keep-build-log": "command-ref/conf-file.html#conf-keep-build-log",
"#conf-keep-derivations": "command-ref/conf-file.html#conf-keep-derivations",
"#conf-keep-env-derivations": "command-ref/conf-file.html#conf-keep-env-derivations",
"#conf-keep-outputs": "command-ref/conf-file.html#conf-keep-outputs",
"#conf-max-build-log-size": "command-ref/conf-file.html#conf-max-build-log-size",
"#conf-max-free": "command-ref/conf-file.html#conf-max-free",
"#conf-max-jobs": "command-ref/conf-file.html#conf-max-jobs",
"#conf-max-silent-time": "command-ref/conf-file.html#conf-max-silent-time",
"#conf-min-free": "command-ref/conf-file.html#conf-min-free",
"#conf-narinfo-cache-negative-ttl": "command-ref/conf-file.html#conf-narinfo-cache-negative-ttl",
"#conf-narinfo-cache-positive-ttl": "command-ref/conf-file.html#conf-narinfo-cache-positive-ttl",
"#conf-netrc-file": "command-ref/conf-file.html#conf-netrc-file",
"#conf-plugin-files": "command-ref/conf-file.html#conf-plugin-files",
"#conf-post-build-hook": "command-ref/conf-file.html#conf-post-build-hook",
"#conf-pre-build-hook": "command-ref/conf-file.html#conf-pre-build-hook",
"#conf-repeat": "command-ref/conf-file.html#conf-repeat",
"#conf-require-sigs": "command-ref/conf-file.html#conf-require-sigs",
"#conf-restrict-eval": "command-ref/conf-file.html#conf-restrict-eval",
"#conf-run-diff-hook": "command-ref/conf-file.html#conf-run-diff-hook",
"#conf-sandbox": "command-ref/conf-file.html#conf-sandbox",
"#conf-sandbox-dev-shm-size": "command-ref/conf-file.html#conf-sandbox-dev-shm-size",
"#conf-sandbox-paths": "command-ref/conf-file.html#conf-sandbox-paths",
"#conf-secret-key-files": "command-ref/conf-file.html#conf-secret-key-files",
"#conf-show-trace": "command-ref/conf-file.html#conf-show-trace",
"#conf-stalled-download-timeout": "command-ref/conf-file.html#conf-stalled-download-timeout",
"#conf-substitute": "command-ref/conf-file.html#conf-substitute",
"#conf-substituters": "command-ref/conf-file.html#conf-substituters",
"#conf-system": "command-ref/conf-file.html#conf-system",
"#conf-system-features": "command-ref/conf-file.html#conf-system-features",
"#conf-tarball-ttl": "command-ref/conf-file.html#conf-tarball-ttl",
"#conf-timeout": "command-ref/conf-file.html#conf-timeout",
"#conf-trace-function-calls": "command-ref/conf-file.html#conf-trace-function-calls",
"#conf-trusted-binary-caches": "command-ref/conf-file.html#conf-trusted-binary-caches",
"#conf-trusted-public-keys": "command-ref/conf-file.html#conf-trusted-public-keys",
"#conf-trusted-substituters": "command-ref/conf-file.html#conf-trusted-substituters",
"#conf-trusted-users": "command-ref/conf-file.html#conf-trusted-users",
"#extra-sandbox-paths": "command-ref/conf-file.html#extra-sandbox-paths",
"#sec-conf-file": "command-ref/conf-file.html",
"#env-NIX_PATH": "command-ref/env-common.html#env-NIX_PATH",
"#env-common": "command-ref/env-common.html",
"#envar-remote": "command-ref/env-common.html#env-NIX_REMOTE",
"#sec-common-env": "command-ref/env-common.html",
"#ch-files": "command-ref/files.html",
"#ch-main-commands": "command-ref/main-commands.html",
"#opt-out-link": "command-ref/nix-build.html#opt-out-link",
"#sec-nix-build": "command-ref/nix-build.html",
"#sec-nix-channel": "command-ref/nix-channel.html",
"#sec-nix-collect-garbage": "command-ref/nix-collect-garbage.html",
"#sec-nix-copy-closure": "command-ref/nix-copy-closure.html",
"#sec-nix-daemon": "command-ref/nix-daemon.html",
"#refsec-nix-env-install-examples": "command-ref/nix-env.html#examples",
"#rsec-nix-env-install": "command-ref/nix-env.html#operation---install",
"#rsec-nix-env-set": "command-ref/nix-env.html#operation---set",
"#rsec-nix-env-set-flag": "command-ref/nix-env.html#operation---set-flag",
"#rsec-nix-env-upgrade": "command-ref/nix-env.html#operation---upgrade",
"#sec-nix-env": "command-ref/nix-env.html",
"#ssec-version-comparisons": "command-ref/nix-env.html#versions",
"#sec-nix-hash": "command-ref/nix-hash.html",
"#sec-nix-instantiate": "command-ref/nix-instantiate.html",
"#sec-nix-prefetch-url": "command-ref/nix-prefetch-url.html",
"#sec-nix-shell": "command-ref/nix-shell.html",
"#ssec-nix-shell-shebang": "command-ref/nix-shell.html#use-as-a--interpreter",
"#nixref-queries": "command-ref/nix-store.html#queries",
"#opt-add-root": "command-ref/nix-store.html#opt-add-root",
"#refsec-nix-store-dump": "command-ref/nix-store.html#operation---dump",
"#refsec-nix-store-export": "command-ref/nix-store.html#operation---export",
"#refsec-nix-store-import": "command-ref/nix-store.html#operation---import",
"#refsec-nix-store-query": "command-ref/nix-store.html#operation---query",
"#refsec-nix-store-verify": "command-ref/nix-store.html#operation---verify",
"#rsec-nix-store-gc": "command-ref/nix-store.html#operation---gc",
"#rsec-nix-store-generate-binary-cache-key": "command-ref/nix-store.html#operation---generate-binary-cache-key",
"#rsec-nix-store-realise": "command-ref/nix-store.html#operation---realise",
"#rsec-nix-store-serve": "command-ref/nix-store.html#operation---serve",
"#sec-nix-store": "command-ref/nix-store.html",
"#opt-I": "command-ref/opt-common.html#opt-I",
"#opt-attr": "command-ref/opt-common.html#opt-attr",
"#opt-common": "command-ref/opt-common.html",
"#opt-cores": "command-ref/opt-common.html#opt-cores",
"#opt-log-format": "command-ref/opt-common.html#opt-log-format",
"#opt-max-jobs": "command-ref/opt-common.html#opt-max-jobs",
"#opt-max-silent-time": "command-ref/opt-common.html#opt-max-silent-time",
"#opt-timeout": "command-ref/opt-common.html#opt-timeout",
"#sec-common-options": "command-ref/opt-common.html",
"#ch-utilities": "command-ref/utilities.html",
"#chap-hacking": "contributing/hacking.html",
"#adv-attr-allowSubstitutes": "language/advanced-attributes.html#adv-attr-allowSubstitutes",
"#adv-attr-allowedReferences": "language/advanced-attributes.html#adv-attr-allowedReferences",
"#adv-attr-allowedRequisites": "language/advanced-attributes.html#adv-attr-allowedRequisites",
"#adv-attr-disallowedReferences": "language/advanced-attributes.html#adv-attr-disallowedReferences",
"#adv-attr-disallowedRequisites": "language/advanced-attributes.html#adv-attr-disallowedRequisites",
"#adv-attr-exportReferencesGraph": "language/advanced-attributes.html#adv-attr-exportReferencesGraph",
"#adv-attr-impureEnvVars": "language/advanced-attributes.html#adv-attr-impureEnvVars",
"#adv-attr-outputHash": "language/advanced-attributes.html#adv-attr-outputHash",
"#adv-attr-outputHashAlgo": "language/advanced-attributes.html#adv-attr-outputHashAlgo",
"#adv-attr-outputHashMode": "language/advanced-attributes.html#adv-attr-outputHashMode",
"#adv-attr-passAsFile": "language/advanced-attributes.html#adv-attr-passAsFile",
"#adv-attr-preferLocalBuild": "language/advanced-attributes.html#adv-attr-preferLocalBuild",
"#fixed-output-drvs": "language/advanced-attributes.html#adv-attr-outputHash",
"#sec-advanced-attributes": "language/advanced-attributes.html",
"#builtin-abort": "language/builtins.html#builtins-abort",
"#builtin-add": "language/builtins.html#builtins-add",
"#builtin-all": "language/builtins.html#builtins-all",
"#builtin-any": "language/builtins.html#builtins-any",
"#builtin-attrNames": "language/builtins.html#builtins-attrNames",
"#builtin-attrValues": "language/builtins.html#builtins-attrValues",
"#builtin-baseNameOf": "language/builtins.html#builtins-baseNameOf",
"#builtin-bitAnd": "language/builtins.html#builtins-bitAnd",
"#builtin-bitOr": "language/builtins.html#builtins-bitOr",
"#builtin-bitXor": "language/builtins.html#builtins-bitXor",
"#builtin-builtins": "language/builtins.html#builtins-builtins",
"#builtin-compareVersions": "language/builtins.html#builtins-compareVersions",
"#builtin-concatLists": "language/builtins.html#builtins-concatLists",
"#builtin-concatStringsSep": "language/builtins.html#builtins-concatStringsSep",
"#builtin-currentSystem": "language/builtins.html#builtins-currentSystem",
"#builtin-deepSeq": "language/builtins.html#builtins-deepSeq",
"#builtin-derivation": "language/builtins.html#builtins-derivation",
"#builtin-dirOf": "language/builtins.html#builtins-dirOf",
"#builtin-div": "language/builtins.html#builtins-div",
"#builtin-elem": "language/builtins.html#builtins-elem",
"#builtin-elemAt": "language/builtins.html#builtins-elemAt",
"#builtin-fetchGit": "language/builtins.html#builtins-fetchGit",
"#builtin-fetchTarball": "language/builtins.html#builtins-fetchTarball",
"#builtin-fetchurl": "language/builtins.html#builtins-fetchurl",
"#builtin-filterSource": "language/builtins.html#builtins-filterSource",
"#builtin-foldl-prime": "language/builtins.html#builtins-foldl-prime",
"#builtin-fromJSON": "language/builtins.html#builtins-fromJSON",
"#builtin-functionArgs": "language/builtins.html#builtins-functionArgs",
"#builtin-genList": "language/builtins.html#builtins-genList",
"#builtin-getAttr": "language/builtins.html#builtins-getAttr",
"#builtin-getEnv": "language/builtins.html#builtins-getEnv",
"#builtin-hasAttr": "language/builtins.html#builtins-hasAttr",
"#builtin-hashFile": "language/builtins.html#builtins-hashFile",
"#builtin-hashString": "language/builtins.html#builtins-hashString",
"#builtin-head": "language/builtins.html#builtins-head",
"#builtin-import": "language/builtins.html#builtins-import",
"#builtin-intersectAttrs": "language/builtins.html#builtins-intersectAttrs",
"#builtin-isAttrs": "language/builtins.html#builtins-isAttrs",
"#builtin-isBool": "language/builtins.html#builtins-isBool",
"#builtin-isFloat": "language/builtins.html#builtins-isFloat",
"#builtin-isFunction": "language/builtins.html#builtins-isFunction",
"#builtin-isInt": "language/builtins.html#builtins-isInt",
"#builtin-isList": "language/builtins.html#builtins-isList",
"#builtin-isNull": "language/builtins.html#builtins-isNull",
"#builtin-isString": "language/builtins.html#builtins-isString",
"#builtin-length": "language/builtins.html#builtins-length",
"#builtin-lessThan": "language/builtins.html#builtins-lessThan",
"#builtin-listToAttrs": "language/builtins.html#builtins-listToAttrs",
"#builtin-map": "language/builtins.html#builtins-map",
"#builtin-match": "language/builtins.html#builtins-match",
"#builtin-mul": "language/builtins.html#builtins-mul",
"#builtin-parseDrvName": "language/builtins.html#builtins-parseDrvName",
"#builtin-path": "language/builtins.html#builtins-path",
"#builtin-pathExists": "language/builtins.html#builtins-pathExists",
"#builtin-placeholder": "language/builtins.html#builtins-placeholder",
"#builtin-readDir": "language/builtins.html#builtins-readDir",
"#builtin-readFile": "language/builtins.html#builtins-readFile",
"#builtin-removeAttrs": "language/builtins.html#builtins-removeAttrs",
"#builtin-replaceStrings": "language/builtins.html#builtins-replaceStrings",
"#builtin-seq": "language/builtins.html#builtins-seq",
"#builtin-sort": "language/builtins.html#builtins-sort",
"#builtin-split": "language/builtins.html#builtins-split",
"#builtin-splitVersion": "language/builtins.html#builtins-splitVersion",
"#builtin-stringLength": "language/builtins.html#builtins-stringLength",
"#builtin-sub": "language/builtins.html#builtins-sub",
"#builtin-substring": "language/builtins.html#builtins-substring",
"#builtin-tail": "language/builtins.html#builtins-tail",
"#builtin-throw": "language/builtins.html#builtins-throw",
"#builtin-toFile": "language/builtins.html#builtins-toFile",
"#builtin-toJSON": "language/builtins.html#builtins-toJSON",
"#builtin-toPath": "language/builtins.html#builtins-toPath",
"#builtin-toString": "language/builtins.html#builtins-toString",
"#builtin-toXML": "language/builtins.html#builtins-toXML",
"#builtin-trace": "language/builtins.html#builtins-trace",
"#builtin-tryEval": "language/builtins.html#builtins-tryEval",
"#builtin-typeOf": "language/builtins.html#builtins-typeOf",
"#ssec-builtins": "language/builtins.html",
"#attr-system": "language/derivations.html#attr-system",
"#ssec-derivation": "language/derivations.html",
"#ch-expression-language": "language/index.html",
"#sec-constructs": "language/constructs.html",
"#sect-let-language": "language/constructs.html#let-language",
"#ss-functions": "language/constructs.html#functions",
"#sec-language-operators": "language/operators.html",
"#table-operators": "language/operators.html",
"#ssec-values": "language/values.html",
"#gloss-closure": "glossary.html#gloss-closure",
"#gloss-derivation": "glossary.html#gloss-derivation",
"#gloss-deriver": "glossary.html#gloss-deriver",
"#gloss-nar": "glossary.html#gloss-nar",
"#gloss-output-path": "glossary.html#gloss-output-path",
"#gloss-profile": "glossary.html#gloss-profile",
"#gloss-reachable": "glossary.html#gloss-reachable",
"#gloss-reference": "glossary.html#gloss-reference",
"#gloss-substitute": "glossary.html#gloss-substitute",
"#gloss-user-env": "glossary.html#gloss-user-env",
"#gloss-validity": "glossary.html#gloss-validity",
"#part-glossary": "glossary.html",
"#sec-building-source": "installation/building-source.html",
"#ch-env-variables": "installation/env-variables.html",
"#sec-installer-proxy-settings": "installation/env-variables.html#proxy-environment-variables",
"#sec-nix-ssl-cert-file": "installation/env-variables.html#nix_ssl_cert_file",
"#sec-nix-ssl-cert-file-with-nix-daemon-and-macos": "installation/env-variables.html#nix_ssl_cert_file-with-macos-and-the-nix-daemon",
"#chap-installation": "installation/installation.html",
"#ch-installing-binary": "installation/installing-binary.html",
"#sect-macos-installation": "installation/installing-binary.html#macos-installation",
"#sect-macos-installation-change-store-prefix": "installation/installing-binary.html#macos-installation",
"#sect-macos-installation-encrypted-volume": "installation/installing-binary.html#macos-installation",
"#sect-macos-installation-recommended-notes": "installation/installing-binary.html#macos-installation",
"#sect-macos-installation-symlink": "installation/installing-binary.html#macos-installation",
"#sect-multi-user-installation": "installation/installing-binary.html#multi-user-installation",
"#sect-nix-install-binary-tarball": "installation/installing-binary.html#installing-from-a-binary-tarball",
"#sect-nix-install-pinned-version-url": "installation/installing-binary.html#installing-a-pinned-nix-version-from-a-url",
"#sect-single-user-installation": "installation/installing-binary.html#single-user-installation",
"#ch-installing-source": "installation/installing-source.html",
"#ssec-multi-user": "installation/multi-user.html",
"#ch-nix-security": "installation/nix-security.html",
"#sec-obtaining-source": "installation/obtaining-source.html",
"#sec-prerequisites-source": "installation/prerequisites-source.html",
"#sec-single-user": "installation/single-user.html",
"#ch-supported-platforms": "installation/supported-platforms.html",
"#ch-upgrading-nix": "installation/upgrading.html",
"#ch-about-nix": "introduction.html",
"#chap-introduction": "introduction.html",
"#ch-basic-package-mgmt": "package-management/basic-package-mgmt.html",
"#ssec-binary-cache-substituter": "package-management/binary-cache-substituter.html",
"#sec-channels": "package-management/channels.html",
"#ssec-copy-closure": "package-management/copy-closure.html",
"#sec-garbage-collection": "package-management/garbage-collection.html",
"#ssec-gc-roots": "package-management/garbage-collector-roots.html",
"#chap-package-management": "package-management/package-management.html",
"#sec-profiles": "package-management/profiles.html",
"#ssec-s3-substituter": "package-management/s3-substituter.html",
"#ssec-s3-substituter-anonymous-reads": "package-management/s3-substituter.html#anonymous-reads-to-your-s3-compatible-binary-cache",
"#ssec-s3-substituter-authenticated-reads": "package-management/s3-substituter.html#authenticated-reads-to-your-s3-binary-cache",
"#ssec-s3-substituter-authenticated-writes": "package-management/s3-substituter.html#authenticated-writes-to-your-s3-compatible-binary-cache",
"#sec-sharing-packages": "package-management/sharing-packages.html",
"#ssec-ssh-substituter": "package-management/ssh-substituter.html",
"#chap-quick-start": "quick-start.html",
"#sec-relnotes": "release-notes/release-notes.html",
"#ch-relnotes-0.10.1": "release-notes/rl-0.10.1.html",
"#ch-relnotes-0.10": "release-notes/rl-0.10.html",
"#ssec-relnotes-0.11": "release-notes/rl-0.11.html",
"#ssec-relnotes-0.12": "release-notes/rl-0.12.html",
"#ssec-relnotes-0.13": "release-notes/rl-0.13.html",
"#ssec-relnotes-0.14": "release-notes/rl-0.14.html",
"#ssec-relnotes-0.15": "release-notes/rl-0.15.html",
"#ssec-relnotes-0.16": "release-notes/rl-0.16.html",
"#ch-relnotes-0.5": "release-notes/rl-0.5.html",
"#ch-relnotes-0.6": "release-notes/rl-0.6.html",
"#ch-relnotes-0.7": "release-notes/rl-0.7.html",
"#ch-relnotes-0.8.1": "release-notes/rl-0.8.1.html",
"#ch-relnotes-0.8": "release-notes/rl-0.8.html",
"#ch-relnotes-0.9.1": "release-notes/rl-0.9.1.html",
"#ch-relnotes-0.9.2": "release-notes/rl-0.9.2.html",
"#ch-relnotes-0.9": "release-notes/rl-0.9.html",
"#ssec-relnotes-1.0": "release-notes/rl-1.0.html",
"#ssec-relnotes-1.1": "release-notes/rl-1.1.html",
"#ssec-relnotes-1.10": "release-notes/rl-1.10.html",
"#ssec-relnotes-1.11.10": "release-notes/rl-1.11.10.html",
"#ssec-relnotes-1.11": "release-notes/rl-1.11.html",
"#ssec-relnotes-1.2": "release-notes/rl-1.2.html",
"#ssec-relnotes-1.3": "release-notes/rl-1.3.html",
"#ssec-relnotes-1.4": "release-notes/rl-1.4.html",
"#ssec-relnotes-1.5.1": "release-notes/rl-1.5.1.html",
"#ssec-relnotes-1.5.2": "release-notes/rl-1.5.2.html",
"#ssec-relnotes-1.5": "release-notes/rl-1.5.html",
"#ssec-relnotes-1.6.1": "release-notes/rl-1.6.1.html",
"#ssec-relnotes-1.6.0": "release-notes/rl-1.6.html",
"#ssec-relnotes-1.7": "release-notes/rl-1.7.html",
"#ssec-relnotes-1.8": "release-notes/rl-1.8.html",
"#ssec-relnotes-1.9": "release-notes/rl-1.9.html",
"#ssec-relnotes-2.0": "release-notes/rl-2.0.html",
"#ssec-relnotes-2.1": "release-notes/rl-2.1.html",
"#ssec-relnotes-2.2": "release-notes/rl-2.2.html",
"#ssec-relnotes-2.3": "release-notes/rl-2.3.html"
};
var isRoot = (document.location.pathname.endsWith('/') || document.location.pathname.endsWith('/index.html')) && path_to_root === '';
if (isRoot && redirects[document.location.hash]) {
document.location.href = path_to_root + redirects[document.location.hash];
}

View File

@@ -26,21 +26,14 @@
- [Copying Closures via SSH](package-management/copy-closure.md)
- [Serving a Nix store via SSH](package-management/ssh-substituter.md)
- [Serving a Nix store via S3](package-management/s3-substituter.md)
- [Writing Nix Expressions](expressions/writing-nix-expressions.md)
- [A Simple Nix Expression](expressions/simple-expression.md)
- [Expression Syntax](expressions/expression-syntax.md)
- [Build Script](expressions/build-script.md)
- [Arguments and Variables](expressions/arguments-variables.md)
- [Building and Testing](expressions/simple-building-testing.md)
- [Generic Builder Syntax](expressions/generic-builder.md)
- [Writing Nix Expressions](expressions/expression-language.md)
- [Values](expressions/language-values.md)
- [Language Constructs](expressions/language-constructs.md)
- [Operators](expressions/language-operators.md)
- [Derivations](expressions/derivations.md)
- [Advanced Attributes](expressions/advanced-attributes.md)
- [Built-in Constants](expressions/builtin-constants.md)
- [Built-in Functions](expressions/builtins.md)
- [Nix Language](language/index.md)
- [Data Types](language/values.md)
- [Language Constructs](language/constructs.md)
- [Operators](language/operators.md)
- [Derivations](language/derivations.md)
- [Advanced Attributes](language/advanced-attributes.md)
- [Built-in Constants](language/builtin-constants.md)
- [Built-in Functions](language/builtins.md)
- [Advanced Topics](advanced-topics/advanced-topics.md)
- [Remote Builds](advanced-topics/distributed-builds.md)
- [Tuning Cores and Jobs](advanced-topics/cores-vs-jobs.md)
@@ -72,6 +65,11 @@
- [CLI guideline](contributing/cli-guideline.md)
- [Release Notes](release-notes/release-notes.md)
- [Release X.Y (202?-??-??)](release-notes/rl-next.md)
- [Release 2.11 (2022-08-25)](release-notes/rl-2.11.md)
- [Release 2.10 (2022-07-11)](release-notes/rl-2.10.md)
- [Release 2.9 (2022-05-30)](release-notes/rl-2.9.md)
- [Release 2.8 (2022-04-19)](release-notes/rl-2.8.md)
- [Release 2.7 (2022-03-07)](release-notes/rl-2.7.md)
- [Release 2.6 (2022-01-24)](release-notes/rl-2.6.md)
- [Release 2.5 (2021-12-13)](release-notes/rl-2.5.md)
- [Release 2.4 (2021-11-01)](release-notes/rl-2.4.md)

View File

@@ -101,7 +101,7 @@ In particular, notice the
`/nix/store/krpqk0l9ib0ibi1d2w52z293zw455cap-unstable.check` output. Nix
has copied the build results to that directory where you can examine it.
> **Note**
> []{#check-dirs-are-unregistered} **Note**
>
> Check paths are not protected against garbage collection, and this
> path will be deleted on the next garbage collection.

View File

@@ -12,14 +12,14 @@ machine is accessible via SSH and that it has Nix installed. You can
test whether connecting to the remote Nix instance works, e.g.
```console
$ nix ping-store --store ssh://mac
$ nix store ping --store ssh://mac
```
will try to connect to the machine named `mac`. It is possible to
specify an SSH identity file as part of the remote store URI, e.g.
```console
$ nix ping-store --store ssh://mac?ssh-key=/home/alice/my-key
$ nix store ping --store ssh://mac?ssh-key=/home/alice/my-key
```
Since builds should be non-interactive, the key should not have a
@@ -110,7 +110,7 @@ default, set it to `-`.
7. A comma-separated list of *mandatory features*. A machine will only
be used to build a derivation if all of the machines mandatory
features appear in the derivations `requiredSystemFeatures`
attribute..
attribute.
8. The (base64-encoded) public host key of the remote machine. If omitted, SSH
will use its regular known-hosts file. Specifically, the field is calculated

View File

@@ -2,11 +2,11 @@
Most Nix commands interpret the following environment variables:
- `IN_NIX_SHELL`\
- [`IN_NIX_SHELL`]{#env-IN_NIX_SHELL}\
Indicator that tells if the current environment was set up by
`nix-shell`. Since Nix 2.0 the values are `"pure"` and `"impure"`
`nix-shell`. It can have the values `pure` or `impure`.
- `NIX_PATH`\
- [`NIX_PATH`]{#env-NIX_PATH}\
A colon-separated list of directories used to look up Nix
expressions enclosed in angle brackets (i.e., `<path>`). For
instance, the value
@@ -44,7 +44,7 @@ Most Nix commands interpret the following environment variables:
The Nix search path can also be extended using the `-I` option to
many Nix commands, which takes precedence over `NIX_PATH`.
- `NIX_IGNORE_SYMLINK_STORE`\
- [`NIX_IGNORE_SYMLINK_STORE`]{#env-NIX_IGNORE_SYMLINK_STORE}\
Normally, the Nix store directory (typically `/nix/store`) is not
allowed to contain any symlink components. This is to prevent
“impure” builds. Builders sometimes “canonicalise” paths by
@@ -66,41 +66,41 @@ Most Nix commands interpret the following environment variables:
Consult the mount 8 manual page for details.
- `NIX_STORE_DIR`\
- [`NIX_STORE_DIR`]{#env-NIX_STORE_DIR}\
Overrides the location of the Nix store (default `prefix/store`).
- `NIX_DATA_DIR`\
- [`NIX_DATA_DIR`]{#env-NIX_DATA_DIR}\
Overrides the location of the Nix static data directory (default
`prefix/share`).
- `NIX_LOG_DIR`\
- [`NIX_LOG_DIR`]{#env-NIX_LOG_DIR}\
Overrides the location of the Nix log directory (default
`prefix/var/log/nix`).
- `NIX_STATE_DIR`\
- [`NIX_STATE_DIR`]{#env-NIX_STATE_DIR}\
Overrides the location of the Nix state directory (default
`prefix/var/nix`).
- `NIX_CONF_DIR`\
- [`NIX_CONF_DIR`]{#env-NIX_CONF_DIR}\
Overrides the location of the system Nix configuration directory
(default `prefix/etc/nix`).
- `NIX_CONFIG`\
- [`NIX_CONFIG`]{#env-NIX_CONFIG}\
Applies settings from Nix configuration from the environment.
The content is treated as if it was read from a Nix configuration file.
Settings are separated by the newline character.
- `NIX_USER_CONF_FILES`\
- [`NIX_USER_CONF_FILES`]{#env-NIX_USER_CONF_FILES}\
Overrides the location of the user Nix configuration files to load
from (defaults to the XDG spec locations). The variable is treated
as a list separated by the `:` token.
- `TMPDIR`\
- [`TMPDIR`]{#env-TMPDIR}\
Use the specified directory to store temporary files. In particular,
this includes temporary build directories; these can take up
substantial amounts of disk space. The default is `/tmp`.
- `NIX_REMOTE`\
- [`NIX_REMOTE`]{#env-NIX_REMOTE}\
This variable should be set to `daemon` if you want to use the Nix
daemon to execute Nix operations. This is necessary in [multi-user
Nix installations](../installation/multi-user.md). If the Nix
@@ -108,16 +108,16 @@ Most Nix commands interpret the following environment variables:
should be set to `unix://path/to/socket`. Otherwise, it should be
left unset.
- `NIX_SHOW_STATS`\
- [`NIX_SHOW_STATS`]{#env-NIX_SHOW_STATS}\
If set to `1`, Nix will print some evaluation statistics, such as
the number of values allocated.
- `NIX_COUNT_CALLS`\
- [`NIX_COUNT_CALLS`]{#env-NIX_COUNT_CALLS}\
If set to `1`, Nix will print how often functions were called during
Nix expression evaluation. This is useful for profiling your Nix
expressions.
- `GC_INITIAL_HEAP_SIZE`\
- [`GC_INITIAL_HEAP_SIZE`]{#env-GC_INITIAL_HEAP_SIZE}\
If Nix has been configured to use the Boehm garbage collector, this
variable sets the initial size of the heap in bytes. It defaults to
384 MiB. Setting it to a low value reduces memory consumption, but

View File

@@ -12,6 +12,12 @@
[`--dry-run`]
[{`--out-link` | `-o`} *outlink*]
# Disambiguation
This man page describes the command `nix-build`, which is distinct from `nix
build`. For documentation on the latter, run `nix build --help` or see `man
nix3-build`.
# Description
The `nix-build` command builds the derivations described by the Nix
@@ -47,16 +53,16 @@ All options not listed here are passed to `nix-store
--realise`, except for `--arg` and `--attr` / `-A` which are passed to
`nix-instantiate`.
- `--no-out-link`\
- [`--no-out-link`]{#opt-no-out-link}\
Do not create a symlink to the output path. Note that as a result
the output does not become a root of the garbage collector, and so
might be deleted by `nix-store
--gc`.
- `--dry-run`\
- [`--dry-run`]{#opt-dry-run}\
Show what store paths would be built or downloaded.
- `--out-link` / `-o` *outlink*\
- [`--out-link`]{#opt-out-link} / `-o` *outlink*\
Change the name of the symlink to the output path created from
`result` to *outlink*.

View File

@@ -30,8 +30,8 @@ Since `nix-copy-closure` calls `ssh`, you may be asked to type in the
appropriate password or passphrase. In fact, you may be asked _twice_
because `nix-copy-closure` currently connects twice to the remote
machine, first to get the set of paths missing on the target machine,
and second to send the dump of those paths. If this bothers you, use
`ssh-agent`.
and second to send the dump of those paths. When using public key
authentication, you can avoid typing the passphrase with `ssh-agent`.
# Options

View File

@@ -31,7 +31,7 @@ subcommand to be performed. These are documented below.
Several commands, such as `nix-env -q` and `nix-env -i`, take a list of
arguments that specify the packages on which to operate. These are
extended regular expressions that must match the entire name of the
package. (For details on regular expressions, see regex7.) The match is
package. (For details on regular expressions, see **regex**(7).) The match is
case-sensitive. The regular expression can optionally be followed by a
dash and a version number; if omitted, any version of the package will
match. Here are some examples:
@@ -198,7 +198,7 @@ a number of possible ways:
another.
- If `--from-expression` is given, *args* are Nix
[functions](../expressions/language-constructs.md#functions)
[functions](../language/constructs.md#functions)
that are called with the active Nix expression as their single
argument. The derivations returned by those function calls are
installed. This allows derivations to be specified in an
@@ -412,7 +412,7 @@ The upgrade operation determines whether a derivation `y` is an upgrade
of a derivation `x` by looking at their respective `name` attributes.
The names (e.g., `gcc-3.3.1` are split into two parts: the package name
(`gcc`), and the version (`3.3.1`). The version part starts after the
first dash not followed by a letter. `x` is considered an upgrade of `y`
first dash not followed by a letter. `y` is considered an upgrade of `x`
if their package names match, and the version of `y` is higher than that
of `x`.

View File

@@ -51,7 +51,7 @@ standard input.
- `--strict`\
When used with `--eval`, recursively evaluate list elements and
attributes. Normally, such sub-expressions are left unevaluated
(since the Nix expression language is lazy).
(since the Nix language is lazy).
> **Warning**
>
@@ -66,7 +66,7 @@ standard input.
When used with `--eval`, print the resulting value as an XML
representation of the abstract syntax tree rather than as an ATerm.
The schema is the same as that used by the [`toXML`
built-in](../expressions/builtins.md).
built-in](../language/builtins.md).
- `--read-write-mode`\
When used with `--eval`, perform evaluation in read/write mode so

View File

@@ -15,6 +15,12 @@
[`--keep` *name*]
{{`--packages` | `-p`} {*packages* | *expressions*} … | [*path*]}
# Disambiguation
This man page describes the command `nix-shell`, which is distinct from `nix
shell`. For documentation on the latter, run `nix shell --help` or see `man
nix3-shell`.
# Description
The command `nix-shell` will build the dependencies of the specified

View File

@@ -22,7 +22,7 @@ This section lists the options that are common to all operations. These
options are allowed for every subcommand, though they may not always
have an effect.
- `--add-root` *path*\
- [`--add-root`]{#opt-add-root} *path*\
Causes the result of a realisation (`--realise` and
`--force-realise`) to be registered as a root of the garbage
collector. *path* will be created as a symlink to the resulting
@@ -121,7 +121,7 @@ Special exit codes:
- `102`\
Hash mismatch, the build output was rejected because it does not
match the [`outputHash` attribute of the
derivation](../expressions/advanced-attributes.md).
derivation](../language/advanced-attributes.md).
- `104`\
Not deterministic, the build succeeded in check mode but the

View File

@@ -2,13 +2,13 @@
Most Nix commands accept the following command-line options:
- `--help`\
- [`--help`]{#opt-help}\
Prints out a summary of the command syntax and exits.
- `--version`\
- [`--version`]{#opt-version}\
Prints out the Nix version number on standard output and exits.
- `--verbose` / `-v`\
- [`--verbose`]{#opt-verbose} / `-v`\
Increases the level of verbosity of diagnostic messages printed on
standard error. For each Nix operation, the information printed on
standard output is well-defined; any diagnostic information is
@@ -37,14 +37,14 @@ Most Nix commands accept the following command-line options:
- 5\
“Vomit”: print vast amounts of debug information.
- `--quiet`\
- [`--quiet`]{#opt-quiet}\
Decreases the level of verbosity of diagnostic messages printed on
standard error. This is the inverse option to `-v` / `--verbose`.
This option may be specified repeatedly. See the previous verbosity
levels list.
- `--log-format` *format*\
- [`--log-format`]{#opt-log-format} *format*\
This option can be used to change the output of the log format, with
*format* being one of:
@@ -66,14 +66,14 @@ Most Nix commands accept the following command-line options:
- bar-with-logs\
Display the raw logs, with the progress bar at the bottom.
- `--no-build-output` / `-Q`\
- [`--no-build-output`]{#opt-no-build-output} / `-Q`\
By default, output written by builders to standard output and
standard error is echoed to the Nix command's standard error. This
option suppresses this behaviour. Note that the builder's standard
output and error are always written to a log file in
`prefix/nix/var/log/nix`.
- `--max-jobs` / `-j` *number*\
- [`--max-jobs`]{#opt-max-jobs} / `-j` *number*\
Sets the maximum number of build jobs that Nix will perform in
parallel to the specified number. Specify `auto` to use the number
of CPUs in the system. The default is specified by the `max-jobs`
@@ -83,7 +83,7 @@ Most Nix commands accept the following command-line options:
Setting it to `0` disallows building on the local machine, which is
useful when you want builds to happen only on remote builders.
- `--cores`\
- [`--cores`]{#opt-cores}\
Sets the value of the `NIX_BUILD_CORES` environment variable in
the invocation of builders. Builders can use this variable at
their discretion to control the maximum amount of parallelism. For
@@ -94,18 +94,18 @@ Most Nix commands accept the following command-line options:
means that the builder should use all available CPU cores in the
system.
- `--max-silent-time`\
- [`--max-silent-time`]{#opt-max-silent-time}\
Sets the maximum number of seconds that a builder can go without
producing any data on standard output or standard error. The
default is specified by the `max-silent-time` configuration
setting. `0` means no time-out.
- `--timeout`\
- [`--timeout`]{#opt-timeout}\
Sets the maximum number of seconds that a builder can run. The
default is specified by the `timeout` configuration setting. `0`
means no timeout.
- `--keep-going` / `-k`\
- [`--keep-going`]{#opt-keep-going} / `-k`\
Keep going in case of failed builds, to the greatest extent
possible. That is, if building an input of some derivation fails,
Nix will still build the other inputs, but not the derivation
@@ -113,13 +113,13 @@ Most Nix commands accept the following command-line options:
for builds of substitutes), possibly killing builds in progress (in
case of parallel or distributed builds).
- `--keep-failed` / `-K`\
- [`--keep-failed`]{#opt-keep-failed} / `-K`\
Specifies that in case of a build failure, the temporary directory
(usually in `/tmp`) in which the build takes place should not be
deleted. The path of the build directory is printed as an
informational message.
- `--fallback`\
- [`--fallback`]{#opt-fallback}\
Whenever Nix attempts to build a derivation for which substitutes
are known for each output path, but realising the output paths
through the substitutes fails, fall back on building the derivation.
@@ -134,18 +134,18 @@ Most Nix commands accept the following command-line options:
failure in obtaining the substitutes to lead to a full build from
source (with the related consumption of resources).
- `--readonly-mode`\
- [`--readonly-mode`]{#opt-readonly-mode}\
When this option is used, no attempt is made to open the Nix
database. Most Nix operations do need database access, so those
operations will fail.
- `--arg` *name* *value*\
- [`--arg`]{#opt-arg} *name* *value*\
This option is accepted by `nix-env`, `nix-instantiate`,
`nix-shell` and `nix-build`. When evaluating Nix expressions, the
expression evaluator will automatically try to call functions that
it encounters. It can automatically call functions for which every
argument has a [default
value](../expressions/language-constructs.md#functions) (e.g.,
value](../language/constructs.md#functions) (e.g.,
`{ argName ? defaultValue }: ...`). With `--arg`, you can also
call functions that have arguments without a default value (or
override a default value). That is, if the evaluator encounters a
@@ -164,19 +164,19 @@ Most Nix commands accept the following command-line options:
So if you call this Nix expression (e.g., when you do `nix-env -iA
pkgname`), the function will be called automatically using the
value [`builtins.currentSystem`](../expressions/builtins.md) for
value [`builtins.currentSystem`](../language/builtins.md) for
the `system` argument. You can override this using `--arg`, e.g.,
`nix-env -iA pkgname --arg system \"i686-freebsd\"`. (Note that
since the argument is a Nix string literal, you have to escape the
quotes.)
- `--argstr` *name* *value*\
- [`--argstr`]{#opt-argstr} *name* *value*\
This option is like `--arg`, only the value is not a Nix
expression but a string. So instead of `--arg system
\"i686-linux\"` (the outer quotes are to keep the shell happy) you
can say `--argstr system i686-linux`.
- `--attr` / `-A` *attrPath*\
- [`--attr`]{#opt-attr} / `-A` *attrPath*\
Select an attribute from the top-level Nix expression being
evaluated. (`nix-env`, `nix-instantiate`, `nix-build` and
`nix-shell` only.) The *attribute path* *attrPath* is a sequence
@@ -191,7 +191,7 @@ Most Nix commands accept the following command-line options:
attribute of the fourth element of the array in the `foo` attribute
of the top-level expression.
- `--expr` / `-E`\
- [`--expr`]{#opt-expr} / `-E`\
Interpret the command line arguments as a list of Nix expressions to
be parsed and evaluated, rather than as a list of file names of Nix
expressions. (`nix-instantiate`, `nix-build` and `nix-shell` only.)
@@ -202,17 +202,17 @@ Most Nix commands accept the following command-line options:
use, give your expression to the `nix-shell -p` convenience flag
instead.
- `-I` *path*\
- [`-I`]{#opt-I} *path*\
Add a path to the Nix expression search path. This option may be
given multiple times. See the `NIX_PATH` environment variable for
information on the semantics of the Nix search path. Paths added
through `-I` take precedence over `NIX_PATH`.
- `--option` *name* *value*\
- [`--option`]{#opt-option} *name* *value*\
Set the Nix configuration option *name* to *value*. This overrides
settings in the Nix configuration file (see nix.conf5).
- `--repair`\
- [`--repair`]{#opt-repair}\
Fix corrupted or missing store paths by redownloading or rebuilding
them. Note that this is slow because it requires computing a
cryptographic hash of the contents of every path in the closure of

View File

@@ -42,7 +42,7 @@ $ nix develop
```
To get a shell with a different compilation environment (e.g. stdenv,
gccStdenv, clangStdenv, clang11Stdenv):
gccStdenv, clangStdenv, clang11Stdenv, ccacheStdenv):
```console
$ nix-shell -A devShells.x86_64-linux.clang11StdenvPackages
@@ -54,6 +54,9 @@ or if you have a flake-enabled nix:
$ nix develop .#clang11StdenvPackages
```
Note: you can use `ccacheStdenv` to drastically improve rebuild
time. By default, ccache keeps artifacts in `~/.cache/ccache/`.
To build Nix itself in this shell:
```console
@@ -71,18 +74,6 @@ To install it in `$(pwd)/outputs` and test it:
nix (Nix) 3.0
```
To run a functional test:
```console
make tests/test-name-should-auto-complete.sh.test
```
To run the unit-tests for C++ code:
```
make check
```
If you have a flakes-enabled Nix you can replace:
```console
@@ -94,3 +85,96 @@ by:
```console
$ nix develop
```
## Running tests
### Unit-tests
The unit-tests for each Nix library (`libexpr`, `libstore`, etc..) are defined
under `src/{library_name}/tests` using the
[googletest](https://google.github.io/googletest/) framework.
You can run the whole testsuite with `make check`, or the tests for a specific component with `make libfoo-tests_RUN`. Finer-grained filtering is also possible using the [--gtest_filter](https://google.github.io/googletest/advanced.html#running-a-subset-of-the-tests) command-line option.
### Functional tests
The functional tests reside under the `tests` directory and are listed in `tests/local.mk`.
The whole testsuite can be run with `make install && make installcheck`.
Individual tests can be run with `make tests/{testName}.sh.test`.
### Integration tests
The integration tests are defined in the Nix flake under the `hydraJobs.tests` attribute.
These tests include everything that needs to interact with external services or run Nix in a non-trivial distributed setup.
Because these tests are expensive and require more than what the standard github-actions setup provides, they only run on the master branch (on <https://hydra.nixos.org/jobset/nix/master>).
You can run them manually with `nix build .#hydraJobs.tests.{testName}` or `nix-build -A hydraJobs.tests.{testName}`
### Installer tests
After a one-time setup, the Nix repository's GitHub Actions continuous integration (CI) workflow can test the installer each time you push to a branch.
Creating a Cachix cache for your installer tests and adding its authorization token to GitHub enables [two installer-specific jobs in the CI workflow](https://github.com/NixOS/nix/blob/88a45d6149c0e304f6eb2efcc2d7a4d0d569f8af/.github/workflows/ci.yml#L50-L91):
- The `installer` job generates installers for the platforms below and uploads them to your Cachix cache:
- `x86_64-linux`
- `armv6l-linux`
- `armv7l-linux`
- `x86_64-darwin`
- The `installer_test` job (which runs on `ubuntu-latest` and `macos-latest`) will try to install Nix with the cached installer and run a trivial Nix command.
#### One-time setup
1. Have a GitHub account with a fork of the [Nix repository](https://github.com/NixOS/nix).
2. At cachix.org:
- Create or log in to an account.
- Create a Cachix cache using the format `<github-username>-nix-install-tests`.
- Navigate to the new cache > Settings > Auth Tokens.
- Generate a new Cachix auth token and copy the generated value.
3. At github.com:
- Navigate to your Nix fork > Settings > Secrets > Actions > New repository secret.
- Name the secret `CACHIX_AUTH_TOKEN`.
- Paste the copied value of the Cachix cache auth token.
#### Using the CI-generated installer for manual testing
After the CI run completes, you can check the output to extract the installer URL:
1. Click into the detailed view of the CI run.
2. Click into any `installer_test` run (the URL you're here to extract will be the same in all of them).
3. Click into the `Run cachix/install-nix-action@v...` step and click the detail triangle next to the first log line (it will also be `Run cachix/install-nix-action@v...`)
4. Copy the value of `install_url`
5. To generate an install command, plug this `install_url` and your GitHub username into this template:
```console
sh <(curl -L <install_url>) --tarball-url-prefix https://<github-username>-nix-install-tests.cachix.org/serve
```
<!-- #### Manually generating test installers
There's obviously a manual way to do this, and it's still the only way for
platforms that lack GA runners.
I did do this back in Fall 2020 (before the GA approach encouraged here). I'll
sketch what I recall in case it encourages someone to fill in detail, but: I
didn't know what I was doing at the time and had to fumble/ask around a lot--
so I don't want to uphold any of it as "right". It may have been dumb or
the _hard_ way from the getgo. Fundamentals may have changed since.
Here's the build command I used to do this on and for x86_64-darwin:
nix build --out-link /tmp/foo ".#checks.x86_64-darwin.binaryTarball"
I used the stable out-link to make it easier to script the next steps:
link=$(readlink /tmp/foo)
cp $link/*-darwin.tar.xz ~/somewheres
I've lost the last steps and am just going from memory:
From here, I think I had to extract and modify the `install` script to point
it at this tarball (which I scped to my own site, but it might make more sense
to just share them locally). I extracted this script once and then just
search/replaced in it for each new build.
The installer now supports a `--tarball-url-prefix` flag which _may_ have
solved this need?
-->

View File

@@ -1,80 +0,0 @@
# Arguments and Variables
The [Nix expression for GNU Hello](expression-syntax.md) is a
function; it is missing some arguments that have to be filled in
somewhere. In the Nix Packages collection this is done in the file
`pkgs/top-level/all-packages.nix`, where all Nix expressions for
packages are imported and called with the appropriate arguments. Here
are some fragments of `all-packages.nix`, with annotations of what
they mean:
```nix
...
rec {
hello = import ../applications/misc/hello/ex-1 {
inherit fetchurl stdenv perl;
};
perl = import ../development/interpreters/perl {
inherit fetchurl stdenv;
};
fetchurl = import ../build-support/fetchurl {
inherit stdenv; ...
};
stdenv = ...;
}
```
1. This file defines a set of attributes, all of which are concrete
derivations (i.e., not functions). In fact, we define a *mutually
recursive* set of attributes. That is, the attributes can refer to
each other. This is precisely what we want since we want to “plug”
the various packages into each other.
2. Here we *import* the Nix expression for GNU Hello. The import
operation just loads and returns the specified Nix expression. In
fact, we could just have put the contents of the Nix expression
for GNU Hello in `all-packages.nix` at this point. That would be
completely equivalent, but it would make `all-packages.nix` rather
bulky.
Note that we refer to `../applications/misc/hello/ex-1`, not
`../applications/misc/hello/ex-1/default.nix`. When you try to
import a directory, Nix automatically appends `/default.nix` to the
file name.
3. This is where the actual composition takes place. Here we *call* the
function imported from `../applications/misc/hello/ex-1` with a set
containing the things that the function expects, namely `fetchurl`,
`stdenv`, and `perl`. We use inherit again to use the attributes
defined in the surrounding scope (we could also have written
`fetchurl = fetchurl;`, etc.).
The result of this function call is an actual derivation that can be
built by Nix (since when we fill in the arguments of the function,
what we get is its body, which is the call to `stdenv.mkDerivation`
in the [Nix expression for GNU Hello](expression-syntax.md)).
> **Note**
>
> Nixpkgs has a convenience function `callPackage` that imports and
> calls a function, filling in any missing arguments by passing the
> corresponding attribute from the Nixpkgs set, like this:
>
> ```nix
> hello = callPackage ../applications/misc/hello/ex-1 { };
> ```
>
> If necessary, you can set or override arguments:
>
> ```nix
> hello = callPackage ../applications/misc/hello/ex-1 { stdenv = myStdenv; };
> ```
4. Likewise, we have to instantiate Perl, `fetchurl`, and the standard
environment.

View File

@@ -1,70 +0,0 @@
# Build Script
Here is the builder referenced from Hello's Nix expression (stored in
`pkgs/applications/misc/hello/ex-1/builder.sh`):
```bash
source $stdenv/setup ①
PATH=$perl/bin:$PATH
tar xvfz $src
cd hello-*
./configure --prefix=$out
make ⑤
make install
```
The builder can actually be made a lot shorter by using the *generic
builder* functions provided by `stdenv`, but here we write out the build
steps to elucidate what a builder does. It performs the following steps:
1. When Nix runs a builder, it initially completely clears the
environment (except for the attributes declared in the derivation).
This is done to prevent undeclared inputs from being used in the
build process. If for example the `PATH` contained `/usr/bin`, then
you might accidentally use `/usr/bin/gcc`.
So the first step is to set up the environment. This is done by
calling the `setup` script of the standard environment. The
environment variable `stdenv` points to the location of the
standard environment being used. (It wasn't specified explicitly
as an attribute in Hello's Nix expression, but `mkDerivation` adds
it automatically.)
2. Since Hello needs Perl, we have to make sure that Perl is in the
`PATH`. The `perl` environment variable points to the location of
the Perl package (since it was passed in as an attribute to the
derivation), so `$perl/bin` is the directory containing the Perl
interpreter.
3. Now we have to unpack the sources. The `src` attribute was bound to
the result of fetching the Hello source tarball from the network, so
the `src` environment variable points to the location in the Nix
store to which the tarball was downloaded. After unpacking, we `cd`
to the resulting source directory.
The whole build is performed in a temporary directory created in
`/tmp`, by the way. This directory is removed after the builder
finishes, so there is no need to clean up the sources afterwards.
Also, the temporary directory is always newly created, so you don't
have to worry about files from previous builds interfering with the
current build.
4. GNU Hello is a typical Autoconf-based package, so we first have to
run its `configure` script. In Nix every package is stored in a
separate location in the Nix store, for instance
`/nix/store/9a54ba97fb71b65fda531012d0443ce2-hello-2.1.1`. Nix
computes this path by cryptographically hashing all attributes of
the derivation. The path is passed to the builder through the `out`
environment variable. So here we give `configure` the parameter
`--prefix=$out` to cause Hello to be installed in the expected
location.
5. Finally we build Hello (`make`) and install it into the location
specified by `out` (`make install`).
If you are wondering about the absence of error checking on the result
of various commands called in the builder: this is because the shell
script is evaluated with Bash's `-e` option, which causes the script to
be aborted if any command fails without an error check.

View File

@@ -1,12 +0,0 @@
# Nix Expression Language
The Nix expression language is a pure, lazy, functional language. Purity
means that operations in the language don't have side-effects (for
instance, there is no variable assignment). Laziness means that
arguments to functions are evaluated only when they are needed.
Functional means that functions are “normal” values that can be passed
around and manipulated in interesting ways. The language is not a
full-featured, general purpose language. Its main job is to describe
packages, compositions of packages, and the variability within packages.
This section presents the various features of the language.

View File

@@ -1,93 +0,0 @@
# Expression Syntax
Here is a Nix expression for GNU Hello:
```nix
{ stdenv, fetchurl, perl }:
stdenv.mkDerivation {
name = "hello-2.1.1";
builder = ./builder.sh;
src = fetchurl {
url = "ftp://ftp.nluug.nl/pub/gnu/hello/hello-2.1.1.tar.gz";
sha256 = "1md7jsfd8pa45z73bz1kszpp01yw6x5ljkjk2hx7wl800any6465";
};
inherit perl;
}
```
This file is actually already in the Nix Packages collection in
`pkgs/applications/misc/hello/ex-1/default.nix`. It is customary to
place each package in a separate directory and call the single Nix
expression in that directory `default.nix`. The file has the following
elements (referenced from the figure by number):
1. This states that the expression is a *function* that expects to be
called with three arguments: `stdenv`, `fetchurl`, and `perl`. They
are needed to build Hello, but we don't know how to build them here;
that's why they are function arguments. `stdenv` is a package that
is used by almost all Nix Packages; it provides a
“standard” environment consisting of the things you would expect
in a basic Unix environment: a C/C++ compiler (GCC, to be precise),
the Bash shell, fundamental Unix tools such as `cp`, `grep`, `tar`,
etc. `fetchurl` is a function that downloads files. `perl` is the
Perl interpreter.
Nix functions generally have the form `{ x, y, ..., z }: e` where
`x`, `y`, etc. are the names of the expected arguments, and where
*e* is the body of the function. So here, the entire remainder of
the file is the body of the function; when given the required
arguments, the body should describe how to build an instance of
the Hello package.
2. So we have to build a package. Building something from other stuff
is called a *derivation* in Nix (as opposed to sources, which are
built by humans instead of computers). We perform a derivation by
calling `stdenv.mkDerivation`. `mkDerivation` is a function
provided by `stdenv` that builds a package from a set of
*attributes*. A set is just a list of key/value pairs where each
key is a string and each value is an arbitrary Nix
expression. They take the general form `{ name1 = expr1; ...
nameN = exprN; }`.
3. The attribute `name` specifies the symbolic name and version of
the package. Nix doesn't really care about these things, but they
are used by for instance `nix-env -q` to show a “human-readable”
name for packages. This attribute is required by `mkDerivation`.
4. The attribute `builder` specifies the builder. This attribute can
sometimes be omitted, in which case `mkDerivation` will fill in a
default builder (which does a `configure; make; make install`, in
essence). Hello is sufficiently simple that the default builder
would suffice, but in this case, we will show an actual builder
for educational purposes. The value `./builder.sh` refers to the
shell script shown in the [next section](build-script.md),
discussed below.
5. The builder has to know what the sources of the package are. Here,
the attribute `src` is bound to the result of a call to the
`fetchurl` function. Given a URL and a SHA-256 hash of the expected
contents of the file at that URL, this function builds a derivation
that downloads the file and checks its hash. So the sources are a
dependency that like all other dependencies is built before Hello
itself is built.
Instead of `src` any other name could have been used, and in fact
there can be any number of sources (bound to different attributes).
However, `src` is customary, and it's also expected by the default
builder (which we don't use in this example).
6. Since the derivation requires Perl, we have to pass the value of the
`perl` function argument to the builder. All attributes in the set
are actually passed as environment variables to the builder, so
declaring an attribute
```nix
perl = perl;
```
will do the trick: it binds an attribute `perl` to the function
argument which also happens to be called `perl`. However, it looks a
bit silly, so there is a shorter syntax. The `inherit` keyword
causes the specified attributes to be bound to whatever variables
with the same name happen to be in scope.

View File

@@ -1,66 +0,0 @@
# Generic Builder Syntax
Recall that the [build script for GNU Hello](build-script.md) looked
something like this:
```bash
PATH=$perl/bin:$PATH
tar xvfz $src
cd hello-*
./configure --prefix=$out
make
make install
```
The builders for almost all Unix packages look like this — set up some
environment variables, unpack the sources, configure, build, and
install. For this reason the standard environment provides some Bash
functions that automate the build process. Here is what a builder using
the generic build facilities looks like:
```bash
buildInputs="$perl"
source $stdenv/setup ②
genericBuild ③
```
Here is what each line means:
1. The `buildInputs` variable tells `setup` to use the indicated
packages as “inputs”. This means that if a package provides a `bin`
subdirectory, it's added to `PATH`; if it has a `include`
subdirectory, it's added to GCC's header search path; and so on.
(This is implemented in a modular way: `setup` tries to source the
file `pkg/nix-support/setup-hook` of all dependencies. These “setup
hooks” can then set up whatever environment variables they want; for
instance, the setup hook for Perl sets the `PERL5LIB` environment
variable to contain the `lib/site_perl` directories of all inputs.)
2. The function `genericBuild` is defined in the file `$stdenv/setup`.
3. The final step calls the shell function `genericBuild`, which
performs the steps that were done explicitly in the previous build
script. The generic builder is smart enough to figure out whether
to unpack the sources using `gzip`, `bzip2`, etc. It can be
customised in many ways; see the Nixpkgs manual for details.
Discerning readers will note that the `buildInputs` could just as well
have been set in the Nix expression, like this:
```nix
buildInputs = [ perl ];
```
The `perl` attribute can then be removed, and the builder becomes even
shorter:
```bash
source $stdenv/setup
genericBuild
```
In fact, `mkDerivation` provides a default builder that looks exactly
like that, so it is actually possible to omit the builder for Hello
entirely.

View File

@@ -1,251 +0,0 @@
# Values
## Simple Values
Nix has the following basic data types:
- *Strings* can be written in three ways.
The most common way is to enclose the string between double quotes,
e.g., `"foo bar"`. Strings can span multiple lines. The special
characters `"` and `\` and the character sequence `${` must be
escaped by prefixing them with a backslash (`\`). Newlines, carriage
returns and tabs can be written as `\n`, `\r` and `\t`,
respectively.
You can include the result of an expression into a string by
enclosing it in `${...}`, a feature known as *antiquotation*. The
enclosed expression must evaluate to something that can be coerced
into a string (meaning that it must be a string, a path, or a
derivation). For instance, rather than writing
```nix
"--with-freetype2-library=" + freetype + "/lib"
```
(where `freetype` is a derivation), you can instead write the more
natural
```nix
"--with-freetype2-library=${freetype}/lib"
```
The latter is automatically translated to the former. A more
complicated example (from the Nix expression for
[Qt](http://www.trolltech.com/products/qt)):
```nix
configureFlags = "
-system-zlib -system-libpng -system-libjpeg
${if openglSupport then "-dlopen-opengl
-L${mesa}/lib -I${mesa}/include
-L${libXmu}/lib -I${libXmu}/include" else ""}
${if threadSupport then "-thread" else "-no-thread"}
";
```
Note that Nix expressions and strings can be arbitrarily nested; in
this case the outer string contains various antiquotations that
themselves contain strings (e.g., `"-thread"`), some of which in
turn contain expressions (e.g., `${mesa}`).
The second way to write string literals is as an *indented string*,
which is enclosed between pairs of *double single-quotes*, like so:
```nix
''
This is the first line.
This is the second line.
This is the third line.
''
```
This kind of string literal intelligently strips indentation from
the start of each line. To be precise, it strips from each line a
number of spaces equal to the minimal indentation of the string as a
whole (disregarding the indentation of empty lines). For instance,
the first and second line are indented two spaces, while the third
line is indented four spaces. Thus, two spaces are stripped from
each line, so the resulting string is
```nix
"This is the first line.\nThis is the second line.\n This is the third line.\n"
```
Note that the whitespace and newline following the opening `''` is
ignored if there is no non-whitespace text on the initial line.
Antiquotation (`${expr}`) is supported in indented strings.
Since `${` and `''` have special meaning in indented strings, you
need a way to quote them. `$` can be escaped by prefixing it with
`''` (that is, two single quotes), i.e., `''$`. `''` can be escaped
by prefixing it with `'`, i.e., `'''`. `$` removes any special
meaning from the following `$`. Linefeed, carriage-return and tab
characters can be written as `''\n`, `''\r`, `''\t`, and `''\`
escapes any other character.
Indented strings are primarily useful in that they allow multi-line
string literals to follow the indentation of the enclosing Nix
expression, and that less escaping is typically necessary for
strings representing languages such as shell scripts and
configuration files because `''` is much less common than `"`.
Example:
```nix
stdenv.mkDerivation {
...
postInstall =
''
mkdir $out/bin $out/etc
cp foo $out/bin
echo "Hello World" > $out/etc/foo.conf
${if enableBar then "cp bar $out/bin" else ""}
'';
...
}
```
Finally, as a convenience, *URIs* as defined in appendix B of
[RFC 2396](http://www.ietf.org/rfc/rfc2396.txt) can be written *as
is*, without quotes. For instance, the string
`"http://example.org/foo.tar.bz2"` can also be written as
`http://example.org/foo.tar.bz2`.
- Numbers, which can be *integers* (like `123`) or *floating point*
(like `123.43` or `.27e13`).
Numbers are type-compatible: pure integer operations will always
return integers, whereas any operation involving at least one
floating point number will have a floating point number as a result.
- *Paths*, e.g., `/bin/sh` or `./builder.sh`. A path must contain at
least one slash to be recognised as such. For instance, `builder.sh`
is not a path: it's parsed as an expression that selects the
attribute `sh` from the variable `builder`. If the file name is
relative, i.e., if it does not begin with a slash, it is made
absolute at parse time relative to the directory of the Nix
expression that contained it. For instance, if a Nix expression in
`/foo/bar/bla.nix` refers to `../xyzzy/fnord.nix`, the absolute path
is `/foo/xyzzy/fnord.nix`.
If the first component of a path is a `~`, it is interpreted as if
the rest of the path were relative to the user's home directory.
e.g. `~/foo` would be equivalent to `/home/edolstra/foo` for a user
whose home directory is `/home/edolstra`.
Paths can also be specified between angle brackets, e.g.
`<nixpkgs>`. This means that the directories listed in the
environment variable `NIX_PATH` will be searched for the given file
or directory name.
Antiquotation is supported in any paths except those in angle brackets.
`./${foo}-${bar}.nix` is a more convenient way of writing
`./. + "/" + foo + "-" + bar + ".nix"` or `./. + "/${foo}-${bar}.nix"`. At
least one slash must appear *before* any antiquotations for this to be
recognized as a path. `a.${foo}/b.${bar}` is a syntactically valid division
operation. `./a.${foo}/b.${bar}` is a path.
- *Booleans* with values `true` and `false`.
- The null value, denoted as `null`.
## Lists
Lists are formed by enclosing a whitespace-separated list of values
between square brackets. For example,
```nix
[ 123 ./foo.nix "abc" (f { x = y; }) ]
```
defines a list of four elements, the last being the result of a call to
the function `f`. Note that function calls have to be enclosed in
parentheses. If they had been omitted, e.g.,
```nix
[ 123 ./foo.nix "abc" f { x = y; } ]
```
the result would be a list of five elements, the fourth one being a
function and the fifth being a set.
Note that lists are only lazy in values, and they are strict in length.
## Sets
Sets are really the core of the language, since ultimately the Nix
language is all about creating derivations, which are really just sets
of attributes to be passed to build scripts.
Sets are just a list of name/value pairs (called *attributes*) enclosed
in curly brackets, where each value is an arbitrary expression
terminated by a semicolon. For example:
```nix
{ x = 123;
text = "Hello";
y = f { bla = 456; };
}
```
This defines a set with attributes named `x`, `text`, `y`. The order of
the attributes is irrelevant. An attribute name may only occur once.
Attributes can be selected from a set using the `.` operator. For
instance,
```nix
{ a = "Foo"; b = "Bar"; }.a
```
evaluates to `"Foo"`. It is possible to provide a default value in an
attribute selection using the `or` keyword. For example,
```nix
{ a = "Foo"; b = "Bar"; }.c or "Xyzzy"
```
will evaluate to `"Xyzzy"` because there is no `c` attribute in the set.
You can use arbitrary double-quoted strings as attribute names:
```nix
{ "foo ${bar}" = 123; "nix-1.0" = 456; }."foo ${bar}"
```
This will evaluate to `123` (Assuming `bar` is antiquotable). In the
case where an attribute name is just a single antiquotation, the quotes
can be dropped:
```nix
{ foo = 123; }.${bar} or 456
```
This will evaluate to `123` if `bar` evaluates to `"foo"` when coerced
to a string and `456` otherwise (again assuming `bar` is antiquotable).
In the special case where an attribute name inside of a set declaration
evaluates to `null` (which is normally an error, as `null` is not
antiquotable), that attribute is simply not added to the set:
```nix
{ ${if foo then "bar" else null} = true; }
```
This will evaluate to `{}` if `foo` evaluates to `false`.
A set that has a `__functor` attribute whose value is callable (i.e. is
itself a function or a set with a `__functor` attribute whose value is
callable) can be applied as if it were a function, with the set itself
passed in first , e.g.,
```nix
let add = { __functor = self: x: x + self.x; };
inc = add // { x = 1; };
in inc 1
```
evaluates to `2`. This can be used to attach metadata to a function
without the caller needing to treat it specially, or to implement a form
of object-oriented programming, for example.

View File

@@ -1,61 +0,0 @@
# Building and Testing
You can now try to build Hello. Of course, you could do `nix-env -f . -iA
hello`, but you may not want to install a possibly broken package just
yet. The best way to test the package is by using the command
`nix-build`, which builds a Nix expression and creates a symlink named
`result` in the current directory:
```console
$ nix-build -A hello
building path `/nix/store/632d2b22514d...-hello-2.1.1'
hello-2.1.1/
hello-2.1.1/intl/
hello-2.1.1/intl/ChangeLog
...
$ ls -l result
lrwxrwxrwx ... 2006-09-29 10:43 result -> /nix/store/632d2b22514d...-hello-2.1.1
$ ./result/bin/hello
Hello, world!
```
The `-A` option selects the `hello` attribute. This is faster than
using the symbolic package name specified by the `name` attribute
(which also happens to be `hello`) and is unambiguous (there can be
multiple packages with the symbolic name `hello`, but there can be
only one attribute in a set named `hello`).
`nix-build` registers the `./result` symlink as a garbage collection
root, so unless and until you delete the `./result` symlink, the output
of the build will be safely kept on your system. You can use
`nix-build`s `-o` switch to give the symlink another name.
Nix has transactional semantics. Once a build finishes successfully, Nix
makes a note of this in its database: it registers that the path denoted
by `out` is now “valid”. If you try to build the derivation again, Nix
will see that the path is already valid and finish immediately. If a
build fails, either because it returns a non-zero exit code, because Nix
or the builder are killed, or because the machine crashes, then the
output paths will not be registered as valid. If you try to build the
derivation again, Nix will remove the output paths if they exist (e.g.,
because the builder died half-way through `make
install`) and try again. Note that there is no “negative caching”: Nix
doesn't remember that a build failed, and so a failed build can always
be repeated. This is because Nix cannot distinguish between permanent
failures (e.g., a compiler error due to a syntax error in the source)
and transient failures (e.g., a disk full condition).
Nix also performs locking. If you run multiple Nix builds
simultaneously, and they try to build the same derivation, the first Nix
instance that gets there will perform the build, while the others block
(or perform other derivations if available) until the build finishes:
```console
$ nix-build -A hello
waiting for lock on `/nix/store/0h5b7hp8d4hqfrw8igvx97x1xawrjnac-hello-2.1.1x'
```
So it is always safe to run multiple instances of Nix in parallel (which
isnt the case with, say, `make`).

View File

@@ -1,23 +0,0 @@
# A Simple Nix Expression
This section shows how to add and test the [GNU Hello
package](http://www.gnu.org/software/hello/hello.html) to the Nix
Packages collection. Hello is a program that prints out the text “Hello,
world\!”.
To add a package to the Nix Packages collection, you generally need to
do three things:
1. Write a Nix expression for the package. This is a file that
describes all the inputs involved in building the package, such as
dependencies, sources, and so on.
2. Write a *builder*. This is a shell script that builds the package
from the inputs. (In fact, it can be written in any language, but
typically it's a `bash` shell script.)
3. Add the package to the file `pkgs/top-level/all-packages.nix`. The
Nix expression written in the first step is a *function*; it
requires other packages in order to build it. In this step you put
it all together, i.e., you call the function with the right
arguments to build the actual package.

View File

@@ -1,12 +0,0 @@
This chapter shows you how to write Nix expressions, which instruct Nix
how to build packages. It starts with a simple example (a Nix expression
for GNU Hello), and then moves on to a more in-depth look at the Nix
expression language.
> **Note**
>
> This chapter is mostly about the Nix expression language. For more
> extensive information on adding packages to the Nix Packages
> collection (such as functions in the standard environment and coding
> conventions), please consult [its
> manual](http://nixos.org/nixpkgs/manual/).

View File

@@ -1,48 +1,100 @@
# Glossary
- derivation\
- [derivation]{#gloss-derivation}\
A description of a build action. The result of a derivation is a
store object. Derivations are typically specified in Nix expressions
using the [`derivation` primitive](expressions/derivations.md). These are
using the [`derivation` primitive](language/derivations.md). These are
translated into low-level *store derivations* (implicitly by
`nix-env` and `nix-build`, or explicitly by `nix-instantiate`).
- store\
- [content-addressed derivation]{#gloss-content-addressed-derivation}\
A derivation which has the
[`__contentAddressed`](language/advanced-attributes.md#adv-attr-__contentAddressed)
attribute set to `true`.
- [fixed-output derivation]{#gloss-fixed-output-derivation}\
A derivation which includes the
[`outputHash`](language/advanced-attributes.md#adv-attr-outputHash) attribute.
- [store]{#gloss-store}\
The location in the file system where store objects live. Typically
`/nix/store`.
- store path\
From the perspective of the location where Nix is
invoked, the Nix store can be referred to
as a "_local_" or a "_remote_" one:
+ A *local store* exists on the filesystem of
the machine where Nix is invoked. You can use other
local stores by passing the `--store` flag to the
`nix` command. Local stores can be used for building derivations.
+ A *remote store* exists anywhere other than the
local filesystem. One example is the `/nix/store`
directory on another machine, accessed via `ssh` or
served by the `nix-serve` Perl script.
- [chroot store]{#gloss-chroot-store}\
A local store whose canonical path is anything other than `/nix/store`.
- [binary cache]{#gloss-binary-cache}\
A *binary cache* is a Nix store which uses a different format: its
metadata and signatures are kept in `.narinfo` files rather than in a
Nix database. This different format simplifies serving store objects
over the network, but cannot host builds. Examples of binary caches
include S3 buckets and the [NixOS binary
cache](https://cache.nixos.org).
- [store path]{#gloss-store-path}\
The location in the file system of a store object, i.e., an
immediate child of the Nix store directory.
- store object\
- [store object]{#gloss-store-object}\
A file that is an immediate child of the Nix store directory. These
can be regular files, but also entire directory trees. Store objects
can be sources (objects copied from outside of the store),
derivation outputs (objects produced by running a build action), or
derivations (files describing a build action).
- substitute\
- [input-addressed store object]{#gloss-input-addressed-store-object}\
A store object produced by building a
non-[content-addressed](#gloss-content-addressed-derivation),
non-[fixed-output](#gloss-fixed-output-derivation)
derivation.
- [output-addressed store object]{#gloss-output-addressed-store-object}\
A store object whose store path hashes its content. This
includes derivations, the outputs of
[content-addressed derivations](#gloss-content-addressed-derivation),
and the outputs of
[fixed-output derivations](#gloss-fixed-output-derivation).
- [substitute]{#gloss-substitute}\
A substitute is a command invocation stored in the Nix database that
describes how to build a store object, bypassing the normal build
mechanism (i.e., derivations). Typically, the substitute builds the
store object by downloading a pre-built version of the store object
from some server.
- purity\
- [substituter]{#gloss-substituter}\
A *substituter* is an additional store from which Nix will
copy store objects it doesn't have. For details, see the
[`substituters` option](command-ref/conf-file.html#conf-substituters).
- [purity]{#gloss-purity}\
The assumption that equal Nix derivations when run always produce
the same output. This cannot be guaranteed in general (e.g., a
builder can rely on external inputs such as the network or the
system time) but the Nix model assumes it.
- Nix expression\
- [Nix expression]{#gloss-nix-expression}\
A high-level description of software packages and compositions
thereof. Deploying software using Nix entails writing Nix
expressions for your packages. Nix expressions are translated to
derivations that are stored in the Nix store. These derivations can
then be built.
- reference\
- [reference]{#gloss-reference}\
A store path `P` is said to have a reference to a store path `Q` if
the store object at `P` contains the path `Q` somewhere. The
*references* of a store path are the set of store paths to which it
@@ -52,11 +104,11 @@
output paths), whereas an output path only references other output
paths.
- reachable\
- [reachable]{#gloss-reachable}\
A store path `Q` is reachable from another store path `P` if `Q`
is in the *closure* of the *references* relation.
- closure\
- [closure]{#gloss-closure}\
The closure of a store path is the set of store paths that are
directly or indirectly “reachable” from that store path; that is,
its the closure of the path under the *references* relation. For
@@ -71,34 +123,34 @@
to path `Q`, then `Q` is in the closure of `P`. Further, if `Q`
references `R` then `R` is also in the closure of `P`.
- output path\
- [output path]{#gloss-output-path}\
A store path produced by a derivation.
- deriver\
- [deriver]{#gloss-deriver}\
The deriver of an *output path* is the store
derivation that built it.
- validity\
- [validity]{#gloss-validity}\
A store path is considered *valid* if it exists in the file system,
is listed in the Nix database as being valid, and if all paths in
its closure are also valid.
- user environment\
- [user environment]{#gloss-user-env}\
An automatically generated store object that consists of a set of
symlinks to “active” applications, i.e., other store paths. These
are generated automatically by
[`nix-env`](command-ref/nix-env.md). See *profiles*.
- profile\
- [profile]{#gloss-profile}\
A symlink to the current *user environment* of a user, e.g.,
`/nix/var/nix/profiles/default`.
- NAR\
- [NAR]{#gloss-nar}\
A *N*ix *AR*chive. This is a serialisation of a path in the Nix
store. It can contain regular files, directories and symbolic
links. NARs are generated and unpacked using `nix-store --dump`
and `nix-store --restore`.
- `∅` \
- [`∅`]{#gloss-emtpy-set}\
The empty set symbol. In the context of profile history, this denotes a package is not present in a particular version of the profile.
- `ε` \
- [`ε`]{#gloss-epsilon}\
The epsilon symbol. In the context of a package, this means the version is empty. More precisely, the derivation does not have a version attribute.

View File

@@ -13,7 +13,7 @@ for your platform:
- multi-user on macOS
> **Notes on read-only filesystem root in macOS 10.15 Catalina +**
>
>
> - It took some time to support this cleanly. You may see posts,
> examples, and tutorials using obsolete workarounds.
> - Supporting it cleanly made macOS installs too complex to qualify
@@ -31,8 +31,8 @@ $ sh <(curl -L https://nixos.org/nix/install) --no-daemon
```
This will perform a single-user installation of Nix, meaning that `/nix`
is owned by the invoking user. You should run this under your usual user
account, *not* as root. The script will invoke `sudo` to create `/nix`
is owned by the invoking user. You can run this under your usual user
account or root. The script will invoke `sudo` to create `/nix`
if it doesnt already exist. If you dont have `sudo`, you should
manually create `/nix` first as root, e.g.:
@@ -71,11 +71,11 @@ $ sh <(curl -L https://nixos.org/nix/install) --daemon
The multi-user installation of Nix will create build users between the
user IDs 30001 and 30032, and a group with the group ID 30000. You
should run this under your usual user account, *not* as root. The script
can run this under your usual user account or root. The script
will invoke `sudo` as needed.
> **Note**
>
>
> If you need Nix to use a different group ID or user ID set, you will
> have to download the tarball manually and [edit the install
> script](#installing-from-a-binary-tarball).
@@ -84,7 +84,9 @@ The installer will modify `/etc/bashrc`, and `/etc/zshrc` if they exist.
The installer will first back up these files with a `.backup-before-nix`
extension. The installer will also create `/etc/profile.d/nix.sh`.
You can uninstall Nix with the following commands:
## Uninstalling
### Linux
```console
sudo rm -rf /etc/profile/nix.sh /etc/nix /nix ~root/.nix-profile ~root/.nix-defexpr ~root/.nix-channels ~/.nix-profile ~/.nix-defexpr ~/.nix-channels
@@ -95,21 +97,114 @@ sudo systemctl stop nix-daemon.service
sudo systemctl disable nix-daemon.socket
sudo systemctl disable nix-daemon.service
sudo systemctl daemon-reload
# If you are on macOS, you will need to run:
sudo launchctl unload /Library/LaunchDaemons/org.nixos.nix-daemon.plist
sudo rm /Library/LaunchDaemons/org.nixos.nix-daemon.plist
```
There may also be references to Nix in `/etc/profile`, `/etc/bashrc`,
and `/etc/zshrc` which you may remove.
# macOS Installation <a name="sect-macos-installation-change-store-prefix"></a><a name="sect-macos-installation-encrypted-volume"></a><a name="sect-macos-installation-symlink"></a><a name="sect-macos-installation-recommended-notes"></a>
### macOS
1. Edit `/etc/zshrc` and `/etc/bashrc` to remove the lines sourcing
`nix-daemon.sh`, which should look like this:
```bash
# Nix
if [ -e '/nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh' ]; then
. '/nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh'
fi
# End Nix
```
If these files haven't been altered since installing Nix you can simply put
the backups back in place:
```console
sudo mv /etc/zshrc.backup-before-nix /etc/zshrc
sudo mv /etc/bashrc.backup-before-nix /etc/bashrc
```
This will stop shells from sourcing the file and bringing everything you
installed using Nix in scope.
2. Stop and remove the Nix daemon services:
```console
sudo launchctl unload /Library/LaunchDaemons/org.nixos.nix-daemon.plist
sudo rm /Library/LaunchDaemons/org.nixos.nix-daemon.plist
sudo launchctl unload /Library/LaunchDaemons/org.nixos.darwin-store.plist
sudo rm /Library/LaunchDaemons/org.nixos.darwin-store.plist
```
This stops the Nix daemon and prevents it from being started next time you
boot the system.
3. Remove the `nixbld` group and the `_nixbuildN` users:
```console
sudo dscl . -delete /Groups/nixbld
for u in $(sudo dscl . -list /Users | grep _nixbld); do sudo dscl . -delete /Users/$u; done
```
This will remove all the build users that no longer serve a purpose.
4. Edit fstab using `sudo vifs` to remove the line mounting the Nix Store
volume on `/nix`, which looks like
`UUID=<uuid> /nix apfs rw,noauto,nobrowse,suid,owners` or
`LABEL=Nix\040Store /nix apfs rw,nobrowse`. This will prevent automatic
mounting of the Nix Store volume.
5. Edit `/etc/synthetic.conf` to remove the `nix` line. If this is the only
line in the file you can remove it entirely, `sudo rm /etc/synthetic.conf`.
This will prevent the creation of the empty `/nix` directory to provide a
mountpoint for the Nix Store volume.
6. Remove the files Nix added to your system:
```console
sudo rm -rf /etc/nix /var/root/.nix-profile /var/root/.nix-defexpr /var/root/.nix-channels ~/.nix-profile ~/.nix-defexpr ~/.nix-channels
```
This gets rid of any data Nix may have created except for the store which is
removed next.
7. Remove the Nix Store volume:
```console
sudo diskutil apfs deleteVolume /nix
```
This will remove the Nix Store volume and everything that was added to the
store.
If the output indicates that the command couldn't remove the volume, you should
make sure you don't have an _unmounted_ Nix Store volume. Look for a
"Nix Store" volume in the output of the following command:
```console
diskutil list
```
If you _do_ see a "Nix Store" volume, delete it by re-running the diskutil
deleteVolume command, but replace `/nix` with the store volume's `diskXsY`
identifier.
> **Note**
>
> After you complete the steps here, you will still have an empty `/nix`
> directory. This is an expected sign of a successful uninstall. The empty
> `/nix` directory will disappear the next time you reboot.
>
> You do not have to reboot to finish uninstalling Nix. The uninstall is
> complete. macOS (Catalina+) directly controls root directories and its
> read-only root will prevent you from manually deleting the empty `/nix`
> mountpoint.
# macOS Installation
[]{#sect-macos-installation-change-store-prefix}[]{#sect-macos-installation-encrypted-volume}[]{#sect-macos-installation-symlink}[]{#sect-macos-installation-recommended-notes}
<!-- Note: anchors above to catch permalinks to old explanations -->
We believe we have ironed out how to cleanly support the read-only root
on modern macOS. New installs will do this automatically, and you can
also re-run a new installer to convert your existing setup.
on modern macOS. New installs will do this automatically.
This section previously detailed the situation, options, and trade-offs,
but it now only outlines what the installer does. You don't need to know

View File

@@ -2,7 +2,7 @@
Derivations can declare some infrequently used optional attributes.
- `allowedReferences`\
- [`allowedReferences`]{#adv-attr-allowedReferences}\
The optional attribute `allowedReferences` specifies a list of legal
references (dependencies) of the output of the builder. For example,
@@ -17,7 +17,7 @@ Derivations can declare some infrequently used optional attributes.
booting Linux dont have accidental dependencies on other paths in
the Nix store.
- `allowedRequisites`\
- [`allowedRequisites`]{#adv-attr-allowedRequisites}\
This attribute is similar to `allowedReferences`, but it specifies
the legal requisites of the whole closure, so all the dependencies
recursively. For example,
@@ -30,7 +30,7 @@ Derivations can declare some infrequently used optional attributes.
runtime dependency than `foobar`, and in addition it enforces that
`foobar` itself doesn't introduce any other dependency itself.
- `disallowedReferences`\
- [`disallowedReferences`]{#adv-attr-disallowedReferences}\
The optional attribute `disallowedReferences` specifies a list of
illegal references (dependencies) of the output of the builder. For
example,
@@ -42,7 +42,7 @@ Derivations can declare some infrequently used optional attributes.
enforces that the output of a derivation cannot have a direct
runtime dependencies on the derivation `foo`.
- `disallowedRequisites`\
- [`disallowedRequisites`]{#adv-attr-disallowedRequisites}\
This attribute is similar to `disallowedReferences`, but it
specifies illegal requisites for the whole closure, so all the
dependencies recursively. For example,
@@ -55,7 +55,7 @@ Derivations can declare some infrequently used optional attributes.
dependency on `foobar` or any other derivation depending recursively
on `foobar`.
- `exportReferencesGraph`\
- [`exportReferencesGraph`]{#adv-attr-exportReferencesGraph}\
This attribute allows builders access to the references graph of
their inputs. The attribute is a list of inputs in the Nix store
whose references graph the builder needs to know. The value of
@@ -84,7 +84,7 @@ Derivations can declare some infrequently used optional attributes.
with a Nix store containing the closure of a bootable NixOS
configuration).
- `impureEnvVars`\
- [`impureEnvVars`]{#adv-attr-impureEnvVars}\
This attribute allows you to specify a list of environment variables
that should be passed from the environment of the calling user to
the builder. Usually, the environment is cleared completely when the
@@ -112,7 +112,7 @@ Derivations can declare some infrequently used optional attributes.
> environmental variables come from the environment of the
> `nix-build`.
- `outputHash`; `outputHashAlgo`; `outputHashMode`\
- [`outputHash`]{#adv-attr-outputHash}; [`outputHashAlgo`]{#adv-attr-outputHashAlgo}; [`outputHashMode`]{#adv-attr-outputHashMode}\
These attributes declare that the derivation is a so-called
*fixed-output derivation*, which means that a cryptographic hash of
the output is already known in advance. When the build of a
@@ -208,7 +208,7 @@ Derivations can declare some infrequently used optional attributes.
[`nix-hash` command](../command-ref/nix-hash.md) for information
about converting to and from base-32 notation.)
- `__contentAddressed`
- [`__contentAddressed`]{#adv-attr-__contentAddressed}
If this **experimental** attribute is set to true, then the derivation
outputs will be stored in a content-addressed location rather than the
traditional input-addressed one.
@@ -216,7 +216,7 @@ Derivations can declare some infrequently used optional attributes.
Setting this attribute also requires setting `outputHashMode` and `outputHashAlgo` like for *fixed-output derivations* (see above).
- `passAsFile`\
- [`passAsFile`]{#adv-attr-passAsFile}\
A list of names of attributes that should be passed via files rather
than environment variables. For example, if you have
@@ -234,7 +234,7 @@ Derivations can declare some infrequently used optional attributes.
builder, since most operating systems impose a limit on the size
of the environment (typically, a few hundred kilobyte).
- `preferLocalBuild`\
- [`preferLocalBuild`]{#adv-attr-preferLocalBuild}\
If this attribute is set to `true` and [distributed building is
enabled](../advanced-topics/distributed-builds.md), then, if
possible, the derivation will be built locally instead of forwarded
@@ -242,7 +242,7 @@ Derivations can declare some infrequently used optional attributes.
where the cost of doing a download or remote build would exceed
the cost of building locally.
- `allowSubstitutes`\
- [`allowSubstitutes`]{#adv-attr-allowSubstitutes}\
If this attribute is set to `false`, then Nix will always build this
derivation; it will not try to substitute its outputs. This is
useful for very trivial derivations (such as `writeText` in Nixpkgs)

View File

@@ -14,7 +14,7 @@ Here are the constants built into the Nix expression evaluator:
This allows a Nix expression to fall back gracefully on older Nix
installations that dont have the desired built-in function.
- `builtins.currentSystem`\
- [`builtins.currentSystem`]{#builtins-currentSystem}\
The built-in value `currentSystem` evaluates to the Nix platform
identifier for the Nix installation on which the expression is being
evaluated, such as `"i686-linux"` or `"x86_64-darwin"`.

View File

@@ -4,7 +4,7 @@ The most important built-in function is `derivation`, which is used to
describe a single derivation (a build action). It takes as input a set,
the attributes of which specify the inputs of the build.
- There must be an attribute named `system` whose value must be a
- There must be an attribute named [`system`]{#attr-system} whose value must be a
string specifying a Nix system type, such as `"i686-linux"` or
`"x86_64-darwin"`. (To figure out your system type, run `nix -vv
--version`.) The build can only be performed on a machine and

View File

@@ -0,0 +1,33 @@
# Nix Language
The Nix language is
- *domain-specific*
It only exists for the Nix package manager:
to describe packages and configurations as well as their variants and compositions.
It is not intended for general purpose use.
- *declarative*
There is no notion of executing sequential steps.
Dependencies between operations are established only through data.
- *pure*
Values cannot change during computation.
Functions always produce the same output if their input does not change.
- *functional*
Functions are like any other value.
Functions can be assigned to names, taken as arguments, or returned by functions.
- *lazy*
Expressions are only evaluated when their value is needed.
- *dynamically typed*
Type errors are only detected when expressions are evaluated.

View File

@@ -1,6 +1,6 @@
# Operators
The table below lists the operators in the Nix expression language, in
The table below lists the operators in the Nix language, in
order of precedence (from strongest to weakest binding).
| Name | Syntax | Associativity | Description | Precedence |

View File

@@ -0,0 +1,261 @@
# Data Types
## Primitives
- <a id="type-string" href="#type-string">String</a>
*Strings* can be written in three ways.
The most common way is to enclose the string between double quotes,
e.g., `"foo bar"`. Strings can span multiple lines. The special
characters `"` and `\` and the character sequence `${` must be
escaped by prefixing them with a backslash (`\`). Newlines, carriage
returns and tabs can be written as `\n`, `\r` and `\t`,
respectively.
You can include the result of an expression into a string by
enclosing it in `${...}`, a feature known as *antiquotation*. The
enclosed expression must evaluate to something that can be coerced
into a string (meaning that it must be a string, a path, or a
derivation). For instance, rather than writing
```nix
"--with-freetype2-library=" + freetype + "/lib"
```
(where `freetype` is a derivation), you can instead write the more
natural
```nix
"--with-freetype2-library=${freetype}/lib"
```
The latter is automatically translated to the former. A more
complicated example (from the Nix expression for
[Qt](http://www.trolltech.com/products/qt)):
```nix
configureFlags = "
-system-zlib -system-libpng -system-libjpeg
${if openglSupport then "-dlopen-opengl
-L${mesa}/lib -I${mesa}/include
-L${libXmu}/lib -I${libXmu}/include" else ""}
${if threadSupport then "-thread" else "-no-thread"}
";
```
Note that Nix expressions and strings can be arbitrarily nested; in
this case the outer string contains various antiquotations that
themselves contain strings (e.g., `"-thread"`), some of which in
turn contain expressions (e.g., `${mesa}`).
The second way to write string literals is as an *indented string*,
which is enclosed between pairs of *double single-quotes*, like so:
```nix
''
This is the first line.
This is the second line.
This is the third line.
''
```
This kind of string literal intelligently strips indentation from
the start of each line. To be precise, it strips from each line a
number of spaces equal to the minimal indentation of the string as a
whole (disregarding the indentation of empty lines). For instance,
the first and second line are indented two spaces, while the third
line is indented four spaces. Thus, two spaces are stripped from
each line, so the resulting string is
```nix
"This is the first line.\nThis is the second line.\n This is the third line.\n"
```
Note that the whitespace and newline following the opening `''` is
ignored if there is no non-whitespace text on the initial line.
Antiquotation (`${expr}`) is supported in indented strings.
Since `${` and `''` have special meaning in indented strings, you
need a way to quote them. `$` can be escaped by prefixing it with
`''` (that is, two single quotes), i.e., `''$`. `''` can be escaped
by prefixing it with `'`, i.e., `'''`. `$` removes any special
meaning from the following `$`. Linefeed, carriage-return and tab
characters can be written as `''\n`, `''\r`, `''\t`, and `''\`
escapes any other character.
Indented strings are primarily useful in that they allow multi-line
string literals to follow the indentation of the enclosing Nix
expression, and that less escaping is typically necessary for
strings representing languages such as shell scripts and
configuration files because `''` is much less common than `"`.
Example:
```nix
stdenv.mkDerivation {
...
postInstall =
''
mkdir $out/bin $out/etc
cp foo $out/bin
echo "Hello World" > $out/etc/foo.conf
${if enableBar then "cp bar $out/bin" else ""}
'';
...
}
```
Finally, as a convenience, *URIs* as defined in appendix B of
[RFC 2396](http://www.ietf.org/rfc/rfc2396.txt) can be written *as
is*, without quotes. For instance, the string
`"http://example.org/foo.tar.bz2"` can also be written as
`http://example.org/foo.tar.bz2`.
- <a id="type-number" href="#type-number">Number</a>
Numbers, which can be *integers* (like `123`) or *floating point*
(like `123.43` or `.27e13`).
Numbers are type-compatible: pure integer operations will always
return integers, whereas any operation involving at least one
floating point number will have a floating point number as a result.
- <a id="type-path" href="#type-path">Path</a>
*Paths*, e.g., `/bin/sh` or `./builder.sh`. A path must contain at
least one slash to be recognised as such. For instance, `builder.sh`
is not a path: it's parsed as an expression that selects the
attribute `sh` from the variable `builder`. If the file name is
relative, i.e., if it does not begin with a slash, it is made
absolute at parse time relative to the directory of the Nix
expression that contained it. For instance, if a Nix expression in
`/foo/bar/bla.nix` refers to `../xyzzy/fnord.nix`, the absolute path
is `/foo/xyzzy/fnord.nix`.
If the first component of a path is a `~`, it is interpreted as if
the rest of the path were relative to the user's home directory.
e.g. `~/foo` would be equivalent to `/home/edolstra/foo` for a user
whose home directory is `/home/edolstra`.
Paths can also be specified between angle brackets, e.g.
`<nixpkgs>`. This means that the directories listed in the
environment variable `NIX_PATH` will be searched for the given file
or directory name.
Antiquotation is supported in any paths except those in angle brackets.
`./${foo}-${bar}.nix` is a more convenient way of writing
`./. + "/" + foo + "-" + bar + ".nix"` or `./. + "/${foo}-${bar}.nix"`. At
least one slash must appear *before* any antiquotations for this to be
recognized as a path. `a.${foo}/b.${bar}` is a syntactically valid division
operation. `./a.${foo}/b.${bar}` is a path.
- <a id="type-boolean" href="#type-boolean">Boolean</a>
*Booleans* with values `true` and `false`.
- <a id="type-null" href="#type-null">Null</a>
The null value, denoted as `null`.
## List
Lists are formed by enclosing a whitespace-separated list of values
between square brackets. For example,
```nix
[ 123 ./foo.nix "abc" (f { x = y; }) ]
```
defines a list of four elements, the last being the result of a call to
the function `f`. Note that function calls have to be enclosed in
parentheses. If they had been omitted, e.g.,
```nix
[ 123 ./foo.nix "abc" f { x = y; } ]
```
the result would be a list of five elements, the fourth one being a
function and the fifth being a set.
Note that lists are only lazy in values, and they are strict in length.
## Attribute Set
An attribute set is a collection of name-value-pairs (called *attributes*) enclosed in curly brackets (`{ }`).
Names and values are separated by an equal sign (`=`).
Each value is an arbitrary expression terminated by a semicolon (`;`).
Attributes can appear in any order.
An attribute name may only occur once.
Example:
```nix
{
x = 123;
text = "Hello";
y = f { bla = 456; };
}
```
This defines a set with attributes named `x`, `text`, `y`.
Attributes can be selected from a set using the `.` operator. For
instance,
```nix
{ a = "Foo"; b = "Bar"; }.a
```
evaluates to `"Foo"`. It is possible to provide a default value in an
attribute selection using the `or` keyword. For example,
```nix
{ a = "Foo"; b = "Bar"; }.c or "Xyzzy"
```
will evaluate to `"Xyzzy"` because there is no `c` attribute in the set.
You can use arbitrary double-quoted strings as attribute names:
```nix
{ "foo ${bar}" = 123; "nix-1.0" = 456; }."foo ${bar}"
```
This will evaluate to `123` (Assuming `bar` is antiquotable). In the
case where an attribute name is just a single antiquotation, the quotes
can be dropped:
```nix
{ foo = 123; }.${bar} or 456
```
This will evaluate to `123` if `bar` evaluates to `"foo"` when coerced
to a string and `456` otherwise (again assuming `bar` is antiquotable).
In the special case where an attribute name inside of a set declaration
evaluates to `null` (which is normally an error, as `null` is not
antiquotable), that attribute is simply not added to the set:
```nix
{ ${if foo then "bar" else null} = true; }
```
This will evaluate to `{}` if `foo` evaluates to `false`.
A set that has a `__functor` attribute whose value is callable (i.e. is
itself a function or a set with a `__functor` attribute whose value is
callable) can be applied as if it were a function, with the set itself
passed in first , e.g.,
```nix
let add = { __functor = self: x: x + self.x; };
inc = add // { x = 1; };
in inc 1
```
evaluates to `2`. This can be used to attach metadata to a function
without the caller needing to treat it specially, or to implement a form
of object-oriented programming, for example.

View File

@@ -1,5 +1,4 @@
This chapter discusses how to do package management with Nix, i.e.,
how to obtain, install, upgrade, and erase packages. This is the
“users” perspective of the Nix system — people who want to *create*
packages should consult the [chapter on writing Nix
expressions](../expressions/writing-nix-expressions.md).
packages should consult the chapter on the [Nix language](../language/index.md).

View File

@@ -0,0 +1,31 @@
# Release 2.10 (2022-07-11)
* `nix repl` now takes installables on the command line, unifying the usage
with other commands that use `--file` and `--expr`. Primary breaking change
is for the common usage of `nix repl '<nixpkgs>'` which can be recovered with
`nix repl --file '<nixpkgs>'` or `nix repl --expr 'import <nixpkgs>{}'`.
This is currently guarded by the `repl-flake` experimental feature.
* A new function `builtins.traceVerbose` is available. It is similar
to `builtins.trace` if the `trace-verbose` setting is set to true,
and it is a no-op otherwise.
* `nix search` has a new flag `--exclude` to filter out packages.
* On Linux, if `/nix` doesn't exist and cannot be created and you're
not running as root, Nix will automatically use
`~/.local/share/nix/root` as a chroot store. This enables non-root
users to download the statically linked Nix binary and have it work
out of the box, e.g.
```
# ~/nix run nixpkgs#hello
warning: '/nix' does not exists, so Nix will use '/home/ubuntu/.local/share/nix/root' as a chroot store
Hello, world!
```
* `flake-registry.json` is now fetched from `channels.nixos.org`.
* Nix can now be built with LTO by passing `--enable-lto` to `configure`.
LTO is currently only supported when building with GCC.

View File

@@ -0,0 +1,5 @@
# Release 2.11 (2022-08-24)
* `nix copy` now copies the store paths in parallel as much as possible (again).
This doesn't apply for the `daemon` and `ssh-ng` stores which copy everything
in one batch to avoid latencies issues.

View File

@@ -0,0 +1,33 @@
# Release 2.7 (2022-03-07)
* Nix will now make some helpful suggestions when you mistype
something on the command line. For instance, if you type `nix build
nixpkgs#thunderbrd`, it will suggest `thunderbird`.
* A number of "default" flake output attributes have been
renamed. These are:
* `defaultPackage.<system>``packages.<system>.default`
* `defaultApps.<system>``apps.<system>.default`
* `defaultTemplate``templates.default`
* `defaultBundler.<system>``bundlers.<system>.default`
* `overlay``overlays.default`
* `devShell.<system>``devShells.<system>.default`
The old flake output attributes still work, but `nix flake check`
will warn about them.
* Breaking API change: `nix bundle` now supports bundlers of the form
`bundler.<system>.<name>= derivation: another-derivation;`. This
supports additional functionality to inspect evaluation information
during bundling. A new
[repository](https://github.com/NixOS/bundlers) has various bundlers
implemented.
* `nix store ping` now reports the version of the remote Nix daemon.
* `nix flake {init,new}` now display information about which files have been
created.
* Templates can now define a `welcomeText` attribute, which is printed out by
`nix flake {init,new} --template <template>`.

View File

@@ -0,0 +1,53 @@
# Release 2.8 (2022-04-19)
* New experimental command: `nix fmt`, which applies a formatter
defined by the `formatter.<system>` flake output to the Nix
expressions in a flake.
* Various Nix commands can now read expressions from standard input
using `--file -`.
* New experimental builtin function `builtins.fetchClosure` that
copies a closure from a binary cache at evaluation time and rewrites
it to content-addressed form (if it isn't already). Like
`builtins.storePath`, this allows importing pre-built store paths;
the difference is that it doesn't require the user to configure
binary caches and trusted public keys.
This function is only available if you enable the experimental
feature `fetch-closure`.
* New experimental feature: *impure derivations*. These are
derivations that can produce a different result every time they're
built. Here is an example:
```nix
stdenv.mkDerivation {
name = "impure";
__impure = true; # marks this derivation as impure
buildCommand = "date > $out";
}
```
Running `nix build` twice on this expression will build the
derivation twice, producing two different content-addressed store
paths. Like fixed-output derivations, impure derivations have access
to the network. Only fixed-output derivations and impure derivations
can depend on an impure derivation.
* `nix store make-content-addressable` has been renamed to `nix store
make-content-addressed`.
* The `nixosModule` flake output attribute has been renamed consistent
with the `.default` renames in Nix 2.7.
* `nixosModule` → `nixosModules.default`
As before, the old output will continue to work, but `nix flake check` will
issue a warning about it.
* `nix run` is now stricter in what it accepts: members of the `apps`
flake output are now required to be apps (as defined in [the
manual](https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-run.html#apps)),
and members of `packages` or `legacyPackages` must be derivations
(not apps).

View File

@@ -0,0 +1,47 @@
# Release 2.9 (2022-05-30)
* Running Nix with the new `--debugger` flag will cause it to start a
repl session if an exception is thrown during evaluation, or if
`builtins.break` is called. From there you can inspect the values
of variables and evaluate Nix expressions. In debug mode, the
following new repl commands are available:
```
:env Show env stack
:bt Show trace stack
:st Show current trace
:st <idx> Change to another trace in the stack
:c Go until end of program, exception, or builtins.break().
:s Go one step
```
Read more about the debugger
[here](https://www.zknotes.com/note/5970).
* Nix now provides better integration with zsh's `run-help`
feature. It is now included in the Nix installation in the form of
an autoloadable shell function, `run-help-nix`. It picks up Nix
subcommands from the currently typed in command and directs the user
to the associated man pages.
* `nix repl` has a new build-and-link (`:bl`) command that builds a
derivation while creating GC root symlinks.
* The path produced by `builtins.toFile` is now allowed to be imported
or read even with restricted evaluation. Note that this will not
work with a read-only store.
* `nix build` has a new `--print-out-paths` flag to print the
resulting output paths. This matches the default behaviour of
`nix-build`.
* You can now specify which outputs of a derivation `nix` should
operate on using the syntax `installable^outputs`,
e.g. `nixpkgs#glibc^dev,static` or `nixpkgs#glibc^*`. By default,
`nix` will use the outputs specified by the derivation's
`meta.outputsToInstall` attribute if it exists, or all outputs
otherwise.
* `builtins.fetchTree` (and flake inputs) can now be used to fetch
plain files over the `http(s)` and `file` protocols in addition to
directory tarballs.

View File

@@ -1,15 +1,7 @@
# Release X.Y (202?-??-??)
* `nix bundle` breaking API change now supports bundlers of the form
`bundler.<system>.<name>= derivation: another-derivation;`. This supports
additional functionality to inspect evaluation information during bundling. A
new [repository](https://github.com/NixOS/bundlers) has various bundlers
implemented.
* `<nix/fetchurl.nix>` now accepts an additional argument `impure` which
defaults to `false`. If it is set to `true`, the `hash` and `sha256`
arguments will be ignored and the resulting derivation will have
`__impure` set to `true`, making it an impure derivation.
* `nix store ping` now reports the version of the remote Nix daemon.
* `nix flake {init,new}` now display information about which files have been
created.
* Templates can now define a `welcomeText` attribute, which is printed out by
`nix flake {init,new} --template <template>`.

View File

@@ -5,6 +5,32 @@ rec {
concatStrings = concatStringsSep "";
replaceStringsRec = from: to: string:
# recursively replace occurrences of `from` with `to` within `string`
# example:
# replaceStringRec "--" "-" "hello-----world"
# => "hello-world"
let
replaced = replaceStrings [ from ] [ to ] string;
in
if replaced == string then string else replaceStringsRec from to replaced;
squash = replaceStringsRec "\n\n\n" "\n\n";
trim = string:
# trim trailing spaces and squash non-leading spaces
let
trimLine = line:
let
# separate leading spaces from the rest
parts = split "(^ *)" line;
spaces = head (elemAt parts 1);
rest = elemAt parts 2;
# drop trailing spaces
body = head (split " *$" rest);
in spaces + replaceStringsRec " " " " body;
in concatStringsSep "\n" (map trimLine (splitLines string));
# FIXME: O(n^2)
unique = foldl' (acc: e: if elem e acc then acc else acc ++ [ e ]) [];

View File

@@ -2,8 +2,12 @@
, lib ? pkgs.lib
, name ? "nix"
, tag ? "latest"
, bundleNixpkgs ? true
, channelName ? "nixpkgs"
, channelURL ? "https://nixos.org/channels/nixpkgs-unstable"
, extraPkgs ? []
, maxLayers ? 100
, nixConf ? {}
}:
let
defaultPkgs = with pkgs; [
@@ -22,13 +26,14 @@ let
findutils
iana-etc
git
];
openssh
] ++ extraPkgs;
users = {
root = {
uid = 0;
shell = "/bin/bash";
shell = "${pkgs.bashInteractive}/bin/bash";
home = "/root";
gid = 0;
};
@@ -120,20 +125,27 @@ let
(lib.attrValues (lib.mapAttrs groupToGroup groups))
);
nixConf = {
defaultNixConf = {
sandbox = "false";
build-users-group = "nixbld";
trusted-public-keys = "cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=";
trusted-public-keys = [ "cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=" ];
};
nixConfContents = (lib.concatStringsSep "\n" (lib.mapAttrsFlatten (n: v: "${n} = ${v}") nixConf)) + "\n";
nixConfContents = (lib.concatStringsSep "\n" (lib.mapAttrsFlatten (n: v:
let
vStr = if builtins.isList v then lib.concatStringsSep " " v else v;
in
"${n} = ${vStr}") (defaultNixConf // nixConf))) + "\n";
baseSystem =
let
nixpkgs = pkgs.path;
channel = pkgs.runCommand "channel-nixos" { } ''
channel = pkgs.runCommand "channel-nixos" { inherit bundleNixpkgs; } ''
mkdir $out
ln -s ${nixpkgs} $out/nixpkgs
echo "[]" > $out/manifest.nix
if [ "$bundleNixpkgs" ]; then
ln -s ${nixpkgs} $out/nixpkgs
echo "[]" > $out/manifest.nix
fi
'';
rootEnv = pkgs.buildPackages.buildEnv {
name = "root-profile-env";
@@ -228,7 +240,7 @@ let
in
pkgs.dockerTools.buildLayeredImageWithNixDb {
inherit name tag;
inherit name tag maxLayers;
contents = [ baseSystem ];

18
flake.lock generated
View File

@@ -18,17 +18,18 @@
},
"nixpkgs": {
"locked": {
"lastModified": 1632864508,
"narHash": "sha256-d127FIvGR41XbVRDPVvozUPQ/uRHbHwvfyKHwEt5xFM=",
"lastModified": 1657693803,
"narHash": "sha256-G++2CJ9u0E7NNTAi9n5G8TdDmGJXcIjkJ3NF8cetQB8=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "82891b5e2c2359d7e58d08849e4c89511ab94234",
"rev": "365e1b3a859281cf11b94f87231adeabbdd878a2",
"type": "github"
},
"original": {
"id": "nixpkgs",
"ref": "nixos-21.05-small",
"type": "indirect"
"owner": "NixOS",
"ref": "nixos-22.05-small",
"repo": "nixpkgs",
"type": "github"
}
},
"nixpkgs-regression": {
@@ -41,9 +42,10 @@
"type": "github"
},
"original": {
"id": "nixpkgs",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "215d4d0fd80ca5163643b03a33fde804a29cc1e2",
"type": "indirect"
"type": "github"
}
},
"root": {

150
flake.nix
View File

@@ -1,8 +1,8 @@
{
description = "The purely functional package manager";
inputs.nixpkgs.url = "nixpkgs/nixos-21.05-small";
inputs.nixpkgs-regression.url = "nixpkgs/215d4d0fd80ca5163643b03a33fde804a29cc1e2";
inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-22.05-small";
inputs.nixpkgs-regression.url = "github:NixOS/nixpkgs/215d4d0fd80ca5163643b03a33fde804a29cc1e2";
inputs.lowdown-src = { url = "github:kristapsdz/lowdown"; flake = false; };
outputs = { self, nixpkgs, nixpkgs-regression, lowdown-src }:
@@ -23,7 +23,7 @@
crossSystems = [ "armv6l-linux" "armv7l-linux" ];
stdenvs = [ "gccStdenv" "clangStdenv" "clang11Stdenv" "stdenv" ];
stdenvs = [ "gccStdenv" "clangStdenv" "clang11Stdenv" "stdenv" "libcxxStdenv" "ccacheStdenv" ];
forAllSystems = f: nixpkgs.lib.genAttrs systems (system: f system);
forAllSystemsAndStdenvs = f: forAllSystems (system:
@@ -36,7 +36,7 @@
)
);
forAllStdenvs = stdenvs: f: nixpkgs.lib.genAttrs stdenvs (stdenv: f stdenv);
forAllStdenvs = f: nixpkgs.lib.genAttrs stdenvs (stdenv: f stdenv);
# Memoize nixpkgs for different platforms for efficiency.
nixpkgsFor =
@@ -54,7 +54,7 @@
# we want most of the time and for backwards compatibility
forAllSystems (system: stdenvsPackages.${system} // stdenvsPackages.${system}.stdenvPackages);
commonDeps = pkgs: with pkgs; rec {
commonDeps = { pkgs, isStatic ? false }: with pkgs; rec {
# Use "busybox-sandbox-shell" if present,
# if not (legacy) fallback and hope it's sufficient.
sh = pkgs.busybox-sandbox-shell or (busybox.override {
@@ -85,10 +85,11 @@
lib.optionals stdenv.isLinux [
"--with-boost=${boost}/lib"
"--with-sandbox-shell=${sh}/bin/busybox"
]
++ lib.optionals (stdenv.isLinux && !(isStatic && stdenv.system == "aarch64-linux")) [
"LDFLAGS=-fuse-ld=gold"
];
nativeBuildDeps =
[
buildPackages.bison
@@ -102,12 +103,12 @@
# Tests
buildPackages.git
buildPackages.mercurial # FIXME: remove? only needed for tests
buildPackages.jq
buildPackages.jq # Also for custom mdBook preprocessor.
]
++ lib.optionals stdenv.hostPlatform.isLinux [(buildPackages.util-linuxMinimal or buildPackages.utillinuxMinimal)];
buildDeps =
[ curl
[ (curl.override { patchNetrcRegression = true; })
bzip2 xz brotli editline
openssl sqlite
libarchive
@@ -135,11 +136,6 @@
}))
nlohmann_json
];
perlDeps =
[ perl
perlPackages.DBDSQLite
];
};
installScriptFor = systems:
@@ -176,7 +172,7 @@
echo "file installer $out/install" >> $out/nix-support/hydra-build-products
'';
testNixVersions = pkgs: client: daemon: with commonDeps pkgs; with pkgs.lib; pkgs.stdenv.mkDerivation {
testNixVersions = pkgs: client: daemon: with commonDeps { inherit pkgs; }; with pkgs.lib; pkgs.stdenv.mkDerivation {
NIX_DAEMON_PACKAGE = daemon;
NIX_CLIENT_PACKAGE = client;
name =
@@ -264,6 +260,7 @@
echo "file binary-dist $fn" >> $out/nix-support/hydra-build-products
tar cvfJ $fn \
--owner=0 --group=0 --mode=u+rw,uga+r \
--mtime='1970-01-01' \
--absolute-names \
--hard-dereference \
--transform "s,$TMPDIR/install,$dir/install," \
@@ -287,7 +284,7 @@
# Forward from the previous stage as we dont want it to pick the lowdown override
nixUnstable = prev.nixUnstable;
nix = with final; with commonDeps pkgs; currentStdenv.mkDerivation {
nix = with final; with commonDeps { inherit pkgs; }; currentStdenv.mkDerivation {
name = "nix-${version}";
inherit version;
@@ -319,6 +316,7 @@
for LIB in $out/lib/*.dylib; do
chmod u+w $LIB
install_name_tool -id $LIB $LIB
install_name_tool -delete_rpath ${boost}/lib/ $LIB || true
done
install_name_tool -change ${boost}/lib/libboost_system.dylib $out/lib/libboost_system.dylib $out/lib/libboost_thread.dylib
''}
@@ -353,7 +351,7 @@
strictDeps = true;
passthru.perl-bindings = with final; currentStdenv.mkDerivation {
passthru.perl-bindings = with final; perl.pkgs.toPerlModule (currentStdenv.mkDerivation {
name = "nix-perl-${version}";
src = self;
@@ -366,7 +364,7 @@
buildInputs =
[ nix
curl
(curl.override { patchNetrcRegression = true; })
bzip2
xz
pkgs.perl
@@ -375,16 +373,17 @@
++ lib.optional (currentStdenv.isLinux || currentStdenv.isDarwin) libsodium
++ lib.optional currentStdenv.isDarwin darwin.apple_sdk.frameworks.Security;
configureFlags = ''
--with-dbi=${perlPackages.DBI}/${pkgs.perl.libPrefix}
--with-dbd-sqlite=${perlPackages.DBDSQLite}/${pkgs.perl.libPrefix}
'';
configureFlags = [
"--with-dbi=${perlPackages.DBI}/${pkgs.perl.libPrefix}"
"--with-dbd-sqlite=${perlPackages.DBDSQLite}/${pkgs.perl.libPrefix}"
];
enableParallelBuilding = true;
postUnpack = "sourceRoot=$sourceRoot/perl";
};
});
meta.platforms = systems;
};
lowdown-nix = with final; currentStdenv.mkDerivation rec {
@@ -409,7 +408,7 @@
# A Nixpkgs overlay that overrides the 'nix' and
# 'nix.perl-bindings' packages.
overlay = overlayFor (p: p.stdenv);
overlays.default = overlayFor (p: p.stdenv);
hydraJobs = {
@@ -434,7 +433,7 @@
value = let
nixpkgsCross = import nixpkgs {
inherit system crossSystem;
overlays = [ self.overlay ];
overlays = [ self.overlays.default ];
};
in binaryTarball nixpkgsFor.${system} self.packages.${system}."nix-${crossSystem}" nixpkgsCross;
}) crossSystems));
@@ -452,7 +451,7 @@
# Line coverage analysis.
coverage =
with nixpkgsFor.x86_64-linux;
with commonDeps pkgs;
with commonDeps { inherit pkgs; };
releaseTools.coverageAnalysis {
name = "nix-coverage-${version}";
@@ -480,31 +479,31 @@
tests.remoteBuilds = import ./tests/remote-builds.nix {
system = "x86_64-linux";
inherit nixpkgs;
inherit (self) overlay;
overlay = self.overlays.default;
};
tests.nix-copy-closure = import ./tests/nix-copy-closure.nix {
system = "x86_64-linux";
inherit nixpkgs;
inherit (self) overlay;
overlay = self.overlays.default;
};
tests.nssPreload = (import ./tests/nss-preload.nix rec {
system = "x86_64-linux";
inherit nixpkgs;
inherit (self) overlay;
overlay = self.overlays.default;
});
tests.githubFlakes = (import ./tests/github-flakes.nix rec {
system = "x86_64-linux";
inherit nixpkgs;
inherit (self) overlay;
overlay = self.overlays.default;
});
tests.sourcehutFlakes = (import ./tests/sourcehut-flakes.nix rec {
system = "x86_64-linux";
inherit nixpkgs;
inherit (self) overlay;
overlay = self.overlays.default;
});
tests.setuid = nixpkgs.lib.genAttrs
@@ -512,7 +511,7 @@
(system:
import ./tests/setuid.nix rec {
inherit nixpkgs system;
inherit (self) overlay;
overlay = self.overlays.default;
});
# Make sure that nix-env still produces the exact same result
@@ -547,6 +546,11 @@
# againstLatestStable = testNixVersions pkgs pkgs.nix pkgs.nixStable;
} "touch $out");
installerTests = import ./tests/installer {
binaryTarballs = self.hydraJobs.binaryTarball;
inherit nixpkgsFor;
};
};
checks = forAllSystems (system: {
@@ -557,12 +561,13 @@
dockerImage = self.hydraJobs.dockerImage.${system};
});
packages = forAllSystems (system: {
packages = forAllSystems (system: rec {
inherit (nixpkgsFor.${system}) nix;
default = nix;
} // (nixpkgs.lib.optionalAttrs (builtins.elem system linux64BitSystems) {
nix-static = let
nixpkgs = nixpkgsFor.${system}.pkgsStatic;
in with commonDeps nixpkgs; nixpkgs.stdenv.mkDerivation {
in with commonDeps { pkgs = nixpkgs; isStatic = true; }; nixpkgs.stdenv.mkDerivation {
name = "nix-${version}";
src = self;
@@ -574,14 +579,24 @@
nativeBuildInputs = nativeBuildDeps;
buildInputs = buildDeps ++ propagatedDeps;
configureFlags = [ "--sysconfdir=/etc" ];
# Work around pkgsStatic disabling all tests.
# Remove in NixOS 22.11, see https://github.com/NixOS/nixpkgs/pull/140271.
preHook =
''
doCheck=1
doInstallCheck=1
'';
configureFlags =
configureFlags ++
[ "--sysconfdir=/etc"
"--enable-embedded-sandbox-shell"
];
enableParallelBuilding = true;
makeFlags = "profiledir=$(out)/etc/profile.d";
doCheck = true;
installFlags = "sysconfdir=$(out)/etc";
postInstall = ''
@@ -591,7 +606,6 @@
echo "file binary-dist $out/bin/nix" >> $out/nix-support/hydra-build-products
'';
doInstallCheck = true;
installCheckFlags = "sysconfdir=$(out)/etc";
stripAllList = ["bin"];
@@ -600,6 +614,7 @@
hardeningDisable = [ "pie" ];
};
dockerImage =
let
pkgs = nixpkgsFor.${system};
@@ -614,14 +629,16 @@
ln -s ${image} $image
echo "file binary-dist $image" >> $out/nix-support/hydra-build-products
'';
} // builtins.listToAttrs (map (crossSystem: {
}
// builtins.listToAttrs (map (crossSystem: {
name = "nix-${crossSystem}";
value = let
nixpkgsCross = import nixpkgs {
inherit system crossSystem;
overlays = [ self.overlay ];
overlays = [ self.overlays.default ];
};
in with commonDeps nixpkgsCross; nixpkgsCross.stdenv.mkDerivation {
in with commonDeps { pkgs = nixpkgsCross; }; nixpkgsCross.stdenv.mkDerivation {
name = "nix-${version}";
src = self;
@@ -653,44 +670,45 @@
doInstallCheck = true;
installCheckFlags = "sysconfdir=$(out)/etc";
};
}) crossSystems)) // (builtins.listToAttrs (map (stdenvName:
}) (if system == "x86_64-linux" then crossSystems else [])))
// (builtins.listToAttrs (map (stdenvName:
nixpkgsFor.${system}.lib.nameValuePair
"nix-${stdenvName}"
nixpkgsFor.${system}."${stdenvName}Packages".nix
) stdenvs)));
defaultPackage = forAllSystems (system: self.packages.${system}.nix);
devShells = forAllSystems (system:
forAllStdenvs (stdenv:
with nixpkgsFor.${system};
with commonDeps { inherit pkgs; };
nixpkgsFor.${system}.${stdenv}.mkDerivation {
name = "nix";
devShell = forAllSystems (system: self.devShells.${system}.stdenvPackages);
outputs = [ "out" "dev" "doc" ];
devShells = forAllSystemsAndStdenvs (system: stdenv:
with nixpkgsFor.${system};
with commonDeps pkgs;
nativeBuildInputs = nativeBuildDeps;
buildInputs = buildDeps ++ propagatedDeps ++ awsDeps;
nixpkgsFor.${system}.${stdenv}.mkDerivation {
name = "nix";
inherit configureFlags;
outputs = [ "out" "dev" "doc" ];
enableParallelBuilding = true;
nativeBuildInputs = nativeBuildDeps;
buildInputs = buildDeps ++ propagatedDeps ++ awsDeps ++ perlDeps;
installFlags = "sysconfdir=$(out)/etc";
inherit configureFlags;
shellHook =
''
PATH=$prefix/bin:$PATH
unset PYTHONPATH
export MANPATH=$out/share/man:$MANPATH
enableParallelBuilding = true;
installFlags = "sysconfdir=$(out)/etc";
shellHook =
''
PATH=$prefix/bin:$PATH
unset PYTHONPATH
export MANPATH=$out/share/man:$MANPATH
# Make bash completion work.
XDG_DATA_DIRS+=:$out/share
'';
});
# Make bash completion work.
XDG_DATA_DIRS+=:$out/share
'';
}
)
// { default = self.devShells.${system}.stdenv; }
);
};
}

View File

@@ -15,7 +15,7 @@ function _complete_nix {
else
COMPREPLY+=("$completion")
fi
done < <(NIX_GET_COMPLETIONS=$cword "${words[@]/#\~/$HOME}" 2>/dev/null)
done < <(NIX_GET_COMPLETIONS=$cword "${words[@]}" 2>/dev/null)
__ltrim_colon_completions "$cur"
}

View File

@@ -1,7 +1,8 @@
ifdef HOST_LINUX
$(foreach n, nix-daemon.socket nix-daemon.service, $(eval $(call install-file-in, $(d)/$(n), $(prefix)/lib/systemd/system, 0644)))
$(foreach n, nix-daemon.conf, $(eval $(call install-file-in, $(d)/$(n), $(prefix)/lib/tmpfiles.d, 0644)))
clean-files += $(d)/nix-daemon.socket $(d)/nix-daemon.service
clean-files += $(d)/nix-daemon.socket $(d)/nix-daemon.service $(d)/nix-daemon.conf
endif

View File

@@ -0,0 +1 @@
d @localstatedir@/nix/daemon-socket 0755 root root - -

View File

@@ -1,12 +1,15 @@
[Unit]
Description=Nix Daemon
Documentation=man:nix-daemon https://nixos.org/manual
RequiresMountsFor=@storedir@
RequiresMountsFor=@localstatedir@
RequiresMountsFor=@localstatedir@/nix/db
ConditionPathIsReadWrite=@localstatedir@/nix/daemon-socket
[Service]
ExecStart=@@bindir@/nix-daemon nix-daemon --daemon
KillMode=process
LimitNOFILE=4096
[Install]
WantedBy=multi-user.target

View File

@@ -10,14 +10,15 @@ function _nix() {
local -a suggestions
declare -a suggestions
for suggestion in ${res:1}; do
# FIXME: This doesn't work properly if the suggestion word contains a `:`
# itself
suggestions+="${suggestion/ /:}"
suggestions+=("${suggestion%% *}")
done
local -a args
if [[ "$tpe" == filenames ]]; then
compadd -f
args+=('-f')
elif [[ "$tpe" == attrs ]]; then
args+=('-S' '')
fi
_describe 'nix' suggestions
compadd -J nix "${args[@]}" -a suggestions
}
_nix "$@"

View File

@@ -1 +1,2 @@
$(eval $(call install-file-as, $(d)/completion.zsh, $(datarootdir)/zsh/site-functions/_nix, 0644))
$(eval $(call install-file-as, $(d)/run-help-nix, $(datarootdir)/zsh/site-functions/run-help-nix, 0644))

42
misc/zsh/run-help-nix Normal file
View File

@@ -0,0 +1,42 @@
emulate -L zsh
# run-help is a zsh widget that can be bound to a key. It mainly looks up the
# man page for the currently typed in command.
#
# Although run-help works for any command without requiring special support,
# it can only deduce the right man page based solely on the name of the
# command. Programs like Nix provide better integration with run-help by
# helping zsh identify Nix subcommands and their corresponding man pages. This
# is what this function does.
#
# To actually use run-help on zsh, place the following lines in your .zshrc:
#
# (( $+aliases[run-help] )) && unalias run-help
# autoload -Uz run-help run-help-nix
#
# Then also assign run-help to any key of choice:
#
# bindkey '^[h' run-help
while [[ "$#" != 0 && "$1" == -* ]]; do
shift
done
local -a subcommands; subcommands=( nix3 )
local arg
for arg in "$@"; do
if man -w "${(j:-:)subcommands}-$arg" >/dev/null 2>&1; then
subcommands+="$arg"
else
break
fi
done
if (( $#subcommands > 1 )); then
man "${(j:-:)subcommands}"
else
man nix
fi
return $?

View File

@@ -91,7 +91,7 @@ define build-library
$(1)_PATH := $$(_d)/$$($(1)_NAME).$(SO_EXT)
$$($(1)_PATH): $$($(1)_OBJS) $$(_libs) | $$(_d)/
$$(trace-ld) $(CXX) -o $$(abspath $$@) -shared $$(LDFLAGS) $$(GLOBAL_LDFLAGS) $$($(1)_OBJS) $$($(1)_LDFLAGS) $$($(1)_LDFLAGS_PROPAGATED) $$(foreach lib, $$($(1)_LIBS), $$($$(lib)_LDFLAGS_USE)) $$($(1)_LDFLAGS_UNINSTALLED)
+$$(trace-ld) $(CXX) -o $$(abspath $$@) -shared $$(LDFLAGS) $$(GLOBAL_LDFLAGS) $$($(1)_OBJS) $$($(1)_LDFLAGS) $$($(1)_LDFLAGS_PROPAGATED) $$(foreach lib, $$($(1)_LIBS), $$($$(lib)_LDFLAGS_USE)) $$($(1)_LDFLAGS_UNINSTALLED)
ifndef HOST_DARWIN
$(1)_LDFLAGS_USE += -Wl,-rpath,$$(abspath $$(_d))
@@ -105,7 +105,7 @@ define build-library
$$(eval $$(call create-dir, $$($(1)_INSTALL_DIR)))
$$($(1)_INSTALL_PATH): $$($(1)_OBJS) $$(_libs_final) | $(DESTDIR)$$($(1)_INSTALL_DIR)/
$$(trace-ld) $(CXX) -o $$@ -shared $$(LDFLAGS) $$(GLOBAL_LDFLAGS) $$($(1)_OBJS) $$($(1)_LDFLAGS) $$($(1)_LDFLAGS_PROPAGATED) $$(foreach lib, $$($(1)_LIBS), $$($$(lib)_LDFLAGS_USE_INSTALLED))
+$$(trace-ld) $(CXX) -o $$@ -shared $$(LDFLAGS) $$(GLOBAL_LDFLAGS) $$($(1)_OBJS) $$($(1)_LDFLAGS) $$($(1)_LDFLAGS_PROPAGATED) $$(foreach lib, $$($(1)_LIBS), $$($$(lib)_LDFLAGS_USE_INSTALLED))
$(1)_LDFLAGS_USE_INSTALLED += -L$$(DESTDIR)$$($(1)_INSTALL_DIR) -l$$(patsubst lib%,%,$$(strip $$($(1)_NAME)))
ifndef HOST_DARWIN
@@ -125,7 +125,7 @@ define build-library
$(1)_PATH := $$(_d)/$$($(1)_NAME).a
$$($(1)_PATH): $$($(1)_OBJS) | $$(_d)/
$$(trace-ld) $(LD) -Ur -o $$(_d)/$$($(1)_NAME).o $$?
+$$(trace-ld) $(LD) -Ur -o $$(_d)/$$($(1)_NAME).o $$^
$$(trace-ar) $(AR) crs $$@ $$(_d)/$$($(1)_NAME).o
$(1)_LDFLAGS_USE += $$($(1)_PATH) $$($(1)_LDFLAGS)

View File

@@ -32,7 +32,7 @@ define build-program
$$(eval $$(call create-dir, $$(_d)))
$$($(1)_PATH): $$($(1)_OBJS) $$(_libs) | $$(_d)/
$$(trace-ld) $(CXX) -o $$@ $$(LDFLAGS) $$(GLOBAL_LDFLAGS) $$($(1)_OBJS) $$($(1)_LDFLAGS) $$(foreach lib, $$($(1)_LIBS), $$($$(lib)_LDFLAGS_USE))
+$$(trace-ld) $(CXX) -o $$@ $$(LDFLAGS) $$(GLOBAL_LDFLAGS) $$($(1)_OBJS) $$($(1)_LDFLAGS) $$(foreach lib, $$($(1)_LIBS), $$($$(lib)_LDFLAGS_USE))
$(1)_INSTALL_DIR ?= $$(bindir)
@@ -49,7 +49,7 @@ define build-program
_libs_final := $$(foreach lib, $$($(1)_LIBS), $$($$(lib)_INSTALL_PATH))
$(DESTDIR)$$($(1)_INSTALL_PATH): $$($(1)_OBJS) $$(_libs_final) | $(DESTDIR)$$($(1)_INSTALL_DIR)/
$$(trace-ld) $(CXX) -o $$@ $$(LDFLAGS) $$(GLOBAL_LDFLAGS) $$($(1)_OBJS) $$($(1)_LDFLAGS) $$(foreach lib, $$($(1)_LIBS), $$($$(lib)_LDFLAGS_USE_INSTALLED))
+$$(trace-ld) $(CXX) -o $$@ $$(LDFLAGS) $$(GLOBAL_LDFLAGS) $$($(1)_OBJS) $$($(1)_LDFLAGS) $$(foreach lib, $$($(1)_LIBS), $$($$(lib)_LDFLAGS_USE_INSTALLED))
else

View File

@@ -14,9 +14,27 @@ if [ -t 1 ]; then
yellow=""
normal=""
fi
(cd tests && env ${TESTS_ENVIRONMENT} init.sh 2>/dev/null > /dev/null)
log="$(cd $(dirname $1) && env ${TESTS_ENVIRONMENT} $(basename $1) 2>&1)"
status=$?
run_test () {
(cd tests && env ${TESTS_ENVIRONMENT} init.sh 2>/dev/null > /dev/null)
log="$(cd $(dirname $1) && env ${TESTS_ENVIRONMENT} $(basename $1) 2>&1)"
status=$?
}
run_test "$1"
# Hack: Retry the test if it fails with “unexpected EOF reading a line” as these
# appear randomly without anyone knowing why.
# See https://github.com/NixOS/nix/issues/3605 for more info
if [[ $status -ne 0 && $status -ne 99 && \
"$(uname)" == "Darwin" && \
"$log" =~ "unexpected EOF reading a line" \
]]; then
echo "$post_run_msg [${yellow}FAIL$normal] (possibly flaky, so will be retried)"
echo "$log" | sed 's/^/ /'
run_test "$1"
fi
if [ $status -eq 0 ]; then
echo "$post_run_msg [${green}PASS$normal]"
elif [ $status -eq 99 ]; then

View File

@@ -240,7 +240,7 @@ SV * convertHash(char * algo, char * s, int toBase32)
PPCODE:
try {
auto h = Hash::parseAny(s, parseHashType(algo));
string s = h.to_string(toBase32 ? Base32 : Base16, false);
auto s = h.to_string(toBase32 ? Base32 : Base16, false);
XPUSHs(sv_2mortal(newSVpv(s.c_str(), 0)));
} catch (Error & e) {
croak("%s", e.what());

85
roadmap.md Normal file
View File

@@ -0,0 +1,85 @@
# Nix is everywhere
Nix is the universal build and [configuration management] tool.
[configuration management]: https://www.sebokwiki.org/wiki/Configuration_Management
Software developers use Nix as a matter of course every day, mostly without even noticing.
Nix runs trivially, anywhere.
For individuals to large organizations, Nix underpins the entire software supply chain:
- Developer tooling
- Build automation
- Binary distribution
To this end, the Nix team will work towards the following goals.
## Make Nix easy to adopt
- Well-defined target user base
- anyone who wants to manage the complexity of - and build - software
- transform bits in a declarative and reproducible way
- Well-defined core user stories
- Ad hoc environments
- One-liner setup (nix-shell)
- Declarative environments
- One-liner setup ("templates")
- Easy modification/extension
- Easy and transparent usage ("direnv"/"lorri")
- Secret management as first-class citizen
- Configurations as first-class citizens
- Configuration/Modules/Nickel/etc.
- Language bindings
- Supply chain trust solution
- Content-addressed derivation
- Build result signing, key distribution
- SBOM/SLSA
- Unprivileged installation and use
<!-- valentin: this needs clarification, I still don't know what rewriting is -->
- Portable store?
- restricted-root
- ACLs
- rewriting
- ...
- Linux, MacOS and Windows support at feature parity
## Make Nix a tool that users can rely on
- Reliable installer
- Effective testing
- Test coverage for all major use cases
- Memory safety validation (sanitizers, ...)
- Benchmarking infrastructure
- Test reports published and accessible
- Executable language specification
## Make Nix a good investment for users
- Explicit compatibility guarantees (and non-guarantees)
- Commitment to uncompromising reproducibility
- Well-defined release process
- Feature support status
- Deprecation strategy
- LTS commitments
- Close Flakes schism, remove uncertanity/ambiguity/confusion
- Exemplary contributor and maintainer experience
- Recommended development setup
- Testing guidance
- Formalize review criteria
- Formalize design criteria (technical invariants)
- Well-defined architecture of isolated components
- Swappable store
- Formalize store protocol
- Swappable Nix language evaluator
- Swappable scheduler and remote-build system
- Integrate Hydra (modulo UI) into Nix
- Remote protocol speed and reliability improvements
- Binary cache protocol speed improvements
- Minimal custom code base (proven off-the-shelf components where possible)
- Git file hashing
- Sandboxing, containers
- Capnproto for RPC
- Bazel RBE protocol
- ...

View File

@@ -14,7 +14,7 @@ curl -sS -H 'Accept: application/json' https://hydra.nixos.org/jobset/nix/master
someBuildFailed=0
for buildId in $BUILDS_FOR_LATEST_EVAL; do
buildInfo=$(curl -sS -H 'Accept: application/json' "https://hydra.nixos.org/build/$buildId")
buildInfo=$(curl --fail -sS -H 'Accept: application/json' "https://hydra.nixos.org/build/$buildId")
finished=$(echo "$buildInfo" | jq -r '.finished')

View File

@@ -246,7 +246,8 @@ get_volume_pass() {
verify_volume_pass() {
local volume_special="$1" # (i.e., disk1s7)
local volume_uuid="$2"
/usr/sbin/diskutil apfs unlockVolume "$volume_special" -verify -stdinpassphrase -user "$volume_uuid"
_sudo "to confirm the password actually unlocks the volume" \
/usr/sbin/diskutil apfs unlockVolume "$volume_special" -verify -stdinpassphrase -user "$volume_uuid"
}
volume_pass_works() {
@@ -441,9 +442,14 @@ add_nix_vol_fstab_line() {
local escaped_mountpoint="${NIX_ROOT/ /'\\\'040}"
shift
# wrap `ex` to work around a problem with vim plugins breaking exit codes;
# (see https://github.com/NixOS/nix/issues/5468)
# we'd prefer EDITOR="/usr/bin/ex --noplugin" but vifs doesn't word-split
# wrap `ex` to work around problems w/ vim features breaking exit codes
# - plugins (see github.com/NixOS/nix/issues/5468): -u NONE
# - swap file: -n
#
# the first draft used `--noplugin`, but github.com/NixOS/nix/issues/6462
# suggests we need the less-semantic `-u NONE`
#
# we'd prefer EDITOR="/usr/bin/ex -u NONE" but vifs doesn't word-split
# the EDITOR env.
#
# TODO: at some point we should switch to `--clean`, but it wasn't added
@@ -451,7 +457,7 @@ add_nix_vol_fstab_line() {
# minver 10.12.6 seems to have released with vim 7.4
cat > "$SCRATCH/ex_cleanroom_wrapper" <<EOF
#!/bin/sh
/usr/bin/ex --noplugin "\$@"
/usr/bin/ex -u NONE -n "\$@"
EOF
chmod 755 "$SCRATCH/ex_cleanroom_wrapper"
@@ -645,8 +651,9 @@ EOF
task "Configuring /etc/synthetic.conf to make a mount-point at $NIX_ROOT" >&2
# technically /etc/synthetic.d/nix is supported in Big Sur+
# but handling both takes even more code...
# See earlier note; `-u NONE` disables vim plugins/rc, `-n` skips swapfile
_sudo "to add Nix to /etc/synthetic.conf" \
/usr/bin/ex --noplugin /etc/synthetic.conf <<EOF
/usr/bin/ex -u NONE -n /etc/synthetic.conf <<EOF
:a
${NIX_ROOT:1}
.
@@ -685,22 +692,27 @@ encrypt_volume() {
local volume_uuid="$1"
local volume_label="$2"
local password
task "Encrypt the Nix volume" >&2
# Note: mount/unmount are late additions to support the right order
# of operations for creating the volume and then baking its uuid into
# other artifacts; not as well-trod wrt to potential errors, race
# conditions, etc.
/usr/sbin/diskutil mount "$volume_label"
_sudo "to mount your Nix volume for encrypting" \
/usr/sbin/diskutil mount "$volume_label"
password="$(/usr/bin/xxd -l 32 -p -c 256 /dev/random)"
_sudo "to add your Nix volume's password to Keychain" \
/usr/bin/security -i <<EOF
add-generic-password -a "$volume_label" -s "$volume_uuid" -l "$volume_label encryption password" -D "Encrypted volume password" -j "Added automatically by the Nix installer for use by $NIX_VOLUME_MOUNTD_DEST" -w "$password" -T /System/Library/CoreServices/APFSUserAgent -T /System/Library/CoreServices/CSUserAgent -T /usr/bin/security "/Library/Keychains/System.keychain"
EOF
builtin printf "%s" "$password" | _sudo "to encrypt your Nix volume" \
builtin printf "%s" "$password" | _sudo "to actually encrypt your Nix volume" \
/usr/sbin/diskutil apfs encryptVolume "$volume_label" -user disk -stdinpassphrase
/usr/sbin/diskutil unmount force "$volume_label"
_sudo "to unmount the encrypted volume" \
/usr/sbin/diskutil unmount force "$volume_label"
}
create_volume() {
@@ -809,7 +821,8 @@ setup_volume_daemon() {
local volume_uuid="$2"
if ! test_voldaemon; then
task "Configuring LaunchDaemon to mount '$NIX_VOLUME_LABEL'" >&2
_sudo "to install the Nix volume mounter" /usr/bin/ex --noplugin "$NIX_VOLUME_MOUNTD_DEST" <<EOF
# See earlier note; `-u NONE` disables vim plugins/rc, `-n` skips swapfile
_sudo "to install the Nix volume mounter" /usr/bin/ex -u NONE -n "$NIX_VOLUME_MOUNTD_DEST" <<EOF
:a
$(generate_mount_daemon "$cmd_type" "$volume_uuid")
.

View File

@@ -167,7 +167,7 @@ poly_user_shell_get() {
}
poly_user_shell_set() {
_sudo "in order to give $1 a safe home directory" \
_sudo "in order to give $1 a safe shell" \
/usr/bin/dscl . -create "/Users/$1" "UserShell" "$2"
}

View File

@@ -23,10 +23,10 @@ readonly RED='\033[31m'
# installer allows overriding build user count to speed up installation
# as creating each user takes non-trivial amount of time on macos
readonly NIX_USER_COUNT=${NIX_USER_COUNT:-32}
readonly NIX_BUILD_GROUP_ID="30000"
readonly NIX_BUILD_GROUP_ID="${NIX_BUILD_GROUP_ID:-30000}"
readonly NIX_BUILD_GROUP_NAME="nixbld"
# darwin installer needs to override these
NIX_FIRST_BUILD_UID="30001"
NIX_FIRST_BUILD_UID="${NIX_FIRST_BUILD_UID:-30001}"
NIX_BUILD_USER_NAME_TEMPLATE="nixbld%d"
# Please don't change this. We don't support it, because the
# default shell profile that comes with Nix doesn't support it.
@@ -37,6 +37,19 @@ readonly PROFILE_TARGETS=("/etc/bashrc" "/etc/profile.d/nix.sh" "/etc/zshrc" "/e
readonly PROFILE_BACKUP_SUFFIX=".backup-before-nix"
readonly PROFILE_NIX_FILE="$NIX_ROOT/var/nix/profiles/default/etc/profile.d/nix-daemon.sh"
# Fish has different syntax than zsh/bash, treat it separate
readonly PROFILE_FISH_SUFFIX="conf.d/nix.fish"
readonly PROFILE_FISH_PREFIXES=(
# each of these are common values of $__fish_sysconf_dir,
# under which Fish will look for a file named
# $PROFILE_FISH_SUFFIX.
"/etc/fish" # standard
"/usr/local/etc/fish" # their installer .pkg for macOS
"/opt/homebrew/etc/fish" # homebrew
"/opt/local/etc/fish" # macports
)
readonly PROFILE_NIX_FILE_FISH="$NIX_ROOT/var/nix/profiles/default/etc/profile.d/nix-daemon.fish"
readonly NIX_INSTALLED_NIX="@nix@"
readonly NIX_INSTALLED_CACERT="@cacert@"
#readonly NIX_INSTALLED_NIX="/nix/store/j8dbv5w6jl34caywh2ygdy88knx1mdf7-nix-2.3.6"
@@ -59,6 +72,30 @@ headless() {
fi
}
is_root() {
if [ "$EUID" -eq 0 ]; then
return 0
else
return 1
fi
}
is_os_linux() {
if [ "$(uname -s)" = "Linux" ]; then
return 0
else
return 1
fi
}
is_os_darwin() {
if [ "$(uname -s)" = "Darwin" ]; then
return 0
else
return 1
fi
}
contact_us() {
echo "You can open an issue at https://github.com/nixos/nix/issues"
echo ""
@@ -313,14 +350,23 @@ __sudo() {
_sudo() {
local expl="$1"
shift
if ! headless; then
if ! headless || is_root; then
__sudo "$expl" "$*" >&2
fi
sudo "$@"
if is_root; then
env "$@"
else
sudo "$@"
fi
}
# Ensure that $TMPDIR exists if defined.
if [[ -n "${TMPDIR:-}" ]] && [[ ! -d "${TMPDIR:-}" ]]; then
mkdir -m 0700 -p "${TMPDIR:-}"
fi
readonly SCRATCH=$(mktemp -d "${TMPDIR:-/tmp/}tmp.XXXXXXXXXX")
readonly SCRATCH=$(mktemp -d)
finish_cleanup() {
rm -rf "$SCRATCH"
}
@@ -329,7 +375,7 @@ finish_fail() {
finish_cleanup
failure <<EOF
Jeeze, something went wrong. If you can take all the output and open
Oh no, something went wrong. If you can take all the output and open
an issue, we'd love to fix the problem so nobody else has this issue.
:(
@@ -423,6 +469,18 @@ EOF
fi
done
if is_os_linux && [ ! -e /run/systemd/system ]; then
warning <<EOF
We did not detect systemd on your system. With a multi-user install
without systemd you will have to manually configure your init system to
launch the Nix daemon after installation.
EOF
if ! ui_confirm "Do you want to proceed with a multi-user installation?"; then
failure <<EOF
You have aborted the installation.
EOF
fi
fi
}
setup_report() {
@@ -626,6 +684,17 @@ place_channel_configuration() {
fi
}
check_selinux() {
if command -v getenforce > /dev/null 2>&1; then
if [ "$(getenforce)" = "Enforcing" ]; then
failure <<EOF
Nix does not work with selinux enabled yet!
see https://github.com/NixOS/nix/issues/2374
EOF
fi
fi
}
welcome_to_nix() {
ok "Welcome to the Multi-User Nix Installation"
@@ -739,7 +808,7 @@ install_from_extracted_nix() {
cd "$EXTRACTED_NIX_PATH"
_sudo "to copy the basic Nix files to the new store at $NIX_ROOT/store" \
cp -RLp ./store/* "$NIX_ROOT/store/"
cp -RPp ./store/* "$NIX_ROOT/store/"
_sudo "to make the new store non-writable at $NIX_ROOT/store" \
chmod -R ugo-w "$NIX_ROOT/store/"
@@ -754,7 +823,7 @@ EOF
fi
_sudo "to load data for the first time in to the Nix Database" \
"$NIX_INSTALLED_NIX/bin/nix-store" --load-db < ./.reginfo
HOME="$ROOT_HOME" "$NIX_INSTALLED_NIX/bin/nix-store" --load-db < ./.reginfo
echo " Just finished getting the nix database ready."
)
@@ -772,6 +841,19 @@ fi
EOF
}
# Fish has differing syntax
fish_source_lines() {
cat <<EOF
# Nix
if test -e '$PROFILE_NIX_FILE_FISH'
. '$PROFILE_NIX_FILE_FISH'
end
# End Nix
EOF
}
configure_shell_profile() {
task "Setting up shell profiles: ${PROFILE_TARGETS[*]}"
for profile_target in "${PROFILE_TARGETS[@]}"; do
@@ -793,6 +875,27 @@ configure_shell_profile() {
tee -a "$profile_target"
fi
done
task "Setting up shell profiles for Fish with with ${PROFILE_FISH_SUFFIX} inside ${PROFILE_FISH_PREFIXES[*]}"
for fish_prefix in "${PROFILE_FISH_PREFIXES[@]}"; do
if [ ! -d "$fish_prefix" ]; then
# this specific prefix (ie: /etc/fish) is very likely to exist
# if Fish is installed with this sysconfdir.
continue
fi
profile_target="${fish_prefix}/${PROFILE_FISH_SUFFIX}"
conf_dir=$(dirname "$profile_target")
if [ ! -d "$conf_dir" ]; then
_sudo "create $conf_dir for our Fish hook" \
mkdir "$conf_dir"
fi
fish_source_lines \
| _sudo "write nix-daemon settings to $profile_target" \
tee "$profile_target"
done
# TODO: should we suggest '. $PROFILE_NIX_FILE'? It would get them on
# their way less disruptively, but a counter-argument is that they won't
# immediately notice if something didn't get set up right?
@@ -842,22 +945,14 @@ EOF
install -m 0664 "$SCRATCH/nix.conf" /etc/nix/nix.conf
}
main() {
# TODO: I've moved this out of validate_starting_assumptions so we
# can fail faster in this case. Sourcing install-darwin... now runs
# `touch /` to detect Read-only root, but it could update times on
# pre-Catalina macOS if run as root user.
if [ "$EUID" -eq 0 ]; then
failure <<EOF
Please do not run this script with root privileges. I will call sudo
when I need to.
EOF
fi
if [ "$(uname -s)" = "Darwin" ]; then
main() {
check_selinux
if is_os_darwin; then
# shellcheck source=./install-darwin-multi-user.sh
. "$EXTRACTED_NIX_PATH/install-darwin-multi-user.sh"
elif [ "$(uname -s)" = "Linux" ]; then
elif is_os_linux; then
# shellcheck source=./install-systemd-multi-user.sh
. "$EXTRACTED_NIX_PATH/install-systemd-multi-user.sh" # most of this works on non-systemd distros also
else
@@ -865,7 +960,10 @@ EOF
fi
welcome_to_nix
chat_about_sudo
if ! is_root; then
chat_about_sudo
fi
cure_artifacts
# TODO: there's a tension between cure and validate. I moved the

View File

@@ -148,7 +148,9 @@ if ! [ -w "$dest" ]; then
exit 1
fi
mkdir -p "$dest/store"
# The auto-chroot code in openFromNonUri() checks for the
# non-existence of /nix/var/nix, so we need to create it here.
mkdir -p "$dest/store" "$dest/var/nix"
printf "copying Nix to %s..." "${dest}/store" >&2
# Insert a newline if no progress is shown.
@@ -207,31 +209,50 @@ if [ -z "$NIX_INSTALLER_NO_CHANNEL_ADD" ]; then
fi
added=
p=$HOME/.nix-profile/etc/profile.d/nix.sh
p=
p_sh=$HOME/.nix-profile/etc/profile.d/nix.sh
p_fish=$HOME/.nix-profile/etc/profile.d/nix.fish
if [ -z "$NIX_INSTALLER_NO_MODIFY_PROFILE" ]; then
# Make the shell source nix.sh during login.
for i in .bash_profile .bash_login .profile; do
fn="$HOME/$i"
if [ -w "$fn" ]; then
if ! grep -q "$p" "$fn"; then
if ! grep -q "$p_sh" "$fn"; then
echo "modifying $fn..." >&2
printf '\nif [ -e %s ]; then . %s; fi # added by Nix installer\n' "$p" "$p" >> "$fn"
printf '\nif [ -e %s ]; then . %s; fi # added by Nix installer\n' "$p_sh" "$p_sh" >> "$fn"
fi
added=1
p=${p_sh}
break
fi
done
for i in .zshenv .zshrc; do
fn="$HOME/$i"
if [ -w "$fn" ]; then
if ! grep -q "$p" "$fn"; then
if ! grep -q "$p_sh" "$fn"; then
echo "modifying $fn..." >&2
printf '\nif [ -e %s ]; then . %s; fi # added by Nix installer\n' "$p" "$p" >> "$fn"
printf '\nif [ -e %s ]; then . %s; fi # added by Nix installer\n' "$p_sh" "$p_sh" >> "$fn"
fi
added=1
p=${p_sh}
break
fi
done
if [ -d "$HOME/.config/fish" ]; then
fishdir=$HOME/.config/fish/conf.d
if [ ! -d "$fishdir" ]; then
mkdir -p "$fishdir"
fi
fn="$fishdir/nix.fish"
echo "placing $fn..." >&2
printf '\nif test -e %s; . %s; end # added by Nix installer\n' "$p_fish" "$p_fish" > "$fn"
added=1
p=${p_fish}
fi
else
p=${p_sh}
fi
if [ -z "$added" ]; then

View File

@@ -9,6 +9,8 @@ readonly SERVICE_DEST=/etc/systemd/system/nix-daemon.service
readonly SOCKET_SRC=/lib/systemd/system/nix-daemon.socket
readonly SOCKET_DEST=/etc/systemd/system/nix-daemon.socket
readonly TMPFILES_SRC=/lib/tmpfiles.d/nix-daemon.conf
readonly TMPFILES_DEST=/etc/tmpfiles.d/nix-daemon.conf
# Path for the systemd override unit file to contain the proxy settings
readonly SERVICE_OVERRIDE=${SERVICE_DEST}.d/override.conf
@@ -83,6 +85,13 @@ EOF
poly_configure_nix_daemon_service() {
if [ -e /run/systemd/system ]; then
task "Setting up the nix-daemon systemd service"
_sudo "to create the nix-daemon tmpfiles config" \
ln -sfn /nix/var/nix/profiles/default/$TMPFILES_SRC $TMPFILES_DEST
_sudo "to run systemd-tmpfiles once to pick that path up" \
systemd-tmpfiles --create --prefix=/nix/var/nix
_sudo "to set up the nix-daemon service" \
systemctl link "/nix/var/nix/profiles/default$SERVICE_SRC"

View File

@@ -40,12 +40,12 @@ case "$(uname -s).$(uname -m)" in
path=@tarballPath_aarch64-linux@
system=aarch64-linux
;;
Linux.armv6l_linux)
Linux.armv6l)
hash=@tarballHash_armv6l-linux@
path=@tarballPath_armv6l-linux@
system=armv6l-linux
;;
Linux.armv7l_linux)
Linux.armv7l)
hash=@tarballHash_armv7l-linux@
path=@tarballPath_armv7l-linux@
system=armv7l-linux
@@ -82,7 +82,7 @@ if [ "$(uname -s)" != "Darwin" ]; then
fi
if command -v curl > /dev/null 2>&1; then
fetch() { curl -L "$1" -o "$2"; }
fetch() { curl --fail -L "$1" -o "$2"; }
elif command -v wget > /dev/null 2>&1; then
fetch() { wget "$1" -O "$2"; }
else

View File

@@ -6,6 +6,8 @@ noinst-scripts += $(nix_noinst_scripts)
profiledir = $(sysconfdir)/profile.d
$(eval $(call install-file-as, $(d)/nix-profile.sh, $(profiledir)/nix.sh, 0644))
$(eval $(call install-file-as, $(d)/nix-profile.fish, $(profiledir)/nix.fish, 0644))
$(eval $(call install-file-as, $(d)/nix-profile-daemon.sh, $(profiledir)/nix-daemon.sh, 0644))
$(eval $(call install-file-as, $(d)/nix-profile-daemon.fish, $(profiledir)/nix-daemon.fish, 0644))
clean-files += $(nix_noinst_scripts)

View File

@@ -0,0 +1,35 @@
# Only execute this file once per shell.
if test -n "$__ETC_PROFILE_NIX_SOURCED"
exit
end
set __ETC_PROFILE_NIX_SOURCED 1
set --export NIX_PROFILES "@localstatedir@/nix/profiles/default $HOME/.nix-profile"
# Set $NIX_SSL_CERT_FILE so that Nixpkgs applications like curl work.
if test -n "$NIX_SSH_CERT_FILE"
: # Allow users to override the NIX_SSL_CERT_FILE
else if test -e /etc/ssl/certs/ca-certificates.crt # NixOS, Ubuntu, Debian, Gentoo, Arch
set --export NIX_SSL_CERT_FILE /etc/ssl/certs/ca-certificates.crt
else if test -e /etc/ssl/ca-bundle.pem # openSUSE Tumbleweed
set --export NIX_SSL_CERT_FILE /etc/ssl/ca-bundle.pem
else if test -e /etc/ssl/certs/ca-bundle.crt # Old NixOS
set --export NIX_SSL_CERT_FILE /etc/ssl/certs/ca-bundle.crt
else if test -e /etc/pki/tls/certs/ca-bundle.crt # Fedora, CentOS
set --export NIX_SSL_CERT_FILE /etc/pki/tls/certs/ca-bundle.crt
else if test -e "$NIX_LINK/etc/ssl/certs/ca-bundle.crt" # fall back to cacert in Nix profile
set --export NIX_SSL_CERT_FILE "$NIX_LINK/etc/ssl/certs/ca-bundle.crt"
else if test -e "$NIX_LINK/etc/ca-bundle.crt" # old cacert in Nix profile
set --export NIX_SSL_CERT_FILE "$NIX_LINK/etc/ca-bundle.crt"
else
# Fall back to what is in the nix profiles, favouring whatever is defined last.
for i in $NIX_PROFILES
if test -e "$i/etc/ssl/certs/ca-bundle.crt"
set --export NIX_SSL_CERT_FILE "$i/etc/ssl/certs/ca-bundle.crt"
end
end
end
fish_add_path --prepend --global "@localstatedir@/nix/profiles/default/bin"
fish_add_path --prepend --global "$HOME/.nix-profile/bin"

View File

@@ -0,0 +1,35 @@
if test -n "$HOME" && test -n "$USER"
# Set up the per-user profile.
set NIX_LINK $HOME/.nix-profile
# Set up environment.
# This part should be kept in sync with nixpkgs:nixos/modules/programs/environment.nix
set --export NIX_PROFILES "@localstatedir@/nix/profiles/default $HOME/.nix-profile"
# Set $NIX_SSL_CERT_FILE so that Nixpkgs applications like curl work.
if test -n "$NIX_SSH_CERT_FILE"
: # Allow users to override the NIX_SSL_CERT_FILE
else if test -e /etc/ssl/certs/ca-certificates.crt # NixOS, Ubuntu, Debian, Gentoo, Arch
set --export NIX_SSL_CERT_FILE /etc/ssl/certs/ca-certificates.crt
else if test -e /etc/ssl/ca-bundle.pem # openSUSE Tumbleweed
set --export NIX_SSL_CERT_FILE /etc/ssl/ca-bundle.pem
else if test -e /etc/ssl/certs/ca-bundle.crt # Old NixOS
set --export NIX_SSL_CERT_FILE /etc/ssl/certs/ca-bundle.crt
else if test -e /etc/pki/tls/certs/ca-bundle.crt # Fedora, CentOS
set --export NIX_SSL_CERT_FILE /etc/pki/tls/certs/ca-bundle.crt
else if test -e "$NIX_LINK/etc/ssl/certs/ca-bundle.crt" # fall back to cacert in Nix profile
set --export NIX_SSL_CERT_FILE "$NIX_LINK/etc/ssl/certs/ca-bundle.crt"
else if test -e "$NIX_LINK/etc/ca-bundle.crt" # old cacert in Nix profile
set --export NIX_SSL_CERT_FILE "$NIX_LINK/etc/ca-bundle.crt"
end
# Only use MANPATH if it is already set. In general `man` will just simply
# pick up `.nix-profile/share/man` because is it close to `.nix-profile/bin`
# which is in the $PATH. For more info, run `manpath -d`.
set --export --prepend --path MANPATH "$NIX_LINK/share/man"
fish_add_path --prepend --global "$NIX_LINK/bin"
set --erase NIX_LINK
end

View File

@@ -1,7 +1,6 @@
if [ -n "$HOME" ] && [ -n "$USER" ]; then
# Set up the per-user profile.
# This part should be kept in sync with nixpkgs:nixos/modules/programs/shell.nix
NIX_LINK=$HOME/.nix-profile

View File

@@ -14,6 +14,7 @@
#include "pathlocks.hh"
#include "globals.hh"
#include "serialise.hh"
#include "build-result.hh"
#include "store-api.hh"
#include "derivations.hh"
#include "local-store.hh"
@@ -32,7 +33,7 @@ std::string escapeUri(std::string uri)
return uri;
}
static string currentLoad;
static std::string currentLoad;
static AutoCloseFD openSlotLock(const Machine & m, uint64_t slot)
{
@@ -97,7 +98,7 @@ static int main_build_remote(int argc, char * * argv)
}
std::optional<StorePath> drvPath;
string storeUri;
std::string storeUri;
while (true) {
@@ -183,7 +184,7 @@ static int main_build_remote(int argc, char * * argv)
else
{
// build the hint template.
string errorText =
std::string errorText =
"Failed to find a machine for remote build!\n"
"derivation: %s\nrequired (system, features): (%s, %s)";
errorText += "\n%s available machines:";
@@ -193,7 +194,7 @@ static int main_build_remote(int argc, char * * argv)
errorText += "\n(%s, %s, %s, %s)";
// add the template values.
string drvstr;
std::string drvstr;
if (drvPath.has_value())
drvstr = drvPath->to_string();
else
@@ -208,7 +209,7 @@ static int main_build_remote(int argc, char * * argv)
for (auto & m : machines)
error
% concatStringsSep<std::vector<string>>(", ", m.systemTypes)
% concatStringsSep<std::vector<std::string>>(", ", m.systemTypes)
% m.maxJobs
% concatStringsSep<StringSet>(", ", m.supportedFeatures)
% concatStringsSep<StringSet>(", ", m.mandatoryFeatures);
@@ -299,7 +300,7 @@ connected:
std::set<Realisation> missingRealisations;
StorePathSet missingPaths;
if (settings.isExperimentalFeatureEnabled(Xp::CaDerivations) && !derivationHasKnownOutputPaths(drv.type())) {
if (settings.isExperimentalFeatureEnabled(Xp::CaDerivations) && !drv.type().hasKnownOutputPaths()) {
for (auto & outputName : wantedOutputs) {
auto thisOutputHash = outputHashes.at(outputName);
auto thisOutputId = DrvOutput{ thisOutputHash, outputName };

View File

@@ -86,6 +86,11 @@ ref<Store> CopyCommand::getDstStore()
EvalCommand::EvalCommand()
{
addFlag({
.longName = "debugger",
.description = "start an interactive environment if evaluation fails",
.handler = {&startReplOnEvalErrors, true},
});
}
EvalCommand::~EvalCommand()
@@ -103,7 +108,7 @@ ref<Store> EvalCommand::getEvalStore()
ref<EvalState> EvalCommand::getEvalState()
{
if (!evalState)
if (!evalState) {
evalState =
#if HAVE_BOEHMGC
std::allocate_shared<EvalState>(traceable_allocator<EvalState>(),
@@ -113,6 +118,11 @@ ref<EvalState> EvalCommand::getEvalState()
searchPath, getEvalStore(), getStore())
#endif
;
if (startReplOnEvalErrors) {
evalState->debugRepl = &runRepl;
};
}
return ref<EvalState>(evalState);
}
@@ -153,7 +163,7 @@ void BuiltPathsCommand::run(ref<Store> store)
for (auto & p : store->queryAllValidPaths())
paths.push_back(BuiltPath::Opaque{p});
} else {
paths = toBuiltPaths(getEvalStore(), store, realiseMode, operateOn, installables);
paths = Installable::toBuiltPaths(getEvalStore(), store, realiseMode, operateOn, installables);
if (recursive) {
// XXX: This only computes the store path closure, ignoring
// intermediate realisations
@@ -197,16 +207,17 @@ void StorePathCommand::run(ref<Store> store, std::vector<StorePath> && storePath
run(store, *storePaths.begin());
}
Strings editorFor(const Pos & pos)
Strings editorFor(const Path & file, uint32_t line)
{
auto editor = getEnv("EDITOR").value_or("cat");
auto args = tokenizeString<Strings>(editor);
if (pos.line > 0 && (
if (line > 0 && (
editor.find("emacs") != std::string::npos ||
editor.find("nano") != std::string::npos ||
editor.find("vim") != std::string::npos))
args.push_back(fmt("+%d", pos.line));
args.push_back(pos.file);
editor.find("vim") != std::string::npos ||
editor.find("kak") != std::string::npos))
args.push_back(fmt("+%d", line));
args.push_back(file);
return args;
}

View File

@@ -5,7 +5,6 @@
#include "common-eval-args.hh"
#include "path.hh"
#include "flake/lockfile.hh"
#include "store-api.hh"
#include <optional>
@@ -58,6 +57,9 @@ struct CopyCommand : virtual StoreCommand
struct EvalCommand : virtual StoreCommand, MixEvalArgs
{
bool startReplOnEvalErrors = false;
bool ignoreExceptionsDuringTry = false;
EvalCommand();
~EvalCommand();
@@ -76,29 +78,28 @@ struct MixFlakeOptions : virtual Args, EvalCommand
{
flake::LockFlags lockFlags;
std::optional<std::string> needsFlakeInputCompletion = {};
MixFlakeOptions();
virtual std::optional<FlakeRef> getFlakeRefForCompletion()
virtual std::vector<std::string> getFlakesForCompletion()
{ return {}; }
};
/* How to handle derivations in commands that operate on store paths. */
enum class OperateOn {
/* Operate on the output path. */
Output,
/* Operate on the .drv path. */
Derivation
void completeFlakeInput(std::string_view prefix);
void completionHook() override;
};
struct SourceExprCommand : virtual Args, MixFlakeOptions
{
std::optional<Path> file;
std::optional<std::string> expr;
bool readOnlyMode = false;
// FIXME: move this; not all commands (e.g. 'nix run') use it.
OperateOn operateOn = OperateOn::Output;
SourceExprCommand();
SourceExprCommand(bool supportReadOnlyMode = false);
std::vector<std::shared_ptr<Installable>> parseInstallables(
ref<Store> store, std::vector<std::string> ss);
@@ -113,19 +114,6 @@ struct SourceExprCommand : virtual Args, MixFlakeOptions
void completeInstallable(std::string_view prefix);
};
enum class Realise {
/* Build the derivation. Postcondition: the
derivation outputs exist. */
Outputs,
/* Don't build the derivation. Postcondition: the store derivation
exists. */
Derivation,
/* Evaluate in dry-run mode. Postcondition: nothing. */
// FIXME: currently unused, but could be revived if we can
// evaluate derivations in-memory.
Nothing
};
/* A command that operates on a list of "installables", which can be
store paths, attribute paths, Nix expressions, etc. */
struct InstallablesCommand : virtual Args, SourceExprCommand
@@ -135,12 +123,13 @@ struct InstallablesCommand : virtual Args, SourceExprCommand
InstallablesCommand();
void prepare() override;
Installables load();
virtual bool useDefaultInstallables() { return true; }
std::optional<FlakeRef> getFlakeRefForCompletion() override;
std::vector<std::string> getFlakesForCompletion() override;
private:
protected:
std::vector<std::string> _installables;
};
@@ -150,13 +139,13 @@ struct InstallableCommand : virtual Args, SourceExprCommand
{
std::shared_ptr<Installable> installable;
InstallableCommand();
InstallableCommand(bool supportReadOnlyMode = false);
void prepare() override;
std::optional<FlakeRef> getFlakeRefForCompletion() override
std::vector<std::string> getFlakesForCompletion() override
{
return parseFlakeRef(_installable, absPath("."));
return {_installable};
}
private:
@@ -238,41 +227,9 @@ static RegisterCommand registerCommand2(std::vector<std::string> && name)
return RegisterCommand(std::move(name), [](){ return make_ref<T>(); });
}
BuiltPaths build(
ref<Store> evalStore,
ref<Store> store, Realise mode,
const std::vector<std::shared_ptr<Installable>> & installables,
BuildMode bMode = bmNormal);
std::set<StorePath> toStorePaths(
ref<Store> evalStore,
ref<Store> store,
Realise mode,
OperateOn operateOn,
const std::vector<std::shared_ptr<Installable>> & installables);
StorePath toStorePath(
ref<Store> evalStore,
ref<Store> store,
Realise mode,
OperateOn operateOn,
std::shared_ptr<Installable> installable);
std::set<StorePath> toDerivations(
ref<Store> store,
const std::vector<std::shared_ptr<Installable>> & installables,
bool useDeriver = false);
BuiltPaths toBuiltPaths(
ref<Store> evalStore,
ref<Store> store,
Realise mode,
OperateOn operateOn,
const std::vector<std::shared_ptr<Installable>> & installables);
/* Helper function to generate args that invoke $EDITOR on
filename:lineno. */
Strings editorFor(const Pos & pos);
Strings editorFor(const Path & file, uint32_t line);
struct MixProfile : virtual StoreCommand
{
@@ -323,4 +280,8 @@ void printClosureDiff(
const StorePath & afterPath,
std::string_view indent);
void runRepl(
ref<EvalState> evalState,
const ValMap & extraEnv);
}

View File

@@ -7,6 +7,7 @@
#include "registry.hh"
#include "flake/flakeref.hh"
#include "store-api.hh"
#include "command.hh"
namespace nix {
@@ -59,6 +60,9 @@ MixEvalArgs::MixEvalArgs()
fetchers::Attrs extraAttrs;
if (to.subdir != "") extraAttrs["dir"] = to.subdir;
fetchers::overrideRegistry(from.input, to.input, extraAttrs);
}},
.completer = {[&](size_t, std::string_view prefix) {
completeFlakeRef(openStore(), prefix);
}}
});
@@ -77,7 +81,7 @@ Bindings * MixEvalArgs::getAutoArgs(EvalState & state)
for (auto & i : autoArgs) {
auto v = state.allocValue();
if (i.second[0] == 'E')
state.mkThunk_(*v, state.parseExprFromString(string(i.second, 1), absPath(".")));
state.mkThunk_(*v, state.parseExprFromString(i.second.substr(1), absPath(".")));
else
v->mkString(((std::string_view) i.second).substr(1));
res.insert(state.symbols.create(i.first), v);
@@ -85,17 +89,17 @@ Bindings * MixEvalArgs::getAutoArgs(EvalState & state)
return res.finish();
}
Path lookupFileArg(EvalState & state, string s)
Path lookupFileArg(EvalState & state, std::string_view s)
{
if (isUri(s)) {
return state.store->toRealPath(
fetchers::downloadTarball(
state.store, resolveUri(s), "source", false).first.storePath);
} else if (s.size() > 2 && s.at(0) == '<' && s.at(s.size() - 1) == '>') {
Path p = s.substr(1, s.size() - 2);
Path p(s.substr(1, s.size() - 2));
return state.findFile(p);
} else
return absPath(s);
return absPath(std::string(s));
}
}

View File

@@ -22,6 +22,6 @@ private:
std::map<std::string, std::string> autoArgs;
};
Path lookupFileArg(EvalState & state, string s);
Path lookupFileArg(EvalState & state, std::string_view s);
}

View File

@@ -1,4 +1,6 @@
#include "globals.hh"
#include "installables.hh"
#include "util.hh"
#include "command.hh"
#include "attr-path.hh"
#include "common-eval-args.hh"
@@ -12,6 +14,7 @@
#include "eval-cache.hh"
#include "url.hh"
#include "registry.hh"
#include "build-result.hh"
#include <regex>
#include <queue>
@@ -20,17 +23,6 @@
namespace nix {
void completeFlakeInputPath(
ref<EvalState> evalState,
const FlakeRef & flakeRef,
std::string_view prefix)
{
auto flake = flake::getFlake(*evalState, flakeRef, true);
for (auto & input : flake.inputs)
if (hasPrefix(input.first, prefix))
completions->add(input.first);
}
MixFlakeOptions::MixFlakeOptions()
{
auto category = "Common flake-related options";
@@ -83,8 +75,7 @@ MixFlakeOptions::MixFlakeOptions()
lockFlags.inputUpdates.insert(flake::parseInputPath(s));
}},
.completer = {[&](size_t, std::string_view prefix) {
if (auto flakeRef = getFlakeRefForCompletion())
completeFlakeInputPath(getEvalState(), *flakeRef, prefix);
needsFlakeInputCompletion = {std::string(prefix)};
}}
});
@@ -98,6 +89,12 @@ MixFlakeOptions::MixFlakeOptions()
lockFlags.inputOverrides.insert_or_assign(
flake::parseInputPath(inputPath),
parseFlakeRef(flakeRef, absPath("."), true));
}},
.completer = {[&](size_t n, std::string_view prefix) {
if (n == 0)
needsFlakeInputCompletion = {std::string(prefix)};
else if (n == 1)
completeFlakeRef(getEvalState()->store, prefix);
}}
});
@@ -128,12 +125,33 @@ MixFlakeOptions::MixFlakeOptions()
});
}
SourceExprCommand::SourceExprCommand()
void MixFlakeOptions::completeFlakeInput(std::string_view prefix)
{
auto evalState = getEvalState();
for (auto & flakeRefS : getFlakesForCompletion()) {
auto flakeRef = parseFlakeRefWithFragment(expandTilde(flakeRefS), absPath(".")).first;
auto flake = flake::getFlake(*evalState, flakeRef, true);
for (auto & input : flake.inputs)
if (hasPrefix(input.first, prefix))
completions->add(input.first);
}
}
void MixFlakeOptions::completionHook()
{
if (auto & prefix = needsFlakeInputCompletion)
completeFlakeInput(*prefix);
}
SourceExprCommand::SourceExprCommand(bool supportReadOnlyMode)
{
addFlag({
.longName = "file",
.shortName = 'f',
.description = "Interpret installables as attribute paths relative to the Nix expression stored in *file*.",
.description =
"Interpret installables as attribute paths relative to the Nix expression stored in *file*. "
"If *file* is the character -, then a Nix expression will be read from standard input. "
"Implies `--impure`.",
.category = installablesCategory,
.labels = {"file"},
.handler = {&file},
@@ -154,11 +172,25 @@ SourceExprCommand::SourceExprCommand()
.category = installablesCategory,
.handler = {&operateOn, OperateOn::Derivation},
});
if (supportReadOnlyMode) {
addFlag({
.longName = "read-only",
.description =
"Do not instantiate each evaluated derivation. "
"This improves performance, but can cause errors when accessing "
"store paths of derivations during evaluation.",
.handler = {&readOnlyMode, true},
});
}
}
Strings SourceExprCommand::getDefaultFlakeAttrPaths()
{
return {"defaultPackage." + settings.thisSystem.get()};
return {
"packages." + settings.thisSystem.get() + ".default",
"defaultPackage." + settings.thisSystem.get()
};
}
Strings SourceExprCommand::getDefaultFlakeAttrPathPrefixes()
@@ -176,6 +208,8 @@ Strings SourceExprCommand::getDefaultFlakeAttrPathPrefixes()
void SourceExprCommand::completeInstallable(std::string_view prefix)
{
if (file) {
completionType = ctAttrs;
evalSettings.pureEval = false;
auto state = getEvalState();
Expr *e = state->parseExprFromFile(
@@ -204,13 +238,14 @@ void SourceExprCommand::completeInstallable(std::string_view prefix)
Value v2;
state->autoCallFunction(*autoArgs, v1, v2);
completionType = ctAttrs;
if (v2.type() == nAttrs) {
for (auto & i : *v2.attrs) {
std::string name = i.name;
std::string name = state->symbols[i.name];
if (name.find(searchWord) == 0) {
completions->add(i.name);
if (prefix_ == "")
completions->add(name);
else
completions->add(prefix_ + "." + name);
}
}
}
@@ -238,10 +273,11 @@ void completeFlakeRefWithFragment(
if (hash == std::string::npos) {
completeFlakeRef(evalState->store, prefix);
} else {
completionType = ctAttrs;
auto fragment = prefix.substr(hash + 1);
auto flakeRefS = std::string(prefix.substr(0, hash));
// FIXME: do tilde expansion.
auto flakeRef = parseFlakeRef(flakeRefS, absPath("."));
auto flakeRef = parseFlakeRef(expandTilde(flakeRefS), absPath("."));
auto evalCache = openEvalCache(*evalState,
std::make_shared<flake::LockedFlake>(lockFlake(*evalState, flakeRef, lockFlags)));
@@ -253,8 +289,6 @@ void completeFlakeRefWithFragment(
flake. */
attrPathPrefixes.push_back("");
completionType = ctAttrs;
for (auto & attrPathPrefixS : attrPathPrefixes) {
auto attrPathPrefix = parseAttrPath(*evalState, attrPathPrefixS);
auto attrPathS = attrPathPrefixS + std::string(fragment);
@@ -262,19 +296,19 @@ void completeFlakeRefWithFragment(
std::string lastAttr;
if (!attrPath.empty() && !hasSuffix(attrPathS, ".")) {
lastAttr = attrPath.back();
lastAttr = evalState->symbols[attrPath.back()];
attrPath.pop_back();
}
auto attr = root->findAlongAttrPath(attrPath);
if (!attr) continue;
for (auto & attr2 : attr->getAttrs()) {
if (hasPrefix(attr2, lastAttr)) {
auto attrPath2 = attr->getAttrPath(attr2);
for (auto & attr2 : (*attr)->getAttrs()) {
if (hasPrefix(evalState->symbols[attr2], lastAttr)) {
auto attrPath2 = (*attr)->getAttrPath(attr2);
/* Strip the attrpath prefix. */
attrPath2.erase(attrPath2.begin(), attrPath2.begin() + attrPathPrefix.size());
completions->add(flakeRefS + "#" + concatStringsSep(".", attrPath2));
completions->add(flakeRefS + "#" + concatStringsSep(".", evalState->symbols.resolve(attrPath2)));
}
}
}
@@ -328,16 +362,16 @@ DerivedPath Installable::toDerivedPath()
return std::move(buildables[0]);
}
std::vector<std::pair<std::shared_ptr<eval_cache::AttrCursor>, std::string>>
std::vector<ref<eval_cache::AttrCursor>>
Installable::getCursors(EvalState & state)
{
auto evalCache =
std::make_shared<nix::eval_cache::EvalCache>(std::nullopt, state,
[&]() { return toValue(state).first; });
return {{evalCache->getRoot(), ""}};
return {evalCache->getRoot()};
}
std::pair<std::shared_ptr<eval_cache::AttrCursor>, std::string>
ref<eval_cache::AttrCursor>
Installable::getCursor(EvalState & state)
{
auto cursors = getCursors(state);
@@ -411,10 +445,8 @@ DerivedPaths InstallableValue::toDerivedPaths()
// Group by derivation, helps with .all in particular
for (auto & drv : toDerivations()) {
auto outputName = drv.outputName;
if (outputName == "")
throw Error("derivation '%s' lacks an 'outputName' attribute", state->store->printStorePath(drv.drvPath));
drvsToOutputs[drv.drvPath].insert(outputName);
for (auto & outputName : drv.outputsToInstall)
drvsToOutputs[drv.drvPath].insert(outputName);
drvsToCopy.insert(drv.drvPath);
}
@@ -437,14 +469,24 @@ struct InstallableAttrPath : InstallableValue
SourceExprCommand & cmd;
RootValue v;
std::string attrPath;
OutputsSpec outputsSpec;
InstallableAttrPath(ref<EvalState> state, SourceExprCommand & cmd, Value * v, const std::string & attrPath)
: InstallableValue(state), cmd(cmd), v(allocRootValue(v)), attrPath(attrPath)
InstallableAttrPath(
ref<EvalState> state,
SourceExprCommand & cmd,
Value * v,
const std::string & attrPath,
OutputsSpec outputsSpec)
: InstallableValue(state)
, cmd(cmd)
, v(allocRootValue(v))
, attrPath(attrPath)
, outputsSpec(std::move(outputsSpec))
{ }
std::string what() const override { return attrPath; }
std::pair<Value *, Pos> toValue(EvalState & state) override
std::pair<Value *, PosIdx> toValue(EvalState & state) override
{
auto [vRes, pos] = findAlongAttrPath(state, attrPath, *cmd.getAutoArgs(state), **v);
state.forceValue(*vRes, pos);
@@ -465,10 +507,21 @@ std::vector<InstallableValue::DerivationInfo> InstallableAttrPath::toDerivations
std::vector<DerivationInfo> res;
for (auto & drvInfo : drvInfos) {
res.push_back({
state->store->parseStorePath(drvInfo.queryDrvPath()),
state->store->maybeParseStorePath(drvInfo.queryOutPath()),
drvInfo.queryOutputName()
auto drvPath = drvInfo.queryDrvPath();
if (!drvPath)
throw Error("'%s' is not a derivation", what());
std::set<std::string> outputsToInstall;
if (auto outputNames = std::get_if<OutputNames>(&outputsSpec))
outputsToInstall = *outputNames;
else
for (auto & output : drvInfo.queryOutputs(false, std::get_if<DefaultOutputs>(&outputsSpec)))
outputsToInstall.insert(output.first);
res.push_back(DerivationInfo {
.drvPath = *drvPath,
.outputsToInstall = std::move(outputsToInstall)
});
}
@@ -546,6 +599,7 @@ InstallableFlake::InstallableFlake(
ref<EvalState> state,
FlakeRef && flakeRef,
std::string_view fragment,
OutputsSpec outputsSpec,
Strings attrPaths,
Strings prefixes,
const flake::LockFlags & lockFlags)
@@ -553,6 +607,7 @@ InstallableFlake::InstallableFlake(
flakeRef(flakeRef),
attrPaths(fragment == "" ? attrPaths : Strings{(std::string) fragment}),
prefixes(fragment == "" ? Strings{} : prefixes),
outputsSpec(std::move(outputsSpec)),
lockFlags(lockFlags)
{
if (cmd && cmd->getAutoArgs(*state)->size())
@@ -561,37 +616,55 @@ InstallableFlake::InstallableFlake(
std::tuple<std::string, FlakeRef, InstallableValue::DerivationInfo> InstallableFlake::toDerivation()
{
auto lockedFlake = getLockedFlake();
Activity act(*logger, lvlTalkative, actUnknown, fmt("evaluating derivation '%s'", what()));
auto cache = openEvalCache(*state, lockedFlake);
auto root = cache->getRoot();
auto attr = getCursor(*state);
for (auto & attrPath : getActualAttrPaths()) {
debug("trying flake output attribute '%s'", attrPath);
auto attrPath = attr->getAttrPathStr();
auto attr = root->findAlongAttrPath(
parseAttrPath(*state, attrPath),
true
);
if (!attr->isDerivation())
throw Error("flake output attribute '%s' is not a derivation", attrPath);
if (!attr) continue;
auto drvPath = attr->forceDerivation();
if (!attr->isDerivation())
throw Error("flake output attribute '%s' is not a derivation", attrPath);
std::set<std::string> outputsToInstall;
std::optional<NixInt> priority;
auto drvPath = attr->forceDerivation();
auto drvInfo = DerivationInfo{
std::move(drvPath),
state->store->maybeParseStorePath(attr->getAttr(state->sOutPath)->getString()),
attr->getAttr(state->sOutputName)->getString()
};
return {attrPath, lockedFlake->flake.lockedRef, std::move(drvInfo)};
if (auto aOutputSpecified = attr->maybeGetAttr(state->sOutputSpecified)) {
if (aOutputSpecified->getBool()) {
if (auto aOutputName = attr->maybeGetAttr("outputName"))
outputsToInstall = { aOutputName->getString() };
}
}
throw Error("flake '%s' does not provide attribute %s",
flakeRef, showAttrPaths(getActualAttrPaths()));
else if (auto aMeta = attr->maybeGetAttr(state->sMeta)) {
if (auto aOutputsToInstall = aMeta->maybeGetAttr("outputsToInstall"))
for (auto & s : aOutputsToInstall->getListOfStrings())
outputsToInstall.insert(s);
if (auto aPriority = aMeta->maybeGetAttr("priority"))
priority = aPriority->getInt();
}
if (outputsToInstall.empty() || std::get_if<AllOutputs>(&outputsSpec)) {
outputsToInstall.clear();
if (auto aOutputs = attr->maybeGetAttr(state->sOutputs))
for (auto & s : aOutputs->getListOfStrings())
outputsToInstall.insert(s);
}
if (outputsToInstall.empty())
outputsToInstall.insert("out");
if (auto outputNames = std::get_if<OutputNames>(&outputsSpec))
outputsToInstall = *outputNames;
auto drvInfo = DerivationInfo {
.drvPath = std::move(drvPath),
.outputsToInstall = std::move(outputsToInstall),
.priority = priority,
};
return {attrPath, getLockedFlake()->flake.lockedRef, std::move(drvInfo)};
}
std::vector<InstallableValue::DerivationInfo> InstallableFlake::toDerivations()
@@ -601,28 +674,12 @@ std::vector<InstallableValue::DerivationInfo> InstallableFlake::toDerivations()
return res;
}
std::pair<Value *, Pos> InstallableFlake::toValue(EvalState & state)
std::pair<Value *, PosIdx> InstallableFlake::toValue(EvalState & state)
{
auto lockedFlake = getLockedFlake();
auto vOutputs = getFlakeOutputs(state, *lockedFlake);
auto emptyArgs = state.allocBindings(0);
for (auto & attrPath : getActualAttrPaths()) {
try {
auto [v, pos] = findAlongAttrPath(state, attrPath, *emptyArgs, *vOutputs);
state.forceValue(*v, pos);
return {v, pos};
} catch (AttrPathNotFound & e) {
}
}
throw Error("flake '%s' does not provide attribute %s",
flakeRef, showAttrPaths(getActualAttrPaths()));
return {&getCursor(state)->forceValue(), noPos};
}
std::vector<std::pair<std::shared_ptr<eval_cache::AttrCursor>, std::string>>
std::vector<ref<eval_cache::AttrCursor>>
InstallableFlake::getCursors(EvalState & state)
{
auto evalCache = openEvalCache(state,
@@ -630,21 +687,55 @@ InstallableFlake::getCursors(EvalState & state)
auto root = evalCache->getRoot();
std::vector<std::pair<std::shared_ptr<eval_cache::AttrCursor>, std::string>> res;
std::vector<ref<eval_cache::AttrCursor>> res;
for (auto & attrPath : getActualAttrPaths()) {
auto attr = root->findAlongAttrPath(parseAttrPath(state, attrPath));
if (attr) res.push_back({attr, attrPath});
if (attr) res.push_back(ref(*attr));
}
return res;
}
ref<eval_cache::AttrCursor> InstallableFlake::getCursor(EvalState & state)
{
auto lockedFlake = getLockedFlake();
auto cache = openEvalCache(state, lockedFlake);
auto root = cache->getRoot();
Suggestions suggestions;
auto attrPaths = getActualAttrPaths();
for (auto & attrPath : attrPaths) {
debug("trying flake output attribute '%s'", attrPath);
auto attrOrSuggestions = root->findAlongAttrPath(
parseAttrPath(state, attrPath),
true
);
if (!attrOrSuggestions) {
suggestions += attrOrSuggestions.getSuggestions();
continue;
}
return *attrOrSuggestions;
}
throw Error(
suggestions,
"flake '%s' does not provide attribute %s",
flakeRef,
showAttrPaths(attrPaths));
}
std::shared_ptr<flake::LockedFlake> InstallableFlake::getLockedFlake() const
{
flake::LockFlags lockFlagsApplyConfig = lockFlags;
lockFlagsApplyConfig.applyNixConfig = true;
if (!_lockedFlake) {
flake::LockFlags lockFlagsApplyConfig = lockFlags;
lockFlagsApplyConfig.applyNixConfig = true;
_lockedFlake = std::make_shared<flake::LockedFlake>(lockFlake(*state, flakeRef, lockFlagsApplyConfig));
}
return _lockedFlake;
@@ -669,6 +760,10 @@ std::vector<std::shared_ptr<Installable>> SourceExprCommand::parseInstallables(
{
std::vector<std::shared_ptr<Installable>> result;
if (readOnlyMode) {
settings.readOnlyMode = true;
}
if (file || expr) {
if (file && expr)
throw UsageError("'--file' and '--expr' are exclusive");
@@ -679,15 +774,24 @@ std::vector<std::shared_ptr<Installable>> SourceExprCommand::parseInstallables(
auto state = getEvalState();
auto vFile = state->allocValue();
if (file)
if (file == "-") {
auto e = state->parseStdin();
state->eval(e, *vFile);
} else if (file)
state->evalFile(lookupFileArg(*state, *file), *vFile);
else {
auto e = state->parseExprFromString(*expr, absPath("."));
state->eval(e, *vFile);
}
for (auto & s : ss)
result.push_back(std::make_shared<InstallableAttrPath>(state, *this, vFile, s == "." ? "" : s));
for (auto & s : ss) {
auto [prefix, outputsSpec] = parseOutputsSpec(s);
result.push_back(
std::make_shared<InstallableAttrPath>(
state, *this, vFile,
prefix == "." ? "" : prefix,
outputsSpec));
}
} else {
@@ -706,12 +810,13 @@ std::vector<std::shared_ptr<Installable>> SourceExprCommand::parseInstallables(
}
try {
auto [flakeRef, fragment] = parseFlakeRefWithFragment(s, absPath("."));
auto [flakeRef, fragment, outputsSpec] = parseFlakeRefWithFragmentAndOutputsSpec(s, absPath("."));
result.push_back(std::make_shared<InstallableFlake>(
this,
getEvalState(),
std::move(flakeRef),
fragment,
outputsSpec,
getDefaultFlakeAttrPaths(),
getDefaultFlakeAttrPathPrefixes(),
lockFlags));
@@ -735,56 +840,20 @@ std::shared_ptr<Installable> SourceExprCommand::parseInstallable(
return installables.front();
}
BuiltPaths getBuiltPaths(ref<Store> evalStore, ref<Store> store, const DerivedPaths & hopefullyBuiltPaths)
BuiltPaths Installable::build(
ref<Store> evalStore,
ref<Store> store,
Realise mode,
const std::vector<std::shared_ptr<Installable>> & installables,
BuildMode bMode)
{
BuiltPaths res;
for (const auto & b : hopefullyBuiltPaths)
std::visit(
overloaded{
[&](const DerivedPath::Opaque & bo) {
res.push_back(BuiltPath::Opaque{bo.path});
},
[&](const DerivedPath::Built & bfd) {
OutputPathMap outputs;
auto drv = evalStore->readDerivation(bfd.drvPath);
auto outputHashes = staticOutputHashes(*evalStore, drv); // FIXME: expensive
auto drvOutputs = drv.outputsAndOptPaths(*store);
for (auto & output : bfd.outputs) {
if (!outputHashes.count(output))
throw Error(
"the derivation '%s' doesn't have an output named '%s'",
store->printStorePath(bfd.drvPath), output);
if (settings.isExperimentalFeatureEnabled(
Xp::CaDerivations)) {
auto outputId =
DrvOutput{outputHashes.at(output), output};
auto realisation =
store->queryRealisation(outputId);
if (!realisation)
throw Error(
"cannot operate on an output of unbuilt "
"content-addressed derivation '%s'",
outputId.to_string());
outputs.insert_or_assign(
output, realisation->outPath);
} else {
// If ca-derivations isn't enabled, assume that
// the output path is statically known.
assert(drvOutputs.count(output));
assert(drvOutputs.at(output).second);
outputs.insert_or_assign(
output, *drvOutputs.at(output).second);
}
}
res.push_back(BuiltPath::Built{bfd.drvPath, outputs});
},
},
b.raw());
for (auto & [_, builtPath] : build2(evalStore, store, mode, installables, bMode))
res.push_back(builtPath);
return res;
}
BuiltPaths build(
std::vector<std::pair<std::shared_ptr<Installable>, BuiltPath>> Installable::build2(
ref<Store> evalStore,
ref<Store> store,
Realise mode,
@@ -795,21 +864,101 @@ BuiltPaths build(
settings.readOnlyMode = true;
std::vector<DerivedPath> pathsToBuild;
std::map<DerivedPath, std::vector<std::shared_ptr<Installable>>> backmap;
for (auto & i : installables) {
auto b = i->toDerivedPaths();
pathsToBuild.insert(pathsToBuild.end(), b.begin(), b.end());
for (auto b : i->toDerivedPaths()) {
pathsToBuild.push_back(b);
backmap[b].push_back(i);
}
}
if (mode == Realise::Nothing || mode == Realise::Derivation)
printMissing(store, pathsToBuild, lvlError);
else if (mode == Realise::Outputs)
store->buildPaths(pathsToBuild, bMode, evalStore);
std::vector<std::pair<std::shared_ptr<Installable>, BuiltPath>> res;
return getBuiltPaths(evalStore, store, pathsToBuild);
switch (mode) {
case Realise::Nothing:
case Realise::Derivation:
printMissing(store, pathsToBuild, lvlError);
for (auto & path : pathsToBuild) {
for (auto & installable : backmap[path]) {
std::visit(overloaded {
[&](const DerivedPath::Built & bfd) {
OutputPathMap outputs;
auto drv = evalStore->readDerivation(bfd.drvPath);
auto outputHashes = staticOutputHashes(*evalStore, drv); // FIXME: expensive
auto drvOutputs = drv.outputsAndOptPaths(*store);
for (auto & output : bfd.outputs) {
auto outputHash = get(outputHashes, output);
if (!outputHash)
throw Error(
"the derivation '%s' doesn't have an output named '%s'",
store->printStorePath(bfd.drvPath), output);
if (settings.isExperimentalFeatureEnabled(Xp::CaDerivations)) {
DrvOutput outputId { *outputHash, output };
auto realisation = store->queryRealisation(outputId);
if (!realisation)
throw Error(
"cannot operate on an output of the "
"unbuilt derivation '%s'",
outputId.to_string());
outputs.insert_or_assign(output, realisation->outPath);
} else {
// If ca-derivations isn't enabled, assume that
// the output path is statically known.
auto drvOutput = get(drvOutputs, output);
assert(drvOutput);
assert(drvOutput->second);
outputs.insert_or_assign(
output, *drvOutput->second);
}
}
res.push_back({installable, BuiltPath::Built { bfd.drvPath, outputs }});
},
[&](const DerivedPath::Opaque & bo) {
res.push_back({installable, BuiltPath::Opaque { bo.path }});
},
}, path.raw());
}
}
break;
case Realise::Outputs: {
if (settings.printMissing)
printMissing(store, pathsToBuild, lvlInfo);
for (auto & buildResult : store->buildPathsWithResults(pathsToBuild, bMode, evalStore)) {
if (!buildResult.success())
buildResult.rethrow();
for (auto & installable : backmap[buildResult.path]) {
std::visit(overloaded {
[&](const DerivedPath::Built & bfd) {
std::map<std::string, StorePath> outputs;
for (auto & path : buildResult.builtOutputs)
outputs.emplace(path.first.outputName, path.second.outPath);
res.push_back({installable, BuiltPath::Built { bfd.drvPath, outputs }});
},
[&](const DerivedPath::Opaque & bo) {
res.push_back({installable, BuiltPath::Opaque { bo.path }});
},
}, buildResult.path.raw());
}
}
break;
}
default:
assert(false);
}
return res;
}
BuiltPaths toBuiltPaths(
BuiltPaths Installable::toBuiltPaths(
ref<Store> evalStore,
ref<Store> store,
Realise mode,
@@ -817,19 +966,19 @@ BuiltPaths toBuiltPaths(
const std::vector<std::shared_ptr<Installable>> & installables)
{
if (operateOn == OperateOn::Output)
return build(evalStore, store, mode, installables);
return Installable::build(evalStore, store, mode, installables);
else {
if (mode == Realise::Nothing)
settings.readOnlyMode = true;
BuiltPaths res;
for (auto & drvPath : toDerivations(store, installables, true))
for (auto & drvPath : Installable::toDerivations(store, installables, true))
res.push_back(BuiltPath::Opaque{drvPath});
return res;
}
}
StorePathSet toStorePaths(
StorePathSet Installable::toStorePaths(
ref<Store> evalStore,
ref<Store> store,
Realise mode, OperateOn operateOn,
@@ -843,7 +992,7 @@ StorePathSet toStorePaths(
return outPaths;
}
StorePath toStorePath(
StorePath Installable::toStorePath(
ref<Store> evalStore,
ref<Store> store,
Realise mode, OperateOn operateOn,
@@ -857,7 +1006,7 @@ StorePath toStorePath(
return *paths.begin();
}
StorePathSet toDerivations(
StorePathSet Installable::toDerivations(
ref<Store> store,
const std::vector<std::shared_ptr<Installable>> & installables,
bool useDeriver)
@@ -893,24 +1042,30 @@ InstallablesCommand::InstallablesCommand()
void InstallablesCommand::prepare()
{
if (_installables.empty() && useDefaultInstallables())
// FIXME: commands like "nix install" should not have a
// default, probably.
_installables.push_back(".");
installables = parseInstallables(getStore(), _installables);
installables = load();
}
std::optional<FlakeRef> InstallablesCommand::getFlakeRefForCompletion()
Installables InstallablesCommand::load() {
Installables installables;
if (_installables.empty() && useDefaultInstallables())
// FIXME: commands like "nix profile install" should not have a
// default, probably.
_installables.push_back(".");
return parseInstallables(getStore(), _installables);
}
std::vector<std::string> InstallablesCommand::getFlakesForCompletion()
{
if (_installables.empty()) {
if (useDefaultInstallables())
return parseFlakeRef(".", absPath("."));
return {"."};
return {};
}
return parseFlakeRef(_installables.front(), absPath("."));
return _installables;
}
InstallableCommand::InstallableCommand()
InstallableCommand::InstallableCommand(bool supportReadOnlyMode)
: SourceExprCommand(supportReadOnlyMode)
{
expectArgs({
.label = "installable",

View File

@@ -5,6 +5,7 @@
#include "path-with-outputs.hh"
#include "derived-path.hh"
#include "eval.hh"
#include "store-api.hh"
#include "flake/flake.hh"
#include <optional>
@@ -29,6 +30,27 @@ struct UnresolvedApp
App resolve(ref<Store> evalStore, ref<Store> store);
};
enum class Realise {
/* Build the derivation. Postcondition: the
derivation outputs exist. */
Outputs,
/* Don't build the derivation. Postcondition: the store derivation
exists. */
Derivation,
/* Evaluate in dry-run mode. Postcondition: nothing. */
// FIXME: currently unused, but could be revived if we can
// evaluate derivations in-memory.
Nothing
};
/* How to handle derivations in commands that operate on store paths. */
enum class OperateOn {
/* Operate on the output path. */
Output,
/* Operate on the .drv path. */
Derivation
};
struct Installable
{
virtual ~Installable() { }
@@ -46,7 +68,7 @@ struct Installable
UnresolvedApp toApp(EvalState & state);
virtual std::pair<Value *, Pos> toValue(EvalState & state)
virtual std::pair<Value *, PosIdx> toValue(EvalState & state)
{
throw Error("argument '%s' cannot be evaluated", what());
}
@@ -58,18 +80,60 @@ struct Installable
return {};
}
virtual std::vector<std::pair<std::shared_ptr<eval_cache::AttrCursor>, std::string>>
virtual std::vector<ref<eval_cache::AttrCursor>>
getCursors(EvalState & state);
std::pair<std::shared_ptr<eval_cache::AttrCursor>, std::string>
virtual ref<eval_cache::AttrCursor>
getCursor(EvalState & state);
virtual FlakeRef nixpkgsFlakeRef() const
{
return FlakeRef::fromAttrs({{"type","indirect"}, {"id", "nixpkgs"}});
}
static BuiltPaths build(
ref<Store> evalStore,
ref<Store> store,
Realise mode,
const std::vector<std::shared_ptr<Installable>> & installables,
BuildMode bMode = bmNormal);
static std::vector<std::pair<std::shared_ptr<Installable>, BuiltPath>> build2(
ref<Store> evalStore,
ref<Store> store,
Realise mode,
const std::vector<std::shared_ptr<Installable>> & installables,
BuildMode bMode = bmNormal);
static std::set<StorePath> toStorePaths(
ref<Store> evalStore,
ref<Store> store,
Realise mode,
OperateOn operateOn,
const std::vector<std::shared_ptr<Installable>> & installables);
static StorePath toStorePath(
ref<Store> evalStore,
ref<Store> store,
Realise mode,
OperateOn operateOn,
std::shared_ptr<Installable> installable);
static std::set<StorePath> toDerivations(
ref<Store> store,
const std::vector<std::shared_ptr<Installable>> & installables,
bool useDeriver = false);
static BuiltPaths toBuiltPaths(
ref<Store> evalStore,
ref<Store> store,
Realise mode,
OperateOn operateOn,
const std::vector<std::shared_ptr<Installable>> & installables);
};
typedef std::vector<std::shared_ptr<Installable>> Installables;
struct InstallableValue : Installable
{
ref<EvalState> state;
@@ -79,8 +143,8 @@ struct InstallableValue : Installable
struct DerivationInfo
{
StorePath drvPath;
std::optional<StorePath> outPath;
std::string outputName;
std::set<std::string> outputsToInstall;
std::optional<NixInt> priority;
};
virtual std::vector<DerivationInfo> toDerivations() = 0;
@@ -95,6 +159,7 @@ struct InstallableFlake : InstallableValue
FlakeRef flakeRef;
Strings attrPaths;
Strings prefixes;
OutputsSpec outputsSpec;
const flake::LockFlags & lockFlags;
mutable std::shared_ptr<flake::LockedFlake> _lockedFlake;
@@ -103,6 +168,7 @@ struct InstallableFlake : InstallableValue
ref<EvalState> state,
FlakeRef && flakeRef,
std::string_view fragment,
OutputsSpec outputsSpec,
Strings attrPaths,
Strings prefixes,
const flake::LockFlags & lockFlags);
@@ -117,11 +183,17 @@ struct InstallableFlake : InstallableValue
std::vector<DerivationInfo> toDerivations() override;
std::pair<Value *, Pos> toValue(EvalState & state) override;
std::pair<Value *, PosIdx> toValue(EvalState & state) override;
std::vector<std::pair<std::shared_ptr<eval_cache::AttrCursor>, std::string>>
/* Get a cursor to every attrpath in getActualAttrPaths() that
exists. */
std::vector<ref<eval_cache::AttrCursor>>
getCursors(EvalState & state) override;
/* Get a cursor to the first attrpath in getActualAttrPaths() that
exists, or throw an exception with suggestions if none exists. */
ref<eval_cache::AttrCursor> getCursor(EvalState & state) override;
std::shared_ptr<flake::LockedFlake> getLockedFlake() const;
FlakeRef nixpkgsFlakeRef() const override;

View File

@@ -6,9 +6,9 @@ libcmd_DIR := $(d)
libcmd_SOURCES := $(wildcard $(d)/*.cc)
libcmd_CXXFLAGS += -I src/libutil -I src/libstore -I src/libexpr -I src/libmain -I src/libfetchers
libcmd_CXXFLAGS += -I src/libutil -I src/libstore -I src/libexpr -I src/libmain -I src/libfetchers -I src/nix
libcmd_LDFLAGS += $(LOWDOWN_LIBS) -pthread
libcmd_LDFLAGS = $(EDITLINE_LIBS) -llowdown -pthread
libcmd_LIBS = libstore libutil libexpr libmain libfetchers

View File

@@ -9,14 +9,16 @@ namespace nix {
std::string renderMarkdownToTerminal(std::string_view markdown)
{
int windowWidth = getWindowSize().second;
struct lowdown_opts opts {
.type = LOWDOWN_TERM,
.maxdepth = 20,
.cols = std::max(getWindowSize().second, (unsigned short) 80),
.cols = (size_t) std::max(windowWidth - 5, 60),
.hmargin = 0,
.vmargin = 0,
.feat = LOWDOWN_COMMONMARK | LOWDOWN_FENCED | LOWDOWN_DEFLIST | LOWDOWN_TABLES,
.oflags = 0,
.oflags = LOWDOWN_TERM_NOLINK,
};
auto doc = lowdown_doc_new(&opts);

View File

@@ -22,9 +22,11 @@ extern "C" {
#include "ansicolor.hh"
#include "shared.hh"
#include "eval.hh"
#include "eval-cache.hh"
#include "eval-inline.hh"
#include "attr-path.hh"
#include "store-api.hh"
#include "log-store.hh"
#include "common-eval-args.hh"
#include "get-drvs.hh"
#include "derivations.hh"
@@ -32,6 +34,8 @@ extern "C" {
#include "command.hh"
#include "finally.hh"
#include "markdown.hh"
#include "local-fs-store.hh"
#include "progress-bar.hh"
#if HAVE_BOEHMGC
#define GC_INCLUDE_NEW
@@ -45,54 +49,65 @@ struct NixRepl
: gc
#endif
{
string curDir;
std::unique_ptr<EvalState> state;
std::string curDir;
ref<EvalState> state;
Bindings * autoArgs;
size_t debugTraceIndex;
Strings loadedFiles;
typedef std::vector<std::pair<Value*,std::string>> AnnotatedValues;
std::function<AnnotatedValues()> getValues;
const static int envSize = 32768;
StaticEnv staticEnv;
std::shared_ptr<StaticEnv> staticEnv;
Env * env;
int displ;
StringSet varNames;
const Path historyFile;
NixRepl(const Strings & searchPath, nix::ref<Store> store);
NixRepl(const Strings & searchPath, nix::ref<Store> store,ref<EvalState> state,
std::function<AnnotatedValues()> getValues);
~NixRepl();
void mainLoop(const std::vector<std::string> & files);
StringSet completePrefix(string prefix);
bool getLine(string & input, const std::string &prompt);
void mainLoop();
StringSet completePrefix(const std::string & prefix);
bool getLine(std::string & input, const std::string & prompt);
StorePath getDerivationPath(Value & v);
bool processLine(string line);
bool processLine(std::string line);
void loadFile(const Path & path);
void loadFlake(const std::string & flakeRef);
void initEnv();
void loadFiles();
void reloadFiles();
void addAttrsToScope(Value & attrs);
void addVarToScope(const Symbol & name, Value & v);
Expr * parseString(string s);
void evalString(string s, Value & v);
void addVarToScope(const Symbol name, Value & v);
Expr * parseString(std::string s);
void evalString(std::string s, Value & v);
void loadDebugTraceEnv(DebugTrace & dt);
typedef std::set<Value *> ValuesSeen;
std::ostream & printValue(std::ostream & str, Value & v, unsigned int maxDepth);
std::ostream & printValue(std::ostream & str, Value & v, unsigned int maxDepth, ValuesSeen & seen);
std::ostream & printValue(std::ostream & str, Value & v, unsigned int maxDepth);
std::ostream & printValue(std::ostream & str, Value & v, unsigned int maxDepth, ValuesSeen & seen);
};
string removeWhitespace(string s)
std::string removeWhitespace(std::string s)
{
s = chomp(s);
size_t n = s.find_first_not_of(" \n\r\t");
if (n != string::npos) s = string(s, n);
if (n != std::string::npos) s = std::string(s, n);
return s;
}
NixRepl::NixRepl(const Strings & searchPath, nix::ref<Store> store)
: state(std::make_unique<EvalState>(searchPath, store))
, staticEnv(false, &state->staticBaseEnv)
NixRepl::NixRepl(const Strings & searchPath, nix::ref<Store> store, ref<EvalState> state,
std::function<NixRepl::AnnotatedValues()> getValues)
: state(state)
, debugTraceIndex(0)
, getValues(getValues)
, staticEnv(new StaticEnv(false, state->staticBaseEnv.get()))
, historyFile(getDataDir() + "/nix/repl-history")
{
curDir = absPath(".");
@@ -104,23 +119,20 @@ NixRepl::~NixRepl()
write_history(historyFile.c_str());
}
string runNix(Path program, const Strings & args,
void runNix(Path program, const Strings & args,
const std::optional<std::string> & input = {})
{
auto subprocessEnv = getEnv();
subprocessEnv["NIX_CONFIG"] = globalConfig.toKeyValue();
auto res = runProgram(RunOptions {
runProgram2(RunOptions {
.program = settings.nixBinDir+ "/" + program,
.args = args,
.environment = subprocessEnv,
.input = input,
});
if (!statusOk(res.first))
throw ExecError(res.first, fmt("program '%1%' %2%", program, statusToString(res.first)));
return res.second;
return;
}
static NixRepl * curRepl; // ugly
@@ -196,20 +208,45 @@ namespace {
}
}
void NixRepl::mainLoop(const std::vector<std::string> & files)
static std::ostream & showDebugTrace(std::ostream & out, const PosTable & positions, const DebugTrace & dt)
{
string error = ANSI_RED "error:" ANSI_NORMAL " ";
if (dt.isError)
out << ANSI_RED "error: " << ANSI_NORMAL;
out << dt.hint.str() << "\n";
// prefer direct pos, but if noPos then try the expr.
auto pos = *dt.pos
? *dt.pos
: positions[dt.expr.getPos() ? dt.expr.getPos() : noPos];
if (pos) {
printAtPos(pos, out);
auto loc = getCodeLines(pos);
if (loc.has_value()) {
out << "\n";
printCodeLines(out, "", pos, *loc);
out << "\n";
}
}
return out;
}
void NixRepl::mainLoop()
{
std::string error = ANSI_RED "error:" ANSI_NORMAL " ";
notice("Welcome to Nix " + nixVersion + ". Type :? for help.\n");
for (auto & i : files)
loadedFiles.push_back(i);
reloadFiles();
if (!loadedFiles.empty()) notice("");
loadFiles();
// Allow nix-repl specific settings in .inputrc
rl_readline_name = "nix-repl";
createDirs(dirOf(historyFile));
try {
createDirs(dirOf(historyFile));
} catch (SysError & e) {
logWarning(e.info());
}
#ifndef READLINE
el_hist_size = 1000;
#endif
@@ -220,14 +257,21 @@ void NixRepl::mainLoop(const std::vector<std::string> & files)
rl_set_list_possib_func(listPossibleCallback);
#endif
/* Stop the progress bar because it interferes with the display of
the repl. */
stopProgressBar();
std::string input;
while (true) {
// When continuing input from previous lines, don't print a prompt, just align to the same
// number of chars as the prompt.
if (!getLine(input, input.empty() ? "nix-repl> " : " "))
if (!getLine(input, input.empty() ? "nix-repl> " : " ")) {
// ctrl-D should exit the debugger.
state->debugStop = false;
state->debugQuit = true;
break;
}
try {
if (!removeWhitespace(input).empty() && !processLine(input)) return;
} catch (ParseError & e) {
@@ -238,6 +282,14 @@ void NixRepl::mainLoop(const std::vector<std::string> & files)
} else {
printMsg(lvlError, e.msg());
}
} catch (EvalError & e) {
// in debugger mode, an EvalError should trigger another repl session.
// when that session returns the exception will land here. No need to show it again;
// show the error for this repl session instead.
if (state->debugRepl && !state->debugTraces.empty())
showDebugTrace(std::cout, state->positions, state->debugTraces.front());
else
printMsg(lvlError, e.msg());
} catch (Error & e) {
printMsg(lvlError, e.msg());
} catch (Interrupted & e) {
@@ -252,7 +304,7 @@ void NixRepl::mainLoop(const std::vector<std::string> & files)
}
bool NixRepl::getLine(string & input, const std::string &prompt)
bool NixRepl::getLine(std::string & input, const std::string & prompt)
{
struct sigaction act, old;
sigset_t savedSignalMask, set;
@@ -297,7 +349,7 @@ bool NixRepl::getLine(string & input, const std::string &prompt)
}
StringSet NixRepl::completePrefix(string prefix)
StringSet NixRepl::completePrefix(const std::string & prefix)
{
StringSet completions;
@@ -313,7 +365,7 @@ StringSet NixRepl::completePrefix(string prefix)
size_t slash, dot;
if ((slash = cur.rfind('/')) != string::npos) {
if ((slash = cur.rfind('/')) != std::string::npos) {
try {
auto dir = std::string(cur, 0, slash);
auto prefix2 = std::string(cur, slash + 1);
@@ -323,11 +375,11 @@ StringSet NixRepl::completePrefix(string prefix)
}
} catch (Error &) {
}
} else if ((dot = cur.rfind('.')) == string::npos) {
} else if ((dot = cur.rfind('.')) == std::string::npos) {
/* This is a variable name; look it up in the current scope. */
StringSet::iterator i = varNames.lower_bound(cur);
while (i != varNames.end()) {
if (string(*i, 0, cur.size()) != cur) break;
if (i->substr(0, cur.size()) != cur) break;
completions.insert(prev + *i);
i++;
}
@@ -336,8 +388,8 @@ StringSet NixRepl::completePrefix(string prefix)
/* This is an expression that should evaluate to an
attribute set. Evaluate it to get the names of the
attributes. */
string expr(cur, 0, dot);
string cur2 = string(cur, dot + 1);
auto expr = cur.substr(0, dot);
auto cur2 = cur.substr(dot + 1);
Expr * e = parseString(expr);
Value v;
@@ -345,9 +397,9 @@ StringSet NixRepl::completePrefix(string prefix)
state->forceAttrs(v, noPos);
for (auto & i : *v.attrs) {
string name = i.name;
if (string(name, 0, cur2.size()) != cur2) continue;
completions.insert(prev + expr + "." + name);
std::string_view name = state->symbols[i.name];
if (name.substr(0, cur2.size()) != cur2) continue;
completions.insert(concatStrings(prev, expr, ".", name));
}
} catch (ParseError & e) {
@@ -365,7 +417,7 @@ StringSet NixRepl::completePrefix(string prefix)
}
bool isVarName(const string & s)
static bool isVarName(std::string_view s)
{
if (s.size() == 0) return false;
char c = s[0];
@@ -384,28 +436,41 @@ StorePath NixRepl::getDerivationPath(Value & v) {
auto drvInfo = getDerivation(*state, v, false);
if (!drvInfo)
throw Error("expression does not evaluate to a derivation, so I can't build it");
Path drvPathRaw = drvInfo->queryDrvPath();
if (drvPathRaw == "")
throw Error("expression did not evaluate to a valid derivation (no drv path)");
StorePath drvPath = state->store->parseStorePath(drvPathRaw);
if (!state->store->isValidPath(drvPath))
throw Error("expression did not evaluate to a valid derivation (invalid drv path)");
return drvPath;
auto drvPath = drvInfo->queryDrvPath();
if (!drvPath)
throw Error("expression did not evaluate to a valid derivation (no 'drvPath' attribute)");
if (!state->store->isValidPath(*drvPath))
throw Error("expression evaluated to invalid derivation '%s'", state->store->printStorePath(*drvPath));
return *drvPath;
}
bool NixRepl::processLine(string line)
void NixRepl::loadDebugTraceEnv(DebugTrace & dt)
{
initEnv();
auto se = state->getStaticEnv(dt.expr);
if (se) {
auto vm = mapStaticEnvBindings(state->symbols, *se.get(), dt.env);
// add staticenv vars.
for (auto & [name, value] : *(vm.get()))
addVarToScope(state->symbols.create(name), *value);
}
}
bool NixRepl::processLine(std::string line)
{
line = trim(line);
if (line == "") return true;
_isInterrupted = false;
string command, arg;
std::string command, arg;
if (line[0] == ':') {
size_t p = line.find_first_of(" \n\r\t");
command = string(line, 0, p);
if (p != string::npos) arg = removeWhitespace(string(line, p));
command = line.substr(0, p);
if (p != std::string::npos) arg = removeWhitespace(line.substr(p));
} else {
arg = line;
}
@@ -418,7 +483,8 @@ bool NixRepl::processLine(string line)
<< " <expr> Evaluate and print expression\n"
<< " <x> = <expr> Bind expression to variable\n"
<< " :a <expr> Add attributes from resulting set to scope\n"
<< " :b <expr> Build derivation\n"
<< " :b <expr> Build a derivation\n"
<< " :bl <expr> Build a derivation, creating GC roots in the working directory\n"
<< " :e <expr> Open package or function in $EDITOR\n"
<< " :i <expr> Build derivation, then install result into current profile\n"
<< " :l <path> Load Nix expression and add it to scope\n"
@@ -426,12 +492,72 @@ bool NixRepl::processLine(string line)
<< " :p <expr> Evaluate and print expression recursively\n"
<< " :q Exit nix-repl\n"
<< " :r Reload all files\n"
<< " :s <expr> Build dependencies of derivation, then start nix-shell\n"
<< " :sh <expr> Build dependencies of derivation, then start nix-shell\n"
<< " :t <expr> Describe result of evaluation\n"
<< " :u <expr> Build derivation, then start nix-shell\n"
<< " :doc <expr> Show documentation of a builtin function\n"
<< " :log <expr> Show logs for a derivation\n"
<< " :st [bool] Enable, disable or toggle showing traces for errors\n";
<< " :te [bool] Enable, disable or toggle showing traces for errors\n"
;
if (state->debugRepl) {
std::cout
<< "\n"
<< " Debug mode commands\n"
<< " :env Show env stack\n"
<< " :bt Show trace stack\n"
<< " :st Show current trace\n"
<< " :st <idx> Change to another trace in the stack\n"
<< " :c Go until end of program, exception, or builtins.break\n"
<< " :s Go one step\n"
;
}
}
else if (state->debugRepl && (command == ":bt" || command == ":backtrace")) {
for (const auto & [idx, i] : enumerate(state->debugTraces)) {
std::cout << "\n" << ANSI_BLUE << idx << ANSI_NORMAL << ": ";
showDebugTrace(std::cout, state->positions, i);
}
}
else if (state->debugRepl && (command == ":env")) {
for (const auto & [idx, i] : enumerate(state->debugTraces)) {
if (idx == debugTraceIndex) {
printEnvBindings(*state, i.expr, i.env);
break;
}
}
}
else if (state->debugRepl && (command == ":st")) {
try {
// change the DebugTrace index.
debugTraceIndex = stoi(arg);
} catch (...) { }
for (const auto & [idx, i] : enumerate(state->debugTraces)) {
if (idx == debugTraceIndex) {
std::cout << "\n" << ANSI_BLUE << idx << ANSI_NORMAL << ": ";
showDebugTrace(std::cout, state->positions, i);
std::cout << std::endl;
printEnvBindings(*state, i.expr, i.env);
loadDebugTraceEnv(i);
break;
}
}
}
else if (state->debugRepl && (command == ":s" || command == ":step")) {
// set flag to stop at next DebugTrace; exit repl.
state->debugStop = true;
return false;
}
else if (state->debugRepl && (command == ":c" || command == ":continue")) {
// set flag to run to next breakpoint or end of program; exit repl.
state->debugStop = false;
return false;
}
else if (command == ":a" || command == ":add") {
@@ -458,21 +584,23 @@ bool NixRepl::processLine(string line)
Value v;
evalString(arg, v);
Pos pos;
if (v.type() == nPath || v.type() == nString) {
PathSet context;
auto filename = state->coerceToString(noPos, v, context);
pos.file = state->symbols.create(*filename);
} else if (v.isLambda()) {
pos = v.lambda.fun->pos;
} else {
// assume it's a derivation
pos = findPackageFilename(*state, v, arg);
}
const auto [file, line] = [&] () -> std::pair<std::string, uint32_t> {
if (v.type() == nPath || v.type() == nString) {
PathSet context;
auto filename = state->coerceToString(noPos, v, context).toOwned();
state->symbols.create(filename);
return {filename, 0};
} else if (v.isLambda()) {
auto pos = state->positions[v.lambda.fun->pos];
return {pos.file, pos.line};
} else {
// assume it's a derivation
return findPackageFilename(*state, v, arg);
}
}();
// Open in EDITOR
auto args = editorFor(pos);
auto args = editorFor(file, line);
auto editor = args.front();
args.pop_front();
@@ -495,24 +623,32 @@ bool NixRepl::processLine(string line)
Value v, f, result;
evalString(arg, v);
evalString("drv: (import <nixpkgs> {}).runCommand \"shell\" { buildInputs = [ drv ]; } \"\"", f);
state->callFunction(f, v, result, Pos());
state->callFunction(f, v, result, PosIdx());
StorePath drvPath = getDerivationPath(result);
runNix("nix-shell", {state->store->printStorePath(drvPath)});
}
else if (command == ":b" || command == ":i" || command == ":s" || command == ":log") {
else if (command == ":b" || command == ":bl" || command == ":i" || command == ":sh" || command == ":log") {
Value v;
evalString(arg, v);
StorePath drvPath = getDerivationPath(v);
Path drvPathRaw = state->store->printStorePath(drvPath);
if (command == ":b") {
if (command == ":b" || command == ":bl") {
state->store->buildPaths({DerivedPath::Built{drvPath}});
auto drv = state->store->readDerivation(drvPath);
logger->cout("\nThis derivation produced the following outputs:");
for (auto & [outputName, outputPath] : state->store->queryDerivationOutputMap(drvPath))
logger->cout(" %s -> %s", outputName, state->store->printStorePath(outputPath));
for (auto & [outputName, outputPath] : state->store->queryDerivationOutputMap(drvPath)) {
auto localStore = state->store.dynamic_pointer_cast<LocalFSStore>();
if (localStore && command == ":bl") {
std::string symlink = "repl-result-" + outputName;
localStore->addPermRoot(outputPath, absPath(symlink));
logger->cout(" ./%s -> %s", symlink, state->store->printStorePath(outputPath));
} else {
logger->cout(" %s -> %s", outputName, state->store->printStorePath(outputPath));
}
}
} else if (command == ":i") {
runNix("nix-env", {"-i", drvPathRaw});
} else if (command == ":log") {
@@ -527,9 +663,16 @@ bool NixRepl::processLine(string line)
bool foundLog = false;
RunPager pager;
for (auto & sub : subs) {
auto log = sub->getBuildLog(drvPath);
auto * logSubP = dynamic_cast<LogStore *>(&*sub);
if (!logSubP) {
printInfo("Skipped '%s' which does not support retrieving build logs", sub->getUri());
continue;
}
auto & logSub = *logSubP;
auto log = logSub.getBuildLog(drvPath);
if (log) {
printInfo("got build log for '%s' from '%s'", drvPathRaw, sub->getUri());
printInfo("got build log for '%s' from '%s'", drvPathRaw, logSub.getUri());
logger->writeToStdout(*log);
foundLog = true;
break;
@@ -547,8 +690,11 @@ bool NixRepl::processLine(string line)
printValue(std::cout, v, 1000000000) << std::endl;
}
else if (command == ":q" || command == ":quit")
else if (command == ":q" || command == ":quit") {
state->debugStop = false;
state->debugQuit = true;
return false;
}
else if (command == ":doc") {
Value v;
@@ -573,7 +719,7 @@ bool NixRepl::processLine(string line)
throw Error("value does not have documentation");
}
else if (command == ":st" || command == ":show-trace") {
else if (command == ":te" || command == ":trace-enable") {
if (arg == "false" || (arg == "" && loggerSettings.showTrace)) {
std::cout << "not showing error traces\n";
loggerSettings.showTrace = false;
@@ -590,13 +736,13 @@ bool NixRepl::processLine(string line)
else {
size_t p = line.find('=');
string name;
if (p != string::npos &&
std::string name;
if (p != std::string::npos &&
p < line.size() &&
line[p + 1] != '=' &&
isVarName(name = removeWhitespace(string(line, 0, p))))
isVarName(name = removeWhitespace(line.substr(0, p))))
{
Expr * e = parseString(string(line, p + 1));
Expr * e = parseString(line.substr(p + 1));
Value & v(*state->allocValue());
v.mkThunk(env, e);
addVarToScope(state->symbols.create(name), v);
@@ -610,7 +756,6 @@ bool NixRepl::processLine(string line)
return true;
}
void NixRepl::loadFile(const Path & path)
{
loadedFiles.remove(path);
@@ -627,8 +772,8 @@ void NixRepl::loadFlake(const std::string & flakeRefS)
throw Error("cannot use ':load-flake' without a path specified. (Use '.' for the current working directory.)");
auto flakeRef = parseFlakeRef(flakeRefS, absPath("."), true);
if (evalSettings.pureEval && !flakeRef.input.isImmutable())
throw Error("cannot use ':load-flake' on mutable flake reference '%s' (use --impure to override)", flakeRefS);
if (evalSettings.pureEval && !flakeRef.input.isLocked())
throw Error("cannot use ':load-flake' on locked flake reference '%s' (use --impure to override)", flakeRefS);
Value v;
@@ -649,11 +794,11 @@ void NixRepl::initEnv()
env = &state->allocEnv(envSize);
env->up = &state->baseEnv;
displ = 0;
staticEnv.vars.clear();
staticEnv->vars.clear();
varNames.clear();
for (auto & i : state->staticBaseEnv.vars)
varNames.insert(i.first);
for (auto & i : state->staticBaseEnv->vars)
varNames.emplace(state->symbols[i.first]);
}
@@ -661,16 +806,24 @@ void NixRepl::reloadFiles()
{
initEnv();
loadFiles();
}
void NixRepl::loadFiles()
{
Strings old = loadedFiles;
loadedFiles.clear();
bool first = true;
for (auto & i : old) {
if (!first) notice("");
first = false;
notice("Loading '%1%'...", i);
loadFile(i);
}
for (auto & [i, what] : getValues()) {
notice("Loading installable '%1%'...", what);
addAttrsToScope(*i);
}
}
@@ -681,37 +834,37 @@ void NixRepl::addAttrsToScope(Value & attrs)
throw Error("environment full; cannot add more variables");
for (auto & i : *attrs.attrs) {
staticEnv.vars.emplace_back(i.name, displ);
staticEnv->vars.emplace_back(i.name, displ);
env->values[displ++] = i.value;
varNames.insert((string) i.name);
varNames.emplace(state->symbols[i.name]);
}
staticEnv.sort();
staticEnv.deduplicate();
staticEnv->sort();
staticEnv->deduplicate();
notice("Added %1% variables.", attrs.attrs->size());
}
void NixRepl::addVarToScope(const Symbol & name, Value & v)
void NixRepl::addVarToScope(const Symbol name, Value & v)
{
if (displ >= envSize)
throw Error("environment full; cannot add more variables");
if (auto oldVar = staticEnv.find(name); oldVar != staticEnv.vars.end())
staticEnv.vars.erase(oldVar);
staticEnv.vars.emplace_back(name, displ);
staticEnv.sort();
if (auto oldVar = staticEnv->find(name); oldVar != staticEnv->vars.end())
staticEnv->vars.erase(oldVar);
staticEnv->vars.emplace_back(name, displ);
staticEnv->sort();
env->values[displ++] = &v;
varNames.insert((string) name);
varNames.emplace(state->symbols[name]);
}
Expr * NixRepl::parseString(string s)
Expr * NixRepl::parseString(std::string s)
{
Expr * e = state->parseExprFromString(std::move(s), curDir, staticEnv);
return e;
}
void NixRepl::evalString(string s, Value & v)
void NixRepl::evalString(std::string s, Value & v)
{
Expr * e = parseString(s);
e->eval(*state, *env, v);
@@ -780,17 +933,20 @@ std::ostream & NixRepl::printValue(std::ostream & str, Value & v, unsigned int m
str << "«derivation ";
Bindings::iterator i = v.attrs->find(state->sDrvPath);
PathSet context;
Path drvPath = i != v.attrs->end() ? state->coerceToPath(*i->pos, *i->value, context) : "???";
str << drvPath << "»";
if (i != v.attrs->end())
str << state->store->printStorePath(state->coerceToStorePath(i->pos, *i->value, context));
else
str << "???";
str << "»";
}
else if (maxDepth > 0) {
str << "{ ";
typedef std::map<string, Value *> Sorted;
typedef std::map<std::string, Value *> Sorted;
Sorted sorted;
for (auto & i : *v.attrs)
sorted[i.name] = i.value;
sorted.emplace(state->symbols[i.name], i.value);
for (auto & i : sorted) {
if (isVarName(i.first))
@@ -798,7 +954,7 @@ std::ostream & NixRepl::printValue(std::ostream & str, Value & v, unsigned int m
else
printStringValue(str, i.first.c_str());
str << " = ";
if (seen.find(i.second) != seen.end())
if (seen.count(i.second))
str << "«repeated»";
else
try {
@@ -840,7 +996,7 @@ std::ostream & NixRepl::printValue(std::ostream & str, Value & v, unsigned int m
case nFunction:
if (v.isLambda()) {
std::ostringstream s;
s << v.lambda.fun->pos;
s << state->positions[v.lambda.fun->pos];
str << ANSI_BLUE "«lambda @ " << filterANSIEscapes(s.str()) << "»" ANSI_NORMAL;
} else if (v.isPrimOp()) {
str << ANSI_MAGENTA "«primop»" ANSI_NORMAL;
@@ -863,17 +1019,66 @@ std::ostream & NixRepl::printValue(std::ostream & str, Value & v, unsigned int m
return str;
}
struct CmdRepl : StoreCommand, MixEvalArgs
void runRepl(
ref<EvalState>evalState,
const ValMap & extraEnv)
{
auto getValues = [&]()->NixRepl::AnnotatedValues{
NixRepl::AnnotatedValues values;
return values;
};
const Strings & searchPath = {};
auto repl = std::make_unique<NixRepl>(
searchPath,
openStore(),
evalState,
getValues
);
repl->initEnv();
// add 'extra' vars.
for (auto & [name, value] : extraEnv)
repl->addVarToScope(repl->state->symbols.create(name), *value);
repl->mainLoop();
}
struct CmdRepl : InstallablesCommand
{
CmdRepl() {
evalSettings.pureEval = false;
}
void prepare()
{
if (!settings.isExperimentalFeatureEnabled(Xp::ReplFlake) && !(file) && this->_installables.size() >= 1) {
warn("future versions of Nix will require using `--file` to load a file");
if (this->_installables.size() > 1)
warn("more than one input file is not currently supported");
auto filePath = this->_installables[0].data();
file = std::optional(filePath);
_installables.front() = _installables.back();
_installables.pop_back();
}
installables = InstallablesCommand::load();
}
std::vector<std::string> files;
CmdRepl()
Strings getDefaultFlakeAttrPaths() override
{
expectArgs({
.label = "files",
.handler = {&files},
.completer = completePath
});
return {""};
}
bool useDefaultInstallables() override
{
return file.has_value() or expr.has_value();
}
bool forceImpureByDefault() override
{
return true;
}
std::string description() override
@@ -890,10 +1095,37 @@ struct CmdRepl : StoreCommand, MixEvalArgs
void run(ref<Store> store) override
{
evalSettings.pureEval = false;
auto repl = std::make_unique<NixRepl>(searchPath, openStore());
auto state = getEvalState();
auto getValues = [&]()->NixRepl::AnnotatedValues{
auto installables = load();
NixRepl::AnnotatedValues values;
for (auto & installable: installables){
auto what = installable->what();
if (file){
auto [val, pos] = installable->toValue(*state);
auto what = installable->what();
state->forceValue(*val, pos);
auto autoArgs = getAutoArgs(*state);
auto valPost = state->allocValue();
state->autoCallFunction(*autoArgs, *val, *valPost);
state->forceValue(*valPost, pos);
values.push_back( {valPost, what });
} else {
auto [val, pos] = installable->toValue(*state);
values.push_back( {val, what} );
}
}
return values;
};
auto repl = std::make_unique<NixRepl>(
searchPath,
openStore(),
state,
getValues
);
repl->autoArgs = getAutoArgs(*repl->state);
repl->mainLoop(files);
repl->initEnv();
repl->mainLoop();
}
};

View File

@@ -9,7 +9,7 @@ namespace nix {
static Strings parseAttrPath(std::string_view s)
{
Strings res;
string cur;
std::string cur;
auto i = s.begin();
while (i != s.end()) {
if (*i == '.') {
@@ -41,13 +41,13 @@ std::vector<Symbol> parseAttrPath(EvalState & state, std::string_view s)
}
std::pair<Value *, Pos> findAlongAttrPath(EvalState & state, const string & attrPath,
std::pair<Value *, PosIdx> findAlongAttrPath(EvalState & state, const std::string & attrPath,
Bindings & autoArgs, Value & vIn)
{
Strings tokens = parseAttrPath(attrPath);
Value * v = &vIn;
Pos pos = noPos;
PosIdx pos = noPos;
for (auto & attr : tokens) {
@@ -74,10 +74,16 @@ std::pair<Value *, Pos> findAlongAttrPath(EvalState & state, const string & attr
throw Error("empty attribute name in selection path '%1%'", attrPath);
Bindings::iterator a = v->attrs->find(state.symbols.create(attr));
if (a == v->attrs->end())
throw AttrPathNotFound("attribute '%1%' in selection path '%2%' not found", attr, attrPath);
if (a == v->attrs->end()) {
std::set<std::string> attrNames;
for (auto & attr : *v->attrs)
attrNames.insert(state.symbols[attr.name]);
auto suggestions = Suggestions::bestMatches(attrNames, attr);
throw AttrPathNotFound(suggestions, "attribute '%1%' in selection path '%2%' not found", attr, attrPath);
}
v = &*a->value;
pos = *a->pos;
pos = a->pos;
}
else {
@@ -100,7 +106,7 @@ std::pair<Value *, Pos> findAlongAttrPath(EvalState & state, const string & attr
}
Pos findPackageFilename(EvalState & state, Value & v, std::string what)
std::pair<std::string, uint32_t> findPackageFilename(EvalState & state, Value & v, std::string what)
{
Value * v2;
try {
@@ -121,14 +127,12 @@ Pos findPackageFilename(EvalState & state, Value & v, std::string what)
std::string filename(pos, 0, colon);
unsigned int lineno;
try {
lineno = std::stoi(std::string(pos, colon + 1, string::npos));
lineno = std::stoi(std::string(pos, colon + 1, std::string::npos));
} catch (std::invalid_argument & e) {
throw ParseError("cannot parse line number '%s'", pos);
}
Symbol file = state.symbols.create(filename);
return { foFile, file, lineno, 0 };
return { std::move(filename), lineno };
}

View File

@@ -10,11 +10,14 @@ namespace nix {
MakeError(AttrPathNotFound, Error);
MakeError(NoPositionInfo, Error);
std::pair<Value *, Pos> findAlongAttrPath(EvalState & state, const string & attrPath,
Bindings & autoArgs, Value & vIn);
std::pair<Value *, PosIdx> findAlongAttrPath(
EvalState & state,
const std::string & attrPath,
Bindings & autoArgs,
Value & vIn);
/* Heuristic to find the filename and lineno or a nix value. */
Pos findPackageFilename(EvalState & state, Value & v, std::string what);
std::pair<std::string, uint32_t> findPackageFilename(EvalState & state, Value & v, std::string what);
std::vector<Symbol> parseAttrPath(EvalState & state, std::string_view s);

View File

@@ -26,7 +26,7 @@ Bindings * EvalState::allocBindings(size_t capacity)
/* Create a new attribute named 'name' on an existing attribute set stored
in 'vAttrs' and return the newly allocated Value which is associated with
this attribute. */
Value * EvalState::allocAttr(Value & vAttrs, const Symbol & name)
Value * EvalState::allocAttr(Value & vAttrs, Symbol name)
{
Value * v = allocValue();
vAttrs.attrs->push_back(Attr(name, v));
@@ -40,7 +40,7 @@ Value * EvalState::allocAttr(Value & vAttrs, std::string_view name)
}
Value & BindingsBuilder::alloc(const Symbol & name, ptr<Pos> pos)
Value & BindingsBuilder::alloc(Symbol name, PosIdx pos)
{
auto value = state.allocValue();
bindings->push_back(Attr(name, value, pos));
@@ -48,7 +48,7 @@ Value & BindingsBuilder::alloc(const Symbol & name, ptr<Pos> pos)
}
Value & BindingsBuilder::alloc(std::string_view name, ptr<Pos> pos)
Value & BindingsBuilder::alloc(std::string_view name, PosIdx pos)
{
return alloc(state.symbols.create(name), pos);
}

View File

@@ -15,18 +15,27 @@ struct Value;
/* Map one attribute name to its value. */
struct Attr
{
/* the placement of `name` and `pos` in this struct is important.
both of them are uint32 wrappers, they are next to each other
to make sure that Attr has no padding on 64 bit machines. that
way we keep Attr size at two words with no wasted space. */
Symbol name;
PosIdx pos;
Value * value;
ptr<Pos> pos;
Attr(Symbol name, Value * value, ptr<Pos> pos = ptr(&noPos))
: name(name), value(value), pos(pos) { };
Attr() : pos(&noPos) { };
Attr(Symbol name, Value * value, PosIdx pos = noPos)
: name(name), pos(pos), value(value) { };
Attr() { };
bool operator < (const Attr & a) const
{
return name < a.name;
}
};
static_assert(sizeof(Attr) == 2 * sizeof(uint32_t) + sizeof(Value *),
"performance of the evaluator is highly sensitive to the size of Attr. "
"avoid introducing any padding into Attr if at all possible, and do not "
"introduce new fields that need not be present for almost every instance.");
/* Bindings contains all the attributes of an attribute set. It is defined
by its size and its capacity, the capacity being the number of Attr
elements allocated after this structure, while the size corresponds to
@@ -35,13 +44,13 @@ class Bindings
{
public:
typedef uint32_t size_t;
ptr<Pos> pos;
PosIdx pos;
private:
size_t size_, capacity_;
Attr attrs[0];
Bindings(size_t capacity) : pos(&noPos), size_(0), capacity_(capacity) { }
Bindings(size_t capacity) : size_(0), capacity_(capacity) { }
Bindings(const Bindings & bindings) = delete;
public:
@@ -57,7 +66,7 @@ public:
attrs[size_++] = attr;
}
iterator find(const Symbol & name)
iterator find(Symbol name)
{
Attr key(name, 0);
iterator i = std::lower_bound(begin(), end(), key);
@@ -65,7 +74,7 @@ public:
return end();
}
Attr * get(const Symbol & name)
Attr * get(Symbol name)
{
Attr key(name, 0);
iterator i = std::lower_bound(begin(), end(), key);
@@ -73,18 +82,6 @@ public:
return nullptr;
}
Attr & need(const Symbol & name, const Pos & pos = noPos)
{
auto a = get(name);
if (!a)
throw Error({
.msg = hintfmt("attribute '%s' missing", name),
.errPos = pos
});
return *a;
}
iterator begin() { return &attrs[0]; }
iterator end() { return &attrs[size_]; }
@@ -98,14 +95,15 @@ public:
size_t capacity() { return capacity_; }
/* Returns the attributes in lexicographically sorted order. */
std::vector<const Attr *> lexicographicOrder() const
std::vector<const Attr *> lexicographicOrder(const SymbolTable & symbols) const
{
std::vector<const Attr *> res;
res.reserve(size_);
for (size_t n = 0; n < size_; n++)
res.emplace_back(&attrs[n]);
std::sort(res.begin(), res.end(), [](const Attr * a, const Attr * b) {
return (const string &) a->name < (const string &) b->name;
std::sort(res.begin(), res.end(), [&](const Attr * a, const Attr * b) {
std::string_view sa = symbols[a->name], sb = symbols[b->name];
return sa < sb;
});
return res;
}
@@ -130,7 +128,7 @@ public:
: bindings(bindings), state(state)
{ }
void insert(Symbol name, Value * value, ptr<Pos> pos = ptr(&noPos))
void insert(Symbol name, Value * value, PosIdx pos = noPos)
{
insert(Attr(name, value, pos));
}
@@ -145,9 +143,9 @@ public:
bindings->push_back(attr);
}
Value & alloc(const Symbol & name, ptr<Pos> pos = ptr(&noPos));
Value & alloc(Symbol name, PosIdx pos = noPos);
Value & alloc(std::string_view name, ptr<Pos> pos = ptr(&noPos));
Value & alloc(std::string_view name, PosIdx pos = noPos);
Bindings * finish()
{

Some files were not shown because too many files have changed in this diff Show More