Split `common.sh` into the vars and functions definitions vs starting
the daemon (and possibly other initialization logic). This way,
`init.sh` can just `source` the former. Trying to start the daemon
before `nix.conf` is written will fail because `nix daemon` requires
`--experimental-features 'nix-command'`.
`killDaemon` is idempotent, so it's safe to call when no daemon is
running.
`startDaemon` and `killDaemon` use the PID (which is now exported to
subshells) to decide whether there is work to be done, rather than
`NIX_REMOTE`, which might conceivably be set differently even if a
daemon is running.
`startDaemon` and `killDaemon` can save/restore the old `NIX_REMOTE` as
`NIX_REMOTE_OLD`.
`init.sh` kills daemon before deleting everything (including the daemon
socket).
`init.sh` is tested on its own. We used to do that. I deleted it in
4720853129 but I am not sure why. Better
to just restore it; at one point working on this every other test
passed, so seems good to check whether `init.sh` can be run twice.
We don't *need* to run `init.sh` twice, but I want to try to make our
tests as robust as possible so that manual debugging (where tests for
better or worse might be run ways that we didn't expect) is less
fragile.
It would be incorrect to say that the `sourceInfo` has an `outPath`
that isn't the root. `sourceInfo` is about the root, whereas only
the flake may not be about the root. Thanks Eelco for pointing that
out.
Some dependencies supposed to be skipped in the cross build, along with
not using the gold linker. But in https://github.com/NixOS/nix/pull/6538
this was accidentally not preserved.
Also since https://github.com/NixOS/nix/pull/6538 we saw some new
aarch64-linux static build failures. This is a first attempt to try to
fix those failures. If this is not sufficient, there are other things we
can try next.
It is independent of SourceExprCommand, which is about parsing
installables, except for the fact that parsing installables is one of
the many things influenced by read-only mode.
- `nixpkgsFor` does all of native, static, cross, and the different stdenvs.
- The main Nix derivation is no longer duplicated for static.
- DRY nixpkgs.lib and lib.genAttrs calls.
We are looking for *$ because it indicate that it was constructed with a new but
not release. De-referencing shallow copy so deleting as whole might create
dangling pointer that's why we move it so we delete a empty containers + the
nice perf boost.
For static builds, we need to propagate all the static library
dependencies to the link of the program. E.g. if libstore-tests-exe
depends on libnixstore-tests, and libnixstore-tests depends on
libstore, then libstore-tests-exe needs to link against libstore.
https://hydra.nixos.org/build/209007480
- Refer to current version in readme
- Split into flakes and non-flakes section
- Change order to move nix-build to the end, since people often start
with it in the beginning.
- Use proper "Note" syntax
- Add notes about editor integration
- Move information about target platforms and stdenvs into separate
sections
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
Co-authored-by: Alexander Bantyev <alexander.bantyev@tweag.io>
Co-authored-by: Théophane Hufschmitt <theophane.hufschmitt@tweag.io>
Adding a test to ensure there is no regression.
The tests that are split out of `tests/build.sh` are ones that don't yet
work with CA derivation. I have not yet evaluated whether they should or
not.
This behavior, reported missing in issue #4661, already got fixed in
PR #4818, but didn't get a test case then.
Nixpkgs on aarch64-linux is currently stuck on GCC 9
(https://github.com/NixOS/nixpkgs/issues/208412) and using gcc11Stdenv
doesn't work either.
So use c++2a instead of c++20 for now. Unfortunately this means we
can't use some C++20 features for now (like std::span).
Descriptions for commandline flags may not include newlines and should
be rather short for display in a shell. Truncate the description string
of a flag on '\n' or '.' to and add an ellipsis if needed.
XDG Base Directory is a standard for locations for storing various
files. Nix has a few files which seem to fit in the standard, but
currently use a custom location directly in the user's ~, polluting
it:
- ~/.nix-profile
- ~/.nix-defexpr
- ~/.nix-channels
This commit adds a config option (use-xdg-base-directories) to follow
the XDG spec and instead use the following locations:
- $XDG_STATE_HOME/nix/profile
- $XDG_STATE_HOME/nix/defexpr
- $XDG_STATE_HOME/nix/channels
If $XDG_STATE_HOME is not set, it is assumed to be ~/.local/state.
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
Co-authored-by: Tim Fenney <kodekata@gmail.com>
Co-authored-by: pasqui23 <pasqui23@users.noreply.github.com>
Co-authored-by: Artturin <Artturin@artturin.com>
Co-authored-by: John Ericson <Ericson2314@Yahoo.com>
Fixes#3898
The entire `BinaryCaches` row used to get replaced after it became
stale according to the `timestamp` column. In a concurrent scenario,
this leads to foreign key conflicts as different instances of the
in-process `state.caches` cache now differ, with the consequence that
the older process still tries to use the `id` number of the old record.
Furthermore, this phenomenon appears to have caused the cache for
actual narinfos to be erased about every week, while the default
ttl for narinfos was supposed to be 30 days.
This is slightly more accurate considering that an outdated record
may exist in the persistent cache. Possibly-outdated records are
quite relevant as they may be foreign keys to more recent information
that we want to keep, but we will not return them here.
In unprivileged podman containers, /proc is not fully visible (there
are other filesystems mounted on subdirectories of /proc). Therefore
we can't mount a new /proc in the sandbox that matches the PID
namespace of the sandbox. So this commit automatically disables
sandboxing if /proc is not fully visible.
This didn't work because sandboxing doesn't work in Docker. However,
the sandboxing check is done lazily - after clone(CLONE_NEWNS) fails,
we retry with sandboxing disabled. But at that point, we've already
done UID allocation under the assumption that user namespaces are
enabled.
So let's get rid of the "goto fallback" logic and just detect early
whether user / mount namespaces are enabled.
This commit also gets rid of a compatibility hack for some ancient
Linux kernels (<2.13).
Previously we would completely refetch the submodules from the
network, even though the repo might already have them. Now we copy the
.git/modules directory from the repo as an optimisation. This speeds
up evaluating
builtins.fetchTree { type = "git"; url = "/path/to/blender"; submodules = true; }
(where /path/to/blender already has the needed submodules) from 121s
to 57s.
This is still pretty inefficient and a hack, but a better solution is
best done on the lazy-trees branch.
This change also help in the case where the repo already has the
submodules but the origin is unfetchable for whatever reason
(e.g. there have been cases where Nix in a GitHub action doesn't have
the right authentication set up).
We cannot use 'actualUrl', because for file:// repos that's not the
original URL that the repo was fetched from. This is a problem since
submodules may be relative to the original URL.
Fixes e.g.
nix eval --impure --json --expr 'builtins.fetchTree { type = "git"; url = "/path/to/blender"; submodules = true; }'
where /path/to/blender is a clone of
https://github.com/blender/blender.git (which has several relative
submodules like '../blender-addons.git').
Previously, build-remote would show a warning if all build slots were
taken, even if they would open up later. This caused a lot of spam in
the logs. Disable this warning when maxJobs > 0.
See #6263
`clang11StdenvPackages` does not exist
```
│ └───x86_64-linux
│ ├───ccacheStdenv: development environment 'nix'
│ ├───clang11Stdenv: development environment 'nix'
│ ├───clangStdenv: development environment 'nix'
│ ├───default: development environment 'nix'
│ ├───gccStdenv: development environment 'nix'
│ ├───libcxxStdenv: development environment 'nix'
│ └───stdenv: development environment 'nix'
```
Per the old FIXME, this flag was on too many commands, and mostly
ignored. Now it is just on the commands where it actually has an effect.
Per https://github.com/NixOS/nix/issues/7261, I would still like to get
rid of it entirely, but that is a separate project. This change should
be good with or without doing that.
`nix app` had something called `InstallableDerivedPath` which is
actually the same thing. We go with the later's name because it has
become more correct.
I originally did this change (more hurriedly) as part of #6225 --- a
mini store-only Nix and a full Nix need to share this code. In the first
RFC meeting for https://github.com/NixOS/rfcs/pull/134 we discussed how
some splitting of the massive `installables.cc` could begin prior, as
that is a good thing anyways. (@edolstra's words, not mine!) This would
be one such PR.
tl;dr: With this 1 line change I was able to get a speedup of 1.5x on 1Gbit/s
wan connections by enabling zstd compression in nginx.
Also nix already supported all common compression format for http
transfer, webservers usually only enable them if they are advertised
through the Accept-Encoding header.
This pull requests makes nix advertises content compression support for
zstd, br, gzip and deflate.
It's particular useful to add transparent compression for binary caches
that serve packages from the host nix store in particular nix-serve,
nix-serve-ng and harmonia.
I tried so far gzip, brotli and zstd, whereas only zstd was able to bring
me performance improvements for 1Gbit/s WAN connections.
The following nginx configuration was used in combination with the
[zstd module](https://github.com/tokers/zstd-nginx-module) and
[harmonia](https://github.com/nix-community/harmonia/)
```nix
{
services.nginx.virtualHosts."cache.yourhost.com" = {
locations."/".extraConfig = ''
proxy_pass http://127.0.0.1:5000;
proxy_set_header Host $host;
proxy_redirect http:// https://;
proxy_http_version 1.1;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
zstd on;
zstd_types application/x-nix-archive;
'';
};
}
```
For testing I unpacked a linux kernel tarball to the nix store using
this command `nix-prefetch-url --unpack https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-6.1.8.tar.gz`.
Before:
```console
$ nix build && rm -rf /tmp/hello && time ./result/bin/nix copy --no-check-sigs --from https://cache.thalheim.io --to 'file:///tmp/hello?compression=none' '/nix/store/j42mahch5f0jvfmayhzwbb88sw36fvah-linux-6.1.8.tar.gz'
warning: Git tree '/scratch/joerg/nix' is dirty
real 0m18,375s
user 0m2,889s
sys 0m1,558s
```
After:
```console
$ nix build && rm -rf /tmp/hello && time ./result/bin/nix copy --no-check-sigs --from https://cache.thalheim.io --to 'file:///tmp/hello?compression=none' '/nix/store/j42mahch5f0jvfmayhzwb
b88sw36fvah-linux-6.1.8.tar.gz'
real 0m11,884s
user 0m4,130s
sys 0m1,439s
```
Signed-off-by: Jörg Thalheim <joerg@thalheim.io>
Update src/libstore/filetransfer.cc
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
These settings are not needed for libstore at all, they are just used by
the nix daemon *command* for authorization on unix domain sockets. My
moving them to a new configuration struct just in that file, we avoid
them leaking anywhere else.
Also, it is good to break up the mammoth `Settings` struct in general.
Issue #5638 tracks this.
The message is not changed because I do not want to regress in
convenience to the user. Just saying "this connection is not trusted"
doesn't tell them out to fix the issue. The ideal thing to do would be
to somehow parameterize `processCommand` on how the error should be
displayed, so different sorts of connections can display different
information to the user based on how authentication is performed for the
connection in question. This, however, is a good bit more work, so it is
left for the future.
This came up with me thinking about the tcp:// store (#5265). The larger
project is not TCP *per se*, but the idea that it should be possible for
something else to manage access control to services like the Nix Daemon,
and those services simply trust or trust the incoming connection as they
are told. This is a more capability-oriented way of thinking about trust
than "every server implements its own auth separately" as we are used to today.
Its very great that libstore itself already implements just this model,
and so via this refactor I basically want to "enshrine" that so it
continues to be the case.
Since #7478 it's mandatory that `initLibStore()` is called for store
operations. However that's not the case when running `openStore()` in
Perl using the perl-bindings. That breaks e.g. `hydra-eval-jobset` when
built against Nix 2.13 which uses small portions of the store API.
For frameworks it's important that structures are as lazy as possible
to prevent infinite recursions, performance issues and errors that
aren't related to the thing to evaluate. As a consequence, they have
to emit more attributes than strictly (sic) necessary.
However, these attributes with empty values are not useful to the user
so we omit them.
Commit 14bc3ce3d6 (0.13~43) changed the
timestamps in the Nix store from 0 to 1. Update the nix-store man
page to match.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
- `AC_LANG_PUSH(C++)` is needed for the header check
- The library check is hopeless (without lots of third-party macros I
don't feel like getting) because name mangling
Pkg-config would make all this easier. I previously opened
https://github.com/emil-e/rapidcheck/issues/302, I should write a PR
too.
The references set seems to have been unused since `LegacySSHStore`
references were first created in
caa5793b4a.
The method decls never were upstream, and accidentally added by me in
062533f7cd (probably due to `git rerere`).
Sorry!
This reduces the diff from #3746.
No other getDefaultFlakeAttrPaths implementation has this trailing dot,
and the dot can show up in error messages like:
error: flake '...' does not provide attribute 'packages.x86_64-linux.', ...
- Clarify doc comments, Installables::getCursors returns non-empty
vector
- Use vector::at in Installable::getCursor instead of checking for empty
vector and throwing an exception with error message.
Handle the case where none of getActualAttrPaths() actually exists,
in which case instead of returning an empty vector.
This fixes the case where the user misspells the attribute name in nix
search. Instead of getting no search results, now it shows an error with
suggestions.
Also remove InstallableFlake::getCursor() override since it's now
equivalent to the base class version.
Avoid needless work and throwing away invariants.
These conversions date back to when `StorePath` was in Rust and there
were issues with it missing utility methods.
Previously, getDefaultNixPath was called too early: at initialisation
time, before CLI and config have been processed, when `restrictEval` and
`pureEval` both have their default value `false`. Call it when
initialising the EvalState instead, and use `setDefault`.
clangStdenv compiles with a single warning:
```
warning: destructor called on non-final 'nix::PosAdapter' that has virtual functions but non-virtual destructor [-Wdelete-non-abstract-non-virtual-dtor]
```
This fixes the warning by making the destructor of PosAdapter virtual,
deffering to the correct destructor from the concrete child classes.
This has no impact in the end, as none of these classes have specific
destructors.
Technicaly, it may be faster not to have this indirection, but as per
the warning, there is only one place where we have to delete abstract
PosAdapter values.
Not worth bikesheding I guess.
It is still necessary.
Please do your research, or f ask the author, which happens to be me.
An evaluator like this is not an environment where "it compiles, so
it works" will ever hold.
This reverts commit 1c40182b12.
Add an `$` at the end of the `grep` regex. Without it, `checkRef foo`
would always imply `checkRef foo.drv`. We want to tell these situations
apart to more precisely test what is going on.
The issue *seems* to be the cross jobs, which are missing the `CXXFLAGS`
needed to get rapidcheck.
PR #6538 would be really nice to resurrect which will prevent the
`configureFlags` from going out of sync between the regular build and
the cross build again.
Allows checking directory entry type of a single file/directory.
This was added to optimize the use of `builtins.readDir` on some
filesystems and operating systems which cannot detect this information
using POSIX's `readdir`.
Previously `builtins.readDir` would eagerly use system calls to lookup
these filetypes using other interfaces; this change makes these
operations lazy in the attribute values for each file with application
of `builtins.readFileType`.
We had some local variables left over from the older (more
complicated) implementation of this function. They should all be unused,
but one wasn't by mistake.
Delete them all, and replace the one that was still in use as intended.
The original `builtins.getContext` test from
1d757292d0 would have caught this. The
problem is that b30be6b450 adding
`builtins.appendContext` modified that test to make it test too much at
once, rather than adding a separate test.
We now have isolated tests for both functions, and also a property test
showing everything put together (in the form of an eta rule for strings
with context). This is better coverage and properly reproduces the bug.
It's used as the “system” profile in a bunch of places, so better not
touch it. Besides, it doesn't hurt to keep it since it's owned by root
any way, so it doesn't have the `chown` problem that the user profiles
had and that led to wanting to move them on the client-side.
Rather than using `/nix/var/nix/{profiles,gcroots}/per-user/`, put the user
profiles and gcroots under `$XDG_DATA_DIR/nix/{profiles,gcroots}`.
This means that the daemon no longer needs to manage these paths itself
(they are fully handled client-side). In particular, it doesn’t have to
`chown` them anymore (removing one need for root).
This does change the layout of the gc-roots created by nix-env, and is
likely to break some stuff, so I’m not sure how to properly handle that.
Originally there was no `path-info.*`, then there was `path-info.hh`,
then there was `path-info.cc`, but only for new things. Moving this
stuff over makes everything consistent.
Among all the characters that are allowed in a URL, both the percentage
sign "%" and the single quotation mark "'" needs escaping when written
as a environment variable in a systemd service file. While the single
quotation mark may be rare, the percentage sign is widely used to escape
characters in a URL. This is especially common in proxy setting, where
username and password may contain special characters that need
percentage escaping. This patch applies the following replacements:
% -> %%
' -> \'
Instead of needing to run `nix show-config --json | jq -r
'."warn-dirty".value'` to view the value of `warn-dirty`, you can now
run `nix show-config warn-dirty`.
This should be a non-empty set, and so we don't want people doing this
by accident. We remove the zero-0 constructor with a little inheritance
trickery.
`DerivedPath::Built` and `DerivationGoal` were previously using a
regular set with the convention that the empty set means all outputs.
But it is easy to forget about this rule when processing those sets.
Using `OutputSpec` forces us to get it right.
It appears that on current macOS versions, our use of poll() to detect
client disconnects no longer works. As a workaround, poll() for
POLLRDNORM, since this *will* wake up when the client has
disconnected. The downside is that it also wakes up when input is
available. So just sleep for a bit in that case. This means that on
macOS, a client disconnect may take up to a second to be detected,
but that's better than not being detected at all.
Fixes#7584.
This way the links are clearly within the manual (ie not absolute paths),
while allowing snippets to reference the documentation root reliably,
regardless of at which base url they're included.
mdbook-linkcheck is not consistent about its warning setting.
It disables some warnings, but not the warnings about lack of
fragment checking support; hence the extra filtering.
Prior to this change, we had a bunch of ad-hoc string manipulation code
scattered around. This made it hard to figure out what data model for
string contexts is.
Now, we still store string contexts most of the time as encoded strings
--- I was wary of the performance implications of changing that --- but
whenever we parse them we do so only through the
`NixStringContextElem::parse` method, which handles all cases. This
creates a data type that is very similar to `DerivedPath` but:
- Represents the funky `=<drvpath>` case as properly distinct from the
others.
- Only encodes a single output, no wildcards and no set, for the
"built" case.
(I would like to deprecate `=<path>`, after which we are in spitting
distance of `DerivedPath` and could maybe get away with fewer types, but
that is another topic for another day.)
this form is much easier to maintain (also with minimal diffs), and
allows for more details on each operator.
this change a purely mechanical transformation, without changing any contents.
macOS doesn't have user namespacing, so the gid of the builder needs
to be nixbld. The logic got "has sandboxing enabled" confused with
"has user namespaces".
Fixes#7529.
This basically reverts 6e5165b773.
It fixes errors like
sandbox-exec: <internal init prelude>:292:47: unable to open sandbox-minimal.sb: not found
when trying to run a development Nix installed in a user's home
directory.
Also, we're trying to minimize the number of installed files
to make it possible to deploy Nix as a single statically-linked
binary.
The `fish_add_path` function is only available for fish 3.2.0 or newer,
and not on older versions.
This commit adds an alternative way to update the PATH when
`fish_add_path` does not exist.
Adds a new boolean structured attribute
`outputChecks.<output>.unsafeDiscardReferences` which disables scanning
an output for runtime references.
__structuredAttrs = true;
outputChecks.out.unsafeDiscardReferences = true;
This is useful when creating filesystem images containing their own embedded Nix
store: they are self-contained blobs of data with no runtime dependencies.
Setting this attribute requires the experimental feature
`discard-references` to be enabled.
Previously addTempRoot() acquired the LocalStore state lock and waited
for the garbage collector to reply. If the garbage collector is in the
same process (as it the case with auto-GC), this would deadlock as
soon as the garbage collector thread needs the LocalStore state lock.
So now addTempRoot() uses separate Syncs for the state that it
needs. As long at the auto-GC thread doesn't call addTempRoot() (which
it shouldn't), it shouldn't deadlock.
Fixes#3224.
This also moves the file handle into its own Sync object so we're not
holding the _state while acquiring the file lock. There was no real
deadlock risk here since locking a newly created file cannot block,
but it's still a bit nicer.
This has the same goal as b13fd4c58e81b2b2b0d72caa5ce80de861622610,but
achieves it in a different way in order to not break
`nix why-depends --derivation`.
In principle, this should avoid deadlocks where two instances of Nix are
holding a shared lock on big-lock and are both waiting to get an
exclusive lock.
However, it seems like `flock(2)` is supposed to do this automatically,
so it's not clear whether this is actually where the problem comes from.
Without the change checks issue the fllowing warning:
$ nix flake check
trace: warning: The option `nix.useSandbox' defined in `makeTest parameters' has been renamed to `nix.settings.sandbox'.
trace: warning: The option `nix.useSandbox' defined in `makeTest parameters' has been renamed to `nix.settings.sandbox'.
trace: warning: The option `nix.maxJobs' defined in `makeTest parameters' has been renamed to `nix.settings.max-jobs'.
...
This makes 'nix develop' set the Linux personality in the same way
that the actual build does, allowing a command like 'nix develop
nix#devShells.i686-linux.default' on x86_64-linux to work correctly.
Without this, the error is lost, and it makes for a hard to debug
situation. Also remove some of the busyness inside the sqlite_open_v2
args.
The errcode returned is not the extended one. The only way to make open
return an extended code, would be to add SQLITE_OPEN_EXRESCODE to the
flags. In the future it might be worth making this change,
which would also simplify the existing SQLiteError code.
First, logic is consolidated in the shell script instead of being spread
between them and makefiles. That makes understanding what is going on a
little easier.
This would not be super interesting by itself, but it gives us a way to
debug tests more easily. *That* in turn I hope is much more compelling.
See the updated manual for details.
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
* doc/book.toml: Improve config
- `title` value will be added to the HTML <title> - here</title>
- `git-repository-url` adds a link to the GitHub repo in the top right corner
- `edit-url-template` adds an edit link, inviting contributions
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
This makes 'nix build' work on paths (which will be copied to the
store) and store paths (returned as is). E.g. the following flake
output attributes can be built using 'nix build .#foo':
foo = ./src;
foo = self.outPath;
foo = builtins.fetchTarball { ... };
foo = (builtins.fetchTree { .. }).outPath;
foo = builtins.fetchTree { .. } + "/README.md";
foo = builtins.storePath /nix/store/...;
Note that this is potentially risky, e.g.
foo = /.;
will cause Nix to try to copy the entire file system to the store.
What doesn't work yet:
foo = self;
foo = builtins.fetchTree { .. };
because we don't handle attrsets with an outPath attribute in it yet,
and
foo = builtins.storePath /nix/store/.../README.md;
since result symlinks have to point to a store path currently (rather
than a file inside a store path).
Fixes#7417.
They did not include the detailed error message, losing essential
information for troubleshooting.
Example message:
warning: creating statement 'insert or rplace into NARs(cache, hashPart, namePart, url, compression, fileHash, fileSize, narHash, narSize, refs, deriver, sigs, ca, timestamp, present) values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, 1)': at offset 10: SQL logic error, near "rplace": syntax error (in '/tmp/nix-shell.grQ6f7/nix-test/tests/binary-cache/test-home/.cache/nix/binary-cache-v6.sqlite')
It's not the best example; more important information will be in
the message for e.g. a constraint violation.
I don't see why this specific error is printed as a warning, but
that's for another commit.
Unsetting `build-users-group` (without `auto-allocate-uids` enabled)
gives the following error:
```
src/libstore/lock.cc:25: static std::unique_ptr<nix::UserLock> nix::SimpleUserLock::acquire(): Assertion `settings.buildUsersGroup != ""' failed.
```
Fix the logic in `useBuildUsers` and document the default value
for `build-users-group`.
This makes the position object used in exceptions abstract, with a
method getSource() to get the source code of the file in which the
error originated. This is needed for lazy trees because source files
don't necessarily exist in the filesystem, and we don't want to make
libutil depend on the InputAccessor type in libfetcher.
Make everything be in the form "while ..." (most things were already),
and in particular *don't* use other propositions that must go after or
before specific "while ..." clauses to make sense.
When debugging nix expressions the outermost trace tends to be more useful
than the innermost. It is therefore printed last to save developers from
scrolling.
We used to set enforceDeterminism to true in the settings (by default)
and thus did send a non-zero value over the wire. The value should
probably be ignored as it should only matter if nrRounds is non-zero
as well.
Having the old code here where the value is expected to be zero only
works with the same version of Nix where we are sending zero. We
should always test this against older Nix versions being client or
server as otherwise upgrade in larger networks might be a pain.
Fixes 8e0946e8df
Fix#6209
When trying to run `nix log <installable>`, try first to resolve the derivation pointed to
by `<installable>` as it is the resolved one that holds the build log.
This has a couple of shortcomings:
1. It’s expensive as it requires re-reading the derivation
2. It’s brittle because if the derivation doesn’t exist anymore or can’t
be resolved (which is the case if any one of its build inputs is missing),
then we can’t access the log anymore
However, I don’t think we can do better (at least not right now).
The alternatives I see are:
1. Copy the build log for the un-resolved derivation. But that means a
lot of duplication
2. Store the results of the resolving in the db. Which might be the best
long-term solution, but leads to a whole new class of potential
issues.
this avoids incorrect rendering on the man pages, since `lowdown`
neither parses the anchor syntax nor HTML.
this should rather be fixed in lowdown, as adding more anchors
would otherwise produce ever more noise and error-prone repetition.
* docs: Use secret-key-files when demonstrating post-build-hooks
The docs used to recommend calling `nix store sign` in a post-build
hook, but on more recent versions of nix, this results in unsigned
store paths being copied into binary caches. See
https://github.com/NixOS/nix/issues/6960 for details.
Instead, use the `secret-key-files` config option, which signs all
locally-built derivations with the private key.
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
These only functioned if a very narrow combination of conditions held:
- The result path does not yet exist (--check did not result in
repeated builds), AND
- The result path is not available from any configured substituters, AND
- No remote builders that can build the path are available.
If any of these do not hold, a derivation would be built 0 or 1 times
regardless of the repeat option. Thus, remove it to avoid confusion.
being too specific about it requires more maintenance (or otherwise
produced more confusion and churn), since these points of contact change
over time.
this is a quick half-fix for command line examples, as discussed
discussed in [1].
[1]: https://github.com/NixOS/nix/pull/7389
examples which look like this
$ foo bar
baz
are confusing for Unix shell beginners, because it's hard to discern
what is supposed to be entered into the actual command line when the
convention of prefixing `$` is not known, as barely any real-world shell
looks that way any more.
this change prevents selecting the prompt part with the mouse in the
HTML representation of the Nix manual.
it does not prevent selecting the output part of the shell example.
it also does not address that the copy button provided by mdBook takes
the entire sample, including the prompts, into the clipboard.
The old way was not correct.
Here is an example:
```
$ nix-instantiate --eval --expr 'let x = a: throw "asdf"; in x 1' --show-trace
error: asdf
… while evaluating 'x'
at «string»:1:9:
1| let x = a: throw "asdf"; in x 1
| ^
… from call site
at «string»:1:29:
1| let x = a: throw "asdf"; in x 1
| ^
```
and yet also:
```
$ nix-instantiate --eval --expr 'let x = a: throw "asdf"; in x' --show-trace
<LAMBDA>
```
Here is the thing: in both cases we are evaluating `x`!
Nix is a higher-order languages, and functions are a sort of value. When
we write `x = a: ...`, `a: ...` is the expression that `x` is being
defined to be, and that is already a value. Therefore, we should *never*
get an trace that says "while evaluating `x`", because evaluating `a:
...` is *trival* and nothing happens during it!
What is actually happening here is we are applying `x` and evaluating
its *body* with arguments substituted for parameters. I think the
simplest way to say is just "while *calling* `x`", and so that is what I
changed it to.
We need to close the GC server socket before shutting down the active
GC client connections, otherwise a client may (re)connect and get
ECONNRESET. But also handle ECONNRESET for resilience.
Fixes random failures like
GC socket disconnected
connecting to '/tmp/nix-shell.y07M0H/nix-test/default/var/nix/gc-socket/socket'
sending GC root '/tmp/nix-shell.y07M0H/nix-test/default/store/kb5yzija0f1x5xkqkgclrdzldxj6nnc6-non-blocking'
reading GC root from client: error: unexpected EOF reading a line
1 store paths deleted, 0.00 MiB freed
error: reading from file: Connection reset by peer
in gc-non-blocking.sh.
It calls strlen() on the input (rather than simply copying at most
`size` bytes), which can fail if the input is not zero-terminated and
is inefficient in any case.
Fixes#7347.
why-depends assumed that we knew the output path of the second argument.
For CA derivations, we might not know until it's built. One way to solve
this would be to build the second installable to get the output path.
In this case we don't need to, though. If the first installable (A)
depends on the second (B), then getting the store path of A will
necessitate having the store path B. The contrapositive is, if the store
path of B is not known (i.e. it's a CA derivation which hasn't been
built), then A does not depend on B.
We shouldn't skip this if the supplementary group list is empty,
because then the sandbox won't drop the supplementary groups of the
parent (like "root").
The new experimental feature 'cgroups' enables the use of cgroups for
all builds. This allows better containment and enables setting
resource limits and getting some build stats.
It occurred when a output of the dependency was already available,
so it didn't need rebuilding and didn't get added to the
inputDrvOutputs.
This process-related info wasn't suitable for the purpose of finding
the actual input paths for the builder. It is better to do this in
absolute terms by querying the store.
This change is needed to support aws-sdk-cpp 1.10 and newer.
I opted not to make this dependent on the sdk version because
the crt dependency has been in the interface of the older
sdk as well, and it was only coincidence that libstore didn't
make use of any privately defined symbols directly.
When calling `builtins.readFile` on a store path, the references of that
path are currently added to the resulting string's context.
This change makes those references the *possible* context of the string,
but filters them to keep only the references whose hash actually appears
in the string, similarly to what is done for determining the runtime
references of a path.
Cgroups are now only used for derivations that require the uid-range
range feature. This allows auto UID allocation even on systems that
don't have cgroups (like macOS).
Also, make things work on modern systems that use cgroups v2 (where
there is a single hierarchy and no "systemd" controller).
after discussing this with multiple people, I'm convinced that "build
task" is more precise: a derivation is not an action, but inert until it
is built. also it's easier to pronounce.
proposal: use "build task" for the generic concept "description of how
to derive new files from the contents of existing files". then it will
be easier to distinguish what we mean by "derivation" (a specific data
structure and Nix language value type) and "store derivation" (a
serialisation of a derivation into a file in the Nix store).
readline is not re-entrant, so entering the debugger from the
completioncallback results in an eventual segfault.
The workaround is to temporarily disable the debugger when searching
for possible completions.
In addition to consistency, the fancy "Copy to clipboard" button on the
website will copy the prompt character. Retaining the prompt character
would mean having to edit each command after pasting in the terminal.
Call it as `['nix', '__build-remote', ... ]` rather than the previous
`["__build-remote", "nix __build-remote", ... ]` which seemed to have
been most likely unintended
The description of the --profile option talks about the "update" operation.
This is probably meant for operations such as "nix profile install", but the
same option is reused in other subcommands, which do not update the profile,
such as "nix profile {list,history,diff-closures}".
We update the description to make sense in both contexts.
Currently, Nix passes `-a` when it runs commands on a remote machine via
SSH, which disables agent forwarding. This causes issues when the
`ForwardAgent` option is set in SSH config files, as the command line
operation always overrides those.
In particular, this causes issues if the command being run is `sudo`
and the remote machine is configured with the equivalent of NixOS's
`security.pam.enableSSHAgentAuth` option. Not allowing SSH agent
forwarding can cause authentication to fail unexpectedly.
This can currently be worked around by setting `NIX_SSHOPTS="-A"`, but
we should defer to the options in the SSH config files to be least
surprising for users.
This is a really old test case (which was originally written before the
proper Nix syntax). The tested deep comparison behavior was implemented
and reverted soon after due to performance problems, but it has been
restored in today's Nix again (thanks to the derivation comparison
optimization, presumably).
Specifically, explain why Nix does not _re_evaluate paths during a
`nix repl` session. This is a thing that bit me while playing around
with paths and antiquotation in `nix repl` while reading the Nix
language tutorial at https://nix.dev/tutorials/nix-language.
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
* Clarify the documentation of foldl': That the arguments are forced
before application (?) of `op` is necessarily true. What is important
to stress is that we force every application of `op`, even when the
value turns out to be unused.
* Move the example before the comment about strictness to make it less
confusing: It is a general example and doesn't really showcase anything
about foldl' strictness.
* Add test cases which nail down aspects of foldl' strictness:
* The initial accumulator value is not forced unconditionally.
* Applications of op are forced.
* The list elements are not forced unconditionally.
After we've send "\2\n" to the parent, we can't send a serialized
exception anymore. It will show up garbled like
$ nix-build --store /tmp/nix --expr 'derivation { name = "foo"; system = "x86_64-linux"; builder = "/foo/bar"; }'
this derivation will be built:
/nix/store/xmdip0z5x1zqpp6gnxld3vqng7zbpapp-foo.drv
building '/nix/store/xmdip0z5x1zqpp6gnxld3vqng7zbpapp-foo.drv'...
ErrorErrorEexecuting '/foo/bar': No such file or directory
error: builder for '/nix/store/xmdip0z5x1zqpp6gnxld3vqng7zbpapp-foo.drv' failed with exit code 1
While trying to use an alternate directory for my Nix installation, I
noticed that nix's output didn't reflect the updated state
directory. This patch corrects that and now prints the warning before
attempting to create the directory (if the directory creation fails,
it wouldn't have been obvious why nix was attempting to create the
directory in the first place).
With this patch, I now get the following warning:
warning: '/home/deck/.var/app/org.nixos.nix/var/nix' does not
exist, so Nix will use '/home/deck/.local/share/nix/root' as a
chroot store
This commit adds a test covering the discrepancy between parseDrvName's
implementation and documentation (the discrepancy was eliminated in the previous
commit).
The documentation for `parseDrvName` does not agree with the implementation when
the derivation name contains a dash which is followed by something that is
neither a letter nor a digit. This commit corrects the documentation to agree
with the implementation.
Previously the MANPATH was set even if MANPATH was empty beforehand
which resulted in a MANPATH of only ~/.nix-profile/share/man which
omitted the default man page directory (commonly /opt/local/share/man)
from man page results.
If we don't have any github token, we won't be able to fetch private
repos, but we are also more likely to run into API limits since
we don't have a token. To mitigate this only ever use the github api
if we actually have a token.
Older versions of Fish (such as those bundled with Ubuntu LTS 22.04) do
not support return outside of functions. We need to use the equivalent
exit instead.
The current definition of `intersectAttrs` is incorrect:
> Return a set consisting of the attributes in the set e2 that also exist in the
> set e1.
Recall that (Nix manual, section 5.1):
> An attribute set is a collection of name-value-pairs (called attributes)
According to the existing description of `intersectAttrs`, the following should
evaluate to the empty set, since no key-value *pair* (i.e. attribute) exists in
both sets:
```
builtins.intersectAttrs { x=3; } {x="foo";}
```
And yet:
```
nix-repl> builtins.intersectAttrs { x=3; } {x="foo";}
{ x = "foo"; }
```
Clearly the intent here was for the *names* of the resulting attribute set to be
the intersection of the *names* of the two arguments, and for the values of the
resulting attribute set to be the values from the second argument.
This commit corrects the definition, making it match the implementation and intent.
Make sure that people who run Nix in non-interactive mode (and so don't have the possibility to interactively accept the individual flake configuration settings) are aware of this flag.
Fix#7086
These settings seem harmless, they control the same polling
functionality that timeout does, but with different behavior. Should
be safe for untrusted users to pass in.
I just had a colleague get confused by the previous phrase for good
reason. "valid" sounds like an *objective* criterion, e.g. and *invalid
signature* would be one that would be trusted by no one, e.g. because it
misformatted or something.
What is actually going is that there might be a signature which is
perfectly valid to *someone else*, but not to the user, because they
don't trust the corresponding public key. This is a *subjective*
criterion, because it depends on the arbitrary and personal choice of
which public keys to trust.
I therefore think "trustworthy" is a better adjective to use. Whether
something is worthy of trust is clearly subjective, and then "trust"
within that word nicely evokes `trusted-public-keys` and friends.
The history is not critical to the functionality of nix repl, so it's
enough to warn here, rather than refuse to start if the directory Nix
thinks the history should live in can't be created.
- call close explicitly in writeFile to prevent the close exception
from being ignored
- fsync after writing schema file to flush data to disk
- fsync schema file parent to flush metadata to disk
https://github.com/NixOS/nix/issues/7064
Remove the `verify TLS: Nix CA file = 'blah'` message that Nix used to print when fetching anything as it's both useless (`libcurl` prints the same info in its logs) and misleading (gives the impression that a new TLS connection is being established which might not be the case because of multiplexing. See #7011 )
Mainly:
- Try to triangulate between narrative that framed this as
a new/easy process and the need for a reference that will
not quickly grow stale.
- Fix a ~continuity issue where the text was talking about
"your Cachix cache" before saying that you'd need to make
a Cachix cache to enable the installer tests.
- Adopt suggestion on titling, and nest subtitles in the
installer test section.
This commit adds an optional `__impure` parameter to fetchurl.nix, which allows
the caller to use `libfetcher`'s fetcher in an impure derivation. This allows
nixpkgs' patch-normalizing fetcher (fetchpatch) to be rewritten to use nix's
internal fetchurl, thereby eliminating the awkward "you can't use fetchpatch
here" banners scattered all over the place.
See also: https://github.com/NixOS/nixpkgs/pull/188587
This runs the installer in a QEMU VM. Unlike the old installer test
that ran inside a declaratively built RedHat/Debian image, this uses
an image from Vagrant.
Before this patch, installing Nix using the Fish shell did not
work because Fish wasn't configured to add Nix to the PATH. Some
options in #1512 offered workarounds, but they typically involve
extra plugins or packages.
This patch adds native, out-of-the-box support for the Fish shell.
Note that Fish supports a `conf.d` directory, which is intended
for exactly use cases like this: software projects distributing
shell snippets. This patch takes advantage of it. The installer
doesn't append any Nix loader behavior to any Fish config file.
Because of that, the uninstall process is smooth and a reinstall
obliterates the existing nix.fish files that we place instead of
bothering the user with a backup / manual removal.
Both single-user and multi-user cases are covered. It has been
tested on Ubuntu, and a Mac with MacPorts, homebrew, and the
Fish installer pkg.
Closes#1512
Co-authored-by: Graham Christensen <graham@grahamc.com>
Implements the approach suggested by feedback on PR #6994, where
tempdir paths are created in the store (now with an exclusive lock).
As part of this work, the currently-broken and unused
`createTempDirInStore` function is updated to create an exclusive lock
on the temp directory in the store.
The GC now makes a non-blocking attempt to lock any store directories
that "look like" the temp directories created by this function, and if
it can't acquire one, ignores the directory.
Stdenv sets this to a bash that doesn't have readline/completion
support, so running 'nix (develop|shell)' inside a 'nix develop' gives
you a crippled shell. So let's just ignore the derivation's $SHELL.
This could break interactive use of build phases that use $SHELL, but
they appear to be fairly rare.
renaming section headers and changing manually set `id`s will break URLs
in the wild.
this change allows keeping track of all changes to ensure backwards
compatibility.
Disables the SA_RESTART behavior on macOS which causes:
> Restarting of pending calls is requested by setting the SA_RESTART bit
> in sa_flags. The affected system calls include read(2), write(2),
> sendto(2), recvfrom(2), sendmsg(2) and recvmsg(2) on a communications
> channel or a slow device (such as a terminal, but not a regular file)
> and during a wait(2) or ioctl(2).
From: https://man.openbsd.org/sigaction#SA_RESTART
This being set on macOS caused a bug where read() calls to the daemon
socket were blocking after a SIGINT was received. As a result,
checkInterrupt was never reached even though the signal was received
by the signal handler thread.
On Linux, SA_RESTART is disabled by default. This probably effects
other BSDs but I don’t have the ability to test it there right now.
readDerivation is pretty slow, and while it may not be significant for
some use cases, on things like ghc-nix where we have thousands of
derivations is really slows things down.
So, this just doesn’t do the impure derivation check if the impure
derivation experimental feature is disabled. Perhaps we could cache
the result of isPure() and keep the check, but this is a quick fix to
for the slowdown introduced with impure derivations features in 2.8.0.
This matches the behavior of bash. We don’t want to add a space after
completion on attrs. Uses -S.
Switches to new compadd style comppletions instead of _describe.
Shouldn’t have any negative issues from what I can tell.
The darwin_stop_world implementation is slightly different. sp goes to
altstack_lo instead of lo in this case. Assuming that is an
implementation detail.
But the fix is the same, when we detect alstack_lo outside of the
expected stack range, we reset it to hi - stack_limit.
Here stack_limit is calculated with pthread_get_stacksize_np since
that is the BSD equivalent to pthread_attr_getstacksize.
A [recent-ish change](https://github.com/NixOS/nix/pull/6676) logs a warning when a potentially counterintuitive situation happens.
This now causes the multi-user installer to [emit a warning](https://github.com/NixOS/nixpkgs/issues/189043) when it's doing
the "seed the Nix database" step via a low-level `nix-store --load-db` invocation.
`nix-store` functionality implementations don't actually use profiles or channels or homedir as far as i can tell. So why are we
hitting this code at all?
Well, the current command approach for functionality here builds a [fat `nix` binary](https://github.com/NixOS/nix/blob/master/src/nix/local.mk#L23-L26) which has _all_ the functionality of
previous individual binaries (nix-env, nix-store, etc) bundled in, then [uses the invocation name](https://github.com/NixOS/nix/blob/master/src/nix/main.cc#L274-L277) to select the
set of commands to expose. `nix` itself has this behavior, even when just trying to parse the (sub)command and arguments:
```
dave @ davembp2
$ nix
error: no subcommand specified
Try 'nix --help' for more information.
dave @ davembp2
$ sudo nix
warning: $HOME ('/Users/dave') is not owned by you, falling back to the one defined in the 'passwd' file
error: no subcommand specified
Try 'nix --help' for more information.
dave @ davembp2
$ HOME=~root sudo nix
error: no subcommand specified
Try 'nix --help' for more information.
```
This behavior can also be seen pretty easily with an arbitrary `nix-store` invocation:
```
dave @ davembp2
$ nix-store --realize
dave @ davembp2
$ sudo nix-store --realize # what installer is doing now
warning: $HOME ('/Users/dave') is not owned by you, falling back to the one defined in the 'passwd' file
dave @ davembp2
$ sudo HOME=~root nix-store --realize # what this PR effectively does
dave @ davembp2
$
```
this simplifies the setup a lot, and avoids weird looking `./file.md`
links showing up.
it also does not show regular URLs any more. currently the command
reference only has few of them, and not showing them in the offline
documentation is hopefully not a big deal.
instead of building more special-case solutions, clumsily preprocessing
the input, or issuing verbal rules on dealing with URLs, should better
be solved sustainably by not rendering relative links in `lowdown`:
https://github.com/kristapsdz/lowdown/issues/105
This was caused by -L calling setLogFormat() again, which caused the
creation of a new progress bar without destroying the old one. So we
had two progress bars clobbering each other.
We should change 'logger' to be a smart pointer, but I'll do that in a
future PR.
Fixes#6931.
98e361ad4c introduced a regression where
previously stored attributes were replaced by placeholders. As a
result, a command like 'nix build nixpkgs#hello' had to be executed at
least twice to get caching.
This code does not seem necessary for suggestions to work.
This issue made it impossible for clients using a serve protocol of
version <= 2.3 to use the `cmdBuildDerivation` command of servers using
a protocol of version >= 2.6. The faulty version check makes the server
send back build outputs that the client is not expecting.
This hang for some reason didn't trigger in the Nix build, but did
running 'make installcheck' interactively. What happened:
* Store::addMultipleToStore() calls a SinkToSource object to copy a
path, which in turn calls LegacySSHStore::narFromPath(), which
acquires a connection.
* The SinkToSource object is not destroyed after the last bytes has
been read, so the coroutine's stack is still alive and its
destructors are not run. So the connection is not released.
* Then when the next path is copied, because max-connections = 1,
LegacySSHStore::narFromPath() hangs forever waiting for a connection
to be released.
The fix is to make sure that the source object is destroyed when we're
done with it.
Makes `printValueAsJSON` not copy paths to the store for `nix eval
--json`, `nix-instantiate --eval --json` and `nix-env --json`.
Fixes https://github.com/NixOS/nix/issues/5612
RewritingSink can handle being fed input where a reference crosses a
chunk boundary. we don't need to load the whole source into memory, and
in fact *not* loading the whole source lets nix build FODs that do not
fit into memory (eg fetchurl'ing data files larger than system memory).
Some activities are numerous but usually very short (e.g. copying a
source file to the store) which would cause a lot of flickering. So
only show activities that have been running for at least 10 ms.
Nix veterans intuitively know what the following terms mean. They are
used in several places in the nix documentation, but never defined:
- local store
- remote store
- binary cache
- substituter
In particular, I found the last two terms to be confusingly similar.
Let's give definitions for them.
this is not as compact any more, but it more closely resembles the
chapter structure, and clearly shows that the closure property is the
key idea on which most of Nix operates.
clarify that we are copying between different stores. we have not
introduced that notion or why it would be interesting, but for now it
should be fine to keep it in context of the store directory.
we could move that later to a more detailed explanation of different
store types.
at this level of abstraction we do not really care about build instructions or what they are, and also build instructions including their arguments really amount to the build task.
group description of data instead of spreading it across the section.
that should help direct skimming. as it turns out, people do not
actually read any of that.
this leaves open implementation details, especially about store paths
and file system objects, and allows explaining them together were it is
more appropriate. also leaves room to carefully introduce the key
insight behind Nix: applying results from programming language theory to
the operating system paradigm of files and processes.
while this may eventually introduce ugly diffs, the table will now
render readably on the terminal (e.g. for `man nix` or `nix --help`)
without further intervention.
while it appears a bit much for the overview, this way we set the stage
for going directly into data types when describing the store, instead of
first having to say what build tasks are and how they relate to build
plans.
this displays correct composition again. build inputs and build results
are not part of build plans in terms of data objects.
also this is a much less complicated setup. this will be the first
impression of architecture, and we want to get it right.
convention: describe every data type in prose, and illustrate with
a class diagram, and a textual representation of an abstract
data type.
right now we save ourselves the trouble of doing class diagrams, we can
add them later. but they are important.
use file Contents instead of Data, as that flows more naturally in the
prose.
simplify explanation of the idea behind scanning for store paths
remove references to unfinished sections.
attempt to explain used and documented terminology, as well as how
the declarative programming paradigm relates to building software.
in the future one could highlight encouraged terms to shape future
material into higher consistency.
the diagram is a first approximation and only covers that same section.
of course there is much more going on, and other features should at some
point also be illustrated.
we also have to think about presentation format and technicalities
behind it. the manual has to render to `man`, but we may want something
more refined for web view.
there should be a meta section for each chapter to give motivation of
the presented structure. the structure itself is visible from the table
of contents.
idea: sections could be read in different orders by linking them in
different ways (e.g. depth-first or breadth-first). adding hard-coded
transitions makes that confusing.
trying to capture alternative terms in one go here, mirroring everyday
use:
derivation - build plan
realise - execute build
there will be more of that sort.
The idea and most of the execution are @fricklerhandwerk's. I changed a
few things best I could based on @edolstra's corrections, and a Bazel
glossary.
Valentin Gagarin <valentin@fricklerhandwerk.de>
The current docs are all "how to do things" and no "what is Nix" or "why
are things the way they are".
I see lots of misconception on the wider internet, and I also think we
would benefit from a "living document" to answer some questions people
currently turn to the thesis for.
I think a new section of the manual can address all these issues.
it is out of date, all over the place in level of detail, is really
about `nixpkgs`, and in general instructions should not be part of
a reference manual.
also:
- update redirects and internal links
- use "Nix language" consistently
That flag breaks `-lc++fs` (introducing a duplicate symbol for some
reason). Besides, it was apparently needed for bzip2, but we're not using bzip2
anymore.
Rather than directly copying the source to its dest, copy it first to a
temporary location, and eventually move that temporary.
That way, the move is at least atomic from the point-of-view of the destination
The recursive copy from the stl doesn’t exactly do what we need because
1. It doesn’t delete things as we go
2. It doesn’t keep the mtime, which change the nars
So re-implement it ourselves. A bit dull, but that way we have what we want
In `nix::rename`, if the call to `rename` fails with `EXDEV` (failure
because the source and the destination are in a different filesystems)
switch to copying and removing the source.
To avoid having to re-implement the copy manually, I switched the
function to use the c++17 `filesystem` library (which has a `copy`
function that should do what we want).
Fix#6262
Once a derivation goal has been completed, we check whether or not
this goal was meant to be repeated to check its output.
An early return branch was preventing the worker to reach that repeat
code branch, hence breaking the --check command (#2619).
It seems like this early return branch is an artifact of a passed
refactoring. As far as I can tell, buildDone's main branch also
cleanup the tmp directory before returning.
By default, Nix sets the "cores" setting to the number of CPUs which are
physically present on the machine. If cgroups are used to limit the CPU
and memory consumption of a large Nix build, the OOM killer may be
invoked.
For example, consider a GitLab CI pipeline which builds a large software
package. The GitLab runner spawns a container whose CPU is limited to 4
cores and whose memory is limited to 16 GiB. If the underlying machine
has 64 cores, Nix will invoke the build with -j64. In many cases, that
level of parallelism will invoke the OOM killer and the build will
completely fail.
This change sets the default value of "cores" to be
ceil(cpu_quota / cpu_period), with a fallback to
std::thread::hardware_concurrency() if cgroups v2 is not detected.
The workaround for "Some distros patch Linux" mentioned in
local-derivation-goal.cc will not help in the `--option
sandbox-fallback false` case. To provide the user more helpful
guidance on how to get the sandbox working, let's check to see if the
`/proc` node created by the aforementioned patch is present and
configured in a way that will cause us problems. If so, give the user
a suggestion for how to troubleshoot the problem.
local-derivation-goal.cc contains a comment stating that "Some distros
patch Linux to not allow unprivileged user namespaces." Let's give a
pointer to a common version of this patch for those who want more
details about this failure mode.
This commit causes nix to `warn()` if sandbox setup has failed and
`/proc/self/ns/user` does not exist. This is usually a sign that the
kernel was compiled without `CONFIG_USER_NS=y`, which is required for
sandboxing.
This commit uses `warn()` to notify the user if sandbox setup fails
with errno==EPERM and /proc/sys/user/max_user_namespaces is missing or
zero, since that is at least part of the reason why sandbox setup
failed.
Note that `echo -n 0 > /proc/sys/user/max_user_namespaces` or
equivalent at boot time has been the recommended mitigation for
several Linux LPE vulnerabilities over the past few years. Many users
have applied this mitigation and then forgotten that they have done
so.
The failure modes for nix's sandboxing setup are pretty complicated.
When nix is unable to set up the sandbox, let's provide more detail
about what went wrong. Specifically:
* Make sure the error message includes the word "sandbox" so the user
knows that the failure was related to sandboxing.
* If `--option sandbox-fallback false` was provided, and removing it
would have allowed further attempts to make progress, let the user
know.
`--override-input` id snarky because it takes two arguments, so it
doesn't play well when completed in the middle of the CLI (since the
argument just after gets interpreted as its second argument). So use
`--update-input` instead
I recently got fairly confused why the following expression didn't have
any effect
{
description = "Foobar";
inputs.sops-nix = {
url = github:mic92/sops-nix;
inputs.nixpkgs_22_05.follows = "nixpkgs";
};
}
until I found out that the input was called `nixpkgs-22_05` (please note
the dash vs. underscore).
IMHO it's not a good idea to not throw an error in that case and
probably leave end-users rather confused, so I implemented a small check
for that which basically checks whether `follows`-declaration from
overrides actually have corresponding inputs in the transitive flake.
In fact this was done by accident already in our own test-suite where
the removal of a `follows` was apparently forgotten[1].
Since the key of the `std::map` that holds the `overrides` is a vector
and we have to find the last element of each vector (i.e. the override)
this has to be done with a for loop in O(n) complexity with `n` being
the total amount of overrides (which shouldn't be that large though).
Please note that this doesn't work with nested expressions, i.e.
inputs.fenix.inputs.nixpkgs.follows = "...";
which is a known problem[2].
For the expression demonstrated above, an error like this will be
thrown:
error: sops-nix has a `follows'-declaration for a non-existant input nixpkgs_22_05!
[1] 2664a216e5
[2] https://github.com/NixOS/nix/issues/5790
- Don't use `printf` for the expected result, but just use bash's `$' '`
litteral strings
- Quote the `nix` call result
- Invert the order in the comparisons (just because it feels more
natural)
Defers completion of flake inputs until the whole command line is parsed
so that we know what flakes we need to complete the inputs of.
Previously, `nix build flake --update-input <Tab>` always behaved like
`nix build . --update-input <Tab>`.
Before #5150 the copy-to-store phase of the install was idempotent,
but the recursive cp isn't. This is probably baiting a few people
into trying corrective installs that will fail.
Allow `nix build flake1 flake2 --update-input <Tab>` to complete the
inputs of both flakes.
Also do tilde expansion so that `nix build ~/flake --update-input <Tab>`
works.
Bring back the possibility to copy CA paths with no reference (like the
outputs of FO derivations or stuff imported at eval time) between stores
that have a different prefix.
A mips64el Linux MIPS kernel can execute userspace code using any of
three ABIs:
mips64el-linux-*abin64
mips64el-linux-*abin32
mipsel-linux-*
The first of these is the native 64-bit ABI, and the only ABI with
64-bit pointers; this is sometimes called "n64". The last of these is
the old legacy 32-bit ABI, whose binaries can execute natively on
32-bit MIPS hardware; this is sometimes called "o32".
The second ABI, "n32" is essentially the 64-bit ABI with 32-bit
pointers and address space. Hardware 64-bit integer/floating
arithmetic is still allowed, as well as the much larger mips64
register set and more-efficient calling convention.
Let's enable seccomp filters for all of these. Likewise for big
endian (mips64-linux-*).
"uid-range" provides 65536 UIDs to a build and runs the build as root
in its user namespace. "systemd-cgroup" allows the build to mount the
systemd cgroup controller (needed for running systemd-nspawn and NixOS
containers).
Also, add a configuration option "auto-allocate-uids" which is needed
to enable these features, and some experimental feature gates.
So to enable support for containers you need the following in
nix.conf:
experimental-features = auto-allocate-uids systemd-cgroup
auto-allocate-uids = true
system-features = uid-range systemd-cgroup
2^18 was overkill. The idea was to enable multiple containers to run
inside a build. However, those containers can use the same UID range -
we don't really care about perfect isolation between containers inside
a build.
Also, run builds in a cgroup namespace (ensuring /proc/self/cgroup
doesn't leak information about the outside world) and mount /sys. This
enables running systemd-nspawn and thus NixOS containers in a Nix
build.
Rather than rely on a nixbld group, we now allocate UIDs/GIDs
dynamically starting at a configurable ID (872415232 by default).
Also, we allocate 2^18 UIDs and GIDs per build, and run the build as
root in its UID namespace. (This should not be the default since it
breaks some builds. We probably should enable this conditional on a
requiredSystemFeature.) The goal is to be able to run (NixOS)
containers in a build. However, this will also require some cgroup
initialisation.
The 2^18 UIDs/GIDs is intended to provide enough ID space to run
multiple containers per build, e.g. for distributed NixOS tests.
2020-07-06 13:50:33 +02:00
453 changed files with 15771 additions and 7303 deletions
@@ -5,3 +5,7 @@ Please include relevant [release notes](https://github.com/NixOS/nix/blob/master
**Testing**
If this issue is a regression or something that should block release, please consider including a test either in the [testsuite](https://github.com/NixOS/nix/tree/master/tests) or as a [hydraJob]( https://github.com/NixOS/nix/blob/master/flake.nix#L396) so that it can be part of the [automatic checks](https://hydra.nixos.org/jobset/nix/master).
**Priorities**
Add :+1: to [pull requests you find important](https://github.com/NixOS/nix/pulls?q=is%3Aopen+sort%3Areactions-%2B1-desc).
@@ -20,8 +20,8 @@ Information on additional installation methods is available on the [Nix download
## Building And Developing
See our [Hacking guide](https://hydra.nixos.org/job/nix/master/build.x86_64-linux/latest/download-by-type/doc/manual/contributing/hacking.html) in our manual for instruction on how to
build nix from source with nix-build or how to get a development environment.
See our [Hacking guide](https://nixos.org/manual/nix/unstable/contributing/hacking.html) in our manual for instruction on how to
to set up a development environment and build Nix from source.
building '/nix/store/ch6llwpr2h8c3jmnf3f2ghkhx59aa97f-unstable.drv' (round 1/2)...
building '/nix/store/ch6llwpr2h8c3jmnf3f2ghkhx59aa97f-unstable.drv' (round 2/2)...
output '/nix/store/6xg356v9gl03hpbbg8gws77n19qanh02-unstable' of '/nix/store/ch6llwpr2h8c3jmnf3f2ghkhx59aa97f-unstable.drv' differs from '/nix/store/6xg356v9gl03hpbbg8gws77n19qanh02-unstable.check' from previous round
Then, add the public key and the cache URL to your `nix.conf`'s
`trusted-public-keys` and `substituters` options:
Then update [`nix.conf`](../command-ref/conf-file.md) on any machine that will access the cache.
Add the cache URL to [`substituters`](../command-ref/conf-file.md#conf-substituters) and the public key to [`trusted-public-keys`](../command-ref/conf-file.md#conf-trusted-public-keys):
Machines that build for the cache must sign derivations using the private key.
On those machines, add the path to the key file to the [`secret-key-files`](../command-ref/conf-file.md#conf-secret-key-files) field in their [`nix.conf`](../command-ref/conf-file.md):
secret-key-files = /etc/nix/key.private
We will restart the Nix daemon in a later step.
# Implementing the build hook
@@ -52,14 +57,12 @@ set -eu
set -f # disable globbing
exportIFS=' '
echo"Signing paths"$OUT_PATHS
nix store sign --key-file /etc/nix/key.private $OUT_PATHS
The following [concept map] shows its main components (rectangles), the objects they operate on (rounded rectangles), and their interactions (connecting phrases):
At the top is the [command line interface](../command-ref/command-ref.md) that drives the underlying layers.
The [Nix language](../language/index.md) evaluator transforms Nix expressions into self-contained *build plans*, which are used to derive *build results* from referenced *build inputs*.
The command line interface and Nix expressions are what users deal with most.
> **Note**
> The Nix language itself does not have a notion of *packages* or *configurations*.
> As far as we are concerned here, the inputs and results of a build plan are just data.
Underlying the command line interface and the Nix language evaluator is the [Nix store](../glossary.md#gloss-store), a mechanism to keep track of build plans, data, and references between them.
It can also execute build plans to produce new data, which are made available to the operating system as files.
A build plan itself is a series of *build tasks*, together with their build inputs.
> **Important**
> A build task in Nix is called [derivation](../glossary.md#gloss-derivation).
Each build task has a special build input executed as *build instructions* in order to perform the build.
The result of a build task can be input to another build task.
The following [data flow diagram] shows a build plan for illustration.
Build inputs used as instructions to a build task are marked accordingly:
Normally, the Nix store directory (typically `/nix/store`) is not
@@ -122,3 +91,16 @@ Most Nix commands interpret the following environment variables:
variable sets the initial size of the heap in bytes. It defaults to
384 MiB. Setting it to a low value reduces memory consumption, but
will increase runtime due to the overhead of garbage collection.
## XDG Base Directory
New Nix commands conform to the [XDG Base Directory Specification], and use the following environment variables to determine locations of various state and configuration files:
To build Nix for the current operating system/architecture use
The following instructions assume you already have some version of Nix installed locally, so that you can use it to set up the development environment. If you don't have it installed, follow the [installation instructions].
This section assumes you are using Nix with [flakes] enabled. See the [next section](#classic-nix) for equivalent instructions which don't require flakes.
These solutions let Nix perform builds as if you're on the native platform, so
executing the build is as simple as
```console
$ nix develop
$ nix build .#packages.aarch64-linux.default
```
## Testing
for flake-enabled Nix, or
Nix comes with three different flavors of tests: unit, functional and integration.
```console
$ nix-build -A packages.aarch64-linux.default
```
for classic Nix.
You can use any of the other supported platforms in place of `aarch64-linux`.
Cross-compiled builds are available for ARMv6 and ARMv7, and Nix on unsupported platforms can be bootstrapped by adding more `crossSystems` in `flake.nix`.
## Compilation environments
Nix can be compiled using multiple environments:
-`stdenv`: default;
-`gccStdenv`: force the use of `gcc` compiler;
-`clangStdenv`: force the use of `clang` compiler;
-`ccacheStdenv`: enable [ccache], a compiler cache to speed up compilation.
To build with one of those environments, you can use
```console
$ nix build .#nix-ccacheStdenv
```
for flake-enabled Nix, or
```console
$ nix-build -A nix-ccacheStdenv
```
for classic Nix.
You can use any of the other supported environments in place of `nix-ccacheStdenv`.
## Editor integration
The `clangd` LSP server is installed by default on the `clang`-based `devShell`s.
See [supported compilation environments](#compilation-environments) and instructions how to set up a shell [with flakes](#nix-with-flakes) or in [classic Nix](#classic-nix).
To use the LSP with your editor, you first need to [set up `clangd`](https://clangd.llvm.org/installation#project-setup) by running:
```console
make clean && bear -- make -j$NIX_BUILD_CORES install
```
Configure your editor to use the `clangd` from the shell, either by running it inside the development shell, or by using [nix-direnv](https://github.com/nix-community/nix-direnv) and [the appropriate editor plugin](https://github.com/direnv/direnv/wiki#editor-integration).
> **Note**
>
> For some editors (e.g. Visual Studio Code), you may need to install a [special extension](https://open-vsx.org/extension/llvm-vs-code-extensions/vscode-clangd) for the editor to interact with `clangd`.
> Some other editors (e.g. Emacs, Vim) need a plugin to support LSP servers in general (e.g. [lsp-mode](https://github.com/emacs-lsp/lsp-mode) for Emacs and [vim-lsp](https://github.com/prabirshrestha/vim-lsp) for vim).
> Editor-specific setup is typically opinionated, so we will not cover it here in more detail.
## Running tests
### Unit-tests
The unit-tests for each Nix library (`libexpr`, `libstore`, etc..) are defined
You can run the whole testsuite with `make check`, or the tests for a specific component with `make libfoo-tests_RUN`. Finer-grained filtering is also possible using the [--gtest_filter](https://google.github.io/googletest/advanced.html#running-a-subset-of-the-tests) command-line option.
### Functional tests
The functional tests reside under the `tests` directory and are listed in `tests/local.mk`.
The whole testsuite can be run with `make install && make installcheck`.
Individual tests can be run with `make tests/{testName}.sh.test`.
Each test is a bash script.
The whole test suite can be run with:
```shell-session
$ make install && make installcheck
ran test tests/foo.sh... [PASS]
ran test tests/bar.sh... [PASS]
...
```
Individual tests can be run with `make`:
```shell-session
$ make tests/${testName}.sh.test
ran test tests/${testName}.sh... [PASS]
```
or without `make`:
```shell-session
$ ./mk/run-test.sh tests/${testName}.sh
ran test tests/${testName}.sh... [PASS]
```
To see the complete output, one can also run:
```shell-session
$ ./mk/debug-test.sh tests/${testName}.sh
+ foo
output from foo
+ bar
output from bar
...
```
The test script will then be traced with `set -x` and the output displayed as it happens, regardless of whether the test succeeds or fails.
#### Debugging failing functional tests
When a functional test fails, it usually does so somewhere in the middle of the script.
To figure out what's wrong, it is convenient to run the test regularly up to the failing `nix` command, and then run that command with a debugger like GDB.
For example, if the script looks like:
```bash
foo
nix blah blub
bar
```
edit it like so:
```diff
foo
-nix blah blub
+gdb --args nix blah blub
bar
```
Then, running the test with `./mk/debug-test.sh` will drop you into GDB once the script reaches that point:
```shell-session
$ ./mk/debug-test.sh tests/${testName}.sh
...
+ gdb blash blub
GNU gdb (GDB) 12.1
...
(gdb)
```
One can debug the Nix invocation in all the usual ways.
For example, enter `run` to start the Nix invocation.
### Integration tests
@@ -108,3 +287,105 @@ These tests include everything that needs to interact with external services or
Because these tests are expensive and require more than what the standard github-actions setup provides, they only run on the master branch (on <https://hydra.nixos.org/jobset/nix/master>).
You can run them manually with `nix build .#hydraJobs.tests.{testName}` or `nix-build -A hydraJobs.tests.{testName}`
### Installer tests
After a one-time setup, the Nix repository's GitHub Actions continuous integration (CI) workflow can test the installer each time you push to a branch.
Creating a Cachix cache for your installer tests and adding its authorization token to GitHub enables [two installer-specific jobs in the CI workflow](https://github.com/NixOS/nix/blob/88a45d6149c0e304f6eb2efcc2d7a4d0d569f8af/.github/workflows/ci.yml#L50-L91):
- The `installer` job generates installers for the platforms below and uploads them to your Cachix cache:
- `x86_64-linux`
- `armv6l-linux`
- `armv7l-linux`
- `x86_64-darwin`
- The `installer_test` job (which runs on `ubuntu-latest` and `macos-latest`) will try to install Nix with the cached installer and run a trivial Nix command.
#### One-time setup
1. Have a GitHub account with a fork of the [Nix repository](https://github.com/NixOS/nix).
2. At cachix.org:
- Create or log in to an account.
- Create a Cachix cache using the format `<github-username>-nix-install-tests`.
- Navigate to the new cache > Settings > Auth Tokens.
- Generate a new Cachix auth token and copy the generated value.
3. At github.com:
- Navigate to your Nix fork > Settings > Secrets > Actions > New repository secret.
- Name the secret `CACHIX_AUTH_TOKEN`.
- Paste the copied value of the Cachix cache auth token.
#### Using the CI-generated installer for manual testing
After the CI run completes, you can check the output to extract the installer URL:
1. Click into the detailed view of the CI run.
2. Click into any `installer_test` run (the URL you're here to extract will be the same in all of them).
3. Click into the `Run cachix/install-nix-action@v...` step and click the detail triangle next to the first log line (it will also be `Run cachix/install-nix-action@v...`)
4. Copy the value of `install_url`
5. To generate an install command, plug this `install_url` and your GitHub username into this template:
```console
curl -L <install_url> | sh -s -- --tarball-url-prefix https://<github-username>-nix-install-tests.cachix.org/serve
```
<!-- #### Manually generating test installers
There's obviously a manual way to do this, and it's still the only way for
platforms that lack GA runners.
I did do this back in Fall 2020 (before the GA approach encouraged here). I'll
sketch what I recall in case it encourages someone to fill in detail, but: I
didn't know what I was doing at the time and had to fumble/ask around a lot--
so I don't want to uphold any of it as "right". It may have been dumb or
the _hard_ way from the getgo. Fundamentals may have changed since.
Here's the build command I used to do this on and for x86_64-darwin:
`@docroot@` provides a base path for links that occur in reusable snippets or other documentation that doesn't have a base path of its own.
If a broken link occurs in a snippet that was inserted into multiple generated files in different directories, use `@docroot@` to reference the `doc/manual/src` directory.
If the `@docroot@` literal appears in an error message from the `mdbook-linkcheck` tool, the `@docroot@` replacement needs to be applied to the generated source file that mentions it.
See existing `@docroot@` logic in the [Makefile].
Regular markdown files used for the manual have a base path of their own and they can use relative paths instead of `@docroot@`.
| Select | *e*`.`*attrpath* \[ `or`*def* \] | none | Select attribute denoted by the attribute path *attrpath* from set *e*. (An attribute path is a dot-separated list of attribute names.) If the attribute doesn’t exist, return *def* if provided, otherwise abort evaluation. | 1 |
| Application | *e1**e2* | left | Call function *e1* with argument *e2*. | 2 |
| Update | *e1*`//`*e2* | right | Return a set consisting of the attributes in *e1* and *e2* (with the latter taking precedence over the former in case of equally named attributes). | 9 |
| Less Than | *e1*`<`*e2*, | none | Arithmetic/lexicographic comparison. | 10 |
| Less Than or Equal To | *e1*`<=`*e2* | none | Arithmetic/lexicographic comparison. | 10 |
Translate a [derivation] into a [store derivation].
See [`nix-instantiate`](./command-ref/nix-instantiate.md).
[instantiate]: #gloss-instantiate
- [realise]{#gloss-realise}, realisation\
Ensure a [store path] is [valid][validity].
This means either running the `builder` executable as specified in the corresponding [derivation] or fetching a pre-built [store object] from a [substituter].
See [`nix-build`](./command-ref/nix-build.md) and [`nix-store --realise`](./command-ref/nix-store.md#operation---realise).
See [`nix build`](./command-ref/new-cli/nix3-build.md) (experimental).
Select the attribute denoted by attribute path *attrpath* from [attribute set] *attrset*.
If the attribute doesn’t exist, return *value* if provided, otherwise abort evaluation.
<!-- FIXME: the following should to into its own language syntax section, but that needs more work to fit in well -->
An attribute path is a dot-separated list of attribute names.
An attribute name can be an identifier or a string.
> *attrpath* = *name* [ `.` *name* ]...
> *name* = *identifier* | *string*
> *identifier* ~ `[a-zA-Z_][a-zA-Z0-9_'-]*`
[Attribute selection]: #attribute-selection
## Has attribute
> *attrset* `?` *attrpath*
Test whether [attribute set] *attrset* contains the attribute denoted by *attrpath*.
The result is a [Boolean] value.
[Boolean]: ./values.md#type-boolean
[Has attribute]: #has-attribute
## Arithmetic
Numbers are type-compatible:
Pure integer operations will always return integers, whereas any operation involving at least one floating point number return a floating point number.
See also [Comparison] and [Equality].
The `+` operator is overloaded to also work on strings and paths.
[arithmetic]: #arithmetic
## String concatenation
> *string* `+` *string*
Concatenate two [string]s and merge their string contexts.
[String concatenation]: #string-concatenation
## Path concatenation
> *path* `+` *path*
Concatenate two [path]s.
The result is a path.
[Path concatenation]: #path-concatenation
## Path and string concatenation
> *path* + *string*
Concatenate *[path]* with *[string]*.
The result is a path.
> **Note**
>
> The string must not have a string context that refers to a [store path].
[Path and string concatenation]: #path-and-string-concatenation
## String and path concatenation
> *string* + *path*
Concatenate *[string]* with *[path]*.
The result is a string.
> **Important**
>
> The file or directory at *path* must exist and is copied to the [store].
> The path appears in the result as the corresponding [store path].
[store path]: ../glossary.md#gloss-store-path
[store]: ../glossary.md#gloss-store
[String and path concatenation]: #string-and-path-concatenation
## Update
> *attrset1* // *attrset2*
Update [attribute set] *attrset1* with names and values from *attrset2*.
The returned attribute set will have of all the attributes in *attrset1* and *attrset2*.
If an attribute name is present in both, the attribute value from the latter is taken.
[Update]: #update
## Comparison
Comparison is
- [arithmetic] for [number]s
- lexicographic for [string]s and [path]s
- item-wise lexicographic for [list]s:
elements at the same index in both lists are compared according to their type and skipped if they are equal.
All comparison operators are implemented in terms of `<`, and the following equivalencies hold:
| comparison | implementation |
|--------------|-----------------------|
| *a*`<=`*b* | `! (`*b*`<`*a*`)` |
| *a*`>`*b* | *b*`<`*a* |
| *a*`>=`*b* | `! (`*a*`<`*b*`)` |
[Comparison]: #comparison-operators
## Equality
- [Attribute sets][attribute set] and [list]s are compared recursively, and therefore are fully evaluated.
- Comparison of [function]s always returns `false`.
- Numbers are type-compatible, see [arithmetic] operators.
- Floating point numbers only differ up to a limited precision.
String interpolation is a language feature where a [string], [path], or [attribute name] can contain expressions enclosed in `${ }` (dollar-sign with curly brackets).
Such a string is an *interpolated string*, and an expression inside is an *interpolated expression*.
Interpolated expressions must evaluate to one of the following:
- a [string]
- a [path]
- a [derivation]
[string]: ./values.md#type-string
[path]: ./values.md#type-path
[attribute name]: ./values.md#attribute-set
[derivation]: ../glossary.md#gloss-derivation
## Examples
### String
Rather than writing
```nix
"--with-freetype2-library="+freetype+"/lib"
```
(where `freetype` is a [derivation]), you can instead write
```nix
"--with-freetype2-library=${freetype}/lib"
```
The latter is automatically translated to the former.
A more complicated example (from the Nix expression for [Qt](http://www.trolltech.com/products/qt)):
```nix
configureFlags="
-system-zlib-system-libpng-system-libjpeg
${ifopenglSupportthen"-dlopen-opengl
-L${mesa}/lib-I${mesa}/include
-L${libXmu}/lib-I${libXmu}/include"else""}
${ifthreadSupportthen"-thread"else"-no-thread"}
";
```
Note that Nix expressions and strings can be arbitrarily nested;
in this case the outer string contains various interpolated expressions that themselves contain strings (e.g., `"-thread"`), some of which in turn contain interpolated expressions (e.g., `${mesa}`).
### Path
Rather than writing
```nix
./.+"/"+foo+"-"+bar+".nix"
```
or
```nix
./.+"/${foo}-${bar}.nix"
```
you can instead write
```nix
./${foo}-${bar}.nix
```
### Attribute name
Attribute names can be created dynamically with string interpolation:
Numbers, which can be *integers* (like `123`) or *floating point*
(like `123.43` or `.27e13`).
See [arithmetic] and [comparison] operators for semantics.
[arithmetic]: ./operators.md#arithmetic
[comparison]: ./operators.md#comparison
- <a id="type-path" href="#type-path">Path</a>
*Paths*, e.g., `/bin/sh` or `./builder.sh`. A path must contain at
least one slash to be recognised as such. For instance, `builder.sh`
is not a path: it's parsed as an expression that selects the
attribute `sh` from the variable `builder`. If the file name is
relative, i.e., if it does not begin with a slash, it is made
absolute at parse time relative to the directory of the Nix
expression that contained it. For instance, if a Nix expression in
`/foo/bar/bla.nix` refers to `../xyzzy/fnord.nix`, the absolute path
is `/foo/xyzzy/fnord.nix`.
If the first component of a path is a `~`, it is interpreted as if
the rest of the path were relative to the user's home directory.
e.g. `~/foo` would be equivalent to `/home/edolstra/foo` for a user
whose home directory is `/home/edolstra`.
Paths can also be specified between angle brackets, e.g.
`<nixpkgs>`. This means that the directories listed in the
environment variable `NIX_PATH` will be searched for the given file
or directory name.
When an [interpolated string][string interpolation] evaluates to a path, the path is first copied into the Nix store and the resulting string is the [store path] of the newly created [store object].
[store path]: ../glossary.md#gloss-store-path
[store object]: ../glossary.md#gloss-store-object
For instance, evaluating `"${./foo.txt}"` will cause `foo.txt` in the current directory to be copied into the Nix store and result in the string `"/nix/store/<hash>-foo.txt"`.
Note that the Nix language assumes that all input files will remain _unchanged_ while evaluating a Nix expression.
For example, assume you used a file path in an interpolated string during a `nix repl` session.
Later in the same session, after having changed the file contents, evaluating the interpolated string with the file path again might not return a new store path, since Nix might not re-read the file contents.
Paths themselves, except those in angle brackets (`< >`), support [string interpolation].
At least one slash (`/`) must appear *before* any interpolated expression for the result to be recognized as a path.
`a.${foo}/b.${bar}` is a syntactically valid division operation.
1. Triage issues and pull requests from the _No Status_ column (30 min)
2. Discuss issues and pull requests from the _To discuss_ column (30 min)
- Work meeting: [Mondays 13:00-15:00 CET](https://calendar.google.com/calendar/event?eid=NTM1MG1wNGJnOGpmOTZhYms3bTB1bnY5cWxfMjAyMjExMjFUMTIwMDAwWiBiOW81MmZvYnFqYWs4b3E4bGZraGczdDBxZ0Bn)
1. Code review on pull requests from _In review_.
2. Other chores and tasks.
Meeting notes are collected on a [collaborative scratchpad](https://pad.lassul.us/Cv7FpYx-Ri-4VjUykQOLAw), and published on Discourse under the [Nix category](https://discourse.nixos.org/c/dev/nix/50).
## Project board protocol
The team uses a [GitHub project board](https://github.com/orgs/NixOS/projects/19/views/1) for tracking its work.
Issues on the board progress through the following states:
- No Status
During the discussion meeting, the team triages new items.
To be considered, issues and pull requests must have a high-level description to provide the whole team with the necessary context at a glance.
On every meeting, at least one item from each of the following categories is inspected:
To [automatically backport a pull request](https://github.com/NixOS/nix/blob/master/.github/workflows/backport.yml) to a release branch once it's merged, assign it a label of the form [`backport <branch>`](https://github.com/NixOS/nix/labels?q=backport).
Since [GitHub Actions workflows will not trigger other workflows](https://docs.github.com/en/actions/using-workflows/triggering-a-workflow#triggering-a-workflow-from-a-workflow), checks on the automatic backport need to be triggered by another actor.
This is achieved by closing and reopening the backport pull request.
This specifically affects the [`installer_test`] check.
Note that it only runs after the other tests, so it may take a while to appear.
Omit `IS_LATEST=1` when creating a point release that is not on the
most recent stable branch. This prevents `nixos.org` to going back
to an older release.
* Bump the version number of the release branch as above (e.g. to
`2.12.2`).
## Recovering from mistakes
`upload-release.pl` should be idempotent. For instance a wrong `IS_LATEST` value can be fixed that way, by running the script on the actual latest release.
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
