This seems to be slower as of yet.
TODO
- [ ] remove merge pass 2 into 1:
- pass 1: fill UpdateQueue
- pass 2: stats (like max output size)
- pass 3: k-way merge
- [ ] maybe tracking sortedness is useful and cheap? in pass 1, building the UpdateQueue
- [ ] make it actually fast; different algorithm depending on size distribution and number of update ops?
- currently solution is the min-heap or priority queue approach
- divide and conquer (see also Nixpkgs which manually implements this for pkgs/by-name)
binary tree-shaped sorted merge operations
- can we take the size of attrsets into account to balance the work?
1000 + (1 + (1 + 1)) is better than ((1000+1)+1)+1 (where number refers to attrset size)
- iterative pairwise merging is more or less what we had, the tree of updates is flattened. Probably worse.
- perhaps some combination of solutions, depending on a heuristic
- [ ] split callFunction into two parts, effectively
- call2Thunk (create the new scope's Env etc)
- eval that thunk (evaluate the body)
- [ ] implement ExprApply::evalForUpdate - easy after the split
- [ ] make foldl' (//) work?
- [ ] optimize listToAttrs and other primops?
This is because with the split packages of the Meson build, we simply
have no idea what directory the binaries will be installed in when we
build the library.
In the process of doing so, consolidate and make more sophisticated the
logic to cope with a few corner cases (e.g. `NIX_BIN_DIR` exists, but no
binaries are inside it).
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
This ended up motivating a good deal of other infra improvements in
order to get Windows right:
- `OsString` to complement `std::filesystem::path`
- env var code for working with the underlying `OsString`s
- Rename `PATHNG_LITERAL` to `OS_STR`
- `NativePathTrait` renamed to `OsPathTrait`, given a character template
parameter until #9205 is complete.
Split `tests.cc` matching split of `util.{cc,hh}` last year.
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
The test split matches PR #8920, so the utility files and tests files
are once again to 1-1. The string changes continues what was started in
PR #11093.
In the FFI world we have many tools that are not gcc/clang and therefore
not always support the latest C standard. This fixes support with cffi
i.e. used in https://github.com/tweag/python-nix
It was failing with:
error: AWS error fetching 'nix-cache-info': The specified bucket does not exist
because `S3BinaryCacheStoreImpl` had a `bucketName` field that
shadowed the inherited `bucketName from `S3BinaryCacheStoreConfig`.
We didn't even realize you *could* use this syntax with -E and -f, much
less that the attribute path could be *empty*.
Change-Id: Id1a6715609f3a76a5ce477bd43a7832effbbe07b
* docs: unify documentation on search paths
- put all the information on search path semantics into `builtins.findFile`
- put all the information on determining the value of `builtins.nixPath` into the
`nix-path` setting
maybe `builtins.nixPath` is a better place for this, but those bits
can still be moved around now that it's all next to each other.
- link to the syntax page for lookup paths from all places that are
concerned with it
- add or clarify examples
- add a test verifying a claim from documentation
This also bans various sneaking of negative numbers from the language
into unsuspecting builtins as was exposed while auditing the
consequences of changing the Nix language integer type to a newtype.
It's unlikely that this change comprehensively ensures correctness when
passing integers out of the Nix language and we should probably add a
checked-narrowing function or something similar, but that's out of scope
for the immediate change.
During the development of this I found a few fun facts about the
language:
- You could overflow integers by converting from unsigned JSON values.
- You could overflow unsigned integers by converting negative numbers
into them when going into Nix config, into fetchTree, and into flake
inputs.
The flake inputs and Nix config cannot actually be tested properly
since they both ban thunks, however, we put in checks anyway because
it's possible these could somehow be used to do such shenanigans some
other way.
Note that Lix has banned Nix language integer overflows since the very
first public beta, but threw a SIGILL about them because we run with
-fsanitize=signed-overflow -fsanitize-undefined-trap-on-error in
production builds. Since the Nix language uses signed integers, overflow
was simply undefined behaviour, and since we defined that to trap, it
did.
Trapping on it was a bad UX, but we didn't even entirely notice
that we had done this at all until it was reported as a bug a couple of
months later (which is, to be fair, that flag working as intended), and
it's got enough production time that, aside from code that is IMHO buggy
(and which is, in any case, not in nixpkgs) such as
https://git.lix.systems/lix-project/lix/issues/445, we don't think
anyone doing anything reasonable actually depends on wrapping overflow.
Even for weird use cases such as doing funny bit crimes, it doesn't make
sense IMO to have wrapping behaviour, since two's complement arithmetic
overflow behaviour is so *aggressively* not what you want for *any* kind
of mathematics/algorithms. The Nix language exists for package
management, a domain where bit crimes are already only dubiously in
scope to begin with, and it makes a lot more sense for that domain for
the integers to never lose precision, either by throwing errors if they
would, or by being arbitrary-precision.
Fixes: https://github.com/NixOS/nix/issues/10968
Original-CL: https://gerrit.lix.systems/c/lix/+/1596
Change-Id: I51f253840c4af2ea5422b8a420aa5fafbf8fae75
The actual motive here is the avoidance of integer overflow if we were
to make these use checked NixInts and retain the subtraction.
However, the actual *intent* of this code is a three-way comparison,
which can be done with operator<=>, so we should just do *that* instead.
Change-Id: I7f9a7da1f3176424b528af6d1b4f1591e4ab26bf
Few filesystem-related tests rely on PATH_MAX for buffers, and PATH_MAX
is optional in POSIX (and not available on the Hurd). To make them build
and pass, provide a fallback definition of PATH_MAX in case not
available.
Ideally speaking, the tests ought to not unconditionally rely on
PATH_MAX, do alternative strategies (e.g. dynamically allocate buffers,
expand them as needed, etc); OTOH this is test code, so it would be more
work that what it would be worth, so IMHO the define fallback is good
enough.
Set HOST_HURD & HOST_UNIX for GNU/Hurd in the makefile-based build
system; the latter variable is important as it will include all the
commit Unix bits.
- move <sys/resource.h> from a __linux__ block to a !_WIN32 block: this
matches what the actual code does, using getrlimit() & setrlimit() in
!_WIN32 blocks
- drop <sys/mount.h>, which is not portable, and it is not used
This is not allowed in C++20, and GCC 14 warns about it:
../src/libutil/ref.hh:26:20: warning: template-id not allowed for constructor in C++20 [-Wtemplate-id-cdtor]
26 | explicit ref<T>(const std::shared_ptr<T> & p)
| ^
../src/libutil/ref.hh:26:20: note: remove the '< >'
../src/libutil/ref.hh:33:21: warning: template-id not allowed for constructor in C++20 [-Wtemplate-id-cdtor]
33 | explicit ref<T>(T * p)
| ^
../src/libutil/ref.hh:33:21: note: remove the '< >'
This change updates the seccomp profile to return ENOTSUP for getxattr
functions family. This reflects the behavior of filesystems that don’t
support extended attributes (or have an option to disable them), e.g.
ext2.
The current behavior is confusing for some programs because we can read
extended attributes, but only get to know that they are not supported
when setting them. In addition to that, ACLs on Linux are implemented
via extended attributes internally and if we don’t return ENOTSUP, acl
library converts file mode to ACL.
https://git.savannah.nongnu.org/cgit/acl.git/tree/libacl/acl_get_file.c?id=d9bb1759d4dad2f28a6dcc8c1742ff75d16dd10d#n69
The internal "completionCallback" and "listPossibleCallback" helpers
are used only when building with editline; hence, do not build then
when using readline, matching their usage in
"ReadlineLikeInteracter::init()".
This seems to have been the intent all along.
The odd combination of unit tests, but no functional tests caused a
build error where some data for the unit test was source-filtered out.
Apparently. It's unclear to me why that happened, so I'm proposing this
alternate "fix" to get the buildNoTests to pass.
It would be nice to test more configurations, but this mode of building
is on the way out anyway, so let's just make it pass and see what
configurations make sense to test as part of the meson migration.
This should make the test more robust, considering the strange hang
in https://hydra.nixos.org/build/267517233/nixlog/8
`builder` seems to have reached `multi-user.target` before the
SSH connection was established, but this seems to be coincidental.
This does tell us that enforcing this has a minimal cost in terms
of runtime.
Waiting for `multi-user.target` on the client is honestly paranoid,
but flaky tests are very bad for productivity.
Trying to learn more about enigmatic spurious hang at
https://hydra.nixos.org/build/267517233/nixlog/8
- builder1 seems to have started properly
- ssh connection and session are established
- ssh client doesn't exit or client.succeed does not return
for some reason.
Seeing the stdout on the console might give a tiny bit more info.
By syncing with Nixpkgs, we reuse the same derivation, which is
generally a good idea, and has the benefit that it is transitively
a channel blocker.
Changes:
- https://github.com/NixOS/nixpkgs/pull/163313 (SuperSandro2000)
> nix: disable big-parallel for aws-sdk-cpp
> aws-sdk-cpp only takes ~1m52s on a 4 core machine under 50% load
> which does not justify the requirement on big parallel.
> Tested with `nix-build -A nixVersions.nix_2_6.aws-sdk-cpp`.
> I can finally build nix without requiring a big-parallel machine.
- https://github.com/NixOS/nixpkgs/pull/227506 (Artturin)
> nix: use [ ] instead null to empty requiredSystemFeatures
> fixes 'error: value is null while a list was expected' with 'nixpkgs.hostPlatform.gcc.arch = "x86_64";'
Found by looking for interesting asan reports from the test suite.
What happened here is that name got overwritten, but it was what
actually held the backing memory for the thing it got overwritten by,
which was a by-reference value coming out of std::regex.
Due to absurd reasons I cannot seem to use a string_view iterator here,
so I just copy the string with a longer lifetime instead. idk lol
==3796364==ERROR: AddressSanitizer: heap-use-after-free on address 0x503000014c61 at pc 0x74843523bf1d bp 0x7ffc68351330 sp 0x7ffc68350af0
READ of size 3 at 0x503000014c61 thread T0
0 0x74843523bf1c in __asan_memcpy (/nix/store/mzhqknx2mc94jdz4n320hn1lml86398y-clang-wrapper-17.0.6/resource-root/lib/linux/libclang_rt.asan-x86_64.so+0x159f1c)
1 0x6403cf6cbff4 in std::char_traits<char>::copy(char*, char const*, unsigned long) /nix/store/14c6s4xzhy14i2b05s00rjns2j93gzz4-gcc-13.2.0/include/c++/13.2.0/bits/char_traits.h:445:33
<...>
7 0x6403cf6cbff4 in std::__cxx11::sub_match<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>>::str() const /nix/store/14c6s4xzhy14i2b05s00rjns2j93gzz4-gcc-13.2.0/include/c++/13.2.0/bits/regex.h:966:6
8 0x6403cf6cbff4 in std::__cxx11::sub_match<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>>::operator std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>() const /nix/store/14c6s4xzhy14i2b05s00rjns2j93gzz4-gcc-13.2.0/include/c++/13.2.0/bits/regex.h:955:16
9 0x6403cf6cbff4 in nix::getClosureInfo[abi:cxx11](nix::ref<nix::Store>, nix::StorePath const&) /home/jade/lix/lix2/build/src/nix/diff-closures.cc:37:26
10 0x6403cf6cd70c in nix::printClosureDiff(nix::ref<nix::Store>, nix::StorePath const&, nix::StorePath const&, std::basic_string_view<char, std::char_traits<char>>) /home/jade/lix/lix2/build/src/nix/diff-closures.cc:54:25
11 0x6403cf873331 in CmdProfileDiffClosures::run(nix::ref<nix::Store>) /home/jade/lix/lix2/build/src/nix/profile.cc:479:17
<...>
0x503000014c61 is located 17 bytes inside of 21-byte region [0x503000014c50,0x503000014c65)
freed by thread T0 here:
0 0x748435250470 in operator delete(void*) (/nix/store/mzhqknx2mc94jdz4n320hn1lml86398y-clang-wrapper-17.0.6/resource-root/lib/linux/libclang_rt.asan-x86_64.so+0x16e470)
<...>
6 0x6403cf6cbda2 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::~basic_string() /nix/store/14c6s4xzhy14i2b05s00rjns2j93gzz4-gcc-13.2.0/include/c++/13.2.0/bits/basic_string.h:792:9
7 0x6403cf6cbda2 in nix::getClosureInfo[abi:cxx11](nix::ref<nix::Store>, nix::StorePath const&) /home/jade/lix/lix2/build/src/nix/diff-closures.cc:36:13
8 0x6403cf6cd70c in nix::printClosureDiff(nix::ref<nix::Store>, nix::StorePath const&, nix::StorePath const&, std::basic_string_view<char, std::char_traits<char>>) /home/jade/lix/lix2/build/src/nix/diff-closures.cc:54:25
<...>
previously allocated by thread T0 here:
0 0x74843524fa38 in operator new(unsigned long) (/nix/store/mzhqknx2mc94jdz4n320hn1lml86398y-clang-wrapper-17.0.6/resource-root/lib/linux/libclang_rt.asan-x86_64.so+0x16da38)
<...>
9 0x6403cf6cb68c in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::basic_string<std::basic_string_view<char, std::char_traits<char>>, void>(std::basic_string_view<char, std::char_traits<char>> const&, std::allocator<char> const&) /nix/store/14c6s4xzhy14i2b05s00rjns2j93gzz4-gcc-13.2.0/include/c++/13.2.0/bits/basic_string.h:784:4
10 0x6403cf6cb68c in nix::getClosureInfo[abi:cxx11](nix::ref<nix::Store>, nix::StorePath const&) /home/jade/lix/lix2/build/src/nix/diff-closures.cc:33:21
11 0x6403cf6cd70c in nix::printClosureDiff(nix::ref<nix::Store>, nix::StorePath const&, nix::StorePath const&, std::basic_string_view<char, std::char_traits<char>>) /home/jade/lix/lix2/build/src/nix/diff-closures.cc:54:25
12 0x6403cf873331 in CmdProfileDiffClosures::run(nix::ref<nix::Store>) /home/jade/lix/lix2/build/src/nix/profile.cc:479:17
<...>
(cherry-picked from https://git.lix.systems/lix-project/lix/commit/b9b1bbd22fc3b34a4d8e2090f0d033f742c32ee0)
We should use a metric that weighs the related issues.
Counterbalancing time doesn't make much sense to me.
If it's around for longer, the fix will be relevant to more people.
* manual: Contributing -> Development, Hacking -> Building
what's currently called "hacking" are really instructions for setting up
a development environment and compiling from source. we have
a contribution guide in the repo (which rightly focuses on GitHub
workflows), and the material in the manual is more about working
on the code itself.
since we'd otherwise have three headings that amount to "Building Nix",
this change also moves the "classic Nix" instructions to the top.
we may want to reorganise this in the future, and bring
contributor-oriented information closer to the code, but for now let's
stick to more accurate names to ease navigation.
Meson uses a venerable GNU convention described in
https://www.gnu.org/software/automake/manual/html_node/Scripts_002dbased-Testsuites.html
in which:
> When no test protocol is in use, an exit status of 0 from a test
> script will denote a success, an exit status of 77 a skipped test, an
> exit status of 99 a hard error, and any other exit status will denote
> a failure.
77 is thus what we want, not 99.
* fix NIX_PATH overriding
- test restricted evaluation
- test precedence for setting the search path
Co-authored-by: Robert Hensing <robert@roberthensing.nl>
Co-authored-by: John Ericson <git@JohnEricson.me>
We are piping curl downloads into `unpackTarfileToSink()`, but the
latter is typically slower than the former if you're on a fast
connection. So the download could appear unnecessarily slow. (There is
even a risk that if the Git import is *really* slow for whatever
reason, the TCP connection could time out.)
So let's make the download buffer bigger by default - 64 MiB is big
enough for the Nixpkgs tarball. Perhaps in the future, we could have
an unlimited buffer that spills data to disk beyond a certain
threshold, but that's probably overkill.
Currently, the worker protocol has a version number that we increment
whenever we change something in the protocol. However, this can cause
a collision between Nix PRs / forks that make protocol changes
(e.g. PR #9857 increments the version, which could collide with
another PR). So instead, the client and daemon now exchange a set of
protocol features (such as `auth-forwarding`). They will use the
intersection of the sets of features, i.e. the features they both
support.
Note that protocol features are completely distinct from
`ExperimentalFeature`s.
This is in accordance with ARM's naming convention.
"Low" is confusing, because it could refer to either the cold end
of the stack as an abstract data type, or a low address.
These are different places, because the stack grows down through
the address space.
... as well as match buildReadlineNoMarkdown.
Unfortunately it doesn't support long inputs or multiline inputs
for now.
This needs to make better use of the interacter interface.
In addition to adding the missing thread deps in the last commit, we
also appear to need to skip `-Wl,--as-needed` flags that Meson wants to
use, but doesn't work with our *BSD toolchains.
See https://github.com/mesonbuild/meson/issues/3593
This avoids the double warning
warning: 'ping-store' is a deprecated alias for 'store ping'
warning: 'nix store ping' is a deprecated alias for 'nix store info'
* Only build perl subproject on Linux
* Fix various Windows regressions
* Don't put the emulator hook in test builds
We run the tests in a separate derivation. Only need it for the dev shell.
* Fix native dev shells
* Fix cross dev shells we don't know how to emulate
Co-authored-by: PoweredByPie <poweredbypie@users.noreply.github.com>
Co-authored-by: Joachim Schiele <js@lastlog.de>
Co-authored-by: John Ericson <John.Ericson@Obsidian.Systems>
In _very_ rare cases (I had about 7 cases out of 32200 files!),
the order of how inherit-from bindings are printed when using
`nix-instantiate --parse` gets messed up.
The cause of this seems to be because the std::map the bindings are
placed in is keyed on a _pointer_, which then uses an
[implementation-defined strict total order](https://en.cppreference.com/w/cpp/language/operator_comparison#Pointer_total_order).
The fix here is to key the bindings on their displacement instead,
which maintains the same order as they appear in the file.
Unfortunately I wasn't able to make a reproducible test for this in the
source, there's something about the local environment that makes it
unreproducible for me.
However I was able to make a reproducible test in a Nix build on a Nix
version from a very recent master:
nix build github:infinisil/non-det-nix-parsing-repro
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Following what is outlined in #10766 refactor the uds-remote-store such
that the member variables (state) don't live in the store itself but in
the config object.
Additionally, the config object includes a new necessary constructor
that takes a scheme & authority.
Tests are commented out because of linking errors with the current config system.
When there is a new config system we can reenable them.
Co-authored-by: John Ericson <John.Ericson@Obsidian.Systems>
This generally gives a better experience with bindings generators,
possibly other tooling.
A possible risk is that some generators may not represent unknown
codes correctly.
Rust bindgen by default generates suitable code:
* a type alias nix_err = c_int
* individual constants for the known enum values
It does _not_ generate a closed type that can only hold the values
that were known at code generation time.
If this proves to be a problem, we could instead split the type:
`typedef int nix_err;` for return values
`enum nix_known_err` for code generation.
This would complicate the interface, so let's not do it unless it
is shown to be needed.
This was accidentally introduced
in f71b4da0b3. We didn't notice this
because the version got interpreted by the daemon as the obsolete "CPU
affinity will follow" field, and being non-zero, it would then read
another integer for the ignored CPU affinity.
The default value for the setting was evaluated by
calling a method on the object _being currently constructed_,
so we were using it before all fields were initialized.
This has been fixed by making the called method static,
and not using the previously used fields at all.
But functionality hasn't changed!
The fields were usually always zero (by chance?) anyway,
meaning the conditional path was always taken.
Thus the current logic has been kept, the code simplified,
and UB removed.
This was found with the helper of UBSan.
Splitting it out immediately answers questions like [this],
without increasing the number of compilation units.
I did consider using boost::hash_combine instead, but it doesn't seem
to be quite as capable, accepting only two arguments.
[this]: https://github.com/NixOS/nix/pull/11113#discussion_r1679991573
Progress towards #10766
I thought that #10768 achieved, but when I went to use this stuff (in
Hydra), turns out it did not. (Those `using FooConfig;` lines were not
working --- they are so finicky!) This PR gets the job done, and adds
some trivial unit tests to make sure I did what I intended.
I had to add add a header to expose `SSHStoreConfig`, after which the
preexisting `ssh-store-config.*` were very confusingly named files, so I
renamed them to `common-ssh-store-config.hh` to match the type defined
therein.
They are not actually part of the store layer, but instead part of the
Nix executable infra (libraries don't need plugins, executables do).
This is part of a larger project of moving all of our legacy settings
infra to libmain, and having the underlying libraries just have plain
configuration structs detached from any settings infra / UI layer.
Progress on #5638
Previous test implementation assumed that grep supports newlines
in patterns. It doesn't, so tests spuriously passed, even though
some tests outputs were broken.
This patches output (and expected output) before grepping,
so there're no newlines in pattern.
This makes it possible to certain discern failures from empty
snippets, which I think is an ok review comment.
Maybe it should do so for swapped column indexes too, but I'm not
sure.
I don't think it matters in the grand scheme. We don't even have
a real use case for `nullopt` now anyway.
Since we don't have a use case, I'm not applying this logic to
higher level functions yet.
Unfortunately these don't render correctly, because they go into the
markdown renderer, instead of the terminal.
```
nix-repl> :doc lib.version
Attribute '[35;1mversion[0m'
… defined at [35;1m/home/user/h/nixpkgs/lib/default.nix:73:40[0m
```
We could switch that to go direct to the terminal, but then we should
do the same for the primops, to get a consistent look.
Reverting for now.
This reverts commit 3413e0338cbee1c7734d5cb614b5325e51815cde.
Got shellcheck passing for misc/systemv/nix-daemon
Not sure how to test this since it's not running on my NixOS machine and
I see no references to it in the directory otherwise.
See #10795
fetchurl can be given a name and url aside from just the url.
Giving a name can be useful if the url has invalid characters such as
tilde for the store.
... at call sites that are may be in the hot path.
I do not know how clever the compiler gets at these sites.
My primary concern is to not regress performance and I am confident
that this achieves it the easy way.
When the separator is empty, no difference is observable.
Note that concatStringsSep has centralized definitions. This adds the
required definitions. Alternatively, `strings-inline.hh` could be
included at call sites.
Considering that `value` was probably parsed with tokenizeString
prior, it's unlikely to contain empty strings, and we have no
reason to remove them either.
Empty attributes are probably not well supported, but the least we
could do is leave a hint.
Attribute path rendering and parsing should be done according to
Nix expression syntax in my opinion.
(System) features are unlikely to be empty strings, but when they
come in through structuredAttrs, they probably can.
I don't think this means we should drop them, but most likely they
will be dropped after this because next time they'll be parsed with
tokenizeString.
TODO: We should forbid empty features.
I don't think it's completely impossible, but I can't construct
one easily as derivationStrict seems to (re)tokenize the outputs
attribute, dropping the empty output.
It's not a scenario we have to account for here.
Bug not reported in 6 years, but here you go.
Also it is safe to switch to normal concatStringsSep behavior
because tokenizeString does not produce empty items.
The empty attribute name should not be dropped from attribute paths.
Rendering attribute paths with concatStringsSep is lossy and wrong,
but this is just a first improvement while dealing with the
dropEmptyInitThenConcatStringsSep problem.
Known behavior changes:
- `MemorySourceAccessor`'s comparison operators no longer forget to
compare the `SourceAccessor` base class.
Progress on #10832
What remains for that issue is hopefully much easier!
- Fix eval cache not being persisted in `nix develop` (since #10570)
- Don't attempt to commit cache transaction if there is no active transaction, which will spew errors in edge cases
- Drive-by: trivial typo fix
On some systems, previous usage of `match` may cause a stackoverflow
(presumably due to the large size of the match result). Avoid this by
(ab)using `replaceStrings` to test for containment without using
regexes, thereby avoiding the issue. The causal configuration seems to
be the stack size hard limit, which e.g. Amazon Linux sets, whereas most
Linux distros leave unlimited.
Match the fn name to similar fn in nixpkgs.lib, but different
implementation that does not use `match`. This impl gives perhaps
unexpected results when the needle is `""`, but the scope of this is
narrow and that case is a bit odd anyway.
This makes for some duplication-of-work as we do a different
`replaceStrings` if this one is true, but this only runs during doc
generation at build time so has no runtime impact.
See https://github.com/NixOS/nix/issues/11085 for details.
Progress on #5638
There are still a global fetcher and eval settings, but they are pushed
down into `libnixcmd`, which is a lot less bad a place for this sort of
thing.
Continuing process pioneered in
52bfccf8d8.
The move assignment was implicitly generated and used in
src/libstore/build/goal.cc:90:22:
90 | this->ex = std::move(*ex);
Clang warns about this generated method being deprecated, so making
them explicit fixes the warning.
It is unclear to me why this worked when not in a VM test, but the
explanation would be in the part of nix-shell we're getting rid of
with the devShell attribute.
When --unpack was used the nix would add the current directory to the
nix store instead of the content of unpacked.
The reason for this is that std::distance already consumes the iterator.
To fix this we re-instantiate the directory iterator in case the
directory only contains a single entry.
This improves the error message of nix-env -qa, among others, which
is crucial for understanding some ofborg eval error reports, such as
https://gist.github.com/GrahamcOfBorg/89101ca9c2c855d288178f1d3c78efef
After this change, it will report the same trace, but also start with
```
error:
… while evaluating the attribute 'devShellTools'
… while evaluating the attribute 'nixos'
… while evaluating the attribute 'docker-tools-nix-shell'
… while evaluating the attribute 'aarch64-darwin'
… from call site
at /home/user/h/nixpkgs/outpaths.nix:48:6:
47| tweak = lib.mapAttrs
48| (name: val:
| ^
49| if name == "recurseForDerivations" then true
<same>
```
The recent fix for CVE-2024-38531 broke the sandbox on macOS
completely. As it’s not practical to use `chroot(2)` on
macOS, the build takes place in the main filesystem tree, and the
world‐unreadable wrapper directory prevents the build from accessing
its `$TMPDIR` at all.
The macOS sandbox probably shouldn’t be treated as any kind of a
security boundary in its current state, but this specific vulnerability
wasn’t possible to exploit on macOS anyway, as creating `set{u,g}id`
binaries is blocked by sandbox policy.
Locking down the build sandbox further may be a good idea in future,
but it already has significant compatibility issues. For now, restore
the previous status quo on macOS.
Thanks to @alois31 for helping me come to a better understanding of
the vulnerability.
Fixes: 1d3696f0fbCloses: #11002
- use the iterator in `CanonPath` to count `level`
- use the `CanonPath::basename` method
- use `CanonPath::root` instead of `CanonPath{""}`
- remove `Path` and `PathView`, use `std::filesystem::path` directly
move together all syntactic and semantic information into one
page, and add a page on data types, which in turn links to the syntax and
semantics.
also split out the note on scoping rules into its own page.
Co-authored-by: Ryan Hendrickson <ryan.hendrickson@alum.mit.edu>
Inspired by
https://git.lix.systems/lix-project/lix/commit/010ff57ebb40f1a9aaff99867d2886f0e59f774a
From the original PR:
> We do not have any of these warnings appearing at the moment, but
> it seems like a good idea to enable [[nodiscard]] checking anyway.
> Once we start introducing more functions with must-use conditions we will
> need such checking, and the rust stdlib has proven them very useful.
* doc: fix `directory` definition in nix-archive.md
Before the change the document implied that directory of a single entry
contained entry:
"type" "directory" "type" directory" "entry" ...
After the change document should expand into:
"type" "directory" "entry" ...
Co-authored-by: John Ericson <git@JohnEricson.me>
The code that counts the number of elided attrs incorrectly used the
per-printer "global" attribute counter instead of a counter that
was relevant only to the current attribute set.
This bug flew under the radar because often the attribute sets aren't
nested, not big enough, or we wouldn't pay attention to the numbers.
I've noticed the issue because the difference underflowed.
Although this behavior is tested by the functional test
lang/eval-fail-bad-string-interpolation-4.nix, the underflow slipped
through review. A simpler reproducer would be as follows, but I
haven't added it to the test suite to keep it simple and marginally
faster.
```
$ nix run nix/2.23.1 -- eval --expr '"" + (let v = { a = { a = 1; b = 2; c = 1; d = 1; e = 1; f = 1; g = 1; h = 1; }; b = { a = 1; b = 1; c = 1; }; }; in builtins.deepSeq v v)'
error:
… while evaluating a path segment
at «string»:1:6:
1| "" + (let v = { a = { a = 1; b = 2; c = 1; d = 1; e = 1; f = 1; g = 1; h = 1; }; b = { a = 1; b = 1; c = 1; }; }; in builtins.deepSeq v v)
| ^
error: cannot coerce a set to a string: { a = { a = 1; b = 2; c = 1; d = 1; e = 1; f = 1; g = 1; h = 1; }; b = { a = 1; «4294967289 attributes elided» }; }
```
* string interpolation escape example
Make it easier to find the documentation, and the example might be enough for most cases.
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
We don't apply any patches to it, and vendoring it locks users into
bugs (it hasn't been updated since its introduction in late 2021).
Closes https://git.lix.systems/lix-project/lix/issues/164
Change-Id: Ied071c841fc30b0dfb575151afd1e7f66970fdb9
(cherry picked from commit 80405d06264f0de1c16ee2646388ab501df20628)
On one hand, new things should be formatted. On the other, we just
bacported this file to many prior branches, and if we need to make
changes to it and backport them also, formatting the file on master but
not the release branches would cause issues.
We're building a bit of Darwin meson indirectly through `checks`,
but it'd be annoying to encounter broken un-`check`-ed stuff
during the porting process, so let's just do the right thing now.
This avoids polluting nixComponents with things that aren't our
components.
Fixes the extraction of passthru tests, which failed for boehmgc
which had many irrelevant ones anyway.
flatMapAttrs is easier to read because it introduces the values
before using them, kind of like a `let` bindings with multiple
values.
The repeated comments remind the reader of the purpose of the
innermost attrsets, which is actually very simple.
Knowing that they go right into the result should help a lot
with building a mental model for this pattern.
The old `std::variant` is bad because we aren't adding a new case to
`FileIngestionMethod` so much as we are defining a separate concept ---
store object content addressing rather than file system object content
addressing. As such, it is more correct to just create a fresh
enumeration.
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
This tests the parser and JSON format using the DRV files from the tests
added in the previous commit.
Co-Authored-By: John Ericson <John.Ericson@Obsidian.Systems>
This tests the Nix language side of things.
We are purposely skipping most of `common.sh` because it is overkill for
this test: we don't want to have an "overfit" test environment.
Co-Authored-By: John Ericson <John.Ericson@Obsidian.Systems>
Previously, the .chroot directory had permission 750 or 755 (depending
on the uid-range system feature) and was owned by root/nixbld. This
makes it possible for any nixbld user (if uid-range is disabled) or
any user (if uid-range is enabled) to inspect the contents of the
chroot of an active build and maybe interfere with it (e.g. via /tmp
in the chroot, which has 1777 permission).
To prevent this, the root is now a subdirectory of .chroot, which has
permission 700 and is owned by root/root.
Instead of running the builds under
`$TMPDIR/{unique-build-directory-owned-by-the-build-user}`, run them
under `$TMPDIR/{unique-build-directory-owned-by-the-daemon}/{subdir-owned-by-the-build-user}`
where the build directory is only readable and traversable by the daemon user.
This achieves two things:
1. It prevents builders from making their build directory world-readable
(or even writeable), which would allow the outside world to interact
with them.
2. It prevents external processes running as the build user (either
because that somehow leaked, maybe as a consequence of 1., or because
`build-users` isn't in use) from gaining access to the build
directory.
the `std::filesystem::create_directories` can fail due to insufficient
permissions. We convert this error into a `SysError` and catch it
wherever required.
#2230 broadened the scope of macOS hardlink exclusion but did not change the comments. This was a little confusing for me, so I figured the comments should be updated.
This will avoid some out-of-memory issues in GitHub actions that result
from num jobs > 1 and num cores = 4. Once we only have the Meson build
system, this problem should go away, and we can reenable these jobs.
The documentation is clear about the supported formats (with at least
`builtins.fetchTarball`). The way the code was written previously it
supported all the formats that libarchive supported. That is a
surprisingly large amount of formats that are likely not on the radar
of the Nix developers and users. Before people end up relying on
this (or if they do) it is better to break it now before it becomes a
widespread "feature".
Zip file support has been retained as (at least to my knowledge)
historically that has been used to fetch nixpkgs in some shell
expressions *many* years back.
Fixes https://github.com/NixOS/nix/issues/10917
I hope this will make it easier to maintain, and also make it easier for
others to assist with porting the rest of the build system to Meson.
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Previously (in cfc18a7739), we forgot to
compare the algo at all. This means we keep the same ordering as before
by making the stuff we always have compared take priority.
The idea is two-fold:
- Replace autotools with Meson
- Build each library in its own derivation
The interaction of these two features is that Meson's "subprojects"
feature (https://mesonbuild.com/Subprojects) allows us to have single
dev shell for building all libraries still, while also building things
separately. This allows us to break up the build without a huge
productivity lost.
I tested the Linux native build, and NetBSD and Windows cross builds.
Also do some clean ups of the Flake in the process of supporting new
jobs.
Special thanks to everyone that has worked on a Meson port so far,
@p01arst0rm and @Qyriad in particular.
Co-Authored-By: p01arst0rm <polar@ever3st.com>
Co-Authored-By: Artemis Tosini <lix@artem.ist>
Co-Authored-By: Artemis Tosini <me@artem.ist>
Co-Authored-By: Felix Uhl <felix.uhl@outlook.com>
Co-Authored-By: Jade Lovelace <lix@jade.fyi>
Co-Authored-By: Lunaphied <lunaphied@lunaphied.me>
Co-Authored-By: Maximilian Bosch <maximilian@mbosch.me>
Co-Authored-By: Pierre Bourdon <delroth@gmail.com>
Co-Authored-By: Qyriad <qyriad@qyriad.me>
Co-Authored-By: Rebecca Turner <rbt@sent.as>
Co-Authored-By: Winter <winter@winter.cafe>
Co-Authored-By: eldritch horrors <pennae@lix.systems>
Co-Authored-By: jade <lix@jade.fyi>
Co-Authored-By: julia <midnight@trainwit.ch>
Co-Authored-By: rebecca “wiggles” turner <rbt@sent.as>
Co-Authored-By: wiggles dog <rbt@sent.as>
Co-Authored-By: fricklerhandwerk <valentin@fricklerhandwerk.de>
Co-authored-By: Eli Schwartz <eschwartz93@gmail.com>
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
This is not really part of the evaluator: it is just an integration
between Boehm GC and Boost coroutines usable for any purpose. The
evaluator (merely) optionally uses it.
Since 24.05 (I think) we need to pass `-c` or Clang thinks we want to
compile *both* a final executable and precompiled header file, and
complains that we cannot use `-o` with multiple outputs. `-c` seems fine
with GCC too, so I just put it in there conditionally.
the thesis is still the defining document with all the motivation and
explanations.
adding it here for greater visibility.
also more emphasis and clarity around the community aspect.
the hydra build job seems a bit arbitrary right there. may be better for
the contributing guide.
The implementation of `nix::createDirs` allows it to be a simple wrapper
around `std::filesystem::create_directories` as its return value is not
used anywhere.
throwing exceptions is fine, but throwing exceptions during exception
handling is hard enough to do correctly that we should just forbid it
entirely out of an overabundance of caution. in cases where terminate
is the correct answer the users of Finally must call it manually now.
Source: https://git.lix.systems/lix-project/lix/commit/6c777476c9e97abfc5232f0707985caf6df2baea
* docs: fix python nix-shell example
This Python code snippet depended on Python 2 which has been marked as insecure in 24.05.
I modernized the example so new users will not be surprised upon copying and pasting the snippet for exploration.
Co-authored-by: John Ericson <git@JohnEricson.me>
Before, `-lnixutil` was just stuck in `nix-store.pc`, but that doesn't
seem so nice.
This prepares us to distribute `libnixutil` in a separate package if we
want, but it should be a good change either way. I suspect it wasn't
done before because libutil was an extra unstable interface, but I don't
think we need worry about that. *All* the C++ is less stable than the C
(or that's the goal at least).
For what it's worth, Lix also created this pkg-config file *en passant*
during their rename:
https://git.lix.systems/lix-project/lix/commit/c97e17144e0d0b666d7b79d8b4b0d581bfdf373b#diff-3c4f60cc44a0e35444c7f45331cfa50f76637118
This re-enables support for older bwdgc versions without complicating
the code too much.
Coroutines generally only interfere with GC during source filtering,
so it's not too bad of a regression on older bdwgc.
This seems preferable over conditional compilation to enable the patch
etc; we've already spent a lot of complexity budget on this GC-coroutine
interaction...
Manually tested by printing to stderr in both branches (sp in os
stack, or not), and triggering a GC in a filterSource function,
e.g.:
let
generateTree = n: if n == 0 then "ha" else { left = generateTree (n - 1); right = generateTree (n - 1); };
in
builtins.deepSeq (generateTree 18) ...
Note that the darwin still uses the strategy of disabling GC, despite
having an implementation that compiles. The proper solution will be
enabled and tested later.
... so that we may perhaps later extend the interface.
Note that Nixpkgs' lib.warn already requires a string coercible
argument, so this is reasonable. Also note that string coercible
values aren't all strings, but in practice, for warn, they are.
Progress on #10832
This doesn't switch to auto-deriving the fields, but by defining `<=>`
we allow deriving `<=>` in downstream types where `Hash` is used.
Fixes assertion failure if outputsToInstall is empty by defaulting to the "out"
output. That is, behavior between the following commands should be consistent:
$ nix build --no-link --json .#nothing-to-install-no-out
error: derivation '/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-nothing-to-install-no-out.drv' does not have wanted outputs 'out'
$ nix build --no-link --file default.nix --json nothing-to-install-no-out
error: derivation '/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-nothing-to-install-no-out.drv' does not have wanted outputs 'out'
Real-world example of this issue:
$ nix build --json .#.legacyPackages.aarch64-linux.texlive.pkgs.iwona
error: derivation '/nix/store/dj0h6b0pnlnan5nidnhqa0bmzq4rv6sx-iwona-0.995b.drv' does not have wanted outputs 'out'
$ git rev-parse HEAD
eee33247cf6941daea8398c976bd2dda7962b125
$ nix build --json --file . texlive.pkgs.iwona
nix: src/libstore/outputs-spec.hh:46: nix::OutputsSpec::Names::Names(std::set<std::__cxx11::basic_string<char> >&&): Assertion `!empty()' failed.
Aborted (core dumped)
1. Fix build by making the legacy SSH Storey's secret `logFD` setting
not a setting on Windows. (It doesn't make sense to specify `void *`
handles by integer cross-proccess, I don't think.)
2. Move some files that don't need to be Unix-only anymore back to their
original locations.
Before:
$ nix flake lock --override-input nixpkgs gitlab:simple-nixos-mailserver/nixos-mailserver/nonexistent
fetching git input 'git+file:///home/linus/projects/lix'
fetching gitlab input 'gitlab:simple-nixos-mailserver/nixos-mailserver/nonexistent'
error: [json.exception.type_error.302] type must be string, but is null
After:
/tmp/inst/bin/nix flake lock --override-input nixpkgs gitlab:simple-nixos-mailserver/nixos-mailserver/nonexistent
warning: unknown experimental feature 'repl-flake'
error:
… while updating the lock file of flake 'git+file:///home/joerg/git/nix?ref=refs/heads/master&rev=62693c2c37c8edd92f95114eb1387b461fc671df'
… while updating the flake input 'nixpkgs'
… while fetching the input 'gitlab:simple-nixos-mailserver/nixos-mailserver/nonexistent'
error: No commits returned by GitLab API -- does the git ref really exist?
Adapted from: https://git.lix.systems/lix-project/lix/commit/3df013597d7a2b5e400839e6625c05bd47de4dca
This turns errors like:
error: flake output attribute 'hydraJobs' is not a derivation or path
into errors like:
error: expected flake output attribute 'hydraJobs' to be a derivation or
path but found a set: { binaryTarball = «thunk»; build = «thunk»; etc> }
This change affects all InstallableFlake commands.
Source: https://git.lix.systems/lix-project/lix/commit/20981461d4a2a62c68f4bc7c4258473f7cd7d8e1
Signed-off-by: Jörg Thalheim <joerg@thalheim.io>
* docs: mention importNative/exec in allow-unsafe-native-code-during-evaluation
Both of these still needs their own actual documentation, but they are
at least now mentioned that they exist and what they're enabled by.
Co-authored-by: Qyriad <qyriad@qyriad.me>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
- Get a rump derivation goal: hook instance will come later, local
derivation goal will come after that.
- Start cleaning up the channel / waiting code with an abstraction.
By moving `host` to the config, we can do a lot further cleanups and
dedups. This anticipates a world where we always go `StoreReference` ->
`*StoreConfig` -> `Store*` rather than skipping the middle step too.
Progress on #10766
Progress on https://github.com/NixOS/hydra/issues/1164
This increases test coverage, and gets the worker protocol ready to be
used by Hydra.
Why don't we just try to use the store interface in Hydra? Well, the
problem is that the store interface works on connection pools, with each
opreation getting potentially a different connection, but the way temp
roots work requires that we keep one logical "transaction" (temp root
session) using the same connection.
The longer-term solution probably is making connections themselves
implement the store interface, but that is something that builds on
this, so I feel OK that this is not churn in the wrong direction.
Fixes#9584
We don't want to rely on how C assigns numbers for enums in the wire
format. Sure, this is totally determined by the ABI, but it obscures the
code and makes it harder to safely change the enum definition (should we
need to) without accidentally breaking the wire format.
Do this instead of an unchecked cast
I redid this to use the serialisation framework (including a unit test),
but I am keeping the reference to credit Jade for spotting the issue.
Change-Id: Icf6af7935e8f139bef36b40ad475e973aa48855c
(adapted from commit 2a7a824d83dc5fb33326b8b89625685f283a743b)
Co-Authored-By: Jade Lovelace <lix@jade.fyi>
File not found while importing is not currently caught by the tab-completion handler.
Original bug report: https://git.lix.systems/lix-project/lix/issues/340
Fix has been adapted from https://gerrit.lix.systems/c/lix/+/1189
Example crash:
$ cat /tmp/foo.nix
{
someImport = import ./this_file_does_not_exist;
}
$ ./src/nix/nix repl --file /tmp/foo.nix
warning: unknown experimental feature 'repl-flake'
Nix 2.23.0pre20240517_dirty
Type :? for help.
Loading installable ''...
Added 1 variables.
nix-repl> someImport.<TAB>
This way we can commit the same amount of stack size (64 MB) without a conditional.
Includes nix, libnixexpr-tests, libnixfetchers-tests, libnixstore-tests, libnixutil-tests.
https://github.com/NixOS/nix/pull/10555 added a check requiring
that output parameters always have an uninitialized Value as argument.
Unfortunately the output parameter of the primop callback received
a thunk instead.
See the comment for implementation considerations.
This was accidentally removed in
e989c83b44. I restored it and also did a
few other cleanups:
- Make a static method for namespacing purposes
- Put the test files in the data dir with the other test data
- Avoid mutating globals in the machine config tests
This will be used by Hydra.
This is useful for diagnosing whether an evaluation is copying large
paths to the store. Example:
$ nix build .#packages.x86_64-linux.default --large-path-warning-threshold 1000000
warning: copied large path '/home/eelco/Dev/nix-master/' to the store (6271792 bytes)
warning: copied large path '«github:NixOS/nixpkgs/b550fe4b4776908ac2a861124307045f8e717c8e?narHash=sha256-7kkJQd4rZ%2BvFrzWu8sTRtta5D1kBG0LSRYAfhtmMlSo%3D»/' to the store (155263768 bytes)
warning: copied large path '«github:libgit2/libgit2/45fd9ed7ae1a9b74b957ef4f337bc3c8b3df01b5?narHash=sha256-oX4Z3S9WtJlwvj0uH9HlYcWv%2Bx1hqp8mhXl7HsLu2f0%3D»/' to the store (22175416 bytes)
warning: copied large path '/nix/store/z985088mcd6w23qwdlirsinnyzayagki-source' to the store (5885872 bytes)
Building derivations is a lot harder, but the downloading goals is
portable enough.
The "common channel" code is due to Volth. I wonder if there is a way we
can factor it out into separate functions / files to avoid some
within-function CPP.
Co-authored-by: volth <volth@volth.com>
Previously, the "file:./" prefix was not correctly recognized in
fixGitURL; instead, it was mistaken as a file path, which resulted in a
parsed url of the form "file://file:./".
This commit fixes the issue by properly detecting the "file:" prefix.
Note, however, that unlike "file://", the "file:./" URI is _not_
standardized, but has been widely used to referred to relative file
paths. In particular, the "git+file:./" did work for nix<=2.18, and was
broken since nix 2.19.0.
Finally, this commit fixes the issue completely for the 2.19 series, but
is still inadequate for the 2.20 series due to new behaviors from the
switch to libgit2. However, it does improve the correctness of parsing
even though it is not yet a complete solution.
When writing a shebang script, you expect your path to be relative to
the script, not the cwd. We previously handled this correctly for
relative file paths, but not for expressions.
This handles both -p & -E args. My understanding is this should be
what we want in any cases I can think of - people run scripts from
many different working directories. @edolstra is there any reason to
handle -p args differently in this case?
Fixes#4232
2023-11-27 14:40:38 -05:00
1024 changed files with 26502 additions and 22449 deletions
# 2. Create a cache for $githubuser-nix-install-tests
# 3. Create a cachix auth token and save it in https://github.com/$githubuser/nix/settings/secrets/actions in "Repository secrets" as CACHIX_AUTH_TOKEN
# Dockerhub:
# 1. Sign-up for https://hub.docker.com/
# 2. Store your dockerhub username as DOCKERHUB_USERNAME in "Repository secrets" of your fork repository settings (https://github.com/$githubuser/nix/settings/secrets/actions)
# 3. Create an access token in https://hub.docker.com/settings/security and store it as DOCKERHUB_TOKEN in "Repository secrets" of your fork
@@ -41,9 +41,9 @@ Check out the [security policy](https://github.com/NixOS/nix/security/policy).
There are many open pull requests that might already do what you intend to work on.
You can use [labels](https://github.com/NixOS/nix/labels) to filter for relevant topics.
3. Check the [Nix reference manual](https://nixos.org/manual/nix/unstable/contributing/hacking.html) for information on building Nix and running its tests.
3. Check the [Nix reference manual](https://nix.dev/manual/nix/development/development/building.html) for information on building Nix and running its tests.
For contributions to the command line interface, please check the [CLI guidelines](https://nixos.org/manual/nix/unstable/contributing/cli-guideline.html).
For contributions to the command line interface, please check the [CLI guidelines](https://nix.dev/manual/nix/development/development/cli-guideline.html).
4. Make your change!
@@ -69,7 +69,7 @@ Check out the [security policy](https://github.com/NixOS/nix/security/policy).
- [ ] API documentation in header files
- [ ] Code and comments are self-explanatory
- [ ] Commit message explains **why** the change was made
- [ ] New feature or incompatible change: [add a release note](https://nixos.org/manual/nix/stable/contributing/hacking#add-a-release-note)
- [ ] New feature or incompatible change: [add a release note](https://nix.dev/manual/nix/development/development/contributing.html#add-a-release-note)
7. If you need additional feedback or help to getting pull request into shape, ask other contributors using [@mentions](https://docs.github.com/en/get-started/writing-on-github/getting-started-with-writing-and-formatting-on-github/basic-writing-and-formatting-syntax#mentioning-people-and-teams).
@@ -78,7 +78,7 @@ Check out the [security policy](https://github.com/NixOS/nix/security/policy).
The Nix reference manual is hosted on https://nixos.org/manual/nix.
The underlying source files are located in [`doc/manual/src`](./doc/manual/src).
For small changes you can [use GitHub to edit these files](https://docs.github.com/en/repositories/working-with-files/managing-files/editing-files)
For larger changes see the [Nix reference manual](https://nixos.org/manual/nix/unstable/contributing/hacking.html).
For larger changes see the [Nix reference manual](https://nix.dev/manual/nix/development/development/contributing.html).
Nix is a powerful package manager for Linux and other Unix systems that makes package
management reliable and reproducible. Please refer to the [Nix manual](https://nixos.org/nix/manual)
management reliable and reproducible. Please refer to the [Nix manual](https://nix.dev/reference/nix-manual)
for more details.
## Installation and first steps
Visit [nix.dev](https://nix.dev) for [installation instructions](https://nix.dev/tutorials/install-nix) and [beginner tutorials](https://nix.dev/tutorials/first-steps).
Full reference documentation can be found in the [Nix manual](https://nixos.org/nix/manual).
Full reference documentation can be found in the [Nix manual](https://nix.dev/reference/nix-manual).
## Building And Developing
## Building and developing
See our [Hacking guide](https://nixos.org/manual/nix/unstable/contributing/hacking.html) in our manual for instruction on how to
set up a development environment and build Nix from source.
Follow instructions in the Nix reference manual to [set up a development environment and build Nix from source](https://nix.dev/manual/nix/development/building.html).
## Contributing
Check the [contributing guide](./CONTRIBUTING.md) if you want to get involved with developing Nix.
## Additional Resources
## Additional resources
- [Nix manual](https://nixos.org/nix/manual)
- [Nix jobsets on hydra.nixos.org](https://hydra.nixos.org/project/nix)
Nix was created by Eelco Dolstra and developed as the subject of his PhD thesis [The Purely Functional Software Deployment Model](https://edolstra.github.io/pubs/phd-thesis.pdf), published 2006.
Today, a world-wide developer community contributes to Nix and the ecosystem that has grown around it.
- [The Nix, Nixpkgs, NixOS Community on nixos.org](https://nixos.org/)
- [Official documentation on nix.dev](https://nix.dev)
- [Nixpkgs](https://github.com/NixOS/nixpkgs) is [the largest, most up-to-date free software repository in the world](https://repology.org/repositories/graphs)
- [NixOS](https://github.com/NixOS/nixpkgs/tree/master/nixos) is a Linux distribution that can be configured fully declaratively
[test "$ENABLE_BUILD" == "no" && test "$ENABLE_UNIT_TESTS" == "yes"],
[AC_MSG_ERROR([Cannot enable unit tests when building overall is disabled. Please do not pass '--enable-unit-tests' or do not pass '--disable-build'.])])
@@ -171,15 +166,6 @@ AS_IF(
[test "$ENABLE_BUILD" == "no" && test "$ENABLE_DOC_GEN" == "yes"],
[AC_MSG_ERROR([Cannot enable generated docs when building overall is disabled. Please do not pass '--enable-doc-gen' or do not pass '--disable-build'.])])
# Building without API docs is the default as Nix' C++ interfaces are internal and unstable.
AC_ARG_ENABLE(internal-api-docs, AS_HELP_STRING([--enable-internal-api-docs],[Build API docs for Nix's internal unstable C++ interfaces]),
synopsis:Define integer overflow in the Nix language as an error
issues:[10968]
prs:[11188]
---
Previously, integer overflow in the Nix language invoked C++ level signed overflow, which was undefined behaviour, but *usually* manifested as wrapping around on overflow.
Since prior to the public release of Lix, Lix had C++ signed overflow defined to crash the process and nobody noticed this having accidentally removed overflow from the Nix language for three months until it was caught by fiddling around.
Given the significant body of actual Nix code that has been evaluated by Lix in that time, it does not appear that nixpkgs or much of importance depends on integer overflow, so it appears safe to turn into an error.
Some other overflows were fixed:
-`builtins.fromJSON` of values greater than the maximum representable value in a signed 64-bit integer will generate an error.
-`nixConfig` in flakes will no longer accept negative values for configuration options.
Integer overflow now looks like the following:
```
$ nix eval --expr '9223372036854775807 + 1'
error: integer overflow in adding 9223372036854775807 + 1
The `build-hook` setting's default is less useful when using `libnixstore` as a library
prs:
- 11178
---
*This is an obscure issue that only affects usage of the `libnixstore` library outside of the Nix executable.*
As part the ongoing [rewrite of the build system](https://github.com/NixOS/nix/issues/2503) to use [Meson](https://mesonbuild.com/), we are also switching to packaging individual Nix components separately (and building them in separate derivations).
This means that when building `libnixstore` we do not know where the Nix binaries will be installed --- `libnixstore` doesn't know about downstream consumers like the Nix binaries at all.
*This is also unrelated to the _`post`_-`build-hook`*, which is often used for pushing to a cache.*
This has a small adverse affect on remote building --- the `build-remote` executable that is specified from the [`build-hook`](@docroot@/command-ref/conf-file.md#conf-build-hook) setting will not be gotten from the (presumed) installation location, but instead looked up on the `PATH`.
This means that other applications linking `libnixstore` that wish to use remote building must arrange for the `nix` command to be on the PATH (or manually overriding `build-hook`) in order for that to work.
Long term we don't envision this being a downside, because we plan to [get rid of `build-remote` and the build hook setting entirely](https://github.com/NixOS/nix/issues/1221).
There is simply no need to add a second layer of remote-procedure-calling when we want to connect to a remote builder.
The build hook protocol did in principle support custom ways of remote building, but that can also be accomplished with a custom service for the ssh or daemon/ssh-ng protocols, or with a custom [store type](@docroot@/store/types/) i.e. `Store` subclass. <!-- we normally don't mention classes, but consider that this release note is about a library use case -->
The Perl bindings no longer expose `getBinDir` either, since they libraries those bindings wrap no longer know the location of installed binaries as described above.
synopsis:Modify `nix derivation {add,show}` JSON format
issues:9866
prs:10722
---
The JSON format for derivations has been slightly revised to better conform to our [JSON guidelines](@docroot@contributing/cli-guideline#returning-future-proof-json).
In particular, the hash algorithm and content addressing method of content-addresed derivation outputs is now separated into two fields `hashAlgo` and `method`,
rather than one field with an arcane `:`-separated format.
This JSON format is only used by the experimental `nix derivation` family of commands, at this time.
Future revisions are expected as the JSON format is still not entirely in compliance even after these changes.
A colon-separated list of search path entries used to resolve [lookup paths](@docroot@/language/constructs/lookup-path.md).
If `NIX_PATH` is not set at all, Nix will fall back to the following list in [impure](@docroot@/command-ref/conf-file.md#conf-pure-eval) and [unrestricted](@docroot@/command-ref/conf-file.md#conf-restrict-eval) evaluation mode:
This environment variable overrides the value of the [`nix-path` configuration setting](@docroot@/command-ref/conf-file.md#conf-nix-path).
This is the equivalent of invoking [`nix-env --delete-generations old`](@docroot@/command-ref/nix-env/delete-generations.md#generations-old) on each found profile.
Attempt to download missing store objects on the target from [substituters](@docroot@/command-ref/conf-file.md#conf-substituters).
Any store objects that cannot be substituted on the target are still copied normally from the source.
This is useful, for instance, if the connection between the source and target machine is slow, but the connection between the target machine and `cache.nixos.org` (the default binary cache server) is fast.
Attempt to download missing store objects on the target from [substituters](@docroot@/command-ref/conf-file.md#conf-substituters).
Any store objects that cannot be substituted on the target are still copied normally from the source.
This is useful, for instance, if the connection between the source and target machine is slow, but the connection between the target machine and `cache.nixos.org` (the default binary cache server) is fast.
{{#include ./opt-common.md}}
# Environment variables
-`NIX_SSHOPTS`
-`NIX_SSHOPTS`
Additional options to be passed to `ssh` on the command line.
Additional options to be passed to `ssh` on the command line.
@@ -21,125 +21,125 @@ It is based on the current generation of the active [profile](@docroot@/command-
The arguments *args* map to store paths in a number of possible ways:
- By default, *args* is a set of [derivation] names denoting derivations in the [default Nix expression].
These are [realised], and the resulting output paths are installed.
Currently installed derivations with a name equal to the name of a derivation being added are removed unless the option `--preserve-installed` is specified.
- By default, *args* is a set of [derivation] names denoting derivations in the [default Nix expression].
These are [realised], and the resulting output paths are installed.
Currently installed derivations with a name equal to the name of a derivation being added are removed unless the option `--preserve-installed` is specified.
If there are multiple derivations matching a name in *args* that
have the same name (e.g., `gcc-3.3.6` and `gcc-4.1.1`), then the
derivation with the highest *priority* is used. A derivation can
define a priority by declaring the `meta.priority` attribute. This
attribute should be a number, with a higher value denoting a lower
priority. The default priority is `5`.
If there are multiple derivations matching a name in *args* that
have the same name (e.g., `gcc-3.3.6` and `gcc-4.1.1`), then the
derivation with the highest *priority* is used. A derivation can
define a priority by declaring the `meta.priority` attribute. This
attribute should be a number, with a higher value denoting a lower
priority. The default priority is `5`.
If there are multiple matching derivations with the same priority,
then the derivation with the highest version will be installed.
If there are multiple matching derivations with the same priority,
then the derivation with the highest version will be installed.
You can force the installation of multiple derivations with the same
name by being specific about the versions. For instance, `nix-env --install
gcc-3.3.6 gcc-4.1.1` will install both version of GCC (and will
probably cause a user environment conflict\!).
You can force the installation of multiple derivations with the same
name by being specific about the versions. For instance, `nix-env --install
gcc-3.3.6 gcc-4.1.1` will install both version of GCC (and will
probably cause a user environment conflict\!).
- If [`--attr`](#opt-attr) / `-A` is specified, the arguments are *attribute paths* that select attributes from the [default Nix expression].
This is faster than using derivation names and unambiguous.
Show the attribute paths of available packages with [`nix-env --query`](./query.md):
- If [`--attr`](#opt-attr) / `-A` is specified, the arguments are *attribute paths* that select attributes from the [default Nix expression].
This is faster than using derivation names and unambiguous.
Show the attribute paths of available packages with [`nix-env --query`](./query.md):
```console
nix-env --query --available --attr-path
```
```console
nix-env --query --available --attr-path
```
- If `--from-profile` *path* is given, *args* is a set of names
denoting installed [store paths] in the profile *path*. This is an
easy way to copy user environment elements from one profile to
another.
- If `--from-profile` *path* is given, *args* is a set of names
denoting installed [store paths] in the profile *path*. This is an
easy way to copy user environment elements from one profile to
another.
- If `--from-expression` is given, *args* are [Nix language functions](@docroot@/language/constructs.md#functions) that are called with the [default Nix expression] as their single argument.
The derivations returned by those function calls are installed.
This allows derivations to be specified in an unambiguous way, which is necessary if there are multiple derivations with the same name.
- If `--from-expression` is given, *args* are [Nix language functions](@docroot@/language/syntax.md#functions) that are called with the [default Nix expression] as their single argument.
The derivations returned by those function calls are installed.
This allows derivations to be specified in an unambiguous way, which is necessary if there are multiple derivations with the same name.
- If *args* are [store derivations](@docroot@/glossary.md#gloss-store-derivation), then these are [realised], and the resulting output paths are installed.
- If *args* are [store derivations](@docroot@/glossary.md#gloss-store-derivation), then these are [realised], and the resulting output paths are installed.
- If *args* are [store paths] that are not store derivations, then these are [realised] and installed.
- If *args* are [store paths] that are not store derivations, then these are [realised] and installed.
- By default all [outputs](@docroot@/language/derivations.md#attr-outputs) are installed for each [derivation].
This can be overridden by adding a `meta.outputsToInstall` attribute on the derivation listing a subset of the output names.
- By default all [outputs](@docroot@/language/derivations.md#attr-outputs) are installed for each [derivation].
This can be overridden by adding a `meta.outputsToInstall` attribute on the derivation listing a subset of the output names.
Example:
Example:
The file `example.nix` defines a derivation with two outputs `foo` and `bar`, each containing a file.
The file `example.nix` defines a derivation with two outputs `foo` and `bar`, each containing a file.
```nix
# example.nix
let
pkgs = import <nixpkgs> {};
command = ''
${pkgs.coreutils}/bin/mkdir -p $foo $bar
echo foo > $foo/foo-file
echo bar > $bar/bar-file
'';
in
derivation {
name = "example";
builder = "${pkgs.bash}/bin/bash";
args = [ "-c" command ];
outputs = [ "foo" "bar" ];
system = builtins.currentSystem;
}
```
```nix
# example.nix
let
pkgs = import <nixpkgs> {};
command = ''
${pkgs.coreutils}/bin/mkdir -p $foo $bar
echo foo > $foo/foo-file
echo bar > $bar/bar-file
'';
in
derivation {
name = "example";
builder = "${pkgs.bash}/bin/bash";
args = [ "-c" command ];
outputs = [ "foo" "bar" ];
system = builtins.currentSystem;
}
```
Installing from this Nix expression will make files from both outputs appear in the current profile.
Installing from this Nix expression will make files from both outputs appear in the current profile.
```console
$ nix-env --install --file example.nix
installing 'example'
$ ls ~/.nix-profile
foo-file
bar-file
manifest.nix
```
```console
$ nix-env --install --file example.nix
installing 'example'
$ ls ~/.nix-profile
foo-file
bar-file
manifest.nix
```
Adding `meta.outputsToInstall` to that derivation will make `nix-env` only install files from the specified outputs.
Adding `meta.outputsToInstall` to that derivation will make `nix-env` only install files from the specified outputs.
The script's file name is passed as the first argument to the interpreter specified by the `-i` flag.
Aside from the very first line, which is a directive to the operating system, the additional `#! nix-shell` lines do not need to be at the beginning of the file.
This allows wrapping them in block comments for languages where `#` does not start a comment, such as ECMAScript, Erlang, PHP, or Ruby.
@@ -37,7 +37,7 @@ Most Nix commands accept the following command-line options:
Print even more informational messages.
-`4` “Debug”
Print debug information.
-`5` “Vomit”
@@ -143,7 +143,7 @@ Most Nix commands accept the following command-line options:
This option is accepted by `nix-env`, `nix-instantiate`, `nix-shell` and `nix-build`.
When evaluating Nix expressions, the expression evaluator will automatically try to call functions that it encounters.
It can automatically call functions for which every argument has a [default value](@docroot@/language/constructs.md#functions) (e.g., `{ argName ? defaultValue }: ...`).
It can automatically call functions for which every argument has a [default value](@docroot@/language/syntax.md#functions) (e.g., `{ argName ? defaultValue }: ...`).
With `--arg`, you can also call functions that have arguments without a default value (or override a default value).
That is, if the evaluator encounters a function with an argument named *name*, it will call it with value *value*.
@@ -187,11 +187,12 @@ Most Nix commands accept the following command-line options:
For `nix-shell`, this option is commonly used to give you a shell in which you can build the packages returned by the expression.
If you want to get a shell which contain the *built* packages ready for use, give your expression to the `nix-shell --packages ` convenience flag instead.
Add an entry to the [Nix expression search path](@docroot@/command-ref/conf-file.md#conf-nix-path).
Add an entry to the list of search paths used to resolve [lookup paths](@docroot@/language/constructs/lookup-path.md).
This option may be given multiple times.
Paths added through `-I` take precedence over [`NIX_PATH`](@docroot@/command-ref/env-common.md#env-NIX_PATH).
Paths added through `-I` take precedence over the [`nix-path` configuration setting](@docroot@/command-ref/conf-file.md#conf-nix-path) and the [`NIX_PATH` environment variable](@docroot@/command-ref/env-common.md#env-NIX_PATH).
This section provides some notes on how to hack on Nix. To get the
latest version of Nix from GitHub:
This section provides some notes on how to start hacking on Nix.
To get the latest version of Nix from GitHub:
```console
$ git clone https://github.com/NixOS/nix.git
$ cd nix
```
The following instructions assume you already have some version of Nix installed locally, so that you can use it to set up the development environment. If you don't have it installed, follow the [installation instructions].
> **Note**
>
> The following instructions assume you already have some version of Nix installed locally, so that you can use it to set up the development environment.
> If you don't have it installed, follow the [installation instructions](../installation/index.md).
@@ -389,88 +389,6 @@ colors, no emojis and using ASCII instead of Unicode symbols). The same should
happen when TTY is not detected on STDERR. We should not display progress /
status section, but only print warnings and errors.
## Returning future proof JSON
The schema of JSON output should allow for backwards compatible extension. This section explains how to achieve this.
Two definitions are helpful here, because while JSON only defines one "key-value"
object type, we use it to cover two use cases:
- **dictionary**: a map from names to value that all have the same type. In
C++ this would be a `std::map` with string keys.
- **record**: a fixed set of attributes each with their own type. In C++, this
would be represented by a `struct`.
It is best not to mix these use cases, as that may lead to incompatibilities when the schema changes. For example, adding a record field to a dictionary breaks consumers that assume all JSON object fields to have the same meaning and type.
This leads to the following guidelines:
- The top-level (root) value must be a record.
Otherwise, one can not change the structure of a command's output.
- The value of a dictionary item must be a record.
Otherwise, the item type can not be extended.
- List items should be records.
Otherwise, one can not change the structure of the list items.
If the order of the items does not matter, and each item has a unique key that is a string, consider representing the list as a dictionary instead. If the order of the items needs to be preserved, return a list of records.
- Streaming JSON should return records.
An example of a streaming JSON format is [JSON lines](https://jsonlines.org/), where each line represents a JSON value. These JSON values can be considered top-level values or list items, and they must be records.
### Examples
This is bad, because all keys must be assumed to be store types:
```json
{
"local": { ... },
"remote": { ... },
"http": { ... }
}
```
This is good, because the it is extensible at the root, and is somewhat self-documenting:
```json
{
"storeTypes": { "local": { ... }, ... },
"pluginSupport": true
}
```
While the dictionary of store types seems like a very complete response at first, a use case may arise that warrants returning additional information.
For example, the presence of plugin support may be crucial information for a client to proceed when their desired store type is missing.
The following representation is bad because it is not extensible:
```json
{ "outputs": [ "out" "bin" ] }
```
However, simply converting everything to records is not enough, because the order of outputs must be preserved:
```json
{ "outputs": { "bin": {}, "out": {} } }
```
The first item is the default output. Deriving this information from the outputs ordering is not great, but this is how Nix currently happens to work.
While it is possible for a JSON parser to preserve the order of fields, we can not rely on this capability to be present in all JSON libraries.
This representation is extensible and preserves the ordering:
@@ -5,4 +5,4 @@ Check the [contributing guide](https://github.com/NixOS/nix/blob/master/CONTRIBU
This chapter is a collection of guides for making changes to the code and documentation.
If you're not sure where to start, try to [compile Nix from source](./hacking.md) and consider [making improvements to documentation](./documentation.md).
If you're not sure where to start, try to [compile Nix from source](./building.md) and consider [making improvements to documentation](./documentation.md).
Nix consumes and produces JSON in a variety of contexts.
These guidelines ensure consistent practices for all our JSON interfaces, for ease of use, and so that experience in one part carries over to another.
## Extensibility
The schema of JSON input and output should allow for backwards compatible extension.
This section explains how to achieve this.
Two definitions are helpful here, because while JSON only defines one "key-value" object type, we use it to cover two use cases:
- **dictionary**: a map from names to value that all have the same type.
In C++ this would be a `std::map` with string keys.
- **record**: a fixed set of attributes each with their own type.
In C++, this would be represented by a `struct`.
It is best not to mix these use cases, as that may lead to incompatibilities when the schema changes.
For example, adding a record field to a dictionary breaks consumers that assume all JSON object fields to have the same meaning and type, and dictionary items with a colliding name can not be represented anymore.
This leads to the following guidelines:
- The top-level (root) value must be a record.
Otherwise, one can not change the structure of a command's output.
- The value of a dictionary item must be a record.
Otherwise, the item type can not be extended.
- List items should be records.
Otherwise, one can not change the structure of the list items.
If the order of the items does not matter, and each item has a unique key that is a string, consider representing the list as a dictionary instead.
If the order of the items needs to be preserved, return a list of records.
- Streaming JSON should return records.
An example of a streaming JSON format is [JSON lines](https://jsonlines.org/), where each line represents a JSON value.
These JSON values can be considered top-level values or list items, and they must be records.
### Examples
This is bad, because all keys must be assumed to be store types:
```json
{
"local": { ... },
"remote": { ... },
"http": { ... }
}
```
This is good, because the it is extensible at the root, and is somewhat self-documenting:
```json
{
"storeTypes": { "local": { ... }, ... },
"pluginSupport": true
}
```
While the dictionary of store types seems like a very complete response at first, a use case may arise that warrants returning additional information.
For example, the presence of plugin support may be crucial information for a client to proceed when their desired store type is missing.
The following representation is bad because it is not extensible:
```json
{ "outputs": [ "out" "bin" ] }
```
However, simply converting everything to records is not enough, because the order of outputs must be preserved:
```json
{ "outputs": { "bin": {}, "out": {} } }
```
The first item is the default output. Deriving this information from the outputs ordering is not great, but this is how Nix currently happens to work.
While it is possible for a JSON parser to preserve the order of fields, we can not rely on this capability to be present in all JSON libraries.
This representation is extensible and preserves the ordering:
@@ -59,15 +59,15 @@ The unit tests are defined using the [googletest] and [rapidcheck] frameworks.
> …
> ```
The tests for each Nix library (`libnixexpr`, `libnixstore`, etc..) live inside a directory `tests/unit/${library_name_without-nix}`.
Given an interface (header) and implementation pair in the original library, say, `src/libexpr/value/context.{hh,cc}`, we write tests for it in `tests/unit/libexpr/tests/value/context.cc`, and (possibly) declare/define additional interfaces for testing purposes in `tests/unit/libexpr-support/tests/value/context.{hh,cc}`.
The tests for each Nix library (`libnixexpr`, `libnixstore`, etc..) live inside a directory `src/${library_name_without-nix}-test`.
Given an interface (header) and implementation pair in the original library, say, `src/libexpr/value/context.{hh,cc}`, we write tests for it in `src/nix-expr-tests/value/context.cc`, and (possibly) declare/define additional interfaces for testing purposes in `src/nix-expr-test-support/tests/value/context.{hh,cc}`.
Data for unit tests is stored in a `data` subdir of the directory for each unit test executable.
For example, `libnixstore` code is in `src/libstore`, and its test data is in `tests/unit/libstore/data`.
The path to the `tests/unit/data` directory is passed to the unit test executable with the environment variable `_NIX_TEST_UNIT_DATA`.
For example, `libnixstore` code is in `src/libstore`, and its test data is in `src/nix-store-tests/data`.
The path to the `src/${library_name_without-nix}-test/data` directory is passed to the unit test executable with the environment variable `_NIX_TEST_UNIT_DATA`.
Note that each executable only gets the data for its tests.
The unit test libraries are in `tests/unit/${library_name_without-nix}-lib`.
The unit test libraries are in `src/${library_name_without-nix}-test-support`.
All headers are in a `tests` subdirectory so they are included with `#include "tests/"`.
The use of all these separate directories for the unit tests might seem inconvenient, as for example the tests are not "right next to" the part of the code they are testing.
@@ -76,8 +76,25 @@ there is no risk of any build-system wildcards for the library accidentally pick
### Running tests
You can run the whole testsuite with `make check`, or the tests for a specific component with `make libfoo-tests_RUN`.
Finer-grained filtering is also possible using the [--gtest_filter](https://google.github.io/googletest/advanced.html#running-a-subset-of-the-tests) command-line option, orthe `GTEST_FILTER` environment variable, e.g. `GTEST_FILTER='ErrorTraceTest.*' make check`.
You can run the whole testsuite with `meson test` from the Meson build directory, or the tests for a specific component with `meson test nix-store-tests`.
A environment variables that Google Test accepts are also worth knowing:
@@ -86,7 +103,7 @@ See [functional characterisation testing](#characterisation-testing-functional)
Like with the functional characterisation, `_NIX_TEST_ACCEPT=1` is also used.
For example:
```shell-session
$ _NIX_TEST_ACCEPT=1 make libstore-tests_RUN
$ _NIX_TEST_ACCEPT=1 meson test nix-store-tests -v
...
[ SKIPPED ] WorkerProtoTest.string_read
[ SKIPPED ] WorkerProtoTest.string_write
@@ -114,6 +131,8 @@ On other platforms they wouldn't be run at all.
The functional tests reside under the `tests/functional` directory and are listed in `tests/functional/local.mk`.
Each test is a bash script.
Functional tests are run during `installCheck` in the `nix` package build, as well as separately from the build, in VM tests.
### Running the whole test suite
The whole test suite can be run with:
@@ -252,13 +271,30 @@ Regressions are caught, and improvements always show up in code review.
To ensure that characterisation testing doesn't make it harder to intentionally change these interfaces, there always must be an easy way to regenerate the expected output, as we do with `_NIX_TEST_ACCEPT=1`.
### Running functional tests on NixOS
We run the functional tests not just in the build, but also in VM tests.
This helps us ensure that Nix works correctly on NixOS, and environments that have similar characteristics that are hard to reproduce in a build environment.
The recommended way to run these tests during development is:
The `quickBuild` attribute configures the test to use a `nix` package that's built without integration tests, so that you can iterate on the tests without performing recompilations due to the changed sources for `installCheck`.
Generally, this build is sufficient, but in nightly or CI we also test the attributes `functional_root` and `functional_trusted`, in which the test suite is run with different levels of authorization.
## Integration tests
The integration tests are defined in the Nix flake under the `hydraJobs.tests` attribute.
These tests include everything that needs to interact with external services or run Nix in a non-trivial distributed setup.
Because these tests are expensive and require more than what the standard github-actions setup provides, they only run on the master branch (on <https://hydra.nixos.org/jobset/nix/master>).
You can run them manually with `nix build .#hydraJobs.tests.{testName}` or `nix-build -A hydraJobs.tests.{testName}`
You can run them manually with `nix build .#hydraJobs.tests.{testName}` or `nix-build -A hydraJobs.tests.{testName}`.
If you are testing a build of `nix` that you haven't compiled yet, you may iterate faster by appending the `quickBuild` attribute: `nix build .#hydraJobs.tests.{testName}.quickBuild`.
Software Heritage's writing on [*Intrinsic and Extrinsic identifiers*](https://www.softwareheritage.org/2020/07/09/intrinsic-vs-extrinsic-identifiers) is also a good introduction to the value of content-addressing over other referencing schemes.
A [derivation] where a cryptographic hash of the [output] is determined in advance using the [`outputHash`](./language/advanced-attributes.md#adv-attr-outputHash) attribute, and where the [`builder`](@docroot@/language/derivations.md#attr-builder) executable has access to the network.
- [store]{#gloss-store}
@@ -137,9 +136,12 @@
- [content-addressed store object]{#gloss-content-addressed-store-object}
A [store object] whose [store path] is determined by its contents.
A [store object] which is [content-addressed](#gloss-content-address),
i.e. whose [store path] is determined by its contents.
This includes derivations, the outputs of [content-addressed derivations](#gloss-content-addressed-derivation), and the outputs of [fixed-output derivations](#gloss-fixed-output-derivation).
See [Content-Addressing Store Objects](@docroot@/store/store-object/content-address.md) for details.
- [substitute]{#gloss-substitute}
A substitute is a command invocation stored in the [Nix database] that
@@ -166,7 +168,7 @@
- [impure derivation]{#gloss-impure-derivation}
[An experimental feature](#@docroot@/contributing/experimental-features.md#xp-feature-impure-derivations) that allows derivations to be explicitly marked as impure,
[An experimental feature](#@docroot@/development/experimental-features.md#xp-feature-impure-derivations) that allows derivations to be explicitly marked as impure,
so that they are always rebuilt, and their outputs not reused by subsequent calls to realise them.
- [Nix database]{#gloss-nix-database}
@@ -310,7 +312,7 @@
- [package attribute set]{#package-attribute-set}
An [attribute set](@docroot@/language/values.md#attribute-set) containing the attribute `type = "derivation";` (derivation for historical reasons), as well as other attributes, such as
An [attribute set](@docroot@/language/types.md#attribute-set) containing the attribute `type = "derivation";` (derivation for historical reasons), as well as other attributes, such as
- attributes that refer to the files of a [package], typically in the form of [derivation outputs](#output),
- attributes that declare something about how the package is supposed to be installed or used,
- other metadata or arbitrary attributes.
@@ -323,9 +325,9 @@
See [String interpolation](./language/string-interpolation.md) for details.
These attributes declare that the derivation is a so-called
*fixed-output derivation*, which means that a cryptographic hash of
the output is already known in advance. When the build of a
fixed-output derivation finishes, Nix computes the cryptographic
hash of the output and compares it to the hash declared with these
attributes. If there is a mismatch, the build fails.
These attributes declare that the derivation is a so-called*fixed-output derivation* (FOD), which means that a cryptographic hash of the output is already known in advance.
As opposed to regular derivations, the [`builder`] executable of a fixed-output derivation has access to the network.
Nix computes a cryptographic hash of its output and compares that to the hash declared with these attributes.
If there is a mismatch, the derivation fails.
The rationale for fixed-output derivations is derivations such as
those produced by the `fetchurl` function. This function downloads a
@@ -227,7 +226,7 @@ Derivations can declare some infrequently used optional attributes.
@@ -279,7 +278,9 @@ Derivations can declare some infrequently used optional attributes.
> **Note**
>
> If set to `false`, the [`builder`](./derivations.md#attr-builder) should be able to run on the system type specified in the [`system` attribute](./derivations.md#attr-system), since the derivation cannot be substituted.
> If set to `false`, the [`builder`] should be able to run on the system type specified in the [`system` attribute](./derivations.md#attr-system), since the derivation cannot be substituted.
If the special attribute `__structuredAttrs` is set to `true`, the other derivation
@@ -301,6 +302,12 @@ Derivations can declare some infrequently used optional attributes.
(associative) arrays. For example, the attribute `hardening.format = true`
ends up as the Bash associative array element `${hardening[format]}`.
> **Warning**
>
> If set to `true`, other advanced attributes such as [`allowedReferences`](#adv-attr-allowedReferences), [`allowedReferences`](#adv-attr-allowedReferences), [`allowedRequisites`](#adv-attr-allowedRequisites),
[`disallowedReferences`](#adv-attr-disallowedReferences) and [`disallowedRequisites`](#adv-attr-disallowedRequisites), maxSize, and maxClosureSize.
will have no effect.
- [`outputChecks`]{#adv-attr-outputChecks}\
When using [structured attributes](#adv-attr-structuredAttrs), the `outputChecks`
attribute allows defining checks per-output.
@@ -363,6 +370,6 @@ Derivations can declare some infrequently used optional attributes.
ensures that the derivation can only be built on a machine with the `kvm` feature.
A lookup path is an identifier with an optional path suffix that resolves to a [path value](@docroot@/language/values.md#type-path) if the identifier matches a search path entry.
A lookup path is an identifier with an optional path suffix that resolves to a [path value](@docroot@/language/types.md#type-path) if the identifier matches a search path entry.
The value of a lookup path is determined by [`builtins.nixPath`](@docroot@/language/builtin-constants.md#builtins-nixPath).
The value of a lookup path is determined by [`builtins.nixPath`](@docroot@/language/builtins.md#builtins-nixPath).
See [`builtins.findFile`](@docroot@/language/builtins.md#builtins-findFile) for details on lookup path resolution.
The system type on which the [`builder`](#attr-builder) executable is meant to be run.
@@ -64,9 +64,9 @@ It outputs an attribute set, and produces a [store derivation] as a side effect
> }
> ```
>
> [`builtins.currentSystem`](@docroot@/language/builtin-constants.md#builtins-currentSystem) has the value of the [`system` configuration option], and defaults to the system type of the current Nix installation.
> [`builtins.currentSystem`](@docroot@/language/builtins.md#builtins-currentSystem) has the value of the [`system` configuration option], and defaults to the system type of the current Nix installation.
An *identifier* is an [ASCII](https://en.wikipedia.org/wiki/ASCII) character sequence that:
- Starts with a letter (`a-z`, `A-Z`) or underscore (`_`)
- Can contain any number of:
- Letters (`a-z`, `A-Z`)
- Digits (`0-9`)
- Underscores (`_`)
- Apostrophes (`'`)
- Hyphens (`-`)
- Is not one of the [keywords](#keywords)
> **Syntax**
>
> *identifier* ~ `[A-Za-z_][A-Za-z0-9_'-]*`
# Names
A *name* can be written as an [identifier](#identifier) or a [string literal](./string-literals.md).
> **Syntax**
>
> *name* → *identifier* | *string*
Names are used in [attribute sets](./syntax.md#attrs-literal), [`let` bindings](./syntax.md#let-expressions), and [`inherit`](./syntax.md#inheriting-attributes).
Two names are the same if they represent the same sequence of characters, regardless of whether they are written as identifiers or strings.
# Keywords
These keywords are reserved and cannot be used as [identifiers](#identifiers):
- [`assert`](./syntax.md#assertions)
- [`else`][if]
- [`if`][if]
- [`in`][let]
- [`inherit`](./syntax.md#inheriting-attributes)
- [`let`][let]
- [`or`](./operators.md#attribute-selection) (see note)
- [`rec`](./syntax.md#recursive-sets)
- [`then`][if]
- [`with`](./syntax.md#with-expressions)
[if]: ./syntax.md#conditionals
[let]: ./syntax.md#let-expressions
> **Note**
>
> The Nix language evaluator currently allows `or` to be used as a name in some contexts, for backwards compatibility reasons.
> Users are advised not to rely on this.
>
> There are long-standing issues with how `or` is parsed as a name, which can't be resolved without making a breaking change to the language.
@@ -500,7 +500,7 @@ This is an incomplete overview of language features, by example.
</td>
<td>
A [function](@docroot@/language/constructs.md#functions) that expects an integer and returns it increased by 1.
A [function](@docroot@/language/syntax.md#functions) that expects an integer and returns it increased by 1.
</td>
</tr>
@@ -512,7 +512,7 @@ This is an incomplete overview of language features, by example.
</td>
<td>
Curried [function](@docroot@/language/constructs.md#functions), equivalent to `x: (y: x + y)`. Can be used like a function that takes two arguments and returns their sum.
Curried [function](@docroot@/language/syntax.md#functions), equivalent to `x: (y: x + y)`. Can be used like a function that takes two arguments and returns their sum.
</td>
</tr>
@@ -524,7 +524,7 @@ This is an incomplete overview of language features, by example.
</td>
<td>
A [function](@docroot@/language/constructs.md#functions) call (evaluates to 101)
A [function](@docroot@/language/syntax.md#functions) call (evaluates to 101)
</td>
</tr>
@@ -536,7 +536,7 @@ This is an incomplete overview of language features, by example.
</td>
<td>
A [function](@docroot@/language/constructs.md#functions) bound to a variable and subsequently called by name (evaluates to 103)
A [function](@docroot@/language/syntax.md#functions) bound to a variable and subsequently called by name (evaluates to 103)
</td>
</tr>
@@ -548,7 +548,7 @@ This is an incomplete overview of language features, by example.
</td>
<td>
A [function](@docroot@/language/constructs.md#functions) that expects a set with required attributes `x` and `y` and concatenates them
A [function](@docroot@/language/syntax.md#functions) that expects a set with required attributes `x` and `y` and concatenates them
</td>
</tr>
@@ -560,7 +560,7 @@ This is an incomplete overview of language features, by example.
</td>
<td>
A [function](@docroot@/language/constructs.md#functions) that expects a set with required attribute `x` and optional `y`, using `"bar"` as default value for `y`
A [function](@docroot@/language/syntax.md#functions) that expects a set with required attribute `x` and optional `y`, using `"bar"` as default value for `y`
</td>
</tr>
@@ -572,7 +572,7 @@ This is an incomplete overview of language features, by example.
</td>
<td>
A [function](@docroot@/language/constructs.md#functions) that expects a set with required attributes `x` and `y` and ignores any other attributes
A [function](@docroot@/language/syntax.md#functions) that expects a set with required attributes `x` and `y` and ignores any other attributes
</td>
</tr>
@@ -586,7 +586,7 @@ This is an incomplete overview of language features, by example.
</td>
<td>
A [function](@docroot@/language/constructs.md#functions) that expects a set with required attributes `x` and `y`, and binds the whole set to `args`
A [function](@docroot@/language/syntax.md#functions) that expects a set with required attributes `x` and `y`, and binds the whole set to `args`
<!-- TODO(@rhendric, #10970): ^ rationalize number -> int/float -->
## Attribute selection
@@ -42,12 +46,6 @@
Select the attribute denoted by attribute path *attrpath* from [attribute set] *attrset*.
If the attribute doesn’t exist, return the *expr* after `or` if provided, otherwise abort evaluation.
An attribute path is a dot-separated list of [attribute names](./values.md#attribute-set).
> **Syntax**
>
> *attrpath* = *name* [ `.`*name* ]...
[Attribute selection]: #attribute-selection
## Has attribute
@@ -61,7 +59,7 @@ The result is a [Boolean] value.
See also: [`builtins.hasAttr`](@docroot@/language/builtins.md#builtins-hasAttr)
[Boolean]: ./values.md#type-boolean
[Boolean]: ./types.md#type-boolean
[Has attribute]: #has-attribute
@@ -69,8 +67,12 @@ After evaluating *attrset* and *attrpath*, the computational complexity is O(log
## Arithmetic
Numbers are type-compatible:
Pure integer operations will always return integers, whereas any operation involving at least one floating point number return a floating point number.
Numbers will retain their type unless mixed with other numeric types:
Pure integer operations will always return integers, whereas any operation involving at least one floating point number returns a floating point number.
Evaluation of the following numeric operations throws an evaluation error:
- Division by zero
- Integer overflow, that is, any operation yielding a result outside of the representable range of [Nix language integers](./syntax.md#number-literal)
See also [Comparison] and [Equality].
@@ -141,7 +143,7 @@ The result is a string.
Update [attribute set] *attrset1* with names and values from *attrset2*.
The returned attribute set will have of all the attributes in *attrset1* and *attrset2*.
The returned attribute set will have all of the attributes in *attrset1* and *attrset2*.
If an attribute name is present in both, the attribute value from the latter is taken.
[Update]: #update
@@ -172,7 +174,7 @@ All comparison operators are implemented in terms of `<`, and the following equi
- Numbers are type-compatible, see [arithmetic] operators.
- Floating point numbers only differ up to a limited precision.
[function]: ./constructs.md#functions
[function]: ./syntax.md#functions
[Equality]: #equality
@@ -182,3 +184,34 @@ Equivalent to `!`*b1* `||` *b2*.
A *scope* in the Nix language is a dictionary keyed by [name](./identifiers.md#names), mapping each name to an expression and a *definition type*.
The definition type is either *explicit* or *implicit*.
Each entry in this dictionary is a *definition*.
Explicit definitions are created by the following expressions:
- [let-expressions](syntax.md#let-expressions)
- [recursive attribute set literals](syntax.md#recursive-sets) (`rec`)
- [function literals](syntax.md#functions)
Implicit definitions are only created by [with-expressions](./syntax.md#with-expressions).
Every expression is *enclosed* by a scope.
The outermost expression is enclosed by the [built-in, global scope](./builtins.md), which contains only explicit definitions.
The expressions listed above *extend* their enclosing scope by adding new definitions, or replacing existing ones with the same name.
An explicit definition can replace a definition of any type; an implicit definition can only replace another implicit definition.
Each of the above expressions defines which of its subexpressions are enclosed by the extended scope.
In all other cases, the same scope that encloses an expression is the enclosing scope for its subexpressions.
The Nix language is [statically scoped](https://en.wikipedia.org/wiki/Scope_(computer_science)#Lexical_scope);
the value of a variable is determined only by the variable's enclosing scope, and not by the dynamic context in which the variable is evaluated.
> **Note**
>
> Expressions entered into the [Nix REPL](@docroot@/command-ref/new-cli/nix3-repl.md) are enclosed by a scope that can be extended by command line arguments or previous REPL commands.
> These ways of extending scope are not, strictly speaking, part of the Nix language.
@@ -4,9 +4,13 @@ String interpolation is a language feature where a [string], [path], or [attribu
Such a construct is called *interpolated string*, and the expression inside is an [interpolated expression](#interpolated-expression).
[string]: ./values.md#type-string
[path]: ./values.md#type-path
[attribute set]: ./values.md#attribute-set
[string]: ./types.md#type-string
[path]: ./types.md#type-path
[attribute set]: ./types.md#attribute-set
> **Syntax**
>
> *interpolation_element* → `${`*expression*`}`
## Examples
@@ -43,6 +47,47 @@ configureFlags = "
Note that Nix expressions and strings can be arbitrarily nested;
in this case the outer string contains various interpolated expressions that themselves contain strings (e.g., `"-thread"`), some of which in turn contain interpolated expressions (e.g., `${mesa}`).
To write a literal `${` in an regular string, escape it with a backslash (`\`).
> **Example**
>
> ```nix
> "echo \${PATH}"
> ```
>
> "echo ${PATH}"
To write a literal `${` in an indented string, escape it with two single quotes (`''`).
> **Example**
>
> ```nix
> ''
> echo ''${PATH}
> ''
> ```
>
> "echo ${PATH}\n"
`$${` can be written literally in any string.
> **Example**
>
> In Make, `$` in file names or recipes is represented as `$$`, see [GNU `make`: Basics of Variable Reference](https://www.gnu.org/software/make/manual/html_node/Reference.html#Basics-of-Variable-References).
> This can be expressed directly in the Nix language strings:
Note that negative numeric literals are actually parsed as unary negation of positive numeric literals.
This means that the minimum integer `-9223372036854775808` cannot be written as-is as a literal, since the positive number `9223372036854775808` is one past the maximum range.
See [arithmetic] and [comparison] operators for semantics.
[arithmetic]: ./operators.md#arithmetic
[comparison]: ./operators.md#comparison
### Path {#path-literal}
*Paths* can be expressed by path literals such as `./builder.sh`.
A path literal must contain at least one slash to be recognised as such.
For instance, `builder.sh` is not a path:
it's parsed as an expression that selects the attribute `sh` from the variable `builder`.
Path literals are resolved relative to their [base directory](@docroot@/glossary.md#gloss-base-directory).
Path literals may also refer to absolute paths by starting with a slash.
> **Note**
>
> Absolute paths make expressions less portable.
> In the case where a function translates a path literal into an absolute path string for a configuration file, it is recommended to write a string literal instead.
> This avoids some confusion about whether files at that location will be used during evaluation.
> It also avoids unintentional situations where some function might try to copy everything at the location into the store.
If the first component of a path is a `~`, it is interpreted such that the rest of the path were relative to the user's home directory.
For example, `~/foo` would be equivalent to `/home/edolstra/foo` for a user whose home directory is `/home/edolstra`.
Path literals that start with `~` are not allowed in [pure](@docroot@/command-ref/conf-file.md#conf-pure-eval) evaluation.
Path literals can also include [string interpolation], besides being [interpolated into other expressions].
[interpolated into other expressions]: ./string-interpolation.md#interpolated-expressions
At least one slash (`/`) must appear *before* any interpolated expression for the result to be recognized as a path.
`a.${foo}/b.${bar}` is a syntactically valid number division operation.
`./a.${foo}/b.${bar}` is a path.
[Lookup path](./constructs/lookup-path.md) literals such as `<nixpkgs>` also resolve to path values.
## List {#list-literal}
Lists are formed by enclosing a whitespace-separated list of values
between square brackets. For example,
```nix
[ 123 ./foo.nix "abc" (f { x = y; }) ]
```
defines a list of four elements, the last being the result of a call to
the function `f`. Note that function calls have to be enclosed in
parentheses. If they had been omitted, e.g.,
```nix
[ 123 ./foo.nix "abc" f { x = y; } ]
```
the result would be a list of five elements, the fourth one being a
function and the fifth being a set.
Note that lists are only lazy in values, and they are strict in length.
Elements in a list can be accessed using [`builtins.elemAt`](./builtins.md#builtins-elemAt).
## Attribute Set {#attrs-literal}
An attribute set is a collection of name-value-pairs called *attributes*.
Attribute sets are written enclosed in curly brackets (`{ }`).
Attribute names and attribute values are separated by an equal sign (`=`).
Each value can be an arbitrary expression, terminated by a semicolon (`;`)
An attribute name is a string without context, and is denoted by a [name] (an [identifier](./identifiers.md#identifiers) or [string literal](string-literals.md)).
[name]: ./identifiers.md#names
> **Syntax**
>
> *attrset* → `{` { *name*`=`*expr*`;` } `}`
Attributes can appear in any order.
An attribute name may only occur once in each attribute set.
> **Example**
>
> This defines an attribute set with attributes named:
> - `x` with the value `123`, an integer
> - `text` with the value `"Hello"`, a string
> - `y` where the value is the result of applying the function `f` to the attribute set `{ bla = 456; }`
>
> ```nix
> {
> x = 123;
> text = "Hello";
> y = f { bla = 456; };
> }
> ```
Attributes in nested attribute sets can be written using *attribute paths*.
> **Syntax**
>
> *attrset* → `{` { *attrpath*`=`*expr*`;` } `}`
An attribute path is a dot-separated list of [names][name].
> **Syntax**
>
> *attrpath* = *name* { `.`*name* }
<!-- -->
> **Example**
>
> ```nix
> { a.b.c = 1; a.b.d = 2; }
> ```
>
> {
> a = {
> b = {
> c = 1;
> d = 2;
> };
> };
> }
Attribute names can also be set implicitly by using the [`inherit` keyword](#inheriting-attributes).
> **Example**
>
> ```nix
> { inherit (builtins) true; }
> ```
>
> { true = true; }
Attributes can be accessed with the [`.` operator](./operators.md#attribute-selection).
Example:
```nix
{ a = "Foo"; b = "Bar"; }.a
```
This evaluates to `"Foo"`.
It is possible to provide a default value in an attribute selection using the `or` keyword.
Example:
```nix
{ a = "Foo"; b = "Bar"; }.c or "Xyzzy"
```
```nix
{ a = "Foo"; b = "Bar"; }.c.d.e.f.g or "Xyzzy"
```
will both evaluate to `"Xyzzy"` because there is no `c` attribute in the set.
You can use arbitrary double-quoted strings as attribute names:
```nix
{ "$!@#?" = 123; }."$!@#?"
```
```nix
let bar = "bar"; in
{ "foo ${bar}" = 123; }."foo ${bar}"
```
Both will evaluate to `123`.
Attribute names support [string interpolation]:
```nix
let bar = "foo"; in
{ foo = 123; }.${bar}
```
```nix
let bar = "foo"; in
{ ${bar} = 123; }.foo
```
Both will evaluate to `123`.
In the special case where an attribute name inside of a set declaration
evaluates to `null` (which is normally an error, as `null` cannot be coerced to
a string), that attribute is simply not added to the set:
```nix
{ ${if foo then "bar" else null} = true; }
```
This will evaluate to `{}` if `foo` evaluates to `false`.
A set that has a `__functor` attribute whose value is callable (i.e. is
itself a function or a set with a `__functor` attribute whose value is
callable) can be applied as if it were a function, with the set itself
passed in first , e.g.,
```nix
let add = { __functor = self: x: x + self.x; };
inc = add // { x = 1; };
in inc 1
```
evaluates to `2`. This can be used to attach metadata to a function
without the caller needing to treat it specially, or to implement a form
of object-oriented programming, for example.
## Recursive sets
Recursive sets are like normal [attribute sets](./values.md#attribute-set), but the attributes can refer to each other.
Recursive sets are like normal [attribute sets](./types.md#attribute-set), but the attributes can refer to each other.
When defining an [attribute set](./values.md#attribute-set) or in a [let-expression](#let-expressions) it is often convenient to copy variables from the surrounding lexical scope (e.g., when you want to propagate attributes).
When defining an [attribute set](./types.md#attribute-set) or in a [let-expression](#let-expressions) it is often convenient to copy variables from the surrounding lexical scope (e.g., when you want to propagate attributes).
This can be shortened using the `inherit` keyword.
Example:
@@ -470,17 +703,3 @@ Does evaluate to `"inner"`.
> ```console
> 1
> ```
## Scoping rules
Nix is [statically scoped](https://en.wikipedia.org/wiki/Scope_(computer_science)#Lexical_scope), but with multiple scopes and shadowing rules.
* primary scope --- explicitly-bound variables
* [`let`](#let-expressions)
* [`inherit`](#inheriting-attributes)
* function arguments
* secondary scope --- implicitly-bound variables
* [`with`](#with-expressions)
Primary scope takes precedence over secondary scope.
See [`with`](#with-expressions) for a detailed example.
Every value in the Nix language has one of the following types:
* [Integer](#type-int)
* [Float](#type-float)
* [Boolean](#type-bool)
* [String](#type-string)
* [Path](#type-path)
* [Null](#type-null)
* [Attribute set](#type-attrs)
* [List](#type-list)
* [Function](#type-function)
* [External](#type-external)
## Primitives
### Integer {#type-int}
An _integer_ in the Nix language is a signed 64-bit integer.
Non-negative integers can be expressed as [integer literals](syntax.md#number-literal).
Negative integers are created with the [arithmetic negation operator](./operators.md#arithmetic).
The function [`builtins.isInt`](builtins.md#builtins-isInt) can be used to determine if a value is an integer.
### Float {#type-float}
A _float_ in the Nix language is a 64-bit [IEEE 754](https://en.wikipedia.org/wiki/IEEE_754) floating-point number.
Most non-negative floats can be expressed as [float literals](syntax.md#number-literal).
Negative floats are created with the [arithmetic negation operator](./operators.md#arithmetic).
The function [`builtins.isFloat`](builtins.md#builtins-isFloat) can be used to determine if a value is a float.
### Boolean {#type-bool}
A _boolean_ in the Nix language is one of _true_ or _false_.
<!-- TODO: mention the top-level environment -->
These values are available as attributes of [`builtins`](builtins.md#builtins-builtins) as [`builtins.true`](builtins.md#builtins-true) and [`builtins.false`](builtins.md#builtins-false).
The function [`builtins.isBool`](builtins.md#builtins-isBool) can be used to determine if a value is a boolean.
### String {#type-string}
A _string_ in the Nix language is an immutable, finite-length sequence of bytes, along with a [string context](string-context.md).
Nix does not assume or support working natively with character encodings.
String values without string context can be expressed as [string literals](string-literals.md).
The function [`builtins.isString`](builtins.md#builtins-isString) can be used to determine if a value is a string.
### Path {#type-path}
A _path_ in the Nix language is an immutable, finite-length sequence of bytes starting with `/`, representing a POSIX-style, canonical file system path.
Path values are distinct from string values, even if they contain the same sequence of bytes.
Operations that produce paths will simplify the result as the standard C function [`realpath`] would, except that there is no symbolic link resolution.
A file is not required to exist at a given path in order for that path value to be valid, but a path that is converted to a string with [string interpolation] or [string-and-path concatenation] must resolve to a readable file or directory which will be copied into the Nix store.
For instance, evaluating `"${./foo.txt}"` will cause `foo.txt` from the same directory to be copied into the Nix store and result in the string `"/nix/store/<hash>-foo.txt"`.
Operations such as [`import`] can also expect a path to resolve to a readable file or directory.
> The Nix language assumes that all input files will remain _unchanged_ while evaluating a Nix expression.
> For example, assume you used a file path in an interpolated string during a `nix repl` session.
> Later in the same session, after having changed the file contents, evaluating the interpolated string with the file path again might not return a new [store path], since Nix might not re-read the file contents.
> Use `:r` to reset the repl as needed.
[store path]: @docroot@/store/store-path.md
Path values can be expressed as [path literals](syntax.md#path-literal).
The function [`builtins.isPath`](builtins.md#builtins-isPath) can be used to determine if a value is a path.
### Null {#type-null}
There is a single value of type _null_ in the Nix language.
<!-- TODO: mention the top-level environment -->
This value is available as an attribute on the [`builtins`](builtins.md#builtins-builtins) attribute set as [`builtins.null`](builtins.md#builtins-null).
## Compound values
### Attribute set {#type-attrs}
<!-- TODO(@rhendric, #10970): fill this out -->
An attribute set can be constructed with an [attribute set literal](syntax.md#attrs-literal).
The function [`builtins.isAttrs`](builtins.md#builtins-isAttrs) can be used to determine if a value is an attribute set.
### List {#type-list}
<!-- TODO(@rhendric, #10970): fill this out -->
A list can be constructed with a [list literal](syntax.md#list-literal).
The function [`builtins.isList`](builtins.md#builtins-isList) can be used to determine if a value is a list.
## Function {#type-function}
<!-- TODO(@rhendric, #10970): fill this out -->
A function can be constructed with a [function expression](syntax.md#functions).
The function [`builtins.isFunction`](builtins.md#builtins-isFunction) can be used to determine if a value is a function.
## External {#type-external}
An _external_ value is an opaque value created by a Nix [plugin](../command-ref/conf-file.md#conf-plugin-files).
Such a value can be substituted in Nix expressions but only created and used by plugin code.
Numbers, which can be *integers* (like `123`) or *floating point*
(like `123.43` or `.27e13`).
See [arithmetic] and [comparison] operators for semantics.
[arithmetic]: ./operators.md#arithmetic
[comparison]: ./operators.md#comparison
- <a id="type-path" href="#type-path">Path</a>
*Paths* are distinct from strings and can be expressed by path literals such as `./builder.sh`.
Paths are suitable for referring to local files, and are often preferable over strings.
- Path values do not contain trailing slashes, `.` and `..`, as they are resolved when evaluating a path literal.
- Path literals are automatically resolved relative to their [base directory](@docroot@/glossary.md#gloss-base-directory).
- The files referred to by path values are automatically copied into the Nix store when used in a string interpolation or concatenation.
- Tooling can recognize path literals and provide additional features, such as autocompletion, refactoring automation and jump-to-file.
A path literal must contain at least one slash to be recognised as such.
For instance, `builder.sh` is not a path:
it's parsed as an expression that selects the attribute `sh` from the variable `builder`.
Path literals may also refer to absolute paths by starting with a slash.
> **Note**
>
> Absolute paths make expressions less portable.
> In the case where a function translates a path literal into an absolute path string for a configuration file, it is recommended to write a string literal instead.
> This avoids some confusion about whether files at that location will be used during evaluation.
> It also avoids unintentional situations where some function might try to copy everything at the location into the store.
If the first component of a path is a `~`, it is interpreted such that the rest of the path were relative to the user's home directory.
For example, `~/foo` would be equivalent to `/home/edolstra/foo` for a user whose home directory is `/home/edolstra`.
Path literals that start with `~` are not allowed in [pure](@docroot@/command-ref/conf-file.md#conf-pure-eval) evaluation.
Paths can be used in [string interpolation] and string concatenation.
For instance, evaluating `"${./foo.txt}"` will cause `foo.txt` from the same directory to be copied into the Nix store and result in the string `"/nix/store/<hash>-foo.txt"`.
Note that the Nix language assumes that all input files will remain _unchanged_ while evaluating a Nix expression.
For example, assume you used a file path in an interpolated string during a `nix repl` session.
Later in the same session, after having changed the file contents, evaluating the interpolated string with the file path again might not return a new [store path], since Nix might not re-read the file contents. Use `:r` to reset the repl as needed.
[store path]: @docroot@/store/store-path.md
Path literals can also include [string interpolation], besides being [interpolated into other expressions].
[interpolated into other expressions]: ./string-interpolation.md#interpolated-expressions
At least one slash (`/`) must appear *before* any interpolated expression for the result to be recognized as a path.
`a.${foo}/b.${bar}` is a syntactically valid number division operation.
`./a.${foo}/b.${bar}` is a path.
[Lookup path](./constructs/lookup-path.md) literals such as `<nixpkgs>` also resolve to path values.
Lists are formed by enclosing a whitespace-separated list of values
between square brackets. For example,
```nix
[ 123 ./foo.nix "abc" (f { x = y; }) ]
```
defines a list of four elements, the last being the result of a call to
the function `f`. Note that function calls have to be enclosed in
parentheses. If they had been omitted, e.g.,
```nix
[ 123 ./foo.nix "abc" f { x = y; } ]
```
the result would be a list of five elements, the fourth one being a
function and the fifth being a set.
Note that lists are only lazy in values, and they are strict in length.
Elements in a list can be accessed using [`builtins.elemAt`](./builtins.md#builtins-elemAt).
## Attribute Set
An attribute set is a collection of name-value-pairs (called *attributes*) enclosed in curly brackets (`{ }`).
An attribute name can be an identifier or a [string](#string).
An identifier must start with a letter (`a-z`, `A-Z`) or underscore (`_`), and can otherwise contain letters (`a-z`, `A-Z`), numbers (`0-9`), underscores (`_`), apostrophes (`'`), or dashes (`-`).
> **Syntax**
>
> *name* = *identifier* | *string* \
> *identifier* ~ `[a-zA-Z_][a-zA-Z0-9_'-]*`
Names and values are separated by an equal sign (`=`).
Each value is an arbitrary expression terminated by a semicolon (`;`).
> **Syntax**
>
> *attrset* = `{` [ *name*`=`*expr*`;` ]... `}`
Attributes can appear in any order.
An attribute name may only occur once.
Example:
```nix
{
x = 123;
text = "Hello";
y = f { bla = 456; };
}
```
This defines a set with attributes named `x`, `text`, `y`.
Attributes can be accessed with the [`.` operator](./operators.md#attribute-selection).
Example:
```nix
{ a = "Foo"; b = "Bar"; }.a
```
This evaluates to `"Foo"`.
It is possible to provide a default value in an attribute selection using the `or` keyword.
Example:
```nix
{ a = "Foo"; b = "Bar"; }.c or "Xyzzy"
```
```nix
{ a = "Foo"; b = "Bar"; }.c.d.e.f.g or "Xyzzy"
```
will both evaluate to `"Xyzzy"` because there is no `c` attribute in the set.
You can use arbitrary double-quoted strings as attribute names:
```nix
{ "$!@#?" = 123; }."$!@#?"
```
```nix
let bar = "bar"; in
{ "foo ${bar}" = 123; }."foo ${bar}"
```
Both will evaluate to `123`.
Attribute names support [string interpolation]:
```nix
let bar = "foo"; in
{ foo = 123; }.${bar}
```
```nix
let bar = "foo"; in
{ ${bar} = 123; }.foo
```
Both will evaluate to `123`.
In the special case where an attribute name inside of a set declaration
evaluates to `null` (which is normally an error, as `null` cannot be coerced to
a string), that attribute is simply not added to the set:
```nix
{ ${if foo then "bar" else null} = true; }
```
This will evaluate to `{}` if `foo` evaluates to `false`.
A set that has a `__functor` attribute whose value is callable (i.e. is
itself a function or a set with a `__functor` attribute whose value is
callable) can be applied as if it were a function, with the set itself
passed in first , e.g.,
```nix
let add = { __functor = self: x: x + self.x; };
inc = add // { x = 1; };
in inc 1
```
evaluates to `2`. This can be used to attach metadata to a function
without the caller needing to treat it specially, or to implement a form
- `nix-shell` shebang lines now support single-quoted arguments.
- `builtins.fetchTree` is now its own experimental feature, [`fetch-tree`](@docroot@/contributing/experimental-features.md#xp-fetch-tree).
This allows stabilising it independently of the rest of what is encompassed by [`flakes`](@docroot@/contributing/experimental-features.md#xp-fetch-tree).
- `builtins.fetchTree` is now its own experimental feature, [`fetch-tree`](@docroot@/development/experimental-features.md#xp-fetch-tree).
This allows stabilising it independently of the rest of what is encompassed by [`flakes`](@docroot@/development/experimental-features.md#xp-fetch-tree).
- The interface for creating and updating lock files has been overhauled:
@@ -33,7 +33,7 @@
- The flake-specific flags `--recreate-lock-file` and `--update-input` have been removed from all commands operating on installables.
They are superceded by `nix flake update`.
- Commit signature verification for the [`builtins.fetchGit`](@docroot@/language/builtins.md#builtins-fetchGit) is added as the new [`verified-fetches` experimental feature](@docroot@/contributing/experimental-features.md#xp-feature-verified-fetches).
- Commit signature verification for the [`builtins.fetchGit`](@docroot@/language/builtins.md#builtins-fetchGit) is added as the new [`verified-fetches` experimental feature](@docroot@/development/experimental-features.md#xp-feature-verified-fetches).
(experimental) now returns a JSON map rather than JSON list.
Loaded 100 of 1024 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.