Compare commits

...
628 Commits
Author SHA1 Message Date
Théophane Hufschmitt a7b8624910 Rename nix show-config to nix config show
Part of #7672
2023-11-27 20:02:14 +01:00
Robert Hensing f0180487a0 Merge pull request #9462 from trofi/eval.o-dependency
libexpr: add missing dependency on 'flake/call-flake.nix.gen.hh'
2023-11-27 17:25:21 +01:00
Robert Hensing 928f0c1341 Merge pull request #9393 from hercules-ci/changelog-d
Automatically compile hand-written release notes with `changelog-d`
2023-11-27 16:29:16 +01:00
Robert Hensing 68c48756fe libexpr/local.mk: Make eval compile deps regular
Dependency is now entirely through the eval.cc rule.
All gen.hh deps are now there.
2023-11-27 15:52:24 +01:00
Robert HensingandValentin Gagarin f7bfec2806 maintainers/release-notes: Improve DATE check
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-11-27 15:18:29 +01:00
tomberek 0c6fe1be7d Merge pull request #9459 from fricklerhandwerk/page-redirects
add path based redirects
2023-11-27 08:28:27 -05:00
Valentin Gagarin 384ffb4443 add deprecation warnings in documentation
this is hacky, but can serve as a stopgap until we can do it
programmatically.
2023-11-27 14:07:32 +01:00
Théophane Hufschmitt d46230eff5 Merge pull request #7762 from yorickvP/boost-regex
Switch from std::regex to boost::regex
2023-11-27 10:59:22 +01:00
Théophane Hufschmitt e986d20bed Remove an obsolete comment 2023-11-27 10:25:18 +01:00
Sergei Trofimovich 75134b7513 libexpr: add missing dependency on 'flake/call-flake.nix.gen.hh'
Without the change build for `eval.o` fails occasionally as:

    $ make src/libexpr/eval.o
      GEN    Makefile.config
      GEN    src/libexpr/primops/derivation.nix.gen.hh
      GEN    src/libexpr/fetchurl.nix.gen.hh
      GEN    src/libexpr/parser-tab.cc
      GEN    src/libexpr/lexer-tab.cc
    src/libexpr/lexer.l:314: warning, -s option given but default rule can be matched
      CXX    src/libexpr/eval.o
    src/libexpr/eval.cc:519:18: fatal error: flake/call-flake.nix.gen.hh: No such file or directory
      519 |         #include "flake/call-flake.nix.gen.hh"
          |                  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    compilation terminated.
    make: *** [mk/patterns.mk:3: src/libexpr/eval.o] Error 1

Noticed in https://github.com/NixOS/nixpkgs/pull/269439
2023-11-27 08:56:24 +00:00
Moritz AngermannandCole Helbling f56401a114 nix flake update add deprecation warnings.
This builds on #8817, to add additional UX help for people with existing
muscle memory (or shell history) with --update-input and tries to gently
guide them towards the newly evolved CLI UI.

Co-authored-by: Cole Helbling <cole.e.helbling@outlook.com>
2023-11-27 09:08:15 +01:00
Valentin Gagarin 2b7016cc56 add path based redirects
up to now, those were managed outside of this repo, which as
unsurprisingly a real hassle to deal with if one wanted to prevent URLs
from breaking when moving pages around. this change removes a large part
of the friction involved in moving content in the Nix manual.

possible next steps for further automation:
- check for content that moved and warn if it's not reachable from
  links that were valid prior to a change
- create redirect rules automatically based on this information
2023-11-27 08:33:03 +01:00
Robert Hensing e7e21aa0c8 flake.nix: Use top level changelog-d
It is about 2 MB now, as only it and libffi (tiny) are new in the
build or shell closures.
2023-11-26 21:21:55 +01:00
Robert Hensing c5d49ec7ab flake.lock: Update
Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/decdf666c833a325cb4417041a90681499e06a41' (2023-11-18)
  → 'github:NixOS/nixpkgs/9ba29e2346bc542e9909d1021e8fd7d4b3f64db0' (2023-11-23)
2023-11-26 21:21:55 +01:00
Robert Hensing d63f72197c Don't run changelog-d in the build
This way we lose the preview of release notes on master, as well
as on https://nixos.org/manual/nix/unstable/release-notes/rl-next
but we can come back to this.
2023-11-26 21:21:55 +01:00
Jan Tojnar f25c06d7a3 docs: Fix broken link
Link target definitions need to be in a separate paragraph to be collected.

Fixup for https://github.com/NixOS/nix/commit/217d863f7a251a4d8a08ff3294944b45146c61c9
2023-11-26 19:32:28 +01:00
John Ericson fdac6e3818 Merge pull request #9451 from kolloch/feature/git-ignore-ds-store
gitignore: Also ignore .DS_Store
2023-11-25 11:47:49 -05:00
Peter Kolloch 213594721a gitignore: Also ignore .DS_Store
This is a file that Finder on Mac OS loves to add into various folders.
2023-11-25 17:30:03 +01:00
Yorick van Pelt d2f5e263e3 Switch from std::regex to boost::regex 2023-11-25 15:14:18 +01:00
John Ericson dda0e34ecf Merge pull request #9443 from ivan770/reproducibility
doc: fix machine-specific capabilities leaking
2023-11-24 22:59:38 -05:00
ivan770 54b6847655 doc: fix machine-specific capabilities leaking 2023-11-24 11:17:35 -05:00
Robert Hensing 857f9168f7 Migrate rl-next.md to doc/manual/rl-next directory 2023-11-24 16:53:33 +01:00
Robert Hensing 6971c4adc0 maintainers/release-notes <- scripts/release-notes 2023-11-24 16:53:33 +01:00
Robert Hensing 7c4ee5c813 scripts/release-notes: Avoid mutating variables 2023-11-24 16:53:33 +01:00
Robert Hensing b1ea30f21d scripts/release-notes: Support patch releases
This also fixes the broken case statement, which has globs, not regexes.
2023-11-24 16:53:33 +01:00
Robert Hensing 2a538c571b Add scripts/release-notes 2023-11-24 16:53:33 +01:00
Théophane Hufschmitt 7f626dba33 Merge pull request #9398 from Qyriad/fixes/flake-not-found
flakes: bare minimum fix the error message for untracked flake.nix
2023-11-24 15:33:08 +01:00
Valentin Gagarin 9aa63f70d7 fricklerhandwerk: subscribe to documentation changes (#9422)
* fricklerhandwerk: subscribe to documentation changes
2023-11-24 14:32:02 +00:00
Robert Hensing b26038c517 doc: Rename 2X.XX to "Upcoming release", and only generate if applicable 2023-11-24 15:13:23 +01:00
Robert Hensing b7982372d2 Compile hand-written release notes with changelog-d 2023-11-24 15:13:21 +01:00
Robert Hensing 8a52325d7d Merge pull request #9444 from phip1611/foldl-doc
doc: primops: fix typo
2023-11-24 13:36:18 +01:00
Robert Hensing 43fb39ca29 Merge pull request #9445 from NixOS/allow-input-in-git-commit
Allow user input in `git commit`
2023-11-24 13:34:59 +01:00
Théophane Hufschmitt 6a94755b12 Allow user input in git commit
We occasionnally commit to git repositories (like with `nix flake update --commit-lock-file`).
This shells out to `git commit`, which might wait for user input (for a signing key passphrase for instance).
Disable the progress bar while this is running to make sure that the
user can enter it.
2023-11-24 11:45:37 +01:00
Philipp Schuster 5be0e6b314 doc: primops: fix typo 2023-11-24 10:50:01 +01:00
Philipp SchusterandValentin Gagarin 2ce8c9650b doc: primops: add more info for foldl (#9254)
* doc: primops: add more info for foldl

From the existing doc it is not obvious whether the first or the
second argument is the accumulator. This is however relevant to
know, as for certain scenarios, this might change the behavior.

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-11-23 21:02:20 +00:00
Théophane Hufschmitt bf13943206 Merge pull request #9173 from vkryachko/transitive-input-overrides
Fix transitive input locking.
2023-11-23 08:54:45 +01:00
Robert Hensing 030fb86dd1 Merge pull request #9394 from NixOS/separate-file-canonicalise
Put `canonicaliseTimestampAndPermissions` in its own header/file
2023-11-22 18:27:52 +01:00
Eelco Dolstra 38844943d0 Merge pull request #9413 from edolstra/fetchToStore-cache
Persistently cache `InputAccessor::fetchToStore()`
2023-11-22 12:19:25 +01:00
Eelco Dolstra 533eafc5f2 Merge pull request #9438 from R-VdP/fix_unused_variable_warning
Fix compile warning due to unused variable binding.
2023-11-22 12:09:10 +01:00
Eelco Dolstra b1ab592f28 Use the StorePath-based cache interface 2023-11-22 11:44:02 +01:00
r-vdp 5292f36426 Fix compile warning due to unused variable binding.
We still need the check, since we don't have narinfo for locally built store paths.
2023-11-22 11:34:53 +01:00
Eelco DolstraandJohn Ericson 61b76f5f34 Apply suggestion
Co-authored-by: John Ericson <git@JohnEricson.me>
2023-11-22 11:26:12 +01:00
John Ericson 7155df091f Merge pull request #9431 from obsidiansystems/fix-rapidcheck-shared
Add missing `-lrapidcheck` fixing build with shared lib
2023-11-21 23:47:40 -05:00
John Ericson 46131567da Add missing -lrapidcheck fixing build with shared lib
https://github.com/NixOS/nixpkgs/pull/269064 makes rapidcheck be build
as a shared lib, but that broke Nix because the `-lrapidcheck` was
missing. This fixes that (and doesn't break Nix what the library is a
static archive as today).
2023-11-21 23:19:25 -05:00
John Ericson a6b315ae80 Merge pull request #7912 from mupdt/gcstore-add-perm-root
Mounted SSH Store
2023-11-21 14:04:45 -05:00
Matej Urbas 03c3af1bf9 mounted-ssh-ng store: integration tests 2023-11-21 13:35:12 -05:00
mupdt b32b20a6d7 release note entry for the mounted-ssh-ng:// store 2023-11-21 13:35:07 -05:00
Matej Urbas 06b8902562 MountedSSHStore: stores on shared filesystems 2023-11-21 13:34:01 -05:00
Matej Urbas 226b0f3956 Extend the worker protocol with wopAddPermRoot 2023-11-21 13:26:55 -05:00
Matej Urbas 9796ebd7ef Add --process-ops flag to nix-daemon 2023-11-21 13:18:19 -05:00
John Ericson 949f5841f8 Add the MountedSSHStore experimental feature
It will be implemented in the subsequent commits of this PR.
2023-11-21 13:16:46 -05:00
John Ericson f880469173 Put canonicaliseTimestampAndPermissions in its own header/file
It is not inherently tied to `LocalStore`, it could probably even go in
`libnixutil`. Functions not attached to `LocalStore` should not be
declared in `local-store.hh`.

I am moving it to facilitate experimenting for #9344. If
canonicalisation should be done client-side in client-side builds, there
wouldn't be a `LocalStore` at all so having to include that header to
get this freestanding function is cumbersome and wrong.

Perhaps canonicalisation should still be done server-side for security
reasons --- I don't mean to make that judgement call now --- but even if
so, this freestanding function still isn't connected to `LocalStore` so
while less urgent it is still better to move out of this header.
2023-11-21 12:57:59 -05:00
Robert Hensing 9cd69e1c39 Merge pull request #9411 from edolstra/path-display
Improve SourceAccessor path display
2023-11-21 18:12:01 +01:00
Lin Jian 4d8decbd13 doc: fix number of template attributes
This number is not updated when welcomeText is added[1][2].

[1]: f3a2940e70
[2]: https://github.com/NixOS/nix/pull/6103
2023-11-21 16:52:25 +01:00
Théophane Hufschmitt 9ad9e988b8 Merge pull request #9425 from iFreilicht/fix-unbound-variable-errors
Fix "unbound variable" errors in bash
2023-11-21 15:12:32 +01:00
Felix Uhl 64827360be Fix "unbound variable" errors in bash
Fixes #9414
2023-11-21 14:49:48 +01:00
Eelco Dolstra c185e45ed7 Merge pull request #9410 from edolstra/shallow
Some shallow git fetching improvements
2023-11-21 12:58:01 +01:00
John Ericson 516e7ddc41 Merge pull request #9257 from Artturin/nixenvjsondrvpath
`nix-env --query`: fix `--json` ignoring `--drv-path`
2023-11-20 16:32:35 -05:00
Eelco Dolstra 99d5204baa Persistently cache InputAccessor::fetchToStore()
This avoids repeated copying of the same source tree between Nix
invocations. It requires the accessor to have a "fingerprint" (e.g. a
Git revision) that uniquely determines its contents.
2023-11-20 20:04:37 +01:00
Eelco Dolstra a0162d5732 Improve SourceAccessor path display
Backported from lazy-trees. This allows SourceAccessors to show the
origin of the accessor. E.g. we now get

  copying '«git+https://github.com/blender/blender.git?ref=refs/heads/main&rev=4edc1389337dd3679ff66969c332d2aff52e1992»/' to the store

instead of

  copying '/' to the store
2023-11-20 18:54:36 +01:00
Eelco Dolstra 071f14a0bb Don't do shallow fetches over ssh 2023-11-20 17:25:40 +01:00
Eelco Dolstra e4066c0444 Fetch specific Git revisions
This is more efficient, and necessary when using shallow=1 with a rev.
2023-11-20 17:25:40 +01:00
Eelco Dolstra f450c8773c Merge pull request #9408 from roblabla/fix-bad-format-string
Fix bad_format_string error when builder stdout contains %
2023-11-20 17:22:40 +01:00
Eelco Dolstra e29e952a25 Merge pull request #9386 from hercules-ci/flake-update
flake.nix: Update nixpkgs: release-23.05 -> nixos-23.05-small
2023-11-20 16:51:20 +01:00
Eelco Dolstra 065cd5bed7 Merge pull request #9401 from NixOS/bump-2.20.0
Bump version
2023-11-20 15:42:53 +01:00
roblabla e2b6821ca0 Fix bad_format_string error when builder stdout contains % 2023-11-20 15:41:38 +01:00
John Ericson e540d48c4f Merge pull request #9400 from hercules-ci/refactors-from-5e3986f59cb
Refactors from 5e3986f59c
2023-11-20 09:38:18 -05:00
Théophane Hufschmitt cfc6494d57 Merge pull request #9390 from DavHau/git-shallow-docs
fetchTree: clarify docs for shallow flag
2023-11-20 15:15:25 +01:00
Eelco Dolstra 2a96445d75 Bump version 2023-11-20 15:06:28 +01:00
John Ericson f9970fdc19 Merge pull request #9240 from edolstra/libgit2
Introduce libgit2
2023-11-20 09:00:27 -05:00
John Ericson 4ab27e5595 Merge remote-tracking branch 'upstream/master' into libgit2 2023-11-20 08:30:26 -05:00
Robert Hensing 6832d18ac7 Merge pull request #9399 from edolstra/revert-vlas
Revert use of boost::container::small_vector in the evaluator
2023-11-20 13:37:50 +01:00
Robert Hensing a5e51a9e02 refactor Worker::childStarted/Terminated: use switch
Preparation for RFC 92 dynamic derivations.
2023-11-20 13:32:48 +01:00
Robert Hensing 7ac39ff05c refactor Store::buildPaths: convert to string earlier
Preparation for RFC 92 dynamic derivations.
2023-11-20 13:11:58 +01:00
Eelco Dolstra 1d6abec993 Revert use of boost::container::small_vector in the evaluator
It caused random crashes (https://hydra.nixos.org/build/241514506,
https://hydra.nixos.org/build/241443330) because the heap allocation
done by small_vector in the not-small case is not scanned for GC
roots.
2023-11-20 12:35:35 +01:00
Qyriad 19993398a1 flakes: check for flake.nix before complaining that lstat on it fails
getFlake currently calls lstat (via isLink via canonPath) before it
performs the sanity check that a flake.nix exists in the first place.
This commit moves the check to before path canonicalization, so that
failed symlink check operations don't throw before the check does.
2023-11-20 03:37:02 -07:00
tomberek fb68699456 Merge pull request #9370 from hercules-ci/add-value-types
refactor: Add `Value` types, use `std::span` for list iteration
2023-11-20 01:32:32 -05:00
Robert Hensing 4292d99756 Merge pull request #9395 from nbraud/buitlins
builtins.concatMap: Fix typo in error message
2023-11-20 00:25:03 +01:00
nicoo d5928085d5 builtins.concatMap: Fix typo in error message 2023-11-19 19:57:07 +01:00
DavHau 796a7eb92d fetchTree: clarify docs for shallow flag 2023-11-19 20:32:23 +07:00
Robert Hensing fe4f573d49 flake.nix: Update nixpkgs: release-23.05 -> nixos-23.05-small
Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/9eb24edd6a0027fed010ccfe300a9734d029983c' (2023-11-01)
  → 'github:NixOS/nixpkgs/decdf666c833a325cb4417041a90681499e06a41' (2023-11-18)
2023-11-19 11:38:58 +01:00
John Ericson c4a74d6dcc Merge pull request #9382 from tweag/filterSource-link
doc: Add link to filterSource from path
2023-11-19 00:06:18 -05:00
John Ericson f739fc0729 Merge pull request #9379 from NixOS/shebang-virtual-dtor
Shebang parser: add virtual destructor
2023-11-19 00:05:34 -05:00
Silvan Mosberger 70ddf298e0 doc: Add link to filterSource from path 2023-11-19 04:09:14 +01:00
Robert Hensing 251fb23aea Shebang parser: add virtual destructor
Fixes:

    warning: destructor called on non-final 'nix::ParseUnquoted' that has virtual functions but non-virtual destructor [-Wdelete-non-abstract-non-virtual-dtor]
2023-11-19 01:48:08 +01:00
John Ericson 87ac33f29a Merge branch 'master' into nixenvjsondrvpath 2023-11-18 13:47:14 -05:00
Robert Hensing 1d86bb4f70 Merge pull request #9373 from obsidiansystems/fix-make-check
Fix `make check`
2023-11-17 19:09:38 +01:00
Robert Hensing 7762335a9c Merge pull request #9374 from hercules-ci/fix-nogc
Fix buildNoGc
2023-11-17 18:16:11 +01:00
Robert Hensing 4a539ac3ea Fix buildNoGc
Fixes https://hydra.nixos.org/build/241067941/nixlog/1

    src/libexpr/eval.cc:1776:54: error: variable 'boost::container::small_vector<nix::Value*, 4> vArgs' has initializer but incomplete type
2023-11-17 17:38:08 +01:00
John Ericson 293ae59257 Fix make check
After 9c7749e135, `libutil-tests_RUN`
doesn't exist. It needs to become `libutil-tests-exe_RUN`.
2023-11-17 11:26:45 -05:00
Eelco Dolstra 6acc9b11d9 Merge pull request #9372 from edolstra/release-notes
2.19 release notes
2023-11-17 14:52:40 +01:00
Théophane Hufschmitt b1e7d7cad6 Merge pull request #9280 from R-VdP/rvdp/fix_remote_logging_phase_reporting
Include phase reporting in log file for ssh-ng builds
2023-11-17 14:37:09 +01:00
Eelco Dolstra f7d59d0dda Release notes 2023-11-17 14:21:17 +01:00
Eelco Dolstra 3a7f024a9c Merge pull request #9363 from edolstra/symlink-regression
Fix symlink handling
2023-11-17 14:11:50 +01:00
Robert Hensing d4370d8850 Merge pull request #9368 from frogamic/let-inherit
doc: Add example of inherit in a let expression
2023-11-17 11:22:27 +01:00
Robert HensingandValentin Gagarin 7b0e8c5c2c Apply suggestions from code review
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-11-17 10:56:23 +01:00
Robert Hensing 121665f377 nix-env: Use state.mkList, required for correct stats 2023-11-17 10:23:32 +01:00
Robert Hensing 260c614762 Value: use std::span, change use of const
**`Value` and `const`**

These two deserve some explanation. We'll get to lists later.

Values can normally be thought of as immutable, except they are
are also the vehicle for call by need, which must be implemented
using mutation.

This circumstance makes a `const Value` a rather useless thing:

 - If it's a thunk, you can't evaluate it, except by copying, but
   that would not be call by need.

 - If it's not a thunk, you know the type, so the method that
   acquired it for you should have returned something more specific,
   such as a `const Bindings &` (which actually does make sense
   because that's an immutable span of pointers to mutable `Value`s.

 - If you don't care about the type yet, you might establish the
   convention that `const Value` means `deepSeq`-ed data, but
   this is hardly useful and not actually as safe as you would
   supposedly want to trust it to be - just convention.

**Lists**

`std::span` is a tuple of pointer and size - just what we need.

We don't return them as `const Value`, because considering the
first bullet point we discussed before, we'd have to force all
the list values, which isn't what we want.

So what we end up with is a nice representation of a list in
weak head normal form: the spine is immutable, but the
items may need some evaluation later.
2023-11-17 10:19:03 +01:00
Robert Hensing 7055c65285 Value: extract Value::Lambda 2023-11-17 10:19:03 +01:00
Robert Hensing 6af1d9f7b9 Value: extract Value::FunctionApplicationThunk 2023-11-17 10:19:03 +01:00
Robert Hensing b55203e874 Value: extract Value::ClosureThunk 2023-11-17 10:19:03 +01:00
Robert Hensing d8ff5cfe8e Value: extract Value::Path 2023-11-17 10:19:03 +01:00
Robert Hensing 2eb59c34b5 Value: extract Value::StringWithContext 2023-11-17 10:19:03 +01:00
Dominic Shelton c819375769 doc: Add example of inherit in a let expression 2023-11-17 17:50:17 +11:00
Théophane Hufschmitt 7ba4e073e8 Merge pull request #6469 from gbpdt/fix/skipped_build_locking
Unlock output paths when a derivation is already built
2023-11-16 21:59:25 +01:00
Théophane Hufschmitt fabae98ab4 Merge pull request #9360 from obsidiansystems/install-unit-tests
Allow installing unit tests
2023-11-16 21:25:21 +01:00
Théophane Hufschmitt ac4431e9d0 Merge pull request #7348 from thufschmitt/dont-use-vlas
Remove the usage of VLAs in the code
2023-11-16 19:05:37 +01:00
Eelco Dolstra 57de482e00 Merge pull request #9364 from edolstra/fix-coverage
Fix a broken generated header file dependency
2023-11-16 17:37:05 +01:00
Eelco Dolstra 96d67620d5 Fix a broken generated header file dependency
https://hydra.nixos.org/build/240882042
2023-11-16 17:12:06 +01:00
Eelco Dolstra 31ebc6028b Fix symlink handling
This restores the symlink handling behaviour prior to
94812cca98.

Fixes #9298.
2023-11-16 16:45:14 +01:00
John Ericson 6c8f4ef350 Allow installing unit tests
Closes #9343

See that issue for motivation.

Installing these is disabled by default, but we enable it (and the
additional output we want isntall these too so as not to clutter the
existing ones) to use in cross builds and dev shells.
2023-11-16 09:55:42 -05:00
Robert Hensing 4e27f1947a libexpr: Reduce nonRecursiveStackReservation
128 is still beyond the point where the allocation overhead is
insignificant, but we don't anticipate to overflow for these
use cases, so it's fine.
2023-11-16 15:38:21 +01:00
Robert Hensing a96be29db5 removeAttrs: increase stack reservation to 64 2023-11-16 15:38:21 +01:00
Robert Hensing 1b9813e4e6 primops: Name stack reservation limits 2023-11-16 15:38:21 +01:00
Robert Hensing 898c47384f primops: Err on the side of less stack usage
Try to stay away from stack overflows.

These small vectors use stack space. Most instances will not need
to allocate because in general most things are small, and large
things are worth heap allocating.

16 * 3 * word = 384 bytes is still quite a bit, but these functions
tend not to be part of deep recursions.
2023-11-16 15:38:21 +01:00
Robert Hensing 91114a6fa4 ExprCall::eval: Heap allocate at arity 5+ 2023-11-16 15:38:21 +01:00
Robert Hensing 206ece0f41 builtins.{any,all}: Use constant errorCtx
Clang warned that the expanded code used to have a buffer overflow.
Very strange, but also very avoidable.
2023-11-16 15:38:21 +01:00
Robert Hensing 9fa133dde5 readProcLink: Replace unnecessary value judgement by actual info 2023-11-16 15:38:21 +01:00
Robert Hensing 12c91a823e maxPrimOpArity: 64 -> 8
This makes stack usage significantly more compact, allowing larger
amounts of data to be processed on the same stack.

PrimOp functions with more than 8 positional (curried) arguments
should use an attrset instead.
2023-11-16 15:38:21 +01:00
Eelco Dolstra 2bd2702127 Merge pull request #9354 from vicky1999/fix/8875
Renamed nix nar dump-path to nix nar pack
2023-11-16 15:06:33 +01:00
Eelco Dolstra 16c052e4e7 Merge pull request #9307 from flox/tomberek.nix_key_newline
nix key: no need for progressBar
2023-11-16 15:05:34 +01:00
Eelco Dolstra c47f769391 Merge pull request #9350 from NixOS/dependabot/github_actions/zeebe-io/backport-action-2.1.1
build(deps): bump zeebe-io/backport-action from 2.1.0 to 2.1.1
2023-11-16 15:01:49 +01:00
Robert Hensing 0daccb1121 libexpr: Check primop arity earlier 2023-11-16 12:28:32 +01:00
Théophane Hufschmitt ba3cb4a049 Remove all the occurences of VLAs
There's generally no strict reason for using them, and they are somewhat
fishy, so let's avoid them.
2023-11-16 12:27:30 +01:00
Théophane Hufschmitt 5196613e82 Use boost small vectors instead of VLAs in the primops
VLAs are a dangerous feature, and their usage triggers an undefined
behavior since theire size can be zero in some cases.
So replace them with `boost::small_vector`s which fit the same goal but
are safer.

It's also incidentally consistently 1% faster on the benchmarks.
2023-11-16 12:27:30 +01:00
John Ericson e34c424279 Merge pull request #9357 from NixOS/nix-store-add
Add a new `nix store add` command
2023-11-15 13:49:44 -05:00
Théophane Hufschmitt 84128461b6 Add a new nix store add command
Deprecate `nix store add-file` and `nix store add-path`, and replace
them with a single `nix store add` command.
2023-11-15 19:21:17 +01:00
Eelco Dolstra 7ab91e7238 Implement shallow fetching 2023-11-15 15:00:17 +01:00
Eelco Dolstra 5dd4ae8687 Remove unused cacheType field 2023-11-15 14:08:34 +01:00
Eelco Dolstra 2890999911 Show Git fetch progress 2023-11-15 13:57:20 +01:00
Silvan Mosberger 70b396649c doc: logical implication is right-associative
nix-repl> bools = [ false true ]

    nix-repl> combinations = builtins.concatMap (a: builtins.concatMap (b: map (c: { inherit a b c; }) bools) bools) bools

    nix-repl> builtins.all ({ a, b, c }: (a -> b -> c) == (a -> (b -> c))) combinations
    true

    nix-repl> builtins.all ({ a, b, c }: (a -> b -> c) == ((a -> b) -> c)) combinations
    false
2023-11-15 07:04:09 +01:00
John Ericson d23d0a074d Merge pull request #9355 from obsidiansystems/fix-tiny-libutil-unit-test-make-bug
Fix makefile bug confusing `libnixutil-test` exe vs lib
2023-11-14 12:13:41 -05:00
John Ericson 9c7749e135 Fix makefile bug confusing libnixutil-test exe vs lib
The `-exe` variant is the program, the unsuffixed variant is the
library.

The corrected usage matches `libnixstore-test`.
2023-11-14 11:45:52 -05:00
Eelco Dolstra 2964a9f562 Fix relative submodule handling
Tested on

  nix flake prefetch 'git+https://github.com/blender/blender.git?rev=4ed8a360e956daf2591add4d3c9ec0719e2628fe&submodules=1'
2023-11-14 16:00:21 +01:00
Eelco Dolstra 6ec6b8aa36 Improve git submodule error reporting 2023-11-14 15:52:18 +01:00
vicky1999 e07e3c106a code cleanup 2023-11-14 20:02:33 +05:30
vicky1999 4944cdb94d nar dump-path command renamed to nar pack 2023-11-14 19:59:48 +05:30
Eelco Dolstra c257c82447 Cleanup 2023-11-14 14:47:17 +01:00
Eelco Dolstra 7f576f5dfe Rename UnionInputAccessor to MountedInputAccessor 2023-11-14 14:02:57 +01:00
Eelco Dolstra 21140c987b Fix doxygen comments 2023-11-14 14:02:57 +01:00
Eelco Dolstra 4329bdf6a3 Move comment 2023-11-14 14:02:57 +01:00
Eelco DolstraandRobert Hensing 25cf8f1071 src/libfetchers/union-input-accessor.cc: Apply suggestion
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-11-14 13:57:24 +01:00
Eelco DolstraandRobert Hensing 38b07d6347 src/libfetchers/git.cc: Apply suggestion
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-11-14 13:38:03 +01:00
Eelco Dolstra d74d2fdaa7 Move statusCallbackTrampoline 2023-11-14 13:35:26 +01:00
Eelco Dolstra 21bb180547 Use libgit2 with ssh-exec support
See https://github.com/libgit2/libgit2/pull/6617. This ensures that we
get support for ~/.ssh/config, known_hosts etc.
2023-11-14 13:30:51 +01:00
Théophane Hufschmitt c889881cbb Merge pull request #9351 from Trundle/update-chroot-store-comment
Update comment to reflect bind mounts are now used for store in chroot
2023-11-14 09:09:27 +01:00
Andreas Stührk ad99c8950b Update comment to reflect bind mounts are now used for store in chroot 2023-11-13 23:29:48 +01:00
dependabot[bot] 742a63b98f build(deps): bump zeebe-io/backport-action from 2.1.0 to 2.1.1
Bumps [zeebe-io/backport-action](https://github.com/zeebe-io/backport-action) from 2.1.0 to 2.1.1.
- [Release notes](https://github.com/zeebe-io/backport-action/releases)
- [Commits](https://github.com/zeebe-io/backport-action/compare/v2.1.0...v2.1.1)

---
updated-dependencies:
- dependency-name: zeebe-io/backport-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-11-13 22:01:50 +00:00
Tom Bereknyei 0be84c83b2 key and cat: no need for progressBar
otherwise the output will be invisible in common terminal configurations
2023-11-12 15:40:44 -05:00
Théophane Hufschmitt 2afe2e4fbc Merge pull request #9333 from DeterminateSystems/option-for-nix-upgrade-url
nix upgrade-nix: make the source URL an option
2023-11-12 19:45:30 +01:00
Graham Christensen fd5a4a8467 nix upgrade-nix: make the source URL an option
This new option enables organizations to more easily manage their Nix
fleet's deployment, and ensure a consistent and planned rollout of Nix
upgrades.
2023-11-10 12:12:28 -05:00
John Ericson 458e511dc4 Merge pull request #9294 from obsidiansystems/minimal-git
Git object hashing in libutil
2023-11-10 11:24:23 -05:00
20b95d6223 Git object hashing in libutil
This is the core functionality but just unit-tested and not yet made
part of the store layer. This is because there is some tech debt around
(a) repeated boilerplate hashing objects (b) better integration of the
new `SourceAccessor` type that needs to be cleaned up first.

Part of RFC 133

Co-Authored-By: Matthew Bauer <mjbauer95@gmail.com>
Co-Authored-By: Carlo Nucera <carlo.nucera@protonmail.com>
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Florian Klink <flokli@flokli.de>
2023-11-10 11:02:44 -05:00
John EricsonandRobert Hensing 9afa697ab6 Refactor bash test build system a bit
The basic idea here is to separate a few intertwined notions:

1. Not all "run bash tests" are "install tests"

2. Not all "run bash tests" use `tests/functional/init.sh`, or any
   pre-test initialization at all.

This will used in the next commit when we have a test that check unit
test golden master data.

Also, move our custom `PS4` from the test to the test runner, as it is
part of how we want to display the tests, not the test themselves.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-11-10 11:02:37 -05:00
John Ericson 3d9d5dc189 Create MemorySink
This is for writing to a `MemorySourceAccessor`.
2023-11-10 11:02:37 -05:00
Eelco Dolstra cf59ea83ec configure: Check for libgit2 2023-11-10 13:58:59 +01:00
Eelco Dolstra d1a1888a3e Merge pull request #9328 from obsidiansystems/editorconfig-perl
`.editorconfig`: Also affect Perl FFI `xs` file
2023-11-10 13:45:07 +01:00
John Ericson 1d5a48240c .editorconfig: Also affect Perl FFI xs file
This way `perl/lib/Nix/Store.xs` is affected.
2023-11-09 23:11:52 -05:00
Artturin a903f85f84 nix-env --query: fix --json ignoring --drv-path
```json
{
  "AMB-plugins": {
    "drvPath": "/nix/store/l99cb7h2hy8dg005arsjbd9kx0w05d3h-AMB-plugins-0.8.1.drv",
    "name": "AMB-plugins-0.8.1",
    "outputName": "out",
    "outputs": {
      "out": null
    },
    "pname": "AMB-plugins",
    "system": "x86_64-linux",
    "version": "0.8.1"
  },
  "ArchiSteamFarm": {
    "drvPath": "/nix/store/nhplgyjj34fz6hjmnyih25gxscfh8s7b-ArchiSteamFarm-5.4.12.5.drv",
    "name": "ArchiSteamFarm-5.4.12.5",
    "outputName": "out",
    "outputs": {
      "out": null
    },
    "pname": "ArchiSteamFarm",
    "system": "x86_64-linux",
    "version": "5.4.12.5"
  },
...
```
2023-11-09 22:31:32 +02:00
Eelco Dolstra 98a120b8b8 Merge remote-tracking branch 'origin/master' into libgit2 2023-11-09 16:48:41 +01:00
Robert Hensing df9bd755a1 Merge pull request #9326 from NixOS/unstable-fetchTree-git
Mark `fetchTree` as unstable again
2023-11-09 07:44:48 +01:00
Robert Hensing 12953b942c Fixup docs 2023-11-09 07:08:56 +01:00
John Ericson f0adb72c23 Mark fetchTree as unstable again
As discussed in our last meeting, we need a bit more time, but we are
"time boxing" the work left to do to ensure there is not unbounded
delay.

Rather than putting it back underneath `flakes`, though, put it
underneath its own `fetch-tree` experimental feature (which `flakes`
includes/implies). This signals our commitment to the plan to stabilize
it first without waiting to go through the rest of Flakes, and also will
give users a "release candidate" when we get closer to stabilization.

This reverts commit 4112dd1fc9.
2023-11-08 23:23:56 -05:00
John Ericson d854e8696b Specify the size of the experimental feature array in a more robust way
See doc comment for details.
2023-11-08 22:58:10 -05:00
John Ericson c14ba93290 Merge pull request #9317 from tfc/libstore-improvementswq
Libstore improvements
2023-11-08 15:36:38 -05:00
Jacek Galowicz df8bfe84cc Fix consts and casts 2023-11-08 17:29:55 +01:00
Jacek Galowicz c581143e0c Use structured binding for json iteration 2023-11-08 17:29:55 +01:00
Jacek Galowicz 77dceb2844 Drop obsolete assert and cast 2023-11-08 17:29:55 +01:00
Jacek Galowicz 6a47629530 Fix initialization of struct members (wrong order) 2023-11-08 17:29:55 +01:00
John Ericson d8e3b9aafc Merge pull request #9313 from NixOS/maintainer-mission
maintainers: refine the mission statement phrasing
2023-11-08 10:12:09 -05:00
Felix Uhl c0c7c4b6cd Link to shebang interpreter docs from release notes 2023-11-07 22:11:48 +01:00
Robert Hensing ab69dc4da3 Test parseShebangContent round trip 2023-11-07 13:24:01 -05:00
Robert Hensing 589d338776 parseShebangs: Make strings with backtick sequences representable 2023-11-07 13:24:01 -05:00
Robert Hensing ffd414eb75 Fix nix shebang interaction with #8131 overhaul completions 2023-11-07 13:24:01 -05:00
Robert Hensing e91fd837ee Move shebang docs from rl-next to nix.md 2023-11-07 13:24:01 -05:00
Robert Hensing 51bb69535b nix/installables.cc: Use getCommandBaseDir() where possible
These usages of the working directory are perhaps unlikely to
interact with shebangs, but the code is more consistent this way,
and we're less likely to miss usages that do interact.
2023-11-07 13:24:01 -05:00
Robert Hensing 466271568b nix: Parse --file relative to shebang script 2023-11-07 13:24:01 -05:00
Robert Hensing 198bc22e3b nix: Add command baseDir to parse --expr relative to shebang script 2023-11-07 13:24:01 -05:00
Robert Hensing 20ff61ab25 nix: Reserve shebang line syntax and only parse double backtick quotes
Being restrictive about syntax leaves opportunity to improve the
syntax and functionality later.
2023-11-07 13:24:01 -05:00
Robert Hensing cc68ed8ff7 libcmd: lookupFileArg(): add baseDir
This will allow a different base directory to be used, matching
a shebang script location instead of the working directory.
2023-11-07 13:24:01 -05:00
Tom Bereknyei bbeddf0602 fix: refactor parseCmdline interface 2023-11-07 13:24:01 -05:00
Tom Bereknyei e6ed729243 doc: remove reference to nix-shell 2023-11-07 13:24:01 -05:00
Tom Bereknyei 06f3583b1c feat: break out of shebang processing for non-comments 2023-11-07 13:24:01 -05:00
Tom Bereknyei 01f61cefcb Read file incrementally 2023-11-07 13:24:01 -05:00
Tom Bereknyei bfcbf3b5bf doc: shebang release notes, docs, tests
fix: release notes
2023-11-07 13:24:01 -05:00
Tom Bereknyei eea5a003d9 fix: test to ensure arguments are passed 2023-11-07 13:24:01 -05:00
Eelco Dolstra 5f9b5758b6 src/libutil/util.hh: Formatting 2023-11-07 13:24:01 -05:00
Tom Bereknyei 74210c12fe Shellbang support with flakes
Enables shebang usage of nix shell. All arguments with `#! nix` get
added to the nix invocation. This implementation does NOT set any
additional arguments other than placing the script path itself as the
first argument such that the interpreter can utilize it.

Example below:

```
    #!/usr/bin/env nix
    #! nix shell --quiet
    #! nix nixpkgs#bash
    #! nix nixpkgs#shellcheck
    #! nix nixpkgs#hello
    #! nix --ignore-environment --command bash
    # shellcheck shell=bash
    set -eu
    shellcheck "$0" || exit 1
    function main {
        hello
        echo 0:"$0" 1:"$1" 2:"$2"
    }
    "$@"
```

fix: include programName usage

EDIT: For posterity I've changed shellwords to shellwords2 in order
      not to interfere with other changes during a rebase.
      shellwords2 is removed in a later commit. -- roberth
2023-11-07 13:24:01 -05:00
Eelco Dolstra ba4e07782c Merge pull request #9311 from NixOS/dependabot/github_actions/zeebe-io/backport-action-2.1.0
build(deps): bump zeebe-io/backport-action from 2.0.0 to 2.1.0
2023-11-07 14:45:11 +01:00
Felix Uhl 1362a0a55a Fix logic for default XDG_DATA_DIRS value
The [POSIX test manpage](https://pubs.opengroup.org/onlinepubs/9699919799/utilities/test.html)
as well as the [fish test manpage](https://fishshell.com/docs/current/cmds/test.html#operators-for-text-strings)
specify that `-z` will be "True if the length of string string is zero;
otherwise, false."

The `-n` was likely a mixup and not caught during testing of
https://github.com/NixOS/nix/pull/8985 due to a lack of missing
conflicting entries in `XDG_DATA_DIRS`.
2023-11-07 03:41:19 +01:00
Valentin Gagarin b733f4ab29 maintainers: refine the mission statement phrasing
setting a direction falls short of what we're already doing: guide contributors.

the direction aspect is still important, as that is the authoritative part. guidance is the supportive part.
2023-11-07 01:12:39 +01:00
Valentin Gagarin fe62cb79a6 Merge pull request #9309 from iFreilicht/patch-2
Add release note on XDG_DATA_DIRS change
2023-11-06 23:55:12 +01:00
dependabot[bot] 9fec62a100 build(deps): bump zeebe-io/backport-action from 2.0.0 to 2.1.0
Bumps [zeebe-io/backport-action](https://github.com/zeebe-io/backport-action) from 2.0.0 to 2.1.0.
- [Release notes](https://github.com/zeebe-io/backport-action/releases)
- [Commits](https://github.com/zeebe-io/backport-action/compare/v2.0.0...v2.1.0)

---
updated-dependencies:
- dependency-name: zeebe-io/backport-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-11-06 22:21:07 +00:00
Felix Uhl c60eba3276 Add release note on XDG_DATA_DIRS change
Follow-up to https://github.com/NixOS/nix/pull/8985
2023-11-06 23:00:00 +01:00
Ana HobdenandValentin Gagarin 150b5aba50 Update scripts/nix-profile-daemon.fish.in
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-11-06 20:43:27 +01:00
Ana Hobden 896013ec0c Fix bad copy-paste 2023-11-06 20:43:27 +01:00
Ana Hobden 867f894289 Populate $XDG_DATA_DIRS with appropriate folder from Nix profile
On non-NixOS systems, the default `nix` install does not populate the
`$XDG_DATA_DIRS`. This populates it and enables things like bash-completion
and `.desktop` file detection for `nix` profile installed packages.

Signed-off-by: Ana Hobden <operator@hoverbear.org>
2023-11-06 20:43:27 +01:00
Mel Zuser 61d6fe059e Fix boost::bad_format_string exception in builtins.addErrorContext (#9291)
* Fix boost::bad_format_string exception in builtins.addErrorContext

The message passed to addTrace was incorrectly being used as a format
string and this this would cause an exception when the string contained
a '%', which can be hit in places where arbitrary file paths are
interpolated.

* add test
2023-11-06 19:13:40 +00:00
Robert Hensing 1fa87c6aa8 Merge pull request #9283 from obsidiansystems/mememory-source-accessor
Factor out `MemorySourceAccessor`, implement missing features
2023-11-06 19:51:02 +01:00
John Ericson dcc49744ed Merge pull request #9293 from ThinkChaos/ssh-misc-improvments
SSH small improvments
2023-11-06 13:33:51 -05:00
John Ericson dea63bb810 Merge pull request #9299 from tfc/config-improvements
Improvements in src/libutil/config.*
2023-11-06 13:03:54 -05:00
Théophane Hufschmitt 06d0d51895 Merge pull request #9242 from obsidiansystems/path-info-map
Make `nix path-info --json` return an object not array
2023-11-06 18:00:42 +01:00
John Ericson cc46ea1630 Make nix path-info --json return an object not array
Before it returned a list of JSON objects with store object information,
including the path in each object. Now, it maps the paths to JSON
objects with the metadata sans path.

This matches how `nix derivation show` works.

Quite hillariously, none of our existing functional tests caught this
change to `path-info --json` though they did use it. So just new
functional tests need to be added.
2023-11-06 11:06:31 -05:00
John Ericson a7212e169b Include compression in the NarInfo JSON format
It was forgotten before.
2023-11-06 11:06:31 -05:00
John Ericson 937e02e7b9 Shuffle ValidPathInfo JSON rendering
`Store::pathInfoToJSON` was a rather baroque functions, being full of
parameters to support both parsed derivations and `nix path-info`. The
common core of each, a simple `dValidPathInfo::toJSON` function, is
factored out, but the rest of the logic is just duplicated and then
specialized to its use-case (at which point it is no longer that
duplicated).

This keeps the human oriented CLI logic (which is currently unstable)
and the core domain logic (export reference graphs with structured
attrs, which is stable), separate, which I think is better.
2023-11-06 11:06:31 -05:00
Jacek Galowicz a4b7df7bfa More const, scope reductions, move fixes 2023-11-06 16:00:25 +01:00
Jacek Galowicz f404e9b3b3 Make toJSONObject const 2023-11-06 16:00:25 +01:00
Jacek Galowicz ad385f9ec4 Minor improvements 2023-11-06 16:00:25 +01:00
Jacek Galowicz 07ac53732b Fix moves in appendOrSet 2023-11-06 16:00:25 +01:00
John Ericson 0b0d1b5214 Add comparison functions for NarInfo
We will need these for tests.
2023-11-06 09:51:21 -05:00
ThinkChaos 2fb49759b8 fix(ssh): log first line of stdout
Spent a while debugging why `nix-copy-closure` wasn't working anymore
and it was my shell RC printing something I added for debug.
Hopefully this can save someone else some time.
2023-11-06 08:46:19 -05:00
ThinkChaos 6472c3bf0d fix(ssh): extraneous master processes 2023-11-06 08:45:14 -05:00
Eelco Dolstra 28dddde0ac Merge pull request #9225 from drupol/nixpkgs-pr-107251/make-gnutar-reproducible
fix: make sure `tar` reproducibility flags are set
2023-11-06 10:51:42 +01:00
Robert Hensing 1a14ce8381 Merge pull request #8920 from obsidiansystems/split-util-cchh
Split up `util.{hh,cc}`
2023-11-05 21:53:11 +01:00
John Ericson ac89bb064a Split up util.{hh,cc}
All OS and IO operations should be moved out, leaving only some misc
portable pure functions.

This is useful to avoid copious CPP when doing things like Windows and
Emscripten ports.

Newly exposed functions to break cycles:

 - `restoreSignals`
 - `updateWindowSize`
2023-11-05 12:20:02 -05:00
John Ericson 2678b51b31 Narrower scope for nativeSystem
I don't think we need a CPP defininition and a header entry, and this
way allows constant expression elimination.
2023-11-05 12:18:28 -05:00
John EricsonandEelco Dolstra 9b880e3e29 Factor out MemorySourceAccessor, implement missing features
The new `MemorySourceAccessor` rather than being a slightly lossy flat
map is a complete in-memory model of file system objects.

Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2023-11-04 15:24:20 -04:00
Valentin Gagarin 8e222fbb12 Merge pull request #8848 from BootRhetoric/flake-authentication 2023-11-04 00:23:28 +01:00
BootRhetoric 271932782d fetchGit and flake: add commit signature verification tests
This adds simple tests of the commit signature verification mechanism of
fetchGit and its flake input wrapper.
OpenSSH is added to the build dependencies since it's needed to create
a key when testing the functionality. It is neither a built- nor a
runtime dependency.
2023-11-03 20:23:31 +01:00
BootRhetoricandValentin Gagarin 098f0615c9 fetchGit and flake: add publicKeys list input
This adds publicKeys as an optional fetcher input attribute to flakes
and builtins.fetchGit to provide a nix interface for the json-encoded
`publicKeys` attribute of the git fetcher.

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-11-03 20:23:31 +01:00
BootRhetoric 6df32889a5 Add git commit verification input attributes
This implements the git input attributes `verifyCommit`, `keytype`,
`publicKey` and `publicKeys` as experimental feature
`verified-fetches`. `publicKeys` should be a json string.
This representation was chosen because all attributes must be of type bool,
int or string so they can be included in flake uris (see definition of
fetchers::Attr).
2023-11-03 20:15:12 +01:00
Robert Hensing 66cb364f58 Merge pull request #9289 from edolstra/fix-warnings
Fix gcc warnings
2023-11-03 12:41:52 +01:00
r-vdp 60b363936d libstore/ssh-ng: Fix phase reporting in log files.
When doing local builds, we get phase reporting lines in the log file,
they look like '@nix {"action":"setPhase","phase":"unpackPhase"}'.
With the ssh-ng protocol, we do have access to these messages, but since we
are only including messages of type resBuildLogLine in the logs, the phase
information does not end up in the log file.

The phase reporting could probably be improved altoghether (it looks like it
is kind of accidental that these JSON messages for phase reporting show up
but others don't, just because they are actually emitted by nixpkgs' stdenv),
but as a first step I propose to make ssh-ng behave in the same way as local builds do.
2023-11-03 12:30:25 +01:00
Eelco Dolstra b0455e9931 Fix uninitialized variable warnings on i686-linux
https://hydra.nixos.org/build/239849607
2023-11-03 12:04:20 +01:00
Eelco Dolstra 55dd1244d2 parseDerivation(): Fix warning about uninitialized 'version' variable 2023-11-03 12:04:20 +01:00
Eelco Dolstra e9a857e418 Merge pull request #9288 from edolstra/fix-nar-access-test
Fix nar-access test on macOS
2023-11-03 11:39:48 +01:00
Eelco Dolstra e5908212e2 Fix nar-access test on macOS 2023-11-03 11:03:58 +01:00
John Ericson 727ada1a41 Merge pull request #9268 from obsidiansystems/systematize-characterization-tests
Systematize characterization tests a bit more
2023-11-02 12:38:28 -04:00
John Ericson d15c3a33e6 Don't use std::invocable C++ concept yet
It s not supported on all platforms yet. Can revert this once it is.
2023-11-02 12:06:29 -04:00
John EricsonandJacek Galowicz b107431816 Systematize characterization tests a bit more
Deduplicating code moreover enforcing the pattern means:

 - It is easier to write new characterization tests because less boilerplate

 - It is harder to mess up new tests because there are fewer places to
   make mistakes.

Co-authored-by: Jacek Galowicz <jacek@galowicz.de>
2023-11-02 12:06:08 -04:00
Eelco Dolstra 39ea46abb1 Merge remote-tracking branch 'origin/master' into libgit2 2023-11-02 16:16:37 +01:00
John Ericson 1e61c007be Merge pull request #9234 from fricklerhandwerk/doc-style-examples
styleguide: add note on highlighting examples and syntax definitions
2023-11-02 09:49:38 -04:00
John Ericson 7f56ed8b90 Merge pull request #9207 from fricklerhandwerk/doc-store-path
document store paths
2023-11-02 09:48:18 -04:00
John Ericson 8c99d40cac Merge pull request #9232 from fricklerhandwerk/doc-store-object
document store objects in terms of their constituent parts
2023-11-02 09:45:48 -04:00
John Ericson 90de958637 Merge pull request #9265 from obsidiansystems/better-parse-sink
Make `ParseSink` a bit better
2023-11-02 09:28:29 -04:00
Eelco Dolstra 5223114c93 Merge pull request #9269 from edolstra/unify-accessor
Unify `FSAccessor` and `SourceAccessor`
2023-11-02 14:23:10 +01:00
Eelco DolstraandJohn Ericson d26c317b14 Use expect
Co-authored-by: John Ericson <git@JohnEricson.me>
2023-11-02 13:40:54 +01:00
Robert Hensing a80fb9fa39 Merge pull request #9277 from keszybz/file-permissions
Remove stray executable permissions on source files
2023-11-02 13:19:38 +01:00
Zbigniew Jędrzejewski-Szmek 55ed09c4f2 Remove stray executable permissions on source files
Noticed because of a warning during an rpm build:
*** WARNING: ./usr/src/debug/nix-2.18.1-1.fc40.x86_64/src/nix-copy-closure/nix-copy-closure.cc is executable but has no shebang, removing executable bit
*** WARNING: ./usr/src/debug/nix-2.18.1-1.fc40.x86_64/src/nix-channel/nix-channel.cc is executable but has no shebang, removing executable bit
2023-11-02 09:24:21 +01:00
Valentin GagarinandJohn Ericson d7b7a79f3e document store paths
update the glossary to point to the new page.

since this is a cross-cutting concern, it warrants its own section in
the manual.

Co-authored-by: John Ericson <git@JohnEricson.me>
2023-11-02 06:01:51 +01:00
Valentin Gagarin 4ba8b182be document store objects in terms of their constituent parts
this also rephrases the introductory sentence to be more general, in order to
avoid the same word being repeated in short succession.
2023-11-02 06:01:30 +01:00
John Ericson 12f9719162 Merge pull request #9250 from fricklerhandwerk/fix-anchors
fix anchor in conf-file
2023-11-01 21:30:22 -04:00
John Ericson d7710a40be flake: Temporarily get Nixpkgs ahead of Hydra
Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/31ed632c692e6a36cfc18083b88ece892f863ed4' (2023-09-21)
  → 'github:NixOS/nixpkgs/9eb24edd6a0027fed010ccfe300a9734d029983c' (2023-11-01)
2023-11-01 20:09:55 -04:00
Eelco DolstraandJohn Ericson e47984ce0b Fix whitespace
Co-authored-by: John Ericson <git@JohnEricson.me>
2023-11-01 20:19:08 +01:00
John Ericson 8d66346293 Merge pull request #9271 from keszybz/if-fix
fix: gcc complains about if which doesn't guard the indented statement
2023-11-01 14:02:33 -04:00
Zbigniew Jędrzejewski-Szmek eab9292738 fix: gcc complains about if which doesn't guard the indented statement 2023-11-01 18:10:06 +01:00
Eelco Dolstra 2f5c1a27dc LocalStoreAccessor: Reuse PosixSourceAccessor 2023-11-01 17:22:25 +01:00
Eelco Dolstra 1a902f5fa7 Merge FSAccessor into SourceAccessor 2023-11-01 17:09:28 +01:00
Eelco Dolstra 581693bdea fmt(): Handle std::string_view 2023-11-01 16:33:22 +01:00
Eelco Dolstra 50aae0a14c FSAccessor: Make the fileSize and narOffset fields optional
The narOffset field only applies to NAR accessors. The fileSize field
may be too expensive to compute for certain accessors (e.g. libgit).
2023-11-01 15:39:40 +01:00
Eelco Dolstra 5381123879 Unify DirEntries types 2023-11-01 15:33:35 +01:00
Eelco Dolstra cdb27c1519 SourceAccessor: Change the main interface from lstat() to maybeLstat() 2023-11-01 15:26:07 +01:00
Eelco Dolstra 8ffd1695ce Unify FSAccessor::Type and SourceAccessor::Type 2023-11-01 14:43:20 +01:00
John Ericson 6132ffd825 Merge pull request #9266 from hercules-ci/doc-hacking-clangd-fix-tests
doc/hacking: Fix clangd for tests
2023-11-01 09:38:43 -04:00
John Ericson 0707db2b1c Merge pull request #9255 from NixOS/libfetcher-docs-json
libfetcher doc automation
2023-11-01 09:38:02 -04:00
Eelco Dolstra b2ac6fc040 Remove FSAccessor::Type::tMissing
Instead stat() now returns std::nullopt to denote that the file
doesn't exist.
2023-11-01 14:36:40 +01:00
Robert Hensing bc4a1695ac doc/hacking: Fix clangd for tests 2023-11-01 11:44:16 +01:00
John Ericson 1093d6585f Make ParseSink a bit better
I wouldn't call it *good* yet, but this will do for now.

- `RetrieveRegularNARSink` renamed to `RegularFileSink` and moved
  accordingly because it actually has nothing to do with NARs in
  particular.

  - its `fd` field is also marked private

- `copyRecursive` introduced to dump a `SourceAccessor` into a
  `ParseSink`.

- `NullParseSink` made so `ParseSink` no longer has sketchy default
  methods.

This was done while updating #8918 to work with the new
`SourceAccessor`.
2023-11-01 02:36:43 -04:00
John Ericson e3febfcd53 Merge pull request #6258 from obsidiansystems/gcc-bug-ergonomics
Remove bug-avoiding `StoreConfig *` casts for settings
2023-10-31 12:48:27 -04:00
John Ericson b2cae33aef Remove bug-avoiding StoreConfig * casts for settings
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=80431 has been fixed, and
per the previous commit we now check that is the case at build time.
2023-10-31 12:09:46 -04:00
John Ericson 1f4525531e Add configure test to ensure GCC bug is fixed
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=80431

(test is adapted from issue, test does not test for GCC-specific
behavior but rather absence of bug, so test is good with other compilers
too.)
2023-10-31 12:09:46 -04:00
Théophane Hufschmitt 12a0ae73db Merge pull request #8817 from iFreilicht/flake-update-lock-overhaul
Overhaul `nix flake update` and `nix flake lock` UX
2023-10-31 16:19:05 +01:00
Eelco Dolstra f8da50644c Merge pull request #9256 from NixOS/dependabot/github_actions/zeebe-io/backport-action-2.0.0
build(deps): bump zeebe-io/backport-action from 1.4.0 to 2.0.0
2023-10-31 16:07:25 +01:00
Eelco Dolstra 0c5eac9c45 Git fetcher: Handle submodules for workdirs 2023-10-31 15:59:25 +01:00
Felix Uhl f282ef5a56 fix: segfault in positional arg completion
Adding the inputPath as a positional feature uncovered this bug.
As positional argument forms were discarded from the `expectedArgs`
list, their closures were not. When the `.completer` closure was then
called, part of the surrounding object did not exist anymore.

This didn't cause an issue before, but with the new call to
`getEvalState()` in the "inputs" completer in nix/flake.cc, a segfault
was triggered reproducibly on invalid memory access to the `this`
pointer, which was always 0.

The solution of splicing the argument forms into a new list to extend
their lifetime is a bit of a hack, but I was unable to get the "nicer"
iterator-based solution to work.
2023-10-31 15:33:57 +01:00
Felix Uhl c7dcdb8325 Overhaul nix flake update and lock commands
Closes #5110
2023-10-31 15:33:57 +01:00
Felix Uhl c762b65dc5 Fix documentation of flake command output 2023-10-31 15:33:57 +01:00
Felix Uhl 1fd0867389 Fix missing output when creating lockfile 2023-10-31 15:33:50 +01:00
Eelco Dolstra 669b074f51 Cleanup 2023-10-31 14:52:21 +01:00
Eelco Dolstra d88106df24 Git fetcher: Improve submodule handling
Instead of making a complete copy of the repo, fetching the
submodules, and writing the result to the store (which is all
superexpensive), we now fetch the submodules recursively using the Git
fetcher, and return a union accessor that "mounts" the accessors for
the submodules on top of the root accessor.
2023-10-31 14:52:21 +01:00
Eelco Dolstra ee36a44bf2 GitInputScheme: Use libgit2
This replaces most calls to the "git" binary with libgit2.
2023-10-31 14:52:21 +01:00
Eelco Dolstra 1d0e3d84b6 Provide a InputScheme::fetch() built on top of InputScheme::getAccessor()
This is for graceful migration to lazy-trees fetchers (which are all
accessor-based). Eventually fetch() will be removed.
2023-10-31 14:52:17 +01:00
Eelco Dolstra e1b8442fa1 Fetcher cache: Add support for caching facts not related to store paths 2023-10-31 14:51:39 +01:00
Eelco Dolstra fa6bc33604 Merge pull request #9239 from edolstra/putFile
Input: Replace markFileChanged() by putFile()
2023-10-31 14:49:42 +01:00
dependabot[bot] 95f3f9eac9 build(deps): bump zeebe-io/backport-action from 1.4.0 to 2.0.0
Bumps [zeebe-io/backport-action](https://github.com/zeebe-io/backport-action) from 1.4.0 to 2.0.0.
- [Release notes](https://github.com/zeebe-io/backport-action/releases)
- [Commits](https://github.com/zeebe-io/backport-action/compare/v1.4.0...v2.0.0)

---
updated-dependencies:
- dependency-name: zeebe-io/backport-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-10-30 22:21:34 +00:00
Eelco Dolstra 05316d401f Cleanup 2023-10-30 17:03:06 +01:00
John Ericson 077de2968e Include fetcher input scheme info in the CLI dump
Leverages the previous commit.
2023-10-30 10:30:59 -04:00
John Ericson 8381eeda6f Systematize fetcher input attribute validation
We now have `schemeName` and `allowedAttrs` functions for this purpose.
We look up the schema with the former; we restrict the set of input
attributes with the latter.
2023-10-30 10:17:24 -04:00
Valentin Gagarin 325db01d26 fix anchor in conf-file
I inadvertently switched it to `opt-` when refactoring, but it should
have been `conf` to begin with.
2023-10-27 07:30:24 +02:00
John Ericson a6e587923c Merge pull request #9247 from obsidiansystems/derivation-test-with-files
Turn derivation unit tests into unit characterization tests
2023-10-26 18:39:00 -04:00
John Ericson a419b61497 Turn derivation unit tests into unit characterization tests
The brings a number of advantages, including:

- Easier to update test data if design changes (and I do think our
  derivation JSON is not yet complaint with the guidelines).

- Easier to reuse test data in other implementations, inching closer to
  compliance tests for Nix *the concept* rather than any one
  implementation.
2023-10-26 18:09:01 -04:00
John Ericson 232e236672 Merge pull request #9246 from trofi/clarify-builder-uid-exhaustion
local-derivation-goal.cc: slightly clarify waiting message
2023-10-26 16:31:49 -04:00
Sergei Trofimovich e69c764708 local-derivation-goal.cc: slightly clarify waiting message
Before the change builder ID exhaustion printed the following message:

    [0/1 built] waiting for UID to build '/nix/store/hiy9136x0iyib4ssh3w3r5m8pxjnad50-python3.11-breathe-4.35.0.drv'

After the change it should be:

    [0/1 built] waiting for a free build user ID for '/nix/store/hiy9136x0iyib4ssh3w3r5m8pxjnad50-python3.11-breathe-4.35.0.drv'
2023-10-26 20:54:21 +01:00
John Ericson 1dc6a65d36 Merge pull request #9238 from tfc/small-improvements2
Small improvements 2
2023-10-26 09:35:03 -04:00
Jacek Galowicz 28c39c370c Provide default value for id to fix warning 2023-10-26 09:47:40 +02:00
Jacek Galowicz b66381e8d8 Use using instead of typedef 2023-10-26 09:47:40 +02:00
Silvan Mosberger 46028ff764 doc: Fix fetchGit default name (#9241) 2023-10-26 07:05:48 +02:00
John Ericson bfd51a4137 Merge pull request #9235 from fricklerhandwerk/doc-style-guide
add notes on comments in code samples
2023-10-25 15:32:07 -04:00
Eelco Dolstra 622191c2b5 Merge pull request #8965 from Artturin/bindfilesinchroot
Bindmount files instead of hardlinking or copying to chroot
2023-10-25 19:10:03 +02:00
Eelco Dolstra 15c430f389 Remove unused LockFile::write() 2023-10-25 18:44:09 +02:00
Eelco Dolstra 95d657c8b3 Input: Replace markFileChanged() by putFile()
Committing a lock file using markFileChanged() required the input to
be writable by the caller in the local filesystem (using the path
returned by getSourcePath()). putFile() abstracts over this.
2023-10-25 18:30:29 +02:00
John Ericson 5ac87a75dd Merge pull request #9229 from tfc/small-improvements
Remove warnings, small improvements
2023-10-25 10:50:46 -04:00
Théophane Hufschmitt d571f17ff1 Merge pull request #9237 from iFreilicht/clarify-flake-implicit-types
docs: clarify flake types and implied defaults
2023-10-25 16:13:47 +02:00
Jacek Galowicz f555c98a34 Improve loop over gid container 2023-10-25 16:10:35 +02:00
Felix Uhl 7bc45c6136 docs: clarify flake types and implied defaults 2023-10-25 15:10:35 +02:00
Valentin Gagarin 00c90eae95 add note on highlighting examples and syntax definitions 2023-10-25 12:04:03 +02:00
Valentin Gagarin 78278f2b3f add notes on comments in code samples 2023-10-25 12:00:56 +02:00
Eelco Dolstra 3ca0dade5b Merge pull request #9212 from trofkm/args-refactoring
Clean up `args.hh`
2023-10-25 10:33:53 +02:00
Valentin GagarinandJohn Ericson 8d9e0b7aed document the store concept (#9206)
* document the store concept and its purpose

reword the glossary to link to more existing information instead of
repeating it.

move the store documentation to the top of the table of contents, in
front of the Nix language. this will provide a natural place to
document other aspects of the store as well as the various store types.

move the package management section after the Nix language and before
Advanced Topics to follow the pattern to layer more complex concepts on
top of each other.

this structure of the manual will also nudge beginners to learn Nix
bottom-up and hopefully make more likely that they understand underlying
concepts first before delving into complex use cases that may or may not
be easy to implement with what's currently there.

[John adds this note] The sort of beginner who likes to dive straight into reference documentation should prefer this approach. Conversely, the sort of beginner who would prefer the opposite top-down approach of trying to solve problems before they understand everything that is going on is better off reading other tutorial/guide material anyways, and will just "random-access" the reference manual as a last resort. For such random-access the order doesn't matter, so this restructure doesn't make them any worse off.

Co-authored-by: John Ericson <git@JohnEricson.me>
2023-10-25 02:28:35 +00:00
Pol Dellaiera 7f71fc7540 fix: make sure tar reproducibility flags are set 2023-10-24 22:14:35 +02:00
Jacek Galowicz b113d925de Fix warning 2023-10-24 19:57:38 +02:00
Jacek Galowicz eaced12c94 Fix signed vs. unsigned comparison warning and improve code 2023-10-24 19:57:38 +02:00
Silvan Mosberger f269911641 Document builtins.substring negative length behavior (#9226) 2023-10-24 11:22:02 +02:00
Vignesh abb1c829c8 Release notes updated for #9150 reverted (#9227) 2023-10-24 11:18:00 +02:00
Robert Hensing c7c5d370e0 Merge pull request #9218 from NixOS/system-features-darwin-typo
system-features: Typo
2023-10-23 20:22:59 +02:00
Robert Hensing cde3c63617 system-features: Typo
There I was, thinking all of Apple's OSes started with lower case.
2023-10-23 19:30:00 +02:00
John Ericson a58d7f143e Merge pull request #9216 from obsidiansystems/addDrvOutputDependencies-pre
Add `builtins.addDrvOutputDependencies`
2023-10-23 13:26:51 -04:00
Valentin GagarinandJohn Ericson cd680bd53d Merge how-to section on S3 buckets into S3 store docs (#7972)
Rather than having a misc tutorial page in the grab-bag "package management" section, this information should just be part of the S3 store docs.

---------

Co-authored-by: John Ericson <John.Ericson@Obsidian.Systems>
2023-10-23 13:22:33 -04:00
Кирилл Трофимов c9528d2081 fix: Remove extra to from README.md (#9213) 2023-10-23 19:20:23 +02:00
John Ericson 765436e300 Add builtins.addDrvOutputDependencies
End goal: make `(mkDerivation x).drvPath` behave like a non-DrvDeep
context.

Problem: users won't be able to recover the DrvDeep behavior when
nixpkgs makes this change.

Solution: add this primop.

The new primop is fairly simple, and is supposed to complement other
existing ones (`builtins.storePath`, `builtins.outputOf`) so there are
simple ways to construct strings with every type of string context
element.

(It allows nothing we couldn't already do with `builtins.getContext` and `builtins.appendContext`, which is also true of those other two primops.)

This was originally in #8595, but then it was proposed to land some doc
changes separately. So now the code changes proper is just moved to
this, and the doc will be done in that.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.nore
github.com>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io
2023-10-23 12:49:14 -04:00
Robert Hensing 3b99c62913 Merge pull request #8470 from ncfavier/shebang-single-quotes
nix-shell: support single quotes in shebangs, fix whitespace parsing
2023-10-23 18:23:48 +02:00
Naïm Favier e053eeb272 tests: test nix-shell shebang quoting 2023-10-23 17:32:50 +02:00
Kirill Trofimov 90e3ed06f8 fix: Use default destructor. 2023-10-23 18:07:57 +03:00
Kirill Trofimov a31fc5cc86 fix: Use using instead of typedef for type aliasing.
Since C++ 11 we shouldn't use c-style `typedefs`. In addition, `using` can be templated.
2023-10-23 18:07:17 +03:00
Kirill Trofimov b205da16ef fix: Explicitly pass lambda scope variables.
Default capture implicitly also capture *this, which would automatically be used if for example you referenced a method from the enclosing scope.
2023-10-23 18:06:15 +03:00
Kirill Trofimov c82066cf73 fix: Declare constructor as default 2023-10-23 16:59:19 +03:00
Naïm Favier 595010b631 nix-shell: fix shebang whitespace parsing
Leading whitespace after `nix-shell` used to produce an empty argument,
while an empty argument at the end of the line was ignored.

Fix the first issue by consuming the initial whitespace before calling
shellwords; fix the second issue by returning immediately if whitespace
is found at the end of the string instead of checking for an empty
string.

Also throw if quotes aren't terminated.
2023-10-23 15:56:07 +02:00
Naïm Favier fa9642ec45 nix-shell: support single quotes in shebangs
Single quotes are a basic feature of shell syntax that people expect to
work. They are also more convenient for writing literal code expressions
with less escaping.
2023-10-23 15:56:05 +02:00
John Ericson 8b68bbb777 Merge pull request #6223 from obsidiansystems/worker-proto-with-version
Give `nix daemon` and `nix-store --serve` protocols separate serializers with version info
2023-10-23 09:16:23 -04:00
John EricsonandValentin Gagarin b461cac21a Overhaul completions, redo #6693 (#8131)
As I complained in
https://github.com/NixOS/nix/pull/6784#issuecomment-1421777030 (a
comment on the wrong PR, sorry again!), #6693 introduced a second
completions mechanism to fix a bug. Having two completion mechanisms
isn't so nice.

As @thufschmitt also pointed out, it was a bummer to go from `FlakeRef`
to `std::string` when collecting flake refs. Now it is `FlakeRefs`
again.

The underlying issue that sought to work around was that completion of
arguments not at the end can still benefit from the information from
latter arguments.

To fix this better, we rip out that change and simply defer all
completion processing until after all the (regular, already-complete)
arguments have been passed.

In addition, I noticed the original completion logic used some global
variables. I do not like global variables, because even if they save
lines of code, they also obfuscate the architecture of the code.

I got rid of them  moved them to a new `RootArgs` class, which now has
`parseCmdline` instead of `Args`. The idea is that we have many argument
parsers from subcommands and what-not, but only one root args that owns
the other per actual parsing invocation. The state that was global is
now part of the root args instead.

This did, admittedly, add a bunch of new code. And I do feel bad about
that. So I went and added a lot of API docs to try to at least make the
current state of things clear to the next person.

--

This is needed for RFC 134 (tracking issue #7868). It was very hard to
modularize `Installable` parsing when there were two completion
arguments. I wouldn't go as far as to say it is *easy* now, but at least
it is less hard (and the completions test finally passed).

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-10-23 15:03:11 +02:00
Eelco Dolstra 955bbe53c5 Merge pull request #9177 from edolstra/input-accessors
Backport FSInputAccessor and MemoryInputAccessor from lazy-trees
2023-10-23 11:42:04 +02:00
Eelco Dolstra 34a42f0d0a Move PosixSourceAccessor into its own file 2023-10-23 11:05:50 +02:00
Valentin Gagarin 256dfb98e8 remove Basic Package Management section (#7974)
this is the first thing most beginners see, and it misleads them into
assuming `nix-env` is appropriate for doing anything but setting and
reverting profile generations.

this chapter is the root of most evil around the ecosystem, and today we
finally close it for good.
2023-10-23 04:05:02 +02:00
Кирилл Трофимов 201a4af9a4 Clean up app.cc (#9201)
- Rename `expected` to `expectedType`

- Use early `return` and `continue` to reduce nesting
2023-10-22 22:56:46 +00:00
John Ericson edc07588ec Merge pull request #9191 from tfc/libutil-implementation
libutil: Small improvements
2023-10-22 10:02:03 -04:00
John Ericson 70f8b96c11 Factor out UnkeyedValidPathInfo and test
This makes the path info serialisers ideomatic again, which allows me to
test them.
2023-10-20 15:21:04 -04:00
John Ericson 596bd469cc Move ValidPathInfo serialization code to worker-protocol.{cc.hh}
It does not belong with the data type itself.

This also materializes the fact that `copyPath` does not do any version
negotiation just just hard-codes "16".

The non-standard interface of these serializers makes it harder to test,
but this is fixed in the next commit which then adds those tests.
2023-10-20 15:20:48 -04:00
John Ericson ab822af0df Factor out serialization for BuildResult
Worker Protocol:

Note that the worker protocol already had a serialization for
`BuildResult`; this was added in
a4604f1928. It didn't have any versioning
support because at that time reusable seralizers were not away for the protocol
version. It could thus only be used for new messages also introduced in
that commit.

Now that we do support versioning in reusable serializers, we can expand
it to support all known versions and use it in many more places.

The exist test data becomes the version 1.29 tests: note that those
files' contents are unchanged. 1.28 and 1.27 tests are added to cover
the older code-paths.

The keyered build result test only has 1.29 because the keying was also
added in a4604f19284254ac98f19a13ff7c2216de7fe176; the older
serializations are always used unkeyed.

Serve Protocol:

Conversely, no attempt was made to factor out such a serializer for the
serve protocol, so our work there in this commit for that protocol
proceeds from scratch.
2023-10-20 15:19:28 -04:00
John Ericson 4372738efe Systematize the worker protocol derived path serialiser
It was some ad-hoc functions to account for versions, while the already
factored-out serializer just supported the latest version.

Now, we can fold that version-specific logic into the factored out one,
and so we do.
2023-10-20 15:19:28 -04:00
John Ericson 96c58550b8 Test more derived paths 2023-10-20 15:19:28 -04:00
Valentin GagarinandJohn Ericson 97a0c08873 Expand derivation examples (#9048)
Also use fancier formatting so the example blocks are easier to discern
from the description.

Co-authored-by: John Ericson <git@JohnEricson.me>
2023-10-20 15:17:28 -04:00
Eelco Dolstra 7d3cd54282 Merge pull request #9199 from edolstra/remove-tree
Remove fetchers::Tree and move tarball-related stuff into its own header
2023-10-20 20:26:29 +02:00
Eelco Dolstra 935c9981de Remove fetchers::Tree and move tarball-related stuff into its own header 2023-10-20 19:56:52 +02:00
Eelco Dolstra 091e5b4513 Merge pull request #9198 from edolstra/remove-direct
Input: Remove 'direct' field
2023-10-20 19:49:53 +02:00
John Ericson 81ed1d56ce Merge pull request #9197 from obsidiansystems/delete-dead-code
Delete dead code
2023-10-20 13:32:06 -04:00
Arthur GautierandSandro 85e5ac403f docker: publish images to ghcr.io (#8066)
* docker: publish images to ghcr.io

docker.com announced their intention to remove the free plan used by
OSS. The nixos/nix image is essential to various CI runs to build with
nix. To provide a continuity plan, this commit pushes the image to
ghcr.io as well.

Co-authored-by: Sandro <sandro.jaeckel@gmail.com>
2023-10-20 19:28:26 +02:00
Eelco Dolstra 0f7e9d0513 Input: Remove 'direct' field 2023-10-20 19:14:04 +02:00
Eelco Dolstra e824ab30cf Merge pull request #9195 from edolstra/remove-hasAllInfo
Input::hasAllInfo(): Remove
2023-10-20 18:57:56 +02:00
John Ericson 862d16436b Remove the ValidPathInfo == operator
It is dead code. It was added in
8e0946e8df as part of the repeated /
enforce-determinism feature, but that was removed in
8fdd156a65.

It is not good because it skips many fields. For testing purposes we
will soon want to add a new one that doesn't skip fields, but we want to
make sure making == sensitive to those fields won't change how Nix
works. Proving in this commit that the old version is dead code achieves
that.
2023-10-20 12:39:17 -04:00
John Ericson 3e6b9f9357 Remove prevInfos as its dead code
It is unused since 8e0946e8df removed
support for the repeat and enforce-determinism options.
2023-10-20 12:39:17 -04:00
Eelco Dolstra af302267e5 Input::hasAllInfo(): Remove 2023-10-20 17:19:36 +02:00
Eelco Dolstra bcf5c31950 Add future FIXME 2023-10-20 16:58:33 +02:00
Eelco Dolstra 57db3be9e4 SourceAccessor::readFile(): Support reading into a sink 2023-10-20 16:36:41 +02:00
Robert Hensing 4d17c59d8d Merge pull request #9157 from obsidiansystems/protocol-versions
Add protocol versions to `{Worker,Serve}Proto::*Conn`
2023-10-20 15:34:26 +02:00
John Ericson 8a28ed2e8b Merge pull request #9187 from hercules-ci/issue-7619-apple-virt-system-feature
libstore: Add `apple-virt` to system features when available
2023-10-20 09:09:20 -04:00
Eelco Dolstra 7a086a32bc fetchToStore(): Handle flat ingestion method and add test 2023-10-20 13:32:15 +02:00
Eelco Dolstra 173abec0bc coerceToPath(): Handle __toString, add tests 2023-10-20 13:06:44 +02:00
Eelco Dolstra bacceaea91 Move getLastModified(), remove setPathDisplay() 2023-10-20 12:40:46 +02:00
Eelco DolstraandJohn Ericson df10dc630f Doxygen
Co-authored-by: John Ericson <git@JohnEricson.me>
2023-10-20 12:36:18 +02:00
Robert Hensing bb645c5d02 system-features doc: kvm is Linux-only 2023-10-20 10:21:39 +02:00
Robert Hensing 9277eb276b libstore: Add apple-virt to system features when available
I'm sure that we'll adjust the implementation over time, but this
at least discerns between an apple silicon bare metal machine and
a tart VM.
2023-10-20 10:21:39 +02:00
Johannes KirschbauerandValentin Gagarin e58566a057 doc: add reference to hasAttr in ? operator (#9185)
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-10-20 03:11:03 +00:00
42f26eb42e doc: complexity for '?' operator (#9184)
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-10-20 02:45:47 +02:00
John Ericson 7ed294f07d Merge pull request #9182 from fricklerhandwerk/popular-issues
link to popular issues from the contributing guide
2023-10-19 16:25:36 -04:00
John Ericson 6016a0ed09 Merge pull request #9181 from fricklerhandwerk/good-first-issue-label
update link to label
2023-10-19 16:24:57 -04:00
John Ericson ce42655ef6 Merge pull request #9180 from fricklerhandwerk/meeting-notes-link
add a link to all maintainer meeting notes
2023-10-19 16:24:37 -04:00
Jacek Galowicz b0f4ac29d3 libutil: Use c++ style cast 2023-10-19 18:26:49 +01:00
Jacek Galowicz 55f06b6f30 libutil: Remove non-needed constructor 2023-10-19 18:26:49 +01:00
Jacek Galowicz 87c4f4a972 libutil: Move some non-template implememntations from config.hh to
config.cc
2023-10-19 18:26:49 +01:00
Eelco Dolstra 5be7705ddf Remove stuff we don't need yet 2023-10-19 19:20:21 +02:00
Eelco Dolstra 50156302c0 Deduplicate FSSourceAccessor and FSInputAccessor 2023-10-19 15:20:10 +02:00
Eelco Dolstra 9f572eb0e3 Unify the two implementations of dumpPath() 2023-10-19 15:07:56 +02:00
Eelco Dolstra fb6a3910c4 Move most of InputAccessor into libutil 2023-10-19 14:45:26 +02:00
Johannes KirschbauerandValentin Gagarin 9bc7b4f463 doc: generic closure supported key types (#9183)
* doc: generic closure supported key types

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-10-19 12:39:41 +00:00
Eelco Dolstra 06c57899e3 Remove FIXME 2023-10-19 14:22:05 +02:00
Eelco Dolstra f16af08e83 Fix macOS compilation 2023-10-19 14:20:50 +02:00
Eelco Dolstra 12214fef09 InputAccessor::fetchToStore(): Support arbitrary ingestion methods 2023-10-19 14:19:10 +02:00
Valentin Gagarin 36b15d905e link to popular issues from the contributing guide
this also adds a hint to contributors about making far-reaching changes,
complementing the recent update to the maintainers' handbook on how to
deal with those.
2023-10-19 11:47:10 +02:00
Valentin Gagarin 9adac237e7 update link to label
GitHub now displays a banner and has a dedicated page[1] for good first
issues, but that uses a different label name as we had in place.
I renamed the label on GitHub, this is updating the link.

[1]: https://github.com/NixOS/nix/contribute
2023-10-19 11:40:30 +02:00
Valentin Gagarin 8b48fb146b Merge pull request #7708 from ShamrockLee/primop-rebasehash
primops: add builtins.convertHash
2023-10-19 11:38:47 +02:00
Valentin Gagarin 34c5593525 add a link to all maintainer meeting notes
linking to the discourse category will by default show a view sorted by
most recent post, which makes it hard to find particular meeting notes.

this also adds a procedural detail about the notes, to make that more
explicit and less dependent on being present in the meetings.
2023-10-19 10:57:45 +02:00
Eelco Dolstra d2c0051784 Remove obsolete corepkgs references 2023-10-18 23:35:07 +02:00
Yueh-Shun LiandValentin Gagarin 5088e6563a primops: add builtins.convertHash
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-10-19 00:58:56 +08:00
Yueh-Shun LiandValentin Gagarin 6b47635180 Add helper function parseHashFormat[Opt] printHashFormat
Add hash format analogy of
parseHashTypeOpt, parseHashType, and printHashType.

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-10-19 00:56:44 +08:00
Yueh-Shun Li 231b0fca6d Migrate HashFormat to scoped enumeration (enum struct) 2023-10-19 00:56:44 +08:00
Yueh-Shun Li e026f3e1ae treewide: Reference HashFormat members with scope
Base* -> HashFormat::Base*
2023-10-19 00:56:41 +08:00
Yueh-Shun Li 5043e6cf4e Document HashFormat 2023-10-19 00:38:32 +08:00
Yueh-Shun Li 838c70f621 treewide: Rename hashBase to hashFormat
hashBase is ambiguous, since it's not about the digital bases, but about
the format of hashes. Base16, Base32 and Base64 are all character maps
for binary encoding.

Rename the enum Base to HashFormat.

Rename variables of type HashFormat from [hash]Base to hashFormat,
including CmdHashBase::hashFormat and CmdToBase::hashFormat.
2023-10-19 00:38:32 +08:00
Yueh-Shun Li aff177d860 Elaborate the "unknown hash algorithm" error
List the allowed hash formats
2023-10-19 00:38:32 +08:00
Yueh-Shun Li e9ddf0b400 Simplify parseHashTypeOpt
Remove redundant "else" after "return".

Use std::nullopt to increase readability.
2023-10-19 00:38:32 +08:00
Eelco Dolstra df73c6eb8c Introduce MemoryInputAccessor and use it for corepkgs
MemoryInputAccessor is an in-memory virtual filesystem that returns
files like <nix/fetchurl.nix>. This removes the need for special hacks
to handle those files.
2023-10-18 17:38:11 +02:00
Eelco Dolstra ea38605d11 Introduce FSInputAccessor and use it
Backported from the lazy-trees branch. Note that this doesn't yet use
the access control features of FSInputAccessor.
2023-10-18 17:37:32 +02:00
Vladimir Kryachko 311e2ad024 Address review comments 2023-10-18 10:37:06 -04:00
Vladimir Kryachko a2f0ba6a6d Fix transitive input locking.
Fixes reproducibility issue described in #9143

Fixes #9143
2023-10-18 10:31:36 -04:00
Eelco Dolstra 201c115c3e Merge pull request #9151 from edolstra/stabilize-fetchTree
Stabilize fetchTree
2023-10-18 10:54:08 +02:00
Théophane Hufschmitt c1a1766c46 Merge pull request #9169 from vkryachko/follow_cycle
Detect cycles in flake follows.
2023-10-18 07:34:03 +02:00
John Ericson 9d1f42db52 Merge pull request #9150 from vicky1999/fix/8914
`nix store ping` -> `nix store info`
2023-10-17 22:52:28 -04:00
vicky1999 891dfb4359 updated store ping to store info in files 2023-10-18 00:14:11 +05:30
vicky1999 a0f071f1d3 store info sh renamed 2023-10-18 00:12:10 +05:30
John Ericson e36c9175f4 Add protocol versions to {Worker,Serve}Proto::*Conn
This will allow us to factor out logic, which is currently scattered
inline, into several reusable instances

The tests are also updated to support versioning. Currently all Worker
and Serve protocol tests are using the minimum version, since no
version-specific serialisers have been created yet. But in subsequent
commits when that changes, we will test individual versions to ensure
complete coverage.
2023-10-17 11:21:10 -04:00
John Ericson ff68426095 Name the protocol version types
This makes the code clearer, and will help us replace them with proper
structs and get rid of the macros later.
2023-10-17 11:20:39 -04:00
Eelco Dolstra 3470cd68c4 Mark some fetchers as experimental 2023-10-17 14:57:29 +02:00
Eelco Dolstra f62b5500ff fetchTree: Require the flakes experimental feature for the URL syntax 2023-10-17 14:52:34 +02:00
Eelco Dolstra e92cac789f Merge pull request #9168 from obsidiansystems/fix-lang-tests-read-only
Fix language tests a bit
2023-10-17 14:45:38 +02:00
Eelco Dolstra a9b8595fd9 Merge pull request #9162 from eclairevoyant/add-mainprogram
explicitly set meta.mainProgram
2023-10-17 14:44:29 +02:00
vicky1999 dcc5f801f4 Store info command help updates 2023-10-17 09:39:59 +05:30
Artturin b8dfa3d53b use doBind in addDependency 2023-10-17 01:26:34 +03:00
Artturin 11e47e7dfb factor out doBind from runChild 2023-10-17 01:26:34 +03:00
Artturin 630c2545d1 remove linkOrCopy and use bindmounts for files in addDependency 2023-10-17 01:26:34 +03:00
Artturin 5649229394 Bindmount files instead of hardlinking or copying to chroot
https://github.com/nixos/nix/commit/16591eb3cccf86da8cd3f20c56e2dd847576ff5e#diff-19f999107b609d37cfb22c58e7f0bc1cf76edf1180e238dd6389e03cc279b604 (2013) added support for files to doBind

This is work towards allowing users to change the location of chrootRootDir, to, for example, a tmpfs.

inspired by trofi on matrix

> It looks like build sandbox created by nix-daemon runs on the same filesystem, as /nix/store including things like /tmp which makes all small temporary files hit the disk. Is it intentional? If it is is there an easy way to redirect chroot's root to be tmpfs?

dirsInChroot -> pathsInChroot
2023-10-17 01:26:34 +03:00
John Ericson 8c049a9f04 Merge pull request #9172 from tfc/bad-moves
Fix/remove some bad std::moves
2023-10-16 17:46:41 -04:00
Jacek Galowicz 54b350d517 Drop some moves that would happen anyway but forbid NRVO where appicable 2023-10-16 21:48:35 +01:00
Jacek Galowicz abf7df2b37 Fix moves that accidentally copy anyway 2023-10-16 21:48:35 +01:00
Vladimir Kryachko d6066c90f8 Don't convert InputPaths to strings prematurely. 2023-10-16 15:47:28 -04:00
John Ericson aa4f41d796 Merge pull request #9170 from tfc/fix-broken-move
Fix broken move
2023-10-16 15:13:03 -04:00
Jacek Galowicz add066cc7b Fix broken move 2023-10-16 19:32:47 +01:00
Vladimir Kryachko b3fd7db63f Detect cycles in flake follows.
This change results in an error thrown as opposed to segfaulting due to
stack overflow.

Fixes #9144
2023-10-16 13:00:49 -04:00
John Ericson f7a36f9812 Fix language tests a bit
- Remove some stray saved error messages that didn't correspond to any
  test, because they were renamed in
  d11faa01b5.

- Need `--eval` in test failure test in order to get in "read-only" mode
  where we don't try to write to the store. (The other tests already do
  this.)

- Need `--strict` so top-level attribute sets are still forced, like
  they are without `--eval`.
2023-10-16 12:52:59 -04:00
John Ericson 5442d9b472 Merge pull request #9167 from obsidiansystems/pre-overhaul-completions
Improve tests and docs prior to refactoring completions
2023-10-16 12:12:28 -04:00
John Ericson 483d99c622 Add API docs to some args-related functionality 2023-10-16 11:25:15 -04:00
John EricsonandNaïm Favier c27d2f8da9 Add two more completions tests
Thanks @ncfavier for catching these regressions in my PR.

Co-Authored-By: Naïm Favier <n@monade.li>
2023-10-16 10:09:10 -04:00
John Ericson d12c614ac7 Merge pull request #9152 from obsidiansystems/split-out-perl-nix
Factor out Perl bindings Nix package
2023-10-16 09:57:37 -04:00
Eelco Dolstra 6bf68c1e64 Merge pull request #8904 from wentasah/metadata-shows-last-modified
nix flakes metadata: Show lastModified timestamp for each input
2023-10-16 13:52:51 +02:00
éclairevoyant e5ce53f3db explicitly set meta.mainProgram 2023-10-15 15:08:07 -04:00
John Ericson a23cc147cb Factor out Perl bindings Nix package
Progress breaking up `flake.nix` by introducing separate `default.nix`
files which make sense on their own. (This one is a regular
`callPackage`-able package.)
2023-10-13 11:04:51 -04:00
John Ericson d070d8b746 Merge pull request #9137 from obsidiansystems/serve-protocol
Introduce separate Serve protocol serialisers
2023-10-13 10:51:46 -04:00
Eelco Dolstra 4112dd1fc9 Mark fetchTree as stable 2023-10-13 16:45:08 +02:00
Robert Hensing da2b59a088 Merge pull request #8047 from lovesegfault/always-allow-substitutes
feat: add always-allow-substitutes
2023-10-13 15:42:11 +02:00
Eelco Dolstra 2084312313 Merge pull request #9061 from edolstra/stabilize-fetchTree
fetchTree cleanup
2023-10-13 15:11:14 +02:00
Eelco Dolstra 8eb4f735dc fetchTree: Only use the registry if flakes are enabled 2023-10-13 14:34:32 +02:00
Eelco Dolstra 4ce7a53a9c Update fetchTree docs 2023-10-13 14:34:32 +02:00
Eelco Dolstra 856fe13533 fetchTree cleanup
Two changes:

* The (probably unintentional) hack to handle paths as tarballs has
  been removed. This is almost certainly not what users expect and is
  inconsistent with flakeref handling everywhere else.

* The hack to support scp-style Git URLs has been moved to the Git
  fetcher, so it's now supported not just by fetchTree but by flake
  inputs.
2023-10-13 14:34:23 +02:00
Michal SojkaandValentin Gagarin db0d94560b Document builtins.fetchTree
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>

Supersedes #6740
2023-10-13 14:24:10 +02:00
vicky1999 5c65379b22 info store alias added to store-ping 2023-10-13 07:16:05 +05:30
vicky1999 b4b1a07f97 store info alias created 2023-10-13 06:48:35 +05:30
Ninlives 94e91566ed Allow CLI to pass environment variables to FOD builder (#8830)
Add a new experimental `impure-env` setting that is a key-value list of
environment variables to inject into FOD derivations that specify the
corresponding `impureEnvVars`.

This allows clients to make use of this feature (without having to change the
environment of the daemon itself) and might eventually deprecate the current
behaviour (pick whatever is in the environment of the daemon) as it's more
principled and might prevent information leakage.
2023-10-11 11:58:42 +00:00
Valentin Gagarin 301623f3a3 Merge pull request #9134 from fricklerhandwerk/syntax-admonitions 2023-10-10 18:02:52 +02:00
John Ericson f7b8f8aff6 Introduce separate Serve protocol serialisers
To start, it is just a clone of the common protocol. But now that we
have the separate protocol implementations, we can add versioning
information without the versions of one protocol leaking into another.

Using the infrastructure from the previous commit, we don't have to
duplicate code for shared behavior.

Motivation: No more perverse incentives. [0] did some awkward things
because the serialisers did not store the version. I don't want anyone
making changes to be pushed towards keeping the serialization logic with
the core data types just because it's easier or the alternative is
tedious.

The actual versioning of the Worker and Serve protocol serialisers
(Common remains unversioned as the underlying mini-protocols are not
versioned) will happen in subsequent commits / PRs.

[0]: fe1f34fa60
2023-10-10 11:52:45 -04:00
John Ericson c7f1d86b80 Merge pull request #9115 from fricklerhandwerk/lookup-algorithm
reword documentation on lookup path resolution
2023-10-10 10:44:32 -04:00
Valentin Gagarin 7642894a4e reword documentation on lookup path resolution 2023-10-10 00:49:27 +02:00
Valentin Gagarin 1e1f08c045 annotate admonitions showing syntax
also fix typos
2023-10-10 00:45:19 +02:00
John Ericson 4b1a97338f Merge pull request #9099 from obsidiansystems/common-proto
Factor out bits of the worker protocol to use elsewhere
2023-10-09 17:31:54 -04:00
John EricsonandValentin Gagarin 4de54b2190 Unit test the "common protocol" too
Copy the relevant tests to ensure the new interfaces added in the last
commit are tested.

Perhaps I should try to deduplicat these tests some more. However its
not clear how to do that outside of a big ugly C++ macro.
https://github.com/google/googletest/blob/main/docs/advanced.md has some
stuff but it is cumbersome and I didn't figure it out yet.

This is done in a separate commit in order to be sure that the first
commit really didn't change any behavior; if we changed the
implementation and the tests at once, it would be harder to tell whether
or not some behavioral changes slipped in what is supposed to be a "pure
refactor".

Co-Authored-By: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-10-09 16:57:03 -04:00
John Ericson be81764320 Factor out bits of the worker protocol to use elsewhere
This introduces some shared infrastructure for our notion of protocols.
We can then define multiple protocols in terms of that notion.
We an also express how particular protocols depend on each other.

For example, we can define a common protocol and a worker protocol,
where the second depends on the first in terms of the data types it can
read and write.

The "serve" protocol can just use the common one for now, but will
eventually need its own machinary just like the worker protocol for
version-aware serialisers
2023-10-09 16:55:12 -04:00
Valentin Gagarin aaef47a08e Merge pull request #9131 from obsidiansystems/delete-bootstrap-script
Get rid of `bootstrap.sh`
2023-10-09 21:20:29 +02:00
John Ericson 2f0b508c29 Get rid of bootstrap.sh
For people working on Nix with `nix develop`, it's better to just use
`autoreconfPhase` and `configurePhase`, which is standard Nixpkgs / nix
shell make from Nixpkgs practice --- it is good to emphasize the degree
to which Nix is *just* a regular C++ project which can be worked on in
the regular way.

(For people running `nix-shell`, the story is similar, except
`configurePhase` would use non-writable store paths, which matters for
hte times we use output paths before `make install`, so I kept the
existing `./configure ...` instruction.)

For people building Nix without Nix (e.g. packaging it for another
distro) they also don't need `bootstrap.sh`, and can just run
`autoreconf -vfi` directly. (More likely, they have their own idioms to
do this just as we have `autoreconfPhase`.)
2023-10-09 12:55:58 -04:00
John Ericson 22513c91dc Merge pull request #8942 from fricklerhandwerk/option-anchors
always show anchors on setting listings
2023-10-09 11:29:36 -04:00
John Ericson 838be5e4a0 Merge pull request #9114 from fricklerhandwerk/lookup-path
introduce lookup paths as a distinct language construct
2023-10-09 11:28:40 -04:00
Robert Hensing aeea49609b Merge pull request #8895 from hercules-ci/gc-before-stats
eval: Run a full GC before printing stats
2023-10-09 17:14:45 +02:00
John Ericson 3953901796 Merge pull request #9112 from fricklerhandwerk/restrict-eval-nix-path
describe the effect of `restrict-eval` in a more focused manner
2023-10-09 11:07:21 -04:00
John Ericson a7a9386c96 Merge pull request #9111 from fricklerhandwerk/impure-constants
document that pure-eval also disables `builtins.nixPath`
2023-10-09 11:04:47 -04:00
John Ericson ead65d3d8e Merge pull request #9110 from fricklerhandwerk/interpolated-expression
expand on interpolated expressions
2023-10-09 10:56:19 -04:00
Eelco Dolstra c8eb3b82e9 Merge pull request #9129 from fricklerhandwerk/ae-be
AE -> BE; fix redirects
2023-10-09 16:56:07 +02:00
John Ericson 217d863f7a Merge branch 'master' into lookup-path 2023-10-09 10:54:54 -04:00
Robert Hensing c32084a12c printStats -> maybePrintStats 2023-10-09 16:34:35 +02:00
John Ericson 67eddc05ef Merge pull request #9116 from fricklerhandwerk/doc-import
reword and reformat description of `builtins.import`
2023-10-09 10:26:41 -04:00
John Ericson 81d3a8542a doc: Slight reword of "interpolated expression" in paragraph
I was sleepy and confused that "interpolated expression" was a new type of thing at first. This nudges the reader to understand that its just a regular expression, and these conditions are imposed by the interpolation operation.
2023-10-09 10:21:18 -04:00
Robert Hensing 6b6bd90030 Merge pull request #9106 from Ericson2314/positive-source-filtering
Use positive source filtering for the standalone functional tests job and Perl bindings
2023-10-09 16:14:26 +02:00
John Ericson 3015375197 Merge pull request #9126 from fricklerhandwerk/dedent
remove unnecessary indentation from markdown list
2023-10-09 09:11:04 -04:00
John Ericson 6654b4e3b4 Use positive source filtering for the Perl bindings 2023-10-09 08:29:45 -04:00
John Ericson 47b3508665 Use positive source filtering for the standalone functional tests job
Additionally this skipping of the building is reimplemented to be a bit
more robust and use the same idioms as the functionality for skipping
the tests. In particular, it will now work even if the source files
exist, so we can do this during development too.
2023-10-09 08:29:27 -04:00
Valentin Gagarin 0246de1896 remove unnecessary indentation from markdown list
this makes it a bit easier to work with, as some tooling doesn't work
well with too much indentation.
2023-10-09 10:14:29 +02:00
Valentin Gagarin 896a905202 AE -> BE; fix redirects 2023-10-09 10:07:46 +02:00
Robert Hensing c29b8ba142 Merge pull request #9105 from Ericson2314/split-out-nixos-tests
Define NixOS tests in `tests/nixos/default.nix` rather than `flake.nix`
2023-10-07 11:30:22 +02:00
Valentin Gagarin f00a5eb11b introduce lookup paths as a distinct language construct
so far they did not really have a name, and were at best referred to as
"angle bracket syntax".
2023-10-07 04:44:09 +02:00
Valentin Gagarin 6305801626 reword and reformat description of builtins.import 2023-10-07 03:57:08 +02:00
Valentin Gagarin 2fe1ccf797 describe the effect of restrict-eval in a more focused manner 2023-10-07 03:20:20 +02:00
Valentin Gagarin a86a3e5e59 document that pure-eval also disables builtins.nixPath 2023-10-07 03:12:25 +02:00
Valentin Gagarin a7ba8c3f4a complete example on attribute name interpolation 2023-10-07 02:49:55 +02:00
Valentin Gagarin a67cee965a expand on interpolated expressions 2023-10-07 02:49:55 +02:00
Valentin Gagarin c167ffe1cf Merge pull request #9108 from fricklerhandwerk/fix-redirect
remove duplicate redirects entry
2023-10-07 00:06:33 +02:00
Valentin Gagarin 517c547dec remove duplicate redirects entry 2023-10-06 23:34:08 +02:00
John Ericson 644ebaab5f Define NixOS tests in tests/nixos/default.nix rather than flake.nix
I think the our `flake.nix` is currently too large and too scary looking.
I think this matters --- if Nix cannot dog-food itself in a way that is
elegant, why should other people have confidence that their own code can
be elegant and easy to maintain?

We could do this at many points in time, but I think around now, when we
are thinking about stabilizing parts of Flakes, is an especially good
time.

This is a first step to make the `flake.nix` smaller, and make
individual components responsible for their own packaging. I hope we can
do this many more follow-ups like it, until the top-level `flake.nix` is
very small and just coordinates between other things.
2023-10-06 10:58:17 -04:00
John Ericson 61720d0035 Merge pull request #9103 from obsidiansystems/functional-tests-dir
Put functional tests in `tests/functional`
2023-10-06 09:34:11 -04:00
John Ericson 68c81c7375 Put functional tests in tests/functional
I think it is bad for these reasons when `tests/` contains a mix of
functional and integration tests

 - Concepts is harder to understand, the documentation makes a good
   unit vs functional vs integration distinction, but when the
   integration tests are just two subdirs within `tests/` this is not
   clear.

 - Source filtering in the `flake.nix` is more complex. We need to
   filter out some of the dirs from `tests/`, rather than simply pick
   the dirs we want and take all of them. This is a good sign the
   structure of what we are trying to do is not matching the structure
   of the files.

With this change we have a clean:
```shell-session
$ git show 'HEAD:tests'
tree HEAD:tests

functional/
installer/
nixos/
```
2023-10-06 09:05:56 -04:00
Eelco Dolstra 3dd4475826 Merge pull request #8905 from hercules-ci/no-unknown-location
Don't print unknown locations unless requested for dev purposes
2023-10-06 14:41:01 +02:00
Eelco Dolstra 72c3990bb4 Merge pull request #9104 from fricklerhandwerk/nix-env-links
add links and anchors
2023-10-06 14:24:01 +02:00
Eelco Dolstra 8cfa582f43 Merge pull request #9096 from NixOS/dont-rebuild-twice-on-ci
Don't run the tests that require building if we're not building
2023-10-06 14:17:05 +02:00
Eelco Dolstra 62434951d9 Merge pull request #9095 from edef1c/reject-dot-paths
StorePath: reject names starting with '.'
2023-10-06 14:12:53 +02:00
Valentin Gagarin 369b076986 add links and anchors 2023-10-06 11:46:41 +02:00
Robert Hensing f95364a803 eval: Run a full GC before printing stats
This makes the numbers more deterministic, especially when it comes
to the final heap size.
2023-10-06 00:21:05 +02:00
Robert Hensing 66a9fb0314 Merge pull request #9010 from fricklerhandwerk/first-steps
refer to nix.dev for installation instructions
2023-10-06 00:07:24 +02:00
Théophane Hufschmitt 92e8e1b1bb Poison the build on the test derivation
Make sure that we're not accidentally rebuilding Nix here as it's just
wasteful and awful for CI times.
2023-10-05 16:43:26 +02:00
Théophane Hufschmitt eb68454be6 Don't run the tests that require building if we're not building
A couple of tests require building some libraries that depend on Nix,
and assume it to be built locally.
Don't run these if we only want to run the install tests.

This prevents the CI from rebuilding several times Nix (like in
https://github.com/NixOS/nix/actions/runs/6404422275/job/17384964033#step:6:6412), thus removing a fair amount of build time.
2023-10-05 16:43:26 +02:00
Valentin Gagarin d12fb4b1f1 Merge pull request #9017 from fricklerhandwerk/contributing-docs
add contributing guide for documentation
2023-10-05 09:23:27 +02:00
Valentin Gagarin 80f734a803 more specific links to nix.dev 2023-10-05 09:20:42 +02:00
John Ericson d344c112f7 Merge pull request #9098 from obsidiansystems/test-proto
Enable most of the third `BuildResult` worker protocol test
2023-10-04 22:44:22 -04:00
Valentin Gagarin e0e47c0a68 accommodate inconsistent output from lowdown
the `term` output mode leaves inline HTML around verbatim, while `nroff`
mode (used for `man` pages) does not.

the correct solution would be to pre-render all output with a more
benign tool so we have less liabilities in our own code, but this has to
do for now.
2023-10-05 01:20:26 +02:00
edef 24bda0c7b3 StorePath: reject names starting with '.'
This has been the behaviour before Nix 2.4. It was dropped in a rewrite
in 759947bf72, allowing the creation of
store paths that aren't considered valid by older Nix versions or other
Nix tooling.

Nix 2.4 didn't ship in NixOS until 22.05, and stdenv.mkDerivation in
nixpkgs drops leading periods since April 2022, so it's unlikely anyone
is relying on the current lax behaviour.

Closes #9091.

Change-Id: I4a57bd9899e1b0dba56870ae5a1b680918a18ce9
2023-10-04 22:10:52 +00:00
Valentin Gagarin 8232711c9f fix wiring of baked-in Nix expressions 2023-10-05 00:05:08 +02:00
Valentin Gagarin 64b73476ce always show anchors on setting listings
refactor the templates for readability
2023-10-05 00:05:08 +02:00
John Ericson e1af175707 Enable most of the third BuildResult worker protocol test
This was somewhat of a false alarm. The problem was not that the
protocol implementation actually failed to round trip, but that two of
the fields were ignored entirely --- not serialized and deserialized at
all.

For reference, those fields were added in
fa68eb367e.
2023-10-04 18:03:50 -04:00
John Ericson 2f1c16dfa2 Merge pull request #9094 from obsidiansystems/test-proto
Test the rest of the worker protocol serializers
2023-10-04 16:13:00 -04:00
John Ericson 632f24166d Test the rest of the worker protocol serializers
Part of the `BuildResult` test is commented out because we have caught a
roundtrip bug! A future PR will fix the bug and uncomment that test.
2023-10-04 15:31:52 -04:00
Robert Hensing 3c042f3b0b Merge pull request #9032 from Ma27/structured-attrs-env-vars
structured attrs: improve support / usage of NIX_ATTRS_{SH,JSON}_FILE
2023-10-04 11:57:26 +02:00
John Ericson b80207fad8 Merge pull request #9081 from obsidiansystems/hacky-fix-9052
Revert "Adapt scheduler to work with dynamic derivations"
2023-10-02 08:10:25 -04:00
John Ericson 8440afbed7 Revert "Adapt scheduler to work with dynamic derivations"
This reverts commit 5e3986f59c. This
un-implements RFC 92 but fixes the critical bug #9052 which many people
are hitting. This is a decent stop-gap until a minimal reproduction of
that bug is found and a proper fix can be made.

Mostly fixed #9052, but I would like to leave that issue open until we
have a regression test, so I can then properly fix the bug (unbreaking
RFC 92) later.
2023-10-01 23:43:12 -04:00
Valentin GagarinandRobert Hensing 78e886bc5f refine the maintainer's process to unblock discussions more quickly
this addresses that we're too often running into open-ended discussions
about attempts to solve problems where neither the problem nor the
solution is well-understood enough to make decisions in a reasonable
amount of time.

this also prevents us from doing more work asynchronously.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-10-01 12:54:59 -04:00
Robert Hensing 7a0886e3cc tests/structured-attrs.sh: grep -q -> grepQuiet 2023-10-01 13:25:32 +01:00
Maximilian Bosch 42e3c6d658 doc: reference NIX_ATTRS_*_FILE vars at the env var reference for drvs 2023-10-01 13:25:29 +01:00
Maximilian Bosch bfdd908f7d structured attrs: improve support / usage of NIX_ATTRS_{SH,JSON}_FILE
In #4770 I implemented proper `nix-shell(1)` support for derivations
using `__structuredAttrs = true;`. Back then we decided to introduce two
new environment variables, `NIX_ATTRS_SH_FILE` for `.attrs.sh` and
`NIX_ATTRS_JSON_FILE` for `.attrs.json`. This was to avoid having to
copy these files to `$NIX_BUILD_TOP` in a `nix-shell(1)` session which
effectively meant copying these files to the project dir without
cleaning up afterwords[1].

On last NixCon I resumed hacking on `__structuredAttrs = true;` by
default for `nixpkgs` with a few other folks and getting back to it,
I identified a few problems with the how it's used in `nixpkgs`:

* A lot of builders in `nixpkgs` don't care about the env vars and
  assume that `.attrs.sh` and `.attrs.json` are in `$NIX_BUILD_TOP`.
  The sole reason why this works is that `nix-shell(1)` sources
  the contents of `.attrs.sh` and then sources `$stdenv/setup` if it
  exists. This may not be pretty, but it mostly works. One notable
  difference when using nixpkgs' stdenv as of now is however that
  `$__structuredAttrs` is set to `1` on regular builds, but set to
  an empty string in a shell session.

  Also, `.attrs.json` cannot be used in shell sessions because
  it can only be accessed by `$NIX_ATTRS_JSON_FILE` and not by
  `$NIX_BUILD_TOP/.attrs.json`.

  I considered changing Nix to be compatible with what nixpkgs
  effectively does, but then we'd have to either move $NIX_BUILD_TOP for
  shell sessions to a temporary location (and thus breaking a lot of
  assumptions) or we'd reintroduce all the problems we solved back then
  by using these two env vars.

  This is partly because I didn't document these variables back
  then (mea culpa), so I decided to drop all mentions of
  `.attrs.{json,sh}` in the  manual and only refer to `$NIX_ATTRS_SH_FILE`
  and `$NIX_ATTRS_JSON_FILE`. The same applies to all our integration tests.
  Theoretically we could deprecated using `"$NIX_BUILD_TOP"/.attrs.sh` in
  the future now.

* `nix develop` and `nix print-dev-env` don't support this environment
  variable at all even though they're supposed to be part of the replacement
  for `nix-shell` - for the drv debugging part to be precise.

  This isn't a big deal for the vast majority of derivations, i.e.
  derivations relying on nixpkgs' `stdenv` wiring things together
  properly. This is because `nix develop` effectively "clones" the
  derivation and replaces the builder with a script that dumps all of
  the environment, shell variables, functions etc, so the state of
  structured attrs being "sourced" is transmitted into the dev shell and
  most of the time you don't need to worry about `.attrs.sh` not
  existing because the shell is correctly configured and the

      if [ -e .attrs.sh ]; then source .attrs.sh; fi

  is simply omitted.

  However, this will break when having a derivation that reads e.g. from
  `.attrs.json` like

      with import <nixpkgs> {};
      runCommand "foo" { __structuredAttrs = true; foo.bar = 23; } ''
        cat $NIX_ATTRS_JSON_FILE # doesn't work because it points to /build/.attrs.json
      ''

  To work around this I employed a similar approach as it exists for
  `nix-shell`: the `NIX_ATTRS_{JSON,SH}_FILE` vars are replaced with
  temporary locations.

  The contents of `.attrs.sh` and `.attrs.json` are now written into the
  JSON by `get-env.sh`, the builder that `nix develop` injects into the
  derivation it's debugging. So finally the exact file contents are
  present and exported by `nix develop`.

  I also made `.attrs.json` a JSON string in the JSON printed by
  `get-env.sh` on purpose because then it's not necessary to serialize
  the object structure again. `nix develop` only needs the JSON
  as string because it's only written into the temporary file.

  I'm not entirely sure if it makes sense to also use a temporary
  location for `nix print-dev-env` (rather than just skipping the
  rewrite in there), but this would probably break certain cases where
  it's relied upon `$NIX_ATTRS_SH_FILE` to exist (prime example are the
  `nix print-dev-env` test-cases I wrote in this patch using
  `tests/shell.nix`, these would fail because the env var exists, but it
  cannot read from it).

[1] https://github.com/NixOS/nix/pull/4770#issuecomment-836799719
2023-10-01 13:22:48 +01:00
John Ericson ea2f74cbe1 Merge pull request #9022 from hercules-ci/fix-issue-8838-pathExists-isDir-slash-dot
pathExists: isDir when endswith /.
2023-09-30 00:49:17 -04:00
Robert Hensing f8a3893e8d pathExists: isDir when endswith /. 2023-09-30 02:35:26 +01:00
Robert Hensing 9c84054f97 Merge pull request #9073 from fricklerhandwerk/pr-guidelines
contributor guide: emphasize solving a well-specified problem with each pull request
2023-09-29 18:11:54 +01:00
John Ericson 461902b860 Merge pull request #9069 from obsidiansystems/libfetchers-prep-0
`libfetchers` improvements without `libflake`
2023-09-29 10:39:45 -04:00
Valentin GagarinandRobert Hensing 08145a5be5 contributor guide: emphasize solving a well-specified problem with each pull request
this moves the orientation step to the beginning, and adds notes how to
make sure that a problem is well-spefified and the according change more
likely to get accepted

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-09-29 16:19:14 +02:00
Théophane Hufschmitt 784c7df5bf Merge pull request #9004 from fricklerhandwerk/release-support
add information on release cycle and backports
2023-09-29 15:04:00 +02:00
Théophane Hufschmitt a66bd8f1f2 Merge pull request #9065 from fricklerhandwerk/testing-hints
add hint for troubleshooting tests
2023-09-29 13:52:54 +02:00
Valentin Gagarin 1dd03c62ad add hint for troubleshooting tests 2023-09-29 10:46:42 +02:00
John Ericson 89b3952063 Make the indirect fetcher input scheme part of the Flakes XP feature
I don't know much about it, but by the number of times "flake" appears
in the code it seems like is part of flakes, at least for now.
2023-09-28 21:35:36 -04:00
John Ericson bfe1308d3f Add infra for InputSchemes to be experimental 2023-09-28 21:35:30 -04:00
John Ericson c816c67eed Reword some comments/API docs to reflect libfetcher's multiple users
It's not just flakes, but also `builtins.fetchTree`. Also try to provide
some more info in general.
2023-09-28 21:10:51 -04:00
John Ericson b912f3a937 Move flakeIdRegex{,S} from libutil to flakeref.{cc,hh
It isn't used, and doesn't belong in `libutil`.
2023-09-28 20:55:41 -04:00
John Ericson cede94dbf7 builtins.fetchTree: Mark experimental the new way
This helps ensure uniform docs/error message.
2023-09-28 20:51:25 -04:00
John Ericson b7e712f9fd Merge pull request #8509 from wentasah/fetch-tree-doc
Document fetchTree
2023-09-28 15:13:53 -04:00
Andrea Bedini add7c99c3b Include "original" and "locked" in nix flake prefetch --json 2023-09-28 12:34:06 -04:00
Robert Hensing 62ddb6851f Merge pull request #8852 from flox/tomberek.absolute.attrpath.notation
Absolute attrPath notation ("flakeref#.attrPath")
2023-09-28 15:52:39 +01:00
tomberek 976f596579 Merge branch 'master' into tomberek.absolute.attrpath.notation 2023-09-28 10:01:57 -04:00
Robert Hensing 13a9090ffc Merge pull request #9047 from flox/tomberek.string_refactor
string Value refactor
2023-09-28 02:58:57 +01:00
Ilan Joselevich 13ed5d7106 flakes: adopt repl-flake behavior as default 2023-09-27 20:47:10 -04:00
Robert Hensing ea2fa8b6f3 Merge pull request #8241 from Luabee/patch-1
Mention `$DRV_PATH` in post-build-hook docs
2023-09-28 00:11:00 +01:00
Robert Hensing 11a3dc99b2 Merge pull request #7003 from SuperSandro2000/patch-1
Improve experimental-features error wording
2023-09-27 23:26:27 +01:00
Robert HensingandValentin Gagarin dafa38213b Update doc/manual/src/advanced-topics/post-build-hook.md
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-09-27 23:10:39 +01:00
Robert Hensing 16a6ea7249 Merge pull request #9049 from inclyc/users/inclyc/move-path
libexpr: construct ExprPath by move ctor, not copy cotr
2023-09-27 22:30:44 +01:00
Robert Hensing 4b78a66bc5 Merge pull request #9056 from aakropotkin/patch-1
Respect `NOCOLOR`
2023-09-27 22:29:09 +01:00
Alex Ameen 5bc540a8ca Respect NOCOLOR
While `nix` has always been respectful towards requests for `NO_COLOR=1`, this change asks represents a new stage of maturity for `nix` - making it also respect quests for `NOCOLOR=1`.

This ideally makes the tool more accessible to folks like me, who are exhausted by guessing whether `NO_COLOR` or `NOCOLOR` is the right environment variable to set.

<3
2023-09-27 14:49:52 -05:00
Valentin Gagarin 57eb62d230 Merge pull request #9051 from cafkafk/patch-2 2023-09-27 07:19:00 +02:00
Tom Bereknyei 399ef84420 refactor: use string accessors
Create context, string_view, and c_str, accessors throughout in order to
better support improvements to the underlying string representation.
2023-09-27 00:33:01 -04:00
Christina Sørensen 1eeea01931 Fix repl.md duplicate typo
Seems like `legacyPackages.x86_64-linux.emacs.name` is accidentally shown twice.
2023-09-27 02:07:43 +00:00
Yingchi Long 5b902ce9d6 libexpr: construct ExprPath by move ctor, not copy cotr 2023-09-26 23:30:32 +08:00
Théophane Hufschmitt c6faef61a6 Merge pull request #8923 from obsidiansystems/test-proto
Unit test some worker protocol serializers
2023-09-26 17:12:24 +02:00
Robert Hensing 57202969d0 Merge pull request #9040 from waalge/waalge/tail-docstr
fix docstring
2023-09-26 15:32:00 +01:00
Eelco Dolstra 706b91ef62 Merge pull request #9044 from fricklerhandwerk/doc-realise
realisation: reformat for readability
2023-09-26 15:28:20 +02:00
Théophane Hufschmitt 1da1642527 Merge pull request #9041 from trofi/profiles-sign
src/libstore/profiles.cc: fix comparison of sign difference
2023-09-26 07:50:17 +02:00
Valentin Gagarin a757749fcf reword for readability 2023-09-26 06:28:17 +02:00
Valentin Gagarin 9428d7dcd1 Merge pull request #9045 from fricklerhandwerk/fix-redirects
fix broken redirects script
2023-09-26 04:27:33 +02:00
Valentin Gagarin 503b02d3f8 Merge pull request #9046 from fricklerhandwerk/fixup-derivation
fix broken reference link
2023-09-26 04:27:23 +02:00
Valentin Gagarin 60a155d01c Merge pull request #8706 from fricklerhandwerk/doc-system-features
document system features
2023-09-26 04:21:31 +02:00
b17f200b11 Document "Import From Derivation" (#7332)
* document "Import From Derivation"

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: John Ericson <git@JohnEricson.me>
2023-09-26 01:49:03 +00:00
Tom Bereknyei 7e24dc606b fix(tests): fix assumption that string.s is a char* 2023-09-25 21:37:32 -04:00
Valentin Gagarin c6f8247032 fix broken reference link 2023-09-26 03:10:12 +02:00
Théophane Hufschmitt 9a78d87bc0 Merge pull request #6614 from RasmusRendal/spaces
Implement support for percent encoded filepaths for flakerefs
2023-09-26 02:27:09 +02:00
Valentin Gagarin b7d88fe56a Merge pull request #7320 from fricklerhandwerk/doc-derivation
restructure attribute listing to `derivation`
2023-09-26 01:42:07 +02:00
Valentin Gagarin 45de35bcf1 fix broken redirects script 2023-09-26 01:24:07 +02:00
Valentin Gagarin 887cbcd395 add contributing guide for documentation 2023-09-26 01:06:47 +02:00
Valentin Gagarin 5b0336b3b1 reword example for clarity 2023-09-26 00:58:50 +02:00
Valentin GagarinandRobert Hensing 7de66f19f8 example: headers -> dev
make the example more realistic, since `headers` is not an output name
used in Nixpkgs

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-09-26 00:46:27 +02:00
Valentin Gagarin 75a231147f be more precise about system semantics 2023-09-26 00:45:00 +02:00
Valentin Gagarin 5e4734a08b reword to avoid saying "build"
derivations are about data transformation, so the term "build" does not
add any information. there was also some feedback that "build task" is
not more helpful than "derivation" if you have no prior experience with
Nix or build systems, while existing associations may be misleading.
2023-09-26 00:45:00 +02:00
Valentin GagarinandRobert Hensing 17884f54d1 clarification on extra attributes
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-09-26 00:45:00 +02:00
Valentin Gagarin 026c24e378 add example for store path using the given name 2023-09-26 00:45:00 +02:00
Valentin GagarinandRobert Hensing d621dd17f2 more precise wording
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-09-26 00:45:00 +02:00
Valentin Gagarin e2f118efed restructure and reword input attributes section on derivation 2023-09-26 00:43:58 +02:00
Valentin Gagarin 541890463d make separate section for builder execution 2023-09-26 00:43:58 +02:00
Valentin Gagarin cba53b3a18 reformat for readability 2023-09-26 00:42:23 +02:00
Robert Hensing b19bd4f348 Merge pull request #8970 from hercules-ci/eval-stuff
Expr: remove redundant fields, add nrExprs
2023-09-25 19:49:22 +02:00
John Ericson 1f3fc08c59 Merge pull request #8887 from obsidiansystems/bsd-cross-ci
Support cross compiling to BSD and CI it
2023-09-25 13:46:55 -04:00
Robert HensingandYingchi Long bd24176ac5 libexpr/nixexpr.hh: Remove redundant inline
This is redundant since definitions in C++ record are implicitly inline-ed.

Co-authored-by: Yingchi Long <i@lyc.dev>
2023-09-25 17:51:17 +01:00
Sergei Trofimovich ad213103d8 src/libstore/profiles.cc: fix comparison of sign difference
Detected by `gcc` as:

      CXX    src/libstore/profiles.o
    src/libstore/profiles.cc: In function 'void nix::deleteGenerationsGreaterThan(const Path&, GenerationNumber, bool)':
    src/libstore/profiles.cc:186:50: warning: comparison of integer expressions of different signedness: 'int' and 'nix::GenerationNumber' {aka 'long unsigned int'} [-Wsign-compare]
      186 |     for (auto keep = 0; i != gens.rend() && keep < max; ++i, ++keep);
          |                                             ~~~~~^~~~~
2023-09-25 17:45:57 +01:00
Robert Hensing b21c41529d Merge pull request #9024 from obsidiansystems/git-objects-prep
Shuffle `ParseSink` code in preparation for git hashing support
2023-09-25 16:55:11 +02:00
waalge 70b5e6050c fix docstring 2023-09-25 13:39:11 +00:00
John Ericson 728767db03 Merge pull request #9028 from Ericson2314/nix3-config-options
Misc options rendering adjustments
2023-09-25 09:04:56 -04:00
Eelco Dolstra de99647b9c Merge pull request #9029 from inclyc/users/lyc/pass-value-2
libexpr: const rvalue reference -> value for nix::Expr nodes
2023-09-25 14:37:39 +02:00
John Ericson 4606a07bb6 generate-manpage.nix: Add comment explaining one bit 2023-09-25 08:20:39 -04:00
Théophane Hufschmitt e304e8d053 Merge pull request #7593 from miallo/ignore-identical-rc-backup-files-for-installation
install-multi-user: ignore profile_target backups that have no change
2023-09-25 14:14:53 +02:00
Eelco Dolstra 1e99b7a425 Merge pull request #9026 from Gerg-L/master
flake: complete update to 23.05
2023-09-25 10:43:34 +02:00
Théophane HufschmittandValentin Gagarin b3433099d4 Apply suggestions from code review
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-09-25 09:56:49 +02:00
Maximilian Bosch 89e5e68799 doc/hacking: fix make target to build the docs (#9033)
Was confused why `make html` didn't work while working on #9032, but
then I realized that after this section was written, the target was
renamed to `manual-html` in 6910f5dcb6.
2023-09-24 20:28:34 +02:00
Yingchi Long e4b83fbfe2 libexpr: const rvalue reference -> value for nix::Expr nodes 2023-09-24 14:54:41 +08:00
John Ericson 9f93972c4d manual / manpages: Make option category names a proper subheader
Before they were an "ad-hoc" header with bold and a colon; now they are
a proper subheader.

For the man pages, this doesn't make much of a difference, but it will
help more on for the HTML manual, where things can be restyled. Again,
good separation of content vs presentation.
2023-09-23 00:37:44 -04:00
John Ericson 1d9fd3a6f8 manual / manpages: Adjust option filter filtering, move from C++ to Nix
Behavior change:

Before we only showed uption if the command-specific options were
non-empty. But that is somewhat odd since we also show common options.
Now, we do everything based on the union of both sorts of options (with
hidden-categories filtered, as before).

Implementation change:

The JSON dumping once again includes all options; the filtering of
hidden categories is done in the Nix instead. This is better separation
of "content" vs "presentation", and prepare the way for the HTML manual
vs manpages / `--help` doing different things.
2023-09-23 00:34:51 -04:00
John Ericson 9c640c1229 doc: showOptions: Simplify code with builtins.groupBy
This makes grouping options by category much nicer. No behavior should
be changed.
2023-09-23 00:28:16 -04:00
John Ericson 694810ba34 doc: showOptions: Move union to caller
`showOptions` itself doesn't care, so it shouldn't take two separate
arguments.
2023-09-22 23:54:04 -04:00
f2e201fbdb Expose RestoreSink in header (fs-sink.hh)
Co-Authored-By: Matthew Bauer <mjbauer95@gmail.com>
Co-Authored-By: Carlo Nucera <carlo.nucera@protonmail.com>
2023-09-22 09:11:29 -04:00
8a416e819c Move RestoreSink to fs-sink.cc
Co-Authored-By: Matthew Bauer <mjbauer95@gmail.com>
Co-Authored-By: Carlo Nucera <carlo.nucera@protonmail.com>
2023-09-22 09:10:32 -04:00
9d6114313b Move ParseSink to its own header
We will soon add a new implemenation so the one for NARs in `archive.cc`
isn't the only one.

Co-Authored-By: Matthew Bauer <mjbauer95@gmail.com>
Co-Authored-By: Carlo Nucera <carlo.nucera@protonmail.com>
2023-09-22 09:10:32 -04:00
39ba81a4eb Improve internal API docs for two file hashing functions
Co-Authored-By: Matthew Bauer <mjbauer95@gmail.com>
Co-Authored-By: Carlo Nucera <carlo.nucera@protonmail.com>
2023-09-22 09:10:32 -04:00
Théophane Hufschmitt f89b84919c Merge pull request #8931 from fricklerhandwerk/nix3-config-options
do not show configuration override flags for each command
2023-09-22 14:13:51 +02:00
Rasmus Rendal 3411507696 Document the percent-encoding mechanism 2023-09-22 10:07:14 +02:00
Théophane Hufschmitt e8113747e1 Split the parseFlakeRefWithFragment function
Was starting to be very complex and hard to follow.
Now the different cases should be easier to understand.
2023-09-22 10:06:43 +02:00
Théophane HufschmittandRendal 50e61f579c Allow special characters in flake paths
Support using nix flakes in paths with spaces or abitrary unicode characters.
This introduces the convention that the path part of the URL should be
percent-encoded when dealing with `path:` urls and not when using
filepaths (following the convention of firefox).

Co-authored-by: Rendal <rasmus@rend.al>
2023-09-22 10:06:43 +02:00
Rasmus Rendal d8cebae939 Add a test for flake paths with spaces in them 2023-09-22 10:06:43 +02:00
Gerg-L f264d9ff08 flake: complete update to 23.05 2023-09-21 21:00:53 -04:00
Valentin GagarinandJohn Ericson 1b560ea502 more detail on backports
Co-authored-by: John Ericson <git@JohnEricson.me>
2023-09-21 23:23:14 +02:00
John Ericson 2d2aff5b92 Merge pull request #9016 from fricklerhandwerk/contributing-readme
README: link to CONTRIBUTING
2023-09-21 12:43:17 -04:00
John Ericson 6b6cfec8a5 Merge pull request #8990 from fricklerhandwerk/example-nix-env-install-outputs
add clarifying example to `nix-env` output selection
2023-09-21 12:27:35 -04:00
John Ericson 914cc7dc02 Merge pull request #9018 from fricklerhandwerk/readme-irc
remove IRC from links in README
2023-09-21 12:00:21 -04:00
John Ericson fd61799c0b Merge pull request #9006 from fricklerhandwerk/fix-links
fix links to configuration settings
2023-09-21 11:58:07 -04:00
John Ericson 0273ad73fa Merge pull request #9012 from fricklerhandwerk/fix-multiline-doc
fix rendering error for consecutive spaces
2023-09-21 11:57:39 -04:00
John Ericson 7bafcdba79 Merge pull request #9013 from fricklerhandwerk/test-coverage-docs
Move test coverage section to testing page
2023-09-21 11:50:04 -04:00
Robert Hensing 89a604a6b6 Merge pull request #9021 from fricklerhandwerk/settings-docstring
fix typo in docstring
2023-09-21 13:20:13 +02:00
Robert Hensing fb1c22df94 Merge pull request #9019 from fricklerhandwerk/doc-delete-unused
delete unused files
2023-09-21 13:16:03 +02:00
Valentin Gagarin 1a412a8d78 fix typo in docstring 2023-09-21 11:38:13 +02:00
Valentin Gagarin cf6ba7256f delete unused files 2023-09-21 02:57:00 +02:00
Valentin Gagarin 984bd4cb0e README: link to CONTRIBUTING 2023-09-21 02:53:36 +02:00
Valentin Gagarin 8e25450ff4 refer to nix.dev for installation instructions
there are currently multiple places with installation instructions that
all have to be updated when a change to any of them is accepted.

this reduces the number of places by one, and directs beginners to the
maintained and curated resource for Nix learning materials.
2023-09-21 02:53:11 +02:00
Valentin Gagarin 4685260a77 fix links to configuration settings 2023-09-21 02:52:58 +02:00
Valentin Gagarin 02649d247b move test coverage section to testing page 2023-09-21 02:52:28 +02:00
Valentin Gagarin 954890a42f add information on release cycle and backports 2023-09-21 02:52:05 +02:00
Valentin Gagarin 747b2baf21 fix rendering error for consecutive spaces 2023-09-21 02:51:54 +02:00
Valentin Gagarin e0e5943db2 remove IRC from links in README
the community has moved away from IRC a long time ago
2023-09-21 02:51:39 +02:00
Eelco Dolstra 3ce793d3e9 Merge pull request #9008 from cole-h/re-enable-nspawn-test
Re-enable systemd-nspawn test
2023-09-20 19:03:13 +02:00
Cole Helbling 883092e3f7 Re-enable systemd-nspawn test
It was disabled in c6953d1ff6 because
a recent Nixpkgs bump brought in a new systemd which changed how
systemd-nspawn worked.

As far as I can tell, the issue was caused by this upstream systemd
commit:
https://github.com/systemd/systemd/commit/b71a0192c040f585397cfc6fc2ca025bf839733d

Bind-mounting the host's `/sys` and `/proc` into the container's
`/run/host/{sys,proc}` fixes the issue and allows the test to succeed.
2023-09-20 09:09:01 -07:00
Eelco Dolstra d3f866cf11 Merge pull request #9003 from NixOS/bump-2.19.0
Bump version
2023-09-20 16:09:37 +02:00
Eelco Dolstra b7acef1ceb Bump version 2023-09-20 15:21:21 +02:00
John Ericson c18911602e Fix boehmgc-coroutine-sp-fallback.diff for FreeBSD
Our FreeBSD headers have `pthread_getattr_np`, but we get a link-time
error that is missing. The good news is that there is another similar
function which does exist, and the upstream project elsewhere does just
the [fallback code] we need.

As the fallback code indicates, the two functions are not identical
however as the other one needs explicit initialization. NetBSD supports
both in fact, and its [manpage] is therefore a good
resource on what the differences are.

[fallback code]: https://github.com/ivmai/bdwgc/blob/07a6d0ee8889bca5eaeadc13cabadc363725d216/os_dep.c#L1266-L1272

[manpage]: https://man.netbsd.org/pthread_attr_get_np.3
2023-09-20 09:04:42 -04:00
John Ericson 7f76d7f038 Rename an identifier of ours called stdout
This is a reserved identifier on NetBSD --- it is replaced by a macro on
that platform --- and so we cannot use it.
2023-09-20 09:04:42 -04:00
John Ericson 0db251e4ad Do not build docs in cross devShell
Coppied from the main build; we really should deduplicate this more.
2023-09-20 09:04:42 -04:00
John Ericson 564392b57b Make libsodium an unconditional dependency
The configure script will not tolerate it being missing.
2023-09-20 09:04:42 -04:00
John Ericson 28850ee900 Make dev shells work for cross
Need to get tools from right package set. Could build clang tools but I
don't want to wait :D.
2023-09-20 09:04:42 -04:00
John Ericson e44d2a6bbe Add FreeBSD and NetBSD cross to Nix's flake 2023-09-20 09:04:42 -04:00
Valentin Gagarin 408055a9dd add clarifying example to nix-env output selection
there is a very confusing warning in the Nixpkgs manual that
mischaracterises `nix-env` behavior, and this example shows what's
really happening.

note that it doesn't use `pkgs.runCommand` or other `pkgs.stdenv`
facilities, as deep down those set `meta.outputsToInstall` to very
particular defaults that do not generally apply to Nix.
2023-09-18 20:11:42 +02:00
Robert Hensing bf8deb4991 Expr: remove redundant int and float fields 2023-09-12 13:45:45 +02:00
Robert Hensing 3720e811fa libexpr: Add nrExprs to NIX_SHOW_STATS 2023-09-12 13:21:55 +02:00
Graham Bennett 82ddb13098 Unlock output paths when a derivation is already built
Without this change, nix build processes will not drop the locks for derivation goals
which have already been built by another process when the current process gets
round to building them. This means the locks are held until the process
terminates.

If there are other nix build processes in a similar state, they will also try to
acquire the same locks when they try to build the same derivation, and so will
wait until the lock holder terminates (which might be a very long time if it has
a lot to build). In some pathological cases, those processes might be holding
their own locks on other derivations due to the same issue, and this can lead to
deadlock.

Resolves #6468
2023-09-09 10:11:11 +01:00
Valentin Gagarin aa46f536e8 add note on overriding settings for stable commands 2023-09-07 02:02:27 +02:00
Valentin Gagarin 09eb7f1ef6 do not show configuration override flags for each command
this removes a lot of noise from the web search, which precludes finding
the actual documentation.

some configuration settings have enough documentation to warrant
individual pages, so the alternative of including full setting
documentation in each command page doesn't make much sense here.

this change technically means that the command line flags to override
settings are "invisible", and not exported as JSON. this may or may not
be desirable. a more explicit approach would be adding a `hidden` field
to the flag's JSON output, but would also require adjusting
post-processing of that JSON for manual rendering.
2023-09-07 02:02:27 +02:00
Valentin Gagarin b0fe7f560d add missing link 2023-09-07 01:38:53 +02:00
Valentin Gagarin 589fd897fb add todo on store docs 2023-09-07 01:38:53 +02:00
John EricsonandAndreas Rammhold 7ff43435f9 Unit test some worker protocol serializers
Continue with the characterization testing idioms begun in
c70484454f, but this time for unit tests.

Co-authored-by: Andreas Rammhold <andreas@rammhold.de>
2023-09-05 10:48:02 -04:00
Robert Hensing 477bc617bb traces: Add _NIX_DEVELOPER_SHOW_UNKNOWN_LOCATIONS 2023-09-03 14:08:03 +02:00
Robert Hensing f1aeeea32b traces: DRY printPosMaybe 2023-09-03 14:00:55 +02:00
Robert Hensing 4c50f5d130 traces: Do not print unknown location
Solves 1/3 of the infinite recursion at unknown location meme.
See #8879 for ensuring we always have a trace (for stack overflows)
We might want to re-add this for finding missing location info
*while hacking on that problem only*.
2023-09-03 13:44:32 +02:00
Walter Franzini 3ae1489847 nix flakes metadata: Show lastModified timestamp for each input
Before this commit, the output of `nix flake metadata` in nix repo
looked like this:

    ...
    Last modified: 2023-07-09 16:00:16
    Inputs:
    ├───flake-compat: github:edolstra/flake-compat/35bb57c0c8d8b62bbfd284272c928ceb64ddbde9
    ├───lowdown-src: github:kristapsdz/lowdown/d2c2b44ff6c27b936ec27358a2653caaef8f73b8
    ├───nixpkgs: github:NixOS/nixpkgs/04a75b2eecc0acf6239acf9dd04485ff8d14f425
    └───nixpkgs-regression: github:NixOS/nixpkgs/215d4d0fd80ca5163643b03a33fde804a29cc1e2

This commit changes that to:

    ...
    Last modified: 2023-07-09 16:00:16
    Inputs:
    ├───flake-compat: github:edolstra/flake-compat/35bb57c0c8d8b62bbfd284272c928ceb64ddbde9 (2023-01-17 11:47:33)
    ├───lowdown-src: github:kristapsdz/lowdown/d2c2b44ff6c27b936ec27358a2653caaef8f73b8 (2021-10-06 10:00:07)
    ├───nixpkgs: github:NixOS/nixpkgs/04a75b2eecc0acf6239acf9dd04485ff8d14f425 (2022-12-08 01:04:00)
    └───nixpkgs-regression: github:NixOS/nixpkgs/215d4d0fd80ca5163643b03a33fde804a29cc1e2 (2022-01-24 19:20:45)
2023-09-03 12:35:29 +02:00
Valentin Gagarin 880fef9cdf do not change existing release notes 2023-08-28 20:52:30 +02:00
Tom Bereknyei 696eb79b15 test: test behavior of .-prefixed attrPaths 2023-08-27 04:42:52 -04:00
Tom Bereknyei c609be4072 doc: explain the . attrPath prefix notation 2023-08-19 17:19:52 -04:00
Tom Bereknyei 52248b1c27 feat: notation to refer to no attribute search prefix
An attrPath prefix of "." indicates no need to try default attrPath prefixes. For example 1nixpkgs#legacyPackages.x86_64-linux.ERROR` searches through

```
trying flake output attribute 'packages.x86_64-linux.legacyPackages.x86_64-linux.ERROR'
using cached attrset attribute ''
trying flake output attribute 'legacyPackages.x86_64-linux.legacyPackages.x86_64-linux.ERROR'
using cached attrset attribute 'legacyPackages.x86_64-linux'
trying flake output attribute 'legacyPackages.x86_64-linux.ERROR'
using cached attrset attribute 'legacyPackages.x86_64-linux'
```

And there is no way to specify that one does not want the automatic
search behavior. Now one can specify
`nixpkgs#.legacyPackages.x86_64-linux.ERROR` to only refer to the rooted
attribute path without any default injection of attribute search path or
system.
2023-08-19 17:07:46 -04:00
Valentin Gagarin 2fa90e5824 add more details on CA derivations 2023-07-19 13:59:18 +02:00
Valentin Gagarin 5f37ebcf83 document all special system features and their behavior 2023-07-19 13:34:03 +02:00
Valentin Gagarin 493ddf617f reformat system-features setting documentation 2023-07-19 11:07:47 +02:00
Valentin Gagarin feb01b22ed add links to store API documentation 2023-07-19 11:07:47 +02:00
Michal SojkaandValentin Gagarin f11445952f Document builtins.fetchTree
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>

Supersedes #6740
2023-06-26 00:59:06 +02:00
Bernardo Meurer bf693319f6 feat: add always-allow-substitutes
This adds a new configuration option to Nix, `always-allow-substitutes`,
whose effect is simple: it causes the `allowSubstitutes` attribute in
derivations to be ignored, and for substituters to always be used.

This is extremely valuable for users of Nix in CI, where usually
`nix-build-uncached` is used. There, derivations which disallow
substitutes cause headaches as the inputs for building already-cached
derivations need to be fetched to spuriously rebuild some simple text
file.

This option should be a good middle-ground, since it doesn't imply
rebuilding the world, such as the approach I took in
https://github.com/NixOS/nixpkgs/pull/221048
2023-05-22 19:42:29 +01:00
Rynn Blackmon c4df53f154 Mention $DRV_PATH in post-build-hook docs 2023-04-19 16:32:47 -07:00
Michael Lohmann c74248c56e install-multi-user: ignore profile_target backups that have no change
If there was a prior nix installation that created this backup file and
then you tried to install it again, it would stop to tell you there is
this file. But if the file and its backup are identical in content,
there is no harm in continuing and in a later step overwriting the
existing backup file with the identical one. This is just a convenience
feature.
2023-01-12 15:53:49 +01:00
SandroandValentin Gagarin 481e4082bf Update src/libutil/experimental-features.cc
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2022-11-09 12:03:53 +01:00
Sandro 408dcfc0d3 Improve experimental-features error wording 2022-09-05 15:42:10 +02:00
1077 changed files with 17999 additions and 8715 deletions

No files matched your search

+1 -1
View File
@@ -17,7 +17,7 @@ indent_style = space
indent_size = 2
# Match c++/shell/perl, set indent to spaces with width of four
[*.{hpp,cc,hh,sh,pl}]
[*.{hpp,cc,hh,sh,pl,xs}]
indent_style = space
indent_size = 4
+7
View File
@@ -14,5 +14,12 @@
/doc @fricklerhandwerk
*.md @fricklerhandwerk
# Documentation of built-in functions
src/libexpr/primops.cc @fricklerhandwerk @roberth
# Documentation on experimental features
src/libutil/experimental-features.cc @fricklerhandwerk
# Documentation on configuration settings
src/libstore/globals.hh @fricklerhandwerk
# Libstore layer
/src/libstore @thufschmitt
+1 -1
View File
@@ -20,4 +20,4 @@
# Unit tests
- src/*/tests/**/*
# Functional and integration tests
- tests/**/*
- tests/functional/**/*
+1 -1
View File
@@ -21,7 +21,7 @@ jobs:
fetch-depth: 0
- name: Create backport PRs
# should be kept in sync with `version`
uses: zeebe-io/backport-action@v1.4.0
uses: zeebe-io/backport-action@v2.1.1
with:
# Config README: https://github.com/zeebe-io/backport-action#backport-action
github_token: ${{ secrets.GITHUB_TOKEN }}
+23
View File
@@ -101,6 +101,9 @@ jobs:
docker_push_image:
needs: [check_secrets, tests]
permissions:
contents: read
packages: write
if: >-
github.event_name == 'push' &&
github.ref_name == 'master' &&
@@ -126,6 +129,9 @@ jobs:
- run: docker load -i ./result/image.tar.gz
- run: docker tag nix:$NIX_VERSION nixos/nix:$NIX_VERSION
- run: docker tag nix:$NIX_VERSION nixos/nix:master
# We'll deploy the newly built image to both Docker Hub and Github Container Registry.
#
# Push to Docker Hub first
- name: Login to Docker Hub
uses: docker/login-action@v3
with:
@@ -133,3 +139,20 @@ jobs:
password: ${{ secrets.DOCKERHUB_TOKEN }}
- run: docker push nixos/nix:$NIX_VERSION
- run: docker push nixos/nix:master
# Push to GitHub Container Registry as well
- name: Login to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Push image
run: |
IMAGE_ID=ghcr.io/${{ github.repository_owner }}/nix
# Change all uppercase to lowercase
IMAGE_ID=$(echo $IMAGE_ID | tr '[A-Z]' '[a-z]')
docker tag nix:$NIX_VERSION $IMAGE_ID:$NIX_VERSION
docker tag nix:$NIX_VERSION $IMAGE_ID:master
docker push $IMAGE_ID:$NIX_VERSION
docker push $IMAGE_ID:master
+24 -17
View File
@@ -21,12 +21,14 @@ perl/Makefile.config
/doc/manual/language.json
/doc/manual/xp-features.json
/doc/manual/src/SUMMARY.md
/doc/manual/src/SUMMARY-rl-next.md
/doc/manual/src/command-ref/new-cli
/doc/manual/src/command-ref/conf-file.md
/doc/manual/src/command-ref/experimental-features-shortlist.md
/doc/manual/src/contributing/experimental-feature-descriptions.md
/doc/manual/src/language/builtins.md
/doc/manual/src/language/builtin-constants.md
/doc/manual/src/release-notes/rl-next.md
# /scripts/
/scripts/nix-profile.sh
@@ -79,24 +81,24 @@ perl/Makefile.config
/src/build-remote/build-remote
# /tests/
/tests/test-tmp
/tests/common/vars-and-functions.sh
/tests/result*
/tests/restricted-innocent
/tests/shell
/tests/shell.drv
/tests/config.nix
/tests/ca/config.nix
/tests/dyn-drv/config.nix
/tests/repl-result-out
/tests/test-libstoreconsumer/test-libstoreconsumer
# /tests/functional/
/tests/functional/test-tmp
/tests/functional/common/vars-and-functions.sh
/tests/functional/result*
/tests/functional/restricted-innocent
/tests/functional/shell
/tests/functional/shell.drv
/tests/functional/config.nix
/tests/functional/ca/config.nix
/tests/functional/dyn-drv/config.nix
/tests/functional/repl-result-out
/tests/functional/test-libstoreconsumer/test-libstoreconsumer
# /tests/lang/
/tests/lang/*.out
/tests/lang/*.out.xml
/tests/lang/*.err
/tests/lang/*.ast
# /tests/functional/lang/
/tests/functional/lang/*.out
/tests/functional/lang/*.out.xml
/tests/functional/lang/*.err
/tests/functional/lang/*.ast
/perl/lib/Nix/Config.pm
/perl/lib/Nix/Store.cc
@@ -138,7 +140,12 @@ nix-rust/target
result
# IDE
.vscode/
.idea/
# clangd and possibly more
.cache/
# Mac OS
.DS_Store
+1 -1
View File
@@ -1 +1 @@
2.18.0
2.20.0
+19 -11
View File
@@ -24,25 +24,33 @@ Check out the [security policy](https://github.com/NixOS/nix/security/policy).
## Making changes to Nix
1. Check for [pull requests](https://github.com/NixOS/nix/pulls) that might already cover the contribution you are about to make.
There are many open pull requests that might already do what you intent to work on.
1. Search for related issues that cover what you're going to work on.
It could help to mention there that you will work on the issue.
Issues labeled [good first issue](https://github.com/NixOS/nix/labels/good%20first%20issue) should be relatively easy to fix and are likely to get merged quickly.
Pull requests addressing issues labeled [idea approved](https://github.com/NixOS/nix/labels/idea%20approved) or [RFC](https://github.com/NixOS/nix/labels/RFC) are especially welcomed by maintainers and will receive prioritised review.
If you are proficient with C++, addressing one of the [popular issues](https://github.com/NixOS/nix/issues?q=is%3Aissue+is%3Aopen+sort%3Areactions-%2B1-desc) will be highly appreciated by maintainers and Nix users all over the world.
For far-reaching changes, please investigate possible blockers and design implications, and coordinate with maintainers before investing too much time in writing code that may not end up getting merged.
If there is no relevant issue yet and you're not sure whether your change is likely to be accepted, [open an issue](https://github.com/NixOS/nix/issues/new/choose) yourself.
2. Check for [pull requests](https://github.com/NixOS/nix/pulls) that might already cover the contribution you are about to make.
There are many open pull requests that might already do what you intend to work on.
You can use [labels](https://github.com/NixOS/nix/labels) to filter for relevant topics.
2. Search for related issues that cover what you're going to work on. It could help to mention there that you will work on the issue.
Issues labeled [good first issue](https://github.com/NixOS/nix/labels/good-first-issue) should be relatively easy to fix and are likely to get merged quickly.
Pull requests addressing issues labeled [idea approved](https://github.com/NixOS/nix/labels/idea%20approved) are especially welcomed by maintainers and will receive prioritised review.
3. Check the [Nix reference manual](https://nixos.org/manual/nix/unstable/contributing/hacking.html) for information on building Nix and running its tests.
For contributions to the command line interface, please check the [CLI guidelines](https://nixos.org/manual/nix/unstable/contributing/cli-guideline.html).
4. Make your changes!
4. Make your change!
5. [Create a pull request](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request) for your changes.
* Link related issues in your pull request to inform interested parties and future contributors about your change.
* Clearly explain the problem that you're solving.
Link related issues to inform interested parties and future contributors about your change.
If your pull request closes one or multiple issues, mention that in the description using `Closes: #<number>`, as it will then happen automatically when your change is merged.
* Make sure to have [a clean history of commits on your branch by using rebase](https://www.digitalocean.com/community/tutorials/how-to-rebase-and-update-a-pull-request).
If your pull request closes one or multiple issues, note that in the description using `Closes: #<number>`, as it will then happen automatically when your change is merged.
* [Mark the pull request as draft](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/changing-the-stage-of-a-pull-request) if you're not done with the changes.
6. Do not expect your pull request to be reviewed immediately.
@@ -52,7 +60,7 @@ Check out the [security policy](https://github.com/NixOS/nix/security/policy).
- [ ] Fixes an [idea approved](https://github.com/NixOS/nix/labels/idea%20approved) issue
- [ ] Tests, as appropriate:
- Functional tests – [`tests/**.sh`](./tests)
- Functional tests – [`tests/functional/**.sh`](./tests/functional)
- Unit tests – [`src/*/tests`](./src/)
- Integration tests – [`tests/nixos/*`](./tests/nixos)
- [ ] User documentation in the [manual](..doc/manual/src)
+17 -9
View File
@@ -1,3 +1,7 @@
-include Makefile.config
clean-files += Makefile.config
ifeq ($(ENABLE_BUILD), yes)
makefiles = \
mk/precompiled-headers.mk \
local.mk \
@@ -18,19 +22,23 @@ makefiles = \
misc/upstart/local.mk \
doc/manual/local.mk \
doc/internal-api/local.mk
endif
-include Makefile.config
ifeq ($(tests), yes)
ifeq ($(ENABLE_BUILD)_$(ENABLE_TESTS), yes_yes)
UNIT_TEST_ENV = _NIX_TEST_UNIT_DATA=unit-test-data
makefiles += \
src/libutil/tests/local.mk \
src/libstore/tests/local.mk \
src/libexpr/tests/local.mk \
tests/local.mk \
tests/ca/local.mk \
tests/dyn-drv/local.mk \
tests/test-libstoreconsumer/local.mk \
tests/plugins/local.mk
src/libexpr/tests/local.mk
endif
ifeq ($(ENABLE_TESTS), yes)
makefiles += \
tests/functional/local.mk \
tests/functional/ca/local.mk \
tests/functional/dyn-drv/local.mk \
tests/functional/test-libstoreconsumer/local.mk \
tests/functional/plugins/local.mk
else
makefiles += \
mk/disable-tests.mk
+7 -2
View File
@@ -8,15 +8,19 @@ CXX = @CXX@
CXXFLAGS = @CXXFLAGS@
CXXLTO = @CXXLTO@
EDITLINE_LIBS = @EDITLINE_LIBS@
ENABLE_BUILD = @ENABLE_BUILD@
ENABLE_S3 = @ENABLE_S3@
ENABLE_TESTS = @ENABLE_TESTS@
GTEST_LIBS = @GTEST_LIBS@
HAVE_LIBCPUID = @HAVE_LIBCPUID@
HAVE_SECCOMP = @HAVE_SECCOMP@
HOST_OS = @host_os@
INSTALL_UNIT_TESTS = @INSTALL_UNIT_TESTS@
LDFLAGS = @LDFLAGS@
LIBARCHIVE_LIBS = @LIBARCHIVE_LIBS@
LIBBROTLI_LIBS = @LIBBROTLI_LIBS@
LIBCURL_LIBS = @LIBCURL_LIBS@
LIBGIT2_LIBS = @LIBGIT2_LIBS@
LIBSECCOMP_LIBS = @LIBSECCOMP_LIBS@
LOWDOWN_LIBS = @LOWDOWN_LIBS@
OPENSSL_LIBS = @OPENSSL_LIBS@
@@ -28,6 +32,8 @@ SODIUM_LIBS = @SODIUM_LIBS@
SQLITE3_LIBS = @SQLITE3_LIBS@
bash = @bash@
bindir = @bindir@
checkbindir = @checkbindir@
checklibdir = @checklibdir@
datadir = @datadir@
datarootdir = @datarootdir@
doc_generate = @doc_generate@
@@ -35,6 +41,7 @@ docdir = @docdir@
embedded_sandbox_shell = @embedded_sandbox_shell@
exec_prefix = @exec_prefix@
includedir = @includedir@
internal_api_docs = @internal_api_docs@
libdir = @libdir@
libexecdir = @libexecdir@
localstatedir = @localstatedir@
@@ -46,5 +53,3 @@ sandbox_shell = @sandbox_shell@
storedir = @storedir@
sysconfdir = @sysconfdir@
system = @system@
tests = @tests@
internal_api_docs = @internal_api_docs@
+8 -10
View File
@@ -7,21 +7,20 @@ Nix is a powerful package manager for Linux and other Unix systems that makes pa
management reliable and reproducible. Please refer to the [Nix manual](https://nixos.org/nix/manual)
for more details.
## Installation
## Installation and first steps
On Linux and macOS the easiest way to install Nix is to run the following shell command
(as a user other than root):
Visit [nix.dev](https://nix.dev) for [installation instructions](https://nix.dev/tutorials/install-nix) and [beginner tutorials](https://nix.dev/tutorials/first-steps).
```console
$ curl -L https://nixos.org/nix/install | sh
```
Information on additional installation methods is available on the [Nix download page](https://nixos.org/download.html).
Full reference documentation can be found in the [Nix manual](https://nixos.org/nix/manual).
## Building And Developing
See our [Hacking guide](https://nixos.org/manual/nix/unstable/contributing/hacking.html) in our manual for instruction on how to
to set up a development environment and build Nix from source.
set up a development environment and build Nix from source.
## Contributing
Check the [contributing guide](./CONTRIBUTING.md) if you want to get involved with developing Nix.
## Additional Resources
@@ -29,7 +28,6 @@ to set up a development environment and build Nix from source.
- [Nix jobsets on hydra.nixos.org](https://hydra.nixos.org/project/nix)
- [NixOS Discourse](https://discourse.nixos.org/)
- [Matrix - #nix:nixos.org](https://matrix.to/#/#nix:nixos.org)
- [IRC - #nixos on libera.chat](irc://irc.libera.chat/#nixos)
## License
+8 -2
View File
@@ -59,12 +59,18 @@ index b5d71e62..aed7b0bf 100644
GC_bool found_me = FALSE;
size_t nthreads = 0;
int i;
@@ -851,6 +853,31 @@ GC_INNER void GC_push_all_stacks(void)
@@ -851,6 +853,37 @@ GC_INNER void GC_push_all_stacks(void)
hi = p->altstack + p->altstack_size;
/* FIXME: Need to scan the normal stack too, but how ? */
/* FIXME: Assume stack grows down */
+ } else {
+ if (pthread_getattr_np(p->id, &pattr)) {
+#ifdef HAVE_PTHREAD_ATTR_GET_NP
+ if (!pthread_attr_init(&pattr)
+ || !pthread_attr_get_np(p->id, &pattr))
+#else /* HAVE_PTHREAD_GETATTR_NP */
+ if (pthread_getattr_np(p->id, &pattr))
+#endif
+ {
+ ABORT("GC_push_all_stacks: pthread_getattr_np failed!");
+ }
+ if (pthread_attr_getstacksize(&pattr, &stack_limit)) {
-4
View File
@@ -1,4 +0,0 @@
#! /bin/sh -e
rm -f aclocal.m4
mkdir -p config
exec autoreconf -vfi
+29 -3
View File
@@ -68,6 +68,9 @@ case "$host_os" in
esac
ENSURE_NO_GCC_BUG_80431
# Check for pubsetbuf.
AC_MSG_CHECKING([for pubsetbuf])
AC_LANG_PUSH(C++)
@@ -152,12 +155,29 @@ if test "x$GCC_ATOMIC_BUILTINS_NEED_LIBATOMIC" = xyes; then
LDFLAGS="-latomic $LDFLAGS"
fi
# Running the functional tests without building Nix is useful for testing
# different pre-built versions of Nix against each other.
AC_ARG_ENABLE(build, AS_HELP_STRING([--disable-build],[Do not build nix]),
ENABLE_BUILD=$enableval, ENABLE_BUILD=yes)
AC_SUBST(ENABLE_BUILD)
# Building without tests is useful for bootstrapping with a smaller footprint
# or running the tests in a separate derivation. Otherwise, we do compile and
# run them.
AC_ARG_ENABLE(tests, AS_HELP_STRING([--disable-tests],[Do not build the tests]),
tests=$enableval, tests=yes)
AC_SUBST(tests)
ENABLE_TESTS=$enableval, ENABLE_TESTS=yes)
AC_SUBST(ENABLE_TESTS)
AC_ARG_ENABLE(install-unit-tests, AS_HELP_STRING([--enable-install-unit-tests],[Install the unit tests for running later (default no)]),
INSTALL_UNIT_TESTS=$enableval, INSTALL_UNIT_TESTS=no)
AC_SUBST(INSTALL_UNIT_TESTS)
AC_ARG_WITH(check-bin-dir, AS_HELP_STRING([--with-check-bin-dir=PATH],[path to install unit tests for running later (defaults to $libexecdir/nix)]),
checkbindir=$withval, checkbindir=$libexecdir/nix)
AC_SUBST(checkbindir)
AC_ARG_WITH(check-lib-dir, AS_HELP_STRING([--with-check-lib-dir=PATH],[path to install unit tests for running later (defaults to $libdir)]),
checklibdir=$withval, checklibdir=$libdir)
AC_SUBST(checklibdir)
# Building without API docs is the default as Nix' C++ interfaces are internal and unstable.
AC_ARG_ENABLE(internal_api_docs, AS_HELP_STRING([--enable-internal-api-docs],[Build API docs for Nix's internal unstable C++ interfaces]),
@@ -289,7 +309,7 @@ if test "$gc" = yes; then
fi
if test "$tests" = yes; then
if test "$ENABLE_TESTS" = yes; then
# Look for gtest.
PKG_CHECK_MODULES([GTEST], [gtest_main])
@@ -327,9 +347,15 @@ AC_ARG_ENABLE(doc-gen, AS_HELP_STRING([--disable-doc-gen],[disable documentation
doc_generate=$enableval, doc_generate=yes)
AC_SUBST(doc_generate)
# Look for lowdown library.
PKG_CHECK_MODULES([LOWDOWN], [lowdown >= 0.9.0], [CXXFLAGS="$LOWDOWN_CFLAGS $CXXFLAGS"])
# Look for libgit2.
PKG_CHECK_MODULES([LIBGIT2], [libgit2])
# Setuid installations.
AC_CHECK_FUNCS([setresuid setreuid lchown])
+30
View File
@@ -54,6 +54,23 @@ INPUT = \
src/nix-env \
src/nix-store
# If the MACRO_EXPANSION tag is set to YES, doxygen will expand all macro names
# in the source code. If set to NO, only conditional compilation will be
# performed. Macro expansion can be done in a controlled way by setting
# EXPAND_ONLY_PREDEF to YES.
# The default value is: NO.
# This tag requires that the tag ENABLE_PREPROCESSING is set to YES.
MACRO_EXPANSION = YES
# If the EXPAND_ONLY_PREDEF and MACRO_EXPANSION tags are both set to YES then
# the macro expansion is limited to the macros specified with the PREDEFINED and
# EXPAND_AS_DEFINED tags.
# The default value is: NO.
# This tag requires that the tag ENABLE_PREPROCESSING is set to YES.
EXPAND_ONLY_PREDEF = YES
# The INCLUDE_PATH tag can be used to specify one or more directories that
# contain include files that are not input files but should be processed by the
# preprocessor. Note that the INCLUDE_PATH is not recursive, so the setting of
@@ -61,3 +78,16 @@ INPUT = \
# This tag requires that the tag SEARCH_INCLUDES is set to YES.
INCLUDE_PATH = @RAPIDCHECK_HEADERS@
# If the MACRO_EXPANSION and EXPAND_ONLY_PREDEF tags are set to YES then this
# tag can be used to specify a list of macro names that should be expanded. The
# macro definition that is found in the sources will be used. Use the PREDEFINED
# tag if you want to use a different macro definition that overrules the
# definition found in the source code.
# This tag requires that the tag ENABLE_PREPROCESSING is set to YES.
EXPAND_AS_DEFINED = \
DECLARE_COMMON_SERIALISER \
DECLARE_WORKER_SERIALISER \
DECLARE_SERVE_SERIALISER \
LENGTH_PREFIXED_PROTO_HELPER
+30
View File
@@ -0,0 +1,30 @@
# redirect rules for paths (server-side) to prevent link rot.
# see ./redirects.js for redirects based on URL fragments (client-side)
#
# concrete user story this supports:
# - user finds URL to the manual for Nix x.y
# - Nix x.z (z > y) is the most recent release
# - updating the version in the URL will show the right thing
#
# format documentation:
# - https://docs.netlify.com/routing/redirects/#syntax-for-the-redirects-file
# - https://docs.netlify.com/routing/redirects/redirect-options/
#
# conventions:
# - always force (<CODE>!) since this allows re-using file names
# - group related paths to ease readability
# - always append new redirects to the end of the file
# - redirects that should have been there but are missing can be inserted where they belong
/expressions/expression-language /language/ 301!
/expressions/language-values /language/values 301!
/expressions/language-constructs /language/constructs 301!
/expressions/language-operators /language/operators 301!
/expressions/* /language/:splat 301!
/package-management/basic-package-mgmt /command-ref/nix-env 301!
/package-management/channels* /command-ref/nix-channel 301!
/package-management/s3-substituter* /command-ref/new-cli/nix3-help-stores#s3-binary-cache-store 301!
+45 -58
View File
@@ -1,11 +1,12 @@
let
inherit (builtins)
attrNames attrValues fromJSON listToAttrs mapAttrs
attrNames attrValues fromJSON listToAttrs mapAttrs groupBy
concatStringsSep concatMap length lessThan replaceStrings sort;
inherit (import ./utils.nix) concatStrings optionalString filterAttrs trim squash unique showSettings;
inherit (import <nix/utils.nix>) attrsToList concatStrings optionalString filterAttrs trim squash unique;
showStoreDocs = import ./generate-store-info.nix;
in
commandDump:
inlineHTML: commandDump:
let
@@ -30,7 +31,7 @@ let
${maybeSubcommands}
${maybeDocumentation}
${maybeStoreDocs}
${maybeOptions}
'';
@@ -40,15 +41,15 @@ let
showArgument = arg: "*${arg.label}*" + optionalString (! arg ? arity) "...";
arguments = concatStringsSep " " (map showArgument args);
in ''
`${command}` [*option*...] ${arguments}
`${command}` [*option*...] ${arguments}
'';
maybeSubcommands = optionalString (details ? commands && details.commands != {})
''
where *subcommand* is one of the following:
''
where *subcommand* is one of the following:
${subcommands}
'';
${subcommands}
'';
subcommands = if length categories > 1
then listCategories
@@ -70,40 +71,57 @@ let
* [`${command} ${name}`](./${appendName filename name}.md) - ${subcmd.description}
'';
maybeDocumentation = optionalString
(details ? doc)
(replaceStrings ["@stores@"] [storeDocs] details.doc);
# FIXME: this is a hack.
# store parameters should not be part of command documentation to begin
# with, but instead be rendered on separate pages.
maybeStoreDocs = optionalString (details ? doc)
(replaceStrings [ "@stores@" ] [ (showStoreDocs inlineHTML commandInfo.stores) ] details.doc);
maybeOptions = optionalString (details.flags != {}) ''
maybeOptions = let
allVisibleOptions = filterAttrs
(_: o: ! o.hiddenCategory)
(details.flags // toplevel.flags);
in optionalString (allVisibleOptions != {}) ''
# Options
${showOptions details.flags toplevel.flags}
${showOptions inlineHTML allVisibleOptions}
> **Note**
>
> See [`man nix.conf`](@docroot@/command-ref/conf-file.md#command-line-flags) for overriding configuration settings with command line flags.
'';
showOptions = options: commonOptions:
showOptions = inlineHTML: allOptions:
let
allOptions = options // commonOptions;
showCategory = cat: ''
${optionalString (cat != "") "**${cat}:**"}
showCategory = cat: opts: ''
${optionalString (cat != "") "## ${cat}"}
${listOptions (filterAttrs (n: v: v.category == cat) allOptions)}
${concatStringsSep "\n" (attrValues (mapAttrs showOption opts))}
'';
listOptions = opts: concatStringsSep "\n" (attrValues (mapAttrs showOption opts));
showOption = name: option:
let
result = trim ''
- ${item}
${option.description}
'';
item = if inlineHTML
then ''<span id="opt-${name}">[`--${name}`](#opt-${name})</span> ${shortName} ${labels}''
else "`--${name}` ${shortName} ${labels}";
shortName = optionalString
(option ? shortName)
("/ `-${option.shortName}`");
labels = optionalString
(option ? labels)
(concatStringsSep " " (map (s: "*${s}*") option.labels));
in trim ''
- <span id="opt-${name}">[`--${name}`](#opt-${name})</span> ${shortName} ${labels}
${option.description}
'';
categories = sort lessThan (unique (map (cmd: cmd.category) (attrValues allOptions)));
in concatStrings (map showCategory categories);
in result;
categories = mapAttrs
# Convert each group from a list of key-value pairs back to an attrset
(_: listToAttrs)
(groupBy
(cmd: cmd.value.category)
(attrsToList allOptions));
in concatStrings (attrValues (mapAttrs showCategory categories));
in squash result;
appendName = filename: name: (if filename == "nix" then "nix3" else filename) + "-" + name;
@@ -135,35 +153,4 @@ let
" - [${page.command}](command-ref/new-cli/${page.name})";
in concatStringsSep "\n" (map showEntry manpages) + "\n";
storeDocs =
let
showStore = name: { settings, doc, experimentalFeature }:
let
experimentalFeatureNote = optionalString (experimentalFeature != null) ''
> **Warning**
> This store is part of an
> [experimental feature](@docroot@/contributing/experimental-features.md).
To use this store, you need to make sure the corresponding experimental feature,
[`${experimentalFeature}`](@docroot@/contributing/experimental-features.md#xp-feature-${experimentalFeature}),
is enabled.
For example, include the following in [`nix.conf`](#):
```
extra-experimental-features = ${experimentalFeature}
```
'';
in ''
## ${name}
${doc}
${experimentalFeatureNote}
**Settings**:
${showSettings { useAnchors = false; } settings}
'';
in concatStrings (attrValues (mapAttrs showStore commandInfo.stores));
in (listToAttrs manpages) // { "SUMMARY.md" = tableOfContents; }
+66
View File
@@ -0,0 +1,66 @@
let
inherit (builtins) attrValues concatStringsSep isAttrs isBool mapAttrs;
inherit (import ./utils.nix) concatStrings indent optionalString squash;
in
# `inlineHTML` is a hack to accommodate inconsistent output from `lowdown`
{ prefix, inlineHTML ? true }: settingsInfo:
let
showSetting = prefix: setting: { description, documentDefault, defaultValue, aliases, value, experimentalFeature }:
let
result = squash ''
- ${item}
${indent " " body}
'';
item = if inlineHTML
then ''<span id="${prefix}-${setting}">[`${setting}`](#${prefix}-${setting})</span>''
else "`${setting}`";
# separate body to cleanly handle indentation
body = ''
${description}
${experimentalFeatureNote}
**Default:** ${showDefault documentDefault defaultValue}
${showAliases aliases}
'';
experimentalFeatureNote = optionalString (experimentalFeature != null) ''
> **Warning**
> This setting is part of an
> [experimental feature](@docroot@/contributing/experimental-features.md).
To change this setting, you need to make sure the corresponding experimental feature,
[`${experimentalFeature}`](@docroot@/contributing/experimental-features.md#xp-feature-${experimentalFeature}),
is enabled.
For example, include the following in [`nix.conf`](#):
```
extra-experimental-features = ${experimentalFeature}
${setting} = ...
```
'';
showDefault = documentDefault: defaultValue:
if documentDefault then
# a StringMap value type is specified as a string, but
# this shows the value type. The empty stringmap is `null` in
# JSON, but that converts to `{ }` here.
if defaultValue == "" || defaultValue == [] || isAttrs defaultValue
then "*empty*"
else if isBool defaultValue then
if defaultValue then "`true`" else "`false`"
else "`${toString defaultValue}`"
else "*machine-specific*";
showAliases = aliases:
optionalString (aliases != [])
"**Deprecated alias:** ${(concatStringsSep ", " (map (s: "`${s}`") aliases))}";
in result;
in concatStrings (attrValues (mapAttrs (showSetting prefix) settingsInfo))
+45
View File
@@ -0,0 +1,45 @@
let
inherit (builtins) attrValues mapAttrs;
inherit (import ./utils.nix) concatStrings optionalString;
showSettings = import ./generate-settings.nix;
in
inlineHTML: storesInfo:
let
showStore = name: { settings, doc, experimentalFeature }:
let
result = ''
## ${name}
${doc}
${experimentalFeatureNote}
### Settings
${showSettings { prefix = "store-${slug}"; inherit inlineHTML; } settings}
'';
# markdown doesn't like spaces in URLs
slug = builtins.replaceStrings [ " " ] [ "-" ] name;
experimentalFeatureNote = optionalString (experimentalFeature != null) ''
> **Warning**
> This store is part of an
> [experimental feature](@docroot@/contributing/experimental-features.md).
To use this store, you need to make sure the corresponding experimental feature,
[`${experimentalFeature}`](@docroot@/contributing/experimental-features.md#xp-feature-${experimentalFeature}),
is enabled.
For example, include the following in [`nix.conf`](#):
```
extra-experimental-features = ${experimentalFeature}
```
'';
in result;
in concatStrings (attrValues (mapAttrs showStore storesInfo))
+34 -12
View File
@@ -24,7 +24,7 @@ man-pages += $(foreach subcommand, \
clean-files += $(d)/*.1 $(d)/*.5 $(d)/*.8
# Provide a dummy environment for nix, so that it will not access files outside the macOS sandbox.
# Set cores to 0 because otherwise nix show-config resolves the cores based on the current machine
# Set cores to 0 because otherwise `nix config show` resolves the cores based on the current machine
dummy-env = env -i \
HOME=/dummy \
NIX_CONF_DIR=/dummy \
@@ -32,7 +32,7 @@ dummy-env = env -i \
NIX_STATE_DIR=/dummy \
NIX_CONFIG='cores = 0'
nix-eval = $(dummy-env) $(bindir)/nix eval --experimental-features nix-command -I nix/corepkgs=corepkgs --store dummy:// --impure --raw
nix-eval = $(dummy-env) $(bindir)/nix eval --experimental-features nix-command -I nix=doc/manual --store dummy:// --impure --raw
# re-implement mdBook's include directive to make it usable for terminal output and for proper @docroot@ substitution
define process-includes
@@ -92,18 +92,18 @@ $(d)/nix-profiles.5: $(d)/src/command-ref/files/profiles.md
$(trace-gen) lowdown -sT man --nroff-nolinks -M section=5 $^.tmp -o $@
@rm $^.tmp
$(d)/src/SUMMARY.md: $(d)/src/SUMMARY.md.in $(d)/src/command-ref/new-cli $(d)/src/contributing/experimental-feature-descriptions.md
$(d)/src/SUMMARY.md: $(d)/src/SUMMARY.md.in $(d)/src/SUMMARY-rl-next.md $(d)/src/command-ref/new-cli $(d)/src/contributing/experimental-feature-descriptions.md
@cp $< $@
@$(call process-includes,$@,$@)
$(d)/src/command-ref/new-cli: $(d)/nix.json $(d)/utils.nix $(d)/generate-manpage.nix $(bindir)/nix
$(d)/src/command-ref/new-cli: $(d)/nix.json $(d)/utils.nix $(d)/generate-manpage.nix $(d)/generate-settings.nix $(d)/generate-store-info.nix $(bindir)/nix
@rm -rf $@ $@.tmp
$(trace-gen) $(nix-eval) --write-to $@.tmp --expr 'import doc/manual/generate-manpage.nix (builtins.readFile $<)'
$(trace-gen) $(nix-eval) --write-to $@.tmp --expr 'import doc/manual/generate-manpage.nix true (builtins.readFile $<)'
@mv $@.tmp $@
$(d)/src/command-ref/conf-file.md: $(d)/conf-file.json $(d)/utils.nix $(d)/src/command-ref/conf-file-prefix.md $(d)/src/command-ref/experimental-features-shortlist.md $(bindir)/nix
$(d)/src/command-ref/conf-file.md: $(d)/conf-file.json $(d)/utils.nix $(d)/generate-settings.nix $(d)/src/command-ref/conf-file-prefix.md $(d)/src/command-ref/experimental-features-shortlist.md $(bindir)/nix
@cat doc/manual/src/command-ref/conf-file-prefix.md > $@.tmp
$(trace-gen) $(nix-eval) --expr '(import doc/manual/utils.nix).showSettings { useAnchors = true; } (builtins.fromJSON (builtins.readFile $<))' >> $@.tmp;
$(trace-gen) $(nix-eval) --expr 'import doc/manual/generate-settings.nix { prefix = "conf"; } (builtins.fromJSON (builtins.readFile $<))' >> $@.tmp;
@mv $@.tmp $@
$(d)/nix.json: $(bindir)/nix
@@ -111,7 +111,7 @@ $(d)/nix.json: $(bindir)/nix
@mv $@.tmp $@
$(d)/conf-file.json: $(bindir)/nix
$(trace-gen) $(dummy-env) $(bindir)/nix show-config --json --experimental-features nix-command > $@.tmp
$(trace-gen) $(dummy-env) $(bindir)/nix config show --json --experimental-features nix-command > $@.tmp
@mv $@.tmp $@
$(d)/src/contributing/experimental-feature-descriptions.md: $(d)/xp-features.json $(d)/utils.nix $(d)/generate-xp-features.nix $(bindir)/nix
@@ -125,7 +125,7 @@ $(d)/src/command-ref/experimental-features-shortlist.md: $(d)/xp-features.json $
@mv $@.tmp $@
$(d)/xp-features.json: $(bindir)/nix
$(trace-gen) $(dummy-env) NIX_PATH=nix/corepkgs=corepkgs $(bindir)/nix __dump-xp-features > $@.tmp
$(trace-gen) $(dummy-env) $(bindir)/nix __dump-xp-features > $@.tmp
@mv $@.tmp $@
$(d)/src/language/builtins.md: $(d)/language.json $(d)/generate-builtins.nix $(d)/src/language/builtins-prefix.md $(bindir)/nix
@@ -141,9 +141,27 @@ $(d)/src/language/builtin-constants.md: $(d)/language.json $(d)/generate-builtin
@mv $@.tmp $@
$(d)/language.json: $(bindir)/nix
$(trace-gen) $(dummy-env) NIX_PATH=nix/corepkgs=corepkgs $(bindir)/nix __dump-language > $@.tmp
$(trace-gen) $(dummy-env) $(bindir)/nix __dump-language > $@.tmp
@mv $@.tmp $@
# Generate "Upcoming release" notes (or clear it and remove from menu)
$(d)/src/release-notes/rl-next.md: $(d)/rl-next $(d)/rl-next/*
@if type -p changelog-d > /dev/null; then \
echo " GEN " $@; \
changelog-d doc/manual/rl-next > $@; \
else \
echo " NULL " $@; \
true > $@; \
fi
$(d)/src/SUMMARY-rl-next.md: $(d)/src/release-notes/rl-next.md
$(trace-gen) true
@if [ -s $< ]; then \
echo ' - [Upcoming release](release-notes/rl-next.md)' > $@; \
else \
true > $@; \
fi
# Generate the HTML manual.
.PHONY: manual-html
manual-html: $(docdir)/manual/index.html
@@ -173,14 +191,18 @@ doc/manual/generated/man1/nix3-manpages: $(d)/src/command-ref/new-cli
done
@touch $@
$(docdir)/manual/index.html: $(MANUAL_SRCS) $(d)/book.toml $(d)/anchors.jq $(d)/custom.css $(d)/src/SUMMARY.md $(d)/src/command-ref/new-cli $(d)/src/contributing/experimental-feature-descriptions.md $(d)/src/command-ref/conf-file.md $(d)/src/language/builtins.md $(d)/src/language/builtin-constants.md
# the `! -name 'contributing.md'` filter excludes the one place where
# `@docroot@` is to be preserved for documenting the mechanism
# FIXME: maybe contributing guides should live right next to the code
# instead of in the manual
$(docdir)/manual/index.html: $(MANUAL_SRCS) $(d)/book.toml $(d)/anchors.jq $(d)/custom.css $(d)/src/SUMMARY.md $(d)/src/command-ref/new-cli $(d)/src/contributing/experimental-feature-descriptions.md $(d)/src/command-ref/conf-file.md $(d)/src/language/builtins.md $(d)/src/language/builtin-constants.md $(d)/src/release-notes/rl-next.md
$(trace-gen) \
tmp="$$(mktemp -d)"; \
cp -r doc/manual "$$tmp"; \
find "$$tmp" -name '*.md' | while read -r file; do \
$(call process-includes,$$file,$$file); \
done; \
find "$$tmp" -name '*.md' | while read -r file; do \
find "$$tmp" -name '*.md' ! -name 'documentation.md' | while read -r file; do \
docroot="$$(realpath --relative-to="$$(dirname "$$file")" $$tmp/manual/src)"; \
sed -i "s,@docroot@,$$docroot,g" "$$file"; \
done; \
+7 -5
View File
@@ -1,7 +1,9 @@
// redirect rules for anchors ensure backwards compatibility of URLs.
// this must be done on the client side, as web servers do not see the anchor part of the URL.
// redirect rules for URL fragments (client-side) to prevent link rot.
// this must be done on the client side, as web servers do not see the fragment part of the URL.
// it will only work with JavaScript enabled in the browser, but this is the best we can do here.
// see ./_redirects for path redirects (client-side)
// redirections are declared as follows:
// redirects are declared as follows:
// each entry has as its key a path matching the requested URL path, relative to the mdBook document root.
//
// IMPORTANT: it must specify the full path with file name and suffix
@@ -336,14 +338,13 @@ const redirects = {
"simple-values": "#primitives",
"lists": "#list",
"strings": "#string",
"lists": "#list",
"attribute-sets": "#attribute-set",
},
"installation/installing-binary.html": {
"linux": "uninstall.html#linux",
"macos": "uninstall.html#macos",
"uninstalling": "uninstall.html",
}
},
"contributing/hacking.html": {
"nix-with-flakes": "#building-nix-with-flakes",
"classic-nix": "#building-nix",
@@ -355,6 +356,7 @@ const redirects = {
"installer-tests": "testing.html#installer-tests",
"one-time-setup": "testing.html#one-time-setup",
"using-the-ci-generated-installer-for-manual-testing": "testing.html#using-the-ci-generated-installer-for-manual-testing",
"characterization-testing": "#characterisation-testing-unit",
}
};
+2
View File
@@ -0,0 +1,2 @@
organization: NixOS
repository: nix
+9
View File
@@ -0,0 +1,9 @@
synopsis: Mounted SSH Store
issues: #7890
prs: #7912
description: {
Introduced the store [`mounted-ssh-ng://`](@docroot@/command-ref/new-cli/nix3-help-stores.md).
This store allows full access to a Nix store on a remote machine and additionally requires that the store be mounted in the local filesystem.
}
+8
View File
@@ -0,0 +1,8 @@
synopsis: `nix config show`
issues: #7672
prs: #9477
description: {
`nix show-config` was renamed to `nix config show` to be more consistent with the rest of the command-line
}
@@ -0,0 +1,9 @@
synopsis: Fix `nix-env --query --drv-path --json`
prs: #9257
description: {
Fixed a bug where `nix-env --query` ignored `--drv-path` when `--json` was set.
}
+19 -13
View File
@@ -16,26 +16,31 @@
- [Environment Variables](installation/env-variables.md)
- [Upgrading Nix](installation/upgrading.md)
- [Uninstalling Nix](installation/uninstall.md)
- [Nix Store](store/index.md)
- [File System Object](store/file-system-object.md)
- [Store Object](store/store-object.md)
- [Store Path](store/store-path.md)
- [Nix Language](language/index.md)
- [Data Types](language/values.md)
- [Language Constructs](language/constructs.md)
- [String interpolation](language/string-interpolation.md)
- [Lookup path](language/constructs/lookup-path.md)
- [Operators](language/operators.md)
- [Derivations](language/derivations.md)
- [Advanced Attributes](language/advanced-attributes.md)
- [Import From Derivation](language/import-from-derivation.md)
- [Built-in Constants](language/builtin-constants.md)
- [Built-in Functions](language/builtins.md)
- [Package Management](package-management/package-management.md)
- [Basic Package Management](package-management/basic-package-mgmt.md)
- [Profiles](package-management/profiles.md)
- [Garbage Collection](package-management/garbage-collection.md)
- [Garbage Collector Roots](package-management/garbage-collector-roots.md)
- [Advanced Topics](advanced-topics/advanced-topics.md)
- [Sharing Packages Between Machines](package-management/sharing-packages.md)
- [Serving a Nix store via HTTP](package-management/binary-cache-substituter.md)
- [Copying Closures via SSH](package-management/copy-closure.md)
- [Serving a Nix store via SSH](package-management/ssh-substituter.md)
- [Serving a Nix store via S3](package-management/s3-substituter.md)
- [Nix Language](language/index.md)
- [Data Types](language/values.md)
- [Language Constructs](language/constructs.md)
- [String interpolation](language/string-interpolation.md)
- [Operators](language/operators.md)
- [Derivations](language/derivations.md)
- [Advanced Attributes](language/advanced-attributes.md)
- [Built-in Constants](language/builtin-constants.md)
- [Built-in Functions](language/builtins.md)
- [Advanced Topics](advanced-topics/advanced-topics.md)
- [Remote Builds](advanced-topics/distributed-builds.md)
- [Tuning Cores and Jobs](advanced-topics/cores-vs-jobs.md)
- [Verifying Build Reproducibility](advanced-topics/diff-hook.md)
@@ -97,7 +102,6 @@
- [Channels](command-ref/files/channels.md)
- [Default Nix expression](command-ref/files/default-nix-expression.md)
- [Architecture and Design](architecture/architecture.md)
- [File System Object](architecture/file-system-object.md)
- [Protocols](protocols/protocols.md)
- [Serving Tarball Flakes](protocols/tarball-fetcher.md)
- [Derivation "ATerm" file format](protocols/derivation-aterm.md)
@@ -105,11 +109,13 @@
- [Contributing](contributing/contributing.md)
- [Hacking](contributing/hacking.md)
- [Testing](contributing/testing.md)
- [Documentation](contributing/documentation.md)
- [Experimental Features](contributing/experimental-features.md)
- [CLI guideline](contributing/cli-guideline.md)
- [C++ style guide](contributing/cxx.md)
- [Release Notes](release-notes/release-notes.md)
- [Release X.Y (202?-??-??)](release-notes/rl-next.md)
{{#include ./SUMMARY-rl-next.md}}
- [Release 2.19 (2023-11-17)](release-notes/rl-2.19.md)
- [Release 2.18 (2023-09-20)](release-notes/rl-2.18.md)
- [Release 2.17 (2023-07-24)](release-notes/rl-2.17.md)
- [Release 2.16 (2023-05-31)](release-notes/rl-2.16.md)
@@ -12,14 +12,14 @@ machine is accessible via SSH and that it has Nix installed. You can
test whether connecting to the remote Nix instance works, e.g.
```console
$ nix store ping --store ssh://mac
$ nix store info --store ssh://mac
```
will try to connect to the machine named `mac`. It is possible to
specify an SSH identity file as part of the remote store URI, e.g.
```console
$ nix store ping --store ssh://mac?ssh-key=/home/alice/my-key
$ nix store info --store ssh://mac?ssh-key=/home/alice/my-key
```
Since builds should be non-interactive, the key should not have a
@@ -17,9 +17,8 @@ the build loop.
# Prerequisites
This tutorial assumes you have [configured an S3-compatible binary
cache](../package-management/s3-substituter.md), and that the `root`
user's default AWS profile can upload to the bucket.
This tutorial assumes you have configured an [S3-compatible binary cache](@docroot@/command-ref/new-cli/nix3-help-stores.md#s3-binary-cache-store) as a [substituter](../command-ref/conf-file.md#conf-substituters),
and that the `root` user's default AWS profile can upload to the bucket.
# Set up a Signing Key
@@ -69,6 +68,8 @@ exec nix copy --to "s3://example-nix-cache" $OUT_PATHS
> store sign`. Nix guarantees the paths will not contain any spaces,
> however a store path might contain glob characters. The `set -f`
> disables globbing in the shell.
> If you want to upload the `.drv` file too, the `$DRV_PATH` variable
> is also defined for the script and works just like `$OUT_PATHS`.
Then make sure the hook program is executable by the `root` user:
+2 -1
View File
@@ -59,10 +59,11 @@ The [Nix language](../language/index.md) evaluator transforms Nix expressions in
The command line interface and Nix expressions are what users deal with most.
> **Note**
>
> The Nix language itself does not have a notion of *packages* or *configurations*.
> As far as we are concerned here, the inputs and results of a build plan are just data.
Underlying the command line interface and the Nix language evaluator is the [Nix store](../glossary.md#gloss-store), a mechanism to keep track of build plans, data, and references between them.
Underlying the command line interface and the Nix language evaluator is the [Nix store](../store/index.md), a mechanism to keep track of build plans, data, and references between them.
It can also execute build plans to produce new data, which are made available to the operating system as files.
A build plan itself is a series of *build tasks*, together with their build inputs.
+95 -80
View File
@@ -2,109 +2,124 @@
Most Nix commands interpret the following environment variables:
- <span id="env-IN_NIX_SHELL">[`IN_NIX_SHELL`](#env-IN_NIX_SHELL)</span>\
Indicator that tells if the current environment was set up by
`nix-shell`. It can have the values `pure` or `impure`.
- <span id="env-IN_NIX_SHELL">[`IN_NIX_SHELL`](#env-IN_NIX_SHELL)</span>
- <span id="env-NIX_PATH">[`NIX_PATH`](#env-NIX_PATH)</span>\
A colon-separated list of directories used to look up the location of Nix
expressions using [paths](@docroot@/language/values.md#type-path)
enclosed in angle brackets (i.e., `<path>`),
e.g. `/home/eelco/Dev:/etc/nixos`. It can be extended using the
[`-I` option](@docroot@/command-ref/opt-common.md#opt-I).
Indicator that tells if the current environment was set up by
`nix-shell`. It can have the values `pure` or `impure`.
If `NIX_PATH` is not set at all, Nix will fall back to the following list in [impure](@docroot@/command-ref/conf-file.md#conf-pure-eval) and [unrestricted](@docroot@/command-ref/conf-file.md#conf-restrict-eval) evaluation mode:
- <span id="env-NIX_PATH">[`NIX_PATH`](#env-NIX_PATH)</span>
1. `$HOME/.nix-defexpr/channels`
2. `nixpkgs=/nix/var/nix/profiles/per-user/root/channels/nixpkgs`
3. `/nix/var/nix/profiles/per-user/root/channels`
A colon-separated list of directories used to look up the location of Nix
expressions using [paths](@docroot@/language/values.md#type-path)
enclosed in angle brackets (i.e., `<path>`),
e.g. `/home/eelco/Dev:/etc/nixos`. It can be extended using the
[`-I` option](@docroot@/command-ref/opt-common.md#opt-I).
If `NIX_PATH` is set to an empty string, resolving search paths will always fail.
For example, attempting to use `<nixpkgs>` will produce:
If `NIX_PATH` is not set at all, Nix will fall back to the following list in [impure](@docroot@/command-ref/conf-file.md#conf-pure-eval) and [unrestricted](@docroot@/command-ref/conf-file.md#conf-restrict-eval) evaluation mode:
error: file 'nixpkgs' was not found in the Nix search path
1. `$HOME/.nix-defexpr/channels`
2. `nixpkgs=/nix/var/nix/profiles/per-user/root/channels/nixpkgs`
3. `/nix/var/nix/profiles/per-user/root/channels`
- <span id="env-NIX_IGNORE_SYMLINK_STORE">[`NIX_IGNORE_SYMLINK_STORE`](#env-NIX_IGNORE_SYMLINK_STORE)</span>\
Normally, the Nix store directory (typically `/nix/store`) is not
allowed to contain any symlink components. This is to prevent
“impure” builds. Builders sometimes “canonicalise” paths by
resolving all symlink components. Thus, builds on different machines
(with `/nix/store` resolving to different locations) could yield
different results. This is generally not a problem, except when
builds are deployed to machines where `/nix/store` resolves
differently. If you are sure that you’re not going to do that, you
can set `NIX_IGNORE_SYMLINK_STORE` to `1`.
If `NIX_PATH` is set to an empty string, resolving search paths will always fail.
For example, attempting to use `<nixpkgs>` will produce:
Note that if you’re symlinking the Nix store so that you can put it
on another file system than the root file system, on Linux you’re
better off using `bind` mount points, e.g.,
error: file 'nixpkgs' was not found in the Nix search path
```console
$ mkdir /nix
$ mount -o bind /mnt/otherdisk/nix /nix
```
- <span id="env-NIX_IGNORE_SYMLINK_STORE">[`NIX_IGNORE_SYMLINK_STORE`](#env-NIX_IGNORE_SYMLINK_STORE)</span>
Consult the mount 8 manual page for details.
Normally, the Nix store directory (typically `/nix/store`) is not
allowed to contain any symlink components. This is to prevent
“impure” builds. Builders sometimes “canonicalise” paths by
resolving all symlink components. Thus, builds on different machines
(with `/nix/store` resolving to different locations) could yield
different results. This is generally not a problem, except when
builds are deployed to machines where `/nix/store` resolves
differently. If you are sure that you’re not going to do that, you
can set `NIX_IGNORE_SYMLINK_STORE` to `1`.
- <span id="env-NIX_STORE_DIR">[`NIX_STORE_DIR`](#env-NIX_STORE_DIR)</span>\
Overrides the location of the Nix store (default `prefix/store`).
Note that if you’re symlinking the Nix store so that you can put it
on another file system than the root file system, on Linux you’re
better off using `bind` mount points, e.g.,
- <span id="env-NIX_DATA_DIR">[`NIX_DATA_DIR`](#env-NIX_DATA_DIR)</span>\
Overrides the location of the Nix static data directory (default
`prefix/share`).
```console
$ mkdir /nix
$ mount -o bind /mnt/otherdisk/nix /nix
```
- <span id="env-NIX_LOG_DIR">[`NIX_LOG_DIR`](#env-NIX_LOG_DIR)</span>\
Overrides the location of the Nix log directory (default
`prefix/var/log/nix`).
Consult the mount 8 manual page for details.
- <span id="env-NIX_STATE_DIR">[`NIX_STATE_DIR`](#env-NIX_STATE_DIR)</span>\
Overrides the location of the Nix state directory (default
`prefix/var/nix`).
- <span id="env-NIX_STORE_DIR">[`NIX_STORE_DIR`](#env-NIX_STORE_DIR)</span>
- <span id="env-NIX_CONF_DIR">[`NIX_CONF_DIR`](#env-NIX_CONF_DIR)</span>\
Overrides the location of the system Nix configuration directory
(default `prefix/etc/nix`).
Overrides the location of the Nix store (default `prefix/store`).
- <span id="env-NIX_CONFIG">[`NIX_CONFIG`](#env-NIX_CONFIG)</span>\
Applies settings from Nix configuration from the environment.
The content is treated as if it was read from a Nix configuration file.
Settings are separated by the newline character.
- <span id="env-NIX_DATA_DIR">[`NIX_DATA_DIR`](#env-NIX_DATA_DIR)</span>
- <span id="env-NIX_USER_CONF_FILES">[`NIX_USER_CONF_FILES`](#env-NIX_USER_CONF_FILES)</span>\
Overrides the location of the Nix user configuration files to load from.
Overrides the location of the Nix static data directory (default
`prefix/share`).
The default are the locations according to the [XDG Base Directory Specification].
See the [XDG Base Directories](#xdg-base-directories) sub-section for details.
- <span id="env-NIX_LOG_DIR">[`NIX_LOG_DIR`](#env-NIX_LOG_DIR)</span>
The variable is treated as a list separated by the `:` token.
Overrides the location of the Nix log directory (default
`prefix/var/log/nix`).
- <span id="env-TMPDIR">[`TMPDIR`](#env-TMPDIR)</span>\
Use the specified directory to store temporary files. In particular,
this includes temporary build directories; these can take up
substantial amounts of disk space. The default is `/tmp`.
- <span id="env-NIX_STATE_DIR">[`NIX_STATE_DIR`](#env-NIX_STATE_DIR)</span>
- <span id="env-NIX_REMOTE">[`NIX_REMOTE`](#env-NIX_REMOTE)</span>\
This variable should be set to `daemon` if you want to use the Nix
daemon to execute Nix operations. This is necessary in [multi-user
Nix installations](@docroot@/installation/multi-user.md). If the Nix
daemon's Unix socket is at some non-standard path, this variable
should be set to `unix://path/to/socket`. Otherwise, it should be
left unset.
Overrides the location of the Nix state directory (default
`prefix/var/nix`).
- <span id="env-NIX_SHOW_STATS">[`NIX_SHOW_STATS`](#env-NIX_SHOW_STATS)</span>\
If set to `1`, Nix will print some evaluation statistics, such as
the number of values allocated.
- <span id="env-NIX_CONF_DIR">[`NIX_CONF_DIR`](#env-NIX_CONF_DIR)</span>
- <span id="env-NIX_COUNT_CALLS">[`NIX_COUNT_CALLS`](#env-NIX_COUNT_CALLS)</span>\
If set to `1`, Nix will print how often functions were called during
Nix expression evaluation. This is useful for profiling your Nix
expressions.
Overrides the location of the system Nix configuration directory
(default `prefix/etc/nix`).
- <span id="env-GC_INITIAL_HEAP_SIZE">[`GC_INITIAL_HEAP_SIZE`](#env-GC_INITIAL_HEAP_SIZE)</span>\
If Nix has been configured to use the Boehm garbage collector, this
variable sets the initial size of the heap in bytes. It defaults to
384 MiB. Setting it to a low value reduces memory consumption, but
will increase runtime due to the overhead of garbage collection.
- <span id="env-NIX_CONFIG">[`NIX_CONFIG`](#env-NIX_CONFIG)</span>
Applies settings from Nix configuration from the environment.
The content is treated as if it was read from a Nix configuration file.
Settings are separated by the newline character.
- <span id="env-NIX_USER_CONF_FILES">[`NIX_USER_CONF_FILES`](#env-NIX_USER_CONF_FILES)</span>
Overrides the location of the Nix user configuration files to load from.
The default are the locations according to the [XDG Base Directory Specification].
See the [XDG Base Directories](#xdg-base-directories) sub-section for details.
The variable is treated as a list separated by the `:` token.
- <span id="env-TMPDIR">[`TMPDIR`](#env-TMPDIR)</span>
Use the specified directory to store temporary files. In particular,
this includes temporary build directories; these can take up
substantial amounts of disk space. The default is `/tmp`.
- <span id="env-NIX_REMOTE">[`NIX_REMOTE`](#env-NIX_REMOTE)</span>
This variable should be set to `daemon` if you want to use the Nix
daemon to execute Nix operations. This is necessary in [multi-user
Nix installations](@docroot@/installation/multi-user.md). If the Nix
daemon's Unix socket is at some non-standard path, this variable
should be set to `unix://path/to/socket`. Otherwise, it should be
left unset.
- <span id="env-NIX_SHOW_STATS">[`NIX_SHOW_STATS`](#env-NIX_SHOW_STATS)</span>
If set to `1`, Nix will print some evaluation statistics, such as
the number of values allocated.
- <span id="env-NIX_COUNT_CALLS">[`NIX_COUNT_CALLS`](#env-NIX_COUNT_CALLS)</span>
If set to `1`, Nix will print how often functions were called during
Nix expression evaluation. This is useful for profiling your Nix
expressions.
- <span id="env-GC_INITIAL_HEAP_SIZE">[`GC_INITIAL_HEAP_SIZE`](#env-GC_INITIAL_HEAP_SIZE)</span>
If Nix has been configured to use the Boehm garbage collector, this
variable sets the initial size of the heap in bytes. It defaults to
384 MiB. Setting it to a low value reduces memory consumption, but
will increase runtime due to the overhead of garbage collection.
## XDG Base Directories
+85 -33
View File
@@ -14,16 +14,21 @@
# Description
The install operation creates a new user environment, based on the
current generation of the active profile, to which a set of store paths
described by *args* is added. The arguments *args* map to store paths in
a number of possible ways:
The install operation creates a new user environment.
It is based on the current generation of the active [profile](@docroot@/command-ref/files/profiles.md), to which a set of [store paths] described by *args* is added.
- By default, *args* is a set of derivation names denoting derivations
in the active Nix expression. These are realised, and the resulting
output paths are installed. Currently installed derivations with a
name equal to the name of a derivation being added are removed
unless the option `--preserve-installed` is specified.
[store paths]: @docroot@/glossary.md#gloss-store-path
The arguments *args* map to store paths in a number of possible ways:
- By default, *args* is a set of [derivation] names denoting derivations in the [default Nix expression].
These are [realised], and the resulting output paths are installed.
Currently installed derivations with a name equal to the name of a derivation being added are removed unless the option `--preserve-installed` is specified.
[derivation]: @docroot@/glossary.md#gloss-derivation
[default Nix expression]: @docroot@/command-ref/files/default-nix-expression.md
[realised]: @docroot@/glossary.md#gloss-realise
If there are multiple derivations matching a name in *args* that
have the same name (e.g., `gcc-3.3.6` and `gcc-4.1.1`), then the
@@ -40,44 +45,90 @@ a number of possible ways:
gcc-3.3.6 gcc-4.1.1` will install both version of GCC (and will
probably cause a user environment conflict\!).
- If `--attr` (`-A`) is specified, the arguments are *attribute
paths* that select attributes from the top-level Nix
expression. This is faster than using derivation names and
unambiguous. To find out the attribute paths of available
packages, use `nix-env --query --available --attr-path `.
- If [`--attr`](#opt-attr) / `-A` is specified, the arguments are *attribute paths* that select attributes from the [default Nix expression].
This is faster than using derivation names and unambiguous.
Show the attribute paths of available packages with [`nix-env --query`](./query.md):
```console
nix-env --query --available --attr-path`
```
- If `--from-profile` *path* is given, *args* is a set of names
denoting installed store paths in the profile *path*. This is an
denoting installed [store paths] in the profile *path*. This is an
easy way to copy user environment elements from one profile to
another.
- If `--from-expression` is given, *args* are Nix
[functions](@docroot@/language/constructs.md#functions)
that are called with the active Nix expression as their single
argument. The derivations returned by those function calls are
installed. This allows derivations to be specified in an
unambiguous way, which is necessary if there are multiple
derivations with the same name.
- If `--from-expression` is given, *args* are [Nix language functions](@docroot@/language/constructs.md#functions) that are called with the [default Nix expression] as their single argument.
The derivations returned by those function calls are installed.
This allows derivations to be specified in an unambiguous way, which is necessary if there are multiple derivations with the same name.
- If *args* are [store derivations](@docroot@/glossary.md#gloss-store-derivation), then these are
[realised](@docroot@/command-ref/nix-store/realise.md), and the resulting output paths
are installed.
- If *args* are [store derivations](@docroot@/glossary.md#gloss-store-derivation), then these are [realised], and the resulting output paths are installed.
- If *args* are store paths that are not store derivations, then these
are [realised](@docroot@/command-ref/nix-store/realise.md) and installed.
- If *args* are [store paths] that are not store derivations, then these are [realised] and installed.
- By default all outputs are installed for each derivation. That can
be reduced by setting `meta.outputsToInstall`.
- By default all [outputs](@docroot@/language/derivations.md#attr-outputs) are installed for each [derivation].
This can be overridden by adding a `meta.outputsToInstall` attribute on the derivation listing a subset of the output names.
# Flags
Example:
The file `example.nix` defines a derivation with two outputs `foo` and `bar`, each containing a file.
```nix
# example.nix
let
pkgs = import <nixpkgs> {};
command = ''
${pkgs.coreutils}/bin/mkdir -p $foo $bar
echo foo > $foo/foo-file
echo bar > $bar/bar-file
'';
in
derivation {
name = "example";
builder = "${pkgs.bash}/bin/bash";
args = [ "-c" command ];
outputs = [ "foo" "bar" ];
system = builtins.currentSystem;
}
```
Installing from this Nix expression will make files from both outputs appear in the current profile.
```console
$ nix-env --install --file example.nix
installing 'example'
$ ls ~/.nix-profile
foo-file
bar-file
manifest.nix
```
Adding `meta.outputsToInstall` to that derivation will make `nix-env` only install files from the specified outputs.
```nix
# example-outputs.nix
import ./example.nix // { meta.outputsToInstall = [ "bar" ]; }
```
```console
$ nix-env --install --file example-outputs.nix
installing 'example'
$ ls ~/.nix-profile
bar-file
manifest.nix
```
# Options
- `--prebuilt-only` / `-b`
- `--prebuilt-only` / `-b`\
Use only derivations for which a substitute is registered, i.e.,
there is a pre-built binary available that can be downloaded in lieu
of building the derivation. Thus, no packages will be built from
source.
- `--preserve-installed` / `-P`\
- `--preserve-installed` / `-P`
Do not remove derivations with a name matching one of the
derivations being installed. Usually, trying to have two versions of
the same package installed in the same generation of a profile will
@@ -85,7 +136,8 @@ a number of possible ways:
clashes between the two versions. However, this is not the case for
all packages.
- `--remove-all` / `-r`\
- `--remove-all` / `-r`
Remove all previously installed packages first. This is equivalent
to running `nix-env --uninstall '.*'` first, except that everything happens
in a single transaction.
+3 -3
View File
@@ -235,14 +235,14 @@ package like Terraform:
```bash
#! /usr/bin/env nix-shell
#! nix-shell -i bash --packages "terraform.withPlugins (plugins: [ plugins.openstack ])"
#! nix-shell -i bash --packages 'terraform.withPlugins (plugins: [ plugins.openstack ])'
terraform apply
```
> **Note**
>
> You must use double quotes (`"`) when passing a simple Nix expression
> You must use single or double quotes (`'`, `"`) when passing a simple Nix expression
> in a nix-shell shebang.
Finally, using the merging of multiple nix-shell shebangs the following
@@ -251,7 +251,7 @@ branch):
```haskell
#! /usr/bin/env nix-shell
#! nix-shell -i runghc --packages "haskellPackages.ghcWithPackages (ps: [ps.download-curl ps.tagsoup])"
#! nix-shell -i runghc --packages 'haskellPackages.ghcWithPackages (ps: [ps.download-curl ps.tagsoup])'
#! nix-shell -I nixpkgs=https://github.com/NixOS/nixpkgs/archive/nixos-20.03.tar.gz
import Network.Curl.Download
@@ -15,8 +15,12 @@ Each of *paths* is processed as follows:
1. If it is not [valid], substitute the store derivation file itself.
2. Realise its [output paths]:
- Try to fetch from [substituters] the [store objects] associated with the output paths in the store derivation's [closure].
- With [content-addressed derivations] (experimental): Determine the output paths to realise by querying content-addressed realisation entries in the [Nix database].
- For any store paths that cannot be substituted, produce the required store objects. This involves first realising all outputs of the derivation's dependencies and then running the derivation's [`builder`](@docroot@/language/derivations.md#attr-builder) executable. <!-- TODO: Link to build process page #8888 -->
- With [content-addressed derivations] (experimental):
Determine the output paths to realise by querying content-addressed realisation entries in the [Nix database].
- For any store paths that cannot be substituted, produce the required store objects:
1. Realise all outputs of the derivation's dependencies
2. Run the derivation's [`builder`](@docroot@/language/derivations.md#attr-builder) executable
<!-- TODO: Link to build process page #8888 -->
- Otherwise, and if the path is not already valid: Try to fetch the associated [store objects] in the path's [closure] from [substituters].
If no substitutes are available and no store derivation is given, realisation fails.
@@ -1,57 +0,0 @@
\--help
\--version
\--verbose
\-v
\--quiet
\--log-format
format
\--no-build-output
\-Q
\--max-jobs
\-j
number
\--cores
number
\--max-silent-time
number
\--timeout
number
\--keep-going
\-k
\--keep-failed
\-K
\--fallback
\--readonly-mode
\-I
path
\--option
name
value
+4
View File
@@ -203,3 +203,7 @@ Most Nix commands accept the following command-line options:
Fix corrupted or missing store paths by redownloading or rebuilding them.
Note that this is slow because it requires computing a cryptographic hash of the contents of every path in the closure of the build.
Also note the warning under `nix-store --repair-path`.
> **Note**
>
> See [`man nix.conf`](@docroot@/command-ref/conf-file.md#command-line-flags) for overriding configuration settings with command line flags.
@@ -1,15 +0,0 @@
\--prebuilt-only
\-b
\--attr
\-A
\--from-expression
\-E
\--from-profile
path
+8 -1
View File
@@ -1 +1,8 @@
# Contributing
# Development
Nix is developed on GitHub.
Check the [contributing guide](https://github.com/NixOS/nix/blob/master/CONTRIBUTING.md) if you want to get involved.
This chapter is a collection of guides for making changes to the code and documentation.
If you're not sure where to start, try to [compile Nix from source](./hacking.md) and consider [making improvements to documentation](./documentation.md).
@@ -0,0 +1,210 @@
# Contributing documentation
Improvements to documentation are very much appreciated, and a good way to start out with contributing to Nix.
This is how you can help:
- Address [open issues with documentation](https://github.com/NixOS/nix/issues?q=is%3Aissue+is%3Aopen+label%3Adocumentation)
- Review [pull requests concerning documentation](https://github.com/NixOS/nix/pulls?q=is%3Apr+is%3Aopen+label%3Adocumentation)
Incremental refactorings of the documentation build setup to make it faster or easier to understand and maintain are also welcome.
## Building the manual
Build the manual from scratch:
```console
nix-build $(nix-instantiate)'!doc'
```
or
```console
nix build .#^doc
```
and open `./result-doc/share/doc/nix/manual/index.html`.
To build the manual incrementally, [enter the development shell](./hacking.md) and run:
```console
make manual-html -j $NIX_BUILD_CORES
```
and open `./outputs/out/share/doc/nix/manual/language/index.html`.
In order to reflect changes to the [Makefile for the manual], clear all generated files before re-building:
[Makefile for the manual]: https://github.com/NixOS/nix/blob/master/doc/manual/local.mk
```console
rm $(git ls-files doc/manual/ -o | grep -F '.md') && rmdir doc/manual/src/command-ref/new-cli && make manual-html -j $NIX_BUILD_CORES
```
## Style guide
The goal of this style guide is to make it such that
- The manual is easy to search and skim for relevant information
- Documentation sources are easy to edit
- Changes to documentation are easy to review
You will notice that this is not implemented consistently yet.
Please follow the guide when making additions or changes to existing documentation.
Do not make sweeping changes, unless they are programmatic and can be validated easily.
### Language
This manual is [reference documentation](https://diataxis.fr/reference/).
The typical usage pattern is to look up isolated pieces of information.
It should therefore aim to be correct, consistent, complete, and easy to navigate at a glance.
- Aim for clarity and brevity.
Please take the time to read the [plain language guidelines](https://www.plainlanguage.gov/guidelines/) for details.
- Describe the subject factually.
In particular, do not make value judgements or recommendations.
Check the code or add tests if in doubt.
- Provide complete, minimal examples, and explain them.
Readers should be able to try examples verbatim and get the same results as shown in the manual.
Always describe in words what a given example does.
Non-trivial examples may need additional explanation, especially if they use concepts from outside the given context.
- Always explain code examples in the text.
Use comments in code samples very sparingly, for instance to highlight a particular aspect.
Readers tend to glance over large amounts of code when scanning for information.
Especially beginners will likely find reading more complex-looking code strenuous and may therefore avoid it altogether.
If a code sample appears to require a lot of inline explanation, consider replacing it with a simpler one.
If that's not possible, break the example down into multiple parts, explain them separately, and then show the combined result at the end.
This should be a last resort, as that would amount to writing a [tutorial](https://diataxis.fr/tutorials/) on the given subject.
- Use British English.
This is a somewhat arbitrary choice to force consistency, and accounts for the fact that a majority of Nix users and developers are from Europe.
### Links and anchors
Reference documentation must be readable in arbitrary order.
Readers cannot be expected to have any particular prerequisite knowledge about Nix.
While the table of contents can provide guidance and full-text search can help, they are most likely to find what they need by following sensible cross-references.
- Link to technical terms
When mentioning Nix-specific concepts, commands, options, settings, etc., link to appropriate documentation.
Also link to external tools or concepts, especially if their meaning may be ambiguous.
You may also want to link to definitions of less common technical terms.
Then readers won't have to actively search for definitions and are more likely to discover relevant information on their own.
> **Note**
>
> `man` and `--help` pages don't display links.
> Use appropriate link texts such that readers of terminal output can infer search terms.
- Do not break existing URLs between releases.
There are countless links in the wild pointing to old versions of the manual.
We want people to find up-to-date documentation when following popular advice.
- When moving files, update [redirects on nixos.org](https://github.com/NixOS/nixos-homepage/blob/master/netlify.toml).
This is especially important when moving information out of the Nix manual to other resources.
- When changing anchors, update [client-side redirects](https://github.com/NixOS/nix/blob/master/doc/manual/redirects.js)
The current setup is cumbersome, and help making better automation is appreciated.
The build checks for broken internal links with.
This happens late in the process, so [building the whole manual](#building-the-manual) is not suitable for iterating quickly.
[`mdbook-linkcheck`] does not implement checking [URI fragments] yet.
[`mdbook-linkcheck`]: https://github.com/Michael-F-Bryan/mdbook-linkcheck
[URI fragments]: https://en.wikipedia.org/wiki/URI_fragment
### Markdown conventions
The manual is written in markdown, and rendered with [mdBook](https://github.com/rust-lang/mdBook) for the web and with [lowdown](https://github.com/kristapsdz/lowdown) for `man` pages and `--help` output.
For supported markdown features, refer to:
- [mdBook documentation](https://rust-lang.github.io/mdBook/format/markdown.html)
- [lowdown documentation](https://kristaps.bsd.lv/lowdown/)
Please observe these guidelines to ease reviews:
- Write one sentence per line.
This makes long sentences immediately visible, and makes it easier to review changes and make direct suggestions.
- Use reference links – sparingly – to ease source readability.
Put definitions close to their first use.
Example:
```
A [store object] contains a [file system object] and [references] to other store objects.
[store object]: @docroot@/glossary.md#gloss-store-object
[file system object]: @docroot@/architecture/file-system-object.md
[references]: @docroot@/glossary.md#gloss-reference
```
- Use admonitions of the following form:
```
> **Note**
>
> This is a note.
```
Highlight examples as such:
````
> **Example**
>
> ```console
> $ nix --version
> ```
````
Highlight syntax definiions as such, using [EBNF](https://en.wikipedia.org/wiki/Extended_Backus%E2%80%93Naur_form) notation:
````
> **Syntax**
>
> *attribute-set* = `{` [ *attribute-name* `=` *expression* `;` ... ] `}`
````
### The `@docroot@` variable
`@docroot@` provides a base path for links that occur in reusable snippets or other documentation that doesn't have a base path of its own.
If a broken link occurs in a snippet that was inserted into multiple generated files in different directories, use `@docroot@` to reference the `doc/manual/src` directory.
If the `@docroot@` literal appears in an error message from the [`mdbook-linkcheck`] tool, the `@docroot@` replacement needs to be applied to the generated source file that mentions it.
See existing `@docroot@` logic in the [Makefile for the manual].
Regular markdown files used for the manual have a base path of their own and they can use relative paths instead of `@docroot@`.
## API documentation
[Doxygen API documentation] is available online.
You can also build and view it yourself:
[Doxygen API documentation]: https://hydra.nixos.org/job/nix/master/internal-api-docs/latest/download-by-type/doc/internal-api-docs
```console
# nix build .#hydraJobs.internal-api-docs
# xdg-open ./result/share/doc/nix/internal-api/html/index.html
```
or inside `nix-shell` or `nix develop`:
```
# make internal-api-html
# xdg-open ./outputs/doc/share/doc/nix/internal-api/html/index.html
```
+29 -56
View File
@@ -42,8 +42,8 @@ $ nix develop .#native-clang11StdenvPackages
To build Nix itself in this shell:
```console
[nix-shell]$ ./bootstrap.sh
[nix-shell]$ ./configure $configureFlags --prefix=$(pwd)/outputs/out
[nix-shell]$ autoreconfPhase
[nix-shell]$ configurePhase
[nix-shell]$ make -j $NIX_BUILD_CORES
```
@@ -86,7 +86,7 @@ $ nix-shell --attr devShells.x86_64-linux.native-clang11StdenvPackages
To build Nix itself in this shell:
```console
[nix-shell]$ ./bootstrap.sh
[nix-shell]$ autoreconfPhase
[nix-shell]$ ./configure $configureFlags --prefix=$(pwd)/outputs/out
[nix-shell]$ make -j $NIX_BUILD_CORES
```
@@ -210,7 +210,7 @@ See [supported compilation environments](#compilation-environments) and instruct
To use the LSP with your editor, you first need to [set up `clangd`](https://clangd.llvm.org/installation#project-setup) by running:
```console
make clean && bear -- make -j$NIX_BUILD_CORES install
make clean && bear -- make -j$NIX_BUILD_CORES default check install
```
Configure your editor to use the `clangd` from the shell, either by running it inside the development shell, or by using [nix-direnv](https://github.com/nix-community/nix-direnv) and [the appropriate editor plugin](https://github.com/direnv/direnv/wiki#editor-integration).
@@ -221,67 +221,40 @@ Configure your editor to use the `clangd` from the shell, either by running it i
> Some other editors (e.g. Emacs, Vim) need a plugin to support LSP servers in general (e.g. [lsp-mode](https://github.com/emacs-lsp/lsp-mode) for Emacs and [vim-lsp](https://github.com/prabirshrestha/vim-lsp) for vim).
> Editor-specific setup is typically opinionated, so we will not cover it here in more detail.
### Checking links in the manual
## Add a release note
The build checks for broken internal links.
This happens late in the process, so `nix build` is not suitable for iterating.
To build the manual incrementally, run:
`doc/manual/rl-next` contains release notes entries for all unreleased changes.
```console
make html -j $NIX_BUILD_CORES
```
User-visible changes should come with a release note.
In order to reflect changes to the [Makefile], clear all generated files before re-building:
### Add an entry
[Makefile]: https://github.com/NixOS/nix/blob/master/doc/manual/local.mk
```console
rm $(git ls-files doc/manual/ -o | grep -F '.md') && rmdir doc/manual/src/command-ref/new-cli && make html -j $NIX_BUILD_CORES
```
[`mdbook-linkcheck`] does not implement checking [URI fragments] yet.
[`mdbook-linkcheck`]: https://github.com/Michael-F-Bryan/mdbook-linkcheck
[URI fragments]: https://en.wikipedia.org/wiki/URI_fragment
#### `@docroot@` variable
`@docroot@` provides a base path for links that occur in reusable snippets or other documentation that doesn't have a base path of its own.
If a broken link occurs in a snippet that was inserted into multiple generated files in different directories, use `@docroot@` to reference the `doc/manual/src` directory.
If the `@docroot@` literal appears in an error message from the `mdbook-linkcheck` tool, the `@docroot@` replacement needs to be applied to the generated source file that mentions it.
See existing `@docroot@` logic in the [Makefile].
Regular markdown files used for the manual have a base path of their own and they can use relative paths instead of `@docroot@`.
## API documentation
Doxygen API documentation is [available
online](https://hydra.nixos.org/job/nix/master/internal-api-docs/latest/download-by-type/doc/internal-api-docs). You
can also build and view it yourself:
```console
# nix build .#hydraJobs.internal-api-docs
# xdg-open ./result/share/doc/nix/internal-api/html/index.html
```
or inside a `nix develop` shell by running:
Here's what a complete entry looks like. The file name is not incorporated in the document.
```
# make internal-api-html
# xdg-open ./outputs/doc/share/doc/nix/internal-api/html/index.html
synopsis: Basically a title
issues: #1234
prs: #1238
description: {
Here's one or more paragraphs that describe the change.
- It's markdown
- Add references to the manual using @docroot@
}
```
## Coverage analysis
A coverage analysis report is [available
online](https://hydra.nixos.org/job/nix/master/coverage/latest/download-by-type/report/coverage). You
can build it yourself:
Significant changes should add the following header, which moves them to the top.
```
# nix build .#hydraJobs.coverage
# xdg-open ./result/coverage/index.html
significance: significant
```
Metrics about the change in line/function coverage over time are also
[available](https://hydra.nixos.org/job/nix/master/coverage#tabs-charts).
<!-- Keep an eye on https://codeberg.org/fgaz/changelog-d/issues/1 -->
See also the [format documentation](https://github.com/haskell/cabal/blob/master/CONTRIBUTING.md#changelog).
### Build process
Releases have a precomputed `rl-MAJOR.MINOR.md`, and no `rl-next.md`.
Set `buildUnreleasedNotes = true;` in `flake.nix` to build the release notes on the fly.
+107 -25
View File
@@ -1,18 +1,88 @@
# Running tests
## Coverage analysis
A [coverage analysis report] is available online
You can build it yourself:
[coverage analysis report]: https://hydra.nixos.org/job/nix/master/coverage/latest/download-by-type/report/coverage
```
# nix build .#hydraJobs.coverage
# xdg-open ./result/coverage/index.html
```
[Extensive records of build metrics](https://hydra.nixos.org/job/nix/master/coverage#tabs-charts), such as test coverage over time, are also available online.
## Unit-tests
The unit-tests for each Nix library (`libexpr`, `libstore`, etc..) are defined
under `src/{library_name}/tests` using the
[googletest](https://google.github.io/googletest/) and
[rapidcheck](https://github.com/emil-e/rapidcheck) frameworks.
The unit tests are defined using the [googletest] and [rapidcheck] frameworks.
[googletest]: https://google.github.io/googletest/
[rapidcheck]: https://github.com/emil-e/rapidcheck
### Source and header layout
> An example of some files, demonstrating much of what is described below
>
> ```
> src
> ├── libexpr
> │ ├── value/context.hh
> │ ├── value/context.cc
> │ │
> │ …
> └── tests
> │ ├── value/context.hh
> │ ├── value/context.cc
> │ │
> │ …
> │
> ├── unit-test-data
> │ ├── libstore
> │ │ ├── worker-protocol/content-address.bin
> │ │ …
> │ …
> …
> ```
The unit tests for each Nix library (`libnixexpr`, `libnixstore`, etc..) live inside a directory `src/${library_shortname}/tests` within the directory for the library (`src/${library_shortname}`).
The data is in `unit-test-data`, with one subdir per library, with the same name as where the code goes.
For example, `libnixstore` code is in `src/libstore`, and its test data is in `unit-test-data/libstore`.
The path to the `unit-test-data` directory is passed to the unit test executable with the environment variable `_NIX_TEST_UNIT_DATA`.
> **Note**
> Due to the way googletest works, downstream unit test executables will actually include and re-run upstream library tests.
> Therefore it is important that the same value for `_NIX_TEST_UNIT_DATA` be used with the tests for each library.
> That is why we have the test data nested within a single `unit-test-data` directory.
### Running tests
You can run the whole testsuite with `make check`, or the tests for a specific component with `make libfoo-tests_RUN`.
Finer-grained filtering is also possible using the [--gtest_filter](https://google.github.io/googletest/advanced.html#running-a-subset-of-the-tests) command-line option, or the `GTEST_FILTER` environment variable.
### Characterisation testing { #characaterisation-testing-unit }
See [functional characterisation testing](#characterisation-testing-functional) for a broader discussion of characterisation testing.
Like with the functional characterisation, `_NIX_TEST_ACCEPT=1` is also used.
For example:
```shell-session
$ _NIX_TEST_ACCEPT=1 make libstore-tests-exe_RUN
...
[ SKIPPED ] WorkerProtoTest.string_read
[ SKIPPED ] WorkerProtoTest.string_write
[ SKIPPED ] WorkerProtoTest.storePath_read
[ SKIPPED ] WorkerProtoTest.storePath_write
...
```
will regenerate the "golden master" expected result for the `libnixstore` characterisation tests.
The characterisation tests will mark themselves "skipped" since they regenerated the expected result instead of actually testing anything.
## Functional tests
The functional tests reside under the `tests` directory and are listed in `tests/local.mk`.
The functional tests reside under the `tests/functional` directory and are listed in `tests/functional/local.mk`.
Each test is a bash script.
### Running the whole test suite
@@ -21,8 +91,8 @@ The whole test suite can be run with:
```shell-session
$ make install && make installcheck
ran test tests/foo.sh... [PASS]
ran test tests/bar.sh... [PASS]
ran test tests/functional/foo.sh... [PASS]
ran test tests/functional/bar.sh... [PASS]
...
```
@@ -30,14 +100,14 @@ ran test tests/bar.sh... [PASS]
Sometimes it is useful to group related tests so they can be easily run together without running the entire test suite.
Each test group is in a subdirectory of `tests`.
For example, `tests/ca/local.mk` defines a `ca` test group for content-addressed derivation outputs.
For example, `tests/functional/ca/local.mk` defines a `ca` test group for content-addressed derivation outputs.
That test group can be run like this:
```shell-session
$ make ca.test-group -j50
ran test tests/ca/nix-run.sh... [PASS]
ran test tests/ca/import-derivation.sh... [PASS]
ran test tests/functional/ca/nix-run.sh... [PASS]
ran test tests/functional/ca/import-derivation.sh... [PASS]
...
```
@@ -56,24 +126,24 @@ install-tests-groups += $(test-group-name)
Individual tests can be run with `make`:
```shell-session
$ make tests/${testName}.sh.test
ran test tests/${testName}.sh... [PASS]
$ make tests/functional/${testName}.sh.test
ran test tests/functional/${testName}.sh... [PASS]
```
or without `make`:
```shell-session
$ ./mk/run-test.sh tests/${testName}.sh
ran test tests/${testName}.sh... [PASS]
$ ./mk/run-test.sh tests/functional/${testName}.sh tests/functional/init.sh
ran test tests/functional/${testName}.sh... [PASS]
```
To see the complete output, one can also run:
```shell-session
$ ./mk/debug-test.sh tests/${testName}.sh
+ foo
$ ./mk/debug-test.sh tests/functional/${testName}.sh tests/functional/init.sh
+(${testName}.sh:1) foo
output from foo
+ bar
+(${testName}.sh:2) bar
output from bar
...
```
@@ -105,7 +175,7 @@ edit it like so:
Then, running the test with `./mk/debug-test.sh` will drop you into GDB once the script reaches that point:
```shell-session
$ ./mk/debug-test.sh tests/${testName}.sh
$ ./mk/debug-test.sh tests/functional/${testName}.sh tests/functional/init.sh
...
+ gdb blash blub
GNU gdb (GDB) 12.1
@@ -116,17 +186,29 @@ GNU gdb (GDB) 12.1
One can debug the Nix invocation in all the usual ways.
For example, enter `run` to start the Nix invocation.
### Characterization testing
### Troubleshooting
Occasionally, Nix utilizes a technique called [Characterization Testing](https://en.wikipedia.org/wiki/Characterization_test) as part of the functional tests.
Sometimes running tests in the development shell may leave artefacts in the local repository.
To remove any traces of that:
```console
git clean -x --force tests
```
### Characterisation testing { #characterisation-testing-functional }
Occasionally, Nix utilizes a technique called [Characterisation Testing](https://en.wikipedia.org/wiki/Characterization_test) as part of the functional tests.
This technique is to include the exact output/behavior of a former version of Nix in a test in order to check that Nix continues to produce the same behavior going forward.
For example, this technique is used for the language tests, to check both the printed final value if evaluation was successful, and any errors and warnings encountered.
It is frequently useful to regenerate the expected output.
To do that, rerun the failed test with `_NIX_TEST_ACCEPT=1`.
(At least, this is the convention we've used for `tests/lang.sh`.
If we add more characterization testing we should always strive to be consistent.)
To do that, rerun the failed test(s) with `_NIX_TEST_ACCEPT=1`.
For example:
```bash
_NIX_TEST_ACCEPT=1 make tests/functional/lang.sh.test
```
This convention is shared with the [characterisation unit tests](#characterisation-testing-unit) too.
An interesting situation to document is the case when these tests are "overfitted".
The language tests are, again, an example of this.
@@ -139,7 +221,7 @@ Diagnostic outputs are indeed not a stable interface, but they still are importa
By recording the expected output, the test suite guards against accidental changes, and ensure the *result* (not just the code that implements it) of the diagnostic code paths are under code review.
Regressions are caught, and improvements always show up in code review.
To ensure that characterization testing doesn't make it harder to intentionally change these interfaces, there always must be an easy way to regenerate the expected output, as we do with `_NIX_TEST_ACCEPT=1`.
To ensure that characterisation testing doesn't make it harder to intentionally change these interfaces, there always must be an easy way to regenerate the expected output, as we do with `_NIX_TEST_ACCEPT=1`.
## Integration tests
@@ -153,7 +235,7 @@ You can run them manually with `nix build .#hydraJobs.tests.{testName}` or `nix-
After a one-time setup, the Nix repository's GitHub Actions continuous integration (CI) workflow can test the installer each time you push to a branch.
Creating a Cachix cache for your installer tests and adding its authorization token to GitHub enables [two installer-specific jobs in the CI workflow](https://github.com/NixOS/nix/blob/88a45d6149c0e304f6eb2efcc2d7a4d0d569f8af/.github/workflows/ci.yml#L50-L91):
Creating a Cachix cache for your installer tests and adding its authorisation token to GitHub enables [two installer-specific jobs in the CI workflow](https://github.com/NixOS/nix/blob/88a45d6149c0e304f6eb2efcc2d7a4d0d569f8af/.github/workflows/ci.yml#L50-L91):
- The `installer` job generates installers for the platforms below and uploads them to your Cachix cache:
- `x86_64-linux`
+31 -22
View File
@@ -33,11 +33,15 @@
Ensure a [store path] is [valid][validity].
This means either running the [`builder`](@docroot@/language/derivations.md#attr-builder) executable as specified in the corresponding [derivation], or fetching a pre-built [store object] from a [substituter], or delegating to a [remote builder](@docroot@/advanced-topics/distributed-builds.html) and retrieving the outputs. <!-- TODO: link [running] to build process page, #8888 -->
This can be achieved by:
- Fetching a pre-built [store object] from a [substituter]
- Running the [`builder`](@docroot@/language/derivations.md#attr-builder) executable as specified in the corresponding [derivation]
- Delegating to a [remote builder](@docroot@/advanced-topics/distributed-builds.html) and retrieving the outputs
<!-- TODO: link [running] to build process page, #8888 -->
See [`nix-build`](./command-ref/nix-build.md) and [`nix-store --realise`](@docroot@/command-ref/nix-store/realise.md).
See [`nix-store --realise`](@docroot@/command-ref/nix-store/realise.md) for a detailed description of the algorithm.
See [`nix build`](./command-ref/new-cli/nix3-build.md) (experimental).
See also [`nix-build`](./command-ref/nix-build.md) and [`nix build`](./command-ref/new-cli/nix3-build.md) (experimental).
[realise]: #gloss-realise
@@ -54,22 +58,16 @@
- [store]{#gloss-store}
The location in the file system where store objects live. Typically
`/nix/store`.
A collection of store objects, with operations to manipulate that collection.
See [Nix store](./store/index.md) for details.
From the perspective of the location where Nix is
invoked, the Nix store can be referred to
as a "_local_" or a "_remote_" one:
There are many types of stores.
See [`nix help-stores`](@docroot@/command-ref/new-cli/nix3-help-stores.md) for a complete list.
+ A [local store]{#gloss-local-store} exists on the filesystem of
the machine where Nix is invoked. You can use other
local stores by passing the `--store` flag to the
`nix` command. Local stores can be used for building derivations.
+ A *remote store* exists anywhere other than the
local filesystem. One example is the `/nix/store`
directory on another machine, accessed via `ssh` or
served by the `nix-serve` Perl script.
From the perspective of the location where Nix is invoked, the Nix store can be referred to _local_ or _remote_.
Only a [local store]{#gloss-local-store} exposes a location in the file system of the machine where Nix is invoked that allows access to store objects, typically `/nix/store`.
Local stores can be used for building [derivations](#derivation).
See [Local Store](@docroot@/command-ref/new-cli/nix3-help-stores.md#local-store) for details.
[store]: #gloss-store
[local store]: #gloss-local-store
@@ -88,10 +86,13 @@
- [store path]{#gloss-store-path}
The location of a [store object] in the file system, i.e., an
immediate child of the Nix store directory.
The location of a [store object](@docroot@/store/index.md#store-object) in the file system, i.e., an immediate child of the Nix store directory.
Example: `/nix/store/a040m110amc4h71lds2jmr8qrkj2jhxd-git-2.38.1`
> **Example**
>
> `/nix/store/a040m110amc4h71lds2jmr8qrkj2jhxd-git-2.38.1`
See [Store Path](@docroot@/store/store-path.md) for details.
[store path]: #gloss-store-path
@@ -99,18 +100,25 @@
The Nix data model for representing simplified file system data.
See [File System Object](@docroot@/architecture/file-system-object.md) for details.
See [File System Object](@docroot@/store/file-system-object.md) for details.
[file system object]: #gloss-file-system-object
- [store object]{#gloss-store-object}
Part of the contents of a [store].
A store object consists of a [file system object], [reference]s to other store objects, and other metadata.
A store object consists of a [file system object], [references][reference] to other store objects, and other metadata.
It can be referred to by a [store path].
See [Store Object](@docroot@/store/index.md#store-object) for details.
[store object]: #gloss-store-object
- [IFD]{#gloss-ifd}
[Import From Derivation](./language/import-from-derivation.md)
- [input-addressed store object]{#gloss-input-addressed-store-object}
A store object produced by building a
@@ -200,6 +208,7 @@
- [output]{#gloss-output}
A [store object] produced by a [derivation].
See [the `outputs` argument to the `derivation` function](@docroot@/language/derivations.md#attr-outputs) for details.
[output]: #gloss-output
@@ -3,7 +3,7 @@
After cloning Nix's Git repository, issue the following commands:
```console
$ ./bootstrap.sh
$ autoreconf -vfi
$ ./configure options...
$ make
$ make install
@@ -3,14 +3,14 @@
To run the latest stable release of Nix with Docker run the following command:
```console
$ docker run -ti nixos/nix
Unable to find image 'nixos/nix:latest' locally
latest: Pulling from nixos/nix
$ docker run -ti ghcr.io/nixos/nix
Unable to find image 'ghcr.io/nixos/nix:latest' locally
latest: Pulling from ghcr.io/nixos/nix
5843afab3874: Pull complete
b52bf13f109c: Pull complete
1e2415612aa3: Pull complete
Digest: sha256:27f6e7f60227e959ee7ece361f75d4844a40e1cc6878b6868fe30140420031ff
Status: Downloaded newer image for nixos/nix:latest
Status: Downloaded newer image for ghcr.io/nixos/nix:latest
35ca4ada6e96:/# nix --version
nix (Nix) 2.3.12
35ca4ada6e96:/# exit
+34 -7
View File
@@ -112,6 +112,13 @@ Derivations can declare some infrequently used optional attributes.
> environmental variables come from the environment of the
> `nix-build`.
If the [`configurable-impure-env` experimental
feature](@docroot@/contributing/experimental-features.md#xp-feature-configurable-impure-env)
is enabled, these environment variables can also be controlled
through the
[`impure-env`](@docroot@/command-ref/conf-file.md#conf-impure-env)
configuration setting.
- [`outputHash`]{#adv-attr-outputHash}; [`outputHashAlgo`]{#adv-attr-outputHashAlgo}; [`outputHashMode`]{#adv-attr-outputHashMode}\
These attributes declare that the derivation is a so-called
*fixed-output derivation*, which means that a cryptographic hash of
@@ -229,6 +236,8 @@ Derivations can declare some infrequently used optional attributes.
[`outputHashAlgo`](#adv-attr-outputHashAlgo)
like for *fixed-output derivations* (see above).
It also implicitly requires that the machine to build the derivation must have the `ca-derivations` [system feature](@docroot@/command-ref/conf-file.md#conf-system-features).
- [`passAsFile`]{#adv-attr-passAsFile}\
A list of names of attributes that should be passed via files rather
than environment variables. For example, if you have
@@ -261,6 +270,9 @@ Derivations can declare some infrequently used optional attributes.
useful for very trivial derivations (such as `writeText` in Nixpkgs)
that are cheaper to build than to substitute from a binary cache.
You may disable the effects of this attibute by enabling the
`always-allow-substitutes` configuration option in Nix.
> **Note**
>
> You need to have a builder configured which satisfies the
@@ -271,18 +283,21 @@ Derivations can declare some infrequently used optional attributes.
- [`__structuredAttrs`]{#adv-attr-structuredAttrs}\
If the special attribute `__structuredAttrs` is set to `true`, the other derivation
attributes are serialised in JSON format and made available to the
builder via the file `.attrs.json` in the builder’s temporary
directory. This obviates the need for [`passAsFile`](#adv-attr-passAsFile) since JSON files
have no size restrictions, unlike process environments.
attributes are serialised into a file in JSON format. The environment variable
`NIX_ATTRS_JSON_FILE` points to the exact location of that file both in a build
and a [`nix-shell`](../command-ref/nix-shell.md). This obviates the need for
[`passAsFile`](#adv-attr-passAsFile) since JSON files have no size restrictions,
unlike process environments.
It also makes it possible to tweak derivation settings in a structured way; see
[`outputChecks`](#adv-attr-outputChecks) for example.
As a convenience to Bash builders,
Nix writes a script named `.attrs.sh` to the builder’s directory
that initialises shell variables corresponding to all attributes
that are representable in Bash. This includes non-nested
Nix writes a script that initialises shell variables
corresponding to all attributes that are representable in Bash. The
environment variable `NIX_ATTRS_SH_FILE` points to the exact
location of the script, both in a build and a
[`nix-shell`](../command-ref/nix-shell.md). This includes non-nested
(associative) arrays. For example, the attribute `hardening.format = true`
ends up as the Bash associative array element `${hardening[format]}`.
@@ -335,3 +350,15 @@ Derivations can declare some infrequently used optional attributes.
This is useful, for example, when generating self-contained filesystem images with
their own embedded Nix store: hashes found inside such an image refer
to the embedded store and not to the host's Nix store.
- [`requiredSystemFeatures`]{#adv-attr-requiredSystemFeatures}\
If a derivation has the `requiredSystemFeatures` attribute, then Nix will only build it on a machine that has the corresponding features set in its [`system-features` configuration](@docroot@/command-ref/conf-file.md#conf-system-features).
For example, setting
```nix
requiredSystemFeatures = [ "kvm" ];
```
ensures that the derivation can only be built on a machine with the `kvm` feature.
+40 -14
View File
@@ -132,6 +132,32 @@ a = src-set.a; b = src-set.b; c = src-set.c;
when used while defining local variables in a let-expression or while
defining a set.
In a `let` expression, `inherit` can be used to selectively bring specific attributes of a set into scope. For example
```nix
let
x = { a = 1; b = 2; };
inherit (builtins) attrNames;
in
{
names = attrNames x;
}
```
is equivalent to
```nix
let
x = { a = 1; b = 2; };
in
{
names = builtins.attrNames x;
}
```
both evaluate to `{ names = [ "a" "b" ]; }`.
## Functions
Functions have the following form:
@@ -146,65 +172,65 @@ three kinds of patterns:
- If a pattern is a single identifier, then the function matches any
argument. Example:
```nix
let negate = x: !x;
concat = x: y: x + y;
in if negate true then concat "foo" "bar" else ""
```
Note that `concat` is a function that takes one argument and returns
a function that takes another argument. This allows partial
parameterisation (i.e., only filling some of the arguments of a
function); e.g.,
```nix
map (concat "foo") [ "bar" "bla" "abc" ]
```
evaluates to `[ "foobar" "foobla" "fooabc" ]`.
- A *set pattern* of the form `{ name1, name2, …, nameN }` matches a
set containing the listed attributes, and binds the values of those
attributes to variables in the function body. For example, the
function
```nix
{ x, y, z }: z + y + x
```
can only be called with a set containing exactly the attributes `x`,
`y` and `z`. No other attributes are allowed. If you want to allow
additional arguments, you can use an ellipsis (`...`):
```nix
{ x, y, z, ... }: z + y + x
```
This works on any set that contains at least the three named
attributes.
It is possible to provide *default values* for attributes, in
which case they are allowed to be missing. A default value is
specified by writing `name ? e`, where *e* is an arbitrary
expression. For example,
```nix
{ x, y ? "foo", z ? "bar" }: z + y + x
```
specifies a function that only requires an attribute named `x`, but
optionally accepts `y` and `z`.
- An `@`-pattern provides a means of referring to the whole value
being matched:
```nix
args@{ x, y, z, ... }: z + y + x + args.a
```
but can also be written as:
```nix
{ x, y, z, ... } @ args: z + y + x + args.a
```
@@ -0,0 +1,27 @@
# Lookup path
> **Syntax**
>
> *lookup-path* = `<` *identifier* [ `/` *identifier* ]... `>`
A lookup path is an identifier with an optional path suffix that resolves to a [path value](@docroot@/language/values.md#type-path) if the identifier matches a search path entry.
The value of a lookup path is determined by [`builtins.nixPath`](@docroot@/language/builtin-constants.md#builtins-nixPath).
See [`builtins.findFile`](@docroot@/language/builtins.md#builtins-findFile) for details on lookup path resolution.
> **Example**
>
> ```nix
> <nixpkgs>
>```
>
> /nix/var/nix/profiles/per-user/root/channels/nixpkgs
> **Example**
>
> ```nix
> <nixpkgs/nixos>
>```
>
> /nix/var/nix/profiles/per-user/root/channels/nixpkgs/nixos
+296 -142
View File
@@ -1,161 +1,315 @@
# Derivations
The most important built-in function is `derivation`, which is used to
describe a single derivation (a build task). It takes as input a set,
the attributes of which specify the inputs of the build.
The most important built-in function is `derivation`, which is used to describe a single derivation:
a specification for running an executable on precisely defined input files to repeatably produce output files at uniquely determined file system paths.
- There must be an attribute named [`system`]{#attr-system} whose value must be a
string specifying a Nix system type, such as `"i686-linux"` or
`"x86_64-darwin"`. (To figure out your system type, run `nix -vv
--version`.) The build can only be performed on a machine and
operating system matching the system type. (Nix can automatically
[forward builds for other
platforms](../advanced-topics/distributed-builds.md) by forwarding
them to other machines.)
It takes as input an attribute set, the attributes of which specify the inputs to the process.
It outputs an attribute set, and produces a [store derivation] as a side effect of evaluation.
- There must be an attribute named `name` whose value must be a
string. This is used as a symbolic name for the package by
`nix-env`, and it is appended to the output paths of the derivation.
[store derivation]: @docroot@/glossary.md#gloss-store-derivation
- There must be an attribute named [`builder`]{#attr-builder} that identifies the
program that is executed to perform the build. It can be either a
derivation or a source (a local file reference, e.g.,
`./builder.sh`).
## Input attributes
- Every attribute is passed as an environment variable to the builder.
Attribute values are translated to environment variables as follows:
- Strings and numbers are just passed verbatim.
- A *path* (e.g., `../foo/sources.tar`) causes the referenced file
to be copied to the store; its location in the store is put in
the environment variable. The idea is that all sources should
reside in the Nix store, since all inputs to a derivation should
reside in the Nix store.
- A *derivation* causes that derivation to be built prior to the
present derivation; its default output path is put in the
environment variable.
- Lists of the previous types are also allowed. They are simply
concatenated, separated by spaces.
- `true` is passed as the string `1`, `false` and `null` are
passed as an empty string.
### Required
- The optional attribute `args` specifies command-line arguments to be
passed to the builder. It should be a list.
- [`name`]{#attr-name} ([String](@docroot@/language/values.md#type-string))
- The optional attribute `outputs` specifies a list of symbolic
outputs of the derivation. By default, a derivation produces a
single output path, denoted as `out`. However, derivations can
produce multiple output paths. This is useful because it allows
outputs to be downloaded or garbage-collected separately. For
instance, imagine a library package that provides a dynamic library,
header files, and documentation. A program that links against the
library doesn’t need the header files and documentation at runtime,
and it doesn’t need the documentation at build time. Thus, the
library package could specify:
```nix
outputs = [ "lib" "headers" "doc" ];
```
This will cause Nix to pass environment variables `lib`, `headers`
and `doc` to the builder containing the intended store paths of each
output. The builder would typically do something like
```bash
./configure \
--libdir=$lib/lib \
--includedir=$headers/include \
--docdir=$doc/share/doc
```
for an Autoconf-style package. You can refer to each output of a
derivation by selecting it as an attribute, e.g.
```nix
buildInputs = [ pkg.lib pkg.headers ];
```
The first element of `outputs` determines the *default output*.
Thus, you could also write
```nix
buildInputs = [ pkg pkg.headers ];
```
since `pkg` is equivalent to `pkg.lib`.
A symbolic name for the derivation.
It is added to the [store path] of the corresponding [store derivation] as well as to its [output paths](@docroot@/glossary.md#gloss-output-path).
The function `mkDerivation` in the Nixpkgs standard environment is a
wrapper around `derivation` that adds a default value for `system` and
always uses Bash as the builder, to which the supplied builder is passed
as a command-line argument. See the Nixpkgs manual for details.
[store path]: @docroot@/glossary.md#gloss-store-path
The builder is executed as follows:
> **Example**
>
> ```nix
> derivation {
> name = "hello";
> # ...
> }
> ```
>
> The store derivation's path will be `/nix/store/<hash>-hello.drv`.
> The [output](#attr-outputs) paths will be of the form `/nix/store/<hash>-hello[-<output>]`
- A temporary directory is created under the directory specified by
`TMPDIR` (default `/tmp`) where the build will take place. The
current directory is changed to this directory.
- [`system`]{#attr-system} ([String](@docroot@/language/values.md#type-string))
- The environment is cleared and set to the derivation attributes, as
specified above.
The system type on which the [`builder`](#attr-builder) executable is meant to be run.
- In addition, the following variables are set:
- `NIX_BUILD_TOP` contains the path of the temporary directory for
this build.
- Also, `TMPDIR`, `TEMPDIR`, `TMP`, `TEMP` are set to point to the
temporary directory. This is to prevent the builder from
accidentally writing temporary files anywhere else. Doing so
might cause interference by other processes.
- `PATH` is set to `/path-not-set` to prevent shells from
initialising it to their built-in default value.
- `HOME` is set to `/homeless-shelter` to prevent programs from
using `/etc/passwd` or the like to find the user's home
directory, which could cause impurity. Usually, when `HOME` is
set, it is used as the location of the home directory, even if
it points to a non-existent path.
- `NIX_STORE` is set to the path of the top-level Nix store
directory (typically, `/nix/store`).
- For each output declared in `outputs`, the corresponding
environment variable is set to point to the intended path in the
Nix store for that output. Each output path is a concatenation
of the cryptographic hash of all build inputs, the `name`
attribute and the output name. (The output name is omitted if
it’s `out`.)
A necessary condition for Nix to build derivations locally is that the `system` attribute matches the current [`system` configuration option].
It can automatically [build on other platforms](../advanced-topics/distributed-builds.md) by forwarding build requests to other machines.
- If an output path already exists, it is removed. Also, locks are
acquired to prevent multiple Nix instances from performing the same
build at the same time.
[`system` configuration option]: @docroot@/command-ref/conf-file.md#conf-system
- A log of the combined standard output and error is written to
`/nix/var/log/nix`.
> **Example**
>
> Declare a derivation to be built on a specific system type:
>
> ```nix
> derivation {
> # ...
> system = "x86_64-linux";
> # ...
> }
> ```
- The builder is executed with the arguments specified by the
attribute `args`. If it exits with exit code 0, it is considered to
have succeeded.
> **Example**
>
> Declare a derivation to be built on the system type that evaluates the expression:
>
> ```nix
> derivation {
> # ...
> system = builtins.currentSystem;
> # ...
> }
> ```
>
> [`builtins.currentSystem`](@docroot@/language/builtin-constants.md#builtins-currentSystem) has the value of the [`system` configuration option], and defaults to the system type of the current Nix installation.
- The temporary directory is removed (unless the `-K` option was
specified).
- [`builder`]{#attr-builder} ([Path](@docroot@/language/values.md#type-path) | [String](@docroot@/language/values.md#type-string))
- If the build was successful, Nix scans each output path for
references to input paths by looking for the hash parts of the input
paths. Since these are potential runtime dependencies, Nix registers
them as dependencies of the output paths.
Path to an executable that will perform the build.
- After the build, Nix sets the last-modified timestamp on all files
in the build result to 1 (00:00:01 1/1/1970 UTC), sets the group to
the default group, and sets the mode of the file to 0444 or 0555
(i.e., read-only, with execute permission enabled if the file was
originally executable). Note that possible `setuid` and `setgid`
bits are cleared. Setuid and setgid programs are not currently
supported by Nix. This is because the Nix archives used in
deployment have no concept of ownership information, and because it
makes the build result dependent on the user performing the build.
> **Example**
>
> Use the file located at `/bin/bash` as the builder executable:
>
> ```nix
> derivation {
> # ...
> builder = "/bin/bash";
> # ...
> };
> ```
<!-- -->
> **Example**
>
> Copy a local file to the Nix store for use as the builder executable:
>
> ```nix
> derivation {
> # ...
> builder = ./builder.sh;
> # ...
> };
> ```
<!-- -->
> **Example**
>
> Use a file from another derivation as the builder executable:
>
> ```nix
> let pkgs = import <nixpkgs> {}; in
> derivation {
> # ...
> builder = "${pkgs.python}/bin/python";
> # ...
> };
> ```
### Optional
- [`args`]{#attr-args} ([List](@docroot@/language/values.md#list) of [String](@docroot@/language/values.md#type-string))
Default: `[ ]`
Command-line arguments to be passed to the [`builder`](#attr-builder) executable.
> **Example**
>
> Pass arguments to Bash to interpret a shell command:
>
> ```nix
> derivation {
> # ...
> builder = "/bin/bash";
> args = [ "-c" "echo hello world > $out" ];
> # ...
> };
> ```
- [`outputs`]{#attr-outputs} ([List](@docroot@/language/values.md#list) of [String](@docroot@/language/values.md#type-string))
Default: `[ "out" ]`
Symbolic outputs of the derivation.
Each output name is passed to the [`builder`](#attr-builder) executable as an environment variable with its value set to the corresponding [store path].
By default, a derivation produces a single output called `out`.
However, derivations can produce multiple outputs.
This allows the associated [store objects](@docroot@/glossary.md#gloss-store-object) and their [closures](@docroot@/glossary.md#gloss-closure) to be copied or garbage-collected separately.
> **Example**
>
> Imagine a library package that provides a dynamic library, header files, and documentation.
> A program that links against such a library doesn’t need the header files and documentation at runtime, and it doesn’t need the documentation at build time.
> Thus, the library package could specify:
>
> ```nix
> derivation {
> # ...
> outputs = [ "lib" "dev" "doc" ];
> # ...
> }
> ```
>
> This will cause Nix to pass environment variables `lib`, `dev`, and `doc` to the builder containing the intended store paths of each output.
> The builder would typically do something like
>
> ```bash
> ./configure \
> --libdir=$lib/lib \
> --includedir=$dev/include \
> --docdir=$doc/share/doc
> ```
>
> for an Autoconf-style package.
The name of an output is combined with the name of the derivation to create the name part of the output's store path, unless it is `out`, in which case just the name of the derivation is used.
> **Example**
>
>
> ```nix
> derivation {
> name = "example";
> outputs = [ "lib" "dev" "doc" "out" ];
> # ...
> }
> ```
>
> The store derivation path will be `/nix/store/<hash>-example.drv`.
> The output paths will be
> - `/nix/store/<hash>-example-lib`
> - `/nix/store/<hash>-example-dev`
> - `/nix/store/<hash>-example-doc`
> - `/nix/store/<hash>-example`
You can refer to each output of a derivation by selecting it as an attribute.
The first element of `outputs` determines the *default output* and ends up at the top-level.
> **Example**
>
> Select an output by attribute name:
>
> ```nix
> let
> myPackage = derivation {
> name = "example";
> outputs = [ "lib" "dev" "doc" "out" ];
> # ...
> };
> in myPackage.dev
> ```
>
> Since `lib` is the first output, `myPackage` is equivalent to `myPackage.lib`.
<!-- FIXME: refer to the output attributes when we have one -->
- See [Advanced Attributes](./advanced-attributes.md) for more, infrequently used, optional attributes.
<!-- FIXME: This should be moved here -->
- Every other attribute is passed as an environment variable to the builder.
Attribute values are translated to environment variables as follows:
- Strings are passed unchanged.
- Integral numbers are converted to decimal notation.
- Floating point numbers are converted to simple decimal or scientific notation with a preset precision.
- A *path* (e.g., `../foo/sources.tar`) causes the referenced file
to be copied to the store; its location in the store is put in
the environment variable. The idea is that all sources should
reside in the Nix store, since all inputs to a derivation should
reside in the Nix store.
- A *derivation* causes that derivation to be built prior to the
present derivation. The environment variable is set to the [store path] of the derivation's default [output](#attr-outputs).
- Lists of the previous types are also allowed. They are simply
concatenated, separated by spaces.
- `true` is passed as the string `1`, `false` and `null` are
passed as an empty string.
<!-- FIXME: add a section on output attributes -->
## Builder execution
The [`builder`](#attr-builder) is executed as follows:
- A temporary directory is created under the directory specified by
`TMPDIR` (default `/tmp`) where the build will take place. The
current directory is changed to this directory.
- The environment is cleared and set to the derivation attributes, as
specified above.
- In addition, the following variables are set:
- `NIX_BUILD_TOP` contains the path of the temporary directory for
this build.
- Also, `TMPDIR`, `TEMPDIR`, `TMP`, `TEMP` are set to point to the
temporary directory. This is to prevent the builder from
accidentally writing temporary files anywhere else. Doing so
might cause interference by other processes.
- `PATH` is set to `/path-not-set` to prevent shells from
initialising it to their built-in default value.
- `HOME` is set to `/homeless-shelter` to prevent programs from
using `/etc/passwd` or the like to find the user's home
directory, which could cause impurity. Usually, when `HOME` is
set, it is used as the location of the home directory, even if
it points to a non-existent path.
- `NIX_STORE` is set to the path of the top-level Nix store
directory (typically, `/nix/store`).
- `NIX_ATTRS_JSON_FILE` & `NIX_ATTRS_SH_FILE` if `__structuredAttrs`
is set to `true` for the dervation. A detailed explanation of this
behavior can be found in the
[section about structured attrs](./advanced-attributes.md#adv-attr-structuredAttrs).
- For each output declared in `outputs`, the corresponding
environment variable is set to point to the intended path in the
Nix store for that output. Each output path is a concatenation
of the cryptographic hash of all build inputs, the `name`
attribute and the output name. (The output name is omitted if
it’s `out`.)
- If an output path already exists, it is removed. Also, locks are
acquired to prevent multiple Nix instances from performing the same
build at the same time.
- A log of the combined standard output and error is written to
`/nix/var/log/nix`.
- The builder is executed with the arguments specified by the
attribute `args`. If it exits with exit code 0, it is considered to
have succeeded.
- The temporary directory is removed (unless the `-K` option was
specified).
- If the build was successful, Nix scans each output path for
references to input paths by looking for the hash parts of the input
paths. Since these are potential runtime dependencies, Nix registers
them as dependencies of the output paths.
- After the build, Nix sets the last-modified timestamp on all files
in the build result to 1 (00:00:01 1/1/1970 UTC), sets the group to
the default group, and sets the mode of the file to 0444 or 0555
(i.e., read-only, with execute permission enabled if the file was
originally executable). Note that possible `setuid` and `setgid`
bits are cleared. Setuid and setgid programs are not currently
supported by Nix. This is because the Nix archives used in
deployment have no concept of ownership information, and because it
makes the build result dependent on the user performing the build.
@@ -0,0 +1,139 @@
# Import From Derivation
The value of a Nix expression can depend on the contents of a [store object](@docroot@/glossary.md#gloss-store-object).
Passing an expression `expr` that evaluates to a [store path](@docroot@/glossary.md#gloss-store-path) to any built-in function which reads from the filesystem constitutes Import From Derivation (IFD):
- [`import`](./builtins.md#builtins-import)` expr`
- [`builtins.readFile`](./builtins.md#builtins-readFile)` expr`
- [`builtins.readFileType`](./builtins.md#builtins-readFileType)` expr`
- [`builtins.readDir`](./builtins.md#builtins-readDir)` expr`
- [`builtins.pathExists`](./builtins.md#builtins-pathExists)` expr`
- [`builtins.filterSource`](./builtins.md#builtins-filterSource)` f expr`
- [`builtins.path`](./builtins.md#builtins-path)` { path = expr; }`
- [`builtins.hashFile`](./builtins.md#builtins-hashFile)` t expr`
- `builtins.scopedImport x drv`
When the store path needs to be accessed, evaluation will be paused, the corresponding store object [realised], and then evaluation resumed.
[realised]: @docroot@/glossary.md#gloss-realise
This has performance implications:
Evaluation can only finish when all required store objects are realised.
Since the Nix language evaluator is sequential, it only finds store paths to read from one at a time.
While realisation is always parallel, in this case it cannot be done for all required store paths at once, and is therefore much slower than otherwise.
Realising store objects during evaluation can be disabled by setting [`allow-import-from-derivation`](../command-ref/conf-file.md#conf-allow-import-from-derivation) to `false`.
Without IFD it is ensured that evaluation is complete and Nix can produce a build plan before starting any realisation.
## Example
In the following Nix expression, the inner derivation `drv` produces a file with contents `hello`.
```nix
# IFD.nix
let
drv = derivation {
name = "hello";
builder = "/bin/sh";
args = [ "-c" "echo -n hello > $out" ];
system = builtins.currentSystem;
};
in "${builtins.readFile drv} world"
```
```shellSession
nix-instantiate IFD.nix --eval --read-write-mode
```
```
building '/nix/store/348q1cal6sdgfxs8zqi9v8llrsn4kqkq-hello.drv'...
"hello world"
```
The contents of the derivation's output have to be [realised] before they can be read with [`readFile`](./builtins.md#builtins-readFile).
Only then evaluation can continue to produce the final result.
## Illustration
As a first approximation, the following data flow graph shows how evaluation and building are interleaved, if the value of a Nix expression depends on realising a [store object].
Boxes are data structures, arrow labels are transformations.
```
+----------------------+ +------------------------+
| Nix evaluator | | Nix store |
| .----------------. | | |
| | Nix expression | | | |
| '----------------' | | |
| | | | |
| evaluate | | |
| | | | |
| V | | |
| .------------. | | .------------------. |
| | derivation |----|-instantiate-|->| store derivation | |
| '------------' | | '------------------' |
| | | | |
| | | realise |
| | | | |
| | | V |
| .----------------. | | .--------------. |
| | Nix expression |<-|----read-----|----| store object | |
| '----------------' | | '--------------' |
| | | | |
| evaluate | | |
| | | | |
| V | | |
| .------------. | | |
| | value | | | |
| '------------' | | |
+----------------------+ +------------------------+
```
In more detail, the following sequence diagram shows how the expression is evaluated step by step, and where evaluation is blocked to wait for the build output to appear.
```
.-------. .-------------. .---------.
|Nix CLI| |Nix evaluator| |Nix store|
'-------' '-------------' '---------'
| | |
|evaluate IFD.nix| |
|--------------->| |
| | |
| evaluate `"${readFile drv} world"` |
| | |
| evaluate `readFile drv` |
| | |
| evaluate `drv` as string |
| | |
| |instantiate /nix/store/...-hello.drv|
| |----------------------------------->|
| : |
| : realise /nix/store/...-hello.drv |
| :----------------------------------->|
| : |
| |--------.
| : | |
| (evaluation blocked) | echo hello > $out
| : | |
| |<-------'
| : /nix/store/...-hello |
| |<-----------------------------------|
| | |
| resume `readFile /nix/store/...-hello` |
| | |
| | readFile /nix/store/...-hello |
| |----------------------------------->|
| | |
| | hello |
| |<-----------------------------------|
| | |
| resume `"${"hello"} world"` |
| | |
| resume `"hello world"` |
| | |
| "hello world" | |
|<---------------| |
.-------. .-------------. .---------.
|Nix CLI| |Nix evaluator| |Nix store|
'-------' '-------------' '---------'
```
+2 -1
View File
@@ -83,7 +83,8 @@ This is an incomplete overview of language features, by example.
</td>
<td>
A multi-line string. Strips common prefixed whitespace. Evaluates to `"multi\n line\n string"`.
<!-- FIXME: using two no-break spaces, because apparently mdBook swallows the second regular space! -->
A multi-line string. Strips common prefixed whitespace. Evaluates to `"multi\n line\n  string"`.
</td>
</tr>
+21 -1
View File
@@ -25,7 +25,7 @@
| Inequality | *expr* `!=` *expr* | none | 11 |
| Logical conjunction (`AND`) | *bool* `&&` *bool* | left | 12 |
| Logical disjunction (`OR`) | *bool* <code>\|\|</code> *bool* | left | 13 |
| [Logical implication] | *bool* `->` *bool* | none | 14 |
| [Logical implication] | *bool* `->` *bool* | right | 14 |
[string]: ./values.md#type-string
[path]: ./values.md#type-path
@@ -35,6 +35,8 @@
## Attribute selection
> **Syntax**
>
> *attrset* `.` *attrpath* \[ `or` *expr* \]
Select the attribute denoted by attribute path *attrpath* from [attribute set] *attrset*.
@@ -42,21 +44,29 @@ If the attribute doesn’t exist, return the *expr* after `or` if provided, othe
An attribute path is a dot-separated list of [attribute names](./values.md#attribute-set).
> **Syntax**
>
> *attrpath* = *name* [ `.` *name* ]...
[Attribute selection]: #attribute-selection
## Has attribute
> **Syntax**
>
> *attrset* `?` *attrpath*
Test whether [attribute set] *attrset* contains the attribute denoted by *attrpath*.
The result is a [Boolean] value.
See also: [`builtins.hasAttr`](@docroot@/language/builtins.md#builtins-hasAttr)
[Boolean]: ./values.md#type-boolean
[Has attribute]: #has-attribute
After evaluating *attrset* and *attrpath*, the computational complexity is O(log(*n*)) for *n* attributes in the *attrset*
## Arithmetic
Numbers are type-compatible:
@@ -70,6 +80,8 @@ The `+` operator is overloaded to also work on strings and paths.
## String concatenation
> **Syntax**
>
> *string* `+` *string*
Concatenate two [string]s and merge their string contexts.
@@ -78,6 +90,8 @@ Concatenate two [string]s and merge their string contexts.
## Path concatenation
> **Syntax**
>
> *path* `+` *path*
Concatenate two [path]s.
@@ -87,6 +101,8 @@ The result is a path.
## Path and string concatenation
> **Syntax**
>
> *path* + *string*
Concatenate *[path]* with *[string]*.
@@ -100,6 +116,8 @@ The result is a path.
## String and path concatenation
> **Syntax**
>
> *string* + *path*
Concatenate *[string]* with *[path]*.
@@ -117,6 +135,8 @@ The result is a string.
## Update
> **Syntax**
>
> *attrset1* // *attrset2*
Update [attribute set] *attrset1* with names and values from *attrset2*.
+134 -18
View File
@@ -1,19 +1,12 @@
# String interpolation
String interpolation is a language feature where a [string], [path], or [attribute name] can contain expressions enclosed in `${ }` (dollar-sign with curly brackets).
String interpolation is a language feature where a [string], [path], or [attribute name][attribute set] can contain expressions enclosed in `${ }` (dollar-sign with curly brackets).
Such a string is an *interpolated string*, and an expression inside is an *interpolated expression*.
Interpolated expressions must evaluate to one of the following:
- a [string]
- a [path]
- a [derivation]
Such a construct is called *interpolated string*, and the expression inside is an [interpolated expression](#interpolated-expression).
[string]: ./values.md#type-string
[path]: ./values.md#type-path
[attribute name]: ./values.md#attribute-set
[derivation]: ../glossary.md#gloss-derivation
[attribute set]: ./values.md#attribute-set
## Examples
@@ -70,13 +63,136 @@ you can instead write
### Attribute name
Attribute names can be created dynamically with string interpolation:
<!--
FIXME: these examples are redundant with the main page on attribute sets.
figure out what to do about that
-->
```nix
let name = "foo"; in
{
${name} = "bar";
}
```
Attribute names can be interpolated strings.
{ foo = "bar"; }
> **Example**
>
> ```nix
> let name = "foo"; in
> { ${name} = 123; }
> ```
>
> { foo = 123; }
Attributes can be selected with interpolated strings.
> **Example**
>
> ```nix
> let name = "foo"; in
> { foo = 123; }.${name}
> ```
>
> 123
# Interpolated expression
An expression that is interpolated must evaluate to one of the following:
- a [string]
- a [path]
- an [attribute set] that has a `__toString` attribute or an `outPath` attribute
- `__toString` must be a function that takes the attribute set itself and returns a string
- `outPath` must be a string
This includes [derivations](./derivations.md) or [flake inputs](@docroot@/command-ref/new-cli/nix3-flake.md#flake-inputs) (experimental).
A string interpolates to itself.
A path in an interpolated expression is first copied into the Nix store, and the resulting string is the [store path] of the newly created [store object](../glossary.md#gloss-store-object).
[store path]: ../glossary.md#gloss-store-path
> **Example**
>
> ```console
> $ mkdir foo
> ```
>
> Reference the empty directory in an interpolated expression:
>
> ```nix
> "${./foo}"
> ```
>
> "/nix/store/2hhl2nz5v0khbn06ys82nrk99aa1xxdw-foo"
A derivation interpolates to the [store path] of its first [output](./derivations.md#attr-outputs).
> **Example**
>
> ```nix
> let
> pkgs = import <nixpkgs> {};
> in
> "${pkgs.hello}"
> ```
>
> "/nix/store/4xpfqf29z4m8vbhrqcz064wfmb46w5r7-hello-2.12.1"
An attribute set interpolates to the return value of the function in the `__toString` applied to the attribute set itself.
> **Example**
>
> ```nix
> let
> a = {
> value = 1;
> __toString = self: toString (self.value + 1);
> };
> in
> "${a}"
> ```
>
> "2"
An attribute set also interpolates to the value of its `outPath` attribute.
> **Example**
>
> ```nix
> let
> a = { outPath = "foo"; };
> in
> "${a}"
> ```
>
> "foo"
If both `__toString` and `outPath` are present in an attribute set, `__toString` takes precedence.
> **Example**
>
> ```nix
> let
> a = { __toString = _: "yes"; outPath = throw "no"; };
> in
> "${a}"
> ```
>
> "yes"
If neither is present, an error is thrown.
> **Example**
>
> ```nix
> let
> a = {};
> in
> "${a}"
> ```
>
> error: cannot coerce a set to a string
>
> at «string»:4:2:
>
> 3| in
> 4| "${a}"
> | ^
+13 -13
View File
@@ -107,29 +107,25 @@
e.g. `~/foo` would be equivalent to `/home/edolstra/foo` for a user
whose home directory is `/home/edolstra`.
Paths can also be specified between angle brackets, e.g.
`<nixpkgs>`. This means that the directories listed in the
environment variable `NIX_PATH` will be searched for the given file
or directory name.
When an [interpolated string][string interpolation] evaluates to a path, the path is first copied into the Nix store and the resulting string is the [store path] of the newly created [store object].
[store path]: ../glossary.md#gloss-store-path
[store object]: ../glossary.md#gloss-store-object
For instance, evaluating `"${./foo.txt}"` will cause `foo.txt` in the current directory to be copied into the Nix store and result in the string `"/nix/store/<hash>-foo.txt"`.
Note that the Nix language assumes that all input files will remain _unchanged_ while evaluating a Nix expression.
For example, assume you used a file path in an interpolated string during a `nix repl` session.
Later in the same session, after having changed the file contents, evaluating the interpolated string with the file path again might not return a new store path, since Nix might not re-read the file contents.
Later in the same session, after having changed the file contents, evaluating the interpolated string with the file path again might not return a new [store path], since Nix might not re-read the file contents.
Paths themselves, except those in angle brackets (`< >`), support [string interpolation].
[store path]: ../glossary.md#gloss-store-path
Paths can include [string interpolation] and can themselves be [interpolated in other expressions].
[interpolated in other expressions]: ./string-interpolation.md#interpolated-expressions
At least one slash (`/`) must appear *before* any interpolated expression for the result to be recognized as a path.
`a.${foo}/b.${bar}` is a syntactically valid division operation.
`./a.${foo}/b.${bar}` is a path.
[Lookup paths](./constructs/lookup-path.md) such as `<nixpkgs>` resolve to path values.
- <a id="type-boolean" href="#type-boolean">Boolean</a>
*Booleans* with values `true` and `false`.
@@ -167,13 +163,17 @@ An attribute set is a collection of name-value-pairs (called *attributes*) enclo
An attribute name can be an identifier or a [string](#string).
An identifier must start with a letter (`a-z`, `A-Z`) or underscore (`_`), and can otherwise contain letters (`a-z`, `A-Z`), numbers (`0-9`), underscores (`_`), apostrophes (`'`), or dashes (`-`).
> **Syntax**
>
> *name* = *identifier* | *string* \
> *identifier* ~ `[a-zA-Z_][a-zA-Z0-9_'-]*`
Names and values are separated by an equal sign (`=`).
Each value is an arbitrary expression terminated by a semicolon (`;`).
> *attrset* = `{` [ *name* `=` *expr* `;` `]`... `}`
> **Syntax**
>
> *attrset* = `{` [ *name* `=` *expr* `;` ]... `}`
Attributes can appear in any order.
An attribute name may only occur once.
@@ -1,179 +0,0 @@
# Basic Package Management
The main command for package management is
[`nix-env`](../command-ref/nix-env.md). You can use it to install,
upgrade, and erase packages, and to query what packages are installed
or are available for installation.
In Nix, different users can have different “views” on the set of
installed applications. That is, there might be lots of applications
present on the system (possibly in many different versions), but users
can have a specific selection of those active — where “active” just
means that it appears in a directory in the user’s `PATH`. Such a view
on the set of installed applications is called a *user environment*,
which is just a directory tree consisting of symlinks to the files of
the active applications.
Components are installed from a set of *Nix expressions* that tell Nix
how to build those packages, including, if necessary, their
dependencies. There is a collection of Nix expressions called the
Nixpkgs package collection that contains packages ranging from basic
development stuff such as GCC and Glibc, to end-user applications like
Mozilla Firefox. (Nix is however not tied to the Nixpkgs package
collection; you could write your own Nix expressions based on Nixpkgs,
or completely new ones.)
You can manually download the latest version of Nixpkgs from
<https://github.com/NixOS/nixpkgs>. However, it’s much more
convenient to use the Nixpkgs [*channel*](../command-ref/nix-channel.md), since it makes
it easy to stay up to date with new versions of Nixpkgs. Nixpkgs is
automatically added to your list of “subscribed” channels when you
install Nix. If this is not the case for some reason, you can add it
as follows:
```console
$ nix-channel --add https://nixos.org/channels/nixpkgs-unstable
$ nix-channel --update
```
> **Note**
>
> On NixOS, you’re automatically subscribed to a NixOS channel
> corresponding to your NixOS major release (e.g.
> <http://nixos.org/channels/nixos-21.11>). A NixOS channel is identical
> to the Nixpkgs channel, except that it contains only Linux binaries
> and is updated only if a set of regression tests succeed.
You can view the set of available packages in Nixpkgs:
```console
$ nix-env --query --available --attr-path
nixpkgs.aterm aterm-2.2
nixpkgs.bash bash-3.0
nixpkgs.binutils binutils-2.15
nixpkgs.bison bison-1.875d
nixpkgs.blackdown blackdown-1.4.2
nixpkgs.bzip2 bzip2-1.0.2
…
```
The flag `-q` specifies a query operation, `-a` means that you want
to show the “available” (i.e., installable) packages, as opposed to the
installed packages, and `-P` prints the attribute paths that can be used
to unambiguously select a package for installation (listed in the first column).
If you downloaded Nixpkgs yourself, or if you checked it out from GitHub,
then you need to pass the path to your Nixpkgs tree using the `-f` flag:
```console
$ nix-env --query --available --attr-path --file /path/to/nixpkgs
aterm aterm-2.2
bash bash-3.0
…
```
where */path/to/nixpkgs* is where you’ve unpacked or checked out
Nixpkgs.
You can filter the packages by name:
```console
$ nix-env --query --available --attr-path firefox
nixpkgs.firefox-esr firefox-91.3.0esr
nixpkgs.firefox firefox-94.0.1
```
and using regular expressions:
```console
$ nix-env --query --available --attr-path 'firefox.*'
```
It is also possible to see the *status* of available packages, i.e.,
whether they are installed into the user environment and/or present in
the system:
```console
$ nix-env --query --available --attr-path --status
…
-PS nixpkgs.bash bash-3.0
--S nixpkgs.binutils binutils-2.15
IPS nixpkgs.bison bison-1.875d
…
```
The first character (`I`) indicates whether the package is installed in
your current user environment. The second (`P`) indicates whether it is
present on your system (in which case installing it into your user
environment would be a very quick operation). The last one (`S`)
indicates whether there is a so-called *substitute* for the package,
which is Nix’s mechanism for doing binary deployment. It just means that
Nix knows that it can fetch a pre-built package from somewhere
(typically a network server) instead of building it locally.
You can install a package using `nix-env --install --attr `. For instance,
```console
$ nix-env --install --attr nixpkgs.subversion
```
will install the package called `subversion` from `nixpkgs` channel (which is, of course, the
[Subversion version management system](http://subversion.tigris.org/)).
> **Note**
>
> When you ask Nix to install a package, it will first try to get it in
> pre-compiled form from a *binary cache*. By default, Nix will use the
> binary cache <https://cache.nixos.org>; it contains binaries for most
> packages in Nixpkgs. Only if no binary is available in the binary
> cache, Nix will build the package from source. So if `nix-env
> -iA nixpkgs.subversion` results in Nix building stuff from source, then either
> the package is not built for your platform by the Nixpkgs build
> servers, or your version of Nixpkgs is too old or too new. For
> instance, if you have a very recent checkout of Nixpkgs, then the
> Nixpkgs build servers may not have had a chance to build everything
> and upload the resulting binaries to <https://cache.nixos.org>. The
> Nixpkgs channel is only updated after all binaries have been uploaded
> to the cache, so if you stick to the Nixpkgs channel (rather than
> using a Git checkout of the Nixpkgs tree), you will get binaries for
> most packages.
Naturally, packages can also be uninstalled. Unlike when installing, you will
need to use the derivation name (though the version part can be omitted),
instead of the attribute path, as `nix-env` does not record which attribute
was used for installing:
```console
$ nix-env --uninstall subversion
```
Upgrading to a new version is just as easy. If you have a new release of
Nix Packages, you can do:
```console
$ nix-env --upgrade --attr nixpkgs.subversion
```
This will *only* upgrade Subversion if there is a “newer” version in the
new set of Nix expressions, as defined by some pretty arbitrary rules
regarding ordering of version numbers (which generally do what you’d
expect of them). To just unconditionally replace Subversion with
whatever version is in the Nix expressions, use `-i` instead of `-u`;
`-i` will remove whatever version is already installed.
You can also upgrade all packages for which there are newer versions:
```console
$ nix-env --upgrade
```
Sometimes it’s useful to be able to ask what `nix-env` would do, without
actually doing it. For instance, to find out what packages would be
upgraded by `nix-env --upgrade `, you can do
```console
$ nix-env --upgrade --dry-run
(dry run; not doing anything)
upgrading `libxslt-1.1.0' to `libxslt-1.1.10'
upgrading `graphviz-1.10' to `graphviz-1.12'
upgrading `coreutils-5.0' to `coreutils-5.2.1'
```
@@ -1,115 +0,0 @@
# Serving a Nix store via S3
Nix has [built-in support](@docroot@/command-ref/new-cli/nix3-help-stores.md#s3-binary-cache-store)
for storing and fetching store paths from
Amazon S3 and S3-compatible services. This uses the same *binary*
cache mechanism that Nix usually uses to fetch prebuilt binaries from
[cache.nixos.org](https://cache.nixos.org/).
In this example we will use the bucket named `example-nix-cache`.
## Anonymous Reads to your S3-compatible binary cache
If your binary cache is publicly accessible and does not require
authentication, the simplest and easiest way to use Nix with your S3
compatible binary cache is to use the HTTP URL for that cache.
For AWS S3 the binary cache URL for example bucket will be exactly
<https://example-nix-cache.s3.amazonaws.com> or
<s3://example-nix-cache>. For S3 compatible binary caches, consult that
cache's documentation.
Your bucket will need the following bucket policy:
```json
{
"Id": "DirectReads",
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowDirectReads",
"Action": [
"s3:GetObject",
"s3:GetBucketLocation"
],
"Effect": "Allow",
"Resource": [
"arn:aws:s3:::example-nix-cache",
"arn:aws:s3:::example-nix-cache/*"
],
"Principal": "*"
}
]
}
```
## Authenticated Reads to your S3 binary cache
For AWS S3 the binary cache URL for example bucket will be exactly
<s3://example-nix-cache>.
Nix will use the [default credential provider
chain](https://docs.aws.amazon.com/sdk-for-cpp/v1/developer-guide/credentials.html)
for authenticating requests to Amazon S3.
Nix supports authenticated reads from Amazon S3 and S3 compatible binary
caches.
Your bucket will need a bucket policy allowing the desired users to
perform the `s3:GetObject` and `s3:GetBucketLocation` action on all
objects in the bucket. The [anonymous policy given
above](#anonymous-reads-to-your-s3-compatible-binary-cache) can be
updated to have a restricted `Principal` to support this.
## Authenticated Writes to your S3-compatible binary cache
Nix support fully supports writing to Amazon S3 and S3 compatible
buckets. The binary cache URL for our example bucket will be
<s3://example-nix-cache>.
Nix will use the [default credential provider
chain](https://docs.aws.amazon.com/sdk-for-cpp/v1/developer-guide/credentials.html)
for authenticating requests to Amazon S3.
Your account will need the following IAM policy to upload to the cache:
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "UploadToCache",
"Effect": "Allow",
"Action": [
"s3:AbortMultipartUpload",
"s3:GetBucketLocation",
"s3:GetObject",
"s3:ListBucket",
"s3:ListBucketMultipartUploads",
"s3:ListMultipartUploadParts",
"s3:PutObject"
],
"Resource": [
"arn:aws:s3:::example-nix-cache",
"arn:aws:s3:::example-nix-cache/*"
]
}
]
}
```
## Examples
To upload with a specific credential profile for Amazon S3:
```console
$ nix copy nixpkgs.hello \
--to 's3://example-nix-cache?profile=cache-upload&region=eu-west-2'
```
To upload to an S3-compatible binary cache:
```console
$ nix copy nixpkgs.hello --to \
's3://example-nix-cache?profile=cache-upload&scheme=https&endpoint=minio.example.com'
```
@@ -1 +1,12 @@
# Nix Release Notes
Nix has a release cycle of roughly 6 weeks.
Notable changes and additions are announced in the release notes for each version.
Bugfixes can be backported on request to previous Nix releases.
We typically backport only as far back as the Nix version used in the latest NixOS release, which is announced in the [NixOS release notes](https://nixos.org/manual/nixos/stable/release-notes.html#ch-release-notes).
Backports never skip releases.
If a feature is backported to version `x.y`, it must also be available in version `x.(y+1)`.
This ensures that upgrading from an older version with backports is still safe and no backported functionality will go missing.
-1
View File
@@ -2,7 +2,6 @@
* On Linux, Nix can now run builds in a user namespace where they run
as root (UID 0) and have 65,536 UIDs available.
<!-- FIXME: move this to its own section about system features -->
This is primarily useful for running containers such as `systemd-nspawn`
inside a Nix build. For an example, see [`tests/systemd-nspawn/nix`][nspawn].
+77
View File
@@ -0,0 +1,77 @@
# Release 2.19 (2023-11-17)
- The experimental `nix` command can now act as a [shebang interpreter](@docroot@/command-ref/new-cli/nix.md#shebang-interpreter)
by appending the contents of any `#! nix` lines and the script's location into a single call.
- [URL flake references](@docroot@/command-ref/new-cli/nix3-flake.md#flake-references) now support [percent-encoded](https://datatracker.ietf.org/doc/html/rfc3986#section-2.1) characters.
- [Path-like flake references](@docroot@/command-ref/new-cli/nix3-flake.md#path-like-syntax) now accept arbitrary unicode characters (except `#` and `?`).
- The experimental feature `repl-flake` is no longer needed, as its functionality is now part of the `flakes` experimental feature. To get the previous behavior, use the `--file/--expr` flags accordingly.
- There is a new flake installable syntax `flakeref#.attrPath` where the "." prefix specifies that `attrPath` is interpreted from the root of the flake outputs, with no searching of default attribute prefixes like `packages.<SYSTEM>` or `legacyPackages.<SYSTEM>`.
- Nix adds `apple-virt` to the default system features on macOS systems that support virtualization. This is similar to what's done for the `kvm` system feature on Linux hosts.
- Add a new built-in function [`builtins.convertHash`](@docroot@/language/builtins.md#builtins-convertHash).
- `nix-shell` shebang lines now support single-quoted arguments.
- `builtins.fetchTree` is now its own experimental feature, [`fetch-tree`](@docroot@/contributing/experimental-features.md#xp-fetch-tree).
As described in the documentation for that feature, this is because we anticipate polishing it and then stabilizing it before the rest of flakes.
- The interface for creating and updating lock files has been overhauled:
- [`nix flake lock`](@docroot@/command-ref/new-cli/nix3-flake-lock.md) only creates lock files and adds missing inputs now.
It will *never* update existing inputs.
- [`nix flake update`](@docroot@/command-ref/new-cli/nix3-flake-update.md) does the same, but *will* update inputs.
- Passing no arguments will update all inputs of the current flake, just like it already did.
- Passing input names as arguments will ensure only those are updated. This replaces the functionality of `nix flake lock --update-input`
- To operate on a flake outside the current directory, you must now pass `--flake path/to/flake`.
- The flake-specific flags `--recreate-lock-file` and `--update-input` have been removed from all commands operating on installables.
They are superceded by `nix flake update`.
- Commit signature verification for the [`builtins.fetchGit`](@docroot@/language/builtins.md#builtins-fetchGit) is added as the new [`verified-fetches` experimental feature](@docroot@/contributing/experimental-features.md#xp-feature-verified-fetches).
- [`nix path-info --json`](@docroot@/command-ref/new-cli/nix3-path-info.md)
(experimental) now returns a JSON map rather than JSON list.
The `path` field of each object has instead become the key in the outer map, since it is unique.
The `valid` field also goes away because we just use `null` instead.
- Old way:
```json5
[
{
"path": "/nix/store/8fv91097mbh5049i9rglc73dx6kjg3qk-bash-5.2-p15",
"valid": true,
// ...
},
{
"path": "/nix/store/wffw7l0alvs3iw94cbgi1gmmbmw99sqb-home-manager-path",
"valid": false
}
]
```
- New way
```json5
{
"/nix/store/8fv91097mbh5049i9rglc73dx6kjg3qk-bash-5.2-p15": {
// ...
},
"/nix/store/wffw7l0alvs3iw94cbgi1gmmbmw99sqb-home-manager-path": null,
}
```
This makes it match `nix derivation show`, which also maps store paths to information.
- When Nix is installed using the [binary installer](@docroot@/installation/installing-binary.md), in supported shells (Bash, Zsh, Fish)
[`XDG_DATA_DIRS`](https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html#variables) is now populated with the path to the `/share` subdirectory of the current profile.
This means that command completion scripts, `.desktop` files, and similar artifacts installed via [`nix-env`](@docroot@/command-ref/nix-env.md) or [`nix profile`](@docroot@/command-ref/new-cli/nix3-profile.md)
(experimental) can be found by any program that follows the [XDG Base Directory Specification](https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html).
- A new command `nix store add` has been added. It replaces `nix store add-file` and `nix store add-path` which are now deprecated.
-1
View File
@@ -1 +0,0 @@
# Release X.Y (202?-??-??)
+5
View File
@@ -0,0 +1,5 @@
# Nix Store
The *Nix store* is an abstraction to store immutable file system data (such as software packages) that can have dependencies on other such data.
There are multiple implementations of Nix stores with different capabilities, such as the actual filesystem (`/nix/store`) or binary caches.
+10
View File
@@ -0,0 +1,10 @@
## Store Object
A Nix store is a collection of *store objects* with *references* between them.
A store object consists of
- A [file system object](./file-system-object.md) as data
- A set of [store paths](./store-path.md) as references to other store objects
Store objects are [immutable](https://en.wikipedia.org/wiki/Immutable_object):
Once created, they do not change until they are deleted.
+69
View File
@@ -0,0 +1,69 @@
# Store Path
Nix implements references to [store objects](./index.md#store-object) as *store paths*.
Think of a store path as an [opaque], [unique identifier]:
The only way to obtain store path is by adding or building store objects.
A store path will always reference exactly one store object.
[opaque]: https://en.m.wikipedia.org/wiki/Opaque_data_type
[unique identifier]: https://en.m.wikipedia.org/wiki/Unique_identifier
Store paths are pairs of
- A 20-byte digest for identification
- A symbolic name for people to read
> **Example**
>
> - Digest: `b6gvzjyb2pg0kjfwrjmg1vfhh54ad73z`
> - Name: `firefox-33.1`
To make store objects accessible to operating system processes, stores have to expose store objects through the file system.
A store path is rendered to a file system path as the concatenation of
- [Store directory](#store-directory) (typically `/nix/store`)
- Path separator (`/`)
- Digest rendered in a custom variant of [Base32](https://en.wikipedia.org/wiki/Base32) (20 arbitrary bytes become 32 ASCII characters)
- Hyphen (`-`)
- Name
> **Example**
>
> ```
> /nix/store/b6gvzjyb2pg0kjfwrjmg1vfhh54ad73z-firefox-33.1
> |--------| |------------------------------| |----------|
> store directory digest name
> ```
## Store Directory
Every [Nix store](./index.md) has a store directory.
Not every store can be accessed through the file system.
But if the store has a file system representation, the store directory contains the store’s [file system objects], which can be addressed by [store paths](#store-path).
[file system objects]: ./file-system-object.md
This means a store path is not just derived from the referenced store object itself, but depends on the store the store object is in.
> **Note**
>
> The store directory defaults to `/nix/store`, but is in principle arbitrary.
It is important which store a given store object belongs to:
Files in the store object can contain store paths, and processes may read these paths.
Nix can only guarantee referential integrity if store paths do not cross store boundaries.
Therefore one can only copy store objects to a different store if
- The source and target stores' directories match
or
- The store object in question has no references, that is, contains no store paths
One cannot copy a store object to a store with a different store directory.
Instead, it has to be rebuilt, together with all its dependencies.
It is in general not enough to replace the store directory string in file contents, as this may render executables unusable by invalidating their internal offsets or checksums.
-57
View File
@@ -44,63 +44,6 @@ rec {
optionalString = cond: string: if cond then string else "";
showSetting = { useAnchors }: name: { description, documentDefault, defaultValue, aliases, value, experimentalFeature }:
let
result = squash ''
- ${if useAnchors
then ''<span id="conf-${name}">[`${name}`](#conf-${name})</span>''
else ''`${name}`''}
${indent " " body}
'';
experimentalFeatureNote = optionalString (experimentalFeature != null) ''
> **Warning**
> This setting is part of an
> [experimental feature](@docroot@/contributing/experimental-features.md).
To change this setting, you need to make sure the corresponding experimental feature,
[`${experimentalFeature}`](@docroot@/contributing/experimental-features.md#xp-feature-${experimentalFeature}),
is enabled.
For example, include the following in [`nix.conf`](#):
```
extra-experimental-features = ${experimentalFeature}
${name} = ...
```
'';
# separate body to cleanly handle indentation
body = ''
${description}
${experimentalFeatureNote}
**Default:** ${showDefault documentDefault defaultValue}
${showAliases aliases}
'';
showDefault = documentDefault: defaultValue:
if documentDefault then
# a StringMap value type is specified as a string, but
# this shows the value type. The empty stringmap is `null` in
# JSON, but that converts to `{ }` here.
if defaultValue == "" || defaultValue == [] || isAttrs defaultValue
then "*empty*"
else if isBool defaultValue then
if defaultValue then "`true`" else "`false`"
else "`${toString defaultValue}`"
else "*machine-specific*";
showAliases = aliases:
optionalString (aliases != [])
"**Deprecated alias:** ${(concatStringsSep ", " (map (s: "`${s}`") aliases))}";
in result;
indent = prefix: s:
concatStringsSep "\n" (map (x: if x == "" then x else "${prefix}${x}") (splitLines s));
showSettings = args: settingsInfo: concatStrings (attrValues (mapAttrs (showSetting args) settingsInfo));
}
Generated
+23 -6
View File
@@ -16,6 +16,22 @@
"type": "github"
}
},
"libgit2": {
"flake": false,
"locked": {
"lastModified": 1697646580,
"narHash": "sha256-oX4Z3S9WtJlwvj0uH9HlYcWv+x1hqp8mhXl7HsLu2f0=",
"owner": "libgit2",
"repo": "libgit2",
"rev": "45fd9ed7ae1a9b74b957ef4f337bc3c8b3df01b5",
"type": "github"
},
"original": {
"owner": "libgit2",
"repo": "libgit2",
"type": "github"
}
},
"lowdown-src": {
"flake": false,
"locked": {
@@ -34,16 +50,16 @@
},
"nixpkgs": {
"locked": {
"lastModified": 1695124524,
"narHash": "sha256-trXDytVCqf3KryQQQrHOZKUabu1/lB8/ndOAuZKQrOE=",
"owner": "edolstra",
"lastModified": 1700748986,
"narHash": "sha256-/nqLrNU297h3PCw4QyDpZKZEUHmialJdZW2ceYFobds=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "a3d30b525535e3158221abc1a957ce798ab159fe",
"rev": "9ba29e2346bc542e9909d1021e8fd7d4b3f64db0",
"type": "github"
},
"original": {
"owner": "edolstra",
"ref": "fix-aws-sdk-cpp",
"owner": "NixOS",
"ref": "nixos-23.05-small",
"repo": "nixpkgs",
"type": "github"
}
@@ -67,6 +83,7 @@
"root": {
"inputs": {
"flake-compat": "flake-compat",
"libgit2": "libgit2",
"lowdown-src": "lowdown-src",
"nixpkgs": "nixpkgs",
"nixpkgs-regression": "nixpkgs-regression"
+141 -135
View File
@@ -1,19 +1,22 @@
{
description = "The purely functional package manager";
#inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-23.05-small";
inputs.nixpkgs.url = "github:edolstra/nixpkgs/fix-aws-sdk-cpp";
inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-23.05-small";
inputs.nixpkgs-regression.url = "github:NixOS/nixpkgs/215d4d0fd80ca5163643b03a33fde804a29cc1e2";
inputs.lowdown-src = { url = "github:kristapsdz/lowdown"; flake = false; };
inputs.flake-compat = { url = "github:edolstra/flake-compat"; flake = false; };
inputs.libgit2 = { url = "github:libgit2/libgit2"; flake = false; };
outputs = { self, nixpkgs, nixpkgs-regression, lowdown-src, flake-compat }:
outputs = { self, nixpkgs, nixpkgs-regression, lowdown-src, flake-compat, libgit2 }:
let
inherit (nixpkgs) lib;
officialRelease = false;
# Set to true to build the release notes for the next release.
buildUnreleasedNotes = false;
version = lib.fileContents ./.version + versionSuffix;
versionSuffix =
if officialRelease
@@ -25,8 +28,11 @@
linuxSystems = linux32BitSystems ++ linux64BitSystems;
darwinSystems = [ "x86_64-darwin" "aarch64-darwin" ];
systems = linuxSystems ++ darwinSystems;
crossSystems = [ "armv6l-linux" "armv7l-linux" ];
crossSystems = [
"armv6l-linux" "armv7l-linux"
"x86_64-freebsd13" "x86_64-netbsd"
];
stdenvs = [ "gccStdenv" "clangStdenv" "clang11Stdenv" "stdenv" "libcxxStdenv" "ccacheStdenv" ];
@@ -57,44 +63,55 @@
# that would interfere with repo semantics.
fileset.fileFilter (f: f.name != ".gitignore") ./.;
configureFiles = fileset.unions [
./.version
./configure.ac
./m4
# TODO: do we really need README.md? It doesn't seem used in the build.
./README.md
];
topLevelBuildFiles = fileset.unions [
./local.mk
./Makefile
./Makefile.config.in
./mk
];
functionalTestFiles = fileset.unions [
./tests/functional
(fileset.fileFilter (f: lib.strings.hasPrefix "nix-profile" f.name) ./scripts)
];
nixSrc = fileset.toSource {
root = ./.;
fileset = fileset.intersect baseFiles (
fileset.difference
(fileset.unions [
./.version
./boehmgc-coroutine-sp-fallback.diff
./bootstrap.sh
./configure.ac
./doc
./local.mk
./m4
./Makefile
./Makefile.config.in
./misc
./mk
./precompiled-headers.h
./src
./tests
./COPYING
./scripts/local.mk
(fileset.fileFilter (f: lib.strings.hasPrefix "nix-profile" f.name) ./scripts)
# TODO: do we really need README.md? It doesn't seem used in the build.
./README.md
])
(fileset.unions [
# Removed file sets
./tests/nixos
./tests/installer
])
);
fileset = fileset.intersect baseFiles (fileset.unions [
configureFiles
topLevelBuildFiles
./boehmgc-coroutine-sp-fallback.diff
./doc
./misc
./precompiled-headers.h
./src
./unit-test-data
./COPYING
./scripts/local.mk
functionalTestFiles
]);
};
# Memoize nixpkgs for different platforms for efficiency.
nixpkgsFor = forAllSystems
(system: let
make-pkgs = crossSystem: stdenv: import nixpkgs {
inherit system crossSystem;
localSystem = {
inherit system;
};
crossSystem = if crossSystem == null then null else {
system = crossSystem;
} // lib.optionalAttrs (crossSystem == "x86_64-freebsd13") {
useLLVM = true;
};
overlays = [
(overlayFor (p: p.${stdenv}))
];
@@ -140,7 +157,7 @@
configureFlags =
lib.optionals stdenv.isLinux [
"--with-boost=${boost}/lib"
"--with-boost=${boost-nix}/lib"
"--with-sandbox-shell=${sh}/bin/busybox"
]
++ lib.optionals (stdenv.isLinux && !(isStatic && stdenv.system == "aarch64-linux")) [
@@ -149,12 +166,18 @@
testConfigureFlags = [
"RAPIDCHECK_HEADERS=${lib.getDev rapidcheck}/extras/gtest/include"
] ++ lib.optionals (stdenv.hostPlatform != stdenv.buildPlatform) [
"--enable-install-unit-tests"
"--with-check-bin-dir=${builtins.placeholder "check"}/bin"
"--with-check-lib-dir=${builtins.placeholder "check"}/lib"
];
internalApiDocsConfigureFlags = [
"--enable-internal-api-docs"
];
changelog-d = pkgs.buildPackages.changelog-d;
nativeBuildDeps =
[
buildPackages.bison
@@ -170,19 +193,28 @@
buildPackages.git
buildPackages.mercurial # FIXME: remove? only needed for tests
buildPackages.jq # Also for custom mdBook preprocessor.
buildPackages.openssh # only needed for tests (ssh-keygen)
]
++ lib.optionals stdenv.hostPlatform.isLinux [(buildPackages.util-linuxMinimal or buildPackages.utillinuxMinimal)];
++ lib.optionals stdenv.hostPlatform.isLinux [(buildPackages.util-linuxMinimal or buildPackages.utillinuxMinimal)]
# Official releases don't have rl-next, so we don't need to compile a changelog
++ lib.optional (!officialRelease && buildUnreleasedNotes) changelog-d
;
buildDeps =
[ curl
bzip2 xz brotli editline
openssl sqlite
libarchive
boost
(pkgs.libgit2.overrideAttrs (attrs: {
src = libgit2;
version = libgit2.lastModifiedDate;
cmakeFlags = (attrs.cmakeFlags or []) ++ ["-DUSE_SSH=exec"];
}))
boost-nix
lowdown-nix
libsodium
]
++ lib.optionals stdenv.isLinux [libseccomp]
++ lib.optional (stdenv.isLinux || stdenv.isDarwin) libsodium
++ lib.optional stdenv.hostPlatform.isx86_64 libcpuid;
checkDeps = [
@@ -258,7 +290,14 @@
"-${client.version}-against-${daemon.version}";
inherit version;
src = nixSrc;
src = fileset.toSource {
root = ./.;
fileset = fileset.intersect baseFiles (fileset.unions [
configureFiles
topLevelBuildFiles
functionalTestFiles
]);
};
VERSION_SUFFIX = versionSuffix;
@@ -268,7 +307,9 @@
enableParallelBuilding = true;
configureFlags = testConfigureFlags; # otherwise configure fails
configureFlags =
testConfigureFlags # otherwise configure fails
++ [ "--disable-build" ];
dontBuild = true;
doInstallCheck = true;
@@ -276,7 +317,10 @@
mkdir -p $out
'';
installCheckPhase = "make installcheck -j$NIX_BUILD_CORES -l$NIX_BUILD_CORES";
installCheckPhase = ''
mkdir -p src/nix-channel
make installcheck -j$NIX_BUILD_CORES -l$NIX_BUILD_CORES
'';
};
binaryTarball = nix: pkgs:
@@ -376,7 +420,8 @@
src = nixSrc;
VERSION_SUFFIX = versionSuffix;
outputs = [ "out" "dev" "doc" ];
outputs = [ "out" "dev" "doc" ]
++ lib.optional (currentStdenv.hostPlatform != currentStdenv.buildPlatform) "check";
nativeBuildInputs = nativeBuildDeps;
buildInputs = buildDeps
@@ -386,14 +431,14 @@
propagatedBuildInputs = propagatedDeps;
disallowedReferences = [ boost ];
disallowedReferences = [ boost-nix ];
preConfigure = lib.optionalString (! currentStdenv.hostPlatform.isStatic)
''
# Copy libboost_context so we don't get all of Boost in our closure.
# https://github.com/NixOS/nixpkgs/issues/45462
mkdir -p $out/lib
cp -pd ${boost}/lib/{libboost_context*,libboost_thread*,libboost_system*} $out/lib
cp -pd ${boost-nix}/lib/{libboost_context*,libboost_thread*,libboost_system*,libboost_regex*} $out/lib
rm -f $out/lib/*.a
${lib.optionalString currentStdenv.hostPlatform.isLinux ''
chmod u+w $out/lib/*.so.*
@@ -403,9 +448,9 @@
for LIB in $out/lib/*.dylib; do
chmod u+w $LIB
install_name_tool -id $LIB $LIB
install_name_tool -delete_rpath ${boost}/lib/ $LIB || true
install_name_tool -delete_rpath ${boost-nix}/lib/ $LIB || true
done
install_name_tool -change ${boost}/lib/libboost_system.dylib $out/lib/libboost_system.dylib $out/lib/libboost_thread.dylib
install_name_tool -change ${boost-nix}/lib/libboost_system.dylib $out/lib/libboost_system.dylib $out/lib/libboost_thread.dylib
''}
'';
@@ -433,9 +478,13 @@
''}
${lib.optionalString currentStdenv.isDarwin ''
install_name_tool \
-change ${boost}/lib/libboost_context.dylib \
-change ${boost-nix}/lib/libboost_context.dylib \
$out/lib/libboost_context.dylib \
$out/lib/libnixutil.dylib
install_name_tool \
-change ${boost-nix}/lib/libboost_regex.dylib \
$out/lib/libboost_regex.dylib \
$out/lib/libnixexpr.dylib
''}
'';
@@ -449,41 +498,19 @@
hardeningDisable = lib.optional stdenv.hostPlatform.isStatic "pie";
passthru.perl-bindings = with final; perl.pkgs.toPerlModule (currentStdenv.mkDerivation {
name = "nix-perl-${version}";
src = self;
nativeBuildInputs =
[ buildPackages.autoconf-archive
buildPackages.autoreconfHook
buildPackages.pkg-config
];
buildInputs =
[ nix
curl
bzip2
xz
pkgs.perl
boost
]
++ lib.optional (currentStdenv.isLinux || currentStdenv.isDarwin) libsodium
++ lib.optional currentStdenv.isDarwin darwin.apple_sdk.frameworks.Security;
configureFlags = [
"--with-dbi=${perlPackages.DBI}/${pkgs.perl.libPrefix}"
"--with-dbd-sqlite=${perlPackages.DBDSQLite}/${pkgs.perl.libPrefix}"
];
enableParallelBuilding = true;
postUnpack = "sourceRoot=$sourceRoot/perl";
});
passthru.perl-bindings = final.callPackage ./perl {
inherit fileset;
stdenv = currentStdenv;
};
meta.platforms = lib.platforms.unix;
meta.mainProgram = "nix";
});
boost-nix = final.boost.override {
enableIcu = false;
};
lowdown-nix = with final; currentStdenv.mkDerivation rec {
name = "lowdown-0.9.0";
@@ -502,18 +529,6 @@
};
};
nixos-lib = import (nixpkgs + "/nixos/lib") { };
# https://nixos.org/manual/nixos/unstable/index.html#sec-calling-nixos-tests
runNixOSTestFor = system: test: nixos-lib.runTest {
imports = [ test ];
hostPkgs = nixpkgsFor.${system}.native;
defaults = {
nixpkgs.pkgs = nixpkgsFor.${system}.native;
};
_module.args.nixpkgs = nixpkgs;
};
in {
# A Nixpkgs overlay that overrides the 'nix' and
# 'nix.perl-bindings' packages.
@@ -620,49 +635,29 @@
};
# System tests.
tests.authorization = runNixOSTestFor "x86_64-linux" ./tests/nixos/authorization.nix;
tests = import ./tests/nixos { inherit lib nixpkgs nixpkgsFor; } // {
tests.remoteBuilds = runNixOSTestFor "x86_64-linux" ./tests/nixos/remote-builds.nix;
# Make sure that nix-env still produces the exact same result
# on a particular version of Nixpkgs.
evalNixpkgs =
with nixpkgsFor.x86_64-linux.native;
runCommand "eval-nixos" { buildInputs = [ nix ]; }
''
type -p nix-env
# Note: we're filtering out nixos-install-tools because https://github.com/NixOS/nixpkgs/pull/153594#issuecomment-1020530593.
time nix-env --store dummy:// -f ${nixpkgs-regression} -qaP --drv-path | sort | grep -v nixos-install-tools > packages
[[ $(sha1sum < packages | cut -c1-40) = ff451c521e61e4fe72bdbe2d0ca5d1809affa733 ]]
mkdir $out
'';
tests.nix-copy-closure = runNixOSTestFor "x86_64-linux" ./tests/nixos/nix-copy-closure.nix;
tests.nix-copy = runNixOSTestFor "x86_64-linux" ./tests/nixos/nix-copy.nix;
tests.nssPreload = runNixOSTestFor "x86_64-linux" ./tests/nixos/nss-preload.nix;
tests.githubFlakes = runNixOSTestFor "x86_64-linux" ./tests/nixos/github-flakes.nix;
tests.sourcehutFlakes = runNixOSTestFor "x86_64-linux" ./tests/nixos/sourcehut-flakes.nix;
tests.tarballFlakes = runNixOSTestFor "x86_64-linux" ./tests/nixos/tarball-flakes.nix;
tests.containers = runNixOSTestFor "x86_64-linux" ./tests/nixos/containers/containers.nix;
tests.setuid = lib.genAttrs
["i686-linux" "x86_64-linux"]
(system: runNixOSTestFor system ./tests/nixos/setuid.nix);
# Make sure that nix-env still produces the exact same result
# on a particular version of Nixpkgs.
tests.evalNixpkgs =
with nixpkgsFor.x86_64-linux.native;
runCommand "eval-nixos" { buildInputs = [ nix ]; }
''
type -p nix-env
# Note: we're filtering out nixos-install-tools because https://github.com/NixOS/nixpkgs/pull/153594#issuecomment-1020530593.
time nix-env --store dummy:// -f ${nixpkgs-regression} -qaP --drv-path | sort | grep -v nixos-install-tools > packages
[[ $(sha1sum < packages | cut -c1-40) = ff451c521e61e4fe72bdbe2d0ca5d1809affa733 ]]
mkdir $out
'';
tests.nixpkgsLibTests =
forAllSystems (system:
import (nixpkgs + "/lib/tests/release.nix")
{ pkgs = nixpkgsFor.${system}.native;
nixVersions = [ self.packages.${system}.nix ];
}
);
nixpkgsLibTests =
forAllSystems (system:
import (nixpkgs + "/lib/tests/release.nix")
{ pkgs = nixpkgsFor.${system}.native;
nixVersions = [ self.packages.${system}.nix ];
}
);
};
metrics.nixpkgs = import "${nixpkgs-regression}/pkgs/top-level/metrics.nix" {
pkgs = nixpkgsFor.x86_64-linux.native;
@@ -733,20 +728,31 @@
devShells = let
makeShell = pkgs: stdenv:
let
canRunInstalled = stdenv.buildPlatform.canExecute stdenv.hostPlatform;
in
with commonDeps { inherit pkgs; };
stdenv.mkDerivation {
name = "nix";
outputs = [ "out" "dev" "doc" ];
outputs = [ "out" "dev" "doc" ]
++ lib.optional (stdenv.hostPlatform != stdenv.buildPlatform) "check";
nativeBuildInputs = nativeBuildDeps
++ (lib.optionals stdenv.cc.isClang [ pkgs.bear pkgs.clang-tools ]);
++ lib.optional stdenv.cc.isClang pkgs.buildPackages.bear
++ lib.optional
(stdenv.cc.isClang && stdenv.hostPlatform == stdenv.buildPlatform)
pkgs.buildPackages.clang-tools
# We want changelog-d in the shell even if the current build doesn't need it
++ lib.optional (officialRelease || ! buildUnreleasedNotes) changelog-d
;
buildInputs = buildDeps ++ propagatedDeps
++ awsDeps ++ checkDeps ++ internalApiDocsDeps;
configureFlags = configureFlags
++ testConfigureFlags ++ internalApiDocsConfigureFlags;
++ testConfigureFlags ++ internalApiDocsConfigureFlags
++ lib.optional (!canRunInstalled) "--disable-doc-gen";
enableParallelBuilding = true;
-2
View File
@@ -1,5 +1,3 @@
clean-files += Makefile.config
GLOBAL_CXXFLAGS += -Wno-deprecated-declarations -Werror=switch
# Allow switch-enum to be overridden for files that do not support it, usually because of dependency headers.
ERROR_SWITCH_ENUM = -Werror=switch-enum
+64
View File
@@ -0,0 +1,64 @@
# Ensure that this bug is not present in the C++ toolchain we are using.
#
# URL for bug: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=80431
#
# The test program is from that issue, with only a slight modification
# to set an exit status instead of printing strings.
AC_DEFUN([ENSURE_NO_GCC_BUG_80431],
[
AC_MSG_CHECKING([that GCC bug 80431 is fixed])
AC_LANG_PUSH(C++)
AC_RUN_IFELSE(
[AC_LANG_PROGRAM(
[[
#include <cstdio>
static bool a = true;
static bool b = true;
struct Options { };
struct Option
{
Option(Options * options)
{
a = false;
}
~Option()
{
b = false;
}
};
struct MyOptions : Options { };
struct MyOptions2 : virtual MyOptions
{
Option foo{this};
};
]],
[[
{
MyOptions2 opts;
}
return (a << 1) | b;
]])],
[status_80431=0],
[status_80431=$?],
[
# Assume we're bug-free when cross-compiling
])
AC_LANG_POP(C++)
AS_CASE([$status_80431],
[0],[
AC_MSG_RESULT(yes)
],
[2],[
AC_MSG_RESULT(no)
AC_MSG_ERROR(Cannot build Nix with C++ compiler with this bug)
],
[
AC_MSG_RESULT(unexpected result $status_80431: not expected failure with bug, ignoring)
])
])
+7 -3
View File
@@ -2,7 +2,7 @@
## Motivation
The team's main responsibility is to set a direction for the development of Nix and ensure that the code is in good shape.
The team's main responsibility is to guide and direct the development of Nix and ensure that the code is in good shape.
We aim to achieve this by improving the contributor experience and attracting more maintainers – that is, by helping other people contributing to Nix and eventually taking responsibility – in order to scale the development process to match users' needs.
@@ -50,7 +50,9 @@ The team meets twice a week:
1. Code review on pull requests from [In review](#in-review).
2. Other chores and tasks.
Meeting notes are collected on a [collaborative scratchpad](https://pad.lassul.us/Cv7FpYx-Ri-4VjUykQOLAw), and published on Discourse under the [Nix category](https://discourse.nixos.org/c/dev/nix/50).
Meeting notes are collected on a [collaborative scratchpad](https://pad.lassul.us/Cv7FpYx-Ri-4VjUykQOLAw).
Notes on issues and pull requests are posted as comments and linked from the meeting notes, so they are easy to find from both places.
[All meeting notes](https://discourse.nixos.org/search?expanded=true&q=Nix%20team%20meeting%20minutes%20%23%20%23dev%3Anix%20in%3Atitle%20order%3Alatest_topic) are published on Discourse under the [Nix category](https://discourse.nixos.org/c/dev/nix/50).
## Project board protocol
@@ -96,8 +98,10 @@ What constitutes a trivial pull request is up to maintainers' judgement.
Pull requests and issues that are deemed important and controversial are discussed by the team during discussion meetings.
This may be where the merit of the change itself or the implementation strategy is contested by a team member.
Whenever the discussion opens up questions about the process or this team's goals, this may indicate that the change is too large in scope.
In that case it is taken off the board to be reconsidered by the author or broken down into smaller pieces that are less far-reaching and can be reviewed independently.
As a general guideline, the order of items is determined as follows:
As a general guideline, the order of items to discuss is determined as follows:
- Prioritise pull requests over issues
+179
View File
@@ -0,0 +1,179 @@
#!/usr/bin/env nix-shell
#!nix-shell -i bash ../shell.nix -I nixpkgs=channel:nixos-unstable-small
# ^^^^^^^
# Only used for bash. shell.nix goes to the flake.
# --- CONFIGURATION ---
# This does double duty for
# - including rl-next
# - marking where to insert new links (right after)
SUMMARY_MARKER_LINE='{{#include ./SUMMARY-rl-next.md}}'
# --- LIB ---
log() {
echo 1>&2 "release-notes:" "$@"
}
logcmd() {
local cmd="$1"
shift
logcmd2 "$cmd" "${*@Q}" "$cmd" "$@"
}
logcmd2() {
local fakecmd="$1"
local fakeargs="$2"
shift
shift
printf 1>&2 "release-notes: \033[34;1m$fakecmd\033[0m "
echo "$fakeargs" 1>&2
"$@"
}
die() {
# ANSI red
printf 1>&2 "release-notes: \033[31;1merror:\033[0m"
echo 1>&2 "" "$@"
exit 1
}
confirm() {
local answer
echo 1>&2 "$@" "[y/n]"
read -r answer
case "$answer" in
y|Y|yes|Yes|YES)
return 0
;;
n|N|no|No|NO)
return 1
;;
*)
echo 1>&2 "please answer y or n"
confirm "$@"
;;
esac
}
report_done() {
logcmd2 "git" "show" git -c pager.show=false show
printf 1>&2 "release-notes: \033[32;1mdone\033[0m\n"
}
# --- PARSE ARGS ---
if [[ $# -gt 0 ]]; then
die "Release notes takes no arguments, but make sure to set VERSION."
fi
# --- CHECKS ---
if [[ ! -e flake.nix ]] || [[ ! -e .git ]]; then
die "must run in repo root"
exit 1
fi
# repo must be clean
if ! git diff --quiet; then
die "repo is dirty, please commit or stash changes"
fi
if ! git diff --quiet --cached; then
die "repo has staged changes, please commit or stash them"
fi
if ! grep "$SUMMARY_MARKER_LINE" doc/manual/src/SUMMARY.md.in >/dev/null; then
# would have been nice to catch this early, but won't be worth the extra infra
die "SUMMARY.md.in is missing the marker line '$SUMMARY_MARKER_LINE', which would be used for inserting a new release notes page. Please fix the script."
fi
if [[ ! -n "${VERSION:-}" ]]; then
die "please set the VERSION environment variable before invoking this script"
exit 1
fi
# version_major_minor: MAJOR.MINOR
# version_full: MAJOR.MINOR.PATCH
# IS_PATCH: true if this is a patch release; append instead of create
if grep -E '^[0-9]+\.[0-9]+$' <<< "$VERSION" >/dev/null; then
log 'is minor'
IS_PATCH=false
version_full="$VERSION.0"
version_major_minor="$VERSION"
elif grep -E '^[0-9]+\.[0-9]+\.0$' <<< "$VERSION" >/dev/null; then
log 'is minor (.0)'
IS_PATCH=false
version_full="$VERSION"
version_major_minor="$(echo "$VERSION" | sed -e 's/\.0$//')"
elif grep -E '^[0-9]+\.[0-9]+\.[0-9]+$' <<< "$VERSION" >/dev/null; then
log 'is patch'
IS_PATCH=true
version_full="$VERSION"
version_major_minor="$(echo "$VERSION" | sed -e 's/\.[0-9]*$//')"
else
die "VERSION must be MAJOR.MINOR[.PATCH], where each is a number, e.g. 2.20 or 2.20.1 (VERSION was set to $VERSION)"
fi
unset VERSION
log "version_major_minor=$version_major_minor"
log "version_full=$version_full"
log "IS_PATCH=$IS_PATCH"
basename=rl-${version_major_minor}.md
file=doc/manual/src/release-notes/$basename
if ! $IS_PATCH; then
if [[ -e $file ]]; then
die "release notes file $file already exists. If you'd like to make a minor release, pass a patch version, e.g. 2.20.1"
fi
fi
# --- DEFAULTS ---
if [[ ! -n "${DATE:-}" ]]; then
DATE="$(date +%Y-%m-%d)"
log "DATE not set, using $DATE"
fi
case "$DATE" in
[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9])
;;
*)
die "DATE must be YYYY-MM-DD, e.g. 2021-12-31 (DATE was set to $DATE)"
;;
esac
# --- DO THE WORK ---
# menu
title="Release $version_major_minor ($DATE)"
# section on page
section_title="Release $version_full ($DATE)"
(
# TODO add minor number, and append?
echo "# $section_title"
echo
changelog-d doc/manual/rl-next | sed -e 's/ *$//'
) | tee -a $file
log "Wrote $file"
if ! $IS_PATCH; then
NEW_SUMMARY_LINE=" - [$title](release-notes/$basename)"
# find the marker line, insert new link after it
escaped_marker="$(echo "$SUMMARY_MARKER_LINE" | sed -e 's/\//\\\//g' -e 's/ /\\ /g')"
escaped_line="$(echo "$NEW_SUMMARY_LINE" | sed -e 's/\//\\\//g' -e 's/ /\\ /g')"
logcmd sed -i -e "/$escaped_marker/a $escaped_line" doc/manual/src/SUMMARY.md.in
fi
for f in doc/manual/rl-next/*.md; do
if [[ config != "$(basename $f)" ]]; then
logcmd git rm $f
fi
done
logcmd git add $file doc/manual/src/SUMMARY.md.in
logcmd git status
logcmd git commit -m "release notes: $version_full"
report_done
+42 -27
View File
@@ -24,34 +24,23 @@ release:
* In a checkout of the Nix repo, make sure you're on `master` and run
`git pull`.
* Move the contents of `doc/manual/src/release-notes/rl-next.md`
(except the first line) to
`doc/manual/src/release-notes/rl-$VERSION.md` (where `$VERSION` is
the contents of `.version` *without* the patch level, e.g. `2.12`
rather than `2.12.0`).
* Add a header to `doc/manual/src/release-notes/rl-$VERSION.md` like
```
# Release 2.12 (2022-12-06)
```
* Proof-read / edit / rearrange the release notes. Breaking changes
and highlights should go to the top.
* Add a link to the release notes to `doc/manual/src/SUMMARY.md.in`
(*not* `SUMMARY.md`), e.g.
```
- [Release 2.12 (2022-12-06)](release-notes/rl-2.12.md)
```
* Run
* Compile the release notes by running
```console
$ git checkout -b release-notes
$ git add doc/manual/src/release-notes/rl-$VERSION.md
$ git commit -a -m 'Release notes'
$ VERSION=X.YY ./maintainers/release-notes
```
where `X.YY` is *without* the patch level, e.g. `2.12` rather than ~~`2.12.0`~~.
A commit is created.
* Proof-read / edit / rearrange the release notes if needed. Breaking changes
and highlights should go to the top.
* Push.
```console
$ git push --set-upstream $REMOTE release-notes
```
@@ -67,15 +56,17 @@ release:
$ git checkout -b $VERSION-maintenance
```
* Mark the release as stable:
* Mark the release as official:
```console
$ git cherry-pick f673551e71942a52b6d7ae66af8b67140904a76a
$ sed -e 's/officialRelease = false;/officialRelease = true;/' -i flake.nix
```
This removes the link to `rl-next.md` from the manual and sets
`officialRelease = true` in `flake.nix`.
* Commit
* Push the release branch:
```console
@@ -159,6 +150,30 @@ release:
## Creating a point release
* Checkout.
```console
$ git checkout XX.YY-maintenance
```
* Determine the next patch version.
```console
$ export VERSION=XX.YY.ZZ
```
* Update release notes.
```console
$ ./maintainers/release-notes
```
* Push.
```console
$ git push
```
* Wait for the desired evaluation of the maintenance jobset to finish
building.
+19 -3
View File
@@ -1,11 +1,27 @@
TESTS_ENVIRONMENT=("TEST_NAME=${test%.*}" 'NIX_REMOTE=')
# Remove overall test dir (at most one of the two should match) and
# remove file extension.
test_name=$(echo -n "$test" | sed \
-e "s|^unit-test-data/||" \
-e "s|^tests/functional/||" \
-e "s|\.sh$||" \
)
TESTS_ENVIRONMENT=(
"TEST_NAME=$test_name"
'NIX_REMOTE='
'PS4=+(${BASH_SOURCE[0]-$0}:$LINENO) '
)
: ${BASH:=/usr/bin/env bash}
run () {
cd "$(dirname $1)" && env "${TESTS_ENVIRONMENT[@]}" $BASH -x -e -u -o pipefail $(basename $1)
}
init_test () {
cd tests && env "${TESTS_ENVIRONMENT[@]}" $BASH -e init.sh 2>/dev/null > /dev/null
run "$init" 2>/dev/null > /dev/null
}
run_test_proper () {
cd $(dirname $test) && env "${TESTS_ENVIRONMENT[@]}" $BASH -e $(basename $test)
run "$test"
}
+4 -1
View File
@@ -3,9 +3,12 @@
set -eu -o pipefail
test=$1
init=${2-}
dir="$(dirname "${BASH_SOURCE[0]}")"
source "$dir/common-test.sh"
(init_test)
if [ -n "$init" ]; then
(init_test)
fi
run_test_proper
+4 -3
View File
@@ -122,14 +122,15 @@ $(foreach script, $(bin-scripts), $(eval $(call install-program-in,$(script),$(b
$(foreach script, $(bin-scripts), $(eval programs-list += $(script)))
$(foreach script, $(noinst-scripts), $(eval programs-list += $(script)))
$(foreach template, $(template-files), $(eval $(call instantiate-template,$(template))))
install_test_init=tests/functional/init.sh
$(foreach test, $(install-tests), \
$(eval $(call run-install-test,$(test))) \
$(eval $(call run-test,$(test),$(install_test_init))) \
$(eval installcheck: $(test).test))
$(foreach test-group, $(install-tests-groups), \
$(eval $(call run-install-test-group,$(test-group))) \
$(eval $(call run-test-group,$(test-group),$(install_test_init))) \
$(eval installcheck: $(test-group).test-group) \
$(foreach test, $($(test-group)-tests), \
$(eval $(call run-install-test,$(test))) \
$(eval $(call run-test,$(test),$(install_test_init))) \
$(eval $(test-group).test-group: $(test).test)))
$(foreach file, $(man-pages), $(eval $(call install-data-in, $(file), $(mandir)/man$(patsubst .%,%,$(suffix $(file))))))
+1 -1
View File
@@ -87,6 +87,6 @@ define build-program
# Phony target to run this program (typically as a dependency of 'check').
.PHONY: $(1)_RUN
$(1)_RUN: $$($(1)_PATH)
$(trace-test) $$($(1)_PATH)
$(trace-test) $$(UNIT_TEST_ENV) $$($(1)_PATH)
endef
+4 -1
View File
@@ -8,6 +8,7 @@ yellow=""
normal=""
test=$1
init=${2-}
dir="$(dirname "${BASH_SOURCE[0]}")"
source "$dir/common-test.sh"
@@ -21,7 +22,9 @@ if [ -t 1 ]; then
fi
run_test () {
(init_test 2>/dev/null > /dev/null)
if [ -n "$init" ]; then
(init_test 2>/dev/null > /dev/null)
fi
log="$(run_test_proper 2>&1)" && status=0 || status=$?
}
+12 -9
View File
@@ -2,19 +2,22 @@
test-deps =
define run-install-test
define run-bash
.PHONY: $1.test
$1.test: $1 $(test-deps)
@env BASH=$(bash) $(bash) mk/run-test.sh $1 < /dev/null
.PHONY: $1.test-debug
$1.test-debug: $1 $(test-deps)
@env BASH=$(bash) $(bash) mk/debug-test.sh $1 < /dev/null
.PHONY: $1
$1: $2
@env BASH=$(bash) $(bash) $3 < /dev/null
endef
define run-install-test-group
define run-test
$(eval $(call run-bash,$1.test,$1 $(test-deps),mk/run-test.sh $1 $2))
$(eval $(call run-bash,$1.test-debug,$1 $(test-deps),mk/debug-test.sh $1 $2))
endef
define run-test-group
.PHONY: $1.test-group
+7 -1
View File
@@ -1,6 +1,12 @@
makefiles = local.mk
GLOBAL_CXXFLAGS += -g -Wall -std=c++2a -I ../src
GLOBAL_CXXFLAGS += -g -Wall -std=c++2a
# A convenience for concurrent development of Nix and its Perl bindings.
# Not needed in a standalone build of the Perl bindings.
ifneq ("$(wildcard ../src)", "")
GLOBAL_CXXFLAGS += -I ../src
endif
-include Makefile.config
+51
View File
@@ -0,0 +1,51 @@
{ lib, fileset
, stdenv
, perl, perlPackages
, autoconf-archive, autoreconfHook, pkg-config
, nix, curl, bzip2, xz, boost, libsodium, darwin
}:
perl.pkgs.toPerlModule (stdenv.mkDerivation {
name = "nix-perl-${nix.version}";
src = fileset.toSource {
root = ../.;
fileset = fileset.unions [
../.version
../m4
../mk
./MANIFEST
./Makefile
./Makefile.config.in
./configure.ac
./lib
./local.mk
];
};
nativeBuildInputs =
[ autoconf-archive
autoreconfHook
pkg-config
];
buildInputs =
[ nix
curl
bzip2
xz
perl
boost
]
++ lib.optional (stdenv.isLinux || stdenv.isDarwin) libsodium
++ lib.optional stdenv.isDarwin darwin.apple_sdk.frameworks.Security;
configureFlags = [
"--with-dbi=${perlPackages.DBI}/${perl.libPrefix}"
"--with-dbd-sqlite=${perlPackages.DBDSQLite}/${perl.libPrefix}"
];
enableParallelBuilding = true;
postUnpack = "sourceRoot=$sourceRoot/perl";
})
+6 -7
View File
@@ -11,7 +11,6 @@
#include "derivations.hh"
#include "globals.hh"
#include "store-api.hh"
#include "util.hh"
#include "crypto.hh"
#include <sodium.h>
@@ -78,7 +77,7 @@ SV * queryReferences(char * path)
SV * queryPathHash(char * path)
PPCODE:
try {
auto s = store()->queryPathInfo(store()->parseStorePath(path))->narHash.to_string(Base32, true);
auto s = store()->queryPathInfo(store()->parseStorePath(path))->narHash.to_string(HashFormat::Base32, true);
XPUSHs(sv_2mortal(newSVpv(s.c_str(), 0)));
} catch (Error & e) {
croak("%s", e.what());
@@ -104,7 +103,7 @@ SV * queryPathInfo(char * path, int base32)
XPUSHs(&PL_sv_undef);
else
XPUSHs(sv_2mortal(newSVpv(store()->printStorePath(*info->deriver).c_str(), 0)));
auto s = info->narHash.to_string(base32 ? Base32 : Base16, true);
auto s = info->narHash.to_string(base32 ? HashFormat::Base32 : HashFormat::Base16, true);
XPUSHs(sv_2mortal(newSVpv(s.c_str(), 0)));
mXPUSHi(info->registrationTime);
mXPUSHi(info->narSize);
@@ -206,7 +205,7 @@ SV * hashPath(char * algo, int base32, char * path)
PPCODE:
try {
Hash h = hashPath(parseHashType(algo), path).first;
auto s = h.to_string(base32 ? Base32 : Base16, false);
auto s = h.to_string(base32 ? HashFormat::Base32 : HashFormat::Base16, false);
XPUSHs(sv_2mortal(newSVpv(s.c_str(), 0)));
} catch (Error & e) {
croak("%s", e.what());
@@ -217,7 +216,7 @@ SV * hashFile(char * algo, int base32, char * path)
PPCODE:
try {
Hash h = hashFile(parseHashType(algo), path);
auto s = h.to_string(base32 ? Base32 : Base16, false);
auto s = h.to_string(base32 ? HashFormat::Base32 : HashFormat::Base16, false);
XPUSHs(sv_2mortal(newSVpv(s.c_str(), 0)));
} catch (Error & e) {
croak("%s", e.what());
@@ -228,7 +227,7 @@ SV * hashString(char * algo, int base32, char * s)
PPCODE:
try {
Hash h = hashString(parseHashType(algo), s);
auto s = h.to_string(base32 ? Base32 : Base16, false);
auto s = h.to_string(base32 ? HashFormat::Base32 : HashFormat::Base16, false);
XPUSHs(sv_2mortal(newSVpv(s.c_str(), 0)));
} catch (Error & e) {
croak("%s", e.what());
@@ -239,7 +238,7 @@ SV * convertHash(char * algo, char * s, int toBase32)
PPCODE:
try {
auto h = Hash::parseAny(s, parseHashType(algo));
auto s = h.to_string(toBase32 ? Base32 : Base16, false);
auto s = h.to_string(toBase32 ? HashFormat::Base32 : HashFormat::Base16, false);
XPUSHs(sv_2mortal(newSVpv(s.c_str(), 0)));
} catch (Error & e) {
croak("%s", e.what());
+8
View File
@@ -452,6 +452,14 @@ EOF
# a row for different files.
if [ -e "$profile_target$PROFILE_BACKUP_SUFFIX" ]; then
# this backup process first released in Nix 2.1
if diff -q "$profile_target$PROFILE_BACKUP_SUFFIX" "$profile_target" > /dev/null; then
# a backup file for the rc-file exist, but they are identical,
# so we can safely ignore it and overwrite it with the same
# content later
continue
fi
failure <<EOF
I back up shell profile/rc scripts before I add Nix to them.
I need to back up $profile_target to $profile_target$PROFILE_BACKUP_SUFFIX,
+8
View File
@@ -19,6 +19,14 @@ set __ETC_PROFILE_NIX_SOURCED 1
set --export NIX_PROFILES "@localstatedir@/nix/profiles/default $HOME/.nix-profile"
# Populate bash completions, .desktop files, etc
if test -z "$XDG_DATA_DIRS"
# According to XDG spec the default is /usr/local/share:/usr/share, don't set something that prevents that default
set --export XDG_DATA_DIRS "/usr/local/share:/usr/share:/nix/var/nix/profiles/default/share"
else
set --export XDG_DATA_DIRS "$XDG_DATA_DIRS:/nix/var/nix/profiles/default/share"
end
# Set $NIX_SSL_CERT_FILE so that Nixpkgs applications like curl work.
if test -n "$NIX_SSH_CERT_FILE"
: # Allow users to override the NIX_SSL_CERT_FILE
+8
View File
@@ -30,6 +30,14 @@ fi
export NIX_PROFILES="@localstatedir@/nix/profiles/default $NIX_LINK"
# Populate bash completions, .desktop files, etc
if [ -z "${XDG_DATA_DIRS-}" ]; then
# According to XDG spec the default is /usr/local/share:/usr/share, don't set something that prevents that default
export XDG_DATA_DIRS="/usr/local/share:/usr/share:$NIX_LINK/share:/nix/var/nix/profiles/default/share"
else
export XDG_DATA_DIRS="$XDG_DATA_DIRS:$NIX_LINK/share:/nix/var/nix/profiles/default/share"
fi
# Set $NIX_SSL_CERT_FILE so that Nixpkgs applications like curl work.
if [ -n "${NIX_SSL_CERT_FILE:-}" ]; then
: # Allow users to override the NIX_SSL_CERT_FILE
+8
View File
@@ -20,6 +20,14 @@ if test -n "$HOME" && test -n "$USER"
# This part should be kept in sync with nixpkgs:nixos/modules/programs/environment.nix
set --export NIX_PROFILES "@localstatedir@/nix/profiles/default $HOME/.nix-profile"
# Populate bash completions, .desktop files, etc
if test -z "$XDG_DATA_DIRS"
# According to XDG spec the default is /usr/local/share:/usr/share, don't set something that prevents that default
set --export XDG_DATA_DIRS "/usr/local/share:/usr/share:$NIX_LINK/share:/nix/var/nix/profiles/default/share"
else
set --export XDG_DATA_DIRS "$XDG_DATA_DIRS:$NIX_LINK/share:/nix/var/nix/profiles/default/share"
end
# Set $NIX_SSL_CERT_FILE so that Nixpkgs applications like curl work.
if test -n "$NIX_SSH_CERT_FILE"
: # Allow users to override the NIX_SSL_CERT_FILE
+8
View File
@@ -32,6 +32,14 @@ if [ -n "$HOME" ] && [ -n "$USER" ]; then
# This part should be kept in sync with nixpkgs:nixos/modules/programs/environment.nix
export NIX_PROFILES="@localstatedir@/nix/profiles/default $NIX_LINK"
# Populate bash completions, .desktop files, etc
if [ -z "${XDG_DATA_DIRS-}" ]; then
# According to XDG spec the default is /usr/local/share:/usr/share, don't set something that prevents that default
export XDG_DATA_DIRS="/usr/local/share:/usr/share:$NIX_LINK/share:/nix/var/nix/profiles/default/share"
else
export XDG_DATA_DIRS="$XDG_DATA_DIRS:$NIX_LINK/share:/nix/var/nix/profiles/default/share"
fi
# Set $NIX_SSL_CERT_FILE so that Nixpkgs applications like curl work.
if [ -e /etc/ssl/certs/ca-certificates.crt ]; then # NixOS, Ubuntu, Debian, Gentoo, Arch
export NIX_SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt
+3
View File
@@ -1,3 +1,6 @@
#pragma once
///@file
#include "derived-path.hh"
#include "realisation.hh"
+3 -3
View File
@@ -98,7 +98,7 @@ EvalCommand::EvalCommand()
EvalCommand::~EvalCommand()
{
if (evalState)
evalState->printStats();
evalState->maybePrintStats();
}
ref<Store> EvalCommand::getEvalStore()
@@ -175,7 +175,7 @@ void BuiltPathsCommand::run(ref<Store> store, Installables && installables)
throw UsageError("'--all' does not expect arguments");
// XXX: Only uses opaque paths, ignores all the realisations
for (auto & p : store->queryAllValidPaths())
paths.push_back(BuiltPath::Opaque{p});
paths.emplace_back(BuiltPath::Opaque{p});
} else {
paths = Installable::toBuiltPaths(getEvalStore(), store, realiseMode, operateOn, installables);
if (recursive) {
@@ -188,7 +188,7 @@ void BuiltPathsCommand::run(ref<Store> store, Installables && installables)
}
store->computeFSClosure(pathsRoots, pathsClosure);
for (auto & path : pathsClosure)
paths.push_back(BuiltPath::Opaque{path});
paths.emplace_back(BuiltPath::Opaque{path});
}
}
+84 -26
View File
@@ -34,21 +34,28 @@ struct NixMultiCommand : virtual MultiCommand, virtual Command
// For the overloaded run methods
#pragma GCC diagnostic ignored "-Woverloaded-virtual"
/* A command that requires a Nix store. */
/**
* A command that requires a \ref Store "Nix store".
*/
struct StoreCommand : virtual Command
{
StoreCommand();
void run() override;
ref<Store> getStore();
virtual ref<Store> createStore();
/**
* Main entry point, with a `Store` provided
*/
virtual void run(ref<Store>) = 0;
private:
std::shared_ptr<Store> _store;
};
/* A command that copies something between `--from` and `--to`
stores. */
/**
* A command that copies something between `--from` and `--to` \ref
* Store stores.
*/
struct CopyCommand : virtual StoreCommand
{
std::string srcUri, dstUri;
@@ -60,6 +67,9 @@ struct CopyCommand : virtual StoreCommand
ref<Store> getDstStore();
};
/**
* A command that needs to evaluate Nix language expressions.
*/
struct EvalCommand : virtual StoreCommand, MixEvalArgs
{
bool startReplOnEvalErrors = false;
@@ -79,20 +89,26 @@ private:
std::shared_ptr<EvalState> evalState;
};
/**
* A mixin class for commands that process flakes, adding a few standard
* flake-related options/flags.
*/
struct MixFlakeOptions : virtual Args, EvalCommand
{
flake::LockFlags lockFlags;
std::optional<std::string> needsFlakeInputCompletion = {};
MixFlakeOptions();
virtual std::vector<std::string> getFlakesForCompletion()
/**
* The completion for some of these flags depends on the flake(s) in
* question.
*
* This method should be implemented to gather all flakerefs the
* command is operating with (presumably specified via some other
* arguments) so that the completions for these flags can use them.
*/
virtual std::vector<FlakeRef> getFlakeRefsForCompletion()
{ return {}; }
void completeFlakeInput(std::string_view prefix);
void completionHook() override;
};
struct SourceExprCommand : virtual Args, MixFlakeOptions
@@ -112,15 +128,35 @@ struct SourceExprCommand : virtual Args, MixFlakeOptions
virtual Strings getDefaultFlakeAttrPathPrefixes();
void completeInstallable(std::string_view prefix);
/**
* Complete an installable from the given prefix.
*/
void completeInstallable(AddCompletions & completions, std::string_view prefix);
/**
* Convenience wrapper around the underlying function to make setting the
* callback easier.
*/
CompleterClosure getCompleteInstallable();
};
/**
* A mixin class for commands that need a read-only flag.
*
* What exactly is "read-only" is unspecified, but it will usually be
* the \ref Store "Nix store".
*/
struct MixReadOnlyOption : virtual Args
{
MixReadOnlyOption();
};
/* Like InstallablesCommand but the installables are not loaded */
/**
* Like InstallablesCommand but the installables are not loaded.
*
* This is needed by `CmdRepl` which wants to load (and reload) the
* installables itself.
*/
struct RawInstallablesCommand : virtual Args, SourceExprCommand
{
RawInstallablesCommand();
@@ -129,19 +165,22 @@ struct RawInstallablesCommand : virtual Args, SourceExprCommand
void run(ref<Store> store) override;
// FIXME make const after CmdRepl's override is fixed up
// FIXME make const after `CmdRepl`'s override is fixed up
virtual void applyDefaultInstallables(std::vector<std::string> & rawInstallables);
bool readFromStdIn = false;
std::vector<std::string> getFlakesForCompletion() override;
std::vector<FlakeRef> getFlakeRefsForCompletion() override;
private:
std::vector<std::string> rawInstallables;
};
/* A command that operates on a list of "installables", which can be
store paths, attribute paths, Nix expressions, etc. */
/**
* A command that operates on a list of "installables", which can be
* store paths, attribute paths, Nix expressions, etc.
*/
struct InstallablesCommand : RawInstallablesCommand
{
virtual void run(ref<Store> store, Installables && installables) = 0;
@@ -149,7 +188,9 @@ struct InstallablesCommand : RawInstallablesCommand
void run(ref<Store> store, std::vector<std::string> && rawInstallables) override;
};
/* A command that operates on exactly one "installable" */
/**
* A command that operates on exactly one "installable".
*/
struct InstallableCommand : virtual Args, SourceExprCommand
{
InstallableCommand();
@@ -158,10 +199,7 @@ struct InstallableCommand : virtual Args, SourceExprCommand
void run(ref<Store> store) override;
std::vector<std::string> getFlakesForCompletion() override
{
return {_installable};
}
std::vector<FlakeRef> getFlakeRefsForCompletion() override;
private:
@@ -175,7 +213,12 @@ struct MixOperateOnOptions : virtual Args
MixOperateOnOptions();
};
/* A command that operates on zero or more store paths. */
/**
* A command that operates on zero or more extant store paths.
*
* If the argument the user passes is a some sort of recipe for a path
* not yet built, it must be built first.
*/
struct BuiltPathsCommand : InstallablesCommand, virtual MixOperateOnOptions
{
private:
@@ -207,7 +250,9 @@ struct StorePathsCommand : public BuiltPathsCommand
void run(ref<Store> store, BuiltPaths && paths) override;
};
/* A command that operates on exactly one store path. */
/**
* A command that operates on exactly one store path.
*/
struct StorePathCommand : public StorePathsCommand
{
virtual void run(ref<Store> store, const StorePath & storePath) = 0;
@@ -215,7 +260,9 @@ struct StorePathCommand : public StorePathsCommand
void run(ref<Store> store, StorePaths && storePaths) override;
};
/* A helper class for registering commands globally. */
/**
* A helper class for registering \ref Command commands globally.
*/
struct RegisterCommand
{
typedef std::map<std::vector<std::string>, std::function<ref<Command>()>> Commands;
@@ -271,13 +318,24 @@ struct MixEnvironment : virtual Args {
MixEnvironment();
/* Modify global environ based on ignoreEnvironment, keep, and unset. It's expected that exec will be called before this class goes out of scope, otherwise environ will become invalid. */
/***
* Modify global environ based on `ignoreEnvironment`, `keep`, and
* `unset`. It's expected that exec will be called before this class
* goes out of scope, otherwise `environ` will become invalid.
*/
void setEnviron();
};
void completeFlakeRef(ref<Store> store, std::string_view prefix);
void completeFlakeInputPath(
AddCompletions & completions,
ref<EvalState> evalState,
const std::vector<FlakeRef> & flakeRefs,
std::string_view prefix);
void completeFlakeRef(AddCompletions & completions, ref<Store> store, std::string_view prefix);
void completeFlakeRefWithFragment(
AddCompletions & completions,
ref<EvalState> evalState,
flake::LockFlags lockFlags,
Strings attrPathPrefixes,
+7 -7
View File
@@ -2,13 +2,13 @@
#include "common-eval-args.hh"
#include "shared.hh"
#include "filetransfer.hh"
#include "util.hh"
#include "eval.hh"
#include "fetchers.hh"
#include "registry.hh"
#include "flake/flakeref.hh"
#include "store-api.hh"
#include "command.hh"
#include "tarball.hh"
namespace nix {
@@ -132,8 +132,8 @@ MixEvalArgs::MixEvalArgs()
if (to.subdir != "") extraAttrs["dir"] = to.subdir;
fetchers::overrideRegistry(from.input, to.input, extraAttrs);
}},
.completer = {[&](size_t, std::string_view prefix) {
completeFlakeRef(openStore(), prefix);
.completer = {[&](AddCompletions & completions, size_t, std::string_view prefix) {
completeFlakeRef(completions, openStore(), prefix);
}}
});
@@ -164,18 +164,18 @@ Bindings * MixEvalArgs::getAutoArgs(EvalState & state)
return res.finish();
}
SourcePath lookupFileArg(EvalState & state, std::string_view s)
SourcePath lookupFileArg(EvalState & state, std::string_view s, CanonPath baseDir)
{
if (EvalSettings::isPseudoUrl(s)) {
auto storePath = fetchers::downloadTarball(
state.store, EvalSettings::resolvePseudoUrl(s), "source", false).tree.storePath;
state.store, EvalSettings::resolvePseudoUrl(s), "source", false).storePath;
return state.rootPath(CanonPath(state.store->toRealPath(storePath)));
}
else if (hasPrefix(s, "flake:")) {
experimentalFeatureSettings.require(Xp::Flakes);
auto flakeRef = parseFlakeRef(std::string(s.substr(6)), {}, true, false);
auto storePath = flakeRef.resolve(state.store).fetchTree(state.store).first.storePath;
auto storePath = flakeRef.resolve(state.store).fetchTree(state.store).first;
return state.rootPath(CanonPath(state.store->toRealPath(storePath)));
}
@@ -185,7 +185,7 @@ SourcePath lookupFileArg(EvalState & state, std::string_view s)
}
else
return state.rootPath(CanonPath::fromCwd(s));
return state.rootPath(CanonPath(s, baseDir));
}
}
+2 -1
View File
@@ -2,6 +2,7 @@
///@file
#include "args.hh"
#include "canon-path.hh"
#include "common-args.hh"
#include "search-path.hh"
@@ -28,6 +29,6 @@ private:
std::map<std::string, std::string> autoArgs;
};
SourcePath lookupFileArg(EvalState & state, std::string_view s);
SourcePath lookupFileArg(EvalState & state, std::string_view s, CanonPath baseDir = CanonPath::fromCwd());
}
+1 -1
View File
@@ -1,5 +1,5 @@
#include "util.hh"
#include "editor-for.hh"
#include "environment-variables.hh"
namespace nix {
-1
View File
@@ -4,7 +4,6 @@
#include "globals.hh"
#include "installable-value.hh"
#include "outputs-spec.hh"
#include "util.hh"
#include "command.hh"
#include "attr-path.hh"
#include "common-eval-args.hh"
+5
View File
@@ -28,6 +28,11 @@ namespace nix {
std::vector<std::string> InstallableFlake::getActualAttrPaths()
{
std::vector<std::string> res;
if (attrPaths.size() == 1 && attrPaths.front().starts_with(".")){
attrPaths.front().erase(0,1);
res.push_back(attrPaths.front());
return res;
}
for (auto & prefix : prefixes)
res.push_back(prefix + *attrPaths.begin());
+111 -61
View File
@@ -4,6 +4,7 @@
#include "installable-attr-path.hh"
#include "installable-flake.hh"
#include "outputs-spec.hh"
#include "users.hh"
#include "util.hh"
#include "command.hh"
#include "attr-path.hh"
@@ -28,15 +29,38 @@
namespace nix {
void completeFlakeInputPath(
AddCompletions & completions,
ref<EvalState> evalState,
const std::vector<FlakeRef> & flakeRefs,
std::string_view prefix)
{
for (auto & flakeRef : flakeRefs) {
auto flake = flake::getFlake(*evalState, flakeRef, true);
for (auto & input : flake.inputs)
if (hasPrefix(input.first, prefix))
completions.add(input.first);
}
}
MixFlakeOptions::MixFlakeOptions()
{
auto category = "Common flake-related options";
addFlag({
.longName = "recreate-lock-file",
.description = "Recreate the flake's lock file from scratch.",
.description = R"(
Recreate the flake's lock file from scratch.
> **DEPRECATED**
>
> Use [`nix flake update`](@docroot@/command-ref/new-cli/nix3-flake-update.md) instead.
)",
.category = category,
.handler = {&lockFlags.recreateLockFile, true}
.handler = {[&]() {
lockFlags.recreateLockFile = true;
warn("'--recreate-lock-file' is deprecated and will be removed in a future version; use 'nix flake update' instead.");
}}
});
addFlag({
@@ -55,8 +79,13 @@ MixFlakeOptions::MixFlakeOptions()
addFlag({
.longName = "no-registries",
.description =
"Don't allow lookups in the flake registries. This option is deprecated; use `--no-use-registries`.",
.description = R"(
Don't allow lookups in the flake registries.
> **DEPRECATED**
>
> Use [`--no-use-registries`](#opt-no-use-registries) instead.
)",
.category = category,
.handler = {[&]() {
lockFlags.useRegistries = false;
@@ -73,14 +102,21 @@ MixFlakeOptions::MixFlakeOptions()
addFlag({
.longName = "update-input",
.description = "Update a specific flake input (ignoring its previous entry in the lock file).",
.description = R"(
Update a specific flake input (ignoring its previous entry in the lock file).
> **DEPRECATED**
>
> Use [`nix flake update`](@docroot@/command-ref/new-cli/nix3-flake-update.md) instead.
)",
.category = category,
.labels = {"input-path"},
.handler = {[&](std::string s) {
warn("'--update-input' is a deprecated alias for 'flake update' and will be removed in a future version.");
lockFlags.inputUpdates.insert(flake::parseInputPath(s));
}},
.completer = {[&](size_t, std::string_view prefix) {
needsFlakeInputCompletion = {std::string(prefix)};
.completer = {[&](AddCompletions & completions, size_t, std::string_view prefix) {
completeFlakeInputPath(completions, getEvalState(), getFlakeRefsForCompletion(), prefix);
}}
});
@@ -93,13 +129,14 @@ MixFlakeOptions::MixFlakeOptions()
lockFlags.writeLockFile = false;
lockFlags.inputOverrides.insert_or_assign(
flake::parseInputPath(inputPath),
parseFlakeRef(flakeRef, absPath("."), true));
parseFlakeRef(flakeRef, absPath(getCommandBaseDir()), true));
}},
.completer = {[&](size_t n, std::string_view prefix) {
if (n == 0)
needsFlakeInputCompletion = {std::string(prefix)};
else if (n == 1)
completeFlakeRef(getEvalState()->store, prefix);
.completer = {[&](AddCompletions & completions, size_t n, std::string_view prefix) {
if (n == 0) {
completeFlakeInputPath(completions, getEvalState(), getFlakeRefsForCompletion(), prefix);
} else if (n == 1) {
completeFlakeRef(completions, getEvalState()->store, prefix);
}
}}
});
@@ -134,7 +171,7 @@ MixFlakeOptions::MixFlakeOptions()
auto evalState = getEvalState();
auto flake = flake::lockFlake(
*evalState,
parseFlakeRef(flakeRef, absPath(".")),
parseFlakeRef(flakeRef, absPath(getCommandBaseDir())),
{ .writeLockFile = false });
for (auto & [inputName, input] : flake.lockFile.root->inputs) {
auto input2 = flake.lockFile.findInput({inputName}); // resolve 'follows' nodes
@@ -146,30 +183,12 @@ MixFlakeOptions::MixFlakeOptions()
}
}
}},
.completer = {[&](size_t, std::string_view prefix) {
completeFlakeRef(getEvalState()->store, prefix);
.completer = {[&](AddCompletions & completions, size_t, std::string_view prefix) {
completeFlakeRef(completions, getEvalState()->store, prefix);
}}
});
}
void MixFlakeOptions::completeFlakeInput(std::string_view prefix)
{
auto evalState = getEvalState();
for (auto & flakeRefS : getFlakesForCompletion()) {
auto flakeRef = parseFlakeRefWithFragment(expandTilde(flakeRefS), absPath(".")).first;
auto flake = flake::getFlake(*evalState, flakeRef, true);
for (auto & input : flake.inputs)
if (hasPrefix(input.first, prefix))
completions->add(input.first);
}
}
void MixFlakeOptions::completionHook()
{
if (auto & prefix = needsFlakeInputCompletion)
completeFlakeInput(*prefix);
}
SourceExprCommand::SourceExprCommand()
{
addFlag({
@@ -226,11 +245,18 @@ Strings SourceExprCommand::getDefaultFlakeAttrPathPrefixes()
};
}
void SourceExprCommand::completeInstallable(std::string_view prefix)
Args::CompleterClosure SourceExprCommand::getCompleteInstallable()
{
return [this](AddCompletions & completions, size_t, std::string_view prefix) {
completeInstallable(completions, prefix);
};
}
void SourceExprCommand::completeInstallable(AddCompletions & completions, std::string_view prefix)
{
try {
if (file) {
completionType = ctAttrs;
completions.setType(AddCompletions::Type::Attrs);
evalSettings.pureEval = false;
auto state = getEvalState();
@@ -265,14 +291,15 @@ void SourceExprCommand::completeInstallable(std::string_view prefix)
std::string name = state->symbols[i.name];
if (name.find(searchWord) == 0) {
if (prefix_ == "")
completions->add(name);
completions.add(name);
else
completions->add(prefix_ + "." + name);
completions.add(prefix_ + "." + name);
}
}
}
} else {
completeFlakeRefWithFragment(
completions,
getEvalState(),
lockFlags,
getDefaultFlakeAttrPathPrefixes(),
@@ -285,6 +312,7 @@ void SourceExprCommand::completeInstallable(std::string_view prefix)
}
void completeFlakeRefWithFragment(
AddCompletions & completions,
ref<EvalState> evalState,
flake::LockFlags lockFlags,
Strings attrPathPrefixes,
@@ -296,12 +324,19 @@ void completeFlakeRefWithFragment(
try {
auto hash = prefix.find('#');
if (hash == std::string::npos) {
completeFlakeRef(evalState->store, prefix);
completeFlakeRef(completions, evalState->store, prefix);
} else {
completionType = ctAttrs;
completions.setType(AddCompletions::Type::Attrs);
auto fragment = prefix.substr(hash + 1);
std::string prefixRoot = "";
if (fragment.starts_with(".")){
fragment = fragment.substr(1);
prefixRoot = ".";
}
auto flakeRefS = std::string(prefix.substr(0, hash));
// TODO: ideally this would use the command base directory instead of assuming ".".
auto flakeRef = parseFlakeRef(expandTilde(flakeRefS), absPath("."));
auto evalCache = openEvalCache(*evalState,
@@ -309,6 +344,9 @@ void completeFlakeRefWithFragment(
auto root = evalCache->getRoot();
if (prefixRoot == "."){
attrPathPrefixes.clear();
}
/* Complete 'fragment' relative to all the
attrpath prefixes as well as the root of the
flake. */
@@ -333,7 +371,7 @@ void completeFlakeRefWithFragment(
auto attrPath2 = (*attr)->getAttrPath(attr2);
/* Strip the attrpath prefix. */
attrPath2.erase(attrPath2.begin(), attrPath2.begin() + attrPathPrefix.size());
completions->add(flakeRefS + "#" + concatStringsSep(".", evalState->symbols.resolve(attrPath2)));
completions.add(flakeRefS + "#" + prefixRoot + concatStringsSep(".", evalState->symbols.resolve(attrPath2)));
}
}
}
@@ -344,7 +382,7 @@ void completeFlakeRefWithFragment(
for (auto & attrPath : defaultFlakeAttrPaths) {
auto attr = root->findAlongAttrPath(parseAttrPath(*evalState, attrPath));
if (!attr) continue;
completions->add(flakeRefS + "#");
completions.add(flakeRefS + "#" + prefixRoot);
}
}
}
@@ -353,15 +391,15 @@ void completeFlakeRefWithFragment(
}
}
void completeFlakeRef(ref<Store> store, std::string_view prefix)
void completeFlakeRef(AddCompletions & completions, ref<Store> store, std::string_view prefix)
{
if (!experimentalFeatureSettings.isEnabled(Xp::Flakes))
return;
if (prefix == "")
completions->add(".");
completions.add(".");
completeDir(0, prefix);
Args::completeDir(completions, 0, prefix);
/* Look for registry entries that match the prefix. */
for (auto & registry : fetchers::getRegistries(store)) {
@@ -370,10 +408,10 @@ void completeFlakeRef(ref<Store> store, std::string_view prefix)
if (!hasPrefix(prefix, "flake:") && hasPrefix(from, "flake:")) {
std::string from2(from, 6);
if (hasPrefix(from2, prefix))
completions->add(from2);
completions.add(from2);
} else {
if (hasPrefix(from, prefix))
completions->add(from);
completions.add(from);
}
}
}
@@ -447,10 +485,12 @@ Installables SourceExprCommand::parseInstallables(
auto e = state->parseStdin();
state->eval(e, *vFile);
}
else if (file)
state->evalFile(lookupFileArg(*state, *file), *vFile);
else if (file) {
state->evalFile(lookupFileArg(*state, *file, CanonPath::fromCwd(getCommandBaseDir())), *vFile);
}
else {
auto e = state->parseExprFromString(*expr, state->rootPath(CanonPath::fromCwd()));
CanonPath dir(CanonPath::fromCwd(getCommandBaseDir()));
auto e = state->parseExprFromString(*expr, state->rootPath(dir));
state->eval(e, *vFile);
}
@@ -485,7 +525,7 @@ Installables SourceExprCommand::parseInstallables(
}
try {
auto [flakeRef, fragment] = parseFlakeRefWithFragment(std::string { prefix }, absPath("."));
auto [flakeRef, fragment] = parseFlakeRefWithFragment(std::string { prefix }, absPath(getCommandBaseDir()));
result.push_back(make_ref<InstallableFlake>(
this,
getEvalState(),
@@ -669,7 +709,7 @@ BuiltPaths Installable::toBuiltPaths(
BuiltPaths res;
for (auto & drvPath : Installable::toDerivations(store, installables, true))
res.push_back(BuiltPath::Opaque{drvPath});
res.emplace_back(BuiltPath::Opaque{drvPath});
return res;
}
}
@@ -739,9 +779,7 @@ RawInstallablesCommand::RawInstallablesCommand()
expectArgs({
.label = "installables",
.handler = {&rawInstallables},
.completer = {[&](size_t, std::string_view prefix) {
completeInstallable(prefix);
}}
.completer = getCompleteInstallable(),
});
}
@@ -754,6 +792,17 @@ void RawInstallablesCommand::applyDefaultInstallables(std::vector<std::string> &
}
}
std::vector<FlakeRef> RawInstallablesCommand::getFlakeRefsForCompletion()
{
applyDefaultInstallables(rawInstallables);
std::vector<FlakeRef> res;
for (auto i : rawInstallables)
res.push_back(parseFlakeRefWithFragment(
expandTilde(i),
absPath(getCommandBaseDir())).first);
return res;
}
void RawInstallablesCommand::run(ref<Store> store)
{
if (readFromStdIn && !isatty(STDIN_FILENO)) {
@@ -767,10 +816,13 @@ void RawInstallablesCommand::run(ref<Store> store)
run(store, std::move(rawInstallables));
}
std::vector<std::string> RawInstallablesCommand::getFlakesForCompletion()
std::vector<FlakeRef> InstallableCommand::getFlakeRefsForCompletion()
{
applyDefaultInstallables(rawInstallables);
return rawInstallables;
return {
parseFlakeRefWithFragment(
expandTilde(_installable),
absPath(getCommandBaseDir())).first
};
}
void InstallablesCommand::run(ref<Store> store, std::vector<std::string> && rawInstallables)
@@ -786,9 +838,7 @@ InstallableCommand::InstallableCommand()
.label = "installable",
.optional = true,
.handler = {&_installable},
.completer = {[&](size_t, std::string_view prefix) {
completeInstallable(prefix);
}}
.completer = getCompleteInstallable(),
});
}
-1
View File
@@ -1,7 +1,6 @@
#pragma once
///@file
#include "util.hh"
#include "path.hh"
#include "outputs-spec.hh"
#include "derived-path.hh"
+1
View File
@@ -1,6 +1,7 @@
#include "markdown.hh"
#include "util.hh"
#include "finally.hh"
#include "terminal.hh"
#include <sys/queue.h>
#include <lowdown.h>
+4 -1
View File
@@ -22,6 +22,7 @@ extern "C" {
#include "repl.hh"
#include "ansicolor.hh"
#include "signals.hh"
#include "shared.hh"
#include "eval.hh"
#include "eval-cache.hh"
@@ -36,6 +37,8 @@ extern "C" {
#include "globals.hh"
#include "flake/flake.hh"
#include "flake/lockfile.hh"
#include "users.hh"
#include "terminal.hh"
#include "editor-for.hh"
#include "finally.hh"
#include "markdown.hh"
@@ -922,7 +925,7 @@ std::ostream & NixRepl::printValue(std::ostream & str, Value & v, unsigned int m
case nString:
str << ANSI_WARNING;
printLiteralString(str, v.string.s);
printLiteralString(str, v.string_view());
str << ANSI_NORMAL;
break;
+1 -2
View File
@@ -1,6 +1,5 @@
#include "attr-path.hh"
#include "eval-inline.hh"
#include "util.hh"
namespace nix {
@@ -132,7 +131,7 @@ std::pair<SourcePath, uint32_t> findPackageFilename(EvalState & state, Value & v
if (colon == std::string::npos) fail();
std::string filename(fn, 0, colon);
auto lineno = std::stoi(std::string(fn, colon + 1, std::string::npos));
return {CanonPath(fn.substr(0, colon)), lineno};
return {SourcePath{path.accessor, CanonPath(fn.substr(0, colon))}, lineno};
} catch (std::invalid_argument & e) {
fail();
abort();
+6 -5
View File
@@ -1,3 +1,4 @@
#include "users.hh"
#include "eval-cache.hh"
#include "sqlite.hh"
#include "eval.hh"
@@ -50,7 +51,7 @@ struct AttrDb
Path cacheDir = getCacheDir() + "/nix/eval-cache-v5";
createDirs(cacheDir);
Path dbPath = cacheDir + "/" + fingerprint.to_string(Base16, false) + ".sqlite";
Path dbPath = cacheDir + "/" + fingerprint.to_string(HashFormat::Base16, false) + ".sqlite";
state->db = SQLite(dbPath);
state->db.isCache();
@@ -440,8 +441,8 @@ Value & AttrCursor::forceValue()
if (root->db && (!cachedValue || std::get_if<placeholder_t>(&cachedValue->second))) {
if (v.type() == nString)
cachedValue = {root->db->setString(getKey(), v.string.s, v.string.context),
string_t{v.string.s, {}}};
cachedValue = {root->db->setString(getKey(), v.c_str(), v.context()),
string_t{v.c_str(), {}}};
else if (v.type() == nPath) {
auto path = v.path().path;
cachedValue = {root->db->setString(getKey(), path.abs()), string_t{path.abs(), {}}};
@@ -582,7 +583,7 @@ std::string AttrCursor::getString()
if (v.type() != nString && v.type() != nPath)
root->state.error("'%s' is not a string but %s", getAttrPathStr()).debugThrow<TypeError>();
return v.type() == nString ? v.string.s : v.path().to_string();
return v.type() == nString ? v.c_str() : v.path().to_string();
}
string_t AttrCursor::getStringWithContext()
@@ -624,7 +625,7 @@ string_t AttrCursor::getStringWithContext()
if (v.type() == nString) {
NixStringContext context;
copyContext(v, context);
return {v.string.s, std::move(context)};
return {v.c_str(), std::move(context)};
}
else if (v.type() == nPath)
return {v.path().to_string(), {}};
+1
View File
@@ -1,3 +1,4 @@
#include "users.hh"
#include "globals.hh"
#include "profiles.hh"
#include "eval.hh"
+18 -10
View File
@@ -1,4 +1,6 @@
#pragma once
///@file
#include "config.hh"
namespace nix {
@@ -29,10 +31,12 @@ struct EvalSettings : Config
this, false, "restrict-eval",
R"(
If set to `true`, the Nix evaluator will not allow access to any
files outside of the Nix search path (as set via the `NIX_PATH`
environment variable or the `-I` option), or to URIs outside of
[`allowed-uris`](../command-ref/conf-file.md#conf-allowed-uris).
The default is `false`.
files outside of
[`builtins.nixPath`](@docroot@/language/builtin-constants.md#builtins-nixPath),
or to URIs outside of
[`allowed-uris`](@docroot@/command-ref/conf-file.md#conf-allowed-uris).
Also the default value for [`nix-path`](#conf-nix-path) is ignored, such that only explicitly set search path entries are taken into account.
)"};
Setting<bool> pureEval{this, false, "pure-eval",
@@ -40,18 +44,22 @@ struct EvalSettings : Config
Pure evaluation mode ensures that the result of Nix expressions is fully determined by explicitly declared inputs, and not influenced by external state:
- Restrict file system and network access to files specified by cryptographic hash
- Disable [`bultins.currentSystem`](@docroot@/language/builtin-constants.md#builtins-currentSystem) and [`builtins.currentTime`](@docroot@/language/builtin-constants.md#builtins-currentTime)
- Disable impure constants:
- [`bultins.currentSystem`](@docroot@/language/builtin-constants.md#builtins-currentSystem)
- [`builtins.currentTime`](@docroot@/language/builtin-constants.md#builtins-currentTime)
- [`builtins.nixPath`](@docroot@/language/builtin-constants.md#builtins-nixPath)
)"
};
Setting<bool> enableImportFromDerivation{
this, true, "allow-import-from-derivation",
R"(
By default, Nix allows you to `import` from a derivation, allowing
building at evaluation time. With this option set to false, Nix will
throw an error when evaluating an expression that uses this feature,
allowing users to ensure their evaluation will not require any
builds to take place.
By default, Nix allows [Import from Derivation](@docroot@/language/import-from-derivation.md).
With this option set to `false`, Nix will throw an error when evaluating an expression that uses this feature,
even when the required store object is readily available.
This ensures that evaluation will not require any builds to take place,
regardless of the state of the store.
)"};
Setting<Strings> allowedUris{this, {}, "allowed-uris",
Loaded 100 of 1077 files, more files were not shown because too many files have changed in this diff. Show more