Compare commits

..
39 Commits
Author SHA1 Message Date
Robert Hensing 2cc8387cb3 WIP: accumulate attrset updates, use k-way merge
This seems to be slower as of yet.

TODO

- [ ] remove merge pass 2 into 1:
  - pass 1: fill UpdateQueue
  - pass 2: stats (like max output size)
  - pass 3: k-way merge
- [ ] maybe tracking sortedness is useful and cheap? in pass 1, building the UpdateQueue
- [ ] make it actually fast; different algorithm depending on size distribution and number of update ops?
  - currently solution is the min-heap or priority queue approach
  - divide and conquer (see also Nixpkgs which manually implements this for pkgs/by-name)
    binary tree-shaped sorted merge operations
    - can we take the size of attrsets into account to balance the work?
      1000 + (1 + (1 + 1)) is better than ((1000+1)+1)+1  (where number refers to attrset size)
  - iterative pairwise merging is more or less what we had, the tree of updates is flattened. Probably worse.
  - perhaps some combination of solutions, depending on a heuristic
- [ ] split callFunction into two parts, effectively
        - call2Thunk (create the new scope's Env etc)
        - eval that thunk (evaluate the body)
- [ ] implement ExprApply::evalForUpdate - easy after the split
- [ ] make foldl' (//) work?
- [ ] optimize listToAttrs and other primops?
2024-08-13 21:09:11 +02:00
John Ericson 59def6c23b Merge pull request #11178 from obsidiansystems/better-exe-lookup
Move `NIX_BIN_DIR` and all logic using it to the Nix executable itself
2024-08-12 12:21:56 -05:00
John EricsonandRobert Hensing 58b03ef1cd Move NIX_BIN_DIR and all logic using it to the Nix executable itself
This is because with the split packages of the Meson build, we simply
have no idea what directory the binaries will be installed in when we
build the library.

In the process of doing so, consolidate and make more sophisticated the
logic to cope with a few corner cases (e.g. `NIX_BIN_DIR` exists, but no
binaries are inside it).

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-08-12 12:29:17 -04:00
Robert Hensing 18485d2d53 Merge pull request #11188 from lf-/jade/kill-int-overflow
Ban integer overflow in the Nix language
2024-08-11 04:24:16 +02:00
Robert Hensing 3cc2e2a0ac Edit docs 2024-08-11 03:31:44 +02:00
John Ericson cfe66dbec3 Merge pull request #11218 from obsidiansystems/better-executable-path
Factor out `lookupExecutable` and other PATH improvements
2024-08-07 23:21:19 -05:00
John EricsonandRobert Hensing 0646b6cd61 Update comments / documentation.
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-08-07 22:29:56 -05:00
John EricsonandRobert Hensing 6c861b9c51 Factor out lookupExecutable and other PATH improvments
This ended up motivating a good deal of other infra improvements in
order to get Windows right:

- `OsString` to complement `std::filesystem::path`

- env var code for working with the underlying `OsString`s

- Rename `PATHNG_LITERAL` to `OS_STR`

- `NativePathTrait` renamed to `OsPathTrait`, given a character template
  parameter until #9205 is complete.

Split `tests.cc` matching split of `util.{cc,hh}` last year.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-08-07 18:12:58 -04:00
tomberek 0836888002 Merge pull request #11233 from rhendric/rhendric/reference-manual-2
docs: add language/string-literals.md
2024-08-07 16:17:16 -04:00
tomberek 7354681804 Merge branch 'master' into rhendric/reference-manual-2 2024-08-07 15:25:02 -04:00
Eelco Dolstra 2ed075ffc0 Merge pull request #11257 from DeterminateSystems/no-failed-substitution
PathSubstitutionGoal: Fix spurious "failed" count in the progress bar
2024-08-06 10:33:26 +02:00
John Ericson 79abf816b9 Merge pull request #11256 from obsidiansystems/test-organize
Split tests, organize more string functions
2024-08-05 12:44:39 -05:00
Eelco Dolstra 0a00bd07b2 PathSubstitutionGoal: Fix spurious "failed" count in the progress bar
It is not an error if queryPathInfo() indicates that a path does not
exist in the substituter.

Fixes #11198. This was broken in 846869da0e.
2024-08-05 18:56:02 +02:00
John Ericson 9d2d4d11e6 Split tests, organize more string functions
The test split matches PR #8920, so the utility files and tests files
are once again to 1-1. The string changes continues what was started in
PR #11093.
2024-08-05 12:50:13 -04:00
John Ericson 1fce591cbc Merge pull request #11255 from DeterminateSystems/test-s3-binary-cache-store
Add a VM test for S3BinaryCacheStore
2024-08-05 11:12:08 -05:00
Eelco Dolstra 2950f9e18a Add a VM test for S3BinaryCacheStore
Fixes #11238.
2024-08-05 11:48:50 +02:00
a-kenji 5a6e28e166 docs: installable remove alternate expression flag (#11254) 2024-08-04 23:03:21 +00:00
tomberek ea1f87ecda Merge pull request #11145 from rhendric/rhendric/parser-lalr
Make parser LALR, conflict-free
2024-08-04 03:59:10 -04:00
John Ericson 09199a40cd Merge pull request #11244 from Mic92/make-c-backwards-compat
allow to c api with older c versions
2024-08-02 12:30:43 -05:00
Jörg Thalheim 739418504c allow to c api with older c versions
In the FFI world we have many tools that are not gcc/clang and therefore
not always support the latest C standard. This fixes support with cffi
i.e. used in https://github.com/tweag/python-nix
2024-08-02 17:19:45 +02:00
Eelco Dolstra 838b666a80 Merge pull request #11237 from DeterminateSystems/fix-s3-store
Fix the S3 store
2024-08-01 17:46:29 +02:00
Eelco Dolstra 9b5b7b7963 Fix the S3 store
It was failing with:

   error: AWS error fetching 'nix-cache-info': The specified bucket does not exist

because `S3BinaryCacheStoreImpl` had a `bucketName` field that
shadowed the inherited `bucketName from `S3BinaryCacheStoreConfig`.
2024-08-01 16:51:57 +02:00
Eelco Dolstra 854346045a Merge pull request #11236 from edolstra/bump-2.25.0
Bump version
2024-08-01 11:33:48 +02:00
Eelco Dolstra 2edc570e3e Merge pull request #11235 from edolstra/fix-release-script
Fix the release script
2024-08-01 11:33:22 +02:00
Ryan Hendrickson b291b61089 docs: editorial quibbles (#11232) 2024-08-01 11:14:49 +02:00
Eelco Dolstra 30aca6f243 Bump version 2024-08-01 10:43:00 +02:00
Eelco Dolstra 617e711820 'build' is now 'build.nix' 2024-08-01 10:41:42 +02:00
Ryan Hendrickson 17318bc70d docs: fix string literal example formatting 2024-07-31 19:22:17 -04:00
Ryan Hendrickson 9e8afc68e5 docs: add language/string-literals.md 2024-07-31 19:07:57 -04:00
Ryan HendricksonandValentin Gagarin 6ed67d35ed docs: add variables; rework scope (#11062)
Co-authored-by: Valentin Gagarin <valentin@gagarin.work>
2024-07-31 21:39:43 +00:00
Jade Lovelace 5878b1475f doc: release notes for banning integer overflow
Change-Id: Ib75ab5b8b4d879035d7ee7678f9cd0c491a39c0a
2024-07-30 18:13:05 -07:00
Jade Lovelace bf050d9e96 docs: update to define integer overflow
Change-Id: Ie8a1b31035f2d27a220e5df2e9e178ec3b39ee68
2024-07-30 18:13:05 -07:00
Jade Lovelace 7b6622d733 language: cleanly ban integer overflows
This also bans various sneaking of negative numbers from the language
into unsuspecting builtins as was exposed while auditing the
consequences of changing the Nix language integer type to a newtype.

It's unlikely that this change comprehensively ensures correctness when
passing integers out of the Nix language and we should probably add a
checked-narrowing function or something similar, but that's out of scope
for the immediate change.

During the development of this I found a few fun facts about the
language:
- You could overflow integers by converting from unsigned JSON values.
- You could overflow unsigned integers by converting negative numbers
  into them when going into Nix config, into fetchTree, and into flake
  inputs.

  The flake inputs and Nix config cannot actually be tested properly
  since they both ban thunks, however, we put in checks anyway because
  it's possible these could somehow be used to do such shenanigans some
  other way.

Note that Lix has banned Nix language integer overflows since the very
first public beta, but threw a SIGILL about them because we run with
-fsanitize=signed-overflow -fsanitize-undefined-trap-on-error in
production builds. Since the Nix language uses signed integers, overflow
was simply undefined behaviour, and since we defined that to trap, it
did.

Trapping on it was a bad UX, but we didn't even entirely notice
that we had done this at all until it was reported as a bug a couple of
months later (which is, to be fair, that flag working as intended), and
it's got enough production time that, aside from code that is IMHO buggy
(and which is, in any case, not in nixpkgs) such as
https://git.lix.systems/lix-project/lix/issues/445, we don't think
anyone doing anything reasonable actually depends on wrapping overflow.

Even for weird use cases such as doing funny bit crimes, it doesn't make
sense IMO to have wrapping behaviour, since two's complement arithmetic
overflow behaviour is so *aggressively* not what you want for *any* kind
of mathematics/algorithms. The Nix language exists for package
management, a domain where bit crimes are already only dubiously in
scope to begin with, and it makes a lot more sense for that domain for
the integers to never lose precision, either by throwing errors if they
would, or by being arbitrary-precision.

Fixes: https://github.com/NixOS/nix/issues/10968
Original-CL: https://gerrit.lix.systems/c/lix/+/1596

Change-Id: I51f253840c4af2ea5422b8a420aa5fafbf8fae75
2024-07-30 18:13:05 -07:00
Jade Lovelace e28cb67d41 libutil: add checked arithmetic tools
This is in preparation for adding checked arithmetic to the evaluator.

Change-Id: I6e115ce8f5411feda1706624977a4dcd5efd4d13
2024-07-30 18:13:05 -07:00
Jade Lovelace dd75711895 Use std::strong_ordering for version comparison
The actual motive here is the avoidance of integer overflow if we were
to make these use checked NixInts and retain the subtraction.

However, the actual *intent* of this code is a three-way comparison,
which can be done with operator<=>, so we should just do *that* instead.

Change-Id: I7f9a7da1f3176424b528af6d1b4f1591e4ab26bf
2024-07-30 18:13:05 -07:00
Ryan Hendrickson 18db46a6cb parser.y: GLR -> LALR 2024-07-25 15:48:58 -04:00
Ryan Hendrickson 6e3b9e6a4d parser.y: eliminate conflicts 2024-07-25 15:48:58 -04:00
Ryan Hendrickson b0a8430e85 parser.y: move attr doc setting into addAttr 2024-07-25 15:48:58 -04:00
Ryan Hendrickson 429a197d24 parser.y: use names where I'll be refactoring 2024-07-25 15:48:58 -04:00
574 changed files with 9243 additions and 12166 deletions

No files matched your search

+1
View File
@@ -31,3 +31,4 @@ AlwaysBreakBeforeMultilineStrings: true
IndentPPDirectives: AfterHash
PPIndentWidth: 2
BinPackArguments: false
BreakBeforeTernaryOperators: true
+32
View File
@@ -0,0 +1,32 @@
name: Backport
on:
pull_request_target:
types: [closed, labeled]
permissions:
contents: read
jobs:
backport:
name: Backport Pull Request
permissions:
# for zeebe-io/backport-action
contents: write
pull-requests: write
if: github.repository_owner == 'NixOS' && github.event.pull_request.merged == true && (github.event_name != 'labeled' || startsWith('backport', github.event.label.name))
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.head.sha }}
# required to find all branches
fetch-depth: 0
- name: Create backport PRs
# should be kept in sync with `version`
uses: zeebe-io/backport-action@v3.0.2
with:
# Config README: https://github.com/zeebe-io/backport-action#backport-action
github_token: ${{ secrets.GITHUB_TOKEN }}
github_workspace: ${{ github.workspace }}
pull_description: |-
Automatic backport to `${target_branch}`, triggered by a label in #${pull_number}.
# should be kept in sync with `uses`
version: v0.0.5
+103 -87
View File
@@ -7,28 +7,14 @@ on:
permissions: read-all
jobs:
eval:
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: cachix/install-nix-action@v30
- run: nix --experimental-features 'nix-command flakes' flake show --all-systems --json
tests:
needs: [check_secrets]
strategy:
fail-fast: false
matrix:
include:
- scenario: on ubuntu
runs-on: ubuntu-24.04
os: linux
- scenario: on macos
runs-on: macos-14
os: darwin
name: tests ${{ matrix.scenario }}
runs-on: ${{ matrix.runs-on }}
os: [ubuntu-latest, macos-latest]
runs-on: ${{ matrix.os }}
timeout-minutes: 60
steps:
- uses: actions/checkout@v4
@@ -37,54 +23,102 @@ jobs:
- uses: cachix/install-nix-action@V27
with:
# The sandbox would otherwise be disabled by default on Darwin
extra_nix_config: |
sandbox = true
max-jobs = 1
- uses: DeterminateSystems/magic-nix-cache-action@main
# Since ubuntu 22.30, unprivileged usernamespaces are no longer allowed to map to the root user:
# https://ubuntu.com/blog/ubuntu-23-10-restricted-unprivileged-user-namespaces
- run: sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0
if: matrix.os == 'linux'
- run: scripts/build-checks
- run: scripts/prepare-installer-for-github-actions
- name: Upload installer tarball
uses: actions/upload-artifact@v4
extra_nix_config: "sandbox = true"
- run: echo CACHIX_NAME="$(echo $GITHUB_REPOSITORY-install-tests | tr "[A-Z]/" "[a-z]-")" >> $GITHUB_ENV
- uses: cachix/cachix-action@v15
if: needs.check_secrets.outputs.cachix == 'true'
with:
name: installer-${{matrix.os}}
path: out/*
name: '${{ env.CACHIX_NAME }}'
signingKey: '${{ secrets.CACHIX_SIGNING_KEY }}'
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
- if: matrix.os == 'ubuntu-latest'
run: |
free -h
swapon --show
swap=$(swapon --show --noheadings | head -n 1 | awk '{print $1}')
echo "Found swap: $swap"
sudo swapoff $swap
# resize it (fallocate)
sudo fallocate -l 10G $swap
sudo mkswap $swap
sudo swapon $swap
free -h
(
while sleep 60; do
free -h
done
) &
- run: nix --experimental-features 'nix-command flakes' flake check -L
# Steps to test CI automation in your own fork.
# Cachix:
# 1. Sign-up for https://www.cachix.org/
# 2. Create a cache for $githubuser-nix-install-tests
# 3. Create a cachix auth token and save it in https://github.com/$githubuser/nix/settings/secrets/actions in "Repository secrets" as CACHIX_AUTH_TOKEN
# Dockerhub:
# 1. Sign-up for https://hub.docker.com/
# 2. Store your dockerhub username as DOCKERHUB_USERNAME in "Repository secrets" of your fork repository settings (https://github.com/$githubuser/nix/settings/secrets/actions)
# 3. Create an access token in https://hub.docker.com/settings/security and store it as DOCKERHUB_TOKEN in "Repository secrets" of your fork
check_secrets:
permissions:
contents: none
name: Check Cachix and Docker secrets present for installer tests
runs-on: ubuntu-latest
outputs:
cachix: ${{ steps.secret.outputs.cachix }}
docker: ${{ steps.secret.outputs.docker }}
steps:
- name: Check for secrets
id: secret
env:
_CACHIX_SECRETS: ${{ secrets.CACHIX_SIGNING_KEY }}${{ secrets.CACHIX_AUTH_TOKEN }}
_DOCKER_SECRETS: ${{ secrets.DOCKERHUB_USERNAME }}${{ secrets.DOCKERHUB_TOKEN }}
run: |
echo "::set-output name=cachix::${{ env._CACHIX_SECRETS != '' }}"
echo "::set-output name=docker::${{ env._DOCKER_SECRETS != '' }}"
installer:
needs: [tests, check_secrets]
if: github.event_name == 'push' && needs.check_secrets.outputs.cachix == 'true'
runs-on: ubuntu-latest
outputs:
installerURL: ${{ steps.prepare-installer.outputs.installerURL }}
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- run: echo CACHIX_NAME="$(echo $GITHUB_REPOSITORY-install-tests | tr "[A-Z]/" "[a-z]-")" >> $GITHUB_ENV
- uses: cachix/install-nix-action@V27
with:
install_url: https://releases.nixos.org/nix/nix-2.20.3/install
- uses: cachix/cachix-action@v15
with:
name: '${{ env.CACHIX_NAME }}'
signingKey: '${{ secrets.CACHIX_SIGNING_KEY }}'
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
cachixArgs: '-v'
- id: prepare-installer
run: scripts/prepare-installer-for-github-actions
installer_test:
needs: [tests]
needs: [installer, check_secrets]
if: github.event_name == 'push' && needs.check_secrets.outputs.cachix == 'true'
strategy:
fail-fast: false
matrix:
include:
- scenario: on ubuntu
runs-on: ubuntu-24.04
os: linux
- scenario: on macos
runs-on: macos-14
os: darwin
name: installer test ${{ matrix.scenario }}
runs-on: ${{ matrix.runs-on }}
os: [ubuntu-latest, macos-latest]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v4
- name: Download installer tarball
uses: actions/download-artifact@v4
- run: echo CACHIX_NAME="$(echo $GITHUB_REPOSITORY-install-tests | tr "[A-Z]/" "[a-z]-")" >> $GITHUB_ENV
- uses: cachix/install-nix-action@V27
with:
name: installer-${{matrix.os}}
path: out
- name: Serving installer
id: serving_installer
run: ./scripts/serve-installer-for-github-actions
- uses: cachix/install-nix-action@v30
with:
install_url: 'http://localhost:8126/install'
install_options: "--tarball-url-prefix http://localhost:8126/"
install_url: '${{needs.installer.outputs.installerURL}}'
install_options: "--tarball-url-prefix https://${{ env.CACHIX_NAME }}.cachix.org/serve"
- run: sudo apt install fish zsh
if: matrix.os == 'linux'
if: matrix.os == 'ubuntu-latest'
- run: brew install fish
if: matrix.os == 'darwin'
if: matrix.os == 'macos-latest'
- run: exec bash -c "nix-instantiate -E 'builtins.currentTime' --eval"
- run: exec sh -c "nix-instantiate -E 'builtins.currentTime' --eval"
- run: exec zsh -c "nix-instantiate -E 'builtins.currentTime' --eval"
@@ -92,50 +126,32 @@ jobs:
- run: exec bash -c "nix-channel --add https://releases.nixos.org/nixos/unstable/nixos-23.05pre466020.60c1d71f2ba nixpkgs"
- run: exec bash -c "nix-channel --update && nix-env -iA nixpkgs.hello && hello"
# Steps to test CI automation in your own fork.
# 1. Sign-up for https://hub.docker.com/
# 2. Store your dockerhub username as DOCKERHUB_USERNAME in "Repository secrets" of your fork repository settings (https://github.com/$githubuser/nix/settings/secrets/actions)
# 3. Create an access token in https://hub.docker.com/settings/security and store it as DOCKERHUB_TOKEN in "Repository secrets" of your fork
check_secrets:
permissions:
contents: none
name: Check Docker secrets present for installer tests
runs-on: ubuntu-24.04
outputs:
docker: ${{ steps.secret.outputs.docker }}
steps:
- name: Check for secrets
id: secret
env:
_DOCKER_SECRETS: ${{ secrets.DOCKERHUB_USERNAME }}${{ secrets.DOCKERHUB_TOKEN }}
run: |
echo "::set-output name=docker::${{ env._DOCKER_SECRETS != '' }}"
docker_push_image:
needs: [tests, vm_tests, check_secrets]
needs: [check_secrets, tests]
permissions:
contents: read
packages: write
if: >-
needs.check_secrets.outputs.docker == 'true' &&
github.event_name == 'push' &&
github.ref_name == 'master'
runs-on: ubuntu-24.04
github.ref_name == 'master' &&
needs.check_secrets.outputs.cachix == 'true' &&
needs.check_secrets.outputs.docker == 'true'
runs-on: ubuntu-latest
steps:
- name: Check for secrets
id: secret
env:
_DOCKER_SECRETS: ${{ secrets.DOCKERHUB_USERNAME }}${{ secrets.DOCKERHUB_TOKEN }}
run: |
echo "::set-output name=docker::${{ env._DOCKER_SECRETS != '' }}"
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: cachix/install-nix-action@V27
with:
install_url: https://releases.nixos.org/nix/nix-2.20.3/install
- uses: DeterminateSystems/magic-nix-cache-action@main
- run: echo NIX_VERSION="$(nix --experimental-features 'nix-command flakes' eval .\#nix.version | tr -d \")" >> $GITHUB_ENV
- run: echo CACHIX_NAME="$(echo $GITHUB_REPOSITORY-install-tests | tr "[A-Z]/" "[a-z]-")" >> $GITHUB_ENV
- run: echo NIX_VERSION="$(nix --experimental-features 'nix-command flakes' eval .\#default.version | tr -d \")" >> $GITHUB_ENV
- uses: cachix/cachix-action@v15
if: needs.check_secrets.outputs.cachix == 'true'
with:
name: '${{ env.CACHIX_NAME }}'
signingKey: '${{ secrets.CACHIX_SIGNING_KEY }}'
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
- run: nix --experimental-features 'nix-command flakes' build .#dockerImage -L
- run: docker load -i ./result/image.tar.gz
- run: docker tag nix:$NIX_VERSION ${{ secrets.DOCKERHUB_USERNAME }}/nix:$NIX_VERSION
@@ -172,7 +188,7 @@ jobs:
docker push $IMAGE_ID:master
vm_tests:
runs-on: ubuntu-24.04
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v4
- uses: DeterminateSystems/nix-installer-action@main
@@ -195,7 +211,7 @@ jobs:
flake_regressions:
needs: vm_tests
runs-on: ubuntu-24.04
runs-on: ubuntu-22.04
steps:
- name: Checkout nix
uses: actions/checkout@v4
+1 -1
View File
@@ -15,7 +15,7 @@ permissions:
jobs:
labels:
runs-on: ubuntu-24.04
runs-on: ubuntu-latest
if: github.repository_owner == 'NixOS'
steps:
- uses: actions/labeler@v5
-94
View File
@@ -1,94 +0,0 @@
queue_rules:
- name: default
# all required tests need to go here
merge_conditions:
- check-success=installer
- check-success=installer_test (macos-latest)
- check-success=installer_test (ubuntu-latest)
- check-success=tests on macos
- check-success=tests on ubuntu
- check-success=installer test on macos
- check-success=installer test on ubuntu
- check-success=vm_tests
merge_method: rebase
batch_size: 5
pull_request_rules:
- name: merge using the merge queue
conditions:
- base=master
- label~=merge-queue|dependencies
actions:
queue: {}
# The rules below will first create backport pull requests and put those in a merge queue.
- name: backport patches to 2.18
conditions:
- label=backport 2.18-maintenance
actions:
backport:
branches:
- 2.18-maintenance
labels:
- merge-queue
- name: backport patches to 2.19
conditions:
- label=backport 2.19-maintenance
actions:
backport:
branches:
- 2.19-maintenance
labels:
- merge-queue
- name: backport patches to 2.20
conditions:
- label=backport 2.20-maintenance
actions:
backport:
branches:
- 2.20-maintenance
labels:
- merge-queue
- name: backport patches to 2.21
conditions:
- label=backport 2.21-maintenance
actions:
backport:
branches:
- 2.21-maintenance
labels:
- merge-queue
- name: backport patches to 2.22
conditions:
- label=backport 2.22-maintenance
actions:
backport:
branches:
- 2.22-maintenance
labels:
- merge-queue
- name: backport patches to 2.23
conditions:
- label=backport 2.23-maintenance
actions:
backport:
branches:
- 2.23-maintenance
labels:
- merge-queue
- name: backport patches to 2.24
conditions:
- label=backport 2.24-maintenance
actions:
backport:
branches:
- "2.24-maintenance"
labels:
- merge-queue
+1 -1
View File
@@ -1 +1 @@
2.24.14
2.25.0
+4 -13
View File
@@ -62,16 +62,12 @@ AC_CHECK_TOOL([AR], [ar])
AC_SYS_LARGEFILE
# OS-specific stuff.
# Solaris-specific stuff.
case "$host_os" in
solaris*)
# Solaris requires -lsocket -lnsl for network functions
LDFLAGS="-lsocket -lnsl $LDFLAGS"
;;
darwin*)
# Need to link to libsandbox.
LDFLAGS="-lsandbox $LDFLAGS"
;;
esac
@@ -93,10 +89,9 @@ AC_LANG_POP(C++)
AC_CHECK_FUNCS([statvfs pipe2])
# Check for lutimes and utimensat, optionally used for changing the
# mtime of symlinks.
AC_CHECK_DECLS([AT_SYMLINK_NOFOLLOW], [], [], [[#include <fcntl.h>]])
AC_CHECK_FUNCS([lutimes utimensat])
# Check for lutimes, optionally used for changing the mtime of
# symlinks.
AC_CHECK_FUNCS([lutimes])
# Check whether the store optimiser can optimise symlinks.
@@ -175,10 +170,6 @@ AS_IF(
[test "$ENABLE_FUNCTIONAL_TESTS" == "yes" || test "$ENABLE_DOC_GEN" == "yes"],
[NEED_PROG(jq, jq)])
AS_IF(
[test "$ENABLE_DOC_GEN" == "yes"],
[NEED_PROG(man, man)])
AS_IF([test "$ENABLE_BUILD" == "yes"],[
# Look for boost, a required dependency.
+10 -9
View File
@@ -1,9 +1,10 @@
(import (
let
lock = builtins.fromJSON (builtins.readFile ./flake.lock);
in
fetchTarball {
url = "https://github.com/edolstra/flake-compat/archive/${lock.nodes.flake-compat.locked.rev}.tar.gz";
sha256 = lock.nodes.flake-compat.locked.narHash;
}
) { src = ./.; }).defaultNix
(import
(
let lock = builtins.fromJSON (builtins.readFile ./flake.lock); in
fetchTarball {
url = "https://github.com/edolstra/flake-compat/archive/${lock.nodes.flake-compat.locked.rev}.tar.gz";
sha256 = lock.nodes.flake-compat.locked.narHash;
}
)
{ src = ./.; }
).defaultNix
+1 -9
View File
@@ -5,15 +5,7 @@ in
builtinsInfo:
let
showBuiltin =
name:
{
doc,
type ? null,
args ? [ ],
experimental-feature ? null,
impure-only ? false,
}:
showBuiltin = name: { doc, type ? null, args ? [ ], experimental-feature ? null, impure-only ? false }:
let
type' = optionalString (type != null) " (${type})";
+62 -90
View File
@@ -32,13 +32,7 @@ let
commandInfo = fromJSON commandDump;
showCommand =
{
command,
details,
filename,
toplevel,
}:
showCommand = { command, details, filename, toplevel }:
let
result = ''
@@ -62,27 +56,26 @@ let
${maybeOptions}
'';
showSynopsis =
command: args:
showSynopsis = command: args:
let
showArgument = arg: "*${arg.label}*" + optionalString (!arg ? arity) "...";
showArgument = arg: "*${arg.label}*" + optionalString (! arg ? arity) "...";
arguments = concatStringsSep " " (map showArgument args);
in
''
in ''
`${command}` [*option*...] ${arguments}
'';
maybeSubcommands = optionalString (details ? commands && details.commands != { }) ''
where *subcommand* is one of the following:
maybeSubcommands = optionalString (details ? commands && details.commands != {})
''
where *subcommand* is one of the following:
${subcommands}
'';
${subcommands}
'';
subcommands = if length categories > 1 then listCategories else listSubcommands details.commands;
subcommands = if length categories > 1
then listCategories
else listSubcommands details.commands;
categories = sort (x: y: x.id < y.id) (
unique (map (cmd: cmd.category) (attrValues details.commands))
);
categories = sort (x: y: x.id < y.id) (unique (map (cmd: cmd.category) (attrValues details.commands)));
listCategories = concatStrings (map showCategory categories);
@@ -106,39 +99,38 @@ let
${allStores}
'';
index =
replaceStrings
[ "@store-types@" "./local-store.md" "./local-daemon-store.md" ]
[ storesOverview "#local-store" "#local-daemon-store" ]
details.doc;
index = replaceStrings
[ "@store-types@" "./local-store.md" "./local-daemon-store.md" ]
[ storesOverview "#local-store" "#local-daemon-store" ]
details.doc;
storesOverview =
let
showEntry = store: "- [${store.name}](#${store.slug})";
showEntry = store:
"- [${store.name}](#${store.slug})";
in
concatStringsSep "\n" (map showEntry storesList) + "\n";
allStores = concatStringsSep "\n" (attrValues storePages);
storePages = listToAttrs (
map (s: {
name = s.filename;
value = s.page;
}) storesList
);
storePages = listToAttrs
(map (s: { name = s.filename; value = s.page; }) storesList);
storesList = showStoreDocs {
storeInfo = commandInfo.stores;
inherit inlineHTML;
};
hasInfix =
infix: content:
hasInfix = infix: content:
builtins.stringLength content != builtins.stringLength (replaceStrings [ infix ] [ "" ] content);
in
optionalString (details ? doc) (
# An alternate implementation with builtins.match stack overflowed on some systems.
if hasInfix "@store-types@" details.doc then help-stores else details.doc
if hasInfix "@store-types@" details.doc
then help-stores
else details.doc
);
maybeOptions =
let
allVisibleOptions = filterAttrs (_: o: !o.hiddenCategory) (details.flags // toplevel.flags);
allVisibleOptions = filterAttrs
(_: o: ! o.hiddenCategory)
(details.flags // toplevel.flags);
in
optionalString (allVisibleOptions != { }) ''
# Options
@@ -150,73 +142,55 @@ let
> See [`man nix.conf`](@docroot@/command-ref/conf-file.md#command-line-flags) for overriding configuration settings with command line flags.
'';
showOptions =
inlineHTML: allOptions:
showOptions = inlineHTML: allOptions:
let
showCategory = cat: opts: ''
${optionalString (cat != "") "## ${cat}"}
${concatStringsSep "\n" (attrValues (mapAttrs showOption opts))}
'';
showOption =
name: option:
showOption = name: option:
let
result = trim ''
- ${item}
${option.description}
'';
item =
if inlineHTML then
''<span id="opt-${name}">[`--${name}`](#opt-${name})</span> ${shortName} ${labels}''
else
"`--${name}` ${shortName} ${labels}";
shortName = optionalString (option ? shortName) ("/ `-${option.shortName}`");
labels = optionalString (option ? labels) (concatStringsSep " " (map (s: "*${s}*") option.labels));
in
result;
categories =
mapAttrs
# Convert each group from a list of key-value pairs back to an attrset
(_: listToAttrs)
(groupBy (cmd: cmd.value.category) (attrsToList allOptions));
in
concatStrings (attrValues (mapAttrs showCategory categories));
in
squash result;
item = if inlineHTML
then ''<span id="opt-${name}">[`--${name}`](#opt-${name})</span> ${shortName} ${labels}''
else "`--${name}` ${shortName} ${labels}";
shortName = optionalString
(option ? shortName)
("/ `-${option.shortName}`");
labels = optionalString
(option ? labels)
(concatStringsSep " " (map (s: "*${s}*") option.labels));
in result;
categories = mapAttrs
# Convert each group from a list of key-value pairs back to an attrset
(_: listToAttrs)
(groupBy
(cmd: cmd.value.category)
(attrsToList allOptions));
in concatStrings (attrValues (mapAttrs showCategory categories));
in squash result;
appendName = filename: name: (if filename == "nix" then "nix3" else filename) + "-" + name;
processCommand =
{
command,
details,
filename,
toplevel,
}:
processCommand = { command, details, filename, toplevel }:
let
cmd = {
inherit command;
name = filename + ".md";
value = showCommand {
inherit
command
details
filename
toplevel
;
};
value = showCommand { inherit command details filename toplevel; };
};
subcommand =
subCmd:
processCommand {
command = command + " " + subCmd;
details = details.commands.${subCmd};
filename = appendName filename subCmd;
inherit toplevel;
};
in
[ cmd ] ++ concatMap subcommand (attrNames details.commands or { });
subcommand = subCmd: processCommand {
command = command + " " + subCmd;
details = details.commands.${subCmd};
filename = appendName filename subCmd;
inherit toplevel;
};
in [ cmd ] ++ concatMap subcommand (attrNames details.commands or {});
manpages = processCommand {
command = "nix";
@@ -225,11 +199,9 @@ let
toplevel = commandInfo.args;
};
tableOfContents =
let
showEntry = page: " - [${page.command}](command-ref/new-cli/${page.name})";
in
concatStringsSep "\n" (map showEntry manpages) + "\n";
tableOfContents = let
showEntry = page:
" - [${page.command}](command-ref/new-cli/${page.name})";
in concatStringsSep "\n" (map showEntry manpages) + "\n";
in
(listToAttrs manpages) // { "SUMMARY.md" = tableOfContents; }
in (listToAttrs manpages) // { "SUMMARY.md" = tableOfContents; }
+42 -74
View File
@@ -1,99 +1,67 @@
let
inherit (builtins)
attrValues
concatStringsSep
isAttrs
isBool
mapAttrs
;
inherit (import <nix/utils.nix>)
concatStrings
indent
optionalString
squash
;
inherit (builtins) attrValues concatStringsSep isAttrs isBool mapAttrs;
inherit (import <nix/utils.nix>) concatStrings indent optionalString squash;
in
# `inlineHTML` is a hack to accommodate inconsistent output from `lowdown`
{
prefix,
inlineHTML ? true,
}:
settingsInfo:
{ prefix, inlineHTML ? true }: settingsInfo:
let
showSetting =
prefix: setting:
{
description,
documentDefault,
defaultValue,
aliases,
value,
experimentalFeature,
}:
showSetting = prefix: setting: { description, documentDefault, defaultValue, aliases, value, experimentalFeature }:
let
result = squash ''
- ${item}
- ${item}
${indent " " body}
'';
item =
if inlineHTML then
''<span id="${prefix}-${setting}">[`${setting}`](#${prefix}-${setting})</span>''
else
"`${setting}`";
${indent " " body}
'';
item = if inlineHTML
then ''<span id="${prefix}-${setting}">[`${setting}`](#${prefix}-${setting})</span>''
else "`${setting}`";
# separate body to cleanly handle indentation
body = ''
${experimentalFeatureNote}
${experimentalFeatureNote}
${description}
${description}
**Default:** ${showDefault documentDefault defaultValue}
**Default:** ${showDefault documentDefault defaultValue}
${showAliases aliases}
'';
${showAliases aliases}
'';
experimentalFeatureNote = optionalString (experimentalFeature != null) ''
> **Warning**
>
> This setting is part of an
> [experimental feature](@docroot@/development/experimental-features.md).
>
> To change this setting, make sure the
> [`${experimentalFeature}` experimental feature](@docroot@/development/experimental-features.md#xp-feature-${experimentalFeature})
> is enabled.
> For example, include the following in [`nix.conf`](@docroot@/command-ref/conf-file.md):
>
> ```
> extra-experimental-features = ${experimentalFeature}
> ${setting} = ...
> ```
'';
> **Warning**
>
> This setting is part of an
> [experimental feature](@docroot@/development/experimental-features.md).
>
> To change this setting, make sure the
> [`${experimentalFeature}` experimental feature](@docroot@/development/experimental-features.md#xp-feature-${experimentalFeature})
> is enabled.
> For example, include the following in [`nix.conf`](@docroot@/command-ref/conf-file.md):
>
> ```
> extra-experimental-features = ${experimentalFeature}
> ${setting} = ...
> ```
'';
showDefault =
documentDefault: defaultValue:
showDefault = documentDefault: defaultValue:
if documentDefault then
# a StringMap value type is specified as a string, but
# this shows the value type. The empty stringmap is `null` in
# JSON, but that converts to `{ }` here.
if defaultValue == "" || defaultValue == [ ] || isAttrs defaultValue then
"*empty*"
else if isBool defaultValue then
if defaultValue then "`true`" else "`false`"
else
"`${toString defaultValue}`"
else
"*machine-specific*";
if defaultValue == "" || defaultValue == [] || isAttrs defaultValue
then "*empty*"
else if isBool defaultValue then
if defaultValue then "`true`" else "`false`"
else "`${toString defaultValue}`"
else "*machine-specific*";
showAliases =
aliases:
optionalString (aliases != [ ])
"**Deprecated alias:** ${(concatStringsSep ", " (map (s: "`${s}`") aliases))}";
showAliases = aliases:
optionalString (aliases != [])
"**Deprecated alias:** ${(concatStringsSep ", " (map (s: "`${s}`") aliases))}";
in
result;
in result;
in
concatStrings (attrValues (mapAttrs (showSetting prefix) settingsInfo))
in concatStrings (attrValues (mapAttrs (showSetting prefix) settingsInfo))
+14 -37
View File
@@ -1,20 +1,6 @@
let
inherit (builtins)
attrNames
listToAttrs
concatStringsSep
readFile
replaceStrings
;
inherit (import <nix/utils.nix>)
optionalString
filterAttrs
trim
squash
toLower
unique
indent
;
inherit (builtins) attrNames listToAttrs concatStringsSep readFile replaceStrings;
inherit (import <nix/utils.nix>) optionalString filterAttrs trim squash toLower unique indent;
showSettings = import <nix/generate-settings.nix>;
in
@@ -28,13 +14,7 @@ in
let
showStore =
{ name, slug }:
{
settings,
doc,
experimentalFeature,
}:
showStore = { name, slug }: { settings, doc, experimentalFeature }:
let
result = squash ''
# ${name}
@@ -45,10 +25,7 @@ let
## Settings
${showSettings {
prefix = "store-${slug}";
inherit inlineHTML;
} settings}
${showSettings { prefix = "store-${slug}"; inherit inlineHTML; } settings}
'';
experimentalFeatureNote = optionalString (experimentalFeature != null) ''
@@ -66,15 +43,15 @@ let
> extra-experimental-features = ${experimentalFeature}
> ```
'';
in
result;
in result;
storesList = map (name: rec {
inherit name;
slug = replaceStrings [ " " ] [ "-" ] (toLower name);
filename = "${slug}.md";
page = showStore { inherit name slug; } storeInfo.${name};
}) (attrNames storeInfo);
storesList = map
(name: rec {
inherit name;
slug = replaceStrings [ " " ] [ "-" ] (toLower name);
filename = "${slug}.md";
page = showStore { inherit name slug; } storeInfo.${name};
})
(attrNames storeInfo);
in
storesList
in storesList
+10 -16
View File
@@ -1,11 +1,5 @@
let
inherit (builtins)
attrNames
listToAttrs
concatStringsSep
readFile
replaceStrings
;
inherit (builtins) attrNames listToAttrs concatStringsSep readFile replaceStrings;
showSettings = import <nix/generate-settings.nix>;
showStoreDocs = import <nix/generate-store-info.nix>;
in
@@ -20,26 +14,26 @@ let
index =
let
showEntry = store: "- [${store.name}](./${store.filename})";
showEntry = store:
"- [${store.name}](./${store.filename})";
in
concatStringsSep "\n" (map showEntry storesList);
"index.md" = replaceStrings [ "@store-types@" ] [ index ] (readFile ./src/store/types/index.md.in);
"index.md" = replaceStrings
[ "@store-types@" ] [ index ]
(readFile ./src/store/types/index.md.in);
tableOfContents =
let
showEntry = store: " - [${store.name}](store/types/${store.filename})";
showEntry = store:
" - [${store.name}](store/types/${store.filename})";
in
concatStringsSep "\n" (map showEntry storesList) + "\n";
"SUMMARY.md" = tableOfContents;
storePages = listToAttrs (
map (s: {
name = s.filename;
value = s.page;
}) storesList
);
storePages = listToAttrs
(map (s: { name = s.filename; value = s.page; }) storesList);
in
storePages // { inherit "index.md" "SUMMARY.md"; }
@@ -2,8 +2,8 @@ with builtins;
with import <nix/utils.nix>;
let
showExperimentalFeature = name: doc: ''
- [`${name}`](@docroot@/development/experimental-features.md#xp-feature-${name})
'';
in
xps: indent " " (concatStrings (attrValues (mapAttrs showExperimentalFeature xps)))
showExperimentalFeature = name: doc:
''
- [`${name}`](@docroot@/development/experimental-features.md#xp-feature-${name})
'';
in xps: indent " " (concatStrings (attrValues (mapAttrs showExperimentalFeature xps)))
+1 -2
View File
@@ -2,8 +2,7 @@ with builtins;
with import <nix/utils.nix>;
let
showExperimentalFeature =
name: doc:
showExperimentalFeature = name: doc:
squash ''
## [`${name}`]{#xp-feature-${name}}
+1
View File
@@ -344,6 +344,7 @@ const redirects = {
},
"language/syntax.html": {
"scoping-rules": "scoping.html",
"string-literal": "string-literals.html",
},
"installation/installing-binary.html": {
"linux": "uninstall.html#linux",
@@ -0,0 +1,21 @@
---
synopsis: Define integer overflow in the Nix language as an error
issues: [10968]
prs: [11188]
---
Previously, integer overflow in the Nix language invoked C++ level signed overflow, which was undefined behaviour, but *usually* manifested as wrapping around on overflow.
Since prior to the public release of Lix, Lix had C++ signed overflow defined to crash the process and nobody noticed this having accidentally removed overflow from the Nix language for three months until it was caught by fiddling around.
Given the significant body of actual Nix code that has been evaluated by Lix in that time, it does not appear that nixpkgs or much of importance depends on integer overflow, so it appears safe to turn into an error.
Some other overflows were fixed:
- `builtins.fromJSON` of values greater than the maximum representable value in a signed 64-bit integer will generate an error.
- `nixConfig` in flakes will no longer accept negative values for configuration options.
Integer overflow now looks like the following:
```
$ nix eval --expr '9223372036854775807 + 1'
error: integer overflow in adding 9223372036854775807 + 1
```
+22
View File
@@ -0,0 +1,22 @@
---
synopsis: |-
The `build-hook` setting's default is less useful when using `libnixstore` as a library
prs:
- 11178
---
*This is an obscure issue that only affects usage of the `libnixstore` library outside of the Nix executable.*
As part the ongoing [rewrite of the build system](https://github.com/NixOS/nix/issues/2503) to use [Meson](https://mesonbuild.com/), we are also switching to packaging individual Nix components separately (and building them in separate derivations).
This means that when building `libnixstore` we do not know where the Nix binaries will be installed --- `libnixstore` doesn't know about downstream consumers like the Nix binaries at all.
*This is also unrelated to the _`post`_-`build-hook`*, which is often used for pushing to a cache.*
This has a small adverse affect on remote building --- the `build-remote` executable that is specified from the [`build-hook`](@docroot@/command-ref/conf-file.md#conf-build-hook) setting will not be gotten from the (presumed) installation location, but instead looked up on the `PATH`.
This means that other applications linking `libnixstore` that wish to use remote building must arrange for the `nix` command to be on the PATH (or manually overriding `build-hook`) in order for that to work.
Long term we don't envision this being a downside, because we plan to [get rid of `build-remote` and the build hook setting entirely](https://github.com/NixOS/nix/issues/1221).
There is simply no need to add a second layer of remote-procedure-calling when we want to connect to a remote builder.
The build hook protocol did in principle support custom ways of remote building, but that can also be accomplished with a custom service for the ssh or daemon/ssh-ng protocols, or with a custom [store type](@docroot@/store/types/) i.e. `Store` subclass. <!-- we normally don't mention classes, but consider that this release note is about a library use case -->
The Perl bindings no longer expose `getBinDir` either, since they libraries those bindings wrap no longer know the location of installed binaries as described above.
-10
View File
@@ -1,10 +0,0 @@
---
synopsis: Set FD_CLOEXEC on sockets created by curl
issues: []
prs: [12439]
---
Curl creates sockets without setting FD_CLOEXEC/SOCK_CLOEXEC, this can cause connections to remain open forever when using commands like `nix shell`
This change sets the FD_CLOEXEC flag using a CURLOPT_SOCKOPTFUNCTION callback.
+2
View File
@@ -28,6 +28,8 @@
- [Data Types](language/types.md)
- [String context](language/string-context.md)
- [Syntax and semantics](language/syntax.md)
- [Variables](language/variables.md)
- [String literals](language/string-literals.md)
- [Identifiers](language/identifiers.md)
- [Scoping rules](language/scope.md)
- [String interpolation](language/string-interpolation.md)
@@ -62,15 +62,6 @@ These options are for deleting old [profiles] prior to deleting unreachable [sto
This is the equivalent of invoking [`nix-env --delete-generations <period>`](@docroot@/command-ref/nix-env/delete-generations.md#generations-time) on each found profile.
See the documentation of that command for additional information about the *period* argument.
- <span id="opt-max-freed">[`--max-freed`](#opt-max-freed)</span> *bytes*
<!-- duplication from https://github.com/NixOS/nix/blob/442a2623e48357ff72c77bb11cf2cf06d94d2f90/doc/manual/source/command-ref/nix-store/gc.md?plain=1#L39-L44 -->
Keep deleting paths until at least *bytes* bytes have been deleted,
then stop. The argument *bytes* can be followed by the
multiplicative suffix `K`, `M`, `G` or `T`, denoting KiB, MiB, GiB
or TiB units.
{{#include ./opt-common.md}}
{{#include ./env-common.md}}
+1 -1
View File
@@ -62,7 +62,7 @@ These pages can be viewed offline:
Several operations, such as [`nix-env --query`](./nix-env/query.md) and [`nix-env --install`](./nix-env/install.md), take a list of *arguments* that specify the packages on which to operate.
Packages are identified based on a `name` part and a `version` part of a [symbolic derivation name](@docroot@/language/derivations.md#attr-name):
Packages are identified based on a `name` part and a `version` part of a [symbolic derivation name](@docroot@/language/derivations.md#attr-names):
- `name`: Everything up to but not including the first dash (`-`) that is *not* followed by a letter.
- `version`: The rest, excluding the separating dash.
@@ -21,9 +21,6 @@ This operation has the following options:
Use recursive instead of flat hashing mode, used when adding
directories to the store.
*paths* that refer to symlinks are not dereferenced, but added to the store
as symlinks with the same target.
{{#include ./opt-common.md}}
{{#include ../opt-common.md}}
@@ -11,9 +11,6 @@
The operation `--add` adds the specified paths to the Nix store. It
prints the resulting paths in the Nix store on standard output.
*paths* that refer to symlinks are not dereferenced, but added to the store
as symlinks with the same target.
{{#include ./opt-common.md}}
{{#include ../opt-common.md}}
@@ -104,7 +104,7 @@ symlink.
Prints a set of derivation files (`.drv`) which are supposed produce
said paths when realized. Might print nothing, for example for source paths
or paths substituted from a binary cache.
or paths subsituted from a binary cache.
- `--graph`
@@ -241,3 +241,4 @@ $ nix-store --query --roots $(which svn)
/nix/var/nix/profiles/default-82-link
/home/eelco/.local/state/nix/profiles/profile-97-link
```
+1 -1
View File
@@ -90,7 +90,7 @@ This representation is extensible and preserves the ordering:
## Self-describing values
As described in the previous section, it's crucial that schemas can be extended with new fields without breaking compatibility.
As described in the previous section, it's crucial that schemas can be extended with with new fields without breaking compatibility.
However, that should *not* mean we use the presence/absence of fields to indicate optional information *within* a version of the schema.
Instead, always include the field, and use `null` to indicate the "nothing" case.
+1 -1
View File
@@ -308,7 +308,7 @@ Creating a Cachix cache for your installer tests and adding its authorisation to
- `armv7l-linux`
- `x86_64-darwin`
- The `installer_test` job (which runs on `ubuntu-24.04` and `macos-14`) will try to install Nix with the cached installer and run a trivial Nix command.
- The `installer_test` job (which runs on `ubuntu-latest` and `macos-latest`) will try to install Nix with the cached installer and run a trivial Nix command.
### One-time setup
-8
View File
@@ -14,14 +14,6 @@ This option requires either:
* Linux running systemd, with SELinux disabled
* MacOS
> **Updating to macOS 15 Sequoia**
>
> If you recently updated to macOS 15 Sequoia and are getting
> ```console
> error: the user '_nixbld1' in the group 'nixbld' does not exist
> ```
> when running Nix commands, refer to GitHub issue [NixOS/nix#10892](https://github.com/NixOS/nix/issues/10892) for instructions to fix your installation without reinstalling.
```console
$ bash <(curl -L https://nixos.org/nix/install) --daemon
```
@@ -1,13 +1,5 @@
# Installing a Binary Distribution
> **Updating to macOS 15 Sequoia**
>
> If you recently updated to macOS 15 Sequoia and are getting
> ```console
> error: the user '_nixbld1' in the group 'nixbld' does not exist
> ```
> when running Nix commands, refer to GitHub issue [NixOS/nix#10892](https://github.com/NixOS/nix/issues/10892) for instructions to fix your installation without reinstalling.
To install the latest version Nix, run the following command:
```console
@@ -39,6 +39,8 @@
`pkgconfig` and the Boehm garbage collector, and pass the flag
`--enable-gc` to `configure`.
For `bdw-gc` <= 8.2.4 Nix needs a [small patch](https://github.com/NixOS/nix/blob/ac4d2e7b857acdfeac35ac8a592bdecee2d29838/boehmgc-traceable_allocator-public.diff) to be applied.
- The `boost` library of version 1.66.0 or higher. It can be obtained
from the official web site <https://www.boost.org/>.
+2 -12
View File
@@ -19,7 +19,7 @@ If you are on Linux with systemd:
Remove files created by Nix:
```console
sudo rm -rf /etc/nix /etc/profile.d/nix.sh /etc/tmpfiles.d/nix-daemon.conf /nix ~root/.nix-channels ~root/.nix-defexpr ~root/.nix-profile ~root/.cache/nix
sudo rm -rf /etc/nix /etc/profile.d/nix.sh /etc/tmpfiles.d/nix-daemon.conf /nix ~root/.nix-channels ~root/.nix-defexpr ~root/.nix-profile
```
Remove build users and their group:
@@ -43,14 +43,6 @@ which you may remove.
### macOS
> **Updating to macOS 15 Sequoia**
>
> If you recently updated to macOS 15 Sequoia and are getting
> ```console
> error: the user '_nixbld1' in the group 'nixbld' does not exist
> ```
> when running Nix commands, refer to GitHub issue [NixOS/nix#10892](https://github.com/NixOS/nix/issues/10892) for instructions to fix your installation without reinstalling.
1. If system-wide shell initialisation files haven't been altered since installing Nix, use the backups made by the installer:
```console
@@ -141,9 +133,7 @@ which you may remove.
diskutil list
```
If you _do_ find a "Nix Store" volume, delete it by running `diskutil apfs deleteVolume` with the store volume's `diskXsY` identifier.
If you get an error that the volume is in use by the kernel, reboot and immediately delete the volume before starting any other process.
If you _do_ find a "Nix Store" volume, delete it by running `diskutil deleteVolume` with the store volume's `diskXsY` identifier.
> **Note**
>
+2 -2
View File
@@ -113,7 +113,7 @@ It outputs an attribute set, and produces a [store derivation] as a side effect
### Optional
- [`args`]{#attr-args} ([List](@docroot@/language/types.md#type-list) of [String](@docroot@/language/types.md#type-string))
- [`args`]{#attr-args} ([List](@docroot@/language/types.md#list) of [String](@docroot@/language/types.md#type-string))
Default: `[ ]`
@@ -132,7 +132,7 @@ It outputs an attribute set, and produces a [store derivation] as a side effect
> };
> ```
- [`outputs`]{#attr-outputs} ([List](@docroot@/language/types.md#type-list) of [String](@docroot@/language/types.md#type-string))
- [`outputs`]{#attr-outputs} ([List](@docroot@/language/types.md#list) of [String](@docroot@/language/types.md#type-string))
Default: `[ "out" ]`
+3 -2
View File
@@ -16,13 +16,14 @@ An *identifier* is an [ASCII](https://en.wikipedia.org/wiki/ASCII) character seq
# Names
A name can be an [identifier](#identifier) or a [string literal](./syntax.md#string-literal).
A *name* can be written as an [identifier](#identifier) or a [string literal](./string-literals.md).
> **Syntax**
>
> *name* → *identifier* | *string*
Names are used in [attribute sets](./syntax.md#attrs-literal), [`let` bindings](./syntax.md#let-expressions), and [`inherit`](./syntax.md#inheriting attributes).
Names are used in [attribute sets](./syntax.md#attrs-literal), [`let` bindings](./syntax.md#let-expressions), and [`inherit`](./syntax.md#inheriting-attributes).
Two names are the same if they represent the same sequence of characters, regardless of whether they are written as identifiers or strings.
# Keywords
+10 -24
View File
@@ -3,7 +3,7 @@
| Name | Syntax | Associativity | Precedence |
|----------------------------------------|--------------------------------------------|---------------|------------|
| [Attribute selection] | *attrset* `.` *attrpath* \[ `or` *expr* \] | none | 1 |
| [Function application] | *func* *expr* | left | 2 |
| Function application | *func* *expr* | left | 2 |
| [Arithmetic negation][arithmetic] | `-` *number* | none | 3 |
| [Has attribute] | *attrset* `?` *attrpath* | none | 4 |
| List concatenation | *list* `++` *list* | right | 5 |
@@ -32,8 +32,8 @@
[string]: ./types.md#type-string
[path]: ./types.md#type-path
[number]: ./types.md#type-float
[list]: ./types.md#type-list
[attribute set]: ./types.md#type-attrs
[list]: ./types.md#list
[attribute set]: ./types.md#attribute-set
<!-- TODO(@rhendric, #10970): ^ rationalize number -> int/float -->
@@ -48,22 +48,6 @@ If the attribute doesn’t exist, return the *expr* after `or` if provided, othe
[Attribute selection]: #attribute-selection
## Function application
> **Syntax**
>
> *func* *expr*
Apply the callable value *func* to the argument *expr*. Note the absence of any visible operator symbol.
A callable value is either:
- a [user-defined function][function]
- a [built-in][builtins] function
- an attribute set with a [`__functor` attribute](./syntax.md#attr-__functor)
> **Warning**
>
> [List][list] items are also separated by whitespace, which means that function calls in list items must be enclosed by parentheses.
## Has attribute
> **Syntax**
@@ -75,7 +59,7 @@ The result is a [Boolean] value.
See also: [`builtins.hasAttr`](@docroot@/language/builtins.md#builtins-hasAttr)
[Boolean]: ./types.md#type-bool
[Boolean]: ./types.md#type-boolean
[Has attribute]: #has-attribute
@@ -83,8 +67,12 @@ After evaluating *attrset* and *attrpath*, the computational complexity is O(log
## Arithmetic
Numbers are type-compatible:
Pure integer operations will always return integers, whereas any operation involving at least one floating point number return a floating point number.
Numbers will retain their type unless mixed with other numeric types:
Pure integer operations will always return integers, whereas any operation involving at least one floating point number returns a floating point number.
Evaluation of the following numeric operations throws an evaluation error:
- Division by zero
- Integer overflow, that is, any operation yielding a result outside of the representable range of [Nix language integers](./syntax.md#number-literal)
See also [Comparison] and [Equality].
@@ -227,5 +215,3 @@ Equivalent to `!`*b1* `||` *b2*.
> ```
[Pipe operator]: #pipe-operators
[builtins]: ./builtins.md
[Function application]: #function-application
+23 -9
View File
@@ -1,14 +1,28 @@
# Scoping rules
Nix is [statically scoped](https://en.wikipedia.org/wiki/Scope_(computer_science)#Lexical_scope), but with multiple scopes and shadowing rules.
A *scope* in the Nix language is a dictionary keyed by [name](./identifiers.md#names), mapping each name to an expression and a *definition type*.
The definition type is either *explicit* or *implicit*.
Each entry in this dictionary is a *definition*.
* primary scope: explicitly-bound variables
* [`let`](./syntax.md#let-expressions)
* [`inherit`](./syntax.md#inheriting-attributes)
* [function](./syntax.md#functions) arguments
Explicit definitions are created by the following expressions:
- [let-expressions](syntax.md#let-expressions)
- [recursive attribute set literals](syntax.md#recursive-sets) (`rec`)
- [function literals](syntax.md#functions)
* secondary scope: implicitly-bound variables
* [`with`](./syntax.md#with-expressions)
Implicit definitions are only created by [with-expressions](./syntax.md#with-expressions).
Primary scope takes precedence over secondary scope.
See [`with`](./syntax.md#with-expressions) for a detailed example.
Every expression is *enclosed* by a scope.
The outermost expression is enclosed by the [built-in, global scope](./builtins.md), which contains only explicit definitions.
The expressions listed above *extend* their enclosing scope by adding new definitions, or replacing existing ones with the same name.
An explicit definition can replace a definition of any type; an implicit definition can only replace another implicit definition.
Each of the above expressions defines which of its subexpressions are enclosed by the extended scope.
In all other cases, the same scope that encloses an expression is the enclosing scope for its subexpressions.
The Nix language is [statically scoped](https://en.wikipedia.org/wiki/Scope_(computer_science)#Lexical_scope);
the value of a variable is determined only by the variable's enclosing scope, and not by the dynamic context in which the variable is evaluated.
> **Note**
>
> Expressions entered into the [Nix REPL](@docroot@/command-ref/new-cli/nix3-repl.md) are enclosed by a scope that can be extended by command line arguments or previous REPL commands.
> These ways of extending scope are not, strictly speaking, part of the Nix language.
@@ -8,6 +8,10 @@ Such a construct is called *interpolated string*, and the expression inside is a
[path]: ./types.md#type-path
[attribute set]: ./types.md#attribute-set
> **Syntax**
>
> *interpolation_element* → `${` *expression* `}`
## Examples
### String
@@ -150,21 +150,6 @@ These special characters are escaped as follows:
`''\` escapes any other character.
A "dollar-curly" (`${`) can be written as follows:
> **Example**
>
> ```nix
> ''
> echo ''${PATH}
> ''
> ```
>
> "echo ${PATH}\n"
> **Note**
>
> This differs from the syntax for escaping a dollar-curly within double quotes (`"\${"`). Be aware of which one is needed at a given moment.
A "double-dollar-curly" (`$${`) can be written literally.
> **Example**
+10 -171
View File
@@ -6,175 +6,7 @@ This section covers syntax and semantics of the Nix language.
### String {#string-literal}
*Strings* can be written in three ways.
The most common way is to enclose the string between double quotes, e.g., `"foo bar"`.
Strings can span multiple lines.
The results of other expressions can be included into a string by enclosing them in `${ }`, a feature known as [string interpolation].
[string interpolation]: ./string-interpolation.md
The following must be escaped to represent them within a string, by prefixing with a backslash (`\`):
- Double quote (`"`)
> **Example**
>
> ```nix
> "\""
> ```
>
> "\""
- Backslash (`\`)
> **Example**
>
> ```nix
> "\\"
> ```
>
> "\\"
- Dollar sign followed by an opening curly bracket (`${`) – "dollar-curly"
> **Example**
>
> ```nix
> "\${"
> ```
>
> "\${"
The newline, carriage return, and tab characters can be written as `\n`, `\r` and `\t`, respectively.
A "double-dollar-curly" (`$${`) can be written literally.
> **Example**
>
> ```nix
> "$${"
> ```
>
> "$\${"
String values are output on the terminal with Nix-specific escaping.
Strings written to files will contain the characters encoded by the escaping.
The second way to write string literals is as an *indented string*, which is enclosed between pairs of *double single-quotes* (`''`), like so:
```nix
''
This is the first line.
This is the second line.
This is the third line.
''
```
This kind of string literal intelligently strips indentation from
the start of each line. To be precise, it strips from each line a
number of spaces equal to the minimal indentation of the string as a
whole (disregarding the indentation of empty lines). For instance,
the first and second line are indented two spaces, while the third
line is indented four spaces. Thus, two spaces are stripped from
each line, so the resulting string is
```nix
"This is the first line.\nThis is the second line.\n This is the third line.\n"
```
> **Note**
>
> Whitespace and newline following the opening `''` is ignored if there is no non-whitespace text on the initial line.
> **Warning**
>
> Prefixed tab characters are not stripped.
>
> > **Example**
> >
> > The following indented string is prefixed with tabs:
> >
> > ''
> > all:
> > @echo hello
> > ''
> >
> > "\tall:\n\t\t@echo hello\n"
Indented strings support [string interpolation].
The following must be escaped to represent them in an indented string:
- `$` is escaped by prefixing it with two single quotes (`''`)
> **Example**
>
> ```nix
> ''
> ''$
> ''
> ```
>
> "$\n"
- `''` is escaped by prefixing it with one single quote (`'`)
> **Example**
>
> ```nix
> ''
> '''
> ''
> ```
>
> "''\n"
These special characters are escaped as follows:
- Linefeed (`\n`): `''\n`
- Carriage return (`\r`): `''\r`
- Tab (`\t`): `''\t`
`''\` escapes any other character.
A "double-dollar-curly" (`$${`) can be written literally.
> **Example**
>
> ```nix
> ''
> $${
> ''
> ```
>
> "$\${\n"
Indented strings are primarily useful in that they allow multi-line
string literals to follow the indentation of the enclosing Nix
expression, and that less escaping is typically necessary for
strings representing languages such as shell scripts and
configuration files because `''` is much less common than `"`.
Example:
```nix
stdenv.mkDerivation {
...
postInstall =
''
mkdir $out/bin $out/etc
cp foo $out/bin
echo "Hello World" > $out/etc/foo.conf
${if enableBar then "cp bar $out/bin" else ""}
'';
...
}
```
Finally, as a convenience, *URIs* as defined in appendix B of
[RFC 2396](http://www.ietf.org/rfc/rfc2396.txt) can be written *as
is*, without quotes. For instance, the string
`"http://example.org/foo.tar.bz2"` can also be written as
`http://example.org/foo.tar.bz2`.
See [String literals](string-literals.md).
### Number {#number-literal}
@@ -183,6 +15,13 @@ This section covers syntax and semantics of the Nix language.
Numbers, which can be *integers* (like `123`) or *floating point*
(like `123.43` or `.27e13`).
Integers in the Nix language are 64-bit [two's complement] signed integers, with a range of -9223372036854775808 to 9223372036854775807, inclusive.
[two's complement]: https://en.wikipedia.org/wiki/Two%27s_complement
Note that negative numeric literals are actually parsed as unary negation of positive numeric literals.
This means that the minimum integer `-9223372036854775808` cannot be written as-is as a literal, since the positive number `9223372036854775808` is one past the maximum range.
See [arithmetic] and [comparison] operators for semantics.
[arithmetic]: ./operators.md#arithmetic
@@ -253,7 +92,7 @@ Attribute sets are written enclosed in curly brackets (`{ }`).
Attribute names and attribute values are separated by an equal sign (`=`).
Each value can be an arbitrary expression, terminated by a semicolon (`;`)
An attribute name is a string without context, and is denoted by a [name] (an [identifier](./identifiers.md#identifiers) or [string literal](#string-literal)).
An attribute name is a string without context, and is denoted by a [name] (an [identifier](./identifiers.md#identifiers) or [string literal](string-literals.md)).
[name]: ./identifiers.md#names
@@ -379,7 +218,7 @@ a string), that attribute is simply not added to the set:
This will evaluate to `{}` if `foo` evaluates to `false`.
A set that has a [`__functor`]{#attr-__functor} attribute whose value is callable (i.e. is
A set that has a `__functor` attribute whose value is callable (i.e. is
itself a function or a set with a `__functor` attribute whose value is
callable) can be applied as if it were a function, with the set itself
passed in first , e.g.,
+1 -1
View File
@@ -45,7 +45,7 @@ The function [`builtins.isBool`](builtins.md#builtins-isBool) can be used to det
A _string_ in the Nix language is an immutable, finite-length sequence of bytes, along with a [string context](string-context.md).
Nix does not assume or support working natively with character encodings.
String values without string context can be expressed as [string literals](syntax.md#string-literal).
String values without string context can be expressed as [string literals](string-literals.md).
The function [`builtins.isString`](builtins.md#builtins-isString) can be used to determine if a value is a string.
### Path {#type-path}
+10
View File
@@ -0,0 +1,10 @@
# Variables
A *variable* is an [identifier](identifiers.md) used as an expression.
> **Syntax**
>
> *expression* → *identifier*
A variable must have the same name as a definition in the [scope](./scope.md) that encloses it.
The value of a variable is the value of the corresponding expression in the enclosing scope.
+1 -1
View File
@@ -82,7 +82,7 @@ where
- if `type` = `"source:" ...`:
the hash of the [Nix Archive (NAR)] serialization of the [file system object](@docroot@/store/file-system-object.md) of the store object.
the the hash of the [Nix Archive (NAR)] serialization of the [file system object](@docroot@/store/file-system-object.md) of the store object.
- if `type` = `"output:" id`:
-4
View File
@@ -75,7 +75,3 @@
(experimental) can be found by any program that follows the [XDG Base Directory Specification](https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html).
- A new command `nix store add` has been added. It replaces `nix store add-file` and `nix store add-path` which are now deprecated.
- A new option [`always-allow-substitutes`](@docroot@/command-ref/conf-file.md#conf-always-allow-substitutes) has been added.
When set to `true`, Nix will always try to substitute a derivation, even if it has the [`allowSubstitutes`]{#adv-attr-allowSubstitutes} attribute set to `false`.
+2 -2
View File
@@ -85,7 +85,7 @@
- Store object info JSON format now uses `null` rather than omitting fields [#9995](https://github.com/NixOS/nix/pull/9995)
The [store object info JSON format](@docroot@/protocols/json/store-object-info.md), used for e.g. `nix path-info`, no longer omits fields to indicate absent information, but instead includes the fields with a `null` value.
For example, `"ca": null` is used to indicate a store object that isn't content-addressed rather than omitting the `ca` field entirely.
For example, `"ca": null` is used to to indicate a store object that isn't content-addressed rather than omitting the `ca` field entirely.
This makes records of this sort more self-describing, and easier to consume programmatically.
We will follow this design principle going forward;
@@ -96,7 +96,7 @@
Nix can now warn when evaluation of a Nix expression causes a large
path to be copied to the Nix store. The threshold for this warning can
be configured using [the `warn-large-path-threshold`
setting](@docroot@/command-ref/conf-file.md#conf-warn-large-path-threshold),
setting](@docroot@/command-ref/conf-file.md#warn-large-path-threshold),
e.g. `--warn-large-path-threshold 100M` will warn about paths larger
than 100 MiB.
-15
View File
@@ -274,21 +274,6 @@
be configured using the `warn-large-path-threshold` setting,
e.g. `--warn-large-path-threshold 100M`.
- Wrap filesystem exceptions more correctly [#11378](https://github.com/NixOS/nix/pull/11378)
With the switch to `std::filesystem` in different places, Nix started to throw `std::filesystem::filesystem_error` in many places instead of its own exceptions.
This led to no longer generating error traces, for example when listing a non-existing directory.
This version catches these types of exception correctly and wraps them into Nix's own exeception type.
Author: [**@Mic92**](https://github.com/Mic92)
- `<nix/fetchurl.nix>` uses TLS verification [#11585](https://github.com/NixOS/nix/pull/11585)
Previously `<nix/fetchurl.nix>` did not do TLS verification. This was because the Nix sandbox in the past did not have access to TLS certificates, and Nix checks the hash of the fetched file anyway. However, this can expose authentication data from `netrc` and URLs to man-in-the-middle attackers. In addition, Nix now in some cases (such as when using impure derivations) does *not* check the hash. Therefore we have now enabled TLS verification. This means that downloads by `<nix/fetchurl.nix>` will now fail if you're fetching from a HTTPS server that does not have a valid certificate.
`<nix/fetchurl.nix>` is also known as the builtin derivation builder `builtin:fetchurl`. It's not to be confused with the evaluation-time function `builtins.fetchurl`, which was not affected by this issue.
# Contributors
+13 -31
View File
@@ -11,15 +11,10 @@ rec {
concatStrings = concatStringsSep "";
attrsToList =
a:
map (name: {
inherit name;
value = a.${name};
}) (builtins.attrNames a);
attrsToList = a:
map (name: { inherit name; value = a.${name}; }) (builtins.attrNames a);
replaceStringsRec =
from: to: string:
replaceStringsRec = from: to: string:
# recursively replace occurrences of `from` with `to` within `string`
# example:
# replaceStringRec "--" "-" "hello-----world"
@@ -27,18 +22,16 @@ rec {
let
replaced = replaceStrings [ from ] [ to ] string;
in
if replaced == string then string else replaceStringsRec from to replaced;
if replaced == string then string else replaceStringsRec from to replaced;
toLower = replaceStrings upperChars lowerChars;
squash = replaceStringsRec "\n\n\n" "\n\n";
trim =
string:
trim = string:
# trim trailing spaces and squash non-leading spaces
let
trimLine =
line:
trimLine = line:
let
# separate leading spaces from the rest
parts = split "(^ *)" line;
@@ -46,30 +39,19 @@ rec {
rest = elemAt parts 2;
# drop trailing spaces
body = head (split " *$" rest);
in
spaces + replaceStringsRec " " " " body;
in
concatStringsSep "\n" (map trimLine (splitLines string));
in spaces + replaceStringsRec " " " " body;
in concatStringsSep "\n" (map trimLine (splitLines string));
# FIXME: O(n^2)
unique = foldl' (acc: e: if elem e acc then acc else acc ++ [ e ]) [ ];
unique = foldl' (acc: e: if elem e acc then acc else acc ++ [ e ]) [];
nameValuePair = name: value: { inherit name value; };
filterAttrs =
pred: set:
listToAttrs (
concatMap (
name:
let
v = set.${name};
in
if pred name v then [ (nameValuePair name v) ] else [ ]
) (attrNames set)
);
filterAttrs = pred: set:
listToAttrs (concatMap (name: let v = set.${name}; in if pred name v then [(nameValuePair name v)] else []) (attrNames set));
optionalString = cond: string: if cond then string else "";
indent =
prefix: s: concatStringsSep "\n" (map (x: if x == "" then x else "${prefix}${x}") (splitLines s));
indent = prefix: s:
concatStringsSep "\n" (map (x: if x == "" then x else "${prefix}${x}") (splitLines s));
}
+181 -200
View File
@@ -1,72 +1,70 @@
{
pkgs ? import <nixpkgs> { },
lib ? pkgs.lib,
name ? "nix",
tag ? "latest",
bundleNixpkgs ? true,
channelName ? "nixpkgs",
channelURL ? "https://nixos.org/channels/nixpkgs-unstable",
extraPkgs ? [ ],
maxLayers ? 100,
nixConf ? { },
flake-registry ? null,
{ pkgs ? import <nixpkgs> { }
, lib ? pkgs.lib
, name ? "nix"
, tag ? "latest"
, bundleNixpkgs ? true
, channelName ? "nixpkgs"
, channelURL ? "https://nixos.org/channels/nixpkgs-unstable"
, extraPkgs ? []
, maxLayers ? 100
, nixConf ? {}
, flake-registry ? null
}:
let
defaultPkgs =
with pkgs;
[
nix
bashInteractive
coreutils-full
gnutar
gzip
gnugrep
which
curl
less
wget
man
cacert.out
findutils
iana-etc
git
openssh
]
++ extraPkgs;
defaultPkgs = with pkgs; [
nix
bashInteractive
coreutils-full
gnutar
gzip
gnugrep
which
curl
less
wget
man
cacert.out
findutils
iana-etc
git
openssh
] ++ extraPkgs;
users =
{
users = {
root = {
uid = 0;
shell = "${pkgs.bashInteractive}/bin/bash";
home = "/root";
gid = 0;
groups = [ "root" ];
description = "System administrator";
};
root = {
uid = 0;
shell = "${pkgs.bashInteractive}/bin/bash";
home = "/root";
gid = 0;
groups = [ "root" ];
description = "System administrator";
};
nobody = {
uid = 65534;
shell = "${pkgs.shadow}/bin/nologin";
home = "/var/empty";
gid = 65534;
groups = [ "nobody" ];
description = "Unprivileged account (don't use!)";
};
nobody = {
uid = 65534;
shell = "${pkgs.shadow}/bin/nologin";
home = "/var/empty";
gid = 65534;
groups = [ "nobody" ];
description = "Unprivileged account (don't use!)";
};
}
// lib.listToAttrs (
map (n: {
name = "nixbld${toString n}";
value = {
uid = 30000 + n;
gid = 30000;
groups = [ "nixbld" ];
description = "Nix build user ${toString n}";
};
}) (lib.lists.range 1 32)
);
} // lib.listToAttrs (
map
(
n: {
name = "nixbld${toString n}";
value = {
uid = 30000 + n;
gid = 30000;
groups = [ "nixbld" ];
description = "Nix build user ${toString n}";
};
}
)
(lib.lists.range 1 32)
);
groups = {
root.gid = 0;
@@ -76,20 +74,24 @@ let
userToPasswd = (
k:
{
uid,
gid ? 65534,
home ? "/var/empty",
description ? "",
shell ? "/bin/false",
groups ? [ ],
}:
"${k}:x:${toString uid}:${toString gid}:${description}:${home}:${shell}"
{ uid
, gid ? 65534
, home ? "/var/empty"
, description ? ""
, shell ? "/bin/false"
, groups ? [ ]
}: "${k}:x:${toString uid}:${toString gid}:${description}:${home}:${shell}"
);
passwdContents = (
lib.concatStringsSep "\n"
(lib.attrValues (lib.mapAttrs userToPasswd users))
);
passwdContents = (lib.concatStringsSep "\n" (lib.attrValues (lib.mapAttrs userToPasswd users)));
userToShadow = k: { ... }: "${k}:!:1::::::";
shadowContents = (lib.concatStringsSep "\n" (lib.attrValues (lib.mapAttrs userToShadow users)));
shadowContents = (
lib.concatStringsSep "\n"
(lib.attrValues (lib.mapAttrs userToShadow users))
);
# Map groups to members
# {
@@ -99,35 +101,42 @@ let
let
# Create a flat list of user/group mappings
mappings = (
builtins.foldl' (
acc: user:
let
groups = users.${user}.groups or [ ];
in
acc
++ map (group: {
inherit user group;
}) groups
) [ ] (lib.attrNames users)
builtins.foldl'
(
acc: user:
let
groups = users.${user}.groups or [ ];
in
acc ++ map
(group: {
inherit user group;
})
groups
)
[ ]
(lib.attrNames users)
);
in
(builtins.foldl' (
acc: v:
acc
// {
${v.group} = acc.${v.group} or [ ] ++ [ v.user ];
}
) { } mappings)
(
builtins.foldl'
(
acc: v: acc // {
${v.group} = acc.${v.group} or [ ] ++ [ v.user ];
}
)
{ }
mappings)
);
groupToGroup =
k:
{ gid }:
groupToGroup = k: { gid }:
let
members = groupMemberMap.${k} or [ ];
in
"${k}:x:${toString gid}:${lib.concatStringsSep "," members}";
groupContents = (lib.concatStringsSep "\n" (lib.attrValues (lib.mapAttrs groupToGroup groups)));
groupContents = (
lib.concatStringsSep "\n"
(lib.attrValues (lib.mapAttrs groupToGroup groups))
);
defaultNixConf = {
sandbox = "false";
@@ -135,17 +144,11 @@ let
trusted-public-keys = [ "cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=" ];
};
nixConfContents =
(lib.concatStringsSep "\n" (
lib.mapAttrsFlatten (
n: v:
let
vStr = if builtins.isList v then lib.concatStringsSep " " v else v;
in
"${n} = ${vStr}"
) (defaultNixConf // nixConf)
))
+ "\n";
nixConfContents = (lib.concatStringsSep "\n" (lib.mapAttrsFlatten (n: v:
let
vStr = if builtins.isList v then lib.concatStringsSep " " v else v;
in
"${n} = ${vStr}") (defaultNixConf // nixConf))) + "\n";
baseSystem =
let
@@ -164,29 +167,21 @@ let
manifest = pkgs.buildPackages.runCommand "manifest.nix" { } ''
cat > $out <<EOF
[
${lib.concatStringsSep "\n" (
builtins.map (
drv:
let
outputs = drv.outputsToInstall or [ "out" ];
in
''
{
${lib.concatStringsSep "\n" (
builtins.map (output: ''
${output} = { outPath = "${lib.getOutput output drv}"; };
'') outputs
)}
outputs = [ ${lib.concatStringsSep " " (builtins.map (x: "\"${x}\"") outputs)} ];
name = "${drv.name}";
outPath = "${drv}";
system = "${drv.system}";
type = "derivation";
meta = { };
}
''
) defaultPkgs
)}
${lib.concatStringsSep "\n" (builtins.map (drv: let
outputs = drv.outputsToInstall or [ "out" ];
in ''
{
${lib.concatStringsSep "\n" (builtins.map (output: ''
${output} = { outPath = "${lib.getOutput output drv}"; };
'') outputs)}
outputs = [ ${lib.concatStringsSep " " (builtins.map (x: "\"${x}\"") outputs)} ];
name = "${drv.name}";
outPath = "${drv}";
system = "${drv.system}";
type = "derivation";
meta = { };
}
'') defaultPkgs)}
]
EOF
'';
@@ -195,22 +190,16 @@ let
cp -a ${rootEnv}/* $out/
ln -s ${manifest} $out/manifest.nix
'';
flake-registry-path =
if (flake-registry == null) then
null
else if (builtins.readFileType (toString flake-registry)) == "directory" then
"${flake-registry}/flake-registry.json"
else
flake-registry;
flake-registry-path = if (flake-registry == null) then
null
else if (builtins.readFileType (toString flake-registry)) == "directory" then
"${flake-registry}/flake-registry.json"
else
flake-registry;
in
pkgs.runCommand "base-system"
{
inherit
passwdContents
groupContents
shadowContents
nixConfContents
;
inherit passwdContents groupContents shadowContents nixConfContents;
passAsFile = [
"passwdContents"
"groupContents"
@@ -219,66 +208,62 @@ let
];
allowSubstitutes = false;
preferLocalBuild = true;
}
(
''
env
set -x
mkdir -p $out/etc
} (''
env
set -x
mkdir -p $out/etc
mkdir -p $out/etc/ssl/certs
ln -s /nix/var/nix/profiles/default/etc/ssl/certs/ca-bundle.crt $out/etc/ssl/certs
mkdir -p $out/etc/ssl/certs
ln -s /nix/var/nix/profiles/default/etc/ssl/certs/ca-bundle.crt $out/etc/ssl/certs
cat $passwdContentsPath > $out/etc/passwd
echo "" >> $out/etc/passwd
cat $passwdContentsPath > $out/etc/passwd
echo "" >> $out/etc/passwd
cat $groupContentsPath > $out/etc/group
echo "" >> $out/etc/group
cat $groupContentsPath > $out/etc/group
echo "" >> $out/etc/group
cat $shadowContentsPath > $out/etc/shadow
echo "" >> $out/etc/shadow
cat $shadowContentsPath > $out/etc/shadow
echo "" >> $out/etc/shadow
mkdir -p $out/usr
ln -s /nix/var/nix/profiles/share $out/usr/
mkdir -p $out/usr
ln -s /nix/var/nix/profiles/share $out/usr/
mkdir -p $out/nix/var/nix/gcroots
mkdir -p $out/nix/var/nix/gcroots
mkdir $out/tmp
mkdir $out/tmp
mkdir -p $out/var/tmp
mkdir -p $out/var/tmp
mkdir -p $out/etc/nix
cat $nixConfContentsPath > $out/etc/nix/nix.conf
mkdir -p $out/etc/nix
cat $nixConfContentsPath > $out/etc/nix/nix.conf
mkdir -p $out/root
mkdir -p $out/nix/var/nix/profiles/per-user/root
mkdir -p $out/root
mkdir -p $out/nix/var/nix/profiles/per-user/root
ln -s ${profile} $out/nix/var/nix/profiles/default-1-link
ln -s /nix/var/nix/profiles/default-1-link $out/nix/var/nix/profiles/default
ln -s /nix/var/nix/profiles/default $out/root/.nix-profile
ln -s ${profile} $out/nix/var/nix/profiles/default-1-link
ln -s $out/nix/var/nix/profiles/default-1-link $out/nix/var/nix/profiles/default
ln -s /nix/var/nix/profiles/default $out/root/.nix-profile
ln -s ${channel} $out/nix/var/nix/profiles/per-user/root/channels-1-link
ln -s /nix/var/nix/profiles/per-user/root/channels-1-link $out/nix/var/nix/profiles/per-user/root/channels
ln -s ${channel} $out/nix/var/nix/profiles/per-user/root/channels-1-link
ln -s $out/nix/var/nix/profiles/per-user/root/channels-1-link $out/nix/var/nix/profiles/per-user/root/channels
mkdir -p $out/root/.nix-defexpr
ln -s /nix/var/nix/profiles/per-user/root/channels $out/root/.nix-defexpr/channels
echo "${channelURL} ${channelName}" > $out/root/.nix-channels
mkdir -p $out/root/.nix-defexpr
ln -s $out/nix/var/nix/profiles/per-user/root/channels $out/root/.nix-defexpr/channels
echo "${channelURL} ${channelName}" > $out/root/.nix-channels
mkdir -p $out/bin $out/usr/bin
ln -s ${pkgs.coreutils}/bin/env $out/usr/bin/env
ln -s ${pkgs.bashInteractive}/bin/bash $out/bin/sh
mkdir -p $out/bin $out/usr/bin
ln -s ${pkgs.coreutils}/bin/env $out/usr/bin/env
ln -s ${pkgs.bashInteractive}/bin/bash $out/bin/sh
''
+ (lib.optionalString (flake-registry-path != null) ''
nixCacheDir="/root/.cache/nix"
mkdir -p $out$nixCacheDir
globalFlakeRegistryPath="$nixCacheDir/flake-registry.json"
ln -s ${flake-registry-path} $out$globalFlakeRegistryPath
mkdir -p $out/nix/var/nix/gcroots/auto
rootName=$(${pkgs.nix}/bin/nix --extra-experimental-features nix-command hash file --type sha1 --base32 <(echo -n $globalFlakeRegistryPath))
ln -s $globalFlakeRegistryPath $out/nix/var/nix/gcroots/auto/$rootName
'')
);
'' + (lib.optionalString (flake-registry-path != null) ''
nixCacheDir="/root/.cache/nix"
mkdir -p $out$nixCacheDir
globalFlakeRegistryPath="$nixCacheDir/flake-registry.json"
ln -s ${flake-registry-path} $out$globalFlakeRegistryPath
mkdir -p $out/nix/var/nix/gcroots/auto
rootName=$(${pkgs.nix}/bin/nix --extra-experimental-features nix-command hash file --type sha1 --base32 <(echo -n $globalFlakeRegistryPath))
ln -s $globalFlakeRegistryPath $out/nix/var/nix/gcroots/auto/$rootName
''));
in
pkgs.dockerTools.buildLayeredImageWithNixDb {
@@ -300,19 +285,15 @@ pkgs.dockerTools.buildLayeredImageWithNixDb {
Cmd = [ "/root/.nix-profile/bin/bash" ];
Env = [
"USER=root"
"PATH=${
lib.concatStringsSep ":" [
"/root/.nix-profile/bin"
"/nix/var/nix/profiles/default/bin"
"/nix/var/nix/profiles/default/sbin"
]
}"
"MANPATH=${
lib.concatStringsSep ":" [
"/root/.nix-profile/share/man"
"/nix/var/nix/profiles/default/share/man"
]
}"
"PATH=${lib.concatStringsSep ":" [
"/root/.nix-profile/bin"
"/nix/var/nix/profiles/default/bin"
"/nix/var/nix/profiles/default/sbin"
]}"
"MANPATH=${lib.concatStringsSep ":" [
"/root/.nix-profile/share/man"
"/nix/var/nix/profiles/default/share/man"
]}"
"SSL_CERT_FILE=/nix/var/nix/profiles/default/etc/ssl/certs/ca-bundle.crt"
"GIT_SSL_CAINFO=/nix/var/nix/profiles/default/etc/ssl/certs/ca-bundle.crt"
"NIX_SSL_CERT_FILE=/nix/var/nix/profiles/default/etc/ssl/certs/ca-bundle.crt"
Generated
+3 -3
View File
@@ -80,11 +80,11 @@
},
"nixpkgs": {
"locked": {
"lastModified": 1723688146,
"narHash": "sha256-sqLwJcHYeWLOeP/XoLwAtYjr01TISlkOfz+NG82pbdg=",
"lastModified": 1721548954,
"narHash": "sha256-7cCC8+Tdq1+3OPyc3+gVo9dzUNkNIQfwSDJ2HSi2u3o=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "c3d4ac725177c030b1e289015989da2ad9d56af0",
"rev": "63d37ccd2d178d54e7fb691d7ec76000740ea24a",
"type": "github"
},
"original": {
+234 -339
View File
@@ -4,14 +4,8 @@
inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-24.05";
inputs.nixpkgs-regression.url = "github:NixOS/nixpkgs/215d4d0fd80ca5163643b03a33fde804a29cc1e2";
inputs.nixpkgs-23-11.url = "github:NixOS/nixpkgs/a62e6edd6d5e1fa0329b8653c801147986f8d446";
inputs.flake-compat = {
url = "github:edolstra/flake-compat";
flake = false;
};
inputs.libgit2 = {
url = "github:libgit2/libgit2/v1.8.1";
flake = false;
};
inputs.flake-compat = { url = "github:edolstra/flake-compat"; flake = false; };
inputs.libgit2 = { url = "github:libgit2/libgit2/v1.8.1"; flake = false; };
# dev tooling
inputs.flake-parts.url = "github:hercules-ci/flake-parts";
@@ -24,39 +18,24 @@
inputs.git-hooks-nix.inputs.flake-compat.follows = "";
inputs.git-hooks-nix.inputs.gitignore.follows = "";
outputs =
inputs@{
self,
nixpkgs,
nixpkgs-regression,
libgit2,
...
}:
outputs = inputs@{ self, nixpkgs, nixpkgs-regression, libgit2, ... }:
let
inherit (nixpkgs) lib;
officialRelease = true;
officialRelease = false;
version = lib.fileContents ./.version + versionSuffix;
versionSuffix =
if officialRelease then
""
else
"pre${
builtins.substring 0 8 (self.lastModifiedDate or self.lastModified or "19700101")
}_${self.shortRev or "dirty"}";
if officialRelease
then ""
else "pre${builtins.substring 0 8 (self.lastModifiedDate or self.lastModified or "19700101")}_${self.shortRev or "dirty"}";
linux32BitSystems = [ "i686-linux" ];
linux64BitSystems = [
"x86_64-linux"
"aarch64-linux"
];
linux64BitSystems = [ "x86_64-linux" "aarch64-linux" ];
linuxSystems = linux32BitSystems ++ linux64BitSystems;
darwinSystems = [
"x86_64-darwin"
"aarch64-darwin"
];
darwinSystems = [ "x86_64-darwin" "aarch64-darwin" ];
systems = linuxSystems ++ darwinSystems;
crossSystems = [
@@ -85,76 +64,62 @@
(Provided that the names are unique.)
See https://nixos.org/manual/nixpkgs/stable/index.html#function-library-lib.attrsets.concatMapAttrs
*/
*/
flatMapAttrs = attrs: f: lib.concatMapAttrs f attrs;
forAllSystems = lib.genAttrs systems;
forAllCrossSystems = lib.genAttrs crossSystems;
forAllStdenvs =
f:
lib.listToAttrs (
map (stdenvName: {
name = "${stdenvName}Packages";
value = f stdenvName;
}) stdenvs
);
forAllStdenvs = f:
lib.listToAttrs
(map
(stdenvName: {
name = "${stdenvName}Packages";
value = f stdenvName;
})
stdenvs);
# We don't apply flake-parts to the whole flake so that non-development attributes
# load without fetching any development inputs.
devFlake = inputs.flake-parts.lib.mkFlake { inherit inputs; } {
imports = [ ./maintainers/flake-module.nix ];
systems = lib.subtractLists crossSystems systems;
perSystem =
{ system, ... }:
{
_module.args.pkgs = nixpkgsFor.${system}.native;
};
perSystem = { system, ... }: {
_module.args.pkgs = nixpkgsFor.${system}.native;
};
};
# Memoize nixpkgs for different platforms for efficiency.
nixpkgsFor = forAllSystems (
system:
let
make-pkgs =
crossSystem: stdenv:
import nixpkgs {
localSystem = {
inherit system;
};
crossSystem =
if crossSystem == null then
null
else
{
config = crossSystem;
}
// lib.optionalAttrs (crossSystem == "x86_64-unknown-freebsd13") {
useLLVM = true;
};
overlays = [
(overlayFor (p: p.${stdenv}))
];
nixpkgsFor = forAllSystems
(system: let
make-pkgs = crossSystem: stdenv: import nixpkgs {
localSystem = {
inherit system;
};
crossSystem = if crossSystem == null then null else {
config = crossSystem;
} // lib.optionalAttrs (crossSystem == "x86_64-unknown-freebsd13") {
useLLVM = true;
};
overlays = [
(overlayFor (p: p.${stdenv}))
];
};
stdenvs = forAllStdenvs (make-pkgs null);
native = stdenvs.stdenvPackages;
in
{
in {
inherit stdenvs native;
static = native.pkgsStatic;
cross = forAllCrossSystems (crossSystem: make-pkgs crossSystem "stdenv");
}
);
});
binaryTarball =
nix: pkgs:
pkgs.callPackage ./scripts/binary-tarball.nix {
inherit nix;
};
binaryTarball = nix: pkgs: pkgs.callPackage ./scripts/binary-tarball.nix {
inherit nix;
};
overlayFor =
getStdenv: final: prev:
overlayFor = getStdenv: final: prev:
let
stdenv = getStdenv final;
in
@@ -169,12 +134,10 @@
# The dependencies are in their own scope, so that they don't have to be
# in Nixpkgs top level `pkgs` or `nixComponents`.
nixDependencies = lib.makeScope final.newScope (
import ./packaging/dependencies.nix {
inherit inputs stdenv versionSuffix;
pkgs = final;
}
);
nixDependencies = lib.makeScope final.newScope (import ./packaging/dependencies.nix {
inherit inputs stdenv versionSuffix;
pkgs = final;
});
nix = final.nixComponents.nix;
@@ -186,20 +149,13 @@
# See https://github.com/NixOS/nixpkgs/pull/214409
# Remove when fixed in this flake's nixpkgs
pre-commit =
if prev.stdenv.hostPlatform.system == "i686-linux" then
(prev.pre-commit.override (o: {
dotnet-sdk = "";
})).overridePythonAttrs
(o: {
doCheck = false;
})
else
prev.pre-commit;
if prev.stdenv.hostPlatform.system == "i686-linux"
then (prev.pre-commit.override (o: { dotnet-sdk = ""; })).overridePythonAttrs (o: { doCheck = false; })
else prev.pre-commit;
};
in
{
in {
# A Nixpkgs overlay that overrides the 'nix' and
# 'nix-perl-bindings' packages.
overlays.default = overlayFor (p: p.stdenv);
@@ -217,261 +173,200 @@
;
};
checks = forAllSystems (
system:
{
installerScriptForGHA = self.hydraJobs.installerScriptForGHA.${system};
installTests = self.hydraJobs.installTests.${system};
nixpkgsLibTests = self.hydraJobs.tests.nixpkgsLibTests.${system};
rl-next =
let
pkgs = nixpkgsFor.${system}.native;
in
pkgs.buildPackages.runCommand "test-rl-next-release-notes" { } ''
LANG=C.UTF-8 ${pkgs.changelog-d}/bin/changelog-d ${./doc/manual/rl-next} >$out
'';
repl-completion = nixpkgsFor.${system}.native.callPackage ./tests/repl-completion.nix { };
}
// (lib.optionalAttrs (builtins.elem system linux64BitSystems)) {
dockerImage = self.hydraJobs.dockerImage.${system};
}
// (lib.optionalAttrs (!(builtins.elem system linux32BitSystems))) {
# Some perl dependencies are broken on i686-linux.
# Since the support is only best-effort there, disable the perl
# bindings
checks = forAllSystems (system: {
binaryTarball = self.hydraJobs.binaryTarball.${system};
installTests = self.hydraJobs.installTests.${system};
nixpkgsLibTests = self.hydraJobs.tests.nixpkgsLibTests.${system};
rl-next =
let pkgs = nixpkgsFor.${system}.native;
in pkgs.buildPackages.runCommand "test-rl-next-release-notes" { } ''
LANG=C.UTF-8 ${pkgs.changelog-d}/bin/changelog-d ${./doc/manual/rl-next} >$out
'';
repl-completion = nixpkgsFor.${system}.native.callPackage ./tests/repl-completion.nix { };
} // (lib.optionalAttrs (builtins.elem system linux64BitSystems)) {
dockerImage = self.hydraJobs.dockerImage.${system};
} // (lib.optionalAttrs (!(builtins.elem system linux32BitSystems))) {
# Some perl dependencies are broken on i686-linux.
# Since the support is only best-effort there, disable the perl
# bindings
# Temporarily disabled because GitHub Actions OOM issues. Once
# the old build system is gone and we are back to one build
# system, we should reenable this.
#perlBindings = self.hydraJobs.perlBindings.${system};
}
# Add "passthru" tests
//
flatMapAttrs
(
{
"" = nixpkgsFor.${system}.native;
}
// lib.optionalAttrs (!nixpkgsFor.${system}.native.stdenv.hostPlatform.isDarwin) {
# TODO: enable static builds for darwin, blocked on:
# https://github.com/NixOS/nixpkgs/issues/320448
"static-" = nixpkgsFor.${system}.static;
}
# Temporarily disabled because GitHub Actions OOM issues. Once
# the old build system is gone and we are back to one build
# system, we should reenable this.
#perlBindings = self.hydraJobs.perlBindings.${system};
}
# Add "passthru" tests
// flatMapAttrs ({
"" = nixpkgsFor.${system}.native;
} // lib.optionalAttrs (! nixpkgsFor.${system}.native.stdenv.hostPlatform.isDarwin) {
# TODO: enable static builds for darwin, blocked on:
# https://github.com/NixOS/nixpkgs/issues/320448
"static-" = nixpkgsFor.${system}.static;
})
(nixpkgsPrefix: nixpkgs:
flatMapAttrs nixpkgs.nixComponents
(pkgName: pkg:
flatMapAttrs pkg.tests or {}
(testName: test: {
"${nixpkgsPrefix}${pkgName}-${testName}" = test;
})
)
(
nixpkgsPrefix: nixpkgs:
flatMapAttrs nixpkgs.nixComponents (
pkgName: pkg:
flatMapAttrs pkg.tests or { } (
testName: test: {
"${nixpkgsPrefix}${pkgName}-${testName}" = test;
}
)
)
)
// devFlake.checks.${system} or { }
)
// devFlake.checks.${system} or {}
);
packages = forAllSystems (
system:
{
# Here we put attributes that map 1:1 into packages.<system>, ie
packages = forAllSystems (system:
{ # Here we put attributes that map 1:1 into packages.<system>, ie
# for which we don't apply the full build matrix such as cross or static.
inherit (nixpkgsFor.${system}.native)
changelog-d
;
changelog-d;
default = self.packages.${system}.nix;
binaryTarball = self.hydraJobs.binaryTarball.${system};
installerScriptForGHA = self.hydraJobs.installerScriptForGHA.${system};
nix-internal-api-docs = nixpkgsFor.${system}.native.nixComponents.nix-internal-api-docs;
nix-external-api-docs = nixpkgsFor.${system}.native.nixComponents.nix-external-api-docs;
}
# We need to flatten recursive attribute sets of derivations to pass `flake check`.
//
flatMapAttrs
{
# Components we'll iterate over in the upcoming lambda
"nix" = { };
# Temporarily disabled because GitHub Actions OOM issues. Once
# the old build system is gone and we are back to one build
# system, we should reenable these.
#"nix-util" = { };
#"nix-store" = { };
#"nix-fetchers" = { };
}
(
pkgName:
{ }:
{
# These attributes go right into `packages.<system>`.
"${pkgName}" = nixpkgsFor.${system}.native.nixComponents.${pkgName};
"${pkgName}-static" = nixpkgsFor.${system}.static.nixComponents.${pkgName};
}
// flatMapAttrs (lib.genAttrs crossSystems (_: { })) (
crossSystem:
{ }:
{
# These attributes go right into `packages.<system>`.
"${pkgName}-${crossSystem}" = nixpkgsFor.${system}.cross.${crossSystem}.nixComponents.${pkgName};
}
)
// flatMapAttrs (lib.genAttrs stdenvs (_: { })) (
stdenvName:
{ }:
{
# These attributes go right into `packages.<system>`.
"${pkgName}-${stdenvName}" =
nixpkgsFor.${system}.stdenvs."${stdenvName}Packages".nixComponents.${pkgName};
}
)
)
// lib.optionalAttrs (builtins.elem system linux64BitSystems) {
dockerImage =
let
pkgs = nixpkgsFor.${system}.native;
image = import ./docker.nix {
inherit pkgs;
tag = version;
};
in
pkgs.runCommand "docker-image-tarball-${version}"
{ meta.description = "Docker image with Nix for ${system}"; }
''
mkdir -p $out/nix-support
image=$out/image.tar.gz
ln -s ${image} $image
echo "file binary-dist $image" >> $out/nix-support/hydra-build-products
'';
}
);
devShells =
let
makeShell =
pkgs: stdenv:
(pkgs.nix.override {
inherit stdenv;
forDevShell = true;
}).overrideAttrs
(
attrs:
let
modular = devFlake.getSystem stdenv.buildPlatform.system;
transformFlag =
prefix: flag:
assert builtins.isString flag;
let
rest = builtins.substring 2 (builtins.stringLength flag) flag;
in
"-D${prefix}:${rest}";
havePerl = stdenv.buildPlatform == stdenv.hostPlatform && stdenv.hostPlatform.isUnix;
ignoreCrossFile = flags: builtins.filter (flag: !(lib.strings.hasInfix "cross-file" flag)) flags;
in
{
pname = "shell-for-" + attrs.pname;
# Remove the version suffix to avoid unnecessary attempts to substitute in nix develop
version = lib.fileContents ./.version;
name = attrs.pname;
installFlags = "sysconfdir=$(out)/etc";
shellHook = ''
PATH=$prefix/bin:$PATH
unset PYTHONPATH
export MANPATH=$out/share/man:$MANPATH
# Make bash completion work.
XDG_DATA_DIRS+=:$out/share
'';
# We use this shell with the local checkout, not unpackPhase.
src = null;
env = {
# Needed for Meson to find Boost.
# https://github.com/NixOS/nixpkgs/issues/86131.
BOOST_INCLUDEDIR = "${lib.getDev pkgs.nixDependencies.boost}/include";
BOOST_LIBRARYDIR = "${lib.getLib pkgs.nixDependencies.boost}/lib";
# For `make format`, to work without installing pre-commit
_NIX_PRE_COMMIT_HOOKS_CONFIG = "${(pkgs.formats.yaml { }).generate "pre-commit-config.yaml"
modular.pre-commit.settings.rawConfig
}";
};
mesonFlags =
map (transformFlag "libutil") (ignoreCrossFile pkgs.nixComponents.nix-util.mesonFlags)
++ map (transformFlag "libstore") (ignoreCrossFile pkgs.nixComponents.nix-store.mesonFlags)
++ map (transformFlag "libfetchers") (ignoreCrossFile pkgs.nixComponents.nix-fetchers.mesonFlags)
++ lib.optionals havePerl (
map (transformFlag "perl") (ignoreCrossFile pkgs.nixComponents.nix-perl-bindings.mesonFlags)
)
++ map (transformFlag "libexpr") (ignoreCrossFile pkgs.nixComponents.nix-expr.mesonFlags)
++ map (transformFlag "libcmd") (ignoreCrossFile pkgs.nixComponents.nix-cmd.mesonFlags);
nativeBuildInputs =
attrs.nativeBuildInputs or [ ]
++ pkgs.nixComponents.nix-util.nativeBuildInputs
++ pkgs.nixComponents.nix-store.nativeBuildInputs
++ pkgs.nixComponents.nix-fetchers.nativeBuildInputs
++ lib.optionals havePerl pkgs.nixComponents.nix-perl-bindings.nativeBuildInputs
++ pkgs.nixComponents.nix-internal-api-docs.nativeBuildInputs
++ pkgs.nixComponents.nix-external-api-docs.nativeBuildInputs
++ lib.optional (
!stdenv.buildPlatform.canExecute stdenv.hostPlatform
# Hack around https://github.com/nixos/nixpkgs/commit/bf7ad8cfbfa102a90463433e2c5027573b462479
&& !(stdenv.hostPlatform.isWindows && stdenv.buildPlatform.isDarwin)
&& stdenv.hostPlatform.emulatorAvailable pkgs.buildPackages
&& lib.meta.availableOn stdenv.buildPlatform (stdenv.hostPlatform.emulator pkgs.buildPackages)
) pkgs.buildPackages.mesonEmulatorHook
++ [
pkgs.buildPackages.cmake
pkgs.buildPackages.shellcheck
pkgs.buildPackages.changelog-d
modular.pre-commit.settings.package
(pkgs.writeScriptBin "pre-commit-hooks-install" modular.pre-commit.settings.installationScript)
]
# TODO: Remove the darwin check once
# https://github.com/NixOS/nixpkgs/pull/291814 is available
++ lib.optional (stdenv.cc.isClang && !stdenv.buildPlatform.isDarwin) pkgs.buildPackages.bear
++ lib.optional (
stdenv.cc.isClang && stdenv.hostPlatform == stdenv.buildPlatform
) pkgs.buildPackages.clang-tools;
buildInputs =
attrs.buildInputs or [ ]
++ [
pkgs.gtest
pkgs.rapidcheck
]
++ lib.optional havePerl pkgs.perl;
}
);
in
forAllSystems (
system:
let
makeShells =
prefix: pkgs:
lib.mapAttrs' (k: v: lib.nameValuePair "${prefix}-${k}" v) (
forAllStdenvs (stdenvName: makeShell pkgs pkgs.${stdenvName})
);
in
(makeShells "native" nixpkgsFor.${system}.native)
// (
lib.optionalAttrs (!nixpkgsFor.${system}.native.stdenv.isDarwin) (
makeShells "static" nixpkgsFor.${system}.static
)
// (forAllCrossSystems (
crossSystem:
let
pkgs = nixpkgsFor.${system}.cross.${crossSystem};
in
makeShell pkgs pkgs.stdenv
))
)
// {
default = self.devShells.${system}.native-stdenvPackages;
// flatMapAttrs
{ # Components we'll iterate over in the upcoming lambda
"nix" = { };
# Temporarily disabled because GitHub Actions OOM issues. Once
# the old build system is gone and we are back to one build
# system, we should reenable these.
#"nix-util" = { };
#"nix-store" = { };
#"nix-fetchers" = { };
}
(pkgName: {}: {
# These attributes go right into `packages.<system>`.
"${pkgName}" = nixpkgsFor.${system}.native.nixComponents.${pkgName};
"${pkgName}-static" = nixpkgsFor.${system}.static.nixComponents.${pkgName};
}
// flatMapAttrs (lib.genAttrs crossSystems (_: { })) (crossSystem: {}: {
# These attributes go right into `packages.<system>`.
"${pkgName}-${crossSystem}" = nixpkgsFor.${system}.cross.${crossSystem}.nixComponents.${pkgName};
})
// flatMapAttrs (lib.genAttrs stdenvs (_: { })) (stdenvName: {}: {
# These attributes go right into `packages.<system>`.
"${pkgName}-${stdenvName}" = nixpkgsFor.${system}.stdenvs."${stdenvName}Packages".nixComponents.${pkgName};
})
)
// lib.optionalAttrs (builtins.elem system linux64BitSystems) {
dockerImage =
let
pkgs = nixpkgsFor.${system}.native;
image = import ./docker.nix { inherit pkgs; tag = version; };
in
pkgs.runCommand
"docker-image-tarball-${version}"
{ meta.description = "Docker image with Nix for ${system}"; }
''
mkdir -p $out/nix-support
image=$out/image.tar.gz
ln -s ${image} $image
echo "file binary-dist $image" >> $out/nix-support/hydra-build-products
'';
});
devShells = let
makeShell = pkgs: stdenv: (pkgs.nix.override { inherit stdenv; forDevShell = true; }).overrideAttrs (attrs:
let
modular = devFlake.getSystem stdenv.buildPlatform.system;
transformFlag = prefix: flag:
assert builtins.isString flag;
let
rest = builtins.substring 2 (builtins.stringLength flag) flag;
in
"-D${prefix}:${rest}";
havePerl = stdenv.buildPlatform == stdenv.hostPlatform && stdenv.hostPlatform.isUnix;
ignoreCrossFile = flags: builtins.filter (flag: !(lib.strings.hasInfix "cross-file" flag)) flags;
in {
pname = "shell-for-" + attrs.pname;
# Remove the version suffix to avoid unnecessary attempts to substitute in nix develop
version = lib.fileContents ./.version;
name = attrs.pname;
installFlags = "sysconfdir=$(out)/etc";
shellHook = ''
PATH=$prefix/bin:$PATH
unset PYTHONPATH
export MANPATH=$out/share/man:$MANPATH
# Make bash completion work.
XDG_DATA_DIRS+=:$out/share
'';
# We use this shell with the local checkout, not unpackPhase.
src = null;
env = {
# Needed for Meson to find Boost.
# https://github.com/NixOS/nixpkgs/issues/86131.
BOOST_INCLUDEDIR = "${lib.getDev pkgs.nixDependencies.boost}/include";
BOOST_LIBRARYDIR = "${lib.getLib pkgs.nixDependencies.boost}/lib";
# For `make format`, to work without installing pre-commit
_NIX_PRE_COMMIT_HOOKS_CONFIG =
"${(pkgs.formats.yaml { }).generate "pre-commit-config.yaml" modular.pre-commit.settings.rawConfig}";
};
mesonFlags =
map (transformFlag "libutil") (ignoreCrossFile pkgs.nixComponents.nix-util.mesonFlags)
++ map (transformFlag "libstore") (ignoreCrossFile pkgs.nixComponents.nix-store.mesonFlags)
++ map (transformFlag "libfetchers") (ignoreCrossFile pkgs.nixComponents.nix-fetchers.mesonFlags)
++ lib.optionals havePerl (map (transformFlag "perl") (ignoreCrossFile pkgs.nixComponents.nix-perl-bindings.mesonFlags))
++ map (transformFlag "libexpr") (ignoreCrossFile pkgs.nixComponents.nix-expr.mesonFlags)
++ map (transformFlag "libcmd") (ignoreCrossFile pkgs.nixComponents.nix-cmd.mesonFlags)
;
nativeBuildInputs = attrs.nativeBuildInputs or []
++ pkgs.nixComponents.nix-util.nativeBuildInputs
++ pkgs.nixComponents.nix-store.nativeBuildInputs
++ pkgs.nixComponents.nix-fetchers.nativeBuildInputs
++ lib.optionals havePerl pkgs.nixComponents.nix-perl-bindings.nativeBuildInputs
++ pkgs.nixComponents.nix-internal-api-docs.nativeBuildInputs
++ pkgs.nixComponents.nix-external-api-docs.nativeBuildInputs
++ lib.optional
(!stdenv.buildPlatform.canExecute stdenv.hostPlatform
# Hack around https://github.com/nixos/nixpkgs/commit/bf7ad8cfbfa102a90463433e2c5027573b462479
&& !(stdenv.hostPlatform.isWindows && stdenv.buildPlatform.isDarwin)
&& stdenv.hostPlatform.emulatorAvailable pkgs.buildPackages
&& lib.meta.availableOn stdenv.buildPlatform (stdenv.hostPlatform.emulator pkgs.buildPackages))
pkgs.buildPackages.mesonEmulatorHook
++ [
pkgs.buildPackages.cmake
pkgs.buildPackages.shellcheck
pkgs.buildPackages.changelog-d
modular.pre-commit.settings.package
(pkgs.writeScriptBin "pre-commit-hooks-install"
modular.pre-commit.settings.installationScript)
]
# TODO: Remove the darwin check once
# https://github.com/NixOS/nixpkgs/pull/291814 is available
++ lib.optional (stdenv.cc.isClang && !stdenv.buildPlatform.isDarwin) pkgs.buildPackages.bear
++ lib.optional (stdenv.cc.isClang && stdenv.hostPlatform == stdenv.buildPlatform) pkgs.buildPackages.clang-tools;
buildInputs = attrs.buildInputs or []
++ [
pkgs.gtest
pkgs.rapidcheck
]
++ lib.optional havePerl pkgs.perl
;
});
in
forAllSystems (system:
let
makeShells = prefix: pkgs:
lib.mapAttrs'
(k: v: lib.nameValuePair "${prefix}-${k}" v)
(forAllStdenvs (stdenvName: makeShell pkgs pkgs.${stdenvName}));
in
(makeShells "native" nixpkgsFor.${system}.native) //
(lib.optionalAttrs (!nixpkgsFor.${system}.native.stdenv.isDarwin)
(makeShells "static" nixpkgsFor.${system}.static) //
(forAllCrossSystems (crossSystem: let pkgs = nixpkgsFor.${system}.cross.${crossSystem}; in makeShell pkgs pkgs.stdenv))) //
{
default = self.devShells.${system}.native-stdenvPackages;
}
);
};
};
}
File diff suppressed because it is too large. Load diff
+1 -4
View File
@@ -112,7 +112,7 @@ sub copyManual {
system("xz -d < '$manualNar' | nix-store --restore $tmpDir/manual.tmp") == 0
or die "unable to unpack $manualNar\n";
rename("$tmpDir/manual.tmp/share/doc/nix/manual", "$tmpDir/manual") or die;
File::Path::remove_tree("$tmpDir/manual.tmp", {safe => 1});
system("rm -rf '$tmpDir/manual.tmp'") == 0 or die;
}
system("aws s3 sync '$tmpDir/manual' s3://$releasesBucketName/$releaseDir/manual") == 0
@@ -281,6 +281,3 @@ system("git remote update origin") == 0 or die;
system("git tag --force --sign $version $nixRev -m 'Tagging release $version'") == 0 or die;
system("git push --tags") == 0 or die;
system("git push --force-with-lease origin $nixRev:refs/heads/latest-release") == 0 or die if $isLatest;
File::Path::remove_tree($narCache, {safe => 1});
File::Path::remove_tree($tmpDir, {safe => 1});
+2 -9
View File
@@ -12,16 +12,9 @@ function _complete_nix {
elif [[ $completion == attrs ]]; then
compopt -o nospace
fi
continue
else
COMPREPLY+=("$completion")
fi
if [[ "${cur}" =~ "=" ]]; then
# drop everything up to the first =. if a = is included, bash assumes this to be
# an arg=value argument and the completion gets mangled (see #11208)
completion="${completion#*=}"
fi
COMPREPLY+=("${completion}")
done < <(NIX_GET_COMPLETIONS=$cword "${words[@]}" 2>/dev/null)
__ltrim_colon_completions "$cur"
}
+1 -3
View File
@@ -86,9 +86,7 @@ define build-library
else
ifndef HOST_DARWIN
ifndef HOST_WINDOWS
ifndef HOST_OPENBSD
$(1)_LDFLAGS += -Wl,-z,defs
endif
$(1)_LDFLAGS += -Wl,-z,defs
endif
endif
endif
-4
View File
@@ -21,10 +21,6 @@ ifdef HOST_OS
HOST_NETBSD = 1
HOST_UNIX = 1
endif
ifeq ($(patsubst openbsd%,,$(HOST_KERNEL)),)
HOST_OPENBSD = 1
HOST_UNIX = 1
endif
ifeq ($(HOST_KERNEL), linux)
HOST_LINUX = 1
HOST_UNIX = 1
+304 -364
View File
@@ -1,116 +1,112 @@
{
lib,
stdenv,
releaseTools,
autoconf-archive,
autoreconfHook,
aws-sdk-cpp,
boehmgc,
nlohmann_json,
bison,
boost,
brotli,
bzip2,
curl,
editline,
readline,
flex,
git,
gtest,
jq,
libarchive,
libcpuid,
libgit2,
libseccomp,
libsodium,
man,
darwin,
lowdown,
mdbook,
mdbook-linkcheck,
mercurial,
openssh,
openssl,
pkg-config,
rapidcheck,
sqlite,
toml11,
unixtools,
xz,
{ lib
, stdenv
, releaseTools
, autoconf-archive
, autoreconfHook
, aws-sdk-cpp
, boehmgc
, nlohmann_json
, bison
, boost
, brotli
, bzip2
, curl
, editline
, readline
, flex
, git
, gtest
, jq
, libarchive
, libcpuid
, libgit2
, libseccomp
, libsodium
, man
, lowdown
, mdbook
, mdbook-linkcheck
, mercurial
, openssh
, openssl
, pkg-config
, rapidcheck
, sqlite
, toml11
, unixtools
, xz
busybox-sandbox-shell ? null,
, busybox-sandbox-shell ? null
# Configuration Options
#:
# This probably seems like too many degrees of freedom, but it
# faithfully reflects how the underlying configure + make build system
# work. The top-level flake.nix will choose useful combinations of these
# options to CI.
# Configuration Options
#:
# This probably seems like too many degrees of freedom, but it
# faithfully reflects how the underlying configure + make build system
# work. The top-level flake.nix will choose useful combinations of these
# options to CI.
pname ? "nix",
, pname ? "nix"
versionSuffix ? "",
, versionSuffix ? ""
# Whether to build Nix. Useful to skip for tasks like testing existing pre-built versions of Nix
doBuild ? true,
# Whether to build Nix. Useful to skip for tasks like testing existing pre-built versions of Nix
, doBuild ? true
# Run the unit tests as part of the build. See `installUnitTests` for an
# alternative to this.
doCheck ? __forDefaults.canRunInstalled,
# Run the unit tests as part of the build. See `installUnitTests` for an
# alternative to this.
, doCheck ? __forDefaults.canRunInstalled
# Run the functional tests as part of the build.
doInstallCheck ? test-client != null || __forDefaults.canRunInstalled,
# Run the functional tests as part of the build.
, doInstallCheck ? test-client != null || __forDefaults.canRunInstalled
# Check test coverage of Nix. Probably want to use with at least
# one of `doCHeck` or `doInstallCheck` enabled.
withCoverageChecks ? false,
# Check test coverage of Nix. Probably want to use with with at least
# one of `doCHeck` or `doInstallCheck` enabled.
, withCoverageChecks ? false
# Whether to build the regular manual
enableManual ? __forDefaults.canRunInstalled,
# Whether to build the regular manual
, enableManual ? __forDefaults.canRunInstalled
# Whether to use garbage collection for the Nix language evaluator.
#
# If it is disabled, we just leak memory, but this is not as bad as it
# sounds so long as evaluation just takes places within short-lived
# processes. (When the process exits, the memory is reclaimed; it is
# only leaked *within* the process.)
#
# Temporarily disabled on Windows because the `GC_throw_bad_alloc`
# symbol is missing during linking.
#
# Disabled on OpenBSD because of missing `_data_start` symbol while linking
enableGC ? !stdenv.hostPlatform.isWindows && !stdenv.hostPlatform.isOpenBSD,
# Whether to use garbage collection for the Nix language evaluator.
#
# If it is disabled, we just leak memory, but this is not as bad as it
# sounds so long as evaluation just takes places within short-lived
# processes. (When the process exits, the memory is reclaimed; it is
# only leaked *within* the process.)
#
# Temporarily disabled on Windows because the `GC_throw_bad_alloc`
# symbol is missing during linking.
, enableGC ? !stdenv.hostPlatform.isWindows
# Whether to enable Markdown rendering in the Nix binary.
enableMarkdown ? !stdenv.hostPlatform.isWindows,
# Whether to enable Markdown rendering in the Nix binary.
, enableMarkdown ? !stdenv.hostPlatform.isWindows
# Which interactive line editor library to use for Nix's repl.
#
# Currently supported choices are:
#
# - editline (default)
# - readline
readlineFlavor ? if stdenv.hostPlatform.isWindows then "readline" else "editline",
# Which interactive line editor library to use for Nix's repl.
#
# Currently supported choices are:
#
# - editline (default)
# - readline
, readlineFlavor ? if stdenv.hostPlatform.isWindows then "readline" else "editline"
# Whether to install unit tests. This is useful when cross compiling
# since we cannot run them natively during the build, but can do so
# later.
installUnitTests ? doBuild && !__forDefaults.canExecuteHost,
# Whether to install unit tests. This is useful when cross compiling
# since we cannot run them natively during the build, but can do so
# later.
, installUnitTests ? doBuild && !__forDefaults.canExecuteHost
# For running the functional tests against a pre-built Nix. Probably
# want to use in conjunction with `doBuild = false;`.
test-daemon ? null,
test-client ? null,
# For running the functional tests against a pre-built Nix. Probably
# want to use in conjunction with `doBuild = false;`.
, test-daemon ? null
, test-client ? null
# Avoid setting things that would interfere with a functioning devShell
forDevShell ? false,
# Avoid setting things that would interfere with a functioning devShell
, forDevShell ? false
# Not a real argument, just the only way to approximate let-binding some
# stuff for argument defaults.
__forDefaults ? {
# Not a real argument, just the only way to approximate let-binding some
# stuff for argument defaults.
, __forDefaults ? {
canExecuteHost = stdenv.buildPlatform.canExecute stdenv.hostPlatform;
canRunInstalled = doBuild && __forDefaults.canExecuteHost;
},
}
}:
let
@@ -126,304 +122,248 @@ let
};
mkDerivation =
if withCoverageChecks then
if withCoverageChecks
then
# TODO support `finalAttrs` args function in
# `releaseTools.coverageAnalysis`.
argsFun:
releaseTools.coverageAnalysis (
let
args = argsFun args;
in
args
)
else
stdenv.mkDerivation;
releaseTools.coverageAnalysis (let args = argsFun args; in args)
else stdenv.mkDerivation;
in
mkDerivation (
finalAttrs:
let
mkDerivation (finalAttrs: let
inherit (finalAttrs)
doCheck
doInstallCheck
;
inherit (finalAttrs)
doCheck
doInstallCheck
;
doBuild = !finalAttrs.dontBuild;
doBuild = !finalAttrs.dontBuild;
# Either running the unit tests during the build, or installing them
# to be run later, requiresthe unit tests to be built.
buildUnitTests = doCheck || installUnitTests;
# Either running the unit tests during the build, or installing them
# to be run later, requiresthe unit tests to be built.
buildUnitTests = doCheck || installUnitTests;
in
{
inherit pname version;
in {
inherit pname version;
src =
let
baseFiles = fileset.fileFilter (f: f.name != ".gitignore") ./.;
in
src =
let
baseFiles = fileset.fileFilter (f: f.name != ".gitignore") ./.;
in
fileset.toSource {
root = ./.;
fileset = fileset.intersection baseFiles (
fileset.unions (
[
# For configure
./.version
./configure.ac
./m4
# TODO: do we really need README.md? It doesn't seem used in the build.
./README.md
# This could be put behind a conditional
./maintainers/local.mk
# For make, regardless of what we are building
./local.mk
./Makefile
./Makefile.config.in
./mk
(fileset.fileFilter (f: lib.strings.hasPrefix "nix-profile" f.name) ./scripts)
]
++ lib.optionals doBuild [
./doc
./misc
./precompiled-headers.h
(fileset.difference ./src ./src/perl)
./COPYING
./scripts/local.mk
]
++ lib.optionals enableManual [
./doc/manual
]
++ lib.optionals buildUnitTests [
./tests/unit
]
++ lib.optionals doInstallCheck [
./tests/functional
]
)
);
fileset = fileset.intersection baseFiles (fileset.unions ([
# For configure
./.version
./configure.ac
./m4
# TODO: do we really need README.md? It doesn't seem used in the build.
./README.md
# This could be put behind a conditional
./maintainers/local.mk
# For make, regardless of what we are building
./local.mk
./Makefile
./Makefile.config.in
./mk
(fileset.fileFilter (f: lib.strings.hasPrefix "nix-profile" f.name) ./scripts)
] ++ lib.optionals doBuild [
./doc
./misc
./precompiled-headers.h
(fileset.difference ./src ./src/perl)
./COPYING
./scripts/local.mk
] ++ lib.optionals enableManual [
./doc/manual
] ++ lib.optionals buildUnitTests [
./tests/unit
] ++ lib.optionals doInstallCheck [
./tests/functional
]));
};
VERSION_SUFFIX = versionSuffix;
VERSION_SUFFIX = versionSuffix;
outputs =
[ "out" ]
++ lib.optional doBuild "dev"
# If we are doing just build or just docs, the one thing will use
# "out". We only need additional outputs if we are doing both.
++ lib.optional (doBuild && enableManual) "doc"
++ lib.optional installUnitTests "check"
++ lib.optional doCheck "testresults";
outputs = [ "out" ]
++ lib.optional doBuild "dev"
# If we are doing just build or just docs, the one thing will use
# "out". We only need additional outputs if we are doing both.
++ lib.optional (doBuild && enableManual) "doc"
++ lib.optional installUnitTests "check"
++ lib.optional doCheck "testresults"
;
nativeBuildInputs =
[
autoconf-archive
autoreconfHook
pkg-config
]
++ lib.optionals doBuild [
bison
flex
]
++ lib.optionals enableManual [
(lib.getBin lowdown)
mdbook
mdbook-linkcheck
]
++ lib.optionals doInstallCheck [
git
mercurial
openssh
]
++ lib.optionals (doInstallCheck || enableManual) [
jq # Also for custom mdBook preprocessor.
]
++ lib.optionals enableManual [
man
]
++ lib.optional stdenv.hostPlatform.isStatic unixtools.hexdump;
nativeBuildInputs = [
autoconf-archive
autoreconfHook
pkg-config
] ++ lib.optionals doBuild [
bison
flex
] ++ lib.optionals enableManual [
(lib.getBin lowdown)
mdbook
mdbook-linkcheck
] ++ lib.optionals doInstallCheck [
git
mercurial
openssh
man # for testing `nix-* --help`
] ++ lib.optionals (doInstallCheck || enableManual) [
jq # Also for custom mdBook preprocessor.
] ++ lib.optional stdenv.hostPlatform.isStatic unixtools.hexdump
;
buildInputs = lib.optionals doBuild (
[
brotli
bzip2
curl
libarchive
libgit2
libsodium
openssl
sqlite
toml11
xz
({ inherit readline editline; }.${readlineFlavor})
]
++ lib.optionals enableMarkdown [
lowdown
]
++ lib.optionals buildUnitTests [
gtest
rapidcheck
]
++ lib.optional stdenv.isLinux libseccomp
++ lib.optional stdenv.hostPlatform.isDarwin darwin.apple_sdk.libs.sandbox
++ lib.optional stdenv.hostPlatform.isx86_64 libcpuid
# There have been issues building these dependencies
++ lib.optional (
stdenv.hostPlatform == stdenv.buildPlatform && (stdenv.isLinux || stdenv.isDarwin)
) aws-sdk-cpp
);
buildInputs = lib.optionals doBuild (
[
brotli
bzip2
curl
libarchive
libgit2
libsodium
openssl
sqlite
toml11
xz
({ inherit readline editline; }.${readlineFlavor})
] ++ lib.optionals enableMarkdown [
lowdown
] ++ lib.optionals buildUnitTests [
gtest
rapidcheck
] ++ lib.optional stdenv.isLinux libseccomp
++ lib.optional stdenv.hostPlatform.isx86_64 libcpuid
# There have been issues building these dependencies
++ lib.optional (stdenv.hostPlatform == stdenv.buildPlatform && (stdenv.isLinux || stdenv.isDarwin))
aws-sdk-cpp
);
propagatedBuildInputs = lib.optionals doBuild (
[
boost
nlohmann_json
]
++ lib.optional enableGC boehmgc
);
propagatedBuildInputs = lib.optionals doBuild ([
boost
nlohmann_json
] ++ lib.optional enableGC boehmgc
);
dontBuild = !attrs.doBuild;
doCheck = attrs.doCheck;
dontBuild = !attrs.doBuild;
doCheck = attrs.doCheck;
configureFlags =
[
(lib.enableFeature doBuild "build")
(lib.enableFeature buildUnitTests "unit-tests")
(lib.enableFeature doInstallCheck "functional-tests")
(lib.enableFeature enableManual "doc-gen")
(lib.enableFeature enableGC "gc")
(lib.enableFeature enableMarkdown "markdown")
(lib.enableFeature installUnitTests "install-unit-tests")
(lib.withFeatureAs true "readline-flavor" readlineFlavor)
]
++ lib.optionals (!forDevShell) [
"--sysconfdir=/etc"
]
++ lib.optionals installUnitTests [
"--with-check-bin-dir=${builtins.placeholder "check"}/bin"
"--with-check-lib-dir=${builtins.placeholder "check"}/lib"
]
++ lib.optionals (doBuild) [
"--with-boost=${boost}/lib"
]
++ lib.optionals (doBuild && stdenv.isLinux) [
"--with-sandbox-shell=${busybox-sandbox-shell}/bin/busybox"
]
++ lib.optional (
doBuild && stdenv.isLinux && !(stdenv.hostPlatform.isStatic && stdenv.system == "aarch64-linux")
) "LDFLAGS=-fuse-ld=gold"
++ lib.optional (doBuild && stdenv.hostPlatform.isStatic) "--enable-embedded-sandbox-shell";
configureFlags = [
(lib.enableFeature doBuild "build")
(lib.enableFeature buildUnitTests "unit-tests")
(lib.enableFeature doInstallCheck "functional-tests")
(lib.enableFeature enableManual "doc-gen")
(lib.enableFeature enableGC "gc")
(lib.enableFeature enableMarkdown "markdown")
(lib.enableFeature installUnitTests "install-unit-tests")
(lib.withFeatureAs true "readline-flavor" readlineFlavor)
] ++ lib.optionals (!forDevShell) [
"--sysconfdir=/etc"
] ++ lib.optionals installUnitTests [
"--with-check-bin-dir=${builtins.placeholder "check"}/bin"
"--with-check-lib-dir=${builtins.placeholder "check"}/lib"
] ++ lib.optionals (doBuild) [
"--with-boost=${boost}/lib"
] ++ lib.optionals (doBuild && stdenv.isLinux) [
"--with-sandbox-shell=${busybox-sandbox-shell}/bin/busybox"
] ++ lib.optional (doBuild && stdenv.isLinux && !(stdenv.hostPlatform.isStatic && stdenv.system == "aarch64-linux"))
"LDFLAGS=-fuse-ld=gold"
++ lib.optional (doBuild && stdenv.hostPlatform.isStatic) "--enable-embedded-sandbox-shell"
;
enableParallelBuilding = true;
enableParallelBuilding = true;
makeFlags = "profiledir=$(out)/etc/profile.d PRECOMPILE_HEADERS=1";
makeFlags = "profiledir=$(out)/etc/profile.d PRECOMPILE_HEADERS=1";
preCheck = ''
mkdir $testresults
preCheck = ''
mkdir $testresults
'';
installTargets = lib.optional doBuild "install";
installFlags = "sysconfdir=$(out)/etc";
# In this case we are probably just running tests, and so there isn't
# anything to install, we just make an empty directory to signify tests
# succeeded.
installPhase = if finalAttrs.installTargets != [] then null else ''
mkdir -p $out
'';
postInstall = lib.optionalString doBuild (
lib.optionalString stdenv.hostPlatform.isStatic ''
mkdir -p $out/nix-support
echo "file binary-dist $out/bin/nix" >> $out/nix-support/hydra-build-products
''
) + lib.optionalString enableManual ''
mkdir -p ''${!outputDoc}/nix-support
echo "doc manual ''${!outputDoc}/share/doc/nix/manual" >> ''${!outputDoc}/nix-support/hydra-build-products
'';
# So the check output gets links for DLLs in the out output.
preFixup = lib.optionalString (stdenv.hostPlatform.isWindows && builtins.elem "check" finalAttrs.outputs) ''
ln -s "$check/lib/"*.dll "$check/bin"
ln -s "$out/bin/"*.dll "$check/bin"
'';
doInstallCheck = attrs.doInstallCheck;
installCheckFlags = "sysconfdir=$(out)/etc";
# Work around buggy detection in stdenv.
installCheckTarget = "installcheck";
# Work around weird bug where it doesn't think there is a Makefile.
installCheckPhase = if (!doBuild && doInstallCheck) then ''
runHook preInstallCheck
mkdir -p src/nix-channel
make installcheck -j$NIX_BUILD_CORES -l$NIX_BUILD_CORES
'' else null;
# Needed for tests if we are not doing a build, but testing existing
# built Nix.
preInstallCheck =
lib.optionalString (! doBuild) ''
mkdir -p src/nix-channel
''
# See https://github.com/NixOS/nix/issues/2523
# Occurs often in tests since https://github.com/NixOS/nix/pull/9900
+ lib.optionalString stdenv.hostPlatform.isDarwin ''
export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES
'';
installTargets = lib.optional doBuild "install";
separateDebugInfo = !stdenv.hostPlatform.isStatic;
installFlags = "sysconfdir=$(out)/etc";
# TODO Always true after https://github.com/NixOS/nixpkgs/issues/318564
strictDeps = !withCoverageChecks;
# In this case we are probably just running tests, and so there isn't
# anything to install, we just make an empty directory to signify tests
# succeeded.
installPhase =
if finalAttrs.installTargets != [ ] then
null
else
''
mkdir -p $out
'';
hardeningDisable = lib.optional stdenv.hostPlatform.isStatic "pie";
postInstall =
lib.optionalString doBuild (
lib.optionalString stdenv.hostPlatform.isStatic ''
mkdir -p $out/nix-support
echo "file binary-dist $out/bin/nix" >> $out/nix-support/hydra-build-products
''
)
+ lib.optionalString enableManual ''
mkdir -p ''${!outputDoc}/nix-support
echo "doc manual ''${!outputDoc}/share/doc/nix/manual" >> ''${!outputDoc}/nix-support/hydra-build-products
'';
meta = {
platforms = lib.platforms.unix ++ lib.platforms.windows;
mainProgram = "nix";
broken = !(lib.all (a: a) [
# We cannot run or install unit tests if we don't build them or
# Nix proper (which they depend on).
(installUnitTests -> doBuild)
(doCheck -> doBuild)
# The build process for the manual currently requires extracting
# data from the Nix executable we are trying to document.
(enableManual -> doBuild)
]);
};
# So the check output gets links for DLLs in the out output.
preFixup =
lib.optionalString (stdenv.hostPlatform.isWindows && builtins.elem "check" finalAttrs.outputs)
''
ln -s "$check/lib/"*.dll "$check/bin"
ln -s "$out/bin/"*.dll "$check/bin"
'';
} // lib.optionalAttrs withCoverageChecks {
lcovFilter = [ "*/boost/*" "*-tab.*" ];
doInstallCheck = attrs.doInstallCheck;
hardeningDisable = ["fortify"];
installCheckFlags = "sysconfdir=$(out)/etc";
# Work around buggy detection in stdenv.
installCheckTarget = "installcheck";
NIX_CFLAGS_COMPILE = "-DCOVERAGE=1";
# Work around weird bug where it doesn't think there is a Makefile.
installCheckPhase =
if (!doBuild && doInstallCheck) then
''
runHook preInstallCheck
mkdir -p src/nix-channel
make installcheck -j$NIX_BUILD_CORES -l$NIX_BUILD_CORES
''
else
null;
# Needed for tests if we are not doing a build, but testing existing
# built Nix.
preInstallCheck =
lib.optionalString (!doBuild) ''
mkdir -p src/nix-channel
''
# See https://github.com/NixOS/nix/issues/2523
# Occurs often in tests since https://github.com/NixOS/nix/pull/9900
+ lib.optionalString stdenv.hostPlatform.isDarwin ''
export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES
'';
separateDebugInfo = !stdenv.hostPlatform.isStatic;
# TODO Always true after https://github.com/NixOS/nixpkgs/issues/318564
strictDeps = !withCoverageChecks;
hardeningDisable = lib.optional stdenv.hostPlatform.isStatic "pie";
meta = {
platforms = lib.platforms.unix ++ lib.platforms.windows;
mainProgram = "nix";
broken =
!(lib.all (a: a) [
# We cannot run or install unit tests if we don't build them or
# Nix proper (which they depend on).
(installUnitTests -> doBuild)
(doCheck -> doBuild)
# The build process for the manual currently requires extracting
# data from the Nix executable we are trying to document.
(enableManual -> doBuild)
]);
};
}
// lib.optionalAttrs withCoverageChecks {
lcovFilter = [
"*/boost/*"
"*-tab.*"
];
hardeningDisable = [ "fortify" ];
NIX_CFLAGS_COMPILE = "-DCOVERAGE=1";
dontInstall = false;
}
// lib.optionalAttrs (test-daemon != null) {
NIX_DAEMON_PACKAGE = test-daemon;
}
// lib.optionalAttrs (test-client != null) {
NIX_CLIENT_PACKAGE = test-client;
}
)
dontInstall = false;
} // lib.optionalAttrs (test-daemon != null) {
NIX_DAEMON_PACKAGE = test-daemon;
} // lib.optionalAttrs (test-client != null) {
NIX_CLIENT_PACKAGE = test-client;
})
+65 -76
View File
@@ -20,7 +20,9 @@ let
root = ../.;
stdenv = if prevStdenv.isDarwin && prevStdenv.isx86_64 then darwinStdenv else prevStdenv;
stdenv = if prevStdenv.isDarwin && prevStdenv.isx86_64
then darwinStdenv
else prevStdenv;
# Fix the following error with the default x86_64-darwin SDK:
#
@@ -37,8 +39,7 @@ let
# Indirection for Nixpkgs to override when package.nix files are vendored
filesetToSource = lib.fileset.toSource;
localSourceLayer =
finalAttrs: prevAttrs:
localSourceLayer = finalAttrs: prevAttrs:
let
workDirPath =
# Ideally we'd pick finalAttrs.workDir, but for now `mkDerivation` has
@@ -47,13 +48,8 @@ let
prevAttrs.workDir;
workDirSubpath = lib.path.removePrefix root workDirPath;
sources =
assert prevAttrs.fileset._type == "fileset";
prevAttrs.fileset;
src = lib.fileset.toSource {
fileset = sources;
inherit root;
};
sources = assert prevAttrs.fileset._type == "fileset"; prevAttrs.fileset;
src = lib.fileset.toSource { fileset = sources; inherit root; };
in
{
@@ -67,35 +63,30 @@ let
# Work around weird `--as-needed` linker behavior with BSD, see
# https://github.com/mesonbuild/meson/issues/3593
bsdNoLinkAsNeeded =
finalAttrs: prevAttrs:
bsdNoLinkAsNeeded = finalAttrs: prevAttrs:
lib.optionalAttrs stdenv.hostPlatform.isBSD {
mesonFlags = [ (lib.mesonBool "b_asneeded" false) ] ++ prevAttrs.mesonFlags or [ ];
mesonFlags = [ (lib.mesonBool "b_asneeded" false) ] ++ prevAttrs.mesonFlags or [];
};
miscGoodPractice = finalAttrs: prevAttrs: {
strictDeps = prevAttrs.strictDeps or true;
enableParallelBuilding = true;
};
miscGoodPractice = finalAttrs: prevAttrs:
{
strictDeps = prevAttrs.strictDeps or true;
enableParallelBuilding = true;
};
in
scope: {
inherit stdenv versionSuffix;
version = lib.fileContents ../.version + versionSuffix;
aws-sdk-cpp =
(pkgs.aws-sdk-cpp.override {
apis = [
"s3"
"transfer"
];
customMemoryManagement = false;
}).overrideAttrs
{
# only a stripped down version is built, which takes a lot less resources
# to build, so we don't need a "big-parallel" machine.
requiredSystemFeatures = [ ];
};
aws-sdk-cpp = (pkgs.aws-sdk-cpp.override {
apis = [ "s3" "transfer" ];
customMemoryManagement = false;
}).overrideAttrs {
# only a stripped down version is built, which takes a lot less resources
# to build, so we don't need a "big-parallel" machine.
requiredSystemFeatures = [ ];
};
libseccomp = pkgs.libseccomp.overrideAttrs (_: rec {
version = "2.5.5";
@@ -110,50 +101,48 @@ scope: {
};
# TODO Hack until https://github.com/NixOS/nixpkgs/issues/45462 is fixed.
boost =
(pkgs.boost.override {
extraB2Args = [
"--with-container"
"--with-context"
"--with-coroutine"
];
}).overrideAttrs
(old: {
# Need to remove `--with-*` to use `--with-libraries=...`
buildPhase = lib.replaceStrings [ "--without-python" ] [ "" ] old.buildPhase;
installPhase = lib.replaceStrings [ "--without-python" ] [ "" ] old.installPhase;
});
boost = (pkgs.boost.override {
extraB2Args = [
"--with-container"
"--with-context"
"--with-coroutine"
];
}).overrideAttrs (old: {
# Need to remove `--with-*` to use `--with-libraries=...`
buildPhase = lib.replaceStrings [ "--without-python" ] [ "" ] old.buildPhase;
installPhase = lib.replaceStrings [ "--without-python" ] [ "" ] old.installPhase;
});
libgit2 = pkgs.libgit2.overrideAttrs (attrs: {
src = inputs.libgit2;
version = inputs.libgit2.lastModifiedDate;
cmakeFlags = attrs.cmakeFlags or [ ] ++ [ "-DUSE_SSH=exec" ];
cmakeFlags = attrs.cmakeFlags or []
++ [ "-DUSE_SSH=exec" ];
});
busybox-sandbox-shell =
pkgs.busybox-sandbox-shell or (pkgs.busybox.override {
useMusl = true;
enableStatic = true;
enableMinimal = true;
extraConfig = ''
CONFIG_FEATURE_FANCY_ECHO y
CONFIG_FEATURE_SH_MATH y
CONFIG_FEATURE_SH_MATH_64 y
busybox-sandbox-shell = pkgs.busybox-sandbox-shell or (pkgs.busybox.override {
useMusl = true;
enableStatic = true;
enableMinimal = true;
extraConfig = ''
CONFIG_FEATURE_FANCY_ECHO y
CONFIG_FEATURE_SH_MATH y
CONFIG_FEATURE_SH_MATH_64 y
CONFIG_ASH y
CONFIG_ASH_OPTIMIZE_FOR_SIZE y
CONFIG_ASH y
CONFIG_ASH_OPTIMIZE_FOR_SIZE y
CONFIG_ASH_ALIAS y
CONFIG_ASH_BASH_COMPAT y
CONFIG_ASH_CMDCMD y
CONFIG_ASH_ECHO y
CONFIG_ASH_GETOPTS y
CONFIG_ASH_INTERNAL_GLOB y
CONFIG_ASH_JOB_CONTROL y
CONFIG_ASH_PRINTF y
CONFIG_ASH_TEST y
'';
});
CONFIG_ASH_ALIAS y
CONFIG_ASH_BASH_COMPAT y
CONFIG_ASH_CMDCMD y
CONFIG_ASH_ECHO y
CONFIG_ASH_GETOPTS y
CONFIG_ASH_INTERNAL_GLOB y
CONFIG_ASH_JOB_CONTROL y
CONFIG_ASH_PRINTF y
CONFIG_ASH_TEST y
'';
});
# TODO change in Nixpkgs, Windows works fine. First commit of
# https://github.com/NixOS/nixpkgs/pull/322977 backported will fix.
@@ -163,14 +152,14 @@ scope: {
inherit resolvePath filesetToSource;
mkMesonDerivation =
f:
let
exts = [
miscGoodPractice
bsdNoLinkAsNeeded
localSourceLayer
];
in
stdenv.mkDerivation (lib.extends (lib.foldr lib.composeExtensions (_: _: { }) exts) f);
mkMesonDerivation = f: let
exts = [
miscGoodPractice
bsdNoLinkAsNeeded
localSourceLayer
];
in stdenv.mkDerivation
(lib.extends
(lib.foldr lib.composeExtensions (_: _: {}) exts)
f);
}
+72 -94
View File
@@ -1,31 +1,28 @@
{
inputs,
binaryTarball,
forAllCrossSystems,
forAllSystems,
lib,
linux64BitSystems,
nixpkgsFor,
self,
{ inputs
, binaryTarball
, forAllCrossSystems
, forAllSystems
, lib
, linux64BitSystems
, nixpkgsFor
, self
}:
let
inherit (inputs) nixpkgs nixpkgs-regression;
installScriptFor =
tarballs:
installScriptFor = tarballs:
nixpkgsFor.x86_64-linux.native.callPackage ../scripts/installer.nix {
inherit tarballs;
};
testNixVersions =
pkgs: client: daemon:
testNixVersions = pkgs: client: daemon:
pkgs.callPackage ../package.nix {
pname =
"nix-tests"
+ lib.optionalString (
lib.versionAtLeast daemon.version "2.4pre20211005"
&& lib.versionAtLeast client.version "2.4pre20211005"
) "-${client.version}-against-${daemon.version}";
+ lib.optionalString
(lib.versionAtLeast daemon.version "2.4pre20211005" &&
lib.versionAtLeast client.version "2.4pre20211005")
"-${client.version}-against-${daemon.version}";
test-client = client;
test-daemon = daemon;
@@ -62,35 +59,27 @@ let
in
{
# Binary package for various platforms.
build = forAllPackages (
pkgName: forAllSystems (system: nixpkgsFor.${system}.native.nixComponents.${pkgName})
);
build = forAllPackages (pkgName:
forAllSystems (system: nixpkgsFor.${system}.native.nixComponents.${pkgName}));
shellInputs = forAllSystems (system: self.devShells.${system}.default.inputDerivation);
buildStatic = forAllPackages (
pkgName:
lib.genAttrs linux64BitSystems (system: nixpkgsFor.${system}.static.nixComponents.${pkgName})
);
buildStatic = forAllPackages (pkgName:
lib.genAttrs linux64BitSystems (system: nixpkgsFor.${system}.static.nixComponents.${pkgName}));
buildCross = forAllPackages (
pkgName:
forAllCrossSystems (
crossSystem:
lib.genAttrs [ "x86_64-linux" ] (
system: nixpkgsFor.${system}.cross.${crossSystem}.nixComponents.${pkgName}
)
)
);
buildCross = forAllPackages (pkgName:
forAllCrossSystems (crossSystem:
lib.genAttrs [ "x86_64-linux" ] (system: nixpkgsFor.${system}.cross.${crossSystem}.nixComponents.${pkgName})));
buildNoGc = forAllSystems (system: self.packages.${system}.nix.override { enableGC = false; });
buildNoGc = forAllSystems (system:
self.packages.${system}.nix.override { enableGC = false; }
);
buildNoTests = forAllSystems (system: nixpkgsFor.${system}.native.nix_noTests);
# Toggles some settings for better coverage. Windows needs these
# library combinations, and Debian build Nix with GNU readline too.
buildReadlineNoMarkdown = forAllSystems (
system:
buildReadlineNoMarkdown = forAllSystems (system:
self.packages.${system}.nix.override {
enableMarkdown = false;
readlineFlavor = "readline";
@@ -103,18 +92,13 @@ in
# Binary tarball for various platforms, containing a Nix store
# with the closure of 'nix' package, and the second half of
# the installation script.
binaryTarball = forAllSystems (
system: binaryTarball nixpkgsFor.${system}.native.nix nixpkgsFor.${system}.native
);
binaryTarball = forAllSystems (system: binaryTarball nixpkgsFor.${system}.native.nix nixpkgsFor.${system}.native);
binaryTarballCross = lib.genAttrs [ "x86_64-linux" ] (
system:
forAllCrossSystems (
crossSystem:
binaryTarball nixpkgsFor.${system}.cross.${crossSystem}.nix
nixpkgsFor.${system}.cross.${crossSystem}
)
);
binaryTarballCross = lib.genAttrs [ "x86_64-linux" ] (system:
forAllCrossSystems (crossSystem:
binaryTarball
nixpkgsFor.${system}.cross.${crossSystem}.nix
nixpkgsFor.${system}.cross.${crossSystem}));
# The first half of the installation script. This is uploaded
# to https://nixos.org/nix/install. It downloads the binary
@@ -132,13 +116,15 @@ in
self.hydraJobs.binaryTarballCross."x86_64-linux"."armv7l-unknown-linux-gnueabihf"
self.hydraJobs.binaryTarballCross."x86_64-linux"."riscv64-unknown-linux-gnu"
];
installerScriptForGHA = forAllSystems (
system:
nixpkgsFor.${system}.native.callPackage ../scripts/installer.nix {
tarballs = [ self.hydraJobs.binaryTarball.${system} ];
}
);
installerScriptForGHA = installScriptFor [
# Native
self.hydraJobs.binaryTarball."x86_64-linux"
self.hydraJobs.binaryTarball."aarch64-darwin"
# Cross
self.hydraJobs.binaryTarballCross."x86_64-linux"."armv6l-unknown-linux-gnueabihf"
self.hydraJobs.binaryTarballCross."x86_64-linux"."armv7l-unknown-linux-gnueabihf"
self.hydraJobs.binaryTarballCross."x86_64-linux"."riscv64-unknown-linux-gnu"
];
# docker image with Nix inside
dockerImage = lib.genAttrs linux64BitSystems (system: self.packages.${system}.dockerImage);
@@ -156,24 +142,16 @@ in
external-api-docs = nixpkgsFor.x86_64-linux.native.nixComponents.nix-external-api-docs;
# System tests.
tests =
import ../tests/nixos {
inherit
lib
nixpkgs
nixpkgsFor
self
;
}
// {
tests = import ../tests/nixos { inherit lib nixpkgs nixpkgsFor self; } // {
# Make sure that nix-env still produces the exact same result
# on a particular version of Nixpkgs.
evalNixpkgs =
let
inherit (nixpkgsFor.x86_64-linux.native) runCommand nix;
in
runCommand "eval-nixos" { buildInputs = [ nix ]; } ''
# Make sure that nix-env still produces the exact same result
# on a particular version of Nixpkgs.
evalNixpkgs =
let
inherit (nixpkgsFor.x86_64-linux.native) runCommand nix;
in
runCommand "eval-nixos" { buildInputs = [ nix ]; }
''
type -p nix-env
# Note: we're filtering out nixos-install-tools because https://github.com/NixOS/nixpkgs/pull/153594#issuecomment-1020530593.
(
@@ -184,36 +162,36 @@ in
mkdir $out
'';
nixpkgsLibTests = forAllSystems (
system:
import (nixpkgs + "/lib/tests/test-with-nix.nix") {
lib = nixpkgsFor.${system}.native.lib;
nix = self.packages.${system}.nix;
pkgs = nixpkgsFor.${system}.native;
}
nixpkgsLibTests =
forAllSystems (system:
import (nixpkgs + "/lib/tests/test-with-nix.nix")
{
lib = nixpkgsFor.${system}.native.lib;
nix = self.packages.${system}.nix;
pkgs = nixpkgsFor.${system}.native;
}
);
};
};
metrics.nixpkgs = import "${nixpkgs-regression}/pkgs/top-level/metrics.nix" {
pkgs = nixpkgsFor.x86_64-linux.native;
nixpkgs = nixpkgs-regression;
};
installTests = forAllSystems (
system:
let
pkgs = nixpkgsFor.${system}.native;
in
pkgs.runCommand "install-tests" {
againstSelf = testNixVersions pkgs pkgs.nix pkgs.pkgs.nix;
againstCurrentLatest =
# FIXME: temporarily disable this on macOS because of #3605.
if system == "x86_64-linux" then testNixVersions pkgs pkgs.nix pkgs.nixVersions.latest else null;
# Disabled because the latest stable version doesn't handle
# `NIX_DAEMON_SOCKET_PATH` which is required for the tests to work
# againstLatestStable = testNixVersions pkgs pkgs.nix pkgs.nixStable;
} "touch $out"
);
installTests = forAllSystems (system:
let pkgs = nixpkgsFor.${system}.native; in
pkgs.runCommand "install-tests"
{
againstSelf = testNixVersions pkgs pkgs.nix pkgs.pkgs.nix;
againstCurrentLatest =
# FIXME: temporarily disable this on macOS because of #3605.
if system == "x86_64-linux"
then testNixVersions pkgs pkgs.nix pkgs.nixVersions.latest
else null;
# Disabled because the latest stable version doesn't handle
# `NIX_DAEMON_SOCKET_PATH` which is required for the tests to work
# againstLatestStable = testNixVersions pkgs pkgs.nix pkgs.nixStable;
} "touch $out");
installerTests = import ../tests/installer {
binaryTarballs = self.hydraJobs.binaryTarball;
+1 -1
View File
@@ -1,6 +1,6 @@
#!/usr/bin/env bash
((NEW_NIX_FIRST_BUILD_UID=351))
((NEW_NIX_FIRST_BUILD_UID=301))
id_available(){
dscl . list /Users UniqueID | grep -E '\b'"$1"'\b' >/dev/null
+7 -11
View File
@@ -1,18 +1,14 @@
{
runCommand,
system,
buildPackages,
cacert,
nix,
{ runCommand
, system
, buildPackages
, cacert
, nix
}:
let
installerClosureInfo = buildPackages.closureInfo {
rootPaths = [
nix
cacert
];
rootPaths = [ nix cacert ];
};
inherit (nix) version;
@@ -69,7 +65,7 @@ runCommand "nix-binary-tarball-${version}" env ''
fn=$out/$dir.tar.xz
mkdir -p $out/nix-support
echo "file binary-dist $fn" >> $out/nix-support/hydra-build-products
tar cfJ $fn \
tar cvfJ $fn \
--owner=0 --group=0 --mode=u+rw,uga+r \
--mtime='1970-01-01' \
--absolute-names \
-6
View File
@@ -1,6 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
system=$(nix eval --raw --impure --expr builtins.currentSystem)
nix eval --json ".#checks.$system" --apply builtins.attrNames | \
jq -r '.[]' | \
xargs -P0 -I '{}' sh -c "nix build -L .#checks.$system.{} || { echo 'FAILED: \033[0;31mnix build -L .#checks.$system.{}\\033[0m'; kill 0; }"
+1 -1
View File
@@ -463,7 +463,7 @@ EOF
EDITOR="$SCRATCH/ex_cleanroom_wrapper" _sudo "to add nix to fstab" "$@" <<EOF
:a
UUID=$uuid $escaped_mountpoint apfs rw,noauto,nobrowse,nosuid,noatime,owners
UUID=$uuid $escaped_mountpoint apfs rw,noauto,nobrowse,suid,owners
.
:x
EOF
+1 -11
View File
@@ -4,17 +4,7 @@ set -eu
set -o pipefail
# System specific settings
# Notes:
# - up to macOS Big Sur we used the same GID/UIDs as Linux (30000:30001-32)
# - we changed UID to 301 because Big Sur updates failed into recovery mode
# we're targeting the 200-400 UID range for role users mentioned in the
# usage note for sysadminctl
# - we changed UID to 351 because Sequoia now uses UIDs 300-304 for its own
# daemon users
# - we changed GID to 350 alongside above just because it hides the nixbld
# group from the Users & Groups settings panel :)
export NIX_FIRST_BUILD_UID="${NIX_FIRST_BUILD_UID:-351}"
export NIX_BUILD_GROUP_ID="${NIX_BUILD_GROUP_ID:-350}"
export NIX_FIRST_BUILD_UID="${NIX_FIRST_BUILD_UID:-301}"
export NIX_BUILD_USER_NAME_TEMPLATE="_nixbld%d"
readonly NIX_DAEMON_DEST=/Library/LaunchDaemons/org.nixos.nix-daemon.plist
+4 -2
View File
@@ -23,10 +23,10 @@ readonly RED='\033[31m'
# installer allows overriding build user count to speed up installation
# as creating each user takes non-trivial amount of time on macos
readonly NIX_USER_COUNT=${NIX_USER_COUNT:-32}
readonly NIX_BUILD_GROUP_ID="${NIX_BUILD_GROUP_ID:-30000}"
readonly NIX_BUILD_GROUP_NAME="nixbld"
# each system specific installer must set these:
# NIX_FIRST_BUILD_UID
# NIX_BUILD_GROUP_ID
# NIX_BUILD_USER_NAME_TEMPLATE
# Please don't change this. We don't support it, because the
# default shell profile that comes with Nix doesn't support it.
@@ -530,7 +530,9 @@ It seems the build group $NIX_BUILD_GROUP_NAME already exists, but
with the UID $primary_group_id. This script can't really handle
that right now, so I'm going to give up.
You can export NIX_BUILD_GROUP_ID=$primary_group_id and re-run.
You can fix this by editing this script and changing the
NIX_BUILD_GROUP_ID variable near the top to from $NIX_BUILD_GROUP_ID
to $primary_group_id and re-run.
EOF
else
row " Exists" "Yes"
-4
View File
@@ -5,7 +5,6 @@ set -o pipefail
# System specific settings
export NIX_FIRST_BUILD_UID="${NIX_FIRST_BUILD_UID:-30001}"
export NIX_BUILD_GROUP_ID="${NIX_BUILD_GROUP_ID:-30000}"
export NIX_BUILD_USER_NAME_TEMPLATE="nixbld%d"
readonly SERVICE_SRC=/lib/systemd/system/nix-daemon.service
@@ -96,9 +95,6 @@ poly_configure_nix_daemon_service() {
if [ -e /run/systemd/system ]; then
task "Setting up the nix-daemon systemd service"
_sudo "to create parent of the nix-daemon tmpfiles config" \
mkdir -p "$(dirname "$TMPFILES_DEST")"
_sudo "to create the nix-daemon tmpfiles config" \
ln -sfn "/nix/var/nix/profiles/default$TMPFILES_SRC" "$TMPFILES_DEST"
+31 -37
View File
@@ -1,42 +1,36 @@
{
lib,
runCommand,
nix,
tarballs,
{ lib
, runCommand
, nix
, tarballs
}:
runCommand "installer-script"
{
buildInputs = [ nix ];
runCommand "installer-script" {
buildInputs = [ nix ];
} ''
mkdir -p $out/nix-support
# Converts /nix/store/50p3qk8k...-nix-2.4pre20201102_550e11f/bin/nix to 50p3qk8k.../bin/nix.
tarballPath() {
# Remove the store prefix
local path=''${1#${builtins.storeDir}/}
# Get the path relative to the derivation root
local rest=''${path#*/}
# Get the derivation hash
local drvHash=''${path%%-*}
echo "$drvHash/$rest"
}
''
mkdir -p $out/nix-support
# Converts /nix/store/50p3qk8k...-nix-2.4pre20201102_550e11f/bin/nix to 50p3qk8k.../bin/nix.
tarballPath() {
# Remove the store prefix
local path=''${1#${builtins.storeDir}/}
# Get the path relative to the derivation root
local rest=''${path#*/}
# Get the derivation hash
local drvHash=''${path%%-*}
echo "$drvHash/$rest"
}
substitute ${./install.in} $out/install \
${lib.concatMapStrings
(tarball: let
inherit (tarball.stdenv.hostPlatform) system;
in '' \
--replace '@tarballHash_${system}@' $(nix --experimental-features nix-command hash-file --base16 --type sha256 ${tarball}/*.tar.xz) \
--replace '@tarballPath_${system}@' $(tarballPath ${tarball}/*.tar.xz) \
''
)
tarballs
} --replace '@nixVersion@' ${nix.version}
substitute ${./install.in} $out/install \
${
lib.concatMapStrings (
tarball:
let
inherit (tarball.stdenv.hostPlatform) system;
in
''
\
--replace '@tarballHash_${system}@' $(nix --experimental-features nix-command hash-file --base16 --type sha256 ${tarball}/*.tar.xz) \
--replace '@tarballPath_${system}@' $(tarballPath ${tarball}/*.tar.xz) \
''
) tarballs
} --replace '@nixVersion@' ${nix.version}
echo "file installer $out/install" >> $out/nix-support/hydra-build-products
''
echo "file installer $out/install" >> $out/nix-support/hydra-build-products
''
+1 -1
View File
@@ -1,7 +1,7 @@
# Only execute this file once per shell.
# This file is tested by tests/installer/default.nix.
if [ -n "${__ETC_PROFILE_NIX_SOURCED:-}" ]; then return; fi
export __ETC_PROFILE_NIX_SOURCED=1
__ETC_PROFILE_NIX_SOURCED=1
NIX_LINK=$HOME/.nix-profile
if [ -n "${XDG_STATE_HOME-}" ]; then
+1 -1
View File
@@ -1,5 +1,5 @@
# This file is tested by tests/installer/default.nix.
if [ -n "${HOME-}" ] && [ -n "${USER-}" ]; then
if [ -n "$HOME" ] && [ -n "$USER" ]; then
# Set up the per-user profile.
+6 -7
View File
@@ -1,11 +1,10 @@
#!/usr/bin/env bash
set -euo pipefail
set -e
nix build -L ".#installerScriptForGHA" ".#binaryTarball"
script=$(nix-build -A outputs.hydraJobs.installerScriptForGHA --no-out-link)
installerHash=$(echo "$script" | cut -b12-43 -)
mkdir -p out
cp ./result/install "out/install"
name="$(basename "$(realpath ./result-1)")"
# everything before the first dash
cp -r ./result-1 "out/${name%%-*}"
installerURL=https://$CACHIX_NAME.cachix.org/serve/$installerHash/install
echo "::set-output name=installerURL::$installerURL"
-172
View File
@@ -1,172 +0,0 @@
#!/usr/bin/env bash
set -eo pipefail
# stock path to avoid unexpected command versions
PATH="$(/usr/bin/getconf PATH)"
((NEW_NIX_FIRST_BUILD_UID=351))
((TEMP_NIX_FIRST_BUILD_UID=31000))
nix_user_n() {
printf "_nixbld%d" "$1"
}
id_unavailable(){
dscl . list /Users UniqueID | grep -E '\b'"$1"'\b' >/dev/null
}
any_nixbld(){
dscl . list /Users UniqueID | grep -E '\b_nixbld' >/dev/null
}
dsclattr() {
dscl . -read "$1" | awk "/$2/ { print \$2 }"
}
re_create_nixbld_user(){
local name uid
name="$1"
uid="$2"
gid="$3"
sudo /usr/bin/dscl . -create "/Users/$name" "UniqueID" "$uid"
sudo /usr/bin/dscl . -create "/Users/$name" "IsHidden" "1"
sudo /usr/bin/dscl . -create "/Users/$name" "NFSHomeDirectory" "/var/empty"
sudo /usr/bin/dscl . -create "/Users/$name" "RealName" "Nix build user $name"
sudo /usr/bin/dscl . -create "/Users/$name" "UserShell" "/sbin/nologin"
sudo /usr/bin/dscl . -create "/Users/$name" "PrimaryGroupID" "$gid"
}
hit_id_cap(){
echo "We've hit UID 400 without placing all of your users :("
echo "You should use the commands in this script as a starting"
echo "point to review your UID-space and manually move the"
echo "remaining users (or delete them, if you don't need them)."
}
# evacuate the role-uid space to simplify final placement logic
temporarily_move_existing_nixbld_uids(){
local name uid next_id user_n
((next_id=TEMP_NIX_FIRST_BUILD_UID))
echo ""
echo "Step 1: move existing _nixbld users out of the destination UID range."
while read -r name uid; do
# iterate for a clean ID
while id_unavailable "$next_id"; do
((next_id++))
# We really want to get these all placed, but I guess there's
# some risk we iterate forever--so we'll give up after 9k uids.
if ((next_id >= 40000)); then
echo "We've hit UID 40000 without temporarily placing all of your users :("
echo "You should use the commands in this script as a starting"
echo "point to review your UID-space and manually move the"
echo "remaining users to any open UID over 1000."
exit 1
fi
done
sudo dscl . -create "/Users/$name" UniqueID "$next_id"
echo " Temporarily moved $name from uid $uid -> $next_id"
done < <(dscl . list /Users UniqueID | grep _nixbld | sort -n -k2)
}
change_nixbld_uids(){
local existing_gid name next_id user_n
((next_id=NEW_NIX_FIRST_BUILD_UID))
((user_n=1))
name="$(nix_user_n "$user_n")"
existing_gid="$(dsclattr "/Groups/nixbld" "PrimaryGroupID")"
# we know that we have *some* nixbld users, but macOS may have
# already clobbered the first few users if this system has been
# upgraded
echo ""
echo "Step 2: re-create missing early _nixbld# users."
until dscl . read "/Users/$name" &>/dev/null; do
# iterate for a clean ID
while id_unavailable "$next_id"; do
((next_id++))
if ((next_id >= 400)); then
hit_id_cap
exit 1
fi
done
re_create_nixbld_user "$name" "$next_id" "$existing_gid"
echo " $name was missing; created with uid: $next_id"
((user_n++))
name="$(nix_user_n "$user_n")"
done
echo ""
echo "Step 3: relocate remaining _nixbld# UIDs to $next_id+"
# start at first _nixbld# not re-created above and increment
# until _nixbld<n> doesn't exist
while dscl . read "/Users/$name" &>/dev/null; do
# iterate for a clean ID
while id_unavailable "$next_id"; do
((next_id++))
if ((next_id >= 400)); then
hit_id_cap
exit 1
fi
done
sudo dscl . -create "/Users/$name" UniqueID "$next_id"
echo " $name migrated to uid: $next_id"
((user_n++))
name="$(nix_user_n "$user_n")"
done
if ((user_n == 1)); then
echo "Didn't find _nixbld1. Perhaps you have single-user Nix?"
exit 1
else
echo "Migrated $((user_n - 1)) users. If you want to double-check, try:"
echo "dscl . list /Users UniqueID | grep _nixbld | sort -n -k2"
fi
}
needs_migration(){
local name uid next_id user_n
((next_id=NEW_NIX_FIRST_BUILD_UID))
((user_n=1))
while read -r name uid; do
expected_name="$(nix_user_n "$user_n")"
if [[ "$expected_name" != "$name" ]]; then
return 0
fi
if [[ "$next_id" != "$uid" ]]; then
return 0
fi
((next_id++))
((user_n++))
done < <(dscl . list /Users UniqueID | grep _nixbld | sort -n -k2)
return 1
}
if any_nixbld; then
if needs_migration; then
echo "Attempting to migrate _nixbld users."
temporarily_move_existing_nixbld_uids
change_nixbld_uids
else
echo "_nixbld users already appear to be migrated."
fi
else
echo "Didn't find any _nixbld users. Perhaps you have single-user Nix?"
exit 1
fi
@@ -1,22 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
if [[ ! -d out ]]; then
echo "run prepare-installer-for-github-actions first"
exit 1
fi
cd out
PORT=${PORT:-8126}
nohup python -m http.server "$PORT" >/dev/null 2>&1 &
pid=$!
while ! curl -s "http://localhost:$PORT"; do
sleep 1
if ! kill -0 $pid; then
echo "Failed to start http server"
exit 1
fi
done
echo 'To install nix, run the following command:'
echo "sh <(curl http://localhost:$PORT/install) --tarball-url-prefix http://localhost:$PORT"
+12 -12
View File
@@ -1,14 +1,13 @@
{
lib,
mkMesonDerivation,
{ lib
, mkMesonDerivation
meson,
ninja,
doxygen,
, meson
, ninja
, doxygen
# Configuration Options
# Configuration Options
version,
, version
}:
let
@@ -43,10 +42,11 @@ mkMesonDerivation (finalAttrs: {
doxygen
];
preConfigure = ''
chmod u+w ./.version
echo ${finalAttrs.version} > ./.version
'';
preConfigure =
''
chmod u+w ./.version
echo ${finalAttrs.version} > ./.version
'';
postInstall = ''
mkdir -p ''${!outputDoc}/nix-support
+23 -25
View File
@@ -1,14 +1,13 @@
{
lib,
mkMesonDerivation,
{ lib
, mkMesonDerivation
meson,
ninja,
doxygen,
, meson
, ninja
, doxygen
# Configuration Options
# Configuration Options
version,
, version
}:
let
@@ -20,19 +19,17 @@ mkMesonDerivation (finalAttrs: {
inherit version;
workDir = ./.;
fileset =
let
cpp = fileset.fileFilter (file: file.hasExt "cc" || file.hasExt "hh");
in
fileset.unions [
./.version
../../.version
./meson.build
./doxygen.cfg.in
# Source is not compiled, but still must be available for Doxygen
# to gather comments.
(cpp ../.)
];
fileset = let
cpp = fileset.fileFilter (file: file.hasExt "cc" || file.hasExt "hh");
in fileset.unions [
./.version
../../.version
./meson.build
./doxygen.cfg.in
# Source is not compiled, but still must be available for Doxygen
# to gather comments.
(cpp ../.)
];
nativeBuildInputs = [
meson
@@ -40,10 +37,11 @@ mkMesonDerivation (finalAttrs: {
doxygen
];
preConfigure = ''
chmod u+w ./.version
echo ${finalAttrs.version} > ./.version
'';
preConfigure =
''
chmod u+w ./.version
echo ${finalAttrs.version} > ./.version
'';
postInstall = ''
mkdir -p ''${!outputDoc}/nix-support
+1 -3
View File
@@ -171,9 +171,7 @@ SourcePath lookupFileArg(EvalState & state, std::string_view s, const Path * bas
{
if (EvalSettings::isPseudoUrl(s)) {
auto accessor = fetchers::downloadTarball(
state.store,
state.fetchSettings,
EvalSettings::resolvePseudoUrl(s));
EvalSettings::resolvePseudoUrl(s)).accessor;
auto storePath = fetchToStore(*state.store, SourcePath(accessor), FetchMode::Copy);
return state.rootPath(CanonPath(state.store->toRealPath(storePath)));
}
+10
View File
@@ -32,6 +32,16 @@ InstallableDerivedPath InstallableDerivedPath::parse(
// store path.
[&](const ExtendedOutputsSpec::Default &) -> DerivedPath {
auto storePath = store->followLinksToStorePath(prefix);
// Remove this prior to stabilizing the new CLI.
if (storePath.isDerivation()) {
auto oldDerivedPath = DerivedPath::Built {
.drvPath = makeConstantStorePathRef(storePath),
.outputs = OutputsSpec::All { },
};
warn(
"The interpretation of store paths arguments ending in `.drv` recently changed. If this command is now failing try again with '%s'",
oldDerivedPath.to_string(*store));
};
return DerivedPath::Opaque {
.path = std::move(storePath),
};
+2 -2
View File
@@ -104,12 +104,12 @@ DerivedPathsWithInfo InstallableFlake::toDerivedPaths()
auto drvPath = attr->forceDerivation();
std::optional<NixInt> priority;
std::optional<NixInt::Inner> priority;
if (attr->maybeGetAttr(state->sOutputSpecified)) {
} else if (auto aMeta = attr->maybeGetAttr(state->sMeta)) {
if (auto aPriority = aMeta->maybeGetAttr("priority"))
priority = aPriority->getInt();
priority = aPriority->getInt().value;
}
return {{
+1 -1
View File
@@ -40,7 +40,7 @@ struct ExtraPathInfoValue : ExtraPathInfo
/**
* An optional priority for use with "build envs". See Package
*/
std::optional<NixInt> priority;
std::optional<NixInt::Inner> priority;
/**
* The attribute path associated with this value. The idea is
+1 -1
View File
@@ -86,7 +86,7 @@ MixFlakeOptions::MixFlakeOptions()
> **DEPRECATED**
>
> Use [`--no-use-registries`](@docroot@/command-ref/conf-file.md#conf-use-registries) instead.
> Use [`--no-use-registries`](#opt-no-use-registries) instead.
)",
.category = category,
.handler = {[&]() {
-12
View File
@@ -16,25 +16,13 @@ static std::string doRenderMarkdownToTerminal(std::string_view markdown)
{
int windowWidth = getWindowSize().second;
#if HAVE_LOWDOWN_1_4
struct lowdown_opts_term opts_term {
.cols = (size_t) std::max(windowWidth - 5, 60),
.hmargin = 0,
.vmargin = 0,
};
#endif
struct lowdown_opts opts
{
.type = LOWDOWN_TERM,
#if HAVE_LOWDOWN_1_4
.term = opts_term,
#endif
.maxdepth = 20,
#if !HAVE_LOWDOWN_1_4
.cols = (size_t) std::max(windowWidth - 5, 60),
.hmargin = 0,
.vmargin = 0,
#endif
.feat = LOWDOWN_COMMONMARK | LOWDOWN_FENCED | LOWDOWN_DEFLIST | LOWDOWN_TABLES,
.oflags = LOWDOWN_TERM_NOLINK,
};
-2
View File
@@ -38,8 +38,6 @@ deps_public += nlohmann_json
lowdown = dependency('lowdown', version : '>= 0.9.0', required : get_option('markdown'))
deps_private += lowdown
configdata.set('HAVE_LOWDOWN', lowdown.found().to_int())
# The API changed slightly around terminal initialization.
configdata.set('HAVE_LOWDOWN_1_4', lowdown.version().version_compare('>= 1.4.0').to_int())
readline_flavor = get_option('readline-flavor')
if readline_flavor == 'editline'
+32 -39
View File
@@ -1,38 +1,37 @@
{
lib,
stdenv,
mkMesonDerivation,
releaseTools,
{ lib
, stdenv
, mkMesonDerivation
, releaseTools
meson,
ninja,
pkg-config,
, meson
, ninja
, pkg-config
nix-util,
nix-store,
nix-fetchers,
nix-expr,
nix-flake,
nix-main,
editline,
readline,
lowdown,
nlohmann_json,
, nix-util
, nix-store
, nix-fetchers
, nix-expr
, nix-flake
, nix-main
, editline
, readline
, lowdown
, nlohmann_json
# Configuration Options
# Configuration Options
version,
, version
# Whether to enable Markdown rendering in the Nix binary.
enableMarkdown ? !stdenv.hostPlatform.isWindows,
# Whether to enable Markdown rendering in the Nix binary.
, enableMarkdown ? !stdenv.hostPlatform.isWindows
# Which interactive line editor library to use for Nix's repl.
#
# Currently supported choices are:
#
# - editline (default)
# - readline
readlineFlavor ? if stdenv.hostPlatform.isWindows then "readline" else "editline",
# Which interactive line editor library to use for Nix's repl.
#
# Currently supported choices are:
#
# - editline (default)
# - readline
, readlineFlavor ? if stdenv.hostPlatform.isWindows then "readline" else "editline"
}:
let
@@ -55,10 +54,7 @@ mkMesonDerivation (finalAttrs: {
(fileset.fileFilter (file: file.hasExt "hh") ./.)
];
outputs = [
"out"
"dev"
];
outputs = [ "out" "dev" ];
nativeBuildInputs = [
meson
@@ -93,12 +89,9 @@ mkMesonDerivation (finalAttrs: {
(lib.mesonOption "readline-flavor" readlineFlavor)
];
env =
lib.optionalAttrs
(stdenv.isLinux && !(stdenv.hostPlatform.isStatic && stdenv.system == "aarch64-linux"))
{
LDFLAGS = "-fuse-ld=gold";
};
env = lib.optionalAttrs (stdenv.isLinux && !(stdenv.hostPlatform.isStatic && stdenv.system == "aarch64-linux")) {
LDFLAGS = "-fuse-ld=gold";
};
separateDebugInfo = !stdenv.hostPlatform.isStatic;
+24 -25
View File
@@ -7,7 +7,6 @@
#include "ansicolor.hh"
#include "shared.hh"
#include "config-global.hh"
#include "eval.hh"
#include "eval-settings.hh"
#include "attr-path.hh"
@@ -77,10 +76,14 @@ struct NixRepl
int displ;
StringSet varNames;
RunNix * runNixPtr;
void runNix(Path program, const Strings & args, const std::optional<std::string> & input = {});
std::unique_ptr<ReplInteracter> interacter;
NixRepl(const LookupPath & lookupPath, nix::ref<Store> store,ref<EvalState> state,
std::function<AnnotatedValues()> getValues);
std::function<AnnotatedValues()> getValues, RunNix * runNix);
virtual ~NixRepl() = default;
ReplExitStatus mainLoop() override;
@@ -125,45 +128,32 @@ std::string removeWhitespace(std::string s)
NixRepl::NixRepl(const LookupPath & lookupPath, nix::ref<Store> store, ref<EvalState> state,
std::function<NixRepl::AnnotatedValues()> getValues)
std::function<NixRepl::AnnotatedValues()> getValues, RunNix * runNix = nullptr)
: AbstractNixRepl(state)
, debugTraceIndex(0)
, getValues(getValues)
, staticEnv(new StaticEnv(nullptr, state->staticBaseEnv.get()))
, runNixPtr{runNix}
, interacter(make_unique<ReadlineLikeInteracter>(getDataDir() + "/nix/repl-history"))
{
}
void runNix(Path program, const Strings & args,
const std::optional<std::string> & input = {})
{
auto subprocessEnv = getEnv();
subprocessEnv["NIX_CONFIG"] = globalConfig.toKeyValue();
//isInteractive avoid grabling interactive commands
runProgram2(RunOptions {
.program = settings.nixBinDir+ "/" + program,
.args = args,
.environment = subprocessEnv,
.input = input,
.isInteractive = true,
});
return;
}
static std::ostream & showDebugTrace(std::ostream & out, const PosTable & positions, const DebugTrace & dt)
{
if (dt.isError)
out << ANSI_RED "error: " << ANSI_NORMAL;
out << dt.hint.str() << "\n";
auto pos = dt.getPos(positions);
// prefer direct pos, but if noPos then try the expr.
auto pos = dt.pos
? dt.pos
: positions[dt.expr.getPos() ? dt.expr.getPos() : noPos];
if (pos) {
out << pos;
if (auto loc = pos.getCodeLines()) {
out << *pos;
if (auto loc = pos->getCodeLines()) {
out << "\n";
printCodeLines(out, "", pos, *loc);
printCodeLines(out, "", *pos, *loc);
out << "\n";
}
}
@@ -830,9 +820,18 @@ void NixRepl::evalString(std::string s, Value & v)
}
void NixRepl::runNix(Path program, const Strings & args, const std::optional<std::string> & input)
{
if (runNixPtr)
(*runNixPtr)(program, args, input);
else
throw Error("Cannot run '%s', no method of calling the Nix CLI provided", program);
}
std::unique_ptr<AbstractNixRepl> AbstractNixRepl::create(
const LookupPath & lookupPath, nix::ref<Store> store, ref<EvalState> state,
std::function<AnnotatedValues()> getValues)
std::function<AnnotatedValues()> getValues, RunNix * runNix)
{
return std::make_unique<NixRepl>(
lookupPath,
+12 -2
View File
@@ -19,9 +19,19 @@ struct AbstractNixRepl
typedef std::vector<std::pair<Value*,std::string>> AnnotatedValues;
using RunNix = void(Path program, const Strings & args, const std::optional<std::string> & input);
/**
* @param runNix Function to run the nix CLI to support various
* `:<something>` commands. Optional; if not provided,
* everything else will still work fine, but those commands won't.
*/
static std::unique_ptr<AbstractNixRepl> create(
const LookupPath & lookupPath, nix::ref<Store> store, ref<EvalState> state,
std::function<AnnotatedValues()> getValues);
const LookupPath & lookupPath,
nix::ref<Store> store,
ref<EvalState> state,
std::function<AnnotatedValues()> getValues,
RunNix * runNix = nullptr);
static ReplExitStatus runSimple(
ref<EvalState> evalState,
+6 -2
View File
@@ -306,7 +306,7 @@ int64_t nix_get_int(nix_c_context * context, const nix_value * value)
try {
auto & v = check_value_in(value);
assert(v.type() == nix::nInt);
return v.integer();
return v.integer().value;
}
NIXC_CATCH_ERRS_RES(0);
}
@@ -619,8 +619,12 @@ nix_realised_string * nix_string_realise(nix_c_context * context, EvalState * st
context->last_err_code = NIX_OK;
try {
auto & v = check_value_in(value);
nix::NixStringContext stringContext;
auto rawStr = state->state.coerceToString(nix::noPos, v, stringContext, "while realising a string").toOwned();
nix::StorePathSet storePaths;
auto s = state->state.realiseString(v, &storePaths, isIFD);
auto rewrites = state->state.realiseContext(stringContext, &storePaths);
auto s = nix::rewriteStrings(rawStr, rewrites);
// Convert to the C API StorePath type and convert to vector for index-based access
std::vector<StorePath> vec;
+14 -21
View File
@@ -1,18 +1,17 @@
{
lib,
stdenv,
mkMesonDerivation,
{ lib
, stdenv
, mkMesonDerivation
meson,
ninja,
pkg-config,
, meson
, ninja
, pkg-config
nix-store-c,
nix-expr,
, nix-store-c
, nix-expr
# Configuration Options
# Configuration Options
version,
, version
}:
let
@@ -36,10 +35,7 @@ mkMesonDerivation (finalAttrs: {
(fileset.fileFilter (file: file.hasExt "h") ./.)
];
outputs = [
"out"
"dev"
];
outputs = [ "out" "dev" ];
nativeBuildInputs = [
meson
@@ -63,12 +59,9 @@ mkMesonDerivation (finalAttrs: {
mesonFlags = [
];
env =
lib.optionalAttrs
(stdenv.isLinux && !(stdenv.hostPlatform.isStatic && stdenv.system == "aarch64-linux"))
{
LDFLAGS = "-fuse-ld=gold";
};
env = lib.optionalAttrs (stdenv.isLinux && !(stdenv.hostPlatform.isStatic && stdenv.system == "aarch64-linux")) {
LDFLAGS = "-fuse-ld=gold";
};
separateDebugInfo = !stdenv.hostPlatform.isStatic;
+47 -47
View File
@@ -17,69 +17,69 @@ let
# Resolve a input spec into a node name. An input spec is
# either a node name, or a 'follows' path from the root
# node.
resolveInput =
inputSpec: if builtins.isList inputSpec then getInputByPath lockFile.root inputSpec else inputSpec;
resolveInput = inputSpec:
if builtins.isList inputSpec
then getInputByPath lockFile.root inputSpec
else inputSpec;
# Follow an input path (e.g. ["dwarffs" "nixpkgs"]) from the
# root node, returning the final node.
getInputByPath =
nodeName: path:
if path == [ ] then
nodeName
getInputByPath = nodeName: path:
if path == []
then nodeName
else
getInputByPath
# Since this could be a 'follows' input, call resolveInput.
(resolveInput lockFile.nodes.${nodeName}.inputs.${builtins.head path})
(builtins.tail path);
allNodes = builtins.mapAttrs (
key: node:
let
allNodes =
builtins.mapAttrs
(key: node:
let
sourceInfo =
if overrides ? ${key} then
overrides.${key}.sourceInfo
else
# FIXME: remove obsolete node.info.
fetchTree (node.info or { } // removeAttrs node.locked [ "dir" ]);
sourceInfo =
if overrides ? ${key}
then
overrides.${key}.sourceInfo
else
# FIXME: remove obsolete node.info.
fetchTree (node.info or {} // removeAttrs node.locked ["dir"]);
subdir = overrides.${key}.dir or node.locked.dir or "";
subdir = overrides.${key}.dir or node.locked.dir or "";
outPath = sourceInfo + ((if subdir == "" then "" else "/") + subdir);
outPath = sourceInfo + ((if subdir == "" then "" else "/") + subdir);
flake = import (outPath + "/flake.nix");
flake = import (outPath + "/flake.nix");
inputs = builtins.mapAttrs (inputName: inputSpec: allNodes.${resolveInput inputSpec}) (
node.inputs or { }
);
inputs = builtins.mapAttrs
(inputName: inputSpec: allNodes.${resolveInput inputSpec})
(node.inputs or {});
outputs = flake.outputs (inputs // { self = result; });
outputs = flake.outputs (inputs // { self = result; });
result =
outputs
# We add the sourceInfo attribute for its metadata, as they are
# relevant metadata for the flake. However, the outPath of the
# sourceInfo does not necessarily match the outPath of the flake,
# as the flake may be in a subdirectory of a source.
# This is shadowed in the next //
// sourceInfo
// {
# This shadows the sourceInfo.outPath
inherit outPath;
result =
outputs
# We add the sourceInfo attribute for its metadata, as they are
# relevant metadata for the flake. However, the outPath of the
# sourceInfo does not necessarily match the outPath of the flake,
# as the flake may be in a subdirectory of a source.
# This is shadowed in the next //
// sourceInfo
// {
# This shadows the sourceInfo.outPath
inherit outPath;
inherit inputs;
inherit outputs;
inherit sourceInfo;
_type = "flake";
};
inherit inputs; inherit outputs; inherit sourceInfo; _type = "flake";
};
in
if node.flake or true then
assert builtins.isFunction flake.outputs;
result
else
sourceInfo
) lockFile.nodes;
in
if node.flake or true then
assert builtins.isFunction flake.outputs;
result
else
sourceInfo
)
lockFile.nodes;
in
allNodes.${lockFile.root}
in allNodes.${lockFile.root}
+4 -2
View File
@@ -4,6 +4,8 @@
#include "eval.hh"
#include "eval-inline.hh"
#include "store-api.hh"
// Need specialization involving `SymbolStr` just in this one module.
#include "strings-inline.hh"
namespace nix::eval_cache {
@@ -326,7 +328,7 @@ struct AttrDb
case AttrType::Bool:
return {{rowId, queryAttribute.getInt(2) != 0}};
case AttrType::Int:
return {{rowId, int_t{queryAttribute.getInt(2)}}};
return {{rowId, int_t{NixInt{queryAttribute.getInt(2)}}}};
case AttrType::ListOfStrings:
return {{rowId, tokenizeString<std::vector<std::string>>(queryAttribute.getStr(2), "\t")}};
case AttrType::Missing:
@@ -469,7 +471,7 @@ Value & AttrCursor::forceValue()
else if (v.type() == nBool)
cachedValue = {root->db->setBool(getKey(), v.boolean()), v.boolean()};
else if (v.type() == nInt)
cachedValue = {root->db->setInt(getKey(), v.integer()), int_t{v.integer()}};
cachedValue = {root->db->setInt(getKey(), v.integer().value), int_t{v.integer()}};
else if (v.type() == nAttrs)
; // FIXME: do something?
else
+1 -1
View File
@@ -45,7 +45,7 @@ EvalErrorBuilder<T> & EvalErrorBuilder<T>::withFrame(const Env & env, const Expr
// TODO: check compatibility with nested debugger calls.
// TODO: What side-effects??
error.state.debugTraces.push_front(DebugTrace{
.pos = expr.getPos(),
.pos = error.state.positions[expr.getPos()],
.expr = expr,
.env = env,
.hint = HintFmt("Fake frame for debugging purposes"),
+134
View File
@@ -32,6 +32,122 @@ static void * oomHandler(size_t requested)
throw std::bad_alloc();
}
class BoehmGCStackAllocator : public StackAllocator
{
boost::coroutines2::protected_fixedsize_stack stack{
// We allocate 8 MB, the default max stack size on NixOS.
// A smaller stack might be quicker to allocate but reduces the stack
// depth available for source filter expressions etc.
std::max(boost::context::stack_traits::default_size(), static_cast<std::size_t>(8 * 1024 * 1024))};
// This is specific to boost::coroutines2::protected_fixedsize_stack.
// The stack protection page is included in sctx.size, so we have to
// subtract one page size from the stack size.
std::size_t pfss_usable_stack_size(boost::context::stack_context & sctx)
{
return sctx.size - boost::context::stack_traits::page_size();
}
public:
boost::context::stack_context allocate() override
{
auto sctx = stack.allocate();
// Stacks generally start at a high address and grow to lower addresses.
// Architectures that do the opposite are rare; in fact so rare that
// boost_routine does not implement it.
// So we subtract the stack size.
GC_add_roots(static_cast<char *>(sctx.sp) - pfss_usable_stack_size(sctx), sctx.sp);
return sctx;
}
void deallocate(boost::context::stack_context sctx) override
{
GC_remove_roots(static_cast<char *>(sctx.sp) - pfss_usable_stack_size(sctx), sctx.sp);
stack.deallocate(sctx);
}
};
static BoehmGCStackAllocator boehmGCStackAllocator;
/**
* When a thread goes into a coroutine, we lose its original sp until
* control flow returns to the thread.
* While in the coroutine, the sp points outside the thread stack,
* so we can detect this and push the entire thread stack instead,
* as an approximation.
* The coroutine's stack is covered by `BoehmGCStackAllocator`.
* This is not an optimal solution, because the garbage is scanned when a
* coroutine is active, for both the coroutine and the original thread stack.
* However, the implementation is quite lean, and usually we don't have active
* coroutines during evaluation, so this is acceptable.
*/
void fixupBoehmStackPointer(void ** sp_ptr, void * _pthread_id)
{
void *& sp = *sp_ptr;
auto pthread_id = reinterpret_cast<pthread_t>(_pthread_id);
# ifndef __APPLE__
pthread_attr_t pattr;
# endif
size_t osStackSize;
// The low address of the stack, which grows down.
void * osStackLimit;
void * osStackBase;
# ifdef __APPLE__
osStackSize = pthread_get_stacksize_np(pthread_id);
osStackLimit = pthread_get_stackaddr_np(pthread_id);
# else
if (pthread_attr_init(&pattr)) {
throw Error("fixupBoehmStackPointer: pthread_attr_init failed");
}
# ifdef HAVE_PTHREAD_GETATTR_NP
if (pthread_getattr_np(pthread_id, &pattr)) {
throw Error("fixupBoehmStackPointer: pthread_getattr_np failed");
}
# elif HAVE_PTHREAD_ATTR_GET_NP
if (!pthread_attr_init(&pattr)) {
throw Error("fixupBoehmStackPointer: pthread_attr_init failed");
}
if (!pthread_attr_get_np(pthread_id, &pattr)) {
throw Error("fixupBoehmStackPointer: pthread_attr_get_np failed");
}
# else
# error "Need one of `pthread_attr_get_np` or `pthread_getattr_np`"
# endif
if (pthread_attr_getstack(&pattr, &osStackLimit, &osStackSize)) {
throw Error("fixupBoehmStackPointer: pthread_attr_getstack failed");
}
if (pthread_attr_destroy(&pattr)) {
throw Error("fixupBoehmStackPointer: pthread_attr_destroy failed");
}
# endif
osStackBase = (char *) osStackLimit + osStackSize;
// NOTE: We assume the stack grows down, as it does on all architectures we support.
// Architectures that grow the stack up are rare.
if (sp >= osStackBase || sp < osStackLimit) { // sp is outside the os stack
sp = osStackLimit;
}
}
/* Disable GC while this object lives. Used by CoroutineContext.
*
* Boehm keeps a count of GC_disable() and GC_enable() calls,
* and only enables GC when the count matches.
*/
class BoehmDisableGC
{
public:
BoehmDisableGC()
{
GC_disable();
};
~BoehmDisableGC()
{
GC_enable();
};
};
static inline void initGCReal()
{
/* Initialise the Boehm garbage collector. */
@@ -52,6 +168,24 @@ static inline void initGCReal()
GC_set_oom_fn(oomHandler);
StackAllocator::defaultAllocator = &boehmGCStackAllocator;
// TODO: Remove __APPLE__ condition.
// Comment suggests an implementation that works on darwin and windows
// https://github.com/ivmai/bdwgc/issues/362#issuecomment-1936672196
# if GC_VERSION_MAJOR >= 8 && GC_VERSION_MINOR >= 2 && GC_VERSION_MICRO >= 4 && !defined(__APPLE__)
GC_set_sp_corrector(&fixupBoehmStackPointer);
if (!GC_get_sp_corrector()) {
printTalkative("BoehmGC on this platform does not support sp_corrector; will disable GC inside coroutines");
/* Used to disable GC when entering coroutines on macOS */
create_coro_gc_hook = []() -> std::shared_ptr<void> { return std::make_shared<BoehmDisableGC>(); };
}
# else
# warning \
"BoehmGC version does not support GC while coroutine exists. GC will be disabled inside coroutines. Consider updating bdw-gc to 8.2.4 or later."
# endif
/* Set the initial heap size to something fairly big (25% of
physical RAM, up to a maximum of 384 MiB) so that in most cases
we don't need to garbage collect at all. (Collection has a
+1 -1
View File
@@ -57,7 +57,7 @@ Strings EvalSettings::getDefaultNixPath()
{
Strings res;
auto add = [&](const Path & p, const std::string & s = std::string()) {
if (std::filesystem::exists(p)) {
if (pathAccessible(p)) {
if (s.empty()) {
res.push_back(p);
} else {
+187 -58
View File
@@ -379,16 +379,6 @@ void EvalState::allowPath(const StorePath & storePath)
rootFS2->allowPrefix(CanonPath(store->toRealPath(storePath)));
}
void EvalState::allowClosure(const StorePath & storePath)
{
if (!rootFS.dynamic_pointer_cast<AllowListSourceAccessor>()) return;
StorePathSet closure;
store->computeFSClosure(storePath, closure);
for (auto & p : closure)
allowPath(p);
}
void EvalState::allowAndSetStorePathString(const StorePath & storePath, Value & v)
{
allowPath(storePath);
@@ -438,7 +428,7 @@ void EvalState::checkURI(const std::string & uri)
/* If the URI is a path, then check it against allowedPaths as
well. */
if (isAbsolute(uri)) {
if (hasPrefix(uri, "/")) {
if (auto rootFS2 = rootFS.dynamic_pointer_cast<AllowListSourceAccessor>())
rootFS2->checkAccess(CanonPath(uri));
return;
@@ -756,26 +746,18 @@ void EvalState::runDebugRepl(const Error * error, const Env & env, const Expr &
if (!debugRepl || inDebugger)
return;
auto dts = [&]() -> std::unique_ptr<DebugTraceStacker> {
if (error && expr.getPos()) {
auto trace = DebugTrace{
.pos = [&]() -> std::variant<Pos, PosIdx> {
if (error->info().pos) {
if (auto * pos = error->info().pos.get())
return *pos;
return noPos;
}
return expr.getPos();
}(),
auto dts =
error && expr.getPos()
? std::make_unique<DebugTraceStacker>(
*this,
DebugTrace {
.pos = error->info().pos ? error->info().pos : positions[expr.getPos()],
.expr = expr,
.env = env,
.hint = error->info().msg,
.isError = true};
return std::make_unique<DebugTraceStacker>(*this, std::move(trace));
}
return nullptr;
}();
.isError = true
})
: nullptr;
if (error)
{
@@ -820,7 +802,7 @@ static std::unique_ptr<DebugTraceStacker> makeDebugTraceStacker(
EvalState & state,
Expr & expr,
Env & env,
std::variant<Pos, PosIdx> pos,
std::shared_ptr<Pos> && pos,
const Args & ... formatArgs)
{
return std::make_unique<DebugTraceStacker>(state,
@@ -1017,10 +999,6 @@ void EvalState::mkSingleDerivedPathString(
}
/* Create a thunk for the delayed computation of the given expression
in the given environment. But if the expression is a variable,
then look it up right away. This significantly reduces the number
of thunks allocated. */
Value * Expr::maybeThunk(EvalState & state, Env & env)
{
Value * v = state.allocValue();
@@ -1096,7 +1074,7 @@ void EvalState::evalFile(const SourcePath & path, Value & v, bool mustBeTrivial)
*this,
*e,
this->baseEnv,
e->getPos(),
e->getPos() ? std::make_shared<Pos>(positions[e->getPos()]) : nullptr,
"while evaluating the file '%1%':", resolvedPath.to_string())
: nullptr;
@@ -1322,7 +1300,9 @@ void ExprLet::eval(EvalState & state, Env & env, Value & v)
state,
*this,
env2,
getPos(),
getPos()
? std::make_shared<Pos>(state.positions[getPos()])
: nullptr,
"while evaluating a '%1%' expression",
"let"
)
@@ -1391,7 +1371,7 @@ void ExprSelect::eval(EvalState & state, Env & env, Value & v)
state,
*this,
env,
getPos(),
state.positions[getPos()],
"while evaluating the attribute '%1%'",
showAttrPath(state, env, attrPath))
: nullptr;
@@ -1609,7 +1589,7 @@ void EvalState::callFunction(Value & fun, size_t nrArgs, Value * * args, Value &
try {
auto dts = debugRepl
? makeDebugTraceStacker(
*this, *lambda.body, env2, lambda.pos,
*this, *lambda.body, env2, positions[lambda.pos],
"while calling %s",
lambda.name
? concatStrings("'", symbols[lambda.name], "'")
@@ -1748,7 +1728,9 @@ void ExprCall::eval(EvalState & state, Env & env, Value & v)
state,
*this,
env,
getPos(),
getPos()
? std::make_shared<Pos>(state.positions[getPos()])
: nullptr,
"while calling a function"
)
: nullptr;
@@ -1913,6 +1895,112 @@ void ExprOpImpl::eval(EvalState & state, Env & env, Value & v)
void ExprOpUpdate::eval(EvalState & state, Env & env, Value & v)
{
UpdateQueue q;
e1->evalForUpdate(state, env, q);
e2->evalForUpdate(state, env, q);
// k-way merge
size_t capacity = 0;
struct BindingsCursor {
const Bindings * bindings;
Bindings::size_t offset;
const Attr & front() {
return (*bindings)[offset];
}
/**
* @return false if there are no more elements
*/
bool pop_front() {
offset++;
return offset < bindings->size();
}
};
// a nested queue where the key is equal to the first and lowest name
std::map<std::pair<Symbol, int>, BindingsCursor> next;
// first fill next
int bias = 0;
for (auto b : q) {
if (b->size() == 0) continue;
capacity += b->size();
next.insert_or_assign({b->begin()->name, bias}, BindingsCursor {b, 0});
// The `//` operator is "right-biased", which means `{ a = "L"; } // { a = "R"; }`
// will result in `{ a = "R"; }`.
// This means that we want to find the right most value first, so that
// we can then ignore the earlier attributes of the same name.
// Decreasing the bias integer achieves this.
bias--;
}
auto r = state.allocBindings(capacity);
{
auto it = state.nrMultiConcats.find(capacity);
if (it != state.nrMultiConcats.end()) {
it->second++;
} else {
state.nrMultiConcats[capacity] = 1;
}
}
// std::cerr << "capacity: " << r->capacity() << std::endl;
// Consume the lowest item until empty.
// This changes the order of next, so we need to reinsert it each time.
// The last inserted name, so that we can easily ignore all the values that aren't right-most in the sequence of `//` operations.
// Symbol() is a _null object_ that doesn't match any real symbols.
Symbol lastInsertedName;
while (true) {
auto cur = next.begin();
if (cur == next.end()) break;
// std::cerr << "size: " << r->size() << std::endl;
// std::cerr << "processing attr: " << state.symbols[cur->first] << std::endl;
// Grow `r`
if (lastInsertedName != cur->first.first) {
r->push_back(cur->second.front());
lastInsertedName = cur->first.first;
}
// Shrink `next`
int bias = cur->first.second;
if (cur->second.pop_front()) {
std::pair<Symbol, int> key = {cur->second.front().name, bias};
next.emplace(key, std::move(cur->second));
}
next.erase(cur);
}
// TODO check the size of the bindings. If significantly below capacity, shrink it.
if (capacity > 0) {
auto capacityFraction = (100 * r->size()) / capacity;
auto it = state.nrMultiConcatCapacityPercent.find(capacityFraction);
auto wastedAttrs = capacity - r->size();
if (it != state.nrMultiConcatCapacityPercent.end()) {
it->second += wastedAttrs;
} else {
state.nrMultiConcatCapacityPercent[capacityFraction] = wastedAttrs;
}
state.nrMultiConcatCapacityTotalWaste += wastedAttrs;
}
v.mkAttrs(r);
#if 0
Value v1, v2;
state.evalAttrs(env, e1, v1, pos, "in the left operand of the update (//) operator");
state.evalAttrs(env, e2, v2, pos, "in the right operand of the update (//) operator");
@@ -1946,6 +2034,22 @@ void ExprOpUpdate::eval(EvalState & state, Env & env, Value & v)
v.mkAttrs(attrs.alreadySorted());
state.nrOpUpdateValuesCopied += v.attrs()->size();
#endif
}
void Expr::evalForUpdate(EvalState & state, Env & env, UpdateQueue & q)
{
Value vTmp;
eval(state, env, vTmp);
// TODO add pos and errorCtx params
state.forceAttrs(vTmp, noPos, "while evaluating an attribute set merge operand");
q.push_back(vTmp.attrs());
}
void ExprOpUpdate::evalForUpdate(EvalState &state, Env &env, UpdateQueue &q)
{
e1->evalForUpdate(state, env, q);
e2->evalForUpdate(state, env, q);
}
@@ -1993,7 +2097,7 @@ void ExprConcatStrings::eval(EvalState & state, Env & env, Value & v)
NixStringContext context;
std::vector<BackedStringView> s;
size_t sSize = 0;
NixInt n = 0;
NixInt n{0};
NixFloat nf = 0;
bool first = !forceString;
@@ -2037,17 +2141,22 @@ void ExprConcatStrings::eval(EvalState & state, Env & env, Value & v)
if (firstType == nInt) {
if (vTmp.type() == nInt) {
n += vTmp.integer();
auto newN = n + vTmp.integer();
if (auto checked = newN.valueChecked(); checked.has_value()) {
n = NixInt(*checked);
} else {
state.error<EvalError>("integer overflow in adding %1% + %2%", n, vTmp.integer()).atPos(i_pos).debugThrow();
}
} else if (vTmp.type() == nFloat) {
// Upgrade the type from int to float;
firstType = nFloat;
nf = n;
nf = n.value;
nf += vTmp.fpoint();
} else
state.error<EvalError>("cannot add %1% to an integer", showType(vTmp)).atPos(i_pos).withFrame(env, *this).debugThrow();
} else if (firstType == nFloat) {
if (vTmp.type() == nInt) {
nf += vTmp.integer();
nf += vTmp.integer().value;
} else if (vTmp.type() == nFloat) {
nf += vTmp.fpoint();
} else
@@ -2126,7 +2235,7 @@ void EvalState::forceValueDeep(Value & v)
try {
// If the value is a thunk, we're evaling. Otherwise no trace necessary.
auto dts = debugRepl && i.value->isThunk()
? makeDebugTraceStacker(*this, *i.value->payload.thunk.expr, *i.value->payload.thunk.env, i.pos,
? makeDebugTraceStacker(*this, *i.value->payload.thunk.expr, *i.value->payload.thunk.env, positions[i.pos],
"while evaluating the attribute '%1%'", symbols[i.name])
: nullptr;
@@ -2172,7 +2281,7 @@ NixFloat EvalState::forceFloat(Value & v, const PosIdx pos, std::string_view err
try {
forceValue(v, pos);
if (v.type() == nInt)
return v.integer();
return v.integer().value;
else if (v.type() != nFloat)
error<TypeError>(
"expected a float but found %1%: %2%",
@@ -2359,7 +2468,7 @@ BackedStringView EvalState::coerceToString(
shell scripting convenience, just like `null'. */
if (v.type() == nBool && v.boolean()) return "1";
if (v.type() == nBool && !v.boolean()) return "";
if (v.type() == nInt) return std::to_string(v.integer());
if (v.type() == nInt) return std::to_string(v.integer().value);
if (v.type() == nFloat) return std::to_string(v.fpoint());
if (v.type() == nNull) return "";
@@ -2404,7 +2513,7 @@ StorePath EvalState::copyPathToStore(NixStringContext & context, const SourcePat
: [&]() {
auto dstPath = fetchToStore(
*store,
path.resolveSymlinks(SymlinkResolution::Ancestors),
path.resolveSymlinks(),
settings.readOnlyMode ? FetchMode::DryRun : FetchMode::Copy,
path.baseName(),
ContentAddressMethod::Raw::NixArchive,
@@ -2742,9 +2851,9 @@ bool EvalState::eqValues(Value & v1, Value & v2, const PosIdx pos, std::string_v
// Special case type-compatibility between float and int
if (v1.type() == nInt && v2.type() == nFloat)
return v1.integer() == v2.fpoint();
return v1.integer().value == v2.fpoint();
if (v1.type() == nFloat && v2.type() == nInt)
return v1.fpoint() == v2.integer();
return v1.fpoint() == v2.integer().value;
// All other types are not compatible with each other.
if (v1.type() != v2.type()) return false;
@@ -2914,6 +3023,26 @@ void EvalState::printStatistics()
};
topObj["nrOpUpdates"] = nrOpUpdates;
topObj["nrOpUpdateValuesCopied"] = nrOpUpdateValuesCopied;
topObj["nrOpMultiUpdate"] = {};
for (auto [k, n] : nrMultiConcats)
topObj["nrsKWayUpdate"][fmt("%05i", k)] = n;
topObj["nrMultiConcatCapacityPercent"] = {};
for (auto [k, n] : nrMultiConcatCapacityPercent)
topObj["nrMultiConcatCapacityPercent"][fmt("%03i", k)] = n;
topObj["nrMultiConcatCapacityCumulativePercent"] = {};
{
auto c = 0;
for (auto [k, n] : nrMultiConcatCapacityPercent) {
c += n;
topObj["nrMultiConcatCapacityCumulativePercent"][fmt("%03i", k)] = c;
}
}
topObj["nrMultiConcatCapacityTotalWaste"] = nrMultiConcatCapacityTotalWaste;
#if HAVE_BOEHMGC
topObj["nrMultiConcatCapacityTotalWastePercent"] = nrMultiConcatCapacityTotalWaste * sizeof(Attr) * 100.0 / totalBytes;
#endif
topObj["nrThunks"] = nrThunks;
topObj["nrAvoided"] = nrAvoided;
topObj["nrLookups"] = nrLookups;
@@ -3088,11 +3217,8 @@ std::optional<std::string> EvalState::resolveLookupPathPath(const LookupPath::Pa
auto i = lookupPathResolved.find(value);
if (i != lookupPathResolved.end()) return i->second;
auto finish = [&](std::optional<std::string> res) {
if (res)
debug("resolved search path element '%s' to '%s'", value, *res);
else
debug("failed to resolve search path element '%s'", value);
auto finish = [&](std::string res) {
debug("resolved search path element '%s' to '%s'", value, res);
lookupPathResolved.emplace(value, res);
return res;
};
@@ -3100,9 +3226,7 @@ std::optional<std::string> EvalState::resolveLookupPathPath(const LookupPath::Pa
if (EvalSettings::isPseudoUrl(value)) {
try {
auto accessor = fetchers::downloadTarball(
store,
fetchSettings,
EvalSettings::resolvePseudoUrl(value));
EvalSettings::resolvePseudoUrl(value)).accessor;
auto storePath = fetchToStore(*store, SourcePath(accessor), FetchMode::Copy);
return finish(store->toRealPath(storePath));
} catch (Error & e) {
@@ -3130,7 +3254,10 @@ std::optional<std::string> EvalState::resolveLookupPathPath(const LookupPath::Pa
allowPath(path);
if (store->isInStore(path)) {
try {
allowClosure(store->toStorePath(path).first);
StorePathSet closure;
store->computeFSClosure(store->toStorePath(path).first, closure);
for (auto & p : closure)
allowPath(p);
} catch (InvalidPath &) { }
}
}
@@ -3144,7 +3271,9 @@ std::optional<std::string> EvalState::resolveLookupPathPath(const LookupPath::Pa
}
}
return finish(std::nullopt);
debug("failed to resolve search path element '%s'", value);
return std::nullopt;
}
+4 -32
View File
@@ -155,28 +155,11 @@ struct RegexCache;
std::shared_ptr<RegexCache> makeRegexCache();
struct DebugTrace {
/* WARNING: Converting PosIdx -> Pos should be done with extra care. This is
due to the fact that operator[] of PosTable is incredibly expensive. */
std::variant<Pos, PosIdx> pos;
std::shared_ptr<Pos> pos;
const Expr & expr;
const Env & env;
HintFmt hint;
bool isError;
Pos getPos(const PosTable & table) const
{
return std::visit(
overloaded{
[&](PosIdx idx) {
// Prefer direct pos, but if noPos then try the expr.
if (!idx)
idx = expr.getPos();
return table[idx];
},
[&](Pos pos) { return pos; },
},
pos);
}
};
class EvalState : public std::enable_shared_from_this<EvalState>
@@ -409,11 +392,6 @@ public:
*/
void allowPath(const StorePath & storePath);
/**
* Allow access to the closure of a store path.
*/
void allowClosure(const StorePath & storePath);
/**
* Allow access to a store path and return it as a string.
*/
@@ -803,15 +781,6 @@ public:
*/
[[nodiscard]] StringMap realiseContext(const NixStringContext & context, StorePathSet * maybePaths = nullptr, bool isIFD = true);
/**
* Realise the given string with context, and return the string with outputs instead of downstream output placeholders.
* @param[in] str the string to realise
* @param[out] paths all referenced store paths will be added to this set
* @return the realised string
* @throw EvalError if the value is not a string, path or derivation (see `coerceToString`)
*/
std::string realiseString(Value & str, StorePathSet * storePathsOutMaybe, bool isIFD = true, const PosIdx pos = noPos);
/* Call the binary path filter predicate used builtins.path etc. */
bool callPathFilter(
Value * filterFun,
@@ -850,6 +819,9 @@ private:
unsigned long nrOpUpdates = 0;
unsigned long nrOpUpdateValuesCopied = 0;
unsigned long nrListConcats = 0;
std::map<int, int> nrMultiConcats;
std::map<int, int> nrMultiConcatCapacityPercent;
unsigned long nrMultiConcatCapacityTotalWaste = 0;
unsigned long nrPrimOpCalls = 0;
unsigned long nrFunctionCalls = 0;
+30 -62
View File
@@ -1,72 +1,40 @@
{
system ? "", # obsolete
url,
hash ? "", # an SRI hash
{ system ? "" # obsolete
, url
, hash ? "" # an SRI hash
# Legacy hash specification
md5 ? "",
sha1 ? "",
sha256 ? "",
sha512 ? "",
outputHash ?
if hash != "" then
hash
else if sha512 != "" then
sha512
else if sha1 != "" then
sha1
else if md5 != "" then
md5
else
sha256,
outputHashAlgo ?
if hash != "" then
""
else if sha512 != "" then
"sha512"
else if sha1 != "" then
"sha1"
else if md5 != "" then
"md5"
else
"sha256",
# Legacy hash specification
, md5 ? "", sha1 ? "", sha256 ? "", sha512 ? ""
, outputHash ?
if hash != "" then hash else if sha512 != "" then sha512 else if sha1 != "" then sha1 else if md5 != "" then md5 else sha256
, outputHashAlgo ?
if hash != "" then "" else if sha512 != "" then "sha512" else if sha1 != "" then "sha1" else if md5 != "" then "md5" else "sha256"
executable ? false,
unpack ? false,
name ? baseNameOf (toString url),
impure ? false,
, executable ? false
, unpack ? false
, name ? baseNameOf (toString url)
, impure ? false
}:
derivation (
{
builder = "builtin:fetchurl";
derivation ({
builder = "builtin:fetchurl";
# New-style output content requirements.
outputHashMode = if unpack || executable then "recursive" else "flat";
# New-style output content requirements.
outputHashMode = if unpack || executable then "recursive" else "flat";
inherit
name
url
executable
unpack
;
inherit name url executable unpack;
system = "builtin";
system = "builtin";
# No need to double the amount of network traffic
preferLocalBuild = true;
# No need to double the amount of network traffic
preferLocalBuild = true;
# This attribute does nothing; it's here to avoid changing evaluation results.
impureEnvVars = [
"http_proxy"
"https_proxy"
"ftp_proxy"
"all_proxy"
"no_proxy"
];
# This attribute does nothing; it's here to avoid changing evaluation results.
impureEnvVars = [
"http_proxy" "https_proxy" "ftp_proxy" "all_proxy" "no_proxy"
];
# To make "nix-prefetch-url" work.
urls = [ url ];
}
// (if impure then { __impure = true; } else { inherit outputHashAlgo outputHash; })
)
# To make "nix-prefetch-url" work.
urls = [ url ];
} // (if impure
then { __impure = true; }
else { inherit outputHashAlgo outputHash; }))
+2 -2
View File
@@ -246,8 +246,8 @@ NixInt PackageInfo::queryMetaInt(const std::string & name, NixInt def)
if (v->type() == nString) {
/* Backwards compatibility with before we had support for
integer meta fields. */
if (auto n = string2Int<NixInt>(v->c_str()))
return *n;
if (auto n = string2Int<NixInt::Inner>(v->c_str()))
return NixInt{*n};
}
return def;
}
+15 -21
View File
@@ -1,27 +1,21 @@
attrs@{
drvPath,
outputs,
name,
...
}:
attrs @ { drvPath, outputs, name, ... }:
let
commonAttrs = (builtins.listToAttrs outputsList) // {
all = map (x: x.value) outputsList;
inherit drvPath name;
type = "derivation";
};
outputToAttrListElement = outputName: {
name = outputName;
value = commonAttrs // {
outPath = builtins.getAttr outputName attrs;
inherit outputName;
commonAttrs = (builtins.listToAttrs outputsList) //
{ all = map (x: x.value) outputsList;
inherit drvPath name;
type = "derivation";
};
};
outputToAttrListElement = outputName:
{ name = outputName;
value = commonAttrs // {
outPath = builtins.getAttr outputName attrs;
inherit outputName;
};
};
outputsList = map outputToAttrListElement outputs;
in
(builtins.head outputsList).value
in (builtins.head outputsList).value
+6 -1
View File
@@ -2,6 +2,7 @@
#include "value.hh"
#include "eval.hh"
#include <limits>
#include <variant>
#include <nlohmann/json.hpp>
@@ -101,8 +102,12 @@ public:
return true;
}
bool number_unsigned(number_unsigned_t val) override
bool number_unsigned(number_unsigned_t val_) override
{
if (val_ > std::numeric_limits<NixInt::Inner>::max()) {
throw Error("unsigned json number %1% outside of Nix integer range", val_);
}
NixInt::Inner val = val_;
rs->value(state).mkInt(val);
rs->add();
return true;
+1 -1
View File
@@ -138,7 +138,7 @@ or { return OR_KW; }
{INT} { errno = 0;
std::optional<int64_t> numMay = string2Int<int64_t>(yytext);
if (numMay.has_value()) {
yylval->n = *numMay;
yylval->n = NixInt{*numMay};
} else {
throw ParseError(ErrorInfo{
.msg = HintFmt("invalid integer '%1%'", yytext),
+2
View File
@@ -175,6 +175,8 @@ headers = [config_h] + files(
# internal: 'lexer-helpers.hh',
'nixexpr.hh',
'parser-state.hh',
'pos-idx.hh',
'pos-table.hh',
'primops.hh',
'print-ambiguous.hh',
'print-options.hh',
+35
View File
@@ -601,6 +601,41 @@ void ExprLambda::setDocComment(DocComment docComment) {
}
};
/* Position table. */
Pos PosTable::operator[](PosIdx p) const
{
auto origin = resolve(p);
if (!origin)
return {};
const auto offset = origin->offsetOf(p);
Pos result{0, 0, origin->origin};
auto lines = this->lines.lock();
auto linesForInput = (*lines)[origin->offset];
if (linesForInput.empty()) {
auto source = result.getSource().value_or("");
const char * begin = source.data();
for (Pos::LinesIterator it(source), end; it != end; it++)
linesForInput.push_back(it->data() - begin);
if (linesForInput.empty())
linesForInput.push_back(0);
}
// as above: the first line starts at byte 0 and is always present
auto lineStartOffset = std::prev(
std::upper_bound(linesForInput.begin(), linesForInput.end(), offset));
result.line = 1 + (lineStartOffset - linesForInput.begin());
result.column = 1 + (offset - *lineStartOffset);
return result;
}
/* Symbol table. */
size_t SymbolTable::totalSize() const
+57 -1
View File
@@ -4,6 +4,7 @@
#include <map>
#include <vector>
#include "gc-small-vector.hh"
#include "value.hh"
#include "symbol-table.hh"
#include "eval-error.hh"
@@ -74,6 +75,7 @@ typedef std::vector<AttrName> AttrPath;
std::string showAttrPath(const SymbolTable & symbols, const AttrPath & attrPath);
using UpdateQueue = SmallVector<const Bindings *, 64>;
/* Abstract syntax of Nix expressions. */
@@ -91,8 +93,25 @@ struct Expr
virtual ~Expr() { };
virtual void show(const SymbolTable & symbols, std::ostream & str) const;
virtual void bindVars(EvalState & es, const std::shared_ptr<const StaticEnv> & env);
/** Normal evaluation, implemented directly by all subclasses. */
virtual void eval(EvalState & state, Env & env, Value & v);
/**
* Create a thunk for the delayed computation of the given expression
* in the given environment. But if the expression is a variable,
* then look it up right away. This significantly reduces the number
* of thunks allocated.
*/
virtual Value * maybeThunk(EvalState & state, Env & env);
/**
* Only called when performing an attrset update: `//` or similar.
* Instead of writing to a Value &, this function writes to an UpdateQueue.
* This allows the expression to perform multiple updates in a delayed manner, gathering up all the updates before applying them.
*/
virtual void evalForUpdate(EvalState & state, Env & env, UpdateQueue & q);
virtual void setName(Symbol name);
virtual void setDocComment(DocComment docComment) { };
virtual PosIdx getPos() const { return noPos; }
@@ -107,6 +126,7 @@ struct ExprInt : Expr
{
Value v;
ExprInt(NixInt n) { v.mkInt(n); };
ExprInt(NixInt::Inner n) { v.mkInt(n); };
Value * maybeThunk(EvalState & state, Env & env) override;
COMMON_METHODS
};
@@ -429,9 +449,45 @@ MakeBinOp(ExprOpNEq, "!=")
MakeBinOp(ExprOpAnd, "&&")
MakeBinOp(ExprOpOr, "||")
MakeBinOp(ExprOpImpl, "->")
MakeBinOp(ExprOpUpdate, "//")
MakeBinOp(ExprOpConcatLists, "++")
struct ExprOpUpdate : Expr
{
PosIdx pos;
Expr *e1, *e2;
ExprOpUpdate(Expr * e1, Expr * e2)
: e1(e1)
, e2(e2){};
ExprOpUpdate(const PosIdx & pos, Expr * e1, Expr * e2)
: pos(pos)
, e1(e1)
, e2(e2){};
void show(const SymbolTable & symbols, std ::ostream & str) const override
{
str << "(";
e1->show(symbols, str);
str << " "
"//"
" ";
e2->show(symbols, str);
str << ")";
}
void bindVars(EvalState & es, const std ::shared_ptr<const StaticEnv> & env) override
{
e1->bindVars(es, env);
e2->bindVars(es, env);
}
void eval(EvalState & state, Env & env, Value & v) override;
PosIdx getPos() const override
{
return pos;
}
virtual void evalForUpdate(EvalState & state, Env & env, UpdateQueue & q) override;
};
struct ExprConcatStrings : Expr
{
PosIdx pos;
+38 -46
View File
@@ -1,38 +1,37 @@
{
lib,
stdenv,
mkMesonDerivation,
releaseTools,
{ lib
, stdenv
, mkMesonDerivation
, releaseTools
meson,
ninja,
pkg-config,
bison,
flex,
cmake, # for resolving toml11 dep
, meson
, ninja
, pkg-config
, bison
, flex
, cmake # for resolving toml11 dep
nix-util,
nix-store,
nix-fetchers,
boost,
boehmgc,
nlohmann_json,
toml11,
, nix-util
, nix-store
, nix-fetchers
, boost
, boehmgc
, nlohmann_json
, toml11
# Configuration Options
# Configuration Options
version,
, version
# Whether to use garbage collection for the Nix language evaluator.
#
# If it is disabled, we just leak memory, but this is not as bad as it
# sounds so long as evaluation just takes places within short-lived
# processes. (When the process exits, the memory is reclaimed; it is
# only leaked *within* the process.)
#
# Temporarily disabled on Windows because the `GC_throw_bad_alloc`
# symbol is missing during linking.
enableGC ? !stdenv.hostPlatform.isWindows,
# Whether to use garbage collection for the Nix language evaluator.
#
# If it is disabled, we just leak memory, but this is not as bad as it
# sounds so long as evaluation just takes places within short-lived
# processes. (When the process exits, the memory is reclaimed; it is
# only leaked *within* the process.)
#
# Temporarily disabled on Windows because the `GC_throw_bad_alloc`
# symbol is missing during linking.
, enableGC ? !stdenv.hostPlatform.isWindows
}:
let
@@ -59,10 +58,7 @@ mkMesonDerivation (finalAttrs: {
(fileset.fileFilter (file: file.hasExt "nix") ./.)
];
outputs = [
"out"
"dev"
];
outputs = [ "out" "dev" ];
nativeBuildInputs = [
meson
@@ -97,18 +93,14 @@ mkMesonDerivation (finalAttrs: {
(lib.mesonEnable "gc" enableGC)
];
env =
{
# Needed for Meson to find Boost.
# https://github.com/NixOS/nixpkgs/issues/86131.
BOOST_INCLUDEDIR = "${lib.getDev boost}/include";
BOOST_LIBRARYDIR = "${lib.getLib boost}/lib";
}
// lib.optionalAttrs
(stdenv.isLinux && !(stdenv.hostPlatform.isStatic && stdenv.system == "aarch64-linux"))
{
LDFLAGS = "-fuse-ld=gold";
};
env = {
# Needed for Meson to find Boost.
# https://github.com/NixOS/nixpkgs/issues/86131.
BOOST_INCLUDEDIR = "${lib.getDev boost}/include";
BOOST_LIBRARYDIR = "${lib.getLib boost}/lib";
} // lib.optionalAttrs (stdenv.isLinux && !(stdenv.hostPlatform.isStatic && stdenv.system == "aarch64-linux")) {
LDFLAGS = "-fuse-ld=gold";
};
separateDebugInfo = !stdenv.hostPlatform.isStatic;
+45 -64
View File
@@ -20,6 +20,7 @@ struct StringToken
operator std::string_view() const { return {p, l}; }
};
// This type must be trivially copyable; see YYLTYPE_IS_TRIVIAL in parser.y.
struct ParserLocation
{
int beginOffset;
@@ -87,7 +88,6 @@ struct ParserState
void dupAttr(const AttrPath & attrPath, const PosIdx pos, const PosIdx prevPos);
void dupAttr(Symbol attr, const PosIdx pos, const PosIdx prevPos);
void addAttr(ExprAttrs * attrs, AttrPath && attrPath, const ParserLocation & loc, Expr * e, const ParserLocation & exprLoc);
void addAttr(ExprAttrs * attrs, AttrPath & attrPath, const Symbol & symbol, ExprAttrs::AttrDef && def);
Formals * validateFormals(Formals * formals, PosIdx pos = noPos, Symbol arg = {});
Expr * stripIndentation(const PosIdx pos,
std::vector<std::pair<PosIdx, std::variant<Expr *, StringToken>>> && es);
@@ -120,29 +120,64 @@ inline void ParserState::addAttr(ExprAttrs * attrs, AttrPath && attrPath, const
// Checking attrPath validity.
// ===========================
for (i = attrPath.begin(); i + 1 < attrPath.end(); i++) {
ExprAttrs * nested;
if (i->symbol) {
ExprAttrs::AttrDefs::iterator j = attrs->attrs.find(i->symbol);
if (j != attrs->attrs.end()) {
nested = dynamic_cast<ExprAttrs *>(j->second.e);
if (!nested) {
attrPath.erase(i + 1, attrPath.end());
if (j->second.kind != ExprAttrs::AttrDef::Kind::Inherited) {
ExprAttrs * attrs2 = dynamic_cast<ExprAttrs *>(j->second.e);
if (!attrs2) dupAttr(attrPath, pos, j->second.pos);
attrs = attrs2;
} else
dupAttr(attrPath, pos, j->second.pos);
}
} else {
nested = new ExprAttrs;
ExprAttrs * nested = new ExprAttrs;
attrs->attrs[i->symbol] = ExprAttrs::AttrDef(nested, pos);
attrs = nested;
}
} else {
nested = new ExprAttrs;
ExprAttrs *nested = new ExprAttrs;
attrs->dynamicAttrs.push_back(ExprAttrs::DynamicAttrDef(i->expr, nested, pos));
attrs = nested;
}
attrs = nested;
}
// Expr insertion.
// ==========================
if (i->symbol) {
addAttr(attrs, attrPath, i->symbol, ExprAttrs::AttrDef(e, pos));
ExprAttrs::AttrDefs::iterator j = attrs->attrs.find(i->symbol);
if (j != attrs->attrs.end()) {
// This attr path is already defined. However, if both
// e and the expr pointed by the attr path are two attribute sets,
// we want to merge them.
// Otherwise, throw an error.
auto ae = dynamic_cast<ExprAttrs *>(e);
auto jAttrs = dynamic_cast<ExprAttrs *>(j->second.e);
if (jAttrs && ae) {
if (ae->inheritFromExprs && !jAttrs->inheritFromExprs)
jAttrs->inheritFromExprs = std::make_unique<std::vector<Expr *>>();
for (auto & ad : ae->attrs) {
auto j2 = jAttrs->attrs.find(ad.first);
if (j2 != jAttrs->attrs.end()) // Attr already defined in iAttrs, error.
dupAttr(ad.first, j2->second.pos, ad.second.pos);
jAttrs->attrs.emplace(ad.first, ad.second);
if (ad.second.kind == ExprAttrs::AttrDef::Kind::InheritedFrom) {
auto & sel = dynamic_cast<ExprSelect &>(*ad.second.e);
auto & from = dynamic_cast<ExprInheritFrom &>(*sel.e);
from.displ += jAttrs->inheritFromExprs->size();
}
}
jAttrs->dynamicAttrs.insert(jAttrs->dynamicAttrs.end(), ae->dynamicAttrs.begin(), ae->dynamicAttrs.end());
if (ae->inheritFromExprs) {
jAttrs->inheritFromExprs->insert(jAttrs->inheritFromExprs->end(),
ae->inheritFromExprs->begin(), ae->inheritFromExprs->end());
}
} else {
dupAttr(attrPath, pos, j->second.pos);
}
} else {
// This attr path is not defined. Let's create it.
attrs->attrs.emplace(i->symbol, ExprAttrs::AttrDef(e, pos));
e->setName(i->symbol);
}
} else {
attrs->dynamicAttrs.push_back(ExprAttrs::DynamicAttrDef(i->expr, e, pos));
}
@@ -154,60 +189,6 @@ inline void ParserState::addAttr(ExprAttrs * attrs, AttrPath && attrPath, const
}
}
/**
* Precondition: attrPath is used for error messages and should already contain
* symbol as its last element.
*/
inline void ParserState::addAttr(ExprAttrs * attrs, AttrPath & attrPath, const Symbol & symbol, ExprAttrs::AttrDef && def)
{
ExprAttrs::AttrDefs::iterator j = attrs->attrs.find(symbol);
if (j != attrs->attrs.end()) {
// This attr path is already defined. However, if both
// e and the expr pointed by the attr path are two attribute sets,
// we want to merge them.
// Otherwise, throw an error.
auto ae = dynamic_cast<ExprAttrs *>(def.e);
auto jAttrs = dynamic_cast<ExprAttrs *>(j->second.e);
// N.B. In a world in which we are less bound by our past mistakes, we
// would also test that jAttrs and ae are not recursive. The effect of
// not doing so is that any `rec` marker on ae is discarded, and any
// `rec` marker on jAttrs will apply to the attributes in ae.
// See https://github.com/NixOS/nix/issues/9020.
if (jAttrs && ae) {
if (ae->inheritFromExprs && !jAttrs->inheritFromExprs)
jAttrs->inheritFromExprs = std::make_unique<std::vector<Expr *>>();
for (auto & ad : ae->attrs) {
if (ad.second.kind == ExprAttrs::AttrDef::Kind::InheritedFrom) {
auto & sel = dynamic_cast<ExprSelect &>(*ad.second.e);
auto & from = dynamic_cast<ExprInheritFrom &>(*sel.e);
from.displ += jAttrs->inheritFromExprs->size();
}
attrPath.emplace_back(AttrName(ad.first));
addAttr(jAttrs, attrPath, ad.first, std::move(ad.second));
attrPath.pop_back();
}
ae->attrs.clear();
jAttrs->dynamicAttrs.insert(jAttrs->dynamicAttrs.end(),
std::make_move_iterator(ae->dynamicAttrs.begin()),
std::make_move_iterator(ae->dynamicAttrs.end()));
ae->dynamicAttrs.clear();
if (ae->inheritFromExprs) {
jAttrs->inheritFromExprs->insert(jAttrs->inheritFromExprs->end(),
std::make_move_iterator(ae->inheritFromExprs->begin()),
std::make_move_iterator(ae->inheritFromExprs->end()));
ae->inheritFromExprs = nullptr;
}
} else {
dupAttr(attrPath, def.pos, j->second.pos);
}
} else {
// This attr path is not defined. Let's create it.
attrs->attrs.emplace(symbol, def);
def.e->setName(symbol);
}
}
inline Formals * ParserState::validateFormals(Formals * formals, PosIdx pos, Symbol arg)
{
std::sort(formals->formals.begin(), formals->formals.end(),
Loaded 100 of 574 files, more files were not shown because too many files have changed in this diff. Show more