Compare commits

...
Author SHA1 Message Date
Eelco Dolstra 2d52180455 printError -> printTalkative 2018-03-02 17:31:11 +01:00
Eelco Dolstra 64ea7cc437 Add an 'include' keyword
Inclusion is like importing, except that all variables that in scope
at the 'include' site are in scope of the included file.

Thus, if ./foo.nix contains 'x', then

  let x = 1; in include ./foo.nix

evaluates to 1.

(Note that 'include' has to be a keyword, not a builtin function,
because inclusion takes place within a specific scope. I.e. 'include
./foo.nix' would not be equivalent to 'let f = include; in ... f
./foo.nix ...'.)
2018-03-02 17:12:54 +01:00
Eelco Dolstra 939cf4cceb Fix error message 2018-03-02 14:32:00 +01:00
Shea Levy 4a000cbb39 Merge pull request #1927 from dtzWill/fix/fetchgit-HEAD
fetchGit: use "HEAD" as default ref instead of "master"
2018-02-28 18:44:57 -05:00
Will Dietz e89d02bf03 fetchGit: use "HEAD" as default ref 2018-02-28 16:34:34 -06:00
Daiderd Jordan 05cb8e5c5a launchd: enable keepalive for the nix-daemon service
Without this the daemon won't be restarted if the process ever dies, for
example when sending a SIGHUP to reload nix.conf.
2018-02-28 19:58:48 +01:00
Shea Levy 14ca85688c Actually fix nixDataDir in non-canonical path 2018-02-28 06:19:40 -05:00
Eelco Dolstra 8a5da93841 Merge pull request #1912 from dezgeg/replacestrings-take-2
libexpr: Fix prim_replaceStrings() to work on an empty source string, take 2
2018-02-27 12:53:19 +01:00
Tuomas Tynkkynen 77e9e1ed91 libexpr: Fix prim_replaceStrings() to work on an empty source string
Otherwise, running e.g.

nix-instantiate --eval -E --strict 'builtins.replaceStrings [""] ["X"] "abc"'

would just hang in an infinite loop.

Found by afl-fuzz.

First attempt of this was reverted in e2d71bd186 because it caused
another infinite loop, which is fixed now and a test added.
2018-02-26 19:49:13 +02:00
Eelco Dolstra 24ec750003 nix run: Fix segfault on macOS
Note that clearenv() is not available on macOS.

Fixes #1907.
2018-02-26 18:29:40 +01:00
Graham Christensen 9432f3fb7d Merge pull request #1901 from veprbl/patch-5
Fix a small typo in the release notes
2018-02-24 10:26:14 -05:00
Graham Christensen ab435463d0 Merge pull request #1903 from primeos/release-notes-2.0-url-fix
doc: Fix a URL in the release notes for Nix 2.0
2018-02-24 10:19:39 -05:00
Michael Weiss bd94e63853 doc: Fix a URL in the release notes for Nix 2.0 2018-02-24 16:07:10 +01:00
Dmitry Kalinkin d9d8a84a96 Fix a small typo in the release notes 2018-02-23 23:45:42 -05:00
Eelco Dolstra 64e486ab63 Fix downloadCached() with a chroot store
E.g.

  nix run --store ~/my-nix -f channel:nixos-17.03 hello -c hello

This problem was mentioned in #1897.
2018-02-23 13:00:42 +01:00
Eelco Dolstra 39b4177500 Fix example in release notes 2018-02-22 17:42:06 +01:00
Eelco Dolstra 2691d51a33 Doh 2018-02-22 17:14:35 +01:00
Eelco Dolstra b6dec2f3ca Release script tweaks 2018-02-22 17:11:17 +01:00
Eelco Dolstra 179b896acb Merge branch 'data-dir-non-canon' of https://github.com/shlevy/nix 2018-02-22 14:20:43 +01:00
Eelco Dolstra e8d53bfdc9 Revert "Enable sandbox builds on Linux by default"
This reverts commit ddc58e7896.

https://hydra.nixos.org/eval/1435322
2018-02-22 14:20:07 +01:00
Shea Levy ddbcd01c83 Fix restricted mode when installing in non-canonical data dir 2018-02-22 07:18:14 -05:00
Eelco Dolstra eaa52c34b4 Set release date 2018-02-22 12:44:46 +01:00
Eelco Dolstra ddc58e7896 Enable sandbox builds on Linux by default
The overhead of sandbox builds is a problem on NixOS (since building a
NixOS configuration involves a lot of small derivations) but not for
typical non-NixOS use cases. So outside of NixOS we can enable it.

Issue #179.
2018-02-22 12:27:25 +01:00
Eelco Dolstra 6964131cd7 Merge branch 'fix/sandbox-shell-features' of https://github.com/dtzWill/nix 2018-02-22 12:20:13 +01:00
Eelco Dolstra de4c03d201 Merge branch 'fix/dry-run-partially' of https://github.com/dtzWill/nix 2018-02-22 12:18:20 +01:00
Eelco Dolstra 88c90d5e6d Manual: Put configuration options in sorted order 2018-02-21 18:08:47 +01:00
Eelco Dolstra 0d54671b7b Manual: Update chapter on remote builds
Alos add a command "nix ping-store" to make it easier to see if Nix
can connect to a remote builder (e.g. 'nix ping-store --store
ssh://mac').
2018-02-21 16:24:26 +01:00
Eelco Dolstra e2d71bd186 Revert "libexpr: Fix prim_replaceStrings() to work on an empty source string"
This reverts commit 4ea9707591.

It causes an infinite loop in Nixpkgs evaluation,
e.g. "nix-instantiate -A hello" hung.

PR #1886.
2018-02-21 15:35:28 +01:00
Shea Levy a6c497f526 Merge branch 'nix-copy' of git://github.com/Mic92/nix-1 2018-02-20 21:05:17 -05:00
Jörg Thalheim fa7fd76c5e nix-copy: fix examples
maybe a left-over from nix-store -r ?
2018-02-21 01:13:46 +00:00
Shea Levy 7c377dc5cc Merge remote-tracking branch 'dezgeg/afl-fixes' 2018-02-20 16:32:48 -05:00
Eelco Dolstra 4e44025ac5 Release notes: Add contributors 2018-02-20 15:20:14 +01:00
Eelco Dolstra cea4fb3a31 Fix evaluation of binaryTarball.aarch64-linux 2018-02-20 12:33:32 +01:00
Tuomas Tynkkynen 546f98dace libutil: Fix invalid assert on decoding base64 hashes
The assertion is broken because there is no one-to-one mapping from
length of a base64 string to the length of the output.

E.g.

"1q69lz7Empb06nzfkj651413n9icx0njmyr3xzq1j9q=" results in a 32-byte output.
"1q69lz7Empb06nzfkj651413n9icx0njmyr3xzq1j9qy" results in a 33-byte output.

To reproduce, evaluate:

builtins.derivationStrict {
    name = "0";
    builder = "0";
    system = "0";
    outputHashAlgo = "sha256";
    outputHash = "1q69lz7Empb06nzfkj651413n9icx0njmyr3xzq1j9qy";
}

Found by afl-fuzz.
2018-02-19 23:20:26 +02:00
Tuomas Tynkkynen 4ea9707591 libexpr: Fix prim_replaceStrings() to work on an empty source string
Otherwise, running e.g.

nix-instantiate --eval -E --strict 'builtins.replaceStrings [""] ["X"] "abc"'

would just hang in an infinite loop.

Found by afl-fuzz.
2018-02-19 23:20:26 +02:00
Tuomas Tynkkynen 1d0e42879f libutil: Fix infinite loop in filterANSIEscapes on '\r'
E.g. nix-instantiate --eval -E 'abort "\r"' hangs.

Found by afl-fuzz.
2018-02-19 23:20:26 +02:00
Tuomas Tynkkynen 056d28a601 libexpr: Don't create lots of temporary strings in Bindings::lexicographicOrder
Avoids ~180,000 string temporaries created when evaluating a headless
NixOS system.
2018-02-19 22:47:25 +02:00
Eelco Dolstra d4e93532e2 Fix incorrect (and unnecessary) format string
https://hydra.nixos.org/eval/1434547#tabs-now-fail
2018-02-19 20:46:39 +01:00
Eelco Dolstra d7fdfe322b Remove macOS multi-user instructions
This is already handled by the installer.
2018-02-19 20:40:25 +01:00
Eelco Dolstra 70eb64147e Update release notes
Also add some examples to nix --help.
2018-02-19 20:38:53 +01:00
Will Dietz a6c0b773b7 configure.ac: define HAVE_SECCOMP macro when using seccomp, fix build/tests
Happily the failing tests should prevent anyone from using such a Nix
in situations where they expect sandboxing to be on,
which would otherwise be a risk.
2018-02-19 12:13:51 -06:00
Eelco Dolstra 623fcb071e Merge pull request #1882 from shlevy/no-seccomp-no-filterSyscalls
Don't silently succeed seccomp setup when !HAVE_SECCOMP.
2018-02-19 17:39:46 +01:00
Shea Levy e1eb63a586 Merge branch 'perf-fixes' of git://github.com/dezgeg/nix 2018-02-19 10:11:52 -05:00
Shea Levy e59a8a63e1 Don't silently succeed seccomp setup when !HAVE_SECCOMP.
Running Nix with build users without seccomp on Linux is dangerous,
and administrators should very explicitly opt-in to it.
2018-02-19 09:56:24 -05:00
Eelco Dolstra ed73d40c3b Config::handleUnknownSettings(): Remove unused 'fatal' argument 2018-02-19 14:00:34 +01:00
Eelco Dolstra 75a1d96cfd Merge branch 'register-settings' of https://github.com/shlevy/nix 2018-02-19 13:58:34 +01:00
Eelco Dolstra 7fe5910bf8 Merge pull request #1857 from dtzWill/fix/check-for-lzma-mt
configure.ac: check if lzma has MT support, fix deb build/etc.
2018-02-19 13:06:14 +01:00
Shea Levy 690ac7c90b configure: Add a flag to disable seccomp.
This is needed for new arches where libseccomp support doesn't exist
yet.

Fixes #1878.
2018-02-18 02:35:01 -05:00
Tuomas Tynkkynen 37264ed0ad libexpr: Avoid an unnecessary string copy in prim_derivationStrict 2018-02-17 16:54:21 +02:00
Tuomas Tynkkynen 66eeff3345 libexpr: Remove unnecessary drvName assignment in prim_derivationStrict
drvName is already assigned to the same value right at the start of the
function.
2018-02-17 16:54:21 +02:00
Tuomas Tynkkynen 7e0360504d libexpr: Optimize prim_derivationStrict by using more symbol comparisons 2018-02-17 16:54:21 +02:00
Tuomas Tynkkynen 0845cdf944 libexpr: Rely on Boehm returning zeroed memory in EvalState::allocEnv()
Boehm guarantees that memory returned by GC_malloc() is zeroed, so take
advantage of that.
2018-02-17 16:54:21 +02:00
Tuomas Tynkkynen b8bed7da14 libexpr: Optimize prim_attrNames a bit
Instead of having lexicographicOrder() create a temporary sorted array
of Attr*:s and copying attr names from that, copy the attr names
first and then sort that.
2018-02-17 16:54:21 +02:00
Tuomas Tynkkynen f67a7007a2 libexpr: Pre-reserve space in string in unescapeStr()
Avoids some malloc() traffic.
2018-02-16 04:39:43 +02:00
Eelco Dolstra 3a5a241b32 Merge pull request #1873 from matthewbauer/fix-manpath
Set MANPATH in case man path isn’t set correctly.
2018-02-15 17:37:21 +01:00
Matthew Bauer 8f186722a9 Set backup MANPATH in case man path isn’t set correctly.
Previously, this would fail at startup for non-NixOS installs:

nix-env --help

The fix for this is to just use "nixManDir" as the value for MANPATH
when spawning "man".

To test this, I’m using the following:

$ nix-build release.nix -A build
$ MANPATH= ./result/bin/nix-env --help

Fixes #1627
2018-02-15 10:20:02 -06:00
Eelco Dolstra d26b71fda6 Merge pull request #1872 from shlevy/macOS-poll-fix
monitor-fds: Fix on macOS.
2018-02-15 11:43:56 +01:00
Shea Levy ac973a6d3c monitor-fds: Fix on macOS.
Fixes #1871.
2018-02-14 18:26:37 -05:00
Eelco Dolstra 96d48318cb Merge pull request #1870 from shlevy/split-version
Add splitVersion primop.
2018-02-14 16:39:53 +01:00
Shea Levy b095c06139 Add splitVersion primop.
Fixes #1868.
2018-02-14 09:55:43 -05:00
Shea Levy de4934ab3b Allow plugins to define new settings. 2018-02-13 14:43:32 -05:00
Shea Levy 3fe9767dd3 Fix plugin tests on darwin 2018-02-13 12:49:14 -05:00
Eelco Dolstra 52c777a793 Merge pull request #1863 from shlevy/conf-includes
Allow includes from nix.conf
2018-02-13 17:33:07 +01:00
Eelco Dolstra 7253113fd2 Merge pull request #1862 from shlevy/plugin-dirs
Enable specifying directories in plugin-files.
2018-02-13 17:32:32 +01:00
Shea Levy b8739f2fb3 Enable specifying directories in plugin-files. 2018-02-13 11:25:01 -05:00
Shea Levy 6eb1040e90 Allow includes from nix.conf 2018-02-13 08:16:32 -05:00
Eelco Dolstra f471aacff2 Merge pull request #1775 from LnL7/darwin-build-users
installer: create 'enough' build users
2018-02-13 12:31:53 +01:00
Eelco Dolstra 7828dca9e8 Merge branch 'register-constant' of https://github.com/shlevy/nix 2018-02-13 12:24:48 +01:00
Eelco Dolstra 1c10a74c73 Merge branch 'plugins' of https://github.com/shlevy/nix 2018-02-13 12:15:27 +01:00
Eelco Dolstra c5cc57e962 Merge pull request #1859 from FRidh/flatten
Nix stats: flatten statistics
2018-02-13 11:11:12 +01:00
Eelco Dolstra 9bcb4d2dd9 Fix hang in build-remote 2018-02-12 22:48:55 +01:00
Eelco Dolstra 4f09ce7940 Fix 'deadlock: trying to re-acquire self-held lock'
This was caused by derivations with 'allowSubstitutes = false'. Such
derivations will be built locally. However, if there is another
SubstitionGoal that has the output of the first derivation in its
closure, then the path will be simultaneously built and substituted.

There was a check to catch this situation (via pathIsLockedByMe()),
but it no longer worked reliably because substitutions are now done in
another thread. (Thus the comment 'It can't happen between here and
the lockPaths() call below because we're not allowing multi-threading'
was no longer valid.)

The fix is to handle the path already being locked in both
SubstitutionGoal and DerivationGoal.
2018-02-12 17:06:06 +01:00
Eelco Dolstra 35fd31770c toBuildables -> build 2018-02-12 17:06:06 +01:00
Will Dietz c6209030c4 compression: make parallel sink separate class 2018-02-11 13:50:28 -06:00
Will Dietz a0bdc96726 compression: print warning if parallel requested but not supported 2018-02-11 13:03:47 -06:00
Frederik Rietdijk 60eca58533 Nix stats: flatten statistics
Flattens the list of statistics as suggested in
https://github.com/NixOS/ofborg/issues/67. This makes it easier to work
with.
2018-02-11 14:37:50 +01:00
Will Dietz 5a082ad15a configure.ac: check if lzma has MT support, fix deb build/etc. 2018-02-09 21:02:25 -06:00
Eelco Dolstra 960e9c560e nix: Ensure that the user sees errors from substituters 2018-02-09 15:06:47 +01:00
Eelco Dolstra aa02cdc33c getDefaultSubstituters(): Skip broken substituters
Fixes #1340.
2018-02-09 15:06:47 +01:00
Eelco Dolstra 3d2d207aad Merge pull request #1848 from AmineChikhaoui/parallel-xz
support multi threaded xz encoder
2018-02-09 15:03:25 +01:00
Shea Levy 081f14a169 Allow using RegisterPrimop to define constants.
This enables plugins to add new constants, as well as new primops.
2018-02-08 14:35:50 -05:00
Shea Levy 88cd2d41ac Add plugins to make Nix more extensible.
All plugins in plugin-files will be dlopened, allowing them to
statically construct instances of the various Register* types Nix
supports.
2018-02-08 12:44:37 -05:00
Eelco Dolstra ad97a21834 nix-env: Fix parsing of --system
https://hydra.nixos.org/build/68827814
2018-02-08 15:25:03 +01:00
Eelco Dolstra 444bae44ef dsa -> ed25519
DSS is disabled by default in NixOS 18.03.

https://hydra.nixos.org/build/68788560
2018-02-08 13:46:23 +01:00
AmineChikhaoui 0685a6480a Merge branch 'master' of https://github.com/NixOS/nix into parallel-xz 2018-02-08 12:24:48 +01:00
Will Dietz c7e0be1bfc build-dry: disable failing portion of test until issue is fixed 2018-02-07 15:20:54 -06:00
Will Dietz 98031b6050 nix build: Don't create output links with --dry-run.
Fixes #1849.
2018-02-07 15:20:54 -06:00
Will Dietz 3780435a0e tests: Add (failing) tests for reported --dry-run issues. 2018-02-07 15:20:54 -06:00
Will Dietz f201b7733e More completely fix recursive nix, unbreak tests
See:
https://github.com/NixOS/nix/commit/88b5d0c8e89afefbc547b6243c5aa5a3ec8176e9#commitcomment-27406365
2018-02-07 15:19:28 -06:00
AmineChikhaoui a56637205a Merge branch 'master' of https://github.com/NixOS/nix into parallel-xz 2018-02-07 21:09:04 +01:00
AmineChikhaoui 47ad88099b move the parallel-compression setting to binary-cache-store, the setting
can be done now from the url e.g s3://nix-cache?parallel-compression=1
instead of nix.conf.
2018-02-07 21:06:11 +01:00
Eelco Dolstra 88b5d0c8e8 Prevent accidental recursive Nix 2018-02-07 19:07:38 +01:00
AmineChikhaoui 55ecdfe2a8 make multi threaded compression configurable and use single threaded
by default.
2018-02-07 17:54:08 +01:00
AmineChikhaoui 163e39547a Merge branch 'master' of https://github.com/NixOS/nix into parallel-xz 2018-02-07 17:53:50 +01:00
Shea Levy 48c192ca2d builtins.path test: Don't rely on shlevy's XDG_RUNTIME_DIR 2018-02-07 10:26:53 -05:00
Eelco Dolstra 84989d3af2 Improve filtering of ANSI escape sequences in build logs
All ANSI sequences except color setting are now filtered out. In
particular, terminal resets (such as from NixOS VM tests) are filtered
out.

Also, fix the completely broken tab character handling.
2018-02-07 15:23:57 +01:00
Eelco Dolstra cfdfad5c34 Simplify 2018-02-07 14:15:20 +01:00
Eelco Dolstra 0f3dae1064 Merge branch 'fix-aarch64-test' of https://github.com/grahamc/nix 2018-02-07 14:12:15 +01:00
Eelco Dolstra abe6be578b Merge pull request #1816 from shlevy/add-path
Add path primop.
2018-02-07 13:32:35 +01:00
AmineChikhaoui 9d1e22f743 set block size to 0 to let the lzma lib choose the right one, add
some comments about possible improvements wrt memory usage/threading.
2018-02-07 11:18:55 +01:00
Shea Levy 69d82e5c58 Add path primop.
builtins.path allows specifying the name of a path (which makes paths
with store-illegal names now addable), allows adding paths with flat
instead of recursive hashes, allows specifying a filter (so is a
generalization of filterSource), and allows specifying an expected
hash (enabling safe path adding in pure mode).
2018-02-06 16:48:08 -05:00
AmineChikhaoui bc7e3a4dd6 support multi threaded xz encoder, this might be particularly useful in
the case of hydra where the overhead of single threaded encoding is more
noticeable e.g most of the time spent in "Sending inputs"/"Receiving outputs"
is due to compression while the actual upload to the binary cache seems
to be negligible.
2018-02-06 22:42:02 +01:00
Eelco Dolstra 6f6bfc8205 Update the progress bar at most 20 times per second
Fixes #1834.
2018-02-06 20:51:37 +01:00
Shea Levy 694b6d229b Merge branch 'fix/busybox-sandbox-shell-attribute' of git://github.com/dtzWill/nix 2018-02-06 14:01:22 -05:00
Eelco Dolstra 43f8ef73c6 realiseContext(): Add derivation outputs to the allowed paths
This makes import-from-derivation work in restricted mode again.
2018-02-06 15:38:45 +01:00
Eelco Dolstra f24e726ba5 checkURI(): Check file URIs against allowedPaths
This makes e.g. 'fetchGit ./.' work (assuming that ./. is an allowed
path).
2018-02-06 14:35:33 +01:00
Eelco Dolstra f539085e65 Fix evaluation 2018-02-05 21:48:09 +01:00
Will Dietz 47dc6076af release-common: use shell from nixpkgs, provide fallback for compat 2018-02-05 11:33:18 -06:00
Eelco Dolstra 55012ec0b9 Tweak progress bar message
Say "copying" instead of "fetching" when copying from another local
store. Nice for nixos-install.
2018-02-05 18:32:23 +01:00
Eelco Dolstra bb1d046f5c Allow substituters to be marked as trusted
This is needed by nixos-install, which uses the Nix store on the
installation CD as a substituter. We don't want to disable signature
checking entirely because substitutes from cache.nixos.org should
still be checked. So now we can pas "local?trusted=1" to mark only the
Nix store in /nix as not requiring signatures.

Fixes #1819.
2018-02-05 18:08:30 +01:00
Eelco Dolstra 2175eee9fe Fix segfault using non-binary cache stores as substituters 2018-02-05 17:46:43 +01:00
Will Dietz 0ffa615420 busybox shell: enable various useful/expected features
Matches changes made in nixpkgs:
https://github.com/NixOS/nixpkgs/pull/34628
2018-02-05 10:25:26 -06:00
Eelco Dolstra 84722d67d2 Remove nix-build --hash
Instead, if a fixed-output derivation produces has an incorrect output
hash, we now unconditionally move the outputs to the path
corresponding with the actual hash and register it as valid. Thus,
after correcting the hash in the Nix expression (e.g. in a fetchurl
call), the fixed-output derivation doesn't have to be built again.

It would still be good to have a command for reporting the actual hash
of a fixed-output derivation (instead of throwing an error), but
"nix-build --hash" didn't do that.
2018-02-03 10:08:05 +01:00
Shea Levy de96daf54f Merge branch 'master' of git://github.com/catern/nix 2018-02-01 13:21:45 -05:00
Eelco Dolstra 19477e8815 nix-build: Ignore --indirect
Note that nix-build always creates indirect roots.

Fixes #1830.
2018-02-01 17:09:56 +01:00
Eelco Dolstra e7b23eb5ab Remove docs on removed --drv-link and --add-drv-link options 2018-02-01 16:40:58 +01:00
Eelco Dolstra 855699855f Remove obsolete references to manifests
Closes #323.
2018-02-01 10:39:16 +01:00
Spencer Baugh e5432574e2 document ability to set NIX_REMOTE=unix://path/to/socket 2018-01-31 22:47:16 +00:00
Eelco Dolstra c129fc6ee8 Merge pull request #1801 from catern/master
remote_store: register for NIX_REMOTE=unix://path
2018-01-31 23:12:39 +01:00
Eelco Dolstra 88f4f0231b Fix building without aws-sdk-cpp 2018-01-31 22:34:51 +01:00
Eelco Dolstra c287d73121 Rename 1.12 -> 2.0
Following discussion with Shea and Graham. It's a big enough change
from the last release. Also, from a semver perspective, 2.0 makes more
sense because we did remove some interfaces (like nix-pull/nix-push).
2018-01-31 18:58:45 +01:00
Eelco Dolstra 6fa690291a Add 'nix upgrade-nix' command
This command upgrades Nix to the latest stable version by installing a
store path obtained from

  https://github.com/NixOS/nixpkgs/raw/master/nixos/modules/installer/tools/nix-fallback-paths.nix

which is the same store path that the installer at
https://nixos.org/nix/install.sh uses.

The upgrade fails if Nix is not installed in a profile (e.g. on NixOS,
or when installed outside of the Nix store).
2018-01-31 16:24:43 +01:00
Eelco Dolstra 8af911be5c Merge pull request #1825 from giorgiga/master
Fix macOS installation when umask disallow public read (solves #1582)
2018-01-31 15:16:24 +01:00
Eelco Dolstra 30370f168f Cleanup 2018-01-31 15:14:03 +01:00
Eelco Dolstra 6270b2e50f Merge branch 'http-binary-cache-put-upsert' of https://github.com/adelbertc/nix 2018-01-31 15:10:12 +01:00
Eelco Dolstra f8e8dd827e Manual: Remove old cruft 2018-01-31 15:08:46 +01:00
Eelco Dolstra 478e3e4649 Indent properly 2018-01-31 13:46:38 +01:00
Giorgio Gallo 9f9393df55 solves #1582 2018-01-29 21:33:17 +01:00
Ben Gamari f93e890b4d configure: Use $CPP instead of cpp directly
The latter breaks in the case of cross-compilation, when `cpp` bears a
target prefix.
2018-01-29 02:20:48 -05:00
Eric Wolf 0167eac571 Improve manual on inheriting attributes
Expands first paragraph a bit
Adds a more comprehensive example
2018-01-27 16:18:31 +01:00
Spencer Baugh 746f8aed86 remote_store: register for NIX_REMOTE=unix://path
This allows overriding the socket path so the daemon may be listening at
an arbitrary Unix domain socket location.

Fixes #1800
2018-01-26 22:05:16 +00:00
Shea Levy 1d5d277ac7 HttpBinaryCacheStore: Support upsertFile with PUT.
Some servers, such as Artifactory, allow uploading with PUT and BASIC
auth. This allows nix copy to work to upload binaries to those
servers.

Worked on together with @adelbertc
2018-01-26 11:12:30 -08:00
Eelco Dolstra e09161d05c Remove signed-binary-caches as the default for require-sigs
This was for backward compatibility. However, with security-related
configuration settings, it's best not to have any confusion.

Issue #495.
2018-01-26 17:12:15 +01:00
Eelco Dolstra 98f3c75a0e Merge pull request #1797 from dezgeg/userns-tests-fix
Fix tests using user namespaces on kernels that don't have it
2018-01-24 17:17:48 +01:00
Eelco Dolstra 479692a068 Merge pull request #1811 from dtzWill/update/json-3.0.1
nlohmann-json: 2.1.1 -> 3.0.1
2018-01-24 17:11:25 +01:00
Eelco Dolstra b76e282da8 Merge pull request #1813 from copumpkin/download-pre-resolve-url
Fix obscure corner case in name resolution for builtin:fetchurl in sandboxed environments
2018-01-23 21:05:04 +01:00
Dan Peebles d43a8b25f0 Fix obscure corner case in name resolution for builtin:fetchurl in sandboxed environments 2018-01-23 14:45:50 -05:00
Will Dietz f7c26365eb nlohmann-json: 2.1.1 -> 3.0.1 2018-01-22 12:19:50 -06:00
Eelco Dolstra c382866cd2 Fix test
https://hydra.nixos.org/build/67806811
2018-01-22 17:04:08 +01:00
Eelco Dolstra 9304fde8de Don't access tarballs.nixos.org in a test
https://hydra.nixos.org/build/67806811
2018-01-22 16:56:28 +01:00
Eelco Dolstra 89a2a11d9f Don't use [[noreturn]] 2018-01-19 15:00:38 +01:00
Eelco Dolstra 3c4c30eadd Rewrite builtin derivation environment
Also add a test.

Fixes #1803.
Closes #1805.
2018-01-19 15:00:38 +01:00
Eelco Dolstra 2896bb6826 Don't retry CURLE_URL_MALFORMAT 2018-01-19 14:05:08 +01:00
Eelco Dolstra 87e3d142cc Add a test for --check / --repeat 2018-01-19 13:58:28 +01:00
Eelco Dolstra 0c95776c3e Don't define builtins.{currentSystem,currentTime} in pure mode
This makes it easier to provide a default, e.g.

  system = builtins.currentSystem or "x86_64-linux";
2018-01-18 16:38:48 +01:00
Eelco Dolstra 5647e55f65 Merge pull request #1793 from peterstuart/fix-extra-space
Remove extra space in chat_about_sudo()
2018-01-18 16:19:44 +01:00
Eelco Dolstra 5cc5c3fb83 Merge pull request #1799 from iavael/manpath-fix-1
Fix manpath detection
2018-01-18 16:19:21 +01:00
Eelco Dolstra 27b510af5c nix eval: Stop progress bar before printing the result 2018-01-17 12:04:44 +01:00
Eelco Dolstra 16e0287556 nix eval: Take only one argument
Thus --json no longer produces a list.
2018-01-17 12:03:06 +01:00
Eelco Dolstra cfeff3b273 Move show-trace docs 2018-01-17 11:53:16 +01:00
Eelco Dolstra d8b4cfad82 Typo 2018-01-17 11:53:08 +01:00
Eelco Dolstra 6ddfe9a999 <nix/fetchurl.nix>: Don't access builtins.currentSystem
This doesn't work in pure evaluation mode.
2018-01-16 19:24:23 +01:00
Eelco Dolstra 75b9670df6 Make show-trace a config setting 2018-01-16 19:24:23 +01:00
Eelco Dolstra d4dcffd643 Add pure evaluation mode
In this mode, the following restrictions apply:

* The builtins currentTime, currentSystem and storePath throw an
  error.

* $NIX_PATH and -I are ignored.

* fetchGit and fetchMercurial require a revision hash.

* fetchurl and fetchTarball require a sha256 attribute.

* No file system access is allowed outside of the paths returned by
  fetch{Git,Mercurial,url,Tarball}. Thus 'nix build -f ./foo.nix' is
  not allowed.

Thus, the evaluation result is completely reproducible from the
command line arguments. E.g.

  nix build --pure-eval '(
    let
      nix = fetchGit { url = https://github.com/NixOS/nixpkgs.git; rev = "9c927de4b179a6dd210dd88d34bda8af4b575680"; };
      nixpkgs = fetchGit { url = https://github.com/NixOS/nixpkgs.git; ref = "release-17.09"; rev = "66b4de79e3841530e6d9c6baf98702aa1f7124e4"; };
    in (import (nix + "/release.nix") { inherit nix nixpkgs; }).build.x86_64-linux
  )'

The goal is to enable completely reproducible and traceable
evaluation. For example, a NixOS configuration could be fully
described by a single Git commit hash. 'nixos-rebuild' would do
something like

  nix build --pure-eval '(
    (import (fetchGit { url = file:///my-nixos-config; rev = "..."; })).system
  ')

where the Git repository /my-nixos-config would use further fetchGit
calls or Git externals to fetch Nixpkgs and whatever other
dependencies it has. Either way, the commit hash would uniquely
identify the NixOS configuration and allow it to reproduced.
2018-01-16 19:23:18 +01:00
Eelco Dolstra 23fa7e3606 parseExprFromFile -> evalFile
parseExprFromFile() should be avoided since it doesn't cache anything.
2018-01-16 17:11:58 +01:00
Eelco Dolstra ba75c69e00 Barf when using a diverted store on macOS
Fixes #1792.
2018-01-15 12:14:43 +01:00
Eelco Dolstra 53b520708a Merge pull request #1794 from dtzWill/feature/nix-log-pager
nix log: use pager
2018-01-15 11:25:48 +01:00
Iavael ebc42f8b59 Fix manpath detection
Checking for MANPATH without quotes always returns true, so that it breaks bash-completion for man pages on modern systems without MANPATH environment variable.
2018-01-15 00:43:39 +03:00
Tuomas Tynkkynen 59086e459c Fix tests using user namespaces on kernels that don't have it
Disable various tests if the kernel doesn't support unprivileged user
namespaces (e.g. Arch Linux disables them) or disable them via a sysctl
(Debian, Ubuntu).

Fixes #1521
Fixes #1625
2018-01-13 19:08:41 +02:00
Will Dietz 6454ca393a nix log: use pager 2018-01-12 15:05:26 -06:00
Peter Stuart a65376b01d Remove extra space. 2018-01-12 14:27:29 -05:00
Eelco Dolstra 74f75c8558 import, builtins.readFile: Handle diverted stores
Fixes #1791
2018-01-12 17:31:08 +01:00
Will Dietz 435ccc7980 release: access fetchGit from builtins to fix eval w/1.11 (<1.12) 2018-01-10 14:19:29 -06:00
Eelco Dolstra da85bea7a8 Merge pull request #1777 from bhipple/fix-32bit-rpm-builds
Fix Fedora 25 i386 RPM build
2018-01-10 11:58:20 +01:00
Eelco Dolstra 874ad7d9f8 Merge pull request #1788 from k0001/rem-subs-1
nix.conf: builders-use-substitutes
2018-01-10 11:31:51 +01:00
Renzo Carbonara b0328c244d nix.conf: builders-use-substitutes
Fixes #937
2018-01-09 22:40:07 +01:00
Eelco Dolstra 3cd0704387 Merge pull request #1787 from dtzWill/fix/git-not-on-PATH
fix git cache 'corruption' when git not available, breaks all future use of fetchGit
2018-01-09 16:49:44 +01:00
Will Dietz 428680b307 fetchGit: fix creation of uninitialized cache dir, let git create it
fetchGit test (as modified in previous commit) now passes.
2018-01-09 09:05:18 -06:00
Will Dietz 84d9e213d2 fetchGit.sh: Test we don't "corrupt" cache if invoke w/o git avail 2018-01-09 08:58:19 -06:00
Eelco Dolstra 7b9583680e Improve error message with --repair for untrusted users 2018-01-08 19:13:48 +01:00
Eelco Dolstra 6e0989685a Merge pull request #1774 from LnL7/darwin-no-etc-profile
installer: don't touch /etc/profile
2018-01-05 12:00:44 +01:00
Benjamin Hipple 1882e802e7 Fix Fedora 25 i386 RPM build 2018-01-04 19:44:32 -05:00
Eelco Dolstra 44272d8719 Rename "use-substitutes" to "substitute"
Commit c2154d4c84 renamed
"build-use-substitutes" to "use-substitutes", but that broke
"nix-copy-closure --use-substitutes".
2018-01-04 16:58:39 +01:00
Domen Kožar 1b851ae8f6 Merge pull request #1773 from bhipple/fix-hydra-rpm-builds
Fix RPM builds by increasing VM memory size
2018-01-04 12:00:32 +00:00
Daiderd Jordan d15826164c installer: create 'enough' build users 2018-01-03 22:34:34 +01:00
Daiderd Jordan 27788f4060 installer: don't touch /etc/profile
The default profile already loads /etc/bashrc.
2018-01-03 22:29:54 +01:00
Benjamin Hipple 4cb5c51375 Fix RPM builds by increasing VM memory size
The VM was running out of RAM while handling debug symbols, which caused the
eu-strip to fail while separating debug symbols.
2018-01-02 23:39:42 -05:00
Eelco Dolstra e297aa7b1c Merge pull request #1772 from shlevy/hasContext
Add hasContext primop
2018-01-02 18:53:57 +01:00
Shea Levy 689b2783fc Add hasContext primop 2018-01-02 12:25:14 -05:00
Eelco Dolstra f68c2b5a78 Merge pull request #1770 from dtzWill/fix/run-test-sandbox-ubuntu
run.sh: include lib64 in sandbox-paths to fix on ubuntu 16.XX
2018-01-02 11:32:18 +01:00
Eelco Dolstra 9b67f234c9 Merge pull request #1768 from dtzWill/feature/brotli
use libbrotli directly when available
2018-01-02 11:31:29 +01:00
Eelco Dolstra 099ba37820 Merge pull request #1766 from FRidh/aspell
Do not export ASPELL_CONF
2018-01-02 11:29:13 +01:00
Will Dietz 5afee18726 run.sh: include lib64 in sandbox-paths to fix on ubuntu 16.XX
(cc #1769)
2017-12-30 22:41:49 -06:00
Will Dietz 9dd2b8ac7b use libbrotli directly when available
* Look for both 'brotli' and 'bro' as external command,
  since upstream has renamed it in newer versions.
  If neither are found, current runtime behavior
  is preserved: try to find 'bro' on PATH.
* Limit amount handed to BrotliEncoderCompressStream
  to ensure interrupts are processed in a timely manner.
  Testing shows negligible performance impact.
  (Other compression sinks don't seem to require this)
2017-12-30 20:26:33 -06:00
Frederik Rietdijk ab8ba71205 Do not export ASPELL_CONF
This does not belong in Nix. Setting this env var is already done by the aspell derivation found in Nixpkgs.
2017-12-29 13:45:54 +01:00
Shea Levy 6a0dd63508 Merge branch 'fix/issue-1757' of git://github.com/dtzWill/nix 2017-12-27 18:51:05 -05:00
Shea Levy 25196d0d26 Merge branch 'fix/fetchGit-clean-branch' of git://github.com/dtzWill/nix 2017-12-27 18:50:08 -05:00
Will Dietz bd17ccf1d8 nix repl: use linenoiseKeyType to differentiate ^C and ^D
Fixes #1757.
2017-12-26 19:25:50 -06:00
Eelco Dolstra 4801420893 Remove debug line 2017-12-25 14:53:15 +01:00
Will Dietz 2e6f06c37e fetchGit: Fix handling of local repo when not using 'master' branch
Add tests checking this behavior.
2017-12-22 15:29:52 -06:00
Eelco Dolstra aa43cbb764 Check aws-sdk-cpp version 2017-12-22 12:05:13 +01:00
Eelco Dolstra 6d80870832 release.nix: Use fetchTarball and fetchGit
In particular, using fetchGit means we don't need hackery to clean the
source tree when building from an unclean tree.
2017-12-22 11:35:32 +01:00
Shea Levy 62e214fa6f Merge branch 'fix/issue-1742-empty-completions' of git://github.com/dtzWill/nix 2017-12-20 13:18:36 -05:00
Eelco Dolstra 576abaa650 Merge pull request #1753 from FRidh/fix/escaping
Fix escaping, fixes build
2017-12-20 16:23:18 +01:00
Frederik Rietdijk af1e2ffca1 Fix escaping, fixes build 2017-12-20 13:24:39 +01:00
Will Dietz 71987b18d4 linenoise.cpp: allow completions from empty input
Fixes #1742.
2017-12-19 12:52:10 -06:00
Dan Peebles ca40fbdc50 Add an issue template to cut down on the confusion 2017-12-17 11:08:28 -05:00
Eelco Dolstra 8efe937a35 Fix build on gcc 7
Fixes #1738.
2017-12-16 00:49:31 +01:00
Eelco Dolstra 1dffbff57d Merge pull request #1736 from bgamari/stoi-exceptions
Gracefully handle exceptions from stoi
2017-12-15 11:39:43 +01:00
Ben Gamari 626a94d70e edit: Catch stoi exceptions from line number parsing 2017-12-14 19:11:56 -05:00
Ben Gamari f9bcbddef2 json-to-value: Throw sensible error message on invalid numbers 2017-12-14 19:09:45 -05:00
Eelco Dolstra 5d5b931fb1 Fix build against current aws-sdk-cpp 2017-12-14 16:45:40 +01:00
Graham Christensen e4ece83b1a tests.setuid: only on i686 and x86_64 linuxs 2017-12-12 08:31:31 -05:00
Eelco Dolstra 1ca8e95178 Merge pull request #1722 from bhipple/fix-for-older-libcurl
Fix for builds with system libcurl < 7.30
2017-12-12 11:48:08 +01:00
Eelco Dolstra de38fa9987 Merge pull request #1724 from dtzWill/fix/minor-warnings
fix minor warnings
2017-12-12 11:47:47 +01:00
Eelco Dolstra e3c19ff9bc Merge pull request #1725 from chris-martin/pr/concatStringsSep
Add builtins.concatStringSep to the manual
2017-12-12 11:46:10 +01:00
Chris Martin 2a8f09a8c1 Add builtins.concatStringSep to the manual 2017-12-11 14:28:03 -05:00
Will Dietz 00e0c416ff Mark comparison call operator as const 2017-12-11 12:08:40 -06:00
Will Dietz e4d43f1f94 logging.cc: add missing 'override' 2017-12-11 12:08:40 -06:00
Will Dietz c800f47395 Fix forward-decl of Bindings as "class", match definition.
(appease clang -Wmismatched-tags warning)
2017-12-11 11:51:55 -06:00
Eelco Dolstra 06d4566c2d Merge pull request #1721 from expipiplus1/patch-2
Escape left angle brackets in XML documentation
2017-12-11 17:29:36 +01:00
Shea Levy 5ba5ca7888 Merge branch 'feature/linenoise-ng' of git://github.com/dtzWill/nix 2017-12-11 07:52:20 -05:00
Benjamin Hipple d76c4fe770 Fix for builds with system libcurl < 7.30
CentOS 7.4 and RHEL 7.4 ship with libcurl-devel-7.29.0-42.el7.x86_64; this flag
was added in 7.30.0
https://curl.haxx.se/libcurl/c/CURLMOPT_MAX_TOTAL_CONNECTIONS.html
2017-12-09 20:02:21 -05:00
Joe Hermaszewski 35a49f1d7f Escape left angle brackets in XML documentation 2017-12-09 15:31:03 +00:00
Eelco Dolstra 0b59882fe3 Merge pull request #1717 from grahamc/upload-release-aarch64
upload-release.pl: support aarch64-linux
2017-12-08 14:00:06 +01:00
Eelco Dolstra d73e16df05 Merge pull request #1718 from markus1189/nix-hash-doc
nix-hash: Add sentence and example for nix-prefetch-url hash
2017-12-08 13:59:41 +01:00
Eelco Dolstra 82e6275a7b Merge pull request #1719 from grahamc/fixup-darwin-hs-master
Fixup darwin hs master
2017-12-08 13:58:54 +01:00
Graham Christensen 1db034364a replace lolcat with nix-info 2017-12-08 07:19:32 -05:00
Graham Christensen be79d1f189 darwin installer: fix on High Sierra 2017-12-08 07:19:26 -05:00
Markus Hauck 0af668426d nix-hash: Add sentence and example for nix-prefetch-url hash 2017-12-08 10:27:34 +01:00
Graham Christensen fbab9ed01e upload-release.pl: support aarch64-linux 2017-12-07 19:02:35 -05:00
Eelco Dolstra e3013543d3 Fix test 2017-12-07 01:07:07 +01:00
Eelco Dolstra 2df9cbeb47 Provide random access to cached NARs
E.g.

  $ time nix cat-store --store https://cache.nixos.org?local-nar-cache=/tmp/nars \
    /nix/store/b0w2hafndl09h64fhb86kw6bmhbmnpm1-blender-2.79/share/icons/hicolor/scalable/apps/blender.svg > /dev/null
  real    0m4.139s

  $ time nix cat-store --store https://cache.nixos.org?local-nar-cache=/tmp/nars \
    /nix/store/b0w2hafndl09h64fhb86kw6bmhbmnpm1-blender-2.79/share/icons/hicolor/scalable/apps/blender.svg > /dev/null
  real    0m0.024s

(Before, the second call took ~0.220s.)

This will use a NAR listing in
/tmp/nars/b0w2hafndl09h64fhb86kw6bmhbmnpm1.ls containing all metadata,
including the offsets of regular files inside the NAR. Thus, we don't
need to read the entire NAR. (We do read the entire listing, but
that's generally pretty small. We could use a SQLite DB by borrowing
some more code from nixos-channel-scripts/file-cache.hh.)

This is primarily useful when Hydra is serving files from an S3 binary
cache, in particular when you have giant NARs. E.g. we had some 12 GiB
NARs, so accessing individuals files was pretty slow.
2017-12-07 01:07:07 +01:00
Eelco Dolstra 338f29dbd4 nix ls-{nar,store}: Return offset of files in the NAR if known
E.g.

  $ nix ls-store --json --recursive --store https://cache.nixos.org /nix/store/b0w2hafndl09h64fhb86kw6bmhbmnpm1-blender-2.79 \
    | jq .entries.bin.entries.blender.narOffset
  400
2017-12-07 01:07:07 +01:00
Eelco Dolstra ba334bd6f7 Merge pull request #1714 from LnL7/fix-ssh-store-nar
ssh-store: fix length when writing nar
2017-12-06 13:11:36 +01:00
Daiderd Jordan 2cb8aaa610 ssh-store: fix length when writing nar
This fixes nix copy and other things that use copyStorePath.
2017-12-06 11:41:08 +01:00
Shea Levy 11a7f8ce14 Merge branch 'fetchGit-fast-revision-update' 2017-12-05 11:16:39 -05:00
Eelco Dolstra 7f2c324ed1 Simplify build by including nlohmann/json.hpp 2017-12-04 17:11:36 +01:00
Eelco Dolstra f1efb97075 Fix #1635. 2017-12-04 13:19:16 +01:00
Eelco Dolstra cd74a55afc Merge pull request #1704 from lheckemann/buildenv-eol-fix
Fix "Unexpected EOF reading a line" error
2017-11-29 15:27:57 +01:00
Linus Heckemann f986a44980 Fix "Unexpected EOF reading a line" error
propagated-user-env-packages files in nixpkgs aren't all terminated by
newlines, as buildenv expected. Now it does not require a terminating
newline; note that this introduces a behaviour change: propagated user
env packages may now be spread across multiple lines. However, nix
1.11.x still expects them to be on a single line so this shouldn't be
used in nixpkgs for now.
2017-11-29 11:56:15 +00:00
Eelco Dolstra b52846ab5b Show log tail when a remote build fails 2017-11-28 15:36:28 +01:00
Will Dietz e50d7335bf doc: linenoise -> linenoise-ng 2017-11-27 18:36:20 -06:00
Will Dietz 964349e44d no "linenoiseFree" in linenoise-ng 2017-11-27 18:30:05 -06:00
Will Dietz 21f515724c replace vendored linenoise with linenoise-ng 2017-11-27 18:27:28 -06:00
Eelco Dolstra 3c470c97a1 Merge pull request #1699 from aszlig/fix-remote-build-log
build-remote: Fix missing log output
2017-11-27 13:59:51 +01:00
aszlig 6567ab95a0 build-remote: Fix missing log output
The storeUri variable in the build-remote hook is declared very much to
the start of the main function and a bunch of lines later, the same
variable gets checked via hasPrefix() but it gets assigned *after* that
check when the most suitable machine for the build was choosen.

So I guess this was just a typo in d16fd24973
and what we really want is to either checkd the prefix *after* assigning
storeUri or use bestMachine->storeUri directly.

I choose the latter, because the former could introduce even more
regressions if the try block where the variable gets assigned terminates
early.

Nevertheless, the reason why the log output didn't work is because
hasPrefix() checked for "ssh://" in front of storeUri, but if the
storeUri isn't set correctly (or at all), we don't get the log file
descriptor set up properly, leading to no log output.

I've adjusted the remote-builds test to include a regression test for
this, so that we can make sure we get a build output when using remote
builds.

In addition to that I've tested this with two of my build farms and the
build logs are emitted correctly again.

Signed-off-by: aszlig <aszlig@nix.build>
2017-11-25 01:34:18 +01:00
Eelco Dolstra 549c3706a5 nix path-info: Show URL of NARs in binary caches 2017-11-24 18:08:50 +01:00
Eelco Dolstra 90948a4e3a nix-shell/nix-build: Support .drv files again
Fixes #1663.

Also handle '!<output-name>' (#1694).
2017-11-24 18:08:35 +01:00
Shea Levy eedbc4e06c fetchGit: Ignore tarballTtl if rev is set and not in the repo.
Fixes #1697.
2017-11-24 06:09:24 -05:00
Eelco Dolstra 0fc3e581e0 tests/run.sh: Fix /build conflict when building in a sandbox
https://hydra.nixos.org/build/64519355
2017-11-21 19:43:12 +01:00
Eelco Dolstra d7da6c9ea9 fetchGit/fetchMercurial: Fix directory inclusion check
E.g. the existence of .gitignore would cause .git to be included.
2017-11-21 19:34:46 +01:00
Eelco Dolstra 6cdaa858d0 Propagate flags like --sandbox to the daemon properly 2017-11-21 18:50:56 +01:00
Eelco Dolstra 7536fe31dd Add a warning about the 'trusted-users' option 2017-11-21 18:49:52 +01:00
Eelco Dolstra 4fcf44825f Add tests for verifying/copying content-addressed paths
These don't require signatures.
2017-11-20 19:11:02 +01:00
Eelco Dolstra 0c9718aabc Add tests for signature checking when copying between local stores 2017-11-20 19:02:57 +01:00
Eelco Dolstra 8df60b4ea8 Document secret-key-files 2017-11-20 18:51:04 +01:00
Eelco Dolstra 4cde04f476 Add tests for "nix run" 2017-11-20 18:36:36 +01:00
Eelco Dolstra f5f29dc2b7 nix eval --raw: Use coerceToString()
Thus,

  $ nix eval --raw '("foo")'
  foo

  $ nix eval --raw nixpkgs.hello
  /nix/store/1y6ckg6khrdsvll54s5spcmf3w6ka9k4-hello-2.10

  $ nix eval --raw '(/etc/resolv.conf)'
  /nix/store/vml92ama92i8mz013nny461mlvg8mvap-resolv.conf
2017-11-20 18:27:29 +01:00
Eelco Dolstra 1ff01187e2 nix run: Fix "flag '--command' requires 2 argument(s)" 2017-11-20 18:07:58 +01:00
Eelco Dolstra 4eb9e20028 nix run: Fix accidental removal of /nix/store existence check
Parenthetical to #1686, we don't need to create a new root if we can
just bind-mount on top of the existing /nix/store.
2017-11-20 17:59:32 +01:00
Eelco Dolstra a3aa850f0f nix run: Ignore non-directories while setting up the chroot
Fixes #1686.
2017-11-20 17:58:47 +01:00
Eelco Dolstra 193330d321 Test: Replace --option with the corresponding flag 2017-11-20 17:50:49 +01:00
Eelco Dolstra 91a1987607 signed-binary-caches -> require-sigs
Unlike signed-binary-caches (which could only be '*' or ''),
require-sigs is a proper Boolean option. The default is true.
2017-11-20 17:44:07 +01:00
Eelco Dolstra d0b88db441 Cleanup 2017-11-20 17:32:58 +01:00
Eelco Dolstra 7a2b64e55c binary-cache-public-keys -> trusted-public-keys
The name had become a misnomer since it's not only for substitution
from binary caches, but when adding/copying any
(non-content-addressed) path to a store.
2017-11-20 17:32:34 +01:00
Eelco Dolstra ea94a87493 install-darwin-multi-user.sh: Remove superfluous nix.conf settings 2017-11-20 17:32:34 +01:00
Eelco Dolstra 72804dc0bd Merge pull request #1645 from twhitehead/stat-race
Fix (highly unlikely) race condition in readLink
2017-11-20 17:32:27 +01:00
Eelco Dolstra 7474ac871b nix copy: Abbreviate "daemon" 2017-11-20 15:17:11 +01:00
Eelco Dolstra 8956ae1987 Add a "profile" option to S3BinaryCacheStore
This allows specifying the AWS configuration profile to use. E.g.

  nix copy --from s3://my-cache?profile=aws-dev-account /nix/store/cf3isrlqavvd5w7rpky1fa8j9lcnlggm-...
2017-11-15 14:18:20 +01:00
Eelco Dolstra 897ca33a1c Merge pull request #1678 from dtzWill/feature/anchor-exit-exception
anchor nix::Exit exception
2017-11-15 12:31:47 +01:00
Eelco Dolstra 27131866c9 Doh 2017-11-15 12:23:31 +01:00
Will Dietz 9d7ce0bf45 anchor nix::Exit exception 2017-11-14 12:39:30 -06:00
Eelco Dolstra 2f5789c5d6 Add dependencies for coverage test 2017-11-14 18:47:44 +01:00
Eelco Dolstra 4b45d8c95a Update lcov filter 2017-11-14 18:47:37 +01:00
Eelco Dolstra ec5b04862b nix sign-paths: Support binary caches 2017-11-14 18:44:05 +01:00
Eelco Dolstra d6dbda7004 Add tests for "nix verify", "nix sign-paths" etc. 2017-11-14 18:24:20 +01:00
Eelco Dolstra 2c39e4eca0 Revert "Don't parse "x:x" as a URI"
This reverts commit f90f660b24.

This broke Hydra's release.nix, which contained

  preCheck = ''export LOGNAME=${LOGNAME:-foo}'';
2017-11-14 15:10:52 +01:00
Eelco Dolstra 5773d667ee nix ls-{nar,store}: Don't abort on missing files 2017-11-14 14:49:06 +01:00
Eelco Dolstra b8d446829e Add some tests 2017-11-14 14:37:39 +01:00
Eelco Dolstra 4db0a9555e nix ls-{nar,store} --json: Respect -R 2017-11-14 14:31:38 +01:00
Eelco Dolstra c8155e9f5f Don't indent JSON output 2017-11-14 14:28:03 +01:00
Eelco Dolstra bac8055652 nix ls-{store,nar}: Add --json flag 2017-11-14 14:23:53 +01:00
Eelco Dolstra c0d93a01ee Remove ncurses-bin 2017-11-14 14:16:16 +01:00
Eelco Dolstra 5ef5d70b5e Make config options available to legacy commands 2017-11-14 14:04:09 +01:00
Eelco Dolstra c9c3fc710b Rename tests/nar-index -> tests/nar-access 2017-11-14 13:27:40 +01:00
Eelco Dolstra 9d87d03331 Remove extraneous comment 2017-11-08 16:17:32 +01:00
Eelco Dolstra 513b143cd8 Merge pull request #1650 from copumpkin/darwin-sandbox-unix-socket
Always allow builds to use unix domain sockets in Darwin sandbox
2017-11-08 16:16:42 +01:00
Eelco Dolstra fd10f6f241 Show when tests are skipped
Also, don't depend on tput (ncurses). It's really not needed since
ANSI escape sequences have been standardized for 35 years or so.
2017-11-07 12:09:57 +01:00
Eelco Dolstra dc30856141 Merge pull request #1632 from AmineChikhaoui/sigint-copy
run query paths in parallel during nix copy and handle SIGINT
2017-11-06 13:36:28 +01:00
Eelco Dolstra 7a4d9574d9 fetchgit.cc -> fetchGit.cc 2017-11-03 13:55:31 +01:00
Eelco Dolstra 6cf7c6a6b0 Remove git:// support in NIX_PATH
This didn't support specifying a revision/branch, and was restricted
to git:// URIs (since https:// or ssh:// would be ambiguous).
2017-11-03 13:55:31 +01:00
Eelco Dolstra ee6ac38848 fetchGit/fetchMercurial: Filter out directories with untracked files 2017-11-03 13:55:31 +01:00
Eelco Dolstra 4dee01da7c fetchGit: Add a test 2017-11-03 13:55:30 +01:00
Eelco Dolstra 0e77aa3982 fetchGit: Don't do a remote fetch if we already have the rev 2017-11-03 13:55:30 +01:00
Eelco Dolstra 9ccea31dc2 Merge pull request #1651 from LnL7/darwin-sandbox-getpwuid
Allow getpwuid in the darwin sandbox
2017-11-03 10:55:31 +01:00
Daiderd Jordan 453f675810 Allow getpwuid in the darwin sandbox. 2017-11-03 10:50:49 +01:00
Eelco Dolstra e104842f8b Merge pull request #1660 from 4z3/patch-1
fetchMercurial: fix error message
2017-11-03 10:49:57 +01:00
Eelco Dolstra 4070264613 Merge pull request #1655 from copumpkin/patch-1
Don't freak out if we get a 403 from S3
2017-11-03 10:49:38 +01:00
Daniel Peebles 3105679226 Don't freak out if we get a 403 from S3
As far as we're concerned, not being able to access a file just means
the file is missing. Plus, AWS explicitly goes out of its way to
return a 403 if the file is missing and the requester doesn't have
permission to list the bucket.

Also getting rid of an old hack that Eelco said was only relevant
to an older AWS SDK.
2017-11-03 01:31:42 +01:00
tv 5ab37f0e44 fetchMercurial: fix error message 2017-11-02 23:37:42 +01:00
Eelco Dolstra ac4a1ef0c1 Merge pull request #1657 from plesiv/docs-typeof-fix
Mention "float" type in the docs
2017-11-02 16:26:30 +01:00
Zoran Plesivčak ae506c1ea4 Mention isFloat function in "Built-in Functions" section 2017-11-01 23:00:08 +00:00
Zoran Plesivčak 3050395810 Mention "float" type in builtins.typeOf section of the docs
+ remove trailing whitespace from the file
2017-11-01 21:36:25 +00:00
Eelco Dolstra 212e72c609 Fix build
https://hydra.nixos.org/build/63172338
2017-11-01 21:32:30 +01:00
Eelco Dolstra e026bc3b05 fetchMercurial: Don't fetch hashes we already have 2017-11-01 18:43:11 +01:00
Eelco Dolstra 1969f357b7 Add fetchMercurial primop
E.g.

  $ nix eval '(fetchMercurial https://www.mercurial-scm.org/repo/hello)'
  { branch = "default"; outPath = "/nix/store/alvb9y1kfz42bjishqmyy3pphnrh1pfa-source"; rev = "82e55d328c8ca4ee16520036c0aaace03a5beb65"; revCount = 1; shortRev = "82e55d328c8c"; }

  $ nix eval '(fetchMercurial { url = https://www.mercurial-scm.org/repo/hello; rev = "0a04b987be5ae354b710cefeba0e2d9de7ad41a9"; })'
  { branch = "default"; outPath = "/nix/store/alvb9y1kfz42bjishqmyy3pphnrh1pfa-source"; rev = "0a04b987be5ae354b710cefeba0e2d9de7ad41a9"; revCount = 0; shortRev = "0a04b987be5a"; }

  $ nix eval '(fetchMercurial /tmp/unclean-hg-tree)'
  { branch = "default"; outPath = "/nix/store/cm750cdw1x8wfpm3jq7mz09r30l9r024-source"; rev = "0000000000000000000000000000000000000000"; revCount = 0; shortRev = "000000000000"; }
2017-11-01 17:45:32 +01:00
Eelco Dolstra cd532a9251 Fix filterSource 2017-10-31 16:18:32 +01:00
Dan Peebles bc6b3f7e8f Always allow builds to use unix domain sockets in Darwin sandbox 2017-10-31 15:33:57 +01:00
Eelco Dolstra 72cd52c3cd builtins.fetchgit: Support importing a working tree
For example, you can write

  src = fetchgit ./.;

and if ./. refers to an unclean working tree, that tree will be copied
to the Nix store. This removes the need for "cleanSource".
2017-10-30 19:59:25 +01:00
Eelco Dolstra 197922ea4e Merge pull request #1646 from copumpkin/optional-sandbox-local-network
Allow optional localhost network access to sandboxed derivations
2017-10-30 18:54:40 +01:00
Dan Peebles 4a4a009f78 Allow optional localhost network access to sandboxed derivations
This will allow bind and connect to 127.0.0.1, which can reduce purity/
security (if you're running a vulnerable service on localhost) but is
also needed for a ton of test suites, so I'm leaving it turned off by
default but allowing certain derivations to turn it on as needed.

It also allows DNS resolution of arbitrary hostnames but I haven't found
a way to avoid that. In principle I'd just want to allow resolving
localhost but that doesn't seem to be possible.

I don't think this belongs under `build-use-sandbox = relaxed` because we
want it on Hydra and I don't think it's the end of the world.
2017-10-30 17:59:12 +01:00
Eelco Dolstra f90f660b24 Don't parse "x:x" as a URI
URIs now have to contain "://" or start with "channel:".
2017-10-30 17:58:01 +01:00
Tyson Whitehead 07d2c6d213 Fix (highly unlikely) race condition in readLink
Used to determine symlink size with stat and value with readlink.
This could technically result in garbage if symlink changed between
calls.  Also gets around the broken stat implementation in our
network filesystem (returns size + 1 giving a byte of garbage).
2017-10-30 11:49:55 -04:00
Eelco Dolstra 12991152be nix-build: Fix --hash 2017-10-30 13:31:05 +01:00
Eelco Dolstra 63c80ae26f Make "fetchGit /path" work 2017-10-30 13:18:28 +01:00
Eelco Dolstra a5c392a80e fetchGit: Fix broken assertion
Different URIs can map to the same cache entry if they have the same
revision.
2017-10-30 12:55:46 +01:00
Eelco Dolstra 812e027e1d Add option allowed-uris
This allows network access in restricted eval mode.
2017-10-30 12:41:49 +01:00
Eelco Dolstra f1c555cef8 fetchurl/fetchTarball are *not* allowed in restricted mode
Accidentally committed this change as part of
f9686885be.

Restricted mode != pure mode.
2017-10-30 12:41:48 +01:00
Domen Kožar 5cb78053f0 Merge pull request #1633 from orivej/doc
Update the language documentation
2017-10-30 12:21:54 +01:00
Eelco Dolstra e38382895d builtins.fetchGit: Return an attrset with revision info
This adds rev, shortRev and revCount attributes, equal to what Hydra
provides. E.g.

  $ nix eval '(fetchGit https://github.com/NixOS/patchelf.git)'
  { outPath = "/nix/store/ghigrkw02l440g8vfxa9wj4c3zpfmw99-source"; rev = "29c085fd9d3fc972f75b3961905d6b4ecce7eb2b"; revCount = 303; shortRev = "29c085f"; }
2017-10-30 11:49:03 +01:00
Eelco Dolstra f9686885be enable-http2 -> http2 2017-10-30 11:00:59 +01:00
Eelco Dolstra 049322702b fetchgit -> fetchGit
Almost all other primops are camelCase so no reason not to use that
here.
2017-10-30 10:25:08 +01:00
Eelco Dolstra 23ce4b3393 fetchTarball: Use "source" as the default name
This ensures that it produces the same output as fetchgit:

  $ nix eval --raw '(builtins.fetchgit https://github.com/NixOS/patchelf.git)'
  /nix/store/ghigrkw02l440g8vfxa9wj4c3zpfmw99-source

  $ nix eval --raw '(fetchTarball https://github.com/NixOS/patchelf/archive/master.tar.gz)'
  /nix/store/ghigrkw02l440g8vfxa9wj4c3zpfmw99-source
2017-10-30 10:22:58 +01:00
Eelco Dolstra 66ddbef754 fetchurl/fetchTarball: Respect name changes
The computation of urlHash didn't take the name into account, so
subsequent fetchurl calls with the same URL but a different name would
resolve to the same cached store path.
2017-10-30 10:22:58 +01:00
Eelco Dolstra f25791c196 builtins.fetchgit: Don't mess up the progress indicator 2017-10-30 10:22:58 +01:00
Eelco Dolstra 65b5f177b5 builtins.fetchgit: Support a "name" attribute
The "name" attribute defaults to "source", which we should use for all
similar functions (e.g. fetchTarball and in Hydra) to ensure that we
get a consistent store path regardless of how the tree is fetched.

"source" is not necessarily a correct label, but using an empty name
is problematic: you get an ugly store path ending in a dash, and it's
impossible to have a fixed-output derivation that produces that path
because ".drv" is not a valid store name.

Fixes #904.
2017-10-30 10:22:07 +01:00
Eelco Dolstra c1ae18941a nix repl :t: Indicate strings that have a context 2017-10-29 17:11:22 +01:00
Orivej Desh 842ce8bafd Clarify indented string escaping rules 2017-10-26 18:53:36 +00:00
Orivej Desh 15457c5673 Describe "with" scoping precedence 2017-10-26 18:53:36 +00:00
Orivej Desh e5c499b833 Describe arithmetic operators 2017-10-26 18:53:35 +00:00
AmineChikhaoui 0f9a7225ab respect SIGINT in nix copy during the paths queries #1629 2017-10-25 16:51:45 +01:00
AmineChikhaoui 54a2cd9ce4 Merge branch 'master' of github.com:NixOS/nix into sigint-copy 2017-10-25 16:50:52 +01:00
Eelco Dolstra 9971d875a4 Fix building on clang
https://hydra.nixos.org/build/62945761
2017-10-25 17:20:47 +02:00
AmineChikhaoui 9f01a3f0a8 attempt to fix #1630: make the queries of store paths run in parallel using a thread pool 2017-10-25 16:13:49 +01:00
Eelco Dolstra 82327e3cc4 exportReferencesGraph: Allow exporting a list of store paths 2017-10-25 15:18:49 +02:00
Eelco Dolstra 8191992c83 Bump language version 2017-10-25 15:18:37 +02:00
Eelco Dolstra 3395e3bbc4 Fix exportReferencesGraph in the structured attrs case 2017-10-25 14:08:29 +02:00
Eelco Dolstra 2d5b1b24bf Pass lists/attrsets to bash as (associative) arrays 2017-10-25 13:01:50 +02:00
Eelco Dolstra ac12517f3e Fix RPM build 2017-10-24 15:44:29 +02:00
Eelco Dolstra 86aab6e9a0 Add --store argument to legacy commands 2017-10-24 15:41:11 +02:00
Eelco Dolstra d16fd24973 Allow shorter syntax for chroot stores
You can now say '--store /tmp/nix' instead of '--store local?root=/tmp/nix'.
2017-10-24 15:32:38 +02:00
Eelco Dolstra 3460e4cf00 More progress indicator improvements
In particular, don't show superfluous "fetching path" and "building
path(s)" messages, and show the current round (with --repeat).
2017-10-24 15:32:38 +02:00
Eelco Dolstra 96051dd057 More progress indicator improvements
Fixes #1599.
2017-10-24 14:47:23 +02:00
Eelco Dolstra be220702a7 Progress indicator: Show on what machine we're building
E.g.

  $ nix build nixpkgs.hello --builders 'root@wendy'
  [1/0/1 built] building hello-2.10 on ssh://root@wendy: checking for minix/config.h... no
2017-10-24 14:24:57 +02:00
Eelco Dolstra f8624762ac build-remote: Work properly on a chrooted store 2017-10-24 14:14:28 +02:00
Eelco Dolstra fe9d2f974d Handle log messages from build-remote
This makes the progress indicator show statuses like "connecting to
'root@machine'".
2017-10-24 14:05:22 +02:00
Eelco Dolstra 0d59f1ca49 nix: Respect -I, --arg, --argstr
Also, random cleanup to argument handling.
2017-10-24 12:58:34 +02:00
Eelco Dolstra 25f32625e2 Remove the remote-builds option
This is superfluous since you can now just set "builders" to empty,
e.g. "--builders ''".
2017-10-24 11:00:16 +02:00
Eelco Dolstra af241ae7d3 Remove the builder-files option
You can now include files via the "builders" option, using the syntax
"@<filename>". Having only one option makes it easier to override
builders completely.

For backward compatibility, the default is "@/etc/nix/machines", or
"@<filename>" for each file name in NIX_REMOTE_SYSTEMS.
2017-10-24 10:54:43 +02:00
Eelco Dolstra d4609bb3af NIX_BUILD_HOOK variable is gone 2017-10-23 20:50:28 +02:00
Eelco Dolstra 37fbfffd8e Pass all settings to build-remote
This ensures that command line flags such as --builders get passed
correctly.
2017-10-23 20:50:28 +02:00
Eelco Dolstra f32cdc4fab Turn $NIX_REMOTE into a configuration option 2017-10-23 20:50:28 +02:00
Eelco Dolstra 38dcd0c4b8 Release note updates 2017-10-23 20:50:28 +02:00
Eelco Dolstra 863a45f1f3 build-remote: Put current load under the store state directory
Fixes the error

  error: opening lock file '/nix/var/nix/current-load/main-lock': Permission denied

when using a chroot store.
2017-10-23 20:50:24 +02:00
Eelco Dolstra f31c66d392 Merge pull request #1616 from copumpkin/separate-darwin-sandbox
Shift Darwin sandbox to separate installed files
2017-10-20 12:00:27 +02:00
Eelco Dolstra 311b59001a Merge pull request #1622 from kini/c++14-prerequisite
Mention C++14 dependency in the manual.
2017-10-20 11:35:14 +02:00
Keshav Kini 32940702fc Mention C++14 dependency in the manual.
A couple makefiles in the sources have -std=c++14 in the CFLAGS.
2017-10-19 18:03:26 -07:00
Shea Levy c7af84ce84 nix-shell: Fix file name resolution in shebangs 2017-10-19 15:02:40 -04:00
Eelco Dolstra ef1d0142a0 Revert "Let's not populate the NAR cache from hydra-queue-runner for now"
This reverts commit 908590dc6c. Since
hydra-server can have a different store URI from hydra-queue-runner
now, we don't really need this.
2017-10-19 12:11:07 +02:00
Eelco Dolstra 908590dc6c Let's not populate the NAR cache from hydra-queue-runner for now 2017-10-18 15:45:39 +02:00
Eelco Dolstra 75cd75b1ae Errors writing to the NAR cache should not be fatal 2017-10-18 15:41:14 +02:00
Eelco Dolstra d8306148e0 Suppress "copying 0 paths" message 2017-10-18 15:02:58 +02:00
Dan Peebles 6e5165b773 Shift Darwin sandbox to separate installed files
This makes it slightly more manageable to see at a glance what in a
build's sandbox profile is unique to the build and what is standard. Also
a first step to factoring more of our Darwin logic into scheme functions
that will allow us a bit more flexibility. And of course less of that
nasty codegen in C++! 😀
2017-10-17 19:15:51 -04:00
Eelco Dolstra b24b8ef77c BinaryCacheStore::addToStore(): Add NARs to the local cache 2017-10-17 21:39:48 +02:00
Eelco Dolstra ca580bec35 BinaryCacheStore: Support local caching of NARs
This speeds up commands like "nix cat-store". For example:

  $ time nix cat-store --store https://cache.nixos.org?local-nar-cache=/tmp/nar-cache /nix/store/i60yncmq6w9dyv37zd2k454g0fkl3arl-systemd-234/etc/udev/udev.conf
  real    0m4.336s

  $ time nix cat-store --store https://cache.nixos.org?local-nar-cache=/tmp/nar-cache /nix/store/i60yncmq6w9dyv37zd2k454g0fkl3arl-systemd-234/etc/udev/udev.conf
  real    0m0.045s

The primary motivation is to allow hydra-server to serve files from S3
binary caches. Previously Hydra had a hack to do "nix-store -r
<path>", but that fetches the entire closure so is prohibitively
expensive.

There is no garbage collection of the NAR cache yet. Also, the entire
NAR is read when accessing a single member file. We could generate the
NAR listing to provide random access.

Note: the NAR cache is indexed by the store path hash, not the content
hash, so NAR caches should not be shared between binary caches, unless
you're sure that all your builds are binary-reproducible.
2017-10-17 21:15:33 +02:00
Eelco Dolstra 11ba4302e3 Remove BinaryCacheStoreAccessor
Probably as a result of a bad merge in
4b8f1b0ec0, we had both a
BinaryCacheStoreAccessor and a
RemoteFSAccessor. BinaryCacheStore::getFSAccessor() returned the
latter, but BinaryCacheStore::addToStore() checked for the
former. This probably caused hydra-queue-runner to download paths that
it just uploaded.
2017-10-17 20:51:42 +02:00
Shea Levy cbc216911d fetchgit: Remove incomplete/unneeded ref check.
This check fails for tags and branches, and is made redundant by the
checks git itself will do when fetching the repo.
2017-10-17 12:30:54 -04:00
Shea Levy 8522db1641 Merge branch 'nix-profile-daemon' 2017-10-16 18:17:51 -04:00
Eelco Dolstra be59f07799 Merge pull request #1601 from shlevy/fetchgit-no-uri-check
fetchgit: Remove incomplete/unneeded isURI check.
2017-10-16 21:22:13 +02:00
Shea Levy 6a037a738a Pull nix-profile-daemon from 1.11 2017-10-16 14:51:39 -04:00
Shea Levy 4e58294ae6 fetchgit: Remove incomplete/unneeded isURI check.
This check spuriously fails for e.g. git@github.com:NixOS/nixpkgs.git,
and even for ssh://git@github.com/NixOS/nixpkgs.git, and is made
redundant by the checks git itself will do when fetching the repo. We
instead pass a -- before passing the URI to git to avoid injection.
2017-10-16 14:35:28 -04:00
Eelco Dolstra 1dd29d7aeb Add option to disable the seccomp filter
I needed this to test ACL/xattr removal in
canonicalisePathMetaData(). Might also be useful if you need to build
old Nixpkgs that doesn't have the required patches to remove
setuid/setgid creation.
2017-10-12 18:21:55 +02:00
Eelco Dolstra 97307811ee Merge pull request #1571 from teto/improve_priority_msg
Improve error message for conflicting priorities
2017-10-12 13:10:44 +02:00
Eelco Dolstra 73252aef18 Merge pull request #1591 from shlevy/darwin-installer-no-sudo-i
darwin installer: Fix on systems where sudo -i is disabled.
2017-10-12 13:08:15 +02:00
Eelco Dolstra 29205e0218 Merge pull request #1586 from acowley/patch-2
install-darwin-multi-user: relax assumption check
2017-10-12 13:06:25 +02:00
Eelco Dolstra 177aee07c6 Merge pull request #1580 from copumpkin/retry-after-timeout
Retry in all error cases but a few
2017-10-12 13:05:45 +02:00
Eelco Dolstra 3d3c7fadf7 Merge pull request #1595 from peti/master
docker: ensure that the installation works for users other than 'root'
2017-10-12 13:02:19 +02:00
Eelco Dolstra fda7b95cb0 Fix a hang in ThreadPool
The worker threads could exit prematurely if they finished processing
all items while the main thread was still adding items. In particular,
this caused hanging nix-store --serve processes in the build farm.

Also, process items from the main thread.
2017-10-09 15:41:09 +02:00
Eelco Dolstra 838509d1a0 Whitespace 2017-10-09 15:41:09 +02:00
Eelco Dolstra 1893f78f8e Merge pull request #1594 from dtzWill/fix/busybox-builtins
release-common: Fix busybox builtins (busybox >= 1.27)
2017-10-09 12:09:33 +02:00
Peter Simons 37164ae389 docker: ensure that the installation works for users other than 'root', too
The path /root/.nix-profile is a sym-link to /nix/var/nix/profiles/default. The
latter path, however, works for everyone while the former path works only for
root, so we prefer the public path whenever possible.
2017-10-07 17:28:34 +02:00
Will Dietz 2f1a1c5a49 release-common: Fix busybox builtins (busybox >= 1.27)
See https://github.com/NixOS/nixpkgs/pull/28261
2017-10-07 07:43:55 -05:00
Eelco Dolstra b2e00fd5b7 Merge pull request #1593 from Mic92/bashism
fixing bashisms in test code
2017-10-06 14:58:13 +02:00
Jörg Thalheim e94fc238cf fixing bashisms in test code
This fixed the build on ubuntu/debian, where dash is the sh.
2017-10-06 06:12:33 -05:00
Shea Levy fb98e29067 darwin installer: Fix on systems where sudo -i is disabled. 2017-10-05 09:07:55 -07:00
Eelco Dolstra c6676ea253 Merge branch 'elegant-tests' of https://github.com/copumpkin/nix 2017-10-03 11:16:34 +02:00
Eelco Dolstra fce24b7d6c Merge pull request #1587 from copumpkin/fix-tests
Fix tests
2017-10-03 11:06:03 +02:00
Dan Peebles 60ecbd7934 More elegant test output
I got sick of trying to find the failures in the sea of debug output, so
we now:
- Hide test output unless it fails
- Sprinkle in some simple color
- Pad results for a more tabular look

If Nix is getting a more friendly user interface, we might as well get
a friendlier developer interface, right? :)
2017-10-03 00:59:32 -04:00
Dan Peebles 3139cad9cd Fix tests
`nix copy` no longer accepts a `--recursive` argument
2017-10-02 23:45:27 -04:00
Dan Peebles 476493dbf5 Reverse retry logic to retry in all but a few cases
It was getting too much like whac-a-mole listing all the retriable error
conditions, so we now retry by default and list the cases where retrying
is almost certainly hopeless.
2017-10-02 23:22:02 -04:00
Anthony Cowley 92f9d18aa0 install-darwin-multi-user: relax assumption check
The installer will error out if a user's shell configuration includes any mention of ~nix-profile~, even if this is in a comment. This change is designed to do the bare minimum to ignore lines beginning with a `#`.
2017-10-02 20:07:56 -04:00
Eelco Dolstra f3e0d46821 nix run: Restore signals
Otherwise Ctrl-C doesn't work.
2017-09-28 17:58:59 +02:00
Eelco Dolstra 5421ad243e nix copy: make recursive by default 2017-09-27 18:28:54 +02:00
Eelco Dolstra b8875213dc Tweak message 2017-09-27 13:21:25 +02:00
Eelco Dolstra bfef3ec855 nix build: Fix realising a store path 2017-09-27 13:14:45 +02:00
Eelco Dolstra d1c6e0fe8c nix why-depends: Write to stdout 2017-09-25 16:59:16 +02:00
Eelco Dolstra e8d6ee7c1b Add "nix show-derivation"
This debug command prints a store derivation in JSON format. For
example:

  $ nix show-derivation nixpkgs.hello
  {
    "/nix/store/ayjwpwwiyy04nh9z71rsdgd3q7bra7ch-hello-2.10.drv": {
      "outputs": {
        "out": {
          "path": "/nix/store/w5w4v29ql0qwqhczkdxs94ix2lh7ibgs-hello-2.10"
        }
      },
      "inputSrcs": [
        "/nix/store/9krlzvny65gdc8s7kpb6lkx8cd02c25b-default-builder.sh"
      ],
      "inputDrvs": {
        "/nix/store/13839aqdf6x4k3b785rw5f2l7857l6y3-bash-4.4-p12.drv": [
          "out"
        ],
        "/nix/store/vgdx7fdc7d4iirmnwj2py1nrvr5qwzj7-hello-2.10.tar.gz.drv": [
          "out"
        ],
        "/nix/store/x3kkd0vsqfflbvwf1055l9mr39bg0ms0-stdenv.drv": [
          "out"
        ]
      },
      "platform": "x86_64-linux",
      "builder": "/nix/store/qp5fw57d38bd1n07ss4zxh88zg67c3vg-bash-4.4-p12/bin/bash",
      "args": [
        "-e",
        "/nix/store/9krlzvny65gdc8s7kpb6lkx8cd02c25b-default-builder.sh"
      ],
      "env": {
        "buildInputs": "",
        "builder": "/nix/store/qp5fw57d38bd1n07ss4zxh88zg67c3vg-bash-4.4-p12/bin/bash",
        "configureFlags": "",
        "doCheck": "1",
        "name": "hello-2.10",
        "nativeBuildInputs": "",
        "out": "/nix/store/w5w4v29ql0qwqhczkdxs94ix2lh7ibgs-hello-2.10",
        "propagatedBuildInputs": "",
        "propagatedNativeBuildInputs": "",
        "src": "/nix/store/3x7dwzq014bblazs7kq20p9hyzz0qh8g-hello-2.10.tar.gz",
        "stdenv": "/nix/store/6zngq1rdh0ans9qyckqimqibgnlvlfrm-stdenv",
        "system": "x86_64-linux"
      }
    }
  }

This removes the need for pp-aterm.
2017-09-25 13:43:35 +02:00
Eelco Dolstra e350671737 Doh 2017-09-25 13:25:55 +02:00
Matthieu Coudron 6920c23701 Improve error message for conflicting priorities
I find the error message 'nix-env --set-flag priority NUMBER PKGNAME'
not as helpful as it could be :
- doesn't share the current priorities
- doesn't say that the command must be run on the already installed
PKGNAME (which is confusing the first time)
- the doc needs careful reading:
"If there are multiple derivations matching a name in args that have the same name (e.g., gcc-3.3.6 and gcc-4.1.1), then the derivation with the highest priority is used."
if one stops reading there, he is screwed. Salvation comes with reading "A derivation can define a priority by declaring the meta.priority attribute. This attribute should be a number, with a higher value denoting a lower priority. The default priority is 0."
To sum it up, lower number wins. I tried to convey this idea in the
message too.
2017-09-21 10:25:20 +09:00
Eelco Dolstra 84f112b1c8 nix-shell: Ignore readFile() errors
Fixes #1563.
2017-09-18 13:28:00 +02:00
Eelco Dolstra aca4f7dff0 Don't remove Content-Encoding in fetchurl / nix-prefetch-url
Fixes #1568.
2017-09-18 11:07:28 +02:00
Eelco Dolstra 5324bb9399 Update release notes 2017-09-18 11:07:17 +02:00
Eelco Dolstra 346aeee1cb Remove Debian 8 and Ubuntu 14.10
These have a GCC (4.9) that is too old.

https://hydra.nixos.org/eval/1391740
2017-09-14 18:56:33 +02:00
Eelco Dolstra 89dc62c174 RemoteStore: Add option to drop old connections from the pool
This is a hack to make hydra-queue-runner free its temproots
periodically, thereby ensuring that garbage collection of the
corresponding paths is not blocked until the queue runner is
restarted.

It would be better if temproots could be released earlier than at
process exit. I started working on a RAII object returned by functions
like addToStore() that releases temproots. However, this would be a
pretty massive change so I gave up on it for now.
2017-09-14 18:10:38 +02:00
Eelco Dolstra 308ecf6361 Show the pid of temporary roots 2017-09-14 15:02:52 +02:00
Eelco Dolstra da1e4fdfb5 nix-store -q --roots / --gc --print-roots: Print temporary / in-memory roots
For example,

  $ nix-store -q --roots /nix/store/7phd2sav7068nivgvmj2vpm3v47fd27l-patchelf-0.8pre845_0315148
  {temp:1}

denotes that the path is only being kept alive by a temporary root
(i.e. /nix/var/nix/temproots/). Similarly,

  $ nix-store --gc --print-roots
  ...
  {memory:9} -> /nix/store/094gpjn9f15ip17wzxhma4r51nvsj17p-curl-7.53.1

shows that curl is being used by some process.
2017-09-14 14:38:36 +02:00
Eelco Dolstra 970366266b Add "nix add-to-store" command 2017-09-14 13:29:33 +02:00
Eelco Dolstra 1c58ad2ffa nix why-depends: Fix showing self-references 2017-09-14 11:17:32 +02:00
Eelco Dolstra 359ede1d72 Merge pull request #1562 from peti/master
docker: update to Nix 1.11.14
2017-09-11 16:44:16 +02:00
Eelco Dolstra fc0ded3408 nix why-depends: Add option to show all edges causing a dependency
For example, without --all:

  $ nix why-depends nixpkgs.nixUnstable nixpkgs.libssh2
  /nix/store/s9n5gvj2l49b4n19nz6xl832654nf7n7-nix-1.12pre5511_c94f3d55
  └───lib/libnixstore.so: …/lib:/nix/store/w9ykqpl5v0r3vfwsgn408jqhs72cx96x-curl-7.55.0/lib…
      => /nix/store/w9ykqpl5v0r3vfwsgn408jqhs72cx96x-curl-7.55.0
      └───lib/libcurl.la: …ib -L/nix/store/4mbayl1y5hpjbjzkx8ndyhkv98kqw1wi-libssh2-1.8.0/l…
          => /nix/store/4mbayl1y5hpjbjzkx8ndyhkv98kqw1wi-libssh2-1.8.0

but with --all:

  $ nix why-depends -a nixpkgs.nixUnstable nixpkgs.libssh2
  /nix/store/s9n5gvj2l49b4n19nz6xl832654nf7n7-nix-1.12pre5511_c94f3d55
  ├───lib/libnixstore.so: …/lib:/nix/store/w9ykqpl5v0r3vfwsgn408jqhs72cx96x-curl-7.55.0/lib…
  │   => /nix/store/w9ykqpl5v0r3vfwsgn408jqhs72cx96x-curl-7.55.0
  │   └───lib/libcurl.la: …ib -L/nix/store/4mbayl1y5hpjbjzkx8ndyhkv98kqw1wi-libssh2-1.8.0/l…
  │       lib/libcurl.so.4.4.0: …/lib:/nix/store/4mbayl1y5hpjbjzkx8ndyhkv98kqw1wi-libssh2-1.8.0/l…
  │       => /nix/store/4mbayl1y5hpjbjzkx8ndyhkv98kqw1wi-libssh2-1.8.0
  └───lib/libnixstore.so: …/lib:/nix/store/bx2i9vi76lps6w9rr73fxf6my31s4dg5-aws-sdk-cpp-1.0…
      => /nix/store/bx2i9vi76lps6w9rr73fxf6my31s4dg5-aws-sdk-cpp-1.0.153
      └───lib/libaws-cpp-sdk-core.so: …e.so./nix/store/w9ykqpl5v0r3vfwsgn408jqhs72cx96x-curl-7.55.0/lib…
          lib/libaws-cpp-sdk-s3.so: …/lib:/nix/store/w9ykqpl5v0r3vfwsgn408jqhs72cx96x-curl-7.55.0/lib…
          => /nix/store/w9ykqpl5v0r3vfwsgn408jqhs72cx96x-curl-7.55.0
2017-09-11 16:21:27 +02:00
Peter Simons 1511f01351 docker: update to Nix 1.11.14
- Use the latest Nix version 1.11.14.

- Attempts to download the Nix installation tarball from http://nixos.org
  redirect to https these days, which wget doesn't support unless OpenSSL is
  available.

- Use addgroup and adduser commands to create the Nix build users.

- Link the Nix profile script into /etc/profile.d, where it's run
  automatically.

- Dropped installation of bash and tar. Neither tool is essential for running
  Nix.

Use the command "docker build -t nix . && docker run -it --rm nix sh -"
to build and run the Nix docker container.
2017-09-11 14:26:14 +02:00
Eelco Dolstra d41c5eb13f Add command "nix why-depends"
This command shows why a package has another package in its runtime
closure. For example, to see why VLC has libdrm.dev in its closure:

  $ nix why-depends nixpkgs.vlc nixpkgs.libdrm.dev
  /nix/store/g901z9pcj0n5yy5n6ykxk3qm4ina1d6z-vlc-2.2.5.1:
    lib/libvlccore.so.8.0.0: …nfig:/nix/store/405lmx6jl8lp0ad1vrr6j498chrqhz8g-libdrm-2.4.75-d…

  /nix/store/s3nm7kd8hlcg0facn2q1ff2n7wrwdi2l-mesa-noglu-17.0.7-dev:
    nix-support/propagated-native-build-inputs: …-dev /nix/store/405lmx6jl8lp0ad1vrr6j498chrqhz8g-libdrm-2.4.75-d…

Thus, VLC's lib/libvlccore.so.8.0.0 as well as mesa-noglu's
nix-support/propagated-native-build-inputs cause the dependency.
2017-09-10 22:40:33 +02:00
Eelco Dolstra 8af704eef1 Typo 2017-09-10 22:40:33 +02:00
Eelco Dolstra ad228d84e5 nix build: Only download the requested derivation outputs
Also some refactoring.
2017-09-10 22:40:33 +02:00
Eelco Dolstra 3ed8290e53 LegacySSHStore: Include signatures etc. 2017-09-08 16:55:27 +02:00
Eelco Dolstra e02edb1483 nix copy: Add --substitute flag 2017-09-08 15:32:07 +02:00
Eelco Dolstra b7376edf06 ThreadPool: On exception, interrupt the other worker threads 2017-09-08 15:31:24 +02:00
Eelco Dolstra 6a888ec29a copyStorePath(): Fill in missing narHash regardless of checkSigs
I don't remember what the reasoning was here, but security is provided
by the signatures, not by whether the hash is provided by the other
store.
2017-09-08 14:48:08 +02:00
Eelco Dolstra 3cf1705583 nix copy: Don't open the --from store twice 2017-09-08 14:46:55 +02:00
Eelco Dolstra 8f6b347abd ThreadPool: Improve exception handling
In particular, process() won't return as long as there are active
items. This prevents work item lambdas from referring to stack frames
that no longer exist.
2017-09-08 14:42:09 +02:00
Eelco Dolstra a2740c9ca2 nix copy: Add examples 2017-09-08 11:33:46 +02:00
Eelco Dolstra 88e6bb76de Communicate the structured log FD to builders
Since we may use a dedicated file descriptor in the future, this
allows us to change it. So builders can do

  if [[ -n $NIX_LOG_FD ]]; then
    echo "@nix { message... }" >&$NIX_LOG_FD
  fi
2017-09-08 11:27:10 +02:00
Eelco Dolstra 24e23a1a73 nix search: Add examples 2017-09-07 20:42:11 +02:00
Eelco Dolstra 4967f0509a nix eval: Add examples 2017-09-07 20:37:46 +02:00
Eelco Dolstra 2ebeffcfd4 nix log: Add examples 2017-09-07 20:18:29 +02:00
Eelco Dolstra 17afc42895 nix build: Add examples 2017-09-07 20:14:04 +02:00
Eelco Dolstra 2e9b7c4cb2 nix run: Add some examples 2017-09-07 20:09:04 +02:00
Eelco Dolstra dff440aab3 nix build: Add --out-link and --no-link options 2017-09-06 16:20:34 +02:00
Eelco Dolstra df4342bc17 nix build: Create result symlinks 2017-09-06 16:03:22 +02:00
Eelco Dolstra 1277aab219 Fix abort when the GC thread gets an exception 2017-09-06 11:37:58 +02:00
Eelco Dolstra 0b606aad46 Add automatic garbage collection
Nix can now automatically run the garbage collector during builds or
while adding paths to the store. The option "min-free = <bytes>"
specifies that Nix should run the garbage collector whenever free
space in the Nix store drops below <bytes>. It will then delete
garbage until "max-free" bytes are available.

Garbage collection during builds is asynchronous; running builds are
not paused and new builds are not blocked. However, there also is a
synchronous GC run prior to the first build/substitution.

Currently, no old GC roots are deleted (as in "nix-collect-garbage
-d").
2017-09-05 20:43:42 +02:00
Eelco Dolstra b932ea58ec GC: Don't delete own temproots file
Since file locks are per-process rather than per-file-descriptor, the
garbage collector would always acquire a lock on its own temproots
file and conclude that it's stale.
2017-09-05 20:39:57 +02:00
Eelco Dolstra 8215b75d36 Abort curl downloads if there is no progress for 5 minutes
Maybe this will fix the curl hangs on macOS. (We could also use
CURLOPT_TIMEOUT but that seems more of a sledgehammer.)
2017-09-01 16:51:26 +02:00
Eelco Dolstra 7a108d904e Fix verbosity level for nix build --dry-run 2017-09-01 12:37:09 +02:00
Eelco Dolstra fe38fce2d8 Merge pull request #1545 from trofi/glibc-2.26
src/libmain/stack.cc: fix 'ucontext' usage on glibc-2.26
2017-09-01 11:12:30 +02:00
Sergei Trofimovich c9857ef262 src/libmain/stack.cc: fix 'ucontext' usage on glibc-2.26
Build fails as:

$ make
  CXX    src/libmain/stack.o
src/libmain/stack.cc: In function 'void nix::sigsegvHandler(int, siginfo_t*, void*)':
src/libmain/stack.cc:21:21: error: 'ucontext' was not declared in this scope
     sp = (char *) ((ucontext *) ctx)->uc_mcontext.gregs[REG_RSP];
                     ^~~~~~~~
src/libmain/stack.cc:21:21: note: suggested alternative: 'ucontext_t'
     sp = (char *) ((ucontext *) ctx)->uc_mcontext.gregs[REG_RSP];
                     ^~~~~~~~
                     ucontext_t

It's caused by upstream rename:
https://sourceware.org/git/?p=glibc.git;a=commitdiff;h=251287734e89a52da3db682a8241eb6bccc050c9

which basically changes
    typedef struct ucontext {} ucontext_t;
to
    typedef struct ucontext_t {} ucontext_t;

The change uses ucontext_t.

Signed-off-by: Sergei Trofimovich <slyfox@gentoo.org>
2017-08-31 21:41:36 +01:00
Eelco Dolstra bbdf08bc0f Call queryMissing() prior to building
Without this, substitute info is fetched sequentially, which is
superslow. In the old UI (e.g. nix-build), we call printMissing(),
which calls queryMissing(), thereby preheating the binary cache
cache. But the new UI doesn't do that.
2017-08-31 16:05:07 +02:00
Eelco Dolstra fd73c1e20a Add an activity for binary cache queries 2017-08-31 15:25:58 +02:00
Eelco Dolstra c2154d4c84 Rename a few configuration options
In particular, drop the "build-" and "gc-" prefixes which are
pointless. So now you can say

  nix build --no-sandbox

instead of

  nix build --no-build-use-sandbox
2017-08-31 14:28:25 +02:00
Eelco Dolstra 7d4a7136db More macOS build fixes 2017-08-31 12:52:07 +02:00
Eelco Dolstra fabde432dc Fix build failure on non-Linux
https://hydra.nixos.org/build/59649086
2017-08-31 11:05:18 +02:00
Eelco Dolstra 9f47eac92b Fix mismatched tag warning
https://hydra.nixos.org/build/59649086
2017-08-31 11:04:52 +02:00
Eelco Dolstra e9c07a3b26 nix edit / log: Operate on a single Installable 2017-08-29 16:18:00 +02:00
Eelco Dolstra 9b82ecbae0 nix search: Warn about cached results 2017-08-29 15:22:05 +02:00
Eelco Dolstra c8235c5313 nix run: Flush the progress bar before starting the command 2017-08-29 15:13:30 +02:00
Eelco Dolstra 05d68a6e23 nix run: Add some flags for clearing/keeping the environment
This is useful for testing commands in isolation.

For example,

  $ nix run nixpkgs.geeqie -i -k DISPLAY -k XAUTHORITY -c geeqie

runs geeqie in an empty environment, except for $DISPLAY and
$XAUTHORITY.
2017-08-29 15:00:08 +02:00
Eelco Dolstra 5cc8609e30 nix run: Allow passing a command to execute
E.g.

  nix run nixpkgs.hello -c hello --greeting Hallo

Note that unlike "nix-shell --command", no quoting of arguments is
necessary.

"-c" (short for "--command") cannot be combined with "--" because they
both consume all remaining arguments. But since installables shouldn't
start with a dash, this is unlikely to cause problems.
2017-08-29 14:42:48 +02:00
Eelco Dolstra 93a5ef0516 nix run: Fix chroot execution
Running "nix run" with a diverted store, e.g.

  $ nix run --store local?root=/tmp/nix nixpkgs.hello

stopped working when Nix became multithreaded, because
unshare(CLONE_NEWUSER) doesn't work in multithreaded processes. The
obvious solution is to terminate all other threads first, but 1) there
is no way to terminate Boehm GC marker threads; and 2) it appears that
the kernel has a race where unshare(CLONE_NEWUSER) will still fail for
some indeterminate amount of time after joining other threads.

So instead, "nix run" will now exec() a single-threaded helper ("nix
__run_in_chroot") that performs the actual unshare()/chroot()/exec().
2017-08-29 13:21:07 +02:00
Eelco Dolstra 1c58e13bee Hide commands that don't have a description
These are assumed to be internal.
2017-08-29 11:52:55 +02:00
Eelco Dolstra 05c45f301d readLink(): Handle symlinks in /proc
Symlinks like /proc/self/exe report a stat() size of 0, so use a
buffer of at least PATH_MAX instead.
2017-08-29 11:52:34 +02:00
Eelco Dolstra 2cc345b95f Give activities a verbosity level again
And print them (separately from the progress bar) given sufficient -v
flags.
2017-08-28 19:13:24 +02:00
Eelco Dolstra cfc8132391 Don't send progress messages to older clients 2017-08-28 18:54:23 +02:00
Eelco Dolstra fe34b91289 Tunnel progress messages from the daemon to the client
This makes the progress bar work for non-root users.
2017-08-28 18:49:42 +02:00
Eelco Dolstra e681b1f064 Simplify 2017-08-28 14:30:35 +02:00
Eelco Dolstra 8fff3e7bb5 Make TunnelLogger thread-safe
Now that we use threads in lots of places, it's possible for
TunnelLogger::log() to be called asynchronously from other threads
than the main loop. So we need to ensure that STDERR_NEXT messages
don't clobber other messages.
2017-08-28 14:17:07 +02:00
Eelco Dolstra 94a0548dc4 Simplify 2017-08-25 21:26:37 +02:00
Eelco Dolstra 9b845e6936 Doh 2017-08-25 20:52:34 +02:00
Eelco Dolstra 0ac35b67b8 Allow derivations to update the build phase
So the progress bar can show

  [1/0/1 built, 0.0 MiB DL] building hello-2.10 (configuring): checking whether pread is declared without a macro... yes
2017-08-25 18:04:05 +02:00
Eelco Dolstra c137c0a5eb Allow activities to be nested
In particular, this allows more relevant activities ("substituting X")
to supersede inferior ones ("downloading X").
2017-08-25 17:49:40 +02:00
Eelco Dolstra f194629f96 Fix Debian build
https://hydra.nixos.org/build/59390148
2017-08-25 16:11:18 +02:00
Eelco Dolstra db1d45037c Handle SIGWINCH 2017-08-25 15:59:03 +02:00
Eelco Dolstra ec9e0c03c3 When truncating the progress bar, take ANSI escape sequences into account 2017-08-25 15:59:03 +02:00
Eelco Dolstra a3015db6c3 Typo 2017-08-25 15:59:03 +02:00
Eelco Dolstra 0e9ddcc306 Restore activity metadata
This allows the progress bar to display "building perl-5.22.3" instead
of "building /nix/store/<hash>-perl-5.22.3.drv".
2017-08-25 15:58:35 +02:00
Eelco Dolstra 1f56235438 Clean up JSON construction 2017-08-21 12:18:46 +02:00
Eelco Dolstra 4af2611bd1 Allow builders to create activities
Actually, currently they can only create download activities. Thus,
downloads by builtins.fetchurl show up in the progress bar.
2017-08-21 12:18:46 +02:00
Eelco Dolstra 4c6a26539c Remove debug line 2017-08-21 12:18:46 +02:00
Eelco Dolstra 9400cb36b7 Disallow accidental copy construction 2017-08-21 12:18:42 +02:00
Eelco Dolstra 37db080644 Merge pull request #1529 from matthewbauer/remove-nix-mode
Remove nix-mode.el from Nix.
2017-08-21 10:41:21 +02:00
Matthew Bauer 2c75945de5 Remove nix-mode.el from Nix.
This removes the file nix-mode.el from Nix. The file is now available within the
repository https://github.com/NixOS/nix-mode.

Fixes #662
Fixes #1040
Fixes #1054
Fixes #1055
Closes #1119
Fixes #1419

NOTE: all of the above should be fixed within NixOS/nix-mode. If one of those
hasn’t please reopen within NixOS/nix-mode and not within NixOS/nix.
2017-08-19 21:16:30 -07:00
Domen Kožar 898a3f729c Merge pull request #1527 from chaseadamsio/master
update MD5 to SHA-256 in expression-syntax
2017-08-19 14:02:06 +02:00
Chase Adams 09a38f9125 update MD5 to SHA-256 in expression-syntax 2017-08-18 16:07:33 -07:00
Eelco Dolstra e56e790642 Merge pull request #1525 from wingo/remove-buggy-octal-decoder
Remove unused decodeOctalEscaped
2017-08-18 12:01:41 +02:00
Andy Wingo 6bb4e3e8fe Remove unused decodeOctalEscaped
Besides being unused, this function has a bug that it will incorrectly
decode the path component Ubuntu\04016.04.2\040LTS\040amd64 as
"Ubuntu.04.2 LTS amd64" instead of "Ubuntu 16.04.2 LTS amd64".
2017-08-18 11:07:00 +02:00
Eelco Dolstra e2f9a61dc9 Update release notes 2017-08-16 21:36:47 +02:00
Eelco Dolstra 2ee1b9359b Merge branch 'tokenize' of https://github.com/nbp/nix 2017-08-16 21:21:36 +02:00
Eelco Dolstra c2cab20732 nix verify: Restore the progress indicator 2017-08-16 20:56:03 +02:00
Eelco Dolstra b4ed97e3a3 nix optimise-store: Show how much space has been freed 2017-08-16 20:56:03 +02:00
Eelco Dolstra 23b8b7e096 nix optimise-store: Add
This replaces "nix-store --optimise". Main difference is that it has a
progress indicator.
2017-08-16 20:56:03 +02:00
Eelco Dolstra 40bffe0a43 Progress indicator: Cleanup 2017-08-16 20:56:03 +02:00
Eelco Dolstra dff12b38f9 Progress indicator: More improvements 2017-08-16 20:56:03 +02:00
Eelco Dolstra bf1f123b09 Progress indicator: Show number of active items 2017-08-16 20:56:03 +02:00
Eelco Dolstra 0e0dcf2c7e Progress indicator: Unify "copying" and "substituting"
They're the same thing after all.

Example:

  $ nix build --store local?root=/tmp/nix nixpkgs.firefox-unwrapped
  [0/1 built, 49/98 copied, 16.3/92.8 MiB DL, 55.8/309.2 MiB copied] downloading 'https://cache.nixos.org/nar/0pl9li1jigcj2dany47hpmn0r3r48wc4nz48v5mqhh426lgz3bz6.nar.xz'
2017-08-16 20:56:03 +02:00
Eelco Dolstra c36467ad2e Improve substitution progress indicator
E.g.

  $ nix build --store local?root=/tmp/nix nixpkgs.firefox-unwrapped
  [0/1 built, 1/97/98 fetched, 65.8/92.8 MiB DL, 203.2/309.2 MiB copied] downloading 'https://cache.nixos.org/nar/1czm9fk0svacy4h6a3fzkpafi4f7a9gml36kk8cq1igaghbspg3k.nar.xz'
2017-08-16 20:56:02 +02:00
Eelco Dolstra b29b6feaba nix copy: Improve progress indicator
It now shows the amount of data copied:

  [8/1038 copied, 160.4/1590.9 MiB copied] copying path '...'
2017-08-16 20:56:02 +02:00
Eelco Dolstra c5e4404580 nix copy: Revive progress bar 2017-08-16 20:56:02 +02:00
Eelco Dolstra dffc3fe43b nix copy: Add --no-check-sigs flag 2017-08-16 20:56:02 +02:00
Eelco Dolstra e4bd42f98f Disallow SSH host names starting with a dash 2017-08-16 20:55:58 +02:00
Nicolas B. Pierron b8867a0239 Add builtins.string function.
The function 'builtins.split' takes a POSIX extended regular expression
and an arbitrary string. It returns a list of non-matching substring
interleaved by lists of matched groups of the regular expression.

```nix
with builtins;
assert split "(a)b" "abc"      == [ "" [ "a" ] "c" ];
assert split "([ac])" "abc"    == [ "" [ "a" ] "b" [ "c" ] "" ];
assert split "(a)|(c)" "abc"   == [ "" [ "a" null ] "b" [ null "c" ] "" ];
assert split "([[:upper:]]+)" "  FOO   "
                               == [ "  " [ "FOO" ] "   " ];
```
2017-08-15 20:04:11 +00:00
Eelco Dolstra 82c4b37c6f Merge pull request #1517 from nbp/move-match-doc
Move builtins.match documentation between map and mul.
2017-08-15 20:40:48 +02:00
Nicolas B. Pierron 27417c6160 Move builtins.match documentation between map and mul. 2017-08-15 18:34:24 +00:00
Eelco Dolstra f76e85d8f5 Start of 1.12 release notes 2017-08-10 18:44:45 +02:00
Eelco Dolstra b6ee5e5bf0 Style fix 2017-08-10 13:51:07 +02:00
Eelco Dolstra af765a8eab Use /proc/self/fd to efficiently close all FDs on Linux
Issue #1506.
2017-08-09 16:22:05 +02:00
Eelco Dolstra c6184dec6c nix repl: Support printing floating-point numbers 2017-08-09 15:17:29 +02:00
Eelco Dolstra eb1d1ca780 Merge pull request #1499 from puffnfresh/bug/arm-fixes
Include missing <cstdlib> for abort()
2017-08-09 15:15:09 +02:00
Brian McKenna b39cc4fc81 Include missing <cstdlib> for abort()
This is needed to get Nix compiled using Android NDK.
2017-08-03 07:03:22 +10:00
Eelco Dolstra a2778988f2 Merge branch 'macOS' of https://github.com/davidak/nix 2017-07-31 10:31:51 +02:00
Eelco Dolstra 561e977f51 Merge branch 'quotes' of https://github.com/Mic92/nix-1 2017-07-31 10:21:40 +02:00
Jörg Thalheim 2fd8f8bb99 Replace Unicode quotes in user-facing strings by ASCII
Relevant RFC: NixOS/rfcs#4

$ ag -l | xargs sed -i -e "/\"/s/’/'/g;/\"/s/‘/'/g"
2017-07-30 12:32:45 +01:00
davidak fcb8d6a7a0 replace "OS X" with "macOS" 2017-07-30 12:28:50 +02:00
davidak 92bcb61127 replace "Mac OS X" with "macOS"
except in older release notes where the name was actually Mac OS X.
2017-07-30 12:26:17 +02:00
Eelco Dolstra c7654bc491 nix-build: Fix regression causing all outputs to be built 2017-07-28 15:17:52 +02:00
Eelco Dolstra af4689f9e9 nix-prefetch-url: Fix regression in hash printing 2017-07-28 14:56:39 +02:00
Eelco Dolstra 7480f4f9a4 builtins.fetchgit: Support specifying commit hashes
This adds an argument "rev" specififying the Git commit hash. The
existing argument "rev" is renamed to "ref". The default value for
"ref" is "master". When specifying a hash, it's necessary to specify a
ref since we're not cloning the entire repository but only fetching a
specific ref.

Example usage:

  builtins.fetchgit {
    url = https://github.com/NixOS/nixpkgs.git;
    ref = "release-16.03";
    rev = "c1c0484041ab6f9c6858c8ade80a8477c9ae4442";
  };
2017-07-27 18:08:23 +02:00
Eelco Dolstra 9f64cb89cb builtins.fetchgit: Respect tarball-ttl
I.e. if the local ref is more recent than tarball-ttl seconds, then
don't check the remote.
2017-07-27 17:23:29 +02:00
Eelco Dolstra 69deca194e builtins.fetchgit: Use proper refs locally 2017-07-27 17:02:25 +02:00
Eelco Dolstra 6d7de7f3de builtins.fetchgit: Cache hash -> store path mappings
This prevents an expensive call to addToStore() in the cached case.
2017-07-27 16:16:08 +02:00
Eelco Dolstra 57b9505731 nix search: Add a cache
The package list is now cached in
~/.cache/nix/package-search.json. This gives a substantial speedup to
"nix search" queries. For example (on an SSD):

First run: (no package search cache, cold page cache)

  $ time nix search blender
  Attribute name: nixpkgs.blender
  Package name: blender
  Version: 2.78c
  Description: 3D Creation/Animation/Publishing System

  real    0m6.516s

Second run: (package search cache populated)

  $ time nix search blender
  Attribute name: nixpkgs.blender
  Package name: blender
  Version: 2.78c
  Description: 3D Creation/Animation/Publishing System

  real    0m0.143s
2017-07-26 17:29:10 +02:00
Eelco Dolstra 4c9ff89c26 nix-build/nix-shell: Eliminate call to nix-instantiate / nix-store
Note that this removes the need for a derivation symlink, so the
--drv-path and --add-drv-link flags now do nothing.
2017-07-26 17:29:09 +02:00
Eelco Dolstra c94f3d5575 nix-shell: Use bashInteractive from <nixpkgs>
This adds about 0.1s to nix-shell runtime in the case where
bashInteractive already exists.

See discussion at https://github.com/NixOS/nixpkgs/issues/27493.
2017-07-20 13:50:25 +02:00
Eelco Dolstra 57a30e101b nix search: Ignore top-level eval errors
$NIX_PATH may contain elements that don't evaluate to an attrset (like
"nixos-config"), so ignore those.
2017-07-20 13:33:13 +02:00
Eelco Dolstra fc3568e263 FD_SETSIZE check: BuildError -> Error
BuildError denotes a permanent build failure, which is not the case
here.
2017-07-20 13:33:13 +02:00
Eelco Dolstra b144c4d617 nix search: Add --json flag 2017-07-20 13:33:13 +02:00
Eelco Dolstra 90825dea51 Add "nix search" command 2017-07-20 13:33:13 +02:00
Eelco Dolstra 3162ad5ff4 Merge pull request #1471 from veprbl/fdsetsize
Do not try to fill fd_set with fd>=FD_SETSIZE
2017-07-19 11:15:19 +02:00
Dmitry Kalinkin d5e1bffd2a Do not try to fill fd_set with fd>=FD_SETSIZE
This is UB and causes buffer overflow and crash on linux.
2017-07-18 17:51:50 -04:00
Domen Kožar 72462b4b6e Merge pull request #1470 from grahamc/patch-1
Update mailing list.
2017-07-18 14:10:58 +02:00
Graham Christensen 364f75e03a Update mailing list. 2017-07-18 08:04:01 -04:00
Graham Christensen da2ad30054 Update the mailing list. 2017-07-18 08:02:53 -04:00
Eelco Dolstra bf6792c0df Always use base-16 for hashed mirror lookups
In particular, don't use base-64, which we don't support. (We do have
base-32 redirects for hysterical reasons.)

Also, add a test for the hashed mirror feature.
2017-07-17 13:13:18 +02:00
Eelco Dolstra 49304bae81 Make the hashes mirrors used by builtins.fetchurl configurable
In particular, this allows it to be disabled in our tests.
2017-07-17 13:07:08 +02:00
Eelco Dolstra 4ec6eb1fdf Fix accidental printError 2017-07-17 11:38:15 +02:00
Domen Kožar f3e432305a Merge pull request #1467 from alicebob/nomd5
use sha256 hashes in the examples
2017-07-16 10:58:30 +02:00
Harmen 1e0f59ae14 use sha256 hashes in the examples
And fix a dead link.
2017-07-15 21:06:30 +02:00
Eelco Dolstra 766ad5db3b nix path-info: Show download sizes for binary cache stores
E.g.

  $ nix path-info --json --store https://cache.nixos.org nixpkgs.thunderbird -S
  ...
      "downloadHash": "sha256:1jlixpzi225wwa0f4xdrwrqgi47ip1qpj9p06fyxxg07sfmyi4q0",
      "downloadSize": 43047620,
      "closureDownloadSize": 84745960
    }
  ]
2017-07-14 18:29:10 +02:00
Eelco Dolstra fdc9da034f Avoid a call to derivationFromPath()
This doesn't work in read-only mode, ensuring that operations like

  nix path-info --store https://cache.nixos.org -S nixpkgs.hello

(asking for the closure size of nixpkgs.hello in cache.nixos.org) work
when nixpkgs.hello doesn't exist in the local store.
2017-07-14 18:29:10 +02:00
Eelco Dolstra 3908d3929c nix path-info: Don't barf on invalid paths
Now you get

  [
    {
      "path": "/nix/store/fzvliz4j5xzvnd0w5zgw2l0ksqh578yk-bla",
      "valid": false
    }
  ]
2017-07-14 18:29:10 +02:00
Eelco Dolstra 6438ba22af StorePathsCommand: Don't build installables
On second though this was annoying. E.g. "nix log nixpkgs.hello" would
build/download Hello first, even though the log can be fetched
directly from the binary cache.

May need to revisit this.
2017-07-14 18:29:07 +02:00
Eelco Dolstra 15e8bd3bcb Merge pull request #1466 from grahamc/nix-master-darwin-install
Multi-user installer for Darwin (master edition)
2017-07-14 18:25:02 +02:00
Graham Christensen e0d39c8dc4 Rename PINCH_ME_IM_SILLY to ALLOW_PREEXISTING_INSTALLATION 2017-07-14 12:11:33 -04:00
Graham Christensen 12f6bb33d2 If there is no TTY, also skip verbose sudo messages 2017-07-14 12:11:30 -04:00
Graham Christensen ce2281e6d8 Ensure PINCH_ME_IM_SILLY allows a /nix/store to stick around between builds
Also output in the status report that the user is very silly
2017-07-14 12:11:26 -04:00
Graham Christensen 1c7ce2a018 Assume yes if we have no TTY
Starve the TTY of input to ensure this works, but provide yes to the
current installer to handle the current broken case.
2017-07-14 12:11:23 -04:00
Graham Christensen 73a57a2f22 Cleanup and more specificity around set -e 2017-07-14 12:11:20 -04:00
Graham Christensen 3839dda2ec Only clean if the file exists 2017-07-14 12:11:17 -04:00
Graham Christensen 92ca93528f Clean up nix hints from the old insstaller 2017-07-14 12:11:13 -04:00
Graham Christensen 61ea9e9867 Run nix-build inside a fresh bash login 2017-07-14 12:11:10 -04:00
Graham Christensen bc647fd299 chmod 2017-07-14 12:11:07 -04:00
Graham Christensen a0369b14f4 Test the installer 2017-07-14 12:11:04 -04:00
Graham Christensen 657b47e1b3 Address feedback around printf & exec 2017-07-14 12:11:00 -04:00
Graham Christensen 6a4037ca05 Don't install a second nix after the initial installation, and the rsync change fixes a bug hidden by the nix replacement where the store files were being owned by the installing user due to rsync's -a implying -og. 2017-07-14 12:10:57 -04:00
Graham Christensen 092f447c6d Clean up issues around uninstall directions, and only show
relevant directions
2017-07-14 12:10:54 -04:00
Graham Christensen 6f639943c2 Prompt for sudo before validating assumptions, and check ourselves for root-owned files instead of making a scary warning. 2017-07-14 12:10:51 -04:00
Graham Christensen 2b5ab03524 multi-user install: move the profile in to the nix etc/profiles.d output 2017-07-14 12:10:47 -04:00
Graham Christensen fb40d73e23 Switch to a fancy multi-user installer on Darwin 2017-07-14 12:10:44 -04:00
Graham Christensen a0ad8ba12e Shellcheck the existing installer 2017-07-14 11:42:33 -04:00
Eelco Dolstra 112ff7833d nix: Show help when no arguments are given
Fixes #1464.
2017-07-14 13:44:45 +02:00
Eelco Dolstra 38374a9d35 Tarball job: Include libseccomp on Linux only 2017-07-14 11:41:37 +02:00
Eelco Dolstra 0681f8c907 Shut up a memory leak warning 2017-07-14 11:40:57 +02:00
Eelco Dolstra 2965d40612 replaceSymlink(): Handle the case where the temporary file already exists
Not really necessary anymore for #849, but still nice to have.
2017-07-11 23:21:40 +02:00
Eelco Dolstra 8e8caf7f3e fetchTarball: Prevent concurrent downloads of the same file
Fixes #849.
2017-07-11 23:21:24 +02:00
Eelco Dolstra 9c00fa4179 Merge pull request #1422 from nh2/fix-potential-hash-comparison-crash
Fix potential crash/wrong result two hashes of unequal length are compared
2017-07-10 18:09:49 +02:00
Shea Levy 62a8fe6388 Merge branch 'man2' of git://github.com/robx/nix 2017-07-10 08:43:19 -04:00
Robert Vollmert 30117fb35b fix buggy nix-shell man page 2017-07-10 14:36:55 +02:00
Eelco Dolstra 1762b9616c Merge pull request #1428 from rimmington/clearer-regex-space-error
Clearer error message when regex exceeds space limit
2017-07-10 11:45:05 +02:00
Rhys 17bb00d378 Clearer error message when regex exceeds space limit 2017-07-10 09:35:53 +10:00
Robert Vollmert c85e662004 man page (nix-shell): Fix grouping of -p option
Not sure about the raw ellipsis.
2017-07-07 22:11:46 +02:00
Robert Vollmert 89771a8821 man page (nix-prefetch-url): Add some missing options 2017-07-07 22:11:46 +02:00
Robert Vollmert 772ef22c25 man page (nix-instantiate): -E is optional 2017-07-07 22:11:46 +02:00
Robert Vollmert 8ad898b2cd man page (nix-instantiate): Add --json to synopsis, order variables 2017-07-07 22:11:46 +02:00
Robert Vollmert b1f5995a20 man page (nix-instantiate): Remove non-existent nix-build argument -r 2017-07-07 22:11:46 +02:00
Robert Vollmert 56a1f8f499 man pages: Consistently separate alternatives by / 2017-07-07 22:11:46 +02:00
Robert Vollmert d1643bdaa2 man pages: Argument for --max-jobs 2017-07-07 22:11:45 +02:00
Robert Vollmert 68c626c6b0 man pages: Grouping for option alternatives 2017-07-07 22:11:45 +02:00
Robert Vollmert ce3095e141 glossary: Fix word order 2017-07-07 22:07:46 +02:00
Eelco Dolstra d3713716b6 Merge pull request #1445 from matthewbauer/macos-skip-hardlink
Don’t hardlink disallowed paths in OS X.
2017-07-07 11:05:21 +02:00
Eelco Dolstra eef09c220d Merge pull request #1444 from robx/man
Fix nix-instantiate manpage indentation
2017-07-07 11:04:06 +02:00
Matthew Bauer 72e80c59b5 Don’t hardlink disallowed paths in OS X.
Fixes #1443
2017-07-06 19:30:19 -07:00
Robert Vollmert 01722b3d2c Remove unused variable from test script 2017-07-06 22:37:53 +02:00
Robert Vollmert 60da5d2b8f Fix nix-instantiate manpage indentation
The second command variant is now its own cmdsynopsis, which ensures
it's not indented as was the case using sbrk.
2017-07-06 22:35:36 +02:00
Eelco Dolstra a3dc1e65ab Add X32 to the seccomp filter
Fixes #1432.
2017-07-04 19:00:51 +02:00
Eelco Dolstra 42c5774e78 Sort substituters by priority
Fixes #1438.
2017-07-04 16:34:53 +02:00
Eelco Dolstra b7203e853e getDefaultSubstituters(): Simplify initialisation
As shlevy pointed out, static variables in C++11 have thread-safe
initialisation.
2017-07-04 16:26:48 +02:00
Eelco Dolstra 6cf23c3e8f Add allow-new-privileges option
This allows builds to call setuid binaries. This was previously
possible until we started using seccomp. Turns out that seccomp by
default disallows processes from acquiring new privileges. Generally,
any use of setuid binaries (except those created by the builder
itself) is by definition impure, but some people were relying on this
ability for certain tests.

Example:

  $ nix build '(with import <nixpkgs> {}; runCommand "foo" {} "/run/wrappers/bin/ping -c 1 8.8.8.8; exit 1")' --no-allow-new-privileges
  builder for ‘/nix/store/j0nd8kv85hd6r4kxgnwzvr0k65ykf6fv-foo.drv’ failed with exit code 1; last 2 log lines:
    cannot raise the capability into the Ambient set
    : Operation not permitted

  $ nix build '(with import <nixpkgs> {}; runCommand "foo" {} "/run/wrappers/bin/ping -c 1 8.8.8.8; exit 1")' --allow-new-privileges
  builder for ‘/nix/store/j0nd8kv85hd6r4kxgnwzvr0k65ykf6fv-foo.drv’ failed with exit code 1; last 6 log lines:
    PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
    64 bytes from 8.8.8.8: icmp_seq=1 ttl=46 time=15.2 ms

Fixes #1429.
2017-07-04 15:48:25 +02:00
Eelco Dolstra ad8b96f1f2 Fix handling of expression installables with a / in them 2017-07-04 15:38:23 +02:00
Eelco Dolstra c0015e87af Support base-64 hashes
Also simplify the Hash API.

Fixes #1437.
2017-07-04 15:07:41 +02:00
Eelco Dolstra fe97c69898 <nix/fetchurl.nix>: Support sha512 argument 2017-07-04 14:45:50 +02:00
Eelco Dolstra 0a5a867758 nix-shell: Respect --dry-run
Fixes #824.
2017-07-03 11:54:30 +02:00
Eelco Dolstra fcca702a96 Replace a few bool flags with enums
Functions like copyClosure() had 3 bool arguments, which creates a
severe risk of mixing up arguments.

Also, implement copyClosure() using copyPaths().
2017-07-03 11:38:08 +02:00
Eelco Dolstra 90da34e421 processGraph(): Call getEdges in parallel 2017-07-03 11:38:08 +02:00
Eelco Dolstra 63d6e0ad3f Merge pull request #1417 from corngood/cygwin-fix
Call SetDllDirectory("") after sqlite3 init on cygwin
2017-06-30 19:50:00 +02:00
Niklas Hambüchen b591536e93 Fix potential crash/wrong result two hashes of unequal length are compared 2017-06-24 02:17:45 +02:00
David McFarland 596b0e0a04 Call SetDllDirectory("") after sqlite3 init on cygwin
Cygwin sqlite3 is patched to call SetDllDirectory("/usr/bin") on init, which
affects the current process and is inherited by child processes.  It causes
DLLs to be loaded from /usr/bin/ before $PATH, which breaks all sorts of
things.  A typical failures would be header/lib version mismatches (e.g.
openssl when running checkPhase on openssh).  We'll just set it back to the
default value.

Note that this is a problem with the cygwin version of sqlite3 (currently
3.18.0).  nixpkgs doesn't have the problematic patch.
2017-06-20 10:59:13 -03:00
Eelco Dolstra c7346a275c Restore thunks on any exception
There's no reason to restrict this to Error exceptions. This shouldn't
matter to #1407 since the repl doesn't catch non-Error exceptions
anyway, but you never know...
2017-06-20 12:13:17 +02:00
Shea Levy 04ed11a978 Let hydra choose an alternate list of systems 2017-06-19 14:21:06 -04:00
Eelco Dolstra a1355917ec Disable use of virtual hosting in aws-sdk-cpp
Recently aws-sdk-cpp quietly switched to using S3 virtual host URIs
(https://github.com/aws/aws-sdk-cpp/commit/69d9c53882), i.e. it sends
requests to http://<bucket>.<region>.s3.amazonaws.com rather than
http://<region>.s3.amazonaws.com/<bucket>. However this interacts
badly with curl connection reuse. For example, if we do the following:

1) Check whether a bucket exists using GetBucketLocation.
2) If it doesn't, create it using CreateBucket.
3) Do operations on the bucket.

then 3) will fail for a minute or so with a NoSuchBucket exception,
presumably because the server being hit is a fallback for cases when
buckets don't exist.

Disabling the use of virtual hosts ensures that 3) succeeds
immediately. (I don't know what S3's consistency guarantees are for
bucket creation, but in practice buckets appear to be available
immediately.)
2017-06-19 18:51:33 +02:00
Eelco Dolstra 82a0d614cf Support creating S3 caches in other regions than us-east-1 2017-06-19 18:51:33 +02:00
Eelco Dolstra b33621d425 Handle S3Errors::RESOURCE_NOT_FOUND from aws-sdk-cpp
This is returned by recent versions. Also handle NO_SUCH_KEY even
though the library doesn't actually return that at the moment.
2017-06-19 18:51:32 +02:00
Eelco Dolstra 1c969611ba Suppress "will retry in N ms" for non-retriable errors
Newer versions of aws-sdk-cpp call CalculateDelayBeforeNextRetry()
even for non-retriable errors (like NoSuchKey) whih causes log spam in
hydra-queue-runner.
2017-06-19 18:51:32 +02:00
Eelco Dolstra 00aa7c6705 Show aws-sdk-cpp log messages 2017-06-19 18:51:32 +02:00
Eelco Dolstra 1888f7889b macOS: Ugly hack to make the tests succeed
Sandboxes cannot be nested, so if Nix's build runs inside a sandbox,
it cannot use a sandbox itself. I don't see a clean way to detect
whether we're in a sandbox, so use a test-specific hack.

https://github.com/NixOS/nix/issues/1413
2017-06-19 14:26:05 +02:00
Eelco Dolstra b5bdfdef73 macOS: Remove flags
In particular, UF_IMMUTABLE (uchg) needs to be cleared to allow the
path to be garbage-collected or optimised.

See https://github.com/NixOS/nixpkgs/issues/25819.
+       the file from being garbage-collected.
2017-06-19 14:19:21 +02:00
Eelco Dolstra a10951de08 OS X -> macOS
(cherry picked from commit c20641ce56)
2017-06-14 23:28:06 +02:00
Eelco Dolstra 1dcadadf74 Add 1.11.10 release notes
(cherry picked from commit 0fb60e4e0f)
2017-06-14 23:27:52 +02:00
Eelco Dolstra 38b7d55af1 Remove redundant debug line 2017-06-14 13:45:38 +02:00
Eelco Dolstra 88b291ffc4 canonicalisePathMetaData(): Ignore security.selinux attribute
Untested, hopefully fixes #1406.
2017-06-14 11:41:03 +02:00
Eelco Dolstra 177f3996e2 Suppress spurious "killing process N: Operation not permitted" on macOS 2017-06-12 18:34:48 +02:00
Eelco Dolstra 25230a17a9 On macOS, don't use /var/folders for TMPDIR
This broke "nix-store --serve".
2017-06-12 17:43:19 +02:00
Eelco Dolstra 847f19a5f7 Provide a builtin default for $NIX_SSL_CERT_FILE
This is mostly to ensure that when Nix is started on macOS via a
launchd service or sshd (for a remote build), it gets a certificate
bundle.
2017-06-12 16:44:43 +02:00
Eelco Dolstra 7f5b750b40 Don't run pre-build-hook if we don't have a derivation
This fixes a build failure on OS X when using Hydra or Nix 1.12's
build-remote (since they don't copy the derivation to the build
machine).
2017-06-12 16:07:34 +02:00
Eelco Dolstra 186571965d Don't show flags from config settings in "nix --help" 2017-06-07 18:41:20 +02:00
Eelco Dolstra aa952d5f0b nix: Add --help-config flag 2017-06-07 16:49:54 +02:00
Eelco Dolstra b8283773bd nix: Make all options available as flags
Thus, instead of ‘--option <name> <value>’, you can write ‘--<name>
<value>’. So

  --option http-connections 100

becomes

  --http-connections 100

Apart from brevity, the difference is that it's not an error to set a
non-existent option via --option, but unrecognized arguments are
fatal.

Boolean options have special treatment: they're mapped to the
argument-less flags ‘--<name>’ and ‘--no-<name>’. E.g.

  --option auto-optimise-store false

becomes

  --no-auto-optimise-store
2017-06-07 16:17:17 +02:00
Eelco Dolstra c8cc50d46e Disable the build user mechanism on all platforms except Linux and OS X 2017-06-06 18:52:15 +02:00
Eelco Dolstra 85e93d7b87 Always use the Darwin sandbox
Even with "build-use-sandbox = false", we now use sandboxing with a
permissive profile that allows everything except the creation of
setuid/setgid binaries.
2017-06-06 18:44:49 +02:00
Eelco Dolstra d3f780996c Merge pull request #1399 from Mic92/master
Add .editorconfig
2017-06-06 14:13:36 +02:00
Jörg Thalheim 88acb64610 Add .editorconfig
- Automatically adjust editor to nix coding style
-> less nitpiks/styling issues in pull requests
-> profit(!)

see also nixpkgs' editorconfig: https://github.com/NixOS/nixpkgs/blob/master/.editorconfig
2017-06-05 22:57:28 +01:00
Eelco Dolstra b4b1f4525f Fix coverage job 2017-06-01 14:43:15 +02:00
Eelco Dolstra ab5834f7a1 RPM, Deb: Add dependency on libseccomp 2017-06-01 14:28:21 +02:00
Eelco Dolstra 52fec8dde8 Remove listxattr assertion
It appears that sometimes, listxattr() returns a different value for
the query case (i.e. when the buffer size is 0).
2017-05-31 20:43:47 +02:00
Eelco Dolstra c96e8cd097 OS X sandbox: Improve builtin sandbox profile
Also, add rules to allow fixed-output derivations to access the
network.

These rules are sufficient to build stdenvDarwin without any
__sandboxProfile magic.
2017-05-31 17:25:51 +02:00
Eelco Dolstra 5ea8161b55 resolve-system-dependencies: Misc fixes
This fixes

  Could not find any mach64 blobs in file ‘/usr/lib/libSystem.B.dylib’, continuing...
2017-05-31 16:10:10 +02:00
Eelco Dolstra c368e079ca resolve-system-dependencies: Simplify 2017-05-31 15:34:03 +02:00
Eelco Dolstra 44f3f8048f OS X sandbox: Don't use a deterministic $TMPDIR
This doesn't work because the OS X sandbox cannot bind-mount
path to a different location.
2017-05-31 14:09:00 +02:00
Eelco Dolstra c740c3ce50 OS X sandbox: Store .sb file in $TMPDIR rather than the Nix store
The filename used was not unique and owned by the build user, so
builds could fail with

error: while setting up the build environment: cannot unlink ‘/nix/store/99i210ihnsjacajaw8r33fmgjvzpg6nr-bison-3.0.4.drv.sb’: Permission denied
2017-05-31 14:09:00 +02:00
Eelco Dolstra 683a499ebb resolve-system-dependencies: Fix another segfault
runResolver() was barfing on directories like
/System/Library/Frameworks/Security.framework/Versions/Current/PlugIns. It
should probably do something sophisticated for frameworks, but let's
ignore them for now.
2017-05-30 20:39:40 +02:00
Eelco Dolstra acc889c821 Darwin sandbox: Use sandbox-defaults.sb
Issue #759.

Also, remove nix.conf from the sandbox since I don't really see a
legitimate reason for builders to access the Nix configuration.
2017-05-30 17:40:12 +02:00
Eelco Dolstra 53a1644187 Darwin sandbox: Disallow creating setuid/setgid binaries
Suggested by Daiderd Jordan.
2017-05-30 17:17:17 +02:00
Eelco Dolstra 83eec5a997 resolve-system-dependencies: Several fixes
This fixes

  error: getting attributes of path ‘Versions/Current/CoreFoundation’: No such file or directory

when /System/Library/Frameworks/CoreFoundation.framework/CoreFoundation is a symlink.

Also fixes a segfault when encounting a file that is not a MACH binary (such
as /dev/null, which is included in __impureHostDeps in Nixpkgs).

Possibly fixes #786.
2017-05-30 16:03:37 +02:00
Eelco Dolstra 9bdb88ea6e Only pass --with-sandbox-shell on Linux 2017-05-30 15:56:15 +02:00
Eelco Dolstra fe08d17934 Fix seccomp build failure on clang
Fixes

  src/libstore/build.cc:2321:45: error: non-constant-expression cannot be narrowed from type 'int' to 'scmp_datum_t' (aka 'unsigned long') in initializer list [-Wc++11-narrowing]
2017-05-30 14:37:24 +02:00
Eelco Dolstra d552d38758 Shut up some clang warnings 2017-05-30 14:35:50 +02:00
Eelco Dolstra 2ac99a32da Add a seccomp rule to disallow setxattr() 2017-05-30 13:59:24 +02:00
Eelco Dolstra d798349ede canonicalisePathMetaData(): Remove extended attributes / ACLs
EAs/ACLs are not part of the NAR canonicalisation. Worse, setting an
ACL allows a builder to create writable files in the Nix store. So get
rid of them.

Closes #185.
2017-05-30 13:47:41 +02:00
Eelco Dolstra ff6becafa8 Require seccomp only in multi-user setups 2017-05-30 12:37:04 +02:00
Eelco Dolstra 1d9ab273ba Add test for setuid seccomp filter 2017-05-29 16:14:10 +02:00
Eelco Dolstra cf93397d3f Fix seccomp initialisation on i686-linux 2017-05-29 16:14:10 +02:00
Eelco Dolstra 6cc6c15a2d Add a seccomp filter to prevent creating setuid/setgid binaries
This prevents builders from setting the S_ISUID or S_ISGID bits,
preventing users from using a nixbld* user to create a setuid/setgid
binary to interfere with subsequent builds under the same nixbld* uid.

This is based on aszlig's seccomp code
(47f587700d).

Reported by Linus Heckemann.
2017-05-29 16:14:10 +02:00
Eelco Dolstra 6e01ecd112 Fix nix-copy-closure test
Fixes

  client# error: size mismatch importing path ‘/nix/store/ywf5fihjlxwijm6ygh6s0a353b5yvq4d-libidn2-0.16’; expected 0, got 120264

This is mostly an artifact of the NixOS VM test environment, where the
Nix database doesn't contain hashes/sizes.

http://hydra.nixos.org/build/53537471
2017-05-29 16:08:56 +02:00
Eelco Dolstra 588dad4084 Fix build failure on Debian/Ubuntu
http://hydra.nixos.org/build/53537463
2017-05-29 15:59:18 +02:00
Eelco Dolstra 63145be2a5 Fix typo 2017-05-29 15:52:36 +02:00
Eelco Dolstra d6064dd19b Merge pull request #1393 from pyrtsa/patch-1
Fix variable name typo in derivations doc
2017-05-29 11:32:06 +02:00
Eelco Dolstra 2740a22d2c Merge pull request #1394 from pyrtsa/patch-2
Remove stray `>` in builtins doc
2017-05-29 11:31:40 +02:00
Pyry Jahkola 370428f86d Remove stray > in builtins doc 2017-05-28 20:47:35 +03:00
Pyry Jahkola 86ea7d1566 Fix variable name typo in derivations doc 2017-05-28 15:48:57 +03:00
Eelco Dolstra a7e55151a8 Fix #1314
Also, make nix-shell respect --option. (Previously it only passed it
along to nix-instantiate and nix-build.)
2017-05-24 11:33:42 +02:00
Eelco Dolstra fbe9fe0e75 Merge branch 'topic/cores-master' of https://github.com/neilmayhew/nix 2017-05-24 11:28:36 +02:00
Eelco Dolstra 9e43a4a041 Merge pull request #1376 from Mic92/patch-1
nix-profile.sh: remove sbin from PATH
2017-05-24 11:25:32 +02:00
Eelco Dolstra 9711524188 Fix #1380
It lacked a backslash. Use a raw string and single quotes around PS1
to simplify this.
2017-05-24 11:23:28 +02:00
Eelco Dolstra 01200d07d2 Merge branch 'prompt-terminator' of https://github.com/lheckemann/nix 2017-05-24 11:22:48 +02:00
Eelco Dolstra b303ad012c Merge pull request #1382 from FRidh/patch-1
Document fetchTarball can take a sha256
2017-05-24 11:07:36 +02:00
Eelco Dolstra edbb105e98 Merge branch 'nar-accessor-tree' of https://github.com/bennofs/nix 2017-05-24 11:04:43 +02:00
Eelco Dolstra f134fc4cbe Document that builtins.match takes a POSIX extended RE 2017-05-17 12:18:18 +02:00
Eelco Dolstra e46090edb1 builtins.match: Improve error message for bad regular expression
Issue #1331.
2017-05-17 11:58:01 +02:00
Eelco Dolstra b01d62285c Improve progress indicator 2017-05-16 16:09:57 +02:00
Benno Fünfstück a1f428b13b nar-accessor.cc: remove unused member NarIndexer::currentName 2017-05-15 19:41:59 +02:00
Benno Fünfstück 5ee06e612a nar-accessor: non-recursive NarMember::find
This avoids a possible stack overflow if directories are very deeply nested.
2017-05-15 19:34:18 +02:00
Eelco Dolstra e80257f122 Simplify fixed-output check 2017-05-15 18:50:54 +02:00
Eelco Dolstra c05d9ae7a5 Disallow outputHash being null or an empty string
Fixes #1384.
2017-05-15 18:47:13 +02:00
Eelco Dolstra a2d92bb20e Add --with-sandbox-shell configure flag
And add a 116 KiB ash shell from busybox to the release build. This
helps to make sandbox builds work out of the box on non-NixOS systems
and with diverted stores.
2017-05-15 17:36:32 +02:00
Eelco Dolstra b30f5784d0 Linux sandbox: Don't barf on invalid paths
This is useful when we're using a diverted store (e.g. "--store
local?root=/tmp/nix") in conjunction with a statically-linked sh from
the host store (e.g. "sandbox-paths =/bin/sh=/nix/store/.../bin/busybox").
2017-05-15 17:36:32 +02:00
Eelco Dolstra 2b761d5f50 Make fmt() non-recursive 2017-05-15 17:36:32 +02:00
Benno Fünfstück 4412f7c083 nar-archive.cc: add tests for the nar index 2017-05-15 12:23:21 +02:00
Eelco Dolstra 0124d118ef Merge pull request #1387 from bennofs/nix-ls-slash
nix ls: support '/' for the root directory
2017-05-15 11:02:14 +02:00
Benno Fünfstück 06880d7ed8 nix ls: support '/' for the root directory 2017-05-15 10:25:55 +02:00
Benno Fünfstück 75a1d9849d nar-accessor: use tree, fixes readDirectory missing children
Previously, if a directory `foo` existed and a file `foo-` (where `-` is any character that is sorted before `/`), then  `readDirectory` would return an empty list.

To fix this, we now use a tree where we can just access the children of the node, and do not need to rely on sorting behavior to list the contents of a directory.
2017-05-15 10:23:16 +02:00
Eelco Dolstra 510bc1735b Add an option for extending the user agent header
This is useful e.g. for distinguishing traffic to a binary cache
(e.g. certain machines can use a different tag in the user agent).
2017-05-11 15:10:22 +02:00
Eelco Dolstra 62d476c7ee Fix typo 2017-05-11 14:02:03 +02:00
Eelco Dolstra ea65ae0f9c Tweak error message 2017-05-11 13:59:47 +02:00
Eelco Dolstra 1a8e15053a Don't allow untrusted users to set info.ultimate
Note that a trusted signature was still required in this case so it
was not a huge deal.
2017-05-11 13:58:09 +02:00
Frederik Rietdijk 2b2de5ef6a Document fetchTarball can take a sha256
Note that I refer to `nix-prefetch-url`.
2017-05-11 13:38:13 +02:00
Eelco Dolstra 6f245bf24a Change the meaning of info.ultimate
It now means "paths that were built locally". It no longer includes
paths that were added locally. For those we don't need info.ultimate,
since we have the content-addressability assertion (info.ca).
2017-05-11 13:31:23 +02:00
Eelco Dolstra 45d7b1a9e9 LocalStore::addToStore(): Check info.narSize
It allowed the client to specify bogus narSize values. In particular,
Downloader::downloadCached wasn't setting narSize at all.
2017-05-11 13:26:03 +02:00
Eelco Dolstra 1fd59447d5 Typo 2017-05-10 18:38:17 +02:00
Eelco Dolstra c5f23f10a8 Replace readline by linenoise
Using linenoise avoids a license compatibility issue (#1356), is a lot
smaller and doesn't pull in ncurses.
2017-05-10 18:37:42 +02:00
Linus Heckemann d48edcc3a5 nix-shell: use appropriate prompt terminator
If running nix-shell as root, the terminator should be # and not $.
2017-05-10 12:03:45 +01:00
Domen Kožar 82a9c93c7f doc: builtins.attrNames returns alphabetically sorted list 2017-05-10 11:23:12 +02:00
Eelco Dolstra 03ae5e6459 Add "nix edit" command
This is a little convenience command that opens the Nix expression of
the specified package. For example,

  nix edit nixpkgs.perlPackages.Moose

opens <nixpkgs/pkgs/top-level/perl-packages.nix> in $EDITOR (at the
right line number for some editors).

This requires the package to have a meta.position attribute.
2017-05-08 18:42:30 +02:00
Eelco Dolstra 7689181e4f Minor cleanup 2017-05-08 15:56:52 +02:00
Eelco Dolstra 00b286275c Linux sandbox: Fix compatibility with older kernels 2017-05-08 15:42:59 +02:00
Eelco Dolstra ebfceeb333 build-remote: Check remote build status 2017-05-08 14:27:12 +02:00
Eelco Dolstra 0a97eb6bd7 Remove superfluous #ifdef 2017-05-08 11:27:20 +02:00
Jörg Thalheim 542fe0d8f3 nix-profile.sh: remove sbin from PATH
sbin is a symlink to bin. 
profiles only contains packages, which have this symlink. 
It is a subset of bin.

related to https://github.com/NixOS/nixpkgs/pull/25550
2017-05-07 07:41:19 +01:00
Eelco Dolstra bb50c89319 Make the location of the build directory in the sandbox configurable
This is mostly for use in the sandbox tests, since if the Nix store is
under /build, then we can't use /build as the build directory.
2017-05-05 17:45:22 +02:00
Eelco Dolstra 465cb68244 Figure out the user's home directory if $HOME is not set 2017-05-05 17:08:23 +02:00
Eelco Dolstra eba840c8a1 Linux sandbox: Use /build instead of /tmp as $TMPDIR
There is a security issue when a build accidentally stores its $TMPDIR
in some critical place, such as an RPATH. If
TMPDIR=/tmp/nix-build-..., then any user on the system can recreate
that directory and inject libraries into the RPATH of programs
executed by other users. Since /build probably doesn't exist (or isn't
world-writable), this mitigates the issue.
2017-05-04 16:57:03 +02:00
Eelco Dolstra 2da6a42448 nix dump-path: Add
This is primarily useful for extracting NARs from other stores (like
binary caches), which "nix-store --dump" cannot do.
2017-05-04 14:21:22 +02:00
Eelco Dolstra 44309c5067 Fix Ubuntu 16.10 build
http://hydra.nixos.org/build/52420073
2017-05-03 18:30:47 +02:00
Eelco Dolstra 493d4bd949 perl-bindings: Remove unused --with-store-dir flag 2017-05-03 17:43:52 +02:00
Eelco Dolstra 72fb2a7edc Fix build on gcc 4.9
http://hydra.nixos.org/build/52408843
2017-05-03 16:08:48 +02:00
Eelco Dolstra 08355643ab nix-shell: Implement passAsFile 2017-05-03 15:01:15 +02:00
Eelco Dolstra 782c0bff45 nix eval: Add a --raw flag
Similar to "jq -r", this prints the evaluation result (which must be a
string value) unquoted.
2017-05-03 14:08:18 +02:00
Eelco Dolstra d3dcdfa006 Fix perlBindings.x86_64-darwin
http://hydra.nixos.org/build/52401151
2017-05-03 11:30:22 +02:00
Eelco Dolstra d3449b286b Merge pull request #1371 from regnat/doc_--xml_fix
fix the description of --xml and --json
2017-05-03 11:06:12 +02:00
regnat a786d26dc2 doc: fix the description of --xml and --json
Those options seem to only apply with --eval and not with --parse.
2017-05-03 10:49:34 +02:00
Eelco Dolstra 16535552ad build-remote: Add a basic test
This only runs on Linux because it requires a diverted store (which
uses mount/user namespaces).
2017-05-02 15:46:10 +02:00
Eelco Dolstra cef8c169b1 Fix "nix ... --all"
When "--all" is used, we should not fill in a default installable.
2017-05-02 15:46:10 +02:00
Eelco Dolstra 7dedd3fa24 Add a test for diverted stores 2017-05-02 15:46:09 +02:00
Eelco Dolstra c5bea16611 LocalStoreAccessor: Fix handling of diverted stores 2017-05-02 15:46:09 +02:00
Eelco Dolstra 7f6837a0f6 Replace $NIX_REMOTE_SYSTEMS with an option "builder-files"
Also, to unify with hydra-queue-runner, allow it to be a list of
files.
2017-05-02 15:46:09 +02:00
Eelco Dolstra cd4d2705ec build-remote: Fix fallback to other machines when connecting fails
Opening an SSHStore or LegacySSHStore does not actually establish a
connection, so the try/catch block here did nothing. Added a
Store::connect() method to test whether a connection can be
established.
2017-05-02 15:46:09 +02:00
Eelco Dolstra 1a68710d4d Add an option for specifying remote builders
This is useful for one-off situations where you want to specify a
builder on the command line instead of having to mess with
nix.machines. E.g.

  $ nix-build -A hello --argstr system x86_64-darwin \
    --option builders 'root@macstadium1 x86_64-darwin'

will perform the specified build on "macstadium1".

It also removes the need for a separate nix.machines file since you
can specify builders in nix.conf directly. (In fact nix.machines is
yet another hack that predates the general nix.conf configuration
file, IIRC.)

Note: this option is supported by the daemon for trusted users. The
fact that this allows trusted users to specify paths to SSH keys to
which they don't normally have access is maybe a bit too much trust...
2017-05-02 15:42:43 +02:00
Eelco Dolstra ebc9f36a81 Factor out machines.conf parsing
This allows hydra-queue-runner to use it.
2017-05-02 13:17:37 +02:00
Eelco Dolstra 174b68a2a2 build-hook: If there are no machines defined, quit permanently 2017-05-02 12:16:29 +02:00
Eelco Dolstra 70581b6363 Fix build hook test 2017-05-02 12:07:56 +02:00
Eelco Dolstra feefcb3a98 build-remote: Ugly hackery to get build logs to work
The build hook mechanism expects build log output to go to file
descriptor 4, so do that.
2017-05-02 12:02:23 +02:00
Eelco Dolstra 3a5f04f48c build-remote: Don't require signatures
This restores the old behaviour.
2017-05-01 20:03:25 +02:00
Eelco Dolstra 031d70e500 Support arbitrary store URIs in nix.machines
For backwards compatibility, if the URI is just a hostname, ssh://
(i.e. LegacySSHStore) is prepended automatically.

Also, all fields except the URI are now optional. For example, this is
a valid nix.machines file:

  local?root=/tmp/nix

This is useful for testing the remote build machinery since you don't
have to mess around with ssh.
2017-05-01 17:35:30 +02:00
Eelco Dolstra 3e4bdfedee Minor cleanup 2017-05-01 17:30:17 +02:00
Eelco Dolstra deac171925 Implement LegacySSHStore::buildDerivation()
This makes LegacySSHStore usable by build-remote and
hydra-queue-runner.
2017-05-01 17:30:16 +02:00
Eelco Dolstra 3f5b98e65a Chomp log output from the build hook 2017-05-01 17:30:16 +02:00
Eelco Dolstra d7653dfc6d Remove $NIX_BUILD_HOOK and $NIX_CURRENT_LOAD
This is to simplify remote build configuration. These environment
variables predate nix.conf.

The build hook now has a sensible default (namely build-remote).

The current load is kept in the Nix state directory now.
2017-05-01 17:30:16 +02:00
Eelco Dolstra ca9f589a93 build-remote: Don't copy the .drv closure
Since build-remote uses buildDerivation() now, we don't need to copy
the .drv file anymore. This greatly reduces the set of input paths
copied to the remote side (e.g. from 392 to 51 store paths for GNU
hello on x86_64-darwin).
2017-05-01 17:30:16 +02:00
Eelco Dolstra b986c7f8b1 Pass verbosity level to build hook 2017-05-01 14:43:14 +02:00
Eelco Dolstra 227a48f86f Reduce severity of EMLINK warnings
Fixes #1357.
2017-05-01 14:36:56 +02:00
Eelco Dolstra 0dddcf867a Add a dummy Store::buildPaths() method
This default implementation of buildPaths() does nothing if all
requested paths are already valid, and throws an "unsupported
operation" error otherwise. This fixes a regression introduced by
c30330df6f in binary cache and legacy
SSH stores.
2017-05-01 13:43:34 +02:00
Eelco Dolstra b0b81b7500 Merge pull request #1366 from Mic92/fix-nix-daemon-service
nix-daemon.service: fix startup
2017-05-01 11:16:56 +02:00
Eelco Dolstra 8b039ba74f Merge branch 'remove-catchall' of https://github.com/layus/nix 2017-05-01 11:16:04 +02:00
Guillaume Maudoux a143014d73 lexer: remove catch-all rules hiding real errors
With catch-all rules, we hide potential errors.
It turns out that a4744254 made one cath-all useless. Flex detected that
is was impossible to reach.
The other is more subtle, as it can only trigger on unfinished escapes
in unfinished strings, which only occurs at EOF.
2017-05-01 01:18:06 +02:00
Guillaume Maudoux a474425425 Fix lexer to support $' in multiline strings. 2017-05-01 01:15:40 +02:00
Jörg Thalheim b3f55fdf62 nix-daemon.service: set XDG_CONFIG_HOME
Otherwise starting nix-daemon fails

● nix-daemon.service - Nix Daemon
   Loaded: loaded
(/nix/store/mnf00a6gc55xl47smk0b32gmi7xpvlfp-nix-1.12pre5308_2f21d522/lib/systemd/system/nix-daemon.service;
enabled; vendor preset: enabled)
  Drop-In:
/nix/store/m2rgjp71n4kyp8j5fxgbrlv13scd5vvv-system-units/nix-daemon.service.d
           └─overrides.conf
   Active: failed (Result: exit-code) since Sat 2017-04-29 11:29:21
CEST; 9s ago
  Process: 7299 ExecStart=nix-daemon --daemon (code=exited, status=1/FAILURE)
 Main PID: 7299 (code=exited, status=1/FAILURE)
      CPU: 19ms

... systemd[1]: Started Nix Daemon.
... nix-daemon[7299]: error: $XDG_CONFIG_HOME and $HOME are not set
... systemd[1]: nix-daemon.service: Main process exited, code=exited, status=1/FAILURE
... systemd[1]: nix-daemon.service: Unit entered failed state.
... systemd[1]: nix-daemon.service: Failed with result 'exit-code'.
... systemd[1]: nix-daemon.service: Start request repeated too quickly.
... systemd[1]: Failed to start Nix Daemon.
... systemd[1]: nix-daemon.service: Failed with result 'exit-code'.
2017-04-29 11:33:55 +02:00
Eelco Dolstra 2f21d522c2 Hopefully fix the Darwin build
http://hydra.nixos.org/build/52080911
2017-04-28 17:13:55 +02:00
Eelco Dolstra 895f00c372 Suppress warning about ssh-auth-sock 2017-04-28 16:55:52 +02:00
Eelco Dolstra 73bba12d8b Check for libreadline 2017-04-28 16:53:56 +02:00
Eelco Dolstra a1a5e63e14 Fix brainfart 2017-04-28 16:21:54 +02:00
Eelco Dolstra 41c4558afe Fix hash computation when importing NARs greater than 4 GiB
This caused "nix-store --import" to compute an incorrect hash on NARs
that don't fit in an unsigned int. The import would succeed, but
"nix-store --verify-path" or subsequent exports would detect an
incorrect hash.

A deeper issue is that the export/import format does not contain a
hash, so we can't detect such issues early.

Also, I learned that -Wall does not warn about this.
2017-04-28 15:24:05 +02:00
Eelco Dolstra 39b08f4c0c Merge pull request #1358 from shlevy/store-nesting
Add Store nesting to fix import-from-derivation within filterSource
2017-04-26 20:28:49 +02:00
Shea Levy 4bc00760f9 Add Store nesting to fix import-from-derivation within filterSource 2017-04-26 14:15:47 -04:00
Eelco Dolstra 45ce2c7413 Doh 2017-04-26 17:58:09 +02:00
Eelco Dolstra 98a2adb135 Simplify building nix-perl in nix-shell 2017-04-26 17:04:45 +02:00
Eelco Dolstra 6734c18c99 nix repl: Fix Ctrl-C 2017-04-25 19:19:48 +02:00
Eelco Dolstra 23aa1619da Minor cleanup 2017-04-25 19:10:47 +02:00
Eelco Dolstra 4c95ef3768 Fix nix-shell test 2017-04-25 18:59:18 +02:00
Eelco Dolstra 536f061765 "using namespace std" considered harmful 2017-04-25 18:58:02 +02:00
Eelco Dolstra 5bd8795e1f nix repl: Use $XDG_DATA_HOME for the readline history 2017-04-25 18:56:29 +02:00
Eelco Dolstra 921a2aeb05 Make "nix repl" build 2017-04-25 18:48:40 +02:00
Eelco Dolstra c31000bc93 Merge nix-repl repository 2017-04-25 18:14:13 +02:00
Eelco Dolstra 40daf0d800 Cleanup in preparation of merging nix-repl repo into nix repo 2017-04-25 18:13:23 +02:00
Eelco Dolstra fee93541a4 Merge branch 'master' of https://github.com/olejorgenb/nix-repl 2017-04-25 16:57:07 +02:00
Eelco Dolstra 2dff9556a4 Fix build 2017-04-25 16:55:03 +02:00
Eelco Dolstra c30330df6f StorePathCommands: Build installables
So for instance "nix copy --to ... nixpkgs.hello" will build
nixpkgs.hello first. It's debatable whether this is a good idea. It
seems desirable for commands like "nix copy" but maybe not for
commands like "nix path-info".
2017-04-25 16:19:22 +02:00
Eelco Dolstra d48c973ece Set default installable
Thus

  $ nix build -f foo.nix

will build foo.nix.

And

  $ nix build

will build default.nix. However, this may not be a good idea because
it's kind of inconsistent, given that "nix build foo" will build the
"foo" attribute from the default installation source (i.e. the
synthesis of $NIX_PATH), rather than ./default.nix. So I may revert
this.
2017-04-25 15:18:05 +02:00
Eelco Dolstra 0b6220fbd6 Interpret any installable containing a slash as a path
So "nix path-info ./result" now works.
2017-04-25 14:09:01 +02:00
Eelco Dolstra 7ee81f3887 Make StorePathsCommand a subclass of InstallablesCommand
This allows commands like 'nix path-info', 'nix copy', 'nix verify'
etc. to work on arbitrary installables. E.g. to copy geeqie to a
binary cache:

  $ nix copy -r --to file:///tmp/binary-cache nixpkgs.geeqie

Or to get the closure size of thunderbird:

  $ nix path-info -S nixpkgs.thunderbird
2017-04-25 13:20:26 +02:00
Eelco Dolstra c769841bc4 Move code around 2017-04-25 12:07:31 +02:00
Eelco Dolstra 6267d74889 Add "nix eval" command
This replaces "nix-instantiate --eval". The result is evaluated
strictly since this seems more useful.
2017-04-25 11:23:47 +02:00
Eelco Dolstra bcecc99007 Restructure installables handling in the "nix" command 2017-04-25 11:20:37 +02:00
Eelco Dolstra 1bb87c0487 Remove debug statement 2017-04-24 15:01:28 +02:00
Eelco Dolstra 66577a1c64 Factor out --json 2017-04-24 14:21:36 +02:00
Eelco Dolstra 9b63bb88c8 nix-shell -p: Use runCommandCC
This restores pre-17.03 behaviour by making gcc available.
2017-04-24 12:04:01 +02:00
Eelco Dolstra 1196470e92 Merge pull request #1347 from kennyballou/sm-grammar-fix
Fix small grammar issue about page
2017-04-24 10:49:46 +02:00
Eelco Dolstra 240399e059 Merge pull request #1351 from earldouglas/ellipses
Drop misleading ellipses
2017-04-24 10:49:09 +02:00
Eelco Dolstra 8d6af08530 Merge pull request #1348 from armijnhemel/nix-env
better document --meta option for nix-env
2017-04-24 10:18:23 +02:00
Eelco Dolstra 169edf9407 Merge pull request #1352 from corngood/cygwin-fixes
Cygwin fixes
2017-04-21 17:39:33 +02:00
David McFarland 804ac52489 add helper function to set 'interruptThrown'
this fixes a linker failure on cygwin 64 due to some bad
interaction between tls and shared libraries.

see: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64697
2017-04-21 11:28:14 -03:00
David McFarland d35231ec60 set _GNU_SOURCE on cygwin
this is needed for pipe2()
2017-04-21 11:27:27 -03:00
James Earl Douglas 111d347237 Drop misleading ellipses
This portion of the quick start guide may lead to confusion for
newcomers to Nix.  This change clarifies the example to one that can be
copied in its entirety.
2017-04-20 18:07:23 -06:00
Eelco Dolstra 749696e71c Detect lsof
Also, don't use lsof on Linux since it's not needed.

Fixes #1328.
2017-04-20 19:11:45 +02:00
Eelco Dolstra efa4bdbfcd Improve nix show-config --json
In particular, show descriptions. This could be used for manpage
generation etc.
2017-04-20 17:34:47 +02:00
Eelco Dolstra 4410e9d995 Setting: Remove "Tag" template argument 2017-04-20 16:52:53 +02:00
Eelco Dolstra f05d5f89ff Read per-user settings from ~/.config/nix/nix.conf 2017-04-20 14:58:16 +02:00
Eelco Dolstra 562585e901 binary-caches-parallel-connections -> http-connections 2017-04-20 14:04:00 +02:00
Eelco Dolstra 4222402219 nix.conf man page: binary-caches -> substituters 2017-04-20 13:41:29 +02:00
Eelco Dolstra 76cb3c702c Reimplement trusted-substituters (aka trusted-binary-caches) 2017-04-20 13:41:29 +02:00
Armijn Hemel 1559c596f6 document option 2017-04-19 19:10:12 +02:00
kballou 00b6c6d0c3 Fix small grammar issue about page
Fix subject-verb agreement issue in introduction/about.
2017-04-19 09:27:28 -06:00
Eelco Dolstra 9cc8047f44 Reimplement connect-timeout
Fixes #1339.
2017-04-19 14:54:52 +02:00
Eelco Dolstra b0cb117226 getDerivations(): Filter out packages with bad derivation names
In particular, this disallows attribute names containing dots or
starting with dots. Hydra already disallowed these. This affects the
following packages in Nixpkgs master:

  2048-in-terminal
  2bwm
  389-ds-base
  90secondportraits
  lispPackages.3bmd
  lispPackages.hu.dwim.asdf
  lispPackages.hu.dwim.def

Closes #1342.
2017-04-19 14:18:26 +02:00
Eelco Dolstra 62a07992bd Merge pull request #1337 from lheckemann/doc-tryEval
Manual: document tryEval
2017-04-18 16:54:37 +02:00
Eelco Dolstra 67fe3e07b2 Merge pull request #1321 from shlevy/channel-direct-tarball-error
nix-channel: error out if direct tarball unpack fails.
2017-04-18 16:51:46 +02:00
Linus Heckemann 9622d00afa Manual: document tryEval 2017-04-17 10:33:53 +01:00
Eelco Dolstra f8a2e8a552 Shut up some warnings 2017-04-14 14:42:20 +02:00
Eelco Dolstra dd3714f6ef Doh 2017-04-14 14:42:08 +02:00
Eelco Dolstra da76c72bc9 Build on aarch64-linux 2017-04-14 14:02:43 +02:00
Eelco Dolstra 6520b757c5 Fix 32-bit build
http://hydra.nixos.org/build/51569816
2017-04-14 13:59:39 +02:00
Eelco Dolstra 3b36c64b15 Fix Perl bindings 2017-04-14 13:47:10 +02:00
Eelco Dolstra 3872371f25 Minor cleanup 2017-04-14 13:42:22 +02:00
Eelco Dolstra 01dcdfcf33 nix-daemon: Don't set untrusted-* settings
These are no longer used anywhere.
2017-04-14 11:59:51 +02:00
Eelco Dolstra 1673c373c9 nix-daemon: Don't die if the user sends an unknown setting 2017-04-14 11:57:02 +02:00
Eelco Dolstra 872ba75d8b Add "nix show-config" command
This dumps the entire Nix configuration, including all options that
have default values.
2017-04-13 20:59:38 +02:00
Eelco Dolstra ba9ad29fdb Convert Settings to the new config system
This makes all config options self-documenting.

Unknown or unparseable config settings and --option flags now cause a
warning.
2017-04-13 20:53:23 +02:00
Eelco Dolstra 6bd9576aeb Support arbitrary numeric types for settings 2017-04-13 17:54:05 +02:00
Eelco Dolstra 0bf34de43b Validate Boolean settings better 2017-04-13 16:31:28 +02:00
Eelco Dolstra 1860070548 Merge branch 'rework-options' of https://github.com/copumpkin/nix 2017-04-13 16:15:51 +02:00
Eelco Dolstra 2040240e23 Add a Config class to simplify adding configuration settings
The typical use is to inherit Config and add Setting<T> members:

  class MyClass : private Config
  {
    Setting<int> foo{this, 123, "foo", "the number of foos to use"};
    Setting<std::string> bar{this, "blabla", "bar", "the name of the bar"};

    MyClass() : Config(readConfigFile("/etc/my-app.conf"))
    {
      std::cout << foo << "\n"; // will print 123 unless overriden
    }
  };

Currently, this is used by Store and its subclasses for store
parameters. You now get a warning if you specify a non-existant store
parameter in a store URI.
2017-04-13 16:03:31 +02:00
Eelco Dolstra 568a099c88 canonPath(): Check against empty paths 2017-04-13 16:03:31 +02:00
Eelco Dolstra 6d97d81656 Add warn function 2017-04-13 16:03:31 +02:00
Eelco Dolstra 31cc9366fc Initialise logger 2017-04-13 16:03:31 +02:00
Eelco Dolstra 16c4856027 Merge pull request #1325 from benley/float-rl-1.12
Move note about float support out of the wrong release notes
2017-04-13 16:00:37 +02:00
Benjamin Staffin d267db0d75 Move note about float support out of the wrong release notes
Looks like this snuck into the 1.11 release notes post-release, but
float support isn't actually present until 1.12.
2017-04-12 18:01:43 -04:00
Eelco Dolstra 23304f527a Merge pull request #1302 from dtzWill/fix/nix-options
Process nix.conf options in "new" commands, add test
2017-04-12 11:03:19 +02:00
Eelco Dolstra b134c2d052 Drop WWW::Curl dependency
Somehow this came back after d1da6967b8.
2017-04-11 15:41:50 +02:00
Shea Levy 503cc4431b nix-channel: error out if direct tarball unpack fails.
It's very unlikely a path ending in .tar.gz is a directory

Fixes #1318
2017-04-10 18:16:46 -04:00
Eelco Dolstra 915f62fa19 shell.nix: Remove more dependencies
Thanks @copumpkin.
2017-04-10 17:23:15 +02:00
Eelco Dolstra 1fe1976e0d Merge pull request #1316 from copumpkin/nix-retries-default
Default to 5 download retries
2017-04-10 15:58:57 +02:00
Eelco Dolstra 8945a0ea2c Merge pull request #1317 from copumpkin/nix-retry-write-error
Add CURLE_WRITE_ERROR as a transient error condition
2017-04-10 15:58:17 +02:00
Dan Peebles d1fdade755 Add CURLE_WRITE_ERROR as a transient error condition
We've observed it failing downloads in the wild and retrying the same URL
a few moments later seemed to fix it.
2017-04-10 09:28:44 -04:00
Dan Peebles e43e8be8e7 Default to 5 download retries
This should help certain downloaders that don't request anything special
for the number of retries, like nix-channel.
2017-04-10 09:22:24 -04:00
Eelco Dolstra 53edb55588 shell.nix: Remove obsolete flags 2017-04-10 11:50:01 +02:00
Eelco Dolstra 105f8ffc98 Minor cleanup
Also, possible fix for #1310 on 32-bit systems.
2017-04-10 11:27:33 +02:00
Eelco Dolstra 95295482ea Allow "auto" as a store URI
Using the empty string is likely to be ambiguous in some contexts.
2017-04-10 11:27:29 +02:00
Neil Mayhew f12a048a05 Propagate NIX_BUILD_CORES to nix-shell environments 2017-04-09 08:21:52 -06:00
Will Dietz a0c56197fc tests/timeout: create output so tests don't trivially pass
Timeout tests rely on failed build to determine success,
so make sure these derivations (silent in particular)
don't fail regardless of timeout behavior.
2017-04-08 12:59:42 -05:00
Will Dietz 30f89e0d65 Process nix.conf options in "new" nix commands, add test.
Without this (minor) change, the options set using "--option"
or read from nix.conf were parsed but not used.
2017-04-08 12:59:42 -05:00
Eelco Dolstra d8ff3aaae7 Merge pull request #1306 from copumpkin/retry-on-ssl-error
Retry downloads on transient SSL errors too
2017-04-07 13:53:38 +02:00
Dan Peebles 98283915f5 Retry downloads on transient SSL errors too 2017-04-06 18:18:43 +00:00
Eelco Dolstra ba20730b3f Implement RemoteStore::queryMissing()
This provides a significant speedup, e.g. 64 s -> 12 s for

  nix-build --dry-run -I nixpkgs=channel:nixos-16.03 '<nixpkgs/nixos/tests/misc.nix>' -A test

on a cold local and CloudFront cache.

The alternative is to use lots of concurrent daemon connections but
that seems wasteful.
2017-04-06 18:40:19 +02:00
Eelco Dolstra 963f2bf12b Fix bogus "unexpected Nix daemon error: interrupted by the user" 2017-04-06 17:19:32 +02:00
Eelco Dolstra 6b5e271163 Add a method to allow hydra-queue-runner to flush the path info cache 2017-04-06 15:22:37 +02:00
Eelco Dolstra 256940fc48 nix-daemon: Disable path info cache
This is useless because the client also caches path info, and can
cause problems for long-running clients like hydra-queue-runner
(i.e. it may return cached info about paths that have been
garbage-collected).
2017-04-06 14:30:31 +02:00
Eelco Dolstra 8decb07c31 Allow default sandbox paths to be overriden
E.g. you can now redirect /etc/resolv.conf to a different file.
2017-04-04 17:54:16 +02:00
Eelco Dolstra 488792a87d Make /var/run/nscd/socket optional
Not every distribution uses nscd.
2017-04-04 17:40:50 +02:00
Eelco Dolstra dfcd78d851 Really fix the RPM build 2017-04-03 16:25:20 +02:00
Eelco Dolstra 3b8946e09a Maybe fix the RPM build
http://hydra.nixos.org/build/51095532
2017-03-31 18:30:50 +02:00
Eelco Dolstra 29d35805c6 Sandbox: Fix /dev/ptmx on recent kernels
This fixes "No such file or directory" when opening /dev/ptmx
(e.g. http://hydra.nixos.org/build/51094249).

The reason appears to be some changes to /dev/ptmx / /dev/pts handling
between Linux 4.4 and 4.9. See
https://patchwork.kernel.org/patch/7832531/.

The fix is to go back to mounting a proper /dev/pts instance inside
the sandbox. Happily, this now works inside user namespaces, even for
unprivileged users. So

  NIX_REMOTE=local?root=/tmp/nix nix-build \
    '<nixpkgs/nixos/tests/misc.nix>' -A test

works for non-root users.

The downside is that the fix breaks sandbox builds on older kernels
(probably pre-4.6), since mounting a devpts fails inside user
namespaces for some reason I've never been able to figure out. Builds
on those systems will fail with

  error: while setting up the build environment: mounting /dev/pts: Invalid argument

Ah well.
2017-03-31 18:20:19 +02:00
Shea Levy 3ecb09a40a builtins.exec: Make the argument just a list 2017-03-31 11:58:41 -04:00
Shea Levy d299bd710a Merge branch 'builtins.exec' 2017-03-31 11:22:39 -04:00
Eelco Dolstra b9b8b8a63b Fix evaluation error 2017-03-31 15:54:15 +02:00
Eelco Dolstra e1509adbbb Retry curl error 16 2017-03-31 15:50:11 +02:00
Eelco Dolstra d087700347 Fix perl build 2017-03-31 15:31:34 +02:00
Eelco Dolstra c0745a2531 Merge branch 'remove-perl' of https://github.com/shlevy/nix 2017-03-31 14:13:32 +02:00
Shea Levy a75475ca61 Remove tabs 2017-03-30 16:51:50 -04:00
Shea Levy 0bb8db257d Add exec primop behind allow-unsafe-native-code-during-evaluation.
Execute a given program with the (optional) given arguments as the
user running the evaluation, parsing stdout as an expression to be
evaluated.

There are many use cases for nix that would benefit from being able to
run arbitrary code during evaluation, including but not limited to:

* Automatic git fetching to get a sha256 from a git revision
* git rev-parse HEAD
* Automatic extraction of information from build specifications from
  other tools, particularly language-specific package managers like
  cabal or npm
* Secrets decryption (e.g. with nixops)
* Private repository fetching

Ideally, we would add this functionality in a more principled way to
nix, but in the mean time 'builtins.exec' can be used to get these
tasks done.

The primop is only available when the
'allow-unsafe-native-code-during-evaluation' nix option is true. That
flag also enables the 'importNative' primop, which is strictly more
powerful but less convenient (since it requires compiling a plugin
against the running version of nix).
2017-03-30 08:04:21 -04:00
Eelco Dolstra c60715e937 Ignore broken "Deriver: unknown-deriver" fields in .narinfo
These were generated by a legacy tool.
2017-03-28 13:08:13 +02:00
Daiderd Jordan 023217f07c use std::tuple for ValueMap allocator 2017-03-24 23:05:49 +01:00
Eelco Dolstra 165786dbc0 Merge pull request #1287 from mattaudesse/readme-grammar-its-typo
Fix minor grammatical nitpick ("it's" vs. "its") in `README.md`.
2017-03-22 15:41:46 +01:00
Matt Audesse 8edf107177 Fix minor grammatical nitpick ("it's" vs. "its") in README.md.
See: http://data.grammarbook.com/blog/pronouns/1-grammar-error/
2017-03-22 10:11:23 -04:00
Eelco Dolstra 0a7ca24c26 Fix xz decompression
Fixes #1285.
2017-03-22 11:53:33 +01:00
Eelco Dolstra 895a74a814 LocalFSStore::getBuildLog(): Handle corrupted logs 2017-03-21 19:23:35 +01:00
Eelco Dolstra ed5c0f69f2 Don't hang in decompression if bzip2 data ends prematurely 2017-03-21 19:23:35 +01:00
Eelco Dolstra aa23bba27f Fix tests to reflect the signed-binary-caches default change 2017-03-21 18:06:13 +01:00
Eelco Dolstra e1e49c58e1 Only use cache.nixos.org when the store is /nix/store
This is consistent with the behaviour of the old
download-from-binary-cache substituter.
2017-03-21 17:59:18 +01:00
Eelco Dolstra ecbc3fedd3 Require signatures by default
This corresponds to the NixOS default.
2017-03-21 15:10:48 +01:00
Eelco Dolstra 4bb38591e5 Restore cache.nixos.org as the default substituter
Fixes #1283.
2017-03-21 15:06:46 +01:00
Eelco Dolstra 7347daba8c Don't make brotli a hard dependency 2017-03-21 14:47:35 +01:00
Eelco Dolstra f8b84a3b8c Move istringstream_nocopy to a separate file 2017-03-21 14:43:03 +01:00
Eelco Dolstra 3229f85585 Honor $NIX_SSHOPTS again
NixOps needs this.
2017-03-21 14:35:50 +01:00
Eelco Dolstra 20df50d8e1 Merge branch 'darwin-s3-binary-cache-store' of https://github.com/shlevy/nix 2017-03-20 17:46:35 +01:00
Eelco Dolstra dc931fe1cd Merge pull request #1275 from steveeJ/patch-1
nix-shell/pure: keep environment variable SHLVL
2017-03-20 17:38:40 +01:00
Eelco Dolstra 558eda0115 nix copy: Make -r option use the "from" store
Previously, we tried to compute the closure in the local store, which
obviously doesn't work.
2017-03-16 14:25:54 +01:00
Eelco Dolstra 287084d688 ssh:// -> ssh-ng://, legacy-ssh:// -> ssh:// 2017-03-16 14:19:32 +01:00
Eelco Dolstra c5b83d8913 copyPaths(): Use queryValidPaths() to reduce SSH latency 2017-03-16 13:50:01 +01:00
Eelco Dolstra 91d67692cf copyPaths(): Don't query path info for a path the target already has
For example, this cuts "nix-copy-closure --from" on a NixOS system
closure from 15.9s to 0.5s.
2017-03-16 12:05:51 +01:00
Eelco Dolstra ea7fa88131 LegacySSHStore: Provide a faster implementation of computeFSClosure()
This avoids the latency of the standard implementation, which can make
a huge difference (e.g. 16.5s -> 0.5s on a NixOS system closure).
2017-03-16 11:44:52 +01:00
Eelco Dolstra 7a716ef2a5 Fix nix-copy-closure --to 2017-03-16 10:58:48 +01:00
Eelco Dolstra 0ec7f47b00 Remove "killing process <pid>" messages
They convey no useful information.
2017-03-16 10:52:28 +01:00
Eelco Dolstra 43f158bb08 nix-copy-closure: Fix assertion failure
$ ./inst/bin/nix-copy-closure --to bla $(type -p firefox)
  nix-copy-closure: src/libstore/store-api.cc:80: std::__cxx11::string nix::storePathToHash(const Path&): Assertion `base.size() >= storePathHashLen' failed.
2017-03-16 10:45:45 +01:00
Eelco Dolstra 3f35612c04 More precise compression settings 2017-03-15 17:20:19 +01:00
Eelco Dolstra 25dff2b7db runProgram(): Distinguish between empty input and no input
For example, if we call brotli with an empty input, it shouldn't read
from the caller's stdin.
2017-03-15 16:50:19 +01:00
Eelco Dolstra 042975ea8e Compress NAR listings using the "text-compression" method
So if "text-compression=br", the .ls file in S3 will get a
Content-Encoding of "br". Brotli appears to compress better than xz
for this kind of file and is natively supported by browsers.
2017-03-15 16:50:19 +01:00
Eelco Dolstra 45c70382ac S3BinaryCacheStore: Set Content-Type
This is necessary for serving log files to browsers.
2017-03-15 16:50:19 +01:00
Eelco Dolstra 8b1d65bebe S3BinaryCacheStore: Support compression of narinfo and log files
You can now set the store parameter "text-compression=br" to compress
textual files in the binary cache (i.e. narinfo and logs) using
Brotli. This sets the Content-Encoding header; the extension of
compressed files is unchanged.

You can separately specify the compression of log files using
"log-compression=br". This is useful when you don't want to compress
narinfo files for backward compatibility.
2017-03-15 16:49:28 +01:00
Eelco Dolstra 2691498b5c Fix assertion failure on SIGINT
nix: src/libutil/compression.cc:142: virtual nix::XzSink::~XzSink(): Assertion `finished' failed.
2017-03-15 16:49:06 +01:00
Eelco Dolstra fbbc4d8dda Fix deadlock in runProgram() when input is larger than the pipe buffer size 2017-03-15 16:49:06 +01:00
Eelco Dolstra e8186085e0 Add support for brotli compression
Build logs on cache.nixos.org are compressed using Brotli (since this
allows them to be decompressed automatically by Chrome and Firefox),
so it's handy if "nix log" can decompress them.
2017-03-15 16:49:06 +01:00
Eelco Dolstra 73d7a51ee6 Remove dependency on "curl" binary 2017-03-15 16:48:29 +01:00
Eelco Dolstra 532d73d5d8 BinaryCacheStore: Implement getBuildLog()
We assume that build logs are stored under log/<drv>, e.g.

  /nix/store/q7ab198v13p0f8x8wgnd75dva7d5mip6-friday-devil-0.1.1.1.drv

maps to

  https://cache.nixos.org/log/q7ab198v13p0f8x8wgnd75dva7d5mip6-friday-devil-0.1.1.1.drv
2017-03-15 16:48:29 +01:00
Eelco Dolstra 5b86451f02 Add a "nix log" command
This replaces "nix-store --read-log". It checks the local store and
any configured substituters for the requested logs.
2017-03-15 16:48:29 +01:00
Eelco Dolstra 0afeb7f51e Store: Add a method for getting build logs
This allows various Store implementations to provide different ways to
get build logs. For example, BinaryCacheStore can get the build logs
from the binary cache.

Also, remove the log-servers option since we can use substituters for
this.
2017-03-15 16:48:29 +01:00
Shea Levy 96443e94a1 Merge branch 'configurable-aws-region' 2017-03-12 10:01:11 -04:00
Stefan Junker f628ca2a1f nix-shell/pure: keep environment variable SHLVL 2017-03-12 01:04:21 +01:00
Domen Kožar ae568847f5 Dockerfile: 1.11.2 -> 1.11.7 2017-03-11 19:39:13 +01:00
Eelco Dolstra 19643a781e nix -> Nix 2017-03-09 13:36:56 +01:00
Eelco Dolstra 86227390c5 Merge branch 'allow-import-from-derivation' of https://github.com/shlevy/nix 2017-03-09 13:36:30 +01:00
Eelco Dolstra 6a308c7da4 Merge pull request #1267 from adevress/master
Add missing header <sstream>
2017-03-09 13:35:19 +01:00
Adrien Devresse d853877ce9 Add missing header <sstream> 2017-03-08 22:24:10 +01:00
Shea Levy 5f831c1057 Add docs for allow-import-from-derivation 2017-03-08 09:12:03 -05:00
Shea Levy 93f863be96 Add option to disable import-from-derivation completely, even if the drv is already realized 2017-03-08 08:46:12 -05:00
Eelco Dolstra 121a407eec Merge pull request #1264 from shlevy/s3-ca-file
Properly set the caFile for aws-sdk-cpp s3
2017-03-06 20:54:52 +01:00
Shea Levy 3cc18d3753 Properly set the caFile for aws-sdk-cpp s3 2017-03-06 14:30:35 -05:00
Shea Levy 4fc30922cf istringstream_nocopy: Implement in a standards-compliant way.
Fixes the problem mentioned in e6a61b8da7

See #1135
2017-03-06 13:03:02 -05:00
Eelco Dolstra 1cf4801108 Revert fa125b9b28
This causes quadratic performance.
2017-03-06 13:56:20 +01:00
Shea Levy 689b825627 nix-daemon.plist: Set XDG_CACHE_HOME 2017-03-05 18:15:44 -05:00
Shea Levy b667abc699 Add signing and s3 support on darwin 2017-03-05 07:39:10 -05:00
Shea Levy 5789eaa3f4 Add aws-region param to S3 store URLs 2017-03-03 16:12:17 -05:00
Eelco Dolstra d1158bb816 Cache connection failures 2017-03-03 19:36:01 +01:00
Eelco Dolstra 8490ee37a6 SSHMaster: Make thread-safe 2017-03-03 19:28:27 +01:00
Eelco Dolstra d3eb1cf3bb build-remote: Don't use a SSH master
This is unnecessary because we make only one connection.
2017-03-03 19:23:20 +01:00
Eelco Dolstra d140c75530 Pool: Don't hang if creating a connection fails 2017-03-03 19:21:43 +01:00
Eelco Dolstra 577ebeaefb Improve SSH handling
* Unify SSH code in SSHStore and LegacySSHStore.

* Fix a race starting the SSH master. We now wait synchronously for
  the SSH master to finish starting. This prevents the SSH clients
  from starting their own connections.

* Don't use a master if max-connections == 1.

* Add a "max-connections" store parameter.

* Add a "compress" store parameter.
2017-03-03 19:05:50 +01:00
Eelco Dolstra 7f62be1bcd build-remote: Fix passing SSH key 2017-03-03 16:33:18 +01:00
Eelco Dolstra 5a1fb03b8f build-remote: Misc cleanup 2017-03-03 16:18:49 +01:00
Eelco Dolstra 2f992692e2 Fix fatal "broken pipe" error when $NIX_BUILD_HOOK is missing 2017-03-03 15:40:06 +01:00
Eelco Dolstra ffcf9d24a6 builtins.fetchgit: Support paths 2017-03-02 11:46:28 +01:00
Eelco Dolstra fbf17f1ad7 builtins.fetchgit: Fix bad format string 2017-03-02 11:40:11 +01:00
Eelco Dolstra fa125b9b28 TeeSink: Pre-reserve string space
When receiving a very large file, this can prevent the string from
having tobe copied, which temporarily doubles memory consumption.
2017-03-01 16:16:04 +01:00
Eelco Dolstra f61f67ddee RemoteStore::addToStore(): Send NAR rather than string containing NAR
This allows the NAR to be streamed in the future (though we're not
doing that yet).
2017-03-01 16:07:15 +01:00
Eelco Dolstra 374908726b readString(): Read directly into std::string
When reading a huge string, this halves memory consumption.

(Strictly speaking, this appears only valid in C++17, but who cares...)
2017-03-01 14:54:11 +01:00
Eelco Dolstra 07a0b8ca67 Tweak message 2017-03-01 14:52:16 +01:00
Eelco Dolstra 56e19d970d nix-store --import: Fix importing unsigned paths 2017-03-01 14:47:52 +01:00
Eelco Dolstra e321551d54 Fix assertion failure in nix-store --export
Fixes #1173.
2017-03-01 14:41:54 +01:00
Eelco Dolstra c4a40949d9 Handle importing NARs containing files greater than 4 GiB
Also templatize readInt() to work for various integer types.
2017-03-01 13:52:54 +01:00
Eelco Dolstra 0780805246 Fix building against older curl versions
http://hydra.nixos.org/build/49490928
2017-02-28 14:03:43 +01:00
Eelco Dolstra fd86dd93dd Improve SQLite busy handling 2017-02-28 13:59:11 +01:00
Eelco Dolstra 34b12bad59 NarInfoDiskCache: Handle SQLite busy errors 2017-02-28 13:44:11 +01:00
Eelco Dolstra 80027144ae In SQLite errors, include the database path
This is necessary because we have multiple SQLite databases (e.g. the
binary cache cache).
2017-02-28 13:20:12 +01:00
Eelco Dolstra 7251d048fa Support auto-configuration of build-max-jobs
"build-max-jobs" and the "-j" option can now be set to "auto" to use
the number of CPUs in the system. (Unlike build-cores, it doesn't use
0 to imply auto-configuration, because a) magic values are a bad idea
in general; b) 0 is a legitimate value used to disable local
building.)

Fixes #1198.
2017-02-28 12:54:50 +01:00
Eelco Dolstra 3fab1f04a7 _SC_NPROCESSORS_ONLN -> std::thread::hardware_concurrency() 2017-02-27 16:01:54 +01:00
Eelco Dolstra 4f3fb34844 Fix 1.8 release note about build-max-jobs
We set build-cores automatically, not build-max-jobs. (The commit
message for de4cdd0d47 also got this
wrong.)
2017-02-27 15:58:24 +01:00
Eelco Dolstra 3cb0387d3f Retry downloads on HTTP/2 stream errors
Issue #1254.
2017-02-27 14:36:09 +01:00
Domen Kožar f0bbd153c6 Merge pull request #1252 from bjornfor/channel-rename-fix
unpack-channel.nix: fix 'mv' corner case
2017-02-27 13:12:35 +01:00
Bjørn Forsman f72206b736 unpack-channel.nix: fix 'mv' corner case
unpack-channel.nix fails if the tarball contains a directory named the
same as the channel:

  mv: cannot move 'nixpkgs' to a subdirectory of itself, '.../nixpkgs'

This commit fixes that by not moving the directory if it already has the
correct name.
2017-02-27 10:12:58 +01:00
Dan Peebles e7cb2847ab Explicitly model all settings and fail on unrecognized ones
Previously, the Settings class allowed other code to query for string
properties, which led to a proliferation of code all over the place making
up new options without any sort of central registry of valid options. This
commit pulls all those options back into the central Settings class and
removes the public get() methods, to discourage future abuses like that.

Furthermore, because we know the full set of options ahead of time, we
now fail loudly if someone enters an unrecognized option, thus preventing
subtle typos. With some template fun, we could probably also dump the full
set of options (with documentation, defaults, etc.) to the command line,
but I'm not doing that yet here.
2017-02-22 20:19:25 -05:00
Janus Troelsen 2b4c24f46a Remove Perl dependency listing in doc 2017-02-07 15:56:32 -05:00
Shea Levy f7b7df8d1f Add nix-perl package for the perl bindings 2017-02-07 15:56:32 -05:00
Shea Levy 418a837897 Remove perl dependency.
Fixes #341
2017-02-07 15:56:32 -05:00
Eelco Dolstra 0e49f94120 Merge pull request #35 from orbekk/master
Update <nixpkgs/nixos> path in documentation.
2017-01-02 11:51:21 +01:00
Kjetil Orbekk 5476e987d5 Update <nixpkgs/nixos> path in documentation. 2017-01-01 16:13:11 -05:00
Eelco Dolstra 8b60529231 Strip ANSI escapes from file names
Also, use bright blue for lambdas, otherwise my eyes hurt.
2016-07-21 11:21:59 +02:00
Eelco Dolstra af5ff6c918 Merge branch 'lambda-position' of https://github.com/fkz/nix-repl 2016-07-21 11:16:22 +02:00
Eelco Dolstra df32610dfa Merge branch 'show-trace' of https://github.com/fkz/nix-repl 2016-07-21 11:14:37 +02:00
Eelco Dolstra 00ee039c27 Merge branch 'colorize' of https://github.com/ehmry/nix-repl 2016-07-21 11:12:45 +02:00
Emery Hemingway 34ec98176e fixup "Colorize"
Do not quote escape sequence macro
2016-06-25 13:40:50 +02:00
Emery Hemingway eaabcba1c3 Colorize
Berlin NixOS meetup
2016-06-25 13:28:27 +02:00
Fabian Schmitthenner 828cf7b058 show trace of errors when using --show-trace 2016-03-19 13:55:10 +00:00
Eelco Dolstra e37bca136e Merge pull request #19 from tsion/x-command
Add :u command which works like `nix-shell -p`.
2016-03-07 15:52:02 +01:00
Fabian Schmitthenner 8bec2c07a1 When showing a lambda, also show the position of the definition 2016-03-05 16:48:04 +00:00
Scott Olson e2ff27da07 Rename :x to :u, for 'use'. 2016-03-01 04:03:18 -06:00
Scott Olson 86e93b9f61 Add :x command which works like nix-shell -p. 2016-03-01 04:03:18 -06:00
Eelco Dolstra 79b02dffcb Merge pull request #20 from tsion/more-help
Expand the help message printed from --help.
2016-02-29 12:48:04 +01:00
Eelco Dolstra 2bfb00c66e Merge pull request #17 from tsion/install-cmd
Add :i command to install a derivation to the current profile.
2016-02-29 12:47:00 +01:00
Eelco Dolstra b1cc845413 Merge pull request #25 from fkz/fix-store
fix nix-repl compilation after we don't have a global store variable anymore
2016-02-29 12:46:15 +01:00
Fabian Schmitthenner ff8d0698c7 fix nix-repl after we don't have a global store variable anymore (cf nix@c10c61449f954702ae6d8092120321744acd82ff) 2016-02-28 22:44:08 +00:00
Eelco Dolstra a1ea85e92b Merge pull request #24 from tsion/whitespace
Fix handling of whitespace and ignore blank inputs.
2016-02-24 12:02:06 +01:00
Scott Olson 38816759fc Ignore blank inputs.
Previously, nix-repl would consider this an incomplete parse and wait for the
next line as if it was a multiline input.

Blank lines in the middle of a multiline input will continue to work.
2016-02-23 18:30:21 -06:00
Scott Olson 87e6649fc3 Fix handling of whitespace.
Whitespace will no longer be removed from input lines, which fixes pasting
multiline strings containing end-of-line or beginning-of-line whitespace.
2016-02-23 18:29:56 -06:00
Ole Jørgen Brønner 103c46abc2 Preserve readline history across sessions. Add rl_readline_name. 2016-02-23 23:19:49 +01:00
Eelco Dolstra 5599665a27 Merge pull request #22 from tsion/quote-invalid-vars
Print syntactially invalid attribute names as strings.
2016-02-23 11:07:55 +01:00
Scott Olson 97da6d62f2 Print syntactially invalid attribute names as strings. 2016-02-20 01:14:22 -06:00
Eelco Dolstra a52fd0dbd0 Merge branch 'issue-13' of https://github.com/tsion/nix-repl 2016-02-18 19:40:27 +01:00
Eelco Dolstra 02daf2ec0b Merge branch 'multiline' of https://github.com/tsion/nix-repl 2016-02-18 19:33:46 +01:00
Scott Olson 56c7f0e8c5 Fix typo in comment. 2016-02-18 07:04:55 -06:00
Scott Olson 60ba98242f Fix recognition of REPL commands. 2016-02-18 06:59:51 -06:00
Scott Olson 64080d26fe Cancel multiline input on Ctrl-C. 2016-02-18 06:50:52 -06:00
Scott Olson 2d729e4f6f Support multiline input by detecting "unfinished" parse errors.
Fixes #4,
2016-02-18 06:27:39 -06:00
Scott Olson 287dfee35e Expand the help message printed from --help.
Fixes #10. I consider this a temporary measure, however, until nix-repl has a
manpage (see #14). Then it can just open its manpage on --help like the other
nix tools do.

Much of the text in this commit was copied from nix-build's manpage.
2016-02-18 04:05:11 -06:00
Scott Olson cfc874ee52 Open the store before constructing EvalState.
EvalState requires the `store` global to be initialized before it is constructed
in some cases, e.g. when it needs to download a tarball for something in
NIX_PATH. Hence, this fixes #13.
2016-02-16 18:40:47 -06:00
Scott Olson 82aca33899 Add :i command to install a package to the current profile.
It works by running `nix-env -i <derivation path>`.

Fixes #15.
2016-02-16 00:36:50 -06:00
Scott Olson 30a7bfbebe Fix grammar. 2016-02-15 23:11:26 -06:00
Scott Olson 3cfb8d1584 Remove unused global variable. 2016-02-15 19:16:24 -06:00
Eelco Dolstra dc8b51754b Merge pull request #16 from tsion/dashed-assign
Improve variable name parsing for assignments.
2016-02-15 10:25:45 +01:00
Scott Olson f30fd9c47b Don't consider empty strings or strings beginning with numbers as variable names. 2016-02-14 01:50:47 -06:00
Scott Olson 2111098a3a Don't consider strings starting with - or ' as variable names. 2016-02-14 01:29:48 -06:00
Scott Olson f7980b4712 Parse foo-bar = expr as an assignment. 2016-02-14 01:16:30 -06:00
Eelco Dolstra 8a2f5f0607 Fix building against Nix 1.10
Fixes #12.
2015-09-07 13:05:58 +02:00
Susan Potter 57aeef0b6a Fix nix-repl does not support '--help'
According to popular practice and convention `nix-repl` now supports `--help`
like a good POSIX citizen[1].

[1] https://www.gnu.org/prep/standards/html_node/Command_002dLine-Interfaces.html
2015-07-06 15:53:47 +02:00
Eelco Dolstra 45c6405a30 Fix building against latest Nix
Fixes #8.
Fixes #9.
2015-06-02 13:24:24 +02:00
Eelco Dolstra f92408136e Fix building against current Nix master 2014-12-01 10:07:10 +01:00
Eelco Dolstra 71d61508f2 Support -I flag 2014-08-26 20:05:08 +02:00
Eelco Dolstra 89f9c0d41b Fix building against current Nix master 2014-08-26 20:03:12 +02:00
Eelco Dolstra 02b66e97ba Fix building against current Nix master 2014-07-24 17:53:32 +02:00
Eelco Dolstra 2cf0e67761 Handle non-numeric version strings
Fixes #2.
2014-07-24 17:46:58 +02:00
Eelco Dolstra 66b2d18243 Don't parse 'var == expr' as an assignment 2014-06-16 10:05:09 -04:00
Eelco Dolstra 1734e8a149 Fix crash in tab completion
Fixes #1. Patch by Maxdamantus.
2014-04-11 12:51:15 +02:00
Eelco Dolstra 22a47ab03c Fix building against Nix 1.7 2014-04-11 12:50:46 +02:00
Eelco Dolstra 6a4a8208be Fix building against current Nix 2014-01-28 10:42:23 +01:00
Eelco Dolstra 3beb6f6e76 Show derivations more concisely 2014-01-28 10:42:23 +01:00
Eelco Dolstra 81d658fe4a Fix readme 2013-09-09 18:11:47 +02:00
Eelco Dolstra 853d2e0aa4 Fix markdown 2013-09-09 18:10:57 +02:00
Eelco Dolstra ae50a5e7be Add a README 2013-09-09 18:09:46 +02:00
Eelco Dolstra dc670a173a Make tab-completion work on builtins 2013-09-09 17:22:42 +02:00
Eelco Dolstra e91160021f On reload, wipe the environment 2013-09-09 17:06:14 +02:00
Eelco Dolstra 498f8b0485 Add license 2013-09-09 16:17:45 +02:00
Eelco Dolstra adde4f0c8d Add :reload command 2013-09-09 16:02:46 +02:00
Eelco Dolstra ddd22c37c5 Build on Mac OS X 2013-09-09 16:02:35 +02:00
Eelco Dolstra 03ef6b69be Add system argument 2013-09-09 15:17:08 +02:00
Eelco Dolstra 3567bdb514 Add :quit command 2013-09-09 15:02:56 +02:00
Eelco Dolstra a5dffb3d3d Temporary hack to parameterize nixpkgs 2013-09-09 14:03:28 +02:00
Eelco Dolstra 3c67df928f Add sugar for defining a variable
‘x = <expr>’ is short for ‘:a { x = <expr>; }’.  Note that the
right-hand side refers to the original scope, so you get:

  nix-repl> x = 1
  nix-repl> x = x + 1
  nix-repl> x
  2

rather than an infinite recursion.
2013-09-09 13:56:53 +02:00
Eelco Dolstra 4b33c2dd4c Add help (:?) 2013-09-09 13:22:33 +02:00
Eelco Dolstra e133e91410 Support tab-completion on attribute sets
Example:

$ nix-repl '<nixos>'

> config.services.xserver.desktop<TAB>

comletes to

> config.services.xserver.desktopManager

You also get suggestions if there are multiple matches:

> config.services.xserver.desktopManager.kde4
config.services.xserver.desktopManager.kde4.enable
config.services.xserver.desktopManager.kde4.phononBackends
2013-09-09 12:00:33 +02:00
Eelco Dolstra 8e765b8876 printValue: Show assertion errors inline 2013-09-09 11:37:05 +02:00
Eelco Dolstra e587aec123 printValue: Don't show lists/attribute sets twice 2013-09-09 11:14:43 +02:00
Eelco Dolstra 7e3625f924 Improved value display
By default, we don't recurse into attribute sets or lists when
printing a value.  However, the new :p command does recurse.
2013-09-07 00:35:54 +02:00
Eelco Dolstra c6f2b89c0e Restore affinity 2013-09-06 21:00:36 +02:00
Eelco Dolstra 0abdf4beaa Add basic variable name completion 2013-09-06 19:51:59 +02:00
Eelco Dolstra cf4c29d90a Load files specified on the command line
For example:

  $ nix-repl '<nixpkgs>' '<nixos>'
  Welcome to Nix version 1.6pre3215_2c1ecf8. Type :? for help.

  Loading ‘<nixpkgs>’...
  Added 3337 variables.

  Loading ‘<nixos>’...
  Added 7 variables.

  nix-repl>
2013-09-06 15:20:06 +02:00
Eelco Dolstra b5944ac4ff Add a command :s to start a nix-shell for a derivation 2013-09-06 15:05:18 +02:00
Eelco Dolstra c6712a007f Add a command :b to build a derivation 2013-09-06 14:58:53 +02:00
Eelco Dolstra ad0dd359b4 Don't exit on SIGINT during evaluation
However, this may leave thunks in black-holed state, so it's not
really safe.
2013-09-06 13:20:35 +02:00
Eelco Dolstra 0b419c048b Use readline history file 2013-09-06 13:14:28 +02:00
Eelco Dolstra 4fb82d3d80 Handle SIGINT to cancel the current line 2013-09-06 13:01:02 +02:00
Eelco Dolstra 504563ea44 Fix build 2013-09-06 11:54:49 +02:00
Eelco Dolstra 0f6279d874 Add a command :l for loading a file into scope
Example:

nix-repl> :l <nixpkgs>

nix-repl> lib.range 0 10
[ 0 1 2 3 4 5 6 7 8 9 10 ]

nix-repl> :l <nixos>

nix-repl> config.boot.kernelModules
[ "vboxdrv" "vboxnetadp" ... ]
2013-09-02 16:48:41 +00:00
Eelco Dolstra 3202206d1d Add a command :t for showing the type of an expression 2013-09-02 16:00:48 +00:00
Eelco Dolstra 287c88ca59 Support adding variables to the scope
The command ":a <expr>" evaluates <expr> and adds the attributes in
the resulting attribute set to the interpreter scope.  For instance:

nix-repl> :a import <nixpkgs> {}

nix-repl> lib.range 0 10
[ 0 1 2 3 4 5 6 7 8 9 10 ]
2013-09-02 17:53:58 +02:00
Eelco Dolstra e90569905e Initial version of nix-repl
This program interactively reads a Nix expression from the user,
evaluates it, and prints the result.
2013-09-02 15:18:15 +02:00
316 changed files with 36088 additions and 6429 deletions

No files matched your search

+26
View File
@@ -0,0 +1,26 @@
# EditorConfig configuration for nix
# http://EditorConfig.org
# Top-most EditorConfig file
root = true
# Unix-style newlines with a newline ending every file, utf-8 charset
[*]
end_of_line = lf
insert_final_newline = true
trim_trailing_whitespace = true
charset = utf-8
# Match nix files, set indent to spaces with width of two
[*.nix]
indent_style = space
indent_size = 2
# Match c++/shell/perl, set indent to spaces with width of four
[*.{hpp,cc,hh,sh,pl}]
indent_style = space
indent_size = 4
# Match diffs, avoid to trim trailing whitespace
[*.{diff,patch}]
trim_trailing_whitespace = false
+27
View File
@@ -0,0 +1,27 @@
<!--
# Filing a Nix issue
*WAIT* Are you sure you're filing your issue in the right repository?
We appreciate you taking the time to tell us about issues you encounter, but routing the issue to the right place will get you help sooner and save everyone time.
This is the Nix repository, and issues here should be about Nix the build and package management *_tool_*.
If you have a problem with a specific package on NixOS or when using Nix, you probably want to file an issue with _nixpkgs_, whose issue tracker is over at https://github.com/NixOS/nixpkgs/issues.
Examples of _Nix_ issues:
- Nix segfaults when I run `nix-build -A blahblah`
- The Nix language needs a new builtin: `builtins.foobar`
- Regression in the behavior of `nix-env` in Nix 2.0
Examples of _nixpkgs_ issues:
- glibc is b0rked on aarch64
- chromium in NixOS doesn't support U2F but google-chrome does!
- The OpenJDK package on macOS is missing a key component
Chances are if you're a newcomer to the Nix world, you'll probably want the [nixpkgs tracker](https://github.com/NixOS/nixpkgs/issues). It also gets a lot more eyeball traffic so you'll probably get a response a lot more quickly.
-->
+7 -2
View File
@@ -1,4 +1,5 @@
Makefile.config
perl/Makefile.config
# /
/aclocal.m4
@@ -37,6 +38,7 @@ Makefile.config
/scripts/nix-copy-closure
/scripts/nix-reduce-build
/scripts/nix-http-export.cgi
/scripts/nix-profile-daemon.sh
# /src/libexpr/
/src/libexpr/lexer-tab.cc
@@ -47,8 +49,7 @@ Makefile.config
/src/libexpr/nix.tbl
# /src/libstore/
/src/libstore/schema.sql.hh
/src/libstore/sandbox-defaults.sb
/src/libstore/*.gen.hh
/src/nix/nix
@@ -99,16 +100,20 @@ Makefile.config
/misc/systemd/nix-daemon.socket
/misc/upstart/nix-daemon.conf
/src/resolve-system-dependencies/resolve-system-dependencies
inst/
*.a
*.o
*.so
*.dylib
*.dll
*.exe
*.dep
*~
*.pc
*.plist
# GNU Global
GPATH
+2
View File
@@ -0,0 +1,2 @@
os: osx
script: ./tests/install-darwin.sh
+2 -3
View File
@@ -18,15 +18,14 @@ makefiles = \
src/nix-channel/local.mk \
src/nix-build/local.mk \
src/build-remote/local.mk \
perl/local.mk \
scripts/local.mk \
corepkgs/local.mk \
misc/systemd/local.mk \
misc/launchd/local.mk \
misc/upstart/local.mk \
misc/emacs/local.mk \
doc/manual/local.mk \
tests/local.mk
tests/local.mk \
tests/plugins/local.mk
GLOBAL_CXXFLAGS += -std=c++14 -g -Wall -include config.h
+7 -4
View File
@@ -5,6 +5,9 @@ CXX = @CXX@
CXXFLAGS = @CXXFLAGS@
ENABLE_S3 = @ENABLE_S3@
HAVE_SODIUM = @HAVE_SODIUM@
HAVE_READLINE = @HAVE_READLINE@
HAVE_BROTLI = @HAVE_BROTLI@
HAVE_SECCOMP = @HAVE_SECCOMP@
LIBCURL_LIBS = @LIBCURL_LIBS@
OPENSSL_LIBS = @OPENSSL_LIBS@
PACKAGE_NAME = @PACKAGE_NAME@
@@ -12,9 +15,11 @@ PACKAGE_VERSION = @PACKAGE_VERSION@
SODIUM_LIBS = @SODIUM_LIBS@
LIBLZMA_LIBS = @LIBLZMA_LIBS@
SQLITE3_LIBS = @SQLITE3_LIBS@
LIBBROTLI_LIBS = @LIBBROTLI_LIBS@
bash = @bash@
bindir = @bindir@
curl = @curl@
brotli = @brotli@
lsof = @lsof@
datadir = @datadir@
datarootdir = @datarootdir@
docdir = @docdir@
@@ -24,11 +29,9 @@ libdir = @libdir@
libexecdir = @libexecdir@
localstatedir = @localstatedir@
mandir = @mandir@
perl = @perl@
perlbindings = @perlbindings@
perllibdir = @perllibdir@
pkglibdir = $(libdir)/$(PACKAGE_NAME)
prefix = @prefix@
sandbox_shell = @sandbox_shell@
storedir = @storedir@
sysconfdir = @sysconfdir@
doc_generate = @doc_generate@
+1 -1
View File
@@ -1,7 +1,7 @@
Nix, the purely functional package manager
------------------------------------------
Nix is a new take on package management that is fairly unique. Because of it's
Nix is a new take on package management that is fairly unique. Because of its
purity aspects, a lot of issues found in traditional package managers don't
appear with Nix.
+46 -55
View File
@@ -61,6 +61,7 @@ CFLAGS=
CXXFLAGS=
AC_PROG_CC
AC_PROG_CXX
AC_PROG_CPP
AX_CXX_COMPILE_STDCXX_11
@@ -114,14 +115,12 @@ if test -z "$$1"; then
fi
])
NEED_PROG(curl, curl)
NEED_PROG(bash, bash)
NEED_PROG(patch, patch)
AC_PATH_PROG(xmllint, xmllint, false)
AC_PATH_PROG(xsltproc, xsltproc, false)
AC_PATH_PROG(flex, flex, false)
AC_PATH_PROG(bison, bison, false)
NEED_PROG(perl, perl)
NEED_PROG(sed, sed)
NEED_PROG(tar, tar)
NEED_PROG(bzip2, bzip2)
@@ -129,23 +128,8 @@ NEED_PROG(gzip, gzip)
NEED_PROG(xz, xz)
AC_PATH_PROG(dot, dot)
AC_PATH_PROG(pv, pv, pv)
# Test that Perl has the open/fork feature (Perl 5.8.0 and beyond).
AC_MSG_CHECKING([whether Perl is recent enough])
if ! $perl -e 'open(FOO, "-|", "true"); while (<FOO>) { print; }; close FOO or die;'; then
AC_MSG_RESULT(no)
AC_MSG_ERROR([Your Perl version is too old. Nix requires Perl 5.8.0 or newer.])
fi
AC_MSG_RESULT(yes)
# Figure out where to install Perl modules.
AC_MSG_CHECKING([for the Perl installation prefix])
perlversion=$($perl -e 'use Config; print $Config{version};')
perlarchname=$($perl -e 'use Config; print $Config{archname};')
AC_SUBST(perllibdir, [${libdir}/perl5/site_perl/$perlversion/$perlarchname])
AC_MSG_RESULT($perllibdir)
AC_PATH_PROGS(brotli, brotli bro, bro)
AC_PATH_PROG(lsof, lsof, lsof)
NEED_PROG(cat, cat)
@@ -191,16 +175,51 @@ AC_SUBST(HAVE_SODIUM, [$have_sodium])
# Look for liblzma, a required dependency.
PKG_CHECK_MODULES([LIBLZMA], [liblzma], [CXXFLAGS="$LIBLZMA_CFLAGS $CXXFLAGS"])
AC_CHECK_LIB([lzma], [lzma_stream_encoder_mt],
[AC_DEFINE([HAVE_LZMA_MT], [1], [xz multithreaded compression support])])
# Look for libbrotli{enc,dec}, optional dependencies
PKG_CHECK_MODULES([LIBBROTLI], [libbrotlienc libbrotlidec],
[AC_DEFINE([HAVE_BROTLI], [1], [Whether to use libbrotli.])
CXXFLAGS="$LIBBROTLI_CFLAGS $CXXFLAGS"]
have_brotli=1], [have_brotli=])
AC_SUBST(HAVE_BROTLI, [$have_brotli])
# Look for libseccomp, required for Linux sandboxing.
if test "$sys_name" = linux; then
AC_ARG_ENABLE([seccomp-sandboxing],
AC_HELP_STRING([--disable-seccomp-sandboxing],
[Don't build support for seccomp sandboxing (only recommended if your arch doesn't support libseccomp yet!)]
))
if test "x$enable_seccomp_sandboxing" != "xno"; then
PKG_CHECK_MODULES([LIBSECCOMP], [libseccomp],
[CXXFLAGS="$LIBSECCOMP_CFLAGS $CXXFLAGS"])
have_seccomp=1
AC_DEFINE([HAVE_SECCOMP], [1], [Whether seccomp is available and should be used for sandboxing.])
else
have_seccomp=
fi
else
have_seccomp=
fi
AC_SUBST(HAVE_SECCOMP, [$have_seccomp])
# Look for aws-cpp-sdk-s3.
AC_LANG_PUSH(C++)
AC_CHECK_HEADERS([aws/s3/S3Client.h],
[AC_DEFINE([ENABLE_S3], [1], [Whether to enable S3 support via aws-cpp-sdk-s3.])
[AC_DEFINE([ENABLE_S3], [1], [Whether to enable S3 support via aws-sdk-cpp.])
enable_s3=1], [enable_s3=])
AC_SUBST(ENABLE_S3, [$enable_s3])
AC_LANG_POP(C++)
if test -n "$enable_s3"; then
declare -a aws_version_tokens=($(printf '#include <aws/core/VersionConfig.h>\nAWS_SDK_VERSION_STRING' | $CPP - | grep -v '^#.*' | sed 's/"//g' | tr '.' ' '))
AC_DEFINE_UNQUOTED([AWS_VERSION_MAJOR], ${aws_version_tokens@<:@0@:>@}, [Major version of aws-sdk-cpp.])
AC_DEFINE_UNQUOTED([AWS_VERSION_MINOR], ${aws_version_tokens@<:@1@:>@}, [Minor version of aws-sdk-cpp.])
fi
# Whether to use the Boehm garbage collector.
AC_ARG_ENABLE(gc, AC_HELP_STRING([--enable-gc],
@@ -213,40 +232,6 @@ if test "$gc" = yes; then
fi
# Check for the required Perl dependencies (DBI, DBD::SQLite).
perlFlags="-I$perllibdir"
AC_ARG_WITH(dbi, AC_HELP_STRING([--with-dbi=PATH],
[prefix of the Perl DBI library]),
perlFlags="$perlFlags -I$withval")
AC_ARG_WITH(dbd-sqlite, AC_HELP_STRING([--with-dbd-sqlite=PATH],
[prefix of the Perl DBD::SQLite library]),
perlFlags="$perlFlags -I$withval")
AC_MSG_CHECKING([whether DBD::SQLite works])
if ! $perl $perlFlags -e 'use DBI; use DBD::SQLite;' 2>&5; then
AC_MSG_RESULT(no)
AC_MSG_FAILURE([The Perl modules DBI and/or DBD::SQLite are missing.])
fi
AC_MSG_RESULT(yes)
AC_SUBST(perlFlags)
# Whether to build the Perl bindings
AC_MSG_CHECKING([whether to build the Perl bindings])
AC_ARG_ENABLE(perl-bindings, AC_HELP_STRING([--enable-perl-bindings],
[whether to build the Perl bindings (recommended) [default=yes]]),
perlbindings=$enableval, perlbindings=yes)
if test "$enable_shared" = no; then
# Perl bindings require shared libraries.
perlbindings=no
fi
AC_SUBST(perlbindings)
AC_MSG_RESULT($perlbindings)
AC_ARG_ENABLE(init-state, AC_HELP_STRING([--disable-init-state],
[do not initialise DB etc. in `make install']),
init_state=$enableval, init_state=yes)
@@ -265,7 +250,7 @@ AC_CHECK_FUNCS([setresuid setreuid lchown])
# Nice to have, but not essential.
AC_CHECK_FUNCS([strsignal posix_fallocate nanosleep sysconf])
AC_CHECK_FUNCS([strsignal posix_fallocate sysconf])
# This is needed if bzip2 is a static library, and the Nix libraries
@@ -291,6 +276,12 @@ fi
AC_SUBST(tarFlags)
AC_ARG_WITH(sandbox-shell, AC_HELP_STRING([--with-sandbox-shell=PATH],
[path of a statically-linked shell to use as /bin/sh in sandboxes]),
sandbox_shell=$withval)
AC_SUBST(sandbox_shell)
# Expand all variables in config.status.
test "$prefix" = NONE && prefix=$ac_default_prefix
test "$exec_prefix" = NONE && exec_prefix='${prefix}'
+3
View File
@@ -14,6 +14,9 @@ in rec {
nixBinDir = fromEnv "NIX_BIN_DIR" "@bindir@";
nixPrefix = "@prefix@";
nixLibexecDir = fromEnv "NIX_LIBEXEC_DIR" "@libexecdir@";
nixLocalstateDir = "@localstatedir@";
nixSysconfDir = "@sysconfdir@";
nixStoreDir = fromEnv "NIX_STORE_DIR" "@storedir@";
# If Nix is installed in the Nix store, then automatically add it as
# a dependency to the core packages. This ensures that they work
+7 -5
View File
@@ -1,10 +1,10 @@
{ system ? builtins.currentSystem
{ system ? "" # obsolete
, url
, md5 ? "", sha1 ? "", sha256 ? ""
, md5 ? "", sha1 ? "", sha256 ? "", sha512 ? ""
, outputHash ?
if sha1 != "" then sha1 else if md5 != "" then md5 else sha256
if sha512 != "" then sha512 else if sha1 != "" then sha1 else if md5 != "" then md5 else sha256
, outputHashAlgo ?
if sha1 != "" then "sha1" else if md5 != "" then "md5" else "sha256"
if sha512 != "" then "sha512" else if sha1 != "" then "sha1" else if md5 != "" then "md5" else "sha256"
, executable ? false
, unpack ? false
, name ? baseNameOf (toString url)
@@ -17,7 +17,9 @@ derivation {
inherit outputHashAlgo outputHash;
outputHashMode = if unpack || executable then "recursive" else "flat";
inherit name system url executable unpack;
inherit name url executable unpack;
system = "builtin";
# No need to double the amount of network traffic
preferLocalBuild = true;
+3 -1
View File
@@ -15,7 +15,9 @@ let
else
${bzip2} -d < $src | ${tar} xf - ${tarFlags}
fi
mv * $out/$channelName
if [ * != $channelName ]; then
mv * $out/$channelName
fi
if [ -n "$binaryCacheURL" ]; then
mkdir $out/binary-caches
echo -n "$binaryCacheURL" > $out/binary-caches/$channelName
+140 -74
View File
@@ -4,74 +4,109 @@
version="5.0"
xml:id='chap-distributed-builds'>
<title>Distributed Builds</title>
<title>Remote Builds</title>
<para>Nix supports distributed builds, where a local Nix installation can
forward Nix builds to other machines over the network. This allows
multiple builds to be performed in parallel (thus improving
performance) and allows Nix to perform multi-platform builds in a
semi-transparent way. For instance, if you perform a build for a
<literal>x86_64-darwin</literal> on an <literal>i686-linux</literal>
machine, Nix can automatically forward the build to a
<literal>x86_64-darwin</literal> machine, if available.</para>
<para>Nix supports remote builds, where a local Nix installation can
forward Nix builds to other machines. This allows multiple builds to
be performed in parallel and allows Nix to perform multi-platform
builds in a semi-transparent way. For instance, if you perform a
build for a <literal>x86_64-darwin</literal> on an
<literal>i686-linux</literal> machine, Nix can automatically forward
the build to a <literal>x86_64-darwin</literal> machine, if
available.</para>
<para>You can enable distributed builds by setting the environment
variable <envar>NIX_BUILD_HOOK</envar> to point to a program that Nix
will call whenever it wants to build a derivation. The build hook
(typically a shell or Perl script) can decline the build, in which Nix
will perform it in the usual way if possible, or it can accept it, in
which case it is responsible for somehow getting the inputs of the
build to another machine, doing the build there, and getting the
results back. The details of the build hook protocol are described in
the documentation of the <link
linkend="envar-build-hook"><envar>NIX_BUILD_HOOK</envar>
variable</link>.</para>
<para>To forward a build to a remote machine, it’s required that the
remote machine is accessible via SSH and that it has Nix
installed. You can test whether connecting to the remote Nix instance
works, e.g.
<example xml:id='ex-remote-systems'><title>Remote machine configuration:
<filename>remote-systems.conf</filename></title>
<programlisting>
nix@mcflurry.labs.cs.uu.nl x86_64-darwin /home/nix/.ssh/id_quarterpounder_auto 2
nix@scratchy.labs.cs.uu.nl i686-linux /home/nix/.ssh/id_scratchy_auto 8 1 kvm
nix@itchy.labs.cs.uu.nl i686-linux /home/nix/.ssh/id_scratchy_auto 8 2
nix@poochie.labs.cs.uu.nl i686-linux /home/nix/.ssh/id_scratchy_auto 8 2 kvm perf
</programlisting>
</example>
<screen>
$ nix ping-store --store ssh://mac
</screen>
<para>Nix ships with a build hook that should be suitable for most
purposes. It uses <command>ssh</command> and
<command>nix-copy-closure</command> to copy the build inputs and
outputs and perform the remote build. To use it, you should set
<envar>NIX_BUILD_HOOK</envar> to
<filename><replaceable>prefix</replaceable>/libexec/nix/build-remote.pl</filename>.
You should also define a list of available build machines and point
the environment variable <envar>NIX_REMOTE_SYSTEMS</envar> to
it. <envar>NIX_REMOTE_SYSTEMS</envar> must be an absolute path. An
example configuration is shown in <xref linkend='ex-remote-systems'
/>. Each line in the file specifies a machine, with the following
bits of information:
will try to connect to the machine named <literal>mac</literal>. It is
possible to specify an SSH identity file as part of the remote store
URI, e.g.
<screen>
$ nix ping-store --store ssh://mac?ssh-key=/home/alice/my-key
</screen>
Since builds should be non-interactive, the key should not have a
passphrase. Alternatively, you can load identities ahead of time into
<command>ssh-agent</command> or <command>gpg-agent</command>.</para>
<para>If you get the error
<screen>
bash: nix-store: command not found
error: cannot connect to 'mac'
</screen>
then you need to ensure that the <envar>PATH</envar> of
non-interactive login shells contains Nix.</para>
<warning><para>If you are building via the Nix daemon, it is the Nix
daemon user account (that is, <literal>root</literal>) that should
have SSH access to the remote machine. If you can’t or don’t want to
configure <literal>root</literal> to be able to access to remote
machine, you can use a private Nix store instead by passing
e.g. <literal>--store ~/my-nix</literal>.</para></warning>
<para>The list of remote machines can be specified on the command line
or in the Nix configuration file. The former is convenient for
testing. For example, the following command allows you to build a
derivation for <literal>x86_64-darwin</literal> on a Linux machine:
<screen>
$ uname
Linux
$ nix build \
'(with import &lt;nixpkgs> { system = "x86_64-darwin"; }; runCommand "foo" {} "uname > $out")' \
--builders 'ssh://mac x86_64-darwin'
[1/0/1 built, 0.0 MiB DL] building foo on ssh://mac
$ cat ./result
Darwin
</screen>
It is possible to specify multiple builders separated by a semicolon
or a newline, e.g.
<screen>
--builders 'ssh://mac x86_64-darwin ; ssh://beastie x86_64-freebsd'
</screen>
</para>
<para>Each machine specification consists of the following elements,
separated by spaces. Only the first element is required.
<orderedlist>
<listitem><para>The name of the remote machine, with optionally the
user under which the remote build should be performed. This is
actually passed as an argument to <command>ssh</command>, so it can
be an alias defined in your
<listitem><para>The URI of the remote store in the format
<literal>ssh://[<replaceable>username</replaceable>@]<replaceable>hostname</replaceable></literal>,
e.g. <literal>ssh://nix@mac</literal> or
<literal>ssh://mac</literal>. For backward compatibility,
<literal>ssh://</literal> may be omitted. The hostname may be an
alias defined in your
<filename>~/.ssh/config</filename>.</para></listitem>
<listitem><para>A comma-separated list of Nix platform type
identifiers, such as <literal>x86_64-darwin</literal>. It is
possible for a machine to support multiple platform types, e.g.,
<literal>i686-linux,x86_64-linux</literal>.</para></listitem>
<literal>i686-linux,x86_64-linux</literal>. If omitted, this
defaults to the local platform type.</para></listitem>
<listitem><para>The SSH private key to be used to log in to the
remote machine. Since builds should be non-interactive, this key
should not have a passphrase!</para></listitem>
<listitem><para>The SSH identity file to be used to log in to the
remote machine. If omitted, SSH will use its regular
identities.</para></listitem>
<listitem><para>The maximum number of builds that
<filename>build-remote.pl</filename> will execute in parallel on the
machine. Typically this should be equal to the number of CPU cores.
For instance, the machine <literal>itchy</literal> in the example
will execute up to 8 builds in parallel.</para></listitem>
<listitem><para>The maximum number of builds that Nix will execute
in parallel on the machine. Typically this should be equal to the
number of CPU cores. For instance, the machine
<literal>itchy</literal> in the example will execute up to 8 builds
in parallel.</para></listitem>
<listitem><para>The “speed factor”, indicating the relative speed of
the machine. If there are multiple machines of the right type, Nix
@@ -79,38 +114,69 @@ bits of information:
<listitem><para>A comma-separated list of <emphasis>supported
features</emphasis>. If a derivation has the
<varname>requiredSystemFeatures</varname> attribute, then
<filename>build-remote.pl</filename> will only perform the
derivation on a machine that has the specified features. For
instance, the attribute
<varname>requiredSystemFeatures</varname> attribute, then Nix will
only perform the derivation on a machine that has the specified
features. For instance, the attribute
<programlisting>
requiredSystemFeatures = [ "kvm" ];
</programlisting>
will cause the build to be performed on a machine that has the
<literal>kvm</literal> feature (i.e., <literal>scratchy</literal> in
the example above).</para></listitem>
<literal>kvm</literal> feature.</para></listitem>
<listitem><para>A comma-separated list of <emphasis>mandatory
features</emphasis>. A machine will only be used to build a
derivation if all of the machine’s mandatory features appear in the
derivation’s <varname>requiredSystemFeatures</varname> attribute.
Thus, in the example, the machine <literal>poochie</literal> will
only do derivations that have
<varname>requiredSystemFeatures</varname> set to <literal>["kvm"
"perf"]</literal> or <literal>["perf"]</literal>.</para></listitem>
derivation’s <varname>requiredSystemFeatures</varname>
attribute..</para></listitem>
</orderedlist>
You should also set up the environment variable
<envar>NIX_CURRENT_LOAD</envar> to point at a directory (e.g.,
<filename>/var/run/nix/current-load</filename>) that
<filename>build-remote.pl</filename> uses to remember how many builds
it is currently executing remotely. It doesn't look at the actual
load on the remote machine, so if you have multiple instances of Nix
running, they should use the same <envar>NIX_CURRENT_LOAD</envar>
file. Maybe in the future <filename>build-remote.pl</filename> will
look at the actual remote load.</para>
For example, the machine specification
<programlisting>
nix@scratchy.labs.cs.uu.nl i686-linux /home/nix/.ssh/id_scratchy_auto 8 1 kvm
nix@itchy.labs.cs.uu.nl i686-linux /home/nix/.ssh/id_scratchy_auto 8 2
nix@poochie.labs.cs.uu.nl i686-linux /home/nix/.ssh/id_scratchy_auto 1 2 kvm benchmark
</programlisting>
specifies several machines that can perform
<literal>i686-linux</literal> builds. However,
<literal>poochie</literal> will only do builds that have the attribute
<programlisting>
requiredSystemFeatures = [ "benchmark" ];
</programlisting>
or
<programlisting>
requiredSystemFeatures = [ "benchmark" "kvm" ];
</programlisting>
<literal>itchy</literal> cannot do builds that require
<literal>kvm</literal>, but <literal>scratchy</literal> does support
such builds. For regular builds, <literal>itchy</literal> will be
preferred over <literal>scratchy</literal> because it has a higher
speed factor.</para>
<para>Remote builders can also be configured in
<filename>nix.conf</filename>, e.g.
<programlisting>
builders = ssh://mac x86_64-darwin ; ssh://beastie x86_64-freebsd
</programlisting>
Finally, remote builders can be configured in a separate configuration
file included in <option>builders</option> via the syntax
<literal>@<replaceable>file</replaceable></literal>. For example,
<programlisting>
builders = @/etc/nix/machines
</programlisting>
causes the list of machines in <filename>/etc/nix/machines</filename>
to be included. (This is the default.)</para>
</chapter>
File diff suppressed because it is too large. Load diff
+2 -133
View File
@@ -148,145 +148,14 @@ $ mount -o bind /mnt/otherdisk/nix /nix</screen>
</varlistentry>
<varlistentry xml:id="envar-build-hook"><term><envar>NIX_BUILD_HOOK</envar></term>
<listitem>
<para>Specifies the location of the <emphasis>build hook</emphasis>,
which is a program (typically some script) that Nix will call
whenever it wants to build a derivation. This is used to implement
distributed builds<phrase condition="manual"> (see <xref
linkend="chap-distributed-builds" />)</phrase>.</para>
<!--
The protocol by
which the calling Nix process and the build hook communicate is as
follows.
<para>The build hook is called with the following command-line
arguments:
<orderedlist>
<listitem><para>A boolean value <literal>0</literal> or
<literal>1</literal> specifying whether Nix can locally execute
more builds, as per the <link
linkend="opt-max-jobs"><option>- -max-jobs</option> option</link>.
The purpose of this argument is to allow the hook to not have to
maintain bookkeeping for the local machine.</para></listitem>
<listitem><para>The Nix platform identifier for the local machine
(e.g., <literal>i686-linux</literal>).</para></listitem>
<listitem><para>The Nix platform identifier for the derivation,
i.e., its <link linkend="attr-system"><varname>system</varname>
attribute</link>.</para></listitem>
<listitem><para>The store path of the derivation.</para></listitem>
</orderedlist>
</para>
<para>On the basis of this information, and whatever persistent
state the build hook keeps about other machines and their current
load, it has to decide what to do with the build. It should print
out on standard error one of the following responses (terminated by
a newline, <literal>"\n"</literal>):
<variablelist>
<varlistentry><term><literal># decline</literal></term>
<listitem><para>The build hook is not willing or able to perform
the build; the calling Nix process should do the build itself,
if possible.</para></listitem>
</varlistentry>
<varlistentry><term><literal># postpone</literal></term>
<listitem><para>The build hook cannot perform the build now, but
can do so in the future (e.g., because all available build slots
on remote machines are in use). The calling Nix process should
postpone this build until at least one currently running build
has terminated.</para></listitem>
</varlistentry>
<varlistentry><term><literal># accept</literal></term>
<listitem><para>The build hook has accepted the
build.</para></listitem>
</varlistentry>
</variablelist>
</para>
<para>After sending <literal># accept</literal>, the hook should
read one line from standard input, which will be the string
<literal>okay</literal>. It can then proceed with the build.
Before sending <literal>okay</literal>, Nix will store in the hook’s
current directory a number of text files that contain information
about the derivation:
<variablelist>
<varlistentry><term><filename>inputs</filename></term>
<listitem><para>The set of store paths that are inputs to the
build process (one per line). These have to be copied
<emphasis>to</emphasis> the remote machine (in addition to the
store derivation itself).</para></listitem>
</varlistentry>
<varlistentry><term><filename>outputs</filename></term>
<listitem><para>The set of store paths that are outputs of the
derivation (one per line). These have to be copied
<emphasis>from</emphasis> the remote machine if the build
succeeds.</para></listitem>
</varlistentry>
<varlistentry><term><filename>references</filename></term>
<listitem><para>The reference graph of the inputs, in the format
accepted by the command <command>nix-store
- -register-validity</command>. It is necessary to run this
command on the remote machine after copying the inputs to inform
Nix on the remote machine that the inputs are valid
paths.</para></listitem>
</varlistentry>
</variablelist>
</para>
<para>The hook should copy the inputs to the remote machine,
register the validity of the inputs, perform the remote build, and
copy the outputs back to the local machine. An exit code other than
<literal>0</literal> indicates that the hook has failed. An exit
code equal to 100 means that the remote build failed (as opposed to,
e.g., a network error).</para>
-->
</listitem>
</varlistentry>
<varlistentry xml:id="envar-remote"><term><envar>NIX_REMOTE</envar></term>
<listitem><para>This variable should be set to
<literal>daemon</literal> if you want to use the Nix daemon to
execute Nix operations. This is necessary in <link
linkend="ssec-multi-user">multi-user Nix installations</link>.
If the Nix daemon's Unix socket is at some non-standard path,
this variable should be set to <literal>unix://path/to/socket</literal>.
Otherwise, it should be left unset.</para></listitem>
</varlistentry>
-21
View File
@@ -29,8 +29,6 @@
</group>
<replaceable>attrPath</replaceable>
</arg>
<arg><option>--drv-link</option> <replaceable>drvlink</replaceable></arg>
<arg><option>--add-drv-link</option></arg>
<arg><option>--no-out-link</option></arg>
<arg>
<group choice='req'>
@@ -91,25 +89,6 @@ also <xref linkend="sec-common-options" />.</phrase></para>
<variablelist>
<varlistentry><term><option>--drv-link</option> <replaceable>drvlink</replaceable></term>
<listitem><para>Add a symlink named
<replaceable>drvlink</replaceable> to the store derivation
produced by <command>nix-instantiate</command>. The derivation is
a root of the garbage collector until the symlink is deleted or
renamed. If there are multiple derivations, numbers are suffixed
to <replaceable>drvlink</replaceable> to distinguish between
them.</para></listitem>
</varlistentry>
<varlistentry><term><option>--add-drv-link</option></term>
<listitem><para>Shorthand for <option>--drv-link</option>
<filename>./derivation</filename>.</para></listitem>
</varlistentry>
<varlistentry><term><option>--no-out-link</option></term>
<listitem><para>Do not create a symlink to the output path. Note
+4 -2
View File
@@ -27,8 +27,10 @@
<arg><option>- -show-progress</option></arg>
-->
<arg><option>--include-outputs</option></arg>
<arg><option>--use-substitutes</option></arg>
<arg><option>-s</option></arg>
<group>
<arg choice='plain'><option>--use-substitutes</option></arg>
<arg choice='plain'><option>-s</option></arg>
</group>
<arg><option>-v</option></arg>
<arg choice='plain'>
<replaceable>user@</replaceable><replaceable>machine</replaceable>
+3 -5
View File
@@ -146,8 +146,7 @@ also <xref linkend="sec-common-options" />.</phrase></para>
<variablelist>
<varlistentry><term><option>--file</option></term>
<term><option>-f</option></term>
<varlistentry><term><option>--file</option> / <option>-f</option> <replaceable>path</replaceable></term>
<listitem><para>Specifies the Nix expression (designated below as
the <emphasis>active Nix expression</emphasis>) used by the
@@ -166,8 +165,7 @@ also <xref linkend="sec-common-options" />.</phrase></para>
</varlistentry>
<varlistentry><term><option>--profile</option></term>
<term><option>-p</option></term>
<varlistentry><term><option>--profile</option> / <option>-p</option> <replaceable>path</replaceable></term>
<listitem><para>Specifies the profile to be used by those
operations that operate on a profile (designated below as the
@@ -1136,7 +1134,7 @@ user environment elements, etc. -->
<listitem><para>Print all of the meta-attributes of the
derivation. This option is only available with
<option>--xml</option>.</para></listitem>
<option>--xml</option> or <option>--json</option>.</para></listitem>
</varlistentry>
+12 -1
View File
@@ -44,7 +44,9 @@
cryptographic hash of the contents of each
<replaceable>path</replaceable> and prints it on standard output. By
default, it computes an MD5 hash, but other hash algorithms are
available as well. The hash is printed in hexadecimal.</para>
available as well. The hash is printed in hexadecimal. To generate
the same hash as <command>nix-prefetch-url</command> you have to
specify multiple arguments, see below for an example.</para>
<para>The hash is computed over a <emphasis>serialisation</emphasis>
of each path: a dump of the file system tree rooted at the path. This
@@ -122,6 +124,15 @@ cryptographic hash as <literal>nix-store --dump
<refsection><title>Examples</title>
<para>Computing the same hash as <command>nix-prefetch-url</command>:
<screen>
$ nix-prefetch-url file://&lt;(echo test)
1lkgqb6fclns49861dwk9rzb6xnfkxbpws74mxnx01z9qyv1pjpj
$ nix-hash --type sha256 --flat --base32 &lt;(echo test)
1lkgqb6fclns49861dwk9rzb6xnfkxbpws74mxnx01z9qyv1pjpj
</screen>
</para>
<para>Computing hashes:
<screen>
+22 -22
View File
@@ -24,6 +24,7 @@
<arg choice='plain'>
<option>--eval</option>
<arg><option>--strict</option></arg>
<arg><option>--json</option></arg>
<arg><option>--xml</option></arg>
</arg>
</group>
@@ -38,12 +39,13 @@
</arg>
<arg><option>--add-root</option> <replaceable>path</replaceable></arg>
<arg><option>--indirect</option></arg>
<group choice='req'>
<group>
<arg choice='plain'><option>--expr</option></arg>
<arg choice='plain'><option>-E</option></arg>
</group>
<arg choice='plain' rep='repeat'><replaceable>files</replaceable></arg>
<sbr/>
</cmdsynopsis>
<cmdsynopsis>
<command>nix-instantiate</command>
<arg choice='plain'><option>--find-file</option></arg>
<arg choice='plain' rep='repeat'><replaceable>files</replaceable></arg>
@@ -115,26 +117,6 @@ input.</para>
</varlistentry>
<varlistentry><term><option>--xml</option></term>
<listitem><para>When used with <option>--parse</option> and
<option>--eval</option>, print the resulting expression as an
XML representation of the abstract syntax tree rather than as an
ATerm. The schema is the same as that used by the <link
linkend="builtin-toXML"><function>toXML</function>
built-in</link>.</para></listitem>
</varlistentry>
<varlistentry><term><option>--json</option></term>
<listitem><para>When used with <option>--parse</option> and
<option>--eval</option>, print the resulting expression as an
JSON representation of the abstract syntax tree rather than as an
ATerm.</para></listitem>
</varlistentry>
<varlistentry><term><option>--strict</option></term>
<listitem><para>When used with <option>--eval</option>,
@@ -149,6 +131,24 @@ input.</para>
</varlistentry>
<varlistentry><term><option>--json</option></term>
<listitem><para>When used with <option>--eval</option>, print the resulting
value as an JSON representation of the abstract syntax tree rather
than as an ATerm.</para></listitem>
</varlistentry>
<varlistentry><term><option>--xml</option></term>
<listitem><para>When used with <option>--eval</option>, print the resulting
value as an XML representation of the abstract syntax tree rather than as
an ATerm. The schema is the same as that used by the <link
linkend="builtin-toXML"><function>toXML</function> built-in</link>.
</para></listitem>
</varlistentry>
<varlistentry><term><option>--read-write-mode</option></term>
<listitem><para>When used with <option>--eval</option>, perform
+4 -2
View File
@@ -19,14 +19,16 @@
<refsynopsisdiv>
<cmdsynopsis>
<command>nix-prefetch-url</command>
<arg><option>--version</option></arg>
<arg><option>--type</option> <replaceable>hashAlgo</replaceable></arg>
<arg><option>--print-path</option></arg>
<arg><option>--unpack</option></arg>
<arg><option>--name</option> <replaceable>name</replaceable></arg>
<arg choice='plain'><replaceable>url</replaceable></arg>
<arg><replaceable>hash</replaceable></arg>
</cmdsynopsis>
</refsynopsisdiv>
<refsection><title>Description</title>
<para>The command <command>nix-prefetch-url</command> downloads the
@@ -91,7 +93,7 @@ downloaded file in the Nix store is also printed.</para>
</varlistentry>
<varlistentry><term><option>--name</option></term>
<varlistentry><term><option>--name</option> <replaceable>name</replaceable></term>
<listitem><para>Override the name of the file in the Nix store. By
default, this is
+5 -5
View File
@@ -33,13 +33,13 @@
<arg><option>--exclude</option> <replaceable>regexp</replaceable></arg>
<arg><option>--pure</option></arg>
<group choice='req'>
<group choice='plain'>
<group>
<arg choice='plain'>
<group choice='req'>
<arg choice='plain'><option>--packages</option></arg>
<arg choice='plain'><option>-p</option></arg>
</group>
<replaceable>packages</replaceable>
</group>
<arg choice='plain' rep='repeat'><replaceable>packages</replaceable></arg>
</arg>
<arg><replaceable>path</replaceable></arg>
</group>
</cmdsynopsis>
@@ -144,7 +144,7 @@ also <xref linkend="sec-common-options" />.</phrase></para>
</varlistentry>
<varlistentry><term><option>--packages</option> / <option>-p</option></term>
<varlistentry><term><option>--packages</option> / <option>-p</option> <replaceable>packages</replaceable>…</term>
<listitem><para>Set up an environment in which the specified
packages are present. The command line arguments are interpreted
+4 -9
View File
@@ -234,7 +234,7 @@ linkend="sec-nix-build"><command>nix-build</command></link> does.</para>
<para>To test whether a previously-built derivation is deterministic:
<screen>
$ nix-build -r '&lt;nixpkgs>' -A hello --check -K
$ nix-build '&lt;nixpkgs>' -A hello --check -K
</screen>
</para>
@@ -397,9 +397,9 @@ options control what gets deleted and in what order:
</para>
<para>The behaviour of the collector is also influenced by the <link
linkend="conf-gc-keep-outputs"><literal>gc-keep-outputs</literal></link>
linkend="conf-keep-outputs"><literal>keep-outputs</literal></link>
and <link
linkend="conf-gc-keep-derivations"><literal>gc-keep-derivations</literal></link>
linkend="conf-keep-derivations"><literal>keep-derivations</literal></link>
variables in the Nix configuration file.</para>
<para>With <option>--delete</option>, the collector prints the total
@@ -1236,12 +1236,7 @@ the store path is used.</para>
<filename>/nix/var/log/nix/drvs</filename>. However, there is no
guarantee that a build log is available for any particular store path.
For instance, if the path was downloaded as a pre-built binary through
a substitute, then the log is unavailable. If the log is not available
locally, then <command>nix-store</command> will try to download the
log from the servers specified in the Nix option
<option>log-servers</option>. For example, if it’s set to
<literal>http://hydra.nixos.org/log</literal>, then Nix will check
<literal>http://hydra.nixos.org/log/<replaceable>base-name</replaceable></literal>.</para>
a substitute, then the log is unavailable.</para>
</refsection>
+24 -9
View File
@@ -2,10 +2,18 @@
<arg><option>--help</option></arg>
<arg><option>--version</option></arg>
<arg rep='repeat'><option>--verbose</option></arg>
<arg rep='repeat'><option>-v</option></arg>
<arg><option>--no-build-output</option></arg>
<arg><option>-Q</option></arg>
<arg rep='repeat'>
<group choice='req'>
<arg choice='plain'><option>--verbose</option></arg>
<arg choice='plain'><option>-v</option></arg>
</group>
</arg>
<arg>
<group choice='plain'>
<arg choice='plain'><option>--no-build-output</option></arg>
<arg choice='plain'><option>-Q</option></arg>
</group>
</arg>
<arg>
<group choice='req'>
<arg choice='plain'><option>--max-jobs</option></arg>
@@ -25,13 +33,20 @@
<option>--timeout</option>
<replaceable>number</replaceable>
</arg>
<arg><option>--keep-going</option></arg>
<arg><option>-k</option></arg>
<arg><option>--keep-failed</option></arg>
<arg><option>-K</option></arg>
<arg>
<group choice='plain'>
<arg choice='plain'><option>--keep-going</option></arg>
<arg choice='plain'><option>-k</option></arg>
</group>
</arg>
<arg>
<group choice='plain'>
<arg choice='plain'><option>--keep-failed</option></arg>
<arg choice='plain'><option>-K</option></arg>
</group>
</arg>
<arg><option>--fallback</option></arg>
<arg><option>--readonly-mode</option></arg>
<arg><option>--show-trace</option></arg>
<arg>
<option>-I</option>
<replaceable>path</replaceable>
+13 -23
View File
@@ -22,8 +22,7 @@
</varlistentry>
<varlistentry><term><option>--verbose</option></term>
<term><option>-v</option></term>
<varlistentry><term><option>--verbose</option> / <option>-v</option></term>
<listitem>
@@ -76,8 +75,7 @@
</varlistentry>
<varlistentry><term><option>--no-build-output</option></term>
<term><option>-Q</option></term>
<varlistentry><term><option>--no-build-output</option> / <option>-Q</option></term>
<listitem><para>By default, output written by builders to standard
output and standard error is echoed to the Nix command's standard
@@ -89,13 +87,14 @@
</varlistentry>
<varlistentry xml:id="opt-max-jobs"><term><option>--max-jobs</option></term>
<term><option>-j</option></term>
<varlistentry xml:id="opt-max-jobs"><term><option>--max-jobs</option> / <option>-j</option>
<replaceable>number</replaceable></term>
<listitem><para>Sets the maximum number of build jobs that Nix will
perform in parallel to the specified number. The default is
specified by the <link
linkend='conf-build-max-jobs'><literal>build-max-jobs</literal></link>
perform in parallel to the specified number. Specify
<literal>auto</literal> to use the number of CPUs in the system.
The default is specified by the <link
linkend='conf-max-jobs'><literal>max-jobs</literal></link>
configuration setting, which itself defaults to
<literal>1</literal>. A higher value is useful on SMP systems or to
exploit I/O latency.</para></listitem>
@@ -113,7 +112,7 @@
<literal>true</literal>, the builder passes the
<option>-j<replaceable>N</replaceable></option> flag to GNU Make.
It defaults to the value of the <link
linkend='conf-build-cores'><literal>build-cores</literal></link>
linkend='conf-cores'><literal>cores</literal></link>
configuration setting, if set, or <literal>1</literal> otherwise.
The value <literal>0</literal> means that the builder should use all
available CPU cores in the system.</para></listitem>
@@ -126,7 +125,7 @@
<listitem><para>Sets the maximum number of seconds that a builder
can go without producing any data on standard output or standard
error. The default is specified by the <link
linkend='conf-build-max-silent-time'><literal>build-max-silent-time</literal></link>
linkend='conf-max-silent-time'><literal>max-silent-time</literal></link>
configuration setting. <literal>0</literal> means no
time-out.</para></listitem>
@@ -136,14 +135,13 @@
<listitem><para>Sets the maximum number of seconds that a builder
can run. The default is specified by the <link
linkend='conf-build-timeout'><literal>build-timeout</literal></link>
linkend='conf-timeout'><literal>timeout</literal></link>
configuration setting. <literal>0</literal> means no
timeout.</para></listitem>
</varlistentry>
<varlistentry><term><option>--keep-going</option></term>
<term><option>-k</option></term>
<varlistentry><term><option>--keep-going</option> / <option>-k</option></term>
<listitem><para>Keep going in case of failed builds, to the
greatest extent possible. That is, if building an input of some
@@ -155,8 +153,7 @@
</varlistentry>
<varlistentry><term><option>--keep-failed</option></term>
<term><option>-K</option></term>
<varlistentry><term><option>--keep-failed</option> / <option>-K</option></term>
<listitem><para>Specifies that in case of a build failure, the
temporary directory (usually in <filename>/tmp</filename>) in which
@@ -304,13 +301,6 @@
</varlistentry>
<varlistentry><term><option>--show-trace</option></term>
<listitem><para>Causes Nix to print out a stack trace in case of Nix
expression evaluation errors.</para></listitem>
</varlistentry>
<varlistentry xml:id="opt-I"><term><option>-I</option> <replaceable>path</replaceable></term>
@@ -139,7 +139,7 @@ impureEnvVars = [ "http_proxy" "https_proxy" <replaceable>...</replaceable> ];
<programlisting>
fetchurl {
url = http://ftp.gnu.org/pub/gnu/hello/hello-2.1.1.tar.gz;
md5 = "70c9ccf9fac07f762c24f2df2290784d";
sha256 = "1md7jsfd8pa45z73bz1kszpp01yw6x5ljkjk2hx7wl800any6465";
}
</programlisting>
@@ -150,7 +150,7 @@ fetchurl {
<programlisting>
fetchurl {
url = ftp://ftp.nluug.nl/pub/gnu/hello/hello-2.1.1.tar.gz;
md5 = "70c9ccf9fac07f762c24f2df2290784d";
sha256 = "1md7jsfd8pa45z73bz1kszpp01yw6x5ljkjk2hx7wl800any6465";
}
</programlisting>
+212 -38
View File
@@ -65,7 +65,7 @@ available as <function>builtins.derivation</function>.</para>
<replaceable>set</replaceable></term>
<listitem><para>Return the names of the attributes in the set
<replaceable>set</replaceable> in a sorted list. For instance,
<replaceable>set</replaceable> in an alphabetically sorted list. For instance,
<literal>builtins.attrNames { y = 1; x = "foo"; }</literal>
evaluates to <literal>[ "x" "y" ]</literal>.</para></listitem>
@@ -126,6 +126,17 @@ if builtins ? getEnv then builtins.getEnv "PATH" else ""</programlisting>
</varlistentry>
<varlistentry><term><function>builtins.splitVersion</function>
<replaceable>s</replaceable></term>
<listitem><para>Split a string representing a version into its
components, by the same version splitting logic underlying the
version comparison in <link linkend="ssec-version-comparisons">
<command>nix-env -u</command></link>.</para></listitem>
</varlistentry>
<varlistentry><term><function>builtins.concatLists</function>
<replaceable>lists</replaceable></term>
@@ -134,6 +145,14 @@ if builtins ? getEnv then builtins.getEnv "PATH" else ""</programlisting>
</varlistentry>
<varlistentry><term><function>builtins.concatStringsSep</function>
<replaceable>separator</replaceable> <replaceable>list</replaceable></term>
<listitem><para>Concatenate a list of strings with a separator
between each element, e.g. <literal>concatStringsSep "/"
["usr" "local" "bin"] == "usr/local/bin"</literal></para></listitem>
</varlistentry>
<varlistentry
xml:id='builtin-currentSystem'><term><varname>builtins.currentSystem</varname></term>
@@ -210,36 +229,6 @@ if builtins ? getEnv then builtins.getEnv "PATH" else ""</programlisting>
</varlistentry>
<varlistentry><term><function>builtins.match</function>
<replaceable>regex</replaceable> <replaceable>str</replaceable></term>
<listitem><para>Returns a list if
<replaceable>regex</replaceable> matches
<replaceable>str</replaceable> precisely, otherwise returns <literal>null</literal>.
Each item in the list is a regex group.
<programlisting>
builtins.match "ab" "abc"
</programlisting>
Evaluates to <literal>null</literal>.
<programlisting>
builtins.match "abc" "abc"
</programlisting>
Evaluates to <literal>[ ]</literal>.
<programlisting>
builtins.match "a(b)(c)" "abc"
</programlisting>
Evaluates to <literal>[ "b" "c" ]</literal>.
</para></listitem>
</varlistentry>
<varlistentry><term><function>builtins.elem</function>
<replaceable>x</replaceable> <replaceable>xs</replaceable></term>
@@ -289,6 +278,24 @@ Evaluates to <literal>[ "b" "c" ]</literal>.
<programlisting>
with import (fetchTarball https://github.com/NixOS/nixpkgs-channels/archive/nixos-14.12.tar.gz) {};
stdenv.mkDerivation { … }
</programlisting>
Note that when obtaining the hash with <varname>nix-prefetch-url
</varname> the option <varname>--unpack</varname> is required.
</para>
<para>This function can also verify the contents against a hash.
In that case, the function takes a set instead of a URL. The set
requires the attribute <varname>url</varname> and the attribute
<varname>sha256</varname>, e.g.
<programlisting>
with import (fetchTarball {
url = https://github.com/NixOS/nixpkgs-channels/archive/nixos-14.12.tar.gz;
sha256 = "1jppksrfvbk5ypiqdz4cddxdl8z6zyzdb2srq8fcffr327ld5jj2";
}) {};
stdenv.mkDerivation { … }
</programlisting>
@@ -312,8 +319,9 @@ stdenv.mkDerivation { … }
</varlistentry>
<varlistentry><term><function>builtins.filterSource</function>
<replaceable>e1</replaceable> <replaceable>e2</replaceable></term>
<varlistentry xml:id='builtin-filterSource'>
<term><function>builtins.filterSource</function>
<replaceable>e1</replaceable> <replaceable>e2</replaceable></term>
<listitem>
@@ -416,7 +424,7 @@ builtins.fromJSON ''{"x": [1, 2, 3], "y": null}''
<listitem><para>Generate list of size
<replaceable>length</replaceable>, with each element
<replaceable>i></replaceable> equal to the value returned by
<replaceable>i</replaceable> equal to the value returned by
<replaceable>generator</replaceable> <literal>i</literal>. For
example,
@@ -612,6 +620,16 @@ x: x + 456</programlisting>
</varlistentry>
<varlistentry><term><function>builtins.isFloat</function>
<replaceable>e</replaceable></term>
<listitem><para>Return <literal>true</literal> if
<replaceable>e</replaceable> evaluates to a float, and
<literal>false</literal> otherwise.</para></listitem>
</varlistentry>
<varlistentry><term><function>builtins.isBool</function>
<replaceable>e</replaceable></term>
@@ -702,6 +720,42 @@ map (x: "foo" + x) [ "bar" "bla" "abc" ]</programlisting>
</varlistentry>
<varlistentry><term><function>builtins.match</function>
<replaceable>regex</replaceable> <replaceable>str</replaceable></term>
<listitem><para>Returns a list if the <link
xlink:href="http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap09.html#tag_09_04">extended
POSIX regular expression</link> <replaceable>regex</replaceable>
matches <replaceable>str</replaceable> precisely, otherwise returns
<literal>null</literal>. Each item in the list is a regex group.
<programlisting>
builtins.match "ab" "abc"
</programlisting>
Evaluates to <literal>null</literal>.
<programlisting>
builtins.match "abc" "abc"
</programlisting>
Evaluates to <literal>[ ]</literal>.
<programlisting>
builtins.match "a(b)(c)" "abc"
</programlisting>
Evaluates to <literal>[ "b" "c" ]</literal>.
<programlisting>
builtins.match "[[:space:]]+([[:upper:]]+)[[:space:]]+" " FOO "
</programlisting>
Evaluates to <literal>[ "foo" ]</literal>.
</para></listitem>
</varlistentry>
<varlistentry><term><function>builtins.mul</function>
<replaceable>e1</replaceable> <replaceable>e2</replaceable></term>
@@ -726,6 +780,75 @@ map (x: "foo" + x) [ "bar" "bla" "abc" ]</programlisting>
</varlistentry>
<varlistentry>
<term>
<function>builtins.path</function>
<replaceable>args</replaceable>
</term>
<listitem>
<para>
An enrichment of the built-in path type, based on the attributes
present in <replaceable>args</replaceable>. All are optional
except <varname>path</varname>:
</para>
<variablelist>
<varlistentry>
<term>path</term>
<listitem>
<para>The underlying path.</para>
</listitem>
</varlistentry>
<varlistentry>
<term>name</term>
<listitem>
<para>
The name of the path when added to the store. This can
used to reference paths that have nix-illegal characters
in their names, like <literal>@</literal>.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>filter</term>
<listitem>
<para>
A function of the type expected by
<link linkend="builtin-filterSource">builtins.filterSource</link>,
with the same semantics.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>recursive</term>
<listitem>
<para>
When <literal>false</literal>, when
<varname>path</varname> is added to the store it is with a
flat hash, rather than a hash of the NAR serialization of
the file. Thus, <varname>path</varname> must refer to a
regular file, not a directory. This allows similar
behavior to <literal>fetchurl</literal>. Defaults to
<literal>true</literal>.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>sha256</term>
<listitem>
<para>
When provided, this is the expected hash of the file at
the path. Evaluation will fail if the hash is incorrect,
and providing a hash allows
<literal>builtins.path</literal> to be used even when the
<literal>pure-eval</literal> nix config option is on.
</para>
</listitem>
</varlistentry>
</variablelist>
</listitem>
</varlistentry>
<varlistentry><term><function>builtins.pathExists</function>
<replaceable>path</replaceable></term>
@@ -849,6 +972,43 @@ builtins.sort builtins.lessThan [ 483 249 526 147 42 77 ]
</varlistentry>
<varlistentry><term><function>builtins.split</function>
<replaceable>regex</replaceable> <replaceable>str</replaceable></term>
<listitem><para>Returns a list composed of non matched strings interleaved
with the lists of the <link
xlink:href="http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap09.html#tag_09_04">extended
POSIX regular expression</link> <replaceable>regex</replaceable> matches
of <replaceable>str</replaceable>. Each item in the lists of matched
sequences is a regex group.
<programlisting>
builtins.split "(a)b" "abc"
</programlisting>
Evaluates to <literal>[ "" [ "a" ] "c" ]</literal>.
<programlisting>
builtins.split "([ac])" "abc"
</programlisting>
Evaluates to <literal>[ "" [ "a" ] "b" [ "c" ] "" ]</literal>.
<programlisting>
builtins.split "(a)|(c)" "abc"
</programlisting>
Evaluates to <literal>[ "" [ "a" null ] "b" [ null "c" ] "" ]</literal>.
<programlisting>
builtins.split "([[:upper:]]+)" " FOO "
</programlisting>
Evaluates to <literal>[ " " [ "FOO" ] " " ]</literal>.
</para></listitem>
</varlistentry>
<varlistentry><term><function>builtins.stringLength</function>
<replaceable>e</replaceable></term>
@@ -949,8 +1109,8 @@ stdenv.mkDerivation {
";
src = fetchurl {
url = http://nix.cs.uu.nl/dist/tarballs/hello-2.1.1.tar.gz;
md5 = "70c9ccf9fac07f762c24f2df2290784d";
url = http://ftp.nluug.nl/pub/gnu/hello/hello-2.1.1.tar.gz;
sha256 = "1md7jsfd8pa45z73bz1kszpp01yw6x5ljkjk2hx7wl800any6465";
};
inherit perl;
}</programlisting>
@@ -1163,6 +1323,19 @@ stdenv.mkDerivation (rec {
</varlistentry>
<varlistentry><term><function>builtins.tryEval</function>
<replaceable>e</replaceable></term>
<listitem><para>Try to evaluate <replaceable>e</replaceable>.
Return a set containing the attributes <literal>success</literal>
(<literal>true</literal> if <replaceable>e</replaceable> evaluated
successfully, <literal>false</literal> if an error was thrown) and
<literal>value</literal>, equalling <replaceable>e</replaceable>
if successful and <literal>false</literal> otherwise.
</para></listitem>
</varlistentry>
<varlistentry><term><function>builtins.typeOf</function>
<replaceable>e</replaceable></term>
@@ -1171,8 +1344,9 @@ stdenv.mkDerivation (rec {
<replaceable>e</replaceable>, namely <literal>"int"</literal>,
<literal>"bool"</literal>, <literal>"string"</literal>,
<literal>"path"</literal>, <literal>"null"</literal>,
<literal>"set"</literal>, <literal>"list"</literal> or
<literal>"lambda"</literal>.</para></listitem>
<literal>"set"</literal>, <literal>"list"</literal>,
<literal>"lambda"</literal> or
<literal>"float"</literal>.</para></listitem>
</varlistentry>
-34
View File
@@ -1,34 +0,0 @@
<section xmlns="http://docbook.org/ns/docbook"
xmlns:xlink="http://www.w3.org/1999/xlink"
xmlns:xi="http://www.w3.org/2001/XInclude"
version="5.0"
xml:id="sec-debug-build">
<title>Debugging Build Failures</title>
<para>At the beginning of each phase of the build (such as unpacking,
building or installing), the set of all shell variables is written to
the file <filename>env-vars</filename> at the top-level build
directory. This is useful for debugging: it allows you to recreate
the environment in which a build was performed. For instance, if a
build fails, then assuming you used the <option>-K</option> flag, you
can go to the output directory and <quote>switch</quote> to the
environment of the builder:
<screen>
$ nix-build -K ./foo.nix
... fails, keeping build directory `/tmp/nix-1234-0'
$ cd /tmp/nix-1234-0
$ source env-vars
<lineannotation>(edit some files...)</lineannotation>
$ make
<lineannotation>(execution continues with the same GCC, make, etc.)</lineannotation></screen>
</para>
</section>
+1 -1
View File
@@ -100,7 +100,7 @@ outputs = [ "lib" "headers" "doc" ];
<programlisting>
buildInputs = [ pkg.lib pkg.headers ];
</programlisting>
The first element of <varname>output</varname> determines the
The first element of <varname>outputs</varname> determines the
<emphasis>default output</emphasis>. Thus, you could also write
<programlisting>
buildInputs = [ pkg pkg.headers ];
+3 -3
View File
@@ -16,7 +16,7 @@ stdenv.mkDerivation { <co xml:id='ex-hello-nix-co-2' />
builder = ./builder.sh; <co xml:id='ex-hello-nix-co-4' />
src = fetchurl { <co xml:id='ex-hello-nix-co-5' />
url = ftp://ftp.nluug.nl/pub/gnu/hello/hello-2.1.1.tar.gz;
md5 = "70c9ccf9fac07f762c24f2df2290784d";
sha256 = "1md7jsfd8pa45z73bz1kszpp01yw6x5ljkjk2hx7wl800any6465";
};
inherit perl; <co xml:id='ex-hello-nix-co-6' />
}</programlisting>
@@ -108,7 +108,7 @@ the single Nix expression in that directory
<para>The builder has to know what the sources of the package
are. Here, the attribute <varname>src</varname> is bound to the
result of a call to the <command>fetchurl</command> function.
Given a URL and an MD5 hash of the expected contents of the file
Given a URL and a SHA-256 hash of the expected contents of the file
at that URL, this function builds a derivation that downloads the
file and checks its hash. So the sources are a dependency that
like all other dependencies is built before Hello itself is
@@ -145,4 +145,4 @@ perl = perl;</programlisting>
</para>
</section>
</section>
+44 -3
View File
@@ -61,7 +61,7 @@ evaluates to <literal>"foobar"</literal>.
<simplesect><title>Inheriting attributes</title>
<para>When defining a set it is often convenient to copy variables
<para>When defining a set or in a let-expression it is often convenient to copy variables
from the surrounding lexical scope (e.g., when you want to propagate
attributes). This can be shortened using the
<literal>inherit</literal> keyword. For instance,
@@ -72,7 +72,15 @@ let x = 123; in
y = 456;
}</programlisting>
evaluates to <literal>{ x = 123; y = 456; }</literal>. (Note that
is equivalent to
<programlisting>
let x = 123; in
{ x = x;
y = 456;
}</programlisting>
and both evaluate to <literal>{ x = 123; y = 456; }</literal>. (Note that
this works because <varname>x</varname> is added to the lexical scope
by the <literal>let</literal> construct.) It is also possible to
inherit attributes from another set. For instance, in this fragment
@@ -101,6 +109,26 @@ variables from the surrounding scope (<varname>fetchurl</varname>
<varname>libXaw</varname> (the X Athena Widgets) from the
<varname>xlibs</varname> (X11 client-side libraries) set.</para>
<para>
Summarizing the fragment
<programlisting>
...
inherit x y z;
inherit (src-set) a b c;
...</programlisting>
is equivalent to
<programlisting>
...
x = x; y = y; z = z;
a = src-set.a; b = src-set.b; c = src-set.c;
...</programlisting>
when used while defining local variables in a let-expression or
while defining a set.</para>
</simplesect>
@@ -333,7 +361,20 @@ with (import ./definitions.nix); ...</programlisting>
makes all attributes defined in the file
<filename>definitions.nix</filename> available as if they were defined
locally in a <literal>rec</literal>-expression.</para>
locally in a <literal>let</literal>-expression.</para>
<para>The bindings introduced by <literal>with</literal> do not shadow bindings
introduced by other means, e.g.
<programlisting>
let a = 3; in with { a = 1; }; let a = 4; in with { a = 2; }; ...</programlisting>
establishes the same scope as
<programlisting>
let a = 1; in let a = 2; in let a = 3; in let a = 4; in ...</programlisting>
</para>
</simplesect>
+31 -9
View File
@@ -40,6 +40,11 @@ weakest binding).</para>
<entry>Call function <replaceable>e1</replaceable> with
argument <replaceable>e2</replaceable>.</entry>
</row>
<row>
<entry><literal>-</literal> <replaceable>e</replaceable></entry>
<entry>none</entry>
<entry>Arithmetic negation.</entry>
</row>
<row>
<entry><replaceable>e</replaceable> <literal>?</literal>
<replaceable>attrpath</replaceable></entry>
@@ -55,13 +60,24 @@ weakest binding).</para>
<entry>List concatenation.</entry>
</row>
<row>
<entry><replaceable>e1</replaceable> <literal>+</literal> <replaceable>e2</replaceable></entry>
<entry>
<replaceable>e1</replaceable> <literal>*</literal> <replaceable>e2</replaceable>,
<replaceable>e1</replaceable> <literal>/</literal> <replaceable>e2</replaceable>
</entry>
<entry>left</entry>
<entry>String or path concatenation.</entry>
<entry>Arithmetic multiplication and division.</entry>
</row>
<row>
<entry>
<replaceable>e1</replaceable> <literal>+</literal> <replaceable>e2</replaceable>,
<replaceable>e1</replaceable> <literal>-</literal> <replaceable>e2</replaceable>
</entry>
<entry>left</entry>
<entry>Arithmetic addition and subtraction. String or path concatenation (only by <literal>+</literal>).</entry>
</row>
<row>
<entry><literal>!</literal> <replaceable>e</replaceable></entry>
<entry>left</entry>
<entry>none</entry>
<entry>Boolean negation.</entry>
</row>
<row>
@@ -75,16 +91,22 @@ weakest binding).</para>
attributes).</entry>
</row>
<row>
<entry><replaceable>e1</replaceable> <literal>==</literal>
<replaceable>e2</replaceable></entry>
<entry>
<replaceable>e1</replaceable> <literal>&lt;</literal> <replaceable>e2</replaceable>,
<replaceable>e1</replaceable> <literal>&gt;</literal> <replaceable>e2</replaceable>,
<replaceable>e1</replaceable> <literal>&lt;=</literal> <replaceable>e2</replaceable>,
<replaceable>e1</replaceable> <literal>&gt;=</literal> <replaceable>e2</replaceable>
</entry>
<entry>none</entry>
<entry>Equality.</entry>
<entry>Arithmetic comparison.</entry>
</row>
<row>
<entry><replaceable>e1</replaceable> <literal>!=</literal>
<replaceable>e2</replaceable></entry>
<entry>
<replaceable>e1</replaceable> <literal>==</literal> <replaceable>e2</replaceable>,
<replaceable>e1</replaceable> <literal>!=</literal> <replaceable>e2</replaceable>
</entry>
<entry>none</entry>
<entry>Inequality.</entry>
<entry>Equality and inequality.</entry>
</row>
<row>
<entry><replaceable>e1</replaceable> <literal>&amp;&amp;</literal>
+9 -5
View File
@@ -98,13 +98,17 @@ configureFlags = "
<para>Since <literal>${</literal> and <literal>''</literal> have
special meaning in indented strings, you need a way to quote them.
<literal>${</literal> can be escaped by prefixing it with
<literal>$</literal> can be escaped by prefixing it with
<literal>''</literal> (that is, two single quotes), i.e.,
<literal>''${</literal>. <literal>''</literal> can be escaped by
<literal>''$</literal>. <literal>''</literal> can be escaped by
prefixing it with <literal>'</literal>, i.e.,
<literal>'''</literal>. Finally, linefeed, carriage-return and
tab characters can be written as <literal>''\n</literal>,
<literal>''\r</literal>, <literal>''\t</literal>.</para>
<literal>'''</literal>. <literal>$</literal> removes any special meaning
from the following <literal>$</literal>. Linefeed, carriage-return and tab
characters can be written as <literal>''\n</literal>,
<literal>''\r</literal>, <literal>''\t</literal>, and <literal>''\</literal>
escapes any other character.
</para>
<para>Indented strings are primarily useful in that they allow
multi-line string literals to follow the indentation of the
@@ -81,6 +81,4 @@ Just pass the option <link linkend='opt-max-jobs'><option>-j
in parallel, or set. Typically this should be the number of
CPUs.</para>
<xi:include href="debug-build.xml" />
</section>
+1 -1
View File
@@ -54,7 +54,7 @@
<glossdef><para>A substitute is a command invocation stored in the
Nix database that describes how to build a store object, bypassing
normal the build mechanism (i.e., derivations). Typically, the
the normal build mechanism (i.e., derivations). Typically, the
substitute builds the store object by downloading a pre-built
version of the store object from some server.</para></glossdef>
+2 -12
View File
@@ -6,7 +6,7 @@
<title>Installing a Binary Distribution</title>
<para>If you are using Linux or Mac OS X, the easiest way to install
<para>If you are using Linux or macOS, the easiest way to install
Nix is to run the following command:
<screen>
@@ -39,7 +39,7 @@ behaviour.
<!--
<para>You can also manually download and install a binary package.
Binary packages of the latest stable release are available for Fedora,
Debian, Ubuntu, Mac OS X and various other systems from the <link
Debian, Ubuntu, macOS and various other systems from the <link
xlink:href="http://nixos.org/nix/download.html">Nix homepage</link>.
You can also get builds of the latest development release from our
<link
@@ -79,16 +79,6 @@ alice$ ./install
</para>
<para>Nix can be uninstalled using <command>rpm -e nix</command> or
<command>dpkg -r nix</command> on RPM- and Dpkg-based systems,
respectively. After this you should manually remove the Nix store and
other auxiliary data, if desired:
<screen>
$ rm -rf /nix</screen>
</para>
<para>You can uninstall Nix simply by running:
<screen>
-28
View File
@@ -52,34 +52,6 @@ This creates 10 build users. There can never be more concurrent builds
than the number of build users, so you may want to increase this if
you expect to do many builds at the same time.</para>
<para>On Mac OS X, you can create the required group and users by
running the following script:
<programlisting>
#! /bin/bash -e
dseditgroup -o create nixbld -q
gid=$(dscl . -read /Groups/nixbld | awk '($1 == "PrimaryGroupID:") {print $2 }')
echo "created nixbld group with gid $gid"
for i in $(seq 1 10); do
user=/Users/nixbld$i
uid="$((30000 + $i))"
dscl . create $user
dscl . create $user RealName "Nix build user $i"
dscl . create $user PrimaryGroupID "$gid"
dscl . create $user UserShell /usr/bin/false
dscl . create $user NFSHomeDirectory /var/empty
dscl . create $user UniqueID "$uid"
dseditgroup -o edit -a nixbld$i -t user nixbld
echo "created nixbld$i user with uid $uid"
done
</programlisting>
</para>
</simplesect>
@@ -10,16 +10,14 @@
<listitem><para>GNU Make.</para></listitem>
<listitem><para>A version of GCC or Clang that supports C++11.</para></listitem>
<listitem><para>Perl 5.8 or higher.</para></listitem>
<listitem><para>A version of GCC or Clang that supports C++14.</para></listitem>
<listitem><para><command>pkg-config</command> to locate
dependencies. If your distribution does not provide it, you can get
it from <link
xlink:href="http://www.freedesktop.org/wiki/Software/pkg-config"
/>.</para></listitem>
<listitem><para>The OpenSSL library to calculate cryptographic hashes.
If your distribution does not provide it, you can get it from <link
xlink:href="https://www.openssl.org"/>.</para></listitem>
@@ -34,11 +32,6 @@
or higher. If your distribution does not provide it, please install
it from <link xlink:href="http://www.sqlite.org/" />.</para></listitem>
<listitem><para>The Perl DBI and DBD::SQLite libraries, which are
available from <link
xlink:href="http://search.cpan.org/">CPAN</link> if your
distribution does not provide them.</para></listitem>
<listitem><para>The <link
xlink:href="http://www.hboehm.info/gc/">Boehm
garbage collector</link> to reduce the evaluator’s memory
@@ -12,7 +12,7 @@
<listitem><para>Linux (i686, x86_64).</para></listitem>
<listitem><para>Mac OS X (x86_64).</para></listitem>
<listitem><para>macOS (x86_64).</para></listitem>
<!--
<listitem><para>FreeBSD (only tested on Intel).</para></listitem>
@@ -33,7 +33,4 @@
</para>
<para>Nix is fairly portable, so it should work on most platforms that
support POSIX threads and have a C++11 compiler.</para>
</chapter>
+8 -2
View File
@@ -93,7 +93,7 @@ time window in which the package has some files from the old version
and some files from the new version — which would be bad because a
program might well crash if it’s started during that period.</para>
<para>And since package aren’t overwritten, the old versions are still
<para>And since packages aren’t overwritten, the old versions are still
there after an upgrade. This means that you can <emphasis>roll
back</emphasis> to the old version:</para>
@@ -236,7 +236,7 @@ href="[%root%]hydra">a continuous build system</a>.</para>
<simplesect><title>Portability</title>
<para>Nix runs on Linux and Mac OS X.</para>
<para>Nix runs on Linux and macOS.</para>
</simplesect>
@@ -261,6 +261,12 @@ xlink:href="http://nixos.org/">NixOS homepage</link>.</para>
xlink:href="http://www.gnu.org/licenses/old-licenses/lgpl-2.1.html">GNU
LGPLv2.1 or (at your option) any later version</link>.</para>
<para>Nix uses the <link
xlink:href="https://github.com/arangodb/linenoise-ng">linenoise-ng
library</link>, which has the following license:</para>
<programlisting><xi:include href="../../../src/linenoise/LICENSE" parse="text" /></programlisting>
</simplesect>
+2 -2
View File
@@ -15,7 +15,7 @@ to subsequent chapters.</para>
<step><para>Install single-user Nix by running the following:
<screen>
$ curl https://nixos.org/nix/install | sh
$ bash &lt;(curl https://nixos.org/nix/install)
</screen>
This will install Nix in <filename>/nix</filename>. The install script
@@ -40,7 +40,7 @@ libxslt-1.1.28
<step><para>Install some packages from the channel:
<screen>
$ nix-env -i hello <replaceable>...</replaceable> </screen>
$ nix-env -i hello</screen>
This should download pre-built packages; it should not build them
locally (if it does, something went wrong).</para></step>
+1 -7
View File
@@ -12,19 +12,14 @@
<firstname>Eelco</firstname>
<surname>Dolstra</surname>
</personname>
<affiliation>
<orgname>LogicBlox</orgname>
</affiliation>
<contrib>Author</contrib>
</author>
<copyright>
<year>2004-2014</year>
<year>2004-2018</year>
<holder>Eelco Dolstra</holder>
</copyright>
<date>November 2014</date>
</info>
<!--
@@ -41,7 +36,6 @@
<xi:include href="expressions/writing-nix-expressions.xml" />
<xi:include href="advanced-topics/advanced-topics.xml" />
<xi:include href="command-ref/command-ref.xml" />
<xi:include href="troubleshooting/troubleshooting.xml" />
<xi:include href="glossary/glossary.xml" />
<xi:include href="hacking.xml" />
<xi:include href="release-notes/release-notes.xml" />
+2 -1
View File
@@ -12,7 +12,8 @@
</partintro>
-->
<xi:include href="rl-1.12.xml" />
<xi:include href="rl-2.0.xml" />
<xi:include href="rl-1.11.10.xml" />
<xi:include href="rl-1.11.xml" />
<xi:include href="rl-1.10.xml" />
<xi:include href="rl-1.9.xml" />
+31
View File
@@ -0,0 +1,31 @@
<section xmlns="http://docbook.org/ns/docbook"
xmlns:xlink="http://www.w3.org/1999/xlink"
xmlns:xi="http://www.w3.org/2001/XInclude"
version="5.0"
xml:id="ssec-relnotes-1.11.10">
<title>Release 1.11.10 (2017-06-12)</title>
<para>This release fixes a security bug in Nix’s “build user” build
isolation mechanism. Previously, Nix builders had the ability to
create setuid binaries owned by a <literal>nixbld</literal>
user. Such a binary could then be used by an attacker to assume a
<literal>nixbld</literal> identity and interfere with subsequent
builds running under the same UID.</para>
<para>To prevent this issue, Nix now disallows builders to create
setuid and setgid binaries. On Linux, this is done using a seccomp BPF
filter. Note that this imposes a small performance penalty (e.g. 1%
when building GNU Hello). Using seccomp, we now also prevent the
creation of extended attributes and POSIX ACLs since these cannot be
represented in the NAR format and (in the case of POSIX ACLs) allow
bypassing regular Nix store permissions. On macOS, the restriction is
implemented using the existing sandbox mechanism, which now uses a
minimal “allow all except the creation of setuid/setgid binaries”
profile when regular sandboxing is disabled. On other platforms, the
“build user” mechanism is now disabled.</para>
<para>Thanks go to Linus Heckemann for discovering and reporting this
bug.</para>
</section>
-7
View File
@@ -121,13 +121,6 @@ $ diffoscope /nix/store/11a27shh6n2i…-zlib-1.2.8 /nix/store/11a27shh6n2i…-zl
also improves performance.</para>
</listitem>
<listitem>
<para>The Nix language now supports floating point numbers. They are
based on regular C++ <literal>float</literal> and compatible with
existing integers and number-related operations. Export and import to and
from JSON and XML works, too.
</para>
</listitem>
<listitem>
<para>All "chroot"-containing strings got renamed to "sandbox".
In particular, some Nix options got renamed, but the old names
-24
View File
@@ -1,24 +0,0 @@
<section xmlns="http://docbook.org/ns/docbook"
xmlns:xlink="http://www.w3.org/1999/xlink"
xmlns:xi="http://www.w3.org/2001/XInclude"
version="5.0"
xml:id="ssec-relnotes-1.12">
<title>Release 1.12 (TBA)</title>
<para>This release has the following new features:</para>
<itemizedlist>
<listitem>
<para>It is no longer necessary to set the
<envar>NIX_REMOTE</envar> environment variable if you need to use
the Nix daemon. Nix will use the daemon automatically if you don’t
have write access to the Nix database.</para>
</listitem>
</itemizedlist>
<para>This release has contributions from TBD.</para>
</section>
+1 -1
View File
@@ -40,7 +40,7 @@ $ nix-env -i thunderbird --option binary-caches http://cache.nixos.org
<para>Binary caches are created using <command>nix-push</command>.
For details on the operation and format of binary caches, see the
<command>nix-push</command> manpage. More details are provided in
<link xlink:href="http://lists.science.uu.nl/pipermail/nix-dev/2012-September/009826.html">this
<link xlink:href="https://nixos.org/nix-dev/2012-September/009826.html">this
nix-dev posting</link>.</para>
</listitem>
+2 -2
View File
@@ -83,8 +83,8 @@ $ nix-store -l $(which xterm)
caches).</para></listitem>
<listitem><para>The configuration option
<option>build-max-jobs</option> now defaults to the number of
available CPU cores.</para></listitem>
<option>build-cores</option> now defaults to the number of available
CPU cores.</para></listitem>
<listitem><para>Build users are now used by default when Nix is
invoked as root. This prevents builds from accidentally running as
File diff suppressed because it is too large. Load diff
+2 -10
View File
@@ -96,7 +96,6 @@ div.example
margin-right: 1.5em;
background: #f4f4f8;
border-radius: 0.4em;
box-shadow: 0.4em 0.4em 0.5em #e0e0e0;
}
div.example p.title
@@ -106,7 +105,6 @@ div.example p.title
div.example pre
{
box-shadow: none;
}
@@ -116,15 +114,12 @@ div.example pre
pre.screen, pre.programlisting
{
border: 1px solid #b0b0b0;
padding: 3px 3px;
padding: 6px 6px;
margin-left: 1.5em;
margin-right: 1.5em;
color: #600000;
background: #f4f4f8;
font-family: monospace;
border-radius: 0.4em;
box-shadow: 0.4em 0.4em 0.5em #e0e0e0;
}
div.example pre.programlisting
@@ -149,7 +144,6 @@ div.example pre.programlisting
padding: 0.3em 0.3em 0.3em 0.3em;
background: #fffff5;
border-radius: 0.4em;
box-shadow: 0.4em 0.4em 0.5em #e0e0e0;
}
div.note, div.warning
@@ -256,16 +250,14 @@ span.command strong
div.calloutlist table
{
box-shadow: none;
}
table
{
border-collapse: collapse;
box-shadow: 0.4em 0.4em 0.5em #e0e0e0;
}
div.affiliation
{
font-style: italic;
}
}
@@ -1,38 +0,0 @@
<section xmlns="http://docbook.org/ns/docbook"
xmlns:xlink="http://www.w3.org/1999/xlink"
xmlns:xi="http://www.w3.org/2001/XInclude"
version="5.0"
xml:id="sec-collisions-nixenv">
<title>Collisions in <command>nix-env</command></title>
<para>Symptom: when installing or upgrading, you get an error message such as
<screen>
$ nix-env -i docbook-xml
...
adding /nix/store/s5hyxgm62gk2...-docbook-xml-4.2
collision between `/nix/store/s5hyxgm62gk2...-docbook-xml-4.2/xml/dtd/docbook/calstblx.dtd'
and `/nix/store/06h377hr4b33...-docbook-xml-4.3/xml/dtd/docbook/calstblx.dtd'
at /nix/store/...-builder.pl line 62.</screen>
</para>
<para>The cause is that two installed packages in the user environment
have overlapping filenames (e.g.,
<filename>xml/dtd/docbook/calstblx.dtd</filename>. This usually
happens when you accidentally try to install two versions of the same
package. For instance, in the example above, the Nix Packages
collection contains two versions of <literal>docbook-xml</literal>, so
<command>nix-env -i</command> will try to install both. The default
user environment builder has no way to way to resolve such conflicts,
so it just gives up.</para>
<para>Solution: remove one of the offending packages from the user
environment (if already installed) using <command>nix-env
-e</command>, or specify exactly which version should be installed
(e.g., <literal>nix-env -i docbook-xml-4.2</literal>).</para>
<!-- FIXME: describe priorities -->
</section>
@@ -1,43 +0,0 @@
<section xmlns="http://docbook.org/ns/docbook"
xmlns:xlink="http://www.w3.org/1999/xlink"
xmlns:xi="http://www.w3.org/2001/XInclude"
version="5.0"
xml:id="sec-links-nix-store">
<title><quote>Too many links</quote> Error in the Nix store</title>
<para>Symptom: when building something, you get an error message such as
<screen>
...
<literal>mkdir: cannot create directory `/nix/store/<replaceable>name</replaceable>': Too many links</literal></screen>
</para>
<para>This is usually because you have more than 32,000 subdirectories
in <filename>/nix/store</filename>, as can be seen using <command>ls
-l</command>:
<screen>
$ ls -ld /nix/store
drwxrwxrwt 32000 nix nix 4620288 Sep 8 15:08 store</screen>
The <literal>ext2</literal> file system is limited to an inode link
count of 32,000 (each subdirectory increasing the count by one).
Furthermore, the <literal>st_nlink</literal> field of the
<function>stat</function> system call is a 16-bit value.</para>
<para>This only happens on very large Nix installations (such as build
machines).</para>
<para>Quick solution: run the garbage collector. You may want to use
the <option>--max-links</option> option.</para>
<para>Real solution: put the Nix store on a file system that supports
more than 32,000 subdirectories per directory, such as ext4. (This
doesn’t solve the <literal>st_nlink</literal> limit, but ext4 lies to
the kernel by reporting a link count of 1 if it exceeds the
limit.)</para>
</section>
@@ -1,16 +0,0 @@
<appendix xmlns="http://docbook.org/ns/docbook"
xmlns:xlink="http://www.w3.org/1999/xlink"
xmlns:xi="http://www.w3.org/2001/XInclude"
version="5.0"
xml:id="ch-troubleshooting">
<title>Troubleshooting</title>
<para>This section provides solutions for some common problems. See
the <link xlink:href="https://github.com/NixOS/nix/issues">Nix bug
tracker</link> for a list of currently known issues.</para>
<xi:include href="collisions-nixenv.xml" />
<xi:include href="links-nix-store.xml" />
</appendix>
+3 -5
View File
@@ -1,14 +1,12 @@
ifeq ($(MAKECMDGOALS), dist)
# Make sure we are in repo root with `--git-dir`
dist-files += $(shell git --git-dir=.git ls-files || find * -type f)
dist-files += $(shell cat .dist-files)
endif
dist-files += configure config.h.in nix.spec
dist-files += configure config.h.in nix.spec perl/configure
clean-files += Makefile.config
GLOBAL_CXXFLAGS += -I . -I src -I src/libutil -I src/libstore -I src/libmain -I src/libexpr \
-Wno-unneeded-internal-declaration
GLOBAL_CXXFLAGS += -I . -I src -I src/libutil -I src/libstore -I src/libmain -I src/libexpr
$(foreach i, config.h $(call rwildcard, src/lib*, *.hh), \
$(eval $(call install-file-in, $(i), $(includedir)/nix, 0644)))
+12 -4
View File
@@ -6,6 +6,7 @@ use Data::Dumper;
use File::Basename;
use File::Path;
use File::Slurp;
use File::Copy;
use JSON::PP;
use LWP::UserAgent;
@@ -54,7 +55,7 @@ sub downloadFile {
my $buildInfo = decode_json(fetch("$evalUrl/job/$jobName", 'application/json'));
my $srcFile = $buildInfo->{buildproducts}->{$productNr}->{path} or die;
my $srcFile = $buildInfo->{buildproducts}->{$productNr}->{path} or die "job '$jobName' lacks product $productNr\n";
$dstName //= basename($srcFile);
my $dstFile = "$releaseDir/" . $dstName;
@@ -78,11 +79,11 @@ sub downloadFile {
return ($dstFile, $sha256_expected);
}
downloadFile("tarball", "2"); # PDF
downloadFile("tarball", "3"); # .tar.bz2
my ($tarball, $tarballHash) = downloadFile("tarball", "4"); # .tar.xz
downloadFile("tarball", "2"); # .tar.bz2
my ($tarball, $tarballHash) = downloadFile("tarball", "3"); # .tar.xz
my ($tarball_i686_linux, $tarball_i686_linux_hash) = downloadFile("binaryTarball.i686-linux", "1");
my ($tarball_x86_64_linux, $tarball_x86_64_linux_hash) = downloadFile("binaryTarball.x86_64-linux", "1");
my ($tarball_aarch64_linux, $tarball_aarch64_linux_hash) = downloadFile("binaryTarball.aarch64-linux", "1");
my ($tarball_x86_64_darwin, $tarball_x86_64_darwin_hash) = downloadFile("binaryTarball.x86_64-darwin", "1");
# Update Nixpkgs in a very hacky way.
@@ -111,6 +112,7 @@ write_file("$nixpkgsDir/nixos/modules/installer/tools/nix-fallback-paths.nix",
"{\n" .
" x86_64-linux = \"" . getStorePath("build.x86_64-linux") . "\";\n" .
" i686-linux = \"" . getStorePath("build.i686-linux") . "\";\n" .
" aarch64-linux = \"" . getStorePath("build.aarch64-linux") . "\";\n" .
" x86_64-darwin = \"" . getStorePath("build.x86_64-darwin") . "\";\n" .
"}\n");
@@ -144,9 +146,15 @@ write_file("$siteDir/nix-release.tt",
"latestNixVersion = \"$version\"\n" .
"nix_hash_i686_linux = \"$tarball_i686_linux_hash\"\n" .
"nix_hash_x86_64_linux = \"$tarball_x86_64_linux_hash\"\n" .
"nix_hash_aarch64_linux = \"$tarball_aarch64_linux_hash\"\n" .
"nix_hash_x86_64_darwin = \"$tarball_x86_64_darwin_hash\"\n" .
"-%]\n");
system("cd $siteDir && nix-shell --run 'make nix/install nix/install.sig'") == 0 or die;
copy("$siteDir/nix/install", "$siteDir/nix/install-$version") or die;
copy("$siteDir/nix/install.sig", "$siteDir/nix/install-$version.sig") or die;
system("cd $siteDir && git add nix/install-$version nix/install-$version.sig") == 0 or die;
system("cd $siteDir && git commit -a -m 'Nix $version released'") == 0 or die;
+18 -15
View File
@@ -1,23 +1,26 @@
FROM alpine
RUN wget -O- http://nixos.org/releases/nix/nix-1.11.2/nix-1.11.2-x86_64-linux.tar.bz2 | bzcat - | tar xf - \
&& echo "nixbld:x:30000:nixbld1,nixbld2,nixbld3,nixbld4,nixbld5,nixbld6,nixbld7,nixbld8,nixbld9,nixbld10,nixbld11,nixbld12,nixbld13,nixbld14,nixbld15,nixbld16,nixbld17,nixbld18,nixbld19,nixbld20,nixbld21,nixbld22,nixbld23,nixbld24,nixbld25,nixbld26,nixbld27,nixbld28,nixbld29,nixbld30" >> /etc/group \
&& for i in $(seq 1 30); do echo "nixbld$i:x:$((30000 + $i)):30000:::" >> /etc/passwd; done \
&& mkdir -m 0755 /nix && USER=root sh nix-*-x86_64-linux/install \
&& echo ". /root/.nix-profile/etc/profile.d/nix.sh" >> /etc/profile \
&& rm -r /nix-*-x86_64-linux \
&& apk --update add bash tar \
&& rm -rf /var/cache/apk/*
# Enable HTTPS support in wget.
RUN apk add --update openssl
# Download Nix and install it into the system.
RUN wget -O- https://nixos.org/releases/nix/nix-1.11.14/nix-1.11.14-x86_64-linux.tar.bz2 | bzcat - | tar xf - \
&& addgroup -g 30000 -S nixbld \
&& for i in $(seq 1 30); do adduser -S -D -h /var/empty -g "Nix build user $i" -u $((30000 + i)) -G nixbld nixbld$i ; done \
&& mkdir -m 0755 /nix && USER=root sh nix-*-x86_64-linux/install \
&& ln -s /nix/var/nix/profiles/default/etc/profile.d/nix.sh /etc/profile.d/ \
&& rm -r /nix-*-x86_64-linux \
&& rm -r /var/cache/apk/*
ONBUILD ENV \
ENV=/etc/profile \
PATH=/root/.nix-profile/bin:/root/.nix-profile/sbin:/bin:/sbin:/usr/bin:/usr/sbin \
GIT_SSL_CAINFO=/root/.nix-profile/etc/ssl/certs/ca-bundle.crt \
NIX_SSL_CERT_FILE=/root/.nix-profile/etc/ssl/certs/ca-bundle.crt
PATH=/nix/var/nix/profiles/default/bin:/nix/var/nix/profiles/default/sbin:/bin:/sbin:/usr/bin:/usr/sbin \
GIT_SSL_CAINFO=/nix/var/nix/profiles/default/etc/ssl/certs/ca-bundle.crt \
NIX_SSL_CERT_FILE=/nix/var/nix/profiles/default/etc/ssl/certs/ca-bundle.crt
ENV \
ENV=/etc/profile \
PATH=/root/.nix-profile/bin:/root/.nix-profile/sbin:/bin:/sbin:/usr/bin:/usr/sbin \
GIT_SSL_CAINFO=/root/.nix-profile/etc/ssl/certs/ca-bundle.crt \
NIX_SSL_CERT_FILE=/root/.nix-profile/etc/ssl/certs/ca-bundle.crt \
NIX_PATH=/nix/var/nix/profiles/per-user/root/channels/
PATH=/nix/var/nix/profiles/default/bin:/nix/var/nix/profiles/default/sbin:/bin:/sbin:/usr/bin:/usr/sbin \
GIT_SSL_CAINFO=/nix/var/nix/profiles/default/etc/ssl/certs/ca-bundle.crt \
NIX_SSL_CERT_FILE=/nix/var/nix/profiles/default/etc/ssl/certs/ca-bundle.crt \
NIX_PATH=/nix/var/nix/profiles/per-user/root/channels
-10
View File
@@ -1,10 +0,0 @@
The Nix Emacs mode supports syntax highlighting, somewhat sensible
indenting, and refilling of comments.
To enable Nix mode in Emacs, add something like this to your ~/.emacs
file:
(load "/nix/share/emacs/site-lisp/nix-mode.el")
This automatically causes Nix mode to be activated for all files with
extension `.nix'.
-1
View File
@@ -1 +0,0 @@
$(eval $(call install-data-in,$(d)/nix-mode.el,$(datadir)/emacs/site-lisp))
-177
View File
@@ -1,177 +0,0 @@
;;; nix-mode.el --- Major mode for editing Nix expressions
;; Author: Eelco Dolstra
;; URL: https://github.com/NixOS/nix/tree/master/misc/emacs
;; Version: 1.0
;;; Commentary:
;;; Code:
(defun nix-syntax-match-antiquote (limit)
(let ((pos (next-single-char-property-change (point) 'nix-syntax-antiquote
nil limit)))
(when (and pos (> pos (point)))
(goto-char pos)
(let ((char (char-after pos)))
(pcase char
(`?$
(forward-char 2))
(`?}
(forward-char 1)))
(set-match-data (list pos (point)))
t))))
(defconst nix-font-lock-keywords
'("\\_<if\\_>" "\\_<then\\_>" "\\_<else\\_>" "\\_<assert\\_>" "\\_<with\\_>"
"\\_<let\\_>" "\\_<in\\_>" "\\_<rec\\_>" "\\_<inherit\\_>" "\\_<or\\_>"
("\\_<true\\_>" . font-lock-builtin-face)
("\\_<false\\_>" . font-lock-builtin-face)
("\\_<null\\_>" . font-lock-builtin-face)
("\\_<import\\_>" . font-lock-builtin-face)
("\\_<derivation\\_>" . font-lock-builtin-face)
("\\_<baseNameOf\\_>" . font-lock-builtin-face)
("\\_<toString\\_>" . font-lock-builtin-face)
("\\_<isNull\\_>" . font-lock-builtin-face)
("[a-zA-Z][a-zA-Z0-9\\+-\\.]*:[a-zA-Z0-9%/\\?:@&=\\+\\$,_\\.!~\\*'-]+"
. font-lock-constant-face)
("\\<\\([a-zA-Z_][a-zA-Z0-9_'\-\.]*\\)[ \t]*="
(1 font-lock-variable-name-face nil nil))
("<[a-zA-Z0-9._\\+-]+\\(/[a-zA-Z0-9._\\+-]+\\)*>"
. font-lock-constant-face)
("[a-zA-Z0-9._\\+-]*\\(/[a-zA-Z0-9._\\+-]+\\)+"
. font-lock-constant-face)
(nix-syntax-match-antiquote 0 font-lock-preprocessor-face t))
"Font lock keywords for nix.")
(defvar nix-mode-syntax-table
(let ((table (make-syntax-table)))
(modify-syntax-entry ?/ ". 14" table)
(modify-syntax-entry ?* ". 23" table)
(modify-syntax-entry ?# "< b" table)
(modify-syntax-entry ?\n "> b" table)
table)
"Syntax table for Nix mode.")
(defun nix-syntax-propertize-escaped-antiquote ()
"Set syntax properies for escaped antiquote marks."
nil)
(defun nix-syntax-propertize-multiline-string ()
"Set syntax properies for multiline string delimiters."
(let* ((start (match-beginning 0))
(end (match-end 0))
(context (save-excursion (save-match-data (syntax-ppss start))))
(string-type (nth 3 context)))
(pcase string-type
(`t
;; inside a multiline string
;; ending multi-line string delimiter
(put-text-property (1- end) end
'syntax-table (string-to-syntax "|")))
(`nil
;; beginning multi-line string delimiter
(put-text-property start (1+ start)
'syntax-table (string-to-syntax "|"))))))
(defun nix-syntax-propertize-antiquote ()
"Set syntax properties for antiquote marks."
(let* ((start (match-beginning 0)))
(put-text-property start (1+ start)
'syntax-table (string-to-syntax "|"))
(put-text-property start (+ start 2)
'nix-syntax-antiquote t)))
(defun nix-syntax-propertize-close-brace ()
"Set syntax properties for close braces.
If a close brace `}' ends an antiquote, the next character begins a string."
(let* ((start (match-beginning 0))
(end (match-end 0))
(context (save-excursion (save-match-data (syntax-ppss start))))
(open (nth 1 context)))
(when open ;; a corresponding open-brace was found
(let* ((antiquote (get-text-property open 'nix-syntax-antiquote)))
(when antiquote
(put-text-property (+ start 1) (+ start 2)
'syntax-table (string-to-syntax "|"))
(put-text-property start (1+ start)
'nix-syntax-antiquote t))))))
(defun nix-syntax-propertize (start end)
"Special syntax properties for Nix."
;; search for multi-line string delimiters
(goto-char start)
(remove-text-properties start end '(syntax-table nil nix-syntax-antiquote nil))
(funcall
(syntax-propertize-rules
("''\\${"
(0 (ignore (nix-syntax-propertize-escaped-antiquote))))
("''"
(0 (ignore (nix-syntax-propertize-multiline-string))))
("\\${"
(0 (ignore (nix-syntax-propertize-antiquote))))
("}"
(0 (ignore (nix-syntax-propertize-close-brace)))))
start end))
(defun nix-indent-line ()
"Indent current line in a Nix expression."
(interactive)
(indent-relative-maybe))
;;;###autoload
(define-derived-mode nix-mode prog-mode "Nix"
"Major mode for editing Nix expressions.
The following commands may be useful:
'\\[newline-and-indent]'
Insert a newline and move the cursor to align with the previous
non-empty line.
'\\[fill-paragraph]'
Refill a paragraph so that all lines are at most `fill-column'
lines long. This should do the right thing for comments beginning
with `#'. However, this command doesn't work properly yet if the
comment is adjacent to code (i.e., no intervening empty lines).
In that case, select the text to be refilled and use
`\\[fill-region]' instead.
The hook `nix-mode-hook' is run when Nix mode is started.
\\{nix-mode-map}
"
(set-syntax-table nix-mode-syntax-table)
;; Font lock support.
(setq-local font-lock-defaults '(nix-font-lock-keywords nil nil nil nil))
;; Special syntax properties for Nix
(setq-local syntax-propertize-function 'nix-syntax-propertize)
;; Look at text properties when parsing
(setq-local parse-sexp-lookup-properties t)
;; Automatic indentation [C-j].
(set (make-local-variable 'indent-line-function) 'nix-indent-line)
;; Indenting of comments.
(set (make-local-variable 'comment-start) "# ")
(set (make-local-variable 'comment-end) "")
(set (make-local-variable 'comment-start-skip) "\\(^\\|\\s-\\);?#+ *")
;; Filling of comments.
(set (make-local-variable 'adaptive-fill-mode) t)
(set (make-local-variable 'paragraph-start) "[ \t]*\\(#+[ \t]*\\)?$")
(set (make-local-variable 'paragraph-separate) paragraph-start))
;;;###autoload
(progn
(add-to-list 'auto-mode-alist '("\\.nix\\'" . nix-mode))
(add-to-list 'auto-mode-alist '("\\.nix.in\\'" . nix-mode)))
(provide 'nix-mode)
;;; nix-mode.el ends here
+2 -5
View File
@@ -4,6 +4,8 @@
<dict>
<key>Label</key>
<string>org.nixos.nix-daemon</string>
<key>KeepAlive</key>
<true/>
<key>RunAtLoad</key>
<true/>
<key>Program</key>
@@ -12,10 +14,5 @@
<string>/var/log/nix-daemon.log</string>
<key>StandardOutPath</key>
<string>/dev/null</string>
<key>EnvironmentVariables</key>
<dict>
<key>NIX_SSL_CERT_FILE</key>
<string>/nix/var/nix/profiles/default/etc/ssl/certs/ca-bundle.crt</string>
</dict>
</dict>
</plist>
-1
View File
@@ -7,4 +7,3 @@ ConditionPathIsReadWrite=@localstatedir@/nix/daemon-socket
[Service]
ExecStart=@@bindir@/nix-daemon nix-daemon --daemon
KillMode=process
Environment=XDG_CACHE_HOME=/root/.cache
+2 -2
View File
@@ -53,8 +53,8 @@ BUILD_SHARED_LIBS ?= 1
ifeq ($(BUILD_SHARED_LIBS), 1)
ifeq (CYGWIN,$(findstring CYGWIN,$(OS)))
GLOBAL_CFLAGS += -U__STRICT_ANSI__
GLOBAL_CXXFLAGS += -U__STRICT_ANSI__
GLOBAL_CFLAGS += -U__STRICT_ANSI__ -D_GNU_SOURCE
GLOBAL_CXXFLAGS += -U__STRICT_ANSI__ -D_GNU_SOURCE
else
GLOBAL_CFLAGS += -fPIC
GLOBAL_CXXFLAGS += -fPIC
+7
View File
@@ -45,6 +45,11 @@ endif
# - $(1)_INSTALL_DIR: the directory where the library will be
# installed. Defaults to $(libdir).
#
# - $(1)_EXCLUDE_FROM_LIBRARY_LIST: if defined, the library will not
# be automatically marked as a dependency of the top-level all
# target andwill not be listed in the make help output. This is
# useful for libraries built solely for testing, for example.
#
# - BUILD_SHARED_LIBS: if equal to ‘1’, a dynamic library will be
# built, otherwise a static library.
define build-library
@@ -149,7 +154,9 @@ define build-library
$(1)_DEPS := $$(foreach fn, $$($(1)_OBJS), $$(call filename-to-dep, $$(fn)))
-include $$($(1)_DEPS)
ifndef $(1)_EXCLUDE_FROM_LIBRARY_LIST
libs-list += $$($(1)_PATH)
endif
clean-files += $$(_d)/*.a $$(_d)/*.$(SO_EXT) $$(_d)/*.o $$(_d)/.*.dep $$($(1)_DEPS) $$($(1)_OBJS)
dist-files += $$(_srcs)
endef
+25 -6
View File
@@ -7,20 +7,39 @@ define run-install-test
endef
# Color code from https://unix.stackexchange.com/a/10065
installcheck:
@total=0; failed=0; for i in $(_installcheck-list); do \
@total=0; failed=0; \
red=""; \
green=""; \
yellow=""; \
normal=""; \
if [ -t 1 ]; then \
red=""; \
green=""; \
yellow=""; \
normal=""; \
fi; \
for i in $(_installcheck-list); do \
total=$$((total + 1)); \
echo "running test $$i"; \
if (cd $$(dirname $$i) && $(tests-environment) $$(basename $$i)); then \
echo "PASS: $$i"; \
printf "running test $$i..."; \
log="$$(cd $$(dirname $$i) && $(tests-environment) $$(basename $$i) 2>&1)"; \
status=$$?; \
if [ $$status -eq 0 ]; then \
echo " [$${green}PASS$$normal]"; \
elif [ $$status -eq 99 ]; then \
echo " [$${yellow}SKIP$$normal]"; \
else \
echo "FAIL: $$i"; \
echo " [$${red}FAIL$$normal]"; \
echo "$$log" | sed 's/^/ /'; \
failed=$$((failed + 1)); \
fi; \
done; \
if [ "$$failed" != 0 ]; then \
echo "$$failed out of $$total tests failed "; \
echo "$${red}$$failed out of $$total tests failed $$normal"; \
exit 1; \
else \
echo "$${green}All tests succeeded$$normal"; \
fi
.PHONY: check installcheck
+3 -51
View File
@@ -16,18 +16,15 @@ Source0: %{name}-%{version}.tar.bz2
%if 0%{?el5}
BuildRoot: %(mktemp -ud %{_tmppath}/%{name}-%{version}-%{release}-XXXXXX)
%endif
BuildRequires: perl(DBD::SQLite)
BuildRequires: perl(DBI)
BuildRequires: perl(ExtUtils::ParseXS)
Requires: /usr/bin/perl
Requires: curl
Requires: perl-DBD-SQLite
Requires: bzip2
Requires: gzip
Requires: xz
Requires: libseccomp
BuildRequires: bzip2-devel
BuildRequires: sqlite-devel
BuildRequires: libcurl-devel
BuildRequires: libseccomp-devel
# Hack to make that shitty RPM scanning hack shut up.
Provides: perl(Nix::SSH)
@@ -63,40 +60,8 @@ Requires: %{name} = %{version}-%{release}
%description doc
The %{name}-doc package contains documentation files for %{name}.
%package -n emacs-%{name}
Summary: Nix mode for Emacs
%if 0%{?rhel} && 0%{?rhel} < 7
Group: Applications/Editors
%endif
BuildArch: noarch
BuildRequires: emacs
Requires: emacs(bin) >= %{_emacs_version}
%description -n emacs-%{name}
This package provides a major mode for editing Nix expressions.
%package -n emacs-%{name}-el
Summary: Elisp source files for emacs-%{name}
%if 0%{?rhel} && 0%{?rhel} < 7
Group: Applications/Editors
%endif
BuildArch: noarch
Requires: emacs-%{name} = %{version}-%{release}
%description -n emacs-%{name}-el
This package contains the elisp source file for the Nix major mode for
GNU Emacs. You do not need to install this package to run Nix. Install
the emacs-%{name} package to edit Nix expressions with GNU Emacs.
%prep
%setup -q
# Install Perl modules to vendor_perl
# configure.ac need to be changed to make this global; however, this will
# also affect NixOS. Use discretion.
%{__sed} -i 's|perl5/site_perl/$perlversion/$perlarchname|perl5/vendor_perl|' \
configure
%build
@@ -108,7 +73,6 @@ extraFlags=
--docdir=%{_defaultdocdir}/%{name}-doc-%{version} \
$extraFlags
make -j$NIX_BUILD_CORES -l$NIX_BUILD_CORES
%{_emacs_bytecompile} misc/emacs/nix-mode.el
%install
@@ -134,9 +98,6 @@ done
# (until this is fixed in the relevant Makefile)
chmod -x $RPM_BUILD_ROOT%{_sysconfdir}/profile.d/nix.sh
# Copy the byte-compiled mode file by hand
cp -p misc/emacs/nix-mode.elc $RPM_BUILD_ROOT%{_emacs_sitelispdir}/
# we ship this file in the base package
rm -f $RPM_BUILD_ROOT%{_defaultdocdir}/%{name}-doc-%{version}/README
@@ -169,19 +130,17 @@ systemctl start nix-daemon.socket
%files
%{_bindir}/nix*
%{_libdir}/*.so
%{perl_vendorarch}/*
%exclude %dir %{perl_vendorarch}/auto/
%{_prefix}/libexec/*
%if ! 0%{?rhel} || 0%{?rhel} >= 7
%{_prefix}/lib/systemd/system/nix-daemon.socket
%{_prefix}/lib/systemd/system/nix-daemon.service
%endif
%{_datadir}/emacs/site-lisp/nix-mode.el
%{_datadir}/nix
%{_mandir}/man1/*.1*
%{_mandir}/man5/*.5*
%{_mandir}/man8/*.8*
%config(noreplace) %{_sysconfdir}/profile.d/nix.sh
%config(noreplace) %{_sysconfdir}/profile.d/nix-daemon.sh
/nix
%files devel
@@ -191,10 +150,3 @@ systemctl start nix-daemon.socket
%files doc
%docdir %{_defaultdocdir}/%{name}-doc-%{version}
%{_defaultdocdir}/%{name}-doc-%{version}
%files -n emacs-%{name}
%{_emacs_sitelispdir}/*.elc
#{_emacs_sitestartdir}/*.el
%files -n emacs-%{name}-el
%{_emacs_sitelispdir}/*.el
+14
View File
@@ -0,0 +1,14 @@
makefiles = local.mk
GLOBAL_CXXFLAGS += -std=c++14 -g -Wall
-include Makefile.config
OPTIMIZE = 1
ifeq ($(OPTIMIZE), 1)
GLOBAL_CFLAGS += -O3
GLOBAL_CXXFLAGS += -O3
endif
include mk/lib.mk
+18
View File
@@ -0,0 +1,18 @@
CC = @CC@
CFLAGS = @CFLAGS@
CXX = @CXX@
CXXFLAGS = @CXXFLAGS@
HAVE_SODIUM = @HAVE_SODIUM@
PACKAGE_NAME = @PACKAGE_NAME@
PACKAGE_VERSION = @PACKAGE_VERSION@
SODIUM_LIBS = @SODIUM_LIBS@
NIX_CFLAGS = @NIX_CFLAGS@
NIX_LIBS = @NIX_LIBS@
nixbindir = @nixbindir@
curl = @curl@
nixlibexecdir = @nixlibexecdir@
nixlocalstatedir = @nixlocalstatedir@
perl = @perl@
perllibdir = @perllibdir@
nixstoredir = @nixstoredir@
nixsysconfdir = @nixsysconfdir@
+99
View File
@@ -0,0 +1,99 @@
AC_INIT(nix-perl, m4_esyscmd([bash -c "echo -n $(cat ../version)$VERSION_SUFFIX"]))
AC_CONFIG_SRCDIR(MANIFEST)
AC_CONFIG_AUX_DIR(../config)
CFLAGS=
CXXFLAGS=
AC_PROG_CC
AC_PROG_CXX
AX_CXX_COMPILE_STDCXX_11
# Use 64-bit file system calls so that we can support files > 2 GiB.
AC_SYS_LARGEFILE
AC_DEFUN([NEED_PROG],
[
AC_PATH_PROG($1, $2)
if test -z "$$1"; then
AC_MSG_ERROR([$2 is required])
fi
])
NEED_PROG(perl, perl)
NEED_PROG(curl, curl)
NEED_PROG(bzip2, bzip2)
NEED_PROG(xz, xz)
# Test that Perl has the open/fork feature (Perl 5.8.0 and beyond).
AC_MSG_CHECKING([whether Perl is recent enough])
if ! $perl -e 'open(FOO, "-|", "true"); while (<FOO>) { print; }; close FOO or die;'; then
AC_MSG_RESULT(no)
AC_MSG_ERROR([Your Perl version is too old. Nix requires Perl 5.8.0 or newer.])
fi
AC_MSG_RESULT(yes)
# Figure out where to install Perl modules.
AC_MSG_CHECKING([for the Perl installation prefix])
perlversion=$($perl -e 'use Config; print $Config{version};')
perlarchname=$($perl -e 'use Config; print $Config{archname};')
AC_SUBST(perllibdir, [${libdir}/perl5/site_perl/$perlversion/$perlarchname])
AC_MSG_RESULT($perllibdir)
# Look for libsodium, an optional dependency.
PKG_CHECK_MODULES([SODIUM], [libsodium],
[AC_DEFINE([HAVE_SODIUM], [1], [Whether to use libsodium for cryptography.])
CXXFLAGS="$SODIUM_CFLAGS $CXXFLAGS"
have_sodium=1], [have_sodium=])
AC_SUBST(HAVE_SODIUM, [$have_sodium])
# Check for the required Perl dependencies (DBI and DBD::SQLite).
perlFlags="-I$perllibdir"
AC_ARG_WITH(dbi, AC_HELP_STRING([--with-dbi=PATH],
[prefix of the Perl DBI library]),
perlFlags="$perlFlags -I$withval")
AC_ARG_WITH(dbd-sqlite, AC_HELP_STRING([--with-dbd-sqlite=PATH],
[prefix of the Perl DBD::SQLite library]),
perlFlags="$perlFlags -I$withval")
AC_MSG_CHECKING([whether DBD::SQLite works])
if ! $perl $perlFlags -e 'use DBI; use DBD::SQLite;' 2>&5; then
AC_MSG_RESULT(no)
AC_MSG_FAILURE([The Perl modules DBI and/or DBD::SQLite are missing.])
fi
AC_MSG_RESULT(yes)
AC_SUBST(perlFlags)
PKG_CHECK_MODULES([NIX], [nix-store])
NEED_PROG([NIX_INSTANTIATE_PROGRAM], [nix-instantiate])
# Get nix configure values
nixbindir=$("$NIX_INSTANTIATE_PROGRAM" --eval '<nix/config.nix>' -A nixBinDir | tr -d \")
nixlibexecdir=$("$NIX_INSTANTIATE_PROGRAM" --eval '<nix/config.nix>' -A nixLibexecDir | tr -d \")
nixlocalstatedir=$("$NIX_INSTANTIATE_PROGRAM" --eval '<nix/config.nix>' -A nixLocalstateDir | tr -d \")
nixsysconfdir=$("$NIX_INSTANTIATE_PROGRAM" --eval '<nix/config.nix>' -A nixSysconfDir | tr -d \")
nixstoredir=$("$NIX_INSTANTIATE_PROGRAM" --eval '<nix/config.nix>' -A nixStoreDir | tr -d \")
AC_SUBST(nixbindir)
AC_SUBST(nixlibexecdir)
AC_SUBST(nixlocalstatedir)
AC_SUBST(nixsysconfdir)
AC_SUBST(nixstoredir)
# Expand all variables in config.status.
test "$prefix" = NONE && prefix=$ac_default_prefix
test "$exec_prefix" = NONE && exec_prefix='${prefix}'
for name in $ac_subst_vars; do
declare $name="$(eval echo "${!name}")"
declare $name="$(eval echo "${!name}")"
declare $name="$(eval echo "${!name}")"
done
rm -f Makefile.config
ln -sfn ../mk mk
AC_CONFIG_FILES([])
AC_OUTPUT
+14 -31
View File
@@ -4,48 +4,31 @@ use MIME::Base64;
$version = "@PACKAGE_VERSION@";
$binDir = $ENV{"NIX_BIN_DIR"} || "@bindir@";
$libexecDir = $ENV{"NIX_LIBEXEC_DIR"} || "@libexecdir@";
$stateDir = $ENV{"NIX_STATE_DIR"} || "@localstatedir@/nix";
$logDir = $ENV{"NIX_LOG_DIR"} || "@localstatedir@/log/nix";
$confDir = $ENV{"NIX_CONF_DIR"} || "@sysconfdir@/nix";
$storeDir = $ENV{"NIX_STORE_DIR"} || "@storedir@";
$binDir = $ENV{"NIX_BIN_DIR"} || "@nixbindir@";
$libexecDir = $ENV{"NIX_LIBEXEC_DIR"} || "@nixlibexecdir@";
$stateDir = $ENV{"NIX_STATE_DIR"} || "@nixlocalstatedir@/nix";
$logDir = $ENV{"NIX_LOG_DIR"} || "@nixlocalstatedir@/log/nix";
$confDir = $ENV{"NIX_CONF_DIR"} || "@nixsysconfdir@/nix";
$storeDir = $ENV{"NIX_STORE_DIR"} || "@nixstoredir@";
$bzip2 = "@bzip2@";
$xz = "@xz@";
$curl = "@curl@";
$useBindings = "@perlbindings@" eq "yes";
$useBindings = 1;
%config = ();
%binaryCachePublicKeys = ();
$defaultPublicKeys = "cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=";
sub readConfig {
if (defined $ENV{'_NIX_OPTIONS'}) {
foreach my $s (split '\n', $ENV{'_NIX_OPTIONS'}) {
my ($n, $v) = split '=', $s, 2;
$config{$n} = $v;
}
} else {
my $config = "$confDir/nix.conf";
return unless -f $config;
my $config = "$confDir/nix.conf";
return unless -f $config;
open CONFIG, "<$config" or die "cannot open ‘$config’";
while (<CONFIG>) {
/^\s*([\w\-\.]+)\s*=\s*(.*)$/ or next;
$config{$1} = $2;
}
close CONFIG;
}
foreach my $s (split(/ /, $config{"binary-cache-public-keys"} // $defaultPublicKeys)) {
my ($keyName, $publicKey) = split ":", $s;
next unless defined $keyName && defined $publicKey;
$binaryCachePublicKeys{$keyName} = decode_base64($publicKey);
open CONFIG, "<$config" or die "cannot open '$config'";
while (<CONFIG>) {
/^\s*([\w\-\.]+)\s*=\s*(.*)$/ or next;
$config{$1} = $2;
}
close CONFIG;
}
return 1;
+2 -2
View File
@@ -35,14 +35,14 @@ sub copyToOpen {
my $missingSize = 0;
$missingSize += (queryPathInfo($_, 1))[3] foreach @missing;
printf STDERR "copying %d missing paths (%.2f MiB) to ‘$sshHost’...\n",
printf STDERR "copying %d missing paths (%.2f MiB) to '$sshHost'...\n",
scalar(@missing), $missingSize / (1024**2);
return if $dryRun;
# Send the "import paths" command.
syswrite($to, pack("L<x4", 4)) or die;
exportPaths(fileno($to), @missing);
readInt($from) == 1 or die "remote machine ‘$sshHost’ failed to import closure\n";
readInt($from) == 1 or die "remote machine '$sshHost' failed to import closure\n";
}
+6 -6
View File
@@ -60,10 +60,10 @@ sub readManifest_ {
# Decompress the manifest if necessary.
if ($manifest =~ /\.bz2$/) {
open MANIFEST, "$Nix::Config::bzip2 -d < $manifest |"
or die "cannot decompress ‘$manifest’: $!";
or die "cannot decompress '$manifest': $!";
} else {
open MANIFEST, "<$manifest"
or die "cannot open ‘$manifest’: $!";
or die "cannot open '$manifest': $!";
}
my $inside = 0;
@@ -287,7 +287,7 @@ sub parseNARInfo {
# FIXME: might be useful to support multiple signatures per .narinfo.
if (!defined $sig) {
warn "NAR info file ‘$location’ lacks a signature; ignoring\n";
warn "NAR info file '$location' lacks a signature; ignoring\n";
return undef;
}
my ($keyName, $sig64) = split ":", $sig;
@@ -295,7 +295,7 @@ sub parseNARInfo {
my $publicKey = $Nix::Config::binaryCachePublicKeys{$keyName};
if (!defined $publicKey) {
warn "NAR info file ‘$location’ is signed by unknown key ‘$keyName’; ignoring\n";
warn "NAR info file '$location' is signed by unknown key '$keyName'; ignoring\n";
return undef;
}
@@ -306,12 +306,12 @@ sub parseNARInfo {
[ map { "$Nix::Config::storeDir/$_" } @refs ]);
};
if ($@) {
warn "cannot compute fingerprint of ‘$location’; ignoring\n";
warn "cannot compute fingerprint of '$location'; ignoring\n";
return undef;
}
if (!checkSignature($publicKey, decode_base64($sig64), $fingerprint)) {
warn "NAR info file ‘$location’ has an incorrect signature; ignoring\n";
warn "NAR info file '$location' has an incorrect signature; ignoring\n";
return undef;
}
+1 -1
View File
@@ -97,7 +97,7 @@ sub connectToRemoteNix {
syswrite($to, pack("L<x4L<x4", $SERVE_MAGIC_1, $clientVersion)) or die;
$magic = readInt($from);
};
die "unable to connect to ‘$sshHost’\n" if $@;
die "unable to connect to '$sshHost'\n" if $@;
die "did not get valid handshake from remote host\n" if $magic != 0x5452eecb;
my $serverVersion = readInt($from);
+11 -13
View File
@@ -1,3 +1,5 @@
#include "config.h"
#include "EXTERN.h"
#include "perl.h"
#include "XSUB.h"
@@ -25,9 +27,7 @@ static ref<Store> store()
static std::shared_ptr<Store> _store;
if (!_store) {
try {
logger = makeDefaultLogger();
settings.loadConfFile();
settings.update();
settings.lockCPU = false;
_store = openStore();
} catch (Error & e) {
@@ -81,8 +81,7 @@ SV * queryReferences(char * path)
SV * queryPathHash(char * path)
PPCODE:
try {
auto hash = store()->queryPathInfo(path)->narHash;
string s = "sha256:" + printHash32(hash);
auto s = store()->queryPathInfo(path)->narHash.to_string();
XPUSHs(sv_2mortal(newSVpv(s.c_str(), 0)));
} catch (Error & e) {
croak("%s", e.what());
@@ -108,7 +107,7 @@ SV * queryPathInfo(char * path, int base32)
XPUSHs(&PL_sv_undef);
else
XPUSHs(sv_2mortal(newSVpv(info->deriver.c_str(), 0)));
string s = "sha256:" + (base32 ? printHash32(info->narHash) : printHash(info->narHash));
auto s = info->narHash.to_string(base32 ? Base32 : Base16);
XPUSHs(sv_2mortal(newSVpv(s.c_str(), 0)));
mXPUSHi(info->registrationTime);
mXPUSHi(info->narSize);
@@ -184,7 +183,7 @@ void importPaths(int fd, int dontCheckSigs)
PPCODE:
try {
FdSource source(fd);
store()->importPaths(source, 0, dontCheckSigs);
store()->importPaths(source, nullptr, dontCheckSigs ? NoCheckSigs : CheckSigs);
} catch (Error & e) {
croak("%s", e.what());
}
@@ -194,7 +193,7 @@ SV * hashPath(char * algo, int base32, char * path)
PPCODE:
try {
Hash h = hashPath(parseHashType(algo), path).first;
string s = base32 ? printHash32(h) : printHash(h);
auto s = h.to_string(base32 ? Base32 : Base16, false);
XPUSHs(sv_2mortal(newSVpv(s.c_str(), 0)));
} catch (Error & e) {
croak("%s", e.what());
@@ -205,7 +204,7 @@ SV * hashFile(char * algo, int base32, char * path)
PPCODE:
try {
Hash h = hashFile(parseHashType(algo), path);
string s = base32 ? printHash32(h) : printHash(h);
auto s = h.to_string(base32 ? Base32 : Base16, false);
XPUSHs(sv_2mortal(newSVpv(s.c_str(), 0)));
} catch (Error & e) {
croak("%s", e.what());
@@ -216,7 +215,7 @@ SV * hashString(char * algo, int base32, char * s)
PPCODE:
try {
Hash h = hashString(parseHashType(algo), s);
string s = base32 ? printHash32(h) : printHash(h);
auto s = h.to_string(base32 ? Base32 : Base16, false);
XPUSHs(sv_2mortal(newSVpv(s.c_str(), 0)));
} catch (Error & e) {
croak("%s", e.what());
@@ -226,8 +225,8 @@ SV * hashString(char * algo, int base32, char * s)
SV * convertHash(char * algo, char * s, int toBase32)
PPCODE:
try {
Hash h = parseHash16or32(parseHashType(algo), s);
string s = toBase32 ? printHash32(h) : printHash(h);
Hash h(s, parseHashType(algo));
string s = h.to_string(toBase32 ? Base32 : Base16, false);
XPUSHs(sv_2mortal(newSVpv(s.c_str(), 0)));
} catch (Error & e) {
croak("%s", e.what());
@@ -286,8 +285,7 @@ SV * addToStore(char * srcPath, int recursive, char * algo)
SV * makeFixedOutputPath(int recursive, char * algo, char * hash, char * name)
PPCODE:
try {
HashType ht = parseHashType(algo);
Hash h = parseHash16or32(ht, hash);
Hash h(hash, parseHashType(algo));
Path path = store()->makeFixedOutputPath(recursive, h, name);
XPUSHs(sv_2mortal(newSVpv(path.c_str(), 0)));
} catch (Error & e) {
+3 -3
View File
@@ -10,7 +10,7 @@ $urlRE = "(?: [a-zA-Z][a-zA-Z0-9\+\-\.]*\:[a-zA-Z0-9\%\/\?\:\@\&\=\+\$\,\-\_\.\!
sub checkURL {
my ($url) = @_;
die "invalid URL ‘$url’\n" unless $url =~ /^ $urlRE $ /x;
die "invalid URL '$url'\n" unless $url =~ /^ $urlRE $ /x;
}
sub uniq {
@@ -26,7 +26,7 @@ sub uniq {
sub writeFile {
my ($fn, $s) = @_;
open TMP, ">$fn" or die "cannot create file ‘$fn’: $!";
open TMP, ">$fn" or die "cannot create file '$fn': $!";
print TMP "$s" or die;
close TMP or die;
}
@@ -34,7 +34,7 @@ sub writeFile {
sub readFile {
local $/ = undef;
my ($fn) = @_;
open TMP, "<$fn" or die "cannot open file ‘$fn’: $!";
open TMP, "<$fn" or die "cannot open file '$fn': $!";
my $s = <TMP>;
close TMP or die;
return $s;
+28 -33
View File
@@ -1,48 +1,43 @@
nix_perl_sources := \
$(d)/lib/Nix/Store.pm \
$(d)/lib/Nix/Manifest.pm \
$(d)/lib/Nix/SSH.pm \
$(d)/lib/Nix/CopyClosure.pm \
$(d)/lib/Nix/Config.pm.in \
$(d)/lib/Nix/Utils.pm
lib/Nix/Store.pm \
lib/Nix/Manifest.pm \
lib/Nix/SSH.pm \
lib/Nix/CopyClosure.pm \
lib/Nix/Config.pm.in \
lib/Nix/Utils.pm
nix_perl_modules := $(nix_perl_sources:.in=)
$(foreach x, $(nix_perl_modules), $(eval $(call install-data-in, $(x), $(perllibdir)/Nix)))
ifeq ($(perlbindings), yes)
$(d)/lib/Nix/Store.cc: $(d)/lib/Nix/Store.xs
lib/Nix/Store.cc: lib/Nix/Store.xs
$(trace-gen) xsubpp $^ -output $@
libraries += Store
libraries += Store
Store_DIR := $(d)/lib/Nix
Store_DIR := lib/Nix
Store_SOURCES := $(Store_DIR)/Store.cc
Store_SOURCES := $(Store_DIR)/Store.cc
Store_CXXFLAGS = \
-I$(shell $(perl) -e 'use Config; print $$Config{archlibexp};')/CORE \
-D_FILE_OFFSET_BITS=64 \
-Wno-unknown-warning-option -Wno-unused-variable -Wno-literal-suffix \
-Wno-reserved-user-defined-literal -Wno-duplicate-decl-specifier -Wno-pointer-bool-conversion
Store_CXXFLAGS = \
$(NIX_CFLAGS) \
-I$(shell perl -e 'use Config; print $$Config{archlibexp};')/CORE \
-D_FILE_OFFSET_BITS=64 \
-Wno-unknown-warning-option -Wno-unused-variable -Wno-literal-suffix \
-Wno-reserved-user-defined-literal -Wno-duplicate-decl-specifier -Wno-pointer-bool-conversion
Store_LIBS = libstore libutil
Store_LDFLAGS := $(SODIUM_LIBS)
ifeq (CYGWIN,$(findstring CYGWIN,$(OS)))
archlib = $(shell perl -E 'use Config; print $$Config{archlib};')
libperl = $(shell perl -E 'use Config; print $$Config{libperl};')
Store_LDFLAGS += $(shell find ${archlib} -name ${libperl})
endif
Store_ALLOW_UNDEFINED = 1
Store_FORCE_INSTALL = 1
Store_INSTALL_DIR = $(perllibdir)/auto/Nix/Store
Store_LDFLAGS := $(SODIUM_LIBS) $(NIX_LIBS)
ifeq (CYGWIN,$(findstring CYGWIN,$(OS)))
archlib = $(shell perl -E 'use Config; print $$Config{archlib};')
libperl = $(shell perl -E 'use Config; print $$Config{libperl};')
Store_LDFLAGS += $(shell find ${archlib} -name ${libperl})
endif
clean-files += $(d)/lib/Nix/Config.pm $(d)/lib/Nix/Store.cc
Store_ALLOW_UNDEFINED = 1
Store_FORCE_INSTALL = 1
Store_INSTALL_DIR = $(perllibdir)/auto/Nix/Store
clean-files += lib/Nix/Config.pm lib/Nix/Store.cc Makefile.config
+36
View File
@@ -0,0 +1,36 @@
{ pkgs }:
rec {
# Use "busybox-sandbox-shell" if present,
# if not (legacy) fallback and hope it's sufficient.
sh = pkgs.busybox-sandbox-shell or (pkgs.busybox.override {
useMusl = true;
enableStatic = true;
enableMinimal = true;
extraConfig = ''
CONFIG_FEATURE_FANCY_ECHO y
CONFIG_FEATURE_SH_MATH y
CONFIG_FEATURE_SH_MATH_64 y
CONFIG_ASH y
CONFIG_ASH_OPTIMIZE_FOR_SIZE y
CONFIG_ASH_ALIAS y
CONFIG_ASH_BASH_COMPAT y
CONFIG_ASH_CMDCMD y
CONFIG_ASH_ECHO y
CONFIG_ASH_GETOPTS y
CONFIG_ASH_INTERNAL_GLOB y
CONFIG_ASH_JOB_CONTROL y
CONFIG_ASH_PRINTF y
CONFIG_ASH_TEST y
'';
});
configureFlags =
[ "--disable-init-state"
"--enable-gc"
] ++ pkgs.lib.optionals pkgs.stdenv.isLinux [
"--with-sandbox-shell=${sh}/bin/busybox"
];
}
+92 -53
View File
@@ -1,14 +1,12 @@
{ nix ? { outPath = ./.; revCount = 1234; shortRev = "abcdef"; }
, nixpkgs ? { outPath = <nixpkgs>; revCount = 1234; shortRev = "abcdef"; }
{ nix ? builtins.fetchGit ./.
, nixpkgs ? builtins.fetchGit { url = https://github.com/NixOS/nixpkgs.git; ref = "nix-2.0"; }
, officialRelease ? false
, systems ? [ "x86_64-linux" "i686-linux" "x86_64-darwin" "aarch64-linux" ]
}:
let
pkgs = import <nixpkgs> {};
systems = [ "x86_64-linux" "i686-linux" "x86_64-darwin" ];
pkgs = import nixpkgs { system = builtins.currentSystem or "x86_64-linux"; };
jobs = rec {
@@ -24,27 +22,22 @@ let
inherit officialRelease;
buildInputs =
[ curl bison flex perl libxml2 libxslt bzip2 xz
[ curl bison flex libxml2 libxslt
bzip2 xz brotli
pkgconfig sqlite libsodium boehmgc
docbook5 docbook5_xsl
autoconf-archive
git
];
] ++ lib.optional stdenv.isLinux libseccomp;
configureFlags = ''
--with-dbi=${perlPackages.DBI}/${perl.libPrefix}
--with-dbd-sqlite=${perlPackages.DBDSQLite}/${perl.libPrefix}
--enable-gc
'';
configureFlags = "--enable-gc";
postUnpack = ''
# Clean up when building from a working tree.
if [[ -d $sourceRoot/.git ]]; then
git -C $sourceRoot clean -fd
fi
(cd source && find . -type f) | cut -c3- > source/.dist-files
cat source/.dist-files
'';
preConfigure = ''
(cd perl ; autoreconf --install --force --verbose)
# TeX needs a writable font cache.
export VARTEXFONTS=$TMPDIR/texfonts
'';
@@ -66,28 +59,33 @@ let
build = pkgs.lib.genAttrs systems (system:
with import <nixpkgs> { inherit system; };
with import nixpkgs { inherit system; };
with import ./release-common.nix { inherit pkgs; };
releaseTools.nixBuild {
name = "nix";
src = tarball;
buildInputs =
[ curl perl bzip2 xz openssl pkgconfig sqlite boehmgc ]
++ lib.optional stdenv.isLinux libsodium
++ lib.optional stdenv.isLinux
[ curl
bzip2 xz brotli
openssl pkgconfig sqlite boehmgc
# Tests
git
mercurial
]
++ lib.optional stdenv.isLinux libseccomp
++ lib.optional (stdenv.isLinux || stdenv.isDarwin) libsodium
++ lib.optional (stdenv.isLinux || stdenv.isDarwin)
(aws-sdk-cpp.override {
apis = ["s3"];
customMemoryManagement = false;
});
configureFlags = ''
--disable-init-state
--with-dbi=${perlPackages.DBI}/${perl.libPrefix}
--with-dbd-sqlite=${perlPackages.DBDSQLite}/${perl.libPrefix}
--enable-gc
--sysconfdir=/etc
'';
configureFlags = configureFlags ++
[ "--sysconfdir=/etc" ];
enableParallelBuilding = true;
@@ -102,10 +100,34 @@ let
});
perlBindings = pkgs.lib.genAttrs systems (system:
let pkgs = import nixpkgs { inherit system; }; in with pkgs;
releaseTools.nixBuild {
name = "nix-perl";
src = tarball;
buildInputs =
[ (builtins.getAttr system jobs.build) curl bzip2 xz pkgconfig pkgs.perl ]
++ lib.optional (stdenv.isLinux || stdenv.isDarwin) libsodium;
configureFlags = ''
--with-dbi=${perlPackages.DBI}/${pkgs.perl.libPrefix}
--with-dbd-sqlite=${perlPackages.DBDSQLite}/${pkgs.perl.libPrefix}
'';
enableParallelBuilding = true;
postUnpack = "sourceRoot=$sourceRoot/perl";
preBuild = "unset NIX_INDENT_MAKE";
});
binaryTarball = pkgs.lib.genAttrs systems (system:
# FIXME: temporarily use a different branch for the Darwin build.
with import <nixpkgs> { inherit system; };
with import nixpkgs { inherit system; };
let
toplevel = builtins.getAttr system jobs.build;
@@ -114,7 +136,7 @@ let
runCommand "nix-binary-tarball-${version}"
{ exportReferencesGraph = [ "closure1" toplevel "closure2" cacert ];
buildInputs = [ perl ];
buildInputs = [ perl ] ++ lib.optional (system != "aarch64-linux") shellcheck;
meta.description = "Distribution-independent Nix bootstrap binaries for ${system}";
}
''
@@ -123,7 +145,17 @@ let
substitute ${./scripts/install-nix-from-closure.sh} $TMPDIR/install \
--subst-var-by nix ${toplevel} \
--subst-var-by cacert ${cacert}
substitute ${./scripts/install-darwin-multi-user.sh} $TMPDIR/install-darwin-multi-user \
--subst-var-by nix ${toplevel} \
--subst-var-by cacert ${cacert}
if type -p shellcheck; then
shellcheck -e SC1090 $TMPDIR/install
shellcheck -e SC1091,SC2002 $TMPDIR/install-darwin-multi-user
fi
chmod +x $TMPDIR/install
chmod +x $TMPDIR/install-darwin-multi-user
dir=nix-${version}-${system}
fn=$out/$dir.tar.bz2
mkdir -p $out/nix-support
@@ -135,34 +167,32 @@ let
--transform "s,$TMPDIR/install,$dir/install," \
--transform "s,$TMPDIR/reginfo,$dir/.reginfo," \
--transform "s,$NIX_STORE,$dir/store,S" \
$TMPDIR/install $TMPDIR/reginfo $storePaths
$TMPDIR/install $TMPDIR/install-darwin-multi-user $TMPDIR/reginfo $storePaths
'');
coverage =
with import <nixpkgs> { system = "x86_64-linux"; };
with import nixpkgs { system = "x86_64-linux"; };
releaseTools.coverageAnalysis {
name = "nix-build";
src = tarball;
buildInputs =
[ curl perl bzip2 openssl pkgconfig sqlite xz libsodium
[ curl bzip2 openssl pkgconfig sqlite xz libsodium libseccomp
# These are for "make check" only:
graphviz libxml2 libxslt
graphviz libxml2 libxslt git mercurial
];
configureFlags = ''
--disable-init-state
--with-dbi=${perlPackages.DBI}/${perl.libPrefix}
--with-dbd-sqlite=${perlPackages.DBDSQLite}/${perl.libPrefix}
'';
dontInstall = false;
doInstallCheck = true;
lcovFilter = [ "*/boost/*" "*-tab.*" ];
lcovFilter = [ "*/boost/*" "*-tab.*" "*/nlohmann/*" "*/linenoise/*" ];
# We call `dot', and even though we just use it to
# syntax-check generated dot files, it still requires some
@@ -175,11 +205,9 @@ let
rpm_fedora25x86_64 = makeRPM_x86_64 (diskImageFunsFun: diskImageFunsFun.fedora25x86_64) [ "libsodium-devel" ];
deb_debian8i386 = makeDeb_i686 (diskImageFuns: diskImageFuns.debian8i386) [ "libsodium-dev" ] [ "libsodium13" ];
deb_debian8x86_64 = makeDeb_x86_64 (diskImageFunsFun: diskImageFunsFun.debian8x86_64) [ "libsodium-dev" ] [ "libsodium13" ];
#deb_debian8i386 = makeDeb_i686 (diskImageFuns: diskImageFuns.debian8i386) [ "libsodium-dev" ] [ "libsodium13" ];
#deb_debian8x86_64 = makeDeb_x86_64 (diskImageFunsFun: diskImageFunsFun.debian8x86_64) [ "libsodium-dev" ] [ "libsodium13" ];
deb_ubuntu1410i386 = makeDeb_i686 (diskImageFuns: diskImageFuns.ubuntu1410i386) [] [];
deb_ubuntu1410x86_64 = makeDeb_x86_64 (diskImageFuns: diskImageFuns.ubuntu1410x86_64) [] [];
deb_ubuntu1604i386 = makeDeb_i686 (diskImageFuns: diskImageFuns.ubuntu1604i386) [ "libsodium-dev" ] [ "libsodium18" ];
deb_ubuntu1604x86_64 = makeDeb_x86_64 (diskImageFuns: diskImageFuns.ubuntu1604x86_64) [ "libsodium-dev" ] [ "libsodium18" ];
deb_ubuntu1610i386 = makeDeb_i686 (diskImageFuns: diskImageFuns.ubuntu1610i386) [ "libsodium-dev" ] [ "libsodium18" ];
@@ -188,15 +216,25 @@ let
# System tests.
tests.remoteBuilds = (import ./tests/remote-builds.nix rec {
inherit nixpkgs;
nix = build.x86_64-linux; system = "x86_64-linux";
});
tests.nix-copy-closure = (import ./tests/nix-copy-closure.nix rec {
inherit nixpkgs;
nix = build.x86_64-linux; system = "x86_64-linux";
});
tests.setuid = pkgs.lib.genAttrs
["i686-linux" "x86_64-linux"]
(system:
import ./tests/setuid.nix rec {
inherit nixpkgs;
nix = build.${system}; inherit system;
});
tests.binaryTarball =
with import <nixpkgs> { system = "x86_64-linux"; };
with import nixpkgs { system = "x86_64-linux"; };
vmTools.runInLinuxImage (runCommand "nix-binary-tarball-test"
{ diskImage = vmTools.diskImages.ubuntu1204x86_64;
}
@@ -215,7 +253,7 @@ let
''); # */
tests.evalNixpkgs =
import <nixpkgs/pkgs/top-level/make-tarball.nix> {
import (nixpkgs + "/pkgs/top-level/make-tarball.nix") {
inherit nixpkgs;
inherit pkgs;
nix = build.x86_64-linux;
@@ -246,8 +284,8 @@ let
binaryTarball.i686-linux
binaryTarball.x86_64-darwin
binaryTarball.x86_64-linux
deb_debian8i386
deb_debian8x86_64
#deb_debian8i386
#deb_debian8x86_64
deb_ubuntu1604i386
deb_ubuntu1604x86_64
rpm_fedora25i386
@@ -269,16 +307,17 @@ let
makeRPM =
system: diskImageFun: extraPackages:
with import <nixpkgs> { inherit system; };
with import nixpkgs { inherit system; };
releaseTools.rpmBuild rec {
name = "nix-rpm";
src = jobs.tarball;
diskImage = (diskImageFun vmTools.diskImageFuns)
{ extraPackages =
[ "perl-DBD-SQLite" "perl-devel" "sqlite" "sqlite-devel" "bzip2-devel" "emacs" "libcurl-devel" "openssl-devel" "xz-devel" ]
[ "sqlite" "sqlite-devel" "bzip2-devel" "libcurl-devel" "openssl-devel" "xz-devel" "libseccomp-devel" ]
++ extraPackages; };
memSize = 1024;
# At most 2047MB can be simulated in qemu-system-i386
memSize = 2047;
meta.schedulingPriority = 50;
postRPMInstall = "cd /tmp/rpmout/BUILD/nix-* && make installcheck";
#enableParallelBuilding = true;
@@ -291,21 +330,21 @@ let
makeDeb =
system: diskImageFun: extraPackages: extraDebPackages:
with import <nixpkgs> { inherit system; };
with import nixpkgs { inherit system; };
releaseTools.debBuild {
name = "nix-deb";
src = jobs.tarball;
diskImage = (diskImageFun vmTools.diskImageFuns)
{ extraPackages =
[ "libdbd-sqlite3-perl" "libsqlite3-dev" "libbz2-dev" "libwww-curl-perl" "libcurl-dev" "libcurl3-nss" "libssl-dev" "liblzma-dev" ]
[ "libsqlite3-dev" "libbz2-dev" "libcurl-dev" "libcurl3-nss" "libssl-dev" "liblzma-dev" "libseccomp-dev" ]
++ extraPackages; };
memSize = 1024;
meta.schedulingPriority = 50;
postInstall = "make installcheck";
configureFlags = "--sysconfdir=/etc";
debRequires =
[ "curl" "libdbd-sqlite3-perl" "libsqlite3-0" "libbz2-1.0" "bzip2" "xz-utils" "libwww-curl-perl" "libssl1.0.0" "liblzma5" ]
[ "curl" "libsqlite3-0" "libbz2-1.0" "bzip2" "xz-utils" "libssl1.0.0" "liblzma5" "libseccomp2" ]
++ extraDebPackages;
debMaintainer = "Eelco Dolstra <eelco.dolstra@logicblox.com>";
doInstallCheck = true;
+819
View File
@@ -0,0 +1,819 @@
#!/bin/bash
set -eu
set -o pipefail
# Sourced from:
# - https://github.com/LnL7/nix-darwin/blob/8c29d0985d74b4a990238497c47a2542a5616b3c/bootstrap.sh
# - https://gist.github.com/expipiplus1/e571ce88c608a1e83547c918591b149f/ac504c6c1b96e65505fbda437a28ce563408ecb0
# - https://github.com/NixOS/nixos-org-configurations/blob/a122f418797713d519aadf02e677fce0dc1cb446/delft/scripts/nix-mac-installer.sh
# - https://github.com/matthewbauer/macNixOS/blob/f6045394f9153edea417be90c216788e754feaba/install-macNixOS.sh
# - https://gist.github.com/LnL7/9717bd6cdcb30b086fd7f2093e5f8494/86b26f852ce563e973acd30f796a9a416248c34a
#
# however tracking which bits came from which would be impossible.
readonly ESC='\033[0m'
readonly BOLD='\033[38;1m'
readonly BLUE='\033[38;34m'
readonly BLUE_UL='\033[38;4;34m'
readonly GREEN='\033[38;32m'
readonly GREEN_UL='\033[38;4;32m'
readonly RED='\033[38;31m'
readonly RED_UL='\033[38;4;31m'
readonly YELLOW='\033[38;33m'
readonly YELLOW_UL='\033[38;4;33m'
readonly CORES=$(sysctl -n hw.ncpu)
readonly NIX_USER_COUNT="32"
readonly NIX_BUILD_GROUP_ID="30000"
readonly NIX_BUILD_GROUP_NAME="nixbld"
readonly NIX_FIRST_BUILD_UID="30001"
# Please don't change this. We don't support it, because the
# default shell profile that comes with Nix doesn't support it.
readonly NIX_ROOT="/nix"
readonly PLIST_DEST=/Library/LaunchDaemons/org.nixos.nix-daemon.plist
readonly PROFILE_TARGETS=("/etc/bashrc" "/etc/zshrc")
readonly PROFILE_BACKUP_SUFFIX=".backup-before-nix"
readonly PROFILE_NIX_FILE="$NIX_ROOT/var/nix/profiles/default/etc/profile.d/nix-daemon.sh"
readonly NIX_INSTALLED_NIX="@nix@"
readonly NIX_INSTALLED_CACERT="@cacert@"
readonly EXTRACTED_NIX_PATH="$(dirname "$0")"
readonly ROOT_HOME="/var/root"
if [ -t 0 ]; then
readonly IS_HEADLESS='no'
else
readonly IS_HEADLESS='yes'
fi
headless() {
if [ "$IS_HEADLESS" = "yes" ]; then
return 0
else
return 1
fi
}
contactme() {
echo "We'd love to help if you need it."
echo ""
echo "If you can, open an issue at https://github.com/nixos/nix/issues"
echo ""
echo "Or feel free to contact the team,"
echo " - on IRC #nixos on irc.freenode.net"
echo " - on twitter @nixos_org"
}
uninstall_directions() {
subheader "Uninstalling nix:"
local step=0
if [ -e "$PLIST_DEST" ]; then
step=$((step + 1))
cat <<EOF
$step. Delete $PLIST_DEST
sudo launchctl unload $PLIST_DEST
sudo rm $PLIST_DEST
EOF
fi
for profile_target in "${PROFILE_TARGETS[@]}"; do
if [ -e "$profile_target" ] && [ -e "$profile_target$PROFILE_BACKUP_SUFFIX" ]; then
step=$((step + 1))
cat <<EOF
$step. Restore $profile_target$PROFILE_BACKUP_SUFFIX back to $profile_target
sudo mv $profile_target$PROFILE_BACKUP_SUFFIX $profile_target
(after this one, you may need to re-open any terminals that were
opened while it existed.)
EOF
fi
done
step=$((step + 1))
cat <<EOF
$step. Delete the files Nix added to your system:
sudo rm -rf /etc/nix $NIX_ROOT $ROOT_HOME/.nix-profile $ROOT_HOME/.nix-defexpr $ROOT_HOME/.nix-channels $HOME/.nix-profile $HOME/.nix-defexpr $HOME/.nix-channels
and that is it.
EOF
}
nix_user_for_core() {
printf "nixbld%d" "$1"
}
nix_uid_for_core() {
echo $((NIX_FIRST_BUILD_UID + $1 - 1))
}
dsclattr() {
/usr/bin/dscl . -read "$1" \
| awk "/$2/ { print \$2 }"
}
_textout() {
echo -en "$1"
shift
if [ "$*" = "" ]; then
cat
else
echo "$@"
fi
echo -en "$ESC"
}
header() {
follow="---------------------------------------------------------"
header=$(echo "---- $* $follow$follow$follow" | head -c 80)
echo ""
_textout "$BLUE" "$header"
}
warningheader() {
follow="---------------------------------------------------------"
header=$(echo "---- $* $follow$follow$follow" | head -c 80)
echo ""
_textout "$RED" "$header"
}
subheader() {
echo ""
_textout "$BLUE_UL" "$*"
}
row() {
printf "$BOLD%s$ESC:\\t%s\\n" "$1" "$2"
}
task() {
echo ""
ok "~~> $1"
}
bold() {
echo "$BOLD$*$ESC"
}
ok() {
_textout "$GREEN" "$@"
}
warning() {
warningheader "warning!"
cat
echo ""
}
failure() {
header "oh no!"
_textout "$RED" "$@"
echo ""
_textout "$RED" "$(contactme)"
trap finish_cleanup EXIT
exit 1
}
ui_confirm() {
_textout "$GREEN$GREEN_UL" "$1"
if headless; then
echo "No TTY, assuming you would say yes :)"
return 0
fi
local prompt="[y/n] "
echo -n "$prompt"
while read -r y; do
if [ "$y" = "y" ]; then
echo ""
return 0
elif [ "$y" = "n" ]; then
echo ""
return 1
else
_textout "$RED" "Sorry, I didn't understand. I can only understand answers of y or n"
echo -n "$prompt"
fi
done
echo ""
return 1
}
__sudo() {
local expl="$1"
local cmd="$2"
shift
header "sudo execution"
echo "I am executing:"
echo ""
printf " $ sudo %s\\n" "$cmd"
echo ""
echo "$expl"
echo ""
return 0
}
_sudo() {
local expl="$1"
shift
if ! headless; then
__sudo "$expl" "$*"
fi
sudo "$@"
}
readonly SCRATCH=$(mktemp -d -t tmp.XXXXXXXXXX)
function finish_cleanup {
rm -rf "$SCRATCH"
}
function finish_fail {
finish_cleanup
failure <<EOF
Jeeze, something went wrong. If you can take all the output and open
an issue, we'd love to fix the problem so nobody else has this issue.
:(
EOF
}
trap finish_fail EXIT
function finish_success {
finish_cleanup
ok "Alright! We're done!"
cat <<EOF
Before Nix will work in your existing shells, you'll need to close
them and open them again. Other than that, you should be ready to go.
Try it! Open a new terminal, and type:
$ nix-shell -p nix-info --run "nix-info -m"
Thank you for using this installer. If you have any feedback, don't
hesitate:
$(contactme)
EOF
}
validate_starting_assumptions() {
if [ "$(uname -s)" != "Darwin" ]; then
failure "This script is for use with macOS!"
fi
if [ $EUID -eq 0 ]; then
failure <<EOF
Please do not run this script with root privileges. We will call sudo
when we need to.
EOF
fi
if type nix-env 2> /dev/null >&2; then
failure <<EOF
Nix already appears to be installed, and this tool assumes it is
_not_ yet installed.
$(uninstall_directions)
EOF
fi
if [ "${NIX_REMOTE:-}" != "" ]; then
failure <<EOF
For some reason, \$NIX_REMOTE is set. It really should not be set
before this installer runs, and it hints that Nix is currently
installed. Please delete the old Nix installation and start again.
Note: You might need to close your shell window and open a new shell
to clear the variable.
EOF
fi
if echo "${SSL_CERT_FILE:-}" | grep -qE "(nix/var/nix|nix-profile)"; then
failure <<EOF
It looks like \$SSL_CERT_FILE is set to a path that used to be part of
the old Nix installation. Please unset that variable and try again:
$ unset SSL_CERT_FILE
EOF
fi
for file in ~/.bash_profile ~/.bash_login ~/.profile ~/.zshenv ~/.zprofile ~/.zshrc ~/.zlogin; do
if [ -f "$file" ]; then
if grep -l "^[^#].*.nix-profile" "$file"; then
failure <<EOF
I found a reference to a ".nix-profile" in $file.
This has a high chance of breaking a new nix installation. It was most
likely put there by a previous Nix installer.
Please remove this reference and try running this again. You should
also look for similar references in:
- ~/.bash_profile
- ~/.bash_login
- ~/.profile
or other shell init files that you may have.
$(uninstall_directions)
EOF
fi
fi
done
if [ -d /nix ]; then
failure <<EOF
There are some relics of a previous installation of Nix at /nix, and
this scripts assumes Nix is _not_ yet installed. Please delete the old
Nix installation and start again.
$(uninstall_directions)
EOF
fi
if [ -d /etc/nix ]; then
failure <<EOF
There are some relics of a previous installation of Nix at /etc/nix, and
this scripts assumes Nix is _not_ yet installed. Please delete the old
Nix installation and start again.
$(uninstall_directions)
EOF
fi
for profile_target in "${PROFILE_TARGETS[@]}"; do
if [ -e "$profile_target$PROFILE_BACKUP_SUFFIX" ]; then
failure <<EOF
When this script runs, it backs up the current $profile_target to
$profile_target$PROFILE_BACKUP_SUFFIX. This backup file already exists, though.
Please follow these instructions to clean up the old backup file:
1. Copy $profile_target and $profile_target$PROFILE_BACKUP_SUFFIX to another place, just
in case.
2. Take care to make sure that $profile_target$PROFILE_BACKUP_SUFFIX doesn't look like
it has anything nix-related in it. If it does, something is probably
quite wrong. Please open an issue or get in touch immediately.
3. Take care to make sure that $profile_target doesn't look like it has
anything nix-related in it. If it does, and $profile_target _did not_,
run:
$ /usr/bin/sudo /bin/mv $profile_target$PROFILE_BACKUP_SUFFIX $profile_target
and try again.
EOF
fi
if grep -qi "nix" "$profile_target"; then
failure <<EOF
It looks like $profile_target already has some Nix configuration in
there. There should be no reason to run this again. If you're having
trouble, please open an issue.
EOF
fi
done
danger_paths=("$ROOT_HOME/.nix-defexpr" "$ROOT_HOME/.nix-channels" "$ROOT_HOME/.nix-profile")
for danger_path in "${danger_paths[@]}"; do
if _sudo "making sure that $danger_path doesn't exist" \
test -e "$danger_path"; then
failure <<EOF
I found a file at $danger_path, which is a relic of a previous
installation. You must first delete this file before continuing.
$(uninstall_directions)
EOF
fi
done
}
setup_report() {
header "hardware report"
row " Cores" "$CORES"
header "Nix config report"
row " Temp Dir" "$SCRATCH"
row " Nix Root" "$NIX_ROOT"
row " Build Users" "$NIX_USER_COUNT"
row " Build Group ID" "$NIX_BUILD_GROUP_ID"
row "Build Group Name" "$NIX_BUILD_GROUP_NAME"
if [ "${ALLOW_PREEXISTING_INSTALLATION:-}" != "" ]; then
row "Preexisting Install" "Allowed"
fi
subheader "build users:"
row " Username" "UID"
for i in $(seq 1 "$NIX_USER_COUNT"); do
row " $(nix_user_for_core "$i")" "$(nix_uid_for_core "$i")"
done
echo ""
}
create_build_group() {
local primary_group_id
task "Setting up the build group $NIX_BUILD_GROUP_NAME"
if ! /usr/bin/dscl . -read "/Groups/$NIX_BUILD_GROUP_NAME" > /dev/null 2>&1; then
_sudo "Create the Nix build group, $NIX_BUILD_GROUP_NAME" \
/usr/sbin/dseditgroup -o create \
-r "Nix build group for nix-daemon" \
-i "$NIX_BUILD_GROUP_ID" \
"$NIX_BUILD_GROUP_NAME" >&2
row " Created" "Yes"
else
primary_group_id=$(dsclattr "/Groups/$NIX_BUILD_GROUP_NAME" "PrimaryGroupID")
if [ "$primary_group_id" -ne "$NIX_BUILD_GROUP_ID" ]; then
failure <<EOF
It seems the build group $NIX_BUILD_GROUP_NAME already exists, but
with the UID $primary_group_id. This script can't really handle
that right now, so I'm going to give up.
You can fix this by editing this script and changing the
NIX_BUILD_GROUP_ID variable near the top to from $NIX_BUILD_GROUP_ID
to $primary_group_id and re-run.
EOF
else
row " Exists" "Yes"
fi
fi
}
create_build_user_for_core() {
local coreid
local username
local uid
coreid="$1"
username=$(nix_user_for_core "$coreid")
uid=$(nix_uid_for_core "$coreid")
dsclpath="/Users/$username"
task "Setting up the build user $username"
if ! /usr/bin/dscl . -read "$dsclpath" > /dev/null 2>&1; then
_sudo "Creating the Nix build user, $username" \
/usr/bin/dscl . create "$dsclpath" \
UniqueID "${uid}"
row " Created" "Yes"
else
actual_uid=$(dsclattr "$dsclpath" "UniqueID")
if [ "$actual_uid" -ne "$uid" ]; then
failure <<EOF
It seems the build user $username already exists, but with the UID
with the UID $actual_uid. This script can't really handle that right
now, so I'm going to give up.
If you already created the users and you know they start from
$actual_uid and go up from there, you can edit this script and change
NIX_FIRST_BUILD_UID near the top of the file to $actual_uid and try
again.
EOF
else
row " Exists" "Yes"
fi
fi
if [ "$(dsclattr "$dsclpath" "IsHidden")" = "1" ]; then
row " IsHidden" "Yes"
else
_sudo "in order to make $username a hidden user" \
/usr/bin/dscl . -create "$dsclpath" "IsHidden" "1"
row " IsHidden" "Yes"
fi
if [ "$(dsclattr "$dsclpath" "NFSHomeDirectory")" = "/var/empty" ]; then
row " NFSHomeDirectory" "/var/empty"
else
_sudo "in order to give $username a safe home directory" \
/usr/bin/dscl . -create "$dsclpath" "NFSHomeDirectory" "/var/empty"
row " NFSHomeDirectory" "/var/empty"
fi
if [ "$(dsclattr "$dsclpath" "RealName")" = "Nix build user $coreid" ]; then
row " RealName" "Nix build user $coreid"
else
_sudo "in order to give $username a useful name" \
/usr/bin/dscl . -create "$dsclpath" "RealName" "Nix build user $coreid"
row " RealName" "Nix build user $coreid"
fi
if [ "$(dsclattr "$dsclpath" "UserShell")" = "/sbin/nologin" ]; then
row " Logins Disabled" "Yes"
else
_sudo "in order to prevent $username from logging in" \
/usr/bin/dscl . -create "$dsclpath" "UserShell" "/sbin/nologin"
row " Logins Disabled" "Yes"
fi
if dseditgroup -o checkmember -m "$username" "$NIX_BUILD_GROUP_NAME" > /dev/null 2>&1 ; then
row " Member of $NIX_BUILD_GROUP_NAME" "Yes"
else
_sudo "Add $username to the $NIX_BUILD_GROUP_NAME group"\
/usr/sbin/dseditgroup -o edit -t user \
-a "$username" "$NIX_BUILD_GROUP_NAME"
row " Member of $NIX_BUILD_GROUP_NAME" "Yes"
fi
if [ "$(dsclattr "$dsclpath" "PrimaryGroupID")" = "$NIX_BUILD_GROUP_ID" ]; then
row " PrimaryGroupID" "$NIX_BUILD_GROUP_ID"
else
_sudo "to let the nix daemon use this user for builds (this might seem redundant, but there are two concepts of group membership)" \
/usr/bin/dscl . -create "$dsclpath" "PrimaryGroupID" "$NIX_BUILD_GROUP_ID"
row " PrimaryGroupID" "$NIX_BUILD_GROUP_ID"
fi
}
create_build_users() {
for i in $(seq 1 "$NIX_USER_COUNT"); do
create_build_user_for_core "$i"
done
}
create_directories() {
_sudo "to make the basic directory structure of Nix (part 1)" \
mkdir -pv -m 0755 /nix /nix/var /nix/var/log /nix/var/log/nix /nix/var/log/nix/drvs /nix/var/nix{,/db,/gcroots,/profiles,/temproots,/userpool}
_sudo "to make the basic directory structure of Nix (part 2)" \
mkdir -pv -m 1777 /nix/var/nix/{gcroots,profiles}/per-user
_sudo "to make the basic directory structure of Nix (part 3)" \
mkdir -pv -m 1775 /nix/store
_sudo "to make the basic directory structure of Nix (part 4)" \
chgrp "$NIX_BUILD_GROUP_NAME" /nix/store
_sudo "to set up the root user's profile (part 1)" \
mkdir -pv -m 0755 /nix/var/nix/profiles/per-user/root
_sudo "to set up the root user's profile (part 2)" \
mkdir -pv -m 0700 "$ROOT_HOME/.nix-defexpr"
_sudo "to place the default nix daemon configuration (part 1)" \
mkdir -pv -m 0555 /etc/nix
}
place_channel_configuration() {
echo "https://nixos.org/channels/nixpkgs-unstable nixpkgs" > "$SCRATCH/.nix-channels"
_sudo "to set up the default system channel (part 1)" \
install -m 0664 "$SCRATCH/.nix-channels" "$ROOT_HOME/.nix-channels"
}
welcome_to_nix() {
ok "Welcome to the Multi-User Nix Installation"
cat <<EOF
This installation tool will set up your computer with the Nix package
manager. This will happen in a few stages:
1. Make sure your computer doesn't already have Nix. If it does, I
will show you instructions on how to clean up your old one.
2. Show you what we are going to install and where. Then we will ask
if you are ready to continue.
3. Create the system users and groups that the Nix daemon uses to run
builds.
4. Perform the basic installation of the Nix files daemon.
5. Configure your shell to import special Nix Profile files, so you
can use Nix.
6. Start the Nix daemon.
EOF
if ui_confirm "Would you like to see a more detailed list of what we will do?"; then
cat <<EOF
We will:
- make sure your computer doesn't already have Nix files
(if it does, I will tell you how to clean them up.)
- create local users (see the list above for the users we'll make)
- create a local group ($NIX_BUILD_GROUP_NAME)
- install Nix in to $NIX_ROOT
- create a configuration file in /etc/nix
- set up the "default profile" by creating some Nix-related files in
$ROOT_HOME
EOF
for profile_target in "${PROFILE_TARGETS[@]}"; do
if [ -e "$profile_target" ]; then
cat <<EOF
- back up $profile_target to $profile_target$PROFILE_BACKUP_SUFFIX
- update $profile_target to include some Nix configuration
EOF
fi
done
cat <<EOF
- load and start a LaunchDaemon (at $PLIST_DEST) for nix-daemon
EOF
if ! ui_confirm "Ready to continue?"; then
failure <<EOF
Okay, maybe you would like to talk to the team.
EOF
fi
fi
}
chat_about_sudo() {
header "let's talk about sudo"
if headless; then
cat <<EOF
This script is going to call sudo a lot. Normally, it would show you
exactly what commands it is running and why. However, the script is
run in a headless fashion, like this:
$ curl https://nixos.org/nix/install | sh
or maybe in a CI pipeline. Because of that, we're going to skip the
verbose output in the interest of brevity.
If you would like to
see the output, try like this:
$ curl -o install-nix https://nixos.org/nix/install
$ sh ./install-nix
EOF
return 0
fi
cat <<EOF
This script is going to call sudo a lot. Every time we do, it'll
output exactly what it'll do, and why.
Just like this:
EOF
__sudo "to demonstrate how our sudo prompts look" \
echo "this is a sudo prompt"
cat <<EOF
This might look scary, but everything can be undone by running just a
few commands. We used to ask you to confirm each time sudo ran, but it
was too many times. Instead, I'll just ask you this one time:
EOF
if ui_confirm "Can we use sudo?"; then
ok "Yay! Thanks! Let's get going!"
else
failure <<EOF
That is okay, but we can't install.
EOF
fi
}
install_from_extracted_nix() {
(
cd "$EXTRACTED_NIX_PATH"
_sudo "to copy the basic Nix files to the new store at $NIX_ROOT/store" \
rsync -rlpt ./store/* "$NIX_ROOT/store/"
if [ -d "$NIX_INSTALLED_NIX" ]; then
echo " Alright! We have our first nix at $NIX_INSTALLED_NIX"
else
failure <<EOF
Something went wrong, and I didn't find Nix installed at
$NIX_INSTALLED_NIX.
EOF
fi
_sudo "to initialize the Nix Database" \
$NIX_INSTALLED_NIX/bin/nix-store --init
cat ./.reginfo \
| _sudo "to load data for the first time in to the Nix Database" \
"$NIX_INSTALLED_NIX/bin/nix-store" --load-db
echo " Just finished getting the nix database ready."
)
}
shell_source_lines() {
cat <<EOF
# Nix
if [ -e '$PROFILE_NIX_FILE' ]; then
. '$PROFILE_NIX_FILE'
fi
# End Nix
EOF
}
configure_shell_profile() {
for profile_target in "${PROFILE_TARGETS[@]}"; do
if [ -e "$profile_target" ]; then
_sudo "to back up your current $profile_target to $profile_target$PROFILE_BACKUP_SUFFIX" \
cp "$profile_target" "$profile_target$PROFILE_BACKUP_SUFFIX"
shell_source_lines \
| _sudo "extend your $profile_target with nix-daemon settings" \
tee -a "$profile_target"
fi
done
}
setup_default_profile() {
_sudo "to installing a bootstrapping Nix in to the default Profile" \
HOME=$ROOT_HOME "$NIX_INSTALLED_NIX/bin/nix-env" -i "$NIX_INSTALLED_NIX"
_sudo "to installing a bootstrapping SSL certificate just for Nix in to the default Profile" \
HOME=$ROOT_HOME "$NIX_INSTALLED_NIX/bin/nix-env" -i "$NIX_INSTALLED_CACERT"
_sudo "to update the default channel in the default profile" \
HOME=$ROOT_HOME NIX_SSL_CERT_FILE=/nix/var/nix/profiles/default/etc/ssl/certs/ca-bundle.crt "$NIX_INSTALLED_NIX/bin/nix-channel" --update nixpkgs
}
place_nix_configuration() {
cat <<EOF > "$SCRATCH/nix.conf"
build-users-group = $NIX_BUILD_GROUP_NAME
max-jobs = $NIX_USER_COUNT
cores = 1
sandbox = false
EOF
_sudo "to place the default nix daemon configuration (part 2)" \
install -m 0664 "$SCRATCH/nix.conf" /etc/nix/nix.conf
}
configure_nix_daemon_plist() {
_sudo "to set up the nix-daemon as a LaunchDaemon" \
ln -sfn "/nix/var/nix/profiles/default$PLIST_DEST" "$PLIST_DEST"
_sudo "to load the LaunchDaemon plist for nix-daemon" \
launchctl load /Library/LaunchDaemons/org.nixos.nix-daemon.plist
_sudo "to start the nix-daemon" \
launchctl start org.nixos.nix-daemon
}
main() {
welcome_to_nix
chat_about_sudo
if [ "${ALLOW_PREEXISTING_INSTALLATION:-}" = "" ]; then
validate_starting_assumptions
fi
setup_report
if ! ui_confirm "Ready to continue?"; then
ok "Alright, no changes have been made :)"
contactme
trap finish_cleanup EXIT
exit 1
fi
create_build_group
create_build_users
create_directories
place_channel_configuration
install_from_extracted_nix
configure_shell_profile
set +eu
. /etc/profile
set -eu
setup_default_profile
place_nix_configuration
configure_nix_daemon_plist
trap finish_success EXIT
}
main
+19 -9
View File
@@ -8,15 +8,8 @@ nix="@nix@"
cacert="@cacert@"
# macOS support for 10.10 or higher
if [[ "$(uname -s)" = "Darwin" && $(($(sw_vers -productVersion | cut -d '.' -f 2))) -lt 10 ]]; then
echo "$0: macOS $(sw_vers -productVersion) is not supported, upgrade to 10.10 or higher"
exit 1
fi
if ! [ -e "$self/.reginfo" ]; then
echo "$0: incomplete installer (.reginfo is missing)" >&2
exit 1
fi
if [ -z "$USER" ]; then
@@ -24,6 +17,23 @@ if [ -z "$USER" ]; then
exit 1
fi
if [ -z "$HOME" ]; then
echo "$0: \$HOME is not set" >&2
exit 1
fi
# macOS support for 10.10 or higher
if [ "$(uname -s)" = "Darwin" ]; then
if [ $(($(sw_vers -productVersion | cut -d '.' -f 2))) -lt 10 ]; then
echo "$0: macOS $(sw_vers -productVersion) is not supported, upgrade to 10.10 or higher"
exit 1
fi
printf '\e[1;31mSwitching to the Multi-User Darwin Installer\e[0m\n'
exec "$self/install-darwin-multi-user"
exit 0
fi
if [ "$(id -u)" -eq 0 ]; then
printf '\e[1;31mwarning: installing Nix as root is not supported by this script!\e[0m\n'
fi
@@ -34,13 +44,13 @@ if ! [ -e $dest ]; then
cmd="mkdir -m 0755 $dest && chown $USER $dest"
echo "directory $dest does not exist; creating it by running '$cmd' using sudo" >&2
if ! sudo sh -c "$cmd"; then
echo "$0: please manually run ‘$cmd’ as root to create $dest" >&2
echo "$0: please manually run '$cmd' as root to create $dest" >&2
exit 1
fi
fi
if ! [ -w $dest ]; then
echo "$0: directory $dest exists, but is not writable by you. This could indicate that another user has already performed a single-user installation of Nix on this system. If you wish to enable multi-user support see http://nixos.org/nix/manual/#ssec-multi-user. If you wish to continue with a single-user install for $USER please run ‘chown -R $USER $dest’ as root." >&2
echo "$0: directory $dest exists, but is not writable by you. This could indicate that another user has already performed a single-user installation of Nix on this system. If you wish to enable multi-user support see http://nixos.org/nix/manual/#ssec-multi-user. If you wish to continue with a single-user install for $USER please run 'chown -R $USER $dest' as root." >&2
exit 1
fi
+1
View File
@@ -8,5 +8,6 @@ noinst-scripts += $(nix_noinst_scripts)
profiledir = $(sysconfdir)/profile.d
$(eval $(call install-file-as, $(d)/nix-profile.sh, $(profiledir)/nix.sh, 0644))
$(eval $(call install-file-as, $(d)/nix-profile-daemon.sh, $(profiledir)/nix-daemon.sh, 0644))
clean-files += $(nix_noinst_scripts)
+54
View File
@@ -0,0 +1,54 @@
# Only execute this file once per shell.
if [ -n "$__ETC_PROFILE_NIX_SOURCED" ]; then return; fi
__ETC_PROFILE_NIX_SOURCED=1
# Set up secure multi-user builds: non-root users build through the
# Nix daemon.
if [ "$USER" != root -o ! -w @localstatedir@/nix/db ]; then
export NIX_REMOTE=daemon
fi
export NIX_USER_PROFILE_DIR="@localstatedir@/nix/profiles/per-user/$USER"
export NIX_PROFILES="@localstatedir@/nix/profiles/default $HOME/.nix-profile"
# Set up the per-user profile.
mkdir -m 0755 -p $NIX_USER_PROFILE_DIR
if ! test -O "$NIX_USER_PROFILE_DIR"; then
echo "WARNING: bad ownership on $NIX_USER_PROFILE_DIR" >&2
fi
if test -w $HOME; then
if ! test -L $HOME/.nix-profile; then
if test "$USER" != root; then
ln -s $NIX_USER_PROFILE_DIR/profile $HOME/.nix-profile
else
# Root installs in the system-wide profile by default.
ln -s @localstatedir@/nix/profiles/default $HOME/.nix-profile
fi
fi
# Subscribe the root user to the NixOS channel by default.
if [ "$USER" = root -a ! -e $HOME/.nix-channels ]; then
echo "https://nixos.org/channels/nixpkgs-unstable nixpkgs" > $HOME/.nix-channels
fi
# Create the per-user garbage collector roots directory.
NIX_USER_GCROOTS_DIR=@localstatedir@/nix/gcroots/per-user/$USER
mkdir -m 0755 -p $NIX_USER_GCROOTS_DIR
if ! test -O "$NIX_USER_GCROOTS_DIR"; then
echo "WARNING: bad ownership on $NIX_USER_GCROOTS_DIR" >&2
fi
# Set up a default Nix expression from which to install stuff.
if [ ! -e $HOME/.nix-defexpr -o -L $HOME/.nix-defexpr ]; then
rm -f $HOME/.nix-defexpr
mkdir -p $HOME/.nix-defexpr
if [ "$USER" != root ]; then
ln -s @localstatedir@/nix/profiles/per-user/root/channels $HOME/.nix-defexpr/channels_root
fi
fi
fi
export NIX_SSL_CERT_FILE="@localstatedir@/nix/profiles/default/etc/ssl/certs/ca-bundle.crt"
export NIX_PATH="@localstatedir@/nix/profiles/per-user/root/channels"
export PATH="$HOME/.nix-profile/bin:$HOME/.nix-profile/lib/kde4/libexec:@localstatedir@/nix/profiles/default/bin:@localstatedir@/nix/profiles/default:@localstatedir@/nix/profiles/default/lib/kde4/libexec:$PATH"
+2 -8
View File
@@ -60,12 +60,6 @@ if [ -n "$HOME" ] && [ -n "$USER" ]; then
# This part should be kept in sync with nixpkgs:nixos/modules/programs/environment.nix
NIX_PROFILES="@localstatedir@/nix/profiles/default $NIX_USER_PROFILE_DIR"
for i in $NIX_PROFILES; do
if [ -d "$i/lib/aspell" ]; then
export ASPELL_CONF="dict-dir $i/lib/aspell"
fi
done
# Set $NIX_SSL_CERT_FILE so that Nixpkgs applications like curl work.
if [ -e /etc/ssl/certs/ca-certificates.crt ]; then # NixOS, Ubuntu, Debian, Gentoo, Arch
export NIX_SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt
@@ -81,10 +75,10 @@ if [ -n "$HOME" ] && [ -n "$USER" ]; then
export NIX_SSL_CERT_FILE="$NIX_LINK/etc/ca-bundle.crt"
fi
if [ -n ${MANPATH} ]; then
if [ -n "${MANPATH}" ]; then
export MANPATH="$NIX_LINK/share/man:$MANPATH"
fi
export PATH="$NIX_LINK/bin:$NIX_LINK/sbin:$__savedpath"
export PATH="$NIX_LINK/bin:$__savedpath"
unset __savedpath NIX_LINK NIX_USER_PROFILE_DIR NIX_PROFILES
fi
+19 -10
View File
@@ -2,11 +2,14 @@
with import <nixpkgs> {};
with import ./release-common.nix { inherit pkgs; };
(if useClang then clangStdenv else stdenv).mkDerivation {
name = "nix";
buildInputs =
[ curl bison flex perl libxml2 libxslt bzip2 xz
[ curl bison flex libxml2 libxslt
bzip2 xz brotli
pkgconfig sqlite libsodium boehmgc
docbook5 docbook5_xsl
autoconf-archive
@@ -15,15 +18,18 @@ with import <nixpkgs> {};
customMemoryManagement = false;
})
autoreconfHook
perlPackages.DBDSQLite
];
configureFlags =
[ "--disable-init-state"
"--enable-gc"
"--with-dbi=${perlPackages.DBI}/${perl.libPrefix}"
"--with-dbd-sqlite=${perlPackages.DBDSQLite}/${perl.libPrefix}"
];
# For nix-perl
perl
perlPackages.DBDSQLite
# Tests
git
mercurial
]
++ lib.optional stdenv.isLinux libseccomp;
inherit configureFlags;
enableParallelBuilding = true;
@@ -31,6 +37,9 @@ with import <nixpkgs> {};
shellHook =
''
configureFlags+=" --prefix=$(pwd)/inst"
export prefix=$(pwd)/inst
configureFlags+=" --prefix=$prefix"
PKG_CONFIG_PATH=$prefix/lib/pkgconfig:$PKG_CONFIG_PATH
PATH=$prefix/bin:$PATH
'';
}
+7
View File
@@ -37,6 +37,13 @@ namespace {
os.str(emptyStr);
}
void do_pad( std::string & s,
std::streamsize w,
const char c,
std::ios::fmtflags f,
bool center)
__attribute__ ((unused));
void do_pad( std::string & s,
std::streamsize w,
const char c,
+119 -151
View File
@@ -9,160 +9,96 @@
#include <sys/time.h>
#endif
#include "machines.hh"
#include "shared.hh"
#include "pathlocks.hh"
#include "globals.hh"
#include "serialise.hh"
#include "store-api.hh"
#include "derivations.hh"
#include "local-store.hh"
using namespace nix;
using std::cerr;
using std::cin;
static void handle_alarm(int sig) {
static void handleAlarm(int sig) {
}
class machine {
const std::set<string> supportedFeatures;
const std::set<string> mandatoryFeatures;
public:
const string hostName;
const std::vector<string> systemTypes;
const string sshKey;
const unsigned long long maxJobs;
const unsigned long long speedFactor;
bool enabled;
bool allSupported(const std::set<string> & features) const {
return std::all_of(features.begin(), features.end(),
[&](const string & feature) {
return supportedFeatures.count(feature) ||
mandatoryFeatures.count(feature);
});
}
bool mandatoryMet(const std::set<string> & features) const {
return std::all_of(mandatoryFeatures.begin(), mandatoryFeatures.end(),
[&](const string & feature) {
return features.count(feature);
});
}
machine(decltype(hostName) hostName,
decltype(systemTypes) systemTypes,
decltype(sshKey) sshKey,
decltype(maxJobs) maxJobs,
decltype(speedFactor) speedFactor,
decltype(supportedFeatures) supportedFeatures,
decltype(mandatoryFeatures) mandatoryFeatures) :
supportedFeatures{std::move(supportedFeatures)},
mandatoryFeatures{std::move(mandatoryFeatures)},
hostName{std::move(hostName)},
systemTypes{std::move(systemTypes)},
sshKey{std::move(sshKey)},
maxJobs{std::move(maxJobs)},
speedFactor{speedFactor == 0 ? 1 : std::move(speedFactor)},
enabled{true} {};
};;
static std::vector<machine> read_conf()
std::string escapeUri(std::string uri)
{
auto conf = getEnv("NIX_REMOTE_SYSTEMS", SYSCONFDIR "/nix/machines");
auto machines = std::vector<machine>{};
auto lines = std::vector<string>{};
try {
lines = tokenizeString<std::vector<string>>(readFile(conf), "\n");
} catch (const SysError & e) {
if (e.errNo != ENOENT)
throw;
}
for (auto line : lines) {
chomp(line);
line.erase(std::find(line.begin(), line.end(), '#'), line.end());
if (line.empty()) {
continue;
}
auto tokens = tokenizeString<std::vector<string>>(line);
auto sz = tokens.size();
if (sz < 4) {
throw new FormatError(format("Bad machines.conf file %1%")
% conf);
}
machines.emplace_back(tokens[0],
tokenizeString<std::vector<string>>(tokens[1], ","),
tokens[2],
stoull(tokens[3]),
sz >= 5 ? stoull(tokens[4]) : 1LL,
sz >= 6 ?
tokenizeString<std::set<string>>(tokens[5], ",") :
std::set<string>{},
sz >= 7 ?
tokenizeString<std::set<string>>(tokens[6], ",") :
std::set<string>{});
}
return machines;
std::replace(uri.begin(), uri.end(), '/', '_');
return uri;
}
static string currentLoad;
static AutoCloseFD openSlotLock(const machine & m, unsigned long long slot)
static AutoCloseFD openSlotLock(const Machine & m, unsigned long long slot)
{
std::ostringstream fn_stream(currentLoad, std::ios_base::ate | std::ios_base::out);
fn_stream << "/";
for (auto t : m.systemTypes) {
fn_stream << t << "-";
}
fn_stream << m.hostName << "-" << slot;
return openLockFile(fn_stream.str(), true);
return openLockFile(fmt("%s/%s-%d", currentLoad, escapeUri(m.storeUri), slot), true);
}
static char display_env[] = "DISPLAY=";
static char ssh_env[] = "SSH_ASKPASS=";
int main (int argc, char * * argv)
{
return handleExceptions(argv[0], [&]() {
initNix();
logger = makeJSONLogger(*logger);
/* Ensure we don't get any SSH passphrase or host key popups. */
if (putenv(display_env) == -1 ||
putenv(ssh_env) == -1) {
throw SysError("Setting SSH env vars");
}
unsetenv("DISPLAY");
unsetenv("SSH_ASKPASS");
if (argc != 4) {
if (argc != 2)
throw UsageError("called without required arguments");
verbosity = (Verbosity) std::stoll(argv[1]);
FdSource source(STDIN_FILENO);
/* Read the parent's settings. */
while (readInt(source)) {
auto name = readString(source);
auto value = readString(source);
settings.set(name, value);
}
auto store = openStore();
settings.maxBuildJobs.set("1"); // hack to make tests with local?root= work
auto localSystem = argv[1];
settings.maxSilentTime = stoull(string(argv[2]));
settings.buildTimeout = stoull(string(argv[3]));
initPlugins();
currentLoad = getEnv("NIX_CURRENT_LOAD", "/run/nix/current-load");
auto store = openStore().cast<LocalStore>();
/* It would be more appropriate to use $XDG_RUNTIME_DIR, since
that gets cleared on reboot, but it wouldn't work on macOS. */
currentLoad = store->stateDir + "/current-load";
std::shared_ptr<Store> sshStore;
AutoCloseFD bestSlotLock;
auto machines = read_conf();
auto machines = getMachines();
debug("got %d remote builders", machines.size());
if (machines.empty()) {
std::cerr << "# decline-permanently\n";
return;
}
string drvPath;
string hostName;
for (string line; getline(cin, line);) {
auto tokens = tokenizeString<std::vector<string>>(line);
auto sz = tokens.size();
if (sz != 3 && sz != 4) {
throw Error(format("invalid build hook line %1%") % line);
}
auto amWilling = tokens[0] == "1";
auto neededSystem = tokens[1];
drvPath = tokens[2];
auto requiredFeatures = sz == 3 ?
std::set<string>{} :
tokenizeString<std::set<string>>(tokens[3], ",");
auto canBuildLocally = amWilling && (neededSystem == localSystem);
string storeUri;
while (true) {
try {
auto s = readString(source);
if (s != "try") return;
} catch (EndOfFile &) { return; }
auto amWilling = readInt(source);
auto neededSystem = readString(source);
source >> drvPath;
auto requiredFeatures = readStrings<std::set<std::string>>(source);
auto canBuildLocally = amWilling && (neededSystem == settings.thisSystem);
/* Error ignored here, will be caught later */
mkdir(currentLoad.c_str(), 0777);
@@ -174,9 +110,11 @@ int main (int argc, char * * argv)
bool rightType = false;
machine * bestMachine = nullptr;
Machine * bestMachine = nullptr;
unsigned long long bestLoad = 0;
for (auto & m : machines) {
debug("considering building on remote machine '%s'", m.storeUri);
if (m.enabled && std::find(m.systemTypes.begin(),
m.systemTypes.end(),
neededSystem) != m.systemTypes.end() &&
@@ -221,11 +159,10 @@ int main (int argc, char * * argv)
}
if (!bestSlotLock) {
if (rightType && !canBuildLocally) {
cerr << "# postpone\n";
} else {
cerr << "# decline\n";
}
if (rightType && !canBuildLocally)
std::cerr << "# postpone\n";
else
std::cerr << "# decline\n";
break;
}
@@ -238,47 +175,78 @@ int main (int argc, char * * argv)
lock = -1;
try {
sshStore = openStore("ssh://" + bestMachine->hostName + "?key=" + bestMachine->sshKey);
hostName = bestMachine->hostName;
Activity act(*logger, lvlTalkative, actUnknown, fmt("connecting to '%s'", bestMachine->storeUri));
Store::Params storeParams;
if (hasPrefix(bestMachine->storeUri, "ssh://")) {
storeParams["max-connections"] ="1";
storeParams["log-fd"] = "4";
if (bestMachine->sshKey != "")
storeParams["ssh-key"] = bestMachine->sshKey;
}
sshStore = openStore(bestMachine->storeUri, storeParams);
sshStore->connect();
storeUri = bestMachine->storeUri;
} catch (std::exception & e) {
cerr << e.what() << '\n';
cerr << "unable to open SSH connection to ‘" << bestMachine->hostName << "’, trying other available machines...\n";
printError("unable to open SSH connection to '%s': %s; trying other available machines...",
bestMachine->storeUri, e.what());
bestMachine->enabled = false;
continue;
}
goto connected;
}
}
connected:
cerr << "# accept\n";
string line;
if (!getline(cin, line)) {
throw Error("hook caller didn't send inputs");
std::cerr << "# accept\n" << storeUri << "\n";
auto inputs = readStrings<PathSet>(source);
auto outputs = readStrings<PathSet>(source);
AutoCloseFD uploadLock = openLockFile(currentLoad + "/" + escapeUri(storeUri) + ".upload-lock", true);
{
Activity act(*logger, lvlTalkative, actUnknown, fmt("waiting for the upload lock to '%s'", storeUri));
auto old = signal(SIGALRM, handleAlarm);
alarm(15 * 60);
if (!lockFile(uploadLock.get(), ltWrite, true))
printError("somebody is hogging the upload lock for '%s', continuing...");
alarm(0);
signal(SIGALRM, old);
}
auto inputs = tokenizeString<std::list<string>>(line);
if (!getline(cin, line)) {
throw Error("hook caller didn't send outputs");
auto substitute = settings.buildersUseSubstitutes ? Substitute : NoSubstitute;
{
Activity act(*logger, lvlTalkative, actUnknown, fmt("copying dependencies to '%s'", storeUri));
copyPaths(store, ref<Store>(sshStore), inputs, NoRepair, NoCheckSigs, substitute);
}
auto outputs = tokenizeString<Strings>(line);
AutoCloseFD uploadLock = openLockFile(currentLoad + "/" + hostName + ".upload-lock", true);
auto old = signal(SIGALRM, handle_alarm);
alarm(15 * 60);
if (!lockFile(uploadLock.get(), ltWrite, true)) {
cerr << "somebody is hogging the upload lock for " << hostName << ", continuing...\n";
}
alarm(0);
signal(SIGALRM, old);
copyPaths(store, ref<Store>(sshStore), inputs);
uploadLock = -1;
cerr << "building ‘" << drvPath << "’ on ‘" << hostName << "’\n";
sshStore->buildDerivation(drvPath, readDerivation(drvPath));
BasicDerivation drv(readDerivation(store->realStoreDir + "/" + baseNameOf(drvPath)));
drv.inputSrcs = inputs;
std::remove_if(outputs.begin(), outputs.end(), [=](const Path & path) { return store->isValidPath(path); });
if (!outputs.empty()) {
setenv("NIX_HELD_LOCKS", concatStringsSep(" ", outputs).c_str(), 1); /* FIXME: ugly */
copyPaths(ref<Store>(sshStore), store, outputs);
auto result = sshStore->buildDerivation(drvPath, drv);
if (!result.success())
throw Error("build of '%s' on '%s' failed: %s", drvPath, storeUri, result.errorMsg);
PathSet missing;
for (auto & path : outputs)
if (!store->isValidPath(path)) missing.insert(path);
if (!missing.empty()) {
Activity act(*logger, lvlTalkative, actUnknown, fmt("copying outputs from '%s'", storeUri));
store->locksHeld.insert(missing.begin(), missing.end()); /* FIXME: ugly */
copyPaths(ref<Store>(sshStore), store, missing, NoRepair, NoCheckSigs, substitute);
}
return;
});
}
-2
View File
@@ -7,5 +7,3 @@ build-remote_INSTALL_DIR := $(libexecdir)/nix
build-remote_LIBS = libmain libutil libformat libstore
build-remote_SOURCES := $(d)/build-remote.cc
build-remote_CXXFLAGS = -DSYSCONFDIR="\"$(sysconfdir)\""
+14 -13
View File
@@ -12,7 +12,7 @@ static bool isDirectory (const Path & path)
{
struct stat st;
if (stat(path.c_str(), &st) == -1)
throw SysError(format("getting status of ‘%1%’") % path);
throw SysError(format("getting status of '%1%'") % path);
return S_ISDIR(st.st_mode);
}
@@ -53,18 +53,18 @@ static void createLinks(const Path & srcDir, const Path & dstDir, int priority)
} else if (S_ISLNK(dstSt.st_mode)) {
auto target = readLink(dstFile);
if (!isDirectory(target))
throw Error(format("collision between ‘%1%’ and non-directory ‘%2%’")
throw Error(format("collision between '%1%' and non-directory '%2%'")
% srcFile % target);
if (unlink(dstFile.c_str()) == -1)
throw SysError(format("unlinking ‘%1%’") % dstFile);
throw SysError(format("unlinking '%1%'") % dstFile);
if (mkdir(dstFile.c_str(), 0755) == -1)
throw SysError(format("creating directory ‘%1%’"));
throw SysError(format("creating directory '%1%'"));
createLinks(target, dstFile, priorities[dstFile]);
createLinks(srcFile, dstFile, priority);
continue;
}
} else if (errno != ENOENT)
throw SysError(format("getting status of ‘%1%’") % dstFile);
throw SysError(format("getting status of '%1%'") % dstFile);
} else {
struct stat dstSt;
auto res = lstat(dstFile.c_str(), &dstSt);
@@ -74,17 +74,18 @@ static void createLinks(const Path & srcDir, const Path & dstDir, int priority)
auto prevPriority = priorities[dstFile];
if (prevPriority == priority)
throw Error(format(
"collision between ‘%1%’ and ‘%2%’; "
"use ‘nix-env --set-flag priority NUMBER PKGNAME’ "
"packages '%1%' and '%2%' have the same priority %3%; "
"use 'nix-env --set-flag priority NUMBER INSTALLED_PKGNAME' "
"to change the priority of one of the conflicting packages"
) % srcFile % target);
" (0 being the highest priority)"
) % srcFile % target % priority);
if (prevPriority < priority)
continue;
if (unlink(dstFile.c_str()) == -1)
throw SysError(format("unlinking ‘%1%’") % dstFile);
throw SysError(format("unlinking '%1%'") % dstFile);
}
} else if (errno != ENOENT)
throw SysError(format("getting status of ‘%1%’") % dstFile);
throw SysError(format("getting status of '%1%'") % dstFile);
}
createSymlink(srcFile, dstFile);
priorities[dstFile] = priority;
@@ -112,11 +113,11 @@ static void addPkg(const Path & pkgDir, int priority)
if (!fd) {
if (errno == ENOENT)
return;
throw SysError(format("opening ‘%1%’") % propagatedFN);
throw SysError(format("opening '%1%'") % propagatedFN);
}
propagated = readLine(fd.get());
propagated = readFile(fd.get());
}
for (const auto & p : tokenizeString<std::vector<string>>(propagated, " "))
for (const auto & p : tokenizeString<std::vector<string>>(propagated, " \n"))
if (done.find(p) == done.end())
postponed.insert(p);
}
+7 -7
View File
@@ -19,7 +19,7 @@ static Strings parseAttrPath(const string & s)
++i;
while (1) {
if (i == s.end())
throw Error(format("missing closing quote in selection path ‘%1%’") % s);
throw Error(format("missing closing quote in selection path '%1%'") % s);
if (*i == '"') break;
cur.push_back(*i++);
}
@@ -38,7 +38,7 @@ Value * findAlongAttrPath(EvalState & state, const string & attrPath,
Strings tokens = parseAttrPath(attrPath);
Error attrError =
Error(format("attribute selection path ‘%1%’ does not match expression") % attrPath);
Error(format("attribute selection path '%1%' does not match expression") % attrPath);
Value * v = &vIn;
@@ -62,15 +62,15 @@ Value * findAlongAttrPath(EvalState & state, const string & attrPath,
if (v->type != tAttrs)
throw TypeError(
format("the expression selected by the selection path ‘%1%’ should be a set but is %2%")
format("the expression selected by the selection path '%1%' should be a set but is %2%")
% attrPath % showType(*v));
if (attr.empty())
throw Error(format("empty attribute name in selection path ‘%1%’") % attrPath);
throw Error(format("empty attribute name in selection path '%1%'") % attrPath);
Bindings::iterator a = v->attrs->find(state.symbols.create(attr));
if (a == v->attrs->end())
throw Error(format("attribute ‘%1%’ in selection path ‘%2%’ not found") % attr % attrPath);
throw Error(format("attribute '%1%' in selection path '%2%' not found") % attr % attrPath);
v = &*a->value;
}
@@ -78,11 +78,11 @@ Value * findAlongAttrPath(EvalState & state, const string & attrPath,
if (!v->isList())
throw TypeError(
format("the expression selected by the selection path ‘%1%’ should be a list but is %2%")
format("the expression selected by the selection path '%1%' should be a list but is %2%")
% attrPath % showType(*v));
if (attrIndex >= v->listSize())
throw Error(format("list index %1% in selection path ‘%2%’ is out of range") % attrIndex % attrPath);
throw Error(format("list index %1% in selection path '%2%' is out of range") % attrIndex % attrPath);
v = v->listElems()[attrIndex];
}
+2 -1
View File
@@ -7,13 +7,14 @@
namespace nix {
/* Note: Various places expect the allocated memory to be zeroed. */
static void * allocBytes(size_t n)
{
void * p;
#if HAVE_BOEHMGC
p = GC_malloc(n);
#else
p = malloc(n);
p = calloc(n, 1);
#endif
if (!p) throw std::bad_alloc();
return p;
+1 -1
View File
@@ -83,7 +83,7 @@ public:
for (size_t n = 0; n < size_; n++)
res.emplace_back(&attrs[n]);
std::sort(res.begin(), res.end(), [](const Attr * a, const Attr * b) {
return (string) a->name < (string) b->name;
return (const string &) a->name < (const string &) b->name;
});
return res;
}
+57
View File
@@ -0,0 +1,57 @@
#include "common-eval-args.hh"
#include "shared.hh"
#include "download.hh"
#include "util.hh"
#include "eval.hh"
namespace nix {
MixEvalArgs::MixEvalArgs()
{
mkFlag()
.longName("arg")
.description("argument to be passed to Nix functions")
.labels({"name", "expr"})
.handler([&](std::vector<std::string> ss) { autoArgs[ss[0]] = 'E' + ss[1]; });
mkFlag()
.longName("argstr")
.description("string-valued argument to be passed to Nix functions")
.labels({"name", "string"})
.handler([&](std::vector<std::string> ss) { autoArgs[ss[0]] = 'S' + ss[1]; });
mkFlag()
.shortName('I')
.longName("include")
.description("add a path to the list of locations used to look up <...> file names")
.label("path")
.handler([&](std::string s) { searchPath.push_back(s); });
}
Bindings * MixEvalArgs::getAutoArgs(EvalState & state)
{
Bindings * res = state.allocBindings(autoArgs.size());
for (auto & i : autoArgs) {
Value * v = state.allocValue();
if (i.second[0] == 'E')
state.mkThunk_(*v, state.parseExprFromString(string(i.second, 1), absPath(".")));
else
mkString(*v, string(i.second, 1));
res->push_back(Attr(state.symbols.create(i.first), v));
}
res->sort();
return res;
}
Path lookupFileArg(EvalState & state, string s)
{
if (isUri(s))
return getDownloader()->downloadCached(state.store, s, true);
else if (s.size() > 2 && s.at(0) == '<' && s.at(s.size() - 1) == '>') {
Path p = s.substr(1, s.size() - 2);
return state.findFile(p);
} else
return absPath(s);
}
}
+26
View File
@@ -0,0 +1,26 @@
#pragma once
#include "args.hh"
namespace nix {
class Store;
class EvalState;
class Bindings;
struct MixEvalArgs : virtual Args
{
MixEvalArgs();
Bindings * getAutoArgs(EvalState & state);
Strings searchPath;
private:
std::map<std::string, std::string> autoArgs;
};
Path lookupFileArg(EvalState & state, string s);
}
-67
View File
@@ -1,67 +0,0 @@
#include "common-opts.hh"
#include "shared.hh"
#include "download.hh"
#include "util.hh"
namespace nix {
bool parseAutoArgs(Strings::iterator & i,
const Strings::iterator & argsEnd, std::map<string, string> & res)
{
string arg = *i;
if (arg != "--arg" && arg != "--argstr") return false;
UsageError error(format("‘%1%’ requires two arguments") % arg);
if (++i == argsEnd) throw error;
string name = *i;
if (++i == argsEnd) throw error;
string value = *i;
res[name] = (arg == "--arg" ? 'E' : 'S') + value;
return true;
}
Bindings * evalAutoArgs(EvalState & state, std::map<string, string> & in)
{
Bindings * res = state.allocBindings(in.size());
for (auto & i : in) {
Value * v = state.allocValue();
if (i.second[0] == 'E')
state.mkThunk_(*v, state.parseExprFromString(string(i.second, 1), absPath(".")));
else
mkString(*v, string(i.second, 1));
res->push_back(Attr(state.symbols.create(i.first), v));
}
res->sort();
return res;
}
bool parseSearchPathArg(Strings::iterator & i,
const Strings::iterator & argsEnd, Strings & searchPath)
{
if (*i != "-I") return false;
if (++i == argsEnd) throw UsageError("‘-I’ requires an argument");
searchPath.push_back(*i);
return true;
}
Path lookupFileArg(EvalState & state, string s)
{
if (isUri(s))
return getDownloader()->downloadCached(state.store, s, true);
else if (s.size() > 2 && s.at(0) == '<' && s.at(s.size() - 1) == '>') {
Path p = s.substr(1, s.size() - 2);
return state.findFile(p);
} else
return absPath(s);
}
}
-20
View File
@@ -1,20 +0,0 @@
#pragma once
#include "eval.hh"
namespace nix {
class Store;
/* Some common option parsing between nix-env and nix-instantiate. */
bool parseAutoArgs(Strings::iterator & i,
const Strings::iterator & argsEnd, std::map<string, string> & res);
Bindings * evalAutoArgs(EvalState & state, std::map<string, string> & in);
bool parseSearchPathArg(Strings::iterator & i,
const Strings::iterator & argsEnd, Strings & searchPath);
Path lookupFileArg(EvalState & state, string s);
}
+1 -1
View File
@@ -33,7 +33,7 @@ void EvalState::forceValue(Value & v, const Pos & pos)
v.type = tBlackhole;
//checkInterrupt();
expr->eval(*this, *env, v);
} catch (Error & e) {
} catch (...) {
v.type = tThunk;
v.thunk.env = env;
v.thunk.expr = expr;
+135 -53
View File
@@ -43,13 +43,14 @@ static char * dupString(const char * s)
}
/* Note: Various places expect the allocated memory to be zeroed. */
static void * allocBytes(size_t n)
{
void * p;
#if HAVE_BOEHMGC
p = GC_malloc(n);
#else
p = malloc(n);
p = calloc(n, 1);
#endif
if (!p) throw std::bad_alloc();
return p;
@@ -149,7 +150,7 @@ string showType(const Value & v)
switch (v.type) {
case tInt: return "an integer";
case tBool: return "a boolean";
case tString: return "a string";
case tString: return v.string.context ? "a string with context" : "a string";
case tPath: return "a path";
case tNull: return "null";
case tAttrs: return "a set";
@@ -202,7 +203,7 @@ void initGC()
GC_INIT();
GC_oom_fn = oomHandler;
GC_set_oom_fn(oomHandler);
/* Set the initial heap size to something fairly big (25% of
physical RAM, up to a maximum of 384 MiB) so that in most cases
@@ -293,21 +294,35 @@ EvalState::EvalState(const Strings & _searchPath, ref<Store> store)
, sWrong(symbols.create("wrong"))
, sStructuredAttrs(symbols.create("__structuredAttrs"))
, sBuilder(symbols.create("builder"))
, sArgs(symbols.create("args"))
, sOutputHash(symbols.create("outputHash"))
, sOutputHashAlgo(symbols.create("outputHashAlgo"))
, sOutputHashMode(symbols.create("outputHashMode"))
, repair(NoRepair)
, store(store)
, baseEnv(allocEnv(128))
, staticBaseEnv(false, 0)
{
countCalls = getEnv("NIX_COUNT_CALLS", "0") != "0";
restricted = settings.get("restrict-eval", false);
assert(gcInitialised);
/* Initialise the Nix expression search path. */
Strings paths = parseNixPath(getEnv("NIX_PATH", ""));
for (auto & i : _searchPath) addToSearchPath(i);
for (auto & i : paths) addToSearchPath(i);
addToSearchPath("nix=" + settings.nixDataDir + "/nix/corepkgs");
if (!settings.pureEval) {
Strings paths = parseNixPath(getEnv("NIX_PATH", ""));
for (auto & i : _searchPath) addToSearchPath(i);
for (auto & i : paths) addToSearchPath(i);
}
addToSearchPath("nix=" + canonPath(settings.nixDataDir + "/nix/corepkgs", true));
if (settings.restrictEval || settings.pureEval) {
allowedPaths = PathSet();
for (auto & i : searchPath) {
auto r = resolveSearchPathElem(i);
if (!r.first) continue;
allowedPaths->insert(r.second);
}
}
clearValue(vEmptySet);
vEmptySet.type = tAttrs;
@@ -325,36 +340,76 @@ EvalState::~EvalState()
Path EvalState::checkSourcePath(const Path & path_)
{
if (!restricted) return path_;
if (!allowedPaths) return path_;
bool found = false;
for (auto & i : *allowedPaths) {
if (isDirOrInDir(path_, i)) {
found = true;
break;
}
}
if (!found)
throw RestrictedPathError("access to path '%1%' is forbidden in restricted mode", path_);
/* Resolve symlinks. */
debug(format("checking access to ‘%s’") % path_);
debug(format("checking access to '%s'") % path_);
Path path = canonPath(path_, true);
for (auto & i : searchPath) {
auto r = resolveSearchPathElem(i);
if (!r.first) continue;
if (path == r.second || isInDir(path, r.second))
for (auto & i : *allowedPaths) {
if (isDirOrInDir(path, i))
return path;
}
/* To support import-from-derivation, allow access to anything in
the store. FIXME: only allow access to paths that have been
constructed by this evaluation. */
if (store->isInStore(path)) return path;
#if 0
/* Hack to support the chroot dependencies of corepkgs (see
corepkgs/config.nix.in). */
if (path == settings.nixPrefix && isStorePath(settings.nixPrefix))
return path;
#endif
throw RestrictedPathError(format("access to path ‘%1%’ is forbidden in restricted mode") % path_);
throw RestrictedPathError("access to path '%1%' is forbidden in restricted mode", path);
}
void EvalState::addConstant(const string & name, Value & v)
void EvalState::checkURI(const std::string & uri)
{
if (!settings.restrictEval) return;
/* 'uri' should be equal to a prefix, or in a subdirectory of a
prefix. Thus, the prefix https://github.co does not permit
access to https://github.com. Note: this allows 'http://' and
'https://' as prefixes for any http/https URI. */
for (auto & prefix : settings.allowedUris.get())
if (uri == prefix ||
(uri.size() > prefix.size()
&& prefix.size() > 0
&& hasPrefix(uri, prefix)
&& (prefix[prefix.size() - 1] == '/' || uri[prefix.size()] == '/')))
return;
/* If the URI is a path, then check it against allowedPaths as
well. */
if (hasPrefix(uri, "/")) {
checkSourcePath(uri);
return;
}
if (hasPrefix(uri, "file://")) {
checkSourcePath(std::string(uri, 7));
return;
}
throw RestrictedPathError("access to URI '%s' is forbidden in restricted mode", uri);
}
Path EvalState::toRealPath(const Path & path, const PathSet & context)
{
// FIXME: check whether 'path' is in 'context'.
return
!context.empty() && store->isInStore(path)
? store->toRealPath(path)
: path;
};
Value * EvalState::addConstant(const string & name, Value & v)
{
Value * v2 = allocValue();
*v2 = v;
@@ -362,12 +417,18 @@ void EvalState::addConstant(const string & name, Value & v)
baseEnv.values[baseEnvDispl++] = v2;
string name2 = string(name, 0, 2) == "__" ? string(name, 2) : name;
baseEnv.values[0]->attrs->push_back(Attr(symbols.create(name2), v2));
return v2;
}
void EvalState::addPrimOp(const string & name,
Value * EvalState::addPrimOp(const string & name,
unsigned int arity, PrimOpFun primOp)
{
if (arity == 0) {
Value v;
primOp(*this, noPos, nullptr, v);
return addConstant(name, v);
}
Value * v = allocValue();
string name2 = string(name, 0, 2) == "__" ? string(name, 2) : name;
Symbol sym = symbols.create(name2);
@@ -376,6 +437,7 @@ void EvalState::addPrimOp(const string & name,
staticBaseEnv.vars[symbols.create(name)] = baseEnvDispl;
baseEnv.values[baseEnvDispl++] = v;
baseEnv.values[0]->attrs->push_back(Attr(sym, v));
return v;
}
@@ -503,7 +565,7 @@ inline Value * EvalState::lookupVar(Env * env, const ExprVar & var, bool noEval)
return j->value;
}
if (!env->prevWith)
throwUndefinedVarError("undefined variable ‘%1%’ at %2%", var.name, var.pos);
throwUndefinedVarError("undefined variable '%1%' at %2%", var.name, var.pos);
for (unsigned int l = env->prevWith; l; --l, env = env->up) ;
}
}
@@ -525,9 +587,7 @@ Env & EvalState::allocEnv(unsigned int size)
Env * env = (Env *) allocBytes(sizeof(Env) + size * sizeof(Value *));
env->size = size;
/* Clear the values because maybeThunk() and lookupVar fromWith expect this. */
for (unsigned i = 0; i < size; ++i)
env->values[i] = 0;
/* We assume that env->values has been cleared by the allocator; maybeThunk() and lookupVar fromWith expect this. */
return *env;
}
@@ -628,8 +688,10 @@ Value * ExprPath::maybeThunk(EvalState & state, Env & env)
}
void EvalState::evalFile(const Path & path, Value & v)
void EvalState::evalFile(const Path & path_, Value & v)
{
auto path = checkSourcePath(path_);
FileEvalCache::iterator i;
if ((i = fileEvalCache.find(path)) != fileEvalCache.end()) {
v = i->second;
@@ -642,12 +704,12 @@ void EvalState::evalFile(const Path & path, Value & v)
return;
}
Activity act(*logger, lvlTalkative, format("evaluating file ‘%1%’") % path2);
printTalkative("evaluating file '%1%'", path2);
Expr * e = parseExprFromFile(checkSourcePath(path2));
try {
eval(e, v);
} catch (Error & e) {
addErrorPrefix(e, "while evaluating the file ‘%1%’:\n", path2);
addErrorPrefix(e, "while evaluating the file '%1%':\n", path2);
throw;
}
@@ -797,7 +859,7 @@ void ExprAttrs::eval(EvalState & state, Env & env, Value & v)
Symbol nameSym = state.symbols.create(nameVal.string.s);
Bindings::iterator j = v.attrs->find(nameSym);
if (j != v.attrs->end())
throwEvalError("dynamic attribute ‘%1%’ at %2% already defined at %3%", nameSym, i.pos, *j->pos);
throwEvalError("dynamic attribute '%1%' at %2% already defined at %3%", nameSym, i.pos, *j->pos);
i.valueExpr->setName(nameSym);
/* Keep sorted order so find can catch duplicates */
@@ -885,7 +947,7 @@ void ExprSelect::eval(EvalState & state, Env & env, Value & v)
} else {
state.forceAttrs(*vAttrs, pos);
if ((j = vAttrs->attrs->find(name)) == vAttrs->attrs->end())
throwEvalError("attribute ‘%1%’ missing, at %2%", name, pos);
throwEvalError("attribute '%1%' missing, at %2%", name, pos);
}
vAttrs = j->value;
pos2 = j->pos;
@@ -896,7 +958,7 @@ void ExprSelect::eval(EvalState & state, Env & env, Value & v)
} catch (Error & e) {
if (pos2 && pos2->file != state.sDerivationNix)
addErrorPrefix(e, "while evaluating the attribute ‘%1%’ at %2%:\n",
addErrorPrefix(e, "while evaluating the attribute '%1%' at %2%:\n",
showAttrPath(state, env, attrPath), *pos2);
throw;
}
@@ -1038,7 +1100,7 @@ void EvalState::callFunction(Value & fun, Value & arg, Value & v, const Pos & po
for (auto & i : lambda.formals->formals) {
Bindings::iterator j = arg.attrs->find(i.name);
if (j == arg.attrs->end()) {
if (!i.def) throwTypeError("%1% called without required argument ‘%2%’, at %3%",
if (!i.def) throwTypeError("%1% called without required argument '%2%', at %3%",
lambda, i.name, pos);
env2.values[displ++] = i.def->maybeThunk(*this, env2);
} else {
@@ -1054,7 +1116,7 @@ void EvalState::callFunction(Value & fun, Value & arg, Value & v, const Pos & po
user. */
for (auto & i : *arg.attrs)
if (lambda.formals->argNames.find(i.name) == lambda.formals->argNames.end())
throwTypeError("%1% called with unexpected argument ‘%2%’, at %3%", lambda, i.name, pos);
throwTypeError("%1% called with unexpected argument '%2%', at %3%", lambda, i.name, pos);
abort(); // can't happen
}
}
@@ -1112,7 +1174,7 @@ void EvalState::autoCallFunction(Bindings & args, Value & fun, Value & res)
if (j != args.end())
actualArgs->attrs->push_back(*j);
else if (!i.def)
throwTypeError("cannot auto-call a function that has an argument without a default value (‘%1%’)", i.name);
throwTypeError("cannot auto-call a function that has an argument without a default value ('%1%')", i.name);
}
actualArgs->attrs->sort();
@@ -1324,6 +1386,23 @@ void ExprPos::eval(EvalState & state, Env & env, Value & v)
}
void ExprInclude::eval(EvalState & state, Env & env, Value & v)
{
Value vPath;
path->eval(state, env, vPath);
PathSet context;
auto realPath = state.checkSourcePath(
state.toRealPath(state.coerceToPath(pos, vPath, context), context));
printTalkative("including file '%1%'", realPath);
auto e = state.parseExprFromFile(resolveExprPath(realPath), *staticEnv);
e->eval(state, env, v);
}
void EvalState::forceValueDeep(Value & v)
{
std::set<const Value *> seen;
@@ -1341,7 +1420,7 @@ void EvalState::forceValueDeep(Value & v)
try {
recurse(*i.value);
} catch (Error & e) {
addErrorPrefix(e, "while evaluating the attribute ‘%1%’ at %2%:\n", i.name, *i.pos);
addErrorPrefix(e, "while evaluating the attribute '%1%' at %2%:\n", i.name, *i.pos);
throw;
}
}
@@ -1433,10 +1512,10 @@ string EvalState::forceStringNoCtx(Value & v, const Pos & pos)
string s = forceString(v, pos);
if (v.string.context) {
if (pos)
throwEvalError("the string ‘%1%’ is not allowed to refer to a store path (such as ‘%2%’), at %3%",
throwEvalError("the string '%1%' is not allowed to refer to a store path (such as '%2%'), at %3%",
v.string.s, v.string.context[0], pos);
else
throwEvalError("the string ‘%1%’ is not allowed to refer to a store path (such as ‘%2%’)",
throwEvalError("the string '%1%' is not allowed to refer to a store path (such as '%2%')",
v.string.s, v.string.context[0]);
}
return s;
@@ -1518,17 +1597,17 @@ string EvalState::coerceToString(const Pos & pos, Value & v, PathSet & context,
string EvalState::copyPathToStore(PathSet & context, const Path & path)
{
if (nix::isDerivation(path))
throwEvalError("file names are not allowed to end in ‘%1%’", drvExtension);
throwEvalError("file names are not allowed to end in '%1%'", drvExtension);
Path dstPath;
if (srcToStore[path] != "")
dstPath = srcToStore[path];
else {
dstPath = settings.readOnlyMode
? store->computeStorePathForPath(checkSourcePath(path)).first
? store->computeStorePathForPath(baseNameOf(path), checkSourcePath(path)).first
: store->addToStore(baseNameOf(path), checkSourcePath(path), true, htSHA256, defaultPathFilter, repair);
srcToStore[path] = dstPath;
printMsg(lvlChatty, format("copied source ‘%1%’ -> ‘%2%’")
printMsg(lvlChatty, format("copied source '%1%' -> '%2%'")
% path % dstPath);
}
@@ -1541,7 +1620,7 @@ Path EvalState::coerceToPath(const Pos & pos, Value & v, PathSet & context)
{
string path = coerceToString(pos, v, context, false, false);
if (path == "" || path[0] != '/')
throwEvalError("string ‘%1%’ doesn't represent an absolute path, at %2%", path, pos);
throwEvalError("string '%1%' doesn't represent an absolute path, at %2%", path, pos);
return path;
}
@@ -1647,10 +1726,13 @@ void EvalState::printStats()
printMsg(v, format(" time elapsed: %1%") % cpuTime);
printMsg(v, format(" size of a value: %1%") % sizeof(Value));
printMsg(v, format(" size of an attr: %1%") % sizeof(Attr));
printMsg(v, format(" environments allocated: %1% (%2% bytes)") % nrEnvs % bEnvs);
printMsg(v, format(" list elements: %1% (%2% bytes)") % nrListElems % bLists);
printMsg(v, format(" environments allocated count: %1%") % nrEnvs);
printMsg(v, format(" environments allocated bytes: %1%") % bEnvs);
printMsg(v, format(" list elements count: %1%") % nrListElems);
printMsg(v, format(" list elements bytes: %1%") % bLists);
printMsg(v, format(" list concatenations: %1%") % nrListConcats);
printMsg(v, format(" values allocated: %1% (%2% bytes)") % nrValues % bValues);
printMsg(v, format(" values allocated count: %1%") % nrValues);
printMsg(v, format(" values allocated bytes: %1%") % bValues);
printMsg(v, format(" sets allocated: %1% (%2% bytes)") % nrAttrsets % bAttrsets);
printMsg(v, format(" right-biased unions: %1%") % nrOpUpdates);
printMsg(v, format(" values copied in right-biased unions: %1%") % nrOpUpdateValuesCopied);
+22 -7
View File
@@ -14,6 +14,7 @@ namespace nix {
class Store;
class EvalState;
enum RepairFlag : bool;
typedef void (* PrimOpFun) (EvalState & state, const Pos & pos, Value * * args, Value & v);
@@ -68,16 +69,17 @@ public:
const Symbol sWith, sOutPath, sDrvPath, sType, sMeta, sName, sValue,
sSystem, sOverrides, sOutputs, sOutputName, sIgnoreNulls,
sFile, sLine, sColumn, sFunctor, sToString,
sRight, sWrong, sStructuredAttrs, sBuilder;
sRight, sWrong, sStructuredAttrs, sBuilder, sArgs,
sOutputHash, sOutputHashAlgo, sOutputHashMode;
Symbol sDerivationNix;
/* If set, force copying files to the Nix store even if they
already exist there. */
bool repair = false;
RepairFlag repair;
/* If set, don't allow access to files outside of the Nix search
path or to environment variables. */
bool restricted;
/* The allowed filesystem paths in restricted or pure evaluation
mode. */
std::experimental::optional<PathSet> allowedPaths;
Value vEmptySet;
@@ -109,6 +111,17 @@ public:
Path checkSourcePath(const Path & path);
void checkURI(const std::string & uri);
/* When using a diverted store and 'path' is in the Nix store, map
'path' to the diverted location (e.g. /nix/store/foo is mapped
to /home/alice/my-nix/nix/store/foo). However, this is only
done if the context is not empty, since otherwise we're
probably trying to read from the actual /nix/store. This is
intended to distinguish between import-from-derivation and
sources stored in the actual /nix/store. */
Path toRealPath(const Path & path, const PathSet & context);
/* Parse a Nix expression from the specified file. */
Expr * parseExprFromFile(const Path & path);
Expr * parseExprFromFile(const Path & path, StaticEnv & staticEnv);
@@ -117,6 +130,8 @@ public:
Expr * parseExprFromString(const string & s, const Path & basePath, StaticEnv & staticEnv);
Expr * parseExprFromString(const string & s, const Path & basePath);
Expr * parseStdin();
/* Evaluate an expression read from the given file to normal
form. */
void evalFile(const Path & path, Value & v);
@@ -196,9 +211,9 @@ private:
void createBaseEnv();
void addConstant(const string & name, Value & v);
Value * addConstant(const string & name, Value & v);
void addPrimOp(const string & name,
Value * addPrimOp(const string & name,
unsigned int arity, PrimOpFun primOp);
public:
+72 -18
View File
@@ -1,14 +1,70 @@
#include "get-drvs.hh"
#include "util.hh"
#include "eval-inline.hh"
#include "derivations.hh"
#include <cstring>
#include <regex>
namespace nix {
string DrvInfo::queryDrvPath()
DrvInfo::DrvInfo(EvalState & state, const string & attrPath, Bindings * attrs)
: state(&state), attrs(attrs), attrPath(attrPath)
{
}
DrvInfo::DrvInfo(EvalState & state, ref<Store> store, const std::string & drvPathWithOutputs)
: state(&state), attrs(nullptr), attrPath("")
{
auto spec = parseDrvPathWithOutputs(drvPathWithOutputs);
drvPath = spec.first;
auto drv = store->derivationFromPath(drvPath);
name = storePathToName(drvPath);
if (spec.second.size() > 1)
throw Error("building more than one derivation output is not supported, in '%s'", drvPathWithOutputs);
outputName =
spec.second.empty()
? get(drv.env, "outputName", "out")
: *spec.second.begin();
auto i = drv.outputs.find(outputName);
if (i == drv.outputs.end())
throw Error("derivation '%s' does not have output '%s'", drvPath, outputName);
outPath = i->second.path;
}
string DrvInfo::queryName() const
{
if (name == "" && attrs) {
auto i = attrs->find(state->sName);
if (i == attrs->end()) throw TypeError("derivation name missing");
name = state->forceStringNoCtx(*i->value);
}
return name;
}
string DrvInfo::querySystem() const
{
if (system == "" && attrs) {
auto i = attrs->find(state->sSystem);
system = i == attrs->end() ? "unknown" : state->forceStringNoCtx(*i->value, *i->pos);
}
return system;
}
string DrvInfo::queryDrvPath() const
{
if (drvPath == "" && attrs) {
Bindings::iterator i = attrs->find(state->sDrvPath);
@@ -19,7 +75,7 @@ string DrvInfo::queryDrvPath()
}
string DrvInfo::queryOutPath()
string DrvInfo::queryOutPath() const
{
if (outPath == "" && attrs) {
Bindings::iterator i = attrs->find(state->sOutPath);
@@ -61,7 +117,7 @@ DrvInfo::Outputs DrvInfo::queryOutputs(bool onlyOutputsToInstall)
/* Check for `meta.outputsToInstall` and return `outputs` reduced to that. */
const Value * outTI = queryMeta("outputsToInstall");
if (!outTI) return outputs;
const auto errMsg = Error("this derivation has bad ‘meta.outputsToInstall’");
const auto errMsg = Error("this derivation has bad 'meta.outputsToInstall'");
/* ^ this shows during `nix-env -i` right under the bad derivation */
if (!outTI->isList()) throw errMsg;
Outputs result;
@@ -75,7 +131,7 @@ DrvInfo::Outputs DrvInfo::queryOutputs(bool onlyOutputsToInstall)
}
string DrvInfo::queryOutputName()
string DrvInfo::queryOutputName() const
{
if (outputName == "" && attrs) {
Bindings::iterator i = attrs->find(state->sOutputName);
@@ -224,17 +280,12 @@ static bool getDerivation(EvalState & state, Value & v,
if (done.find(v.attrs) != done.end()) return false;
done.insert(v.attrs);
Bindings::iterator i = v.attrs->find(state.sName);
/* !!! We really would like to have a decent back trace here. */
if (i == v.attrs->end()) throw TypeError("derivation name missing");
DrvInfo drv(state, attrPath, v.attrs);
Bindings::iterator i2 = v.attrs->find(state.sSystem);
DrvInfo drv(state, state.forceStringNoCtx(*i->value), attrPath,
i2 == v.attrs->end() ? "unknown" : state.forceStringNoCtx(*i2->value, *i2->pos),
v.attrs);
drv.queryName();
drvs.push_back(drv);
return false;
} catch (AssertionError & e) {
@@ -244,15 +295,14 @@ static bool getDerivation(EvalState & state, Value & v,
}
bool getDerivation(EvalState & state, Value & v, DrvInfo & drv,
std::experimental::optional<DrvInfo> getDerivation(EvalState & state, Value & v,
bool ignoreAssertionFailures)
{
Done done;
DrvInfos drvs;
getDerivation(state, v, "", drvs, done, ignoreAssertionFailures);
if (drvs.size() != 1) return false;
drv = drvs.front();
return true;
if (drvs.size() != 1) return {};
return std::move(drvs.front());
}
@@ -262,6 +312,9 @@ static string addToPath(const string & s1, const string & s2)
}
static std::regex attrRegex("[A-Za-z_][A-Za-z0-9-_+]*");
static void getDerivations(EvalState & state, Value & vIn,
const string & pathPrefix, Bindings & autoArgs,
DrvInfos & drvs, Done & done,
@@ -285,7 +338,9 @@ static void getDerivations(EvalState & state, Value & vIn,
bound to the attribute with the "lower" name should take
precedence). */
for (auto & i : v.attrs->lexicographicOrder()) {
Activity act(*logger, lvlDebug, format("evaluating attribute ‘%1%’") % i->name);
debug("evaluating attribute '%1%'", i->name);
if (!std::regex_match(std::string(i->name), attrRegex))
continue;
string pathPrefix2 = addToPath(pathPrefix, i->name);
if (combineChannels)
getDerivations(state, *i->value, pathPrefix2, autoArgs, drvs, done, ignoreAssertionFailures);
@@ -304,7 +359,6 @@ static void getDerivations(EvalState & state, Value & vIn,
else if (v.isList()) {
for (unsigned int n = 0; n < v.listSize(); ++n) {
Activity act(*logger, lvlDebug, "evaluating list element");
string pathPrefix2 = addToPath(pathPrefix, (format("%1%") % n).str());
if (getDerivation(state, *v.listElems()[n], pathPrefix2, drvs, done, ignoreAssertionFailures))
getDerivations(state, *v.listElems()[n], pathPrefix2, autoArgs, drvs, done, ignoreAssertionFailures);
+22 -26
View File
@@ -17,31 +17,33 @@ public:
private:
EvalState * state;
string drvPath;
string outPath;
string outputName;
mutable string name;
mutable string system;
mutable string drvPath;
mutable string outPath;
mutable string outputName;
Outputs outputs;
bool failed; // set if we get an AssertionError
bool failed = false; // set if we get an AssertionError
Bindings * attrs, * meta;
Bindings * attrs = nullptr, * meta = nullptr;
Bindings * getMeta();
bool checkMeta(Value & v);
public:
string name;
string attrPath; /* path towards the derivation */
string system;
DrvInfo(EvalState & state) : state(&state), failed(false), attrs(0), meta(0) { };
DrvInfo(EvalState & state, const string & name, const string & attrPath, const string & system, Bindings * attrs)
: state(&state), failed(false), attrs(attrs), meta(0), name(name), attrPath(attrPath), system(system) { };
DrvInfo(EvalState & state) : state(&state) { };
DrvInfo(EvalState & state, const string & attrPath, Bindings * attrs);
DrvInfo(EvalState & state, ref<Store> store, const std::string & drvPathWithOutputs);
string queryDrvPath();
string queryOutPath();
string queryOutputName();
string queryName() const;
string querySystem() const;
string queryDrvPath() const;
string queryOutPath() const;
string queryOutputName() const;
/** Return the list of outputs. The "outputs to install" are determined by `mesa.outputsToInstall`. */
Outputs queryOutputs(bool onlyOutputsToInstall = false);
@@ -58,15 +60,9 @@ public:
MetaValue queryMetaInfo(EvalState & state, const string & name) const;
*/
void setDrvPath(const string & s)
{
drvPath = s;
}
void setOutPath(const string & s)
{
outPath = s;
}
void setName(const string & s) { name = s; }
void setDrvPath(const string & s) { drvPath = s; }
void setOutPath(const string & s) { outPath = s; }
void setFailed() { failed = true; };
bool hasFailed() { return failed; };
@@ -80,10 +76,10 @@ typedef list<DrvInfo> DrvInfos;
#endif
/* If value `v' denotes a derivation, store information about the
derivation in `drv' and return true. Otherwise, return false. */
bool getDerivation(EvalState & state, Value & v, DrvInfo & drv,
bool ignoreAssertionFailures);
/* If value `v' denotes a derivation, return a DrvInfo object
describing it. Otherwise return nothing. */
std::experimental::optional<DrvInfo> getDerivation(EvalState & state,
Value & v, bool ignoreAssertionFailures);
void getDerivations(EvalState & state, Value & v, const string & pathPrefix,
Bindings & autoArgs, DrvInfos & drvs,
+13 -7
View File
@@ -57,7 +57,7 @@ static void parseJSON(EvalState & state, const char * & s, Value & v)
values.push_back(v2);
skipWhitespace(s);
if (*s == ']') break;
if (*s != ',') throw JSONParseError("expected ‘,’ or ‘]’ after JSON array element");
if (*s != ',') throw JSONParseError("expected ',' or ']' after JSON array element");
s++;
}
s++;
@@ -74,14 +74,14 @@ static void parseJSON(EvalState & state, const char * & s, Value & v)
if (attrs.empty() && *s == '}') break;
string name = parseJSONString(s);
skipWhitespace(s);
if (*s != ':') throw JSONParseError("expected ‘:’ in JSON object");
if (*s != ':') throw JSONParseError("expected ':' in JSON object");
s++;
Value * v2 = state.allocValue();
parseJSON(state, s, *v2);
attrs[state.symbols.create(name)] = v2;
skipWhitespace(s);
if (*s == '}') break;
if (*s != ',') throw JSONParseError("expected ‘,’ or ‘}’ after JSON member");
if (*s != ',') throw JSONParseError("expected ',' or '}' after JSON member");
s++;
}
state.mkAttrs(v, attrs.size());
@@ -106,10 +106,16 @@ static void parseJSON(EvalState & state, const char * & s, Value & v)
tmp_number += *s++;
}
if (number_type == tFloat)
mkFloat(v, stod(tmp_number));
else
mkInt(v, stoi(tmp_number));
try {
if (number_type == tFloat)
mkFloat(v, stod(tmp_number));
else
mkInt(v, stoi(tmp_number));
} catch (std::invalid_argument e) {
throw JSONParseError("invalid JSON number");
} catch (std::out_of_range e) {
throw JSONParseError("out-of-range JSON number");
}
}
else if (strncmp(s, "true", 4) == 0) {
+27 -17
View File
@@ -49,9 +49,10 @@ static void adjustLoc(YYLTYPE * loc, const char * s, size_t len)
}
static Expr * unescapeStr(SymbolTable & symbols, const char * s)
static Expr * unescapeStr(SymbolTable & symbols, const char * s, size_t length)
{
string t;
t.reserve(length);
char c;
while ((c = *s++)) {
if (c == '\\') {
@@ -108,6 +109,7 @@ in { return IN; }
rec { return REC; }
inherit { return INHERIT; }
or { return OR_KW; }
include { return INCLUDE; }
\.\.\. { return ELLIPSIS; }
\=\= { return EQ; }
@@ -124,13 +126,13 @@ or { return OR_KW; }
{INT} { errno = 0;
yylval->n = strtol(yytext, 0, 10);
if (errno != 0)
throw ParseError(format("invalid integer ‘%1%’") % yytext);
throw ParseError(format("invalid integer '%1%'") % yytext);
return INT;
}
{FLOAT} { errno = 0;
yylval->nf = strtod(yytext, 0);
if (errno != 0)
throw ParseError(format("invalid float ‘%1%’") % yytext);
throw ParseError(format("invalid float '%1%'") % yytext);
return FLOAT;
}
@@ -142,25 +144,34 @@ or { return OR_KW; }
\{ { return '{'; }
<INSIDE_DOLLAR_CURLY>\{ { PUSH_STATE(INSIDE_DOLLAR_CURLY); return '{'; }
<INITIAL,INSIDE_DOLLAR_CURLY>\" { PUSH_STATE(STRING); return '"'; }
<INITIAL,INSIDE_DOLLAR_CURLY>\" {
PUSH_STATE(STRING); return '"';
}
<STRING>([^\$\"\\]|\$[^\{\"\\]|\\.|\$\\.)*\$/\" |
<STRING>([^\$\"\\]|\$[^\{\"\\]|\\.|\$\\.)+ {
/* It is impossible to match strings ending with '$' with one
regex because trailing contexts are only valid at the end
of a rule. (A sane but undocumented limitation.) */
yylval->e = unescapeStr(data->symbols, yytext);
return STR;
}
/* It is impossible to match strings ending with '$' with one
regex because trailing contexts are only valid at the end
of a rule. (A sane but undocumented limitation.) */
yylval->e = unescapeStr(data->symbols, yytext, yyleng);
return STR;
}
<STRING>\$\{ { PUSH_STATE(INSIDE_DOLLAR_CURLY); return DOLLAR_CURLY; }
<STRING>\" { POP_STATE(); return '"'; }
<STRING>. return yytext[0]; /* just in case: shouldn't be reached */
<STRING>\" { POP_STATE(); return '"'; }
<STRING>\$|\\|\$\\ {
/* This can only occur when we reach EOF, otherwise the above
(...|\$[^\{\"\\]|\\.|\$\\.)+ would have triggered.
This is technically invalid, but we leave the problem to the
parser who fails with exact location. */
return STR;
}
<INITIAL,INSIDE_DOLLAR_CURLY>\'\'(\ *\n)? { PUSH_STATE(IND_STRING); return IND_STRING_OPEN; }
<IND_STRING>([^\$\']|\$[^\{\']|\'[^\'\$])+ {
yylval->e = new ExprIndStr(yytext);
return IND_STR;
}
<IND_STRING>\'\'\$ {
<IND_STRING>\'\'\$ |
<IND_STRING>\$ {
yylval->e = new ExprIndStr("$");
return IND_STR;
}
@@ -169,7 +180,7 @@ or { return OR_KW; }
return IND_STR;
}
<IND_STRING>\'\'\\. {
yylval->e = unescapeStr(data->symbols, yytext + 2);
yylval->e = unescapeStr(data->symbols, yytext + 2, yyleng - 2);
return IND_STR;
}
<IND_STRING>\$\{ { PUSH_STATE(INSIDE_DOLLAR_CURLY); return DOLLAR_CURLY; }
@@ -178,17 +189,16 @@ or { return OR_KW; }
yylval->e = new ExprIndStr("'");
return IND_STR;
}
<IND_STRING>. return yytext[0]; /* just in case: shouldn't be reached */
<INITIAL,INSIDE_DOLLAR_CURLY>{
{PATH} { if (yytext[yyleng-1] == '/')
throw ParseError("path ‘%s’ has a trailing slash", yytext);
throw ParseError("path '%s' has a trailing slash", yytext);
yylval->path = strdup(yytext);
return PATH;
}
{HPATH} { if (yytext[yyleng-1] == '/')
throw ParseError("path ‘%s’ has a trailing slash", yytext);
throw ParseError("path '%s' has a trailing slash", yytext);
yylval->path = strdup(yytext);
return HPATH;
}
Loaded 100 of 316 files, more files were not shown because too many files have changed in this diff. Show more