Compare commits

...

33 Commits

Author SHA1 Message Date
Eelco Dolstra
8a8172cd2b Merge pull request #10050 from NixOS/backport-10049-to-2.20-maintenance
[Backport 2.20-maintenance] Don't send settings that depend on disabled experimental features to the daemon
2024-02-21 13:05:51 +01:00
Eelco Dolstra
7b45cc30a1 Merge pull request #10057 from NixOS/backport-10055-to-2.20-maintenance
[Backport 2.20-maintenance] Faster flake.lock parsing
2024-02-21 12:20:21 +01:00
Graham Dennis
e52d384766 Faster flake.lock parsing
This PR reduces the creation of short-lived basic_json objects while
parsing flake.lock files. For large flake.lock files (~1.5MB) I was
observing ~60s being spent for trivial nix build operations while
after this change it is now taking ~1.6s.

(cherry picked from commit 7fd0de38c6)
2024-02-21 11:19:23 +00:00
Eelco Dolstra
0b32c8763b Don't send settings that depend on disabled experimental features to the daemon
This fixes warnings like

   warning: Ignoring setting 'auto-allocate-uids' because experimental feature 'auto-allocate-uids' is not enabled
   warning: Ignoring setting 'impure-env' because experimental feature 'configurable-impure-env' is not enabled

when using the daemon and the user didn't actually set those settings.

Note: this also hides those settings from `nix config show`, but that
seems a good thing.

(cherry picked from commit 0acd783190)
2024-02-20 14:53:28 +00:00
Eelco Dolstra
adb1d56862 Merge pull request #10045 from NixOS/backport-10043-to-2.20-maintenance
[Backport 2.20-maintenance] fetchToStore(): Don't always respect settings.readOnlyMode
2024-02-20 12:50:30 +01:00
Eelco Dolstra
28dd392948 fetchToStore(): Don't always respect settings.readOnlyMode
It's now up to the caller whether readOnlyMode should be applied. In
some contexts (like InputScheme::fetch()), we always need to fetch.

(cherry picked from commit 7cb4d0c5b7)
2024-02-20 11:08:06 +00:00
Eelco Dolstra
7f02d17881 Don't say "copying X to the store" in read-only mode
(cherry picked from commit 6162105675)
2024-02-20 11:08:06 +00:00
Eelco Dolstra
ce23ef4a77 Bump version 2024-02-19 15:37:26 +01:00
Robert Hensing
98c22e8798 Merge pull request #10023 from NixOS/backport-9985-to-2.20-maintenance
[Backport 2.20-maintenance] Restore `builtins.pathExists` behavior on broken symlinks
2024-02-16 22:55:21 +01:00
John Ericson
02f7025deb Add note about this being a temp solution
(cherry picked from commit e27b7e04bf)
2024-02-16 14:24:23 +00:00
Alois Wohlschlager
0571e6e9b4 Restore builtins.pathExists behavior on broken symlinks
Commit 83c067c0fa changed `builtins.pathExists`
to resolve symlinks before checking for existence. Consequently, if the path
refers to a symlink itself, existence of the target of the symlink (instead of
the symlink itself) was checked. Restore the previous behavior by skipping
symlink resolution in the last component.

(cherry picked from commit 89e21ab4bd)
2024-02-16 14:24:23 +00:00
Eelco Dolstra
982d07d009 Merge pull request #10011 from NixOS/backport-10006-to-2.20-maintenance
[Backport 2.20-maintenance] <nix/fetchurl.nix>: Restore support for "impure = true"
2024-02-13 23:03:59 +01:00
Eelco Dolstra
7f66d4f167 <nix/fetchurl.nix>: Restore support for "impure = true"
(cherry picked from commit bb63bd50e6)
2024-02-13 21:51:13 +00:00
Eelco Dolstra
52e53a2983 Merge pull request #9991 from NixOS/backport-9976-to-2.20-maintenance
[Backport 2.20-maintenance] Restore manual pages
2024-02-12 15:31:57 +01:00
Alois Wohlschlager
c5a8b9050c Restore manual pages
Commit d536c57e87 inadvertedly broke build and
installation of all non-autogenerated manual pages (in particular, all the ones
documenting the stable CLI), by moving the definition of the man-pages variable
in doc/manual/local.mk after its usage in mk/lib.mk. Move including the former
earlier so that the correct order is restored.

(cherry picked from commit 8f3253c6f4)
2024-02-12 14:22:06 +00:00
Eelco Dolstra
86dfeebb3d Merge pull request #9958 from NixOS/backport-9949-to-2.20-maintenance
[Backport 2.20-maintenance] fix location of `_redirects` file
2024-02-07 15:26:07 +01:00
Valentin Gagarin
8f14bf4712 fix location of _redirects file
the Netlify `_redirects` file must be in the root directory [0] of the
files to serve, and mdBook copies all the files in `src` that aren't
`.md` to the output directory [1].

[0]: https://docs.netlify.com/routing/redirects/
[1]: https://rust-lang.github.io/mdBook/guide/creating.html#source-files

(cherry picked from commit 2d74b56aee)
2024-02-07 10:35:09 +00:00
Théophane Hufschmitt
10e1579c81 Merge pull request #9910 from NixOS/backport-9902-to-2.20-maintenance
[Backport 2.20-maintenance] builtin:fetchurl: Ensure a fixed-output derivation
2024-02-02 14:13:01 +01:00
Eelco Dolstra
b6bf4a80d8 Better test fix
(cherry picked from commit e67458e5b8)
2024-02-02 13:00:54 +00:00
Eelco Dolstra
955be03476 Fix test
(cherry picked from commit 05535be03a)
2024-02-02 13:00:54 +00:00
Eelco Dolstra
aab4a17258 builtin:fetchurl: Get output hash info from the drv
(cherry picked from commit b8b739e484)
2024-02-02 13:00:54 +00:00
Eelco Dolstra
df2156a5d2 builtin:fetchurl: Ensure a fixed-output derivation
Previously we didn't check that the derivation was fixed-output, so
you could use builtin:fetchurl to impurely fetch a file.

(cherry picked from commit 1ee42c5b88)
2024-02-02 13:00:54 +00:00
John Ericson
db82034fee Merge pull request #9891 from NixOS/backport-9867-to-2.20-maintenance
[Backport 2.20-maintenance] #912 allow leading period
2024-01-31 15:36:03 -05:00
Robert Hensing
b5947b55e2 Disallow store path names that are . or .. (plus opt. -)
As discussed in the maintainer meeting on 2024-01-29.

Mainly this is to avoid a situation where the name is parsed and
treated as a file name, mostly to protect users.
.-* and ..-* are also considered invalid because they might strip
on that separator to remove versions. Doesn't really work, but that's
what we decided, and I won't argue with it, because .-* probably
doesn't seem to have a real world application anyway.
We do still permit a 1-character name that's just "-", which still
poses a similar risk in such a situation. We can't start disallowing
trailing -, because a non-zero number of users will need it and we've
seen how annoying and painful such a change is.

What matters most is preventing a situation where . or .. can be
injected, and to just get this done.

(cherry picked from commit f1b4663805)
2024-01-31 18:11:17 +00:00
Robert Hensing
60fb31a87d test: Generate distinct hashes
Gen::just is the constant generator. Don't just return that!
(cherry picked from commit 8406da2877)
2024-01-31 18:11:17 +00:00
Robert Hensing
b35958bd7c test: Generate distinct path names
Gen::just is the constant generator. Don't just return that!
(cherry picked from commit 69bbd5852a)
2024-01-31 18:11:17 +00:00
Robert Hensing
f36832ce13 parseStorePath: Support leading period
(cherry picked from commit b13e6a76b4)
2024-01-31 18:11:17 +00:00
Robert Hensing
0f4db25957 Revert "StorePath: reject names starting with '.'"
This reverts commit 24bda0c7b3.

(cherry picked from commit 9ddd0f2af8)
2024-01-31 18:11:17 +00:00
Eelco Dolstra
8f42912c80 Bump version 2024-01-30 18:58:56 +01:00
Eelco Dolstra
a4a4ef9b53 Merge pull request #9886 from NixOS/backport-9884-to-2.20-maintenance
[Backport 2.20-maintenance] Resolve symlinks in a few more places
2024-01-30 17:10:35 +01:00
Eelco Dolstra
5ad5b4447c Resolve symlinks in a few more places
Fixes #9882.

(cherry picked from commit b36ff47e7c)
2024-01-30 16:10:21 +00:00
Eelco Dolstra
1b2b240f22 Bump version 2024-01-29 22:56:56 +01:00
Eelco Dolstra
16e1ff3bcb Mark as stable 2024-01-29 18:59:20 +01:00
30 changed files with 286 additions and 101 deletions

View File

@@ -1 +1 @@
2.20.0
2.20.3

View File

@@ -47,6 +47,17 @@ makefiles += \
tests/functional/plugins/local.mk
endif
# Some makefiles require access to built programs and must be included late.
makefiles-late =
ifeq ($(ENABLE_DOC_GEN), yes)
makefiles-late += doc/manual/local.mk
endif
ifeq ($(ENABLE_INTERNAL_API_DOCS), yes)
makefiles-late += doc/internal-api/local.mk
endif
# Miscellaneous global Flags
OPTIMIZE = 1
@@ -95,24 +106,16 @@ installcheck:
@exit 1
endif
# Documentation or else fallback stub rules.
#
# The documentation makefiles be included after `mk/lib.mk` so rules
# refer to variables defined by `mk/lib.mk`. Rules are not "lazy" like
# variables, unfortunately.
# Documentation fallback stub rules.
ifeq ($(ENABLE_DOC_GEN), yes)
$(eval $(call include-sub-makefile, doc/manual/local.mk))
else
ifneq ($(ENABLE_DOC_GEN), yes)
.PHONY: manual-html manpages
manual-html manpages:
@echo "Generated docs are disabled. Configure without '--disable-doc-gen', or avoid calling 'make manpages' and 'make manual-html'."
@exit 1
endif
ifeq ($(ENABLE_INTERNAL_API_DOCS), yes)
$(eval $(call include-sub-makefile, doc/internal-api/local.mk))
else
ifneq ($(ENABLE_INTERNAL_API_DOCS), yes)
.PHONY: internal-api-html
internal-api-html:
@echo "Internal API docs are disabled. Configure with '--enable-internal-api-docs', or avoid calling 'make internal-api-html'."

View File

@@ -0,0 +1,10 @@
---
synopsis: Store paths are allowed to start with `.`
issues: 912
prs: 9867 9091 9095 9120 9121 9122 9130 9219 9224
---
Leading periods were allowed by accident in Nix 2.4. The Nix team has considered this to be a bug, but this behavior has since been relied on by users, leading to unnecessary difficulties.
From now on, leading periods are officially, definitively supported. The names `.` and `..` are disallowed, as well as those starting with `.-` or `..-`.
Nix versions that denied leading periods are documented [in the issue](https://github.com/NixOS/nix/issues/912#issuecomment-1919583286).

View File

@@ -19,7 +19,7 @@
inherit (import (builtins.fetchTarball { url = "https://github.com/NixOS/nix/archive/1bdcd7fc8a6a40b2e805bad759b36e64e911036b.tar.gz"; sha256 = "sha256:14ljlpdsp4x7h1fkhbmc4bd3vsqnx8zdql4h3037wh09ad6a0893"; }))
fileset;
officialRelease = false;
officialRelease = true;
# Set to true to build the release notes for the next release.
buildUnreleasedNotes = false;
@@ -174,7 +174,7 @@
nix =
let
officialRelease = false;
officialRelease = true;
versionSuffix =
if officialRelease
then ""
@@ -186,7 +186,7 @@
stdenv
versionSuffix
;
officialRelease = false;
officialRelease = true;
boehmgc = final.boehmgc-nix;
libgit2 = final.libgit2-nix;
busybox-sandbox-shell = final.busybox-sandbox-shell or final.default-busybox-sandbox-shell;

View File

@@ -97,6 +97,10 @@ $(foreach test-group, $(install-tests-groups), \
$(eval $(call run-test,$(test),$(install_test_init))) \
$(eval $(test-group).test-group: $(test).test)))
# Include makefiles requiring built programs.
$(foreach mf, $(makefiles-late), $(eval $(call include-sub-makefile,$(mf))))
$(foreach file, $(man-pages), $(eval $(call install-data-in, $(file), $(mandir)/man$(patsubst .%,%,$(suffix $(file))))))

View File

@@ -45,7 +45,7 @@ ref<InstallableValue> InstallableValue::require(ref<Installable> installable)
std::optional<DerivedPathWithInfo> InstallableValue::trySinglePathToDerivedPaths(Value & v, const PosIdx pos, std::string_view errorCtx)
{
if (v.type() == nPath) {
auto storePath = fetchToStore(*state->store, v.path());
auto storePath = fetchToStore(*state->store, v.path(), FetchMode::Copy);
return {{
.path = DerivedPath::Opaque {
.path = std::move(storePath),

View File

@@ -2338,7 +2338,14 @@ StorePath EvalState::copyPathToStore(NixStringContext & context, const SourcePat
auto dstPath = i != srcToStore.end()
? i->second
: [&]() {
auto dstPath = fetchToStore(*store, path, path.baseName(), FileIngestionMethod::Recursive, nullptr, repair);
auto dstPath = fetchToStore(
*store,
path.resolveSymlinks(),
settings.readOnlyMode ? FetchMode::DryRun : FetchMode::Copy,
path.baseName(),
FileIngestionMethod::Recursive,
nullptr,
repair);
allowPath(dstPath);
srcToStore.insert_or_assign(path, dstPath);
printMsg(lvlChatty, "copied source '%1%' -> '%2%'", path, store->printStorePath(dstPath));

View File

@@ -107,7 +107,7 @@ LockFile::LockFile(const nlohmann::json & json, const Path & path)
std::string inputKey = i.value();
auto k = nodeMap.find(inputKey);
if (k == nodeMap.end()) {
auto nodes = json["nodes"];
auto & nodes = json["nodes"];
auto jsonNode2 = nodes.find(inputKey);
if (jsonNode2 == nodes.end())
throw Error("lock file references missing node '%s'", inputKey);

View File

@@ -118,7 +118,7 @@ StringMap EvalState::realiseContext(const NixStringContext & context)
return res;
}
static SourcePath realisePath(EvalState & state, const PosIdx pos, Value & v, bool resolveSymlinks = true)
static SourcePath realisePath(EvalState & state, const PosIdx pos, Value & v, std::optional<SymlinkResolution> resolveSymlinks = SymlinkResolution::Full)
{
NixStringContext context;
@@ -130,7 +130,7 @@ static SourcePath realisePath(EvalState & state, const PosIdx pos, Value & v, bo
auto realPath = state.toRealPath(rewriteStrings(path.path.abs(), rewrites), context);
path = {path.accessor, CanonPath(realPath)};
}
return resolveSymlinks ? path.resolveSymlinks() : path;
return resolveSymlinks ? path.resolveSymlinks(*resolveSymlinks) : path;
} catch (Error & e) {
e.addTrace(state.positions[pos], "while realising the context of path '%s'", path);
throw;
@@ -170,7 +170,7 @@ static void mkOutputString(
argument. */
static void import(EvalState & state, const PosIdx pos, Value & vPath, Value * vScope, Value & v)
{
auto path = realisePath(state, pos, vPath, false);
auto path = realisePath(state, pos, vPath, std::nullopt);
auto path2 = path.path.abs();
// FIXME
@@ -1534,13 +1534,16 @@ static void prim_pathExists(EvalState & state, const PosIdx pos, Value * * args,
try {
auto & arg = *args[0];
auto path = realisePath(state, pos, arg);
/* SourcePath doesn't know about trailing slash. */
state.forceValue(arg, pos);
auto mustBeDir = arg.type() == nString
&& (arg.string_view().ends_with("/")
|| arg.string_view().ends_with("/."));
auto symlinkResolution =
mustBeDir ? SymlinkResolution::Full : SymlinkResolution::Ancestors;
auto path = realisePath(state, pos, arg, symlinkResolution);
auto st = path.maybeLstat();
auto exists = st && (!mustBeDir || st->type == SourceAccessor::tDirectory);
v.mkBool(exists);
@@ -1777,7 +1780,7 @@ static std::string_view fileTypeToString(InputAccessor::Type type)
static void prim_readFileType(EvalState & state, const PosIdx pos, Value * * args, Value & v)
{
auto path = realisePath(state, pos, *args[0], false);
auto path = realisePath(state, pos, *args[0], std::nullopt);
/* Retrieve the directory entry type and stringize it. */
v.mkString(fileTypeToString(path.lstat().type));
}
@@ -2241,7 +2244,14 @@ static void addPath(
});
if (!expectedHash || !state.store->isValidPath(*expectedStorePath)) {
auto dstPath = fetchToStore(*state.store, path, name, method, filter.get(), state.repair);
auto dstPath = fetchToStore(
*state.store,
path.resolveSymlinks(),
settings.readOnlyMode ? FetchMode::DryRun : FetchMode::Copy,
name,
method,
filter.get(),
state.repair);
if (expectedHash && expectedStorePath != dstPath)
state.debugThrowLastTrace(Error("store path mismatch in (possibly filtered) path added from '%s'", path));
state.allowAndSetStorePathString(dstPath, v);

View File

@@ -7,6 +7,7 @@ namespace nix {
StorePath fetchToStore(
Store & store,
const SourcePath & path,
FetchMode mode,
std::string_view name,
ContentAddressMethod method,
PathFilter * filter,
@@ -47,18 +48,19 @@ StorePath fetchToStore(
} else
debug("source path '%s' is uncacheable", path);
Activity act(*logger, lvlChatty, actUnknown, fmt("copying '%s' to the store", path));
Activity act(*logger, lvlChatty, actUnknown,
fmt(mode == FetchMode::DryRun ? "hashing '%s'" : "copying '%s' to the store", path));
auto filter2 = filter ? *filter : defaultPathFilter;
auto storePath =
settings.readOnlyMode
mode == FetchMode::DryRun
? store.computeStorePath(
name, *path.accessor, path.path, method, HashAlgorithm::SHA256, {}, filter2).first
: store.addToStore(
name, *path.accessor, path.path, method, HashAlgorithm::SHA256, {}, filter2, repair);
if (cacheKey)
if (cacheKey && mode == FetchMode::Copy)
fetchers::getCache()->add(store, *cacheKey, {}, storePath, true);
return storePath;

View File

@@ -8,12 +8,15 @@
namespace nix {
enum struct FetchMode { DryRun, Copy };
/**
* Copy the `path` to the Nix store.
*/
StorePath fetchToStore(
Store & store,
const SourcePath & path,
FetchMode mode,
std::string_view name = "source",
ContentAddressMethod method = FileIngestionMethod::Recursive,
PathFilter * filter = nullptr,

View File

@@ -376,7 +376,7 @@ void InputScheme::clone(const Input & input, const Path & destDir) const
std::pair<StorePath, Input> InputScheme::fetch(ref<Store> store, const Input & input)
{
auto [accessor, input2] = getAccessor(store, input);
auto storePath = fetchToStore(*store, SourcePath(accessor), input2.getName());
auto storePath = fetchToStore(*store, SourcePath(accessor), FetchMode::Copy, input2.getName());
return {storePath, input2};
}

View File

@@ -16,6 +16,13 @@ void builtinFetchurl(const BasicDerivation & drv, const std::string & netrcData)
writeFile(settings.netrcFile, netrcData, 0600);
}
auto out = get(drv.outputs, "out");
if (!out)
throw Error("'builtin:fetchurl' requires an 'out' output");
if (!(drv.type().isFixed() || drv.type().isImpure()))
throw Error("'builtin:fetchurl' must be a fixed-output or impure derivation");
auto getAttr = [&](const std::string & name) {
auto i = drv.env.find(name);
if (i == drv.env.end()) throw Error("attribute '%s' missing", name);
@@ -59,13 +66,12 @@ void builtinFetchurl(const BasicDerivation & drv, const std::string & netrcData)
};
/* Try the hashed mirrors first. */
if (getAttr("outputHashMode") == "flat")
auto dof = std::get_if<DerivationOutput::CAFixed>(&out->raw);
if (dof && dof->ca.method.getFileIngestionMethod() == FileIngestionMethod::Flat)
for (auto hashedMirror : settings.hashedMirrors.get())
try {
if (!hasSuffix(hashedMirror, "/")) hashedMirror += '/';
std::optional<HashAlgorithm> ht = parseHashAlgoOpt(getAttr("outputHashAlgo"));
Hash h = newHashAllowEmpty(getAttr("outputHash"), ht);
fetch(hashedMirror + printHashAlgo(h.algo) + "/" + h.to_string(HashFormat::Base16, false));
fetch(hashedMirror + printHashAlgo(dof->ca.hash.algo) + "/" + dof->ca.hash.to_string(HashFormat::Base16, false));
return;
} catch (Error & e) {
debug(e.what());

View File

@@ -3,6 +3,11 @@
namespace nix {
static constexpr std::string_view nameRegexStr = R"([0-9a-zA-Z\+\-_\?=][0-9a-zA-Z\+\-\._\?=]*)";
static constexpr std::string_view nameRegexStr =
// This uses a negative lookahead: (?!\.\.?(-|$))
// - deny ".", "..", or those strings followed by '-'
// - when it's not those, start again at the start of the input and apply the next regex, which is [0-9a-zA-Z\+\-\._\?=]+
R"((?!\.\.?(-|$))[0-9a-zA-Z\+\-\._\?=]+)";
}

View File

@@ -9,9 +9,20 @@ static void checkName(std::string_view path, std::string_view name)
if (name.size() > StorePath::MaxPathLen)
throw BadStorePath("store path '%s' has a name longer than %d characters",
path, StorePath::MaxPathLen);
if (name[0] == '.')
throw BadStorePath("store path '%s' starts with illegal character '.'", path);
// See nameRegexStr for the definition
if (name[0] == '.') {
// check against "." and "..", followed by end or dash
if (name.size() == 1)
throw BadStorePath("store path '%s' has invalid name '%s'", path, name);
if (name[1] == '-')
throw BadStorePath("store path '%s' has invalid name '%s': first dash-separated component must not be '%s'", path, name, ".");
if (name[1] == '.') {
if (name.size() == 2)
throw BadStorePath("store path '%s' has invalid name '%s'", path, name);
if (name[2] == '-')
throw BadStorePath("store path '%s' has invalid name '%s': first dash-separated component must not be '%s'", path, name, "..");
}
}
for (auto c : name)
if (!((c >= '0' && c <= '9')
|| (c >= 'a' && c <= 'z')

View File

@@ -4,7 +4,7 @@
*
* Template implementations (as opposed to mere declarations).
*
* This file is an exmample of the "impl.hh" pattern. See the
* This file is an example of the "impl.hh" pattern. See the
* contributing guide.
*
* One only needs to include this when one is declaring a

View File

@@ -84,7 +84,9 @@ void AbstractConfig::reapplyUnknownSettings()
void Config::getSettings(std::map<std::string, SettingInfo> & res, bool overriddenOnly)
{
for (const auto & opt : _settings)
if (!opt.second.isAlias && (!overriddenOnly || opt.second.setting->overridden))
if (!opt.second.isAlias
&& (!overriddenOnly || opt.second.setting->overridden)
&& experimentalFeatureSettings.isEnabled(opt.second.setting->experimentalFeature))
res.emplace(opt.first, SettingInfo{opt.second.setting->to_string(), opt.second.setting->description});
}

View File

@@ -62,7 +62,7 @@ bool SourcePath::operator<(const SourcePath & x) const
return std::tie(*accessor, path) < std::tie(*x.accessor, x.path);
}
SourcePath SourcePath::resolveSymlinks() const
SourcePath SourcePath::resolveSymlinks(SymlinkResolution mode) const
{
auto res = SourcePath(accessor);
@@ -72,6 +72,8 @@ SourcePath SourcePath::resolveSymlinks() const
for (auto & c : path)
todo.push_back(std::string(c));
bool resolve_last = mode == SymlinkResolution::Full;
while (!todo.empty()) {
auto c = *todo.begin();
todo.pop_front();
@@ -81,14 +83,16 @@ SourcePath SourcePath::resolveSymlinks() const
res.path.pop();
else {
res.path.push(c);
if (auto st = res.maybeLstat(); st && st->type == InputAccessor::tSymlink) {
if (!linksAllowed--)
throw Error("infinite symlink recursion in path '%s'", path);
auto target = res.readLink();
res.path.pop();
if (hasPrefix(target, "/"))
res.path = CanonPath::root;
todo.splice(todo.begin(), tokenizeString<std::list<std::string>>(target, "/"));
if (resolve_last || !todo.empty()) {
if (auto st = res.maybeLstat(); st && st->type == InputAccessor::tSymlink) {
if (!linksAllowed--)
throw Error("infinite symlink recursion in path '%s'", path);
auto target = res.readLink();
res.path.pop();
if (hasPrefix(target, "/"))
res.path = CanonPath::root;
todo.splice(todo.begin(), tokenizeString<std::list<std::string>>(target, "/"));
}
}
}
}

View File

@@ -11,6 +11,26 @@
namespace nix {
/**
* Note there is a decent chance this type soon goes away because the problem is solved another way.
* See the discussion in https://github.com/NixOS/nix/pull/9985.
*/
enum class SymlinkResolution {
/**
* Resolve symlinks in the ancestors only.
*
* Only the last component of the result is possibly a symlink.
*/
Ancestors,
/**
* Resolve symlinks fully, realpath(3)-style.
*
* No component of the result will be a symlink.
*/
Full,
};
/**
* An abstraction for accessing source files during
* evaluation. Currently, it's just a wrapper around `CanonPath` that
@@ -102,11 +122,14 @@ struct SourcePath
bool operator<(const SourcePath & x) const;
/**
* Resolve any symlinks in this `SourcePath` (including its
* parents). The result is a `SourcePath` in which no element is a
* symlink.
* Resolve any symlinks in this `SourcePath` according to the
* given resolution mode.
*
* @param mode might only be a temporary solution for this.
* See the discussion in https://github.com/NixOS/nix/pull/9985.
*/
SourcePath resolveSymlinks() const;
SourcePath resolveSymlinks(
SymlinkResolution mode = SymlinkResolution::Full) const;
};
std::ostream & operator << (std::ostream & str, const SourcePath & path);

View File

@@ -31,17 +31,19 @@ source common.sh
NIX_CONFIG='
experimental-features = nix-command
accept-flake-config = true
' nix config show accept-flake-config 1>$TEST_ROOT/stdout 2>$TEST_ROOT/stderr
grepQuiet "false" $TEST_ROOT/stdout
' expect 1 nix config show accept-flake-config 1>$TEST_ROOT/stdout 2>$TEST_ROOT/stderr
[[ $(cat $TEST_ROOT/stdout) = '' ]]
grepQuiet "Ignoring setting 'accept-flake-config' because experimental feature 'flakes' is not enabled" $TEST_ROOT/stderr
grepQuiet "error: could not find setting 'accept-flake-config'" $TEST_ROOT/stderr
# 'flakes' experimental-feature is disabled after, ignore and warn
NIX_CONFIG='
accept-flake-config = true
experimental-features = nix-command
' nix config show accept-flake-config 1>$TEST_ROOT/stdout 2>$TEST_ROOT/stderr
grepQuiet "false" $TEST_ROOT/stdout
' expect 1 nix config show accept-flake-config 1>$TEST_ROOT/stdout 2>$TEST_ROOT/stderr
[[ $(cat $TEST_ROOT/stdout) = '' ]]
grepQuiet "Ignoring setting 'accept-flake-config' because experimental feature 'flakes' is not enabled" $TEST_ROOT/stderr
grepQuiet "error: could not find setting 'accept-flake-config'" $TEST_ROOT/stderr
# 'flakes' experimental-feature is enabled before, process
NIX_CONFIG='

View File

@@ -30,7 +30,10 @@ echo hello >> $TEST_ROOT/worktree/hello
rev2=$(git -C $repo rev-parse HEAD)
git -C $repo tag -a tag2 -m tag2
# Fetch a worktree
# Check whether fetching in read-only mode works.
nix-instantiate --eval -E "builtins.readFile ((builtins.fetchGit file://$TEST_ROOT/worktree) + \"/hello\") == \"utrecht\\n\""
# Fetch a worktree.
unset _NIX_FORCE_HTTP
path0=$(nix eval --impure --raw --expr "(builtins.fetchGit file://$TEST_ROOT/worktree).outPath")
path0_=$(nix eval --impure --raw --expr "(builtins.fetchTree { type = \"git\"; url = file://$TEST_ROOT/worktree; }).outPath")

View File

@@ -78,3 +78,9 @@ outPath=$(nix-build -vvvvv --expr 'import <nix/fetchurl.nix>' --argstr url file:
test -x $outPath/fetchurl.sh
test -L $outPath/symlink
# Make sure that *not* passing a outputHash fails.
requireDaemonNewerThan "2.20"
expected=100
if [[ -v NIX_DAEMON_PACKAGE ]]; then expected=1; fi # work around the daemon not returning a 100 status correctly
expectStderr $expected nix-build --expr '{ url }: builtins.derivation { name = "nix-cache-info"; system = "x86_64-linux"; builder = "builtin:fetchurl"; inherit url; outputHashMode = "flat"; }' --argstr url file://$narxz 2>&1 | grep 'must be a fixed-output or impure derivation'

View File

@@ -63,3 +63,7 @@ path5=$(nix build -L --no-link --json --file ./impure-derivations.nix contentAdd
path6=$(nix build -L --no-link --json --file ./impure-derivations.nix inputAddressedAfterCA | jq -r .[].outputs.out)
[[ $(< $path6) = X ]]
[[ $(< $TEST_ROOT/counter) = 5 ]]
# Test nix/fetchurl.nix.
path7=$(nix build -L --no-link --print-out-paths --expr "import <nix/fetchurl.nix> { impure = true; url = file://$PWD/impure-derivations.sh; }")
cmp $path7 $PWD/impure-derivations.sh

View File

@@ -29,3 +29,6 @@ builtins.pathExists (./lib.nix)
&& builtins.pathExists (builtins.toPath { outPath = builtins.toString ./lib.nix; })
&& builtins.pathExists ./lib.nix
&& !builtins.pathExists ./bla.nix
&& builtins.pathExists ./symlink-resolution/foo/overlays/overlay.nix
&& builtins.pathExists ./symlink-resolution/broken
&& builtins.pathExists (builtins.toString ./symlink-resolution/foo/overlays + "/.")

View File

@@ -0,0 +1 @@
nonexistent

View File

@@ -29,7 +29,8 @@ unset NIX_CONFIG
# Create a channel.
rm -rf $TEST_ROOT/foo
mkdir -p $TEST_ROOT/foo
nix copy --to file://$TEST_ROOT/foo?compression="bzip2" $(nix-store -r $(nix-instantiate dependencies.nix))
drvPath=$(nix-instantiate dependencies.nix)
nix copy --to file://$TEST_ROOT/foo?compression="bzip2" $(nix-store -r "$drvPath")
rm -rf $TEST_ROOT/nixexprs
mkdir -p $TEST_ROOT/nixexprs
cp config.nix dependencies.nix dependencies.builder*.sh $TEST_ROOT/nixexprs/
@@ -64,3 +65,5 @@ grepQuiet 'item.*attrPath="foo".*name="dependencies-top"' $TEST_ROOT/meta.xml
nix-env -i dependencies-top
[ -e $TEST_HOME/.nix-profile/foobar ]
# Test evaluation through a channel symlink (#9882).
nix-instantiate '<foo/dependencies.nix>'

View File

@@ -1,3 +1,4 @@
#include <rapidcheck/gen/Arbitrary.h>
#include <regex>
#include <rapidcheck.h>
@@ -20,63 +21,60 @@ void showValue(const StorePath & p, std::ostream & os)
namespace rc {
using namespace nix;
Gen<StorePathName> Arbitrary<StorePathName>::arbitrary()
Gen<char> storePathChar()
{
auto len = *gen::inRange<size_t>(
1,
StorePath::MaxPathLen - StorePath::HashLen);
std::string pre;
pre.reserve(len);
for (size_t c = 0; c < len; ++c) {
switch (auto i = *gen::inRange<uint8_t>(0, 10 + 2 * 26 + 6)) {
return rc::gen::apply([](uint8_t i) -> char {
switch (i) {
case 0 ... 9:
pre += '0' + i;
return '0' + i;
case 10 ... 35:
pre += 'A' + (i - 10);
break;
return 'A' + (i - 10);
case 36 ... 61:
pre += 'a' + (i - 36);
break;
return 'a' + (i - 36);
case 62:
pre += '+';
break;
return '+';
case 63:
pre += '-';
break;
return '-';
case 64:
// names aren't permitted to start with a period,
// so just fall through to the next case here
if (c != 0) {
pre += '.';
break;
}
return '.';
case 65:
pre += '_';
break;
return '_';
case 66:
pre += '?';
break;
return '?';
case 67:
pre += '=';
break;
return '=';
default:
assert(false);
}
}
},
gen::inRange<uint8_t>(0, 10 + 2 * 26 + 6));
}
return gen::just(StorePathName {
.name = std::move(pre),
});
Gen<StorePathName> Arbitrary<StorePathName>::arbitrary()
{
return gen::construct<StorePathName>(
gen::suchThat(
gen::container<std::string>(storePathChar()),
[](const std::string & s) {
return
!( s == ""
|| s == "."
|| s == ".."
|| s.starts_with(".-")
|| s.starts_with("..-")
);
}
)
);
}
Gen<StorePath> Arbitrary<StorePath>::arbitrary()
{
return gen::just(StorePath {
*gen::arbitrary<Hash>(),
(*gen::arbitrary<StorePathName>()).name,
});
return
gen::construct<StorePath>(
gen::arbitrary<Hash>(),
gen::apply([](StorePathName n){ return n.name; }, gen::arbitrary<StorePathName>())
);
}
} // namespace rc

View File

@@ -39,7 +39,12 @@ TEST_DONT_PARSE(double_star, "**")
TEST_DONT_PARSE(star_first, "*,foo")
TEST_DONT_PARSE(star_second, "foo,*")
TEST_DONT_PARSE(bang, "foo!o")
TEST_DONT_PARSE(dotfile, ".gitignore")
TEST_DONT_PARSE(dot, ".")
TEST_DONT_PARSE(dot_dot, "..")
TEST_DONT_PARSE(dot_dot_dash, "..-1")
TEST_DONT_PARSE(dot_dash, ".-1")
TEST_DONT_PARSE(dot_dot_dash_a, "..-a")
TEST_DONT_PARSE(dot_dash_a, ".-a")
#undef TEST_DONT_PARSE
@@ -63,6 +68,11 @@ TEST_DO_PARSE(underscore, "foo_bar")
TEST_DO_PARSE(period, "foo.txt")
TEST_DO_PARSE(question_mark, "foo?why")
TEST_DO_PARSE(equals_sign, "foo=foo")
TEST_DO_PARSE(dotfile, ".gitignore")
TEST_DO_PARSE(triple_dot_a, "...a")
TEST_DO_PARSE(triple_dot_1, "...1")
TEST_DO_PARSE(triple_dot_dash, "...-")
TEST_DO_PARSE(triple_dot, "...")
#undef TEST_DO_PARSE
@@ -84,6 +94,64 @@ RC_GTEST_FIXTURE_PROP(
RC_ASSERT(p == store->parseStorePath(store->printStorePath(p)));
}
RC_GTEST_FIXTURE_PROP(
StorePathTest,
prop_check_regex_eq_parse,
())
{
static auto nameFuzzer =
rc::gen::container<std::string>(
rc::gen::oneOf(
// alphanum, repeated to weigh heavier
rc::gen::oneOf(
rc::gen::inRange('0', '9'),
rc::gen::inRange('a', 'z'),
rc::gen::inRange('A', 'Z')
),
// valid symbols
rc::gen::oneOf(
rc::gen::just('+'),
rc::gen::just('-'),
rc::gen::just('.'),
rc::gen::just('_'),
rc::gen::just('?'),
rc::gen::just('=')
),
// symbols for scary .- and ..- cases, repeated for weight
rc::gen::just('.'), rc::gen::just('.'),
rc::gen::just('.'), rc::gen::just('.'),
rc::gen::just('-'), rc::gen::just('-'),
// ascii symbol ranges
rc::gen::oneOf(
rc::gen::inRange(' ', '/'),
rc::gen::inRange(':', '@'),
rc::gen::inRange('[', '`'),
rc::gen::inRange('{', '~')
),
// typical whitespace
rc::gen::oneOf(
rc::gen::just(' '),
rc::gen::just('\t'),
rc::gen::just('\n'),
rc::gen::just('\r')
),
// some chance of control codes, non-ascii or other garbage we missed
rc::gen::inRange('\0', '\xff')
));
auto name = *nameFuzzer;
std::string path = store->storeDir + "/575s52sh487i0ylmbs9pvi606ljdszr0-" + name;
bool parsed = false;
try {
store->parseStorePath(path);
parsed = true;
} catch (const BadStorePath &) {
}
RC_ASSERT(parsed == std::regex_match(std::string { name }, nameRegex));
}
#endif
}

View File

@@ -11,10 +11,17 @@ using namespace nix;
Gen<Hash> Arbitrary<Hash>::arbitrary()
{
Hash hash(HashAlgorithm::SHA1);
for (size_t i = 0; i < hash.hashSize; ++i)
hash.hash[i] = *gen::arbitrary<uint8_t>();
return gen::just(hash);
Hash prototype(HashAlgorithm::SHA1);
return
gen::apply(
[](const std::vector<uint8_t> & v) {
Hash hash(HashAlgorithm::SHA1);
assert(v.size() == hash.hashSize);
std::copy(v.begin(), v.end(), hash.hash);
return hash;
},
gen::container<std::vector<uint8_t>>(prototype.hashSize, gen::arbitrary<uint8_t>())
);
}
}