Compare commits

...

24 Commits

Author SHA1 Message Date
John Ericson
ddca102f8b Move StringData to its own header
Now that it isn't just used for `Value`, it doesn't really belong in
there.

Rename `static-string-data.hh` to share the same prefix to keep them
close together when sorting lines, also.
2025-11-18 14:01:32 -05:00
John Ericson
351e3870c9 Use StringData for Nix language value documentation too
This probably isn't much of a perf gain, but it does mean we are
completely rid of non-length prefixed strings. That allows us to delete
some code.
2025-11-18 14:01:32 -05:00
John Ericson
1ea6a71b0c Improve formatting of primop_fromTOML
Trailing comma helps a lot.
2025-11-18 14:01:32 -05:00
John Ericson
2cc0b1b404 Introduce quoteString utility function 2025-11-17 12:33:26 -05:00
John Ericson
5446d6345f Merge pull request #14576 from corngood/cygwin-tests
Fix/disable tests on cygwin
2025-11-17 04:22:10 +00:00
David McFarland
b115c90043 Disable MonitorFdHup test on cygwin 2025-11-16 23:33:28 -04:00
David McFarland
13b896a188 Disable toString/ToStringPrimOpTest.toString/10 on cygwin 2025-11-16 23:32:29 -04:00
Sergei Zimmerman
5462c5eedd Merge pull request #8871 from teto/flake_show_attr
nix flake show: name attribute that must be a derivation
2025-11-16 19:48:15 +00:00
John Ericson
aec59a973a Merge pull request #14573 from corngood/libexpr-leak
nix_api_expr: ensure destructors are called for builder/state
2025-11-16 04:28:08 +00:00
Matthieu Coudron
653d701300 Merge branch 'master' into flake_show_attr 2025-11-15 23:30:42 +01:00
David McFarland
8d881ee3a3 nix_api_expr: ensure destructors are called for builder/state
I found this because of a test failure on cygwin in
nix_api_expr_test.nix_eval_state_lookup_path:

 'std::filesystem::__cxx11::filesystem_error'
   what():  filesystem error: cannot remove all: Device or resource busy
   [...]
   [.../my_state/db/db.sqlite]

LocalState was never getting destroyed due to a reference leak.  These
_free functions use an 'operator delete' which doesn't call the
destructor for the type.

Fixes: 309d55807c
2025-11-15 15:39:39 -04:00
David McFarland
2872c8ede0 Fix leaks in nix_api_store_test.nix_eval_state_lookup_path 2025-11-15 15:38:39 -04:00
David McFarland
57f526ecda Fix nix_api_store_test.nix_eval_state_lookup_path when run on its own
Currently, --gtest_filter=nix_api_store_test.nix_eval_state_lookup_path
will result in:

 terminating due to unexpected unrecoverable internal error: Assertion
 'gcInitialised' failed in void nix::assertGCInitialized() at
 ../src/libexpr/eval-gc.cc:138

Changing the test fixture to _exr_test causes GC to be initialised.
2025-11-15 15:36:49 -04:00
John Ericson
1f2a994fb9 Merge pull request #14568 from NixOS/proper-range-canon-path
libutil: Make CanonPath a proper range
2025-11-15 17:09:13 +00:00
Sergei Zimmerman
0e81a35881 libutil: Make CanonPath a proper range
This was we can use std::ranges algorithms on it. Requires
making the iterator a proper forward iterator type as well.
2025-11-14 22:45:20 +03:00
John Ericson
94c3bb3e4c Merge pull request #14562 from NixOS/no-races-posix-source-accessor
libutil: Make PosixSourceAccessor update mtime only when needed
2025-11-14 04:48:41 +00:00
John Ericson
30dbc7ee0c Merge pull request #14563 from NixOS/dead-variable
libstore: Remove dead PosixSourceAccessor variable in verifyStore
2025-11-14 04:42:38 +00:00
Sergei Zimmerman
19ab65c9d7 libstore: Remove dead PosixSourceAccessor variable in verifyStore 2025-11-14 04:18:53 +03:00
John Ericson
805496657d Merge pull request #14550 from roberth/fetchers-settings-arg
Remove setting from Input
2025-11-13 22:59:27 +00:00
Sergei Zimmerman
e95503cf9a libutil: Make PosixSourceAccessor update mtime only when needed
Typically PosixSourceAccessor can be used from multiple threads,
but mtime is not updated atomically (i.e. with compare_exchange_weak),
so mtime gets raced. It's only needed in dumpPathAndGetMtime and mtime
tracking can be gated behind that.

Also start using getLastModified interface instead of dynamic casts.
2025-11-13 23:54:14 +03:00
Eelco Dolstra
1bcbe652fb Merge pull request #14537 from NixOS/dependabot/github_actions/cachix/install-nix-action-31.8.3
build(deps): bump cachix/install-nix-action from 31.8.2 to 31.8.3
2025-11-13 17:13:59 +00:00
Robert Hensing
292bd390af Remove setting from Input
This is more straightforward and not subject to undocumented memory
safety restrictions.
Also easier to test.
2025-11-12 23:42:09 +01:00
dependabot[bot]
2150d7a754 build(deps): bump cachix/install-nix-action from 31.8.2 to 31.8.3
Bumps [cachix/install-nix-action](https://github.com/cachix/install-nix-action) from 31.8.2 to 31.8.3.
- [Release notes](https://github.com/cachix/install-nix-action/releases)
- [Changelog](https://github.com/cachix/install-nix-action/blob/master/RELEASE.md)
- [Commits](456688f15b...7ec16f2c06)

---
updated-dependencies:
- dependency-name: cachix/install-nix-action
  dependency-version: 31.8.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-11-10 22:00:54 +00:00
Matthieu Coudron
ac9d2a5b06 nix flake show: log attribute name that "must be a derivation"
I would run `nix flake show` on a flake than hit:

===
        ├───ihaskell: package 'ihaskell-wrapper'
        ├───ihaskell-96: package 'ihaskell-wrapper'
        ├───ihaskell-96-dev: package 'ghc-shell-for-ihaskell-0.10.4.0'
error: expected a derivation
===
and it is not obvious what package is the culprit here since nix stops
rightaway.


Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2025-11-08 13:30:57 +01:00
58 changed files with 685 additions and 574 deletions

View File

@@ -174,7 +174,7 @@ jobs:
echo "installer-url=file://$GITHUB_WORKSPACE/out" >> "$GITHUB_OUTPUT"
TARBALL_PATH="$(find "$GITHUB_WORKSPACE/out" -name 'nix*.tar.xz' -print | head -n 1)"
echo "tarball-path=file://$TARBALL_PATH" >> "$GITHUB_OUTPUT"
- uses: cachix/install-nix-action@456688f15bc354bef6d396e4a35f4f89d40bf2b7 # v31.8.2
- uses: cachix/install-nix-action@7ec16f2c061ab07b235a7245e06ed46fe9a1cab6 # v31.8.3
if: ${{ !matrix.experimental-installer }}
with:
install_url: ${{ format('{0}/install', steps.installer-tarball-url.outputs.installer-url) }}

View File

@@ -33,7 +33,8 @@ EvalSettings evalSettings{
// FIXME `parseFlakeRef` should take a `std::string_view`.
auto flakeRef = parseFlakeRef(fetchSettings, std::string{rest}, {}, true, false);
debug("fetching flake search path element '%s''", rest);
auto [accessor, lockedRef] = flakeRef.resolve(state.store).lazyFetch(state.store);
auto [accessor, lockedRef] =
flakeRef.resolve(fetchSettings, state.store).lazyFetch(fetchSettings, state.store);
auto storePath = nix::fetchToStore(
state.fetchSettings,
*state.store,
@@ -131,7 +132,7 @@ MixEvalArgs::MixEvalArgs()
fetchers::Attrs extraAttrs;
if (to.subdir != "")
extraAttrs["dir"] = to.subdir;
fetchers::overrideRegistry(from.input, to.input, extraAttrs);
fetchers::overrideRegistry(fetchSettings, from.input, to.input, extraAttrs);
}},
.completer = {[&](AddCompletions & completions, size_t, std::string_view prefix) {
completeFlakeRef(completions, openStore(), prefix);
@@ -187,7 +188,7 @@ SourcePath lookupFileArg(EvalState & state, std::string_view s, const Path * bas
else if (hasPrefix(s, "flake:")) {
experimentalFeatureSettings.require(Xp::Flakes);
auto flakeRef = parseFlakeRef(fetchSettings, std::string(s.substr(6)), {}, true, false);
auto [accessor, lockedRef] = flakeRef.resolve(state.store).lazyFetch(state.store);
auto [accessor, lockedRef] = flakeRef.resolve(fetchSettings, state.store).lazyFetch(fetchSettings, state.store);
auto storePath = nix::fetchToStore(
state.fetchSettings, *state.store, SourcePath(accessor), FetchMode::Copy, lockedRef.input.getName());
state.allowPath(storePath);

View File

@@ -185,6 +185,7 @@ MixFlakeOptions::MixFlakeOptions()
}
overrideRegistry(
fetchSettings,
fetchers::Input::fromAttrs(fetchSettings, {{"type", "indirect"}, {"id", inputName}}),
input3->lockedRef.input,
extraAttrs);

View File

@@ -656,7 +656,7 @@ ProcessLineResult NixRepl::processLine(std::string line)
+ "\n\n";
}
markdown += stripIndentation(doc->doc);
markdown += stripIndentation(doc->doc.view());
logger->cout(trim(renderMarkdownToTerminal(markdown)));
} else if (fallbackPos) {
@@ -738,7 +738,7 @@ void NixRepl::loadFlake(const std::string & flakeRefS)
}
auto flakeRef = parseFlakeRef(fetchSettings, flakeRefS, cwd.string(), true);
if (evalSettings.pureEval && !flakeRef.input.isLocked())
if (evalSettings.pureEval && !flakeRef.input.isLocked(fetchSettings))
throw Error("cannot use ':load-flake' on locked flake reference '%s' (use --impure to override)", flakeRefS);
Value v;

View File

@@ -137,6 +137,8 @@ nix_eval_state_builder * nix_eval_state_builder_new(nix_c_context * context, Sto
void nix_eval_state_builder_free(nix_eval_state_builder * builder)
{
if (builder)
builder->~nix_eval_state_builder();
operator delete(builder, static_cast<std::align_val_t>(alignof(nix_eval_state_builder)));
}
@@ -203,6 +205,8 @@ EvalState * nix_state_create(nix_c_context * context, const char ** lookupPath_c
void nix_state_free(EvalState * state)
{
if (state)
state->~EvalState();
operator delete(state, static_cast<std::align_val_t>(alignof(EvalState)));
}

View File

@@ -134,7 +134,7 @@ PrimOp * nix_alloc_primop(
.name = name,
.args = {},
.arity = (size_t) arity,
.doc = doc,
.doc = &nix::StringData::make(doc),
.fun = std::bind(nix_c_primop_wrapper, fun, user_data, _1, _2, _3, _4)};
if (args)
for (size_t i = 0; args[i]; i++)

View File

@@ -1,6 +1,6 @@
#include "nix/expr/tests/libexpr.hh"
#include "nix/expr/value-to-json.hh"
#include "nix/expr/static-string-data.hh"
#include "nix/expr/string-data-static.hh"
namespace nix {
// Testing the conversion to JSON

View File

@@ -14,7 +14,7 @@
namespace nixC {
TEST_F(nix_api_store_test, nix_eval_state_lookup_path)
TEST_F(nix_api_expr_test, nix_eval_state_lookup_path)
{
auto tmpDir = nix::createTempDir();
auto delTmpDir = std::make_unique<nix::AutoDelete>(tmpDir, true);
@@ -42,12 +42,16 @@ TEST_F(nix_api_store_test, nix_eval_state_lookup_path)
nix_expr_eval_from_string(ctx, state, "builtins.seq <nixos-config> <nixpkgs>", ".", value);
assert_ctx_ok();
nix_state_free(state);
ASSERT_EQ(nix_get_type(ctx, value), NIX_TYPE_PATH);
assert_ctx_ok();
auto pathStr = nix_get_path_string(ctx, value);
assert_ctx_ok();
ASSERT_EQ(0, strcmp(pathStr, nixpkgs.c_str()));
nix_gc_decref(nullptr, value);
}
TEST_F(nix_api_expr_test, nix_expr_eval_from_string)

View File

@@ -661,8 +661,14 @@ INSTANTIATE_TEST_SUITE_P(
CASE(R"(null)", ""),
CASE(R"({ v = "bar"; __toString = self: self.v; })", "bar"),
CASE(R"({ v = "bar"; __toString = self: self.v; outPath = "foo"; })", "bar"),
CASE(R"({ outPath = "foo"; })", "foo"),
CASE(R"(./test)", "/test")));
CASE(R"({ outPath = "foo"; })", "foo")
// this is broken on cygwin because canonPath("//./test", false) returns //./test
// FIXME: don't use canonPath
#ifndef __CYGWIN__
,
CASE(R"(./test)", "/test")
#endif
));
#undef CASE
TEST_F(PrimOpTest, substring)

View File

@@ -1,5 +1,5 @@
#include "nix/expr/tests/libexpr.hh"
#include "nix/expr/static-string-data.hh"
#include "nix/expr/string-data-static.hh"
#include "nix/expr/value.hh"
#include "nix/expr/print.hh"

View File

@@ -1,5 +1,5 @@
#include "nix/expr/value.hh"
#include "nix/expr/static-string-data.hh"
#include "nix/expr/string-data-static.hh"
#include "nix/store/tests/libstore.hh"
#include <gtest/gtest.h>

View File

@@ -51,36 +51,6 @@ using json = nlohmann::json;
namespace nix {
/**
* Just for doc strings. Not for regular string values.
*/
static char * allocString(size_t size)
{
char * t;
t = (char *) GC_MALLOC_ATOMIC(size);
if (!t)
throw std::bad_alloc();
return t;
}
// When there's no need to write to the string, we can optimize away empty
// string allocations.
// This function handles makeImmutableString(std::string_view()) by returning
// the empty string.
/**
* Just for doc strings. Not for regular string values.
*/
static const char * makeImmutableString(std::string_view s)
{
const size_t size = s.size();
if (size == 0)
return "";
auto t = allocString(size + 1);
memcpy(t, s.data(), size);
t[size] = '\0';
return t;
}
StringData & StringData::alloc(size_t size)
{
void * t = GC_MALLOC_ATOMIC(sizeof(StringData) + size + 1);
@@ -571,7 +541,7 @@ std::optional<EvalState::Doc> EvalState::getDoc(Value & v)
.name = v2->primOp()->name,
.arity = v2->primOp()->arity,
.args = v2->primOp()->args,
.doc = doc,
.doc = *doc,
};
}
if (v.isLambda()) {
@@ -613,9 +583,8 @@ std::optional<EvalState::Doc> EvalState::getDoc(Value & v)
.name = name,
.arity = 0, // FIXME: figure out how deep by syntax only? It's not semantically useful though...
.args = {},
/* N.B. Can't use StringData here, because that would lead to an interior pointer.
NOTE: memory leak when compiled without GC. */
.doc = makeImmutableString(s.view()),
/* NOTE: memory leak when compiled without GC. */
.doc = StringData::make(s.view()),
};
}
if (isFunctor(v)) {

View File

@@ -108,7 +108,7 @@ struct PrimOp
/**
* Optional free-form documentation about the primop.
*/
const char * doc = nullptr;
const StringData * doc = nullptr;
/**
* Add a trace item, while calling the `<name>` builtin.
@@ -156,7 +156,7 @@ struct Constant
/**
* Optional free-form documentation about the constant.
*/
const char * doc = nullptr;
const StringData * doc = nullptr;
/**
* Whether the constant is impure, and not available in pure mode.
@@ -837,11 +837,7 @@ public:
std::optional<std::string> name;
size_t arity;
std::vector<std::string> args;
/**
* Unlike the other `doc` fields in this file, this one should never be
* `null`.
*/
const char * doc;
const StringData & doc;
};
/**

View File

@@ -31,7 +31,8 @@ headers = [ config_pub_h ] + files(
'print.hh',
'repl-exit-status.hh',
'search-path.hh',
'static-string-data.hh',
'string-data.hh',
'string-data-static.hh',
'symbol-table.hh',
'value-to-json.hh',
'value-to-xml.hh',

View File

@@ -12,7 +12,7 @@
#include "nix/expr/value.hh"
#include "nix/expr/symbol-table.hh"
#include "nix/expr/eval-error.hh"
#include "nix/expr/static-string-data.hh"
#include "nix/expr/string-data-static.hh"
#include "nix/util/pos-idx.hh"
#include "nix/expr/counter.hh"
#include "nix/util/pos-table.hh"

View File

@@ -5,7 +5,7 @@
#include "nix/expr/eval.hh"
#include "nix/expr/value.hh"
#include "nix/expr/static-string-data.hh"
#include "nix/expr/string-data-static.hh"
namespace nix {

View File

@@ -1,7 +1,7 @@
#pragma once
///@file
#include "nix/expr/value.hh"
#include "nix/expr/string-data.hh"
namespace nix {

View File

@@ -0,0 +1,96 @@
#pragma once
///@file
#include <cstddef>
#include <cstring>
#include <memory_resource>
#include <string_view>
namespace nix {
class StringData
{
public:
using size_type = std::size_t;
size_type size_;
char data_[];
/*
* This in particular ensures that we cannot have a `StringData`
* that we use by value, which is just what we want!
*
* Dynamically sized types aren't a thing in C++ and even flexible array
* members are a language extension and beyond the realm of standard C++.
* Technically, sizeof data_ member is 0 and the intended way to use flexible
* array members is to allocate sizeof(StrindData) + count * sizeof(char) bytes
* and the compiler will consider alignment restrictions for the FAM.
*
*/
StringData(StringData &&) = delete;
StringData & operator=(StringData &&) = delete;
StringData(const StringData &) = delete;
StringData & operator=(const StringData &) = delete;
~StringData() = default;
private:
StringData() = delete;
explicit StringData(size_type size)
: size_(size)
{
}
public:
/**
* Allocate StringData on the (possibly) GC-managed heap and copy
* the contents of s to it.
*/
static const StringData & make(std::string_view s);
/**
* Allocate StringData on the (possibly) GC-managed heap.
* @param size Length of the string (without the NUL terminator).
*/
static StringData & alloc(size_t size);
size_t size() const
{
return size_;
}
char * data() noexcept
{
return data_;
}
const char * data() const noexcept
{
return data_;
}
const char * c_str() const noexcept
{
return data_;
}
constexpr std::string_view view() const noexcept
{
return std::string_view(data_, size_);
}
template<size_t N>
struct Static;
static StringData & make(std::pmr::memory_resource & resource, std::string_view s)
{
auto & res =
*new (resource.allocate(sizeof(StringData) + s.size() + 1, alignof(StringData))) StringData(s.size());
std::memcpy(res.data_, s.data(), s.size());
res.data_[s.size()] = '\0';
return res;
}
};
} // namespace nix

View File

@@ -3,7 +3,7 @@
#include <memory_resource>
#include "nix/expr/value.hh"
#include "nix/expr/static-string-data.hh"
#include "nix/expr/string-data-static.hh"
#include "nix/util/chunked-vector.hh"
#include "nix/util/error.hh"

View File

@@ -3,16 +3,14 @@
#include <bit>
#include <cassert>
#include <cstddef>
#include <cstring>
#include <memory>
#include <memory_resource>
#include <span>
#include <string_view>
#include <type_traits>
#include <concepts>
#include "nix/expr/eval-gc.hh"
#include "nix/expr/string-data.hh"
#include "nix/expr/value/context.hh"
#include "nix/util/source-path.hh"
#include "nix/expr/print-options.hh"
@@ -193,91 +191,6 @@ public:
friend struct Value;
};
class StringData
{
public:
using size_type = std::size_t;
size_type size_;
char data_[];
/*
* This in particular ensures that we cannot have a `StringData`
* that we use by value, which is just what we want!
*
* Dynamically sized types aren't a thing in C++ and even flexible array
* members are a language extension and beyond the realm of standard C++.
* Technically, sizeof data_ member is 0 and the intended way to use flexible
* array members is to allocate sizeof(StrindData) + count * sizeof(char) bytes
* and the compiler will consider alignment restrictions for the FAM.
*
*/
StringData(StringData &&) = delete;
StringData & operator=(StringData &&) = delete;
StringData(const StringData &) = delete;
StringData & operator=(const StringData &) = delete;
~StringData() = default;
private:
StringData() = delete;
explicit StringData(size_type size)
: size_(size)
{
}
public:
/**
* Allocate StringData on the (possibly) GC-managed heap and copy
* the contents of s to it.
*/
static const StringData & make(std::string_view s);
/**
* Allocate StringData on the (possibly) GC-managed heap.
* @param size Length of the string (without the NUL terminator).
*/
static StringData & alloc(size_t size);
size_t size() const
{
return size_;
}
char * data() noexcept
{
return data_;
}
const char * data() const noexcept
{
return data_;
}
const char * c_str() const noexcept
{
return data_;
}
constexpr std::string_view view() const noexcept
{
return std::string_view(data_, size_);
}
template<size_t N>
struct Static;
static StringData & make(std::pmr::memory_resource & resource, std::string_view s)
{
auto & res =
*new (resource.allocate(sizeof(StringData) + s.size() + 1, alignof(StringData))) StringData(s.size());
std::memcpy(res.data_, s.data(), s.size());
res.data_[s.size()] = '\0';
return res;
}
};
namespace detail {
/**

File diff suppressed because it is too large Load Diff

View File

@@ -1,5 +1,6 @@
#include "nix/expr/primops.hh"
#include "nix/expr/eval-inline.hh"
#include "nix/expr/string-data-static.hh"
#include "nix/store/derivations.hh"
#include "nix/store/store-api.hh"
#include "nix/store/globals.hh"
@@ -17,9 +18,9 @@ static void prim_unsafeDiscardStringContext(EvalState & state, const PosIdx pos,
static RegisterPrimOp primop_unsafeDiscardStringContext({
.name = "__unsafeDiscardStringContext",
.args = {"s"},
.doc = R"(
.doc = &R"(
Discard the [string context](@docroot@/language/string-context.md) from a value that can be coerced to a string.
)",
)"_sds,
.fun = prim_unsafeDiscardStringContext,
});
@@ -33,7 +34,7 @@ static void prim_hasContext(EvalState & state, const PosIdx pos, Value ** args,
static RegisterPrimOp primop_hasContext(
{.name = "__hasContext",
.args = {"s"},
.doc = R"(
.doc = &R"(
Return `true` if string *s* has a non-empty context.
The context can be obtained with
[`getContext`](#builtins-getContext).
@@ -50,7 +51,7 @@ static RegisterPrimOp primop_hasContext(
> then throw "package name cannot contain string context"
> else { ${name} = meta; }
> ```
)",
)"_sds,
.fun = prim_hasContext});
static void prim_unsafeDiscardOutputDependency(EvalState & state, const PosIdx pos, Value ** args, Value & v)
@@ -75,7 +76,7 @@ static void prim_unsafeDiscardOutputDependency(EvalState & state, const PosIdx p
static RegisterPrimOp primop_unsafeDiscardOutputDependency(
{.name = "__unsafeDiscardOutputDependency",
.args = {"s"},
.doc = R"(
.doc = &R"(
Create a copy of the given string where every
[derivation deep](@docroot@/language/string-context.md#string-context-element-derivation-deep)
string context element is turned into a
@@ -91,7 +92,7 @@ static RegisterPrimOp primop_unsafeDiscardOutputDependency(
Replacing a constant string context element with a "derivation deep" element is a safe operation that just enlargens the string context without forgetting anything.
[`builtins.addDrvOutputDependencies`]: #builtins-addDrvOutputDependencies
)",
)"_sds,
.fun = prim_unsafeDiscardOutputDependency});
static void prim_addDrvOutputDependencies(EvalState & state, const PosIdx pos, Value ** args, Value & v)
@@ -143,7 +144,7 @@ static void prim_addDrvOutputDependencies(EvalState & state, const PosIdx pos, V
static RegisterPrimOp primop_addDrvOutputDependencies(
{.name = "__addDrvOutputDependencies",
.args = {"s"},
.doc = R"(
.doc = &R"(
Create a copy of the given string where a single
[constant](@docroot@/language/string-context.md#string-context-constant)
string context element is turned into a
@@ -156,7 +157,7 @@ static RegisterPrimOp primop_addDrvOutputDependencies(
The latter is supported so this function is idempotent.
This is the opposite of [`builtins.unsafeDiscardOutputDependency`](#builtins-unsafeDiscardOutputDependency).
)",
)"_sds,
.fun = prim_addDrvOutputDependencies});
/* Extract the context of a string as a structured Nix value.
@@ -230,7 +231,7 @@ static void prim_getContext(EvalState & state, const PosIdx pos, Value ** args,
static RegisterPrimOp primop_getContext(
{.name = "__getContext",
.args = {"s"},
.doc = R"(
.doc = &R"(
Return the string context of *s*.
The string context tracks references to derivations within a string.
@@ -248,7 +249,7 @@ static RegisterPrimOp primop_getContext(
```
{ "/nix/store/arhvjaf6zmlyn8vh8fgn55rpwnxq0n7l-a.drv" = { outputs = [ "out" ]; }; }
```
)",
)"_sds,
.fun = prim_getContext});
/* Append the given context to a given string.

View File

@@ -1,4 +1,5 @@
#include "nix/expr/primops.hh"
#include "nix/expr/string-data-static.hh"
#include "nix/store/store-open.hh"
#include "nix/store/realisation.hh"
#include "nix/store/make-content-addressed.hh"
@@ -216,7 +217,7 @@ static void prim_fetchClosure(EvalState & state, const PosIdx pos, Value ** args
static RegisterPrimOp primop_fetchClosure({
.name = "__fetchClosure",
.args = {"args"},
.doc = R"(
.doc = &R"(
Fetch a store path [closure](@docroot@/glossary.md#gloss-closure) from a binary cache, and return the store path as a string with context.
This function can be invoked in three ways that we will discuss in order of preference.
@@ -284,7 +285,7 @@ static RegisterPrimOp primop_fetchClosure({
`fetchClosure` is similar to [`builtins.storePath`](#builtins-storePath) in that it allows you to use a previously built store path in a Nix expression.
However, `fetchClosure` is more reproducible because it specifies a binary cache from which the path can be fetched.
Also, using content-addressed store paths does not require users to configure [`trusted-public-keys`](@docroot@/command-ref/conf-file.md#conf-trusted-public-keys) to ensure their authenticity.
)",
)"_sds,
.fun = prim_fetchClosure,
.experimentalFeature = Xp::FetchClosure,
});

View File

@@ -81,7 +81,7 @@ static void prim_fetchMercurial(EvalState & state, const PosIdx pos, Value ** ar
attrs.insert_or_assign("rev", rev->gitRev());
auto input = fetchers::Input::fromAttrs(state.fetchSettings, std::move(attrs));
auto [storePath, input2] = input.fetchToStore(state.store);
auto [storePath, input2] = input.fetchToStore(state.fetchSettings, state.store);
auto attrs2 = state.buildBindings(8);
state.mkStorePathString(storePath, attrs2.alloc(state.s.outPath));

View File

@@ -2,6 +2,7 @@
#include "nix/expr/primops.hh"
#include "nix/expr/eval-inline.hh"
#include "nix/expr/eval-settings.hh"
#include "nix/expr/string-data-static.hh"
#include "nix/store/store-api.hh"
#include "nix/fetchers/fetchers.hh"
#include "nix/store/filetransfer.hh"
@@ -82,7 +83,7 @@ struct FetchTreeParams
static void fetchTree(
EvalState & state, const PosIdx pos, Value ** args, Value & v, const FetchTreeParams & params = FetchTreeParams{})
{
fetchers::Input input{state.fetchSettings};
fetchers::Input input{};
NixStringContext context;
std::optional<std::string> type;
auto fetcher = params.isFetchGit ? "fetchGit" : "fetchTree";
@@ -194,9 +195,9 @@ static void fetchTree(
}
if (!state.settings.pureEval && !input.isDirect() && experimentalFeatureSettings.isEnabled(Xp::Flakes))
input = lookupInRegistries(state.store, input, fetchers::UseRegistries::Limited).first;
input = lookupInRegistries(state.fetchSettings, state.store, input, fetchers::UseRegistries::Limited).first;
if (state.settings.pureEval && !input.isLocked()) {
if (state.settings.pureEval && !input.isLocked(state.fetchSettings)) {
if (input.getNarHash())
warn(
"Input '%s' is unlocked (e.g. lacks a Git revision) but is checked by NAR hash. "
@@ -219,7 +220,8 @@ static void fetchTree(
throw Error("input '%s' is not allowed to use the '__final' attribute", input.to_string());
}
auto cachedInput = state.inputCache->getAccessor(state.store, input, fetchers::UseRegistries::No);
auto cachedInput =
state.inputCache->getAccessor(state.fetchSettings, state.store, input, fetchers::UseRegistries::No);
auto storePath = state.mountInput(cachedInput.lockedInput, input, cachedInput.accessor);
@@ -234,7 +236,7 @@ static void prim_fetchTree(EvalState & state, const PosIdx pos, Value ** args, V
static RegisterPrimOp primop_fetchTree({
.name = "fetchTree",
.args = {"input"},
.doc = R"(
.doc = &R"(
Fetch a file system tree or a plain file using one of the supported backends and return an attribute set with:
- the resulting fixed-output [store path](@docroot@/store/store-path.md)
@@ -456,7 +458,7 @@ static RegisterPrimOp primop_fetchTree({
> ```nix
> builtins.fetchTree "github:NixOS/nixpkgs/ae2e6b3958682513d28f7d633734571fb18285dd"
> ```
)",
)"_sds,
.fun = prim_fetchTree,
.experimentalFeature = Xp::FetchTree,
});
@@ -616,7 +618,7 @@ static void prim_fetchurl(EvalState & state, const PosIdx pos, Value ** args, Va
static RegisterPrimOp primop_fetchurl({
.name = "__fetchurl",
.args = {"arg"},
.doc = R"(
.doc = &R"(
Download the specified URL and return the path of the downloaded file.
`arg` can be either a string denoting the URL, or an attribute set with the following attributes:
@@ -630,7 +632,7 @@ static RegisterPrimOp primop_fetchurl({
characters that are invalid for the store.
Not available in [restricted evaluation mode](@docroot@/command-ref/conf-file.md#conf-restrict-eval).
)",
)"_sds,
.fun = prim_fetchurl,
});
@@ -642,7 +644,7 @@ static void prim_fetchTarball(EvalState & state, const PosIdx pos, Value ** args
static RegisterPrimOp primop_fetchTarball({
.name = "fetchTarball",
.args = {"args"},
.doc = R"(
.doc = &R"(
Download the specified URL, unpack it and return the path of the
unpacked tree. The file must be a tape archive (`.tar`) compressed
with `gzip`, `bzip2` or `xz`. If the tarball consists of a
@@ -680,7 +682,7 @@ static RegisterPrimOp primop_fetchTarball({
```
Not available in [restricted evaluation mode](@docroot@/command-ref/conf-file.md#conf-restrict-eval).
)",
)"_sds,
.fun = prim_fetchTarball,
});
@@ -693,7 +695,7 @@ static void prim_fetchGit(EvalState & state, const PosIdx pos, Value ** args, Va
static RegisterPrimOp primop_fetchGit({
.name = "fetchGit",
.args = {"args"},
.doc = R"(
.doc = &R"(
Fetch a path from git. *args* can be a URL, in which case the HEAD
of the repo at that URL is fetched. Otherwise, it can be an
attribute with the following attributes (all except `url` optional):
@@ -896,7 +898,7 @@ static RegisterPrimOp primop_fetchGit({
given, `fetchGit` uses the current content of the checked-out
files, even if they are not committed or added to Git's index. It
only considers files added to the Git repository, as listed by `git ls-files`.
)",
)"_sds,
.fun = prim_fetchGit,
});

View File

@@ -1,6 +1,6 @@
#include "nix/expr/primops.hh"
#include "nix/expr/eval-inline.hh"
#include "nix/expr/static-string-data.hh"
#include "nix/expr/string-data-static.hh"
#include "expr-config-private.hh"
@@ -170,10 +170,10 @@ static void prim_fromTOML(EvalState & state, const PosIdx pos, Value ** args, Va
}
}
static RegisterPrimOp primop_fromTOML(
{.name = "fromTOML",
.args = {"e"},
.doc = R"(
static RegisterPrimOp primop_fromTOML({
.name = "fromTOML",
.args = {"e"},
.doc = &R"(
Convert a TOML string to a Nix value. For example,
```nix
@@ -186,7 +186,8 @@ static RegisterPrimOp primop_fromTOML(
```
returns the value `{ s = "a"; table = { y = 2; }; x = 1; }`.
)",
.fun = prim_fromTOML});
)"_sds,
.fun = prim_fromTOML,
});
} // namespace nix

View File

@@ -196,7 +196,7 @@ TEST_F(GitTest, submodulePeriodSupport)
{"ref", "main"},
});
auto [accessor, i] = input.getAccessor(store);
auto [accessor, i] = input.getAccessor(settings, store);
ASSERT_EQ(accessor->readFile(CanonPath("deps/sub/lib.txt")), "hello from submodule\n");
}

View File

@@ -89,7 +89,7 @@ Input Input::fromAttrs(const Settings & settings, Attrs && attrs)
// but not all of them. Doing this is to support those other
// operations which are supposed to be robust on
// unknown/uninterpretable inputs.
Input input{settings};
Input input;
input.attrs = attrs;
fixupInput(input);
return input;
@@ -159,9 +159,9 @@ bool Input::isDirect() const
return !scheme || scheme->isDirect(*this);
}
bool Input::isLocked() const
bool Input::isLocked(const Settings & settings) const
{
return scheme && scheme->isLocked(*this);
return scheme && scheme->isLocked(settings, *this);
}
bool Input::isFinal() const
@@ -198,17 +198,17 @@ bool Input::contains(const Input & other) const
}
// FIXME: remove
std::pair<StorePath, Input> Input::fetchToStore(ref<Store> store) const
std::pair<StorePath, Input> Input::fetchToStore(const Settings & settings, ref<Store> store) const
{
if (!scheme)
throw Error("cannot fetch unsupported input '%s'", attrsToJSON(toAttrs()));
auto [storePath, input] = [&]() -> std::pair<StorePath, Input> {
try {
auto [accessor, result] = getAccessorUnchecked(store);
auto [accessor, result] = getAccessorUnchecked(settings, store);
auto storePath =
nix::fetchToStore(*settings, *store, SourcePath(accessor), FetchMode::Copy, result.getName());
nix::fetchToStore(settings, *store, SourcePath(accessor), FetchMode::Copy, result.getName());
auto narHash = store->queryPathInfo(storePath)->narHash;
result.attrs.insert_or_assign("narHash", narHash.to_string(HashFormat::SRI, true));
@@ -297,10 +297,10 @@ void Input::checkLocks(Input specified, Input & result)
}
}
std::pair<ref<SourceAccessor>, Input> Input::getAccessor(ref<Store> store) const
std::pair<ref<SourceAccessor>, Input> Input::getAccessor(const Settings & settings, ref<Store> store) const
{
try {
auto [accessor, result] = getAccessorUnchecked(store);
auto [accessor, result] = getAccessorUnchecked(settings, store);
result.attrs.insert_or_assign("__final", Explicit<bool>(true));
@@ -313,7 +313,7 @@ std::pair<ref<SourceAccessor>, Input> Input::getAccessor(ref<Store> store) const
}
}
std::pair<ref<SourceAccessor>, Input> Input::getAccessorUnchecked(ref<Store> store) const
std::pair<ref<SourceAccessor>, Input> Input::getAccessorUnchecked(const Settings & settings, ref<Store> store) const
{
// FIXME: cache the accessor
@@ -349,7 +349,7 @@ std::pair<ref<SourceAccessor>, Input> Input::getAccessorUnchecked(ref<Store> sto
if (accessor->fingerprint) {
ContentAddressMethod method = ContentAddressMethod::Raw::NixArchive;
auto cacheKey = makeFetchToStoreCacheKey(getName(), *accessor->fingerprint, method, "/");
settings->getCache()->upsert(cacheKey, *store, {}, storePath);
settings.getCache()->upsert(cacheKey, *store, {}, storePath);
}
accessor->setPathDisplay("«" + to_string() + "»");
@@ -360,7 +360,7 @@ std::pair<ref<SourceAccessor>, Input> Input::getAccessorUnchecked(ref<Store> sto
}
}
auto [accessor, result] = scheme->getAccessor(store, *this);
auto [accessor, result] = scheme->getAccessor(settings, store, *this);
if (!accessor->fingerprint)
accessor->fingerprint = result.getFingerprint(store);
@@ -377,10 +377,10 @@ Input Input::applyOverrides(std::optional<std::string> ref, std::optional<Hash>
return scheme->applyOverrides(*this, ref, rev);
}
void Input::clone(const Path & destDir) const
void Input::clone(const Settings & settings, const Path & destDir) const
{
assert(scheme);
scheme->clone(*this, destDir);
scheme->clone(settings, *this, destDir);
}
std::optional<std::filesystem::path> Input::getSourcePath() const
@@ -493,7 +493,7 @@ void InputScheme::putFile(
throw Error("input '%s' does not support modifying file '%s'", input.to_string(), path);
}
void InputScheme::clone(const Input & input, const Path & destDir) const
void InputScheme::clone(const Settings & settings, const Input & input, const Path & destDir) const
{
throw Error("do not know how to clone input '%s'", input.to_string());
}

View File

@@ -229,7 +229,7 @@ struct GitInputScheme : InputScheme
if (auto ref = maybeGetStrAttr(attrs, "ref"); ref && !isLegalRefName(*ref))
throw BadURL("invalid Git branch/tag name '%s'", *ref);
Input input{settings};
Input input{};
input.attrs = attrs;
input.attrs["url"] = fixGitURL(getStrAttr(attrs, "url")).to_string();
getShallowAttr(input);
@@ -278,7 +278,7 @@ struct GitInputScheme : InputScheme
return res;
}
void clone(const Input & input, const Path & destDir) const override
void clone(const Settings & settings, const Input & input, const Path & destDir) const override
{
auto repoInfo = getRepoInfo(input);
@@ -623,7 +623,7 @@ struct GitInputScheme : InputScheme
}
std::pair<ref<SourceAccessor>, Input>
getAccessorFromCommit(ref<Store> store, RepoInfo & repoInfo, Input && input) const
getAccessorFromCommit(const Settings & settings, ref<Store> store, RepoInfo & repoInfo, Input && input) const
{
assert(!repoInfo.workdirInfo.isDirty);
@@ -733,10 +733,10 @@ struct GitInputScheme : InputScheme
auto rev = *input.getRev();
input.attrs.insert_or_assign("lastModified", getLastModified(*input.settings, repoInfo, repoDir, rev));
input.attrs.insert_or_assign("lastModified", getLastModified(settings, repoInfo, repoDir, rev));
if (!getShallowAttr(input))
input.attrs.insert_or_assign("revCount", getRevCount(*input.settings, repoInfo, repoDir, rev));
input.attrs.insert_or_assign("revCount", getRevCount(settings, repoInfo, repoDir, rev));
printTalkative("using revision %s of repo '%s'", rev.gitRev(), repoInfo.locationToArg());
@@ -779,8 +779,8 @@ struct GitInputScheme : InputScheme
attrs.insert_or_assign("submodules", Explicit<bool>{true});
attrs.insert_or_assign("lfs", Explicit<bool>{smudgeLfs});
attrs.insert_or_assign("allRefs", Explicit<bool>{true});
auto submoduleInput = fetchers::Input::fromAttrs(*input.settings, std::move(attrs));
auto [submoduleAccessor, submoduleInput2] = submoduleInput.getAccessor(store);
auto submoduleInput = fetchers::Input::fromAttrs(settings, std::move(attrs));
auto [submoduleAccessor, submoduleInput2] = submoduleInput.getAccessor(settings, store);
submoduleAccessor->setPathDisplay("«" + submoduleInput.to_string() + "»");
mounts.insert_or_assign(submodule.path, submoduleAccessor);
}
@@ -797,7 +797,7 @@ struct GitInputScheme : InputScheme
}
std::pair<ref<SourceAccessor>, Input>
getAccessorFromWorkdir(ref<Store> store, RepoInfo & repoInfo, Input && input) const
getAccessorFromWorkdir(const Settings & settings, ref<Store> store, RepoInfo & repoInfo, Input && input) const
{
auto repoPath = repoInfo.getPath().value();
@@ -829,8 +829,8 @@ struct GitInputScheme : InputScheme
// TODO: fall back to getAccessorFromCommit-like fetch when submodules aren't checked out
// attrs.insert_or_assign("allRefs", Explicit<bool>{ true });
auto submoduleInput = fetchers::Input::fromAttrs(*input.settings, std::move(attrs));
auto [submoduleAccessor, submoduleInput2] = submoduleInput.getAccessor(store);
auto submoduleInput = fetchers::Input::fromAttrs(settings, std::move(attrs));
auto [submoduleAccessor, submoduleInput2] = submoduleInput.getAccessor(settings, store);
submoduleAccessor->setPathDisplay("«" + submoduleInput.to_string() + "»");
/* If the submodule is dirty, mark this repo dirty as
@@ -857,12 +857,12 @@ struct GitInputScheme : InputScheme
input.attrs.insert_or_assign("rev", rev.gitRev());
if (!getShallowAttr(input)) {
input.attrs.insert_or_assign(
"revCount", rev == nullRev ? 0 : getRevCount(*input.settings, repoInfo, repoPath, rev));
"revCount", rev == nullRev ? 0 : getRevCount(settings, repoInfo, repoPath, rev));
}
verifyCommit(input, repo);
} else {
repoInfo.warnDirty(*input.settings);
repoInfo.warnDirty(settings);
if (repoInfo.workdirInfo.headRev) {
input.attrs.insert_or_assign("dirtyRev", repoInfo.workdirInfo.headRev->gitRev() + "-dirty");
@@ -874,14 +874,14 @@ struct GitInputScheme : InputScheme
input.attrs.insert_or_assign(
"lastModified",
repoInfo.workdirInfo.headRev
? getLastModified(*input.settings, repoInfo, repoPath, *repoInfo.workdirInfo.headRev)
: 0);
repoInfo.workdirInfo.headRev ? getLastModified(settings, repoInfo, repoPath, *repoInfo.workdirInfo.headRev)
: 0);
return {accessor, std::move(input)};
}
std::pair<ref<SourceAccessor>, Input> getAccessor(ref<Store> store, const Input & _input) const override
std::pair<ref<SourceAccessor>, Input>
getAccessor(const Settings & settings, ref<Store> store, const Input & _input) const override
{
Input input(_input);
@@ -897,8 +897,8 @@ struct GitInputScheme : InputScheme
}
auto [accessor, final] = input.getRef() || input.getRev() || !repoInfo.getPath()
? getAccessorFromCommit(store, repoInfo, std::move(input))
: getAccessorFromWorkdir(store, repoInfo, std::move(input));
? getAccessorFromCommit(settings, store, repoInfo, std::move(input))
: getAccessorFromWorkdir(settings, store, repoInfo, std::move(input));
return {accessor, std::move(final)};
}
@@ -934,7 +934,7 @@ struct GitInputScheme : InputScheme
}
}
bool isLocked(const Input & input) const override
bool isLocked(const Settings & settings, const Input & input) const override
{
auto rev = input.getRev();
return rev && rev != nullRev;

View File

@@ -92,7 +92,7 @@ struct GitArchiveInputScheme : InputScheme
if (ref && rev)
throw BadURL("URL '%s' contains both a commit hash and a branch/tag name %s %s", url, *ref, rev->gitRev());
Input input{settings};
Input input{};
input.attrs.insert_or_assign("type", std::string{schemeName()});
input.attrs.insert_or_assign("owner", path[0]);
input.attrs.insert_or_assign("repo", path[1]);
@@ -129,7 +129,7 @@ struct GitArchiveInputScheme : InputScheme
getStrAttr(attrs, "owner");
getStrAttr(attrs, "repo");
Input input{settings};
Input input{};
input.attrs = attrs;
return input;
}
@@ -233,9 +233,9 @@ struct GitArchiveInputScheme : InputScheme
std::optional<Hash> treeHash;
};
virtual RefInfo getRevFromRef(nix::ref<Store> store, const Input & input) const = 0;
virtual RefInfo getRevFromRef(const Settings & settings, nix::ref<Store> store, const Input & input) const = 0;
virtual DownloadUrl getDownloadUrl(const Input & input) const = 0;
virtual DownloadUrl getDownloadUrl(const Settings & settings, const Input & input) const = 0;
struct TarballInfo
{
@@ -243,7 +243,7 @@ struct GitArchiveInputScheme : InputScheme
time_t lastModified;
};
std::pair<Input, TarballInfo> downloadArchive(ref<Store> store, Input input) const
std::pair<Input, TarballInfo> downloadArchive(const Settings & settings, ref<Store> store, Input input) const
{
if (!maybeGetStrAttr(input.attrs, "ref"))
input.attrs.insert_or_assign("ref", "HEAD");
@@ -252,7 +252,7 @@ struct GitArchiveInputScheme : InputScheme
auto rev = input.getRev();
if (!rev) {
auto refInfo = getRevFromRef(store, input);
auto refInfo = getRevFromRef(settings, store, input);
rev = refInfo.rev;
upstreamTreeHash = refInfo.treeHash;
debug("HEAD revision for '%s' is %s", input.to_string(), refInfo.rev.gitRev());
@@ -261,7 +261,7 @@ struct GitArchiveInputScheme : InputScheme
input.attrs.erase("ref");
input.attrs.insert_or_assign("rev", rev->gitRev());
auto cache = input.settings->getCache();
auto cache = settings.getCache();
Cache::Key treeHashKey{"gitRevToTreeHash", {{"rev", rev->gitRev()}}};
Cache::Key lastModifiedKey{"gitRevToLastModified", {{"rev", rev->gitRev()}}};
@@ -270,7 +270,7 @@ struct GitArchiveInputScheme : InputScheme
if (auto lastModifiedAttrs = cache->lookup(lastModifiedKey)) {
auto treeHash = getRevAttr(*treeHashAttrs, "treeHash");
auto lastModified = getIntAttr(*lastModifiedAttrs, "lastModified");
if (input.settings->getTarballCache()->hasObject(treeHash))
if (settings.getTarballCache()->hasObject(treeHash))
return {std::move(input), TarballInfo{.treeHash = treeHash, .lastModified = (time_t) lastModified}};
else
debug("Git tree with hash '%s' has disappeared from the cache, refetching...", treeHash.gitRev());
@@ -278,7 +278,7 @@ struct GitArchiveInputScheme : InputScheme
}
/* Stream the tarball into the tarball cache. */
auto url = getDownloadUrl(input);
auto url = getDownloadUrl(settings, input);
auto source = sinkToSource([&](Sink & sink) {
FileTransferRequest req(url.url);
@@ -290,7 +290,7 @@ struct GitArchiveInputScheme : InputScheme
*logger, lvlInfo, actUnknown, fmt("unpacking '%s' into the Git cache", input.to_string()));
TarArchive archive{*source};
auto tarballCache = input.settings->getTarballCache();
auto tarballCache = settings.getTarballCache();
auto parseSink = tarballCache->getFileSystemObjectSink();
auto lastModified = unpackTarfileToSink(archive, *parseSink);
auto tree = parseSink->flush();
@@ -315,9 +315,10 @@ struct GitArchiveInputScheme : InputScheme
return {std::move(input), tarballInfo};
}
std::pair<ref<SourceAccessor>, Input> getAccessor(ref<Store> store, const Input & _input) const override
std::pair<ref<SourceAccessor>, Input>
getAccessor(const Settings & settings, ref<Store> store, const Input & _input) const override
{
auto [input, tarballInfo] = downloadArchive(store, _input);
auto [input, tarballInfo] = downloadArchive(settings, store, _input);
#if 0
input.attrs.insert_or_assign("treeHash", tarballInfo.treeHash.gitRev());
@@ -325,19 +326,18 @@ struct GitArchiveInputScheme : InputScheme
input.attrs.insert_or_assign("lastModified", uint64_t(tarballInfo.lastModified));
auto accessor =
input.settings->getTarballCache()->getAccessor(tarballInfo.treeHash, false, "«" + input.to_string() + "»");
settings.getTarballCache()->getAccessor(tarballInfo.treeHash, false, "«" + input.to_string() + "»");
return {accessor, input};
}
bool isLocked(const Input & input) const override
bool isLocked(const Settings & settings, const Input & input) const override
{
/* Since we can't verify the integrity of the tarball from the
Git revision alone, we also require a NAR hash for
locking. FIXME: in the future, we may want to require a Git
tree hash instead of a NAR hash. */
return input.getRev().has_value()
&& (input.settings->trustTarballsFromGitForges || input.getNarHash().has_value());
return input.getRev().has_value() && (settings.trustTarballsFromGitForges || input.getNarHash().has_value());
}
std::optional<ExperimentalFeature> experimentalFeature() const override
@@ -387,7 +387,7 @@ struct GitHubInputScheme : GitArchiveInputScheme
return getStrAttr(input.attrs, "repo");
}
RefInfo getRevFromRef(nix::ref<Store> store, const Input & input) const override
RefInfo getRevFromRef(const Settings & settings, nix::ref<Store> store, const Input & input) const override
{
auto host = getHost(input);
auto url = fmt(
@@ -397,9 +397,9 @@ struct GitHubInputScheme : GitArchiveInputScheme
getRepo(input),
*input.getRef());
Headers headers = makeHeadersWithAuthTokens(*input.settings, host, input);
Headers headers = makeHeadersWithAuthTokens(settings, host, input);
auto downloadResult = downloadFile(store, *input.settings, url, "source", headers);
auto downloadResult = downloadFile(store, settings, url, "source", headers);
auto json = nlohmann::json::parse(
store->requireStoreObjectAccessor(downloadResult.storePath)->readFile(CanonPath::root));
@@ -408,11 +408,11 @@ struct GitHubInputScheme : GitArchiveInputScheme
.treeHash = Hash::parseAny(std::string{json["commit"]["tree"]["sha"]}, HashAlgorithm::SHA1)};
}
DownloadUrl getDownloadUrl(const Input & input) const override
DownloadUrl getDownloadUrl(const Settings & settings, const Input & input) const override
{
auto host = getHost(input);
Headers headers = makeHeadersWithAuthTokens(*input.settings, host, input);
Headers headers = makeHeadersWithAuthTokens(settings, host, input);
// If we have no auth headers then we default to the public archive
// urls so we do not run into rate limits.
@@ -426,12 +426,12 @@ struct GitHubInputScheme : GitArchiveInputScheme
return DownloadUrl{parseURL(url), headers};
}
void clone(const Input & input, const Path & destDir) const override
void clone(const Settings & settings, const Input & input, const Path & destDir) const override
{
auto host = getHost(input);
Input::fromURL(*input.settings, fmt("git+https://%s/%s/%s.git", host, getOwner(input), getRepo(input)))
Input::fromURL(settings, fmt("git+https://%s/%s/%s.git", host, getOwner(input), getRepo(input)))
.applyOverrides(input.getRef(), input.getRev())
.clone(destDir);
.clone(settings, destDir);
}
};
@@ -461,7 +461,7 @@ struct GitLabInputScheme : GitArchiveInputScheme
return std::make_pair(token.substr(0, fldsplit), token.substr(fldsplit + 1));
}
RefInfo getRevFromRef(nix::ref<Store> store, const Input & input) const override
RefInfo getRevFromRef(const Settings & settings, nix::ref<Store> store, const Input & input) const override
{
auto host = maybeGetStrAttr(input.attrs, "host").value_or("gitlab.com");
// See rate limiting note below
@@ -472,9 +472,9 @@ struct GitLabInputScheme : GitArchiveInputScheme
getStrAttr(input.attrs, "repo"),
*input.getRef());
Headers headers = makeHeadersWithAuthTokens(*input.settings, host, input);
Headers headers = makeHeadersWithAuthTokens(settings, host, input);
auto downloadResult = downloadFile(store, *input.settings, url, "source", headers);
auto downloadResult = downloadFile(store, settings, url, "source", headers);
auto json = nlohmann::json::parse(
store->requireStoreObjectAccessor(downloadResult.storePath)->readFile(CanonPath::root));
@@ -488,7 +488,7 @@ struct GitLabInputScheme : GitArchiveInputScheme
}
}
DownloadUrl getDownloadUrl(const Input & input) const override
DownloadUrl getDownloadUrl(const Settings & settings, const Input & input) const override
{
// This endpoint has a rate limit threshold that may be
// server-specific and vary based whether the user is
@@ -503,19 +503,19 @@ struct GitLabInputScheme : GitArchiveInputScheme
getStrAttr(input.attrs, "repo"),
input.getRev()->to_string(HashFormat::Base16, false));
Headers headers = makeHeadersWithAuthTokens(*input.settings, host, input);
Headers headers = makeHeadersWithAuthTokens(settings, host, input);
return DownloadUrl{parseURL(url), headers};
}
void clone(const Input & input, const Path & destDir) const override
void clone(const Settings & settings, const Input & input, const Path & destDir) const override
{
auto host = maybeGetStrAttr(input.attrs, "host").value_or("gitlab.com");
// FIXME: get username somewhere
Input::fromURL(
*input.settings,
settings,
fmt("git+https://%s/%s/%s.git", host, getStrAttr(input.attrs, "owner"), getStrAttr(input.attrs, "repo")))
.applyOverrides(input.getRef(), input.getRev())
.clone(destDir);
.clone(settings, destDir);
}
};
@@ -536,7 +536,7 @@ struct SourceHutInputScheme : GitArchiveInputScheme
// Once it is implemented, however, should work as expected.
}
RefInfo getRevFromRef(nix::ref<Store> store, const Input & input) const override
RefInfo getRevFromRef(const Settings & settings, nix::ref<Store> store, const Input & input) const override
{
// TODO: In the future, when the sourcehut graphql API is implemented for mercurial
// and with anonymous access, this method should use it instead.
@@ -547,11 +547,11 @@ struct SourceHutInputScheme : GitArchiveInputScheme
auto base_url =
fmt("https://%s/%s/%s", host, getStrAttr(input.attrs, "owner"), getStrAttr(input.attrs, "repo"));
Headers headers = makeHeadersWithAuthTokens(*input.settings, host, input);
Headers headers = makeHeadersWithAuthTokens(settings, host, input);
std::string refUri;
if (ref == "HEAD") {
auto downloadFileResult = downloadFile(store, *input.settings, fmt("%s/HEAD", base_url), "source", headers);
auto downloadFileResult = downloadFile(store, settings, fmt("%s/HEAD", base_url), "source", headers);
auto contents = store->requireStoreObjectAccessor(downloadFileResult.storePath)->readFile(CanonPath::root);
auto remoteLine = git::parseLsRemoteLine(getLine(contents).first);
@@ -564,8 +564,7 @@ struct SourceHutInputScheme : GitArchiveInputScheme
}
std::regex refRegex(refUri);
auto downloadFileResult =
downloadFile(store, *input.settings, fmt("%s/info/refs", base_url), "source", headers);
auto downloadFileResult = downloadFile(store, settings, fmt("%s/info/refs", base_url), "source", headers);
auto contents = store->requireStoreObjectAccessor(downloadFileResult.storePath)->readFile(CanonPath::root);
std::istringstream is(contents);
@@ -583,7 +582,7 @@ struct SourceHutInputScheme : GitArchiveInputScheme
return RefInfo{.rev = Hash::parseAny(*id, HashAlgorithm::SHA1)};
}
DownloadUrl getDownloadUrl(const Input & input) const override
DownloadUrl getDownloadUrl(const Settings & settings, const Input & input) const override
{
auto host = maybeGetStrAttr(input.attrs, "host").value_or("git.sr.ht");
auto url =
@@ -593,18 +592,18 @@ struct SourceHutInputScheme : GitArchiveInputScheme
getStrAttr(input.attrs, "repo"),
input.getRev()->to_string(HashFormat::Base16, false));
Headers headers = makeHeadersWithAuthTokens(*input.settings, host, input);
Headers headers = makeHeadersWithAuthTokens(settings, host, input);
return DownloadUrl{parseURL(url), headers};
}
void clone(const Input & input, const Path & destDir) const override
void clone(const Settings & settings, const Input & input, const Path & destDir) const override
{
auto host = maybeGetStrAttr(input.attrs, "host").value_or("git.sr.ht");
Input::fromURL(
*input.settings,
settings,
fmt("git+https://%s/%s/%s", host, getStrAttr(input.attrs, "owner"), getStrAttr(input.attrs, "repo")))
.applyOverrides(input.getRef(), input.getRev())
.clone(destDir);
.clone(settings, destDir);
}
};

View File

@@ -36,13 +36,6 @@ struct Input
{
friend struct InputScheme;
const Settings * settings;
Input(const Settings & settings)
: settings{&settings}
{
}
std::shared_ptr<InputScheme> scheme; // note: can be null
Attrs attrs;
@@ -87,7 +80,7 @@ public:
* attributes like a Git revision or NAR hash that uniquely
* identify its contents.
*/
bool isLocked() const;
bool isLocked(const Settings & settings) const;
/**
* Only for relative path flakes, i.e. 'path:./foo', returns the
@@ -120,7 +113,7 @@ public:
* Fetch the entire input into the Nix store, returning the
* location in the Nix store and the locked input.
*/
std::pair<StorePath, Input> fetchToStore(ref<Store> store) const;
std::pair<StorePath, Input> fetchToStore(const Settings & settings, ref<Store> store) const;
/**
* Check the locking attributes in `result` against
@@ -140,17 +133,17 @@ public:
* input without copying it to the store. Also return a possibly
* unlocked input.
*/
std::pair<ref<SourceAccessor>, Input> getAccessor(ref<Store> store) const;
std::pair<ref<SourceAccessor>, Input> getAccessor(const Settings & settings, ref<Store> store) const;
private:
std::pair<ref<SourceAccessor>, Input> getAccessorUnchecked(ref<Store> store) const;
std::pair<ref<SourceAccessor>, Input> getAccessorUnchecked(const Settings & settings, ref<Store> store) const;
public:
Input applyOverrides(std::optional<std::string> ref, std::optional<Hash> rev) const;
void clone(const Path & destDir) const;
void clone(const Settings & settings, const Path & destDir) const;
std::optional<std::filesystem::path> getSourcePath() const;
@@ -223,7 +216,7 @@ struct InputScheme
virtual Input applyOverrides(const Input & input, std::optional<std::string> ref, std::optional<Hash> rev) const;
virtual void clone(const Input & input, const Path & destDir) const;
virtual void clone(const Settings & settings, const Input & input, const Path & destDir) const;
virtual std::optional<std::filesystem::path> getSourcePath(const Input & input) const;
@@ -233,7 +226,8 @@ struct InputScheme
std::string_view contents,
std::optional<std::string> commitMsg) const;
virtual std::pair<ref<SourceAccessor>, Input> getAccessor(ref<Store> store, const Input & input) const = 0;
virtual std::pair<ref<SourceAccessor>, Input>
getAccessor(const Settings & settings, ref<Store> store, const Input & input) const = 0;
/**
* Is this `InputScheme` part of an experimental feature?
@@ -250,7 +244,7 @@ struct InputScheme
return std::nullopt;
}
virtual bool isLocked(const Input & input) const
virtual bool isLocked(const Settings & settings, const Input & input) const
{
return false;
}

View File

@@ -3,6 +3,7 @@
namespace nix::fetchers {
enum class UseRegistries : int;
struct Settings;
struct InputCache
{
@@ -14,7 +15,8 @@ struct InputCache
Attrs extraAttrs;
};
CachedResult getAccessor(ref<Store> store, const Input & originalInput, UseRegistries useRegistries);
CachedResult
getAccessor(const Settings & settings, ref<Store> store, const Input & originalInput, UseRegistries useRegistries);
struct CachedInput
{

View File

@@ -59,7 +59,7 @@ Path getUserRegistryPath();
Registries getRegistries(const Settings & settings, ref<Store> store);
void overrideRegistry(const Input & from, const Input & to, const Attrs & extraAttrs);
void overrideRegistry(const Settings & settings, const Input & from, const Input & to, const Attrs & extraAttrs);
enum class UseRegistries : int {
No,
@@ -71,6 +71,7 @@ enum class UseRegistries : int {
* Rewrite a flakeref using the registries. If `filter` is set, only
* use the registries for which the filter function returns true.
*/
std::pair<Input, Attrs> lookupInRegistries(ref<Store> store, const Input & input, UseRegistries useRegistries);
std::pair<Input, Attrs>
lookupInRegistries(const Settings & settings, ref<Store> store, const Input & input, UseRegistries useRegistries);
} // namespace nix::fetchers

View File

@@ -44,7 +44,7 @@ struct IndirectInputScheme : InputScheme
// FIXME: forbid query params?
Input input{settings};
Input input{};
input.attrs.insert_or_assign("type", "indirect");
input.attrs.insert_or_assign("id", id);
if (rev)
@@ -76,7 +76,7 @@ struct IndirectInputScheme : InputScheme
if (!std::regex_match(id, flakeRegex))
throw BadURL("'%s' is not a valid flake ID", id);
Input input{settings};
Input input{};
input.attrs = attrs;
return input;
}
@@ -106,7 +106,8 @@ struct IndirectInputScheme : InputScheme
return input;
}
std::pair<ref<SourceAccessor>, Input> getAccessor(ref<Store> store, const Input & input) const override
std::pair<ref<SourceAccessor>, Input>
getAccessor(const Settings & settings, ref<Store> store, const Input & input) const override
{
throw Error("indirect input '%s' cannot be fetched directly", input.to_string());
}

View File

@@ -5,23 +5,23 @@
namespace nix::fetchers {
InputCache::CachedResult
InputCache::getAccessor(ref<Store> store, const Input & originalInput, UseRegistries useRegistries)
InputCache::CachedResult InputCache::getAccessor(
const Settings & settings, ref<Store> store, const Input & originalInput, UseRegistries useRegistries)
{
auto fetched = lookup(originalInput);
Input resolvedInput = originalInput;
if (!fetched) {
if (originalInput.isDirect()) {
auto [accessor, lockedInput] = originalInput.getAccessor(store);
auto [accessor, lockedInput] = originalInput.getAccessor(settings, store);
fetched.emplace(CachedInput{.lockedInput = lockedInput, .accessor = accessor});
} else {
if (useRegistries != UseRegistries::No) {
auto [res, extraAttrs] = lookupInRegistries(store, originalInput, useRegistries);
auto [res, extraAttrs] = lookupInRegistries(settings, store, originalInput, useRegistries);
resolvedInput = std::move(res);
fetched = lookup(resolvedInput);
if (!fetched) {
auto [accessor, lockedInput] = resolvedInput.getAccessor(store);
auto [accessor, lockedInput] = resolvedInput.getAccessor(settings, store);
fetched.emplace(
CachedInput{.lockedInput = lockedInput, .accessor = accessor, .extraAttrs = extraAttrs});
}

View File

@@ -89,7 +89,7 @@ struct MercurialInputScheme : InputScheme
throw BadURL("invalid Mercurial branch/tag name '%s'", *ref);
}
Input input{settings};
Input input{};
input.attrs = attrs;
return input;
}
@@ -154,7 +154,7 @@ struct MercurialInputScheme : InputScheme
return {isLocal, isLocal ? renderUrlPathEnsureLegal(url.path) : url.to_string()};
}
StorePath fetchToStore(ref<Store> store, Input & input) const
StorePath fetchToStore(const Settings & settings, ref<Store> store, Input & input) const
{
auto origRev = input.getRev();
@@ -176,10 +176,10 @@ struct MercurialInputScheme : InputScheme
/* This is an unclean working tree. So copy all tracked
files. */
if (!input.settings->allowDirty)
if (!settings.allowDirty)
throw Error("Mercurial tree '%s' is unclean", actualUrl);
if (input.settings->warnDirty)
if (settings.warnDirty)
warn("Mercurial tree '%s' is unclean", actualUrl);
input.attrs.insert_or_assign("ref", chomp(runHg({"branch", "-R", actualUrl})));
@@ -240,13 +240,13 @@ struct MercurialInputScheme : InputScheme
Cache::Key refToRevKey{"hgRefToRev", {{"url", actualUrl}, {"ref", *input.getRef()}}};
if (!input.getRev()) {
if (auto res = input.settings->getCache()->lookupWithTTL(refToRevKey))
if (auto res = settings.getCache()->lookupWithTTL(refToRevKey))
input.attrs.insert_or_assign("rev", getRevAttr(*res, "rev").gitRev());
}
/* If we have a rev, check if we have a cached store path. */
if (auto rev = input.getRev()) {
if (auto res = input.settings->getCache()->lookupStorePath(revInfoKey(*rev), *store))
if (auto res = settings.getCache()->lookupStorePath(revInfoKey(*rev), *store))
return makeResult(res->value, res->storePath);
}
@@ -300,7 +300,7 @@ struct MercurialInputScheme : InputScheme
/* Now that we have the rev, check the cache again for a
cached store path. */
if (auto res = input.settings->getCache()->lookupStorePath(revInfoKey(rev), *store))
if (auto res = settings.getCache()->lookupStorePath(revInfoKey(rev), *store))
return makeResult(res->value, res->storePath);
Path tmpDir = createTempDir();
@@ -317,18 +317,19 @@ struct MercurialInputScheme : InputScheme
});
if (!origRev)
input.settings->getCache()->upsert(refToRevKey, {{"rev", rev.gitRev()}});
settings.getCache()->upsert(refToRevKey, {{"rev", rev.gitRev()}});
input.settings->getCache()->upsert(revInfoKey(rev), *store, infoAttrs, storePath);
settings.getCache()->upsert(revInfoKey(rev), *store, infoAttrs, storePath);
return makeResult(infoAttrs, std::move(storePath));
}
std::pair<ref<SourceAccessor>, Input> getAccessor(ref<Store> store, const Input & _input) const override
std::pair<ref<SourceAccessor>, Input>
getAccessor(const Settings & settings, ref<Store> store, const Input & _input) const override
{
Input input(_input);
auto storePath = fetchToStore(store, input);
auto storePath = fetchToStore(settings, store, input);
auto accessor = store->requireStoreObjectAccessor(storePath);
accessor->setPathDisplay("«" + input.to_string() + "»");
@@ -336,7 +337,7 @@ struct MercurialInputScheme : InputScheme
return {accessor, input};
}
bool isLocked(const Input & input) const override
bool isLocked(const Settings & settings, const Input & input) const override
{
return (bool) input.getRev();
}

View File

@@ -17,7 +17,7 @@ struct PathInputScheme : InputScheme
if (url.authority && url.authority->host.size())
throw Error("path URL '%s' should not have an authority ('%s')", url, *url.authority);
Input input{settings};
Input input{};
input.attrs.insert_or_assign("type", "path");
input.attrs.insert_or_assign("path", renderUrlPathEnsureLegal(url.path));
@@ -60,7 +60,7 @@ struct PathInputScheme : InputScheme
{
getStrAttr(attrs, "path");
Input input{settings};
Input input{};
input.attrs = attrs;
return input;
}
@@ -101,7 +101,7 @@ struct PathInputScheme : InputScheme
return path;
}
bool isLocked(const Input & input) const override
bool isLocked(const Settings & settings, const Input & input) const override
{
return (bool) input.getNarHash();
}
@@ -116,7 +116,8 @@ struct PathInputScheme : InputScheme
throw Error("cannot fetch input '%s' because it uses a relative path", input.to_string());
}
std::pair<ref<SourceAccessor>, Input> getAccessor(ref<Store> store, const Input & _input) const override
std::pair<ref<SourceAccessor>, Input>
getAccessor(const Settings & settings, ref<Store> store, const Input & _input) const override
{
Input input(_input);
auto path = getStrAttr(input.attrs, "path");
@@ -145,7 +146,7 @@ struct PathInputScheme : InputScheme
auto info = store->queryPathInfo(*storePath);
accessor->fingerprint =
fmt("path:%s", store->queryPathInfo(*storePath)->narHash.to_string(HashFormat::SRI, true));
input.settings->getCache()->upsert(
settings.getCache()->upsert(
makeFetchToStoreCacheKey(
input.getName(), *accessor->fingerprint, ContentAddressMethod::Raw::NixArchive, "/"),
*store,

View File

@@ -131,9 +131,9 @@ std::shared_ptr<Registry> getFlagRegistry(const Settings & settings)
return flagRegistry;
}
void overrideRegistry(const Input & from, const Input & to, const Attrs & extraAttrs)
void overrideRegistry(const Settings & settings, const Input & from, const Input & to, const Attrs & extraAttrs)
{
getFlagRegistry(*from.settings)->add(from, to, extraAttrs);
getFlagRegistry(settings)->add(from, to, extraAttrs);
}
static std::shared_ptr<Registry> getGlobalRegistry(const Settings & settings, ref<Store> store)
@@ -172,7 +172,8 @@ Registries getRegistries(const Settings & settings, ref<Store> store)
return registries;
}
std::pair<Input, Attrs> lookupInRegistries(ref<Store> store, const Input & _input, UseRegistries useRegistries)
std::pair<Input, Attrs>
lookupInRegistries(const Settings & settings, ref<Store> store, const Input & _input, UseRegistries useRegistries)
{
Attrs extraAttrs;
int n = 0;
@@ -187,7 +188,7 @@ restart:
if (n > 100)
throw Error("cycle detected in flake registry for '%s'", input.to_string());
for (auto & registry : getRegistries(*input.settings, store)) {
for (auto & registry : getRegistries(settings, store)) {
if (useRegistries == UseRegistries::Limited
&& !(registry->type == fetchers::Registry::Flag || registry->type == fetchers::Registry::Global))
continue;

View File

@@ -224,7 +224,7 @@ ref<SourceAccessor> downloadTarball(ref<Store> store, const Settings & settings,
auto input = Input::fromAttrs(settings, std::move(attrs));
return input.getAccessor(store).first;
return input.getAccessor(settings, store).first;
}
// An input scheme corresponding to a curl-downloadable resource.
@@ -252,7 +252,7 @@ struct CurlInputScheme : InputScheme
if (!isValidURL(_url, requireTree))
return std::nullopt;
Input input{settings};
Input input{};
auto url = _url;
@@ -302,7 +302,7 @@ struct CurlInputScheme : InputScheme
std::optional<Input> inputFromAttrs(const Settings & settings, const Attrs & attrs) const override
{
Input input{settings};
Input input{};
input.attrs = attrs;
// input.locked = (bool) maybeGetStrAttr(input.attrs, "hash");
@@ -319,7 +319,7 @@ struct CurlInputScheme : InputScheme
return url;
}
bool isLocked(const Input & input) const override
bool isLocked(const Settings & settings, const Input & input) const override
{
return (bool) input.getNarHash();
}
@@ -340,7 +340,8 @@ struct FileInputScheme : CurlInputScheme
: (!requireTree && !hasTarballExtension(url)));
}
std::pair<ref<SourceAccessor>, Input> getAccessor(ref<Store> store, const Input & _input) const override
std::pair<ref<SourceAccessor>, Input>
getAccessor(const Settings & settings, ref<Store> store, const Input & _input) const override
{
auto input(_input);
@@ -348,7 +349,7 @@ struct FileInputScheme : CurlInputScheme
the Nix store directly, since there is little deduplication
benefit in using the Git cache for single big files like
tarballs. */
auto file = downloadFile(store, *input.settings, getStrAttr(input.attrs, "url"), input.getName());
auto file = downloadFile(store, settings, getStrAttr(input.attrs, "url"), input.getName());
auto narHash = store->queryPathInfo(file.storePath)->narHash;
input.attrs.insert_or_assign("narHash", narHash.to_string(HashFormat::SRI, true));
@@ -377,15 +378,15 @@ struct TarballInputScheme : CurlInputScheme
: (requireTree || hasTarballExtension(url)));
}
std::pair<ref<SourceAccessor>, Input> getAccessor(ref<Store> store, const Input & _input) const override
std::pair<ref<SourceAccessor>, Input>
getAccessor(const Settings & settings, ref<Store> store, const Input & _input) const override
{
auto input(_input);
auto result =
downloadTarball_(*input.settings, getStrAttr(input.attrs, "url"), {}, "«" + input.to_string() + "»");
auto result = downloadTarball_(settings, getStrAttr(input.attrs, "url"), {}, "«" + input.to_string() + "»");
if (result.immutableUrl) {
auto immutableInput = Input::fromURL(*input.settings, *result.immutableUrl);
auto immutableInput = Input::fromURL(settings, *result.immutableUrl);
// FIXME: would be nice to support arbitrary flakerefs
// here, e.g. git flakes.
if (immutableInput.getType() != "tarball")
@@ -398,9 +399,7 @@ struct TarballInputScheme : CurlInputScheme
input.attrs.insert_or_assign(
"narHash",
input.settings->getTarballCache()
->treeHashToNarHash(*input.settings, result.treeHash)
.to_string(HashFormat::SRI, true));
settings.getTarballCache()->treeHashToNarHash(settings, result.treeHash).to_string(HashFormat::SRI, true));
return {result.accessor, input};
}

View File

@@ -19,6 +19,7 @@
#include "nix/expr/eval-settings.hh"
#include "nix/expr/symbol-table.hh"
#include "nix/expr/value.hh"
#include "nix/expr/string-data-static.hh"
#include "nix/fetchers/attrs.hh"
#include "nix/fetchers/fetchers.hh"
#include "nix/util/configuration.hh"
@@ -38,7 +39,7 @@ PrimOp getFlake(const Settings & settings)
std::string flakeRefS(
state.forceStringNoCtx(*args[0], pos, "while evaluating the argument passed to builtins.getFlake"));
auto flakeRef = nix::parseFlakeRef(state.fetchSettings, flakeRefS, {}, true);
if (state.settings.pureEval && !flakeRef.input.isLocked())
if (state.settings.pureEval && !flakeRef.input.isLocked(state.fetchSettings))
throw Error(
"cannot call 'getFlake' on unlocked flake reference '%s', at %s (use --impure to override)",
flakeRefS,
@@ -62,7 +63,7 @@ PrimOp getFlake(const Settings & settings)
return PrimOp{
.name = "__getFlake",
.args = {"args"},
.doc = R"(
.doc = &R"(
Fetch a flake from a flake reference, and return its output attributes and some metadata. For example:
```nix
@@ -76,7 +77,7 @@ PrimOp getFlake(const Settings & settings)
```nix
(builtins.getFlake "github:edolstra/dwarffs").rev
```
)",
)"_sds,
.fun = prim_getFlake,
.experimentalFeature = Xp::Flakes,
};
@@ -104,7 +105,7 @@ static void prim_parseFlakeRef(EvalState & state, const PosIdx pos, Value ** arg
nix::PrimOp parseFlakeRef({
.name = "__parseFlakeRef",
.args = {"flake-ref"},
.doc = R"(
.doc = &R"(
Parse a flake reference, and return its exploded form.
For example:
@@ -118,7 +119,7 @@ nix::PrimOp parseFlakeRef({
```nix
{ dir = "lib"; owner = "NixOS"; ref = "23.05"; repo = "nixpkgs"; type = "github"; }
```
)",
)"_sds,
.fun = prim_parseFlakeRef,
.experimentalFeature = Xp::Flakes,
});
@@ -162,7 +163,7 @@ static void prim_flakeRefToString(EvalState & state, const PosIdx pos, Value **
nix::PrimOp flakeRefToString({
.name = "__flakeRefToString",
.args = {"attrs"},
.doc = R"(
.doc = &R"(
Convert a flake reference from attribute set format to URL format.
For example:
@@ -178,7 +179,7 @@ nix::PrimOp flakeRefToString({
```nix
"github:NixOS/nixpkgs/23.05?dir=lib"
```
)",
)"_sds,
.fun = prim_flakeRefToString,
.experimentalFeature = Xp::Flakes,
});

View File

@@ -372,7 +372,8 @@ static Flake getFlake(
const InputAttrPath & lockRootAttrPath)
{
// Fetch a lazy tree first.
auto cachedInput = state.inputCache->getAccessor(state.store, originalRef.input, useRegistries);
auto cachedInput =
state.inputCache->getAccessor(state.fetchSettings, state.store, originalRef.input, useRegistries);
auto subdir = fetchers::maybeGetStrAttr(cachedInput.extraAttrs, "dir").value_or(originalRef.subdir);
auto resolvedRef = FlakeRef(std::move(cachedInput.resolvedInput), subdir);
@@ -388,7 +389,8 @@ static Flake getFlake(
debug("refetching input '%s' due to self attribute", newLockedRef);
// FIXME: need to remove attrs that are invalidated by the changed input attrs, such as 'narHash'.
newLockedRef.input.attrs.erase("narHash");
auto cachedInput2 = state.inputCache->getAccessor(state.store, newLockedRef.input, fetchers::UseRegistries::No);
auto cachedInput2 = state.inputCache->getAccessor(
state.fetchSettings, state.store, newLockedRef.input, fetchers::UseRegistries::No);
cachedInput.accessor = cachedInput2.accessor;
lockedRef = FlakeRef(std::move(cachedInput2.lockedInput), newLockedRef.subdir);
}
@@ -704,7 +706,8 @@ lockFlake(const Settings & settings, EvalState & state, const FlakeRef & topRef,
this input. */
debug("creating new input '%s'", inputAttrPathS);
if (!lockFlags.allowUnlocked && !input.ref->input.isLocked() && !input.ref->input.isRelative())
if (!lockFlags.allowUnlocked && !input.ref->input.isLocked(state.fetchSettings)
&& !input.ref->input.isRelative())
throw Error("cannot update unlocked flake input '%s' in pure mode", inputAttrPathS);
/* Note: in case of an --override-input, we use
@@ -753,7 +756,7 @@ lockFlake(const Settings & settings, EvalState & state, const FlakeRef & topRef,
return {*resolvedPath, *input.ref};
} else {
auto cachedInput = state.inputCache->getAccessor(
state.store, input.ref->input, useRegistriesInputs);
state.fetchSettings, state.store, input.ref->input, useRegistriesInputs);
auto lockedRef = FlakeRef(std::move(cachedInput.lockedInput), input.ref->subdir);

View File

@@ -64,9 +64,10 @@ std::ostream & operator<<(std::ostream & str, const FlakeRef & flakeRef)
return str;
}
FlakeRef FlakeRef::resolve(ref<Store> store, fetchers::UseRegistries useRegistries) const
FlakeRef FlakeRef::resolve(
const fetchers::Settings & fetchSettings, ref<Store> store, fetchers::UseRegistries useRegistries) const
{
auto [input2, extraAttrs] = lookupInRegistries(store, input, useRegistries);
auto [input2, extraAttrs] = lookupInRegistries(fetchSettings, store, input, useRegistries);
return FlakeRef(std::move(input2), fetchers::maybeGetStrAttr(extraAttrs, "dir").value_or(subdir));
}
@@ -287,9 +288,10 @@ FlakeRef FlakeRef::fromAttrs(const fetchers::Settings & fetchSettings, const fet
fetchers::maybeGetStrAttr(attrs, "dir").value_or(""));
}
std::pair<ref<SourceAccessor>, FlakeRef> FlakeRef::lazyFetch(ref<Store> store) const
std::pair<ref<SourceAccessor>, FlakeRef>
FlakeRef::lazyFetch(const fetchers::Settings & fetchSettings, ref<Store> store) const
{
auto [accessor, lockedInput] = input.getAccessor(store);
auto [accessor, lockedInput] = input.getAccessor(fetchSettings, store);
return {accessor, FlakeRef(std::move(lockedInput), subdir)};
}

View File

@@ -71,11 +71,15 @@ struct FlakeRef
fetchers::Attrs toAttrs() const;
FlakeRef resolve(ref<Store> store, fetchers::UseRegistries useRegistries = fetchers::UseRegistries::All) const;
FlakeRef resolve(
const fetchers::Settings & fetchSettings,
ref<Store> store,
fetchers::UseRegistries useRegistries = fetchers::UseRegistries::All) const;
static FlakeRef fromAttrs(const fetchers::Settings & fetchSettings, const fetchers::Attrs & attrs);
std::pair<ref<SourceAccessor>, FlakeRef> lazyFetch(ref<Store> store) const;
std::pair<ref<SourceAccessor>, FlakeRef>
lazyFetch(const fetchers::Settings & fetchSettings, ref<Store> store) const;
/**
* Canonicalize a flakeref for the purpose of comparing "old" and

View File

@@ -74,7 +74,7 @@ LockedNode::LockedNode(const fetchers::Settings & fetchSettings, const nlohmann:
, parentInputAttrPath(
json.find("parent") != json.end() ? (std::optional<InputAttrPath>) json["parent"] : std::nullopt)
{
if (!lockedRef.input.isLocked() && !lockedRef.input.isRelative()) {
if (!lockedRef.input.isLocked(fetchSettings) && !lockedRef.input.isRelative()) {
if (lockedRef.input.getNarHash())
warn(
"Lock file entry '%s' is unlocked (e.g. lacks a Git revision) but is checked by NAR hash. "
@@ -282,7 +282,7 @@ std::optional<FlakeRef> LockFile::isUnlocked(const fetchers::Settings & fetchSet
latter case, we can verify the input but we may not be able to
fetch it from anywhere. */
auto isConsideredLocked = [&](const fetchers::Input & input) {
return input.isLocked() || (fetchSettings.allowDirtyLocks && input.getNarHash());
return input.isLocked(fetchSettings) || (fetchSettings.allowDirtyLocks && input.getNarHash());
};
for (auto & i : nodes) {

View File

@@ -1385,7 +1385,6 @@ bool LocalStore::verifyStore(bool checkContents, RepairFlag repair)
checkInterrupt();
auto name = link.path().filename();
printMsg(lvlTalkative, "checking contents of %s", name);
PosixSourceAccessor accessor;
std::string hash = hashPath(
PosixSourceAccessor::createAtRoot(link.path()),
FileIngestionMethod::NixArchive,

View File

@@ -1,4 +1,5 @@
#ifndef _WIN32
// TODO: investigate why this is hanging on cygwin
#if !defined(_WIN32) && !defined(__CYGWIN__)
# include "nix/util/util.hh"
# include "nix/util/monitor-fd.hh"

View File

@@ -103,9 +103,9 @@ void SourceAccessor::dumpPath(const CanonPath & path, Sink & sink, PathFilter &
time_t dumpPathAndGetMtime(const Path & path, Sink & sink, PathFilter & filter)
{
auto path2 = PosixSourceAccessor::createAtRoot(path);
auto path2 = PosixSourceAccessor::createAtRoot(path, /*trackLastModified=*/true);
path2.dumpPath(sink, filter);
return path2.accessor.dynamic_pointer_cast<PosixSourceAccessor>()->mtime;
return path2.accessor->getLastModified().value();
}
void dumpPath(const Path & path, Sink & sink, PathFilter & filter)

View File

@@ -8,6 +8,7 @@
#include <iostream>
#include <set>
#include <vector>
#include <ranges>
#include <boost/container_hash/hash.hpp>
@@ -122,33 +123,70 @@ public:
return &cs[1];
}
struct Iterator
class Iterator
{
/**
* Helper class with overloaded operator-> for "drill-down" behavior.
* This was a "temporary" string_view doesn't have to be stored anywhere.
*/
class PointerProxy
{
std::string_view segment;
public:
PointerProxy(std::string_view segment_)
: segment(segment_)
{
}
const std::string_view * operator->() const
{
return &segment;
}
};
public:
using value_type = std::string_view;
using reference_type = const std::string_view;
using pointer_type = PointerProxy;
using difference_type = std::ptrdiff_t;
using iterator_category = std::forward_iterator_tag;
std::string_view remaining;
size_t slash;
/**
* Dummy default constructor required for forward iterators. Doesn't return
* a usable iterator.
*/
Iterator()
: remaining()
, slash(0)
{
}
Iterator(std::string_view remaining)
: remaining(remaining)
, slash(remaining.find('/'))
{
}
bool operator!=(const Iterator & x) const
{
return remaining.data() != x.remaining.data();
}
bool operator==(const Iterator & x) const
{
return !(*this != x);
return remaining.data() == x.remaining.data();
}
const std::string_view operator*() const
reference_type operator*() const
{
return remaining.substr(0, slash);
}
void operator++()
pointer_type operator->() const
{
return PointerProxy(**this);
}
Iterator & operator++()
{
if (slash == remaining.npos)
remaining = remaining.substr(remaining.size());
@@ -156,9 +194,19 @@ public:
remaining = remaining.substr(slash + 1);
slash = remaining.find('/');
}
return *this;
}
Iterator operator++(int)
{
auto tmp = *this;
++*this;
return tmp;
}
};
static_assert(std::forward_iterator<Iterator>);
Iterator begin() const
{
return Iterator(rel());
@@ -265,6 +313,8 @@ public:
friend std::size_t hash_value(const CanonPath &);
};
static_assert(std::ranges::forward_range<CanonPath>);
std::ostream & operator<<(std::ostream & stream, const CanonPath & path);
inline std::size_t hash_value(const CanonPath & path)

View File

@@ -9,7 +9,7 @@ struct SourcePath;
/**
* A source accessor that uses the Unix filesystem.
*/
struct PosixSourceAccessor : virtual SourceAccessor
class PosixSourceAccessor : virtual public SourceAccessor
{
/**
* Optional root path to prefix all operations into the native file
@@ -18,8 +18,12 @@ struct PosixSourceAccessor : virtual SourceAccessor
*/
const std::filesystem::path root;
const bool trackLastModified = false;
public:
PosixSourceAccessor();
PosixSourceAccessor(std::filesystem::path && root);
PosixSourceAccessor(std::filesystem::path && root, bool trackLastModified = false);
/**
* The most recent mtime seen by lstat(). This is a hack to
@@ -43,6 +47,9 @@ struct PosixSourceAccessor : virtual SourceAccessor
* Create a `PosixSourceAccessor` and `SourcePath` corresponding to
* some native path.
*
* @param Whether the accessor should return a non-null getLastModified.
* When true the accessor must be used only by a single thread.
*
* The `PosixSourceAccessor` is rooted as far up the tree as
* possible, (e.g. on Windows it could scoped to a drive like
* `C:\`). This allows more `..` parent accessing to work.
@@ -64,7 +71,12 @@ struct PosixSourceAccessor : virtual SourceAccessor
* and
* [`std::filesystem::path::relative_path`](https://en.cppreference.com/w/cpp/filesystem/path/relative_path).
*/
static SourcePath createAtRoot(const std::filesystem::path & path);
static SourcePath createAtRoot(const std::filesystem::path & path, bool trackLastModified = false);
std::optional<std::time_t> getLastModified() override
{
return trackLastModified ? std::optional{mtime} : std::nullopt;
}
private:

View File

@@ -33,15 +33,28 @@ auto concatStrings(Parts &&... parts)
return concatStringsSep({}, views);
}
/**
* Add quotes around a string.
*/
inline std::string quoteString(std::string_view s, char quote = '\'')
{
std::string result;
result.reserve(s.size() + 2);
result += quote;
result += s;
result += quote;
return result;
}
/**
* Add quotes around a collection of strings.
*/
template<class C>
Strings quoteStrings(const C & c)
Strings quoteStrings(const C & c, char quote = '\'')
{
Strings res;
for (auto & s : c)
res.push_back("'" + s + "'");
res.push_back(quoteString(s, quote));
return res;
}

View File

@@ -7,8 +7,9 @@
namespace nix {
PosixSourceAccessor::PosixSourceAccessor(std::filesystem::path && argRoot)
PosixSourceAccessor::PosixSourceAccessor(std::filesystem::path && argRoot, bool trackLastModified)
: root(std::move(argRoot))
, trackLastModified(trackLastModified)
{
assert(root.empty() || root.is_absolute());
displayPrefix = root.string();
@@ -19,11 +20,11 @@ PosixSourceAccessor::PosixSourceAccessor()
{
}
SourcePath PosixSourceAccessor::createAtRoot(const std::filesystem::path & path)
SourcePath PosixSourceAccessor::createAtRoot(const std::filesystem::path & path, bool trackLastModified)
{
std::filesystem::path path2 = absPath(path);
return {
make_ref<PosixSourceAccessor>(path2.root_path()),
make_ref<PosixSourceAccessor>(path2.root_path(), trackLastModified),
CanonPath{path2.relative_path().string()},
};
}
@@ -114,9 +115,12 @@ std::optional<SourceAccessor::Stat> PosixSourceAccessor::maybeLstat(const CanonP
auto st = cachedLstat(path);
if (!st)
return std::nullopt;
// This makes the accessor thread-unsafe, but we only seem to use the actual value in a single threaded context in
// `src/libfetchers/path.cc`.
mtime = std::max(mtime, st->st_mtime);
/* The contract is that trackLastModified implies that the caller uses the accessor
from a single thread. Thus this is not a CAS loop. */
if (trackLastModified)
mtime = std::max(mtime, st->st_mtime);
return Stat{
.type = S_ISREG(st->st_mode) ? tRegular
: S_ISDIR(st->st_mode) ? tDirectory

View File

@@ -45,7 +45,7 @@ struct CmdFlakePrefetchInputs : FlakeCommand
if (auto lockedNode = dynamic_cast<const LockedNode *>(&node)) {
try {
Activity act(*logger, lvlInfo, actUnknown, fmt("fetching '%s'", lockedNode->lockedRef));
auto accessor = lockedNode->lockedRef.input.getAccessor(store).first;
auto accessor = lockedNode->lockedRef.input.getAccessor(fetchSettings, store).first;
fetchToStore(
fetchSettings, *store, accessor, FetchMode::Copy, lockedNode->lockedRef.input.getName());
} catch (Error & e) {

View File

@@ -245,7 +245,7 @@ struct CmdFlakeMetadata : FlakeCommand, MixJSON
printJSON(j);
} else {
logger->cout(ANSI_BOLD "Resolved URL:" ANSI_NORMAL " %s", flake.resolvedRef.to_string());
if (flake.lockedRef.input.isLocked())
if (flake.lockedRef.input.isLocked(fetchSettings))
logger->cout(ANSI_BOLD "Locked URL:" ANSI_NORMAL " %s", flake.lockedRef.to_string());
if (flake.description)
logger->cout(ANSI_BOLD "Description:" ANSI_NORMAL " %s", *flake.description);
@@ -1049,7 +1049,7 @@ struct CmdFlakeClone : FlakeCommand
if (destDir.empty())
throw Error("missing flag '--dest'");
getFlakeRef().resolve(store).input.clone(destDir);
getFlakeRef().resolve(fetchSettings, store).input.clone(fetchSettings, destDir);
}
};
@@ -1100,7 +1100,7 @@ struct CmdFlakeArchive : FlakeCommand, MixJSON, MixDryRun, MixNoCheckSigs
std::optional<StorePath> storePath;
if (!(*inputNode)->lockedRef.input.isRelative()) {
storePath = dryRun ? (*inputNode)->lockedRef.input.computeStorePath(*store)
: (*inputNode)->lockedRef.input.fetchToStore(store).first;
: (*inputNode)->lockedRef.input.fetchToStore(fetchSettings, store).first;
sources.insert(*storePath);
}
if (json) {
@@ -1324,8 +1324,10 @@ struct CmdFlakeShow : FlakeCommand, MixJSON
try {
if (visitor.isDerivation())
showDerivation();
else
throw Error("expected a derivation");
else {
auto name = visitor.getAttrPathStr(state->s.name);
logger->warn(fmt("%s is not a derivation", name));
}
} catch (IFDError & e) {
if (!json) {
logger->cout(
@@ -1496,8 +1498,8 @@ struct CmdFlakePrefetch : FlakeCommand, MixJSON
void run(ref<Store> store) override
{
auto originalRef = getFlakeRef();
auto resolvedRef = originalRef.resolve(store);
auto [accessor, lockedRef] = resolvedRef.lazyFetch(store);
auto resolvedRef = originalRef.resolve(fetchSettings, store);
auto [accessor, lockedRef] = resolvedRef.lazyFetch(getEvalState()->fetchSettings, store);
auto storePath =
fetchToStore(getEvalState()->fetchSettings, *store, accessor, FetchMode::Copy, lockedRef.input.getName());
auto hash = store->queryPathInfo(storePath)->narHash;

View File

@@ -440,7 +440,7 @@ void mainWrapped(int argc, char ** argv)
if (!primOp->doc)
continue;
b["args"] = primOp->args;
b["doc"] = trim(stripIndentation(primOp->doc));
b["doc"] = trim(stripIndentation(primOp->doc->view()));
if (primOp->experimentalFeature)
b["experimental-feature"] = primOp->experimentalFeature;
builtinsJson.emplace(state.symbols[builtin.name], std::move(b));
@@ -449,7 +449,7 @@ void mainWrapped(int argc, char ** argv)
auto b = nlohmann::json::object();
if (!info.doc)
continue;
b["doc"] = trim(stripIndentation(info.doc));
b["doc"] = trim(stripIndentation(info.doc->view()));
b["type"] = showType(info.type, false);
if (info.impureOnly)
b["impure-only"] = true;

View File

@@ -9,7 +9,7 @@
#include "nix/expr/eval-inline.hh"
#include "nix/store/profiles.hh"
#include "nix/expr/print-ambiguous.hh"
#include "nix/expr/static-string-data.hh"
#include "nix/expr/string-data-static.hh"
#include <limits>
#include <sstream>

View File

@@ -711,7 +711,7 @@ struct CmdProfileUpgrade : virtual SourceExprCommand, MixDefaultProfile, MixProf
element.identifier());
continue;
}
if (element.source->originalRef.input.isLocked()) {
if (element.source->originalRef.input.isLocked(getEvalState()->fetchSettings)) {
warn(
"Found package '%s', but it was added from a locked flake reference so it can't be upgraded!",
element.identifier());
@@ -740,7 +740,8 @@ struct CmdProfileUpgrade : virtual SourceExprCommand, MixDefaultProfile, MixProf
assert(infop);
auto & info = *infop;
if (info.flake.lockedRef.input.isLocked() && element.source->lockedRef == info.flake.lockedRef)
if (info.flake.lockedRef.input.isLocked(getEvalState()->fetchSettings)
&& element.source->lockedRef == info.flake.lockedRef)
continue;
printInfo(

View File

@@ -190,8 +190,9 @@ struct CmdRegistryPin : RegistryCommand, EvalCommand
auto ref = parseFlakeRef(fetchSettings, url);
auto lockedRef = parseFlakeRef(fetchSettings, locked);
registry->remove(ref.input);
auto resolved = lockedRef.resolve(store).input.getAccessor(store).second;
if (!resolved.isLocked())
auto resolvedInput = lockedRef.resolve(fetchSettings, store).input;
auto resolved = resolvedInput.getAccessor(fetchSettings, store).second;
if (!resolved.isLocked(fetchSettings))
warn("flake '%s' is not locked", resolved.to_string());
fetchers::Attrs extraAttrs;
if (ref.subdir != "")

View File

@@ -107,3 +107,26 @@ in
assert show_output.packages.${builtins.currentSystem}.default == { };
true
'
# Test that nix keeps going even when packages.$SYSTEM contains not derivations
cat >flake.nix <<EOF
{
outputs = inputs: {
packages.$system = {
drv1 = import ./simple.nix;
not-a-derivation = 42;
drv2 = import ./simple.nix;
};
};
}
EOF
nix flake show --json --all-systems > show-output.json
# shellcheck disable=SC2016
nix eval --impure --expr '
let show_output = builtins.fromJSON (builtins.readFile ./show-output.json);
in
assert show_output.packages.${builtins.currentSystem}.not-a-derivation == {};
true
'