Compare commits

...

23 Commits

Author SHA1 Message Date
Eelco Dolstra
0cd1fce3c3 Merge pull request #13230 from NixOS/mergify/bp/2.29-maintenance/pr-13228
libutil-tests/json-utils: fix -Werror=sign-compare error (backport #13228)
2025-05-19 16:46:37 +02:00
Jörg Thalheim
90eb2f759c libutil-tests/json-utils: fix -Werror=sign-compare error
I am on a newer different nixpkgs branch, so I am getting this error

(cherry picked from commit 1290b7e53d)
2025-05-19 14:16:43 +00:00
Jörg Thalheim
d1e4be6fb4 Merge pull request #13227 from NixOS/mergify/bp/2.29-maintenance/pr-13142
libstore: Use `boost::regex` for GC root discovery (backport #13142)
2025-05-18 22:26:54 +02:00
Sergei Zimmerman
29d98da636 libstore: Depend on boost_regex explicitly
(cherry picked from commit 18a5589f9a)
2025-05-18 19:46:17 +00:00
Sergei Zimmerman
91dc6e7fa0 packaging/dependencies: Use boost without enableIcu
This reduces the closure size on master by 40MiB.

```
$ nix build github:nixos/nix/1e822bd4149a8bce1da81ee2ad9404986b07914c#nix-store --out-link closure-on-master
$ nix build .#nix-store -L --out-link closure-without-icu
$ nix path-info --closure-size -h ./closure-on-master
/nix/store/8gwr38m5h6p7245ji9jv28a2a11w1isx-nix-store-2.29.0pre  124.4 MiB
$ nix path-info --closure-size -h ./closure-without-icu
/nix/store/k0gwfykjqpnmaqbwh23nk55lhanc9g24-nix-store-2.29.0pre   86.6 MiB
```

(cherry picked from commit f3090ef703)
2025-05-18 19:46:17 +00:00
Sergei Zimmerman
b33fd1e4fb libstore: Use boost::regex for GC root discovery
As it turns out using `std::regex` is actually the bottleneck
for root discovery. Just substituting `std::` -> `boost::`
makes root discovery twice as fast (3x if counting only userspace time).

Some rather ad-hoc measurements to motivate the switch:

(On master)

```
nix build github:nixos/nix/1e822bd4149a8bce1da81ee2ad9404986b07914c#nix-cli --out-link result-1e822bd4149a8bce1da81ee2ad9404986b07914c
taskset -c 2,3 hyperfine "result-1e822bd4149a8bce1da81ee2ad9404986b07914c/bin/nix store gc --dry-run --max 0"
Benchmark 1: result-1e822bd4149a8bce1da81ee2ad9404986b07914c/bin/nix store gc --dry-run --max 0
  Time (mean ± σ):     481.6 ms ±   3.9 ms    [User: 336.2 ms, System: 142.0 ms]
  Range (min … max):   474.6 ms … 487.7 ms    10 runs
```

(After this patch)

```
taskset -c 2,3 hyperfine "result/bin/nix store gc --dry-run --max 0"
Benchmark 1: result/bin/nix store gc --dry-run --max 0
  Time (mean ± σ):     254.7 ms ±   9.7 ms    [User: 111.1 ms, System: 141.3 ms]
  Range (min … max):   246.5 ms … 281.3 ms    10 runs
```

`boost::regex` is a drop-in replacement for `std::regex`, but much faster.
Doing a simple before/after comparison doesn't surface any change in behavior:

```
result/bin/nix store gc --dry-run -vvvvv --max 0 |& grep "got additional" | wc -l
result-1e822bd4149a8bce1da81ee2ad9404986b07914c/bin/nix store gc --dry-run -vvvvv --max 0 |& grep "got additional" | wc -l
```

(cherry picked from commit 3a1301cd6d)
2025-05-18 19:46:16 +00:00
mergify[bot]
1c618a9d87 Merge pull request #13222 from NixOS/mergify/bp/2.29-maintenance/pr-13221
doc: Render verbatim `@docroot@` on contributing page (backport #13221)
2025-05-17 18:00:31 +00:00
Robert Hensing
ab2abebfc3 doc: Render verbatim @docroot@ on contributing page
In rendered form:

```diff
-Add references to the manual using ..
+Add references to the manual using [links like this](@docroot@/example.md)
```

(cherry picked from commit 147a34c573)
2025-05-17 17:19:51 +00:00
John Ericson
0f132fc129 Merge pull request #13218 from NixOS/mergify/bp/2.29-maintenance/pr-13212
docs: remove repeated "allowedReferences" and other lexical illusion (backport #13212)
2025-05-16 12:36:46 -04:00
Peder Bergebakken Sundt
6fc6db3496 docs: remove lexical illusions detected with write-good
I made this this non-markdown aware tool somewhat behave with some cursed fd+pandoc invocations

(cherry picked from commit ea5302c4a2)
2025-05-16 15:56:26 +00:00
Peder Bergebakken Sundt
ca0bde3578 docs: remove repeated "allowedReferences"
This is what write-good lints as a "lexical illusion"

(cherry picked from commit cb16cd707c)
2025-05-16 15:56:26 +00:00
mergify[bot]
f51b537239 Merge pull request #13210 from NixOS/mergify/bp/2.29-maintenance/pr-13207
dev-shell: Drop bear dependency (backport #13207)
2025-05-15 22:32:16 +00:00
Sergei Zimmerman
2b7e3e9b81 dev-shell: Drop bear dependency
Since the autotools-based build system has been removed
and meson already generates compile database there's no
need to have it in the devshell.

(cherry picked from commit 67535263a5)
2025-05-15 21:49:21 +00:00
Jörg Thalheim
fee8454dd0 Merge pull request #13205 from NixOS/mergify/bp/2.29-maintenance/pr-13202
Fix nix-copy-closure VM test (backport #13202)
2025-05-15 18:34:32 +02:00
Eelco Dolstra
dc238ba102 Fix nix-copy-closure VM test
https://hydra.nixos.org/build/297112538
(cherry picked from commit d626348f42)
2025-05-15 15:51:43 +00:00
mergify[bot]
fa7a5ab07a Merge pull request #13201 from NixOS/mergify/bp/2.29-maintenance/pr-13197
Update Nixpkgs to fix static builds (backport #13197)
2025-05-15 11:48:30 +00:00
Eelco Dolstra
27932ae6da Merge pull request #13199 from NixOS/mergify/bp/2.29-maintenance/pr-13196
rename StoreDirConfigItself to StoreDirConfigBase (backport #13196)
2025-05-15 12:44:09 +02:00
Eelco Dolstra
631d23788e Merge pull request #13198 from NixOS/mergify/bp/2.29-maintenance/pr-13195
Remove otherNixes.nix_2_18 (backport #13195)
2025-05-15 12:24:24 +02:00
Eelco Dolstra
4f03bfebd9 flake.lock: Update
Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/f02fddb8acef29a8b32f10a335d44828d7825b78?narHash=sha256-IgBWhX7A2oJmZFIrpRuMnw5RAufVnfvOgHWgIdds%2Bhc%3D' (2025-05-01)
  → 'github:NixOS/nixpkgs/adaa24fbf46737f3f1b5497bf64bae750f82942e?narHash=sha256-qhFMmDkeJX9KJwr5H32f1r7Prs7XbQWtO0h3V0a0rFY%3D' (2025-05-13)

(cherry picked from commit 3ba49d7ec2)
2025-05-15 10:21:25 +00:00
Jörg Thalheim
7f488dc7d3 rename StoreDirConfigItself to StoreDirConfigBase
context: https://github.com/NixOS/nix/pull/13154#discussion_r2081904653
(cherry picked from commit 2dd2142754)
2025-05-15 10:04:22 +00:00
Eelco Dolstra
1b2e88effd Remove otherNixes.nix_2_18
Nixpkgs no longer has Nix 2.18, so this fails to evaluate.

(cherry picked from commit bc85e20fb9)
2025-05-15 09:44:21 +00:00
John Ericson
4dcf21a2f6 Merge branch 'master' into 2.29-maintenance 2025-05-14 19:59:35 -04:00
John Ericson
ff24751bdd Mark official release 2025-05-14 19:29:50 -04:00
16 changed files with 37 additions and 43 deletions

View File

@@ -20,8 +20,9 @@ prs: 1238
Here's one or more paragraphs that describe the change.
- It's markdown
- Add references to the manual using @docroot@
- Add references to the manual using [links like this](@_at_docroot@/example.md)
```
<!-- for the raw markdown readers: that means using @docroot@ -->
Significant changes should add the following header, which moves them to the top.

View File

@@ -73,7 +73,7 @@ Derivations can declare some infrequently used optional attributes.
> **Warning**
>
> If set to `true`, other advanced attributes such as [`allowedReferences`](#adv-attr-allowedReferences), [`allowedReferences`](#adv-attr-allowedReferences), [`allowedRequisites`](#adv-attr-allowedRequisites),
> If set to `true`, other advanced attributes such as [`allowedReferences`](#adv-attr-allowedReferences), [`allowedRequisites`](#adv-attr-allowedRequisites),
[`disallowedReferences`](#adv-attr-disallowedReferences) and [`disallowedRequisites`](#adv-attr-disallowedRequisites), maxSize, and maxClosureSize.
will have no effect.

View File

@@ -23,7 +23,7 @@ The output spec for an output with a fixed content addresses additionally contai
> **Design note**
>
> In principle, the output spec could also specify the references the store object should have, since the references and file system objects are equally parts of a content-addressed store object proper that contribute to its content-addressed.
> However, at this time, the references are not not done because all fixed content-addressed outputs are required to have no references (including no self-reference).
> However, at this time, the references are not done because all fixed content-addressed outputs are required to have no references (including no self-reference).
>
> Also in principle, rather than specifying the references and file system object data with separate hashes, a single hash that constraints both could be used.
> This could be done with the final store path's digest, or better yet, the hash that will become the store path's digest before it is truncated.
@@ -116,7 +116,7 @@ Because the derivation output is not fixed (just like with [input addressing]),
> (The "environment", in this case, consists of attributes such as the Operating System Nix runs atop, along with the operating-system-specific privileges that Nix has been granted.
> Because of how conventional operating systems like macos, Linux, etc. work, granting builders *fewer* privileges may ironically require that Nix be run with *more* privileges.)
That said, derivations producing floating content-addressed outputs may declare their builders as impure (like the builders of derivations producing producing fixed outputs).
That said, derivations producing floating content-addressed outputs may declare their builders as impure (like the builders of derivations producing fixed outputs).
This is provisionally supported as part of the [`impure-derivations`][xp-feature-impure-derivations] experimental feature.
### Compatibility negotiation
@@ -144,7 +144,7 @@ A *deterministic* content-addressing derivation should produce outputs with the
The choice of provisional store path can be thought of as an impurity, since it is an arbitrary choice.
If provisional outputs paths are deterministically chosen, we are in the first branch of part (1).
The builder the data it produces based on it in arbitrary ways, but this gets us closer to to [input addressing].
The builder the data it produces based on it in arbitrary ways, but this gets us closer to [input addressing].
Deterministically choosing the provisional path may be considered "complete sandboxing" by removing an impurity, but this is unsatisfactory
<!--

View File

@@ -83,7 +83,7 @@ The rules for this are fairly concise:
- A content-addressing derivation may be pure or impure
- If it is impure, it may be be fixed (typical), or it may be floating if the additional [`impure-derivations`][xp-feature-impure-derivations] experimental feature is enabled.
- If it is impure, it may be fixed (typical), or it may be floating if the additional [`impure-derivations`][xp-feature-impure-derivations] experimental feature is enabled.
- If it is pure, it must be floating.

View File

@@ -57,6 +57,9 @@ def recursive_replace(data: dict[str, t.Any], book_root: Path, search_path: Path
).replace(
'@docroot@',
("../" * len(path_to_chapter.parent.parts) or "./")[:-1]
).replace(
'@_at_',
'@'
),
sub_items = [
recursive_replace(sub_item, book_root, search_path)

6
flake.lock generated
View File

@@ -63,11 +63,11 @@
},
"nixpkgs": {
"locked": {
"lastModified": 1746141548,
"narHash": "sha256-IgBWhX7A2oJmZFIrpRuMnw5RAufVnfvOgHWgIdds+hc=",
"lastModified": 1747179050,
"narHash": "sha256-qhFMmDkeJX9KJwr5H32f1r7Prs7XbQWtO0h3V0a0rFY=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "f02fddb8acef29a8b32f10a335d44828d7825b78",
"rev": "adaa24fbf46737f3f1b5497bf64bae750f82942e",
"type": "github"
},
"original": {

View File

@@ -32,7 +32,7 @@
let
inherit (nixpkgs) lib;
officialRelease = false;
officialRelease = true;
linux32BitSystems = [ "i686-linux" ];
linux64BitSystems = [

View File

@@ -63,6 +63,7 @@ scope: {
"--with-coroutine"
"--with-iostreams"
];
enableIcu = false;
}).overrideAttrs
(old: {
# Need to remove `--with-*` to use `--with-libraries=...`

View File

@@ -119,9 +119,6 @@ pkgs.nixComponents2.nix-util.overrideAttrs (
(pkgs.writeScriptBin "pre-commit-hooks-install" modular.pre-commit.settings.installationScript)
pkgs.buildPackages.nixfmt-rfc-style
]
# TODO: Remove the darwin check once
# https://github.com/NixOS/nixpkgs/pull/291814 is available
++ lib.optional (stdenv.cc.isClang && !stdenv.buildPlatform.isDarwin) pkgs.buildPackages.bear
++ lib.optional (stdenv.cc.isClang && stdenv.hostPlatform == stdenv.buildPlatform) (
lib.hiPrio pkgs.buildPackages.clang-tools
)

View File

@@ -13,10 +13,11 @@
# include "nix/util/processes.hh"
#endif
#include <boost/regex.hpp>
#include <functional>
#include <queue>
#include <algorithm>
#include <regex>
#include <random>
#include <climits>
@@ -331,8 +332,8 @@ static void readProcLink(const std::filesystem::path & file, UncheckedRoots & ro
static std::string quoteRegexChars(const std::string & raw)
{
static auto specialRegex = std::regex(R"([.^$\\*+?()\[\]{}|])");
return std::regex_replace(raw, specialRegex, R"(\$&)");
static auto specialRegex = boost::regex(R"([.^$\\*+?()\[\]{}|])");
return boost::regex_replace(raw, specialRegex, R"(\$&)");
}
#ifdef __linux__
@@ -354,12 +355,12 @@ void LocalStore::findRuntimeRoots(Roots & roots, bool censor)
auto procDir = AutoCloseDir{opendir("/proc")};
if (procDir) {
struct dirent * ent;
auto digitsRegex = std::regex(R"(^\d+$)");
auto mapRegex = std::regex(R"(^\s*\S+\s+\S+\s+\S+\s+\S+\s+\S+\s+(/\S+)\s*$)");
auto storePathRegex = std::regex(quoteRegexChars(storeDir) + R"(/[0-9a-z]+[0-9a-zA-Z\+\-\._\?=]*)");
static const auto digitsRegex = boost::regex(R"(^\d+$)");
static const auto mapRegex = boost::regex(R"(^\s*\S+\s+\S+\s+\S+\s+\S+\s+\S+\s+(/\S+)\s*$)");
auto storePathRegex = boost::regex(quoteRegexChars(storeDir) + R"(/[0-9a-z]+[0-9a-zA-Z\+\-\._\?=]*)");
while (errno = 0, ent = readdir(procDir.get())) {
checkInterrupt();
if (std::regex_match(ent->d_name, digitsRegex)) {
if (boost::regex_match(ent->d_name, digitsRegex)) {
try {
readProcLink(fmt("/proc/%s/exe" ,ent->d_name), unchecked);
readProcLink(fmt("/proc/%s/cwd", ent->d_name), unchecked);
@@ -386,15 +387,15 @@ void LocalStore::findRuntimeRoots(Roots & roots, bool censor)
std::filesystem::path mapFile = fmt("/proc/%s/maps", ent->d_name);
auto mapLines = tokenizeString<std::vector<std::string>>(readFile(mapFile.string()), "\n");
for (const auto & line : mapLines) {
auto match = std::smatch{};
if (std::regex_match(line, match, mapRegex))
auto match = boost::smatch{};
if (boost::regex_match(line, match, mapRegex))
unchecked[match[1]].emplace(mapFile.string());
}
auto envFile = fmt("/proc/%s/environ", ent->d_name);
auto envString = readFile(envFile);
auto env_end = std::sregex_iterator{};
for (auto i = std::sregex_iterator{envString.begin(), envString.end(), storePathRegex}; i != env_end; ++i)
auto env_end = boost::sregex_iterator{};
for (auto i = boost::sregex_iterator{envString.begin(), envString.end(), storePathRegex}; i != env_end; ++i)
unchecked[i->str()].emplace(envFile);
} catch (SystemError & e) {
if (errno == ENOENT || errno == EACCES || errno == ESRCH)
@@ -413,12 +414,12 @@ void LocalStore::findRuntimeRoots(Roots & roots, bool censor)
// Because of this we disable lsof when running the tests.
if (getEnv("_NIX_TEST_NO_LSOF") != "1") {
try {
std::regex lsofRegex(R"(^n(/.*)$)");
boost::regex lsofRegex(R"(^n(/.*)$)");
auto lsofLines =
tokenizeString<std::vector<std::string>>(runProgram(LSOF, true, { "-n", "-w", "-F", "n" }), "\n");
for (const auto & line : lsofLines) {
std::smatch match;
if (std::regex_match(line, match, lsofRegex))
boost::smatch match;
if (boost::regex_match(line, match, lsofRegex))
unchecked[match[1].str()].emplace("{lsof}");
}
} catch (ExecError & e) {

View File

@@ -104,7 +104,7 @@ struct MixStoreDirMethods
* Need to make this a separate class so I can get the right
* initialization order in the constructor for `StoreDirConfig`.
*/
struct StoreDirConfigItself : Config
struct StoreDirConfigBase : Config
{
using Config::Config;
@@ -118,12 +118,12 @@ struct StoreDirConfigItself : Config
};
/**
* The order of `StoreDirConfigItself` and then `MixStoreDirMethods` is
* very important. This ensures that `StoreDirConfigItself::storeDir_`
* The order of `StoreDirConfigBase` and then `MixStoreDirMethods` is
* very important. This ensures that `StoreDirConfigBase::storeDir_`
* is initialized before we have our one chance (because references are
* immutable) to initialize `MixStoreDirMethods::storeDir`.
*/
struct StoreDirConfig : StoreDirConfigItself, MixStoreDirMethods
struct StoreDirConfig : StoreDirConfigBase, MixStoreDirMethods
{
using Params = std::map<std::string, std::string>;

View File

@@ -94,7 +94,7 @@ subdir('nix-meson-build-support/libatomic')
boost = dependency(
'boost',
modules : ['container'],
modules : ['container', 'regex'],
include_type: 'system',
)
# boost is a public dependency, but not a pkg-config dependency unfortunately, so we

View File

@@ -5,7 +5,7 @@
namespace nix {
StoreDirConfig::StoreDirConfig(const Params & params)
: StoreDirConfigItself(params)
: StoreDirConfigBase(params)
, MixStoreDirMethods{storeDir_}
{
}

View File

@@ -131,7 +131,7 @@ TEST(getString, wrongAssertions) {
TEST(getIntegralNumber, rightAssertions) {
auto simple = R"({ "int": 0, "signed": -1 })"_json;
ASSERT_EQ(getUnsigned(valueAt(getObject(simple), "int")), 0);
ASSERT_EQ(getUnsigned(valueAt(getObject(simple), "int")), 0u);
ASSERT_EQ(getInteger<int8_t>(valueAt(getObject(simple), "int")), 0);
ASSERT_EQ(getInteger<int8_t>(valueAt(getObject(simple), "signed")), -1);
}

View File

@@ -94,13 +94,6 @@ let
);
};
otherNixes.nix_2_18.setNixPackage =
{ lib, pkgs, ... }:
{
imports = [ checkOverrideNixVersion ];
nix.package = lib.mkForce pkgs.nixVersions.nix_2_18;
};
in
{

View File

@@ -61,12 +61,10 @@ in
"${pkgs.openssh}/bin/ssh-keygen", "-t", "ed25519", "-f", "key", "-N", ""
], capture_output=True, check=True)
client.succeed("mkdir -m 700 /root/.ssh")
client.copy_from_host("key", "/root/.ssh/id_ed25519")
client.succeed("chmod 600 /root/.ssh/id_ed25519")
# Install the SSH key on the server.
server.succeed("mkdir -m 700 /root/.ssh")
server.copy_from_host("key.pub", "/root/.ssh/authorized_keys")
server.wait_for_unit("sshd")
server.wait_for_unit("multi-user.target")