Compare commits

...

12 Commits

Author SHA1 Message Date
Eelco Dolstra
338418790a Bump version 2022-06-29 20:49:03 +02:00
Eelco Dolstra
f5070e2096 Bump version 2022-06-16 15:09:41 +02:00
Eelco Dolstra
424bb9b509 Merge pull request #6668 from lovesegfault/fix-getgrouplist-2.9
backport #1527
2022-06-14 16:36:29 +02:00
Bernardo Meurer
ad5b46ba27 fix(libstore/lock): support users that belong to more than 10 groups
The manpage for `getgrouplist` says:

> If the number of groups of which user is a member is less than or
> equal to *ngroups, then the value *ngroups is returned.
>
> If the user is a member of more than *ngroups groups, then
> getgrouplist() returns -1.  In this case, the value returned in
> *ngroups can be used to resize the buffer passed to a further
> call getgrouplist().

In our original code, however, we allocated a list of size `10` and, if
`getgrouplist` returned `-1` threw an exception. In practice, this
caused the code to fail for any user belonging to more than 10 groups.

While unusual for single-user systems, large companies commonly have a
huge number of POSIX groups users belong to, causing this issue to crop
up and make multi-user Nix unusable in such settings.

The fix is relatively simple, when `getgrouplist` fails, it stores the
real number of GIDs in `ngroups`, so we must resize our list and retry.
Only then, if it errors once more, we can raise an exception.

This should be backported to, at least, 2.9.x.

(cherry picked from commit 931930feb1)
2022-06-14 10:29:52 -04:00
Théophane Hufschmitt
ad41cd0d8a Merge pull request #6658 from NixOS/backport-6653-to-2.9-maintenance
[Backport 2.9-maintenance] fix sourcehut brach/tag resolving regression
2022-06-13 08:58:10 +02:00
Gabriel Fontes
f651fc250b fix sourcehut brach/tag resolving regression
nixos/nix#6290 introduced a regex pattern to account for tags when
resolving sourcehut refs. nixos/nix#4638 reafactored the code,
accidentally treating the pattern as a regular string, causing all
non-HEAD ref resolving to break.

This fixes the regression and adds more test cases to avoid future
breakage.

(cherry picked from commit 9f6b4639c2)
2022-06-13 06:57:53 +00:00
Théophane Hufschmitt
1a994cc35b Merge pull request #6644 from NixOS/backport-6643-to-2.9-maintenance
[Backport 2.9-maintenance] libfetchers/git: add missing `--git-dir` flags
2022-06-10 16:04:05 +02:00
Naïm Favier
51c2dbb575 libfetchers/git: add missing --git-dir flags
(cherry picked from commit da8f8668ca)
2022-06-10 14:40:48 +02:00
Eelco Dolstra
624e38aa43 Bump version 2022-05-31 18:30:17 +02:00
Eelco Dolstra
53555dfb39 Merge pull request #6597 from NixOS/backport-6582-to-2.9-maintenance
[Backport 2.9-maintenance] Fix a segfault in the git fetcher
2022-05-31 17:19:28 +02:00
Théophane Hufschmitt
bc47593455 Fix a segfault in the git fetcher
The git fetcher code used to dereference the (potentially empty) `ref`
input attribute. This was magically working, probably because the
compiler somehow outsmarted us, but is now blowing up with newer nixpkgs
versions.

Fix that by not trying to access this field while we don't know for sure
that it has been defined.

Fix #6554

(cherry picked from commit 027fd45230)
2022-05-31 15:18:58 +00:00
Eelco Dolstra
daa14b8910 Mark official release 2022-05-30 20:49:35 +02:00
7 changed files with 47 additions and 20 deletions

View File

@@ -1 +1 @@
2.9.0
2.9.3

View File

@@ -71,7 +71,6 @@
- [Hacking](contributing/hacking.md)
- [CLI guideline](contributing/cli-guideline.md)
- [Release Notes](release-notes/release-notes.md)
- [Release X.Y (202?-??-??)](release-notes/rl-next.md)
- [Release 2.9 (2022-05-30)](release-notes/rl-2.9.md)
- [Release 2.8 (2022-04-19)](release-notes/rl-2.8.md)
- [Release 2.7 (2022-03-07)](release-notes/rl-2.7.md)

View File

@@ -15,7 +15,7 @@
then ""
else "pre${builtins.substring 0 8 (self.lastModifiedDate or self.lastModified or "19700101")}_${self.shortRev or "dirty"}";
officialRelease = false;
officialRelease = true;
linux64BitSystems = [ "x86_64-linux" "aarch64-linux" ];
linuxSystems = linux64BitSystems ++ [ "i686-linux" ];

View File

@@ -85,8 +85,9 @@ std::optional<std::string> readHead(const Path & path)
bool storeCachedHead(const std::string& actualUrl, const std::string& headRef)
{
Path cacheDir = getCachePath(actualUrl);
auto gitDir = ".";
try {
runProgram("git", true, { "-C", cacheDir, "symbolic-ref", "--", "HEAD", headRef });
runProgram("git", true, { "-C", cacheDir, "--git-dir", gitDir, "symbolic-ref", "--", "HEAD", headRef });
} catch (ExecError &e) {
if (!WIFEXITED(e.status)) throw;
return false;
@@ -182,7 +183,7 @@ WorkdirInfo getWorkdirInfo(const Input & input, const Path & workdir)
if (hasHead) {
// Using git diff is preferrable over lower-level operations here,
// because its conceptually simpler and we only need the exit code anyways.
auto gitDiffOpts = Strings({ "-C", workdir, "diff", "HEAD", "--quiet"});
auto gitDiffOpts = Strings({ "-C", workdir, "--git-dir", gitDir, "diff", "HEAD", "--quiet"});
if (!submodules) {
// Changes in submodules should only make the tree dirty
// when those submodules will be copied as well.
@@ -203,6 +204,7 @@ WorkdirInfo getWorkdirInfo(const Input & input, const Path & workdir)
std::pair<StorePath, Input> fetchFromWorkdir(ref<Store> store, Input & input, const Path & workdir, const WorkdirInfo & workdirInfo)
{
const bool submodules = maybeGetBoolAttr(input.attrs, "submodules").value_or(false);
auto gitDir = ".git";
if (!fetchSettings.allowDirty)
throw Error("Git tree '%s' is dirty", workdir);
@@ -210,7 +212,7 @@ std::pair<StorePath, Input> fetchFromWorkdir(ref<Store> store, Input & input, co
if (fetchSettings.warnDirty)
warn("Git tree '%s' is dirty", workdir);
auto gitOpts = Strings({ "-C", workdir, "ls-files", "-z" });
auto gitOpts = Strings({ "-C", workdir, "--git-dir", gitDir, "ls-files", "-z" });
if (submodules)
gitOpts.emplace_back("--recurse-submodules");
@@ -240,7 +242,7 @@ std::pair<StorePath, Input> fetchFromWorkdir(ref<Store> store, Input & input, co
// modified dirty file?
input.attrs.insert_or_assign(
"lastModified",
workdirInfo.hasHead ? std::stoull(runProgram("git", true, { "-C", actualPath, "log", "-1", "--format=%ct", "--no-show-signature", "HEAD" })) : 0);
workdirInfo.hasHead ? std::stoull(runProgram("git", true, { "-C", actualPath, "--git-dir", gitDir, "log", "-1", "--format=%ct", "--no-show-signature", "HEAD" })) : 0);
return {std::move(storePath), input};
}
@@ -449,11 +451,10 @@ struct GitInputScheme : InputScheme
}
}
const Attrs unlockedAttrs({
Attrs unlockedAttrs({
{"type", cacheType},
{"name", name},
{"url", actualUrl},
{"ref", *input.getRef()},
});
Path repoDir;
@@ -466,6 +467,7 @@ struct GitInputScheme : InputScheme
head = "master";
}
input.attrs.insert_or_assign("ref", *head);
unlockedAttrs.insert_or_assign("ref", *head);
}
if (!input.getRev())
@@ -482,6 +484,7 @@ struct GitInputScheme : InputScheme
head = "master";
}
input.attrs.insert_or_assign("ref", *head);
unlockedAttrs.insert_or_assign("ref", *head);
}
if (auto res = getCache()->lookup(store, unlockedAttrs)) {

View File

@@ -381,7 +381,7 @@ struct SourceHutInputScheme : GitArchiveInputScheme
Headers headers = makeHeadersWithAuthTokens(host);
std::string ref_uri;
std::string refUri;
if (ref == "HEAD") {
auto file = store->toRealPath(
downloadFile(store, fmt("%s/HEAD", base_url), "source", false, headers).storePath);
@@ -393,10 +393,11 @@ struct SourceHutInputScheme : GitArchiveInputScheme
if (!remoteLine) {
throw BadURL("in '%d', couldn't resolve HEAD ref '%d'", input.to_string(), ref);
}
ref_uri = remoteLine->target;
refUri = remoteLine->target;
} else {
ref_uri = fmt("refs/(heads|tags)/%s", ref);
refUri = fmt("refs/(heads|tags)/%s", ref);
}
std::regex refRegex(refUri);
auto file = store->toRealPath(
downloadFile(store, fmt("%s/info/refs", base_url), "source", false, headers).storePath);
@@ -406,7 +407,7 @@ struct SourceHutInputScheme : GitArchiveInputScheme
std::optional<std::string> id;
while(!id && getline(is, line)) {
auto parsedLine = git::parseLsRemoteLine(line);
if (parsedLine && parsedLine->reference == ref_uri)
if (parsedLine && parsedLine->reference && std::regex_match(*parsedLine->reference, refRegex))
id = parsedLine->target;
}

View File

@@ -67,13 +67,26 @@ bool UserLock::findFreeUser() {
#if __linux__
/* Get the list of supplementary groups of this build user. This
is usually either empty or contains a group such as "kvm". */
supplementaryGIDs.resize(10);
int ngroups = supplementaryGIDs.size();
int err = getgrouplist(pw->pw_name, pw->pw_gid,
supplementaryGIDs.data(), &ngroups);
if (err == -1)
throw Error("failed to get list of supplementary groups for '%1%'", pw->pw_name);
int ngroups = 32; // arbitrary initial guess
supplementaryGIDs.resize(ngroups);
int err = getgrouplist(pw->pw_name, pw->pw_gid, supplementaryGIDs.data(),
&ngroups);
// Our initial size of 32 wasn't sufficient, the correct size has
// been stored in ngroups, so we try again.
if (err == -1) {
supplementaryGIDs.resize(ngroups);
err = getgrouplist(pw->pw_name, pw->pw_gid, supplementaryGIDs.data(),
&ngroups);
}
// If it failed once more, then something must be broken.
if (err == -1)
throw Error("failed to get list of supplementary groups for '%1%'",
pw->pw_name);
// Finally, trim back the GID list to its real size
supplementaryGIDs.resize(ngroups);
#endif

View File

@@ -59,7 +59,7 @@ let
echo 'ref: refs/heads/master' > $out/HEAD
mkdir -p $out/info
echo -e '${nixpkgs.rev}\trefs/heads/master' > $out/info/refs
echo -e '${nixpkgs.rev}\trefs/heads/master\n${nixpkgs.rev}\trefs/tags/foo-bar' > $out/info/refs
'';
in
@@ -132,6 +132,17 @@ makeTest (
client.succeed("curl -v https://git.sr.ht/ >&2")
client.succeed("nix registry list | grep nixpkgs")
# Test that it resolves HEAD
rev = client.succeed("nix flake info sourcehut:~NixOS/nixpkgs --json | jq -r .revision")
assert rev.strip() == "${nixpkgs.rev}", "revision mismatch"
# Test that it resolves branches
rev = client.succeed("nix flake info sourcehut:~NixOS/nixpkgs/master --json | jq -r .revision")
assert rev.strip() == "${nixpkgs.rev}", "revision mismatch"
# Test that it resolves tags
rev = client.succeed("nix flake info sourcehut:~NixOS/nixpkgs/foo-bar --json | jq -r .revision")
assert rev.strip() == "${nixpkgs.rev}", "revision mismatch"
# Registry and pinning test
rev = client.succeed("nix flake info nixpkgs --json | jq -r .revision")
assert rev.strip() == "${nixpkgs.rev}", "revision mismatch"