Commit Graph

2725 Commits

Author SHA1 Message Date
John Ericson
bbcf2041e1 File system error improvements
- Make `descriptorToPath` cross-platform (renamed from
  `windows::handleToPath`). Uses `/proc/self/fd` on Linux and
  `F_GETPATH` on macOS. Add `HAVE_F_GETPATH` meson check.

  This is based on 7226a116a0, which was
  removed in 479c356510, but is now
  introduced more judiciously.

- Unix error messages in `readFull`, `writeFull`, `readLine` now include
  file paths via `descriptorToPath`.

- Convert `std::filesystem::filesystem_error` to `SystemError`

  Wrappers like `readLink`, `createDirs`, `DirectoryIterator`, etc. now
  catch `std::filesystem::filesystem_error` and rethrow as `SystemError`
  with the error code preserved. This ensures consistent exception types
  throughout the codebase.

  Call sites that previously caught `filesystem_error` and rethrew with
  `throw;` now throw `SystemError(e.code(), ...)` instead.

  Some call sites can stop catching `filesystem_error` at all,
  because they only call the wrapped functions.

- Rework `SystemError` constructors to auto-append error message

  The public `SystemError(std::error_code, ...)` constructor now
  automatically appends `errorCode.message()` to the error message.
  A protected constructor takes an explicit error message string for
  subclasses.

  `SysError` delegates to the protected constructor with `strerror(errNo)`.
  `WinError` delegates with `renderError(lastError)` (now static).

  This removes the need to manually append `e.code().message()` at call
  sites when converting `filesystem_error` to `SystemError`.

- Use perfect forwarding (`Args &&...` with `std::forward`) consistently
  in `BaseError`, `SystemError`, `SysError`, and `WinError` constructors.

Co-authored-by: Sergei Zimmerman <sergei@zimmerman.foo>
2026-02-18 12:29:11 -05:00
Sergei Zimmerman
db853cf4fb libutil: More robust check for NIX_UBSAN_ENABLED
In 3df91bea62 I forgot that the header
might get included out-of-tree with -Wundef. Let's make this a public
config option for libutil as it can affect function bodies in headers.
2026-02-18 17:33:51 +03:00
Sergei Zimmerman
3df91bea62 libexpr: Optimise Value::type(), ValueStorage::getInternalType()
Using nix::unreachable() in getInternalType() and type() turns
out to be quite expensive and prevents inlining. Also Value::type
got compiled to a jump table which has a high overhead from indirect
jumps. Using an explicit lookup table turns out to be more efficient.

This does mean that we lose out on nice diagnostics from nix::unreachable
calls, but this code is probably one of the hottests functions in the whole
evaluator, so I think the tradeoff is worth it. The nixUnreachableWhenHardened
boils down to nix::unreachable when UBSan is enabled so we still have good
coverage there.
2026-02-17 16:50:07 +03:00
John Ericson
06d4d5779f Merge pull request #15251 from obsidiansystems/file-system-at
Split `file-system-at.{cc,hh}` from `file-descriptor.{cc,hh}`
2026-02-16 20:10:28 +00:00
John Ericson
af7e585009 Split file-system-at.{cc,hh} from file-descriptor.{cc,hh}
`file-descriptor.{cc,hh}` was getting too big, split out
`file-system-at.{cc,hh}` for the FD-based file system stuff,
`file-descriptor.{cc,hh}` will only be for the fundamental primitives
that are file-system agnostic and work on almost all file types.

Review with `git show --color-moved` to see that this is indeed all
moving.
2026-02-16 14:21:52 -05:00
Robert Hensing
771421a34e fix(ref): improve cast exception type and add demangled type names
When ref::cast() fails, the error message was cryptic ("null pointer
cast to ref"). Now it throws a proper bad_ref_cast (a std::bad_cast
subclass) with a clear message showing the actual types involved:

    ref<nix::Base> cannot be cast to ref<nix::Derived>

This also adds a demangle.hh utility.
2026-02-16 17:07:40 +01:00
Robert Hensing
5aaa0cc4a6 refactor(ref): clarify implicit conversion semantics with requires clause
ref<Derived> was already implicitly convertible to ref<Base>, but the
mechanism was unclear and error messages for rejected downcasts were
more cryptic than necessary. This change:

- Adds RefImplicitlyUpcastableTo concept to constrain the conversion
  operator, making the intent explicit and improving error messages
- Documents .cast() and .dynamic_pointer_cast() as alternatives for
  explicit downcasting
- Adds unit tests for covariance behavior
2026-02-16 16:43:08 +01:00
John Ericson
95251a51dd Merge pull request #15241 from obsidiansystems/fix-isindir
libutil: fix `isInDir` rejecting paths starting with dot
2026-02-15 15:52:37 +00:00
Amaan Qureshi
ad0055e67c libutil: fix isInDir rejecting paths starting with dot
The old check rejected any relative path whose first character was a
dot, producing false negatives for valid descendants like `.ssh` or
`.config`. This commit changes the logic such that now it inspects the
first path component via `path::begin()`, only rejects `.` and `..`
rather than anything dot-prefixed. Fixes #15207.
2026-02-15 10:04:08 -05:00
Artemis Tosini
36d0e9580f Implement Pid::kill for Windows
Co-authored-by: John Ericson <John.Ericson@Obsidian.Systems>
2026-02-14 20:39:32 -05:00
Artemis Tosini
c9abefbc30 libutil-tests: Fix crash on Windows
libutil tests were crashing on Windows due to issues finding `environ`.
Replace process creation of `getEnv` with a new `getEnvOs` function that
uses native windows APIs.

Also convert a bunch of `RunOptions` fields to use `OsString` to better
reflect the underlying interfaces.

Co-authored-by: John Ericson <John.Ericson@Obsidian.Systems>
2026-02-14 20:39:32 -05:00
John Ericson
92d0fe000b Merge pull request #15188 from obsidiansystems/handleexceptions
Move `nix::handleExceptions` to libutil
2026-02-10 18:55:14 +00:00
Artemis Tosini
c79ff97c07 Move nix::handleExceptions to libutil
This is a fairly simple function, isolated from the rest of libmain
and could be useful if new programs are made that are not part of the
main nix-cli subproject.
2026-02-10 13:06:27 -05:00
John Ericson
ef659136ca std::filesystem::path in some Settings fields 2026-02-10 12:50:17 -05:00
Sergei Zimmerman
a900bf1548 meson: Add -Werror=return-type and -Werror=non-virtual-dtor flags
Some easy compile-time safety features to catch mistakes earlier.
Fixes some missing virtual destructors.
2026-02-10 00:02:00 +03:00
Sergei Zimmerman
36ad2962ca Merge pull request #14944 from iljah/patch-2
Use unreachable in nix::listNarImpl()
2026-02-09 20:12:33 +00:00
Sergei Zimmerman
3cd840d7f1 libutil: Fix error message in readLinkAt
More correctly describes the error, since we are always reading a relative path.
2026-02-09 00:57:13 +03:00
Sergei Zimmerman
6b90755cad libutil: Fix uninitialised variable in AutoDelete
Diagnosed by UBSan in hydra [1]:

SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /nix/store/m1k4nxs8r0fl0pjxqp5n37vxgms7gdlb-gcc-14.3.0/include/c++/14.3.0/bits/move.h:234:11 in
+(prefetch.sh:6) path=
++(prefetch.sh:6) onError
++(/build/source/tests/functional/common/functions.sh:241) set +x
prefetch.sh: test failed at:
  main in prefetch.sh:6

[1]: https://hydra.nixos.org/build/321098757/nixlog/1
2026-02-09 00:57:07 +03:00
Ilja
23ddb0bfc7 Throw on unsupported type in nix::listNarImpl()
Prevent
```
../src/libutil/nar-accessor.cc: In function ”nix::ListNarResult<deep> nix::listNarImpl(SourceAccessor&, const CanonPath&) [with bool deep = true]”:
../src/libutil/nar-accessor.cc:335:1: varoitus: ei-void-tyyppisen funktion loppu saavutettu [-Wreturn-type]
  335 | }
      | ^
../src/libutil/nar-accessor.cc: In function ”nix::ListNarResult<deep> nix::listNarImpl(SourceAccessor&, const CanonPath&) [with bool deep = false]”:
../src/libutil/nar-accessor.cc:335:1: varoitus: ei-void-tyyppisen funktion loppu saavutettu [-Wreturn-type]
```
2026-02-07 17:31:45 +03:00
Sergei Zimmerman
499ffaf940 NarIndexer: Implement skip
This improves the performance of parseNarListing, which is used by commands
like `nix nar ls` when the underlying source allows cheap seeks (like StringSource
or FdSource that does lseek).

For `nix nar ls` of a NAR for linux source tarball this cuts down the runtime almost
in half (from 300ms -> 175ms).
2026-02-06 20:55:04 +03:00
Sergei Zimmerman
ecda8c2329 Merge pull request #15149 from NixOS/align-up-robustness
libutil: Add overflow check to alignUp
2026-02-05 00:51:20 +00:00
Sergei Zimmerman
ea53914e47 Merge pull request #15147 from NixOS/fix-mingw-read-line
libutil: Fix mingw build
2026-02-04 23:14:52 +00:00
Sergei Zimmerman
d77c131df3 libutil: Add overflow check to alignUp
Old code with size + (size % 8 ? 8 - (size % 8) : 0) also suffered from this.
2026-02-05 02:04:33 +03:00
Sergei Zimmerman
72c2954625 libutil: Fix mingw build
This was broken by b038500b47.
2026-02-05 00:53:23 +03:00
Eelco Dolstra
27d5cc39c8 Merge pull request #14957 from NixOS/invalidate-lstat-cache
SourceAccessor: Allow the lstat cache to be invalidated
2026-02-04 21:44:04 +00:00
John Ericson
25ab7f5850 Merge pull request #15038 from obsidiansystems/drainfd-improvements
More IO portability cleanups
2026-02-04 21:36:18 +00:00
Eelco Dolstra
139d05af6f SourceAccessor: Allow cached information to be invalidated
After lockFlake() creates flake.lock in a PosixSourceAccessor, it
needs to be able to invalidate the lstat cache.
2026-02-04 21:56:17 +01:00
John Ericson
b489c8ea15 More IO portability cleanups
- options structs for `drainFD` (both versions)
- portable `read` wrapper
- portable `GetFileSize` wrapper
- dedup `readFile` and `drainFD` using the above
- Use `drainFD` in `PosixSourceAccessor`, avoiding manual IO
- Remove `fromDescriptorReadOnly` entirely!
2026-02-04 15:49:21 -05:00
John Ericson
a4c0295822 Merge pull request #15060 from NixOS/read-link-at
Support `readLinkAt` and `openFileEnsureBeneathNoSymlinks` on Windows too
2026-02-04 20:40:08 +00:00
Artemis Tosini
b038500b47 libutil: add terminator option to readLine
Some APIs use "lines" that end in `\0` instead of `\n`.
2026-02-04 11:58:34 -05:00
Sergei Zimmerman
47f261cc19 libutil: Add openNewFileForWrite helper function, wrap callsites
This is purely a fix to use CreateFileW in mingw builds. Also adds some
FIXMEs for suspicious symlink following on truncation that can probably
be tightened down without any problems (other than nix-channel), but for
now this is a no-op change other than consistently using O_CLOEXEC, which
is harmless.
2026-02-04 02:10:12 +03:00
Amaan Qureshi
de88141cdf build-result: Make Failure an alias for BuildError and remove exception parameters from goal 2026-01-28 19:32:41 -05:00
John Ericson
da9426b8fc Merge pull request #15095 from NixOS/small-fixes
Some small fixes
2026-01-28 21:27:06 +00:00
John Ericson
bcbc8ae4e3 Merge pull request #15110 from obsidiansystems/pid-cleanup
libutil: add useful functions to Pid
2026-01-28 20:34:00 +00:00
Artemis Tosini
538e82aa0b libutil: add useful functions to Pid
The C++ rule of five suggests that when a custom destructor is needed
then several other functions are as well. The lack of those makes
certain operations challenging
2026-01-28 13:47:09 -05:00
Artemis Tosini
0fc20e3e20 Refactor fstat into a wrapper in libutil
We use a different fstat on posix and windows systems,
and not all fstat users were using the correct one.
Factor out fstat to make the change easier.

See also a13de50df3 for other stat
functions
2026-01-28 12:53:54 -05:00
John Ericson
c0ab135860 Some small fixes
I think this has to do with the 25.11 bump.
2026-01-27 00:06:59 -05:00
John Ericson
5dfd81cbc0 Clean up AutoRemoveJail, AutoDelete, and AutoUnmount
- Extract destructor logic into named methods (`deletePath()`,
  `unmount()`, and `remove()`) that can be called explicitly. These ones
  will throw exceptions normally, unlike the destructor which must quash
  them to avoid double-exceptions.

- Use `std::filesystem::path` in `AutoUnmount` (changed from `Path`)

- Remove `del` field from `AutoRemoveJail`, using `INVALID_JAIL`
  sentinel value instead.

- Add move assignment operators implemented via friend `swap` functions
  for all three RAII classes.

- Remove old `reset(...)` methods that took parameters. These were a bit
  misleading --- do they cancel or immediately destroy? --- and doing it
  explicitly with cancel and then assignment is not hard.

Co-authored-by: Sergei Zimmerman <sergei@zimmerman.foo>
2026-01-26 17:26:59 -05:00
John Ericson
e8d1cb0668 Merge pull request #15086 from JustAGuyTryingHisBest/darwin-pathfmt
libutil: unix filesystem add more PathFmt
2026-01-26 20:49:51 +00:00
John Ericson
252e4ee5ca Merge pull request #15077 from NixOS/stat-wrapper
Cleanup stat usage
2026-01-26 19:55:00 +00:00
Some Guy
7f95112fac libutil: unix filesystem add more PathFmt 2026-01-26 11:39:34 -08:00
John Ericson
b190548c83 Merge pull request #15040 from NixOS/factor-out-nar-cache-0
Factor out `NarCache` from `RemoteFSAccessor`
2026-01-26 19:19:39 +00:00
John Ericson
a13de50df3 Cleanup stat usage
Use wrappers to make error handling easier.

On Windows we are using proper 64-bit time and size info.

We still have the problem of no `lstat` on Windows, but this will be
dealt with in future PRs.
2026-01-26 14:00:19 -05:00
John Ericson
6e2e53a8d2 Share the exception-using chmod wrapper with more code
It is not just useful to `DerivationBuilder`.
2026-01-24 17:03:48 -05:00
John Ericson
1100c9dc23 Support readLinkAt and openFileEnsureBeneathNoSymlinks on Windows too
This means that `RestoreSink` can work in the TOCTOU-resilliant way on
Windows too. And it also bodes will for the upcoming OS source accessor
improvements.

A few misc little refactors around error handling and whatnot are done
along the way too. (No more attempt to support pre Windows Vista! lol.)

This cannot be realiably automatically tested until we have a newer
version of Wine, but it does build, so I am inclined to say we just try
it for now.
2026-01-22 17:35:28 -05:00
John Ericson
b7ddbb8e2d Merge pull request #15039 from NixOS/read-link-at
libutil: Add unix::readLinkAt function
2026-01-22 15:11:54 +00:00
John Ericson
fe8f574471 Clean up NarAccessor construction
We had a minor combinatorial explosion of ways to do things. We can get
rid of those by just having the caller call `parseNarListing` intead.
2026-01-21 19:25:52 -05:00
Eelco Dolstra
0b2dffefea Factor out NarCache from RemoteFSAccessor
Use

```
git show --color-moved --patience --color-moved-ws=ignore-all-space
```

to review and see that this is mostly code motion.

Co-Authored-By: John Ericson <John.Ericson@Obsidian.Systems>
2026-01-21 19:15:51 -05:00
Sergei Zimmerman
52100c6ee1 libutil: Add unix::readLinkAt function
This will be used for TOCTOU-free NAR serialisation and recursive copying.
2026-01-22 02:50:39 +03:00
John Ericson
fa53a9cec8 Deduplicate copyFdRange with new readOffset 2026-01-21 18:16:28 -05:00