Befator-Inc-Firmen-Netzwerk/nix
3
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
21,094 Commits 207 Branches 226 Tags
8d257f5510dbfbab8a74e2b7b0ff60bcd720e141
Commit Graph
12683 Commits
Author SHA1 Message Date
Eelco Dolstra 8d257f5510 EvalState: Make the counters atomic 2025-09-25 08:03:24 +02:00
John Ericson 30691c38c2 Add JSON tests for Realisation 2025-09-24 18:09:24 -04:00
John Ericson 9f26c20ebd Merge pull request #14057 from obsidiansystems/derived-path-json 
Modernize and test derived path JSON
 2025-09-24 18:08:06 -04:00
John Ericson f2cd6235a7 Merge pull request #14056 from obsidiansystems/better-output-spec-json-tests 
Convert `{Extended,}OutputsSpec` JSON tests to characterization tests
 2025-09-24 17:41:25 -04:00
Sergei Zimmerman 05279f2ba0 Merge pull request #14061 from xokdvium/k-way-update 
libexpr: Preparation for more k-way updates of attribute sets (NFC)
 2025-09-25 00:29:54 +03:00
John EricsonandSergei Zimmerman a97d6d89d8 Create a second Store::getFSAccessor for a single store object 
This is sometimes easier / more performant to implement, and
independently it is also a more convenient interface for many callers.

The existing store-wide `getFSAccessor` is only used for

 - `nix why-depends`
 - the evaluator

I hope we can get rid of it for those, too, and then we have the option
of getting rid of the store-wide method.

Co-authored-by: Sergei Zimmerman <sergei@zimmerman.foo>
 2025-09-24 15:49:14 -04:00
Sergei Zimmerman 97ce7759d0 libexpr: Use same naive iterative merging but with evalForUpdate 2025-09-24 21:47:59 +03:00
Sergei Zimmerman 9789019a50 libexpr: Move *StackReservation constants to gc-small-vector.hh 
There are other places where it's useful to use these constants
(notably in eval.hh).
 2025-09-24 01:05:18 +03:00
Sergei Zimmerman b7c6cf900f libexpr: Explicitly define ExprOpUpdate 2025-09-24 01:04:26 +03:00
Sergei Zimmerman e282175f48 libexpr: Split out MakeBinOpMembers from MakeBinOp 2025-09-24 01:04:23 +03:00
John Ericson d23e59bb6b Modernize and test derived path JSON 
Old code is now just used for `nix build` --- there is no CLI breaking
change.

Test the new format, too.

The new format is not currently used, but will be used going forward,
for example in the C API.

Progress on #13570
 2025-09-23 15:05:56 -04:00
Eelco Dolstra 73d3ab05b6 Merge pull request #14054 from obsidiansystems/to-json-no-copy 
Fix `JSON_IMPL` macro to avoid extraneous copies
 2025-09-23 20:58:12 +02:00
John Ericson f24e00710e Convert {Extended,}OutputsSpec JSON tests to characterization tests 
This brings them in line with the other tests, and furthers my goals of
separating unit test data from code.

Doing this cleanup as part of my #13570 effort, but strictly-speaking,
this is separate as these data types' JSON never contained and store
paths or store dirs, just simple output name strings.
 2025-09-23 14:47:00 -04:00
John Ericson 1c71cb4005 Remove some pointless std::visit 
These are not needed, because the `toJSON` methods are already
implemented for the variant wrapper too.
 2025-09-23 13:59:27 -04:00
John Ericson af71a9dbd9 Fix JSON_IMPL macro to avoid extraneous copies 
Should take the thing we're serializing by reference.
 2025-09-23 13:05:12 -04:00
Jörg Thalheim af82c847a7 Merge pull request #14048 from roberth/shellcheck 
Shellcheck
 2025-09-22 21:57:42 +02:00
John Ericson 5292b0e49e Merge pull request #14038 from NixOS/thread-safe-dummy 
libstore: Make writable dummy store thread-safe
 2025-09-22 14:26:35 -04:00
Robert Hensing 5af644492b nix develop: Apply shellcheck 2025-09-22 19:31:22 +02:00
Sergei Zimmerman 5915fe3190 Revert "Use shared pointers in the memory source accessor" 
This is no longer necessary.

This reverts commit 4df60e639b.
 2025-09-22 20:22:17 +03:00
Sergei Zimmerman c4c92c4c61 libstore: Make writable dummy store thread-safe 
Tested by building with b_sanitize=thread and running:

nix flake prefetch-inputs --store "dummy://?read-only=false"

It might make sense to move this utility class out of dummy-store.cc,
but it seems fine for now.
 2025-09-22 20:22:16 +03:00
Sergei Zimmerman c71f80b6eb libstore: Implement boost::hash for StorePath 2025-09-22 20:16:11 +03:00
John EricsonandRobert Hensing 91593a237f Convert Realisation JSON logic to standard style 
No behavior is changed, just:

- Declare a canonical `nlohmnan::json::adl_serializer`

- Use `json-utils.hh` to shorten code without getting worse error
  messages.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
 2025-09-22 12:59:37 -04:00
Robert Hensing 169a368459 Merge pull request #14040 from NixOS/import-thunk 
Ensure that files are parsed/evaluated only once (2nd attempt)
 2025-09-22 18:33:00 +02:00
Eelco Dolstra a0103fc302 Merge pull request #13852 from lovesegfault/warn-no-kvm 
feat(libstore): warn when kvm is enabled but /dev/kvm isn't available
 2025-09-22 18:31:57 +02:00
David McFarland 32d4ea8140 fix cross-build for cygwin 2025-09-22 12:27:04 -03:00
Eelco Dolstra d32d77f4d4 Allocate ExprParseFile on the heap for now 
https://github.com/NixOS/nix/pull/14013#issuecomment-3308085755
 2025-09-22 12:06:51 +02:00
Eelco Dolstra 5f60602875 Reapply "Merge pull request #13938 from NixOS/import-thunk" 
This reverts commit fd034814dc.
 2025-09-22 11:48:58 +02:00
Robert Hensing 71b27774f0 libexpr: Document {eval,maybeThunk} methods 2025-09-22 01:41:02 +03:00
Sergei Zimmerman b66c357b58 libstore: Implement DummyStore::narFromPath 2025-09-21 13:19:50 +03:00
Sergei Zimmerman 3a4c618483 libstore: Fix typo in description of dummy store 2025-09-21 13:19:49 +03:00
Sergei Zimmerman ed9b377928 libstore: Disable path info cache for dummy store 2025-09-21 13:19:48 +03:00
Sergei Zimmerman 341878ce0f libstore: Make dummy store also store path info 2025-09-21 13:19:47 +03:00
Sergei Zimmerman 02c9ac445f libutil: Improve handling of non-directory root in MemorySourceAccessor 2025-09-21 13:19:45 +03:00
John Ericson 4df60e639b Use shared pointers in the memory source accessor 
This allows aliasing, like hard links.
 2025-09-21 13:19:44 +03:00
Sergei Zimmerman e04381edbd libfetchers/github: Use getFSAccessor for downloadFile result 
We should use proper abstractions for reading files from the store.
E.g. this caused errors when trying to download github flakes into
an in-memory store in #14023.
 2025-09-21 01:12:42 +03:00
Robert Hensing 2d1b412e5b libexpr-tests: Enable when test setup for building succeeds 
Accidentally disabled by 9bc218ca3f
 2025-09-19 23:39:00 +02:00
Bernardo Meurer Costa b63d9fbc87 test(libstore): additional ParsedS3Url tests 
Extracted from the work in #13752
 2025-09-19 20:52:44 +00:00
John Ericson 936334638d Merge pull request #14020 from xokdvium/dummy-store-fixes 
libstore: Set display prefix for dummy store
 2025-09-19 00:00:00 -04:00
Sergei Zimmerman 72e2b0efea libstore: Set display prefix for dummy store 
Otherwise the prefix is «unknown».
 2025-09-19 02:28:59 +03:00
Sergei ZimmermanandJade Lovelace 94d37e62fc treewide: Support builds with ASAN, enable in CI 
Enables builds with ASAN to catch memory corruption
bugs faster and in CI. This is an incredibly valuable
instrument that must be used as much as possible.

Somewhat based on jade's work from Lix, though there's a lot that
we have to do differently:

https://git.lix.systems/lix-project/lix/commit/19ae87e5cec71912c7e7ecec5dc8ff18d18c60ee

Co-authored-by: Jade Lovelace <lix@jade.fyi>
 2025-09-19 01:33:57 +03:00
Jörg Thalheim 4808b0e24f Merge pull request #14014 from xokdvium/fish delete 
libexpr-c: Fix mismatched new/delete
 2025-09-18 08:08:39 +02:00
Sergei Zimmerman 309d55807c libexpr-c: Fix mismatched new/delete 
This leads to ASAN errors:

==1137785==ERROR: AddressSanitizer: new-delete-type-mismatch on 0x523000001d00 in thread T0:
  object passed to delete has wrong type:
  size of the allocated type:   5968 bytes;
  size of the deallocated type: 5968 bytes.
  alignment of the allocated type:   8 bytes;
  alignment of the deallocated type: default-aligned.
 2025-09-18 02:58:32 +03:00
Sergei Zimmerman fd034814dc Revert "Merge pull request #13938 from NixOS/import-thunk" 
This has multiple dangling pointer issues that lead to segfaults in e.g.:

nix eval --expr '(builtins.getFlake "github:nixos/nixpkgs/25.05")' --impure

This reverts commit ad175727e4, reversing
changes made to d314750174.
 2025-09-18 01:52:46 +03:00
Robert Hensing b4fcb64276 Merge pull request #13980 from obsidiansystems/drv-json-issue-13570 
Make the JSON format for derivation use basename store paths
 2025-09-17 23:31:41 +02:00
Sergei Zimmerman 6138bc3de3 libexpr: Structural sharing of attrsets 
This changes the implementation of Bindings to allow
for a more space-efficient implementation of attribute
set merges. This is accomplished by "layering" over the "base" Bindings.
The top "layer" is naturally the right-hand-side of the update operator //.

Such an implementation leads to significantly better memory usage on
something like nixpkgs:

nix-env --query --available --out-path --file ../nixpkgs --eval-system x86_64-linux > /dev/null

Comparison against 2b0fd88324 for x86_64-linux on nixpkgs f06c7c3b6f5074dbffcf02542fb86af3a5526afa:

| metric                 | mean_before     | mean_after      | mean_diff       | mean_%_change | p_value | t_stat  |
| -                      | -               | -               | -               | -             | -       | -       |
| cpuTime                | 21.1520         | 21.3414         | 0.1894          | 0.7784        | 0.3190  | 1.0219  |
| envs.bytes             | 461451951.6190  | 461451951.6190  | -               | -             | -       | -       |
| envs.elements          | 34344544.8571   | 34344544.8571   | -               | -             | -       | -       |
| envs.number            | 23336949.0952   | 23336949.0952   | -               | -             | -       | -       |
| gc.cycles              | 7.5238          | 7.2857          | -0.2381         | -4.6825       | 0.0565  | -2.0244 |
| gc.heapSize            | 1777848124.9524 | 1252162023.6190 | -525686101.3333 | -29.9472      | 0.0000  | -8.7041 |
| gc.totalBytes          | 3102787383.6190 | 2498431578.6667 | -604355804.9524 | -19.7704      | 0.0000  | -9.3502 |
| list.bytes             | 59928225.9048   | 59928225.9048   | -               | -             | -       | -       |
| list.concats           | 1240028.2857    | 1240028.2857    | -               | -             | -       | -       |
| list.elements          | 7491028.2381    | 7491028.2381    | -               | -             | -       | -       |
| nrAvoided              | 28165342.2381   | 28165342.2381   | -               | -             | -       | -       |
| nrExprs                | 1577412.9524    | 1577412.9524    | -               | -             | -       | -       |
| nrFunctionCalls        | 20970743.4286   | 20970743.4286   | -               | -             | -       | -       |
| nrLookups              | 10867306.0952   | 10867306.0952   | -               | -             | -       | -       |
| nrOpUpdateValuesCopied | 61206062.0000   | 25748169.5238   | -35457892.4762  | -58.8145      | 0.0000  | -8.9189 |
| nrOpUpdates            | 2167097.4286    | 2167097.4286    | -               | -             | -       | -       |
| nrPrimOpCalls          | 12337423.4286   | 12337423.4286   | -               | -             | -       | -       |
| nrThunks               | 29361806.7619   | 29361806.7619   | -               | -             | -       | -       |
| sets.bytes             | 1393822818.6667 | 897587655.2381  | -496235163.4286 | -36.7168      | 0.0000  | -9.1115 |
| sets.elements          | 84504465.3333   | 48270845.9524   | -36233619.3810  | -43.8698      | 0.0000  | -8.9181 |
| sets.number            | 5218921.6667    | 5218921.6667    | -               | -             | -       | -       |
| sizes.Attr             | 16.0000         | 16.0000         | -               | -             | -       | -       |
| sizes.Bindings         | 8.0000          | 24.0000         | 16.0000         | 200.0000      | -       | inf     |
| sizes.Env              | 8.0000          | 8.0000          | -               | -             | -       | -       |
| sizes.Value            | 16.0000         | 16.0000         | -               | -             | -       | -       |
| symbols.bytes          | 1368494.0952    | 1368494.0952    | -               | -             | -       | -       |
| symbols.number         | 109147.1905     | 109147.1905     | -               | -             | -       | -       |
| time.cpu               | 21.1520         | 21.3414         | 0.1894          | 0.7784        | 0.3190  | 1.0219  |
| time.gc                | 1.6011          | 0.8508          | -0.7503         | -37.1507      | 0.0017  | -3.6328 |
| time.gcFraction        | 0.0849          | 0.0399          | -0.0450         | -37.4504      | 0.0035  | -3.3116 |
| values.bytes           | 615968144.7619  | 615968144.7619  | -               | -             | -       | -       |
| values.number          | 38498009.0476   | 38498009.0476   | -               | -             | -       | -       |

Overall this does slow down the evaluator slightly (no more than ~10% in most cases),
but this seems like a very decent tradeoff for shaving off 33% of memory usage.
 2025-09-17 23:54:45 +03:00
John Ericson 9d7229a2a4 Make the JSON format for derivation use basename store paths 
See #13570 for details --- the idea is that included the store dir in
store paths makes systematic JSON parting with e.g. Serde, Aeson,
nlohmann, or similiar harder.

After talking to Eelco, we are changing the `Derivation` format right
away because not only is `nix derivation` technically experimental, we think it is
also less widely used in practice than, say, `nix path-info`.

Progress on #13570
 2025-09-17 16:38:17 -04:00
John EricsonHaeNoeEelco Dolstra
 		d5ce8c3caa Use MemorySourceAccessor in DummyStore 
Add `read-only` setting to `dummy://` store for back compat.

Test by changing an existing test to use this instead, fixing a TODO.

Co-Authored-By: HaeNoe <git@haenoe.party>
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
 2025-09-17 16:09:53 -04:00
John Ericson 168c24b605 Declare DummyStoreConfig in a header 
This will useful for unit tests.
 2025-09-17 15:54:30 -04:00
John Ericson 613de9d9cc Add missing #pragma once 2025-09-17 15:21:21 -04:00
John Ericson 9bc218ca3f Add new C API for working with derivations 
Also test the APIs we just added.
 2025-09-16 13:25:36 -04:00