4fac767b52
During garbage collection we cache several things -- a set of known-dead paths, a set of known-alive paths, and a map of paths to their derivers. Currently they use STL maps and sets, which are ordered structures that typically are backed by binary trees. Since we are putting pseudorandom paths into these and looking them up by exact key, we don't need the ordering, and we're paying a nontrivial cost per insertion. The existing maps require O(n log n) memory and have O(log n) insertion and lookup time. We could instead use unordered maps, which are typically backed by hashmaps. These require O(n) memory and have O(1) insertion and lookup time. On my system this appears to result in a dramatic speedup -- prior to this patch I was able to delete 400k paths out of 9.5 million over the course of 34.5 hours. After this patch the same result took 89 minutes. This result should NOT be taken at face value because the two runs aren't really comparable; in particular the first started when I had 9.5 million store paths and the seconcd started with 7.8 million, so we are deleting a different set of paths starting from a much cleaner filesystem. But I do think it's indicative. Related: https://github.com/NixOS/nix/issues/9581
Nix
Nix is a powerful package manager for Linux and other Unix systems that makes package management reliable and reproducible. Please refer to the Nix manual for more details.
Installation and first steps
Visit nix.dev for installation instructions and beginner tutorials.
Full reference documentation can be found in the Nix manual.
Building and developing
Follow instructions in the Nix reference manual to set up a development environment and build Nix from source.
Contributing
Check the contributing guide if you want to get involved with developing Nix.
Additional resources
Nix was created by Eelco Dolstra and developed as the subject of his PhD thesis The Purely Functional Software Deployment Model, published 2006. Today, a world-wide developer community contributes to Nix and the ecosystem that has grown around it.
- The Nix, Nixpkgs, NixOS Community on nixos.org
- Official documentation on nix.dev
- Nixpkgs is the largest, most up-to-date free software repository in the world
- NixOS is a Linux distribution that can be configured fully declaratively
- Discourse
- Matrix
License
Nix is released under the LGPL v2.1.