Compare commits

..
172 Commits
Author SHA1 Message Date
Eelco Dolstra 3e95bb49be * Move this back temporarily. 2004-11-14 15:04:51 +00:00
Eelco Dolstra 84cc89b585 * Tag release 0.6. 2004-11-14 12:58:20 +00:00
Eelco Dolstra ac76ee5088 * Mark stable release. 2004-11-14 12:09:31 +00:00
Eelco Dolstra f8e4f731d4 * Release branch for Nix 0.6. 2004-11-14 12:04:21 +00:00
Eelco Dolstra 9f8964a062 * More manual fixes. 2004-11-14 10:42:16 +00:00
Eelco Dolstra 0b79a12082 * Manual fixes. 2004-11-14 00:24:57 +00:00
Eelco Dolstra 54c7a870d5 * Document --delete-generations and other nix-env options. 2004-11-12 23:56:37 +00:00
Eelco Dolstra c2b0d6b02f * Document --eval-only and --parse-only options in nix-instantiate. 2004-11-12 23:22:08 +00:00
Eelco Dolstra 32c7326850 * Typos. 2004-11-09 14:06:56 +00:00
Eelco Dolstra 92ee003dc9 * Fix broken format string. 2004-11-08 15:20:52 +00:00
Eelco Dolstra d6db574ec1 * Check exit status of pipe elements. 2004-11-08 11:32:10 +00:00
Eelco Dolstra b8aaef5e4e * Documented the standard environment, including the generic builder. 2004-11-07 22:12:16 +00:00
Eelco Dolstra 2c3b29c5ca * Everything you always wanted to know about functions and derivations
but were afraid to ask.
2004-11-07 20:36:45 +00:00
Eelco Dolstra ea6581b691 * Drop the grammar appendix. 2004-11-07 20:36:10 +00:00
Eelco Dolstra 09e7f06818 * Put something in here. 2004-11-07 20:30:02 +00:00
Eelco Dolstra 1bac7a10e6 * Operators, comments. 2004-11-07 18:58:49 +00:00
Eelco Dolstra 55b35d6d77 * Lets, inheritance, assertions. 2004-11-07 13:53:07 +00:00
Eelco Dolstra 0b1ee4802b * Typo fix. 2004-11-05 21:12:23 +00:00
Eelco Dolstra 5f0300d18c * Generic builders. 2004-11-05 21:11:01 +00:00
Eelco Dolstra 3e9d2038b4 * Start of language reference. 2004-11-05 15:39:30 +00:00
Eelco Dolstra 6ca9c7f0a9 * Finished GNU Hello walkthrough. 2004-11-05 13:10:08 +00:00
Eelco Dolstra 8b934694f2 * Manual: writing Nix expressions. 2004-11-04 20:21:08 +00:00
Eelco Dolstra feb3ceaee0 * Better error messages. 2004-11-04 20:20:39 +00:00
Eelco Dolstra cb7ccb528b * string2ATerm -> overloaded toATerm. 2004-11-03 18:12:03 +00:00
Eelco Dolstra 4cbd845aa4 * Don't propagate our CFLAGS to the ATerm library since it breaks at
-O2.
2004-11-03 16:51:09 +00:00
Eelco Dolstra 5f2c5a306c * chapter -> appendix. 2004-11-02 08:25:29 +00:00
Eelco Dolstra 0913f5a615 * Section about channels. 2004-11-01 16:21:37 +00:00
Eelco Dolstra ee5dcfade2 * Section about garbage collection. 2004-11-01 16:03:35 +00:00
Eelco Dolstra cbe8de592d * Profiles section. 2004-11-01 12:02:44 +00:00
Eelco Dolstra b05a596d61 * Document setuid Nix installs. 2004-10-31 17:08:48 +00:00
Eelco Dolstra 0d80d237c5 * Add figures to make install' / make dist'. 2004-10-31 16:13:25 +00:00
Eelco Dolstra 2aa1f4717b * Fix File exists' errors if the result' symlink exists but is
dangling.
2004-10-31 12:01:16 +00:00
Eelco Dolstra f8ac8d1ec8 * Began adding build farm docs. 2004-10-29 15:26:26 +00:00
Eelco Dolstra a69534fc21 * Drop ATmake / ATMatcher also in handling store expressions. 2004-10-29 11:22:49 +00:00
Eelco Dolstra ed09821859 * Use atdiff' instead of cmp' for checking test output.
* Don't use local file names in tests since they will produce
  different parse trees depending on the current directory.
2004-10-27 13:12:58 +00:00
Eelco Dolstra 3277c9432a * Bug fix in parsing of /* ... */ comments; due to longest match
regexp there could be only one such comment per file.
2004-10-27 13:00:31 +00:00
Eelco Dolstra 463e2817c5 * Remove ancient Fix tests.
* Add automated Nix expression language tests.
2004-10-27 12:41:53 +00:00
Eelco Dolstra f09618b63a * Turn on read-only mode in queries. This prevents redundant store I/O. 2004-10-27 10:24:44 +00:00
Eelco Dolstra c7bea941b0 * Oops, I did it again. 2004-10-27 10:05:51 +00:00
Eelco Dolstra 210ab0296d * Add file to `make dist'. 2004-10-27 00:02:31 +00:00
Eelco Dolstra c52dda95a6 * Bug: check that term is an application. 2004-10-26 23:30:18 +00:00
Eelco Dolstra 5fe9222b36 * Don't use ATmake / ATmatch anymore, nor the ATMatcher class.
Instead we generate data bindings (build and match functions) for
  the constructors specified in `constructors.def'.  In particular
  this removes the conversions between AFuns and strings, and Nix
  expression evaluation now seems 3 to 4 times faster.
2004-10-26 22:54:26 +00:00
Eelco Dolstra eb8284ddaa * Evaluate argument to `import'. 2004-10-26 17:10:09 +00:00
Eelco Dolstra 033d7c6593 * Doh! 2004-10-26 17:04:55 +00:00
Eelco Dolstra 9fa07b376d * String/path concatenation operator (`+'). 2004-10-26 17:01:35 +00:00
Eelco Dolstra ee401afad8 * Mode --parse-only' to parse the input (on stdin, -'), and print
out the AST as an ATerm.
* Mode `--eval-only' to parse and evaluate the input, and print the
  resulting normal form as an ATerm.

Neither of these modes require store/DB write permission.
2004-10-26 16:59:36 +00:00
Eelco Dolstra 37d7abd694 * New language feature: with expressions.
The expression `with E1; E2' evaluates to E2 with all bindings in
  the attribute set E1 substituted.  E.g.,

    with {x = 123;}; x

  evaluates to 123.  That is, the attribute set E1 is in scope in E2.

  This is particularly useful when importing files containing lots
  definitions.  E.g., instead of

    let {
      inherit (import ./foo.nix) a b c d e f;

      body = ... a ... f ...;
    }

  we can now say

    with import ./foo.nix;

    ... a ... f ...

  I.e., we don't have to say what variables should be brought into scope.
2004-10-25 16:54:56 +00:00
Eelco Dolstra f4d44a0026 * Allow certain operations to succeed even if we don't have write
permission to the Nix store or database.  E.g., `nix-env -qa' will
  work, but `nix-env -qas' won't (the latter needs DB access).  The
  option `--readonly-mode' forces this mode; otherwise, it's only
  activated when the database cannot be opened.
2004-10-25 14:38:23 +00:00
Eelco Dolstra 3ade3e7721 * Revert r1594 - it didn't solve the problem. Instead add
svn-revision to distributions, which should fix it.
2004-10-25 13:51:34 +00:00
Eelco Dolstra 2248becfd3 * Make sure that the prerelease version is included in `--version'. 2004-10-25 12:15:50 +00:00
Eelco Dolstra 50b9caac14 * Updated NEWS for the upcoming 0.6 release. 2004-10-21 09:22:16 +00:00
Eelco Dolstra 2155c0a673 * Register channels as roots of the garbage collector (in
$(localstatedir)/nix/gcroots/channels). 
* In setuid installations, create gcroots/tmp and gcroots/channels
  group-writable.
2004-10-20 14:42:38 +00:00
Eelco Dolstra 88888160d2 * Fix nix-prefetch-url in setuid Nix installations. 2004-10-20 14:40:54 +00:00
Eelco Dolstra 99da51d4de * Show error messages from curl. 2004-10-20 14:05:48 +00:00
Eelco Dolstra 2cd590d96c * Instead of — use the actual Unicode character. By the way, to
edit the manual, you should have something like

    (modify-coding-system-alist 'file "\\.xml\\>" 'utf-8)

  in your ~/.emacs.
2004-10-18 12:22:14 +00:00
Eelco Dolstra 692204e0c5 * Rewrite of package management stuff. 2004-10-14 16:43:09 +00:00
Eelco Dolstra d830b2c1df * In `nix-env -q', sort derivations by name *without* case
sensitivity.
2004-10-14 15:09:55 +00:00
Eelco Dolstra febd8bed1b * Split overview chapter into a chapter on package management and a
chapter on writing Nix expressions.
2004-10-14 11:55:12 +00:00
Eelco Dolstra 98c69e5172 * Unindent. 2004-10-14 11:54:41 +00:00
Eelco Dolstra 371c57d8a7 * Updated the quick start section. Use channels instead of
downloading Nix expressions and calling nix-pull.  This is so
  user-friendly that even a Mac user can do it! :-)
2004-10-13 15:35:47 +00:00
Eelco Dolstra 2b20701f78 * Better introduction.
* Set notes in a different color than warnings.
2004-10-13 15:08:35 +00:00
Eelco Dolstra 1317242780 * Make store objects created by substitutes read-only. 2004-09-22 12:15:04 +00:00
Niels Janssen 995d08208e * prevent collision on log directory 2004-09-19 15:53:37 +00:00
Eelco Dolstra b357284a32 * Fallback didn't work for subderivations of an unnormalised the main
derivation, since NormalisationGoal would first run a
  NormalisationGoal on the subderivation (a no-op, since in a
  situation where we need fallback the successor is known), and then
  runs a RealisationGoal on the normal form, which then cannot do a
  fallback because it doesn't know the derivation expression for which
  it is a normal form.

  Tossed out the 2-phase normalisation/realisation in
  NormalisationGoal and SubstitutionGoal since it's no longer needed -
  a RealisationGoal will run a NormalisationGoal if necessary.
2004-09-12 19:08:57 +00:00
Eelco Dolstra dcc433de47 * Operation `--delete-generations' to delete generations of a
profile.  Arguments are either generation number, or `old' to delete
  all non-current generations.  Typical use:

  $ nix-env --delete-generations old
  $ nix-collect-garbage

* istringstream -> string2Int.
2004-09-10 13:32:08 +00:00
Eelco Dolstra c16be6ac92 * Remove write permission from store objects after they have been
added to the store.  Bug reported by Martin.
2004-09-09 21:19:20 +00:00
Eelco Dolstra 47f87072ad * A very dirty hack to make setuid installations a bit nicer to use.
Previously there was the problem that all files read by nix-env
  etc. should be reachable and readable by the Nix user.  So for
  instance building a Nix expression in your home directory meant that
  the home directory should have at least g+x or o+x permission so
  that the Nix user could reach the Nix expression.  Now we just
  switch back to the original user just prior to reading sources and
  the like.  The places where this happens are somewhat arbitrary,
  however.  Any scope that has a live SwitchToOriginalUser object in
  it is executed as the original user.

* Back out r1385.  setreuid() sets the saved uid to the new
  real/effective uid, which prevents us from switching back to the
  original uid.  setresuid() doesn't have this problem (although the
  manpage has a bug: specifying -1 for the saved uid doesn't leave it
  unchanged; an explicit value must be specified).
2004-09-09 21:12:53 +00:00
Eelco Dolstra 5396304c73 * Use setre[ug]id() instead of setres[ug]id(), since the former is
more common than the latter (which exists only on Linux and
  FreeBSD).  We don't really care about dropping the saved IDs since
  there apparently is no way to quiry them in any case, so it can't
  influence the build (unlike the effective IDs which are checked by
  Perl for instance).
2004-09-09 15:55:31 +00:00
Eelco Dolstra e043fc7d0b * Set the umask to known value (0022). This is important in a
setuid installation, since the calling user may have a more fascist
  umask (say, 0077), which would cause the store objects built by Nix
  to be unreadable to anyone other than the Nix user.
2004-09-09 14:16:02 +00:00
Eelco Dolstra 550d960586 * Hack for perl(readmanifest) dependency. 2004-09-08 12:07:19 +00:00
Eelco Dolstra 17c8252fc9 * Spec file options to create the Nix user and group in the RPM
pre-install script.  By default this is turned off; you should edit
  the spec file to enable it.
2004-09-06 10:05:21 +00:00
Eelco Dolstra fb28cfc86d * Add some variability to RPM spec files: allow setuid options to be
set on the rpmbuild command line.
2004-09-06 08:17:55 +00:00
Eelco Dolstra 5c443b6550 * Main the `substitutes-rev' table again, but now in a way that
doesn't take \Theta(n^2) space/time complexity.
2004-08-31 16:13:10 +00:00
Eelco Dolstra c25f2883b1 * Quadruple the Berkeley DB locking limits to get rid of out of memory
errors while running `nix-store --verify'.
2004-08-31 10:50:08 +00:00
Eelco Dolstra fe122c5a15 * Removed nrWaitees field. It was redundant with waitees.size() and
could get out of sync if multiple input derivations mapped to the
  same closure expression (since waitees is a set).
2004-08-30 11:51:36 +00:00
Eelco Dolstra eb233e728f * `--min-age' flag in nix-store and nix-collect-garbage to only delete
unreachable paths that haven't been used for N hours.  For instance,
  `nix-collect-garbage --min-age 168' only deletes paths that haven't
  been accessed in the last week.

  This is useful for instance in the build farm where many derivations
  can be shared between consecutive builds, and we wouldn't want a
  garbage collect to throw them all away.  We could of course register
  them as roots, but then we'd to unregister them at some point, which
  would be a pain to manage.  The `--min-age' flag gives us a sort of
  MRU caching scheme.

  BUG: this really shouldn't be in gc.cc since that violates
  mechanism/policy separation.
2004-08-25 16:54:08 +00:00
Eelco Dolstra fdec72c6cc * `nix-collect-garbage' now actually performs a garbage collection, it
doesn't just print the set of paths that should be deleted.  So
  there is no more need to pipe the result into `nix-store --delete'
  (which doesn't even exist anymore).
2004-08-25 15:39:13 +00:00
Eelco Dolstra 818047881e * Put the garbage collector in nix-store: operation `--gc',
suboperations `--print-live', `--print-dead', and `--delete'.  The
  roots are not determined by nix-store; they are read from standard
  input.  This is to make it easy to customise what the roots are.

  The collector now no longer fails when store expressions are missing
  (which legally happens when using substitutes).  It never tries to
  fetch paths through substitutes.

  TODO: acquire a global lock on the store while garbage collecting.
  
* Removed `nix-store --delete'.
2004-08-25 11:43:49 +00:00
Eelco Dolstra 9994c1dd9f * Validate derivation names. In particular don't allow spaces.
* Drop support for the outPath attribute in derivations.
2004-08-24 11:46:05 +00:00
Eelco Dolstra 8f58733ef1 * The gid should also match. 2004-08-20 15:47:58 +00:00
Eelco Dolstra 1c90fabccc * Unbreak programs that are not setuid (such as nix-hash). 2004-08-20 15:31:46 +00:00
Eelco Dolstra e77fbe0fa2 * On systems that have the setresuid() and setresgid() system calls to
set the real uid and gid to the effective uid and gid, the Nix
  binaries can be installed as owned by the Nix user and group instead
  of root, so no root involvement of any kind is necessary.

  Linux and FreeBSD have these functions.
2004-08-20 15:22:33 +00:00
Eelco Dolstra 2d35116c13 * Setuid support for sharing a Nix installation between multiple
users.

  If the configure flag `--enable-setuid' is used, the Nix programs
  nix-env, nix-store, etc. are installed with the setuid bit turned on
  so that they are executed as the user and group specified by
  `--with-nix-user=USER' and `--with-nix-group=GROUP', respectively
  (with defaults `nix' and `nix').

  The setuid programs drop all special privileges if they are executed
  by a user who is not a member of the Nix group.

  The setuid feature is a quick hack to enable sharing of a Nix
  installation between users who trust each other.  It is not
  generally secure, since any user in the Nix group can modify (by
  building an appropriate derivation) any object in the store, and for
  instance inject trojans into binaries used by other users.

  The setuid programs are owned by root, not the Nix user.  This is
  because on Unix normal users cannot change the real uid, only the
  effective uid.  Many programs don't work properly when the real uid
  differs from the effective uid.  For instance, Perl will turn on
  taint mode.  However, the setuid programs drop all root privileges
  immediately, changing all uids and gids to the Nix user and group.
2004-08-20 14:49:05 +00:00
Eelco Dolstra 8f1dcdfc0a * Make sure that no build hook is set by default in the tests.
* Don't use `seq' - some primitive, obsolete operating systems
  (Darwin) don't have it.
2004-08-19 09:09:09 +00:00
Eelco Dolstra 1eddee59f2 * The default verbosity level of all Nix commands is now lvlInfo.
* Builder output is written to standard error by default.
  * The option `-B' is gone.
  * The option `-Q' suppresses builder output.

The result of this is that most Nix invocations shouldn't need any
flags w.r.t. logging.
2004-08-18 12:19:06 +00:00
Eelco Dolstra 937ce0cd21 * Flag `--no-link' suppresses symlinking to the output path.
* Handle multiple derivations correctly.
2004-08-18 12:11:31 +00:00
Eelco Dolstra 966bd9d19f * WTF? More canonical system name problems ("athlon-linux" instead of
"i686-linux").
2004-08-13 09:57:51 +00:00
Eelco Dolstra 62fe5c4a22 * The predecessor of a successor need not be present. This in
particular happens on distributed builds or when using push/pull.
2004-08-11 19:03:13 +00:00
Eelco Dolstra ae1a1efa41 * Clean up the temporary directory for hook communication (and don't
print out incorrect "build failed" messages).
2004-08-05 14:53:27 +00:00
Eelco Dolstra d8989b1fb4 * Every real language has a `map' function. 2004-08-04 11:27:53 +00:00
Eelco Dolstra bbfdd64741 * Allow primops with more that 1 arguments. 2004-08-04 10:59:20 +00:00
Eelco Dolstra e3a50f7e25 * Creating a file nix-support/no-scan in the output path of a
derivation disables scanning for dependencies.  Use at your own
  risk.  This is a quick hack to speed up UML image generation (image
  are very big, say 1 GB).

  It would be better if the scanner were faster, and didn't read the
  whole file into memory.
2004-08-04 09:25:21 +00:00
Eelco Dolstra 18ebd7b030 * Doh! 2004-07-30 14:18:48 +00:00
Eelco Dolstra 5373aed1a8 * Use ATerm 2.2.
* Include bootstrap.sh in dist.
2004-07-30 14:17:05 +00:00
Eelco Dolstra 16c8b4c8e5 * A script to generate the Auto* stuff. 2004-07-30 13:45:13 +00:00
Eelco Dolstra e8a95108c0 * Nix-build places a symlink `result' in the current directory to the
store object just built.
2004-07-28 13:32:45 +00:00
Eelco Dolstra 9bf7a5f516 * Don't pass `--with-system'. 2004-07-18 21:08:24 +00:00
Eelco Dolstra 39eaecbc98 * Slightly better heuristic for picking the canonical system type.
Now SuSE and Red Hat should yield the same type (`i686-linux').  Mac
  OS X should now give `powerpc-darwin' (i.e., the version number is
  gone).
2004-07-18 21:07:27 +00:00
Eelco Dolstra 064a36cb54 * Hardcode the system id to be `i686-linux'. 2004-07-09 13:06:12 +00:00
Eelco Dolstra c1a18f543e * Fixed format string error. 2004-07-06 11:21:34 +00:00
Eelco Dolstra 056cd1d3b7 * Don't go into a (sometimes infinite) loop calling the build hook. 2004-07-01 16:24:35 +00:00
Eelco Dolstra 638ce339a5 * Nix-instantiate now accepts sets of derivations (just like nix-env). 2004-07-01 14:25:26 +00:00
Eelco Dolstra 292d6468ec * Nix-env operations now by default filter out any derivations for
system types other than the current system.  I.e., `nix-env -i'
  won't install derivations for other system types, and `nix-env -q'
  won't show them.  The flag `--system-filter SYSTEM' can be used to
  override the system type used for filtering (but not for
  building!).  The value `*' can be used not to filter anything.
2004-07-01 13:56:56 +00:00
Eelco Dolstra 8f6254e823 * Align the columns in the output of `nix-env -q'. 2004-07-01 13:35:10 +00:00
Eelco Dolstra 593bc23d8b * Allow the system attribute of derivations to be queried in
`nix-env -q'.
* Queries can now be combined, e.g., `nix-env -q --status --system'.
2004-07-01 13:13:37 +00:00
Eelco Dolstra b584253af4 * Include some missing headers. 2004-07-01 11:11:16 +00:00
Eelco Dolstra f5d5ffe536 * Write build logs to disk again. 2004-06-29 09:41:50 +00:00
Eelco Dolstra 00aadf478b * Use ATerm 2.1. 2004-06-28 14:51:42 +00:00
Eelco Dolstra 151e61fa5a * By default, `nix-env -i' now deletes previously installed
derivations with names matching the derivations being installed.
  The option `--preserve-installed / -P' overrides this behaviour.
2004-06-28 14:40:26 +00:00
Eelco Dolstra 4d2946c516 * In a realisation goal, check the result of the corresponding
normalisation goal.
2004-06-28 13:51:24 +00:00
Eelco Dolstra 24286e15c9 * `nix-env -u' now allows a specific version to be specified when
upgrading.

  This fixes a bug reported by Martin:

    $ nix-env -i foo-1.0
    $ nix-env -u foo-1.0
    upgrading foo-1.0 to foo-1.1
2004-06-28 13:37:05 +00:00
Eelco Dolstra 2746a879e2 * Typo. 2004-06-28 12:07:07 +00:00
Eelco Dolstra 91dc023665 * Added a switch `--fallback'. From the manual:
Whenever Nix attempts to realise a derivation for which a closure is
  already known, but this closure cannot be realised, fall back on
  normalising the derivation.

  The most common scenario in which this is useful is when we have
  registered substitutes in order to perform binary distribution from,
  say, a network repository.  If the repository is down, the
  realisation of the derivation will fail.  When this option is
  specified, Nix will build the derivation instead.  Thus, binary
  installation falls back on a source installation.  This option is
  not the default since it is generally not desirable for a transient
  failure in obtaining the substitutes to lead to a full build from
  source (with the related consumption of resources).
2004-06-28 10:42:57 +00:00
Eelco Dolstra b113edeab7 * A flag `--keep-going / -k' to keep building goals if one fails, as
much as possible.  (This is similar to GNU Make's `-k' flag.)

* Refactoring to implement this: previously we just bombed out when
  a build failed, but now we have to clean up.  In particular this
  means that goals must be freed quickly --- they shouldn't hang
  around until the worker exits.  So the worker now maintains weak
  pointers in order not to prevent garbage collection.

* Documented the `-k' and `-j' flags.
2004-06-25 15:36:09 +00:00
Eelco Dolstra e4883211f9 * Don't throw an exception when a build fails. Just terminate the
goal and allow the problem to be handled elsewhere (e.g., at
  top-level).
2004-06-25 10:21:44 +00:00
Eelco Dolstra 795d9f8b08 * Obsolete. 2004-06-24 14:36:50 +00:00
Eelco Dolstra a29c8ac51c * Add a test to check that when we cannot realise a closure
expression, we should invalidate it and go back to the derivation
  for which it is a successor.
2004-06-24 14:35:01 +00:00
Eelco Dolstra ec32627621 * Multiple and/or failing substitutes now work. 2004-06-24 13:40:38 +00:00
Eelco Dolstra 8052aef486 * A test for multiple and/or failing substitutes. 2004-06-24 12:56:24 +00:00
Eelco Dolstra 66c7f34759 * Arghhhhhh 2004-06-22 17:07:32 +00:00
Eelco Dolstra 05a5362d63 * Some more diagnostics changes. 2004-06-22 17:04:10 +00:00
Eelco Dolstra d051cd40e1 * Nix-instantiate can return multiple store expressions. 2004-06-22 15:17:04 +00:00
Eelco Dolstra 3093af58a7 * A utility script `nix-build' that builds Nix expressions and prints
their output paths (and only that) on standard output.
2004-06-22 15:12:34 +00:00
Eelco Dolstra b302e5f63b * We don't really need this here. 2004-06-22 15:06:43 +00:00
Eelco Dolstra 83ae1723da * Well, it's better than printf. 2004-06-22 15:01:01 +00:00
Eelco Dolstra 72bc9a522f * Started making Nix's diagnostic messages a bit more useful. 2004-06-22 14:48:59 +00:00
Eelco Dolstra 5e2cf44a4d * Put WEXITSTATUS stuff somewhere else. 2004-06-22 11:03:41 +00:00
Eelco Dolstra 84007a0958 * Reduce gratuitous cut & pasting. 2004-06-22 10:21:44 +00:00
Eelco Dolstra c9fbd2dfd5 * Wrapper class around pids. 2004-06-22 09:51:44 +00:00
Eelco Dolstra 155d7c8dfa * Substitutes should occupy a build slot. 2004-06-22 09:00:31 +00:00
Eelco Dolstra c4cb6ea2bc * Refactoring. 2004-06-22 08:50:25 +00:00
Eelco Dolstra 88fb4f6e53 * Missing files added to `make dist'. 2004-06-21 12:20:47 +00:00
Eelco Dolstra 2db9748221 * Remove debug output. 2004-06-21 10:01:17 +00:00
Eelco Dolstra 37ee6cef99 * Adapted nix-pull to use the new substitute mechanism. 2004-06-21 09:51:23 +00:00
Eelco Dolstra 3f3a3ae87b * Acquire a lock on the output path when running a substitute. Also
delete obstructing invalid paths.
2004-06-21 09:35:50 +00:00
Eelco Dolstra 72c857f0eb * Ugh, nasty Heisenbug due to an uninitialiased variable. The bug
only caused a crash if the program was *not* invoked with a high
  verbosity level.
2004-06-21 08:51:55 +00:00
Eelco Dolstra be1a917beb * Remove obstructing invalid store paths add[Text]ToStore(). 2004-06-21 07:46:02 +00:00
Eelco Dolstra daf0a923c7 * Wrap calls to registerSubstitute() in a single transaction to
improve throughput.
* Don't build the `substitute-rev' table for now, since it caused
  Theta(N^2) time and log file consumption when adding N substitutes.
  Maybe we can do without it.
2004-06-21 07:38:17 +00:00
Eelco Dolstra 15c60ca1b6 * Disable calls to fsync() since Berkeley DB's DB_TXN_WRITE_NOSYNC
flag doesn't seem to work as advertised.
2004-06-21 07:36:01 +00:00
Eelco Dolstra 112ee89501 * Re-enable support for substitutes in the normaliser.
* A better substitute mechanism.

  Instead of generating a store expression for each store path for
  which we have a substitute, we can have a single store expression
  that builds a generic program that is invoked to build the desired
  store path, which is passed as an argument.

  This means that operations like `nix-pull' only produce O(1) files
  instead of O(N) files in the store when registering N substitutes.
  (It consumes O(N) database storage, of course, but that's not a
  performance problem).

* Added a test for the substitute mechanism.
  
* `nix-store --substitute' reads the substitutes from standard input,
  instead of from the command line.  This prevents us from running
  into the kernel's limit on command line length.
2004-06-20 19:17:54 +00:00
Eelco Dolstra bafb2357d1 * README for the Emacs mode (written a while ago, but forgot to commit). 2004-06-20 19:08:59 +00:00
Eelco Dolstra 85ae781765 * Refactoring. 2004-06-20 13:37:51 +00:00
Eelco Dolstra 23bb902d1f * Re-enable build hooks. 2004-06-19 21:45:04 +00:00
Eelco Dolstra 41ec982f31 * Big refactoring. Move to a much more explicitly state machine based
approach.  This makes it much easier to add extra complexity in the
  normaliser / realiser (e.g., build hooks, substitutes).
2004-06-18 18:09:32 +00:00
Eelco Dolstra 3454c685ee * This is also useful. 2004-06-18 16:52:31 +00:00
Eelco Dolstra 6ba26f27c3 * Shared (garbage collecting) pointers. Copied from Boost. 2004-06-18 09:20:51 +00:00
Eelco Dolstra 0b70231b9d * Refactoring. 2004-06-15 13:49:42 +00:00
Eelco Dolstra 1bc6afefac * Cleanup. 2004-06-08 13:21:03 +00:00
Eelco Dolstra e8411948ff * A Nix mode for Emacs. 2004-06-04 14:31:57 +00:00
Eelco Dolstra 5e4a2272bf * Drain the output of the build hook to show error messages. Ugly
hack.
2004-05-18 14:52:35 +00:00
Eelco Dolstra 19479899fb * Don't set the rpath here --- it's not portable. 2004-05-18 12:57:26 +00:00
Eelco Dolstra 1d08093b48 * Go back to the old way of generating the system name, and allow it
to be specified in configure (using `--with-system=SYSTEM').
2004-05-18 09:45:46 +00:00
Eelco Dolstra 8e9fd57ef9 * setpgrp() is not POSIX (and on Mac OS X it's different than on
Linux), so use setpgid().
2004-05-18 09:45:18 +00:00
Eelco Dolstra ace8872706 * execl() requires a terminating 0.
* When a fast build wakes up a goal, try to start that goal in the
  same iteration of the startBuild() loop of run().  Otherwise no job
  might be started until the next job terminates.
2004-05-14 12:24:29 +00:00
Eelco Dolstra 4fc00cbec1 * Distributed builds and load balancing now seem to work pretty well.
(Though the `build-remote.pl' script has a gigantic race condition).
2004-05-13 22:52:37 +00:00
Eelco Dolstra 2fa3304933 * Set the executable bit. 2004-05-13 20:28:20 +00:00
Eelco Dolstra 25db622454 * Load balancing. `build-remote.pl' will only execute up to a
per-machine maximum number of parallel jobs on a remote machine.
2004-05-13 19:35:46 +00:00
Eelco Dolstra 3426d19547 * Perform all tests. 2004-05-13 19:16:48 +00:00
Eelco Dolstra a8306cb98f * The build hooks used to implement distributed builds can now be run
in parallel.  Hooks are more efficient: locks on output paths are
  only acquired when the hook says that it is willing to accept a
  build job.  Hooks now work in two phases.  First, they should first
  tell Nix whether they are willing to accept a job.  Nix guarantuees
  that no two hooks will ever be in the first phase at the same time
  (this simplifies the implementation of hooks, since they don't have
  to perform locking (?)).  Second, if they accept a job, they are
  then responsible for building it (on the remote system), and copying
  the result back.  These can be run in parallel with other hooks and
  locally executed jobs.

  The implementation is a bit messy right now, though.  

* The directory `distributed' shows a (hacky) example of a hook that
  distributes build jobs over a set of machines listed in a
  configuration file.
2004-05-13 19:14:49 +00:00
Eelco Dolstra 5087c8f645 * Use `-j0'. 2004-05-12 14:30:57 +00:00
Eelco Dolstra efa5fa1a91 * A switch `-j NUMBER' to set the maximum number of parallel jobs (0 =
no limit).
* Add missing file to distribution.
2004-05-12 14:20:32 +00:00
Eelco Dolstra 1f48aa0be7 * Broken test. 2004-05-12 13:49:10 +00:00
Eelco Dolstra aa5a5084e4 * Pass to the build hook all sorts of information useful for
distributing a build action to another machine.  In particular, the
  paths in the input closures, the output paths, and successor mapping
  for sub-derivations.
2004-05-12 13:32:26 +00:00
Eelco Dolstra 8c0b42f857 * An quick and dirty hack to support distributed builds. 2004-05-12 09:35:51 +00:00
Eelco Dolstra c8d3882cdc * True parallel builds. Nix can now run as many build jobs in
parallel as possible (similar to GNU Make's `-j' switch).  This is
  useful on SMP systems, but it is especially useful for doing builds
  on multiple machines.  The idea is that a large derivation is
  initiated on one master machine, which then distributes
  sub-derivations to any number of slave machines.  This should not
  happen synchronously or in lock-step, so the master must be capable
  of dealing with multiple parallel build jobs.  We now have the
  infrastructure to support this.

  TODO: substitutes are currently broken.
2004-05-11 18:05:44 +00:00
Eelco Dolstra aea436503e * Ignore interrupt signals while handling an exception.
* Ignore EINTR in reads and writes.
2004-05-11 13:48:25 +00:00
Eelco Dolstra a9858c9f26 * A test to verify that Nix executes build jobs in parallel, if
possible.

  This test fails right now because this hasn't been implemented right
  now.  Yes, I'm doing Test-Driven Development! ;-)
2004-05-04 17:04:17 +00:00
Eelco Dolstra f044ccf702 * 1000th revision!
* A test to verify that locking of output paths (caused by concurrent
  invocations of Nix) works correctly.
2004-05-04 13:56:30 +00:00
Eelco Dolstra a7bbe73971 * Another test. 2004-05-04 13:22:33 +00:00
Eelco Dolstra ef093aac8f * Grrr. TESTS are not included in EXTRA_DIST. 2004-05-04 12:45:04 +00:00
Eelco Dolstra 256eeab711 * Allow the location of the store etc. to be specified using
environment variables.
* Started adding some automatic tests.
* Do a `make check' when building RPMs.
2004-05-04 12:15:30 +00:00
Eelco Dolstra fd927c5d25 * Bump the version number. 2004-04-26 13:44:26 +00:00
170 changed files with 9485 additions and 2620 deletions

No files matched your search

+8
View File
@@ -0,0 +1,8 @@
The following people contributed to Nix, in alphabetical order:
Martin Bravenboer
Eelco Dolstra
Niels Janssen
Armijn Hemel
Rob Vermaas
Eelco Visser
+17 -9
View File
@@ -1,5 +1,5 @@
SUBDIRS = externals src scripts corepkgs doc
EXTRA_DIST = substitute.mk nix.spec nix.spec.in
SUBDIRS = externals src scripts corepkgs doc misc tests
EXTRA_DIST = substitute.mk nix.spec nix.spec.in bootstrap.sh svn-revision
include ./substitute.mk
@@ -14,17 +14,25 @@ relname:
install-data-local: init-state
if INIT_STATE
if SETUID_HACK
INIT_FLAGS = -g @NIX_GROUP@ -o @NIX_USER@
GROUP_WRITABLE = -m 775
endif
init-state:
$(INSTALL) -d $(DESTDIR)$(localstatedir)/nix
$(INSTALL) -d $(DESTDIR)$(localstatedir)/nix/db
$(INSTALL) -d $(DESTDIR)$(localstatedir)/log/nix
$(INSTALL) -d $(DESTDIR)$(localstatedir)/nix/profiles
$(INSTALL) -d $(DESTDIR)$(localstatedir)/nix/gcroots
$(INSTALL) -d $(DESTDIR)$(localstatedir)/nix/gcroots/tmp
$(INSTALL) $(INIT_FLAGS) -d $(DESTDIR)$(localstatedir)/nix
$(INSTALL) $(INIT_FLAGS) -d $(DESTDIR)$(localstatedir)/nix/db
$(INSTALL) $(INIT_FLAGS) -d $(DESTDIR)$(localstatedir)/log/nix
$(INSTALL) $(INIT_FLAGS) -d $(DESTDIR)$(localstatedir)/nix/profiles
$(INSTALL) $(INIT_FLAGS) -d $(DESTDIR)$(localstatedir)/nix/gcroots
$(INSTALL) $(INIT_FLAGS) $(GROUP_WRITABLE) -d $(DESTDIR)$(localstatedir)/nix/gcroots/tmp
$(INSTALL) $(INIT_FLAGS) $(GROUP_WRITABLE) -d $(DESTDIR)$(localstatedir)/nix/gcroots/channels
rm -f $(DESTDIR)$(localstatedir)/nix/gcroots/profiles
ln -s $(localstatedir)/nix/profiles $(DESTDIR)$(localstatedir)/nix/gcroots/profiles
$(INSTALL) -d $(DESTDIR)$(prefix)/store
$(INSTALL) $(INIT_FLAGS) -d $(DESTDIR)$(prefix)/store
# $(bindir)/nix-store --init
else
init-state:
endif
svn-revision:
svnversion . > svn-revision
+69
View File
@@ -0,0 +1,69 @@
Version 0.6
Major changes include the following:
* Rewrite of the normalisation engine.
* Multiple builds can now be performed in parallel (option `-j').
* Distributed builds. Nix can now call a shell script to forward
builds to Nix installations on remote machines, which may or may
not be of the same platform type.
* Option `--fallback' allows recovery from broken substitutes.
* Option `--keep-going' causes building of other (unaffected)
derivations to continue if one failed.
* Improvements to the garbage collector (i.e., it should actually work
now).
* Setuid Nix installations allow a Nix store to be shared among
multiple users.
* Substitute registration is much faster now.
* A utility `nix-build' to build a Nix expression and create a symlink
to the result int the current directory; useful for testing Nix
derivations.
* Manual updates.
* `nix-env' changes:
* Derivations for other platforms are filtered out (which can be
overriden using `--system-filter').
* `--install' by default now uninstall previous derivations with the
same name.
* `--upgrade' allows upgrading to a specific version.
* New operation `--delete-generations' to remove profile
generations (necessary for effective garbage collection).
* Nicer output (sorted, columnised).
* More sensible verbosity levels all around (builder output is now
shown always, unless `-Q' is given).
* Nix expression language changes:
* New language construct: `with E1; E2' brings all attributes
defined in the attribute set E1 in scope in E2.
* Added a `map' function.
* Various new operators (e.g., string concatenation).
* Expression evaluation is much faster.
* An Emacs mode for editing Nix expressions (with syntax highlighting
and indentation) has been added.
* Many bug fixes.
Version 0.5 and earlier
Please refer to the Subversion commit log messages.
Executable
+5
View File
@@ -0,0 +1,5 @@
#! /bin/sh -e
aclocal
autoheader
automake --add-missing --copy
autoconf
+57 -7
View File
@@ -1,4 +1,4 @@
AC_INIT(nix, "0.5")
AC_INIT(nix, "0.6")
AC_CONFIG_SRCDIR(README)
AC_CONFIG_AUX_DIR(config)
AM_INIT_AUTOMAKE
@@ -22,9 +22,24 @@ AC_CANONICAL_HOST
# Construct a Nix system name (like "i686-linux").
AC_MSG_CHECKING([for the canonical Nix system name])
machine_name=`uname -m`
sys_name=`uname -s | tr [A-Z] [a-z]`
system="${machine_name}-${sys_name}"
cpu_name=$(uname -p | tr 'A-Z ' 'a-z_')
machine_name=$(uname -m | tr 'A-Z ' 'a-z_')
case $machine_name in
i*86)
machine_name=i686
;;
*)
if test "$cpu_name" != "unknown"; then
machine_name=$cpu_name
fi
;;
esac
sys_name=$(uname -s | tr 'A-Z ' 'a-z_')
AC_ARG_WITH(system, AC_HELP_STRING([--with-system=SYSTEM],
[platform identifier (e.g., `i686-linux')]),
system=$withval, system="${machine_name}-${sys_name}")
AC_MSG_RESULT($system)
AC_SUBST(system)
AC_DEFINE_UNQUOTED(SYSTEM, ["$system"], [platform identifier (`cpu-os')])
@@ -53,7 +68,7 @@ AC_DEFUN([NEED_PROG],
[
AC_PATH_PROG($1, $2)
if test -z "$$1"; then
AC_MSG_ERROR([$1 is required])
AC_MSG_ERROR([$2 is required])
fi
])
@@ -100,7 +115,7 @@ if test -z "$bdb"; then
bdb_lib='-L${top_builddir}/externals/inst-bdb/lib -ldb_cxx'
bdb_include='-I${top_builddir}/externals/inst-bdb/include'
else
bdb_lib="-L$bdb/lib -Wl,-rpath,$bdb/lib -ldb_cxx"
bdb_lib="-L$bdb/lib -ldb_cxx"
bdb_include="-I$bdb/include"
fi
AC_SUBST(bdb_lib)
@@ -113,12 +128,15 @@ AM_CONDITIONAL(HAVE_ATERM, test -n "$aterm")
if test -z "$aterm"; then
aterm_lib='-L${top_builddir}/externals/inst-aterm/lib -lATerm'
aterm_include='-I${top_builddir}/externals/inst-aterm/include'
aterm_bin='${top_builddir}/externals/inst-aterm/bin'
else
aterm_lib="-L$aterm/lib -Wl,-rpath,$aterm/lib -lATerm"
aterm_lib="-L$aterm/lib -lATerm"
aterm_include="-I$aterm/include"
aterm_bin="$aterm/bin"
fi
AC_SUBST(aterm_lib)
AC_SUBST(aterm_include)
AC_SUBST(aterm_bin)
AC_CHECK_LIB(pthread, pthread_mutex_init)
@@ -127,6 +145,34 @@ AC_ARG_ENABLE(init-state, AC_HELP_STRING([--disable-init-state],
init_state=$enableval, init_state=yes)
AM_CONDITIONAL(INIT_STATE, test "$init_state" = "yes")
AC_ARG_ENABLE(setuid, AC_HELP_STRING([--enable-setuid],
[install Nix setuid]),
setuid_hack=$enableval, setuid_hack=no)
AM_CONDITIONAL(SETUID_HACK, test "$setuid_hack" = "yes")
if test "$setuid_hack" = "yes"; then
AC_DEFINE(SETUID_HACK, 1, [whether to install Nix setuid])
fi
AC_CHECK_FUNC(setresuid, [HAVE_SETRESUID=1], [HAVE_SETRESUID=])
AM_CONDITIONAL(HAVE_SETRESUID, test "$HAVE_SETRESUID" = "1")
if test "$HAVE_SETRESUID" = "1"; then
AC_DEFINE(HAVE_SETRESUID, 1, [whether we have setresuid()])
fi
AC_ARG_WITH(nix-user, AC_HELP_STRING([--with-nix-user=USER],
[user for Nix setuid binaries]),
NIX_USER=$withval, NIX_USER=nix)
AC_SUBST(NIX_USER)
AC_DEFINE_UNQUOTED(NIX_USER, ["$NIX_USER"], [Nix user])
AC_ARG_WITH(nix-group, AC_HELP_STRING([--with-nix-group=USER],
[group for Nix setuid binaries]),
NIX_GROUP=$withval, NIX_GROUP=nix)
AC_SUBST(NIX_GROUP)
AC_DEFINE_UNQUOTED(NIX_GROUP, ["$NIX_GROUP"], [Nix group])
AM_CONFIG_HEADER([config.h])
AC_CONFIG_FILES([Makefile
externals/Makefile
@@ -149,7 +195,11 @@ AC_CONFIG_FILES([Makefile
corepkgs/nar/Makefile
corepkgs/buildenv/Makefile
corepkgs/channels/Makefile
corepkgs/nix-pull/Makefile
doc/Makefile
doc/manual/Makefile
misc/Makefile
misc/emacs/Makefile
tests/Makefile
])
AC_OUTPUT
+1 -1
View File
@@ -1 +1 @@
SUBDIRS = fetchurl nar buildenv channels
SUBDIRS = fetchurl nar buildenv channels nix-pull
+3 -2
View File
@@ -24,9 +24,10 @@ sub createLinks {
my $dstFile = "$dstDir/$baseName";
if ($srcFile =~ /\/propagated-build-inputs$/ ||
$srcFile =~ /\/nix-support$/)
$srcFile =~ /\/nix-support$/ ||
$srcFile =~ /\/log$/)
{
# Do noting.
# Do nothing.
}
elsif (-d $srcFile) {
+2
View File
@@ -8,6 +8,8 @@ mkdir $out
dst=$out/$(basename $path).nar.bz2
@bindir@/nix-store --dump "$path" | @bzip2@ > $dst
if test "${PIPESTATUS[0]}" != "0"; then exit 1; fi
md5=$(md5sum -b $dst | cut -c1-32)
if test $? != 0; then exit 1; fi
echo $md5 > $out/md5
+11
View File
@@ -0,0 +1,11 @@
all-local: builder.sh
install-exec-local:
$(INSTALL) -d $(DESTDIR)$(datadir)/nix/corepkgs
$(INSTALL) -d $(DESTDIR)$(datadir)/nix/corepkgs/nix-pull
$(INSTALL_DATA) default.nix $(DESTDIR)$(datadir)/nix/corepkgs/nix-pull
$(INSTALL_PROGRAM) builder.sh $(DESTDIR)$(datadir)/nix/corepkgs/nix-pull
include ../../substitute.mk
EXTRA_DIST = default.nix builder.sh.in
+34
View File
@@ -0,0 +1,34 @@
#! @shell@ -e
export PATH=/bin:/usr/bin
mkdir $out
cat > $out/fetch <<EOF
#! @shell@ -e
export PATH=/bin:/usr/bin
echo "downloading \$2..."
export PRINT_PATH=1
result=(\$(@bindir@/nix-prefetch-url \$2))
hash=\${result[0]}
path=\${result[1]}
if test "\$hash" != "\$3"; then
echo "hash is \$hash, expected \$3"
exit 1
fi
echo "unpacking into \$1..."
if ! @bunzip2@ < "\$path" | @bindir@/nix-store --restore "\$1"; then
exit 1
fi
exit 0
EOF
chmod +x $out/fetch
+7
View File
@@ -0,0 +1,7 @@
{system}:
derivation {
name = "nix-pull";
builder = ./builder.sh;
inherit system;
}
+10 -3
View File
@@ -4,13 +4,18 @@ XMLLINT = $(ENV) $(xmllint) $(xmlflags) --catalogs
XSLTPROC = $(ENV) $(xsltproc) $(xmlflags) --catalogs \
--param section.autolabel 1 \
--param section.label.includes.component.label 1 \
--param html.stylesheet \'style.css\'
--param html.stylesheet \'style.css\' \
--param xref.with.number.and.title 0
man1_MANS = nix-env.1 nix-store.1 nix-instantiate.1 \
nix-collect-garbage.1 nix-push.1 nix-pull.1 \
nix-prefetch-url.1
SOURCES = manual.xml introduction.xml installation.xml overview.xml \
FIGURES = figures/user-environments.png
SOURCES = manual.xml introduction.xml installation.xml \
package-management.xml writing-nix-expressions.xml \
build-farm.xml \
$(man1_MANS:.1=.xml) \
troubleshooting.xml bugs.xml opt-common.xml opt-common-syn.xml \
quick-start.xml nix-lang-ref.xml style.css images
@@ -35,6 +40,8 @@ install-data-local: manual.html
$(INSTALL_DATA) manual.html $(DESTDIR)$(datadir)/nix/manual
$(INSTALL_DATA) style.css $(DESTDIR)$(datadir)/nix/manual
cp -r images $(DESTDIR)$(datadir)/nix/manual/images
$(INSTALL) -d $(DESTDIR)$(datadir)/nix/manual/figures
$(INSTALL_DATA) $(FIGURES) $(DESTDIR)$(datadir)/nix/manual/figures
images:
mkdir images
@@ -45,6 +52,6 @@ images:
KEEP = manual.html manual.is-valid version.xml $(MANS)
EXTRA_DIST = $(SOURCES) $(KEEP)
EXTRA_DIST = $(SOURCES) $(FIGURES) $(KEEP)
DISTCLEANFILES = $(KEEP)
+18 -25
View File
@@ -1,7 +1,6 @@
<appendix>
<title>Bugs / To-Do</title>
<appendix><title>Bugs / To-Do</title>
<itemizedlist>
<itemizedlist>
<listitem>
<para>
@@ -53,17 +52,6 @@
</para>
</listitem>
<listitem>
<para>
The current garbage collector is a hack. It should be
integrated into <command>nix-store</command>. It should
delete derivations in an order determined by topologically
sorting derivations under the points-to relation. This
ensures that no store paths ever exist that point to
non-existant store paths.
</para>
</listitem>
<listitem>
<para>
There are race conditions between the garbage collector and
@@ -99,17 +87,22 @@ $ nix-store -r $(cat /nix/var/nix/roots/bla)</screen>
</para>
</listitem>
<listitem>
<para>
For security, <command>nix-push</command> manifests should be
digitally signed, and <command>nix-pull</command> should
verify the signatures. The actual NAR archives in the cache
do not need to be signed, since the manifest contains
cryptographic hashes of these files (and
<filename>fetchurl.nix</filename> checks them).
</para>
</listitem>
<listitem><para>For security, <command>nix-push</command> manifests
should be digitally signed, and <command>nix-pull</command> should
verify the signatures. The actual NAR archives in the cache do not
need to be signed, since the manifest contains cryptographic hashes of
these files (and <filename>fetchurl.nix</filename> checks
them).</para></listitem>
</itemizedlist>
<listitem><para>We should switch away from MD5, since it has been
more-or-less cracked. We don't currently depend very much on the
collision-resistance of MD5, but we will once we start sharing build
results between users.</para></listitem>
<listitem><para>It would be useful to have an option in
<command>nix-env --delete-generations</command> to remove non-current
generations older than a certain age.</para></listitem>
</itemizedlist>
</appendix>
+129
View File
@@ -0,0 +1,129 @@
<chapter id='chap-build-farm'><title>Setting up a Build Farm</title>
<para>This chapter provides some sketchy information on how to set up
a Nix-based build farm. Nix is particularly suited as a basis for a
build farm, since:
<itemizedlist>
<listitem><para>Nix supports distributed builds: a local Nix
installation can forward Nix builds to other machines over the
network. This allows multiple builds to be performed in parallel
(thus improving performance), but more in importantly, it allows Nix
to perform multi-platform builds in a semi-transparent way. For
instance, if you perform a build for a
<literal>powerpc-darwin</literal> on an
<literal>i686-linux</literal> machine, Nix can automatically forward
the build to a <literal>powerpc-darwin</literal> machine, if
available.</para></listitem>
<listitem><para>The Nix expression language is ideal for describing
build jobs, plus all their dependencies. For instance, if your
package has some dependency, you don't have to manually install it
on all the machines in the build farm; they will be built
automatically.</para></listitem>
<listitem><para>Proper release management requires that builds (if
deployed) are traceable: it should be possible to figure out from
exactly what sources they were built, in what configuration, etc.;
and it should be possible to reproduce the build, if necessary. Nix
makes this possible since Nix's hashing scheme uniquely identifies
builds, and Nix expressions are self-contained.</para></listitem>
<listitem><para>Nix will only rebuild things that have actually
changed. For instance, if the sources of a component haven't
changed between runs of the build farm, the component won't be
rebuild (unless it was garbage-collected). Also, dependencies
typically don't change very often, so they only need to be built
once.</para></listitem>
<listitem><para>The results of a Nix build farm can be made
available through a channel, so successful builds can be deployed to
users immediately.</para></listitem>
</itemizedlist>
</para>
<section><title>Overview</title>
<para>TODO</para>
<para>The sources of the Nix build farm are at <ulink
url='https://svn.cs.uu.nl:12443/repos/trace/release/trunk' />.</para>
</section>
<section id='sec-distributed-builds'><title>Setting up distributed builds</title>
<para>You can enable distributed builds by setting the environment
variable <envar>NIX_BUILD_HOOK</envar> to point to a program that Nix
will call whenever it wants to build a derivation. The build hook
(typically a shell or Perl script) can decline the build, in which Nix
will perform it in the usual way if possible, or it can accept it, in
which case it is responsible for somehow getting the inputs of the
build to another machine, doing the build there, and getting the
results back.</para>
<example id='ex-remote-systems'><title>Remote machine configuration:
<filename>remote-systems.conf</filename></title>
<programlisting>
nix@mcflurry.labs.cs.uu.nl powerpc-darwin /home/nix/.ssh/id_quarterpounder_auto 2
nix@scratchy.labs.cs.uu.nl i686-linux /home/nix/.ssh/id_scratchy_auto 1
</programlisting>
</example>
<para>An example build hook can be found in the Nix build farm
sources: <ulink
url='https://svn.cs.uu.nl:12443/repos/trace/release/trunk/common/distributed/build-remote.pl'
/>. It should be suitable for most purposes, with maybe some minor
adjustments. It uses <command>ssh</command> and
<command>rsync</command> to copy the build inputs and outputs and
perform the remote build. You should define a list of available build
machines and set the environment variable
<envar>REMOTE_SYSTEMS</envar> to point to it. An example
configuration is shown in <xref linkend='ex-remote-systems' />. Each
line in the file specifies a machine, with the following bits of
information:
<orderedlist>
<listitem><para>The name of the remote machine, with optionally the
user under which the remote build should be performed. This is
actually passed as an argument to <command>ssh</command>, so it can
be an alias defined in your
<filename>~/.ssh/config</filename>.</para></listitem>
<listitem><para>The Nix platform type identifier, such as
<literal>powerpc-darwin</literal>.</para></listitem>
<listitem><para>The SSH private key to be used to log in to the
remote machine. Since builds should be non-interactive, this key
should not have a passphrase!</para></listitem>
<listitem><para>The maximum <quote>load</quote> of the remote
machine. This is just the maximum number of jobs that
<filename>build-remote.pl</filename> will execute in parallel on the
machine. Typically this should be equal to the number of
CPUs.</para></listitem>
</orderedlist>
You should also set up the environment variable
<envar>CURRENT_LOAD</envar> to point at a file that
<filename>build-remote.pl</filename> uses to remember how many jobs it
is currently executing remotely. It doesn't look at the actual load
on the remote machine, so if you have multiple instances of Nix
running, they should use the same <envar>CURRENT_LOAD</envar>
file<footnote><para>Although there are probably some race conditions
in the script right now.</para></footnote>. Maybe in the future
<filename>build-remote.pl</filename> will look at the actual remote
load. The load file should exist, so you should just create it as an
empty file initially.</para>
</section>
</chapter>
Binary file not shown.

After

Width:  |  Height:  |  Size: 16 KiB

+176 -175
View File
@@ -1,211 +1,212 @@
<chapter id='chap-installation'>
<title>Installation</title>
<chapter id='chap-installation'><title>Installation</title>
<sect1>
<title>Obtaining Nix</title>
<para>
The easiest way to obtain Nix is to download a <ulink
url='http://www.cs.uu.nl/groups/ST/Trace/Nix'>source
distribution</ulink>. RPMs for SuSE and Red Hat are also
available. These distributions are generated automatically.
</para>
<sect1><title>Obtaining Nix</title>
<para>
Alternatively, the most recent sources of Nix can be obtained from its
<ulink url='https://svn.cs.uu.nl:12443/repos/trace/nix/trunk'>Subversion
repository</ulink>. For example, the following command will check out
the latest revision into a directory called <filename>nix</filename>:
</para>
<para>The easiest way to obtain Nix is to download a <ulink
url='http://www.cs.uu.nl/groups/ST/Trace/Nix'>source
distribution</ulink>. RPMs for Red Hat 9 are also available. These
distributions are generated automatically.</para>
<screen>
<para>Alternatively, the most recent sources of Nix can be obtained
from its <ulink
url='https://svn.cs.uu.nl:12443/repos/trace/nix/trunk'>Subversion
repository</ulink>. For example, the following command will check out
the latest revision into a directory called <filename>nix</filename>:</para>
<screen>
$ svn checkout https://svn.cs.uu.nl:12443/repos/trace/nix/trunk nix</screen>
<para>
Likewise, specific releases can be obtained from the <ulink
url='https://svn.cs.uu.nl:12443/repos/trace/nix/tags'>tags
directory</ulink> of the repository. If you don't have Subversion, you
can also download an automatically generated <ulink
url='https://svn.cs.uu.nl:12443/dist/trace/'>compressed
tar-file</ulink> of the head revision of the trunk.
<para>Likewise, specific releases can be obtained from the <ulink
url='https://svn.cs.uu.nl:12443/repos/trace/nix/tags'>tags
directory</ulink> of the repository. If you don't have Subversion,
you can also download an automatically generated <ulink
url='https://svn.cs.uu.nl:12443/dist/trace/'>compressed
tar-file</ulink> of the head revision of the trunk.</para>
</sect1>
<sect1><title>Prerequisites</title>
<para>The following prerequisites only apply when you build from
source. Binary releases (e.g., RPMs) have no prerequisites.</para>
<para>A fairly recent version of GCC/G++ is required. Version 2.95
and higher should work.</para>
<para>To build this manual and the man-pages you need the
<command>xmllint</command> and <command>xsltproc</command> programs,
which are part of the <literal>libxml2</literal> and
<literal>libxslt</literal> packages, respectively. You also need the
<ulink url='http://docbook.sourceforge.net/projects/xsl/'>DocBook XSL
stylesheets</ulink> and optionally the <ulink
url='http://www.oasis-open.org/docbook/xml/4.2/docbook-xml-4.2.zip'>
DocBook XML 4.2 DTD</ulink>. Note that these are only required if you
modify the manual sources or when you are building from the Subversion
repository.</para>
<para>To build the parser, very <emphasis>recent</emphasis> versions
of Bison and Flex are required. (This is because Nix needs GLR
support in Bison and reentrancy support in Flex.) For Bison, you need
version 1.875c or higher (1.875 does <emphasis>not</emphasis> work),
which can be obtained from the <ulink
url='ftp://alpha.gnu.org/pub/gnu/bison'>GNU FTP server</ulink>. For
Flex, you need version 2.5.31, which is available on <ulink
url='http://lex.sourceforge.net/'>SourceForge</ulink>. Slightly older
versions may also work, but ancient versions like the ubiquitous
2.5.4a won't. Note that these are only required if you modify the
parser or when you are building from the Subversion repository.</para>
<para>Nix uses Sleepycat's Berkeley DB and CWI's ATerm library. These
are included in the Nix source distribution. If you build from the
Subversion repository, you must download them yourself and place them
in the <filename>externals/</filename> directory. See
<filename>externals/Makefile.am</filename> for the precise URLs of
these packages. Alternatively, if you already have them installed,
you can use <command>configure</command>'s <option>--with-bdb</option>
and <option>--with-aterm</option> options to point to their respective
locations. Note that Berkeley DB <emphasis>must</emphasis> be version
4.2; other versions may not have compatible database formats.</para>
</sect1>
<sect1><title>Building Nix from source</title>
<para>After unpacking or checking out the Nix sources, issue the
following commands:
</para>
</sect1>
<sect1>
<title>Prerequisites</title>
<para>
The following prerequisites only apply when you build from
source. Binary releases (e.g., RPMs) have no prerequisites.
</para>
<para>
A fairly recent version of GCC/G++ is required. Version 2.95
and higher should work.
</para>
<para>
To build this manual and the man-pages you need the
<command>xmllint</command> and <command>xsltproc</command>
programs, which are part of the <literal>libxml2</literal> and
<literal>libxslt</literal> packages, respectively. You also
need the <ulink
url='http://docbook.sourceforge.net/projects/xsl/'>DocBook XSL
stylesheets</ulink> and optionally the <ulink
url='http://www.oasis-open.org/docbook/xml/4.2/docbook-xml-4.2.zip'>
DocBook XML 4.2 DTD</ulink>. Note that these are only required
if you modify the manual sources or when you are building from
the Subversion repository.
</para>
<para>
To build the parser, very <emphasis>recent</emphasis> versions
of Bison and Flex are required. (This is because Nix needs GLR
support in Bison and reentrancy support in Flex.) For Bison,
you need version 1.875c or higher (1.875 does
<emphasis>not</emphasis> work), which can be obtained from the
<ulink url='ftp://alpha.gnu.org/pub/gnu/bison'>GNU FTP
server</ulink>. For Flex, you need version 2.5.31, which is
available on <ulink
url='http://lex.sourceforge.net/'>SourceForge</ulink>. Slightly
older versions may also work, but ancient versions like the
ubiquitous 2.5.4a won't. Note that these are only required if
you modify the parser or when you are building from the
Subversion repository.
</para>
<para>
Nix uses Sleepycat's Berkeley DB and CWI's ATerm library. These
are included in the Nix source distribution. If you build from
the Subversion repository, you must download them yourself and
place them in the <filename>externals/</filename> directory.
See <filename>externals/Makefile.am</filename> for the precise
URLs of these packages.
</para>
</sect1>
<sect1>
<title>Building Nix from source</title>
<para>
After unpacking or checking out the Nix sources, issue the following
commands:
</para>
<screen>
<screen>
$ ./configure <replaceable>options...</replaceable>
$ make
$ make install</screen>
<para>
When building from the Subversion repository, these should be preceded by
the command:
<para>When building from the Subversion repository, these should be
preceded by the command:
</para>
<screen>
<screen>
$ autoreconf -i</screen>
<para>
The installation path can be specified by passing the
<option>--prefix=<replaceable>prefix</replaceable></option> to
<command>configure</command>. The default installation directory is
<filename>/nix</filename>. You can change this to any location you like.
You must have write permission to the <replaceable>prefix</replaceable>
path.
</para>
<para>The installation path can be specified by passing the
<option>--prefix=<replaceable>prefix</replaceable></option> to
<command>configure</command>. The default installation directory is
<filename>/nix</filename>. You can change this to any location you
like. You must have write permission to the
<replaceable>prefix</replaceable> path.</para>
<warning>
<para>
It is advisable <emphasis>not</emphasis> to change the installation
prefix from its default, since doing so will in all likelihood make it
impossible to use derivations built on other systems.
</para>
</warning>
<warning><para>It is advisable <emphasis>not</emphasis> to change the
installation prefix from its default, since doing so will in all
likelihood make it impossible to use derivations built on other
systems.</para></warning>
<para>
If you want to rebuilt the documentation, pass the full path to the
DocBook XML catalog file (<filename>docbook.cat</filename>) and to the
DocBook XSL stylesheets using the
<option>--with-docbook-catalog=<replaceable>path</replaceable></option>
and <option>--with-docbook-xsl=<replaceable>path</replaceable></option>
options.
</para>
<para>If you want to rebuilt the documentation, pass the full path to
the DocBook XML catalog file (<filename>docbook.cat</filename>) and to
the DocBook XSL stylesheets using the
<option>--with-docbook-catalog=<replaceable>path</replaceable></option>
and
<option>--with-docbook-xsl=<replaceable>path</replaceable></option>
options.</para>
</sect1>
</sect1>
<sect1>
<title>Installing from RPMs</title>
<sect1><title>Installing from RPMs</title>
<para>
RPM packages of Nix can be downloaded from <ulink
url='http://www.cs.uu.nl/groups/ST/Trace/Nix' />. These RPMs
should work for most fairly recent releases of SuSE and Red Hat
Linux. They have been known to work work on SuSE Linux 8.1 and
9.0, and Red Hat 9.0. In fact, it should work on any RPM-based
Linux distribution based on <literal>glibc</literal> 2.3 or
later.
</para>
<para>RPM packages of Nix can be downloaded from <ulink
url='http://www.cs.uu.nl/groups/ST/Trace/Nix' />. These RPMs should
work for most fairly recent releases of SuSE and Red Hat Linux. They
have been known to work work on SuSE Linux 8.1 and 9.0, and Red Hat
9.0. In fact, it should work on any RPM-based Linux distribution
based on <literal>glibc</literal> 2.3 or later.</para>
<para>
Once downloaded, the RPMs can be installed or upgraded using
<command>rpm -U</command>. For example,
</para>
<para>Once downloaded, the RPMs can be installed or upgraded using
<command>rpm -U</command>. For example,</para>
<screen>
rpm -U nix-0.5pre664-1.i386.rpm</screen>
<screen>
$ rpm -U nix-0.5pre664-1.i386.rpm</screen>
<para>
The RPMs install into the directory <filename>/nix</filename>.
Nix can be uninstalled using <command>rpm -e nix</command>.
After this it will be necessary to manually remove the Nix store
and other auxiliary data:
</para>
<para>The RPMs install into the directory <filename>/nix</filename>.
Nix can be uninstalled using <command>rpm -e nix</command>. After
this it will be necessary to manually remove the Nix store and other
auxiliary data:</para>
<screen>
rm -rf /nix/store
rm -rf /nix/var</screen>
<screen>
$ rm -rf /nix/store
$ rm -rf /nix/var</screen>
</sect1>
</sect1>
<sect1>
<title>Permissions</title>
<para>
All Nix operations must be performed under the user ID that owns
the Nix store and database
(<filename><replaceable>prefix</replaceable>/store</filename>
and
<filename><replaceable>prefix</replaceable>/var/nix/db</filename>,
respectively). When installed from the RPM packages, these
directories are owned by <systemitem
class='username'>root</systemitem>.
</para>
<sect1><title>Permissions</title>
</sect1>
<para>All Nix operations must be performed under the user ID that owns
the Nix store and database
(<filename><replaceable>prefix</replaceable>/store</filename> and
<filename><replaceable>prefix</replaceable>/var/nix/db</filename>,
respectively). When installed from the RPM packages, these
directories are owned by <systemitem
class='username'>root</systemitem>.</para>
<sect1>
<title>Using Nix</title>
<sect2><title>Setuid installation</title>
<para>
To use Nix, some environment variables should be set. In
particular, <envar>PATH</envar> should contain the directories
<filename><replaceable>prefix</replaceable>/bin</filename> and
<filename>~/.nix-profile/bin</filename>. The first directory
contains the Nix tools themselves, while
<filename>~/.nix-profile</filename> is a symbolic link to the
current <emphasis>user environment</emphasis> (an automatically
generated package consisting of symlinks to installed packages).
The simplest way to set the required environment variables is to
include the file
<filename><replaceable>prefix</replaceable>/etc/profile.d/nix.sh</filename>
in your <filename>~/.bashrc</filename> (or similar), like this:
</para>
<para>As a somewhat <emphasis>ad hoc</emphasis> hack, you can also
install the Nix binaries <quote>setuid</quote> so that a Nix store can
be shared among several users. To do this, configure Nix with the
<emphasis>--enable-setuid</emphasis> option. Nix will be installed as
owned by a user and group specified by the
<option>--with-nix-user=<parameter>user</parameter></option> and
<option>--with-nix-group=<parameter>group</parameter></option>
options. E.g.,
<screen>
. <replaceable>prefix</replaceable>/etc/profile.d/nix.sh</screen>
<screen>
$ ./configure --enable-setuid --with-nix-user=my_nix_user --with-nix-group=my_nix_group</screen>
The user and group default to <literal>nix</literal>. You should make
sure that both the user and the group exist. Any <quote>real</quote>
users that you want to allow access should be added to the Nix
group.</para>
<warning><para>A setuid installation should only by used if the users
in the Nix group are mutually trusted, since any user in that group
has the ability to change anything in the Nix store or database. For
instance, they could install a trojan horse in executables used by
other users.</para></warning>
<warning><para>On some platforms, the Nix binaries will be installed
as setuid <literal>root</literal>. They drop root privileges
immediately after startup and switch to the Nix user. The reason for
this is that both the real and effective user must be set to the Nix
user, and POSIX has no system call to do this. This is not the case
on systems that have the <function>setresuid()</function> system call
(such as Linux and FreeBSD), so on those systems the binaries are
simply owned by the Nix user.</para></warning>
</sect2>
</sect1>
<sect1><title>Using Nix</title>
<para>To use Nix, some environment variables should be set. In
particular, <envar>PATH</envar> should contain the directories
<filename><replaceable>prefix</replaceable>/bin</filename> and
<filename>~/.nix-profile/bin</filename>. The first directory contains
the Nix tools themselves, while <filename>~/.nix-profile</filename> is
a symbolic link to the current <emphasis>user environment</emphasis>
(an automatically generated package consisting of symlinks to
installed packages). The simplest way to set the required environment
variables is to include the file
<filename><replaceable>prefix</replaceable>/etc/profile.d/nix.sh</filename>
in your <filename>~/.bashrc</filename> (or similar), like this:</para>
<screen>
source <replaceable>prefix</replaceable>/etc/profile.d/nix.sh</screen>
</sect1>
</sect1>
</chapter>
+128 -78
View File
@@ -1,93 +1,143 @@
<chapter>
<title>Introduction</title>
<chapter><title>Introduction</title>
<epigraph>
<para><quote>The number of Nix installations in the world has grown to 5,
with more expected.</quote></para>
</epigraph>
<!--
<epigraph><para><quote>The number of Nix installations in the world
has grown to 5, with more expected.</quote></para></epigraph>
-->
<para>
Nix is a system for software deployment. It supports the
creation and distribution of software packages, as well as the installation
and subsequent management of these on target machines (i.e., it is also a
package manager).
</para>
<para>Nix is a system for the deployment of software. Software
deployment is concerned with the creation, distribution, and
management of software components (<quote>packages</quote>). Its main
features are:
<para>
Nix solves some large problems that exist in most current deployment and
package management systems. <emphasis>Dependency determination</emphasis>
is a big one: the correct installation of a software component requires
that all dependencies of that component (i.e., other components used by it)
are also installed. Most systems have no way to verify that the specified
dependencies of a component are actually sufficient.
</para>
<itemizedlist>
<para>
Another big problem is the lack of support for concurrent availability of
multiple <emphasis>variants</emphasis> of a component. It must be possible
to have several versions of a component installed at the same time, or
several instances of the same version built with different parameters.
Unfortunately, components are in general not properly isolated from each
other. For instance, upgrading a component that is a dependency for some
other component might break the latter.
</para>
<listitem><para>It helps you make sure that dependency specifications
are complete. In general in a deployment system you have to specify
for each component what its dependencies are, but there are no
guarantees that this specification is complete. If you forget a
dependency, then the component will build and work correctly on
<emphasis>your</emphasis> machine if you have the dependency
installed, but not on the end user's machine if it's not
there.</para></listitem>
<para>
Nix solves these problems by building and storing packages in paths that
are infeasible to predict in advance. For example, the artifacts of a
package <literal>X</literal> might be stored in
<filename>/nix/store/d58a0606ed616820de291d594602665d-X</filename>, rather
than in, say, <filename>/usr/lib</filename>. The path component
<filename>d58a...</filename> is actually a cryptographic hash of all the
inputs (i.e., sources, requisites, and build flags) used in building
<literal>X</literal>, and as such is very fragile: any change to the inputs
will change the hash. Therefore it is not sensible to
<emphasis>hard-code</emphasis> such a path into the build scripts of a
package <literal>Y</literal> that uses <literal>X</literal> (as does happen
with <quote>fixed</quote> paths such as <filename>/usr/lib</filename>).
Rather, the build script of package <literal>Y</literal> is parameterised
with the actual location of <literal>X</literal>, which is supplied by the
Nix system.
</para>
<listitem><para>It is possible to have <emphasis>multiple versions or
variants</emphasis> of a component installed at the same time. In
contrast, in systems such as RPM different versions of the same
package tend to install to the same location in the file system, so
installing one version will remove the other. This is especially
important if you want to use applications that have conflicting
requirements on different versions of a component (e.g., application A
requires version 1.0 of library X, while application B requires a
non-backwards compatible version 1.1).</para></listitem>
<para>
As stated above, the path name of a file system object contain a
cryptographic hash of all inputs involved in building it. A change to any
of the inputs will cause the hash to change--and by extension, the path
name. These inputs include both sources (variation in time) and
configuration options (variation in space). Therefore variants of the same
package don't clash---they can co-exist peacefully within the same file
system.
</para>
<listitem><para>Users can have different <quote>views</quote>
(<quote>profiles</quote> in Nix parlance) on the set of installed
applications in a system. For instance, one user can have version 1.0
of some package visible, while another is using version 1.1, and a
third doesn't use it at all.</para></listitem>
<para>
Other features:
</para>
<listitem><para>It is possible to atomically
<emphasis>upgrade</emphasis> software. I.e., there is no time window
during an upgrade in which part of the old version and part of the new
version are simultaneously visible (which might well cause the
component to fail).</para></listitem>
<para>
<emphasis>Transparent source/binary deployment.</emphasis>
</para>
<listitem><para>Likewise, it is possible to atomically roll back after
an install, upgrade, or uninstall action. That is, in a fast (O(1))
operation the previous configuration of the system can be restored.
This is because upgrade or uninstall actions don't actually remove
components from the system.</para></listitem>
<para>
<emphasis>Unambiguous identification of configuration.</emphasis>
</para>
<listitem><para>Unused components can be
<emphasis>garbage-collected</emphasis> automatically and safely: when
you remove an application from a profile, its dependencies will be
deleted by the garbage collector only if there are no other active
applications using them.</para></listitem>
<para>
<emphasis>Automatic storage management.</emphasis>
</para>
<listitem><para>Nix supports both source-based deployment models
(where you distribute <emphasis>Nix expressions</emphasis> that tell
Nix how to build software from source) and binary-based deployment
models. The latter is more-or-less transparent: installation of
components is always based on Nix expressions, but if the expressions
have been built before and Nix knows that the resulting binaries are
available somewhere, it will use those instead.</para></listitem>
<para>
<emphasis>Atomic upgrades and rollbacks.</emphasis>
</para>
<listitem><para>Nix is flexible in the deployment policies that it
supports. There is a clear separation between the tools that
implement basic Nix <emphasis>mechanisms</emphasis> (e.g., building
Nix expressions), and the tools that implement various deployment
<emphasis>policies</emphasis>. For instance, there is a concept of
<quote>Nix channels</quote> that can be used to keep software
installations up-to-date automatically from a network source. This is
a policy that is implemented by a fairly short Perl script, which can
be adapted easily to achieve similar policies.</para></listitem>
<para>
<emphasis>Support for many simultaneous configurations.</emphasis>
</para>
<listitem><para>Nix component builds aim to be <quote>pure</quote>;
that is, unaffected by anything other than the declared dependencies.
This means that if a component was built succesfully once, it can be
rebuilt again on another machine and the result will be the same. We
cannot <emphasis>guarantee</emphasis> this (e.g., if the build depends
on the time-of-day), but Nix (and the tools in the Nix Packages
collection) takes special care to help achieve this.</para></listitem>
<para>
<emphasis>Portability.</emphasis> Nix is quite portable. Contrary to
build systems like those in, e.g., Vesta and ClearCase, it does not rely on
operating system extensions.
</para>
<listitem><para>Nix expressions (the things that tell Nix how to build
components) are self-contained: they describe not just components but
complete compositions. In other words, Nix expressions also describe
how to build all the dependencies. This is in contrast to component
specification languages like RPM spec files, which might say that a
component X depends on some other component Y, but since it does not
describe <emphasis>exactly</emphasis> what Y is, the result of
building or running X might be different on different machines.
Combined with purity, self-containedness ensures that a component that
<quote>works</quote> on one machine also works on another, when
deployed using Nix.</para></listitem>
<listitem><para>The Nix expression language makes it easy to describe
variability in components (e.g., optional features or
dependencies).</para></listitem>
<listitem><para>Nix is ideal for building build farms that do
continuous builds of software from a version management system, since
it can take care of building all the dependencies as well. Also, Nix
only rebuilds components that have changed, so there are no
unnecessary builds. In addition, Nix can transparently distribute
build jobs over different machines, including different
platforms.</para></listitem>
<listitem><para>Nix can be used not only for software deployment, but
also for <emphasis>service deployment</emphasis>, such as the
deployment of a complete web server with all its configuration files,
static pages, software dependencies, and so on. Nix's advantages for
software deployment also apply here: for instance, the ability
trivially to have multiple configurations at the same time, or the
ability to do rollbacks.</para></listitem>
</itemizedlist>
</para>
<para>This manual tells you how to install and use Nix and how to
write Nix expressions for software not already in the Nix Packages
collection. It also discusses some advanced topics, such as setting
up a Nix-based build farm, and doing service deployment using
Nix.</para>
<warning><para>This manual is a work in progress. It's quite likely
to be incomplete, inconsistent with the current implementation, or
simply wrong.</para></warning>
<note><para>Some background information on Nix can be found in two
papers. The ICSE 2004 paper <ulink
url='http://www.cs.uu.nl/~eelco/pubs/immdsd-icse2004-final.pdf'><citetitle>Imposing
a Memory Management Discipline on Software
Deployment</citetitle></ulink> discusses the hashing mechanism used to
ensure reliable dependency identification and non-interference between
different versions and variants of packages. The LISA 2004 paper
<ulink
url='http://www.cs.uu.nl/~eelco/pubs/nspfssd-lisa2004-final.pdf'><citetitle>Nix:
A Safe and Policy-Free System for Software
Deployment</citetitle></ulink> gives a more general discussion of Nix
from a system-administration perspective.</para></note>
</chapter>
+11 -10
View File
@@ -1,15 +1,14 @@
<?xml version="1.0"?>
<!DOCTYPE book
PUBLIC "-//OASIS//DTD DocBook EBNF Module V1.0//EN"
"http://www.oasis-open.org/docbook/xml/ebnf/1.0/dbebnf.dtd"
[
<!-- <!DOCTYPE book
PUBLIC "-//OASIS//DTD DocBook XML V4.3//EN"
"http://www.docbook.org/xml/4.3/docbook-xml-4.3.zip" -->
"http://www.docbook.org/xml/4.3/docbook-xml-4.3.zip"
[
<!ENTITY introduction SYSTEM "introduction.xml">
<!ENTITY quick-start SYSTEM "quick-start.xml">
<!ENTITY installation SYSTEM "installation.xml">
<!ENTITY overview SYSTEM "overview.xml">
<!ENTITY package-management SYSTEM "package-management.xml">
<!ENTITY writing-nix-expressions SYSTEM "writing-nix-expressions.xml">
<!ENTITY build-farm SYSTEM "build-farm.xml">
<!ENTITY opt-common SYSTEM "opt-common.xml">
<!ENTITY opt-common-syn SYSTEM "opt-common-syn.xml">
<!ENTITY nix-env SYSTEM "nix-env.xml">
@@ -19,14 +18,14 @@
<!ENTITY nix-push SYSTEM "nix-push.xml">
<!ENTITY nix-pull SYSTEM "nix-pull.xml">
<!ENTITY nix-prefetch-url SYSTEM "nix-prefetch-url.xml">
<!ENTITY nix-lang-ref SYSTEM "nix-lang-ref.xml">
<!-- <!ENTITY nix-lang-ref SYSTEM "nix-lang-ref.xml"> -->
<!ENTITY troubleshooting SYSTEM "troubleshooting.xml">
<!ENTITY bugs SYSTEM "bugs.xml">
<!ENTITY version SYSTEM "version.xml">
]>
<book>
<title>Nix: A System for Software Deployment</title>
<title>Nix User's Guide</title>
<subtitle>Draft (Version &version;)</subtitle>
@@ -44,7 +43,9 @@
&introduction;
&quick-start;
&installation;
&overview;
&package-management;
&writing-nix-expressions;
&build-farm;
<appendix>
<title>Command Reference</title>
@@ -78,7 +79,7 @@
</sect1>
</appendix>
&nix-lang-ref;
<!-- &nix-lang-ref; -->
&troubleshooting;
&bugs;
+33 -21
View File
@@ -1,14 +1,17 @@
<refentry>
<refnamediv>
<refname>nix-collect-garbage</refname>
<refpurpose>determine the set of unreachable store paths</refpurpose>
<refpurpose>remove unreachable store paths</refpurpose>
</refnamediv>
<refsynopsisdiv>
<cmdsynopsis>
<command>nix-collect-garbage</command>
<arg><option>--invert</option></arg>
<arg><option>--no-successors</option></arg>
<group choice='opt'>
<arg choice='plain'><option>--print-live</option></arg>
<arg choice='plain'><option>--print-dead</option></arg>
</group>
<arg><option>--min-age</option> <replaceable>age</replaceable></arg>
</cmdsynopsis>
</refsynopsisdiv>
@@ -16,10 +19,22 @@
<title>Description</title>
<para>
The command <command>nix-collect-garbage</command> determines
the paths in the Nix store that are garbage, that is, not
reachable from outside of the store. These paths can be safely
deleted without affecting the integrity of the system.
The command <command>nix-collect-garbage</command> performs a
garbage collection on the Nix store: any paths in the Nix store
that are garbage (not reachable from a set of root store
expressions) are deleted.
</para>
<para>
The roots of the garbage collector are the store expressions
mentioned in the files in the directory
<filename><replaceable>prefix</replaceable>/var/nix/gcroots</filename>.
By default, the roots are all user environments in
<filename><replaceable>prefix</replaceable>/var/nix/profiles</filename>.
You can register other store expressions as roots by writing the
full path of the store expression to an arbitrary file in the
<filename>gcroots</filename> directory (or a subdirectory
thereof).
</para>
</refsection>
@@ -30,27 +45,24 @@
<variablelist>
<varlistentry>
<term><option>--invert</option></term>
<term><option>--print-live</option> / <option>--print-dead</option></term>
<listitem>
<para>
Causes the set of <emphasis>reachable</emphasis> paths to
be printed, rather than the unreachable paths. These are
the paths that may <emphasis>not</emphasis> be deleted.
These options cause the set of live or dead paths to be
printed, respectively, rather than performing an actual
garbage collector. They correspond exactly with the
sub-operations in <command>nix-store
<option>--gc</option></command>.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><option>--no-successors</option></term>
<term><option>--min-age</option> <replaceable>age</replaceable></term>
<listitem>
<para>
Causes <command>nix-collect-garbage</command> not to
follow successor relations. By default, if a derivation
store expression is reachable, its successor (i.e., a
closure store expression) is also considered to be
reachable. This option is always safe, but garbage
collecting successors may cause undesirable rebuilds later
on.
This option corresponds to the <option>--min-age</option>
option in <command>nix-store <option>--gc</option></command>.
</para>
</listitem>
</varlistentry>
@@ -63,10 +75,10 @@
<title>Examples</title>
<para>
To delete all unreachable paths, do the following:
To delete all unreachable paths, just do:
<screen>
$ nix-collect-garbage | xargs nix-store --delete</screen>
$ nix-collect-garbage</screen>
</para>
+118
View File
@@ -22,6 +22,11 @@
</group>
<replaceable>path</replaceable>
</arg>
<arg><option>--preserve-installed</option></arg>
<arg>
<arg choice='plain'><option>--system-filter</option></arg>
<replaceable>system</replaceable>
</arg>
<arg><option>--dry-run</option></arg>
<arg choice='plain'><replaceable>operation</replaceable></arg>
<arg rep='repeat'><replaceable>options</replaceable></arg>
@@ -121,6 +126,38 @@
</listitem>
</varlistentry>
<varlistentry>
<term><option>--preserve-installed</option></term>
<listitem>
<para>
By default, when you install a derivation with the
<option>--install</option> operation, it will replace
previously installed versions with the same derivation
name (regardless of the version number). This option
causes those previously installed versions to be kept in
the new generation of the profile. Note that this will
generally cause conflicts in the creation of the user
environment (since multiple versions of a package
typically contain the same programs).
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><option>--system-filter</option> <replaceable>system</replaceable></term>
<listitem>
<para>
By default, operations such as <option>--query
--available</option> only include derivations matching the
current platform. This option allows you to use
derivations for the specified platform
<replaceable>system</replaceable>. The special value
<literal>*</literal> causes derivations for any platform
to be included.
</para>
</listitem>
</varlistentry>
</variablelist>
</refsection>
@@ -183,6 +220,10 @@
<arg choice='plain'><option>--install</option></arg>
<arg choice='plain'><option>-i</option></arg>
</group>
<group choice='opt'>
<arg choice='plain'><option>--preserve-installed</option></arg>
<arg choice='plain'><option>-P</option></arg>
</group>
<arg choice='plain' rep='repeat'><replaceable>drvnames</replaceable></arg>
</cmdsynopsis>
</refsection>
@@ -197,6 +238,35 @@
in the active Nix expression are added.
</para>
<para>
Currently installed derivations with a name equal to the name
of a derivation being added are removed unless the option
<option>--preserve-installed</option> is specified.
</para>
</refsection>
<refsection>
<title>Flags</title>
<variablelist>
<varlistentry>
<term><option>--preserve-installed</option> / <option>-P</option></term>
<listitem>
<para>
Do not remove derivations with a name matching one of
the derivations being installed. Usually, trying to
have two versions of the same package installed in the
same generation of a profile will lead to an error in
building the generation, due to file name clashes
between the two versions. However, this is not the case
for all packages.
</para>
</listitem>
</varlistentry>
</variablelist>
</refsection>
<refsection>
@@ -204,7 +274,11 @@
<screen>
$ nix-env --install gcc-3.3.2 <lineannotation>(install specific version)</lineannotation>
installing `gcc-3.3.2'
uninstalling `gcc-3.1' <lineannotation>(previously installed version is removed)</lineannotation>
$ nix-env --install gcc <lineannotation>(just pick any version)</lineannotation>
$ nix-env -f ~/foo.nix -i '*' <lineannotation>(install everything in <filename>foo.nix</filename>)</lineannotation></screen>
</refsection>
@@ -312,6 +386,9 @@ $ nix-env -f ~/foo.nix -i '*' <lineannotation>(install everything in <filename>f
$ nix-env --upgrade gcc
upgrading `gcc-3.3.1' to `gcc-3.4'
$ nix-env -u gcc-3.3.2 --always <lineannotation>(switch to a specific version)</lineannotation>
upgrading `gcc-3.4' to `gcc-3.3.2'
$ nix-env --upgrade pan
<lineannotation>(no upgrades available, so nothing happens)</lineannotation>
@@ -679,6 +756,47 @@ $ nix-env --list-generations
<!--######################################################################-->
<refsection>
<title>Operation <option>--delete-generations</option></title>
<refsection>
<title>Synopsis</title>
<cmdsynopsis>
<command>nix-env</command>
<arg choice='req'><option>--delete-generations</option></arg>
<arg choice='plain' rep='repeat'><replaceable>generations</replaceable></arg>
</cmdsynopsis>
</refsection>
<refsection>
<title>Description</title>
<para>
This operation deletes the specified generations of the
current profile. The generations can be a list of generation
numbers, or the special value <literal>old</literal> to delete
all non-current generations. Periodically deleting old
generations is important to make garbage collection effective.
</para>
</refsection>
<refsection>
<title>Examples</title>
<screen>
$ nix-env --delete-generations 3 4 8
$ nix-env -p other_profile --delete-generations old</screen>
</refsection>
</refsection>
<!--######################################################################-->
<refsection>
+25
View File
@@ -8,6 +8,10 @@
<cmdsynopsis>
<command>nix-instantiate</command>
&opt-common-syn;
<group choice='opt'>
<arg choice='plain'><option>--parse-only</option></arg>
<arg choice='plain'><option>--eval-only</option></arg>
</group>
<arg choice='plain' rep='repeat'><replaceable>files</replaceable></arg>
</cmdsynopsis>
</refsynopsisdiv>
@@ -39,6 +43,27 @@
&opt-common;
<varlistentry>
<term><option>--parse-only</option></term>
<listitem>
<para>
Just parse the input files, and print their abstract
syntax trees on standard output in ATerm format.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><option>--eval-only</option></term>
<listitem>
<para>
Just parse and evaluate the input files, and print the
resulting values on standard output. No instantiation of
store expressions takes place.
</para>
</listitem>
</varlistentry>
</variablelist>
</refsection>
+2 -2
View File
@@ -1,4 +1,4 @@
<chapter>
<appendix>
<title>Nix Language Reference</title>
<sect1>
@@ -274,4 +274,4 @@
</sect1>
</chapter>
</appendix>
+70 -12
View File
@@ -135,17 +135,19 @@
<!--######################################################################-->
<refsection>
<title>Operation <option>--delete</option></title>
<title>Operation <option>--gc</option></title>
<refsection>
<title>Synopsis</title>
<cmdsynopsis>
<command>nix-store</command>
<arg choice='plain'><option>--gc</option></arg>
<group choice='req'>
<arg choice='plain'><option>--print-live</option></arg>
<arg choice='plain'><option>--print-dead</option></arg>
<arg choice='plain'><option>--delete</option></arg>
<arg choice='plain'><option>-d</option></arg>
</group>
<arg choice='plain' rep='repeat'><replaceable>paths</replaceable></arg>
<arg><option>--min-age</option> <replaceable>age</replaceable></arg>
</cmdsynopsis>
</refsection>
@@ -153,19 +155,75 @@
<title>Description</title>
<para>
The operation <option>--delete</option> unconditionally deletes the
paths <replaceable>paths</replaceable> from the Nix store. It is an
error to attempt to delete paths outside of the store.
The operation <option>--gc</option> performs a garbage
collection on the Nix store. What it does specifically is
determined by the sub-operation, which is one of the
following:
</para>
<variablelist>
<varlistentry>
<term><option>--print-live</option></term>
<listitem>
<para>
This operation prints on standard output the set of
<quote>live</quote> store paths, which are all the store
paths reachable from a set of <quote>root</quote> store
expressions read from standard input. Live paths should
never be deleted, since that would break consistency —
it would become possible that applications are installed
that reference things that are no longer present in the
store.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><option>--print-dead</option></term>
<listitem>
<para>
This operation prints out on standard output the set of
<quote>dead</quote> store paths, which is just the
opposite of the set of live paths: any path in the store
that is not live (with respect to the roots) is dead.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><option>--delete</option></term>
<listitem>
<para>
This operation performs an actual garbage collection.
All dead paths are removed from the store.
</para>
</listitem>
</varlistentry>
</variablelist>
<para>
The set of root store expressions is read from standard input.
Each line should contain exactly one store path.
</para>
<para>
The option <option>--min-age</option> specifies a minimum time
in hours that an unreachable store path must not have been
used before it is considered dead. The default is 0 (consider
all unreachable store paths dead). Whether a store path has
been used is determined by looking at its access time
(<literal>atime</literal>), so this does not work if the store
is located on a file system that has the
<literal>noatime</literal> option set.
</para>
<warning>
<para>
This operation should almost never be called directly, since no
attempt is made to verify that no references exist to the paths to
be deleted. Therefore, careless deletion can result in an
inconsistent system. Deletion of paths in the store is done by the
garbage collector (which uses <option>--delete</option> to delete
unreferenced paths).
You generally will want to use the command
<command>nix-collect-garbage</command>, which figures out
the roots and then calls this command automatically.
</para>
</warning>
+13 -2
View File
@@ -2,7 +2,18 @@
<arg><option>--version</option></arg>
<arg rep='repeat'><option>--verbose</option></arg>
<arg rep='repeat'><option>-v</option></arg>
<arg><option>--build-output</option></arg>
<arg><option>-B</option></arg>
<arg><option>--no-build-output</option></arg>
<arg><option>-Q</option></arg>
<arg>
<group choice='req'>
<arg choice='plain'><option>--max-jobs</option></arg>
<arg choice='plain'><option>-j</option></arg>
</group>
<replaceable>number</replaceable>
</arg>
<arg><option>--keep-going</option></arg>
<arg><option>-k</option></arg>
<arg><option>--keep-failed</option></arg>
<arg><option>-K</option></arg>
<arg><option>--fallback</option></arg>
<arg><option>--readonly-mode</option></arg>
+69 -7
View File
@@ -50,7 +50,7 @@
<para>
<quote>Informational</quote>: print
<emphasis>useful</emphasis> messages about what Nix is
doing.
doing. This is the default.
</para>
</listitem>
</varlistentry>
@@ -94,20 +94,46 @@
<varlistentry>
<term><option>--build-output</option> / <option>-B</option></term>
<term><option>--no-build-output</option> / <option>-Q</option></term>
<listitem>
<para>
Causes the output written by build actions to standard output
and standard error to be echoed to standard error, regardless of
verbosity level. By default, it is only echoed at a verbosity
level of at least 4 (<quote>Debug</quote>), and is suppressed at
lower levels. Note that it is always written to a log file in
By default, output written by builders to standard output and
standard error is echoed to the Nix command's standard error.
This option suppresses this behaviour. Note that the builder's
standard output and error are always written to a log file in
<filename><replaceable>prefix</replaceable>/nix/var/log/nix</filename>.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><option>--max-jobs</option> / <option>-j</option></term>
<listitem>
<para>
Sets the maximum number of build jobs that Nix will perform in
parallel to the specified number. The default is 1. A higher
value is useful on SMP systems or to exploit I/O latency.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><option>--keep-going</option> / <option>-k</option></term>
<listitem>
<para>
Keep going in case of failed builds, to the greatest extent
possible. That is, if building an input of some derivation
fails, Nix will still build the other inputs, but not the
derivation itself. Without this option, Nix stops if any build
fails (except for builds of substitutes), possibly killing
builds in progress (in case of parallel or distributed builds).
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><option>--keep-failed</option> / <option>-K</option></term>
<listitem>
@@ -120,3 +146,39 @@
</listitem>
</varlistentry>
<varlistentry>
<term><option>--fallback</option></term>
<listitem>
<para>
Whenever Nix attempts to realise a derivation for which a
closure is already known, but this closure cannot be realised,
fall back on normalising the derivation.
</para>
<para>
The most common scenario in which this is useful is when we have
registered substitutes in order to perform binary distribution
from, say, a network repository. If the repository is down, the
realisation of the derivation will fail. When this option is
specified, Nix will build the derivation instead. Thus,
binary installation falls back on a source installation. This
option is not the default since it is generally not desirable
for a transient failure in obtaining the substitutes to lead to
a full build from source (with the related consumption of
resources).
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><option>--readonly-mode</option></term>
<listitem>
<para>
When this option is used, no attempt is made to open the Nix
database. Most Nix operations do need database access, so those
operations will fail.
</para>
</listitem>
</varlistentry>
-450
View File
@@ -1,450 +0,0 @@
<chapter id='chap-overview'>
<title>Overview</title>
<para>
This chapter provides a guided tour of Nix.
</para>
<!--######################################################################-->
<sect1>
<title>Basic package management</title>
<para>
Let's start from the perspective of an end user. Common operations at
this level are to install and remove packages, ask what packages are
installed or available for installation, and so on. These are operations
on the <emphasis>user environment</emphasis>: the set of packages that a
user <quote>sees</quote>. In a command line Unix environment, this means
the set of programs that are available through the <envar>PATH</envar>
environment variable. (In other environments it might mean the set of
programs available on the desktop, through the start menu, and so on.)
</para>
<para>
The terms <quote>installation</quote> and <quote>uninstallation</quote>
are used in this context to denote the act of adding or removing packages
from the user environment. In Nix, these operations are dissociated from
the physical copying or deleting of files. Installation requires that
the files constituting the package are present, but they may be present
beforehand. Likewise, uninstallation does not actually delete any files;
this is done automatically by running a garbage collector.
</para>
<para>
User environments are manipulated through the <command>nix-env</command>
command. The query operation can be used to see what packages are
currently installed.
</para>
<screen>
$ nix-env -q
MozillaFirebird-0.7
sylpheed-0.9.7
pan-0.14.2</screen>
<para>
(<option>-q</option> is actually short for <option>--query
--installed</option>.) The package names are symbolic: they don't have
any particular significance to Nix (as they shouldn't, since they are not
unique&mdash;there can be many derivations with the same name). Note that
these packages have many dependencies (e.g., Mozilla uses the
<literal>gtk+</literal> package) but these have not been installed in the
user environment, though they are present on the system. Generally,
there is no need to install such packages; only packages containing
programs should be installed.
</para>
<para>
To install packages, a <emphasis>Nix expression</emphasis> is required
that tells Nix how to build that package. There is a <ulink
url='https://svn.cs.uu.nl:12443/dist/trace/trace-nixpkgs-trunk.tar.bz2'>set
of standard of Nix expressions</ulink> for many common packages.
Assuming that you have downloaded and unpacked these, you can view the
set of available packages:
</para>
<screen>
$ nix-env -qaf pkgs/system/i686-linux.nix
gettext-0.12.1
sylpheed-0.9.7
aterm-2.0
gtk+-1.2.10
apache-httpd-2.0.48
pan-0.14.2
...</screen>
<para>
The Nix expression in the file <filename>i686-linux.nix</filename> yields
the set of packages for a Linux system running on x86 hardware. For
other platforms, copy and modify this file for your platform as
appropriate. [TODO: improve this]
</para>
<para>
It is also possible to see the <emphasis>status</emphasis> of available
packages, i.e., whether they are installed into the user environment
and/or present in the system:
</para>
<screen>
$ nix-env -qasf pkgs/system/i686-linux.nix
-P gettext-0.12.1
IP sylpheed-0.9.7
-- aterm-2.0
-P gtk+-1.2.10</screen>
<para>
This reveals that the <literal>sylpheed</literal> package is already
installed, or more precisely, that exactly the same instantiation of
<literal>sylpheed</literal> is installed. This guarantees that the
available package is exactly the same as the installed package with
regard to sources, dependencies, build flags, and so on. Similarly, we
see that the <literal>gettext</literal> and <literal>gtk+</literal>
packages are present but not installed in the user environment, while the
<literal>aterm</literal> package is not installed or present at all (so,
if we were to install it, it would have to be built or downloaded first).
</para>
<para>
The install operation is used install available packages from a Nix
environment. To install the <literal>pan</literal> package (a
newsreader), you would do:
</para>
<screen>
$ nix-env -if pkgs/system/i686-linux.nix pan</screen>
<para>
Since installation may take a long time, depending on whether any
packages need to be built or downloaded, it's a good idea to make
<command>nix-env</command> run verbosely by using the <option>-v</option>
(<option>--verbose</option>) option. This option may be repeated to
increase the level of verbosity. A good value is 3
(<option>-vvv</option>).
</para>
<para>
In fact, if you run this command verbosely you will observe that Nix
starts to build many packages, including large and fundamental ones such
as <literal>glibc</literal> and <literal>gcc</literal>. I.e., you are
performing a source installation. This is generally undesirable, since
installation from sources may require large amounts of disk and CPU
resources. Therefore a <quote>binary</quote> installation is generally
preferable.
</para>
<para>
Rather than provide different mechanisms to create and perform
the installation of binary packages, Nix supports binary deployment
<emphasis>transparently</emphasis> through a generic mechanism of
<emphasis>substitute expressions</emphasis>. If an request is made to
build some Nix expression, Nix will first try to build any substitutes
for that expression. These substitutes presumably perform an identical
build operation with respect to the result, but require less resources.
For instance, a substitute that downloads a pre-built package from the
network requires less CPU and disk resources, and possibly less time.
</para>
<para>
Nix's use of cryptographic hashes makes this entirely safe. It is not
possible, for instance, to accidentally substitute a build of some
package for a Solaris or Windows system for a build on a SuSE/x86 system.
</para>
<para>
While the substitute mechanism is a generic mechanism, Nix provides two
standard tools called <command>nix-pull</command> and
<command>nix-push</command> that maintain and use a shared cache of
prebuilt derivations on some network site (reachable through HTTP). If
you attempt to install some package that someone else has previously
built and <quote>pushed</quote> into the cache, and you have done a
<quote>pull</quote> to register substitutes that download these prebuilt
packages, then the installation will automatically use these.
</para>
<para>
For example, to pull from our <ulink
url='http://losser.st-lab.cs.uu.nl/~eelco/nix-dist/'>cache</ulink> of
prebuilt packages (at the time of writing, for SuSE Linux/x86), use the
following command:
</para>
<screen>
$ nix-pull http://catamaran.labs.cs.uu.nl/dist/nix/nixpkgs-<replaceable>version</replaceable>/MANIFEST
obtaining list of Nix archives at http://catamaran.labs.cs.uu.nl/dist/nix/nixpkgs-<replaceable>version</replaceable>/MANIFEST...
...</screen>
<para>
If <command>nix-pull</command> is run without any arguments, it will pull
from the URLs specified in the file
<filename><replaceable>prefix</replaceable>/etc/nix/prebuilts.conf</filename>.
</para>
<para>
Assuming that the <literal>pan</literal> installation produced no errors,
it can be used immediately, that is, it now appears in a directory in the
<envar>PATH</envar> environment variable. Specifically,
<envar>PATH</envar> includes the entry
<filename><replaceable>prefix</replaceable>/var/nix/profiles/default/bin</filename>,
where
<filename><replaceable>prefix</replaceable>/var/nix/profiles/default</filename>
is just a symlink to the current user environment:
</para>
<screen>
$ ls -l /nix/var/nix/profiles/
...
lrwxrwxrwx 1 eelco ... default-15-link -> /nix/store/1871...12b0-user-environment
lrwxrwxrwx 1 eelco ... default-16-link -> /nix/store/59ba...df6b-user-environment
lrwxrwxrwx 1 eelco ... default -> default-16-link</screen>
<para>
That is, <filename>default</filename> in this example is a link
to <filename>default-16-link</filename>, which is the current
user environment. Before the installation, it pointed to
<filename>default-15-link</filename>. Note that this means that
you can atomically roll-back to the previous user environment by
pointing the symlink <filename>default</filename> at
<filename>default-15-link</filename> again. This also shows
that operations such as installation are atomic in the Nix
system: any arbitrarily complex set of installation,
uninstallation, or upgrade actions eventually boil down to the
single operation of pointing a symlink somewhere else (which can
be implemented atomically in Unix).
</para>
<para>
What's in a user environment? It's just a set of symlinks to the files
that constitute the installed packages. For instance:
</para>
<screen>
$ ls -l /nix/var/nix/profiles/default-16-link/bin
lrwxrwxrwx 1 eelco ... MozillaFirebird -> /nix/store/35f8...4ae6-MozillaFirebird-0.7/bin/MozillaFirebird
lrwxrwxrwx 1 eelco ... svn -> /nix/store/3829...fb5d-subversion-0.32.1/bin/svn
...</screen>
<para>
Note that, e.g., <filename>svn</filename> =
<filename>/nix/var/nix/profiles/default/bin/svn</filename> =
<filename>/nix/var/nix/profiles/default-16-link/bin/svn</filename> =
<filename>/nix/store/59ba...df6b-user-environment/bin/svn</filename> =
<filename>/nix/store/3829...fb5d-subversion-0.32.1/bin/svn</filename>.
</para>
<para>
Naturally, packages can also be uninstalled:
</para>
<screen>
$ nix-env -e pan</screen>
<para>
This means that the package is removed from the user
environment. It is <emphasis>not</emphasis> yet removed from
the system. When a package is uninstalled from a user
environment, it may still be used by other packages, or may
still be present in other user environments. Deleting it under
such conditions would break those other packages or user
environments. To prevent this, packages are only
<quote>physically</quote> deleted by running the Nix garbage
collector, which searches for all packages in the Nix store that
are no longer <quote>reachable</quote> from outside the store.
Thus, uninstalling a package is always safe: it cannot break
other packages.
</para>
<para>
Upgrading packages is easy. Given a Nix expression that
contains newer versions of installed packages (that is, packages
with the same package name, but a higher version number),
<command>nix-env -u</command> will replace the installed package
in the user environment with the newer package. For example,
<screen>
$ nix-env -uf pkgs/system/i686-linux.nix pan</screen>
looks for a newer version of Pan, and installs it if found.
Also useful is the ability to upgrade <emphasis>all</emphasis>
packages:
<screen>
$ nix-env -uf pkgs/system/i686-linux.nix '*'</screen>
The asterisk matches all installed packages<footnote><para>No,
we don't support arbitrary regular
expressions</para></footnote>. Note that <literal>*</literal>
must be quoted to prevent shell globbing.
</para>
</sect1>
<!--######################################################################-->
<sect1>
<title>Writing Nix expressions</title>
<sect2>
<title>A simple Nix expression</title>
<para>
This section shows how to write simple Nix expressions&mdash;the things
that describe how to build a package.
</para>
<example id='ex-hello-nix'>
<title>Nix expression for GNU Hello</title>
<programlisting>
{stdenv, fetchurl, perl}: <co id='ex-hello-nix-co-1' />
derivation { <co id='ex-hello-nix-co-2' />
name = "hello-2.1.1"; <co id='ex-hello-nix-co-3' />
system = stdenv.system; <co id='ex-hello-nix-co-4' />
builder = ./builder.sh; <co id='ex-hello-nix-co-5' />
src = fetchurl { <co id='ex-hello-nix-co-6' />
url = ftp://ftp.nluug.nl/pub/gnu/hello/hello-2.1.1.tar.gz;
md5 = "70c9ccf9fac07f762c24f2df2290784d";
};
stdenv = stdenv; <co id='ex-hello-nix-co-7' />
perl = perl;
}</programlisting>
</example>
<para>
A simple Nix expression is shown in <xref linkend='ex-hello-nix' />. It
describes how to the build the <ulink
url='http://www.gnu.org/directory/GNU/hello.html'>GNU Hello
package</ulink>. This package has several dependencies. First, it
requires a number of other packages, such as a C compiler, standard
Unix shell tools, and Perl. Rather than have this Nix expression refer
to and use specific versions of these packages, it should be generic;
that is, it should be a <emphasis>function</emphasis> that takes the
required packages as inputs and yield a build of the GNU Hello package
as a result. This Nix expression defines a function with three
arguments <xref linkend='ex-hello-nix-co-1' />, namely:
<orderedlist>
<listitem><para><varname>stdenv</varname>, which should be a
<emphasis>standard environment package</emphasis>. The standard
environment is a set of tools and other components that would be
expected in a fairly minimal Unix-like environment: a C compiler
and linker, Unix shell tools, and so on.</para>
</listitem>
<listitem><para><varname>fetchurl</varname>, which should be a
function that given parameters <varname>url</varname> and
<varname>md5</varname>, will fetch a file from the specified
location and check that this file has the given MD5 hash code.
The hash is required because build operations must be
<emphasis>pure</emphasis>: given the same inputs they should
always yield the same output. Since network resources can change
at any time, we must in some way guarantee what the result will
be.</para>
</listitem>
<listitem><para><varname>perl</varname>, which should be a Perl
interpreter.</para>
</listitem>
</orderedlist>
</para>
<para>
The remainder of the file is the body of the function, which happens to
be a <emphasis>derivation</emphasis> <xref
linkend='ex-hello-nix-co-2' />, which is the built-in function
<varname>derivation</varname> applied to a set of attributes that
encode all the necessary information for building the GNU Hello
package.
</para>
<example>
<title>Build script (<filename>builder.sh</filename>) for GNU
Hello</title>
<programlisting>
#! /bin/sh
buildinputs="$perl"
. $stdenv/setup || exit 1
tar xvfz $src || exit 1
cd hello-* || exit 1
./configure --prefix=$out || exit 1
make || exit 1
make install || exit 1</programlisting>
</example>
</sect2>
<sect2>
<title>A more complex Nix expression</title>
<example id='ex-svn-nix'>
<title>Nix expression for Subversion</title>
<programlisting>
{ localServer ? false <co id='ex-svn-nix-co-1' />
, httpServer ? false
, sslSupport ? false
, swigBindings ? false
, stdenv, fetchurl
, openssl ? null, httpd ? null, db4 ? null, expat, swig ? null
}:
assert !isNull expat; <co id='ex-svn-nix-co-2' />
assert localServer -> !isNull db4;
assert httpServer -> !isNull httpd &amp;&amp; httpd.expat == expat; <co id='ex-svn-nix-co-3' />
assert sslSupport -> !isNull openssl &amp;&amp; (httpServer -> httpd.openssl == openssl);
assert swigBindings -> !isNull swig;
derivation {
name = "subversion-0.32.1";
system = stdenv.system;
builder = ./builder.sh;
src = fetchurl {
url = http://svn.collab.net/tarballs/subversion-0.32.1.tar.gz;
md5 = "b06717a8ef50db4b5c4d380af00bd901";
};
localServer = localServer;
httpServer = httpServer;
sslSupport = sslSupport;
swigBindings = swigBindings;
stdenv = stdenv;
openssl = if sslSupport then openssl else null; <co id='ex-svn-nix-co-4' />
httpd = if httpServer then httpd else null;
expat = expat;
db4 = if localServer then db4 else null;
swig = if swigBindings then swig else null;
}</programlisting>
</example>
<para>
This example shows several features. Default parameters <xref
linkend='ex-svn-nix-co-1'/> can be used to simplify call sites: if an
argument that has a default is omitted, its default value is used.
</para>
<para>
You can use <emphasis>assertions</emphasis> to test whether arguments
satisfy certain constraints. The simple assertion <xref
linkend='ex-svn-nix-co-2'/> tests whether the
<varname>expat</varname> argument is not a null value. The more
complex assertion <xref linkend='ex-svn-nix-co-3'/> says that if
Subversion is built with Apache support, then <varname>httpd</varname>
(the Apache package) must not be null and it must have been built using
the same instance of the <varname>expat</varname> library as was passed
to the Subversion expression. This is since the Subversion code is
dynamically linked against the Apache code and they both use Expat,
they must be linked against the same instance&mdash;otherwise a
conflict might occur.
</para>
</sect2>
</sect1>
</chapter>
+455
View File
@@ -0,0 +1,455 @@
<chapter id='chap-package-management'><title>Package Management</title>
<para>This chapter discusses how to do package management with Nix,
i.e., how to obtain, install, upgrade, and erase components. This is
the <quote>user's</quote> perspective of the Nix system — people
who want to <emphasis>create</emphasis> components should consult
<xref linkend='chap-writing-nix-expressions' />.</para>
<sect1><title>Basic package management</title>
<para>The main command for package management is
<command>nix-env</command>. You can use it to install, upgrade, and
erase components, and to query what components are installed or are
available for installation.</para>
<para>In Nix, different users can have different <quote>views</quote>
on the set of installed applications. That is, there might be lots of
applications present on the system (possibly in many different
versions), but users can have a specific selection of those active —
where <quote>active</quote> just means that it appears in a directory
in the user's <envar>PATH</envar>. Such a view on the set of
installed applications is called a <emphasis>user
environment</emphasis>, which is just a directory tree consisting of
symlinks to the files of the active applications. </para>
<para>Components are installed from a set of <emphasis>Nix
expressions</emphasis> that tell Nix how to build those components,
including, if necessary, their dependencies. There is a collection of
Nix expressions called the Nix Package collection that contains
components ranging from basic development stuff such as GCC and Glibc,
to end-user applications like Mozilla Firefox. (Nix is however not
tied to the Nix Package collection; you could write your own Nix
expression based on it, or completely new ones.) You can download the
latest version from <ulink
url='http://catamaran.labs.cs.uu.nl/dist/nix' />. You probably want
the latest unstable release; currently the stable releases tend to lag
behind quite a bit.</para>
<para>Assuming that you have downloaded and unpacked a release of Nix
Packages, you can view the set of available components in the release:
<screen>
$ nix-env -qaf nixpkgs-<replaceable>version</replaceable>
ant-blackdown-1.4.2
aterm-2.2
bash-3.0
binutils-2.15
bison-1.875d
blackdown-1.4.2
bzip2-1.0.2
...</screen>
where <literal>nixpkgs-<replaceable>version</replaceable></literal> is
where you've unpacked the release.</para>
<para>It is also possible to see the <emphasis>status</emphasis> of
available components, i.e., whether they are installed into the user
environment and/or present in the system:
<screen>
$ nix-env -qasf nixpkgs-<replaceable>version</replaceable>
...
-PS bash-3.0
--S binutils-2.15
IPS bison-1.875d
...</screen>
The first character (<literal>I</literal>) indicates whether the
component is installed in your current user environment. The second
(<literal>P</literal>) indicates whether it is present on your system
(in which case installing it into your user environment would be a
very quick operation). The last one (<literal>S</literal>) indicates
whether there is a so-called <emphasis>substitute</emphasis> for the
component, which is Nix's mechanism for doing binary deployment. It
just means that Nix know that it can fetch a pre-built component from
somewhere (typically a network server) instead of building it
locally.</para>
<para>So now that we have a set of Nix expressions we can build the
components contained in them. This is done using <literal>nix-env
-i</literal>. For instance,
<screen>
$ nix-env -f nixpkgs-<replaceable>version</replaceable> -i subversion</screen>
will install the component called <literal>subversion</literal> (which
is, of course, the <ulink
url='http://subversion.tigris.org/'>Subversion version management
system</ulink>).</para>
<para>When you do this for the first time, Nix will start building
Subversion and all its dependencies. This will take quite a while —
typically an hour or two on modern machines. Fortunately, there is a
faster way (so do a Ctrl-C on that install operation!): you just need
to tell Nix that pre-built binaries of all those components are
available somewhere. This is done using the
<command>nix-pull</command> command, which must be supplied with a URL
containing a <emphasis>manifest</emphasis> describing what binaries
are available. This URL should correspond to the Nix Packages release
that you're using. For instance, if you obtained a release from
<ulink
url='http://catamaran.labs.cs.uu.nl/dist/nix/nixpkgs-0.6pre1554/' />,
then you should do:
<screen>
$ nix-pull http://catamaran.labs.cs.uu.nl/dist/nix/nixpkgs-0.6pre1554/MANIFEST</screen>
If you then issue the installation command, it should start
downloading binaries from <systemitem
class='fqdomainname'>catamaran.labs.cs.uu.nl</systemitem>, instead of
building them from source. This might still take a while since all
dependencies must be downloaded, but on a reasonably fast connection
such as an DSL line it's on the order of a few minutes.</para>
<para>Naturally, packages can also be uninstalled:
<screen>
$ nix-env -e subversion</screen>
</para>
<para>Upgrading to a new version is just as easy. If you have a new
release of Nix Packages, you can do:
<screen>
$ nix-env -f nixpkgs-<replaceable>version</replaceable> -u subversion</screen>
This will <emphasis>only</emphasis> upgrade Subversion if there is a
<quote>newer</quote> version in the new set of Nix expressions, as
defined by some pretty arbitrary rules regarding ordering of version
numbers (which generally do what you'd expect of them). To just
unconditionally replace Subversion with whatever version is in the Nix
expressions, use <parameter>-i</parameter> instead of
<parameter>-u</parameter>; <parameter>-i</parameter> will remove
whatever version is already installed.</para>
<para>You can also upgrade all components for which there are newer
versions:
<screen>
$ nix-env -f nixpkgs-<replaceable>version</replaceable> -u '*'</screen>
</para>
<para>Sometimes it's useful to be able to ask what
<command>nix-env</command> would do, without actually doing it. For
instance, to find out what packages would be upgraded by
<literal>nix-env -u '*'</literal>, you can do
<screen>
$ nix-env ... -u '*' --dry-run
(dry run; not doing anything)
upgrading `libxslt-1.1.0' to `libxslt-1.1.10'
upgrading `graphviz-1.10' to `graphviz-1.12'
upgrading `coreutils-5.0' to `coreutils-5.2.1'</screen>
</para>
<para>If you grow bored of specifying the Nix expressions using
<parameter>-f</parameter> all the time, you can set a default
location:
<screen>
$ nix-env -I nixpkgs-<replaceable>version</replaceable></screen>
After this you can just say, for instance, <literal>nix-env -u
'*'</literal>.<footnote><para>Setting a default using
<parameter>-I</parameter> currently clashes with using Nix channels,
since <literal>nix-channel --update</literal> calls <literal>nix-env
-I</literal> to set the default to the Nix expressions it downloaded
from the channel, replacing whatever default you had
set.</para></footnote></para>
</sect1>
<sect1><title>Profiles</title>
<para>Profiles and user environments are Nix's mechanism for
implementing the ability to allow differens users to have different
configurations, and to do atomic upgrades and rollbacks. To
understand how they work, it's useful to know a bit about how Nix
works. In Nix, components are stored in unique locations in the
<emphasis>Nix store</emphasis> (typically,
<filename>/nix/store</filename>). For instance, a particular version
of the Subversion component might be stored in a directory
<filename>/nix/store/eeeeaf42e56b...-subversion-0.32.1/</filename>,
while another version might be stored in
<filename>/nix/store/58823d558a6a...-subversion-0.34/</filename>. The
long hexadecimal numbers prefixed to the directory names are
cryptographic hashes<footnote><para>128 bit MD5 hashes, to be
precise.</para></footnote> of <emphasis>all</emphasis> inputs involved
in building the component — sources, dependencies, compiler flags, and
so on. So if two components differ in any way, they end up in
different locations in the file system, so they don't interfere with
each other. <xref linkend='fig-user-environments'
/><footnote><para>TODO: the figure isn't entirely up to date. It
should show multiple profiles and
<filename>~/.nix-profile</filename>.</para></footnote> shows a part of
a typical Nix store.</para>
<figure id='fig-user-environments'><title>User environments</title>
<mediaobject>
<imageobject>
<imagedata fileref='figures/user-environments.png' format='PNG' />
</imageobject>
</mediaobject>
</figure>
<para>Of course, you wouldn't want to type
<screen>
$ /nix/store/eeeeaf42e56b...-subversion-0.32.1/bin/svn</screen>
every time you want to run Subversion. Of course we could set up the
<envar>PATH</envar> environment variable to include the
<filename>bin</filename> directory of every component we want to use,
but this is not very convenient since changing <envar>PATH</envar>
doesn't take effect for already existing processes. The solution Nix
uses is to create directory trees of symlinks to
<emphasis>activated</emphasis> components. These are called
<emphasis>user environments</emphasis> and they are components
themselves (though automatically generated by
<command>nix-env</command>), so they too reside in the Nix store. For
instance, in <xref linkend='fig-user-environments' /> the user
environment <filename>/nix/store/068150f63831...-user-env</filename>
contains a symlink to just Subversion 0.32.1 (arrows in the figure
indicate symlinks). This would be what we would obtain if we had done
<screen>
$ nix-env -i subversion</screen>
on a set of Nix expressions that contained Subversion 0.32.1.</para>
<para>This doesn't in itself solve the problem, of course; you
wouldn't want to type
<filename>/nix/store/068150f63831...-user-env/bin/svn</filename>
either. Therefore there are symlinks outside of the store that point
to the user environments in the store; for instance, the symlinks
<filename>42</filename> and <filename>43</filename> in the example.
These are called <emphasis>generations</emphasis> since every time you
perform a <command>nix-env</command> operation, a new user environment
is generated based on the current one. For instance, generation 43
was created from generation 42 when we did
<screen>
$ nix-env -i subversion mozilla</screen>
on a set of Nix expressions that contained Mozilla and a new version
of Subversion.</para>
<para>Generations are grouped together into
<emphasis>profiles</emphasis> so that different users don't interfere
with each other if they don't want to. For example:
<screen>
$ ls -l /nix/var/nix/profiles/
...
lrwxrwxrwx 1 eelco ... default-42-link -> /nix/store/068150f63831...-user-env
lrwxrwxrwx 1 eelco ... default-43-link -> /nix/store/84c85f89ddbf...-user-env
lrwxrwxrwx 1 eelco ... default -> default-43-link</screen>
This shows a profile called <filename>default</filename>. The file
<filename>default</filename> itself is actually a symlink that points
to the current generation. When we do a <command>nix-env</command>
operation, a new user environment and generation link are created
based on the current one, and finally the <filename>default</filename>
symlink is made to point at the new generation. This last step is
atomic on Unix, which explains how we can do atomic upgrades. (Note
that the building/installing of new components doesn't interfere in
any way with old components, since they are stored in different
locations in the Nix store.)</para>
<para>If you find that you want to undo a <command>nix-env</command>
operation, you can just do
<screen>
$ nix-env --rollback</screen>
which will just make the current generation link point at the previous
link. E.g., <filename>default</filename> would be made to point at
<filename>default-42-link</filename>. You can also switch to a
specific generation:
<screen>
$ nix-env --switch-generation 43</screen>
which in this example would roll forward to generation 43 again. You
can also see all available generations:
<screen>
$ nix-env --list-generations</screen></para>
<para>Actually, there is another level of indirection not shown in the
figure above. You generally wouldn't have
<filename>/nix/var/nix/profiles/<replaceable>some-profile</replaceable>/bin</filename>
in your <envar>PATH</envar>. Rather, there is a symlink
<filename>~/.nix-profile</filename> that points to your current
profile. This means that you should put
<filename>~/.nix-profile/bin</filename> in your <envar>PATH</envar>
(and indeed, that's what the initialisation script
<filename>/nix/etc/profile.d/nix.sh</filename> does). This makes it
easier to switch to a different profile. You can do that using the
command <command>nix-env --switch-profile</command>:
<screen>
$ nix-env --switch-profile /nix/var/nix/profiles/my-profile
$ nix-env --switch-profile /nix/var/nix/profiles/default</screen>
These commands switch to the <filename>my-profile</filename> and
default profile, respectively. If the profile doesn't exist, it will
be created automatically. You should be careful about storing a
profile in another location than the <filename>profiles</filename>
directory, since otherwise it might not be used as a root of the
garbage collector (see section <xref linkend='sec-garbage-collection'
/>).</para>
<para>All <command>nix-env</command> operations work on the profile
pointed to by <command>~/.nix-profile</command>, but you can override
this using the <option>--profile</option> option (abbreviation
<option>-p</option>):
<screen>
$ nix-env -p /nix/var/nix/profiles/other-profile -i subversion</screen>
This will <emphasis>not</emphasis> change the
<command>~/.nix-profile</command> symlink.</para>
</sect1>
<sect1 id='sec-garbage-collection'><title>Garbage collection</title>
<para><command>nix-env</command> operations such as upgrades
(<option>-u</option>) and uninstall (<option>-e</option>) never
actually delete components from the system. All they do (as shown
above) is to create a new user environment that no longer contains
symlinks to the <quote>deleted</quote> components.</para>
<para>Of course, since disk space is not infinite, unused components
should be removed at some point. You can do this by running the Nix
garbage collector. It will remove from the Nix store any component
not used (directly or indirectly) by any generation of any
profile.</para>
<para>Note however that as long as old generations reference a
component, it will not be deleted. After all, we wouldn't be able to
do a rollback otherwise. So in order for garbage collection to be
effective, you should also delete (some) old generations. Of course,
this should only be done if you are certain that you will not need to
roll back.</para>
<para>To delete all old (non-current) generations of your current
profile:
<screen>
$ nix-env --delete-generations old</screen>
Instead of <literal>old</literal> you can also specify a list of
generations, e.g.,
<screen>
$ nix-env --delete-generations 10 11 14</screen>
</para>
<para>After removing appropriate old generations you can run the
garbage collector as follows:
<screen>
$ nix-collect-garbage</screen>
You can alo first view what files would be deleted:
<screen>
$ nix-collect-garbage --print-dead</screen>
Likewise, the option <option>--print-live</option> will show the paths
that <emphasis>won't</emphasis> be deleted.</para>
<sect2><title>Garbage collector roots</title>
<para>TODO</para>
<para>The garbage collector uses as roots all store expressions
mentioned in all files with extension <filename>.gcroot</filename> in
the directory
<filename><replaceable>prefix</replaceable>/var/nix/gcroots/</filename>,
or in any file or directory symlinked to from that directory. E.g.,
by default,
<filename><replaceable>prefix</replaceable>/var/nix/gcroots/</filename>
contains a symlink to
<filename><replaceable>prefix</replaceable>/var/nix/profiles/</filename>,
so all generations of all profiles are also roots of the collector.</para>
</sect2>
</sect1>
<sect1><title>Channels</title>
<para>If you want to stay up to date with a set of packages, it's not
very convenient to manually download the latest set of Nix expressions
for those packages, use <command>nix-pull</command> to register
pre-built binaries (if available), and upgrade using
<command>nix-env</command>. Fortunately, there's a better way:
<emphasis>Nix channels</emphasis>.</para>
<para>A Nix channel is just a URL that points to a place that contains
a set of Nix expressions and a manifest. Using the command
<command>nix-channel</command> you can automatically stay up to date
with whatever is available at that URL.</para>
<para>You can <quote>subscribe</quote> to a channel using
<command>nix-channel --add</command>, e.g.,
<screen>
$ nix-channel --add http://catamaran.labs.cs.uu.nl/dist/nix/channels/nixpkgs-unstable</screen>
subscribes you to a channel that always contains that latest version
of the Nix Packages collection. (Instead of
<literal>nixpkgs-unstable</literal> you could also subscribe to
<literal>nixpkgs-stable</literal>, which should have a higher level of
stability, but right now is just outdated.) Subscribing really just
means that the URL is added to the file
<filename>~/.nix-channels</filename>. Right now there is no command
to <quote>unsubscribe</quote>; you should just edit that file manually
and delete the offending URL.</para>
<para>To obtain the latest Nix expressions available in a channel, do
<screen>
$ nix-channel --update</screen>
This downloads the Nix expressions in every channel (downloaded from
<literal><replaceable>url</replaceable>/nixexprs.tar.bz2</literal>)
and registers any available pre-built binaries in every channel
(by <command>nix-pull</command>ing
<literal><replaceable>url</replaceable>/MANIFEST</literal>). It also
makes the union of each channel's Nix expressions the default for
<command>nix-env</command> operations. Consequently, you can then say
<screen>
$ nix-env -u '*'</screen>
to upgrade all components in your profile to the latest versions
available in the subscribed channels.</para>
</sect1>
</chapter>
+75 -98
View File
@@ -1,141 +1,118 @@
<chapter>
<title>Quick Start</title>
<chapter><title>Quick Start</title>
<para>
This chapter is for impatient people who don't like reading
documentation. For more in-depth information you are kindly
referred to <xref linkend='chap-installation' /> and <xref
linkend='chap-overview' />.
</para>
<para>This chapter is for impatient people who don't like reading
documentation. For more in-depth information you are kindly referred
to the following chapters.</para>
<orderedlist>
<orderedlist>
<listitem>
<para>
Download a source tarball or RPM from <ulink
url='http://www.cs.uu.nl/groups/ST/Trace/Nix'/>.
Build source distributions using the regular sequence:
<listitem><para>Download a source tarball or RPM from <ulink
url='http://www.cs.uu.nl/groups/ST/Trace/Nix'/>. Build source
distributions using the regular sequence:
<screen>
<screen>
$ tar xvfj nix-<replaceable>version</replaceable>.tar.bz2
$ ./configure
$ make
$ make install <lineannotation>(as root)</lineannotation></screen>
This will install Nix in <filename>/nix</filename>. You
should also add <filename>/nix/etc/profile.d/nix.sh</filename>
to your <filename>~/.bashrc</filename> (or some other login
file).
</para>
</listitem>
This will install Nix in <filename>/nix</filename>. You shouldn't
change the prefix if at all possible since that will make it
impossible to use our pre-built components. Alternatively, you could
grab an RPM if you're on an RPM-based system. You should also add
<filename>/nix/etc/profile.d/nix.sh</filename> to your
<filename>~/.bashrc</filename> (or some other login
file).</para></listitem>
<listitem>
<para>
Get some Nix expressions for pre-built packages by downloading
the latest <literal>nixpkgs</literal> distribution (from the
same location), and unpack them.
<listitem><para>Subscribe to the Nix Packages channel.
<screen>
$ wget http://<replaceable>...</replaceable>/nix/nixpkgs-<replaceable>version</replaceable>/nixpkgs-<replaceable>version</replaceable>.tar.bz2
$ tar xfj nixpkgs-<replaceable>version</replaceable>.tar.bz2</screen>
<screen>
$ nix-channel --add http://catamaran.labs.cs.uu.nl/dist/nix/channels/nixpkgs-unstable</screen>
This will unpack the distribution into a directory
<filename>nixpkgs-<replaceable>version</replaceable>/</filename>.
</para>
</listitem>
</para></listitem>
<listitem>
<para>
Pull the Nix cache. This ensures that when you install
packages they are downloaded in pre-built form from the
network, rather than built from source.
<listitem><para>Download the latest Nix expressions available in the channel.
<screen>
$ nix-channel --update</screen>
<screen>
$ nix-pull http://<replaceable>...</replaceable>/nix/nixpkgs-<replaceable>version</replaceable>/MANIFEST</screen>
Note that this in itself doesn't download any components, it just
downloads the Nix expressions that build them and stores them
somewhere (under <filename>~/.nix-defexpr</filename>, in case you're
curious). Also, it registers the fact that pre-built binaries are
available remotely.</para></listitem>
</para>
<listitem><para>See what installable components are currently
available in the channel:
<para>
Note that currently we only pre-build for Linux on x86
platforms.
</para>
</listitem>
<listitem>
<para>
See what's available:
<screen>
$ nix-env -qaf nixpkgs-<replaceable>version</replaceable>/
MozillaFirebird-0.7
hello-2.1.1
<screen>
$ nix-env -qa
docbook-xml-4.2
firefox-1.0pre-PR-0.10.1
hello-2.1.1
libxslt-1.1.0
<replaceable>...</replaceable></screen>
</para>
</listitem>
</para></listitem>
<listitem>
<para>
Install some packages:
<listitem><para>Install some components from the channel:
<screen>
$ nix-env -iBf nixpkgs-<replaceable>version</replaceable>/ hello MozillaFirebird <replaceable>...</replaceable> </screen>
<screen>
$ nix-env -i hello firefox <replaceable>...</replaceable> </screen>
</para>
</listitem>
This should download the pre-built components; it should not build
them locally (if it does, something went wrong).</para></listitem>
<listitem>
<para>
Test that they work:
<listitem><para>Test that they work:
<screen>
<screen>
$ which hello
/home/eelco/.nix-profile/bin/hello
$ hello
Hello, world!
$ MozillaFirebird
$ firefox
<lineannotation>(read Slashdot or something)</lineannotation></screen>
</para>
</listitem>
</para></listitem>
<listitem>
<para>
Uninstall a package:
<listitem><para>Uninstall a package:
<screen>
<screen>
$ nix-env -e hello</screen>
</para>
</listitem>
</para></listitem>
<listitem>
<para>
If a new release of <literal>nixpkgs</literal> comes along,
you can upgrade all installed packages to the latest versions
by downloading and unpacking the new release and doing:
<listitem><para>To keep up-to-date with the channel, do:
<screen>
$ nix-env -uBf nixpkgs-<replaceable>version</replaceable>/ '*'</screen>
<screen>
$ nix-channel --update
$ nix-env -u '*'</screen>
</para>
</listitem>
The latter command will upgrade each installed component for which
there is a <quote>newer</quote> version (as determined by comparing
the version numbers).</para></listitem>
<listitem>
<para>
You should periodically run the Nix garbage collector to get
rid of unused packages, since uninstalls or upgrades don't
actual delete them:
<listitem><para>If you're unhappy with the result of a
<command>nix-env</command> action (e.g., an upgraded component turned
out not to work properly), you can go back:
<screen>
$ nix-collect-garbage | xargs nix-store --delete</screen>
<screen>
$ nix-env --rollback</screen>
</para>
</listitem>
</para></listitem>
</orderedlist>
<listitem><para>You should periodically run the Nix garbage collector
to get rid of unused packages, since uninstalls or upgrades don't
actually delete them:
</chapter>
<screen>
$ nix-env --delete-generations old
$ nix-collect-garbage</screen>
The first command deletes old <quote>generations</quote> of your
profile (making rollbacks impossible, but also making the components
in those old generations available for garbage collection), while the
second command actually deletes them.</para></listitem>
</orderedlist>
</chapter>
+1 -1
View File
@@ -1,4 +1,4 @@
<?xml version="1.0"?>
<locatingRules xmlns="http://thaiopensource.com/ns/locating-rules/1.0">
<uri pathSuffix=".xml" typeId="DocBook"/>
<uri pattern="*.xml" typeId="DocBook"/>
</locatingRules>
+8 -2
View File
@@ -112,10 +112,16 @@ div.note,div.warning
font-style: italic;
}
div.note h3,div.warning h3
div.warning h3
{
color: red;
text-decoration: underline;
font-size: 100%;
}
div.note h3
{
color: blue;
font-size: 100%;
}
div.navfooter *
File diff suppressed because it is too large. Load diff
+4 -5
View File
@@ -34,12 +34,12 @@ endif
# CWI ATerm
ATERM = aterm-2.0.5
ATERM = aterm-2.2
$(ATERM).tar.gz:
@echo "Nix requires the CWI ATerm library to build."
@echo "Please download version 2.0.5 from"
@echo " http://www.cwi.nl/projects/MetaEnv/aterm/aterm-2.0.5.tar.gz"
@echo "Please download version 2.2 from"
@echo " http://www.cwi.nl/projects/MetaEnv/aterm/aterm-2.2.tar.gz"
@echo "and place it in the externals/ directory."
false
@@ -56,8 +56,7 @@ else
build-aterm: have-aterm
(pfx=`pwd` && \
cd $(ATERM) && \
CC="$(CC)" ./configure --prefix=$$pfx/inst-aterm \
--with-cflags="-DNDEBUG -DXGC_VERBOSE -DXHASHPEM -DWITH_STATS $(CFLAGS)" && \
CC="$(CC)" ./configure --prefix=$$pfx/inst-aterm && \
make && \
make install)
touch build-aterm
+1
View File
@@ -0,0 +1 @@
SUBDIRS = emacs
+5
View File
@@ -0,0 +1,5 @@
EXTRA_DIST = nix-mode.el
install-data-local:
$(INSTALL) -d $(DESTDIR)$(datadir)/emacs/site-lisp
$(INSTALL_DATA) nix-mode.el $(DESTDIR)$(datadir)/emacs/site-lisp
+10
View File
@@ -0,0 +1,10 @@
The Nix Emacs mode supports syntax highlighting, somewhat sensible
indenting, and refilling of comments.
To enable Nix mode in Emacs, add something like this to your ~/.emacs
file:
(load "/nix/share/emacs/site-lisp/nix-mode.el")
This automatically causes Nix mode to be activated for all files with
extension `.nix'.
+109
View File
@@ -0,0 +1,109 @@
(defun nix-mode ()
"Major mode for editing Nix expressions.
The following commands may be useful:
'\\[newline-and-indent]'
Insert a newline and move the cursor to align with the previous
non-empty line.
'\\[fill-paragraph]'
Refill a paragraph so that all lines are at most `fill-column'
lines long. This should do the right thing for comments beginning
with `#'. However, this command doesn't work properly yet if the
comment is adjacent to code (i.e., no intervening empty lines).
In that case, select the text to be refilled and use
`\\[fill-region]' instead.
The hook `nix-mode-hook' is run when Nix mode is started.
\\{nix-mode-map}
"
(interactive)
(kill-all-local-variables)
(setq major-mode 'nix-mode)
(setq mode-name "Nix")
(use-local-map nix-mode-map)
(set-syntax-table nix-mode-syntax-table)
;; Font lock support.
(setq font-lock-defaults '(nix-keywords nil nil nil nil))
;; Automatic indentation [C-j].
(make-local-variable 'indent-line-function)
(setq indent-line-function 'nix-indent-line)
;; Indenting of comments.
(make-local-variable 'comment-start)
(setq comment-start "# ")
(make-local-variable 'comment-end)
(setq comment-end "")
(make-local-variable 'comment-start-skip)
(setq comment-start-skip "\\(^\\|\\s-\\);?#+ *")
;; Filling of comments.
(make-local-variable 'adaptive-fill-mode)
(setq adaptive-fill-mode t)
(make-local-variable 'paragraph-start)
(setq paragraph-start "[ \t]*\\(#+[ \t]*\\)?$")
(make-local-variable 'paragraph-separate)
(setq paragraph-separate paragraph-start)
(run-hooks 'nix-mode-hook)
)
(defvar nix-mode-map nil
"Keymap for Nix mode.")
(setq nix-mode-map (make-sparse-keymap))
;(define-key nix-mode-map [tab] 'tab-to-tab-stop)
(defvar nix-keywords
'("\\<if\\>" "\\<then\\>" "\\<else\\>" "\\<assert\\>"
"\\<let\\>" "\\<rec\\>" "\\<inherit\\>"
("\\<true\\>" . font-lock-builtin-face)
("\\<false\\>" . font-lock-builtin-face)
("\\<null\\>" . font-lock-builtin-face)
("\\<import\\>" . font-lock-builtin-face)
("\\<derivation\\>" . font-lock-builtin-face)
("\\<baseNameOf\\>" . font-lock-builtin-face)
("\\<toString\\>" . font-lock-builtin-face)
("\\<isNull\\>" . font-lock-builtin-face)
("\\<\\([a-zA-Z_][a-zA-Z0-9_']*\\)[ \t]*="
(1 font-lock-variable-name-face nil nil))
("[a-zA-Z][a-zA-Z0-9\\+-\\.]*:[a-zA-Z0-9%/\\?:@&=\\+\\$,_\\.!~\\*'-]+"
. font-lock-constant-face)
("[a-zA-Z0-9._\\+-]*\\(/[a-zA-Z0-9._\\+-]+\\)+"
. font-lock-constant-face)
))
(defvar nix-mode-syntax-table nil
"Syntax table for Nix mode.")
(if nix-mode-syntax-table
nil
(progn
(setq nix-mode-syntax-table (make-syntax-table))
(modify-syntax-entry ?/ ". 14" nix-mode-syntax-table)
(modify-syntax-entry ?* ". 23" nix-mode-syntax-table)
(modify-syntax-entry ?# "< b" nix-mode-syntax-table)
(modify-syntax-entry ?\n "> b" nix-mode-syntax-table)
))
(defun nix-indent-line ()
"Indent current line in a Nix expression."
(interactive)
(indent-relative-maybe))
(setq auto-mode-alist (cons '("\\.nix\\'" . nix-mode) auto-mode-alist))
(setq auto-mode-alist (cons '("\\.nix.in\\'" . nix-mode) auto-mode-alist))
+38 -5
View File
@@ -1,26 +1,49 @@
%define enable_setuid ""
%define nix_user "nix"
%define nix_group "nix"
# If set, the Nix user and group will be created by the RPM
# pre-install script.
%define nix_user_uid ""
%define nix_group_gid ""
Summary: The Nix software deployment system
Name: nix
Version: @version@
Release: 1
License: GPL
Group: WeetNiet
URL: http://www.cs.uu.nl/groups/ST/twiki/bin/view/Trace/NixDeploymentSystem
Group: Software Deployment
URL: http://www.cs.uu.nl/groups/ST/Trace/Nix
Source0: %{name}-@version@.tar.gz
BuildRoot: %{_tmppath}/%{name}-%{version}-buildroot
%define _prefix /nix
Prefix: %{_prefix}
Requires: /usr/bin/perl
# Hack to make that shitty RPM scanning hack shut up.
Provides: perl(readmanifest)
%description
Nix is a software deployment system.
Nix is a system for software deployment.
%prep
%setup -q
%build
./configure --prefix=%{_prefix}
extraFlags=
if test -n "%{enable_setuid}"; then
extraFlags="$extraFlags --enable-setuid"
if test -n "%{nix_user}"; then
extraFlags="$extraFlags --with-nix-user=%{nix_user}"
fi
if test -n "%{nix_group}"; then
extraFlags="$extraFlags --with-nix-group=%{nix_group}"
fi
fi
./configure --prefix=%{_prefix} $extraFlags
make
make check
%install
rm -rf $RPM_BUILD_ROOT
@@ -30,8 +53,18 @@ strip $RPM_BUILD_ROOT/%{_prefix}/bin/* || true
%clean
rm -rf $RPM_BUILD_ROOT
%pre
if test -n "%{nix_group_gid}"; then
/usr/sbin/groupadd -g %{nix_group_gid} %{nix_group} || true
fi
if test -n "%{nix_user_uid}"; then
/usr/sbin/useradd -c "Nix" -u %{nix_user_uid} \
-s /sbin/nologin -r -d /var/empty %{nix_user} \
-g %{nix_group} || true
fi
%files
%defattr(-,root,root)
#%defattr(-,root,root)
%{_prefix}/bin
%{_prefix}/libexec
%{_prefix}/var
+7 -6
View File
@@ -1,6 +1,6 @@
bin_SCRIPTS = nix-collect-garbage \
nix-pull nix-push nix-prefetch-url \
nix-install-package nix-channel
nix-pull nix-push nix-prefetch-url \
nix-install-package nix-channel nix-build
noinst_SCRIPTS = nix-profile.sh
@@ -18,7 +18,8 @@ install-exec-local: readmanifest.pm
include ../substitute.mk
EXTRA_DIST = nix-collect-garbage.in \
nix-pull.in nix-push.in nix-profile.sh.in \
nix-prefetch-url.in nix-install-package.in \
nix-channel.in \
prebuilts.conf readmanifest.pm.in
nix-pull.in nix-push.in nix-profile.sh.in \
nix-prefetch-url.in nix-install-package.in \
nix-channel.in \
prebuilts.conf readmanifest.pm.in \
nix-build.in
+41
View File
@@ -0,0 +1,41 @@
#! @shell@ -e
nixExpr=$1
if test -z "$nixExpr"; then
echo "syntax: $0 NIX-EXPR..." >&2
exit 1
fi
extraArgs=
noLink=
for i in "$@"; do
case "$i" in
--no-link)
noLink=1
;;
-*)
extraArgs="$extraArgs $i"
;;
*)
storeExprs=$(nix-instantiate "$i")
for j in $storeExprs; do
echo "store expression is $j" >&2
done
outPaths=$(nix-store -qnfv $extraArgs $storeExprs)
for j in $outPaths; do
echo "$j"
if test -z "$noLink"; then
if test -L result; then
rm result
elif test -e result; then
echo "cannot remove \`result' (not a symlink)"
exit 1
fi
ln -s "$j" result
fi
done
;;
esac
done
+21 -4
View File
@@ -2,6 +2,8 @@
use strict;
my $rootsDir = "@localstatedir@/nix/gcroots/channels";
# Figure out the name of the `.nix-channels' file to use.
my $home = $ENV{"HOME"};
@@ -15,7 +17,7 @@ my @channels;
# Reads the list of channels from the file $channelsList;
sub readChannels {
return if (!-f $channelsList);
open CHANNELS, "<$channelsList" or die "cannot open `$channelsList'";
open CHANNELS, "<$channelsList" or die "cannot open `$channelsList': $!";
while (<CHANNELS>) {
chomp;
push @channels, $_;
@@ -26,7 +28,7 @@ sub readChannels {
# Writes the list of channels to the file $channelsList;
sub writeChannels {
open CHANNELS, ">$channelsList" or die "cannot open `$channelsList'";
open CHANNELS, ">$channelsList" or die "cannot open `$channelsList': $!";
foreach my $url (@channels) {
print CHANNELS "$url\n";
}
@@ -79,18 +81,33 @@ sub update {
"(import @datadir@/nix/corepkgs/channels/unpack.nix) " .
"{inputs = $nixExpr; system = \"@system@\";}";
# Instantiate and realise it.
# Instantiate the Nix expression.
my $storeExpr = `echo '$nixExpr' | @bindir@/nix-instantiate -`
or die "cannot instantiate Nix expression";
chomp $storeExpr;
my $outPath = `nix-store -qnfB '$storeExpr'`
# Register the store expression as a root of the garbage
# collector.
my $userName = getpwuid($<);
die "who ARE you? go away" unless defined $userName;
my $rootFile = "$rootsDir/$userName.gcroot";
my $tmpRootFile = "$rootsDir/$userName-tmp.gcroot";
open ROOT, ">$tmpRootFile" or die "cannot create `$tmpRootFile': $!";
print ROOT "$storeExpr";
close ROOT;
# Realise the store expression.
my $outPath = `nix-store -qnf '$storeExpr'`
or die "cannot realise store expression";
chomp $outPath;
# Make it the default Nix expression for `nix-env'.
system "@bindir@/nix-env --import '$outPath'";
die "cannot pull set default Nix expression to `$outPath'" if ($? != 0);
rename $tmpRootFile, $rootFile or die "cannot rename `$tmpRootFile' to `$rootFile': $!";
}
+22 -32
View File
@@ -8,16 +8,24 @@ my $storeDir = "@storedir@";
my %alive;
my $keepSuccessors = 1;
my $invert = 0;
my $gcOper = "--delete";
my $minAge = 0;
my @roots = ();
# Parse the command line.
foreach my $arg (@ARGV) {
if ($arg eq "--no-successors") { $keepSuccessors = 0; }
elsif ($arg eq "--invert") { $invert = 1; }
for (my $i = 0; $i < scalar @ARGV; $i++) {
my $arg = $ARGV[$i];
if ($arg eq "--delete" || $arg eq "--print-live" || $arg eq "--print-dead") {
$gcOper = $arg;
}
elsif ($arg eq "--min-age") {
$i++;
$minAge = undef;
$minAge = $ARGV[$i];
die "invalid minimum age" unless defined $minAge && $minAge =~ /^\d*$/;
}
else { die "unknown argument `$arg'" };
}
@@ -68,33 +76,15 @@ sub findRoots {
findRoots 1, $rootsDir;
# Determine all store paths reachable from the roots.
my $extraarg = "";
if ($keepSuccessors) { $extraarg = "--include-successors"; };
my $pid = open2(\*READ, \*WRITE, "@bindir@/nix-store --query --requisites $extraarg @roots")
or die "determining live paths";
close WRITE;
while (<READ>) {
chomp;
$alive{$_} = 1;
if ($invert) { print "$_\n"; };
# Run the collector with the roots we found.
my $pid = open2(">&1", \*WRITE, "@bindir@/nix-store --gc $gcOper --min-age $minAge")
or die "cannot run `nix-store --gc'";
foreach my $root (@roots) {
print WRITE "$root\n";
}
close READ;
close WRITE;
waitpid $pid, 0;
$? == 0 or die "determining live paths";
exit 0 if ($invert);
# Using that information, find all store paths *not* reachable from
# the roots.
opendir(DIR, $storeDir) or die "cannot open directory $storeDir: $!";
foreach my $name (readdir DIR) {
next if ($name eq "." || $name eq "..");
$name = "$storeDir/$name";
if (!$alive{$name}) {
print "$name\n";
}
}
closedir DIR;
$? == 0 or die "`nix-store --gc' failed";
+25 -10
View File
@@ -7,9 +7,18 @@ if test -z "$url"; then
exit 1
fi
# !!! race? should be relatively safe, `svn export' barfs if $tmpPath exists.
# !!! race
tmpPath1=@storedir@/nix-prefetch-url-$$
# Test whether we have write permission in the store. If not, fetch
# to /tmp and don't copy to the store. This is a hack to make this
# script at least work somewhat in setuid installations.
if ! touch $tmpPath1 2> /dev/null; then
echo "(cannot write to the store, result won't be cached)" >&2
dummyMode=1
tmpPath1=/tmp/nix-prefetch-url-$$ # !!! security?
fi
# Perform the checkout.
@curl@ --fail --location --max-redirs 20 "$url" > $tmpPath1
@@ -17,21 +26,27 @@ tmpPath1=@storedir@/nix-prefetch-url-$$
hash=$(@bindir@/nix-hash --flat $tmpPath1)
echo "hash is $hash" >&2
# Rename it so that the fetchsvn builder can find it.
tmpPath2=@storedir@/nix-prefetch-url-$hash
test -e $tmpPath2 || mv $tmpPath1 $tmpPath2 # !!! race
# Rename it so that the fetchurl builder can find it.
if test "$dummyMode" != 1; then
tmpPath2=@storedir@/nix-prefetch-url-$hash
test -e $tmpPath2 || mv $tmpPath1 $tmpPath2 # !!! race
fi
# Create a Nix expression that does a fetchsvn.
# Create a Nix expression that does a fetchurl.
storeExpr=$( \
echo "(import @datadir@/nix/corepkgs/fetchurl) \
echo "(import @datadir@/nix/corepkgs/fetchurl) \
{url = $url; md5 = \"$hash\"; system = \"@system@\";}" \
| nix-instantiate -)
| @bindir@/nix-instantiate -)
# Realise it.
finalPath=$(nix-store -qnB --force-realise $storeExpr)
finalPath=$(@bindir@/nix-store -qnB --force-realise $storeExpr)
echo "path is $finalPath" >&2
rm -rf $tmpPath2 || true
rm -rf $tmpPath1 $tmpPath2 || true
echo $hash
if test -n "$PRINT_PATH"; then
echo $finalPath
fi
+33 -49
View File
@@ -10,18 +10,18 @@ do { $tmpdir = tmpnam(); }
until mkdir $tmpdir, 0777;
my $manifest = "$tmpdir/manifest";
my $conffile = "@sysconfdir@/nix/prebuilts.conf";
my $confFile = "@sysconfdir@/nix/prebuilts.conf";
#END { unlink $manifest; rmdir $tmpdir; }
# Obtain URLs either from the command line or from a configuration file.
my %storepaths2urls;
my %storePaths2urls;
my %urls2hashes;
my %successors;
sub doURL {
my $url = shift;
processURL $manifest, $url, \%storepaths2urls, \%urls2hashes, \%successors;
processURL $manifest, $url, \%storePaths2urls, \%urls2hashes, \%successors;
}
if (scalar @ARGV > 0) {
while (@ARGV) {
@@ -29,7 +29,7 @@ if (scalar @ARGV > 0) {
doURL $url;
}
} else {
open CONFFILE, "<$conffile";
open CONFFILE, "<$confFile";
while (<CONFFILE>) {
chomp;
if (/^\s*(\S+)\s*(\#.*)?$/) {
@@ -41,60 +41,44 @@ if (scalar @ARGV > 0) {
}
# Create a Nix expression for the substitutes.
my $fullexpr = "[";
my @storepaths;
foreach my $storepath (keys %storepaths2urls) {
# Construct a Nix expression that fetches and unpacks a
# Nix archive from the network.
my $url = $storepaths2urls{$storepath};
my $hash = $urls2hashes{$url};
my $fetch =
"(import @datadir@/nix/corepkgs/fetchurl) " .
"{url = $url; md5 = \"$hash\"; system = \"@system@\";}";
my $nixexpr =
"((import @datadir@/nix/corepkgs/nar/unnar.nix) " .
"{narFile = ($fetch); outPath = \"$storepath\"; system = \"@system@\";}) ";
$fullexpr .= $nixexpr; # !!! O(n^2)?
push @storepaths, $storepath;
}
$fullexpr .= "]";
my $size = scalar (keys %storePaths2urls);
print "$size store paths in manifest\n";
# Instantiate store expressions from the Nix expressions we created above.
print STDERR "instantiating store expressions...\n";
my $pid = open2(\*READ, \*WRITE, "@bindir@/nix-instantiate -") or die "cannot run nix-instantiate";
# Instantiate a store expression that builds the substitute program
# (the program that fetches URLs and unpacks them into the store).
my $nixExpr =
"(import @datadir@/nix/corepkgs/nix-pull) " .
"{system = \"@system@\";}";
print WRITE $fullexpr;
close WRITE;
my $i = 0;
my %substitutes;
while (<READ>) {
chomp;
die unless /^\//;
my $subpath = $_;
die unless ($i < scalar @storepaths);
$substitutes{$storepaths[$i++]} = $subpath;
}
waitpid $pid, 0;
$? == 0 or die "nix-instantiate failed";
print STDERR "instantiating store expression...\n";
my $storeExpr = `echo '$nixExpr' | @bindir@/nix-instantiate -`
or die "cannot instantiate Nix expression";
chomp $storeExpr;
# Register all substitutes.
print STDERR "registering substitutes...\n";
my @subs = %substitutes;
while (scalar @subs > 0) {
my $n = scalar @subs;
if ($n > 256) { $n = 256 };
my @subs2 = @subs[0..$n - 1];
@subs = @subs[$n..scalar @subs - 1];
system "@bindir@/nix-store --substitute @subs2";
if ($?) { die "`nix-store --substitute' failed"; }
my $pid = open2(\*READ, \*WRITE, "@bindir@/nix-store --substitute")
or die "cannot run nix-store";
close READ;
foreach my $storePath (keys %storePaths2urls) {
print WRITE "$storePath\n";
print WRITE "$storeExpr\n";
print WRITE "/fetch\n";
print WRITE "2\n";
print WRITE "$storePaths2urls{$storePath}\n";
print WRITE "$urls2hashes{$storePaths2urls{$storePath}}\n";
}
close WRITE;
waitpid $pid, 0;
$? == 0 or die "nix-store failed";
# Register all successors.
print STDERR "registering successors...\n";
+1 -1
View File
@@ -92,7 +92,7 @@ while (scalar @tmp > 0) {
my @tmp2 = @tmp[0..$n - 1];
@tmp = @tmp[$n..scalar @tmp - 1];
system "@bindir@/nix-store --realise -B @tmp2 > /dev/null";
system "@bindir@/nix-store --realise @tmp2 > /dev/null";
if ($?) { die "`nix-store --realise' failed"; }
open NARPATHS, "@bindir@/nix-store --query --list @tmp2 |" or die "cannot run nix";
+7 -7
View File
@@ -3,7 +3,7 @@ use strict;
sub processURL {
my $manifest = shift;
my $url = shift;
my $storepaths2urls = shift;
my $storePaths2urls = shift;
my $urls2hashes = shift;
my $successors = shift;
@@ -11,14 +11,14 @@ sub processURL {
print "obtaining list of Nix archives at $url...\n";
system("@curl@ --fail --silent --show-error --location --max-redirs 20 " .
"'$url' > '$manifest' 2> /dev/null") == 0
"'$url' > '$manifest'") == 0
or die "curl failed: $?";
open MANIFEST, "<$manifest";
my $inside = 0;
my $storepath;
my $storePath;
my $narurl;
my $hash;
my @preds;
@@ -31,7 +31,7 @@ sub processURL {
if (!$inside) {
if (/^\{$/) {
$inside = 1;
undef $storepath;
undef $storePath;
undef $narurl;
undef $hash;
@preds = ();
@@ -41,16 +41,16 @@ sub processURL {
if (/^\}$/) {
$inside = 0;
$$storepaths2urls{$storepath} = $narurl;
$$storePaths2urls{$storePath} = $narurl;
$$urls2hashes{$narurl} = $hash;
foreach my $p (@preds) {
$$successors{$p} = $storepath;
$$successors{$p} = $storePath;
}
}
elsif (/^\s*StorePath:\s*(\/\S+)\s*$/) {
$storepath = $1;
$storePath = $1;
}
elsif (/^\s*NarURL:\s*(\S+)\s*$/) {
$narurl = $1;
+13
View File
@@ -1,2 +1,15 @@
SUBDIRS = bin2c boost libutil libstore libmain nix-store nix-hash \
libexpr nix-instantiate nix-env log2xml
EXTRA_DIST = aterm-helper.pl
SETUID_PROGS = nix-store nix-instantiate nix-env
install-exec-hook:
if SETUID_HACK
if HAVE_SETRESUID
cd $(DESTDIR)$(bindir) && chown @NIX_USER@ $(SETUID_PROGS) \
&& chgrp @NIX_GROUP@ $(SETUID_PROGS) && chmod ug+s $(SETUID_PROGS)
else
cd $(DESTDIR)$(bindir) && chown root $(SETUID_PROGS) && chmod u+s $(SETUID_PROGS)
endif
endif
+143
View File
@@ -0,0 +1,143 @@
#! /usr/bin/perl -w
# This program generates C/C++ code for efficiently manipulating
# ATerms. It generates functions to build and match ATerms according
# to a set of constructor definitions defined in a file read from
# standard input. A constructor is defined by a line with the
# following format:
#
# SYM | ARGS | TYPE | FUN?
#
# where SYM is the name of the constructor, ARGS is a
# whitespace-separated list of argument types, TYPE is the type of the
# resulting ATerm (which should be `ATerm' or a type synonym for
# `ATerm'), and the optional FUN is used to construct the names of the
# build and match functions (it defaults to SYM; overriding it is
# useful if there are overloaded constructors, e.g., with different
# arities). Note that SYM may be empty.
#
# A line of the form
#
# VAR = EXPR
#
# causes a ATerm variable to be generated that is initialised to the
# value EXPR.
#
# Finally, a line of the form
#
# init NAME
#
# causes the initialisation function to be called `NAME'. This
# function must be called before any of the build/match functions or
# the generated variables are used.
die if scalar @ARGV != 2;
my $syms = "";
my $init = "";
my $initFun = "init";
open HEADER, ">$ARGV[0]";
open IMPL, ">$ARGV[1]";
while (<STDIN>) {
next if (/^\s*$/);
if (/^\s*(\w*)\s*\|([^\|]*)\|\s*(\w+)\s*\|\s*(\w+)?/) {
my $const = $1;
my @types = split ' ', $2;
my $result = $3;
my $funname = $4;
$funname = $const unless defined $funname;
my $formals = "";
my $formals2 = "";
my $args = "";
my $unpack = "";
my $n = 1;
foreach my $type (@types) {
$args .= ", ";
if ($type eq "string") {
# $args .= "(ATerm) ATmakeAppl0(ATmakeAFun((char *) e$n, 0, ATtrue))";
# $type = "const char *";
$type = "ATerm";
$args .= "e$n";
# !!! in the matcher, we should check that the
# argument is a string (i.e., a nullary application).
} elsif ($type eq "int") {
$args .= "(ATerm) ATmakeInt(e$n)";
} elsif ($type eq "ATermList" || $type eq "ATermBlob") {
$args .= "(ATerm) e$n";
} else {
$args .= "e$n";
}
$formals .= ", " if $formals ne "";
$formals .= "$type e$n";
$formals2 .= ", ";
$formals2 .= "$type & e$n";
my $m = $n - 1;
# !!! more checks here
if ($type eq "int") {
$unpack .= " e$n = ATgetInt((ATermInt) ATgetArgument(e, $m));\n";
} elsif ($type eq "ATermList") {
$unpack .= " e$n = (ATermList) ATgetArgument(e, $m);\n";
} elsif ($type eq "ATermBlob") {
$unpack .= " e$n = (ATermBlob) ATgetArgument(e, $m);\n";
} else {
$unpack .= " e$n = ATgetArgument(e, $m);\n";
}
$n++;
}
my $arity = scalar @types;
print HEADER "extern AFun sym$funname;\n\n";
print IMPL "AFun sym$funname = 0;\n";
print HEADER "static inline $result make$funname($formals) {\n";
print HEADER " return (ATerm) ATmakeAppl$arity(sym$funname$args);\n";
print HEADER "}\n\n";
print HEADER "#ifdef __cplusplus\n";
print HEADER "static inline bool match$funname(ATerm e$formals2) {\n";
print HEADER " if (ATgetType(e) != AT_APPL || ATgetAFun(e) != sym$funname) return false;\n";
print HEADER "$unpack";
print HEADER " return true;\n";
print HEADER "}\n";
print HEADER "#endif\n\n\n";
$init .= " sym$funname = ATmakeAFun(\"$const\", $arity, ATfalse);\n";
$init .= " ATprotectAFun(sym$funname);\n";
}
elsif (/^\s*(\w+)\s*=\s*(.*)$/) {
my $name = $1;
my $value = $2;
print HEADER "extern ATerm $name;\n";
print IMPL "ATerm $name = 0;\n";
$init .= " $name = $value;\n";
}
elsif (/^\s*init\s+(\w+)\s*$/) {
$initFun = $1;
}
else {
die "bad line: `$_'";
}
}
print HEADER "void $initFun();\n\n";
print HEADER "static inline const char * aterm2String(ATerm t) {\n";
print HEADER " return (const char *) ATgetName(ATgetAFun(t));\n";
print HEADER "}\n\n";
print IMPL "\n";
print IMPL "void $initFun() {\n";
print IMPL "$init";
print IMPL "}\n";
close HEADER;
close IMPL;
+4 -1
View File
@@ -1,3 +1,6 @@
SUBDIRS = format
EXTRA_DIST = format.hpp
EXTRA_DIST = assert.hpp checked_delete.hpp format.hpp \
shared_ptr.hpp weak_ptr.hpp throw_exception.hpp \
enable_shared_from_this.hpp \
detail/shared_count.hpp detail/workaround.hpp
+38
View File
@@ -0,0 +1,38 @@
//
// boost/assert.hpp - BOOST_ASSERT(expr)
//
// Copyright (c) 2001, 2002 Peter Dimov and Multi Media Ltd.
//
// Permission to copy, use, modify, sell and distribute this software
// is granted provided this copyright notice appears in all copies.
// This software is provided "as is" without express or implied
// warranty, and with no claim as to its suitability for any purpose.
//
// Note: There are no include guards. This is intentional.
//
// See http://www.boost.org/libs/utility/assert.html for documentation.
//
#undef BOOST_ASSERT
#if defined(BOOST_DISABLE_ASSERTS)
# define BOOST_ASSERT(expr) ((void)0)
#elif defined(BOOST_ENABLE_ASSERT_HANDLER)
#include <boost/current_function.hpp>
namespace boost
{
void assertion_failed(char const * expr, char const * function, char const * file, long line); // user defined
} // namespace boost
#define BOOST_ASSERT(expr) ((expr)? ((void)0): ::boost::assertion_failed(#expr, BOOST_CURRENT_FUNCTION, __FILE__, __LINE__))
#else
# include <assert.h>
# define BOOST_ASSERT(expr) assert(expr)
#endif
+71
View File
@@ -0,0 +1,71 @@
#ifndef BOOST_CHECKED_DELETE_HPP_INCLUDED
#define BOOST_CHECKED_DELETE_HPP_INCLUDED
// MS compatible compilers support #pragma once
#if defined(_MSC_VER) && (_MSC_VER >= 1020)
# pragma once
#endif
//
// boost/checked_delete.hpp
//
// Copyright (c) 1999, 2000, 2001, 2002 boost.org
// Copyright (c) 2002, 2003 Peter Dimov
// Copyright (c) 2003 Daniel Frey
// Copyright (c) 2003 Howard Hinnant
//
// Permission to copy, use, modify, sell and distribute this software
// is granted provided this copyright notice appears in all copies.
// This software is provided "as is" without express or implied
// warranty, and with no claim as to its suitability for any purpose.
//
// See http://www.boost.org/libs/utility/checked_delete.html for documentation.
//
namespace boost
{
// verify that types are complete for increased safety
template<class T> inline void checked_delete(T * x)
{
// intentionally complex - simplification causes regressions
typedef char type_must_be_complete[ sizeof(T)? 1: -1 ];
(void) sizeof(type_must_be_complete);
delete x;
}
template<class T> inline void checked_array_delete(T * x)
{
typedef char type_must_be_complete[ sizeof(T)? 1: -1 ];
(void) sizeof(type_must_be_complete);
delete [] x;
}
template<class T> struct checked_deleter
{
typedef void result_type;
typedef T * argument_type;
void operator()(T * x) const
{
// boost:: disables ADL
boost::checked_delete(x);
}
};
template<class T> struct checked_array_deleter
{
typedef void result_type;
typedef T * argument_type;
void operator()(T * x) const
{
boost::checked_array_delete(x);
}
};
} // namespace boost
#endif // #ifndef BOOST_CHECKED_DELETE_HPP_INCLUDED
+555
View File
@@ -0,0 +1,555 @@
#ifndef BOOST_DETAIL_SHARED_COUNT_HPP_INCLUDED
#define BOOST_DETAIL_SHARED_COUNT_HPP_INCLUDED
// MS compatible compilers support #pragma once
#if defined(_MSC_VER) && (_MSC_VER >= 1020)
# pragma once
#endif
//
// detail/shared_count.hpp
//
// Copyright (c) 2001, 2002, 2003 Peter Dimov and Multi Media Ltd.
//
// Permission to copy, use, modify, sell and distribute this software
// is granted provided this copyright notice appears in all copies.
// This software is provided "as is" without express or implied
// warranty, and with no claim as to its suitability for any purpose.
//
//#include <boost/config.hpp>
#if defined(BOOST_SP_USE_STD_ALLOCATOR) && defined(BOOST_SP_USE_QUICK_ALLOCATOR)
# error BOOST_SP_USE_STD_ALLOCATOR and BOOST_SP_USE_QUICK_ALLOCATOR are incompatible.
#endif
#include <boost/checked_delete.hpp>
#include <boost/throw_exception.hpp>
//#include <boost/detail/lightweight_mutex.hpp>
#if defined(BOOST_SP_USE_QUICK_ALLOCATOR)
#include <boost/detail/quick_allocator.hpp>
#endif
#include <memory> // std::auto_ptr, std::allocator
#include <functional> // std::less
#include <exception> // std::exception
#include <new> // std::bad_alloc
#include <typeinfo> // std::type_info in get_deleter
#include <cstddef> // std::size_t
#ifdef __BORLANDC__
# pragma warn -8026 // Functions with excep. spec. are not expanded inline
# pragma warn -8027 // Functions containing try are not expanded inline
#endif
namespace boost
{
// Debug hooks
#if defined(BOOST_SP_ENABLE_DEBUG_HOOKS)
void sp_scalar_constructor_hook(void * px, std::size_t size, void * pn);
void sp_array_constructor_hook(void * px);
void sp_scalar_destructor_hook(void * px, std::size_t size, void * pn);
void sp_array_destructor_hook(void * px);
#endif
// The standard library that comes with Borland C++ 5.5.1
// defines std::exception and its members as having C calling
// convention (-pc). When the definition of bad_weak_ptr
// is compiled with -ps, the compiler issues an error.
// Hence, the temporary #pragma option -pc below. The version
// check is deliberately conservative.
#if defined(__BORLANDC__) && __BORLANDC__ == 0x551
# pragma option push -pc
#endif
class bad_weak_ptr: public std::exception
{
public:
virtual char const * what() const throw()
{
return "boost::bad_weak_ptr";
}
};
#if defined(__BORLANDC__) && __BORLANDC__ == 0x551
# pragma option pop
#endif
namespace detail
{
class sp_counted_base
{
private:
// typedef detail::lightweight_mutex mutex_type;
public:
sp_counted_base(): use_count_(1), weak_count_(1)
{
}
virtual ~sp_counted_base() // nothrow
{
}
// dispose() is called when use_count_ drops to zero, to release
// the resources managed by *this.
virtual void dispose() = 0; // nothrow
// destruct() is called when weak_count_ drops to zero.
virtual void destruct() // nothrow
{
delete this;
}
virtual void * get_deleter(std::type_info const & ti) = 0;
void add_ref_copy()
{
#if defined(BOOST_HAS_THREADS)
mutex_type::scoped_lock lock(mtx_);
#endif
++use_count_;
}
void add_ref_lock()
{
#if defined(BOOST_HAS_THREADS)
mutex_type::scoped_lock lock(mtx_);
#endif
if(use_count_ == 0) boost::throw_exception(boost::bad_weak_ptr());
++use_count_;
}
void release() // nothrow
{
{
#if defined(BOOST_HAS_THREADS)
mutex_type::scoped_lock lock(mtx_);
#endif
long new_use_count = --use_count_;
if(new_use_count != 0) return;
}
dispose();
weak_release();
}
void weak_add_ref() // nothrow
{
#if defined(BOOST_HAS_THREADS)
mutex_type::scoped_lock lock(mtx_);
#endif
++weak_count_;
}
void weak_release() // nothrow
{
long new_weak_count;
{
#if defined(BOOST_HAS_THREADS)
mutex_type::scoped_lock lock(mtx_);
#endif
new_weak_count = --weak_count_;
}
if(new_weak_count == 0)
{
destruct();
}
}
long use_count() const // nothrow
{
#if defined(BOOST_HAS_THREADS)
mutex_type::scoped_lock lock(mtx_);
#endif
return use_count_;
}
private:
sp_counted_base(sp_counted_base const &);
sp_counted_base & operator= (sp_counted_base const &);
long use_count_; // #shared
long weak_count_; // #weak + (#shared != 0)
#if defined(BOOST_HAS_THREADS) || defined(BOOST_LWM_WIN32)
mutable mutex_type mtx_;
#endif
};
#if defined(BOOST_SP_ENABLE_DEBUG_HOOKS)
template<class T> void cbi_call_constructor_hook(sp_counted_base * pn, T * px, checked_deleter<T> const &, int)
{
boost::sp_scalar_constructor_hook(px, sizeof(T), pn);
}
template<class T> void cbi_call_constructor_hook(sp_counted_base *, T * px, checked_array_deleter<T> const &, int)
{
boost::sp_array_constructor_hook(px);
}
template<class P, class D> void cbi_call_constructor_hook(sp_counted_base *, P const &, D const &, long)
{
}
template<class T> void cbi_call_destructor_hook(sp_counted_base * pn, T * px, checked_deleter<T> const &, int)
{
boost::sp_scalar_destructor_hook(px, sizeof(T), pn);
}
template<class T> void cbi_call_destructor_hook(sp_counted_base *, T * px, checked_array_deleter<T> const &, int)
{
boost::sp_array_destructor_hook(px);
}
template<class P, class D> void cbi_call_destructor_hook(sp_counted_base *, P const &, D const &, long)
{
}
#endif
//
// Borland's Codeguard trips up over the -Vx- option here:
//
#ifdef __CODEGUARD__
# pragma option push -Vx-
#endif
template<class P, class D> class sp_counted_base_impl: public sp_counted_base
{
private:
P ptr; // copy constructor must not throw
D del; // copy constructor must not throw
sp_counted_base_impl(sp_counted_base_impl const &);
sp_counted_base_impl & operator= (sp_counted_base_impl const &);
typedef sp_counted_base_impl<P, D> this_type;
public:
// pre: initial_use_count <= initial_weak_count, d(p) must not throw
sp_counted_base_impl(P p, D d): ptr(p), del(d)
{
#if defined(BOOST_SP_ENABLE_DEBUG_HOOKS)
detail::cbi_call_constructor_hook(this, p, d, 0);
#endif
}
virtual void dispose() // nothrow
{
#if defined(BOOST_SP_ENABLE_DEBUG_HOOKS)
detail::cbi_call_destructor_hook(this, ptr, del, 0);
#endif
del(ptr);
}
virtual void * get_deleter(std::type_info const & ti)
{
return ti == typeid(D)? &del: 0;
}
#if defined(BOOST_SP_USE_STD_ALLOCATOR)
void * operator new(std::size_t)
{
return std::allocator<this_type>().allocate(1, static_cast<this_type *>(0));
}
void operator delete(void * p)
{
std::allocator<this_type>().deallocate(static_cast<this_type *>(p), 1);
}
#endif
#if defined(BOOST_SP_USE_QUICK_ALLOCATOR)
void * operator new(std::size_t)
{
return quick_allocator<this_type>::alloc();
}
void operator delete(void * p)
{
quick_allocator<this_type>::dealloc(p);
}
#endif
};
#if defined(BOOST_SP_ENABLE_DEBUG_HOOKS)
int const shared_count_id = 0x2C35F101;
int const weak_count_id = 0x298C38A4;
#endif
class weak_count;
class shared_count
{
private:
sp_counted_base * pi_;
#if defined(BOOST_SP_ENABLE_DEBUG_HOOKS)
int id_;
#endif
friend class weak_count;
public:
shared_count(): pi_(0) // nothrow
#if defined(BOOST_SP_ENABLE_DEBUG_HOOKS)
, id_(shared_count_id)
#endif
{
}
template<class P, class D> shared_count(P p, D d): pi_(0)
#if defined(BOOST_SP_ENABLE_DEBUG_HOOKS)
, id_(shared_count_id)
#endif
{
#ifndef BOOST_NO_EXCEPTIONS
try
{
pi_ = new sp_counted_base_impl<P, D>(p, d);
}
catch(...)
{
d(p); // delete p
throw;
}
#else
pi_ = new sp_counted_base_impl<P, D>(p, d);
if(pi_ == 0)
{
d(p); // delete p
boost::throw_exception(std::bad_alloc());
}
#endif
}
#ifndef BOOST_NO_AUTO_PTR
// auto_ptr<Y> is special cased to provide the strong guarantee
template<class Y>
explicit shared_count(std::auto_ptr<Y> & r): pi_(new sp_counted_base_impl< Y *, checked_deleter<Y> >(r.get(), checked_deleter<Y>()))
#if defined(BOOST_SP_ENABLE_DEBUG_HOOKS)
, id_(shared_count_id)
#endif
{
r.release();
}
#endif
~shared_count() // nothrow
{
if(pi_ != 0) pi_->release();
#if defined(BOOST_SP_ENABLE_DEBUG_HOOKS)
id_ = 0;
#endif
}
shared_count(shared_count const & r): pi_(r.pi_) // nothrow
#if defined(BOOST_SP_ENABLE_DEBUG_HOOKS)
, id_(shared_count_id)
#endif
{
if(pi_ != 0) pi_->add_ref_copy();
}
explicit shared_count(weak_count const & r); // throws bad_weak_ptr when r.use_count() == 0
shared_count & operator= (shared_count const & r) // nothrow
{
sp_counted_base * tmp = r.pi_;
if(tmp != 0) tmp->add_ref_copy();
if(pi_ != 0) pi_->release();
pi_ = tmp;
return *this;
}
void swap(shared_count & r) // nothrow
{
sp_counted_base * tmp = r.pi_;
r.pi_ = pi_;
pi_ = tmp;
}
long use_count() const // nothrow
{
return pi_ != 0? pi_->use_count(): 0;
}
bool unique() const // nothrow
{
return use_count() == 1;
}
friend inline bool operator==(shared_count const & a, shared_count const & b)
{
return a.pi_ == b.pi_;
}
friend inline bool operator<(shared_count const & a, shared_count const & b)
{
return std::less<sp_counted_base *>()(a.pi_, b.pi_);
}
void * get_deleter(std::type_info const & ti) const
{
return pi_? pi_->get_deleter(ti): 0;
}
};
#ifdef __CODEGUARD__
# pragma option pop
#endif
class weak_count
{
private:
sp_counted_base * pi_;
#if defined(BOOST_SP_ENABLE_DEBUG_HOOKS)
int id_;
#endif
friend class shared_count;
public:
weak_count(): pi_(0) // nothrow
#if defined(BOOST_SP_ENABLE_DEBUG_HOOKS)
, id_(weak_count_id)
#endif
{
}
weak_count(shared_count const & r): pi_(r.pi_) // nothrow
#if defined(BOOST_SP_ENABLE_DEBUG_HOOKS)
, id_(shared_count_id)
#endif
{
if(pi_ != 0) pi_->weak_add_ref();
}
weak_count(weak_count const & r): pi_(r.pi_) // nothrow
#if defined(BOOST_SP_ENABLE_DEBUG_HOOKS)
, id_(shared_count_id)
#endif
{
if(pi_ != 0) pi_->weak_add_ref();
}
~weak_count() // nothrow
{
if(pi_ != 0) pi_->weak_release();
#if defined(BOOST_SP_ENABLE_DEBUG_HOOKS)
id_ = 0;
#endif
}
weak_count & operator= (shared_count const & r) // nothrow
{
sp_counted_base * tmp = r.pi_;
if(tmp != 0) tmp->weak_add_ref();
if(pi_ != 0) pi_->weak_release();
pi_ = tmp;
return *this;
}
weak_count & operator= (weak_count const & r) // nothrow
{
sp_counted_base * tmp = r.pi_;
if(tmp != 0) tmp->weak_add_ref();
if(pi_ != 0) pi_->weak_release();
pi_ = tmp;
return *this;
}
void swap(weak_count & r) // nothrow
{
sp_counted_base * tmp = r.pi_;
r.pi_ = pi_;
pi_ = tmp;
}
long use_count() const // nothrow
{
return pi_ != 0? pi_->use_count(): 0;
}
friend inline bool operator==(weak_count const & a, weak_count const & b)
{
return a.pi_ == b.pi_;
}
friend inline bool operator<(weak_count const & a, weak_count const & b)
{
return std::less<sp_counted_base *>()(a.pi_, b.pi_);
}
};
inline shared_count::shared_count(weak_count const & r): pi_(r.pi_)
#if defined(BOOST_SP_ENABLE_DEBUG_HOOKS)
, id_(shared_count_id)
#endif
{
if(pi_ != 0)
{
pi_->add_ref_lock();
}
else
{
boost::throw_exception(boost::bad_weak_ptr());
}
}
} // namespace detail
} // namespace boost
#ifdef __BORLANDC__
# pragma warn .8027 // Functions containing try are not expanded inline
# pragma warn .8026 // Functions with excep. spec. are not expanded inline
#endif
#endif // #ifndef BOOST_DETAIL_SHARED_COUNT_HPP_INCLUDED
+74
View File
@@ -0,0 +1,74 @@
// Copyright David Abrahams 2002. Permission to copy, use,
// modify, sell and distribute this software is granted provided this
// copyright notice appears in all copies. This software is provided
// "as is" without express or implied warranty, and with no claim as
// to its suitability for any purpose.
#ifndef WORKAROUND_DWA2002126_HPP
# define WORKAROUND_DWA2002126_HPP
// Compiler/library version workaround macro
//
// Usage:
//
// #if BOOST_WORKAROUND(BOOST_MSVC, <= 1200)
// ... // workaround code here
// #endif
//
// When BOOST_STRICT_CONFIG is defined, expands to 0. Otherwise, the
// first argument must be undefined or expand to a numeric
// value. The above expands to:
//
// (BOOST_MSVC) != 0 && (BOOST_MSVC) <= 1200
//
// When used for workarounds that apply to the latest known version
// and all earlier versions of a compiler, the following convention
// should be observed:
//
// #if BOOST_WORKAROUND(BOOST_MSVC, BOOST_TESTED_AT(1301))
//
// The version number in this case corresponds to the last version in
// which the workaround was known to have been required. When
// BOOST_DETECT_OUTDATED_WORKAROUNDS is not the defined, the macro
// BOOST_TESTED_AT(x) expands to "!= 0", which effectively activates
// the workaround for any version of the compiler. When
// BOOST_DETECT_OUTDATED_WORKAROUNDS is defined, a compiler warning or
// error will be issued if the compiler version exceeds the argument
// to BOOST_TESTED_AT(). This can be used to locate workarounds which
// may be obsoleted by newer versions.
# ifndef BOOST_STRICT_CONFIG
# define BOOST_WORKAROUND(symbol, test) \
((symbol != 0) && (1 % (( (symbol test) ) + 1)))
// ^ ^ ^ ^
// The extra level of parenthesis nesting above, along with the
// BOOST_OPEN_PAREN indirection below, is required to satisfy the
// broken preprocessor in MWCW 8.3 and earlier.
//
// The basic mechanism works as follows:
// (symbol test) + 1 => if (symbol test) then 2 else 1
// 1 % ((symbol test) + 1) => if (symbol test) then 1 else 0
//
// The complication with % is for cooperation with BOOST_TESTED_AT().
// When "test" is BOOST_TESTED_AT(x) and
// BOOST_DETECT_OUTDATED_WORKAROUNDS is #defined,
//
// symbol test => if (symbol <= x) then 1 else -1
// (symbol test) + 1 => if (symbol <= x) then 2 else 0
// 1 % ((symbol test) + 1) => if (symbol <= x) then 1 else divide-by-zero
//
# ifdef BOOST_DETECT_OUTDATED_WORKAROUNDS
# define BOOST_OPEN_PAREN (
# define BOOST_TESTED_AT(value) > value) ?(-1): BOOST_OPEN_PAREN 1
# else
# define BOOST_TESTED_AT(value) != ((value)-(value))
# endif
# else
# define BOOST_WORKAROUND(symbol, test) 0
# endif
#endif // WORKAROUND_DWA2002126_HPP
+68
View File
@@ -0,0 +1,68 @@
#ifndef BOOST_ENABLE_SHARED_FROM_THIS_HPP_INCLUDED
#define BOOST_ENABLE_SHARED_FROM_THIS_HPP_INCLUDED
//
// enable_shared_from_this.hpp
//
// Copyright (c) 2002 Peter Dimov
//
// Permission to copy, use, modify, sell and distribute this software
// is granted provided this copyright notice appears in all copies.
// This software is provided "as is" without express or implied
// warranty, and with no claim as to its suitability for any purpose.
//
// http://www.boost.org/libs/smart_ptr/enable_shared_from_this.html
//
#include <boost/weak_ptr.hpp>
#include <boost/shared_ptr.hpp>
#include <boost/assert.hpp>
//#include <boost/config.hpp>
namespace boost
{
template<class T> class enable_shared_from_this
{
protected:
enable_shared_from_this()
{
}
enable_shared_from_this(enable_shared_from_this const &)
{
}
enable_shared_from_this & operator=(enable_shared_from_this const &)
{
return *this;
}
~enable_shared_from_this()
{
}
public:
shared_ptr<T> shared_from_this()
{
shared_ptr<T> p(_internal_weak_this);
BOOST_ASSERT(p.get() == this);
return p;
}
shared_ptr<T const> shared_from_this() const
{
shared_ptr<T const> p(_internal_weak_this);
BOOST_ASSERT(p.get() == this);
return p;
}
typedef T _internal_element_type; // for bcc 5.5.1
weak_ptr<_internal_element_type> _internal_weak_this;
};
} // namespace boost
#endif // #ifndef BOOST_ENABLE_SHARED_FROM_THIS_HPP_INCLUDED
-10
View File
@@ -34,16 +34,6 @@
#include <boost/format/macros_default.hpp>
namespace boost
{
template<class E> void throw_exception(E const & e)
{
throw e;
}
}
#define BOOST_ASSERT(expr) assert(expr)
// **** Forward declarations ----------------------------------
#include <boost/format/format_fwd.hpp> // basic_format<Ch,Tr>, and other frontends
+1 -1
View File
@@ -24,7 +24,7 @@
#include "boost/format/format_class.hpp"
#include "boost/format/group.hpp"
//#include "boost/throw_exception.hpp"
#include "boost/throw_exception.hpp"
namespace boost {
namespace io {
+2 -2
View File
@@ -20,8 +20,8 @@
#ifndef BOOST_FORMAT_IMPLEMENTATION_HPP
#define BOOST_FORMAT_IMPLEMENTATION_HPP
//#include <boost/throw_exception.hpp>
//#include <boost/assert.hpp>
#include <boost/throw_exception.hpp>
#include <boost/assert.hpp>
#include <boost/format.hpp>
namespace boost {
+1 -1
View File
@@ -20,7 +20,7 @@
#define BOOST_FORMAT_FUNCS_HPP
#include "boost/format.hpp"
//#include "boost/throw_exception.hpp"
#include "boost/throw_exception.hpp"
namespace boost {
+2 -2
View File
@@ -23,8 +23,8 @@
#include <boost/format.hpp>
//#include <boost/throw_exception.hpp>
//#include <boost/assert.hpp>
#include <boost/throw_exception.hpp>
#include <boost/assert.hpp>
namespace boost {
+474
View File
@@ -0,0 +1,474 @@
#ifndef BOOST_SHARED_PTR_HPP_INCLUDED
#define BOOST_SHARED_PTR_HPP_INCLUDED
//
// shared_ptr.hpp
//
// (C) Copyright Greg Colvin and Beman Dawes 1998, 1999.
// Copyright (c) 2001, 2002, 2003 Peter Dimov
//
// Permission to copy, use, modify, sell and distribute this software
// is granted provided this copyright notice appears in all copies.
// This software is provided "as is" without express or implied
// warranty, and with no claim as to its suitability for any purpose.
//
// See http://www.boost.org/libs/smart_ptr/shared_ptr.htm for documentation.
//
//#include <boost/config.hpp> // for broken compiler workarounds
#if defined(BOOST_NO_MEMBER_TEMPLATES) && !defined(BOOST_MSVC6_MEMBER_TEMPLATES)
#include <boost/detail/shared_ptr_nmt.hpp>
#else
#include <boost/assert.hpp>
#include <boost/checked_delete.hpp>
#include <boost/throw_exception.hpp>
#include <boost/detail/shared_count.hpp>
#include <boost/detail/workaround.hpp>
#include <memory> // for std::auto_ptr
#include <algorithm> // for std::swap
#include <functional> // for std::less
#include <typeinfo> // for std::bad_cast
#include <iosfwd> // for std::basic_ostream
#ifdef BOOST_MSVC // moved here to work around VC++ compiler crash
# pragma warning(push)
# pragma warning(disable:4284) // odd return type for operator->
#endif
namespace boost
{
template<class T> class weak_ptr;
template<class T> class enable_shared_from_this;
namespace detail
{
struct static_cast_tag {};
struct const_cast_tag {};
struct dynamic_cast_tag {};
struct polymorphic_cast_tag {};
template<class T> struct shared_ptr_traits
{
typedef T & reference;
};
template<> struct shared_ptr_traits<void>
{
typedef void reference;
};
#if !defined(BOOST_NO_CV_VOID_SPECIALIZATIONS)
template<> struct shared_ptr_traits<void const>
{
typedef void reference;
};
template<> struct shared_ptr_traits<void volatile>
{
typedef void reference;
};
template<> struct shared_ptr_traits<void const volatile>
{
typedef void reference;
};
#endif
// enable_shared_from_this support
template<class T, class Y> void sp_enable_shared_from_this(boost::enable_shared_from_this<T> * pe, Y * px, shared_count const & pn)
{
if(pe != 0) pe->_internal_weak_this._internal_assign(px, pn);
}
inline void sp_enable_shared_from_this(void const volatile *, void const volatile *, shared_count const &)
{
}
} // namespace detail
//
// shared_ptr
//
// An enhanced relative of scoped_ptr with reference counted copy semantics.
// The object pointed to is deleted when the last shared_ptr pointing to it
// is destroyed or reset.
//
template<class T> class shared_ptr
{
private:
// Borland 5.5.1 specific workaround
typedef shared_ptr<T> this_type;
public:
typedef T element_type;
typedef T value_type;
typedef T * pointer;
typedef typename detail::shared_ptr_traits<T>::reference reference;
shared_ptr(): px(0), pn() // never throws in 1.30+
{
}
template<class Y>
explicit shared_ptr(Y * p): px(p), pn(p, checked_deleter<Y>()) // Y must be complete
{
detail::sp_enable_shared_from_this(p, p, pn);
}
//
// Requirements: D's copy constructor must not throw
//
// shared_ptr will release p by calling d(p)
//
template<class Y, class D> shared_ptr(Y * p, D d): px(p), pn(p, d)
{
detail::sp_enable_shared_from_this(p, p, pn);
}
// generated copy constructor, assignment, destructor are fine...
// except that Borland C++ has a bug, and g++ with -Wsynth warns
#if defined(__BORLANDC__) || defined(__GNUC__)
shared_ptr & operator=(shared_ptr const & r) // never throws
{
px = r.px;
pn = r.pn; // shared_count::op= doesn't throw
return *this;
}
#endif
template<class Y>
explicit shared_ptr(weak_ptr<Y> const & r): pn(r.pn) // may throw
{
// it is now safe to copy r.px, as pn(r.pn) did not throw
px = r.px;
}
template<class Y>
shared_ptr(shared_ptr<Y> const & r): px(r.px), pn(r.pn) // never throws
{
}
template<class Y>
shared_ptr(shared_ptr<Y> const & r, detail::static_cast_tag): px(static_cast<element_type *>(r.px)), pn(r.pn)
{
}
template<class Y>
shared_ptr(shared_ptr<Y> const & r, detail::const_cast_tag): px(const_cast<element_type *>(r.px)), pn(r.pn)
{
}
template<class Y>
shared_ptr(shared_ptr<Y> const & r, detail::dynamic_cast_tag): px(dynamic_cast<element_type *>(r.px)), pn(r.pn)
{
if(px == 0) // need to allocate new counter -- the cast failed
{
pn = detail::shared_count();
}
}
template<class Y>
shared_ptr(shared_ptr<Y> const & r, detail::polymorphic_cast_tag): px(dynamic_cast<element_type *>(r.px)), pn(r.pn)
{
if(px == 0)
{
boost::throw_exception(std::bad_cast());
}
}
#ifndef BOOST_NO_AUTO_PTR
template<class Y>
explicit shared_ptr(std::auto_ptr<Y> & r): px(r.get()), pn()
{
Y * tmp = r.get();
pn = detail::shared_count(r);
detail::sp_enable_shared_from_this(tmp, tmp, pn);
}
#endif
#if !defined(BOOST_MSVC) || (BOOST_MSVC > 1200)
template<class Y>
shared_ptr & operator=(shared_ptr<Y> const & r) // never throws
{
px = r.px;
pn = r.pn; // shared_count::op= doesn't throw
return *this;
}
#endif
#ifndef BOOST_NO_AUTO_PTR
template<class Y>
shared_ptr & operator=(std::auto_ptr<Y> & r)
{
this_type(r).swap(*this);
return *this;
}
#endif
void reset() // never throws in 1.30+
{
this_type().swap(*this);
}
template<class Y> void reset(Y * p) // Y must be complete
{
BOOST_ASSERT(p == 0 || p != px); // catch self-reset errors
this_type(p).swap(*this);
}
template<class Y, class D> void reset(Y * p, D d)
{
this_type(p, d).swap(*this);
}
reference operator* () const // never throws
{
BOOST_ASSERT(px != 0);
return *px;
}
T * operator-> () const // never throws
{
BOOST_ASSERT(px != 0);
return px;
}
T * get() const // never throws
{
return px;
}
// implicit conversion to "bool"
#if defined(__SUNPRO_CC) && BOOST_WORKAROUND(__SUNPRO_CC, <= 0x530)
operator bool () const
{
return px != 0;
}
#elif defined(__MWERKS__) && BOOST_WORKAROUND(__MWERKS__, BOOST_TESTED_AT(0x3003))
typedef T * (this_type::*unspecified_bool_type)() const;
operator unspecified_bool_type() const // never throws
{
return px == 0? 0: &this_type::get;
}
#else
typedef T * this_type::*unspecified_bool_type;
operator unspecified_bool_type() const // never throws
{
return px == 0? 0: &this_type::px;
}
#endif
// operator! is redundant, but some compilers need it
bool operator! () const // never throws
{
return px == 0;
}
bool unique() const // never throws
{
return pn.unique();
}
long use_count() const // never throws
{
return pn.use_count();
}
void swap(shared_ptr<T> & other) // never throws
{
std::swap(px, other.px);
pn.swap(other.pn);
}
template<class Y> bool _internal_less(shared_ptr<Y> const & rhs) const
{
return pn < rhs.pn;
}
void * _internal_get_deleter(std::type_info const & ti) const
{
return pn.get_deleter(ti);
}
// Tasteless as this may seem, making all members public allows member templates
// to work in the absence of member template friends. (Matthew Langston)
#ifndef BOOST_NO_MEMBER_TEMPLATE_FRIENDS
private:
template<class Y> friend class shared_ptr;
template<class Y> friend class weak_ptr;
#endif
T * px; // contained pointer
detail::shared_count pn; // reference counter
}; // shared_ptr
template<class T, class U> inline bool operator==(shared_ptr<T> const & a, shared_ptr<U> const & b)
{
return a.get() == b.get();
}
template<class T, class U> inline bool operator!=(shared_ptr<T> const & a, shared_ptr<U> const & b)
{
return a.get() != b.get();
}
#if __GNUC__ == 2 && __GNUC_MINOR__ <= 96
// Resolve the ambiguity between our op!= and the one in rel_ops
template<class T> inline bool operator!=(shared_ptr<T> const & a, shared_ptr<T> const & b)
{
return a.get() != b.get();
}
#endif
template<class T, class U> inline bool operator<(shared_ptr<T> const & a, shared_ptr<U> const & b)
{
return a._internal_less(b);
}
template<class T> inline void swap(shared_ptr<T> & a, shared_ptr<T> & b)
{
a.swap(b);
}
template<class T, class U> shared_ptr<T> static_pointer_cast(shared_ptr<U> const & r)
{
return shared_ptr<T>(r, detail::static_cast_tag());
}
template<class T, class U> shared_ptr<T> const_pointer_cast(shared_ptr<U> const & r)
{
return shared_ptr<T>(r, detail::const_cast_tag());
}
template<class T, class U> shared_ptr<T> dynamic_pointer_cast(shared_ptr<U> const & r)
{
return shared_ptr<T>(r, detail::dynamic_cast_tag());
}
// shared_*_cast names are deprecated. Use *_pointer_cast instead.
template<class T, class U> shared_ptr<T> shared_static_cast(shared_ptr<U> const & r)
{
return shared_ptr<T>(r, detail::static_cast_tag());
}
template<class T, class U> shared_ptr<T> shared_dynamic_cast(shared_ptr<U> const & r)
{
return shared_ptr<T>(r, detail::dynamic_cast_tag());
}
template<class T, class U> shared_ptr<T> shared_polymorphic_cast(shared_ptr<U> const & r)
{
return shared_ptr<T>(r, detail::polymorphic_cast_tag());
}
template<class T, class U> shared_ptr<T> shared_polymorphic_downcast(shared_ptr<U> const & r)
{
BOOST_ASSERT(dynamic_cast<T *>(r.get()) == r.get());
return shared_static_cast<T>(r);
}
// get_pointer() enables boost::mem_fn to recognize shared_ptr
template<class T> inline T * get_pointer(shared_ptr<T> const & p)
{
return p.get();
}
// operator<<
#if defined(__GNUC__) && (__GNUC__ < 3)
template<class Y> std::ostream & operator<< (std::ostream & os, shared_ptr<Y> const & p)
{
os << p.get();
return os;
}
#else
# if defined(BOOST_MSVC) && BOOST_WORKAROUND(BOOST_MSVC, <= 1200 && __SGI_STL_PORT)
// MSVC6 has problems finding std::basic_ostream through the using declaration in namespace _STL
using std::basic_ostream;
template<class E, class T, class Y> basic_ostream<E, T> & operator<< (basic_ostream<E, T> & os, shared_ptr<Y> const & p)
# else
template<class E, class T, class Y> std::basic_ostream<E, T> & operator<< (std::basic_ostream<E, T> & os, shared_ptr<Y> const & p)
# endif
{
os << p.get();
return os;
}
#endif
// get_deleter (experimental)
#if (defined(__GNUC__) && (__GNUC__ < 3)) || (defined(__EDG_VERSION__) && (__EDG_VERSION__ <= 238))
// g++ 2.9x doesn't allow static_cast<X const *>(void *)
// apparently EDG 2.38 also doesn't accept it
template<class D, class T> D * get_deleter(shared_ptr<T> const & p)
{
void const * q = p._internal_get_deleter(typeid(D));
return const_cast<D *>(static_cast<D const *>(q));
}
#else
template<class D, class T> D * get_deleter(shared_ptr<T> const & p)
{
return static_cast<D *>(p._internal_get_deleter(typeid(D)));
}
#endif
} // namespace boost
#ifdef BOOST_MSVC
# pragma warning(pop)
#endif
#endif // #if defined(BOOST_NO_MEMBER_TEMPLATES) && !defined(BOOST_MSVC6_MEMBER_TEMPLATES)
#endif // #ifndef BOOST_SHARED_PTR_HPP_INCLUDED
+47
View File
@@ -0,0 +1,47 @@
#ifndef BOOST_THROW_EXCEPTION_HPP_INCLUDED
#define BOOST_THROW_EXCEPTION_HPP_INCLUDED
// MS compatible compilers support #pragma once
#if defined(_MSC_VER) && (_MSC_VER >= 1020)
# pragma once
#endif
//
// boost/throw_exception.hpp
//
// Copyright (c) 2002 Peter Dimov and Multi Media Ltd.
//
// Permission to copy, use, modify, sell and distribute this software
// is granted provided this copyright notice appears in all copies.
// This software is provided "as is" without express or implied
// warranty, and with no claim as to its suitability for any purpose.
//
// http://www.boost.org/libs/utility/throw_exception.html
//
//#include <boost/config.hpp>
#ifdef BOOST_NO_EXCEPTIONS
# include <exception>
#endif
namespace boost
{
#ifdef BOOST_NO_EXCEPTIONS
void throw_exception(std::exception const & e); // user defined
#else
template<class E> void throw_exception(E const & e)
{
throw e;
}
#endif
} // namespace boost
#endif // #ifndef BOOST_THROW_EXCEPTION_HPP_INCLUDED
+193
View File
@@ -0,0 +1,193 @@
#ifndef BOOST_WEAK_PTR_HPP_INCLUDED
#define BOOST_WEAK_PTR_HPP_INCLUDED
//
// weak_ptr.hpp
//
// Copyright (c) 2001, 2002, 2003 Peter Dimov
//
// Permission to copy, use, modify, sell and distribute this software
// is granted provided this copyright notice appears in all copies.
// This software is provided "as is" without express or implied
// warranty, and with no claim as to its suitability for any purpose.
//
// See http://www.boost.org/libs/smart_ptr/weak_ptr.htm for documentation.
//
#include <boost/shared_ptr.hpp>
#ifdef BOOST_MSVC // moved here to work around VC++ compiler crash
# pragma warning(push)
# pragma warning(disable:4284) // odd return type for operator->
#endif
namespace boost
{
template<class T> class weak_ptr
{
private:
// Borland 5.5.1 specific workarounds
typedef weak_ptr<T> this_type;
public:
typedef T element_type;
weak_ptr(): px(0), pn() // never throws in 1.30+
{
}
// generated copy constructor, assignment, destructor are fine
//
// The "obvious" converting constructor implementation:
//
// template<class Y>
// weak_ptr(weak_ptr<Y> const & r): px(r.px), pn(r.pn) // never throws
// {
// }
//
// has a serious problem.
//
// r.px may already have been invalidated. The px(r.px)
// conversion may require access to *r.px (virtual inheritance).
//
// It is not possible to avoid spurious access violations since
// in multithreaded programs r.px may be invalidated at any point.
//
template<class Y>
weak_ptr(weak_ptr<Y> const & r): pn(r.pn) // never throws
{
px = r.lock().get();
}
template<class Y>
weak_ptr(shared_ptr<Y> const & r): px(r.px), pn(r.pn) // never throws
{
}
#if !defined(BOOST_MSVC) || (BOOST_MSVC > 1200)
template<class Y>
weak_ptr & operator=(weak_ptr<Y> const & r) // never throws
{
px = r.lock().get();
pn = r.pn;
return *this;
}
template<class Y>
weak_ptr & operator=(shared_ptr<Y> const & r) // never throws
{
px = r.px;
pn = r.pn;
return *this;
}
#endif
shared_ptr<T> lock() const // never throws
{
#if defined(BOOST_HAS_THREADS)
// optimization: avoid throw overhead
if(expired())
{
return shared_ptr<element_type>();
}
try
{
return shared_ptr<element_type>(*this);
}
catch(bad_weak_ptr const &)
{
// Q: how can we get here?
// A: another thread may have invalidated r after the use_count test above.
return shared_ptr<element_type>();
}
#else
// optimization: avoid try/catch overhead when single threaded
return expired()? shared_ptr<element_type>(): shared_ptr<element_type>(*this);
#endif
}
long use_count() const // never throws
{
return pn.use_count();
}
bool expired() const // never throws
{
return pn.use_count() == 0;
}
void reset() // never throws in 1.30+
{
this_type().swap(*this);
}
void swap(this_type & other) // never throws
{
std::swap(px, other.px);
pn.swap(other.pn);
}
void _internal_assign(T * px2, detail::shared_count const & pn2)
{
px = px2;
pn = pn2;
}
template<class Y> bool _internal_less(weak_ptr<Y> const & rhs) const
{
return pn < rhs.pn;
}
// Tasteless as this may seem, making all members public allows member templates
// to work in the absence of member template friends. (Matthew Langston)
#ifndef BOOST_NO_MEMBER_TEMPLATE_FRIENDS
private:
template<class Y> friend class weak_ptr;
template<class Y> friend class shared_ptr;
#endif
T * px; // contained pointer
detail::weak_count pn; // reference counter
}; // weak_ptr
template<class T, class U> inline bool operator<(weak_ptr<T> const & a, weak_ptr<U> const & b)
{
return a._internal_less(b);
}
template<class T> void swap(weak_ptr<T> & a, weak_ptr<T> & b)
{
a.swap(b);
}
// deprecated, provided for backward compatibility
template<class T> shared_ptr<T> make_shared(weak_ptr<T> const & r)
{
return r.lock();
}
} // namespace boost
#ifdef BOOST_MSVC
# pragma warning(pop)
#endif
#endif // #ifndef BOOST_WEAK_PTR_HPP_INCLUDED
+12 -3
View File
@@ -1,10 +1,11 @@
noinst_LIBRARIES = libexpr.a
libexpr_a_SOURCES = nixexpr.cc nixexpr.hh parser.cc parser.hh \
eval.cc eval.hh primops.cc primops.hh \
lexer-tab.c lexer-tab.h parser-tab.c parser-tab.h
eval.cc eval.hh primops.cc \
lexer-tab.c lexer-tab.h parser-tab.c parser-tab.h \
nixexpr-ast.hh
EXTRA_DIST = lexer.l parser.y
EXTRA_DIST = lexer.l parser.y nixexpr-ast.def nixexpr-ast.cc
AM_CXXFLAGS = \
-I.. ${bdb_include} ${aterm_include} -I../libutil -I../libstore
@@ -23,4 +24,12 @@ lexer-tab.c lexer-tab.h: lexer.l
$(flex) --outfile lexer-tab.c --header-file=lexer-tab.h lexer.l
# ATerm helper function generation.
nixexpr-ast.cc nixexpr-ast.hh: ../aterm-helper.pl nixexpr-ast.def
$(perl) ../aterm-helper.pl nixexpr-ast.hh nixexpr-ast.cc < nixexpr-ast.def
nixexpr.hh: nixexpr-ast.hh
CLEANFILES =
+123 -112
View File
@@ -1,70 +1,40 @@
#include "eval.hh"
#include "parser.hh"
#include "primops.hh"
static void addPrimOp(ATermMap & map, const string & name, void * f)
{
map.set(name, (ATerm) ATmakeBlob(0, f));
}
static void * lookupPrimOp(ATermMap & map, ATerm name)
{
ATermBlob b = (ATermBlob) map.get(name);
if (!b) return 0;
return ATgetBlobData(b);
}
#include "nixexpr-ast.hh"
EvalState::EvalState()
: normalForms(32768, 50)
{
blackHole = ATmake("BlackHole()");
if (!blackHole) throw Error("cannot build black hole");
blackHole = makeBlackHole();
nrEvaluated = nrCached = 0;
addPrimOp0("true", primTrue);
addPrimOp0("false", primFalse);
addPrimOp0("null", primNull);
initNixExprHelpers();
addPrimOp1("import", primImport);
addPrimOp1("derivation", primDerivation);
addPrimOp1("baseNameOf", primBaseNameOf);
addPrimOp1("toString", primToString);
addPrimOp1("isNull", primIsNull);
primOpsAll.add(primOps0);
primOpsAll.add(primOps1);
addPrimOps();
}
void EvalState::addPrimOp0(const string & name, PrimOp0 primOp)
void EvalState::addPrimOp(const string & name,
unsigned int arity, PrimOp primOp)
{
addPrimOp(primOps0, name, (void *) primOp);
}
void EvalState::addPrimOp1(const string & name, PrimOp1 primOp)
{
addPrimOp(primOps1, name, (void *) primOp);
primOps.set(name, makePrimOpDef(arity, ATmakeBlob(0, (void *) primOp)));
}
/* Substitute an argument set into the body of a function. */
static Expr substArgs(Expr body, ATermList formals, Expr arg)
{
ATMatcher m;
ATermMap subs;
Expr undefined = ATmake("Undefined");
Expr undefined = makeUndefined();
/* Get the formal arguments. */
for (ATermIterator i(formals); i; ++i) {
Expr name, def;
if (atMatch(m, *i) >> "NoDefFormal" >> name)
if (matchNoDefFormal(*i, name))
subs.set(name, undefined);
else if (atMatch(m, *i) >> "DefFormal" >> name >> def)
else if (matchDefFormal(*i, name, def))
subs.set(name, def);
else abort(); /* can't happen */
}
@@ -99,38 +69,32 @@ static Expr substArgs(Expr body, ATermList formals, Expr arg)
(e.x) (e.y); y = e.x;}'. */
ATerm expandRec(ATerm e, ATermList rbnds, ATermList nrbnds)
{
ATMatcher m;
ATerm name;
Expr e2;
Pos pos;
/* Create the substitution list. */
ATermMap subs;
for (ATermIterator i(rbnds); i; ++i) {
if (!(atMatch(m, *i) >> "Bind" >> name >> e2))
abort(); /* can't happen */
subs.set(name, ATmake("Select(<term>, <term>)", e, name));
if (!matchBind(*i, name, e2, pos)) abort(); /* can't happen */
subs.set(name, makeSelect(e, name));
}
for (ATermIterator i(nrbnds); i; ++i) {
if (!(atMatch(m, *i) >> "Bind" >> name >> e2))
abort(); /* can't happen */
if (!matchBind(*i, name, e2, pos)) abort(); /* can't happen */
subs.set(name, e2);
}
/* Create the non-recursive set. */
ATermMap as;
for (ATermIterator i(rbnds); i; ++i) {
ATerm pos;
if (!(atMatch(m, *i) >> "Bind" >> name >> e2 >> pos))
abort(); /* can't happen */
as.set(name, ATmake("(<term>, <term>)", substitute(subs, e2), pos));
if (!matchBind(*i, name, e2, pos)) abort(); /* can't happen */
as.set(name, makeAttrRHS(substitute(subs, e2), pos));
}
/* Copy the non-recursive bindings. !!! inefficient */
for (ATermIterator i(nrbnds); i; ++i) {
ATerm pos;
if (!(atMatch(m, *i) >> "Bind" >> name >> e2 >> pos))
abort(); /* can't happen */
as.set(name, ATmake("(<term>, <term>)", e2, pos));
if (!matchBind(*i, name, e2, pos)) abort(); /* can't happen */
as.set(name, makeAttrRHS(e2, pos));
}
return makeAttrs(as);
@@ -152,73 +116,73 @@ static Expr updateAttrs(Expr e1, Expr e2)
string evalString(EvalState & state, Expr e)
{
e = evalExpr(state, e);
ATMatcher m;
string s;
if (!(atMatch(m, e) >> "Str" >> s))
throw Error("string expected");
return s;
ATerm s;
if (!matchStr(e, s)) throw Error("string expected");
return aterm2String(s);
}
Path evalPath(EvalState & state, Expr e)
{
e = evalExpr(state, e);
ATMatcher m;
string s;
if (!(atMatch(m, e) >> "Path" >> s))
throw Error("path expected");
return s;
ATerm s;
if (!matchPath(e, s)) throw Error("path expected");
return aterm2String(s);
}
bool evalBool(EvalState & state, Expr e)
{
e = evalExpr(state, e);
ATMatcher m;
if (atMatch(m, e) >> "Bool" >> "True") return true;
else if (atMatch(m, e) >> "Bool" >> "False") return false;
if (e == eTrue) return true;
else if (e == eFalse) return false;
else throw Error("boolean expected");
}
Expr evalExpr2(EvalState & state, Expr e)
{
ATMatcher m;
Expr e1, e2, e3, e4;
ATerm name, pos;
AFun sym = ATgetAFun(e);
/* Normal forms. */
string cons;
if (atMatch(m, e) >> cons &&
(cons == "Str" ||
cons == "Path" ||
cons == "SubPath" ||
cons == "Uri" ||
cons == "Null" ||
cons == "Int" ||
cons == "Bool" ||
cons == "Function" ||
cons == "Function1" ||
cons == "Attrs" ||
cons == "List"))
if (sym == symStr ||
sym == symPath ||
sym == symSubPath ||
sym == symUri ||
sym == symNull ||
sym == symInt ||
sym == symBool ||
sym == symFunction ||
sym == symFunction1 ||
sym == symAttrs ||
sym == symList ||
sym == symPrimOp)
return e;
/* The `Closed' constructor is just a way to prevent substitutions
into expressions not containing free variables. */
if (atMatch(m, e) >> "Closed" >> e1)
if (matchClosed(e, e1))
return evalExpr(state, e1);
/* Any encountered variables must be undeclared or primops. */
if (atMatch(m, e) >> "Var" >> name) {
PrimOp0 primOp = (PrimOp0) lookupPrimOp(state.primOps0, name);
if (primOp)
return primOp(state);
/* Any encountered variables must be primops (since undefined
variables are detected after parsing). */
if (matchVar(e, name)) {
ATerm primOp = state.primOps.get(name);
if (!primOp)
throw Error(format("impossible: undefined variable `%1%'") % name);
int arity;
ATermBlob fun;
if (!matchPrimOpDef(primOp, arity, fun)) abort();
if (arity == 0)
return ((PrimOp) ATgetBlobData(fun)) (state, ATermVector());
else
return e;
return makePrimOp(arity, fun, ATempty);
}
/* Function application. */
if (atMatch(m, e) >> "Call" >> e1 >> e2) {
if (matchCall(e, e1, e2)) {
ATermList formals;
ATerm pos;
@@ -227,28 +191,41 @@ Expr evalExpr2(EvalState & state, Expr e)
e1 = evalExpr(state, e1);
/* Is it a primop or a function? */
if (atMatch(m, e1) >> "Var" >> name) {
PrimOp1 primOp = (PrimOp1) lookupPrimOp(state.primOps1, name);
if (primOp) return primOp(state, e2); else abort();
int arity;
ATermBlob fun;
ATermList args;
if (matchPrimOp(e1, arity, fun, args)) {
args = ATinsert(args, e2);
if (ATgetLength(args) == arity) {
/* Put the arguments in a vector in reverse (i.e.,
actual) order. */
ATermVector args2(arity);
for (ATermIterator i(args); i; ++i)
args2[--arity] = *i;
return ((PrimOp) ATgetBlobData((ATermBlob) fun))
(state, args2);
} else
/* Need more arguments, so propagate the primop. */
return makePrimOp(arity, fun, args);
}
else if (atMatch(m, e1) >> "Function" >> formals >> e4 >> pos) {
else if (matchFunction(e1, formals, e4, pos)) {
e2 = evalExpr(state, e2);
try {
return evalExpr(state, substArgs(e4, formals, e2));
} catch (Error & e) {
throw Error(format("while evaluating function at %1%:\n%2%")
throw Error(format("while evaluating the function at %1%:\n%2%")
% showPos(pos) % e.msg());
}
}
else if (atMatch(m, e1) >> "Function1" >> name >> e4 >> pos) {
else if (matchFunction1(e1, name, e4, pos)) {
try {
ATermMap subs;
subs.set(name, e2);
return evalExpr(state, substitute(subs, e4));
} catch (Error & e) {
throw Error(format("while evaluating function at %1%:\n%2%")
throw Error(format("while evaluating the function at %1%:\n%2%")
% showPos(pos) % e.msg());
}
}
@@ -257,26 +234,26 @@ Expr evalExpr2(EvalState & state, Expr e)
}
/* Attribute selection. */
string s1;
if (atMatch(m, e) >> "Select" >> e1 >> s1) {
if (matchSelect(e, e1, name)) {
ATerm pos;
string s1 = aterm2String(name);
Expr a = queryAttr(evalExpr(state, e1), s1, pos);
if (!a) throw Error(format("attribute `%1%' missing") % s1);
try {
return evalExpr(state, a);
} catch (Error & e) {
throw Error(format("while evaluating attribute `%1%' at %2%:\n%3%")
throw Error(format("while evaluating the attribute `%1%' at %2%:\n%3%")
% s1 % showPos(pos) % e.msg());
}
}
/* Mutually recursive sets. */
ATermList rbnds, nrbnds;
if (atMatch(m, e) >> "Rec" >> rbnds >> nrbnds)
if (matchRec(e, rbnds, nrbnds))
return expandRec(e, rbnds, nrbnds);
/* Conditionals. */
if (atMatch(m, e) >> "If" >> e1 >> e2 >> e3) {
if (matchIf(e, e1, e2, e3)) {
if (evalBool(state, e1))
return evalExpr(state, e2);
else
@@ -284,47 +261,81 @@ Expr evalExpr2(EvalState & state, Expr e)
}
/* Assertions. */
if (atMatch(m, e) >> "Assert" >> e1 >> e2 >> pos) {
if (matchAssert(e, e1, e2, pos)) {
if (!evalBool(state, e1))
throw Error(format("assertion failed at %1%") % showPos(pos));
return evalExpr(state, e2);
}
/* Withs. */
if (matchWith(e, e1, e2, pos)) {
ATermMap attrs;
try {
e1 = evalExpr(state, e1);
queryAllAttrs(e1, attrs);
} catch (Error & e) {
throw Error(format("while evaluating the `with' definitions at %1%:\n%2%")
% showPos(pos) % e.msg());
}
try {
e2 = substitute(attrs, e2);
checkVarDefs(state.primOps, e2);
return evalExpr(state, e2);
} catch (Error & e) {
throw Error(format("while evaluating the `with' body at %1%:\n%2%")
% showPos(pos) % e.msg());
}
}
/* Generic equality. */
if (atMatch(m, e) >> "OpEq" >> e1 >> e2)
if (matchOpEq(e, e1, e2))
return makeBool(evalExpr(state, e1) == evalExpr(state, e2));
/* Generic inequality. */
if (atMatch(m, e) >> "OpNEq" >> e1 >> e2)
if (matchOpNEq(e, e1, e2))
return makeBool(evalExpr(state, e1) != evalExpr(state, e2));
/* Negation. */
if (atMatch(m, e) >> "OpNot" >> e1)
if (matchOpNot(e, e1))
return makeBool(!evalBool(state, e1));
/* Implication. */
if (atMatch(m, e) >> "OpImpl" >> e1 >> e2)
if (matchOpImpl(e, e1, e2))
return makeBool(!evalBool(state, e1) || evalBool(state, e2));
/* Conjunction (logical AND). */
if (atMatch(m, e) >> "OpAnd" >> e1 >> e2)
if (matchOpAnd(e, e1, e2))
return makeBool(evalBool(state, e1) && evalBool(state, e2));
/* Disjunction (logical OR). */
if (atMatch(m, e) >> "OpOr" >> e1 >> e2)
if (matchOpOr(e, e1, e2))
return makeBool(evalBool(state, e1) || evalBool(state, e2));
/* Attribute set update (//). */
if (atMatch(m, e) >> "OpUpdate" >> e1 >> e2)
if (matchOpUpdate(e, e1, e2))
return updateAttrs(evalExpr(state, e1), evalExpr(state, e2));
/* Attribute existence test (?). */
if (atMatch(m, e) >> "OpHasAttr" >> e1 >> name) {
if (matchOpHasAttr(e, e1, name)) {
ATermMap attrs;
queryAllAttrs(evalExpr(state, e1), attrs);
return makeBool(attrs.get(name) != 0);
}
/* String or path concatenation. */
if (matchOpPlus(e, e1, e2)) {
e1 = evalExpr(state, e1);
e2 = evalExpr(state, e2);
ATerm s1, s2;
if (matchStr(e1, s1) && matchStr(e2, s2))
return makeStr(toATerm(
(string) aterm2String(s1) + (string) aterm2String(s2)));
else if (matchPath(e1, s1) && matchPath(e2, s2))
return makePath(toATerm(canonPath(
(string) aterm2String(s1) + "/" + (string) aterm2String(s2))));
else throw Error("wrong argument types in `+' operator");
}
/* Barf. */
throw badTerm("invalid expression", e);
}
@@ -364,7 +375,7 @@ Expr evalFile(EvalState & state, const Path & path)
try {
return evalExpr(state, e);
} catch (Error & e) {
throw Error(format("while evaluating file `%1%':\n%2%")
throw Error(format("while evaluating the file `%1%':\n%2%")
% path % e.msg());
}
}
+10 -9
View File
@@ -8,21 +8,21 @@
#include "nixexpr.hh"
typedef map<Path, PathSet> DrvPaths;
typedef map<Path, PathSet> DrvRoots;
typedef map<Path, Hash> DrvHashes;
struct EvalState;
typedef Expr (* PrimOp0) (EvalState &);
typedef Expr (* PrimOp1) (EvalState &, Expr arg);
/* Note: using a ATermVector is safe here, since when we call a primop
we also have an ATermList on the stack. */
typedef Expr (* PrimOp) (EvalState &, const ATermVector & args);
struct EvalState
{
ATermMap normalForms;
ATermMap primOps0; /* nullary primops */
ATermMap primOps1; /* unary primops */
ATermMap primOpsAll;
DrvPaths drvPaths;
ATermMap primOps;
DrvRoots drvRoots;
DrvHashes drvHashes; /* normalised derivation hashes */
Expr blackHole;
@@ -31,8 +31,9 @@ struct EvalState
EvalState();
void addPrimOp0(const string & name, PrimOp0 primOp);
void addPrimOp1(const string & name, PrimOp1 primOp);
void addPrimOps();
void addPrimOp(const string & name,
unsigned int arity, PrimOp primOp);
};
+2 -1
View File
@@ -48,6 +48,7 @@ if { return IF; }
then { return THEN; }
else { return ELSE; }
assert { return ASSERT; }
with { return WITH; }
let { return LET; }
rec { return REC; }
inherit { return INHERIT; }
@@ -75,7 +76,7 @@ inherit { return INHERIT; }
[ \t\n]+ /* eat up whitespace */
\#[^\n]* /* single-line comments */
\/\*(.|\n)*\*\/ /* long comments */
\/\*([^*]|\*[^\/])*\*\/ /* long comments */
. return yytext[0];
+55
View File
@@ -0,0 +1,55 @@
init initNixExprHelpers
Pos | string int int | Pos |
NoPos | | Pos |
Function | ATermList Expr Pos | Expr |
Function1 | string Expr Pos | Expr |
Assert | Expr Expr Pos | Expr |
With | Expr Expr Pos | Expr |
If | Expr Expr Expr | Expr |
OpNot | Expr | Expr |
OpEq | Expr Expr | Expr |
OpNEq | Expr Expr | Expr |
OpAnd | Expr Expr | Expr |
OpOr | Expr Expr | Expr |
OpImpl | Expr Expr | Expr |
OpUpdate | Expr Expr | Expr |
SubPath | Expr Expr | Expr |
OpHasAttr | Expr string | Expr |
OpPlus | Expr Expr | Expr |
Call | Expr Expr | Expr |
Select | Expr string | Expr |
Var | string | Expr |
Int | int | Expr |
Str | string | Expr |
Path | string | Expr |
Uri | string | Expr |
List | ATermList | Expr |
BlackHole | | Expr |
Undefined | | Expr |
PrimOp | int ATermBlob ATermList | Expr |
Attrs | ATermList | Expr |
Closed | Expr | Expr |
Rec | ATermList ATermList | Expr |
Bool | ATerm | Expr |
Null | | Expr |
Bind | string Expr Pos | ATerm |
Bind | string Expr | ATerm | Bind2
Inherit | Expr ATermList Pos | ATerm |
Scope | | Expr |
NoDefFormal | string | ATerm |
DefFormal | string Expr | ATerm |
True | | ATerm |
False | | ATerm |
PrimOpDef | int ATermBlob | ATerm |
AttrRHS | Expr Pos | ATerm |
eTrue = makeBool(makeTrue())
eFalse = makeBool(makeFalse())
+58 -74
View File
@@ -2,6 +2,10 @@
#include "storeexpr.hh"
#include "nixexpr-ast.hh"
#include "nixexpr-ast.cc"
ATermMap::ATermMap(unsigned int initialSize, unsigned int maxLoadPct)
{
this->maxLoadPct = maxLoadPct;
@@ -38,7 +42,7 @@ void ATermMap::set(ATerm key, ATerm value)
void ATermMap::set(const string & key, ATerm value)
{
set(string2ATerm(key), value);
set(toATerm(key), value);
}
@@ -50,7 +54,7 @@ ATerm ATermMap::get(ATerm key) const
ATerm ATermMap::get(const string & key) const
{
return get(string2ATerm(key));
return get(toATerm(key));
}
@@ -62,7 +66,7 @@ void ATermMap::remove(ATerm key)
void ATermMap::remove(const string & key)
{
remove(string2ATerm(key));
remove(toATerm(key));
}
@@ -94,28 +98,14 @@ void ATermMap::reset()
}
ATerm string2ATerm(const string & s)
{
return (ATerm) ATmakeAppl0(ATmakeAFun((char *) s.c_str(), 0, ATtrue));
}
string aterm2String(ATerm t)
{
return ATgetName(ATgetAFun(t));
}
string showPos(ATerm pos)
{
ATMatcher m;
Path path;
ATerm path;
int line, column;
if (atMatch(m, pos) >> "NoPos")
return "undefined position";
if (!(atMatch(m, pos) >> "Pos" >> path >> line >> column))
if (matchNoPos(pos)) return "undefined position";
if (!matchPos(pos, path, line, column))
throw badTerm("position expected", pos);
return (format("`%1%', line %2%") % path % line).str();
return (format("`%1%', line %2%") % aterm2String(path) % line).str();
}
@@ -150,18 +140,16 @@ ATerm bottomupRewrite(TermFun & f, ATerm e)
void queryAllAttrs(Expr e, ATermMap & attrs, bool withPos)
{
ATMatcher m;
ATermList bnds;
if (!(atMatch(m, e) >> "Attrs" >> bnds))
if (!matchAttrs(e, bnds))
throw Error("attribute set expected");
for (ATermIterator i(bnds); i; ++i) {
string s;
ATerm name;
Expr e;
ATerm pos;
if (!(atMatch(m, *i) >> "Bind" >> s >> e >> pos))
abort(); /* can't happen */
attrs.set(s, withPos ? ATmake("(<term>, <term>)", e, pos) : e);
if (!matchBind(*i, name, e, pos)) abort(); /* can't happen */
attrs.set(name, withPos ? makeAttrRHS(e, pos) : e);
}
}
@@ -175,18 +163,16 @@ Expr queryAttr(Expr e, const string & name)
Expr queryAttr(Expr e, const string & name, ATerm & pos)
{
ATMatcher m;
ATermList bnds;
if (!(atMatch(m, e) >> "Attrs" >> bnds))
if (!matchAttrs(e, bnds))
throw Error("attribute set expected");
for (ATermIterator i(bnds); i; ++i) {
string s;
ATerm name2, pos2;
Expr e;
ATerm pos2;
if (!(atMatch(m, *i) >> "Bind" >> s >> e >> pos2))
if (!matchBind(*i, name2, e, pos2))
abort(); /* can't happen */
if (s == name) {
if (aterm2String(name2) == name) {
pos = pos2;
return e;
}
@@ -198,17 +184,15 @@ Expr queryAttr(Expr e, const string & name, ATerm & pos)
Expr makeAttrs(const ATermMap & attrs)
{
ATMatcher m;
ATermList bnds = ATempty;
for (ATermIterator i(attrs.keys()); i; ++i) {
Expr e;
ATerm pos;
if (!(atMatch(m, attrs.get(*i)) >> "" >> e >> pos))
if (!matchAttrRHS(attrs.get(*i), e, pos))
abort(); /* can't happen */
bnds = ATinsert(bnds,
ATmake("Bind(<term>, <term>, <term>)", *i, e, pos));
bnds = ATinsert(bnds, makeBind(*i, e, pos));
}
return ATmake("Attrs(<term>)", ATreverse(bnds));
return makeAttrs(ATreverse(bnds));
}
@@ -216,57 +200,53 @@ Expr substitute(const ATermMap & subs, Expr e)
{
checkInterrupt();
ATMatcher m;
ATerm name, pos;
ATerm name, pos, e2;
/* As an optimisation, don't substitute in subterms known to be
closed. */
if (atMatch(m, e) >> "Closed") return e;
if (matchClosed(e, e2)) return e;
if (atMatch(m, e) >> "Var" >> name) {
if (matchVar(e, name)) {
Expr sub = subs.get(name);
return sub ? ATmake("Closed(<term>)", sub) : e;
return sub ? makeClosed(sub) : e;
}
/* In case of a function, filter out all variables bound by this
function. */
ATermList formals;
ATerm body;
if (atMatch(m, e) >> "Function" >> formals >> body >> pos) {
ATerm body, def;
if (matchFunction(e, formals, body, pos)) {
ATermMap subs2(subs);
for (ATermIterator i(formals); i; ++i) {
if (!(atMatch(m, *i) >> "NoDefFormal" >> name) &&
!(atMatch(m, *i) >> "DefFormal" >> name))
if (!matchNoDefFormal(*i, name) &&
!matchDefFormal(*i, name, def))
abort();
subs2.remove(name);
}
return ATmake("Function(<term>, <term>, <term>)",
substitute(subs, (ATerm) formals),
return makeFunction(
(ATermList) substitute(subs, (ATerm) formals),
substitute(subs2, body), pos);
}
if (atMatch(m, e) >> "Function1" >> name >> body >> pos) {
if (matchFunction1(e, name, body, pos)) {
ATermMap subs2(subs);
subs2.remove(name);
return ATmake("Function1(<term>, <term>, <term>)", name,
substitute(subs2, body), pos);
return makeFunction1(name, substitute(subs2, body), pos);
}
/* Idem for a mutually recursive attribute set. */
ATermList rbnds, nrbnds;
if (atMatch(m, e) >> "Rec" >> rbnds >> nrbnds) {
if (matchRec(e, rbnds, nrbnds)) {
ATermMap subs2(subs);
for (ATermIterator i(rbnds); i; ++i)
if (atMatch(m, *i) >> "Bind" >> name)
subs2.remove(name);
if (matchBind(*i, name, e2, pos)) subs2.remove(name);
else abort(); /* can't happen */
for (ATermIterator i(nrbnds); i; ++i)
if (atMatch(m, *i) >> "Bind" >> name)
subs2.remove(name);
if (matchBind(*i, name, e2, pos)) subs2.remove(name);
else abort(); /* can't happen */
return ATmake("Rec(<term>, <term>)",
substitute(subs2, (ATerm) rbnds),
substitute(subs, (ATerm) nrbnds));
return makeRec(
(ATermList) substitute(subs2, (ATerm) rbnds),
(ATermList) substitute(subs, (ATerm) nrbnds));
}
if (ATgetType(e) == AT_APPL) {
@@ -293,24 +273,23 @@ Expr substitute(const ATermMap & subs, Expr e)
void checkVarDefs(const ATermMap & defs, Expr e)
{
ATMatcher m;
ATerm name;
ATerm name, pos, value;
ATermList formals;
ATerm body;
ATerm with, body;
ATermList rbnds, nrbnds;
if (atMatch(m, e) >> "Var" >> name) {
if (matchVar(e, name)) {
if (!defs.get(name))
throw Error(format("undefined variable `%1%'")
% aterm2String(name));
}
else if (atMatch(m, e) >> "Function" >> formals >> body) {
else if (matchFunction(e, formals, body, pos)) {
ATermMap defs2(defs);
for (ATermIterator i(formals); i; ++i) {
Expr deflt;
if (!(atMatch(m, *i) >> "NoDefFormal" >> name))
if (atMatch(m, *i) >> "DefFormal" >> name >> deflt)
if (!matchNoDefFormal(*i, name))
if (matchDefFormal(*i, name, deflt))
checkVarDefs(defs, deflt);
else
abort();
@@ -319,27 +298,32 @@ void checkVarDefs(const ATermMap & defs, Expr e)
checkVarDefs(defs2, body);
}
else if (atMatch(m, e) >> "Function1" >> name >> body) {
else if (matchFunction1(e, name, body, pos)) {
ATermMap defs2(defs);
defs2.set(name, (ATerm) ATempty);
checkVarDefs(defs2, body);
}
else if (atMatch(m, e) >> "Rec" >> rbnds >> nrbnds) {
else if (matchRec(e, rbnds, nrbnds)) {
checkVarDefs(defs, (ATerm) nrbnds);
ATermMap defs2(defs);
for (ATermIterator i(rbnds); i; ++i) {
if (!(atMatch(m, *i) >> "Bind" >> name))
abort(); /* can't happen */
if (!matchBind(*i, name, value, pos)) abort(); /* can't happen */
defs2.set(name, (ATerm) ATempty);
}
for (ATermIterator i(nrbnds); i; ++i) {
if (!(atMatch(m, *i) >> "Bind" >> name))
abort(); /* can't happen */
if (!matchBind(*i, name, value, pos)) abort(); /* can't happen */
defs2.set(name, (ATerm) ATempty);
}
checkVarDefs(defs2, (ATerm) rbnds);
}
else if (matchWith(e, with, body, pos)) {
/* We can't check the body without evaluating the definitions
(which is an arbitrary expression), so we don't do that
here but only when actually evaluating the `with'. */
checkVarDefs(defs, with);
}
else if (ATgetType(e) == AT_APPL) {
int arity = ATgetArity(ATgetAFun(e));
@@ -355,5 +339,5 @@ void checkVarDefs(const ATermMap & defs, Expr e)
Expr makeBool(bool b)
{
return b ? ATmake("Bool(True)") : ATmake("Bool(False)");
return b ? eTrue : eFalse;
}
+13 -4
View File
@@ -13,6 +13,8 @@
normals forms efficiently. */
typedef ATerm Expr;
typedef ATerm Pos;
/* Mappings from ATerms to ATerms. This is just a wrapper around
ATerm tables. */
@@ -47,10 +49,11 @@ private:
};
/* Convert a string to an ATerm (i.e., a quoted nullary function
applicaton). */
ATerm string2ATerm(const string & s);
string aterm2String(ATerm t);
/* A STL vector of ATerms. Should be used with great care since it's
stored on the heap, and the elements are therefore not roots to the
ATerm garbage collector. */
typedef vector<ATerm> ATermVector;
/* Show a position. */
string showPos(ATerm pos);
@@ -86,5 +89,11 @@ void checkVarDefs(const ATermMap & def, Expr e);
/* Create an expression representing a boolean. */
Expr makeBool(bool b);
/* Create an expression representing a string. */
Expr makeString(const string & s);
/* Create an expression representing a path. */
Expr makePath(const Path & path);
#endif /* !__NIXEXPR_H */
+11 -12
View File
@@ -7,6 +7,7 @@
#include "aterm.hh"
#include "parser.hh"
#include "nixexpr-ast.hh"
struct ParseData
@@ -34,7 +35,7 @@ void setParseResult(ParseData * data, ATerm t)
ATerm absParsedPath(ParseData * data, ATerm t)
{
return string2ATerm(absPath(aterm2String(t), data->basePath).c_str());
return toATerm(absPath(aterm2String(t), data->basePath));
}
void parseError(ParseData * data, char * error, int line, int column)
@@ -45,28 +46,24 @@ void parseError(ParseData * data, char * error, int line, int column)
ATerm fixAttrs(int recursive, ATermList as)
{
ATMatcher m;
ATermList bs = ATempty, cs = ATempty;
ATermList * is = recursive ? &cs : &bs;
for (ATermIterator i(as); i; ++i) {
ATermList names;
Expr src;
ATerm pos;
if (atMatch(m, *i) >> "Inherit" >> src >> names >> pos) {
bool fromScope = atMatch(m, src) >> "Scope";
if (matchInherit(*i, src, names, pos)) {
bool fromScope = matchScope(src);
for (ATermIterator j(names); j; ++j) {
Expr rhs = fromScope
? ATmake("Var(<term>)", *j)
: ATmake("Select(<term>, <term>)", src, *j);
*is = ATinsert(*is, ATmake("Bind(<term>, <term>, <term>)",
*j, rhs, pos));
Expr rhs = fromScope ? makeVar(*j) : makeSelect(src, *j);
*is = ATinsert(*is, makeBind(*j, rhs, pos));
}
} else bs = ATinsert(bs, *i);
}
if (recursive)
return ATmake("Rec(<term>, <term>)", bs, cs);
return makeRec(bs, cs);
else
return ATmake("Attrs(<term>)", bs);
return makeAttrs(bs);
}
const char * getPath(ParseData * data)
@@ -95,7 +92,7 @@ static Expr parse(EvalState & state,
if (res) throw Error(data.error);
try {
checkVarDefs(state.primOpsAll, data.result);
checkVarDefs(state.primOps, data.result);
} catch (Error & e) {
throw Error(format("%1%, in `%2%'") % e.msg() % path);
}
@@ -106,6 +103,8 @@ static Expr parse(EvalState & state,
Expr parseExprFromFile(EvalState & state, Path path)
{
SwitchToOriginalUser sw;
assert(path[0] == '/');
#if 0
+48 -38
View File
@@ -15,6 +15,11 @@
#include "parser-tab.h"
#include "lexer-tab.h"
typedef ATerm Expr;
typedef ATerm Pos;
#include "nixexpr-ast.hh"
void setParseResult(void * data, ATerm t);
void parseError(void * data, char * error, int line, int column);
ATerm absParsedPath(void * data, ATerm t);
@@ -26,13 +31,18 @@ void yyerror(YYLTYPE * loc, yyscan_t scanner, void * data, char * s)
parseError(data, s, loc->first_line, loc->first_column);
}
ATerm makePos(YYLTYPE * loc, void * data)
ATerm toATerm(const char * s)
{
return ATmake("Pos(<str>, <int>, <int>)",
getPath(data), loc->first_line, loc->first_column);
return (ATerm) ATmakeAppl0(ATmakeAFun((char *) s, 0, ATtrue));
}
#define CUR_POS makePos(yylocp, data)
static Pos makeCurPos(YYLTYPE * loc, void * data)
{
return makePos(toATerm(getPath(data)),
loc->first_line, loc->first_column);
}
#define CUR_POS makeCurPos(yylocp, data)
%}
@@ -41,11 +51,11 @@ ATerm makePos(YYLTYPE * loc, void * data)
ATermList ts;
}
%type <t> start expr expr_function expr_assert expr_if expr_op
%type <t> start expr expr_function expr_if expr_op
%type <t> expr_app expr_select expr_simple bind inheritsrc formal
%type <ts> binds ids expr_list formals
%token <t> ID INT STR PATH URI
%token IF THEN ELSE ASSERT LET REC INHERIT EQ NEQ AND OR IMPL
%token IF THEN ELSE ASSERT WITH LET REC INHERIT EQ NEQ AND OR IMPL
%nonassoc IMPL
%left OR
@@ -53,6 +63,7 @@ ATerm makePos(YYLTYPE * loc, void * data)
%nonassoc EQ NEQ
%right UPDATE
%left NEG
%left '+'
%nonassoc '?'
%nonassoc '~'
@@ -64,65 +75,64 @@ expr: expr_function;
expr_function
: '{' formals '}' ':' expr_function
{ $$ = ATmake("Function(<term>, <term>, <term>)", $2, $5, CUR_POS); }
{ $$ = makeFunction($2, $5, CUR_POS); }
| ID ':' expr_function
{ $$ = ATmake("Function1(<term>, <term>, <term>)", $1, $3, CUR_POS); }
| expr_assert
;
expr_assert
: ASSERT expr ';' expr_assert
{ $$ = ATmake("Assert(<term>, <term>, <term>)", $2, $4, CUR_POS); }
{ $$ = makeFunction1($1, $3, CUR_POS); }
| ASSERT expr ';' expr_function
{ $$ = makeAssert($2, $4, CUR_POS); }
| WITH expr ';' expr_function
{ $$ = makeWith($2, $4, CUR_POS); }
| expr_if
;
expr_if
: IF expr THEN expr ELSE expr
{ $$ = ATmake("If(<term>, <term>, <term>)", $2, $4, $6); }
{ $$ = makeIf($2, $4, $6); }
| expr_op
;
expr_op
: '!' expr_op %prec NEG { $$ = ATmake("OpNot(<term>)", $2); }
| expr_op EQ expr_op { $$ = ATmake("OpEq(<term>, <term>)", $1, $3); }
| expr_op NEQ expr_op { $$ = ATmake("OpNEq(<term>, <term>)", $1, $3); }
| expr_op AND expr_op { $$ = ATmake("OpAnd(<term>, <term>)", $1, $3); }
| expr_op OR expr_op { $$ = ATmake("OpOr(<term>, <term>)", $1, $3); }
| expr_op IMPL expr_op { $$ = ATmake("OpImpl(<term>, <term>)", $1, $3); }
| expr_op UPDATE expr_op { $$ = ATmake("OpUpdate(<term>, <term>)", $1, $3); }
| expr_op '~' expr_op { $$ = ATmake("SubPath(<term>, <term>)", $1, $3); }
| expr_op '?' ID { $$ = ATmake("OpHasAttr(<term>, <term>)", $1, $3); }
: '!' expr_op %prec NEG { $$ = makeOpNot($2); }
| expr_op EQ expr_op { $$ = makeOpEq($1, $3); }
| expr_op NEQ expr_op { $$ = makeOpNEq($1, $3); }
| expr_op AND expr_op { $$ = makeOpAnd($1, $3); }
| expr_op OR expr_op { $$ = makeOpOr($1, $3); }
| expr_op IMPL expr_op { $$ = makeOpImpl($1, $3); }
| expr_op UPDATE expr_op { $$ = makeOpUpdate($1, $3); }
| expr_op '~' expr_op { $$ = makeSubPath($1, $3); }
| expr_op '?' ID { $$ = makeOpHasAttr($1, $3); }
| expr_op '+' expr_op { $$ = makeOpPlus($1, $3); }
| expr_app
;
expr_app
: expr_app expr_select
{ $$ = ATmake("Call(<term>, <term>)", $1, $2); }
{ $$ = makeCall($1, $2); }
| expr_select { $$ = $1; }
;
expr_select
: expr_select '.' ID
{ $$ = ATmake("Select(<term>, <term>)", $1, $3); }
{ $$ = makeSelect($1, $3); }
| expr_simple { $$ = $1; }
;
expr_simple
: ID { $$ = ATmake("Var(<term>)", $1); }
| INT { $$ = ATmake("Int(<term>)", $1); }
| STR { $$ = ATmake("Str(<term>)", $1); }
| PATH { $$ = ATmake("Path(<term>)", absParsedPath(data, $1)); }
| URI { $$ = ATmake("Uri(<term>)", $1); }
: ID { $$ = makeVar($1); }
| INT { $$ = makeInt(ATgetInt((ATermInt) $1)); }
| STR { $$ = makeStr($1); }
| PATH { $$ = makePath(absParsedPath(data, $1)); }
| URI { $$ = makeUri($1); }
| '(' expr ')' { $$ = $2; }
/* Let expressions `let {..., body = ...}' are just desugared
into `(rec {..., body = ...}).body'. */
| LET '{' binds '}'
{ $$ = ATmake("Select(<term>, \"body\")", fixAttrs(1, $3)); }
{ $$ = makeSelect(fixAttrs(1, $3), toATerm("body")); }
| REC '{' binds '}'
{ $$ = fixAttrs(1, $3); }
| '{' binds '}'
{ $$ = fixAttrs(0, $2); }
| '[' expr_list ']' { $$ = ATmake("List(<term>)", $2); }
| '[' expr_list ']' { $$ = makeList($2); }
;
binds
@@ -132,14 +142,14 @@ binds
bind
: ID '=' expr ';'
{ $$ = ATmake("Bind(<term>, <term>, <term>)", $1, $3, CUR_POS); }
{ $$ = makeBind($1, $3, CUR_POS); }
| INHERIT inheritsrc ids ';'
{ $$ = ATmake("Inherit(<term>, <term>, <term>)", $2, $3, CUR_POS); }
{ $$ = makeInherit($2, $3, CUR_POS); }
;
inheritsrc
: '(' expr ')' { $$ = $2; }
| { $$ = ATmake("Scope"); }
| { $$ = makeScope(); }
;
ids: ids ID { $$ = ATinsert($1, $2); } | { $$ = ATempty; };
@@ -158,8 +168,8 @@ formals
;
formal
: ID { $$ = ATmake("NoDefFormal(<term>)", $1); }
| ID '?' expr { $$ = ATmake("DefFormal(<term>, <term>)", $1, $3); }
: ID { $$ = makeNoDefFormal($1); }
| ID '?' expr { $$ = makeDefFormal($1, $3); }
;
%%
+125 -63
View File
@@ -1,26 +1,29 @@
#include "primops.hh"
#include "normalise.hh"
#include "eval.hh"
#include "globals.hh"
#include "nixexpr-ast.hh"
Expr primImport(EvalState & state, Expr arg)
/* Load and evaluate an expression from path specified by the
argument. */
static Expr primImport(EvalState & state, const ATermVector & args)
{
ATMatcher m;
string path;
if (!(atMatch(m, arg) >> "Path" >> path))
ATerm path;
Expr fn = evalExpr(state, args[0]);
if (!matchPath(fn, path))
throw Error("path expected");
return evalFile(state, path);
return evalFile(state, aterm2String(path));
}
static PathSet storeExprRootsCached(EvalState & state, const Path & nePath)
{
DrvPaths::iterator i = state.drvPaths.find(nePath);
if (i != state.drvPaths.end())
DrvRoots::iterator i = state.drvRoots.find(nePath);
if (i != state.drvRoots.end())
return i->second;
else {
PathSet paths = storeExprRoots(nePath);
state.drvPaths[nePath] = paths;
state.drvRoots[nePath] = paths;
return paths;
}
}
@@ -59,6 +62,8 @@ static Path copyAtom(EvalState & state, const Path & srcPath)
Path drvPath = writeTerm(unparseStoreExpr(ne), "");
state.drvHashes[drvPath] = drvHash;
state.drvRoots[drvPath] = ne.closure.roots;
printMsg(lvlChatty, format("copied `%1%' -> closure `%2%'")
% srcPath % drvPath);
return drvPath;
@@ -81,24 +86,23 @@ static void processBinding(EvalState & state, Expr e, StoreExpr & ne,
{
e = evalExpr(state, e);
ATMatcher m;
string s;
ATerm s;
ATermList es;
int n;
Expr e1, e2;
if (atMatch(m, e) >> "Str" >> s) ss.push_back(s);
else if (atMatch(m, e) >> "Uri" >> s) ss.push_back(s);
else if (atMatch(m, e) >> "Bool" >> "True") ss.push_back("1");
else if (atMatch(m, e) >> "Bool" >> "False") ss.push_back("");
if (matchStr(e, s)) ss.push_back(aterm2String(s));
else if (matchUri(e, s)) ss.push_back(aterm2String(s));
else if (e == eTrue) ss.push_back("1");
else if (e == eFalse) ss.push_back("");
else if (atMatch(m, e) >> "Int" >> n) {
else if (matchInt(e, n)) {
ostringstream st;
st << n;
ss.push_back(st.str());
}
else if (atMatch(m, e) >> "Attrs") {
else if (matchAttrs(e, es)) {
Expr a = queryAttr(e, "type");
if (a && evalString(state, a) == "derivation") {
a = queryAttr(e, "drvPath");
@@ -109,37 +113,42 @@ static void processBinding(EvalState & state, Expr e, StoreExpr & ne,
if (!a) throw Error("derivation hash missing");
Hash drvHash = parseHash(evalString(state, a));
state.drvHashes[drvPath] = drvHash;
a = queryAttr(e, "outPath");
if (!a) throw Error("output path missing");
PathSet drvRoots;
drvRoots.insert(evalPath(state, a));
state.drvHashes[drvPath] = drvHash;
state.drvRoots[drvPath] = drvRoots;
ss.push_back(addInput(state, drvPath, ne));
} else
throw Error("invalid derivation attribute");
}
else if (atMatch(m, e) >> "Path" >> s) {
Path drvPath = copyAtom(state, s);
else if (matchPath(e, s)) {
Path drvPath = copyAtom(state, aterm2String(s));
ss.push_back(addInput(state, drvPath, ne));
}
else if (atMatch(m, e) >> "List" >> es) {
else if (matchList(e, es)) {
for (ATermIterator i(es); i; ++i) {
startNest(nest, lvlVomit, format("processing list element"));
processBinding(state, evalExpr(state, *i), ne, ss);
}
}
else if (atMatch(m, e) >> "Null") ss.push_back("");
else if (matchNull(e)) ss.push_back("");
else if (atMatch(m, e) >> "SubPath" >> e1 >> e2) {
else if (matchSubPath(e, e1, e2)) {
Strings ss2;
processBinding(state, evalExpr(state, e1), ne, ss2);
if (ss2.size() != 1)
throw Error("left-hand side of `~' operator cannot be a list");
e2 = evalExpr(state, e2);
if (!(atMatch(m, e2) >> "Str" >> s ||
(atMatch(m, e2) >> "Path" >> s)))
if (!(matchStr(e2, s) || matchPath(e2, s)))
throw Error("right-hand side of `~' operator must be a path or string");
ss.push_back(canonPath(ss2.front() + "/" + s));
ss.push_back(canonPath(ss2.front() + "/" + aterm2String(s)));
}
else throw Error("invalid derivation attribute");
@@ -158,11 +167,19 @@ static string concatStrings(const Strings & ss)
}
Expr primDerivation(EvalState & state, Expr args)
/* Construct (as a unobservable side effect) a Nix derivation
expression that performs the derivation described by the argument
set. Returns the original set extended with the following
attributes: `outPath' containing the primary output path of the
derivation; `drvPath' containing the path of the Nix expression;
and `type' set to `derivation' to indicate that this is a
derivation. */
static Expr primDerivation(EvalState & state, const ATermVector & _args)
{
startNest(nest, lvlVomit, "evaluating derivation");
ATermMap attrs;
Expr args = _args[0];
args = evalExpr(state, args);
queryAllAttrs(args, attrs, true);
@@ -171,7 +188,6 @@ Expr primDerivation(EvalState & state, Expr args)
ne.type = StoreExpr::neDerivation;
string drvName;
Path outPath;
Hash outHash;
bool outHashGiven = false;
@@ -180,15 +196,14 @@ Expr primDerivation(EvalState & state, Expr args)
ATerm value;
Expr pos;
ATerm rhs = attrs.get(key);
ATMatcher m;
if (!(atMatch(m, rhs) >> "" >> value >> pos)) abort();
if (!matchAttrRHS(rhs, value, pos)) abort();
startNest(nest, lvlVomit, format("processing attribute `%1%'") % key);
Strings ss;
try {
processBinding(state, value, ne, ss);
} catch (Error & e) {
throw Error(format("while processing derivation attribute `%1%' at %2%:\n%3%")
throw Error(format("while processing the derivation attribute `%1%' at %2%:\n%3%")
% key % showPos(pos) % e.msg());
}
@@ -207,7 +222,6 @@ Expr primDerivation(EvalState & state, Expr args)
if (key == "builder") ne.derivation.builder = s;
else if (key == "system") ne.derivation.platform = s;
else if (key == "name") drvName = s;
else if (key == "outPath") outPath = s;
else if (key == "id") {
outHash = parseHash(s);
outHashGiven = true;
@@ -222,14 +236,27 @@ Expr primDerivation(EvalState & state, Expr args)
throw Error("required attribute `system' missing");
if (drvName == "")
throw Error("required attribute `name' missing");
/* Check the derivation name. It shouldn't contain whitespace,
but we are conservative here: we check whether only
alphanumerics and some other characters appear. */
string validChars = "+-._?=";
for (string::iterator i = drvName.begin(); i != drvName.end(); ++i)
if (!((*i >= 'A' && *i <= 'Z') ||
(*i >= 'a' && *i <= 'z') ||
(*i >= '0' && *i <= '9') ||
validChars.find(*i) != string::npos))
{
throw Error(format("invalid character `%1%' in derivation name `%2%'")
% *i % drvName);
}
/* Determine the output path. */
/* Determine the output path by hashing the Nix expression with no
outputs to produce a unique but deterministic path name for
this derivation. */
if (!outHashGiven) outHash = hashDerivation(state, ne);
if (outPath == "")
/* Hash the Nix expression with no outputs to produce a
unique but deterministic path name for this derivation. */
outPath = canonPath(nixStore + "/" +
((string) outHash).c_str() + "-" + drvName);
Path outPath = canonPath(nixStore + "/" +
((string) outHash).c_str() + "-" + drvName);
ne.derivation.env["out"] = outPath;
ne.derivation.outputs.insert(outPath);
@@ -242,56 +269,91 @@ Expr primDerivation(EvalState & state, Expr args)
printMsg(lvlChatty, format("instantiated `%1%' -> `%2%'")
% drvName % drvPath);
attrs.set("outPath", ATmake("(Path(<str>), NoPos)", outPath.c_str()));
attrs.set("drvPath", ATmake("(Path(<str>), NoPos)", drvPath.c_str()));
attrs.set("drvHash", ATmake("(Str(<str>), NoPos)", ((string) drvHash).c_str()));
attrs.set("type", ATmake("(Str(\"derivation\"), NoPos)"));
attrs.set("outPath", makeAttrRHS(makePath(toATerm(outPath)), makeNoPos()));
attrs.set("drvPath", makeAttrRHS(makePath(toATerm(drvPath)), makeNoPos()));
attrs.set("drvHash",
makeAttrRHS(makeStr(toATerm((string) drvHash)), makeNoPos()));
attrs.set("type", makeAttrRHS(makeStr(toATerm("derivation")), makeNoPos()));
return makeAttrs(attrs);
}
Expr primBaseNameOf(EvalState & state, Expr arg)
/* Return the base name of the given string, i.e., everything
following the last slash. */
static Expr primBaseNameOf(EvalState & state, const ATermVector & args)
{
string s = evalString(state, arg);
return ATmake("Str(<str>)", baseNameOf(s).c_str());
return makeStr(toATerm(baseNameOf(evalString(state, args[0]))));
}
Expr primToString(EvalState & state, Expr arg)
/* Convert the argument (which can be a path or a uri) to a string. */
static Expr primToString(EvalState & state, const ATermVector & args)
{
arg = evalExpr(state, arg);
ATMatcher m;
string s;
if (atMatch(m, arg) >> "Str" >> s ||
atMatch(m, arg) >> "Path" >> s ||
atMatch(m, arg) >> "Uri" >> s)
return ATmake("Str(<str>)", s.c_str());
Expr arg = evalExpr(state, args[0]);
ATerm s;
if (matchStr(arg, s) || matchPath(arg, s) || matchUri(arg, s))
return makeStr(s);
else throw Error("cannot coerce value to string");
}
Expr primTrue(EvalState & state)
/* Boolean constructors. */
static Expr primTrue(EvalState & state, const ATermVector & args)
{
return ATmake("Bool(True)");
return eTrue;
}
Expr primFalse(EvalState & state)
static Expr primFalse(EvalState & state, const ATermVector & args)
{
return ATmake("Bool(False)");
return eFalse;
}
Expr primNull(EvalState & state)
/* Return the null value. */
Expr primNull(EvalState & state, const ATermVector & args)
{
return ATmake("Null");
return makeNull();
}
Expr primIsNull(EvalState & state, Expr arg)
/* Determine whether the argument is the null value. */
Expr primIsNull(EvalState & state, const ATermVector & args)
{
arg = evalExpr(state, arg);
ATMatcher m;
return makeBool(atMatch(m, arg) >> "Null");
return makeBool(matchNull(evalExpr(state, args[0])));
}
/* Apply a function to every element of a list. */
Expr primMap(EvalState & state, const ATermVector & args)
{
Expr fun = evalExpr(state, args[0]);
Expr list = evalExpr(state, args[1]);
ATermList list2;
if (!matchList(list, list2))
throw Error("`map' expects a list as its second argument");
ATermList list3 = ATempty;
for (ATermIterator i(list2); i; ++i)
list3 = ATinsert(list3, makeCall(fun, *i));
return makeList(ATreverse(list3));
}
void EvalState::addPrimOps()
{
addPrimOp("true", 0, primTrue);
addPrimOp("false", 0, primFalse);
addPrimOp("null", 0, primNull);
addPrimOp("import", 1, primImport);
addPrimOp("derivation", 1, primDerivation);
addPrimOp("baseNameOf", 1, primBaseNameOf);
addPrimOp("toString", 1, primToString);
addPrimOp("isNull", 1, primIsNull);
addPrimOp("map", 2, primMap);
}
-38
View File
@@ -1,38 +0,0 @@
#ifndef __PRIMOPS_H
#define __PRIMOPS_H
#include "eval.hh"
/* Load and evaluate an expression from path specified by the
argument. */
Expr primImport(EvalState & state, Expr arg);
/* Construct (as a unobservable side effect) a Nix derivation
expression that performs the derivation described by the argument
set. Returns the original set extended with the following
attributes: `outPath' containing the primary output path of the
derivation; `drvPath' containing the path of the Nix expression;
and `type' set to `derivation' to indicate that this is a
derivation. */
Expr primDerivation(EvalState & state, Expr args);
/* Return the base name of the given string, i.e., everything
following the last slash. */
Expr primBaseNameOf(EvalState & state, Expr arg);
/* Convert the argument (which can be a path or a uri) to a string. */
Expr primToString(EvalState & state, Expr arg);
/* Boolean constructors. */
Expr primTrue(EvalState & state);
Expr primFalse(EvalState & state);
/* Return the null value. */
Expr primNull(EvalState & state);
/* Determine whether the argument is the null value. */
Expr primIsNull(EvalState & state, Expr arg);
#endif /* !__PRIMOPS_H */
+207 -19
View File
@@ -5,6 +5,9 @@
#include <sys/stat.h>
#include <unistd.h>
#include <pwd.h>
#include <grp.h>
extern "C" {
#include <aterm2.h>
}
@@ -15,9 +18,15 @@ extern "C" {
#include "config.h"
volatile sig_atomic_t blockInt = 0;
void sigintHandler(int signo)
{
_isInterrupted = 1;
if (!blockInt) {
_isInterrupted = 1;
blockInt = 1;
}
}
@@ -46,27 +55,30 @@ void checkStoreNotSymlink(Path path)
}
void initStoreExprHelpers();
/* Initialize and reorder arguments, then call the actual argument
processor. */
static void initAndRun(int argc, char * * argv)
{
char * root = getenv("NIX_ROOT");
if (root) {
if (chroot(root) != 0)
string root = getEnv("NIX_ROOT");
if (root != "") {
if (chroot(root.c_str()) != 0)
throw SysError(format("changing root to `%1%'") % root);
}
/* Setup Nix paths. */
nixStore = canonPath(NIX_STORE_DIR);
nixDataDir = canonPath(NIX_DATA_DIR);
nixLogDir = canonPath(NIX_LOG_DIR);
nixStateDir = canonPath(NIX_STATE_DIR);
nixDBPath = canonPath(NIX_STATE_DIR) + "/db";
nixStore = getEnv("NIX_STORE_DIR", canonPath(NIX_STORE_DIR));
nixDataDir = getEnv("NIX_DATA_DIR", canonPath(NIX_DATA_DIR));
nixLogDir = getEnv("NIX_LOG_DIR", canonPath(NIX_LOG_DIR));
nixStateDir = getEnv("NIX_STATE_DIR", canonPath(NIX_STATE_DIR));
nixDBPath = getEnv("NIX_DB_DIR", nixStateDir + "/db");
/* Check that the store directory and its parent are not
symlinks. */
checkStoreNotSymlink(nixStore);
if (getEnv("NIX_IGNORE_SYMLINK_STORE") != "1")
checkStoreNotSymlink(nixStore);
/* Catch SIGINT. */
struct sigaction act, oact;
@@ -76,10 +88,25 @@ static void initAndRun(int argc, char * * argv)
if (sigaction(SIGINT, &act, &oact))
throw SysError("installing handler for SIGINT");
/* Process the NIX_LOG_TYPE environment variable. */
char * lt = getenv("NIX_LOG_TYPE");
if (lt) setLogType(lt);
/* Ignore SIGPIPE. */
act.sa_handler = SIG_IGN;
act.sa_flags = 0;
if (sigaction(SIGPIPE, &act, &oact))
throw SysError("ignoring SIGPIPE");
/* There is no privacy in the Nix system ;-) At least not for
now. In particular, store objects should be readable by
everybody. This prevents nasty surprises when using a shared
store (with the setuid() hack). */
umask(0022);
/* Process the NIX_LOG_TYPE environment variable. */
string lt = getEnv("NIX_LOG_TYPE");
if (lt != "") setLogType(lt);
/* ATerm stuff. !!! find a better place to put this */
initStoreExprHelpers();
/* Put the arguments in a vector. */
Strings args, remaining;
while (argc--) args.push_back(*argv++);
@@ -112,16 +139,35 @@ static void initAndRun(int argc, char * * argv)
++i;
if (i == args.end()) throw UsageError("`--log-type' requires an argument");
setLogType(*i);
} else if (arg == "--build-output" || arg == "-B")
buildVerbosity = lvlError; /* lowest */
}
else if (arg == "--build-output" || arg == "-B")
; /* !!! obsolete - remove eventually */
else if (arg == "--no-build-output" || arg == "-Q")
buildVerbosity = lvlVomit;
else if (arg == "--help") {
printHelp();
return;
} else if (arg == "--version") {
}
else if (arg == "--version") {
cout << format("%1% (Nix) %2%") % programId % NIX_VERSION << endl;
return;
} else if (arg == "--keep-failed" || arg == "-K")
}
else if (arg == "--keep-failed" || arg == "-K")
keepFailed = true;
else if (arg == "--keep-going" || arg == "-k")
keepGoing = true;
else if (arg == "--fallback")
tryFallback = true;
else if (arg == "--max-jobs" || arg == "-j") {
++i;
if (i == args.end()) throw UsageError("`--max-jobs' requires an argument");
int n;
if (!string2Int(*i, n) || n < 0)
throw UsageError(format("`--max-jobs' requires a non-negative integer"));
maxBuildJobs = n;
}
else if (arg == "--readonly-mode")
readOnlyMode = true;
else remaining.push_back(arg);
}
@@ -129,10 +175,142 @@ static void initAndRun(int argc, char * * argv)
}
static bool haveSwitched;
static uid_t savedUid, nixUid;
static gid_t savedGid, nixGid;
#if HAVE_SETRESUID
#define _setuid(uid) setresuid(uid, uid, savedUid)
#define _setgid(gid) setresgid(gid, gid, savedGid)
#else
/* Only works properly when run by root. */
#define _setuid(uid) setuid(uid)
#define _setgid(gid) setgid(gid)
#endif
SwitchToOriginalUser::SwitchToOriginalUser()
{
#if SETUID_HACK && HAVE_SETRESUID
/* Temporarily switch the effective uid/gid back to the saved
uid/gid (which is the uid/gid of the user that executed the Nix
program; it's *not* the real uid/gid, since we changed that to
the Nix user in switchToNixUser()). */
if (haveSwitched) {
if (setuid(savedUid) == -1)
throw SysError(format("temporarily restoring uid to `%1%'") % savedUid);
if (setgid(savedGid) == -1)
throw SysError(format("temporarily restoring gid to `%1%'") % savedGid);
}
#endif
}
SwitchToOriginalUser::~SwitchToOriginalUser()
{
#if SETUID_HACK && HAVE_SETRESUID
/* Switch the effective uid/gid back to the Nix user. */
if (haveSwitched) {
if (setuid(nixUid) == -1)
throw SysError(format("restoring uid to `%1%'") % nixUid);
if (setgid(nixGid) == -1)
throw SysError(format("restoring gid to `%1%'") % nixGid);
}
#endif
}
void switchToNixUser()
{
#if SETUID_HACK
/* Don't do anything if this is not a setuid binary. */
if (getuid() == geteuid() && getgid() == getegid()) return;
/* Here we set the uid and gid to the Nix user and group,
respectively, IF the current (real) user is a member of the Nix
group. Otherwise we just drop all privileges. */
/* Lookup the Nix gid. */
struct group * gr = getgrnam(NIX_GROUP);
if (!gr) {
cerr << format("missing group `%1%'\n") % NIX_GROUP;
exit(1);
}
/* Get the supplementary group IDs for the current user. */
int maxGids = 512, nrGids;
gid_t gids[maxGids];
if ((nrGids = getgroups(maxGids, gids)) == -1) {
cerr << format("unable to query gids\n");
exit(1);
}
/* !!! Apparently it is unspecified whether getgroups() includes
the effective gid. In that case the following test is always
true *if* the program is installed setgid (which we do when we
have setresuid()). On Linux this doesn't appear to be the
case, but we should switch to the real gid before doing this
test, and then switch back to the saved gid. */
/* Check that the current user is a member of the Nix group. */
bool found = false;
for (int i = 0; i < nrGids; ++i)
if (gids[i] == gr->gr_gid) {
found = true;
break;
}
if (!found) {
/* Not in the Nix group - drop all root/Nix privileges. */
_setgid(getgid());
_setuid(getuid());
return;
}
savedUid = getuid();
savedGid = getgid();
/* Set the real, effective and saved gids to gr->gr_gid. Also
make very sure that this succeeded. We switch the gid first
because we cannot do it after we have dropped root uid. */
nixGid = gr->gr_gid;
if (_setgid(nixGid) != 0 || getgid() != nixGid || getegid() != nixGid) {
cerr << format("unable to set gid to `%1%'\n") % NIX_GROUP;
exit(1);
}
/* Lookup the Nix uid. */
struct passwd * pw = getpwnam(NIX_USER);
if (!pw) {
cerr << format("missing user `%1%'\n") % NIX_USER;
exit(1);
}
/* This will drop all root privileges, setting the real, effective
and saved uids to pw->pw_uid. Also make very sure that this
succeeded.*/
nixUid = pw->pw_uid;
if (_setuid(nixUid) != 0 || getuid() != nixUid || geteuid() != nixUid) {
cerr << format("unable to set uid to `%1%'\n") % NIX_USER;
exit(1);
}
haveSwitched = true;
#endif
}
static char buf[1024];
int main(int argc, char * * argv)
{
/* If we are setuid root, we have to get rid of the excess
privileges ASAP. */
switchToNixUser();
/* ATerm setup. */
ATerm bottomOfStack;
ATinit(argc, argv, &bottomOfStack);
@@ -143,7 +321,17 @@ int main(int argc, char * * argv)
#endif
try {
initAndRun(argc, argv);
try {
initAndRun(argc, argv);
} catch (...) {
/* Subtle: we have to make sure that any `interrupted'
condition is discharged before we reach printMsg()
below, since otherwise it will throw an (uncaught)
exception. */
blockInt = 1; /* ignore further SIGINTs */
_isInterrupted = 0;
throw;
}
} catch (UsageError & e) {
printMsg(lvlError,
format(
+10 -2
View File
@@ -2,9 +2,17 @@ noinst_LIBRARIES = libstore.a
libstore_a_SOURCES = \
store.cc store.hh storeexpr.cc storeexpr.hh \
normalise.cc normalise.hh exec.cc exec.hh \
normalise.cc misc.cc normalise.hh \
globals.cc globals.hh db.cc db.hh \
references.cc references.hh pathlocks.cc pathlocks.hh
references.cc references.hh pathlocks.cc pathlocks.hh \
gc.cc gc.hh storeexpr-ast.hh
EXTRA_DIST = storeexpr-ast.def storeexpr-ast.cc
AM_CXXFLAGS = -Wall \
-I.. ${bdb_include} ${aterm_include} -I../libutil
storeexpr-ast.cc storeexpr-ast.hh: ../aterm-helper.pl storeexpr-ast.def
$(perl) ../aterm-helper.pl storeexpr-ast.hh storeexpr-ast.cc < storeexpr-ast.def
storeexpr.cc: storeexpr-ast.hh
+45 -45
View File
@@ -1,6 +1,7 @@
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <errno.h>
#include <memory>
@@ -81,12 +82,16 @@ void Transaction::moveTo(Transaction & t)
void Database::requireEnv()
{
checkInterrupt();
if (!env) throw Error("database environment not open");
if (!env)throw Error("database environment is not open "
"(maybe you don't have sufficient permission?)");
}
Db * Database::getDb(TableId table)
{
if (table == 0)
throw Error("database table is not open "
"(maybe you don't have sufficient permission?)");
map<TableId, Db *>::iterator i = tables.find(table);
if (i == tables.end())
throw Error("unknown table id");
@@ -146,6 +151,12 @@ void openEnv(DbEnv * env, const string & path, u_int32_t flags)
}
static int my_fsync(int fd)
{
return 0;
}
void Database::open(const string & path)
{
if (env) throw Error(format("environment already open"));
@@ -159,10 +170,31 @@ void Database::open(const string & path)
DbEnv * env = 0; /* !!! close on error */
env = new DbEnv(0);
/* Smaller log files. */
env->set_lg_bsize(32 * 1024); /* default */
env->set_lg_max(256 * 1024); /* must be > 4 * lg_bsize */
env->set_lk_detect(DB_LOCK_DEFAULT);
/* Write the log, but don't sync. This protects transactions
against application crashes, but if the system crashes,
some transactions may be undone. An acceptable risk, I
think. */
env->set_flags(DB_TXN_WRITE_NOSYNC | DB_LOG_AUTOREMOVE, 1);
/* Increase the locking limits. If you ever get `Dbc::get:
Cannot allocate memory' or similar, especially while
running `nix-store --verify', just increase the following
number, then run db_recover on the database to remove the
existing DB environment (since changes only take effect on
new environments). */
env->set_lk_max_locks(4000);
env->set_lk_max_lockers(4000);
env->set_lk_max_objects(4000);
env->set_lk_detect(DB_LOCK_DEFAULT);
/* Dangerous, probably, but from the docs it *seems* that BDB
shouldn't sync when DB_TXN_WRITE_NOSYNC is used, but it
still fsync()s sometimes. */
db_env_set_func_fsync(my_fsync);
/* The following code provides automatic recovery of the
@@ -183,7 +215,11 @@ void Database::open(const string & path)
string accessorsPath = path + "/accessor_count";
fdAccessors = ::open(accessorsPath.c_str(), O_RDWR | O_CREAT, 0666);
if (fdAccessors == -1)
throw SysError(format("opening file `%1%'") % accessorsPath);
if (errno == EACCES)
throw DbNoPermission(
format("permission denied to database in `%1%'") % accessorsPath);
else
throw SysError(format("opening file `%1%'") % accessorsPath);
/* Open the lock file. */
string lockPath = path + "/access_lock";
@@ -337,32 +373,9 @@ bool Database::queryStrings(const Transaction & txn, TableId table,
const string & key, Strings & data)
{
string d;
if (!queryString(txn, table, key, d))
return false;
string::iterator it = d.begin();
while (it != d.end()) {
if (it + 4 > d.end())
throw Error(format("short db entry: `%1%'") % d);
unsigned int len;
len = (unsigned char) *it++;
len |= ((unsigned char) *it++) << 8;
len |= ((unsigned char) *it++) << 16;
len |= ((unsigned char) *it++) << 24;
if (it + len > d.end())
throw Error(format("short db entry: `%1%'") % d);
string s;
while (len--) s += *it++;
data.push_back(s);
}
data = unpackStrings(d);
return true;
}
@@ -381,25 +394,12 @@ void Database::setString(const Transaction & txn, TableId table,
void Database::setStrings(const Transaction & txn, TableId table,
const string & key, const Strings & data)
const string & key, const Strings & data, bool deleteEmpty)
{
string d;
for (Strings::const_iterator it = data.begin();
it != data.end(); it++)
{
string s = *it;
unsigned int len = s.size();
d += len & 0xff;
d += (len >> 8) & 0xff;
d += (len >> 16) & 0xff;
d += (len >> 24) & 0xff;
d += s;
}
setString(txn, table, key, d);
if (deleteEmpty && data.size() == 0)
delPair(txn, table, key);
else
setString(txn, table, key, packStrings(data));
}
+9 -1
View File
@@ -76,7 +76,8 @@ public:
const string & key, const string & data);
void setStrings(const Transaction & txn, TableId table,
const string & key, const Strings & data);
const string & key, const Strings & data,
bool deleteEmpty = true);
void delPair(const Transaction & txn, TableId table,
const string & key);
@@ -86,4 +87,11 @@ public:
};
class DbNoPermission : public Error
{
public:
DbNoPermission(const format & f) : Error(f) { };
};
#endif /* !__DB_H */
-140
View File
@@ -1,140 +0,0 @@
#include <iostream>
#include <cstdio>
#include <sys/types.h>
#include <sys/stat.h>
#include <sys/wait.h>
#include <unistd.h>
#include <fcntl.h>
#include "exec.hh"
#include "util.hh"
#include "globals.hh"
static string pathNullDevice = "/dev/null";
/* Run a program. */
void runProgram(const string & program,
const Strings & args, Environment env,
const string & logFileName)
{
/* Create a log file. */
string logCommand =
verbosity >= buildVerbosity
? "tee " + logFileName + " >&2"
: "cat > " + logFileName;
/* !!! auto-pclose on exit */
FILE * logFile = popen(logCommand.c_str(), "w"); /* !!! escaping */
if (!logFile)
throw SysError(format("creating log file `%1%'") % logFileName);
/* Create a temporary directory where the build will take
place. */
Path tmpDir = createTempDir();
AutoDelete delTmpDir(tmpDir);
/* For convenience, set an environment pointing to the top build
directory. */
env["NIX_BUILD_TOP"] = tmpDir;
/* Also set TMPDIR and variants to point to this directory. */
env["TMPDIR"] = tmpDir;
env["TEMPDIR"] = tmpDir;
env["TMP"] = tmpDir;
env["TEMP"] = tmpDir;
/* Fork a child to build the package. */
pid_t pid;
switch (pid = fork()) {
case -1:
throw SysError("unable to fork");
case 0:
try { /* child */
if (chdir(tmpDir.c_str()) == -1)
throw SysError(format("changing into to `%1%'") % tmpDir);
/* Fill in the arguments. */
const char * argArr[args.size() + 2];
const char * * p = argArr;
string progName = baseNameOf(program);
*p++ = progName.c_str();
for (Strings::const_iterator i = args.begin();
i != args.end(); i++)
*p++ = i->c_str();
*p = 0;
/* Fill in the environment. */
Strings envStrs;
const char * envArr[env.size() + 1];
p = envArr;
for (Environment::const_iterator i = env.begin();
i != env.end(); i++)
*p++ = envStrs.insert(envStrs.end(),
i->first + "=" + i->second)->c_str();
*p = 0;
/* Dup the log handle into stderr. */
if (dup2(fileno(logFile), STDERR_FILENO) == -1)
throw SysError("cannot pipe standard error into log file");
/* Dup stderr to stdin. */
if (dup2(STDERR_FILENO, STDOUT_FILENO) == -1)
throw SysError("cannot dup stderr into stdout");
/* Reroute stdin to /dev/null. */
int fdDevNull = open(pathNullDevice.c_str(), O_RDWR);
if (fdDevNull == -1)
throw SysError(format("cannot open `%1%'") % pathNullDevice);
if (dup2(fdDevNull, STDIN_FILENO) == -1)
throw SysError("cannot dup null device into stdin");
/* Execute the program. This should not return. */
execve(program.c_str(), (char * *) argArr, (char * *) envArr);
throw SysError(format("unable to execute %1%") % program);
} catch (exception & e) {
cerr << format("build error: %1%\n") % e.what();
}
_exit(1);
}
/* parent */
/* Close the logging pipe. Note that this should not cause
the logger to exit until builder exits (because the latter
has an open file handle to the former). */
pclose(logFile);
/* Wait for the child to finish. */
int status;
if (waitpid(pid, &status, 0) != pid)
throw Error("unable to wait for child");
checkInterrupt();
if (!WIFEXITED(status) || WEXITSTATUS(status) != 0) {
if (keepFailed) {
printMsg(lvlTalkative,
format("program `%1%' failed; keeping build directory `%2%'")
% program % tmpDir);
delTmpDir.cancel();
}
if (WIFEXITED(status))
throw Error(format("program `%1%' failed with exit code %2%")
% program % WEXITSTATUS(status));
else if (WIFSIGNALED(status))
throw Error(format("program `%1%' failed due to signal %2%")
% program % WTERMSIG(status));
else
throw Error(format("program `%1%' died abnormally") % program);
}
}
-22
View File
@@ -1,22 +0,0 @@
#ifndef __EXEC_H
#define __EXEC_H
#include <string>
#include <map>
#include "util.hh"
using namespace std;
/* A Unix environment is a mapping from strings to strings. */
typedef map<string, string> Environment;
/* Run a program. */
void runProgram(const string & program,
const Strings & args, Environment env,
const string & logFileName);
#endif /* !__EXEC_H */
+98
View File
@@ -0,0 +1,98 @@
#include "normalise.hh"
#include "globals.hh"
#include <sys/types.h>
#include <sys/stat.h>
#include <unistd.h>
void followLivePaths(Path nePath, PathSet & live)
{
/* Just to be sure, canonicalise the path. It is important to do
this here and in findDeadPath() to ensure that a live path is
not mistaken for a dead path due to some non-canonical
representation. */
nePath = canonPath(nePath);
if (live.find(nePath) != live.end()) return;
live.insert(nePath);
startNest(nest, lvlDebug, format("following `%1%'") % nePath);
assertStorePath(nePath);
if (isValidPath(nePath)) {
/* !!! should make sure that no substitutes are used */
StoreExpr ne = storeExprFromPath(nePath);
/* !!! painfully similar to requisitesWorker() */
if (ne.type == StoreExpr::neClosure)
for (ClosureElems::iterator i = ne.closure.elems.begin();
i != ne.closure.elems.end(); ++i)
{
Path p = canonPath(i->first);
if (live.find(p) == live.end()) {
debug(format("found live `%1%'") % p);
assertStorePath(p);
live.insert(p);
}
}
else if (ne.type == StoreExpr::neDerivation)
for (PathSet::iterator i = ne.derivation.inputs.begin();
i != ne.derivation.inputs.end(); ++i)
followLivePaths(*i, live);
else abort();
}
Path nfPath;
if (querySuccessor(nePath, nfPath))
followLivePaths(nfPath, live);
}
PathSet findLivePaths(const Paths & roots)
{
PathSet live;
startNest(nest, lvlDebug, "finding live paths");
for (Paths::const_iterator i = roots.begin(); i != roots.end(); ++i)
followLivePaths(*i, live);
return live;
}
PathSet findDeadPaths(const PathSet & live, time_t minAge)
{
PathSet dead;
startNest(nest, lvlDebug, "finding dead paths");
time_t now = time(0);
Strings storeNames = readDirectory(nixStore);
for (Strings::iterator i = storeNames.begin(); i != storeNames.end(); ++i) {
Path p = canonPath(nixStore + "/" + *i);
if (minAge > 0) {
struct stat st;
if (lstat(p.c_str(), &st) != 0)
throw SysError(format("obtaining information about `%1%'") % p);
if (st.st_atime + minAge >= now) continue;
}
if (live.find(p) == live.end()) {
debug(format("dead path `%1%'") % p);
dead.insert(p);
} else
debug(format("live path `%1%'") % p);
}
return dead;
}
+26
View File
@@ -0,0 +1,26 @@
#ifndef __GC_H
#define __GC_H
#include "storeexpr.hh"
/* Determine the set of "live" store paths, given a set of root store
expressions. The live store paths are those that are reachable
from the roots. The roots are reachable by definition. Any path
mentioned in a reachable store expression is also reachable. If a
derivation store expression is reachable, then its successor (if it
exists) if also reachable. It is not an error for store
expressions not to exist (since this can happen on derivation store
expressions, for instance, due to the substitute mechanism), but
successor links are followed even for non-existant derivations. */
PathSet findLivePaths(const Paths & roots);
/* Given a set of "live" store paths, determine the set of "dead"
store paths (which are simply all store paths that are not in the
live set). The value `minAge' specifies the minimum age in seconds
for an unreachable file to be considered dead (0 meaning that any
unreachable file is dead). */
PathSet findDeadPaths(const PathSet & live, time_t minAge);
#endif /* !__GC_H */
+9 -1
View File
@@ -8,4 +8,12 @@ string nixDBPath = "/UNINIT";
bool keepFailed = false;
Verbosity buildVerbosity = lvlDebug;
bool keepGoing = false;
bool tryFallback = false;
Verbosity buildVerbosity = lvlInfo;
unsigned int maxBuildJobs = 1;
bool readOnlyMode = false;
+15
View File
@@ -29,8 +29,23 @@ extern string nixDBPath;
/* Whether to keep temporary directories of failed builds. */
extern bool keepFailed;
/* Whether to keep building subgoals when a sibling (another subgoal
of the same goal) fails. */
extern bool keepGoing;
/* Whether, if we cannot realise the known closure corresponding to a
derivation, we should try to normalise the derivation instead. */
extern bool tryFallback;
/* Verbosity level for build output. */
extern Verbosity buildVerbosity;
/* Maximum number of parallel build jobs. 0 means unlimited. */
extern unsigned int maxBuildJobs;
/* Read-only mode. Don't copy stuff to the store, don't change the
database. */
extern bool readOnlyMode;
#endif /* !__GLOBALS_H */
+72
View File
@@ -0,0 +1,72 @@
#include "normalise.hh"
StoreExpr storeExprFromPath(const Path & path)
{
assertStorePath(path);
ensurePath(path);
ATerm t = ATreadFromNamedFile(path.c_str());
if (!t) throw Error(format("cannot read aterm from `%1%'") % path);
return parseStoreExpr(t);
}
PathSet storeExprRoots(const Path & nePath)
{
PathSet paths;
StoreExpr ne = storeExprFromPath(nePath);
if (ne.type == StoreExpr::neClosure)
paths.insert(ne.closure.roots.begin(), ne.closure.roots.end());
else if (ne.type == StoreExpr::neDerivation)
paths.insert(ne.derivation.outputs.begin(),
ne.derivation.outputs.end());
else abort();
return paths;
}
static void requisitesWorker(const Path & nePath,
bool includeExprs, bool includeSuccessors,
PathSet & paths, PathSet & doneSet)
{
checkInterrupt();
if (doneSet.find(nePath) != doneSet.end()) return;
doneSet.insert(nePath);
StoreExpr ne = storeExprFromPath(nePath);
if (ne.type == StoreExpr::neClosure)
for (ClosureElems::iterator i = ne.closure.elems.begin();
i != ne.closure.elems.end(); ++i)
paths.insert(i->first);
else if (ne.type == StoreExpr::neDerivation)
for (PathSet::iterator i = ne.derivation.inputs.begin();
i != ne.derivation.inputs.end(); ++i)
requisitesWorker(*i,
includeExprs, includeSuccessors, paths, doneSet);
else abort();
if (includeExprs) paths.insert(nePath);
Path nfPath;
if (includeSuccessors && querySuccessor(nePath, nfPath))
requisitesWorker(nfPath, includeExprs, includeSuccessors,
paths, doneSet);
}
PathSet storeExprRequisites(const Path & nePath,
bool includeExprs, bool includeSuccessors)
{
PathSet paths;
PathSet doneSet;
requisitesWorker(nePath, includeExprs, includeSuccessors,
paths, doneSet);
return paths;
}
+1781 -227
View File
File diff suppressed because it is too large. Load diff
+10 -14
View File
@@ -8,23 +8,23 @@
derivation, a path containing an equivalent closure expression is
returned. This requires that the derivation is performed, unless a
successor is known. */
Path normaliseStoreExpr(const Path & nePath, PathSet pending = PathSet());
Path normaliseStoreExpr(const Path & nePath);
/* Realise a closure store expression in the file system.
The pending paths are those that are already being realised. This
prevents infinite recursion for paths realised through a substitute
(since when we build the substitute, we would first try to realise
its output paths through substitutes... kaboom!). */
void realiseClosure(const Path & nePath, PathSet pending = PathSet());
/* Realise a store expression. If the expression is a derivation, it
is first normalised into a closure. The closure is then realised
in the file system (i.e., it is ensured that each path in the
closure exists in the file system, if necessary by using the
substitute mechanism). Returns the normal form of the expression
(i.e., its closure expression). */
Path realiseStoreExpr(const Path & nePath);
/* Ensure that a path exists, possibly by instantiating it by
realising a substitute. */
void ensurePath(const Path & path, PathSet pending = PathSet());
void ensurePath(const Path & path);
/* Read a store expression, after ensuring its existence through
ensurePath(). */
StoreExpr storeExprFromPath(const Path & path, PathSet pending = PathSet());
StoreExpr storeExprFromPath(const Path & path);
/* Get the list of root (output) paths of the given store
expression. */
@@ -40,9 +40,5 @@ PathSet storeExprRoots(const Path & nePath);
PathSet storeExprRequisites(const Path & nePath,
bool includeExprs, bool includeSuccessors);
/* Return the list of the paths of all known store expressions whose
output paths are completely contained in the set `outputs'. */
PathSet findGenerators(const PathSet & outputs);
#endif /* !__NORMALISE_H */
+20 -2
View File
@@ -45,9 +45,24 @@ bool lockFile(int fd, LockType lockType, bool wait)
static StringSet lockedPaths; /* !!! not thread-safe */
PathLocks::PathLocks(const PathSet & _paths)
PathLocks::PathLocks()
: deletePaths(false)
{
}
PathLocks::PathLocks(const PathSet & paths)
: deletePaths(false)
{
lockPaths(paths);
}
void PathLocks::lockPaths(const PathSet & _paths)
{
/* May be called only once! */
assert(this->paths.empty());
/* Note that `fds' is built incrementally so that the destructor
will only release those locks that we have already acquired. */
@@ -80,6 +95,8 @@ PathLocks::PathLocks(const PathSet & _paths)
/* Acquire an exclusive lock. */
lockFile(fd, ltWrite, true);
debug(format("lock acquired on `%1%'") % lockPath);
lockedPaths.insert(lockPath);
}
}
@@ -88,7 +105,7 @@ PathLocks::PathLocks(const PathSet & _paths)
PathLocks::~PathLocks()
{
for (list<int>::iterator i = fds.begin(); i != fds.end(); i++)
close(*i);
if (close(*i) != 0) throw SysError("closing fd");
for (Paths::iterator i = paths.begin(); i != paths.end(); i++) {
checkInterrupt();
@@ -99,6 +116,7 @@ PathLocks::~PathLocks()
the lock file is an optimisation, not a necessity. */
}
lockedPaths.erase(*i);
debug(format("lock released on `%1%'") % *i);
}
}
+3 -1
View File
@@ -17,7 +17,9 @@ private:
bool deletePaths;
public:
PathLocks(const PathSet & _paths);
PathLocks();
PathLocks(const PathSet & paths);
void lockPaths(const PathSet & _paths);
~PathLocks();
void setDeletion(bool deletePaths);
};
+202 -95
View File
@@ -21,7 +21,7 @@ static Database nixDB;
The existence of a key $p$ indicates that path $p$ is valid (that
is, produced by a succesful build). */
static TableId dbValidPaths;
static TableId dbValidPaths = 0;
/* dbSuccessors :: Path -> Path
@@ -32,19 +32,21 @@ static TableId dbValidPaths;
Note that a term $y$ is a successor of $x$ iff there exists a
sequence of rewrite steps that rewrites $x$ into $y$.
*/
static TableId dbSuccessors;
static TableId dbSuccessors = 0;
/* dbSuccessorsRev :: Path -> [Path]
The reverse mapping of dbSuccessors (i.e., it stores the
predecessors of a Nix expression).
*/
static TableId dbSuccessorsRev;
static TableId dbSuccessorsRev = 0;
/* dbSubstitutes :: Path -> [Path]
/* dbSubstitutes :: Path -> [(Path, Path, [string])]
Each pair $(p, [ps])$ tells Nix that it can realise any of the
Nix expressions stored at paths $ps$ to produce a path $p$.
Each pair $(p, subs)$ tells Nix that it can use any of the
substitutes in $subs$ to build path $p$. Each substitute is a
tuple $(storeExpr, program, args)$ (see the type `Substitute' in
`store.hh').
The main purpose of this is for distributed caching of derivates.
One system can compute a derivate and put it on a website (as a Nix
@@ -52,18 +54,34 @@ static TableId dbSuccessorsRev;
substitute for that derivate. The substitute in this case might be
a Nix expression that fetches the Nix archive.
*/
static TableId dbSubstitutes;
static TableId dbSubstitutes = 0;
/* dbSubstitutesRev :: Path -> [Path]
The reverse mapping of dbSubstitutes.
The reverse mapping of dbSubstitutes; it maps store expressions
back to the paths for which they are substitutes.
*/
static TableId dbSubstitutesRev;
static TableId dbSubstitutesRev = 0;
bool Substitute::operator == (const Substitute & sub)
{
return storeExpr == sub.storeExpr
&& program == sub.program
&& args == sub.args;
}
void openDB()
{
nixDB.open(nixDBPath);
if (readOnlyMode) return;
try {
nixDB.open(nixDBPath);
} catch (DbNoPermission & e) {
printMsg(lvlTalkative, "cannot access Nix database; continuing anyway");
readOnlyMode = true;
return;
}
dbValidPaths = nixDB.openTable("validpaths");
dbSuccessors = nixDB.openTable("successors");
dbSuccessorsRev = nixDB.openTable("successors-rev");
@@ -116,21 +134,22 @@ void copyPath(const Path & src, const Path & dst)
use a thread). */
/* Create a pipe. */
int fds[2];
if (pipe(fds) == -1) throw SysError("creating pipe");
Pipe pipe;
pipe.create();
/* Fork. */
pid_t pid;
switch (pid = fork()) {
Pid pid;
pid = fork();
switch (pid) {
case -1:
throw SysError("unable to fork");
case 0: /* child */
try {
close(fds[1]);
pipe.writeSide.close();
CopySource source;
source.fd = fds[0];
source.fd = pipe.readSide;
restorePath(dst, source);
_exit(0);
} catch (exception & e) {
@@ -139,21 +158,22 @@ void copyPath(const Path & src, const Path & dst)
_exit(1);
}
close(fds[0]);
/* Parent. */
pipe.readSide.close();
CopySink sink;
sink.fd = fds[1];
dumpPath(src, sink);
sink.fd = pipe.writeSide;
{
SwitchToOriginalUser sw;
dumpPath(src, sink);
}
/* Wait for the child to finish. */
int status;
if (waitpid(pid, &status, 0) != pid)
throw SysError("waiting for child");
if (!WIFEXITED(status) || WEXITSTATUS(status) != 0)
throw Error("cannot copy file: child died");
int status = pid.wait(true);
if (!statusOk(status))
throw Error(format("cannot copy `%1% to `%2%': child %3%")
% src % dst % statusToString(status));
}
@@ -227,6 +247,30 @@ void registerSuccessor(const Transaction & txn,
}
void unregisterSuccessor(const Path & srcPath)
{
assertStorePath(srcPath);
Transaction txn(nixDB);
Path sucPath;
if (!nixDB.queryString(txn, dbSuccessors, srcPath, sucPath)) {
txn.abort();
return;
}
nixDB.delPair(txn, dbSuccessors, srcPath);
Paths revs;
nixDB.queryStrings(txn, dbSuccessorsRev, sucPath, revs);
Paths::iterator i = find(revs.begin(), revs.end(), srcPath);
assert(i != revs.end());
revs.erase(i);
nixDB.setStrings(txn, dbSuccessorsRev, sucPath, revs);
txn.commit();
}
bool querySuccessor(const Path & srcPath, Path & sucPath)
{
return nixDB.queryString(noTxn, dbSuccessors, srcPath, sucPath);
@@ -241,44 +285,97 @@ Paths queryPredecessors(const Path & sucPath)
}
void registerSubstitute(const Path & srcPath, const Path & subPath)
static Substitutes readSubstitutes(const Transaction & txn,
const Path & srcPath)
{
assertStorePath(srcPath);
assertStorePath(subPath);
Strings ss;
nixDB.queryStrings(txn, dbSubstitutes, srcPath, ss);
Substitutes subs;
if (!isValidPathTxn(subPath, noTxn)) throw Error(
format("path `%1%' cannot be a substitute, since it is not valid")
% subPath);
Transaction txn(nixDB);
Paths subs;
nixDB.queryStrings(txn, dbSubstitutes, srcPath, subs);
if (find(subs.begin(), subs.end(), subPath) != subs.end()) {
/* Nothing to do if the substitute is already known. */
txn.abort();
return;
for (Strings::iterator i = ss.begin(); i != ss.end(); ++i) {
if (i->size() < 4 || (*i)[3] != 0) {
/* Old-style substitute. !!! remove this code
eventually? */
break;
}
Strings ss2 = unpackStrings(*i);
if (ss2.size() != 3) throw Error("malformed substitute");
Strings::iterator j = ss2.begin();
Substitute sub;
sub.storeExpr = *j++;
sub.program = *j++;
sub.args = unpackStrings(*j++);
subs.push_back(sub);
}
subs.push_front(subPath); /* new substitutes take precedence */
Paths revs;
nixDB.queryStrings(txn, dbSubstitutesRev, subPath, revs);
if (find(revs.begin(), revs.end(), srcPath) == revs.end())
revs.push_back(srcPath);
nixDB.setStrings(txn, dbSubstitutes, srcPath, subs);
nixDB.setStrings(txn, dbSubstitutesRev, subPath, revs);
txn.commit();
return subs;
}
Paths querySubstitutes(const Path & srcPath)
static void writeSubstitutes(const Transaction & txn,
const Path & srcPath, const Substitutes & subs)
{
Paths subPaths;
nixDB.queryStrings(noTxn, dbSubstitutes, srcPath, subPaths);
return subPaths;
Strings ss;
for (Substitutes::const_iterator i = subs.begin();
i != subs.end(); ++i)
{
Strings ss2;
ss2.push_back(i->storeExpr);
ss2.push_back(i->program);
ss2.push_back(packStrings(i->args));
ss.push_back(packStrings(ss2));
}
nixDB.setStrings(txn, dbSubstitutes, srcPath, ss);
}
typedef map<Path, Paths> SubstitutesRev;
void registerSubstitutes(const Transaction & txn,
const SubstitutePairs & subPairs)
{
SubstitutesRev revMap;
for (SubstitutePairs::const_iterator i = subPairs.begin();
i != subPairs.end(); ++i)
{
const Path & srcPath(i->first);
const Substitute & sub(i->second);
assertStorePath(srcPath);
assertStorePath(sub.storeExpr);
Substitutes subs = readSubstitutes(txn, srcPath);
/* New substitutes take precedence over old ones. If the
substitute is already present, it's moved to the front. */
remove(subs.begin(), subs.end(), sub);
subs.push_front(sub);
writeSubstitutes(txn, srcPath, subs);
Paths & revs = revMap[sub.storeExpr];
if (revs.empty())
nixDB.queryStrings(txn, dbSubstitutesRev, sub.storeExpr, revs);
if (find(revs.begin(), revs.end(), srcPath) == revs.end())
revs.push_back(srcPath);
}
/* Re-write the reverse mapping in one go to prevent Theta(n^2)
performance. (This would occur because the data fields of the
`substitutes-rev' table are lists). */
for (SubstitutesRev::iterator i = revMap.begin(); i != revMap.end(); ++i)
nixDB.setStrings(txn, dbSubstitutesRev, i->first, i->second);
}
Substitutes querySubstitutes(const Path & srcPath)
{
return readSubstitutes(noTxn, srcPath);
}
@@ -291,16 +388,6 @@ void registerValidPath(const Transaction & txn, const Path & _path)
}
static void setOrClearStrings(Transaction & txn,
TableId table, const string & key, const Strings & value)
{
if (value.size() > 0)
nixDB.setStrings(txn, table, key, value);
else
nixDB.delPair(txn, table, key);
}
static void invalidatePath(const Path & path, Transaction & txn)
{
debug(format("unregistering path `%1%'") % path);
@@ -319,12 +406,15 @@ static void invalidatePath(const Path & path, Transaction & txn)
revs.clear();
nixDB.queryStrings(txn, dbSubstitutesRev, path, revs);
for (Paths::iterator i = revs.begin(); i != revs.end(); ++i) {
Paths subs;
nixDB.queryStrings(txn, dbSubstitutes, *i, subs);
if (find(subs.begin(), subs.end(), path) == subs.end())
throw Error("integrity error in substitutes mapping");
subs.remove(path);
setOrClearStrings(txn, dbSubstitutes, *i, subs);
Substitutes subs = readSubstitutes(txn, *i), subs2;
bool found = false;
for (Substitutes::iterator j = subs.begin(); j != subs.end(); ++j)
if (j->storeExpr != path)
subs2.push_back(*j);
else
found = true;
// !!! if (!found) throw Error("integrity error in substitutes mapping");
writeSubstitutes(txn, *i, subs);
/* If path *i now has no substitutes left, and is not valid,
then it too should be invalidated. This is because it may
@@ -341,12 +431,16 @@ Path addToStore(const Path & _srcPath)
Path srcPath(absPath(_srcPath));
debug(format("adding `%1%' to the store") % srcPath);
Hash h = hashPath(srcPath);
Hash h;
{
SwitchToOriginalUser sw;
h = hashPath(srcPath);
}
string baseName = baseNameOf(srcPath);
Path dstPath = canonPath(nixStore + "/" + (string) h + "-" + baseName);
if (!isValidPath(dstPath)) {
if (!readOnlyMode && !isValidPath(dstPath)) {
/* The first check above is an optimisation to prevent
unnecessary lock acquisition. */
@@ -356,8 +450,16 @@ Path addToStore(const Path & _srcPath)
PathLocks outputLock(lockPaths);
if (!isValidPath(dstPath)) {
if (pathExists(dstPath)) deletePath(dstPath);
/* !!! race: srcPath might change between hashPath() and
here! */
copyPath(srcPath, dstPath);
makePathReadOnly(dstPath);
Transaction txn(nixDB);
registerValidPath(txn, dstPath);
txn.commit();
@@ -381,8 +483,13 @@ void addTextToStore(const Path & dstPath, const string & s)
PathLocks outputLock(lockPaths);
if (!isValidPath(dstPath)) {
if (pathExists(dstPath)) deletePath(dstPath);
writeStringToFile(dstPath, s);
makePathReadOnly(dstPath);
Transaction txn(nixDB);
registerValidPath(txn, dstPath);
txn.commit();
@@ -438,28 +545,28 @@ void verifyStore()
/* Check that the values of the substitute mappings are valid
paths. */
Paths subs;
nixDB.enumTable(txn, dbSubstitutes, subs);
for (Paths::iterator i = subs.begin(); i != subs.end(); ++i) {
Paths subPaths, subPaths2;
nixDB.queryStrings(txn, dbSubstitutes, *i, subPaths);
for (Paths::iterator j = subPaths.begin(); j != subPaths.end(); ++j)
if (validPaths.find(*j) == validPaths.end())
Paths subKeys;
nixDB.enumTable(txn, dbSubstitutes, subKeys);
for (Paths::iterator i = subKeys.begin(); i != subKeys.end(); ++i) {
Substitutes subs = readSubstitutes(txn, *i), subs2;
for (Substitutes::iterator j = subs.begin(); j != subs.end(); ++j)
if (validPaths.find(j->storeExpr) == validPaths.end())
printMsg(lvlError,
format("found substitute mapping to non-existent path `%1%'") % *j);
format("found substitute mapping to non-existent path `%1%'")
% j->storeExpr);
else
subPaths2.push_back(*j);
if (subPaths.size() != subPaths2.size())
setOrClearStrings(txn, dbSubstitutes, *i, subPaths2);
if (subPaths2.size() > 0)
subs2.push_back(*j);
if (subs.size() != subs2.size())
writeSubstitutes(txn, *i, subs2);
if (subs2.size() > 0)
usablePaths.insert(*i);
}
/* Check that the keys of the reverse substitute mappings are
valid paths. */
Paths rsubs;
nixDB.enumTable(txn, dbSubstitutesRev, rsubs);
for (Paths::iterator i = rsubs.begin(); i != rsubs.end(); ++i) {
Paths rsubKeys;
nixDB.enumTable(txn, dbSubstitutesRev, rsubKeys);
for (Paths::iterator i = rsubKeys.begin(); i != rsubKeys.end(); ++i) {
if (validPaths.find(*i) == validPaths.end()) {
printMsg(lvlError,
format("found reverse substitute mapping for non-existent path `%1%'") % *i);
@@ -469,9 +576,9 @@ void verifyStore()
/* Check that the values of the successor mappings are usable
paths. */
Paths sucs;
nixDB.enumTable(txn, dbSuccessors, sucs);
for (Paths::iterator i = sucs.begin(); i != sucs.end(); ++i) {
Paths sucKeys;
nixDB.enumTable(txn, dbSuccessors, sucKeys);
for (Paths::iterator i = sucKeys.begin(); i != sucKeys.end(); ++i) {
/* Note that *i itself does not have to be valid, just its
successor. */
Path sucPath;
@@ -486,9 +593,9 @@ void verifyStore()
/* Check that the keys of the reverse successor mappings are valid
paths. */
Paths rsucs;
nixDB.enumTable(txn, dbSuccessorsRev, rsucs);
for (Paths::iterator i = rsucs.begin(); i != rsucs.end(); ++i) {
Paths rsucKeys;
nixDB.enumTable(txn, dbSuccessorsRev, rsucKeys);
for (Paths::iterator i = rsucKeys.begin(); i != rsucKeys.end(); ++i) {
if (usablePaths.find(*i) == usablePaths.end()) {
printMsg(lvlError,
format("found reverse successor mapping for non-existent path `%1%'") % *i);
+30 -2
View File
@@ -9,6 +9,29 @@
using namespace std;
/* A substitute is a program invocation that constructs some store
path (typically by fetching it from somewhere, e.g., from the
network). */
struct Substitute
{
/* Store expression to be normalised and realised in order to
obtain `program'. */
Path storeExpr;
/* Program to be executed to create the store path. Must be in
the output path of `storeExpr'. */
Path program;
/* Extra arguments to be passed to the program (the first argument
is the store path to be substituted). */
Strings args;
bool operator == (const Substitute & sub);
};
typedef list<Substitute> Substitutes;
/* Open the database environment. */
void openDB();
@@ -31,6 +54,9 @@ void copyPath(const Path & src, const Path & dst);
void registerSuccessor(const Transaction & txn,
const Path & srcPath, const Path & sucPath);
/* Remove a successor mapping. */
void unregisterSuccessor(const Path & srcPath);
/* Return the predecessors of the Nix expression stored at the given
path. */
bool querySuccessor(const Path & srcPath, Path & sucPath);
@@ -40,10 +66,12 @@ bool querySuccessor(const Path & srcPath, Path & sucPath);
Paths queryPredecessors(const Path & sucPath);
/* Register a substitute. */
void registerSubstitute(const Path & srcPath, const Path & subPath);
typedef list<pair<Path, Substitute> > SubstitutePairs;
void registerSubstitutes(const Transaction & txn,
const SubstitutePairs & subPairs);
/* Return the substitutes expression for the given path. */
Paths querySubstitutes(const Path & srcPath);
Substitutes querySubstitutes(const Path & srcPath);
/* Register the validity of a path. */
void registerValidPath(const Transaction & txn, const Path & path);
+7
View File
@@ -0,0 +1,7 @@
init initStoreExprHelpers
Closure | ATermList ATermList | ATerm |
Derive | ATermList ATermList string string ATermList ATermList | ATerm |
| string string | ATerm | EnvBinding |
| string ATermList | ATerm | ClosureElem |
+32 -31
View File
@@ -2,6 +2,9 @@
#include "globals.hh"
#include "store.hh"
#include "storeexpr-ast.hh"
#include "storeexpr-ast.cc"
Hash hashTerm(ATerm t)
{
@@ -17,7 +20,7 @@ Path writeTerm(ATerm t, const string & suffix)
Path path = canonPath(nixStore + "/" +
(string) h + suffix + ".store");
if (!isValidPath(path)) {
if (!readOnlyMode && !isValidPath(path)) {
char * s = ATwriteToString(t);
if (!s) throw Error(format("cannot write aterm to `%1%'") % path);
addTextToStore(path, string(s));
@@ -29,10 +32,11 @@ Path writeTerm(ATerm t, const string & suffix)
static void parsePaths(ATermList paths, PathSet & out)
{
ATMatcher m;
for (ATermIterator i(paths); i; ++i) {
string s;
if (!(atMatch(m, *i) >> s))
if (ATgetType(*i) != AT_APPL)
throw badTerm("not a path", *i);
string s = aterm2String(*i);
if (s.size() == 0 || s[0] != '/')
throw badTerm("not a path", *i);
out.insert(s);
}
@@ -69,21 +73,20 @@ static void checkClosure(const Closure & closure)
static bool parseClosure(ATerm t, Closure & closure)
{
ATermList roots, elems;
ATMatcher m;
if (!(atMatch(m, t) >> "Closure" >> roots >> elems))
if (!matchClosure(t, roots, elems))
return false;
parsePaths(roots, closure.roots);
for (ATermIterator i(elems); i; ++i) {
string path;
ATerm path;
ATermList refs;
if (!(atMatch(m, *i) >> "" >> path >> refs))
if (!matchClosureElem(*i, path, refs))
throw badTerm("not a closure element", *i);
ClosureElem elem;
parsePaths(refs, elem.refs);
closure.elems[path] = elem;
closure.elems[aterm2String(path)] = elem;
}
checkClosure(closure);
@@ -93,32 +96,29 @@ static bool parseClosure(ATerm t, Closure & closure)
static bool parseDerivation(ATerm t, Derivation & derivation)
{
ATMatcher m;
ATermList outs, ins, args, bnds;
string builder, platform;
ATerm builder, platform;
if (!(atMatch(m, t) >> "Derive" >> outs >> ins >> platform
>> builder >> args >> bnds))
if (!matchDerive(t, outs, ins, platform, builder, args, bnds))
return false;
parsePaths(outs, derivation.outputs);
parsePaths(ins, derivation.inputs);
derivation.builder = builder;
derivation.platform = platform;
derivation.builder = aterm2String(builder);
derivation.platform = aterm2String(platform);
for (ATermIterator i(args); i; ++i) {
string s;
if (!(atMatch(m, *i) >> s))
if (ATgetType(*i) != AT_APPL)
throw badTerm("string expected", *i);
derivation.args.push_back(s);
derivation.args.push_back(aterm2String(*i));
}
for (ATermIterator i(bnds); i; ++i) {
string s1, s2;
if (!(atMatch(m, *i) >> "" >> s1 >> s2))
ATerm s1, s2;
if (!matchEnvBinding(*i, s1, s2))
throw badTerm("tuple of strings expected", *i);
derivation.env[s1] = s2;
derivation.env[aterm2String(s1)] = aterm2String(s2);
}
return true;
@@ -142,7 +142,7 @@ static ATermList unparsePaths(const PathSet & paths)
ATermList l = ATempty;
for (PathSet::const_iterator i = paths.begin();
i != paths.end(); i++)
l = ATinsert(l, ATmake("<str>", i->c_str()));
l = ATinsert(l, toATerm(*i));
return ATreverse(l);
}
@@ -155,11 +155,11 @@ static ATerm unparseClosure(const Closure & closure)
for (ClosureElems::const_iterator i = closure.elems.begin();
i != closure.elems.end(); i++)
elems = ATinsert(elems,
ATmake("(<str>, <term>)",
i->first.c_str(),
makeClosureElem(
toATerm(i->first),
unparsePaths(i->second.refs)));
return ATmake("Closure(<term>, <term>)", roots, elems);
return makeClosure(roots, elems);
}
@@ -168,20 +168,21 @@ static ATerm unparseDerivation(const Derivation & derivation)
ATermList args = ATempty;
for (Strings::const_iterator i = derivation.args.begin();
i != derivation.args.end(); i++)
args = ATinsert(args, ATmake("<str>", i->c_str()));
args = ATinsert(args, toATerm(*i));
ATermList env = ATempty;
for (StringPairs::const_iterator i = derivation.env.begin();
i != derivation.env.end(); i++)
env = ATinsert(env,
ATmake("(<str>, <str>)",
i->first.c_str(), i->second.c_str()));
makeEnvBinding(
toATerm(i->first),
toATerm(i->second)));
return ATmake("Derive(<term>, <term>, <str>, <str>, <term>, <term>)",
return makeDerive(
unparsePaths(derivation.outputs),
unparsePaths(derivation.inputs),
derivation.platform.c_str(),
derivation.builder.c_str(),
toATerm(derivation.platform),
toATerm(derivation.builder),
ATreverse(args),
ATreverse(env));
}
-162
View File
@@ -1,162 +0,0 @@
#include <iostream>
#include <sys/stat.h>
#include <sys/types.h>
#include "hash.hh"
#include "archive.hh"
#include "util.hh"
#include "normalise.hh"
#include "globals.hh"
void realise(Path nePath)
{
Nest nest(lvlDebug, format("TEST: realising `%1%'") % nePath);
realiseClosure(normaliseNixExpr(nePath));
}
struct MySink : DumpSink
{
virtual void operator () (const unsigned char * data, unsigned int len)
{
/* Don't use cout, it's slow as hell! */
writeFull(STDOUT_FILENO, data, len);
}
};
struct MySource : RestoreSource
{
virtual void operator () (unsigned char * data, unsigned int len)
{
readFull(STDIN_FILENO, data, len);
}
};
void runTests()
{
verbosity = (Verbosity) 100;
/* Hashing. */
string s = "0b0ffd0538622bfe20b92c4aa57254d9";
Hash h = parseHash(s);
if ((string) h != s) abort();
try {
h = parseHash("blah blah");
abort();
} catch (Error err) { };
try {
h = parseHash("0b0ffd0538622bfe20b92c4aa57254d99");
abort();
} catch (Error err) { };
/* Path canonicalisation. */
cout << canonPath("/./../././//") << endl;
cout << canonPath("/foo/bar") << endl;
cout << canonPath("///foo/////bar//") << endl;
cout << canonPath("/././/foo/////bar//.") << endl;
cout << canonPath("/foo////bar//..///x/") << endl;
cout << canonPath("/foo////bar//..//..//x/y/../z/") << endl;
cout << canonPath("/foo/bar/../../../..///") << endl;
/* Dumping. */
#if 0
MySink sink;
dumpPath("scratch", sink);
cout << (string) hashPath("scratch") << endl;
#endif
/* Restoring. */
#if 0
MySource source;
restorePath("outdir", source);
cout << (string) hashPath("outdir") << endl;
return;
#endif
/* Set up the test environment. */
mkdir("scratch", 0777);
mkdir("scratch/db", 0777);
string testDir = absPath("scratch");
cout << testDir << endl;
nixStore = testDir;
nixLogDir = testDir;
nixDBPath = testDir + "/db";
openDB();
initDB();
/* Expression evaluation. */
Path builder1fn;
builder1fn = addToStore("./test-builder-1.sh");
ATerm fs1 = ATmake(
"Closure([<str>], [(<str>, [])])",
builder1fn.c_str(),
builder1fn.c_str());
Path fs1ne = writeTerm(fs1, "-c");
realise(fs1ne);
realise(fs1ne);
string out1h = hashString("foo"); /* !!! bad */
Path out1fn = nixStore + "/" + (string) out1h + "-hello.txt";
ATerm fs3 = ATmake(
"Derive([<str>], [<str>], <str>, <str>, [], [(\"out\", <str>)])",
out1fn.c_str(),
fs1ne.c_str(),
thisSystem.c_str(),
builder1fn.c_str(),
out1fn.c_str());
debug(printTerm(fs3));
Path fs3ne = writeTerm(fs3, "-d");
realise(fs3ne);
realise(fs3ne);
Path builder4fn = addToStore("./test-builder-2.sh");
ATerm fs4 = ATmake(
"Closure([<str>], [(<str>, [])])",
builder4fn.c_str(),
builder4fn.c_str());
Path fs4ne = writeTerm(fs4, "-c");
realise(fs4ne);
string out5h = hashString("bar"); /* !!! bad */
Path out5fn = nixStore + "/" + (string) out5h + "-hello2";
ATerm fs5 = ATmake(
"Derive([<str>], [<str>], <str>, <str>, [], [(\"out\", <str>), (\"builder\", <str>)])",
out5fn.c_str(),
fs4ne.c_str(),
thisSystem.c_str(),
builder4fn.c_str(),
out5fn.c_str(),
builder4fn.c_str());
debug(printTerm(fs5));
Path fs5ne = writeTerm(fs5, "-d");
realise(fs5ne);
realise(fs5ne);
}
void run(Strings args)
{
runTests();
}
string programId = "test";
-7
View File
@@ -4,10 +4,3 @@ libutil_a_SOURCES = util.cc util.hh hash.cc hash.hh \
archive.cc archive.hh md5.c md5.h aterm.cc aterm.hh
AM_CXXFLAGS = -Wall -I.. ${aterm_include}
check_PROGRAMS = test-aterm
test_aterm_SOURCES = test-aterm.cc
test_aterm_LDADD = ./libutil.a ../boost/format/libformat.a \
${aterm_lib}
+12 -89
View File
@@ -16,95 +16,6 @@ ostream & operator << (ostream & stream, ATerm e)
}
ATMatcher & atMatch(ATMatcher & pos, ATerm t)
{
pos.t = t;
pos.pos = ATMatcher::funPos;
return pos;
}
static inline bool failed(const ATMatcher & pos)
{
return pos.pos == ATMatcher::failPos;
}
static inline ATMatcher & fail(ATMatcher & pos)
{
pos.pos = ATMatcher::failPos;
return pos;
}
ATMatcher & operator >> (ATMatcher & pos, ATerm & out)
{
out = 0;
if (failed(pos)) return pos;
if (pos.pos == ATMatcher::funPos ||
ATgetType(pos.t) != AT_APPL ||
pos.pos >= (int) ATgetArity(ATgetAFun(pos.t)))
return fail(pos);
out = ATgetArgument(pos.t, pos.pos);
pos.pos++;
return pos;
}
ATMatcher & operator >> (ATMatcher & pos, string & out)
{
out = "";
if (pos.pos == ATMatcher::funPos) {
if (ATgetType(pos.t) != AT_APPL) return fail(pos);
out = ATgetName(ATgetAFun(pos.t));
pos.pos = 0;
} else {
ATerm t;
pos = pos >> t;
if (failed(pos)) return pos;
if (ATgetType(t) != AT_APPL ||
ATgetArity(ATgetAFun(t)) != 0)
return fail(pos);
out = ATgetName(ATgetAFun(t));
}
return pos;
}
ATMatcher & operator >> (ATMatcher & pos, const string & s)
{
string s2;
pos = pos >> s2;
if (failed(pos)) return pos;
if (s != s2) return fail(pos);
return pos;
}
ATMatcher & operator >> (ATMatcher & pos, int & n)
{
n = 0;
ATerm t;
pos = pos >> t;
if (failed(pos)) return pos;
if (ATgetType(t) != AT_INT) return fail(pos);
n = ATgetInt((ATermInt) t);
return pos;
}
ATMatcher & operator >> (ATMatcher & pos, ATermList & out)
{
out = 0;
ATerm t;
pos = pos >> t;
if (failed(pos)) return pos;
if (ATgetType(t) != AT_LIST) return fail(pos);
out = (ATermList) t;
return pos;
}
Error badTerm(const format & f, ATerm t)
{
char * s = ATwriteToString(t);
@@ -116,3 +27,15 @@ Error badTerm(const format & f, ATerm t)
}
return Error(format("%1%, in `%2%'") % f.str() % (string) s);
}
ATerm toATerm(const char * s)
{
return (ATerm) ATmakeAppl0(ATmakeAFun((char *) s, 0, ATtrue));
}
ATerm toATerm(const string & s)
{
return toATerm(s.c_str());
}
+5 -42
View File
@@ -36,51 +36,14 @@ public:
};
/* Type-safe matching. */
struct ATMatcher
{
ATerm t;
int pos;
const static int failPos = -2;
const static int funPos = -1;
ATMatcher() : t(0), pos(failPos)
{
}
operator bool() const
{
return pos != failPos;
}
};
/* Initiate matching of a term. */
ATMatcher & atMatch(ATMatcher & pos, ATerm t);
/* Get the next argument of an application. */
ATMatcher & operator >> (ATMatcher & pos, ATerm & out);
/* Get the name of the function symbol of an application, or the next
argument of an application as a string. */
ATMatcher & operator >> (ATMatcher & pos, string & out);
/* Like the previous, but check that the string is equal to the given
string. */
ATMatcher & operator >> (ATMatcher & pos, const string & s);
/* Get the next argument of an application, and verify that it is a
integer. */
ATMatcher & operator >> (ATMatcher & pos, int & n);
/* Get the next argument of an application, and verify that it is a
list. */
ATMatcher & operator >> (ATMatcher & pos, ATermList & out);
/* Throw an exception with an error message containing the given
aterm. */
Error badTerm(const format & f, ATerm t);
/* Convert strings to ATerms. */
ATerm toATerm(const char * s);
ATerm toATerm(const string & s);
#endif /* !__ATERM_H */
-66
View File
@@ -1,66 +0,0 @@
#include "aterm.hh"
#include <iostream>
void runTests()
{
verbosity = lvlDebug;
ATMatcher pos;
ATerm t = ATmake("Call(Foo, Bar, \"xyz\")");
debug(format("term: %1%") % t);
string fun, arg3;
ATerm lhs, rhs;
if (!(atMatch(pos, t) >> "Call" >> lhs >> rhs >> arg3))
throw Error("should succeed");
if (arg3 != "xyz") throw Error("bad 1");
if (!(atMatch(pos, t) >> fun >> lhs >> rhs >> arg3))
throw Error("should succeed");
if (fun != "Call") throw Error("bad 2");
if (arg3 != "xyz") throw Error("bad 3");
if (!(atMatch(pos, t) >> fun >> lhs >> rhs >> "xyz"))
throw Error("should succeed");
if (atMatch(pos, t) >> fun >> lhs >> rhs >> "abc")
throw Error("should fail");
if (atMatch(pos, t) >> "Call" >> lhs >> rhs >> "abc")
throw Error("should fail");
t = ATmake("X([A, B, C], \"abc\")");
ATerm t1, t2, t3;
if (atMatch(pos, t) >> "X" >> t1 >> t2 >> t3)
throw Error("should fail");
if (!(atMatch(pos, t) >> "X" >> t1 >> t2))
throw Error("should succeed");
ATermList ts;
if (!(atMatch(pos, t) >> "X" >> ts >> t2))
throw Error("should succeed");
if (ATgetLength(ts) != 3)
throw Error("bad");
if (atMatch(pos, t) >> "X" >> t1 >> ts)
throw Error("should fail");
}
int main(int argc, char * * argv)
{
ATerm bottomOfStack;
ATinit(argc, argv, &bottomOfStack);
try {
runTests();
} catch (Error & e) {
printMsg(lvlError, format("error: %1%") % e.msg());
return 1;
}
return 0;
}
+234 -10
View File
@@ -7,9 +7,11 @@
#include <sys/types.h>
#include <sys/stat.h>
#include <sys/wait.h>
#include <unistd.h>
#include <dirent.h>
#include <fcntl.h>
#include <signal.h>
#include "util.hh"
@@ -29,6 +31,13 @@ SysError::SysError(const format & f)
}
string getEnv(const string & key, const string & def)
{
char * value = getenv(key.c_str());
return value ? string(value) : def;
}
Path absPath(Path path, Path dir)
{
if (path[0] != '/') {
@@ -106,7 +115,7 @@ bool pathExists(const Path & path)
struct stat st;
res = lstat(path.c_str(), &st);
if (!res) return true;
if (errno != ENOENT)
if (errno != ENOENT && errno != ENOTDIR)
throw SysError(format("getting status of %1%") % path);
return false;
}
@@ -162,7 +171,7 @@ static void _deletePath(const Path & path)
/* Make the directory writable. */
if (!(st.st_mode & S_IWUSR)) {
if (chmod(path.c_str(), st.st_mode | S_IWUSR) == -1)
throw SysError(format("making `%1%' writable"));
throw SysError(format("making `%1%' writable") % path);
}
for (Strings::iterator i = names.begin(); i != names.end(); ++i)
@@ -206,8 +215,7 @@ void makePathReadOnly(const Path & path)
static Path tempName()
{
static int counter = 0;
char * s = getenv("TMPDIR");
Path tmpRoot = s ? canonPath(Path(s)) : "/tmp";
Path tmpRoot = canonPath(getEnv("TMPDIR", "/tmp"));
return (format("%1%/nix-%2%-%3%") % tmpRoot % getpid() % counter++).str();
}
@@ -235,7 +243,7 @@ void writeStringToFile(const Path & path, const string & s)
LogType logType = ltPretty;
Verbosity verbosity = lvlError;
Verbosity verbosity = lvlInfo;
static int nestingLevel = 0;
@@ -305,7 +313,10 @@ void readFull(int fd, unsigned char * buf, size_t count)
while (count) {
checkInterrupt();
ssize_t res = read(fd, (char *) buf, count);
if (res == -1) throw SysError("reading from file");
if (res == -1) {
if (errno == EINTR) continue;
throw SysError("reading from file");
}
if (res == 0) throw Error("unexpected end-of-file");
count -= res;
buf += res;
@@ -318,13 +329,20 @@ void writeFull(int fd, const unsigned char * buf, size_t count)
while (count) {
checkInterrupt();
ssize_t res = write(fd, (char *) buf, count);
if (res == -1) throw SysError("writing to file");
if (res == -1) {
if (errno == EINTR) continue;
throw SysError("writing to file");
}
count -= res;
buf += res;
}
}
//////////////////////////////////////////////////////////////////////
AutoDelete::AutoDelete(const string & p) : path(p)
{
del = true;
@@ -341,62 +359,268 @@ void AutoDelete::cancel()
}
//////////////////////////////////////////////////////////////////////
AutoCloseFD::AutoCloseFD()
{
fd = -1;
}
AutoCloseFD::AutoCloseFD(int fd)
{
this->fd = fd;
}
AutoCloseFD::~AutoCloseFD()
{
if (fd != -1) close(fd);
try {
close();
} catch (Error & e) {
printMsg(lvlError, format("error (ignored): %1%") % e.msg());
}
}
void AutoCloseFD::operator =(int fd)
{
if (this->fd != fd) close();
this->fd = fd;
}
AutoCloseFD::operator int()
{
return fd;
}
void AutoCloseFD::close()
{
if (fd != -1) {
if (::close(fd) == -1)
/* This should never happen. */
throw SysError("closing file descriptor");
fd = -1;
}
}
bool AutoCloseFD::isOpen()
{
return fd != -1;
}
void Pipe::create()
{
int fds[2];
if (pipe(fds) != 0) throw SysError("creating pipe");
readSide = fds[0];
writeSide = fds[1];
}
//////////////////////////////////////////////////////////////////////
AutoCloseDir::AutoCloseDir()
{
dir = 0;
}
AutoCloseDir::AutoCloseDir(DIR * dir)
{
this->dir = dir;
}
AutoCloseDir::~AutoCloseDir()
{
if (dir) closedir(dir);
}
void AutoCloseDir::operator =(DIR * dir)
{
this->dir = dir;
}
AutoCloseDir::operator DIR *()
{
return dir;
}
//////////////////////////////////////////////////////////////////////
Pid::Pid()
{
pid = -1;
separatePG = false;
}
Pid::~Pid()
{
kill();
}
void Pid::operator =(pid_t pid)
{
if (this->pid != pid) kill();
this->pid = pid;
}
Pid::operator pid_t()
{
return pid;
}
void Pid::kill()
{
if (pid == -1) return;
printMsg(lvlError, format("killing process %1%") % pid);
/* Send a KILL signal to the child. If it has its own process
group, send the signal to every process in the child process
group (which hopefully includes *all* its children). */
if (::kill(separatePG ? -pid : pid, SIGKILL) != 0)
printMsg(lvlError, (SysError(format("killing process %1%") % pid).msg()));
/* Wait until the child dies, disregarding the exit status. */
int status;
while (waitpid(pid, &status, 0) == -1)
if (errno != EINTR) printMsg(lvlError,
(SysError(format("waiting for process %1%") % pid).msg()));
pid = -1;
}
int Pid::wait(bool block)
{
while (1) {
int status;
int res = waitpid(pid, &status, block ? 0 : WNOHANG);
if (res == pid) {
pid = -1;
return status;
}
if (res == 0 && !block) return -1;
if (errno != EINTR)
throw SysError("cannot get child exit status");
}
}
void Pid::setSeparatePG(bool separatePG)
{
this->separatePG = separatePG;
}
//////////////////////////////////////////////////////////////////////
volatile sig_atomic_t _isInterrupted = 0;
void _interrupted()
{
_isInterrupted = 0;
throw Error("interrupted by the user");
/* Block user interrupts while an exception is being handled.
Throwing an exception while another exception is being handled
kills the program! */
if (!uncaught_exception()) {
_isInterrupted = 0;
throw Error("interrupted by the user");
}
}
//////////////////////////////////////////////////////////////////////
string packStrings(const Strings & strings)
{
string d;
for (Strings::const_iterator i = strings.begin();
i != strings.end(); ++i)
{
unsigned int len = i->size();
d += len & 0xff;
d += (len >> 8) & 0xff;
d += (len >> 16) & 0xff;
d += (len >> 24) & 0xff;
d += *i;
}
return d;
}
Strings unpackStrings(const string & s)
{
Strings strings;
string::const_iterator i = s.begin();
while (i != s.end()) {
if (i + 4 > s.end())
throw Error(format("short db entry: `%1%'") % s);
unsigned int len;
len = (unsigned char) *i++;
len |= ((unsigned char) *i++) << 8;
len |= ((unsigned char) *i++) << 16;
len |= ((unsigned char) *i++) << 24;
if (i + len > s.end())
throw Error(format("short db entry: `%1%'") % s);
strings.push_back(string(i, i + len));
i += len;
}
return strings;
}
string statusToString(int status)
{
if (!WIFEXITED(status) || WEXITSTATUS(status) != 0) {
if (WIFEXITED(status))
return (format("failed with exit code %1%") % WEXITSTATUS(status)).str();
else if (WIFSIGNALED(status))
return (format("failed due to signal %1%") % WTERMSIG(status)).str();
else
return "died abnormally";
} else return "succeeded";
}
bool statusOk(int status)
{
return WIFEXITED(status) && WEXITSTATUS(status) == 0;
}
bool string2Int(const string & s, int & n)
{
istringstream str(s);
str >> n;
return str && str.eof();
}
+56
View File
@@ -55,6 +55,9 @@ typedef set<Path> PathSet;
extern string thisSystem;
/* Return an environment variable. */
string getEnv(const string & key, const string & def = "");
/* Return an absolutized path, resolving paths relative to the
specified directory, or the current directory otherwise. The path
is also canonicalised. */
@@ -169,6 +172,7 @@ public:
void cancel();
};
class AutoCloseFD
{
int fd;
@@ -178,8 +182,19 @@ public:
~AutoCloseFD();
void operator =(int fd);
operator int();
void close();
bool isOpen();
};
class Pipe
{
public:
AutoCloseFD readSide, writeSide;
void create();
};
class AutoCloseDir
{
DIR * dir;
@@ -192,6 +207,21 @@ public:
};
class Pid
{
pid_t pid;
bool separatePG;
public:
Pid();
~Pid();
void operator =(pid_t pid);
operator pid_t();
void kill();
int wait(bool block);
void setSeparatePG(bool separatePG);
};
/* User interruption. */
extern volatile sig_atomic_t _isInterrupted;
@@ -204,4 +234,30 @@ void inline checkInterrupt()
}
/* String packing / unpacking. */
string packStrings(const Strings & strings);
Strings unpackStrings(const string & s);
/* Convert the exit status of a child as returned by wait() into an
error string. */
string statusToString(int status);
bool statusOk(int status);
/* Parse a string into an integer. */
bool string2Int(const string & s, int & n);
/* !!! HACK HACK HACK - this should be in shared.hh, but it's to
facilitate a quick hack - will remove this eventually (famous last
words). */
struct SwitchToOriginalUser
{
SwitchToOriginalUser();
~SwitchToOriginalUser();
};
#endif /* !__UTIL_H */
+4
View File
@@ -16,6 +16,8 @@ name `*' may be used to indicate all derivations.
--switch-generation / -G NUMBER: switch to specified generation of profile
--rollback: switch to the previous generation
--list-generations: list available generations of a profile
--delete-generations GENERATIONS...: deleted listed generations,
`old' for all non-current generations
--import / -I FILE: set default Nix expression
@@ -49,3 +51,5 @@ Options:
--file / -f FILE: use Nix expression FILE for installation, etc.
--verbose / -v: verbose operation (may be repeated)
--keep-failed / -K: keep temporary directories of failed builds
--preserve-installed: do not replace currently installed versions in `-i'
--system-filter SYSTEM: only use derivations for specified platform
+230 -94
View File
@@ -6,6 +6,7 @@
#include "parser.hh"
#include "eval.hh"
#include "help.txt.hh"
#include "nixexpr-ast.hh"
#include <cerrno>
#include <ctime>
@@ -17,6 +18,8 @@ struct Globals
Path nixExprPath;
EvalState state;
bool dryRun;
bool preserveInstalled;
string systemFilter;
};
@@ -27,6 +30,7 @@ typedef void (* Operation) (Globals & globals,
struct DrvInfo
{
string name;
string system;
Path drvPath;
Path outPath;
Hash drvHash;
@@ -44,10 +48,9 @@ void printHelp()
bool parseDerivation(EvalState & state, Expr e, DrvInfo & drv)
{
ATMatcher m;
ATermList es;
e = evalExpr(state, e);
if (!(atMatch(m, e) >> "Attrs")) return false;
if (!matchAttrs(e, es)) return false;
Expr a = queryAttr(e, "type");
if (!a || evalString(state, a) != "derivation") return false;
@@ -55,6 +58,12 @@ bool parseDerivation(EvalState & state, Expr e, DrvInfo & drv)
if (!a) throw badTerm("derivation name missing", e);
drv.name = evalString(state, a);
a = queryAttr(e, "system");
if (!a)
drv.system = "unknown";
else
drv.system = evalString(state, a);
a = queryAttr(e, "drvPath");
if (!a) throw badTerm("derivation path missing", e);
drv.drvPath = evalPath(state, a);
@@ -73,7 +82,6 @@ bool parseDerivation(EvalState & state, Expr e, DrvInfo & drv)
bool parseDerivations(EvalState & state, Expr e, DrvInfos & drvs)
{
ATMatcher m;
ATermList es;
DrvInfo drv;
@@ -82,7 +90,7 @@ bool parseDerivations(EvalState & state, Expr e, DrvInfos & drvs)
if (parseDerivation(state, e, drv))
drvs[drv.drvPath] = drv;
else if (atMatch(m, e) >> "Attrs") {
else if (matchAttrs(e, es)) {
ATermMap drvMap;
queryAllAttrs(e, drvMap);
for (ATermIterator i(drvMap.keys()); i; ++i) {
@@ -94,7 +102,7 @@ bool parseDerivations(EvalState & state, Expr e, DrvInfos & drvs)
}
}
else if (atMatch(m, e) >> "List" >> es) {
else if (matchList(e, es)) {
for (ATermIterator i(es); i; ++i) {
debug(format("evaluating list element"));
if (parseDerivation(state, *i, drv))
@@ -108,17 +116,26 @@ bool parseDerivations(EvalState & state, Expr e, DrvInfos & drvs)
}
void loadDerivations(EvalState & state, Path nePath, DrvInfos & drvs)
void loadDerivations(EvalState & state, Path nePath, DrvInfos & drvs,
string systemFilter)
{
Expr e = parseExprFromFile(state, absPath(nePath));
if (!parseDerivations(state, e, drvs))
throw Error("set of derivations expected");
/* Filter out all derivations not applicable to the current
system. */
for (DrvInfos::iterator i = drvs.begin(), j; i != drvs.end(); i = j) {
j = i; j++;
if (systemFilter != "*" && i->second.system != systemFilter)
drvs.erase(i);
}
}
static Path getHomeDir()
{
Path homeDir(getenv("HOME"));
Path homeDir(getEnv("HOME", ""));
if (homeDir == "") throw Error("HOME environment variable not set");
return homeDir;
}
@@ -134,12 +151,10 @@ struct AddPos : TermFun
{
ATerm operator () (ATerm e)
{
ATMatcher m;
ATerm x, y, z;
if (atMatch(m, e) >> "Bind" >> x >> y >> z)
return e;
if (atMatch(m, e) >> "Bind" >> x >> y)
return ATmake("Bind(<term>, <term>, NoPos)", x, y);
if (matchBind(e, x, y, z)) return e;
if (matchBind2(e, x, y))
return makeBind(x, y, makeNoPos());
return e;
}
};
@@ -176,34 +191,36 @@ void createUserEnv(EvalState & state, const DrvInfos & drvs,
for (DrvInfos::const_iterator i = drvs.begin();
i != drvs.end(); ++i)
{
ATerm t = ATmake(
"Attrs(["
"Bind(\"type\", Str(\"derivation\"), NoPos), "
"Bind(\"name\", Str(<str>), NoPos), "
"Bind(\"drvPath\", Path(<str>), NoPos), "
"Bind(\"drvHash\", Str(<str>), NoPos), "
"Bind(\"outPath\", Path(<str>), NoPos)"
"])",
i->second.name.c_str(),
i->second.drvPath.c_str(),
((string) i->second.drvHash).c_str(),
i->second.outPath.c_str());
ATerm t = makeAttrs(ATmakeList6(
makeBind(toATerm("type"),
makeStr(toATerm("derivation")), makeNoPos()),
makeBind(toATerm("name"),
makeStr(toATerm(i->second.name)), makeNoPos()),
makeBind(toATerm("system"),
makeStr(toATerm(i->second.system)), makeNoPos()),
makeBind(toATerm("drvPath"),
makePath(toATerm(i->second.drvPath)), makeNoPos()),
makeBind(toATerm("drvHash"),
makeStr(toATerm((string) i->second.drvHash)), makeNoPos()),
makeBind(toATerm("outPath"),
makePath(toATerm(i->second.outPath)), makeNoPos())
));
inputs = ATinsert(inputs, t);
}
ATerm inputs2 = ATmake("List(<term>)", ATreverse(inputs));
ATerm inputs2 = makeList(ATreverse(inputs));
/* Also write a copy of the list of inputs to the store; we need
it for future modifications of the environment. */
Path inputsFile = writeTerm(inputs2, "-env-inputs");
Expr topLevel = ATmake(
"Call(<term>, Attrs(["
"Bind(\"system\", Str(<str>), NoPos), "
"Bind(\"derivations\", <term>, NoPos), " // !!! redundant
"Bind(\"manifest\", Path(<str>), NoPos)"
"]))",
envBuilder, thisSystem.c_str(), inputs2, inputsFile.c_str());
Expr topLevel = makeCall(envBuilder, makeAttrs(ATmakeList3(
makeBind(toATerm("system"),
makeStr(toATerm(thisSystem)), makeNoPos()),
makeBind(toATerm("derivations"), inputs2, makeNoPos()),
makeBind(toATerm("manifest"),
makePath(toATerm(inputsFile)), makeNoPos())
)));
/* Instantiate it. */
debug(format("evaluating builder expression `%1%'") % topLevel);
@@ -213,8 +230,7 @@ void createUserEnv(EvalState & state, const DrvInfos & drvs,
/* Realise the resulting store expression. */
debug(format("realising user environment"));
Path nfPath = normaliseStoreExpr(topLevelDrv.drvPath);
realiseClosure(nfPath);
Path nfPath = realiseStoreExpr(topLevelDrv.drvPath);
/* Switch the current user environment to the output path. */
debug(format("switching to new user environment"));
@@ -226,16 +242,17 @@ void createUserEnv(EvalState & state, const DrvInfos & drvs,
static void installDerivations(EvalState & state,
Path nePath, DrvNames & selectors, const Path & profile,
bool dryRun)
bool dryRun, bool preserveInstalled, string systemFilter)
{
debug(format("installing derivations from `%1%'") % nePath);
/* Fetch all derivations from the input file. */
DrvInfos availDrvs;
loadDerivations(state, nePath, availDrvs);
loadDerivations(state, nePath, availDrvs, systemFilter);
/* Filter out the ones we're not interested in. */
DrvInfos selectedDrvs;
StringSet selectedNames;
for (DrvInfos::iterator i = availDrvs.begin();
i != availDrvs.end(); ++i)
{
@@ -248,6 +265,7 @@ static void installDerivations(EvalState & state,
format("installing `%1%'") % i->second.name);
j->hits++;
selectedDrvs.insert(*i);
selectedNames.insert(drvName.name);
}
}
}
@@ -262,7 +280,18 @@ static void installDerivations(EvalState & state,
/* Add in the already installed derivations. */
DrvInfos installedDrvs;
queryInstalled(state, installedDrvs, profile);
selectedDrvs.insert(installedDrvs.begin(), installedDrvs.end());
for (DrvInfos::iterator i = installedDrvs.begin();
i != installedDrvs.end(); ++i)
{
DrvName drvName(i->second.name);
if (!preserveInstalled &&
selectedNames.find(drvName.name) != selectedNames.end())
printMsg(lvlInfo,
format("uninstalling `%1%'") % i->second.name);
else
selectedDrvs.insert(*i);
}
if (dryRun) return;
@@ -279,7 +308,8 @@ static void opInstall(Globals & globals,
DrvNames drvNames = drvNamesFromArgs(opArgs);
installDerivations(globals.state, globals.nixExprPath,
drvNames, globals.profile, globals.dryRun);
drvNames, globals.profile, globals.dryRun,
globals.preserveInstalled, globals.systemFilter);
}
@@ -288,7 +318,7 @@ typedef enum { utLt, utLeq, utAlways } UpgradeType;
static void upgradeDerivations(EvalState & state,
Path nePath, DrvNames & selectors, const Path & profile,
UpgradeType upgradeType, bool dryRun)
UpgradeType upgradeType, bool dryRun, string systemFilter)
{
debug(format("upgrading derivations from `%1%'") % nePath);
@@ -303,7 +333,7 @@ static void upgradeDerivations(EvalState & state,
/* Fetch all derivations from the input file. */
DrvInfos availDrvs;
loadDerivations(state, nePath, availDrvs);
loadDerivations(state, nePath, availDrvs, systemFilter);
/* Go through all installed derivations. */
DrvInfos newDrvs;
@@ -311,14 +341,16 @@ static void upgradeDerivations(EvalState & state,
i != installedDrvs.end(); ++i)
{
DrvName drvName(i->second.name);
DrvName selector;
/* Do we want to upgrade this derivation? */
bool upgrade = false;
for (DrvNames::iterator j = selectors.begin();
j != selectors.end(); ++j)
{
if (j->matches(drvName)) {
if (j->name == "*" || j->name == drvName.name) {
j->hits++;
selector = *j;
upgrade = true;
break;
}
@@ -345,9 +377,10 @@ static void upgradeDerivations(EvalState & state,
upgradeType == utLeq && d <= 0 ||
upgradeType == utAlways)
{
if (bestDrv == availDrvs.end() ||
compareVersions(
bestName.version, newName.version) < 0)
if (selector.matches(newName) &&
(bestDrv == availDrvs.end() ||
compareVersions(
bestName.version, newName.version) < 0))
{
bestDrv = j;
bestName = newName;
@@ -386,7 +419,8 @@ static void opUpgrade(Globals & globals,
DrvNames drvNames = drvNamesFromArgs(opArgs);
upgradeDerivations(globals.state, globals.nixExprPath,
drvNames, globals.profile, upgradeType, globals.dryRun);
drvNames, globals.profile, upgradeType, globals.dryRun,
globals.systemFilter);
}
@@ -428,23 +462,70 @@ static void opUninstall(Globals & globals,
}
static bool cmpChars(char a, char b)
{
return toupper(a) < toupper(b);
}
static bool cmpDrvByName(const DrvInfo & a, const DrvInfo & b)
{
return a.name < b.name;
return lexicographical_compare(
a.name.begin(), a.name.end(),
b.name.begin(), b.name.end(), cmpChars);
}
typedef list<Strings> Table;
void printTable(Table & table)
{
unsigned int nrColumns = table.size() > 0 ? table.front().size() : 0;
vector<unsigned int> widths;
widths.resize(nrColumns);
for (Table::iterator i = table.begin(); i != table.end(); ++i) {
assert(i->size() == nrColumns);
Strings::iterator j;
unsigned int column;
for (j = i->begin(), column = 0; j != i->end(); ++j, ++column)
if (j->size() > widths[column]) widths[column] = j->size();
}
for (Table::iterator i = table.begin(); i != table.end(); ++i) {
Strings::iterator j;
unsigned int column;
for (j = i->begin(), column = 0; j != i->end(); ++j, ++column)
{
cout << *j;
if (column < nrColumns - 1)
cout << string(widths[column] - j->size() + 2, ' ');
}
cout << endl;
}
}
static void opQuery(Globals & globals,
Strings opFlags, Strings opArgs)
{
enum { qName, qDrvPath, qStatus } query = qName;
bool printStatus = false;
bool printName = true;
bool printSystem = false;
bool printDrvPath = false;
enum { sInstalled, sAvailable } source = sInstalled;
readOnlyMode = true; /* makes evaluation a bit faster */
for (Strings::iterator i = opFlags.begin();
i != opFlags.end(); ++i)
if (*i == "--name") query = qName;
else if (*i == "--expr") query = qDrvPath;
else if (*i == "--status" || *i == "-s") query = qStatus;
if (*i == "--status" || *i == "-s") printStatus = true;
else if (*i == "--no-name") printName = false;
else if (*i == "--system") printSystem = true;
else if (*i == "--expr") printDrvPath = true;
else if (*i == "--installed") source = sInstalled;
else if (*i == "--available" || *i == "-a") source = sAvailable;
else throw UsageError(format("unknown flag `%1%'") % *i);
@@ -459,7 +540,8 @@ static void opQuery(Globals & globals,
break;
case sAvailable: {
loadDerivations(globals.state, globals.nixExprPath, drvs);
loadDerivations(globals.state, globals.nixExprPath,
drvs, globals.systemFilter);
break;
}
@@ -473,45 +555,46 @@ static void opQuery(Globals & globals,
for (DrvInfos::iterator i = drvs.begin(); i != drvs.end(); ++i)
drvs2.push_back(i->second);
sort(drvs2.begin(), drvs2.end(), cmpDrvByName);
/* We only need to know the installed paths when we are querying
the status of the derivation. */
PathSet installedPaths; /* output paths of installed drvs */
/* Perform the specified query on the derivations. */
switch (query) {
case qName: {
for (DrvInfoList::iterator i = drvs2.begin(); i != drvs2.end(); ++i)
cout << format("%1%\n") % i->name;
break;
}
if (printStatus) {
DrvInfos installed;
queryInstalled(globals.state, installed, globals.profile);
case qDrvPath: {
for (DrvInfoList::iterator i = drvs2.begin(); i != drvs2.end(); ++i)
cout << format("%1%\n") % i->drvPath;
break;
}
case qStatus: {
DrvInfos installed;
queryInstalled(globals.state, installed, globals.profile);
PathSet installedPaths; /* output paths of installed drvs */
for (DrvInfos::iterator i = installed.begin();
i != installed.end(); ++i)
installedPaths.insert(i->second.outPath);
for (DrvInfoList::iterator i = drvs2.begin(); i != drvs2.end(); ++i) {
Paths subs = querySubstitutes(i->drvPath);
cout << format("%1%%2%%3% %4%\n")
% (installedPaths.find(i->outPath)
!= installedPaths.end() ? 'I' : '-')
% (isValidPath(i->outPath) ? 'P' : '-')
% (subs.size() > 0 ? 'S' : '-')
% i->name;
}
break;
}
default: abort();
for (DrvInfos::iterator i = installed.begin();
i != installed.end(); ++i)
installedPaths.insert(i->second.outPath);
}
/* Print the desired columns. */
Table table;
for (DrvInfoList::iterator i = drvs2.begin(); i != drvs2.end(); ++i) {
Strings columns;
if (printStatus) {
Substitutes subs = querySubstitutes(i->drvPath);
columns.push_back(
(string) (installedPaths.find(i->outPath)
!= installedPaths.end() ? "I" : "-")
+ (isValidPath(i->outPath) ? "P" : "-")
+ (subs.size() > 0 ? "S" : "-"));
}
if (printName) columns.push_back(i->name);
if (printSystem) columns.push_back(i->system);
if (printDrvPath) columns.push_back(i->drvPath);
table.push_back(columns);
}
printTable(table);
}
@@ -526,6 +609,7 @@ static void opSwitchProfile(Globals & globals,
Path profile = opArgs.front();
Path profileLink = getHomeDir() + "/.nix-profile";
SwitchToOriginalUser sw;
switchLink(profileLink, profile);
}
@@ -568,10 +652,8 @@ static void opSwitchGeneration(Globals & globals,
if (opArgs.size() != 1)
throw UsageError(format("exactly one argument expected"));
istringstream str(opArgs.front());
int dstGen;
str >> dstGen;
if (!str || !str.eof())
if (!string2Int(opArgs.front(), dstGen))
throw UsageError(format("expected a generation number"));
switchGeneration(globals, dstGen);
@@ -613,6 +695,49 @@ static void opListGenerations(Globals & globals,
}
static void deleteGeneration2(const Path & profile, unsigned int gen)
{
printMsg(lvlInfo, format("removing generation %1%") % gen);
deleteGeneration(profile, gen);
}
static void opDeleteGenerations(Globals & globals,
Strings opFlags, Strings opArgs)
{
if (opFlags.size() > 0)
throw UsageError(format("unknown flag `%1%'") % opFlags.front());
int curGen;
Generations gens = findGenerations(globals.profile, curGen);
for (Strings::iterator i = opArgs.begin(); i != opArgs.end(); ++i) {
if (*i == "old") {
for (Generations::iterator j = gens.begin(); j != gens.end(); ++j)
if (j->number != curGen)
deleteGeneration2(globals.profile, j->number);
}
else {
int n;
if (!string2Int(*i, n) || n < 0)
throw UsageError(format("invalid generation specifier `%1%'") % *i);
bool found = false;
for (Generations::iterator j = gens.begin(); j != gens.end(); ++j) {
if (j->number == n) {
deleteGeneration2(globals.profile, j->number);
found = true;
break;
}
}
if (!found)
printMsg(lvlError, format("generation %1% does not exist") % n);
}
}
}
static void opDefaultExpr(Globals & globals,
Strings opFlags, Strings opArgs)
{
@@ -624,21 +749,21 @@ static void opDefaultExpr(Globals & globals,
Path defNixExpr = absPath(opArgs.front());
Path defNixExprLink = getDefNixExprPath();
SwitchToOriginalUser sw;
switchLink(defNixExprLink, defNixExpr);
}
void run(Strings args)
{
/* Use a higher default verbosity (lvlInfo). */
verbosity = (Verbosity) ((int) verbosity + 1);
Strings opFlags, opArgs;
Operation op = 0;
Globals globals;
globals.nixExprPath = getDefNixExprPath();
globals.dryRun = false;
globals.preserveInstalled = false;
globals.systemFilter = thisSystem;
for (Strings::iterator i = args.begin(); i != args.end(); ++i) {
string arg = *i;
@@ -675,10 +800,20 @@ void run(Strings args)
op = opRollback;
else if (arg == "--list-generations")
op = opListGenerations;
else if (arg == "--delete-generations")
op = opDeleteGenerations;
else if (arg == "--dry-run") {
printMsg(lvlInfo, "(dry run; not doing anything)");
globals.dryRun = true;
}
else if (arg == "--preserve-installed" || arg == "-P")
globals.preserveInstalled = true;
else if (arg == "--system-filter") {
++i;
if (i == args.end()) throw UsageError(
format("`%1%' requires an argument") % arg);
globals.systemFilter = *i;
}
else if (arg[0] == '-')
opFlags.push_back(arg);
else
@@ -691,6 +826,7 @@ void run(Strings args)
if (!op) throw UsageError("no operation specified");
if (globals.profile == "") {
SwitchToOriginalUser sw;
Path profileLink = getHomeDir() + "/.nix-profile";
globals.profile = pathExists(profileLink)
? absPath(readLink(profileLink), dirOf(profileLink))
Loaded 100 of 170 files, more files were not shown because too many files have changed in this diff. Show more