Commit graph

695 commits

Author SHA1 Message Date
Eelco Dolstra
e17910cfb5 * And yet another installation source: the ability to copy user
environment elements from one user environment to another, e.g.,

    $ nix-env -i --from-profile /nix/var/nix/profiles/other-profile aterm
    
  copies the `aterm' component installed in the `other-profile' to the
  user's current profile.
2005-02-15 10:49:31 +00:00
Eelco Dolstra
8992fce3da * It is now possible to add store derivations or paths directly to a
user environment, e.g.,

    $ nix-env -i /nix/store/z58v41v21xd3ywrqk1vmvdwlagjx7f10-aterm-2.3.1.drv

  or 

    $ nix-env -i /nix/store/hsyj5pbn0d9iz7q0aj0fga7cpaadvp1l-aterm-2.3.1

  This is useful because it allows Nix expressions to be bypassed
  entirely.  For instance, if only a nix-pull manifest is provided,
  plus the top-level path of some component, it can be installed
  without having to supply the Nix expression (e.g., for obfuscation,
  or to be independent of Nix expression language changes or context
  dependencies).
2005-02-14 17:35:10 +00:00
Eelco Dolstra
e446d342b7 * Added an installation source --from-expression' (or -E') to
install derivations from a Nix expression specified on the command
  line.  This is particularly useful for disambiguation if there are
  multiple derivations with the same name.  For instance, in Nixpkgs,
  to install the Firefox wrapper rather than the plain Firefox
  component:

    $ nix-env -f .../i686-linux.nix -i -E 'x: x.firefoxWrapper'

  The Nix expressions should be functions to which the default Nix
  expression (in this case, `i686-linux.nix') is passed, hence `x:
  ...'.

  This might also be a nice way to deal with high-level (user-level)
  variability, e.g.,

    $ nix-env -f ./server.nix -i -E 'x: x {port = 8080; ssl = false;}'
2005-02-14 17:07:43 +00:00
Eelco Dolstra
0cb016c209 * Refactoring. Hope this doesn't break the semantics of `-u' ;-) 2005-02-14 16:16:02 +00:00
Eelco Dolstra
a04a5de8f7 * Implement the `gc-keep-derivations' global configuretion flag. 2005-02-14 14:16:56 +00:00
Eelco Dolstra
6a8ef36fe6 * Global configuration option `env-keep-derivations' to store pointer
to derivations in user environments.  Nice for developers (since it
  prevents build-time-only dependencies from being GC'ed, in
  conjunction with `gc-keep-outputs').  Turned off by default.
2005-02-14 13:07:09 +00:00
Eelco Dolstra
b0aba6ec2a * Don't keep the derivation symlink when creating profile generations. 2005-02-14 10:44:57 +00:00
Eelco Dolstra
32429142cd * Type error in constructor call (caught by GCC 3.3, but not 3.4!). 2005-02-14 09:53:11 +00:00
Eelco Dolstra
20ce2642fc * Refactoring to support different installation sources in nix-env.
* Set the references for the user environment manifest properly.
* Don't copy the manifest (this was accidental).
* Don't store derivation paths in the manifest (maybe this should be
  made optional).  This cleans up the semantics of nix-env, which were
  weird.
* Hash on the output paths of activated components, not on derivation
  paths.  This is because we don't know the derivation path of already
  installed components anymore, and it allows the installation of
  components by store path (skipping Nix expressions entirely).
* Query options `--out-path' and `--drv-path' to show the output and
  derivation paths of components, respectively (the latter replaces
  the `--expr' query).
2005-02-11 16:56:45 +00:00
Eelco Dolstra
80870d9291 * Input sources should be in the set of all referenceable paths too. 2005-02-11 16:03:47 +00:00
Eelco Dolstra
3a99616968 * Commit more often to prevent out-of-memory errors. 2005-02-09 14:37:24 +00:00
Eelco Dolstra
98df735b51 * Propagate the deriver of a path through the substitute mechanism.
* Removed some dead code (successor stuff) from nix-push.
* Updated terminology in the tests (store expr -> drv path).
* Check that the deriver is set properly in the tests.
2005-02-09 12:57:13 +00:00
Eelco Dolstra
582e01c06f * Automatically upgrade <= 0.7 Nix stores to the new schema (so that
existing user environments continue to work).
* `nix-store --verify': detect incomplete closures.
2005-02-09 09:50:29 +00:00
Eelco Dolstra
c547439843 * Subflag in --verify': nix-store --verify --check-contents' checks
that the contents of store paths has not changed by comparing hashes
  of their current contents to the hashes stored in the database.
2005-02-08 13:48:53 +00:00
Eelco Dolstra
3d74274b37 * Updated `nix-store --verify' to the new schema. 2005-02-08 13:23:55 +00:00
Eelco Dolstra
fbc434ee4c * `nix-store -qb' to query derivation environment bindings. Useful
for finding build-time dependencies (possibly after a build).  E.g.,

    $ nix-store -qb aterm $(nix-store -qd $(which strc))
    /nix/store/jw7c7s65n1gwhxpn35j9rgcci6ilzxym-aterm-2.3.1

* Arguments to nix-store can be files within store objects, e.g.,
  /nix/store/jw7c...-aterm-2.3.1/bin/baffle.

* Idem for garbage collector roots.
2005-02-07 14:32:44 +00:00
Eelco Dolstra
450c358e20 * Maintain a database table (`derivers') that maps output paths to the
derivation that produced them.
* `nix-store -qd PATH' prints out the derivation that produced a path.
2005-02-07 13:40:40 +00:00
Eelco Dolstra
a37338815d * A GC setting `gc-keep-outputs' to specify whether output paths of
derivations should be kept.
2005-02-01 22:07:48 +00:00
Eelco Dolstra
65b6c8ab4c * Move root finding from nix-collect-garbage' to nix-store --gc'.
This was necessary becase root finding must be done after
  acquisition of the global GC lock.

  This makes `nix-collect-garbage' obsolete; it is now just a wrapper
  around `nix-store --gc'.

* Automatically remove stale GC roots (i.e., indirect GC roots that
  point to non-existent paths).
2005-02-01 15:05:32 +00:00
Eelco Dolstra
630ae0c9d7 * nix-build: use an indirection scheme to make it easier for users to
get rid of GC roots.  Nix-build places a symlink `result' in the
  current directory.  Previously, removing that symlink would not
  remove the store path being linked to as a GC root.  Now, the GC
  root created by nix-build is actually a symlink in
  `/nix/var/nix/gcroots/auto' to `result'.  So if that symlink is
  removed the GC root automatically becomes invalid (since it can no
  longer be resolved).  The root itself is not automatically removed -
  the garbage collector should delete dangling roots.
2005-02-01 13:48:46 +00:00
Eelco Dolstra
dcc37c236c * nix-store, nix-instantiate: added an option `--add-root' to
immediately add the result as a permanent GC root.  This is the only
  way to prevent a race with the garbage collector.  For instance, the
  old style

    ln -s $(nix-store -r $(nix-instantiate foo.nix)) \
      /nix/var/nix/gcroots/result

  has two time windows in which the garbage collector can interfere
  (by GC'ing the derivation and the output, respectively).  On the
  other hand,

    nix-store --add-root /nix/var/nix/gcroots/result -r \
      $(nix-instantiate --add-root /nix/var/nix/gcroots/drv \
        foo.nix)

  is safe.

* nix-build: use `--add-root' to prevent GC races.
2005-02-01 12:36:25 +00:00
Eelco Dolstra
06b4424286 * Add missing files to dist.
* Fix GC and substitute bugs related to self-references.  Add a
  regression test.
2005-02-01 09:23:38 +00:00
Eelco Dolstra
32fa82a56a * Acquire a global GC lock to prevent new temporary root files from
being created after the garbage collector has read the temproots
  directory.  This blocks the creation of new processes, but the
  garbage collector could periodically release the GC lock to allow
  them to run.
2005-01-31 22:23:49 +00:00
Eelco Dolstra
89c9bc11ab * Add a test for a more subtle race: a process starting after the
temporary root files have been read but creating outputs before the
  store directory has been read.
2005-01-31 22:01:55 +00:00
Eelco Dolstra
207bdcbe86 * Automatically remove temporary root files. 2005-01-31 21:20:59 +00:00
Eelco Dolstra
252c9c91ab * Topologically sort paths under the references relation to ensure
that they are deleted in an order that maintains the closure
  invariant.
* Presence of a path in a temporary roots file does not imply that all
  paths in its closure are also present, so add the closure.
2005-01-31 14:00:43 +00:00
Eelco Dolstra
33c5d23b81 * Don't delete active lock files. 2005-01-31 12:19:53 +00:00
Eelco Dolstra
1328aa3307 * Start of concurrent garbage collection. Processes write temporary
roots to a per-process temporary file in /nix/var/nix/temproots
  while holding a write lock on that file.  The garbage collector
  acquires read locks on all those files, thus blocking further
  progress in other Nix processes, and reads the sets of temporary
  roots.
2005-01-31 10:27:25 +00:00
Eelco Dolstra
22cfdfa246 * Use NIX_STORE environment variable to locate the store (in addition
to NIX_STORE_DIR) so that Nix invocations in builders in `make
  check' work correctly if the store doesn't exist.
2005-01-28 13:19:16 +00:00
Eelco Dolstra
0ea8b6993a * Only invalidate paths when they are in fact valid. 2005-01-28 11:05:46 +00:00
Eelco Dolstra
a85d1849af * Missing dependency; only a problem when building from Subversion. 2005-01-27 19:00:48 +00:00
Eelco Dolstra
8a3eef22e3 * Fix deadlock. 2005-01-27 17:48:14 +00:00
Eelco Dolstra
c60a4943ba * Update referers mappings when updating/clearing the references
mapping.
* Do things in the right order in invalidatePath().
2005-01-27 16:18:39 +00:00
Eelco Dolstra
c505702265 * Fix and simplify the garbage collector (it's still not concurrent,
though).  In particular it's now much easier to register a GC root.
  Just place a symlink to whatever store path it is that you want to
  keep in /nix/var/nix/gcroots.
2005-01-27 15:21:29 +00:00
Eelco Dolstra
59682e6188 * Make lock removal safe by signalling to blocked processes that the
lock they are waiting on has become stale (we do this by writing a
  meaningless token to the unlinked file).
2005-01-27 12:19:25 +00:00
Eelco Dolstra
a24b78e9f1 * Maintain the references/referers relation also for derivations.
This simplifies garbage collection and `nix-store --query
  --requisites' since we no longer need to treat derivations
  specially.

* Better maintaining of the invariants, e.g., setReferences() can only
  be called on a valid/substitutable path.
2005-01-25 21:28:25 +00:00
Eelco Dolstra
2a2756b856 * Simplification: registerSubstitutes -> registerSubstitute. We no
longer need the former since there we no longer have the
  substitutes-rev table (which triggered a O(n^2) cost in updating
  them).
2005-01-25 20:27:40 +00:00
Eelco Dolstra
066da4ab85 * Really fix the substitute mechanism, i.e., ensure the closure
invariant by registering references through the manifest.
* Added a test for nix-pull.
2005-01-25 17:08:52 +00:00
Eelco Dolstra
c6290e42bc * Fix the `--fallback' switch.
* Fix the substitutes tests.
2005-01-25 13:00:12 +00:00
Eelco Dolstra
581fc47783 * Fix the build hook mechanism; pass the pointer graph to the hook. 2005-01-25 11:55:43 +00:00
Eelco Dolstra
52bf9b86bb * In nix-store: added query `--referers-closure' that returns the
closure of the referers relation rather than the references
  relation, i.e., the set of all paths that directly or indirectly
  refer to the given path.  Note that contrary to the references
  closure this set is not fixed; it can change as paths are added to
  or removed from the store.
2005-01-25 11:18:03 +00:00
Eelco Dolstra
80faa2f98a * In nix-store: change --build' back to --realise'. Also brought
back the query flag `--force-realise'.
* Fixed some of the tests.
2005-01-25 10:55:33 +00:00
Eelco Dolstra
6a0a2d5593 * Terminology fixes. 2005-01-20 16:01:07 +00:00
Eelco Dolstra
6bb5efadec * Ensure that derivation names and sources don't end in `.drv'. 2005-01-20 15:25:01 +00:00
Eelco Dolstra
05f0430de1 * Another change to low-level derivations. The last one this year, I
promise :-) This allows derivations to specify on *what* output
  paths of input derivations they are dependent.  This helps to
  prevent unnecessary downloads.  For instance, a build might be
  dependent on the `devel' and `lib' outputs of some library
  component, but not the `docs' output.
2005-01-20 14:10:19 +00:00
Eelco Dolstra
e0f4e587c3 * Nix-store queries --references' and referers' to query the pointer
graph.  That is, `nix-store --query --references PATH' shows the set
  of paths referenced by PATH, and `nix-store --query --referers PATH'
  shows the set of paths referencing PATH.
2005-01-19 16:59:56 +00:00
Eelco Dolstra
96de272b48 * Renamed normalise.cc' -> build.cc', `storeexprs.cc' ->
`derivations.cc', etc.
* Store the SHA-256 content hash of store paths in the database after
  they have been built/added.  This is so that we can check whether
  the store has been messed with (a la `rpm --verify').
* When registering path validity, verify that the closure property
  holds.
2005-01-19 16:39:47 +00:00
Eelco Dolstra
ef5f254a55 * `nix-store --build' now builds its arguments in parallel instead of
sequentially (within the limits set by `--jobs').  This should
  greatly improve the utilisation of the build farm when doing Nixpkgs
  builds.
2005-01-19 15:02:02 +00:00
Eelco Dolstra
06c77bf7a8 * Change extension .store' to .drv'.
* Re-enable `nix-store --query --requisites'.
2005-01-19 14:36:00 +00:00
Eelco Dolstra
863dcff6c5 * Started removing closure store expressions, i.e., the explicit
representation of closures as ATerms in the Nix store.  Instead, the
  file system pointer graph is now stored in the Nix database.  This
  has many advantages:

  - It greatly simplifies the implementation (we can drop the notion
    of `successors', and so on).

  - It makes registering roots for the garbage collector much easier.
    Instead of specifying the closure expression as a root, you can
    simply specify the store path that must be retained as a root.
    This could not be done previously, since there was no way to find
    the closure store expression containing a given store path.
    
  - Better traceability: it is now possible to query what paths are
    referenced by a path, and what paths refer to a path.
2005-01-19 11:16:11 +00:00
Eelco Dolstra
e9762e2d10 * Support arities > 6. 2005-01-19 11:04:24 +00:00
Eelco Dolstra
6d493751c3 * Get --readonly-mode to work again. 2005-01-18 11:15:50 +00:00
Eelco Dolstra
32aac8748a * Actually check that the result of fixed-output derivations matches
the specified hash.
2005-01-17 19:01:48 +00:00
Eelco Dolstra
f3dc231250 * Removed the `id' attribute hack.
* Formalise the notion of fixed-output derivations, i.e., derivations
  for which a cryptographic hash of the output is known in advance.
  Changes to such derivations should not propagate upwards through the
  dependency graph.  Previously this was done by specifying the hash
  component of the output path through the `id' attribute, but this is
  insecure since you can lie about it (i.e., you can specify any hash
  and then produce a completely different output).  Now the
  responsibility for checking the output is moved from the builder to
  Nix itself.

  A fixed-output derivation can be created by specifying the
  `outputHash' and `outputHashAlgo' attributes, the latter taking
  values `md5', `sha1', and `sha256', and the former specifying the
  actual hash in hexadecimal or in base-32 (auto-detected by looking
  at the length of the attribute value).  MD5 is included for
  compatibility but should be considered deprecated.

* Removed the `drvPath' pseudo-attribute in derivation results.  It's
  no longer necessary.

* Cleaned up the support for multiple output paths in derivation store
  expressions.  Each output now has a unique identifier (e.g., `out',
  `devel', `docs').  Previously there was no way to tell output paths
  apart at the store expression level.

* `nix-hash' now has a flag `--base32' to specify that the hash should
  be printed in base-32 notation.

* `fetchurl' accepts parameters `sha256' and `sha1' in addition to
  `md5'.

* `nix-prefetch-url' now prints out a SHA-1 hash in base-32.  (TODO: a
  flag to specify the hash.)
2005-01-17 16:55:19 +00:00
Eelco Dolstra
d58a11e019 * Shorten SHA-256 hashes used in store path name generation to 160
bits, then encode them in a radix-32 representation (using digits
  and letters except e, o, u, and t).  This produces store paths like
  /nix/store/4i0zb0z7f88mwghjirkz702a71dcfivn-aterm-2.3.1.  The nice
  thing about this is that the hash part of the file name is still 32
  characters, as before with MD5.

  (Of course, shortening SHA-256 to 160 bits makes it no better than
  SHA-160 in theory, but hopefully it's a bit more resistant to
  attacks; it's certainly a lot slower.)
2005-01-14 16:04:03 +00:00
Eelco Dolstra
9530cc3170 * Start move towards SHA-256 hashes instead of MD5.
* Start cleaning up unique store path generation (they weren't always
  unique; in particular the suffix ("-aterm-2.2", "-builder.sh") was
  not part of the hash, therefore changes to the suffix would cause
  multiple store objects with the same hash).
2005-01-14 13:51:38 +00:00
Eelco Dolstra
63791eb05b * Add SHA-256.
* Tests for the various hashes.
2005-01-14 12:03:04 +00:00
Eelco Dolstra
37b51a9aa6 * Removed some dead code. 2005-01-14 10:16:33 +00:00
Eelco Dolstra
7e8961f720 * Added SHA-1 support. nix-hash' now has an option --type sha1' to
select SHA-1 hashing.
2005-01-13 17:39:26 +00:00
Eelco Dolstra
73992371a3 * Refactoring to support SHA-1. 2005-01-13 15:44:44 +00:00
Eelco Dolstra
a03397be4c * Cygwin compatibility. 2005-01-04 17:38:26 +00:00
Eelco Dolstra
4f07ebc67e * Integrated bsdiff/bspatch 4.2 (from
http://www.daemonology.net/bsdiff/bsdiff-4.2.tar.gz) into the source
  tree.  The license is a bit peculiar, but it does allow verbatim
  copying, which is what we do here (i.e., so don't make any changes
  to the sources).
2004-12-29 22:08:48 +00:00
Eelco Dolstra
8b9697e575 * An operation `nix-store --clear-substitutes' to remove all
registered substitute mappings.
2004-12-20 14:16:55 +00:00
Eelco Dolstra
fa9259f5f5 * Simplify the substitute mechanism:
- Drop the store expression.  So now a substitute is just a
    command-line invocation (a program name + arguments).  If you
    register a substitute you are responsible for registering the
    expression that built it (if any) as a root of the garbage
    collector.
  - Drop the substitutes-rev DB table.
2004-12-20 13:43:32 +00:00
Eelco Dolstra
13f77276d1 * utime() follows symlinks, so don't change the mtime if the file is a
symlink.
2004-11-29 19:22:16 +00:00
Eelco Dolstra
f17553a212 * Remove debug statement. 2004-11-29 15:30:44 +00:00
Eelco Dolstra
4115d8d8ce * Canonicalise metadata of all files in store paths (set the mtime to
0, set the mode to either 444 or 555, set the group to the default).
2004-11-29 15:09:29 +00:00
Eelco Dolstra
54c7a870d5 * Document --delete-generations and other nix-env options. 2004-11-12 23:56:37 +00:00
Eelco Dolstra
c2b0d6b02f * Document --eval-only and --parse-only options in nix-instantiate. 2004-11-12 23:22:08 +00:00
Eelco Dolstra
92ee003dc9 * Fix broken format string. 2004-11-08 15:20:52 +00:00
Eelco Dolstra
cb7ccb528b * string2ATerm -> overloaded toATerm. 2004-11-03 18:12:03 +00:00
Eelco Dolstra
a69534fc21 * Drop ATmake / ATMatcher also in handling store expressions. 2004-10-29 11:22:49 +00:00
Eelco Dolstra
3277c9432a * Bug fix in parsing of /* ... */ comments; due to longest match
regexp there could be only one such comment per file.
2004-10-27 13:00:31 +00:00
Eelco Dolstra
f09618b63a * Turn on read-only mode in queries. This prevents redundant store I/O. 2004-10-27 10:24:44 +00:00
Eelco Dolstra
c7bea941b0 * Oops, I did it again. 2004-10-27 10:05:51 +00:00
Eelco Dolstra
210ab0296d * Add file to `make dist'. 2004-10-27 00:02:31 +00:00
Eelco Dolstra
c52dda95a6 * Bug: check that term is an application. 2004-10-26 23:30:18 +00:00
Eelco Dolstra
5fe9222b36 * Don't use ATmake / ATmatch anymore, nor the ATMatcher class.
Instead we generate data bindings (build and match functions) for
  the constructors specified in `constructors.def'.  In particular
  this removes the conversions between AFuns and strings, and Nix
  expression evaluation now seems 3 to 4 times faster.
2004-10-26 22:54:26 +00:00
Eelco Dolstra
eb8284ddaa * Evaluate argument to `import'. 2004-10-26 17:10:09 +00:00
Eelco Dolstra
033d7c6593 * Doh! 2004-10-26 17:04:55 +00:00
Eelco Dolstra
9fa07b376d * String/path concatenation operator (`+'). 2004-10-26 17:01:35 +00:00
Eelco Dolstra
ee401afad8 * Mode --parse-only' to parse the input (on stdin, -'), and print
out the AST as an ATerm.
* Mode `--eval-only' to parse and evaluate the input, and print the
  resulting normal form as an ATerm.

Neither of these modes require store/DB write permission.
2004-10-26 16:59:36 +00:00
Eelco Dolstra
37d7abd694 * New language feature: with expressions.
The expression `with E1; E2' evaluates to E2 with all bindings in
  the attribute set E1 substituted.  E.g.,

    with {x = 123;}; x

  evaluates to 123.  That is, the attribute set E1 is in scope in E2.

  This is particularly useful when importing files containing lots
  definitions.  E.g., instead of

    let {
      inherit (import ./foo.nix) a b c d e f;

      body = ... a ... f ...;
    }

  we can now say

    with import ./foo.nix;

    ... a ... f ...

  I.e., we don't have to say what variables should be brought into scope.
2004-10-25 16:54:56 +00:00
Eelco Dolstra
f4d44a0026 * Allow certain operations to succeed even if we don't have write
permission to the Nix store or database.  E.g., `nix-env -qa' will
  work, but `nix-env -qas' won't (the latter needs DB access).  The
  option `--readonly-mode' forces this mode; otherwise, it's only
  activated when the database cannot be opened.
2004-10-25 14:38:23 +00:00
Eelco Dolstra
3ade3e7721 * Revert r1594 - it didn't solve the problem. Instead add
svn-revision to distributions, which should fix it.
2004-10-25 13:51:34 +00:00
Eelco Dolstra
2248becfd3 * Make sure that the prerelease version is included in `--version'. 2004-10-25 12:15:50 +00:00
Eelco Dolstra
d830b2c1df * In `nix-env -q', sort derivations by name *without* case
sensitivity.
2004-10-14 15:09:55 +00:00
Eelco Dolstra
1317242780 * Make store objects created by substitutes read-only. 2004-09-22 12:15:04 +00:00
Eelco Dolstra
b357284a32 * Fallback didn't work for subderivations of an unnormalised the main
derivation, since NormalisationGoal would first run a
  NormalisationGoal on the subderivation (a no-op, since in a
  situation where we need fallback the successor is known), and then
  runs a RealisationGoal on the normal form, which then cannot do a
  fallback because it doesn't know the derivation expression for which
  it is a normal form.

  Tossed out the 2-phase normalisation/realisation in
  NormalisationGoal and SubstitutionGoal since it's no longer needed -
  a RealisationGoal will run a NormalisationGoal if necessary.
2004-09-12 19:08:57 +00:00
Eelco Dolstra
dcc433de47 * Operation `--delete-generations' to delete generations of a
profile.  Arguments are either generation number, or `old' to delete
  all non-current generations.  Typical use:

  $ nix-env --delete-generations old
  $ nix-collect-garbage

* istringstream -> string2Int.
2004-09-10 13:32:08 +00:00
Eelco Dolstra
c16be6ac92 * Remove write permission from store objects after they have been
added to the store.  Bug reported by Martin.
2004-09-09 21:19:20 +00:00
Eelco Dolstra
47f87072ad * A very dirty hack to make setuid installations a bit nicer to use.
Previously there was the problem that all files read by nix-env
  etc. should be reachable and readable by the Nix user.  So for
  instance building a Nix expression in your home directory meant that
  the home directory should have at least g+x or o+x permission so
  that the Nix user could reach the Nix expression.  Now we just
  switch back to the original user just prior to reading sources and
  the like.  The places where this happens are somewhat arbitrary,
  however.  Any scope that has a live SwitchToOriginalUser object in
  it is executed as the original user.

* Back out r1385.  setreuid() sets the saved uid to the new
  real/effective uid, which prevents us from switching back to the
  original uid.  setresuid() doesn't have this problem (although the
  manpage has a bug: specifying -1 for the saved uid doesn't leave it
  unchanged; an explicit value must be specified).
2004-09-09 21:12:53 +00:00
Eelco Dolstra
5396304c73 * Use setre[ug]id() instead of setres[ug]id(), since the former is
more common than the latter (which exists only on Linux and
  FreeBSD).  We don't really care about dropping the saved IDs since
  there apparently is no way to quiry them in any case, so it can't
  influence the build (unlike the effective IDs which are checked by
  Perl for instance).
2004-09-09 15:55:31 +00:00
Eelco Dolstra
e043fc7d0b * Set the umask to known value (0022). This is important in a
setuid installation, since the calling user may have a more fascist
  umask (say, 0077), which would cause the store objects built by Nix
  to be unreadable to anyone other than the Nix user.
2004-09-09 14:16:02 +00:00
Eelco Dolstra
5c443b6550 * Main the `substitutes-rev' table again, but now in a way that
doesn't take \Theta(n^2) space/time complexity.
2004-08-31 16:13:10 +00:00
Eelco Dolstra
c25f2883b1 * Quadruple the Berkeley DB locking limits to get rid of out of memory
errors while running `nix-store --verify'.
2004-08-31 10:50:08 +00:00
Eelco Dolstra
fe122c5a15 * Removed nrWaitees field. It was redundant with waitees.size() and
could get out of sync if multiple input derivations mapped to the
  same closure expression (since waitees is a set).
2004-08-30 11:51:36 +00:00
Eelco Dolstra
eb233e728f * `--min-age' flag in nix-store and nix-collect-garbage to only delete
unreachable paths that haven't been used for N hours.  For instance,
  `nix-collect-garbage --min-age 168' only deletes paths that haven't
  been accessed in the last week.

  This is useful for instance in the build farm where many derivations
  can be shared between consecutive builds, and we wouldn't want a
  garbage collect to throw them all away.  We could of course register
  them as roots, but then we'd to unregister them at some point, which
  would be a pain to manage.  The `--min-age' flag gives us a sort of
  MRU caching scheme.

  BUG: this really shouldn't be in gc.cc since that violates
  mechanism/policy separation.
2004-08-25 16:54:08 +00:00
Eelco Dolstra
fdec72c6cc * `nix-collect-garbage' now actually performs a garbage collection, it
doesn't just print the set of paths that should be deleted.  So
  there is no more need to pipe the result into `nix-store --delete'
  (which doesn't even exist anymore).
2004-08-25 15:39:13 +00:00
Eelco Dolstra
818047881e * Put the garbage collector in nix-store: operation `--gc',
suboperations `--print-live', `--print-dead', and `--delete'.  The
  roots are not determined by nix-store; they are read from standard
  input.  This is to make it easy to customise what the roots are.

  The collector now no longer fails when store expressions are missing
  (which legally happens when using substitutes).  It never tries to
  fetch paths through substitutes.

  TODO: acquire a global lock on the store while garbage collecting.
  
* Removed `nix-store --delete'.
2004-08-25 11:43:49 +00:00
Eelco Dolstra
9994c1dd9f * Validate derivation names. In particular don't allow spaces.
* Drop support for the outPath attribute in derivations.
2004-08-24 11:46:05 +00:00
Eelco Dolstra
8f58733ef1 * The gid should also match. 2004-08-20 15:47:58 +00:00
Eelco Dolstra
1c90fabccc * Unbreak programs that are not setuid (such as nix-hash). 2004-08-20 15:31:46 +00:00
Eelco Dolstra
e77fbe0fa2 * On systems that have the setresuid() and setresgid() system calls to
set the real uid and gid to the effective uid and gid, the Nix
  binaries can be installed as owned by the Nix user and group instead
  of root, so no root involvement of any kind is necessary.

  Linux and FreeBSD have these functions.
2004-08-20 15:22:33 +00:00
Eelco Dolstra
2d35116c13 * Setuid support for sharing a Nix installation between multiple
users.

  If the configure flag `--enable-setuid' is used, the Nix programs
  nix-env, nix-store, etc. are installed with the setuid bit turned on
  so that they are executed as the user and group specified by
  `--with-nix-user=USER' and `--with-nix-group=GROUP', respectively
  (with defaults `nix' and `nix').

  The setuid programs drop all special privileges if they are executed
  by a user who is not a member of the Nix group.

  The setuid feature is a quick hack to enable sharing of a Nix
  installation between users who trust each other.  It is not
  generally secure, since any user in the Nix group can modify (by
  building an appropriate derivation) any object in the store, and for
  instance inject trojans into binaries used by other users.

  The setuid programs are owned by root, not the Nix user.  This is
  because on Unix normal users cannot change the real uid, only the
  effective uid.  Many programs don't work properly when the real uid
  differs from the effective uid.  For instance, Perl will turn on
  taint mode.  However, the setuid programs drop all root privileges
  immediately, changing all uids and gids to the Nix user and group.
2004-08-20 14:49:05 +00:00
Eelco Dolstra
1eddee59f2 * The default verbosity level of all Nix commands is now lvlInfo.
* Builder output is written to standard error by default.
  * The option `-B' is gone.
  * The option `-Q' suppresses builder output.

The result of this is that most Nix invocations shouldn't need any
flags w.r.t. logging.
2004-08-18 12:19:06 +00:00
Eelco Dolstra
62fe5c4a22 * The predecessor of a successor need not be present. This in
particular happens on distributed builds or when using push/pull.
2004-08-11 19:03:13 +00:00
Eelco Dolstra
ae1a1efa41 * Clean up the temporary directory for hook communication (and don't
print out incorrect "build failed" messages).
2004-08-05 14:53:27 +00:00
Eelco Dolstra
d8989b1fb4 * Every real language has a `map' function. 2004-08-04 11:27:53 +00:00
Eelco Dolstra
bbfdd64741 * Allow primops with more that 1 arguments. 2004-08-04 10:59:20 +00:00
Eelco Dolstra
e3a50f7e25 * Creating a file nix-support/no-scan in the output path of a
derivation disables scanning for dependencies.  Use at your own
  risk.  This is a quick hack to speed up UML image generation (image
  are very big, say 1 GB).

  It would be better if the scanner were faster, and didn't read the
  whole file into memory.
2004-08-04 09:25:21 +00:00
Eelco Dolstra
c1a18f543e * Fixed format string error. 2004-07-06 11:21:34 +00:00
Eelco Dolstra
056cd1d3b7 * Don't go into a (sometimes infinite) loop calling the build hook. 2004-07-01 16:24:35 +00:00
Eelco Dolstra
638ce339a5 * Nix-instantiate now accepts sets of derivations (just like nix-env). 2004-07-01 14:25:26 +00:00
Eelco Dolstra
292d6468ec * Nix-env operations now by default filter out any derivations for
system types other than the current system.  I.e., `nix-env -i'
  won't install derivations for other system types, and `nix-env -q'
  won't show them.  The flag `--system-filter SYSTEM' can be used to
  override the system type used for filtering (but not for
  building!).  The value `*' can be used not to filter anything.
2004-07-01 13:56:56 +00:00
Eelco Dolstra
8f6254e823 * Align the columns in the output of `nix-env -q'. 2004-07-01 13:35:10 +00:00
Eelco Dolstra
593bc23d8b * Allow the system attribute of derivations to be queried in
`nix-env -q'.
* Queries can now be combined, e.g., `nix-env -q --status --system'.
2004-07-01 13:13:37 +00:00
Eelco Dolstra
b584253af4 * Include some missing headers. 2004-07-01 11:11:16 +00:00
Eelco Dolstra
f5d5ffe536 * Write build logs to disk again. 2004-06-29 09:41:50 +00:00
Eelco Dolstra
151e61fa5a * By default, `nix-env -i' now deletes previously installed
derivations with names matching the derivations being installed.
  The option `--preserve-installed / -P' overrides this behaviour.
2004-06-28 14:40:26 +00:00
Eelco Dolstra
4d2946c516 * In a realisation goal, check the result of the corresponding
normalisation goal.
2004-06-28 13:51:24 +00:00
Eelco Dolstra
24286e15c9 * `nix-env -u' now allows a specific version to be specified when
upgrading.

  This fixes a bug reported by Martin:

    $ nix-env -i foo-1.0
    $ nix-env -u foo-1.0
    upgrading foo-1.0 to foo-1.1
2004-06-28 13:37:05 +00:00
Eelco Dolstra
91dc023665 * Added a switch `--fallback'. From the manual:
Whenever Nix attempts to realise a derivation for which a closure is
  already known, but this closure cannot be realised, fall back on
  normalising the derivation.

  The most common scenario in which this is useful is when we have
  registered substitutes in order to perform binary distribution from,
  say, a network repository.  If the repository is down, the
  realisation of the derivation will fail.  When this option is
  specified, Nix will build the derivation instead.  Thus, binary
  installation falls back on a source installation.  This option is
  not the default since it is generally not desirable for a transient
  failure in obtaining the substitutes to lead to a full build from
  source (with the related consumption of resources).
2004-06-28 10:42:57 +00:00
Eelco Dolstra
b113edeab7 * A flag `--keep-going / -k' to keep building goals if one fails, as
much as possible.  (This is similar to GNU Make's `-k' flag.)

* Refactoring to implement this: previously we just bombed out when
  a build failed, but now we have to clean up.  In particular this
  means that goals must be freed quickly --- they shouldn't hang
  around until the worker exits.  So the worker now maintains weak
  pointers in order not to prevent garbage collection.

* Documented the `-k' and `-j' flags.
2004-06-25 15:36:09 +00:00
Eelco Dolstra
e4883211f9 * Don't throw an exception when a build fails. Just terminate the
goal and allow the problem to be handled elsewhere (e.g., at
  top-level).
2004-06-25 10:21:44 +00:00
Eelco Dolstra
795d9f8b08 * Obsolete. 2004-06-24 14:36:50 +00:00
Eelco Dolstra
ec32627621 * Multiple and/or failing substitutes now work. 2004-06-24 13:40:38 +00:00
Eelco Dolstra
66c7f34759 * Arghhhhhh 2004-06-22 17:07:32 +00:00
Eelco Dolstra
05a5362d63 * Some more diagnostics changes. 2004-06-22 17:04:10 +00:00
Eelco Dolstra
83ae1723da * Well, it's better than printf. 2004-06-22 15:01:01 +00:00
Eelco Dolstra
72bc9a522f * Started making Nix's diagnostic messages a bit more useful. 2004-06-22 14:48:59 +00:00
Eelco Dolstra
5e2cf44a4d * Put WEXITSTATUS stuff somewhere else. 2004-06-22 11:03:41 +00:00
Eelco Dolstra
84007a0958 * Reduce gratuitous cut & pasting. 2004-06-22 10:21:44 +00:00
Eelco Dolstra
c9fbd2dfd5 * Wrapper class around pids. 2004-06-22 09:51:44 +00:00
Eelco Dolstra
155d7c8dfa * Substitutes should occupy a build slot. 2004-06-22 09:00:31 +00:00
Eelco Dolstra
c4cb6ea2bc * Refactoring. 2004-06-22 08:50:25 +00:00
Eelco Dolstra
2db9748221 * Remove debug output. 2004-06-21 10:01:17 +00:00
Eelco Dolstra
3f3a3ae87b * Acquire a lock on the output path when running a substitute. Also
delete obstructing invalid paths.
2004-06-21 09:35:50 +00:00
Eelco Dolstra
72c857f0eb * Ugh, nasty Heisenbug due to an uninitialiased variable. The bug
only caused a crash if the program was *not* invoked with a high
  verbosity level.
2004-06-21 08:51:55 +00:00
Eelco Dolstra
be1a917beb * Remove obstructing invalid store paths add[Text]ToStore(). 2004-06-21 07:46:02 +00:00
Eelco Dolstra
daf0a923c7 * Wrap calls to registerSubstitute() in a single transaction to
improve throughput.
* Don't build the `substitute-rev' table for now, since it caused
  Theta(N^2) time and log file consumption when adding N substitutes.
  Maybe we can do without it.
2004-06-21 07:38:17 +00:00
Eelco Dolstra
15c60ca1b6 * Disable calls to fsync() since Berkeley DB's DB_TXN_WRITE_NOSYNC
flag doesn't seem to work as advertised.
2004-06-21 07:36:01 +00:00
Eelco Dolstra
112ee89501 * Re-enable support for substitutes in the normaliser.
* A better substitute mechanism.

  Instead of generating a store expression for each store path for
  which we have a substitute, we can have a single store expression
  that builds a generic program that is invoked to build the desired
  store path, which is passed as an argument.

  This means that operations like `nix-pull' only produce O(1) files
  instead of O(N) files in the store when registering N substitutes.
  (It consumes O(N) database storage, of course, but that's not a
  performance problem).

* Added a test for the substitute mechanism.
  
* `nix-store --substitute' reads the substitutes from standard input,
  instead of from the command line.  This prevents us from running
  into the kernel's limit on command line length.
2004-06-20 19:17:54 +00:00
Eelco Dolstra
85ae781765 * Refactoring. 2004-06-20 13:37:51 +00:00
Eelco Dolstra
23bb902d1f * Re-enable build hooks. 2004-06-19 21:45:04 +00:00
Eelco Dolstra
41ec982f31 * Big refactoring. Move to a much more explicitly state machine based
approach.  This makes it much easier to add extra complexity in the
  normaliser / realiser (e.g., build hooks, substitutes).
2004-06-18 18:09:32 +00:00
Eelco Dolstra
3454c685ee * This is also useful. 2004-06-18 16:52:31 +00:00
Eelco Dolstra
6ba26f27c3 * Shared (garbage collecting) pointers. Copied from Boost. 2004-06-18 09:20:51 +00:00
Eelco Dolstra
0b70231b9d * Refactoring. 2004-06-15 13:49:42 +00:00
Eelco Dolstra
1bc6afefac * Cleanup. 2004-06-08 13:21:03 +00:00
Eelco Dolstra
5e4a2272bf * Drain the output of the build hook to show error messages. Ugly
hack.
2004-05-18 14:52:35 +00:00
Eelco Dolstra
8e9fd57ef9 * setpgrp() is not POSIX (and on Mac OS X it's different than on
Linux), so use setpgid().
2004-05-18 09:45:18 +00:00
Eelco Dolstra
ace8872706 * execl() requires a terminating 0.
* When a fast build wakes up a goal, try to start that goal in the
  same iteration of the startBuild() loop of run().  Otherwise no job
  might be started until the next job terminates.
2004-05-14 12:24:29 +00:00
Eelco Dolstra
4fc00cbec1 * Distributed builds and load balancing now seem to work pretty well.
(Though the `build-remote.pl' script has a gigantic race condition).
2004-05-13 22:52:37 +00:00
Eelco Dolstra
25db622454 * Load balancing. `build-remote.pl' will only execute up to a
per-machine maximum number of parallel jobs on a remote machine.
2004-05-13 19:35:46 +00:00
Eelco Dolstra
a8306cb98f * The build hooks used to implement distributed builds can now be run
in parallel.  Hooks are more efficient: locks on output paths are
  only acquired when the hook says that it is willing to accept a
  build job.  Hooks now work in two phases.  First, they should first
  tell Nix whether they are willing to accept a job.  Nix guarantuees
  that no two hooks will ever be in the first phase at the same time
  (this simplifies the implementation of hooks, since they don't have
  to perform locking (?)).  Second, if they accept a job, they are
  then responsible for building it (on the remote system), and copying
  the result back.  These can be run in parallel with other hooks and
  locally executed jobs.

  The implementation is a bit messy right now, though.  

* The directory `distributed' shows a (hacky) example of a hook that
  distributes build jobs over a set of machines listed in a
  configuration file.
2004-05-13 19:14:49 +00:00
Eelco Dolstra
efa5fa1a91 * A switch `-j NUMBER' to set the maximum number of parallel jobs (0 =
no limit).
* Add missing file to distribution.
2004-05-12 14:20:32 +00:00
Eelco Dolstra
aa5a5084e4 * Pass to the build hook all sorts of information useful for
distributing a build action to another machine.  In particular, the
  paths in the input closures, the output paths, and successor mapping
  for sub-derivations.
2004-05-12 13:32:26 +00:00
Eelco Dolstra
8c0b42f857 * An quick and dirty hack to support distributed builds. 2004-05-12 09:35:51 +00:00
Eelco Dolstra
c8d3882cdc * True parallel builds. Nix can now run as many build jobs in
parallel as possible (similar to GNU Make's `-j' switch).  This is
  useful on SMP systems, but it is especially useful for doing builds
  on multiple machines.  The idea is that a large derivation is
  initiated on one master machine, which then distributes
  sub-derivations to any number of slave machines.  This should not
  happen synchronously or in lock-step, so the master must be capable
  of dealing with multiple parallel build jobs.  We now have the
  infrastructure to support this.

  TODO: substitutes are currently broken.
2004-05-11 18:05:44 +00:00
Eelco Dolstra
aea436503e * Ignore interrupt signals while handling an exception.
* Ignore EINTR in reads and writes.
2004-05-11 13:48:25 +00:00
Eelco Dolstra
256eeab711 * Allow the location of the store etc. to be specified using
environment variables.
* Started adding some automatic tests.
* Do a `make check' when building RPMs.
2004-05-04 12:15:30 +00:00
Eelco Dolstra
d4779abc04 * Pass SYSTEM through config.h, and allow spaces. 2004-04-23 15:16:08 +00:00
Eelco Dolstra
d7238bc84e * Don't create $(localstatedir)/nix/profiles if --disable-init-state
is specified.
2004-04-22 07:47:41 +00:00
Eelco Dolstra
f79e9c2d22 * Do initialise state (the DB etc.) when doing a `make install',
unless `--disable-init-state' is passed to configure.
2004-04-21 10:54:46 +00:00
Eelco Dolstra
8e459d919d * Recurse into attribute sets and lists when getting derivations from
an expression.
2004-04-21 09:37:37 +00:00
Eelco Dolstra
a4d2b22c8c * Be stricter in verifying store paths. 2004-04-14 08:08:55 +00:00
Eelco Dolstra
59b94ee18a * When something goes wrong in the evaluation of a Nix expression,
print a nice backtrace of the stack, rather than vomiting a gigantic
  (and useless) aterm on the screen.  Example:

    error: while evaluating file `.../pkgs/system/test.nix':
    while evaluating attribute `subversion' at `.../pkgs/system/all-packages-generic.nix', line 533:
    while evaluating function at `.../pkgs/applications/version-management/subversion/default.nix', line 1:
    assertion failed at `.../pkgs/applications/version-management/subversion/default.nix', line 13

  Since the Nix expression language is lazy, the trace may be
  misleading.  The purpose is to provide a hint as to the location of
  the problem.
2004-04-05 22:27:41 +00:00
Eelco Dolstra
a520b1cbc3 * Print a more useful error message in case of an invalid derivation
binding.
2004-04-02 10:49:37 +00:00
Eelco Dolstra
c4ac2a164a * The recent change in nixpkgs of calling `stdenv.mkDerivation'
instead of `derivation' triggered a huge slowdown in the Nix
  expression evaluator.  Total execution time of `nix-env -qa' went up
  by a factor of 60 or so.

  This scalability problem was caused by expressions such as

    (x: y: ... x ...) a b

  where `a' is a large term (say, the one in
  `all-packages-generic.nix').  Then the first beta-reduction would
  produce

    (y: ... a ...) b

  by substituting `a' for `x'.  The second beta-reduction would then
  substitute `b' for `y' into the body `... a ...', which is a large
  term due to `a', and thus causes a large traversal to be performed
  by substitute() in the second reduction.  This is however entirely
  redundant, since `a' cannot contain free variables (since we never
  substitute below a weak head normal form).

  The solution is to wrap substituted terms into a `Closed'
  constructor, i.e.,

    subst(subs, Var(x)) = Closed(e) iff subs[x] = e

  have substitution not descent into closed terms,

    subst(subs, Closed(x)) = Closed(x)

  and otherwise ignore them for evaluation,

    eval(Closed(x)) = eval(x).

* Fix a typo that caused incorrect substitutions to be performed in
  simple lambdas, e.g., `(x: x: x) a' would reduce to `(x: a)'.
2004-03-30 15:05:35 +00:00
Eelco Dolstra
ac4d39f9db * Added an operator `?' to test for attribute existence, e.g.,
`attrs ? x' yields true iff `attrs' has an attribute named `x'.
2004-03-28 21:15:01 +00:00
Eelco Dolstra
f958bcdf1f * Added an operator `~' to select paths within a derivation. E.g.,
{stdenv, bash}: derivation {
      builder = bash ~ /bin/sh;
      args = ["-e" "-x" ./builder.sh];
      ...
    }

  Here the attribute `builder' will evaluate to, e.g.,
  `/nix/store/1234abcd...-bash-2.0.1/bin/sh'.
2004-03-28 20:58:28 +00:00
Eelco Dolstra
db3e644c1c * Added plain lambdas, e.g., `let { id = x: x; const = x: y: x; }'.
`bla:' is now no longer parsed as a URL.

* Re-enabled support for the `args' attribute in derivations to
  specify command line arguments to the builder, e.g.,

    ...
    builder = /usr/bin/python;
    args = ["-c" ./builder.py];
    ...
2004-03-28 20:34:22 +00:00
Eelco Dolstra
f8cd904e05 * Disallow the Nix store or any of its parents from being symlinks.
This is because the contents of these symlinks are not incorporated
  into the hashes of derivations, and could therefore cause a mismatch
  between the build system and the target system.  E.g., if
  `/nix/store' is a symlink to `/data/nix/store', then a builder could
  expand this path and store the result.  If on the target system
  `/nix/store' is not a symlink, or is a symlink that points somewhere
  else, we have a dangling pointer.

  The trigger for this change is that gcc 3.3.3 does exactly that (it
  applies realpath() to some files, such as libraries, which causes
  our impurity checker to bail out.)

  An annoying side-effect of this change is that it makes it harder to
  move the Nix store to a different file system.  On Linux, bind
  mounts can be used instead of symlink for this purpose (e.g., `mount
  -o bind /data/nix/store /nix/store').
2004-03-27 17:58:04 +00:00
Eelco Dolstra
f0f7a9f299 * Do not close a nesting level twice after close() has been
called explicitly on a Nest object.
2004-03-27 15:33:19 +00:00
Eelco Dolstra
7823db2137 * Some more nesting. 2004-03-22 21:42:28 +00:00
Eelco Dolstra
777e13b94b * Nix now has three different formats for the log information it
writes to stderr:
  
  - `pretty': the old nested style (default)
  - `escapes': uses escape codes to indicate nesting and message
    level; can be processed using `log2xml'
  - `flat': just plain text, no nesting

  These can be set using `--log-type TYPE' or the NIX_LOG_TYPE
  environment variable.
2004-03-22 20:53:49 +00:00
Eelco Dolstra
79bb0008ec * `null' is a normal form. 2004-03-19 14:45:45 +00:00
Eelco Dolstra
e6253b58cd * Escape codes to force line breaks to be ignored. 2004-03-18 21:32:15 +00:00
Eelco Dolstra
3f3c4cce5a * Added an extra escape code to signal "unimportant" messages. If a tree only has
unimportant messages, it is collapsed by the default.
* Also added an optional integer argument to the escape code for opening a nesting 
  level to indicate lack of importance.  If set, the tree is collapsed by default.
2004-03-18 21:04:14 +00:00
Eelco Dolstra
84c617966b * Collapsable trees. 2004-03-18 18:26:22 +00:00
Eelco Dolstra
c2fc2c13c9 * Use unordered lists, which is more sensible semantically for
representing tree structures.
2004-03-18 14:58:16 +00:00
Eelco Dolstra
a784fd5792 * Don't use tables. Konqueror likes this much better. 2004-03-18 13:04:05 +00:00
Eelco Dolstra
8ce3dd4887 * Display the popup directly over the abbreviation. 2004-03-17 16:55:53 +00:00
Eelco Dolstra
8330c8202a * A simpler way of implementing the store reference popups, thanks to
Martin and CSS guru Martijn Vermaat.
2004-03-17 16:52:48 +00:00
Eelco Dolstra
b5539e7a30 * Store paths are now abbreviated in the generated HTML file.
Hovering over the abbreviated path will reveal the full path.  This
  probably only works in Mozilla.
2004-03-16 12:47:09 +00:00
Eelco Dolstra
9d2669d218 * Added a utility that can be used to produce nice HTML pages from Nix
build logs.  The program `log2xml' converts a Nix build log (read
  from standard input) into XML file that can then be converted to
  XHTML by the `log2html.xsl' stylesheet.  The CSS stylesheet
  `logfile.css' is necessary to make it look good.

  This is primarily useful if the log file has a *tree structure*,
  i.e., that sub-tasks such as the various phases of a build (unpack,
  configure, make, etc.) or recursive invocations of Make are
  represented as such.  While a log file is in principle an
  unstructured plain text file, builders can communicate this tree
  structure to `log2xml' by using escape sequences:

  - "\e[p" starts a new nesting level; the first line following the
    escape code is the header;

  - "\e[q" ends the current nesting level.

  The generic builder in nixpkgs (not yet committed) uses this.  It
  shouldn't be to hard to patch GNU Make to speak this protocol.

  Further improvements to the generated HTML pages are to allow
  collapsing/expanding of subtrees, and to abbreviate store paths (but
  to show the full path by hovering the mouse over it).
2004-03-15 21:51:14 +00:00
Eelco Dolstra
a5619f1dff * Set the NIX_STORE and NIX_BUILD_TOP environment variables in
builders to point to the store and the temporary build directory,
  respectively.  Useful for purity checking.
* Also set TEMPDIR, TMPDIR, TEMP, and TEMP to NIX_BUILD_TOP to make
  sure that tools in the builder store temporary files in the right
  location.
2004-03-12 10:45:08 +00:00
Eelco Dolstra
dbf547645d * Resolve an ambiguity between ifs and attribute selection, e.g., `if
b then x else y.z'.
2004-02-19 13:11:12 +00:00
Eelco Dolstra
0dfdafdf6d * Allow linking against an external Berkeley DB / ATerm library. 2004-02-16 16:37:16 +00:00
Eelco Dolstra
f34de12140 * Allow the location of the store to be specified (--with-store-dir).
* Do not create stuff in localstatedir when doing `make install'
  (since we may not have write access).  In general, installation of
  constant code/data should be separate from the initialisation of
  mutable state.
2004-02-16 15:23:19 +00:00
Eelco Dolstra
fbc48a469c * Inherited attributes in recursive attribute sets are in scope of the
non-inherited attributes.
2004-02-16 09:18:35 +00:00
Eelco Dolstra
76c0e85929 * The environment variable NIX_ROOT can now be set to execute Nix in a
chroot() environment.
* A operation `--validpath' to register path validity.  Useful for
  bootstrapping in a pure Nix environment.
* Safety checks: ensure that files involved in store operations are in
  the store.
2004-02-14 21:44:18 +00:00
Eelco Dolstra
6f5a5ea5ea * Regression fix: realise substitutes and detect cycles. 2004-02-13 10:45:09 +00:00
Eelco Dolstra
00fe1a506f * When creating a new generation, also make the normal form of the
derivation (i.e., the closure store expression) a root of the
  garbage collector.  This ensures that running `nix-collect-garbage
  --no-successors' is safe.
2004-02-11 10:25:31 +00:00
Eelco Dolstra
92e832348d * Lots of manual stuff. Reference pages for most Nix commands.
* nix-pull now requires the full url to the manifest, i.e.,
  `/MANIFEST/' is no longer automatically appended.
* nix-prefetch-url works again.
2004-02-10 16:14:47 +00:00
Eelco Dolstra
6551b36790 * Print what generation we are switching to; honour --dry-run flag. 2004-02-10 13:42:58 +00:00
Eelco Dolstra
618aa69b01 * In --upgrade': added flags --lt', --leq', --always' to specify
whether we want to upgrade if the current version is less than the
  available version (default), when it is less or equal, or always.
  
* Added a flag `--dry-run' to show what would happen in `--install',
  `--uninstall', and `--upgrade', without actually performing the
  operation.
2004-02-09 11:59:39 +00:00
Eelco Dolstra
06a75a7e0c * A command `--switch-generation' to switch to a specific generation
of the current profile, e.g.,

    $ nix-env --list-generations
    ...
    39   2004-02-02 17:53:53
    40   2004-02-02 17:55:18
    41   2004-02-02 17:55:41
    42   2004-02-02 17:55:50   (current)

    $ nix-env --switch-generation 39

    $ ls -l /nix/var/nix/profiles/default
    ... default -> default-39-link

* Also a command `--rollback' which is just a convenience operation to
  rollback to the oldest generation younger than the current one.

  Note that generations properly form a tree.  E.g., if after
  switching to generation 39, we perform an installation action,
  a generation 43 is created which is a descendant of 39, not 42.  So
  a rollback from 43 ought to go back to 39.  This is not currently
  implemented; generations form a linear sequence.
2004-02-08 14:07:43 +00:00
Eelco Dolstra
b8675aee54 * In `--list-generations', show what the current generation is. 2004-02-06 16:16:55 +00:00
Eelco Dolstra
73ab2ed4fd * A command `--list-generations' to show all generations for a
profile.
2004-02-06 16:03:27 +00:00
Eelco Dolstra
7c0fa4474f * More refactoring. 2004-02-06 14:57:10 +00:00
Eelco Dolstra
7abf9911d9 * Refactoring. 2004-02-06 14:49:41 +00:00
Eelco Dolstra
49bafe1faf * Use the profile pointed to by ~/.nix-profile if no --profile
argument is specified.
2004-02-06 10:59:06 +00:00
Eelco Dolstra
66e94d3275 * Improvements to profiles. Generations are now per-profile, e.g.,
default -> default-94-link
  default-82-link -> /nix/store/cc4480...
  default-83-link -> /nix/store/caeec8...
  ...
  default-94-link -> /nix/store/2896ca...
  experimental -> experimental-2-link
  experimental-1-link -> /nix/store/cc4480...
  experimental-2-link -> /nix/store/a3148f...

* `--profile' / `-p' -> `--switch-profile' / `-S'
* `--link' / `-l' -> `--profile' / `-p'
* The default profile is stored in $prefix/var/nix/profiles.
  $prefix/var/nix/links is gone.  Profiles can be stored anywhere.
* The current profile is now referenced from ~/.nix-profile, not
  ~/.nix-userenv.
* The roots to the garbage collector now have extension `.gcroot', not
  `.id'.
2004-02-06 10:30:20 +00:00
Eelco Dolstra
d445da7a7b * Extended the `inherit' syntax to optionally select attributes from
other attribute sets, rather than the current scope.  E.g.,
  
    {inherit (pkgs) gcc binutils;}

  is equivalent to

    {gcc = pkgs.gcc; binutils = pkgs.binutils;}

  I am not so happy about the syntax.
2004-02-04 17:23:26 +00:00
Eelco Dolstra
9d25466b34 * An attribute set update operator (//). E.g.,
{x=1; y=2; z=3;} // {y=4;}  =>  {x=1; y=4; z=3;}
2004-02-04 16:49:51 +00:00
Eelco Dolstra
9b44480612 * Use a map to lookup primops.
* Various performance improvements in the evaluator.
* Do not link against unused (and missing!) libraries (-lsglr, etc.).
2004-02-04 16:03:29 +00:00
Eelco Dolstra
c4f7ae4aa5 * Verify that all variables in a Nix expression are defined. 2004-02-03 14:45:34 +00:00
Eelco Dolstra
1c9c0a5a46 * Added syntactic sugar to the construction of attribute sets to
`inherit' variables from the surrounding lexical scope.

  E.g.,

    {stdenv, libfoo}: derivation {
      builder = ./bla;
      inherit stdenv libfoo;
      xyzzy = 1;
    }

  is equivalent to

    {stdenv, libfoo}: derivation {
      builder = ./bla;
      stdenv = stdenv;
      libfoo = libfoo;
      xyzzy = 1;
    }

  Note that for mutually recursive attribute set definitions (`rec
  {...}'), this also works, that is, `rec {inherit x;}' is equivalent
  to `let {fresh = x; body = rec {x = fresh;};}', *not*
  `rec {x = x}'.
2004-02-02 21:39:33 +00:00
Eelco Dolstra
d9f30fe7c7 * Sort `nix-env -q' output by derivation name.
* `--version' flag for all commands.
* Manual updates.
2004-02-02 10:51:54 +00:00
Eelco Dolstra
47c003cb59 * Doh! 2004-01-30 17:14:08 +00:00
Eelco Dolstra
619f20775d * Parser numbers again.
* Include missing files in distributions.
2004-01-30 17:06:03 +00:00
Eelco Dolstra
c625718513 * Detect flex and bison; updated the manual. 2004-01-30 16:32:14 +00:00
Eelco Dolstra
c5baaafae6 * Replaced the SDF parser by a substantially faster Bison/Flex
parser (roughly 80x faster).

  The absolutely latest version of Bison (1.875c) is required for
  reentrant GLR support, as well as a recent version of Flex (say,
  2.5.31).  Note that most Unix distributions ship with the
  prehistoric Flex 2.5.4, which doesn't support reentrancy.
2004-01-30 15:21:42 +00:00
Eelco Dolstra
abd1878b26 * Optimised the SDF grammar. 2004-01-29 14:24:53 +00:00
Eelco Dolstra
840551ebdb * Extra bit S' in --query --status' output: show whether there are
any substitutes for the derivation.
2004-01-21 16:41:17 +00:00
Eelco Dolstra
1109ea0680 * Fixed a subtle uninitialised variable bug in ATermMaps copied from
ATermMaps.  Found thanks to Valgrind!
2004-01-21 14:49:32 +00:00
Eelco Dolstra
47f19b6293 * Absolutise the specified path in --import' and --profile'. 2004-01-20 20:36:58 +00:00
Eelco Dolstra
291030b900 * Remove debug message. 2004-01-15 20:58:44 +00:00
Eelco Dolstra
447089a5f6 * Catch SIGINT to terminate cleanly when the user tries to interrupt
Nix.  This is to prevent Berkeley DB from becoming wedged.

  Unfortunately it is not possible to throw C++ exceptions from a
  signal handler.  In fact, you can't do much of anything except
  change variables of type `volatile sig_atomic_t'.  So we set an
  interrupt flag in the signal handler and check it at various
  strategic locations in the code (by calling checkInterrupt()).
  Since this is unlikely to cover all cases (e.g., (semi-)infinite
  loops), sometimes SIGTERM may now be required to kill Nix.
2004-01-15 20:23:55 +00:00
Eelco Dolstra
08719c6c97 * Obsolete. 2004-01-15 20:13:54 +00:00
Eelco Dolstra
55e11bc0d3 * In nix-env --query --status', determine the I' bit by looking at
the output path of a derivation, not the path of its store
  expression.  This ensures that changes that affect the path of the
  store expression but not the output path, do not affect the
  `installed' state of a derivation.
2004-01-15 14:43:00 +00:00
Eelco Dolstra
ff9af107d3 * Option `-B' to always show the output of builders, regardless of
verbosity level.
2004-01-13 16:35:43 +00:00
Eelco Dolstra
3495d153b3 * Periodically checkpoint the log. 2004-01-13 13:37:25 +00:00
Eelco Dolstra
698e880c9f * Tricky: make sure that the accessor count is not reset to 0 if
recovery fails.
2004-01-13 12:36:43 +00:00
Eelco Dolstra
23fbc72f5d * Print error messages, not debug messages. 2004-01-13 11:53:12 +00:00
Eelco Dolstra
4c4fe7a114 * Changed the extension for store expressions from ".nix" to ".store"
(following the Usenix paper).
2004-01-12 10:44:48 +00:00
Eelco Dolstra
46a71c857c * Option --force-realise' in nix-store --query'. 2004-01-09 14:18:28 +00:00
Eelco Dolstra
7959354379 * Upgraded to Berkeley DB 4.2.52. The main advantage of 4.2 is that
it automatically removes log files when they are no longer needed.

  *** IMPORTANT ***

  If you have an existing Nix installation, you must checkpoint the
  Nix database to prevent recent transactions from being undone.  Do
  the following:

  - optional: make a backup of $prefix/var/nix/db.

  - run `db_checkpoint' from Berkeley DB 4.1:
  
    $ db_checkpoint -h $prefix/var/nix/db -1

  - optional (?): run `db_recover' from Berkeley DB 4.1:

    $ db_recover -h $prefix/var/nix/db

  - remove $prefix/var/nix/db/log* and $prefix/var/nix/db/__db*
2004-01-07 15:53:14 +00:00
Eelco Dolstra
4a373a3e9a * Implemented Eelco V.'s `nix-env -I' command to specify the default
path of the Nix expression to be used with the import, upgrade, and
  query commands.  For instance,

  $ nix-env -I ~/nixpkgs/pkgs/system/i686-linux.nix

  $ nix-env --query --available   [aka -qa]
  sylpheed-0.9.7
  bison-1.875
  pango-1.2.5
  subversion-0.35.1
  ...

  $ nix-env -i sylpheed

  $ nix-env -u subversion

  There can be only one default at a time.

* If the path to a Nix expression is a symlink, follow the symlink
  prior to resolving relative path references in the expression.
2004-01-05 16:26:43 +00:00
Eelco Dolstra
f83c5e3e5f * Implemented Eelco V.'s `-p' command to switch profiles. It switches
the symlink ~/.nix-userenv to the given argument (which defaults to
  .../links/current).  /etc/profile.d/nix-profile creates this symlink
  if it doesn't exist yet.  Example use:

  $ nix-env -l my_profile -i foo.nix subversion quake
  $ nix-env -p my_profile

  I don't like the term "profile".  Let's deprecate it :-)
2004-01-05 11:18:59 +00:00
Eelco Dolstra
94175e978a * RPM spec file.
* Respect DESTDIR variable.
2003-12-30 20:09:00 +00:00
Eelco Dolstra
392b7e0f8e * Fixed a bug in the upgrade operation. 2003-12-23 22:13:36 +00:00
Eelco Dolstra
833f2fc92d * GCC 2.95 compatibility. 2003-12-22 16:40:46 +00:00
Eelco Dolstra
cf0287c09e * Upgrade operation in `nix-env'. For instance, you can say
nix-env -u foo.nix strategoxt

  to replace the installed `strategoxt' derivation with the one from `foo.nix', if 
  the latter has a higher version number.  This is a no-op if `strategoxt' is not 
  installed.  Wildcards are also accepted, so

    nix-env -u foo.nix '*'

  will replace any installed derivation with newer versions from `foo.nix', if 
  available.

  The notion of "version number" is somewhat ad hoc, but should be useful in most 
  cases, as evidenced by the following unit tests for the version comparator:

    TEST("1.0", "2.3", -1);
    TEST("2.1", "2.3", -1);
    TEST("2.3", "2.3", 0);
    TEST("2.5", "2.3", 1);
    TEST("3.1", "2.3", 1);
    TEST("2.3.1", "2.3", 1);
    TEST("2.3.1", "2.3a", 1);
    TEST("2.3pre1", "2.3", -1);
    TEST("2.3pre3", "2.3pre12", -1);
    TEST("2.3a", "2.3c", -1);
    TEST("2.3pre1", "2.3c", -1);
    TEST("2.3pre1", "2.3q", -1);

  (-1 = less, 0 = equal, 1 = greater)

* A new verbosity level `lvlInfo', between `lvlError' and `lvlTalkative'.  This is 
  the default for `nix-env', so without any `-v' flags users should get useful 
  output, e.g.,

$ nix-env -u foo.nix strategoxt
upgrading `strategoxt-0.9.2' to `strategoxt-0.9.3'
2003-12-22 16:04:00 +00:00
Eelco Dolstra
f3c9783846 * Version numbers can be omitted in install/uninstall. E.g.,
nix-env -i foo.nix subversion

  The version number part of a derivation name is defined as everything following the 
  first dash not followed by a letter.
2003-12-21 23:58:56 +00:00
Eelco Dolstra
a81b621202 * -u' -> -e'.
* `--link' / `-l' flag to specify the switch symlink to use (by default, 
  /nix/var/nix/links/current).
2003-12-21 22:34:41 +00:00
Eelco Dolstra
528f1d1867 * Bug fix: parallel builds of the same derivation failed due to lock file removal. 2003-12-21 17:09:16 +00:00
Eelco Dolstra
cff6fd22eb * Allow successors that don't exist but have a substitute.
* Integrity: check in successor / substitute registration whether
  the target path exists or has a substitute.
2003-12-05 11:05:19 +00:00
Eelco Dolstra
0d3a1a8582 * Add missing files to `make dist'. 2003-12-02 12:37:37 +00:00
Eelco Dolstra
0c804c6775 * Regression fix: query flags (e.g., "-qsf") were broken. 2003-12-02 10:21:40 +00:00
Eelco Dolstra
078e20885e * Help text for all (non-script) programs, so no more:
$ nix-instantiate --help
error: unknown flag `--help`
Try `nix-instantiate --help' for more information.

:-)
2003-12-01 15:55:05 +00:00
Eelco Dolstra
83ffd4f282 * Fix `make check'. 2003-12-01 14:40:24 +00:00
Eelco Dolstra
5d2b424804 * Use a system name that does not include the OS manufacturer (i.e.,
"i686-linux" instead of "i686-suse-linux").
2003-12-01 14:36:50 +00:00
Eelco Dolstra
ba73f94b3b * Another fix. 2003-11-25 13:03:48 +00:00
Eelco Dolstra
66c115ef5f * More `make dist' fixes. 2003-11-25 13:01:21 +00:00
Eelco Dolstra
c3ee8c9166 * `make dist' fix. 2003-11-25 12:35:52 +00:00
Eelco Dolstra
6e8c19714a * Allow integer bindings in derivations. 2003-11-25 12:05:48 +00:00
Eelco Dolstra
b857267893 * Allow the top-level expression to be a derivation.
* Hack: `nix-env -i *' installs all available derivations.
2003-11-24 11:01:19 +00:00