Commit graph

1727 commits

Author SHA1 Message Date
Eelco Dolstra
efe4b690ae * Refactoring: combine functions that take an attribute set and
functions that take a single argument (plain lambdas) into one AST
  node (Function) that contains a Pattern node describing the
  arguments.  Current patterns are single lazy arguments (VarPat) and
  matching against an attribute set (AttrsPat).

  This refactoring allows other kinds of patterns to be added easily,
  such as Haskell-style @-patterns, or list pattern matching.
2008-08-14 10:04:22 +00:00
Eelco Dolstra
c03b729319 * Increase the sleep periods a bit to make the test less likely to
fail on slow machines.  Of course it would be better if this test
  wasn't timing dependent...
2008-08-14 09:26:30 +00:00
Eelco Dolstra
5664b6d7ba * Removed the "valid values" feature. Nobody uses it anyway. 2008-08-11 13:36:40 +00:00
Eelco Dolstra
98b07466fb * Better error checking of the data from the substituters. 2008-08-05 10:57:53 +00:00
Eelco Dolstra
339c142009 * Use optimistic profile locking for nix-env operations like `-i' and
`-u'.  Instead of acquiring an exclusive lock on the profile for the
  entire duration of the operation, we just perform the operation
  optimistically (without an exclusive lock), and check at the end
  whether the profile changed while we were busy (i.e., the symlink
  target changed).  If so, the operation is restarted.  Restarting is
  generally cheap, since the build results are still in the Nix store.
  Most of the time, only the user environment has to be rebuilt.
2008-08-04 16:21:45 +00:00
Eelco Dolstra
001b3f06ec * `nix-env --set': support --dry-run. 2008-08-04 14:58:50 +00:00
Eelco Dolstra
a1d310b6b5 * `nix-store --realise': print what paths will be built/downloaded,
just like nix-env.
* `nix-store --realise': --dry-run option.
2008-08-04 13:44:46 +00:00
Eelco Dolstra
5adbb0aabe * build.cc: only use a substituter if it returns info for a path. 2008-08-04 13:15:35 +00:00
Eelco Dolstra
5b1052663a * Always show what paths we're going to build/download (as in
--dry-run).  Maybe there should be an option to turn this on/off?
2008-08-04 13:11:09 +00:00
Eelco Dolstra
c4f98941ed * nix-env --dry-run: show the total size of the substituter
downloads.
2008-08-04 12:29:04 +00:00
Eelco Dolstra
03427e76f1 * querySubstitutablePathInfo: work properly when run via the daemon.
* --dry-run: print the paths that we don't know how to build/substitute.
2008-08-04 11:44:50 +00:00
Eelco Dolstra
3c92ea399d * Make nix-env --dry-run print the paths to be substituted correctly
again.  (After the previous substituter mechanism refactoring I
  didn't update the code that obtains the references of substitutable
  paths.)  This required some refactoring: the substituter programs
  are now kept running and receive/respond to info requests via
  stdin/stdout.
2008-08-02 12:54:35 +00:00
Eelco Dolstra
fc691e1cbd * Print a better error message when a non-derivation attribute set is
coerced to a string.
2008-07-24 14:52:25 +00:00
Eelco Dolstra
989176c56e * Allow read-only access to the store (e.g., non-root users on NixOS
can do operations like "nix-store -qR <path>" even without the Nix
  daemon).
2008-07-18 15:34:46 +00:00
Eelco Dolstra
8bc591a6f0 * Use the copy-from-other-stores substituter by default. Of course,
it only does something if $NIX_OTHER_STORES (not really a good
  name...) is set.
* Do globbing on the elements of $NIX_OTHER_STORES.  E.g. you could
  set it to /mnts/*/nix or something.
* Install substituters in libexec/nix/substituters.
2008-07-18 13:05:10 +00:00
Eelco Dolstra
7cd88b1dec * Generalised the dependencyClosure primop to builtins.genericClosure,
which is hopefully more useful.
* New primops: length, mul, div.
2008-07-11 13:29:04 +00:00
Eelco Dolstra
d567baabbd * Export the nix-env derivation name parsing and version comparison
logic through the `parseDrvName' and `compareVersions' primops.
  This will allow expressions to easily check whether some dependency
  is a specific needed version or falls in some version range.  See
  tests/lang/eval-okay-versions.nix for examples.
2008-07-01 10:10:32 +00:00
Eelco Dolstra
6c8641a542 2008-06-18 19:17:05 +00:00
Eelco Dolstra
5af84139a8 * --max-freed: support values >= 4 GB. 2008-06-18 15:20:33 +00:00
Eelco Dolstra
d3aa183beb * Garbage collector: option `--max-freed' to stop after at least N
bytes have been freed, `--max-links' to stop when the Nix store
  directory has fewer than N hard links (the latter being important
  for very large Nix stores on filesystems with a 32000 subdirectories
  limit).
2008-06-18 14:20:16 +00:00
Eelco Dolstra
a8f3b02092 * `nix-store --optimise': handle files with >= 32000 hard links.
(There can easily be more than 32000 occurrences of the empty file.)
2008-06-18 14:13:00 +00:00
Eelco Dolstra
a72709afd8 * Some refactoring: put the GC options / results in separate structs.
* The garbage collector now also prints the number of blocks freed.
2008-06-18 09:34:17 +00:00
Eelco Dolstra
f351834f77 * nix-worker: clean up the temporary root for the worker processes
in /nix/var/nix/temproots.
2008-06-14 16:03:02 +00:00
Eelco Dolstra
94fd46fa1c * Note. 2008-06-14 16:02:31 +00:00
Eelco Dolstra
826b271d9a * Garbage collector: don't do a complete topological sort of the Nix
store under the reference relation, since that means that the
  garbage collector will need a long time to start deleting paths.
  Instead just delete the referrers of a path first.
2008-06-13 18:25:24 +00:00
Eelco Dolstra
30c9f909b2 * Print some progress info during the early GC stages. 2008-06-13 17:21:20 +00:00
Eelco Dolstra
b0e92f6d47 * Merged the no-bdb branch (-r10900:HEAD
https://svn.nixos.org/repos/nix/nix/branches/no-bdb).
2008-06-09 13:52:45 +00:00
Sander van der Burg
c41a3ec3a9 First attempt to update Nix SDF grammar to match the actual bison grammar 2008-06-04 14:36:46 +00:00
Eelco Dolstra
bd955e15e1 * GCC 4.3.0 (Fedora 9) compatibility fixes. Reported by Gour and
Armijn Hemel.
2008-05-21 11:17:31 +00:00
Eelco Dolstra
72034ab35d * sockaddr_un doesn't allow path names of more than 108 characters.
This isn't usually a problem, except that it causes tests to fail
  when performed in a directory with a very long path name.  So chdir
  to the socket directory and use a relative path name.
2008-04-09 05:57:01 +00:00
Eelco Dolstra
329025253d * Use /tmp/nix-build-<drvpath>-<counter> instead of
/tmp/nix-<pid>-<counter> for temporary build directories.  This
  increases purity a bit: many packages store the temporary build path
  in their output, causing (generally unimportant) binary differences.
2008-03-27 13:45:17 +00:00
Eelco Dolstra
98968fbb63 * Disable the don't-run-as-root sanity check because it breaks RPM
builds (which are done as root...).
2008-03-20 18:15:20 +00:00
Eelco Dolstra
2f1e2cf632 * Note that the SDF grammar isn't used. 2008-03-20 14:59:33 +00:00
Eelco Dolstra
f789ea1d09 * A transaction is unnecessary since the path cannot be
garbage-collected (it's a temporary root).
2008-03-01 21:05:33 +00:00
Eelco Dolstra
0a84137c45 * checkVarDefs: don't check in closed terms, which don't have
undefined variables by definition.  This matters for the
  implementation of "with", which does a call to checkVarDefs to see
  if the body of the with has no undefined variables.  (It can't be
  checked at parse time because you don't know which variables are in
  the "with" attribute set.)  If we check closed terms, then we check
  not just the with body but also the substituted terms, which are
  typically very large.  This is the cause of the poor nix-env
  performance on Nixpkgs lately.  It didn't happen earlier because
  "with" wasn't used very often in the past.

  This fix improves nix-env performance roughly 60x on current Nixpkgs.
  nix-env -qa is down from 29.3s to 0.5s on my laptop, and nix-env -qa
  --out-path is down from 229s to 3.39s.  Not bad for a 1-line fix :-)
2008-02-21 12:01:24 +00:00
Eelco Dolstra
0ed89c569f * Fix the parsing of
''
      '${foo}'
    ''
    
  where the antiquote should work as expected, instead of giving the
  string "'${foo}'".
2008-02-05 13:38:07 +00:00
Eelco Dolstra
66c51dc215 * nix-store --dump-db / --load-db to dump/load the Nix DB.
* nix-store --register-validity: option to supply the content hash of
  each path.
* Removed compatibility with Nix <= 0.7 stores.
2008-01-29 18:17:36 +00:00
Michael Raskin
5b5a3af983 Probably fixed __exprToString 2008-01-20 20:44:03 +00:00
Michael Raskin
5eb5c23447 Fixed exportBuildReferenceGraph 2008-01-15 04:32:08 +00:00
Eelco Dolstra
7d0f6aed59 * New primop `unsafeDiscardStringContext' to get rid of string
contexts.  Needed to prevent unnecessary dependencies when building
  the NixOS manual.
2008-01-04 14:22:49 +00:00
Eelco Dolstra
e0ca671491 * More documentation. 2007-12-31 02:52:17 +00:00
Eelco Dolstra
dedd62dd0c * More release notes. 2007-12-31 00:08:09 +00:00
Michael Raskin
2196cb67c5 Added a way to include derivation with its outputs and dependencies - exportBuildReferenceGraph 2007-12-30 09:30:56 +00:00
Eelco Dolstra
df303666bc * Use strsignal if available to give better error messages for
builders that fail due to a signal.
2007-12-14 14:49:35 +00:00
Eelco Dolstra
30e4653783 * Mac OS X 10.5 compatibility. 2007-12-13 22:06:19 +00:00
Michael Raskin
3a2833daff corrected --help message for nix-store a bit 2007-12-10 22:16:47 +00:00
Eelco Dolstra
b42ef9c054 * Syntax to escape '', ${. 2007-12-06 10:20:58 +00:00
Eelco Dolstra
6d6c68c0d2 * Added a new kind of multi-line string literal delimited by two
single quotes.  Example (from NixOS):

    job = ''
      start on network-interfaces

      start script

        rm -f /var/run/opengl-driver
        ${if videoDriver == "nvidia"        
          then "ln -sf ${nvidiaDrivers} /var/run/opengl-driver"
          else if cfg.driSupport
          then "ln -sf ${mesa} /var/run/opengl-driver"
          else ""
        }

        rm -f /var/log/slim.log

      end script
    '';

  This style has two big advantages:

  - \, ' and " aren't special, only '' and ${.  So you get a lot less
    escaping in shell scripts / configuration files in Nixpkgs/NixOS.
    The delimiter '' is rare in scripts (and can usually be written as
    "").  ${ is also fairly rare.

    Other delimiters such as <<...>>, {{...}} and <|...|> were also
    considered but this one appears to have the fewest drawbacks
    (thanks Martin).

  - Indentation is intelligently stripped so that multi-line strings
    can follow the nesting structure of the containing Nix
    expression.  E.g. in the example above 6 spaces are stripped from
    the start of each line.  This prevents unnecessary indentation in
    generated files (which sometimes even breaks things).

  See tests/lang/eval-okay-ind-string.nix for some examples.
2007-11-30 16:48:45 +00:00
Eelco Dolstra
633518628f * nix-env -e: support uninstalling by path, so that one can say
$ nix-env -e $(which firefox)

  or

    $ nix-env -e /nix/store/nywzlygrkfcgz7dfmhm5xixlx1l0m60v-pan-0.132

* nix-env -i: if an argument contains a slash anywhere, treat it as a
  path and follow it through symlinks into the Nix store.  This allows
  things like

    $ nix-build -A firefox
    $ nix-env -i ./result

* nix-env -q/-i/-e: don't complain when the `*' selector doesn't match
  anything.  In particular, `nix-env -q \*' doesn't fail anymore on an
  empty profile.
2007-11-29 16:18:24 +00:00
Eelco Dolstra
06f95dd07c * New primop `readFile' to get the contents of a file as a string. 2007-11-21 13:49:59 +00:00
Eelco Dolstra
c370755583 * Flag `--no-build-hook' to disable distributed builds.
* queryDeriver in daemon mode: don't barf if the other side returns an
  empty string (which means there is no deriver).
2007-11-16 16:15:26 +00:00
Eelco Dolstra
c05783ad67 * nix-store --import: add a flag --require-signature. 2007-11-16 16:10:27 +00:00
Eelco Dolstra
149e273023 * Don't need gc.hh. 2007-11-15 15:07:27 +00:00
Michael Raskin
6fc60e2060 Added possibility to specify garbage collection levels for store paths; so packages can share intermediate results of compilation and GC will collect it automatically while never touching tarballs, for example. 2007-11-15 03:47:12 +00:00
Eelco Dolstra
fa44e401a8 * Documented multi-user Nix. 2007-10-31 18:01:56 +00:00
Eelco Dolstra
bfea7b1f35 * Make the --prebuilt-only' / -b' option work not just for queries
but installations/upgrades as well.  So `nix-env -ub \*' will
  upgrade only those packages for which a substitute is available (or
  to be precise, it will upgrade each package to the highest version
  for which a substitute is available).
2007-10-29 14:31:45 +00:00
Eelco Dolstra
42d80d1170 * On FreeBSD, sys/mount.h needs sys/param.h. 2007-10-29 10:03:07 +00:00
Eelco Dolstra
d91cd30563 * Detect whether chroot / bind-mount support is available. 2007-10-27 16:51:55 +00:00
Eelco Dolstra
dc6f373842 * Delete the chroot directory automatically.
* Removed some debug messages.
2007-10-27 16:06:38 +00:00
Eelco Dolstra
9397cd30c8 * Support for doing builds in a chroot under Linux. The builder is
executed in a chroot that contains just the Nix store, the temporary
  build directory, and a configurable set of additional directories
  (/dev and /proc by default).  This allows a bit more purity
  enforcement: hidden build-time dependencies on directories such as
  /usr or /nix/var/nix/profiles are no longer possible.  As an added
  benefit, accidental network downloads (cf. NIXPKGS-52) are prevented
  as well (because files such as /etc/resolv.conf are not available in
  the chroot).

  However the usefulness of chroots is diminished by the fact that
  many builders depend on /bin/sh, so you need /bin in the list of
  additional directories.  (And then on non-NixOS you need /lib as
  well...)
2007-10-27 00:46:59 +00:00
Eelco Dolstra
0b4ed64d29 * "trace" primop: write the trace to standard error. 2007-10-26 18:25:50 +00:00
Eelco Dolstra
f90d85107e 2007-10-22 12:05:30 +00:00
Eelco Dolstra
14e47e9c2c * Doh. 2007-10-10 14:58:00 +00:00
Eelco Dolstra
d04291cfab * Doh! Don't change the permissions on /nix/store. 2007-10-10 13:52:38 +00:00
Eelco Dolstra
315183f194 * nix-store --optimise: flag "--dry-run" to just query what the disk
savings would be.
2007-10-10 13:43:04 +00:00
Eelco Dolstra
a8629de827 * New command `nix-store --optimise' to reduce Nix store disk space
usage by finding identical files in the store and hard-linking them
  to each other.  It typically reduces the size of the store by
  something like 25-35%.  This is what the optimise-store.pl script
  did, but the new command is faster and more correct (it's safe wrt
  garbage collection and concurrent builds).
2007-10-09 22:14:27 +00:00
Eelco Dolstra
27a0662828 * listToAttrs: the list now should consist of {name, value} attribute
sets instead of {attr, value}.  "name" is better than "attr" because
  the *combination* of the two forms the attribute.
2007-10-09 12:51:25 +00:00
Eelco Dolstra
27f57c9018 * Ignore dangling symlinks in ~/.nix-defexpr. 2007-09-18 14:01:14 +00:00
Eelco Dolstra
f3441e6122 * Pass various options to the worker so that flags like -K or -j work
in multi-user Nix (NIX-72).
* Client/worker: exchange a protocol version number for future
  compatibility.
2007-09-18 09:11:20 +00:00
Eelco Dolstra
c8ea8a09b8 * nix-env -qa: make the "-A" flag do the expected thing, namely follow
the given attribute path (just as -A does with other option)
  (NIX-83).  So you can now say

  $ nix-env -qa -A nixpkgs_unstable.gnome \*
  atk-1.12.4
  esound-0.2.36
  ...

  to see the packages in the "gnome" attribute in Nixpkgs.

  To *print* the attribute path, you should now use "--attr-path" /
  "-P" (running out of letters...).
2007-09-17 19:24:07 +00:00
Eelco Dolstra
055608227f * nix-env: allow ~/.nix-defexpr to be a directory. If it is, then the
Nix expressions in that directory are combined into an attribute set
  {file1 = import file1; file2 = import file2; ...}, i.e. each Nix
  expression is an attribute with the file name as the attribute
  name.  Also recurses into directories.

* nix-env: removed the "--import" (-I) option which set the
  ~/.nix-defexpr symlink.

* nix-channel: don't use "nix-env --import", instead symlink
  ~/.nix-defexpr/channels.  So finally nix-channel --update doesn't
  override any default Nix expressions but combines with them.

  This means that you can have (say) a local Nixpkgs SVN tree and use
  it as a default for nix-env:

  $ ln -s .../path-to-nixpkgs-tree ~/.nix-defexpr/nixpkgs_svn

  and be subscribed to channels (including Nixpkgs) at the same time.
  (If there is any ambiguity, the -A flag can be used to
  disambiguate, e.g. "nix-env -i -A nixpkgs_svn.pan".)
2007-09-17 16:08:24 +00:00
Eelco Dolstra
0d65fc08e2 * Create the Nix daemon socket in a separate directory
(/nix/var/nix/daemon-socket).  This allows access to the Nix daemon
  to be restricted by setting the mode/ownership on that directory as
  desired, e.g.

    $ chmod 770 /nix/var/nix/daemon-socket
    $ chown root.wheel /nix/var/nix/daemon-socket

  to allow only users in the wheel group to use Nix.

  Setting the ownership on a socket is much trickier, since the socket
  must be deleted and recreated every time the daemon is started
  (which would require additional Nix configuration file directives to
  specify the mode/ownership, and wouldn't support arbitrary ACLs),
  some BSD variants appear to ignore permissions on sockets, and it's
  not clear whether the umask is respected on every platform when
  creating sockets.
2007-08-30 09:50:44 +00:00
Eelco Dolstra
cb1c1004cd * When there are multiple substituters, make sure to release the
lock on the output path after trying each.  Otherwise the
  pathIsLockedByMe() test gets confused.
2007-08-28 16:22:08 +00:00
Eelco Dolstra
c970b28ba0 * Fix a race condition with parallel builds where multiple
fixed-output derivations or substitutions try to build the same
  store path at the same time.  Locking generally catches this, but
  not between multiple goals in the same process.  This happened
  especially often (actually, only) in the build farm with fetchurl
  downloads of the same file being executed on multiple machines and
  then copied back to the main machine where they would clobber each
  other (NIXBF-13).

  Solution: if a goal notices that the output path is already locked,
  then go to sleep until another goal finishes (hopefully the one
  locking the path) and try again.
2007-08-28 11:36:17 +00:00
Eelco Dolstra
e0d7e47862 * PathLocks::lockPaths: don't allow reacquiring a lock we already
hold.
2007-08-28 09:39:03 +00:00
Marc Weber
2629998e91 primop functions listToAttrs (+test), __isAttrs, __trace added
new configuration style proposal in lib/default-unstable.nix
2007-08-18 22:12:00 +00:00
Eelco Dolstra
3757ee589f * Bump the Nix database schema version number; delete the substitutes
table.
2007-08-13 11:37:39 +00:00
Eelco Dolstra
59afc1a15c * Backwards compatibility. 2007-08-13 11:15:13 +00:00
Eelco Dolstra
9e975458b4 * Get rid of the substitutes database table (NIX-47). Instead, if we
need any info on substitutable paths, we just call the substituters
  (such as download-using-manifests.pl) directly.  This means that
  it's no longer necessary for nix-pull to register substitutes or for
  nix-channel to clear them, which makes those operations much faster
  (NIX-95).  Also, we don't have to worry about keeping nix-pull
  manifests (in /nix/var/nix/manifests) and the database in sync with
  each other.

  The downside is that there is some overhead in calling an external
  program to get the substitutes info.  For instance, "nix-env -qas"
  takes a bit longer.

  Abolishing the substitutes table also makes the logic in
  local-store.cc simpler, as we don't need to store info for invalid
  paths.  On the downside, you cannot do things like "nix-store -qR"
  on a substitutable but invalid path (but nobody did that anyway).

* Never catch interrupts (the Interrupted exception).
2007-08-12 00:29:28 +00:00
Eelco Dolstra
d71cc503a6 * Don't allocate input files on the stack. 2007-08-07 15:00:13 +00:00
Eelco Dolstra
4cad125e08 * Optionally warn about packages that give an assertion failure. 2007-08-06 16:08:09 +00:00
Eelco Dolstra
6da6fbfbe9 * Properly keep packages during upgrades. 2007-07-05 20:33:04 +00:00
Eelco Dolstra
6d1a1191b0 * Support queryDeriver() in multi-user installations. 2007-06-12 16:53:44 +00:00
Eelco Dolstra
9bff7ad728 * Check against creation of GC roots in the store. Those roots don't
work, because findRoots() stops when it encounters a symlink to the
  store.  And of course the store is supposed to be read-only.
2007-06-11 11:36:22 +00:00
Eelco Dolstra
bddc83a148 * New builtin function "isFunction". You're not supposed to use it
;-)
* Channels: fix channels that are plain lists of derivations (like
  strategoxt-unstable) instead  of functions (like nixpkgs-unstable).
  This fixes the error message "error: the left-hand side of the
  function call is neither a function nor a primop (built-in
  operation) but a list".
2007-05-16 16:17:04 +00:00
Eelco Dolstra
ca00aa1171 * Allow empty argument lists in function definitions (e.g., `{}:
bla').  Also allow trailing commas (`{x, y,}: ...') as a unintented
  consequence.  Hopefully the reduce/reduce conflict won't cause any
  problems.
2007-05-15 12:14:37 +00:00
Eelco Dolstra
7046c35843 * Typo (reported by Marc Weber). 2007-05-14 12:16:41 +00:00
Eelco Dolstra
e5da9c8803 * Create the database directory if it doesn't exist. 2007-05-07 16:59:19 +00:00
Eelco Dolstra
93aefd9fc0 * Give unpacked channels more sensible names than 0, 1, ... They now
get the basename of the channel URL (e.g., nixpkgs-unstable).  The
  top-level Nix expression of the channel is now an attribute set, the
  attributes of which are the individual channels (e.g.,
  {nixpkgs_unstable = ...; strategoxt_unstable = ...}).  This makes
  attribute paths ("nix-env -qaA" and "nix-env -iA") more sensible,
  e.g., "nix-env -iA nixpkgs_unstable.subversion".
2007-05-01 23:16:38 +00:00
Eelco Dolstra
a9d15d4f43 * nix-env -i: instead of breaking package ties by version, break them
by priority and version install.  That is, if there are multiple
  packages with the same name, then pick the package with the highest
  priority, and only use the version if there are multiple packages
  with the same priority.

  This makes it possible to mark specific versions/variant in Nixpkgs
  more or less desirable than others.  A typical example would be a
  beta version of some package (e.g., "gcc-4.2.0rc1") which should not
  be installed even though it is the highest version, except when it
  is explicitly selected (e.g., "nix-env -i gcc-4.2.0rc1").

* Idem for nix-env -u, only the semantics are a bit trickier since we
  also need to take into account the priority of the currently
  installed package (we never upgrade to a lower priority, unless
  --always is given).
2007-05-01 20:33:18 +00:00
Eelco Dolstra
cbfac2fdcc * Set a terminate() handler to ensure that we leave the BDB
environment cleanly even when an exception is thrown from a
  destructor.  We still crash, but we don't take all other Nix
  processes with us.
2007-05-01 15:16:17 +00:00
Eelco Dolstra
644946feed * Make --verify more interruptable. 2007-05-01 13:21:05 +00:00
Eelco Dolstra
bd1f66453a * `nix-env -q --xml --meta' to show all meta attributes. 2007-05-01 11:30:52 +00:00
Eelco Dolstra
e20f0da22c * Doh. 2007-04-30 18:41:27 +00:00
Eelco Dolstra
89c1d2b202 * Package flag "keep" that prevents a package from being removed from
a user environment by an install or upgrade action.  This is
  particularly useful if you have a version installed that you don't
  want to upgrade (e.g., because the newer versions are broken).
  Example:

    $ nix-env -u zapping --dry-run
    (dry run; not doing anything)
    upgrading `zapping-0.9.6' to `zapping-0.10cvs6'

    $ nix-env --set-flag keep true zapping

    $ nix-env -u zapping --dry-run
    (dry run; not doing anything)

  However, "-e" will still uninstall the package.  (Maybe we should
  require the keep flag to be explicitly set to false before it can be
  uninstalled.)
2007-04-27 23:48:14 +00:00
Eelco Dolstra
b7f0f65c19 * nix-env -q now has a flag --prebuilt-only (-b<) that causes nix-env
to show only those derivations whose output is already in the Nix
  store or that can be substituted (i.e., downloaded from somewhere).
  In other words, it shows the packages that can be installed “quickly”,
  i.e., don’t need to be built from source.
2007-04-26 14:20:31 +00:00
Eelco Dolstra
5f2492eaec * New primop "throw <string>" to throw an error. This is like abort,
only thrown errors are caught by the top-level derivation evaluation
  in nix-env -qa / -i.
2007-04-16 15:03:19 +00:00
Eelco Dolstra
0a8eeea9d8 * Remove a warning. 2007-04-16 14:45:25 +00:00
Eelco Dolstra
ae7990cc88 * Work around a bug in Apple's GCC preprocessor. 2007-03-30 13:24:35 +00:00
Eelco Dolstra
17b506c0c7 * Handle ECONNRESET from the client. Also, don't abort() if there are
unexpected conditions in the SIGPOLL handler, since that messes up
  the Berkeley DB environment (which a client must never be able to
  trigger).
2007-03-28 15:46:21 +00:00
Eelco Dolstra
803cb6e3b9 * Override the setuid helper using NIX_SETUID_HELPER. 2007-03-20 22:04:25 +00:00
Eelco Dolstra
8ab229ddf2 * Terminate build hooks and substitutes with a TERM signal, not a KILL
signal.  This is necessary because those processes may have joined
  the BDB environment, so they have to be given a chance to clean up.
  (NIX-85)
2007-03-19 12:48:45 +00:00
Eelco Dolstra
b2b6cf3fc8 * Undocumented option `gc-check-reachability' to allow reachability
checking to be turned off on machines with way too many roots.
2007-03-19 09:16:47 +00:00
Eelco Dolstra
917e06bf63 * Delete the output paths before invoking the build hook. 2007-03-07 15:53:11 +00:00
Eelco Dolstra
db1973d012 * Look for the openssl program at compile time. If not found, call
openssl through $PATH at runtime.
2007-03-01 13:30:46 +00:00
Eelco Dolstra
b4a040e52b * Don't check the signature unless we have to. 2007-03-01 12:30:24 +00:00
Eelco Dolstra
2ea3bebc23 * Doh! The deriver can be empty. 2007-02-27 23:18:57 +00:00
Eelco Dolstra
044b6482c1 * Greatly reduced the amount of stack space used by the Nix expression
evaluator.  This was important because the NixOS expressions started
  to hit 2 MB default stack size on Linux.

  GCC is really dumb about stack space: it just adds up all the local
  variables and temporaries of every scope into one huge stack frame.
  This is really bad for deeply recursive functions.  For instance,
  every `throw Error(format("error message"))' causes a format object
  of a few hundred bytes to be allocated on the stack.  As a result,
  every recursive call to evalExpr2() consumed 4680 bytes.  By
  splitting evalExpr2() and by moving the exception-throwing code out
  of the main functions, evalExpr2() now only consumes 40 bytes.
  Similar for evalExpr().
2007-02-27 19:10:45 +00:00
Eelco Dolstra
adce01a8d0 * When NIX_SHOW_STATS=1, show the amount of stack space consumed by
the Nix expression evaluator.
2007-02-27 17:28:51 +00:00
Eelco Dolstra
ddde8e2f32 * Handle EINTR in select(). 2007-02-22 18:15:29 +00:00
Eelco Dolstra
7f6161ab3a * Flush cout to show progress. 2007-02-21 23:08:55 +00:00
Eelco Dolstra
0db450024d * Export/import many paths in one go. 2007-02-21 23:00:31 +00:00
Eelco Dolstra
9da367b7d5 * `nix-store -qR' and friends: print the paths sorted topologically
under the references relation.  This is useful for commands that
  want to copy paths to another Nix store in the right order.
2007-02-21 22:45:10 +00:00
Eelco Dolstra
881feb9698 * Flag --print-invalid' in nix-store --check-validity' to print out
which paths specified on the command line are invalid (i.e., don't
  barf when encountering an invalid path, just print it).  This is
  useful for build-remote.pl to figure out which paths need to be
  copied to a remote machine.  (Currently we use rsync, but that's
  rather inefficient.)
2007-02-21 17:57:59 +00:00
Eelco Dolstra
65f195f4c7 * Check that the file containing the secret key is secret. 2007-02-21 17:51:10 +00:00
Eelco Dolstra
bdadb98de8 * `nix-store --import' now also works in remote mode. The worker
always requires a signature on the archive.  This is to ensure that
  unprivileged users cannot add Trojan horses to the Nix store.
2007-02-21 17:34:02 +00:00
Eelco Dolstra
0f5da8a83c * Support exportPath() in remote mode. 2007-02-21 16:34:00 +00:00
Eelco Dolstra
dc7d594776 * importPath(): set the deriver.
* exportPath(): lock the path, use a transaction.
2007-02-21 16:23:25 +00:00
Eelco Dolstra
43c4d18c6a * nix-store --import': import an archive created by nix-store
--export' into the Nix store, and optionally check the cryptographic
  signatures against /nix/etc/nix/signing-key.pub.  (TODO: verify
  against a set of public keys.)
2007-02-21 15:45:32 +00:00
Eelco Dolstra
46e0919ced * `nix-store --export --sign': sign the Nix archive using the RSA key
in /nix/etc/nix/signing-key.sec
2007-02-21 14:31:42 +00:00
Eelco Dolstra
b824a1daee * Start of `nix-store --export' operation for serialising a store
path.  This is like `nix-store --dump', only it also dumps the
  meta-information of the store path (references, deriver).  Will add
  a `--sign' flag later to add a cryptographic signature, which we
  will use for exchanging store paths between build farm machines in a
  secure manner.
2007-02-20 23:17:20 +00:00
Eelco Dolstra
46605fb4f5 * Fix 64-bit compiler warnings. 2007-02-06 20:03:53 +00:00
Eelco Dolstra
451dbf687f * nix-env now maintains meta info (from the `meta' derivation
attribute) about installed packages in user environments.  Thus, an
  operation like `nix-env -q --description' shows useful information
  not only on available packages but also on installed packages.

* nix-env now passes the entire manifest as an argument to the Nix
  expression of the user environment builder (not just a list of
  paths), so that in particular the user environment builder has
  access to the meta attributes.
  
* New operation `--set-flag' in nix-env to change meta info of
  installed packages.  This will be useful to pass per-package
  policies to the user environment builder (e.g., how to resolve
  collision or whether to disable a package (NIX-80)) or upgrade
  policies in nix-env (e.g., that a package should be "masked", that
  is, left untouched by upgrade actions).  Example:

  $ nix-env --set-flag enabled false ghc-6.4
2007-02-02 01:52:42 +00:00
Eelco Dolstra
f52de527c7 * Doh! 2007-01-29 15:55:49 +00:00
Eelco Dolstra
b618fa6eb6 * computeStorePathForText: take the references into account when
computing the store path (NIX-77).  This is an important security
  property in multi-user Nix stores.

  Note that this changes the store paths of derivations (since the
  derivation aterms are added using addTextToStore), but not most
  outputs (unless they use builtins.toFile).
2007-01-29 15:51:37 +00:00
Eelco Dolstra
c558b1583c * Don't capitalise the primop functions. 2007-01-29 15:15:37 +00:00
Eelco Dolstra
18e6096105 * Organise primops.cc a bit better. 2007-01-29 15:11:32 +00:00
Eelco Dolstra
7349bd0176 New primitives:
* `sub' to subtract two numbers.
* `stringLength' to get the length of a string.
* `substring' to get a substring of a string.  These should be enough
  to allow most string operations to be expressed.
2007-01-29 14:23:09 +00:00
Eelco Dolstra
7dedbd896a * filterSource: pass strings to the predicate function instead of
paths.  Paths can have unexpected semantics.
2007-01-29 13:32:50 +00:00
Eelco Dolstra
84a84afb0e * Nasty: Glibc clears the TMPDIR environment variable in setuid
programs, so if a builder uses TMPDIR, then it will fail when
  executed through nix-setuid-helper.  In fact Glibc clears a whole
  bunch of variables (see sysdeps/generic/unsecvars.h in the Glibc
  sources), but only TMPDIR should matter in practice.  As a
  workaround, we reinitialise TMPDIR from NIX_BUILD_TOP.
2007-01-24 13:31:20 +00:00
Eelco Dolstra
fac63d6416 * exportReferencesGraph: work on paths within store paths as well. 2007-01-23 16:57:43 +00:00
Eelco Dolstra
71ceb1c161 * Handle multiple indirect symlinks when loading a Nix expression. 2007-01-15 14:50:25 +00:00
Eelco Dolstra
e4b0666f8e * builtins.filterSource: pass the type of the file ("regular",
"directory", "symlink") as the second argument to the filter
  predicate.
2007-01-15 08:54:51 +00:00
Eelco Dolstra
63f3ce6d9a * `nix-store --verify': revive checking the referrers table. This is
important to get garbage collection to work if there is any
  inconsistency in the database (because the referrer table is used to
  determine whether it is safe to delete a path).
* `nix-store --verify': show some progress.
2007-01-14 17:28:30 +00:00
Eelco Dolstra
8f67b35886 * Make the garbage collector more resilient to certain consistency
errors: in-use paths now cause a warning, not a fatal error.
2007-01-14 16:24:49 +00:00
Eelco Dolstra
e418976107 * Option --argstr for passing string arguments easily. (NIX-75) 2007-01-14 12:32:44 +00:00
Eelco Dolstra
afe23b5f38 * nix-pack-closure: store the top-level store paths in the closure.
* nix-unpack-closure: extract the top-level paths from the closure and
  print them on stdout.  This allows them to be installed, e.g.,
  "nix-env -i $(nix-unpack-closure)".  (NIX-64)
2007-01-13 19:50:42 +00:00
Eelco Dolstra
f25f900045 * Allow multiple --attr / -A arguments in nix-build / nix-instantiate
(NIX-74).
2007-01-13 18:25:30 +00:00
Eelco Dolstra
215505bb46 * Removed chroot support. 2007-01-13 17:54:01 +00:00
Eelco Dolstra
f23dcdd603 * Canonicalise ASTs in `nix-instantiate --eval': remove position
info, sort attribute sets.
2007-01-13 16:17:07 +00:00
Eelco Dolstra
05879db628 * Memoize strict evaluation. 2007-01-13 15:41:54 +00:00
Eelco Dolstra
5011588459 * printTermAsXML: treat derivations specially; emit an element
<derivation outPath=... drvPath=...> attrs </derivation>.  Only emit
  the attributes of any specific derivation only.  This prevents
  exponententially large XML output due to the absense of sharing.
2007-01-13 15:11:10 +00:00
Eelco Dolstra
792878af91 * Make printing an expression as XML interruptible. 2007-01-13 14:48:41 +00:00
Eelco Dolstra
11158028be * Cleanup. 2007-01-13 14:21:49 +00:00
Eelco Dolstra
1073b1780a * Remove debug message. 2006-12-13 14:29:05 +00:00
Eelco Dolstra
a3e6415ba8 * New primop builtins.filterSource, which can be used to filter files
from a source directory.  All files for which a predicate function
  returns true are copied to the store.  Typical example is to leave
  out the .svn directory:

    stdenv.mkDerivation {
      ...
      src = builtins.filterSource
        (path: baseNameOf (toString path) != ".svn")
        ./source-dir;
      # as opposed to
      #   src = ./source-dir;
    }

  This is important because the .svn directory influences the hash in
  a rather unpredictable and variable way.
2006-12-12 23:05:01 +00:00
Eelco Dolstra
b438d37558 * In dumpPath(): pass a function object that allows files to be
selectively in/excluded from the dump.
2006-12-12 21:51:02 +00:00
Eelco Dolstra
7ace29dae7 * New operation `nix-env --set' which sets a user environment to a
single derivation specified by the argument.  This is useful when we
  want to have a profile for a single derivation, such as a server
  configuration.  Then we can just say (e.g.)

  $ nix-env -p /.../server-profile -f server.nix --set -A server

  We can't do queries or upgrades on such a profile, but we can do
  rollbacks.  The advantage over -i is that we don't have to worry
  about other packages having been installed in the profile
  previously; --set gets rid of them.
2006-12-12 19:06:02 +00:00
Eelco Dolstra
1a7e88bbd9 * New built-in function `builtins.attrNames' that returns the
names of the attributes in an attribute set.
2006-12-12 16:14:31 +00:00
Eelco Dolstra
b17677462c * Use lchown() instead of chown() in canonicalisePathMetaData(). This
matters when running as root, since then we don't use the setuid
  helper (which already used lchown()).
  
* Also check for an obscure security problem on platforms that don't
  have lchown.  Then we can't change the ownership of symlinks, which
  doesn't matter *except* when the containing directory is writable by
  the owner (which is the case with the top-level Nix store directory).
2006-12-09 20:02:27 +00:00
Eelco Dolstra
5f681988f2 * Use deletePathWrapped() in more places. 2006-12-09 00:26:24 +00:00
Eelco Dolstra
fa33303146 * Goal cancellation inside the waitForInput() loop needs to be handled
very carefully, since it can invalidate iterators into the
  `children' map.
2006-12-08 18:41:48 +00:00
Eelco Dolstra
06c4929958 * Some refactoring.
* Throw more exceptions as BuildErrors instead of Errors.  This
  matters when --keep-going is turned on.  (A BuildError is caught
  and terminates the goal in question, an Error terminates the
  program.)
2006-12-08 17:26:21 +00:00
Eelco Dolstra
9dbfe242e3 * Kill a build if it has gone for more than a certain number of
seconds without producing output on stdout or stderr (NIX-65).  This
  timeout can be specified using the `--max-silent-time' option or the
  `build-max-silent-time' configuration setting.  The default is
  infinity (0).

* Fix a tricky race condition: if we kill the build user before the
  child has done its setuid() to the build user uid, then it won't be
  killed, and we'll potentially lock up in pid.wait().  So also send a
  conventional kill to the child.
2006-12-08 15:44:00 +00:00
Eelco Dolstra
d3fe6ab024 * Also for convenience, change the ownership of the build output even
in case of failure.
2006-12-08 00:19:50 +00:00
Eelco Dolstra
096194ab29 * Remove ancient terminology. 2006-12-07 23:58:36 +00:00
Eelco Dolstra
6833e8bbe8 * When keeping the temporary build directory (-K), change the owner
back to the Nix account.
2006-12-07 23:27:40 +00:00
Eelco Dolstra
e24d0201c2 * Doh! 2006-12-07 22:07:05 +00:00
Eelco Dolstra
2819eb36a4 * Be less verbose. 2006-12-07 21:43:35 +00:00
Eelco Dolstra
4ca01065c3 * Rename all those main.cc files. 2006-12-07 20:47:30 +00:00
Eelco Dolstra
d03f0d4117 * Check for lchown. 2006-12-07 18:51:11 +00:00
Eelco Dolstra
c3286ec020 * Don't count on the Pid deconstructor to kill the child process,
since if we're running a build user in non-root mode, we can't.  Let
  the setuid helper do it.
2006-12-07 17:52:58 +00:00
Eelco Dolstra
a82d80ddeb * Move setuidCleanup() to libutil. 2006-12-07 16:40:41 +00:00
Eelco Dolstra
f76fdb6d42 * If not running as root, let the setuid helper kill the build user's
processes before and after the build.
2006-12-07 16:33:31 +00:00
Eelco Dolstra
ec23ecc64d * In the garbage collector, if deleting a path fails, try to fix its
ownership, then try again.
2006-12-07 15:54:52 +00:00
Eelco Dolstra
a0a43c3206 * When not running as root, call the setuid helper to change the
ownership of the build result after the build.
2006-12-07 15:18:14 +00:00
Eelco Dolstra
6a07ff1ec0 * Change the ownership of store paths to the Nix account before
deleting them using the setuid helper.
2006-12-07 14:14:35 +00:00
Eelco Dolstra
7d8cf316ee * Pass the actual build user to the setuid helper. 2006-12-07 11:27:32 +00:00
Eelco Dolstra
a45c498e4e * If Nix is not running as root, call the setuid helper to start the
builder under the desired build user.
2006-12-07 00:42:30 +00:00
Eelco Dolstra
813a7c65c9 * Sanity check. 2006-12-07 00:19:27 +00:00
Eelco Dolstra
6a8e60913a * Move killUser() to libutil so that the setuid helper can use it. 2006-12-07 00:16:07 +00:00
Eelco Dolstra
79875c5e42 * Change the ownership of the current directory to the build user. 2006-12-06 23:52:25 +00:00
Eelco Dolstra
62ab131412 * Verify that the desired target user is in the build users group (as
specified in the setuid config file).
2006-12-06 23:15:26 +00:00
Eelco Dolstra
f07ac41656 * Check that the caller is allowed to call the setuid helper. The
allowed uid is specified in a configuration file in
  /etc/nix-setuid.conf.
2006-12-06 22:45:41 +00:00
Eelco Dolstra
ef281b93c2 * Fix the safety check. 2006-12-06 20:18:29 +00:00
Eelco Dolstra
6e5ec1029a * Get rid of `build-users'. We'll just take all the members of
`build-users-group'.  This makes configuration easier: you can just
  add users in /etc/group.
2006-12-06 20:00:15 +00:00
Eelco Dolstra
751f6d2157 * nix-setuid-helper: allow running programs under a different uid. 2006-12-06 17:29:10 +00:00
Eelco Dolstra
9f0efa6611 * Start of the setuid helper (the program that performs the operations
that have to be done as root: running builders under different uids,
  changing ownership of build results, and deleting paths in the store
  with the wrong ownership).
2006-12-06 01:24:02 +00:00
Eelco Dolstra
2b558843a2 * Be less chatty. 2006-12-05 19:01:19 +00:00
Eelco Dolstra
44cad9630f * Urgh. Do setgid() before setuid(), because the semantics of setgid()
changes completely depending on whether you're root...
2006-12-05 18:28:15 +00:00
Eelco Dolstra
6f0d050324 * Tricky: child processes should not send data to the client since
that might mess up the protocol.  And besides, the socket file
  descriptor is probably closed.
2006-12-05 18:21:16 +00:00
Eelco Dolstra
4c1c37d0b6 * FreeBSD returns ESRCH when there are no processes to kill. 2006-12-05 18:07:46 +00:00
Eelco Dolstra
8d1854c3f1 * Oops! In daemon mode, we can't run as root either if build-users is empty. 2006-12-05 17:44:19 +00:00
Eelco Dolstra
99655245ae * Use an explicit handler for SIGCHLD, since SIG_IGN doesn't do the
right thing on FreeBSD 4 (it leaves zombies).
2006-12-05 17:21:42 +00:00
Eelco Dolstra
62b0497c0f * Better message. 2006-12-05 16:17:01 +00:00
Eelco Dolstra
c808e6252f * Ugly hack to handle spurious SIGPOLLs. 2006-12-05 15:36:31 +00:00
Eelco Dolstra
fd4a9db91f * Some renaming. 2006-12-05 14:15:51 +00:00
Eelco Dolstra
a9c4f66cfb * Allow unprivileged users to run the garbage collector and to do
`nix-store --delete'.  But unprivileged users are not allowed to
  ignore liveness.
* `nix-store --delete --ignore-liveness': ignore the runtime roots as
  well.
2006-12-05 02:18:46 +00:00
Eelco Dolstra
29cf434a35 * The determination of the root set should be made by the privileged
process, so forward the operation.
* Spam the user about GC misconfigurations (NIX-71).
* findRoots: skip all roots that are unreadable - the warnings with
  which we spam the user should be enough.
2006-12-05 01:31:45 +00:00
Eelco Dolstra
8623256f48 * findRoots: return a map from the symlink (outside of the store) to
the store path (inside the store).
2006-12-05 00:48:36 +00:00
Eelco Dolstra
d27a73b1a9 * In addPermRoot, check that the root that we just registered can be
found by the garbage collector.  This addresses NIX-71 and is a
  particular concern in multi-user stores.
2006-12-05 00:34:42 +00:00
Eelco Dolstra
74033a844f * Add indirect root registration to the protocol so that unprivileged
processes can register indirect roots.  Of course, there is still
  the problem that the garbage collector can only read the targets of
  the indirect roots when it's running as root...
2006-12-04 23:29:16 +00:00
Eelco Dolstra
0d40f6d7bb * Not every OS knows about SIGPOLL. 2006-12-04 22:58:44 +00:00
Eelco Dolstra
7751160e9f * Don't redirect stderr. 2006-12-04 19:10:23 +00:00
Eelco Dolstra
40c3529909 * Handle exceptions and stderr for all protocol functions.
* SIGIO -> SIGPOLL (POSIX calls it that).
* Use sigaction instead of signal to register the SIGPOLL handler.
  Sigaction is better defined, and a handler registered with signal
  appears not to interrupt fcntl(..., F_SETLKW, ...), which is bad.
2006-12-04 17:55:14 +00:00
Eelco Dolstra
0130ef88ea * Daemon mode (`nix-worker --daemon'). Clients connect to the server
via the Unix domain socket in /nix/var/nix/daemon.socket.  The
  server forks a worker process per connection.
* readString(): use the heap, not the stack.
* Some protocol fixes.
2006-12-04 17:17:13 +00:00
Eelco Dolstra
4740baf3a6 * When NIX_REMOTE=daemon, connect to /nix/var/nix/daemon.socket
instead of forking a worker.
2006-12-04 14:21:39 +00:00
Eelco Dolstra
f5f0cf423f * Refactoring. 2006-12-04 13:28:14 +00:00
Eelco Dolstra
052b6fb149 * Pass the verbosity level to the worker. 2006-12-04 13:15:29 +00:00
Eelco Dolstra
1e16d20655 * Install the worker in bindir, not libexecdir.
* Allow the worker path to be overriden through the NIX_WORKER
  environment variable.
2006-12-04 13:09:16 +00:00
Eelco Dolstra
9322b399f3 * Doh. 2006-12-03 20:41:22 +00:00
Eelco Dolstra
f4279bcde0 * Don't run setuid root when build-users is empty.
* Send startup errors to the client.
2006-12-03 16:25:19 +00:00
Eelco Dolstra
35247c4c9f * Removed `build-allow-root'.
* Added `build-users-group', the group under which builds are to be
  performed.
* Check that /nix/store has 1775 permission and is owner by the
  build-users-group.
2006-12-03 15:32:38 +00:00
Eelco Dolstra
84d6459bd5 * Use setreuid if setresuid is not available. 2006-12-03 14:32:22 +00:00
Eelco Dolstra
a9f9241054 * Handle a subtle race condition: the client closing the socket
between the last worker read/write and the enabling of the signal
  handler.
2006-12-03 03:16:27 +00:00
Eelco Dolstra
3ed9e4ad9b * Some hardcore magic to handle asynchronous client disconnects.
The problem is that when we kill the client while the worker is
  building, and the builder is not writing anything to stderr, then
  the worker never notice that the socket is closed on the other side,
  so it just continues indefinitely.  The solution is to catch SIGIO,
  which is sent when the far side of the socket closes, and simulate
  an normal interruption.  Of course, SIGIO is also sent every time
  the client sends data over the socket, so we only enable the signal
  handler when we're not expecting any data...
2006-12-03 03:03:36 +00:00
Eelco Dolstra
4251f94b32 * Use a Unix domain socket instead of pipes. 2006-12-03 02:36:44 +00:00
Eelco Dolstra
8c76df93e6 * Better error message if the worker doesn't start. 2006-12-03 02:22:04 +00:00
Eelco Dolstra
363f40022f * Pid::kill() should be interruptable. 2006-12-03 02:12:26 +00:00
Eelco Dolstra
7951c3c546 * Some hackery to propagate the worker's stderr and exceptions to the
client.
2006-12-03 02:08:13 +00:00
Eelco Dolstra
714fa24cfb * Run the worker in a separate session to prevent terminal signals
from interfering.
2006-12-03 00:52:27 +00:00
Eelco Dolstra
e25fad691a * Move addTempRoot() to the store API, and add another function
syncWithGC() to allow clients to register GC roots without needing
  write access to the global roots directory or the GC lock.
2006-12-02 16:41:36 +00:00
Eelco Dolstra
30bf547f4f * Doh. 2006-12-02 15:46:17 +00:00
Eelco Dolstra
536595b072 * Remove most of the old setuid code.
* Much simpler setuid code for the worker in slave mode.
2006-12-02 15:45:51 +00:00
Eelco Dolstra
9c9cdb06d0 * Remove SwitchToOriginalUser, we're not going to need it anymore. 2006-12-02 14:34:14 +00:00
Eelco Dolstra
8ba5d32769 * Remove queryPathHash().
* Help for nix-worker.
2006-12-02 14:27:24 +00:00
Eelco Dolstra
fcd9900d74 * Replace read-only calls to addTextToStore. 2006-12-01 21:00:39 +00:00
Eelco Dolstra
a824d58b56 * Merge addToStore and addToStoreFixed.
* addToStore now adds unconditionally, it doesn't use readOnlyMode.
  Read-only operation is up to the caller (who can call
  computeStorePathForPath).
2006-12-01 20:51:18 +00:00
Eelco Dolstra
ceb982a1be * Right name. 2006-12-01 18:02:05 +00:00
Eelco Dolstra
b0d8e05be1 * More operations.
* addToStore() and friends: don't do a round-trip to the worker if
  we're only interested in the path (i.e., in read-only mode).
2006-12-01 18:00:01 +00:00
Eelco Dolstra
0565b5f2b3 * More remote operations.
* Added new operation hasSubstitutes(), which is more efficient than
  querySubstitutes().size() > 0.
2006-11-30 22:43:55 +00:00
Eelco Dolstra
aac547a8b3 * Doh. 2006-11-30 21:32:46 +00:00
Eelco Dolstra
0263279071 * More operations. 2006-11-30 20:45:20 +00:00
Eelco Dolstra
a711689368 * First remote operation: isValidPath(). 2006-11-30 20:13:59 +00:00
Eelco Dolstra
765bdfe542 * When NIX_REMOTE is set to "slave", fork off nix-worker in slave
mode.  Presumably nix-worker would be setuid to the Nix store user.
  The worker performs all operations on the Nix store and database, so
  the caller can be completely unprivileged.

  This is already much more secure than the old setuid scheme, since
  the worker doesn't need to do Nix expression evaluation and so on.
  Most importantly, this means that it doesn't need to access any user
  files, with all resulting security risks; it only performs pure
  store operations.

  Once this works, it is easy to move to a daemon model that forks off
  a worker for connections established through a Unix domain socket.
  That would be even more secure.
2006-11-30 19:54:43 +00:00
Eelco Dolstra
40b3f64b55 * Skeleton of the privileged worker program.
* Some refactoring: put the NAR archive integer/string serialisation
  code in a separate file so it can be reused by the worker protocol
  implementation.
2006-11-30 19:19:59 +00:00
Eelco Dolstra
9adc074dc3 * Oops. 2006-11-30 18:35:50 +00:00
Eelco Dolstra
9cf1948993 * Skeleton of remote store implementation. 2006-11-30 18:35:36 +00:00
Eelco Dolstra
6ecb840fd1 * Put building in the store API. 2006-11-30 18:02:04 +00:00
Eelco Dolstra
e2ef5e07fd * Refactoring. There is now an abstract interface class StoreAPI
containing functions that operate on the Nix store.  One
  implementation is LocalStore, which operates on the Nix store
  directly.  The next step, to enable secure multi-user Nix, is to
  create a different implementation RemoteStore that talks to a
  privileged daemon process that uses LocalStore to perform the actual
  operations.
2006-11-30 17:43:04 +00:00
Eelco Dolstra
80b742dd52 * Don't spam. 2006-11-29 22:07:49 +00:00
Roy van den Broek
92417600a1 * Example script to set permissions for setuid operation. 2006-11-29 21:58:09 +00:00
Eelco Dolstra
71e867c5f5 * Remove --enable-setuid, --with-nix-user and --with-nix-group.
Rather, setuid support is now always compiled in (at least on
  platforms that have the setresuid system call, e.g., Linux and
  FreeBSD), but it must enabled by chowning/chmodding the Nix
  binaries.
2006-11-29 21:06:58 +00:00
Eelco Dolstra
c6a97e3b74 * Doh! Path sizes need to be computed recursively of course.
(NIX-70)
2006-11-24 20:24:14 +00:00
Eelco Dolstra
a76efaeb3f * Dead files. 2006-11-24 20:07:30 +00:00
Eelco Dolstra
0541ddc7e3 * Turn off synchronisation between C and C++ I/O functions. This
gives a huge speedup in operations that read or write from standard
  input/output.  (So libstdc++'s I/O isn't that bad, you just have to
  call std::ios::sync_with_stdio(false).)  For instance, `nix-store
  --register-substitutes' went from 1.4 seconds to 0.1 seconds on a
  certain input.  Another victory for Valgrind.
2006-11-18 18:56:30 +00:00
Eelco Dolstra
f459a5bb3a * Remove the undocumented `noscan' feature. It's no longer necessary
now that reference scanning is sufficiently streamy.
2006-11-13 18:19:05 +00:00
Eelco Dolstra
e2a70b7ec0 * Magic attribute `exportReferencesGraph' that allows the references
graph to be passed to a builder.  This attribute should be a list of
  pairs [name1 path1 name2 path2 ...].  The references graph of each
  `pathN' will be stored in a text file `nameN' in the temporary build
  directory.  The text files have the format used by `nix-store
  --register-validity'.  However, the deriver fields are left empty.

  `exportReferencesGraph' is useful for builders that want to do
  something with the closure of a store path.  Examples: the builders
  that make initrds and ISO images for NixOS.

  `exportReferencesGraph' is entirely pure.  It's necessary because
  otherwise the only way for a builder to get this information would
  be to call `nix-store' directly, which is not allowed (though
  unfortunately possible).
2006-11-13 18:18:13 +00:00
Eelco Dolstra
e40d4a5604 * Option --reregister' in nix-store --register-validity'. We need
this in the NixOS installer (or in the buildfarm) to ensure that the
  cryptographic hash of the path contents still matches the actual
  contents.
2006-11-13 16:48:27 +00:00
Eelco Dolstra
7e85a2af5f * Fix importing of derivation outputs. 2006-11-03 16:17:39 +00:00
Eelco Dolstra
8478cd260f * readFile: don't overflow the stack on large files. 2006-10-30 11:56:09 +00:00
Eelco Dolstra
ae6fb27f18 * `nix-store --read-log / -l PATH' shows the build log of PATH, if
available.  For instance,

    $ nix-store -l $(which svn) | less

  lets you read the build log of the Subversion instance in your
  profile.

* `nix-store -qb': if applied to a non-derivation, take the deriver.
2006-10-28 16:33:54 +00:00
Eelco Dolstra
dd300fb48d * Some better error messages. 2006-10-23 16:45:19 +00:00
Eelco Dolstra
17f4883bfe * Better message. 2006-10-19 17:43:58 +00:00
Eelco Dolstra
9bd93f7606 * toFile: maintain the references. 2006-10-19 17:39:02 +00:00
Eelco Dolstra
6a67556f71 * Special derivation attribute `allowedReferences' that causes Nix to
check that the references of the output of a derivation are in the
  specified set.  For instance,

    allowedReferences = [];

  specifies that the output cannot have any references.  (This is
  useful, for instance, for the generation of bootstrap binaries for
  stdenv-linux, which must not have any references for purity).  It
  could also be used to guard against undesired runtime dependencies,
  e.g.,

    {gcc, dynlib}: derivation {
      ...
      allowedReferences = [dynlib];
    }

  says that the output can refer to the path of `dynlib' but not
  `gcc'.  A `forbiddedReferences' attribute would be more useful for
  this, though.
2006-10-19 16:09:24 +00:00
Eelco Dolstra
daa8f85fcd * Backwards compatibility hack for user environments made by Nix <= 0.10. 2006-10-17 14:13:15 +00:00
Eelco Dolstra
24737f279e * Backwards compatibility with old user environment manifests. 2006-10-17 14:01:45 +00:00
Eelco Dolstra
4bd5cdb90b * Print out the offending path. 2006-10-17 14:01:28 +00:00
Eelco Dolstra
58ff6939f4 * An awful backwards compatibility hack. 2006-10-17 12:58:42 +00:00
Eelco Dolstra
3059df0f1e * baseNameOf: paths don't have to be absolute. 2006-10-17 12:34:13 +00:00
Eelco Dolstra
822dba2210 * Maintain the references for the user environment properly. 2006-10-17 12:15:15 +00:00
Eelco Dolstra
be1961c9f8 * toPath: should be the identity on paths. 2006-10-17 11:07:11 +00:00
Eelco Dolstra
cba913c521 * dirOf: return a path if the argument is a path. 2006-10-17 11:05:34 +00:00
Eelco Dolstra
cf705eaf78 * toString: don't copy paths. So toString can be used to pass
non-store paths to a builder.
2006-10-17 10:58:12 +00:00
Eelco Dolstra
7de5fe2fc2 * Do the path check on the normal form. 2006-10-17 10:57:25 +00:00
Eelco Dolstra
46b631b6c4 * Don't generate an empty drvPath attribute in the manifest. 2006-10-17 10:15:42 +00:00
Eelco Dolstra
d7efd76394 * Big cleanup of the semantics of paths, strings, contexts, string
concatenation and string coercion.  This was a big mess (see
  e.g. NIX-67).  Contexts are now folded into strings, so that they
  don't cause evaluation errors when they're not expected.  The
  semantics of paths has been clarified (see nixexpr-ast.def).
  toString() and coerceToString() have been merged.

  Semantic change: paths are now copied to the store when they're in a
  concatenation (and in most other situations - that's the
  formalisation of the meaning of a path).  So

    "foo " + ./bla

  evaluates to "foo /nix/store/hash...-bla", not "foo
  /path/to/current-dir/bla".  This prevents accidental impurities, and
  is more consistent with the treatment of derivation outputs, e.g.,
  `"foo " + bla' where `bla' is a derivation.  (Here `bla' would be
  replaced by the output path of `bla'.)
2006-10-16 15:55:34 +00:00
Eelco Dolstra
37c8a664f3 * A helpful message. 2006-10-13 11:49:55 +00:00
Eelco Dolstra
e4af398681 * Don't crash when upgrading the Berkeley DB environment. 2006-10-13 11:15:53 +00:00
Eelco Dolstra
7d4567f2cc * Removed URIs from the evaluator (NIX-66). They are now just another
kind of notation for strings.
2006-10-11 21:59:33 +00:00
Eelco Dolstra
0c4c5c2020 * Quick hack to fix NIX-67: evaluation result differing if the Nix
expression resides in the store.
2006-10-10 21:23:35 +00:00
Eelco Dolstra
bd0c40e1e9 * import': unwrap the context. Necessary to make import (x + y)'
work, where x is a store path.
2006-10-10 15:07:23 +00:00
Eelco Dolstra
9e08f5efe1 * Documented nix-store --dump / --restore. 2006-10-05 22:57:07 +00:00
Eelco Dolstra
5fd44654db * toXML: propagate the context to allow derivations to be used in the
argument.
2006-10-03 15:38:59 +00:00
Eelco Dolstra
d20c3011a0 * toFile: added an additional argument to specify the store path
suffix, e.g., `builtins.toFile "builder.sh" "..."'.
* toFile: handle references to other files correctly.
2006-10-03 14:55:54 +00:00
Eelco Dolstra
ac19b333b3 * Finally, a real "let" syntax: `let x = ...; ... z = ...; in ...'. 2006-10-02 15:52:44 +00:00
Eelco Dolstra
7581cfdee4 * Hack for Bison 2.3 compatability. 2006-10-02 14:43:15 +00:00
Eelco Dolstra
4ad6fb7ea3 * Fix setuid builds. 2006-09-27 21:04:07 +00:00
Eelco Dolstra
3632019b73 * Quick hack to let nix-install-package set the package name properly
(e.g., "java-front-0.9pre15899" instead of "java-front";
  particularly important when doing upgrades later on).
2006-09-25 14:00:59 +00:00
Eelco Dolstra
e347033f71 * The result of a concatenation with a derivation on the left-hand
side should be a path, I guess.
* Handle paths that are in the store but not direct children of the
  store directory.
* Ugh, hack to prevent double context wrapping.
2006-09-24 21:39:57 +00:00
Eelco Dolstra
0e705391db * Primop `toPath' to convert a string to a path.
* Primop `pathExists' to check for path existence.
2006-09-24 18:23:32 +00:00
Eelco Dolstra
e47e0c2dbe * Builtin function `getEnv' for getting environment variables. 2006-09-24 17:48:41 +00:00
Eelco Dolstra
df8873e14a * lessThan primitive for integer comparison. 2006-09-24 15:21:48 +00:00
Eelco Dolstra
2ab4bc44c7 * Builtin function `add' to add integers.
* Put common test functions in tests/lang/lib.nix.
2006-09-22 15:29:21 +00:00
Eelco Dolstra
d315210612 * Added a builtin function `isList' to test whether a value is a list.
With this primitive, a list-flattening function can be implemented
  (NIX-55, example is in tests/lang/eval-okay-flatten.nix).
2006-09-22 14:55:19 +00:00
Eelco Dolstra
c02a44183f * Builtin functions head' and tail' to return the head and tail of
list.  Useful for lots of things, such as implementing a fold
  function (see NIX-30, example is in tests/lang/eval-okay-list.nix).
2006-09-22 14:46:36 +00:00
Eelco Dolstra
8a1ab709a4 * New builtin functions builtins.{hasAttr, getAttr} to check for
attribute existence and to return an attribute from an attribute
  set, respectively.  Example: `hasAttr "foo" {foo = 1;}'.  They
  differ from the `?' and `.' operators in that the attribute name is
  an arbitrary expression.  (NIX-61)
2006-09-22 14:31:55 +00:00
Eelco Dolstra
666babbbfa * Use a bounded amount of memory in scanForReferences() by not reading
regular files into memory all at once.
2006-09-22 13:10:30 +00:00
Eelco Dolstra
d22d7565f3 * Don't allocate the buffer twice. 2006-09-22 11:28:23 +00:00
Eelco Dolstra
b43aeadbc9 * Don't allocate more than SIZE_MAX bytes. 2006-09-22 11:13:35 +00:00
Eelco Dolstra
25df501704 * GC options in nix-store --help (NIX-15). 2006-09-21 19:06:34 +00:00
Eelco Dolstra
4e91d8621f * Fix comment. 2006-09-21 18:52:05 +00:00
Eelco Dolstra
1bdc152931 * Shut up a warning. 2006-09-20 16:36:29 +00:00
Eelco Dolstra
0623359fbc * Print a better error message for wrong hashes (NIX-49). 2006-09-20 16:15:32 +00:00
Eelco Dolstra
e0afaf1857 * Wow, that bug has been there since r764. 2006-09-14 22:48:59 +00:00
Eelco Dolstra
86cbd93ec1 * nix-env --switch-generation / --list-generations /
--delete-generations: lock the profile to prevent (extremely
  unlikely) race conditions.
2006-09-14 22:33:53 +00:00
Eelco Dolstra
5c38c863bd * Fix a huge gaping hole in nix-env w.r.t. the garbage collector.
Nix-env failed to call addPermRoot(), which is necessary to safely
  add a new root.  So if nix-env started after and finished before the
  garbage collector, the user environment (plus all other new stuff)
  it built might be garbage collected, leading to a dangling symlink
  chain in ~/.nix-profile...

* Be more explicit if we block on the GC lock ("waiting for the big
  garbage collector lock...").

* Don't loop trying to create a new generation.  It's not necessary
  anymore since profiles are locked nowadays.
2006-09-14 22:30:33 +00:00
Eelco Dolstra
01d169f817 * Support `++'.
* More follow restrictions on layout.
2006-09-11 13:05:15 +00:00
Eelco Dolstra
feb63da431 * Remove debug message. 2006-09-08 09:31:07 +00:00
Eelco Dolstra
a89a201598 * Missing #include. 2006-09-05 10:32:47 +00:00
Eelco Dolstra
fc195519b5 * Sone missing #includes. 2006-09-05 08:54:48 +00:00
Eelco Dolstra
bafc1690fc * Move setuid stuff to libutil.
* Install libexpr header files.
2006-09-04 22:55:28 +00:00
Eelco Dolstra
e5a6c09b12 * Install header files in /nix/include/nix. 2006-09-04 22:41:36 +00:00
Eelco Dolstra
4be5443882 * Remove unnecessary inclusions of aterm2.h. 2006-09-04 22:08:40 +00:00
Eelco Dolstra
2382a729e0 * Don't need extern "C". 2006-09-04 21:50:39 +00:00
Eelco Dolstra
e3ce954582 * Compile the lexer as C++ code. Remove all the redundant C/C++
marshalling code.
2006-09-04 21:36:15 +00:00
Eelco Dolstra
75068e7d75 * Use a proper namespace.
* Optimise header file usage a bit.
* Compile the parser as C++.
2006-09-04 21:06:23 +00:00
Eelco Dolstra
aab8812732 * Store the Nix libraries in ${libdir}/nix instead of ${libdir}. 2006-09-04 15:12:24 +00:00
Eelco Dolstra
7974aae81c * New primop: builtins.toFile, which writes a string into the store
and returns its path.  This can be used to (for instance) write
  builders inside a Nix expression, e.g.,

  stdenv.mkDerivation {
    builder = "
      source $stdenv/setup
      ...
    ";
    ...
  }
2006-09-01 12:07:31 +00:00
Eelco Dolstra
de90fdf908 * Allow "$" in strings as long as they are not followed by "{". (Too
bad flex doesn't have lexical restrictions, the current solution
  isn't quite right...)
2006-09-01 12:04:06 +00:00
Eelco Dolstra
c25f688e23 * Doh! Doh! Doh! 2006-08-31 15:38:43 +00:00
Eelco Dolstra
354d58b3d7 * Better error checking. 2006-08-31 11:40:39 +00:00
Eelco Dolstra
f93f7b75be * Okay, that's a bit harder than expected. 2006-08-30 13:10:04 +00:00
Eelco Dolstra
3151bdea55 * Uninitialised variable. 2006-08-30 12:00:27 +00:00
Eelco Dolstra
547b119f25 * Support singleton values and nested lists again in `args', but print
a warning.
2006-08-29 15:40:49 +00:00
Eelco Dolstra
2132d9ddeb * Fix the ~ operator. 2006-08-29 15:29:38 +00:00
Eelco Dolstra
1ec9f55741 * In toString, deal with nested lists properly (i.e., flatten them). 2006-08-28 21:47:42 +00:00
Eelco Dolstra
1fca76870b * Removed processBinding, instead we now apply toString to all
derivation attributes to flatten them into strings.  This is
  possible since string can nowadays be wrapped in contexts that
  describe the derivations/sources referenced by the evaluation of the
  string.
2006-08-28 13:31:06 +00:00
Eelco Dolstra
8a6080eb14 * Refactoring. 2006-08-26 16:48:01 +00:00
Eelco Dolstra
4b66cebe7b * Remove those storePath attribute sets, we don't need 'em. 2006-08-25 17:09:55 +00:00
Eelco Dolstra
3e8dccf6ab * Escape newlines in XML attributes to prevent them from being
normalised away.
2006-08-24 15:02:39 +00:00
Eelco Dolstra
215ec2ddc6 * New primop __toXML (or builtins.toXML) to convert an expression to
an XML representation stored in a string.  This should be useful to
  pass structured information to builders.
2006-08-24 14:34:29 +00:00
Eelco Dolstra
f793caf936 * Refactoring. 2006-08-24 14:16:55 +00:00
Eelco Dolstra
da25d80152 * Strict evaluation and XML printing of lists. 2006-08-24 14:03:39 +00:00
Eelco Dolstra
943ab38a0d * Refactoring: move strictEval to libexpr. 2006-08-24 13:39:22 +00:00
Eelco Dolstra
f41297fdce * Allow --arg in nix-env as well, example:
$ nix-env -qa --system-filter \* --arg system '"powerpc-darwin"'

  to override the system from the default value (__currentSystem in
  all-packages.nix).
2006-08-23 16:33:21 +00:00
Eelco Dolstra
9638f3f393 * Pass the autoArgs to findAlongAttrPath so that "nix-instantiate
foo.nix -A attr --arg name value" will work if (name, value) is
  needed in the evaluation leading up to "attr".
2006-08-23 16:20:14 +00:00
Eelco Dolstra
b19cebc513 * Quotes. 2006-08-23 15:46:27 +00:00
Eelco Dolstra
38f18aa6d4 * New primop: abort "error message". 2006-08-23 15:46:00 +00:00
Eelco Dolstra
4a053bfdfd * A new primop `builtins', which returns an attribute set containing
all the primops.  This allows Nix expressions to test for new
  primops and take appropriate action if they're not available.  For
  instance, rather than calling a primop `foo' directly, they could
  say `if builtins ? foo then builtins.foo ... else ...'.
2006-08-23 14:39:11 +00:00
Eelco Dolstra
24e234a2fa * Print attributes in sorted order, rather than the arbitrary order
produced by ATermMap.  Necessary for testing.
* `--strict' should also work on stdin.
2006-08-17 08:53:08 +00:00
Eelco Dolstra
22ba63df16 * More XML output. `--strict' to strictly evaluate attribute sets and
so on.
* Removed `--print-args', it's subsumed by `--eval-only --xml'.
2006-08-16 21:59:53 +00:00
Eelco Dolstra
18e4ac0fc6 * `nix-instantiate --{eval|parse}-only --xml': print an XML
representation instead of an ATerm.
* Indent XML output.
2006-08-16 10:32:30 +00:00
Eelco Dolstra
2670642733 * Handle carriage returns. Fixes NIX-53. 2006-08-16 10:28:44 +00:00
Martin Bravenboer
3e5b68068b On cygwin, disable the check that the output is not group or world
writable. File permissions on Cygwin are rather complex, and in this
case this check introduced a problem with build jobs invoke from
outside of Cygwin (MSYS). It seemed almost impossible to fix the
permissions of the directory, so for now this safety check is disabled
on Cygwin.
2006-08-15 21:37:48 +00:00
Eelco Dolstra
7455fd8835 * Put the value in an attribute. 2006-08-14 14:24:18 +00:00
Eelco Dolstra
4250b641d8 * `nix-store --gc --print-dead': print the total size of the store
objects that would be freed.
2006-08-11 20:26:49 +00:00
Eelco Dolstra
92f7dfa5b7 * Don't assume that paths returned by the runtime root finder are
valid.
2006-08-11 20:15:20 +00:00
Eelco Dolstra
d19b6521fc * New configuration setting `build-max-jobs' which sets the default
for the `-j' flag (i.e., the maximum number of jobs to execute in
  parallel).  Useful on multi-processor machines.
2006-08-10 20:19:13 +00:00
Eelco Dolstra
f1aa71a92f * Fix the help message wrt --attr. 2006-08-09 15:12:34 +00:00
Eelco Dolstra
a18d02e0b0 * Print a warning that the subpath operator (~) is deprecated. 2006-08-09 15:08:47 +00:00
Eelco Dolstra
a0607be7f4 * Workaround for a bug (?) in GCC 2.95. 2006-08-07 19:48:38 +00:00
Eelco Dolstra
1854f84e83 * Fix a few warnings. 2006-08-04 17:07:13 +00:00
Eelco Dolstra
dcff8cdb76 * Weird issue on Cygwin with the include file order. 2006-08-04 16:01:26 +00:00
Eelco Dolstra
339e6f0e1d * `nix-env -q --xml': show query result in XML format for easier
automated processing.
2006-08-03 15:52:09 +00:00
Eelco Dolstra
0e267e2625 * `nix-instantiate --print-args': produce XML output so that the
result can be used more easily by scripts.
2006-08-03 14:49:57 +00:00
Eelco Dolstra
4750f6c5ed * Simple class for writing XML files. 2006-08-03 13:21:21 +00:00
Eelco Dolstra
6ac237e242 * Show some progress. 2006-08-01 09:43:41 +00:00
Eelco Dolstra
4661282fde * `nix-instantiate ... --arg NAME VALUE': allow arguments to be passed
to functions from the command line.
* nix-build: started removing backticks.
2006-07-28 16:03:28 +00:00
Eelco Dolstra
c11839d7b2 * `nix-instantiate --print-args': print out the valid values for
functions arguments that have a domain.
2006-07-28 14:01:29 +00:00
Eelco Dolstra
ca2238cf81 * Refactoring: get the selection path stuff out of getDerivations()
and put it into a separate function findAlongAttrPath().
2006-07-26 15:05:15 +00:00
Eelco Dolstra
2317d8f671 * `nix-instantiate --print-args' prints out the arguments of a
top-level function.
2006-07-25 21:21:50 +00:00
Eelco Dolstra
0e6dc72a7a * Applied rbroek's patch from the branch at
https://svn.cs.uu.nl:12443/repos/trace/buildfarm-control/trunk/ext/nix/,
  with some modifications.  This allows `nix-env -qa' to show the
  attribute path that can be used to unambiguously install a package
  using `nix-env -i -A'.  Example:

    $ nix-env -f top-level/all-packages.nix -qaA subversion xorg-server
    subversionWithJava  subversion-1.2.3
    subversion          subversion-1.3.2
    subversion14        subversion-1.4.0pre-rc1
    xorg.xorgserver     xorg-server-1.1.0
2006-07-25 16:40:38 +00:00
Eelco Dolstra
5744dd5480 * Support the --attr / -A flag in nix-env as well. So now we can do,
e.g.,

  $ nix-env -i -A subversion xorg.xorgserver

  The main advantage over using symbolic names is that using attribute
  names is unambiguous and much, much faster.
2006-07-25 11:53:22 +00:00
Eelco Dolstra
7a3a5d1608 * When there is a domain check, we have to evaluate the argument.
Can't be lazy!
2006-07-24 16:49:28 +00:00
Eelco Dolstra
f4c5531d92 * New language feature: domain checks, which check whether a function
argument has a valid value, i.e., is in a certain domain.  E.g.,

    { foo : [true false]
    , bar : ["a" "b" "c"]
    }: ...

  This previously could be done using assertions, but domain checks
  will allow the buildfarm to automatically extract the configuration
  space from functions.
2006-07-24 16:35:34 +00:00
Eelco Dolstra
57751fdb55 * Refactoring to support domain checks. 2006-07-24 15:16:03 +00:00
Eelco Dolstra
a4273156c4 * Use $(libexecdir) to find find-runtime-roots.pl. 2006-07-20 13:21:37 +00:00
Eelco Dolstra
ee2cf45d76 * Use debug(). 2006-07-20 12:19:55 +00:00
Eelco Dolstra
c15f544356 * Call find-runtime-roots.pl from the garbage collector to prevent
running applications etc. from being garbage collected.
2006-07-20 12:17:25 +00:00
Eelco Dolstra
4f3725b167 * Better error messages (especially wrt types). 2006-07-19 15:36:15 +00:00
Eelco Dolstra
e10b830251 * Doh! Of couse we cannot memoize across scopes. 2006-07-11 10:29:52 +00:00
Eelco Dolstra
d51aede4af * Allow the canonical system name to be specified at runtime in the
Nix config file.
2006-07-06 15:30:37 +00:00
Eelco Dolstra
a945fb7905 * `nix-env --upgrade --eq': only upgrade if the old version is equal
to the new version.  This is actually useful.
2006-06-27 12:17:25 +00:00
Eelco Dolstra
dbf6d7e783 * Concurrent GC on Cygwin. 2006-06-20 17:48:10 +00:00
Eelco Dolstra
cc51f9c539 * Oops. 2006-06-19 16:35:35 +00:00
Eelco Dolstra
5bb3444032 * _exit() doesn't seem to work right on Cygwin. 2006-06-19 16:24:15 +00:00
Eelco Dolstra
b35735d8b2 * On Windows we cannot delete open (lock) files, so we delete lock
files after we've closed them.  Since this only succeeds if the lock
  is no longer opened by any process, the token trick used on Unix is
  not necessary.
2006-06-19 14:43:13 +00:00
Eelco Dolstra
0e783e5579 * Write messages to stderr in a slightly more atomic way. Useful when
there are several parallel processes.
2006-06-19 14:37:35 +00:00
Eelco Dolstra
c937b73622 * Show when we're blocked waiting for a lock. 2006-06-16 10:13:03 +00:00
Eelco Dolstra
588cb0eade * In `nix-env -i|-u|-e', lock the profile to prevent races between
concurrent nix-env operations on the same profile.  Fixes NIX-7.
2006-06-15 11:56:49 +00:00
Eelco Dolstra
b454977909 * Fix for a problem with BSD's group ownership semantics when the user
is not in the "wheel" group.
2006-06-14 11:53:55 +00:00
Eelco Dolstra
23960e92df * Minor cleanup. 2006-06-01 18:13:33 +00:00
Eelco Dolstra
50fe85f016 * For fixed-output derivations, pass the environment variables listed
in the attribute variable `impureEnvVars' from the caller to the
  builder.
2006-05-31 09:51:45 +00:00
Eelco Dolstra
1390ce4142 * Not all platforms have sys/select.h. 2006-05-30 11:37:21 +00:00
Eelco Dolstra
b1c63dc362 * Don't use badTerm, it gives awful error messages. 2006-05-30 11:31:33 +00:00
Eelco Dolstra
58b4198ed8 * Disable the concurrent garbage collector on Cygwin for now. 2006-05-29 20:46:51 +00:00
Eelco Dolstra
d764409d97 * Some Cygwin fixes. 2006-05-24 13:23:20 +00:00
Eelco Dolstra
b5988004d6 * Support for srcdir != builddir (NIX-41). 2006-05-12 11:47:45 +00:00
Eelco Dolstra
9d72bf8835 * 64-bit compatibility fixes (for problems revealed by building on an Athlon
64 running 64-bit SUSE).  A patched ATerm library is required to run Nix
  succesfully.
2006-05-11 02:19:43 +00:00
Eelco Dolstra
c54287eafe * GCC 2.95 compatibility. 2006-05-08 15:15:13 +00:00
Eelco Dolstra
5cabd47394 * Allow function argument default values to refer to other arguments
of the function.  Implements NIX-45.
2006-05-08 12:52:47 +00:00
Eelco Dolstra
310e605995 * Show evaluation stats when NIX_SHOW_STATS=1. 2006-05-08 10:00:37 +00:00
Eelco Dolstra
0832956089 * Use the new ATermMap. 2006-05-04 12:21:08 +00:00
Eelco Dolstra
d300b4383d * Optimise null-ary term builders. Also declare all term builder
functions as pure, which might improve performance a bit.
2006-05-02 21:58:46 +00:00
Eelco Dolstra
68174bdc7d * Use a linked list of substitutions. This reduces the amount of
copying.
2006-05-02 21:39:02 +00:00
Eelco Dolstra
c791e94aee * Removed a bunch of ATreverses. 2006-05-02 17:51:50 +00:00
Eelco Dolstra
b52e711910 * Huge reduction in memory use (2/3 or so on large nix-env -qas
operations): share ATermMaps between DrvInfos.
2006-05-02 17:12:03 +00:00
Eelco Dolstra
11ae2d1e7a * Memory reduction: replaced expensive calls to ATmakeApplList by
ATmakeApplArray, and got rid of ATreverse in substitute().
2006-05-02 14:07:28 +00:00
Eelco Dolstra
dc719e6ba5 * Some preliminaries towards NIX-45. 2006-05-02 13:39:55 +00:00
Eelco Dolstra
7276e194ee * Disallow unescaped $ in string literals. 2006-05-01 15:29:46 +00:00
Eelco Dolstra
0064599a27 * String interpolation. Expressions like
"--with-freetype2-library=" + freetype + "/lib"

  can now be written as

    "--with-freetype2-library=${freetype}/lib"

  An arbitrary expression can be enclosed within ${...}, not just
  identifiers.

* Escaping in string literals: \n, \r, \t interpreted as in C, any
  other character following \ is interpreted as-is.
  
* Newlines are now allowed in string literals.
2006-05-01 14:01:47 +00:00
Eelco Dolstra
6cecad2be0 * Allow string concatenations involving derivations, e.g.,
configureFlags = "--with-freetype2-library="
      + freetype + "/lib";
2006-05-01 09:56:56 +00:00
Eelco Dolstra
ef2d4a2da9 * Print a more useful stack trace when an error occurs deep in the
derivation dependency graph.
2006-03-24 14:02:44 +00:00
Eelco Dolstra
b69e469328 * In `nix-env', look for derivations inside attribute sets that have
the `recurseForDerivations' attribute set to `true'.
2006-03-23 16:43:07 +00:00
Eelco Dolstra
49ce8b57dd * Hm. 2006-03-23 16:37:49 +00:00
Eelco Dolstra
fdea084c36 * Allow `make check' to work in directories that have symlink
components.
2006-03-10 22:27:26 +00:00
Eelco Dolstra
37d1b1cafd * `nix-env -qa --description' shows human-readable descriptions of
packages (provided that they have a `meta.description' attribute).
  E.g.,

  $ ./src/nix-env/nix-env -qa --description gcc
  gcc-4.0.2   GNU Compiler Collection, 4.0.x (cross-compiler for sparc-linux)
  gcc-4.0.2   GNU Compiler Collection, 4.0.x (cross-compiler for mips-linux)
  gcc-4.0.2   GNU Compiler Collection, 4.0.x (cross-compiler for arm-linux)
  gcc-4.0.2   GNU Compiler Collection, 4.0.x
2006-03-10 16:20:42 +00:00
Eelco Dolstra
2b3b6c9b34 * In theory, this should reduce the number of ATermMap
re-allocations.
2006-03-10 16:14:13 +00:00
Eelco Dolstra
4ada6db1fc * `nix-env -q' now accepts arguments that allow specific derivations
to be queried, e.g., `nix-env -qa firefox'.  This does require the
  argument '*' to be passed if one wants information about all
  derivations, so the old `nix-env -qa' now is `nix-env -qa "*"'.
2006-03-10 10:24:46 +00:00
Eelco Dolstra
18c321308d * Ugh, printHash() was very inefficient because it used
ostringstreams.  Around 11% of execution time was spent here (now
  it's 0.5%).
2006-03-09 17:07:25 +00:00
Eelco Dolstra
b90c00e63f * Regression: semantics of the result of getDerivation() changed. 2006-03-09 15:10:01 +00:00
Eelco Dolstra
922697c8b2 * Big speedup (factor > 2.5) in all nix-env operations that do actual
instantiation, e.g. "nix-env -i" and "nix-env -qas" (but not
  "nix-env -qa").  It turns out that many redundant calls to
  addToStore(path) were made, which reads and hashes the entire path.
  For instance, the bash bootstrap binary in Nixpkgs would be read and
  hashed many times.  As a result nix-env would spend around 92% of
  its time in the function sha256_block (according to callgrind).
  Some simple memoization fixes this.
2006-03-09 15:09:18 +00:00
Eelco Dolstra
6dca5c9099 * When obtaining derivations from Nix expressions, ignore all
expressions that cause an assertion failure (like `assert system ==
  "i686-linux"').  This allows all-packages.nix in Nixpkgs to be used
  on all platforms, even if some Nix expressions don't work on all
  platforms.

  Not sure if this is a good idea; it's a bit hacky.  In particular,
  due to laziness some derivations might appear in `nix-env -qa' but
  disappear in `nix-env -qas' or `nix-env -i'.

  Commit 5000!
2006-03-08 16:03:58 +00:00
Eelco Dolstra
9088dee9e2 * Some refactoring of the exception handling code so that we can catch
Nix expression assertion failures.
2006-03-08 14:11:19 +00:00
Eelco Dolstra
fa72ae1e9c * GCC 4.1 compatibility. 2006-03-06 14:40:10 +00:00
Eelco Dolstra
c8bfb11b34 * `nix-env (-i|-u) --dry-run' now shows exactly which missing paths
will be built or substituted.
2006-03-06 11:21:15 +00:00
Eelco Dolstra
7ba1fd2029 * Regularise help text a bit. 2006-03-06 11:04:39 +00:00
Eelco Dolstra
70dee0f8ca * Flags --to-base32' and --to-base16' to convert between hex and
base-32 hashes.
2006-03-01 18:05:47 +00:00
Eelco Dolstra
e136532800 * Ouch, parseHash32 was completely broken. All digits >= 4 were
parsed as 4.

  For a moment I worried that printHash32 was broken, and that would
  have been really, *really* bad ;-)
2006-03-01 17:59:08 +00:00
Eelco Dolstra
74166f2f44 * db.hh shouldn't depend on the Berkeley DB headers. 2006-03-01 17:44:28 +00:00
Eelco Dolstra
d822bf32e4 * Close the database before the destructor runs. 2006-03-01 16:36:35 +00:00
Eelco Dolstra
b6780b9e10 * Uninitialised variable. Fixes the --delete test. 2006-03-01 14:39:42 +00:00
Eelco Dolstra
b602d2dfdf * Wrong delete. Thanks valgrind. 2006-03-01 14:39:10 +00:00
Eelco Dolstra
458820df6c * Generate valid HTML. 2006-03-01 13:24:47 +00:00
Eelco Dolstra
6b8bb8d74a * Remove dead code. 2006-03-01 12:51:39 +00:00
Eelco Dolstra
46f0cb0869 * In "nix-env -i", print a warning if there are multiple derivations
with the same name *and* version number, and pick the first one
  (this means that the order in which channels appear in
  ~/.nix-channels matters).  E.g.:

    $ nix-env ii aterm
    warning: there are multiple derivations named `aterm-2.4.2'; using the first one
    installing `aterm-2.4.2'
2006-02-17 18:11:45 +00:00
Eelco Dolstra
7a3e715980 * Fix for NIX-31: "nix-env -i foo" installing all derivations named
foo.  Now it will only install the one with the highest version
  number.
2006-02-17 17:47:54 +00:00
Eelco Dolstra
889ef564fd * Fix the infamous NIX-17: nix-env -i prints misleading messages
("installing `foo'" followed by "uninstalling `foo'").
2006-02-17 16:26:28 +00:00
Eelco Dolstra
345a95afe9 * Allow the size of the GC reserved file to be specified in nix.conf
through the new `gc-reserved-space' option.
2006-02-16 13:58:10 +00:00
Eelco Dolstra
651ab439cf * A simple hack to fix NIX-18: the garbage collector cannot run when
the disk is full (because to delete something from the Nix store, we
  need a Berkeley DB transaction, which takes up disk space).  Under
  normal operation, we make sure that there exists a file
  /nix/var/nix/db/reserved of 1 MB.  When running the garbage
  collector, we delete that file before we open the Berkeley DB
  environment.
2006-02-16 13:19:15 +00:00
Eelco Dolstra
d6f586d0ea * Optional switch "--with-openssl=<PATH>" to use OpenSSL's
implementations of MD5, SHA-1 and SHA-256.  The main benefit is that
  we get assembler-optimised implementations of MD5 and SHA-1 (though
  not SHA-256 (at least on x86), unfortunately).  OpenSSL's SHA-1
  implementation on Intel is twice as fast as ours.
2006-02-13 19:52:43 +00:00
Eelco Dolstra
e8475bbd5b * Use a union. 2006-02-13 18:00:08 +00:00
Eelco Dolstra
2d2e28d02c * Override YYMALLOC and YYFREE so that we can call AT[un]protectMemory
on the Bison parse stack.  Otherwise, a garbage collect during
  parsing could lead to a crash.
2006-02-13 13:09:23 +00:00
Eelco Dolstra
982399bb14 * Enable the --attr in nix-build as well (and add -A as an alias).
Example:

    $ nix-build ./all-packages.nix -A xlibs.libX11

  So finally it's easy to perform a test build of a Nix expression!
2006-02-10 17:37:35 +00:00
Eelco Dolstra
c6120352b3 * In nix-instantiate, allow us to specify a "path" to the
derivation(s) we're interested, e.g.,

    $ nix-instantiate ./all-packages.nix --attr xlibs.libX11

  List elements can also be selected:

    $ nix-instantiate ./build-for-release.nix --attr 0.subversion

  This allows a non-ambiguous specification of a derivation.  Of
  course, this should also be added to nix-env and nix-build.
2006-02-10 17:25:59 +00:00
Eelco Dolstra
9e51abc7dc * Make --parse-only work when *not* reading from stdin. 2006-02-10 15:28:47 +00:00
Eelco Dolstra
f848a45739 * Cleanup: use the code shared with nix-env. 2006-02-10 15:14:57 +00:00
Eelco Dolstra
4eb637c799 * When querying all derivations, filter out syntactically equal derivations. 2006-02-08 16:15:07 +00:00
Eelco Dolstra
8688e83194 * When evaluating, automatically call functions with default arguments. 2006-02-08 15:22:30 +00:00
Eelco Dolstra
287d0ef41c * Oops, fix breakage. 2006-02-08 14:32:33 +00:00
Eelco Dolstra
39f50db731 * Refactoring: move derivation evaluation to libexpr. 2006-02-08 13:21:16 +00:00
Eelco Dolstra
81de538e46 * Use setsid instead of setpgrp in child processes. This not only
creates a new process group but also a new session.  New sessions
  have no controlling tty, so child processes like ssh cannot open
  /dev/tty (which is bad).
2006-02-03 14:20:59 +00:00
Eelco Dolstra
b90daaaf6c * When killing a build hook, kill the entire process group (as
intended).  This ensures that any ssh child processes to remote
  machines are also killed, and thus the Nix process on the remote
  machine also exits.  Without this, the remote Nix process will
  continue until it exists or until its stdout buffer gets full and it
  locks up.  (Partially fixes NIX-35.)
2006-02-02 16:27:31 +00:00
Eelco Dolstra
6e2eaaec96 * Print a better error message when writing the patched file (e.g.,
"No space left on device" instead of "Success").  Reported by Karina
  Olmos.
2006-02-01 17:28:13 +00:00
Eelco Dolstra
8291f6d968 * bsdiff updated to 4.3. This makes Nix depend on libbz2. 2006-02-01 16:48:49 +00:00
Eelco Dolstra
9e4ffc43a2 * The "S" bit should be based on the output path, not the derivation path. 2006-02-01 16:47:51 +00:00
Eelco Dolstra
db2275cb99 * Oops, the "I" bit in "nix-env -qas" was broken. Reported by Nicolae Vintila. 2006-01-26 23:18:26 +00:00
Eelco Dolstra
1029716a8a * Don't show cycles, they're not very useful. 2006-01-19 15:35:34 +00:00
Eelco Dolstra
5b527901ae * dirOf: return "/", not "", for paths in the root directory. Fixes NIX-26. 2006-01-09 14:52:46 +00:00
Eelco Dolstra
04be39734f * Resolve all symlink components in the location of the temporary
build directory (TMPDIR, i.e., /tmp).  Fixes NIX-26.
2006-01-08 17:16:03 +00:00
Eelco Dolstra
0f8d3c871b * More GCC 2.95 compatibility. 2005-12-25 11:29:12 +00:00
Eelco Dolstra
52d20ef124 * Hack around a GCC 2.95 bug. 2005-12-25 02:02:29 +00:00
Eelco Dolstra
1440419b45 * GCC 2.95 compatibility. 2005-12-24 23:32:59 +00:00
Eelco Dolstra
1d2460ee5d * Documentation fixes. 2005-12-24 23:22:05 +00:00
Eelco Dolstra
f96d2dea26 * Added a flag --ignore-liveness' to nix-store --delete'. It
deletes a path even if it is reachable from a root.  However, it
  won't delete a path that still has referrers (since that would
  violate store invariants).

  Don't try this at home.  It's a useful hack for recovering from
  certain situations in a somewhat clean way (e.g., holes in closures
  due to disk corruption).
2005-12-23 21:36:44 +00:00
Eelco Dolstra
4b9e7f59ca * Revived the old "nix-store --delete" operation that deletes the
specified paths from the Nix store.  However, this operation is
  safe: it refuses to delete anything that the garbage collector
  wouldn't delete.
2005-12-23 21:08:42 +00:00
Eelco Dolstra
530b27df1e * `nix-store --gc' prints out the number of bytes freed on stdout
(even when it is interrupted by a signal).
2005-12-15 21:11:39 +00:00
Eelco Dolstra
b1eed6b586 * Split the database upgrade into multiple transactions to prevent
Berkeley DB from running out of locks.
2005-12-15 16:53:21 +00:00
Eelco Dolstra
ab5c6bb3a3 * Change referer' to referrer' throughout. In particular, the
nix-store query options `--referer' and `--referer-closure' have
  been changed to `--referrer' and `--referrer-closure' (but the old
  ones are still accepted for compatibility).
2005-12-13 21:04:48 +00:00
Eelco Dolstra
d87549c1c7 * Automatically delete the old referers table. 2005-12-12 19:14:38 +00:00
Eelco Dolstra
8463f27d8c * Fix NIX-23: quadratic complexity in maintaining the referers
mapping.  The referer table is replaced by a referrer table (note
  spelling fix) that stores each referrer separately.  That is,
  instead of having

    referer[P] = {Q_1, Q_2, Q_3, ...}

  we store

    referer[(P, Q_1)] = ""
    referer[(P, Q_2)] = ""
    referer[(P, Q_3)] = ""
    ...

  To find the referrers of P, we enumerate over the keys with a value
  lexicographically greater than P.  This requires the referrer table
  to be stored as a B-Tree rather than a hash table.

  (The tuples (P, Q) are stored as P + null-byte + Q.)

  Old Nix databases are upgraded automatically to the new schema.
2005-12-12 18:24:42 +00:00
Eelco Dolstra
a33b561a6b * Use Berkeley DB 4.4's process registry feature to recover from
crashed Nix instances, and toss out our own recovery code.
2005-12-09 22:55:07 +00:00
Eelco Dolstra
760264bffe * Require Berkeley DB 4.4.
* Checkpoint after an upgrade.
2005-12-06 15:00:04 +00:00
Eelco Dolstra
b7f008fc35 * Did something useful while waiting at IAD: reference scanning is now
much faster.
2005-11-16 08:27:06 +00:00
Eelco Dolstra
9311ab76a5 * Install signal handlers for SIGTERM and SIGHUP. This ensures that
Nix is properly shut down when it receives those signals.  In
  particular this ensures that killing the garbage collector doesn't
  cause a subsequent database recovery.
2005-11-04 15:34:09 +00:00
Eelco Dolstra
5bf939885a * Memoise checkVarDefs since internally produced terms (i.e., not the
result of parsing) can have very heavy sharing, causing exponential
  complexity if we naively recurse into them.  ATerms are graphs, not
  trees!
2005-11-04 15:17:05 +00:00
Eelco Dolstra
221c79013f * Turn off build hooks in nix-push because of an impurity (NIX-21). 2005-10-29 18:17:45 +00:00
Rob Vermaas
f0856fd905 * Repair the referers table from the references table. 2005-10-29 15:44:02 +00:00
Eelco Dolstra
92d599c6a7 * Prevent uids from being used for more than one build
simultaneously.  We do this using exclusive locks on uid files in
  /nix/var/nix/userpool, e.g., /nix/var/nix/userpool/123 for uid 123.
2005-10-20 16:58:34 +00:00
Eelco Dolstra
e932c40f8e * Oops. Fixed-output derivations were broken. 2005-10-19 14:27:44 +00:00
Eelco Dolstra
13b089c890 * Also kill all processes of the build user after the build. This is
critical to prevent certain kinds of 0wnage.
2005-10-17 17:43:21 +00:00
Eelco Dolstra
f1b3a418fa * Before starting a build under some uid, kill all current processes
running under that uid.
2005-10-17 17:35:37 +00:00
Eelco Dolstra
439823ae80 * Check that the build result is owned by the build user, and that
nobody else has write permission to the build result.  This catches
  most hack attempts.
2005-10-17 16:59:25 +00:00
Eelco Dolstra
7ef574e5d0 * Don't use FIFOs to make Nix create the output path on behalf of the
builder.  Instead, require that the Nix store has sticky permission
  (S_ISVTX); everyone can created files in the Nix store, but they
  cannot delete, rename or modify files created by others.
2005-10-17 16:52:29 +00:00
Eelco Dolstra
32282abcea * Beginning of secure multi-user Nix stores. If Nix is started as
root (or setuid root), then builds will be performed under one of
  the users listed in the `build-users' configuration variables.  This
  is to make it impossible to influence build results externally,
  allowing locally built derivations to be shared safely between
  users (see ASE-2005 paper).

  To do: only one builder should be active per build user.
2005-10-17 15:33:24 +00:00
Eelco Dolstra
dfbf520ec2 * Swap the system and version comparion columns. 2005-10-06 15:51:59 +00:00
Eelco Dolstra
cec2be64f3 * Only colorise if we are attached to a terminal. 2005-10-06 15:01:46 +00:00
Eelco Dolstra
b87b9c0d1f * New query option: --compare-versions' or -c' to compare installed
versions to available versions, or vice versa.

  For example, the following compares installed versions to available
  versions:

    $ nix-env -qc
    autoconf-2.59            = 2.59
    automake-1.9.4           < 1.9.6
    f-spot-0.0.10            - ?
    firefox-1.0.4            < 1.0.7
    ...

  I.e., there are newer versions available (in the current default Nix
  expression) for Automake and Firefox, but not for Autoconf, and
  F-Spot is missing altogether.

  Conversely, the available versions can be compared to the installed
  versions:

    $ nix-env -qac
    autoconf-2.59                  = 2.59
    automake-1.9.6                 > 1.9.4
    bash-3.0                       - ?
    firefox-1.0.7                  > 1.0.4
    ...

  Note that bash is available but no version of it is installed.

  If multiple versions are available for comparison, then the highest
  is used.  E.g., if Subversion 1.2.0 is installed, and Subversion
  1.1.4 and 1.2.3 are available, then `nix-env -qc' will print `<
  1.2.3', not `> 1.1.4'.

  If higher versions are available, the version column is printed in
  red (using ANSI escape codes).
2005-10-06 14:44:54 +00:00
Eelco Dolstra
0e38578433 * log2xml -> nix-log2xml. 2005-10-05 09:42:13 +00:00
Eelco Dolstra
d47e03fccd * Install the XSL stylesheets for log to html conversion. 2005-10-05 09:37:58 +00:00
Eelco Dolstra
ebfe57166d * Lets not go wild with templates. 2005-09-22 17:23:43 +00:00
Eelco Dolstra
4578a490ce * Parse multi-valued options. 2005-09-22 15:43:22 +00:00
Eelco Dolstra
8d06842a76 * Configuration options for trusted local builds. 2005-09-21 12:19:39 +00:00
Eelco Dolstra
ef9e2c8e73 * Typo. 2005-09-20 16:14:00 +00:00
Eelco Dolstra
ed1db42915 * List concatenation must be right-associative for efficiency. 2005-09-14 11:41:59 +00:00
Eelco Dolstra
cf2bb91ec8 * Missing #include. 2005-09-13 13:17:01 +00:00
Eelco Dolstra
2bcd65ecf6 * `nix-env -e' corrupts memory due to incorrect use of iterators.
Reported by Rob Vermaas.
2005-09-01 18:14:04 +00:00
Eelco Dolstra
e1a6fb7870 * `dependencyClosure' now allows a search path, e.g.,
dependencyClosure { ... searchPath = [ ../foo ../bar ]; ... }

* Primop `dirOf' to return the directory part of a path (e.g., dirOf
  /a/b/c == /a/b).

* Primop `relativise' (according to Webster that's a real word!) that
  given paths A and B returns a string representing path B relative
  path to A; e.g., relativise /a/b/c a/b/x/y => "../x/y".
2005-08-14 14:00:39 +00:00
Eelco Dolstra
08c53923db * A primitive operation `dependencyClosure' to do automatic dependency
determination (e.g., finding the header files dependencies of a C
  file) in Nix low-level builds automatically.

  For instance, in the function `compileC' in make/lib/default.nix, we
  find the header file dependencies of C file `main' as follows:

    localIncludes =
      dependencyClosure {
        scanner = file:
          import (findIncludes {
            inherit file;
          });
        startSet = [main];
      };

  The function works by "growing" the set of dependencies, starting
  with the set `startSet', and calling the function `scanner' for each
  file to get its dependencies (which should yield a list of strings
  representing relative paths).  For instance, when `scanner' is
  called on a file `foo.c' that includes the line

    #include "../bar/fnord.h"

  then `scanner' should yield ["../bar/fnord.h"].  This list of
  dependencies is absolutised relative to the including file and added
  to the set of dependencies.  The process continues until no more
  dependencies are found (hence its a closure).

  `dependencyClosure' yields a list that contains in alternation a
  dependency, and its relative path to the directory of the start
  file, e.g.,

    [ /bla/bla/foo.c
      "foo.c"
      /bla/bar/fnord.h
      "../bar/fnord.h"
    ]

  These relative paths are necessary for the builder that compiles
  foo.c to reconstruct the relative directory structure expected by
  foo.c.

  The advantage of `dependencyClosure' over the old approach (using
  the impure `__currentTime') is that it's completely pure, and more
  efficient because it only rescans for dependencies (i.e., by
  building the derivations yielded by `scanner') if sources have
  actually changed.  The old approach rescanned every time.
2005-08-14 12:38:47 +00:00
Eelco Dolstra
0399365675 * nix-hash: option `--truncate' to truncate the hash to 160 bits. Hmm,
kind of ad hoc ;-)
2005-08-14 10:09:56 +00:00
Eelco Dolstra
c680f835c9 * Escape ASCII characters < 32 to Unicode FFFD (REPLACEMENT CHARACTER)
so that we don't produce un-wellformed XML.
2005-08-01 13:39:19 +00:00
Eelco Dolstra
f450c8ea2f * Oops. XSL stylesheet to mark errors. 2005-08-01 13:24:04 +00:00
Eelco Dolstra
800a6ff845 * Mark error lines in red, and expand subtrees containing errors
automatically.
2005-08-01 13:23:43 +00:00
Eelco Dolstra
991a130b1e * Added a list concatenation operator:
[1 2 3] ++ [4 5 6] => [1 2 3 4 5 6]
2005-07-25 15:05:34 +00:00
Eelco Dolstra
4c20a08293 * Build dynamic libraries. 2005-07-22 14:52:45 +00:00
Eelco Dolstra
0f827cc607 * Prevent repeated wrapping of closed terms
(closed(closed(closed(...)))) since this reduces performance by
  producing bigger terms and killing caching (which incidentally also
  prevents useful infinite recursion detection).
2005-07-19 11:48:05 +00:00
Eelco Dolstra
2135e7c041 * Wat cleanups. 2005-07-16 23:19:20 +00:00
Eelco Dolstra
6f82a78de7 * Define paths using regexps, as is done in the Flex definition. 2005-07-16 21:38:15 +00:00
Eelco Dolstra
6f91f02f75 * Make the rejects a bit more compact.
* Add lexical restrictions for keywords.
2005-07-16 20:43:58 +00:00
Eelco Dolstra
9590009a74 * Fix ambiguity. 2005-07-16 15:41:27 +00:00
Eelco Dolstra
f797cb5855 * Revive and update the SDF grammar for Nix expressions. 2005-07-16 14:07:35 +00:00
Eelco Dolstra
22d3587f3b * In nix-instantiate, at top-level, call functions that have arguments
with default values automatically.  I.e., e -> e {}.

  This feature makes convenience expressions such as
  pkgs/system/i686-linux.nix in Nixpkgs obsolete, since we can just do

  $ nix-instantiate ./pkgs/system/all-packages.nix

  since all-packages.nix takes a single argument (system) that has a
  default value (__thisSystem).
2005-07-12 16:06:25 +00:00
Eelco Dolstra
040140dd1c * Added a primop `removeAttrs' to remove attributes from a set, e.g.,
`removeAttrs attrs ["x", "y"]' returns the set `attrs' with the
  attributes named `x' and `y' removed.  It is not an error for the
  named attributes to be missing from the input set.
2005-05-18 17:19:21 +00:00
Eelco Dolstra
109cde6706 * Ignore (with a warning) invalid garbage collector roots. 2005-05-10 14:56:10 +00:00
Eelco Dolstra
456f3251d2 * Typo. 2005-05-10 14:21:46 +00:00
Eelco Dolstra
9ec7e58aa4 * Handle store path arguments in `nix-env -i' correctly again. 2005-05-09 17:55:35 +00:00
Eelco Dolstra
8f57634c14 * Automatically upgrade the Berkeley DB environment if necessary. 2005-05-09 15:25:47 +00:00
Eelco Dolstra
edd145d2fb * Lazily compute the derivation and output paths of derivations. This
makes most query and installation operations much faster (e.g.,
  `nix-env -qa' on the current Nixpkgs is about 10 times faster).
2005-05-08 10:32:09 +00:00
Eelco Dolstra
426593162e * ATermMap needs an assignment operator, otherwise we are screwed. 2005-05-08 10:28:19 +00:00
Eelco Dolstra
77557a6f06 Commit 3000!
* Make the `derivation' primitive much more lazy.  The expression
  `derivation attrs' now evaluates to (essentially)

    attrs // {
      type = "derivation";
      outPath = derivation! attrs;
      drvPath = derivation! attrs;
    }

  where `derivation!' is a primop that does the actual derivation
  instantiation (i.e., it does what `derivation' used to do).  The
  advantage is that it allows commands such as `nix-env -qa' and
  `nix-env -i' to be much faster since they no longer need to
  instantiate all derivations, just the `name' attribute.  (However,
  `nix-env' doesn't yet take advantage of this since it still always
  evaluates the `outPath' and `drvPath' attributes).

  Also, this allows derivations to cyclically reference each other,
  for example,

    webServer = derivation {
      ...
      hostName = "svn.cs.uu.nl";
      services = [svnService];
    };

    svnService = derivation {
      ...
      hostName = webServer.hostName;
    };

  Previously, this would yield a black hole (infinite recursion).
2005-05-07 21:48:49 +00:00
Eelco Dolstra
6057b51835 * Don't try to register GC roots in read-only mode. 2005-05-07 21:33:31 +00:00
Eelco Dolstra
52a2f41320 * Include some required header files. 2005-05-04 16:33:20 +00:00
Eelco Dolstra
26fd28432d * FreeBSD 4.x doesn't have stdint.h, use inttypes.h instead (which is
also part of ISO C).
2005-05-04 16:32:54 +00:00
Eelco Dolstra
5dea0622d1 * Idem (constness fix).
* `compare' in GCC 2.95 is broken.
2005-05-04 16:31:49 +00:00
Eelco Dolstra
4a266e35d4 * GCC 2.95 compatibility fix in constness; strangely, I think this
should not have worked at all.
2005-05-04 16:31:24 +00:00
Eelco Dolstra
d7b3cdbd91 * GCC 2.95 compatibility. Prevents internal compiler error in member
template friends.
2005-05-04 16:30:35 +00:00
Eelco Dolstra
ae6d9033a1 * The eof() state isn't guaranteed to be set non-lazily. GCC 2.95
compatibility fix.
2005-05-04 16:29:44 +00:00
Eelco Dolstra
02f2da0142 * Merging from nix-make branch:
- Add __currentTime primitive (dangerous!).
  - Allow imports of derivations.
2005-05-02 14:44:58 +00:00
Eelco Dolstra
f913283570 * Remove redundant message. 2005-04-13 09:20:27 +00:00
Eelco Dolstra
f3660b1c8c * Garbage collector fix: allow deletion of paths that have invalid
(but substitutable) referers.
2005-04-12 10:51:00 +00:00
Eelco Dolstra
cab7816b56 * Slightly nicer message. 2005-04-11 08:07:41 +00:00
Eelco Dolstra
c9c58dba55 * Primop `__currentSystem' to return the current platform identifier. 2005-04-10 17:38:19 +00:00
Eelco Dolstra
c702dfca3f * nix-store: --substitute' -> --register-substitutes'. 2005-04-08 13:48:41 +00:00
Eelco Dolstra
4271385a73 * Make `nix-store --query --tree' work on non-derivations (i.e., on
any store path).
2005-04-08 12:57:16 +00:00
Eelco Dolstra
10c429c757 * If store paths are specified as sources in Nix expressions, don't
copy them, but use them directly.
2005-04-07 14:35:01 +00:00
Eelco Dolstra
f9848d4f31 * Support base-32 hash representations. 2005-04-07 14:33:32 +00:00
Eelco Dolstra
c815aff21b * `nix-store --add-fixed' to preload the outputs of fixed-output
derivations.  This is mostly to simplify the implementation of
  nix-prefetch-{url, svn}, which now work properly in setuid
  installations.

* Enforce valid store names in `nix-store --add / --add-fixed'.
2005-04-07 14:01:51 +00:00
Eelco Dolstra
6f788880b6 * Re-enable dot graph generation. 2005-03-26 22:06:57 +00:00
Eelco Dolstra
298dd487bb * When finding live paths, the deriver need not be valid. 2005-03-25 14:31:12 +00:00
Eelco Dolstra
7eaf038763 * `nix-store --verify': repair bad referer mappings. 2005-03-25 14:21:49 +00:00
Eelco Dolstra
c6178f0b03 * Create missing log and temproots directories automatically (reported
by Rob).
2005-03-24 17:46:38 +00:00
Eelco Dolstra
cff6bc06df * Fix endianness bug. 2005-03-23 19:18:22 +00:00
Eelco Dolstra
0df9f08078 * Export the references graph to the build hook. 2005-03-23 13:16:36 +00:00
Eelco Dolstra
3f236f01ae * `nix-store --register-validity': allow a path to refer to a path
listed later in the list of new valid paths.
2005-03-23 13:07:28 +00:00
Eelco Dolstra
a04c62e0c4 * Canonicalise path meta-data in `nix-store --register-validity'. 2005-03-23 12:06:57 +00:00
Eelco Dolstra
f20f081560 * nix-store: --isvalid' -> --check-validity', `--validpath' ->
`--register-validity'.
* `nix-store --register-validity': read arguments from stdin, and
  allow the references and deriver to be set.
2005-03-23 11:25:20 +00:00
Eelco Dolstra
c757d16c8c * Bug in clearSubstitutes(). 2005-03-14 18:54:40 +00:00
Eelco Dolstra
08df443618 * Check for duplicate attributes and formal parameters in Nix
expressions.
2005-03-10 11:33:46 +00:00
Eelco Dolstra
4bbdcfbb45 * Don't use fork() in copyPath(), but a string buffer. 2005-03-03 13:58:02 +00:00
Eelco Dolstra
86cb3cc554 * Increase Berkeley DB limits a bit more.
* Maintain the cleanup invariant in clearSubstitutes().
2005-03-03 13:10:44 +00:00
Eelco Dolstra
07b4399fb6 * `nix-store -q --hash' to quickly query the hash of the contents of a
store path (which is stored in the database).
2005-03-02 15:57:06 +00:00
Eelco Dolstra
bfaf83a0fd * When multiple derivations are specified in `nix-store -r', don't
continue building when one fails unless `--keep-going' is
  specified.
* When `--keep-going' is specified, print out the set of failing
  derivations at the end (otherwise it can be hard to find out which
  failed).
2005-02-23 11:19:27 +00:00
Eelco Dolstra
3a2c3f0cf2 * Support for fixed-output hashes over directory trees (i.e., over the
NAR dump of the path).
2005-02-22 21:14:41 +00:00
Eelco Dolstra
eda2c3c253 * Compatibility hack so that Nixpkgs can continue to do hash checking
in `fetchurl' in Nix <= 0.7, but doesn't in Nix 0.8.
2005-02-22 15:23:24 +00:00
Eelco Dolstra
3c1630131e * Subtle bug in the builder: if a subgoal that is instantiated
multiple times is also a top-level goal, then the second and later
  instantiations would never be created because there would be a
  stable pointer to the first one that would keep it alive in the
  WeakGoalMap.
* Some tracing code for debugging this kind of problem.
2005-02-18 09:50:20 +00:00
Eelco Dolstra
e0181f56be * `nix-store -q --tree' shows a tree representing the dependency graph
of the given derivation.  Useful for getting a quick overview of how
  something was built.  E.g., to find out how the `baffle' program in
  your user environment was built, you can do

    $ nix-store -q --tree $(nix-store -qd $(which baffle))

  Tree nesting depth is minimised (?) by topologically sorting paths
  under the relation A < B iff A \in closure(B).
2005-02-17 15:57:46 +00:00
Eelco Dolstra
74ab0695b5 * Compatibility hack with older user environments. 2005-02-17 15:48:50 +00:00
Eelco Dolstra
8a3a96dd5b * Switch to the calling user context for some more operations in a
setuid installation.
2005-02-17 13:55:18 +00:00
Eelco Dolstra
88273f9574 * Put build logs in $prefix/var/nix/log/drvs/. 2005-02-17 13:54:45 +00:00
Eelco Dolstra
202d5bbda5 * Compatibility with older GCCs. 2005-02-15 12:05:47 +00:00
Eelco Dolstra
e17910cfb5 * And yet another installation source: the ability to copy user
environment elements from one user environment to another, e.g.,

    $ nix-env -i --from-profile /nix/var/nix/profiles/other-profile aterm
    
  copies the `aterm' component installed in the `other-profile' to the
  user's current profile.
2005-02-15 10:49:31 +00:00
Eelco Dolstra
8992fce3da * It is now possible to add store derivations or paths directly to a
user environment, e.g.,

    $ nix-env -i /nix/store/z58v41v21xd3ywrqk1vmvdwlagjx7f10-aterm-2.3.1.drv

  or 

    $ nix-env -i /nix/store/hsyj5pbn0d9iz7q0aj0fga7cpaadvp1l-aterm-2.3.1

  This is useful because it allows Nix expressions to be bypassed
  entirely.  For instance, if only a nix-pull manifest is provided,
  plus the top-level path of some component, it can be installed
  without having to supply the Nix expression (e.g., for obfuscation,
  or to be independent of Nix expression language changes or context
  dependencies).
2005-02-14 17:35:10 +00:00
Eelco Dolstra
e446d342b7 * Added an installation source --from-expression' (or -E') to
install derivations from a Nix expression specified on the command
  line.  This is particularly useful for disambiguation if there are
  multiple derivations with the same name.  For instance, in Nixpkgs,
  to install the Firefox wrapper rather than the plain Firefox
  component:

    $ nix-env -f .../i686-linux.nix -i -E 'x: x.firefoxWrapper'

  The Nix expressions should be functions to which the default Nix
  expression (in this case, `i686-linux.nix') is passed, hence `x:
  ...'.

  This might also be a nice way to deal with high-level (user-level)
  variability, e.g.,

    $ nix-env -f ./server.nix -i -E 'x: x {port = 8080; ssl = false;}'
2005-02-14 17:07:43 +00:00
Eelco Dolstra
0cb016c209 * Refactoring. Hope this doesn't break the semantics of `-u' ;-) 2005-02-14 16:16:02 +00:00
Eelco Dolstra
a04a5de8f7 * Implement the `gc-keep-derivations' global configuretion flag. 2005-02-14 14:16:56 +00:00
Eelco Dolstra
6a8ef36fe6 * Global configuration option `env-keep-derivations' to store pointer
to derivations in user environments.  Nice for developers (since it
  prevents build-time-only dependencies from being GC'ed, in
  conjunction with `gc-keep-outputs').  Turned off by default.
2005-02-14 13:07:09 +00:00
Eelco Dolstra
b0aba6ec2a * Don't keep the derivation symlink when creating profile generations. 2005-02-14 10:44:57 +00:00
Eelco Dolstra
32429142cd * Type error in constructor call (caught by GCC 3.3, but not 3.4!). 2005-02-14 09:53:11 +00:00
Eelco Dolstra
20ce2642fc * Refactoring to support different installation sources in nix-env.
* Set the references for the user environment manifest properly.
* Don't copy the manifest (this was accidental).
* Don't store derivation paths in the manifest (maybe this should be
  made optional).  This cleans up the semantics of nix-env, which were
  weird.
* Hash on the output paths of activated components, not on derivation
  paths.  This is because we don't know the derivation path of already
  installed components anymore, and it allows the installation of
  components by store path (skipping Nix expressions entirely).
* Query options `--out-path' and `--drv-path' to show the output and
  derivation paths of components, respectively (the latter replaces
  the `--expr' query).
2005-02-11 16:56:45 +00:00
Eelco Dolstra
80870d9291 * Input sources should be in the set of all referenceable paths too. 2005-02-11 16:03:47 +00:00
Eelco Dolstra
3a99616968 * Commit more often to prevent out-of-memory errors. 2005-02-09 14:37:24 +00:00
Eelco Dolstra
98df735b51 * Propagate the deriver of a path through the substitute mechanism.
* Removed some dead code (successor stuff) from nix-push.
* Updated terminology in the tests (store expr -> drv path).
* Check that the deriver is set properly in the tests.
2005-02-09 12:57:13 +00:00
Eelco Dolstra
582e01c06f * Automatically upgrade <= 0.7 Nix stores to the new schema (so that
existing user environments continue to work).
* `nix-store --verify': detect incomplete closures.
2005-02-09 09:50:29 +00:00
Eelco Dolstra
c547439843 * Subflag in --verify': nix-store --verify --check-contents' checks
that the contents of store paths has not changed by comparing hashes
  of their current contents to the hashes stored in the database.
2005-02-08 13:48:53 +00:00
Eelco Dolstra
3d74274b37 * Updated `nix-store --verify' to the new schema. 2005-02-08 13:23:55 +00:00
Eelco Dolstra
fbc434ee4c * `nix-store -qb' to query derivation environment bindings. Useful
for finding build-time dependencies (possibly after a build).  E.g.,

    $ nix-store -qb aterm $(nix-store -qd $(which strc))
    /nix/store/jw7c7s65n1gwhxpn35j9rgcci6ilzxym-aterm-2.3.1

* Arguments to nix-store can be files within store objects, e.g.,
  /nix/store/jw7c...-aterm-2.3.1/bin/baffle.

* Idem for garbage collector roots.
2005-02-07 14:32:44 +00:00
Eelco Dolstra
450c358e20 * Maintain a database table (`derivers') that maps output paths to the
derivation that produced them.
* `nix-store -qd PATH' prints out the derivation that produced a path.
2005-02-07 13:40:40 +00:00
Eelco Dolstra
a37338815d * A GC setting `gc-keep-outputs' to specify whether output paths of
derivations should be kept.
2005-02-01 22:07:48 +00:00
Eelco Dolstra
65b6c8ab4c * Move root finding from nix-collect-garbage' to nix-store --gc'.
This was necessary becase root finding must be done after
  acquisition of the global GC lock.

  This makes `nix-collect-garbage' obsolete; it is now just a wrapper
  around `nix-store --gc'.

* Automatically remove stale GC roots (i.e., indirect GC roots that
  point to non-existent paths).
2005-02-01 15:05:32 +00:00
Eelco Dolstra
630ae0c9d7 * nix-build: use an indirection scheme to make it easier for users to
get rid of GC roots.  Nix-build places a symlink `result' in the
  current directory.  Previously, removing that symlink would not
  remove the store path being linked to as a GC root.  Now, the GC
  root created by nix-build is actually a symlink in
  `/nix/var/nix/gcroots/auto' to `result'.  So if that symlink is
  removed the GC root automatically becomes invalid (since it can no
  longer be resolved).  The root itself is not automatically removed -
  the garbage collector should delete dangling roots.
2005-02-01 13:48:46 +00:00
Eelco Dolstra
dcc37c236c * nix-store, nix-instantiate: added an option `--add-root' to
immediately add the result as a permanent GC root.  This is the only
  way to prevent a race with the garbage collector.  For instance, the
  old style

    ln -s $(nix-store -r $(nix-instantiate foo.nix)) \
      /nix/var/nix/gcroots/result

  has two time windows in which the garbage collector can interfere
  (by GC'ing the derivation and the output, respectively).  On the
  other hand,

    nix-store --add-root /nix/var/nix/gcroots/result -r \
      $(nix-instantiate --add-root /nix/var/nix/gcroots/drv \
        foo.nix)

  is safe.

* nix-build: use `--add-root' to prevent GC races.
2005-02-01 12:36:25 +00:00
Eelco Dolstra
06b4424286 * Add missing files to dist.
* Fix GC and substitute bugs related to self-references.  Add a
  regression test.
2005-02-01 09:23:38 +00:00
Eelco Dolstra
32fa82a56a * Acquire a global GC lock to prevent new temporary root files from
being created after the garbage collector has read the temproots
  directory.  This blocks the creation of new processes, but the
  garbage collector could periodically release the GC lock to allow
  them to run.
2005-01-31 22:23:49 +00:00
Eelco Dolstra
89c9bc11ab * Add a test for a more subtle race: a process starting after the
temporary root files have been read but creating outputs before the
  store directory has been read.
2005-01-31 22:01:55 +00:00
Eelco Dolstra
207bdcbe86 * Automatically remove temporary root files. 2005-01-31 21:20:59 +00:00
Eelco Dolstra
252c9c91ab * Topologically sort paths under the references relation to ensure
that they are deleted in an order that maintains the closure
  invariant.
* Presence of a path in a temporary roots file does not imply that all
  paths in its closure are also present, so add the closure.
2005-01-31 14:00:43 +00:00
Eelco Dolstra
33c5d23b81 * Don't delete active lock files. 2005-01-31 12:19:53 +00:00
Eelco Dolstra
1328aa3307 * Start of concurrent garbage collection. Processes write temporary
roots to a per-process temporary file in /nix/var/nix/temproots
  while holding a write lock on that file.  The garbage collector
  acquires read locks on all those files, thus blocking further
  progress in other Nix processes, and reads the sets of temporary
  roots.
2005-01-31 10:27:25 +00:00
Eelco Dolstra
22cfdfa246 * Use NIX_STORE environment variable to locate the store (in addition
to NIX_STORE_DIR) so that Nix invocations in builders in `make
  check' work correctly if the store doesn't exist.
2005-01-28 13:19:16 +00:00
Eelco Dolstra
0ea8b6993a * Only invalidate paths when they are in fact valid. 2005-01-28 11:05:46 +00:00
Eelco Dolstra
a85d1849af * Missing dependency; only a problem when building from Subversion. 2005-01-27 19:00:48 +00:00
Eelco Dolstra
8a3eef22e3 * Fix deadlock. 2005-01-27 17:48:14 +00:00
Eelco Dolstra
c60a4943ba * Update referers mappings when updating/clearing the references
mapping.
* Do things in the right order in invalidatePath().
2005-01-27 16:18:39 +00:00
Eelco Dolstra
c505702265 * Fix and simplify the garbage collector (it's still not concurrent,
though).  In particular it's now much easier to register a GC root.
  Just place a symlink to whatever store path it is that you want to
  keep in /nix/var/nix/gcroots.
2005-01-27 15:21:29 +00:00
Eelco Dolstra
59682e6188 * Make lock removal safe by signalling to blocked processes that the
lock they are waiting on has become stale (we do this by writing a
  meaningless token to the unlinked file).
2005-01-27 12:19:25 +00:00
Eelco Dolstra
a24b78e9f1 * Maintain the references/referers relation also for derivations.
This simplifies garbage collection and `nix-store --query
  --requisites' since we no longer need to treat derivations
  specially.

* Better maintaining of the invariants, e.g., setReferences() can only
  be called on a valid/substitutable path.
2005-01-25 21:28:25 +00:00
Eelco Dolstra
2a2756b856 * Simplification: registerSubstitutes -> registerSubstitute. We no
longer need the former since there we no longer have the
  substitutes-rev table (which triggered a O(n^2) cost in updating
  them).
2005-01-25 20:27:40 +00:00
Eelco Dolstra
066da4ab85 * Really fix the substitute mechanism, i.e., ensure the closure
invariant by registering references through the manifest.
* Added a test for nix-pull.
2005-01-25 17:08:52 +00:00
Eelco Dolstra
c6290e42bc * Fix the `--fallback' switch.
* Fix the substitutes tests.
2005-01-25 13:00:12 +00:00
Eelco Dolstra
581fc47783 * Fix the build hook mechanism; pass the pointer graph to the hook. 2005-01-25 11:55:43 +00:00
Eelco Dolstra
52bf9b86bb * In nix-store: added query `--referers-closure' that returns the
closure of the referers relation rather than the references
  relation, i.e., the set of all paths that directly or indirectly
  refer to the given path.  Note that contrary to the references
  closure this set is not fixed; it can change as paths are added to
  or removed from the store.
2005-01-25 11:18:03 +00:00
Eelco Dolstra
80faa2f98a * In nix-store: change --build' back to --realise'. Also brought
back the query flag `--force-realise'.
* Fixed some of the tests.
2005-01-25 10:55:33 +00:00
Eelco Dolstra
6a0a2d5593 * Terminology fixes. 2005-01-20 16:01:07 +00:00
Eelco Dolstra
6bb5efadec * Ensure that derivation names and sources don't end in `.drv'. 2005-01-20 15:25:01 +00:00
Eelco Dolstra
05f0430de1 * Another change to low-level derivations. The last one this year, I
promise :-) This allows derivations to specify on *what* output
  paths of input derivations they are dependent.  This helps to
  prevent unnecessary downloads.  For instance, a build might be
  dependent on the `devel' and `lib' outputs of some library
  component, but not the `docs' output.
2005-01-20 14:10:19 +00:00
Eelco Dolstra
e0f4e587c3 * Nix-store queries --references' and referers' to query the pointer
graph.  That is, `nix-store --query --references PATH' shows the set
  of paths referenced by PATH, and `nix-store --query --referers PATH'
  shows the set of paths referencing PATH.
2005-01-19 16:59:56 +00:00
Eelco Dolstra
96de272b48 * Renamed normalise.cc' -> build.cc', `storeexprs.cc' ->
`derivations.cc', etc.
* Store the SHA-256 content hash of store paths in the database after
  they have been built/added.  This is so that we can check whether
  the store has been messed with (a la `rpm --verify').
* When registering path validity, verify that the closure property
  holds.
2005-01-19 16:39:47 +00:00
Eelco Dolstra
ef5f254a55 * `nix-store --build' now builds its arguments in parallel instead of
sequentially (within the limits set by `--jobs').  This should
  greatly improve the utilisation of the build farm when doing Nixpkgs
  builds.
2005-01-19 15:02:02 +00:00
Eelco Dolstra
06c77bf7a8 * Change extension .store' to .drv'.
* Re-enable `nix-store --query --requisites'.
2005-01-19 14:36:00 +00:00
Eelco Dolstra
863dcff6c5 * Started removing closure store expressions, i.e., the explicit
representation of closures as ATerms in the Nix store.  Instead, the
  file system pointer graph is now stored in the Nix database.  This
  has many advantages:

  - It greatly simplifies the implementation (we can drop the notion
    of `successors', and so on).

  - It makes registering roots for the garbage collector much easier.
    Instead of specifying the closure expression as a root, you can
    simply specify the store path that must be retained as a root.
    This could not be done previously, since there was no way to find
    the closure store expression containing a given store path.
    
  - Better traceability: it is now possible to query what paths are
    referenced by a path, and what paths refer to a path.
2005-01-19 11:16:11 +00:00
Eelco Dolstra
e9762e2d10 * Support arities > 6. 2005-01-19 11:04:24 +00:00
Eelco Dolstra
6d493751c3 * Get --readonly-mode to work again. 2005-01-18 11:15:50 +00:00
Eelco Dolstra
32aac8748a * Actually check that the result of fixed-output derivations matches
the specified hash.
2005-01-17 19:01:48 +00:00
Eelco Dolstra
f3dc231250 * Removed the `id' attribute hack.
* Formalise the notion of fixed-output derivations, i.e., derivations
  for which a cryptographic hash of the output is known in advance.
  Changes to such derivations should not propagate upwards through the
  dependency graph.  Previously this was done by specifying the hash
  component of the output path through the `id' attribute, but this is
  insecure since you can lie about it (i.e., you can specify any hash
  and then produce a completely different output).  Now the
  responsibility for checking the output is moved from the builder to
  Nix itself.

  A fixed-output derivation can be created by specifying the
  `outputHash' and `outputHashAlgo' attributes, the latter taking
  values `md5', `sha1', and `sha256', and the former specifying the
  actual hash in hexadecimal or in base-32 (auto-detected by looking
  at the length of the attribute value).  MD5 is included for
  compatibility but should be considered deprecated.

* Removed the `drvPath' pseudo-attribute in derivation results.  It's
  no longer necessary.

* Cleaned up the support for multiple output paths in derivation store
  expressions.  Each output now has a unique identifier (e.g., `out',
  `devel', `docs').  Previously there was no way to tell output paths
  apart at the store expression level.

* `nix-hash' now has a flag `--base32' to specify that the hash should
  be printed in base-32 notation.

* `fetchurl' accepts parameters `sha256' and `sha1' in addition to
  `md5'.

* `nix-prefetch-url' now prints out a SHA-1 hash in base-32.  (TODO: a
  flag to specify the hash.)
2005-01-17 16:55:19 +00:00
Eelco Dolstra
d58a11e019 * Shorten SHA-256 hashes used in store path name generation to 160
bits, then encode them in a radix-32 representation (using digits
  and letters except e, o, u, and t).  This produces store paths like
  /nix/store/4i0zb0z7f88mwghjirkz702a71dcfivn-aterm-2.3.1.  The nice
  thing about this is that the hash part of the file name is still 32
  characters, as before with MD5.

  (Of course, shortening SHA-256 to 160 bits makes it no better than
  SHA-160 in theory, but hopefully it's a bit more resistant to
  attacks; it's certainly a lot slower.)
2005-01-14 16:04:03 +00:00
Eelco Dolstra
9530cc3170 * Start move towards SHA-256 hashes instead of MD5.
* Start cleaning up unique store path generation (they weren't always
  unique; in particular the suffix ("-aterm-2.2", "-builder.sh") was
  not part of the hash, therefore changes to the suffix would cause
  multiple store objects with the same hash).
2005-01-14 13:51:38 +00:00
Eelco Dolstra
63791eb05b * Add SHA-256.
* Tests for the various hashes.
2005-01-14 12:03:04 +00:00
Eelco Dolstra
37b51a9aa6 * Removed some dead code. 2005-01-14 10:16:33 +00:00
Eelco Dolstra
7e8961f720 * Added SHA-1 support. nix-hash' now has an option --type sha1' to
select SHA-1 hashing.
2005-01-13 17:39:26 +00:00
Eelco Dolstra
73992371a3 * Refactoring to support SHA-1. 2005-01-13 15:44:44 +00:00
Eelco Dolstra
a03397be4c * Cygwin compatibility. 2005-01-04 17:38:26 +00:00
Eelco Dolstra
4f07ebc67e * Integrated bsdiff/bspatch 4.2 (from
http://www.daemonology.net/bsdiff/bsdiff-4.2.tar.gz) into the source
  tree.  The license is a bit peculiar, but it does allow verbatim
  copying, which is what we do here (i.e., so don't make any changes
  to the sources).
2004-12-29 22:08:48 +00:00
Eelco Dolstra
8b9697e575 * An operation `nix-store --clear-substitutes' to remove all
registered substitute mappings.
2004-12-20 14:16:55 +00:00
Eelco Dolstra
fa9259f5f5 * Simplify the substitute mechanism:
- Drop the store expression.  So now a substitute is just a
    command-line invocation (a program name + arguments).  If you
    register a substitute you are responsible for registering the
    expression that built it (if any) as a root of the garbage
    collector.
  - Drop the substitutes-rev DB table.
2004-12-20 13:43:32 +00:00
Eelco Dolstra
13f77276d1 * utime() follows symlinks, so don't change the mtime if the file is a
symlink.
2004-11-29 19:22:16 +00:00
Eelco Dolstra
f17553a212 * Remove debug statement. 2004-11-29 15:30:44 +00:00
Eelco Dolstra
4115d8d8ce * Canonicalise metadata of all files in store paths (set the mtime to
0, set the mode to either 444 or 555, set the group to the default).
2004-11-29 15:09:29 +00:00
Eelco Dolstra
54c7a870d5 * Document --delete-generations and other nix-env options. 2004-11-12 23:56:37 +00:00
Eelco Dolstra
c2b0d6b02f * Document --eval-only and --parse-only options in nix-instantiate. 2004-11-12 23:22:08 +00:00
Eelco Dolstra
92ee003dc9 * Fix broken format string. 2004-11-08 15:20:52 +00:00
Eelco Dolstra
cb7ccb528b * string2ATerm -> overloaded toATerm. 2004-11-03 18:12:03 +00:00
Eelco Dolstra
a69534fc21 * Drop ATmake / ATMatcher also in handling store expressions. 2004-10-29 11:22:49 +00:00
Eelco Dolstra
3277c9432a * Bug fix in parsing of /* ... */ comments; due to longest match
regexp there could be only one such comment per file.
2004-10-27 13:00:31 +00:00
Eelco Dolstra
f09618b63a * Turn on read-only mode in queries. This prevents redundant store I/O. 2004-10-27 10:24:44 +00:00
Eelco Dolstra
c7bea941b0 * Oops, I did it again. 2004-10-27 10:05:51 +00:00
Eelco Dolstra
210ab0296d * Add file to `make dist'. 2004-10-27 00:02:31 +00:00
Eelco Dolstra
c52dda95a6 * Bug: check that term is an application. 2004-10-26 23:30:18 +00:00
Eelco Dolstra
5fe9222b36 * Don't use ATmake / ATmatch anymore, nor the ATMatcher class.
Instead we generate data bindings (build and match functions) for
  the constructors specified in `constructors.def'.  In particular
  this removes the conversions between AFuns and strings, and Nix
  expression evaluation now seems 3 to 4 times faster.
2004-10-26 22:54:26 +00:00
Eelco Dolstra
eb8284ddaa * Evaluate argument to `import'. 2004-10-26 17:10:09 +00:00
Eelco Dolstra
033d7c6593 * Doh! 2004-10-26 17:04:55 +00:00
Eelco Dolstra
9fa07b376d * String/path concatenation operator (`+'). 2004-10-26 17:01:35 +00:00
Eelco Dolstra
ee401afad8 * Mode --parse-only' to parse the input (on stdin, -'), and print
out the AST as an ATerm.
* Mode `--eval-only' to parse and evaluate the input, and print the
  resulting normal form as an ATerm.

Neither of these modes require store/DB write permission.
2004-10-26 16:59:36 +00:00
Eelco Dolstra
37d7abd694 * New language feature: with expressions.
The expression `with E1; E2' evaluates to E2 with all bindings in
  the attribute set E1 substituted.  E.g.,

    with {x = 123;}; x

  evaluates to 123.  That is, the attribute set E1 is in scope in E2.

  This is particularly useful when importing files containing lots
  definitions.  E.g., instead of

    let {
      inherit (import ./foo.nix) a b c d e f;

      body = ... a ... f ...;
    }

  we can now say

    with import ./foo.nix;

    ... a ... f ...

  I.e., we don't have to say what variables should be brought into scope.
2004-10-25 16:54:56 +00:00
Eelco Dolstra
f4d44a0026 * Allow certain operations to succeed even if we don't have write
permission to the Nix store or database.  E.g., `nix-env -qa' will
  work, but `nix-env -qas' won't (the latter needs DB access).  The
  option `--readonly-mode' forces this mode; otherwise, it's only
  activated when the database cannot be opened.
2004-10-25 14:38:23 +00:00
Eelco Dolstra
3ade3e7721 * Revert r1594 - it didn't solve the problem. Instead add
svn-revision to distributions, which should fix it.
2004-10-25 13:51:34 +00:00
Eelco Dolstra
2248becfd3 * Make sure that the prerelease version is included in `--version'. 2004-10-25 12:15:50 +00:00
Eelco Dolstra
d830b2c1df * In `nix-env -q', sort derivations by name *without* case
sensitivity.
2004-10-14 15:09:55 +00:00
Eelco Dolstra
1317242780 * Make store objects created by substitutes read-only. 2004-09-22 12:15:04 +00:00
Eelco Dolstra
b357284a32 * Fallback didn't work for subderivations of an unnormalised the main
derivation, since NormalisationGoal would first run a
  NormalisationGoal on the subderivation (a no-op, since in a
  situation where we need fallback the successor is known), and then
  runs a RealisationGoal on the normal form, which then cannot do a
  fallback because it doesn't know the derivation expression for which
  it is a normal form.

  Tossed out the 2-phase normalisation/realisation in
  NormalisationGoal and SubstitutionGoal since it's no longer needed -
  a RealisationGoal will run a NormalisationGoal if necessary.
2004-09-12 19:08:57 +00:00
Eelco Dolstra
dcc433de47 * Operation `--delete-generations' to delete generations of a
profile.  Arguments are either generation number, or `old' to delete
  all non-current generations.  Typical use:

  $ nix-env --delete-generations old
  $ nix-collect-garbage

* istringstream -> string2Int.
2004-09-10 13:32:08 +00:00
Eelco Dolstra
c16be6ac92 * Remove write permission from store objects after they have been
added to the store.  Bug reported by Martin.
2004-09-09 21:19:20 +00:00
Eelco Dolstra
47f87072ad * A very dirty hack to make setuid installations a bit nicer to use.
Previously there was the problem that all files read by nix-env
  etc. should be reachable and readable by the Nix user.  So for
  instance building a Nix expression in your home directory meant that
  the home directory should have at least g+x or o+x permission so
  that the Nix user could reach the Nix expression.  Now we just
  switch back to the original user just prior to reading sources and
  the like.  The places where this happens are somewhat arbitrary,
  however.  Any scope that has a live SwitchToOriginalUser object in
  it is executed as the original user.

* Back out r1385.  setreuid() sets the saved uid to the new
  real/effective uid, which prevents us from switching back to the
  original uid.  setresuid() doesn't have this problem (although the
  manpage has a bug: specifying -1 for the saved uid doesn't leave it
  unchanged; an explicit value must be specified).
2004-09-09 21:12:53 +00:00
Eelco Dolstra
5396304c73 * Use setre[ug]id() instead of setres[ug]id(), since the former is
more common than the latter (which exists only on Linux and
  FreeBSD).  We don't really care about dropping the saved IDs since
  there apparently is no way to quiry them in any case, so it can't
  influence the build (unlike the effective IDs which are checked by
  Perl for instance).
2004-09-09 15:55:31 +00:00
Eelco Dolstra
e043fc7d0b * Set the umask to known value (0022). This is important in a
setuid installation, since the calling user may have a more fascist
  umask (say, 0077), which would cause the store objects built by Nix
  to be unreadable to anyone other than the Nix user.
2004-09-09 14:16:02 +00:00
Eelco Dolstra
5c443b6550 * Main the `substitutes-rev' table again, but now in a way that
doesn't take \Theta(n^2) space/time complexity.
2004-08-31 16:13:10 +00:00
Eelco Dolstra
c25f2883b1 * Quadruple the Berkeley DB locking limits to get rid of out of memory
errors while running `nix-store --verify'.
2004-08-31 10:50:08 +00:00
Eelco Dolstra
fe122c5a15 * Removed nrWaitees field. It was redundant with waitees.size() and
could get out of sync if multiple input derivations mapped to the
  same closure expression (since waitees is a set).
2004-08-30 11:51:36 +00:00
Eelco Dolstra
eb233e728f * `--min-age' flag in nix-store and nix-collect-garbage to only delete
unreachable paths that haven't been used for N hours.  For instance,
  `nix-collect-garbage --min-age 168' only deletes paths that haven't
  been accessed in the last week.

  This is useful for instance in the build farm where many derivations
  can be shared between consecutive builds, and we wouldn't want a
  garbage collect to throw them all away.  We could of course register
  them as roots, but then we'd to unregister them at some point, which
  would be a pain to manage.  The `--min-age' flag gives us a sort of
  MRU caching scheme.

  BUG: this really shouldn't be in gc.cc since that violates
  mechanism/policy separation.
2004-08-25 16:54:08 +00:00
Eelco Dolstra
fdec72c6cc * `nix-collect-garbage' now actually performs a garbage collection, it
doesn't just print the set of paths that should be deleted.  So
  there is no more need to pipe the result into `nix-store --delete'
  (which doesn't even exist anymore).
2004-08-25 15:39:13 +00:00
Eelco Dolstra
818047881e * Put the garbage collector in nix-store: operation `--gc',
suboperations `--print-live', `--print-dead', and `--delete'.  The
  roots are not determined by nix-store; they are read from standard
  input.  This is to make it easy to customise what the roots are.

  The collector now no longer fails when store expressions are missing
  (which legally happens when using substitutes).  It never tries to
  fetch paths through substitutes.

  TODO: acquire a global lock on the store while garbage collecting.
  
* Removed `nix-store --delete'.
2004-08-25 11:43:49 +00:00
Eelco Dolstra
9994c1dd9f * Validate derivation names. In particular don't allow spaces.
* Drop support for the outPath attribute in derivations.
2004-08-24 11:46:05 +00:00
Eelco Dolstra
8f58733ef1 * The gid should also match. 2004-08-20 15:47:58 +00:00
Eelco Dolstra
1c90fabccc * Unbreak programs that are not setuid (such as nix-hash). 2004-08-20 15:31:46 +00:00
Eelco Dolstra
e77fbe0fa2 * On systems that have the setresuid() and setresgid() system calls to
set the real uid and gid to the effective uid and gid, the Nix
  binaries can be installed as owned by the Nix user and group instead
  of root, so no root involvement of any kind is necessary.

  Linux and FreeBSD have these functions.
2004-08-20 15:22:33 +00:00
Eelco Dolstra
2d35116c13 * Setuid support for sharing a Nix installation between multiple
users.

  If the configure flag `--enable-setuid' is used, the Nix programs
  nix-env, nix-store, etc. are installed with the setuid bit turned on
  so that they are executed as the user and group specified by
  `--with-nix-user=USER' and `--with-nix-group=GROUP', respectively
  (with defaults `nix' and `nix').

  The setuid programs drop all special privileges if they are executed
  by a user who is not a member of the Nix group.

  The setuid feature is a quick hack to enable sharing of a Nix
  installation between users who trust each other.  It is not
  generally secure, since any user in the Nix group can modify (by
  building an appropriate derivation) any object in the store, and for
  instance inject trojans into binaries used by other users.

  The setuid programs are owned by root, not the Nix user.  This is
  because on Unix normal users cannot change the real uid, only the
  effective uid.  Many programs don't work properly when the real uid
  differs from the effective uid.  For instance, Perl will turn on
  taint mode.  However, the setuid programs drop all root privileges
  immediately, changing all uids and gids to the Nix user and group.
2004-08-20 14:49:05 +00:00
Eelco Dolstra
1eddee59f2 * The default verbosity level of all Nix commands is now lvlInfo.
* Builder output is written to standard error by default.
  * The option `-B' is gone.
  * The option `-Q' suppresses builder output.

The result of this is that most Nix invocations shouldn't need any
flags w.r.t. logging.
2004-08-18 12:19:06 +00:00
Eelco Dolstra
62fe5c4a22 * The predecessor of a successor need not be present. This in
particular happens on distributed builds or when using push/pull.
2004-08-11 19:03:13 +00:00
Eelco Dolstra
ae1a1efa41 * Clean up the temporary directory for hook communication (and don't
print out incorrect "build failed" messages).
2004-08-05 14:53:27 +00:00
Eelco Dolstra
d8989b1fb4 * Every real language has a `map' function. 2004-08-04 11:27:53 +00:00
Eelco Dolstra
bbfdd64741 * Allow primops with more that 1 arguments. 2004-08-04 10:59:20 +00:00
Eelco Dolstra
e3a50f7e25 * Creating a file nix-support/no-scan in the output path of a
derivation disables scanning for dependencies.  Use at your own
  risk.  This is a quick hack to speed up UML image generation (image
  are very big, say 1 GB).

  It would be better if the scanner were faster, and didn't read the
  whole file into memory.
2004-08-04 09:25:21 +00:00
Eelco Dolstra
c1a18f543e * Fixed format string error. 2004-07-06 11:21:34 +00:00
Eelco Dolstra
056cd1d3b7 * Don't go into a (sometimes infinite) loop calling the build hook. 2004-07-01 16:24:35 +00:00
Eelco Dolstra
638ce339a5 * Nix-instantiate now accepts sets of derivations (just like nix-env). 2004-07-01 14:25:26 +00:00
Eelco Dolstra
292d6468ec * Nix-env operations now by default filter out any derivations for
system types other than the current system.  I.e., `nix-env -i'
  won't install derivations for other system types, and `nix-env -q'
  won't show them.  The flag `--system-filter SYSTEM' can be used to
  override the system type used for filtering (but not for
  building!).  The value `*' can be used not to filter anything.
2004-07-01 13:56:56 +00:00
Eelco Dolstra
8f6254e823 * Align the columns in the output of `nix-env -q'. 2004-07-01 13:35:10 +00:00
Eelco Dolstra
593bc23d8b * Allow the system attribute of derivations to be queried in
`nix-env -q'.
* Queries can now be combined, e.g., `nix-env -q --status --system'.
2004-07-01 13:13:37 +00:00
Eelco Dolstra
b584253af4 * Include some missing headers. 2004-07-01 11:11:16 +00:00
Eelco Dolstra
f5d5ffe536 * Write build logs to disk again. 2004-06-29 09:41:50 +00:00
Eelco Dolstra
151e61fa5a * By default, `nix-env -i' now deletes previously installed
derivations with names matching the derivations being installed.
  The option `--preserve-installed / -P' overrides this behaviour.
2004-06-28 14:40:26 +00:00
Eelco Dolstra
4d2946c516 * In a realisation goal, check the result of the corresponding
normalisation goal.
2004-06-28 13:51:24 +00:00
Eelco Dolstra
24286e15c9 * `nix-env -u' now allows a specific version to be specified when
upgrading.

  This fixes a bug reported by Martin:

    $ nix-env -i foo-1.0
    $ nix-env -u foo-1.0
    upgrading foo-1.0 to foo-1.1
2004-06-28 13:37:05 +00:00
Eelco Dolstra
91dc023665 * Added a switch `--fallback'. From the manual:
Whenever Nix attempts to realise a derivation for which a closure is
  already known, but this closure cannot be realised, fall back on
  normalising the derivation.

  The most common scenario in which this is useful is when we have
  registered substitutes in order to perform binary distribution from,
  say, a network repository.  If the repository is down, the
  realisation of the derivation will fail.  When this option is
  specified, Nix will build the derivation instead.  Thus, binary
  installation falls back on a source installation.  This option is
  not the default since it is generally not desirable for a transient
  failure in obtaining the substitutes to lead to a full build from
  source (with the related consumption of resources).
2004-06-28 10:42:57 +00:00
Eelco Dolstra
b113edeab7 * A flag `--keep-going / -k' to keep building goals if one fails, as
much as possible.  (This is similar to GNU Make's `-k' flag.)

* Refactoring to implement this: previously we just bombed out when
  a build failed, but now we have to clean up.  In particular this
  means that goals must be freed quickly --- they shouldn't hang
  around until the worker exits.  So the worker now maintains weak
  pointers in order not to prevent garbage collection.

* Documented the `-k' and `-j' flags.
2004-06-25 15:36:09 +00:00
Eelco Dolstra
e4883211f9 * Don't throw an exception when a build fails. Just terminate the
goal and allow the problem to be handled elsewhere (e.g., at
  top-level).
2004-06-25 10:21:44 +00:00
Eelco Dolstra
795d9f8b08 * Obsolete. 2004-06-24 14:36:50 +00:00
Eelco Dolstra
ec32627621 * Multiple and/or failing substitutes now work. 2004-06-24 13:40:38 +00:00
Eelco Dolstra
66c7f34759 * Arghhhhhh 2004-06-22 17:07:32 +00:00
Eelco Dolstra
05a5362d63 * Some more diagnostics changes. 2004-06-22 17:04:10 +00:00
Eelco Dolstra
83ae1723da * Well, it's better than printf. 2004-06-22 15:01:01 +00:00
Eelco Dolstra
72bc9a522f * Started making Nix's diagnostic messages a bit more useful. 2004-06-22 14:48:59 +00:00
Eelco Dolstra
5e2cf44a4d * Put WEXITSTATUS stuff somewhere else. 2004-06-22 11:03:41 +00:00
Eelco Dolstra
84007a0958 * Reduce gratuitous cut & pasting. 2004-06-22 10:21:44 +00:00
Eelco Dolstra
c9fbd2dfd5 * Wrapper class around pids. 2004-06-22 09:51:44 +00:00
Eelco Dolstra
155d7c8dfa * Substitutes should occupy a build slot. 2004-06-22 09:00:31 +00:00
Eelco Dolstra
c4cb6ea2bc * Refactoring. 2004-06-22 08:50:25 +00:00
Eelco Dolstra
2db9748221 * Remove debug output. 2004-06-21 10:01:17 +00:00
Eelco Dolstra
3f3a3ae87b * Acquire a lock on the output path when running a substitute. Also
delete obstructing invalid paths.
2004-06-21 09:35:50 +00:00
Eelco Dolstra
72c857f0eb * Ugh, nasty Heisenbug due to an uninitialiased variable. The bug
only caused a crash if the program was *not* invoked with a high
  verbosity level.
2004-06-21 08:51:55 +00:00
Eelco Dolstra
be1a917beb * Remove obstructing invalid store paths add[Text]ToStore(). 2004-06-21 07:46:02 +00:00
Eelco Dolstra
daf0a923c7 * Wrap calls to registerSubstitute() in a single transaction to
improve throughput.
* Don't build the `substitute-rev' table for now, since it caused
  Theta(N^2) time and log file consumption when adding N substitutes.
  Maybe we can do without it.
2004-06-21 07:38:17 +00:00
Eelco Dolstra
15c60ca1b6 * Disable calls to fsync() since Berkeley DB's DB_TXN_WRITE_NOSYNC
flag doesn't seem to work as advertised.
2004-06-21 07:36:01 +00:00
Eelco Dolstra
112ee89501 * Re-enable support for substitutes in the normaliser.
* A better substitute mechanism.

  Instead of generating a store expression for each store path for
  which we have a substitute, we can have a single store expression
  that builds a generic program that is invoked to build the desired
  store path, which is passed as an argument.

  This means that operations like `nix-pull' only produce O(1) files
  instead of O(N) files in the store when registering N substitutes.
  (It consumes O(N) database storage, of course, but that's not a
  performance problem).

* Added a test for the substitute mechanism.
  
* `nix-store --substitute' reads the substitutes from standard input,
  instead of from the command line.  This prevents us from running
  into the kernel's limit on command line length.
2004-06-20 19:17:54 +00:00
Eelco Dolstra
85ae781765 * Refactoring. 2004-06-20 13:37:51 +00:00
Eelco Dolstra
23bb902d1f * Re-enable build hooks. 2004-06-19 21:45:04 +00:00
Eelco Dolstra
41ec982f31 * Big refactoring. Move to a much more explicitly state machine based
approach.  This makes it much easier to add extra complexity in the
  normaliser / realiser (e.g., build hooks, substitutes).
2004-06-18 18:09:32 +00:00
Eelco Dolstra
3454c685ee * This is also useful. 2004-06-18 16:52:31 +00:00
Eelco Dolstra
6ba26f27c3 * Shared (garbage collecting) pointers. Copied from Boost. 2004-06-18 09:20:51 +00:00
Eelco Dolstra
0b70231b9d * Refactoring. 2004-06-15 13:49:42 +00:00
Eelco Dolstra
1bc6afefac * Cleanup. 2004-06-08 13:21:03 +00:00
Eelco Dolstra
5e4a2272bf * Drain the output of the build hook to show error messages. Ugly
hack.
2004-05-18 14:52:35 +00:00
Eelco Dolstra
8e9fd57ef9 * setpgrp() is not POSIX (and on Mac OS X it's different than on
Linux), so use setpgid().
2004-05-18 09:45:18 +00:00
Eelco Dolstra
ace8872706 * execl() requires a terminating 0.
* When a fast build wakes up a goal, try to start that goal in the
  same iteration of the startBuild() loop of run().  Otherwise no job
  might be started until the next job terminates.
2004-05-14 12:24:29 +00:00
Eelco Dolstra
4fc00cbec1 * Distributed builds and load balancing now seem to work pretty well.
(Though the `build-remote.pl' script has a gigantic race condition).
2004-05-13 22:52:37 +00:00
Eelco Dolstra
25db622454 * Load balancing. `build-remote.pl' will only execute up to a
per-machine maximum number of parallel jobs on a remote machine.
2004-05-13 19:35:46 +00:00
Eelco Dolstra
a8306cb98f * The build hooks used to implement distributed builds can now be run
in parallel.  Hooks are more efficient: locks on output paths are
  only acquired when the hook says that it is willing to accept a
  build job.  Hooks now work in two phases.  First, they should first
  tell Nix whether they are willing to accept a job.  Nix guarantuees
  that no two hooks will ever be in the first phase at the same time
  (this simplifies the implementation of hooks, since they don't have
  to perform locking (?)).  Second, if they accept a job, they are
  then responsible for building it (on the remote system), and copying
  the result back.  These can be run in parallel with other hooks and
  locally executed jobs.

  The implementation is a bit messy right now, though.  

* The directory `distributed' shows a (hacky) example of a hook that
  distributes build jobs over a set of machines listed in a
  configuration file.
2004-05-13 19:14:49 +00:00
Eelco Dolstra
efa5fa1a91 * A switch `-j NUMBER' to set the maximum number of parallel jobs (0 =
no limit).
* Add missing file to distribution.
2004-05-12 14:20:32 +00:00
Eelco Dolstra
aa5a5084e4 * Pass to the build hook all sorts of information useful for
distributing a build action to another machine.  In particular, the
  paths in the input closures, the output paths, and successor mapping
  for sub-derivations.
2004-05-12 13:32:26 +00:00
Eelco Dolstra
8c0b42f857 * An quick and dirty hack to support distributed builds. 2004-05-12 09:35:51 +00:00
Eelco Dolstra
c8d3882cdc * True parallel builds. Nix can now run as many build jobs in
parallel as possible (similar to GNU Make's `-j' switch).  This is
  useful on SMP systems, but it is especially useful for doing builds
  on multiple machines.  The idea is that a large derivation is
  initiated on one master machine, which then distributes
  sub-derivations to any number of slave machines.  This should not
  happen synchronously or in lock-step, so the master must be capable
  of dealing with multiple parallel build jobs.  We now have the
  infrastructure to support this.

  TODO: substitutes are currently broken.
2004-05-11 18:05:44 +00:00
Eelco Dolstra
aea436503e * Ignore interrupt signals while handling an exception.
* Ignore EINTR in reads and writes.
2004-05-11 13:48:25 +00:00
Eelco Dolstra
256eeab711 * Allow the location of the store etc. to be specified using
environment variables.
* Started adding some automatic tests.
* Do a `make check' when building RPMs.
2004-05-04 12:15:30 +00:00
Eelco Dolstra
d4779abc04 * Pass SYSTEM through config.h, and allow spaces. 2004-04-23 15:16:08 +00:00
Eelco Dolstra
d7238bc84e * Don't create $(localstatedir)/nix/profiles if --disable-init-state
is specified.
2004-04-22 07:47:41 +00:00
Eelco Dolstra
f79e9c2d22 * Do initialise state (the DB etc.) when doing a `make install',
unless `--disable-init-state' is passed to configure.
2004-04-21 10:54:46 +00:00
Eelco Dolstra
8e459d919d * Recurse into attribute sets and lists when getting derivations from
an expression.
2004-04-21 09:37:37 +00:00
Eelco Dolstra
a4d2b22c8c * Be stricter in verifying store paths. 2004-04-14 08:08:55 +00:00
Eelco Dolstra
59b94ee18a * When something goes wrong in the evaluation of a Nix expression,
print a nice backtrace of the stack, rather than vomiting a gigantic
  (and useless) aterm on the screen.  Example:

    error: while evaluating file `.../pkgs/system/test.nix':
    while evaluating attribute `subversion' at `.../pkgs/system/all-packages-generic.nix', line 533:
    while evaluating function at `.../pkgs/applications/version-management/subversion/default.nix', line 1:
    assertion failed at `.../pkgs/applications/version-management/subversion/default.nix', line 13

  Since the Nix expression language is lazy, the trace may be
  misleading.  The purpose is to provide a hint as to the location of
  the problem.
2004-04-05 22:27:41 +00:00
Eelco Dolstra
a520b1cbc3 * Print a more useful error message in case of an invalid derivation
binding.
2004-04-02 10:49:37 +00:00
Eelco Dolstra
c4ac2a164a * The recent change in nixpkgs of calling `stdenv.mkDerivation'
instead of `derivation' triggered a huge slowdown in the Nix
  expression evaluator.  Total execution time of `nix-env -qa' went up
  by a factor of 60 or so.

  This scalability problem was caused by expressions such as

    (x: y: ... x ...) a b

  where `a' is a large term (say, the one in
  `all-packages-generic.nix').  Then the first beta-reduction would
  produce

    (y: ... a ...) b

  by substituting `a' for `x'.  The second beta-reduction would then
  substitute `b' for `y' into the body `... a ...', which is a large
  term due to `a', and thus causes a large traversal to be performed
  by substitute() in the second reduction.  This is however entirely
  redundant, since `a' cannot contain free variables (since we never
  substitute below a weak head normal form).

  The solution is to wrap substituted terms into a `Closed'
  constructor, i.e.,

    subst(subs, Var(x)) = Closed(e) iff subs[x] = e

  have substitution not descent into closed terms,

    subst(subs, Closed(x)) = Closed(x)

  and otherwise ignore them for evaluation,

    eval(Closed(x)) = eval(x).

* Fix a typo that caused incorrect substitutions to be performed in
  simple lambdas, e.g., `(x: x: x) a' would reduce to `(x: a)'.
2004-03-30 15:05:35 +00:00
Eelco Dolstra
ac4d39f9db * Added an operator `?' to test for attribute existence, e.g.,
`attrs ? x' yields true iff `attrs' has an attribute named `x'.
2004-03-28 21:15:01 +00:00
Eelco Dolstra
f958bcdf1f * Added an operator `~' to select paths within a derivation. E.g.,
{stdenv, bash}: derivation {
      builder = bash ~ /bin/sh;
      args = ["-e" "-x" ./builder.sh];
      ...
    }

  Here the attribute `builder' will evaluate to, e.g.,
  `/nix/store/1234abcd...-bash-2.0.1/bin/sh'.
2004-03-28 20:58:28 +00:00
Eelco Dolstra
db3e644c1c * Added plain lambdas, e.g., `let { id = x: x; const = x: y: x; }'.
`bla:' is now no longer parsed as a URL.

* Re-enabled support for the `args' attribute in derivations to
  specify command line arguments to the builder, e.g.,

    ...
    builder = /usr/bin/python;
    args = ["-c" ./builder.py];
    ...
2004-03-28 20:34:22 +00:00
Eelco Dolstra
f8cd904e05 * Disallow the Nix store or any of its parents from being symlinks.
This is because the contents of these symlinks are not incorporated
  into the hashes of derivations, and could therefore cause a mismatch
  between the build system and the target system.  E.g., if
  `/nix/store' is a symlink to `/data/nix/store', then a builder could
  expand this path and store the result.  If on the target system
  `/nix/store' is not a symlink, or is a symlink that points somewhere
  else, we have a dangling pointer.

  The trigger for this change is that gcc 3.3.3 does exactly that (it
  applies realpath() to some files, such as libraries, which causes
  our impurity checker to bail out.)

  An annoying side-effect of this change is that it makes it harder to
  move the Nix store to a different file system.  On Linux, bind
  mounts can be used instead of symlink for this purpose (e.g., `mount
  -o bind /data/nix/store /nix/store').
2004-03-27 17:58:04 +00:00
Eelco Dolstra
f0f7a9f299 * Do not close a nesting level twice after close() has been
called explicitly on a Nest object.
2004-03-27 15:33:19 +00:00
Eelco Dolstra
7823db2137 * Some more nesting. 2004-03-22 21:42:28 +00:00
Eelco Dolstra
777e13b94b * Nix now has three different formats for the log information it
writes to stderr:
  
  - `pretty': the old nested style (default)
  - `escapes': uses escape codes to indicate nesting and message
    level; can be processed using `log2xml'
  - `flat': just plain text, no nesting

  These can be set using `--log-type TYPE' or the NIX_LOG_TYPE
  environment variable.
2004-03-22 20:53:49 +00:00
Eelco Dolstra
79bb0008ec * `null' is a normal form. 2004-03-19 14:45:45 +00:00
Eelco Dolstra
e6253b58cd * Escape codes to force line breaks to be ignored. 2004-03-18 21:32:15 +00:00
Eelco Dolstra
3f3c4cce5a * Added an extra escape code to signal "unimportant" messages. If a tree only has
unimportant messages, it is collapsed by the default.
* Also added an optional integer argument to the escape code for opening a nesting 
  level to indicate lack of importance.  If set, the tree is collapsed by default.
2004-03-18 21:04:14 +00:00
Eelco Dolstra
84c617966b * Collapsable trees. 2004-03-18 18:26:22 +00:00
Eelco Dolstra
c2fc2c13c9 * Use unordered lists, which is more sensible semantically for
representing tree structures.
2004-03-18 14:58:16 +00:00
Eelco Dolstra
a784fd5792 * Don't use tables. Konqueror likes this much better. 2004-03-18 13:04:05 +00:00
Eelco Dolstra
8ce3dd4887 * Display the popup directly over the abbreviation. 2004-03-17 16:55:53 +00:00
Eelco Dolstra
8330c8202a * A simpler way of implementing the store reference popups, thanks to
Martin and CSS guru Martijn Vermaat.
2004-03-17 16:52:48 +00:00
Eelco Dolstra
b5539e7a30 * Store paths are now abbreviated in the generated HTML file.
Hovering over the abbreviated path will reveal the full path.  This
  probably only works in Mozilla.
2004-03-16 12:47:09 +00:00
Eelco Dolstra
9d2669d218 * Added a utility that can be used to produce nice HTML pages from Nix
build logs.  The program `log2xml' converts a Nix build log (read
  from standard input) into XML file that can then be converted to
  XHTML by the `log2html.xsl' stylesheet.  The CSS stylesheet
  `logfile.css' is necessary to make it look good.

  This is primarily useful if the log file has a *tree structure*,
  i.e., that sub-tasks such as the various phases of a build (unpack,
  configure, make, etc.) or recursive invocations of Make are
  represented as such.  While a log file is in principle an
  unstructured plain text file, builders can communicate this tree
  structure to `log2xml' by using escape sequences:

  - "\e[p" starts a new nesting level; the first line following the
    escape code is the header;

  - "\e[q" ends the current nesting level.

  The generic builder in nixpkgs (not yet committed) uses this.  It
  shouldn't be to hard to patch GNU Make to speak this protocol.

  Further improvements to the generated HTML pages are to allow
  collapsing/expanding of subtrees, and to abbreviate store paths (but
  to show the full path by hovering the mouse over it).
2004-03-15 21:51:14 +00:00
Eelco Dolstra
a5619f1dff * Set the NIX_STORE and NIX_BUILD_TOP environment variables in
builders to point to the store and the temporary build directory,
  respectively.  Useful for purity checking.
* Also set TEMPDIR, TMPDIR, TEMP, and TEMP to NIX_BUILD_TOP to make
  sure that tools in the builder store temporary files in the right
  location.
2004-03-12 10:45:08 +00:00
Eelco Dolstra
dbf547645d * Resolve an ambiguity between ifs and attribute selection, e.g., `if
b then x else y.z'.
2004-02-19 13:11:12 +00:00
Eelco Dolstra
0dfdafdf6d * Allow linking against an external Berkeley DB / ATerm library. 2004-02-16 16:37:16 +00:00
Eelco Dolstra
f34de12140 * Allow the location of the store to be specified (--with-store-dir).
* Do not create stuff in localstatedir when doing `make install'
  (since we may not have write access).  In general, installation of
  constant code/data should be separate from the initialisation of
  mutable state.
2004-02-16 15:23:19 +00:00
Eelco Dolstra
fbc48a469c * Inherited attributes in recursive attribute sets are in scope of the
non-inherited attributes.
2004-02-16 09:18:35 +00:00
Eelco Dolstra
76c0e85929 * The environment variable NIX_ROOT can now be set to execute Nix in a
chroot() environment.
* A operation `--validpath' to register path validity.  Useful for
  bootstrapping in a pure Nix environment.
* Safety checks: ensure that files involved in store operations are in
  the store.
2004-02-14 21:44:18 +00:00
Eelco Dolstra
6f5a5ea5ea * Regression fix: realise substitutes and detect cycles. 2004-02-13 10:45:09 +00:00
Eelco Dolstra
00fe1a506f * When creating a new generation, also make the normal form of the
derivation (i.e., the closure store expression) a root of the
  garbage collector.  This ensures that running `nix-collect-garbage
  --no-successors' is safe.
2004-02-11 10:25:31 +00:00
Eelco Dolstra
92e832348d * Lots of manual stuff. Reference pages for most Nix commands.
* nix-pull now requires the full url to the manifest, i.e.,
  `/MANIFEST/' is no longer automatically appended.
* nix-prefetch-url works again.
2004-02-10 16:14:47 +00:00
Eelco Dolstra
6551b36790 * Print what generation we are switching to; honour --dry-run flag. 2004-02-10 13:42:58 +00:00
Eelco Dolstra
618aa69b01 * In --upgrade': added flags --lt', --leq', --always' to specify
whether we want to upgrade if the current version is less than the
  available version (default), when it is less or equal, or always.
  
* Added a flag `--dry-run' to show what would happen in `--install',
  `--uninstall', and `--upgrade', without actually performing the
  operation.
2004-02-09 11:59:39 +00:00
Eelco Dolstra
06a75a7e0c * A command `--switch-generation' to switch to a specific generation
of the current profile, e.g.,

    $ nix-env --list-generations
    ...
    39   2004-02-02 17:53:53
    40   2004-02-02 17:55:18
    41   2004-02-02 17:55:41
    42   2004-02-02 17:55:50   (current)

    $ nix-env --switch-generation 39

    $ ls -l /nix/var/nix/profiles/default
    ... default -> default-39-link

* Also a command `--rollback' which is just a convenience operation to
  rollback to the oldest generation younger than the current one.

  Note that generations properly form a tree.  E.g., if after
  switching to generation 39, we perform an installation action,
  a generation 43 is created which is a descendant of 39, not 42.  So
  a rollback from 43 ought to go back to 39.  This is not currently
  implemented; generations form a linear sequence.
2004-02-08 14:07:43 +00:00
Eelco Dolstra
b8675aee54 * In `--list-generations', show what the current generation is. 2004-02-06 16:16:55 +00:00
Eelco Dolstra
73ab2ed4fd * A command `--list-generations' to show all generations for a
profile.
2004-02-06 16:03:27 +00:00
Eelco Dolstra
7c0fa4474f * More refactoring. 2004-02-06 14:57:10 +00:00
Eelco Dolstra
7abf9911d9 * Refactoring. 2004-02-06 14:49:41 +00:00
Eelco Dolstra
49bafe1faf * Use the profile pointed to by ~/.nix-profile if no --profile
argument is specified.
2004-02-06 10:59:06 +00:00
Eelco Dolstra
66e94d3275 * Improvements to profiles. Generations are now per-profile, e.g.,
default -> default-94-link
  default-82-link -> /nix/store/cc4480...
  default-83-link -> /nix/store/caeec8...
  ...
  default-94-link -> /nix/store/2896ca...
  experimental -> experimental-2-link
  experimental-1-link -> /nix/store/cc4480...
  experimental-2-link -> /nix/store/a3148f...

* `--profile' / `-p' -> `--switch-profile' / `-S'
* `--link' / `-l' -> `--profile' / `-p'
* The default profile is stored in $prefix/var/nix/profiles.
  $prefix/var/nix/links is gone.  Profiles can be stored anywhere.
* The current profile is now referenced from ~/.nix-profile, not
  ~/.nix-userenv.
* The roots to the garbage collector now have extension `.gcroot', not
  `.id'.
2004-02-06 10:30:20 +00:00
Eelco Dolstra
d445da7a7b * Extended the `inherit' syntax to optionally select attributes from
other attribute sets, rather than the current scope.  E.g.,
  
    {inherit (pkgs) gcc binutils;}

  is equivalent to

    {gcc = pkgs.gcc; binutils = pkgs.binutils;}

  I am not so happy about the syntax.
2004-02-04 17:23:26 +00:00
Eelco Dolstra
9d25466b34 * An attribute set update operator (//). E.g.,
{x=1; y=2; z=3;} // {y=4;}  =>  {x=1; y=4; z=3;}
2004-02-04 16:49:51 +00:00
Eelco Dolstra
9b44480612 * Use a map to lookup primops.
* Various performance improvements in the evaluator.
* Do not link against unused (and missing!) libraries (-lsglr, etc.).
2004-02-04 16:03:29 +00:00
Eelco Dolstra
c4f7ae4aa5 * Verify that all variables in a Nix expression are defined. 2004-02-03 14:45:34 +00:00
Eelco Dolstra
1c9c0a5a46 * Added syntactic sugar to the construction of attribute sets to
`inherit' variables from the surrounding lexical scope.

  E.g.,

    {stdenv, libfoo}: derivation {
      builder = ./bla;
      inherit stdenv libfoo;
      xyzzy = 1;
    }

  is equivalent to

    {stdenv, libfoo}: derivation {
      builder = ./bla;
      stdenv = stdenv;
      libfoo = libfoo;
      xyzzy = 1;
    }

  Note that for mutually recursive attribute set definitions (`rec
  {...}'), this also works, that is, `rec {inherit x;}' is equivalent
  to `let {fresh = x; body = rec {x = fresh;};}', *not*
  `rec {x = x}'.
2004-02-02 21:39:33 +00:00
Eelco Dolstra
d9f30fe7c7 * Sort `nix-env -q' output by derivation name.
* `--version' flag for all commands.
* Manual updates.
2004-02-02 10:51:54 +00:00
Eelco Dolstra
47c003cb59 * Doh! 2004-01-30 17:14:08 +00:00
Eelco Dolstra
619f20775d * Parser numbers again.
* Include missing files in distributions.
2004-01-30 17:06:03 +00:00
Eelco Dolstra
c625718513 * Detect flex and bison; updated the manual. 2004-01-30 16:32:14 +00:00
Eelco Dolstra
c5baaafae6 * Replaced the SDF parser by a substantially faster Bison/Flex
parser (roughly 80x faster).

  The absolutely latest version of Bison (1.875c) is required for
  reentrant GLR support, as well as a recent version of Flex (say,
  2.5.31).  Note that most Unix distributions ship with the
  prehistoric Flex 2.5.4, which doesn't support reentrancy.
2004-01-30 15:21:42 +00:00
Eelco Dolstra
abd1878b26 * Optimised the SDF grammar. 2004-01-29 14:24:53 +00:00
Eelco Dolstra
840551ebdb * Extra bit S' in --query --status' output: show whether there are
any substitutes for the derivation.
2004-01-21 16:41:17 +00:00
Eelco Dolstra
1109ea0680 * Fixed a subtle uninitialised variable bug in ATermMaps copied from
ATermMaps.  Found thanks to Valgrind!
2004-01-21 14:49:32 +00:00