Commit graph

396 commits

Author SHA1 Message Date
Eelco Dolstra
d86bd22d24 * Define _GNU_SOURCE. Hopefully this fixes the build on Debian 4.0
(http://hydra.nixos.org/nixlog/384/1).
2008-12-12 10:20:19 +00:00
Eelco Dolstra
ac5478eb52 * Don't provide the whole Nix store in the chroot, but only the
closure of the inputs.  This really enforces that there can't be any
  undeclared dependencies on paths in the store.  This is done by
  creating a fake Nix store and creating bind-mounts or hard-links in
  the fake store for all paths in the closure.  After the build, the
  build output is moved from the fake store to the real store.  TODO:
  the chroot has to be on the same filesystem as the Nix store for
  this to work, but this isn't enforced yet.  (I.e. it only works
  currently if /tmp is on the same FS as /nix/store.)
2008-12-11 18:57:10 +00:00
Eelco Dolstra
652817046b * Revert r13150: now that we use private namespaces for the chroot, we
don't have to put the chroot in /nix/var/nix/chroots anymore.
  They're back in /tmp now.
2008-12-11 17:52:34 +00:00
Eelco Dolstra
5a569509b4 * Provide a minimal /etc/passwd in the chroot to keep some builders
happy.
2008-12-11 17:44:02 +00:00
Eelco Dolstra
7c54f1603f * Do chroot builds in a private namespace. This means that all the
bind-mounts we do are only visible to the builder process and its
  children.  So accidentally doing "rm -rf" on the chroot directory
  won't wipe out /nix/store and other bind-mounted directories
  anymore.  Also, the bind-mounts in the private namespace disappear
  automatically when the builder exits.
2008-12-11 17:00:12 +00:00
Eelco Dolstra
07cdfb09fb * Open the connection to the daemon lazily (on demand) so that
read-only operations (like nix-env -qa) work properly when the
  daemon isn't running.
2008-12-11 14:30:25 +00:00
Eelco Dolstra
a0766eca27 * Build on Debian 4.0 (GCC 4.1.1). 2008-12-04 21:07:29 +00:00
Eelco Dolstra
9ac3f5df9c * Propagate --max-silent-time to remote machines. 2008-12-04 16:51:16 +00:00
Eelco Dolstra
909fbb9de1 * When using build hooks, for any nix-store -r build operation, it is
necessary that at least one build hook doesn't return "postpone",
  otherwise nix-store will barf ("waiting for a build slot, yet there
  are no running children").  So inform the build hook when this is
  the case, so that it can start a build even when that would exceed
  the maximum load on a machine.
2008-12-04 14:29:41 +00:00
Eelco Dolstra
82ae85de27 * addToStore() in nix-worker: don't write the NAR dump received from
the client to a temporary directory, as that is highly inefficient.
2008-12-03 18:05:14 +00:00
Eelco Dolstra
ff762fb499 * Pass HashType values instead of strings. 2008-12-03 16:10:17 +00:00
Eelco Dolstra
1307b22223 * Made addToStore() a lot more efficient: it no longer reads the path
being copied 3 times in the worst case.  It doesn't run in constant space,
  but it didn't do that anyway.
2008-12-03 15:51:17 +00:00
Eelco Dolstra
64519cfd65 * Unify the treatment of sources copied to the store, and recursive
SHA-256 outputs of fixed-output derivations.  I.e. they now produce
  the same store path:

  $ nix-store --add x
  /nix/store/j2fq9qxvvxgqymvpszhs773ncci45xsj-x

  $ nix-store --add-fixed --recursive sha256 x
  /nix/store/j2fq9qxvvxgqymvpszhs773ncci45xsj-x

  the latter being the same as the path that a derivation

    derivation {
      name = "x";
      outputHashAlgo = "sha256";
      outputHashMode = "recursive";
      outputHash = "...";
      ...
    };

  produces.

  This does change the output path for such fixed-output derivations.
  Fortunately they are quite rare.  The most common use is fetchsvn
  calls with SHA-256 hashes.  (There are a handful of those is
  Nixpkgs, mostly unstable development packages.)
  
* Documented the computation of store paths (in store-api.cc).
2008-12-03 15:06:30 +00:00
Eelco Dolstra
a55113411f * Nix daemon: reload the configuration file after forking (NIX-100). 2008-11-20 12:25:11 +00:00
Eelco Dolstra
e13da525a7 * Files in the info directory starting with "." are temporary files
and don't indicate path validity.
2008-11-19 16:27:07 +00:00
Eelco Dolstra
6fedb7aa0f * Restore SIGPIPE to SIG_DFL when running the builder. This prevents
subtle and often hard-to-reproduce bugs where programs in pipes
  either barf with a "Broken pipe" message or not, depending on the
  exact timing conditions.  This particularly happened in GNU M4 (and
  Bison, which uses M4).
2008-11-14 15:46:45 +00:00
Eelco Dolstra
a519bb0635 * Some somewhat ad hoc mechanism to allow the build farm to monitor
build progress.
2008-11-12 11:08:27 +00:00
Eelco Dolstra
96598e7b06 * Pass the --no-build-output flag to the daemon. 2008-11-11 15:11:10 +00:00
Eelco Dolstra
4166b11a53 * Add /dev/pts to the default nix.conf. 2008-11-11 14:59:20 +00:00
Eelco Dolstra
709b55ee02 * Put the chroots under /nix/var/nix/chroots to reduce the risk of
disasters involving `rm -rf' on bind mounts.  Will try the
  definitive fix (per-process mounts, apparently possible via the
  CLONE_NEWNS flag in clone()) some other time.
2008-10-29 15:34:48 +00:00
Ludovic Courtès
c98ea254dc libstore: Always mount `/dev/pts' individually.
This fixes problems such as Tcl's PTY handling:

  ERROR: The system has no more ptys.  Ask your system administrator to
  create more.
2008-10-16 21:04:32 +00:00
Eelco Dolstra
9d6d50269b * Bug fix for building on some old installations (contributed by Pjotr). 2008-10-16 14:16:03 +00:00
Eelco Dolstra
f32fef1b07 * GC option `--max-atime' that specifies an upper limit to the last
accessed time of paths that may be deleted.  Anything more recently
  used won't be deleted.  The time is specified in time_t,
  e.g. seconds since 1970-01-01 00:00:00 UTC; use `date +%s' to
  convert to time_t from the command line. 

  Example: to delete everything that hasn't been used in the last two
  months:

  $ nix-store --gc -v --max-atime $(date +%s -d "2 months ago")
2008-09-17 14:52:35 +00:00
Eelco Dolstra
4af2fdba6d * Typo. 2008-09-17 13:00:55 +00:00
Eelco Dolstra
c987061aa4 * Some refactoring. Better output with `-v' for --use-atime. 2008-09-17 12:54:07 +00:00
Eelco Dolstra
7ab68961e4 * Garbage collector: added an option `--use-atime' to delete paths in
order of ascending last access time.  This is useful in conjunction
  with --max-freed or --max-links to prefer deleting non-recently used
  garbage, which is good (especially in the build farm) since garbage
  may become live again.

  The code could easily be modified to accept other criteria for
  ordering garbage by changing the comparison operator used by the
  priority queue in collectGarbage().
2008-09-17 10:02:55 +00:00
Eelco Dolstra
7718b19389 * Explicitly set PWD to prevent problems with chroot builds. In
particular, dietlibc cannot figure out the cwd because the inode of
  the current directory doesn't appear in .. (because getdents returns
  the inode of the mount point).
2008-08-27 17:20:25 +00:00
Eelco Dolstra
9cc0da8453 * Create a /tmp with 1777 permission in the chroot. Some builders
need a writable /tmp (they don't respect $TMPDIR).
2008-08-27 16:03:03 +00:00
Eelco Dolstra
d06be428f6 * Disable chroot builds for fixed-output derivations so that we don't
need /etc in the chroot (in particular, /etc/resolv.conf for
  fetchurl).  Not having /etc/resolv.conf in the chroot is a good
  thing, since we don't want normal derivations to download files.
2008-08-25 15:49:22 +00:00
Eelco Dolstra
b428adc267 * Strip off the `.nix' suffix from the attribute name for files in
~/.nix-defexpr,  otherwise the attribute cannot be selected with the
  `-A' option.  Useful if you want to stick a Nix expression directly
  in ~/.nix-defexpr.
2008-08-25 13:31:57 +00:00
Eelco Dolstra
db4f4a8425 * Backward compatibility check to prevent nixos-rebuild from barfing
when upgrading Nix.
2008-08-14 13:02:19 +00:00
Eelco Dolstra
c03b729319 * Increase the sleep periods a bit to make the test less likely to
fail on slow machines.  Of course it would be better if this test
  wasn't timing dependent...
2008-08-14 09:26:30 +00:00
Eelco Dolstra
98b07466fb * Better error checking of the data from the substituters. 2008-08-05 10:57:53 +00:00
Eelco Dolstra
5adbb0aabe * build.cc: only use a substituter if it returns info for a path. 2008-08-04 13:15:35 +00:00
Eelco Dolstra
c4f98941ed * nix-env --dry-run: show the total size of the substituter
downloads.
2008-08-04 12:29:04 +00:00
Eelco Dolstra
03427e76f1 * querySubstitutablePathInfo: work properly when run via the daemon.
* --dry-run: print the paths that we don't know how to build/substitute.
2008-08-04 11:44:50 +00:00
Eelco Dolstra
3c92ea399d * Make nix-env --dry-run print the paths to be substituted correctly
again.  (After the previous substituter mechanism refactoring I
  didn't update the code that obtains the references of substitutable
  paths.)  This required some refactoring: the substituter programs
  are now kept running and receive/respond to info requests via
  stdin/stdout.
2008-08-02 12:54:35 +00:00
Eelco Dolstra
989176c56e * Allow read-only access to the store (e.g., non-root users on NixOS
can do operations like "nix-store -qR <path>" even without the Nix
  daemon).
2008-07-18 15:34:46 +00:00
Eelco Dolstra
6c8641a542 2008-06-18 19:17:05 +00:00
Eelco Dolstra
d3aa183beb * Garbage collector: option `--max-freed' to stop after at least N
bytes have been freed, `--max-links' to stop when the Nix store
  directory has fewer than N hard links (the latter being important
  for very large Nix stores on filesystems with a 32000 subdirectories
  limit).
2008-06-18 14:20:16 +00:00
Eelco Dolstra
a8f3b02092 * `nix-store --optimise': handle files with >= 32000 hard links.
(There can easily be more than 32000 occurrences of the empty file.)
2008-06-18 14:13:00 +00:00
Eelco Dolstra
a72709afd8 * Some refactoring: put the GC options / results in separate structs.
* The garbage collector now also prints the number of blocks freed.
2008-06-18 09:34:17 +00:00
Eelco Dolstra
94fd46fa1c * Note. 2008-06-14 16:02:31 +00:00
Eelco Dolstra
826b271d9a * Garbage collector: don't do a complete topological sort of the Nix
store under the reference relation, since that means that the
  garbage collector will need a long time to start deleting paths.
  Instead just delete the referrers of a path first.
2008-06-13 18:25:24 +00:00
Eelco Dolstra
30c9f909b2 * Print some progress info during the early GC stages. 2008-06-13 17:21:20 +00:00
Eelco Dolstra
b0e92f6d47 * Merged the no-bdb branch (-r10900:HEAD
https://svn.nixos.org/repos/nix/nix/branches/no-bdb).
2008-06-09 13:52:45 +00:00
Eelco Dolstra
bd955e15e1 * GCC 4.3.0 (Fedora 9) compatibility fixes. Reported by Gour and
Armijn Hemel.
2008-05-21 11:17:31 +00:00
Eelco Dolstra
72034ab35d * sockaddr_un doesn't allow path names of more than 108 characters.
This isn't usually a problem, except that it causes tests to fail
  when performed in a directory with a very long path name.  So chdir
  to the socket directory and use a relative path name.
2008-04-09 05:57:01 +00:00
Eelco Dolstra
329025253d * Use /tmp/nix-build-<drvpath>-<counter> instead of
/tmp/nix-<pid>-<counter> for temporary build directories.  This
  increases purity a bit: many packages store the temporary build path
  in their output, causing (generally unimportant) binary differences.
2008-03-27 13:45:17 +00:00
Eelco Dolstra
f789ea1d09 * A transaction is unnecessary since the path cannot be
garbage-collected (it's a temporary root).
2008-03-01 21:05:33 +00:00