Commit graph

566 commits

Author SHA1 Message Date
Eelco Dolstra
22144afa8d Don't keep "disabled" substituters running
For instance, it's pointless to keep copy-from-other-stores running if
there are no other stores, or download-using-manifests if there are no
manifests.  This also speeds things up because we don't send queries
to those substituters.
2013-06-20 11:55:15 +02:00
Eelco Dolstra
5c06e5297d download-from-binary-cache.pl: Respect $NIX_CONNECT_TIMEOUT 2013-06-07 16:25:12 +02:00
Eelco Dolstra
24a356bf71 Replace $NIX_DEBUG_SUBST with an option ‘debug-subst’
Thus passing ‘--option debug-subst 1’ allows daemon users to turn on
debug info and see what the substituter is doing.
2013-06-07 15:36:36 +02:00
Eelco Dolstra
24e063efdc download-from-binary-cache.pl: Show if we're waiting for a URL
Previously, if a binary cache is hanging/unreachable/slow,
download-from-binary-cache.pl would also hang without any indication
to the user.  Now, if fetching a URL takes more than 5 seconds, it
will print a message to that effect.
2013-06-07 15:33:44 +02:00
Eelco Dolstra
75e12b8e66 download-from-binary-cache.pl: Fix race condition
Fixes the error "DBD::SQLite::db do failed: column url is not unique".
2013-06-05 16:25:07 +02:00
Eelco Dolstra
f0576d6775 Update the default binary cache URL to cache.nixos.org 2013-06-05 13:36:43 +02:00
Eelco Dolstra
ff08306746 download-from-binary-cache.pl: Treat a 403 error as a 404
Amazon S3 returns HTTP status code 403 if a file doesn't exist and the
user has no permission to list the contents of the bucket.  So treat
it as 404 (meaning it's cached in the NARExistence table).
2013-06-04 15:20:37 +02:00
Eelco Dolstra
bc2e43f3c8 build-remote.pl: Remove meaningless signing when importing the output paths
The "$UID != 0" makes no sense: if the local side has write access to
the Nix store (which is always the case) then it doesn't matter if
we're root - we can import unsigned paths either way.
2013-05-29 18:18:59 +02:00
Eelco Dolstra
107505e13a build-remote.pl: Copy all outputs in one operation 2013-05-23 14:39:58 -04:00
Eelco Dolstra
72d8209548 build-remote.pl: Indicate if remote machine is refusing builds
Fixes #120.
2013-05-23 14:04:46 -04:00
Eelco Dolstra
a4cb62ac25 download-from-binary-cache.pl: Get rid of an uninitialized value warning
Reported by Pablo Costa.
2013-05-15 15:47:05 +02:00
Eelco Dolstra
3a0cc43ac8 build-remote.pl: Properly close the SSH connection between attempts 2013-05-10 02:38:50 +02:00
Eelco Dolstra
be0b9dda31 build-remote.pl: Pass /dev/null as SSH's stdin
Otherwise it will set the parent's stdin to non-blocking mode, causing
the subsequent read of the set of inputs/outputs to fail randomly.
That's insane.
2013-05-10 02:32:13 +02:00
Eelco Dolstra
78206f06ec build-remote.pl: Allow a machine to refuse a build
Before selecting a machine, build-remote.pl will try to run the
command "nix-builds-inhibited" on the machine.  If this command exists
and returns a 0 exit code, then the machine won't be used.  It's up to
the user to provide this command, but it would typically be a script
that checks whether there is enough disk space and whether the load is
not too high.
2013-05-10 01:09:46 +02:00
Eelco Dolstra
7a03cbf09d build-remote.pl: Create one process fewer on the remote side 2013-05-09 17:30:07 +02:00
Eelco Dolstra
69b8f9980f build-remote.pl: Enforce timeouts locally
Don't pass --timeout / --max-silent-time to the remote builder.
Instead, let the local Nix process terminate the build if it exceeds a
timeout.  The remote builder will be killed as a side-effect.  This
gives better error reporting (since the timeout message from the
remote side wasn't properly propagated) and handles non-Nix problems
like SSH hangs.
2013-05-09 17:17:17 +02:00
Eelco Dolstra
ea019e9a26 Add option ‘extra-binary-caches’
This allows providing additional binary caches, useful in scripts like
Hydra's build reproduction scripts, in particular because untrusted
caches are ignored.
2013-05-07 15:37:28 +02:00
Eelco Dolstra
c51b6a893c nix-copy-closure: Show a proper error message if no host name is given 2013-05-03 11:12:11 +02:00
Lluís Batlle i Rossell
e6c44d166a Fixing the pv position regarding compression
Problem noticed by niksnut.
2013-05-03 11:08:51 +02:00
Lluís Batlle i Rossell
7391533ea5 Fixing the pv reference; I didn't mean to change it 2013-05-03 11:08:51 +02:00
Lluís Batlle i Rossell
5cc2fc46ec Adding ETA support to the --show-progress in nix-copy-closure
Based on https://github.com/NixOS/nix/pull/6 from shlevy
2013-05-03 11:08:51 +02:00
Eelco Dolstra
4ddd077bfa find-runtime-roots.pl: Don't hardcode /nix/store 2013-04-26 12:15:54 +02:00
Eelco Dolstra
00f698eb8b find-runtime-roots.pl: Search process environments for roots
For instance, this prevents paths from being deleted that are in use
by a "nix-build --run-env" session.
2013-04-26 12:06:39 +02:00
Eelco Dolstra
938092a213 find-runtime-roots.pl: Use Nix::Utils::readFile 2013-04-26 11:44:19 +02:00
Eelco Dolstra
6955d41f2b nix-build: Respect --timeout 2013-04-23 17:16:01 +02:00
Eelco Dolstra
08d96ffad0 Fix --fallback with the binary cache substituter
Reported by Peter Simons.
2013-04-23 12:45:01 +02:00
Eelco Dolstra
c642441beb Test NAR info caching 2013-04-23 12:43:28 +02:00
Eelco Dolstra
bdd4646338 Revert "Prevent config.h from being clobbered"
This reverts commit 28bba8c44f.
2013-03-08 01:24:59 +01:00
Eelco Dolstra
28bba8c44f Prevent config.h from being clobbered 2013-03-07 23:55:55 +01:00
Eelco Dolstra
dadf7a5b46 build-remote: Use the --quiet flag
‘--option verbosity 0’ doesn't actually do anything.
2013-02-19 16:56:31 +01:00
Eelco Dolstra
96fbbbde55 build-remote.pl: Don't keep build logs on the build slave 2013-01-21 22:26:25 +01:00
Eelco Dolstra
0a4e90395c Urgggh
http://hydra.nixos.org/build/3661100
2013-01-02 23:52:15 +01:00
Eelco Dolstra
aebea2e489 Reinstate the http://nixos.org/binary-cache default for the binary-caches setting 2013-01-02 22:16:37 +01:00
Eelco Dolstra
68dcbb187e download-from-binary-cache: Use the channels of the calling user rather than root
This should make live easier for single-user (non-daemon)
installations.  Note that when the daemon is used, the "calling user"
is root so we're not using any untrusted caches.
2012-12-21 15:00:07 +01:00
Eelco Dolstra
8cc19ed089 Set a long SQLite timeout in the binary cache substituter 2012-12-06 11:43:34 +01:00
Eelco Dolstra
566afa1e9c Support xz compression of nixexprs.tar in channels 2012-12-05 11:02:44 +01:00
Eelco Dolstra
08964d7328 Undo accidental debug change 2012-11-26 17:57:14 +01:00
Eelco Dolstra
46a369ad95 Make "nix-build -A <derivation>.<output>" do the right thing
For example, given a derivation with outputs "out", "man" and "bin":

  $ nix-build -A pkg

produces ./result pointing to the "out" output;

  $ nix-build -A pkg.man

produces ./result-man pointing to the "man" output;

  $ nix-build -A pkg.all

produces ./result, ./result-man and ./result-bin;

  $ nix-build -A pkg.all -A pkg2

produces ./result, ./result-man, ./result-bin and ./result-2.
2012-11-26 15:39:10 +01:00
Eelco Dolstra
a3d6585c5a nix-copy-closure: Add flag ‘--use-substitutes’ 2012-11-23 16:20:16 +01:00
Eelco Dolstra
a9a8baaccb Use a shorter Nixpkgs channel URL 2012-11-09 13:33:35 +01:00
Eelco Dolstra
e28b683324 download-from-binary-cache: Try next cache if downloading a NAR fails 2012-11-06 17:55:53 +01:00
Eelco Dolstra
bbc107ef1e Process binary caches in order of priority
Binary caches can now specify a priority in their nix-cache-info file.
The binary cache substituter checks caches in order of priority.  This
is to ensure that fast, static caches like nixos.org/binary-cache are
processed before slow, dynamic caches like hydra.nixos.org.
2012-11-06 17:45:20 +01:00
Eelco Dolstra
82951e5582 nix-push: Handle pushing a symlink 2012-11-06 13:53:53 +01:00
Eelco Dolstra
620e92e880 Add an option ‘use-binary-caches’
This allows disabling the use of binary caches, e.g.

  $ nix-build ... --option use-binary-caches false

Note that

  $ nix-build ... --option binary-caches ''

does not disable all binary caches, since the caches defined by
channels will still be used.
2012-11-06 13:44:59 +01:00
Eelco Dolstra
df27db712d Fix "Not an ARRAY reference" error
It's a mystery why this error is not triggered in the build farm
(e.g. http://hydra.nixos.org/build/3265602).  Ah well.
2012-11-06 13:43:46 +01:00
Eelco Dolstra
a28b4445a4 nix-prefetch-url: Improve option handling 2012-10-23 18:04:11 +02:00
Eelco Dolstra
dde6486eab nix-push: Add a flag ‘--manifest-path’ to write the manifest to another directory 2012-10-18 10:36:16 -04:00
Eelco Dolstra
c8daeba303 nix-push: Add ‘--link’ flag
If ‘--link’ is given, nix-push will create hard links to the NAR files
in the store, rather than copying them.  This is faster and requires
less disk space.  However, it doesn't work if the store is on a
different file system.
2012-10-17 17:01:23 -04:00
Eelco Dolstra
167e36a5c3 nix-push: Only generate and copy a NAR if it doesn't already exist
This prevents unnecessary and slow rebuilds of NARs that already exist
in the binary cache.
2012-10-17 16:58:05 -04:00
Eelco Dolstra
600daf972f download-from-binary-cache: Remove duplicate entries in trustedURLs 2012-10-11 14:21:22 -04:00