Commit graph

353 commits

Author SHA1 Message Date
Eelco Dolstra
1042c10fd0 Add NAR / Store accessor abstraction
This is primary to allow hydra-queue-runner to extract files like
"nix-support/hydra-build-products" from NARs in binary caches.
2016-02-25 17:43:19 +01:00
Eelco Dolstra
152b1d6bf9 deletePath(): Succeed if path doesn't exist
Also makes it robust against concurrent deletions.
2016-02-24 17:44:55 +01:00
Eelco Dolstra
9ccbd55c5b BinaryCacheStore: Implement addToStore()
So now you can do

  $ NIX_REMOTE=file:///tmp/binary-cache nix-instantiate '<nixpkgs>' -A hello

and lots of other operations.
2016-02-24 16:52:28 +01:00
Eelco Dolstra
45c83e5f9b Typo 2016-02-24 14:49:28 +01:00
Eelco Dolstra
263187a2ec Move BinaryCacheStore / LocalBinaryCacheStore from Hydra
So you can now do:

  $ NIX_REMOTE=file:///tmp/binary-cache nix-store -qR /nix/store/...
2016-02-24 14:48:16 +01:00
Eelco Dolstra
bf2adf72c4 std::condition_variable_any -> std::condition_variable
The latter is supposed to be more efficient.
2016-02-24 13:31:46 +01:00
Eelco Dolstra
ccdbf589a4 C++ templates are just a glorified macro facility 2016-02-24 13:07:32 +01:00
Eelco Dolstra
5f862658c3 Remove bad daemon connections from the pool
This is necessary for long-running processes like hydra-queue-runner:
if a nix-daemon worker is killed, we need to stop reusing that
connection.
2016-02-24 11:39:56 +01:00
Eelco Dolstra
d5626bf4c1 Pool<T>: Allow a maximum pool size 2016-02-23 16:40:16 +01:00
Eelco Dolstra
e292144d46 RemoteStore: Make thread-safe
This allows a RemoteStore object to be used safely from multiple
threads concurrently. It will make multiple daemon connections if
necessary.

Note: pool.hh and sync.hh have been copied from the Hydra source tree.
2016-02-23 15:00:59 +01:00
Eelco Dolstra
c0b7a8a0b5 Move ref<t> into a separate header 2016-02-23 13:53:31 +01:00
Eelco Dolstra
eff5021eaa Add xz compression function
This is used by the Hydra queue runner, but since it may also be
useful for the C++ rewrite of nix-push, I'm putting it here.
2016-02-15 21:45:56 +01:00
Eelco Dolstra
bfdacb712c decompressXZ: Ensure that lzma_end() is called
Otherwise we might leak memory.
2016-02-15 21:26:28 +01:00
Eelco Dolstra
b3e8d72770 Merge pull request #762 from ctheune/ctheune-floats
Implement floats
2016-02-12 12:49:59 +01:00
Eelco Dolstra
fd205fb6f8 ref<T>: Add cast operator 2016-02-11 15:32:48 +01:00
Eelco Dolstra
c10c61449f Eliminate the "store" global variable
Also, move a few free-standing functions into StoreAPI and Derivation.

Also, introduce a non-nullable smart pointer, ref<T>, which is just a
wrapper around std::shared_ptr ensuring that the pointer is never
null. (For reference-counted values, this is better than passing a
"T&", because the latter doesn't maintain the refcount. Usually, the
caller will have a shared_ptr keeping the value alive, but that's not
always the case, e.g., when passing a reference to a std::thread via
std::bind.)
2016-02-04 14:28:26 +01:00
Eelco Dolstra
7899fc959d Define HashType's size to allow it to be forward-declared 2016-01-27 17:46:01 +01:00
Eelco Dolstra
739bab0be7 Trivia 2016-01-27 17:18:31 +01:00
Eelco Dolstra
d45ad8fcf5 Make hashLength32() a method of Hash 2016-01-27 17:18:20 +01:00
Eelco Dolstra
5b8c09c124 string2Int: Barf on negative numbers for unsigned types 2016-01-20 16:45:05 +01:00
Eelco Dolstra
9a4706eb19 Fix compilation 2016-01-05 14:05:11 +01:00
Eelco Dolstra
6298afc047 Merge pull request #685 from vizanto/master
POSIX compliant directory access (fixes build on Solaris)
2016-01-05 13:49:55 +01:00
FrankHB
607d91517e Fixed unresolved 'BUFSIZ'
`BUFSIZ` is defined in header `<cstdio>`.
2016-01-05 13:33:40 +01:00
Christian Theune
14ebde5289 First hit at providing support for floats in the language. 2016-01-05 00:40:40 +01:00
Eelco Dolstra
8f67325a7c Build sandbox support etc. unconditionally on Linux
Also, use "#if __APPLE__" instead of "#if SANDBOX_ENABLED" to prevent
ambiguity.
2015-12-10 11:47:17 +01:00
Danny Wilson
69b9d8fdbd Use DT_UNKNOWN when dirent d_type extension is not supported.
edolstra:
“…since callers of readDirectory have to handle the possibility of
 DT_UNKNOWN anyway, and we don't want to do a stat call for every
 directory entry unless it's really needed.”
2015-11-25 21:57:19 +01:00
Shea Levy
1d3529e93a Default arguments belong at declaration, not definition 2015-11-16 05:55:55 -05:00
Shea Levy
9b4cd20752 Fix copy-paste error 2015-11-16 05:54:34 -05:00
Shea Levy
58d2fac91d AutoDelete: Add default constructor with deletion disabled 2015-11-16 05:53:10 -05:00
Eelco Dolstra
7759a56bed Fix namespace issue 2015-11-09 21:28:25 +01:00
Danny Wilson
cdb346c65e Fix build on Solaris
d_type is not part of the POSIX spec unfortunately.
2015-11-07 04:51:33 +01:00
Eelco Dolstra
6f1743b1a5 Support SHA-512 hashes
Fixes #679.

Note: on x86_64, SHA-512 is considerably faster than SHA-256 (198 MB/s
versus 131 MB/s).
2015-11-04 16:37:49 +01:00
Eelco Dolstra
a6ca68a70c Require OpenSSL 2015-11-04 16:37:49 +01:00
Eelco Dolstra
1f735a3440 <nix/fetchurl.nix>: Support xz-compressed NARs 2015-10-30 12:34:30 +01:00
Eelco Dolstra
5c28943e8f int2String() -> std::to_string() 2015-10-29 13:26:55 +01:00
Jude Taylor
af1218a86a revert libutil change 2015-10-21 14:40:35 -07:00
Jude Taylor
e770f941d6 make sandbox builds more permissive 2015-10-21 12:38:52 -07:00
Eelco Dolstra
bec3c31608 nix-prefetch-url: Rewrite in C++ 2015-10-01 16:47:43 +02:00
Eelco Dolstra
4ba6bc184c Shut up clang warnings 2015-09-18 01:22:35 +02:00
Eelco Dolstra
71a5161365 Implement buildDerivation() via the daemon 2015-09-03 12:56:59 +02:00
Kirill Elagin
3b0f60e5c2 baseNameOf: Enhance basename compatibility
* If the path ends with a slash, drop it.
* If the remaining path doesn’t contain slashes, just return it.

Fixes #574.
2015-08-07 03:35:02 +02:00
Eelco Dolstra
db55940d9e Support systemd log severity prefixes
This is mostly useful for hydra-queue-runner.
2015-07-20 01:39:48 +02:00
Eelco Dolstra
b3491c781c More cleanup 2015-07-20 01:16:16 +02:00
Eelco Dolstra
6bd2c7bb38 OCD: foreach -> C++11 ranged for 2015-07-17 20:13:56 +02:00
Eelco Dolstra
1511aa9f48 Allow remote builds without sending the derivation closure
Previously, to build a derivation remotely, we had to copy the entire
closure of the .drv file to the remote machine, even though we only
need the top-level derivation. This is very wasteful: the closure can
contain thousands of store paths, and in some Hydra use cases, include
source paths that are very large (e.g. Git/Mercurial checkouts).

So now there is a new operation, StoreAPI::buildDerivation(), that
performs a build from an in-memory representation of a derivation
(BasicDerivation) rather than from a on-disk .drv file. The only files
that need to be in the Nix store are the sources of the derivation
(drv.inputSrcs), and the needed output paths of the dependencies (as
described by drv.inputDrvs). "nix-store --serve" exposes this
interface.

Note that this is a privileged operation, because you can construct a
derivation that builds any store path whatsoever. Fixing this will
require changing the hashing scheme (i.e., the output paths should be
computed from the other fields in BasicDerivation, allowing them to be
verified without access to other derivations). However, this would be
quite nice because it would allow .drv-free building (e.g. "nix-env
-i" wouldn't have to write any .drv files to disk).

Fixes #173.
2015-07-17 17:57:40 +02:00
Eelco Dolstra
65f17cd330 Support URLs in $NIX_PATH
This didn't work (despite claims in the manual), because the colon in
"http://" was parsed as a element separator. So handle "://"
specially.
2015-06-17 16:20:11 +02:00
Eelco Dolstra
7c4501886d Use std::vector::data() 2015-06-09 10:54:46 +02:00
Rok Garbas
dad754843a cygwin: looks like stdout/stdin are reserved words 2015-05-13 09:37:56 +02:00
Eelco Dolstra
60340ce3e2 Implement caching of fetchurl/fetchTarball results
ETags are used to prevent redownloading unchanged files.
2015-04-09 11:42:04 +02:00
Eelco Dolstra
a596c525ad Add base64 encoder/decoder 2015-02-10 11:33:33 +01:00