mdebray/tvl-depot
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
4571 commits 5 branches 0 tags 172 MiB
Commit graph

2575 commits

Author SHA1 Message Date
Eelco Dolstra
ce113c32d2 Add warning about "nix" being experimental 2016-03-04 15:54:41 +01:00
Eelco Dolstra
7c9d7a253c Merge branch 'new-cli' 2016-03-04 15:21:23 +01:00
Nikolay Amiantov
f7f0116dd7 Propagate path context via builtins.readFile 2016-03-04 14:04:51 +03:00
Eelco Dolstra
76f1ba4f3b Add file missing from 201b48de60 2016-03-03 18:03:34 +01:00
Nathan Zadoks
fe2be8f016 build.cc: fs.h doesn't appear to be necessary anymore 2016-03-03 14:11:00 +01:00
Eelco Dolstra
5a8455c85e Provide function required by Hydra 2016-03-02 18:21:48 +01:00
Eelco Dolstra
25eea97873 Force stack trace for boost format errors 2016-03-02 15:46:07 +01:00
Eelco Dolstra
201b48de60 Add an HTTP binary cache store 
Allowing stuff like

  NIX_REMOTE=https://cache.nixos.org nix-store -qR /nix/store/x1p1gl3a4kkz5ci0nfbayjqlqmczp1kq-geeqie-1.1

or

  NIX_REMOTE=https://cache.nixos.org nix-store --export /nix/store/x1p1gl3a4kkz5ci0nfbayjqlqmczp1kq-geeqie-1.1 | nix-store --import
 2016-03-02 15:46:07 +01:00
Eelco Dolstra
0402b6398d Eliminate local-binary-cache-store.hh 2016-03-02 15:46:07 +01:00
Eelco Dolstra
68a5414982 Make store implementations pluggable 
This for instance allows hydra-queue-runner to add the S3 backend
at runtime.
 2016-03-02 15:46:07 +01:00
Eelco Dolstra
6055d84beb Fix reading symlinks 
The st_size field of a symlink doesn't have to be correct, e.g. for
/dev/fd symlinks.
 2016-03-02 15:46:07 +01:00
Eelco Dolstra
0a62d9b3d7 Remove bad assertion 2016-03-02 15:46:07 +01:00
Eelco Dolstra
89a2fa68ac FdSource: track number of bytes read 2016-03-02 15:46:07 +01:00
Eelco Dolstra
cca4a8dc1a importPaths(): Optionally add NARs to binary cache accessor 
This enables an optimisation in hydra-queue-runner, preventing a
download of a NAR it just uploaded to the cache when reading files
like hydra-build-products.
 2016-03-02 15:46:07 +01:00
Eelco Dolstra
e9c50064b5 Add an HTTP binary cache store 
Allowing stuff like

  NIX_REMOTE=https://cache.nixos.org nix-store -qR /nix/store/x1p1gl3a4kkz5ci0nfbayjqlqmczp1kq-geeqie-1.1

or

  NIX_REMOTE=https://cache.nixos.org nix-store --export /nix/store/x1p1gl3a4kkz5ci0nfbayjqlqmczp1kq-geeqie-1.1 | nix-store --import
 2016-02-29 18:15:20 +01:00
Eelco Dolstra
6170bb474b Eliminate local-binary-cache-store.hh 2016-02-29 16:14:39 +01:00
Eelco Dolstra
0b907321cc Make store implementations pluggable 
This for instance allows hydra-queue-runner to add the S3 backend
at runtime.
 2016-02-29 16:11:11 +01:00
Eelco Dolstra
012f8d187c Fix reading symlinks 
The st_size field of a symlink doesn't have to be correct, e.g. for
/dev/fd symlinks.
 2016-02-28 03:27:34 +01:00
Eelco Dolstra
d0344dd2c1 Remove bad assertion 2016-02-26 21:43:59 +01:00
Eelco Dolstra
581e1bc5b4 FdSource: track number of bytes read 2016-02-26 16:16:08 +01:00
Eelco Dolstra
bcc9943cee importPaths(): Optionally add NARs to binary cache accessor 
This enables an optimisation in hydra-queue-runner, preventing a
download of a NAR it just uploaded to the cache when reading files
like hydra-build-products.
 2016-02-26 15:20:10 +01:00
Eelco Dolstra
00b2c05749 nix: Add commands to query contents of NARs / binary caches 
For example,

  $ NIX_REMOTE=file:///my-cache nix ls-store -lR /nix/store/f4kbgl8shhyy76rkk3nbxr0lz8d2ip7q-binutils-2.23.1
  dr-xr-xr-x                    0 ./bin
  -r-xr-xr-x                30748 ./bin/addr2line
  -r-xr-xr-x                66973 ./bin/ar
  ...

Similarly, "nix ls-nar" lists the contents of a NAR file, "nix
cat-nar" extracts a file from a NAR file, and "nix cat-store" extract
a file from a Nix store.
 2016-02-25 17:57:00 +01:00
Eelco Dolstra
24a8f9e27b Merge branch 'master' into new-cli 2016-02-25 17:48:35 +01:00
Eelco Dolstra
1042c10fd0 Add NAR / Store accessor abstraction 
This is primary to allow hydra-queue-runner to extract files like
"nix-support/hydra-build-products" from NARs in binary caches.
 2016-02-25 17:43:19 +01:00
Eelco Dolstra
c5bc571861 Fix short boolean flags 2016-02-25 13:31:34 +01:00
Eelco Dolstra
7873cfb18d Fix build 2016-02-25 11:55:05 +01:00
Eelco Dolstra
f1bdeac986 Merge branch 'master' into new-cli 2016-02-25 11:25:11 +01:00
Eelco Dolstra
152b1d6bf9 deletePath(): Succeed if path doesn't exist 
Also makes it robust against concurrent deletions.
 2016-02-24 17:44:55 +01:00
Eelco Dolstra
28e7e29abd Eliminate reserveSpace flag 2016-02-24 17:44:55 +01:00
Eelco Dolstra
5a64e66268 LocalStore::addTextToStore(): Don't read the path we just wrote 
This eliminates some unnecessary (presumably cached) I/O.
 2016-02-24 17:11:31 +01:00
Eelco Dolstra
9ccbd55c5b BinaryCacheStore: Implement addToStore() 
So now you can do

  $ NIX_REMOTE=file:///tmp/binary-cache nix-instantiate '<nixpkgs>' -A hello

and lots of other operations.
 2016-02-24 16:52:28 +01:00
Eelco Dolstra
30e9d01516 openStoreAt(): Initialise the binary cache 2016-02-24 16:52:07 +01:00
Eelco Dolstra
ba0a81d14f Show a proper error message for unimplemented binary cache operations 2016-02-24 14:57:30 +01:00
Eelco Dolstra
45c83e5f9b Typo 2016-02-24 14:49:28 +01:00
Eelco Dolstra
263187a2ec Move BinaryCacheStore / LocalBinaryCacheStore from Hydra 
So you can now do:

  $ NIX_REMOTE=file:///tmp/binary-cache nix-store -qR /nix/store/...
 2016-02-24 14:48:16 +01:00
Eelco Dolstra
b584a0e7de Merge pull request #816 from tsion/incomplete-parse 
Throw a specific error for incomplete parse errors.
 2016-02-24 13:34:56 +01:00
Eelco Dolstra
bf2adf72c4 std::condition_variable_any -> std::condition_variable 
The latter is supposed to be more efficient.
 2016-02-24 13:31:46 +01:00
Eelco Dolstra
ccdbf589a4 C++ templates are just a glorified macro facility 2016-02-24 13:07:32 +01:00
Eelco Dolstra
5f862658c3 Remove bad daemon connections from the pool 
This is necessary for long-running processes like hydra-queue-runner:
if a nix-daemon worker is killed, we need to stop reusing that
connection.
 2016-02-24 11:39:56 +01:00
Scott Olson
6498adb002 Throw a specific error for incomplete parse errors. 
`nix-repl` will use this for deciding whether to keep waiting for input or
error out right away.
 2016-02-24 04:32:21 -06:00
Eelco Dolstra
d5626bf4c1 Pool<T>: Allow a maximum pool size 2016-02-23 16:40:16 +01:00
Eelco Dolstra
e292144d46 RemoteStore: Make thread-safe 
This allows a RemoteStore object to be used safely from multiple
threads concurrently. It will make multiple daemon connections if
necessary.

Note: pool.hh and sync.hh have been copied from the Hydra source tree.
 2016-02-23 15:00:59 +01:00
Vladimír Čunát
03cbb9ad59 nix-env: respect meta.outputsToInstall 
Discussed on https://github.com/NixOS/nixpkgs/pull/12653#discussion_r51601849
 2016-02-23 14:19:14 +01:00
Eelco Dolstra
c0b7a8a0b5 Move ref<t> into a separate header 2016-02-23 13:53:31 +01:00
Eelco Dolstra
8f71bc33d5 Doh 2016-02-22 17:33:27 +01:00
Eelco Dolstra
840056af04 Make OpenSSL usage thread-safe 
OpenSSL can randomly segfault unless we register a callback function
to do locking.

https://www.openssl.org/docs/manmaster/crypto/threads.html
 2016-02-22 15:51:14 +01:00
Eelco Dolstra
d361901bfe curl: Set CURLOPT_NOSIGNAL 
Otherwise using curl is not safe in multi-threaded applications
because it installs a SIGALRM handler.
 2016-02-22 15:51:14 +01:00
Eelco Dolstra
7a173a7be1 JSONObject: Support floats and booleans 2016-02-19 14:24:04 +01:00
Eelco Dolstra
b49d323ce2 Fix build without sodium 
http://hydra.nixos.org/build/32085949
 2016-02-17 12:42:14 +01:00
Eelco Dolstra
c4d22997f3 Add C++ functions for .narinfo processing / signing 
This is currently only used by the Hydra queue runner rework, but like
eff5021eaa it presumably will be useful
for the C++ rewrite of nix-push and
download-from-binary-cache. (@shlevy)
 2016-02-16 16:38:44 +01:00
Eelco Dolstra
5ac27053e9 Rename ValidPathInfo::hash -> narHash for consistency 2016-02-16 11:49:12 +01:00
Tristan Hume
9b05d5848c Fix to-base16 description 2016-02-16 10:15:58 +01:00
Eelco Dolstra
eff5021eaa Add xz compression function 
This is used by the Hydra queue runner, but since it may also be
useful for the C++ rewrite of nix-push, I'm putting it here.
 2016-02-15 21:45:56 +01:00
Eelco Dolstra
bfdacb712c decompressXZ: Ensure that lzma_end() is called 
Otherwise we might leak memory.
 2016-02-15 21:26:28 +01:00
Eelco Dolstra
c8f4d89a34 Expose the export magic value and move LocalStore::queryReferences to Store 2016-02-15 15:01:26 +01:00
Eelco Dolstra
e03d6e0998 Fix broken number parsing in fromJSON 
The call to tmp_number.append had its arguments mixed up. Also, JSON
does not allow a trailing "," after array/object members.
 2016-02-15 15:01:26 +01:00
Eelco Dolstra
d089372565 Add function to extract hash part of a store path 2016-02-15 15:01:26 +01:00
Eelco Dolstra
b3e8d72770 Merge pull request #762 from ctheune/ctheune-floats 
Implement floats
 2016-02-12 12:49:59 +01:00
Eelco Dolstra
ae4a3cfa03 Move addPermRoot into Store 2016-02-11 16:14:42 +01:00
Eelco Dolstra
fd205fb6f8 ref<T>: Add cast operator 2016-02-11 15:32:48 +01:00
Eelco Dolstra
206bbb5dc9 Add basic "nix build" command 
Currently only builds by attribute from <nixpkgs> or the specified
file, e.g. "nix build hello".
 2016-02-09 21:34:24 +01:00
Eelco Dolstra
cd2196b089 Start of new Nix command-line interface 2016-02-09 21:28:29 +01:00
Eelco Dolstra
0db9e6cd1a New command line parsing infrastructure 2016-02-09 21:07:48 +01:00
Eelco Dolstra
c780c1124e More of the same 2016-02-04 15:10:47 +01:00
Eelco Dolstra
fa7cd5369b StoreAPI -> Store 
Calling a class an API is a bit redundant...
 2016-02-04 14:48:42 +01:00
Eelco Dolstra
c10c61449f Eliminate the "store" global variable 
Also, move a few free-standing functions into StoreAPI and Derivation.

Also, introduce a non-nullable smart pointer, ref<T>, which is just a
wrapper around std::shared_ptr ensuring that the pointer is never
null. (For reference-counted values, this is better than passing a
"T&", because the latter doesn't maintain the refcount. Usually, the
caller will have a shared_ptr keeping the value alive, but that's not
always the case, e.g., when passing a reference to a std::thread via
std::bind.)
 2016-02-04 14:28:26 +01:00
Eelco Dolstra
d367b8e787 Add build mode to compute fixed-output derivation hashes 
For example,

  $ nix-build --hash -A nix-repl.src

will build the fixed-output derivation nix-repl.src (a fetchFromGitHub
call), but instead of *verifying* the hash given in the Nix
expression, it prints out the resulting hash, and then moves the
result to its content-addressed location in the Nix store. E.g

  build produced path ‘/nix/store/504a4k6zi69dq0yjc0bm12pa65bccxam-nix-repl-8a2f5f0607540ffe56b56d52db544373e1efb980-src’ with sha256 hash ‘0cjablz01i0g9smnavhf86imwx1f9mnh5flax75i615ml71gsr88’

The goal of this is to make all nix-prefetch-* scripts unnecessary: we
can just let Nix run the real thing (i.e., the corresponding fetch*
derivation).

Another example:

  $ nix-build --hash -E 'with import <nixpkgs> {}; fetchgit { url = "https://github.com/NixOS/nix.git"; sha256 = "ffffffffffffffffffffffffffffffffffffffffffffffffffff"; }'
  ...
  git revision is 9e7c1a4bbd
  ...
  build produced path ‘/nix/store/gmsnh9i7x4mb7pyd2ns7n3c9l90jfsi1-nix’ with sha256 hash ‘1188xb621diw89n25rifqg9lxnzpz7nj5bfh4i1y3dnis0dmc0zp’

(Having to specify a fake sha256 hash is a bit annoying...)
 2016-01-31 12:18:09 +01:00
Eelco Dolstra
9e7c1a4bbd Use the daemon when we don't have write access to the Nix database 2016-01-31 10:28:48 +01:00
Eelco Dolstra
4fa08f3edb Untabify 2016-01-28 16:03:32 +01:00
Eelco Dolstra
26566cd28e Untabify 2016-01-28 16:01:01 +01:00
Eelco Dolstra
9f14d7d33a printMissing(): Propagate store argument 2016-01-28 15:16:06 +01:00
Eelco Dolstra
7899fc959d Define HashType's size to allow it to be forward-declared 2016-01-27 17:46:01 +01:00
Eelco Dolstra
739bab0be7 Trivia 2016-01-27 17:18:31 +01:00
Eelco Dolstra
d45ad8fcf5 Make hashLength32() a method of Hash 2016-01-27 17:18:20 +01:00
Eelco Dolstra
5b8c09c124 string2Int: Barf on negative numbers for unsigned types 2016-01-20 16:45:05 +01:00
Eelco Dolstra
5d8b7eb3e1 Revert "Revert "next try for "don't abort when given unmatched '}' with 'start-condition stack underflow'. This fixes #751""" 
This reverts commit b669d3d2e8.
 2016-01-20 16:34:42 +01:00
Eelco Dolstra
b669d3d2e8 Revert "next try for "don't abort when given unmatched '}' with 'start-condition stack underflow'. This fixes #751"" 
This reverts commit ed23c8568e. Let's
merge this *after* the 1.11.1 release.
 2016-01-20 00:05:28 +01:00
Fabian Schmitthenner
ed23c8568e next try for "don't abort when given unmatched '}' with 'start-condition stack underflow'. This fixes #751" 
This reverts commit 8120b6fb8a and fixes the regression introduced in
8d22b26448.
 2016-01-19 20:35:35 +00:00
Eelco Dolstra
8120b6fb8a Revert "don't abort when given unmatched '}' with 'start-condition stack underflow'. This fixes #751" 
This reverts commit 8d22b26448. It
breaks Nixpkgs:

$ nix-env -qa
error: syntax error, unexpected IND_STR, expecting '}', at /home/eelco/Dev/nixpkgs-stable/pkgs/top-level/python-packages.nix:7605:8
 2016-01-19 20:33:32 +01:00
Shea Levy
a6d52fcd95 Merge branch 'lexer' of git://github.com/fkz/nix 2016-01-17 10:08:03 -05:00
Fabian Schmitthenner
8d22b26448 don't abort when given unmatched '}' with 'start-condition stack underflow'. This fixes #751 2016-01-12 20:40:41 +00:00
Eelco Dolstra
786046cf13 --option build-repeat: Keep the differing output if -K is given 
Similar to 00903fa799. Regardless of -K,
we now also print which output differs.
 2016-01-12 18:26:24 +01:00
Eelco Dolstra
8906eda2f9 Canonicalize gids to 0 
Previously files in the Nix store were owned by root or by nixbld,
depending on whether they were created by a substituter or by a
builder. This doesn't matter much, but causes spurious diffoscope
differences. So use root everywhere.
 2016-01-12 17:27:40 +01:00
Eelco Dolstra
00903fa799 --check: Keep the differing output if -K is given 
This makes it easier to investigate the non-determinism, e.g.

  $ nix-build pkgs/stdenv/linux -A stage1.pkgs.zlib --check -K
  error: derivation ‘/nix/store/l54i8wlw22656i4pk05c52ngv9rpl39q-zlib-1.2.8.drv’ may not be deterministic: output ‘/nix/store/11a27shh6n2ivi4a7s964i65ql80cf27-zlib-1.2.8’ differs from ‘/nix/store/11a27shh6n2ivi4a7s964i65ql80cf27-zlib-1.2.8-check’

  $ diffoscope /nix/store/11a27shh6n2ivi4a7s964i65ql80cf27-zlib-1.2.8 /nix/store/11a27shh6n2ivi4a7s964i65ql80cf27-zlib-1.2.8-check
  ...
  ├── lib/libz.a
  │   ├── metadata
  │   │ @@ -1,15 +1,15 @@
  │   │ -rw-r--r-- 30001/30000   3096 Jan 12 15:20 2016 adler32.o
  ...
  │   │ +rw-r--r-- 30001/30000   3096 Jan 12 15:28 2016 adler32.o
  ...
 2016-01-12 16:44:26 +01:00
Eelco Dolstra
0cad1f8049 --check: Fix "failed to produce output path" 
This occured when sandbox building is disabled, at least one output
exists, and at least one other output does not.
 2016-01-12 15:13:47 +01:00
Eelco Dolstra
1c57ab8b31 --check: Fix assertion failure when some outputs are missing 
E.g.

  $ nix-build pkgs/stdenv/linux/ -A stage1.pkgs.perl --check
  nix-store: src/libstore/build.cc:1323: void nix::DerivationGoal::tryToBuild(): Assertion `buildMode != bmCheck || validPaths.size() == drv->outputs.size()' failed.

when perl.out exists but perl.man doesn't. The fix is to only check
the outputs that exist. Note that "nix-build -A stage1.pkgs.all
--check" will still give a (proper) error in this case.
 2016-01-12 14:54:39 +01:00
Eelco Dolstra
458711e4ee Fix "Bad address" executing build hook 
This was observed in the deb_debian7x86_64 build:
http://hydra.nixos.org/build/29973215

Calling c_str() on a temporary should be fine because the temporary
shouldn't be destroyed until after the execl() call, but who knows...
 2016-01-07 15:10:14 +01:00
Eelco Dolstra
9aac1861f7 Fix some signedness warnings 2016-01-07 14:37:39 +01:00
Eelco Dolstra
02a66b3fd7 nix-store --repair-path: Rebuild if there is no substituter 2016-01-06 22:07:59 +01:00
Eelco Dolstra
caaaff3954 Fix --repair failure on multiple-output derivations 
If repair found a corrupted/missing path that depended on a
multiple-output derivation, and some of the outputs of the latter were
not present, it failed with a message like

  error: path ‘/nix/store/cnfn9d5fjys1y93cz9shld2xwaibd7nn-bash-4.3-p42-doc’ is not valid
 2016-01-06 21:49:32 +01:00
Christian Theune
934642155c @eelco's feedback: downgrade to regular float for size, remove unused function. 2016-01-06 08:25:58 +01:00
Fabian Schmitthenner
39d1da7b51 Better error message 
Also show types when nix cannot compare values of different types.
This is also more consistent since types are already shown when comparing values of the same not comparable type.
 2016-01-05 14:13:54 +01:00
Eelco Dolstra
9a4706eb19 Fix compilation 2016-01-05 14:05:11 +01:00
Shea Levy
14080f3e4b Use __toString when coercing sets to strings. 
For example, "${{ foo = "bar"; __toString = x: x.foo; }}" evaluates
to "bar".

With this, we can delay calling functions like mkDerivation,
buildPythonPackage, etc. until we actually need a derivation, enabling
overrides and other modifications to happen by simple attribute set
update.
 2016-01-05 13:55:06 +01:00
Eelco Dolstra
6298afc047 Merge pull request #685 from vizanto/master 
POSIX compliant directory access (fixes build on Solaris)
 2016-01-05 13:49:55 +01:00
FrankHB
607d91517e Fixed unresolved 'BUFSIZ' 
`BUFSIZ` is defined in header `<cstdio>`.
 2016-01-05 13:33:40 +01:00
Eelco Dolstra
743e310046 Fix non-Darwin build 2016-01-05 13:31:15 +01:00
Vladimír Čunát
29c0199d5c manual: document builtins.functionArgs 
The text is just a conversion of comment from src/libexpr/primops.cc
 2016-01-05 13:29:17 +01:00
Tuomas Tynkkynen
f770b9e6c5 libstore: mmap() returns MAP_FAILED, not NULL on failure 2016-01-05 13:26:35 +01:00
Christian Theune
a12a43046b Edge condition: parser did not pick up floats starting exactly with 0. 2016-01-05 09:54:49 +01:00