Commit graph

2429 commits

Author SHA1 Message Date
Jude Taylor
3b52e54e31 darwin: allow reading system locale and zoneinfo 2015-11-03 08:43:09 -08:00
Jude Taylor
d6bd51ffbe allow reading ICU data 2015-10-31 02:52:42 -07:00
Jude Taylor
95c66cc950 add special devices to sandbox-defaults 2015-10-30 20:32:11 -07:00
Eelco Dolstra
1f735a3440 <nix/fetchurl.nix>: Support xz-compressed NARs 2015-10-30 12:34:30 +01:00
Eelco Dolstra
dae5dc7ade <nix/fetchurl.nix>: Support downloading and unpacking NARs
This removes the need to have multiple downloads in the stdenv
bootstrap process (like a separate busybox binary for Linux, or
curl/mkdir/sh/bzip2 for Darwin). Now all those files can be combined
into a single NAR.
2015-10-30 11:27:47 +01:00
Eelco Dolstra
5c28943e8f int2String() -> std::to_string() 2015-10-29 13:26:55 +01:00
Eelco Dolstra
45b2f32cc1 Merge pull request #668 from svanderburg/master
Fix compilation error due to missing ENOENT on cygwin
2015-10-26 11:12:01 +01:00
Jude Taylor
9155fbd386 use nixDataDir instead of appending /share to PREFIX 2015-10-21 14:44:44 -07:00
Jude Taylor
af1218a86a revert libutil change 2015-10-21 14:40:35 -07:00
Jude Taylor
992cda1b11 clarifying comment 2015-10-21 14:39:16 -07:00
Jude Taylor
76f3ba42fd move preBuildHook defaulting to globals.cc 2015-10-21 14:39:08 -07:00
Jude Taylor
2dc8e19a17 restore old DEFAULT_ALLOWED_IMPURE_PREFIXES 2015-10-21 14:38:49 -07:00
Jude Taylor
ff6953cb03 Add resolve-system-dependencies.pl 2015-10-21 12:38:52 -07:00
Jude Taylor
15c7a79ed3 remove usr paths from allowed inputs 2015-10-21 12:38:52 -07:00
Jude Taylor
5bf4085000 allow access to SystemVersion for python builders 2015-10-21 12:38:52 -07:00
Jude Taylor
6a8cee19b3 fix line reading in preBuildHook 2015-10-21 12:38:52 -07:00
Jude Taylor
1fc0fe1baf remove sandbox defaults into a new file 2015-10-21 12:38:52 -07:00
Jude Taylor
d4cac051f7 restore allowed impure prefixes 2015-10-21 12:38:52 -07:00
Jude Taylor
4b8c71b4ef remove an unneeded default impure-dep 2015-10-21 12:38:52 -07:00
Jude Taylor
e770f941d6 make sandbox builds more permissive 2015-10-21 12:38:52 -07:00
Jude Taylor
1ea2aa9e99 add a few more permissions 2015-10-21 12:38:52 -07:00
Eelco Dolstra
f5b32ff47d Allow builtin fetchurl regardless of the derivation's system attribute 2015-10-21 15:14:42 +02:00
Eelco Dolstra
01615b5f63 Show progress indicator for builtin fetchurl 2015-10-21 15:14:42 +02:00
Eelco Dolstra
5db358d4d7 Disable TLS verification for builtin fetchurl
This makes it consistent with the Nixpkgs fetchurl and makes it work
in chroots. We don't need verification because the hash of the result
is checked anyway.
2015-10-21 15:14:42 +02:00
Eelco Dolstra
357d31b339 Fix segfault in builtin fetchurl
The stack allocated for the builder was way too small (32 KB). This is
sufficient for normal derivations, because they just do some setup and
then exec() the actual builder. But for the fetchurl builtin
derivation it's not enough. Also, allocating the stack on the caller's
stack was fishy business.
2015-10-21 15:14:42 +02:00
Sander van der Burg
058d1950c0 Fix compilation error due to missing ENOENT on cygwin 2015-10-18 19:04:24 +00:00
Tuomas Tynkkynen
2075ec83e3 Allow building ARMv6 stuff on ARMv7
This allows building a Raspberry Pi image on modern, faster boards.
2015-10-08 13:32:28 +02:00
Eelco Dolstra
51cf4455b2 isFunctor: Simplify 2015-10-08 13:22:11 +02:00
Mathnerd314
8a87521636 forceFunction: allow functors as well 2015-10-08 13:19:44 +02:00
Eelco Dolstra
14f7e1fc42 Revert to CURLOPT_PROGRESSFUNCTION
CURLOPT_XFERINFOFUNCTION isn't widely supported yet.

http://hydra.nixos.org/build/26679495
2015-10-08 13:15:21 +02:00
Eelco Dolstra
21ecd106ed Show progress during downloads 2015-10-07 17:31:50 +02:00
Eelco Dolstra
f08449ccbd nix-prefetch-url: Add --name option
This allows overriding the name component of the resulting Nix store
path, which is necessary if the base name of the URI contains
"illegal" characters.
2015-10-07 14:55:33 +02:00
Eelco Dolstra
e4abf8610b nix-prefetch-url -A: Use "name" attribute from Nix expression
This is in particular useful for fetchFromGitHub et al., ensuring that
the store path produced by nix-prefetch-url corresponds to what those
functions expect.
2015-10-07 14:47:39 +02:00
Eelco Dolstra
b54f447df9 nix-prefetch-url: Support unpacking tarballs
This allows nix-prefetch-url to prefetch the output of fetchzip and
its wrappers (like fetchFromGitHub). For example:

  $ nix-prefetch-url --unpack https://github.com/NixOS/patchelf/archive/0.8.tar.gz

or from a Nix expression:

  $ nix-prefetch-url -A nix-repl.src

In the latter case, --unpack can be omitted because nix-repl.src is a
fetchFromGitHub derivation and thus has "outputHashMode" set to
"recursive".
2015-10-07 14:40:10 +02:00
Manuel Jacob
7888b2ba15 Define SOL_LOCAL if not defined already (e.g. on FreeBSD).
Some evidence that defining it to be 0 is right:
* OS X headers define it to be 0.
* Other code uses 0 instead of SOL_LOCAL to check for peer credentials
  (e.g. FreeBSD's implementation of getpeereid).
2015-10-06 22:28:30 +02:00
Manuel Jacob
12c1776df9 Don't try to link libdl on FreeBSD.
There is no libdl on FreeBSD.  Instead the symbols are included in libc.
2015-10-06 22:28:30 +02:00
Manuel Jacob
cbc177ca01 Use pkg-config-provided LDFLAGS for libsqlite3 and libcurl.
Previously, pkg-config was already queried for libsqlite3's and
libcurl's link flags.  However they were not used, but hardcoded
instead.  This commit replaces the hardcoded LDFLAGS by the ones
provided by pkg-config in a similar pattern as already used for
libsodium.
2015-10-06 22:28:30 +02:00
Eelco Dolstra
1abda8e173 nix-store --serve: Implement log size limit 2015-10-06 17:33:30 +02:00
Eelco Dolstra
06f29fafe8 nix-prefetch-url: Support prefetching from a Nix expression
For example,

  $ nix-prefetch-url -A hello.src

will prefetch the file specified by the fetchurl call in the attribute
‘hello.src’ from the Nix expression in the current directory. This
differs from ‘nix-build -A hello.src’ in that it doesn't verify the
hash.

You can also specify a path to the Nix expression:

  $ nix-prefetch-url ~/Dev/nixpkgs -A hello.src

List elements (typically used in ‘patches’ attributes) also work:

  $ nix-prefetch-url -A portmidi.patches.0
2015-10-01 18:07:56 +02:00
Eelco Dolstra
a3c4eb0964 nix-prefetch-url: $PRINT_PATH -> --print-path 2015-10-01 16:53:07 +02:00
Eelco Dolstra
bec3c31608 nix-prefetch-url: Rewrite in C++ 2015-10-01 16:47:43 +02:00
Eelco Dolstra
4ba6bc184c Shut up clang warnings 2015-09-18 01:22:35 +02:00
Vladimír Čunát
42808fa281 nix-env --upgrade: show "downgrading" when doing so
It was strange to show "upgrading" when the version was getting lower.
This is left on "upgrading" when the versions are the same,
as I can't see any better wording.
2015-09-17 12:09:54 +02:00
Vladimír Čunát
64f9b511be nix-env --upgrade: avoid unexpected downgrades
Until now, if one explicitly installed a low-priority version,
nix-env --upgrade would downgrade it by default and even with --leq.
Let's never accept an upgrade with version not matching the upgradeType.
Additionally, let's never decrease the priority of an installed package;
you can use --install to force that.

Also refactor to use variable bestVersion instead of bestName,
as only version was used from it.
2015-09-17 12:06:26 +02:00
Eelco Dolstra
71a5161365 Implement buildDerivation() via the daemon 2015-09-03 12:56:59 +02:00
Eelco Dolstra
73332fd342 Filter build-chroot-dirs entries that conflict with derivation outputs
Fixes https://github.com/NixOS/nixpkgs/issues/9504.

Note that this means we may have a non-functional /bin/sh in the
chroot while rebuilding Bash or one of its dependencies. Ideally those
packages don't rely on /bin/sh though.
2015-09-02 14:54:12 +02:00
Eelco Dolstra
d711fe8f5e Remove unused variable 2015-09-02 14:46:04 +02:00
Eelco Dolstra
e12cf82782 Prevent .chroot from being GC'ed when using LocalStore::buildDerivation()
Fixes #616.
2015-08-24 11:13:31 +02:00
Eelco Dolstra
eadb86f447 nix-collect-garbage: Revive --max-freed
Fixes #609.
2015-08-21 13:57:53 +02:00
Manolis Ragkousis
26221e44eb Remove unneeded HAVE_UNSHARE.
* src/libstore/build.cc (CHROOT_ENABLED): Remove HAVE_UNSHARE.
2015-08-19 14:11:15 +03:00
Kirill Elagin
3b0f60e5c2 baseNameOf: Enhance basename compatibility
* If the path ends with a slash, drop it.
* If the remaining path doesn’t contain slashes, just return it.

Fixes #574.
2015-08-07 03:35:02 +02:00
Eelco Dolstra
34dfbd9394 Don't include <iostream> before config.h
This breaks the build on 32-bit systems.

http://hydra.nixos.org/build/24373658
2015-08-04 11:12:31 +02:00
Eelco Dolstra
30d19a2bdc Handle debug messages from runChild()
Turns out that "nix-build -vvv" with chroots enabled has been broken
for some time, because some debug message got interpreted as an error.
2015-08-03 18:04:32 +02:00
Iwan Aucamp
3db950aab7 Removed unnecessary included 2015-08-03 11:48:34 +02:00
Eelco Dolstra
2bac04c5ff Fix stack consumption 2015-07-31 20:28:25 +02:00
Iwan Aucamp
75837651f1 Output line number on infinite recursion 2015-07-31 20:26:44 +02:00
Eelco Dolstra
76cc8e97a2 Add sort primop 2015-07-28 18:39:39 +02:00
Eelco Dolstra
50807f3dd5 Add primop genList
This can be used to implement functions like ‘imap’ (or for that
matter, ‘map’) without the quadratic complexity incurred by calling
‘++’ repeatedly.
2015-07-28 17:28:35 +02:00
Eelco Dolstra
f3dda728a4 Remove unnecessary parentheses 2015-07-26 12:07:14 +02:00
Eelco Dolstra
d6d5885c15 Add replaceStrings primop
This is a generalisation of replaceChars in Nixpkgs.
2015-07-24 15:32:24 +02:00
Eelco Dolstra
2e8fd4c5cd Add concatStringsSep as a primop
This fixes the quadratic behaviour of concatStrings/concatStringsSep
in Nixpkgs.
2015-07-24 02:38:09 +02:00
Eelco Dolstra
cb4320c1a0 Cleanup 2015-07-23 23:14:07 +02:00
Eelco Dolstra
c8bb2371eb Optimize empty sets
This reduces the number of Bindings allocations by about 10%.
2015-07-23 23:11:08 +02:00
Eelco Dolstra
16c9935fa9 Fix fetchurl of executable file
Pointed out by @cstrahan, thanks!
2015-07-23 22:25:04 +02:00
Eelco Dolstra
19eddecc0f Merge branch 'attr-set-hh' of https://github.com/nbp/nix
Conflicts:
	src/libexpr/eval.cc
2015-07-23 22:16:01 +02:00
Eelco Dolstra
b83801f8b3 Optimize small lists
The value pointers of lists with 1 or 2 elements are now stored in the
list value itself. In particular, this makes the "concatMap (x: if
cond then [(f x)] else [])" idiom cheaper.
2015-07-23 22:05:09 +02:00
Eelco Dolstra
14be783676 Add primops all and any
These are used thousands of times during NixOS evaluation, so it's
useful to speed them up.
2015-07-23 19:23:11 +02:00
Shea Levy
39e27a04b8 Importing derivations: Add name attribute to make a valid drv 2015-07-23 17:04:07 +02:00
Shea Levy
1ed55234d9 Allow derivations-as-srcs in the context of builtins.toFile files 2015-07-23 17:03:47 +02:00
Eelco Dolstra
61af14a921 Add foldl' primop 2015-07-23 17:03:02 +02:00
Eelco Dolstra
887bb5fa5a --version: Print some config info
Such as whether Nix is built with signed binary cache support, and the
location of the configuration file.
2015-07-23 14:38:00 +02:00
Eelco Dolstra
1993b10d11 Fix Darwin build
Turns out getgrouplist() is not POSIX.

http://hydra.nixos.org/build/23881243
2015-07-21 14:45:24 +02:00
Eelco Dolstra
0a2bee307b Make <nix/fetchurl.nix> a builtin builder
This ensures that 1) the derivation doesn't change when Nix changes;
2) the derivation closure doesn't contain Nix and its dependencies; 3)
we don't have to rely on ugly chroot hacks.
2015-07-20 04:38:46 +02:00
Eelco Dolstra
eda2f36c2a Provide more detailed info about build status to hydra-queue-runner
In particular, hydra-queue-runner can now distinguish between remote
build / substitution / already-valid. For instance, if a path already
existed on the remote side, we don't want to store a log file.
2015-07-20 03:20:03 +02:00
Eelco Dolstra
ccf31dbc25 nix-copy-closure: Add -v flag
And make exportPath() less spammy by default.
2015-07-20 01:52:07 +02:00
Eelco Dolstra
db55940d9e Support systemd log severity prefixes
This is mostly useful for hydra-queue-runner.
2015-07-20 01:39:48 +02:00
Eelco Dolstra
b3491c781c More cleanup 2015-07-20 01:16:16 +02:00
Eelco Dolstra
6bd2c7bb38 OCD: foreach -> C++11 ranged for 2015-07-17 20:13:56 +02:00
Eelco Dolstra
1511aa9f48 Allow remote builds without sending the derivation closure
Previously, to build a derivation remotely, we had to copy the entire
closure of the .drv file to the remote machine, even though we only
need the top-level derivation. This is very wasteful: the closure can
contain thousands of store paths, and in some Hydra use cases, include
source paths that are very large (e.g. Git/Mercurial checkouts).

So now there is a new operation, StoreAPI::buildDerivation(), that
performs a build from an in-memory representation of a derivation
(BasicDerivation) rather than from a on-disk .drv file. The only files
that need to be in the Nix store are the sources of the derivation
(drv.inputSrcs), and the needed output paths of the dependencies (as
described by drv.inputDrvs). "nix-store --serve" exposes this
interface.

Note that this is a privileged operation, because you can construct a
derivation that builds any store path whatsoever. Fixing this will
require changing the hashing scheme (i.e., the output paths should be
computed from the other fields in BasicDerivation, allowing them to be
verified without access to other derivations). However, this would be
quite nice because it would allow .drv-free building (e.g. "nix-env
-i" wouldn't have to write any .drv files to disk).

Fixes #173.
2015-07-17 17:57:40 +02:00
Eelco Dolstra
f39979c6d3 Make printValue() interruptible
Fixes #572.
2015-07-17 11:33:39 +02:00
Eelco Dolstra
7c9d0a5969 nix-collect-garbage: Handle ENOENT
Don't barf trying to read a link that just got deleted.

Fixes #575.
2015-07-17 11:24:25 +02:00
Nicolas B. Pierron
db21cfa688 Move attribute set data structures into their own header file.
This modification moves Attr and Bindings structures into their own header
file which is dedicated to the attribute set representation. The goal of to
isolate pieces of code which are related to the attribute set
representation. Thus future modifications of the attribute set
representation will only have to modify these files, and not every other
file across the evaluator.
2015-07-14 19:23:17 +02:00
Guillaume Maudoux
467977f203 Fix the parsing of "$"'s in strings. 2015-07-03 14:09:58 +02:00
Guillaume Maudoux
65e4dcd69b Fix the hack that resets the scanner state. 2015-07-03 13:53:36 +02:00
Eelco Dolstra
dd48c06bb6 Typo 2015-07-02 00:30:16 +02:00
Ludovic Courtès
9aed117395 Preserve supplementary groups of build users
The following patch is an attempt to address this bug (see
<http://bugs.gnu.org/18994>) by preserving the supplementary groups of
build users in the build environment.

In practice, I would expect that supplementary groups would contain only
one or two groups: the build users group, and possibly the “kvm” group.

[Changed &at(0) to data() and removed tabs - Eelco]
2015-07-01 14:57:48 +02:00
Eelco Dolstra
ff4de4cb27 GC: Handle ENOSPC creating/moving to the trash directory
Issue #564.
2015-06-30 21:41:26 +02:00
Eelco Dolstra
2bc9c84327 Use posix_fallocate to create /nix/var/nix/db/reserved 2015-06-22 15:54:55 +02:00
Eelco Dolstra
ba63ec6f39 Make /nix/var/nix/db/reserved bigger
Issue #564.
2015-06-22 15:47:40 +02:00
Eelco Dolstra
65f17cd330 Support URLs in $NIX_PATH
This didn't work (despite claims in the manual), because the colon in
"http://" was parsed as a element separator. So handle "://"
specially.
2015-06-17 16:20:11 +02:00
Eelco Dolstra
b65875f859 Export outputPaths function
This is useful for the new hydra-queue-runner.
2015-06-10 16:17:06 +02:00
Eelco Dolstra
7c4501886d Use std::vector::data() 2015-06-09 10:54:46 +02:00
Eelco Dolstra
b64988bb35 Allow substitutes for builds that have preferLocalBuild set
Not substituting builds with "preferLocalBuild = true" was a bad idea,
because it didn't take the cost of dependencies into account. For
instance, if we can't substitute a fetchgit call, then we have to
download/build git and all its dependencies.

Partially reverts 5558652709 and adds a
new derivation attribute "allowSubstitutes" to specify whether a
derivation may be substituted.
2015-06-04 16:30:22 +02:00
Eelco Dolstra
07d7e7df84 Chown files created for passAsFile
Nixpkgs' writeTextAsFile does this:

  mv "$textPath" "$n"

Since $textPath was owned by root, if $textPath is on the same
filesystem as $n, $n will be owned as root. As a result, the build
result was rejected as having suspicious ownership.

http://hydra.nixos.org/build/22836807
2015-06-04 14:07:43 +02:00
Eelco Dolstra
d8ddf994e7 Don't let unprivileged users repair paths 2015-06-02 02:21:54 +02:00
Eelco Dolstra
7106bb0611 Use StoreAPI::verifyStore() 2015-06-02 02:21:15 +02:00
Ludovic Courtès
b755752f76 Add a ‘verifyStore’ RPC
Hello!

The patch below adds a ‘verifyStore’ RPC with the same signature as the
current LocalStore::verifyStore method.

Thanks,
Ludo’.

>From aef46c03ca77eb6344f4892672eb6d9d06432041 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@gnu.org>
Date: Mon, 1 Jun 2015 23:17:10 +0200
Subject: [PATCH] Add a 'verifyStore' remote procedure call.
2015-06-02 02:14:24 +02:00
Eelco Dolstra
bc51175dc0 Add tarball tests 2015-06-01 16:18:23 +02:00
Eelco Dolstra
da196ec68f Document tarball downloading 2015-06-01 15:14:44 +02:00
Eelco Dolstra
c1323b53e3 Fix fetchurl/fetchTarball 2015-06-01 15:08:09 +02:00