Commit graph

1501 commits

Author SHA1 Message Date
Eelco Dolstra
4ec6eb1fdf
Fix accidental printError 2017-07-17 11:38:15 +02:00
Eelco Dolstra
766ad5db3b
nix path-info: Show download sizes for binary cache stores
E.g.

  $ nix path-info --json --store https://cache.nixos.org nixpkgs.thunderbird -S
  ...
      "downloadHash": "sha256:1jlixpzi225wwa0f4xdrwrqgi47ip1qpj9p06fyxxg07sfmyi4q0",
      "downloadSize": 43047620,
      "closureDownloadSize": 84745960
    }
  ]
2017-07-14 18:29:10 +02:00
Eelco Dolstra
3908d3929c
nix path-info: Don't barf on invalid paths
Now you get

  [
    {
      "path": "/nix/store/fzvliz4j5xzvnd0w5zgw2l0ksqh578yk-bla",
      "valid": false
    }
  ]
2017-07-14 18:29:10 +02:00
Eelco Dolstra
8e8caf7f3e fetchTarball: Prevent concurrent downloads of the same file
Fixes #849.
2017-07-11 23:21:24 +02:00
Eelco Dolstra
d3713716b6 Merge pull request #1445 from matthewbauer/macos-skip-hardlink
Don’t hardlink disallowed paths in OS X.
2017-07-07 11:05:21 +02:00
Matthew Bauer
72e80c59b5 Don’t hardlink disallowed paths in OS X.
Fixes #1443
2017-07-06 19:30:19 -07:00
Eelco Dolstra
a3dc1e65ab
Add X32 to the seccomp filter
Fixes #1432.
2017-07-04 19:00:51 +02:00
Eelco Dolstra
42c5774e78
Sort substituters by priority
Fixes #1438.
2017-07-04 16:34:53 +02:00
Eelco Dolstra
b7203e853e
getDefaultSubstituters(): Simplify initialisation
As shlevy pointed out, static variables in C++11 have thread-safe
initialisation.
2017-07-04 16:26:48 +02:00
Eelco Dolstra
6cf23c3e8f
Add allow-new-privileges option
This allows builds to call setuid binaries. This was previously
possible until we started using seccomp. Turns out that seccomp by
default disallows processes from acquiring new privileges. Generally,
any use of setuid binaries (except those created by the builder
itself) is by definition impure, but some people were relying on this
ability for certain tests.

Example:

  $ nix build '(with import <nixpkgs> {}; runCommand "foo" {} "/run/wrappers/bin/ping -c 1 8.8.8.8; exit 1")' --no-allow-new-privileges
  builder for ‘/nix/store/j0nd8kv85hd6r4kxgnwzvr0k65ykf6fv-foo.drv’ failed with exit code 1; last 2 log lines:
    cannot raise the capability into the Ambient set
    : Operation not permitted

  $ nix build '(with import <nixpkgs> {}; runCommand "foo" {} "/run/wrappers/bin/ping -c 1 8.8.8.8; exit 1")' --allow-new-privileges
  builder for ‘/nix/store/j0nd8kv85hd6r4kxgnwzvr0k65ykf6fv-foo.drv’ failed with exit code 1; last 6 log lines:
    PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
    64 bytes from 8.8.8.8: icmp_seq=1 ttl=46 time=15.2 ms

Fixes #1429.
2017-07-04 15:48:25 +02:00
Eelco Dolstra
c0015e87af
Support base-64 hashes
Also simplify the Hash API.

Fixes #1437.
2017-07-04 15:07:41 +02:00
Eelco Dolstra
fcca702a96
Replace a few bool flags with enums
Functions like copyClosure() had 3 bool arguments, which creates a
severe risk of mixing up arguments.

Also, implement copyClosure() using copyPaths().
2017-07-03 11:38:08 +02:00
David McFarland
596b0e0a04 Call SetDllDirectory("") after sqlite3 init on cygwin
Cygwin sqlite3 is patched to call SetDllDirectory("/usr/bin") on init, which
affects the current process and is inherited by child processes.  It causes
DLLs to be loaded from /usr/bin/ before $PATH, which breaks all sorts of
things.  A typical failures would be header/lib version mismatches (e.g.
openssl when running checkPhase on openssh).  We'll just set it back to the
default value.

Note that this is a problem with the cygwin version of sqlite3 (currently
3.18.0).  nixpkgs doesn't have the problematic patch.
2017-06-20 10:59:13 -03:00
Eelco Dolstra
a1355917ec
Disable use of virtual hosting in aws-sdk-cpp
Recently aws-sdk-cpp quietly switched to using S3 virtual host URIs
(https://github.com/aws/aws-sdk-cpp/commit/69d9c53882), i.e. it sends
requests to http://<bucket>.<region>.s3.amazonaws.com rather than
http://<region>.s3.amazonaws.com/<bucket>. However this interacts
badly with curl connection reuse. For example, if we do the following:

1) Check whether a bucket exists using GetBucketLocation.
2) If it doesn't, create it using CreateBucket.
3) Do operations on the bucket.

then 3) will fail for a minute or so with a NoSuchBucket exception,
presumably because the server being hit is a fallback for cases when
buckets don't exist.

Disabling the use of virtual hosts ensures that 3) succeeds
immediately. (I don't know what S3's consistency guarantees are for
bucket creation, but in practice buckets appear to be available
immediately.)
2017-06-19 18:51:33 +02:00
Eelco Dolstra
82a0d614cf
Support creating S3 caches in other regions than us-east-1 2017-06-19 18:51:33 +02:00
Eelco Dolstra
b33621d425
Handle S3Errors::RESOURCE_NOT_FOUND from aws-sdk-cpp
This is returned by recent versions. Also handle NO_SUCH_KEY even
though the library doesn't actually return that at the moment.
2017-06-19 18:51:32 +02:00
Eelco Dolstra
1c969611ba
Suppress "will retry in N ms" for non-retriable errors
Newer versions of aws-sdk-cpp call CalculateDelayBeforeNextRetry()
even for non-retriable errors (like NoSuchKey) whih causes log spam in
hydra-queue-runner.
2017-06-19 18:51:32 +02:00
Eelco Dolstra
00aa7c6705
Show aws-sdk-cpp log messages 2017-06-19 18:51:32 +02:00
Eelco Dolstra
1888f7889b
macOS: Ugly hack to make the tests succeed
Sandboxes cannot be nested, so if Nix's build runs inside a sandbox,
it cannot use a sandbox itself. I don't see a clean way to detect
whether we're in a sandbox, so use a test-specific hack.

https://github.com/NixOS/nix/issues/1413
2017-06-19 14:26:05 +02:00
Eelco Dolstra
b5bdfdef73
macOS: Remove flags
In particular, UF_IMMUTABLE (uchg) needs to be cleared to allow the
path to be garbage-collected or optimised.

See https://github.com/NixOS/nixpkgs/issues/25819.
+       the file from being garbage-collected.
2017-06-19 14:19:21 +02:00
Eelco Dolstra
38b7d55af1
Remove redundant debug line 2017-06-14 13:45:38 +02:00
Eelco Dolstra
88b291ffc4
canonicalisePathMetaData(): Ignore security.selinux attribute
Untested, hopefully fixes #1406.
2017-06-14 11:41:03 +02:00
Eelco Dolstra
847f19a5f7
Provide a builtin default for $NIX_SSL_CERT_FILE
This is mostly to ensure that when Nix is started on macOS via a
launchd service or sshd (for a remote build), it gets a certificate
bundle.
2017-06-12 16:44:43 +02:00
Eelco Dolstra
7f5b750b40
Don't run pre-build-hook if we don't have a derivation
This fixes a build failure on OS X when using Hydra or Nix 1.12's
build-remote (since they don't copy the derivation to the build
machine).
2017-06-12 16:07:34 +02:00
Eelco Dolstra
186571965d
Don't show flags from config settings in "nix --help" 2017-06-07 18:41:20 +02:00
Eelco Dolstra
b8283773bd
nix: Make all options available as flags
Thus, instead of ‘--option <name> <value>’, you can write ‘--<name>
<value>’. So

  --option http-connections 100

becomes

  --http-connections 100

Apart from brevity, the difference is that it's not an error to set a
non-existent option via --option, but unrecognized arguments are
fatal.

Boolean options have special treatment: they're mapped to the
argument-less flags ‘--<name>’ and ‘--no-<name>’. E.g.

  --option auto-optimise-store false

becomes

  --no-auto-optimise-store
2017-06-07 16:17:17 +02:00
Eelco Dolstra
c8cc50d46e
Disable the build user mechanism on all platforms except Linux and OS X 2017-06-06 18:52:15 +02:00
Eelco Dolstra
85e93d7b87
Always use the Darwin sandbox
Even with "build-use-sandbox = false", we now use sandboxing with a
permissive profile that allows everything except the creation of
setuid/setgid binaries.
2017-06-06 18:44:49 +02:00
Eelco Dolstra
52fec8dde8
Remove listxattr assertion
It appears that sometimes, listxattr() returns a different value for
the query case (i.e. when the buffer size is 0).
2017-05-31 20:43:47 +02:00
Eelco Dolstra
c96e8cd097
OS X sandbox: Improve builtin sandbox profile
Also, add rules to allow fixed-output derivations to access the
network.

These rules are sufficient to build stdenvDarwin without any
__sandboxProfile magic.
2017-05-31 17:25:51 +02:00
Eelco Dolstra
44f3f8048f OS X sandbox: Don't use a deterministic $TMPDIR
This doesn't work because the OS X sandbox cannot bind-mount
path to a different location.
2017-05-31 14:09:00 +02:00
Eelco Dolstra
c740c3ce50 OS X sandbox: Store .sb file in $TMPDIR rather than the Nix store
The filename used was not unique and owned by the build user, so
builds could fail with

error: while setting up the build environment: cannot unlink ‘/nix/store/99i210ihnsjacajaw8r33fmgjvzpg6nr-bison-3.0.4.drv.sb’: Permission denied
2017-05-31 14:09:00 +02:00
Eelco Dolstra
acc889c821
Darwin sandbox: Use sandbox-defaults.sb
Issue #759.

Also, remove nix.conf from the sandbox since I don't really see a
legitimate reason for builders to access the Nix configuration.
2017-05-30 17:40:12 +02:00
Eelco Dolstra
53a1644187
Darwin sandbox: Disallow creating setuid/setgid binaries
Suggested by Daiderd Jordan.
2017-05-30 17:17:17 +02:00
Eelco Dolstra
fe08d17934
Fix seccomp build failure on clang
Fixes

  src/libstore/build.cc:2321:45: error: non-constant-expression cannot be narrowed from type 'int' to 'scmp_datum_t' (aka 'unsigned long') in initializer list [-Wc++11-narrowing]
2017-05-30 14:37:24 +02:00
Eelco Dolstra
d552d38758
Shut up some clang warnings 2017-05-30 14:35:50 +02:00
Eelco Dolstra
2ac99a32da
Add a seccomp rule to disallow setxattr() 2017-05-30 13:59:24 +02:00
Eelco Dolstra
d798349ede
canonicalisePathMetaData(): Remove extended attributes / ACLs
EAs/ACLs are not part of the NAR canonicalisation. Worse, setting an
ACL allows a builder to create writable files in the Nix store. So get
rid of them.

Closes #185.
2017-05-30 13:47:41 +02:00
Eelco Dolstra
ff6becafa8
Require seccomp only in multi-user setups 2017-05-30 12:37:04 +02:00
Eelco Dolstra
cf93397d3f
Fix seccomp initialisation on i686-linux 2017-05-29 16:14:10 +02:00
Eelco Dolstra
6cc6c15a2d
Add a seccomp filter to prevent creating setuid/setgid binaries
This prevents builders from setting the S_ISUID or S_ISGID bits,
preventing users from using a nixbld* user to create a setuid/setgid
binary to interfere with subsequent builds under the same nixbld* uid.

This is based on aszlig's seccomp code
(47f587700d).

Reported by Linus Heckemann.
2017-05-29 16:14:10 +02:00
Eelco Dolstra
6e01ecd112
Fix nix-copy-closure test
Fixes

  client# error: size mismatch importing path ‘/nix/store/ywf5fihjlxwijm6ygh6s0a353b5yvq4d-libidn2-0.16’; expected 0, got 120264

This is mostly an artifact of the NixOS VM test environment, where the
Nix database doesn't contain hashes/sizes.

http://hydra.nixos.org/build/53537471
2017-05-29 16:08:56 +02:00
Eelco Dolstra
63145be2a5
Fix typo 2017-05-29 15:52:36 +02:00
Eelco Dolstra
edbb105e98
Merge branch 'nar-accessor-tree' of https://github.com/bennofs/nix 2017-05-24 11:04:43 +02:00
Eelco Dolstra
b01d62285c
Improve progress indicator 2017-05-16 16:09:57 +02:00
Benno Fünfstück
a1f428b13b nar-accessor.cc: remove unused member NarIndexer::currentName 2017-05-15 19:41:59 +02:00
Benno Fünfstück
5ee06e612a nar-accessor: non-recursive NarMember::find
This avoids a possible stack overflow if directories are very deeply nested.
2017-05-15 19:34:18 +02:00
Eelco Dolstra
e80257f122
Simplify fixed-output check 2017-05-15 18:50:54 +02:00
Eelco Dolstra
a2d92bb20e
Add --with-sandbox-shell configure flag
And add a 116 KiB ash shell from busybox to the release build. This
helps to make sandbox builds work out of the box on non-NixOS systems
and with diverted stores.
2017-05-15 17:36:32 +02:00
Eelco Dolstra
b30f5784d0
Linux sandbox: Don't barf on invalid paths
This is useful when we're using a diverted store (e.g. "--store
local?root=/tmp/nix") in conjunction with a statically-linked sh from
the host store (e.g. "sandbox-paths =/bin/sh=/nix/store/.../bin/busybox").
2017-05-15 17:36:32 +02:00