Eelco Dolstra
c905d8b0a8
GC_malloc -> GC_MALLOC
...
This makes it possible to build with -DGC_DEBUG.
2018-06-12 17:49:55 +02:00
Eelco Dolstra
455d1f01d0
Don't scan for roots in dynamic libraries
...
This reduces the risk of object liveness misdetection. For example,
Glibc has an internal variable "mp_" that often points to a Boehm
object, keeping it alive unnecessarily. Since we don't store any
actual roots in global variables, we can just disable data segment
scanning.
With this, the max RSS doing 100 evaluations of
nixos.tests.firefox.x86_64-linux.drvPath went from 718 MiB to 455 MiB.
2018-06-12 17:49:55 +02:00
Eelco Dolstra
30964103dc
Add temporary stats
2018-06-12 17:49:55 +02:00
Eelco Dolstra
24c6806994
Cache parse trees
...
This prevents EvalState::resetFileCache() from parsing everything all
over again.
2018-06-12 17:49:55 +02:00
Eelco Dolstra
6ad0a2f749
Remove duplicate definition of allocBytes()
2018-06-12 17:49:51 +02:00
Yorick van Pelt
72a78beb34
Fix #2162 : use getaddrinfo instead of curl to preload NSS
2018-06-12 13:05:14 +02:00
Symphorien Gibol
8c567afe35
libstore/gc.cc: ignore ESRCH when reading /proc
...
If a process disappears between the time /proc/[pid]/maps is opened and
the time it is read, the read() syscall will return ESRCH. This should be ignored.
2018-06-11 16:29:47 +02:00
AmineChikhaoui
170165ee5a
use debug instead of printTalkative to avoid spamming hydra queue
...
runner logs.
2018-06-08 11:43:32 +01:00
Eelco Dolstra
4ac4f675df
Don't require --fallback to recover from disappeared binary cache NARs
2018-06-05 16:04:41 +02:00
Eelco Dolstra
691b7582c7
Handle HTTP 410 Gone errors
2018-06-05 16:03:32 +02:00
Eelco Dolstra
ca06a9cea7
Improve binary cache fallback test
2018-06-05 15:26:12 +02:00
Eelco Dolstra
a2ec7a3bfd
Further improve upload messages
2018-06-05 14:37:26 +02:00
Will Dietz
54b1c59643
download.cc: fix readCallback return type (int -> size_t)
...
I'm not sure if curl ever asks for enough data at once
for truncation to occur but better safe than sorry.
2018-06-01 10:59:26 -05:00
Eelco Dolstra
73357500ac
Improve binary cache upload messages
...
Don't say "download" when we mean "upload".
2018-06-01 14:14:22 +02:00
Peter Simons
93aa3bea2e
Merge pull request #767 from mogorman/garbage_collect_keep_last_few
...
Implement --delete-generations + flag for keeping last N number of gens
2018-05-31 10:00:21 +02:00
Eelco Dolstra
5d4a9d5677
Fix static assertion failure on 32-bit systems
2018-05-30 17:47:30 +02:00
Eelco Dolstra
3c2de9830d
RemoteStore::addToStore(): Ensure capacity is decreased on exceptions
2018-05-30 13:48:31 +02:00
Eelco Dolstra
a4c1618876
Fix deadlock in RemoteStore::queryPathInfoUncached()
2018-05-30 13:46:14 +02:00
Eelco Dolstra
7d21863bb3
Make <nix/fetchurl.nix> run in constant memory
...
E.g.
nix-build --store ~/my-nix/ -E 'import <nix/fetchurl.nix> { url = https://cache.nixos.org/nar/0nwi996rgq4b914qyx0mv2wq4k80hjac7xilikavagw7kxmn2iiv.nar.xz ; sha256 = "0nwi996rgq4b914qyx0mv2wq4k80hjac7xilikavagw7kxmn2iiv"; }'
now runs in 17 MiB (was 70 MiB), while
nix-build --store ~/my-nix/ -E 'import <nix/fetchurl.nix> { url = https://cache.nixos.org/nar/0nwi996rgq4b914qyx0mv2wq4k80hjac7xilikavagw7kxmn2iiv.nar.xz ; sha256 = "0d2fxljdih3nc5dqx41hjzic3141ajil94m8kdbpryq569dpsbvb"; unpack = true; }'
runs in 17 MiB (was 346 MiB).
2018-05-30 13:42:51 +02:00
Eelco Dolstra
5a654fd7dd
getDownloader(): Simplify
2018-05-30 13:42:30 +02:00
Eelco Dolstra
e87e4a60d6
Make HttpBinaryCacheStore::narFromPath() run in constant memory
...
This reduces memory consumption of
nix copy --from https://cache.nixos.org --to ~/my-nix /nix/store/95cwv4q54dc6giaqv6q6p4r02ia2km35-blender-2.79
from 176 MiB to 82 MiB. (The remaining memory is probably due to xz
decompression overhead.)
Issue https://github.com/NixOS/nix/issues/1681 .
Issue https://github.com/NixOS/nix/issues/1969 .
2018-05-30 13:42:29 +02:00
Eelco Dolstra
08ec757726
Make LocalBinaryCacheStore::narFromPath() run in constant memory
...
This reduces memory consumption of
nix copy --from file://... --to ~/my-nix /nix/store/95cwv4q54dc6giaqv6q6p4r02ia2km35-blender-2.79
from 514 MiB to 18 MiB for an uncompressed binary cache, and from 192
MiB to 53 MiB for a bzipped binary cache. It may also be faster
because fetching can happen concurrently with decompression/writing.
Continuation of 48662d151bhttps://github.com/NixOS/nix/issues/1681 .
2018-05-30 13:34:57 +02:00
Eelco Dolstra
81ea8bd5ce
Simplify the callback mechanism
2018-05-30 13:34:37 +02:00
Eelco Dolstra
1672bcd230
Move evaluator-specific settings out of libstore
2018-05-30 13:29:50 +02:00
Eelco Dolstra
c1d445ecec
Move some Download-specific settings to download.cc
2018-05-30 13:29:50 +02:00
Eelco Dolstra
737ed88f35
Modularize config settings
...
Allow global config settings to be defined in multiple Config
classes. For example, this means that libutil can have settings and
evaluator settings can be moved out of libstore. The Config classes
are registered in a new GlobalConfig class to which config files
etc. are applied.
Relevant to https://github.com/NixOS/nix/issues/2009 in that it
removes the need for ad hoc handling of useCaseHack, which was the
underlying cause of that issue.
2018-05-30 13:28:01 +02:00
Eelco Dolstra
e606cd412f
Fix assertion failure in storePathToHash()
...
Fixes https://github.com/NixOS/nix/issues/2015 .
2018-05-30 13:27:19 +02:00
Eelco Dolstra
6185d25e52
Make 'nix copy --to daemon' run in constant memory (daemon side)
...
Continuation of 97002b684chttps://github.com/NixOS/nix/issues/1681 .
2018-05-30 13:26:48 +02:00
Eelco Dolstra
23d6bb583a
LocalStore::addToStore: Throw an exception if no hash is given
2018-05-30 13:25:40 +02:00
Eelco Dolstra
3cab639e21
Use extended SQLite error codes
2018-05-30 13:25:03 +02:00
Eelco Dolstra
bd56b5fe3f
Merge pull request #2187 from bgamari/stoi-exceptions
...
json-to-value: Use strtol instead of strtoi
2018-05-28 14:54:49 +02:00
Ben Gamari
7f560b81ea
json-to-value: Use strtol instead of strtoi
...
NixInt is long, so strtoi is too restrictive.
2018-05-26 18:43:46 -04:00
Will Dietz
93ae90de0f
eval.cc: add message to static_assert, message can be omitted w/c++17
2018-05-26 14:04:41 -05:00
Will Dietz
14c464b6c8
nix-daemon: remove unused "pendingMsgs" variable
2018-05-26 13:09:55 -05:00
Will Dietz
ca9c6cb95d
libstore/download: remove unused function "getTime()"
2018-05-25 19:49:45 -05:00
Eelco Dolstra
743359bc8a
Merge pull request #2157 from volth/bitwise
...
add builtins: __bitAnd, __bitOr, __bitXor
2018-05-24 15:00:39 +02:00
Eelco Dolstra
4bb8741b98
Make Env self-describing
...
If the Env denotes a 'with', then values[0] may be an Expr* cast to a
Value*. For code that generically traverses Values/Envs, it's useful
to know this.
2018-05-22 16:02:32 +02:00
Eelco Dolstra
9fd7cf98db
Memoise checkSourcePath()
...
This prevents hydra-eval-jobs from statting the same files over and
over again.
2018-05-22 13:02:14 +02:00
Will Dietz
b08923b4a0
serialise: fix buffer size used, hide method for internal use only
...
Fixes #2169 .
2018-05-21 17:30:38 -05:00
volth
6cc28c0589
add `mod' and bitwise builtins: camel-case function names
2018-05-16 10:52:19 +00:00
volth
f3c090f91c
add mod' and bitwise builtins: remove mod' and shifts
2018-05-16 06:57:11 +00:00
volth
49b7cf1813
add `mod' and bitwise builtins: remove infix functions
2018-05-16 06:55:24 +00:00
Eelco Dolstra
966407bcf1
Merge pull request #2145 from serokell/uncached-channel
...
make sure not to use cached channels for nix-channel --update
2018-05-15 14:57:18 +02:00
volth
8a6a14e1f5
add `mod' and bitwise builtins
2018-05-12 16:50:39 +00:00
Eelco Dolstra
1ad19232c4
Don't return negative numbers from the flex tokenizer
...
Fixes #1374 .
Closes #2129 .
2018-05-11 12:05:12 +02:00
Eelco Dolstra
f3c85f9eb3
Revert "Throw a specific error for incomplete parse errors."
...
This reverts commit 6498adb002
2018-05-11 11:40:50 +02:00
Eelco Dolstra
2b3c1b3a88
Remove mention of default since we already know the default
2018-05-10 12:06:28 +02:00
Eelco Dolstra
38def17627
Merge branch 's3-multipart-uploads' of https://github.com/AmineChikhaoui/nix
2018-05-10 12:06:02 +02:00
Yorick van Pelt
b9289e4855
make sure not to use cached channels for nix-channel --update
...
fixes #1964
2018-05-09 16:18:20 +02:00
Eelco Dolstra
a91c4ca01f
In restricted eval mode, allow access to the closure of store paths
...
E.g. this makes
nix eval --restrict-eval -I /nix/store/foo '(builtins.readFile "/nix/store/foo/symlink/bla")'
(where /nix/store/foo/symlink is a symlink to another path in the
closure of /nix/store/foo) succeed.
This fixes a regression in Hydra compared to Nix 1.x (where there were
no restrictions at all on access to the Nix store).
2018-05-09 15:45:05 +02:00
AmineChikhaoui
854c0860f4
share the executor between multiple copy threads.
2018-05-07 15:07:00 +01:00
AmineChikhaoui
591e75cd01
add a FIXME note to find a way to abort the multipart uploads in
...
case the nix command is interrupted.
2018-05-07 14:27:53 +01:00
AmineChikhaoui
129394fb95
Support multi-part uploads for large NARs that exceed the size of
...
5Gb.
2018-05-07 14:23:51 +01:00
Eelco Dolstra
a1adcdf087
nix-shell: Only download requested outputs
...
Fixes #2116 .
2018-05-07 14:58:25 +02:00
Eelco Dolstra
c717d8e3bd
Don't set a CA assertion for paths with references
...
Really fixes #2133 .
2018-05-03 14:49:56 +02:00
Eelco Dolstra
80a7b16593
LocalStore::addValidPath(): Throw error when the CA assertion is wrong
...
Closes #2133 .
2018-05-03 11:02:40 +02:00
Eelco Dolstra
53ec5ac69f
Fix some random -Wconversion warnings
2018-05-02 13:56:34 +02:00
Eelco Dolstra
4a2c948943
Fix bzip2 compression of files > 4 GiB
...
Bzip2's 'avail_in' parameter is declared as an unsigned int, so
assigning a size_t length to it led to silent truncation.
Fixes #2111 .
2018-05-02 12:54:30 +02:00
Will Dietz
8e7d77d494
globals.hh: don't use '==' to compare string literals
...
Saw this in logs, also reported here:
8e6108ff71 (r28707288)
2018-04-23 20:57:05 -05:00
Shea Levy
8e6108ff71
Merge branch 'aarch64-armv7' of git://github.com/lheckemann/nix
...
Support extra compatible architectures (#1916 )
2018-04-23 08:48:22 -04:00
Shea Levy
e2b028353b
Merge branch 'pos-crash-fix' of git://github.com/dezgeg/nix
2018-04-23 08:43:54 -04:00
Shea Levy
58f9ae0dcd
Merge branch 'hash-mismatch' of git://github.com/LnL7/nix
2018-04-23 08:40:34 -04:00
Andrew Dunham
3a918014b2
Fix library ordering in Makefiles
...
The existing ordering linked `libutil` before `libstore`, which causes
link failures when building statically. This is due to `libstore` using
functions from `libutil`, and the fact that symbol resolution works
"forward" - i.e. if you pass `-lfoo -lbar -lbaz`, any symbols that
`libbar` uses from `libbaz` will be resolved, but symbols from `libfoo`
will not since it comes first in the command line.
All this to say: this commit reorders the libraries which fixes the link
errors.
2018-04-21 21:10:52 -07:00
Peter Simons
040acdcee2
Merge pull request #1707 from dtzWill/fix/issue-1692
...
ignore "interrupted" exception in progress callback
2018-04-21 16:51:19 +02:00
Daiderd Jordan
13d4d1c0a2
download: make hash mismatch error consistent with fetchurl
2018-04-20 21:07:32 +02:00
Will Dietz
6d9129014d
build.cc: fix bind-mount of /dev/{pts,ptmx} fallback
...
Don't bind-mount these to themselves,
mount them into the chroot directory.
Fixes pty issues when using sandbox on CentOS 7.4.
(build of perlPackages.IOTty fails before this change)
2018-04-20 12:24:23 -05:00
Daniel Poelzleithner
b712d4674b
Allow multiple search experssions in nix search
...
The common use case is to search for packages containing multiple words
like a "git" "frontend". Having only one expressions makes this simple regular
use case very complicated. Instead, search accepts multiple regular epressions
which all need to match.
nix search git 'gui|frontend'
returns a list of all git uis for example
2018-04-18 21:08:35 +02:00
Shea Levy
b37f5ae31d
isFunction: True on primops.
...
Fixes #2073
2018-04-17 14:33:12 -04:00
Eelco Dolstra
a4aac7f88c
Handle arguments in $EDITOR
...
Fixes #2079 .
2018-04-17 12:16:04 +02:00
Eelco Dolstra
c74f838620
exportReferences: Check whether the path is in the input closure
...
Fixes #2082 .
2018-04-17 12:03:27 +02:00
Félix Baylac-Jacqué
a91fb422fe
nix repl: Fix multiline SIGINT handling.
...
Fixes #2076
2018-04-16 16:09:30 +02:00
Eelco Dolstra
d34fa2bcc3
Fix #1921
2018-04-13 15:42:35 +02:00
Shea Levy
dc0a542c9f
initPlugins: Fix dlopen error message.
2018-04-11 21:02:50 -04:00
Eelco Dolstra
1839a5542a
Fix #2057
2018-04-11 11:42:17 +02:00
Eelco Dolstra
d6aaa35478
Fix perl-bindings
...
https://hydra.nixos.org/build/72636781
2018-04-10 13:11:43 +02:00
Shea Levy
a38fe5c1a8
Export required C++ version in pkgconfig.
2018-04-09 11:32:43 -04:00
Shea Levy
7459388448
Merge branch 'fix/ruby-shebang' of git://github.com/samueldr/nix
2018-04-09 10:54:39 -04:00
Shea Levy
fd98fca7bb
Make prim_exec and prim_importNative available to plugins
2018-04-09 10:26:50 -04:00
Shea Levy
346c0ac361
nix-daemon: Exit successfully when interrupted.
...
Fixes #2058 .
2018-04-09 08:05:54 -04:00
Samuel Dionne-Riel
399f43c3d5
nix-shell: Fixes use with ruby shebangs.
...
The ported code in 80ebc553ec
2018-04-07 22:38:45 -04:00
AmineChikhaoui
62d75ad3e1
rename the options to mention it's a narinfo TTL as disk cache is used all over the place for other operations
2018-04-06 11:05:15 +01:00
AmineChikhaoui
86930ed414
add documentation for the local disk cache TTL config
2018-04-06 10:51:18 +01:00
AmineChikhaoui
2855c3d965
Make the TTL for disk cache configurable, we can now completely disable
...
disk cache lookup for example by doing:
nix copy --from <binary-cahe> <store-path> --option \
positive-disk-cache-ttl 0
Issues: #1885 #2035
2018-04-06 10:51:18 +01:00
Eelco Dolstra
1d9742b95f
<nix/buildenv.nix>: Ignore dangling symlinks
...
Fixes
error: getting status of '/nix/store/j8p0vv89k1pf0cn7kmfsdcs7bshwga1i-firefox-52.7.2esr/share/icons/hicolor/48x48/apps/firefox.png': No such file or directory
https://github.com/NixOS/nix/issues/1934
Also improve error message on directory/non-directory collisions.
2018-04-05 20:04:42 +02:00
Eelco Dolstra
5c904e10eb
<nix/buildenv.nix>: Skip non-directory top-level paths
...
Fixes https://github.com/NixOS/nix/issues/1934 (at least the "error:
opening directory '/nix/store/...-linux-config-4.4.14': Not a
directory" issue).
2018-04-05 19:34:56 +02:00
Tuomas Tynkkynen
af86132e1a
libexpr: Make unsafeGetAttrPos not crash on noPos
...
Currently e.g. `builtins.unsafeGetAttrPos "abort" builtins` will
eventually segfault because pos->file is an unset Symbol.
Found by afl-fuzz.
2018-04-03 15:54:42 +03:00
zimbatm
865ca2402f
nix copy: add an example with S3
...
I couldn't find a good example how to use it with non-us-east-1 buckets.
2018-03-30 02:31:14 +01:00
Eelco Dolstra
446bb88f13
Remove reference to non-existent manpages
2018-03-29 12:29:10 +02:00
Eelco Dolstra
2b61c74922
Process --option use-case-hack properly
...
Fixes https://github.com/NixOS/nix/issues/2009 .
2018-03-29 12:28:29 +02:00
Eelco Dolstra
9d1220a01d
ValidPathInfo::isContentAddressed(): Ensure there are no references
2018-03-29 12:27:42 +02:00
Will Dietz
74da813912
download: improve error for hash mismatch ("store mismatch")
...
Fixes #1905
2018-03-22 15:22:06 -05:00
Will Dietz
59b32403f2
download.cc: fix path for cached files
2018-03-22 15:05:46 -05:00
Eelco Dolstra
28eb9621cc
Fix build
2018-03-22 13:46:19 +01:00
Eelco Dolstra
dce8af59a6
Tweak
2018-03-22 13:32:18 +01:00
Eelco Dolstra
f87e286e82
Merge branch 'fix/avoid-large-stack-buffers' of https://github.com/dtzWill/nix
2018-03-22 13:19:25 +01:00
Shea Levy
c24e0439b2
Merge branch 'fix/execl-null-sentinel' of git://github.com/dtzWill/nix
2018-03-21 23:25:53 -04:00
Eelco Dolstra
ec91840e70
Also make the backwards compatible case of RemoteStore::addToStore run in constant memory
...
Just because we can.
2018-03-21 23:46:03 +01:00
Eelco Dolstra
97002b684c
Make 'nix copy --to daemon' run in constant memory
2018-03-21 23:42:52 +01:00
Will Dietz
f66ae22288
shared.cc: style: prefer "nullptr" over '(char *)NULL'
2018-03-21 17:17:37 -05:00
Eelco Dolstra
11898d6a37
Make 'nix copy --to ssh://...' run in constant memory
2018-03-21 23:13:06 +01:00
