Commit graph

10822 commits

Author SHA1 Message Date
Eelco Dolstra
0b606aad46
Add automatic garbage collection
Nix can now automatically run the garbage collector during builds or
while adding paths to the store. The option "min-free = <bytes>"
specifies that Nix should run the garbage collector whenever free
space in the Nix store drops below <bytes>. It will then delete
garbage until "max-free" bytes are available.

Garbage collection during builds is asynchronous; running builds are
not paused and new builds are not blocked. However, there also is a
synchronous GC run prior to the first build/substitution.

Currently, no old GC roots are deleted (as in "nix-collect-garbage
-d").
2017-09-05 20:43:42 +02:00
Eelco Dolstra
b932ea58ec
GC: Don't delete own temproots file
Since file locks are per-process rather than per-file-descriptor, the
garbage collector would always acquire a lock on its own temproots
file and conclude that it's stale.
2017-09-05 20:39:57 +02:00
Eelco Dolstra
8215b75d36
Abort curl downloads if there is no progress for 5 minutes
Maybe this will fix the curl hangs on macOS. (We could also use
CURLOPT_TIMEOUT but that seems more of a sledgehammer.)
2017-09-01 16:51:26 +02:00
Eelco Dolstra
7a108d904e
Fix verbosity level for nix build --dry-run 2017-09-01 12:37:09 +02:00
Eelco Dolstra
fe38fce2d8 Merge pull request #1545 from trofi/glibc-2.26
src/libmain/stack.cc: fix 'ucontext' usage on glibc-2.26
2017-09-01 11:12:30 +02:00
Sergei Trofimovich
c9857ef262 src/libmain/stack.cc: fix 'ucontext' usage on glibc-2.26
Build fails as:

$ make
  CXX    src/libmain/stack.o
src/libmain/stack.cc: In function 'void nix::sigsegvHandler(int, siginfo_t*, void*)':
src/libmain/stack.cc:21:21: error: 'ucontext' was not declared in this scope
     sp = (char *) ((ucontext *) ctx)->uc_mcontext.gregs[REG_RSP];
                     ^~~~~~~~
src/libmain/stack.cc:21:21: note: suggested alternative: 'ucontext_t'
     sp = (char *) ((ucontext *) ctx)->uc_mcontext.gregs[REG_RSP];
                     ^~~~~~~~
                     ucontext_t

It's caused by upstream rename:
https://sourceware.org/git/?p=glibc.git;a=commitdiff;h=251287734e89a52da3db682a8241eb6bccc050c9

which basically changes
    typedef struct ucontext {} ucontext_t;
to
    typedef struct ucontext_t {} ucontext_t;

The change uses ucontext_t.

Signed-off-by: Sergei Trofimovich <slyfox@gentoo.org>
2017-08-31 21:41:36 +01:00
Vincent Ambo
b20bc5f57a fix templater: Don't template default.yml files
After the change from #84 default variable files with the '.yml'
extension got templated as resource set templates accidentally.

This resolves the issue by moving the list reserved default file names
to a common place and reusing it in both the templater and context pkg.

This fixes #85
2017-08-31 18:41:57 +02:00
Eelco Dolstra
bbdf08bc0f
Call queryMissing() prior to building
Without this, substitute info is fetched sequentially, which is
superslow. In the old UI (e.g. nix-build), we call printMissing(),
which calls queryMissing(), thereby preheating the binary cache
cache. But the new UI doesn't do that.
2017-08-31 16:05:07 +02:00
Eelco Dolstra
fd73c1e20a
Add an activity for binary cache queries 2017-08-31 15:25:58 +02:00
Eelco Dolstra
c2154d4c84
Rename a few configuration options
In particular, drop the "build-" and "gc-" prefixes which are
pointless. So now you can say

  nix build --no-sandbox

instead of

  nix build --no-build-use-sandbox
2017-08-31 14:28:25 +02:00
Eelco Dolstra
7d4a7136db
More macOS build fixes 2017-08-31 12:52:07 +02:00
Eelco Dolstra
fabde432dc
Fix build failure on non-Linux
https://hydra.nixos.org/build/59649086
2017-08-31 11:05:18 +02:00
Eelco Dolstra
9f47eac92b
Fix mismatched tag warning
https://hydra.nixos.org/build/59649086
2017-08-31 11:04:52 +02:00
Eelco Dolstra
e9c07a3b26
nix edit / log: Operate on a single Installable 2017-08-29 16:18:00 +02:00
Eelco Dolstra
9b82ecbae0
nix search: Warn about cached results 2017-08-29 15:22:05 +02:00
Eelco Dolstra
c8235c5313
nix run: Flush the progress bar before starting the command 2017-08-29 15:13:30 +02:00
Eelco Dolstra
05d68a6e23
nix run: Add some flags for clearing/keeping the environment
This is useful for testing commands in isolation.

For example,

  $ nix run nixpkgs.geeqie -i -k DISPLAY -k XAUTHORITY -c geeqie

runs geeqie in an empty environment, except for $DISPLAY and
$XAUTHORITY.
2017-08-29 15:00:08 +02:00
Eelco Dolstra
5cc8609e30
nix run: Allow passing a command to execute
E.g.

  nix run nixpkgs.hello -c hello --greeting Hallo

Note that unlike "nix-shell --command", no quoting of arguments is
necessary.

"-c" (short for "--command") cannot be combined with "--" because they
both consume all remaining arguments. But since installables shouldn't
start with a dash, this is unlikely to cause problems.
2017-08-29 14:42:48 +02:00
Eelco Dolstra
93a5ef0516
nix run: Fix chroot execution
Running "nix run" with a diverted store, e.g.

  $ nix run --store local?root=/tmp/nix nixpkgs.hello

stopped working when Nix became multithreaded, because
unshare(CLONE_NEWUSER) doesn't work in multithreaded processes. The
obvious solution is to terminate all other threads first, but 1) there
is no way to terminate Boehm GC marker threads; and 2) it appears that
the kernel has a race where unshare(CLONE_NEWUSER) will still fail for
some indeterminate amount of time after joining other threads.

So instead, "nix run" will now exec() a single-threaded helper ("nix
__run_in_chroot") that performs the actual unshare()/chroot()/exec().
2017-08-29 13:21:07 +02:00
Eelco Dolstra
1c58e13bee
Hide commands that don't have a description
These are assumed to be internal.
2017-08-29 11:52:55 +02:00
Eelco Dolstra
05c45f301d
readLink(): Handle symlinks in /proc
Symlinks like /proc/self/exe report a stat() size of 0, so use a
buffer of at least PATH_MAX instead.
2017-08-29 11:52:34 +02:00
Eelco Dolstra
2cc345b95f
Give activities a verbosity level again
And print them (separately from the progress bar) given sufficient -v
flags.
2017-08-28 19:13:24 +02:00
Eelco Dolstra
cfc8132391
Don't send progress messages to older clients 2017-08-28 18:54:23 +02:00
Eelco Dolstra
fe34b91289
Tunnel progress messages from the daemon to the client
This makes the progress bar work for non-root users.
2017-08-28 18:49:42 +02:00
Eelco Dolstra
e681b1f064
Simplify 2017-08-28 14:30:35 +02:00
Eelco Dolstra
8fff3e7bb5
Make TunnelLogger thread-safe
Now that we use threads in lots of places, it's possible for
TunnelLogger::log() to be called asynchronously from other threads
than the main loop. So we need to ensure that STDERR_NEXT messages
don't clobber other messages.
2017-08-28 14:17:07 +02:00
Eelco Dolstra
94a0548dc4
Simplify 2017-08-25 21:26:37 +02:00
Eelco Dolstra
9b845e6936
Doh 2017-08-25 20:52:34 +02:00
Eelco Dolstra
0ac35b67b8
Allow derivations to update the build phase
So the progress bar can show

  [1/0/1 built, 0.0 MiB DL] building hello-2.10 (configuring): checking whether pread is declared without a macro... yes
2017-08-25 18:04:05 +02:00
Eelco Dolstra
c137c0a5eb
Allow activities to be nested
In particular, this allows more relevant activities ("substituting X")
to supersede inferior ones ("downloading X").
2017-08-25 17:49:40 +02:00
Eelco Dolstra
f194629f96
Fix Debian build
https://hydra.nixos.org/build/59390148
2017-08-25 16:11:18 +02:00
Eelco Dolstra
db1d45037c
Handle SIGWINCH 2017-08-25 15:59:03 +02:00
Eelco Dolstra
ec9e0c03c3
When truncating the progress bar, take ANSI escape sequences into account 2017-08-25 15:59:03 +02:00
Eelco Dolstra
a3015db6c3
Typo 2017-08-25 15:59:03 +02:00
Eelco Dolstra
0e9ddcc306
Restore activity metadata
This allows the progress bar to display "building perl-5.22.3" instead
of "building /nix/store/<hash>-perl-5.22.3.drv".
2017-08-25 15:58:35 +02:00
Vincent Ambo
063a3e9d30 fix context: Support ".yml" extension on default files
In other places in Kontemplate that deal with YAML files, both the
".yaml" and ".yml" extension are supported.

This fixes context loading code to support that for resource set
variables, too.

This fixes #83
2017-08-25 14:44:24 +02:00
Vincent Ambo
8ccaab322a chore brew: Update Homebrew formula to v1.2.0 2017-08-22 20:02:10 +02:00
Vincent Ambo
4c7ff46c73 chore image: Bump Docker image to build v1.2.0 2017-08-22 20:02:10 +02:00
Vincent Ambo
f8b6ad652d
Version 1.2.0
This release comes with some new features, usability improvements and
a better build & release process.

Features:

* Documentation has been improved significantly, check out the new
  [README][] and follow the links within!
* Extra variables can now be loaded from files on disk. Simply specify
  a list of YAML/JSON files under the 'import' key in your cluster
  context file. Check out #66 for details!
* Resource set paths can now be overridden by users. By default it is
  assumed that the path to a resource set is the same as its name,
  however this is now user-controllable.
  This means the same resource set can be included multiple times
  under different names, for easier including/excluding. See #71 for
  details!
* Kontemplate is currently getting a website that is under construction
  at [kontemplate.works][] - feel free to check it out and
  [give feedback][]!

Fixes:

* Windows release binaries now have the correct filename
* Several potential warning and error messages have been improved

Release binaries are signed with GPG key `66F505681DB8F43B` which is
verified on my Github profile.

[README]: https://github.com/tazjin/kontemplate/blob/master/README.md
[kontemplate.works]: http://kontemplate.works/
[give feedback]: https://github.com/tazjin/kontemplate-website/issues
2017-08-22 19:18:14 +02:00
Vincent Ambo
d22b3694fe feat main: Warn if resource set contains no templates
If a resource set is specified by the user and does _not_ contain any
templates, a warning will now be printed.

This fixes #79
2017-08-22 19:08:29 +02:00
Vincent Ambo
825506d2e9 chore build: Bump dependency versions
Several bugfixes, nothing major. Skipped one "outdated" dependency
because the only change was a new text file.
2017-08-22 18:53:48 +02:00
Vincent Ambo
e8cfa9c119 refactor build: Keep GPG-signatures outside of tarballs
Instead of signing the binary and adding the signature in the release
tarball, keep the GPG-signatures *outside* of the tarball.

This makes it easier to use the built-in GPG-signature verification
features of package managers such as pacman.
2017-08-22 18:41:14 +02:00
Vincent Ambo
a9c450c5a3 fix build: Build Windows executable with correct name
Windows executable filenames must end in ".exe" because the operating
system can't execute them otherwise.

This fixes #73
2017-08-22 18:41:14 +02:00
Eelco Dolstra
1f56235438
Clean up JSON construction 2017-08-21 12:18:46 +02:00
Eelco Dolstra
4af2611bd1
Allow builders to create activities
Actually, currently they can only create download activities. Thus,
downloads by builtins.fetchurl show up in the progress bar.
2017-08-21 12:18:46 +02:00
Eelco Dolstra
4c6a26539c
Remove debug line 2017-08-21 12:18:46 +02:00
Eelco Dolstra
9400cb36b7
Disallow accidental copy construction 2017-08-21 12:18:42 +02:00
Eelco Dolstra
37db080644 Merge pull request #1529 from matthewbauer/remove-nix-mode
Remove nix-mode.el from Nix.
2017-08-21 10:41:21 +02:00
Matthew Bauer
2c75945de5
Remove nix-mode.el from Nix.
This removes the file nix-mode.el from Nix. The file is now available within the
repository https://github.com/NixOS/nix-mode.

Fixes #662
Fixes #1040
Fixes #1054
Fixes #1055
Closes #1119
Fixes #1419

NOTE: all of the above should be fixed within NixOS/nix-mode. If one of those
hasn’t please reopen within NixOS/nix-mode and not within NixOS/nix.
2017-08-19 21:16:30 -07:00
Domen Kožar
898a3f729c Merge pull request #1527 from chaseadamsio/master
update MD5 to SHA-256 in expression-syntax
2017-08-19 14:02:06 +02:00