ecoppens/colmena
1
0
Fork 0
forked from DGNum/colmena
Code Pull requests Activity
352 commits 2 branches 0 tags 994 KiB
Commit graph

196 commits

Author SHA1 Message Date
Zhaofeng Li
abb74b9f49 Add EditorConfig rules, fix trailing whitespaces 2022-01-08 01:20:36 -08:00
Zhaofeng Li
deca292b53 Refactor NixOptions 2022-01-08 01:20:36 -08:00
Zhaofeng Li
31fd1e49ac Move nix::{NixResult, NixError} to error::{ColmenaResult, ColmenaError} 2022-01-08 01:20:36 -08:00
Zhaofeng Li
16ed9d8c66 Move nix::NixCommand to util::CommandExt 2022-01-08 01:20:36 -08:00
Zhaofeng Li
f92236da46 Refactor current profile detection 2022-01-08 01:20:36 -08:00
Zhaofeng Li
6401ce4c3c cli: Remove target from logs 
format_target is enabled by default in env_logger 0.9.0. It makes
the logs look cluttered and is not very useful for our use case.
 2022-01-05 14:01:05 -08:00
Zhaofeng Li
da7907c874 General code cleanup 2022-01-05 14:01:05 -08:00
Zhaofeng Li
f234e16e80 manual: Fix colorized CLI help 
In clap 3.0, it's no longer possible to get `App::write_long_help` to
output colorized text (it doesn't invoke the Colorizer at all). So let's
move the generation outside of Rust.
 2022-01-03 10:37:03 -08:00
Zhaofeng Li
d2762757f0 command/eval: Fix introspect alias 
On clap 3.0, resetting the App::name still causes clap to complain
that the subcommand doesn't exist in debug mode.
 2022-01-03 10:37:03 -08:00
Zhaofeng Li
79d1010356 command: Change wording for consistency 2022-01-03 10:37:03 -08:00
Zhaofeng Li
81e2982bbb Migrate to clap 0.3 2022-01-03 10:37:03 -08:00
Zhaofeng Li
f9fab83030 deployment: Remove useless Arc clones 2022-01-02 13:16:24 -08:00
Zhaofeng Li
7bf57fd1ad nix/node_filter: Forgot to add 2022-01-01 16:47:38 -08:00
Zhaofeng Li
98897bf4de Support building on target nodes 
This partially addresses #33, and allows Colmena to be used more
easily on bandwidth-constrained hosts and macOS.

With `deployment.buildOnTarget = true;` deployment works fine from
macOS without designated builders, except when IFD is involved.
 2022-01-01 16:41:35 -08:00
Zhaofeng Li
872f944743 eval.nix: Skip recursive type checking when cross-referencing configs through the nodes argument 
This makes evaluation faster especially when `nodes` is heavily used in
the configuration (like in my auto-meshing setup). This matches the
behavior of Morph.
 2022-01-01 16:41:35 -08:00
Zhaofeng Li
cdbb69617f eval.nix: Support specifying a list of configs 2022-01-01 16:41:35 -08:00
Zhaofeng Li
bc3450b3ce cli: Disable apply-local on non-Linux platforms 2022-01-01 16:41:35 -08:00
Zhaofeng Li
f9f4dd5f63 Remove another ugly toJSON hack 2022-01-01 16:41:35 -08:00
Zhaofeng Li
111bc9dea7 Remove test-progress subcommand from release builds 2022-01-01 16:41:35 -08:00
Zhaofeng Li
f809d3b21c nix/host: Always copy outputs to remote along with derivations when realizing 
This prevents useless rebuilds when trying to realize a derivation on
a remote host. This code path isn't actually used by Colmena at the
moment.
 2022-01-01 16:41:35 -08:00
Zhaofeng Li
2b652f7236 hive: Add another test for nixpkgs.system 2022-01-01 16:41:35 -08:00
Zhaofeng Li
8f77184d58 eval.nix: Inherit pkgs.system in evalConfig 2021-12-18 14:35:06 -08:00
Zhaofeng Li
bd4493da73 nix: Move eval.nix tests into /hive 2021-12-18 14:35:06 -08:00
Zhaofeng Li
31fc98cfa7 nix: Remove ugly toJSON hack 
We now evaluate with --strict.
 2021-12-07 23:13:31 -08:00
Zhaofeng Li
eebded1786 Build each node individually 
Now nodes that take a long time to build won't bottleneck the
deployment of other nodes in the same chunk.

Fixes #47.
 2021-12-07 23:13:31 -08:00
Zhaofeng Li
39f597f778 hive: Pass --read-write-mode to nix-instantiate --eval 
This enables the use of IFD inside `meta` and node `deployment.*`
options, from which we obtain the values with `nix-instantiate --eval`.

Fixes #45.
 2021-12-05 12:34:44 -08:00
Zhaofeng Li
7433661aed Add deployment.keys.<name>.name 2021-12-05 01:14:12 -08:00
Zhaofeng Li
0f8873027f utils: Don't panic in capture_stream 2021-12-05 01:14:12 -08:00
Zhaofeng Li
ddccad4fb9 eval.nix: Auto-call functors as well 
This makes some very cursed setup work :P
 2021-12-04 01:03:26 -08:00
Zhaofeng Li
7757a0e45d command/apply: Use verbose output when goal is "dry-activate" 
It doesn't make sense to use the spinners since you can't see
the full output from the activation script.

Fixes #44.
 2021-12-04 01:03:26 -08:00
Zhaofeng Li
7b1feffb9e util: Log the configuration in use 2021-12-04 01:03:26 -08:00
Zhaofeng Li
9cd2af9438 cli: Re-order arguments in help message 2021-12-04 01:03:26 -08:00
Zhaofeng Li
f253e6eb18 cli: Allow configuring output colorization 
We now follow the <https://bixense.com/clicolors> standard.
 2021-12-04 01:03:26 -08:00
Zhaofeng Li
dc57b489ea progress: Restore tty-detection after refactoring 2021-12-04 01:03:26 -08:00
Zhaofeng Li
064432f38e nix/deployment: Misc cleanup 2021-12-04 01:03:26 -08:00
Zhaofeng Li
04976e5fda job: Make event formatting less ugly 2021-12-03 00:06:21 -08:00
Zhaofeng Li
a3e292d7fb nix: Another deployment parallelization oops 2021-12-03 00:06:17 -08:00
Zhaofeng Li
2bba64a002 nix: Fix deployment parallelization 2021-12-02 23:49:46 -08:00
Zhaofeng Li
a42e8c5bf0 Misc cleanup 2021-11-23 14:12:01 -08:00
Zhaofeng Li
ec51f5703f deployment: A few oops 2021-11-22 01:53:08 -08:00
Zhaofeng Li
572f77dbf5 A couple of fixes to progress output 
Need to have some integration tests.
 2021-11-21 16:02:45 -08:00
Zhaofeng Li
b3016ad3a7 progress/plain: Fix label alignment 2021-11-21 16:02:45 -08:00
Zhaofeng Li
46090598eb apply-local: Fix output after rewrite 2021-11-21 15:30:52 -08:00
Zhaofeng Li
0cb3f8e968 Redesign deployment process (again) 
We now ship Events from different parts of the deployment process
via a channel to a job monitor.
 2021-11-21 00:11:37 -08:00
Zhaofeng Li
5c84134af3 Refactor node names 2021-11-20 23:34:52 -08:00
Zhaofeng Li
47ccc6c9c4 nix/info.rs: Update message about nixUnstable for 21.11 2021-11-18 15:41:02 -08:00
Zhaofeng Li
c271780b63 Improve error reporting 2021-11-18 13:15:20 -08:00
Zhaofeng Li
fae58994e4 Prepare for 0.2.0 release 2021-11-18 13:15:20 -08:00
Zhaofeng Li
e5665775b2 Preserve elapsed time in store path listing after build 
Fixes #36.
 2021-11-18 01:38:58 -08:00
Zhaofeng Li
80a71d9023 cli.rs: Point user at manual 2021-11-17 22:21:00 -08:00
Zhaofeng Li
4497ef296e Add manual 
We finally have some real documentation :)
 2021-11-17 22:21:00 -08:00
Zhaofeng Li
c42c97d2f6 Add internal command to render CLI usage text for manual 2021-11-17 22:21:00 -08:00
Zhaofeng Li
1535857acc Also disallow pinning to a Nixpkgs lambda in Flakes 
Somehow missed this one *shrugs*
 2021-11-17 22:21:00 -08:00
Zhaofeng Li
006cb2c5ee eval.nix: Make the uninitialized nixpkgs error more informative 2021-11-16 21:01:40 -08:00
Zhaofeng Li
f716daa3a1 Migrate to indicatif 0.7 beta 
Now there is no need for the ugly hack where we drove the spinner
in a separate thread :)
 2021-11-16 21:01:33 -08:00
Zhaofeng Li
7d15d08d6d Add test command for progress spinners 2021-11-16 21:01:33 -08:00
Zhaofeng Li
86eeeece3c command: Rename introspect to eval 2021-10-28 17:27:30 -07:00
Zhaofeng Li
f7eb121260 Disallow uninitialized meta.nixpkgs in Flakes 2021-10-28 17:10:58 -07:00
Zhaofeng Li
765f42fa24 introspect: Support actually instantiating the expression 2021-10-28 14:09:35 -07:00
Zhaofeng Li
58f2bf391f src/util.rs: No need to list 2021-10-28 14:09:35 -07:00
Zhaofeng Li
0e0a1e84f0 Make flake resolution (slightly) less terrible 
Instead of using `path:` which always copies the entire directory,
we now try to resolve the Flake URI using `nix flake metadata` which
may give us a `git+file:`.
 2021-10-25 23:38:10 -07:00
Zhaofeng Li
b48753239a hive.rs: Canonicalize flake path 
Relative paths are no longer allowed in newer Nix versions.
 2021-10-25 21:53:38 -07:00
Zhaofeng Li
6d6e33fcd4 nix: Remove unneeded ok() 2021-10-23 20:49:14 -07:00
Bjørn Forsman
4106a73e75 Allow selecting ssh user dynamically 
...by setting `deployment.targetUser = null`.

This allows sharing a deployment file (hive.nix/flake.nix) between
multiple admins, without having to use a shared root account.
 2021-10-23 15:06:56 +02:00
Zhaofeng Li
37b43cd6d7 eval.nix: Support autocall for hive configuration 2021-08-26 19:59:22 -07:00
Zhaofeng Li
7cc6552ee3 hive.rs: Remove unwrap in builder_args() 2021-08-26 19:59:22 -07:00
Zhaofeng Li
7b69946d98 Ensure key ownerships are set correctly 
Depending on when keys are uploaded (`deployment.keys.<name>.uploadAt`):

`pre-activation`:
We set the ownerships in the uploader script opportunistically and
continue if the user/group does not exist. Then, in the activation
script, we set the ownerships of all pre-activation keys.

`post-activation`:
We set the ownerships in the uploader script and fail if the
user/group does not exist.

The ownerships will be correct regardless of which mode is in use.

Fixes #23. Also a more complete solution to #10.
 2021-08-26 12:54:41 -07:00
Zhaofeng Li
24339bcca7 Add deployment.keys.<name>.uploadAt 
This mirrors the functionality recently added in morph and allows
for the uploading of keys after system profile activation.

Fixes #10.
 2021-08-24 23:25:46 -07:00
Zhaofeng Li
135a42b20f eval.nix: Add meta.specialArgs 2021-07-16 22:52:23 -07:00
Zhaofeng Li
671cf38796 hive.rs: Pass --builders to nix-instantiate as well 2021-07-13 01:38:52 -07:00
Zhaofeng Li
c644f79ad1 cli.rs: Indicate support for flakes 2021-06-29 01:19:13 -07:00
Zhaofeng Li
67db0e73d1 Add check for Flakes support 2021-06-29 01:02:43 -07:00
Zhaofeng Li
e50ba82bf2 Add basic Flakes support 
Co-authored-by: Alex Zero <joseph@marsden.space>
 2021-06-29 01:02:43 -07:00
Zhaofeng Li
22ae18f5e7 Exit with non-zero code if any node fails to deploy 
The exit codes are in flux and should not be relied upon.

Fixes #28.
 2021-05-24 00:15:38 -07:00
Zhaofeng Li
960af8f793 Add deployment.privilegeEscalationCommand 
This adds a NixOps-equivalent option for non-root deployment
on remote hosts.

Fixes #27.
 2021-05-24 00:15:38 -07:00
Zhaofeng Li
39d612a5e7 ssh: Remove dead code 2021-05-24 00:15:38 -07:00
Zhaofeng Li
99ba8db335
Merge pull request #21 from jasonrm/machines-file 
eval.nix: Adds meta.machinesFile option that is passed to Nix as builder option
 2021-05-07 16:25:36 -07:00
Zhaofeng Li
16ccdbc700 Better handling of killed processes 2021-04-28 15:09:40 -07:00
Zhaofeng Li
44b421c2c7 key.rs: Fix typo (user -> group) 
Fixes #22.
 2021-04-19 15:40:19 -07:00
Jason R. McNeil
3ee97c2a76 apply: Add deployment.replaceUnknownProfiles option and --force-replace-unknown-profiles switch 
If `deployment.replaceUnknownProfiles` is set to false, a diverged hive
config (in a shared git repo for example) won't result in accidentally
undoing another applied configuration profile.

The deployment option is set to true so that fiction is minimized from
aggressive garbage collection, first time profile application and low
contention hives.
 2021-04-10 13:42:38 -07:00
Jason R. McNeil
e0465567b2 eval.nix: Adds meta.machinesFile option that is passed to Nix as builders argument 2021-04-09 23:54:13 -07:00
Zhaofeng Li
0927fe9dc1 cli: Add hidden command to generate shell autocompletion 2021-03-23 14:14:04 -07:00
Zhaofeng Li
53b55a102e cli: Set bin_name to be lower case 2021-03-23 14:14:04 -07:00
Zhaofeng Li
ba2574755a Separate global CLI setup into module 2021-03-23 14:14:04 -07:00
Zhaofeng Li
8abcd5d53b "Successfully built" -> "Build successful" for consistency 2021-03-18 15:13:34 -07:00
Zhaofeng Li
b44dd1f877 apply_local: Don't bother evaluating other hosts 2021-03-18 15:05:05 -07:00
Zhaofeng Li
e9487ced9e host: Use the key uploader script for both SSH and local 2021-03-17 22:39:05 -07:00
Zhaofeng Li
29cfd45141 Miscellaneous doc fixes 2021-03-17 19:07:26 -07:00
Zhaofeng Li
610a725ba2 Add --keep-result to create GC roots for profiles 
This resembles the behavior of morph.

Reference: #18
 2021-03-17 14:59:57 -07:00
Zhaofeng Li
81375e71b2 deployment: Display the resulting paths if the goal is to build only 
Reference: #18
 2021-03-17 14:59:43 -07:00
Zhaofeng Li
180d2f2435
Merge pull request #17 from jasonrm/nix-copy-ssh-options 
Makes SSH options available to nix-copy-closure
 2021-03-14 23:06:04 -07:00
Jason R. McNeil
4098bf73bc Makes SSH options available to nix-copy-closure 2021-03-14 22:20:47 -07:00
Zhaofeng Li
fb5ff6f9c9 Allow specifying alternative commands for privilege escalation 
Fixes #16.
 2021-03-10 08:42:51 -08:00
Zhaofeng Li
082a033443 eval.nix: Exclude internal Nixpkgs config options from node override warning 2021-02-17 23:06:22 -08:00
Zhaofeng Li
3b005b0949 src/progress.rs: Oops, forgot to add 2021-02-17 22:56:34 -08:00
Zhaofeng Li
e32e130621 Always print the entire log for failures in eval and build 
This makes it easier to debug configuration issues without -v.

Fixes #14.
 2021-02-17 22:48:26 -08:00
Zhaofeng Li
d16a13654c Merge nixpkgs.config and nixpkgs.overlays 
This replaces #12, and allows for Nixpkgs overlays and config to be overridden
in machine configs. With #12, overlays set in machine configurations
(`nixpkgs.overlays`) get silently ignored.
 2021-02-17 22:46:01 -08:00
Zhaofeng Li
9eae937b42 apply: Disable configuration of build process limit 2021-02-17 08:09:15 -08:00
Justinas Stankevicius
10f98d715f Propagate same 'pkgs' to all modules 2021-02-16 20:53:43 +02:00
Zhaofeng Li
6a9282e121 exec: Remove outdated doc 2021-02-12 16:21:38 -08:00