ecoppens/colmena
1
0
Fork 0
forked from DGNum/colmena
Code Pull requests Activity
487 commits 2 branches 0 tags 994 KiB
Commit graph

261 commits

Author SHA1 Message Date
Zhaofeng Li
81bda9d595 Update deps 2022-08-16 20:16:00 -06:00
Zhaofeng Li
68d0036a2c Thanks clippy 2022-08-16 20:16:00 -06:00
Zhaofeng Li
62a3d1e6f8 Run rustfmt 2022-07-29 22:13:09 -07:00
Zhaofeng Li
7d0a6eed90 Apply quit::main to main function 
This makes `quit::with_code` actually work with the correct exit code.

Fixes #111.
 2022-07-17 14:42:34 -07:00
David Arnold
b3ca4b1c4d
ref: transparent eval.nix contract 
- at least to my eyes, this code formatting gives a better view about
  the gist of the eval.nix interface
 2022-07-09 16:04:23 -05:00
David Arnold
e309b2d4b2
ref: reduce eval.nix contract even further 
- meta and evalAll contracts are found to be not consumed
 2022-07-09 15:41:17 -05:00
David Arnold
83a919bdb0
ref: reduce eval.nix contract forther 
- module export contratcts can now bypass the eval.nix iface
- enabled by:9bd5e7bb2540664fe1c728c85f80375fe0cbf9fa
 2022-07-09 15:41:17 -05:00
David Arnold
c82517fc12
ref: reduce eval.nix contract 
- documentation contratcts can now bypass the eval.nix iface
- enabled by:9bd5e7bb2540664fe1c728c85f80375fe0cbf9fa
 2022-07-09 15:41:15 -05:00
Zhaofeng Li
b8bdf5824c hive: Use absolute paths for non-flake hives 
Fixes #107.
 2022-07-04 14:05:50 -07:00
Zhaofeng Li
ea1e7684f1 hive: Add tests for nodeSpecialArgs 2022-07-01 16:31:13 -07:00
Zhaofeng Li
1b045eb38d
Merge pull request #100 from fooker/pr/specialArgs 
eval.nix: per-node specialArgs
 2022-07-01 16:29:59 -07:00
NeverBehave
dfe55ed461 fix: rename no-substitutes -> no-substitute 
Fixes #59.
 2022-07-01 16:06:01 -07:00
Zhaofeng Li
432d4212fc apply: Don't enforce meta.allowApplyAll for builds 
Fixes #99.
 2022-06-24 17:34:37 -07:00
Zhaofeng Li
0209b6dc40 Add name of key to key upload errors 
> example | Key upload failed: Error processing key "meow": I/O Error: No such file or directory (os error 2)

Fixes #101.
 2022-06-24 17:34:37 -07:00
Dustin Frisch
07499a1eaf
eval.nix: per-node specialArgs 2022-06-23 12:09:29 +02:00
NeverBehave
dd7a2924ca feat(meta): add disallowApplyAll options 
unify meta access for machinesFile

update release note and config name
 2022-06-20 18:47:53 -07:00
Zhaofeng Li
dd3f4b252c host/local: Upload keys with sudo as well 2022-06-10 11:29:25 -07:00
Zhaofeng Li
d7aa4ab864 hive/tests: Fix editorconfig 2022-06-09 11:35:02 -07:00
Zhaofeng Li
9db25bd914 deployment: Move GC root creation after build 
Previously GC roots were only created after successful activation.
 2022-06-09 11:31:49 -07:00
Zhaofeng Li
2b0bafa155 eval.nix: Restore signature in introspect 
It was broken in 9bd5e7bb25.
 2022-06-09 11:31:49 -07:00
Zhaofeng Li
b5629dca83 eval.nix: Enable config merging for meta.nixpkgs on 21.11+ 
Ref: #74 #15
 2022-06-03 23:51:32 -07:00
Zhaofeng Li
1cd3aa6401 eval.nix: Accept path-likes in meta.nixpkgs as well 
This fixes usage with Niv.
 2022-06-03 23:51:32 -07:00
Zhaofeng Li
b87da69558 Don't import impure overlays if meta.nixpkgs is a path 
Also update documentations to recommend specifying `overlays = []`
explicitly.

Fixes #39.
 2022-06-03 23:51:32 -07:00
Zhaofeng Li
ca12be27ed apply-local: Escalate privileges only during activation 
Fixes #85.
 2022-06-03 23:51:32 -07:00
Zhaofeng Li
fa07814abf hive/assets.rs: Use OpenOptions::new() instead 2022-06-03 23:51:32 -07:00
Zhaofeng Li
4037b61a2b apply: Fix typo in help 2022-06-03 23:51:32 -07:00
Zhaofeng Li
af3247faeb Update docs after #91 2022-06-03 23:51:32 -07:00
Zhaofeng Li
ed91898e59 Clippy 2022-06-03 23:51:32 -07:00
Zhaofeng Li
ac8852dcdf hive: Refactor static file creation 2022-06-02 16:39:45 -07:00
Wanja Hentze
6fe48ccd04 nix-eval-jobs: Strip quotes surrounding attribute names 
Fixes #92.
 2022-06-02 16:39:45 -07:00
Victor Nawothnig
e937eb8faa Do not use an explicit user for ssh when deploymentUser is null 2022-06-01 12:30:49 +02:00
Zhaofeng Li
429a0f5aa1 eval.nix: Suppress _module.args docs 2022-05-22 02:15:33 -07:00
Zhaofeng Li
1be16d552d apply: Add --reboot flag 
Fixes #46.
 2022-05-22 02:15:33 -07:00
Zhaofeng Li
e239cbd260 host: Add get_current_system_profile 2022-05-22 02:15:33 -07:00
Zhaofeng Li
c8b79e7e2d ssh: Clean up 2022-05-22 02:15:33 -07:00
David Arnold
9bd5e7bb25 imp: soc of eval, modules & options 
- the reason for this change is to have more transparent separation
  of concern between effectuations of the module system and pre-module
  system effectuations

- with improved flakes support down the line, pre-module system
  effectuations will get more complex

- this also allows to patch the aspects of the evaluation individually
  while tracking other components from upstream. eg. path options & eval
  but not modules
 2022-05-22 01:54:52 -07:00
Zhaofeng Li
ea4f2ba6dc Migrate other doc links to new domain 2022-05-13 19:14:58 -07:00
Zhaofeng Li
3e7341a98c utils.rs: Tolerate invalid UTF-8 in streams 
This will substitute any invalid UTF-8 sequence with `\u{fffd}`. Such
substitutions are okay for human-readable log outputs.

Correctness: A side-effect of this is that it also affects the code
that captures filesystem paths from stdout, in which case any substitution
is unacceptable. Currently we only capture store paths from Nix, and Nix
restricts characters allowed in derivation names. Nevertheless we will
refactor this for correctness.

Fixes #75.
 2022-04-25 00:03:57 -07:00
Zhaofeng Li
6a64625fda eval.nix: Ignore typed allowAliases config when checking unoverridden config keys 
This was added in NixOS/nixpkgs#166792.

Fixes #74.
 2022-04-25 00:03:57 -07:00
Zhaofeng Li
ab754ac2e3 Squelch dead code warnings on non-Linux platforms (aka macOS) 2022-04-19 00:29:35 -07:00
Zhaofeng Li
5a81996afb ssh: Enable BatchMode 2022-04-19 00:29:35 -07:00
Zhaofeng Li
26b2203da9 Update deps 2022-03-07 22:02:04 -08:00
Zhaofeng Li
108c135700 Rename: NixFailure -> ChildFailure, NixKilled -> ChildKilled 2022-03-07 22:02:04 -08:00
Glenn McDonald
9c179b0db8
apply_local: Fix nixos detection from os-release (#63) 2022-03-07 20:52:22 -08:00
Zhaofeng Li
2b281286d0
eval.nix: Add "${name}-key.service" units for keys (#51) 
Fixes #48.
 2022-02-16 10:23:27 -08:00
Zhaofeng Li
85b0107b59 eval.nix: Skip recursive type checking for nodes in colmena eval as well 
This matches the behavior of the `nodes` argument in NixOS modules.
 2022-02-16 01:11:06 -08:00
Linus Heckemann
efa7322587
streaming evaluator: respect eval-nodes-limit (#58) 2022-02-16 01:10:52 -08:00
Zhaofeng Li
c088925e0d Misc doc tweaks 2022-02-05 18:43:43 -08:00
Zhaofeng Li
45b135c1b1 nix_eval_jobs: Add timeouts to tests 
Sometimes nix-eval-jobs would lock up indefinitely, so let's make it
fail fast in CI.

Also ignore test_json_global_error while we investigate.
 2022-02-04 20:58:17 -08:00
Zhaofeng Li
856f82644e apply_local: Fix argument name typo 
Caught by debug assertions in clap 3.0. We should switch to the Derive
API where possible to prevent this from happening in the future.
 2022-01-27 18:48:25 -08:00
Zhaofeng Li
f674ddf173 key: Compute full path to key file in Nix 2022-01-25 14:22:26 -08:00
Zhaofeng Li
e58dde1be0 eval.nix: Don't override modulesPath 
`eval-config.nix` will set the canonical path for us that will then be used
to strip the module path prefixes in `nixos/modules/misc/documentation.nix` [1].
This should fix split docs build problems described in #50.

[1] 1511e72b75/nixos/modules/misc/documentation.nix (L79)
 2022-01-25 14:22:26 -08:00
Zhaofeng Li
7cd9b6f70c eval.nix: Don't pass baseModules to modules 
This is unnecessary and breaks the separation of user `modules`
and `baseModules` in `eval-config.nix`.
 2022-01-25 14:22:26 -08:00
Zhaofeng Li
9251019723 nix_eval_jobs: Errors may not have an attribute name attached 2022-01-25 14:22:26 -08:00
Zhaofeng Li
c0107b21e0 General code cleanup 2022-01-25 14:22:26 -08:00
Paul Haerle
5b8971a0f4
eval.nix: expose nixosModules.deploymentOptions (#49) 
Allow flake users to import .#nixosModules.deploymentOptions
into their flake, so that the same expression can be used for both,
.#colmena.$host as well as .#nixosConfiguration.$host, without the
latter complaining about undefined options in "deployment".
 2022-01-23 10:06:41 -08:00
Zhaofeng Li
0d9198c351 spinner: Work around the lack of newline after completion 2022-01-22 17:50:53 -08:00
Zhaofeng Li
87189ec3df nix_info: Display pinned nix-eval-jobs path 2022-01-22 17:50:53 -08:00
Zhaofeng Li
cf9a72a1d4 nix: Add initial nix-eval-job integration 
A DrvSetEvaluator is able to evaluate attribute sets of derivations,
streaming results as they come in.
 2022-01-22 17:50:53 -08:00
Zhaofeng Li
3e40e84e19 nix: Add NixExpression abstraction 
This decouples expression generation from evaluation. `NixExpression`s
that evaluate to a set of derivations can be fed to `DrvSetEvaluator`s
which may be able to parallelize evaluation.
 2022-01-22 17:50:53 -08:00
Zhaofeng Li
9fbb6b133b job: Make it possible to have a JobHandle not connected to a JobMonitor 2022-01-22 17:50:53 -08:00
Zhaofeng Li
55ce6d078e Remove unused stuff 2022-01-22 17:50:53 -08:00
Zhaofeng Li
d3e556027f deployment: Move chunked mode into self-contained function 2022-01-21 00:45:12 -08:00
Zhaofeng Li
82361e5ea5 hive: Clean up, make way for nix-eval-jobs 2022-01-21 00:45:12 -08:00
Zhaofeng Li
3fa0dee6b0 hive: Convert derivations to drvPath in separate attribute 
This is to make way for the nix-eval-jobs streaming evaluator.
 2022-01-21 00:45:12 -08:00
Zhaofeng Li
abb74b9f49 Add EditorConfig rules, fix trailing whitespaces 2022-01-08 01:20:36 -08:00
Zhaofeng Li
deca292b53 Refactor NixOptions 2022-01-08 01:20:36 -08:00
Zhaofeng Li
31fd1e49ac Move nix::{NixResult, NixError} to error::{ColmenaResult, ColmenaError} 2022-01-08 01:20:36 -08:00
Zhaofeng Li
16ed9d8c66 Move nix::NixCommand to util::CommandExt 2022-01-08 01:20:36 -08:00
Zhaofeng Li
f92236da46 Refactor current profile detection 2022-01-08 01:20:36 -08:00
Zhaofeng Li
6401ce4c3c cli: Remove target from logs 
format_target is enabled by default in env_logger 0.9.0. It makes
the logs look cluttered and is not very useful for our use case.
 2022-01-05 14:01:05 -08:00
Zhaofeng Li
da7907c874 General code cleanup 2022-01-05 14:01:05 -08:00
Zhaofeng Li
f234e16e80 manual: Fix colorized CLI help 
In clap 3.0, it's no longer possible to get `App::write_long_help` to
output colorized text (it doesn't invoke the Colorizer at all). So let's
move the generation outside of Rust.
 2022-01-03 10:37:03 -08:00
Zhaofeng Li
d2762757f0 command/eval: Fix introspect alias 
On clap 3.0, resetting the App::name still causes clap to complain
that the subcommand doesn't exist in debug mode.
 2022-01-03 10:37:03 -08:00
Zhaofeng Li
79d1010356 command: Change wording for consistency 2022-01-03 10:37:03 -08:00
Zhaofeng Li
81e2982bbb Migrate to clap 0.3 2022-01-03 10:37:03 -08:00
Zhaofeng Li
f9fab83030 deployment: Remove useless Arc clones 2022-01-02 13:16:24 -08:00
Zhaofeng Li
7bf57fd1ad nix/node_filter: Forgot to add 2022-01-01 16:47:38 -08:00
Zhaofeng Li
98897bf4de Support building on target nodes 
This partially addresses #33, and allows Colmena to be used more
easily on bandwidth-constrained hosts and macOS.

With `deployment.buildOnTarget = true;` deployment works fine from
macOS without designated builders, except when IFD is involved.
 2022-01-01 16:41:35 -08:00
Zhaofeng Li
872f944743 eval.nix: Skip recursive type checking when cross-referencing configs through the nodes argument 
This makes evaluation faster especially when `nodes` is heavily used in
the configuration (like in my auto-meshing setup). This matches the
behavior of Morph.
 2022-01-01 16:41:35 -08:00
Zhaofeng Li
cdbb69617f eval.nix: Support specifying a list of configs 2022-01-01 16:41:35 -08:00
Zhaofeng Li
bc3450b3ce cli: Disable apply-local on non-Linux platforms 2022-01-01 16:41:35 -08:00
Zhaofeng Li
f9f4dd5f63 Remove another ugly toJSON hack 2022-01-01 16:41:35 -08:00
Zhaofeng Li
111bc9dea7 Remove test-progress subcommand from release builds 2022-01-01 16:41:35 -08:00
Zhaofeng Li
f809d3b21c nix/host: Always copy outputs to remote along with derivations when realizing 
This prevents useless rebuilds when trying to realize a derivation on
a remote host. This code path isn't actually used by Colmena at the
moment.
 2022-01-01 16:41:35 -08:00
Zhaofeng Li
2b652f7236 hive: Add another test for nixpkgs.system 2022-01-01 16:41:35 -08:00
Zhaofeng Li
8f77184d58 eval.nix: Inherit pkgs.system in evalConfig 2021-12-18 14:35:06 -08:00
Zhaofeng Li
bd4493da73 nix: Move eval.nix tests into /hive 2021-12-18 14:35:06 -08:00
Zhaofeng Li
31fc98cfa7 nix: Remove ugly toJSON hack 
We now evaluate with --strict.
 2021-12-07 23:13:31 -08:00
Zhaofeng Li
eebded1786 Build each node individually 
Now nodes that take a long time to build won't bottleneck the
deployment of other nodes in the same chunk.

Fixes #47.
 2021-12-07 23:13:31 -08:00
Zhaofeng Li
39f597f778 hive: Pass --read-write-mode to nix-instantiate --eval 
This enables the use of IFD inside `meta` and node `deployment.*`
options, from which we obtain the values with `nix-instantiate --eval`.

Fixes #45.
 2021-12-05 12:34:44 -08:00
Zhaofeng Li
7433661aed Add deployment.keys.<name>.name 2021-12-05 01:14:12 -08:00
Zhaofeng Li
0f8873027f utils: Don't panic in capture_stream 2021-12-05 01:14:12 -08:00
Zhaofeng Li
ddccad4fb9 eval.nix: Auto-call functors as well 
This makes some very cursed setup work :P
 2021-12-04 01:03:26 -08:00
Zhaofeng Li
7757a0e45d command/apply: Use verbose output when goal is "dry-activate" 
It doesn't make sense to use the spinners since you can't see
the full output from the activation script.

Fixes #44.
 2021-12-04 01:03:26 -08:00
Zhaofeng Li
7b1feffb9e util: Log the configuration in use 2021-12-04 01:03:26 -08:00
Zhaofeng Li
9cd2af9438 cli: Re-order arguments in help message 2021-12-04 01:03:26 -08:00
Zhaofeng Li
f253e6eb18 cli: Allow configuring output colorization 
We now follow the <https://bixense.com/clicolors> standard.
 2021-12-04 01:03:26 -08:00
Zhaofeng Li
dc57b489ea progress: Restore tty-detection after refactoring 2021-12-04 01:03:26 -08:00
Zhaofeng Li
064432f38e nix/deployment: Misc cleanup 2021-12-04 01:03:26 -08:00