ecoppens/colmena
1
0
Fork 0
forked from DGNum/colmena
Code Pull requests Activity
514 commits 2 branches 0 tags 996 KiB
Commit graph

283 commits

Author SHA1 Message Date
Zhaofeng Li
0ee3bb2e62 Update inotify-tools alias 
No longer exists in unstable since https://github.com/NixOS/nixpkgs/pull/192681.
 2022-09-28 18:47:42 -06:00
Zhaofeng Li
a8e6b999cf host/ssh: Fix extension matching for .drv 
Oopsie
 2022-09-21 00:40:27 -06:00
Zhaofeng Li
3bdd0b1cc7 hive/flake.nix: Fix trailing whitespce 2022-09-18 17:45:43 -06:00
Zhaofeng Li
3e8ec98a26 ssh: Support using the experimental nix copy command to copy 
The new SSH store protocol (ssh-ng://) provides better performance.
This is now enabled for flake deployments.
 2022-09-18 17:27:46 -06:00
Zhaofeng Li
695ec0c36f Use --extra-experimental-features as opposed to --experimental-features 2022-09-18 17:27:46 -06:00
Zhaofeng Li
d4dcf1c6e9 Add support for using an already-evaluated hive 
Note that the interface hasn't been finalized yet.
After this commit, the discovery order is:

1. flake.nix `colmenaHive` output (prepared hive)
2. flake.nix `colmena` output (raw hive)
3. hive.nix (raw hive)
 2022-09-18 17:27:46 -06:00
Zhaofeng Li
bd048e98d8 flake.nix: Add lib.makeHive 
Ground work to make the evaluation logic more clearly separated from
the deployment logic.

The `colmenaHive` output will be the integration point between the two
halves.
 2022-09-18 17:27:46 -06:00
Zhaofeng Li
52b594a167 apply: Run rustfmt 2022-09-01 18:49:11 -06:00
Zhaofeng Li
f01f6ac197 deployment: Move post-activation key upload after reboot 
Ref: #113
 2022-09-01 18:42:37 -06:00
Zhaofeng Li
3af3751d8e apply: Default to the "boot" goal if --reboot is passed 
Fixes #113.
 2022-09-01 18:42:37 -06:00
Zhaofeng Li
e740a50b73 nix/assets: Use more descriptive prefix for temp dir 2022-08-28 00:45:23 -06:00
Zhaofeng Li
3fbc5eeb11 repl: Add arguments depending on Nix version 2022-08-17 11:56:30 -06:00
Zhaofeng Li
741d5c4fbb repl: Use a temporary file for the expression 
This makes it compatible with Nix 2.3+.
 2022-08-17 11:56:30 -06:00
Zhaofeng Li
7aa1607a3c nix_eval_jobs: Pass the expression on the command line 
This gets rid of the temporary file.
 2022-08-17 01:09:43 -06:00
Zhaofeng Li
06168dabf3 nix_eval_jobs: Remove --impure 
Note that this still does not make flake evaluation pure in
nix-eval-jobs yet, since it forcibly resets `evalSettings.pureEval`
prior to starting the evaluation.
 2022-08-16 20:38:42 -06:00
Zhaofeng Li
c540e00861 Add repl subcommand 
Fixes #115.
 2022-08-16 20:16:00 -06:00
Zhaofeng Li
11add6501e exec: Set label width in output 2022-08-16 20:16:00 -06:00
Zhaofeng Li
80426aa814 cli: Support passing --impure 2022-08-16 20:16:00 -06:00
Zhaofeng Li
8aca525788 Make flake evaluation pure 
This seems to be the easiest way to get pure-eval working with
existing evaluation mechinery (nix-instantiate, nix-eval-jobs).

Now `--pure-eval` is forced for flakes with user being able to
add `--impure` as needed.
 2022-08-16 20:16:00 -06:00
Zhaofeng Li
092e5848ab Directly serialize Nix expressions as quoted strings 2022-08-16 20:16:00 -06:00
Zhaofeng Li
271d9ae576 nix/assets: Make assets a flake 
Groundwork to make flake evaluation pure. `pure-eval` works when
the git workspace is clean.
 2022-08-16 20:16:00 -06:00
Zhaofeng Li
a98d1f8963 nix/flake: Always resolve flake metadata 2022-08-16 20:16:00 -06:00
Zhaofeng Li
81bda9d595 Update deps 2022-08-16 20:16:00 -06:00
Zhaofeng Li
68d0036a2c Thanks clippy 2022-08-16 20:16:00 -06:00
Zhaofeng Li
62a3d1e6f8 Run rustfmt 2022-07-29 22:13:09 -07:00
Zhaofeng Li
7d0a6eed90 Apply quit::main to main function 
This makes `quit::with_code` actually work with the correct exit code.

Fixes #111.
 2022-07-17 14:42:34 -07:00
David Arnold
b3ca4b1c4d
ref: transparent eval.nix contract 
- at least to my eyes, this code formatting gives a better view about
  the gist of the eval.nix interface
 2022-07-09 16:04:23 -05:00
David Arnold
e309b2d4b2
ref: reduce eval.nix contract even further 
- meta and evalAll contracts are found to be not consumed
 2022-07-09 15:41:17 -05:00
David Arnold
83a919bdb0
ref: reduce eval.nix contract forther 
- module export contratcts can now bypass the eval.nix iface
- enabled by:9bd5e7bb2540664fe1c728c85f80375fe0cbf9fa
 2022-07-09 15:41:17 -05:00
David Arnold
c82517fc12
ref: reduce eval.nix contract 
- documentation contratcts can now bypass the eval.nix iface
- enabled by:9bd5e7bb2540664fe1c728c85f80375fe0cbf9fa
 2022-07-09 15:41:15 -05:00
Zhaofeng Li
b8bdf5824c hive: Use absolute paths for non-flake hives 
Fixes #107.
 2022-07-04 14:05:50 -07:00
Zhaofeng Li
ea1e7684f1 hive: Add tests for nodeSpecialArgs 2022-07-01 16:31:13 -07:00
Zhaofeng Li
1b045eb38d
Merge pull request #100 from fooker/pr/specialArgs 
eval.nix: per-node specialArgs
 2022-07-01 16:29:59 -07:00
NeverBehave
dfe55ed461 fix: rename no-substitutes -> no-substitute 
Fixes #59.
 2022-07-01 16:06:01 -07:00
Zhaofeng Li
432d4212fc apply: Don't enforce meta.allowApplyAll for builds 
Fixes #99.
 2022-06-24 17:34:37 -07:00
Zhaofeng Li
0209b6dc40 Add name of key to key upload errors 
> example | Key upload failed: Error processing key "meow": I/O Error: No such file or directory (os error 2)

Fixes #101.
 2022-06-24 17:34:37 -07:00
Dustin Frisch
07499a1eaf
eval.nix: per-node specialArgs 2022-06-23 12:09:29 +02:00
NeverBehave
dd7a2924ca feat(meta): add disallowApplyAll options 
unify meta access for machinesFile

update release note and config name
 2022-06-20 18:47:53 -07:00
Zhaofeng Li
dd3f4b252c host/local: Upload keys with sudo as well 2022-06-10 11:29:25 -07:00
Zhaofeng Li
d7aa4ab864 hive/tests: Fix editorconfig 2022-06-09 11:35:02 -07:00
Zhaofeng Li
9db25bd914 deployment: Move GC root creation after build 
Previously GC roots were only created after successful activation.
 2022-06-09 11:31:49 -07:00
Zhaofeng Li
2b0bafa155 eval.nix: Restore signature in introspect 
It was broken in 9bd5e7bb25.
 2022-06-09 11:31:49 -07:00
Zhaofeng Li
b5629dca83 eval.nix: Enable config merging for meta.nixpkgs on 21.11+ 
Ref: #74 #15
 2022-06-03 23:51:32 -07:00
Zhaofeng Li
1cd3aa6401 eval.nix: Accept path-likes in meta.nixpkgs as well 
This fixes usage with Niv.
 2022-06-03 23:51:32 -07:00
Zhaofeng Li
b87da69558 Don't import impure overlays if meta.nixpkgs is a path 
Also update documentations to recommend specifying `overlays = []`
explicitly.

Fixes #39.
 2022-06-03 23:51:32 -07:00
Zhaofeng Li
ca12be27ed apply-local: Escalate privileges only during activation 
Fixes #85.
 2022-06-03 23:51:32 -07:00
Zhaofeng Li
fa07814abf hive/assets.rs: Use OpenOptions::new() instead 2022-06-03 23:51:32 -07:00
Zhaofeng Li
4037b61a2b apply: Fix typo in help 2022-06-03 23:51:32 -07:00
Zhaofeng Li
af3247faeb Update docs after #91 2022-06-03 23:51:32 -07:00
Zhaofeng Li
ed91898e59 Clippy 2022-06-03 23:51:32 -07:00
Zhaofeng Li
ac8852dcdf hive: Refactor static file creation 2022-06-02 16:39:45 -07:00
Wanja Hentze
6fe48ccd04 nix-eval-jobs: Strip quotes surrounding attribute names 
Fixes #92.
 2022-06-02 16:39:45 -07:00
Victor Nawothnig
e937eb8faa Do not use an explicit user for ssh when deploymentUser is null 2022-06-01 12:30:49 +02:00
Zhaofeng Li
429a0f5aa1 eval.nix: Suppress _module.args docs 2022-05-22 02:15:33 -07:00
Zhaofeng Li
1be16d552d apply: Add --reboot flag 
Fixes #46.
 2022-05-22 02:15:33 -07:00
Zhaofeng Li
e239cbd260 host: Add get_current_system_profile 2022-05-22 02:15:33 -07:00
Zhaofeng Li
c8b79e7e2d ssh: Clean up 2022-05-22 02:15:33 -07:00
David Arnold
9bd5e7bb25 imp: soc of eval, modules & options 
- the reason for this change is to have more transparent separation
  of concern between effectuations of the module system and pre-module
  system effectuations

- with improved flakes support down the line, pre-module system
  effectuations will get more complex

- this also allows to patch the aspects of the evaluation individually
  while tracking other components from upstream. eg. path options & eval
  but not modules
 2022-05-22 01:54:52 -07:00
Zhaofeng Li
ea4f2ba6dc Migrate other doc links to new domain 2022-05-13 19:14:58 -07:00
Zhaofeng Li
3e7341a98c utils.rs: Tolerate invalid UTF-8 in streams 
This will substitute any invalid UTF-8 sequence with `\u{fffd}`. Such
substitutions are okay for human-readable log outputs.

Correctness: A side-effect of this is that it also affects the code
that captures filesystem paths from stdout, in which case any substitution
is unacceptable. Currently we only capture store paths from Nix, and Nix
restricts characters allowed in derivation names. Nevertheless we will
refactor this for correctness.

Fixes #75.
 2022-04-25 00:03:57 -07:00
Zhaofeng Li
6a64625fda eval.nix: Ignore typed allowAliases config when checking unoverridden config keys 
This was added in NixOS/nixpkgs#166792.

Fixes #74.
 2022-04-25 00:03:57 -07:00
Zhaofeng Li
ab754ac2e3 Squelch dead code warnings on non-Linux platforms (aka macOS) 2022-04-19 00:29:35 -07:00
Zhaofeng Li
5a81996afb ssh: Enable BatchMode 2022-04-19 00:29:35 -07:00
Zhaofeng Li
26b2203da9 Update deps 2022-03-07 22:02:04 -08:00
Zhaofeng Li
108c135700 Rename: NixFailure -> ChildFailure, NixKilled -> ChildKilled 2022-03-07 22:02:04 -08:00
Glenn McDonald
9c179b0db8
apply_local: Fix nixos detection from os-release (#63) 2022-03-07 20:52:22 -08:00
Zhaofeng Li
2b281286d0
eval.nix: Add "${name}-key.service" units for keys (#51) 
Fixes #48.
 2022-02-16 10:23:27 -08:00
Zhaofeng Li
85b0107b59 eval.nix: Skip recursive type checking for nodes in colmena eval as well 
This matches the behavior of the `nodes` argument in NixOS modules.
 2022-02-16 01:11:06 -08:00
Linus Heckemann
efa7322587
streaming evaluator: respect eval-nodes-limit (#58) 2022-02-16 01:10:52 -08:00
Zhaofeng Li
c088925e0d Misc doc tweaks 2022-02-05 18:43:43 -08:00
Zhaofeng Li
45b135c1b1 nix_eval_jobs: Add timeouts to tests 
Sometimes nix-eval-jobs would lock up indefinitely, so let's make it
fail fast in CI.

Also ignore test_json_global_error while we investigate.
 2022-02-04 20:58:17 -08:00
Zhaofeng Li
856f82644e apply_local: Fix argument name typo 
Caught by debug assertions in clap 3.0. We should switch to the Derive
API where possible to prevent this from happening in the future.
 2022-01-27 18:48:25 -08:00
Zhaofeng Li
f674ddf173 key: Compute full path to key file in Nix 2022-01-25 14:22:26 -08:00
Zhaofeng Li
e58dde1be0 eval.nix: Don't override modulesPath 
`eval-config.nix` will set the canonical path for us that will then be used
to strip the module path prefixes in `nixos/modules/misc/documentation.nix` [1].
This should fix split docs build problems described in #50.

[1] 1511e72b75/nixos/modules/misc/documentation.nix (L79)
 2022-01-25 14:22:26 -08:00
Zhaofeng Li
7cd9b6f70c eval.nix: Don't pass baseModules to modules 
This is unnecessary and breaks the separation of user `modules`
and `baseModules` in `eval-config.nix`.
 2022-01-25 14:22:26 -08:00
Zhaofeng Li
9251019723 nix_eval_jobs: Errors may not have an attribute name attached 2022-01-25 14:22:26 -08:00
Zhaofeng Li
c0107b21e0 General code cleanup 2022-01-25 14:22:26 -08:00
Paul Haerle
5b8971a0f4
eval.nix: expose nixosModules.deploymentOptions (#49) 
Allow flake users to import .#nixosModules.deploymentOptions
into their flake, so that the same expression can be used for both,
.#colmena.$host as well as .#nixosConfiguration.$host, without the
latter complaining about undefined options in "deployment".
 2022-01-23 10:06:41 -08:00
Zhaofeng Li
0d9198c351 spinner: Work around the lack of newline after completion 2022-01-22 17:50:53 -08:00
Zhaofeng Li
87189ec3df nix_info: Display pinned nix-eval-jobs path 2022-01-22 17:50:53 -08:00
Zhaofeng Li
cf9a72a1d4 nix: Add initial nix-eval-job integration 
A DrvSetEvaluator is able to evaluate attribute sets of derivations,
streaming results as they come in.
 2022-01-22 17:50:53 -08:00
Zhaofeng Li
3e40e84e19 nix: Add NixExpression abstraction 
This decouples expression generation from evaluation. `NixExpression`s
that evaluate to a set of derivations can be fed to `DrvSetEvaluator`s
which may be able to parallelize evaluation.
 2022-01-22 17:50:53 -08:00
Zhaofeng Li
9fbb6b133b job: Make it possible to have a JobHandle not connected to a JobMonitor 2022-01-22 17:50:53 -08:00
Zhaofeng Li
55ce6d078e Remove unused stuff 2022-01-22 17:50:53 -08:00
Zhaofeng Li
d3e556027f deployment: Move chunked mode into self-contained function 2022-01-21 00:45:12 -08:00
Zhaofeng Li
82361e5ea5 hive: Clean up, make way for nix-eval-jobs 2022-01-21 00:45:12 -08:00
Zhaofeng Li
3fa0dee6b0 hive: Convert derivations to drvPath in separate attribute 
This is to make way for the nix-eval-jobs streaming evaluator.
 2022-01-21 00:45:12 -08:00
Zhaofeng Li
abb74b9f49 Add EditorConfig rules, fix trailing whitespaces 2022-01-08 01:20:36 -08:00
Zhaofeng Li
deca292b53 Refactor NixOptions 2022-01-08 01:20:36 -08:00
Zhaofeng Li
31fd1e49ac Move nix::{NixResult, NixError} to error::{ColmenaResult, ColmenaError} 2022-01-08 01:20:36 -08:00
Zhaofeng Li
16ed9d8c66 Move nix::NixCommand to util::CommandExt 2022-01-08 01:20:36 -08:00
Zhaofeng Li
f92236da46 Refactor current profile detection 2022-01-08 01:20:36 -08:00
Zhaofeng Li
6401ce4c3c cli: Remove target from logs 
format_target is enabled by default in env_logger 0.9.0. It makes
the logs look cluttered and is not very useful for our use case.
 2022-01-05 14:01:05 -08:00
Zhaofeng Li
da7907c874 General code cleanup 2022-01-05 14:01:05 -08:00
Zhaofeng Li
f234e16e80 manual: Fix colorized CLI help 
In clap 3.0, it's no longer possible to get `App::write_long_help` to
output colorized text (it doesn't invoke the Colorizer at all). So let's
move the generation outside of Rust.
 2022-01-03 10:37:03 -08:00
Zhaofeng Li
d2762757f0 command/eval: Fix introspect alias 
On clap 3.0, resetting the App::name still causes clap to complain
that the subcommand doesn't exist in debug mode.
 2022-01-03 10:37:03 -08:00
Zhaofeng Li
79d1010356 command: Change wording for consistency 2022-01-03 10:37:03 -08:00
Zhaofeng Li
81e2982bbb Migrate to clap 0.3 2022-01-03 10:37:03 -08:00
Zhaofeng Li
f9fab83030 deployment: Remove useless Arc clones 2022-01-02 13:16:24 -08:00
Zhaofeng Li
7bf57fd1ad nix/node_filter: Forgot to add 2022-01-01 16:47:38 -08:00