DGNum/colmena
13
0
Fork 1
Code Issues 6 Pull requests 2 Projects Releases Packages Wiki Activity

Add manual

Browse source 
We finally have some real documentation :)
This commit is contained in:
Zhaofeng Li 2021-11-17 22:21:00 -08:00
parent c42c97d2f6
commit 4497ef296e
27 changed files with 1979 additions and 122 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

123
README.md
View file

@ -1,8 +1,9 @@
# Colmena
![Build](https://github.com/zhaofengli/colmena/workflows/Build/badge.svg)
[![Manual](https://img.shields.io/badge/Manual-Pages-informational)](https://zhaofengli.github.io/colmena)
[![Build](https://github.com/zhaofengli/colmena/workflows/Build/badge.svg)](https://github.com/zhaofengli/colmena/actions/workflows/build.yml)
Colmena is a simple, stateless [NixOS](https://nixos.org) deployment tool modeled after [NixOps](https://github.com/NixOS/nixops) and [Morph](https://github.com/DBCDK/morph), written in Rust.
Colmena is a simple, stateless [NixOS](https://nixos.org) deployment tool modeled after [NixOps](https://github.com/NixOS/nixops) and [morph](https://github.com/DBCDK/morph), written in Rust.
It's a thin wrapper over Nix commands like `nix-instantiate` and `nix-copy-closure`, and supports parallel deployment.
Now with 100% more flakes! See *Tutorial with Flakes* below.
@ -36,7 +37,7 @@ nix-env -if default.nix
*See Tutorial with Flakes for usage with Nix Flakes.*
Colmena should work with your existing NixOps and Morph configurations with minimal modification.
Colmena should work with your existing NixOps and morph configurations with minimal modification.
Here is a sample `hive.nix` with two nodes, with some common configurations applied to both nodes:
```nix
@ -95,7 +96,7 @@ Here is a sample `hive.nix` with two nodes, with some common configurations appl
};
host-b = {
# Like NixOps and Morph, Colmena will attempt to connect to
# Like NixOps and morph, Colmena will attempt to connect to
# the remote host using the attribute name by default. You
# can override it like:
deployment.targetHost = "host-b.mydomain.tld";
@ -126,7 +127,7 @@ Here is a sample `hive.nix` with two nodes, with some common configurations appl
}
```
The full set of options can be found at `src/nix/eval.nix`.
The full set of options can be found in [the manual](https://zhaofengli.github.io/colmena/stable/reference).
Run `colmena build` in the same directory to build the configuration, or do `colmena apply` to build and deploy it to all nodes.
## Tutorial with Flakes
@ -167,118 +168,12 @@ Here is a short example:
}
```
The full set of options can be found at `src/nix/eval.nix`.
The full set of options can be found in [the manual](https://zhaofengli.github.io/colmena/stable/reference).
Run `colmena build` in the same directory to build the configuration, or do `colmena apply` to build and deploy it to all nodes.
## `colmena eval`
## Manual
Sometimes you may want to extract values from your Hive configuration for consumption in another program (e.g., [OctoDNS](https://github.com/octodns/octodns)).
To do that, create a `.nix` file with a lambda:
```nix
{ nodes, pkgs, lib, ... }:
# Feels like a NixOS module - But you can return any JSON-serializable value
lib.attrsets.mapAttrs (k: v: v.config.deployment.targetHost) nodes
```
Then you can evaluate with:
```
colmena eval your-lambda.nix
```
## `colmena apply-local`
For some machines, you may still want to stick with the manual `nixos-rebuild`-type of workflow.
Colmena allows you to build and activate configurations on the host running Colmena itself, provided that:
1. The node must be running NixOS.
1. The node must have `deployment.allowLocalDeployment` set to `true`.
1. The node's _attribute name_ must match the hostname of the machine.
If you invoke `apply-local` with `--sudo`, Colmena will attempt to elevate privileges with `sudo` if it's not run as root.
You may also find it helpful to set `deployment.targetHost` to `null` if you don't intend to deploy to the host via SSH.
As an example, the following `hive.nix` includes a node (`laptop`) that is meant to be only deployed with `apply-local`:
```nix
{
meta = {
nixpkgs = ./deps/nixpkgs-stable;
# I'd like to use the unstable version of Nixpkgs on
# my desktop machines.
nodeNixpkgs = {
laptop = ./deps/nixpkgs-unstable;
};
};
# This attribute name must match the output of `hostname` on your machine
laptop = { name, nodes, ... }: {
networking.hostName = "laptop";
deployment = {
# Allow local deployment with `colmena apply-local`
allowLocalDeployment = true;
# Disable SSH deployment. This node will be skipped in a
# normal`colmena apply`.
targetHost = null;
};
# Rest of configuration...
};
server-a = { pkgs, ... }: {
# This node will use the default Nixpkgs checkout specified
# in `meta.nixpkgs`.
# Rest of configuration...
};
}
```
On `laptop`, run `colmena apply-local --sudo` to activate the configuration.
## Secrets
Colmena allows you to upload secret files to nodes that will not be stored in the Nix store.
It implements a subset of the `deployment.keys` options supported by NixOps.
For example, to deploy DNS-01 credentials for use with `security.acme`:
```nix
{
shared-box = {
security.acme.certs."my-site.tld".credentialsFile = "/run/keys/acme-credentials.secret";
deployment.keys."acme-credentials.secret" = {
# Alternatively, `text` (string) or `keyFile` (path to file)
# may be specified.
keyCommand = [ "vault" "read" "-field=env" "secret/dns01" ];
destDir = "/run/keys"; # Default: /run/keys
user = "acme"; # Default: root
group = "nginx"; # Default: root
permissions = "0640"; # Default: 0600
uploadAt = "pre-activation"; # Default: pre-activation, Alternative: post-activation
};
# Rest of configuration...
};
}
```
Take note that if you use the default path (`/run/keys`), the secret files are only stored in-memory and will not survive reboots.
To upload your secrets without performing a full deployment, use `colmena upload-keys`.
## Parallelism
Colmena is built from the ground up to support parallel deployments.
Evaluation, build, and deployment of node configurations can happen at the same time.
This parallelism can be controlled primarily through two flags:
- `--limit <number>`: Number of hosts to deploy at once in the final step (pushing closures and activating new profiles).
- `--eval-node-limit <number>`: By default, Colmena will automatically determine the maximum number of nodes to evaluate at the same time according to available RAM. This flag allows you to set the limit to a predetermined value.
Read [the Colmena Manual](https://zhaofengli.github.io/colmena).
## Environment variables

22
flake.nix
View file

@ -17,8 +17,27 @@
pkgs = import nixpkgs { inherit system; };
in rec {
# We still maintain the expression in a Nixpkgs-acceptable form
packages.colmena = import ./default.nix { inherit pkgs; };
defaultPackage = self.packages.${system}.colmena;
packages = rec {
colmena = import ./default.nix { inherit pkgs; };
manual = let
colmena = self.packages.${system}.colmena;
evalNix = import ./src/nix/eval.nix {
hermetic = true;
};
deploymentOptionsMd = (pkgs.nixosOptionsDoc {
options = evalNix.docs.deploymentOptions pkgs;
}).optionsMDDoc;
metaOptionsMd = (pkgs.nixosOptionsDoc {
options = evalNix.docs.metaOptions pkgs;
}).optionsMDDoc;
in pkgs.callPackage ./manual {
inherit colmena deploymentOptionsMd metaOptionsMd;
};
manualFast = manual.override { colmena = null; };
};
defaultApp = self.apps.${system}.colmena;
apps.colmena = {
@ -28,6 +47,7 @@
devShell = pkgs.mkShell {
inputsFrom = [ defaultPackage ];
nativeBuildInputs = with pkgs; [ mdbook ];
shellHook = ''
export NIX_PATH=nixpkgs=${pkgs.path}
'';

1
manual/.gitignore vendored Normal file
View file

@ -0,0 +1 @@
book

9
manual/README.md Normal file
View file

@ -0,0 +1,9 @@
# Documentation
You can read the rendered version [here](https://zhaofengli.github.io/colmena).
## Building the Documentation
The manual is rendered using [mdBook](https://github.com/rust-lang/mdBook).
To build the manual, do `nix build .#manual`.
You can also do `nix build .#manualFast` for a version without the CLI usage reference.

11
manual/book.release.toml Normal file
View file

@ -0,0 +1,11 @@
[book]
authors = ["Zhaofeng Li"]
language = "en"
multilingual = false
src = "src"
title = "Colmena RELEASE"
[output.html]
site-url = "/colmena/RELEASE/"
git-repository-url = "https://github.com/zhaofengli/colmena"
edit-url-template = "https://github.com/zhaofengli/colmena/edit/main/manual/{path}"

11
manual/book.toml Normal file
View file

@ -0,0 +1,11 @@
[book]
authors = ["Zhaofeng Li"]
language = "en"
multilingual = false
src = "src"
title = "Colmena (Unstable)"
[output.html]
site-url = "/colmena/unstable/"
git-repository-url = "https://github.com/zhaofengli/colmena"
edit-url-template = "https://github.com/zhaofengli/colmena/edit/main/manual/{path}"

43
manual/default.nix Normal file
View file

@ -0,0 +1,43 @@
let
notFound = typ: "<span style=\"color: red;\">Error: No ${typ} passed to the builder</span>";
in
{ stdenv, nix-gitignore, mdbook
, release ? null
, deploymentOptionsMd ? notFound "deployment options text"
, metaOptionsMd ? notFound "meta options text"
, colmena ? null
}:
stdenv.mkDerivation {
inherit colmena deploymentOptionsMd metaOptionsMd release;
name = if release == null then "colmena-manual-dev" else "colmena-manual-${release}";
src = nix-gitignore.gitignoreSource [] ./.;
nativeBuildInputs = [ mdbook ];
patchPhase = ''
if [ -n "$release" ]; then
find . -name '*.md' -exec sed -i "/REMOVE_FOR_RELEASE/d" {} \;
sed "s/RELEASE/$release/g" book.release.toml > book.toml
fi
'';
buildPhase = ''
if [ -n "$colmena" ]; then
echo "Generating CLI help text"
$colmena/bin/colmena gen-help-markdown >> src/reference/cli.md
else
echo "Error: No colmena executable passed to the builder" >> src/reference/cli.md
fi
echo "$deploymentOptionsMd" >> src/reference/deployment.md
echo "$metaOptionsMd" >> src/reference/meta.md
mdbook build -d $out
'';
installPhase = "true";
}

19
manual/src/SUMMARY.md Normal file
View file

@ -0,0 +1,19 @@
# Summary
- [Introduction](./introduction.md)
- [Tutorial](./tutorial/index.md)
- [Usage with Flakes](./tutorial/flakes.md)
- [Migrating from NixOps/morph](./tutorial/migration.md)
- [Features](./features/index.md)
- [Node Tagging](./features/tags.md)
- [Local Deployment](./features/apply-local.md)
- [Secrets](./features/keys.md)
- [Ad Hoc Evaluation](./features/eval.md)
- [Parallelism](./features/parallelism.md)
- [Examples](./examples/index.md)
- [Multi-Architecture Deployments](./examples/multi-arch.md)
- [Reference](./reference/index.md)
- [Deployment Options](./reference/deployment.md)
- [Meta Options](./reference/meta.md)
- [Command Line Arguments](./reference/cli.md)
- [Contributing](./contributing.md)

14
manual/src/contributing.md Normal file
View file

@ -0,0 +1,14 @@
# Contributing
Contributions are welcome!
## Code
You can checkout the source code and submit pull requests at [the GitHub repo](https://github.com/zhaofengli/colmena).
By contributing code to Colmena, you agree that your contributions will be available under [the MIT License](https://github.com/zhaofengli/colmena/blob/main/LICENSE).
## Issues & Feature Requests
Bumped into a problem? We would like to fix it!
Please open a new issue at [the GitHub repo](https://github.com/zhaofengli/colmena).

5
manual/src/examples/index.md Normal file
View file

@ -0,0 +1,5 @@
# Examples
This section contains examples of setups people commonly use:
- **[Multi-Architecture Deployments](multi-arch.md)** - Deploying to hosts running a foreign architecture

36
manual/src/examples/multi-arch.md Normal file
View file

@ -0,0 +1,36 @@
# Multi-Architecture Deployments
You can deploy to hosts running different architectures with a single configuration.
This requires you to either [set up remote builders](https://nixos.org/manual/nix/stable/advanced-topics/distributed-builds.html) running the foreign architecture(s), or [set up binfmt emulation](https://nixos.wiki/wiki/NixOS_on_ARM#Compiling_through_QEMU) on the host running Colmena.
## `binfmt` Emulation
This following example sets up binfmt, allowing an X86-64 host (`laptop`) to build derivations for an AArch64 host (`rpi`) through QEMU:
```nix
{
# The NixOS machine you are running Colmena on (x86_64-linux)
laptop = { pkgs, ... }: {
# Enable binfmt emulation for aarch64-linux
boot.binfmt.emulatedSystems = [ "aarch64-linux" ];
# ... Rest of configuration ...
};
# The remote machine running a foreign architecture (aarch64-linux)
rpi = { pkgs, ... }: {
# Override nixpkgs architecture
nixpkgs.system = "aarch64-linux";
# ... Rest of configuration ...
};
}
```
*(For Flake users, the above attribute set is the value of `outputs.colmena`)*
First, deploy the local configuration with `colmena apply-local --sudo`.
For more information on what is required on the local system, see [Local Deployment](../features/apply-local.md).
After the new configuration is activated, binfmt emulation will be set up on the local machine.
You can then deploy to the `rpi` node with `colmena apply --on rpi`.

52
manual/src/features/apply-local.md Normal file
View file

@ -0,0 +1,52 @@
# Local Deployment
For some machines, you may still want to stick with the manual `nixos-rebuild`-type of workflow.
Colmena allows you to build and activate configurations on the host running Colmena itself, provided that:
1. The node must be running NixOS.
1. The node must have `deployment.allowLocalDeployment` set to `true`.
1. The node's _attribute name_ must match the hostname of the machine.
If you invoke `apply-local` with `--sudo`, Colmena will attempt to elevate privileges with `sudo` if it's not run as root.
You may also find it helpful to set `deployment.targetHost` to `null` if you don't intend to deploy to the host via SSH.
As an example, the following `hive.nix` includes a node (`laptop`) that is meant to be only deployed with `apply-local`:
```nix
{
meta = {
nixpkgs = ./deps/nixpkgs-stable;
# I'd like to use the unstable version of Nixpkgs on
# my desktop machines.
nodeNixpkgs = {
laptop = ./deps/nixpkgs-unstable;
};
};
# This attribute name must match the output of `hostname` on your machine
laptop = { name, nodes, ... }: {
networking.hostName = "laptop";
deployment = {
# Allow local deployment with `colmena apply-local`
allowLocalDeployment = true;
# Disable SSH deployment. This node will be skipped in a
# normal`colmena apply`.
targetHost = null;
};
# ... Rest of configuration ...
};
server-a = { pkgs, ... }: {
# This node will use the default Nixpkgs checkout specified
# in `meta.nixpkgs`.
# ... Rest of configuration ...
};
}
```
On `laptop`, run `colmena apply-local --sudo` to activate the configuration.

32
manual/src/features/eval.md Normal file
View file

@ -0,0 +1,32 @@
# Ad Hoc Evaluation
Sometimes you may want to extract values from your Hive configuration for consumption in another program (e.g., [OctoDNS](https://github.com/octodns/octodns)).
To do that, create a `.nix` file with a lambda:
```nix
{ nodes, pkgs, lib, ... }:
# Feels like a NixOS module - But you can return any JSON-serializable value
lib.attrsets.mapAttrs (k: v: v.config.deployment.targetHost) nodes
```
Then you can obtain a JSON output with:
```console
$ colmena eval target-hosts.nix
{"alpha":"fd12:3456::1","beta":"fd12:3456::2"}
```
You can also specify an expression directly on the command line:
```console
$ colmena eval -E '{ nodes, pkgs, lib, ... }: ...'
```
## Instantiation
You may directly instantiate an expression that evaluates to a derivation:
```console
$ colmena eval --instantiate -E '{ nodes, ... }: nodes.alpha.config.boot.kernelPackages.kernel'
/nix/store/7ggmhnwvywrqcd1z2sdpan8afz55sw7z-linux-5.14.14.drv
```

9
manual/src/features/index.md Normal file
View file

@ -0,0 +1,9 @@
# Features
This section introduces the main features in Colmena:
- **[Node Tagging](tags.md)** - Deploying to a subset of tagged nodes
- **[Local Deployment](apply-local.md)** - Deploying to the host running Colmena itself
- **[Secrets](keys.md)** - Deploying sensitive files separate from the main configuration
- **[Ad Hoc Evaluation](eval.md)** - Evaluating a Nix expression with access to your configuration
- **[Parallelism](parallelism.md)** - Controlling how Colmena parallelizes the deployment process

30
manual/src/features/keys.md Normal file
View file

@ -0,0 +1,30 @@
# Secrets
Colmena allows you to upload secret files that will not be stored in the Nix store to nodes.
It implements a subset of the `deployment.keys` options supported by NixOps.
For example, to deploy DNS-01 credentials for use with `security.acme`:
```nix
{
shared-box = {
security.acme.certs."my-site.tld".credentialsFile = "/run/keys/acme-credentials.secret";
deployment.keys."acme-credentials.secret" = {
# Alternatively, `text` (string) or `keyFile` (path to file)
# may be specified.
keyCommand = [ "vault" "read" "-field=env" "secret/dns01" ];
destDir = "/run/keys"; # Default: /run/keys
user = "acme"; # Default: root
group = "nginx"; # Default: root
permissions = "0640"; # Default: 0600
uploadAt = "pre-activation"; # Default: pre-activation, Alternative: post-activation
};
# Rest of configuration...
};
}
```
Take note that if you use the default path (`/run/keys`), the secret files are only stored in-memory and will not survive reboots.
To upload your secrets without performing a full deployment, use `colmena upload-keys`.

15
manual/src/features/parallelism.md Normal file
View file

@ -0,0 +1,15 @@
# Parallelism
Colmena is built from the ground up to support parallel deployments.
Evaluation, build, and deployment of node configurations can happen at the same time.
This parallelism can be controlled primarily through two flags:
**`--limit <number>`**
Number of hosts to deploy at once in the final step (pushing closures and activating new profiles). The default value is 10.
**`--eval-node-limit <number>`**
By default, Colmena will automatically determine the maximum number of nodes to evaluate at the same time according to available RAM.
This flag allows you to set the limit to a predetermined value.

35
manual/src/features/tags.md Normal file
View file

@ -0,0 +1,35 @@
# Node Tagging
With node tags, you can quickly select a subset of nodes for deployment.
You can specify tags using the `deployment.tags` option:
```nix
{
alpha = { pkgs, ... }: {
deployment.tags = [ "web" "infra-lax" ];
# ... Rest of configuration ...
};
beta = { pkgs, ... }: {
deployment.tags = [ "infra-sfo" ];
# ... Rest of configuration ...
};
}
```
You can filter hosts by tags or names with `--on`, which accepts a comma-separated list of node names or @tags.
To select all nodes with `web`:
```console
$ colmena apply --on @web
```
Wildcards are supported as well.
To select all nodes with a tag beginning with `infra-`:
```console
$ colmena apply --on '@infra-*'
```
*(Note the quotes around the argument)*

26
manual/src/introduction.md Normal file
View file

@ -0,0 +1,26 @@
# Introduction
**Colmena** is a simple, stateless [NixOS](https://nixos.org) deployment tool modeled after [NixOps](https://github.com/NixOS/nixops) and [morph](https://github.com/DBCDK/morph), written in Rust.
It's a thin wrapper over Nix commands like `nix-instantiate` and `nix-copy-closure`, and supports parallel deployment.
Interested? Get started [here](tutorial)!
<pre><div class="hljs">$ <b>colmena apply --on @tag-a</b>
[INFO ] Enumerating nodes...
[INFO ] Selected 7 out of 45 hosts.
(...) ✅ 0s Build successful
<b>sigma</b> 🕗 7s copying path '/nix/store/h6qpk8rwm3dh3zsl1wlj1jharzf8aw9f-unit-haigha-agent.service' to 'ssh://root@sigma.redacted'...
<b>theta</b> ✅ 7s Activation successful
<b>gamma</b> 🕘 8s Starting...
<b>alpha</b> ✅ 1s Activation successful
<b>epsilon</b> 🕗 7s copying path '/nix/store/fhh4rfixny8b21l6jqzk7nqwxva5k20h-nixos-system-epsilon-20.09pre-git' to 'ssh://root@epsilon.redacted'...
<b>beta</b> 🕗 7s removing obsolete file /boot/kernels/z28ayg10kpnlrz0s2qrb9pzv82lc20s2-initrd-linux-5.4.89-initrd
<b>kappa</b> ✅ 2s Activation successful
</div></pre>
You are currently reading **the unstable version** of the Colmena Manual, built against the tip of [the development branch](https://github.com/zhaofengli/colmena).<!-- REMOVE_FOR_RELEASE -->
## Links
- [GitHub](https://github.com/zhaofengli/colmena)
- [Deployment Options Reference](reference/deployment.md)

7
manual/src/reference/cli.md Normal file
View file

@ -0,0 +1,7 @@
# Command Line Arguments
You are currently reading **the unstable version** of the Colmena Manual, built against the tip of [the development branch](https://github.com/zhaofengli/colmena). <!-- REMOVE_FOR_RELEASE -->
The following are the auto-generated help messages that are printed when you invoke any sub-command with `--help`:
<!-- The following is injected by the build system -->

7
manual/src/reference/deployment.md Normal file
View file

@ -0,0 +1,7 @@
# Deployment Options
You are currently reading **the unstable version** of the Colmena Manual, built against the tip of [the development branch](https://github.com/zhaofengli/colmena). <!-- REMOVE_FOR_RELEASE -->
Colmena adds a set of extra options that can be used in your NixOS configurations under the `deployment` prefix.
<!-- The following is injected by the build system -->

9
manual/src/reference/index.md Normal file
View file

@ -0,0 +1,9 @@
# Reference
You are currently reading **the unstable version** of the Colmena Manual, built against the tip of [the development branch](https://github.com/zhaofengli/colmena). Go back to [the home page](../introduction.md) for links to other versions of the manual. <!-- REMOVE_FOR_RELEASE -->
This section contains detailed listings of options and parameters accepted by Colmena:
- **[Deployment Options](deployment.md)**
- **[Meta Options](meta.md)**
- **[Command Line Arguments](cli.md)**

10
manual/src/reference/meta.md Normal file
View file

@ -0,0 +1,10 @@
# Meta Options
You are currently reading **the unstable version** of the Colmena Manual, built against the tip of [the development branch](https://github.com/zhaofengli/colmena). <!-- REMOVE_FOR_RELEASE -->
The following is a list of options that can be added to the `meta` attribute set.
For compatibility with NixOps, you may name it `network` instead of `meta`.
However, you cannot specify both at the same time.
<!-- The following is injected by the build system -->

75
manual/src/tutorial/flakes.md Normal file
View file

@ -0,0 +1,75 @@
# Usage with Flakes
## Installation
Colmena doesn't have a stable release yet.
To quickly try Colmena out, use the following command to enter an ephemeral environment with `colmena`:
```console
$ nix shell github:zhaofengli/colmena
```
To install the latest development version to the user profile, use the following command:
```console
$ nix-env -if https://github.com/zhaofengli/colmena/tarball/main
```
You can also add `github:zhaofengli/colmena` as an input in your Flake and add the `colmena` package to your `devShell`.
## Basic Configuration
Colmena reads the `colmena` output in your Flake.
Here is a short example:
```nix
{
inputs = {
nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
};
outputs = { nixpkgs, ... }: {
colmena = {
meta = {
nixpkgs = import nixpkgs {
system = "x86_64-linux";
};
};
host-a = { name, nodes, pkgs, ... }: {
boot.isContainer = true;
time.timeZone = nodes.host-b.config.time.timeZone;
};
host-b = {
deployment = {
targetHost = "somehost.tld";
targetPort = 1234;
targetUser = "luser";
};
boot.isContainer = true;
time.timeZone = "America/Los_Angeles";
};
};
};
}
```
The full set of `deployment` options can be found [here](../reference/deployment.md).
For some inspiration, you can check out the short example in [the main tutorial](index.md).
Now you are ready to use Colmena! To build the configuration:
```console
$ colmena build
```
To build and deploy to all nodes:
```console
$ colmena apply
```
## Next Steps
- Head to the [Features](../features/index.md) section to see what else Colmena can do.
- Read more about options available in Colmena in the [Reference](../reference/index.md) section.

125
manual/src/tutorial/index.md Normal file
View file

@ -0,0 +1,125 @@
# Tutorial
## Installation
Colmena doesn't have a stable release yet.
To install the latest development version to the user profile, use the following command:
```console
$ nix-env -if https://github.com/zhaofengli/colmena/tarball/main
```
## Basic Configuration
*If you use Nix Flakes, follow the Flake version [here](flakes.md).*
Colmena should work with your existing NixOps and morph configurations with minimal modification (see [Migrating from NixOps/morph](migration.md)).
Here is a sample `hive.nix` with two nodes, with some common configurations applied to both nodes:
```nix
{
meta = {
# Override to pin the Nixpkgs version (recommended). This option
# accepts one of the following:
# - A path to a Nixpkgs checkout
# - The Nixpkgs lambda (e.g., import <nixpkgs>)
# - An initialized Nixpkgs attribute set
nixpkgs = <nixpkgs>;
# You can also override Nixpkgs by node!
nodeNixpkgs = {
node-b = ./another-nixos-checkout;
};
# If your Colmena host has nix configured to allow for remote builds
# (for nix-daemon, your user being included in trusted-users)
# you can set a machines file that will be passed to the underlying
# nix-store command during derivation realization as a builders option.
# For example, if you support multiple orginizations each with their own
# build machine(s) you can ensure that builds only take place on your
# local machine and/or the machines specified in this file.
# machinesFile = ./machines.client-a;
};
defaults = { pkgs, ... }: {
# This module will be imported by all hosts
environment.systemPackages = with pkgs; [
vim wget curl
];
# By default, Colmena will replace unknown remote profile
# (unknown means the profile isn't in the nix store on the
# host running Colmena) during apply (with the default goal,
# boot, and switch).
# If you share a hive with others, or use multiple machines,
# and are not careful to always commit/push/pull changes
# you can accidentaly overwrite a remote profile so in those
# scenarios you might want to change this default to false.
# deployment.replaceUnknownProfiles = true;
};
host-a = { name, nodes, ... }: {
# The name and nodes parameters are supported in Colmena,
# allowing you to reference configurations in other nodes.
networking.hostName = name;
time.timeZone = nodes.host-b.config.time.timeZone;
boot.loader.grub.device = "/dev/sda";
fileSystems."/" = {
device = "/dev/sda1";
fsType = "ext4";
};
};
host-b = {
# Like NixOps and Morph, Colmena will attempt to connect to
# the remote host using the attribute name by default. You
# can override it like:
deployment.targetHost = "host-b.mydomain.tld";
# It's also possible to override the target SSH port.
# For further customization, use the SSH_CONFIG_FILE
# environment variable to specify a ssh_config file.
deployment.targetPort = 1234;
# Override the default for this target host
deployment.replaceUnknownProfiles = false;
# You can filter hosts by tags with --on @tag-a,@tag-b.
# In this example, you can deploy to hosts with the "web" tag using:
# colmena apply --on @web
# You can use globs in tag matching as well:
# colmena apply --on '@infra-*'
deployment.tags = [ "web" "infra-lax" ];
time.timeZone = "America/Los_Angeles";
boot.loader.grub.device = "/dev/sda";
fileSystems."/" = {
device = "/dev/sda1";
fsType = "ext4";
};
};
}
```
The full set of `deployment` options can be found [here](../reference/deployment.md).
Now you are ready to use Colmena! To build the configuration:
```console
$ colmena build
```
To build and deploy to all nodes:
```console
$ colmena apply
```
## Next Steps
- Head to the [Features](../features/index.md) section to see what else Colmena can do.
- Read more about options available in Colmena in the [Reference](../reference/index.md) section.

26
manual/src/tutorial/migration.md Normal file
View file

@ -0,0 +1,26 @@
# Migrating from NixOps/morph
Colmena should work with existing NixOps and morph configurations with minimal modification.
That said, there are a few things to look out for:
## Colmena deploys to *existing* NixOS hosts
Unlike NixOps which can be configured to manage the entire lifecycles of NixOS machines (e.g., spinning up AWS EC2 instances), Colmena can only deploy to hosts already running NixOS.
## `network` vs `meta`
Colmena accepts [a set of options](/reference/meta.md) to configure the deployment itself as `meta`.
For NixOps compatibility, it also accepts `network` as an alias so you don't have to change your existing configuration.
## Pinning Nixpkgs
You can pin the nixpkgs version by setting `meta.nixpkgs` (or `network.nixpkgs` if you use the alias).
This is required if you [use Flakes](flakes.md).
The options accepts one of the following:
- Path to a Nixpkgs checkout *(not supported in Flakes)*
- Example: `./nixpkgs`
- The Nixpkgs lambda returned by importing its `default.nix` *(not supported in Flakes)*
- Example: `import ./nixpkgs`
- A fully initialized Nixpkgs attribute set
- Example: `import ./nixpkgs { system = "x86_64-linux"; }`

1313
manual/theme/highlight.js Normal file
View file

File diff suppressed because one or more lines are too long

36
src/nix/eval.nix
View file

@ -14,10 +14,10 @@ let
defaults = {};
};
types = lib.types;
# Hive-wide options
metaOptions = { lib, ... }: {
metaOptions = { lib, ... }: let
types = lib.types;
in {
options = {
name = lib.mkOption {
description = ''
@ -44,7 +44,7 @@ let
This option must be specified when using Flakes.
'';
type = types.unspecified;
default = <nixpkgs>;
default = if !hermetic then <nixpkgs> else null;
};
nodeNixpkgs = lib.mkOption {
description = ''
@ -88,7 +88,9 @@ let
# Colmena-specific options
#
# Largely compatible with NixOps/Morph.
deploymentOptions = { name, lib, ... }: {
deploymentOptions = { name, lib, ... }: let
types = lib.types;
in {
options = {
deployment = {
targetHost = lib.mkOption {
@ -152,7 +154,7 @@ let
Secrets are transferred to the node out-of-band and
never ends up in the Nix store.
'';
type = types.attrsOf keyType;
type = types.attrsOf (types.submodule keyType);
default = {};
};
replaceUnknownProfiles = lib.mkOption {
@ -185,7 +187,9 @@ let
};
};
keyType = types.submodule {
keyType = { lib, ... }: let
types = lib.types;
in {
options = {
text = lib.mkOption {
description = ''
@ -468,4 +472,22 @@ let
in {
inherit nodes deploymentConfigJson toplevel buildAll buildSelected introspect;
meta = hive.meta;
docs = {
deploymentOptions = pkgs: let
eval = pkgs.lib.evalModules {
modules = [ deploymentOptions ];
specialArgs = {
name = "nixos";
nodes = {};
};
};
in eval.options;
metaOptions = pkgs: let
eval = pkgs.lib.evalModules {
modules = [ metaOptions ];
};
in eval.options;
};
}