DGNum/infrastructure
11
6
Fork 2
Code Issues 20 Pull requests 14 Projects 1 Releases Packages Wiki Activity Actions

meta-isp #295

Open
thubrecht wants to merge 4 commits from meta-isp into main
pull from: meta-isp
merge into: DGNum:main
Conversation 0 Commits 4 Files changed 9 +313 -672
9 changed files with 313 additions and 672 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

468
lib/nix-lib/nixpkgs.nix
View file

@ -1,468 +0,0 @@
# SPDX-FileCopyrightText: 2024 Tom Hubrecht <tom.hubrecht@dgnum.eu>
#
# SPDX-License-Identifier: EUPL-1.2
###
# Collection of nixpkgs library functions, those are necessary for defining our own lib
#
# They have been simplified and builtins are used in some places, instead of lib shims.
rec {
/**
Does the same as the update operator '//' except that attributes are
merged until the given predicate is verified. The predicate should
accept 3 arguments which are the path to reach the attribute, a part of
the first attribute set and a part of the second attribute set. When
the predicate is satisfied, the value of the first attribute set is
replaced by the value of the second attribute set.
# Inputs
`pred`
: Predicate, taking the path to the current attribute as a list of strings for attribute names, and the two values at that path from the original arguments.
`lhs`
: Left attribute set of the merge.
`rhs`
: Right attribute set of the merge.
# Type
```
recursiveUpdateUntil :: ( [ String ] -> AttrSet -> AttrSet -> Bool ) -> AttrSet -> AttrSet -> AttrSet
```
# Examples
:::{.example}
## `lib.attrsets.recursiveUpdateUntil` usage example
```nix
recursiveUpdateUntil (path: l: r: path == ["foo"]) {
# first attribute set
foo.bar = 1;
foo.baz = 2;
bar = 3;
} {
#second attribute set
foo.bar = 1;
foo.quz = 2;
baz = 4;
}
=> {
foo.bar = 1; # 'foo.*' from the second set
foo.quz = 2; #
bar = 3; # 'bar' from the first set
baz = 4; # 'baz' from the second set
}
```
:::
*/
recursiveUpdateUntil =
pred: lhs: rhs:
let
f =
attrPath:
builtins.zipAttrsWith (
n: values:
let
here = attrPath ++ [ n ];
in
if builtins.length values == 1 || pred here (builtins.elemAt values 1) (builtins.head values) then
builtins.head values
else
f here values
);
in
f
[ ]
[
rhs
lhs
];
/**
A recursive variant of the update operator ‘//’. The recursion
stops when one of the attribute values is not an attribute set,
in which case the right hand side value takes precedence over the
left hand side value.
# Inputs
`lhs`
: Left attribute set of the merge.
`rhs`
: Right attribute set of the merge.
# Type
```
recursiveUpdate :: AttrSet -> AttrSet -> AttrSet
```
# Examples
:::{.example}
## `lib.attrsets.recursiveUpdate` usage example
```nix
recursiveUpdate {
boot.loader.grub.enable = true;
boot.loader.grub.device = "/dev/hda";
} {
boot.loader.grub.device = "";
}
returns: {
boot.loader.grub.enable = true;
boot.loader.grub.device = "";
}
```
:::
*/
recursiveUpdate =
lhs: rhs:
recursiveUpdateUntil (
_: lhs: rhs:
!(builtins.isAttrs lhs && builtins.isAttrs rhs)
) lhs rhs;
/**
Determine whether a string has given prefix.
# Inputs
`pref`
: Prefix to check for
`str`
: Input string
# Type
```
hasPrefix :: string -> string -> bool
```
# Examples
:::{.example}
## `lib.strings.hasPrefix` usage example
```nix
hasPrefix "foo" "foobar"
=> true
hasPrefix "foo" "barfoo"
=> false
```
:::
*/
hasPrefix = pref: str: (builtins.substring 0 (builtins.stringLength pref) str == pref);
/**
Escape occurrence of the elements of `list` in `string` by
prefixing it with a backslash.
# Inputs
`list`
: 1\. Function argument
`string`
: 2\. Function argument
# Type
```
escape :: [string] -> string -> string
```
# Examples
:::{.example}
## `lib.strings.escape` usage example
```nix
escape ["(" ")"] "(foo)"
=> "\\(foo\\)"
```
:::
*/
escape = list: builtins.replaceStrings list (builtins.map (c: "\\${c}") list);
/**
Convert a string `s` to a list of characters (i.e. singleton strings).
This allows you to, e.g., map a function over each character. However,
note that this will likely be horribly inefficient; Nix is not a
general purpose programming language. Complex string manipulations
should, if appropriate, be done in a derivation.
Also note that Nix treats strings as a list of bytes and thus doesn't
handle unicode.
# Inputs
`s`
: 1\. Function argument
# Type
```
stringToCharacters :: string -> [string]
```
# Examples
:::{.example}
## `lib.strings.stringToCharacters` usage example
```nix
stringToCharacters ""
=> [ ]
stringToCharacters "abc"
=> [ "a" "b" "c" ]
stringToCharacters "🦄"
=> [ "<EFBFBD>" "<EFBFBD>" "<EFBFBD>" "<EFBFBD>" ]
```
:::
*/
stringToCharacters = s: builtins.genList (p: builtins.substring p 1 s) (builtins.stringLength s);
/**
Turn a string `s` into an exact regular expression
# Inputs
`s`
: 1\. Function argument
# Type
```
escapeRegex :: string -> string
```
# Examples
:::{.example}
## `lib.strings.escapeRegex` usage example
```nix
escapeRegex "[^a-z]*"
=> "\\[\\^a-z]\\*"
```
:::
*/
escapeRegex = escape (stringToCharacters "\\[{()^$?*+|.");
/**
Appends string context from string like object `src` to `target`.
:::{.warning}
This is an implementation
detail of Nix and should be used carefully.
:::
Strings in Nix carry an invisible `context` which is a list of strings
representing store paths. If the string is later used in a derivation
attribute, the derivation will properly populate the inputDrvs and
inputSrcs.
# Inputs
`src`
: The string to take the context from. If the argument is not a string,
it will be implicitly converted to a string.
`target`
: The string to append the context to. If the argument is not a string,
it will be implicitly converted to a string.
# Type
```
addContextFrom :: string -> string -> string
```
# Examples
:::{.example}
## `lib.strings.addContextFrom` usage example
```nix
pkgs = import <nixpkgs> { };
addContextFrom pkgs.coreutils "bar"
=> "bar"
```
The context can be displayed using the `toString` function:
```nix
nix-repl> builtins.getContext (lib.strings.addContextFrom pkgs.coreutils "bar")
{
"/nix/store/m1s1d2dk2dqqlw3j90jl3cjy2cykbdxz-coreutils-9.5.drv" = { ... };
}
```
:::
*/
addContextFrom = src: target: builtins.substring 0 0 src + target;
/**
Cut a string with a separator and produces a list of strings which
were separated by this separator.
# Inputs
`sep`
: 1\. Function argument
`s`
: 2\. Function argument
# Type
```
splitString :: string -> string -> [string]
```
# Examples
:::{.example}
## `lib.strings.splitString` usage example
```nix
splitString "." "foo.bar.baz"
=> [ "foo" "bar" "baz" ]
splitString "/" "/usr/local/bin"
=> [ "" "usr" "local" "bin" ]
```
:::
*/
splitString =
sep: s:
let
splits = builtins.filter builtins.isString (
builtins.split (escapeRegex (builtins.toString sep)) (builtins.toString s)
);
in
builtins.map (addContextFrom s) splits;
/**
Remove duplicate elements from the `list`. O(n^2) complexity.
# Inputs
`list`
: Input list
# Type
```
unique :: [a] -> [a]
```
# Examples
:::{.example}
## `lib.lists.unique` usage example
```nix
unique [ 3 2 3 4 ]
=> [ 3 2 4 ]
```
:::
*/
unique = builtins.foldl' (acc: e: if builtins.elem e acc then acc else acc ++ [ e ]) [ ];
/**
Flip the order of the arguments of a binary function.
# Inputs
`f`
: 1\. Function argument
`a`
: 2\. Function argument
`b`
: 3\. Function argument
# Type
```
flip :: (a -> b -> c) -> (b -> a -> c)
```
# Examples
:::{.example}
## `lib.trivial.flip` usage example
```nix
flip concat [1] [2]
=> [ 2 1 ]
```
:::
*/
flip =
f: a: b:
f b a;
/**
`warn` *`message`* *`value`*
Print a warning before returning the second argument.
See [`builtins.warn`](https://nix.dev/manual/nix/latest/language/builtins.html#builtins-warn) (Nix >= 2.23).
On older versions, the Nix 2.23 behavior is emulated with [`builtins.trace`](https://nix.dev/manual/nix/latest/language/builtins.html#builtins-warn), including the [`NIX_ABORT_ON_WARN`](https://nix.dev/manual/nix/latest/command-ref/conf-file#conf-abort-on-warn) behavior, but not the `nix.conf` setting or command line option.
# Inputs
*`message`* (String)
: Warning message to print before evaluating *`value`*.
*`value`* (any value)
: Value to return as-is.
# Type
```
String -> a -> a
```
*/
warn =
# Since Nix 2.23, https://github.com/NixOS/nix/pull/10592
builtins.warn or (
let
mustAbort = builtins.elem (builtins.getEnv "NIX_ABORT_ON_WARN") [
"1"
"true"
"yes"
];
in
# Do not eta reduce v, so that we have the same strictness as `builtins.warn`.
msg: v:
# `builtins.warn` requires a string message, so we enforce that in our implementation, so that callers aren't accidentally incompatible with newer Nix versions.
assert builtins.isString msg;
if mustAbort then
builtins.trace "evaluation warning: ${msg}" (
abort "NIX_ABORT_ON_WARN=true; warnings are treated as unrecoverable errors."
)
else
builtins.trace "evaluation warning: ${msg}" v
);
}

22
machines/nixos/vault01/k-radius/default.nix
View file

@ -2,7 +2,16 @@
#
# SPDX-License-Identifier: EUPL-1.2
{ config, ... }:
{
config,
lib,
meta,
...
}:
let
inherit (lib) filterAttrs mapAttrsToList;
in
{
imports = [ ./module.nix ];
@ -40,13 +49,14 @@
radius_required_groups = [ "radius_access@sso.dgnum.eu" ];
# A mapping between Kanidm groups and VLANS
radius_groups = map (
{ vlan, ... }:
radius_groups = mapAttrsToList (
_:
{ id, ... }:
{
inherit vlan;
spn = "vlan_${toString vlan}@sso.dgnum.eu";
vlan = id;
spn = "vlan_${builtins.toString id}@sso.dgnum.eu";
}
) config.networking.vlans-info;
) (filterAttrs (_: { userOnly, ... }: userOnly) meta.isp.vlans);
};
authTokenFile = config.age.secrets."radius-auth_token_file".path;

184
machines/nixos/vault01/networking.nix
View file

@ -3,165 +3,38 @@
# SPDX-License-Identifier: EUPL-1.2
{
pkgs,
lib,
meta,
name,
config,
lib,
pkgs,
meta,
...
}:
let
inherit (lib) mapAttrs' mkOption nameValuePair;
inherit (lib.types) listOf attrs;
inherit (lib)
concatMapStringsSep
filterAttrs
mapAttrs'
nameValuePair
;
uplink = {
ip = "10.120.33.250";
prefix = 30;
router = "10.120.33.249";
};
mkNetwork =
name:
{
address ? [ ],
extraNetwork ? { },
...
}:
nameValuePair "10-${name}" ({ inherit name address; } // extraNetwork);
mkNetwork = name: { settings, ... }: nameValuePair "10-${name}" ({ inherit name; } // settings);
mkNetdev =
name:
{ Id, ... }:
{ id, ... }:
nameValuePair "10-${name}" {
netdevConfig = {
Name = name;
Kind = "vlan";
};
vlanConfig.Id = Id;
vlanConfig.Id = id;
};
mkUserVlan =
{
vlan,
netIP,
servIP,
interfaceName,
...
}:
{
name = interfaceName;
value = {
Id = vlan;
extraNetwork = {
networkConfig = {
LinkLocalAddressing = "no";
DHCPServer = "yes";
};
linkConfig = {
Promiscuous = true;
MTUBytes = 1500;
};
addresses = [
{
Address = "${servIP}/27";
AddPrefixRoute = false;
}
];
routes = [
{
Destination = "${netIP}/27";
Table = "user";
}
];
};
};
};
userVlans = builtins.genList (id: rec {
vlan = 4094 - id;
prefix24nb = (id + 1) / 8;
prefix27nb = (id + 1 - prefix24nb * 8) * 32;
netIP = "10.0.${toString prefix24nb}.${toString prefix27nb}";
servIP = "10.0.${toString prefix24nb}.${toString (prefix27nb + 1)}";
interfaceName = "vlan-user-${toString vlan}";
prefixLen = 27;
}) 850;
vlans = {
vlan-uplink-cri = {
Id = 223;
address = with uplink; [ "${ip}/${builtins.toString prefix}" ];
extraNetwork = {
routes = [
{
# Get the public ip from the metadata
PreferredSource = builtins.head meta.network.${name}.addresses.ipv4;
Gateway = uplink.router;
}
];
linkConfig.MTUBytes = 1500;
};
};
vlan-admin = {
Id = 3000;
address = [ "fd26:baf9:d250:8000::1/64" ];
};
vlan-admin-ap = {
Id = 3001;
address = [
"fd26:baf9:d250:8001::1/64"
# FIXME: ipv4 is temporary for APs in production
"10.0.253.1/24"
];
extraNetwork = {
networkConfig = {
IPv6SendRA = true;
DHCPServer = "yes";
};
ipv6Prefixes = [
{
AddressAutoconfiguration = false;
OnLink = false;
Prefix = "fd26:baf9:d250:8001::/64";
}
];
};
};
vlan-apro = {
Id = 2000;
address = [ "10.0.255.1/24" ];
extraNetwork = {
networkConfig.DHCPServer = "yes";
linkConfig.MTUBytes = 1500;
};
};
vlan-hypervisor = {
Id = 2001;
address = [ "10.0.254.1/24" ];
extraNetwork = {
networkConfig.DHCPServer = "yes";
linkConfig.MTUBytes = 1500;
};
};
} // builtins.listToAttrs (map mkUserVlan userVlans);
vlans = mapAttrs' (name: nameValuePair "vlan-${name}") meta.isp.vlans;
in
{
options.networking.vlans-info = mkOption {
type = listOf attrs;
description = ''
Information about vlans for log analysis.
'';
readOnly = true;
};
config = {
lbailly commented 2025-02-10 15:58:23 +01:00
Review
Copy link

Si on retire l'option (ce qui est le but), on peut retirer ce config =

Si on retire l'option (ce qui est le but), on peut retirer ce `config =`
systemd = {
network = {
@ -274,7 +147,7 @@ in
}
];
};
} // mapAttrs' mkNetdev vlans;
} // (mapAttrs' mkNetdev vlans);
};
services = {
@ -296,20 +169,21 @@ in
systemd-networkd.serviceConfig.LimitNOFILE = 4096;
net-checker = {
net-checker =
let
userVlans = builtins.attrNames (filterAttrs (_: { userOnly, ... }: userOnly) vlans);
networkctl = action: concatMapStringsSep "\n " (name: "networkctl ${action} ${name}") userVlans;
in
{
path = [
pkgs.iputils
pkgs.systemd
];
script = ''
if ping -c 1 8.8.8.8 > /dev/null || ping -c 1 1.1.1.1 > /dev/null; then
${lib.concatMapStringsSep "\n " (
{ interfaceName, ... }: "networkctl up ${interfaceName}"
) userVlans}
${networkctl "up"}
else
${lib.concatMapStringsSep "\n " (
{ interfaceName, ... }: "networkctl down ${interfaceName}"
) userVlans}
${networkctl "down"}
fi
'';
};
@ -322,18 +196,6 @@ in
};
networking = {
vlans-info = [
{
vlan = 2001;
netIP = "10.0.254.0";
prefixLen = 24;
}
{
vlan = 3001;
netIP = "10.0.253.0";
prefixLen = 24;
}
] ++ userVlans;
nftables = {
enable = true;
tables = {

4
machines/nixos/vault01/ulogd/default.nix
View file

@ -6,8 +6,10 @@
config,
lib,
pkgs,
meta,
...
}:
{
services = {
ulogd = {
@ -59,7 +61,7 @@
};
environment.defaultPackages = [
(pkgs.callPackage ./fill-vlan_prefixes.nix {
inherit (config.networking) vlans-info;
inherit (meta.isp) vlans;
postgresql = config.services.postgresql.package;
})
(pkgs.callPackage ./nat-request-daddr.nix {

31
machines/nixos/vault01/ulogd/fill-vlan_prefixes.nix
View file

@ -6,12 +6,19 @@
lib,
writeShellApplication,
writeText,
vlans-info,
vlans,
postgresql,
}:
let
inherit (lib) concatMapStringsSep;
sql-script = writeText "vlan-filling.sql" ''
inherit (lib) attrValues filter concatMapStringsSep;
in
writeShellApplication {
name = "fill-vlan_prefixes";
runtimeInputs = [ postgresql ];
text = ''
psql -d ulogd -U ulogd -f ${writeText "vlan-filling.sql" ''
DROP TABLE IF EXISTS vlan_prefixes;
CREATE TABLE vlan_prefixes (
vlan_id smallint PRIMARY KEY UNIQUE NOT NULL,
@ -19,21 +26,9 @@ let
);
INSERT INTO vlan_prefixes VALUES
${concatMapStringsSep ",\n " (
{
vlan,
netIP,
prefixLen,
...
}:
"(${toString vlan}, inet '${netIP}/${toString prefixLen}')"
) vlans-info}
vlan: "(${builtins.toString vlan.id}, inet '${vlan.internal.cidr}')"
) (filter ({ internal, ... }: internal.cidr != null) (attrValues vlans))}
;
'';
in
writeShellApplication {
name = "fill-vlan_prefixes";
runtimeInputs = [ postgresql ];
text = ''
psql -d ulogd -U ulogd -f ${sql-script}
''}
'';
}

11
meta/default.nix
View file

@ -10,11 +10,12 @@
lib:
(lib.evalModules {
modules = [
./options.nix
./network.nix
./nodes
./organization.nix
modules = lib.extra.mkImports ./. [
"isp"
"network"
"nodes"
"options"
"organization"
];
class = "dgnumMeta";
}).config

131
meta/isp/default.nix Normal file
View file

@ -0,0 +1,131 @@
# SPDX-FileCopyrightText: 2024 Tom Hubrecht <tom.hubrecht@dgnum.eu>
# SPDX-FileCopyrightText: 2025 Lubin Bailly <lubin.bailly@dgnum.eu>
#
# SPDX-License-Identifier: EUPL-1.2
{ config, lib, ... }:
let
inherit (lib) genList listToAttrs nameValuePair;
mkCIDR = address: prefix: "${address}/${builtins.toString prefix}";
in
{
imports = [ ./module.nix ];
isp = {
vlans =
{
uplink-cri = {
id = 223;
settings = {
address = [ (mkCIDR "10.120.33.250" 30) ];
routes = [
{
PreferredSource = builtins.head config.network.vault01.addresses.ipv4;
Gateway = "10.120.33.249";
}
];
linkConfig.MTUBytes = 1500;
};
};
admin = {
id = 3000;
settings = {
address = [ "fd26:baf9:d250:8000::1/64" ];
};
};
admin-ap = {
id = 3001;
settings = {
address = [
"fd26:baf9:d250:8001::1/64"
# FIXME: ipv4 is temporary for APs in production
"10.0.253.1/24"
];
networkConfig = {
IPv6SendRA = true;
DHCPServer = "yes";
};
ipv6Prefixes = [
{
AddressAutoconfiguration = false;
OnLink = false;
Prefix = "fd26:baf9:d250:8001::/64";
}
];
};
internal = {
network = "10.0.253.0";
prefix = 24;
};
};
apro = {
id = 2000;
settings = {
address = [ "10.0.255.1/24" ];
networkConfig.DHCPServer = "yes";
linkConfig.MTUBytes = 1500;
};
};
hypervisor = {
id = 2001;
settings = {
address = [ "10.0.254.1/24" ];
networkConfig.DHCPServer = "yes";
linkConfig.MTUBytes = 1500;
};
internal = {
network = "10.0.254.0";
prefix = 24;
};
};
}
// listToAttrs (
genList (
base:
let
id = (4096 - 2) - base;
range24 = (base + 1) / 8;
range27 = (base + 1 - range24 * 8) * 32;
in
nameValuePair "user-${builtins.toString id}" rec {
inherit id;
internal = {
network = "10.0.${builtins.toString range24}.${builtins.toString range27}";
address = "10.0.${builtins.toString range24}.${builtins.toString (range27 + 1)}";
prefix = 27;
};
settings = {
networkConfig = {
LinkLocalAddressing = "no";
DHCPServer = "yes";
};
linkConfig = {
Promiscuous = true;
MTUBytes = 1500;
};
addresses = [
{
Address = mkCIDR internal.address internal.prefix;
AddPrefixRoute = false;
}
];
routes = [
{
Destination = mkCIDR internal.network internal.prefix;
Table = "user";
}
];
};
userOnly = true;
}
) 850
);
};
}

106
meta/isp/module.nix Normal file
View file

@ -0,0 +1,106 @@
# SPDX-FileCopyrightText: 2024 Tom Hubrecht <tom.hubrecht@dgnum.eu>
#
# SPDX-License-Identifier: EUPL-1.2
{ lib, ... }:
let
inherit (lib)
mkDefault
mkIf
mkOption
;
inherit (lib.types)
attrs
attrsOf
bool
ints
nullOr
submodule
str
;
in
{
options.isp = {
vlans = mkOption {
type = attrsOf (
submodule (
{ config, ... }:
{
options = {
id = mkOption {
type = ints.between 0 (4096 - 1);
description = ''
The VLAN id to use.
'';
};
settings = mkOption {
type = attrs;
default = { };
description = ''
Settings for the configuration of networkd.
'';
};
lbailly commented 2025-02-10 15:57:21 +01:00
Review
Copy link

Je suis très peu convaincu de l'idée de mettre de la configuration systemd-networkd dans le meta, surtout pour des choses type MTU, adresses ou DHCP.
Notamment, il faudrait garantir que c'est utilisé à un unique endroit, ce qui me parait peu scallable, sans apporter beaucoup au meta, vu qu'on est sensé déjà avoir mis toute les informations pertinente dans les autres fields.

Je suis très peu convaincu de l'idée de mettre de la configuration systemd-networkd dans le meta, surtout pour des choses type MTU, adresses ou DHCP. Notamment, il faudrait garantir que c'est utilisé à un unique endroit, ce qui me parait peu scallable, sans apporter beaucoup au meta, vu qu'on est sensé déjà avoir mis toute les informations pertinente dans les autres fields.
internal = {
network = mkOption {
type = nullOr str;
description = ''
The internal network address of this VLAN.
'';
};
prefix = mkOption {
type = nullOr (ints.between 0 32);
default = if config.userOnly then 27 else null;
description = ''
The prefix length of the network associated to this VLAN.
'';
};
address = mkOption {
type = nullOr str;
description = ''
The router address in the VLAN. It should be the first ipv4 in the network.
'';
};
cidr = mkOption {
type = nullOr str;
default =
with config.internal;
if (prefix != null && network != null) then "${network}/${builtins.toString prefix}" else null;
description = ''
The CIDR notation of the network associated to the VLAN.
'';
};
};
userOnly = mkOption {
type = bool;
default = false;
description = ''
Whether this VLAN is only used by a user in the context of the IPS.
I.e. this is not an administration VLAN.
'';
};
};
# The address is null by default when not on a user VLAN
config.internal = mkIf (!config.userOnly) {
address = mkDefault null;
network = mkDefault null;
};
}
)
);
default = [ ];
description = ''
The list of VLANs known to our ISP.
'';
};
};
}

2
meta/options.nix
View file

@ -121,6 +121,7 @@ in
'';
};
# FIXME: Unused
vpnKeys = mkOption {
type = attrsOf vpnKeyType;
default = { };
@ -404,6 +405,7 @@ in
'';
};
# FIXME: Unused
vpnKeys = mkOption {
type = attrsOf vpnKeyType;
default = { };