7.9 KiB
Liminix
A Nix-based system for configuring consumer wifi routers.
What is this?
This is a Nix-based collection of software tailored for domestic wifi router or IoT device devices, of the kind that OpenWrt or DD-WRT or Gargoyle or Tomato run on. It's a reboot/restart/rewrite of NixWRT.
This is not NixOS-on-your-router: it's aimed at devices that are underpowered for the full NixOS experience. It uses busybox tools, musl instead of GNU libc, and s6-rc instead of systemd.
The Liminix name comes from Liminis, in Latin the genitive declension of "limen", or "of the threshold". Your router stands at the threshold of your (online) home and everything you send to/receive from the outside word goes across it.
What about NixWRT?
This is an in-progress rewrite of NixWRT, incorporating Lessons Learned. That said, as of today (September 2022) it is not yet anywhere near feature parity.
Liminix will eventually provide these differentiators over NixWRT:
-
a writable filesystem so that software updates or reconfiguration (e.g. changing passwords) don't require taking the device offline to reflash it.
-
more flexible service management with dependencies, to allow configurations such as "route through PPPoE if it is healthy, with fallback to LTE"
-
a spec for valid configuration options (a la NixOS module options) to that we can detect errors at evaluation time instead of producing a bad image.
-
a network-based mechanism for secrets management so that changes can be pushed from a central location to several Liminix devices at once
-
send device metrics and logs to a monitoring/alerting/o11y infrastructure
Today though, it does approximately none of these things and certainly not on real hardware.
Building
For the device
These instructions assume you have nixpkgs checked out in a peer directory of this one.
You need a configuration.nix file pointed to by <liminix-config>, a
hardware device definition as argument device, and to choose an
appropriate output attribute depending on what your device is and how
you plan to install onto it. For example:
NIX_PATH=nixpkgs=../nixpkgs:$NIX_PATH NIXPKGS_ALLOW_UNSUPPORTED_SYSTEM=1 nix-build -I liminix-config=./tests/smoke/configuration.nix --arg device "import ./devices/qemu" -A outputs.default
outputs.default is intended to do something appropriate for the
device, whatever that is. For the qemu device, it creates a directory
containing a squashfs root image and a kernel.
For the build machine
Liminix also includes some tools intended for the build machine. You can run
nix-shell -A buildEnv --arg device '(import ./devices/qemu)'
to get a shell environment with (currently) a tftp server and a script to start a PPPoE server in QEMU for testing against.
QEMU
QEMU is useful for developing userland without needing to keep flashing or messing with U-Boot: it also enables testing against emulated network peers using QEMU socket networking, which may be preferable to letting Liminix loose on your actual LAN.
We observe these conventions for QEMU network sockets, so that we can run multiple emulated instances and have them wired up to each other in the right way
- multicast 230.0.0.1:1234 : access (interconnect between router and "isp")
- multicast 230.0.0.1:1235 : lan
- multicast 230.0.0.1:1236 : world (the internet)
Running Liminix in Qemu
In a buildEnv nix-shell, you can use the mips-vm command
to run Qemu with appropriate config for two ethernet interfaces
hooked up to "lan" and "access" respectively. It connects the Liminix
serial console and the QEMU monitor to stdin/stdout. Use ^P (not ^A) to switch to the monitor.
nix-shell -A buildEnv --arg device '(import ./devices/qemu)' --run "mips-vm result/vmlinux result/squashfs"
If you run with --background /path/to/some/directory as the first
parameter, it will fork into the background and open Unix sockets in
that directory for console and monitor. Use connect-vm (also in the
buildEnv environment) to connect to either of these sockets, and ^O
to disconnect.
Emulated upstream connection
In pkgs/routeros there is a derivation to install and configure
Mikrotik RouterOS as a PPPoE access
concentrator connected to the access and world networks, so that
Liminix PPPoE client support can be tested.
This is made available in the buildEnv, so you can do something like
mkdir ros-sockets
nix-shell -A buildEnv --arg device '(import ./devices/qemu)'
nix-shell$ routeros ros-sockets
nix-shell$ connect-vm ./ros-sockets/console
to start it and connect to it.
_Liminix does not provide RouterOS licences and it is your own responsibility if you use this to ensure you're compliant with the terms of Mikrotik's licencing._It may be supplemented or replaced in time with configurations for RP-PPPoE and/or Accel PPP.
Running tests
Assuming you have nixpkgs checked out in a peer directory of this one,
you can run all of the tests by evaluating ci.nix:
nix-build --argstr liminix `pwd` --argstr nixpkgs `pwd`/../nixpkgs --argstr unstable `pwd`/../unstable-nixpkgs/ ci.nix
or to run a named test, use the -A flag. For example, -A pppoe
Hardware
How you get the thing onto hardware will vary according to the device, but is likely to involve U-Boot and TFTP.
There is a rudimentary TFTP server bundled with the system which runs
from the command line, has an allowlist for client connections, and
follows symlinks, so you can have your device download images direct
from the ./result directory without exposing /nix/store/ to the
internet or mucking about copying files to /tftproot. If the
permitted device is to be given the IP address 192.168.8.251 you might
do something like this:
nix-shell -A buildEnv --arg device '(import ./devices/qemu)' \
--run "tufted -a 192.168.8.251 result"
Troubleshooting
Diagnosing unexpectedly large images
Sometimes you can add a package and it causes the image size to balloon
because it has dependencies on other things you didn't know about. Build the
outputs.manifest attribute, which is a json representation of the
filesystem, and you can run nix-store --query on it.
NIX_PATH=nixpkgs=../nixpkgs:$NIX_PATH nix-build -I liminix-config=path/to/your/configuration.nix --arg device "import ./devices/qemu" -A outputs.manifest -o manifest
nix-store -q --tree manifest
Contributing
Contributions are welcome, though in these early days there may be a bit of back and forth involved before patches are merged. Have a read of CONTRIBUTING and STYLE and try to intuit the unarticulated vision :-)
Liminix' primary repo is https://gti.telent.net/dan/liminix. There's a mirror on Github for convenience and visibility: you can open PRs against that but be aware that the process of merging them may be arcane. Some day, we will have federated Gitea using ActivityPub.
Articles of interest
-
Build Safety of Software in 28 Popular Home Routers: "of the access points and routers we reviewed, not a single one took full advantage of the basic application armoring features provided by the operating system. Indeed, only one or two models even came close, and no brand did well consistently across all models tested"
-
A PPPoE Implementation for Linux: "Many DSL service providers use PPPoE for residential broadband Internet access. This paper briefly describes the PPPoE protocol, presents strategies for implementing it under Linux and describes in detail a user-space implementation of a PPPoE client."