forked from DGNum/liminix
doc WIP: build "hello net" example
This commit is contained in:
parent
35c7f1643f
commit
c6faf88dd1
6 changed files with 165 additions and 738 deletions
|
@ -1,336 +0,0 @@
|
||||||
Developer Manual
|
|
||||||
################
|
|
||||||
|
|
||||||
As a developer working on Liminix, or implementing a service or
|
|
||||||
module, you probably want to test your changes more conveniently
|
|
||||||
than by building and flashing a new image every time. This manual
|
|
||||||
documents various affordances for iteration and experiments.
|
|
||||||
|
|
||||||
In general, packages and tools that run on the "build" machine are
|
|
||||||
available in the ``buildEnv`` derivation and can most easily
|
|
||||||
be added to your environment by running :command:`nix-shell`
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
Emulated devices
|
|
||||||
****************
|
|
||||||
|
|
||||||
Liminix has a ``qemu`` device, which generates images suitable for
|
|
||||||
running on your build machine using the free `QEMU machine emulator <http://www.qemu.org>`_.
|
|
||||||
This is useful for developing userland without needing to keep
|
|
||||||
flashing or messing with U-Boot: it also enables testing against
|
|
||||||
emulated network peers using `QEMU socket networking <https://wiki.qemu.org/Documentation/Networking#Socket>`_,
|
|
||||||
which may be preferable to letting Liminix loose on your actual LAN.
|
|
||||||
To build it,
|
|
||||||
|
|
||||||
.. code-block:: console
|
|
||||||
|
|
||||||
nix-build -I liminix-config=path/to/your/configuration.nix --arg device "import ./devices/qemu" -A outputs.default
|
|
||||||
|
|
||||||
In a ``buildEnv`` nix-shell, you can use the :command:`mips-vm` command
|
|
||||||
to run Qemu with appropriate options. It connects the Liminix
|
|
||||||
serial console and the `QEMU monitor <https://www.qemu.org/docs/master/system/monitor.html>`_ to stdin/stdout. Use ^P (not ^A) to switch to the monitor.
|
|
||||||
|
|
||||||
.. code-block:: console
|
|
||||||
|
|
||||||
nix-shell --run "mips-vm result/vmlinux result/squashfs"
|
|
||||||
|
|
||||||
If you run with ``--background /path/to/some/directory`` as the first
|
|
||||||
parameter, it will fork into the background and open Unix sockets in
|
|
||||||
that directory for console and monitor. Use :command:`connect-vm`
|
|
||||||
(also in the ``buildEnv`` environment) to connect to either of these
|
|
||||||
sockets, and ^O to disconnect.
|
|
||||||
|
|
||||||
Networking
|
|
||||||
==========
|
|
||||||
|
|
||||||
VMs can network with each other using QEMU
|
|
||||||
socket networking. We observe these conventions, so that we can run
|
|
||||||
multiple emulated instances and have them wired up to each other in
|
|
||||||
the right way:
|
|
||||||
|
|
||||||
* multicast 230.0.0.1:1234 : access (interconnect between router and "isp")
|
|
||||||
* multicast 230.0.0.1:1235 : lan
|
|
||||||
* multicast 230.0.0.1:1236 : world (the internet)
|
|
||||||
|
|
||||||
A VM started with :command:`mips-vm` is connected to "lan" and "access", and
|
|
||||||
the emulated border network gateway (see below) runs PPPoE and is
|
|
||||||
connected to "access" and "world".
|
|
||||||
|
|
||||||
Border Network Gateway
|
|
||||||
----------------------
|
|
||||||
|
|
||||||
In pkgs/routeros there is a derivation to install and configure
|
|
||||||
`Mikrotik RouterOS <https://mikrotik.com/software>`_ as a PPPoE access
|
|
||||||
concentrator connected to the ``access`` and ``world`` networks, so that
|
|
||||||
Liminix PPPoE client support can be tested without actual hardware.
|
|
||||||
|
|
||||||
This is made available as the :command:`routeros` command in
|
|
||||||
``buildEnv``, so you can do something like::
|
|
||||||
|
|
||||||
mkdir ros-sockets
|
|
||||||
nix-shell
|
|
||||||
nix-shell$ routeros ros-sockets
|
|
||||||
nix-shell$ connect-vm ./ros-sockets/console
|
|
||||||
|
|
||||||
to start it and connect to it. Note that by default it runs in the
|
|
||||||
background. It is connected to "access" and "world" virtual networks
|
|
||||||
and runs a PPPoE service on "access" - so a Liminix VM with a
|
|
||||||
PPPOE client can connect to it and thus reach the virtual internet.
|
|
||||||
[ check, but pretty sure this is not the actual internet ]
|
|
||||||
|
|
||||||
`Liminix does not provide RouterOS licences and it is your own
|
|
||||||
responsibility if you use this to ensure you're compliant with the
|
|
||||||
terms of Mikrotik's licencing. It may be supplemented or replaced in
|
|
||||||
time with configurations for RP-PPPoE and/or Accel PPP.`
|
|
||||||
|
|
||||||
Hardware devices
|
|
||||||
****************
|
|
||||||
|
|
||||||
How you get your image onto hardware will vary according to the
|
|
||||||
device, but is likely to involve taking it apart to add wires to
|
|
||||||
serial console pads/headers, then using U-Boot to fetch images over
|
|
||||||
TFTP. The OpenWrt documentation has a `good explanation <https://openwrt.org/docs/techref/hardware/port.serial>`_ of what you may expect to find on
|
|
||||||
the device.
|
|
||||||
|
|
||||||
There is a rudimentary TFTP server bundled with the system which runs
|
|
||||||
from the command line, has an allowlist for client connections, and
|
|
||||||
follows symlinks, so you can have your device download images direct
|
|
||||||
from the :file:`./result` directory without exposing :file:`/nix/store/` to the
|
|
||||||
internet or mucking about copying files to :file:`/tftproot`. If the
|
|
||||||
permitted device is to be given the IP address 192.168.8.251 you might
|
|
||||||
do something like this:
|
|
||||||
|
|
||||||
.. code-block:: console
|
|
||||||
|
|
||||||
nix-shell --run "tufted -a 192.168.8.251 result"
|
|
||||||
|
|
||||||
Now add the device and server IP addresses to your configuration:
|
|
||||||
|
|
||||||
.. code-block:: nix
|
|
||||||
|
|
||||||
boot.tftp = {
|
|
||||||
serverip = "192.168.8.111";
|
|
||||||
ipaddr = "192.168.8.251";
|
|
||||||
};
|
|
||||||
|
|
||||||
and then build the derivation for ``outputs.default`` or
|
|
||||||
``outputs.flashimage`` (for which it will be an alias on any device
|
|
||||||
where this is applicable). You should find it has created
|
|
||||||
|
|
||||||
* :file:`result/firmware.bin` which is the file you are going to flash
|
|
||||||
* :file:`result/flash.scr` which is a set of instructions to U-Boot to
|
|
||||||
download the image and write it to flash after erasing the appropriate
|
|
||||||
flash partition.
|
|
||||||
|
|
||||||
.. NOTE::
|
|
||||||
|
|
||||||
TTL serial connections typically have no form of flow control and
|
|
||||||
so don't always like having massive chunks of text pasted into
|
|
||||||
them - and U-Boot may drop characters while it's busy. So don't
|
|
||||||
necessarily expect to copy-paste the whole of :file:`boot.scr` into
|
|
||||||
a terminal emulator and have it work just like that. You may need
|
|
||||||
to paste each line one at a time, or even retype it.
|
|
||||||
|
|
||||||
|
|
||||||
For a faster edit-compile-test cycle, you can build a TFTP-bootable
|
|
||||||
image instead of flashing. In your device configuration add
|
|
||||||
|
|
||||||
.. code-block:: nix
|
|
||||||
|
|
||||||
imports = [
|
|
||||||
./modules/tftpboot.nix
|
|
||||||
];
|
|
||||||
|
|
||||||
and then build ``outputs.tftpboot``. This creates a file in
|
|
||||||
``result/`` called ``boot.scr``, which you can copy and paste into
|
|
||||||
U-Boot to transfer the kernel and filesystem over TFTP and boot the
|
|
||||||
kernel from RAM.
|
|
||||||
|
|
||||||
|
|
||||||
Networking
|
|
||||||
==========
|
|
||||||
|
|
||||||
You probably don't want to be testing a device that might serve DHCP,
|
|
||||||
DNS and routing protocols on the same LAN as you (or your colleagues,
|
|
||||||
employees, or family) are using for anything else, because it will
|
|
||||||
interfere. You also might want to test the device against an
|
|
||||||
"upstream" connection without having to unplug your regular home
|
|
||||||
router from the internet so you can borrow the cable/fibre/DSL.
|
|
||||||
|
|
||||||
``bordervm`` is included for this purpose. You will need
|
|
||||||
|
|
||||||
* a Linux machine with a spare (PCI or USB) ethernet device which you can dedicate to Liminix
|
|
||||||
|
|
||||||
* an L2TP service such as https://www.aa.net.uk/broadband/l2tp-service/
|
|
||||||
|
|
||||||
You need to "hide" the Ethernet device from the host - for PCI this
|
|
||||||
means configuring it for VFIO passthru; for USB you need to unload the
|
|
||||||
module(s) it uses. I have this segment in configuration.nix which you
|
|
||||||
may be able to adapt:
|
|
||||||
|
|
||||||
.. code-block:: nix
|
|
||||||
|
|
||||||
boot = {
|
|
||||||
kernelParams = [ "intel_iommu=on" ];
|
|
||||||
kernelModules = [
|
|
||||||
"kvm-intel" "vfio_virqfd" "vfio_pci" "vfio_iommu_type1" "vfio"
|
|
||||||
];
|
|
||||||
|
|
||||||
postBootCommands = ''
|
|
||||||
# modprobe -i vfio-pci
|
|
||||||
# echo vfio-pci > /sys/bus/pci/devices/0000:01:00.0/driver_override
|
|
||||||
'';
|
|
||||||
blacklistedKernelModules = [
|
|
||||||
"r8153_ecm" "cdc_ether"
|
|
||||||
];
|
|
||||||
};
|
|
||||||
services.udev.extraRules = ''
|
|
||||||
SUBSYSTEM=="usb", ATTRS{idVendor}=="0bda", ATTRS{idProduct}=="8153", OWNER="dan"
|
|
||||||
'';
|
|
||||||
|
|
||||||
Then
|
|
||||||
you can execute :command:`run-border-vm` in a ``buildEnv`` shell,
|
|
||||||
which starts up QEMU using the NixOS configuration in
|
|
||||||
:file:`bordervm-configuration.nix`.
|
|
||||||
|
|
||||||
In this VM
|
|
||||||
|
|
||||||
* your Liminix checkout is mounted under :file:`/home/liminix/liminix`
|
|
||||||
|
|
||||||
* TFTP is listening on the ethernet device and serving
|
|
||||||
:file:`/home/liminix/liminix`. The server IP address is 10.0.0.1
|
|
||||||
|
|
||||||
* a PPPOE-L2TP relay is running on the same ethernet card. When the
|
|
||||||
connected Liminix device makes PPPoE requests, the relay spawns
|
|
||||||
L2TPv2 Access Concentrator sessions to your specified L2TP LNS.
|
|
||||||
Note that authentication is expected at the PPP layer not the L2TP
|
|
||||||
layer, so the PAP/CHAP credentials provided by your L2TP service can
|
|
||||||
be configured into your test device - bordervm doesn't need to know
|
|
||||||
about them.
|
|
||||||
|
|
||||||
To configure bordervm, you need a file called :file:`bordervm.conf.nix`
|
|
||||||
which you can create by copying and appropriately editing :file:`bordervm.conf-example.nix`
|
|
||||||
|
|
||||||
.. note::
|
|
||||||
|
|
||||||
If you make changes to the bordervm configuration after executing
|
|
||||||
:command:`run-border-vm`, you need to remove the :file:`border.qcow2` disk
|
|
||||||
image file otherwise the changes won't get picked up.
|
|
||||||
|
|
||||||
|
|
||||||
Running tests
|
|
||||||
*************
|
|
||||||
|
|
||||||
You can run all of the tests by evaluating :file:`ci.nix`, which is the
|
|
||||||
input I use in Hydra. Note that it expects Nixpkgs stable `and` unstable
|
|
||||||
as inputs, because it builds the qemu device against both.
|
|
||||||
|
|
||||||
.. code-block:: console
|
|
||||||
|
|
||||||
nix-build --argstr liminix `pwd` --arg nixpkgs "<nixpkgs>" \
|
|
||||||
--argstr unstable `pwd`/../unstable-nixpkgs/ ci.nix
|
|
||||||
|
|
||||||
To run a single named test, use the ``-A`` flag. For example, ``-A pppoe``
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
Troubleshooting
|
|
||||||
***************
|
|
||||||
|
|
||||||
Diagnosing unexpectedly large images
|
|
||||||
====================================
|
|
||||||
|
|
||||||
Sometimes you can add a package and it causes the image size to balloon
|
|
||||||
because it has dependencies on other things you didn't know about. Build the
|
|
||||||
``outputs.manifest`` attribute, which is a JSON representation of the
|
|
||||||
filesystem, and you can run :command:`nix-store --query` on it.
|
|
||||||
|
|
||||||
.. code-block:: console
|
|
||||||
|
|
||||||
nix-build -I liminix-config=path/to/your/configuration.nix \
|
|
||||||
--arg device "import ./devices/qemu" -A outputs.manifest \
|
|
||||||
-o manifest
|
|
||||||
nix-store -q --tree manifest
|
|
||||||
|
|
||||||
|
|
||||||
Contributing
|
|
||||||
************
|
|
||||||
|
|
||||||
Contributions are welcome, though in these early days there may be a
|
|
||||||
bit of back and forth involved before patches are merged:
|
|
||||||
Please get in touch somehow `before` you invest a lot of time into a
|
|
||||||
code contribution I haven't asked for. Just so I know it's expected
|
|
||||||
and you're not wasting time doing something I won't accept or have
|
|
||||||
already started on.
|
|
||||||
|
|
||||||
|
|
||||||
Nix language style
|
|
||||||
==================
|
|
||||||
|
|
||||||
This section describes some Nix language style points that we
|
|
||||||
attempt to adhere to in this repo.
|
|
||||||
|
|
||||||
* favour ``callPackage`` over raw ``import`` for calling derivations
|
|
||||||
or any function that may generate one - any code that might need
|
|
||||||
``pkgs`` or parts of it.
|
|
||||||
|
|
||||||
* prefer ``let inherit (quark) up down strange charm`` over
|
|
||||||
``with quark``, in any context where the scope is more than a single
|
|
||||||
expression or there is more than one reference to ``up``, ``down``
|
|
||||||
etc. ``with pkgs; [ foo bar baz]`` is OK,
|
|
||||||
``with lib; stdenv.mkDerivation { ... }`` is usually not.
|
|
||||||
|
|
||||||
* ``<liminix>`` is defined only when running tests, so don't refer to it
|
|
||||||
in "application" code
|
|
||||||
|
|
||||||
* the parameters to a derivation are sorted alphabetically, except for
|
|
||||||
``lib``, ``stdenv`` and maybe other non-package "special cases"
|
|
||||||
|
|
||||||
* indentation is whatever emacs nix-mode says it is.
|
|
||||||
|
|
||||||
* where a ``let`` form defines multiple names, put a newline after the
|
|
||||||
token ``let``, and indent each name two characters
|
|
||||||
|
|
||||||
* to decide whether some code should be a package or a module?
|
|
||||||
Packages are self-contained - they live in ``/nix/store/eeeeeee-name``
|
|
||||||
and don't directly change system behaviour by their presence or
|
|
||||||
absense. modules can add to
|
|
||||||
``/etc`` or ``/bin`` or other global state, create services, all that
|
|
||||||
side-effecty stuff. Generally it should be a package unless it
|
|
||||||
can't be.
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
Copyright
|
|
||||||
=========
|
|
||||||
|
|
||||||
The Nix code in Liminix is MIT-licenced (same as Nixpkgs), but the
|
|
||||||
code it combines from other places (e.g. Linux, OpenWrt) may have a
|
|
||||||
variety of licences. I have no intention of asking for copyright
|
|
||||||
assignment: just like when submitting to the Linux kernel you retain
|
|
||||||
the copyright on the code you contribute.
|
|
||||||
|
|
||||||
|
|
||||||
Code of Conduct
|
|
||||||
===============
|
|
||||||
|
|
||||||
Please govern yourself in Liminix project venues according to the
|
|
||||||
`Code of Conduct <https://gti.telent.net/dan/liminix/src/commit/7bcf6b15c3fdddafeda13f65b3cd4a422dc52cd3/CODE-OF-CONDUCT.md>`_
|
|
||||||
|
|
||||||
|
|
||||||
Where to send patches
|
|
||||||
=====================
|
|
||||||
|
|
||||||
|
|
||||||
Liminix' primary repo is https://gti.telent.net/dan/liminix but you
|
|
||||||
can't send code there directly because it doesn't have open registrations.
|
|
||||||
|
|
||||||
* There's a `mirror on Github <https://github.com/telent/liminix>`_ for
|
|
||||||
convenience and visibility: you can open PRs against that
|
|
||||||
|
|
||||||
* or, you can send me your patch by email using `git send-email <https://git-send-email.io/>`_
|
|
||||||
|
|
||||||
* or in the future, some day, we will have federated Gitea using
|
|
||||||
ActivityPub.
|
|
51
doc/etc.rst
51
doc/etc.rst
|
@ -1,51 +0,0 @@
|
||||||
The Future
|
|
||||||
##########
|
|
||||||
|
|
||||||
What about NixWRT?
|
|
||||||
|
|
||||||
This is an in-progress rewrite of NixWRT, incorporating Lessons
|
|
||||||
Learned. That said, as of today it is not yet at feature parity.
|
|
||||||
|
|
||||||
Liminix will eventually provide these differentiators over NixWRT:
|
|
||||||
|
|
||||||
* a writable filesystem so that software updates or reconfiguration
|
|
||||||
(e.g. changing passwords) don't require taking the device offline to
|
|
||||||
reflash it.
|
|
||||||
|
|
||||||
* more flexible service management with dependencies, to allow
|
|
||||||
configurations such as "route through PPPoE if it is healthy, with
|
|
||||||
fallback to LTE"
|
|
||||||
|
|
||||||
* a spec for valid configuration options (a la NixOS module options)
|
|
||||||
to that we can detect errors at evaluation time instead of producing
|
|
||||||
a bad image.
|
|
||||||
|
|
||||||
* a network-based mechanism for secrets management so that changes can
|
|
||||||
be pushed from a central location to several Liminix devices at once
|
|
||||||
|
|
||||||
* send device metrics and logs to a monitoring/alerting/o11y
|
|
||||||
infrastructure
|
|
||||||
|
|
||||||
Today though, it does approximately none of these things and certainly
|
|
||||||
not on real hardware.
|
|
||||||
|
|
||||||
|
|
||||||
Articles of interest
|
|
||||||
####################
|
|
||||||
|
|
||||||
* `Build Safety of Software in 28 Popular Home Routers <https://cyber-itl.org/assets/papers/2018/build_safety_of_software_in_28_popular_home_routers.pdf>`_: "of the access
|
|
||||||
points and routers we reviewed, not a single one took full
|
|
||||||
advantage of the basic application armoring features provided by
|
|
||||||
the operating system. Indeed, only one or two models even came
|
|
||||||
close, and no brand did well consistently across all models tested"
|
|
||||||
|
|
||||||
* `A PPPoE Implementation for Linux <https://static.usenix.org/publications/library/proceedings/als00/2000papers/papers/full_papers/skoll/skoll_html/index.html>`_:
|
|
||||||
"Many DSL service providers use PPPoE for residential broadband
|
|
||||||
Internet access. This paper briefly describes the PPPoE protocol,
|
|
||||||
presents strategies for implementing it under Linux and describes in
|
|
||||||
detail a user-space implementation of a PPPoE client."
|
|
||||||
|
|
||||||
* `PPP IPV6CP vs DHCPv6 at AAISP <https://www.revk.uk/2011/01/ppp-ipv6cp-vs-dhcpv6.html>`_
|
|
||||||
|
|
||||||
|
|
||||||
* `Creating a Home IPv6 Network (James Bottomley) <https://blog.hansenpartnership.com/creating-a-home-ipv6-network/>`_
|
|
|
@ -5,10 +5,7 @@ Liminix
|
||||||
:maxdepth: 3
|
:maxdepth: 3
|
||||||
:caption: Contents:
|
:caption: Contents:
|
||||||
|
|
||||||
intro
|
new
|
||||||
user
|
|
||||||
developer
|
|
||||||
etc
|
|
||||||
|
|
||||||
|
|
||||||
Indices and tables
|
Indices and tables
|
||||||
|
|
114
doc/new.rst
Normal file
114
doc/new.rst
Normal file
|
@ -0,0 +1,114 @@
|
||||||
|
Getting Started
|
||||||
|
###############
|
||||||
|
|
||||||
|
Liminix is very configurable, which can make it initially daunting
|
||||||
|
especially if you're learning Nix or Linux or networking concepts at
|
||||||
|
the same time. In this section we build some "worked example" Liminix
|
||||||
|
images to introduce the concepts. If you follow the examples exactly,
|
||||||
|
they should work. If you change things as you go along, they may work
|
||||||
|
differently or not at all, but the experience should be educational
|
||||||
|
either way.
|
||||||
|
|
||||||
|
|
||||||
|
.. warning:: The first example we will look at runs under emulation,
|
||||||
|
so there is no danger of bricking your hardware
|
||||||
|
device. For the second example you may (if you have
|
||||||
|
appropriate hardware and choose to do so) flash the
|
||||||
|
configuration onto an actual router. There is always a
|
||||||
|
risk of rendering the device unbootable when you do this,
|
||||||
|
and various ways to recover depending on what went wrong.
|
||||||
|
We'll write more about that at the appropriate point
|
||||||
|
|
||||||
|
|
||||||
|
Requirements
|
||||||
|
************
|
||||||
|
|
||||||
|
You will need a reasonably powerful computer running Nix. Devices
|
||||||
|
that run Liminix are unlikely tohave the CPU power and disk space to
|
||||||
|
be able to build it in situ, so the build process is based around
|
||||||
|
"cross-compilation" from another computer. The build machine can be
|
||||||
|
any reasonably powerful desktop/laptop/server PC running NixOS.
|
||||||
|
Standalone Nixpkgs installations on other Linux distribuions or MacOS
|
||||||
|
also ought to work (but I haven't tested that configuration)
|
||||||
|
|
||||||
|
|
||||||
|
Running in Qemu
|
||||||
|
***************
|
||||||
|
|
||||||
|
Clone the Liminix git repository and change into its directory
|
||||||
|
|
||||||
|
|
||||||
|
.. code-block:: console
|
||||||
|
|
||||||
|
git clone https://gti.telent.net/dan/liminix
|
||||||
|
cd liminix
|
||||||
|
|
||||||
|
Now build Liminix
|
||||||
|
|
||||||
|
.. code-block:: console
|
||||||
|
|
||||||
|
nix-build -I liminix-config=./examples/hellonet.nix \
|
||||||
|
--arg device "import ./devices/qemu" -A outputs.default
|
||||||
|
|
||||||
|
In this command ``liminix-config`` points to the configuration for the
|
||||||
|
device (services, users, filesystem, secrets) and ``device`` is the
|
||||||
|
file for your hardware device definition. ``outputs.default`` tells
|
||||||
|
Liminix to build the appropriate image output appropriate to
|
||||||
|
flash to the hardware device: for the qemu "hardware" it's an alias
|
||||||
|
for ``outputs.vmbuild``, which creates a directory containing a root
|
||||||
|
filesystem image and a kernel.
|
||||||
|
|
||||||
|
.. tip:: The first time you run this it may take several hours,
|
||||||
|
because it builds all of the dependencies including a full
|
||||||
|
MIPS gcc and library toolchain. Once those intermediate build
|
||||||
|
products are in the nix store, subsequent builds will be much
|
||||||
|
faster - practically instant, if nothing has changed.
|
||||||
|
|
||||||
|
Now you can try it:
|
||||||
|
|
||||||
|
.. code-block:: console
|
||||||
|
|
||||||
|
nix-shell --run "mips-vm ./result/vmlinux ./result/rootfs"
|
||||||
|
|
||||||
|
This starts the Qemu emulator to run the Liminix configuration you
|
||||||
|
just built. It connects the Liminix serial console and the `QEMU
|
||||||
|
monitor <https://www.qemu.org/docs/master/system/monitor.html>`_ to
|
||||||
|
stdin/stdout. Use ^P (not ^A) to switch to the monitor.
|
||||||
|
|
||||||
|
You should now see Linux boot messages and after a few seconds be
|
||||||
|
presented with a login prompt. You can login on the console as
|
||||||
|
``root`` (no password) and poke around to see what processes are
|
||||||
|
running. Run ``shutdown`` to shut it down cleanly, or press ^P then
|
||||||
|
type ``exit`` at the monitor to stop it suddenly.
|
||||||
|
|
||||||
|
To see that it running an ssh service we need to connect to its
|
||||||
|
emulated network. Start the machine again, if you had stopped it,
|
||||||
|
and open up a second terminal on your build machine. We're going to
|
||||||
|
run another virtual machine attached to the virtual network, which will
|
||||||
|
request an IP address from our Liminix system and give you a shell
|
||||||
|
you can run ssh from.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
- using modules
|
||||||
|
|
||||||
|
- link to module reference
|
||||||
|
|
||||||
|
- creating custom services
|
||||||
|
|
||||||
|
- longrun or oneshot
|
||||||
|
- dependencies
|
||||||
|
- outputs
|
||||||
|
|
||||||
|
- creating your own modules
|
||||||
|
|
||||||
|
- hacking on Liminix itself
|
||||||
|
|
||||||
|
- contributing
|
||||||
|
|
||||||
|
- external links and resources
|
||||||
|
|
||||||
|
- module reference
|
||||||
|
|
||||||
|
- hardware device reference
|
347
doc/user.rst
347
doc/user.rst
|
@ -1,347 +0,0 @@
|
||||||
User Manual
|
|
||||||
###########
|
|
||||||
|
|
||||||
This manual is an early work in progress, not least because Liminix is
|
|
||||||
not yet really ready for users who are not also developers. Your
|
|
||||||
feedback to improve it is very welcome.
|
|
||||||
|
|
||||||
Installation
|
|
||||||
************
|
|
||||||
|
|
||||||
The Liminix installation process is not quite like installing NixOS on
|
|
||||||
a real computer, but some NixOS experience will nevertheless be
|
|
||||||
helpful in understanding it. The steps are as follows:
|
|
||||||
|
|
||||||
* Decide whether you want the device to be updatable in-place (there
|
|
||||||
are advantages and disadvantages), or if you are happy to generate
|
|
||||||
and flash a new image whenever changes are required.
|
|
||||||
|
|
||||||
* Create a :file:`configuration.nix` describing the system you want
|
|
||||||
|
|
||||||
* Build an image
|
|
||||||
|
|
||||||
* Flash it to the device
|
|
||||||
|
|
||||||
Supported devices
|
|
||||||
=================
|
|
||||||
|
|
||||||
For a list of devices that Liminix (present or previous versions)
|
|
||||||
has run on, refer to `devices/ in the source repo <https://gti.telent.net/dan/liminix/src/branch/main/devices>`_. For devices that _currently_ build,
|
|
||||||
cross-reference it with `the CI status <https://build.liminix.org/jobset/liminix/build#tabs-jobs>`_. Everything that builds is (usually) expected
|
|
||||||
to run, so if you end up with an image that builds but doesn't
|
|
||||||
boot, please report it as a bug.
|
|
||||||
|
|
||||||
As of June 2023 the device list is a little thin. Adding devices based
|
|
||||||
on the Atheros or Mediatek (Ralink) platform should be quite
|
|
||||||
straightforward if you have some C/Linux kernel experience and are
|
|
||||||
prepared to open it up and attach serial wires: please refer to the
|
|
||||||
Developer Manual.
|
|
||||||
|
|
||||||
|
|
||||||
Choosing a flavour (read-only or updatable)
|
|
||||||
===========================================
|
|
||||||
|
|
||||||
Liminix installations come in two "flavours"- read-only or in-place
|
|
||||||
updatable:
|
|
||||||
|
|
||||||
* a read-only installation can't be updated once it is flashed to your
|
|
||||||
device, and so must be reinstalled in its entirety every time you
|
|
||||||
want to change it. It uses the ``squashfs`` filesystem which has
|
|
||||||
very good compression ratios and so you can pack quite a lot of
|
|
||||||
useful stuff onto your device. This is good if you don't expect
|
|
||||||
to change it often.
|
|
||||||
|
|
||||||
* an updatable installation has a writable filesystem so that you can
|
|
||||||
update configuration, upgrade packages and install new packages over
|
|
||||||
the network after installation. This uses the `jffs2
|
|
||||||
<http://www.linux-mtd.infradead.org/doc/jffs2.html>`_ filesystem:
|
|
||||||
although it does compress the data, the need to support writes means
|
|
||||||
that it can't pack quite as small as squashfs, so you will not have
|
|
||||||
as much space to play with.
|
|
||||||
|
|
||||||
Updatability caveats
|
|
||||||
~~~~~~~~~~~~~~~~~~~~
|
|
||||||
|
|
||||||
At the time of writing this manual the read-only squashfs support is
|
|
||||||
much more mature. Consider also that it may not be possible to perform
|
|
||||||
"larger" updates in-place even if you do opt for updatability. If you
|
|
||||||
have (for example) an 11MB system on a 16MB device, you won't be able
|
|
||||||
to do an in-place update of something fundamental like the C library
|
|
||||||
(libc), as this will temporarily require 22MB to install all the
|
|
||||||
packages needing the new library before the packages using the old
|
|
||||||
library can be removed. A writable system will be more useful for
|
|
||||||
smaller updates such as installing a new package (perhaps you
|
|
||||||
temporarily need tcpdump to diagnose a network problem) or for
|
|
||||||
changing configuration files.
|
|
||||||
|
|
||||||
Note also that the kernel is not part of the filesystem so cannot be
|
|
||||||
updated this way. Kernel changes require a full reflash.
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
Creating configuration.nix
|
|
||||||
==========================
|
|
||||||
|
|
||||||
|
|
||||||
You need to create a :file:`configuration.nix` that describes your
|
|
||||||
device and the services that you want to run on it. The best way to
|
|
||||||
get started is by reading one of the examples such as
|
|
||||||
:file:`examples/rotuer.nix` and modifying it to your needs.
|
|
||||||
|
|
||||||
:file:`configuration.nix` conventionally describes the packages, services,
|
|
||||||
user accounts etc of the device. It does not describe the hardware
|
|
||||||
itself, which is specified separately in the build command (as you
|
|
||||||
will see below).
|
|
||||||
|
|
||||||
Most of the functionality of a Liminix system is driven by *services*
|
|
||||||
which are declared by *modules*: thus, to add for example an NTP service
|
|
||||||
you first add :file:`modules/ntp` to your ``imports`` list, then
|
|
||||||
you create a service by calling :code:`config.system.service.ntp.build { .... }`
|
|
||||||
with the appropriate service-dependent configuration parameters.
|
|
||||||
|
|
||||||
.. code-block:: nix
|
|
||||||
|
|
||||||
let svc = config.system.service;
|
|
||||||
in {
|
|
||||||
# ...
|
|
||||||
imports = [
|
|
||||||
./modules/ntp
|
|
||||||
# ....
|
|
||||||
];
|
|
||||||
config.services.ntp = svc.ntp.build {
|
|
||||||
pools = { "pool.ntp.org" = ["iburst"]; };
|
|
||||||
makestep = { threshold = 1.0; limit = 3; };
|
|
||||||
};
|
|
||||||
|
|
||||||
A :ref:`full list of module options <module-options>` is provided
|
|
||||||
later in this manual.
|
|
||||||
|
|
||||||
You *most likely* want to include the ``standard`` module unless you
|
|
||||||
have a quite unusual use case for a very minimal system, in which case
|
|
||||||
you will understand what it does and what happens if you leave it out.
|
|
||||||
|
|
||||||
.. code-block:: nix
|
|
||||||
|
|
||||||
imports = [
|
|
||||||
./modules/standard.nix
|
|
||||||
]
|
|
||||||
configuration.rootfsType = "jffs2"; # or "squashfs"
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
Building
|
|
||||||
========
|
|
||||||
|
|
||||||
Build Liminix using the :file:`default.nix` in the project toplevel
|
|
||||||
directory, passing it arguments for configuration and hardware. For
|
|
||||||
example:
|
|
||||||
|
|
||||||
.. code-block:: console
|
|
||||||
|
|
||||||
nix-build -I liminix-config=./tests/smoke/configuration.nix \
|
|
||||||
--arg device "import ./devices/qemu" -A outputs.default
|
|
||||||
|
|
||||||
In this command ``<liminix-config>`` points to your
|
|
||||||
:file:`configuration.nix`, ``device`` is the file for your hardware device
|
|
||||||
definition, and ``outputs.default`` will generate some kind of
|
|
||||||
Liminix image output appropriate to that device.
|
|
||||||
|
|
||||||
For the qemu device in this example, ``outputs.default`` is an alias
|
|
||||||
for ``outputs.vmbuild``, which creates a directory containing a
|
|
||||||
squashfs root image and a kernel. You can use the :command:`mips-vm` command to
|
|
||||||
run this.
|
|
||||||
|
|
||||||
For the currently supported hardware devices, ``outputs.default``
|
|
||||||
creates a directory containing a file called ``firmware.bin``. This
|
|
||||||
is a raw image file that can be written directly to the firmware flash
|
|
||||||
partition.
|
|
||||||
|
|
||||||
|
|
||||||
Flashing
|
|
||||||
========
|
|
||||||
|
|
||||||
|
|
||||||
Flashing from the boot monitor
|
|
||||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
||||||
|
|
||||||
If you are prepared to open the device and have a TTL serial adaptor
|
|
||||||
of some kind to connect it to, you can probably flash it using U-Boot.
|
|
||||||
This is quite hardware-specific, and sometimes involves soldering:
|
|
||||||
please refer to the Developer Manual.
|
|
||||||
|
|
||||||
|
|
||||||
Flashing from an existing Liminix system with :command:`flashcp`
|
|
||||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
||||||
|
|
||||||
The flash procedure from an existing Liminix-system is two-step.
|
|
||||||
First we reboot the device (using "kexec") into an "ephemeral"
|
|
||||||
RAM-based version of the new configuration, then when we're happy it
|
|
||||||
works we can flash the image - and if it doesn't work we can reboot
|
|
||||||
the device again and it will boot from the old image.
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
Building the RAM-based image
|
|
||||||
............................
|
|
||||||
|
|
||||||
To create the ephemeral image, build ``outputs.kexecboot`` instead of
|
|
||||||
``outputs.default``. This generates a directory containing the root
|
|
||||||
filesystem image and kernel, along with an executable called `kexec`
|
|
||||||
and a `boot.sh` script that runs it with appropriate arguments.
|
|
||||||
|
|
||||||
For example
|
|
||||||
|
|
||||||
.. code-block:: console
|
|
||||||
|
|
||||||
nix-build --show-trace -I liminix-config=./examples/arhcive.nix \
|
|
||||||
--arg device "import ./devices/gl-ar750"
|
|
||||||
-A outputs.kexecboot && \
|
|
||||||
(tar chf - result | ssh root@the-device tar -C /run -xvf -)
|
|
||||||
|
|
||||||
and then login to the device and run
|
|
||||||
|
|
||||||
.. code-block:: console
|
|
||||||
|
|
||||||
cd /run/result
|
|
||||||
sh ./boot.sh .
|
|
||||||
|
|
||||||
|
|
||||||
This will load the new kernel and map the root filesystem into a RAM
|
|
||||||
disk, then start executing the new kernel. *This is effectively a
|
|
||||||
reboot - be sure to close all open files and finish anything else
|
|
||||||
you were doing first.*
|
|
||||||
|
|
||||||
If the new system crashes or is rebooted, then the device will revert
|
|
||||||
to the old configuration it finds in flash.
|
|
||||||
|
|
||||||
|
|
||||||
Building the second (permanent) image
|
|
||||||
.....................................
|
|
||||||
|
|
||||||
While running in the kexecboot system, you can copy the permanent
|
|
||||||
image to the device with :command:`ssh`
|
|
||||||
|
|
||||||
.. code-block:: console
|
|
||||||
|
|
||||||
build-machine$ tar chf - result/firmware.bin | \
|
|
||||||
ssh root@the-device tar -C /run -xvf -
|
|
||||||
|
|
||||||
Next you need to connect to the device and locate the "firmware"
|
|
||||||
partition, which you can do with a combination of :command:`dmesg`
|
|
||||||
output and the contents of :file:`/proc/mtd`
|
|
||||||
|
|
||||||
.. code-block:: console
|
|
||||||
|
|
||||||
<5>[ 0.469841] Creating 4 MTD partitions on "spi0.0":
|
|
||||||
<5>[ 0.474837] 0x000000000000-0x000000040000 : "u-boot"
|
|
||||||
<5>[ 0.480796] 0x000000040000-0x000000050000 : "u-boot-env"
|
|
||||||
<5>[ 0.487056] 0x000000050000-0x000000060000 : "art"
|
|
||||||
<5>[ 0.492753] 0x000000060000-0x000001000000 : "firmware"
|
|
||||||
|
|
||||||
# cat /proc/mtd
|
|
||||||
dev: size erasesize name
|
|
||||||
mtd0: 00040000 00001000 "u-boot"
|
|
||||||
mtd1: 00010000 00001000 "u-boot-env"
|
|
||||||
mtd2: 00010000 00001000 "art"
|
|
||||||
mtd3: 00fa0000 00001000 "firmware"
|
|
||||||
mtd4: 002a0000 00001000 "kernel"
|
|
||||||
mtd5: 00d00000 00001000 "rootfs"
|
|
||||||
|
|
||||||
Now run (in this example)
|
|
||||||
|
|
||||||
.. code-block:: console
|
|
||||||
|
|
||||||
flashcp -v firmware.bin /dev/mtd3
|
|
||||||
|
|
||||||
|
|
||||||
"I know my new image is good, can I skip the intemediate step?"
|
|
||||||
```````````````````````````````````````````````````````````````
|
|
||||||
|
|
||||||
In addition to giving you a chance to see if the new image works, this
|
|
||||||
two-step process ensures that you're not copying the new image over
|
|
||||||
the top of the active root filesystem. It might work, or it might
|
|
||||||
crash in surprising ways.
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
Flashing from OpenWrt (not currently advised!)
|
|
||||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
||||||
|
|
||||||
.. CAUTION:: At your own risk! This will (at least in some
|
|
||||||
circumstances) lead to bricking the device: we think this
|
|
||||||
flash method is currently incompatible with use of a
|
|
||||||
writeable (jffs2) filesystem.
|
|
||||||
|
|
||||||
If your device is running OpenWrt then it probably has the
|
|
||||||
:command:`mtd` command installed. After transferring the image onto the
|
|
||||||
device using e.g. :command:`ssh`, you can run it as follows:
|
|
||||||
|
|
||||||
.. code-block:: console
|
|
||||||
|
|
||||||
mtd -r write /tmp/firmware.bin firmware
|
|
||||||
|
|
||||||
For more information, please see the `OpenWrt manual <https://openwrt.org/docs/guide-user/installation/sysupgrade.cli>`_ which may also contain (hardware-dependent) instructions on how to flash an image using the vendor firmware - perhaps even from a web interface.
|
|
||||||
|
|
||||||
|
|
||||||
Updating an installed system (JFFS2)
|
|
||||||
************************************
|
|
||||||
|
|
||||||
Adding packages
|
|
||||||
===============
|
|
||||||
|
|
||||||
|
|
||||||
If your device is running a JFFS2 root filesystem, you can build
|
|
||||||
extra packages for it on your build system and copy them to the
|
|
||||||
device: any package in Nixpkgs or in the Liminix overlay is available
|
|
||||||
with the ``pkgs`` prefix:
|
|
||||||
|
|
||||||
.. code-block:: console
|
|
||||||
|
|
||||||
nix-build -I liminix-config=./my-configuration.nix \
|
|
||||||
--arg device "import ./devices/mydevice" -A pkgs.tcpdump
|
|
||||||
|
|
||||||
nix-shell -p min-copy-closure root@the-device result/
|
|
||||||
|
|
||||||
Note that this only copies the package to the device: it doesn't update
|
|
||||||
any profile to add it to ``$PATH``
|
|
||||||
|
|
||||||
|
|
||||||
Rebuilding the system
|
|
||||||
=====================
|
|
||||||
|
|
||||||
:command:`liminix-rebuild` is the Liminix analogue of :command:`nixos-rebuild`, although its operation is a bit different because it expects to run on a build machine and then copy to the host device. Run it with the same ``liminix-config`` and ``device`` parameters as you would run :command:`nix-build`, and it will build any new/changed packages and then copy them to the device using SSH. For example:
|
|
||||||
|
|
||||||
.. code-block:: console
|
|
||||||
|
|
||||||
liminix-rebuild root@the-device -I liminix-config=./examples/rotuer.nix --arg device "import ./devices/gl-ar750"
|
|
||||||
|
|
||||||
This will
|
|
||||||
|
|
||||||
* build anything that needs building
|
|
||||||
* copy new or changed packages to the device
|
|
||||||
* reboot the device
|
|
||||||
|
|
||||||
It doesn't delete old packages automatically: to do that run
|
|
||||||
:command:`min-collect-garbage`, which will delete any packages not in
|
|
||||||
the current system closure. Note that Liminix does not have the NixOS
|
|
||||||
concept of environments or generations, and there is no way back from
|
|
||||||
this except for building the previous configuration again.
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
Caveats
|
|
||||||
~~~~~~~
|
|
||||||
|
|
||||||
* it needs there to be enough free space on the device for all the new
|
|
||||||
packages in addition to all the packages already on it - which may be
|
|
||||||
a problem if a lot of things have changed (e.g. a new version of
|
|
||||||
nixpkgs).
|
|
||||||
|
|
||||||
* it cannot upgrade the kernel, only userland
|
|
||||||
|
|
||||||
Configuration options
|
|
||||||
*********************
|
|
||||||
|
|
||||||
.. _module-options:
|
|
||||||
|
|
||||||
.. include:: modules.rst
|
|
50
examples/hellonet.nix
Normal file
50
examples/hellonet.nix
Normal file
|
@ -0,0 +1,50 @@
|
||||||
|
{ config, pkgs, lib, ... } :
|
||||||
|
let
|
||||||
|
inherit (pkgs) serviceFns;
|
||||||
|
svc = config.system.service;
|
||||||
|
|
||||||
|
in rec {
|
||||||
|
imports = [
|
||||||
|
../modules/network
|
||||||
|
../modules/dnsmasq
|
||||||
|
../modules/ntp
|
||||||
|
../modules/ssh
|
||||||
|
];
|
||||||
|
hostname = "hellonet";
|
||||||
|
|
||||||
|
services.int = svc.network.address.build {
|
||||||
|
interface = config.hardware.networkInterfaces.lan;
|
||||||
|
family = "inet"; address ="10.3.0.1"; prefixLength = 16;
|
||||||
|
};
|
||||||
|
|
||||||
|
services.ntp = svc.ntp.build {
|
||||||
|
pools = { "pool.ntp.org" = ["iburst"]; };
|
||||||
|
makestep = { threshold = 1.0; limit = 3; };
|
||||||
|
};
|
||||||
|
|
||||||
|
services.sshd = svc.ssh.build { };
|
||||||
|
|
||||||
|
users.root = {
|
||||||
|
passwd = "";
|
||||||
|
};
|
||||||
|
|
||||||
|
services.dns =
|
||||||
|
let interface = services.int;
|
||||||
|
in svc.dnsmasq.build {
|
||||||
|
inherit interface;
|
||||||
|
ranges = [
|
||||||
|
"10.3.0.10,10.3.0.240"
|
||||||
|
# ra-stateless: sends router advertisements with the O and A
|
||||||
|
# bits set, and provides a stateless DHCP service. The client
|
||||||
|
# will use a SLAAC address, and use DHCP for other
|
||||||
|
# configuration information.
|
||||||
|
"::,constructor:$(output ${interface} ifname),ra-stateless"
|
||||||
|
];
|
||||||
|
|
||||||
|
domain = "example.org";
|
||||||
|
};
|
||||||
|
|
||||||
|
defaultProfile.packages = with pkgs; [
|
||||||
|
figlet
|
||||||
|
];
|
||||||
|
}
|
Loading…
Reference in a new issue