From 648ac2eb7fd3baaf7a5ecdb1ff34343276294ebe Mon Sep 17 00:00:00 2001 From: Daniel Barlow Date: Wed, 17 May 2023 15:04:46 +0100 Subject: [PATCH] Document jffs2, min-copy-closure, liminix-rebuild Some of the code is now out of date w.r.t. some of the text --- doc/developer.rst | 1 + doc/user.rst | 285 ++++++++++++++++++++++++++++++++-------------- 2 files changed, 200 insertions(+), 86 deletions(-) diff --git a/doc/developer.rst b/doc/developer.rst index 84c4d21..7cfb902 100644 --- a/doc/developer.rst +++ b/doc/developer.rst @@ -149,6 +149,7 @@ module(s) it uses. I have this segment in configuration.nix which you may be able to adapt: .. code-block:: nix + boot = { kernelParams = [ "intel_iommu=on" ]; kernelModules = [ diff --git a/doc/user.rst b/doc/user.rst index 2bf0ea0..60a2576 100644 --- a/doc/user.rst +++ b/doc/user.rst @@ -2,32 +2,101 @@ User Manual ########### This manual is an early work in progress, not least because Liminix is -not yet ready for users who are not also developers. +not yet really ready for users who are not also developers. Your +feedback to improve it is very welcome. -Configuring for your use case -***************************** +Installation +************ -You need to create a ``configuration.nix`` that describes your router +The Liminix installation process is not quite like installing NixOS on +a real computer, but some NixOS experience will nevertheless be +helpful in understanding it. The steps are as follows: + +* Decide whether you want the device to be updatable in-place (there + are advantages and disadvantages), or if you are happy to generate + and flash a new image whenever changes are required. + +* Create a :file:`configuration.nix` describing the system you want + +* Build an image + +* Flash it to the device + + +Choosing a flavour (read-only or updatable) +=========================================== + +Liminix installations come in two "flavours"- read-only or in-place +updatable: + +* a read-only installation can't be updated once it is flashed to your + device, and so must be reinstalled in its entirety every time you + want to change it. It uses the ``squashfs`` filesystem which has + very good compression ratios and so you can pack quite a lot of + useful stuff onto your device. This is good if you don't expect + to change it often. + +* an updatable installation has a writable filesystem so that you can + update configuration, upgrade packages and install new packages over + the network after installation. This uses the `jffs2 + `_ filesystem: + although it does compress the data, the need to support writes means + that it can't pack quite as small as squashfs, so you will not have + as much space to play with. + +Updatability caveats +~~~~~~~~~~~~~~~~~~~~ + +At the time of writing this manual the read-only squashfs support is +much more mature. Consider also that it may not be possible to perform +"larger" updates in-place even if you do opt for updatability. If you +have (for example) an 11MB system on a 16MB device, you won't be able +to do an in-place update of something fundamental like the C library +(libc), as this will temporarily require 22MB to install all the +packages needing the new library before the packages using the old +library can be removed. A writable system will be more useful for +smaller updates such as installing a new package (perhaps you +temporarily need tcpdump to diagnose a network problem) or for +changing configuration files. + +Note also that the kernel is not part of the filesystem so cannot be +updated this way. Kernel changes require a full reflash. + + + +Creating configuration.nix +========================== + + +You need to create a ``configuration.nix`` that describes your device and the services that you want to run on it. Start by copying ``vanilla-configuration.nix`` and adjusting it, or look in the `examples` directory for some pre-written configurations. -Your configuration may include modules and probably _should_ +``configuration.nix`` conventionally describes the packages, services, +user accounts etc of the device. It does not describe the hardware +itself, which is specified separately in the build command (as you +will see below). + +Your configuration may include modules: it probably *should* include the ``standard`` module unless you understand what it does and what happens if you leave it out. -.. code-block: nix +.. code-block:: nix - imports = [ - ./modules/standard.nix - ] + imports = [ + ./modules/standard.nix + ] + configuration.rootfsType = "jffs2"; # or "squashfs" -Building and flashing -********************* +Building +======== -An example command to build Liminix might look like this: +Build Liminix using the :file:`default.nix` in the project toplevel +directory, passing it arguments for configuration and hardware. For +example: .. code-block:: console @@ -50,52 +119,8 @@ is a raw image file that can be written directly to the firmware flash partition. -Flashing with :command:`flashcp` -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -This requires an existing Liminix system, or perhaps some other -operating system on the device which provides the :command:`flashcp` -command. You need to locate the "firmware" partition, which you can do -with a combination of :command:`dmesg` output and the contents of -:file:`/proc/mtd` - -**Don't do this on a device that's running on the same flash partition -as you're about to overwrite, otherwise you're likely to crash it. Use -kexecboot (see "Updates to running devices" below) first to reboot -into a RAM-based system.** - - -.. code-block:: console - - <5>[ 0.469841] Creating 4 MTD partitions on "spi0.0": - <5>[ 0.474837] 0x000000000000-0x000000040000 : "u-boot" - <5>[ 0.480796] 0x000000040000-0x000000050000 : "u-boot-env" - <5>[ 0.487056] 0x000000050000-0x000000060000 : "art" - <5>[ 0.492753] 0x000000060000-0x000001000000 : "firmware" - - # cat /proc/mtd - dev: size erasesize name - mtd0: 00040000 00001000 "u-boot" - mtd1: 00010000 00001000 "u-boot-env" - mtd2: 00010000 00001000 "art" - mtd3: 00fa0000 00001000 "firmware" - mtd4: 002a0000 00001000 "kernel" - mtd5: 00d00000 00001000 "rootfs" - -Then you can copy the image to the device with :command:`ssh` - -.. code-block:: console - - build-machine$ tar chf - result/firmware.bin | \ - ssh root@the-device tar -C /run -xvf - - -and then connect to the device and run - - -.. code-block:: console - - flashcp -v firmware.bin /dev/mtd3 - +Flashing +======== Flashing from OpenWrt (untested) @@ -108,7 +133,7 @@ If your device is running OpenWrt then it probably has the mtd -r write /tmp/firmware_image.bin firmware -For more information, please see the `OpenWrt manual `_ +For more information, please see the `OpenWrt manual `_ which may also contain (hardware-dependent) instructions on how to flash an image using the vendor firmware - perhaps even from a web interface. Flashing from the boot monitor @@ -116,20 +141,26 @@ Flashing from the boot monitor If you are prepared to open the device and have a TTL serial adaptor of some kind to connect it to, you can probably flash it using U-Boot. -This is quite hardware-specific: please refer to the Developer Manual. +This is quite hardware-specific, and sometimes involves soldering: +please refer to the Developer Manual. -Updates to running devices -************************** +Flashing from an existing Liminix system with :command:`flashcp` +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -To mitigate the risk of flashing a new configuration and potentially -render the device unresponsive if the configuration is unbootable or -doesn't bring up a network device, Liminix has a -"try before write" mode. +The flash procedure from an existing Liminix-system is two-step. +First we reboot the device (using "kexec") into an "ephemeral" +RAM-based version of the new configuration, then when we're happy it +works we can flash the image - and if it doesn't work we can reboot +the device again and it will boot from the old image. -To test a configuration without writing it to flash, import the -``kexecboot`` module and build ``outputs.kexecboot`` instead of + +Building the RAM-based image +............................ + + +To creatr the ephemeral image, build ``outputs.kexecboot`` instead of ``outputs.default``. This generates a directory containing the root filesystem image and kernel, along with an executable called `kexec` and a `boot.sh` script that runs it with appropriate arguments. @@ -157,29 +188,111 @@ reboot - be sure to close all open files and finish anything else you were doing first.* If the new system crashes or is rebooted, then the device will revert -to the old configuration it finds in flash. Thus, by combining kexec -boot with a hardware watchdog you can try new images with very little -chance of bricking anything. When you are happy that the new -configuration is correct, build and flash a flashable image of it. +to the old configuration it finds in flash. + + +Building the second (permanent) image +..................................... + + +While running in the kexecboot system, you can copy the permanent +image to the device with :command:`ssh` + +.. code-block:: console + + build-machine$ tar chf - result/firmware.bin | \ + ssh root@the-device tar -C /run -xvf - + +Next you need to connect to the device and locate the "firmware" +partition, which you can do with a combination of :command:`dmesg` +output and the contents of :file:`/proc/mtd` + + +.. code-block:: console + + <5>[ 0.469841] Creating 4 MTD partitions on "spi0.0": + <5>[ 0.474837] 0x000000000000-0x000000040000 : "u-boot" + <5>[ 0.480796] 0x000000040000-0x000000050000 : "u-boot-env" + <5>[ 0.487056] 0x000000050000-0x000000060000 : "art" + <5>[ 0.492753] 0x000000060000-0x000001000000 : "firmware" + + # cat /proc/mtd + dev: size erasesize name + mtd0: 00040000 00001000 "u-boot" + mtd1: 00010000 00001000 "u-boot-env" + mtd2: 00010000 00001000 "art" + mtd3: 00fa0000 00001000 "firmware" + mtd4: 002a0000 00001000 "kernel" + mtd5: 00d00000 00001000 "rootfs" + +Now run (in this example) + +.. code-block:: console + + flashcp -v firmware.bin /dev/mtd3 + + +"I know my new image is good, can I skip the intemediate step?" +``````````````````````````````````````````````````````````````` + +In addition to giving you a chance to see if the new image works, this +two-step process ensures that you're not copying the new image over +the top of the active root filesystem. It might work, or it might +crash in surprising ways. -Module options (tbd) -************** +Updating an installed system (JFFS2) +************************************ + + +Adding packages +=============== + + +If your device is running a JFFS2 root filesystem, you can build +extra packages for it on your build system and copy them to the +device: any package in Nixpkgs or in the Liminix overlay is available +with the ``pkgs`` prefix: + +.. code-block:: console + + nix-build -I liminix-config=./my-configuration.nix \ + --arg device "import ./devices/mydevice" -A pkgs.tcpdump + + nix-shell -p min-copy-closure root@the-device result/ + +Note that this only copies the package to the device: it doesn't update +any profile to add it to ``$PATH`` + + +Rebuilding the system +===================== + +:command:`liminix-rebuild` is the Liminix analogue of :command:`nixos-rebuild`, although its operation is a bit different because it expects to run on a build machine and then copy to the host device. Run it with the same ``liminix-config`` and ``device`` parameters as you would run :command:`nix-build`, and it will build any new/changed packages and then copy them to the device using SSH. For example: + +.. code-block:: console + + liminix-rebuild root@the-device -I liminix-config=./examples/rotuer.nix --arg device "import ./devices/gl-ar750" + + +Caveats +~~~~~~~ + +* it needs there to be enough free space on the device for all the new + packages in addition to all the packages already on it - which may be + a problem if a lot of things have changed (e.g. a new version of + nixpkgs). + +* it cannot upgrade the kernel, only userland + +* it reboots the device! -Foo module -========== +Configuration Options +********************* + + Module docs will go here. This part of the doc should be autogenerated. - - -Bar module -========== - -Baz module -========== - -Quuz net device -===============