We skip 1.20.3 which had a somewhat nasty bug that was immediately fixed
in 1.20.4 with no other changes.
Change-Id: Ib7ef12912b3ed7a7ed2e00fea654d5d7532a92f7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10863
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Autosubmit: sterni <sternenseemann@systemli.org>
1.20.1 doesn't seem to have any substantial changes except bug fixes, so
let's jump to it right away.
Change-Id: If41f06352f5e5351e37af1969fa8152693a227e8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10861
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Autosubmit: sterni <sternenseemann@systemli.org>
nixos-unstable got the bump to libcamera 0.2, no need to also do this
here.
We only need to use the patched libcamera for pipewire and wireplumber
running on the system.
Change-Id: I4ba5d30f68a5ded56cba07a5ad4aaa84d2711b8f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10825
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
"media: ov2740: Fix hts value" got backported into linux-stable:
```
commit c5883a8a3676b94fe7cefde97c8eec50bb879eb2
Author: Hans de Goede <hdegoede@redhat.com>
Date: Mon Dec 4 13:39:42 2023 +0100
media: ov2740: Fix hts value
[ Upstream commit 3735228bbe3511f844e03dfcc4003fadb59dde23 ]
HTS must be more then width, so the 1080 value clearly is wrong,
this is then corrected with some weird math dividing clocks in
to_pixels_per_line() which results in the hts getting multiplied by 2,
resulting in 2160.
Instead just directly set hts to the correct value of 2160 and
drop to_pixels_per_line().
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Sasha Levin <sashal@kernel.org>
```
So drop that patch from the list of kernel patches to apply (and vendor
in the patches).
Change-Id: I314ccb524d156c0f445ed923ce0600f27b8cc699
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10813
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: flokli <flokli@flokli.de>
No default.nix yet, just for development.
Change-Id: Ib8bd0057d697fecd083d5961e635c770b7638e08
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10803
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
Some of the comments didn't yet account for this layout now being
colemak, and the function keys swapped to match stock Miryoku more
closely.
Change-Id: I38ae92153f80855ac21cc62cd603b7020b5be4ac
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10787
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: flokli <flokli@flokli.de>
The machine at Hetzner is gone, only the one in EC2 remains.
Change-Id: Ia7266d56ef1174267b95086c51e6d80015c2f905
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10711
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: flokli <flokli@flokli.de>
This code adds support for the ipu6 webcams via libcamera, based on the work in
https://copr.fedorainfracloud.org/coprs/jwrdegoede/ipu6-softisp/.
It's supposed to be included in your NixOS configuration imports.
Change-Id: Ifb71999ad61161fa23506b97cb449f73fb1270e3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10709
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: flokli <flokli@flokli.de>
Looks like the develop branch at least has everything that's needed.
Change-Id: Ic25d571f4e6fe2a45f8f11f917622fe50a39ce07
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10690
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
The original layout is pretty far from Miryoku. It's not using Colemak
DH, but QWERTY, and the rest also looks quite different.
Align with the official Miryoku reference, from
https://github.com/manna-harbour/miryoku/tree/master/docs/reference.
Notable differences:
- Button layer left out (normally activated by holding a bottom row
pinkie key)
- Media layer is missing external power and bluetooth-related keys
(understandably).
Change-Id: Icd8b2222c9a68650d9d4ee81aa96df0f4bc3bc99
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10673
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
So far, this is still a copy from the "via" target, but makes
customization possible.
Change-Id: I9e23808b72b874901a5f8c074763346181b05c1f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10670
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
This builds the stock "via" layout, which, in addition to via support,
contains some other usefulness from miryoku.
Change-Id: I967a9bfcc87e108d4c0a2cb37b512d0b39912f7a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10666
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
This cleans up the clang-tools_11 attribute, affects tazjin's frog and
aspen's home-manager configuration. achilles remains with pinned
llvmPackages, but it doesn't build at the moment, so it's impossible to
test.
Change-Id: I53416bf7c99f363163de29b32678a82b9eb5adbd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10644
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: aspen <root@gws.fyi>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
This has been broken for a while, and I don't want to keep linking to it
until I get a chance to fix it
Change-Id: Id7e50c0b626c9fb0ca6bb8a7c91bbae1b457c473
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10660
Autosubmit: aspen <root@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: aspen <root@gws.fyi>
Following a discussion on Telegram about window management ... might
come in useful!
Change-Id: If50741e4281c658bccc55e2f591ef945ef4b3b5d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10592
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Creates a simple dot graph that shows the internal dependencies of an
elisp module. Useful for certain kinds of refactoring ...
Change-Id: Id7a7756b866751d0e7288e0d22b72ec8056a9eef
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10591
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
* Make sterni.lv declarative
* Disable gopher server
* Disable likely-music.sterni.lv for now
* Don't give systemd too much leeway with scheduling git syncs
Change-Id: Ie8507d96f2df76ad8e393b2181ed7378c37829d0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10480
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
This thing has been introduced recently and prints annoying warnings
when running manually patchelfed binaries with the default loader
path (which used to work fine!).
This doesn't actually fix running the binaries, which now segfault,
but at least it doesn't print a paragraph of annoying text in the
terminal instead.
Change-Id: Ie0d8c3cc124b71d244e10f3755431fb3401ff81b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10479
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
This has some effect on the Emacs 29 hollow cursor bug, though I
honestly don't fully understand what this does.
Change-Id: I03eaa0a64e81f01a86736ad17b4c1b3a56fe3d5a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10461
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
* all: update wasm-bindgen to 0.2.89 in WASM projects
* users/grfn: explicitly set pinentry for gpg-agent
* 3p/crate2nix: drop patches that were merged upstream
* 3p/rust-crates: fix one more package name that was broken by crates.io
* 3p/overlays: bump telega backend to new required version
The update for agenix has been dropped. It caused strange build errors
with messages like these:
patching script interpreter paths in /nix/store/0g0wpa3vxfb4w461s6ny3s1wr08faj73-agenix-0.15.0
/nix/store/0g0wpa3vxfb4w461s6ny3s1wr08faj73-agenix-0.15.0/bin/agenix: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/q8qq40xg2grfh9ry1d9x4g7lq4ra7n81-bash-5.2-p21/bin/bash"
stripping (with command strip and flags -S -p) in /nix/store/0g0wpa3vxfb4w461s6ny3s1wr08faj73-agenix-0.15.0/bin
Running phase: installCheckPhase
no Makefile or custom installCheckPhase, doing nothing
agenix version: 0.15.0
error: creating directory '/nix/var': Permission denied
There is no rule for secret1.age in ./secrets.nix.
/nix/store/d4jf1cbbk494zwgbqz31pxgigpsbh6w2-stdenv-linux/setup: line 138: test: =: unary operator expected
/nix/store/d4jf1cbbk494zwgbqz31pxgigpsbh6w2-stdenv-linux/setup: line 131: pop_var_context: head of shell_variables not a function context
builder for '/nix/store/0ivvf44hxy0zv4gg8nvchdkp895xw5ri-agenix-0.15.0.drv' failed with exit code 2
I can't be bothered to deal with that right now.
Change-Id: Ia052af0d97dbe9ef0c0d4f3e2214ac00ca8645a2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10458
Reviewed-by: aspen <root@gws.fyi>
Tested-by: BuildkiteCI
This is just intended as a local backup in case things go wrong
horribly, so you can revert to a recent state.
Change-Id: I1d666bad77045a1c807204df144422ba69d1d99f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10417
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
The correct one is cache.tvl.su:kjc6KOMupXc1vHVufJUoDUYeLzbwSr9abcAKdn/U1Jk=,
defined in ops/modules/tvl-cache.nix for example, but as
ssh://nix-ssh@whitby.tvl.fyi is configured, these signatures don't apply
anyways.
Change-Id: I7008a005fe34568d7504b66d979de68bfcfc7acf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10192
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
The correct one is cache.tvl.su:kjc6KOMupXc1vHVufJUoDUYeLzbwSr9abcAKdn/U1Jk=,
defined in ops/modules/tvl-cache.nix for example, but as
ssh://nix-ssh@whitby.tvl.fyi is configured, these signatures don't apply
anyways.
Change-Id: Ib6d429b198f2d4853d6a7d302d91c51dad3c9cab
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10191
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Autosubmit: flokli <flokli@flokli.de>
the keys.gnupg.net keyserver has been down for some time now, let's not
link to that in favor of linking to a self-hosted version of my GPG
public key
Change-Id: I9f645c562c2b04e9fb755f21d1ec8a89f89c2230
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10032
Tested-by: BuildkiteCI
Autosubmit: grfn <grfn@gws.fyi>
Reviewed-by: grfn <grfn@gws.fyi>
This passes all tests I could find so far, but doesn't work on the
website. I wrote an interactive debugger for looking at the input, and
haven't found anything incorrect, so not sure what it wants from me.
Change-Id: I506001735e15b2d02eaaebc6d1da8c26e92acde0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10188
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Deletes some stuff that I think isn't necessary anymore, and
consolidates the modes.el content into settings.el.
Change-Id: Ib682dbdb4eb89b3a7ee2eca89da4151af806a508
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10187
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
The second task was very annoying because you had to guess the actual
rules (overlapping words), as they're not explained correctly in the
task. My solution hardcodes those cases.
Change-Id: Idf24579a78a1b8ede368504d3ff0c58c9978f069
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10183
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: tazjin <tazjin@tvl.su>
Lets see if it's good now. Chrom(e|ium) (and by extension (pun
intended) derivatives) are getting rid of ad blockers, so need to
do *something*.
Change-Id: Id7c69d38ddb0ad929aed9e6fbae4d9919ed0731f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10177
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
I have a suspicion that some strange behaviour I occasionally get is
related to company mode.
Change-Id: I26f25c31967ae092d15248a806acdf4f28cb4c10
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10176
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
I don't really like this package, it's kinda wonky, and now that part
of my mode line logic is in the tab-bar, it's no longer needed.
Change-Id: I4791a75e5ce2f0c49ef0d239cadf6a4f81c73636
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10171
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
I lost a yubikey :(
this is the way to recover my cloudflare account
Change-Id: Iaffb567ec4a072baa93e78073b47e99a1dc2bb36
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10181
Reviewed-by: grfn <grfn@gws.fyi>
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Main objective was to get SMART/md monitoring working, alerts go (via
some awful glue code) to #sterni.lv on hackint. Bot nick should also be
registered in the future.
Change-Id: Ia73c5a64ee9f6df62f5fbe21fc1606477e3d6e73
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10174
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
We currently don't need it actively and it is easy to re-enable if
needed. Due to spawn chunks simulation it is not really idling either.
Change-Id: I2e4e5ff2271fd61ee1affec27a614244d4a87fcf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10173
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
This actually allows disabling a service for the first time, since the
configuration structure for the service is created irrespective of the
enable option.
Change-Id: I08e5e67565d6fe210fb0f65600b8750433ce4712
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10175
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
This will hopefully prevent failures on system deploys where DNS is
briefly not available, so git(1) fails to resolve github.com. Thanks
flokli for the tip.
Change-Id: I6096e9f3655cbe28ca2a71142de22337814e0be1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10172
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
This removes the wonky hack with detecting the bottom right window,
and gives me saner, unified handling of this display in the tab
bar (of which there is only one!)
Change-Id: Id21c6b2472d0c89fc4d000a10a9e90d2ddba86b6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10165
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
While we are at it, rename disk-checkup.nix to btrfs-auto-scrub.nix and
move it into //ops/modules. I originally wanted to have additionally
disk health related services in that module, but the btrfs scrub
functionality is nicely self-contained and reusable, so I think it makes
sense to have this in a more central location.
Change-Id: Iabdd62838eef009540ca71abafd921afda2a9b47
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10128
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Network configuration and initrd setup is basically the same as with
edwin, but we are using md for Software RAID this time as well as LVM
over two partitions with LUKS:
- sda2 <-- RAID1 --> sdb2 (boot-raid)
└ boot partition, ext4 (encrypted-container-raid)
- sda3 <-- RAID1 --> sdb3
└ LUKS container
└ Volume Group vgmain
├ Logical Volume vgmain/swap
│ └ swap
└ Logical Volume vgmain/root
└ btrfs
So we no longer rely on btrfs raid1 due to question marks over its
reliability (I personally did not have any problems though). This also
means that we have less LUKS containers we need to unlock when
booting (kind of neglible improvement). The biggest improvement is that
we have redundancy for the swap, so a disk failure shouldn't cause
memory corruption/loss.
Change-Id: I14f065b659857415917d9a60a7ec019e687f8d1c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10127
Tested-by: BuildkiteCI
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
These settings would also be applied to a machine that'd replace edwin,
so it's useful to have them outside edwin's default.nix.
Change-Id: I4e8f464118a103645e53909a87c6ee4446022fa3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10125
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
This introduces a new structure, as activities are moved to a separate
AWS Account for reasons (tm).
Change-Id: Ic310eca2dc0d4ee81bae8944238b26910067336a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10124
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
This removes everything related to workspaces, as well as the current
RandR configuration, creating a (cleaner) slate for switching to tabs.
This was supposed to be committed earlier, but got broken by
clbot (see b/333).
Change-Id: I2d110bca0d6629a505699210f0aba12882f83d48
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10115
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Sets up tab-bar-mode for EXWM, and adds simple initial configuration
and shortcuts to make it behave sanely. I had to assign tab switching
shortcuts myself, as the EXWM input key stuff doesn't work correctly
with the built-in way of setting these shortcuts in tab-bar-mode.
Change-Id: Icd96f03ae7bd2bd5f7c2d59f9aca7a1ca2e7f788
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10116
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Adds an `exwm-assign-workspaces` function that automatically creates
and assigns workspaces to each currently connected monitor. The first
workspace (index 0) is always on the primary monitor.
This function should be idempotent and can be called at any point to
synchronise X outputs and what EXWM is displaying on them.
This works because tabs are disconnected from workspaces completely,
so I don't have to care about what's going on on other workspaces
anymore.
Still missing:
* functions to connect/disconnect outputs
* switching to other outputs from within emacs commands (i.e. without
the mouse)
Change-Id: I7c24aa1b45218fe35de6939e799852b5d11d1272
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10119
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
This one is a bit stupid because I couldn't figure out a way to
determine the active workspace. It's definitely possible (either
through some XCB calls, or through state management in screen change
hooks), but for now this is fine.
Change-Id: I5e4c531b248caa0021664bad9dc196bef60cfbac
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10122
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
By default, this will be the (only) remaining monitor. In N>2
situations, ask the user.
Change-Id: Id68fcf60d56d0414d1072b8ffeef72c608678f52
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10121
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Uses a bunch of weird xrandr invocations and completing reads to
configure screens the way I want.
Note that this has a known bug where disconnecting a primary screen
will *not* make one of the remaining screens primary.
Change-Id: Ide5322df446685cc4740d4ddd7b6ca8682375050
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10120
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Something changed in the machinery and broke the overriding. I didn't
notice this, as a I was temporarily using an unpatched EXWM.
Change-Id: I1a4e8ea63bd116d86a430e680c2b631474e9a0fe
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10047
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Make these keys behave a bit more like the Fn+F* keys on the thinkpad
keyboards.
Seems there's currently no trivial way to get mic mute, keyboard mute
and wifi toggle sent out, but considering the thinkpad usb keyboard is
able to, this should be possible somehow here too - but not today, left
for a followup.
Change-Id: I529a958c78116dd9f7250c938e2e7989b296d6c6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10076
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
It seems the regex is not perfect, it choked on a single log line:
```
Nov 13 03:10:19 archeology-ec2 59nkrwmih3ywaxrgxqj79pn395fs6m17-parse-bucket-logs-continuously[11105]: Code: 117. DB::Exception: Line "d57bd890fbd1ae16625bdb8168064125e013198099b7e1b3c24878a4d03c3ab8 nix-cache [12/Nov/2023:09:13:02 +0000] xxx.xx.xxx.xxx - VB7SJVZ108DSSN67 REST.POST.OBJECT index.html "POST /index.html HTTP/1.1" 405 MethodNotAllowed 348 - 4 - "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36" - 0bFdGKbi0n9JHXU1a2hijcJwmYdc6lG2xgbdozc3wS6mlUkBE7ssrQCHIDdOLebo78o2cGbhivY= - ECDHE-RSA-AES128-GCM-SHA256 - nix-cache.s3.amazonaws.com TLSv1.2 - -" doesn't match the regexp.: (in file/uri log/2023-11-12-10-19-50-80805A702ECF65EB): (at row 5)
```
This was due to the user-agent field. The regex is now fixed.
The request itself is fun (someone trying to POST an index.html to the
bucket), and we should probably filter this on the Fastly side already,
not via IAM,
In any case, there's no point failing to parse if a single line doesn't
match the regex - we can just skip them.
For the sake of completeness, logs for that day have been reprocessed
and reuploaded.
Change-Id: Id98a7167a381cda06d150ad5118ee9e70ead277e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10034
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
I no longer work at ReadySet.
Change-Id: Idc19e2d68846551b6cd94f84594712692ebe35a9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9976
Tested-by: BuildkiteCI
Autosubmit: grfn <grfn@gws.fyi>
Reviewed-by: grfn <grfn@gws.fyi>
This packages up my keyboard firmware used for the Keychron K6 Pro.
We add a custom keymap to the `keyboards/keychron/k6_pro/ansi/rgb/
keymaps` directory, a copy from the `default` one (with a modified
`keymap.c`), and then build that as a makefile target.
`via` is *disabled*, as their keybindings take priority over keymap.c.
Luckily, only `qmk` seems to be sufficient to build it.
A simple `:flash` target/script is provided as well, it relies on some
udev rules set in the global system
(`hardware.keyboard.qmk.enable = true`).
Change-Id: I9f7a7a992e13516c32033127f94e37aec62d6b67
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10020
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
This allows using awscli inside a shell.
Clickhouse AWS SSO integration still seems broken unfortunately, even
with https://github.com/ClickHouse/ClickHouse/pull/54347 included in
our bump - it seems it's coming up with another token file path than the
AWS SDK:
> SSOCredentialsProvider: Unable to open token file on path: /home/flokli/.aws/sso/cache/da39a3ee5e6b4b0d3255bfef95601890afd80709.json
This is the sha1sum of the sso_start_url, not the sha1sum of the
session-name (nixos / f2f059b8b7298f1ad52636d67cef8b719aa83bf5).
Change-Id: Ia1bdec03c4f269a7415c42c90c1f4fd3d928f770
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10012
Reviewed-by: edef <edef@edef.eu>
Tested-by: BuildkiteCI
This was previously only in my Telegram channel, but it might as well
be on the blog itself.
Change-Id: I301ebeaa4dd1875f3858cee5259a5c689b950790
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10009
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
This adds a `parse-bucket-logs.{service,timer}`, running once every
night at 3AM UTC, figuring out the last time it was run and parsing
bucket logs for all previous days.
It invokes the `archeology-parse-bucket-logs` script to produce
a .parquet file with the bucket logs in `s3://nix-cache-log/log/` for
that day (inside a temporary directory), then on success uploads the
produced parquet file to
`s3://nix-archeologist/nix-cache-bucket-logs/yyyy-mm-dd.parquet`.
Change-Id: Ia75ca8c43f8074fbaa34537ffdba68350c504e52
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10011
Reviewed-by: edef <edef@edef.eu>
Tested-by: BuildkiteCI
Clickhouse also has column compression, configurable with the
output_format_parquet_compression_method setting.
It defaults to lz4, and the previous setting got a a zstd-compressed
parquet file with lz4 data.
Set output_format_parquet_compression_method to zstd instead, and sort
by timestamp before assembling the parquet file.
The existing files were updated to the same format with the following query:
```
SELECT * FROM file('bucket_logs_2023-11-11*.pq', 'Parquet', 'auto') ORDER BY timestamp ASC INTO OUTFILE 'bucket_logs_2023-11-11.parquet' SETTINGS output_format_parquet_compression_method = 'zstd'
```
Change-Id: Id63b14c82e7bf4b9907a500528b569a51e277751
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10008
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
This adds a `archeology-parse-bucket-logs` CLI tool to `$PATH`.
It can be invoked like this:
```
archeology-parse-bucket-logs http://nix-cache-log.s3.amazonaws.com/log/2023-11-10-00-* bucket_logs_2023-11-10-00.pq.zstd
````
… and will produce a zstd-compressed Parquet file for (roughly) that
time range.
As the EC2 instance credentials don't give access to the logs bucket
(yet), other AWS credentials need to be provided.
This can be accomplished by using "AWS_ACCESS_KEY_ID",
"AWS_SECRET_ACCESS_KEY", "AWS_SESSION_TOKEN" from
"Option 2: Manually add a profile to your AWS credentials file (Short-
term credentials)" in AWS IAM Identity Center.
Processing logs for a one-hour range takes a minute or two, the
resulting zstd-compressed Parquet file is around 40-80M in size.
Processing logs for a whole day takes some 25mins, due to the sheer
amount of data (12 GB of raw log data, distributed among 450k individual
files, 20Mio log lines), but at least clickhouse isn't able to parse the
resulting parquet file back in:
> Code: 36. DB::Exception: IOError: Couldn't deserialize thrift: MaxMessageSize reached
For future automation tasks, it's probably better to run this once an
hour, and further join the data later on.
Change-Id: I6c8108c0ec17dc8d4e2dbe923175553325210a5c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10007
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Rather than picking up from clickhouse-specific config files, this gets
it to pick up from the ambient environment, which is closer to (but not
the same as) the AWS default credentials chain.
Change-Id: I9c498c231974ed345c3e3d354ec230052b4d0ff2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10006
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
This behaviour might change (or not), see https://github.com/ClickHouse/
ClickHouse/pull/42003, but as of now, a `--progress` will provide some
progress.
Change-Id: I4891b6e2f96f2656858e71f88a226d24f0d45dc3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10005
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
First pass at an xdotool-based command to edit the current text input in
emacs
Change-Id: I1e04612478292fe83083d197d481e034a9fce97f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9971
Reviewed-by: grfn <grfn@gws.fyi>
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
This can apparently work around some of the CPU throttling bugs on
~modern~ computers.
Change-Id: I807ece85d3eba53857a1cb1e73a33f7924538e96
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9895
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Expose `deps` separately, add a direnv with PATH_add for it to bring
tooling into $PATH.
Change-Id: I432cd2b082cad89e08bef78dc4653e10e137cd6b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9842
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Avoid having to re-enter the shell whenever the config is changed.
Change-Id: Ib9f6bb4075e29acaeb4863d64c017695ca85b60b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9841
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
This add the EC2 box config to the repo.
Change-Id: Id7a888a2cfbf1454cd9f9465018df377e14b4e9f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9836
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
This adds a deploy-archeology script.
I tried getting morph to work first, but passing it a
depot.ops.nixos.nixosFor seems to be very hard - the NixOS module system
doesn't like the arguments it's called with.
Replace morph with a 3 line bash script, which assumes your ssh_config
contains config for an `archeology` host.
Change-Id: I2bf694c60ded39c201efbbb899f3b5512aa4d0f2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9835
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: edef <edef@edef.eu>
And of course I managed to move the cache creation into the handlers,
instead of doing it before starting the webserver.
And now I managed to create a hopeless mess of callbacks, but oh well.
Change-Id: I73c3aeced71923c7372496286a279e326b20c388
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9813
Reviewed-by: Profpatsch <mail@profpatsch.de>
Autosubmit: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
I’ve been wanting to experiment with this stuff for a while,
abstracting away a handler type.
The existentials for parser and body took a bit of mucking about, but
in the end hiding the variable behind a `Body` constructor did the
trick.
Now every handler has its own cache, which means we can start caching
arbitrary results.
Change-Id: If57230c47f97ef4c548683f2c2f27660817a31f2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9812
Autosubmit: Profpatsch <mail@profpatsch.de>
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
Now this is getting cool. After 5 minutes we will ask the backend
again (which takes like 3 seconds), but then we compare the old cached
result with the new result and only send it back to the client iff it
changed.
So the client will still have to wait for the roundtrip time, but
doesn’t have to pay for the content. Plus, it gets some info that
upstream hasn’t been updated.
Change-Id: I6dba40321949da5da6a16b2e799d939573c77ba7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9811
Autosubmit: Profpatsch <mail@profpatsch.de>
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
This is a dumb experiment to see how hard it is to respect cache
headers; turns out, medium hard but doable.
Sets the correct expiry time according to the cache, plus respects
`If-Modified-Since` which is a tiny bit harder.
Change-Id: I9e6166af0fa254df2beb0f3919187b91a407487b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9810
Autosubmit: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
Okay, so I guess you also have to seq the cache and everything in
between the IORef and the data.
Change-Id: I4c79c99afbd09e83e9d7a01d58b31b36862e4d11
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9807
Reviewed-by: Profpatsch <mail@profpatsch.de>
Autosubmit: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
