Commit graph

6 commits

Author SHA1 Message Date
Florian Klink
b65d40261b fix(users/flokli/nixos-tvix-cache): drop private bind mounts
The mount didn't get applied for some reason, explicitly configure the
path.

Change-Id: Ie41eb3c1d5f6416493211fb77709aaeecf61edf0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12924
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
2024-12-27 15:50:23 +00:00
Florian Klink
b36f2e3a32 fix(users/flokli/nixos-tvix-cache): BindPaths is serviceConfig
Putting this into UnitConfig won't work, so the bind mount didn't
happen, causing the blobs to be created on the SSD too.

This was already deployed and the data migrated over.

Change-Id: Ie30c8f458cdad8b764817a48a048ec3ca3c18e64
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12922
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-12-27 12:54:29 +00:00
Florian Klink
b2a2225b8b feat(users/flokli/nixos-tvix-cache): put metadata on SSD
Move the Directory and PathInfo storage to the SSD, and only bind-mount
the blob storage from the HDD.

This should improve IO for random access.

Change-Id: Icf9408a879dec8a52541953682ffac25b31e73d3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12921
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-12-27 11:58:01 +00:00
zimbatm
88e65b5c33 fix(users/flokli/nixos-tvix-cache): bump nginx read timeout
This is a bandaid until we have a proper fix.

Change-Id: Id9f0bab5f309a7796c1efee23071013618c6dd12
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12896
Autosubmit: Jonas Chevalier <zimbatm@zimbatm.com>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2024-12-20 17:04:40 +00:00
Florian Klink
ae76eaa761 feat(users/flokli/nixos-tvix-cache): re-enable http2
With nar-bridge supporting zstd content-encoding, we don't need the
nginx zstd module and can re-enable http2.

We also need to propagate the Accept-Encoding sent by the client to
nar-bridge, so it actually knows it can send zstd.

This reduces the time measured in the microbenchmark from ~13s to this:

```
hyperfine 'rm -rf /tmp/cache; nix copy --from https://nixos.tvix.store/ --to "file:///tmp/cache?compression=none" /nix/store/jlkypcf54nrh4n6r0l62ryx93z752hb2-firefox-132.0'
Benchmark 1: rm -rf /tmp/cache; nix copy --from https://nixos.tvix.store/ --to "file:///tmp/cache?compression=none" /nix/store/jlkypcf54nrh4n6r0l62ryx93z752hb2-firefox-132.0
  Time (mean ± σ):      4.880 s ±  0.207 s    [User: 4.661 s, System: 2.377 s]
  Range (min … max):    4.700 s …  5.274 s    10 runs
```

Change-Id: Id092307423636163ae95ef87ec8fa558b83ce0bb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12835
Reviewed-by: Jörg Thalheim <joerg@thalheim.io>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
2024-11-24 18:34:04 +00:00
Florian Klink
52a8e47ac1 feat(users/flokli/nixos/nixos-tvix-cache): init
This is a fetch-through mirror of cache.nixos.org, hosted by NumTide.

The current machine is a SX65 Hetzner dedicated server with 4x22TB SATA disks,
and 2x1TB NVMe disks.

The goals of this machine:

 - Exercise tvix-store and nar-bridge code
 - Collect usage metrics (see https://nixos.tvix.store/grafana)
 - Identify bottlenecks
 - Replace cache.nixos.org?

Be however aware that there's zero availability guarantees. Since Tvix doesn't
support garbage collection yet, we either will delete data or order a bigger
box.

Change-Id: Id24baa18cae1629a06caaa059c0c75d4a01659d5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12811
Tested-by: BuildkiteCI
Reviewed-by: Jonas Chevalier <zimbatm@zimbatm.com>
Reviewed-by: flokli <flokli@flokli.de>
2024-11-23 09:40:21 +00:00