mdebray/tvl-depot
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

feat(users/flokli/nixos-tvix-cache): re-enable http2

Browse source 
With nar-bridge supporting zstd content-encoding, we don't need the
nginx zstd module and can re-enable http2.

We also need to propagate the Accept-Encoding sent by the client to
nar-bridge, so it actually knows it can send zstd.

This reduces the time measured in the microbenchmark from ~13s to this:

```
hyperfine 'rm -rf /tmp/cache; nix copy --from https://nixos.tvix.store/ --to "file:///tmp/cache?compression=none" /nix/store/jlkypcf54nrh4n6r0l62ryx93z752hb2-firefox-132.0'
Benchmark 1: rm -rf /tmp/cache; nix copy --from https://nixos.tvix.store/ --to "file:///tmp/cache?compression=none" /nix/store/jlkypcf54nrh4n6r0l62ryx93z752hb2-firefox-132.0
  Time (mean ± σ):      4.880 s ±  0.207 s    [User: 4.661 s, System: 2.377 s]
  Range (min … max):    4.700 s …  5.274 s    10 runs
```

Change-Id: Id092307423636163ae95ef87ec8fa558b83ce0bb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12835
Reviewed-by: Jörg Thalheim <joerg@thalheim.io>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
This commit is contained in:
Florian Klink 2024-11-24 09:42:14 +02:00 committed by clbot
parent f6aee3534b
commit ae76eaa761
1 changed files with 5 additions and 20 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

25
users/flokli/nixos/nixos-tvix-cache/nar-bridge.nix
View file

@ -4,25 +4,9 @@
# Microbenchmark # Microbenchmark
# hyperfine --warmup 1 'rm -rf /tmp/cache; nix copy --from https://nixos.tvix.store/ --to "file:///tmp/cache?compression=none" /nix/store/jlkypcf54nrh4n6r0l62ryx93z752hb2-firefox-132.0' # hyperfine --warmup 1 'rm -rf /tmp/cache; nix copy --from https://nixos.tvix.store/ --to "file:///tmp/cache?compression=none" /nix/store/jlkypcf54nrh4n6r0l62ryx93z752hb2-firefox-132.0'
# From a different hetzner machine with 1Gbps uplink:
# - with zstd: 13.384s
# - with gzip: 11.130s
# - with brotli: ~18s
# - without compression: 15.6s
# From a 1Gbit link in TUM:
# - with zstd: 32.292s
# - with gzip: 51s
# - cache.nixos.org from the same connection: 36.559s
services.nginx = { services.nginx = {
package = pkgs.nginxStable.override { package = pkgs.nginxStable;
modules = [ pkgs.nginxModules.zstd ];
};
virtualHosts.${config.machine.domain} = { virtualHosts.${config.machine.domain} = {
# when using http2 we actually see worse throughput,
# because it only uses a single tcp connection,
# which pins nginx to a single core.
http2 = false;
locations."=/" = { locations."=/" = {
tryFiles = "$uri $uri/index.html =404"; tryFiles = "$uri $uri/index.html =404";
root = pkgs.runCommand "index" root = pkgs.runCommand "index"
@ -42,6 +26,10 @@
# nar bridge allows to upload nars via PUT # nar bridge allows to upload nars via PUT
deny all; deny all;
} }
# Propagate content-encoding to the backend
proxy_set_header Accept-Encoding $http_accept_encoding;
# Enable proxy cache # Enable proxy cache
proxy_cache nar-bridge; proxy_cache nar-bridge;
proxy_cache_key "$scheme$proxy_host$request_uri"; proxy_cache_key "$scheme$proxy_host$request_uri";
@ -49,9 +37,6 @@
proxy_cache_valid 404 1m; # Cache 404 responses for 1 minute proxy_cache_valid 404 1m; # Cache 404 responses for 1 minute
proxy_cache_min_uses 2; # Cache only if the object is requested at least twice proxy_cache_min_uses 2; # Cache only if the object is requested at least twice
proxy_cache_use_stale error timeout updating; proxy_cache_use_stale error timeout updating;
zstd on;
zstd_types application/x-nix-nar;
''; '';
}; };
}; };