feat(users/flokli): add ASG Lightning talk presentation
This adds the materials for the lightning talk held at All Systems Go Conference 2023, Berlin. Talk lives at https://media.ccc.de/v/all-systems-go-2023-245-tvix-store Change-Id: I114b1aec9f1953c148dd29ca88888c16b9fc655d Reviewed-on: https://cl.tvl.fyi/c/depot/+/9751 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
This commit is contained in:
parent
b8ef3e5263
commit
8b09ae54b1
3 changed files with 187 additions and 0 deletions
|
@ -0,0 +1,32 @@
|
|||
{ depot, pkgs, ... }:
|
||||
|
||||
let
|
||||
inherit (pkgs)
|
||||
fontconfig qrencode runCommand stdenv;
|
||||
mkQr = url: runCommand "qrcode.png" { } ''
|
||||
${qrencode}/bin/qrencode -o $out -t SVG -s 5 \
|
||||
--background=fafafa \
|
||||
--foreground=000000 \
|
||||
${url}
|
||||
'';
|
||||
in
|
||||
stdenv.mkDerivation {
|
||||
name = "2023-asg-tvix-store";
|
||||
src = ./.;
|
||||
|
||||
FONTCONFIG_FILE = pkgs.makeFontsConf {
|
||||
fontDirectories = with pkgs; [ jetbrains-mono fira fira-code fira-mono lato ];
|
||||
};
|
||||
|
||||
nativeBuildInputs = [ pkgs.reveal-md pkgs.graphviz ];
|
||||
|
||||
buildPhase = ''
|
||||
cp ${depot.tvix.logo}/logo.png tvix-logo.png
|
||||
cp ${mkQr "https://flokli.de"} qrcode-flokli.svg
|
||||
cp ${mkQr "https://tvix.dev"} qrcode-tvix.svg
|
||||
|
||||
mkdir -p $out
|
||||
cp tvix-store-graph-blob-directory.svg $out/
|
||||
reveal-md --static $out presentation.md
|
||||
'';
|
||||
}
|
|
@ -0,0 +1,138 @@
|
|||
---
|
||||
author:
|
||||
- Florian Klink
|
||||
date: 2023-09-09
|
||||
title: "tvix-store: A content-addressed file system and sync protocol"
|
||||
theme: moon
|
||||
revealOptions:
|
||||
transition: 'fade'
|
||||
---
|
||||
|
||||
## tvix-store
|
||||
### A content-addressed file system and sync protocol
|
||||
|
||||
2023-09-13
|
||||
|
||||
Florian Klink / flokli
|
||||
|
||||
---
|
||||
|
||||
## Whoami
|
||||
|
||||
- <!-- .element: class="fragment" -->
|
||||
flokli
|
||||
- <!-- .element: class="fragment" -->
|
||||
Nix/NixOS contributor
|
||||
- maintain systemd, nss and more low-level stuff there
|
||||
- <!-- .element: class="fragment" -->
|
||||
Freelance Nix/DevOps consultant
|
||||
|
||||
Note: more Kubernetes/DevOps exposure with work
|
||||
|
||||
---
|
||||
|
||||
## What is tvix-store?
|
||||
- <!-- .element: class="fragment" -->
|
||||
A new implementation of a content-addressed "storage system"
|
||||
- <!-- .element: class="fragment" -->
|
||||
part of the Tvix Project, a (WIP) reimplementation of Nix and auxillary components in Rust
|
||||
- <!-- .element: class="fragment" -->
|
||||
Storage model: think about git trees and its Merkle DAG…
|
||||
- <!-- .element: class="fragment" -->
|
||||
… but with nicer wire format (`.proto`) and hash function (blake3)
|
||||
|
||||
---
|
||||
|
||||
## Storage model
|
||||
- <!-- .element: class="fragment" -->
|
||||
Once you know the root: everything else is content-addressed
|
||||
- <!-- .element: class="fragment" -->
|
||||
No timestamps, no uid/gid, no xattrs, only one way to represent the same tree
|
||||
- <!-- .element: class="fragment" -->
|
||||
Automatic dedup of identical subtrees in different file system trees
|
||||
- <!-- .element: class="fragment" -->
|
||||
Automatic dedup of identical blobs (and you can do more chunking underneath too)
|
||||
|
||||
---
|
||||
|
||||
## Storage model (cont.)
|
||||
- <!-- .element: class="fragment" -->
|
||||
Granular seekable access into blobs
|
||||
- <!-- .element: class="fragment" -->
|
||||
verified streaming thanks to BLAKE3 and Bao, faulty data is detected early on
|
||||
- <!-- .element: class="fragment" -->
|
||||
Everything below can be retrieved from anyone without having to trust (P2P substitution, CDNs, …)
|
||||
|
||||
---
|
||||
|
||||
## Usecases
|
||||
- <!-- .element: class="fragment" -->
|
||||
File system tree delivery
|
||||
- <!-- .element: class="fragment" -->
|
||||
Container image delivery
|
||||
- <!-- .element: class="fragment" -->
|
||||
Backing store for VCS
|
||||
- <!-- .element: class="fragment" -->
|
||||
Granular access into large datasets
|
||||
|
||||
---
|
||||
|
||||
## Status
|
||||
- <!-- .element: class="fragment" -->
|
||||
In-memory backend, a local K/V backend (Sled)
|
||||
- <!-- .element: class="fragment" -->
|
||||
FUSE filesystem
|
||||
- <!-- .element: class="fragment" -->
|
||||
A gRPC API to transfer things, bindings for golang and rust
|
||||
- <!-- .element: class="fragment" -->
|
||||
some object storage backends in development (GCS, NATS)
|
||||
- <!-- .element: class="fragment" -->
|
||||
FUTUREWORK: more storage backends / store composition / in-kernel module?
|
||||
|
||||
Notes: of course you can use your own network protocol too, like HTTP CAS or iroh....plug different stores together to represent caches, blobfs
|
||||
|
||||
---
|
||||
|
||||
## Contributing
|
||||
|
||||
- <!-- .element: class="fragment" -->
|
||||
Join the IRC channel (`#tvl` on `hackint`), bridged to Matrix and XMPP
|
||||
- <!-- .element: class="fragment" -->
|
||||
Check our issue tracker
|
||||
- <!-- .element: class="fragment" -->
|
||||
Try to use it and tell us how you broke it!
|
||||
|
||||
Note: if this sounds useful to you, reach out!
|
||||
|
||||
---
|
||||
|
||||
# Thanks!
|
||||
|
||||
<style>
|
||||
.container{
|
||||
display: flex;
|
||||
}
|
||||
.col{
|
||||
flex: 1;
|
||||
}
|
||||
</style>
|
||||
|
||||
<div class="container">
|
||||
|
||||
<div class="col">
|
||||
Florian Klink / <a href="https://flokli.de">flokli.de</a><br />
|
||||
<img src="qrcode-flokli.svg" />
|
||||
</div>
|
||||
|
||||
<div class="col">
|
||||
Tvix / <a href="https://tvix.dev">tvix.dev</a><br />
|
||||
<img src="qrcode-tvix.svg" />
|
||||
</div>
|
||||
|
||||
</div>
|
||||
|
||||
---
|
||||
|
||||
## Structure
|
||||
|
||||
[tvix-store graph](tvix-store-graph-blob-directory.svg)
|
File diff suppressed because one or more lines are too long
After Width: | Height: | Size: 67 KiB |
Loading…
Reference in a new issue