tvl-depot/users/flokli/presentations/2024-10-25-nixcon-tvix/presentation.md

author	date	title	theme	revealOptions
Florian Klink	2024-10-25	Tvix: Status update	solarized	transition fade

Tvix: Status update

2024-10-25

Florian Klink (flokli)

---

Whoami

• flokli
• nixpkgs contributor since 2018, maintaining systemd, nsncd and more
• Freelance Nix/DevOps consultant

---

What is Tvix?

---

---

What is Tvix?

• A Rust re-implementation of the components of the Nix package manager
• Uses different underlying approaches while retaining Nix compatibility "on the surface"
• modular architecture, allowing to recombine aspects to solve your usecase
• No "end-user CLI" for now, focus on getting the foundational architecture right and 100% correctness with Nix

---

Topics

• Component overview
• Recent developments
• Next steps

Note: Component overview, to understand a bit better how it's different / updates since the last talk / outlook on roadmap

---

Structure

• tvix-castore, the granular data storage/syncing engine.
• tvix-store, the "Nix store implementation" on top of tvix-castore
• nix-compat, a library providing access to data formats, protocols and concepts
• tvix-eval, a bytecode interpreter evaluator and core Nix language concepts and builtins
• tvix-build, a generic builder interface
• tvix-glue, combines tvix-eval with tvix-[ca]store and tvix-build

Note: one big cargo workspace / go into detail later! / nix-compat: concepts like output path calculation, doesn't depend on tvix crates tvix-eval: language concepts being Scopes, Thunks, Nix values, "core builtins"

---

Note: Nix does do content addressing on a store path level, we're on a per-file/chunk in file. That model allows granular syncing and reuse of parts, which will speedup substitution/copying. Because everything is ca, it'll also allow decentralization and local p2p substitution. Think about everyone in the same room serving chunks. verified streaming.

---

Note: track all metadata about store paths (think about the sqlite db), and link to castore instead of storing NAR. NarCalculation: this computation, is super nice to cache, as this info never needs to expire, and is reconstructable in a pure CA manner.

---

Note: kept as a somewhat separate library, Tvix "first consumer". But use from your code, it doesn't depend on Tvix bits. Regularly factoring out Nix concepts into this library

---

Note: only includes basic builtins, like string manipulation, math, …. Other crates can bring their own builtins

---

Note: Not aware of Nix, store paths etc. just flexible enough to express everything in there. Use it for your own build system!

---

Note: connects the evaluator to store and builders

---

Note: Example on how to use this: Nix HTTP binary cache lens into tvix-[ca]store, allows Nix to download from and push into. It renders NARs on-the-fly

---

Updates

Rough overview. Check blog posts and git log for details!

---

Updates (#1)

• Fixes on error catchability and context behaviour
• More compact Nix Value types (memory-wise)
• tvix-cli REPL global scope manipulation (assign variables and use them in next command)
• firefox.outPath and pkgsCross.aarch64-multiplatform.firefox.outPath correct and added to CI 🎉

Notes: catchability/context to align behaviour with nix / … / assign variables and use them in the next REPL command

---

Updates (#2)

• OpenTelemetry integration, trace propagation throughout the entire stack
• more backends in tvix-[ca]store (object_store/local fs/redb/bigtable/…)
• nar-bridge RIIR, was deployed as a fetch-through cache for cache.nixos.org at Bornhack
• store composition/combinators
• wiring up of builds (without reference propagation yet, but reference scanning)

Notes: o11y already proven super helpful for debugging where time is spent / first version of composition / builds waiting on reference propagation

---

Other Updates

• A lot of the tooling for cache.nixos.org usage / closure analysis making use of and contributing to nix-compat (@edef) 🎉
• "Replit using tvix-[ca]store and reporting 10x storage reduction" (@cbrewster) 🎉
• "Devenv is switching to Tvix" (Talk on Saturday 12:55, @domenkozar) 🎉

---

Next steps (in no specific order):

• Test suite classification system, to decouple test cases from test runner and share with other Nix impls
• (Continuous) Docs deployment, website restructuring
• Interactions with the evaluator (LSP, DAP)
• Persistent deployment of nar-bridge, as fetch-through cache for cache.nixos.org
• Blob / Chunking protocol improvements (use local chunks where possible, allow readahead)

Note: Allows filtering, reusing test cases in Nixcpp and Lix. Mention some behavorial changes found. Mention fetchTree Make it easier for new contributors to get started LSP / Debug adapter protocol / tvix-eval jobs / … / is gonna improve performance for IO into store paths

---

Next steps (in no specific order) (cont.):

• "Build/Fetch realization goal engine" (tradeoff network bandwidth and CPU time)
• More per-store metrics and instance names
• More backends (p2p discovery, ipfs, …)

Note: needed for builds / to get better insights into cache hit ratios etc.

---

Contributing

• Join the IRC channel (#tvix-dev on hackint), bridged to Matrix and XMPP
• Check our issue tracker (b.tvl.fyi), as well as tvix/docs/src/TODO.md (but ask!)
• Try to use it and tell us how you broke it!
• Sponsoring

Note: make sure to ask in the channel to ensure noone is already working on this

---

Thanks to...

• all Tvix contributors
• Nix community members for their input on the architecture
• Sponsors

Note: some drive-by, some sticking around longer / NLNET / Clan

---

Questions?

Florian Klink / flokli.de

Tvix / tvix.dev