No description
Find a file
Vincent Ambo d6c57eb957 refactor(tvix/eval): ensure VM operations fit in a single byte
This replaces the OpCode enum with a new Op enum which is guaranteed to fit in a
single byte. Instead of carrying enum variants with data, every variant that has
runtime data encodes it into the `Vec<u8>` that a `Chunk` now carries.

This has several advantages:

* Less stack space is required at runtime, and fewer allocations are required
  while compiling.
* The OpCode doesn't need to carry "weird" special-cased data variants anymore.
* It is faster (albeit, not by much). On my laptop, results consistently look
  approximately like this:

  Benchmark 1: ./before -E '(import <nixpkgs> {}).firefox.outPath' --log-level ERROR --no-warnings
  Time (mean ± σ):      8.224 s ±  0.272 s    [User: 7.149 s, System: 0.688 s]
  Range (min … max):    7.759 s …  8.583 s    10 runs

  Benchmark 2: ./after -E '(import <nixpkgs> {}).firefox.outPath' --log-level ERROR --no-warnings
  Time (mean ± σ):      8.000 s ±  0.198 s    [User: 7.036 s, System: 0.633 s]
  Range (min … max):    7.718 s …  8.334 s    10 runs

  See notes below for why the performance impact might be less than expected.
* It is faster while at the same time dropping some optimisations we previously
  performed.

This has several disadvantages:

* The code is closer to how one would write it in C or Go.
* Bit shifting!
* There is (for now) slightly more code than before.

On performance I have the following thoughts at the moment:

In order to prepare for adding GC, there's a couple of places in Tvix where I'd
like to fence off certain kinds of complexity (such as mutating bytecode, which,
for various reaons, also has to be part of data that is subject to GC). With
this change, we can drop optimisations like retroactively modifying existing
bytecode and *still* achieve better performance than before.

I believe that this is currently worth it to pave the way for changes that are
more significant for performance.

In general this also opens other avenues of optimisation: For example, we can
profile which argument sizes actually exist and remove the copy overhead of
varint decoding (which does show up in profiles) by using more adequately sized
types for, e.g., constant indices.

Known regressions:

* Op::Constant is no longer printing its values in disassembly (this can be
  fixed, I just didn't get around to it, will do separately).

Change-Id: Id9b3a4254623a45de03069dbdb70b8349e976743
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12191
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2024-08-19 11:02:50 +00:00
.gcroots feat(.envrc): gcroot third_party.sources 2022-09-15 11:27:53 +00:00
.nixery chore(3p/sources): bump to OpenSSH vulnerability hotfix 2024-07-01 17:42:30 +00:00
corp chore(3p/sources): Bump channels & overlays 2024-06-08 15:20:48 +00:00
docs docs: move Registration to the top; minor reword 2024-06-15 17:42:16 +00:00
fun feat(fun/clbot,ops/machines/whitby): filter tvix-dev clbot 2024-06-03 19:35:34 +00:00
lisp chore(depot): update OWNERS files for aspen 2023-12-20 18:35:58 +00:00
net chore(net/crimp): update dependencies 2024-03-11 16:52:45 +00:00
nix chore(3p/sources): Bump channels & overlays (2024-07-28) 2024-08-01 10:06:33 +00:00
ops feat(ops/pipelines): support buildkite retries 2024-08-19 10:07:08 +00:00
third_party chore(3p/sources): Bump channels & overlays 2024-08-14 14:10:44 +00:00
tools fix(tvix): Reuse now exposed depotfmt wrapper in crate2nix-check 2024-07-31 20:07:43 +00:00
tvix refactor(tvix/eval): ensure VM operations fit in a single byte 2024-08-19 11:02:50 +00:00
users fix(aspen/system): Remove duplicate email 2024-08-15 01:28:10 +00:00
views docs(views): update where josh-filter is coming from 2024-08-18 20:44:39 +00:00
web refactor(tvix/eval): ensure VM operations fit in a single byte 2024-08-19 11:02:50 +00:00
.envrc feat(.envrc): gcroot third_party.sources 2022-09-15 11:27:53 +00:00
.git-blame-ignore-revs fix: add cl/4397 (treewide nixpkgs-fmt) to git-blame-ignore-revs 2022-02-07 18:15:09 +00:00
.gitignore feat(buildkite): avoid building extraSteps in pipeline construction 2024-03-07 15:39:56 +00:00
.hgignore chore(hgignore): ignore .git for hg 2020-06-14 18:23:13 +00:00
.mailmap chore(mailmap): Add my name and email 2024-08-11 01:23:46 +00:00
.rgignore chore: Only exclude //third_party/git from ripgrep 2020-05-17 23:58:22 +01:00
buf.gen.yaml feat(nix/bufCheck): ensure .pb.go is up to date 2022-12-27 13:27:40 +00:00
buf.yaml chore(buf): Use nixpkgs-provided buf 2022-10-21 18:39:03 +00:00
default.nix fix(aspen/system): Fix and reenable system build 2024-04-07 14:19:01 +00:00
LICENSE chore: another year of licensing 2023-04-09 13:34:18 +00:00
OWNERS chore(depot): update OWNERS files for aspen 2023-12-20 18:35:58 +00:00
README.md chore(users): grfn -> aspen 2024-02-14 19:37:41 +00:00
RULES feat(whitby): Let sterni bear the wheel 2021-05-23 19:06:15 +00:00
rustfmt.toml feat(depotfmt): Check & format Rust code with rustfmt 2022-02-08 12:06:39 +00:00

depot

Build status

This repository is the monorepo for the community around The Virus Lounge, containing our personal tools and infrastructure. Everything in here is built using Nix.

A large portion of the software here is very self-referential, meaning that it exists to sustain the operation of the repository. This is the case because we partially see this as an experiment in tooling for monorepos.

Highlights

Services

  • Source code is available primarily through Sourcegraph on cs.tvl.fyi, where it is searchable and even semantically indexed. A lower-tech view of the repository is also available via cgit-pink on code.tvl.fyi.

    The repository can be cloned using git from https://cl.tvl.fyi/depot.

  • All code in the depot, with the exception of code that is checked in to individual //users folders, needs to be reviewed. We use Gerrit on cl.tvl.fyi for this.

  • Issues are tracked via our own issue tracker on b.tvl.fyi. Its source code lives at //web/panettone/.

  • Smaller todo-list entries which do not warrant a separate issue are listed at todo.tvl.fyi.

  • We use Buildkite for CI. Recent builds are listed on tvl.fyi/builds and pipelines are configured dynamically via //ops/pipelines.

  • A search service that makes TVL services available via textual shortcuts is available: atward

All services that we host are deployed on NixOS machines that we manage. Their configuration is tracked in //ops/{modules,machines}.

Nix

  • //nix/readTree contains the Nix code which automatically registers projects in our Nix attribute hierarchy based on their in-tree location
  • //tools/nixery contains the source code of Nixery, a container registry that can build images ad-hoc from Nix packages
  • //nix/yants contains Yet Another Nix Type System, which we use for a variety of things throughout the repository
  • //nix/buildGo implements a Nix library that can build Go software in the style of Bazel's rules_go. Go programs in this repository are built using this library.
  • //nix/buildLisp implements a Nix library that can build Common Lisp software. Currently only SBCL is supported. Lisp programs in this repository are built using this library.
  • //web/blog and //web/atom-feed: A Nix-based static site generator which generates the web page and Atom feed for tazj.in (//users/tazjin/homepage) and tvl.fyi (//web/tvl)
  • //web/bubblegum contains a CGI-based web framework written in Nix.
  • //nix/nint: A shebang-compatible interpreter wrapper for Nix.
  • //tvix contains initial work towards a modular architecture for Nix.

We have a variety of other tools and libraries in the //nix folder which may be of interest.

Packages / Libraries

  • //net/alcoholic_jwt contains an easy-to-use JWT-validation library for Rust
  • //net/crimp contains a high-level HTTP client using cURL for Rust
  • //tools/emacs-pkgs contains various useful Emacs libraries, for example:
    • dottime.el provides dottime in the Emacs modeline
    • nix-util.el provides editing utilities for Nix files
    • term-switcher.el is an ivy-function for switching between vterm buffers
    • tvl.el provides helper functions for interacting with the TVL monorepo
  • //lisp/klatre provides a grab-bag utility library for Common Lisp

User packages

Contributors to the repository have user directories under //users, which can be used for personal or experimental code that does not require review.

Some examples:

  • //users/aspen/xanthous: A (WIP) TUI RPG, written in Haskell.
  • //users/tazjin/emacs: tazjin's Emacs & EXWM configuration
  • //users/tazjin/finito: A persistent finite-state machine library for Rust.

Licensing

Unless otherwise stated in a subdirectory, all code is licensed under the MIT license. See LICENSE for details.

Contributing

If you'd like to contribute to any of the tools in here, please check out the contribution guidelines and our code of conduct.

IRC users can find us in #tvl on hackint, which is also reachable via XMPP at #tvl@irc.hackint.org (sic!).

Hackint also provide a web chat.