No description
Find a file
Adam Joseph 1e80b9ea8b chore(tvix/eval): mark async functions which are called by the VM
Given Rust's current lack of support for tail calls, we cannot avoid
using `async` for builtins.  This is the only way to avoid
overflowing the cpu stack when we have arbitrarily deep
builtin/interpreted/builtin/interpreted/... "sandwiches"

There are only five `async fn` functions which are not builtins
(some come in multiple "flavors"):

- add_values
- resolve_with
- force, final_deep_force
- nix_eq, nix_cmp_eq
- coerce_to_string

These can be written iteratively rather than recursively (and in
fact nix_eq used to be written that way!).  I volunteer to rewrite
them.  If written iteratively they would no longer need to be
`async`.

There are two motivations for limiting our reliance on `async` to
only the situation (builtins) where we have no other choice:

1. Performance.

   We don't really have any good measurement of the performance hit
   that the Box<dyn Future>s impose on us.  Right now all of our
   large (nixpkgs-eval) tests are swamped by the cost of other
   things (e.g. fork()ing `nix-store`) so we can't really measure
   it.  Builtins tend to be expensive operations anyways
   (regexp-matching, sorting, etc) that are likely to already cost
   more than the `async` overhead.

2. Preserving the ability to switch to `musttail` calls.

   Clang/LLVM recently got `musttail` (mandatory-elimination tail
   calls).  Rust has refused to add this mainly because WASM doesn't
   support, but WASM `tail_call` has been implemented and was
   recently moved to phase 4 (standardization).  It is very likely
   that Rust will get tail calls sometime in the next year; if it
   does, we won't need async anymore.  In the meantime, I'd like to
   avoid adding any further reliance on `async` in places where it
   wouldn't be straightforward to replace it with a tail call.

https://reviews.llvm.org/D99517

https://github.com/WebAssembly/proposals/pull/157

https: //github.com/rust-lang/rfcs/issues/2691#issuecomment-1462152908
Change-Id: Id15945d5a92bf52c16d93456e3437f91d93bdc57
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8290
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: Adam Joseph <adam@westernsemico.com>
2023-03-13 21:33:58 +00:00
.gcroots feat(.envrc): gcroot third_party.sources 2022-09-15 11:27:53 +00:00
.nixery feat(ops/modules): Add module for running Nixery 2021-08-12 14:55:59 +00:00
corp fix(predlozhnik): use correct link to source code after move 2023-03-10 14:30:37 +00:00
docs docs(REVIEWS): update scp command 2023-02-10 15:15:36 +00:00
fun chore(3p/sources): Bump channels & overlays 2022-12-24 12:42:41 +00:00
lisp chore(gerrit): migrate OWNERS files to code-owners style 2022-09-19 11:13:28 +00:00
net chore(3p/sources): Bump channels & overlays 2022-09-28 08:02:31 +00:00
nix feat(nix/readTree): implement .skip-tree marker for subtree ignoring 2023-02-28 12:15:33 +00:00
ops fix(ops/terraform): s/TARGET_ADDRESS/TARGET_HOST 2023-03-08 16:59:19 +00:00
third_party chore(3p/sources): Bump channels & overlays 2023-03-07 21:09:35 +00:00
tools fix(nixery): fix link to nixery logo 2023-02-06 17:36:54 +00:00
tvix chore(tvix/eval): mark async functions which are called by the VM 2023-03-13 21:33:58 +00:00
users chore(users/lukegb/keys): -porcorosso-wsl +lukegb-build +lukegb-ca 2023-03-12 03:35:24 +00:00
views docs(views): extend README for josh-filter usage 2023-03-03 14:56:04 +00:00
web feat(web/inbox): add landing page for inbox.tvl.su 2022-12-28 08:17:45 +00:00
.envrc feat(.envrc): gcroot third_party.sources 2022-09-15 11:27:53 +00:00
.git-blame-ignore-revs fix: add cl/4397 (treewide nixpkgs-fmt) to git-blame-ignore-revs 2022-02-07 18:15:09 +00:00
.gitignore feat(.envrc): gcroot third_party.sources 2022-09-15 11:27:53 +00:00
.hgignore chore(hgignore): ignore .git for hg 2020-06-14 18:23:13 +00:00
.mailmap chore(mailmap): add my name to mailmap 2020-07-18 18:15:05 +00:00
.rgignore chore: Only exclude //third_party/git from ripgrep 2020-05-17 23:58:22 +01:00
buf.gen.yaml feat(nix/bufCheck): ensure .pb.go is up to date 2022-12-27 13:27:40 +00:00
buf.yaml chore(buf): Use nixpkgs-provided buf 2022-10-21 18:39:03 +00:00
default.nix fix: make depot-gcroot derivation a plain list of out paths 2023-03-08 16:59:08 +00:00
LICENSE chore(LICENSE): happy new year! 2022-11-26 00:40:57 +00:00
OWNERS chore(gerrit): migrate OWNERS files to code-owners style 2022-09-19 11:13:28 +00:00
README.md docs(README.md): reflect recent upheaval in depot 2022-05-27 23:24:28 +00:00
RULES feat(whitby): Let sterni bear the wheel 2021-05-23 19:06:15 +00:00
rustfmt.toml feat(depotfmt): Check & format Rust code with rustfmt 2022-02-08 12:06:39 +00:00

depot

Build status

This repository is the monorepo for the community around The Virus Lounge, containing our personal tools and infrastructure. Everything in here is built using Nix.

A large portion of the software here is very self-referential, meaning that it exists to sustain the operation of the repository. This is the case because we partially see this as an experiment in tooling for monorepos.

Highlights

Services

  • Source code is available primarily through Sourcegraph on cs.tvl.fyi, where it is searchable and even semantically indexed. A lower-tech view of the repository is also available via cgit-pink on code.tvl.fyi.

    The repository can be cloned using git from https://cl.tvl.fyi/depot.

  • All code in the depot, with the exception of code that is checked in to individual //users folders, needs to be reviewed. We use Gerrit on cl.tvl.fyi for this.

  • Issues are tracked via our own issue tracker on b.tvl.fyi. Its source code lives at //web/panettone/.

  • Smaller todo-list entries which do not warrant a separate issue are listed at todo.tvl.fyi.

  • We use Buildkite for CI. Recent builds are listed on tvl.fyi/builds and pipelines are configured dynamically via //ops/pipelines.

  • A search service that makes TVL services available via textual shortcuts is available: atward

All services that we host are deployed on NixOS machines that we manage. Their configuration is tracked in //ops/{modules,machines}.

Nix

  • //nix/readTree contains the Nix code which automatically registers projects in our Nix attribute hierarchy based on their in-tree location
  • //tools/nixery contains the source code of Nixery, a container registry that can build images ad-hoc from Nix packages
  • //nix/yants contains Yet Another Nix Type System, which we use for a variety of things throughout the repository
  • //nix/buildGo implements a Nix library that can build Go software in the style of Bazel's rules_go. Go programs in this repository are built using this library.
  • //nix/buildLisp implements a Nix library that can build Common Lisp software. Currently only SBCL is supported. Lisp programs in this repository are built using this library.
  • //web/blog and //web/atom-feed: A Nix-based static site generator which generates the web page and Atom feed for tazj.in (//users/tazjin/homepage) and tvl.fyi (//web/tvl)
  • //web/bubblegum contains a CGI-based web framework written in Nix.
  • //nix/nint: A shebang-compatible interpreter wrapper for Nix.
  • //tvix contains initial work towards a modular architecture for Nix.

We have a variety of other tools and libraries in the //nix folder which may be of interest.

Packages / Libraries

  • //net/alcoholic_jwt contains an easy-to-use JWT-validation library for Rust
  • //net/crimp contains a high-level HTTP client using cURL for Rust
  • //tools/emacs-pkgs contains various useful Emacs libraries, for example:
    • dottime.el provides dottime in the Emacs modeline
    • nix-util.el provides editing utilities for Nix files
    • term-switcher.el is an ivy-function for switching between vterm buffers
    • tvl.el provides helper functions for interacting with the TVL monorepo
  • //lisp/klatre provides a grab-bag utility library for Common Lisp

User packages

Contributors to the repository have user directories under //users, which can be used for personal or experimental code that does not require review.

Some examples:

  • //users/grfn/xanthous: A (WIP) TUI RPG, written in Haskell.
  • //users/tazjin/emacs: tazjin's Emacs & EXWM configuration
  • //users/tazjin/finito: A persistent finite-state machine library for Rust.

Licensing

Unless otherwise stated in a subdirectory, all code is licensed under the MIT license. See LICENSE for details.

Contributing

If you'd like to contribute to any of the tools in here, please check out the contribution guidelines and our code of conduct.

IRC users can find us in #tvl on hackint, which is also reachable via XMPP at #tvl@irc.hackint.org (sic!).

Hackint also provide a web chat.