3d2e55ad53
The input adapter streams were input streams yielding either binary or character data that could be constructed from a variable data source. The stream would take care not to destroy the underlying data source (i.e. not close it if it was a stream), so similar to with FILE-PORTIONs, but simpler. Unfortunately, the implementation was quite inefficient: They are ultimately defined in terms of a function that retrieves the next character in the source. This only allows for an implementation of READ-CHAR (and READ-BYTE). Thanks to cl/8559, READ-SEQUENCE can be used on e.g. FILE-PORTION, but this was still negated by a input adapter based on one—then, READ-SEQUENCE would need to fall back on READ-CHAR or READ-BYTE again. Luckily, we can replace BINARY-INPUT-ADAPTER-STREAM and CHARACTER-INPUT-ADAPTER-STREAM with a much simpler abstraction: Instead of extra stream classes, we have a function, MAKE-INPUT-ADAPTER, which returns an appropriate instance of FLEXI-STREAM based on a given source. This way, the need for a distinction between binary and character input adapter is eliminated, since FLEXI-STREAMS supports both binary and character reads (external format is not yet handled, though). Consequently, the :binary keyword argument to MIME-BODY-STREAM can be dropped. flexi-streams provides stream classes for everything except a stream that doesn't close the underlying one. Since we have already implemented this in POSITIONED-FLEXI-INPUT-STREAM, we can split this functionality into a new superclass ADAPTER-FLEXI-INPUT-STREAM. This change also allows addressing the performance regression encountered in cl/8559: It seems that flexi-streams performs worse when we are reading byte by byte or char by char. (After this change mblog is still two times slower than on r/6150.) By eliminating the adapter streams, we can start utilizing READ-SEQUENCE via decoding code that supports it (i.e. qbase64) and bring performance on par with r/6150 again. Surely there are also ways to gain back even more performance which has to be determined using profiling. Buffering more aggressively seems like a sure bet, though. Switching to flexi-streams still seems like a no-brainer, as it allows us to drop a lot of code that was quite hacky (e.g. DELIMITED-INPUT- STREAM) and implements en/decoding handling we did not support before, but would need for improved correctness. Change-Id: Ie2d1f4e42b47512a5660a1ccc0deeec2bff9788d Reviewed-on: https://cl.tvl.fyi/c/depot/+/8581 Autosubmit: sterni <sternenseemann@systemli.org> Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI |
||
---|---|---|
.gcroots | ||
.nixery | ||
corp | ||
docs | ||
fun | ||
lisp | ||
net | ||
nix | ||
ops | ||
third_party | ||
tools | ||
tvix | ||
users | ||
views | ||
web | ||
.envrc | ||
.git-blame-ignore-revs | ||
.gitignore | ||
.hgignore | ||
.mailmap | ||
.rgignore | ||
buf.gen.yaml | ||
buf.yaml | ||
default.nix | ||
LICENSE | ||
OWNERS | ||
README.md | ||
RULES | ||
rustfmt.toml |
depot
This repository is the monorepo for the community around The Virus Lounge, containing our personal tools and infrastructure. Everything in here is built using Nix.
A large portion of the software here is very self-referential, meaning that it exists to sustain the operation of the repository. This is the case because we partially see this as an experiment in tooling for monorepos.
Highlights
Services
-
Source code is available primarily through Sourcegraph on cs.tvl.fyi, where it is searchable and even semantically indexed. A lower-tech view of the repository is also available via cgit-pink on code.tvl.fyi.
The repository can be cloned using
git
fromhttps://cl.tvl.fyi/depot
. -
All code in the depot, with the exception of code that is checked in to individual
//users
folders, needs to be reviewed. We use Gerrit on cl.tvl.fyi for this. -
Issues are tracked via our own issue tracker on b.tvl.fyi. Its source code lives at
//web/panettone/
. -
Smaller todo-list entries which do not warrant a separate issue are listed at todo.tvl.fyi.
-
We use Buildkite for CI. Recent builds are listed on tvl.fyi/builds and pipelines are configured dynamically via
//ops/pipelines
. -
A search service that makes TVL services available via textual shortcuts is available: atward
All services that we host are deployed on NixOS machines that we manage. Their
configuration is tracked in //ops/{modules,machines}
.
Nix
//nix/readTree
contains the Nix code which automatically registers projects in our Nix attribute hierarchy based on their in-tree location//tools/nixery
contains the source code of Nixery, a container registry that can build images ad-hoc from Nix packages//nix/yants
contains Yet Another Nix Type System, which we use for a variety of things throughout the repository//nix/buildGo
implements a Nix library that can build Go software in the style of Bazel'srules_go
. Go programs in this repository are built using this library.//nix/buildLisp
implements a Nix library that can build Common Lisp software. Currently only SBCL is supported. Lisp programs in this repository are built using this library.//web/blog
and//web/atom-feed
: A Nix-based static site generator which generates the web page and Atom feed for tazj.in (//users/tazjin/homepage
) and tvl.fyi (//web/tvl
)//web/bubblegum
contains a CGI-based web framework written in Nix.//nix/nint
: A shebang-compatible interpreter wrapper for Nix.//tvix
contains initial work towards a modular architecture for Nix.
We have a variety of other tools and libraries in the //nix
folder which may
be of interest.
Packages / Libraries
//net/alcoholic_jwt
contains an easy-to-use JWT-validation library for Rust//net/crimp
contains a high-level HTTP client using cURL for Rust//tools/emacs-pkgs
contains various useful Emacs libraries, for example:dottime.el
provides dottime in the Emacs modelinenix-util.el
provides editing utilities for Nix filesterm-switcher.el
is an ivy-function for switching between vterm bufferstvl.el
provides helper functions for interacting with the TVL monorepo
//lisp/klatre
provides a grab-bag utility library for Common Lisp
User packages
Contributors to the repository have user directories under
//users
, which can be used for
personal or experimental code that does not require review.
Some examples:
//users/grfn/xanthous
: A (WIP) TUI RPG, written in Haskell.//users/tazjin/emacs
: tazjin's Emacs & EXWM configuration//users/tazjin/finito
: A persistent finite-state machine library for Rust.
Licensing
Unless otherwise stated in a subdirectory, all code is licensed under the MIT license. See LICENSE for details.
Contributing
If you'd like to contribute to any of the tools in here, please check out the contribution guidelines and our code of conduct.
IRC users can find us in #tvl
on hackint, which is also
reachable via XMPP at #tvl@irc.hackint.org
(sic!).
Hackint also provide a web chat.