We aim to produce bit-identical output when roundtripping, and this
applies to legacy formats as well.
Change-Id: Iaec7d6bb5c5e305ec5e1b78c6968226dee9a0d90
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12943
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
We know these names statically, so we can just use &'static str.
Change-Id: I81cb7ecc4d7553f57baca74464c120a143586fe6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12941
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Last one of the year! С наступающим)
Fixes:
* users/wpcarro: remove use-package from emacs packages (it has been built-in
for a while now)
* users/sterni: the same thing
* users/aspen: remove `coz`, forwardport `gdmap` from stable
* users/flokli: dropped corneish_zen firmware from CI
This firmware depends on a non-reproducible FOD which, when updated, causes
build failures. We have worked around this repeatedly, but it needs to be
fixed properly.
* tvix: regenerate Go protobufs
* tvix: address new clippy lints
* tvix/{castore,store,build}-go: update grpc/protobuf libraries
* tvix/eval: formatting fixes
* 3p/overlays/tvl: work around GCC 14 -Werrors
Change-Id: Ice5948ca7780192fb7d2abc6a48971fb875f03c9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12933
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: aspen <root@gws.fyi>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
readTree gives special treatment to the directory on which it is
invoked -- for example, it won't read *.nix files in that directory.
This commit adds the ability to disable this special treatment, which
remains the default behavior.
Example use case:
10029d3682
Change-Id: I306bea95f4d556f7090f3255e6da6bb410adbf57
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12939
Tested-by: BuildkiteCI
Autosubmit: Adam Joseph <adam@westernsemico.com>
Reviewed-by: tazjin <tazjin@tvl.su>
I'm not really describing what the problem here is because I don't
think a writeup is really useful. It would just be speculation and
I don't need to syncronize my efforts with anyone at the moment,
so it's best to keep those notes offline.
Basically, the next problem I want to tackle is that the initial
parsing of a multipart message (to get the number, types, offsets
etc. of the different parts) is very slow. This is because READ-LINE
on a FLEXI-STREAM dispatches to READ-CHAR which is laughably slow.
Change-Id: Ia5d6e335abb23639cfe9c2149ead99ffa5dbbcf5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12936
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
resolve_local isn't exclusively used to compile variable access, so we
shouldn't automatically mark a local as used when it's called.
Specifically, the optimiser would call `is_user_defined`, causing locals
to be marked as used even when they weren't, erroneously silencing the
unused local variable warning.
For resolve_upvalue I've opted not to do the same, instead renaming the
function to make its behavior clearer. The reason for this is that
resolve_upvalue is only used in the code for the purpose of compiling
variable access and mark_used for upvalues would be needlessly
expensive, as it requires recursing through enclosing contexts.
Supersedes / alternative to cl/12708.
Change-Id: Ib4a4a9dfcecf710ab2766b03d0a0cc09245c2d0a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12869
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
For some reason it keeps getting scaled to 1.25, which is too big.
Change-Id: Idbe938bb94f92c72275ee715d21004a3d9372496
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12935
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
It's only used in my user directory and I don't feel it's generally
useful. There's so little to interpolate you may as well just check them
in.
Change-Id: I31dc9917c2b281c7d68388e1a32c8ef5179621df
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12873
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
This makes the awkward withDoctype utility obsolete which is much nicer.
Technically, this is a BREAKING CHANGE since it was possible to create
valid documents without an <html> tag before:
withDoctype (lib.concatStrings [ (<head> { } …) (<body> { } …) ])
I don't think this usecase is worth preserving since this can just be
written as
<html> { } [ (<head> { } …) (<body> { } …) ]
and omitting the <html> tag is not recommended since it should be used
to set the language of the document (which we didn't in the example
above).
Change-Id: Idc5104ce88fe8bee965c076229b79387915c3605
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12907
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
We don't need to use a git checkout here anymore.
Change-Id: I99641a4908f39799c8be6a1610ae458ca6fdc5ee
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12931
Tested-by: BuildkiteCI
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
Autosubmit: flokli <flokli@flokli.de>
escapeSystemdExecArgs is the function that should be used to escape
Exec* service lines.
See a72b1b3c65/nixos/lib/utils.nix (L122-L128)
Reported-By: matrix:u/lukas:luflosi.de
Change-Id: Ia3a628db221a30310154c060a6e29ccb2c94c352
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12930
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
Use dotted keys where we'd else have maps with a single key.
Change-Id: I9389e0fedddad1cf65f870a3a68415a0defaa259
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12929
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
'buf-language-server' was removed as its development has moved to the
'buf' package.
Change-Id: If1ef36ed54926644debc914817bf4afea8bf2264
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12928
Reviewed-by: lukegb <lukegb@tvl.fyi>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
* //users/flokli/keyboards/chocofi:
The hash got invalidated somehow which I've updated (to what
https://buildkite.com/tvl/depot/builds/37991#0193f512-78ba-491f-af60-a23e987def95
showed). This seems to have triggered an update of ZMK and some
options have gotten renamed.
Change-Id: I5a62cd4636c23bfdeae671da7b8acb0f02cc2263
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12905
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
* //tools/nixery/popcount:
replace removed buildGoPackage with buildGoModule.
* //users/aspen/system/system/modules:
pkgs.nerdfonts has been removed. Instead we have a
pkgs.nerd-fonts attribute set that contains all fonts
as individual derivations.
* //users/tazjin/presentations:
The ms package was removed from texlive for some reason
in the latest release. Replace it with the packages it
bundles (according to CTAN).
* //tvix/verify-lang-tests:
Test on latest Nix release 2.25.2.
* //tvix/*-go:
regenerate code from protobufs.
Change-Id: I19fcb3a0267f929f6e7388aa69ad99ac53b62236
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12859
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: flokli <flokli@flokli.de>
Reviewed-by: aspen <root@gws.fyi>
Tested-by: BuildkiteCI
We produce traces bigger than what tempo accepts by default, causing
traces to be rejected with TRACE_TOO_LARGE and to then be incomplete.
Bump the max size.
Change-Id: I8caa245d14db683853485ee5625c9662ea51ce29
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12926
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
There's no need to mkForce anything in that list.
Nix reads nix-cache-info to determine priority.
Change-Id: I08797ed25348f52f5696f80558d206b73d20dead
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12925
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
The mount didn't get applied for some reason, explicitly configure the
path.
Change-Id: Ie41eb3c1d5f6416493211fb77709aaeecf61edf0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12924
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
The data source defaults to 15s of time interval. As alloy scrapes every
60s only, this causes watching dashboards with a smaller time range to
just not show any data, like the CPU graph being empty for a time range
< last 12h.
Fix by setting time interval to 60s.
Co-Authored-By: WilliButz <willibutz@posteo.de>
Change-Id: Ife306b2fda968654cad818a82f99e0011819be3c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12923
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Putting this into UnitConfig won't work, so the bind mount didn't
happen, causing the blobs to be created on the SSD too.
This was already deployed and the data migrated over.
Change-Id: Ie30c8f458cdad8b764817a48a048ec3ca3c18e64
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12922
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Nix and Lix both report this quite wrong when showing progress, they
assume 0.00 MiB of download size:
```
these 3 derivations will be built:
/nix/store/m6dkr67hk87lpcz9wv8f2mp2zrgsyfp9-nix-2.24.11.drv
/nix/store/zcfi6vs0z18309asw8fpa9v3665av44v-niv-0.2.22.drv
/nix/store/g4kvzrs1kk9i13vaa8q1r0y4hgsqsnlp-dev-env.drv
these 112 paths will be fetched (0.00 MiB download, 2649.76 MiB unpacked):
/nix/store/3qzlg8h5qc1slypy99aafdcgkzj6974h-OneTuple-0.4.2
/nix/store/pp4540rig52fnj66kz1kiaj1000ja9v0-QuickCheck-2.14.3
/nix/store/416aqss6p59w6v92127hkz04v0bclx21-StateVar-1.2.2
/nix/store/b5f93spm2cl9g6x73dnx7ns5irs739fz-aeson-2.1.2.1
/nix/store/nqnx6k6y103rargdz2gai2rmi389zn6n-aeson-pretty-0.8.10
/nix/store/11wc4s6a6qi98lxikacw746slhmj5dl7-ansi-terminal-1.0.2
/nix/store/yy7j66av9lwh3lvbxp1zv7572wb4l7dj-ansi-terminal-types-0.11.5
/nix/store/cn6m459cbacdwkvjllgy5hkzf045yc1g-appar-0.1.8
[…]
```
For now, set FileSize to NarSize - it's not more wrong than it was
before, Nix already supports content encoding for compression (via
curl).
Reported-On: https://git.lix.systems/lix-project/lix/issues/606
Change-Id: Ia53506ecf6678ad298f759c95a69feb441cbc26d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12919
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
We currently only had some integration tests (as part of tvix-boot)
testing nar-bridge functionality as a smoketest, but with axum-test we
can test individual handlers and peek at the store afterwards, which is
much more granular.
This adds tests for the nar-specific request handlers.
Change-Id: I7f2345df89ac43b9b372ecc66f696e95e2fcad18
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12916
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: flokli <flokli@flokli.de>
Allow reusing CASTORE_NODE_* and NAR_CONTENTS_* from other crates.
Also, there's no need for NAR_CONTENTS_* to be Vecs of bytes, these can
just be [u8; _].
Change-Id: I435c08a9d20f6a68266d0c9a70bfc7fdb618ce42
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12915
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Move the Directory and PathInfo storage to the SSD, and only bind-mount
the blob storage from the HDD.
This should improve IO for random access.
Change-Id: Icf9408a879dec8a52541953682ffac25b31e73d3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12921
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
We reuse the common buffer size setting from mblog for convenience.
Eventually we probably want to make mail-note an independent library, so
it's good to make this internal dependency explicit and not a blanket
USE-PACKAGE, so the amount of used symbols from config doesn't increase.
Change-Id: I88458493c90d9f52410e34ed2a1db99be751b901
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12914
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Because OPEN-DECODED-FILE-PORTION only knows about transfer encodings it
would only return a character stream for 7bit encoded bodies. This
causes inconsistent behavior where some bodies would return binary and
some character streams. To fix this, we specialize MIME-BODY-STREAM for
MIME-TEXT parts which may or may not be a good enough solution.
We may actually want to make MIME-BODY-STREAM binary always and let the
user handle decoding?! This may be a good idea to take care after yet
another stream machinery redesign.
Since the mime4cl test suite doesn't test MIME-BODY-STREAM (much), add a
message generated by notemap that hits this issue to the mblog golden
test suite.
Change-Id: Ie340c42ced6c693af9b3c84b177408d6b6d2c9c4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12913
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
The type identifier Apple uses is com.apple.mail-note, so “Mail Note” is
actually the best way to refer to this format. Not only doesn't it
include a trademark, but it's also more accurate. The iOS and macOS
Notes.app(s) allow authoring Notes to be saved in iCloud which seems to
use a different API and/or storage format (at least these notes are no
longer accessible via IMAP). In this sense they are “Apple Notes”, but
not “Mail Notes”.
Change-Id: I2fd3d3bd253ed39adf7965008290f7d1e622831d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12815
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
This should allow for refactors with more confidence as we can make sure
base functionality stays the same. It is important to test image
extraction, so unfortunately we need to check in a base64 rendering of
an image file. I've used //users/tvlbot.jpg, so git should at least be
able to deduplicate the extracted content. Note that this was achieved
by altering the note message since I wasn't able to add the picture in
the iOS Notes.app without the image being recompressed.
To get extra benefit, we also add the test note to the mime4cl test suite.
The expected output can be updated with
mblog $(mg build :maildir) expected
Change-Id: I0aa493b206439018ad89745bacbd47af78bd1396
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12911
Tested-by: BuildkiteCI
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: sterni <sternenseemann@systemli.org>
`configFile` is chosen to match the terminology used in NixOS modules. I
think this is quite useful since you can do a lot via the config file
since you can not only set the HTML export setting via the org file
header, but also through various emacs variables (see
<https://orgmode.org/manual/HTML-specific-export-settings.html>).
Change-Id: If4d66348e3043f62782106e7fd34f5cf5c7071a4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12651
Reviewed-by: aspen <root@gws.fyi>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
For some reason the driver does not mount the bus sometimes, in that
case just call the tool directly without a specific bus, so we get at
least some sort of brightness adjustment (lol).
Change-Id: Ie8fe8c500fb1025609b569715e681e053e6e06ed
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12909
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
Currently nix/html requires that the content of an element is either an
HTML string (which may or may not be generated by the library) or a flat
list of HTML strings (which may or may not be generated by the library).
I've found that this requirement makes authoring more complex pages that
have programmatically generated parts cumbersome since one needs to take
care that returned lists are appended, not included as an element. This
leads to confusing code and annoying errors. We don't really care about
the nesting of a content list as long as the order is clear, so we can
just flatten the list making life a little easier:
(<main> { } [
(<section> { } (<h2> { } "static section"))
listOfGeneratedSections
(<section> { } (<h2> { } "another section"))
])
Change-Id: I06016a8eff01d34d7eaea7798a00ed191115f9c8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12908
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Autosubmit: sterni <sternenseemann@systemli.org>
In r/4000, I switched orgExportHTML to Emacs 28 (which bundles org by
default) when nixpkgs defaulted to Emacs 27. Since nixpkgs now uses
Emacs 29 by default, this is no longer necessary.
Change-Id: I6d700e8508be77a1b9866557403a5a4ecaa005f7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12569
Tested-by: BuildkiteCI
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: aspen <root@gws.fyi>
This is occasionally necessary. --force-with-lease should also be
supported in the future, unfortunately getopts(1) doesn't have --long
option support.
Change-Id: Ib054009f48585b1a52ed041a51bcaf7e32dca1b3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12904
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
The gopher server has been disabled for a while and I'm probably not
going to revive it any time soon (though I should fix some stuff on
spacecookie soon-ish…).
Change-Id: I6ef6bbfc013f9924e2d2b7ba116285a32406e5a7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12901
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Small git subcommand that enables you to push a subset of (independently
apply-able) commits from a local chain of commits to a remote ref, e.g.
for review. Useful for a workflow where you work on a chain of commits
and want to submit the ones that have been finished for review without
rebasing the chain.
Change-Id: I7717fe37867acdd826bc03a578104a0c3b2cbf71
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12900
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
I've recently set JetBrains Mono to be my default monospace font because
it has better Unicode coverage than Bitstream Vera Mono and should
consequently include all BQN symbols.
However when investigating why 𝕊 was weirdly small, I discovered that it
was using GNU FreeSerif for some reason. As it turns out, Emacs uses the
default font (or the system wide default monospace font if unset) for
ASCII only. Beyond ASCII emacs falls back to the random assortment that
is fontset-default. Using (set-fontset-font <fontset> NIL …) doesn't
work – neither for preventing a fallback to fontset-default from
fontset-startup nor for prepending a font to all ranges of
fontset-default. Especially the former seems to contradict the Emacs
documentation.
The only solution I could come up with, is to set the relevant charsets
explicitly in fontset-startup and to never touch the default font face.
Change-Id: I640b3207e3cc3449ecd422db0e2ed93fb7d3521f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12899
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
We allow strings with context that represent paths (since they qualify
as `pathLike`). While store path (names) may not contain any characters
that are meaningful in shell, they may contain directories and/or files
with such names since it's permissible in POSIX.
To fix this, we convert the given value `v` to a shell argument in two
stages:
1. Use `${v}` to coerce the value to a string while importing any
necessary paths to store.
2. Escape the resulting string for use as an argument.
Change-Id: Ib989b50df2a921c2abcd1ebc7ca0ff6e2bb79088
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12898
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Autosubmit: sterni <sternenseemann@systemli.org>
Retrieves all diagonals of a two dimensional array (which have differing
lengths). Useful to solve e.g. Project Euler problem 11
(<https://projecteuler.net/problem=11>).
Change-Id: I853bc6eaaf869326d84d8e789fee9995f5a332d8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12897
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
This is a bandaid until we have a proper fix.
Change-Id: Id9f0bab5f309a7796c1efee23071013618c6dd12
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12896
Autosubmit: Jonas Chevalier <zimbatm@zimbatm.com>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
This makes it easier to see what's being communicated, by setting
`RUST_LOG=nix_daemon=debug`.
Change-Id: Ifcd89ff6c5c1727e97569e29d4f63993cc37ed8e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12884
Tested-by: BuildkiteCI
Reviewed-by: Vladimir Kryachko <v.kryachko@gmail.com>
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: flokli <flokli@flokli.de>
For trying out a bunch of themes in quick succession.
Also note down a few of the cooler themes.
Change-Id: I0da80cc0945dba03d3592f28f4c34df4b5969e82
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12893
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
This simple dbus service will use the ddcci interfaces to change
brightness for both the internal and external monitor (roughly in
sync).
Currently in the alacritty-change-color-scheme script because I’m lazy
and still experimenting.
Change-Id: Ib2c4323699ed9d19ee398f84680b755df4b25798
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12891
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
These `tsconfig` will apply to JS files with jsdoc comments. Sweet!
Change-Id: I1a623d0ec7e2d73e99d7c6aaf8162d96f4ff2b29
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12889
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
We want to use the tracers quite similar to how we’d use OTEL for
tracing, meaning you should be able to start a span and use it within
another span to register it as a parent span.
They are also batched up and sent asynchrously, so the won’t incur a
lot of overhead on dbus nor block the main execution flow (done via
sending a nodejs event and a dedicated batch sending process).
Change-Id: If61b85305807e661ffee386f793c11c4b7a858a9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12888
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
This makes defining the interface a little less verbose & more
typesafe (checks are done by the dbus library).
Change-Id: I16df987fd152cabf76ed9878ed1a372a0f7003fb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12886
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
For simplicity’s sake this puts everything into the
alacritty-change-color-scheme script for now.
This implements a simple dbus-opentelemetry proxy adapter, which
allows services to record otel traces without having to depend on the
quite complex otel libraries. Instead, they just send their traces to
the dbus tracing interface, and the service that binds against that
interface forwards the spans to the OTLP collector.
First you create a new Tracer for your service via the `TracerFactory`
interface:
```
> busctl --user call \
de.profpatsch.otel.Tracer \
/de/profpatsch/otel/TracerFactory \
de.profpatsch.otel.TracerFactory CreateTracer \
s hello
s "/de/profpatsch/otel/tracers/hello"
```
(this corresponds to setting up a tracer with properties in OTEL)
Then, you can use the returned object path to call the `Tracer`
interface proper:
```
< busctl --user call \
de.profpatsch.otel.Tracer \
/de/profpatsch/otel/tracers/hello \
de.profpatsch.otel.Tracer \
StartSpan \
s '{"spanId": "111", "name": "111"}'
```
This will create the spans. You can also set their timestamps on the
sending side via `startTime`/`endTime`, but make sure it’s a hrtime
tuple.
Prefer batching multiple spans vie the `BatchSpans` call.
Change-Id: Ie6cfdcb0dc3e2398316a2c1763bc72c1118168b0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12885
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
Reviewed-by: benjaminedwardwebb <benjaminedwardwebb@gmail.com>
Adds a simple-stupid dbus interface for this daemon which allows
on-the-fly changing of the alacritty color scheme.
Example call:
```
busctl --user call de.profpatsch.alacritty.ColorScheme \
/de/profpatsch/alacritty/ColorScheme \
de.profpatsch.alacritty.ColorScheme \
SetColorScheme s 'prefer-dark'
```
Change-Id: Ic895fedefb3f5bd95f2279edf53fe179e8f24f89
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12875
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
they switched from .ch to .sh
Change-Id: I889634ec257b7956b9d2b22a9ad6fc0c889f43c2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12853
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
Uses xapian under the hood to index the contents, then makes it
searchable with a CGI binary on http://localhost:8080
We could in theory index every -doc output this way to get local
documentation search for the current system (similar to `man-db`).
Change-Id: I2588c8f100841cfbed570bb65d376b79747c06ee
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12710
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
rush is like GNU parallel, but in Go, and most importantly, without
the annoying citation.
Change-Id: Id4737e6dee43037c1c2bc814738416410a603e07
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12887
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com>
Tested-by: BuildkiteCI
mkBootTest is not limited to only listing files, it can also be used (is
used) to boot init.
Change-Id: Iaa0d2b5bad3be856aa8a0172450efe166620ba41
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12882
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
This makes it a bit less verbose.
Change-Id: I41835f43628d7a10855b9d89816e8d20eb7546d2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12881
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
Reviewed-by: Domen Kožar <domen@cachix.org>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
There's more than one result in there.
Change-Id: I5d519db51fda050ed293bfb52215a643882e0116
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12879
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
This is a little helper which tries to render different input files into
a fancier, human readable plain text form. This is quite useful to
gether with build.gopherHole.
Change-Id: Ibe0050fa6a55e85745127a287bba0febeeb75849
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12874
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Autosubmit: sterni <sternenseemann@systemli.org>
I never fully set up the hyper key and now this configuration actually started
working by default, which is distracting, so begone!
Change-Id: If4c0b5928360087eb1a674ee2eca2510ced3ca55
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12871
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
These functions are not very useful—as far as I'm aware at least—, are
not implemented very efficiently and totally untested. Remove them for
now. See also r/8978.
Change-Id: If9d277b460c3ed728a171bc29dd626c4c5fc0313
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12868
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
This is far from comprehensive, mainly covering stuff I'm interested for
mblog currently. I should extend it as I go. The cases I've added reveal
something I've noticed recently: The worst performing part of mime4cl
seems to be the initial parsing of the message. My current theory is
that this is due to the use of READ-LINE in DO-MULTIPART-PARTS which
seems to ultimately dispatch to READ-CHAR internally due to the way our
streams are set up. We should look into fixing this soon.
It may be interesting to add this to windtunnel at some point, but I'd
rather not burden a runner with this given that mime4cl is only worked
on once every blue moon and I'm the only user.
Change-Id: I001de3aac01f8aa7ea923b43b2db29cf66a4aac3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12864
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Move back to a proper release containing the opentelemetry bump. Also
enable the `axum` feature, which will give us a per-route accounting.
Change-Id: Icdf4dc73588ef45b6596b320c14d9f44946327b3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12865
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
This didn't work properly for a while, at least expose too many cameras
for browsers to get confused.
It also doesn't apply cleanly to the 6.12.1 kernel.
Change-Id: I8a23cc0ae0547bfde5756ee84953b60f4b0a2df2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12866
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: flokli <flokli@flokli.de>
As it turns out, some of the load/compile time set up the package does
doesn't work in ECL for unknown reasons at the moment. Executables using
closure-* will crash after starting up:
;;; Checking for wide character support... WARNING: Lisp implementation doesn't use UTF-16, but accepts surrogate code points.
yes, using code points.
;;; Building Closure with CHARACTER RUNES
Condition of type: SIMPLE-ERROR
Invalid relative pathname #P"package.lisp" for component ("closure-common" "package")
Change-Id: I4b4bf96835a39696884ec6fea9c249fdeb53c853
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12863
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Only significant implementation specific code at the moment is FILE-SIZE
which isn't very important. We can also easily implement it for CCL.
Additionally, we clean up an unused lexical variable warning and remove
a duplicate definiton of MIME-TYPE-STRING fro MIME-UNKNOWN-PART that CCL
doesn't like.
Change-Id: I7c960e50dcdc1d3e46cb4945f36ea315a3c9838d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12862
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
This is a silly mistake and was not caught because FILE-SIZE isn't
exercised in the test suite. We can probably remove MIME-BODY-SIZE and
look into removing MIME-PART-SIZE as well. I just want to be thorough
here so that we can revert into a non-broken state in case we decide we
need those functions for something.
Change-Id: I5bbb3dde6616220fc3b6feddbf7a39b6a9b0ea0d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12861
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
The only code that used this function was removed in r/7854.
Change-Id: Ia07dcb08ed4a92495085b48018372fb9898a0248
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12860
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
MIME-MESSAGE has a HEADERS slot which is an alist of all headers. Some
of those headers will be parsed again and stored in MIME-PART (or a
subclass of it). Having the header content stored in the HEADERS alist
and in MIME-PART causes problems:
- Requires extra knowledge about how messages are parsed when rendering
messages.
- Makes MIME= depend on the specific whitespace and quoting in those
headers which isn't preserved by how mime4cl parses e.g. Content-Type.
- Gives users two ways that slightly diverge to access the same thing.
To avoid this, we remove these headers after the MIME-PARTs contained in
MIME-MESSAGE have been initialized (since they reuse the HEADERS slot).
Change-Id: I5b221f88bbac47dd81db369e3c1d5881a5a50e5e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12858
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: sterni <sternenseemann@systemli.org>
Content-Transfer-Encoding should default to 7bit when it's not
given (RFC2045). MIME-PART already defaults to this when manually
constructing this, but MAKE-MIME-PART would always set it, so it would
sometimes be NIL which is incorrect. We now correctly fall back to :7bit
in this case.
Additionally, we make sure that KEYWORDIFY-ENCODING immediately returns
when it's given NIL.
Change-Id: I50f86dd649d83a4c3a8881d6e13dcada889d5521
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12857
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Autosubmit: sterni <sternenseemann@systemli.org>
Use the crate name in the user-agent, similar to tvix-[ca]store.
Change-Id: I10527fb1f29006dbfd8630d8bb1f00d7905efdd3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12851
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Vladimir Kryachko <v.kryachko@gmail.com>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
GitHub does not allow downloads by clients without a User-Agent set.
This changes sets the client User-Agent to "tvix".
Change-Id: I829b37e63bcedb5ea785b55eff5b10175f5caaa6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12845
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
I've decided to use the commit view instead of the log view (which cgit
uses) for now. It really depends on how you use it in commit messages:
To refer to a depot state or to a specific change (independently of what
CL gerrit assigned). I'm happy to change it to use the log view.
Change-Id: I472b511fa1322f91304f6543473b51f9c5f21ca2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12837
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
This operation is particularly used when invoking the following
nix commands:
```
nix-store --add-fixed some-path
nix-store --add-fixed --recursive some-path
```
Change-Id: I0f9b129c838c00e10415881f1e6e0d7bc1d7a3a6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12800
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Validation is done inside ingest_nar_and_hash and
is used by Fetch::NAR and the nar-bridge.
Change-Id: I7e2be4cc13d2447035f1e5a444f44b62339988bf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12836
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
With nar-bridge supporting zstd content-encoding, we don't need the
nginx zstd module and can re-enable http2.
We also need to propagate the Accept-Encoding sent by the client to
nar-bridge, so it actually knows it can send zstd.
This reduces the time measured in the microbenchmark from ~13s to this:
```
hyperfine 'rm -rf /tmp/cache; nix copy --from https://nixos.tvix.store/ --to "file:///tmp/cache?compression=none" /nix/store/jlkypcf54nrh4n6r0l62ryx93z752hb2-firefox-132.0'
Benchmark 1: rm -rf /tmp/cache; nix copy --from https://nixos.tvix.store/ --to "file:///tmp/cache?compression=none" /nix/store/jlkypcf54nrh4n6r0l62ryx93z752hb2-firefox-132.0
Time (mean ± σ): 4.880 s ± 0.207 s [User: 4.661 s, System: 2.377 s]
Range (min … max): 4.700 s … 5.274 s 10 runs
```
Change-Id: Id092307423636163ae95ef87ec8fa558b83ce0bb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12835
Reviewed-by: Jörg Thalheim <joerg@thalheim.io>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
We previously didn't handle compression in nar-bridge, and left it up to
a fronting reverse proxy.
However, at least nginx with http2 enabled pins each connection to a
single core, causing compression to be limited by the throughput of a
single CPU.
Change-Id: Ia11c2ff5c012192b25eb8ad05dae5542a2d2f777
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12834
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
Reviewed-by: Jörg Thalheim <joerg@thalheim.io>
We don't need to calculate dependencies between and on targets that are
part of the parent target map since they will be skipped by buildkite
anyways. This speeds up 🦙 considerably for pipeline runs that have a
limited number of changed targets and a parent target map passed
in (i.e. pipeline runs of most CLs, but not canon runs). In my testing
it was about a minute faster (1/6 of the time 🦙 takes currently) for a
pipeline where under five drv targets changed. For the full
pipeline (i.e. no parentTargetMap) 🦙 takes about the same time as
before (it's a few seconds slower as is to be expected, but nothing
significant).
Change-Id: Ia5a80e142da8f40bc591e2c6cfaf48c325b2f577
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12818
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
With more implementing NixDeserialize, this error message changed.
Fixtures were regenerated by running:
```
TRYBUILD=overwrite cargo test -p nix-compat-derive-tests --all-features
```
Unfortunately, it's not possible to loop this into CI, as trybuild
invokes cargo during the build.
Change-Id: Ia0ab07d0907d21366845fe06e01df9fb1fe3e7cb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12831
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
Tested-by: BuildkiteCI
We don't need a separate instance of opentelemetry-collector, alloy can
also do this job for us.
Change-Id: I1b671ba57d70b080f7db112e1afcfe2e0cbdd13e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12829
Reviewed-by: flokli <flokli@flokli.de>
Reviewed-by: Jonas Chevalier <zimbatm@zimbatm.com>
Tested-by: BuildkiteCI
These are quite bursty, and I've seen messages about getting rate
limited.
Change-Id: I73058140957cb5718971fa432c003c2d1b0305e3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12828
Tested-by: BuildkiteCI
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
Use grafana-alloy to collect system metrics.
Change-Id: I592e64ca722701d4f12e69a531a434b54954955a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12827
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
We got some double-quoted strings at narinfo_str, and it didn't align
well with our other field names.
Change-Id: I5c08786d2c4435542bf39ff44b9d4ada5400550d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12826
Tested-by: BuildkiteCI
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
We should be able to distinguish tvix-store and nar-bridge.
Change-Id: I616c8e0c1ce2dbacab92975582dd36141b673aa2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12824
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
Tested-by: BuildkiteCI
This provides some global HTTP statistics.
Change-Id: I8bd3e034123154a49d94720b0c8d0c3babde5ae3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12557
Reviewed-by: Jonas Chevalier <zimbatm@zimbatm.com>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
This creates and registers a global meter provider, which uses the same
mechanism to get notified of flushes.
Change-Id: I856a67f0b282d494de3b2c2a1b79c06ae8ffe252
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12556
Reviewed-by: Jonas Chevalier <zimbatm@zimbatm.com>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
This enables routing of metrics to an instance of VictoriaMetrics, and
configures opentelemetry-collector to route metrics there.
Change-Id: If765191a4cc70ddcaad821d45132b96a10a12148
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12812
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: Jonas Chevalier <zimbatm@zimbatm.com>
This is a fetch-through mirror of cache.nixos.org, hosted by NumTide.
The current machine is a SX65 Hetzner dedicated server with 4x22TB SATA disks,
and 2x1TB NVMe disks.
The goals of this machine:
- Exercise tvix-store and nar-bridge code
- Collect usage metrics (see https://nixos.tvix.store/grafana)
- Identify bottlenecks
- Replace cache.nixos.org?
Be however aware that there's zero availability guarantees. Since Tvix doesn't
support garbage collection yet, we either will delete data or order a bigger
box.
Change-Id: Id24baa18cae1629a06caaa059c0c75d4a01659d5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12811
Tested-by: BuildkiteCI
Reviewed-by: Jonas Chevalier <zimbatm@zimbatm.com>
Reviewed-by: flokli <flokli@flokli.de>
This can be used to transitively enable the `xp-composition-cli` feature
flag in the `tvix-store` crate, which is unnecessarily hard to do with
crate2nix.
Change-Id: I3f7c505a3fd505561c9a7b2f063b6540532cdfd4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12809
Tested-by: BuildkiteCI
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
This got lost somehow, but is necessary to keep `mg` in `$PATH`.
Change-Id: I2100d68225284bfe825bcc5ab01628891ebd09a3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12810
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: flokli <flokli@flokli.de>
When sending nars over the wire to the nix-daemon, nix protocol versions
>= 1.23 use this framing protocol.
This change implements an AsyncRead for this protocol, to be used in
AddToStoreNar and any other operations when necessary.
Change-Id: I5f7972fe1c9ea145780bf449321bd3efeb833d18
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12814
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
When sending nars over the wire to the nix-daemon, nix protocol versions
1.21 to 1.23 use this framing protocol.
This change implements an AsyncRead for this protocol, to be used in
AddToStoreNar and any other operations when necessary.
Change-Id: I571f1adbb2343c14c98503d1a2c12eea4c783ec9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12813
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
The TracingHandle::flush function allowed a user to pass in their
own (optional) oneshot::Sender<()> to get notified once the flush is
completed, but that's making things unnecessary complicated.
By simply having the flush() function await the flush, we make its
interface more intuitive, and callsites (only inside tvix-tracing itself
so far) simpler.
We can also remove the Option around the oneshot::Sender entirely, as we
now always call it with that.
For some more clarity, we can remove the channel from the struct fields
entirely if otlp support isn't compiled in.
Change-Id: I0870b9e8e88c6be6494a9c201c1c70b87e0f0810
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12801
Reviewed-by: flokli <flokli@flokli.de>
Reviewed-by: Marijan Petričević <marijan.petricevic94@gmail.com>
Tested-by: BuildkiteCI
This should get rid of some deprecation warnings in 🦙.
Change-Id: I8588169553a312a4df01d9bf71de7d673f28c431
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12808
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Autosubmit: sterni <sternenseemann@systemli.org>
* //3p/overlays/tvl: build nixos-option with latest Nix version (2.24)
as is required now. It would be nice to avoid this somehow to prevent
NixOS machines in depot having to carry around two versions of Nix.
Maybe we can at least use a statically linked nixos-option?
* //3p/{gerrit,gerrit_plugins}: update deps hash
* //tvix/eval: adjust our nixVersion “user agent” so that it'll pass the
new 2.3.17 minimum version nixpkgs prescribes (to check for zstd
support when substituting from the binary cache).
Change-Id: I4eb715afdc3dbb857340839f08ce86612aa7f117
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12805
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
This machine hasn't existed for a while. Seems like I forgot to remove
the expression after its final month ran out.
Change-Id: I0e4abbd9af75eabfab0db106f851a1e43aa8c90f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12807
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: sterni <sternenseemann@systemli.org>
All repositories under code.sterni.lv are mirrors, so there's no value
in AI startups endlessly crawling the nixpkgs git history on
code.sterni.lv…
Change-Id: Iaac296315f325ced3cfd0852ae1d8d3f3815ea5b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12803
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
I want to add a warpper script to the mdmonitor program anyways, so
there's not really a point in this.
Change-Id: I92166bd44b54507b782a8d19b9676d91d8fa0f99
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12802
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: sterni <sternenseemann@systemli.org>
1.21.1 fixes an exploit in 1.21 without any other changes, so we can
safely skip it.
Change-Id: I72503c9f3869d7bafdfc78842b61804627a1d452
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12798
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
This switches to a ZMK branch with support for mouse movement, and sets
MIRYOKU_KLUDGE_MOUSEKEYSPR so miryoku makes use of the functionality.
Change-Id: I3d4f48f10d50c202f909bec15189106a1bbcc1b3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12796
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
`requires` is not recommended in this context. I think I tried it out of
desperation trying to combat the issue that these units would be started
before they were able to resolve names in switch-to-configuration.
Unfortunately, network access during switch-to-configuration can't be
detected using network-online.target, it seems.
Change-Id: Ia98a0a3b505ffa56eb37fb58a5375a1215d6cb1b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12794
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Mostly it's important (which I did this time at least) to remember that
with how things work at the moment, the initrd sshd is only reachable
via IPv4.
Change-Id: Ie9a87b6a38b2e128a8a2141d2221bbe7cfe24cdb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12792
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
I've discovered that it is possible for irccat to fail enough times to
run into the restart limit before network is online after booting.
Change-Id: Ia54a46d56bdc765a825fee50e7bdc8206718edc0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12790
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: sterni <sternenseemann@systemli.org>
* update wasm-bindgen in all wasm projects
* //users/wpcarro/website: declare missing dependency on
string-conversions. Presumably this was propagated
before from some other dependency which got updated now.
Change-Id: Ib93de576408974441d532196601e6e53d22cdafe
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12770
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: tazjin <tazjin@tvl.su>
I don't have this keyboard anymore, no need to build the firmware
anymore.
Change-Id: I1ab25dfaa71d03dc7006312ecab86453d6f90333
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12788
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
`buildSplitKeyboard`, as well as all the patching of `miryoku_zmk` is
independent of the specific keyboard used, so it can be moved one layer
up.
`config-flat` is now provided through a helper function, accepting the
name of the keymap to use when rendering the config.
This all makes the amount of code added for the new keyboard itself
pleasantly small.
Change-Id: I2216aa246502eddaf9bc4f4d126b0639d574ad87
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12787
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
These are required to support certain nix's local-overlay store
operations, it's safer to return empty results for these
operations than failing with "operation not implemented" errors.
Change-Id: Ic9b69d75dd52af5a826bfb6a8b283b082a0f6bcf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12766
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
This change contains 2 improvements to the tvix-store copy command:
1. Allows reading the reference graph from stdin, using `-` argument
2. Supports json representation produced by `nix path-info --json`
command.
In general it makes is easier and faster to import arbitrary closures
from an existing nix store with e.g the following command:
```
nix path-info ./result --json --closure-size --recursive | \
jq -s '{closure: add}' | \
tvix-store copy -
```
Change-Id: Id6eea2993da233ecfbdc186f1a8c37735b686264
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12765
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
By manually writing out the async function, and moving the owned String
we can avoid for S to be Sync in the NixSerialize for StorePath<S> impl.
Co-Authored-By: edef@edef.eu
Change-Id: I8427b38d2bd61de2818088728cbad79cda69d17d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12764
Reviewed-by: Vladimir Kryachko <v.kryachko@gmail.com>
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: edef <edef@edef.eu>
Tested-by: BuildkiteCI
This describes the current composition system used for BlobService /
DirectoryService / PathInfoService, why it's hidden, how to expose it,
and adds some common examples to explain it.
Change-Id: I2ce7da40992cc988947c3e924499f8157c5e4937
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12749
Tested-by: BuildkiteCI
Reviewed-by: yuka <yuka@yuka.dev>
Implement PartialEq/Eq ourselves instead of deriving, by proxying to
name.as_ref() (and digest of course).
Also implement Hash on our own, clippy doesn't like this to be derived,
while Eq/PartialEq is not.
Change-Id: Idbe289a23ba3bc8dabf893d4d8752792ae2778c3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12744
Tested-by: BuildkiteCI
Reviewed-by: edef <edef@edef.eu>
Autosubmit: flokli <flokli@flokli.de>
NixString doesn't contain a `HashSet<BuildReference>` anymore, there's a
more packed internal data structure. We don't need to be overly specific
in the docs, just say we expose an API.
Change-Id: I13380c49293f9a86d1916909fdfeefbe64d9024b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12755
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: edef <edef@edef.eu>
This conversion was a bit too magic, and we can just use
`NixString::new_context_from` without having to worry about the
distinction between an empty context or no context, as
NixString::new_context_from already deals with that internally.
Change-Id: I3e5d57ecfa0f7456aa6c526863e49f2523afaec3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12754
Tested-by: BuildkiteCI
Reviewed-by: edef <edef@edef.eu>
Autosubmit: flokli <flokli@flokli.de>
This change includes only the basic nix handshake protocol handling and
sets up a client session. The only supported operation at this point is
SetOptions.
Additional operations will be implemented in subsequent cls.
Change-Id: I3eccd9e0ceb270c3865929543c702f1491768852
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12743
Autosubmit: Vladimir Kryachko <v.kryachko@gmail.com>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Reviewed-by: edef <edef@edef.eu>
Reviewed-by: Brian Olsen <me@griff.name>
This prevents the field from getting printed to stderr, apparently.
Change-Id: Ia9860e4ff37224003154db88ee5f83103060e626
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12756
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
Allow this to be overridden via the CLI, also getting rid of the unsafe
because this doesn't need to be const anymore.
Change-Id: I5e51b52e42522a21f59ef69628b464477c0764d1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12753
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
When uploading a Store Path to a Nix HTTP Binary Cache, Nix first does a
HEAD request for $outhash.narinfo, and if that's not found, for
`{narhash}.nar[.compression_suffix]`.
If the NAR is already present, only the NARInfo is uploaded.
Even though we don't have a service allowing to globally look up from
NARHash to root node, `root_nodes` in `AppState` at least contains
recently uploaded NARHashes.
If we find it in there, we can prevent Nix unnecessarily uploading NARs
if the same contents have already been recently uploaded.
We also promote this key, chances are high Nix will subsequently upload
a NARInfo referring to this NARHash.
Change-Id: I34e3fd9b334b695abe945e64cd291e30f303c2a2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12752
Tested-by: BuildkiteCI
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
Autosubmit: flokli <flokli@flokli.de>
This being in combinators makes it harder to find.
Change-Id: If7984bdbd43f164c670548639bb4846d859f6695
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12750
Reviewed-by: yuka <yuka@yuka.dev>
Tested-by: BuildkiteCI
Extend the docstrings of `add_default_services`, and add one for
`addrs_to_configs` as well as the module-wide one at
`tvix_store::composition`.
Change-Id: Ie9b449988eb210cd65b19b174094bbe0c4af2fd6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12748
Tested-by: BuildkiteCI
Reviewed-by: yuka <yuka@yuka.dev>
tvix-castore already supports composition without any additional feature
flags, the only thing that can be explicitly enabled is referring to
other stores via an anonymous url. Rename that feature flag to
"xp-composition-url-refs".
tvix-store effectively only controls the CLI surface, so rename this to
"xp-composition-cli".
The "store" in the feature name was dropped, as it's already apparent
from being in the tvix-[ca]store crate.
Change-Id: I1175dc6280cbba4cbcdfb7fd4b35fce713b45fc4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12747
Tested-by: BuildkiteCI
Reviewed-by: yuka <yuka@yuka.dev>
This becomes the root of the composition. `default` implies we can
directly access anything else, which we cannot. `root` makes this more
understandable, and it's all internal only anyways.
Change-Id: I297511bc05a7c32c59510b9d192b40d1bd937b5f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12746
Reviewed-by: yuka <yuka@yuka.dev>
Tested-by: BuildkiteCI
This groups most `wire` feature gated logic into a single module.
The nix_daemon module will be gated by a feature that adds
nix-compat-derive as a dependency.
All of this is a way to break the crate2nix dependency cycle between
nix-compat and nix-compat-derive(which depends on nix-compat for its
doctests).
Change-Id: I95938a6f280c11967371ff21f8b5a19e6d3d3805
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12761
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Currently it is not possible to distinguish between tracing of the same
*Service type whenever there are multiple of them. Now the instance_name
of ServiceBuilder is passed into the *Service and used in the existing
instrument as the `instance_name` field.
Places that did not already have a instance_name in its context use
`"default"`. In tests I used `"test"`.
Change-Id: Ia20bf2a7bb849a781e370d087ba7ddb3be79f654
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12739
Tested-by: BuildkiteCI
Autosubmit: Bob van der Linden <bobvanderlinden@gmail.com>
Reviewed-by: flokli <flokli@flokli.de>
This will be used by tvix-daemon to write tracing data into the active
client's connection socket.
Change-Id: I8889dd0a638e004ee2c8cb312946b029c9779313
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12734
Tested-by: BuildkiteCI
Autosubmit: Vladimir Kryachko <v.kryachko@gmail.com>
Reviewed-by: flokli <flokli@flokli.de>
This makes it quite hard to press Alt+Number otherwise.
Change-Id: Id1cc4fbfa1575cc213014106f734bec035d46f2c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12737
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
This change implements the serialization part that is needed to
implement the nix daemon protocol. Previously was add deserialization
and derivers for that and this then adds the other part of that equation
so that you can write types that can then be read using deserialization.
Change-Id: I2917de634980a93822a4f5a8ad38897b9ce16d89
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12729
Autosubmit: Brian Olsen <me@griff.name>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
This adds a span to default handling so that any error message
gives a more precise location as a hint for why the error occurred
instead of just pointing to the type.
Change-Id: I7bf6cf38e0284f9726d670ea50a94a6b1edd8a94
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12728
Reviewed-by: Vladimir Kryachko <v.kryachko@gmail.com>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Autosubmit: Brian Olsen <me@griff.name>
Currently Value::Json is used in combination with VMRequest::ToJson to
recursively convert tvix Value to serde_json::Value. This functionality
is used in builtins.toJSON as well as derivation __structuredAttrs.
Both Value::Json and VMRequest::ToJson were removed in this commit.
Related functionality in vm.rs is also removed: vm.rs does not know
about JSON anymore.
Recursively converting to serde_json now happens without going through
the VM.
Thrown errors that are part of the value of toJSON are now directly
propagated as ErrorKind, were-as previously there was a split between
CatchableErrorKind and ErrorKind, where eventually CatchableErrorKind
would be converted to ErrorKind::Catchable.
Change-Id: I066f064926c491e4c087a984f07af43d19124cfe
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12732
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Some references to nixrs had sneaked in while writing the original
CL so this just removes those.
Change-Id: I635adbd2de46c7c1f31e1d449396253a78e1b762
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12727
Reviewed-by: Vladimir Kryachko <v.kryachko@gmail.com>
Autosubmit: Brian Olsen <me@griff.name>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
When running tests for buildservice on MacOS it would fail
because of the oci test. I also got some clippy warnings on
MacOS because of disabled code.
Change-Id: I235739fa4270a4ef46e54d3e2b8cbb55eb20bdda
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12726
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: Brian Olsen <me@griff.name>
Tested-by: BuildkiteCI
Before this commit only deriving of tuple enums where tested.
This adds a few tests to cover the other types of enums.
Change-Id: I62f311e4db98cab84bd4ac164d50051e9aab0d4d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12725
Autosubmit: Brian Olsen <me@griff.name>
Reviewed-by: Vladimir Kryachko <v.kryachko@gmail.com>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
I found some more operations that are obsolete and a few missing
fields. I also did some small changes to make handshake description
more consistent.
Change-Id: I9f853a37e9d50621491f31ea71879d6a600046e2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12724
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Previously such errors showed up as:
error[E006]: expected value of type 'sha256', but found a 'not a sha256'
Now they show up as:
error[E041]: Invalid hash: invalid encoded digest length '31' for algo
sha256
This is consistent with the errors of `builtins.fetchurl`.
Change-Id: Id11b26fc7951778640cc4e41b3bf23203eaf07df
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12719
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
The nixhash errors were wrapped in a generic TvixError. Now it has its
own TvixError with unique error code. The nixhash error is passed along
as a string.
The errors looked like:
error[E997]: invalid encoded digest length '51' for algo sha256
Now they look like:
error[E041]: Invalid hash: invalid encoded digest length '51' for algo
sha256
Change-Id: I5c420815538ba4c6567c95f5d44d60c4d48f43fd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12718
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Existing handshake behavior assumed that the server version is always
at least as new as the client. Meaning that the client's version was
always picked the handshake details as well as for further communication
This change removes that assumption and correctly uses
min(server_version, client_version).
Change-Id: Ia5dad4613dd5f69a0aeb6c9d86982f1f36fe1a4c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12722
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
The goal is to create a drop-in replacement nix-daemon that nix-cpp can
use as a `daemon` store.
Change-Id: Ie092047dcc6a24a3b8d8d1b808f3e6fd2c493bf2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12711
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
This does the same as cl/12585 did for tvix-cli.
Change-Id: Ia2d693f5ad10feb81777c951bcd4489a3a83ad7c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12716
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
This does the same as cl/12585 did for tvix-cli.
Change-Id: I35215947f3616acd65654c2bbdb19d0c360b568b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12714
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
This version has my trackball fixes, and interactive move, both of which are not
in a stable release yet.
Change-Id: Id7356e328772c972db7fb496ac80c318e8c5b330
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12707
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
The copy&paste from the documentation didn't work ...
Change-Id: Ic894356354d6ac2b66562da5aa89590cd94ae347
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12705
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
oci_spec::runtime::Posix* stuff is feature-flagged out on MacOS.
While it might be desirable to emit Linux OCI runtime specs from MacOS to
schedule there, and it being feature-flagged to Linux might be a bug in the
upstream crate, we only have an implementation invoking runc, which certainly
don't work.
Feature-flag this out to make tvix-build and tvix-cli to build on MacOS. This
can be revisited once we actually start thinking about builders on MacOS.
Change-Id: I82002f6d55eddd9c305f1b3a3f3f2b252f617ed0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12694
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
fuse_backend_rs::api::filesystem::Layer is not exposed for non-Linux, and
feature-flagged on virtiofs, so only implement the trait for these cases.
Change-Id: Id8455dc5be502f8375836ba04288d50c59d69d89
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12693
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
runc is not available on MacOS, we only want to include it in the shell if on
Linux.
Point TVIX_BUILD_SANDBOX_SHELL to /bin/sh if not on Linux.
While we cannot use the OCI/runc executor on MacOS, other implementations might
make use of this env var, so it's good to keep setting it.
Also update the other occurence of it, in tvix/utils.nix.
Change-Id: If7b47e1bb7c41bbde84c93016713754a252c4355
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12691
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
Tested-by: BuildkiteCI
This builds the keyboard firmware using zmk-nix.
The miryoku_zmk config is pulled in from manna-harbour/miryoku_zmk, but
slightly patched to add support for RALT and quick-tap.
Change-Id: I91efbbd789526cce7f086f367c7bccb7857b06e6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12695
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: flokli <flokli@flokli.de>
A while ago I decided to experiment with corfu and orderless instead of
company/prescient. This was prompted by issues which I had in EXWM, where some
sort of UI race with company elements could lock up a whole frame.
Now that I'm not on EXWM anymore this doesn't really matter, so I can bring back
company and set up prescient again correctly, fixing a whole bunch of issues
along the way:
* Completion issues where not all matching candidates where shown based on
cursor position (I honestly have no idea what caused this and what was going
on there, never figured it out, but enabling prescient properly fixes it).
* Code completion works again. Corfu, as I understand it, is technically
better/cleaner/whatever than company - but it doesn't work anywhere,
especially not in telega.el where I need completion more than anywhere else.
With this commit my Emacs behaves a lot better again.
Change-Id: I8f082de8211dd3be3bb7a0663d43d414cc320e49
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12690
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
This is only using nix-compat through tvix-glue.
Change-Id: Ib6391987954a28cc05a2c27c1c21585185d82d1a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12681
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
Tested-by: BuildkiteCI
This is only using tvix-castore through tvix-store.
Change-Id: I342ee1e18f8c3946b712cfb446520b732091d58f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12680
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
... because it's on the left side!
Change-Id: I1d67d797bd6137705e69b96349faa32a9c3e94ff
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12678
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
This test will otherwise obviously fail on aarch64-linux, or anything
not x86_64-linux, as it'll produce a different Derivation.
Change-Id: Iedddcb6aeff05e49fc2e6da27f08111cdc9affec
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12677
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: edef <edef@edef.eu>
Tested-by: BuildkiteCI
Deduping early saves a fair bit of memory, but the extra hashing is
costly.
We switch to FxHash, since we don't need a DoS-proof hash, but we do
need it to be *fast*.
Change-Id: Ic6b7010874c417862baa9b882593208c8dd1d5e6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12648
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Parsing of store-paths.xz is now handled by //users/edef/fetchroots.
Change-Id: I78be5aada0c0a321ed79d80c9b615e5f997ac3e0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12670
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
This vastly reduces the memory requirements, so we can run in ~40G RAM.
Change-Id: I4952a780df294bd852a8b4682ba2fd59b9bae675
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12667
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
We switch to working with the underlying Arc<Bytes<u8>> type, since
Buffer<u8> is already essentially an OwningRef-esque subslice.
Because we're now working with an exposed Arc directly, we don't need
to have our own `unsafe impl StableAddress` any more.
Change-Id: I9ce2edc6899177145e15b72aa5380f708a62173c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12668
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
We only care about ordering on the same variable, and we rely on the
release barrier provided by rayon's thread joining.
The comparison failing is always an error path, and we're indifferent
about which thread it fails on.
Change-Id: I592a7eaae5b6935cf424c6576a49786f39909fb6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12666
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
It seems we need more memory these days, and llama frequently ran OOM.
Decrease the number of concurrent evaluations.
Change-Id: I2648ebdedf09b80c9a231c4614004f953a646bc0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12662
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: edef <edef@edef.eu>
When evaluating nixpkgs from the <nixpkgs> channel, store_path_to_node
returns an error, falling back to regular filesystem access.
This currently produces a warning in the CLI, which is misleading, so
degrade to trace level.
Change-Id: I4cb2297cc85a2c0e904a37343748f9051aa6d5c7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12665
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Construct the owned signature in a separate scope, so all borrows to the
original PathInfo are already dropped again, and we can modify the
PathInfo without having to clone it.
Change-Id: I03e7390540c2cfe7a2c61850bdbe8a33d213a5d9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12663
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
This pushes generating spans with pb_show up to the caller.
They usually have more context on how to present things, if at all.
Change-Id: Icfcaa64a8a57dce50c0261f2d06e7c051e3946c2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12657
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
We have two places where we parse protos and want their names to be
empty:
- Receiving a root node in a nar-bridge NAR request
- Processing the CalculateNAR gRPC call
We don't have any place where we want to keep a name as bytes::Bytes
around, yet we used the `into_name_bytes_and_node` method.
It was also a bit wrongly named - it wasn't very clear the name was
not validated, and that the function may fail.
This moves the "splitting off the name as bytes::Bytes" part into a
private helper, only leaving the `try_into_name_and_node` and
`try_into_anonymous_node` methods around.
Change-Id: I2c7fd9871d49ec67450d7efa6a30d96197fb319c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12664
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: Marijan Petričević <marijan.petricevic94@gmail.com>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
This implements BS, DS, PS for Box'ed or Arc'ed variants of it with less
code, and less potential to accidentially forget to proxy default trait
methods for blanked impls, as fixed in cl/12658.
Change-Id: If2cdbb563a73792038ebe7bff45d6f880214855b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12661
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: edef <edef@edef.eu>
If the PathInfoService is using gRPC, expose it via the
nar_calculation_service() method in the PathInfoService.
Also pass nar_calculation_service through in tha blanket impl.
This now causes a `tvix-store import` using the default config to not
fall back to `SimpleRenderer`, which will calculate the NAR hash and
size by downloading the uploaded blobs (and blobs it didn't need to
upload) locally, making such imports faster.
Change-Id: If2c3fe6584e9093cba322d2360f355a3923904ae
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12658
Tested-by: BuildkiteCI
Reviewed-by: edef <edef@edef.eu>
Autosubmit: flokli <flokli@flokli.de>
Clients can (and do) send an empty name in here, and we discard it
anyways.
Change-Id: Iec135d9cba48cc066464e5639319c845079787e9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12659
Reviewed-by: edef <edef@edef.eu>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
With the seekable NAR renderer, figuring out the next few blobs to
render became possible.
Change-Id: I1214302f88e6f9aba74227f84df0f964d587baf2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12652
Tested-by: BuildkiteCI
Reviewed-by: edef <edef@edef.eu>
Autosubmit: flokli <flokli@flokli.de>
This is now supported in the standard library via std::sync::LazyLock,
but requires some manual shuffling around of code.
Change-Id: Ifca792f4d2dbc36b703de4a4dfa406015ab86da7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12614
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: flokli <flokli@flokli.de>
Reviewed-by: edef <edef@edef.eu>
Tested-by: BuildkiteCI
This is now supported in the standard library via std::sync::LazyLock,
but requires some manual shuffling around of code.
Change-Id: Ia0370ca46cb1c6122a452b1d117160536b632c7e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12612
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
This is the only (remaining) occurence of it, and not really
more code than just calling store_path::build_ca_path with
`CAHash::Nar(NixHash::Sha256(…))`, especially considering we need the
CAHash in the PathInfo struct later anyways - so let's remove this
function.
Change-Id: Ia82212086062c366e0280ca0823d9e68a3f91d3a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12632
Reviewed-by: edef <edef@edef.eu>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
This became obsolete, since the introduction of a stricter `Directory`
struct invalid names cannot be represented anymore.
Change-Id: I9e4b1b6cca01831d0a9735f58d8a1f59ac18676b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12615
Reviewed-by: flokli <flokli@flokli.de>
Reviewed-by: edef <edef@edef.eu>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
This is not gonna work out as-is, as we still key PathInfos by
their store path digest, and how to handle thing if we encounter a
Frankenbuild.
For now, let's keep the PathInfoService data as it is, we can record
this information (and more) in the builder structures.
Change-Id: Ic38fc3ecd8096a5fe002e681bdc812a9dbeaa7d2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12607
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: edef <edef@edef.eu>
For mystifying reasons, Type=simple and CREDENTIALS_DIRECTORY in
ExecStop have stopped working (when exactly I don't know, but presumably
256). Apparently, you are supposed to use Type=exec with credentials due
to raciness (I've personally never experienced):
<https://github.com/systemd/systemd/issues/32583>.
Just changing the type did not resolve the issue of
CREDENTIALS_DIRECTORY being unset, though. It appears, though, that the
issue is merely an unset environment variable and not the credentials
being unavailable: We can work around the problem by setting an
appropriate environment variable ourselves.
Change-Id: Ifcdb1f3bce782ea1c568a9bc413f3fb29f0985c5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12649
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: sterni <sternenseemann@systemli.org>
This moves the implementation from builtins.path into a helper function,
which we now call from both builtins.
Most of the Value plumbing stays inside this helper.
We also implemented handling of symlinks at the root, which was handled
in builtins.filterSource, but not builtins.path - by peeking at the
FileType using std::fs::metadata, instead of the EvalIO trait.
For now, this is fine, as our filtered_ingest also goes via the
filesystem directly. It ends up with the same semantics as before and in
Nix - symlinks at the root are followed, except if they point to an
invalid target.
In the future, we should revisit this, and then maybe get both stat and
lstat into EvalIO, though we will need to be very careful about the
semantics for following symlink inside store paths.
Change-Id: I6a941c0187db36165c2f7a338015e4e32d41b298
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12629
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
In a previous refactoring CL this into_bstring method was accidentally
kept, when we don't need it and can just to_str directly.
Change-Id: Idd531d508b8fd530611b213d0164e7aaf0e87d80
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12631
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com>
These nested ifs are a bit confusing, a match block makes this cleaner.
Change-Id: I256fd0bc921fbf2e60ad0f6e1ea51c2e0fb00317
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12628
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
This removes all the intermediate helper functions and reorganizes the
import code to only do the calculations where/when needed, and hopefully
makes things easier to understand as well.
Change-Id: I7e4c89c742bf8569b45e303523f7f801da7127ea
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12627
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: Jörg Thalheim <joerg@thalheim.io>
Reviewed-by: edef <edef@edef.eu>
This makes it easier to understand what the specific test is testing.
Change-Id: I34b2798841c6b9367849668451af2165dc78f997
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12626
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
Tested-by: BuildkiteCI
Reviewed-by: Jörg Thalheim <joerg@thalheim.io>
This didn't support store paths with a subpath joined to them, while
Nix does.
Use state.path_exists, which does. This also means we can drop the
`store_path_exists` helper, which was only used here.
Change-Id: I918ccb270f64acbdc41cb4d2a9c3c5871ce15002
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12618
Tested-by: BuildkiteCI
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
Reviewed-by: Jörg Thalheim <joerg@thalheim.io>
Autosubmit: flokli <flokli@flokli.de>
These are not necessarily strings, and making it paths allows us to stop
converting them to lossy strings.
Change-Id: I11366c721dc5da1778aafe89092a1966b5a43178
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12617
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
Reviewed-by: Jörg Thalheim <joerg@thalheim.io>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Make this generic on the StorePath<SP> that's being used, similar to the
other functions in there.
Change-Id: I453d1fd3749053d4e5aca156abc18da1f95ca264
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12616
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Jörg Thalheim <joerg@thalheim.io>
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
This is now supported in the standard library via std::sync::LazyLock,
but requires some manual shuffling around of code.
Change-Id: I14bee4068dc73c948321481b5a4e1fc922a89a27
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12611
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
This is now supported in the standard library via std::sync::LazyLock,
but requires some manual shuffling around of code.
Change-Id: Ie2af74beda9fcf8aa19fca7d844bcbe732f05bf8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12610
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
This is now supported in the standard library via std::sync::LazyLock, but
requires some manual shuffling around of code.
Change-Id: Ibb3be8458b8a8912ea04c9360d64c5cf914254d4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12609
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
This is now supported in the standard library via std::sync::LazyLock, but
requires some manual shuffling around of code.
I found at least one dead variable along the way, which I deleted.
Change-Id: I8600c87c49078fb5ff72671994c77b919259e67b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12608
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
This is not a core Tvix tool, it's a tool that uses a Tvix component.
Change-Id: I81d2b2374da23489df0097dcabb8295c82652fc1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12606
Reviewed-by: edef <edef@edef.eu>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
This is not a core Tvix tool, it's a tool that uses a Tvix component.
Change-Id: I705f2c4ab87f1512e005007c933e16b84ed4279f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12605
Reviewed-by: edef <edef@edef.eu>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
This was introduced in cl/9925 without any commit message, but this is clearly
not relevant to Tvix itself (it even says so in a comment in Cargo.toml).
Change-Id: I84f12d5145c3f53c9df23863f887bad913856c50
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12604
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: edef <edef@edef.eu>
This is not a core Tvix tool, it's some sort of one-off analysis thing.
Change-Id: I05fcbed45abad27d6b5cfd49db1727249dad3971
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12603
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: edef <edef@edef.eu>
Equivalent logic is now in the standard library, and this dependency is no
longer needed for eval.
Change-Id: Iaa4410d89fdaa5b84cbd9e6bc6ae479c659d92f2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12602
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: edef <edef@edef.eu>
Tested-by: BuildkiteCI
Refactor the `strict` boolean passed into evaluation at the top-level to
be a (two-variant, so far) EvalMode enum of Lazy and Strict.
This is more explicit than a boolean, and if we ever add more EvalModes
it's a simple extension of the enum.
Change-Id: I3de50e74ec971011664f6cd0999d08b792118410
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12186
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: aspen <root@gws.fyi>
Move things around a bit to make it easier to understand what's going on:
- We first validate our fixture invariants
- We then insert into the PathInfoService
- Do all comparisons and checks we can on the returned PathInfo struct
- Only convert to the NarInfo variant to calculate the fingerprint,
and don't keep intermediate let bindings for this
Before cl/12588, this was arguably much harder to do that way, as we
relied on some of the conversions done in the to_narinfo() function.
Change-Id: Iaddbf1079f73ce566ef6d56f69a823e080b2e006
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12595
Reviewed-by: Marijan Petričević <marijan.petricevic94@gmail.com>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Reviewed-by: sinavir <tvix@sinavir.fr>
The store path is already contained in the PathInfo, and the ca bits is
already passed into the function, so known to the caller - there's no
need to duplicate this.
We can also avoid having two separate block_on in our import builtin -
we already know the content hash before constructing, as we pass it in
via ca_hash.
There's still some room to unclutter some more of the code around
importing - we still do NAR calculation twice in some cases, and some of
the code might be share-able from other places producing PathInfo too.
Log a TODO for this cleanup.
Change-Id: I6a5fc427d15bc9293a396310143c7694dd2996c0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12592
Reviewed-by: Marijan Petričević <marijan.petricevic94@gmail.com>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
We also use S in other places in the same file, but that's for the
string-like references.
SP is now consistently used as the type parameter for StorePath<_> (and
build_output_path) gets support for it).
By being a bit more careful in the order of assignments in nix-compat/
src/derivation, we can nudge the compiler to use the type we want.
Change-Id: Ia7c298e110dff98d3b113d2388674ce9e22b80e8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12590
Reviewed-by: flokli <flokli@flokli.de>
Reviewed-by: Marijan Petričević <marijan.petricevic94@gmail.com>
Tested-by: BuildkiteCI
This switches the PathInfoService trait from using the proto-derived
PathInfo struct to a more restrictive struct, and updates all
implementations to use it.
It removes a lot of the previous conversion and checks, as invalid
states became nonrepresentable, and validations are expressed on the
type level.
PathInfoService implementations consuming protobuf need to convert and
do the verification internally, and can only return the strongly typed
variant.
The nix_compat::narinfo::NarInfo conversions for the proto PathInfo
are removed, we only keep a version showing a NarInfo representation for
the strong struct.
Converting back to a PathInfo requires the root node now, but is
otherwise trivial, so left to the users.
Co-Authored-By: Florian Klink <flokli@flokli.de>
Change-Id: I6fdfdb44063efebb44a8f0097b6b81a828717e03
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12588
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Our oci-spec was a bit oudated and there were some renamings in one of
the release, which made building tvix-build fail if it's a dependency.
I encountered this issue while working on tvix-eval-jobs.
Change-Id: I6d982965176b83170a07445e351d3f5e5679ed2e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12586
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com>
Tested-by: BuildkiteCI
This allows specifying an url in place of a named reference to another
composition entry, if the castore crate has been compiled with the
xp-store-composition feature.
Example: `--directory-service-addr cache://?near=memory://&far=memory://`
This would be equivalent to the instantiation via toml file:
```toml
[memory1]
type = "memory"
[memory2]
type = "memory"
[default]
type = "cache"
near = "memory1"
far = "memory2"
```
Note that each anonymous url causes a distinct instance to be created.
Change-Id: Iee5a07a94b063b5e767c704d9cad0114fa843164
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12146
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
This still defaults to the "default" services, but allows users to tell the
nix+http pathinfoservice to ingest the castore nodes into a non-default
blob-/directoryservice when used with the experimental store composition.
Change-Id: I5c0f683ce95d888eadf3f302520a47f42f1a481d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12148
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
I should probably remove the default.nix files in these as well so
they don’t get built on CI.
Change-Id: I09764f2ee198ab4016a1649f1675f7c45d207b09
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12580
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
Want to be able to make changes with low overhead, and having it in a
separate library is just annoying.
Change-Id: I30b76885d8e0e6ebaefe9506cf36672783ed4988
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12577
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
the linters & prettier config should apply to my whole subdir.
This is somewhat nasty, you have to `npm` in the toplevel dir before
it starts working, otoh dev tooling is dev time and I’m working on
these alone.
Change-Id: I96721f549b24a40b7ffbb2d310f37a40d2590b2b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12573
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
Add a tapping command that does not quantize the timestamps.
For the silence warning, we make it BPM-dependent (defaulting to
120BPM as everywhere else), meaning for slower songs we give a higher
possible time difference before we display a warning.
Change-Id: Idefc44166639b23c2105a1a810ac22ed84457225
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12563
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
That was the original intention, but I didn’t understand that lua
would fail with "r+" if the file does not exist (and "w+" truncates
the file, so you have to try "r+" and then fall back to "w+" which
will create the file as well.)
Change-Id: Ib238f0b73ab403ceeaf035d053a14eba718d1b48
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12562
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
This adds support for uploading the lyrics part of an .lrc file to
lrclib, see https://lrclib.net/docs
I pretty much only used ChatGPT to translate the rust “proof of work”
challenge to nodejs and it worked first try lol.
Before uploading the lyrics, I construct a webview with a preview of
what is going to be uploaded, and then only upload when that is
accepted. Pretty sweet.
Also adds two commands for increasing/decreasing the current timestamp
by 100ms and starting playback from 2 seconds before that, very handy
for fine-tuning lines.
Change-Id: Ia6adfe26d0c21c62554c8f8c55e97e2caec95d1e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12561
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
If the bpm header already exists, overwrite it with the new value.
Also use an existing header as suggestion.
Change-Id: If6431e8056504db437c31313d885b5ba0d0e55d5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12553
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
Insert the length and stuff into the .lrc file headers.
Change-Id: Id2565c95c516208f1e46b79d5b8da50f3d6bee62
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12552
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
It’s a bit crappy and really depends on the input field opening
quickly again (which it often doesn’t really do…), but it was the
easiest way I figured how to do it haha.
Aligning to eigth notes is pretty much the easiest way to sync
everything up after tapping in the timestamps (for most songs).
Change-Id: Ibbb072f62b6ee17d983e81b6c1554bc3516fa636
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12551
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
* tap-bpm: simple CLI program that accepts key inputs and averages a
BPM value
* lyric-timing-mpv-script: If you press Ctrl+l, mpv attaches the
current timestamp to a .lrc file named after the song.
This is for manually timing missing songs for uploading them to
https://lrclib.net/
* extension: vscode extension for `.lrc` files, currently with the
following features:
1. A “jump to LRC position” command which reads an .lrc timestamp
from the current line and expects mpv to listen on
`~/tmp/mpv-socket` (via `--input-ipc-server`), and will seek to
the exact timestamp (down to the ms) in the currently playing
song.
2. Some initial linting warnings
- A lint that warns if the difference to the next timestamp is
more than 10s (which usually means there’s an instrumental and
the previous line is stuck)
- A lint that checks that timestamps are monotonically
increasing
Change-Id: I32a4ac0e2c5bbe3d94e45ffcf647f81bc7c08aa0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12537
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
Leaving out any symbol characters improves the search accuracy.
Change-Id: I00c993d4099bb8e9701783b53afc9423f1b2f674
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12480
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
We wanted to enable this earlier but the log level of many of the
messages in redb were too high, they've now been downgraded, so we can
enable logs from redb.
Context on the fix and release:
- https://github.com/cberner/redb/pull/828
- https://github.com/cberner/redb/releases/tag/v2.1.2
Change-Id: I8635e8a0bcb01a7d0b580387ac9134ccdd0205f3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12568
Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
It prevent some shortcuts from being used the "natural way", like
only being able to type Ctrl-Alt-T with the left hand.
Luckily, RALT isn't needed that frequently, so we can expose it right
next to GACS (on the G and M keys).
Change-Id: Iccba3b1f6a5e2b01195f87471fd9972967b4e175
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12572
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Over the past couple of months we've been using redb instead of sled as
the default filesystem-based database in PS and DS. I am confident that
we can get rid of sled completely now, and just keep redb.
Change-Id: I11fa1e4453e280253855f8eade990b37eb6965ae
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12567
Reviewed-by: yuka <yuka@yuka.dev>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com>
Previously the script ignored workspaces that were further up, but in practice I
don't care about their order, I just want them to be gone.
To keep IDs stable, this implements a fix where the current workspace is first
moved to the first position (invisible), and windows are then reaped afterwards.
I've tried this in various combinations and it seems to work fine.
Change-Id: Ifc3eb272af761670ec83305665ec2103eb4f269e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12564
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Previously, the overlap calculation would underflow when
the pattern is empty.
Change-Id: I1f6bf49fafc4b8183a3a5e5e491a5a5bfc41ca97
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12558
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Autosubmit: yuka <yuka@yuka.dev>
With https://cl.tvl.fyi/12533 in, we still need to lookup references to
properly populate `BuildRequest`.
It currently fails as the reference to
h9lc1dpi14z7is86ffhl3ld569138595-audit-tmpdir.sh is not propagated.
We should prevent Frankenbuilds from the go, so let's update our
PathInfo type to accomodate for that.
Change-Id: I26f9215312c258bba222efd390bc135f1a3a3d6d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12560
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
This is just patchset 10 of CL10855, before the color_eyre changes,
rebased to the tvix_castore api.
Change-Id: If4b42412ff8568058908cda971ad7d6f2d9f9b7b
---
This provides a build service invoking runc. It can be used by using the
`oci://$path_to_some_tempdir` builder URL for now.
For now, it can be tested as such:
```
BUILD_SERVICE_ADDR=oci://$PWD/bundles target/debug/tvix
let pkgs = (import <nixpkgs> {}); in builtins.readDir pkgs.perl
```
readDir is to actually trigger IO into the store path (which triggers
the builds).
For now it fails due to missing reference scanning (see followup CLs).
Change-Id: I09b40e410114ce69966a41a0e3c33281b859e443
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12526
Autosubmit: yuka <yuka@yuka.dev>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
After this, attempting to build the nixpkgs still fails in the same way,
because the references are not yet properly used by the code at
`tvix/glue/src/tvix_store_io.rs`.
Change-Id: I8a59ef8ef3c9a6f6aa7b05106dd9eef2e9ac0d0f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12532
Reviewed-by: Brian Olsen <me@griff.name>
Autosubmit: yuka <yuka@yuka.dev>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
This provides a generic interface to let the builder search for needles
in the output, as described in the notes at `docs/src/build/index.md`.
Change-Id: Ic2c5bd563e9aa2e766c157f2b13cdb19aede12f8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12531
Autosubmit: yuka <yuka@yuka.dev>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Reviewed-by: Brian Olsen <me@griff.name>
This changes the only actual state the ReferenceScanner has to use atomic bools, so it no longer requires a mutable borrow for .scan(). This allows passing an immutable borrow of a reference scanner to multiple threads which might be ingesting blobs in parallel, and using
them in the ReferenceReader or calling .scan() there.
Change-Id: Id5c30bcebb06bf15eae8c4451d70eb806cab722e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12528
Autosubmit: yuka <yuka@yuka.dev>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
We don't need to access castore for HEAD requests.
Change-Id: I9365d9520d5a9e52ed92897d3c4972ec5b6e11fb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12547
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
With an implementation of AsyncRead + AsyncSeek, axum-range can answer
range requests.
We only use it if a range has been requested, as it uses more memory
than the linear variant.
Change-Id: I0072b0a09b328f3e932f14567a2caa3a49abcbf7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12509
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Reviewed-by: yuka <yuka@yuka.dev>
turns out emacs over ssh forwarding works well if you use ssh -Y
Change-Id: I9c1134d880265b765b0dc52d587ee331fda29f7f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12546
Autosubmit: aspen <root@gws.fyi>
Reviewed-by: aspen <root@gws.fyi>
Tested-by: BuildkiteCI
Instead of installing and configuring fuzzel and wl-clipboard locally, put them
into Nix, too.
Change-Id: I4b909ac1d0577f8bd176bbaae71583c59aaf5362
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12544
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
I think that's the last module I was missing for this to be "feature-complete".
I think ideally the things displayed by the taskbar would have the same order as
the current Niri layout, but I can tackle this later. This might require a patch
in Niri.
Change-Id: I67cc2366b6c510830021954372e29ce0be7d8363
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12543
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
If the user puts in a string that doesn't match any of the selectables, do one
of the following:
1. When called inside of Emacs, create a new buffer with that name.
2. When called outside of Emacs, do nothing.
Change-Id: I92ba985b5cd7805d37d5d0e0631b20fdce7ce479
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12542
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Adds a command which can invoke something like fuzzel (or any other dmenu
compatible system) for the buffer switching logic.
This can be invoked as `emacsclient -e (niri-go-anywhere-external)` from any
shell, given that the Emacs server is running in the given Emacs.
Change-Id: Iafa421e5ad0c3c6cbfecbfc0835e6db878e452e0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12540
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Adds a completing-read annotation function which can display the "remoteness" of
Emacs buffers, as well as the app ID of Niri windows.
Change-Id: I15550f7403b91aa4be7076290931da74ac539fac
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12536
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Introduces a new buffer switching function which is also capable of switching to
existing Emacs frames that already display the target buffer, or to other
windows displayed in the same Niri session.
Not all behaviour is done yet, and there's an explanatory comment in the package
with more details.
Change-Id: I5a548931a681ba32fdb352ecec66845a75268c19
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12535
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
This release includes support for overlay filesystems
Change-Id: I946cbf346df045209afaea2d720bb57fc2f2659f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12538
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: yuka <yuka@yuka.dev>
Tested-by: BuildkiteCI
This is required to add the optional refscanner parameter to the
ingest functions.
Change-Id: Ib40a7287cf857eb55e31e0df309a79474fefb518
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12527
Autosubmit: yuka <yuka@yuka.dev>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Brings this a LOT closer to the Windows95 style theming that I'm trying to
achieve with the whole thing.
Change-Id: Ifb457a682cb83a78e4a8e2ba2bb4a5b96f3eaf1f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12525
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: tazjin <tazjin@tvl.su>
Harmonia is, ostensibly, faster and better and, most importantly, not a giant
pile of wonky Perl.
I've tested locally that Harmonia works with Nix 2.3 (on both ends), so I think
we should be good to go here.
We have a vendored copy of the upstream module for now. We need to fix Nix 2.3
compatibility in upstream for the module, but the service itself works fine.
Change-Id: I3897bb02b83bd466b6fe7077c05728ac49ea4406
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12517
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
This is now stable enough that I can put it under home-manager management.
Change-Id: Ifd92ca8951e0adfddf962a4cd845c3e4fe80cc21
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12516
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
The previous config was copy&pasted from somewhere, this config makes it look
like an actual thing instead. Much nicer!
Change-Id: I8eb68149964376d6dbe4cef53e86030b5357d17d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12514
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
This otherwise spams the logs quite a bit, for example when uploading
to nar-bridge with the new check from cl/12497.
Change-Id: Idc2bcc513caea6fae38ae04489e4e3ee7be64bce
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12510
Reviewed-by: yuka <yuka@yuka.dev>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
- Add a new PathInfoService implementation that wraps transparently
around another except that it dynamically signs all the incoming
path-infos with the provided signer.
- Add a ServiceBuilder for this PathInfoService that provides a
SigningPathInfoService with a keyfile signer
Change-Id: I845ddfdf01d14c503c796b2b80c720dab98be091
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12032
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: sinavir <tvix@sinavir.fr>
Tested-by: BuildkiteCI
thiserror is much more easier to maintain than manually implementing Error and Display.
Change-Id: Ibf13e2d8a96fba69c8acb362b7515274a593dfd6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12452
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com>
Tested-by: BuildkiteCI
Nar-bridge tried to parse the name of the protobuf node encoded in the
URL into a PathComponent but this name was empty, leading to an error
when the user tried to retrieve the nar file.
This was an oversight from the conversion to stricter types (some of the
CLs in the serious containing cl/12217).
We need a version converting a protobuf without a name to our stricter
types, but an empty PathComponent cannot be constructed.
So we need a into_name_and_node() version that returns the name as
Bytes, not PathComponent.
Change-Id: I2996cdd2e0107133e502748947298f512f1cc521
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12504
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
This is useful for building other NAR writers which use custom (async or optimized) I/O to write the blob parts of the NAR.
Change-Id: I447c09914fb0c99044e2fa910d4213660dc51c64
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12437
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
This should make it possible to use Quassel over Yggdrasil.
Change-Id: Iad81af7c85771928fd50860393d58a14d7878a40
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12502
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
ddclient is back in nixpkgs and nixos[0], so let's just use that, and
remove the backported package from third_party.
[0] 8a8ec36615
Change-Id: Ib14ab68158a6799c78d71e3bea63869ec9fc1a48
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12500
Tested-by: BuildkiteCI
Reviewed-by: aspen <root@gws.fyi>
Autosubmit: aspen <root@gws.fyi>
ConcurrentBlobUploader buffers small blobs in memory, and then uploads
them to the BlobService in the background.
In these cases, we know the hash of the whole blob, so we could check if
it exists first before, uploading it.
We were however not, and this caused rate limiting issues in GCS, as it
has an update limit of one write per second on the same key, which we
ran into especially frequently with the empty blob.
This reduces the amount of writes of the same blob considerably.
In the future, we might be able to drop this, as our chunked blob
uploading protocol gets smarter and covers these cases.
Change-Id: Icf482df815812f80a0b65cec0426f8e686308abb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12497
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
* users/aspen: disable readyset overlay
sysbench + postgresql is broken, which breaks the overlay, but I suspect the
overlay is no longer needed
Change-Id: I1845370c88f5fab35fd700535e6fb0972a4ca556
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12494
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: flokli <flokli@flokli.de>
Reviewed-by: aspen <root@gws.fyi>
This is a stable release that has the IPC features this crate depends on.
Change-Id: I4aaba076a53d7a95a00dd2d59a02c69af6c4d63a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12495
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
This is some sort of calling home analytics thing. Lets not have that.
Change-Id: I59fcf747c8269052dd3d346bb0873adc38834803
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12490
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
This requires overriding build-time configuration, so I've added a little fixed
point that takes care of that.
Change-Id: Ie990e362c6e00aa6e3be66b04af4b62034b03515
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12489
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
For some reason this is not included. The build is based on their experimental
Nix code upstream which I copy&pasted and modified a bit.
Change-Id: I523f9e90a3f5feca0deb317eec1d5283e2a3ca98
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12487
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
First test of running an Yggdrasil service, lets see how that goes!
Change-Id: Iac10b72f7314a45df13ea539c5c1cef6c994154f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12484
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
This was in //ops for legacy reasons, but this is really not necessary.
Change-Id: I758b257838993ef0f7d55809c137118826e2ba85
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12483
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
I'm looking to replace tailscale with this for some use-cases.
Change-Id: I00f765a403879ef048e635a6fedcfdde3f685159
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12482
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Issue a wildcard certificate using the Yandex Cloud DNS plugin (which is where
DNS for tazj.in is hosted).
Change-Id: I44fa48add660f4f4324ec4b056a81d78c45ff4f4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12481
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
if there are not synced lyrics.
Change-Id: I3e8d452fc63847505886ea0bb01d29dd5dcb56d1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12477
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
This never went anywhere.
We had problems with `git2` breaking, so let’s remove everything that
this pulled in.
Change-Id: Ia29d827cd6fc7b97aedca36a37f8418384579c38
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12474
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
Instead of serving a stale table when a torrent gets deleted, fetch
the whole view again. This is a little wasteful, but torrents
shouldn’t get deleted very often, so it’s fine.
Change-Id: If33d517270421881852158f27dbc3e7d24880d3b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12333
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
If transmission does not know about a torrent file anymore, we should
not only delete our local transmission torrent hash, but also the
torrent file.
Before, it would always display the old weighted torrent, even after
removing it from transmission. Now, it will automatically clear and
switch to the new top-weighted mp3 version.
Small bug: since we only clean the torrents after already fetching the
table data, we get a stale result right after clean-up. Maybe we
should re-do the fetch instead.
Also logs what gets deleted and only ever deletes if there is
something to delete, leading to more obvious traces.
Change-Id: I0f6c1dc3669d0f549efaba2c46c34b77c1eb7e33
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12332
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
I changed the seeding weight, so now it would not show already
downloaded torrents, prompting me to fetch stuff that I already have
in new qualities. Obviously, that’s not the best idea.
Kinda surprised it’s this easy to fix heh.
Change-Id: I6bbf325672a91d794d144c006ccf3d702a581bce
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12330
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
AppException would be a console-pretty-printed version for http
errors, which would print all the escape codes in the jaeger traces of
the exception, making it more-or-less unreadable.
So instead, let’s make AppException two cases, an ErrorTree case which
is printed as-is (no color), and a “Pretty” case which is printed
using the pretty module (colors on console, no colors in otel).
Somewhat involved, I guess this is temporary until I figure out what
is really needed.
Change-Id: Iff4a8651c5f5368a5b798541efc19cc7ab9de34b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12232
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
* Prefer MP3 versions to save on ratio
* Definitely never download 24bit flacs, they are not worth it at all
One needs to remove the `seeding_weight` column and restart
for it to have any effect (no real migrations yet, but eh)
bump
Change-Id: I30cc3ea05c4352316e3929f93d90fc27ce833682
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12222
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
I don't use workspaces and don't have them bound to anything in my Niri
configuration. However, when an external screen is unplugged, its workspace
(and windows) move to one of the remaining outputs.
This adds a tool that makes the windows available again by "reaping" them from
the other workspaces and moving them to the current one.
For starters I'll bind this to a key and see how it works in practice.
Change-Id: I18b2d60e93c8397dd637cdc426b4e46af5725558
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12451
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Makes it possible to just keep switching focus or moving a column in the
direction of the target screen. This is MUCH easier to think about.
Change-Id: I0488b8d338261b83327dc476e0173b222de9221a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12467
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
We always read the length before returning a BytesReader, so len()
cannot be called before the length is read.
Change-Id: Ifa1cbc2c0923dd24d59e0af5e135ab8ed7314aa9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12453
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
This will later be used to allow the user to access the original writer.
Change-Id: I1734600c5d580d18b16727f892cd2f6335cc9459
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12442
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Finally no more setting this up manually!
Change-Id: I75d7a53623cc05a0a88f70e20e881a944b6ed818
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12446
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
The bundled tool is sort of convenient, I first tried it when debugging
an issue (that doesn't affect this module) and it seems a bit nicer.
Change-Id: Ief948f8e46e51f05b04fe7628ab9298284f780e9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12444
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Autosubmit: sterni <sternenseemann@systemli.org>
This configuration is not automatically placed anywhere yet, because I'm also
still actively iterating on it.
Change-Id: I0a13fcbc2b9a5587480f6a69a73ab99b36d209e6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12435
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Adds the current (fairly simple) Niri configuration, which is annealed from
a few days of being poked constantly.
This isn't yet deployed through home-manager or anything as I'm still doing
stuff with it manually.
Change-Id: I3dfcafb55d98d9655585d3128e1088fc13e60cc7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12434
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
These don't actually really work the way I want, but I have no idea how font
configuration works.
Change-Id: I228ab234f37d8ae0847cbdc6a3431fc29a2585e6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12433
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
This is a CLI for uploading changes to Gerrit.
Change-Id: I45fe592934174ab86af267c54b1f6a190617be62
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12392
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: Yury Shvedov <yury.shvedov@kaspersky.com>
Tested-by: BuildkiteCI
Apparently something called a "desktop portal" needs to provide a magic
"service" and blabla; the TL;DR is that adding all this Gnome stuff makes screen
sharing work.
Tested using the Mozilla WebRTC test page:
https://mozilla.github.io/webrtc-landing/gum_test.html
Seems like our Chromium/Firefox are new enough not to require magic settings for
Wayland.
Change-Id: I250d857a0e941ef63fffca17a9c9feed14ec1bd9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12432
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
According to discussions in the Niri issue tracker, pipewire must be started
before niri in order for screen sharing to work. Why? No idea.
Change-Id: Id4ff890dd2536b3cc98b14992cd799d257eec9be
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12431
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
logind doesn't like reliably suspending, and apparently it has a bunch of bugs
around dock detection (I don't even use docks), but maybe this helps
Change-Id: Ia822799cde838d8d41a67f9d39d59d1fa0306116
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12429
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
This should be fixed with the switch to a "GitHub App" for
authentication.
Change-Id: I0bd8b4b9c9f53a754a12ce07f193d19f61f56014
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12414
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
wezterm has very strange rendering bugs which I have no interest in debugging
Change-Id: I59a068df9b4474830ba4706bf8e626b45cd015f0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12424
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Bootstraps a plugin (that doesn't do anything yet) for accessing Yandex Tracker
through Eagle Mode.
This commit only initialises the plugin files, it doesn't actually do anything
other than print the word "Loaded".
API docs for future development: https://yandex.cloud/ru/docs/tracker/about-api
The next steps will be to figure out the emModel classes and the threading model
for fetching the remote data.
Change-Id: Ifce8bc2a61c4fd0c4a591013acbf428a9f5803f8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12398
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Adds a buildPlugin function which can build Eagle Mode plugins that can
ultimately be linked into Eagle Mode using the existing etcDir function.
Change-Id: I338171779d3547faecbfb708fbaa78cd1cfd59ac
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12387
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: emery <emery@dmz.rs>
Reviewed-by: azahi <azat@bahawi.net>
Required to bump past the broken time crate.
Change-Id: I10c979f2b3abc10ad66ac438834f4caceb661279
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12408
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
Required to bump past the broken time crate.
Change-Id: Ied9e3367f5fc69db0671732a75f2e410f4f234f6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12407
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Without this, terraform wants to recreate the resource, just because we
do /not/ want to delete the default mappers:
```
# keycloak_ldap_user_federation.tvl_ldap must be replaced
-/+ resource "keycloak_ldap_user_federation" "tvl_ldap" {
+ delete_default_mappers = false # forces replacement
~ id = "4e68e9f0-7aba-4465-8357-f2af6a55fd0e" -> (known after apply)
name = "tvl-ldap"
~ use_truststore_spi = "ALWAYS" -> "ONLY_FOR_LDAPS"
# (27 unchanged attributes hidden)
}
```
Keycloak lists the a few mappers. which are likely the default ones,
but in any case, we don't want to recreate this resource.
Change-Id: I170a91a44b2efa426fae268cf7fc97a7f28a5760
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12412
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
The docs mention this applies to "users of the legacy distribution of keycloak".
However, we get a "failed to perform initial login to Keycloak: error
sending POST request to https://auth.tvl.fyi/realms/master/protocol/openid-connect/token: 404 Not Found"
if we don't set this.
With this, the provider is able to talk to the API, as long as the
secrets are sourced.
Change-Id: I0b9cdd45b1628aa0870a1673491c12c07bf7f8d6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12411
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
The same fix from cl/11021 also needs to be applied to other states.
Change-Id: I205b03aab49130639c79702f4bf16f0bf28d89ab
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12410
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
The same fix from cl/11021 also needs to be applied to other states.
Change-Id: I0df3ee2e8970e0d08a119ecc6347f24aef0448c2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12409
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
This is required because of the `time` breakage due to a bugfix in rustc.
Change-Id: I232dabec8bf5498a9e868e811f5822ba39b44d97
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12406
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
This commit is experimental for now.
The backstory is this: There are use-cases where tiling windows (as in EXWM,
which I normally use) annoy me. There are also use-cases where stacking
windows (as in e.g. XFCE, which I tried for a bit) are annoying.
Scrolling WMs are a new thing which is sort of similar to tiling, but as it adds
more screen real estate doesn't require shuffling around or resizing of existing
windows basically ever.
I'm going to trial this (assuming I can get everything that's relevant for me
set up) and see how it goes.
Change-Id: I603c87b8ebd7fbde763d9d4708e098e9079889c5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12402
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Got into weird states more than once while the private configuration wasn't
available ...
Change-Id: I75b1a242d9cf10f5e4ac27ad7fce3e40a79898cc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12401
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
I'm doing a wayland experiment, this is necessary for now. This disables some
parts of the configuration which are not going to work with wayland.
Change-Id: I61d0042fd52f2c7cade2794c0d5b5849ecaf5229
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12399
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Adds a new function that converts anything to a TGA image (imagemagick really is
magic), with which arbitrary icons can be converted.
This is demoed with the emacsclient command, which just takes the icon out of
the Emacs package tree.
Change-Id: I771bafed7b07a37c847bd07db986228b9fda60a0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12374
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: azahi <azat@bahawi.net>
Reviewed-by: tazjin <tazjin@tvl.su>
In https://cl.tvl.fyi/c/depot/+/12389 I accidentally deleted some
comments. Also fixed some useless no-default-features with futures crate, making it match the previous feature combination.
Change-Id: I72bb2cfb88719ff2f8812d90193de2bd49149cce
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12395
Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Adds a new eaglemode.withConfig function that creates a specially wrapped Eagle
Mode, in which a configuration script joins the user's configuration with the
config passed in.
This produces a fully working and configured Eagle Mode with custom stuff out of
the box.
Change-Id: I6282cafd0b1ac6e77bede90cc91d4ede19ee1d2f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12369
Reviewed-by: azahi <azat@bahawi.net>
Tested-by: BuildkiteCI
Reviewed-by: emery <emery@dmz.rs>
Adds an eaglemode.etcDir function which creates a directory structure suitable
for use with EM_USER_CONFIG_DIR.
The catch is that Eagle Mode requires this to be always writable, so it isn't
possible to just point the environment variable at the Nix store and launch it
from there.
The idea of this function is to make it possible to reuse it in a wrapper
script, a home manager module, a NixOS module or whatever that would make it
possible to provide the result to Eagle Mode in a mutable location.
Change-Id: I95c8b16c6c6fe8510ce9759c9d9b9e36e836e290
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12368
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: azahi <azat@bahawi.net>
Adds a command that opens the current target in the file browser in Emacsclient.
Change-Id: Idbbec3905eb7763ce5bdc2931415db74bc5a5c31
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12367
Tested-by: BuildkiteCI
Reviewed-by: emery <emery@dmz.rs>
Reviewed-by: azahi <azat@bahawi.net>
This generates the correct `.pl` files for adding Eagle Mode commands. This
commit does not yet contain the wiring for adding these into Eagle Mode
directly, which is a bit involved.
Change-Id: I7d88128ba3ddaebfbb618db45e8fa843a3f17dea
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12366
Tested-by: BuildkiteCI
Reviewed-by: emery <emery@dmz.rs>
This is tvix/docs, so asking to be a member of #tvl here is confusing.
Being a member of #tvix-dev is sufficient, we can still mention #tvl
though.
Change-Id: If9fbf7d098c0a8d77a45a95e321fc0a553f4c63b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12396
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: yuka <yuka@yuka.dev>
Tested-by: BuildkiteCI
From now on we will add the dependencies and their version in the root
Cargo.toml and in order to enable the dependency for a workspace member
we set `workspace = true` in the member's Cargo.toml.
Change-Id: I9738c1cf99810b7ace87ca712c3ea965ba846e25
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12389
Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
This updates all the dependencies and their "minimum" versions in
Cargo.{lock,toml} to the latest compatible version using `cargo-edit`'s
`cargo upgrade` command that will eventually be merged into `cargo
update`.
Change-Id: Iccb2aa4a1c84a0465222244a0bd0cafe2a82e781
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12388
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com>
Tested-by: BuildkiteCI
This bumps bigtable_rs to
https://github.com/liufuyang/bigtable_rs/pull/86, allowing us to drop
our second set of prost/tonic/http/axum crates.
Change-Id: I70f9150289c3e8611ebe8a7d99490e3dfd085a6e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12384
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Autosubmit: flokli <flokli@flokli.de>
This interface is the only way to construct lazy builtins outside of eval, which
is actually a useful feature to have.
Change-Id: I386323af20aa3134bb4f669fa66fbb21e9b05fd4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12386
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: Yury Shvedov <yury.shvedov@kaspersky.com>
The external feature flag was there because I couldn't find a way to
refer to crate and nix-compat with the same name so that the generated
code could be the same.
In essence `use nix_compat::nix_daemon:🇩🇪:NixDeserialize` is an error
when used inside nix_compat crate.
So my best fix was the external feature flag until I found the solution
used here which also removes the flag completely.
Change-Id: Ia3e89c6c350c3fb22ca87f974a39c21542aae152
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12376
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: Brian Olsen <me@griff.name>
Exclusive range patterns were stabilized in Rust version 1.80 but Tvix
still uses Rust 1.79 and so would fail this one test when you ran
doctests from `mg shell //tvix:shell`.
It was not caught by CI because that does not currently run doctests.
The fix is just to use an inclusive pattern instead.
Fixes: b/417
Change-Id: Ifea7a3b84bb8f6f8c76e277979833713bdf51f46
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12375
Autosubmit: Brian Olsen <me@griff.name>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
tonic-reflection 0.12.x moved from the v1alpha to v1 of the reflection
protocol.
However, most clients, like Postman, Kreya and evans don't support that
one yet.
Bump tonic-reflection to 0.12.2, which re-introduces v1alpha support
alongside the v1 version of it, registering both services.
This fixes the example documented in tvix/store/README.md, it was
previously failing as evans couldn't find the v1alpha reflection
service.
See https://github.com/hyperium/tonic/pull/1888 for details.
Change-Id: I55438877317f82dc39face13afeb9594cda07a4e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12353
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
These started throwing eval warnings due to usage of cargoSha256, and
are both available in nixpkgs.
Change-Id: Ice636bebc0a839410125bc19b2252158add2b96e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12337
Tested-by: BuildkiteCI
Reviewed-by: aspen <root@gws.fyi>
Autosubmit: flokli <flokli@flokli.de>
When only the wire feature was enabled two methods were unused and so
would faild `cargo check --no-default-features --features wire`.
This feature flags those two methods on async feature since that is the
only place they are used.
Change-Id: I6ec18a670e3c6e3ecee8d1417c99f1a5084e0ae7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12346
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: Brian Olsen <me@griff.name>
Tested-by: BuildkiteCI
This adds a nix-compat-derive derive crate that implements a deriver
for NixDeserialize implementations. This is to reduce the amount of
code needed to implement deserialization for all the types used by
the Nix daemon protocol.
Change-Id: I484724b550e8a1d5e9adad9555d9dc1374ae95c2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12022
Autosubmit: Brian Olsen <me@griff.name>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Add a trait for deserializing a type from a daemon worker connection.
This adds the NixDeserialize trait which is kind of like the serde
Deserialize trait in that individual types are meant to implement it
and it can potentially be derived in the future.
The NixDeserialize trait takes something that implements NixRead as
input so that you can among other things mock the reader.
Change-Id: Ibb59e3562dfc822652f7d18039f00a1c0d422997
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11990
Autosubmit: Brian Olsen <me@griff.name>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Small strings are always copied fully, without allocations.
Large transient strings copy the data and allocate.
Large persistent strings are also a trivial copy.
Change-Id: I319c0b800fa7a4a62e634176b959bb2fa766a4eb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12342
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
This was deleted when removing the Sourcegraph module, but it turns out it is
also needed by panettone.
Change-Id: I8f14165bf783743247894c2b64882fbb032ffbf8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12295
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
This adds a function which can be used across the monorepo to create a
an extra CI step that checks whether the Cargo.nix file is up-to-date.
Change-Id: Idb8298b29ddc2ca5dff1facb1b9ed86a236ee66d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12227
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
This adds almost a gigabyte of closure size for formatting files that almost
never change.
I'll look into building just the formatter somehow, but it's not a very high
priority task.
Change-Id: Ib0f841e1a98133381c5ae154e2a57df8af52dc1f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12293
Tested-by: BuildkiteCI
Reviewed-by: emery <emery@dmz.rs>
Autosubmit: tazjin <tazjin@tvl.su>
This prevents browsers from treating NARInfo and nix-cache-info paths as
a separate "Download", but just show it in plaintext.
Change-Id: If99abe20ef1d24e4fa86c055160861ca47aa81ce
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12267
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
This adds the MIME types Nix uses when uploading files to a S3 bucket /
HTTP endpoint. It ignores them when retrieving.
Change-Id: I75fa96d5e53349f0470bbe8cbba93f961fbd5f05
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12266
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
We're moving away from sourcegraph to livegrep and so on, as Sourcegraph has
gone fully proprietary.
This removes support for redirecting to Sourcegraph.
Relates to b/290
Change-Id: I04ccf8dfef72113cd49d444151cb0c3eb834845d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12268
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
Makes it possible to construct persistent German Strings from data with 'static
lifetime. This doesn't leak any additional data.
Change-Id: Ifb3a36ff72b106724883302e4c2195ac54acd352
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12263
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Adds a StorageClassPtr type that is set up to be able to steal bits from an
aligned pointer to encode the storage class of a German String.
Change-Id: I64591174eac1ebcb73e624a59bd107ba1e02c69d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12262
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Fix a discrepancy in the error message for DirectoryError::SizeOverflow.
The message indicates that the SizeOverflow error occurs when total size
exceeds u32::MAX, but that's not true. All size fields within the
castore's internal Directory ADT are u64, and the SizeOverflow error is
only returned after a call to the checked_add implementation on u64.
See tvix/castore/nodes/directory.rs +111
and tvix/castore/nodes/directory.rs +88
as of this commit.
Change-Id: I74d161ea8927362e1cb601ba163489aa96fb91b1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12259
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
This moves the URL component parsing code we had in nar-bridge to
nix-compat.
We change the function signature to return an Option, not a
Result<_, StatusCode>.
This allows returning more appropriate error codes, as we can
ok_or(…) at the callsite, which we now do: on an upload to an
invalid path, we now return "unauthorized", while on a GET/HEAD, we
return "not found".
This also adds support to parse compression suffixes. While not
supported in nar-bridge, other users of nix-compat might very well want
to parse these paths.
Also fix the error message when parsing NAR urls, it mentioned 32, not
52, which is a copypasta error from the narinfo URL parsing code.
Change-Id: Id1be9a8044814b54ce68b125c52dfe933c9c4f74
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12260
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Similar to how cl/12253 already did this for `Signature`, we apply the
same logic to `StorePath`.
`StorePathRef<'a>'` is now a `StorePath<&'a str>`, and there's less
redundant code for the two different implementation.
`.as_ref()` returns a `StorePathRef<'_>`, `.to_owned()` gives a
`StorePath<String>` (for now).
I briefly thought about only publicly exporting `StorePath<String>`
as `StorePath`, but the diff is not too large and this will make it
easier to gradually introduce more flexibility in which store paths to
accept.
Also, remove some silliness in `StorePath::from_absolute_path_full`,
which now doesn't allocate anymore.
Change-Id: Ife8843857a1a0a3a99177ca997649fd45b8198e6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12258
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Don't call functions bstr or str when they return BString or String,
it's confusing.
Rename them to `string` and `bytes`. We might be able to generalize over
this being BString or Vec<u8> later.
Change-Id: I8198551ed3ba1cfc479bf7e3cbbc13a426faf4c0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12257
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
This allows importing just //tools/depotfmt.nix when using a josh
workspace.
Change-Id: Idf7eb438be1cc1e60445aace982fdca1f4367595
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12254
Tested-by: BuildkiteCI
Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com>
Reviewed-by: flokli <flokli@flokli.de>
This implements a simple DFS locator in listing structures.
It is interoperable with the Rust standard library paths, we also build our
own errors to restrict path values to reasonable secure defaults, e.g.
relative paths with no `..` component.
Tests are added for this new feature for a positive and a negative
check.
In addition, a path validation test was added. The Windows-style prefix
is gated on the Windows platform as UNIX does not parse `C:\\` as a
`Component::Prefix(_)` but as a `Component::Normal(_)`.
Change-Id: Iae2a80bebd8138e41af94aa7d09f2842c3c5a786
Signed-off-by: Ryan Lahfa <tvl@lahfa.xyz>
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12255
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Requiring `name` to be a `&str` means it'll get annoying to pass around
`Signature`, but being able to pass them around in an owned fashion is
kinda a requirement for a stronger typed `PathInfo` struct, where we
want to have full ownership.
Rework the `Signature` struct to become generic over the type of the
`name` field. This means, it becomes possible to have owned versions
of it.
We don't want to impose `String` or `SmolStr` for example, but want to
leave it up to the nix-compat user to decide.
Provide a type alias for the existing `&str` variant (`SignatureRef`),
and use it where we previously used the non-generic `Signature` one.
Add some tests to ensure it's possible to *use* `Signature` with both
`String` and `SmolStr` (but only pull in `smol_str` as dev dependency
for the tests).
Also, add some more docstrings, these were a bit sparse.
Change-Id: I3f75691498c6bda9cd072d2d9dac83c4f6c57287
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12253
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
.ls files are useful to seek in a NAR without parsing it entirely.
The responsibility of validating the files is on the caller.
Change-Id: I5d1da28b5479c38f20ca5babe60e362a2217c9ea
Signed-off-by: Ryan Lahfa <tvl@lahfa.xyz>
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12196
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
We can avoid depending on things outside //tvix by just using a similar
util from nixpkgs.
Change-Id: I9ea3e1f0a8a059ea10caaec173569ba9f316aec6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12247
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Using pkgs.nixos directly allows us to create a smaller nixos closure
for the tests and also not depend on things in depot.ops which can be
beneficial for extending the tvix josh workspace.
Change-Id: Ic6ad2122733418114b43aa692d6e42ac1e308eeb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12251
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com>
It's a `[u8; SIGNATURE_LENGTH]` type alias, and conveys what we're
accepting or returning a bit nicer.
Change-Id: I974cd97d56d383e51417eb0f26e1431a05711922
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12252
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Makes it possible to construct a GermanString from an owned byte vector, without
having to clone the data.
This is done by "disowning" the vector using ManuallyDrop to access its internal
pointer. For transient strings, this memory is then owned (and freed) by the
GermanString instance.
Small strings are copied out of the heap and stored inline as before, to avoid
any dereferencing operations.
Change-Id: I754736099f71d646d430aed73e558a5a7626c394
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12249
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
This can short-circuit two large string comparisons.
Change-Id: If45e7cf33921fe571482dc710c27ef8cc7c70885
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12245
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
This is where one of the advantages of this string representation starts to
shine: For small strings there's no derefencing any heap memory at all, and for
the long representation we can compare the prefix before moving on.
Change-Id: Iac333a52e8d7c9dd09e33dbcf51754e321c880e6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12238
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
//tvix depending less on other parts of depot is prefered as it will
help with extending the josh workspace of tvix.
Change-Id: Ifcac3af1782dfd82e7543cb4c3ae57fbd186edff
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12250
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
It's easier to implement readTree/depot polyfills for gitignoreSource
when it's imported from third_party.sources, rather than in a file at
//third_party.gitignoreSource.
Change-Id: I1323f932bd0feeb2c50ccc76397a80e035842992
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12248
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
All of these strings are currently transient (the storage class is not yet
represented anywhere), and the ones that are heap allocated need to be
deallocated when the transient string dies.
Change-Id: Iba0ca926df5db7594f304c5d5318db9d69c6f42c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12235
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
This adds an initial implementation of the so-called "German Strings" in Rust.
https://cedardb.com/blog/german_strings/
This implementation is *far from* complete, the only thing that can be done
right now is construct a string, and even that I'm not fully happy with.
Change-Id: I2697932a0ef373be76ffd14d59677493a5783b58
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12234
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Removes imbl::OrdMap in favour of an Rc over the standard library's BTreeMap,
which allows us to drop the imbl dependency completely.
In my local tests this is actually slightly faster for `hello` and `firefox`.
Change-Id: Ic9597ead4e98bf9530f290c6a94a3c5c3efd0acc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12201
Reviewed-by: aspen <root@gws.fyi>
Tested-by: BuildkiteCI
This vector type has served us well for now, but it contains internal refcounts
which are incompatible with upcoming changes related to garbage collection.
The performance impact of this change within all benchmarks I ran was within the
margin of error:
[nix-shell:/tmp/perf]$ hyperfine "./before -E '(import <nixpkgs> {}).firefox.outPath' --log-level ERROR --no-warnings"
Benchmark 1: ./u64 -E '(import <nixpkgs> {}).firefox.outPath' --log-level ERROR --no-warnings
Time (mean ± σ): 7.528 s ± 0.272 s [User: 6.578 s, System: 0.631 s]
Range (min … max): 7.160 s … 8.012 s 10 runs
nix-shell:/tmp/perf]$ hyperfine "./std-vec -E '(import <nixpkgs> {}).firefox.outPath' --log-level ERROR --no-warnings"
Benchmark 1: ./std-vec -E '(import <nixpkgs> {}).firefox.outPath' --log-level ERROR --no-warnings
Time (mean ± σ): 7.515 s ± 0.178 s [User: 6.508 s, System: 0.652 s]
Range (min … max): 7.276 s … 7.861 s 10 runs
Change-Id: Ib95f871956e336a1e5771f6293583854b1efb276
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12197
Reviewed-by: aspen <root@gws.fyi>
Tested-by: BuildkiteCI
This replaces the OpCode enum with a new Op enum which is guaranteed to fit in a
single byte. Instead of carrying enum variants with data, every variant that has
runtime data encodes it into the `Vec<u8>` that a `Chunk` now carries.
This has several advantages:
* Less stack space is required at runtime, and fewer allocations are required
while compiling.
* The OpCode doesn't need to carry "weird" special-cased data variants anymore.
* It is faster (albeit, not by much). On my laptop, results consistently look
approximately like this:
Benchmark 1: ./before -E '(import <nixpkgs> {}).firefox.outPath' --log-level ERROR --no-warnings
Time (mean ± σ): 8.224 s ± 0.272 s [User: 7.149 s, System: 0.688 s]
Range (min … max): 7.759 s … 8.583 s 10 runs
Benchmark 2: ./after -E '(import <nixpkgs> {}).firefox.outPath' --log-level ERROR --no-warnings
Time (mean ± σ): 8.000 s ± 0.198 s [User: 7.036 s, System: 0.633 s]
Range (min … max): 7.718 s … 8.334 s 10 runs
See notes below for why the performance impact might be less than expected.
* It is faster while at the same time dropping some optimisations we previously
performed.
This has several disadvantages:
* The code is closer to how one would write it in C or Go.
* Bit shifting!
* There is (for now) slightly more code than before.
On performance I have the following thoughts at the moment:
In order to prepare for adding GC, there's a couple of places in Tvix where I'd
like to fence off certain kinds of complexity (such as mutating bytecode, which,
for various reaons, also has to be part of data that is subject to GC). With
this change, we can drop optimisations like retroactively modifying existing
bytecode and *still* achieve better performance than before.
I believe that this is currently worth it to pave the way for changes that are
more significant for performance.
In general this also opens other avenues of optimisation: For example, we can
profile which argument sizes actually exist and remove the copy overhead of
varint decoding (which does show up in profiles) by using more adequately sized
types for, e.g., constant indices.
Known regressions:
* Op::Constant is no longer printing its values in disassembly (this can be
fixed, I just didn't get around to it, will do separately).
Change-Id: Id9b3a4254623a45de03069dbdb70b8349e976743
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12191
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Since we switched from reference scanning to string context, this only
handles the `__corepkgs__` hack. Update the docstrings.
Change-Id: Ie857c8c99ae1cdb4697323ec738f88be0580df3e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12246
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
cl/12228 did enable automatic retries for some flaky tests, which
generally did work, as can be seen in
https://buildkite.com/tvl/depot/builds/35893
However, "🦆" still reports as failing, because we check the number
of steps to be nonzero, which is not the case if retries have happened.
We cannot check for the overall status of the build, as it's still
"RUNNING", but instead of counting all failed steps so far, we can query
all failed jobs and then filter out the ones that were already retried.
Change-Id: Ib9d27587c8a8ba7970850812c4302fecdc4482e7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12233
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
We started using josh from nixpkgs since cl/11457, but forgot to update
this documentation.
Change-Id: I6e07227bcd3e955076d46146024edd89b69f08f7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12244
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
Tested-by: BuildkiteCI
I'm gonna be doing some poking around in the internals of Context, so in
preparation this pulls it out into its own module.
Change-Id: I72ea7df80b5f36f838934ee07bdba66874c334c9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12189
Autosubmit: aspen <root@gws.fyi>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Don't use ValidateNodeError, but SymlinkTargetError.
Also, add checks for too long symlink targets.
Change-Id: I4b533325d494232ff9d0b3f4f695f5a1a0a36199
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12230
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: edef <edef@edef.eu>
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
Tested-by: BuildkiteCI
Don't use DirectoryError, but PathComponentError.
Also add checks for too long path components.
Change-Id: Ia9deb9dd0351138baadb2e9c9454c3e019d5a45e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12229
Tested-by: BuildkiteCI
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: edef <edef@edef.eu>
Replace the hand-rolled code comparing names with a try_fold. Also, make
it slightly stricter here, detecting duplicates in the same fields
earlier.
Change-Id: I9c560838ece88c3b8b339249a8ecbf3b05969538
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12226
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: edef <edef@edef.eu>
Tested-by: BuildkiteCI
This provides a batched variant to construct a Directory, which reuses
the previously calculated size.
Checking and inserting code is factored out into a check_insert_node
function, taking the current size as a parameter and returning the new
size.
Change-Id: Ia6c2970a0c12181b7c40e63cf7ce8c93298ea37c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12225
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: edef <edef@edef.eu>
Tested-by: BuildkiteCI
The boot tests are sometimes flaky, and we don't want them to
periodically fail other build. Have Buildkite auto-retry them 2 times
on failure.
Logs for individual failures are still available, so it won't hinder
flakiness debuggability.
See https://buildkite.com/docs/pipelines/command-step#retry-attributes
for a documentation of a parameter, and cl/8983 for the introduction of
that feature to //nix/buildkite.
Change-Id: I1c0d25fa1d0ca940b3bdcd145ede87154b0c28eb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12228
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
Tested-by: BuildkiteCI
This is covered by clippy.toml these days.
Change-Id: I2330af5781844d5f9d975793d770efcea48d371b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12223
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Make this a proper struct with named fields. We apparently never access
B3Digest in there, so it can be removed.
Change-Id: Ifc07310393e1afb0a835778eae137a19b54070b9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12224
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Provide a into_nodes() function on a Directory, which consumes self and
returns owned PathComponent and Node.
Use it to provide a proper conversion from Directory to the proto
variant that doesn't clone.
There's no need for the one taking only &Directory, we don't use it
anywhere, and once someone needs that they might as well clone Directory
before converting it.
Update all other users of the `.nodes()` function to use `.into_nodes()`
where applicable, and avoid some more cloning there.
Change-Id: Id4577b9eb173c012e225337458898d3937112bcb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12218
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
This encodes a verified component on the type level. Internally, it
contains a bytes::Bytes.
The castore Path/PathBuf component() and file_name() methods now
return this type, the old ones returning bytes were renamed to
component_bytes() and component_file_name() respectively.
We can drop the directory_reject_invalid_name test - it's not possible
anymore to pass an invalid name to Directories::add.
Invalid names in the Directory proto are still being tested to be
rejected in the validate_invalid_names tests.
Change-Id: Ide4d16415dfd50b7e2d7e0c36d42a3bbeeb9b6c5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12217
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
Add a `SymlinkTarget` type to represent validated symlink targets.
With this, no invalid states are representable, so we can make `Node` be
just an enum of all three kind of types, and allow access to these
fields directly.
Change-Id: I20bdd480c8d5e64a827649f303c97023b7e390f2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12216
Reviewed-by: benjaminedwardwebb <benjaminedwardwebb@gmail.com>
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
Nodes only have names if they're contained inside a Directory, or if
they're a root node and have something else possibly giving them a name
externally.
This removes all `name` fields in the three different Nodes, and instead
maintains it inside a BTreeMap inside the Directory.
It also removes the NamedNode trait (they don't have a get_name()), as
well as Node::rename(self, name), and all [Partial]Ord implementations
for Node (as they don't have names to use for sorting).
The `nodes()`, `directories()`, `files()` iterators inside a `Directory`
now return a tuple of Name and Node, as does the RootNodesProvider.
The different {Directory,File,Symlink}Node struct constructors got
simpler, and the {Directory,File}Node ones became infallible - as
there's no more possibility to represent invalid state.
The proto structs stayed the same - there's now from_name_and_node and
into_name_and_node to convert back and forth between the two `Node`
structs.
Some further cleanups:
The error types for Node validation were renamed. Everything related to
names is now in the DirectoryError (not yet happy about the naming)
There's some leftover cleanups to do:
- There should be a from_(sorted_)iter and into_iter in Directory, so
we can construct and deconstruct in one go.
That should also enable us to implement conversions from and to the
proto representation that moves, rather than clones.
- The BuildRequest and PathInfo structs are still proto-based, so we
still do a bunch of conversions back and forth there (and have some
ugly expect there). There's not much point for error handling here,
this will be moved to stricter types in a followup CL.
Change-Id: I7369a8e3a426f44419c349077cb4fcab2044ebb6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12205
Tested-by: BuildkiteCI
Reviewed-by: yuka <yuka@yuka.dev>
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: benjaminedwardwebb <benjaminedwardwebb@gmail.com>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
The tvix-cli tests now include evaluating the NixOS GNOME installer ISO
image and making sure that its drvPath and outPath matches the one
evaluated by Nix. (This required extending the helper function a bit and
adding docs).
NixOS docs generation is disabled for now, see comments in diff.
Change-Id: Ia510f209b1ec3ef9a823f1e5ac0ef2f5f193976f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12177
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
This splits the existing ReferenceScanner into a ReferenceScanner and
ReferencePattern as well as adds an AsyncRead implementation that can
do a scan while you read from it.
The reason to split the scanner in two is that generating the pattern
is expensive and when ingesting build results with multiple outputs you
want to do several independant scans that look for the same pattern.
The reader is for scanning files without having to load the entire file
into memory.
Change-Id: I993f5a32308c12d9035840f8e04fe82e8dc1d962
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12052
Autosubmit: Brian Olsen <me@griff.name>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
This comment is a bit misleading - everything that's not in this field
is referenced in a content-addressed fashion (by their digest).
Change-Id: I5097131530fd188173393063643c057f588ea2c4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12219
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
These were out of date.
Change-Id: Ideaf888c2851bb9ec36ae01b11d93165b62cd7a5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12209
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
When using the runTests feature of crate2nix the derivation that runs the
tests is put into passthru.test but all default.nix files for Rust crates
in Tvix threw that away.
This commit retains passthru so that you can get access to the test
derivation.
Change-Id: I8b7b7db57a49069348f08c12c00a3b1a41a0c05b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12215
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
These were out of date.
Change-Id: I89df37f088ad6c53b676d965f0dc1023f40481f4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12208
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
This was out of date.
Change-Id: Icea6ee865d389d56bc46941f6049721db293ef7e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12207
Tested-by: BuildkiteCI
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
Autosubmit: flokli <flokli@flokli.de>
This apparently was out of date.
Change-Id: I9768d5e62f200169d7815ad85aa0f12eadc56a8e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12206
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
There's some followup work on making tvix-build use the stricter structs
natively, but that should probably be combined with the overall trait
changes there.
Add an item there so we won't forget, but this isn't really castore
territory anymore.
Change-Id: I6b1f9fa02d5c87c821d4ddc4b1dcc6a98c4eeaa5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12200
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: yuka <yuka@yuka.dev>
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
*Node and Directory are types of the tvix-castore model, not the tvix
DirectoryService model. A DirectoryService only happens to send
Directories.
Move types into individual files in a nodes/ subdirectory, as it's
gotten too cluttered in a single file, and (re-)export all types from
the crate root.
This has the effect that we now cannot poke at private fields directly
from other files inside `crate::directoryservice` (as it's not all in
the same file anymore), but that's a good thing, it now forces us to go
through the proper accessors.
For the same reasons, we currently also need to introduce the `rename`
functions on each *Node directly.
A followup is gonna move the names out of the individual enum kinds, so
we can better represent "unnamed nodes".
Change-Id: Icdb34dcfe454c41c94f2396e8e99973d27db8418
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12199
Reviewed-by: yuka <yuka@yuka.dev>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
We have this (`ObjectStoreDirectoryService`), as well as some store
composition.
Change-Id: I876f85da1d5b183d80b148c378f825033ec080e4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12198
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
Tested-by: BuildkiteCI
This uses our own data type to deal with Directories in the castore model.
It makes some undesired states unrepresentable, removing the need for conversions and checking in various places:
- In the protobuf, blake3 digests could have a wrong length, as proto doesn't know fixed-size fields. We now use `B3Digest`, which makes cloning cheaper, and removes the need to do size-checking everywhere.
- In the protobuf, we had three different lists for `files`, `symlinks` and `directories`. This was mostly a protobuf size optimization, but made interacting with them a bit awkward. This has now been replaced with a list of enums, and convenience iterators to get various nodes, and add new ones.
Change-Id: I7b92691bb06d77ff3f58a5ccea94a22c16f84f04
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12057
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Describe what these structs are used for, and for each of it, explain
which usecases it's used for.
Change-Id: I8b7857bc68ec2b37df9f5163e06d028a64a12c79
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12195
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
Provides a derivation file dumping functionality for tvix-cli that can
be used when passing the --drv-dumpdir CLI arg to tvix-cli.
This will dump all the known derivation files into the specified
directory, making it easier to debug derivation divergences between Tvix
generated drvs and the drvs generated by Nix.
Supersedes: https://cl.tvl.fyi/c/depot/+/11265
Change-Id: I0e10b26eba22032b84ac543af0d4150ad87aed3e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12192
Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Otherwise the flattened `ServiceUrlsMemory` docstring becomes the main
binary description, which doesn't make an awful lot of sense.
The help now describes what this CLI does currently (an interface to use
the different Tvix component.
Also, describe what it does not, so people running it without looking
too much other documentation might not get confused about why this
doesn't do anything in `/nix/store`, or doesn't provide the same CLI
surface as Nix.
Change-Id: Ia4838b444f03a10821801a6171d3e956b3cdfdaf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12194
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
This sentence is a little stale; let's just link to NixString directly
for the authoritative source of truth.
Change-Id: I64e065c4148d29702b09820a0e7724a65fae7c67
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12181
Autosubmit: aspen <root@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Just like tvix-repl does (except we don't force values when printing
them, so... not entirely like tvix-repl does). But it's something.
Change-Id: I2e69b08d7d82b0b2d337f1d4c5d87ed28475fa84
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12180
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Autosubmit: aspen <root@gws.fyi>
Configure the nix path even if globals is already set.
Change-Id: I6598c92ab40ff952f73da04d9e7d3aeb13c16b53
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12176
Tested-by: BuildkiteCI
Autosubmit: aspen <root@gws.fyi>
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
Rather than storing the leaked allocation for the string as the key in
the interner, store the hash (using NoHashHashBuilder). I thought this
would improve performance, but it doesn't:
hello outpath time: [736.85 ms 748.42 ms 760.42 ms]
change: [-2.0754% +0.4798% +2.7096%] (p = 0.72 > 0.05)
No change in performance detected.
but it at least doesn't *hurt* performance, and it *does* avoid an
`unsafe`, so it's probably net good.
Change-Id: Ie413955bdb6f04b1f468f511e5ebce56e329fa37
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12049
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: aspen <root@gws.fyi>
Had to fix some stuff in a typescript project, it's better with highlighting.
Change-Id: I984540a791167427acde3494182521b8839aed6d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12069
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
The `Fetch` is an enum, not a struct, and "Type" probably makes more
sense here. Slightly reword the "generic" to not be confused with
generics.
Also fix the link to Tvixbolt, which got eaten by the line wrapping.
Change-Id: I677229f0365523620a640c2333e76790573f7318
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12174
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
Tested-by: BuildkiteCI
When we moved to building tvix with crate2nix we had to disable the
readDir tests for the same reasons mentioned in
https://cl.tvl.fyi/c/depot/+/12131. But now with the 12131 CL, we can
re-enable the readDir tests.
In this change I re-enabled the readDir tests in nix_tests by moving
them out of notyetpassing and I also deleted them from tvix_tests
because we don't need them in both places.
Change-Id: I82ac39605299a8b22d80f8b51fc8ec2476d21dc9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12133
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com>
builtins.readFileType was added to Nix back in version 2.14.
The tests were also moved out of notyetpassing in addition to the
readDir fixtures they depend on.
I caught a bug where we previously used std::fs::metadata (via the
.metadata() method on File) which follows symlinks so it would always
return false for is_symlink(). Instead we now use
std::fs::symlink_metadata directly which does not follow symlinks, so
tests now pass. This wasn't an issue for builtins.readDir as it uses
walkdir and walkdir doesn't follow symlinks either.
Change-Id: I58eb97bdb5ec95df4f6882f495f8c572fe7c6793
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12130
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com>
Tested-by: BuildkiteCI
The problem with using runCommand and recreating the src directory with
lndir is that it changes the file types of individual files, they will
now be a symlink instead of a regular file. If you have a crate that tests
that a file is of regular type then it will fail inside the crate2nix
derivation.
Also regenerate Cargo.nix for //tvix as it will be needed in the next
commit.
Change-Id: I9275602cc17a428f9fdf0e55daf12cd673bbc030
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12131
Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
With this change the latest Nix version being checked is 2.23
(previously was 2.18). A warning will now be printed when the
latest verified Nix version is out of date.
I added a test to the skip-list because the behavior of
builtins.dirOf has changed in Nix 2.22
Change-Id: Iaf7fa98dbfa599a2d2723df0e42bb459711e2413
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12105
Tested-by: BuildkiteCI
Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com>
Reviewed-by: flokli <flokli@flokli.de>
In newer versions of Nix there's a builtins.readFileType builtin, we
should try to avoid duplicating the enum -> string conversion by
implementating Display before we implement builtins.readFileType.
Change-Id: I579e95949a76eb33d2e7bda0000ed84859df765e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12129
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com>
Tested-by: BuildkiteCI
Use the faster and newer MiMalloc memory allocator for all endpoints in
the workspace.
Change-Id: Ic60237284ed168e46ec6e8f28e2710bae4385c6f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12149
Tested-by: BuildkiteCI
Reviewed-by: aspen <root@gws.fyi>
Rather than making the interner be a global lazy_static mutex, put it in
a thread-local RefCell. This doesn't change anything in terms of
sharing (since we're currently actually just single threaded), but
avoids the overhead of a mutex, for a nice performance boost (compared
to the mutex version):
hello outpath time: [726.71 ms 729.79 ms 735.69 ms]
change: [-5.7277% -3.9733% -2.1144%] (p = 0.00 < 0.05)
Performance has improved.
Change-Id: I240b238dcbaf854ebafc3017b4425fb7d7b91b03
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12048
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: aspen <root@gws.fyi>
This is the most naive version of string interning possible - we store a
map from the string itself to the pointer behind a global mutex, and
memoize the allocation of all strings below a threshold length (16
bytes, for now) into that map. This requires leaking /all/ strings,
since it's not easy to know just from the pointer that a string has been
interned - so interning is disabled if string leaking is also disabled.
In the case where we're leaking strings (the default), even the naive
version of this gets us a pretty nice perfomance boost:
hello outpath time: [742.54 ms 745.89 ms 749.14 ms]
change: [-2.8722% -2.0135% -1.0654%] (p = 0.00 < 0.05)
Performance has improved.
However, in the case where we're not leaking strings, we have to keep
track of which strings have and haven't been interned, which makes this
a little worse:
hello outpath time: [779.30 ms 792.82 ms 808.74 ms]
change: [+2.5258% +4.0884% +5.8931%] (p = 0.00 < 0.05)
Performance has regressed.
Hopefully we can close the gap here a bit with some clever
tricks (coming next).
Change-Id: If08cb48ede703c7fe3bdd8d617443f8a561ad09b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12047
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: aspen <root@gws.fyi>
Per https://nnethercote.github.io/perf-book/hashing.html, we have
basically no reason to use the default hasher over a faster,
non-DoS-resistant hasher. This gives a nice perf boost basically for
free:
hello outpath time: [704.76 ms 714.91 ms 725.63 ms]
change: [-7.2391% -6.1018% -4.9189%] (p = 0.00 < 0.05)
Performance has improved.
Change-Id: If5587f444ed3af69f8af4eead6af3ea303b4ae68
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12046
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
Autosubmit: aspen <root@gws.fyi>
This depends on the ChunkReader work.
Change-Id: I38878d0f822c312151131e55baee4db6ef1c3650
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12142
Tested-by: BuildkiteCI
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
Autosubmit: flokli <flokli@flokli.de>
Explain the current caveats as far as performance tuning is concerned.
Change-Id: I1a9c11c81de09350240fb61e3c130fc401ef6618
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12141
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: yuka <yuka@yuka.dev>
Tested-by: BuildkiteCI
We’d transfer the full json data for each torrent from the db instead
of just the 2 or 3 fields we need.
Adds some more helpers for parsing database values.
Adds some better logging events & traces.
Change-Id: I5db386c4ea247febf5f9fc3815da2e7f11286d41
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12140
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
The queries would not be interpolated anymore, because we didn’t pass
the thing down deep enough.
Also only init the `pgFormatPool` if we want to use the formatter,
this saves on a bunch of subprocesses.
Change-Id: I8d69ef5aab4d8eac1cbfb1c3991d4edaacba254f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12139
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
It turns out the pg_format thing is just too slow for my use-cases
most of the time, even when pooling the mf. Most queries stay 90%+ in
the perl script, even though they are very fast to execute on their
own, screwing up the traces a lot.
So instead I replace the `postgres-simple` quasi-quoter that strips
whitespace (and tends to screw up queries anyway) with a simple one
that just removes the outer indentation up to the first line.
Why did I spend so much time on pg_format haha
Change-Id: I911cd869deec68aa5cf430ff4d111b0662ec6d28
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12138
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
Parked projects: I might pick these up again, but don’t want to
maintain them right now. Might delete later, or reinstate.
Change-Id: Ieb4f51dfaeba33c84820ca804b57c016e05bf566
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12137
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
Ideally there’d be a better generic abstraction of doing basic http
calls (with tracing) in the future, but for now just reexport.
Change-Id: Id7548739ea62e9172f2773f8db79fe726096b7f1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12136
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
There’s a bunch of duplication in how http client things are done,
let’s move that all to a single module.
Change-Id: Ic08c9bce49d562e4fa640a5bdfc15973a28a7bcb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12135
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
Default to always leaking strings, and copying strings by copying
pointers rather than cloning the underlying allocation. This (somewhat
bafflingly) doesn't seem to affect any benchmarks, but paves the way for
some tricks around string allocation that do.
Unfortunately, we can't do this (yet?) for contextful strings, for
reasons I don't currently understand but which I will address later,
when I address contextful strings more holistically.
I've left a flag in here to disable this, both to test the cloning logic
for strings for when/if we decide to bring this back, and to allow
people who care more about memory usage than perf to disable leaking.
Change-Id: Iec44bcbfe9b3d20389d2450b9a551792a79b9b26
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12045
Autosubmit: aspen <root@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Use redb instead of sled for the default filesystem implementation of
PathInfoService and DirectoryService. In the future we'll also drop sled
support completely.
Change-Id: I513ff0c2ff953d59714aa50b9aa1301b02f53d40
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12085
Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
This is a partial revert of https://cl.tvl.fyi/c/depot/+/12068 where I
changed tvix/crate2nix-check to use depotfmt.check. It turns out that we
don't actually want to be using it in this case as the wrapper sets
`--fail-on-change` which would always fail because the Cargo.nix
generated by crate2nix will always need to be changed (It's not
formatted properly).
Change-Id: Ife35c812ca69c90459ce4445f4252d0a24c218fa
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12132
Tested-by: BuildkiteCI
Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com>
Reviewed-by: flokli <flokli@flokli.de>
This provides a DirectoryService implementation which uses
redb (https://github.com/cberner/redb) as the database. It provides both
in-memory and persistent on-filesystem implementations.
Change-Id: Id8f7c812e2cf401cccd1c382b19907b17a6887bc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12038
Tested-by: BuildkiteCI
Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com>
Reviewed-by: flokli <flokli@flokli.de>
The "path" key in the arguments to builtins.path supports any
path-coercible type (a string, a path...). Coerce it to a path in the
argument rather than just requiring it already be one and throwing an
error if it's not.
This is... annoying to test, since it requires a file with known
contents that's available in the build sandbox. But it works! Trust me!
Fixes: b/412
Change-Id: I3c8e339bf344a208d5ed5990193942651f318745
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12053
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: aspen <root@gws.fyi>
This change exposes the already existing wrapper for treefmt/depotfmt
that supports running it inside a sandbox. We now reuse it inside
//tvix/crate2nix-check, where we previously duplicated the code. The
check is now stricter and will also fail on changes, so I had to set the
rust edition in the treefmt config.
Change-Id: I000e52421258979c038ba6b1f1ff2db14e391b0c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12068
Tested-by: BuildkiteCI
Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com>
Reviewed-by: flokli <flokli@flokli.de>
The environment variables caused both nar-bridge and tvix-store daemon
to try to connect to the same store, which fails due to locking issues.
Pass the config to `tvix-store daemon` directly. Also, add the
`--otlp=false` to the instructions to remove spam, most users don't have
a OTLP collector running.
Explain when to use gRPC in `tvix-store virtiofs` and when not.
Also, point out to be able to boot a NixOS closure, it needs to be
copied into the store first.
Change-Id: If4eda07bba28ad0bbe70e468cb727441a21b0588
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12067
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
It was previously pointing to a sled implementation which no longer
exists. It was also using nar-bridge-go instead of the new nar-bridge
(rust).
Closes: https://b.tvl.fyi/issues/413
Change-Id: Id0df61d4728198c3ae95a8e27ba7303434892966
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12063
Reviewed-by: aspen <root@gws.fyi>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com>
Start of an effort to make the app work without javascript
enabled (graceful degradation yay).
We use a trick where buttons are nested into a form element, passing
their value as input; this should be better than depending on
`hx-vals`. If htmx is disabled, just redirect and reload the full page
instead of sending back the snippet. Probably depends on the use-case
of each snippet though.
Change-Id: I6c73e624c4bd29b1cbd5492b2f84f48102edc68b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12056
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
This is nice to test too - it's similar to hello, but runs for a lot
longer (like 7.5 seconds on my laptop) which means we get even better
stats for stuff.
Change-Id: I7935818f10a6d846d446e685b9263a72d7e2aabd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12061
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: aspen <root@gws.fyi>
We already do this for redb and for sled in SledDirectoryService.
Change-Id: I34c7178257a6a04e9f12ed4037a4ef585d7b0d54
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12060
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com>
Add more logging and remove context from errors because that's already
provided by the logs (Errors also need to be refactored anyway, there's
also confusion about StorageError vs InvalidRequest, there's no
consistency)
Change-Id: Ia43c0d237d9075152490c635b05fb3fb343abcc8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12058
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
The decode function didn't check that the input had a valid length and
so would panic when given input with invalid length.
Change-Id: Ie27d006b8fe20f005b4a47a1763821a61e9a95c7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12051
Reviewed-by: aspen <root@gws.fyi>
Tested-by: BuildkiteCI
Autosubmit: Brian Olsen <me@griff.name>
Previously, OpConstant would display some detail about its
ConstantIdx: whether it's a thunk or closure, and what its address
is. This has been expanded to also show when the ConstantIdx is a
blueprint, along with the blueprint's address, and to the other
opcodes that use a ConstantIdx.
Currently, it seems like blueprint addresses don't correspond to the
address of the thunk listed in the bytecode output, but it's still
useful to see that the constant being grabbed is a blueprint, and
maybe this pointer can be made to make more sense in the future.
Change-Id: Ia212b0d52b004c87051542c093274e7106ee08e4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12044
Tested-by: BuildkiteCI
Reviewed-by: aspen <root@gws.fyi>
Autosubmit: chickadee <matthewktromp@gmail.com>
path_exists was returning an error when certain common IO errors were
encountered. e.g. in the path "/dev/null/.", path_exists would throw
an error because the underlying call to Path::try_exists threw an
error because null isn't a directory. But if null isn't a directory,
then the path is invalid, so this should really be returning
false. That's what nix's behavior is and that's what makes sense.
The trait function isn't being changed because some other
implementers (e.g. tvix_store_io) have actual errors they can throw.
Fixes: b/411
Change-Id: I9e810e7a198bffe61365697c6d3d7e71f264c20b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12042
Tested-by: BuildkiteCI
Autosubmit: chickadee <matthewktromp@gmail.com>
Reviewed-by: aspen <root@gws.fyi>
This is the `{fixed,fixed:r,text}:{sha*,md5}` prefix used in various
string representations.
Factor that code out, and use it in the two places it can be used in.
Change-Id: Ic9555fa9e1884198d435e55c7f630b8d3ba2a032
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12041
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: Brian Olsen <me@griff.name>
When retrieving a closure with get_recursive, the following could happen in the GRPC client:
- The first reference to the deduplicated directory is added to expected_directory_digests
- The deduplicated directory is obtained removed from expected_directory_digests
- The second reference to the deduplicated directory is added to expected_directory_digests
- The deduplicated directory has already been sent, but is still in the
expected_directory_digests. It looks to the GRPC client like the
closure is incomplete and the stream ended prematurely.
Change-Id: Ic62bca12e7f8fb85af5fa4dacd199f0f3b8eea8c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12033
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Use the ChunkedReader in CombinedBlobService instead which also supports seeking.
Change-Id: I681331a80763172c27e55362b7044fe81aaa323b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12031
Autosubmit: yuka <yuka@yuka.dev>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Ensure nar-bridge is healthy before connecting to it, don't just check
for the unix socket to be present.
We don't have a proper /health endpoint yet, but nix-cache-info works
fine for now.
Change-Id: I22df2c3b7bffcf52dbd3d00f3ba5382dc06ab03d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12030
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: yuka <yuka@yuka.dev>
Tested-by: BuildkiteCI
Ensure the service is healthy before connecting to it, don't just check
for the unix socket to be present.
Change-Id: If6501828677c247910d91f35b860960802084691
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12029
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
This provides a PathInfoService implementation using redb
(https://github.com/cberner/redb) as the underlying storage engine.
Both an in-memory variant, as well as a filesystem one is provided,
similar how it's done with the sled implementation.
Supersedes: https://cl.tvl.fyi/c/depot/+/11692
Change-Id: I744619c51bf2efd0fb63659b12a27cbe0b2fd6fc
Signed-off-by: Ilan Joselevich <personal@ilanjoselevich.com>
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11995
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Parsing of the narinfo file sets the compression field to None instead
of Some("none"). The mapping selecting the decompression reader expected
the former in //tvix/store/src/pathinfoservice/nix_http.rs.
Change-Id: I254a825b88a4016aab087446bdc0c7b6286de40c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12007
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
This adds a generic `SigningKey` struct that can be used to sign
NARInfos with signers.
It also includes tooling to parse keypairs from bytes generated by Nix,
returning a specialized ed25519_dalek variant.
Change-Id: Ic9780c370939af54e7177c93cde3321adf189fc3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12014
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Align these with the way it's called in the ed25519 crates.
Change-Id: Ia52d3bb9bf831dc6b5f7d5356f5ac62135672883
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12013
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: flokli <flokli@flokli.de>
This documents some thoughts and goals of the Tvix Build protocol, and
how it is possible to express Nix builds with it.
Additionally, it explains a proposed design for reference scanning.
Change-Id: I4b1f3feb2278e3c7ce06de831eb8eb1715cba1c9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12012
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: yuka <yuka@yuka.dev>
Tested-by: BuildkiteCI
We still have the unique store name to identify which instantiation caused the error. For recursion errors, the full chain is still retained inside the CompositionError.
Change-Id: Iaddcece445a5df331e578d7c69d710db3d5f8dcd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12002
Tested-by: BuildkiteCI
Autosubmit: yuka <yuka@yuka.dev>
Reviewed-by: flokli <flokli@flokli.de>
Having `tvix-store daemon` and `nar-bridge` listen on the same port by
default is silly.
Use the same port that nar-bridge-go was using previously.
Change-Id: Id374a12e52ba9dac4318e43d0ca0853866eadabc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11998
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: yuka <yuka@yuka.dev>
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
spf=neutral (google.com: 64.147.123.21 is neither permitted nor denied by best guess record for domain of firefly@firefly.nu) smtp.mailfrom=firefly@firefly.nu
spf=neutral (google.com: 64.147.123.21 is neither permitted nor denied by best guess record for domain of firefly@firefly.nu) smtp.mailfrom=firefly@firefly.nu
Received: from wout5-smtp.messagingengine.com (wout5-smtp.messagingengine.com. [64.147.123.21])
by mx.google.com with ESMTPS id m23si6276235ejo.333.2020.09.27.16.35.13
Received-SPF: neutral (google.com: 64.147.123.21 is neither permitted nor denied by best guess record for domain of firefly@firefly.nu) client-ip=64.147.123.21;
Received: from compute1.internal (compute1.nyi.internal [10.202.2.41])
by mailout.west.internal (Postfix) with ESMTP id AF9D4858
for <depot@tazj.in>; Sun, 27 Sep 2020 19:35:11 -0400 (EDT)
Received: from imap6 ([10.202.2.56])
by compute1.internal (MEProxy); Sun, 27 Sep 2020 19:35:11 -0400
