With this change, the query we make to Buildkite changes from
explicitly querying for a specific number of *ancestor* builds, to
any latest builds of the default branch that have a drvmap.
This is not really supported by Buildkite, it seems, and the query to
do it feels very wonky and requires a lot of fiddling with `jq` to get
the output into the right shape.
We lose the information about which build we downloaded this from in
the output. Adding that information back would make the `jq` query
much more complex.
Change-Id: I9e7cecdffa9ac09f9e0339eb24d98c0e8dd82292
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9168
Tested-by: BuildkiteCI
Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
And of course I managed to move the cache creation into the handlers,
instead of doing it before starting the webserver.
And now I managed to create a hopeless mess of callbacks, but oh well.
Change-Id: I73c3aeced71923c7372496286a279e326b20c388
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9813
Reviewed-by: Profpatsch <mail@profpatsch.de>
Autosubmit: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
I’ve been wanting to experiment with this stuff for a while,
abstracting away a handler type.
The existentials for parser and body took a bit of mucking about, but
in the end hiding the variable behind a `Body` constructor did the
trick.
Now every handler has its own cache, which means we can start caching
arbitrary results.
Change-Id: If57230c47f97ef4c548683f2c2f27660817a31f2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9812
Autosubmit: Profpatsch <mail@profpatsch.de>
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
Now this is getting cool. After 5 minutes we will ask the backend
again (which takes like 3 seconds), but then we compare the old cached
result with the new result and only send it back to the client iff it
changed.
So the client will still have to wait for the roundtrip time, but
doesn’t have to pay for the content. Plus, it gets some info that
upstream hasn’t been updated.
Change-Id: I6dba40321949da5da6a16b2e799d939573c77ba7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9811
Autosubmit: Profpatsch <mail@profpatsch.de>
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
This is a dumb experiment to see how hard it is to respect cache
headers; turns out, medium hard but doable.
Sets the correct expiry time according to the cache, plus respects
`If-Modified-Since` which is a tiny bit harder.
Change-Id: I9e6166af0fa254df2beb0f3919187b91a407487b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9810
Autosubmit: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
Okay, so I guess you also have to seq the cache and everything in
between the IORef and the data.
Change-Id: I4c79c99afbd09e83e9d7a01d58b31b36862e4d11
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9807
Reviewed-by: Profpatsch <mail@profpatsch.de>
Autosubmit: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
Otherwise the table might potentially hold onto data from the website
request, it’s hard to say.
Change-Id: I786478bd1ce2d9775b3d0b57565d79666ef8a96f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9806
Autosubmit: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
Back at my bullshit.
Mostly copied the setup from whatcd-resolver.
Change-Id: I9edd4387ee73c18816b1692d5338735536cce70f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9803
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
Autosubmit: Profpatsch <mail@profpatsch.de>
It is *eventually* followed by a Node, but there is some stuff in
between.
Change-Id: Ie7c7b462828bd3e066f4a7e774895f30b82763ef
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9768
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Equivalent to the existing code, but a little less cryptic.
Change-Id: Ib9b2f9aedddc84d0e79840bba4cce01f92d9bc56
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9766
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
As seen in cl/9761, we only use put() and close() in some cases.
Make sure this is documented, so we don't end up having implementations
not able to deal with that.
Change-Id: I406ad23f2f4c9604eca731d898ae41a4663fb846
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9786
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
`code.tvl.fyi/tvix/store/protos` now points to a directory that only
contains the `.proto` files, while all golang tooling and .pb.go files
live in tvix/store-go.
As discussed in
https://cl.tvl.fyi/c/depot/+/9787/comment/fc5d155c_1bd38e3a/, the amount
of people currently using this is still small, so rename the go.mod now,
while it doesn't yet hurt.
Also, use code.tvl.fyi/tvix/castore-go instead of code.tvl.fyi/tvix/
castore/protos, to make use of cl/9791.
Change-Id: I9ea89957d7c29dfae4c893b9aae8ac8a0bad2d8e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9792
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
`code.tvl.fyi/tvix/castore/protos` now points to a directory that only
contains the `.proto` files, while all golang tooling and .pb.go files
live in tvix/castore-go.
As discussed in
https://cl.tvl.fyi/c/depot/+/9787/comment/fc5d155c_1bd38e3a/, the amount
of people currently using this is still small, so rename the go.mod now,
while it doesn't yet hurt.
Change-Id: Ib3c6a2dac2923b3806ebb05be00af66d0da9f698
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9791
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
This keeps things isolated a bit more.
Change-Id: I437f2f63ee1567c0cbc02298514ad4d89f1dce85
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9790
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
buf lint already happens in other individual targets, and as no
generation happens in here anymore either, there's no need to check for
git status to record changes anymore.
Change-Id: Ieb8fc4760a61dd0f0e03f0ec388062dd2303c37a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9789
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Similar to the castore-go CL before, this also updates the store-go
bindings to the new layout.
Change-Id: Id73d7ad43f7d70171ab021728e303300c5db71f0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9788
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Have `tvix/castore/protos` only contain the protos, no go noise.
Make the `.pb.go` file generation a pure Nix build
at `//tvix/castore/protos:go-bindings`, and have a script at
`//tvix:castore-go-generate` (TBD) that copies the results to
`tvix/castore-go`.
`//tvix:castore-go`, with sources in `tvix/castore-go` now contains the
tooling around the generated bindings, and the generated bindings
themselves (So go mod replace workflows still work).
An additional CI step is added from there to ensure idempotenty of
the .pb.go files.
The code.tvl.fyi webserver config is updated to the new source code
path. I'm still unsure if we want to also update the go.mod name. While
being a backwards-incompatible change, it'll probbaly make it easier
where to find these files, and the amount of external consumers is still
low enough.
Part of b/323.
Change-Id: I2edadd118c22ec08e57c693f6cc2ef3261c62489
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9787
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
After finishing the ingestion, the directory putter was not being
closed. This caused a race where the root directory node was accessed
before the directory node had been flushed to the server.
This patch makes it so we close the putter before returning the root
node which should ensure that the root node exists on the directory
service server before the `ingest_path` function returns.
Fixes b/326
Change-Id: Id16cf46bc48962121dde76d3c9c23a845d87d0f1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9761
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
That's needed to build nar-bridge, as well as things in tvix/*store/
protos.
Change-Id: If5f71b6c528a27689faa3c5ab3489cc528f9e6e5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9759
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
We need to invoke protoc at build time (only).
Change-Id: Ib4c101d2ccdbdbb078725fc98c30d6c00086667d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9754
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
On Darwin, some crates producing binaries need to be able to link
against security.
Change-Id: I5bdd69247c12729b9efd5c4f18527d361ef99e87
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9758
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Also, make some of them buildInputs, not nativeBuildInputs.
We likely need to link against libiconv, so it shouldn't be a native
build input.
Change-Id: I757bc7f5d8c864ecaa3db26988d00b496c085c8f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9757
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Autosubmit: flokli <flokli@flokli.de>
It looks like we need Security Framework in the shell to be able to
imperatively run `cargo build`.
Change-Id: Ia5df9052bafca3610d4ed235b180b812d33d0d15
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9756
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Autosubmit: flokli <flokli@flokli.de>
The previous CLs did already absorb all the logic into the common tests,
no need to write this here again.
Change-Id: I7ba84ba86d5445ed247e5d11d5e59b7fa815670e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9732
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
Due to the lack of a ATerm parser, we were previously loading the JSON
fixtures to construct our Derivation structs to run the output path
calculations with.
However, as we now have a ATerm parser, we can load the ATerm
representation directly.
This also means we can test the output path calculation for non-UTF8
Derivations.
Change-Id: I0e53f41a23566b5ad5f0fed12724e02a10b02707
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9731
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
This provides a nom-based parser for Nix derivations in ATerm format,
which can be reached via `Derivation::from_aterm_bytes`.
Some of the lower-level ATerm primitives are moved into a (new) aterm
module, and some more higher-level ones that construct derivation-
specific types.
Also, move the escape_bytes function into there, this is a generic ATerm
thing.
Change-Id: I2b03b8a1461c7ea2fcb8640c2fc3d1fa3ea719fb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9730
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
Gerrit instructions now recommend fetching the commit-msg hook via HTTP
in all three cases, we should do the same.
Change-Id: I12d5784ccacd3ea68fb81fc15508b209a22cd2c1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9745
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
Bump gerrit plugins dep hashes, and for code-owners rebase it against
master.
Change-Id: If7da0ca391b4a5c0102560ca8d52b6f5a2dfd223
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9734
Autosubmit: lukegb <lukegb@tvl.fyi>
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
We can cross-reference all of these to schema.org, it should work for
most of the fields.
Change-Id: I38d8dbc7e964764886ddd156c4148bcf3ee376f3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9744
Autosubmit: Profpatsch <mail@profpatsch.de>
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
Ideally we can figure out how to search for single songs by grepping
through musicbrainz. For this we kinda need the jsonld results, so
this is a first step which visualizes the structure and makes it
easy-ish to lazily traverse it.
Change-Id: Ieca21674dee8e8c2dacbab4f2f15ccbe067da647
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9743
Reviewed-by: Profpatsch <mail@profpatsch.de>
Autosubmit: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
