Add a system for writing, running, and tracking database
migrations (changes to the database schema) over time, inspired by but
significantly simpler than postmodern-passenger-pigeon.
Migrations can be generated by
running (PANETTONE.MODEL:GENERATE-MIGRATION "name"), and are numerically
ordered lisp files that define (at least) a function called UP, which
runs the migration. The migrations that have been run against the
database are tracked in the `migrations` table, and when the
`(PANETTONE.MODEL:MIGRATE)` function is called (as it is on startup),
all migrations that have not yet been run are run within a transaction.
This includes one migration `1-init-schema.lisp`, which migrates the
database (idempotently) to the current state of the schema.
Change-Id: Id243a47763abea649784b12f25a6d05c2267381c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11253
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
My new laptop has the battery at index 1, not index 0
Change-Id: I64fa16e7f0eecf6b623361645d73b1176e1e5d48
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11311
Tested-by: BuildkiteCI
Autosubmit: aspen <root@gws.fyi>
Reviewed-by: aspen <root@gws.fyi>
The expression needs updating on our side to be compatible with the
nixpkgs bump in cl/11289. It wasn't used anywhere in TVL either.
This is also packaged in nixpkgs, since
f3ea5cfa6ec0a640735f4d6a0b5c91683c9bab25 / Feb 20 2024, so using it from
there feels reasonable.
Change-Id: Icd50ad959ca0de433b847180f1a5be308719f287
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11295
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Bump to the latest u-root release. This seems to be compatible with go
1.22, too, which previously segfaulted the compiler in cl/11289.
Change-Id: Ie8a0d3cb51a435d6a0b194b38a9c83ee260618df
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11291
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Only those that want them
Change-Id: Id1caf5e89f69c51a7da90ff2387a9384abbf4f91
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11308
Autosubmit: aspen <root@gws.fyi>
Reviewed-by: aspen <root@gws.fyi>
Tested-by: BuildkiteCI
For the rare times I still occasionally use vim
Change-Id: Ie3059ab0ec5d5ace729e78b765f251d1fafdfac1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11305
Reviewed-by: aspen <root@gws.fyi>
Autosubmit: aspen <root@gws.fyi>
Tested-by: BuildkiteCI
This is not necessary now that I'm using tailscale everywhere
Change-Id: I36d8060d4ad5f6955cd18c68b5fc667e64669004
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11303
Reviewed-by: aspen <root@gws.fyi>
Autosubmit: aspen <root@gws.fyi>
Tested-by: BuildkiteCI
I never got this working
Change-Id: Iacb68a7d6d5fe1d1f1f390d09f106c4eb9a51b6a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11301
Reviewed-by: aspen <root@gws.fyi>
Autosubmit: aspen <root@gws.fyi>
Tested-by: BuildkiteCI
These are noisy, but more importantly I'm worried I might accidentally
commit a secret at some point
Change-Id: If6f2c358f2803af25ea27ef34d39c7f2108d4186
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11299
Reviewed-by: aspen <root@gws.fyi>
Autosubmit: aspen <root@gws.fyi>
Tested-by: BuildkiteCI
This builtin only transforms any `NixContextElement::Derivation` into the trivial `NixContextElement::Plain`.
This is a forgetful functor on derivation-deep context strings.
The test coverage of this change is done in cl/11264.
Change-Id: Icd00778c97766be6db8a6bdabaa59e9724353ec5
Signed-off-by: Ryan Lahfa <tvl@lahfa.xyz>
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11262
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Derivation that depends on `${d.drvPath}` generates a `NixContextElement::Derivation(drvPath)` context string.
In turn, this makes the dependent derivation depend on *ALL* outputs of that derivation.
Note that a dependency on `${d.drvPath}` generates an input source
dependency too.
This is a complete implementation of the context string system in Nix on
the "input population" side.
The test coverage of this change is taken care in cl/11264.
Change-Id: I97fe5f7c772a6b1cc4366bee071aa691a11fcde6
Signed-off-by: Ryan Lahfa <tvl@lahfa.xyz>
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11261
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
This is currently unused, and even if it would be used, it'd probably
be with `default-features = false`, explicitly enabling what's needed.
Change-Id: I3175a19adf8d65f0b805e5234267e460cd5e68d8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11287
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: picnoir picnoir <picnoir@alternativebit.fr>
Tested-by: BuildkiteCI
This provides an emulator for Google Bigtable.
This has also been sent to nixpkgs in
https://github.com/NixOS/nixpkgs/pull/298044, and can be dropped (modulo
the patches) once we bump past it.
Change-Id: Id7809a02065ce479d6d29813d1de454cd230d2df
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11211
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Nixpkgs make use of it and this is a 2.3 builtin that was missed.
Change-Id: If36d748eed4a5f777344556c0802392c76c8d53c
Signed-off-by: Ryan Lahfa <tvl@lahfa.xyz>
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11260
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Make this function async, and do the block_on on the (single) callsite.
Change-Id: Ib8b0b54ab5370fe02ef95f38a45d8866868a9d60
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11285
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
Replace the (single) callsite with some code interacting with the tokio
runtime to block on the async version.
Change-Id: I3976496ae77b2bb8734603f303655834265e3f0a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11284
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Let's get rid of these sync helpers, they make this less understandable.
Change-Id: I3c7294647849db2747762722247c65e4e2947757
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11283
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
We need to define behaviours and add tests for these.
Change-Id: Id5825fafbf47897d8de42503ea6006eb131b1082
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11281
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
This drops pretty much all of castore/utils.rs.
There were only two things left in there, both a bit messy and only used
for tests:
Some `gen_*_service()` helper functions. These can be expressed by
`from_addr("memory://")`.
The other thing was some plumbing code to test the gRPC layer, by
exposing a in-memory implementation via gRPC, and then connecting to
that channel via a gRPC client again.
Previous CLs moved the connection setup code to
{directory,blob}service::tests::utils, close to where we exercise them,
the new rstest-based tests.
The tests interacting directly on the gRPC types are removed, all
scenarios that were in there show now be covered through the rstest ones
on the trait level.
Change-Id: I450ccccf983b4c62145a25d81c36a40846664814
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11223
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
As for grpc_pathinfoservice, we test this in the rstest-based tests,
with more implementations.
tests/utils.rs is unused now.
Change-Id: I62b192170d181e4fb432bf7dfff2057cb048b52c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11279
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
Remove usage of the gen_{blob,directory}_service() helper functions
from utils. We populate Memory{Blob,Directory}Services here directly, as
test_case and rstest doesn't compose well.
Change-Id: I0fb48aadb8c818f508b18ceb83c85eb91359442a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11278
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
Previous CLs ensured BlobService and DirectoryService is implemented on
AsRef<dyn BlobService> and AsRef<dyn DirectoryService> respectively, so
there's no need to require AsRef in here.
Change-Id: I1e8b93188ca784e1b9a592f3d35fcf6b9df75c75
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11275
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
This stops using the gen_{blob,directory}_service() functions from
tvix_castore::utils in favor of the rstest-based fixtures.
Change-Id: I4c2fff2926f34af63d3bd5c29c8e939dbdf7e062
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11274
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
This introduces rstest-based tests. We also add fixtures for creating
some BlobService / DirectoryService out of thin air.
To test a PathInfoService, we don't really care too much about its
internal storage - ensuring they work is up to the castore tests.
Change-Id: Ia62af076ef9c9fbfcf8b020a781454ad299d972e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11272
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
This makes the fixture more use-able when interacting with the trait,
the Bytes are only useful for the gRPC version.
Change-Id: Iaaea1adc6df18491f236a28c4343f5b4ee5fcfd3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11271
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
This allows StorePathRef as keys in a hashmap, by deriving Hash. The same is already done for StorePath.
Change-Id: I3fc54c45787948116dcb27dfb5dc806b9b505835
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11269
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
At least terraform wants all parameters passed via JSON to be strings.
It can't accept maps.
This means, allowing to pass a `argstr` JSON dict isn't really possible.
However, terraform is perfectly able to JSON-encode a map. So accept a
`argstr_json` argument instead, which `jq` will JSON-decode before
further processing it.
I dropped `argstr` support again so the jq expression still fits on my
screen, if anyone else (started) using this, I'm happy to review a CL
adding this. We should probably move the jq expression to some multiline
format then, though.
Change-Id: I5ab7a1169ab7305d3ab02db31c27732d9d1ab4e8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11228
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
If this is set to true (and only then), also invoke `nix-build` on the
previously-instantiated .drv to cause builds/substitutions on the local
machine.
There's no terraform example for this in here, but this is useful if you
want to perform builds locally, for example to upload nix-built blobs
elsewhere through terraform.
Change-Id: Idcf7b8527aa9c27f6f9ca60ca607c29d82e1cce9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11215
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Implementing the full connection handshake.
The integration test is a bit naive, but there's not much to test yet.
Tested this against cpp nix. We reach the stage where cppnix sends the
opcode.
Change-Id: I98322832848ee5b048f22105731b0adeb44b2ce0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11227
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
We already did all the instantiation, grabbing the calculated output
path too is cheap.
Change-Id: Id591865c65159409da739f706a9de29a9f50456a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11214
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
This allows us to use containers around BlobServices as BlobServices too.
Change-Id: I3c7feb074f42b4e07c550fb8dfa63cf81d448ab5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11249
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
This creates test scenarios (using the DirectoryService trait) that we
want all DirectoryService implementations to pass.
Some of these tests are ported from proto::tests::grpc_directoryservice,
which tested this on the gRPC interface (rather than the trait),
some others ensure certain behaviour for which we only recently
introduced general checking logic (through ClosureValidator).
We also borrow some code related to setting up a gRPC DirectoryService
client (connecting to a server exposing a in-memory DiretoryService)
from castore::utils, this will be deleted once it's all ported over.
Change-Id: I6810215a76101f908e2aaecafa803c70d85bc552
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11247
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
