Commit graph

859 commits

Author SHA1 Message Date
Vincent Ambo
00dab6142e test(tvix/serde): add initial set of deserialisation tests
Change-Id: I0a9779edf0296c25d37fb5f75f8fc5852fe00121
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7715
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: flokli <flokli@flokli.de>
2023-01-03 13:37:12 +00:00
Vincent Ambo
4350be34d1 feat(tvix/serde): handle nested data structures
Change-Id: I543fc05d31bbb9ad2edb887bce4510e9a1cdb102
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7714
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: flokli <flokli@flokli.de>
2023-01-03 13:37:12 +00:00
Florian Klink
8e2b1de3d5 feat(tvix/store/nixpath): DIGEST_SIZE public, use more consts
Rename PATH_HASH_SIZE to DIGEST_SIZE.

It's a digest, not hash (we don't necessarily have the internal hash
state anymore), and the fact it's about (Nix)Paths is already visible
from the module name.

Also expose ENCODED_DIGEST_SIZE, so we don't need to do the calculation
inside from_string() method, and it becomes more clear this is a
constant.

Change-Id: I0e7577dd7a6e503039037b986313b214e995d826
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7725
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: jrhahn <mail.jhahn@gmail.com>
Tested-by: BuildkiteCI
2023-01-03 13:03:07 +00:00
Florian Klink
2df642231c feat(tvix/store/nixpath): expose digest and name fields
These can be accessed directly.

Change-Id: I71dc84f982820d53f319efefbed9b9055034954d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7724
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-01-03 13:03:07 +00:00
Florian Klink
0b56d9f21b feat(src/proto): add PathInfo.validate()
This provides validation of PathInfo messages, and ensures the output
hashes are properly parsed from the root node names.

NixPath already has a more extensive test suite for various wrong
NixPaths, so it's omitted from here.

Change-Id: I5d69118df5816daabb521ddb19d178bddd1caacf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7684
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-01-03 13:03:07 +00:00
Florian Klink
ceb2c0ba89 chore(tvix/store): make importable
This allows other crates to import tvix_store.

Rename the bin crate to tvix-store-bin, to avoid having multiple crates
with the same name (https://github.com/rust-lang/cargo/issues/6313)

Change-Id: I857768d6115640dbf102e79ed03e8474090df2fe
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7728
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-01-03 13:03:07 +00:00
Florian Klink
d0bbc8c821 chore(tvix/store/nixbase32): address clippy
Change-Id: Ib47a55e39ed752492b9732439de5f327a7fa601e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7723
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
2023-01-03 12:06:55 +00:00
Florian Klink
ecd943ea6a chore(tvix/store/main): address clippy
Change-Id: Ie400a7109d3aa6b66153c0ab25a38abf06984cbf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7722
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-01-03 12:06:54 +00:00
Vincent Ambo
90c32eec7a feat(tvix/serde): initial Nix->serde::Deserialize impl
This will make it possible fairly easily use Nix to represent
arbitrary data structures, e.g. for using Nix as a config language.

Only pure Nix (i.e. no `import` etc.) is supported for now.

Not all types, specifically no struct traversal, are implemented in
this commit.

Change-Id: I9ac91a229a0d12bf818e6e3249f3e5a691599a2c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7712
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2023-01-02 22:24:43 +00:00
Vincent Ambo
49ee3e3b14 chore(tvix/eval): implement std::error::Error for tvix_eval::Error
This makes it easier to interface this error with other crates.

Change-Id: I4947ea6097608f8c0427fb94a819ef748d94ea4b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7711
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2023-01-02 22:24:43 +00:00
Jürgen Hahn
31973890a9 refactor(tvix/derivation): refactor the derivation serialization
This refactors the code to serialize a derivation. The original code
has beed moved to seperate crates for better code structure.

Change-Id: I3b1a6b134428fcbc9930c330bced8ec3610cfb4c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7733
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2023-01-02 20:55:14 +00:00
Florian Klink
319c03f634 feat(tvix/store): add logging with tracing
This uses [tracing](https://github.com/tokio-rs/tracing) for logs/
tracing.

Annotate all method handlers with an instrument macro, and warn! a
message for them being unimplemented.

Co-Authored-By: Márton Boros <martonboros@gmail.com>
Change-Id: Id42a41db33782d82abfb8dc0e49a8915000e5d89
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7665
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-12-30 20:25:09 +00:00
Florian Klink
0bf2b0ef11 feat(tvix/store): implement reflection
This implements grpc.reflection.v1alpha.ServerReflection, and will make tools
like evans automatically discover available services, without having to
specify the path to the .proto files client-side.

It's behind a reflection feature flag, which is enabled by default.

Change-Id: Icbcb5eb05ceede5b9952e38a2ba72eaa6fa8a437
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7435
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-12-30 20:25:09 +00:00
edef
51243007f6 refactor(tvix/store): make nixbase32 reversal more idiomatic
Change-Id: Ibe550f9573b0f45ee95453b50c7510e49b07c719
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7685
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-12-30 16:40:17 +00:00
Jürgen Hahn
60bdf619a3 feat(tvix/store): refactor digest conversion
This refactors how the original digest type (Vec<u8>) is converted
to [u8; PATH_HASH_SIZE].

Change-Id: I9441470a3a199620fcf328f2b7c890ca6ae93bde
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7710
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2022-12-30 16:02:38 +00:00
Florian Klink
58f5ff2c17 feat(tvix/store): initial dummy implementation
This replaces the hello world example from tvix-store with an actual
gRPC endpoint, implementing all of BlobService, DirectoryService and
PathInfoService.

All RPC methods currently respond with the unimplemented gRPC status.

Co-Authored-By: Márton Boros <martonboros@gmail.com>
Change-Id: Ieba333cca44dc1e3f2ffbe676ba7a99e672b9bfb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7664
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-12-30 15:21:38 +00:00
Florian Klink
d22a9c8610 chore(tvix/store/protos): more idiomatic go
Pointed out by edef in
https://cl.tvl.fyi/c/depot/+/7648/comment/4551ba4b_e89ade36/#, thanks!

Change-Id: I6b0d317bb0210521622483cdf4cb557bc637a100
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7709
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
2022-12-30 15:18:02 +00:00
Florian Klink
4036740ca7 feat(tvix/store): parameterise validate_digest error
Similar to cl/7682, we also want to make that error configurable.

Change-Id: I64f1a4570b3d75af4741abe10c2855959766e107
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7708
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-12-30 14:43:19 +00:00
Jürgen Hahn
5520bf3682 feat(tvix/store): add nixpath
This implements the NixPath structure. NixPath allow to parse a string to a nix path. If the parsing fails, a DecodeError will be raised.

Change-Id: I28363cdcfb27f04bf21a11c0d130b461667e3720
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7706
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-12-30 14:39:59 +00:00
Florian Klink
357c4d4836 feat(tvix/store): add nixbase32 mod
This implements the nix-specific base32 encoding and decoding, exposing
a subset of the API that the data-encoding crate provides.

Nix uses a custom alphabet, no padding, and encodes bytes in reverse
order. The latter one is the reason we can't just use the data-encoding
crate directly.

Three odd corner case tests ported over from go-nix failed. We opened
b/235 to further investigate.

Change-Id: I73fab6ddd67177d882e4c3f2b48761c95853d558
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7683
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2022-12-30 13:31:46 +00:00
Florian Klink
5ba47a2bc3 feat(tvix/store): parameterise validate_node_name error
We'll be using validate_node_name in other places in a bit, where
returning a ValidateDirectoryError is not appropriate.

Accept a function mapping a string to error as a second argument, and
pass ValidateDirectoryError::InvalidName at the current call sites.

Change-Id: I45cbb0deb4763061ad912c6b18a112c727795a17
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7682
Tested-by: BuildkiteCI
Reviewed-by: jrhahn <mail.jhahn@gmail.com>
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
2022-12-29 21:55:45 +00:00
Florian Klink
e97a429e73 fix(tvix/store): fix typo
Change-Id: I351a2bd4e007443aef1b97f50aecffc095ac60b1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7681
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: jrhahn <mail.jhahn@gmail.com>
Reviewed-by: tazjin <tazjin@tvl.su>
2022-12-29 21:51:43 +00:00
Vincent Ambo
86361f0f4a refactor(tvix/eval): remove extra Rc<..> around Value::Attrs
The `im::OrdMap` is already small and cheap to copy while sharing
memory, so this is not required anymore.

Only the `KV` variant may have slightly larger content, but in
practice this doesn't seem to make a difference when comparing the two
variants and this one is less complicated.

Change-Id: I64a563b209a2444125653777551373cb2989ca7d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7677
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-12-29 17:44:56 +00:00
Vincent Ambo
91465dc78e refactor(tvix/eval): persistent, memory-sharing OrdMap for NixAttrs
This uses the `im::OrdMap` for `NixAttrs` to enable sharing of memory
between different iterations of a map.

This slightly speeds up eval, but not significantly. Future work might
include benchmarking whether using a `HashMap` and only ordering in
cases where order is actually required would help.

This switches to a fork of `im` that fixes some bugs with its OrdMap
implementation.

Change-Id: I2f6a5ff471b6d508c1e8a98b13f889f49c0d9537
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7676
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-12-29 17:44:56 +00:00
Vincent Ambo
610c44ec1e refactor(tvix/eval): use im::Vector directly where possible
The conversion from im::Vector -> Vec is cheaper for NixList
construction (of course), so where possible we should make use of
that.

This updates most builtins dealing with lists to use Vector directly,
and marks the function constructing NixList from Vec as deprecated so
that we get appropriate warnings in places where it's still in use.

These places are currently inside of JSON serialisation logic which is
in flux right now, so lets leave them as-is until it's stabilised.

Change-Id: I037f12a2800f2576db4d9526bd935efd079163f0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7671
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-12-29 16:33:14 +00:00
Jürgen Hahn
bb185b2c6e feat(tvix/derivation): serialize Nix Derivation
This adds a Derivation structure and allows to write it to a structure that implements std::fmt:Write.
The implementation is based on the go-nix version.

Change-Id: Ib54e1202b5c67f5d206b21bc109a751e971064cf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7659
Reviewed-by: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-12-29 14:38:45 +00:00
Vincent Ambo
5d73c06b1a refactor(tvix/eval): use im::Vector for NixList representation
This is a persistent, structurally sharing data structure which is
more efficient in some of our use-cases. I have verified the
efficiency improvement using `hyperfine` repeatedly over expressions
on nixpkgs.

Lists are not the most performance-critical structure in Nix (that
would be attribute sets), but we can already see a small (~5-10%)
improvement.

Note that there are a handful of cases where we still go via `Vec`
that need to be fixed, most notable for `builtins.sort` which can not
currently be implemented directly using `im::Vector` because of a
restrictive type bound.

Change-Id: I237cc50cbd7629a046e5a5e4601fbb40355e551d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7670
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-12-29 12:27:59 +00:00
Florian Klink
6ab8320f07 chore(tvix/store): address clippy warnings
The only warnings left are unused warnings, but that'll change once we
have a real implementation, and not just tests.

Change-Id: I28912281b5e66735be37e999cc8ef4b8b09028fb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7669
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-12-29 12:07:57 +00:00
Florian Klink
b0879917d4 chore(tvix/store): trim Cargo.toml
lazy_static is only used in tests, and anyhow isn't used at all (yet).

This can be dropped.

Change-Id: Ic41ff3f9bb93cfa600c3485e85464f78a3976504
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7668
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-12-29 12:07:57 +00:00
Florian Klink
ff71366f41 chore(tvix/store): move tests into separate file
Move them from the bottom of src/proto.rs to its own src/tests/mod.rs.

Also drop the test_ prefix, this is not golang.

Change-Id: I2e0b6b9812264f3d9721c0766936f08157fadc66
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7667
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-12-29 12:07:57 +00:00
sterni
56555b211e docs(tvix/eval): sketch in place list/attr set update idea
Change-Id: Ic7debbd8cbd3acdf5f3947288f2aa2964bd163a0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7660
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-12-29 09:52:17 +00:00
Florian Klink
c4ee942b1c feat(tvix/store/protos): rename Get to Read, add Stat method
Stat exposes metadata about a given blob,
such as more granular chunking, baos.
It implicitly allows checking for existence too, as asking this for a
non-existing Blob will return a Status::not_found grpc error.

The previous version returned a Status::not_found error on the Get
request too, but there was no chance to prevent the server from starting
to stream (except sending an immediate cancellation).

Being able to check whether something exists in a BlobStore helps to
prevent from uploading in first place.

The granular chunking bits are an optional optimization - if the
BlobStore implements no more granular chunking, the Stat response can
simply contain a single chunk.

Read returns a stream of BlobChunk, which is just a stream of bytes -
not necessarily using the chunking that's returned in the reply of a
Stat() call. It can be used to read blobs or chunks.

Change-Id: I4b6030ef184ace5484c84ca273b49d710433731d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7652
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-12-28 13:55:18 +00:00
Florian Klink
1c15154b83 feat(tvix/store): make blobstore stream chunks
This changes the RPC methods to return/consume a stream of chunks, instead of a
very big message containing the whole blob, to keep message sizes in manageable
sizes (less than 4MiB).

Change-Id: I2a3a50f07b059d8a2f5196860254adff98c8a352
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7651
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-12-28 13:55:18 +00:00
Florian Klink
f879993cc4 feat(tvix): add evans to shell
Change-Id: I7ec8bae236330ec1e38e82da67b47aef2815d4b6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7437
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-12-28 13:55:18 +00:00
Florian Klink
d973c9772d chore(tvix/store/protos): add PathInfoService::CalculateNAR()
Expose the NAR calculation to a separate `CalculateNAR` method, which
responds with the NAR size and sha256 hash.

Contrary to what cl/7618 and cl/7620 initially did, don't add different
other request types.

In the CalculateNARResponse message, there's now some duplication in the
(optional) `narinfo` field of a PathInfo, but I'm not entirely sure if
we want to drop the fields from there yet.

Change-Id: Id797c56e17efedac115fbd43de9dfde9fa1db140
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7663
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-12-28 13:55:18 +00:00
Florian Klink
ce7be00994 chore(tvix/store/protos): drop non-sha256 NAR hashes
Nix actually doesn't support anything else than sha256 NAR hashes, so
there's little reason to communicate anything else in here.

Co-Authored-By: edef <edef@unfathomable.blue>
Change-Id: I760370bd6cabd02028e001a74c454ef9296eb600
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7619
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-12-28 13:55:18 +00:00
Florian Klink
27bfc8cfcd chore(tvix/store/protos): move (Root)Node into separate message
This is gonna get used in another place in a second.

Co-Authored-By: edef <edef@unfathomable.blue>
Change-Id: I347c11c8d24379628b7ed09d2c90670c576e686a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7617
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-12-28 13:55:18 +00:00
Florian Klink
7163d3ad37 feat(tvix/store): implement Directory::validate()
Change-Id: I4c6ae79d705b8e19a3e2ed54812366e88935d7a6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7650
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-12-28 10:58:50 +00:00
Florian Klink
0c7e545fd0 chore(tvix/store): use DUMMY_DIGEST, too
Missed from cl/7649.

Change-Id: Id2d382567f91cba9cdd792fe44bc0a83d040ec69
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7656
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-12-28 10:58:50 +00:00
Florian Klink
f930c91086 chore(tvix/store): use Directory::default in some more places
Change-Id: Iebad5e0b38f9506933fa855e1bfb2611d27ff572
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7655
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-12-28 10:58:50 +00:00
Florian Klink
35b18b2fdb chore(tvix/store): use lazy_static and dummy digests
While there's currently nothing in here checking the size of the digest,
we should use something that passes the to-be-introduced validate()
function.

Change-Id: I0c515d9e3afc79292dedebce659a32485aa3d936
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7649
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-12-27 21:31:33 +00:00
Florian Klink
dfd9286f68 feat(tvix/store/protos): implement Directory.Validate()
Validate thecks the Directory message for invalid data, such as:
- violations of name restrictions
- invalid digest lengths
- not properly sorted lists
- duplicate names in the three lists

Change-Id: I8d43a13797793c64097e526ef3bd482c9606c87b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7648
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-12-27 21:31:33 +00:00
Florian Klink
c3fb6d2218 feat(tvix/store/protos): implement Size() and Digest() for Directory
This adds Size() and Digest() functions for the golang version.

Change-Id: If71445a9bb26100bb4076ac4f5c96945b33919f9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7325
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-12-27 21:31:33 +00:00
Florian Klink
a91e6334ff feat(txix/store/protos): init go.mod, generate .pb.go files
This allows importing the generated .pb.go files into other go projects.

I initially looked at buildGo.protos, but it doesn't work for multi-.proto
files, and actually having LSP support for the generated structs is nice, too.

Change-Id: Idbd448008010790a10a0ea42e4059dbb609eaf1a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7322
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-12-26 11:38:52 +00:00
Florian Klink
2b9330911a feat(tvix/store/protos): add go_package option
Change-Id: I0898b8a0a78e704219da38e5acaabef1e640d4e4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7321
Reviewed-by: Adam Joseph <adam@westernsemico.com>
Tested-by: BuildkiteCI
2022-12-26 11:38:52 +00:00
Florian Klink
e05e8bdd03 docs(tvix/store): add README, document services and store model
These are intended to help digest the protocol definitions for tvix-
store, and how they tie into the whole concept.

Co-Authored-By: Vincent Ambo <mail@tazj.in>
Change-Id: Ic1ba3ba41ef599209453f15d0ac2e07a6144bcca
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7439
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-12-26 11:38:52 +00:00
Florian Klink
3676ce8d6d feat(tvix/store): use tonic-build for protos
While prost-build already exposes protobuf message types as structs, we
actually need tonic-build too, to be able to get traits for all the RPC
services defined in the proto files.

Change-Id: I7f4c08454bf0d280d577975c7cdae13ccc2d933b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7320
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-12-26 11:38:52 +00:00
Vincent Ambo
71174f6626 fix(tvix/eval): fix current clippy warnings
It's been a while since the last time, so quite a lot of stuff has
accumulated here.

Change-Id: I0762827c197b30a917ff470fd8ae8f220f6ba247
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7597
Reviewed-by: grfn <grfn@gws.fyi>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-12-25 18:25:06 +00:00
Adam Joseph
67d508f2ec refactor(tvix/eval): non-recursive thunk forcing
Introduces continuation-passing-based trampolining of thunk forcing to
avoid recursing when forcing deeply nested expressions.

This is required for evaluating large expressions.

This change was extracted out of cl/7362.

Co-authored-by: Vincent Ambo <tazjin@tvl.su>
Co-authored-by: Griffin Smith <grfn@gws.fyi>
Change-Id: Ifc1747e712663684b2fff53095de62b8459a47f3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7551
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-12-25 18:17:46 +00:00
Ryan Lahfa
4cda236c0c feat(tvix/eval): implement From<f64> for Value
Change-Id: I287282a195d6f752260242739332b2357791974a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7625
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-12-25 15:47:40 +00:00