Commit graph

1979 commits

Author SHA1 Message Date
Florian Klink
ddae4860c2 feat(tvix/castore/import): generalize ingest_path
We don't actually care if it's an Arc<dyn BlobService>, or something
else, as long as we can Deref to a BlobService and clone.

Change-Id: I0852aaf723f51c5e6b820be8db1199d17309ab08
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10510
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2024-01-01 00:54:14 +00:00
Florian Klink
fdd7817aad feat(nix-compat/store_path): build_ca_path may fail
Change-Id: Ia74ee870f38b7966501458bace541092256c3213
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10509
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2024-01-01 00:52:41 +00:00
Florian Klink
20c98a51bf refactor(tvix/store/bin): don't reconstruct StorePath twice
We already have that data in output_path.

Change-Id: Iadf615551f0b44d539098446ff5926994deb606d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10508
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-01-01 00:42:38 +00:00
Florian Klink
fa335aaa68 fix(tvix/store/bin): don't unwrap in case of invalid paths
Instead, return an error, and move the entire check before starting to
ingest the data underneath.

Change-Id: Idcfba115cb7d599f5fc72a156aaad9d4d4714fcf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10507
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
2023-12-31 22:23:49 +00:00
Florian Klink
694ed7ea1a refactor(tvix/store/bin): condense subscriber setup a bit
We can use cli.json.then_some(…) to create a Some(…), allowing us to
omit the else { None } lines.

Change-Id: I6c8142a08d8cb88d6c8302e5ca7570698fcf2aa3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10505
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
2023-12-31 22:23:49 +00:00
Florian Klink
f6d1a56c8c refactor(tvix/store): move construct_services helper here
This takes three URLs, and constructs Arc'ed
{Blob,Directory,PathInfo}Service, allowing to remove some of the
boilerplate.

Change-Id: I40e7c2b551442ef2acdc543dfc87ab97e7c742bb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10484
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-12-31 22:23:49 +00:00
Florian Klink
41935fab70 refactor(tvix/castore/directorysvc): return Box, not Arc
While we currently mostly use it in an Arc, as we need to clone it
inside PathInfoService, there might be other usecases not requiring it
to be Clone.

Change-Id: Ia05bb370340792a048e2036be30e285ef1e63870
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10483
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
2023-12-31 22:18:14 +00:00
Florian Klink
9ca1353122 refactor(tvix/castore/blobsvc): return Box, not Arc
While we currently mostly use it in an Arc, as we need to clone it
inside PathInfoService, there might be other usecases not requiring it
to be Clone.

Change-Id: I7bd337cd2e4c2d4154b385461eefa62c9b78345d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10482
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
2023-12-31 22:18:14 +00:00
Florian Klink
e2b6c77bfc feat(tvix/cli): allow configuring different services
At some point, tvix-cli needs to talk to the outside world to persist
things into a real store.

Introduce the same CLI options to configure
{Blob,Directory,PathInfo}Service URLs.

We need to be a bit careful with how we set up stores, and make
this separate from setting up TvixStoreIO, as it's holding a
Rc<RefCell<KnonPath>> which not Send.

At some point, we might make this a Arc<RwLock<_>> later anyways, and
then this can be simplified a bit, but for now, this is sufficient.

Change-Id: I87d84ca3a10ce947e194ff985073791469773f35
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10474
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
2023-12-31 21:55:18 +00:00
Florian Klink
12f9b95a2c feat(tvix/eval): accept impl AsRef<str> for code
We're also happy to consume strings, or other owned stringy types.

Change-Id: I5bead4407976134815d8f879f9f70468e6af1dc4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10476
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
2023-12-31 13:15:24 +00:00
Florian Klink
4fba57c2c9 refactor(tvix/eval): remove code and location from struct
Instead, it's passed in the evaluate/compile_only functions, which feels
more naturally. It lets us set up the Evaluation struct long before
we actually feed it with data to evaluate.

Now that Evaluation::new() would be accepting an empty list of
arguments, we can simply implement Default, making things a bit more
idiomatic.

Change-Id: I4369658634909a0c504fdffa18242a130daa0239
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10475
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
2023-12-31 13:15:24 +00:00
Florian Klink
4d6137964b feat(tvix/build): don't force outputs to be in store_dir
There's no need to require builds to also put their outputs in store_dir.

So rename store_dir to inputs_dir and make outputs consume paths, not
basenames.

Also document the possibility for builds to write to inputs_dir. Let's
see if we want to make this a constraint later.

Change-Id: Ib4df20afcdde2d771c269c422f04c7e95587cd0f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10447
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: flokli <flokli@flokli.de>
2023-12-29 22:14:08 +00:00
Ryan Lahfa
3e63d7be42 feat(tvix/eval): context-aware casting to strings
By default, we don't want contextful strings and we almost always want contextless strings.

To this end, we make taking a contextful string a very explicit operation under `to_contextful_str`
and we implement manually the `to_str` cast which requires a `if !s.has_context()` guard that
the macro cannot cover.

Change-Id: I7aae8e57a7d73e547e62b1edb0b1cc7e8c0c69b6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10425
Autosubmit: raitobezarius <tvl@lahfa.xyz>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-12-29 22:00:04 +00:00
Ryan Lahfa
343ee0de57 feat(tvix/eval): implement hasContext primop
`hasContext` is now functional.

Change-Id: I23b128afc9150b833bc0d9b042d31fee35badadb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10422
Tested-by: BuildkiteCI
Autosubmit: raitobezarius <tvl@lahfa.xyz>
Reviewed-by: tazjin <tazjin@tvl.su>
2023-12-29 21:58:32 +00:00
Ryan Lahfa
5dfb15d2c8 feat(tvix/eval): introduce NixContext
This prepares the data structures to implement string contexts
in Nix.

Change-Id: Idd913c9c881daeb8d446907f4b940e462e730978
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10420
Autosubmit: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-12-29 21:57:30 +00:00
Ryan Lahfa
fc182061c6 docs(tvix/eval): why context strings now
Explain why we had to take the bullet on context strings now
and move the historical approach in a historical section.

Change-Id: Ie3bcc2213b391c6ba06547cc05c850891a41d06b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10446
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
2023-12-29 21:55:58 +00:00
Adam Joseph
4b3f27ebc9 fix(tvix/eval): propagate catchables through builtins.attrNames
Change-Id: Id14e39543239272aed041998fd9a78465c9cb8b2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10359
Autosubmit: Adam Joseph <adam@westernsemico.com>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2023-12-29 21:34:45 +00:00
Adam Joseph
9cb3cd10f5 fix(tvix/eval): propagate catchables through builtins.intersectAttrs
Change-Id: I4ada8cf10611e98519cb1b1da11bfd06815f2932
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10358
Autosubmit: Adam Joseph <adam@westernsemico.com>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2023-12-29 21:34:45 +00:00
Adam Joseph
2883b2d4bd fix(tvix/eval): propagate catchables in string interpolations
Change-Id: I13d7ce0c7328a8e6fbc6d2c4ff5c4fe6095b96ea
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10357
Autosubmit: Adam Joseph <adam@westernsemico.com>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2023-12-29 21:34:45 +00:00
Adam Joseph
7ddea7340f fix(tvix/eval): catchable in type field of nix_eq()
Change-Id: I165ff77764e272cc94d18cb03ad6cbc9a8ebefde
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10348
Autosubmit: Adam Joseph <adam@westernsemico.com>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2023-12-29 21:34:45 +00:00
Adam Joseph
2af8174e2e fix(tvix/eval): builtins.match: propagate catchables
Change-Id: I14c9e625c91369e10d0c00380dca992811ae9059
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10346
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: Adam Joseph <adam@westernsemico.com>
Tested-by: BuildkiteCI
2023-12-29 21:34:45 +00:00
Adam Joseph
11e35a77a6 refactor(tvix/eval): let OpCoerceToString select the CoercionKind
Change-Id: I92d58ef216d7e0766af70f019b3dcd445284a95d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10344
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-12-29 21:34:45 +00:00
Adam Joseph
0c22454bb9 fix(tvix/eval): add stack depth assertion to OpReturn
I'm still trying to work out the exact stack invariants for tvix.
We really should add assertions for them; getting the stack messed
up is no fun.  This commit adds one simple assertion.  It also adds
a missing stack-push (my mistake) in one place, which was uncovered
by the assertion.

Change-Id: I9d8b4bd1702d954e325832c5935b0d7e3eb68422
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10369
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-12-29 21:34:45 +00:00
Florian Klink
8a52c7f1c5 feat(tvix/castore/fs): borrow some matches
We only do things with the reference, so we don't need to locally borrow
it.

Change-Id: I6073f7ec7aff717ae3069e28a00b1cb408a50ceb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10455
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-12-29 17:18:14 +00:00
Florian Klink
5e3683cd53 feat(tvix/shell): add grpcurl to shell
This is a bit more useful than evans when sending over the same message
over and over again.

Change-Id: I600b6b9f591c0c963c5d270005aa1cc84d2a0770
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10453
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: edef <edef@edef.eu>
Tested-by: BuildkiteCI
2023-12-29 15:57:26 +00:00
Florian Klink
22427dc8d0 refactor(tvix/castore/bin/tvix-store) don't import std::io
Make it more clear this is not tokio::io.

Change-Id: Ic2fa56f0baf1c200b6631098d556388a19629a45
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10452
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-12-29 15:57:26 +00:00
Florian Klink
46a372d5d7 feat(tvix/castore/fs): instrument FuseDaemon functions
Change-Id: I696b7ab6b4c08004db147c0fda7312bbebaa0eec
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10451
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
2023-12-29 15:57:26 +00:00
Florian Klink
0af445c2f4 feat(tvix/store): emit more structured log messages
Change-Id: If4748cb800276fc42779ba8973d1eef72df0503e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10450
Reviewed-by: edef <edef@edef.eu>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-12-29 14:37:58 +00:00
Florian Klink
f6c94430c8 feat(tvix/build/protos): add some missing fields
- directory in which the castore input nodes are mounted
 - working directory for the build command
 - scratch paths
 - network access y/n
 - whether a (static) /bin/sh should be provided

Populate these fields appropriately, and extend the tests in tvix-glue
with a FOD example.

Change-Id: I4f9de1483d6696d74694a09784910c407acb0be0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10412
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2023-12-26 10:20:41 +00:00
sterni
d07600dbca fix(tvix/eval/value): correctly emit spaces when coercing lists
r/7176 introduced an incorrect assumption was the benefit of the
nonrecursive coercion algorithm, namely that a coercion operation always
returns a non empty string. This allows to detect whether we are
coercing a list or not by checking if the intermediate result is empty
or not. Unfortunately, coercing null and false yields an empty string,
so we need to explicitly track whether we are coercing a list.

Updated the test case to hopefully catch similar bugs in the future. I'm
not a hundred percent certain I have not introduced a new edge case with
this, so it may be interesting to add a prop test case for this to
nix_oracle down the line. At least lists are the only nested data
structures that can be serialized as nested data structures, so the
problem is kind of limited.

Change-Id: Ia41e904356f1c41a9d35e4e65ec02f2fe5a4100e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10418
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2023-12-25 15:06:23 +00:00
Florian Klink
d4dc28b675 chore(tvix/turbofetch): switch to futures 0.3.30
The bugs have been fixed,
https://github.com/rust-lang/futures-rs/pull/2801 and
https://github.com/rust-lang/futures-rs/pull/2812 were merged and ended
up in that release.

Change-Id: I301c0ffc951f04a5b3b7267e922771c837a3f5a9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10415
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-12-24 21:45:04 +00:00
Florian Klink
acbb613e61 chore(tvix): switch to upstream futures 0.3.30
The bugs have been fixed,
https://github.com/rust-lang/futures-rs/pull/2801 and
https://github.com/rust-lang/futures-rs/pull/2812 were merged and ended
up in that release.

Change-Id: Iefd990d2d1719b884504093343e54e9c5258e2e2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10414
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-12-24 21:45:04 +00:00
Florian Klink
a5865ec7fa refactor(tvix/castore/fs/tests): drop unused args
There's no need to pass in an unused directory service into the
populate_blob_* method, and considering we have one or two invocation of
each of these, we don't really gain much from having all these functions
follow the same structure, at least for now.

Also, update some function names to better describe what they're doing.

Change-Id: I92f680745c157fb0a602b07342f8838bfad23ecd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10411
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
2023-12-24 16:05:52 +00:00
Florian Klink
8d86d2f409 refactor(tvix/castore): add RootNode impl for BTreeMap, mv fs tests
cl/10378 did already move store/fs to castore/fs, but we kept the tests
in tvix-store, as they were populating a PathInfoService to make nodes
appear in the mount root.

Update these tests to now just insert root nodes into a BTreeMap, and
ensure we can use that as a RootNodes too.

Change-Id: Iad7d1ee4f9423eb6e3a1da33f433842c9ae0de1f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10410
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
2023-12-24 15:44:30 +00:00
Aspen Smith
6af17545bf feat(tvix/website): Link to windtunnel results page
Change-Id: If4456c565abf32385b1239467b96e7398190ea4b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10383
Reviewed-by: flokli <flokli@flokli.de>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2023-12-23 17:12:25 +00:00
Florian Klink
a5167c508c chore(tvix): move store/fs to castore/fs
With the recent introduction of the RootNodes trait, there's nothing in
the fs module pulling in tvix-store dependencies, so it can live in
tvix-castore.

This allows other crates to make use of TvixStoreFS, without having to
pull in tvix-store.

For example, a tvix-build using a fuse mountpoint at /nix/store doesn't
need a PathInfoService to hold the root nodes that should be present,
but just a list.

tvix-store now has a pathinfoservice/fs module, which contains the
necessary glue logic to implement the RootNodes trait for a
PathInfoService.

To satisfy Rust orphan rules for trait implementations, we had to add a
small wrapper struct. It's mostly hidden away by the make_fs helper
function returning a TvixStoreFs.

It can't be entirely private, as its still leaking into the concrete
type of TvixStoreFS.

tvix-store still has `fuse` and `virtiofs` features, but they now simply
enable these features in the `tvix-castore` crate they depend on.

The tests for the fuse functionality stay in tvix-store for now, as
they populate the root nodes through a PathInfoService.

Once above mentioned "list of root nodes" implementation exists, we
might want to shuffle this around one more time.

Fixes b/341.

Change-Id: I989f664827a5a361b23b34368d242d10c157c756
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10378
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2023-12-22 16:55:18 +00:00
Florian Klink
52cad86195 refactor(tvix/store): remove Arc<> from PathInfoService::from_addr
This makes PathInfoService::from_addr return a Box<dyn PathInfoService>,
rather than an Arc<dyn …>, and leaves it up to the consumers to rewrap
it into an Arc where needed.

This allows us to drop the Arc for the tvix-store daemon subcommand.

Change-Id: Ic83aa2ade6c51912281bd17c7eef7252e152b2d1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10409
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2023-12-22 16:55:18 +00:00
Florian Klink
93a228b9a4 refactor(tvix/store/bin): move service.clone right before async move
Change-Id: I7a2b951d9c9251a053a0de40f31836bda03a922d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10408
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: sterni <sternenseemann@systemli.org>
2023-12-22 16:55:11 +00:00
Florian Klink
39cddb95be refactor(tvix/store/bin): remove unneeded clone
Change-Id: Ib94376db3a57853d980112919c2d6fcc0f566883
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10407
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: flokli <flokli@flokli.de>
2023-12-22 13:29:04 +00:00
Florian Klink
9627ef15de docs(tvix/castore/protos): remove reference
This is not gonna end up as a interlinked docstring.

Change-Id: I2b0ca106aa75bae0156c0b411da5931da60c725d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10406
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: edef <edef@edef.eu>
Tested-by: BuildkiteCI
2023-12-21 16:44:48 +00:00
Florian Klink
3b274f47af docs(nix-compat/nar/reader): fix reference
Change-Id: I7ccdfe3c042f49aa962d1c7ba6337bd656ef05c8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10405
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: edef <edef@edef.eu>
2023-12-21 16:44:16 +00:00
Florian Klink
12406fe241 docs(nix-compat/derivation): fix docstring
Change-Id: I76200aa474b86cf655308f6710a2298b77fa4b38
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10404
Reviewed-by: edef <edef@edef.eu>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-12-21 16:44:16 +00:00
Florian Klink
b04fdeaa10 docs(nix-compat/derivation): fix reference
Change-Id: I0ebe256c62435901c12ca1895f30e380280e9ea8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10403
Reviewed-by: edef <edef@edef.eu>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-12-21 16:44:16 +00:00
Florian Klink
410abd67f8 docs(nix-compat/aterm/parser): fix docstring
Change-Id: I7a65ce320163c5898df402af3a639f6cee2e17c6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10402
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: edef <edef@edef.eu>
2023-12-21 16:43:44 +00:00
Florian Klink
783a1e314c docs(tvix/castore): fix reference
Change-Id: I00b1d56d58c4d3779b57ab0056cff1c7e6053b9b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10401
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: edef <edef@edef.eu>
Tested-by: BuildkiteCI
2023-12-21 16:43:44 +00:00
Florian Klink
3fc4d295d5 docs(tvix/store): fix reference
Change-Id: If1970cb8eb9a21c30011c9303f2fcbff646ad12b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10400
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: edef <edef@edef.eu>
2023-12-21 16:43:44 +00:00
Florian Klink
5bee6c6be0 docs(tvix/store): fix reference
Change-Id: Ifb77097755f25aa6536c60365941507a592cac3e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10399
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: edef <edef@edef.eu>
2023-12-21 16:43:11 +00:00
Florian Klink
6838dc3fff docs(tvix/glue): fix docstring
This is not an unclosed <html> tag.

Change-Id: I2bd2426fc600de2d96dbab47743f1c7bd5fed35e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10398
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: edef <edef@edef.eu>
2023-12-21 16:43:11 +00:00
Florian Klink
ca0a8f5603 docs(nix-compat/nixhash): fix reference
Change-Id: I95c35a1405601c6f79cbc03462ab7c7cd6f08f2a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10397
Tested-by: BuildkiteCI
Reviewed-by: edef <edef@edef.eu>
Autosubmit: flokli <flokli@flokli.de>
2023-12-21 16:43:11 +00:00
Florian Klink
1c3ea9d83d docs(nix-compat/drvfmt): fix reference
Change-Id: Ic0403e028cc968bef4023ef78dbfa0d502da90b0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10396
Reviewed-by: edef <edef@edef.eu>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-12-21 16:39:07 +00:00
Florian Klink
8c83838c40 docs(nix-compat/narinfo): fix reference
Change-Id: Ib3a5414afed738ba6178ec69d38a4169c9b24714
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10395
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: edef <edef@edef.eu>
2023-12-21 16:36:34 +00:00
Florian Klink
18702c740e docs(nix-compat/narinfo/fingerprint): fix reference
Change-Id: Icc56aa178247b3985a406421b7d8d449c63f8231
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10394
Tested-by: BuildkiteCI
Reviewed-by: edef <edef@edef.eu>
Autosubmit: flokli <flokli@flokli.de>
2023-12-21 16:36:34 +00:00
Florian Klink
e4eb0b9bf4 docs(nix-compat/derivation): fix references
Change-Id: If6962931350edfbae206fa739a673e7ccbc3ee89
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10393
Reviewed-by: edef <edef@edef.eu>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-12-21 16:36:34 +00:00
Florian Klink
329a7d30a7 refactor(nix-compat/store_path): centralize self_reference check
self_reference being set to true is only allowed for
`CAHash::Nar(NixHash::Sha256(_))`, so we can handle this in a check at
the front.

Change-Id: Ic363ade4789a7767cbe26a6959b143bb53e50e5a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10391
Reviewed-by: edef <edef@edef.eu>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-12-21 16:36:29 +00:00
Florian Klink
88adaea12b refactor(nix-compat/store_path/utils): restructure build_ca_path
All match cases essentially construct `ty` and `hash`, which is then
passed to the `build_store_path_from_fingerprint_parts` function.

Change-Id: I01dfd219f9b0ac1afe8af7c6e361ea048117a0e6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10390
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: edef <edef@edef.eu>
2023-12-21 16:28:53 +00:00
Florian Klink
9065089b0c refactor(tvix/nix-compat): have helpers interact with StorePathRef
In most case, we don't actually need an owned `StorePath` struct, but a
`StorePathRef<'_>` is sufficient. The lifetime is only due to it holding
onto the name, but that one is mostly left untouched.

`Derivation::calculate_derivation_path` still needs to return
`StorePath`, as its name has a `.drv` appended.

Change-Id: Ie0d52f369d785711bb0658ea2b0bd2617fd9f45e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10389
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: edef <edef@edef.eu>
2023-12-21 16:28:53 +00:00
Florian Klink
ea8fdb9304 refactor(nix-compat/narinfo/fingerprint): move to StorePathRef
With the introduction of StorePathRef::to_absolute_path(), there's no
need to convert this StorePathRef to a StorePath first.

Change-Id: I634c977c4b63858e4f329fd21726e0611b99da4a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10388
Tested-by: BuildkiteCI
Reviewed-by: edef <edef@edef.eu>
Autosubmit: flokli <flokli@flokli.de>
2023-12-21 16:28:53 +00:00
Florian Klink
91deb67de9 refactor(nix-compat/store_path): use StorePathRef::to_absolute_path
Keep the method around in StorePath for convenience, but move the
implementation to StorePathRef.

Change-Id: Ie1844fa01ce6529dc1a58907563c95c3112c831d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10387
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: edef <edef@edef.eu>
2023-12-21 16:28:41 +00:00
Florian Klink
c7ea57df46 feat(nix-compat/store_path): derive [Partial]Eq
This allows comparing StorePathRef structs.

Change-Id: Ia69967ea9358052e2d6e76042a7e6d394f7f29a9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10386
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: edef <edef@edef.eu>
2023-12-21 16:19:46 +00:00
Luke Granger-Brown
e20848ecf1 chore(depot): update OWNERS files for aspen
Change-Id: Id94b646a6ea035782298c421d6667530da6fc5b6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10384
Tested-by: BuildkiteCI
Owners-Override: lukegb <lukegb@tvl.fyi>
Reviewed-by: lukegb <lukegb@tvl.fyi>
2023-12-20 18:35:58 +00:00
Aspen Smith
c236231910 chore(tvix): Run criterion benchmarks for the whole workspace
Run criterion benchmarks (via `cargo bench`) for the *whole tvix
workspace*, not just `//tvix/eval`, and report the results to
windtunnel.

Change-Id: I9235c3c166ed9121f35c5bb4c46d59dc1f4c4055
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10382
Tested-by: BuildkiteCI
Autosubmit: grfn <grfn@gws.fyi>
Reviewed-by: flokli <flokli@flokli.de>
2023-12-20 18:04:43 +00:00
Florian Klink
9c1358fe44 refactor(tvix/store): remove more clones on pathinfoservice
These are still leftovers from before the traits being async, where we
had to clone before moving into an async closure.

Change-Id: I1b3937edf61ce3e23bb07803306622c37a3572c6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10381
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Autosubmit: flokli <flokli@flokli.de>
2023-12-18 14:10:40 +00:00
Ryan Lahfa
0ae32d45f6 feat(tvix/castore): simple filesystem blob service
The simple filesystem `BlobService` enable a user to write blob store
on an existing filesystem using a prefix-style layout in the provided root directory,
e.g. the two first bytes of the blake3 hashes are used as directories prefixes.

Change-Id: I3451a688a6f39027b9c6517d853b95a87adb3a52
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10071
Autosubmit: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2023-12-17 14:34:13 +00:00
Florian Klink
923a5737e6 refactor(tvix/castore): drop is_closed() from impl DirectoryPutter
This is only used in the gRPC version (GRPCPutter), during the test
automation.

So define it as a method there, behind #[cfg(test)], and remove from
the trait.

Change-Id: Idf170884e3a10be0e96c75d946d9c431171e5e88
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10340
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-12-16 23:07:31 +00:00
Florian Klink
36cc7b7088 docs(tvix/store/fs): fix comment
There's nothing store-path specific here anymore, it's just a name in
the mountpoint root.

Change-Id: I0f8004491baa03ba560d390053a42678ee81154a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10377
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-12-16 21:53:31 +00:00
Florian Klink
7749814953 docs(tvix): refer to futures-rs PR
The PR has since been replaced with another version, using a bit of
unsafe, as rustc doesn't like the lifetimes.
We're still waiting on a backport of that to 0.3.

Change-Id: I8dd344e78162cd956f6e53ff05dc4c02763fbb04
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10376
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-12-16 21:47:01 +00:00
Florian Klink
43ab1543e2 chore(tvix/store): switch back to fuse-backend-rs upstream
The fix mentioned in Cargo.toml did land in
https://github.com/cloud-hypervisor/fuse-backend-rs/pull/162.

Change-Id: I4451f910806f05400f7e0f83581ae786f170b9e2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10375
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-12-16 21:46:30 +00:00
Florian Klink
97dee277b0 refactor(tvix/store/fs): no explicitly required Arc'ed Blob/DirSvc
Change-Id: Ie55026668cd4a6117e7b07174f5ac6638f93d194
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10374
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-12-16 21:46:30 +00:00
Connor Brewster
6815572274 refactor(tvix/store/fs): Decouple TvixStoreFs from Nix concepts
To support tvix builds, we need to be able to use the `TvixStoreFs` to
materialize the sandbox's /nix/store filesystem with just the list of
inputs needed for the build. Currently we'd need to seed an in-memory
`PathInfoService`, which includes more functionality than what is
required for `TvixStoreFs`. Additionally, the `PathInfoService` is
specific to Nix. By decoupling `TvixStoreFs` and `PathInfoService`,
we allow for usage of `TvixStoreFs` with `tvix-castore` without needing
a `PathInfoService`.

This introduces a new `RootNodes` trait which provides a way for the
filesystem to look up CA nodes via their basename in the root directory
of the filesystem. We then implement `RootNodes` for any
`PathInfoService`. Additionally, the filesystem root inode tracker now
stores basenames rather than `StorePath`s since `StorePath`s are
specific to Nix.

As a followup we can rename `TvixStoreFs` to `TvixCaStoreFs` and move
it to the `castore` crate (or its own crate).

b/341

Change-Id: I928372955017c23b1bf2b37190cbc508a4ed10d5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10363
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2023-12-14 21:46:25 +00:00
Aspen Smith
91b6c13f90 fix(tvix): Remove errant backslash in benchmark script
Change-Id: I485986521fd4a25770f2a398896a0f672628df9f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10365
Tested-by: BuildkiteCI
Autosubmit: grfn <grfn@gws.fyi>
Reviewed-by: grfn <grfn@gws.fyi>
2023-12-14 19:52:31 +00:00
Aspen Smith
1d678bab5f fix(tvix): Fix tvix nixpkgs eval benchmark
Fix the nixpkgs eval hyperfine benchmark in tvix/scripts.bench.sh to:

- Eval both a pinned (never-changing) nixpkgs commit and the nixpkgs
  commit used by the depot
- Not depend on fetchTarball, which is not yet implemented.
- Be called bench-windtunnel.sh

Change-Id: Id2ae18f983ab7327625320b5b16c082ecc369a49
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10364
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2023-12-14 19:34:58 +00:00
Florian Klink
e4d6d20aa7 test(tvix/glue): add nonexisting_path_without_import test
This moves the failing example from cl/10285 into its separate test
case.

There were multiple complications: tvix-[ca]store was panicking in some
places, rather than returning an error. This is now fixed.

It needs to live in tvix-glue, so we actually have a "proper" EvalIO
interface doing something.

> toString ({ line = 42; col = 42; file = /deep/thought; }.file)

Should not cause an error, because it shouldn't trigger an import, but
leave the path as-is, and not care about it not being present.

Change-Id: I76f70b3cb1f73a0fb05870375710fd9f67d5603c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10342
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: sterni <sternenseemann@systemli.org>
2023-12-14 13:25:25 +00:00
sterni
7165ebc43b fix(tvix/eval): remove incorrect imports when coercing
The default behavior of string coercion in C++ Nix is to weakly coerce
and import to store if necessary. There is a flag to make it strongly
coerce (coerceMore) and a flag that controls whether path values have
the corresponding file/directory imported into the store before
returning the (store) path as a string (copyToStore). We need to
implement our equivalent to the copyToStore (import_paths) flag for the
benefit of weak coercions that don't import into the store (dirOf,
baseNameOf, readFile, ...) and strong coercions that don't import into
the store (toString).

This makes coerce_to_string as well as CoercionKind weirder and more
versatile, but prevents us from reimplementing parts of the coercion
logic constantly as can be seen in the case of baseNameOf.

Note that it is not possible to test this properly in //tvix/eval tests
due to the lack of an appropriate EvalIO implementation being available.
Tests should be added to //tvix/glue down the line.

Change-Id: I8fb8ab99c7fe08e311d2ba1c36960746bf22f566
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10361
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reviewed-by: Adam Joseph <adam@westernsemico.com>
2023-12-14 13:15:23 +00:00
sterni
a30dd0905a fix(tvix/eval): determine meaning of + exprs based on first type
This matches the behavior of C++ Nix more closely where the decision is
made based on the first type based to ExprConcatStrings:
1f93fa2ed2/src/libexpr/eval.cc (L1967-L2025)

Note that this doesn't make a difference in any successful
evaluation (at least to my knowledge), but ensures that our error
messages will match C++ Nix more closely, e.g. in the case of
`1 + "string"`.

Change-Id: I8059930788f9c8d98baf98e3d93d8a060ef961f2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10360
Tested-by: BuildkiteCI
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: Adam Joseph <adam@westernsemico.com>
2023-12-14 02:59:58 +00:00
Florian Klink
8b0047e277 docs(tvix/castore/directorysvc): update comment
This comment didn't make a lot of sense before.

Change-Id: Ie057a133ca4b1a099ed3c885e32316b0d87c5eb0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10339
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-12-13 21:11:58 +00:00
Florian Klink
3a32963b78 docs(tvix/castore): document expectations about DirectoryService
Namely, all trait implementations should reject invalid data being fed,
and detect invalid data being returned.

b/355 tracks writing some more tests for this, to ensure we're compliant
with this.

Change-Id: I3b05752932837ce208785efb21ffc21508b4b33a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10338
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Autosubmit: flokli <flokli@flokli.de>
2023-12-13 19:57:45 +00:00
Florian Klink
d236b08916 docs(tvix/castore): fix docstrings
There's been some copypasta errors.

Change-Id: I8fcad6cfc951ead6c789e0dce823c798adbfcf97
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10337
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2023-12-13 19:40:40 +00:00
Florian Klink
9c5c717a99 feat(tvix/glue): add single-file import tests
This ensures importing these paths also behave the same way as Nix.

Change-Id: Icaa507bbe3d9867a301fc7a300c5d2b3f9feb911
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10355
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
2023-12-13 13:42:21 +00:00
Florian Klink
3cdde8ad5a feat(tvix/glue): add some import tests
This creates a directory with a .keep file inside, and uses
"${path/to/there}" to coerce it to a string (and import it into the
store), ensuring it calculates the same store paths as Nix does.

Change-Id: Ie14ae075104ce278bc4f2cce93aab5762a2734d1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10343
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-12-13 13:41:44 +00:00
Florian Klink
307675f5e7 fix(tvix/glue/tvix_store_io): don't unwrap ingest_path errors
Fixes b/344.

Change-Id: I1446726e3be3a8fc20801d466a964c4d6b8cbc02
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10331
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
2023-12-12 18:07:43 +00:00
Florian Klink
81ef26ba3f fix(tvix/castore/import): don't unwrap entry
If the path specified doesn't exist, construct a proper error instead
of panicking.

Part of b/344.

Change-Id: Id5c6a91248b0a387f3e8f138f8e686e402009e8f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10330
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
2023-12-12 18:07:11 +00:00
Florian Klink
afd09c3290 feat(tvix/castore/import): log returned errors
This will emit a log event / trace in case this function returns an
error-y type.

Change-Id: I48db6807f3e42304357c422a2b6e177cb8b95228
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10329
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
2023-12-12 18:07:11 +00:00
Florian Klink
30d82efa77 refactor(tvix/castore/blobservice): use io::Result in trait
For all these calls, the caller has enough context about what it did, so
it should be fine to use io::Result here.

We pretty much only constructed crate::Error::StorageError before
anyways, so this conveys *more* information.

Change-Id: I5cabb3769c9c2314bab926d34dda748fda9d3ccc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10328
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
2023-12-12 18:06:40 +00:00
Adam Joseph
91456c3520 refactor(tvix/eval): vm::add_values(): be less verbose
Change-Id: Icf328649fd70bdf0fc3ba6cd7754ae29735f11f7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10035
Autosubmit: Adam Joseph <adam@westernsemico.com>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-12-12 18:00:37 +00:00
Adam Joseph
243a4b5699 fix(tvix/eval): substring: propagate catchables
Change-Id: Ia9b7858c817fbc9c95a3d1c2855b2445f7830e8d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10326
Tested-by: BuildkiteCI
Autosubmit: Adam Joseph <adam@westernsemico.com>
Reviewed-by: tazjin <tazjin@tvl.su>
2023-12-12 17:51:29 +00:00
Adam Joseph
c5f58d4af3 fix(tvix/eval): baseNameOf: propagate catchables
Change-Id: Id8dc772ea8f338dfd243210f4108f79072570c3b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10324
Autosubmit: Adam Joseph <adam@westernsemico.com>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-12-12 17:50:56 +00:00
Adam Joseph
2565a21aa9 fix(tvix/eval): builtins.length: propagate catchables
This commit fixes out builtins.length so it propagates catchables
like cppnix does.

Change-Id: I7670bec5eee1d4cd3f67a04c9a6808979fb56a8d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10315
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: Adam Joseph <adam@westernsemico.com>
Tested-by: BuildkiteCI
2023-12-12 17:49:54 +00:00
Adam Joseph
1ddfed9825 fix(tvix/eval): builtins.filter: propagate catchables
This commit fixes builtins.filter so it propagates catchables
correctly.

Change-Id: Ib23a383bc5e272e42052205ffd1e94649a0ebc47
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10313
Autosubmit: Adam Joseph <adam@westernsemico.com>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-12-12 17:35:39 +00:00
Adam Joseph
25dd1458b9 feat(tvix/eval): builtins.hashString: add placeholder
This adds an unimplemented placeholder for builtins.hashString.

Change-Id: Ibc770103acf5dbc3ea7589ab5ca23fe6e07bd91a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10311
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: Adam Joseph <adam@westernsemico.com>
2023-12-12 17:35:06 +00:00
Adam Joseph
8c8409c0d2 fix(tvix/eval): builtins.getAttr: propagate catchables
Change-Id: I84b6b8f8568d57614a03aff0d6069e0bc27357bf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10310
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: Adam Joseph <adam@westernsemico.com>
2023-12-12 17:34:34 +00:00
Adam Joseph
afba150036 fix(tvix/eval): builtins.elemAt: propagate catchables
This commit fixes builtins.elemAt so it propagates catchables like
cppnix does.

Change-Id: Ieca5e128da17e78af0b14dae4a28a1ff8796e4f2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10308
Autosubmit: Adam Joseph <adam@westernsemico.com>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-12-12 17:34:02 +00:00
Adam Joseph
07ca556a94 feat(tvix/glue): make unimplemented-structuredAttrs catchable
This commit adjusts the error produced by STRUCTURED_ATTRS so that
it is catchable.  This way we are able to enumerate the release
packageset, and the enumeration process will simply treat the few
derivations using structured attributes as being broken, rather than
killing the whole eval session.

Change-Id: I2e17638b8e3227f88543c3718aaf505deaec22ae
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10306
Autosubmit: Adam Joseph <adam@westernsemico.com>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-12-12 17:21:53 +00:00
Adam Joseph
29464fc65b fix(tvix/eval): propagate catchables through builtins.splitVersion
This fixes our implementation of builtins.splitVersion so it
propagates catchables like cppnix does.

Change-Id: Id5d83ea76229f8c8f202aa42353cb609e67de43f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10305
Autosubmit: Adam Joseph <adam@westernsemico.com>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-12-12 17:21:53 +00:00
Adam Joseph
012b44244b feat(tvix/eval): builtins.hasContext: placeholder implementation
Currently this just `throw`s a message explaining that it is not
implemented.  This is necessary in order to allow enumerating the
nixpkgs release attrset (afaict only one package uses this builtin).

Change-Id: I45266d46af579ddb5856b192b6be4b481369543c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10302
Autosubmit: Adam Joseph <adam@westernsemico.com>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-12-12 17:21:53 +00:00
Adam Joseph
531f0c0c42 fix(tvix/eval): baseNameOf should not coerce paths into strings
... since this may import them to the store which changes their
basename.

Fixes b/350.

Change-Id: Iabd08ff4d6a424c66d6d7784d7a96b0c078f0a91
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10298
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: Adam Joseph <adam@westernsemico.com>
2023-12-12 17:21:21 +00:00
Adam Joseph
8d4aa2c15c fix(tvix/eval): add unimplemented __curPos and builtins.filterSource
This commit adds __curPos (to the global scope, yuck) and
builtins.filterSource.  These are not implemented; forcing them will
produce the same result as `throw "message"`.

Unfortunately these two post-2.3 features are used throughout
nixpkgs.  Since an unresolved indentifier is a catchable error, this
breaks the entire release eval.  With this commit, it simply causes
those broken packages that use these features to appear as they are:
broken.

Change-Id: Ib43dea571f6a9fab4d54869349f80ee4ec5424c2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10297
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: Adam Joseph <adam@westernsemico.com>
2023-12-12 17:20:50 +00:00
Adam Joseph
bb1e79e5d1 fix(tvix/eval): propagate catchables through &&
Change-Id: I7bb5ac1ef47b41c47269e64cee0e90eb64c6fcce
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10322
Autosubmit: Adam Joseph <adam@westernsemico.com>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-12-12 17:04:07 +00:00
Adam Joseph
bcc1ea8552 fix(tvix/eval): make || propagate catchables
Change-Id: I42f994d7c9228368d5f6c30c4730c24666f7bc69
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10320
Autosubmit: Adam Joseph <adam@westernsemico.com>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-12-12 17:04:07 +00:00
Adam Joseph
7b9eea71d0 fix(tvix/eval): fix nested assertions b/340
This commit fixes our handling of `throw` within an `assert`
condition.

Fixes: b/340
Change-Id: I40a383639ec266da50a853f16216b1b7868495da
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10318
Autosubmit: Adam Joseph <adam@westernsemico.com>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-12-12 17:03:03 +00:00
Adam Joseph
fc963033ae fix(tvix/eval): ?: propagate catchables
This commit fixes out `?` operator so it correctly propagates
catchables.

Change-Id: Iebaa153a8492101ee3ddd29893c98730ff331547
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10317
Autosubmit: Adam Joseph <adam@westernsemico.com>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-12-12 16:57:55 +00:00