We don't need to spawn in all these places, we can just block_on
directly, this is all IO bound.
This also means, we don't need to clone any of the service handles
(except preserving clone-ability of the BlobService).
Change-Id: I7d90f4d6a263a98491caa071ada538a5197a5472
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10540
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
This was wrongly named, it returns a specific node at a subpath.
Also, this code can be simplified a lot - we don't need to spawn
additional tasks, and can get rid of some clones too.
This is also where we need a certain build - so add some TODO to block /
fetch here.
Change-Id: Id26d7bd80f7a2095121e642b3f7716de78d6b6a6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10539
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Extend our validation function to also check for the None case.
Change-Id: Ib75f880646d7fb3d66588f1988e61ec18be816a2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10534
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
Make this an `AsRef<dyn DirectoryService>`.
This helps dropping some Clone requirements.
Unfortunately, we can't thread this through to TvixStoreIO just yet.
Change-Id: I3f07eb28d6c793d3313fe21506ada84d5a8aa3ac
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10533
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
While doing the obvious thing, ensuring clippy doesn't complain on our
benchmarks and examples, this will also flag build failures on them,
and contrary to cl/10301 not pollute the /nix/store with stuff we can't
execute anyways.
Change-Id: I7ee1c2babbc67dac9794274d05bb9d6e1251ed01
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10544
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
cl/10475 updated the API of tvix_eval::Evaluation, but didn't update
this benchmark to use that new API.
Also, fixes the docstring to no longer specify that there is a "given
snippet".
Change-Id: Ibb8285731849dbeec814e2585bbaa36f22368afe
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10542
Autosubmit: aspen <root@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: flokli <flokli@flokli.de>
With context strings, we are now able to reproduce up to firefox.
Until a more problematic example can be found, we should at least
lean on this particular example.
Change-Id: Ibeaf799d26e1f160b6c93f8ccd978702a2a7e3e1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10460
Reviewed-by: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Raw printing will transform the result into a string
and print it.
In case of a contextful string, this will fail by default, as expected.
Change-Id: I5e564329e7b001adc57a77a9153b4425cb332bb7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10457
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Nix does something like:
```cpp
NixStringContext context;
const auto str = state.forceString(*args[1], context, pos, "while evaluating the second argument passed to builtins.split");
```
And then do nothing with that context, therefore, we follow them and
make `split` aware of the context but still do nothing with it.
Change-Id: I4fee1936600ce86d99d00893ca3f64013213935b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10428
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: raitobezarius <tvl@lahfa.xyz>
We calculate the input context by performing the union of context
over all input of the derivation.
Then, we just pass the rest of it to the remaining machinery.
Finally, we re-emit an `outPath` and a `drvPath` containing the expected
contexts.
Change-Id: I74905fb258b5bee8b08d1208c9eb87f51b92a890
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10436
Autosubmit: raitobezarius <tvl@lahfa.xyz>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
This removes the reference tracking and uses instead the context for
references and returns some.
Change-Id: Ic359ca6b903b63f1a9c679c566004c617b792442
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10435
Autosubmit: raitobezarius <tvl@lahfa.xyz>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
In the past reference tracking system, `tvix-io` glue was appending
plain paths in the known path state.
Now, we make up for this by just making contextful coercion of file
imports.
Change-Id: Ieb9b04dd83302c77909252d5f7733857ac3cf8fd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10443
Tested-by: BuildkiteCI
Autosubmit: raitobezarius <tvl@lahfa.xyz>
Reviewed-by: tazjin <tazjin@tvl.su>
String with contexts are always coerced to a string with the same
context.
Change-Id: I224814febd9cad196bb28876793e76bed564dc72
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10440
Tested-by: BuildkiteCI
Autosubmit: raitobezarius <tvl@lahfa.xyz>
Reviewed-by: sterni <sternenseemann@systemli.org>
Otherwise, you just fail because they are not... contextless strings!
Change-Id: I0b8f63a18cd89c3841b613d41c12ec4ee336f953
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10442
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
`match` silently ignore the input context and do not propagate
it and successful matches.
The why is unclear but nixpkgs does rely implicitly on this behavior
because dynamic attribute selection cannot be done with contextful
strings.
Change-Id: I5167fa9b2c2db8ecab0c2fb3e9895c9cfce6eeb2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10441
Autosubmit: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
And it also preserve the original context if it exists.
Change-Id: I904f7c13b7f003a267aace6301723780fccaafb7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10434
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: raitobezarius <tvl@lahfa.xyz>
We just perform union of contexts of every pieces.
Change-Id: Ief925c1818cd8bbec0503e9c625b0630feebfdda
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10432
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: raitobezarius <tvl@lahfa.xyz>
`substring` has a very funny behavior when it comes to empty strings,
it propagates the context too, this is used in nixpkgs to attach context
to strings without using any builtin: `lib.addContextFrom`.
Change-Id: Id655356799b3485f7519b3d1914c630f9d8416c3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10448
Autosubmit: raitobezarius <tvl@lahfa.xyz>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
We make a case for adding a `reject_context` `CoercionKind` here.
It does happen during concatenation actually for path concats.
Change-Id: I0c196aad917550b9bcd0896cd2127a94f8181ffb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10444
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: raitobezarius <tvl@lahfa.xyz>
I am still undecided whether we need a CoercionKind to control
the coerced context, here's a simple attempt.
Change-Id: Ibe59d09ef26c519a6acfdfe392014446646dd6d8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10426
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: raitobezarius <tvl@lahfa.xyz>
This extends derivation_to_build_request to handle passAsFile the same
way Nix does, and adds a unit test for it.
I opted to making this function fallible (if passAsFile contains a
non-existent env var), rather than pushing all of this into the
Derivation validate function.
Change-Id: I75b635f1f6c0c78d72b9a8fc7824f77e97b69951
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10522
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
While this can be influenced with `OTEL_SERVICE_NAME` or
`OTEL_RESOURCE_ATTRIBUTES`, "unknown_service" is an annoying name.
Let's pick tvix.store as a default.
Change-Id: I9b5c45703d9546775bb9266b29ab54daf7c54a9f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10524
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Prevent code in this function from accessing the Vec, realize it only
when returning the BuildRequest struct.
Change-Id: I385c1adca552c9cb261cd70c5ca3ec45f0d7a9c3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10521
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
This already succeeds, as the output path calculation path does not need
to be aware of any builder-specific custom handling for passAsFile.
Change-Id: I9cc9e4e4351cdeaa3ec33ba58ee1569e7a368150
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10520
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Nix builds also have some other inputs, outside of inputs_dir, used in
derivations setting `passAsFile` and `__structuredAttrs`.
This extends our `BuildRequest` structure to accomodate these usecases.
Change-Id: I8de6516eb467fa48a4961d88933ebd548f131049
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10519
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
And include a test to ensure we show the warning.
Change-Id: Ib6a436dbba2592b398b54e44f15a48d1aa345099
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10470
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
In the past, we had `emit_warning` be no-op and we used `push_warnings` exclusively
but as we have consumers of this function, we need it to work somewhat.
Change-Id: I78a5ece199a473dec9ef5ea1fae60b36e35137b8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10477
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Nix is quite tolerant when it comes to parsing SRI hashes and their
padding (and only for SRI hashes, it funnily is strict about that in the
non-SRI-hash case).
Nix essentially accepts any number of padding characters, no matter if
it's too much or too little. So we do the only sane thing - simply strip
all padding characters, and parse it with BASE64_NOPAD and the length
the algo uses.
Change-Id: I6a721aa289b06cc36741589792b9dd4c4f930b86
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10468
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
This is behind the otlp feature flag (currently enabled by default).
By default, this will try to push traces to a OTLP collector running at
https://localhost:4317.
You can make one available by running:
```
docker run -d --name jaeger \
-e COLLECTOR_ZIPKIN_HOST_PORT=:9411 \
-e COLLECTOR_OTLP_ENABLED=true \
-p 6831:6831/udp \
-p 6832:6832/udp \
-p 5778:5778 \
-p 16686:16686 \
-p 4317:4317 \
-p 4318:4318 \
-p 14250:14250 \
-p 14268:14268 \
-p 14269:14269 \
-p 9411:9411 --rm \
jaegertracing/all-in-one:1.42
```
Started like that, jaeger brings a web interface at
http://localhost:16686/search
As documented in
https://docs.rs/opentelemetry-otlp/latest/opentelemetry_otlp/, you can
point this to another location by setting `OTEL_EXPORTER_OTLP_ENDPOINT`.
Change-Id: Id1dca367d70027b2ea98bb70bcf99a68363ec2be
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8194
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: aaqaishtyaq <aaqaishtyaq@gmail.com>
Only convert to and reuse an Arc<…> where needed.
Change-Id: I2c1bc69cca5a4a3ebd3bdb33d6e28e1f5fb86cb9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10514
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
We can also drop the Clone requirement. Because the trait is async since
some time, there's no need to clone before moving into an async closure,
allowing us to simplify the code a bit.
Change-Id: I9b0a0e10077d8c548d218207b908bfd92c5b8de0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10515
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: flokli <flokli@flokli.de>
This is doing the exact same thing.
Change-Id: Iadc5e13dd192efc91cc3d36b2bdf4b8b99a312b2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10512
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
While at it, make it a bit more generic.
Change-Id: Ic4caefda93aca3ffb656a09f8b4d648b41415532
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10511
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
We don't actually care if it's an Arc<dyn BlobService>, or something
else, as long as we can Deref to a BlobService and clone.
Change-Id: I0852aaf723f51c5e6b820be8db1199d17309ab08
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10510
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
