mdebray/tvl-depot
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

docs(tvix/TODO): drop PathInfo including references by content idea

Browse source 
This is not gonna work out as-is, as we still key PathInfos by
their store path digest, and how to handle thing if we encounter a
Frankenbuild.

For now, let's keep the PathInfoService data as it is, we can record
this information (and more) in the builder structures.

Change-Id: Ic38fc3ecd8096a5fe002e681bdc812a9dbeaa7d2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12607
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: edef <edef@edef.eu>
This commit is contained in:
Florian Klink 2024-10-13 12:52:19 +03:00 committed by clbot
parent 6501ee194b
commit 42377ba235
1 changed files with 0 additions and 27 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

27
tvix/docs/src/TODO.md
View file

@ -136,33 +136,6 @@ Similarly, we also don't properly populate the build environment for
`fetchClosure` yet. (Note there already is `ExportedPathInfo`, so once
`structuredAttrs` is there this should be easy.
### PathInfo: include references by content
In the PathInfo struct, we currently only store references by their names and
store path hash. Getting the castore node for the content at that store path
requires another lookup to the PathInfoService.
Due to this information missing, this also means we currently cannot preserve
information necessary to detect/prevent
[Frankenbuilds](https://tvl.fyi/blog/tvix-update-february-24#builder-protocol-drv-builder).
We should extend/change the `PathInfo` type to maintain references in a
`BTreeMap` from store path basename to castore node. Should probably be done
after PathInfo uses its own data type.
The `NixHTTPPathInfoService` needs to get some more logic to populate the ca
bits of the references:
- If the referenced store path if not present in our PathInfoService hierarchy,
it needs to be ingested too, and persisted.
- If the referenced store path is present, we can use the castore root from there.
In an optional mode, we should parse the .narinfo file for the reference, and
compare the nar hash with our local one, to detect Frankenbuilds in a binary
cache.
As `NixHTTPPathInfoService` now needs to interact with "the PathInfoService"
hierarchy, we need to accept a pointer to a PathInfoService (hierarchy) that's
used to check for presence, and PathInfos are inserted into.
### Builders
Once builds are proven to work with real-world builds, and the corner cases
there are ruled out, adding other types of builders might be interesting.