Commit graph

340 commits

Author SHA1 Message Date
John Ericson
b932cf2d85 refactor(nix-compat) Make nixhash_* mods hierarchical
They can go under `nixhash`

Change-Id: Ia15835c57130b66d58f5df80ae9595dceee00941
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8408
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-03-31 18:46:18 +00:00
John Ericson
df78a4d24e refactor(tvix/nix-compat): Inline texthash mod
It is moved into `store_path::utils` with the other path builders.

Change-Id: I3257170e442af5d83bcf79e63fa7387dd914597c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8410
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-03-31 18:46:08 +00:00
Florian Klink
5f2c2e79e1 refactor(tvix/nix-compat): move build_store_path out of derivation
This doesn't have anything to do with ATerms, we just happen to be using
the aterm representation of a Derivation as contents.

Moving this into store_path/utils.rs makes these things much cleaner -
Have a build_store_path_from_references function, and a
build_store_path_from_fingerprint helper function that makes use of it.

build_store_path_from_references is invoked from the derivation module
which can be used to calculate the derivation path.

In the derivation module, we also invoke
build_store_path_from_fingerprint during the output path calculation.

Change-Id: Ia8d61a5e8e5d3f396f93593676ed3f5d1a3f1d66
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8367
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-03-30 14:03:35 +00:00
Florian Klink
4ab180b1eb refactor(tvix/nix-compat): use Sha256::new_with_prefix
Change-Id: I328983008c0680e0448ccff604dbdae312f7988b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8366
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-03-30 14:03:34 +00:00
Florian Klink
6454769eef refactor(tvix/nix-compat): drop is_derivation in build_store_path
This only added a suffix to the input argument, if build_store_path was
building the path of a Derivation.

As we need to also add the `.drv` suffix to the name we pass into
text_hash_string inside calculate_derivation_path, we can simply add the
suffix there and drop the parameter from build_store_path.

Change-Id: Icd5343dd1458f112b9296b389e81ce2ebdd16a9f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8365
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-03-30 14:03:34 +00:00
Florian Klink
971080c912 refactor(tvix/nix-compat): add text_hash_string function
Use it to calculate the text_hash_string, which is then used in the
calculate_derivation_path and path_with_references functions.

Relates to b/263.

Change-Id: I7478825e2a23a11224212fea5e3fd06daa97d5e5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8364
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-03-30 14:03:34 +00:00
Florian Klink
14c5781389 refactor(tvix/nix-compat): move HashAlgo to separate file
… and keep the pub exports as is.

Change-Id: I2ad21660577553395f05b5ba71083626429b0dfc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8363
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
2023-03-30 13:47:31 +00:00
Florian Klink
e53152866e refactor(tvix/nix-compat): move NixHashWithMode into separate file
… and keep the pub exports as is.

Change-Id: I9f89a738c508c478ddba61303c21ea294f01ee9f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8362
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
2023-03-30 13:47:31 +00:00
Florian Klink
def2d32319 refactor(tvix/nix-compat): remove get_fixed_output() func
The behaviour of this function is a bit unintuitive, and cl/8310 already
inlined the other consumer of it.

Rewrite the last consumer of the function, so we can drop it.

Change-Id: I59c8486037ce3f777667d1d9e4f4a9316d5a0cb9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8311
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-03-15 15:31:55 +00:00
Florian Klink
09b30f895a refactor(tvix/nix-compat): simplify fingerprint calculation
Instead of having two very similar match branches for the FOD and non-
FOD case, detect the FOD case while looping over all outputs.

In the case of anything other than recursive sha256 FODs, the
fingerprint and output path calculation is exactly the same.

Change-Id: Ieb6995653d008766e595cf29d7cd4fb1334e33dd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8310
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
2023-03-15 14:02:24 +00:00
Florian Klink
b55d1f97ce refactor(tvix/nix-compat): -derivation::Hash, +NixHash
This stops using our own custom Hash structure, which was mostly only
used because we had to parse the JSON representation somehow.

Since cl/8217, there's a `NixHash` struct, which is better suited to
hold this data. Converting the format requires a bit of serde labor
though, but that only really matters when interacting with JSON
representations (which we mostly don't).

Change-Id: Idc5ee511e36e6726c71f66face8300a441b0bf4c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8304
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-03-15 11:50:12 +00:00
Florian Klink
a14880903c refactor(tvix/nix-compat): rename helper func
This only trims the output paths from a Derivation struct, not the
output hashes.

Change-Id: I9250fec4602ed05bb64540c4a89ddb6fb052be1f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8303
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
2023-03-14 20:53:12 +00:00
Florian Klink
84a8ed265c refactor(tvix/nix-compat): remove manual map
Change-Id: I1652e24c9be28112d98683de2d2db51dc46001ed
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8302
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-03-14 20:53:12 +00:00
Florian Klink
c699ace11b refactor(tvix/nix-compat): rename test name
Change-Id: Ia6af8d4b53fbd4d9ea26f0094cd98dde66a91f78
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8301
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-03-14 20:53:12 +00:00
Florian Klink
075dcbf1e0 fix(tvix/store/derivation): fix comment
The digest is encoded in LOWERHEX here, not nixbase32.

Change-Id: I8e14742f0135966424073bfdb93ada8cd7bd52e2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8300
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
2023-03-14 20:53:12 +00:00
Florian Klink
5f260edf7f refactor(tvix/nix-compat): replace calculate_drv_replacement_str
Call this function derivation_or_fod_hash, and return a NixHash.

This is more in line with how cppnix calls this, and allows using
to_nix_hash_string() in some places.

Change-Id: Iebf5355f08ed5c9a044844739350f829f874f0ce
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8293
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-03-14 18:26:19 +00:00
Florian Klink
1f9b582239 feat(tvix/nix-compat/derivation): make use of NixPath in some places
Some of these strings are actually just the nix hash representation of
the hash calculated earlier.

There's another one passed around via calculate_drv_replacement_str, but
that's left for a followup.

Change-Id: Id99a2a926a980d679eb49c34ee6a36bf224699b0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8218
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
2023-03-04 12:26:00 +00:00
Florian Klink
c2a681eaff feat(tvix/nix-compat): add NixHash::{new, to_nix_hash_string}
This provides a way to construct a NixHash struct without parsing
strings.

Change-Id: I947d96e15e51e72d5b02929cda8c5fc31d81253a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8217
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
2023-03-04 12:26:00 +00:00
Florian Klink
67c9e2c770 feat(tvix/nix-compat): introduce parsing for Nix Hash strings
We ironically didn't add support parsing for the "native" format that
Nix uses under the hood.

This extends the from_str method to peek at the prefix of the string to
determine whether to try decoding as SRI, Nix string, or whether it
should be a bare digest.

Change-Id: I33efd24968b16f86eff18305b4ca8f112c7131d7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8216
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-03-04 12:26:00 +00:00
Florian Klink
4cc4b2a814 refactor(tvix/nix-compat): move digest decoding into helper function
This will be used for both Nix hash strings and hash strings without the
algo specified.

Change-Id: Iedfe5494fba5f2be00614ba0fc38bf659eafd447
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8215
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-03-04 12:26:00 +00:00
Florian Klink
daba4cc0e5 refactor(tvix/nix-compat): address clippy in nixbase32.rs tests
Change-Id: If8820cba4cf19bf0f7aa27e0d93b70eb7b4ee81f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8221
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-03-04 12:21:28 +00:00
Florian Klink
93b0766485 refactor(tvix/nix-compat): address clippy in derivation/tests/mod.rs
Change-Id: I3f3f7cb590c900abf8b39533ed73fe9135d58f0b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8220
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
2023-03-04 12:19:24 +00:00
Florian Klink
f9dcf54b04 refactor(tvix/nix-compat): address clippy in store_path.rs
Change-Id: Ib10bd93bbb23696d7048e7ed8e405953db94693f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8219
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-03-04 12:19:24 +00:00
Vincent Ambo
248f1c2151 chore(tvix/nix-compat): include invalid hashes in SRI hash error
Change-Id: I379b18cb07da0988f8ce4934976ae7d6566d6bb5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8210
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2023-03-03 23:35:47 +00:00
Florian Klink
d4cbdc2beb feat(tvix/nix-compat/nixhash): support BASE64_NOPAD SRI
Nix accepts SRI hashes that are missing their padding characters
in base64, as seen in
7e49471316/pkgs/development/libraries/kerberos/krb5.nix .

It only seems to work in the SRI case, not with `sha256` being set to a
(nopad) base64 string.

Add regression tests for this, and document why we don't want to support
*additional* characters afterwards.

Reported in https://b.tvl.fyi/issues/252

Change-Id: I9ffc2b417501b426ced1894a9cbf95ff5f0e5159
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8037
Reviewed-by: Alyssa Ross <hi@alyssa.is>
Tested-by: BuildkiteCI
2023-02-08 10:39:26 +00:00
Florian Klink
5fa35d9d49 refactor(tvix/nix-compat/nixhash): fix test function name
Change-Id: I3b089758ce8c01df544064422e56a97a8402513d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8036
Tested-by: BuildkiteCI
Reviewed-by: Alyssa Ross <hi@alyssa.is>
2023-02-08 10:39:26 +00:00
Alyssa Ross
5da1f4161c fix(tvix): fix InvalidEncodedDigestLength message
Before:
	error[E997]: invalid output hash: invalid encoded digest length '43' for algo 43

After: error[E997]: invalid output hash: invalid encoded digest length '43' for algo sha256

Change-Id: Icb533839586b47f90b4e8d22083cce397f9f3655
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8035
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-02-07 17:40:54 +00:00
Florian Klink
f5ccb65f26 feat(tvix/nix-compat/derivation): Display -> to_aterm_string()
Instead of implementing `std::fmt::Display for Derivation` and relying
on the `to_string` method, introduce a `to_aterm_string()` method, which
does the same thing, but makes it clearer what we're producing, rather
than just calling `to_string()``.

Change-Id: I21823de9096a0f2c2eb6f4591e48c1aa9fd94161
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7998
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-02-01 16:31:56 +00:00
Florian Klink
265dd36b98 feat(tvix/nix-compat): add nixhash module
This module takes care of parsing various hashes and algorithms.

It will get used to modify derivation output hashes in the next CL.

Change-Id: Idc07c401dbb7510f49883ac02b8379b9a5d930c7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7990
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-02-01 15:29:48 +00:00
Florian Klink
366d2c5e92 refactor(tvix/nix-compat/derivation): remove module inception
The module `src/derivation/derivation.rs` is a sign of module inception.

Move the Derivation struct definiton up into `src/derivation/mod.rs`,
and some of the helpers in a `util.rs`.

Change-Id: Ib24a5f8a27bdd45df8b1fa2b3482a79b33cab8d5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7997
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
2023-02-01 13:43:25 +00:00
Florian Klink
2551adb4f1 refactor(tvix/nix-compat): operator precedence can trip the unwary
warning: operator precedence can trip the unwary
  --> nix-compat/src/nixbase32.rs:41:23
   |
41 |                 c |= ((input[i + 1] as u16) << 8 - j as u16) as u8
   |                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: consider parenthesizing your expression: `(input[i + 1] as u16) << (8 - j as u16)`
   |
   = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#precedence
   = note: `#[warn(clippy::precedence)]` on by default

Change-Id: I091071d649abf4ed38f5f4e39a0c5d21a0459bff
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7996
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-02-01 13:43:24 +00:00
Florian Klink
361ffd7903 refactor(tvix/derivation): don't create deref'd immediately ref
clippy says:

> This expression creates a reference which is immediately dereferenced
> by the compiler

Change-Id: Ic2c093b043ebee9ae80912075083107e4d216cf1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7995
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
2023-02-01 13:38:21 +00:00
Florian Klink
ce4d87b129 refactor(tvix/nix-compat): remove unneeded return
Change-Id: I97b9fb3ad33d2f51baf29486d4eff11f8dedcbab
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7994
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
2023-02-01 13:38:19 +00:00
Florian Klink
894d17b29b refactor(tvix/nix-compat): const is always 'static
Change-Id: If33bbf47a6cc376f2c0935ea4331a819223b2f00
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7993
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
2023-02-01 13:38:18 +00:00
Vincent Ambo
759f9dbf39 feat(tvix/cli): implement builtins.placeholder
This doesn't require any other corresponding handling *yet*, as the
actual replacements happen in the builder logic (which we delegate to
cppnix at the moment).

Change-Id: I034147c933f05ae427c7a8794647132d108d0ede
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7972
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2023-02-01 10:01:40 +00:00
Florian Klink
a7e20bbef3 docs(tvix/nix-compat/store_path): fix docstrings
Change-Id: I19261233e338b406a87fe680675eb4dfd853fe11
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7970
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
2023-01-31 15:35:45 +00:00
Florian Klink
a23b7e17c0 refactor(tvix/nix-compat): absorb nar writer
Expose it at `nix_compat::nar::writer`.

Change-Id: I98a8acfa6277074f24608d64fb6c0082b41d10f5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7969
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-01-31 15:18:39 +00:00
Florian Klink
1e28cc7ede docs(tvix/nix-compat/derivation): fix docstrings
Change-Id: I3fd02e62c8a24fa7f27461b17cdd824b060d9bdb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7968
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
2023-01-31 15:16:40 +00:00
Florian Klink
2d24c5f260 refactor(tvix/nix-compat): absorb //tvix/derivation
Put this in its src/derivation.

Change-Id: Ic047ab1c2da555a833ee454e10ef60c77537b617
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7967
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
2023-01-31 15:16:39 +00:00
Florian Klink
c27bacd905 refactor(tvix): introduce nix-compat crate
Move nixbase32 and store_path into this.

This allows //tvix/cli to not pull in //tvix/store for now.

Change-Id: Id3a32867205d95794bc0d33b21d4cb3d9bafd02a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7964
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-01-31 13:26:18 +00:00