tvl-depot/tvix/castore/default.nix

{ depot, pkgs, ... }:

(depot.tvix.crates.workspaceMembers.tvix-castore.build.override {
  runTests = true;
  testPreRun = ''
    export SSL_CERT_FILE=${pkgs.cacert}/etc/ssl/certs/ca-bundle.crt;
  '';

  # enable some optional features.
  features = [ "default" "cloud" ];
}).overrideAttrs (_: {
  meta.ci.targets = [ "integration-tests" ];
  passthru.integration-tests = depot.tvix.crates.workspaceMembers.tvix-castore.build.override {
    runTests = true;
    testPreRun = ''
      export SSL_CERT_FILE=${pkgs.cacert}/etc/ssl/certs/ca-bundle.crt;
      export PATH="$PATH:${pkgs.lib.makeBinPath [pkgs.cbtemulator pkgs.google-cloud-bigtable-tool]}"
    '';

    # enable some optional features.
    features = [ "default" "cloud" "integration" ];
  };
})
refactor(tvix): move castore into tvix-castore crate This splits the pure content-addressed layers from tvix-store into a `castore` crate, and only leaves PathInfo related things, as well as the CLI entrypoint in the tvix-store crate. Notable changes: - `fixtures` and `utils` had to be moved out of the `test` cfg, so they can be imported from tvix-store. - Some ad-hoc fixtures in the test were moved to proper fixtures in the same step. - The protos are now created by a (more static) recipe in the protos/ directory. The (now two) golang targets are commented out, as it's not possible to update them properly in the same CL. This will be done by a followup CL once this is merged (and whitby deployed) Bug: https://b.tvl.fyi/issues/301 Change-Id: I8d675d4bf1fb697eb7d479747c1b1e3635718107 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9370 Reviewed-by: tazjin <tazjin@tvl.su> Reviewed-by: flokli <flokli@flokli.de> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com> 2023-09-21 21:32:44 +02:00			`{ depot, pkgs, ... }:`

test(tvix/[ca]store): enable bigtable only with "integration" feature The emulator and bigtable client are quite big. Remove them from the default //tvix:shell. Put the tests behind a `integration` feature flag, and add a variant with that enabled to CI, and drop the bigtable tools from //tvix:shell. Change-Id: Ie042097a0d6fc26542faa96c139b77298ccb160a Reviewed-on: https://cl.tvl.fyi/c/depot/+/11582 Reviewed-by: edef <edef@edef.eu> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI 2024-05-02 23:09:46 +02:00			`(depot.tvix.crates.workspaceMembers.tvix-castore.build.override {`
refactor(tvix): move castore into tvix-castore crate This splits the pure content-addressed layers from tvix-store into a `castore` crate, and only leaves PathInfo related things, as well as the CLI entrypoint in the tvix-store crate. Notable changes: - `fixtures` and `utils` had to be moved out of the `test` cfg, so they can be imported from tvix-store. - Some ad-hoc fixtures in the test were moved to proper fixtures in the same step. - The protos are now created by a (more static) recipe in the protos/ directory. The (now two) golang targets are commented out, as it's not possible to update them properly in the same CL. This will be done by a followup CL once this is merged (and whitby deployed) Bug: https://b.tvl.fyi/issues/301 Change-Id: I8d675d4bf1fb697eb7d479747c1b1e3635718107 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9370 Reviewed-by: tazjin <tazjin@tvl.su> Reviewed-by: flokli <flokli@flokli.de> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com> 2023-09-21 21:32:44 +02:00			`runTests = true;`
fix(tvix/castore): also set SSL_CERT_FILE for tests there For everything using reqwest here during test cases, we also need to set SSL_CERT_FILE. Change-Id: If8aeda65f3d75cb9ac5c9bc64e37a0cb7dffc17c Reviewed-on: https://cl.tvl.fyi/c/depot/+/11092 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI 2024-03-03 16:46:20 +01:00			`testPreRun = ''`
			`export SSL_CERT_FILE=${pkgs.cacert}/etc/ssl/certs/ca-bundle.crt;`
			`'';`
feat(tvix/castore/directory): add bigtable backend This adds a Directory service using https://cloud.google.com/bigtable/docs/ as a K/V store. Directory (closures) are put in individual keys. We don't do any bucketed upload of directory closures (yet), as castore/ fs does query individually, does not request recursively (and buffers). This will be addressed by store composition at some point. Change-Id: I7fada45bf386a78b7ec93be38c5f03879a2a6e22 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11212 Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de> 2024-03-19 11:12:03 +01:00
			`# enable some optional features.`
			`features = [ "default" "cloud" ];`
test(tvix/[ca]store): enable bigtable only with "integration" feature The emulator and bigtable client are quite big. Remove them from the default //tvix:shell. Put the tests behind a `integration` feature flag, and add a variant with that enabled to CI, and drop the bigtable tools from //tvix:shell. Change-Id: Ie042097a0d6fc26542faa96c139b77298ccb160a Reviewed-on: https://cl.tvl.fyi/c/depot/+/11582 Reviewed-by: edef <edef@edef.eu> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI 2024-05-02 23:09:46 +02:00			`}).overrideAttrs (_: {`
			`meta.ci.targets = [ "integration-tests" ];`
			`passthru.integration-tests = depot.tvix.crates.workspaceMembers.tvix-castore.build.override {`
			`runTests = true;`
			`testPreRun = ''`
			`export SSL_CERT_FILE=${pkgs.cacert}/etc/ssl/certs/ca-bundle.crt;`
			`export PATH="$PATH:${pkgs.lib.makeBinPath [pkgs.cbtemulator pkgs.google-cloud-bigtable-tool]}"`
			`'';`

			`# enable some optional features.`
			`features = [ "default" "cloud" "integration" ];`
			`};`
			`})`