This is slightly better than the (mostly untested) mess we had before:
Just implement the one thing we need using the tools the one
implementation we support (SBCL) gives us.
Eventually, we'll want to make this portable, probably using osicat.
Unfortunately, packaging this requires support for cffi-grovel (b/383)
which buildLisp lacks at the moment.
Change-Id: I6960015f80e6a5dfde67baf55537c5274a19e4e2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11356
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Autosubmit: sterni <sternenseemann@systemli.org>
Let's make this usable for the entire crate.
Change-Id: I754408908a00296ee80dd52680f84b8a7cb22317
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11354
Tested-by: BuildkiteCI
Reviewed-by: Brian Olsen <me@griff.name>
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: picnoir picnoir <picnoir@alternativebit.fr>
`appendContext s ctx` will just append a user-crafted context attrs to `s`.
The most important part of this builtin is to perform all the relevant invariant validations
to avoid letting the user craft invalid contexts which can never be built, e.g. invalid store paths,
inexistent derivations, etc.
This version is incomplete and full of TODOs, but passes all the Nix's
context strings tests, so we turn them on.
Change-Id: I625dc5e7c4f5b784f078b390f04b0ee5a8d65a7c
Signed-off-by: Ryan Lahfa <tvl@lahfa.xyz>
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11263
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Fixes b/392.
Output paths were created, depending on a plain store path but no
context string was attached to track that plain dependency.
Context string propagation tests are strengthened to prevent any
regression on this.
Change-Id: Ifd6671aeba6949324b0bb9f0f766b87db728d484
Signed-off-by: Ryan Lahfa <tvl@lahfa.xyz>
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11351
Reviewed-by: Alyssa Ross <hi@alyssa.is>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Remove potential partial reads/writes and instead read/write the full
buffer size: we want those to be 64 bits.
Change-Id: I1f767baf23fa80c2babb8113f61d1a9e72a8d8dd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11350
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
- Set pipefail to make sure the correct exit code gets returned.
- Don't let xargs(1) call realpath(1) if the build fails (and nothing is
printed to stdout).
Change-Id: I526a18df877601d884b83862da5f37336cdf52cd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11312
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Adding a verbosity flag available through the CLI/ENV variable.
Change-Id: If04cc2e6e26e7cb3c2df7821fce222da2b85a95a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11349
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Actually these are all u64 LE encoded on the wire.
Change-Id: I5ca22c7639607ac47117cd946e036a444271885a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11348
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Using all the primitives recently implemented to nix-compat to reach
the point where the Nix client start to send us operation requests.
Using a small integration test script (or the VM test, but let's face
it, it's too slow to be useful), we manage to reach the point where
we're able to read a store operation:
2024-03-21T18:53:27.624876Z INFO tvix_daemon: Incoming connection addr=unix
2024-03-21T18:53:27.625312Z INFO worker:perform_init_handshake: tvix_daemon: Trust sent conn=Connection(unix) conn=Connection(unix)
2024-03-21T18:53:27.625406Z INFO worker: tvix_daemon: Client hanshake succeeded conn=Connection(unix)
2024-03-21T18:53:27.625488Z INFO worker: tvix_daemon: Operation received op=SetOptions conn=Connection(unix)
We had to take some shortcuts wrt. stderr/log management. The CPP Nix
codebase is a bit confusing in that area. I'll need to spend more time
reading this to fully understand what's happening there. For now,
sending the STDERR_LAST command to the client does the trick.
Change-Id: I9b0e20a52d885e64fe29188496aac5334de61edd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11233
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
This is used by the nix client to determine whether or not the daemon
trust it. The trust conditions check are daemon-specific, hence not
part of nix-compat.
Change-Id: Icbcba2f7f1fd58f67e7da72d22a264f5a3f3619d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11231
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Write counterpart of read_bytes. Despite its name, we mostly use it to
write strings (as in ascii strings) to the wire.
We also extract the padding calculation in its own function.
Change-Id: I8d936e989961107261b3089e4275acbd2c093a7f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11230
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Now, it supports almost everything except `recursive = false;`, i.e. `flat`-ingestion
because we have no knob exposed in the tvix store import side to do it.
This has been tested to work.
Change-Id: I2e9da10ceccdfbf45b43c532077ed45d6306aa98
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10597
Tested-by: BuildkiteCI
Autosubmit: raitobezarius <tvl@lahfa.xyz>
Reviewed-by: flokli <flokli@flokli.de>
Instead of enforcing NAR SHA256 all the time, we generalize the
`PathInfo` constructor to take a `CAHash` argument which can drive
whether we are having a flat, NAR or text scheme.
With this, it is now possible to implement flat schemes in our
evaluation builtins, e.g. `builtins.path`.
Change-Id: I15bfee0ef4f0f428bfbd2f30c57c012cdcf6a976
Signed-off-by: Ryan Lahfa <tvl@lahfa.xyz>
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11286
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Update all 3p/sources as we do normally except
- agenix which is still pinned to 0.15.0
- nixpkgs (unstable) which we bump to the HEAD of the staging-next
branch. This branch includes the downgrade of xz from 5.6.1 to
5.4.6 (https://github.com/nixos/nixpkgs/commit/d6dc19adbd). It
also includes the second haskell-updates rotation with GHC 9.6.4
which contains a few build fixes that seem to be required to get
our Haskell targets to work.
Note that this only reverts xz to a version that doesn't contain the now
known backdoor (CVE-2024-3094) which may or may not actually affect
NixOS. Additionally reverting to a version before the malicious
contributor's involvement may be difficult, but prudent:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1068024
Changes required by the updates:
- //3p/overlays/haskell:
- Update ihp-hsx to latest master to fix build with Stackage LTS 22.
- Update tmp-postgres to latest master to work around failure with
ansi-wl-pprint >= 1.
- Patch punycode for mtl >= 2.3.
- //users/Profpatsch:
- Clean up some warnings, mostly about unused dependencies
- my-prelude: Fix build with ghc-boot-9.6.4
- cas-serve: Use crypton over unmaintained cryptonite
- ical-smolify: skip in ci, iCalendar would require heavy patching to
work with Stackage LTS 22.
- //users/{wpcarro,aspen,flokli}:
Disable home-manager / nixos configuration builds that seem to have
transient failures that should disappear as we move away from
staging-next and closer to an actual channel release.
Change-Id: I5cca48e101041c3aedc1d9932dbca2cac885fcc1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11289
Tested-by: BuildkiteCI
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
We just had a minor incident where apparently our build cache for the
critical security fix was deleted by automatic-gc (which I had stopped
manually) being reenabled by an unrelated whitby deploy.
This adds a new mechanism where by touching a file called
`/run/stop-automatic-gc` the GC can be prevented from running.
We might want to configure an occasional alert or something if this
file exists, so we don't forget about it when we are using it.
Change-Id: I041e57e24b2b684696164a2d516581d7f5696ef0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11326
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: flokli <flokli@flokli.de>
- Actually define *static-dir* at build time, to get the search.png in
the search box
- Better logging for migration running at startup time
- Fix and-where to properly exclude nil clauses
- fix looking up build-time vars in the :build package
Change-Id: Ia2ef3b2715d4c2efb62bbb2c72084f0f0ad09562
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11325
Autosubmit: aspen <root@gws.fyi>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
For local dev, it's nice to be able to set the oauth variables by hand
before starting the server - in that case, we don't want to override
them with the defaults.
Change-Id: I530518b4d58df1edf55ce062cf7ff3290784a3ed
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11259
Autosubmit: aspen <root@gws.fyi>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Support basic full text search of issues using postgresql's built-in
text search. There's a new column on the issues table called `tsv`,
which contains a tsvector of the title concatenated with the
description, and a new search form on both the index and closed issues
page which allows searching that tsvector with a user-supplied query.
Results are ranked according to that text query in the case of a search.
This works fine for now, but next up I'd also like to highlight the
results according to the bits that matched the user's query.
Change-Id: I25170bedbbbcdc3ed29a047962e9fcfe280d763a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11258
Autosubmit: aspen <root@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
This is needed for the GENERATED ALWAYS AS...STORED stuff we'll be
doing for text search later on, but also just matches what's being run
in prod
Change-Id: I076ab88c9735f29b5f669107a339f99f80a00cc1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11257
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Add a system for writing, running, and tracking database
migrations (changes to the database schema) over time, inspired by but
significantly simpler than postmodern-passenger-pigeon.
Migrations can be generated by
running (PANETTONE.MODEL:GENERATE-MIGRATION "name"), and are numerically
ordered lisp files that define (at least) a function called UP, which
runs the migration. The migrations that have been run against the
database are tracked in the `migrations` table, and when the
`(PANETTONE.MODEL:MIGRATE)` function is called (as it is on startup),
all migrations that have not yet been run are run within a transaction.
This includes one migration `1-init-schema.lisp`, which migrates the
database (idempotently) to the current state of the schema.
Change-Id: Id243a47763abea649784b12f25a6d05c2267381c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11253
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
My new laptop has the battery at index 1, not index 0
Change-Id: I64fa16e7f0eecf6b623361645d73b1176e1e5d48
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11311
Tested-by: BuildkiteCI
Autosubmit: aspen <root@gws.fyi>
Reviewed-by: aspen <root@gws.fyi>
The expression needs updating on our side to be compatible with the
nixpkgs bump in cl/11289. It wasn't used anywhere in TVL either.
This is also packaged in nixpkgs, since
f3ea5cfa6ec0a640735f4d6a0b5c91683c9bab25 / Feb 20 2024, so using it from
there feels reasonable.
Change-Id: Icd50ad959ca0de433b847180f1a5be308719f287
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11295
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Bump to the latest u-root release. This seems to be compatible with go
1.22, too, which previously segfaulted the compiler in cl/11289.
Change-Id: Ie8a0d3cb51a435d6a0b194b38a9c83ee260618df
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11291
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Only those that want them
Change-Id: Id1caf5e89f69c51a7da90ff2387a9384abbf4f91
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11308
Autosubmit: aspen <root@gws.fyi>
Reviewed-by: aspen <root@gws.fyi>
Tested-by: BuildkiteCI
For the rare times I still occasionally use vim
Change-Id: Ie3059ab0ec5d5ace729e78b765f251d1fafdfac1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11305
Reviewed-by: aspen <root@gws.fyi>
Autosubmit: aspen <root@gws.fyi>
Tested-by: BuildkiteCI
This is not necessary now that I'm using tailscale everywhere
Change-Id: I36d8060d4ad5f6955cd18c68b5fc667e64669004
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11303
Reviewed-by: aspen <root@gws.fyi>
Autosubmit: aspen <root@gws.fyi>
Tested-by: BuildkiteCI
I never got this working
Change-Id: Iacb68a7d6d5fe1d1f1f390d09f106c4eb9a51b6a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11301
Reviewed-by: aspen <root@gws.fyi>
Autosubmit: aspen <root@gws.fyi>
Tested-by: BuildkiteCI
These are noisy, but more importantly I'm worried I might accidentally
commit a secret at some point
Change-Id: If6f2c358f2803af25ea27ef34d39c7f2108d4186
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11299
Reviewed-by: aspen <root@gws.fyi>
Autosubmit: aspen <root@gws.fyi>
Tested-by: BuildkiteCI
This builtin only transforms any `NixContextElement::Derivation` into the trivial `NixContextElement::Plain`.
This is a forgetful functor on derivation-deep context strings.
The test coverage of this change is done in cl/11264.
Change-Id: Icd00778c97766be6db8a6bdabaa59e9724353ec5
Signed-off-by: Ryan Lahfa <tvl@lahfa.xyz>
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11262
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Derivation that depends on `${d.drvPath}` generates a `NixContextElement::Derivation(drvPath)` context string.
In turn, this makes the dependent derivation depend on *ALL* outputs of that derivation.
Note that a dependency on `${d.drvPath}` generates an input source
dependency too.
This is a complete implementation of the context string system in Nix on
the "input population" side.
The test coverage of this change is taken care in cl/11264.
Change-Id: I97fe5f7c772a6b1cc4366bee071aa691a11fcde6
Signed-off-by: Ryan Lahfa <tvl@lahfa.xyz>
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11261
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
