Commit graph

274 commits

Author SHA1 Message Date
sterni
7df7cd6257 feat(nix/buildLisp): pass implementation description instead of name
Instead of using a string to refer to an internal set defined in
buildLisp, we just expose the relevant sets (as nix.buildLisp.sbcl,
nix.buildLisp.ecl, …) and receive them as the `implementation`
argument directly. This has several advantages:

* It becomes easier to extend buildLisp, even for downstream users:
  Since you can just pass your own set, there's nothing stopping you
  from adding support for another implementation in a downstream
  derivation without having to edit the buildLisp file in any way which
  is great if you're using e. g. builtins.fetchGit to import it.

* Users can mess with the implementation set by changing out some parts
  of it for customization purposes. Note that currently the sets use a
  lot of self-references which aren't even bound by a fix-point, so to
  make this work smoothly, we'd need to add some overriding mechanism.

* The buildLisp code becomes quite a bit clearer. Since we're now always
  dealing with the implementation set, the confusing distinction between
  `impl`, `impl.name` and `implementation` no longer exists. `impl` is
  now exclusively an abbreviation of `implementation` (we could make
  this more consistent in the future even).

Change-Id: I36d68069dd1315610b2f7159941507b465469b7c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3368
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2021-08-24 22:00:15 +00:00
sterni
d7e70b1d72 feat(nix/buildLisp): add ccl
This adds support for Clozure's CL implementation to buildLisp. This is
quite trivial in comparison to ECL since SBCL and CCL have very similar
in how they work (so much so that CCL also suffers from b/136).

Also the similarities in the code actually added here are striking, so
I'll try to make an effort to reduce the code duplication in the
future.

To fix builds with CCL the following changes were made:

* //3p/lisp/nibbles: The double inclusion of the types.lisp file was
  fixed. CCL doesn't like double definitions and refuses to compile
  otherwise.

* //3p/lisp/physical-quantities: Update to a new bug fix release which
  contains a compilation fix for CCL.

* //3p/lisp/routes: apply a patch fixing the build which was previously
  failing due to a double definition.

* //3p/lisp/usocket: only depend on sb-bsd-sockets for SBCL and ECL, the
  latter of which seems to have a SBCL compatible implementation of the
  package.

* Conditionally include a few CCL-specific source files and add
  `badImplementation` entries for the remaining failures which are
  //fun/gemma (to be expected) and //web/panettone which fails with an
  incredibly vague message.

Change-Id: I666efdc39a0f16ee1bb6e23225784c709b04e740
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3350
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2021-08-24 22:00:15 +00:00
sterni
02566cdcfb feat(nix/buildLisp): add ecl
Adds ECL as a second supported implementation, specifically a statically
linked ECL. This is interesting because we can create statically linked
binaries, but has a few drawbacks which doesn't make it generally
useful:

* Loading things is very slow: The statically linked ECL only has byte
  compilation available, so when we do load things or use the REPL it is
  significantly worse than with e. g. SBCL.

* We can't load shared objects via the FFI since ECL's dffi is not
  available when linked statically. This means that as it stands, we
  can't build a statically linked //web/panettone for example.

Since ECL is quite slow anyways, I think these drawbacks are worth it
since the biggest reason for using ECL would be to get a statically
linked binary. If we change our minds, it shouldn't be too hard to
provide ecl-static and ecl-dynamic as separate implementations.

ECL is LGPL and some libraries it uses as part of its runtime are as
well. I've outlined in the ecl-static overlay why this should be of no
concern in the context of depot even though we are statically linking.

Currently everything is building except projects that are using cffi to
load shared libaries which have gotten an appropriate
`badImplementations` entry. To get the rest building the following
changes were made:

* Anywhere a dependency on UIOP is expressed as `bundled "uiop"` we now
  use `bundled "asdf"` for all implementations except SBCL. From my
  testing, SBCL seems to be the only implementation to support using
  `(require 'uiop)` to only load the UIOP package. Where both a
  dependency on ASDF and UIOP exists, we just delete the UIOP one.
  `(require 'asdf)` always causes UIOP to be available.

* Where appropriate only conditionally compile SBCL-specific code and
  if any build the corresponding files for ECL.

* //lisp/klatre: Use the standard condition parse-error for all
  implementations except SBCL in try-parse-integer.

* //3p/lisp/ironclad: disable SBCL assembly optimization hack for all
  other platforms as it may interfere with compilation.

* //3p/lisp/trivial-mimes: prevent call to asdf function by substituting
  it out of the source since it always errors out in ECL and we hardcode
  the correct path elsewhere anyways.

As it stands ECL still suffers from a very weird problem which happens
when compiling postmodern and moptilities:
https://gitlab.com/embeddable-common-lisp/ecl/-/issues/651

Change-Id: I0285924f92ac154126b4c42145073c3fb33702ed
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3297
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: eta <tvl@eta.st>
2021-08-24 22:00:15 +00:00
sterni
0285ea7eac feat(nix/buildLisp): expose drvs built w/ the other implementations
For every implementation we support an extra passthru attribute with the
name of the implementation is created which points to a version of the
derivation built with that implementation. E. g. if we support CCL, ECL
and SBCL, third_party.lisp.alexandria would have:

* third_party.lisp.alexandria.sbcl
* third_party.lisp.alexandria.ecl
* third_party.lisp.alexandria.ccl

To make this possible, the REPL derivation which was called `sbcl`
originally has been renamed to `repl`.

Since some things won't build with all implementations, we introduce a
brokenOn argument which influences the meta.targets list that is
created, but won't prevent the passthru attrs from being created to
ease debugging failures.

Change-Id: Icd6af345143593fac30ded10deabf31172e5d48a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3359
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2021-08-24 22:00:15 +00:00
sterni
ee6b2003fc feat(nix/buildLisp): implementation specific deps and srcs
Both the deps and srcs arguments may now have special “filter sets” in
the lists they receive as arguments. When building, buildLisp checks if
such sets either have a attribute named like the current implementation
or a "default" attribute. If yes, the set is replaced by the respective
attribute's value. If no, the set is removed from the list without
replacement.

This can be used to add elements for (a) specific implementation(s):

  { sbcl = buildLisp.bundled "sb-posix"; }

  { sbcl = ./sbcl/optional-sbcl.lisp; }

or to switch between files for different implementations:

  # If a implementation case is missing and no default set present,
  # no file will be added. Compilation will likely fail as a result.
  {
    ecl = ./tf-ecl.lisp;
    ccl = ./tf-ccl.lisp;
    sbcl = ./tf-sbcl.lisp;
  }

or to account for special behavior for a certain implementation:

  {
    ccl = ./ccl-quirk-impl.lisp
    default = ./ansi-impl.lisp;
  }

Change-Id: I082c3701d1f5063b92100bf336a83425471c269d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3321
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2021-08-24 22:00:15 +00:00
sterni
acb5994e87 feat(nix/buildLisp): allow implementation-specifc bundled functions
By implementing a bundled function for an implementation, we can use a
custom one for a specific implementation. This is useful for
implementations like ECL where a require will be compiled as an
instruction rather than importing all new symbols into a dump, so using
the underlying static or shared object directly would be beneficial.

overrideLisp for bundled libraries now only allows overriding the name
and implementation arguments.

Change-Id: I9036b29157e8daa4d86ff87d603b044373711dbf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3301
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2021-08-24 22:00:15 +00:00
sterni
d344637fe2 refactor(nix/buildLisp): prepare multi implementation support
Concept is roughly:

* receive extra argument `implementation` that refers to the name of an
  implementation or rather an attribute in an internal attribute set
  telling buildLisp how to do certain build steps.

* We assume an implementation can execute lisp files as scripts and that
  we can implement the following main tasks in lisp:

  - Building a library (`genCompileLisp`)

  - Building an executable (`genDumpLisp`)

  - Loading a library dynamically (`genLoadLisp`)

  Based on that we can implement:

  - Running a test suite (`genTestLisp`)

  - A REPL preloaded with a libraries and their dependencies (`lispWith`)

  Additional attributes for implementing these parts genericly  are
  added as needed (`faslExt` and `runScript`).

* `genCompileLisp` no longer prints a shell script which concatenates
  the individual FASLs. Instead it does the step previously done by the
  shell script itself. In essence `genCompileLisp` now writes a lisp
  script which compiles and installs the library to build.
  This will allow us extra freedom for different implementations, e. g.
  for ECL we'll want to build a object file archive additionally to fasl
  files in order to be able to link proper executables.

* `genLoadLisp` and `genTestLisp` are almost generic (the former just
  sometimes would need to use different file extensions), but we
  integrate them into the implementation “API” to facilitate minor
  tweaks we need to do like the `fasc` extension for ECL's native FASL
  files.

Change-Id: I1b8ccc0063159638ec7af534e9a6b5384e750193
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3292
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2021-08-24 22:00:15 +00:00
sterni
32de4cbd93 refactor(users/grfn/gws.fyi): implement isDirectory in pure nix
Another day, another import from derivation avoided by
builtins.unsafeDiscardStringContext!

Change-Id: I67274b1ba13ba980bb3346b22f2955c702aa3151
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3372
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: tazjin <mail@tazj.in>
2021-08-24 12:13:06 +00:00
sterni
59195e76fa docs(nix/buildLisp): mention drawback of non-parallel compilation
To be fair this hardly matters since SBCL is quite fast, but compiling
ironclad with ECL is quite the experience…

Change-Id: Ib89cc50e5d557acec51fdb085bcbdfc99736221e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3342
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2021-08-12 15:08:15 +00:00
sterni
0298ec862e fix(nix/buildLisp): remove misplaced makeOverridables
`makeOverriddable` doesn't work for bundled sbclWith as is because it
uses the `//` operator internally which doesn't work with the types
`bundled` and `sbclWith` accept as arguments (string and list
respectively).

What's more, `bundled` already uses `makeOverridable` and allows to
override the internal call to `library` via `overrideLisp`. For
`sbclWith` no such mechanism exists, but this seems to be no concern for
now: Using `overrideLisp` for this hasn't worked so far (and failed with
a _hideous_ evaluation error), so there doesn't seem to be any real
demand for this feature. Maybe a feature for another CL.

Change-Id: I0b2f34c00a2143cd66dd43a6b1b2880af997ee50
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3296
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2021-08-10 14:31:52 +00:00
sterni
c8f521e96a refactor(nix/buildLisp): use passthru for extra attributes
Using passthru and appending the attributes via `//` have the same
effect with a subtle difference: In the latter case re-evaluating
the derivation when using the underlying `mkDerivation`'s
`overrideAttrs` will delete all appended attributes. Using
passthru at least preserves the attributes although the self
reference to the derivation in `passthru.sbcl` will become
outdated (unless updated manually).

Change-Id: I8b85009f386b9375b86a23fd50c4ec8c6a9dea7f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3257
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: grfn <grfn@gws.fyi>
2021-08-01 23:37:03 +00:00
Profpatsch
72924facae fix(nix/writers/rust): remove args override
We can be closed world, so let’s restrict the arguments to the subset
we need for now.

The existing override was wrong, in that `// args` would use the
arguments we already added, again. So instead of deliberating about
how to make this work right in all cases, we don’t need it, we trim
it.

Change-Id: I6443a0808b8bfd5e4db939b669c6afc741954db8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3057
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2021-05-17 23:00:57 +00:00
Profpatsch
713f066297 feat(users/Profpatsch/blog): implement blog engine lol
My horrible blog engine v0.0.0.0.1. Don’t judge.

Change-Id: I427b4e84d67aa49793cb7625e4d8ca2824f00943
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2456
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
2021-04-30 21:47:42 +00:00
Profpatsch
eb41eef612 chore(nix): move rustSimple from users.Profpatsch.writers
I think it’s solid enough to use in a wider context.

Change-Id: If53e8bbb6b90fa88d73fb42730db470e822ea182
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3055
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: lukegb <lukegb@tvl.fyi>
2021-04-24 10:23:55 +00:00
Profpatsch
7e888c3c7b feat(nix): add basic netstring nix generation functions
Moving to toplevel so I can use them with `runExecline`. They should
be pretty atomic, and are proven to work (tests are still in my user
dir, since they test the producers indirectly via the python parser
and I don’t want to pull it out right now).

Change-Id: Id0baa3adcb2ec646458a104c7868c2889b8c64f5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3054
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
2021-04-24 09:54:34 +00:00
Profpatsch
2224731e0d feat(nix/buildGo): add the import position to errmsg
When a foreign dep is missing a dependency, it is good to have a
context.

e.g. the `github.com/charmbracelet/bubblegum` package has a lot of
dependencies that are only used in its `examples/` dir; this is not
obvious, unless we also print where the imports come from.

New error message:

```
error: missing foreign dependency 'github.com/containerd/console' in 'github.com/charmbracelet/bubbletea, imported at /nix/store/0cry4sg6bzxqwk5zl2nxhas6k5663svg-source/tea.go:22:2'
```

Change-Id: If34a3c62b9d77d4aea108b5e011e16fbd03e8554
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2852
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2021-04-23 16:35:57 +00:00
sterni
f88ac5c0b5 feat(nix/utils): add storePathName, a more generic baseNameOf
This is a wrapper around baseNameOf which also can deal with
derivations. Added to //nix/utils since I've found myself introducing an
ad-hoc implementation of this for both //web/bubblegum and //nix/buildC.

Change-Id: I2fcd97a150d6eda21ab323fa0d881ff7442a892e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3049
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2021-04-20 11:00:52 +00:00
Vincent Ambo
f520bd40ca refactor: Replace 'depotPath' with 'depot.path'
Instead of having two ways of accessing the path to the depot (one of
which was stuttering, depot.depotPath) we settle on only one:
depot.path.

This was mostly used for NixOS module imports.

Co-Authored-By: Florian Klink <flokli@flokli.de>
Change-Id: I2c0db23383fc34f6ca76baaad4cc4af2d9dfae15
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2962
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: sterni <sternenseemann@systemli.org>
2021-04-12 21:55:07 +00:00
Vincent Ambo
a559135970 refactor(readTree): Initialise repo roots without recursing
Plumbs an additional internal argument through readTree that indicates
whether the top-level of a tree is being read, and avoids recursing
into itself in that case. This changes the externally visible
behaviour of readTree (it is now expected to be called a level higher
than previously).

This allows us to reduce the amount of boilerplate needed to bootstrap
the TVL repository (by not having to specify the individual folders
that need to be read).

For reasons related to an infinite recursion we could not (be bothered
to) debug, the top-level `config` key (which held the attribute set
passed on by readTree) has been removed. This is not needed, as it is
already passed on by readTree ...

Co-Authored-By: Florian Klink <flokli@flokli.de>
Change-Id: Id6e39b57b2f5b3473c4b695a72dd1d01fcfb7a66
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2961
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
2021-04-12 21:55:07 +00:00
Vincent Ambo
f39b5a8a0d docs(readTree): Add SPDX license header on readTree
Change-Id: I9a6eafa7d4bb3d590dfa35d368adfd25aeed7f64
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2936
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
2021-04-12 13:56:44 +00:00
sterni
8d4b2f3d54 refactor(sterni): use pkgs over third_party to import from nixpkgs
This should ease migrating to a distinction between depot.third_party
and pkgs (as in nixpkgs) in the future.

Ref cl/2910, b/108.

Change-Id: I53a854071fddd7c0d0526cc4c5b16998202082c6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2913
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2021-04-10 11:40:18 +00:00
sterni
811fff2d8b feat(nix/buildManPages): infra for generating man page dirs from nix
Very simple builder which builds (optionally) gzipped man pages from a
list of attrsets and links them into a common man directory with the
correct layout, so it should be installable immediately.

Additionally runs mandoc -T lint, but by default only for informational
purposes as it is very strict and some things are almost never true (for
example all Xrs being present in the respective directory).

buildManPages.single exposes the internal builder for a single,
optionally gzipped man page from a nix attrset.

Change-Id: I43fce011716f4a7cc80521f222800ca99ba54060
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2654
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2021-03-31 23:06:10 +00:00
Profpatsch
d0b635f6f2 fix(nix/runTestsuite): use s6-portable-utils
Same as 221698c603dcb318c609b4d21cb2a9fada44a14c

We had a bunch of instances of
https://github.com/NixOS/nix/issues/2176,
where nix would exit with a “killed by signal 9” error.

According to Eelco in that issue, this is perfectly normal behaviour
of course, and appears if the last command in a loop closes `stdout`
or `stdin`, then the builder will SIGKILL it immediately. This is of
course also a perfectly fine error message for that case.

It turns out that mainly GNU coreutils exhibit this behaviour …

Let’s see if using a more sane tool suite fixes that.

Change-Id: Iaf9e542952ca36c02208a3f067f575ba978272b4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2663
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
2021-03-26 11:01:20 +00:00
Vincent Ambo
d34c527372 refactor: Replace some uses of builtins.toFile with pkgs.writeText
I'm looking at removing some of these because they can cause
unnecessary build steps during CI pipeline generation.

Change-Id: I84742968918090c050d2eedab8a1b42692632a42
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2655
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2021-03-25 19:14:36 +00:00
sterni
5d8490d2fc feat(users/Profpatsch): build attrset members on CI
Setting meta.targets to include all derivations in the different package
sets in Profpatsch's user folder makes them checked by CI until they do
the readTree refactor as promised.

To reduce code duplication we handle this in a simple function which is
exposed from nix.utils which may be a good place for depot specific bits
and bops we accumulate over time.

To get around the issue of too nested sets we perform the following
renames:

* users.Profpatsch.tests gets moved into its own directory
* users.Profpatsch.arglib.netencode now lives in its own file instead of
  the default.nix
* users.Profpatsch.netstring.tests gets moved into its own directory

Change-Id: Icd039c29d7760a711c1c53554504d6b0cd19e120
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2603
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
2021-03-15 22:16:19 +00:00
Luke Granger-Brown
2974d4b4b6 fix(buildGo): Fix stdlib list under Go 1.16.
Go 1.16 makes "go list all" not work. "go list std" is what we should be
using instead anyway.

Change-Id: I3f867fde477030d2358085b3d64b5856fb9c421b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2551
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2021-02-24 20:44:18 +00:00
Profpatsch
83e81def23 feat(nix/readTree): give better error message when not a function
When a file is added to the depot tree that is picked up by read-tree,
but it’s not a function like ({...}: {}), `readTree` will fail on the
function application, leading to a bad error message.

We can do slightly better, by checking the type and throwing a nicer
trace message.

`assertMsg` is copied from `nixpkgs/lib/assert.nix`, since at this
point we don’t have a reference to the lib.

There is another evaluation failure that can happen, which is when the
function we try to call does not have dots; however, nix does not
provide any inflection capabilies for checking whether a function
attrset is open (`builtins.functionArgs` only tells us the attrs it
mentions explicitly). Maybe the locality of the error could be
improved somehow.

Change-Id: Ibe38ce78bb56902075f7c31f2eeeb93485b34be3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2469
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2021-02-19 23:04:04 +00:00
sterni
690994a28c fix(nix/yants): make (typedef …).checkType return a result set
Previously, for types defined using typedef (like all primitive types)
type.checkType would return a boolean. This is largely fine since in
most places `type.checkToBool (type.checkType x)` or similar is used.
However, some functions actually take type.checkType up on the promise
that it returns a set of the form:

  {
    ok = <bool>;
    err = <option string>;
  }

This is the case for restrict which has checkToBool = v: v.ok; and will
generate a proper set except if `t.checkToBool (t.checkType v) == false`
in which case it will return t.checkType v. If t was a primitive type or
defined using typedef, previously `t.checkType v` would be a boolean
which meant as soon as (restrict …).checkToBool was called on a restrict
checkType result in cases where the wrapped type didn't match, an
unrelated error would be thrown:

  nix-repl> with nix.yants; restrict "foo" (_: true) int "lol"
  error: value is a boolean while a set was expected, at /home/lukas/src/depot/nix/yants/default.nix:38:39

This is fixed by making typedef return a proper set from checkType and
adjusting its checkToBool accordingly.

Unfortunately I don't think we can easily add test cases for this except
by using recursive nix or VM tests as there is no way to introspect
error messages.

Change-Id: I96a7be065630f04ca33358f21809284911ec14fe
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2536
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: Profpatsch <mail@profpatsch.de>
2021-02-19 16:14:55 +00:00
sterni
2469f0b423 fix(nix/readTree): make nixFileName only match .nix extensions
Nix unfortunately has terrible escaping syntax: If something is an
escape sequence it does not know, like \0, it just swallows the
backslash and returns the second character (byte) as is (yes,
"\0" == "0" is true). This created the following bug in nixFileName
which should have resulted in at least a parse error: "(.*)\.nix" is
the same as "(.*).nix" which meant that nixFileName matched anything
that is at least 4 characters long and ends in "nix". This lead to
readTree creating double attributes when directories are involved or
attributes for non-nix files.

Change-Id: Ibf3be2bd189d48881c82ee795c50151bfb365627
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2535
Reviewed-by: tazjin <mail@tazj.in>
Tested-by: BuildkiteCI
2021-02-18 00:07:14 +00:00
sterni
abbc538ef3 feat(nix/drvSeqL): make build of a drv depend on unrelated drvs
This adds the star of NixCon 2017 from vuizvui, slightly reformatted and
now using yants. While it has some flaws, I realized that it is ideal to
run the tests of rustSimple{Lib,Bin} where the normal and the -tests
variant would have to be rebuilt if either the tests or the library /
executable itself changes.

Change-Id: Ie8f84f98c51c9fafc046eff916c8f0df7e8f224b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2528
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
2021-02-14 12:40:06 +00:00
sterni
7bd43d15d9 feat(nix/yants/tests): test drv type
By using an extremely trivial derivation we can ensure that it will not
throw if evaluated using deepSeq. When using stdenv.mkDerivation or
similar at some point something will most likely throw or generate some
kind of error which is alright in the context of nixpkgs, but makes
testing yants harder than you'd think it should be.

Change-Id: I61ff7dc01a00a4815ef39066e4e223123356ddd5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2507
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2021-02-12 21:52:36 +00:00
sterni
58020730cd feat(nix/yants/tests): port to runTestsuite
Port existing tests to runTestsuite and add some obvious additional
tests that wouldn't be possible before (using assertThrows and
assertEq).

Change-Id: Ibe950a7a0cda3e23ebb226bdff35f52cdfec5ddf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2479
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
Reviewed-by: tazjin <mail@tazj.in>
2021-02-09 21:57:56 +00:00
sterni
fde23c5d0a feat(nix/runTestsuite): add assertDoesNotThrow
assertDoesNotThrow is like assertThrows, but fails if the expression
throws. In that case the new unexpected-throw branch of AssertErrorContext
is returned.

Change-Id: I7195eb5df8965456e9ab9b69e35ec96b33f00a35
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2476
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
2021-02-09 21:57:56 +00:00
sterni
00f79da358 refactor(nix/runTestsuite): implement in terms of generic assertBool
Refactor assertEq and assertThrows to be implemented in terms of a more
generic assertBool to reduce code duplication and ease adding new assert
types.

To preserve meaningful error messages AssertResult is changed slightly:
nope-eq and nope-throw have been replaced by a single nope branch which
contains an AssertErrorContext which contains error information. To
implement an assert assertBoolContext (which is not exposed) can be
used: It takes an AssertErrorContext which is returned in case of an
error and a boolean determining whether the assert was successful.

The currently possible AssertErrorContext are:

* should-throw: error result of assertThrows, formerly nope-throw
* not-equal: error result of assertEq, formerly nope-eq

Change-Id: Ifd6b3aa4187c90c3add2df63fa7c906c8f03fd2d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2473
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
2021-02-09 21:57:56 +00:00
Profpatsch
e4a7704583 feat(nix): add nix.tag, a way of discriminating via tagged unions
Introduces the concept of a “tag”, a single-keyed attrset which
annotates a nix value with a name.
This can be used to implement tagged unions (by implying the list of
possible tags is well-known), which has some overlap with how
`nix.yants` does it.

However, the more fascinating use-case is in concert with a
so-called discriminator, `match` and hylomorphisms.

The discriminator can take a nix value, and add tags to it based on
some predicate.
With `match`, we can then use that information to convert the
discriminated values again.
With `hylo`, we can combine both the “constructive” discriminator step
with the “destructive” match step to recursively walk over a nix data
structure (based on a description of how to recurse, e.g. through attrset
values or list values), and then apply a transformation in one go.

Change-Id: Ia335ca8b0881447fbbcb6bcd80f49feb835f1715
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2434
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2021-01-30 15:41:08 +00:00
Profpatsch
5da7ee9e8a feat(nix/readTree): add tests of the traversal logic in README
This should closely match the documented behaviour. It might still be
missing some edge cases of course.

Change-Id: I5c75fa045d5f3be8cf5eab787a02644500c14522
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2466
Reviewed-by: tazjin <mail@tazj.in>
Tested-by: BuildkiteCI
2021-01-30 12:06:54 +00:00
Profpatsch
768f5ec301 feat(nix/readTree): add tests of the README example
First step to slowly giving readTree some coverage, so we can do
refactoring without breaking functionality.

Change-Id: If25a8c0fa9c4ac7472c0473372f10a9326cccaf7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2465
Reviewed-by: tazjin <mail@tazj.in>
Tested-by: BuildkiteCI
2021-01-30 12:06:54 +00:00
Profpatsch
5f9e9a60e8 chore(nix/readTree): move function into __functor
We are going to export some tests under `nix.readTree.tests`, so in
order to do that and still have `nix.readTree` be a function, let’s
move it to `__functor`.

This requires wiring the `args` and `initPath` arguments through
explicitly.

Change-Id: Ife7956b85d35e59c22174b42dcb7cca83ed868ea
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2464
Reviewed-by: tazjin <mail@tazj.in>
Tested-by: BuildkiteCI
2021-01-30 12:06:54 +00:00
Profpatsch
f4a4da134b feat(nix/runTestsuite): add assertThrows
Uses `builtins.tryEval` to check that the expression throws when
`deepSeq`-ed.

Change-Id: I0d57cc37f473bb733f57a1b1c0d889084152fd2f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2463
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2021-01-30 11:44:25 +00:00
Profpatsch
7f091079ce fix(nix/runTestsuite): wrap runTestsuite into derivation
Previously we would throw or return `{}`, which doesn’t integrate
nicely into our CI; thus, let’s wrap it into a derivation which either
fails the build or doesn’t.

Change-Id: I65880d86b8393094661e57a0b32aafe748bf1dd5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2462
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2021-01-30 11:44:25 +00:00
Vincent Ambo
8f57ca92bd chore(3p|nix): Remove typed Go
Nobody has actually done any experimentation with typed Go, so we're
getting rid of it for now - it's causing annoying IFD during build
graph generation.

Change-Id: Ibac3dea98ebed1b3ee08acda184d24c500cf695d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2458
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: lukegb <lukegb@tvl.fyi>
Reviewed-by: Profpatsch <mail@profpatsch.de>
2021-01-30 08:20:45 +00:00
V
29db630a39 chore: Remove banned user
Change-Id: Icd61f7c567a327c74a4f381168e94737b2b30702
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2422
Tested-by: BuildkiteCI
Reviewed-by: edef <edef@edef.eu>
Reviewed-by: tazjin <mail@tazj.in>
2021-01-19 10:30:19 +00:00
Profpatsch
4af195c5f2 feat(nix/yants): add restrict
`restrict` uses a predicate function to restrict a type, giving the
restricting a descriptive name in the process.

First, the wrapped type definition is checked (e.g. int) and then the
value is checked with the predicate, so the predicate can already
depend on the value being of the wrapped type.

Change-Id: Ic3edde45a8f34c31bc164414580d0a1aa5a821d5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2312
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
Reviewed-by: tazjin <mail@tazj.in>
2021-01-03 15:53:18 +00:00
Profpatsch
2f063bc5b0 feat(nix): add runExecline.local
Similar to runCommandLocal, this turns off substitutions and prefers
building locally.

Change-Id: I823b34c7fc54990b54a82324172c299aeffdbf41
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2309
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
Reviewed-by: lukegb <lukegb@tvl.fyi>
2021-01-02 14:24:35 +00:00
Profpatsch
c9b985e7bc chore(OWNERS): move Profpatsch up to nix/
I can review all nix-related changes.

Change-Id: I13e5bb7b523d4b9c79dbe2083d9e23c217466651
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2308
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
2021-01-01 19:03:15 +00:00
Profpatsch
4618b57f74 feat(nix): add mergePatch
Change-Id: Id6a9ecbfb04886e6d96750b1451c29dc3f68154e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2307
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
2021-01-01 19:03:02 +00:00
Vincent Ambo
87f1e15baa chore(writeElispBin): Move from //overrides to //nix
... some legacy cleanup ...

Change-Id: I9e73451256f842e75ddbf1382badaf06c775c755
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2212
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2020-12-01 12:55:59 +00:00
Vincent Ambo
d28d30477a Note that or works for nested attribute access 2020-11-25 13:41:38 +01:00
Vincent Ambo
3236b7ff28 Add small section about or expressions 2020-11-25 13:40:19 +01:00
Vincent Ambo
31939acd6c fix(buildLisp): Override LANG in sbclWith for misconfigured machines
Machines on which LANG is misconfigured have trouble with SBCL loading
files that contain characters in certain encodings. This overrides
whichever local LANG (if any) is set.

Change-Id: Ic4341a01c4393e7f697de6cecc58dea4f2d85987
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2076
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
2020-11-08 18:35:28 +00:00
Vincent Ambo
29613e7494 feat(readTree): Use tree location as the value of __readTree
Change-Id: I6a087279b5d819db475f2179cb77c0f55cbd0ac6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1860
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-08-31 00:46:35 +00:00
Vincent Ambo
dbda84bac3 fix(readTree): Add __readTree attribute to folders
This is required to automatically walk the tree (see subsequent
commits).

Note: Lisp packages are removed from the CI builds in this commit
because the attrValues of third_party.lisp will contain an element
that is simply `true`, which causes a type error.

These packages are re-added when CI refactoring is complete.

Change-Id: I21e2b719e6c7161c23d2867a216f4daa1c6c8394
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1848
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-26 23:49:32 +00:00
Griffin Smith
7a344fbb5e feat(nix/buildLisp): Expose an sbcl attribute
Expose an `sbcl` attribute on packages and programs, to allow for easier
development either with SLY or on a REPL.

Change-Id: Ide4d087a5223561e1fe192ef32dc593c54b5a20e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1834
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2020-08-23 22:21:17 +00:00
Kane York
dfc351b463 feat(tools/depot-scanner): init project
depot-scanner is a tool that runs Nix and parses the --trace-file-access output to deduce what files are necessary to evaluate a derivation.

Take DEPOT_ROOT from the environment. If depotRoot doesn't exist, print an error early.

Fix the build of the protobuf library. Switch to the GRPC build rule, as a service is in this proto file.

Create the PathType enum and parse it from cmdline flags.

Change-Id: I537b5c6bceecf76ca510f7ac04ab9dad7785feb1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1769
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-08-20 03:05:40 +00:00
edef
58e3608e25 feat(nix/readTree): allow arguments to depend on tree location
This accepts a function for args, and passes it an attrset containing
the locatedAt parameter.

Change-Id: I3c0f8ca00605e02a787fda88f32b06f5ef9998f2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1639
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2020-08-04 22:55:45 +00:00
edef
1c96e14174 chore(nix/fetchGoModule): don't move downloaded file
unzip(1) doesn't care about the extension, unlike the unpackFile hook.

Change-Id: Ia9d0e17c4b343ee5d32350329eb6267b9dc35605
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1641
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Reviewed-by: tazjin <mail@tazj.in>
2020-08-04 22:51:51 +00:00
edef
ee627e07f1 chore(nix/fetchGoModule): remove overrideAttrs hack
This was based off fetchzip, but unlike fetchzip we don't need to
support arbitrary archive formats, so we don't actually need the unpack
hooks.

Change-Id: I5655c031f420422f13e72bb79f4b4b95f6f1204e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1640
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Reviewed-by: tazjin <mail@tazj.in>
2020-08-04 22:51:51 +00:00
edef
94387dbc25 chore: simplify buildGo expressions
This removes redundant `map (p: p.gopkg)` expressions.

Change-Id: If25d603c27d63ae011f396caf5d26cdfdb8c28d5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1625
Reviewed-by: tazjin <mail@tazj.in>
Tested-by: BuildkiteCI
2020-08-04 07:43:25 +00:00
edef
596e1f7f77 feat(nix/buildGo): use gopkg attribute of deps
This removes the need for `map (x: x.gopkg)` in dependency lists.

Change-Id: Ia19e51f14110bf22f5cdbb64fa7ce182e1b34241
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1624
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2020-08-04 07:42:53 +00:00
edef
c3546c5f9c feat(nix/buildGo): expose gopkg attribute on buildGo.package
This makes the derivations more uniform, since both third-party
packages generated by buildGo.external and native buildGo.package
expose their libraries as gopkg attributes now.

Change-Id: I547f9860082f36e3300139bf67613eb4fc600d24
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1623
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2020-08-04 07:42:53 +00:00
edef
436145a41c fix(nix/fetchGoModule): correctly case-fold mixed-case package paths
Change-Id: I5a93462a23bf3c9f22fc3de0b173763a9bc8d526
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1591
Tested-by: BuildkiteCI
Reviewed-by: edef <edef@edef.eu>
2020-08-03 10:58:24 +00:00
edef
8d6fcbe1a5 chore(nix/fetchGoModule): Move //users/edef/fetchGoModule to //nix
Change-Id: Ie7f52367ef91c0c8017a36c2a6c7edb06f10e57d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1589
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2020-08-03 10:29:18 +00:00
Griffin Smith
3089f6b6ce feat(nix/buildLisp): Add abstraction for test suites
Add support for explicitly specifying tests as part of a buildLisp
program or library.

Change-Id: I733213c1618f0fa60f645465560bce0522641efd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1481
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2020-07-27 14:18:32 +00:00
Griffin Smith
2b7fe6f957 fix(nix/buildLisp): Set LANG to C.UTF-8 on program builds
This was already happening for libraries, but not for programs - as a
result, programs built with libraries that contained unicode (eg
cl-unicode, uax-15, ...) would fail to build due to character encoding
issues when loading the FASLs.

Change-Id: I66149b585e85b213d0c026153140a1925536bd29
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1469
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2020-07-26 21:56:54 +00:00
Vincent Ambo
31516eb9cd fix(buildTypedGo): Use builtins.baseNameOf when renaming inputs
Without this, we would end up with "double" store paths like this:

  /nix/store/848js1fvbjniv5n00hifmhgzzszl97vv--nix-store-ad6piq18wdkxnfzsbyn88ixvv7gfb1dp-main.go.drv

We really only care about the `main.go` bit though.

Change-Id: Ib0644781a0e232a45e1cae3dd05b9b828c9087ee
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1321
Tested-by: BuildkiteCI
Reviewed-by: ericvolp12 <ericvolp12@gmail.com>
2020-07-20 11:33:13 +00:00
Kane York
4dd236be53 feat(ci): run buf check lint in CI
Breaking change detection will run but not enforce.

Emoji of water buffalo was chosen by @pedge fiat in the bufbuild slack.

Change-Id: Ie292f2bfddc0e3bc512e4a138c0b5d0fa2603bad
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1247
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: glittershark <grfn@gws.fyi>
2020-07-17 22:55:13 +00:00
Kane York
6c98d3d81f fix(nix/buildGo/example): use correct package name
Change-Id: Ie191e4154afe5aa47bb9ea5322010db014a1fa42
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1246
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2020-07-17 20:01:29 +00:00
eta
4dca44ac72 fix(buildLisp): use full store path in FASL output pathnames
If you compiled dirA/test.lisp and dirB/test.lisp in the same library,
they'd both get written to /test.fasl and the second would overwrite the
first. Instead, use the whole store path (with / swapped for -) as the fasl
filename.

Change-Id: I4eb88b5d33757751e1f67e72ed328bd58079b1b9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/944
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2020-07-06 22:01:09 +00:00
Profpatsch
2f21e0c8c0 refactor(nix): use our own writeScript(Bin)
We have this nice `runExecline` now, so we don’t need to use
`runCommand` (which spawns bash) just to write a simple script.

Change-Id: I2941ed8c1448fa1d7cc02dc18b24a8a945b2c38b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/704
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: BuildkiteCI
2020-06-30 00:19:27 +00:00
Profpatsch
4402c363b6 feat(nix/runExecline): add runExecline
runExecline is a primitive that just does not care.

It’s similar to `runCommand`, but instead of concatenating bash
scripts left and right, it actually *uses* the features of
`derivation`, passing things to `args` and making it possible to
overwrite the `builder` in a sensible manner.

Additionally, it provides a way to pass a nix string to `stdin` of the
build script.

Similar to `writeExecline`, the passed script is not a string, but a
nested list of nix lists representing execline blocks. Escaping is
done by the implementation, the user can just use normal nix strings.

Change-Id: I890d9e5d921207751cdc8cc4309381395d92742f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/701
Reviewed-by: BuildkiteCI
Reviewed-by: isomer <isomer@tvl.fyi>
Reviewed-by: tazjin <mail@tazj.in>
Tested-by: BuildkiteCI
2020-06-29 22:58:47 +00:00
Profpatsch
7a6a007302 refactor(nix/escapeExecline): move into own directory
The escaping functions are going to be used by both `writeExecline`
and `runExecline`, so let’s move them to their own namespace.

Change-Id: Iccf69eaeca3062573e0751a17c548b7def86196d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/706
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: Kane York <rikingcoding@gmail.com>
2020-06-29 22:58:32 +00:00
Profpatsch
cdfae7de48 feat(nix/emptyDerivation) add emptyDerivation
The most trivial of all derivations. It is more useful than it looks.

Can be used to bind nix expressions (e.g. test suites) to a
derivation, so that `nix-build` does not crap itself.

Change-Id: I61c24d8c129c9505733161207f3c30e820f5b15e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/665
Reviewed-by: tazjin <mail@tazj.in>
2020-06-28 17:59:46 +00:00
Profpatsch
98a990c6a6 feat(nix/getBins): add getBins
This is a simple-stupid “unix import system” for nix, for referencing
binaries in `/bin/` by their name and lifting them to a Nix attrset.

Allows for simple aliasing of executable names.

Change-Id: Ifa23cb377201c3b08050c5026e9751e736afaf56
Reviewed-on: https://cl.tvl.fyi/c/depot/+/664
Reviewed-by: tazjin <mail@tazj.in>
2020-06-28 17:58:15 +00:00
Profpatsch
c8e888c1d2 feat(nix/runTestsuite): add runTestsuite
This is a very simple test suite for nix expressions.
It should help us set up a good suite of unit tests for our nix-based
stuff.

Since we allow import from derivation, these tests can also depend on
derivations and e.g. use `builtins.readFile` to check outputs.

This is a first PoC to get us going, we can always replace it by
something different in the future if we don’t like it.

Change-Id: I206c7b624db2b1dabd9c73ffce4f87e658919958
Reviewed-on: https://cl.tvl.fyi/c/depot/+/662
Reviewed-by: tazjin <mail@tazj.in>
Tested-by: tazjin <mail@tazj.in>
2020-06-28 03:52:40 +00:00
Profpatsch
e2c7e38934 feat(nix/yants): add unit type
Has one less value than bool.

Change-Id: I2f2db07b1eb1f49172942bc1d8c6ceb30d987d84
Reviewed-on: https://cl.tvl.fyi/c/depot/+/661
Reviewed-by: tazjin <mail@tazj.in>
2020-06-27 20:48:15 +00:00
Profpatsch
3fd583d27c feat(nix/writeExecline): add writeExecline
This is a writer, similar to `pkgs.writeBashScript` or
`pkgs.writers.writePython3`.

The difference is that we can correctly write all execline scripts by
using nix lists of lists, so the user doesn’t have to care about
escaping arguments (like they have to in bash scripts with
`lib.escapeShellArg` for example).

Change-Id: I2f2874cf61170ddca07b89b692f762725f4a75dc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/625
Reviewed-by: Kane York <rikingcoding@gmail.com>
Reviewed-by: tazjin <mail@tazj.in>
2020-06-27 17:52:14 +00:00
Profpatsch
17eba437a7 feat(nix/binify): add binify
Create a store path where the executable `exe` is linked to
$out/bin/${name}. This is useful for e.g. including it as a “package”
in `buildInputs` of a shell.nix.

For example, if I have the exeutable /nix/store/…-hello, I can make it
into /nix/store/…-binify-hello/bin/hello with
`binify { exe = …; name = "hello" }`.

Change-Id: I600bdcd8f143bca2dd8dfbb165a9a5a8d6397622
Reviewed-on: https://cl.tvl.fyi/c/depot/+/624
Reviewed-by: tazjin <mail@tazj.in>
2020-06-27 03:42:05 +00:00
Vincent Ambo
b72d232075 fix(buildTypedGo): Pass correct arguments to .package
Change-Id: I535e084e7b3195628609bb43e382e450bc0003ba
Reviewed-on: https://cl.tvl.fyi/c/depot/+/497
Reviewed-by: eta <eta@theta.eu.org>
Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-06-19 15:25:11 +00:00
Vincent Ambo
e1e1764e32 feat(nix/buildTypedGo): Add a //nix/buildGo wrapper for typed Go
These functions work like buildGo.program & buildGo.package, but run
the .go2 sources through go2go first before passing them to the
//nix/buildGo equivalents.

Change-Id: Id1ebab6085c390d6986387370181377b9f5d39e8
2020-06-18 02:29:52 +01:00
Vincent Ambo
81b6bde074 fix(nix/buildGo): Fix import of buildGo in example file
Change-Id: I56b4d19f24c5d8ec45254b7b05c9e49002d2ef85
Reviewed-on: https://cl.tvl.fyi/c/depot/+/382
Reviewed-by: edef <edef@edef.eu>
2020-06-15 16:50:54 +00:00
edef
cec109807b fix(nix/readTree): Ignore hidden files and directories
This skips any directory entries starting with a dot.

Change-Id: I95767f3d35bcb2ed9b3d6e772f3924dd57612711
Reviewed-on: https://cl.tvl.fyi/c/depot/+/123
Reviewed-by: tazjin <mail@tazj.in>
2020-06-13 20:08:05 +00:00
edef
01ddbb4397 fix(nix/buildGo/external): Properly match import path prefixes
Prior to this patch, github.com/hashicorp/terraform-svchost is
erroneously considered a sub-package of github.com/hashicorp/terraform,
breaking dependency searching:

    error: missing local dependency 'github.com.hashicorp.terraform-svchost' in 'github.com/hashicorp/terraform'

Change-Id: Ibcf0f3a9b1742ce46f84cbbf84e90127b8c1df0d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/122
Reviewed-by: tazjin <mail@tazj.in>
2020-06-13 03:04:40 +00:00
edef
f0aca6bd8a fix(nix/buildGo): Pass sane arguments to log.Fatalf
Change-Id: Ie0f2c0a50d8c0618abf6dae2242155fdf1f98e2b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/121
Reviewed-by: tazjin <mail@tazj.in>
2020-06-13 01:32:18 +00:00
Vincent Ambo
980bf5365c fix(nix/buildGo): Do not silently ignore filepath.Walk() errors 2020-05-25 23:24:57 +01:00
Vincent Ambo
6e7006c061 docs(nix/yants): Mention Yants subtree split in README
This subtree split makes it possible for people to clone only
yants (similar to the kontemplate and journaldriver branches).

The subtree continues the history of the old git repository.
2020-05-16 12:37:12 +01:00
Vincent Ambo
db74ffb4b1 fix(nix/buildGo): Remove absolute references to GOROOT from binaries
Setting the GOROOT_FINAL environment variables replaces the absolute
location of the Go standard library sources in the final build
artefacts with a fake location (in this case starting with
go/src/...).

This is despite the documentation for 'trimpath' (in 'go tool
compile') stating that it would affect all source paths: That's only
true for user code!

I figured this out by reading through the implementation of the other
'trimpath' (in 'gob build'):

https://go-review.googlesource.com/c/go/+/173345
2020-03-31 01:29:35 +01:00
Vincent Ambo
4bbbb58cb5 chore: Rename pkgs->depot in all Nix file headers 2020-02-21 13:54:53 +00:00
Vincent Ambo
b4c0292753 fix(nix/tailscale): Fix incorrect Tailscale ACL config type 2020-02-11 21:00:50 +00:00
Vincent Ambo
df1a4fef2b feat(nix/tailscale): Add function for generating tailscale ACLs
... and use it on Camden!
2020-02-11 16:36:28 +00:00
Vincent Ambo
ddc9ee4dcd docs(nix/yants): Fix screenshot URLs
The URLs served on the /about page are relative to the path's dirname,
which is tree/nix not tree/nix/yants.
2020-02-07 19:11:55 +00:00
Vincent Ambo
8e8bbbca04 fix(nix/buildLisp): Ensure SBCL uses UTF-8 encoding 2020-01-29 10:12:07 +00:00
Vincent Ambo
0001dfd7f6 docs(nix/buildLisp): Update the README with actual docs 2020-01-29 00:03:24 +00:00
Vincent Ambo
ca60eafa80 feat(nix/buildLisp): Add 'bundled' function for built-in libraries
Makes it possible to add virtual dependencies on built-in libraries,
e.g. `buildLisp.bundled "sb-posix"`.
2020-01-26 23:58:31 +00:00
edef
30286d5454 chore(buildLisp): use lib.optionalString where applicable 2020-01-24 22:18:19 +00:00
edef
ccbac831d3 chore(buildGo): use lib.optionalString where applicable 2020-01-24 22:18:19 +00:00
Vincent Ambo
21e9a65a35 fix(nix/buildLisp): Don't load binaries in sbclWith
Adds an attribute on each Lisp derivation that specifies whether it is
a binary or not. This attribute is then filtered for in sbclWith.
2020-01-17 16:41:57 +00:00
Vincent Ambo
e1cc4966b7 feat(nix/buildLisp): Support passing programs to sbclWith
Adds the necessary attributes on derivations created by
buildLisp.program for them to be passed to buildLisp.sbclWith.

This makes it possible to easily spin up Lisp environments that
contain everything needed for a given program.
2020-01-17 12:44:24 +01:00
Vincent Ambo
09720e2da2 fix(buildLisp): Wrap executables to set load paths correctly
I can not currently find a way to set the CFFI variables correctly to
get it to load libraries from Nix.

In the absence of that feature, a wrapper also does the trick.
2020-01-09 03:32:29 +00:00
Vincent Ambo
44820827d1 feat(buildLisp): Initial implementation of foreign library loading
Adds a new 'native' parameter to the buildLisp functions in which
libraries can be passed in.

This does not yet work with CFFI packages.
2020-01-09 02:57:02 +00:00
Vincent Ambo
2a170f1ed7 fix(buildLisp): Perform a topological sort of dependencies
This ensures that dependencies are loaded in the correct order in
larger dependency graphs.
2020-01-09 00:46:20 +00:00
Vincent Ambo
e3a8dc9500 fix(buildLisp): Cursed code to fix load ordering
It's not enough to compile in the right order - turns out you also
have to load the compiled objects in the right order.

To achieve this some cursed code has been added that changes the Lisp
generated by Nix to compile the other Lisp so that it also generates
some bash, which Nix can then use to concatenate the FASLs in the
right order to feed them to Lisp again.

It works but I'll replace it with a more elegant solution once one is
needed.
2020-01-08 23:57:34 +00:00
Vincent Ambo
7bc10eb9b7 feat(buildLisp): Add initial, tiny example program 2020-01-08 21:41:43 +00:00
Vincent Ambo
bdad8f6642 feat(buildLisp): Implement buildLisp.program to dump executables
Dumps the executable image from SBCL to $out/bin/$name.

Image compression is disabled.
2020-01-08 21:39:26 +00:00
Vincent Ambo
2bfe073eb2 refactor(buildLisp): Inline dependency loading in genCompileLisp 2020-01-08 21:39:06 +00:00
Vincent Ambo
b5e1e81a3d feat(buildLisp): Add function to wrap SBCL with dependencies
Adds `buildLisp.sbclWith` which creates an SBCL wrapper the contains
all the requested dependencies.
2020-01-08 19:38:29 +00:00
Vincent Ambo
ca199a57d9 feat(buildLisp): Implement dependency loading & propagation
Similar to buildGo.nix, the library derivations carry information
about their dependencies which is merged when a load file is
instantiated.

The load files are created when compiling libraries, but will in the
future also be created when wrapping SBCL and dumping images.
2020-01-08 18:40:53 +00:00
Vincent Ambo
1297afec4b fix(buildLisp): Fail the build on compilation errors
This needs to be handled explicitly in the COMPILE-FILE form.
2020-01-08 17:53:06 +00:00
Vincent Ambo
a954bd8d5e feat(nix/buildLisp): Add initial sketch including buildLisp.library
Adds a Nix function to build a Lisp library out of a specified set of
Nix files. All files are combined into a single FASL.

This is by design only compatible with SBCL (for now).
2020-01-08 02:00:54 +00:00
Vincent Ambo
eb650eb8d8 docs(nix/readTree): Add a very descriptive README for readTree 2019-12-21 05:42:52 +00:00
Vincent Ambo
4a9c6ab6a2 refactor(nix/readTree): Move readTree to its own subfolder 2019-12-21 05:42:49 +00:00
Vincent Ambo
9885036eec chore(yants): Move tests into subfolder & add to CI builds 2019-12-20 21:53:19 +00:00
Vincent Ambo
902500c4ea fix(yants): Allow extra import arguments
Required for readTree compatibility.
2019-12-20 21:49:00 +00:00
Vincent Ambo
b98c60ecca merge(yants): Integrate yants into depot at //depot/nix/yants 2019-12-20 21:47:38 +00:00
Vincent Ambo
210893ce09 chore(yants): Prepare for depot-merge
Yants is being integrated at //depot/nix/yants
2019-12-20 21:46:59 +00:00
Vincent Ambo
03bfe08e1d chore: Significantly restructure folder layout
This moves the various projects from "type-based" folders (such as
"services" or "tools") into more appropriate semantic folders (such as
"nix", "ops" or "web").

Deprecated projects (nixcon-demo & gotest) which only existed for
testing/demonstration purposes have been removed.

(Note: *all* builds are broken with this commit)
2019-12-20 20:18:41 +00:00
Asad Saeeduddin
56a1f03a10 Fix typo in code snippet 2019-12-18 01:00:39 +00:00
Vincent Ambo
5f4664ad5a Fix rendering of pipes in code blocks across dialects
Pipes inside of code blocks inside of tables seem to pose a challenge
for some Markdown renderers. Not a problem for Github's but this page
is occasionally embedded in other documentation (such as nixery.dev)
2019-08-21 10:19:58 +01:00
Vincent Ambo
19d6d5a16c Add table with most important Nix operators 2019-08-07 16:36:28 +01:00
Louis Taylor
d113a91ae8 Fix link to nixdoc 2019-08-02 11:50:35 +01:00
Vincent Ambo
8b325cd791 Make Github's Nix highlighter happy
The highlighter expects that all code blocks are valid expressions (to
some degree) and highlights mismatches in dark red, which is not
particularly pleasant for the reader.

This introduces a "fake" attribute set in the `inherit` section to
please the highlighter.
2019-08-01 23:24:14 +01:00
Vincent Ambo
bb02b01dd0 Add Nix idioms (file lambdas, callPackage, overrides)
These are not strictly speaking language features, but my intention
for this document is that someone can - after reading it - navigate a
large chunk of existing Nix code.
2019-08-01 23:20:56 +01:00
Vincent Ambo
a2610a9b33 Add note about 'rec' in attribute sets 2019-08-01 23:20:38 +01:00
Vincent Ambo
fe3b0d3706 Point people to the issue tracker for ... issue tracking 2019-08-01 22:48:01 +01:00
Vincent Ambo
32e4cfdda4 Check in initial version of nix-1p
Adds the initial version originally published as a gist. This is
already slightly different from the gist in that it contains a section
on derivations.

This is not yet complete.
2019-08-01 22:48:01 +01:00