Commit graph

40 commits

Author SHA1 Message Date
Ilan Joselevich
cd0b92419a docs(tvix/TODO): dibs on crate2nix WASM
Change-Id: Idde9aaa31ecaa9570559ffe3271bc23b7a71133c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11820
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
2024-06-15 04:02:33 +00:00
Florian Klink
86928b5e7f docs(tvix/TODO): add builtins.path roundtrip for flat
Change-Id: I8291d2fdfa2ea31ec24c82dcfb447fa9eaa3a961
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11819
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
2024-06-15 04:02:33 +00:00
Florian Klink
6e321c615c docs(tvix/*store): add some internal links
We can now cross-link between pages, and it actually works.

Change-Id: Id8bd85111672c687118db9adae7f5066f22441f5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11817
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
2024-06-15 03:56:11 +00:00
Florian Klink
46585bee15 docs(tvix/TODO): drop builtins:fetchurl item
This has been implemented in cl/11786, cl/11790 and cl/10975.

This includes support for the `executable` and `unpack` variants of it.

Change-Id: I5bcad5b8e86b9fd341219a59ba86660b692e66e8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11818
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-06-15 01:01:04 +00:00
Simon Hauser
118e69d81d fix(tvix/tracing): reduce the error logs of otlp if collector is offline
The problem is that opentelemetry_otlp tonic batch exporter tries to
exports if either the `scheduled_delay` or if the
`max_export_batch_size` is reached. Per default the
`max_export_batch_size` is set to 512 spans, which means that we try to
export these spans once that counter is reached. Each export will then
try to connect to the exporter (if that not already happening) and will
result in a `tcp connect error`.
Increasing the max_export_batch_size to 4096 will then ensure that the
export only happens if the `scheduled_delay` is met after the 10
seconds.
`max_queue_size` is also increased, because `max_export_batch_size`
should not be greater than `max_queue_size`, so similar to the default
config its set to `max_export_batch_size * 4`.
This will reduce the amount of tries to otlp if the collector is not
available and otlp enabled.

Change-Id: Ic3430006e8a104fa3b34d274678cae55b3620ce9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11791
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: Simon Hauser <simon.hauser@helsinki-systems.de>
2024-06-14 10:20:31 +00:00
Florian Klink
5077ca70de chore(tvix/eval): move eval docs to tvix/docs
Change-Id: I75b33c43456389de6e521b4f0ad46d68bc9e98f6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11809
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2024-06-14 08:00:34 +00:00
Florian Klink
6947dc4349 chore(tvix/docs): move [ca]store docs to tvix/docs
Change-Id: Idd78ffae34b6ea7b93d13de73b98c61a348869fb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11808
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
2024-06-14 08:00:34 +00:00
Florian Klink
adc7353bd1 feat(tvix/docs): add d2 support
This provides a nicer syntax to draw some diagrams.

Change-Id: I2ab580e61399a45f01ed422ee29382e860753ffa
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11807
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: yuka <yuka@yuka.dev>
2024-06-14 07:04:23 +00:00
Florian Klink
4df3284dd0 feat(tvix/docs): add mdbook-admonish
This provides a plugin for callouts.

It needs to have additional CSS file added to `book.toml`,
which can be (re-)generated by `mdbook-admonish install`.

See https://github.com/tommilligan/mdbook-admonish/issues/171#issuecomment-2009166079
for more context.

Use it by adding a warning one to the architecture document.

Change-Id: I75c9a33d00acb603c6da10d3f9ce3485731c1672
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11805
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: yuka <yuka@yuka.dev>
2024-06-14 07:04:23 +00:00
Florian Klink
c30fded880 chore(tvix/docs): move daemon protocol to its own chapter
… and add an introductionary page at the root.

Change-Id: I89ce69a4221c332b7c0d99ac26b4e26bfd248341
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11806
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: Brian Olsen <me@griff.name>
2024-06-13 19:23:20 +00:00
Florian Klink
154e0d71e0 docs(tvix/docs/TODO): document ChunkService split idea
Change-Id: Ie9c88b0d14902c642e2d3d6603265688eef0e10d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11755
Reviewed-by: yuka <yuka@yuka.dev>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
2024-06-11 11:59:29 +00:00
Simon Hauser
825d498908 feat(tvix/tracing): introduce common tvix-tracing crate
Introduce a new common crate that contains tracing boilerplate which then
can be used in the cli, tvix-store and tvix-build crates.
It has otlp as an optional feature, which is currently only used by
tvix-store.

Change-Id: I41468ac4d9c65174515d721513b96fea463d6ed2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11758
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: Simon Hauser <simon.hauser@helsinki-systems.de>
2024-06-10 16:35:08 +00:00
Florian Klink
11a6ff7706 docs(tvix/docs/TODO): document attempt to add more granular errors
Change-Id: I46f99d15a4ce9c98dcdad822d3032bda3b09c12f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11754
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Simon Hauser <simon.hauser@helsinki-systems.de>
Tested-by: BuildkiteCI
2024-06-10 16:31:34 +00:00
Florian Klink
d3bc358bbc docs(tvix/docs/TODO): document crate2nix for WASM attempts
I tried moving web/tvixbolt to tvix/tvixbolt, and adding it to the cargo
workspace.

I then made `crates =` in `default.nix` a function accepting `pkgs`
(so we can pass in another nixpkgs for some invocations), and then
constructed another nixpkgs and crates instance like this:

```
  pkgs-wasm = (import pkgs.path {
    localSystem = localSystem;
    crossSystem = {
      system = localSystem;
      rustc.config = "wasm32-unknown-unknown";
    };
  }
  crates-wasm = (crates pkgs-wasm);
  tvixbolt-test = crates-wasm.workspaceMembers.tvixbolt.build;
```

… leading to the architecture build failures described in the TODO.

Change-Id: I32112d75f8c098d9810ca52b2d07cd76fae8d8d7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11777
Reviewed-by: flokli <flokli@flokli.de>
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2024-06-10 12:03:26 +00:00
Florian Klink
c3572048d5 docs(tvix/docs/TODO): expand Store Composition config section
Write down some of the thoughts after brainstorming with yuka.

Change-Id: I01c94474dc643b8c4993db80e50d3ec65f5c17f4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11749
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: yuka <yuka@yuka.dev>
2024-06-05 17:44:12 +00:00
Florian Klink
0ea55c767a docs(tvix/docs/TODO): extend O11Y section
Expand on tvix-tracing crate strategy, add some more context regarding
OTLP and span propagation.

Change-Id: Ice55c116c20aaf60531100465192ce11969551ac
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11750
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: Simon Hauser <simon.hauser@helsinki-systems.de>
Reviewed-by: flokli <flokli@flokli.de>
2024-06-05 08:51:38 +00:00
Florian Klink
c4c42c8b6c docs(tvix/docs/TODO): add item for string context rework
Change-Id: I4592490a75fb05093b3fd8649db3b57bff748c0d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11707
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: edef <edef@edef.eu>
Tested-by: BuildkiteCI
2024-05-23 15:58:22 +00:00
Florian Klink
6ad3780734 docs(tvix/TODO): toXML string context and self-closing tags is done
Change-Id: I83f6b1863a023ea0bf8518c67eb51f740c3ea89a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11706
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: Alyssa Ross <hi@alyssa.is>
2024-05-23 14:58:41 +00:00
Brian Olsen
7d7d0d150e docs(tvix): Begin documenting the Nix daemon protocol
This adds rough notes documenting the history of the Nix
daemon protocol, how logging works as well as begins
documenting inputs and outputs for all operations.

Change-Id: Id24a9a658c3e4e7c350ca1e4622f63ed96ccef5a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11594
Reviewed-by: Adam Joseph <adam@westernsemico.com>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2024-05-16 22:44:19 +00:00
Florian Klink
11850249c8 docs(tvix/docs): async nar reader is done
Change-Id: I4e8fd03d521b7d30904f84a6afcf842f1505ec7b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11628
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Autosubmit: flokli <flokli@flokli.de>
2024-05-11 15:07:30 +00:00
Florian Klink
728ceda8a6 docs(tvix/todo): add section on [ca]store errors
Change-Id: I832eed8abb62b7e13972ed58649a96e31b17f033
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11610
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-05-11 14:59:57 +00:00
Florian Klink
fd49dfa6ca docs(tvix/todo): update test suite section
nix_oracle is quite misleading, and fooled me into thinking this
validates all .exp files in the repo to match Nix output (which it
doesn't, that's done by verify-lang-tests).

Also, the whole test suite structure thing can use a bit more
explanation.

Change-Id: I2fadcc871843143270ad2ed9ac98de8287280e6f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11609
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-05-11 14:59:57 +00:00
Florian Klink
f5bb9b8467 docs(tvix): add a list of TODOs/ideas
This contains a rough collection of ideas on the TODO list, trying to
keep track of it somewhere.

Change-Id: Ifc5b0cf9f7ac38f7a8e56515882bdf70e349544b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11512
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: picnoir picnoir <picnoir@alternativebit.fr>
2024-04-25 15:07:45 +00:00
Florian Klink
b0e5a932ec docs(tvix): remove differences-drv-paths.md
This was true while we didn't do string contexts, but drvPath should not
differ anymore.

Change-Id: I58e6ed439519bd4d68259d1577946f3c65110dc4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11074
Tested-by: BuildkiteCI
Reviewed-by: edef <edef@edef.eu>
Autosubmit: flokli <flokli@flokli.de>
2024-03-03 16:53:41 +00:00
Florian Klink
c9514b7c0c feat(tvix/docs): ship custom highlight.js with Nix support
Selected languages:
 - bash
 - c
 - cpp
 - diff
 - go
 - ini
 - json
 - markdown
 - nix
 - protobuf
 - rust
 - shell
 - xml
 - yaml

Created with https://highlightjs.org/download

Change-Id: I76a87d2c1e9fa582d1474f8ffb9f3129e9a858bb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11073
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: edef <edef@edef.eu>
Tested-by: BuildkiteCI
2024-03-03 16:53:16 +00:00
Florian Klink
65a810fc0b feat(tvix/docs): switch to mdbook
Some of the docs are still outdated (like architecture and drv path
inconsistencies).

Change-Id: I7a6afceb008ef4cd19a764dd6c637b39fa842a2e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11072
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: edef <edef@edef.eu>
2024-03-03 16:52:38 +00:00
Florian Klink
d991548ea2 fix(tvix/docs): update builder magic string
It's `builtin:fetchurl`, not `builtins:fetchurl`.

Change-Id: I2e90fce5feecc75257a5156aee12e0737a525175
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10961
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: aspen <root@gws.fyi>
2024-02-19 15:52:14 +00:00
Florian Klink
9bb3f74062 docs(tvix/docs): add document about drvPath differences
This question popped up every once in a while. While already explained
quite well at
https://inbox.tvl.su/depot/20230316120039.j4fkp3puzrtbjcpi@tp/T/#t,
it's not easily accessible.

Lift it from there into tvix/docs for better visibility.

Change-Id: I5f2d4aff31ab4adc421e06a7d36c871f45e09100
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9080
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
2023-08-20 22:08:02 +00:00
sterni
8adc9c56f2 docs(tvix): document when pointer equality is preserved in C++ Nix
This explicitly documents behavior of C++ Nix that goes against the
intuition you'd gather from this document: that e.g. a simple select
from an attribute set causes a value to no longer be pointer equal to
its former self.

The point of documenting this is that we can show in a to be written
section on the use of pointer equality in nixpkgs that pointer equality
is only needed in a limited sense for evaluating it (C++ Nix's exterior
pointer equality). Tvix's pointer equality is far more powerful since
value identity preserving operations also preserve pointer equality,
generally speaking (this is because we implement interior pointer
equality in my made up terminology). This should eventually also be
documented.

Change-Id: I6ce7ef2d67b012f5ebc92f9e81bba33fb9dce7d0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8856
Tested-by: BuildkiteCI
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
2023-07-11 16:17:42 +00:00
sterni
2aaabb709f docs(tvix): update C++ Nix code links for pointer equality
Change-Id: Icfd79b36c09607b4183e7378cd3c17f6238297b2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8853
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2023-07-01 21:45:53 +00:00
sterni
864931b297 docs(tvix): start restructuring pointer equality document
I want to expand on the C++ Nix behavior, since it seems relevant to
note that a lot of operations in C++ Nix (like select) don't preserve
pointer equality (see
<https://github.com/NixOS/nix/issues/3371#issuecomment-1596167957>).
It is especially so, as Tvix establishes pointer equality in a different
way and thus shows differing behavior. Therefore I want to additionally
document Tvix's current behavior and make it more explicit to what
extent nixpkgs needs pointer equality.

Change-Id: I9b4ba75dacb749c9fcbba4b9646c6b48bb57bbad
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8852
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2023-07-01 21:45:53 +00:00
sterni
0128323c55 docs(tvix): mention langVersion >= 5 check in nixpkgs
Change-Id: I7aae900012e901fa4d886b1a4bf0657d593e4983
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8771
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-06-14 10:55:55 +00:00
sterni
daa3721f73 docs(tvix): fix minor spelling problems in pointer equality document
Main one is the its-it's mistake in the last paragraph, the rest was
suggested by LanguageTool.

Change-Id: If1b87a11f480452f312fc2759be7ded782d0a522
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7930
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2023-01-25 14:30:50 +00:00
sterni
ca0bd84589 docs(tvix): add reconstructed changelog for the Nix language
This was put together by digging through the C++ Nix git history as
described in the second paragraph. Note that this is only based on
changes given when the language version was actually increased.

There is no guarantee that there have been no other changes inbetween
that would have warranted an increase as well.

Change-Id: I4ddee0d4ecafa1b3e5e1a867e9700d6c32e936ad
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7836
Tested-by: BuildkiteCI
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
2023-01-16 09:32:49 +00:00
sterni
094f07ad9a docs(tvix): document pointer equality in (C++) Nix
semi-obscure = #nix-lang on freenode

Change-Id: Id235f7019bbee137bbde57a552c755261e23b58f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7368
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: Adam Joseph <adam@westernsemico.com>
Tested-by: BuildkiteCI
2022-11-25 10:27:08 +00:00
Florian Klink
d8f667e01e docs(tvix): Update components.md
This extends various ideas around the store and its slightly different
internal model.

Thanks to store composition, lazy substitution and more efficient
transfer, we can also simplify some complicated logic while building.

Change-Id: Ib3380af650fe06e114f54e8dc2df231f18af876b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6585
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-16 11:55:57 +00:00
Vincent Ambo
aa122cbae7 style: format entire depot with nixpkgs-fmt
This CL can be used to compare the style of nixpkgs-fmt against other
formatters (nixpkgs, alejandra).

Change-Id: I87c6abff6bcb546b02ead15ad0405f81e01b6d9e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4397
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: lukegb <lukegb@tvl.fyi>
Reviewed-by: wpcarro <wpcarro@gmail.com>
Reviewed-by: Profpatsch <mail@profpatsch.de>
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: cynthia <cynthia@tvl.fyi>
Reviewed-by: edef <edef@edef.eu>
Reviewed-by: eta <tvl@eta.st>
Reviewed-by: grfn <grfn@gws.fyi>
2022-01-31 16:11:53 +00:00
Vincent Ambo
9c8b904b56 docs(tvix): Begin specification of the Nix language
This is only the introductory text, without any of the components
specified so far.

Change-Id: I6cd8eff6ec2daf14cc2420ee0a8e8983bfaa57d9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3052
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: flokli <flokli@flokli.de>
Reviewed-by: sterni <sternenseemann@systemli.org>
2021-04-24 20:07:18 +00:00
adisbladis
c693e8bf51 chore(tvix): Make docs multi output
We also want to access the produced SVGs separately.

Change-Id: I5bdfd95b8a56323ed8bf5f4585b703f6e728c677
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2742
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2021-03-31 22:41:07 +00:00
Florian Klink
3c4e401c9e chore(tvix/docs): move from doc
Most other docs folders in the repo are called `doc` too, let's make
this consistent.

Change-Id: Icd712429b51763076548c977321e370f2a77877e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2741
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2021-03-31 22:34:59 +00:00