Commit graph

19 commits

Author SHA1 Message Date
sterni
9d8832474c chore(tvix/verify-lang-tests): update latest Nix version
Change-Id: I656c183d970806565d45ffd31f99f73c804dda1d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12203
Tested-by: BuildkiteCI
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
2024-08-14 15:01:03 +00:00
Ilan Joselevich
3511e328ec feat(tvix/eval): Implement builtins.readFileType
builtins.readFileType was added to Nix back in version 2.14.

The tests were also moved out of notyetpassing in addition to the
readDir fixtures they depend on.

I caught a bug where we previously used std::fs::metadata (via the
.metadata() method on File) which follows symlinks so it would always
return false for is_symlink(). Instead we now use
std::fs::symlink_metadata directly which does not follow symlinks, so
tests now pass. This wasn't an issue for builtins.readDir as it uses
walkdir and walkdir doesn't follow symlinks either.

Change-Id: I58eb97bdb5ec95df4f6882f495f8c572fe7c6793
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12130
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com>
Tested-by: BuildkiteCI
2024-08-09 14:35:57 +00:00
Ilan Joselevich
549e86d95b feat(tvix/verify-lang-tests): Test nixVersions.nix_{2_3,2_23}
With this change the latest Nix version being checked is 2.23
(previously was 2.18). A warning will now be printed when the
latest verified Nix version is out of date.

I added a test to the skip-list because the behavior of
builtins.dirOf has changed in Nix 2.22

Change-Id: Iaf7fa98dbfa599a2d2723df0e42bb459711e2413
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12105
Tested-by: BuildkiteCI
Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com>
Reviewed-by: flokli <flokli@flokli.de>
2024-08-09 14:19:25 +00:00
sterni
d1b7e08726 fix(tvix/eval): don't force lambda arg in map, mapAttrs & genList
It is pretty pointless to force the function argument if we are going to
use a suspended call later since forcing the function may fail in ways
that are not covered by Catchables (non-recoverable errors, infinite
recursions). From this, it kind of seems as if using #[catch] is never
correct and should be replaced by #[lazy]. Also we should probably try
to come up with more test cases for stuff where laziness gets us out of
the jam as an equivalent to the catchable tests for nonrecoverable
errors.

Fixes b/386.

Change-Id: Ia926df4ac1b440ec430403ab7b40924a0c97221b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11153
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: sterni <sternenseemann@systemli.org>
2024-03-19 06:20:17 +00:00
Ryan Lahfa
7b1632ec71 feat(tvix/eval): strengthen significantly catchable test suite
Adds a bunch (notably certain overlapping) tests for catchable situations.

This should cover many scenarios, argument is catchable, element in argument is catchable, function returns
catchable in the middle of the processing, etc.

Co-authored-by: Aspen Smith <root@gws.fyi>
Change-Id: Icd722cf8dbc91a24f45cd540a328711e5826f76c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10621
Reviewed-by: aspen <root@gws.fyi>
Tested-by: BuildkiteCI
2024-02-10 15:34:15 +00:00
Adam Joseph
19d13eb070 test(tvix/eval): add test case for b/339
Not yet passing.

Change-Id: I1de3f72d8b3f46567fdba010fc3ab4bace3f1699
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10219
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: Adam Joseph <adam@westernsemico.com>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2023-12-12 14:26:46 +00:00
Adam Joseph
9256621009 fix(tvix/eval): update identifier quoting to match cppnix 2.17
In cppnix 2.17, commit b72bc4a972fe568744d98b89d63adcd504cb586c, the
libexpr pretty-printing routine was fixed so that it would no longer
pretty-print attrsets with keywords in their attrnames incorrectly.

This commit implements the corresponding fix for tvix, fixes our
tests to work with cppnix>=2.17 oracles, and expands our test cases
to cover all the keywords.

Change-Id: I4b51389cd3a9c44babc8ab2a84b383b7b0b116ca
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9283
Autosubmit: Adam Joseph <adam@westernsemico.com>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2023-09-15 00:00:59 +00:00
Vincent Ambo
110328232d chore(3p/sources): bump channels & overlays (2023-08-21)
Included changes:

* tvix/eval: enable some lang tests on nix_latest

  Nix 2.16 contains some breaking language changes which Tvix does
  not yet implement, but the existing tests for them are now passed by
  Nix 2.16 (but not yet by Tvix).

* tvix/eval: disable a lang test on nix_latest

  In Nix 2.17, the identifier formatting test fails because some
  behaviour changed. We have not investigated further yet.

* 3p/overlays: use version of ihp-hsx that works with GHC 9.4

  Originally from the separate cl/9185.

* top-level: introduce a mechanism to exclude build targets from CI in
  the top level. This fixes b/296.

* users/grfn: disable builds of xanthous (and dependents) until the
  CLs fixing its build are submitted

* 3p/overlays: build nixos-option against Nix 2.15, the only version
  with which it builds

* 3p/overlays: bump tdlib to 1.8.16

Change-Id: Ia377f39dbdb08ac45ff830a615e64babc091e5ee
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9125
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
2023-09-06 13:38:15 +00:00
sterni
3b8c9ec9c8 test(tvix/eval): update nix_tests suite to C++ Nix master
Adds new tests for foldl', intersectAttrs as well as fills in missing
.exp files.

New test cases we don't pass:

- fromTOML timestamp capabilities
- path antiquotation
- replaceStrings is lazier on C++ Nix master

The C++ Nix revision used is 7066d21a0ddb421967980094222c4bc1f5a0f45a.

Change-Id: Ic619c96e2d41e6c5ea6fa93f9402b12e564af3c5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8778
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2023-06-15 19:28:16 +00:00
sterni
0f71d8f813 test(tvix/eval): genericClosure (pointer) comparison support
genericClosure has very limited support for pointer equality: It relies
on comparison (not equality!) in C++ Nix, so as soon as C++ Nix supports
comparing lists (langVersion >= 6) we can rely on pointer equality for
key.

Since Tvix uses equality, not comparison for the insert, our behavior is
currently different, as documented by the notyetpassing tests.

Change-Id: Ifcd741ed4fc3ccc3825f7038875d56a9918b786a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8720
Tested-by: BuildkiteCI
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
2023-06-15 11:01:06 +00:00
sterni
0005737f11 fix(tvix/eval): make tvix display values like nix-instantiate(1)
In order for the test suite we have currently to be comparable to C++
Nix, we need to display values in the same way. This was largely the
case except in some weird cases.

* <CODE> for thunks and <CYCLE> for repeated thunks (?) are already in
  use. <CODE> formatting is tested by the oracle test suite already.

* Instead of lambda, we need to use <LAMBDA>

* <<primop>> and <<primop-app>> (a formatting C++ Nix uses nowhere)
  now are <PRIMOP> and <PRIMOP-APP>.

We'll probably want to have a fancier display of values (in a separate
trait) down the line. This could be used for interactive usage, e.g. the
REPL or a potential debugger.

There is a peculiarity with C++ Nix 2.3 formatting primops: import is
considered a <<PRIMOP-APP>>, since it is internally implemented by means
of scopedImport. This implementation detail no longer leaks in C++ Nix
2.13 nor in Tvix.

<CYCLE> display is untested at the moment, since we exhibit a
discrepancy to C++ Nix 2.3. Our current detection is more similar to C++
Nix 2.13—luckily it is also the more consistent of the two. See also
b/245.

Change-Id: I1d534434b02e470bf5475b3758920ea81e3420dc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8760
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2023-06-15 11:01:06 +00:00
sterni
b7090ec874 test(tvix/eval): add test for builtins parity
This will eventually force us to have a base builtins set in common with
C++ Nix, i.e. all 2.3 builtins except the controversial
builtins.valueSize.

Change-Id: I2c767f07d6a14711911658e87da9f18ede57a143
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7747
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-01-06 12:00:38 +00:00
sterni
7bab0c26ee test(tvix/eval): verify pointer equality in list comparisons
Change-Id: I617d402c8ecc7aaf607c4bdcd58a06ebddb71fac
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7370
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: Adam Joseph <adam@westernsemico.com>
2022-12-02 14:06:22 +00:00
Adam Joseph
9c7d8e791a test(tvix/eval): add a test case for groupBy with thunks
We have to be careful implementing `builtins.groupBy`, since the
list may contain thunks, and tvix's to_xxx() functions do not work
on thunks.

Signed-off-by: Adam Joseph <adam@westernsemico.com>
Change-Id: I182b6fc2d4296f864ed16744ef70b153e8e6978a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7039
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-11-04 01:46:27 +00:00
Griffin Smith
d0a836b0e1 feat(tvix/eval): Implement comparison for lists
Lists are compared lexicographically in C++ nix as of [0], and our
updated nix test suites depend on this. This implements comparison of
list values in `Value::nix_cmp` using a very similar algorithm to what
C++ does - similarly to there, this requires passing in the VM so we can
force thunks in the list elements as we go.

[0]: 09471d2680#

Change-Id: I5d8bb07f90647a1fec83f775243e21af856afbb1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7070
Autosubmit: grfn <grfn@gws.fyi>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-10-29 10:45:51 +00:00
James Landrein
8425c2016c feat(tvix/eval): add builtins.{floor,ceil}
Change-Id: I4e6c4f96f6f5097a5c637eb3dbbd7bb8b34b7d52
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7032
Autosubmit: j4m3s <james.landrein@gmail.com>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: grfn <grfn@gws.fyi>
2022-10-25 08:46:37 +00:00
sterni
b410655a5e test(tvix/verify-lang-tests): also check notyetpassing tests
Adding these to the C++ Nix CI ensures that it is possible for the
tests in that directory to pass at all. Mainly this would catch
situations like fixed in a previous CL when moving the tests around
would break them so that they wouldn't even pass in C++ Nix.

For this to work, we need to track skips by basename to be
directory-independent (assuming that every skipped test name is unique
is hopefully okay).

Change-Id: If6cb63ebdef0fc19b082b6a04e79ada2e47c658e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7048
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-10-20 18:21:25 +00:00
sterni
8e188d6235 test(tvix/eval): update C++ Nix test suite to current master
The language test suite actually doesn't require flakes and the
new features are mostly sensible (added builtins) as well as some
tests for regressions the C++ implementation experienced.

The path interpolation test is not included in this update because there
is no way to construct an location-independent .exp file for it (the C++
repo also doesn't have one). We may still want to implement that feature
eventually (in case rnix adds support for it).

The C++ Nix revision used is ac0fb38e8a5a25a84fa17704bd31b453211263eb.

Change-Id: I75f1e780ddeeee6f6b1f28cf3c66c288dca2c20c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7043
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-10-20 02:33:40 +00:00
sterni
292a751388 test(tvix): nix-planned test verification using C++ Nix 2.3 and 2.11
Reimplement the test discovery of the lang tests script in Nix which
allows for a more flexible skipping logic that can e.g. react to the C++
Nix version used. This allows us to run the test suite against both
C++ Nix 2.3 and the latest C++ Nix version 2.11. The latter is mainly
useful, so we can implement newer Nix features and still verify them
against the C++ implementation.

Change-Id: I30c802844133b86b5e49f5e4f4fefacdb6215e0e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7042
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-10-20 02:28:38 +00:00