feat(testing): built-in test blocks + expect assertions, auto-run with pass/fail (#12)
All checks were successful
bootstrap / cfree-fixpoint (push) Successful in 17s
ci / build-and-test (push) Successful in 1m13s
commit-lint / conventional-commits (push) Successful in 4s
docs / build-and-deploy (push) Successful in 19s

A `test "name" { ... }` top-level block is discovered automatically and run by a
synthetic runner @main — no `entry` to write, nothing to register. Inside a test,
expect(cond) / expect_eq(a, b) / expect_near(a, b, tol) assert; on failure they
print `file:line: <what> failed (got G, want W)` and set a per-test fail flag but
keep going, so one run reports every failure. The runner prints `ok   - name` /
`FAIL - name` per test, a `== N passed, M failed ==` summary, and exits non-zero
if any test failed — so `ludic spec_test.ludic` drops straight into bin/x and CI.
expect_near carries the tolerance fixed-point / accumulated-integer game math need.

Frontend: new `test` keyword (parse_test -> N_TEST, collected in g_tests) and the
call node now carries its source line for the file:line messages. Backend:
emit_test_runner synthesises @fn__test_i bodies + the runner @main; the expect*
builtins lower to a branch-print-flag tail (emit_expect_fail). g_src_name (set in
main from the input path) supplies the filename. The compiler's own source has no
`test` blocks, so its self-compiled IR is unchanged and the C-free fixpoint holds.

- `test` wired into the vocabulary (ludic_syntax.h, JetBrains lexer, TextMate
  grammar) and documented (docs/language/testing/)
- examples/library/testing.ludic: a passing spec, guarded by a new spec_case in
  the regression suite (build, run, require exit 0 + the expected summary)

Coverage instrumentation (the biggest lift in #12) is left as a follow-up.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Orkun ÇAKILKAYA 2026-08-31 14:27:45 +03:00
parent a71279a7b9
commit ea2c6ab246
16 changed files with 15911 additions and 14867 deletions

View file

@ -0,0 +1,7 @@
---
id: testing
title: Testing
order: 34
---
A built-in testing framework in the spirit of Go's <code>go test</code>: tests live next to the code, run with one command, and report pass/fail — no harness to wire up. A <a href="kw-test"><code>test "name" { … }</code></a> block is discovered automatically and run by a synthetic entry point that prints <code>ok - name</code> or <code>FAIL - name</code> for each, a <code>== N passed, M failed ==</code> summary, and exits non-zero if anything failed (so CI and the <code>bin/x</code> runner catch it). Inside a test, the <code>expect</code>, <code>expect_eq</code> and <code>expect_near</code> assertions check a condition and, on failure, print <code>file:line: … failed (got …, want …)</code> and mark the test failed — without aborting, so one run reports every failure. <code>expect_near</code> takes a tolerance, which is what fixed-point and accumulated-integer game math need. Everything is deterministic and compiles to a native binary, so a suite runs in the same C-free toolchain as the rest of Ludic. Coverage instrumentation is a planned follow-up. Related: <a href="kw-function"><code>function</code></a>, <a href="fn-print"><code>print</code></a>.

View file

@ -0,0 +1,36 @@
---
id: kw-test
name: test
category: testing
kind: keyword
tokens: test
sig: test "name" { … }
tip: A named test block, run automatically with pass/fail reporting.
order: 1
---
A <code>test "name" { … }</code> block declares a named test. Every test in a file is discovered automatically and run by a generated entry point — there is no <code>entry</code> to write and nothing to register. The runner prints <code>ok - name</code> for a test whose assertions all held and <code>FAIL - name</code> for one that had a failure, then a <code>== N passed, M failed ==</code> summary, and the program exits non-zero if any test failed. Writing a test should feel like writing a function: put a few assertions in a block and run it.
Assertions (each records a failure and prints <code>file:line: … failed</code> but keeps going, so one run reports every failure):
- `expect(cond)` — the boolean `cond` must be true.
- `expect_eq(a, b)` — `a` must equal `b`; on failure prints `(got a, want b)`.
- `expect_near(a, b, tol)` — `a` must be within `tol` of `b` (absolute). Use it for `fixed`-point results and accumulated integer math, where an exact match is too brittle.
Run a spec file directly with the compiler-runner — `ludic mymath_test.ludic` compiles it to a native binary, runs it, and forwards the pass/fail exit code — so it drops straight into `bin/x` and CI.
```ludic
program MathSpec {
function add(a: int, b: int) -> int { return a + b }
test "addition adds" {
expect_eq(add(2, 3), 5)
expect(add(1, 1) == 2)
}
test "fixed math is close enough" {
let half = fixed(1) / 2 # 0.5 in Q16.16
expect_near(half, 32768, 2) # within 2 raw units of 0.5
}
}
```