Follow-up to #12: line coverage for the testing framework (bin/x test --coverage) #45

New issue

Closed

opened 2026-08-31 13:34:32 +02:00 by orkun · 1 comment

orkun commented

2026-08-31 13:34:32 +02:00

Owner

Follow-up to #12, which shipped the testing framework in ea2c6ab: test "name" { … } blocks, expect/expect_eq/expect_near assertions, a synthetic runner that prints per-test pass/fail + a == N passed, M failed == summary and exits non-zero on failure.

This tracks the one remaining acceptance item deliberately left out — the issue's own biggest lift: per-file line coverage.

Scope:

Compiler instrumentation: at each emitted statement, bump a per-source-line hit counter. Gate it behind a --coverage flag (default off) so ordinary builds — and the compiler's own self-compile — stay byte-identical and the C-free bootstrap fixpoint is untouched.
A runtime counter table dumped at exit (line -> hits), keyed by source file.
bin/x test --coverage: aggregate the dumps into a clean per-file line-coverage report.
Docs + a coverage example.

Design note: keep the instrumentation flag-gated and additive so the fixpoint stays trivial to reason about (the same discipline used for panic/expect). Related: #12 (the shipped framework).

Follow-up to #12, which shipped the testing framework in ea2c6ab: `test "name" { … }` blocks, `expect`/`expect_eq`/`expect_near` assertions, a synthetic runner that prints per-test pass/fail + a `== N passed, M failed ==` summary and exits non-zero on failure. This tracks the one remaining acceptance item deliberately left out — the issue's own **biggest lift**: per-file **line coverage**. Scope: - [ ] Compiler instrumentation: at each emitted statement, bump a per-source-line hit counter. Gate it behind a `--coverage` flag (default off) so ordinary builds — and the compiler's own self-compile — stay byte-identical and the C-free bootstrap fixpoint is untouched. - [ ] A runtime counter table dumped at exit (line -> hits), keyed by source file. - [ ] `bin/x test --coverage`: aggregate the dumps into a clean per-file line-coverage report. - [ ] Docs + a coverage example. Design note: keep the instrumentation flag-gated and additive so the fixpoint stays trivial to reason about (the same discipline used for panic/expect). Related: #12 (the shipped framework).

orkun referenced this issue from a commit

2026-08-31 13:51:23 +02:00

feat(testing): line coverage via --coverage + bin/x test --coverage (#45)

orkun commented

2026-08-31 13:51:50 +02:00

Author

Owner

Shipped in 4985933.

--coverage instrumentation — the compiler now bumps a per-source-line hit counter at each emitted statement, gated behind a new --coverage flag (default off). Ordinary builds and the compiler's own self-compile stay byte-identical (verified: snake/testing/reflect IR unchanged without the flag), and the C-free bootstrap fixpoint is untouched (x reseed + x bootstrap-cfree reproduce the seed byte-for-byte).

Runtime dump — a static line table plus a parallel hit-counter array are dumped at exit via an atexit hook (registered through an LLVM global constructor, so it fires regardless of program shape) to the file named by $LUDIC_COVERAGE (default ludic.cov): a FILE <name> header then one <line> <hits> row per instrumented line.

bin/x test --coverage — compiles the test specs with instrumentation, runs each into its own dump, and aggregates a per-file report naming the unreached lines:

== line coverage (bin/x test --coverage) ==
  examples/library/coverage.ludic   16/17 lines  94%   uncovered: 33
  examples/library/testing.ludic    17/17 lines  100%
  ----
  total: 33/34 lines  97%

Docs + example — examples/library/coverage.ludic (a spec whose tests deliberately miss one branch, which the report flags at line 33), the testing docs page, and a changeset. Full suite: 70/70 pass.

This closes the last open acceptance item deliberately left out of #12.

Shipped in 4985933. **`--coverage` instrumentation** — the compiler now bumps a per-source-line hit counter at each emitted statement, gated behind a new `--coverage` flag (default off). Ordinary builds and the compiler's own self-compile stay **byte-identical** (verified: snake/testing/reflect IR unchanged without the flag), and the C-free bootstrap fixpoint is untouched (`x reseed` + `x bootstrap-cfree` reproduce the seed byte-for-byte). **Runtime dump** — a static line table plus a parallel hit-counter array are dumped at exit via an `atexit` hook (registered through an LLVM global constructor, so it fires regardless of program shape) to the file named by `$LUDIC_COVERAGE` (default `ludic.cov`): a `FILE <name>` header then one `<line> <hits>` row per instrumented line. **`bin/x test --coverage`** — compiles the test specs with instrumentation, runs each into its own dump, and aggregates a per-file report naming the unreached lines: ``` == line coverage (bin/x test --coverage) == examples/library/coverage.ludic 16/17 lines 94% uncovered: 33 examples/library/testing.ludic 17/17 lines 100% ---- total: 33/34 lines 97% ``` **Docs + example** — `examples/library/coverage.ludic` (a spec whose tests deliberately miss one branch, which the report flags at line 33), the testing docs page, and a changeset. Full suite: 70/70 pass. This closes the last open acceptance item deliberately left out of #12.

orkun closed this issue

2026-08-31 13:51:56 +02:00