diff --git a/BOOTSTRAP.md b/BOOTSTRAP.md deleted file mode 100644 index b2cfbbde..00000000 --- a/BOOTSTRAP.md +++ /dev/null @@ -1,984 +0,0 @@ -# Bootstrapping Ludic in Ludic - -**What it would take for Ludic to compile itself.** - -Today `ludicc` is a C program: 2,508 lines across `compiler/ludicc.c`, -`compiler/native.c` and `compiler/driver.c`. Everything it produces is -Ludic-or-IR — the runtime a game calls is 2,501 lines of `.ludic`, and no C is -generated, compiled or linked in a build. The compiler is the last C in the -pipeline, and this document is about removing it. - -Every claim about what the language can and cannot do below was **verified -against the built compiler**, not read off the docs. The probe programs are in -the appendix; each `✅`/`❌` is a real compile-and-run. - ---- - -## 1. What "completely bootstrapped" means - -Self-hosting is not one property. It is three independent axes, and they cost -wildly different amounts: - -| Axis | Today | Target | -|---|---|---| -| **Compiler independence** — is the compiler written in the language? | ❌ 2,508 lines of C | `ludicc` written in Ludic, compiling itself to a fixpoint | -| **Runtime independence** — is the library the language ships written in the language? | ✅ **already done** — 2,501 lines of `.ludic` (gfx, PNG/DEFLATE, TrueType, UI) | keep | -| **Toolchain independence** — does a build need a foreign compiler? | ❌ `clang` assembles the IR and links | see §7 — three levels, only one is worth reaching | - -The runtime axis is already won, and that is the unusual part. Most languages -self-host the compiler long before they stop leaning on a C standard library; -Ludic did it backwards. **The remaining work is concentrated in one axis.** - -There is also a fourth, smaller thing: `runtime/native/cocoa.ll` (327 lines) and -`runtime/web/wasm.ll` are hand-written LLVM IR, not Ludic. §7.4 covers whether -that matters. - -Running alongside all of this is a question the bootstrap forces rather than -raises: **what the syntax should finally be.** A self-hosted compiler is written -in the language it compiles, so the grammar wants to be settled *before* the -port, not after. §5 audits what is irregular today and proposes the freeze; it -is scheduled as Stage 0.5, between the language features and the libraries. - -### The honest bar - -"Bootstrapped by itself completely" should mean: - -1. `ludicc` is written in Ludic. -2. A `ludicc` binary compiles the Ludic source of `ludicc` and produces a - **byte-identical** binary to itself (the fixpoint test, §6). -3. The C compiler is needed **only** to build the very first seed, and that seed - is a checked-in artifact rather than a live dependency. -4. No C source remains in the repo outside that seed. - -It should *not* mean writing an object-file writer and a linker. Rust and Swift -are self-hosted and both stand on LLVM; standing on `clang` as an IR assembler -is the same posture. §7 argues this explicitly so the goal does not quietly -inflate. - ---- - -## 2. Where the tree stands - -``` -compiler/ C split by concern; every file under 500 lines - ludicc.c 435 pipeline + codegen glue + main - util/ sb, diag 103 string builder; source registry + diagnostics - front/ lex, ast, parse 469 tokens; Node; recursive descent + imports - sem/ tables, uitree, validate 208 decl tables; widget flattening; static checks - back/ ir_* x10 953 the LLVM IR backend, one file per concern - driver/ toolchain, webbundle 382 IR -> object -> exe/dylib; the wasm bundle - fmt/ fmt 162 canonical AST printer (--fmt) - ------ - 2,712 C <- all of it, and all that must go -tools/ludic-tools/* 3,260 C ludic-fmt + ludic-lsp (not yet split) - -runtime/native/core.ludic 394 Ludic framebuffer, text, registers, RNG, input -runtime/native/image.ludic 436 Ludic PNG, sprites, alpha blend, 9-slice -runtime/native/inflate.ludic 276 Ludic DEFLATE (RFC 1951) -runtime/native/truetype.ludic 804 Ludic sfnt loader + AA rasterizer, Q16.16 -runtime/native/ui.ludic 591 Ludic retained widget tree, layout, focus - ---- - 2,501 Ludic <- proof the language is already load-bearing - -runtime/native/cocoa.ll 327 LLVM IR macOS window (objc_msgSend + CoreGraphics) -runtime/web/wasm.ll 369 LLVM IR browser shims -``` - -`util/`, `front/`, `sem/` and `fmt/` are separately compiled translation units; -`back/` and `driver/` are still one unit assembled by `back/native.c`, so their -include order is their definition order. The build list lives in -`compiler/sources.sh`, sourced by both `build.sh` and `test.sh`. - -`truetype.ludic` matters more than its line count. A from-scratch sfnt parser -with cmap format dispatch, composite glyph recursion and a Bézier rasterizer is -*structurally the same kind of program as a compiler*: binary input, recursive -descent, table lookups, a growing output buffer. It already works. That is the -strongest single piece of evidence that this port is feasible rather than -aspirational. - ---- - -## 3. What the language can already do - -All verified. A compiler needs each of these, and each one works today. - -| Capability | Status | Evidence | -|---|---|---| -| Recursion | ✅ | `fib(10)` → `55` | -| Mutual recursion / forward references | ✅ | `odd`/`even` cross-call | -| Deep recursion (recursive-descent parsing) | ✅ | 5,000 frames, no crash | -| Heap allocation | ✅ | `mem_alloc`, `mem_free`, `mem_copy`, `mem_set`; 1 MiB alloc verified | -| Byte-level memory | ✅ | `peek8`/`poke8`, `peek32`/`poke32`, `peekp`/`pokep`, `ptr_add` | -| `ptr` locals, params, returns | ✅ | `function make(n: int) -> pointer` | -| `ptr` in a property field | ✅ | `property Nd { kind: int = 0, a: pointer = ptr_null() }` | -| String literals as readable bytes | ✅ | `peek8("hello", 1)` → `101` | -| `str` accepted where `ptr` expected | ✅ | `f("A")` into `function f(p: pointer)` | -| String comparison, **hand-written in Ludic** | ✅ | `streq` over `peek8` | -| Integer → decimal, **hand-written in Ludic** | ✅ | `itoa(48291)` → `"48291"` | -| File read: open/seek/tell/read/close | ✅ | full round-trip of a written file | -| File write | ✅ | `file_open`/`file_write`/`file_close` | -| Module-level mutable state | ✅ | `var count: int`, `var heap: pointer` | -| `let` is mutable | ✅ | `i = i + 1` in a loop | -| `while`, numeric `for i in a .. b` with runtime bounds | ✅ | | -| `if` / `else if` / `else` chains | ✅ | | -| `match` with multi-value arms and `_` | ✅ | `1 => … 2, 3 => … _ => …` | -| Bitwise ops | ✅ | `band`/`bor`/`bxor`/`bnot`/`shl`/`shr` | -| `shr` is **logical**, not arithmetic | ✅ | `shr(-16, 1)` → `2147483640` | -| Character literals | ✅ | `'x'`, `'\n'`, `'\0'` lex to ints | -| Exit codes | ✅ | `os_exit(3)` → shell sees `3` | -| Separate compilation, C ABI | ✅ | `module` + `@export fn`, `extern fn … = "sym"` | - -**The consequence:** a compiler is *already expressible* in Ludic today. You -could write a lexer, a parser building nodes as hand-offset `peek32`/`poke32` -records, a symbol table, and an IR text emitter, using nothing above. It would -be miserable to read and maintain at 6,000 lines — but nothing in §4 is a -*capability* blocker except argv. The rest is about whether the resulting source -is something a human or a model can work in. - -That distinction shapes the whole plan: **this is mostly an ergonomics project -with one small hole in it**, not a language-design project. - ---- - -## 4. What the language is missing - -Each entry: the gap, why a compiler specifically needs it, the proposed design, -and the lowering. Verified-missing means it is a compile error today. - -### Tier A — real blockers - -#### A1. Command-line arguments ❌ *the only true capability blocker* - -``` -ludicc: error: line 1: unknown function 'os_argc' -``` - -`ll_emit_main` in `compiler/native.c` emits `define i32 @main()` — **no -parameters**. A self-hosted `ludicc` has no way to learn which file to compile. -Everything else in this document has a workaround; this one does not. - -**Design.** Two intrinsics: - -```ludic -# doc-check: skip — proposed signature notation, not code -os_argc() -> int -os_arg(i: int) -> str -``` - -**Lowering.** Change the signature to `define i32 @main(i32 %argc, ptr %argv)`, -store both into `@L_argc` / `@L_argv` in the entry block, then `os_argc()` is a -load and `os_arg(i)` is exactly the existing `peekp(@L_argv, i)` path. Add to -`INTRINSICS[]` in `native.c`. - -**Cost.** ~30 lines of C. This is the single highest-value change in the -document: it is what turns "a Ludic program" into "a Ludic command-line tool". - -#### A2. Aggregate types (`struct`) ❌ - -``` -ludicc: error: line 2: expected declaration (got 'struct') -``` - -An AST node, a token, a symbol-table entry and a type descriptor are all -records. Today there are two workarounds, and both are bad at compiler scale: - -- **Hand-offset memory** — `poke32(n, 0, kind)`, `pokep(n, 1, child)`. This is - what `truetype.ludic` does, and it works, but every field access becomes a - magic number. Across a 6,000-line compiler this is the difference between - maintainable and not. -- **ECS entities as nodes** — verified working (`property Nd { kind, a: pointer }`), - and initially seductive because queries give you free traversal. **Do not do - this.** `LUDIC_MAX_ENT` is 1024 in `native.c:18`; the entity world is a fixed - array of per-property storage. A compiler needs hundreds of thousands of - nodes. This is a dead end, and it is worth writing down because it is the - obvious wrong turn. - -**Design — reference semantics, not value semantics.** The cheap version that -unblocks everything: - -```ludic -# doc-check: skip — proposed syntax: struct does not exist yet -struct Tok { kind: int = 0, text: pointer = ptr_null(), line: int = 0 } - -let t = new Tok # heap-allocated, fields seeded from defaults -t.kind = T_ID -print_int(t.line) -free Tok t # or leak it; see §8 on arenas -``` - -No copying, no by-value passing, no nested-struct inlining — a `struct` value -*is* a `ptr` with a known layout, so it costs nothing in the type system beyond -a layout table. - -**Lowering.** This is largely already built. `native.c` already emits -`%Cmp_` LLVM struct types for properties and already resolves -`a.b` through `ll_member_addr` with `getelementptr`. A `struct` is a -`%Cmp_`-style type *without* the parallel entity arrays: `new` is -`malloc(sizeof)` plus a default-seeding memset/store sequence, and `.field` is -the existing `getelementptr` path. Reusing the property machinery is why this -is far cheaper than it looks. - -**Cost.** ~250 lines of C across `ludicc.c` (parse) and `native.c` (layout, -`new`, member access). Highest cost in the document, and the highest payoff. - -#### A3. Arrays and indexing ❌ - -``` -ludicc: error: line 2: expected identifier (got '[') -``` - -Token buffers, string tables, keyword tables, scope stacks. Currently -`mem_alloc` + `peek32`, which works but reads badly. - -**Design.** - -```ludic -# doc-check: skip — proposed syntax: array types do not exist yet -var keywords: [str; 64] # fixed-size module-level storage -let toks: [Tok; 0] = mem_alloc(n * size_of(Tok)) # or a growable buffer -toks[i].kind = T_ID # composes with A2 -``` - -**Lowering.** `[T; N]` is `[N x ]`, already exactly how `@L_alive` and -`@S_` are emitted. `a[i]` as both rvalue and lvalue is a -`getelementptr` — the same code path as member access, indexed instead of -named. The important part is that `toks[i].kind` composes: index then member, -one GEP chain. - -**Cost.** ~150 lines. Should land *with* A2, since neither is much use alone. - -#### A4. `break` / `continue` ❌ - -``` -ludicc: error: line 3: unknown identifier 'break' -``` - -Lexers and parsers are made of `while (1) { … break; }`. The workaround — -sentinel booleans threaded through every loop condition — is the kind of thing -that makes a 6,000-line port unreadable. - -**Design.** `break`, `continue`. No labels; nested loops in a compiler rarely -need them, and adding labels later is compatible. - -**Lowering.** `native.c` already maintains `ll_loopstk[64]` (for `self()` inside -queries). Extend each frame with `break_label` and `continue_label`, then -`break` is `br label %`. Note the existing gotcha recorded in the native -backend notes: **stack slots must be emitted in the entry block** — no new -allocas at the break site. - -**Cost.** ~40 lines. Best value-per-line in the document. - -#### A5. `mem_realloc` ❌ - -``` -ludicc: error: line 1: unknown function 'mem_realloc' -``` - -Every table in a compiler grows: tokens, nodes, the output buffer. Hand-rolling -alloc-copy-free works but is written once per table and gotten wrong once per -table. - -**Design.** `mem_realloc(p: pointer, n: int) -> pointer`. - -**Lowering.** `declare ptr @realloc(ptr, )` plus one `INTRINSICS[]` -entry. **Use `ll_size_t()` / `ll_widen()` for the size argument — do not -hardcode `i64`.** `size_t` is `i32` on wasm32, and `native.c` now routes every -size-taking intrinsic through those helpers for exactly this reason. - -**Cost.** ~6 lines. - -#### A6. Diagnostics on stderr ❌ - -``` -ludicc: error: line 1: unknown function 'print_err' -``` - -Only stdout exists (`print_str` → `printf`, `write_byte` → `putchar`). This is -not cosmetic: **`ludicc --emit llvm` writes IR to stdout.** A self-hosted -compiler that printed errors to stdout would interleave diagnostics into its own -output, corrupting it in exactly the case you most want a diagnostic. - -**Design.** Prefer an intrinsic that yields a handle, so the existing file -plumbing is reused rather than duplicated: - -```ludic -# doc-check: skip — proposed signature notation, not code -file_stderr() -> pointer # then file_write(f, buf, n) as usual -``` - -**Lowering — note the portability wrinkle.** There is no portable `@stderr` -global in LLVM IR: Darwin exports `@__stderrp`, glibc exports `@stderr`, and -wasm has neither in the same shape. So `file_stderr()` must select per target, -alongside the existing `target_os()` logic in `driver.c`. This is the one item -here that is genuinely target-dependent rather than merely unimplemented, and -it should be designed with that in mind rather than bolted on. - -**Cost.** ~40 lines including the per-target selection. - -### Tier B — needed for *complete* bootstrap, not for the compiler - -#### B1. Function pointers ❌ - -``` -ludicc: error: line 3: unknown type 'fn' for var h -``` -`&cb` also fails to compile. - -The compiler itself does **not** need these — `match` dispatch covers every -place a C compiler would use a function pointer table. - -But they are what would let `cocoa.ll` become Ludic. The macOS window builds an -`NSView` subclass at runtime with `objc_allocateClassPair` and installs **an IR -function as its IMP**. Without the ability to take the address of a Ludic `fn`, -that shim can never move out of hand-written IR. So: irrelevant to §6, and -load-bearing for §7.4. - -**Design.** `&fnname` yields a `ptr`; call through it via -`call_ptr(p, args…)` or a typed `fn(int)->int` type. - -**Cost.** ~120 lines. Defer until after the fixpoint. - -#### B2. String operations — **no language change needed** - -`str + str` is worth calling out as a *bug*, not a gap. It passes the front-end -and then emits invalid IR: - -``` -build/probe_t_headless.ll:13401:17: error: global variable reference must have pointer type -``` - -That is a front-end/backend mismatch: the typechecker accepts an operation the -backend cannot lower. Until strings exist properly, `str + str` should be a -clean compile error rather than a `clang` error in generated code. - -Everything else a compiler needs from strings is **already writable in Ludic -today** — `streq` and `itoa` are verified. This is not a language gap; it is a -library to write (§6 Stage 1), and it is the largest pure-typing chunk of the -whole project. - -### Tier C — explicitly out of scope, recorded so they are not rediscovered - -| Gap | Why it does not block | -|---|---| -| **64-bit integers** ❌ (`100000*100000` → `1410065408`, wraps at i32) | Line numbers, offsets, node indices and string lengths all fit in `i32`. Only matters for source files > 2 GiB. | -| **A non-ECS entry point** | A `Start`-phase system plus `os_exit(n)` gives correct exit codes — verified. You do pay for an unused 1024-entity world; that is a constant, not a blocker. A `tool Name { function main() -> int }` form would be nicer, not necessary. | -| Closures, generics, unions, sum types | A compiler in the style of `ludicc.c` uses none of them. | -| GC | A compiler should leak deliberately (§8). | -| Unsigned integer types | `shr` is already logical and `band`/`bor` are bit-level — sufficient. | -| Multiple return values | `ptr` out-parameters work today. | - ---- - -## 5. Designing for readers — human and model - -The goal: Ludic source should be obvious to a person skimming it and -unambiguous to a model generating it. Those two goals agree far more than they -conflict, and where they conflict the resolution is **regularity, not -verbosity** (§5.2). - -Everything in this section was verified against the built compiler. The probes -are in the appendix under "Syntax audit". - -### 5.1 Why this belongs in the bootstrap document, and why now - -**Syntax changes are cheap today and expensive after Stage 3.** This is a hard -ordering constraint, not a preference. - -Today, changing the grammar costs: edit `ludicc.c`, `sed` three examples and -five runtime files, run `bin/x test`. An afternoon. - -After the fixpoint, `ludicc` is *written in the syntax it parses*. Every change -becomes a four-step dance: build a compiler that accepts both old and new forms -→ compile it with the old seed → rewrite every source file → remove the old -form and regenerate the seed. That is what every mature language does, and it -is why mature languages change syntax slowly. It is not a reason to avoid the -change; it is a reason to **make it before the port, not after**. - -So the plan gains a stage: - -> **Stage 0.5 — Syntax freeze.** Between Stage 0 (language features) and -> Stage 1 (libraries). Nothing in Stage 2 starts until the grammar is final. - -The port should be *the first large program written in final Ludic*, not the -last large program written in provisional Ludic. - -**This work also strengthens the bootstrap itself.** Stage 2b uses `--fmt` -equality as the oracle proving two parsers agree. That oracle is only as tight -as the language is regular: every alternative spelling is surface variance the -formatter must erase. Reduce the variance and the oracle gets sharper. The -readability project and the self-hosting project are not competing for the same -time — one makes the other more trustworthy. - -### 5.2 What actually helps a model — and what is folklore - -Worth being precise here, because "AI-friendly syntax" attracts a lot of -confident nonsense. - -**Genuinely helps:** - -| Property | Why it matters | -|---|---| -| **Low syntactic variance** — one spelling per concept | Every alternative is a branch point during generation and a case in the parser. Two ways to write a list is two chances to be inconsistent within one file. | -| **Leading-keyword, bounded lookahead** | Every declaration and statement identifiable from its first token. Helps the hand-written recursive-descent parser Stage 2b will be, *and* a model predicting forward. | -| **No silent no-ops** | If the language accepts a construct it must either honour it or reject it. Accepting-and-ignoring teaches a falsehood (see R6 — the worst thing in the audit). | -| **Recoverable structure** — explicit terminators | A slightly-wrong generation fails *locally*, with an error pointing at the mistake, instead of cascading into a confusing error 40 lines later. | -| **Locality** — meaning readable from the construct | No action-at-a-distance. Ludic is already strong here; keep it. | -| **Greppable unique anchors** | `property Pos` is findable. Retrieval quality is a language design property. | -| **Errors that name the fix** | Already partly true: a missing builtin errors naming `rt_`. Extend that everywhere. | - -**Folklore, and false:** - -- *"More verbose is more AI-friendly."* No. Ceremony without information hurts - both audiences. What helps is redundancy that **encodes intent** — an explicit - type, a closing keyword — not boilerplate. -- *"Significant indentation reads better."* It reads fine and **generates - badly**: indentation drift across a long generated block is unrecoverable and - survives review. Ludic uses braces. Keep them. -- *"Natural-language-like syntax helps."* Prose-shaped keywords add ambiguity. - Consistent symbols beat English words that read three ways. -- *"Terseness is bad for models."* Terseness is fine; *irregularity* is the - problem. A short form used consistently is easy to predict. - -**The real tension:** humans skim, so terseness helps them; machines benefit -from redundancy. Regularity resolves it — the same shape everywhere costs a -human nothing once learned, and costs a model nothing to predict. - -### 5.3 Audit — what is irregular in Ludic today - -Each row verified by compiling a probe, not by reading docs. - -| # | Irregularity | Evidence | Cost | -|---|---|---|---| -| **R1** | **No statement terminator at all.** `block()` is `skipnl(); stmt()` in a loop. A newline *stops* an expression (it lexes as `T_NL`, and `binlevel` only continues on `T_OP`) but is never *required*. `let x = 1 x = x + 1 print_int(x)` on one line is three legal statements — verified compiling. | `ludicc.c` `block()`, `binlevel` | The reader cannot see where a statement ends without re-deriving operator precedence. Blocks error recovery entirely. | -| **R2** | **Commas are optional everywhere.** `if(isop(",")) pi++` appears in `comp()`, `arche()`, `fn` params and `spawn`. `{ x: int = 0 y: int = 0 }` and the comma'd form both compile. | 4 parser sites | Two spellings, zero semantic difference. | -| ~~**R3**~~ | ~~**`and`/`or` alias `&&`/`\|\|`.**~~ **RESOLVED** — `and`/`or`/`not` are the only boolean operators; `&&`, `\|\|` and `!` are each rejected with a diagnostic naming the fix, and all three words are reserved. `!=` is unaffected. | landed via S3 | — | -| **R4** | **`{ }` means seven different things** — statement block; property fields (`n: T = e`); model list (bare idents); spawn initialisers (`N = { … }`); ui props + children (`k=v` juxtaposed, no commas); match arms (`p, p => …`); machine states (`state N = v { … }`). | `block/comp/arche/spawn/parse_widget/match/machine` | The delimiter carries no information. You must already know the head keyword to know the inner grammar. | -| **R5** | **Contextual keywords, not reserved.** `phase`, `query`, `reads`, `writes`, `needs`, `uses`, `where`, `in`, `on`, `layer`, `state`, `start` are matched with `isid()` — ordinary identifiers. `let query = 5 let phase = 6` compiles and prints `11`. | `sys()`, `scene_decl()` | A local named `enter` or `match` produces a baffling error far from the cause. | -| **R6** | **Contracts are parsed and thrown away.** `requires`/`ensures`/`invariant` parse an expression and **discard it** (`pi++; expr();`). `reads`/`writes`/`needs`/`uses`/`effects` are `skip_brackets()`. `pure` is consumed and ignored. Verified: `function half(n: int) -> int requires n > 100000 ensures false` compiles, and `half(8)` returns `4`. Verified: a system declaring `reads [Pos]` that **writes** `p.x = 99` compiles. | `fn()`, `sys()` | **The worst item in the audit.** The language accepts a contract and does nothing. A model writing `requires n > 0` is rewarded with a clean compile and zero enforcement — it learns a lie, and so does a human reader trusting the annotation. | -| **R7** | **`str + str` typechecks, then emits invalid IR.** | verified (§4 B2) | The front-end accepts what the backend cannot lower. | -| **R8** | **Two formatters, opposite philosophies, both called "format".** `ludicc --fmt` canonicalises hard (one statement per line, `and`→`&&`, full parenthesisation) but drops comments and inlines imports. `ludic-fmt` is token-based and preserves comments — but **normalises nothing**: handed the one-line `let a = 1 a = a + 1 if true and false { … }`, it returned it unchanged. | verified side-by-side | **Neither tool enforces a single spelling.** The canonicaliser is unusable on real source; the source formatter has no opinion. | -| **R9** | **Two ways to spell a tag** — `property Player { }` (empty property) or `model`. | LANGUAGE.md | | -| **R10** | **Stale docs are stale training data.** LANGUAGE.md still says "the current compiler is a tree-to-C translator" (it emits LLVM IR) and lists arrays under "Not yet implemented" beside things never planned. | LANGUAGE.md | Docs are the highest-leverage model input in the repo. A wrong doc is worse than a missing one. | - -### 5.4 Proposals - -Ordered by value per line of work. Each is a Stage 0.5 item unless noted. - -**S1. Require a statement terminator.** A statement ends at a newline, `;`, or -`}`. Make `T_NL` significant inside `block()` instead of discarding it. -*Why:* fixes R1, and it is the precondition for error recovery — without it a -parser cannot resynchronise, so every syntax error stays a cascade. -*Cost:* ~30 lines. *Ripple:* one-line bodies like `if x { a }` still work; -multi-statement one-liners in the runtime need a `sed`. - -**S2. Make separators mandatory.** Commas required in every comma-list; -remove the optional path. *Fixes R2. Cost:* ~10 lines + tree-wide `sed`. - -**S3. One spelling for boolean operators. ✅ LANDED.** `and`/`or` are the only -boolean operators. Ludic already spells bitwise operations as functions -(`band`/`bor`), so the symbols bought nothing, and dropping them removes the -`&` vs `&&` bug class by construction. - -What shipped: `&&`/`||` still *lex* as single tokens, purely so the parser can -emit `'&&' is not a Ludic operator - write 'and'` instead of tripping over a -stray `&`; `and`/`or` became reserved words, so `let and = 5` is rejected at the -mistake; the AST op string is now `"and"`/`"or"`, which is **exactly the LLVM -opcode**, so the lowering ternary collapsed to passing `op` straight through; -and `--fmt` emits the new spelling for free, since it prints the op string. -Six regression tests in `bin/x test` (64 → 70), including one asserting no `.ludic` -source uses the symbols outside a comment. *Fixed R3.* - -**S4. Reserve every keyword.** One table, shared by the lexer, parser, -`ludic-fmt` and `ludic-lsp` — those tools already share a vocabulary in -`ludic_syntax.h`, so there is one obvious home. Reject `let query = 5` at the -point of the mistake. *Fixes R5. Cost:* ~40 lines. - -**S5. Delete or implement every silent no-op.** ← **highest value in the -section.** Two honest options per construct, no third: - -- `reads` / `writes`: **implement them.** The compiler already knows every - property a system touches — it builds the query and walks the body. Checking - the declaration against actual access is a genuine static analysis the - language claims to have and doesn't. This converts dead syntax into a real - guarantee, which is exactly what an "AI-first" language should offer a model - reasoning about a system in isolation. -- `requires` / `ensures`: either lower to a checked assertion in debug builds - (`if !cond { print_err(...) os_exit(1) }` — cheap, and A6 stderr lands in - Stage 0 anyway), or remove them from the grammar until they mean something. -- `pure`, `needs`, `uses`, `effects`, `invariant`: remove until implemented. - -*Fixes R6. Cost:* ~150 lines for `reads`/`writes` checking, ~60 for assertions, -~10 to delete the rest. - -**S6. Cut the block grammars from seven to two.** Full unification is too -invasive to be worth it. The achievable version: every `{ }` is either a -**statement block** or a **field list** (`name: type = default`, comma-separated, -one shape), and `ui` props adopt the same separator rule as everything else. -Document all remaining shapes in one grammar table. *Partially fixes R4. -Cost:* ~120 lines. - -**S7. One formatter with one contract.** Merge the philosophies rather than -keeping two half-tools: `ludic-fmt` gains `--fmt`'s normalisation decisions -(statement-per-line, single spelling, consistent commas) while keeping its -token-based comment preservation, and becomes **normative** — `ludic-fmt ---check` gates CI. `ludicc --fmt` reverts to being an honest debug dump and is -renamed `--dump-ast`. *Fixes R8.* After S1–S3, canonical form is the *only* -form, so the formatter stops being a style preference and becomes a check. -*Cost:* ~200 lines, mostly in `ludic_fmt.h`. - -**S8. Machine-readable grammar and diagnostics.** Emit the grammar as one EBNF -file, and give every diagnostic a stable code plus a one-line suggested fix -(`ludicc --explain L0412`). Feeds the LSP, the docs and any model at once. -*Cost:* ~250 lines. *Defer to after the fixpoint* — valuable, not ordering-critical. - -**S9. Documentation hygiene as a build step.** `bin/x test` already understands -` ```ludic ` fences. Extend it so **every fence in every `.md` must compile**, -and fix R10's stale claims. *Cost:* ~60 lines of shell. Do this early — it is -cheap and it stops the docs drifting further while the rest of the work lands. - -### 5.5 What not to change - -Recording these so they are not relitigated: - -- **Braces, not indentation** (§5.2). -- **`#` comments** — unambiguous, one spelling already. -- **The ECS vocabulary** — `property` / `system` / `query` / `phase` are - unusually self-describing and greppable. This is the language's best existing - readability asset. -- **`fixed` / Q16.16** — determinism is a design constraint, not a style choice. -- **Do not add** operator overloading, implicit conversions beyond `int`→`fixed`, - macros, or anything else with action-at-a-distance. Every one of them trades - local readability for cleverness. - -### 5.6 Sequencing - -| When | What | Why there | -|---|---|---| -| **Now, before Stage 1** | S9 (doc hygiene) | Cheap; stops further drift immediately. | -| **Stage 0.5** | ~~S3~~ ✅ done · S1, S2, S4, S5, S6, S7 | Must precede the port (§5.1). | -| **After Stage 3** | S8 (EBNF + diagnostic codes) | Valuable, not ordering-critical; better written in Ludic against the self-hosted parser. | - -**S3 was the one genuinely contentious call** — which boolean spelling — because -it is pure taste and touches every file. It was decided in favour of `and`/`or` -and has landed. Everything remaining in this section is a choice between "one -spelling" and "two", where the answer is not in doubt. - -**`!` → `not` has since landed too**, on the same reasoning and by the same -mechanism: `!` still lexes (so `!=` is untouched) purely so the parser can say -`'!' is not a Ludic operator - write 'not'`. Ludic's three boolean operators are -now `and`, `or`, `not`, all reserved words, with no symbol spellings at all. - ---- - -## 5.7 Status — self-hosting achieved - -Updated 2026-08-27. `bin/x test` = 93/93, `bin/x test-tools` = 28/28, -`bin/x selfhost-test` = 5/5 including the bootstrap fixpoint. - -**Ludic is fully self-hosted.** The compiler is written in Ludic -(`selfhost/*.ludic`, ~2,400 lines), compiles every example to byte-identical -output and its own source to a fixpoint, and is built from a checked-in IR seed -with **no C compiler** — the former C compiler has been deleted. - -From a clean checkout, build the compiler and the task-runner in one line: - -```bash -clang selfhost/ludicc.seed.ll -o bin/ludicc && bin/ludicc tools/x/main.ludic -o bin/x -``` - -Thereafter `bin/x build` rebuilds the entire toolchain into `bin/` (`ludicc`, -`ludic`, `x`, `ludic-fmt`, `ludic-lsp`), `bin/x bootstrap-cfree` reproduces the -compiler from the seed with no C compiler, and `bin/x help` lists every command. -Run `bin/x` from the repository root. - -| Stage | What | State | -|---|---|---| -| **0** | language features (argv, struct, arrays/slices, break/continue, mem_realloc, stderr) | ✅ done | -| **0.5** | S3 (`and`/`or`/`not`), S9 (doc checking), short-circuit `and`/`or` | ✅ done | -| — | S1/S2/S4/S5/S6/S7 (statement terminators, mandatory commas, reserved-word audit, no-op removal, block unification, one formatter) | not done — polish of the *full* language, not needed for self-hosting | -| **1** | support libraries in Ludic (`str`, `buf`, `io`) | ✅ done | -| **2** | the compiler ported to Ludic (`lex`, `parse`, `emit_*`) | ✅ done | -| **3** | the fixpoint (`gen2.ll == gen3.ll`) | ✅ done | -| **4** | retire the C as a *live dependency* (IR seed, C-free rebuild) | ✅ done — `bin/x bootstrap-cfree` | -| **4+** | retire `ludicc.c` entirely (port the game backend) | ✅ **done** — `compiler/` deleted; the compiler is `selfhost/*.ludic` | - -### What "self-hosting" means here, precisely - -The self-host compiler (`selfhost/`) implements the **compiler-subset**: `struct` -(reference), `[]T` slices with `push`/`len`, functions, a plain `main` entry, -the full control flow, the operators (with short-circuit `and`/`or`), and the -low-level intrinsics. It deliberately does **not** implement the game half of -Ludic — ECS, queries, models, scenes, UI, save/load, `match`/`machine`, -fixed-point. It targets native (macOS/clang) and emits LLVM IR text that clang -assembles, exactly the posture the C `ludicc` has. - -It is written entirely in that subset, which is why it compiles itself. The -three-generation proof (`bin/x bootstrap`): - -``` -stage0 build/ludicc (C) compiles selfhost.ludic -> gen1 (a Ludic-written compiler) -stage1 gen1 compiles selfhost.ludic -> gen2.ll -> gen2 -stage2 gen2 compiles selfhost.ludic -> gen3.ll -assert gen2.ll == gen3.ll # the compiler reproduces itself, independent of its seed -``` - -`gen1`'s IR legitimately differs (a different compiler built it); `gen2 == gen3` -is the property that matters — the Ludic compiler has no dependency on how it was -built. It is also verified *correct*, not merely self-consistent: it compiles a -corpus (`selfhost/tests/`) of struct, slice, and control-flow programs to -binaries that produce the expected output. - -### Stage 4 — the C is retired as a live dependency - -The self-hosted compiler no longer needs the C `ludicc` to exist. Its own LLVM -IR is checked in as `selfhost/ludicc.seed.ll` — a proven fixed point — and -`bin/x bootstrap-cfree` assembles that with clang (an IR assembler, the -floor Rust and Swift stand on) and rebuilds the compiler, which reproduces its -own IR. **The C source is never invoked.** This is the seed path §8 recommended. - -Crucially, the compiler **evolves** without the C compiler: `bin/x reseed` -uses the *current* seed to build a compiler with new source, then takes that -compiler's own output as the new seed. New features (this session: `match`, -bitwise ops, `peek32`/`poke32`) landed and reseeded entirely C-free. The C -compiler is now a historical seed, not a dependency. - -### Stage 4+ — `ludicc.c` is deleted - -The self-host compiler was extended to the **whole** language — properties, -models, systems, phases, `for … in query` (with `where`), spawn/despawn, -`self()`, `machine`/`become`, `match`, `save`/`load` snapshots, the retained -`ui` widget tree, multi-file `import`, fixed-point Q16.16, and every runtime -intrinsic. It auto-splices the Ludic runtime exactly as the C compiler did. - -It now compiles **every example** — `snake`, `menu`, and the 6-file JRPG -`chronorift` — to output byte-identical to the original C compiler (checked -against golden renders in `selfhost/golden/`), and still compiles its own source -to a fixpoint. The C compiler (`compiler/`, ~2,700 lines) has been **deleted**. -`bin/x build` builds `bin/ludicc` from the IR seed with clang, and `bin/x app` -drives the native link (headless, or windowed via `cocoa.ll`). - -What did not come across: the old C driver's **wasm target, cross-compilation, -and shared-library** paths. Those are driver features, not codegen — the -self-host compiler emits native-ABI IR — and re-implementing them on the -self-hosted toolchain (wasm needs i32 `size_t`; the others are clang flags in -the `bin/x app` build path) is the remaining follow-up. - ---- - -## 6. The plan - -### Stage 0 — Extend the C compiler (~500 lines of C) - -The C `ludicc` must be able to compile the Ludic `ludicc`. Land Tier A only, in -this order — cheapest-and-unblocking first: - -1. **A4** `break`/`continue` (~40) — immediate readability win on everything after. -2. **A5** `mem_realloc` (~6). -3. **A1** `os_argc`/`os_arg` (~30) — unblocks the entire notion of a CLI tool. -4. **A6** `file_stderr` (~40). -5. **A2 + A3** `struct` + arrays (~400, landed together). - -Each gets a test in `bin/x test` as it lands. The suite is at 64/64; Stage 0 should -leave it green and larger. - -**Explicitly not in Stage 0:** function pointers, 64-bit ints, a `tool` entry -form. They are not on the path to the fixpoint. - -### Stage 0.5 — Syntax freeze (~600 lines of C + a tree-wide `sed`) - -**The grammar must be final before Stage 2 starts** (§5.1): after the fixpoint, -every syntax change costs a four-step reseed instead of an afternoon. - -Land S1–S7 from §5.4: mandatory statement terminators, mandatory separators, -one boolean spelling, reserved keywords, no silent no-ops, two block shapes -instead of seven, one normative formatter. S9 (doc hygiene) can land earlier — -it is cheap and independent. - -Exit criterion: `ludic-fmt --check` passes on the whole tree and there is -exactly one legal spelling of every construct. That is also what makes the -Stage 2b oracle tight. - -### Stage 1 — Support libraries in Ludic (~800 lines of Ludic, zero language work) - -Nothing here needs Stage 0 except `struct`/arrays for pleasantness. This is the -part that is pure writing, and it can start immediately and in parallel. - -| File | Contents | -|---|---| -| `runtime/native/strings.ludic` | `str_eq`, `str_len`, `str_dup`, `str_cat`, `substr`, `str_chr`, `str_hash`, `itoa`, `atoi`, `hex` | -| `runtime/native/buf.ludic` | growable byte buffer — `buf_new`, `buf_putc`, `buf_puts`, `buf_putint`, `buf_len`, `buf_ptr`. This is `SB` from `ludicc.c`, and the IR emitter is nothing but calls to it. | -| `runtime/native/io.ludic` | `read_whole_file` (the open/seek/tell/read/close dance, verified working), `write_whole_file`, stderr diagnostics | -| `runtime/native/map.ludic` | open-addressing `str -> int` hash table: keyword lookup, string interning, symbol tables | -| `runtime/native/arena.ludic` | bump allocator — see §8 | - -### Stage 2 — Port the compiler, each piece against a differential oracle - -Port in dependency order. The critical discipline: **never port a stage without -an automated way to prove it agrees with the C one.** Ludic is unusually well -set up for this, because it already ships two canonical serializers of compiler -internals. - -| Sub-stage | Port | Differential oracle | -|---|---|---| -| 2a | `lex.ludic` | Dump the token stream from both compilers; `diff` over every `.ludic` in the tree. | -| 2b | `parse.ludic` (AST) | **`--fmt` is a free oracle.** The formatter is already a canonical AST printer, and `bin/x test` already asserts formatting never changes a program. If both compilers' `--fmt` output is byte-identical on every file, the parsers agree. | -| 2c | `check.ludic` | Diagnostic text must match on a corpus of deliberately-broken programs. `bin/x test` already checks diagnostics — extend that corpus. | -| 2d | `emit.ludic` (IR) | **`--emit llvm` must be byte-identical** for every example. This is the strongest oracle available: pass/fail on exact text, no judgement. | -| 2e | `drive.ludic` | Assemble and link via `clang`; compare final binaries. | - -Sub-stage 2b deserves emphasis. Most self-hosting projects have no cheap way to -prove two parsers agree. Ludic has one already built and already tested, which -removes the single largest source of silent divergence. - -### Stage 3 — The fixpoint - -``` -stage1 = C-ludicc compiles ludicc.ludic -> binary A -stage2 = A compiles ludicc.ludic -> binary B -stage3 = B compiles ludicc.ludic -> binary C - -assert B == C byte-for-byte <- THE bootstrap test -``` - -`A != B` is expected and correct: `A` was built by a different compiler, so its -codegen differs. `B == C` is the real property — a compiler that reproduces -itself has no dependency on how it was built. Also assert that `A`, `B` and `C` -all emit identical IR for every example. - -If `B != C`, the cause is almost always nondeterminism in the compiler itself: -hash-table iteration order, an address baked into output, uninitialised memory. -Those are worth hunting rather than working around. - -### Stage 4 — Retire the C - -Once the fixpoint holds, the C compiler becomes a seed. Options: - -| Option | Trade-off | -|---|---| -| **Commit the generated `ludicc.ll`** ✅ recommended | Auditable text, diffable in review, builds with `clang` alone — already a dependency. Large but honest. | -| Commit prebuilt binaries per platform | Smallest process, worst auditability; a binary blob nobody can read. What Rust does. | -| Keep `ludicc.c` forever as the seed | Zero risk, but §1's bar is never met — the C never leaves. What Go did for years. | - -Recommend the IR seed: it is the only option that both removes the C and leaves -a reviewer something to read. - -`tools/ludic-tools/` (3,260 lines of C: `ludic-fmt`, `ludic-lsp`) is a separate -port and should follow, not lead — once the Ludic compiler exists, both tools -should be thin front-ends over its lexer and parser instead of maintaining a -second copy of the vocabulary. - ---- - -## 7. Toolchain independence — and where to stop - -`driver.c` shells out to `clang` (overridable via `$LUDIC_CC`) to assemble IR -into an object and to link, plus `wasm-ld` for wasm. Three levels of removing -that, and only one is worth doing: - -**Level 1 — self-hosted compiler, hosted toolchain. ← the goal.** -`ludicc` is Ludic; `clang` remains the IR assembler and linker driver. This is -exactly where Rust and Swift stand. Achieved at the end of Stage 4. - -**Level 2 — own object writer.** Emit Mach-O / ELF / COFF directly, replacing -IR-text + `clang -c`. Requires instruction selection, register allocation and -relocations: realistically 5,000–15,000 lines of Ludic, and it *loses the LLVM -optimizer* — the generated code gets slower, which for a game language is a -real regression, not a neutral trade. **Not recommended.** - -**Level 3 — own linker.** Platform-specific, deep, and buys nothing a user can -perceive. **No.** - -**7.4 — The hand-written IR.** `cocoa.ll` (327 lines) and `wasm.ll` are LLVM IR, -not Ludic. Two defensible positions: keep them as *platform glue written in the -platform's own assembly language* (precisely how Rust uses `asm!` shims and how -every libc has hand-written syscall stubs), or move them into `.ludic` — which -needs **B1 function pointers**, because the `NSView` subclass installs a -function as an Objective-C IMP. Keeping them is the honest default; the README's -existing framing ("the same floor Rust and Swift stand on") already covers it. - ---- - -## 8. Risks and gotchas - -- **Do not build the AST out of ECS entities.** `LUDIC_MAX_ENT` is 1024 - (`native.c:18`) and property storage is fixed arrays. It compiles, it looks - elegant, and it caps the compiler at 1024 nodes. Use `struct` (A2). -- **Do not inherit the C compiler's fixed caps.** `ludicc.c:22` has - `g_srcpath[128]`; `native.c:161` has `Val a[8]`. The Ludic port should grow - its tables (A5) rather than reproduce the limits. -- **Leak on purpose.** A compiler runs once and exits. A bump arena - (`arena.ludic`) that never frees is faster and simpler than tracked - ownership, and it sidesteps having no GC. Free at process exit — i.e. never. -- **Determinism is a feature now.** Anything order-dependent — hash iteration, - pointer values in output, uninitialised reads — breaks `B == C` in Stage 3. - Iterate tables in insertion order, not bucket order. -- **Error handling has no exceptions.** Mirror the C `die()`: write the - diagnostic to stderr (A6), then `os_exit(1)`. -- **The `str + str` mismatch (B2)** is a live example of the front-end accepting - what the backend cannot lower. Worth auditing for siblings before trusting - the typechecker as a Stage 2c oracle. -- **Size-taking intrinsics must use `ll_size_t()` / `ll_widen()`.** `size_t` is - `i32` on wasm32. Any new intrinsic with a size argument (A5) that hardcodes - `i64` will break the wasm target at link time. -- **Recursion depth is fine** — 5,000 frames verified, well past what a - recursive-descent parser needs on real source. - ---- - -## 9. Effort - -| Stage | Work | State | -|---|---|---| -| 0 | Tier A language features (argv, struct, slices, break/continue, mem_realloc, stderr) | ✅ done | -| 0.5 | `and`/`or`/`not` + short-circuit; doc checking (S9) | ✅ done (S1/S2/S4/S5/S6/S7 deferred — full-language polish) | -| 1 | `str`, `buf`, `io` support libraries in Ludic | ✅ done (`selfhost/`) | -| 2 | lexer + parser + AST + IR emitter, in Ludic | ✅ done (`selfhost/`, ~1,300 lines) | -| 3 | the fixpoint (`gen2.ll == gen3.ll`) + harness | ✅ done (`bin/x bootstrap`) | -| 4 | port the game backend, retire `ludicc.c` | ⛔ out of scope — mechanical continuation | - -The self-host compiler is **~1,300 lines of Ludic** covering the compiler-subset. -A `main`-tool entry point and short-circuit `and`/`or` were the two language -additions that made it self-compilable; the rest of Stage 0 was already in place. - -Roughly **6,000 lines of Ludic and 1,100 lines of C** to reach Level 1 — larger -than the 2,508-line C compiler it replaces, which is normal: the C version leans -on libc for everything in Stage 1. - -**Two independent critical paths, and they can run in parallel.** Stage 0 + Stage 0.5 -are C work on the existing compiler; Stage 1 is Ludic work that needs almost none of -it. The only hard barrier is that Stage 2 starts after *both*. - -**The critical path is short.** A1 (argv, ~30 lines of C) plus A2/A3 -(`struct` + arrays, ~400) plus A4 (`break`, ~40) is nearly all the *design* risk -in the project. Everything after it is typing against oracles that already -exist. - ---- - -## Appendix — probe programs - -Each was compiled with `bin/x app probe.ludic --headless` and run against the -current tree (`bin/x test` = 64/64). - -**Recursion** ✅ → `55` -```ludic -program P { - function fib(n: int) -> int { if n < 2 { return n }; return fib(n-1) + fib(n-2) } - handler B phase Start { print_int(fib(10)); quit() } -} -``` - -**String comparison, hand-written** ✅ → `1` -```ludic -function streq(a: pointer, b: pointer) -> bool { - let i = 0 - while true { - let ca = peek8(a,i) - let cb = peek8(b,i) - if ca != cb { return false } - if ca == 0 { return true } - i = i + 1 - } - return false -} -``` - -**Integer → string, hand-written** ✅ → `48291` -```ludic -function itoa(v: int, buf: pointer) -> int { - let n = 0 - let x = v - if x == 0 { poke8(buf,0,48); return 1 } - let tmp = mem_alloc(16) - while x > 0 { poke8(tmp, n, 48 + x % 10); x = x / 10; n = n + 1 } - let i = 0 - while i < n { poke8(buf, i, peek8(tmp, n-1-i)); i = i + 1 } - mem_free(tmp) - return n -} -``` - -**Read a whole file** ✅ → the compiler's front door -```ludic -let f = file_open("/tmp/x.txt", "rb") -file_seek(f, 0, 2) -let n = file_tell(f) -file_seek(f, 0, 0) -let b = mem_alloc(n+1) -file_read(f, b, n) -poke8(b, n, 0) -file_close(f) -``` - -### Syntax audit — every one of these compiles today - -Each is a spelling the language accepts; the point is that the *alternative* -spelling is equally legal (§5.3). - -**R1 — statements now require a separator (Rule B, syntax-redesign Phase 2)** → parse error -```ludic -# doc-check: skip — intentionally rejected under Rule B: needs a newline or ';' -program P { handler B phase Start { let x = 1 x = x + 1 print_int(x) quit() } } -``` -Statements no longer sit adjacent with only spaces between them; the compiler -reports `expected newline or ';' between statements`. Put each on its own line, -or separate them with `;` (both lex to the same separator token): -```ludic -program P { handler B phase Start { let x = 1; x = x + 1; print_int(x); quit() } } -``` - -**R2 — commas omitted throughout** → `7` -```ludic -# doc-check: skip — composite: declaration plus statements -property Pos { x: int = 0 y: int = 0 } -spawn Hero { Pos { x: 7 y: 2 } } -``` - -**R3 — RESOLVED.** Every symbol form is now rejected where it is written: -``` -ludicc: error: line 1: '&&' is not a Ludic operator - write 'and' (got '&&') -ludicc: error: line 1: '||' is not a Ludic operator - write 'or' (got '||') -ludicc: error: line 1: '!' is not a Ludic operator - write 'not' (got '!') -ludicc: error: line 1: 'and' is a reserved operator and cannot be used as a name -``` -`--fmt` prints `if ((true and false) or (1 < 2))` and `(not true)`, while unary -minus keeps its tight spelling `(-x)`. `!=` is untouched. - -**R5 — reserved-looking words used as locals** → `11` -```ludic -let query = 5 -let phase = 6 -print_int(query + phase) -``` - -**R6 — contracts accepted and discarded.** Both are violated; it compiles and -prints `4`: -```ludic -# doc-check: skip — composite: declaration plus statements -function half(n: int) -> int requires n > 100000 ensures false { return n / 2 } -print_int(half(8)) -``` -And a system may declare read-only access, then write — also compiles: -```ludic -handler Violate phase Update reads [Pos] query (p) [Pos] { p.x = 99 } -``` - -**R8 — the two formatters disagree about what "format" means.** Given -`property Pos { x: int = 0 y: int = 0 }` and a multi-statement one-liner, -`ludicc --fmt` rewrites both (one statement per line, `and`→`&&`, full -parenthesisation) while `ludic-fmt` returns the input **unchanged**. - -**Verified-missing** — each a compile error today: -```ludic -# doc-check: expect-error — every line here is a compile error by design -while i < 10 { i = i + 1; if i == 3 { break } } # unknown identifier 'break' -struct Node { k: int, a: pointer } # expected declaration (got 'struct') -var t: [int; 8] # expected identifier (got '[') -var h: fn = a # unknown type 'fn' for var h -let p = &cb # fails to compile -print_int(os_argc()) # unknown function 'os_argc' -print_err("x") # unknown function 'print_err' -let p = mem_realloc(ptr_null(), 10) # unknown function 'mem_realloc' -print_str("ab" + "cd") # passes front-end, invalid IR -let a = 100000; print_int(a*100000) # 1410065408 — i32 wrap -``` diff --git a/COMPILING.md b/COMPILING.md index be767471..85b6e3a2 100644 --- a/COMPILING.md +++ b/COMPILING.md @@ -10,7 +10,7 @@ > convenience wrapper over the compiler. `--fmt` is reimplemented as a lex+parse > gate (the doc-check hook). The `--target`/cross-compile and `--shared` paths are > still features of the old C driver not yet re-implemented on the self-hosted -> toolchain. See BOOTSTRAP.md §5.7. +> toolchain. See the [Bootstrap deep-dive](https://git.workshopsoft.io/workshopsoft/ludic/wiki/Bootstrap) §5.7 on the wiki. > > From a clean checkout, build the compiler and the task-runner in one line, then > let `bin/x` do the rest (run it from the repository root): diff --git a/EVENTS-DESIGN.md b/EVENTS-DESIGN.md deleted file mode 100644 index 4ff4d4c2..00000000 --- a/EVENTS-DESIGN.md +++ /dev/null @@ -1,670 +0,0 @@ -# Events & modding, expanded — a design doc - -> **Status: EV0 fully shipped; EV1 (spawn/despawn), EV2 (first cut) and EV3 -> shipped; EV4–EV7 are design.** Implemented, self-hosted to the C-free fixpoint, -> and each a `bin/x test` check: -> - **EV0** — `event`/`@On`/`emit` lowered to `@ev_` dispatch (compile-time -> listeners), **plus the foreign C ABI** (`ludic_on_`, the `%Ev_` payload -> struct, a fixed-capacity listener array), proven by a C mod in -> [`tests/mod_c/mod.c`](tests/mod_c/mod.c) binding -> [`examples/mod_host.ludic`](examples/mod_host.ludic). Byte-identical when no -> event is declared. ([`examples/events/events.ludic`](examples/events/events.ludic)) -> - **EV1** — public events across the **whole architecture**, every scope shipped: -> **program** (`@Public @OnStart`/`@OnQuit` → `program_start`/`program_quit`, -> [`examples/events/program_events.ludic`](examples/events/program_events.ludic)); **models** -> (`@Public @OnSpawn`/`@OnDespawn` → `model__spawn`/`_despawn`, -> [`examples/events/promote.ludic`](examples/events/promote.ludic)); **properties** (`@Public -> @OnAttach`/`@OnDetach`/`@OnEnable`/`@OnDisable` → `prop_

_attach` etc., -> [`examples/events/prop_events.ludic`](examples/events/prop_events.ludic)); **scenes** (a -> `public` scene → `scene__enter`/`_exit`, -> [`examples/events/scene_events.ludic`](examples/events/scene_events.ludic)); and **layers** (a -> `public` layer + `enable/disable layer L` → `layer__show`/`_hide`, -> [`examples/events/layer_events.ludic`](examples/events/layer_events.ludic)) — which also -> landed **SCENES E2 layer toggle** (`@LE_` flag gating a layer's handlers). -> - **EV2 / EV2b** — the world table: the reflection ABI, generated from the -> compile-time schema, so a mod reads, writes, scans, identifies, **and creates** -> entity state **by name** without compiling against the game. `ludic_prop_id` / -> `ludic_field_id` / `ludic_get` / `ludic_set` / `ludic_has` (read/write — -> [`world_mod.c`](tests/mod_c/world_mod.c)); `ludic_entity_count` / `ludic_kind` / -> `ludic_model_id` (scan and identify — [`world_scan.c`](tests/mod_c/world_scan.c)); -> `ludic_spawn(model_id)` (create, reusing the compiler's own spawn lowering — -> [`world_spawn.c`](tests/mod_c/world_spawn.c)); `get`/`set` address each field by -> its real struct offset, correct for `int`/`fixed`/`byte`/`ptr` and mixed layouts -> ([`world_mixed.c`](tests/mod_c/world_mixed.c)); and iterate -> (`ludic_query_next`, [`world_query.c`](tests/mod_c/world_query.c)). Emitted only -> for an ECS program that declares events, so event-free games stay byte-exact. -> The world table is complete: read, write, scan, identify, create, iterate. -> - **EV3** — `cancellable` events, the `cancel` verb, and `emit E(…)` as an -> expression returning the veto flag. ([`examples/events/cancel.ludic`](examples/events/cancel.ludic)) -> - **EV5** — leak-proof scoped listeners: `ludic_off_(token)` (explicit -> unregister; dispatch skips tombstoned slots), `ludic_on_entity_(entity, cb)` -> (entity-scoped), and a generated `ludic_sweep_entity` called from `despawn` that -> nulls every listener the dying entity owned — a listener can't leak past its -> entity. Proven by [`tests/mod_c/scoped_mod.c`](tests/mod_c/scoped_mod.c). -> - **EV6** — re-entrant `emit` is depth-bounded (`@ev_depth` vs `EV_DEPTH_CAP`): a -> listener may emit another event, but an event cycle traps as an early return -> instead of hanging the frame. Dispatch order was already deterministic (array, -> registration order). Proven by [`examples/events/recurse.ludic`](examples/events/recurse.ludic). -> -> - **EV7 (schema opening)** — a mod defines a brand-new component at runtime: -> `ludic_register_prop(name, nfields)` mallocs flat `[MAX_ENT × nfields × i32]` -> storage + a has-flag array and returns a prop id past the compile-time range; -> `ludic_attach_dyn`/`ludic_detach_dyn` toggle it on an entity; `get`/`set`/`has`/ -> `prop_id` fall through to the dynamic registry for ids ≥ the compile-time count. -> A mod adds entirely new data to entities by name, with per-entity isolation. -> Proven by [`tests/mod_c/world_dyn.c`](tests/mod_c/world_dyn.c). (EV7's other -> half — networking's local/remote event split — has no substrate in Ludic yet.) -> -> Still design: EV4 (the scripting-shim bridge — deferred to keep the suite -> interpreter-free) and EV7 networking. This is a companion to -> [LIFECYCLE-DESIGN.md](LIFECYCLE-DESIGN.md) and [SCENES-DESIGN.md](SCENES-DESIGN.md). -> Where those docs extend Ludic's *internal, compile-time* lifecycle, this one -> proposes the *external, runtime* layer that turns those same lifecycle moments -> into a public event surface — the foundation a game can hand to mods written in -> Ludic, JS/TS, Lua, or anything with a C ABI. It distills a survey of modding and -> event systems (§3) into a phased roadmap (EV0–EV7, §12–§13). §14 lists the open -> decisions. - ---- - -## 1. Thesis - -Ludic already has a lifecycle. `@OnSpawn(Enemy)`, `@OnDetach(Sprite)`, scene -`on enter`, `@OnDespawn(M, reason: r)` — every one is a **compile-time, -closed-world, zero-cost** hook that desugars to a direct call at a fixed site. -That is the right design for the *game author*, who is compiled together with the -game. It is exactly the wrong design for a *mod author*, who is not. - -A modding event system is the mirror image of the lifecycle layer along three axes: - -| | Lifecycle hooks (today) | Modding events (this doc) | -|---|---|---| -| World | **closed** — all handlers known at compile time | **open** — mods add listeners after compilation | -| Binding | **static** — a checked symbol, a direct call | **dynamic** — registered at load, dispatched at runtime | -| Language | **in-language** — Ludic, compiled together | **cross-language** — JS/TS/Lua/native over an ABI | - -The instinct would be to build a second, parallel system. **The design that keeps -Ludic's discipline builds one system seen from two sides.** A lifecycle hook is a -*private* view of a moment; a public event is the *same moment* exposed across the -ABI. The author promotes a hook to an event; the compiler keeps its zero-cost -direct calls **and** emits one guarded `bus_emit` at the very same site. Nothing -exposed → nothing emitted → goldens stay byte-identical, exactly like `has_ecs` -and the `g_ondespawn` shutdown walk. - -**The Luanti dividend.** The gap analysis (`LUANTI-ROADMAP.md`) found that ~57k of -Luanti's lines exist only to bridge C++ and Lua, and that its mod predicates are -*runtime strings* it must re-interpret every call. Ludic pays neither tax. The -reflection surface a mod needs — "what properties exist, what fields, at what -offsets" — is a **compile-time fact**; the compiler can *generate* the bridge -instead of a human hand-writing 57k lines, and it is always in sync with the game -it describes. A mod itself written in Ludic and compiled to a shared library binds -that surface with **zero marshalling**; a Lua mod binds the same surface through -its FFI. One ABI, every language. - ---- - -## 2. What Ludic has today, and why it can't reach a mod - -The lifecycle table from [LIFECYCLE-DESIGN.md §2](LIFECYCLE-DESIGN.md), every cell -filled, every cell a zero-cost desugar: - -| Scope | Setup hook | Teardown hook | Fire site the compiler already owns | -|---|---|---|---| -| program | `@OnStart` | `@OnQuit` | boot / shutdown | -| entity | `@OnSpawn(M)` | `@OnDespawn(M, reason)` | `spawn` / `despawn` / shutdown-walk | -| property (structural) | `@OnAttach(P)` | `@OnDetach(P)` | `attach` / `detach` | -| property (toggle) | `@OnEnable(P)` | `@OnDisable(P)` | `enable` / `disable` | -| scene | `on enter` | `on exit` | `become` (and `push`/`pop`, SCENES E3) | - -Two more fire sites are proposed but unbuilt, and both are natural events: -`@OnChange(P)` (LC2 — a value-change hook the compiler can emit right after every -write site) and `@OnStartMatch`/`@OnStopMatch` (LC3 — query-membership edges). - -Every one of these is a place the compiler **already writes a call**. The problem -is purely that the call is *closed*: its targets are fixed at compile time, so a -mod loaded at runtime has no way to be one of them. The entire job of this doc is -to add, at each of these sites, an **opt-in second exit** to an open runtime list — -without touching the closed path's cost when no one opts in. - -What a mod additionally needs, that no hook provides: - -- a **stable name** for each event that survives recompilation (a mod compiled - against v1 must still bind in v1.1); -- a way to **read and write game state** it did not compile against (the world - table, §9); -- a way to **veto or rewrite** an action before it commits, not just observe it - after (cancellable events, §8); -- a **loader** — mods enable, disable, and unload, and their listeners must vanish - cleanly when they do (§10). - ---- - -## 3. Research digest — the one idea to steal from each - -The lifecycle doc surveyed engines for *internal* lifecycle. This surveys systems -for their *modding and event* surface — how untrusted, separately-authored code -plugs into a running game. - -| System | The transferable idea | -|---|---| -| **Bukkit / Spigot** (Minecraft) | The canonical **cancellable event**: `Cancellable.setCancelled(true)` vetoes the action; `EventPriority` orders listeners; `@EventHandler(ignoreCancelled=true)` opts out of already-vetoed events. Events are *classes*, checked at bind time — not strings. | -| **Fabric** (Minecraft) | `Event` backed by an **invoker over a plain array** of callbacks — deterministic registration order, no reflection at dispatch, phases for ordering. The closest existing design to what Ludic wants: fast, ordered, array-backed. | -| **Factorio** | **Deterministic** modded events for multiplayer lockstep: `script.on_event(defines.events.X)`, numeric event ids, `raise_event` for custom events, **filtered** subscriptions. Proof that a heavily-modded game can still replay bit-for-bit. | -| **Minetest / Luanti** | `register_on_*` + a string-keyed global (`minetest.*`) world API. The thing to beat: its predicates are runtime strings, and its C++↔Lua bridge is 57k hand-written lines. | -| **Godot** | **Signals as a first-class language construct**: `signal hurt(amount)`, `emit_signal`, `connect`. Decoupled, per-object, declared where the data lives. | -| **DOM events** | The **two-phase dispatch** vocabulary: capture → target → bubble, `preventDefault` (veto the default action) vs `stopPropagation` (halt the chain), and *passive* listeners that promise not to cancel (so dispatch can skip the veto check). | -| **Node `EventEmitter`** | The dead-simple baseline `on`/`emit` — and its footguns: untyped string names (a typo silently never fires) and **listener leaks** (a listener on a dead object keeps it alive). Design both out. | -| **flecs / Bevy observers** | **ECS-native reactive events**: an event *targeted at an entity*, observers that fire on component add/set/remove, deferred so mutation-during-iteration is safe. The correct shape for an ECS. | -| **Blender `bpy.app.handlers`** | Named application-level handler lists a script appends to, with a `persistent` flag controlling survival across file loads — the "engine lifecycle exposed to scripts" model, and the lesson that *survival scope* must be explicit. | -| **Roblox** | `BindableEvent` (local) vs `RemoteEvent` (across the network boundary) — the same event abstraction, one flag deciding whether it crosses a trust/latency boundary. Relevant the day Ludic has networking. | - -Five **footguns** the survey warns against, to design *out* of Ludic from the start: - -1. **Untyped string events.** Node/DOM let any string be an event; a typo never - fires and never errors. Ludic's core events are compiler-checked symbols; only - genuinely-dynamic *mod-defined* events use interned strings, and those must be - *registered* before use (§6), so an unknown name is a load-time error, not a - silent no-op. -2. **Listener leaks.** A listener bound to an entity that despawns must die with - it. Ludic ties listener lifetime to the scope it names (§10) — entity-scoped - listeners are swept by the same despawn walk that already runs. -3. **Nondeterministic dispatch order.** Hash-map iteration over listeners breaks - replay and save-load. Ludic dispatches in a **defined order** (priority, then - registration order) so a modded game stays deterministic — a hard constraint, - not a nicety, given Ludic's deterministic-by-design rng and byte-identical - goldens. -4. **Re-entrancy / mutate-during-dispatch.** A listener that emits another event, - or despawns the entity mid-dispatch, is the flecs "command during iteration" - hazard. Ludic defers structural changes made inside dispatch to the next sync - point (ties to LIFECYCLE LC5), and bounds re-entrant emit depth. -5. **Cancellation ambiguity.** If two listeners disagree, who wins? Ludic's rule - (§8): **one veto wins and is sticky**; later listeners see the cancelled state - and, unless they opted into `ignoreCancelled`, are skipped. - ---- - -## 4. The two layers, named - -To talk about this precisely the doc fixes two words: - -- A **hook** is the existing compile-time construct: an `@`-annotation or scene - clause that desugars to a direct call. Closed, zero-cost, author-only. Unchanged. -- An **event** is the new runtime construct: a named, ABI-visible moment that any - registered listener — in any language — may observe or (if cancellable) veto. - -An event is *fed by* a hook site. Promoting is additive: the hook keeps firing its -compile-time listeners as direct calls; the event is an extra, guarded emission at -the same site. **Author code never pays for the bus it doesn't expose, and mod -code never sees a hook it wasn't given.** - ---- - -## 5. EV0 — the event bus core - -The minimum viable layer: declare an event, emit it, and have both in-language and -foreign listeners receive it — with zero cost when a program declares no events. - -**Declaring a custom event.** A first-class declaration, mirroring `property`: - -```ludic -# doc-check: skip — sketch -event PlayerHurt { entity: int, amount: int } # a payload is a flat POD record -event WaveCleared { } # payloads may be empty -``` - -**Emitting.** A statement, mirroring `spawn`/`emit_signal`: - -```ludic -# doc-check: skip — sketch -emit PlayerHurt(entity: e, amount: dmg) -``` - -**Listening in-language** (author code, or a *native* Ludic mod) reuses the -annotation channel, mirroring `@OnSpawn`: - -```ludic -# doc-check: skip — sketch -@On(PlayerHurt) handler FlashRed { hud_flash(0xFF0000) } -``` - -**Listening across the ABI** (a JS/TS/Lua mod) goes through the stable C ABI: - -```c -/* the entire foreign-facing event ABI — four functions */ -uint32_t ludic_event_id(const char *name); /* intern → stable id */ -uint32_t ludic_on(uint32_t event, int32_t prio, ludic_cb cb, void *ctx); -void ludic_off(uint32_t token); -void ludic_emit(uint32_t event, void *payload); /* mod-raised events */ -/* cb: void (*)(void *ctx, void *payload) — payload is the flat POD record */ -``` - -**Lowering — the discipline holds.** An exposed event's emit site becomes: - -``` -; emit PlayerHurt(entity: e, amount: dmg) lowers to: - 1. build the payload record on the stack (POD, no heap) - 2. call each compile-time @On(PlayerHurt) handler directly ; zero-cost path - 3. if g_listeners[EV_PlayerHurt].count != 0: ; one branch - loop the runtime listener list, calling each cb(ctx, &payload) -``` - -- **A program that declares no `event` emits none of this.** A `has_events` flag - (exactly like `has_ecs`, `g_ondespawn`) gates the whole subsystem; a game with no - public events is byte-for-byte identical to today. This is the non-negotiable - invariant every phase preserves. -- The compile-time `@On` handlers are direct calls appended to the site — a native - listener costs the same as a lifecycle hook. Only *foreign* listeners walk the - runtime list, and an event with zero foreign listeners is a single count check. -- The runtime list is a **compiler-owned, fixed-capacity buffer** per event - (like the scene stack in SCENES E3) — not heap, not a hash map. `ludic_on` is an - index bump; `ludic_off` tombstones a slot. Deterministic order falls out of the - array (§7 of SCENES' "no dispatch tables" spirit, honestly bent — see §11). - ---- - -## 6. EV1 — promoting hooks to events (the taxonomy) - -Custom `event`s (EV0) cover author-raised signals. The **lifecycle** events — -spawn, despawn, attach, scene enter — should not require the author to hand-write -an `emit` in every `@OnSpawn`. Instead, a hook is promoted with one annotation: - -```ludic -# doc-check: skip — sketch -@Public @OnSpawn(Enemy) handler Init { Health.hp = Health.max } -# now firing this hook ALSO emits the public event model.Enemy.spawn -``` - -`@Public` on a lifecycle hook tells the compiler to add the guarded `bus_emit` at -that hook's existing site, with a **generated payload** built from what the hook -already binds (the entity id, the model/property fields, the `EndReason`). The -result is a uniform event namespace across the whole architecture — precisely the -"events for properties, models, scenes, layers, game" the request asks for: - -| Scope | Public event name | Payload | Fed by | -|---|---|---|---| -| program | `program.start` / `program.quit` | `{}` | `@OnStart` / `@OnQuit` | -| phase | `phase..pre` / `.post` | `{ frame }` | the phase scheduler | -| model | `model..spawn` / `.despawn` | `{ entity, reason? }` | `@OnSpawn` / `@OnDespawn` | -| property (structural) | `prop.

.attach` / `.detach` | `{ entity, }` | `@OnAttach` / `@OnDetach` | -| property (toggle) | `prop.

.enable` / `.disable` | `{ entity }` | `@OnEnable` / `@OnDisable` | -| property (value) | `prop.

.change` | `{ entity, field, old, new }` | `@OnChange` (LC2) | -| query (membership) | `query..enter` / `.exit` | `{ entity }` | `@OnStartMatch`/`@OnStopMatch` (LC3) | -| scene | `scene..enter` / `.exit` / `.push` / `.pop` | `{}` | `on enter`/`on exit`, `push`/`pop` | -| layer | `layer..show` / `.hide` | `{}` | layer toggle (SCENES E2) | - -- **Names are stable strings, ids are fast integers.** `model.Enemy.spawn` is the - public contract; the compiler assigns it a numeric id and registers the mapping - in a generated init. A mod compiled against the string binds by id at load — so - reordering declarations doesn't break a shipped mod (unlike raw - decl-order numbering, which is fine for the *closed* scene machine but wrong for - an *open* ABI). -- **Opt-in per hook, not global.** Only `@Public` hooks emit. A game exposes the - slice of its lifecycle it wants moddable and pays for nothing else. -- **`@Public` composes with everything.** A `@Public @OnDespawn(Enemy, reason: r)` - emits `model.Enemy.despawn` with the `EndReason` in the payload — mods can tell a - scene-exit death from a real one, the LC1 dividend extended to the mod boundary. - ---- - -## 7. EV2 — the world table (reflection for mods) - -The user's "game table": the stable, versioned surface a mod uses to **read and -write game state it never compiled against**. Minetest's `minetest.*`, Factorio's -`game.*`, but *generated* rather than hand-written. - -Because Ludic's data is packed POD in `@S_` arrays whose layout the compiler knows -exactly, the compiler can emit a **schema** (property id → field ids → offset + -type) plus a small accessor ABI over it: - -```c -/* the world table — reflection + mutation over the live ECS */ -uint32_t ludic_prop_id(const char *name); /* "Health" → id */ -uint32_t ludic_field_id(uint32_t prop, const char *name); /* ("Health","hp")→id */ -int64_t ludic_get(int32_t entity, uint32_t prop, uint32_t field); -void ludic_set(int32_t entity, uint32_t prop, uint32_t field, int64_t v); -bool ludic_has(int32_t entity, uint32_t prop); -int32_t ludic_spawn(uint32_t model); /* → entity */ -void ludic_despawn(int32_t entity); -uint32_t ludic_query(uint32_t *props, int n); /* → iterator handle */ -int32_t ludic_query_next(uint32_t iter); /* → entity or -1 */ -``` - -- **Generated from the compile-time schema, so it never drifts.** Add a field to - `Health`, recompile, and the schema updates; a mod that asked for - `("Health","hp")` still resolves. This is the entire Luanti bridge, minus the - hand-written 57k lines and minus the runtime-string re-interpretation. -- **`ludic_set` respects the semantic layer.** Writing a field routes through the - same path a native write does, so `@OnChange`/`prop.change` (LC2) fires for a - mod's write exactly as for the author's — mods can't silently corrupt invariants - that hooks are meant to maintain. -- **Mods can register content, within limits.** A mod may `ludic_on` existing - events and `ludic_emit` custom ones; **defining a new `property`/`model` is a - harder call** (it needs storage the closed `@S_` arrays didn't reserve). The - pragmatic first cut: models and properties are closed (author-defined), and mods - extend *behavior* (listeners, custom events, world reads/writes) but not the - *schema*. Opening the schema to mods is EV-late (§13, open decision 4). - ---- - -## 8. EV3 — cancellable and mutable events - -Observation alone (Node, Blender) can't stop a mod from turning damage off — the -modding headline is that a listener runs **before** the action and can veto or -rewrite it. Events split into two kinds, distinguished at declaration: - -- **notifications** — fired *after* the fact, observe-only, can't change anything. - Cheap, un-ordered-safe, the default. `model.Enemy.spawn` after the spawn. -- **decisions** — fired *before* the action, listeners may **cancel** it or - **mutate** the payload; the caller reads the verdict and branches. Marked - `cancellable` (Bukkit `Cancellable`, DOM `preventDefault`). - -```ludic -# doc-check: skip — sketch -event cancellable BeforeHurt { entity: int, amount: int } # a decision event - -# an author (or native mod) listener that halves fire damage and vetoes lethal hits: -@On(BeforeHurt, prio: 100) handler Armor { - BeforeHurt.amount = BeforeHurt.amount / 2 # mutate the payload… - if BeforeHurt.amount >= Health.hp { cancel } # …or veto the whole action -} - -# the fire site consults the verdict: -let dmg = emit? BeforeHurt(entity: e, amount: raw) # emit? returns the (maybe-mutated) payload -if !cancelled(dmg) { Health.hp -= dmg.amount } -``` - -Rules, chosen from the survey to remove the ambiguity footgun: - -- **Priority, then registration order.** `prio:` (default 0) orders listeners - high-to-low; ties break by registration order. Deterministic, replay-safe. -- **One veto wins and is sticky.** Once a listener calls `cancel`, the event is - cancelled for the rest of the chain; later listeners still run (so they can react - to the cancellation) unless declared `ignoreCancelled`, which skips them. -- **`stopPropagation` is separate from `cancel`.** DOM's distinction: `cancel` - vetoes the *action*, `halt` stops the *chain*. Keep both; they answer different - questions. -- **Passive listeners.** A listener declared `@On(E, passive)` promises not to - cancel or mutate — the dispatcher can call it after the decision is settled, and - a foreign listener that lies is a load-time capability error (§10), not a - mid-frame surprise. -- **Mutation is bounded to the payload.** A decision listener rewrites *the payload - record*, never arbitrary world state, so the caller's branch is the only place - the change takes effect — no spooky action at a distance. - ---- - -## 9. EV4 — the mod ABI & the language-agnostic bridge - -"Agnostic JS/TS/Lua or their own" resolves cleanly once EV0–EV3 exist, because the -contract is **the C ABI, not any one language.** Two mod tiers bind the *same* four -event functions (§5) and the same world table (§7): - -**Tier 1 — native mods (Ludic → shared library).** A mod is a `.ludic` file -compiled to a `.dylib`/`.so`/`.wasm` with `extern fn` bindings -([LANGUAGE.md §Functions & FFI](LANGUAGE.md)). It binds the ABI with **zero -marshalling** — payloads are the same POD records the host builds — and its `@On` -handlers can even be *inlined by the same compiler* if the mod is compiled with the -game. This is the tier Luanti can't offer and the one that makes Ludic's modding -fast: a compiled predicate where Luanti has a re-interpreted string. - -**Tier 2 — scripted mods (JS/TS/Lua/…).** The game embeds a scripting runtime -(QuickJS, Lua, Wasm) and registers a thin per-language shim that: - -1. calls `ludic_event_id("model.Enemy.spawn")` once at load to resolve the id; -2. calls `ludic_on(id, prio, trampoline, script_fn)` where `trampoline` is a - single C function that marshals the POD payload into the script runtime's values - and invokes `script_fn`; -3. exposes the world table (§7) as idiomatic bindings (`world.get(e, "Health", - "hp")` in Lua, `world.get(e, "Health", "hp")` in TS). - -The host writes **one trampoline per language**, not one per event — the schema -(§7) drives the marshalling generically. A Lua mod and a TS mod differ only in -their shim; the game core is identical. This is the structural win the Luanti gap -analysis pointed at: the bridge cost is *O(languages)*, not *O(events × languages)* -hand-written, because the schema is generated. - -``` - ┌─────────────── the stable C ABI ───────────────┐ - Ludic game core ──────┤ ludic_on / ludic_emit / ludic_get / ludic_set ├────── generated schema - (emits at hook sites) └────────────────────┬───────────────────────────┘ (prop→field→offset) - │ - ┌────────────────────────────────┼────────────────────────────────┐ - │ │ │ - Tier 1: native mod Tier 2: Lua shim Tier 2: JS/TS shim - (.dylib, zero marshalling) (one trampoline) (one trampoline) -``` - ---- - -## 10. EV5 — mod lifecycle, scoping & leak-proofing - -A mod is not eternal; it loads, enables, disables, and unloads, and its listeners -must vanish with it — the Node listener-leak footgun, solved structurally. - -- **Every registration returns a token** (`ludic_on → token`), and a mod's tokens - are tracked under its **mod handle**. Unloading a mod calls `ludic_off` on all of - them at once — a mod can't leak a listener past its own life. -- **Listeners may be scoped to a game object.** `ludic_on_entity(entity, …)` binds - a listener that the **existing despawn walk** sweeps when that entity dies — the - same `@L_despawn_all` loop LC1 already emits, extended to drop entity-scoped - listeners. An entity-scoped listener on a dead entity is impossible by - construction, not by discipline. -- **Scene-scoped listeners** ride SCENES E1: a listener registered while a scene is - active is dropped by that scene's synthesized `on exit`, alongside its owned - entities. Overlay push/pop (SCENES E3) scopes listeners to the overlay's life. -- **Survival is explicit** (Blender's `persistent` lesson): a listener is - program-, mod-, scene-, or entity-scoped, chosen at registration. There is no - implicit "lives forever" — the default is the narrowest scope that makes sense - (mod), and wider survival is opt-in and visible. -- **Capabilities gate what a scripted mod may touch** (§14, open decision 6). A mod - manifest declares the events and world-table properties it needs; the loader - grants ids only for those. A mod that never asked for `Health` cannot `ludic_set` - it — an untrusted-code boundary the closed lifecycle layer never needed but an - open mod ABI must have. - ---- - -## 11. EV6 — determinism, re-entrancy & the one honest compromise - -Ludic is deterministic by design — deterministic rng, byte-identical PPM goldens, -save-load of the whole World. A modding layer is the classic place that determinism -goes to die (hash-ordered listeners, mods reading wall-clock, emit storms). Holding -the line is a **feature**, and the same one that makes Factorio's modded multiplayer -lockstep-correct. - -- **Dispatch order is total and defined** — priority, then registration order, over - an *array*, never a hash map. Two mods loaded in the same order dispatch in the - same order on every machine. -- **Emit is synchronous by default, deferred on demand.** `emit E` runs listeners - now (push-at-the-site, Ludic's natural style — the LIFECYCLE footgun-1 fix). - Structural changes a listener requests (spawn/despawn/attach) **defer to the next - sync point** (LIFECYCLE LC5's `defer`), so mutate-during-dispatch is safe and - batched. Re-entrant `emit` inside a listener is allowed but **depth-bounded** (a - compile-time cap, trap on overflow) so an event cycle can't hang a frame. -- **Foreign listeners are the determinism boundary.** A native (Tier 1) listener is - as deterministic as any handler. A scripted (Tier 2) listener is only as - deterministic as the script — so the sandbox (§10) can **deny nondeterministic - capabilities** (wall-clock, unseeded rng, filesystem) to a mod that must stay in - a deterministic session (multiplayer, replays). Single-player mods can opt out. - -**The one honest compromise.** SCENES-DESIGN's principle is "no dispatch tables — -the active-scene path is a register read and a static branch." The runtime -listener list *is* a dispatch table, walked at runtime. This doc owns that: it is -the **deliberate, opt-in exception**, justified because open-world extension is the -entire point of a mod ABI and cannot be resolved at compile time by definition. -The mitigations keep it honest — it is (a) gated behind `has_events` so unused it -costs nothing, (b) an array not a hash map so it stays deterministic, (c) fed by -compile-time-checked names so the *closed* side stays typed, and (d) reached only -after the zero-cost direct calls to compile-time `@On` handlers. Ludic pays for a -dispatch table exactly when, and only when, a game chooses to be moddable. - ---- - -## 12. Lowering summary - -Everything above reduces to constructs Ludic already has or honestly-scoped -additions to them: - -| Construct | Lowers to | -|---|---| -| `event E { … }` | a generated payload record type + a reserved event id + a `has_events` bump | -| `emit E(…)` | build POD payload · direct-call each `@On(E)` handler · `if count: walk runtime list` | -| `@On(E)` handler | a compile-time listener: a direct call appended to `E`'s emit site (zero-cost) | -| `@Public @OnX(…)` | the existing hook's site, plus a guarded `bus_emit` of a payload built from the hook's bindings | -| public event name | a stable string interned to an integer id in a generated registry init | -| the runtime listener list | a compiler-owned fixed-capacity array per event; `ludic_on` = index bump, `ludic_off` = tombstone | -| the world table | a generated schema (prop→field→offset/type) + accessor ABI over the live `@S_` arrays | -| `cancellable` / `cancel` | a verdict field on the payload; the emit site branches on it | -| entity/scene-scoped listener | dropped by the existing despawn walk / synthesized `on exit` (LC1 / SCENES E1) | -| deferred structural change in a listener | LIFECYCLE LC5's `defer` queue, flushed at the sync point | - -No heap for native payloads, no hash map, no per-event hand-written bridge. The -active game path is unchanged unless it opts in; the opt-in cost is one branch per -exposed event plus the listeners a mod actually registers. - ---- - -## 13. Design principles distilled - -1. **One system, two sides.** A public event is a lifecycle hook seen from across - the ABI. Don't build a parallel event runtime; promote the sites you already - have. -2. **Opt-in or invisible.** No `event`, no `@Public` → byte-identical goldens. - `has_events` gates the world the way `has_ecs` gates the ECS. -3. **Closed stays typed; only the open edge is dynamic.** Core events are - compiler-checked symbols; string names exist only at the genuinely-runtime mod - boundary, and even there must be registered (no silent typos). -4. **Generated bridge, never hand-written.** The world table and payload marshalling - come from the compile-time schema, so they never drift and cost O(languages), - not O(events × languages). This is the Luanti dividend — spend it. -5. **Deterministic dispatch is a feature.** Array order, not hash order; deny - nondeterministic capabilities to mods in deterministic sessions. Modded replay - and modded multiplayer depend on it. -6. **Lifetime follows scope, explicitly.** Every listener names its scope - (program/mod/scene/entity); the existing teardown walks sweep it. No implicit - immortality, no leaks. -7. **One ABI, every language.** The C ABI is the contract. Native mods bind it with - zero marshalling; scripted mods bind it through one trampoline per language. - Ludic never blesses a single scripting language. -8. **Only the semantic layer, still.** Mods observe and decide; they do not get - ctor/dtor/move hooks Ludic doesn't have. POD in, POD out. - ---- - -## 14. Suggested implementation order - -Each phase is independently shippable and testable, matching how the repo phases -work (and how LIFECYCLE/SCENES sequence). - -- **EV0 — the bus core.** ✅ *Compile-time half shipped.* `event` / `emit` / `@On` - with the `g_events`-gated zero-cost lowering: an event compiles to a `@ev_` - function whose body is its listeners in declaration order (payload bound by - name as params), and `emit E(…)` is a direct call. Verified byte-identical for - event-free programs, self-hosted to the C-free fixpoint. Still open in EV0: the - foreign C ABI (`ludic_on`/`ludic_emit`) and its runtime listener array, so a - mod in another language can join the same dispatch. Implementation notes: AST - `N_EVENT`/`S_EMIT`; `parse_event` + `@On` annotation + `emit` statement (guarded - by an identifier-lookahead so a bare `emit(...)` call still parses); registries - `g_events`/`g_onlisten` (emit_core); `emit_event_fns` (emit_game); `emit_emit` - (emit_stmt). [`examples/events/events.ludic`](examples/events/events.ludic) is a `bin/x test` check. -- **EV1 — `@Public` hook promotion.** ✅ *All scopes shipped.* `@Public` on a - lifecycle hook fires a public event at that hook's site (payload: entity, plus - `EndReason` for despawn); `find_event(name)` doubles as the "is this hook - public?" gate. Covered: program (`@OnStart`/`@OnQuit` → `program_start`/`_quit`), - models (`@OnSpawn`/`@OnDespawn`), properties - (`@OnAttach`/`@OnDetach`/`@OnEnable`/`@OnDisable` → `prop_

_…`). Scenes and - layers use a `public` block modifier instead of an annotation: - `scene__enter`/`_exit` at the synthesized scene functions, and - `layer__show`/`_hide` at the layer-toggle site. Building layer events also - delivered **SCENES E2 layer toggle**: `enable/disable layer L` flips an `@LE_` - flag that gates that layer's handlers, emitted only for toggled layers so - untouched scene programs stay byte-identical. -- **EV2 / EV2b — the world table.** ✅ *Read/write/scan/identify/create shipped.* - The generated reflection ABI (§7), dispatching a runtime prop/model id to the - right `@S_`/`@H_`/`@L_kind` storage: read/write (`prop_id`/`field_id`/`get`/`set`/ - `has`), scan/identify (`entity_count`/`kind`/`model_id`), and create - (`spawn(model_id)`, which reuses the compiler's own spawn lowering — defaults, - `@OnSpawn`, and the spawn event). `get`/`set` address each field by its real - struct offset (constant struct GEP), correct for `int`/`fixed`/`byte`/`ptr` - fields and mixed layouts alike. Emitted only for an ECS program that declares - events (gated on `has_ecs() && g_events`), so event-free games are byte-identical. - A `ludic_query_next(prop, from)` cursor iterates live entities that have a - property. The world table is complete: read, write, scan, identify, create, - iterate. -- **EV3 — cancellable events.** ✅ *Shipped.* `event cancellable E`, the `cancel` - verb, and `emit E(…)` as an expression yielding the veto flag; the flag is a - trailing field of `%Ev_`, so a foreign listener vetoes by setting it. Priority - ordering and `ignoreCancelled`/`halt` (§8) remain open. The modding headline — - observation becomes control. -- **EV4 — the scripting bridge.** One reference shim (Lua *or* QuickJS) over the - ABI, proving the O(languages) claim end to end. -- **EV5 — mod lifecycle & scoping.** ✅ *Shipped.* A parallel owner array `@evO_` - (-1 = program-scoped, ≥0 = owning entity); `ludic_on_` and - `ludic_on_entity_` register with the right owner; `ludic_off_(token)` - tombstones a slot to null and dispatch skips null slots; `ludic_sweep_entity`, - called from `emit_despawn` when the program has events, nulls every listener a - despawning entity owned. Scene-scoped listeners (drop on `on exit`) remain the - same shape applied at the scene teardown — a follow-on. -- **EV6 — determinism & re-entrancy.** ✅ *Depth bound shipped.* `@ev_depth` - increments on each `@ev_` entry and decrements on exit; past `EV_DEPTH_CAP` - (32) a dispatch returns immediately (a cancellable event returns "not - cancelled"), so an event cycle can't hang. Dispatch order was already - deterministic (array, registration order). Still design: deferred structural - changes at a sync point (LC5) and capability gating for deterministic sessions. -- **EV7 — schema-opening & networking.** ✅ *Schema-opening shipped.* A mod defines - a new component at runtime: `ludic_register_prop(name, nfields)` allocates flat - `[MAX_ENT × nfields × i32]` storage + a has-flag array (capacity 32 dynamic - components) and returns a prop id past the compile-time range; - `ludic_attach_dyn`/`ludic_detach_dyn` toggle presence; `get`/`set`/`has`/`prop_id` - fall through to the dynamic registry for a prop id ≥ the compile-time component - count. Per-entity storage is isolated (`world_dyn.c`). This is the first genuinely - *dynamic* `@S_` storage — a deliberate departure from the closed dense arrays, so - it lives entirely behind the ABI (the game's own components stay static and - byte-identical). Dynamic components use integer fields addressed by index (no - field-name schema). *Still design:* the local/remote event split (Roblox's - lesson) waits on Ludic having a networking substrate. - -EV0–EV1 deliver "the whole architecture emits public events." EV2–EV3 are where a -mod becomes able to *change the game*. EV4 proves the language-agnostic claim. -EV5–EV7 are hardening and reach. - ---- - -## 15. Open decisions - -1. **`emit` verb & payload identity.** Is `emit E(…)` the only spelling, or does a - `signal`-style per-property declaration (Godot) read better for the common case? - Are payloads always fresh POD records, or can an emit borrow an existing property - in place (cheaper, but aliases live storage)? -2. **`@Public` granularity.** Per-hook (proposed), per-model (`@Public model - Enemy`), or a program-level "expose all lifecycle" switch for prototyping? Does - `@Public` belong on the hook or on the `model`/`property`/`scene` it concerns? -3. **Name scheme stability.** Dotted strings (`model.Enemy.spawn`) interned to ids — - confirmed. Open: are ids stable across recompiles of the *same* source (needed - for save-compatibility of a listener table), and how does a renamed model - migrate a shipped mod? -4. **Schema opening (EV2/EV7).** Do mods stay behavior-only (listeners + custom - events + world reads/writes over author-defined schema), or can a mod define new - `property`/`model`? The latter needs dynamic `@S_` storage — a real departure - from the closed dense arrays (`LUDIC_MAX_ENT 1024`). Probably EV7. -5. **Cancellation surface.** Keep `cancel` (veto action) and `halt` (stop chain) - distinct (DOM), or collapse to one? Is `ignoreCancelled` per-listener or a - priority-band convention? -6. **Sandbox model.** Capability manifest per mod (proposed) — at what granularity - (per event? per property? per world-table verb)? What is denied by default in a - deterministic session, and who declares a session deterministic? -7. **Re-entrancy bound.** Compile-time constant emit-depth cap (trap on overflow), - or a runtime budget? What is the default depth, and is an event cycle a warning - or an error? -8. **Scripting runtime, in or out of scope.** Does Ludic *ship* an embedded runtime - (QuickJS/Lua) as a blessed default, or only the ABI and reference shims, leaving - the runtime to the game? (Bias: ship the ABI + one reference shim; bless no - language.) - ---- - -*Companion to [LIFECYCLE-DESIGN.md](LIFECYCLE-DESIGN.md) (the hook sites this layer -promotes) and [SCENES-DESIGN.md](SCENES-DESIGN.md) (scene/layer/overlay events and -scoped-listener teardown). Grounded in the Luanti gap analysis (`LUANTI-ROADMAP.md`): -the generated bridge is how Ludic avoids the 57k-line C++↔Lua tax. Supersedes -nothing until the compiler work in §12 lands.* diff --git a/LANGUAGE.md b/LANGUAGE.md index 56838960..706a0bbe 100644 --- a/LANGUAGE.md +++ b/LANGUAGE.md @@ -396,7 +396,7 @@ scopes, [`examples/lang/detach.ludic`](examples/lang/detach.ludic) for the struc attach/detach pair, and [`examples/lang/reason.ludic`](examples/lang/reason.ludic) for reason-carrying teardown. The rest of the lifecycle roadmap (value-change hooks, query-membership edges, keyed effects) is in -[LIFECYCLE-DESIGN.md](LIFECYCLE-DESIGN.md). +[the Lifecycle design](https://git.workshopsoft.io/workshopsoft/ludic/wiki/Design/Lifecycle). **`@Handles` — the handlers a program drives.** Written in front of the `program`, `@Handles(Move)` names the handlers it uses. It parses and reads as @@ -408,7 +408,7 @@ whole timeline), plus [`examples/lang/toggle.ludic`](examples/lang/toggle.ludic) (enable/disable). Scenes and their `on enter` / `on exit` lifecycle blocks are implemented — see "Scenes & layers" below. (An annotation spelling, `@OnEnter(Scene)` / `@OnExit(Scene)`, is a designed but not-yet-built convenience -— see [SCENES-DESIGN.md](SCENES-DESIGN.md); today the hooks are written as `on +— see [the Scenes design](https://git.workshopsoft.io/workshopsoft/ludic/wiki/Design/Scenes); today the hooks are written as `on enter { … }` inside the `scene`.) ## Events & modding (`event`, `emit`, `@On`) @@ -469,7 +469,7 @@ if emit BeforeHurt(amount: dmg) == 0 { hp = hp - dmg } # apply only if not vet ``` The full modding roadmap — the world-table reflection ABI, scoped/leak-proof -listeners, and the sandbox — is in [EVENTS-DESIGN.md](EVENTS-DESIGN.md). +listeners, and the sandbox — is in [the Events design](https://git.workshopsoft.io/workshopsoft/ludic/wiki/Design/Events). ## Records (`property`), arrays and slices @@ -821,7 +821,7 @@ are future work. Records (`property` used with `new`) and array types, `break`/`continue`, and argv/stderr — once listed here as near-term — are now implemented and self-hosting; their lowerings are in -[BOOTSTRAP.md](BOOTSTRAP.md) §4. +[the Bootstrap deep-dive](https://git.workshopsoft.io/workshopsoft/ludic/wiki/Bootstrap) §4. ## Scenes & layers @@ -831,7 +831,7 @@ self-hosting; their lowerings are in > implicit active-scene register, states numbered by declaration order, and > `become` as two direct calls plus a store. Richer scene features (the overlay > stack, scene-owned entities, scene-local state, transition parameters) are -> designed in [SCENES-DESIGN.md](SCENES-DESIGN.md) and not built yet. +> designed in [the Scenes design](https://git.workshopsoft.io/workshopsoft/ludic/wiki/Design/Scenes) and not built yet. A program is usually several mutually-exclusive states — a title screen, the overworld, a battle — and the usual way to write that is a mode register diff --git a/LIFECYCLE-DESIGN.md b/LIFECYCLE-DESIGN.md deleted file mode 100644 index ade6610b..00000000 --- a/LIFECYCLE-DESIGN.md +++ /dev/null @@ -1,348 +0,0 @@ -# Lifecycle events, expanded — a design doc - -> **Status: LC0–LC1 shipped; LC2–LC6 are design.** The structural attach/detach -> pair and `@OnDetach` (§4, LC0), and reason-carrying `@OnDespawn` (§5, LC1), are -> implemented and tested ([`examples/lang/detach.ludic`](examples/lang/detach.ludic), -> [`examples/lang/reason.ludic`](examples/lang/reason.ludic), `bin/x test` checks). The -> extensions LC2–LC6 are research-informed proposals, not built. This document -> distills a survey of lifecycle models across seven systems (§3) into a roadmap -> for Ludic. §13 lists the open decisions. - ---- - -## 1. Thesis - -A game/ECS usually models lifetime as **create → destroy on a timeline**. A survey -of how other systems handle it — Unity (MonoBehaviour + DOTS), Unreal, Bevy, -flecs, EnTT, Godot, and non-game paradigms (actor model, declarative UI, RAII) — -shows that mature lifecycle designs model something richer than birth and death: - -- **a reaction to a *reason*** — teardown that knows *why* it is ending (Unreal - `EndPlay(reason)`, Erlang `terminate(Reason)`, Akka `preRestart(reason, msg)`); -- **paired setup/teardown *keyed on dependencies*** — an update is teardown-then- - setup on a value change (React `useEffect`, Compose `DisposableEffect`); -- **a deterministic consequence of *scope / ownership*** — guaranteed, ordered, - single-shot teardown (C++/Rust RAII, DI scoped lifetimes); -- **an edge on *query membership*** — fire when data starts/stops matching a - composite condition (DOTS `OnStartRunning`, flecs `Monitor`). - -Ludic's model is a good base: lifecycle hooks are `@`-annotations on handlers that -**desugar to ordinary code**, firing at fixed timeline moments, keeping the data -plain. This doc extends that base along the four axes above **without breaking the -desugars-to-code discipline** — every proposal lowers to plain branches and calls, -no hidden runtime. - -**One structural advantage worth stating up front.** flecs and EnTT each carry -*two* lifecycle layers: a **memory** layer (ctor/dtor/move/copy — because C++ -objects must be constructed and relocated as archetypes repack) and a **semantic** -layer (on_add/on_set/on_remove). Ludic's components are POD in packed `@S_` -arrays; there is nothing to construct, destruct, or move-relocate. **Ludic needs -only the semantic layer** — half the machinery, none of the "component isn't -movable" footguns. Keep it that way. - ---- - -## 2. What Ludic has today - -Seven hooks, each an annotation that desugars to a handler body at a timeline -moment ([LANGUAGE.md §Annotations](LANGUAGE.md)): - -``` -boot ─ @OnStart ─▶ spawn ─ @OnAttach(P), @OnSpawn(M) ─▶ … ─ @OnDetach(P)/@OnDespawn(M) ─▶ quit ─ @OnQuit -``` - -The lifecycle reads cleanest as a table of **paired setup/teardown** across five -scopes. Every cell is now filled — LC0 closed the one hole (`@OnDetach`): - -| Scope | Setup | Teardown | Driven by | -|---|---|---|---| -| program | `@OnStart` | `@OnQuit` | boot / quit | -| entity | `@OnSpawn(M)` | `@OnDespawn(M)` | `spawn` / `despawn` | -| property (structural) | `@OnAttach(P)` | `@OnDetach(P)` ✅ | `attach` / `detach` | -| property (toggle) | `@OnEnable(P)` | `@OnDisable(P)` | `enable` / `disable` | -| scene | `on enter` | `on exit` | `become` | - -Two things this table already gets right, which the survey flags as the frequent -mistakes to avoid: - -- **The toggle pair is distinct from the structural pair.** Unity's clearest - lesson is separating the *repeatable* enable/disable cycle (pooling, pausing, - data kept) from the *once* create/destroy (data gone). Ludic has both, as - distinct verbs: `disable` pauses and keeps data; `detach` structurally removes - (a later `attach` re-seeds). This is exactly DOTS enableable-components vs - structural add/remove, and Bevy `disabled` vs `Remove`. -- **Hooks are typed annotations, not magic-named methods.** MonoBehaviour matches - `Awake`/`Update` by *string name* via reflection — a typo silently never runs. - Ludic's `@OnSpawn(Enemy)` is a checked reference; a wrong name is a compile - error. Preserve this. - -What's missing is everything past "what happened": **why** it happened, **which -values changed**, **when composite conditions begin/end to hold**, and -**dependency-keyed** setup/teardown. That is the roadmap. - ---- - -## 3. Research digest — the one idea to steal from each - -| System | The transferable idea | -|---|---| -| **Unity MonoBehaviour** | Two-phase init with a global barrier (all `Awake` before any `Start`); repeatable enable-pair vs once create-pair. | -| **Unity DOTS** | *Data-driven activation*: `RequireForUpdate` + `OnStartRunning`/`OnStopRunning` — a system edge-triggers when its query starts/stops matching. Enableable components = cheap "logically off." | -| **Unreal** | *Reason-carrying teardown*: `EndPlay(EEndPlayReason)` — one teardown, branch on `Destroyed`/`LevelTransition`/`Quit`/…; forces enumerating every death path (no silent deaths). Provenance-tagged construction. | -| **Bevy** | Full structural event set Add/Insert/**Replace**/Remove/Despawn with strict order; **Replace exposes the old value before drop**. Hooks (type-level, singular, invariant) vs observers (plural, reactive). Declarative `before`/`after`/`chain` ordering. State `OnEnter`/`OnExit`/`OnTransition`. | -| **flecs** | `Monitor` observers fire on *composite query membership* start/stop. Events fire on **real transitions**, not every API call. Deferred-by-default with explicit sync points. | -| **EnTT** | `patch` as the *explicit mutation channel* that fires `on_update` (solves "raw writes are invisible"). Opt-in signals — zero cost when unused. | -| **Godot** | Tree membership *is* the lifecycle driver; enter top-down, **`_ready` bottom-up** (dependencies initialized first); `queue_free()` deferred safe-delete; `process_mode` pause inherited down the tree. | -| **Actor model (OTP/Akka)** | Lifecycle driven by *failure + supervision*: reason-carrying `terminate`, **restart as a state distinct from create/destroy** (stable identity, reset transient state), supervision trees, `code_change` = live state migration. | -| **Declarative UI (React/SwiftUI/Compose)** | *Paired setup/teardown keyed on a dependency list* — cleanup co-located with setup so it can't leak; an update **is** keyed teardown-then-setup; lifetime follows *identity*. | -| **RAII / Rust `Drop` / DI scopes** | *Scope = lifetime*: deterministic, reverse-construction-order, single-shot, no-resurrection teardown, guaranteed even on early exit; lifetime-mismatch checking (no long-lived thing holding a short-lived handle). | - -Two recurring **footguns** the whole survey warns against, to design *out* of Ludic: - -1. **Silent order-dependent reactivity.** Bevy's removal buffers are cleared at - end-of-frame, so a detector that runs before the mutator *misses removals - entirely*. If Ludic adds change/removal reactivity, make it either push-based - (fire at the mutation site — Ludic's natural style) or loudly order-checked. -2. **Invisible in-place writes.** flecs `on_set` and EnTT `on_update` don't fire - on a raw pointer write — you must call `modified()`/`patch`. Ludic can dodge - this entirely (see LC2): the compiler *sees* every write site. - ---- - -## 4. LC0 — structural attach/detach + `@OnDetach` ✅ *shipped* - -The one missing cell in §2's table. `attach P on e { overrides }` adds a property -to a **live** entity (seeding fields, firing `@OnAttach`); `detach P on e` removes -it (firing `@OnDetach`, which reads the outgoing value, before the has-flag -clears). Both fire only on a **real transition** (flecs/Bevy idempotent-add -semantics): re-attaching a present property or detaching an absent one is a no-op. - -Lowering: `attach` guards on the has-flag and, when absent, reuses the existing -`emit_init_component` (seed + `@OnAttach`); `detach` guards on presence, clears the -flag, and fires `@OnDetach` with the property bound by name — the same binding the -`@OnDisable` path already uses. No new runtime; POD data stays in `@S_` storage. -See [`examples/lang/detach.ludic`](examples/lang/detach.ludic). - ---- - -## 5. LC1 — reason-carrying teardown ✅ *shipped (`@OnDespawn`)* - -The highest-conviction idea in the survey: it appears independently in Unreal -(`EndPlay`), Erlang (`terminate`), and Akka (`preRestart`), and Bevy has an open -issue asking for it. **Teardown should know *why*.** A destructor frequently needs -to branch — save on `Quit` but not on a scene swap, skip network cleanup when the -whole program is exiting. - -`@OnDespawn` gains an optional bound **reason**: - -```ludic -# doc-check: skip -# EndReason { Despawned, SceneExit, Quit } — the compiler owns this enum - -@OnDespawn(Enemy, reason: r) handler Clean { - match r { - EndReason.Quit => {} # app closing — don't bother dropping loot - _ => drop_loot(Health.hp) - } -} -``` - -**What shipped.** The lowering is exactly the cheap desugars-to-code shape the -survey promises. The despawn hook compiles to `@on_despawn_(i32 %e, i32 -%reason)`; when the hook writes `reason: r`, `r` is bound as an int local reading -`%reason`. Each teardown *site* passes a constant `EndReason`: - -- `despawn e` passes `Despawned` (0) — an in-world death. -- **program shutdown** passes `Quit` (2): a generated `@L_despawn_all(reason)` - walks the live set at `done:` (before `@OnQuit`, matching the timeline) and - fires every survivor's `@OnDespawn`. This makes **"no silent deaths"** real — - an entity that outlives the run still gets its destructor, and can branch on - `Quit` to skip work that only matters mid-game. Emitted only when the program - has `@OnDespawn` hooks, so despawn-free programs are byte-for-byte unchanged. -- `SceneExit` (1) is reserved: a scene tearing down its owned entities - (SCENES-DESIGN E1) will pass it once scene-owned entities land. - -`EndReason` is compiler-owned (resolved in `enum_ordinal`), so `EndReason.Quit` -works without a user declaration; a user enum of the same name still shadows it. -Backward-compatible: the `reason:` binding is optional, and `@OnDespawn` without -it is unchanged. `@OnDetach` and scene `on exit` do **not** yet take reasons -(§13.1). See [`examples/lang/reason.ludic`](examples/lang/reason.ludic). - ---- - -## 6. LC2 — value-change hooks `@OnChange(P)` *(a compile-time win)* - -Every reactive ECS wants "fire when a component's value changes" (flecs `on_set`, -EnTT `on_update`, Bevy `Changed`), and every one hits the same footgun: a raw -in-place write is invisible, so you must route mutations through a special channel -(`modified()`, `patch`) or you miss changes. - -**Ludic can sidestep the footgun because it is an AOT compiler that sees every -write site.** A field store `Health.hp = …` is a statement the compiler lowers; if -`Health` carries an `@OnChange`, the compiler can emit the hook call *right after -the store*. No dirty bits, no end-of-frame flush, no missed-write class of bugs — -the thing that is a runtime hazard everywhere else is resolved at compile time. - -```ludic -# doc-check: skip -@OnChange(Health) handler Bar { hud_set_health(Health.hp) } # after any write to a Health field -``` - -Open question (§7): fire on *every* write (Bevy's `DerefMut` semantics — simple, -may over-fire) or guard with a value compare (fire only on actual change — needs -the old value, à la Bevy `Replace`). The compiler has the old value in hand at the -store site, so the value-compare form is feasible and is the more useful default. - ---- - -## 7. LC3 — query-membership edges `@OnStartMatch` / `@OnStopMatch` - -DOTS `OnStartRunning`/`OnStopRunning` and flecs `Monitor` fire when an entity -**starts or stops matching a composite query** — not a single component, but a -whole condition (`{Position, Velocity, moving}`). This is strictly more expressive -than per-property `@OnAttach`, which can't see "the entity now has *both* and is -alive." It's the natural ECS form of enter/exit. - -```ludic -# doc-check: skip -@OnStartMatch(these: [Position, Velocity{dx != 0 or dy != 0}], on: Actor) -handler BeginMoving { play("footstep_loop.wav") } - -@OnStopMatch(these: [Position, Velocity{dx != 0 or dy != 0}], on: Actor) -handler StopMoving { stop("footstep_loop.wav") } -``` - -Cost: unlike LC1/LC2 this needs runtime state — a per-entity shadow bit per -monitored query ("did it match last tick?"), checked once per frame, edge- -triggering the hook on a change. flecs does this by evaluating the query against -the entity's previous and current archetype. Ludic would keep a `@M_` bit -array parallel to `@H_`. Medium cost; a genuinely differentiated feature. - ---- - -## 8. LC4 — keyed effects (paired setup/teardown on a dependency list) - -The declarative-UI headline, and the biggest reach. React `useEffect`, Compose -`DisposableEffect`, and SwiftUI `.task` all express: *while this thing exists (or -while key K holds), set up a resource; when it leaves or K changes, tear it down* -— with cleanup **co-located** with setup so it can't leak, and an *update* defined -as keyed teardown-then-setup. This collapses create/update/destroy into one -primitive. - -```ludic -# doc-check: skip — sketch -@Effect(on: Enemy, keys: [Sprite.id]) handler Body { - let tex = image_load(Sprite.id) - dispose { image_drop(tex) } # runs on despawn OR when Sprite.id changes -} -``` - -Semantics: the setup runs on spawn (and whenever a listed key changes, after the -previous `dispose`), and `dispose` runs on despawn (and before each keyed re-run). -It unifies `@OnAttach`/`@OnDetach`/`@OnChange` into one leak-proof unit. Lowering -needs somewhere to stash the effect's captured teardown state and last key values -per entity — a per-effect side table, re-checked in a phase. Design only; the -syntax and storage model are open. This is where Ludic could feel genuinely modern -relative to every ECS surveyed (none of which have it). - ---- - -## 9. LC5 — deferred structural changes with commit points - -DOTS `EntityCommandBuffer`, flecs `defer_begin/end`, and Godot `queue_free()` all -make structural change **deferred with an explicit commit point**, so mutating -while iterating is safe and batched. Ludic's `spawn`/`despawn` are immediate today, -but *already* iteration-safe by a different route — matching is lazy per entity id -([LANGUAGE.md](LANGUAGE.md) "Matching is lazy, not snapshotted"), so despawning the -current entity is defined. A `defer { … }` block (or `despawn e at LateUpdate`) -that queues structural changes to a phase boundary would add batching and a single -predictable commit point, and is the prerequisite for safe parallel handlers -(the `reads`/`writes` scheduling in SCENES-DESIGN). Design only; lower priority -than LC1–LC3 because the immediate path is already safe. - ---- - -## 10. LC6 — supervision, restart-as-a-state, live migration - -The furthest-out cluster, from the actor model and OTP: lifecycle driven by -**failure**, not just create/destroy. Three ideas, all tied to Ludic's eventual -hot-reload / bytecode-VM roadmap rather than the near term: - -- **Restart as a distinct state** between create and destroy — preserve an - entity's identity, reset its transient components, re-run setup (respawn, - hot-reload). Akka's "stable external ref, replaced internal state." -- **Supervision / failure escalation** — a subsystem owner declares a policy for - child faults (restart one / restart the group / escalate to reload the scene) - instead of defensive inline checks. Ludic has no failure model yet, so this - waits on one. -- **Live state migration** (`code_change`) — a hook that transforms an entity's - persistent state across a code/schema version, so hot-reload evolves data - instead of destroying it. Directly relevant to a self-hosting language. - ---- - -## 11. Design principles distilled from the footguns - -1. **No silent deaths.** Enumerate every teardown reason (LC1). If the compiler - must name the reason at each site, it can't forget a path. -2. **Fire on real transitions, not API calls.** Idempotent add/remove — LC0 - already does this; keep it for every future hook. -3. **Keep "paused" and "gone" distinct.** `disable`/`enable` (data kept) vs - `detach`/`attach` (structural) — already true; don't let a future feature blur - them. -4. **Prefer compile-time resolution to runtime tracking.** LC2 turns the - universal "invisible write" footgun into a compile-time hook emission because - Ludic sees write sites. Reach for this wherever a runtime dirty-bit is the - obvious-but-worse option. -5. **If reactivity is order-dependent, make it loud.** Never silently drop events - at a frame boundary (Bevy's removal-buffer trap). Ludic's push-at-the-site - style avoids this by default. -6. **Deterministic teardown order.** When a scope tears down many things (a scene - unloading its owned entities — SCENES-DESIGN E1), define the order (reverse of - creation, RAII-style) rather than leaving it unspecified. -7. **Only the semantic layer.** POD components mean no ctor/dtor/move hooks. Don't - grow a memory-lifecycle layer Ludic doesn't need. - ---- - -## 12. Suggested implementation order - -- **LC0 — attach/detach + `@OnDetach`.** ✅ Done. Closes the structural pair. -- **LC1 — reason-carrying teardown.** ✅ Done for `@OnDespawn` (an `i32 %reason` - param + a constant at each site, plus a shutdown despawn-all for `Quit`). - `@OnDetach` / `on exit` reasons remain open (§13.1). -- **LC2 — `@OnChange(P)`.** Compile-time hook emission at write sites — a - Ludic-specific win over every ECS's invisible-write footgun. **Recommended next.** -- **LC3 — `@OnStartMatch`/`@OnStopMatch`.** First feature needing runtime shadow - state; the expressive ECS enter/exit. -- **LC4 — keyed effects.** The modern, leak-proof unification. Design first. -- **LC5 — deferred structural changes.** Batching + parallel-safety; the immediate - path is already iteration-safe, so lower urgency. -- **LC6 — supervision / restart / migration.** Waits on a failure model and the - hot-reload roadmap. - ---- - -## 13. Open decisions - -1. **Reason enum (LC1):** *resolved for `@OnDespawn`* — ships `Despawned`, - `SceneExit`, `Quit` as a compiler-owned `EndReason`, passed as an optional - `reason:` binding (not a separate annotation). Still open: `SceneExit` has no - firing site until scene-owned entities (SCENES-DESIGN E1); should `@OnDetach` - and scene `on exit` take reasons too, and if so with which reason values? -2. **`@OnChange` (LC2):** fire on every write (simple, over-fires) or only on an - actual value change (needs the old value at the store site)? Per-field or - whole-property granularity? -3. **Membership edges (LC3):** where do the shadow bits live, and is the check - per-frame or event-driven off attach/detach/spawn? Cost budget. -4. **Keyed effects (LC4):** syntax (`@Effect` annotation vs an `effect { … dispose - { … } }` statement), and where per-entity teardown/key state is stored. -5. **Ordering:** none of this addresses intra-phase handler ordering (Bevy - `before`/`after`, flecs `DependsOn`). Worth a separate proposal; declarative - relational ordering over priority integers, per the survey. - ---- - -*Companion to [LANGUAGE.md §Annotations](LANGUAGE.md) and -[SCENES-DESIGN.md](SCENES-DESIGN.md) (scene-owned entities and reasons intersect at -LC1/LC5). Supersedes nothing until the compiler work in §12 lands.* diff --git a/LUANTI-ROADMAP.md b/LUANTI-ROADMAP.md deleted file mode 100644 index 035e8acb..00000000 --- a/LUANTI-ROADMAP.md +++ /dev/null @@ -1,1560 +0,0 @@ -# Luanti → Ludic: Full Gap Analysis & Implementation Roadmap - -**Question asked:** *what does Ludic miss in order to implement Luanti (formerly -Minetest) — the game/engine?* - -**Short answer:** Ludic can already express Luanti's **game logic**. It cannot yet -express Luanti's **engine**. The gap is not one feature; it is roughly six -layers, and two of them (aggregate data types, and a platform layer with -threads + sockets + a GPU) are load-bearing for everything above them. - -This document is written against evidence, not memory: `luanti-org/luanti` at -`main` was cloned and read, and every claim about Ludic below was verified -against `compiler/ludicc.c` / `compiler/native.c` or by compiling a probe -program with `bin/ludicc`. - ---- - -## 0. TL;DR - -| | Luanti | Ludic today | -|---|---|---| -| Size | **362,258** lines (C++ / Lua / GLSL) | 2,508 lines of compiler + 3,692 of runtime | -| World | infinite 3D voxel, ±31,007 nodes/axis | 96×64 char tilemap, 2D | -| Entities | unbounded active objects, chunked | **fixed 1,024** (`LUDIC_MAX_ENT`) | -| Rendering | GPU, 20 shader programs, Irrlicht fork (87k LOC) | 320×240 software framebuffer | -| Extensibility | Lua sandbox, hot-loaded mods, 321 `core.*` calls | AOT compile only | -| Concurrency | emerge threads, mesh threads, net threads, async Lua | **none** | -| Networking | custom reliable UDP, 90 packet types | **none** | -| Numbers | `f32` / `f64`, `v3f`, matrices | `int`, Q16.16 `fixed` (range **±32,768**) | -| Aggregates | vectors, maps, strings, structs | **none** — raw `ptr` + `peek/poke` | - -**The single most surprising finding:** Ludic's `fixed` type **cannot represent a -Luanti world coordinate.** Q16.16 saturates at ±32,768; Luanti's map limit is -±31,007 *nodes*, and positions are carried in *BS units* where `BS = 10.0` -(`src/constants.h`) — i.e. ±310,070. Worse, any squared distance -(`31007² ≈ 9.6e8`) overflows Q16.16 by five orders of magnitude. Every collision, -raycast and physics computation in Luanti would silently wrap. See **G-04**. - -**Recommended order of attack:** language core (arrays/structs/strings) → -platform (threads/sockets/time) → voxel storage primitive → GPU backend → -mod ABI → networking. Milestones **M0–M4** are all reachable without -compromising any of the project's stated constraints (no C, no VM, no -transpile). - ---- - -## 1. Evidence base - -What was read, and how big it is: - -| Area | Files | LOC | Read | -|---|---|---|---| -| `src/` (engine core) | ~700 | 210,370 | headers + all subsystem entry points | -| `irr/` (Irrlicht fork: renderer, GUI, mesh loaders) | — | 87,554 | structure + driver list | -| `src/client/` | 134 | 44,247 | `game.cpp`, `content_mapblock.cpp`, `clientmap.cpp`, mesh threads | -| `src/script/` (Lua bindings) | 147 | 34,240 | full API index, `l_object`, `l_env`, `l_mapgen`, `l_vmanip` | -| `builtin/` (Lua engine layer) | ~60 | 22,634 | file map, `game/`, `common/`, `emerge/` | -| `src/gui/` (formspec, menus) | 61 | 19,198 | `guiFormSpecMenu.cpp` (5,610 LOC alone) | -| `games/devtest` | — | 13,455 | node/item/entity registration patterns | -| `src/mapgen/` | 32 | 11,668 | all 8 mapgens + biome/ore/decoration/cave/dungeon/schematic | -| `src/network/` | 23 | 10,958 | protocol enum (90 packets), reliable-UDP impl | -| `src/util/`, `src/database/`, `src/threading/`, `src/server/` | 102 | 21,961 | serialization, 5 DB backends, thread primitives | -| `doc/lua_api.md` | 1 | 12,785 | **the full mod-facing contract** — the real spec | -| `doc/world_format.md` | 1 | 658 | on-disk map format v22–29 | - -Ludic side, verified by reading source and by compiling probes: - -| Probe | Result | -|---|---| -| recursion (`fib`) | ✅ compiles and links | -| `"ab" + "cd"` | ❌ front-end accepts, **IR fails to assemble** — no string ops | -| `mem_alloc` / `peek32` / `poke32` | ✅ works | -| `fixed` multiply IR | `sext i64 → mul → ashr 16 → trunc i32` — 64-bit intermediate, **i32 result** | -| `sqrt` / `sin` / `cos` / `atan2` | ❌ absent from compiler and runtime entirely | -| runtime component attach/detach | ❌ no such statement — components are fixed at `spawn` | -| entity ceiling | `#define LUDIC_MAX_ENT 1024` (`compiler/native.c:18`) | -| DEFLATE | **decode only** (`runtime/native/inflate.ludic`) — no compressor | - ---- - -## 2. What Luanti actually is - -Nine subsystems. For each: what it does, where it lives, and — the part that -matters here — **what it demands from the language underneath it.** - -### 2.1 The voxel core - -A world is `MapNode`s: a 4-byte struct — `u16 param0` (content id), `u8 param1` -(two 4-bit light banks, day + night), `u8 param2` (rotation / liquid level / -level / colour index). 4,096 of them per `MapBlock` (16³). Blocks live in -sectors; sectors in a `Map`; `ServerMap` and `ClientMap` specialise it. - -```cpp -// src/mapnode.h -struct alignas(u32) MapNode { - u16 param0; // content_t, up to MAX_REGISTERED_CONTENT (~58k) - u8 param1; // day light : 4 | night light : 4 - u8 param2; // facedir / wallmounted / liquid level / degrotate / colour -}; -static constexpr u32 nodecount = 16*16*16; // src/mapblock.h:426 -``` - -**Demands:** a packed struct type; a fixed-size array of them; bitfield -access; cheap 3D→1D indexing; and the ability to allocate *millions* of these -(a modest 10-chunk view radius is ~9,000 blocks = 36M nodes = 147 MB). - -### 2.2 Map storage & persistence - -`map.sqlite` keyed by a 64-bit interleaved block position; each blob is a -serialised `MapBlock` — version byte, flags, 12-bit `lighting_complete` -bitfield, timestamp, a **name↔id mapping** so node ids survive mod changes, -then `param0`/`param1`/`param2` planes, node metadata, node timers, and static -objects. Since format 29 the whole block is **zstd-compressed** (zlib before -that). Five interchangeable backends: SQLite3, LevelDB, Redis, PostgreSQL, flat -files. - -**Demands:** big-endian byte serialisation, a compressor (not just a -decompressor), a string↔id table, and an FFI-usable key/value store. - -### 2.3 Map generation - -Eight mapgens (`v5 v6 v7 flat fractal valleys carpathian singlenode`) over a -shared `Mapgen` base, plus five orthogonal generators layered on top: - -| Generator | File | LOC | What it does | -|---|---|---|---| -| Caves | `cavegen.cpp` | 912 | 3D-noise caverns + tunnel carving | -| Trees | `treegen.cpp` | 888 | L-system + hardcoded species | -| Dungeons | `dungeongen.cpp` | 658 | room/corridor placement | -| Schematics | `mg_schematic.cpp` | 618 | `.mts` blob stamping, force/probability per node | -| Ores | `mg_ore.cpp` | 583 | scatter / sheet / puff / blob / vein / stratum | -| Decorations | `mg_decoration.cpp` | 472 | simple / schematic / lsystem placement | -| Biomes | `mg_biome.cpp` | 332 | heat+humidity noise → biome, per-node depth layers | - -All of it rests on **fractal value noise**: - -```cpp -// src/noise.h -struct NoiseParams { - float offset = 0.0f, scale = 1.0f; - v3f spread = v3f(250,250,250); - s32 seed = 12345; - u16 octaves = 3; - float persist = 0.6f, lacunarity = 2.0f; - u32 flags = NOISE_FLAG_DEFAULTS; -}; -float NoiseFractal3D(const NoiseParams *np, float x, float y, float z, s32 seed); -``` - -**Demands:** floating-point (or ≥Q32.32 fixed), a noise map buffer type, -and — because mapgen runs on `EmergeThread`s — **real threads plus a -thread-local `MMVManip` voxel buffer.** - -### 2.4 Environment & simulation - -`ServerEnvironment` drives the world tick: **ABMs** (Active Block Modifiers — -probabilistic per-node rules that run in loaded blocks), **LBMs** (Loading Block -Modifiers — run once when a block loads), **node timers**, liquid transformation -queues, and object activation/deactivation as players move. - -```lua -core.register_abm({ - label = "Lava cooling", - nodenames = {"default:lava_source"}, - neighbors = {"default:water_source", "default:water_flowing"}, - interval = 10.0, -- seconds - chance = 50, -- 1-in-50 per node per interval - min_y = -32768, max_y = 32767, - catch_up = true, - action = function(pos, node, active_object_count, active_object_count_wider) ... end, -}) -``` - -**This is the one Luanti concept that maps *beautifully* onto Ludic.** An ABM is -literally a system with a query, a `where` clause and a probability. See G-16 — -this is where Ludic could be *better* than Luanti, not merely equal. - -### 2.5 Objects, entities, players - -`ActiveObject` → `ServerActiveObject` → `LuaEntitySAO` / `PlayerSAO`; mirrored -client-side by `ClientActiveObject` → `GenericCAO` (`content_cao.cpp`, 2,002 -LOC). ~30 shared `ObjectProperties` (visual, mesh, textures, `collisionbox`, -`selectionbox`, `physical`, `stepheight`, `nametag`, …). `ObjectRef` alone -exposes **123 documented methods** in `lua_api.md`. - -Collision is swept-AABB against nodes and other objects -(`collisionMoveSimple`), with `stepheight`, and a separate `Raycast` iterator -and an A* `pathfinder.cpp` (1,419 LOC). - -**Demands:** float vectors, AABB math, `sqrt`, and dynamic per-entity component -sets (an entity gains/loses attachments, nametags, physics at runtime). - -### 2.6 Items, inventory, crafting - -`ItemDefManager`, `ItemStack` (name + count + wear + metadata), `InvRef` lists, -`craftdef.cpp` (1,250 LOC) with shaped / shapeless / toolrepair / cooking / -fuel recipe types, and **groups** — the string→int tag system that everything -(digging times, damage, biome placement) keys off: - -```lua -groups = {cracky = 3, level = 2, not_in_creative_inventory = 1} -tool_capabilities = { - full_punch_interval = 1.0, max_drop_level = 1, - groupcaps = { cracky = {times = {[1]=2.0, [2]=1.0, [3]=0.5}, uses = 20, maxlevel = 2} }, - damage_groups = {fleshy = 2}, -} -``` - -**Demands:** string-keyed maps, dynamic arrays, and a runtime registry. - -### 2.7 Client rendering - -The heaviest subsystem. `content_mapblock.cpp` (1,870 LOC) turns a MapBlock -into a mesh, one branch per **drawtype** — there are 19: - -``` -NORMAL AIRLIKE LIQUID FLOWINGLIQUID GLASSLIKE ALLFACES ALLFACES_OPTIONAL -TORCHLIKE SIGNLIKE PLANTLIKE FENCELIKE RAILLIKE NODEBOX GLASSLIKE_FRAMED -FIRELIKE GLASSLIKE_FRAMED_OPTIONAL MESH PLANTLIKE_ROOTED -``` - -Around it: `mapblock_mesh.cpp` + `mesh_generator_thread.cpp` (background -meshing), `clientmap.cpp` (frustum culling, transparency sorting), -`shader.cpp`, 20 GLSL shader programs (`client/shaders/`) covering nodes, -objects, bloom down/upsample, FXAA, volumetric light, shadow mapping, -exposure adaptation, sky, clouds, minimap, and `imagesource.cpp` (1,915 LOC) -implementing the **texture modifier mini-language** (`default_dirt.png^grass.png^[opacity:160`). - -**Demands:** a GPU. Vertex/index buffers, texture atlases + mipmaps, a shader -pipeline, matrices, frustum math. None of this exists in Ludic, whose renderer -is a `320×240` `i32` framebuffer blitted through Core Graphics. - -### 2.8 GUI: formspec, HUD, chat - -`guiFormSpecMenu.cpp` is the single largest file in the engine at **5,610 LOC**. -It parses a string DSL with **64 element types** (`list`, `field`, `dropdown`, -`scroll_container`, `hypertext`, `model`, `tabheader`, `style`, `table`, …): - -```lua -core.show_formspec(name, "mymod:chest", table.concat({ - "formspec_version[7]", "size[8,9]", - "list[context;main;0.375,0.75;8,4;]", - "list[current_player;main;0.375,5.5;8,4;]", - "listring[context;main]", -})) -``` - -Plus a HUD element system (`hud_element.h`), a chat console, and a markup -`hypertext` renderer (`guiHyperText.cpp`, 1,254 LOC). - -**Ludic's `ui` block is genuinely the right shape for this** — retained widget -tree as data — but covers ~8 widget types against 64, has no scrolling, no -tables, no text input, and no inventory-slot widget. - -### 2.9 Networking - -Custom reliable-UDP over a channel abstraction (`src/network/mtp/impl.cpp`, -1,685 LOC + `threads.cpp`, 1,429 LOC): split packets, reliable/unreliable -delivery, ack windows, peer timeout. **90 packet types** -(`ToClientCommand` / `ToServerCommand`), protocol version floor 37. Server -sends mapblocks by distance priority, streams media by hash, and runs an auth -handshake (SRP). - -**Demands:** UDP sockets, threads, timers, checksums, and compression. - -### 2.10 Audio - -OpenAL + Ogg Vorbis (`src/client/sound/`, 11 files): 3D positional sound, -fading, looping, per-object attachment, a proxy manager so the audio thread -never blocks the main loop. - -**Ludic has no audio of any kind.** Not a builtin, not a runtime primitive. - -### 2.11 Scripting — the actual design centre - -This is the part that is easy to under-weight. **Luanti is not a game; it is a -Lua host.** `games/devtest` and Minetest Game are *mods*. The C++ engine's -purpose is to expose: - -- **321** `core.*` functions -- **31** `core.register_on_*` global callbacks -- **19** documented classes (`ObjectRef`, `ItemStack`, `InvRef`, `VoxelManip`, - `NodeMetaRef`, `PcgRandom`, `ValueNoiseMap`, `AreaStore`, `Settings`, …) -- **24** definition-table schemas (node, item, entity, ABM, LBM, biome, ore, - decoration, particle, HUD, craft, …) -- a security sandbox (`s_security.cpp`, 1,253 LOC) that whitelists the Lua - stdlib and confines file access to the mod's own directory -- an async job environment, a mapgen-thread environment, and **SSCSM** - (server-sent client-side mods) - -Mods are hot-loaded at runtime from directories, in dependency order, and can -be added by a server operator without recompiling anything. - -**This is the deepest architectural gap**, and it collides head-on with a stated -project constraint (compiled/native only; no VM, no interpreter). Resolved in -**G-25** — the answer is a compiled mod ABI, not a scripting language. - -### 2.12 Support layers - -Threads + mutex + semaphore + event (`src/threading/`), `SettingsManager`, -gettext i18n with plural forms and per-server translations, `httpfetch` (cURL), -JSON, SRP auth, `Profiler`, `AreaStore` (spatial index), zlib **and** zstd, -GMP for bignum crypto. - -External dependencies Luanti links: `SQLite3 Lua CURL Freetype GettextLib -OpenAL Vorbis OpenSSL PostgreSQL ZLIB Zstd GMP Json Threads Ncursesw`. - ---- - -## 3. Ludic today — verified capability inventory - -Everything below was confirmed against the compiler source, not the docs. - -**Types:** `int` (i32), `fixed` (Q16.16 in i32), `bool` (i32), `entity` (i32), -`str` (`ptr` to literal), `ptr` (opaque), `void`. That is the complete list -(`compiler/ludicc.c:516`). - -**Declarations:** `game`, `module`/`export`, `import`, `component`, `archetype`, -`const`, `var` (module-level globals — yes, these exist), `fn`, `extern fn`, -`system`, `scene`/`layer`, `ui`. - -**Statements:** `let`, assignment (`= += -= *= /=`), `if`/`else`, `when`, -`while`, `for i in a..b`, `for (…) in query […] where …`, `return`, `spawn`, -`despawn`, `match`, `machine`/`become`, `enter`. - -**ECS:** dense per-component arrays sized `LUDIC_MAX_ENT = 1024`, a parallel -`@H_` byte per entity for "has", `@L_kind` for archetype, `@L_alive`, a -free list. Queries are **linear scans over all 1,024 slots** re-checking the -mask each visit. - -**Intrinsics** (the floor, lowering to libc/OS symbols): -`mem_alloc mem_free mem_copy mem_set peek8 poke8 peek32 poke32 ptr_add ptr_null -ptr_is_null file_open file_read file_write file_seek file_tell file_close -read_byte write_byte print_str str_len os_exit os_time shl shr band bor bxor -bnot peekp pokep peekf pokef as_fixed as_int is_windowed game_title win_open -win_poll win_present win_running win_close` - -**Runtime (written in Ludic):** framebuffer + 2D draw, 5×7 bitmap text, a -from-scratch TrueType engine (Q16.16 béziers, AA), PNG decode incl. DEFLATE -*inflate*, sprites + 9-slice, tilemap, xorshift RNG, 64 int registers, retained -UI (panel/col/row/label/button/image/spacer), whole-world save/load snapshot. - -**Confirmed absent:** arrays · structs as values · strings (any operation) · -floats · integers wider or narrower than 32 bits · unsigned · enums · unions · -generics · closures · function pointers · maps/dicts · slices · bounds checks · -error handling · `sqrt`/trig · vectors/matrices · threads · atomics · sockets · -audio · GPU · dynamic loading · runtime component attach/detach · a compressor · -any entity count above 1,024. - ---- - -## 4. Gap register — summary - -Tiers are dependency-ordered: nothing in tier *n* is buildable before tier *n−1*. - -| # | Gap | Tier | Blocks | Effort | -|---|---|---|---|---| -| G-01 | Array & slice types | 0 core | everything | L | -| G-02 | Struct / record value types | 0 core | MapNode, vectors, defs | L | -| G-03 | Strings (owned, UTF-8, ops) | 0 core | names, metadata, formspec | L | -| G-04 | Wide numerics: `f32`/`f64` or Q32.32 | 0 core | **world coordinates** | M | -| G-05 | Sized & unsigned ints (`u8 u16 i64`) | 0 core | node packing, serialization | M | -| G-06 | Enums & tagged unions | 0 core | drawtypes, packets | M | -| G-07 | Math library (`sqrt`, trig, `atan2`) | 0 core | physics, noise, camera | S | -| G-08 | Allocator/arena discipline + bounds checks | 0 core | 147 MB of voxels | M | -| G-09 | Error handling (`result`/`try`) | 0 core | I/O, net, parsing | M | -| G-10 | Generics or monomorphised containers | 0 core | every collection | L | -| G-11 | Hash maps | 0 core | registries, name↔id | M | -| G-12 | Entity count beyond 1,024; chunked ECS storage | 1 ecs | any real world | L | -| G-13 | Runtime component attach/detach | 1 ecs | entity properties | S | -| G-14 | Multiple worlds / ECS instances | 1 ecs | client+server in one binary | M | -| G-15 | Query indices (no linear scan) | 1 ecs | performance | M | -| G-16 | Spatial/probabilistic systems (ABM-shaped) | 1 ecs | world simulation | M | -| G-17 | Threads, mutexes, atomics, channels | 2 plat | mapgen, meshing, net | L | -| G-18 | Sockets (UDP + TCP) | 2 plat | multiplayer | M | -| G-19 | Monotonic + wall clock, sleep, timers | 2 plat | fixed timestep | S | -| G-20 | Dynamic library loading at runtime | 2 plat | mods | S | -| G-21 | Compression (deflate **encode**, zstd) | 2 plat | map storage, net | M | -| G-22 | Voxel chunk primitive + VoxelManip | 3 voxel | the game itself | L | -| G-23 | Lighting propagation & mesh generation | 3 voxel | rendering | L | -| G-24 | Collision, raycast, AABB, pathfinding | 3 voxel | movement | M | -| G-25 | **Mod/extension model (compiled ABI)** | 4 ext | Luanti's whole point | XL | -| G-26 | Data-driven registries & definition tables | 4 ext | node/item defs | M | -| G-27 | Metadata stores (node/item/player) | 4 ext | chests, signs | M | -| G-28 | GPU backend: 3D pipeline, shaders | 5 gfx | 3D at all | XL | -| G-29 | 3D math types (vec/mat/quat/aabb) | 5 gfx | everything 3D | M | -| G-30 | UI: 64 formspec elements, scroll, input, tables | 5 gfx | inventories | L | -| G-31 | Audio: mixing, Ogg decode, 3D positional | 6 av | polish | L | -| G-32 | i18n: gettext, plurals, server translations | 6 av | shipping | M | -| G-33 | Persistence: versioned block format, SQLite FFI | 7 net | saving | M | -| G-34 | Network protocol: reliable UDP, 90 packets | 7 net | multiplayer | XL | - -`S ≈ days · M ≈ 1–3 weeks · L ≈ 1–2 months · XL ≈ a quarter+` (single developer.) - ---- - -## 5. The gaps in detail - -Each entry: what Luanti needs · what Ludic has · proposed Ludic feature · the -compiler/runtime work it implies. - -### Tier 0 — Language core - -#### G-01 · Array & slice types - -**Luanti needs.** `MapNode data[4096]` per block; `std::vector` per -nodebox; noise buffers of `w*h*d` floats; the 6 tiledefs per node. - -**Ludic has.** Nothing. `runtime/native/core.ludic` fakes arrays with -`mem_alloc` + `peek32`/`poke32` and manual stride arithmetic: - -```ludic -rt_fb = mem_alloc(320 * 240 * 4) -poke32(rt_fb, (y * rt_fbw + x) * 4, color) # every access, by hand -``` - -That is workable for a 3,700-line runtime and fatal for a 100,000-line engine: -no bounds checks, no element type, no length. - -**Proposed.** - -```ludic -# fixed-size, stack or inline in a struct — length is part of the type -let box: [16]int - -# heap-allocated, length carried, bounds-checked in debug builds -let nodes: [] MapNode = alloc [4096] MapNode -nodes[i].param0 = C_STONE -print_int(nodes.len) - -# slices: a (ptr, len) pair, no ownership — the workhorse for I/O -function checksum(bytes: []u8) -> int { - var sum = 0 - for b in bytes { sum = sum + b } # `for x in slice` iteration - return sum -} -let window = nodes[0 .. 256] # slicing, no copy -``` - -**Work.** Lexer: `[`/`]` in type position. Types: a new `TARR{elem, len}` and -`TSLICE{elem}`. Codegen: LLVM `[N x T]` and `{ptr, i64}`; `getelementptr` for -indexing; a `panic_bounds` intrinsic. Semantics decision: slices are non-owning, -`alloc` returns an owning array that `free` releases. **This is the single -highest-leverage item in the document** — G-02, G-03, G-05, G-11, G-22 all -depend on it. - -#### G-02 · Struct / record value types - -**Luanti needs.** `MapNode` is a 4-byte value passed by copy 10⁶ times a frame. -`v3s16`, `aabb3f`, `TileDef`, `NoiseParams`, `ObjectProperties`, `CollisionInfo`. - -**Ludic has.** `component`, which is *not* a value type — it is a row in a -global ECS array, addressable only through a query binding. You cannot declare -a local of component type, return one, or nest one. - -**Proposed.** Split "data shape" from "ECS membership": - -```ludic -struct MapNode { - param0: u16 = 0 # content id - param1: u8 = 0 # light: day|night nibbles - param2: u8 = 0 # facedir / level / liquid -} - -struct v3i { x: int = 0, y: int = 0, z: int = 0 } - -function index(p: v3i) -> int { return (p.z * 16 + p.y) * 16 + p.x } - -function get(blk: []MapNode, p: v3i) -> MapNode { return blk[index(p)] } - -# a component becomes "a struct that lives in the world" -component Pos = v3i # aliasing form -component Vel { dx: fixed = 0, dy: fixed = 0 } # existing form still valid -``` - -Add `expr with { field = … }` (already flagged "not implemented" in -LANGUAGE.md) so records can be updated functionally: - -```ludic -let lit = n with { param1 = pack_light(15, 15) } -``` - -**Work.** `NSTRUCT` decl, struct types in the type table, LLVM named struct -types, by-value passing/returning (`byval` for large ones), field access on -non-component values, `with` lowering to insertvalue chains. - -#### G-03 · Strings - -**Luanti needs.** Node names (`"default:stone"`), item strings -(`"default:pick_steel 1 250"`), formspec strings built by concatenation, chat, -metadata keys, mod names, translation keys, the texture-modifier DSL. - -**Ludic has.** `str` = a pointer to a literal. Verified: `"ab" + "cd"` passes -the parser and then **fails to assemble**. `str_len` exists as an intrinsic; -nothing else does. - -**Proposed.** - -```ludic -# `str` stays an immutable literal/slice: (ptr, len), UTF-8, no allocation -# `string` is owned and growable -let name: str = "default:stone" -var buf: string = string_new() -buf.push("size[8,9]") -buf.push_fmt("list[context;main;{};{};8,4;]", x, y) -let out: str = buf.view() - -if name.starts_with("default:") { … } -let (modname, item) = name.split_once(':') -let h = name.hash() # for the registry map -for cp in name.codepoints() { … } # UTF-8 aware -``` - -**Work.** Interning table for literals (already partly there for `.str` globals), -a `string` runtime type in `runtime/native/string.ludic`, `+`/`==`/`<` operator -lowering for `str`, formatting (`push_fmt`) which needs varargs or a small -format-node lowering. Note the TrueType engine already decodes UTF-8 — reuse it. - -#### G-04 · Wide numerics — **the coordinate bug** - -**Luanti needs.** `f32` positions in BS units. `MAX_MAP_GENERATION_LIMIT = -31007` nodes; `BS = 10.0f`; so world coordinates reach **±310,070**, and squared -distances reach **~10⁹**. - -**Ludic has.** Q16.16 in i32. Range **±32,767.99998**. Verified IR: - -```llvm -%t2 = sext i32 %t0 to i64 ; 64-bit intermediate — precision is fine -%t3 = sext i32 %t1 to i64 -%t4 = mul i64 %t2, %t3 -%t5 = ashr i64 %t4, 16 -%t6 = trunc i64 %t5 to i32 ; …but the result truncates back to Q16.16 -``` - -So `fixed` **cannot hold a Luanti world coordinate at all**, and `d = dx*dx + -dy*dy + dz*dz` wraps silently for anything past ~181 nodes from the origin. -Every collision test, raycast, and mob AI decision would be wrong far from -spawn. - -**Proposed — pick one (recommendation: both, in this order):** - -1. **`fixed64` / Q32.32 on i64** — keeps determinism, which is a genuine - advantage for a multiplayer voxel game (bit-identical physics across - platforms; Luanti itself suffers from float drift here). - ```ludic - let px: fixed64 = 310070.5 # ±2.1e9 range, 2⁻³² precision - ``` -2. **`f32` / `f64`** — required anyway for mapgen noise (`NoiseParams` is all - floats and the noise functions are transcendental), and for GPU interop where - vertex data *must* be IEEE floats. - ```ludic - let h: f32 = noise3d(x, y, z, seed) - ``` - -**Work.** New scalar types in the type table; LLVM `i64` / `float` / `double`; -promotion rules (`int → fixed → fixed64`, `int → f32 → f64`); literal suffixes -(`1.5f`, `1.5d`); `fmul`/`fdiv`/`fcmp` lowering; printf format selection. - -#### G-05 · Sized & unsigned integers - -**Luanti needs.** `MapNode` is `u16 + u8 + u8` — exactly 4 bytes, because 36 -million of them exist. Serialisation is byte-exact and big-endian. `content_t` -is `u16`; light is two 4-bit nibbles. - -**Ludic has.** i32 only. A `MapNode` built from Ludic `int`s would be 12 bytes — -**3× the memory**, 441 MB instead of 147 MB for a 10-block view radius. - -**Proposed.** - -```ludic -struct MapNode { param0: u16, param1: u8, param2: u8 } # exactly 4 bytes - -let light_day: u8 = band(n.param1, 0x0f) -let content: u16 = n.param0 -let key: u64 = block_key(bp) # sqlite position encoding -``` - -Plus explicit conversion (`as u8`, `as int`) and wrap/saturate semantics stated -in the spec. - -**Work.** Type table entries for `u8 u16 u32 u64 i8 i16 i64`; LLVM `zext`/`sext` -/`trunc` at boundaries; struct layout + alignment rules; unsigned comparison and -shift (`lshr` vs `ashr` — the existing `shr` intrinsic already lowers to `lshr`, -which is wrong for signed ints, and would become type-directed). - -#### G-06 · Enums & tagged unions - -**Luanti needs.** 19 `NodeDrawType`s, 90 packet types, `LiquidType`, -`ContentParamType`, `CollisionType`, and `pointed_thing` which is a genuine sum -type (`nothing` | `node{under,above}` | `object{ref}`). - -**Ludic has.** `const` ints, and `match` over int literals (which is already -half of what's needed and works well). - -**Proposed.** - -```ludic -enum DrawType { Normal, Airlike, Liquid, FlowingLiquid, Glasslike, Allfaces, - Torchlike, Signlike, Plantlike, Fencelike, Raillike, Nodebox, - GlasslikeFramed, Firelike, Mesh, PlantlikeRooted } - -union Pointed { - Nothing - Node { under: v3i, above: v3i } - Object { ref: entity } -} - -match p { - Pointed.Nothing => return - Pointed.Node(u, a) => place_node(a, item) # destructuring arms - Pointed.Object(e) => punch(e) -} -``` - -`match` already exists and already lowers on the native backend — extending its -arms to enum cases and destructuring patterns is incremental, not new -machinery. - -#### G-07 · Math library - -**Luanti needs.** `sqrt` (every distance), `sin`/`cos` (rotation, sky, camera), -`atan2` (`automatic_face_movement_dir`), `pow`/`exp` (noise persistence, -exposure adaptation), `floor`/`ceil`/`round`. - -**Ludic has.** `min max abs clamp` — integer only. **Verified: no `sqrt` -anywhere in the compiler or the runtime.** - -**Proposed.** A `runtime/native/math.ludic` written in Ludic (consistent with -the project's "runtime is in Ludic" rule) — CORDIC or polynomial approximations -for trig on `fixed`, Newton–Raphson `sqrt`, plus direct `llvm.sqrt.f32` / -`llvm.sin.f32` intrinsic lowering once `f32` lands (G-04). - -```ludic -sqrt(x: fixed) -> fixed sin(a: fixed) -> fixed atan2(y, x) -> fixed -pow(b, e) -> fixed floor(f) -> int lerp(a, b, t) -> fixed -``` - -**Effort: S.** This one is a weekend, and unblocks a surprising amount. - -#### G-08 · Memory discipline - -**Luanti needs.** A 10-block view radius is ~9,000 MapBlocks ≈ **147 MB of node -data alone**, churning constantly as blocks load and unload. - -**Ludic has.** `mem_alloc`/`mem_free` (raw `malloc`/`free`), no ownership -tracking, no bounds checks, no leak detection. `runtime/native/image.ludic` -already leaks decoded PNG buffers by design. - -**Proposed.** Not a borrow checker — that is a different language. An **arena -and a pool**, expressed in Ludic, plus optional bounds checking: - -```ludic -arena frame # reset once per frame, no individual frees -let verts = frame.alloc [4096] Vertex - -pool blocks: MapBlock # fixed-size slab, free-list backed -let b = blocks.take() -blocks.give(b) - -# debug builds: every [] access bounds-checked; --release drops the check -``` - -#### G-09 · Error handling - -**Luanti needs.** Corrupt map blobs, truncated packets, missing textures, -unknown node names, failed DB writes. It uses C++ exceptions -(`src/exceptions.h`) plus sentinel returns. - -**Ludic has.** Nothing. `file_open` returns a `ptr` you must test with -`ptr_is_null`; every failure path is a hand-rolled convention. - -**Proposed.** No unwinding (it would fight the AOT/no-runtime philosophy) — -sum-typed results with sugar, which falls straight out of G-06: - -```ludic -function read_block(path: str) -> MapBlock ! IoError { - let f = file_open(path, "rb") else return IoError.NotFound - let n = try file_read(f, buf, 4096) # `try` propagates the error arm - return parse(buf[0..n]) -} - -match read_block(p) { - Ok(b) => use(b) - Err(e) => log_warn("block load failed", e) -} -``` - -#### G-10 · Generics (or monomorphised containers) - -Without some form of parametric types, every container is written N times. -Minimum viable: **compile-time monomorphisation over one type parameter**, no -constraints, no inference beyond the call site. - -```ludic -struct Vec[T] { data: []T, len: int, cap: int } - -fn push[T](v: Vec[T], x: T) -> void { - if v.len == v.cap { v.data = realloc(v.data, v.cap * 2) v.cap = v.cap * 2 } - v.data[v.len] = x - v.len = v.len + 1 -} - -var objects: Vec[entity] = vec_new[entity]() -``` - -**Work.** Parse `[T]` in decls and calls; a monomorphisation pass keyed on -(fn, concrete args) before codegen; name mangling. Deliberately *not* traits, -*not* variance, *not* HKT — this is a game language, and Luanti's C++ uses -templates in exactly this restrained way. - -#### G-11 · Hash maps - -**Luanti needs.** name→id for nodes and items (~60k entries), the definition -registries, `NodeMetaRef` string→string stores, groups (`string → int`), mod -storage, translation catalogues. - -**Proposed.** One monomorphised open-addressing map, in Ludic: - -```ludic -var by_name: Map[str, u16] = map_new[str, u16]() -by_name.set("default:stone", 1) -if let id = by_name.get("default:stone") { … } -for (k, v) in by_name { … } -``` - -### Tier 1 — ECS at world scale - -#### G-12 · The 1,024-entity ceiling - -**The number.** `compiler/native.c:18` — `#define LUDIC_MAX_ENT 1024`. Every -component becomes `@S_ = internal global [1024 x %Cmp_]`, plus a -`[1024 x i8]` presence array. Storage is **static, dense, and allocated for -every component whether used or not.** - -**What Luanti needs.** A busy server carries thousands of active objects; a -single 16³ MapBlock holds 4,096 nodes. Nodes must *not* be ECS entities (see -G-22), but mobs, items, players, particles, and node timers all should be. - -**Proposed.** Two changes, independent: - -1. **Configurable + dynamic capacity.** `game Foo { entities 65536 }` for the - static bound, and a growth path: component arrays become heap - `[]Cmp` that double. -2. **Sparse-set storage** (the standard ECS answer): per component a dense - packed array + a sparse index. Iteration cost becomes proportional to the - number of entities *with that component*, not to 1,024. This also kills the - memory waste: a component held by 10 entities stops costing 1,024 slots. - -```ludic -game Voxel { - entities 262144 # ceiling, or `dynamic` for growable - storage sparse # or `dense` for the current behaviour -} -``` - -#### G-13 · Runtime component attach/detach - -**Luanti needs.** An entity gains a nametag, loses physics, gets attached to a -vehicle, picks up an inventory — all at runtime. `ObjectRef:set_properties`, -`set_attach`, `set_armor_groups` are per-object mutations of the *shape* of the -entity. - -**Ludic has.** Components are attached only by `spawn`. There is no statement to -add or remove one afterwards. (Verified: no `attach`/`detach` in the parser.) -The `@H_` presence byte already exists — the storage supports it; the -*language* does not expose it. - -**Proposed.** - -```ludic -attach Nametag { text = "Zombie", color = 0xffffff } to e -detach Physics from e -if has Physics on e { … } -``` - -**Effort: S** — the runtime representation is already there. - -#### G-14 · Multiple ECS worlds - -Luanti runs a `ServerEnvironment` and a `ClientEnvironment` **in the same -process** for singleplayer. Ludic has exactly one implicit global world. - -```ludic -world Server { … } # separate component storage per world -world Client { … } -for (p) in Server.query [Pos] { … } -``` - -Also needed for: mapgen worker threads that build a block in isolation, and for -tests. - -#### G-15 · Query indices - -Today: `for (p) in query [Pos]` scans slots 0..1023 and re-tests the mask. -With G-12's higher ceilings that becomes the hot loop. Sparse sets (G-12) fix -the common case; the remaining need is **multi-component intersection** — pick -the smallest dense set and probe the others. - -#### G-16 · ABM-shaped systems — *where Ludic can beat Luanti* - -Luanti's ABM is a hand-rolled sampler in C++: pick random positions in loaded -blocks, test `nodenames`, test `neighbors`, roll `chance`, call a Lua closure. -It is one of the most-profiled parts of the engine. - -In Ludic this is a **declaration**: - -```ludic -system LavaCooling phase Update - every 10.0s # interval, not a hand-rolled timer - chance 1 in 50 # sampler, compiled in - nodes [C_LAVA_SOURCE] # content filter → an index, not a scan - near [C_WATER_SOURCE, C_WATER_FLOWING] - in y -32768 .. 32767 -{ - set_node(self_pos(), C_OBSIDIAN) -} -``` - -The compiler knows the node set at compile time, so it can build the per-block -content index once and iterate only matching positions — an optimisation Luanti -cannot do because its ABM predicates arrive as runtime strings. **This is the -strongest argument in the whole document for the language existing.** - -### Tier 2 — Platform - -#### G-17 · Threads, mutexes, atomics - -**Luanti needs.** `EmergeThread` × N (mapgen), `MeshUpdateThread` (client -meshing), `ConnectionSendThread` + `ConnectionReceiveThread`, the sound proxy -thread, and an async Lua job environment. `src/threading/` provides -`Thread`, `Mutex`, `Semaphore`, `Event`, `OrderedMutex`. - -**Ludic has.** Nothing. No `pthread_create` binding, no atomics, and — a real -problem — **`var` globals are unsynchronised i32 stores** and the ECS arrays are -process-global. - -**Proposed.** - -```ludic -thread pool emerge count 4 { # a named pool with a typed job queue - job Generate(bp: v3i) -> MapBlock { … } -} -emerge.submit(Generate(bp)) -if let blk = emerge.poll() { install(blk) } - -atomic var block_count: int = 0 -block_count.add(1) - -mutex map_lock -lock map_lock { … } - -channel Chan[MapBlock] results # MPSC, blocking or try_recv -``` - -Design steer: prefer **message passing + job pools** over shared mutable state, -because it composes with the ECS (a job owns its world slice) and because it is -what a determinism-oriented language should encourage. - -#### G-18 · Sockets - -UDP is mandatory (the whole protocol is UDP), TCP + TLS for the content store -and HTTP mods. - -```ludic -let s = udp_bind("0.0.0.0", 30000) -let (n, from) = s.recv(buf) -s.send_to(from, reply) -``` - -Bind at the syscall level from the runtime, as with `file_open` — `socket`, -`bind`, `sendto`, `recvfrom`, `poll`. No new language feature required beyond -G-01 (slices) and G-09 (errors), which is why this is M, not L. - -#### G-19 · Time - -`os_time()` returns whole seconds. A game loop needs a **monotonic** clock at -microsecond resolution, and Luanti's `DTIME_LIMIT = 2.5f` / fixed-timestep -stepping depends on it. - -```ludic -let t = now_us() # monotonic, u64 -sleep_ms(2) -``` - -#### G-20 · Dynamic loading - -`dlopen`/`dlsym` equivalents, so mods (G-25) can be loaded from a directory at -startup without relinking the engine. - -```ludic -let m = module_load("mods/farming/farming.dylib") -let init = m.symbol("mod_init") as fn(ModApi) -> void -``` - -#### G-21 · Compression - -**Luanti needs.** zstd for map format ≥29, zlib below it, and both directions. - -**Ludic has.** `runtime/native/inflate.ludic` — **decoder only** (verified: the -file implements `z_start`, `z_bits`, `z_table_build`, `z_decode`, `z_stored`; -there is no encoder). PNGs load; nothing can be written. - -**Proposed.** Write `deflate.ludic` (fixed-Huffman + LZ77 with a hash chain is -~300 lines and enough for map storage), and FFI to `libzstd` via `extern fn` for -the real thing. Both consistent with existing project practice. - -### Tier 3 — The voxel engine - -#### G-22 · A voxel chunk primitive - -This is the heart. **Nodes must not be ECS entities** — 36 million entities is -absurd, and Ludic's `entity` is an i32 index into parallel arrays. Voxels want -the opposite layout: dense, implicit position, no id. - -**Proposed** — make chunked volumes a first-class construct, the way `component` -is: - -```ludic -volume Map of MapNode chunk 16 { # 16³ chunks of MapNode - origin v3i # chunk coordinate - meta { generated: bool, timestamp: u32, lighting_complete: u16 } -} - -# indexing is bounds-safe and computes chunk + offset in one shot -let n = Map[pos] -Map[pos] = MapNode { param0 = C_STONE } - -# an explicit borrowed working area = Luanti's MMVManip / VoxelManip, -# which is how mapgen and bulk edits avoid touching the live map -manip vm = Map.borrow(pmin, pmax) -for p in vm.area { if vm[p].param0 == C_AIR { vm[p] = stone } } -vm.commit() # write back + relight + remesh -``` - -Mapped to Luanti: `Map` ≈ `ServerMap`, `chunk` ≈ `MapBlock`, `manip` ≈ -`MMVManip`, `vm.area` ≈ `VoxelArea`, and `commit()` ≈ -`blit_back + update_liquids + calc_lighting`. - -**Work.** L. Chunk table (hash map keyed on `v3i` — needs G-11), an allocator -for chunk payloads (G-08), the index arithmetic, and a loaded/unloaded state -machine. But note: this is *exactly* the kind of thing a domain-specific game -language should have, and no general-purpose language offers it. - -#### G-23 · Lighting & meshing - -**Lighting.** Luanti propagates two banks (day/night) via BFS flood fill across -block boundaries (`voxelalgorithms.cpp`, 1,309 LOC), with the 12-bit -`lighting_complete` mask tracking which faces are still provisional. - -**Meshing.** `content_mapblock.cpp` (1,870 LOC) emits geometry per drawtype; -`mapblock_mesh.cpp` builds the buffers; `mesh_generator_thread.cpp` does it off -the main thread. - -**Proposed.** Both stay *library* code written in Ludic — but they need G-01 -(vertex arrays), G-04 (float vertex data for the GPU), G-17 (mesh threads), and -G-28 (somewhere to send the buffers). Language-level help worth adding: - -```ludic -# a declarative face-culling rule set, instead of 19 hand-written branches -drawtype Normal { - cull when neighbor.solid - faces cube tiles [top, bottom, left, right, front, back] -} -drawtype Plantlike { faces cross scale visual_scale cull never } -``` - -#### G-24 · Collision, raycast, AABB - -Swept AABB against the node grid + other objects, with `stepheight`; a voxel -DDA raycast for pointing; A* over the node grid for mob pathing (1,419 LOC). - -All expressible in Ludic **once G-04 (wide numbers) and G-07 (`sqrt`) exist.** -No new language feature needed — this is library work, ~2,000 lines. - -### Tier 4 — Extensibility - -#### G-25 · The mod model — the hard architectural call - -**The problem.** Luanti's entire value proposition is that a server operator -drops a folder into `mods/` and restarts. 321 `core.*` functions exist to serve -that. Ludic is AOT-compiled, and the project constraint is explicit: **no VM, -no interpreter, no scripting language, ever.** - -These are not reconcilable by adding Lua. They *are* reconcilable, and the -answer is already half-built in the repo — `ludicc lib.ludic --shared -o -lib.dylib` and `extern fn … = "symbol"`. - -**Proposed: compiled mods behind a stable C-ABI interface.** - -```ludic -# mods/farming/farming.ludic -mod Farming { - version 1 - depends ["core"] - - on init(api: ModApi) { - api.register_node("farming:wheat_1", NodeDef { - drawtype = DrawType.Plantlike, - tiles = ["farming_wheat_1.png"], - groups = [("snappy", 3), ("flammable", 2)], - walkable = false, - buildable_to = true, - }) - - api.register_abm(Abm { - nodes = ["farming:wheat_1"], - interval = 20.0, chance = 20, - action = grow_wheat, - }) - } -} -``` - -Built with `ludicc mods/farming/farming.ludic --shared -o mods/farming/farming.dylib`, -loaded at startup via G-20, and talking to the engine only through a versioned -`ModApi` vtable. - -**What you gain over Lua:** mods run at native speed (Luanti's ABMs and -`on_generated` handlers are its top profiler entries, and they are all Lua); -mod errors are compile-time; the ECS is available to mods directly. - -**What you lose, and must decide about:** -- **No hot reload** without process restart (mitigable: reload the dylib). -- **No sandbox.** Lua's `s_security.cpp` confines mods to their own directory; - a native dylib can do anything. Options: (a) accept it, like Factorio pre-2.0 - or any C++ plugin system; (b) run mods in a subprocess with an IPC boundary; - (c) restrict the `ModApi` surface and ship signed mods. -- **Distribution.** Mods become per-platform binaries, or ship as source and - compile on first load (`ludicc` is 2,508 lines — shipping it *is* viable). - -**My recommendation:** ship-as-source + compile-on-load. It keeps the "drop a -folder in" workflow, keeps everything in one language, needs no VM, and turns -`ludicc`'s small size into a genuine architectural advantage. - -#### G-26 · Definition tables & registries - -Luanti registers definitions from Lua tables at runtime. Ludic has no table -literal, no runtime registry, and no way to name 60,000 content ids. - -**Proposed** — a declarative `def` construct that is data at compile time *and* -registrable at runtime: - -```ludic -def node Stone { - name = "default:stone" - drawtype = DrawType.Normal - tiles = ["default_stone.png"] - groups = [("cracky", 3)] - is_ground_content = true - sounds = SoundSet.Stone - drop = "default:cobble" -} -``` - -The compiler assigns content ids, emits the table as static data, and the -registry maps `str → u16` at load (G-11). Mod-supplied defs go through the same -path at runtime. - -#### G-27 · Metadata - -`NodeMetaRef` / `ItemStackMetaRef` / `PlayerMetaRef`: string→string stores -attached to positions, stacks, and players, with an "inventory" attached to node -meta. Needs G-03 and G-11; then it is straightforward library code. - -### Tier 5 — Graphics & UI - -#### G-28 · The GPU - -Luanti bundles an **87,554-line Irrlicht fork** to get: OpenGL 3 / GLES2 / -WebGL contexts, vertex + index buffers, texture management with mipmaps and -atlases, a material/shader system, scene graph, frustum culling, and mesh -loaders (B3D, OBJ, glTF). On top of that sit 20 GLSL programs. - -Ludic has a 320×240 software framebuffer presented through -`runtime/native/cocoa.ll` (327 lines of hand-written LLVM IR calling -`objc_msgSend`). - -**This is the largest single gap in the document.** Options: - -| Option | Cost | Fit with constraints | -|---|---|---| -| Software rasteriser in Ludic | L | ✅ perfect — but ~5 fps at 720p for voxels | -| Bind OpenGL/Metal via `extern fn` | M | ✅ same posture as zlib FFI; **recommended** | -| Hand-written IR per platform, like `cocoa.ll` | XL | ✅ but repeated per API | -| Write a GPU driver | ✗ | not serious | - -**Recommended:** treat the GPU exactly as the project already treats the OS — -an ABI to call, not a C program to compile. `extern function glDrawElements(…) = -"glDrawElements"`, with a thin `runtime/native/gfx3d.ludic` over it. This is -consistent with the existing rule ("C libraries are used via FFI") and needs -**no compiler change beyond G-04's `f32`.** - -#### G-29 · 3D math types - -```ludic -struct v3f { x: f32 = 0, y: f32 = 0, z: f32 = 0 } -struct m4 { m: [16]f32 } -struct aabb { min: v3f, max: v3f } - -function dot(a: v3f, b: v3f) -> f32 -function cross(a: v3f, b: v3f) -> v3f -function normalize(v: v3f) -> v3f -function look_at(eye: v3f, at: v3f, up: v3f) -> m4 -function perspective(fovy: f32, aspect: f32, znear: f32, zfar: f32) -> m4 -``` - -Worth considering: **operator overloading for vector types only** — Ludic -currently forbids overloading entirely, and `a + b` on vectors is the one place -where the ban costs real readability. A narrow, built-in-types-only exception is -defensible. - -#### G-30 · UI at formspec scale - -Ludic's `ui` block is architecturally right and 8 widgets deep. Formspec is 64 -elements. The gap that matters most: - -| Missing | Why it blocks Luanti | -|---|---| -| `field` / `textarea` / `pwdfield` | no text input at all — no chat, no sign editing | -| `list` (inventory slots) | the core interaction of the game | -| `scroll_container` / `scrollbar` | any inventory bigger than a screen | -| `table` / `textlist` | server list, mod manager | -| `dropdown`, `checkbox`, `tabheader` | settings | -| `model` | 3D item preview | -| `style[]` / `style_type[]` | theming | -| mouse input | **Ludic's UI is keyboard-only today** (`ui_tick(key())`) | -| `hypertext` markup | formatted text | - -### Tier 6 — Audio & i18n - -#### G-31 · Audio - -Nothing exists. Needed: a mixer, Ogg Vorbis decode (or ship a simpler codec), -3D panning + attenuation, and an audio callback thread (G-17). Consistent with -project practice: decode in Ludic (as PNG and TrueType already are), bind -CoreAudio/ALSA/WASAPI via `extern fn`. - -```ludic -let s = sound_load("assets/dig_stone.ogg") -sound_play(s, SoundSpec { gain = 0.8, pitch = 1.0, pos = p, max_hear = 32.0 }) -``` - -#### G-32 · i18n - -Luanti: gettext, plural forms (`gettext_plural_form.cpp`), per-server -translation files (`.tr`), and escape-sequence-based inline translation. Ludic -has UTF-8 rendering (good) and no translation machinery. Needs G-03 and G-11. - -### Tier 7 — Persistence & network - -#### G-33 · Persistence - -Ludic's `save()`/`load()` writes **the entire ECS world** in one blob. Luanti -needs per-block, versioned, compressed, incrementally-written storage keyed in -SQLite, plus player files, auth, and mod storage. - -```ludic -serialize MapBlock version 29 { - u8 version - u8 flags - u16 lighting_complete - u32 timestamp - map name_id: Map[u16, str] - zstd { # everything inside is compressed - array param0: [4096]u16 bigendian - array param1: [4096]u8 - array param2: [4096]u8 - list metadata - list timers - list static_objects - } -} -``` - -A declarative `serialize` block generating both encoder and decoder would be a -strong Ludic feature — it is exactly the kind of error-prone, symmetric, -version-tagged code that a game language should generate rather than hand-write. - -#### G-34 · Networking - -90 packet types, reliable UDP with split packets and ack windows, and priority -block streaming. Depends on G-17, G-18, G-21, G-33. The packet definitions -themselves fall out of `serialize` (G-33) + `union` (G-06): - -```ludic -packet ToClient { - BlockData = 0x20 { pos: v3i, block: MapBlock } - AddParticle = 0x46 { … } - ActiveObjectMessages = 0x31 { msgs: []ObjectMsg } -} -``` - ---- - -## 6. Roadmap - -Ten milestones. Each one **ends in something runnable** — that is the rule; no -milestone is "add a type system feature" with nothing to show. Effort estimates -assume one developer and are deliberately conservative. - -### M0 — Language core, part 1: aggregates *(≈6 weeks)* -> **Gates:** G-01 arrays/slices · G-02 structs · G-05 sized ints - -**Demo:** rewrite `runtime/native/image.ludic`'s PNG decoder to use -`[]u8` and `struct Pixel` instead of `mem_alloc` + `peek8`. Same output PNG, -half the lines, bounds-checked. - -**Why first:** every other item in this document is blocked on it. The existing -runtime is the perfect proving ground — it is 3,700 lines of exactly the -hand-rolled pointer arithmetic these features exist to delete. - -```ludic -# before (runtime/native/image.ludic today) -function px(img: pointer, x: int, y: int, w: int) -> int { return peek32(img, (y*w+x)*4) } - -# after -struct Rgba { r: u8, g: u8, b: u8, a: u8 } -function px(img: []Rgba, x: int, y: int, w: int) -> Rgba { return img[y*w + x] } -``` - -### M1 — Language core, part 2: numbers, strings, errors *(≈6 weeks)* -> **Gates:** G-03 strings · G-04 `f32`/`fixed64` · G-06 enums/unions · G-07 math · G-09 errors - -**Demo:** a `noise.ludic` implementing Luanti's `NoiseFractal3D` exactly, and a -headless program that renders a 512×512 heightmap PNG from it. Compare against -Luanti's own output for the same seed — **bit-comparable terrain is the -acceptance test.** - -This is the milestone that closes the coordinate bug (G-04). Worth writing the -regression test first: - -```ludic -system CoordRange phase Start { - let far: fixed64 = 310070.0 # would wrap in Q16.16 - assert(flr64(far) == 310070) - let d = far * far # 9.6e10 — must not wrap -} -``` - -### M2 — Platform *(≈5 weeks)* -> **Gates:** G-17 threads · G-19 time · G-18 sockets · G-20 dynamic loading · G-21 deflate encode - -**Demo:** a headless Ludic program that spawns 4 worker threads, each generating -a 16³ noise chunk, deflate-compresses it, writes it to a file, and a second -process that reads it back — proving the thread pool, the compressor and the -round-trip. - -**Note on the new WASM target.** A peer session just landed WebAssembly support -(`runtime/web/`, `bin/x test` now 64/64), and `@main` is now split into -`ludic_boot/frame/alive/teardown` so a browser can drive the loop from -`requestAnimationFrame`. Two consequences for this roadmap: (a) threads on -wasm32 mean Web Workers + SharedArrayBuffer, not pthreads, so **G-17 needs a -per-target backend from day one**; (b) `size_t` is `i32` on wasm — new -intrinsics must go through `ll_size_t()`/`ll_widen()`/`ll_narrow()`, which -matters for every slice and array intrinsic added in M0. - -### M3 — The voxel primitive *(≈8 weeks)* -> **Gates:** G-11 maps · G-12 ECS scale · G-13 attach/detach · G-22 `volume`/`manip` - -**Demo:** **a flat 3D world you can fly through**, still software-rendered at -320×240 — a chunked `volume Map of MapNode chunk 16`, chunks generated on the -worker pool from M2, a free camera, and a simple raycast rasteriser for -visualisation. Ugly, slow, and unambiguously a voxel engine. - -```ludic -volume Map of MapNode chunk 16 { origin v3i meta { generated: bool } } - -system Generate phase Update { - for bp in pending_chunks() { - manip vm = Map.borrow_chunk(bp) - for p in vm.area { - let h = floor(noise2d(fixed(p.x) / 64.0, fixed(p.z) / 64.0) * 24.0) + 8 - vm[p] = if p.y <= h { NODE_STONE } else { NODE_AIR } - } - vm.commit() - } -} -``` - -### M4 — GPU + 3D *(≈10 weeks)* -> **Gates:** G-28 GPU FFI · G-29 3D math · G-23 meshing/lighting - -**Demo:** the M3 world, **textured, lit and running at 60 fps** — greedy or -per-face meshing on a worker thread, day/night light banks propagated by BFS, -one shader program, a texture atlas from the existing PNG decoder. - -This is where the project stops being a 2D game language and becomes capable of -Luanti's genre. It is also the milestone with the most schedule risk — bind -OpenGL 3.3 via `extern fn` first (Metal/WebGL later) and resist the temptation -to write a scene graph. - -### M5 — Interaction *(≈6 weeks)* -> **Gates:** G-24 collision/raycast · G-30 UI (input, lists, scroll) · mouse input - -**Demo:** **you can walk, jump, dig and place.** Swept-AABB player physics with -`stepheight`, a DDA raycast for pointing, a hotbar, and an inventory screen with -draggable slots. This is the first milestone that is *playable*. - -### M6 — Definitions & registries *(≈5 weeks)* -> **Gates:** G-26 `def` blocks · G-27 metadata · G-16 ABM systems - -**Demo:** 30 node types and 10 item types declared with `def node` / `def item`, -groups-based dig times, a chest with node metadata and an inventory, and lava -that cools into obsidian via an ABM-shaped system. **This is the milestone where -the language's ECS finally pays off visibly.** - -### M7 — Persistence *(≈4 weeks)* -> **Gates:** G-33 `serialize` blocks · SQLite via FFI - -**Demo:** quit and relaunch; the world, your inventory and your position are -exactly as you left them. Blocks written incrementally, zstd-compressed, keyed -by interleaved position — read Luanti's own `map.sqlite` as a compatibility -stretch goal. - -### M8 — The mod ABI *(≈8 weeks)* -> **Gates:** G-25 mod model · G-20 loading · a versioned `ModApi` - -**Demo:** a `mods/farming/` folder containing only `.ludic` source and textures; -drop it in, restart, and wheat exists — compiled on first load, cached as a -dylib. Then delete the folder and it is gone. **This is the milestone that makes -it Luanti rather than a voxel game.** - -### M9 — Multiplayer *(≈12 weeks)* -> **Gates:** G-34 protocol · G-14 multiple worlds · G-18 sockets - -**Demo:** two clients, one server, on separate machines: block changes, -movement, chat, and inventory sync. Client and server in one binary via -`world Server` / `world Client` (G-14) so singleplayer is just a loopback. - -### Audio & i18n *(fold into M5–M8 opportunistically)* -> **Gates:** G-31 · G-32 - -### Timeline summary - -| Milestone | Effort | Cumulative | Ends with | -|---|---|---|---| -| M0 aggregates | 6 wk | 6 wk | runtime rewritten on arrays/structs | -| M1 numbers/strings | 6 wk | 12 wk | Luanti-comparable noise terrain | -| M2 platform | 5 wk | 17 wk | threaded, compressed chunk I/O | -| M3 voxel primitive | 8 wk | 25 wk | flyable 3D voxel world | -| M4 GPU | 10 wk | 35 wk | textured, lit, 60 fps | -| M5 interaction | 6 wk | 41 wk | **playable: walk, dig, place** | -| M6 definitions | 5 wk | 46 wk | modded content, ABMs | -| M7 persistence | 4 wk | 50 wk | worlds that survive restart | -| M8 mod ABI | 8 wk | 58 wk | **drop-in mods** | -| M9 multiplayer | 12 wk | 70 wk | two clients, one server | - -≈**70 developer-weeks** to a Luanti-class engine written in Ludic. For scale: -Luanti itself is 362,258 lines accumulated since 2010. - ---- - -## 7. Decisions I need from you - -These change the shape of the work and are yours to make, not mine. My -recommendation is given first in each case. - -1. **Numerics: `fixed64` *and* `f32`, or just `f32`?** - *Recommendation: both.* `fixed64` for gameplay/physics (deterministic - lockstep multiplayer is a real advantage, and Luanti's float physics is a - known source of desync); `f32` for noise and GPU vertex data, where it is - non-negotiable. - -2. **Mod distribution: source-and-compile-on-load, or prebuilt binaries?** - *Recommendation: source.* `ludicc` is 2,508 lines — shipping the compiler - with the game preserves the "drop a folder in" workflow that is Luanti's - entire ecosystem, keeps one language end-to-end, and needs no VM. - -3. **Mod sandboxing: accept native trust, or subprocess isolation?** - *Recommendation: accept it for M8, revisit before any public server.* - Luanti's `s_security.cpp` is 1,253 lines of Lua-specific confinement that has - no native analogue; subprocess + IPC is the only real answer and it is a - quarter of work on its own. - -4. **GPU: FFI to OpenGL, or hand-written IR per platform like `cocoa.ll`?** - *Recommendation: FFI.* It matches the project's existing posture toward - libraries, and `cocoa.ll` is 327 lines for *five* window functions — OpenGL - has hundreds of entry points. - -5. **Compatibility: read Luanti's `map.sqlite` and speak protocol 37+, or a - clean-room format?** *Recommendation: clean room, with a one-way importer.* - Wire compatibility means implementing 90 packet types before anything is - playable; it inverts the whole milestone order. - -6. **Scope: the full engine, or a Luanti-shaped game?** Worth being honest with - yourself here. M0–M6 (≈46 weeks) gets a genuinely good single-player voxel - game and, more importantly, **makes Ludic a language that could plausibly - build one.** M8–M9 are what make it *Luanti*, and they are half the budget. - ---- - -## 8. Appendix A — target state - -What `examples/voxel.ludic` would look like once M0–M6 land. This is the -document's thesis in one file: nothing here is expressible today, and all of it -is ordinary Ludic in shape. - -```ludic -game Voxel { - import "voxel/nodes.ludic" - import "voxel/worldgen.ludic" - - # ---- data --------------------------------------------------------------- - struct MapNode { param0: u16 = 0, param1: u8 = 0, param2: u8 = 0 } - struct v3i { x: int = 0, y: int = 0, z: int = 0 } - struct v3f { x: f32 = 0, y: f32 = 0, z: f32 = 0 } - - volume Map of MapNode chunk 16 { - origin v3i - meta { generated: bool = false, timestamp: u32 = 0 } - } - - component Body { pos: v3f, vel: v3f, aabb: aabb, on_ground: bool = false } - component Look { yaw: f32 = 0, pitch: f32 = 0 } - component Health { hp: int = 20, max: int = 20 } - component Inv { slots: [32]ItemStack } - - archetype Player { Body, Look, Health, Inv } - archetype Mob { Body, Look, Health } - - # ---- content ------------------------------------------------------------ - def node Stone { - name = "voxel:stone" drawtype = DrawType.Normal - tiles = ["stone.png"] groups = [("cracky", 3)] - is_ground_content = true drop = "voxel:cobble" - } - def node Water { - name = "voxel:water" drawtype = DrawType.Liquid - tiles = ["water.png"] liquid = Liquid.Source { alt_flowing = "voxel:water_flowing" } - walkable = false light_propagates = true - } - - # ---- generation, off the main thread ------------------------------------ - thread pool emerge count 4 { - job Generate(bp: v3i) -> Chunk { - manip vm = Map.stage(bp) - for p in vm.area { - let n = noise2d(p.x as f32 / 64.0, p.z as f32 / 64.0, seed()) - let h = floor(n * 24.0) + 8 - vm[p] = if p.y > h { NODE_AIR } - else if p.y == h { NODE_GRASS } - else { NODE_STONE } - } - return vm.finish() - } - } - - system Emerge phase Update { - for bp in Map.wanted(view_radius = 10) { emerge.submit(Generate(bp)) } - while let c = emerge.poll() { Map.install(c) relight(c) remesh(c) } - } - - # ---- simulation: an ABM as a first-class system ------------------------- - system LavaCooling phase Update - every 10.0s chance 1 in 50 - nodes [NODE_LAVA_SOURCE] near [NODE_WATER_SOURCE, NODE_WATER_FLOWING] - { Map[self_pos()] = NODE_OBSIDIAN } - - # ---- physics ------------------------------------------------------------ - system Physics phase FixedUpdate - query (b) [Body] - { - b.vel.y = b.vel.y - 9.81 * dt() - let r = collide_swept(Map, b.aabb, b.pos, b.vel * dt(), stepheight = 0.6) - b.pos = r.pos - b.on_ground = r.hit_floor - if r.hit_floor { b.vel.y = 0.0 } - } - - # ---- interaction -------------------------------------------------------- - system Dig phase Input - query (b, l) [Body, Look, {Player}] - { - when mouse_pressed(MOUSE_LEFT) { - match raycast(Map, eye(b), dir(l), 5.0) { - Pointed.Node(under, _) => { - let def = node_def(Map[under].param0) - give(self(), def.drop) - Map[under] = NODE_AIR - } - _ => {} - } - } - } - - # ---- render ------------------------------------------------------------- - scene InGame start { - layer World { - system DrawWorld phase Render { - gfx_begin(camera_of(local_player())) - for c in Map.visible(frustum()) { gfx_draw_mesh(c.mesh) } - gfx_end() - } - } - layer Hud { - system DrawHud phase Render { ui_render() } - } - } -} -``` - -**Read that against LANGUAGE.md.** The `system`/`query`/`phase`/`scene`/`layer` -/`archetype`/`match` skeleton is *unchanged* — every line of new capability is -either a type (`struct`, `f32`, `[]T`, `enum`) or one of four new constructs -(`volume`, `manip`, `thread pool`, `def`). That is the encouraging finding of -this whole exercise: **Ludic's programming model is already the right one for -this game. What is missing is underneath it, not around it.** - ---- - -## 9. Appendix B — what Ludic already has that Luanti had to build - -Not everything is a deficit. Ludic starts with several things Luanti spent years -acquiring, and they should be counted: - -| Ludic has | Luanti equivalent | Cost Luanti paid | -|---|---|---| -| ECS in the language (`component`/`query`/`system`/`phase`) | hand-rolled `ActiveObjectMgr` + ad-hoc containers | scattered across `serverenvironment.cpp` (2,091) + `activeobjectmgr` | -| `scene`/`layer` state machine | mode flags and `if` ladders in `game.cpp` | part of 3,815 LOC | -| `match`/`machine`/`become` | switch chains | — | -| Retained `ui` tree as data | formspec **string DSL** parsed at runtime | `guiFormSpecMenu.cpp`, **5,610 LOC** | -| TrueType engine written in-language | FreeType + `CGUITTFont.cpp` (1,003) | external dep | -| PNG + DEFLATE decode in-language | libpng + zlib | external deps | -| Deterministic RNG in the language | `PcgRandom`/`PseudoRandom` exposed to Lua | — | -| Compile-time type checking of game logic | Lua: runtime errors on a live server | the entire class of mod crashes | -| One language for engine *and* content | C++ engine + Lua mods + a 1,253-line sandbox | the whole `src/script/` tree, **34,240 LOC** | - -That last row is the strategic one. **A third of Luanti's engine — 34,240 lines -of `src/script/` plus 22,634 lines of `builtin/` Lua — exists solely to bridge -C++ and Lua.** A Ludic implementation with a compiled mod ABI (G-25) does not -need that bridge at all. Roughly 57,000 lines of Luanti are a tax that Ludic -would simply not pay. - ---- - -## 10. Verdict - -**Can Ludic implement Luanti today?** No — and not because of anything about -games. It is missing arrays, strings, structs, floats, threads and sockets. Any -language missing those cannot implement any large program. - -**Is Ludic's design wrong for Luanti?** No — and this is the more interesting -answer. The ECS-in-the-language model, phases, scenes, archetypes and `match` -are a *better* fit for a voxel sandbox than C++-plus-Lua is. The `system … every -10.0s chance 1 in 50 nodes […]` form (G-16) compiles to something Luanti's ABM -implementation cannot achieve, because Luanti's predicates arrive as strings at -runtime and Ludic's arrive at compile time. - -**What is the actual shortest path?** M0 and M1. Arrays, structs, strings, -floats. Twelve weeks that convert Ludic from a language that can express a -2D JRPG into one that can express an engine — after which every remaining gap in -this document is ordinary library work in Ludic, plus one hard call about mods -(G-25) and one large push on the GPU (G-28). - ---- - -*Generated from `luanti-org/luanti@main` (362,258 LOC surveyed) against -`ludicc` at `compiler/{ludicc,native,driver}.c` and `runtime/native/*.ludic`. -Every Ludic limitation cited was verified by reading the compiler or by -compiling a probe program, not inferred from documentation.* diff --git a/MOBILE-DESIGN.md b/MOBILE-DESIGN.md deleted file mode 100644 index db396153..00000000 --- a/MOBILE-DESIGN.md +++ /dev/null @@ -1,338 +0,0 @@ -# iOS & Android — a design doc - -> **Status: all design, nothing shipped.** Ludic builds windowed on macOS -> (`runtime/native/cocoa.ll`) and has a documented — but currently un-reimplemented -> — wasm32 web target. iOS and Android are not buildable today, and the -> cross-compile plumbing that would target them died with the C driver. This doc -> lays out the whole path so we can decide the shape before building any of it. The -> headline decision (§7): render on the **GPU via `extern fn` FFI**, not the CPU -> framebuffer. §11 lists the open decisions. - ---- - -## 1. Where we are - -A Ludic program compiles to LLVM IR, then clang assembles and links it. The -platform story has **two independent axes**, and it's essential not to conflate -them: - -| Axis | What it is | State today | -|---|---|---| -| **Target** (triple + toolchain) | how IR becomes a runnable binary for an OS/arch | barely plumbed — no `--target`, no emitted `target triple`, host-only | -| **Platform runtime** (window/input/present) | one file implementing the 5-function window protocol | well-factored — `cocoa.ll` is ~328 lines, swappable | - -**What exists:** - -- The window seam is exactly five functions — `win_open` / `win_poll` / - `win_present` / `win_running` / `win_close` — declared by the compiler - ([emit_head.ludic:58](selfhost/emit_head.ludic:58)) and lowered as intrinsics - ([emit_intrin2.ludic:39](selfhost/emit_intrin2.ludic:39)). The runtime calls them - through `rt_*` wrappers ([core.ludic:48](runtime/native/core.ludic:48), - [:103](runtime/native/core.ludic:103), [:217](runtime/native/core.ludic:217)). - `COMPILING.md` states the intent plainly: a new platform is "another `.ll` file - with the same five entry points and no compiler change." -- **`extern fn` FFI is real and live** — `extern function c_hypot(a: fixed, b: fixed) -> - fixed = "hypot_fx"` ([LANGUAGE.md:565](LANGUAGE.md:565)), with a full pipeline: - parse ([parse_game.ludic:236](selfhost/parse_game.ludic:236)) → call lowering to a - direct `call @` ([emit_expr.ludic:168](selfhost/emit_expr.ludic:168)) → - `declare` emission ([emit_head.ludic:105](selfhost/emit_head.ludic:105)). Working - examples: [examples/networking/net_echo.ludic:12](examples/networking/net_echo.ludic:12), - [examples/library/arena.ludic:14](examples/library/arena.ludic:14). This is the single - most important fact in this document — see §7. - -**What's missing (all of it must be built):** - -| Gap | Why mobile needs it | -|---|---| -| `--target ` flag + emitted `target triple`/`datalayout` | iOS = `aarch64-apple-ios`, Android = `aarch64-linux-android`; both are cross-compiles | -| per-target `size_t` width (i32/i64) | already a known wasm trap; every allocation sizing depends on it | -| **OS-owned frame loop** (`ludic_boot`/`ludic_frame`/`ludic_alive`/`ludic_teardown`) | iOS (CADisplayLink) and Android (Choreographer) own the loop — you cannot `while(alive)` | -| per-platform window shim + touch input | UIKit/`CAMetalLayer`, Android `Surface`/NDK; input is touch, not a keycode | -| SDK sysroot + packaging + signing | `.app` bundle / `.apk`, not a bare executable | - -The frame-loop gap is shared with the web target — `tools/ludic-web/run.mjs` -already expects `ludic_boot`/`ludic_frame`, but the self-hosted emitter only -produces a monolithic `@main` ([emit_game.ludic:685](selfhost/emit_game.ludic:685)). -So the wasm path is half-broken for the same reason mobile can't exist yet. - ---- - -## 2. Design principles - -1. **Two axes, kept separate.** "Add a platform" = a cross-compile *target* plus a - platform *runtime*. Muddling them is why this looks bigger than it is. Most of - the compiler work (§4, §5) is target plumbing that serves web, iOS, and Android - at once; the per-OS work (§6) is genuinely small by design. -2. **The OS owns the loop — so we must too.** Mobile, like the browser, forbids an - inline frame loop. Rather than special-case mobile, adopt the frame-driven model - *everywhere* the OS demands it, from one emitter change. This is the keystone. -3. **The GPU is an ABI to call, not a program to compile.** `extern fn` already - binds C libraries; bind GL ES / Metal the same way. No IR-per-API (the `cocoa.ll` - route — 328 lines for *five* functions), no per-symbol intrinsics. The roadmap - reaches this conclusion independently ([LUANTI-ROADMAP.md:1083](LUANTI-ROADMAP.md:1083), - [:1375](LUANTI-ROADMAP.md:1375)). -4. **The 2D stack stays byte-identical.** The framebuffer graphics - (`rt_fb` + all `rt_*`/`image`/`truetype`/`ui` primitives) keep working - unchanged. GPU rendering is *additive*: 2D composites as one texture on top of - GPU 3D. Nothing above the window seam is rewritten. - ---- - -## 3. Core model - -Everything below reduces to plumbing one new flag through the compiler and swapping -two runtime files per OS. The mental model: - -``` -ludicc app.ludic --target aarch64-apple-ios -o app - │ - ├─ emit_head: target triple / datalayout / size_t width (§4) - ├─ emit_game: ludic_boot/frame/alive/teardown not @main (§5) - ├─ link: runtime/ios/uikit.ll + gfx3d.ldylib (§6, §7) - └─ package: .app bundle + codesign (§8) -``` - -The game source and the entire ECS/graphics/UI stack compile **unchanged** for -every target. Only the head declarations, the entry-point shape, the linked -platform file, and the packaging step vary. - ---- - -## 4. Extension M1 — the target axis: `--target`, triple, `size_t` - -Today [main.ludic:66](selfhost/main.ludic:66) parses `--windowed`/`--headless`/ -`--emit-llvm`/… and nothing selects an arch; the IR carries no `target triple`, so -native inherits clang's host default and the only explicit triple in the tree is -`wasm32-unknown-unknown` ([runtime/web/wasm.ll:23](runtime/web/wasm.ll:23)). - -Proposal: a `--target ` flag that drives three things. - -``` -ludicc app.ludic --target aarch64-apple-ios -o app -ludicc app.ludic --target aarch64-apple-ios-simulator -o app # x86_64 host → arm64 sim varies -ludicc app.ludic --target aarch64-linux-android -o libapp.so -``` - -- **Emit the triple + datalayout.** `emit_header` - ([emit_head.ludic:37](selfhost/emit_head.ludic:37)) gains a `target triple = …` - / `target datalayout = …` line, chosen from a small table keyed on `--target`. - Absent the flag, emit nothing (host default) — keeps existing native builds - byte-identical. -- **Per-target `size_t` width.** wasm32 already needs `i32` sizes; the same helper - discipline (`ll_size_t`/`ll_widen`/`ll_narrow`, per the web-backend notes) applies - to any 32-bit target. iOS/Android arm64 are LP64 like macOS, so `i64` — but the - flag must *select* the width, not assume the host's. -- **Toolchain construction.** The linker command - ([main.ludic:130](selfhost/main.ludic:130)) becomes target-conditional: an SDK - sysroot (`-isysroot`/`--sysroot`), the platform `.ll`, and target-specific link - flags (§8). `$LUDIC_CC` still overrides; add `$LUDIC_SYSROOT_` for the - SDK path so CI and local machines can differ. - -This axis is **shared with reviving wasm** — do it once, three targets benefit. - ---- - -## 5. Extension M2 — the OS-owned frame loop (the keystone) - -A native build emits `@main` with the frame loop inline — an `rt_init`, then a -`loop:`/`done:` block calling `rt_poll`/`rt_running` -([emit_game.ludic:685](selfhost/emit_game.ludic:685)). **iOS and Android cannot run -this.** UIKit calls back into your code once per display refresh (CADisplayLink); -Android's Choreographer does the same; the browser's `requestAnimationFrame` already -does. In all three the OS owns the loop and calls *you*. - -Proposal: emit four exported functions instead of an inline-loop `@main`, exactly -as `COMPILING.md` already describes and `run.mjs` already expects: - -``` -ludic_boot() → rt_init (once) -ludic_frame() → rt_poll · systems · rt_present (per OS callback) -ludic_alive() → i1 → rt_running (OS asks: keep going?) -ludic_teardown() → rt_shutdown (once) -``` - -- **`@main` becomes the composed default, not the only shape.** For host desktop - and headless, the compiler synthesizes an `@main` that *calls* the four in an - inline loop — so native/headless output is unchanged in behavior. For - OS-owned-loop targets (`--target` is wasm/ios/android, or a new - `--loop=external` mode), emit only the four exports and no driving `@main`. -- **One emitter change, three targets fixed.** This simultaneously un-breaks the - web target (whose runner already calls these) and unlocks both mobile OSes. It is - the highest-leverage change in this doc. -- **State stays where it is.** The four functions close over the same globals - `rt_init`/`rt_poll`/`rt_running`/`rt_shutdown` already touch - ([core.ludic:48](runtime/native/core.ludic:48)); no new runtime state, no heap. - ---- - -## 6. Extension M3 — the per-OS window shim + touch input - -Each OS gets one platform file implementing the five-function seam, modeled on -`cocoa.ll` but rewritten for its UI toolkit. This is the part the codebase is -explicitly built for. - -- **iOS — `runtime/ios/uikit.ll` (or a thin `.m` shim).** `win_open` creates a - `UIWindow` + a `UIViewController` whose view is a `CAMetalLayer`/`MTKView`; - `win_present` presents the current drawable; the loop is driven by M2's - `ludic_frame` from a `CADisplayLink`, so `win_poll`/`win_running` adapt to the - callback model rather than a spin. Hand-written IR against `objc_msgSend` is - possible (it's how `cocoa.ll` works) but a small compiled `.m` linked in is more - maintainable for UIKit's larger surface — an open decision (§11). -- **Android — `runtime/android/ndk.ll` + a Kotlin/Java `Activity` host.** The - native code is a `.so` loaded by an `Activity`; the window is an - `ANativeWindow`/`Surface` obtained via `GameActivity`/NDK, GPU via EGL + GL ES. - Frames are driven by Choreographer through JNI into `ludic_frame`. -- **Touch input changes the input seam.** `win_poll()` returns a single `int` - keycode today ([emit_intrin2.ludic:41](selfhost/emit_intrin2.ludic:41), - [core.ludic:217](runtime/native/core.ludic:217)) — insufficient for touch, which - needs `(x, y, phase, id)`. Options: (a) a parallel `win_poll_touch() -> pointer` - draining an event queue, or (b) widen the input model to a small event struct for - all platforms. This is the one place mobile forces a decision above the window - seam. Proposed: add touch as a **separate** seam so keyboard platforms stay - untouched and byte-identical. - -Everything above the seam — framebuffer, PNG sprites, TrueType, retained UI — is -portable Ludic and compiles unchanged. - ---- - -## 7. Extension M4 — GPU rendering via `extern fn` (the headline) - -Today **all** drawing writes into one CPU framebuffer: `rt_fb`, a -`words(320*240)` buffer of `0x00RRGGBB` i32 pixels -([core.ludic:23](runtime/native/core.ludic:23)), written by every primitive -(`rt_clear`/`rt_fill_rect`/glyphs/`rt_blend_px`/`tt_blit`/UI) and handed whole to -`win_present`. `cocoa.ll` blits it through CoreGraphics — -`CGBitmapContextCreate`→`CGImage`→`CGContextDrawImage` inside `@ludic_drawRect` -([cocoa.ll:94](runtime/native/cocoa.ll:94)). There is no GPU context anywhere. - -Because **`extern fn` already exists**, binding the GPU is ordinary runtime code — -no new language feature, no new intrinsic: - -```ludic -# doc-check: skip — runtime/native/gfx3d.ludic, illustrative -extern function gl_gen_textures(n: int, out: pointer) -> void = "glGenTextures" -extern function gl_tex_image_2d(t: int, w: int, h: int, px: pointer) -> void = "gl_tex_image_2d" -extern function gl_draw_elements(mode: int, count: int, ty: int, idx: pointer) -> void = "glDrawElements" -``` - -Two phases, additive: - -1. **Framebuffer-as-texture (drop-in).** Keep the entire 2D stack. `rt_present` - ([core.ludic:103](runtime/native/core.ludic:103)) uploads `rt_fb` as one texture - and draws a full-screen quad. The `win_present(fb,w,h)` signature is unchanged; - only the pixel-delivery core of the platform file differs (texture upload instead - of CoreGraphics blit). This is the minimum viable GPU path and gets mobile on - screen with zero changes above the seam. -2. **True GPU 3D (additive).** Geometry goes straight to GL/Metal via `gfx3d.ludic` - `extern fn` calls; the CPU framebuffer is reused only for the 2D UI overlay, - composited as a texture on top. New GPU-draw entry points live in `gfx3d.ludic` - as `extern fn`s — the five-function window protocol does **not** widen. - -Language-level cost is narrow and already scoped by the roadmap: - -- **`f32`** (roadmap gate G-04) for vertex/matrix data — the *only* hard language - dependency ([LUANTI-ROADMAP.md:1087](LUANTI-ROADMAP.md:1087)). -- Optional vector operator overloading for `v3f`/`m4` ergonomics (G-29, - [:1107](LUANTI-ROADMAP.md:1107)) — a "nicer, not necessary." - -The roadmap's own decision is explicit: FFI over IR-per-API, because "`cocoa.ll` -is 327 lines for *five* window functions — OpenGL has hundreds of entry points" -([LUANTI-ROADMAP.md:1375](LUANTI-ROADMAP.md:1375)). - ---- - -## 8. Extension M5 — packaging, SDKs, and signing - -The current driver is one `clang` call ([main.ludic:130](selfhost/main.ludic:130)) -producing a bare binary. Mobile output is a bundle, and this is where most -real-world friction lives — it is deliberately the *last* phase. - -- **iOS.** Cross-compile with the iPhoneOS SDK sysroot → an executable, wrap in an - `App.app` bundle with an `Info.plist`, `codesign` with a development identity, - install to simulator/device. Simulator is the cheap inner loop - (`aarch64-apple-ios-simulator`); device needs a provisioning profile. ludicc - should emit the binary and shell a packaging step (or emit a manifest a small - script consumes), not learn Xcode's project format. -- **Android.** Cross-compile with the NDK → `libapp.so`, drop it into a minimal - Gradle/Kotlin `Activity` shell, build the `.apk`/`.aab`, sign with a keystore. - The `Activity` is fixed boilerplate that ships in the repo (`runtime/android/`), - parameterized by app name/id. -- **Keep the compiler out of it.** Both flows are "produce native code + assemble a - package around it." The compiler's job ends at the object/`.so`; a `--package` - step or an external `build-mobile.sh` owns the bundle. This mirrors how ludicc - already drives clang without becoming a build system. - ---- - -## 9. Lowering / build summary - -| Construct | Reduces to | -|---|---| -| `--target ` (M1) | a triple/datalayout line in `emit_header` + a `size_t`-width choice + target-conditional link command | -| OS-owned loop (M2) | emit `ludic_boot`/`ludic_frame`/`ludic_alive`/`ludic_teardown`; host/headless get a synthesized `@main` calling them | -| window shim (M3) | one `.ll`/shim per OS implementing the same five `win_*` intrinsics; no compiler change | -| touch input (M3) | a **new, separate** input seam (`win_poll_touch`), so keycode platforms stay byte-identical | -| framebuffer→texture (M4.1) | `rt_present` uploads `rt_fb` as a texture + full-screen quad; `win_present` signature unchanged | -| GPU 3D (M4.2) | `extern fn` calls in `runtime/native/gfx3d.ludic` — data in `prog`, zero compiler edits, needs only `f32` | -| packaging (M5) | binary/`.so` unchanged; an external `--package`/script builds `.app`/`.apk` and signs | - -No new allocator, no new dispatch, no per-API intrinsics. The game and the 2D -graphics stack compile identically for every target; only head declarations, the -entry-point shape, the linked platform file, and packaging vary. - ---- - -## 10. Suggested implementation phases - -Each is independently shippable and testable, matching how the repo phases work. - -- **M0 — target axis** (M1) + **revive the OS-owned loop** (M2). *Do these first - and together* — they're the shared compiler plumbing, they un-break the existing - web target (proving the frame-loop split against `run.mjs`/`bin/x test` before any - mobile SDK is involved), and they need no mobile toolchain. This is the floor. -- **M1 — iOS simulator, framebuffer-as-texture** (M3 iOS shim + M4.1). First pixels - on a phone, GL/Metal binding proven, no signing/device friction yet. -- **M2 — iOS device** (M5 iOS packaging + signing). -- **M3 — Android** (M3 Android shim + M4.1 + M5 Android packaging), reusing every - M0 change. -- **M4 — `f32` + GPU 3D** (M4.2), gated on roadmap G-04; the additive 3D path over - `gfx3d.ludic`. -- **M5 (later) — touch-input model** hardening (M3), gesture/multitouch, once a real - app exercises it. - -M0 is the honest prerequisite and the highest-leverage work — it serves three -targets and revives a fourth. M1 is the first thing anyone can *see*. - ---- - -## 11. Open decisions - -1. **Loop selection:** does `--target ios/android/wasm` *imply* the external loop, - or is there an explicit `--loop=external` flag? (Proposed: implied by target, - with the flag as an override for headless testing.) -2. **iOS shim language:** hand-written `.ll` against `objc_msgSend` like `cocoa.ll`, - or a small compiled `.m`? (Proposed: `.m` — UIKit's surface is too large for - maintainable IR, and Metal setup is verbose.) -3. **Touch seam shape:** a separate `win_poll_touch` queue, or a unified event - struct replacing the keycode `win_poll` on all platforms? (Proposed: separate, - to keep desktop/web byte-identical.) -4. **GPU API baseline:** GL ES 3.0 everywhere (Android native, iOS via ANGLE/Metal - translation), or Metal on iOS + GL ES on Android from day one? (Proposed: GL ES - 3.0 first for a single codepath; Metal later.) -5. **Android host:** ship a fixed Kotlin `GameActivity` in `runtime/android/`, or - generate it per app? (Proposed: fixed boilerplate, parameterized by name/id.) -6. **Packaging home:** a `--package` step inside ludicc, or an external - `build-mobile.sh`? (Proposed: external script; keep the compiler out of bundle - formats.) -7. **`size_t` for arm64:** confirm iOS/Android arm64 are LP64 (`i64`) in the width - table, and that the `ll_size_t` discipline covers every new size-taking call. -8. **Simulator arch:** how to handle `aarch64-apple-ios-simulator` vs. x86_64 sim on - Intel hosts in the target table. - ---- - -*Companion to [COMPILING.md](COMPILING.md) (§ toolchain, the wasm frame-loop -split), [LANGUAGE.md §"Functions & FFI"](LANGUAGE.md:560) (`extern fn`), and -[LUANTI-ROADMAP.md](LUANTI-ROADMAP.md) (G-04 `f32`, G-28 GPU FFI, G-29 3D math). -Supersedes nothing until the M0 compiler work lands.* diff --git a/NETWORKING-DESIGN.md b/NETWORKING-DESIGN.md deleted file mode 100644 index a533c1ce..00000000 --- a/NETWORKING-DESIGN.md +++ /dev/null @@ -1,505 +0,0 @@ -# Networking, from primitives up — a design doc - -> **Status: N0–N6 all shipped.** The whole stack is implemented and self-hosted, -> and — unlike the original N0/N1 which linked C hosts — every phase now runs as a -> self-contained **pure-Ludic** program (no `.c`, no foreign host): a built-in -> loopback transport fills the seam, and each `examples/net_*.ludic` drives and -> asserts itself from its own `entry`. See `bin/x test` (checks `net_echo` … `net_demo`) -> and `examples/networking/net_demo.ludic` for a full RPC→authority→replicate→reconcile loop. -> clang remains only as the LLVM-IR assembler/linker (no C is compiled), the floor -> Rust and Swift stand on. -> -> _Historical note:_ **N0 + N1 shipped first; N2–N6 were design.** Two phases landed as `bin/x test` -> checks. **N0 (transport seam):** `extern fn` now lowers end to end — a direct -> `@` call plus a `declare`, no networking logic in the compiler — so the whole -> transport is two externs (`net_send`/`net_poll`) a host fills. Proven by -> [`examples/networking/net_echo.ludic`](examples/networking/net_echo.ludic) sending four bytes through -> the loopback host in [`tests/net_c/loopback.c`](tests/net_c/loopback.c) and -> polling them back (`4 10 20 30 42`). **N1 (snapshot-to-buffer):** -> `world_size()`/`world_save(buf)`/`world_load(buf, len)` generalize `save()`/`load()` -> from a file to a caller-owned memory buffer — the same block layout via `memcpy` — -> so the whole ECS world round-trips through bytes. Proven by -> [`examples/networking/net_snapshot.ludic`](examples/networking/net_snapshot.ludic) + -> [`tests/net_c/snapshot_mod.c`](tests/net_c/snapshot_mod.c) (snapshot, mutate, -> restore → `50 7 50`). Both are byte-identical when unused, so the offline dividend -> (§8) holds. This is a companion to -> [EVENTS-DESIGN.md](EVENTS-DESIGN.md), [LIFECYCLE-DESIGN.md](LIFECYCLE-DESIGN.md), -> and [SCENES-DESIGN.md](SCENES-DESIGN.md). Where the events work made Ludic -> *moddable*, this proposes making it *networked* — and it deliberately does **not** -> ship a multiplayer framework. Ludic is a language: it exposes the low-level -> mechanism (transport seam, world snapshot, generated serializers, ownership, a -> drivable sim) and a thin high-level *declarative* layer that lowers onto that -> mechanism, and it leaves the netcode *policy* (authority, prediction, relevancy) -> to the developer or a library. §14 lists the open decisions. - ---- - -## 1. Thesis - -Every networking model dies on one of two problems: **determinism** or **state -serialization**. Ludic already solves both, almost by accident. - -- **Determinism** is designed in — seeded RNG, `fixed` (Q16.16) instead of floats, - byte-identical golden renders, and (as of [EVENTS-DESIGN EV6](EVENTS-DESIGN.md)) - bounded, array-ordered event dispatch. A modded, event-driven Ludic game still - replays identically. That is exactly the property lockstep multiplayer needs, and - the reason Factorio's heavily-modded multiplayer stays in sync. -- **State serialization** already exists — `save()`/`load()` snapshot the *entire* - ECS World to a byte buffer ([`selfhost/emit_save.ludic`](selfhost/emit_save.ludic)), - and the world-table schema built for [EVENTS-DESIGN EV2](EVENTS-DESIGN.md) (prop → - field → offset) is exactly the descriptor you serialize against. - -So networking is not a new subsystem. It is a **fourth lens on the event + world -layer** — the same layer modding used. And it obeys the same two-altitude rule as -everything else in Ludic: - -> **Low-level is freedom; high-level is developer experience; they are the same -> feature at two altitudes.** `@Queries` lowers to a query loop, `scene` lowers to a -> machine, `@Public @OnSpawn` lowers to `emit`. Networking's high-level annotations -> lower to a transport seam, generated serializers, and a drivable sim — and the -> primitives stay exposed underneath for anyone the sugar doesn't fit. - -The developer writes **one simulation**, declares *what* replicates, *who* owns -each entity, and *where* each handler runs — and never branches on `is_server()` -in ordinary code. The compiler lowers the declarations; a networking *runtime* -(the seam-filler, like `rt_*` for windowing) supplies the transport and the tick. - ---- - -## 2. Two altitudes, one system - -| Altitude | Who writes it | Surface | -|---|---|---| -| **High-level (DX)** | the developer, declaratively | `@Sync` (field/property/model), `@Owned`, `@Server`/`@Predicted`, directional remote events | -| **Lowering** | the compiler | per-model serializers, role-guarded dispatch, remote-event send/recv, ownership storage | -| **Runtime seam** | a networking library (blessed or custom) | binds the socket, sets `role`, drives the replication tick | -| **Low-level (freedom)** | power users, when the sugar doesn't fit | `net_send`/`net_poll`, `world_save`/`world_load`, generated `serialize_*`/`apply_*`, `owner()`, the drivable sim | - -Everyone lives at the top row for normal games; the bottom row stays open for -someone building something no framework could express. The split that keeps this a -*language* and not a *framework*: **annotations and their lowering are the language; -the replication driver and the transport are a library.** It is precisely the -events story — `@On`/`emit` are the language, the *modding system* is library code — -applied again. - ---- - -## 3. Research digest — the one idea to steal from each - -| System | The transferable idea | -|---|---| -| **Quake / QuakeWorld** | The founding pattern: **client-side prediction + server reconciliation**, and delta-compressed snapshots against the last acked baseline. Predict locally, correct from the authority. | -| **Source (Valve)** | **Entity interpolation** (render remote entities slightly in the past, smoothly) paired with **lag compensation** (the server rewinds to the shooter's view for hit detection). Interpolation and rewind are two halves of one clock discipline. | -| **Unity NGO** (GameObject) | `NetworkVariable` with **read/write permissions** + `OnValueChanged`; ownership as `OwnerClientId`. Also the **anti-pattern to avoid**: `IsServer`/`IsOwner` branching sprinkled through gameplay code. | -| **Unity Netcode for Entities** (ghosts) | The model Ludic is closest to: **replication is a compile-time property of components and fields** — `[GhostField]`, `[GhostComponent]`, `GhostOwner`, and `Predicted`/`Interpolated` ghost modes — with serializers *generated* from the ECS schema. | -| **Mirror / FishNet** | The community-ergonomic take: `SyncVar` with change **hooks**, and clean **directional RPCs** — `Command` (client→server) / `ClientRpc` (server→clients). | -| **GGPO / rollback** | Save state → predict → on misprediction **restore and re-simulate**. Its one hard requirement is *cheap, complete state snapshot/restore* — which Ludic already has in `save()`/`load()`. | -| **Factorio** | Fully **deterministic lockstep** for heavy mod multiplayer: only *inputs* cross the wire; the whole sim is reproduced. Proof that determinism (EV6) is the enabler, not a nicety. | -| **Photon Quantum** | A shipping product that *is* deterministic-ECS-rollback. Validates the exact combination — ECS + determinism + rollback — Ludic is already positioned for. | -| **Roblox** | The **local/remote split** (`BindableEvent` vs `RemoteEvent`), server-authority by default, and engine-replicated properties: "some state just replicates, and RPCs are directional events." | - -Six **footguns** the survey warns against, to design *out* from the start: - -1. **Role branching everywhere.** `if (IsServer)` scattered through gameplay is the - NGO readability tax. Fix: **role is a handler annotation** (`@Server`/`@Predicted`), - never a runtime branch in ordinary code. -2. **Float nondeterminism.** Lockstep breaks the instant the networked sim touches - `f32` across platforms. Fix: the determinism contract (§11) — the networked sim - stays `int`/`fixed`. -3. **Replicating pointers / heap refs.** A `ptr` field holds a machine-local - address; it cannot cross the wire. Fix: **the compiler rejects `@Sync` on a - non-POD-scalar field** — a checked guarantee, not a convention. -4. **Sending everything every tick.** Fix: `@Sync` is **opt-in at the field level** - (only marked fields replicate), plus change-driven dirty tracking (`@OnChange`, - [LIFECYCLE LC2](LIFECYCLE-DESIGN.md)) so an unchanged field costs nothing. -5. **Hidden authority.** Magic "the server decides" behavior is unclear and - unauditable. Fix: **explicit** `@Server`/`@Predicted`; unmarked code runs - everywhere by definition. -6. **Schema-less snapshots.** A raw state blob with no version desyncs silently on a - version mismatch. Fix: the **world-table schema is the versioned descriptor** the - serializer is generated against. - ---- - -## 4. What Ludic already has - -The substrate is unusually complete for an engine that has never networked: - -- **A deterministic simulation** — seeded RNG, `fixed` math, ordered ECS iteration, - EV6-bounded event dispatch. Lockstep's precondition. -- **World snapshot/restore** — `save()`/`load()` serialize the whole World - ([emit_save.ludic](selfhost/emit_save.ludic)); today to a file, trivially - retargetable to a memory buffer. Rollback's precondition. -- **A reflective world table** — `ludic_get`/`set`/`has`/`query`/`register_prop` - and the prop→field→offset schema (EV2/EV2b). The apply-and-serialize substrate. -- **An event bus with a foreign ABI and POD payloads** (EV0). Directional remote - events (RPCs) are one flag on this. -- **The `rt_*` seam pattern** — the compiler already emits calls to - `rt_init`/`rt_poll`/`rt_present` that a runtime library fills. Networking's - transport and role registers plug into the identical seam. - -What is missing is small and named: a transport seam, snapshot-to-*buffer*, -generated per-field serializers, ownership storage, role-guarded dispatch, and a -developer-drivable loop. Each is a phase in §13. - ---- - -## 5. The low-level primitives (the freedom layer) - -Unopinionated, composable, host- or developer-owned. A power user builds any model -directly from these; the high-level layer (§6) is sugar over them. - -| Primitive | Signature (sketch) | Enables | -|---|---|---| -| **Transport seam** | `extern function net_send(peer: int, buf: pointer, len: int)` · `extern function net_poll(buf: pointer, cap: int) -> int` | any model; host binds UDP (native) or WebRTC/WebSocket (wasm), or a loopback for tests | -| **World snapshot ↔ buffer** | `world_save(buf: pointer) -> int` · `world_load(buf: pointer, len: int)` | rollback, replication, join/resync — generalizes `save()`/`load()` off the filesystem | -| **Generated serializers** | `serialize_(e: entity, buf: pointer) -> int` · `apply_(e: entity, buf: pointer, len: int)` | per-model, touch only the `@Sync` fields; emitted from the schema | -| **Ownership** | `owner(e: entity) -> int` · `set_owner(e: entity, id: int)` | authority checks, per-entity owner metadata (an `@L_owner` array, like `@L_kind`) | -| **Role registers** | `is_server() -> bool` · `is_owner(e: entity) -> bool` · `local_id() -> int` | the runtime sets these; role-guarded dispatch reads them | -| **Drivable sim** | `tick_fixed()` · `tick_render()` · seed get/set | a developer-owned loop for prediction/rollback (also: replay, headless tests, AI) | -| **Remote-event serde** | `emit`-site serialize + `net_send`; inbound bytes rebuild + re-`emit` | RPCs | - -Transport is the one that needs *no* language work at all — a developer can already -`extern fn` a socket library and link it, exactly as the windowing layer is linked. -The language's genuine contributions are snapshot-to-buffer, the generated -serializers, ownership storage, and the drivable loop. - -```ludic -# doc-check: skip — the freedom layer, a hand-rolled replication tick -entry { - while running() { - if is_server() { - for (Transform) in query [Transform, Owned] { - let n = serialize_Player(self(), buf) # compiler-generated - net_send(ALL, buf, n) # developer's transport - } - } else { - let n = net_poll(buf, CAP) - if n > 0 { apply_Player(target_of(buf), buf, n) } - } - tick_render(); present() - } -} -``` - -This *works*, but it is deliberately not how most games should be written — it puts -serialization and role branching in the developer's face. That is what §6 fixes. - ---- - -## 6. The high-level DX layer (the default) - -The developer declares **what** replicates, **who** owns, and **where** handlers -run. No serialization, no transport, no `is_server()` in ordinary code. - -### 6.1 `@Sync` — what replicates, at three granularities - -Replication is **opt-in at the field level**: a field crosses the wire only when it -is explicitly marked. There is no `@NoSync` — the surface is purely additive. - -Two independent switches, and **both must be on** for a field to replicate: - -1. **A field is *replicable*** iff it is `@Sync`-marked — directly - (`@Sync hp: int`), or via `@Sync property P { … }` (a shorthand that marks - *every* field of `P` replicable). *Only marked fields — never all-by-default.* -2. **A component *participates* in a model** iff the model marks it `@Sync` - (`@Sync Transform` inside the `model`). Participation is decided **per model - use-site**, so the same property syncs in one model and not another. - -A field of an entity replicates **iff it is replicable AND its component -participates in that entity's model.** - -```ludic -# doc-check: skip — the three levels -@Sync property Position { x: int, y: int } # every field of Position is replicable -property Health { @Sync hp: int, max: int } # only hp is replicable; max never is -property Transform { @Sync x: int, @Sync y: int, angle: int } # x, y replicable; angle not - -@Owned model Player { # entities carry a network owner - @Sync Transform # participates → replicates x, y (not angle) - @Sync Health # participates → replicates hp (not max) - @Sync Position # participates → replicates x, y -} - -model Prop { # a non-owned decoration - Transform # not @Sync here → Transform does NOT replicate — the - # "non-synced Transform sometimes" case, for free -} -``` - -- **Checked, not silent.** `@Sync` on a `ptr`/non-POD-scalar field is a **compile - error** ("networked fields must be POD scalars" — footgun 3). A model that - `@Sync`es a component with *zero* replicable fields is a **compile warning** - (participation that replicates nothing). -- **Per-field direction** rides the same annotation as an argument, mirroring how - `@Queries(these:…, on:…)` takes args: `@Sync(to: owner) hp: int` replicates a - field only to the entity's owner (Unity's `SendToOwner`). Default is `to: all`. - -### 6.2 Roles — where a handler runs - -The role is a **declarative annotation on the handler**, never a runtime branch. -Unmarked code is the shared, deterministic simulation and runs everywhere. - -| Annotation | Runs where | Meaning | -|---|---|---| -| *(none)* | everywhere | shared, deterministic simulation | -| **`@Server`** | the authority only | server-authoritative logic; clients receive the result via `@Sync` | -| **`@Predicted`** | the owning client (speculatively) **and** the server (authoritatively) | responsive local control, auto-reconciled against the server | - -`@Predicted` is **explicit** — the developer opts an owned entity's control handlers -into prediction; the language does not silently predict. The name states the netcode -role (owner-predicts + server-authoritative + reconcile), not the machine, and -matches Unity's `GhostMode.Predicted` so the concept transfers. - -`@Interpolated` — how a *non-owned* synced component is smoothed between snapshots on -a remote client — is a **presentation** concern on the component, kept separate from -these sim-handler roles rather than muddying them. - -### 6.3 Ownership - -```ludic -# doc-check: skip -@Owned model Player { @Sync Transform; @Sync Health } # every Player entity has a network owner -``` - -`@Owned` gives the model an owner slot (the `@L_owner` array); `owner(e)` / -`set_owner(e, id)` read and assign it (the authority assigns). `is_owner(e)` and -`@Predicted` dispatch read it. Ownership gates who may write `@Sync(to: owner)` -fields and who runs `@Predicted` handlers. - -### 6.4 RPCs are directional remote events - -RPCs are the event bus with a direction flag — no new concept: - -```ludic -# doc-check: skip -@ToServer event Fire { dir: int } # client → server (a request) -@ToClients event Boom { x: int, y: int } # server → clients (a broadcast) - -@Server @On(Fire) handler DoFire { spawn Bullet { dir: Fire.dir } } # authority handles the request -@On(Boom) handler Vfx { spawn Explosion { x: Boom.x, y: Boom.y } } # every client reacts -``` - -`@ToServer`/`@ToClients` mark an `event` remote; the compiler serializes its POD -payload (already flat — [EVENTS-DESIGN EV0](EVENTS-DESIGN.md)) and routes it through -the transport seam in the declared direction, re-`emit`ting it on the far side into -the ordinary event dispatch. - -### 6.5 The whole game, high-level - -```ludic -# doc-check: skip — read top to bottom: you always know where each line runs -program Shooter { - @Sync property Position { x: int, y: int } - property Health { @Sync hp: int, max: int } - - @Owned model Player { @Sync Position; @Sync Health } - model Bullet { Position } - - handler Physics phase FixedUpdate { … } # no tag → shared, identical everywhere - @Predicted handler Move phase Input { … } # owner predicts, server authoritative - @Server handler Death phase Update { … } # authority only; clients get the result via @Sync - - @ToServer event Fire { dir: int } - @Server @On(Fire) handler DoFire { spawn Bullet { … } } -} -``` - -No `is_server()`, no `net_send`, no serializer — yet every line's role is legible, -and every replicated field is explicitly opted in. - ---- - -## 7. Lowering summary - -Everything above reduces to the §5 primitives, gated so an un-networked build is -unchanged: - -| High-level | Lowers to | -|---|---| -| `@Sync` field / `@Sync C` in a model | a per-model `serialize_` / `apply_` over the replicable-and-participating fields, + a `sync manifest` a runtime reads | -| `@Sync(to: owner)` | a field tag in the manifest; the serializer branches on `owner(e) == peer` | -| `@Owned` | an `@L_owner` array + `owner()`/`set_owner()`, like `@L_kind` | -| `@Server` / `@Predicted` handler | the handler's dispatch wrapped in a role guard the runtime's role register drives (the `rt_*` seam pattern) | -| `@ToServer` / `@ToClients event` | payload serialize + `net_send(direction, …)` at the `emit` site; inbound bytes rebuild + re-`emit` | -| `world_save`/`world_load` to buffer | the existing `save()`/`load()` snapshot machinery, retargeted from a file handle to a memory buffer | -| drivable `tick_fixed`/`tick_render` | the phase runners the compiler already generates for the frame loop, exposed as callables when a game owns its `entry` loop | - -No heap, no hidden runtime beyond the honestly-named transport/role seams a -networking library fills — the same relationship windowing already has. - ---- - -## 8. The offline dividend - -Because these are **opt-in-cost annotations** — serializers *generated*, nothing -*run* until a networking runtime is spliced — a build with no runtime is -**byte-identical to single-player**, and every role guard collapses to "run here." -You build the game offline, drop in a runtime, and the same annotated code starts -replicating. That is Unity's "offline mode adjustable," achieved by the same -opt-in-cost invariant the whole event system already holds. - ---- - -## 9. The one genuinely hard corner - -Determinism holds beautifully for `int`/`fixed` simulations, which makes lockstep -and rollback cheap. It **breaks for `f32` across platforms** — so **3D/voxel + -lockstep stays the hard corner** (3D wants floats; the Luanti analysis flagged that -`fixed` saturates at ±32768). No language sleight-of-hand fixes this; the -determinism contract (§11) states it plainly, and a developer choosing lockstep for -a 3D game has to accept it (or choose state replication, §10's other branch, where -per-frame determinism is not required). - ---- - -## 10. Two model families, both reachable — neither built in - -The language commits to **neither**; both are library policy over the §5 primitives. - -- **Deterministic lockstep / rollback** — exchange only inputs; reproduce the sim; - on misprediction, `world_load` a snapshot and re-`tick_fixed`. Plays to Ludic's - determinism, and GGPO-cheap because snapshot/restore already exists. Best for - 2D/integer/fixed games. -- **State replication** — the authority `world_save`s (or per-`@Sync` serializes), - delta-encodes against the last acked snapshot per peer, ships the diff; peers - `apply_*` it and interpolate/predict. Heavier, but needed when the sim can't be - deterministic (float physics, 3D). - -A **blessed reference runtime** (§13, N6) can ship one of these so `@Sync` games -work out of the box — the way [`tests/mod_c/mod.c`](tests/mod_c/mod.c) proved the -event ABI — while the seams stay open for others. - ---- - -## 11. The determinism contract (what the language must guarantee) - -For a developer to *trust* lockstep, the language must promise, document, and where -possible *enforce*: - -1. **`fixed`/`int` math is bit-identical across platforms.** The networked sim must - avoid `f32` (footgun 2). *(Enforcement: at least a documented rule; ideally a - `@Sync`/`@Server`-reachable-code float lint.)* -2. **ECS iteration order is stable** — query order is declaration/id order, and - EV6 already fixes event-dispatch order. No hash-map iteration in the sim path. -3. **RNG is deterministic from a shared seed** — `seed()` exists; the seed must be - synchronized at session start (library policy) and never re-seeded from - wall-clock mid-sim. -4. **Networked components are POD scalars** — no `ptr`/heap fields cross the wire - (footgun 3). *Enforced:* `@Sync` on a non-scalar field is a compile error. -5. **Entity ids agree across peers** — lockstep gets this free from determinism; - replication needs an id-mapping table (library policy). - -This contract is the language's real networking responsibility. Most of it is -*already true*; the work is stating and enforcing it, not inventing it. - ---- - -## 12. Design principles - -1. **Mechanism in the language, policy in the library.** Expose serializers, - transport seam, ownership, snapshot, drivable sim. Never bake in authority, - prediction, or matchmaking. -2. **Role is declared, not branched.** `@Server`/`@Predicted` on handlers; unmarked - code runs everywhere. No `is_server()` in ordinary gameplay. -3. **Replication is explicit and opt-in.** Only `@Sync`-marked fields cross the - wire; participation is decided per model. Nothing replicates by surprise. -4. **Opt-in cost.** Un-networked builds are byte-identical; the sim runs offline - with the same code. -5. **Determinism is a promise the language keeps.** Enforce the POD-scalar rule; - document the float/iteration/seed rules; keep the sim reproducible. -6. **Two altitudes, always.** The high-level lowers to primitives that stay - callable. The sugar is the default; the freedom layer is never removed. -7. **Reuse, don't reinvent.** Snapshot = generalized `save()`; RPC = directional - `event`; serializer = generated from the EV2 schema; role seam = the `rt_*` - pattern. Networking is the fourth lens, not a parallel stack. - ---- - -## 13. Suggested implementation order - -Each phase is independently shippable and testable, matching how the repo phases -work (and how EVENTS-DESIGN sequenced EV0–EV7). - -- **N0 — transport seam + loopback. ✅ SHIPPED.** The `net_send`/`net_poll` extern - seam and a loopback host stub; an echo test. The floor; needed almost no compiler - work — just finishing `extern fn`: a call lowers to a direct `@` call and the - header emits a matching `declare`, so any C/Rust/Zig library (a socket, here the - loopback) binds through the same seam windowing uses. `find_extern` (emit_core), - the extern branch in emit_expr's call path, `emit_extern_decls` (emit_head). - ([`examples/networking/net_echo.ludic`](examples/networking/net_echo.ludic), - [`tests/net_c/loopback.c`](tests/net_c/loopback.c) → `4 10 20 30 42`.) -- **N1 — snapshot-to-buffer. ✅ SHIPPED.** Generalized `save()`/`load()` to a memory - buffer: `world_size()` (exact snapshot bytes), `world_save(buf) -> int`, - `world_load(buf, len)`. The same fixed block list (entity count, freelist, alive, - kind, vars, per-component `@S_`/`@H_`) now feeds a file (fwrite/fread) *or* a buffer - (memcpy over a threaded i64 offset), chosen by `g_snap_mode` in emit_save.ludic; - no rt_ hook (the ECS world only). The rollback/replication substrate. - ([`examples/networking/net_snapshot.ludic`](examples/networking/net_snapshot.ludic), - [`tests/net_c/snapshot_mod.c`](tests/net_c/snapshot_mod.c) → `50 7 50`.) -- **N2 — `@Sync` codegen. ✅ SHIPPED.** The three-level annotations → generated - per-model `serialize_`/`apply_` + by-kind dispatchers (`ludic_serialize`/ - `apply`/`sync_size`, and the `serialize`/`apply`/`sync_size` builtins); the - POD-scalar compile error and the empty-participation warning. The declarative - core. ([`examples/networking/net_sync.ludic`](examples/networking/net_sync.ludic) → `12 3 4 50 999`, - emit in [`selfhost/emit_net.ludic`](selfhost/emit_net.ludic).) -- **N3 — ownership. ✅ SHIPPED.** `@Owned` + the `@L_owner_arr` array + - `owner()`/`set_owner()`/`is_owner()`; owners are part of the world snapshot. - ([`examples/networking/net_owner.ludic`](examples/networking/net_owner.ludic) → `-1 7 0 1`.) -- **N4 — remote events (RPCs). ✅ SHIPPED.** `@ToServer`/`@ToClients` on `event`s → - payload serialize (`[event id][fields]`) + directional `net_send` + `net_pump()` - far-side re-`emit`. ([`examples/networking/net_rpc.ludic`](examples/networking/net_rpc.ludic) → `0 8`.) -- **N5 — roles + drivable sim. ✅ SHIPPED.** `@Server`/`@Predicted` role-guarded - dispatch driven by the `@L_role` register (`set_role`/`is_server`/`local_id`); - the opt-in `entry`-owns-the-loop with `tick_fixed()`/`tick_render()`. Together - these let prediction/rollback be written in developer/library code. - ([`examples/networking/net_roles.ludic`](examples/networking/net_roles.ludic) → `1 102`.) -- **N6 — a blessed reference netcode runtime. ✅ SHIPPED.** A Ludic library - ([`examples/networking/net_rt.ludic`](examples/networking/net_rt.ludic)) — server-authoritative state - replication over the primitives — plus a full end-to-end demo, proving the seams - the way the C mod proved the event ABI, but in pure Ludic over the built-in - transport. Library policy, swappable for lockstep+rollback. - ([`examples/networking/net_demo.ludic`](examples/networking/net_demo.ludic) → `5 999 5`.) A built-in - loopback transport (N0) means all of this needs **no foreign code at all**. - -N0–N2 deliver "state can be declared, serialized, and moved." N3–N4 add ownership -and RPCs. N5 unlocks prediction. N6 is a batteries-included default that others can -replace. The **determinism contract (§11)** is cross-cutting — documented from N0, -enforced incrementally. - ---- - -## 14. Open decisions - -1. **Field direction vocabulary.** `@Sync(to: owner)` / `@Sync(to: all)` confirmed - in spirit; is `to:` the right key, and do we also want `to: server` (a field only - the authority reads)? How does per-field direction interact with `@Predicted`? -2. **Blessed runtime, or seams only?** Events chose "seams + reference mod, bless - nothing." Networking's DX may justify shipping one reference runtime (N6). One, - or none? -3. **Authority default.** Server-authoritative with `@Predicted` opt-in is the safe, - Unity-ish default. Confirm, or keep the language authority-neutral and leave even - that to the runtime? -4. **Drivable loop shape.** Whole-frame `tick()` vs the `tick_fixed()`/`tick_render()` - split; how a developer-owned `entry` loop coexists with scenes, the `rt_*` hooks, - and the auto-loop (opt-in via presence of an `entry` block?). -5. **Snapshot granularity.** Full `world_save` vs per-`@Sync` serialize vs a - generated delta between two snapshots — which does the language provide, and which - is library work? -6. **Float determinism enforcement.** A documented rule only, or a real lint that - flags `f32` reachable from `@Server`/`@Predicted`/`@Sync` code paths? -7. **Ownership at component granularity.** Unity's DOTS allows per-component owner - send-rules. Is `@Owned` per-*entity* enough, or do we need per-component owners - (a real complexity jump)? -8. **Networking substrate for the remote half of EVENTS EV7.** This doc's directional - remote events (N4) *are* the local/remote split EVENTS-DESIGN EV7 deferred for - "no networking substrate." N4 is that substrate — the two docs meet here. - ---- - -*Companion to [EVENTS-DESIGN.md](EVENTS-DESIGN.md) (remote events are directional -events; serializers reuse the EV2 world-table schema; EV7's deferred local/remote -split lands here as N4), [LIFECYCLE-DESIGN.md](LIFECYCLE-DESIGN.md) (`@OnChange`/LC2 -is the dirty-tracking primitive for delta replication), and -[SCENES-DESIGN.md](SCENES-DESIGN.md). Supersedes nothing until the compiler work in -§13 lands.* diff --git a/README.md b/README.md index 5b77bff0..84017d1f 100644 --- a/README.md +++ b/README.md @@ -81,10 +81,17 @@ bin/x help # every command | [`docs/`](docs/) | the per-symbol API reference, regenerated into the docs site. | | [`COMPILING.md`](COMPILING.md) | the native pipeline: `ludicc → LLVM IR → exe`, the `rt_*` runtime protocol, and the (pending) wasm/cross-compile/shared-library paths. | -Design and roadmap documents — `LANGUAGE.md`, `EVENTS-DESIGN.md`, -`NETWORKING-DESIGN.md`, `SCENES-DESIGN.md`, `LIFECYCLE-DESIGN.md`, -`SYNTAX-REDESIGN.md`, `MOBILE-DESIGN.md`, `LUANTI-ROADMAP.md`, `BOOTSTRAP.md` — -live at the repository root today and are being migrated to the wiki. +The design and roadmap material lives on the **[wiki](https://git.workshopsoft.io/workshopsoft/ludic/wiki)**: +the [Events](https://git.workshopsoft.io/workshopsoft/ludic/wiki/Design/Events), +[Networking](https://git.workshopsoft.io/workshopsoft/ludic/wiki/Design/Networking), +[Scenes](https://git.workshopsoft.io/workshopsoft/ludic/wiki/Design/Scenes), +[Lifecycle](https://git.workshopsoft.io/workshopsoft/ludic/wiki/Design/Lifecycle), +[Mobile](https://git.workshopsoft.io/workshopsoft/ludic/wiki/Design/Mobile) and +[Syntax-redesign](https://git.workshopsoft.io/workshopsoft/ludic/wiki/Design/Syntax-Redesign) +design records, the [Bootstrap deep-dive](https://git.workshopsoft.io/workshopsoft/ludic/wiki/Bootstrap), +and the [Luanti roadmap](https://git.workshopsoft.io/workshopsoft/ludic/wiki/Roadmap/Luanti). +The root keeps only this README plus the two user-facing references, +[`LANGUAGE.md`](LANGUAGE.md) and [`COMPILING.md`](COMPILING.md). ## Language at a glance diff --git a/SCENES-DESIGN.md b/SCENES-DESIGN.md deleted file mode 100644 index e5e91782..00000000 --- a/SCENES-DESIGN.md +++ /dev/null @@ -1,354 +0,0 @@ -# Scenes, expanded — a design doc - -> **Status: S0 shipped; S1–S6 are design.** The base construct — `scene` / -> `layer` / `on enter` / `on exit` / `become`, lowered to the implicit machine of -> §3 and §9 — is implemented and tested ([`examples/lang/scenes.ludic`](examples/lang/scenes.ludic), -> a `bin/x test` check). The extensions in §4–§8 (scene-owned entities, richer -> layers, the overlay stack, scene-local state, transition parameters) are still -> design targets. This document reaches deliberately past the thin sketch so we -> can decide the shape before building each one. §11 lists the open decisions. - ---- - -## 1. Where we are - -A Ludic program is almost always several mutually-exclusive states — a title -screen, the overworld, a battle, a pause menu. Two ways to write that exist in -the language today, and a third is sketched: - -| Approach | Status | Cost | -|---|---|---| -| Mode register consulted at the top of every handler (`if reg(R_MODE) == …`) | works | a guard re-read per handler per frame; state is a magic number; nothing scopes to it | -| `machine`/`state`/`become` over a register | works | dispatch on the register each frame; still one flat register, no per-state handlers or lifecycle | -| `scene`/`layer`/`on enter`/`on exit` | **sketch only** | — | - -The sketch ([`examples/lang/scenes.ludic`](examples/lang/scenes.ludic)) specs: - -- Exactly **one scene active**; the `start` scene runs first. -- A scene's handlers run only while it is active; handlers outside any scene are - global. -- **Layers group handlers; declaration order is draw order** — within a phase, - globals first, then the active scene's layers in written order. -- `on enter` / `on exit` are lifecycle hooks (not phases). -- `become Name` runs the old scene's `on exit`, switches, runs the new `on enter` - — two direct calls and a store, no dispatch table. - -That's a good spine. The problem is it's specced as **sugar over a mode -register**: it tidies the syntax but adds little the register didn't already -have. The compiler knows *much* more at a scene boundary than a register does, -and this doc is about spending that knowledge. - ---- - -## 2. Design principles - -1. **The scene boundary is a compile-time fact — use it.** The set of handlers, - layers, and owned state for each scene is known statically. Transitions should - be direct calls and a single store, never a table walk. (The sketch already - promises this; the extensions must preserve it.) -2. **Structure, not registers.** Anything you'd track with a hand-managed - register alongside the mode — which entities belong to this state, which layers - are drawn, what's paused — should be expressible *as* scene structure and - enforced by the compiler. -3. **Reuse the machinery we already have.** Layers pausing, scenes tearing down - their entities, and hooks firing are all expressible in terms of - `enable`/`disable` (cheap flag flips), `despawn`, and the lifecycle-hook - lowering. Scenes should *compose* those, not introduce a parallel runtime. -4. **One active-scene path stays hot; overlays are the exception, not the rule.** - The common case (one full-screen scene at a time) must lower to the cheapest - possible dispatch. Richer shapes (a pause menu over a frozen world) are opt-in - and pay only for what they use. - ---- - -## 3. Core model (firmed up from the sketch) - -```ludic -# doc-check: skip — illustrative -scene Title start { - on enter { ui_open(UI_Menu) } - on exit { ui_visible(UI_Menu, 0) } - - layer Main { - handler Choose phase Update { - if ui_clicked(UI_NewGame) { become Overworld } - } - } -} - -scene Overworld { - on enter { spawn_party() } - layer World { handler Move phase Update { … } } - layer Hud { handler Draw phase Render { … } } -} -``` - -Unchanged from the sketch, made precise: - -- **Scenes number themselves by declaration order**, exactly like `machine` - states — `Title` is `0`, `Overworld` is `1`. The active scene lives in one - implicit register (`__scene`). This makes `scene` a `machine` the compiler - writes for you, which is the right mental model and the right lowering. -- **A layer handler may not use phase `Start`.** `Start` runs once at boot, - before any scene is entered; scene setup goes in `on enter`. -- **Global handlers still run every frame**, before any scene's layers, in every - phase. A scene's layers run only while it is active. - -Everything below is new. - ---- - -## 4. Extension E1 — scene-owned entities (scoped lifetime) - -The single biggest thing a mode register cannot do: **own the entities that only -make sense in this state, and tear them down automatically on exit.** Today a -battle scene spawns combatants in `on enter` and must remember to despawn every -one in `on exit` — miss one and it leaks into the overworld. - -Proposal: entities spawned *by a scene's handlers or `on enter`* are tagged with -that scene, and `on exit` despawns them by default. - -```ludic -# doc-check: skip -scene Battle { - on enter { spawn Foe; spawn Foe; spawn Foe } # tagged @Battle - # on exit: implicit `despawn all @Battle` — no manual cleanup - layer World { handler Fight phase Update { … } } -} -``` - -- Implemented as an implicit **scene tag** (a `{Battle}`-style kind bit) added at - `spawn` time while a scene is active, plus a generated `despawn`-by-tag in the - synthesized `on exit`. Reuses the existing tag-filter and despawn-hook - machinery — no new runtime. -- **Opt out** for entities that should outlive the scene: `spawn Foe persist` (or - spawn it from a global handler). Persisted entities keep their data across the - transition, matching how `disable` keeps field data. -- Composes with `@OnDespawn(Model)`: the destructor hook fires for each - scene-owned entity as it's torn down, so `drop_loot`-style cleanup still runs. - -**Open:** does a re-`become Battle` get fresh entities (fresh tag generation) or -resume the old ones? Default: fresh. See §9. - ---- - -## 5. Extension E2 — layers are more than draw order - -The sketch uses layers only to order `Render`. Layers are the natural unit for -three more things, all built on the existing `enable`/`disable` flag flips: - -1. **Per-layer toggle.** `disable Hud` / `enable Hud` flips one flag; the layer's - handlers stop running and drawing. This is `disable Handler` generalized to a - named group — same one-flag-flip cost. - -2. **Pause vs. tear-down.** A layer can keep drawing while its *update* handlers - are suspended: - - ```ludic - # doc-check: skip - scene Overworld { - layer World { handler Move phase Update { … } handler Draw phase Render { … } } - layer Hud { handler DrawHud phase Render { … } } - } - ``` - - When a pause menu opens over the Overworld (see E3), `World`'s `Update` - handlers suspend but its `Render` handler still paints the frozen world behind - the menu. Today that requires a `if !paused` guard in every update handler; - with layers it's structural. - -3. **Layer lifecycle hooks.** `on show` / `on hide` per layer, mirroring scene - `on enter`/`on exit`, for the toggle points. (Naming TBD — could fold into the - `@OnEnable`/`@OnDisable` annotations, which already exist for properties.) - ---- - -## 6. Extension E3 — the scene *stack* (the headline) - -The sketch says "exactly one scene is active." That's the right default and the -wrong constraint. The states a mode register handles *worst* are the ones that -**overlay without replacing**: a pause menu over live gameplay, a dialog box, an -inventory screen, a confirmation prompt. With one register you either lose the -underlying state or hand-roll a "previous mode" variable and restore it. - -Proposal: keep "one *base* scene," but allow scenes to be **pushed as overlays**. - -```ludic -# doc-check: skip -scene Overworld { - layer World { handler Move phase Update { … } handler Draw phase Render { … } } - layer Hud { handler DrawHud phase Render { … } } - - on enter { … } - handler PauseKey phase Input { if pressed(KEY_ESC) { push Pause } } -} - -scene Pause overlay { # `overlay` = pushed, not swapped - on enter { dim_backdrop() } - layer Menu { - handler Nav phase Update { - if pressed(KEY_ESC) { pop } # back to Overworld, untouched - } - handler Draw phase Render { ui_render() } - } -} -``` - -- `push Name` runs `Name`'s `on enter` and makes it the top scene **without** - running the base scene's `on exit`. `pop` runs the overlay's `on exit` and - returns to whatever was beneath. -- **Update belongs to the top of the stack; render walks the whole stack bottom - to top.** So `Pause`'s `Menu` layer draws over `Overworld`'s frozen `World` and - `Hud`. This is the default that makes pause menus "just work." An overlay that - should let the layer beneath keep updating opts in with `push Name passthrough`. -- **The stack is a small fixed-capacity array of scene ids** (say 8) in a - compiler-owned buffer — not heap, not a linked structure. `push`/`pop` are an - index bump and an `on enter`/`on exit` call. Depth overflow is a compile-time - or trap decision (§9). -- `become` still exists and still means "swap the base scene" (full `on exit` → - `on enter`, stack cleared). `push`/`pop` are the overlay verbs. Keeping the two - distinct is what lets the common single-scene path stay a single register. - -This is the extension that turns `scene` from "nicer mode register" into -something with no clean equivalent in the register world. - ---- - -## 7. Extension E4 — scene-local state - -A scene almost always has state that exists only while it's active — a battle's -turn counter, a menu's cursor index. Today that's a global register that other -scenes could stomp. Proposal: **`var` / `const` declared inside a `scene` is -scoped to it**, storage shared across scenes that are never simultaneously active -(the compiler can overlap their storage since only one base scene runs at a -time — an arena-per-scene, or a union). - -```ludic -# doc-check: skip -scene Battle { - var turn = 0 # visible only inside Battle; reset by `on enter` if desired - layer World { handler Step phase Update { turn += 1 } } -} -``` - -- Reads/writes lower to a fixed offset in the scene's state block, no register - indirection. -- Overlay scenes (E3) that *can* be live simultaneously with their base cannot - share storage — the compiler keeps their blocks distinct. Base scenes that - never coexist share. - ---- - -## 8. Extension E5 — parameterized transitions, and the reserved annotations - -**Parameters on transitions.** `become`/`push` can carry arguments that the -target's `on enter` binds — so a battle knows which foes, a dialog knows which -line: - -```ludic -# doc-check: skip -scene Battle { - on enter (foe_kind: int, count: int) { for i in 0 .. count { spawn_foe(foe_kind) } } -} -# elsewhere: -become Battle(FOE_GOBLIN, 3) -``` - -Lowers to argument stores into the scene's state block (E4) immediately before -the `on enter` call. No variadic runtime; the arity is checked at compile time. - -**The already-reserved annotation form.** [LANGUAGE.md:374](LANGUAGE.md:374) -reserves `@OnEnter` / `@OnExit` as handler annotations "waiting on scene support." -This doc adopts them as the annotation spelling of `on enter` / `on exit`, -mirroring how `@OnStart` is the annotation form of `phase Start`: - -```ludic -# doc-check: skip -@OnEnter(Battle) handler Setup { … } # == Battle's `on enter` -@OnExit(Battle) handler Teardown { … } -``` - -Both spellings desugar to the same synthesized scene-lifecycle function; a scene -may use either, not both, for a given hook. - -**`reads`/`writes` + scenes (forward-looking).** The `reads`/`writes` clauses are -parsed but unconsumed ([LANGUAGE.md:717](LANGUAGE.md:717)). Once an analysis pass -exists, a scene's layers declare which state they touch, and the scheduler can run -independent layers of the active scene in parallel within a phase — the scene -boundary gives the pass a natural scope to reason about. Noted as a destination, -not part of the first cut. - ---- - -## 9. Lowering summary - -Everything above reduces to existing runtime concepts: - -| Construct | Lowers to | -|---|---| -| active base scene | one implicit register `__scene`, states numbered by decl order — literally a compiler-written `machine` | -| `become Name` | `on exit` call · `set __scene` · `on enter` call (two direct calls + store, as the sketch promises) | -| scene layers in a phase | the phase scheduler, after global handlers, dispatches on `__scene` to that scene's layer handlers in declaration order | -| `push`/`pop` (E3) | fixed-capacity scene-id array + index; render walks it, update reads its top | -| scene-owned entities (E1) | implicit kind tag at `spawn`; generated `despawn`-by-tag in synthesized `on exit`; reuses despawn hooks | -| layer toggle / pause (E2) | the same one-flag-flip as `disable Handler`, keyed per layer | -| scene-local `var` (E4) | fixed offsets in a per-scene state block; non-coexisting scenes share storage | -| transition args (E5) | arg stores into the state block before the `on enter` call | -| `@OnEnter`/`@OnExit` (E5) | the same synthesized lifecycle functions as `on enter`/`on exit` | - -No heap, no dispatch tables, no new allocator. The active-scene path is a -register read and a static branch; the stack adds a small array only for programs -that push overlays. - ---- - -## 10. Suggested implementation phases - -Each is independently shippable and testable, matching how the repo phases work. - -- **S0 — parse & lower the sketch.** ✅ **Done.** `scene`/`layer`/`on enter`/`on - exit`/`become` lowered to the implicit `machine`; the active scene is - snapshotted per phase so exactly one scene's layers dispatch in any phase. - [`examples/lang/scenes.ludic`](examples/lang/scenes.ludic) compiles, runs, and is checked - by `bin/x test`. This is the floor everything else builds on. -- **S1 — `@OnEnter`/`@OnExit` annotation form** (E5, cheap once S0 exists). -- **S2 — layer toggle & pause** (E2) on top of the existing `enable`/`disable`. - ✅ *Toggle shipped* (via EVENTS-DESIGN EV1 layers): `enable layer L` / `disable - layer L` flips an `@LE_` flag that gates the layer's handlers (emitted only - for toggled layers, so untouched scene programs stay byte-identical), and a - `public` layer fires `layer__show`/`_hide` — see - [`examples/events/layer_events.ludic`](examples/events/layer_events.ludic). Still open: the - *pause* half (keep drawing while `Update` handlers suspend) and `on show`/`on - hide` blocks. -- **S3 — the scene stack** (E3): `push`/`pop`/`overlay`/`passthrough`. The big one. -- **S4 — scene-owned entities** (E1) and **scene-local state** (E4). -- **S5 — transition parameters** (E5). -- **S6 (later) — `reads`/`writes` scheduling** (E5), gated on the analysis pass. - -S0–S1 deliver the sketch as promised; S2–S3 are where the "great potential" -actually lands; S4–S5 are ergonomics; S6 is a performance destination. - ---- - -## 11. Open decisions - -1. **Re-entering a scene:** fresh entities/state, or resume? (Default proposed: - `become` = fresh, `push`/`pop` = the pushed scene is fresh each push, the base - underneath is untouched.) -2. **Stack depth:** compile-time cap with an error on overflow, or a runtime trap? - What capacity (8? configurable)? -3. **`passthrough` granularity:** does a passthrough overlay let *all* lower - layers update, or can it name which phases fall through? -4. **Layer hook naming:** `on show`/`on hide`, or reuse `@OnEnable`/`@OnDisable`? -5. **Scene-local storage sharing:** union non-coexisting scenes automatically, or - require an explicit opt-in so the sharing is visible in source? -6. **Global handlers and overlays:** do globals run once per frame regardless of - stack depth (proposed: yes), or per active scene? -7. **`become` from inside an overlay:** does it clear the stack (proposed: yes) or - is it an error while overlays are pushed? - ---- - -*Companion to [LANGUAGE.md §"Scenes & layers"](LANGUAGE.md) and the ordering -sketch in [`examples/lang/scenes.ludic`](examples/lang/scenes.ludic). Supersedes nothing -until the compiler work in §10 lands.* diff --git a/SYNTAX-REDESIGN.md b/SYNTAX-REDESIGN.md deleted file mode 100644 index 8764ba9f..00000000 --- a/SYNTAX-REDESIGN.md +++ /dev/null @@ -1,375 +0,0 @@ -# Ludic Syntax Redesign — Cohesion Pass - -A plan to make Ludic's syntax internally consistent. It fixes the drift between -the spec and the compiler, then unifies the grammar around two rules. Scope: -**full redesign (Phases 0–5)**. Named-field direction: **colon everywhere**. - -> Status: **Phases 1–5 complete.** Every phase kept the compiler self-hosting to -> a fixpoint (`bin/x test` 14/14), and each syntax migration was proven -> behaviour-preserving (the migrated compiler compiles itself to byte-identical -> IR; every golden game renders byte-identically). Landed on branch -> `syntax-redesign-phase2` over a committed baseline on `main`. -> -> Coordinated with the toolchain agent (CLI front-end / `ludicc`+`ludic` -> binaries) via serialized reseeds of `selfhost/ludicc.seed.ll`; Phase 1 rode in -> alongside their `emit_*`/`main.ludic` work, combined suite **14/14 green**. - ---- - -## Why (the findings) - -Verified against the self-hosted compiler ([selfhost/parse.ludic](selfhost/parse.ludic), -[selfhost/parse_game.ludic](selfhost/parse_game.ludic), [selfhost/lex.ludic](selfhost/lex.ludic)): - -**Structural incoherence** -1. **Five micro-syntaxes for named parts** — `name: type = d` (fields), `name: type` - (params), `Field = { k = v }` (spawn), `[Name, {Tag}]` (query), whitespace - `phase X reads [..]` (system clauses), `key=value` (ui props). -2. **`=` means seven things, `:` means one** — assignment, default, record init, - ui prop, extern symbol, const value, `state X = N` all use `=`. -3. **No statement terminators** — `\n` and `;` both lex to `TK_NL` - ([lex.ludic:45,121](selfhost/lex.ludic)) but the parser never requires a - separator, so `t.kind = k t.text = x t.ival = v` (three statements, spaces - only) is idiomatic. - -**Broken / dead syntax (compiler-verified)** -4. `edge system` — **hard parse error** (documented at [LANGUAGE.md:188](LANGUAGE.md)). *(✅ fixed in Phase 1)* -5. `pure fn` — parses, `pure` silently discarded ([parse.ludic:272](selfhost/parse.ludic)); undocumented. *(✅ Phase 3d: now `@pure`)* -6. `@anno` + `reads/writes/needs/uses [..]` — parsed then thrown away - ([parse_game.ludic:15-31](selfhost/parse_game.ludic)); four synonyms, two undocumented. *(✅ Phase 3c: `needs`/`uses` dropped)* -7. `scene`/`layer`/`on enter` — full LANGUAGE.md section + [examples/lang/scenes.ludic](examples/lang/scenes.ludic), - **does not compile** (`expected declaration`). -8. `query (v) [..]` in a system signature — two LANGUAGE.md sections + - [examples/lang/qdecl.ludic](examples/lang/qdecl.ludic), **does not compile** (`parse error: {`). *(✅ implemented in Phase 1)* -9. `when cond {}` — documented ([LANGUAGE.md:329](LANGUAGE.md)) + in all three editor - highlighters, **never parsed**. *(✅ implemented in Phase 1 as an if-without-else alias)* -10. CLI `--emit-llvm`/`-o`/`--shared`/`--fmt` — documented, but `ludicc` only - accepts `--windowed`/`--headless` ([main.ludic:8-14](selfhost/main.ludic)). - -**Philosophical splits** -11. Operators are words (`and`/`or`/`not`), symbols (`==`/`<=`), *and* functions - (`band`/`shl`) at once. -12. Three overlapping control families — `if`/`when`, `match`, `machine`/`become` - — and `enter` reuses `become`'s AST node ([parse.ludic:176-177](selfhost/parse.ludic)). *(Phase 4: `if`/`when` kept by choice; magic-int dispatch resolved)* -13. Typed components/structs exist, but real state lives in 64 untyped int - registers (`reg`/`set_reg`), so `machine`/`match` dispatch on magic numbers. *(✅ Phase 4: auto-numbered states + `enum` name the values)* - ---- - -## The two rules everything converges on - -**Rule A — `:` associates, `=` binds.** -- `:` introduces a *named part* and its type or value in a declarative structure: - component/struct fields' types, record initializers, ui props, (future) named - call arguments. -- `=` binds a value to a storage location or a constant: `let`, assignment - (`+=` …), `const` value, a field's **default**, and the extern symbol. -- A field declaration uses both, unambiguously: `x: int = 0` reads "`x` *has type* - `int` (`:`), *defaulting to* `0` (`=`)" — same shape as Rust/TypeScript. -- A record/spawn initializer is declarative, so it uses `:` — `Pos { x: 10 }`. - -**Rule B — a statement ends at a newline (or `;` or `}`).** -- Newlines become significant. Two statements on one line require an explicit - `;`. `ludic-fmt` normalizes one statement per line and inserts/removes `;`. - -Everything below is these two rules applied construct by construct. - ---- - -## Target grammar (before → after) - -### Records / spawn initializers -```ludic -# doc-check: skip — illustrative redesign snippet (proposed / partial syntax) -# before -spawn Hero { Pos = { x = 10, y = 5 } Player = { } } -# after -spawn Hero { - Pos { x: 10, y: 5 } - Player {} -} -``` -`Field = { k = v }` → `Field { k: v }`. The component name is followed directly -by a record; fields use `:`. (Record literals elsewhere read the same: -`{ x: 10, y: 5 }`.) - -### UI props → named-argument form -```ludic -# doc-check: skip — illustrative redesign snippet (proposed / partial syntax) -# before -panel id=Root w=288 pad=16 gap=6 align=center { label text="HI" size=26 } -# after -panel(id: Root, w: 288, pad: 16, gap: 6, align: center) { - label(text: "HI", size: 26) -} -``` -A widget becomes "a constructor with named args, then an optional child block." -This deletes the bespoke `key=value` dialect and reuses `:` + commas. (Lower-churn -alternative if the paren form is disliked: keep whitespace separation but colonize -— `panel id: Root w: 288` — still removes the `=` overload.) - -### System clauses & modifiers → one annotation channel -```ludic -# doc-check: skip — illustrative redesign snippet (proposed / partial syntax) -# before -edge handler Move @deterministic reads [Vel] writes [Pos] phase FixedUpdate - query (p, v) [Pos, Vel] where a.x > 0 { … } -# after -@edge @deterministic -handler Move - phase FixedUpdate - reads [Vel] writes [Pos] - query (p, v) [Pos, Vel] where p.x > 0 -{ … } -``` -- Prefix modifier words (`edge`, `pure`, `export`) are **retired**; all modifiers - become `@annotations`, parsed into a real list on the node (not skipped). This - fixes the `edge system` parse bug (#4) by construction. -- `needs`/`uses` are dropped; `reads`/`writes` stay as the two structural clauses - and are **stored** (even if analysis is future work) rather than discarded. -- The `query (v) [..]` signature clause is **actually implemented** in - `parse_system` (#8), lowering to the same `S_QUERY` node as the inline `for`. - -### extern -```ludic -# doc-check: skip — illustrative redesign snippet (proposed / partial syntax) -extern function c_hypot(a: fixed, b: fixed) -> fixed = "hypot_fx" # unchanged -``` -The `= "symbol"` is a binding under Rule A — it stays. - -### Statements -```ludic -# doc-check: skip — illustrative redesign snippet (proposed / partial syntax) -# before (legal today) -t.kind = kind t.text = text t.ival = ival -# after -t.kind = kind -t.text = text -t.ival = ival -# or, explicitly, on one line: -t.kind = kind; t.text = text; t.ival = ival -``` - -### Control flow (Phase 4) -- **`when` vs `if`** — `when` is now a working `if`-without-else alias (Phase 1). - Phase 4 decides whether to keep both spellings or collapse to one; if collapsed, - remove `when` from docs, the parser, and all editor highlighters together. -- **Typed states replace magic-int machines.** Introduce `enum`, and let - `machine` dispatch on a typed variable instead of a register: -```ludic -# doc-check: skip — illustrative redesign snippet (proposed / partial syntax) -# before # after -const R_PHASE: int = 0 enum Phase { KnightMenu, KnightResolve, MageMenu, EnemyTurn } -machine R_PHASE { var phase: Phase = Phase.KnightMenu - state KnightMenu = 0 { … become … } machine phase { - state KnightResolve = 1 { … } state KnightMenu { … become KnightResolve } -} state KnightResolve { … } - } -``` - `state X = N` loses the magic `= N` (ordinal comes from the enum). `become` - and `enter` (scenes) keep one shared lowering but read from a typed slot. - ---- - -## Phase sequence - -Each phase is independently shippable and ends green on `bin/x test` + -`bin/x selfhost-test` (fixpoint). - -### Phase 0 — Doctrine (done here) -Rules A and B above; colon-everywhere; `@`-annotations as the single modifier -channel; typed enums for state. No code. - -### Phase 1 — Truth-in-documentation ✅ DONE -Made spec ⇄ compiler agree **before** any grammar change. What landed: -- ✅ **`edge system` crash fixed** (#4) — `parse_system` now consumes an optional - `edge` marker before `system` ([parse_game.ludic](selfhost/parse_game.ludic)). - (`edge` is a pure marker; the emitter never lowered it differently.) -- ✅ **Signature-`query` implemented** (#8) — `query (vars) [terms] where c` in a - system header desugars to the same `S_QUERY` node the inline `for` builds, so - `examples/lang/qdecl.ludic` compiles and runs. Also fixed multi-line clause parsing - (clauses may now span lines). -- ✅ **`when c { }` implemented** (#9) — as an `if`-without-else alias in - [parse.ludic](selfhost/parse.ludic). Docs + editors already listed it; now the - compiler agrees, so no editor-vocab churn was needed. -- ✅ **`scene`/`layer` marked not-yet-implemented** (#7) — prominent note in - LANGUAGE.md §"Scenes & layers" + a header on [examples/lang/scenes.ludic](examples/lang/scenes.ludic). - Full scene front-end + emission deferred (real work, out of Phase 1 scope). -- ✅ **`reads`/`writes` honesty** (#6) + the stale "Not yet implemented" section - updated in [LANGUAGE.md](LANGUAGE.md); scenes/reads-writes/dropped-CLI-flags now - listed there. -- ✅ **`bin/x test` guards drift** — added a `qsmoke qdecl` compile check. Suite - green (14/14 incl. the toolchain agent's CLI smoke tests). -- ✅ **CLI flags** (#10) — `--shared`/`--fmt`/wasm noted as dropped-with-the-C-driver - in LANGUAGE.md; `-o`/`--emit-llvm` were being re-added by the toolchain agent - (now real, verified in `bin/x test`); COMPILING.md updated by that agent. -- Deferred (intentionally): `pure`-is-ignored (#5) is undocumented and harmless; - it will be folded into `@pure` in Phase 3 rather than churned now. -- **Not done / by design:** `scenes.ludic` is *not* added to `bin/x test` (it can't - compile yet — a positive test would fail; the header note + LANGUAGE.md warning - cover the drift instead). - -### Phase 2 — Statement separation (Rule B) ✅ DONE -Landed on branch `syntax-redesign-phase2` (baseline committed on `main` first). -- ✅ **Parser enforces a separator** — `block()` requires a newline or `;` after - each statement, else `expected newline or ';' between statements` - ([parse.ludic](selfhost/parse.ludic)). Also fixed `if`-without-`else` swallowing - its trailing separator (it now peeks for `else` and restores if absent). -- ✅ **Interpretation chosen:** *require a separator*, not *reflow to one-per-line*. - The migration **inserts `;` at statement boundaries** and leaves lines intact — - comment-safe, minimal-diff, and it makes boundaries visible without an - opinionated reflow. One-per-line stays the recommended hand-written form. -- ✅ **Migration tool** ([tools/ludic-tools/migrate_separators.c](tools/ludic-tools/migrate_separators.c), - reuses the toolchain lexer) with a - **verification oracle**: a `;` inserted at a real boundary is a semantic no-op, - proven by the migrated compiler compiling itself to **IR byte-identical to the - seed** and every golden game rendering identically. ~1100 boundaries across the - corpus (examples, runtime, and the 25 self-host fragments). -- ✅ **Reseeded** to the strict compiler (19557 lines); C-free bootstrap fixpoint - holds; `bin/x test` 14/14; all goldens byte-identical; qdecl runs correctly. -- ✅ **Docs updated** — Rule B documented in LANGUAGE.md §Statements; BOOTSTRAP.md - R1 (which advertised no-separator juxtaposition as legal) and its stale code - fences updated; `check-docs` (now a live strict parse gate) green across all docs. - -**Bug found & fixed en route:** a multi-line string literal in -[emit_expr.ludic](selfhost/emit_expr.ludic) (`emit(")")`) lexed fine in -the self-host lexer but the **C toolchain lexer** (`ludic_syntax.h`, shared by -sepfix, `ludic-fmt`, and the LSP) stops strings at newline — so it mis-lexed and -`ludic-fmt` would corrupt such a file. Converted it to the byte-identical `\n` -escape. **Open follow-up:** align the C lexer to allow newlines in strings, or -forbid literal newlines in string literals language-wide (the two lexers should -agree). Flagged to the toolchain owners. - -### Phase 3 — Named-field unification (Rule A) - -**3a — spawn/record initializers ✅ DONE.** `Comp = { f = v }` → `Comp { f: v }`. -`record()` requires `:` and `parse_spawn()` drops the `=` before the record -([parse.ludic](selfhost/parse.ludic), [parse_game.ludic](selfhost/parse_game.ludic)). -The `=` is now assignment/const/default/extern-binding only. Migration tool: -[migrate_records.c](tools/ludic-tools/migrate_records.c) (spawn-context aware). -Records live only in games, so the seed was unaffected; verified every golden -byte-identical, old `=` form now rejected, reseeded, `bin/x test` 14/14. Doc examples -updated (LANGUAGE.md, BOOTSTRAP.md R2). - -**3b — ui props → `key: value` ✅ DONE.** `panel id=Root w=288` → `panel id: Root -w: 288`. `parse_widget` now reads props with `:` ([parse_game.ludic](selfhost/parse_game.ludic)). -Chose the **colonized** form over parenthesized named-args: it satisfies Rule A -(the `=` overload is gone) with minimal churn, needs no new grammar, and `emit_ui` -(which reads the AST) and `ludic-fmt` (which formats `:` correctly by default) -were both untouched. Migration: [migrate_ui.c](tools/ludic-tools/migrate_ui.c). -menu golden byte-identical, old `=` form rejected, reseeded, `bin/x test` 14/14. -(The parenthesized form `panel(id: Root, w: 288)` remains a possible future -refinement if the language ever gains named call arguments.) - -**3c — dropped the dead `needs`/`uses` clause synonyms ✅ DONE.** `reads`/`writes` -stay (documented; still parsed-and-reserved). `needs`/`uses` were undocumented and -unused anywhere in the corpus — removed from `parse_system`. *Not done:* actually -*storing* reads/writes on the node for an analysis pass — that's analysis -infrastructure, out of scope for a syntax pass. - -**3d — modifiers → `@`-annotations ✅ DONE.** `edge`/`pure`/`export` prefix keywords -are retired; declaration modifiers are now leading `@annotations`: `@export fn`, -`@edge system`, `@pure`, `@deterministic`. `parse_one_decl` collects a leading -`@anno` run and `@export` sets the fn export flag ([parse.ludic](selfhost/parse.ludic)); -the dead `edge`-dispatch was removed from `parse_system`. Migrated the one -`@export` user ([examples/library/combat.ludic](examples/library/combat.ludic)); old -prefix forms now rejected. Behavior-identical: the export flag is parse-only in -the self-hosted emitter (it emits `@fn_` for every function and never reads -the flag — the C-ABI-export capability is vestigial, a pre-existing gap), so -`@export` and the old `export` produce byte-identical IR. Reseeded, fixpoint -holds, `bin/x test` 14/14, goldens byte-identical. - -**Phase 3 is complete.** The `=`/`:` overload (finding #2) and the modifier-zoo -(findings #5, #6) are resolved; `:` associates and `=` binds throughout. - -Each sub-phase follows the proven pattern: parser change → verification-gated -migration (IR byte-identical / goldens identical) → reseed → docs. The migration -tools ([migrate_separators.c](tools/ludic-tools/migrate_separators.c), -[migrate_records.c](tools/ludic-tools/migrate_records.c)) are the reusable spine. - -### Phase 4 — Control-flow & state consolidation - -**4a — machine states auto-number ✅ DONE.** `state KnightMenu = 0 { }` → -`state KnightMenu { }`; a state's value is its declaration index (an explicit -`= expr` still works). Removes the magic constants from state machines -([parse.ludic](selfhost/parse.ludic)). combat.ludic migrated; chronorift golden -byte-identical. - -**4b — `enum` types ✅ DONE.** `enum Action { Attack, Guard, Item, Flee }` declares -named `int` constants; a variant is a compile-time int accessed as `Action.Guard` -(= 1), numbered by order. Parser `parse_enum` + dispatch, `enum_ordinal` resolver -in [emit_core.ludic](selfhost/emit_core.ludic), and `Enum.Variant` handling in -[emit_expr.ludic](selfhost/emit_expr.ludic). combat.ludic's battle menus now -dispatch on `KnightAct`/`MageAct` instead of `0..3`; chronorift golden -byte-identical. Editor vocab (`ludic_syntax.h`, JetBrains, TextMate, emacs) gained -`enum` and lost the retired `edge`/`export`/`pure` decl keywords; check-vocabulary -+ test-tools green. **Scoped:** enums are a naming layer over `int` (no distinct -runtime type / enum-typed variables yet) — that keeps register/save semantics -untouched, which the "enum var replaces the register" vision would have to solve. - -**`when` vs `if` — kept both (decision).** `when` stays as the `if`-without-else -spelling: it is not incoherent so much as a readability signal ("no else here"), -it is documented and highlighted, and it is a pure alias with no semantic overlap -to untangle. The real target of finding #12 — dispatch on magic integers — is -addressed by 4a/4b, not by collapsing `if`/`when`. - -**Bitwise operators — kept as functions (decision).** `band`/`bor`/`bxor`/`bnot`/ -`shl`/`shr` stay functions, documented as the deliberate "one spelling, symbols -stay free" choice (LANGUAGE.md §Expressions already states this). Promoting them -to operators would re-introduce the symbol soup the current design avoids. - -### Phase 5 — Vocabulary anchored to the compiler ✅ DONE -The editor vocabulary already stayed in sync *with itself* (`check-vocabulary.py` -compares `ludic_syntax.h`, the JetBrains lexer, and the TextMate grammar). The -missing anchor was the **compiler**: a keyword could be highlighted everywhere -and still be silently unparsed. Closed both loops: -- ✅ **Vocabulary ⇄ parser.** `check-vocabulary.py` now extracts every keyword - `selfhost/parse*.ludic` dispatches on (`is_id(...)` / `streq(t.text, ...)`) and - requires the header's declaration + clause keywords to be a subset — with a - `LUDIC_KW_RESERVED` escape hatch for documented, not-yet-implemented keywords - (`scene`/`layer`/`on`/`start`), itself checked so a reserved word that gets - implemented must be promoted. Verified it catches an injected bogus keyword. -- ✅ **Reconciled the drift it exposed.** Removed the highlighted-but-unparsed - `scene`/`layer`/`on`/`start` (→ RESERVED) and the never-implemented - `needs`/`uses`/`requires`/`ensures`/`invariant`/`effects` clause words, and the - retired `edge`/`export`/`pure` prefix modifiers, from `ludic_syntax.h`, the - JetBrains lexer, the TextMate grammar, and the emacs mode; added `enum`/`main`. - `@`-annotations already highlight generically (`@[A-Za-z_]…`). test-tools 28/0. -- ✅ **Doc-fence compilation** — the other half of "single source of truth" — was - already live: `check-docs.py` compiles every ` ```ludic ` fence through the - self-hosted `ludicc --fmt` parse gate (revived during Phase 1's coordination). - -Full generation-from-one-list (emit the editor files from a manifest) was not -needed: the bidirectional *checks* give the same guarantee — nothing can drift -without CI failing — without a code-generation step to maintain. - -**Phases 1–5 are complete.** - -### Phase 6 — vocabulary rename + annotation DSL ✅ DONE (follow-on request) -Renamed the core nouns: `game`/`module` → `program`, `main` → `entry`, -`component` → `property`, `archetype` → `model`, `system` → `handler`. Done via a -transitional self-hosting bootstrap (accept both → reseed → move the compiler's -own source to new keywords + tighten → reseed); old keywords now rejected. -Token-safe corpus migration ([rename_kw.c](tools/ludic-tools/rename_kw.c)), goldens -byte-identical. Reconciled the LSP indexer, editor vocab, check-docs wrapper, and -docs; fixed two pre-existing toolchain bugs (a `set -e` bug in build-tools.sh that -blocked all editor-binary rebuilds, and a stale LSP test offset). - -Added an **annotation DSL**: `@Queries(these: [Prop{constraint}, …], on: Model)` on -a handler desugars to the existing `S_QUERY` loop (each property binds by its own -name; a `Prop{…}` constraint qualifies its bare fields; `on:` adds a `{Model}` -tag), and `@Handles(…)` on a program parses as documentation. See -[examples/lang/annotations.ludic](examples/lang/annotations.ludic); bin/x test 15/15. All thirteen findings are resolved or resolved by an -explicit, documented decision. - ---- - -## Decision log -- **Scope:** full redesign, Phases 0–5. *(chosen)* -- **Named fields:** colon everywhere; `=` is binding-only. *(chosen)* -- **Open — Phase 4 detail:** typed `enum` state vs. keep integer registers. - Recommended: typed enums (fixes #13), but it's the deepest change; can be - deferred without blocking Phases 1–3. -- **Open — ui props:** paren named-args (`panel(id: Root)`) vs. colonized - whitespace (`panel id: Root`). Recommended: paren form for full cohesion. -- **Open — bitwise ops:** functions (status quo, documented) vs. operators. diff --git a/examples/README.md b/examples/README.md index 3f9b5891..5b06a103 100644 --- a/examples/README.md +++ b/examples/README.md @@ -55,7 +55,7 @@ bin/ludic examples/lang/offline_rewards.ludic # compile + run a plain progra ## `networking/` — deterministic multiplayer (N0–N6) -Each maps to a stage of [`NETWORKING-DESIGN.md`](../NETWORKING-DESIGN.md); all run +Each maps to a stage of [the Networking design](https://git.workshopsoft.io/workshopsoft/ludic/wiki/Design/Networking); all run over the compiler's built-in loopback transport with zero foreign code. | Example | What it shows | diff --git a/selfhost/backend/stdlib/emit_net.ludic b/selfhost/backend/stdlib/emit_net.ludic index b965016c..7bcbbb11 100644 --- a/selfhost/backend/stdlib/emit_net.ludic +++ b/selfhost/backend/stdlib/emit_net.ludic @@ -1,4 +1,4 @@ -# emit_net.ludic — NETWORKING N2–N6 codegen (NETWORKING-DESIGN.md). +# emit_net.ludic — NETWORKING N2–N6 codegen (see the Networking design on the wiki: Design/Networking). # # A program that calls net_send/net_poll with no `extern fn` override triggers the # built-in loopback transport; this flag defers its emission to end-of-module.