workshopsoft/ludic

Fork 0

Proposal: package/module distribution + package-declarable components & engine-systems (prerequisite for controller libs #57) #62

New issue

Closed

opened 2026-09-01 05:19:15 +02:00 by orkun · 1 comment

orkun commented

2026-09-01 05:19:15 +02:00

Owner

Prerequisite for the builtin-controllers layer (#57). The controllers (#58–#61) are not core language — they should ship as external, versioned packages, not be welded into the compiler or runtime/native/ stdlib. This issue is the mechanism that makes that possible. Planning only — no implementation.

Context & decision

The gameplay controllers (platformer / RPG / shooter / NPC AI) are genre content, not language. Baking them into the compiler or the in-repo runtime/native/*.ludic stdlib would (a) couple game-genre opinions to the compiler's release cadence, (b) bloat every build's surface area, and (c) prevent the community from forking/extending them — which is the entire stated goal. So they ship as packages.

The important sub-decision (discussed on #57): a Ludic package is distributed as source (Ludic modules), not as a precompiled OS binary (.dylib/.so/.a). Because Ludic is AOT (ludicc → LLVM IR → native/wasm), a source package is still fully compiled — into the consumer's binary — so we get distribution and compilation without an ABI seam. Precompiled native libraries were rejected as the default for three reasons:

They kill the wasm/web target and cross-compilation. You can't dlopen a native dylib in wasm32, and a macOS .dylib won't serve a Linux .so build. Source packages compile to every target for free.
They forfeit determinism's payoff. Lockstep / replay / rollback / byte-identical goldens rest on compiling the whole world together with fixed-point semantics; a separate binary reintroduces a reproducibility hole.
The compile-time ECS won't let a binary participate first-class. property/model/query/spawn are codegen in the consumer's binary; a precompiled lib can only reach the game through the slower dynamic reflection ABI (EV7 ludic_register_prop/ludic_get/set by name).

Precompiled-binary distribution is kept only as an escape hatch for closed-source or other-language authors shipping a mod over the existing C-ABI + dynamic reflection — second-class by design (native-only, dynamic components, perf cost), fine for mods, wrong as the default for first-party controllers.

What has to change in the compiler

Two hooks that make stdlib namespaces work today are hardcoded, and both must become data-driven / package-declarable so a package can register a namespace, a component schema, and an engine-owned system without editing the compiler:

1. Namespace splice/import trigger (today: string literals in the parser)

The on-demand splice is a per-namespace literal in p_postfix — e.g. if e.a.s == "Query" { g_uses_query = true } (parse.ludic ~L182) with a matching do_import("runtime/native/query.ludic") in maybe_splice_runtime. A package can't add a namespace without patching these.

Proposed: a package manifest declares the namespaces it provides and the module files backing them. Import resolution registers Ns → module from manifests, so p_postfix looks the namespace up in a table (populated from imported packages) instead of matching hardcoded names. emit_ns_call dispatch likewise resolves Ns.method via the manifest (alias-style, preserving the runtime fn's return type — the existing path).

2. Engine-owned system registration (today: a hardcoded component/phase list)

uses_engine_systems() and emit_engine_systems_for_phase(phase) carry a fixed list of (component, esys_fn, phase) (SpriteAnim/Motion/Light2D/…). A package's controller is an engine-owned system, so it needs to add to this list.

Proposed: a package declares its engine systems as data — (well-known-component-name, esys-fn-name, phase) — in the manifest (or via a first-class system P phase Update { ... } decl that lowers to the same registration). The compiler builds the splice list + the frame-loop insertion from the union of core + imported-package registrations. Keep the existing "resolve fields by name, no-op when absent" reflection-ABI contract so registration stays layout-independent and additive (unused package ⇒ byte-identical build).
This also generalizes the disable system <Name> lever from #57: with systems in a registry, disabling one is a registry flag, not a special case.

These two generalizations are independently useful — they turn "Ludic stdlib" into "Ludic ecosystem" and are the honest prerequisite for #58–#61 being real packages rather than more in-repo stdlib.

Package/module distribution

Today import is raw import "path" (include-guarded, path-relative) — enough for one repo, not for shipping. Proposed, kept deliberately small and determinism-first:

Manifest per package (name, version, provided namespaces, engine-system registrations, dependency list, min compiler version). One file, declarative.
Import by package name, resolved through the manifest graph: import pkg "platformer" then Platformer { ... } — vs. today's file-path import (which stays for intra-package modules).
Versioning + lockfile for reproducible builds — a resolved dependency set pinned so a project builds byte-identically across machines/time (mirrors the determinism ethos; a lockfile is to packages what fixed-point is to math).
Resolution sources — start local/vendored (a packages/ dir, monorepo-friendly), design the manifest so a URL/registry fetch can slot in later without changing the consumer-facing import.
Transitive dependency + namespace-collision handling (two packages providing the same Ns = resolution error, surfaced at compile time like every other Ludic drift check).
No new runtime — resolution happens at compile time in ludicc/bin/x; the output is the same self-contained binary/wasm.

Escape hatch: binary mods (not packages)

For authors who cannot or will not ship source (closed-source middleware, a controller written in C/Rust), the existing seam already works and needs no new mechanism: link a native object over the EV0–EV7 C-ABI + dynamic reflection (ludic_register_prop/get/set, ludic_on_<E>). Document it as the explicit, second-class path: native-only, dynamic (by-name) components, no static query/spawn, a perf cost — appropriate for post-ship user mods, not first-party genre packages.

Determinism / security / tooling notes

Package code is ordinary Ludic → all existing guarantees hold (fixed-point, no float, snapshot/replay). A lockfile extends reproducibility to the dependency set.
Compile-time resolution means no arbitrary code runs at build resolution beyond compiling Ludic source (no install scripts — a deliberate non-feature).
Editor/vocab tooling: namespaces provided by packages must feed the LSP/formatter/grammars the way core namespaces do — likely by the tools reading the same manifests (keeps check-vocabulary/check-impl honest across packages).
Docs: per-package doc pages generated from the same docs/language/** machinery, scoped to the package.

Phasing

Registry-ize the two compiler hooks (namespace splice table + engine-system registration) with core stdlib migrated onto them — byte-identical output is the acceptance test (the compiler's own namespaces now flow through the generic path). No packaging yet.
Manifest + local package resolution (packages/ dir, import pkg), version field, namespace-collision errors.
Lockfile + reproducible resolution; min-compiler-version checks.
Tooling integration (LSP/formatter/grammar/docs read manifests); document the binary-mod escape hatch.
Ship #58–#61 as the first external packages on top of this, proving the mechanism end-to-end. (Registry/URL fetch is later, out of scope here.)

References

Cargo / crates.io (manifest + lockfile + reproducible builds; the model to emulate for determinism).
Bevy ecosystem crates and flecs modules/prefabs (composable gameplay shipped as packages, not engine core).
Deno / Go modules (URL-addressed, minimal-ceremony resolution) as the later registry direction.
Ludic internals to generalize: p_postfix splice triggers + maybe_splice_runtime (parse.ludic), emit_ns_call dispatch (emit_call.ludic), uses_engine_systems/emit_engine_systems_for_phase (emit_ecs/emit_game), the EV0–EV7 C-ABI + EV7 dynamic reflection (the escape hatch), do_import include-guarding.

_Prerequisite for the builtin-controllers layer (#57). The controllers (#58–#61) are **not** core language — they should ship as **external, versioned packages**, not be welded into the compiler or `runtime/native/` stdlib. This issue is the mechanism that makes that possible. Planning only — no implementation._ ## Context & decision The gameplay controllers (platformer / RPG / shooter / NPC AI) are genre content, not language. Baking them into the compiler or the in-repo `runtime/native/*.ludic` stdlib would (a) couple game-genre opinions to the compiler's release cadence, (b) bloat every build's surface area, and (c) prevent the community from forking/extending them — which is the entire stated goal. So they ship as **packages**. The important sub-decision (discussed on #57): a Ludic package is **distributed as source (Ludic modules), not as a precompiled OS binary** (`.dylib`/`.so`/`.a`). Because Ludic is AOT (`ludicc → LLVM IR → native/wasm`), a source package is **still fully compiled** — into the consumer's binary — so we get distribution *and* compilation without an ABI seam. Precompiled native libraries were rejected as the default for three reasons: 1. **They kill the wasm/web target and cross-compilation.** You can't `dlopen` a native dylib in wasm32, and a macOS `.dylib` won't serve a Linux `.so` build. Source packages compile to every target for free. 2. **They forfeit determinism's payoff.** Lockstep / replay / rollback / byte-identical goldens rest on compiling the whole world together with fixed-point semantics; a separate binary reintroduces a reproducibility hole. 3. **The compile-time ECS won't let a binary participate first-class.** `property`/`model`/`query`/`spawn` are codegen in the *consumer's* binary; a precompiled lib can only reach the game through the slower **dynamic reflection ABI** (EV7 `ludic_register_prop`/`ludic_get`/`set` by name). Precompiled-binary distribution is kept only as an **escape hatch** for closed-source or other-language authors shipping a **mod** over the existing C-ABI + dynamic reflection — second-class by design (native-only, dynamic components, perf cost), fine for mods, wrong as the default for first-party controllers. ## What has to change in the compiler Two hooks that make stdlib namespaces work today are **hardcoded**, and both must become **data-driven / package-declarable** so a package can register a namespace, a component schema, and an engine-owned system *without editing the compiler*: ### 1. Namespace splice/import trigger (today: string literals in the parser) The on-demand splice is a per-namespace literal in `p_postfix` — e.g. `if e.a.s == "Query" { g_uses_query = true }` (parse.ludic ~L182) with a matching `do_import("runtime/native/query.ludic")` in `maybe_splice_runtime`. A package can't add a namespace without patching these. - **Proposed:** a package **manifest** declares the namespaces it provides and the module files backing them. Import resolution registers `Ns → module` from manifests, so `p_postfix` looks the namespace up in a table (populated from imported packages) instead of matching hardcoded names. `emit_ns_call` dispatch likewise resolves `Ns.method` via the manifest (alias-style, preserving the runtime fn's return type — the existing path). ### 2. Engine-owned system registration (today: a hardcoded component/phase list) `uses_engine_systems()` and `emit_engine_systems_for_phase(phase)` carry a fixed list of `(component, esys_fn, phase)` (SpriteAnim/Motion/Light2D/…). A package's controller *is* an engine-owned system, so it needs to add to this list. - **Proposed:** a package declares its engine systems as data — `(well-known-component-name, esys-fn-name, phase)` — in the manifest (or via a first-class `system P phase Update { ... }` decl that lowers to the same registration). The compiler builds the splice list + the frame-loop insertion from the union of core + imported-package registrations. Keep the existing "resolve fields by name, no-op when absent" reflection-ABI contract so registration stays layout-independent and additive (unused package ⇒ byte-identical build). - This also generalizes the **`disable system <Name>`** lever from #57: with systems in a registry, disabling one is a registry flag, not a special case. > These two generalizations are independently useful — they turn "Ludic stdlib" into "Ludic ecosystem" and are the honest prerequisite for #58–#61 being real packages rather than more in-repo stdlib. ## Package/module distribution Today import is raw `import "path"` (include-guarded, path-relative) — enough for one repo, not for shipping. Proposed, kept deliberately small and determinism-first: - **Manifest** per package (name, version, provided namespaces, engine-system registrations, dependency list, min compiler version). One file, declarative. - **Import by package name**, resolved through the manifest graph: `import pkg "platformer"` then `Platformer { ... }` — vs. today's file-path import (which stays for intra-package modules). - **Versioning + lockfile** for **reproducible builds** — a resolved dependency set pinned so a project builds byte-identically across machines/time (mirrors the determinism ethos; a lockfile is to packages what fixed-point is to math). - **Resolution sources** — start local/vendored (a `packages/` dir, monorepo-friendly), design the manifest so a URL/registry fetch can slot in later without changing the consumer-facing `import`. - **Transitive dependency + namespace-collision** handling (two packages providing the same `Ns` = resolution error, surfaced at compile time like every other Ludic drift check). - **No new runtime** — resolution happens at compile time in `ludicc`/`bin/x`; the output is the same self-contained binary/wasm. ## Escape hatch: binary mods (not packages) For authors who cannot or will not ship source (closed-source middleware, a controller written in C/Rust), the existing seam already works and needs no new mechanism: link a native object over the **EV0–EV7 C-ABI** + **dynamic reflection** (`ludic_register_prop`/`get`/`set`, `ludic_on_<E>`). Document it as the explicit, second-class path: native-only, dynamic (by-name) components, no static `query`/`spawn`, a perf cost — appropriate for post-ship user mods, not first-party genre packages. ## Determinism / security / tooling notes - Package code is ordinary Ludic → all existing guarantees hold (fixed-point, no float, snapshot/replay). A lockfile extends reproducibility to the dependency set. - Compile-time resolution means **no arbitrary code runs at build resolution** beyond compiling Ludic source (no install scripts — a deliberate non-feature). - Editor/vocab tooling: namespaces provided by packages must feed the LSP/formatter/grammars the way core namespaces do — likely by the tools reading the same manifests (keeps `check-vocabulary`/`check-impl` honest across packages). - Docs: per-package doc pages generated from the same `docs/language/**` machinery, scoped to the package. ## Phasing 1. **Registry-ize the two compiler hooks** (namespace splice table + engine-system registration) with core stdlib migrated onto them — byte-identical output is the acceptance test (the compiler's own namespaces now flow through the generic path). No packaging yet. 2. **Manifest + local package resolution** (`packages/` dir, `import pkg`), version field, namespace-collision errors. 3. **Lockfile + reproducible resolution**; min-compiler-version checks. 4. **Tooling integration** (LSP/formatter/grammar/docs read manifests); document the binary-mod escape hatch. 5. **Ship #58–#61 as the first external packages** on top of this, proving the mechanism end-to-end. (Registry/URL fetch is later, out of scope here.) ## References - **Cargo / crates.io** (manifest + lockfile + reproducible builds; the model to emulate for determinism). - **Bevy** ecosystem crates and **flecs modules/prefabs** (composable gameplay shipped as packages, not engine core). - **Deno** / **Go modules** (URL-addressed, minimal-ceremony resolution) as the later registry direction. - Ludic internals to generalize: `p_postfix` splice triggers + `maybe_splice_runtime` (parse.ludic), `emit_ns_call` dispatch (emit_call.ludic), `uses_engine_systems`/`emit_engine_systems_for_phase` (emit_ecs/emit_game), the EV0–EV7 C-ABI + EV7 dynamic reflection (the escape hatch), `do_import` include-guarding.

orkun referenced this issue

2026-09-01 05:19:32 +02:00

Proposal: builtin gameplay controllers — the extensibility contract (platformer / RPG / shooter / NPC AI) #57

orkun referenced this issue

2026-09-01 05:19:40 +02:00

Proposal: builtin Platformer controller — movement feel, tile collision, platforms, score/lives/levels (base + extensible) #58

orkun referenced this issue

2026-09-01 05:19:41 +02:00

Proposal: builtin RPG systems — movement (grid/free, 4/8-axis), inventory, crafting, quests, dialog, puzzles (base + extensible) #59

orkun referenced this issue

2026-09-01 05:19:41 +02:00

Proposal: builtin Shooter controller — top-down rotatable movement, weapons registry, projectiles/bullet-hell (base + extensible) #60

orkun referenced this issue

2026-09-01 05:19:41 +02:00

Proposal: builtin NPC AI — perception/decision/steering, FSM+BT+utility, friendly & enemy, director (base + extensible) #61

orkun referenced this issue

2026-09-01 06:12:01 +02:00

RFC: package manager for Ludic (fetch + resolve + namespace registration) #63

orkun referenced this issue from a commit

2026-09-01 11:32:05 +02:00

feat(compiler): package-declarable engine systems + namespaces (#62)

orkun referenced this issue from a commit

2026-09-01 11:32:05 +02:00

docs(pkg): document @Namespace / @EngineSystem / @System + package hooks (#62)

orkun commented

2026-09-01 11:32:36 +02:00

Author

Owner

Implemented across this session and shipped to main. The packaging prerequisite is met — a gameplay-controller library can now ship as a real package: declare components/models, register engine-owned systems and Foo.* namespaces, and be distributed/versioned/locked, all without editing the compiler. #58–#61 are unblocked.

Delivered

Distribution / manifest / lockfile (was Phases 2–3) — done in #63:

package.ludic manifest (name, version, provided namespaces, deps, kind); package.lock.ludic lockfile; MVS resolution; local/vendored resolution (LUDIC_PKG_PROXY, x vendor) with URL/git later; transitive deps; namespace collision = compile-time error; no build-time code execution.

Phase 1 — the two hardcoded hooks are now registries (commit 7cbd5d1):

Namespaces: @Namespace(Name) on a function opens Name.method(…) → dispatches to the bare name_method through the same generic path the core namespaces use (applied after them, so it never shadows a core one).
Engine systems: @EngineSystem(Component, Phase) registers an engine-owned system the frame loop runs each phase when the component is present — the package-declarable form of the built-in SpriteAnim/Motion/Light2D systems, reading components by name through the reflection ABI (unused registration ⇒ byte-identical). uses_engine_systems / emit_engine_systems_for_phase are now driven by that registry, with the core three seeded in their historical order.
Both annotations are keyword-free (no grammar/vocab churn). Acceptance: the engine-system hook migrated onto the registry with byte-identical IR (anim_ecs/light_ecs/snake) and 87/0 golden renders; the C-free bootstrap fixpoint holds. Proven end-to-end (hermetic, source path): a package registers Score+esys_score and Coach.bonus(); a consumer imports it and prints 4 99 with no compiler edit.

Binary-mod escape hatch — #62 only asked to document it; #64 built it in full: --emit-module + ludic_register_system + @System(Phase) + x build-lib + prebuilt consumption, over the EV0–EV7 C-ABI + dynamic reflection (native-only, second-class, by design).

Docs: docs/PACKAGES.md + per-annotation pages for @Namespace / @EngineSystem / @System.

import pkg "name" name-based sugar — today a package is consumed by import path via the ludic_modules/ resolution fallback (#63); the short-name form is convenience over that.
min_compiler manifest field + check.
Migrating the remaining ~15 core stdlib namespaces onto the generic path — the engine-system hook was fully migrated (byte-identical); for namespaces the generic path is proven and packages ride it, but the core namespace blocks stay hardcoded by design (byte-identity + determinism; migrating them risks the golden renders for no functional gain).
Deep LSP/grammar completion for package-provided namespaces (doc pages + inventory are in; live manifest-driven LSP is polish).
Shipping #58–#61 as the first packages (their own issues).

Closing as done — the prerequisite this issue exists for is delivered.

Implemented across this session and shipped to `main`. The **packaging prerequisite is met** — a gameplay-controller library can now ship as a real package: declare components/models, register engine-owned systems and `Foo.*` namespaces, and be distributed/versioned/locked, all without editing the compiler. #58–#61 are unblocked. ## Delivered **Distribution / manifest / lockfile (was Phases 2–3)** — done in #63: - `package.ludic` manifest (name, version, provided namespaces, deps, kind); `package.lock.ludic` lockfile; **MVS** resolution; local/vendored resolution (`LUDIC_PKG_PROXY`, `x vendor`) with URL/git later; transitive deps; **namespace collision = compile-time error**; no build-time code execution. **Phase 1 — the two hardcoded hooks are now registries** (commit 7cbd5d1): - **Namespaces:** `@Namespace(Name)` on a function opens `Name.method(…)` → dispatches to the bare `name_method` through the same generic path the core namespaces use (applied *after* them, so it never shadows a core one). - **Engine systems:** `@EngineSystem(Component, Phase)` registers an engine-owned system the frame loop runs each phase when the component is present — the package-declarable form of the built-in SpriteAnim/Motion/Light2D systems, reading components by name through the reflection ABI (unused registration ⇒ byte-identical). `uses_engine_systems` / `emit_engine_systems_for_phase` are now driven by that registry, with the core three seeded in their historical order. - Both annotations are keyword-free (no grammar/vocab churn). **Acceptance:** the engine-system hook migrated onto the registry with byte-identical IR (anim_ecs/light_ecs/snake) and 87/0 golden renders; the C-free bootstrap fixpoint holds. Proven end-to-end (hermetic, source path): a package registers `Score`+`esys_score` and `Coach.bonus()`; a consumer imports it and prints `4 99` with no compiler edit. **Binary-mod escape hatch** — #62 only asked to document it; #64 built it in full: `--emit-module` + `ludic_register_system` + `@System(Phase)` + `x build-lib` + prebuilt consumption, over the EV0–EV7 C-ABI + dynamic reflection (native-only, second-class, by design). Docs: `docs/PACKAGES.md` + per-annotation pages for `@Namespace` / `@EngineSystem` / `@System`. ## Deferred (refinements, not blockers — happy to file a follow-up) - `import pkg "name"` name-based sugar — today a package is consumed by import path via the `ludic_modules/` resolution fallback (#63); the short-name form is convenience over that. - `min_compiler` manifest field + check. - Migrating the remaining ~15 **core** stdlib namespaces onto the generic path — the engine-system hook was fully migrated (byte-identical); for namespaces the generic path is proven and packages ride it, but the core namespace blocks stay hardcoded **by design** (byte-identity + determinism; migrating them risks the golden renders for no functional gain). - Deep LSP/grammar completion for package-provided namespaces (doc pages + inventory are in; live manifest-driven LSP is polish). - Shipping #58–#61 as the first packages (their own issues). Closing as done — the prerequisite this issue exists for is delivered.

orkun closed this issue

2026-09-01 11:32:37 +02:00