feat: prebuilt binary packages — closed-source functions/systems/components over the reflection ABI #64

New issue

Closed

opened 2026-09-01 06:29:51 +02:00 by orkun · 1 comment

orkun commented

2026-09-01 06:29:51 +02:00

Owner

Summary

Follow-up to #63. Source packages compile into the consumer (AOT, first-class compile-time ECS) — shipped. This issue adds prebuilt binary packages: a developer ships a compiled artifact and a consumer uses its exported functions, systems and components without the source, resolved through the stable reflection C-ABI.

Decision (chosen over a source-recompile hybrid): full binary via the reflection ABI — true closed-source machine-code modules. This is native-only (no wasm/cross-compile), second-class ECS (by-name dynamic components, an indirect call per registered system), and not deterministic-first-class — the documented escape hatch, exactly as the controllers roadmap (#57–#62) framed binary distribution.

Model

A binary module is a .dylib that, at load, registers its components and systems against the host's world through the C-ABI and exposes its functions as symbols the consumer binds:

Components — ludic_register_prop(name, nfields) (already exists, EV7) + world_get/set/has/attach_dyn. A module declares its components dynamically; the host owns the single world.
Systems — a new ludic_register_system(fn, phase) + a runtime system registry the host's phase loop dispatches (mirrors the existing EV6 foreign event-listener array). A module registers a void() system into a phase; it runs each frame after the host's own handlers.
Functions — compiled to symbols; the consumer binds them with extern function … = "sym". Resolved at dylib load.
Load-time registration — the module carries a global constructor that calls the registration ABI; -undefined dynamic_lookup binds ludic_* back to the host image.

Plan (layered, each independently testable)

Runtime ABI — ludic_register_system(fn, phase_id) + a stable phase-id enum + dynamic dispatch appended to every phase in the frame loop (emit_world/emit_game). Prove in-process.
Module compile mode — ludicc --emit-module: no main/world-table/runtime; emit the module's functions, declare the host reflection ABI it references, and a @llvm.global_ctors entry calling a conventional init. Build as a dylib with dynamic_lookup.
Package manager — x build-lib (compile a package to a per-target dylib in the content-addressed store + an interface stub of extern signatures) and prebuilt consumption (kind prebuilt links the artifact for the build target, rpath into the store; hard error when the target is missing).

Non-goals

wasm/cross-compile for binary modules (source packages remain the portable path).
Making binary systems first-class in the deterministic/replay world (they ride the reflection ABI by design).

## Summary Follow-up to #63. Source packages compile into the consumer (AOT, first-class compile-time ECS) — shipped. This issue adds **prebuilt binary packages**: a developer ships a compiled artifact and a consumer uses its exported **functions, systems and components** without the source, resolved through the stable reflection C-ABI. Decision (chosen over a source-recompile hybrid): **full binary via the reflection ABI** — true closed-source machine-code modules. This is native-only (no wasm/cross-compile), second-class ECS (by-name dynamic components, an indirect call per registered system), and not deterministic-first-class — the documented escape hatch, exactly as the controllers roadmap (#57–#62) framed binary distribution. ## Model A binary module is a `.dylib` that, at load, registers its components and systems against the host's world through the C-ABI and exposes its functions as symbols the consumer binds: - **Components** — `ludic_register_prop(name, nfields)` (already exists, EV7) + `world_get/set/has/attach_dyn`. A module declares its components dynamically; the host owns the single world. - **Systems** — a new `ludic_register_system(fn, phase)` + a runtime system registry the host's phase loop dispatches (mirrors the existing EV6 foreign event-listener array). A module registers a `void()` system into a phase; it runs each frame after the host's own handlers. - **Functions** — compiled to symbols; the consumer binds them with `extern function … = "sym"`. Resolved at dylib load. - **Load-time registration** — the module carries a global constructor that calls the registration ABI; `-undefined dynamic_lookup` binds `ludic_*` back to the host image. ## Plan (layered, each independently testable) 1. **Runtime ABI** — `ludic_register_system(fn, phase_id)` + a stable phase-id enum + dynamic dispatch appended to every phase in the frame loop (emit_world/emit_game). Prove in-process. 2. **Module compile mode** — `ludicc --emit-module`: no `main`/world-table/runtime; emit the module's functions, `declare` the host reflection ABI it references, and a `@llvm.global_ctors` entry calling a conventional init. Build as a dylib with dynamic_lookup. 3. **Package manager** — `x build-lib` (compile a package to a per-target dylib in the content-addressed store + an interface stub of `extern` signatures) and prebuilt consumption (`kind prebuilt` links the artifact for the build target, rpath into the store; hard error when the target is missing). ## Non-goals - wasm/cross-compile for binary modules (source packages remain the portable path). - Making binary systems first-class in the deterministic/replay world (they ride the reflection ABI by design).

orkun referenced this issue from a commit

2026-09-01 06:52:20 +02:00

feat(compiler): dynamic system registration + --emit-module for binary packages (#64)

orkun referenced this issue from a commit

2026-09-01 06:52:20 +02:00

feat(pkg): x build-lib / prebuilt consumption — binary packages end-to-end (#64)

orkun commented

2026-09-01 06:52:34 +02:00

Author

Owner

Implemented and shipped to main (commits e1537d2 compiler, 075a7b1 package manager).

A developer can now ship a compiled artifact whose exported functions, systems and components a consumer uses without the source — full binary via the reflection ABI, exactly as decided.

Compiler (e1537d2):

ludic_register_system(fn, phase) + a registry the frame loop dispatches after each phase's own handlers — the systems analogue of the existing ludic_register_prop (dynamic components) and a mirror of the EV6 foreign event-listener array.
--emit-module: compiles a package to a position-independent module — no main, no world table — that declares the host reflection ABI it calls and carries a load-time constructor registering its @System(Phase) functions and running module_init. Built as a dylib with -undefined dynamic_lookup, it binds ludic_* back to the host at load.
@System(Phase) annotation marks a module system (the compiler supplies its address — Ludic source cannot take one).
The reflection ABI is now emitted for every ECS program, so any game hosts binary modules with no flag; unused defs dead-strip at -O2, so golden renders stay byte-identical and the C-free bootstrap fixpoint holds.

Package manager (075a7b1):

x build-lib <module.ludic> → lib/<target>/lib<name>.dylib (with an @rpath install name).
kind prebuilt deps resolve + link like any dependency; x link-flags prints the clang flags for any build system (x app splices them in-repo). Missing the build target is a hard error.

Proven end-to-end (hermetic, macOS-gated in x test-pkg): a module exporting a component + an @System(Update) + a function is built with x build-lib, fetched as a prebuilt dependency, and linked into a consumer game that never saw its source — the module's system mutates the shared world and its function is callable (output 3 42). Package suite 17/0; full suite 87/0; fixpoint intact.

Boundaries (as designed): native/dylib-only (no wasm/cross-compile), second-class ECS (dynamic by-name components, one indirect call per registered system), not part of the deterministic/replay core. Source packages (#63) remain the portable, first-class path. Docs: docs/PACKAGES.md.

Closing as done.

Implemented and shipped to `main` (commits e1537d2 compiler, 075a7b1 package manager). A developer can now ship a **compiled artifact** whose exported **functions, systems and components** a consumer uses without the source — full binary via the reflection ABI, exactly as decided. **Compiler (e1537d2):** - `ludic_register_system(fn, phase)` + a registry the frame loop dispatches after each phase's own handlers — the systems analogue of the existing `ludic_register_prop` (dynamic components) and a mirror of the EV6 foreign event-listener array. - `--emit-module`: compiles a package to a position-independent module — no `main`, no world table — that declares the host reflection ABI it calls and carries a load-time constructor registering its `@System(Phase)` functions and running `module_init`. Built as a dylib with `-undefined dynamic_lookup`, it binds `ludic_*` back to the host at load. - `@System(Phase)` annotation marks a module system (the compiler supplies its address — Ludic source cannot take one). - The reflection ABI is now emitted for every ECS program, so any game hosts binary modules with no flag; unused defs dead-strip at `-O2`, so golden renders stay byte-identical and the C-free bootstrap fixpoint holds. **Package manager (075a7b1):** - `x build-lib <module.ludic>` → `lib/<target>/lib<name>.dylib` (with an `@rpath` install name). - `kind prebuilt` deps resolve + link like any dependency; `x link-flags` prints the clang flags for any build system (`x app` splices them in-repo). Missing the build target is a hard error. **Proven end-to-end** (hermetic, macOS-gated in `x test-pkg`): a module exporting a component + an `@System(Update)` + a function is built with `x build-lib`, fetched as a prebuilt dependency, and linked into a consumer game that never saw its source — the module's system mutates the shared world and its function is callable (output `3 42`). Package suite 17/0; full suite 87/0; fixpoint intact. **Boundaries (as designed):** native/dylib-only (no wasm/cross-compile), second-class ECS (dynamic by-name components, one indirect call per registered system), not part of the deterministic/replay core. Source packages (#63) remain the portable, first-class path. Docs: `docs/PACKAGES.md`. Closing as done.

orkun closed this issue

2026-09-01 06:52:34 +02:00

orkun referenced this issue from a commit

2026-09-01 11:32:05 +02:00

feat(compiler): package-declarable engine systems + namespaces (#62)

orkun referenced this issue

2026-09-01 11:32:36 +02:00

Proposal: package/module distribution + package-declarable components & engine-systems (prerequisite for controller libs #57) #62