feat(pkg): phase 15 - a package can carry a native library

native "<target>" "<path>" in a package's package.ludic; the compiler records the libraries of
every package a program imports and writes them into the IR (; ludic-native:), so ludicc -o,
ludic build, ludic test and ludic bundle all link one list. macOS: an rpath to the package and to
Contents/Frameworks, where ludic bundle copies and signs each library and drops the build
machine's rpath. Windows: the import library, the .dll copied beside the exe (--natives-out for
the bundle). tools/native/lib.sh builds from a pinned, checksummed source with clang on both
machines; ludic.nativeecho is the worked example; the shim rules are in packages/README.md.
Linked at build time rather than dlopen (docs/PACKAGES.md says why). Reseeded.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Orkun ÇAKILKAYA 2026-09-26 23:35:45 +03:00
parent 5df0747642
commit 5d3a799e33
25 changed files with 54474 additions and 51707 deletions

1
.gitattributes vendored Normal file
View file

@ -0,0 +1 @@
packages/*/lib/** filter=lfs diff=lfs merge=lfs -text

3
.gitignore vendored
View file

@ -57,3 +57,6 @@ assets/polyhaven/textures/
assets/polyhaven/models/ assets/polyhaven/models/
# `ludic run` beside an example writes its binary into a build/ there # `ludic run` beside an example writes its binary into a build/ there
examples/**/build/ examples/**/build/
# a package native/build.sh writes its objects under the package (phase 15)
packages/*/build/

View file

@ -0,0 +1,10 @@
bump: minor
type: feat
**A package can carry a native library.** `native "<target>" "<path>"` in a package's
`package.ludic` names a C/C++ library per target (`macos-arm64`, `windows-x64`, ...). The
compiler records the libraries of every package a program imports, and every link - `ludicc -o`,
`ludic build`, `ludic test`, `ludic bundle` - links them: on macOS with an rpath to the package
and to `Contents/Frameworks`, where `ludic bundle` places and signs them; on Windows through the
import library, the `.dll` copied beside the executable. `tools/native/lib.sh` builds a library
from a pinned, checksummed source; `ludic.nativeecho` is the worked example, and
`packages/README.md` says what a shim may pass across.

View file

@ -235,6 +235,47 @@ clang -O2 game.ll $(ludic link-flags) -o game
not ship the build target, `ludic get` fails — build from source instead where the not ship the build target, `ludic get` fails — build from source instead where the
package offers it. package offers it.
## Native libraries - a source package that carries a C/C++ library (phase 15)
A source package may link a native library of its own - a physics engine, a navmesh builder, an
audio renderer - by naming it per target in its `package.ludic`:
```
native "macos-arm64" "lib/macos-arm64/libjoltc.dylib"
native "windows-x64" "lib/windows-x64/joltc.dll"
```
The targets are `macos-arm64`, `macos-x64`, `windows-x64` and `linux-x64`
(`$LUDIC_NATIVE_TARGET` names another). The package's Ludic reaches the library through
`extern function` declarations, which only a package (or the runtime) may call (L7).
**Linked at build time, not opened with `dlopen`.** When the compiler loads a file from a
package, it reads that package's manifest once and keeps the libraries for the build's target;
they are written into the IR as `; ludic-native: <path>` lines, and every link reads that one
list - `ludicc -o`, `ludic build`, `ludic test` and `ludic bundle`. A program that imports no
such package links nothing new. Linking was chosen over a Vulkan-style loader because a missing
or mismatched library then fails at start-up, with the loader's own message naming it, rather
than at the first call deep in a frame; because an `extern function` is all the binding needs (no
generated thunk table per library); and because nothing here is optional the way Vulkan is - a
game that imports the physics package cannot run without it.
- **macOS**: the dylib's install name is `@rpath/lib<name>.dylib`. A build gets an rpath to the
package's `lib/<target>/` (so `ludic run` and `ludic test` work in place) and one to
`@executable_path/../Frameworks`. `ludic bundle` copies each library into
`Contents/Frameworks` - never `Contents/MacOS`, which holds mach-o executables only (a Velopack
update strips a detached signature) - removes the build machine's rpath from the executable,
and signs every library before the app.
- **Windows**: the `.dll` is linked through its import library, `<name>.lib` beside it; `ludicc`
copies the `.dll` beside the executable it writes, and `ludic bundle` beside the game's `.exe`.
**Built here from a pinned source.** A package's `native/build.sh` fetches the upstream tag,
checks its SHA-256, compiles the library and the package's shim with clang, and writes
`lib/<target>/` (committed through Git LFS). `tools/native/lib.sh` is what every such script
shares - `native_fetch`, `native_link`, the target and compiler - and it runs unchanged in Git
Bash on Windows with the LLVM installer's clang. The shim's rules (what may cross, how a
callback comes back) are in [packages/README.md](../packages/README.md#native-libraries);
`ludic.nativeecho` is the worked example.
## Offline / hermetic builds ## Offline / hermetic builds
`ludic vendor` copies the resolved packages out of the store into `./vendor`. Build `ludic vendor` copies the resolved packages out of the store into `./vendor`. Build

View file

@ -52,6 +52,39 @@ section. The rules are in [ludic.base](ludic.base/README.md).
| [ludic.lab](ludic.lab/README.md) | the visual lab: a scene on a lit plate, fixed cameras, PNG shots and a contact sheet | | [ludic.lab](ludic.lab/README.md) | the visual lab: a scene on a lit plate, fixed cameras, PNG shots and a contact sheet |
| [ludic.core](ludic.core/) | the canonical engine-ABI components | | [ludic.core](ludic.core/) | the canonical engine-ABI components |
| [ludic.prefs](ludic.prefs/) | a small `key=value` store for preferences and records | | [ludic.prefs](ludic.prefs/) | a small `key=value` store for preferences and records |
| [ludic.nativeecho](ludic.nativeecho/README.md) | the worked example of a package carrying a native library: a C sum over a slice, callbacks drained as facts |
## Native libraries
A package may carry a C or C++ library (`native` lines in its `package.ludic`,
[docs/PACKAGES.md](../docs/PACKAGES.md#native-libraries---a-source-package-that-carries-a-cc-library-phase-15)).
The library is never bound directly: a thin C shim in the package's `native/shim/` is, and every
shim keeps the same shape so the Ludic side stays safe.
- **Only `int`, `long`, `float` and opaque handles cross.** A Ludic `float` is a C `float`, a
`long` an `int64_t`, a `pointer` a `void *`. No struct is passed by value, no C++ type, no
exception: the shim catches and turns it into a return code.
- **A handle is the library's pointer, kept in the package's state and never exported.** The
game names what the package hands it - an int id - and the package maps it to the handle.
- **Nothing calls back into Ludic.** No function pointer crosses the boundary. What a library
reports through a callback (a contact, a finished path, a mixed buffer) the shim records in a
buffer of its own, and the package drains it after the call returns (`ne_scan` then
`ne_drain`), pushing each record onto its facts queue. A record that did not fit is counted,
not dropped silently.
- **Bulk data crosses as a slice the package owns.** A `[]float` or `[]int` passed where the
shim takes a pointer is its data; the package makes it once and reuses it (L7), and passes its
length beside it.
- **Errors are return codes.** 0 is success and a negative number says what failed; nothing
aborts, nothing prints.
- **Threads a library starts stay inside C** (a job system, an audio device). The Ludic side is
called on the game's thread only.
- **The symbols are prefixed** with the package's short name (`ne_`, `jph_`), exported explicitly
(`__attribute__((visibility("default")))`, `__declspec(dllexport)`), and everything else is
hidden.
`native/build.sh` builds `lib/<target>/` from the pinned upstream source and the shim through
[tools/native/lib.sh](../tools/native/lib.sh); run it on the Mac and, in Git Bash, on the PC.
The package's tests run the real library - there is no fake for it.
## The builtin controllers ## The builtin controllers

View file

@ -0,0 +1,26 @@
# ludic.nativeecho
The worked example of a package that carries a native library (phase 15). It exists to be copied
and to be tested: nothing in a game needs it.
```ludic
import "ludic.nativeecho"
```
- `package.ludic` names the library per target (`native "macos-arm64" "lib/macos-arm64/libnativeecho.dylib"`).
- `native/shim/nativeecho.c` is the whole library: a sum over a float slice, a scan that
"calls back" once per value over a threshold, and the drain the package empties those records
through.
- `native/build.sh` builds `lib/<target>/` through `tools/native/lib.sh` (no upstream to fetch).
- `native.ludic` declares the symbols; `echo.ludic` is the state, the verbs and the facts.
| | |
| --- | --- |
| `EchoState` | the slices that cross to C (made once), and the facts queue |
| `echo_put(st, x) -> bool`, `echo_clear(st)` | fill the slice |
| `echo_sum(st) -> float` | the sum, in C |
| `echo_scan(st, over) -> int` | C records each value over `over`; they arrive on `echo_facts(st)` as `EchoFact { index, value }` |
| `echo_lost() -> int`, `echo_twice(x: long) -> long` | the lost-record count, and a 64-bit round trip |
The rules it keeps are the shim rules in [packages/README.md](../README.md#native-libraries).
`ludic test packages/ludic.nativeecho` runs it against the real library.

View file

@ -0,0 +1,69 @@
# echo.ludic - the state, its verbs and its facts. The slices crossing to C are made once and
# reused (L7), and what C recorded is drained into the queue after each call returns.
export const ECHO_CAP: int = 64
export property EchoFact {
index: int = 0 # which element went over
value: float = 0.0
}
export state EchoState {
ec_xs: []float = null
ec_n: int = 0
ec_kinds: []int = null
ec_values: []float = null
ec_facts: Queue<EchoFact> = null
}
function ec_ready(echo_st: mut EchoState) -> void {
if echo_st.ec_xs == null { echo_st.ec_xs = floats(ECHO_CAP) }
if echo_st.ec_kinds == null { echo_st.ec_kinds = words(ECHO_CAP) }
if echo_st.ec_values == null { echo_st.ec_values = floats(ECHO_CAP) }
}
# what went over, oldest first
export function echo_facts(echo_st: mut EchoState) -> Queue<EchoFact> {
if echo_st.ec_facts == null { echo_st.ec_facts = queue_new("nativeecho.facts") }
return echo_st.ec_facts
}
# put a value in the next slot; false when the slice is full
export function echo_put(echo_st: mut EchoState, x: float) -> bool {
ec_ready(echo_st)
if echo_st.ec_n >= ECHO_CAP { return false }
echo_st.ec_xs[echo_st.ec_n] = x
echo_st.ec_n += 1
return true
}
export function echo_clear(echo_st: mut EchoState) -> void { echo_st.ec_n = 0 }
# the sum of what was put, worked out in C
export function echo_sum(echo_st: mut EchoState) -> float {
ec_ready(echo_st)
return ne_sum(echo_st.ec_xs, echo_st.ec_n)
}
# C walks the values and records each one over `over`; the records come back as facts
export function echo_scan(echo_st: mut EchoState, over: float) -> int {
ec_ready(echo_st)
let hits = ne_scan(echo_st.ec_xs, echo_st.ec_n, over)
ec_drain(echo_st)
return hits
}
function ec_drain(echo_st: mut EchoState) -> void {
let n = ne_drain(echo_st.ec_kinds, echo_st.ec_values, ECHO_CAP)
for i in 0 .. n {
let f = new EchoFact
f.index = echo_st.ec_kinds[i]
f.value = echo_st.ec_values[i]
q_push(echo_facts(echo_st), f)
}
}
# how many records C had to drop because nobody drained them
export function echo_lost() -> int { return ne_lost_count() }
# a 64-bit value there and back
export function echo_twice(x: long) -> long { return ne_twice(x) }

View file

@ -0,0 +1,7 @@
# ludic.nativeecho - the worked example of a package carrying a native library (phase 15): a C
# sum over a slice it owns, and a C scan whose "callbacks" come back as facts on its queue
module ludic_nativeecho uses ludic_base
numbers float
import "ludic.base"
import "native.ludic"
import "echo.ludic"

BIN
packages/ludic.nativeecho/lib/macos-arm64/libnativeecho.dylib (Stored with Git LFS) Executable file

Binary file not shown.

View file

@ -0,0 +1,7 @@
# native.ludic - the shim's symbols (native/shim/nativeecho.c), in the types that cross:
# int, long, float and a slice's data as a pointer. Nothing here is exported.
extern function ne_sum(xs: pointer, n: int) -> float = "ne_sum"
extern function ne_scan(xs: pointer, n: int, over: float) -> int = "ne_scan"
extern function ne_drain(kinds: pointer, values: pointer, cap: int) -> int = "ne_drain"
extern function ne_lost_count() -> int = "ne_lost_count"
extern function ne_twice(x: long) -> long = "ne_twice"

View file

@ -0,0 +1,10 @@
#!/bin/sh
# builds lib/<target>/ for ludic.nativeecho: no upstream library, only the shim
# (tools/native/lib.sh; a package that wraps a library adds native_fetch first - see ludic.physics)
set -eu
PKG="$(cd "$(dirname "$0")/.." && pwd)"
. "$PKG/../../tools/native/lib.sh"
OBJ="$PKG/build/native"
mkdir -p "$OBJ"
"$(native_cc)" $(native_cflags) -c "$PKG/native/shim/nativeecho.c" -o "$OBJ/nativeecho.o"
native_link "$PKG" nativeecho "$OBJ/nativeecho.o"

View file

@ -0,0 +1,59 @@
/* nativeecho - the shape every shim in packages/ has (packages/README.md, "Native libraries"):
int, long, float and opaque handles cross; no struct by value, no callback into Ludic. What a
library would report through a callback is recorded here and drained after the call returns. */
#include <stdint.h>
#if defined(_WIN32)
#define NE_API __declspec(dllexport)
#else
#define NE_API __attribute__((visibility("default")))
#endif
#define NE_CAP 64
static int ne_kinds[NE_CAP];
static float ne_values[NE_CAP];
static int ne_count = 0;
static int ne_lost = 0;
/* the sum of n floats the package owns */
NE_API float ne_sum(const float *xs, int n) {
float s = 0.0f;
for (int i = 0; i < n; i++) s += xs[i];
return s;
}
/* what a callback would have reported: kept until drained, counted as lost past the cap */
NE_API int ne_record(int kind, float value) {
if (ne_count >= NE_CAP) { ne_lost++; return -1; }
ne_kinds[ne_count] = kind;
ne_values[ne_count] = value;
ne_count++;
return 0;
}
/* a library's work that "calls back" once per element over a threshold */
NE_API int ne_scan(const float *xs, int n, float over) {
int hits = 0;
for (int i = 0; i < n; i++) {
if (xs[i] > over) { ne_record(i, xs[i]); hits++; }
}
return hits;
}
/* copy up to cap pending records into two slices the package owns; returns how many */
NE_API int ne_drain(int *kinds, float *values, int cap) {
int n = ne_count < cap ? ne_count : cap;
for (int i = 0; i < n; i++) { kinds[i] = ne_kinds[i]; values[i] = ne_values[i]; }
for (int i = n; i < ne_count; i++) {
ne_kinds[i - n] = ne_kinds[i];
ne_values[i - n] = ne_values[i];
}
ne_count -= n;
return n;
}
NE_API int ne_lost_count(void) { return ne_lost; }
/* a 64-bit value round trip: handles and counts wider than 32 bits cross as long */
NE_API int64_t ne_twice(int64_t x) { return x * 2; }

View file

@ -0,0 +1,8 @@
# ludic.nativeecho - the worked example of a package that carries a native library (phase 15):
# a C function that sums a float slice, and one that records what a callback would have said
# into a buffer the package drains. Uses ludic.base and nothing else.
package "ludic.nativeecho"
version "0.1.0"
kind source
native "macos-arm64" "lib/macos-arm64/libnativeecho.dylib"
native "windows-x64" "lib/windows-x64/nativeecho.dll"

View file

@ -0,0 +1,41 @@
# echo_test.ludic - the native library is linked and called: a sum over a slice, a scan whose
# records come back as facts in order, a 64-bit value, and a drain that leaves nothing behind
import "ludic.nativeecho"
import "ludic.base"
program EchoTest {
numbers float
test "a sum is worked out in C over the package's own slice" (echo_st: mut EchoState) {
echo_put(echo_st, 1.5)
echo_put(echo_st, 2.0)
echo_put(echo_st, 0.25)
expect(echo_sum(echo_st) == 3.75)
}
test "what C would have called back arrives as facts, in order" (echo_st: mut EchoState) {
echo_put(echo_st, 1.0)
echo_put(echo_st, 9.0)
echo_put(echo_st, 2.0)
echo_put(echo_st, 7.5)
expect(echo_scan(echo_st, 5.0) == 2)
let fs = q_drain(echo_facts(echo_st))
expect(len(fs) == 2)
expect(fs[0].index == 1)
expect(fs[0].value == 9.0)
expect(fs[1].index == 3)
expect(fs[1].value == 7.5)
expect(echo_scan(echo_st, 100.0) == 0)
expect(q_len(echo_facts(echo_st)) == 0)
expect(echo_lost() == 0)
}
test "a long crosses whole" {
expect(echo_twice(long(3000000000)) == long(6000000000))
}
test "the slice holds what it was made for and no more" (echo_st: mut EchoState) {
for i in 0 .. ECHO_CAP { expect(echo_put(echo_st, 1.0)) }
expect(not echo_put(echo_st, 1.0))
expect(echo_sum(echo_st) == float(ECHO_CAP))
}
}

View file

@ -439,7 +439,7 @@ function emit_cov_runtime() -> void {
# drivers read); with a path it is written to that file so ludicc can hand it to # drivers read); with a path it is written to that file so ludicc can hand it to
# clang itself. # clang itself.
function ir_flush(path: pointer) -> bool { function ir_flush(path: pointer) -> bool {
let h = buf_str(head) let h = native_ir_lines() + buf_str(head)
let c = buf_str(code) let c = buf_str(code)
if (path == null) { # raw IR to stdout (no trailing newline) if (path == null) { # raw IR to stdout (no trailing newline)
let out = file_stdout() let out = file_stdout()

View file

@ -0,0 +1,105 @@
# natives.ludic - a package can carry a native library (phase 15).
#
# A package's package.ludic says `native "<target>" "<path>"` for each library it links, per
# target. When an import is found in a package, its manifest is read once and the libraries for
# this build's target are kept, as absolute paths. ir_flush writes them into the IR as
# `; ludic-native: <path>` lines, so every link - ludicc's own, `ludic build`, `ludic test` and
# `ludic bundle` - reads one list, and a program that imports no such package links nothing new.
var g_native_libs: []pointer = new []pointer
var g_native_pkgs: []pointer = new []pointer
# the target a `native` line names: windows-x64, macos-arm64 or linux-x64.
# $LUDIC_NATIVE_TARGET names another (an Intel Mac says macos-x64).
function native_target() -> pointer {
let env = getenv("LUDIC_NATIVE_TARGET")
if (env != null) { return env }
if g_target_win { return "windows-x64" }
if Fs.is_dir("/System/Library") { return "macos-arm64" }
return "linux-x64"
}
# the words of one manifest line, a quoted word as one word without its quotes
function native_words(line: pointer) -> []pointer {
let out = new []pointer
let n = len(line)
var i = 0
while i < n {
while (i < n) and ((line[i] == ' ') or (line[i] == 9)) { i += 1 }
if (i >= n) or (line[i] == '#') { return out }
if line[i] == '"' {
let s = i + 1
i = s
while (i < n) and (line[i] != '"') { i += 1 }
push(out, line[s..i])
i += 1
} else {
let s = i
while (i < n) and (line[i] != ' ') and (line[i] != 9) { i += 1 }
push(out, line[s..i])
}
}
return out
}
function native_have(xs: []pointer, s: pointer) -> bool {
var i = 0
while i < len(xs) { if xs[i] == s { return true }; i += 1 }
return false
}
# `full` was found as `rel` under a package root (ludic_modules or $LUDIC_HOME/packages):
# read that package's manifest once and keep its libraries for this target
function native_note_pkg(full: pointer, rel: pointer) -> void {
var n = 0
while (n < len(rel)) and (rel[n] != '/') { n += 1 }
let root = full[0..len(full) - len(rel) + n] + "/"
if native_have(g_native_pkgs, root) { return }
push(g_native_pkgs, root)
let txt = read_file(root + "package.ludic")
if (txt == null) { return }
let want = native_target()
var i = 0
let tn = len(txt)
while i < tn {
var e = i
while (e < tn) and (txt[e] != '\n') { e += 1 }
var le = e
if (le > i) and (txt[le - 1] == 13) { le -= 1 }
let ws = native_words(txt[i..le])
if (len(ws) == 3) and (ws[0] == "native") and (ws[1] == want) {
let lib = root + ws[2]
if not native_have(g_native_libs, lib) { push(g_native_libs, lib) }
}
i = e + 1
}
}
# the `; ludic-native:` lines ir_flush puts at the head of the IR
function native_ir_lines() -> pointer {
var s = ""
var i = 0
while i < len(g_native_libs) { s = s + `; ludic-native: {g_native_libs[i]}\n`; i += 1 }
return s
}
# what ludicc's own link adds: on macOS the library and an rpath to its directory (and to where
# a bundle keeps it); on Windows the import library beside the .dll, the .dll copied beside the exe
function native_link_args(out: pointer) -> pointer {
var s = ""
var i = 0
while i < len(g_native_libs) {
let lib = g_native_libs[i]
let dir = dir_of(lib)
if g_target_win {
let n = len(lib)
s = s + ` "{lib[0..n - 4]}.lib"`
var od = dir_of(out)
if len(od) == 0 { od = "." }
run(`copy /y "{win_path(lib)}" "{win_path(od)}" >nul 2>nul`)
} else {
s = s + ` "{lib}" -Wl,-rpath,{dir} -Wl,-rpath,@executable_path/../Frameworks`
}
i += 1
}
return s
}

View file

@ -1228,6 +1228,7 @@ function do_import(rel: pointer) -> void {
if is_runtime_path(rel) or not beside or (beside and unsafe_trusted(g_parse_file)) { trust_file(full) } if is_runtime_path(rel) or not beside or (beside and unsafe_trusted(g_parse_file)) { trust_file(full) }
if beside and not is_runtime_path(rel) and not (module_of(g_parse_file) == "") { module_set(full, module_of(g_parse_file)) } if beside and not is_runtime_path(rel) and not (module_of(g_parse_file) == "") { module_set(full, module_of(g_parse_file)) }
if not beside and not is_runtime_path(rel) { pkg_file_set(full, pkg_name_of(rel)) } # modules.ludic if not beside and not is_runtime_path(rel) { pkg_file_set(full, pkg_name_of(rel)) } # modules.ludic
if not beside and not is_runtime_path(rel) and (src != null) { native_note_pkg(full, rel) } # natives.ludic
if beside and not is_runtime_path(rel) and not (pkg_of_file(g_parse_file) == "") { pkg_file_set(full, pkg_of_file(g_parse_file)) } if beside and not is_runtime_path(rel) and not (pkg_of_file(g_parse_file) == "") { pkg_file_set(full, pkg_of_file(g_parse_file)) }
if (src == null) { perr(`cannot open import {full}`) } if (src == null) { perr(`cannot open import {full}`) }
# the audio runtime can arrive through atlas.ludic's own import or the Assets # the audio runtime can arrive through atlas.ludic's own import or the Assets

File diff suppressed because it is too large Load diff

File diff suppressed because it is too large Load diff

View file

@ -138,6 +138,7 @@ entry {
var title = null # --title <name>: game_title() - the window's first title - instead of the program name var title = null # --title <name>: game_title() - the window's first title - instead of the program name
var gui = false # --gui: a Windows GUI-subsystem program (no console behind the window) var gui = false # --gui: a Windows GUI-subsystem program (no console behind the window)
var links = new []pointer # --link <file>: an extra input to the link (a compiled resource) var links = new []pointer # --link <file>: an extra input to the link (a compiled resource)
var natives_out = null # --natives-out <file>: the package libraries linked, one a line (ludic bundle)
g_coverage = false # --coverage: instrument each statement with a per-line hit counter g_coverage = false # --coverage: instrument each statement with a per-line hit counter
var ai = 1 var ai = 1
@ -166,6 +167,7 @@ entry {
else if a == "--target" { ai += 1; if ai < arg_count() { target = arg(ai) } } else if a == "--target" { ai += 1; if ai < arg_count() { target = arg(ai) } }
else if a == "--gui" { gui = true } else if a == "--gui" { gui = true }
else if a == "--title" { ai += 1; if ai < arg_count() { title = arg(ai) } } else if a == "--title" { ai += 1; if ai < arg_count() { title = arg(ai) } }
else if a == "--natives-out" { ai += 1; if ai < arg_count() { natives_out = arg(ai) } }
else if a == "--link" { ai += 1; if ai < arg_count() { push(links, arg(ai)) } } else if a == "--link" { ai += 1; if ai < arg_count() { push(links, arg(ai)) } }
else if a == "-o" { ai += 1; if ai < arg_count() { out = arg(ai) } } else if a == "-o" { ai += 1; if ai < arg_count() { out = arg(ai) } }
else if a[0] == '-' { else if a[0] == '-' {
@ -359,6 +361,8 @@ entry {
# console-subsystem one opens a console window behind the game. mainCRTStartup keeps # console-subsystem one opens a console window behind the game. mainCRTStartup keeps
# the C runtime's ordinary main() entry rather than WinMain. # the C runtime's ordinary main() entry rather than WinMain.
if g_target_win and gui { cmd = `{cmd} -Wl,/subsystem:windows -Wl,/entry:mainCRTStartup` } if g_target_win and gui { cmd = `{cmd} -Wl,/subsystem:windows -Wl,/entry:mainCRTStartup` }
# a package's native libraries (natives.ludic)
cmd = cmd + native_link_args(out)
var li = 0 var li = 0
while li < len(links) { cmd = `{cmd} "{links[li]}"`; li += 1 } while li < len(links) { cmd = `{cmd} "{links[li]}"`; li += 1 }
cmd = `{cmd} -o {out}` cmd = `{cmd} -o {out}`
@ -371,6 +375,13 @@ entry {
let rc = run(line) let rc = run(line)
if not save { remove_file(ll) } if not save { remove_file(ll) }
if rc != 0 { die("ludicc: link failed\n") } if rc != 0 { die("ludicc: link failed\n") }
if natives_out != null {
var nl = ""
var ni = 0
while ni < len(g_native_libs) { nl = nl + g_native_libs[ni] + "\n"; ni += 1 }
let nf = file_open(natives_out, "wb")
if nf != null { file_write(nf, nl, len(nl)); file_close(nf) }
}
# run mode: execute the binary we just built and forward its exit code. # run mode: execute the binary we just built and forward its exit code.
# system() hands back a wait status on POSIX and the exit code itself on Windows. # system() hands back a wait status on POSIX and the exit code itself on Windows.

View file

@ -27,6 +27,8 @@ function ensure_ludicc() -> void {
# rendered to build/out.ppm from piped input — deterministic output for tests). # rendered to build/out.ppm from piped input — deterministic output for tests).
# Returns true on success. `ludic build`, `ludic run` and the contributor tool all # Returns true on success. `ludic build`, `ludic run` and the contributor tool all
# go through here. # go through here.
# the native libraries (phase 15) the last compile_app linked, for `ludic bundle` to place
var g_built_natives: []pointer = new []pointer
function compile_app(src: pointer, out: pointer, mode: int, save: bool) -> bool { function compile_app(src: pointer, out: pointer, mode: int, save: bool) -> bool {
ensure_ludicc() # in a checkout: assemble the seed if bin/ludicc is missing or stale ensure_ludicc() # in a checkout: assemble the seed if bin/ludicc is missing or stale
let home = ludic_home() let home = ludic_home()
@ -52,7 +54,8 @@ function compile_app(src: pointer, out: pointer, mode: int, save: bool) -> bool
if mode == 2 { if mode == 2 {
if not shq(`{ludicc()} --headless{title_flag()}{unsafe_flag()} {src} --emit-llvm -o {ll}`) { return false } if not shq(`{ludicc()} --headless{title_flag()}{unsafe_flag()} {src} --emit-llvm -o {ll}`) { return false }
if not shq(`{cc()} -O2 {ll}{gl_link_flags(ll)}{vk_link_flags(ll)}{http_link_flags(ll)}{udp_link_flags(ll)}{process_link_flags(ll)}{threads_link_flags(ll)}{pbf} -o {out}`) { return false } if not shq(`{cc()} -O2 {ll}{gl_link_flags(ll)}{vk_link_flags(ll)}{http_link_flags(ll)}{udp_link_flags(ll)}{process_link_flags(ll)}{threads_link_flags(ll)}{native_link_flags(ll)}{pbf} -o {out}`) { return false }
g_built_natives = native_libs_of(ll)
if not save { shell(`rm -f {ll}`) } if not save { shell(`rm -f {ll}`) }
return true return true
} }
@ -62,7 +65,8 @@ function compile_app(src: pointer, out: pointer, mode: int, save: bool) -> bool
# canonical `ludicc -o` path links it only when Audio.* is used. # canonical `ludicc -o` path links it only when Audio.* is used.
let cocoa = `{home}runtime/native/cocoa.ll` let cocoa = `{home}runtime/native/cocoa.ll`
let audio = `{home}runtime/native/audio.ll` let audio = `{home}runtime/native/audio.ll`
if not shq(`{cc()} -O2 {ll} {cocoa} {audio} -framework Cocoa -Wl,-needed_framework,GameController -Wl,-needed_framework,AVFoundation -Wl,-rpath,@loader_path{gl_link_flags(ll)}{vk_link_flags(ll)}{http_link_flags(ll)}{udp_link_flags(ll)}{process_link_flags(ll)}{threads_link_flags(ll)}{pbf} -o {out}`) { return false } if not shq(`{cc()} -O2 {ll} {cocoa} {audio} -framework Cocoa -Wl,-needed_framework,GameController -Wl,-needed_framework,AVFoundation -Wl,-rpath,@loader_path{gl_link_flags(ll)}{vk_link_flags(ll)}{http_link_flags(ll)}{udp_link_flags(ll)}{process_link_flags(ll)}{threads_link_flags(ll)}{native_link_flags(ll)}{pbf} -o {out}`) { return false }
g_built_natives = native_libs_of(ll)
if not save { shell(`rm -f {ll}`) } if not save { shell(`rm -f {ll}`) }
return true return true
} }
@ -105,6 +109,30 @@ function vk_link_flags(ll: pointer) -> pointer {
return ` {home}runtime/native/vk_thunks.ll {home}runtime/native/vk_mac.ll` return ` {home}runtime/native/vk_thunks.ll {home}runtime/native/vk_mac.ll`
} }
# A package that carries a native library (phase 15) is written into the IR by the compiler as
# `; ludic-native: <path>` lines: link each, with an rpath to its directory for a run from the
# build and one to Contents/Frameworks, where `ludic bundle` puts it.
function native_libs_of(ll: pointer) -> []pointer {
let out = new []pointer
let lines = split_lines(capture(`grep "^; ludic-native: " {ll} 2>/dev/null`))
var i = 0
while i < len(lines) {
if s_starts(lines[i], "; ludic-native: ") { push(out, sslice(lines[i], 16, slen(lines[i]))) }
i += 1
}
return out
}
function native_link_flags(ll: pointer) -> pointer {
let libs = native_libs_of(ll)
var f = ""
var i = 0
while i < len(libs) {
f = f + ` "{libs[i]}" -Wl,-rpath,{dir_of_path(libs[i])} -Wl,-rpath,@executable_path/../Frameworks`
i += 1
}
return f
}
# A program that uses Http.* calls the @hs_* transport; link http.ll and Foundation only # A program that uses Http.* calls the @hs_* transport; link http.ll and Foundation only
# then, in both modes, as `ludicc -o` does (selfhost/main.ludic). Without it a Http.* # then, in both modes, as `ludicc -o` does (selfhost/main.ludic). Without it a Http.*
# program built through `ludic build` failed to link on every hs_* symbol. # program built through `ludic build` failed to link on every hs_* symbol.

View file

@ -272,6 +272,35 @@ function windows_icon_res(icon: pointer) -> pointer {
return res return res
} }
# copy what the last build linked into Contents/Frameworks
# and drop the rpath to where each was built: a shipped binary names no path on this machine
function bundle_natives_mac(contents: pointer, exe: pointer) -> bool {
if len(g_built_natives) == 0 { return true }
shell(`mkdir -p "{contents}/Frameworks"`)
for i in 0 .. len(g_built_natives) {
shq(`install_name_tool -delete_rpath "{dir_of_path(g_built_natives[i])}" "{exe}" 2>/dev/null`)
if not shq(`cp "{g_built_natives[i]}" "{contents}/Frameworks/"`) {
err(`ludic bundle: could not place {g_built_natives[i]}\n`)
return false
}
}
return true
}
# copy the .dll files ludicc wrote into <exe>.natives beside the executable
function bundle_natives_win(list: pointer, root: pointer) -> bool {
let txt = read_file(list)
if txt == null { return true }
let libs = split_lines(txt)
for i in 0 .. len(libs) {
if libs[i] != "" and not shq(`cp "{libs[i]}" "{root}/"`) {
err(`ludic bundle: could not place {libs[i]}\n`)
return false
}
}
return true
}
function cmd_bundle_windows() -> int { function cmd_bundle_windows() -> int {
let entry = parse_build_args(2) let entry = parse_build_args(2)
if g_argerr { return 1 } if g_argerr { return 1 }
@ -288,11 +317,12 @@ function cmd_bundle_windows() -> int {
let staged = tmp_path("bundle_exe") let staged = tmp_path("bundle_exe")
var flags = "--windowed --gui" var flags = "--windowed --gui"
if res != "" { flags = flags + ` --link "{res}"` } if res != "" { flags = flags + ` --link "{res}"` }
if not shq(`{ludicc()} {flags}{title_flag()} {entry} -o {staged}`) { return 1 } if not shq(`{ludicc()} {flags}{title_flag()} {entry} --natives-out {staged}.natives -o {staged}`) { return 1 }
if not shq(`cp "{staged}.exe" "{root}/{name}.exe"`) { if not shq(`cp "{staged}.exe" "{root}/{name}.exe"`) {
err("ludic bundle: could not place the executable\n") err("ludic bundle: could not place the executable\n")
return 1 return 1
} }
if not bundle_natives_win(`{staged}.natives`, root) { return 1 }
# 2. the assets, exactly as the .app packs them # 2. the assets, exactly as the .app packs them
let roots = pack_roots(m, 999) let roots = pack_roots(m, 999)
@ -376,6 +406,11 @@ function cmd_bundle() -> int {
return 1 return 1
} }
# 1b. a package's native libraries (phase 15) go in Contents/Frameworks, where the binary's
# @executable_path/../Frameworks rpath finds them - never in MacOS, which must hold mach-o
# executables only (a Velopack update strips anything else's detached signature)
if not bundle_natives_mac(contents, exe) { return 1 }
# 2. the assets # 2. the assets
let roots = pack_roots(m, 999) # manifest/convention only, no argv let roots = pack_roots(m, 999) # manifest/convention only, no argv
var packed = false var packed = false
@ -430,6 +465,13 @@ function cmd_bundle() -> int {
# ID and notarisation - but it does make it run here. # ID and notarisation - but it does make it run here.
var ident = manifest_app(m, "sign") var ident = manifest_app(m, "sign")
if ident == "" { ident = "-" } if ident == "" { ident = "-" }
# nested code is signed before what contains it
for i in 0 .. len(g_built_natives) {
let fw = `{contents}/Frameworks/{file_of_path(g_built_natives[i])}`
if not shq(`codesign --force --timestamp=none --sign "{ident}" "{fw}" 2>/dev/null`) {
err(`ludic bundle: warning: codesign failed on {fw}\n`)
}
}
if not shq(`codesign --force --timestamp=none --sign "{ident}" "{root}" 2>/dev/null`) { if not shq(`codesign --force --timestamp=none --sign "{ident}" "{root}" 2>/dev/null`) {
err(`ludic bundle: warning: codesign failed; the app may not launch\n`) err(`ludic bundle: warning: codesign failed; the app may not launch\n`)
} }

View file

@ -21,6 +21,7 @@ function selfhost_frags() -> []pointer {
push(f, "selfhost/frontend/generics.ludic") push(f, "selfhost/frontend/generics.ludic")
push(f, "selfhost/frontend/aliases.ludic") push(f, "selfhost/frontend/aliases.ludic")
push(f, "selfhost/frontend/modules.ludic") push(f, "selfhost/frontend/modules.ludic")
push(f, "selfhost/frontend/natives.ludic")
push(f, "selfhost/frontend/ports.ludic") push(f, "selfhost/frontend/ports.ludic")
push(f, "selfhost/frontend/privates.ludic") push(f, "selfhost/frontend/privates.ludic")
push(f, "selfhost/frontend/privates_types.ludic") push(f, "selfhost/frontend/privates_types.ludic")

View file

@ -36,7 +36,7 @@ function test_parallel(files: []pointer, only: pointer, verbose: bool) -> int {
let bin = bins[i] let bin = bins[i]
let ll = `{bin}.ll` let ll = `{bin}.ll`
if test_rc(`{bin}.rc1`) == 0 { if test_rc(`{bin}.rc1`) == 0 {
let flags = `{gl_link_flags(ll)}{vk_link_flags(ll)}{http_link_flags(ll)}{udp_link_flags(ll)}{process_link_flags(ll)}{threads_link_flags(ll)}{pbf}` let flags = `{gl_link_flags(ll)}{vk_link_flags(ll)}{http_link_flags(ll)}{udp_link_flags(ll)}{process_link_flags(ll)}{threads_link_flags(ll)}{native_link_flags(ll)}{pbf}`
write_file(`{jd}/l{string(i)}.sh`, `{cc()} -O2 {ll}{flags} -o {bin} >> {bin}.log 2>&1; echo $? > {bin}.rc2\n`) write_file(`{jd}/l{string(i)}.sh`, `{cc()} -O2 {ll}{flags} -o {bin} >> {bin}.log 2>&1; echo $? > {bin}.rc2\n`)
} }
} }

81
tools/native/lib.sh Normal file
View file

@ -0,0 +1,81 @@
# tools/native/lib.sh - what every package's native/build.sh shares (phase 15). Sourced, not run.
#
# A native library is built here from a pinned source, never downloaded as a binary: the upstream
# tag is fetched, its SHA-256 checked, and it and the package's shim compiled with clang into
# lib/<target>/. The same script runs on the Mac (Terminal) and on the PC (Git Bash, with the LLVM
# installer's clang), so there is one recipe per library, not two.
set -eu
# macos-arm64, macos-x64, windows-x64 or linux-x64 - the names `native` lines use
native_target() {
if [ -n "${LUDIC_NATIVE_TARGET:-}" ]; then echo "$LUDIC_NATIVE_TARGET"; return; fi
case "$(uname -s)" in
Darwin) if [ "$(uname -m)" = arm64 ]; then echo macos-arm64; else echo macos-x64; fi ;;
MINGW*|MSYS*|CYGWIN*) echo windows-x64 ;;
*) echo linux-x64 ;;
esac
}
# the clang that builds: $LUDIC_CC, else clang on PATH, else the LLVM installer's on Windows
native_cc() {
if [ -n "${LUDIC_CC:-}" ]; then echo "$LUDIC_CC"; return; fi
if command -v clang >/dev/null 2>&1; then echo clang; return; fi
if [ -x "/c/Program Files/LLVM/bin/clang.exe" ]; then echo "/c/Program Files/LLVM/bin/clang.exe"; return; fi
echo "native: no clang (set LUDIC_CC)" >&2; exit 1
}
native_cxx() {
local cc; cc="$(native_cc)"
case "$cc" in *clang.exe) echo "${cc%clang.exe}clang++.exe" ;; *clang) echo "${cc}++" ;; *) echo "$cc" ;; esac
}
native_sha256() {
if command -v shasum >/dev/null 2>&1; then shasum -a 256 "$1" | cut -d' ' -f1
else sha256sum "$1" | cut -d' ' -f1; fi
}
# native_fetch <dir> <url> <sha256>: the archive at url, checked, unpacked into <dir> (its one
# top-level directory stripped). A checksum that does not match stops the build.
native_fetch() {
local dir="$1" url="$2" want="$3"
local tgz="$dir.tar.gz"
mkdir -p "$(dirname "$dir")"
if [ ! -f "$tgz" ]; then curl -fsSL "$url" -o "$tgz.part" && mv "$tgz.part" "$tgz"; fi
local got; got="$(native_sha256 "$tgz")"
if [ "$got" != "$want" ]; then
echo "native: $url is $got, pinned $want" >&2; rm -f "$tgz"; exit 1
fi
if [ ! -d "$dir" ]; then
mkdir -p "$dir.part" && tar -xzf "$tgz" -C "$dir.part" --strip-components 1 && mv "$dir.part" "$dir"
fi
}
# native_link <package dir> <name> <objects...>: lib<name>.dylib (install name @rpath, so the
# executable finds it beside itself, in Contents/Frameworks or in lib/<target> while developing)
# or <name>.dll with its import library <name>.lib, into <package>/lib/<target>/
native_link() {
local pkg="$1" name="$2"; shift 2
local t; t="$(native_target)"
local out="$pkg/lib/$t"
mkdir -p "$out"
local cxx; cxx="$(native_cxx)"
case "$t" in
macos-*)
"$cxx" -dynamiclib -O2 -Wl,-install_name,@rpath/lib$name.dylib -Wl,-dead_strip "$@" -o "$out/lib$name.dylib"
echo "native: $out/lib$name.dylib" ;;
windows-*)
"$cxx" -shared -O2 "$@" -o "$out/$name.dll" -Wl,/implib:"$out/$name.lib"
echo "native: $out/$name.dll + $name.lib" ;;
*)
"$cxx" -shared -O2 -fPIC -Wl,-soname,lib$name.so "$@" -o "$out/lib$name.so"
echo "native: $out/lib$name.so" ;;
esac
}
# the flags every object is compiled with: optimised, position independent, symbols hidden unless
# the shim exports them
native_cflags() {
case "$(native_target)" in
windows-*) echo "-O2" ;;
*) echo "-O2 -fPIC -fvisibility=hidden" ;;
esac
}