feat(pkg): phase 15 - a package can carry a native library
native "<target>" "<path>" in a package's package.ludic; the compiler records the libraries of every package a program imports and writes them into the IR (; ludic-native:), so ludicc -o, ludic build, ludic test and ludic bundle all link one list. macOS: an rpath to the package and to Contents/Frameworks, where ludic bundle copies and signs each library and drops the build machine's rpath. Windows: the import library, the .dll copied beside the exe (--natives-out for the bundle). tools/native/lib.sh builds from a pinned, checksummed source with clang on both machines; ludic.nativeecho is the worked example; the shim rules are in packages/README.md. Linked at build time rather than dlopen (docs/PACKAGES.md says why). Reseeded. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
parent
5df0747642
commit
5d3a799e33
25 changed files with 54474 additions and 51707 deletions
1
.gitattributes
vendored
Normal file
1
.gitattributes
vendored
Normal file
|
|
@ -0,0 +1 @@
|
|||
packages/*/lib/** filter=lfs diff=lfs merge=lfs -text
|
||||
3
.gitignore
vendored
3
.gitignore
vendored
|
|
@ -57,3 +57,6 @@ assets/polyhaven/textures/
|
|||
assets/polyhaven/models/
|
||||
# `ludic run` beside an example writes its binary into a build/ there
|
||||
examples/**/build/
|
||||
|
||||
# a package native/build.sh writes its objects under the package (phase 15)
|
||||
packages/*/build/
|
||||
|
|
|
|||
10
changes/native-libraries.md
Normal file
10
changes/native-libraries.md
Normal file
|
|
@ -0,0 +1,10 @@
|
|||
bump: minor
|
||||
type: feat
|
||||
**A package can carry a native library.** `native "<target>" "<path>"` in a package's
|
||||
`package.ludic` names a C/C++ library per target (`macos-arm64`, `windows-x64`, ...). The
|
||||
compiler records the libraries of every package a program imports, and every link - `ludicc -o`,
|
||||
`ludic build`, `ludic test`, `ludic bundle` - links them: on macOS with an rpath to the package
|
||||
and to `Contents/Frameworks`, where `ludic bundle` places and signs them; on Windows through the
|
||||
import library, the `.dll` copied beside the executable. `tools/native/lib.sh` builds a library
|
||||
from a pinned, checksummed source; `ludic.nativeecho` is the worked example, and
|
||||
`packages/README.md` says what a shim may pass across.
|
||||
|
|
@ -235,6 +235,47 @@ clang -O2 game.ll $(ludic link-flags) -o game
|
|||
not ship the build target, `ludic get` fails — build from source instead where the
|
||||
package offers it.
|
||||
|
||||
## Native libraries - a source package that carries a C/C++ library (phase 15)
|
||||
|
||||
A source package may link a native library of its own - a physics engine, a navmesh builder, an
|
||||
audio renderer - by naming it per target in its `package.ludic`:
|
||||
|
||||
```
|
||||
native "macos-arm64" "lib/macos-arm64/libjoltc.dylib"
|
||||
native "windows-x64" "lib/windows-x64/joltc.dll"
|
||||
```
|
||||
|
||||
The targets are `macos-arm64`, `macos-x64`, `windows-x64` and `linux-x64`
|
||||
(`$LUDIC_NATIVE_TARGET` names another). The package's Ludic reaches the library through
|
||||
`extern function` declarations, which only a package (or the runtime) may call (L7).
|
||||
|
||||
**Linked at build time, not opened with `dlopen`.** When the compiler loads a file from a
|
||||
package, it reads that package's manifest once and keeps the libraries for the build's target;
|
||||
they are written into the IR as `; ludic-native: <path>` lines, and every link reads that one
|
||||
list - `ludicc -o`, `ludic build`, `ludic test` and `ludic bundle`. A program that imports no
|
||||
such package links nothing new. Linking was chosen over a Vulkan-style loader because a missing
|
||||
or mismatched library then fails at start-up, with the loader's own message naming it, rather
|
||||
than at the first call deep in a frame; because an `extern function` is all the binding needs (no
|
||||
generated thunk table per library); and because nothing here is optional the way Vulkan is - a
|
||||
game that imports the physics package cannot run without it.
|
||||
|
||||
- **macOS**: the dylib's install name is `@rpath/lib<name>.dylib`. A build gets an rpath to the
|
||||
package's `lib/<target>/` (so `ludic run` and `ludic test` work in place) and one to
|
||||
`@executable_path/../Frameworks`. `ludic bundle` copies each library into
|
||||
`Contents/Frameworks` - never `Contents/MacOS`, which holds mach-o executables only (a Velopack
|
||||
update strips a detached signature) - removes the build machine's rpath from the executable,
|
||||
and signs every library before the app.
|
||||
- **Windows**: the `.dll` is linked through its import library, `<name>.lib` beside it; `ludicc`
|
||||
copies the `.dll` beside the executable it writes, and `ludic bundle` beside the game's `.exe`.
|
||||
|
||||
**Built here from a pinned source.** A package's `native/build.sh` fetches the upstream tag,
|
||||
checks its SHA-256, compiles the library and the package's shim with clang, and writes
|
||||
`lib/<target>/` (committed through Git LFS). `tools/native/lib.sh` is what every such script
|
||||
shares - `native_fetch`, `native_link`, the target and compiler - and it runs unchanged in Git
|
||||
Bash on Windows with the LLVM installer's clang. The shim's rules (what may cross, how a
|
||||
callback comes back) are in [packages/README.md](../packages/README.md#native-libraries);
|
||||
`ludic.nativeecho` is the worked example.
|
||||
|
||||
## Offline / hermetic builds
|
||||
|
||||
`ludic vendor` copies the resolved packages out of the store into `./vendor`. Build
|
||||
|
|
|
|||
|
|
@ -52,6 +52,39 @@ section. The rules are in [ludic.base](ludic.base/README.md).
|
|||
| [ludic.lab](ludic.lab/README.md) | the visual lab: a scene on a lit plate, fixed cameras, PNG shots and a contact sheet |
|
||||
| [ludic.core](ludic.core/) | the canonical engine-ABI components |
|
||||
| [ludic.prefs](ludic.prefs/) | a small `key=value` store for preferences and records |
|
||||
| [ludic.nativeecho](ludic.nativeecho/README.md) | the worked example of a package carrying a native library: a C sum over a slice, callbacks drained as facts |
|
||||
|
||||
## Native libraries
|
||||
|
||||
A package may carry a C or C++ library (`native` lines in its `package.ludic`,
|
||||
[docs/PACKAGES.md](../docs/PACKAGES.md#native-libraries---a-source-package-that-carries-a-cc-library-phase-15)).
|
||||
The library is never bound directly: a thin C shim in the package's `native/shim/` is, and every
|
||||
shim keeps the same shape so the Ludic side stays safe.
|
||||
|
||||
- **Only `int`, `long`, `float` and opaque handles cross.** A Ludic `float` is a C `float`, a
|
||||
`long` an `int64_t`, a `pointer` a `void *`. No struct is passed by value, no C++ type, no
|
||||
exception: the shim catches and turns it into a return code.
|
||||
- **A handle is the library's pointer, kept in the package's state and never exported.** The
|
||||
game names what the package hands it - an int id - and the package maps it to the handle.
|
||||
- **Nothing calls back into Ludic.** No function pointer crosses the boundary. What a library
|
||||
reports through a callback (a contact, a finished path, a mixed buffer) the shim records in a
|
||||
buffer of its own, and the package drains it after the call returns (`ne_scan` then
|
||||
`ne_drain`), pushing each record onto its facts queue. A record that did not fit is counted,
|
||||
not dropped silently.
|
||||
- **Bulk data crosses as a slice the package owns.** A `[]float` or `[]int` passed where the
|
||||
shim takes a pointer is its data; the package makes it once and reuses it (L7), and passes its
|
||||
length beside it.
|
||||
- **Errors are return codes.** 0 is success and a negative number says what failed; nothing
|
||||
aborts, nothing prints.
|
||||
- **Threads a library starts stay inside C** (a job system, an audio device). The Ludic side is
|
||||
called on the game's thread only.
|
||||
- **The symbols are prefixed** with the package's short name (`ne_`, `jph_`), exported explicitly
|
||||
(`__attribute__((visibility("default")))`, `__declspec(dllexport)`), and everything else is
|
||||
hidden.
|
||||
|
||||
`native/build.sh` builds `lib/<target>/` from the pinned upstream source and the shim through
|
||||
[tools/native/lib.sh](../tools/native/lib.sh); run it on the Mac and, in Git Bash, on the PC.
|
||||
The package's tests run the real library - there is no fake for it.
|
||||
|
||||
## The builtin controllers
|
||||
|
||||
|
|
|
|||
26
packages/ludic.nativeecho/README.md
Normal file
26
packages/ludic.nativeecho/README.md
Normal file
|
|
@ -0,0 +1,26 @@
|
|||
# ludic.nativeecho
|
||||
|
||||
The worked example of a package that carries a native library (phase 15). It exists to be copied
|
||||
and to be tested: nothing in a game needs it.
|
||||
|
||||
```ludic
|
||||
import "ludic.nativeecho"
|
||||
```
|
||||
|
||||
- `package.ludic` names the library per target (`native "macos-arm64" "lib/macos-arm64/libnativeecho.dylib"`).
|
||||
- `native/shim/nativeecho.c` is the whole library: a sum over a float slice, a scan that
|
||||
"calls back" once per value over a threshold, and the drain the package empties those records
|
||||
through.
|
||||
- `native/build.sh` builds `lib/<target>/` through `tools/native/lib.sh` (no upstream to fetch).
|
||||
- `native.ludic` declares the symbols; `echo.ludic` is the state, the verbs and the facts.
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| `EchoState` | the slices that cross to C (made once), and the facts queue |
|
||||
| `echo_put(st, x) -> bool`, `echo_clear(st)` | fill the slice |
|
||||
| `echo_sum(st) -> float` | the sum, in C |
|
||||
| `echo_scan(st, over) -> int` | C records each value over `over`; they arrive on `echo_facts(st)` as `EchoFact { index, value }` |
|
||||
| `echo_lost() -> int`, `echo_twice(x: long) -> long` | the lost-record count, and a 64-bit round trip |
|
||||
|
||||
The rules it keeps are the shim rules in [packages/README.md](../README.md#native-libraries).
|
||||
`ludic test packages/ludic.nativeecho` runs it against the real library.
|
||||
69
packages/ludic.nativeecho/echo.ludic
Normal file
69
packages/ludic.nativeecho/echo.ludic
Normal file
|
|
@ -0,0 +1,69 @@
|
|||
# echo.ludic - the state, its verbs and its facts. The slices crossing to C are made once and
|
||||
# reused (L7), and what C recorded is drained into the queue after each call returns.
|
||||
export const ECHO_CAP: int = 64
|
||||
|
||||
export property EchoFact {
|
||||
index: int = 0 # which element went over
|
||||
value: float = 0.0
|
||||
}
|
||||
|
||||
export state EchoState {
|
||||
ec_xs: []float = null
|
||||
ec_n: int = 0
|
||||
ec_kinds: []int = null
|
||||
ec_values: []float = null
|
||||
ec_facts: Queue<EchoFact> = null
|
||||
}
|
||||
|
||||
function ec_ready(echo_st: mut EchoState) -> void {
|
||||
if echo_st.ec_xs == null { echo_st.ec_xs = floats(ECHO_CAP) }
|
||||
if echo_st.ec_kinds == null { echo_st.ec_kinds = words(ECHO_CAP) }
|
||||
if echo_st.ec_values == null { echo_st.ec_values = floats(ECHO_CAP) }
|
||||
}
|
||||
|
||||
# what went over, oldest first
|
||||
export function echo_facts(echo_st: mut EchoState) -> Queue<EchoFact> {
|
||||
if echo_st.ec_facts == null { echo_st.ec_facts = queue_new("nativeecho.facts") }
|
||||
return echo_st.ec_facts
|
||||
}
|
||||
|
||||
# put a value in the next slot; false when the slice is full
|
||||
export function echo_put(echo_st: mut EchoState, x: float) -> bool {
|
||||
ec_ready(echo_st)
|
||||
if echo_st.ec_n >= ECHO_CAP { return false }
|
||||
echo_st.ec_xs[echo_st.ec_n] = x
|
||||
echo_st.ec_n += 1
|
||||
return true
|
||||
}
|
||||
|
||||
export function echo_clear(echo_st: mut EchoState) -> void { echo_st.ec_n = 0 }
|
||||
|
||||
# the sum of what was put, worked out in C
|
||||
export function echo_sum(echo_st: mut EchoState) -> float {
|
||||
ec_ready(echo_st)
|
||||
return ne_sum(echo_st.ec_xs, echo_st.ec_n)
|
||||
}
|
||||
|
||||
# C walks the values and records each one over `over`; the records come back as facts
|
||||
export function echo_scan(echo_st: mut EchoState, over: float) -> int {
|
||||
ec_ready(echo_st)
|
||||
let hits = ne_scan(echo_st.ec_xs, echo_st.ec_n, over)
|
||||
ec_drain(echo_st)
|
||||
return hits
|
||||
}
|
||||
|
||||
function ec_drain(echo_st: mut EchoState) -> void {
|
||||
let n = ne_drain(echo_st.ec_kinds, echo_st.ec_values, ECHO_CAP)
|
||||
for i in 0 .. n {
|
||||
let f = new EchoFact
|
||||
f.index = echo_st.ec_kinds[i]
|
||||
f.value = echo_st.ec_values[i]
|
||||
q_push(echo_facts(echo_st), f)
|
||||
}
|
||||
}
|
||||
|
||||
# how many records C had to drop because nobody drained them
|
||||
export function echo_lost() -> int { return ne_lost_count() }
|
||||
|
||||
# a 64-bit value there and back
|
||||
export function echo_twice(x: long) -> long { return ne_twice(x) }
|
||||
7
packages/ludic.nativeecho/index.ludic
Normal file
7
packages/ludic.nativeecho/index.ludic
Normal file
|
|
@ -0,0 +1,7 @@
|
|||
# ludic.nativeecho - the worked example of a package carrying a native library (phase 15): a C
|
||||
# sum over a slice it owns, and a C scan whose "callbacks" come back as facts on its queue
|
||||
module ludic_nativeecho uses ludic_base
|
||||
numbers float
|
||||
import "ludic.base"
|
||||
import "native.ludic"
|
||||
import "echo.ludic"
|
||||
BIN
packages/ludic.nativeecho/lib/macos-arm64/libnativeecho.dylib
(Stored with Git LFS)
Executable file
BIN
packages/ludic.nativeecho/lib/macos-arm64/libnativeecho.dylib
(Stored with Git LFS)
Executable file
Binary file not shown.
7
packages/ludic.nativeecho/native.ludic
Normal file
7
packages/ludic.nativeecho/native.ludic
Normal file
|
|
@ -0,0 +1,7 @@
|
|||
# native.ludic - the shim's symbols (native/shim/nativeecho.c), in the types that cross:
|
||||
# int, long, float and a slice's data as a pointer. Nothing here is exported.
|
||||
extern function ne_sum(xs: pointer, n: int) -> float = "ne_sum"
|
||||
extern function ne_scan(xs: pointer, n: int, over: float) -> int = "ne_scan"
|
||||
extern function ne_drain(kinds: pointer, values: pointer, cap: int) -> int = "ne_drain"
|
||||
extern function ne_lost_count() -> int = "ne_lost_count"
|
||||
extern function ne_twice(x: long) -> long = "ne_twice"
|
||||
10
packages/ludic.nativeecho/native/build.sh
Executable file
10
packages/ludic.nativeecho/native/build.sh
Executable file
|
|
@ -0,0 +1,10 @@
|
|||
#!/bin/sh
|
||||
# builds lib/<target>/ for ludic.nativeecho: no upstream library, only the shim
|
||||
# (tools/native/lib.sh; a package that wraps a library adds native_fetch first - see ludic.physics)
|
||||
set -eu
|
||||
PKG="$(cd "$(dirname "$0")/.." && pwd)"
|
||||
. "$PKG/../../tools/native/lib.sh"
|
||||
OBJ="$PKG/build/native"
|
||||
mkdir -p "$OBJ"
|
||||
"$(native_cc)" $(native_cflags) -c "$PKG/native/shim/nativeecho.c" -o "$OBJ/nativeecho.o"
|
||||
native_link "$PKG" nativeecho "$OBJ/nativeecho.o"
|
||||
59
packages/ludic.nativeecho/native/shim/nativeecho.c
Normal file
59
packages/ludic.nativeecho/native/shim/nativeecho.c
Normal file
|
|
@ -0,0 +1,59 @@
|
|||
/* nativeecho - the shape every shim in packages/ has (packages/README.md, "Native libraries"):
|
||||
int, long, float and opaque handles cross; no struct by value, no callback into Ludic. What a
|
||||
library would report through a callback is recorded here and drained after the call returns. */
|
||||
#include <stdint.h>
|
||||
|
||||
#if defined(_WIN32)
|
||||
#define NE_API __declspec(dllexport)
|
||||
#else
|
||||
#define NE_API __attribute__((visibility("default")))
|
||||
#endif
|
||||
|
||||
#define NE_CAP 64
|
||||
|
||||
static int ne_kinds[NE_CAP];
|
||||
static float ne_values[NE_CAP];
|
||||
static int ne_count = 0;
|
||||
static int ne_lost = 0;
|
||||
|
||||
/* the sum of n floats the package owns */
|
||||
NE_API float ne_sum(const float *xs, int n) {
|
||||
float s = 0.0f;
|
||||
for (int i = 0; i < n; i++) s += xs[i];
|
||||
return s;
|
||||
}
|
||||
|
||||
/* what a callback would have reported: kept until drained, counted as lost past the cap */
|
||||
NE_API int ne_record(int kind, float value) {
|
||||
if (ne_count >= NE_CAP) { ne_lost++; return -1; }
|
||||
ne_kinds[ne_count] = kind;
|
||||
ne_values[ne_count] = value;
|
||||
ne_count++;
|
||||
return 0;
|
||||
}
|
||||
|
||||
/* a library's work that "calls back" once per element over a threshold */
|
||||
NE_API int ne_scan(const float *xs, int n, float over) {
|
||||
int hits = 0;
|
||||
for (int i = 0; i < n; i++) {
|
||||
if (xs[i] > over) { ne_record(i, xs[i]); hits++; }
|
||||
}
|
||||
return hits;
|
||||
}
|
||||
|
||||
/* copy up to cap pending records into two slices the package owns; returns how many */
|
||||
NE_API int ne_drain(int *kinds, float *values, int cap) {
|
||||
int n = ne_count < cap ? ne_count : cap;
|
||||
for (int i = 0; i < n; i++) { kinds[i] = ne_kinds[i]; values[i] = ne_values[i]; }
|
||||
for (int i = n; i < ne_count; i++) {
|
||||
ne_kinds[i - n] = ne_kinds[i];
|
||||
ne_values[i - n] = ne_values[i];
|
||||
}
|
||||
ne_count -= n;
|
||||
return n;
|
||||
}
|
||||
|
||||
NE_API int ne_lost_count(void) { return ne_lost; }
|
||||
|
||||
/* a 64-bit value round trip: handles and counts wider than 32 bits cross as long */
|
||||
NE_API int64_t ne_twice(int64_t x) { return x * 2; }
|
||||
8
packages/ludic.nativeecho/package.ludic
Normal file
8
packages/ludic.nativeecho/package.ludic
Normal file
|
|
@ -0,0 +1,8 @@
|
|||
# ludic.nativeecho - the worked example of a package that carries a native library (phase 15):
|
||||
# a C function that sums a float slice, and one that records what a callback would have said
|
||||
# into a buffer the package drains. Uses ludic.base and nothing else.
|
||||
package "ludic.nativeecho"
|
||||
version "0.1.0"
|
||||
kind source
|
||||
native "macos-arm64" "lib/macos-arm64/libnativeecho.dylib"
|
||||
native "windows-x64" "lib/windows-x64/nativeecho.dll"
|
||||
41
packages/ludic.nativeecho/tests/echo_test.ludic
Normal file
41
packages/ludic.nativeecho/tests/echo_test.ludic
Normal file
|
|
@ -0,0 +1,41 @@
|
|||
# echo_test.ludic - the native library is linked and called: a sum over a slice, a scan whose
|
||||
# records come back as facts in order, a 64-bit value, and a drain that leaves nothing behind
|
||||
import "ludic.nativeecho"
|
||||
import "ludic.base"
|
||||
program EchoTest {
|
||||
numbers float
|
||||
|
||||
test "a sum is worked out in C over the package's own slice" (echo_st: mut EchoState) {
|
||||
echo_put(echo_st, 1.5)
|
||||
echo_put(echo_st, 2.0)
|
||||
echo_put(echo_st, 0.25)
|
||||
expect(echo_sum(echo_st) == 3.75)
|
||||
}
|
||||
|
||||
test "what C would have called back arrives as facts, in order" (echo_st: mut EchoState) {
|
||||
echo_put(echo_st, 1.0)
|
||||
echo_put(echo_st, 9.0)
|
||||
echo_put(echo_st, 2.0)
|
||||
echo_put(echo_st, 7.5)
|
||||
expect(echo_scan(echo_st, 5.0) == 2)
|
||||
let fs = q_drain(echo_facts(echo_st))
|
||||
expect(len(fs) == 2)
|
||||
expect(fs[0].index == 1)
|
||||
expect(fs[0].value == 9.0)
|
||||
expect(fs[1].index == 3)
|
||||
expect(fs[1].value == 7.5)
|
||||
expect(echo_scan(echo_st, 100.0) == 0)
|
||||
expect(q_len(echo_facts(echo_st)) == 0)
|
||||
expect(echo_lost() == 0)
|
||||
}
|
||||
|
||||
test "a long crosses whole" {
|
||||
expect(echo_twice(long(3000000000)) == long(6000000000))
|
||||
}
|
||||
|
||||
test "the slice holds what it was made for and no more" (echo_st: mut EchoState) {
|
||||
for i in 0 .. ECHO_CAP { expect(echo_put(echo_st, 1.0)) }
|
||||
expect(not echo_put(echo_st, 1.0))
|
||||
expect(echo_sum(echo_st) == float(ECHO_CAP))
|
||||
}
|
||||
}
|
||||
|
|
@ -439,7 +439,7 @@ function emit_cov_runtime() -> void {
|
|||
# drivers read); with a path it is written to that file so ludicc can hand it to
|
||||
# clang itself.
|
||||
function ir_flush(path: pointer) -> bool {
|
||||
let h = buf_str(head)
|
||||
let h = native_ir_lines() + buf_str(head)
|
||||
let c = buf_str(code)
|
||||
if (path == null) { # raw IR to stdout (no trailing newline)
|
||||
let out = file_stdout()
|
||||
|
|
|
|||
105
selfhost/frontend/natives.ludic
Normal file
105
selfhost/frontend/natives.ludic
Normal file
|
|
@ -0,0 +1,105 @@
|
|||
# natives.ludic - a package can carry a native library (phase 15).
|
||||
#
|
||||
# A package's package.ludic says `native "<target>" "<path>"` for each library it links, per
|
||||
# target. When an import is found in a package, its manifest is read once and the libraries for
|
||||
# this build's target are kept, as absolute paths. ir_flush writes them into the IR as
|
||||
# `; ludic-native: <path>` lines, so every link - ludicc's own, `ludic build`, `ludic test` and
|
||||
# `ludic bundle` - reads one list, and a program that imports no such package links nothing new.
|
||||
var g_native_libs: []pointer = new []pointer
|
||||
var g_native_pkgs: []pointer = new []pointer
|
||||
|
||||
# the target a `native` line names: windows-x64, macos-arm64 or linux-x64.
|
||||
# $LUDIC_NATIVE_TARGET names another (an Intel Mac says macos-x64).
|
||||
function native_target() -> pointer {
|
||||
let env = getenv("LUDIC_NATIVE_TARGET")
|
||||
if (env != null) { return env }
|
||||
if g_target_win { return "windows-x64" }
|
||||
if Fs.is_dir("/System/Library") { return "macos-arm64" }
|
||||
return "linux-x64"
|
||||
}
|
||||
|
||||
# the words of one manifest line, a quoted word as one word without its quotes
|
||||
function native_words(line: pointer) -> []pointer {
|
||||
let out = new []pointer
|
||||
let n = len(line)
|
||||
var i = 0
|
||||
while i < n {
|
||||
while (i < n) and ((line[i] == ' ') or (line[i] == 9)) { i += 1 }
|
||||
if (i >= n) or (line[i] == '#') { return out }
|
||||
if line[i] == '"' {
|
||||
let s = i + 1
|
||||
i = s
|
||||
while (i < n) and (line[i] != '"') { i += 1 }
|
||||
push(out, line[s..i])
|
||||
i += 1
|
||||
} else {
|
||||
let s = i
|
||||
while (i < n) and (line[i] != ' ') and (line[i] != 9) { i += 1 }
|
||||
push(out, line[s..i])
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
function native_have(xs: []pointer, s: pointer) -> bool {
|
||||
var i = 0
|
||||
while i < len(xs) { if xs[i] == s { return true }; i += 1 }
|
||||
return false
|
||||
}
|
||||
|
||||
# `full` was found as `rel` under a package root (ludic_modules or $LUDIC_HOME/packages):
|
||||
# read that package's manifest once and keep its libraries for this target
|
||||
function native_note_pkg(full: pointer, rel: pointer) -> void {
|
||||
var n = 0
|
||||
while (n < len(rel)) and (rel[n] != '/') { n += 1 }
|
||||
let root = full[0..len(full) - len(rel) + n] + "/"
|
||||
if native_have(g_native_pkgs, root) { return }
|
||||
push(g_native_pkgs, root)
|
||||
let txt = read_file(root + "package.ludic")
|
||||
if (txt == null) { return }
|
||||
let want = native_target()
|
||||
var i = 0
|
||||
let tn = len(txt)
|
||||
while i < tn {
|
||||
var e = i
|
||||
while (e < tn) and (txt[e] != '\n') { e += 1 }
|
||||
var le = e
|
||||
if (le > i) and (txt[le - 1] == 13) { le -= 1 }
|
||||
let ws = native_words(txt[i..le])
|
||||
if (len(ws) == 3) and (ws[0] == "native") and (ws[1] == want) {
|
||||
let lib = root + ws[2]
|
||||
if not native_have(g_native_libs, lib) { push(g_native_libs, lib) }
|
||||
}
|
||||
i = e + 1
|
||||
}
|
||||
}
|
||||
|
||||
# the `; ludic-native:` lines ir_flush puts at the head of the IR
|
||||
function native_ir_lines() -> pointer {
|
||||
var s = ""
|
||||
var i = 0
|
||||
while i < len(g_native_libs) { s = s + `; ludic-native: {g_native_libs[i]}\n`; i += 1 }
|
||||
return s
|
||||
}
|
||||
|
||||
# what ludicc's own link adds: on macOS the library and an rpath to its directory (and to where
|
||||
# a bundle keeps it); on Windows the import library beside the .dll, the .dll copied beside the exe
|
||||
function native_link_args(out: pointer) -> pointer {
|
||||
var s = ""
|
||||
var i = 0
|
||||
while i < len(g_native_libs) {
|
||||
let lib = g_native_libs[i]
|
||||
let dir = dir_of(lib)
|
||||
if g_target_win {
|
||||
let n = len(lib)
|
||||
s = s + ` "{lib[0..n - 4]}.lib"`
|
||||
var od = dir_of(out)
|
||||
if len(od) == 0 { od = "." }
|
||||
run(`copy /y "{win_path(lib)}" "{win_path(od)}" >nul 2>nul`)
|
||||
} else {
|
||||
s = s + ` "{lib}" -Wl,-rpath,{dir} -Wl,-rpath,@executable_path/../Frameworks`
|
||||
}
|
||||
i += 1
|
||||
}
|
||||
return s
|
||||
}
|
||||
|
|
@ -1228,6 +1228,7 @@ function do_import(rel: pointer) -> void {
|
|||
if is_runtime_path(rel) or not beside or (beside and unsafe_trusted(g_parse_file)) { trust_file(full) }
|
||||
if beside and not is_runtime_path(rel) and not (module_of(g_parse_file) == "") { module_set(full, module_of(g_parse_file)) }
|
||||
if not beside and not is_runtime_path(rel) { pkg_file_set(full, pkg_name_of(rel)) } # modules.ludic
|
||||
if not beside and not is_runtime_path(rel) and (src != null) { native_note_pkg(full, rel) } # natives.ludic
|
||||
if beside and not is_runtime_path(rel) and not (pkg_of_file(g_parse_file) == "") { pkg_file_set(full, pkg_of_file(g_parse_file)) }
|
||||
if (src == null) { perr(`cannot open import {full}`) }
|
||||
# the audio runtime can arrive through atlas.ludic's own import or the Assets
|
||||
|
|
|
|||
52792
selfhost/ludicc.seed.ll
52792
selfhost/ludicc.seed.ll
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
|
|
@ -138,6 +138,7 @@ entry {
|
|||
var title = null # --title <name>: game_title() - the window's first title - instead of the program name
|
||||
var gui = false # --gui: a Windows GUI-subsystem program (no console behind the window)
|
||||
var links = new []pointer # --link <file>: an extra input to the link (a compiled resource)
|
||||
var natives_out = null # --natives-out <file>: the package libraries linked, one a line (ludic bundle)
|
||||
g_coverage = false # --coverage: instrument each statement with a per-line hit counter
|
||||
|
||||
var ai = 1
|
||||
|
|
@ -166,6 +167,7 @@ entry {
|
|||
else if a == "--target" { ai += 1; if ai < arg_count() { target = arg(ai) } }
|
||||
else if a == "--gui" { gui = true }
|
||||
else if a == "--title" { ai += 1; if ai < arg_count() { title = arg(ai) } }
|
||||
else if a == "--natives-out" { ai += 1; if ai < arg_count() { natives_out = arg(ai) } }
|
||||
else if a == "--link" { ai += 1; if ai < arg_count() { push(links, arg(ai)) } }
|
||||
else if a == "-o" { ai += 1; if ai < arg_count() { out = arg(ai) } }
|
||||
else if a[0] == '-' {
|
||||
|
|
@ -359,6 +361,8 @@ entry {
|
|||
# console-subsystem one opens a console window behind the game. mainCRTStartup keeps
|
||||
# the C runtime's ordinary main() entry rather than WinMain.
|
||||
if g_target_win and gui { cmd = `{cmd} -Wl,/subsystem:windows -Wl,/entry:mainCRTStartup` }
|
||||
# a package's native libraries (natives.ludic)
|
||||
cmd = cmd + native_link_args(out)
|
||||
var li = 0
|
||||
while li < len(links) { cmd = `{cmd} "{links[li]}"`; li += 1 }
|
||||
cmd = `{cmd} -o {out}`
|
||||
|
|
@ -371,6 +375,13 @@ entry {
|
|||
let rc = run(line)
|
||||
if not save { remove_file(ll) }
|
||||
if rc != 0 { die("ludicc: link failed\n") }
|
||||
if natives_out != null {
|
||||
var nl = ""
|
||||
var ni = 0
|
||||
while ni < len(g_native_libs) { nl = nl + g_native_libs[ni] + "\n"; ni += 1 }
|
||||
let nf = file_open(natives_out, "wb")
|
||||
if nf != null { file_write(nf, nl, len(nl)); file_close(nf) }
|
||||
}
|
||||
|
||||
# run mode: execute the binary we just built and forward its exit code.
|
||||
# system() hands back a wait status on POSIX and the exit code itself on Windows.
|
||||
|
|
|
|||
|
|
@ -27,6 +27,8 @@ function ensure_ludicc() -> void {
|
|||
# rendered to build/out.ppm from piped input — deterministic output for tests).
|
||||
# Returns true on success. `ludic build`, `ludic run` and the contributor tool all
|
||||
# go through here.
|
||||
# the native libraries (phase 15) the last compile_app linked, for `ludic bundle` to place
|
||||
var g_built_natives: []pointer = new []pointer
|
||||
function compile_app(src: pointer, out: pointer, mode: int, save: bool) -> bool {
|
||||
ensure_ludicc() # in a checkout: assemble the seed if bin/ludicc is missing or stale
|
||||
let home = ludic_home()
|
||||
|
|
@ -52,7 +54,8 @@ function compile_app(src: pointer, out: pointer, mode: int, save: bool) -> bool
|
|||
|
||||
if mode == 2 {
|
||||
if not shq(`{ludicc()} --headless{title_flag()}{unsafe_flag()} {src} --emit-llvm -o {ll}`) { return false }
|
||||
if not shq(`{cc()} -O2 {ll}{gl_link_flags(ll)}{vk_link_flags(ll)}{http_link_flags(ll)}{udp_link_flags(ll)}{process_link_flags(ll)}{threads_link_flags(ll)}{pbf} -o {out}`) { return false }
|
||||
if not shq(`{cc()} -O2 {ll}{gl_link_flags(ll)}{vk_link_flags(ll)}{http_link_flags(ll)}{udp_link_flags(ll)}{process_link_flags(ll)}{threads_link_flags(ll)}{native_link_flags(ll)}{pbf} -o {out}`) { return false }
|
||||
g_built_natives = native_libs_of(ll)
|
||||
if not save { shell(`rm -f {ll}`) }
|
||||
return true
|
||||
}
|
||||
|
|
@ -62,7 +65,8 @@ function compile_app(src: pointer, out: pointer, mode: int, save: bool) -> bool
|
|||
# canonical `ludicc -o` path links it only when Audio.* is used.
|
||||
let cocoa = `{home}runtime/native/cocoa.ll`
|
||||
let audio = `{home}runtime/native/audio.ll`
|
||||
if not shq(`{cc()} -O2 {ll} {cocoa} {audio} -framework Cocoa -Wl,-needed_framework,GameController -Wl,-needed_framework,AVFoundation -Wl,-rpath,@loader_path{gl_link_flags(ll)}{vk_link_flags(ll)}{http_link_flags(ll)}{udp_link_flags(ll)}{process_link_flags(ll)}{threads_link_flags(ll)}{pbf} -o {out}`) { return false }
|
||||
if not shq(`{cc()} -O2 {ll} {cocoa} {audio} -framework Cocoa -Wl,-needed_framework,GameController -Wl,-needed_framework,AVFoundation -Wl,-rpath,@loader_path{gl_link_flags(ll)}{vk_link_flags(ll)}{http_link_flags(ll)}{udp_link_flags(ll)}{process_link_flags(ll)}{threads_link_flags(ll)}{native_link_flags(ll)}{pbf} -o {out}`) { return false }
|
||||
g_built_natives = native_libs_of(ll)
|
||||
if not save { shell(`rm -f {ll}`) }
|
||||
return true
|
||||
}
|
||||
|
|
@ -105,6 +109,30 @@ function vk_link_flags(ll: pointer) -> pointer {
|
|||
return ` {home}runtime/native/vk_thunks.ll {home}runtime/native/vk_mac.ll`
|
||||
}
|
||||
|
||||
# A package that carries a native library (phase 15) is written into the IR by the compiler as
|
||||
# `; ludic-native: <path>` lines: link each, with an rpath to its directory for a run from the
|
||||
# build and one to Contents/Frameworks, where `ludic bundle` puts it.
|
||||
function native_libs_of(ll: pointer) -> []pointer {
|
||||
let out = new []pointer
|
||||
let lines = split_lines(capture(`grep "^; ludic-native: " {ll} 2>/dev/null`))
|
||||
var i = 0
|
||||
while i < len(lines) {
|
||||
if s_starts(lines[i], "; ludic-native: ") { push(out, sslice(lines[i], 16, slen(lines[i]))) }
|
||||
i += 1
|
||||
}
|
||||
return out
|
||||
}
|
||||
function native_link_flags(ll: pointer) -> pointer {
|
||||
let libs = native_libs_of(ll)
|
||||
var f = ""
|
||||
var i = 0
|
||||
while i < len(libs) {
|
||||
f = f + ` "{libs[i]}" -Wl,-rpath,{dir_of_path(libs[i])} -Wl,-rpath,@executable_path/../Frameworks`
|
||||
i += 1
|
||||
}
|
||||
return f
|
||||
}
|
||||
|
||||
# A program that uses Http.* calls the @hs_* transport; link http.ll and Foundation only
|
||||
# then, in both modes, as `ludicc -o` does (selfhost/main.ludic). Without it a Http.*
|
||||
# program built through `ludic build` failed to link on every hs_* symbol.
|
||||
|
|
|
|||
|
|
@ -272,6 +272,35 @@ function windows_icon_res(icon: pointer) -> pointer {
|
|||
return res
|
||||
}
|
||||
|
||||
# copy what the last build linked into Contents/Frameworks
|
||||
# and drop the rpath to where each was built: a shipped binary names no path on this machine
|
||||
function bundle_natives_mac(contents: pointer, exe: pointer) -> bool {
|
||||
if len(g_built_natives) == 0 { return true }
|
||||
shell(`mkdir -p "{contents}/Frameworks"`)
|
||||
for i in 0 .. len(g_built_natives) {
|
||||
shq(`install_name_tool -delete_rpath "{dir_of_path(g_built_natives[i])}" "{exe}" 2>/dev/null`)
|
||||
if not shq(`cp "{g_built_natives[i]}" "{contents}/Frameworks/"`) {
|
||||
err(`ludic bundle: could not place {g_built_natives[i]}\n`)
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
# copy the .dll files ludicc wrote into <exe>.natives beside the executable
|
||||
function bundle_natives_win(list: pointer, root: pointer) -> bool {
|
||||
let txt = read_file(list)
|
||||
if txt == null { return true }
|
||||
let libs = split_lines(txt)
|
||||
for i in 0 .. len(libs) {
|
||||
if libs[i] != "" and not shq(`cp "{libs[i]}" "{root}/"`) {
|
||||
err(`ludic bundle: could not place {libs[i]}\n`)
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
function cmd_bundle_windows() -> int {
|
||||
let entry = parse_build_args(2)
|
||||
if g_argerr { return 1 }
|
||||
|
|
@ -288,11 +317,12 @@ function cmd_bundle_windows() -> int {
|
|||
let staged = tmp_path("bundle_exe")
|
||||
var flags = "--windowed --gui"
|
||||
if res != "" { flags = flags + ` --link "{res}"` }
|
||||
if not shq(`{ludicc()} {flags}{title_flag()} {entry} -o {staged}`) { return 1 }
|
||||
if not shq(`{ludicc()} {flags}{title_flag()} {entry} --natives-out {staged}.natives -o {staged}`) { return 1 }
|
||||
if not shq(`cp "{staged}.exe" "{root}/{name}.exe"`) {
|
||||
err("ludic bundle: could not place the executable\n")
|
||||
return 1
|
||||
}
|
||||
if not bundle_natives_win(`{staged}.natives`, root) { return 1 }
|
||||
|
||||
# 2. the assets, exactly as the .app packs them
|
||||
let roots = pack_roots(m, 999)
|
||||
|
|
@ -376,6 +406,11 @@ function cmd_bundle() -> int {
|
|||
return 1
|
||||
}
|
||||
|
||||
# 1b. a package's native libraries (phase 15) go in Contents/Frameworks, where the binary's
|
||||
# @executable_path/../Frameworks rpath finds them - never in MacOS, which must hold mach-o
|
||||
# executables only (a Velopack update strips anything else's detached signature)
|
||||
if not bundle_natives_mac(contents, exe) { return 1 }
|
||||
|
||||
# 2. the assets
|
||||
let roots = pack_roots(m, 999) # manifest/convention only, no argv
|
||||
var packed = false
|
||||
|
|
@ -430,6 +465,13 @@ function cmd_bundle() -> int {
|
|||
# ID and notarisation - but it does make it run here.
|
||||
var ident = manifest_app(m, "sign")
|
||||
if ident == "" { ident = "-" }
|
||||
# nested code is signed before what contains it
|
||||
for i in 0 .. len(g_built_natives) {
|
||||
let fw = `{contents}/Frameworks/{file_of_path(g_built_natives[i])}`
|
||||
if not shq(`codesign --force --timestamp=none --sign "{ident}" "{fw}" 2>/dev/null`) {
|
||||
err(`ludic bundle: warning: codesign failed on {fw}\n`)
|
||||
}
|
||||
}
|
||||
if not shq(`codesign --force --timestamp=none --sign "{ident}" "{root}" 2>/dev/null`) {
|
||||
err(`ludic bundle: warning: codesign failed; the app may not launch\n`)
|
||||
}
|
||||
|
|
|
|||
|
|
@ -21,6 +21,7 @@ function selfhost_frags() -> []pointer {
|
|||
push(f, "selfhost/frontend/generics.ludic")
|
||||
push(f, "selfhost/frontend/aliases.ludic")
|
||||
push(f, "selfhost/frontend/modules.ludic")
|
||||
push(f, "selfhost/frontend/natives.ludic")
|
||||
push(f, "selfhost/frontend/ports.ludic")
|
||||
push(f, "selfhost/frontend/privates.ludic")
|
||||
push(f, "selfhost/frontend/privates_types.ludic")
|
||||
|
|
|
|||
|
|
@ -36,7 +36,7 @@ function test_parallel(files: []pointer, only: pointer, verbose: bool) -> int {
|
|||
let bin = bins[i]
|
||||
let ll = `{bin}.ll`
|
||||
if test_rc(`{bin}.rc1`) == 0 {
|
||||
let flags = `{gl_link_flags(ll)}{vk_link_flags(ll)}{http_link_flags(ll)}{udp_link_flags(ll)}{process_link_flags(ll)}{threads_link_flags(ll)}{pbf}`
|
||||
let flags = `{gl_link_flags(ll)}{vk_link_flags(ll)}{http_link_flags(ll)}{udp_link_flags(ll)}{process_link_flags(ll)}{threads_link_flags(ll)}{native_link_flags(ll)}{pbf}`
|
||||
write_file(`{jd}/l{string(i)}.sh`, `{cc()} -O2 {ll}{flags} -o {bin} >> {bin}.log 2>&1; echo $? > {bin}.rc2\n`)
|
||||
}
|
||||
}
|
||||
|
|
|
|||
81
tools/native/lib.sh
Normal file
81
tools/native/lib.sh
Normal file
|
|
@ -0,0 +1,81 @@
|
|||
# tools/native/lib.sh - what every package's native/build.sh shares (phase 15). Sourced, not run.
|
||||
#
|
||||
# A native library is built here from a pinned source, never downloaded as a binary: the upstream
|
||||
# tag is fetched, its SHA-256 checked, and it and the package's shim compiled with clang into
|
||||
# lib/<target>/. The same script runs on the Mac (Terminal) and on the PC (Git Bash, with the LLVM
|
||||
# installer's clang), so there is one recipe per library, not two.
|
||||
set -eu
|
||||
|
||||
# macos-arm64, macos-x64, windows-x64 or linux-x64 - the names `native` lines use
|
||||
native_target() {
|
||||
if [ -n "${LUDIC_NATIVE_TARGET:-}" ]; then echo "$LUDIC_NATIVE_TARGET"; return; fi
|
||||
case "$(uname -s)" in
|
||||
Darwin) if [ "$(uname -m)" = arm64 ]; then echo macos-arm64; else echo macos-x64; fi ;;
|
||||
MINGW*|MSYS*|CYGWIN*) echo windows-x64 ;;
|
||||
*) echo linux-x64 ;;
|
||||
esac
|
||||
}
|
||||
|
||||
# the clang that builds: $LUDIC_CC, else clang on PATH, else the LLVM installer's on Windows
|
||||
native_cc() {
|
||||
if [ -n "${LUDIC_CC:-}" ]; then echo "$LUDIC_CC"; return; fi
|
||||
if command -v clang >/dev/null 2>&1; then echo clang; return; fi
|
||||
if [ -x "/c/Program Files/LLVM/bin/clang.exe" ]; then echo "/c/Program Files/LLVM/bin/clang.exe"; return; fi
|
||||
echo "native: no clang (set LUDIC_CC)" >&2; exit 1
|
||||
}
|
||||
native_cxx() {
|
||||
local cc; cc="$(native_cc)"
|
||||
case "$cc" in *clang.exe) echo "${cc%clang.exe}clang++.exe" ;; *clang) echo "${cc}++" ;; *) echo "$cc" ;; esac
|
||||
}
|
||||
|
||||
native_sha256() {
|
||||
if command -v shasum >/dev/null 2>&1; then shasum -a 256 "$1" | cut -d' ' -f1
|
||||
else sha256sum "$1" | cut -d' ' -f1; fi
|
||||
}
|
||||
|
||||
# native_fetch <dir> <url> <sha256>: the archive at url, checked, unpacked into <dir> (its one
|
||||
# top-level directory stripped). A checksum that does not match stops the build.
|
||||
native_fetch() {
|
||||
local dir="$1" url="$2" want="$3"
|
||||
local tgz="$dir.tar.gz"
|
||||
mkdir -p "$(dirname "$dir")"
|
||||
if [ ! -f "$tgz" ]; then curl -fsSL "$url" -o "$tgz.part" && mv "$tgz.part" "$tgz"; fi
|
||||
local got; got="$(native_sha256 "$tgz")"
|
||||
if [ "$got" != "$want" ]; then
|
||||
echo "native: $url is $got, pinned $want" >&2; rm -f "$tgz"; exit 1
|
||||
fi
|
||||
if [ ! -d "$dir" ]; then
|
||||
mkdir -p "$dir.part" && tar -xzf "$tgz" -C "$dir.part" --strip-components 1 && mv "$dir.part" "$dir"
|
||||
fi
|
||||
}
|
||||
|
||||
# native_link <package dir> <name> <objects...>: lib<name>.dylib (install name @rpath, so the
|
||||
# executable finds it beside itself, in Contents/Frameworks or in lib/<target> while developing)
|
||||
# or <name>.dll with its import library <name>.lib, into <package>/lib/<target>/
|
||||
native_link() {
|
||||
local pkg="$1" name="$2"; shift 2
|
||||
local t; t="$(native_target)"
|
||||
local out="$pkg/lib/$t"
|
||||
mkdir -p "$out"
|
||||
local cxx; cxx="$(native_cxx)"
|
||||
case "$t" in
|
||||
macos-*)
|
||||
"$cxx" -dynamiclib -O2 -Wl,-install_name,@rpath/lib$name.dylib -Wl,-dead_strip "$@" -o "$out/lib$name.dylib"
|
||||
echo "native: $out/lib$name.dylib" ;;
|
||||
windows-*)
|
||||
"$cxx" -shared -O2 "$@" -o "$out/$name.dll" -Wl,/implib:"$out/$name.lib"
|
||||
echo "native: $out/$name.dll + $name.lib" ;;
|
||||
*)
|
||||
"$cxx" -shared -O2 -fPIC -Wl,-soname,lib$name.so "$@" -o "$out/lib$name.so"
|
||||
echo "native: $out/lib$name.so" ;;
|
||||
esac
|
||||
}
|
||||
|
||||
# the flags every object is compiled with: optimised, position independent, symbols hidden unless
|
||||
# the shim exports them
|
||||
native_cflags() {
|
||||
case "$(native_target)" in
|
||||
windows-*) echo "-O2" ;;
|
||||
*) echo "-O2 -fPIC -fvisibility=hidden" ;;
|
||||
esac
|
||||
}
|
||||
Loading…
Add table
Add a link
Reference in a new issue