feat(pkg): package manager — fetch + MVS resolve + namespace registration (#63)
Implements the v1 direction decided in the RFC as a set of `x` subcommands
plus a small, contained compiler change.
* URL-as-identity, no registry — a dependency is named by its git import
path and a `git tag vX.Y.Z` publishes a version.
* Minimum Version Selection — a `require` is a minimum; the resolver picks
the greatest required minimum per module, then the reachable closure at
those versions. Deterministic, no SAT solver (tools/x/pkg.ludic).
* Content-addressed global store + per-project links — packages live once in
~/.ludic/store keyed by a content hash; each project links them under
ludic_modules/. package.ludic (manifest) + package.lock.ludic (lock).
* Namespace registration for source packages via a module-root import
fallback in the compiler: do_import resolves a non-local, non-absolute
import under $LUDIC_MODULES (default ludic_modules/), so a fetched
package's Ludic compiles into the consumer the way the built-in stdlib
does. Collisions and missing prebuilt targets are hard errors.
Commands: x add / x get / x update / x verify / x vendor. New hermetic suite
`x test-pkg` (stands up throwaway git repos, offline) is gated inside `x test`.
Existing programs compile byte-for-byte identically (the import fallback only
fires when the local path is absent); the C-free bootstrap fixpoint holds and
the seed is regenerated. Full suite: 87 passed, package suite: 12 passed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
parent
237e13c95e
commit
2c44bae496
10 changed files with 17409 additions and 16387 deletions
7
.gitignore
vendored
7
.gitignore
vendored
|
|
@ -14,6 +14,13 @@
|
||||||
# only thing published is the source and the LLVM-IR seed (selfhost/ludicc.seed.ll).
|
# only thing published is the source and the LLVM-IR seed (selfhost/ludicc.seed.ll).
|
||||||
/bin/
|
/bin/
|
||||||
|
|
||||||
|
# package manager (issue #63): the per-project linked view into the global
|
||||||
|
# content-addressed store, and the optional hermetic copy from `x vendor`. Both
|
||||||
|
# are regenerated by `x get` / `x vendor` — package.ludic + package.lock.ludic
|
||||||
|
# are the tracked source of truth, so these stay out of the tree.
|
||||||
|
ludic_modules/
|
||||||
|
vendor/
|
||||||
|
|
||||||
# editor toolchain build artifacts
|
# editor toolchain build artifacts
|
||||||
tools/editors/vscode/node_modules/
|
tools/editors/vscode/node_modules/
|
||||||
tools/editors/vscode/*.vsix
|
tools/editors/vscode/*.vsix
|
||||||
|
|
|
||||||
10
README.md
10
README.md
|
|
@ -66,6 +66,16 @@ bin/x selfhost-test # correctness + the self-hosting / C-free bootstrap fixpoi
|
||||||
bin/x help # every command
|
bin/x help # every command
|
||||||
```
|
```
|
||||||
|
|
||||||
|
Add a dependency (URL-as-identity, MVS resolution, content-addressed store):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
bin/x add git.workshopsoft.io/user/pkg # resolve + fetch + link into ludic_modules/
|
||||||
|
bin/x get # install everything in package.ludic, write the lock
|
||||||
|
bin/x verify # check the locked packages against the store
|
||||||
|
```
|
||||||
|
|
||||||
|
See [`docs/PACKAGES.md`](docs/PACKAGES.md) for the manifest, lockfile and package model.
|
||||||
|
|
||||||
## Layout
|
## Layout
|
||||||
|
|
||||||
| Path | What it is |
|
| Path | What it is |
|
||||||
|
|
|
||||||
3
changes/package-manager.md
Normal file
3
changes/package-manager.md
Normal file
|
|
@ -0,0 +1,3 @@
|
||||||
|
bump: minor
|
||||||
|
type: feat
|
||||||
|
Package manager (#63) — `x add` / `x get` / `x update` / `x verify` / `x vendor` bring third-party packages to Ludic with no new infrastructure. Dependencies are named by their git import path (URL-as-identity, no registry — a `git tag vX.Y.Z` is publishing), resolved by Go-style Minimum Version Selection, fetched into a content-addressed global store (`~/.ludic/store`, keyed by a file-content hash) and linked into each project under `ludic_modules/`. A `package.ludic` manifest declares dependencies, the provided `Foo.*` namespace(s), the kind (source or prebuilt) and, for prebuilt libs, the shipped targets; `package.lock.ludic` pins the resolved versions and content hashes for reproducible, verifiable builds. A source package's Ludic compiles into the consumer via a new module-root import fallback in the compiler (`import "git.workshopsoft.io/user/pkg/foo.ludic"` resolves against `$LUDIC_MODULES`, default `ludic_modules/`), so a package registers a namespace the same way the built-in stdlib does. Namespace collisions and missing prebuilt targets are hard errors. Existing programs compile byte-for-byte identically; the C-free bootstrap fixpoint is untouched. See docs/PACKAGES.md.
|
||||||
120
docs/PACKAGES.md
Normal file
120
docs/PACKAGES.md
Normal file
|
|
@ -0,0 +1,120 @@
|
||||||
|
# Ludic packages
|
||||||
|
|
||||||
|
Ludic has a package manager built into the task runner (`bin/x`). It fetches,
|
||||||
|
resolves, stores and links third-party packages with no new infrastructure to
|
||||||
|
run — it drives plain `git` and rides on the Forgejo host and its release tags.
|
||||||
|
|
||||||
|
This is the v1 implementation of the direction decided in issue #63.
|
||||||
|
|
||||||
|
## The four decisions
|
||||||
|
|
||||||
|
| Axis | Ludic's choice |
|
||||||
|
| --- | --- |
|
||||||
|
| Where source comes from | **URL-as-identity, no registry.** The import path *is* the git location — `git.workshopsoft.io/user/pkg`. A `git tag vX.Y.Z` publishes a version. No account, no publish step, no index to run. |
|
||||||
|
| Version resolution | **Minimum Version Selection (MVS), Go-style.** A `require` is a *minimum*; the resolver picks, per module, the greatest of every required minimum, then keeps the reachable closure at those versions. Deterministic, no SAT solver. |
|
||||||
|
| Where dependencies live | **A content-addressed global store + per-project links (pnpm-style).** One immutable store keyed by a file-content hash (`~/.ludic/store/<sha256>`); each project gets a light linked view under `ludic_modules/` instead of a full copy. |
|
||||||
|
| Manifest / lock | **`package.ludic`** (declared deps + provided namespaces + kind + targets) and **`package.lock.ludic`** (resolved versions + content hashes). |
|
||||||
|
|
||||||
|
## Commands
|
||||||
|
|
||||||
|
```
|
||||||
|
x add <module>[@version] add a dependency to package.ludic, then resolve + fetch + link
|
||||||
|
x get resolve every dependency in package.ludic, link them, write the lock
|
||||||
|
x update [module] bump a dependency (or all) to its latest published version, then relock
|
||||||
|
x verify check every locked package against the store by content hash
|
||||||
|
x vendor copy the resolved packages into ./vendor for hermetic/offline builds
|
||||||
|
```
|
||||||
|
|
||||||
|
`x add` with no `@version` picks the latest published tag and records it as the
|
||||||
|
minimum. All the install commands print the resolved build list and write
|
||||||
|
`package.lock.ludic`.
|
||||||
|
|
||||||
|
## The manifest — `package.ludic`
|
||||||
|
|
||||||
|
A line-oriented manifest. `#` starts a comment; strings are double-quoted.
|
||||||
|
|
||||||
|
```
|
||||||
|
package "git.workshopsoft.io/orkun/greeter" # this package's import path
|
||||||
|
version "1.2.0" # the version this checkout publishes
|
||||||
|
kind source # source | prebuilt
|
||||||
|
provides "Greet" # the Foo.* namespace(s) it registers (repeatable)
|
||||||
|
|
||||||
|
require "git.workshopsoft.io/orkun/util" "1.0.0" # a dependency and its minimum version
|
||||||
|
|
||||||
|
# a prebuilt lib also declares the targets it ships:
|
||||||
|
# kind prebuilt
|
||||||
|
# targets "native-arm64" "wasm32"
|
||||||
|
```
|
||||||
|
|
||||||
|
A consumer project's manifest is the same file, usually with only `require`
|
||||||
|
lines (the `package`/`version` fields describe a *publishable* package and are
|
||||||
|
optional for a leaf application).
|
||||||
|
|
||||||
|
## The lockfile — `package.lock.ludic`
|
||||||
|
|
||||||
|
Generated by `x get`; do not edit by hand. One line per resolved module, pinning
|
||||||
|
its selected version, content hash, kind and provided namespaces:
|
||||||
|
|
||||||
|
```
|
||||||
|
# package.lock.ludic — generated by `x get`. Do not edit by hand.
|
||||||
|
lock 1
|
||||||
|
module "git.workshopsoft.io/orkun/greeter" version "1.2.0" hash "sha256:…" kind "source" provides "Greet"
|
||||||
|
module "git.workshopsoft.io/orkun/util" version "1.0.0" hash "sha256:…" kind "source" provides "Util"
|
||||||
|
```
|
||||||
|
|
||||||
|
`x verify` rehashes each store entry and confirms the project links to it, so a
|
||||||
|
tampered or missing dependency is caught before it reaches a build.
|
||||||
|
|
||||||
|
## The store and the project view
|
||||||
|
|
||||||
|
Fetched packages live once in a global, immutable, content-addressed store:
|
||||||
|
|
||||||
|
```
|
||||||
|
~/.ludic/store/<sha256>/… the package tree at a version (no .git)
|
||||||
|
~/.ludic/store/cache/<module>/ a git clone cache used during resolution
|
||||||
|
```
|
||||||
|
|
||||||
|
Each project gets a lightweight view — `ludic_modules/<import-path>` is a symlink
|
||||||
|
into the store — so many projects share one copy and nothing is duplicated
|
||||||
|
per-project. Override the store location with `$LUDIC_STORE`.
|
||||||
|
|
||||||
|
## Consuming a package — namespace registration
|
||||||
|
|
||||||
|
A **source package** ships plain Ludic. The consumer imports the package files by
|
||||||
|
their import path:
|
||||||
|
|
||||||
|
```
|
||||||
|
program App {
|
||||||
|
import "git.workshopsoft.io/orkun/greeter/greet.ludic"
|
||||||
|
entry { print(greet_hello()) }
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
The compiler resolves an import first relative to the importing file, then — for
|
||||||
|
a non-absolute path that is not found — under the package module root
|
||||||
|
(`$LUDIC_MODULES`, default `ludic_modules/`). So a fetched package's code is
|
||||||
|
spliced into the build and its namespace becomes available exactly the way the
|
||||||
|
built-in stdlib namespaces (Regex.\*, Grid.\*, …) are. Because Ludic compiles
|
||||||
|
ahead-of-time, a source package is compiled *into* the consumer's binary — no
|
||||||
|
ABI seam, and the whole-program guarantees (determinism, replay, `world_save`)
|
||||||
|
still hold.
|
||||||
|
|
||||||
|
Two packages may not register the same `Foo.*` namespace — a collision is a hard
|
||||||
|
error naming both modules.
|
||||||
|
|
||||||
|
## Prebuilt libraries
|
||||||
|
|
||||||
|
A `kind prebuilt` package ships a compiled artifact per target it declares in
|
||||||
|
`targets`. Resolution selects the artifact for the build target
|
||||||
|
(`$LUDIC_TARGET`, else `native-<arch>` for the host). If a needed target is not
|
||||||
|
shipped it is a hard error — unless the package also ships source, in which case
|
||||||
|
the source path is used. Prebuilt libs are the escape hatch for closed-source or
|
||||||
|
other-language code over the engine's stable C-ABI; source packages are the
|
||||||
|
default because they keep cross-compilation (including the wasm target) and the
|
||||||
|
compile-time ECS first-class.
|
||||||
|
|
||||||
|
## Offline / hermetic builds
|
||||||
|
|
||||||
|
`x vendor` copies the resolved packages out of the store into `./vendor`. Build
|
||||||
|
against the copy with `LUDIC_MODULES=vendor`, so the build needs neither the
|
||||||
|
network nor the global store.
|
||||||
|
|
@ -573,12 +573,26 @@ function parse_test() -> Node {
|
||||||
return n
|
return n
|
||||||
}
|
}
|
||||||
|
|
||||||
# lex and parse an imported fragment into `prog`, saving/restoring lexer state
|
# lex and parse an imported fragment into `prog`, saving/restoring lexer state.
|
||||||
|
#
|
||||||
|
# An import is resolved first relative to the importing file (the historical
|
||||||
|
# behaviour). When that fails and the spec is not absolute, it is looked up
|
||||||
|
# under the package module root — $LUDIC_MODULES, default "ludic_modules" — so a
|
||||||
|
# fetched source package resolves by its import path, e.g.
|
||||||
|
# import "git.workshopsoft.io/user/pkg/foo.ludic"
|
||||||
|
# materialised by `x get` at ludic_modules/git.workshopsoft.io/user/pkg/foo.ludic.
|
||||||
|
# The fallback only fires when the local path does not exist, so every existing
|
||||||
|
# import resolves exactly as before (the emitted IR is byte-identical).
|
||||||
function do_import(rel: pointer) -> void {
|
function do_import(rel: pointer) -> void {
|
||||||
let full = path_join(cur_dir, rel)
|
var full = path_join(cur_dir, rel)
|
||||||
|
var src = read_file(full)
|
||||||
|
if (src == null) and (rel[0] != 47) {
|
||||||
|
let cand = ensure_slash(getenv_or("LUDIC_MODULES", "ludic_modules")) + rel
|
||||||
|
let s2 = read_file(cand)
|
||||||
|
if (s2 != null) { full = cand; src = s2 }
|
||||||
|
}
|
||||||
if already_loaded(full) { return }
|
if already_loaded(full) { return }
|
||||||
push(loaded_paths, full)
|
push(loaded_paths, full)
|
||||||
let src = read_file(full)
|
|
||||||
if (src == null) { perr(`cannot open import {full}`) }
|
if (src == null) { perr(`cannot open import {full}`) }
|
||||||
let saved_toks = toks; let saved_pi = pi; let saved_dir = cur_dir
|
let saved_toks = toks; let saved_pi = pi; let saved_dir = cur_dir
|
||||||
cur_dir = dir_of(full)
|
cur_dir = dir_of(full)
|
||||||
|
|
|
||||||
32818
selfhost/ludicc.seed.ll
32818
selfhost/ludicc.seed.ll
File diff suppressed because it is too large
Load diff
|
|
@ -23,6 +23,8 @@ program X {
|
||||||
import "docgen_gen.ludic"
|
import "docgen_gen.ludic"
|
||||||
import "docgen_check.ludic"
|
import "docgen_check.ludic"
|
||||||
import "release.ludic"
|
import "release.ludic"
|
||||||
|
import "pkg.ludic"
|
||||||
|
import "pkg_test.ludic"
|
||||||
|
|
||||||
function usage() -> void {
|
function usage() -> void {
|
||||||
print("x — the Ludic task runner (run from the repository root)")
|
print("x — the Ludic task runner (run from the repository root)")
|
||||||
|
|
@ -35,8 +37,16 @@ program X {
|
||||||
print(" x tools [--install] [--test] build the editor toolchain (ludic-fmt, ludic-lsp)")
|
print(" x tools [--install] [--test] build the editor toolchain (ludic-fmt, ludic-lsp)")
|
||||||
print(" x clean remove build/ (incl. build/out.ppm) and stray artifacts (keeps bin/)")
|
print(" x clean remove build/ (incl. build/out.ppm) and stray artifacts (keeps bin/)")
|
||||||
print("")
|
print("")
|
||||||
|
print("packages (issue #63 — MVS resolution, content-addressed store, ludic_modules/ links):")
|
||||||
|
print(" x add <module>[@version] add a dependency to package.ludic, then resolve + fetch + link")
|
||||||
|
print(" x get resolve every package.ludic dependency, link them, write the lock")
|
||||||
|
print(" x update [module] bump a dep (or all) to its latest published version, then relock")
|
||||||
|
print(" x verify check every locked package against the store by content hash")
|
||||||
|
print(" x vendor copy the resolved packages into ./vendor for offline builds")
|
||||||
|
print("")
|
||||||
print("test:")
|
print("test:")
|
||||||
print(" x test the full regression suite")
|
print(" x test the full regression suite")
|
||||||
|
print(" x test-pkg the package-manager suite (hermetic git repos, offline)")
|
||||||
print(" x test --coverage per-file line coverage over the test specs")
|
print(" x test --coverage per-file line coverage over the test specs")
|
||||||
print(" x selfhost-test the self-hosting suite (correctness + bootstrap fixpoints)")
|
print(" x selfhost-test the self-hosting suite (correctness + bootstrap fixpoints)")
|
||||||
print(" x test-tools the editor-toolchain suite")
|
print(" x test-tools the editor-toolchain suite")
|
||||||
|
|
@ -106,6 +116,12 @@ program X {
|
||||||
if (arg_count() < 5) { err("usage: x game-build <ludicc> <game.ludic> <out>\n"); exit(1) }
|
if (arg_count() < 5) { err("usage: x game-build <ludicc> <game.ludic> <out>\n"); exit(1) }
|
||||||
exit(cmd_game_build(arg(2), arg(3), arg(4)))
|
exit(cmd_game_build(arg(2), arg(3), arg(4)))
|
||||||
}
|
}
|
||||||
|
if (cmd == "add") { exit(cmd_pkg_add()) }
|
||||||
|
if (cmd == "get") { exit(cmd_pkg_get()) }
|
||||||
|
if (cmd == "update") { exit(cmd_pkg_update()) }
|
||||||
|
if (cmd == "verify") { exit(cmd_pkg_verify()) }
|
||||||
|
if (cmd == "vendor") { exit(cmd_pkg_vendor()) }
|
||||||
|
if (cmd == "test-pkg") { exit(cmd_test_pkg()) }
|
||||||
if (cmd == "version") or (cmd == "--version") or (cmd == "-v") { exit(cmd_version()) }
|
if (cmd == "version") or (cmd == "--version") or (cmd == "-v") { exit(cmd_version()) }
|
||||||
if (cmd == "release") { exit(cmd_release()) }
|
if (cmd == "release") { exit(cmd_release()) }
|
||||||
if (cmd == "help") or (cmd == "--help") or (cmd == "-h") { usage(); exit(0) }
|
if (cmd == "help") or (cmd == "--help") or (cmd == "-h") { usage(); exit(0) }
|
||||||
|
|
|
||||||
646
tools/x/pkg.ludic
Normal file
646
tools/x/pkg.ludic
Normal file
|
|
@ -0,0 +1,646 @@
|
||||||
|
# pkg.ludic — the Ludic package manager (issue #63), a set of `x` subcommands.
|
||||||
|
#
|
||||||
|
# It realises the v1 direction decided in the RFC:
|
||||||
|
#
|
||||||
|
# * URL-as-identity, no registry — the import path IS the git location
|
||||||
|
# (git.workshopsoft.io/user/pkg). A `git tag` is publishing.
|
||||||
|
# * Minimum Version Selection (MVS, Go-style) — a require is a *minimum*; the
|
||||||
|
# resolver picks, per module, the greatest of the required minimums, then
|
||||||
|
# the reachable closure at those versions. Deterministic, no SAT solver.
|
||||||
|
# * A content-addressed global store + per-project symlinks (pnpm-style): one
|
||||||
|
# immutable store keyed by a content hash, each project a light linked view
|
||||||
|
# under ludic_modules/ (which the compiler resolves imports against).
|
||||||
|
# * package.ludic (declared deps + provided namespaces + kind + targets) and
|
||||||
|
# package.lock.ludic (resolved versions + content hashes) as the manifest
|
||||||
|
# and lockfile.
|
||||||
|
#
|
||||||
|
# Fetching drives plain `git` through the `run`/`capture` intrinsics — no new
|
||||||
|
# infrastructure. A source package's Ludic is compiled together with the
|
||||||
|
# consumer via the module-root import fallback in the compiler (do_import); a
|
||||||
|
# prebuilt lib declares its shipped targets and is resolved against the build
|
||||||
|
# target. Everything is content-verified against the lockfile by `x verify`.
|
||||||
|
|
||||||
|
# ---- the manifest / lock model ----------------------------------------------
|
||||||
|
|
||||||
|
property Dep { module: pointer = "", ver: pointer = "" }
|
||||||
|
|
||||||
|
property Manifest {
|
||||||
|
module: pointer = "",
|
||||||
|
ver: pointer = "",
|
||||||
|
kind: pointer = "source", # source | prebuilt
|
||||||
|
hash: pointer = "", # content hash, filled in after a snapshot
|
||||||
|
provides: []pointer, # the Foo.* namespace(s) this package registers
|
||||||
|
targets: []pointer, # prebuilt: the targets it ships (native-arm64, wasm32, …)
|
||||||
|
deps: []Dep
|
||||||
|
}
|
||||||
|
|
||||||
|
function manifest_new() -> Manifest {
|
||||||
|
let m = new Manifest
|
||||||
|
m.module = ""
|
||||||
|
m.ver = ""
|
||||||
|
m.kind = "source"
|
||||||
|
m.hash = ""
|
||||||
|
m.provides = new []pointer
|
||||||
|
m.targets = new []pointer
|
||||||
|
m.deps = new []Dep
|
||||||
|
return m
|
||||||
|
}
|
||||||
|
|
||||||
|
# a bare newline string (the lexer has no reliable \n inside interpolation)
|
||||||
|
function nl() -> pointer { let b = bytes(2); b[0] = 10; b[1] = 0; return b }
|
||||||
|
|
||||||
|
# ---- version arithmetic (semver major.minor.patch, pre-release ignored) ------
|
||||||
|
|
||||||
|
# drop a leading 'v' (118) from a tag/version string
|
||||||
|
function strip_v(s: pointer) -> pointer {
|
||||||
|
if slen(s) > 0 and s[0] == 118 { return sslice(s, 1, slen(s)) }
|
||||||
|
return s
|
||||||
|
}
|
||||||
|
|
||||||
|
# major/minor/patch as three ints (missing fields are 0); stops at -/+ metadata
|
||||||
|
function ver_nums(s: pointer) -> []int {
|
||||||
|
let v = strip_v(s)
|
||||||
|
var parts = new []int
|
||||||
|
var cur = 0
|
||||||
|
let n = slen(v)
|
||||||
|
var i = 0
|
||||||
|
while i < n {
|
||||||
|
let c = v[i]
|
||||||
|
if c >= 48 and c <= 57 { cur = cur * 10 + (c - 48) }
|
||||||
|
else if c == 46 { push(parts, cur); cur = 0 }
|
||||||
|
else { i = n } # a '-'/'+' ends the numeric core
|
||||||
|
i = i + 1
|
||||||
|
}
|
||||||
|
push(parts, cur)
|
||||||
|
while len(parts) < 3 { push(parts, 0) }
|
||||||
|
return parts
|
||||||
|
}
|
||||||
|
|
||||||
|
# -1 / 0 / 1 for a < b / a == b / a > b
|
||||||
|
function ver_cmp(a: pointer, b: pointer) -> int {
|
||||||
|
let pa = ver_nums(a)
|
||||||
|
let pb = ver_nums(b)
|
||||||
|
var i = 0
|
||||||
|
while i < 3 {
|
||||||
|
if pa[i] < pb[i] { return 0 - 1 }
|
||||||
|
if pa[i] > pb[i] { return 1 }
|
||||||
|
i = i + 1
|
||||||
|
}
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
function ver_gt(a: pointer, b: pointer) -> bool { return ver_cmp(a, b) > 0 }
|
||||||
|
|
||||||
|
# is `s` a version tag (optional leading v, then a digit)?
|
||||||
|
function is_ver_tag(s: pointer) -> bool {
|
||||||
|
let v = strip_v(s)
|
||||||
|
if slen(v) == 0 { return false }
|
||||||
|
return v[0] >= 48 and v[0] <= 57
|
||||||
|
}
|
||||||
|
|
||||||
|
# ---- tiny line/token scanners for the manifest & lock ------------------------
|
||||||
|
|
||||||
|
# split a manifest/lock line into tokens: "quoted strings" (unquoted) or bare
|
||||||
|
# words. A '#' outside a quote starts a comment that ends the line.
|
||||||
|
function tok_line(line: pointer) -> []pointer {
|
||||||
|
var toks = new []pointer
|
||||||
|
let n = slen(line)
|
||||||
|
var i = 0
|
||||||
|
while i < n {
|
||||||
|
let c = line[i]
|
||||||
|
if c == 35 { i = n } # '#' comment
|
||||||
|
else if c == 32 or c == 9 { i = i + 1 } # whitespace
|
||||||
|
else if c == 34 { # "quoted"
|
||||||
|
i = i + 1
|
||||||
|
let start = i
|
||||||
|
while i < n and line[i] != 34 { i = i + 1 }
|
||||||
|
push(toks, sslice(line, start, i))
|
||||||
|
if i < n { i = i + 1 }
|
||||||
|
} else { # bare word
|
||||||
|
let start = i
|
||||||
|
while i < n and line[i] != 32 and line[i] != 9 and line[i] != 35 { i = i + 1 }
|
||||||
|
push(toks, sslice(line, start, i))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return toks
|
||||||
|
}
|
||||||
|
|
||||||
|
# split a captured multi-line string into trimmed non-empty lines
|
||||||
|
function split_lines(s: pointer) -> []pointer {
|
||||||
|
var out = new []pointer
|
||||||
|
let n = slen(s)
|
||||||
|
var i = 0
|
||||||
|
while i < n {
|
||||||
|
let line = line_at(s, i)
|
||||||
|
i = i + slen(line) + 1
|
||||||
|
let t = s_trim(line)
|
||||||
|
if slen(t) > 0 { push(out, t) }
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
# ---- manifest / lock parsing -------------------------------------------------
|
||||||
|
|
||||||
|
function parse_manifest(text: pointer) -> Manifest {
|
||||||
|
let m = manifest_new()
|
||||||
|
let n = slen(text)
|
||||||
|
var i = 0
|
||||||
|
while i < n {
|
||||||
|
let line = line_at(text, i)
|
||||||
|
i = i + slen(line) + 1
|
||||||
|
let ts = tok_line(line)
|
||||||
|
if len(ts) > 0 {
|
||||||
|
let head = ts[0]
|
||||||
|
if head == "package" { if len(ts) > 1 { m.module = ts[1] } }
|
||||||
|
else if head == "version" { if len(ts) > 1 { m.ver = ts[1] } }
|
||||||
|
else if head == "kind" { if len(ts) > 1 { m.kind = ts[1] } }
|
||||||
|
else if head == "provides" { var k = 1; while k < len(ts) { push(m.provides, ts[k]); k = k + 1 } }
|
||||||
|
else if head == "targets" { var k = 1; while k < len(ts) { push(m.targets, ts[k]); k = k + 1 } }
|
||||||
|
else if head == "require" {
|
||||||
|
if len(ts) > 2 { let d = new Dep; d.module = ts[1]; d.ver = ts[2]; push(m.deps, d) }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return m
|
||||||
|
}
|
||||||
|
|
||||||
|
# read a package.lock.ludic body into a list of pinned entries
|
||||||
|
function parse_lock(text: pointer) -> []Manifest {
|
||||||
|
var out = new []Manifest
|
||||||
|
let n = slen(text)
|
||||||
|
var i = 0
|
||||||
|
while i < n {
|
||||||
|
let line = line_at(text, i)
|
||||||
|
i = i + slen(line) + 1
|
||||||
|
let ts = tok_line(line)
|
||||||
|
if len(ts) > 1 and ts[0] == "module" {
|
||||||
|
let m = manifest_new()
|
||||||
|
m.module = ts[1]
|
||||||
|
var k = 2
|
||||||
|
while k < len(ts) {
|
||||||
|
let key = ts[k]
|
||||||
|
if key == "version" and k + 1 < len(ts) { m.ver = ts[k + 1]; k = k + 2 }
|
||||||
|
else if key == "hash" and k + 1 < len(ts) { m.hash = ts[k + 1]; k = k + 2 }
|
||||||
|
else if key == "kind" and k + 1 < len(ts) { m.kind = ts[k + 1]; k = k + 2 }
|
||||||
|
else if key == "provides" and k + 1 < len(ts) { push(m.provides, ts[k + 1]); k = k + 2 }
|
||||||
|
else { k = k + 1 }
|
||||||
|
}
|
||||||
|
push(out, m)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
# ---- the content-addressed store & git fetch layer --------------------------
|
||||||
|
|
||||||
|
# the global store root: $LUDIC_STORE, else ~/.ludic/store. Trailing slash.
|
||||||
|
function store_root() -> pointer {
|
||||||
|
let s = getenv("LUDIC_STORE")
|
||||||
|
if s != null { return ensure_dir_slash(s) }
|
||||||
|
return ensure_dir_slash(`{getenv_or("HOME", "/tmp")}/.ludic/store`)
|
||||||
|
}
|
||||||
|
function ensure_dir_slash(d: pointer) -> pointer {
|
||||||
|
let n = slen(d)
|
||||||
|
if n == 0 { return d }
|
||||||
|
if d[n - 1] == 47 { return d }
|
||||||
|
return d + "/"
|
||||||
|
}
|
||||||
|
|
||||||
|
# the git URL for a module: a local $LUDIC_PKG_PROXY tree (used by tests and for
|
||||||
|
# mirrors/offline) keyed by the import path, else https://<import-path>.
|
||||||
|
function repo_url(module: pointer) -> pointer {
|
||||||
|
let proxy = getenv("LUDIC_PKG_PROXY")
|
||||||
|
if proxy != null { return `{proxy}/{module}` }
|
||||||
|
return `https://{module}`
|
||||||
|
}
|
||||||
|
|
||||||
|
# ensure a clone cache for `module` under the store; refresh tags if present.
|
||||||
|
# Returns the cache dir, or "" if the clone failed.
|
||||||
|
function ensure_clone(module: pointer) -> pointer {
|
||||||
|
let cache = `{store_root()}cache/{flat(module)}`
|
||||||
|
if file_exists(cache) {
|
||||||
|
run(`git -C {cache} fetch -q --tags 2>/dev/null`)
|
||||||
|
return cache
|
||||||
|
}
|
||||||
|
run(`mkdir -p {store_root()}cache`)
|
||||||
|
if not shq(`git clone -q {repo_url(module)} {cache} 2>/dev/null`) {
|
||||||
|
err(`x: cannot fetch {module} (git clone {repo_url(module)} failed)\n`)
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
return cache
|
||||||
|
}
|
||||||
|
|
||||||
|
# check out version `ver` in the cache — a v-prefixed tag first, then bare.
|
||||||
|
function checkout_ver(cache: pointer, ver: pointer) -> bool {
|
||||||
|
if shq(`git -C {cache} checkout -q v{ver} 2>/dev/null`) { return true }
|
||||||
|
return shq(`git -C {cache} checkout -q {ver} 2>/dev/null`)
|
||||||
|
}
|
||||||
|
|
||||||
|
# the greatest published version tag of `module` (normalised, no leading v)
|
||||||
|
function latest_version(module: pointer) -> pointer {
|
||||||
|
let cache = ensure_clone(module)
|
||||||
|
if cache == "" { return "" }
|
||||||
|
let lines = split_lines(capture(`git -C {cache} tag`))
|
||||||
|
var best = ""
|
||||||
|
var k = 0
|
||||||
|
while k < len(lines) {
|
||||||
|
let tg = lines[k]
|
||||||
|
if is_ver_tag(tg) {
|
||||||
|
let cand = strip_v(tg)
|
||||||
|
if best == "" or ver_gt(cand, best) { best = cand }
|
||||||
|
}
|
||||||
|
k = k + 1
|
||||||
|
}
|
||||||
|
return best
|
||||||
|
}
|
||||||
|
|
||||||
|
# read a package's manifest at `ver`; a package may omit package.ludic, in which
|
||||||
|
# case it is a leaf source package with no declared deps.
|
||||||
|
function fetch_manifest(module: pointer, ver: pointer) -> Manifest {
|
||||||
|
let cache = ensure_clone(module)
|
||||||
|
var m = manifest_new()
|
||||||
|
m.module = module
|
||||||
|
m.ver = ver
|
||||||
|
if cache == "" { return m }
|
||||||
|
if not checkout_ver(cache, ver) {
|
||||||
|
err(`x: {module} has no version {ver}\n`)
|
||||||
|
return m
|
||||||
|
}
|
||||||
|
let txt = read_file(`{cache}/package.ludic`)
|
||||||
|
if txt != null { m = parse_manifest(txt) }
|
||||||
|
m.module = module
|
||||||
|
m.ver = ver
|
||||||
|
return m
|
||||||
|
}
|
||||||
|
|
||||||
|
# the sha256 pick — shasum on macOS, sha256sum elsewhere. Both print "<hash> f".
|
||||||
|
function sha_cmd() -> pointer {
|
||||||
|
if shq("command -v shasum >/dev/null 2>&1") { return "shasum -a 256" }
|
||||||
|
return "sha256sum"
|
||||||
|
}
|
||||||
|
|
||||||
|
# a deterministic content hash of a directory tree (its files' names + bytes),
|
||||||
|
# excluding .git. Independent of file metadata, so it is stable across machines.
|
||||||
|
function dir_hash(dir: pointer) -> pointer {
|
||||||
|
let sc = sha_cmd()
|
||||||
|
return capture_line(`( cd {dir} && find . -type f -not -path './.git/*' | LC_ALL=C sort | while IFS= read -r f; do {sc} "$f"; done | {sc} | cut -d' ' -f1 )`)
|
||||||
|
}
|
||||||
|
|
||||||
|
# snapshot `module`@`ver` into the content-addressed store; returns its hash.
|
||||||
|
# The store entry is immutable and shared: if the hash is already present the
|
||||||
|
# copy is skipped. The .git directory is never stored.
|
||||||
|
function snapshot(module: pointer, ver: pointer) -> pointer {
|
||||||
|
let cache = ensure_clone(module)
|
||||||
|
if cache == "" { return "" }
|
||||||
|
if not checkout_ver(cache, ver) { return "" }
|
||||||
|
let h = dir_hash(cache)
|
||||||
|
if slen(h) == 0 { return "" }
|
||||||
|
let dest = `{store_root()}{h}`
|
||||||
|
if not file_exists(dest) {
|
||||||
|
run(`rm -rf {dest}.tmp`)
|
||||||
|
run(`cp -R {cache} {dest}.tmp`)
|
||||||
|
run(`rm -rf {dest}.tmp/.git`)
|
||||||
|
run(`mv {dest}.tmp {dest}`)
|
||||||
|
}
|
||||||
|
return h
|
||||||
|
}
|
||||||
|
|
||||||
|
# link a project's ludic_modules/<module> view at the store entry `hash`
|
||||||
|
function link_module(module: pointer, hash: pointer) -> void {
|
||||||
|
let dest = `{store_root()}{hash}`
|
||||||
|
let link = `ludic_modules/{module}`
|
||||||
|
run(`mkdir -p "$(dirname {link})"`)
|
||||||
|
run(`rm -rf {link}`)
|
||||||
|
run(`ln -s {dest} {link}`)
|
||||||
|
}
|
||||||
|
|
||||||
|
# ---- MVS resolution ----------------------------------------------------------
|
||||||
|
|
||||||
|
function find_mod(mods: []pointer, m: pointer) -> int {
|
||||||
|
var i = 0
|
||||||
|
while i < len(mods) { if mods[i] == m { return i }; i = i + 1 }
|
||||||
|
return 0 - 1
|
||||||
|
}
|
||||||
|
|
||||||
|
# resolve a root manifest to the selected build list (one Manifest per module,
|
||||||
|
# each at its chosen version). MVS: a module's version is the greatest of every
|
||||||
|
# required minimum; the returned list is the reachable closure at those versions.
|
||||||
|
function resolve(root: Manifest) -> []Manifest {
|
||||||
|
var mods = new []pointer # module -> selected version (parallel arrays)
|
||||||
|
var vers = new []pointer
|
||||||
|
var fm = new []pointer # the current requirement frontier
|
||||||
|
var fv = new []pointer
|
||||||
|
var di = 0
|
||||||
|
while di < len(root.deps) { push(fm, root.deps[di].module); push(fv, root.deps[di].ver); di = di + 1 }
|
||||||
|
|
||||||
|
while len(fm) > 0 {
|
||||||
|
var nm = new []pointer
|
||||||
|
var nv = new []pointer
|
||||||
|
var i = 0
|
||||||
|
while i < len(fm) {
|
||||||
|
let m = fm[i]
|
||||||
|
let v = fv[i]
|
||||||
|
let idx = find_mod(mods, m)
|
||||||
|
var take = false
|
||||||
|
if idx < 0 { push(mods, m); push(vers, v); take = true }
|
||||||
|
else if ver_gt(v, vers[idx]) { vers[idx] = v; take = true }
|
||||||
|
if take {
|
||||||
|
let man = fetch_manifest(m, v) # expand the chosen version's requires
|
||||||
|
var k = 0
|
||||||
|
while k < len(man.deps) { push(nm, man.deps[k].module); push(nv, man.deps[k].ver); k = k + 1 }
|
||||||
|
}
|
||||||
|
i = i + 1
|
||||||
|
}
|
||||||
|
fm = nm
|
||||||
|
fv = nv
|
||||||
|
}
|
||||||
|
|
||||||
|
# reachability closure: keep only modules reachable from the root, each pinned
|
||||||
|
# to its selected version (a module required only by a version later bumped
|
||||||
|
# away drops out — the minimal MVS build list).
|
||||||
|
var out = new []Manifest
|
||||||
|
var seen = new []pointer
|
||||||
|
var q = new []pointer
|
||||||
|
var d2 = 0
|
||||||
|
while d2 < len(root.deps) { push(q, root.deps[d2].module); d2 = d2 + 1 }
|
||||||
|
var qi = 0
|
||||||
|
while qi < len(q) {
|
||||||
|
let m = q[qi]
|
||||||
|
qi = qi + 1
|
||||||
|
if find_mod(seen, m) < 0 {
|
||||||
|
push(seen, m)
|
||||||
|
let idx = find_mod(mods, m)
|
||||||
|
if idx >= 0 {
|
||||||
|
let ver = vers[idx]
|
||||||
|
let man = fetch_manifest(m, ver)
|
||||||
|
push(out, man)
|
||||||
|
var k = 0
|
||||||
|
while k < len(man.deps) { push(q, man.deps[k].module); k = k + 1 }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
# sort a build list by module path (stable enough — insertion sort)
|
||||||
|
function sort_mans(a: []Manifest) -> []Manifest {
|
||||||
|
var i = 1
|
||||||
|
while i < len(a) {
|
||||||
|
let x = a[i]
|
||||||
|
var j = i - 1
|
||||||
|
while j >= 0 and (a[j].module > x.module) { a[j + 1] = a[j]; j = j - 1 }
|
||||||
|
a[j + 1] = x
|
||||||
|
i = i + 1
|
||||||
|
}
|
||||||
|
return a
|
||||||
|
}
|
||||||
|
|
||||||
|
# ---- namespace-collision policy (v1: hard error) ----------------------------
|
||||||
|
# Two packages may not both register the same Foo.* namespace. Returns "" when
|
||||||
|
# clean, else a human message naming the conflicting modules.
|
||||||
|
function collision(sels: []Manifest) -> pointer {
|
||||||
|
var ns = new []pointer
|
||||||
|
var owner = new []pointer
|
||||||
|
var i = 0
|
||||||
|
while i < len(sels) {
|
||||||
|
var p = 0
|
||||||
|
while p < len(sels[i].provides) {
|
||||||
|
let name = sels[i].provides[p]
|
||||||
|
let at = find_mod(ns, name)
|
||||||
|
if at >= 0 { return `namespace {name} provided by both {owner[at]} and {sels[i].module}` }
|
||||||
|
push(ns, name)
|
||||||
|
push(owner, sels[i].module)
|
||||||
|
p = p + 1
|
||||||
|
}
|
||||||
|
i = i + 1
|
||||||
|
}
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
|
||||||
|
# ---- lockfile writing --------------------------------------------------------
|
||||||
|
|
||||||
|
function write_lock(sels: []Manifest) -> bool {
|
||||||
|
var body = "# package.lock.ludic — generated by `x get`. Do not edit by hand." + nl()
|
||||||
|
body = body + "lock 1" + nl()
|
||||||
|
let sorted = sort_mans(sels)
|
||||||
|
var i = 0
|
||||||
|
while i < len(sorted) {
|
||||||
|
let m = sorted[i]
|
||||||
|
var line = `module "{m.module}" version "{m.ver}" hash "{m.hash}" kind "{m.kind}"`
|
||||||
|
var p = 0
|
||||||
|
while p < len(m.provides) { line = line + ` provides "{m.provides[p]}"`; p = p + 1 }
|
||||||
|
body = body + line + nl()
|
||||||
|
i = i + 1
|
||||||
|
}
|
||||||
|
return write_file("package.lock.ludic", body)
|
||||||
|
}
|
||||||
|
|
||||||
|
# ---- the shared get/install pipeline ----------------------------------------
|
||||||
|
# resolve -> collision check -> snapshot each into the store -> link the project
|
||||||
|
# view -> write the lockfile. Returns 0 on success.
|
||||||
|
function do_install(root: Manifest) -> int {
|
||||||
|
let sels = resolve(root)
|
||||||
|
if len(sels) == 0 {
|
||||||
|
print("no dependencies to resolve")
|
||||||
|
write_lock(sels)
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
let clash = collision(sels)
|
||||||
|
if slen(clash) > 0 { err(`x: namespace collision — {clash}\n`); return 1 }
|
||||||
|
|
||||||
|
var i = 0
|
||||||
|
while i < len(sels) {
|
||||||
|
let m = sels[i]
|
||||||
|
if m.kind == "prebuilt" {
|
||||||
|
let t = target_id()
|
||||||
|
if not has_target(m, t) {
|
||||||
|
err(`x: {m.module}@{m.ver} is a prebuilt lib and ships no artifact for target {t}\n`)
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
}
|
||||||
|
let h = snapshot(m.module, m.ver)
|
||||||
|
if slen(h) == 0 { err(`x: failed to snapshot {m.module}@{m.ver}\n`); return 1 }
|
||||||
|
m.hash = `sha256:{h}`
|
||||||
|
link_module(m.module, h)
|
||||||
|
print(` {m.module} {m.ver} ({m.kind}) sha256:{sslice(h, 0, 12)}…`)
|
||||||
|
i = i + 1
|
||||||
|
}
|
||||||
|
if not write_lock(sels) { err("x: cannot write package.lock.ludic\n"); return 1 }
|
||||||
|
print(`resolved {string(len(sels))} package(s) — see package.lock.ludic; linked under ludic_modules/`)
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
|
||||||
|
# the build target id a prebuilt lib is matched against
|
||||||
|
function target_id() -> pointer {
|
||||||
|
let t = getenv("LUDIC_TARGET")
|
||||||
|
if t != null { return t }
|
||||||
|
let os = capture_line("uname -s")
|
||||||
|
let arch = capture_line("uname -m")
|
||||||
|
if os == "Darwin" { return `native-{arch}` }
|
||||||
|
return `native-{arch}`
|
||||||
|
}
|
||||||
|
function has_target(m: Manifest, t: pointer) -> bool {
|
||||||
|
var i = 0
|
||||||
|
while i < len(m.targets) { if m.targets[i] == t { return true }; i = i + 1 }
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
# ---- reading / editing the project manifest ---------------------------------
|
||||||
|
|
||||||
|
# the project manifest in the current directory, or a fresh default one
|
||||||
|
function read_root_manifest() -> Manifest {
|
||||||
|
let txt = read_file("package.ludic")
|
||||||
|
if txt == null { return manifest_new() }
|
||||||
|
return parse_manifest(txt)
|
||||||
|
}
|
||||||
|
|
||||||
|
# split "module@version" into a 2-element list [module, version]; version is ""
|
||||||
|
# when absent
|
||||||
|
function split_spec(spec: pointer) -> []pointer {
|
||||||
|
var out = new []pointer
|
||||||
|
let n = slen(spec)
|
||||||
|
var at = 0 - 1
|
||||||
|
var i = 0
|
||||||
|
while i < n { if spec[i] == 64 { at = i }; i = i + 1 } # '@' = 64
|
||||||
|
if at < 0 { push(out, spec); push(out, "") }
|
||||||
|
else { push(out, sslice(spec, 0, at)); push(out, sslice(spec, at + 1, n)) }
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
# rewrite package.ludic so `module` requires exactly `ver`, creating the file
|
||||||
|
# with sane defaults when it does not exist yet.
|
||||||
|
function set_require(module: pointer, ver: pointer) -> void {
|
||||||
|
var txt = read_file("package.ludic")
|
||||||
|
if txt == null {
|
||||||
|
txt = "# package.ludic — Ludic package manifest" + nl()
|
||||||
|
txt = txt + `package "app"` + nl()
|
||||||
|
txt = txt + `version "0.0.0"` + nl()
|
||||||
|
txt = txt + "kind source" + nl()
|
||||||
|
}
|
||||||
|
var out = ""
|
||||||
|
let n = slen(txt)
|
||||||
|
var i = 0
|
||||||
|
var replaced = false
|
||||||
|
while i < n {
|
||||||
|
let line = line_at(txt, i)
|
||||||
|
i = i + slen(line) + 1
|
||||||
|
let ts = tok_line(line)
|
||||||
|
if len(ts) >= 2 and ts[0] == "require" and ts[1] == module {
|
||||||
|
out = out + `require "{module}" "{ver}"` + nl()
|
||||||
|
replaced = true
|
||||||
|
} else {
|
||||||
|
out = out + line + nl()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if not replaced { out = out + `require "{module}" "{ver}"` + nl() }
|
||||||
|
write_file("package.ludic", out)
|
||||||
|
}
|
||||||
|
|
||||||
|
# ---- the commands ------------------------------------------------------------
|
||||||
|
|
||||||
|
# x add <module>[@version] — add/update a dependency then install
|
||||||
|
function cmd_pkg_add() -> int {
|
||||||
|
if arg_count() < 3 { err("usage: x add <module>[@version]\n"); return 1 }
|
||||||
|
let spec = split_spec(arg(2))
|
||||||
|
let module = spec[0]
|
||||||
|
var ver = spec[1]
|
||||||
|
if slen(ver) == 0 {
|
||||||
|
ver = latest_version(module)
|
||||||
|
if slen(ver) == 0 { err(`x: {module} has no published version tags (git tag vX.Y.Z to publish)\n`); return 1 }
|
||||||
|
print(`x add: {module} -> latest v{ver}`)
|
||||||
|
}
|
||||||
|
set_require(module, ver)
|
||||||
|
return do_install(read_root_manifest())
|
||||||
|
}
|
||||||
|
|
||||||
|
# x get — resolve + fetch + link every dependency in package.ludic, write lock
|
||||||
|
function cmd_pkg_get() -> int {
|
||||||
|
let txt = read_file("package.ludic")
|
||||||
|
if txt == null { err("x: no package.ludic in the current directory (x add <module> to start one)\n"); return 1 }
|
||||||
|
print("resolving dependencies (MVS)…")
|
||||||
|
return do_install(parse_manifest(txt))
|
||||||
|
}
|
||||||
|
|
||||||
|
# x update [module] — bump a dep (or all) to its latest published version, relock
|
||||||
|
function cmd_pkg_update() -> int {
|
||||||
|
let root = read_root_manifest()
|
||||||
|
if len(root.deps) == 0 { err("x: package.ludic declares no dependencies\n"); return 1 }
|
||||||
|
let only = argn(2, "")
|
||||||
|
var i = 0
|
||||||
|
while i < len(root.deps) {
|
||||||
|
let d = root.deps[i]
|
||||||
|
if only == "" or only == d.module {
|
||||||
|
let latest = latest_version(d.module)
|
||||||
|
if slen(latest) > 0 and ver_gt(latest, d.ver) {
|
||||||
|
print(`x update: {d.module} {d.ver} -> {latest}`)
|
||||||
|
set_require(d.module, latest)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
i = i + 1
|
||||||
|
}
|
||||||
|
return do_install(read_root_manifest())
|
||||||
|
}
|
||||||
|
|
||||||
|
# x verify — check every locked package against the store by content hash and
|
||||||
|
# confirm the project view links to it
|
||||||
|
function cmd_pkg_verify() -> int {
|
||||||
|
let txt = read_file("package.lock.ludic")
|
||||||
|
if txt == null { err("x: no package.lock.ludic (run x get first)\n"); return 1 }
|
||||||
|
let locked = parse_lock(txt)
|
||||||
|
if len(locked) == 0 { print("lockfile lists no packages"); return 0 }
|
||||||
|
var bad_count = 0
|
||||||
|
var i = 0
|
||||||
|
while i < len(locked) {
|
||||||
|
let m = locked[i]
|
||||||
|
let want = m.hash
|
||||||
|
let raw = strip_prefix(want, "sha256:")
|
||||||
|
let dest = `{store_root()}{raw}`
|
||||||
|
if not file_exists(dest) {
|
||||||
|
print(` MISSING {m.module}@{m.ver} (store entry {want} absent)`)
|
||||||
|
bad_count = bad_count + 1
|
||||||
|
} else {
|
||||||
|
let got = dir_hash(dest)
|
||||||
|
if got == raw {
|
||||||
|
let link = `ludic_modules/{m.module}`
|
||||||
|
if file_exists(link) { print(` ok {m.module}@{m.ver}`) }
|
||||||
|
else { print(` UNLINKED {m.module}@{m.ver} (ludic_modules view missing — run x get)`); bad_count = bad_count + 1 }
|
||||||
|
} else {
|
||||||
|
print(` TAMPERED {m.module}@{m.ver} (want sha256:{sslice(raw, 0, 12)}… got sha256:{sslice(got, 0, 12)}…)`)
|
||||||
|
bad_count = bad_count + 1
|
||||||
|
}
|
||||||
|
}
|
||||||
|
i = i + 1
|
||||||
|
}
|
||||||
|
if bad_count == 0 { print(`verified {string(len(locked))} package(s) against the store`); return 0 }
|
||||||
|
err(`x: {string(bad_count)} package(s) failed verification\n`)
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
|
||||||
|
# x vendor — copy the resolved packages into ./vendor for hermetic/offline
|
||||||
|
# builds. Build against them with LUDIC_MODULES=vendor.
|
||||||
|
function cmd_pkg_vendor() -> int {
|
||||||
|
let txt = read_file("package.lock.ludic")
|
||||||
|
if txt == null { err("x: no package.lock.ludic (run x get first)\n"); return 1 }
|
||||||
|
let locked = parse_lock(txt)
|
||||||
|
run("rm -rf vendor")
|
||||||
|
var i = 0
|
||||||
|
while i < len(locked) {
|
||||||
|
let m = locked[i]
|
||||||
|
let raw = strip_prefix(m.hash, "sha256:")
|
||||||
|
let dest = `{store_root()}{raw}`
|
||||||
|
if not file_exists(dest) { err(`x: {m.module}@{m.ver} not in the store — run x get\n`); return 1 }
|
||||||
|
let vdir = `vendor/{m.module}`
|
||||||
|
run(`mkdir -p "$(dirname {vdir})"`)
|
||||||
|
run(`cp -R {dest} {vdir}`)
|
||||||
|
print(` vendored {m.module}@{m.ver}`)
|
||||||
|
i = i + 1
|
||||||
|
}
|
||||||
|
print(`vendored {string(len(locked))} package(s) into ./vendor — build offline with LUDIC_MODULES=vendor`)
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
|
||||||
|
# drop a leading `prefix` from `s` if present
|
||||||
|
function strip_prefix(s: pointer, prefix: pointer) -> pointer {
|
||||||
|
let pn = slen(prefix)
|
||||||
|
if pn <= slen(s) and s_starts(s, prefix) { return sslice(s, pn, slen(s)) }
|
||||||
|
return s
|
||||||
|
}
|
||||||
145
tools/x/pkg_test.ludic
Normal file
145
tools/x/pkg_test.ludic
Normal file
|
|
@ -0,0 +1,145 @@
|
||||||
|
# pkg_test.ludic — the package-manager suite (issue #63), hermetic and offline.
|
||||||
|
#
|
||||||
|
# It stands up throwaway git repositories under /tmp and points the package
|
||||||
|
# manager at them with LUDIC_PKG_PROXY, so the whole fetch → MVS resolve →
|
||||||
|
# content-addressed store → project link → compile → run → verify pipeline runs
|
||||||
|
# with no network. Every check drives a fresh `bin/x` / `bin/ludicc` process the
|
||||||
|
# way a real project would.
|
||||||
|
|
||||||
|
# write `body` to `path`, creating parent directories
|
||||||
|
function pt_write(path: pointer, body: pointer) -> void {
|
||||||
|
run(`mkdir -p "$(dirname {path})"`)
|
||||||
|
write_file(path, body)
|
||||||
|
}
|
||||||
|
|
||||||
|
# commit the current tree of `dir` and tag it `tag`
|
||||||
|
function pt_commit_tag(dir: pointer, tag: pointer) -> void {
|
||||||
|
run(`git -C {dir} add -A`)
|
||||||
|
run(`git -C {dir} -c user.email=t@t.test -c user.name=tester commit -q -m {tag}`)
|
||||||
|
run(`git -C {dir} tag {tag}`)
|
||||||
|
}
|
||||||
|
|
||||||
|
function cmd_test_pkg() -> int {
|
||||||
|
PASS = 0
|
||||||
|
FAIL = 0
|
||||||
|
print("== package manager (issue #63): fetch + MVS resolve + store + link + build ==")
|
||||||
|
|
||||||
|
if not shq("command -v git >/dev/null 2>&1") {
|
||||||
|
skip("git unavailable — package-manager suite needs git")
|
||||||
|
return report()
|
||||||
|
}
|
||||||
|
|
||||||
|
let root = capture_line("pwd")
|
||||||
|
let work = "/tmp/x_pkg_test"
|
||||||
|
let proxy = `{work}/proxy`
|
||||||
|
let store = `{work}/store`
|
||||||
|
let proj = `{work}/proj`
|
||||||
|
run(`rm -rf {work}`)
|
||||||
|
run(`mkdir -p {proxy} {store} {proj}`)
|
||||||
|
|
||||||
|
# ---- fixture packages -----------------------------------------------------
|
||||||
|
# example.test/util: a leaf source package, two published versions.
|
||||||
|
let util = `{proxy}/example.test/util`
|
||||||
|
run(`git -C {util} init -q 2>/dev/null || ( mkdir -p {util} && git -C {util} init -q )`)
|
||||||
|
pt_write(`{util}/package.ludic`, `package "example.test/util"` + nl() + `version "1.0.0"` + nl() + "kind source" + nl() + `provides "Util"` + nl())
|
||||||
|
pt_write(`{util}/util.ludic`, "function util_tag() -> pointer { return \"u1\" }\n")
|
||||||
|
pt_commit_tag(util, "v1.0.0")
|
||||||
|
pt_write(`{util}/package.ludic`, `package "example.test/util"` + nl() + `version "1.2.0"` + nl() + "kind source" + nl() + `provides "Util"` + nl())
|
||||||
|
pt_write(`{util}/util.ludic`, "function util_tag() -> pointer { return \"u12\" }\n")
|
||||||
|
pt_commit_tag(util, "v1.2.0")
|
||||||
|
|
||||||
|
# example.test/greeter: requires util >=1.0.0, provides Greet.
|
||||||
|
let greeter = `{proxy}/example.test/greeter`
|
||||||
|
run(`mkdir -p {greeter} && git -C {greeter} init -q`)
|
||||||
|
pt_write(`{greeter}/package.ludic`, `package "example.test/greeter"` + nl() + `version "1.0.0"` + nl() + "kind source" + nl() + `provides "Greet"` + nl() + `require "example.test/util" "1.0.0"` + nl())
|
||||||
|
pt_write(`{greeter}/greet.ludic`, "function greet_hello() -> pointer { return \"hi\" }\n")
|
||||||
|
pt_commit_tag(greeter, "v1.0.0")
|
||||||
|
|
||||||
|
# ---- the consumer project -------------------------------------------------
|
||||||
|
# requires util 1.2.0 directly; greeter (which requires util 1.0.0) is added
|
||||||
|
# too. MVS must pick util 1.2.0 — the greatest required minimum.
|
||||||
|
pt_write(`{proj}/package.ludic`, `package "app"` + nl() + `version "0.0.0"` + nl() + `require "example.test/greeter" "1.0.0"` + nl() + `require "example.test/util" "1.2.0"` + nl())
|
||||||
|
pt_write(`{proj}/app.ludic`, "program App {\n import \"example.test/greeter/greet.ludic\"\n import \"example.test/util/util.ludic\"\n entry { print(greet_hello()); print(util_tag()) }\n}\n")
|
||||||
|
|
||||||
|
let envp = `LUDIC_PKG_PROXY={proxy} LUDIC_STORE={store}`
|
||||||
|
|
||||||
|
# ---- x get: resolve, fetch, store, link, lock -----------------------------
|
||||||
|
if shq(`( cd {proj} && {envp} {root}/bin/x get > {work}/get.out 2>&1 )`) {
|
||||||
|
ok("x get resolves + fetches the dependency graph")
|
||||||
|
} else {
|
||||||
|
bad2("x get failed", capture_line(`tail -2 {work}/get.out`))
|
||||||
|
}
|
||||||
|
|
||||||
|
var lock = read_file(`{proj}/package.lock.ludic`)
|
||||||
|
if lock == null { lock = "" }
|
||||||
|
if s_contains(lock, "example.test/greeter") and s_contains(lock, "example.test/util") {
|
||||||
|
ok("package.lock.ludic pins the whole build list")
|
||||||
|
} else { bad("lockfile is missing a resolved package") }
|
||||||
|
|
||||||
|
# MVS: util resolves to 1.2.0 (max of the 1.0.0 and 1.2.0 minimums), not 1.0.0
|
||||||
|
if s_contains(lock, `"example.test/util" version "1.2.0"`) {
|
||||||
|
ok("MVS selects util 1.2.0 (greatest required minimum)")
|
||||||
|
} else { bad2("MVS picked the wrong util version", capture_line(`grep util {proj}/package.lock.ludic`)) }
|
||||||
|
|
||||||
|
if s_contains(lock, "sha256:") { ok("lockfile records content hashes") } else { bad("lockfile has no content hash") }
|
||||||
|
|
||||||
|
# the project view links into the content-addressed store
|
||||||
|
if file_exists(`{proj}/ludic_modules/example.test/greeter/greet.ludic`) and file_exists(`{proj}/ludic_modules/example.test/util/util.ludic`) {
|
||||||
|
ok("ludic_modules/ links the store entries into the project")
|
||||||
|
} else { bad("ludic_modules view was not linked") }
|
||||||
|
|
||||||
|
# ---- compile + run the consumer against the fetched packages --------------
|
||||||
|
if shq(`( cd {proj} && LUDIC_MODULES=ludic_modules {root}/bin/ludicc app.ludic -o app > {work}/build.out 2>&1 )`) {
|
||||||
|
ok("consumer compiles against the linked packages (do_import module-root fallback)")
|
||||||
|
let got = capture_line(`( cd {proj} && ./app )`)
|
||||||
|
if got == "hi u12" { ok("the running program uses the MVS-selected package code (\"hi u12\")") }
|
||||||
|
else { bad2("program output mismatch", `got [{got}] want [hi u12]`) }
|
||||||
|
} else {
|
||||||
|
bad2("consumer build failed", capture_line(`tail -2 {work}/build.out`))
|
||||||
|
}
|
||||||
|
|
||||||
|
# ---- x verify: content integrity against the store ------------------------
|
||||||
|
if shq(`( cd {proj} && {envp} {root}/bin/x verify > {work}/verify.out 2>&1 )`) {
|
||||||
|
ok("x verify passes on an untouched store")
|
||||||
|
} else { bad2("x verify failed unexpectedly", capture_line(`tail -2 {work}/verify.out`)) }
|
||||||
|
|
||||||
|
# tamper with a stored file — verify must now flag it
|
||||||
|
let uhash = capture_line(`grep 'example.test/util' {proj}/package.lock.ludic | sed 's/.*sha256://; s/\".*//'`)
|
||||||
|
run(`echo tampered >> {store}/{uhash}/util.ludic`)
|
||||||
|
if not shq(`( cd {proj} && {envp} {root}/bin/x verify > {work}/verify2.out 2>&1 )`) {
|
||||||
|
ok("x verify detects a tampered store entry")
|
||||||
|
} else { bad("x verify missed a tampered store entry") }
|
||||||
|
# restore the store for the remaining checks
|
||||||
|
run(`( cd {proj} && {envp} {root}/bin/x get > /dev/null 2>&1 )`)
|
||||||
|
|
||||||
|
# ---- namespace collision policy (v1: hard error) --------------------------
|
||||||
|
let dupe = `{proxy}/example.test/dupe`
|
||||||
|
run(`mkdir -p {dupe} && git -C {dupe} init -q`)
|
||||||
|
pt_write(`{dupe}/package.ludic`, `package "example.test/dupe"` + nl() + `version "1.0.0"` + nl() + "kind source" + nl() + `provides "Util"` + nl())
|
||||||
|
pt_write(`{dupe}/dupe.ludic`, "function dupe_tag() -> pointer { return \"d\" }\n")
|
||||||
|
pt_commit_tag(dupe, "v1.0.0")
|
||||||
|
if not shq(`( cd {proj} && {envp} {root}/bin/x add example.test/dupe@1.0.0 > {work}/dupe.out 2>&1 )`) and shq(`grep -q collision {work}/dupe.out`) {
|
||||||
|
ok("two packages claiming the same Foo.* namespace is a hard error")
|
||||||
|
} else { bad2("collision not rejected", capture_line(`tail -1 {work}/dupe.out`)) }
|
||||||
|
# drop the bad dep again so the manifest is clean
|
||||||
|
pt_write(`{proj}/package.ludic`, `package "app"` + nl() + `version "0.0.0"` + nl() + `require "example.test/greeter" "1.0.0"` + nl() + `require "example.test/util" "1.2.0"` + nl())
|
||||||
|
|
||||||
|
# ---- prebuilt target matrix ----------------------------------------------
|
||||||
|
let pb = `{proxy}/example.test/prebuilt`
|
||||||
|
run(`mkdir -p {pb} && git -C {pb} init -q`)
|
||||||
|
pt_write(`{pb}/package.ludic`, `package "example.test/prebuilt"` + nl() + `version "1.0.0"` + nl() + "kind prebuilt" + nl() + `provides "Blob"` + nl() + `targets "made-up-target"` + nl())
|
||||||
|
pt_write(`{pb}/README`, "a prebuilt lib shipping no artifact for this host\n")
|
||||||
|
pt_commit_tag(pb, "v1.0.0")
|
||||||
|
if not shq(`( cd {proj} && {envp} {root}/bin/x add example.test/prebuilt@1.0.0 > {work}/pb.out 2>&1 )`) and shq(`grep -q 'prebuilt lib' {work}/pb.out`) {
|
||||||
|
ok("a prebuilt lib with no artifact for the build target is a hard error")
|
||||||
|
} else { bad2("prebuilt target matrix not enforced", capture_line(`tail -1 {work}/pb.out`)) }
|
||||||
|
pt_write(`{proj}/package.ludic`, `package "app"` + nl() + `version "0.0.0"` + nl() + `require "example.test/greeter" "1.0.0"` + nl() + `require "example.test/util" "1.2.0"` + nl())
|
||||||
|
|
||||||
|
# ---- x vendor: hermetic copy ---------------------------------------------
|
||||||
|
run(`( cd {proj} && {envp} {root}/bin/x get > /dev/null 2>&1 )`)
|
||||||
|
if shq(`( cd {proj} && {envp} {root}/bin/x vendor > {work}/vendor.out 2>&1 )`) and file_exists(`{proj}/vendor/example.test/util/util.ludic`) {
|
||||||
|
ok("x vendor copies the resolved packages into ./vendor")
|
||||||
|
} else { bad2("x vendor failed", capture_line(`tail -1 {work}/vendor.out`)) }
|
||||||
|
|
||||||
|
return report()
|
||||||
|
}
|
||||||
|
|
@ -326,6 +326,17 @@ function cmd_test() -> int {
|
||||||
if (rc == 42) { ok("ludic app.ludic -> compiles, runs, forwards exit code") }
|
if (rc == 42) { ok("ludic app.ludic -> compiles, runs, forwards exit code") }
|
||||||
else { bad2("ludic run: expected exit 42", `got {string(rc)}`) }
|
else { bad2("ludic run: expected exit 42", `got {string(rc)}`) }
|
||||||
|
|
||||||
|
# the package manager (issue #63): fetch + MVS resolve + content-addressed
|
||||||
|
# store + ludic_modules/ links + build, run as its own hermetic suite (it
|
||||||
|
# stands up throwaway git repos, so like selfhost-test it runs as a sub-process
|
||||||
|
# and this stage checks its footer).
|
||||||
|
print("== package manager (x test-pkg) ==")
|
||||||
|
run("bin/x test-pkg > /tmp/x_pkg.out 2>&1")
|
||||||
|
if shq("grep -q '0 failed' /tmp/x_pkg.out") {
|
||||||
|
let pc = capture_line("grep -c PASS /tmp/x_pkg.out")
|
||||||
|
ok(`package-manager suite: {pc} checks passed (see: x test-pkg)`)
|
||||||
|
} else { bad2("x test-pkg", capture_line("grep -i fail /tmp/x_pkg.out | head -1")) }
|
||||||
|
|
||||||
# docs site generator + guard, in Ludic (no Python). docs-gen emits the whole
|
# docs site generator + guard, in Ludic (no Python). docs-gen emits the whole
|
||||||
# pages payload; docs-check is its coverage/integrity guard; docs-palette is
|
# pages payload; docs-check is its coverage/integrity guard; docs-palette is
|
||||||
# the named-colour source of truth (its output is the tracked emit_color.ludic
|
# the named-colour source of truth (its output is the tracked emit_color.ludic
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue