workshopsoft/ludic
2
0
Fork 0
Code Issues Pull requests Projects Releases 26 Packages Wiki Activity Actions

Merge lang/i18nkeys (27024de) into lang/syntax: the vocabulary gains the text keys - the Key type, @TextKey (a registry's), the k"..." and kn"..." literals - and @Text's doc; docs/language pages for Key and @TextKey; ludic-dev syntax rewrote the grammars; reseeded, bootstrap-cfree fixpoint holds

Browse source 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Orkun ÇAKILKAYA 2026-09-30 00:02:48 +03:00
commit 9f8aec3bc1
63 changed files with 145288 additions and 114562 deletions
Download patch file Download diff file Expand all files Collapse all files

78
LANGUAGE.md
View file

@ -800,6 +800,7 @@ object with `"schema_version": 1`:
`handler` the `fn` value it passes last (else null). A tag worked out at run time cannot be known
and is left out.
- `units` - `@Unit`'s canonical spellings.
- `lang` - the text keys and the languages (see Text keys), or null without a `lang` line.
Each list is sorted by name (then file and line), a registry's entries are in index order, and the
paths are the ones the compiler read, so two runs over the same source write the same file. The
@ -810,6 +811,82 @@ worked out once and only when the program names it - so a tool talking to a runn
whether it was compiled from the data in front of it. It is 0 in a release (`ludicc --release`,
which `ludic bundle` passes).
### Text keys (`Key`, `k"..."`, `lang`)
A text the player reads is named by a KEY, and the key's English is a language file like any
other's. `package.ludic` says where the languages are and which is the source:
```
lang "assets/lang" en # the directory, and the source language: assets/lang/en.po
```
In code a key is a literal of the builtin type `Key` - `k"module.purpose"`, or `kn"..."` for a key
whose text has plural forms - written against its quote (`k "x"` is a name and a string, as ever):
```ludic
# doc-check: skip — tr / trf / trn are ludic.i18n's
let title = tr(k"pause.resume") # tr(key: Key) -> string
let day = trf(k"hud.day", txt_num(n)) # the English's holes {1}..{4}, in order
let got = trn(kn"catch.count", n, fish) # {1} is the count, the rest follow
```
A `Key` is not a `string`, and a `string` is not a `Key`: giving one where the other is wanted is an
error either way, and so is `+` on a key. Keys compare with `==` and `!=`, and are fields, parameters,
results, list elements and registry values like any other type (`null` is none). At run time a key is
its text after a marker byte - byte 1, or 2 for `kn"..."` - so `k"pause.resume"` is the string
`"\x01pause.resume"`: the runtime's `tr` is a cast, and a translator knows a key from English.
`string(key)` gives that marked text, for a runtime that needs it; a program itself makes text with
`tr`, and a key in a template literal's hole (`` `at {k}` ``) is an error - write `trf(k"...", ...)`.
`trn` takes a plural key and `tr` / `trf` refuse one, with or without a `lang` line. (A program that
declares its own type named `Key` keeps it; the builtin is then out of reach.)
**The data.** A registry's `@Text` field is text by derivation: its key is
`<registry>.<row>.<field>` - the registry's name in snake case (`Items` -> `items`, `GearKinds` ->
`gear_kinds`), a list field adding `.<index>` and a nested record `.<field>` - or, with
`@TextKey("steps")` on the registry, `steps.<row>.<field>`. A `@PerMap` table's is
`maps.<map>.<table>.<row>.<field>`. When the field's type is `Key` and a row of a compiled registry
gives it no value, the compiler fills in the derived key (marked, as a literal is), so code writes
`tr(Items[i].name)` and the `.lres` never spells a key; a row may still name one, `name:
k"items.lamp.name"`, in a resource file or a map's file alike.
```ludic
# doc-check: skip — the data file is the game's
property Item {
key: string = ""
@Text name: Key = null # items.<row>.name, filled in when the row gives none
}
registry Items of Item as IT from "data/items.lres"
@TextKey("steps") registry StepKinds of StepKind from "steps.lres" # steps.<row>.<field>
```
**The templates.** A component's text is a key too: `{t('pause.resume')}`, `title="{t('pause.close')}"`,
`{t('hud.day', day)}`; `t(expr)` takes a key worked out at run time (a `Key` field reaches a template
as its marked text). Words outside an element with `translate="no"` are English still waiting for a
key.
**The checks.** With a `lang` line and its source `.po` there, the compiler holds every key to it,
each diagnostic at its `file:line:col`:
- a `k"..."` literal, a template's `t('...')` literal, or a key a data row names or derives that the
source `.po` does not have is an **error**; so is a `kn"..."` whose entry has no `msgid_plural`;
- a call of `trf` or `trn` with a key literal, and a template's `t('key', ...)`, gives as many values
after the key as the English's highest hole `{n}` (`trn`'s count is `{1}`), or it is a **warning**;
- **English left** - a template's words outside `translate="no"`, a text attribute's words (`title`,
`label`, `hint`, `text`, `caption`, and any other whose value is not a keyword), a quoted choice
inside a hole that reads as words, and a `@Text` row whose value is still English - is a
**warning**, and `ludic deps` counts them as `english_left`, a ratchet like every number there;
- one English under several keys (a split) wants a `#.` description on each, for a translator to
tell them apart: a **warning** at the `.po`'s line.
With no `lang` line nothing is checked and nothing is said (a package test uses key literals freely); a
`lang` line whose source `.po` is missing checks nothing, and the first key literal says so once.
The source `.po` is gettext's: `msgid` (the key), `msgid_plural` and `msgstr[n]`, `#,` flags
(`fuzzy`), `#.` descriptions, `msgctxt` (read and ignored: the key is the context), strings over several lines; `#~`
entries are obsolete. `ludic schema` carries a `"lang"` object (null without a `lang` line): every
key used with its kind (`code`, `template`, `data`), its sites, its English, whether it is a plural,
and its description; `unused` (the source's keys used nowhere), `undescribed`, `split`, and per other
language in the directory its `missing`, `fuzzy` and `extra` keys.
### Default parameters, and calls that name what they change
A parameter can have a default, and a call leaves out what it does not change - the last ones when
@ -1330,6 +1407,7 @@ for a complete title screen.
| `bool` | boolean | `i32` |
| `entity` | entity handle | `i32` |
| `string` | text (a string literal, an interpolation, a concatenation) | `ptr` |
| `Key` | a text key, `k"pause.resume"` (see Text keys) - not a `string`, and no string is one | `ptr` |
| `pointer` | raw address (runtime/FFI, records, anything untyped) | `ptr` |
| `byte` | one byte value (what `p[i]` on a `bytes` buffer reads) | `i8` |
| `bytes` | buffer of bytes — `b[i]` reads/writes one byte | `ptr` |

16
changes/text-keys.md Normal file
View file

@ -0,0 +1,16 @@
bump: minor
type: feature
**Text keys: `k"pause.resume"`, a `Key`, checked against `en.po`.** A text the player reads is named by
a key of the new builtin type `Key` (`kn"..."` for a plural) - not a string, and no string one; at run
time its text after a marker byte, so a runtime's `tr` is a cast. `lang "assets/lang" en` in
`package.ludic` names the source language's `.po`, and the compiler holds every key to it: a key
literal, a template's `t('key', ...)` and a registry's derived `@Text` key (`<registry>.<row>.<field>`,
or `@TextKey("steps")`'s prefix; filled into a `Key` field a row leaves out) must be there, a plural
one with a `msgid_plural`; `trf` / `trn` / `t()` given another number of values than the English has
holes, text still written in English (a template's words, a `@Text` row's value - counted as
`english_left` by `ludic deps`) and one English split into keys without `#.` descriptions are
warnings. `ludic schema` gains `"lang"`: every key with its sites, unused and undescribed keys, splits,
and each other language's missing, fuzzy and extra keys.
A `Key` in a template literal's hole is an error (write `trf(k"...", ...)`), `trn` takes only a plural
key and `tr` / `trf` refuse one; a program with no `lang` line (a package test) uses key literals
unchecked and silently.

1
docs/SHIPPING.md
View file

@ -49,6 +49,7 @@ pack "assets"
| `app out` | where to write the bundle | `build/<name>.app` |
| `pack` | an asset root, repeatable | `assets/` if it exists |
| `maps` | where the maps' own tables are, one directory a map (`@PerMap` registries, LANGUAGE.md) - the compiler builds it into the program and `ludicc --check` reads every map under it | `assets/maps` |
| `lang` | the languages' directory and the source language (`lang "assets/lang" en`): the compiler checks every text key against `<dir>/<source>.po` (LANGUAGE.md, Text keys) | none: no key is checked |
Only `app name` is worth setting deliberately. The identifier is derived from the
package path when you omit it - `git.workshopsoft.io/workshopsoft/maroon-lake`

4
docs/language/annotations/annot-text.md
View file

@ -4,9 +4,9 @@ name: @Text
category: annotations
kind: annotation
tokens: @Text @Multiline
sig: @Text @Multiline field: string = ""
sig: @Text name: Key = null / @Text @Multiline blurb: string = ""
tip: Read by the player, so translated; and prose, edited over several lines.
order: 72
---
<code>@Text</code> marks a string the player reads, so it is translated; <code>@Multiline</code> says it is prose, edited as several lines rather than one.
<code>@Text</code> marks what the player reads, so it is translated. On a <code>Key</code> field a row that gives no value gets its derived key, <code>&lt;registry&gt;.&lt;row&gt;.&lt;field&gt;</code> (see <code>@TextKey</code>); English still written in a <code>@Text</code> row is counted as <code>english_left</code>. <code>@Multiline</code> says it is prose, edited as several lines rather than one.

12
docs/language/annotations/annot-textkey.md Normal file
View file

@ -0,0 +1,12 @@
---
id: annot-textkey
name: @TextKey
category: annotations
kind: annotation
tokens: @TextKey
sig: @TextKey("steps") registry StepKinds of StepKind from "steps.lres"
tip: The prefix of a registry's derived text keys, in place of its name in snake case.
order: 73
---
A registry's <code>@Text</code> fields are keyed <code>&lt;registry&gt;.&lt;row&gt;.&lt;field&gt;</code>, the registry's name in snake case (<code>GearKinds</code> is <code>gear_kinds</code>). <code>@TextKey("steps")</code> names the prefix instead, so its rows' keys are <code>steps.&lt;row&gt;.&lt;field&gt;</code>. Every derived key is checked against the source language's <code>.po</code> (LANGUAGE.md, Text keys).

19
docs/language/types/type-key.md Normal file
View file

@ -0,0 +1,19 @@
---
id: type-key
name: Key
category: types
kind: type
tokens: Key
sig: k"module.purpose" / kn"module.purpose"
tip: A text key: it names a text and is not one - tr(key) makes the text.
order: 4
---
A `Key` names a text the player reads; its English lives in the source language's `.po`, like every other language's text. It is written `k"module.purpose"`, or `kn"..."` for a text with plural forms, against its quote. A `Key` is not a `string` and a `string` is not a `Key`: text is made with `tr(key)`, `trf(key, ...)` or, for a plural, `trn(kn"...", n, ...)`, and a key in a template literal's hole is an error. Keys compare with `==`, and with a `lang` line in `package.ludic` the compiler checks every one against the `.po`.
```ludic
# doc-check: skip — tr / trf / trn are ludic.i18n's
let title = tr(k"pause.resume")
let day = trf(k"hud.day", txt_num(n))
if key == k"pause.resume" { Screen.status(title) }
```

3
examples/lang/i18n/data/i18n_items.lres Normal file
View file

@ -0,0 +1,3 @@
# i18n.ludic's items: a @Text name is a Key, derived when the row gives none (items.rope.name)
rope { weight: 0.4 }
lamp { name: k"items.lamp.name", weight: 1.2 }

41
examples/lang/i18n/i18n.ludic Normal file
View file

@ -0,0 +1,41 @@
# i18n.ludic — keys, not English (LANGUAGE.md, "Text keys"). A text the player reads is named by a
# key, k"pause.resume", of the builtin type Key: not a string, and no string is one. Its English is
# lang/en.po (package.ludic says `lang "lang" en`), and the compiler checks every key against it. A
# registry's @Text Key field a row leaves out is filled with its derived key - <registry>.<row>.<field>,
# or the registry's @TextKey prefix. At run time a Key is its text after a marker byte (1, or 2 for a
# plural kn"..."), so ludic.i18n's tr() is a cast; here tr, trf and trn are stand-ins that show the key.
#
# Running it prints: pause.resume yes no 13 12 hud.day(3) catch.count[2](trout) items.rope.name items.lamp.name steps.gather_wood.hint
program I18n {
property Item {
key: string = ""
@Text name: Key = null
weight: float = 0.0
}
registry Items of Item as IT from "data/i18n_items.lres"
property StepKind {
key: string = ""
@Text hint: Key = null
}
@TextKey("steps") registry StepKinds of StepKind as SK
def StepKinds gather_wood { }
# ludic.i18n's three, stood in for: the key's text without its marker
function tr(key: Key) -> string {
let s = string(key)
return s[1..len(s)]
}
function trf(key: Key, a: string) -> string { return `{tr(key)}({a})` }
function trn(key: Key, n: int, a: string) -> string { return `{tr(key)}[{n}]({a})` }
function yn(b: bool) -> string {
if b { return "yes" }
return "no"
}
entry {
let resume = k"pause.resume"
let same = resume == k"pause.resume"
let other = resume == k"pause.close"
var out = `{tr(resume)} {yn(same)} {yn(other)} {len(string(resume))} {len(string(kn"catch.count"))} {trf(k"hud.day", "3")}`
out = out + ` {trn(kn"catch.count", 2, "trout")} {tr(Items[IT_ROPE].name)} {tr(Items[IT_LAMP].name)} {tr(StepKinds[SK_GATHER_WOOD].hint)}`
print(out)
}
}

9
examples/lang/i18n/i18n_ui.ludic Normal file
View file

@ -0,0 +1,9 @@
# i18n_ui.ludic — a component's template against lang/en.po: every t('key', ...) in a {hole} names
# a key en.po has, and gives as many values as its English has holes. Words a player would read that
# are not a key are "English left" (a warning, counted by ludic deps as english_left) - here the
# "Paused" line; an element with translate="no" is left alone. `ludicc --check` reports both.
import "ludic.ui"
program I18nUi {
import "parts/Pause.ludic"
entry { print(1) }
}

40
examples/lang/i18n/lang/en.po Normal file
View file

@ -0,0 +1,40 @@
# The example's source language: the English of every key i18n.ludic and i18n_ui.ludic use.
msgid ""
msgstr ""
"Content-Type: text/plain; charset=UTF-8\n"
"Plural-Forms: nplurals=2; plural=(n != 1);\n"
#. On the pause menu: returns to play.
msgid "pause.resume"
msgstr "Resume"
#. On the pause menu: closes the menu and returns to play.
msgid "pause.close"
msgstr "Close"
#. On the map screen: closes the map.
msgid "map.close"
msgstr "Close"
#. The day on the HUD. {1} = the day number
msgid "hud.day"
msgstr "Day {1}"
#. A catch, counted. {1} = how many, {2} = the species
msgid "catch.count"
msgid_plural "catch.count"
msgstr[0] "{1} {2} caught"
msgstr[1] "{1} {2} "
"caught"
#. An item's name: a coil of rope.
msgid "items.rope.name"
msgstr "Rope"
#. An item's name: a brass lamp.
msgid "items.lamp.name"
msgstr "Lamp"
#. What a chapter step says to do: gather firewood.
msgid "steps.gather_wood.hint"
msgstr "Gather wood"

32
examples/lang/i18n/lang/tr.po Normal file
View file

@ -0,0 +1,32 @@
# Turkish: one key missing (hud.day), one fuzzy (pause.close) and one en.po does not have (old.title)
msgid ""
msgstr ""
"Content-Type: text/plain; charset=UTF-8\n"
"Plural-Forms: nplurals=2; plural=(n != 1);\n"
msgid "pause.resume"
msgstr "Devam et"
#, fuzzy
msgid "pause.close"
msgstr "Kapat"
msgid "map.close"
msgstr "Kapat"
msgid "catch.count"
msgid_plural "catch.count"
msgstr[0] "{1} {2} tutuldu"
msgstr[1] "{1} {2} tutuldu"
msgid "items.rope.name"
msgstr "Halat"
msgid "items.lamp.name"
msgstr "Fener"
msgid "steps.gather_wood.hint"
msgstr "Odun topla"
msgid "old.title"
msgstr "Eski"

3
examples/lang/i18n/package.ludic Normal file
View file

@ -0,0 +1,3 @@
# the key-based text example (i18n.ludic, i18n_ui.ludic): its languages are lang/<code>.po, and
# lang/en.po is the source every key is checked against
lang "lang" en

7
examples/lang/i18n/parts/Pause.ludic Normal file
View file

@ -0,0 +1,7 @@
# Pause.ludic - i18n_ui.ludic's component: its text is keys (Pause.xml), one of them a field the
# template reads - a Key reaches a template as its marked text, for t(expr)
component Pause {
prop day: int = 1
prop caught: int = 0
back: Key = k"pause.close"
}

8
examples/lang/i18n/parts/Pause.xml Normal file
View file

@ -0,0 +1,8 @@
<div class="pause">
<p>{t('pause.resume')}</p>
<button title="{t('pause.close')}">{t('hud.day', day)}</button>
<button>{t(back)}</button>
<p>{t('catch.count', caught, 'trout')}</p>
<p translate="no">Maroon Lake 0.5 <b>beta</b></p>
<p>Paused</p>
</div>

12
examples/rejected/i18n_keys/data_missing.ludic Normal file
View file

@ -0,0 +1,12 @@
# phase 26: a @Text Key field a row leaves out takes its derived key (things.axe.name), and en.po must
# have it; a @Text string field still holding English is a warning, counted as english_left
program DataMissing {
property Thing {
key: string = ""
@Text name: Key = null
@Text blurb: string = ""
}
registry Things of Thing as TH
def Things axe { blurb: "A sharp axe" }
entry { print(TH_COUNT) }
}

9
examples/rejected/i18n_keys/key_hole.ludic Normal file
View file

@ -0,0 +1,9 @@
# phase 26: a Key in a template literal's {hole} is refused - it would draw its marked name, never
# translated; text is made with tr / trf
program KeyHole {
entry {
let k = k"pause.resume"
print(`{k}`)
print(`at {k}!`)
}
}

12
examples/rejected/i18n_keys/key_missing.ludic Normal file
View file

@ -0,0 +1,12 @@
# phase 26: a key literal en.po does not have is refused, and so is a plural key (kn"...") whose
# en.po entry has no msgid_plural; trf given more values than the English has holes is a warning
program KeyMissing {
function tr(key: Key) -> string { return string(key) }
function trf(key: Key, a: string, b: string) -> string { return string(key) + a + b }
function trn(key: Key, n: int) -> string { return string(key) }
entry {
print(tr(k"pause.gone"))
print(trn(kn"pause.resume", 1))
print(trf(k"hud.day", "1", "2"))
}
}

11
examples/rejected/i18n_keys/key_mixup.ludic Normal file
View file

@ -0,0 +1,11 @@
# phase 26: a Key is not a string and a string is not a Key - text is made from a key, and a key is
# written k"..."; nor does a key join text with +
program KeyMixup {
function show(s: string) -> void { print(s) }
function name_of(k: Key) -> Key { return k }
entry {
show(k"pause.resume")
let k2 = name_of("Resume")
print(k2 + "!")
}
}

12
examples/rejected/i18n_keys/key_rules.ludic Normal file
View file

@ -0,0 +1,12 @@
# phase 26: trn wants a plural key kn"...", and tr / trf refuse one
program KeyRules {
function tr(key: Key) -> string { return string(key) }
function trf(key: Key, a: string) -> string { return string(key) + a }
function trn(key: Key, n: int) -> string { return string(key) }
entry {
print(trn(k"hud.day", 2))
print(trf(kn"catch.count", "1"))
print(tr(kn"catch.count"))
print(tr(k"pause.resume"))
}
}

24
examples/rejected/i18n_keys/lang/en.po Normal file
View file

@ -0,0 +1,24 @@
# The source language of the refused examples: two keys, and one English under two undescribed keys.
msgid ""
msgstr ""
"Content-Type: text/plain; charset=UTF-8\n"
#. On the pause menu: returns to play.
msgid "pause.resume"
msgstr "Resume"
#. The day on the HUD. {1} = the day number
msgid "hud.day"
msgstr "Day {1}"
msgid "shop.close"
msgstr "Close"
msgid "map.close"
msgstr "Close"
#. The catch on the HUD. {1} = how many
msgid "catch.count"
msgid_plural "catch.count"
msgstr[0] "{1} fish"
msgstr[1] "{1} fish"

2
examples/rejected/i18n_keys/package.ludic Normal file
View file

@ -0,0 +1,2 @@
# the key checks' refusals (key_missing, t_missing, key_mixup, data_missing, key_hole, key_rules): lang/en.po is the source
lang "lang" en

4
examples/rejected/i18n_keys/parts/Menu.ludic Normal file
View file

@ -0,0 +1,4 @@
# Menu.ludic - t_missing.ludic's component: one key en.po lacks
component Menu {
prop day: int = 1
}

5
examples/rejected/i18n_keys/parts/Menu.xml Normal file
View file

@ -0,0 +1,5 @@
<div class="menu">
<p>{t('pause.resume')}</p>
<p>{t('menu.gone')}</p>
<p>{t('hud.day')}</p>
</div>

7
examples/rejected/i18n_keys/t_missing.ludic Normal file
View file

@ -0,0 +1,7 @@
# phase 26: a template's t('key') is checked against en.po like a key literal - refused when en.po lacks
# it; t('hud.day') with no value for its {1} is a warning
import "ludic.ui"
program TMissing {
import "parts/Menu.ludic"
entry { print(1) }
}

46
runtime/native/lres.ludic
View file

@ -4,7 +4,7 @@
# A @PerMap registry's rows are not compiled in: they are read when a map loads, or a chunk comes
# in, from `<maps root>/<map>/<file>`. This file reads one .lres - the grammar the compiler's
# resource files use: entries `key { field: value, ... }`, nested records `{ ... }`, lists `[ ... ]`,
# "strings" (\n \r \t \0 and \x for any other x), numbers (-3, 0x1F, 1.5, 2e-3), true / false, a
# "strings" (\n \r \t \0 and \x for any other x), text keys k"a.b" and kn"a.b", numbers (-3, 0x1F, 1.5, 2e-3), true / false, a
# constant by NAME, `fn name`, and `#` comments to the end of a line; commas and newlines separate.
#
# ludicc splices it when a program declares a @PerMap registry, and writes the typed fill for each
@ -27,6 +27,7 @@ const LRES_K_STR: int = 5
const LRES_K_BOOL: int = 6
const LRES_K_NAME: int = 7
const LRES_K_FN: int = 8
const LRES_K_KEY: int = 9 # k"pause.resume" / kn"catch.count": a text key, kept after its marker byte
# one reader: the file, its tree, the program's constants by name, and the first error
property LresTree {
@ -431,6 +432,7 @@ function lres_value(t: LresTree) -> int {
if c == '[' { return lres_list(t) }
if c == '"' { return lres_string(t) }
if c == '-' or c == '+' or c == '.' or lres_is_digit(c) { return lres_number(t) }
if c == 'k' and lres_key_ahead(t) { return lres_keylit(t) }
if lres_is_alpha(c) {
let line = t.ln
let col = t.p - t.ls + 1
@ -545,13 +547,43 @@ function lres_number_end(t: LresTree, k: int) -> int {
if lres_is_alpha(c) or lres_is_digit(c) { lres_fail(t, "this is not a number") }
return k
}
function lres_string(t: LresTree) -> int {
# k"..." or kn"...", the cursor on the k
function lres_key_ahead(t: LresTree) -> bool {
let buf = t.buf
if t.p + 1 < t.n and buf[t.p + 1] == '"' { return true }
return t.p + 2 < t.n and buf[t.p + 1] == 'n' and buf[t.p + 2] == '"'
}
# the key's text after its marker byte (1, or 2 for a plural), as the compiler writes k"..."
function lres_keylit(t: LresTree) -> int {
let line = t.ln
let col = t.p - t.ls + 1
let buf = t.buf
var mark = 1
t.p += 1
if buf[t.p] == 'n' {
mark = 2
t.p += 1
}
let s = lres_string_marked(t, mark)
t.kind[s] = LRES_K_KEY
t.line[s] = line
t.col[s] = col
return s
}
function lres_string(t: LresTree) -> int { return lres_string_marked(t, 0) }
function lres_string_marked(t: LresTree, mark: int) -> int {
let s = lres_node(t, LRES_K_STR, t.ln, t.p - t.ls + 1)
let oline = t.ln
let ocol = t.p - t.ls + 1
t.p += 1
let buf = t.buf
var k = 0
if mark > 0 {
lres_sb_room(t, 2)
let sb0 = t.sb
sb0[0] = mark
k = 1
}
while true {
if t.p >= t.n {
lres_fail_at(t, oline, ocol, "this string is never closed with \"")
@ -601,6 +633,7 @@ function lres_kind_text(k: int) -> string {
if k == LRES_K_STR { return "a string" }
if k == LRES_K_BOOL { return "true or false" }
if k == LRES_K_NAME { return "a name" }
if k == LRES_K_KEY { return "a key" }
return "fn name"
}
function lres_want(t: LresTree, n: int, what: string) -> void {
@ -646,6 +679,15 @@ function lres_str(t: LresTree, n: int) -> string {
lres_want(t, n, "a \"string\"")
return ""
}
# a Key field's value: k"..." (its marker kept, as the compiler writes the literal)
function lres_key(t: LresTree, n: int) -> Key {
if t.kind[n] == LRES_K_KEY {
let p: pointer = t.sval[n]
return p
}
lres_want(t, n, "a key k\"...\"")
return null
}
# `fn name`: the name, which the fill resolves among the program's functions of the field's type
function lres_fn(t: LresTree, n: int) -> string {
if t.kind[n] == LRES_K_FN { return t.sval[n] }

5
selfhost/backend/emit_call.ludic
View file

@ -1092,7 +1092,10 @@ function emit_expr_raw(e: Node) -> Val {
# a copy, so it is fresh: `s[0 .. n - 4] + ".dds"` and `s[a .. b] == "x"` free it once read
return fresh_val(emit_bind(`call ptr @lp_str_slice(ptr {base.code}, i32 {lo.code}, i32 {hi.code})`), "string")
}
if e.kind == E_STR { return val(emit_str_const(e.s), "string") }
if e.kind == E_STR {
if e.ival == KEY_LIT or e.ival == KEY_PLURAL { return val(emit_str_const(e.s), "Key") } # its marker byte, then the key
return val(emit_str_const(e.s), "string")
}
if e.kind == E_NEW {
if is_slice_ty(e.s) { return emit_new_slice(e.s) }
return emit_new_struct(e.s, e.a, e.b)

1
selfhost/backend/emit_deps.ludic
View file

@ -121,6 +121,7 @@ function deps_flush() -> void {
}
deps_widths(f)
deps_reach(f)
deps_line(f, `english_left {itoa(g_i18n_english_left)}`) # phase 26: text left in English (i18n.ludic)
file_close(f)
}

2
selfhost/backend/emit_expr.ludic
View file

@ -174,7 +174,7 @@ function emit_bin(e: Node) -> Val {
# text types: `==` on two of these compares content, not the pointers. Besides
# `string`, that is the untyped raw pointer — `pointer`, and `ptr`, the element of
# a `pointers` buffer — which runtime code uses to carry text.
function is_textish(t: pointer) -> bool { return (t == "string") or (t == "pointer") or (t == "ptr") }
function is_textish(t: pointer) -> bool { return (t == "string") or (t == "pointer") or (t == "ptr") or ((t == "Key") and find_comp(t) == null) }
# lower `a <op> b` on two already-evaluated operands. Shared by binary expressions
# and compound assignment (`x += y` is exactly `x = x + y`), so both agree on

4
selfhost/backend/emit_schema.ludic
View file

@ -45,7 +45,9 @@ function schema_buf() -> Buf {
sc_natives(b)
buf_puts(b, "\n ],\n \"units\": [")
sc_units(b)
buf_puts(b, "]\n}\n")
buf_puts(b, "],\n \"lang\": ")
sc_lang(b) # emit_schema_lang.ludic: the keys and the languages
buf_puts(b, "\n}\n")
return b
}

224
selfhost/backend/emit_schema_lang.ludic Normal file
View file

@ -0,0 +1,224 @@
# emit_schema_lang.ludic — the schema's "lang" (emit_schema.ludic): what a language editor needs of
# the keys (i18n.ludic). null when the program's package.ludic has no `lang` line; otherwise
#
# dir, source, file the languages' directory, the source language and its .po (null if missing)
# keys every key used, sorted: its kind (code, template or data - the first site's, and
# `kinds` all of them), every site as "file:line:col", whether en.po has it, its
# English, whether it is a plural, and en.po's #. description (or null)
# unused en.po's keys used nowhere
# undescribed en.po's keys with no #. description
# split one English under several keys: [{english, keys}] - each wants a description
# languages every other .po in the directory: {code: {file, missing, fuzzy, extra}} - en.po's
# keys it lacks (or leaves empty), its fuzzy ones, and what it has that en.po does not
function sc_lang(b: Buf) -> void {
if not g_i18n_on {
buf_puts(b, "null")
return
}
let en = g_i18n_en
buf_puts(b, "{\n \"dir\": ")
jq_put(b, g_i18n_dir)
buf_puts(b, ",\n \"source\": ")
jq_put(b, g_i18n_src)
buf_puts(b, ",\n \"file\": ")
if en == null { buf_puts(b, "null") } else { jq_put(b, g_i18n_en_path) }
buf_puts(b, ",\n \"keys\": [")
let used = ss_new(1024)
sc_lang_keys(b, used)
buf_puts(b, "\n ],\n \"unused\": [")
if en != null { sc_lang_ids(b, en, used, 0) }
buf_puts(b, "],\n \"undescribed\": [")
if en != null { sc_lang_ids(b, en, used, 1) }
buf_puts(b, "],\n \"split\": [")
if en != null { sc_lang_split(b, en) }
buf_puts(b, "\n ],\n \"languages\": {")
sc_lang_others(b, en)
buf_puts(b, "\n }\n }")
}
function sc_lang_keys(b: Buf, used: StrSet) -> void {
let xs = new []int
let keys = new []pointer
var i = 0
while i < len(g_ks_key) {
push(xs, i)
push(keys, `{g_ks_key[i]}\t{g_ks_file[i]}\t{itoa(1000000 + g_ks_line[i])}\t{itoa(1000000 + g_ks_col[i])}`)
i += 1
}
sc_sort(xs, keys)
var a = 0
var first = true
while a < len(xs) {
let key = g_ks_key[xs[a]]
var z = a + 1
while z < len(xs) and (g_ks_key[xs[z]] == key) { z += 1 }
let dummy = ss_add(used, key)
sc_sep(b, first, " ")
first = false
buf_puts(b, "{\"key\": ")
jq_put(b, key)
buf_puts(b, ", \"kind\": ")
jq_put(b, i18n_ks_name(g_ks_kind[xs[a]]))
buf_puts(b, ", \"kinds\": [")
var kd = 0
var nk = 0
while kd < 3 {
var has = false
var q = a
while q < z {
if g_ks_kind[xs[q]] == kd { has = true }
q += 1
}
if has {
if nk > 0 { buf_puts(b, ", ") }
jq_put(b, i18n_ks_name(kd))
nk += 1
}
kd += 1
}
buf_puts(b, "], \"sites\": [")
var s = a
while s < z {
let j = xs[s]
if s > a { buf_puts(b, ", ") }
jq_put(b, `{g_ks_file[j]}:{itoa(g_ks_line[j])}:{itoa(g_ks_col[j])}`)
s += 1
}
buf_puts(b, "]")
let e = po_find(g_i18n_en, key)
buf_puts(b, ", \"in_source\": ")
sc_bool(b, e >= 0)
buf_puts(b, ", \"english\": ")
if e >= 0 { jq_put(b, g_i18n_en.first[e]) } else { buf_puts(b, "null") }
buf_puts(b, ", \"plural\": ")
sc_bool(b, e >= 0 and g_i18n_en.plural[e])
buf_puts(b, ", \"description\": ")
if e >= 0 { sc_str(b, g_i18n_en.desc[e]) } else { buf_puts(b, "null") }
buf_puts(b, "}")
a = z
}
}
# en.po's ids, sorted: those used nowhere (what 0), or those with no description (what 1)
function sc_lang_ids(b: Buf, en: PoFile, used: StrSet, what: int) -> void {
let xs = new []int
var i = 0
while i < len(en.ids) {
var take = false
if what == 0 { take = not ss_has(used, en.ids[i]) }
if what == 1 { take = en.desc[i] == null }
if take { push(xs, i) }
i += 1
}
sc_sort(xs, en.ids)
i = 0
while i < len(xs) {
if i > 0 { buf_puts(b, ", ") }
jq_put(b, en.ids[xs[i]])
i += 1
}
}
function sc_lang_split(b: Buf, en: PoFile) -> void {
let xs = new []int
let keys = new []pointer
var i = 0
while i < len(en.ids) {
push(xs, i)
push(keys, `{en.first[i]}\t{en.ids[i]}`)
i += 1
}
sc_sort(xs, keys)
var first = true
var a = 0
while a < len(xs) {
var z = a + 1
while z < len(xs) and (en.first[xs[z]] == en.first[xs[a]]) { z += 1 }
if z - a > 1 and len(en.first[xs[a]]) > 0 {
sc_sep(b, first, " ")
first = false
buf_puts(b, "{\"english\": ")
jq_put(b, en.first[xs[a]])
buf_puts(b, ", \"keys\": [")
var k = a
while k < z {
if k > a { buf_puts(b, ", ") }
jq_put(b, en.ids[xs[k]])
k += 1
}
buf_puts(b, "]}")
}
a = z
}
}
# every other language in the directory, against en.po
function sc_lang_others(b: Buf, en: PoFile) -> void {
let fs = pmk_list(g_i18n_dir, false)
var first = true
var i = 0
while i < len(fs) {
let f = fs[i]
let n = len(f)
if n > 3 and (f[n - 3 .. n] == ".po") and not (f[0 .. n - 3] == g_i18n_src) {
let p = po_read(`{g_i18n_dir}/{f}`)
if p != null {
sc_sep(b, first, " ")
first = false
jq_put(b, f[0 .. n - 3])
buf_puts(b, ": {\"file\": ")
jq_put(b, p.path)
buf_puts(b, ", \"missing\": [")
sc_lang_diff(b, en, p, 0)
buf_puts(b, "], \"fuzzy\": [")
sc_lang_diff(b, en, p, 1)
buf_puts(b, "], \"extra\": [")
sc_lang_diff(b, en, p, 2)
buf_puts(b, "]}")
}
}
i += 1
}
}
# 0: en.po's keys p lacks or leaves empty; 1: p's fuzzy keys; 2: p's keys en.po does not have
function sc_lang_diff(b: Buf, en: PoFile, p: PoFile, what: int) -> void {
let names = new []pointer
var i = 0
if what == 0 and en != null {
while i < len(en.ids) {
let j = po_find(p, en.ids[i])
if j < 0 or sc_lang_empty(p.strs[j]) { push(names, en.ids[i]) }
i += 1
}
}
if what == 1 {
while i < len(p.ids) {
if p.fuzzy[i] { push(names, p.ids[i]) }
i += 1
}
}
if what == 2 {
while i < len(p.ids) {
if po_find(en, p.ids[i]) < 0 { push(names, p.ids[i]) }
i += 1
}
}
let xs = new []int
i = 0
while i < len(names) {
push(xs, i)
i += 1
}
sc_sort(xs, names)
i = 0
while i < len(xs) {
if i > 0 { buf_puts(b, ", ") }
jq_put(b, names[xs[i]])
i += 1
}
}
# every form empty (the forms are joined by newlines)
function sc_lang_empty(s: pointer) -> bool {
var i = 0
while i < len(s) {
if s[i] != '\n' { return false }
i += 1
}
return true
}

10
selfhost/check/check_call.ludic
View file

@ -100,6 +100,7 @@ function ck_builtin(e: Node, name: pointer) -> pointer {
ck_walk_args(e)
return "string"
}
if (name == "string") and ck_tpl_hole(e) { return ck_hole(e) }
if (name == "string") or (name == "int") or (name == "float") or (name == "fixed") or (name == "long") or (name == "double") {
ck_walk_args(e)
return name
@ -313,3 +314,12 @@ function ck_emit(s: Node) -> void {
i += 1
}
}
# a template literal's {hole}: text, never a key - a Key there would draw its marked name untranslated
function ck_tpl_hole(e: Node) -> bool {
return e.a != null and (e.a.uns == TPL_HOLE or e.a.uns == TPL_COPY) and len(e.kids) == 1
}
function ck_hole(e: Node) -> pointer {
let t = ck_expr(e.kids[0])
if ck_is_key(t) { ck_err("key", e.kids[0], "a Key in a template's {hole}: a key names a text, it is not one - translate it with trf(k\"...\", ...) (tr(key) for no values), or say string(key) outside the template if the marked name is really wanted") }
return "string"
}

8
selfhost/check/check_env.ludic
View file

@ -74,5 +74,13 @@ function ck_give(to: pointer, from: pointer, e: Node, what: pointer) -> void {
ck_err(cat, e, `{what} wants {ck_a(to)} and this is a pointer: a slice is made with words(n), floats(n) or new, not from an address`)
return
}
if (cat == "key") and ck_is_key(to) {
ck_err(cat, e, `{what} wants a Key and this is {ck_a(from)}: a key is written k"module.purpose" (kn"..." for a plural)`)
return
}
if (cat == "key") {
ck_err(cat, e, `{what} wants {ck_a(to)} and this is a Key: text is made from a key (tr(key)), never the key itself`)
return
}
ck_err(cat, e, `{what} wants {ck_a(to)} and this is {ck_a(from)}`)
}

5
selfhost/check/check_expr.ludic
View file

@ -40,7 +40,10 @@ function ck_expr(e: Node) -> pointer {
if is_float_file(e.file) { return "float" }
return "fixed"
}
if k == E_STR { return "string" }
if k == E_STR {
if e.ival == KEY_LIT or e.ival == KEY_PLURAL { return "Key" } # k"..." / kn"..." (i18n.ludic)
return "string"
}
if k == E_SLICE { ck_walk(e); return "string" }
if k == E_BOOL { return "bool" }
if k == E_NULL { return "null" }

5
selfhost/check/check_ops.ludic
View file

@ -34,6 +34,7 @@ function ck_compare(e: Node, op: pointer, l: pointer, r: pointer, a: Node, b: No
let cr = ck_class(r)
# a pointer holding text is compared with a string all through the runtime and the compiler
if ((l == "pointer") and (cr == "string")) or ((r == "pointer") and (cl == "string")) { return }
if ((l == "pointer") and (cr == "key")) or ((r == "pointer") and (cl == "key")) { return }
if cl != cr {
ck_err("compare", e, `{op} between {ck_a(l)} and {ck_a(r)}`)
return
@ -46,6 +47,10 @@ function ck_compare(e: Node, op: pointer, l: pointer, r: pointer, a: Node, b: No
# `string` and the untyped `pointer` are both text to an operator (the emitter's is_textish)
function ck_is_text(t: pointer) -> bool { return (t == "string") or (t == "pointer") }
function ck_arith(e: Node, op: pointer, l: pointer, r: pointer, a: Node, b: Node) -> pointer {
if ck_is_key(l) or ck_is_key(r) {
ck_err("key", e, `{op} on a Key: a key names a text, it is not one - make the text with tr(key)`)
return "?"
}
if ck_is_text(l) or ck_is_text(r) {
if ck_unknown(l) or ck_unknown(r) { return "string" }
if ck_is_text(l) and ck_is_text(r) and (op == "+") { return "string" }

1
selfhost/check/check_tynames.ludic
View file

@ -60,6 +60,7 @@ function ck_ty_args(s: pointer, tps: pointer) -> pointer {
function ck_ty_known(name: pointer, tps: pointer) -> bool {
if ck_tab_get(ck_ty_k, ck_ty_v, name) != null { return true }
if (name == "Vector") or (name == "IVec2") or (name == "Rect") { return true } # the packed value types
if (name == "Key") { return true } # k"..." (i18n.ludic)
if is_state_ty(name) or ck_record(name) != null or ck_enum(name) != null { return true }
if gen_template(g_gen_recs, name) != null { return true }
if tps != null {

8
selfhost/check/check_types.ludic
View file

@ -5,6 +5,11 @@
# functions are kept apart, and a pointer is trusted to be whatever it is given as.
var ck_extern_arg: bool = false # an extern's arguments are being given: a slice goes as its data
function ck_unknown(t: pointer) -> bool { return t == null or (t == "?") }
# the builtin Key, the type of k"..." - unless the program declares a type of that name itself
function ck_is_key(t: pointer) -> bool {
if t == null or not (t == "Key") { return false }
return ck_record(t) == null and ck_enum(t) == null
}
function ck_is_int(t: pointer) -> bool {
if (t == "int") or (t == "long") or (t == "byte") or (t == "i64") or (t == "u8") { return true }
let en = ck_enum(t)
@ -38,6 +43,7 @@ function ck_is_lit(e: Node) -> bool {
function ck_class(t: pointer) -> pointer {
if ck_is_num(t) { return "number" }
if (t == "string") { return "string" }
if ck_is_key(t) { return "key" }
if (t == "bool") { return "bool" }
return "reference"
}
@ -85,6 +91,8 @@ function ck_mismatch(to: pointer, from: pointer, e: Node) -> pointer {
}
if (from == "pointer") and is_slice_ty(to) { return "pointer-slice" }
if (to == "pointer") or (from == "pointer") { return null }
# a Key (k"...") is not text and text is not a Key, either way round (i18n.ludic)
if ck_is_key(to) or ck_is_key(from) { return "key" }
if (to == "string") { return "kind" }
if ck_is_rec(to) and ck_is_rec(from) { return "record" }
if ck_is_rec(to) and ((from == "string") or is_slice_ty(from)) { return "kind" }

1
selfhost/frontend/attrs.ludic
View file

@ -176,6 +176,7 @@ function at_decl_clear() -> void {
g_at_decl_args = new []pointer
g_pm_pending = false # permap.ludic: @PerMap and @Chunked belong to one declaration
g_pm_chunk_pending = 0
g_tk_pending = null # i18n.ludic: @TextKey belongs to one declaration too
}
# the registry just declared takes the attributes read before it
function at_registry_take() -> int {

481
selfhost/frontend/i18n.ludic Normal file
View file

@ -0,0 +1,481 @@
# i18n.ludic — keys, not English (phase 26 of Maroon Lake's plan). A text the player reads is named
# by a key, and the key's English is en.po, a language like any other:
#
# lang "assets/lang" en # package.ludic: the languages' directory and the source language
# tr(k"pause.resume") # a Key, the builtin type of k"...": not a string, nor a string one
# trn(kn"catch.count", n, name) # a plural key: en.po's entry has a msgid_plural
# <button>{t('pause.resume')}</button> # a template's text (i18n_xml.ludic)
# @TextKey("steps") registry StepKinds of StepKind from "steps.lres"
# property StepKind { @Text hint: Key = null } # the row's key is steps.<row>.hint, filled in
#
# At run time a Key is its text after a marker byte - 1 for k"...", 2 for kn"..." - so the runtime's
# tr(k) is a cast and the translator knows a key from English. What the compiler checks, once the
# program is read (i18n_check, after the types): every key literal is in en.po (a kn"" one with a
# msgid_plural); trf / trn and a template's t() give as many values as the English has holes (a
# warning); a registry's @Text field left in the data as English, and a template's words outside
# translate="no", are "English left" (a warning each, counted as `english_left` by ludic deps); a
# @Text Key field with no value is filled with its derived key, which en.po must have; and the same
# English under several keys wants a `#.` description on each (a warning at en.po's line). With no
# `lang` line, or no en.po, nothing is checked, and a program's first key literal says so once.
const KEY_LIT: int = 1
const KEY_PLURAL: int = 2
const KS_CODE: int = 0
const KS_TEMPLATE: int = 1
const KS_DATA: int = 2
var g_i18n_on: bool = false # a `lang` line was read
var g_i18n_dir: pointer = null # its directory, as the program opens it
var g_i18n_src: pointer = "en"
var g_i18n_en: PoFile = null # <dir>/<src>.po, or null when it is not there
var g_i18n_en_path: pointer = ""
var g_i18n_nopo_warned: bool = false
var g_i18n_english_left: int = 0 # ludic deps' english_left
var g_i18n_done: bool = false
var g_rg_textkey: []pointer = new []pointer # per registry: its @TextKey prefix, or null
var g_tk_pending: pointer = null # @TextKey("x") read before the registry being parsed
# every key used, one row a site: the key, where, and what kind of source (KS_*)
var g_ks_key: []pointer = new []pointer
var g_ks_kind: []int = new []int
var g_ks_file: []pointer = new []pointer
var g_ks_line: []int = new []int
var g_ks_col: []int = new []int
var g_ks_seen: StrSet = null
var g_i18n_what: pointer = null # how a message names the key, when not as its literal
# ---- the literal ---------------------------------------------------------------------------------
# k"..." (ival 1) or kn"..." (ival 2): a string node holding the marker byte and the key
function key_lit_node(t: Tok) -> Node {
let n = node(E_STR)
n.line = t.line
n.col = t.col
n.s = key_marked(t.ival, t.text)
n.ival = t.ival
return n
}
function key_marked(kind: int, key: pointer) -> pointer {
let k = len(key)
let b = bytes(k + 2)
b[0] = kind
var i = 0
while i < k {
b[i + 1] = key[i]
i += 1
}
b[k + 1] = 0
return b
}
function i18n_is_key(e: Node) -> bool { return e != null and e.kind == E_STR and (e.ival == KEY_LIT or e.ival == KEY_PLURAL) }
function i18n_key_text(e: Node) -> pointer { return e.s[1 .. len(e.s)] }
# ---- a set of strings, grown as it fills ---------------------------------------------------------
property StrSet {
keys: []pointer = null
n: int = 0
}
function ss_new(cap: int) -> StrSet {
let s = new StrSet
s.keys = new []pointer
var i = 0
while i < cap {
push(s.keys, null)
i += 1
}
return s
}
function ss_slot(s: StrSet, k: pointer) -> int {
let cap = len(s.keys)
var h = po_hash(k) & (cap - 1)
while s.keys[h] != null and not (s.keys[h] == k) { h = (h + 1) & (cap - 1) }
return h
}
function ss_has(s: StrSet, k: pointer) -> bool { return s.keys[ss_slot(s, k)] != null }
# true when k was not in the set
function ss_add(s: StrSet, k: pointer) -> bool {
if (s.n + 1) * 2 > len(s.keys) {
let old = s.keys
let bigger = ss_new(len(old) * 2)
var i = 0
while i < len(old) {
if old[i] != null { bigger.keys[ss_slot(bigger, old[i])] = old[i] }
i += 1
}
s.keys = bigger.keys
}
let h = ss_slot(s, k)
if s.keys[h] != null { return false }
s.keys[h] = k
s.n += 1
return true
}
# ---- the manifest --------------------------------------------------------------------------------
# package.ludic's `lang "assets/lang" en`, looked for as `maps` is: the nearest package.ludic up to four
# directories above the program, the directory taken relative to it
function i18n_manifest(entry: pointer) -> void {
var dir = dir_of(entry)
var up = 0
while up < 5 {
var root = dir
if len(root) > 0 and not is_sep(root[len(root) - 1]) { root = root + "/" }
let txt = read_file(root + "package.ludic")
if txt != null {
i18n_manifest_read(txt, root)
return
}
if len(dir) == 0 { dir = ".." } else { dir = dir + "/.." }
up += 1
}
}
function i18n_manifest_read(txt: pointer, root: pointer) -> void {
var i = 0
let tn = len(txt)
while i < tn {
var e = i
while (e < tn) and (txt[e] != '\n') { e += 1 }
var le = e
if (le > i) and (txt[le - 1] == 13) { le -= 1 }
let ws = native_words(txt[i..le])
if (len(ws) == 2 or len(ws) == 3) and (ws[0] == "lang") {
g_i18n_on = true
g_i18n_dir = pm_norm(join_path(root, pm_trim_slash(ws[1])))
if len(ws) == 3 { g_i18n_src = ws[2] }
}
i = e + 1
}
if g_i18n_on {
g_i18n_en_path = `{g_i18n_dir}/{g_i18n_src}.po`
g_i18n_en = po_read(g_i18n_en_path)
}
}
# ---- @TextKey and the derived keys ---------------------------------------------------------------
# @TextKey("steps") before a registry, its '@' and name read
function i18n_parse_textkey() -> void {
eat_op("(")
if toks[pi].kind != TK_STR or len(toks[pi].text) == 0 { perr("@TextKey takes the prefix of the registry's text keys: @TextKey(\"steps\")") }
g_tk_pending = toks[pi].text
pi += 1
eat_op(")")
let b = buf_new()
jq_put(b, g_tk_pending)
at_decl_with("TextKey", `[{buf_str(b)}]`)
}
# the registry just declared takes it
function i18n_registry_declared() -> void {
push(g_rg_textkey, g_tk_pending)
g_tk_pending = null
}
# where a registry's derived keys start: its @TextKey, else its name in snake case (Items -> items,
# GearKinds -> gear_kinds). One rule, asked by the fill, the checks and the schema alike
function i18n_prefix(r: int) -> pointer {
if r >= 0 and r < len(g_rg_textkey) and g_rg_textkey[r] != null { return g_rg_textkey[r] }
return pm_snake(vis_plain(g_rg_name[r]))
}
function i18n_has_text(f: Node) -> bool {
if f == null or f.at < 0 { return false }
var j = g_at_start[f.at]
while j < g_at_start[f.at] + g_at_count[f.at] {
if (g_at_name[j] == "Text") { return true }
j += 1
}
return false
}
function i18n_is_key_ty(t: pointer) -> bool { return t != null and (t == "Key") and find_comp(t) == null }
# a compiled registry's row (reg_fill): a @Text Key field it gives no value takes its derived key
function i18n_fill_row(r: int, key: pointer, rec: Node, comp: Node, kline: int, kcol: int) -> void {
if comp == null { return }
var i = 0
while i < len(comp.kids) {
let f = comp.kids[i]
if f.kind == N_FIELD and i18n_has_text(f) and i18n_is_key_ty(f.ty) and not reg_rec_has(rec, f.s) {
let fi = node(E_FINIT)
fi.s = f.s
fi.file = rec.file
fi.line = kline
fi.col = kcol
let ks = node(E_STR)
ks.s = key_marked(KEY_LIT, `{i18n_prefix(r)}.{key}.{f.s}`)
ks.ival = KEY_LIT
ks.ty = "derived" # the checks know it was not written
ks.file = rec.file
ks.line = kline
ks.col = kcol
fi.a = ks
push(rec.kids, fi)
}
i += 1
}
}
# ---- the sites and the key checks ----------------------------------------------------------------
function i18n_site(key: pointer, kind: int, file: pointer, line: int, col: int) -> bool {
if g_ks_seen == null { g_ks_seen = ss_new(1024) }
var f = file
if f == null { f = "" }
if not ss_add(g_ks_seen, `{f}:{itoa(line)}:{itoa(col)}:{key}`) { return false }
push(g_ks_key, key)
push(g_ks_kind, kind)
push(g_ks_file, f)
push(g_ks_line, line)
push(g_ks_col, col)
return true
}
function i18n_ks_name(kind: int) -> pointer {
if kind == KS_TEMPLATE { return "template" }
if kind == KS_DATA { return "data" }
return "code"
}
# a key used: in en.po, and a plural one as a plural there. A lang line with no en.po: said once;
# no lang line at all (a package test) checks nothing and says nothing
function i18n_use(key: pointer, plural: bool, kind: int, file: pointer, line: int, col: int) -> void {
if not i18n_site(key, kind, file, line, col) { return }
if g_i18n_en == null {
if g_i18n_on and kind != KS_TEMPLATE and not g_i18n_nopo_warned {
g_i18n_nopo_warned = true
pwarn_at(file, line, col, `k"{key}": no en.po to check against - {g_i18n_en_path} is not there; no key is checked`)
}
return
}
let i = po_find(g_i18n_en, key)
var lit = `k"{key}"`
if plural { lit = `kn"{key}"` }
if kind == KS_TEMPLATE { lit = `t('{key}')` }
if g_i18n_what != null { lit = g_i18n_what }
if i < 0 {
perr_at(file, line, col, `{lit}: the key {key} is not in {g_i18n_en_path}`)
return
}
if plural and not g_i18n_en.plural[i] {
perr_at(file, line, col, `{lit} is a plural key, and {g_i18n_en_path}:{itoa(g_i18n_en.line[i])} gives {key} no msgid_plural`)
}
}
# trf / trn / t(): as many values as en.po's English has holes ({1} is trn's count)
function i18n_holes(key: pointer, given: int, what: pointer, file: pointer, line: int, col: int) -> void {
let i = po_find(g_i18n_en, key)
if i < 0 { return }
let want = po_max_hole(g_i18n_en.strs[i])
if want == given { return }
pwarn_at(file, line, col, `{what}: {key} is given {itoa(given)} value(s), and its English in {g_i18n_en_path} has {itoa(want)} hole(s)`)
}
# words a player would read: two letters together, or any byte of a non-ASCII character
function i18n_wordy(s: pointer) -> bool {
if s == null { return false }
var run = 0
var i = 0
while i < len(s) {
let c = s[i]
if c < 0 or c >= 128 { return true }
if (c >= 'a' and c <= 'z') or (c >= 'A' and c <= 'Z') { run += 1 } else { run = 0 }
if run >= 2 { return true }
i += 1
}
return false
}
function i18n_short(s: pointer) -> pointer {
var t = s
if len(t) > 40 { t = t[0 .. 37] + "..." }
let b = buf_new()
var i = 0
while i < len(t) {
if t[i] == '\n' { buf_puts(b, " ") } else { buf_putc(b, t[i]) }
i += 1
}
return buf_str(b)
}
function i18n_english_left(file: pointer, line: int, col: int, msg: pointer) -> void {
g_i18n_english_left += 1
pwarn_at(file, line, col, `English left: {msg}`)
}
# ---- the whole check -----------------------------------------------------------------------------
function i18n_check() -> void {
if g_i18n_done { return }
g_i18n_done = true
if g_i18n_on { i18n_walk_data() } # first: a def's key literal is a data site
var i = 0
while i < len(prog) {
if not is_runtime_file(prog[i].file) { i18n_walk(prog[i]) }
i += 1
}
if not g_i18n_on { return }
i18n_walk_templates() # i18n_xml.ludic
i18n_split_check()
}
function i18n_is_data_file(f: pointer) -> bool {
if f == null { return false }
let n = len(f)
return n > 5 and (f[n - 5 .. n] == ".lres")
}
function i18n_walk(n: Node) -> void {
if n == null { return }
if i18n_is_key(n) and not i18n_is_data_file(n.file) {
i18n_use(i18n_key_text(n), n.ival == KEY_PLURAL, KS_CODE, n.file, n.line, i18n_col(n))
}
if n.kind == E_CALL and n.a != null and n.a.kind == E_ID { i18n_call(n) }
i18n_walk(n.a)
i18n_walk(n.b)
i18n_walk(n.c)
var i = 0
while i < len(n.kids) {
i18n_walk(n.kids[i])
i += 1
}
}
# tr(k"...") / trf(k"...", a, b) / trn(kn"...", n, a): a plural key only with trn, and trn only with
# one (en.po or not); then the values after the key against its holes
function i18n_call(n: Node) -> void {
let nm = vis_plain(n.a.s)
if not (nm == "tr") and not (nm == "trf") and not (nm == "trn") { return }
var k = 0
while k < len(n.kids) and not i18n_is_key(n.kids[k]) { k += 1 }
if k >= len(n.kids) { return }
let e = n.kids[k]
let key = i18n_key_text(e)
let plural = e.ival == KEY_PLURAL
if nm == "trn" and not plural {
perr_at(e.file, e.line, i18n_col(e), `trn(k"{key}", ...): trn counts, so its key is a plural one - write kn"{key}" (and give en.po a msgid_plural)`)
}
if plural and not (nm == "trn") {
perr_at(e.file, e.line, i18n_col(e), `{nm}(kn"{key}", ...): a plural key needs its count - write trn(kn"{key}", n, ...)`)
}
if nm == "tr" or g_i18n_en == null { return }
i18n_holes(key, len(n.kids) - k - 1, nm, e.file, e.line, i18n_col(e))
}
# a node's column in its file: a template literal's hole is lexed on its own, so its tokens' columns
# are the hole's - the file offset says where it really is
function i18n_col(n: Node) -> int {
if n.pos < 0 or n.file == null { return n.col }
let src = fm_get(g_fm_src, n.file)
if src == null or len(src) <= n.pos { return n.col }
var a = n.pos
while a > 0 and src[a - 1] != '\n' { a -= 1 }
return n.pos - a + 1
}
# ---- the data: every compiled registry's rows ----------------------------------------------------
function i18n_walk_data() -> void {
var d = 0
while d < len(g_df_reg) {
let r = reg_find(g_df_reg[d])
if r >= 0 and not pm_is(r) {
let comp = find_comp(g_rg_type[r])
if comp != null { i18n_data_rec(g_df_rec[d], comp, `{i18n_prefix(r)}.{g_df_key[d]}`) }
}
d += 1
}
}
# a @PerMap registry's rows in one map, once --check has read them (permap_check.ludic)
function i18n_permap_rows(md: pointer) -> void {
if not g_i18n_on { return }
var m = md
var cut = len(m)
while cut > 0 and is_sep(m[cut - 1]) { cut -= 1 }
m = m[0 .. cut]
var s = cut
while s > 0 and not is_sep(m[s - 1]) { s -= 1 }
let map = m[s .. cut]
var e = 0
while e < len(g_pmk_rec) {
let r = g_pmk_reg[e]
let comp = find_comp(g_rg_type[r])
if comp != null { i18n_data_rec(g_pmk_rec[e], comp, `maps.{map}.{i18n_prefix(r)}.{g_pmk_key[e]}`) }
e += 1
}
}
function i18n_field_init(rec: Node, name: pointer) -> Node {
var i = 0
while i < len(rec.kids) {
let fi = rec.kids[i]
if fi.kind == E_FINIT and (fi.s == name) { return fi }
i += 1
}
return null
}
function i18n_data_rec(rec: Node, comp: Node, base: pointer) -> void {
if rec == null or comp == null { return }
var i = 0
while i < len(comp.kids) {
let f = comp.kids[i]
if f.kind == N_FIELD {
let fi = i18n_field_init(rec, f.s)
let key = `{base}.{f.s}`
let text = i18n_has_text(f)
if fi != null { i18n_data_val(fi.a, f.ty, key, text) }
else if text and i18n_is_key_ty(f.ty) {
# a map's row: its key is derived and must be in en.po (the fill is compiled registries')
g_i18n_what = `{key} (the derived key of a @Text field the row gives no value)`
i18n_use(key, false, KS_DATA, rec.file, rec.line, rec.col)
g_i18n_what = null
}
}
i += 1
}
}
function i18n_data_val(e: Node, ty: pointer, key: pointer, text: bool) -> void {
if e == null { return }
if i18n_is_key(e) {
if e.ty != null and (e.ty == "derived") { g_i18n_what = `{key} (the derived key of a @Text field the row gives no value)` }
i18n_use(i18n_key_text(e), e.ival == KEY_PLURAL, KS_DATA, e.file, e.line, e.col)
g_i18n_what = null
return
}
if e.kind == E_STR {
if text and i18n_wordy(e.s) {
let dummy = i18n_site(key, KS_DATA, e.file, e.line, e.col)
i18n_english_left(e.file, e.line, e.col, `{key} is "{i18n_short(e.s)}" in the data - its English goes to en.po under {key}`)
}
return
}
if e.kind == E_LIST {
var el: pointer = null
if ty != null and is_slice_ty(ty) { el = slice_elem(ty) }
var i = 0
while i < len(e.kids) {
i18n_data_val(e.kids[i], el, `{key}.{itoa(i)}`, text)
i += 1
}
return
}
if e.kind == E_NEW and e.a != null and e.a.kind == E_REC {
i18n_data_rec(e.a, find_comp(e.s), key)
return
}
if e.kind == E_REC and ty != null { i18n_data_rec(e, find_comp(ty), key) }
}
# ---- one English, several keys -------------------------------------------------------------------
# the English a key was split into wants a `#.` on each key, so a translator can choose other words
function i18n_split_check() -> void {
let p = g_i18n_en
if p == null { return }
let xs = new []int
let keys = new []pointer
var i = 0
while i < len(p.ids) {
push(xs, i)
push(keys, p.first[i])
i += 1
}
sc_sort(xs, keys)
var a = 0
while a < len(xs) {
var b = a + 1
while b < len(xs) and (p.first[xs[b]] == p.first[xs[a]]) { b += 1 }
if b - a > 1 and len(p.first[xs[a]]) > 0 {
var names: pointer = ""
var k = a
while k < b {
if k > a { names = names + ", " }
names = names + p.ids[xs[k]]
k += 1
}
k = a
while k < b {
let j = xs[k]
if p.desc[j] == null { pwarn_at(p.path, p.line[j], 1, `{p.ids[j]} has no #. description, and its English "{i18n_short(p.first[j])}" is split into {itoa(b - a)} keys ({names}) - say where it appears, so a translator can tell them apart`) }
k += 1
}
}
a = b
}
}

252
selfhost/frontend/i18n_po.ludic Normal file
View file

@ -0,0 +1,252 @@
# i18n_po.ludic — a gettext .po file, read for the key checks (i18n.ludic). What is kept of each
# entry: its msgid (a key, since phase 26), whether it has a msgid_plural, its msgstr (or every
# msgstr[n], joined by a newline, so a hole in any form is seen), the first form alone (the English a
# split key is found by), the `#, fuzzy` flag, the `#.` extracted comments (the translator's
# description, lines joined by a newline) and the line its msgid is written on. msgctxt is read and
# kept but keys nothing; `#~` obsolete entries, `#:` references, `#|` previous ids and translator
# comments are skipped; a string may run over several "..." lines. The header (msgid "") is left out.
property PoFile {
path: pointer = null
ids: []pointer = null
plural: []bool = null
strs: []pointer = null # every form, joined
first: []pointer = null # msgstr, or msgstr[0]
fuzzy: []bool = null
desc: []pointer = null # the #. lines, or null
ctxt: []pointer = null
line: []int = null
slots: []int = null # the index by msgid: open addressing, -1 empty
}
# the entry being read
var g_po_cid: pointer = null
var g_po_cpl: bool = false
var g_po_cstr: []pointer = new []pointer
var g_po_cfuzzy: bool = false
var g_po_cdesc: pointer = null
var g_po_cctxt: pointer = null
var g_po_cline: int = 0
var g_po_field: int = 0 # what a continuation line adds to: 1 msgid, 2 plural, 3 msgstr (form g_po_form), 4 msgctxt
var g_po_form: int = 0
var g_po_seen_str: bool = false
function po_new(path: pointer) -> PoFile {
let p = new PoFile
p.path = path
p.ids = new []pointer
p.plural = new []bool
p.strs = new []pointer
p.first = new []pointer
p.fuzzy = new []bool
p.desc = new []pointer
p.ctxt = new []pointer
p.line = new []int
p.slots = new []int
return p
}
function po_reset_cur() -> void {
g_po_cid = null
g_po_cpl = false
g_po_cstr = new []pointer
g_po_cfuzzy = false
g_po_cdesc = null
g_po_cctxt = null
g_po_cline = 0
g_po_field = 0
g_po_form = 0
g_po_seen_str = false
}
# the entry read so far goes into the file (the header and an id-less fragment do not)
function po_flush(p: PoFile) -> void {
if g_po_cid != null and len(g_po_cid) > 0 {
var all: pointer = ""
var first: pointer = ""
var i = 0
while i < len(g_po_cstr) {
var f = g_po_cstr[i]
if f == null { f = "" }
if i == 0 { first = f }
if i > 0 { all = all + "\n" }
all = all + f
i += 1
}
push(p.ids, g_po_cid)
push(p.plural, g_po_cpl)
push(p.strs, all)
push(p.first, first)
push(p.fuzzy, g_po_cfuzzy)
push(p.desc, g_po_cdesc)
push(p.ctxt, g_po_cctxt)
push(p.line, g_po_cline)
}
po_reset_cur()
}
# the text of the "..." starting at or after s[i], its escapes read
function po_quoted(s: pointer, i0: int) -> pointer {
var i = i0
let n = len(s)
while i < n and s[i] != '"' { i += 1 }
if i >= n { return "" }
i += 1
let b = buf_new()
while i < n and s[i] != '"' {
var c = s[i]
if c == CH_BACKSLASH and i + 1 < n {
i += 1
c = unescape(s[i])
}
buf_putc(b, c)
i += 1
}
return buf_str(b)
}
function po_add_to(field: int, text: pointer) -> void {
if field == 1 { g_po_cid = g_po_cid + text }
else if field == 4 { g_po_cctxt = g_po_cctxt + text }
else if field == 3 {
while len(g_po_cstr) <= g_po_form { push(g_po_cstr, "") }
g_po_cstr[g_po_form] = g_po_cstr[g_po_form] + text
}
}
# a keyword line starts a new entry when the last one already had its msgstr
function po_maybe_new(p: PoFile) -> void {
if g_po_seen_str { po_flush(p) }
}
function po_read(path: pointer) -> PoFile {
let src = read_file(path)
if src == null { return null }
let p = po_new(path)
po_reset_cur()
var i = 0
var ln = 1
let n = len(src)
while i < n {
var e = i
while e < n and src[e] != '\n' { e += 1 }
var le = e
if le > i and src[le - 1] == 13 { le -= 1 }
var a = i
while a < le and (src[a] == ' ' or src[a] == 9) { a += 1 }
let line = src[a .. le]
po_line(p, line, ln)
i = e + 1
ln += 1
}
po_flush(p)
po_index(p)
return p
}
function po_line(p: PoFile, line: pointer, ln: int) -> void {
let n = len(line)
if n == 0 {
po_flush(p)
return
}
if line[0] == '#' {
if n > 1 and line[1] == '~' { return } # obsolete
po_maybe_new(p)
if n > 1 and line[1] == '.' {
var t = 2
if t < n and line[t] == ' ' { t += 1 }
let d = line[t .. n]
if g_po_cdesc == null { g_po_cdesc = d } else { g_po_cdesc = g_po_cdesc + "\n" + d }
}
if n > 1 and line[1] == ',' and has_sub(line, "fuzzy") { g_po_cfuzzy = true }
return
}
if line[0] == '"' {
po_add_to(g_po_field, po_quoted(line, 0))
return
}
if str_starts(line, "msgctxt") {
po_maybe_new(p)
g_po_cctxt = po_quoted(line, 7)
g_po_field = 4
return
}
if str_starts(line, "msgid_plural") {
g_po_cpl = true
g_po_field = 2
return
}
if str_starts(line, "msgid") {
po_maybe_new(p)
g_po_cid = po_quoted(line, 5)
g_po_cline = ln
g_po_field = 1
return
}
if str_starts(line, "msgstr") {
g_po_seen_str = true
g_po_form = 0
if n > 6 and line[6] == '[' {
var k = 7
var v = 0
while k < n and char_is_digit(line[k]) { v = v * 10 + (line[k] - 48); k += 1 }
g_po_form = v
}
g_po_field = 3
po_add_to(3, po_quoted(line, 6))
}
}
# ---- the index by msgid --------------------------------------------------------------------------
function po_hash(s: pointer) -> int {
var h = 5381
var i = 0
let n = len(s)
while i < n {
h = (h * 33 + s[i]) & 16777215
i += 1
}
return h
}
function po_index(p: PoFile) -> void {
var cap = 64
while cap < len(p.ids) * 2 + 1 { cap = cap * 2 }
p.slots = new []int
var i = 0
while i < cap {
push(p.slots, -1)
i += 1
}
i = 0
while i < len(p.ids) {
var h = po_hash(p.ids[i]) & (cap - 1)
var dup = false
while p.slots[h] >= 0 and not dup {
if (p.ids[p.slots[h]] == p.ids[i]) { dup = true } else { h = (h + 1) & (cap - 1) }
}
if not dup { p.slots[h] = i } # the first entry of an id is the one read
i += 1
}
}
# the entry for key, or -1
function po_find(p: PoFile, key: pointer) -> int {
if p == null or key == null { return -1 }
let cap = len(p.slots)
if cap == 0 { return -1 }
var h = po_hash(key) & (cap - 1)
while p.slots[h] >= 0 {
if (p.ids[p.slots[h]] == key) { return p.slots[h] }
h = (h + 1) & (cap - 1)
}
return -1
}
# the highest hole {n} in a text, 0 when it has none ({{ is a brace, not a hole)
function po_max_hole(s: pointer) -> int {
var best = 0
var i = 0
let n = len(s)
while i < n {
if s[i] == '{' and i + 1 < n and s[i + 1] == '{' { i += 2; continue }
if s[i] == '{' {
var k = i + 1
var v = 0
while k < n and char_is_digit(s[k]) { v = v * 10 + (s[k] - 48); k += 1 }
if k > i + 1 and k < n and s[k] == '}' and v > best { best = v }
}
i += 1
}
return best
}

326
selfhost/frontend/i18n_xml.ludic Normal file
View file

@ -0,0 +1,326 @@
# i18n_xml.ludic — a component's template (NAME.xml) against en.po (i18n.ludic). The compiler reads
# the template's text, so it checks it: in every {hole} - an element's text or an attribute's value -
# `t('key', ...)` with a literal key names a key en.po has, and gives as many values after it as the
# English has holes; `t(expr)` with a key worked out at run time is left alone. What a player would
# read that is not a key is "English left" (a warning, counted): an element's words outside its
# holes, a text attribute's words (text, label, hint, title, caption, and any other attribute whose
# value is not a keyword - as tools/i18n/extract.py decided), and a quoted choice inside a hole that
# reads as words (a capital or a space) and is not a key. An element with translate="no" and all it
# holds are left alone. Comments, <? ?> and <! > are skipped; `&name;` is not a word.
var g_xk_src: pointer = ""
var g_xk_file: pointer = ""
var g_xk_i: int = 0
var g_xk_n: int = 0
function i18n_walk_templates() -> void {
var c = 0
while c < len(g_cm_name) {
let xml = g_cm_xml[c]
if xml != null and len(xml) > 0 and not is_runtime_file(g_cm_file[c]) { i18n_xml(g_cm_xml_path[c], xml) }
c += 1
}
}
# the cursor on to `to`
function xk_move(to: int) -> void {
g_xk_i = to
if g_xk_i > g_xk_n { g_xk_i = g_xk_n }
}
var g_xk_at_line: int = 0
var g_xk_at_col: int = 0
function xk_find(s: pointer, from: int, pat: pointer) -> int {
let n = len(s)
let m = len(pat)
var i = from
while i + m <= n {
if s[i .. i + m] == pat { return i }
i += 1
}
return n
}
function i18n_xml(path: pointer, src: pointer) -> void {
g_xk_src = src
g_xk_file = path
g_xk_i = 0
g_xk_n = len(src)
let names = new []pointer
var no_depth = -1 # the depth an untranslated element opened at
while g_xk_i < g_xk_n {
let c = src[g_xk_i]
if c == '<' {
if g_xk_i + 3 < g_xk_n and (src[g_xk_i + 1 .. g_xk_i + 4] == "!--") {
xk_move(xk_find(src, g_xk_i + 4, "-->") + 3)
continue
}
if g_xk_i + 1 < g_xk_n and (src[g_xk_i + 1] == '?' or src[g_xk_i + 1] == '!') {
xk_move(xk_find(src, g_xk_i, ">") + 1)
continue
}
if g_xk_i + 1 < g_xk_n and src[g_xk_i + 1] == '/' {
xk_move(xk_find(src, g_xk_i, ">") + 1)
if len(names) > 0 { let gone = List.pop(names) }
if no_depth >= 0 and len(names) <= no_depth { no_depth = -1 }
continue
}
let depth = len(names)
let r = xk_tag(no_depth >= 0)
if r > 0 and no_depth < 0 and r == 2 { no_depth = depth }
if g_xk_last_open { push(names, g_xk_last_name) }
else if no_depth == depth { no_depth = -1 } # a self-closed translate="no" holds nothing
continue
}
xk_text(no_depth >= 0)
}
}
# text up to the next tag: its holes checked, its words outside them English left
function xk_text(quiet: bool) -> void {
let src = g_xk_src
var words_at = -1
var run = 0
while g_xk_i < g_xk_n and src[g_xk_i] != '<' {
let c = src[g_xk_i]
if c == '{' {
let e = hole_end(src, g_xk_i + 1, g_xk_n)
xk_hole(g_xk_i + 1, e, quiet)
xk_move(e + 1)
run = 0
continue
}
if c == '&' {
var j = g_xk_i + 1
while j < g_xk_n and j < g_xk_i + 12 and src[j] != ';' and src[j] != '<' and src[j] != ' ' { j += 1 }
if j < g_xk_n and src[j] == ';' {
xk_move(j + 1)
run = 0
continue
}
}
if words_at < 0 {
if c < 0 or c >= 128 { words_at = g_xk_i }
if (c >= 'a' and c <= 'z') or (c >= 'A' and c <= 'Z') { run += 1 } else { run = 0 }
if run >= 2 { words_at = g_xk_i - 1 }
}
xk_move(g_xk_i + 1)
}
if words_at >= 0 and not quiet {
let a = xk_trim_start(words_at)
xk_where_back(a)
i18n_english_left(g_xk_file, g_xk_at_line, g_xk_at_col, `a template's words ("{i18n_short(xk_words_from(a))}") - write {{t('key')}} and give en.po the English, or mark the element translate="no"`)
}
}
# where a word starts: back from a letter to the space before it
function xk_trim_start(at: int) -> int {
var a = at
while a > 0 and g_xk_src[a - 1] != '>' and g_xk_src[a - 1] != '}' and g_xk_src[a - 1] != ' ' and g_xk_src[a - 1] != '\n' and g_xk_src[a - 1] != 9 { a -= 1 }
return a
}
function xk_words_from(a: int) -> pointer {
var e = a
while e < g_xk_n and g_xk_src[e] != '<' and g_xk_src[e] != '{' { e += 1 }
return g_xk_src[a .. e]
}
# the line and column of a byte already passed
function xk_where_back(at: int) -> void {
var ln = 1
var ls = 0
var i = 0
while i < at {
if g_xk_src[i] == '\n' {
ln += 1
ls = i + 1
}
i += 1
}
g_xk_at_line = ln
g_xk_at_col = at - ls + 1
}
# ---- a tag ---------------------------------------------------------------------------------------
var g_xk_last_open: bool = false # the tag read opens an element (it is not <x ... />)
var g_xk_last_name: pointer = ""
function xk_name_ch(c: int) -> bool { return char_is_alnum(c) or c == '_' or c == '-' or c == ':' or c == '.' or c == '@' }
function xk_space(c: int) -> bool { return c == ' ' or c == '\n' or c == 9 or c == 13 }
# `<name attr="v" ...>` from the cursor: 2 when it says translate="no", else 1
function xk_tag(quiet: bool) -> int {
let src = g_xk_src
var i = g_xk_i + 1
let a0 = i
while i < g_xk_n and xk_name_ch(src[i]) { i += 1 }
g_xk_last_name = src[a0 .. i]
g_xk_last_open = true
let an = new []pointer
let va = new []int
let vb = new []int
var no = false
while i < g_xk_n {
while i < g_xk_n and xk_space(src[i]) { i += 1 }
if i >= g_xk_n { break }
if src[i] == '>' {
i += 1
break
}
if src[i] == '/' {
g_xk_last_open = false
i += 1
continue
}
let na = i
while i < g_xk_n and not xk_space(src[i]) and src[i] != '=' and src[i] != '>' and src[i] != '/' { i += 1 }
if i == na { i += 1; continue }
let name = src[na .. i]
while i < g_xk_n and xk_space(src[i]) { i += 1 }
var a = i
var b = i
if i < g_xk_n and src[i] == '=' {
i += 1
while i < g_xk_n and xk_space(src[i]) { i += 1 }
if i < g_xk_n and (src[i] == '"' or src[i] == CH_SQUOTE) {
let q = src[i]
i += 1
a = i
while i < g_xk_n and src[i] != q {
if src[i] == '{' { i = hole_end(src, i + 1, g_xk_n) }
i += 1
}
b = i
if i < g_xk_n { i += 1 }
} else {
a = i
while i < g_xk_n and not xk_space(src[i]) and src[i] != '>' { i += 1 }
b = i
}
}
push(an, name)
push(va, a)
push(vb, b)
if (name == "translate") and (src[a .. b] == "no") { no = true }
}
var k = 0
while k < len(an) {
xk_attr(an[k], va[k], vb[k], quiet or no)
k += 1
}
g_xk_i = i
if no { return 2 }
return 1
}
function xk_not_text(n: pointer) -> bool {
if (n == "class") or (n == "id") or (n == "src") or (n == "style") or (n == "icon") or (n == "name") or (n == "type") { return true }
if (n == "key") or (n == "as") or (n == "in") or (n == "test") or (n == "index") or (n == "slot") or (n == "export") { return true }
if (n == "color") or (n == "bg") or (n == "background") or (n == "state") or (n == "min") or (n == "max") or (n == "width") { return true }
if (n == "height") or (n == "w") or (n == "h") or (n == "shade") or (n == "rows") or (n == "maxlength") or (n == "checked") { return true }
if (n == "enabled") or (n == "disabled") or (n == "hidden") or (n == "primary") or (n == "selected") or (n == "kind") { return true }
if (n == "translate") { return true }
return str_starts(n, "on") or str_starts(n, "--")
}
function xk_words_attr(n: pointer) -> bool {
return (n == "text") or (n == "label") or (n == "hint") or (n == "title") or (n == "caption")
}
# a keyword: nothing but a-z 0-9 _ . # % -
function xk_keyword(s: pointer) -> bool {
var i = 0
while i < len(s) {
let c = s[i]
let ok = (c >= 'a' and c <= 'z') or (c >= '0' and c <= '9') or c == '_' or c == '.' or c == '#' or c == '%' or c == '-'
if not ok { return false }
i += 1
}
return true
}
# one attribute's value, src[a..b]: its holes, and its words if it is text
function xk_attr(name: pointer, a: int, b: int, quiet: bool) -> void {
let src = g_xk_src
let textual = not xk_not_text(name)
let rest = buf_new()
var i = a
while i < b {
if src[i] == '{' {
let e = hole_end(src, i + 1, b)
xk_hole(i + 1, e, quiet or not textual)
i = e + 1
continue
}
buf_putc(rest, src[i])
i += 1
}
if quiet or not textual { return }
let words = buf_str(rest)
if not xk_words_attr(name) and xk_keyword(words) { return }
if not i18n_wordy(words) { return }
xk_where_back(a)
i18n_english_left(g_xk_file, g_xk_at_line, g_xk_at_col, `{name}="{i18n_short(words)}" in a template - write {name}="{{t('key')}}" and give en.po the English`)
}
# ---- a hole --------------------------------------------------------------------------------------
# the code of a hole, src[a..e]: each t('key', ...) checked; a quoted choice that reads as words is
# English left unless the text around it is quiet
function xk_hole(a: int, e: int, quiet: bool) -> void {
let src = g_xk_src
var i = a
while i < e {
let c = src[i]
if c == '"' or c == CH_SQUOTE {
var j = i + 1
while j < e and src[j] != c {
if src[j] == CH_BACKSLASH { j += 1 }
j += 1
}
let lit = src[i + 1 .. j]
if not quiet and xk_choice_wordy(lit) {
xk_where_back(i)
i18n_english_left(g_xk_file, g_xk_at_line, g_xk_at_col, `'{i18n_short(lit)}' in a template's hole - a choice the player reads is a key too: t('key')`)
}
i = j + 1
continue
}
if c == 't' and i + 1 < e and src[i + 1] == '(' and (i == a or not (xk_name_ch(src[i - 1]))) {
i = xk_t_call(i + 2, e)
continue
}
i += 1
}
}
function xk_choice_wordy(s: pointer) -> bool {
if not i18n_wordy(s) { return false }
var i = 0
while i < len(s) {
let c = s[i]
if (c >= 'A' and c <= 'Z') or c == ' ' or c < 0 or c >= 128 { return true }
i += 1
}
return false
}
# t( read, the cursor after it: a literal key is checked, and the values after it counted; where
# the scan goes on (just past the key, so a t() inside the values is found as well)
function xk_t_call(at: int, e: int) -> int {
let src = g_xk_src
var k = at
while k < e and xk_space(src[k]) { k += 1 }
if k >= e or (src[k] != CH_SQUOTE and src[k] != '"') { return at }
let q = src[k]
var j = k + 1
while j < e and src[j] != q { j += 1 }
let key = src[k + 1 .. j]
var depth = 0
var commas = 0
var m = j + 1
while m < e {
let c = src[m]
if c == '"' or c == CH_SQUOTE {
m += 1
while m < e and src[m] != c {
if src[m] == CH_BACKSLASH { m += 1 }
m += 1
}
} else if c == '(' or c == '[' or c == '{' { depth += 1 }
else if c == ')' or c == ']' or c == '}' {
if depth == 0 { break }
depth -= 1
}
else if c == ',' and depth == 0 { commas += 1 }
m += 1
}
xk_where_back(k)
i18n_use(key, false, KS_TEMPLATE, g_xk_file, g_xk_at_line, g_xk_at_col)
if g_i18n_en != null { i18n_holes(key, commas, "t()", g_xk_file, g_xk_at_line, g_xk_at_col) }
return j + 1
}

18
selfhost/frontend/lex.ludic
View file

@ -10,6 +10,7 @@ const TK_NL: int = 4
const TK_EOF: int = 5
const TK_FLOAT: int = 6
const TK_INTERP: int = 7 # `text {expr} text` — raw content, split by the parser
const TK_KEY: int = 8 # k"pause.resume" / kn"catch.count" — a text key (i18n.ludic); ival 1 or 2
property Tok { kind: int = 0, text: pointer = null, ival: int = 0, line: int = 0, pos: int = -1, end: int = -1, col: int = 0, off: int = -1, oend: int = -1 }
# col: the token's column (1-based) on its line; off / oend: where it starts and ends in the text
@ -244,6 +245,23 @@ function lex_at(src: pointer, first_line: int) -> void {
if char_is_alpha(c) {
let start = i
while i < n and char_is_alnum(src[i]) { i += 1 }
# `k"..."` / `kn"..."`: a key literal, the name written against the quote (`k "x"` is a name
# and a string, as it always was)
let kw = i - start
if i < n and src[i] == '"' and ((kw == 1 and c == 'k') or (kw == 2 and c == 'k' and src[start + 1] == 'n')) {
i += 1
let kout = bytes(n)
var kj = 0
while i < n and src[i] != '"' and src[i] != '\n' {
if src[i] == CH_BACKSLASH { kout[kj] = unescape(src[i + 1]); kj += 1; i += 2 }
else { kout[kj] = src[i]; kj += 1; i += 1 }
}
if i >= n or src[i] != '"' { lex_fail(line, "a key literal is never closed with \" on its line") }
i += 1
kout[kj] = 0
tok_push(TK_KEY, kout, kw, line)
continue
}
tok_push(TK_ID, src[start..i], 0, line)
continue
}

5
selfhost/frontend/parse.ludic
View file

@ -248,7 +248,7 @@ function interp_add(acc: Node, part: Node) -> Node {
return mkbin("+", acc, part)
}
function interp_str(e: Node) -> Node { # wrap a hole in string(...)
let c = node(E_CALL); let id = node(E_ID); id.s = "string"; c.a = id; push(c.kids, e); return c
let c = node(E_CALL); let id = node(E_ID); id.s = "string"; id.uns = TPL_HOLE; c.a = id; push(c.kids, e); return c
}
var g_interp_line: int = 1 # the line the interpolated string sits on
var g_interp_pos: int = -1 # the backtick's place in its file
@ -297,6 +297,7 @@ function parse_interp(raw: pointer) -> Node {
return acc
}
const TPL_COPY: int = 91 # on the callee of a template's lone string(hole)
const TPL_HOLE: int = 92 # on the callee of every other string(hole) (a Key is refused there)
# emit E(field: v, ...) — shared by the statement form and the expression form.
# As an expression it yields a cancellable event's cancelled flag (0/1); a
@ -334,6 +335,7 @@ function p_primary() -> Node {
if t.kind == TK_INT { let n = node(E_INT); n.ival = t.ival; n.s = t.text; pi += 1; return n } # s: a long literal's digits
if t.kind == TK_FLOAT { let n = node(E_FLOAT); n.ival = t.ival; n.s = t.text; pi += 1; return n }
if t.kind == TK_STR { let n = node(E_STR); n.s = t.text; pi += 1; return n }
if t.kind == TK_KEY { pi += 1; return key_lit_node(t) } # k"..." / kn"..." (i18n.ludic)
if t.kind == TK_ID {
if (t.text == "true") { let n = node(E_BOOL); n.ival = 1; pi += 1; return n }
if (t.text == "false") { let n = node(E_BOOL); n.ival = 0; pi += 1; return n }
@ -1012,6 +1014,7 @@ function parse_one_decl() -> void {
g_pm_pending = true
}
else if a == "Chunked" { pm_parse_chunked() } # @Chunked(64): read a chunk at a time
else if a == "TextKey" { i18n_parse_textkey() } # @TextKey("steps"): the registry's text keys (i18n.ludic)
else if a == "alloc_ok" { # 25.2: @alloc_ok("why") - this allocates in a frame on purpose
eat_op("(")
let why = expr()

1
selfhost/frontend/permap.ludic
View file

@ -139,6 +139,7 @@ function pm_kind(ty: pointer) -> int {
if (ty == "float") { return PM_FLOAT }
if (ty == "bool") { return PM_BOOL }
if (ty == "string") { return PM_STR }
if i18n_is_key_ty(ty) { return PM_STR } # a Key is text at run time: k"..." in a map's row
if str_starts(ty, "fn(") { return PM_FN }
if is_slice_ty(ty) { return PM_LIST }
let c = find_comp(ty)

8
selfhost/frontend/permap_check.ludic
View file

@ -220,6 +220,7 @@ function pmk_map(md: pointer) -> void {
}
pmk_refs()
pmk_game_assets()
i18n_permap_rows(md) # i18n.ludic: its text keys and English left
}
# a key written twice in one file
function pmk_dups() -> void {
@ -321,6 +322,7 @@ function pmk_rec(rec: Node, comp: Node, row: bool) -> void {
function pmk_desc(e: Node) -> pointer {
if e.kind == E_INT or (e.kind == E_UN and e.a != null and e.a.kind == E_INT) { return "an int" }
if e.kind == E_FLOAT or (e.kind == E_UN and e.a != null and e.a.kind == E_FLOAT) { return "a float" }
if i18n_is_key(e) { return "a key" }
if e.kind == E_STR { return "a string" }
if e.kind == E_BOOL { return "true or false" }
if e.kind == E_ID { return `the name {e.s}` }
@ -376,8 +378,12 @@ function pmk_value(e: Node, f: Node, ty: pointer, at: Node) -> void {
if e.kind != E_BOOL { pmk_want(e, f, "true or false") }
return
}
if k == PM_STR and i18n_is_key_ty(ty) {
if not i18n_is_key(e) { pmk_want(e, f, "a key k\"...\"") }
return
}
if k == PM_STR {
if e.kind != E_STR { pmk_want(e, f, "a \"string\""); return }
if e.kind != E_STR or i18n_is_key(e) { pmk_want(e, f, "a \"string\""); return }
pmk_attrs_str(e, f)
return
}

6
selfhost/frontend/permap_gen.ludic
View file

@ -233,6 +233,10 @@ function pm_gen_field(b: Buf, sn: pointer, fd: Node, i: int) -> void {
let k = pm_kind(fd.ty)
let x = fd.s
let n = itoa(i)
if k == PM_STR and i18n_is_key_ty(fd.ty) {
pm_l(b, ` r.{x} = lres_key(t, c)`)
return
}
if k == PM_INT or k == PM_FLOAT or k == PM_BOOL or k == PM_STR {
pm_l(b, ` r.{x} = {pm_scalar_read(k)}(t, c)`)
return
@ -256,6 +260,8 @@ function pm_gen_field(b: Buf, sn: pointer, fd: Node, i: int) -> void {
pm_l(b, ` {sn}__reset_{el}(pl, e{n})`)
pm_l(b, ` if lres_is_rec(t, k{n}) {{ {sn}__fill_{el}(t, pl, e{n}, k{n}) }}`)
pm_l(b, ` push(r.{x}, e{n})`)
} else if i18n_is_key_ty(el) {
pm_l(b, ` push(r.{x}, lres_key(t, k{n}))`)
} else {
pm_l(b, ` push(r.{x}, {pm_scalar_read(ek)}(t, k{n}))`)
}

1
selfhost/frontend/registry.ludic
View file

@ -92,6 +92,7 @@ function parse_registry() -> void {
push(g_rg_var, v)
push(g_rg_open, 0)
push(g_rg_from, from)
i18n_registry_declared() # i18n.ludic: its @TextKey, before the attributes are taken
push(g_rg_at, at_registry_take()) # attrs.ludic
pm_declared(pm, chunk, v.s)
if from != null and not pm { res_read(v, from) }

1
selfhost/frontend/registry_finish.ludic
View file

@ -61,6 +61,7 @@ function reg_fill(r: int) -> void {
push(rec.kids, fi)
}
if comp != null { res_type_fields(rec, comp) } # L9: nested records take their field's type
i18n_fill_row(r, key, rec, comp, g_df_kline[d], g_df_kcol[d]) # a @Text Key left out: its derived key
let nw = node(E_NEW)
nw.s = t
nw.a = rec

1
selfhost/frontend/view_conv.ludic
View file

@ -6,6 +6,7 @@ function vw_conv(v: int, ty: pointer, x: pointer, head: Node) -> pointer {
if (ty == "int") { return `value_int({x})` }
if (ty == "float") { return `value_float({x})` }
if (ty == "string") { return `value_str({x})` }
if i18n_is_key_ty(ty) { return `value_str(string({x}))` } # a Key: its marked text, for a template's t(expr)
if (ty == "bool") {
vw_helper(`function view_val_bool(b: bool) -> Val {{\n if b {{ return value_bool(1) }}\n return value_bool(0)\n}}\n`, "view_val_bool", head)
return `view_val_bool({x})`

1
selfhost/frontend/view_fill.ludic
View file

@ -6,6 +6,7 @@ function vw_set(v: int, ty: pointer, o: pointer, k: pointer, x: pointer, head: N
if (ty == "int") { return ` value_set_int({o}, "{k}", {x})\n` }
if (ty == "float") { return ` value_set_float({o}, "{k}", {x})\n` }
if (ty == "string") { return ` value_set_str({o}, "{k}", {x})\n` }
if i18n_is_key_ty(ty) { return ` value_set_str({o}, "{k}", string({x}))\n` } # a Key: its marked text, for t(expr)
if (ty == "bool") { return ` value_set_bool({o}, "{k}", {x})\n` }
if (ty == "[]int") { return ` value_set_ints({o}, "{k}", {x})\n` }
if (ty == "[]float") { return ` value_set_floats({o}, "{k}", {x})\n` }

6
selfhost/frontend/vocab.ludic
View file

@ -163,6 +163,7 @@ function vocab_types() -> void {
vc_ty("bool", "primitive", "true or false.")
vc_ty("byte", "primitive", "An unsigned 8-bit value.")
vc_ty("string", "primitive", "Text, UTF-8.")
vc_ty("Key", "primitive", "A text key, k\"module.purpose\" (kn\"...\" a plural): names a text, is not one; tr(key) makes the text.")
vc_ty("pointer", "primitive", "An untyped address (raw memory needs unsafe).")
vc_ty("void", "primitive", "No value: a function that returns nothing.")
vc_ty("entity", "primitive", "A handle to an entity.")
@ -203,7 +204,7 @@ function vocab_attributes() -> void {
vc_at("Material", "checked", "field", "(model_field)", "A material inside the glTF that field names.")
vc_at("Tint", "checked", "field", "(SLOT)", "A colour for that tint slot (a constant).")
vc_at("Derived", "checked", "field", "", "Worked out at boot: anything written in the data is overwritten.")
vc_at("Text", "checked", "field", "", "Read by the player, so translated.")
vc_at("Text", "checked", "field", "", "Read by the player, so translated: a Key field's key is derived, <registry>.<row>.<field>.")
vc_at("Multiline", "checked", "field", "", "Prose: edited as several lines.")
vc_at("Key", "checked", "field", "", "A key code.")
# a field's, with an effect
@ -217,6 +218,7 @@ function vocab_attributes() -> void {
vc_at("ByKey", "checked", "registry", "", "Entries are saved by key, so their order is free.")
vc_at("PerMap", "checked", "registry", "", "Its rows are read per map, from that map's directory, when the map loads.")
vc_at("Chunked", "checked", "registry", "(n)", "A @PerMap table read a chunk of n by n metres at a time.")
vc_at("TextKey", "checked", "registry", "(\"prefix\")", "The prefix of its @Text fields' derived keys, in place of the registry's name in snake case.")
# a declaration's
vc_at("export", "checked", "declaration", "", "The declaration is exported (the same as the export keyword).")
vc_at("Public", "checked", "handler", "", "Promotes a lifecycle hook to an event other code may hear.")
@ -294,6 +296,8 @@ function vocab_literals() -> void {
vc("literal", "1.5", "decimal", "", "", "A fixed, or a float in a numbers float file.")
vc("literal", "'a'", "char", "", "", "A character's code, an int.")
vc("literal", "\"text\"", "string", "", "", "A string; \\n \\t \\r \\0 \\\\ \\\" are escapes.")
vc("literal", "k\"module.purpose\"", "key", "", "", "A Key, checked against the source language's .po; written against its quote.")
vc("literal", "kn\"module.purpose\"", "plural key", "", "", "A plural Key, for trn: its .po entry has a msgid_plural.")
vc("literal", "`a {expr} b`", "template", "", "", "A string with holes; {{ and }} are braces.")
vc("literal", "# ...", "comment", "", "", "A comment to the end of the line.")
}

124348
selfhost/ludicc.seed.ll
View file

File diff suppressed because one or more lines are too long

133621
selfhost/ludicc.win.seed.ll
View file

File diff suppressed because it is too large Load diff

2
selfhost/main.ludic
View file

@ -223,6 +223,7 @@ entry {
maybe_splice_runtime()
fence_manifest(path) # 25.1: a `fence` line in the program's package.ludic
i18n_manifest(path) # phase 26: a `lang` line, and its en.po (i18n.ludic)
ui_register_gotos() # `goto:` buttons -> generated UiClicked listeners
# a game gets a window by default; a plain program stays headless. An explicit
@ -241,6 +242,7 @@ entry {
if g_migrate { mg_collect() } # 0.S2: the vars that move (migrate.ludic)
ix_start() # the lookups by name, as tables (emit_core.ludic)
check_program() # L4: the types agree before anything is emitted
i18n_check() # phase 26: keys against en.po, English left (i18n.ludic)
if g_es_report { escape_report() } # 25.3: which allocations never outlive their frame
else if g_arena { escape_analyse() } # 25.3: and they allocate from the frame's scratch
if g_arena_strict and (getenv_or("LUDIC_DEPS", "") == "") { # 25.3: the region rule, before anything is emitted

2
tools/editors/emacs/ludic-mode.el
View file

@ -26,7 +26,7 @@
"continue" "new" "spawn" "despawn" "enable" "disable" "attach" "detach" "emit" "cancel"
"dispatch" "try" "and" "or" "not"))
(defconst ludic--types
'("int" "long" "float" "double" "fixed" "bool" "byte" "string" "pointer" "void" "entity"
'("int" "long" "float" "double" "fixed" "bool" "byte" "string" "Key" "pointer" "void" "entity"
"countdown" "words" "floats" "doubles" "fixeds" "pointers" "Vector" "IVec2" "Rect"))
(defconst ludic--phases
'("Start" "Input" "FixedUpdate" "Update" "LateUpdate" "Render" "Overlay"))

4
tools/editors/jetbrains/src/main/kotlin/io/ludic/ide/LudicTokens.kt
View file

@ -78,8 +78,8 @@ object LudicVocabulary {
"emit", "cancel", "dispatch", "try", "and", "or", "not"
)
val PRIMITIVES = setOf(
"int", "long", "float", "double", "fixed", "bool", "byte", "string", "pointer", "void",
"entity", "countdown", "words", "floats", "doubles", "fixeds", "pointers", "Vector",
"int", "long", "float", "double", "fixed", "bool", "byte", "string", "Key", "pointer",
"void", "entity", "countdown", "words", "floats", "doubles", "fixeds", "pointers", "Vector",
"IVec2", "Rect"
)
val PHASES = setOf(

2
tools/editors/shared/ludic.tmLanguage.json
View file

@ -184,7 +184,7 @@
{ "name": "keyword.operator.logical.ludic", "comment": "ludic-dev syntax: operator", "match": "\\b(and|or|not)\\b" },
{ "name": "keyword.other.clause.ludic", "comment": "ludic-dev syntax: modifier", "match": "\\b(export|internal|friend|uses|open|of|as|from|numbers|unsafe|mut|phase|query|on|cancellable|public|layer|start|shows|lasts|then|loads|system)\\b" },
{ "name": "storage.type.ludic", "comment": "ludic-dev syntax: declaration", "match": "\\b(program|module|import|property|state|event|action|reducer|enum|model|prefab|scene|handler|ui|namespace|alias|port|bind|registry|def|component|view|prop|const|var|function|fn|extern|entry|test)\\b" },
{ "name": "support.type.primitive.ludic", "comment": "ludic-dev syntax: types", "match": "\\b(int|long|float|double|fixed|bool|byte|string|pointer|void|entity|countdown|words|floats|doubles|fixeds|pointers|Vector|IVec2|Rect)\\b" },
{ "name": "support.type.primitive.ludic", "comment": "ludic-dev syntax: types", "match": "\\b(int|long|float|double|fixed|bool|byte|string|Key|pointer|void|entity|countdown|words|floats|doubles|fixeds|pointers|Vector|IVec2|Rect)\\b" },
{ "name": "constant.language.boolean.ludic", "comment": "ludic-dev syntax: constant", "match": "\\b(true|false|null)\\b" },
{ "name": "constant.language.phase.ludic", "comment": "ludic-dev syntax: phases", "match": "\\b(Start|Input|FixedUpdate|Update|LateUpdate|Render|Overlay)\\b" }
]

2
tools/editors/vscode/syntaxes/ludic.tmLanguage.json
View file

@ -184,7 +184,7 @@
{ "name": "keyword.operator.logical.ludic", "comment": "ludic-dev syntax: operator", "match": "\\b(and|or|not)\\b" },
{ "name": "keyword.other.clause.ludic", "comment": "ludic-dev syntax: modifier", "match": "\\b(export|internal|friend|uses|open|of|as|from|numbers|unsafe|mut|phase|query|on|cancellable|public|layer|start|shows|lasts|then|loads|system)\\b" },
{ "name": "storage.type.ludic", "comment": "ludic-dev syntax: declaration", "match": "\\b(program|module|import|property|state|event|action|reducer|enum|model|prefab|scene|handler|ui|namespace|alias|port|bind|registry|def|component|view|prop|const|var|function|fn|extern|entry|test)\\b" },
{ "name": "support.type.primitive.ludic", "comment": "ludic-dev syntax: types", "match": "\\b(int|long|float|double|fixed|bool|byte|string|pointer|void|entity|countdown|words|floats|doubles|fixeds|pointers|Vector|IVec2|Rect)\\b" },
{ "name": "support.type.primitive.ludic", "comment": "ludic-dev syntax: types", "match": "\\b(int|long|float|double|fixed|bool|byte|string|Key|pointer|void|entity|countdown|words|floats|doubles|fixeds|pointers|Vector|IVec2|Rect)\\b" },
{ "name": "constant.language.boolean.ludic", "comment": "ludic-dev syntax: constant", "match": "\\b(true|false|null)\\b" },
{ "name": "constant.language.phase.ludic", "comment": "ludic-dev syntax: phases", "match": "\\b(Start|Input|FixedUpdate|Update|LateUpdate|Render|Overlay)\\b" }
]

10
tools/ludic-cli/deps.ludic
View file

@ -33,6 +33,9 @@
# Both leave out the ROOTS - a function that reads fn values out of a table (a dispatcher), which reaches
# every state by definition - and count every other function without going through one, so the frame
# and the boot count their own work; the roots are listed on a line of their own.
# english_left: text still written in English where a key belongs (phase 26, the compiler's i18n
# checks) - a template's words, a text attribute, a @Text row's English in the data; 0 without a
# `lang` line in package.ludic.
var dp_mods: []pointer = null
var dp_pkg: []int = null
var dp_uses: []pointer = null
@ -71,6 +74,7 @@ var dp_rdrops: []pointer = new []pointer # 25.5e: `rdrop <kind> <file:l
var dp_resource_drops: int = 0
var dp_oleaks: []pointer = new []pointer # 25.5e: `oleak <Type.field> <file:line> <fn> <released>`
var dp_owned_leaks: int = 0
var dp_english_left: int = 0 # phase 26: a template's words or a @Text row's English, not yet keys
var dp_wreach_n: int = 0
var dp_wreach_at: pointer = ""
@ -115,6 +119,7 @@ function dp_load(path: pointer) -> bool {
dp_fbirths = new []pointer; dp_birth_leaks = 0
dp_rdrops = new []pointer; dp_resource_drops = 0
dp_oleaks = new []pointer; dp_owned_leaks = 0
dp_english_left = 0
dp_writes = new []pointer; dp_aliases = new []pointer; dp_wowner = new []pointer; dp_wname = new []pointer; dp_wfrom = new []pointer
let lines = split_lines(text)
for i in 0 .. len(lines) {
@ -146,6 +151,7 @@ function dp_load(path: pointer) -> bool {
if len(w) == 2 and w[0] == "resource_drops" { dp_resource_drops = s_to_int(w[1]) }
if len(w) >= 5 and w[0] == "oleak" { push(dp_oleaks, lines[i]) }
if len(w) == 2 and w[0] == "owned_leaks" { dp_owned_leaks = s_to_int(w[1]) }
if len(w) == 2 and w[0] == "english_left" { dp_english_left = s_to_int(w[1]) }
if len(w) >= 5 and w[0] == "width" and w[4] == "0" and s_to_int(w[1]) > dp_wide_n {
dp_wide_n = s_to_int(w[1])
dp_wide_at = `{dp_fn_name(w[2])} ({w[3]})`
@ -365,7 +371,7 @@ var dp_vals: []int = null
var dp_cycle: []int = null
var dp_cycle_all: []int = null # the same with the layers' own edges counted
function dp_numbers() -> void {
dp_names = ["modules", "dependencies", "largest_cycle", "cross_writes", "globals_written_from_outside", "widest_function", "widest_reach", "widest_write_reach", "frame_allocs", "frame_keeps", "birth_leaks", "resource_drops", "owned_leaks"]
dp_names = ["modules", "dependencies", "largest_cycle", "cross_writes", "globals_written_from_outside", "widest_function", "widest_reach", "widest_write_reach", "frame_allocs", "frame_keeps", "birth_leaks", "resource_drops", "owned_leaks", "english_left"]
var mods = 0
for k in 0 .. len(dp_mods) { if dp_own(k) { mods += 1 } }
var deps = 0
@ -387,7 +393,7 @@ function dp_numbers() -> void {
}
}
dp_reach_best()
dp_vals = [mods, deps, len(dp_cycle), writes, len(seen), dp_wide_n, dp_reach_n, dp_wreach_n, dp_frame_allocs, dp_frame_keeps, dp_birth_leaks, dp_resource_drops, dp_owned_leaks]
dp_vals = [mods, deps, len(dp_cycle), writes, len(seen), dp_wide_n, dp_reach_n, dp_wreach_n, dp_frame_allocs, dp_frame_keeps, dp_birth_leaks, dp_resource_drops, dp_owned_leaks, dp_english_left]
}
# ---- the command --------------------------------------------------------------

4
tools/ludic-cli/selfhost.ludic
View file

@ -50,6 +50,9 @@ function selfhost_frags() -> []pointer {
push(f, "selfhost/frontend/component_gen.ludic")
push(f, "selfhost/frontend/component_gen2.ludic")
push(f, "selfhost/frontend/component_class.ludic")
push(f, "selfhost/frontend/i18n_po.ludic")
push(f, "selfhost/frontend/i18n.ludic")
push(f, "selfhost/frontend/i18n_xml.ludic")
push(f, "selfhost/backend/emit_core.ludic")
push(f, "selfhost/backend/emit_fence.ludic")
push(f, "selfhost/backend/emit_fence_ir.ludic")
@ -97,6 +100,7 @@ function selfhost_frags() -> []pointer {
push(f, "selfhost/backend/emit_schema.ludic")
push(f, "selfhost/backend/emit_schema_ui.ludic")
push(f, "selfhost/backend/emit_schema_hash.ludic")
push(f, "selfhost/backend/emit_schema_lang.ludic")
push(f, "selfhost/check/check_state.ludic")
push(f, "selfhost/check/check_index.ludic")
push(f, "selfhost/check/check_types.ludic")

35
tools/ludic-cli/test.ludic
View file

@ -74,6 +74,18 @@ function permap_check_case() -> void {
if not s_contains(capture(`cat {js}`), "the unit is spelled") or not s_contains(capture(`cat {js}`), "deg") { bad2(lbl, capture_line(`head -3 {js}`)); return }
ok(lbl)
}
# phase 26: a template checks clean against en.po, its one line of words a warning, and ludic deps
# counts it as english_left
function i18n_check_case() -> void {
let lbl = "--check: a component's template against en.po - clean, \"Paused\" English left (a warning), english_left 1 in ludic deps"
let js = `{tmp_dir()}/dj_i18n.json`
if not shq(`bin/ludicc examples/lang/i18n/i18n_ui.ludic --check --diagnostics=json > {js} 2>/dev/null`) { bad2(lbl, capture_line(`head -3 {js}`)); return }
let got = capture(`cat {js}`)
if not s_contains(got, "English left: a template's words") or not s_contains(got, "Paused") { bad2(lbl, capture_line(`head -3 {js}`)); return }
let dp = capture(`bin/ludic deps examples/lang/i18n/i18n_ui.ludic 2>&1`)
if not s_contains(dp, "english_left: 1") { bad2(lbl, `ludic deps said [{s_trim(dp)}]`); return }
ok(lbl)
}
# --check --diagnostics=json: refused, with exactly `n` ERRORS on stdout (warnings beside them are not
# counted), one of the diagnostics holding `want`
function diag_json_case(path: pointer, n: int, want: pointer, label: pointer) -> void {
@ -514,7 +526,7 @@ function deps_case() -> void {
let lbl = "ludic deps: the numbers, --check against a baseline, --dot, --writes"
let p = "examples/modules/tangle.ludic"
let got = capture(`bin/ludic deps {p} 2>&1`)
let want = "modules: 3\ndependencies: 2\nlargest_cycle: 2\ncross_writes: 1\nglobals_written_from_outside: 1\nwidest_function: 1\nwidest_reach: 1\nwidest_write_reach: 1\nframe_allocs: 0\nframe_keeps: 0\nbirth_leaks: 0\nresource_drops: 0\nowned_leaks: 0\nin the largest cycle: a b\nthe widest function: a_run (examples/modules/tangle/a/index.ludic:3), 1 state\nthe widest write reach: a_run (examples/modules/tangle/a/index.ludic:3), 1 state it can come to change\n"
let want = "modules: 3\ndependencies: 2\nlargest_cycle: 2\ncross_writes: 1\nglobals_written_from_outside: 1\nwidest_function: 1\nwidest_reach: 1\nwidest_write_reach: 1\nframe_allocs: 0\nframe_keeps: 0\nbirth_leaks: 0\nresource_drops: 0\nowned_leaks: 0\nenglish_left: 0\nin the largest cycle: a b\nthe widest function: a_run (examples/modules/tangle/a/index.ludic:3), 1 state\nthe widest write reach: a_run (examples/modules/tangle/a/index.ludic:3), 1 state it can come to change\n"
if not (s_trim(got) == s_trim(want)) { bad2(lbl, `said [{s_trim(got)}]`); return }
let base = `{tmp_dir()}/deps-base.txt`
if not shq(`bin/ludic deps {p} --baseline {base} > /dev/null`) { bad2(lbl, "--baseline failed"); return }
@ -1284,6 +1296,27 @@ function cmd_dev_test() -> int {
diag_json_case("rejected/permap_maps/permap_maps", 6, "there is no Spots row nowhere in this map", "--check reads every map: a cross-row @Ref names a row of the same map")
diag_json_case("rejected/permap_maps/permap_maps", 6, "the key t1 is also written at", "--check reads every map: a chunked table's key is unique across the map's chunk files, both places named")
diag_json_case("rejected/permap_maps/permap_maps", 6, "this map has no examples/rejected/permap_maps/maps/m1/blades.png", "--check reads every map: a game-wide row's @Asset(..., map) file is in each map's directory, unless optional")
# phase 26: text keys - k"..." / kn"..." of type Key, checked against package.ludic's lang en.po
feat_case("lang/i18n/i18n", "", "pause.resume yes no 13 12 hud.day(3) catch.count[2](trout) items.rope.name items.lamp.name steps.gather_wood.hint", "i18n.ludic (phase 26: k\"...\" and kn\"...\" of type Key, their marker byte, == on keys, a @Text Key field filled with its derived key, @TextKey's prefix)")
i18n_check_case()
schema_case("lang/i18n/i18n", "{\"key\": \"items.rope.name\", \"kind\": \"data\", \"kinds\": [\"data\"], \"sites\": [\"examples/lang/i18n/data/i18n_items.lres:2:1\"], \"in_source\": true, \"english\": \"Rope\"", "--emit-schema: lang.keys - a derived key, its kind, its site and en.po's English")
schema_case("lang/i18n/i18n", "\"unused\": [\"map.close\"]", "--emit-schema: lang.unused - en.po's keys the program uses nowhere")
schema_case("lang/i18n/i18n", "\"missing\": [\"hud.day\"], \"fuzzy\": [\"pause.close\"], \"extra\": [\"old.title\"]", "--emit-schema: lang.languages - another .po's missing, fuzzy and extra keys against en.po")
schema_case("lang/i18n/i18n", "{\"english\": \"Close\", \"keys\": [\"map.close\", \"pause.close\"]}", "--emit-schema: lang.split - one English under several keys")
diag_json_case("rejected/i18n_keys/key_missing", 2, "the key pause.gone is not in examples/rejected/i18n_keys/lang/en.po", "a key literal en.po does not have is refused, at its line and column")
diag_json_case("rejected/i18n_keys/key_missing", 2, "is a plural key, and examples/rejected/i18n_keys/lang/en.po:7 gives pause.resume no msgid_plural", "kn\"...\" is a plural key: en.po's entry has a msgid_plural")
diag_json_case("rejected/i18n_keys/key_missing", 2, "trf: hud.day is given 2 value(s), and its English in examples/rejected/i18n_keys/lang/en.po has 1 hole(s)", "trf given more values than the key's English has holes: a warning")
diag_json_case("rejected/i18n_keys/key_missing", 2, "shop.close has no #. description, and its English", "one English split into several keys wants a #. description on each: a warning at en.po's line")
diag_json_case("rejected/i18n_keys/key_mixup", 3, "argument 1 of show wants a string and this is a Key", "a Key is not a string")
diag_json_case("rejected/i18n_keys/key_mixup", 3, "argument 1 of name_of wants a Key and this is a string", "a string is not a Key")
diag_json_case("rejected/i18n_keys/key_mixup", 3, "+ on a Key", "a Key joins no text")
diag_json_case("rejected/i18n_keys/key_hole", 2, "a Key in a template's {hole}", "a Key in a template literal's hole is refused: it would draw its marked name, untranslated")
diag_json_case("rejected/i18n_keys/key_rules", 3, "trn counts, so its key is a plural one", "trn wants a plural key, kn\"...\"")
diag_json_case("rejected/i18n_keys/key_rules", 3, "a plural key needs its count", "tr / trf refuse a plural key")
diag_json_case("rejected/i18n_keys/data_missing", 1, "things.axe.name (the derived key of a @Text field the row gives no value): the key things.axe.name is not in", "a @Text Key field a row leaves out: its derived key must be in en.po")
diag_json_case("rejected/i18n_keys/data_missing", 1, "English left: things.axe.blurb is", "a @Text string field still holding English: a warning")
diag_json_case("rejected/i18n_keys/t_missing", 1, "t('menu.gone'): the key menu.gone is not in", "a template's t('key') is checked against en.po")
diag_json_case("rejected/i18n_keys/t_missing", 1, "t(): hud.day is given 0 value(s)", "a template's t('key') given fewer values than the English has holes: a warning")
feat_case("lang/defaults", "", "a 300.0 8.0 / b 120.0 8.0 / c! 300.0 8.0 / d 300.0 2.0 / x / y", "defaults.ludic (L11: default parameters, positional then named arguments, in a generic too)")
reject_case("rejected/default_missing", "this call to f leaves out a, which has no default", "a call leaves out only what has a default")
reject_case("rejected/default_unknown", "f has no parameter c", "a named argument names a parameter the function has")

6
tools/ludic-tools/fmt.ludic
View file

@ -216,6 +216,12 @@ program LudicFmt {
if char_is_alpha(c) {
let st = i; while char_is_alnum(s[i]) { i += 1 }
let w = s[st..i]
if s[i] == '"' and ((w == "k") or (w == "kn")) { # k"pause.resume": a key literal, one token
i += 1
while s[i] != 0 and s[i] != '"' and s[i] != '\n' { if s[i] == '\\' and s[i + 1] != 0 { i += 2 } else { i += 1 } }
if s[i] == '"' { i += 1 }
push_tok(LT_STR, st, i, line); continue
}
var k = LT_ID
if (w == "true") or (w == "false") or (w == "null") { k = LT_BOOL }
else { if is_type_word(w) { k = LT_TYPE }

6
tools/ludic-tools/lsp.ludic
View file

@ -236,9 +236,9 @@ program LudicLsp {
# ludic-dev syntax: begin - generated from `ludicc --emit-syntax` (selfhost/frontend/vocab.ludic); run `ludic-dev syntax`, do not edit
function is_type_word(w: pointer) -> bool {
if (w == "int") or (w == "long") or (w == "float") or (w == "double") or (w == "fixed") or (w == "bool") { return true }
if (w == "byte") or (w == "string") or (w == "pointer") or (w == "void") or (w == "entity") or (w == "countdown") { return true }
if (w == "words") or (w == "floats") or (w == "doubles") or (w == "fixeds") or (w == "pointers") or (w == "Vector") { return true }
if (w == "IVec2") or (w == "Rect") { return true }
if (w == "byte") or (w == "string") or (w == "Key") or (w == "pointer") or (w == "void") or (w == "entity") { return true }
if (w == "countdown") or (w == "words") or (w == "floats") or (w == "doubles") or (w == "fixeds") or (w == "pointers") { return true }
if (w == "Vector") or (w == "IVec2") or (w == "Rect") { return true }
return false
}
function is_phase_word(w: pointer) -> bool {

10
tools/ludic-tools/ludic_syntax.h
View file

@ -75,7 +75,7 @@ static const char* LUDIC_KW_STMT[] = {
"dispatch","try","and","or","not", 0
};
static const char* LUDIC_TYPES[] = {
"int","long","float","double","fixed","bool","byte","string","pointer","void","entity",
"int","long","float","double","fixed","bool","byte","string","Key","pointer","void","entity",
"countdown","words","floats","doubles","fixeds","pointers","Vector","IVec2","Rect", 0
};
static const char* LUDIC_PHASES[] = {
@ -84,10 +84,10 @@ static const char* LUDIC_PHASES[] = {
static const char* LUDIC_ATTRIBUTES[] = {
"Ref","OneOf","Range","Unit","Asset","Color","Node","Clip","Material","Tint","Derived","Text",
"Multiline","Key","Computed","Sync","frame","max","owns","AppendOnly","ByKey","PerMap",
"Chunked","export","Public","On","Queries","OnSpawn","OnDespawn","OnAttach","OnDetach",
"OnEnable","OnDisable","OnStart","OnQuit","Owned","Server","Predicted","ToServer","ToClients",
"System","EngineSystem","Namespace","deterministic","ClearColor","alloc_ok","creates",
"releases","Handles","Reads","Writes", 0
"Chunked","TextKey","export","Public","On","Queries","OnSpawn","OnDespawn","OnAttach",
"OnDetach","OnEnable","OnDisable","OnStart","OnQuit","Owned","Server","Predicted","ToServer",
"ToClients","System","EngineSystem","Namespace","deterministic","ClearColor","alloc_ok",
"creates","releases","Handles","Reads","Writes", 0
};
/* ludic-dev syntax: end */
static const char* LUDIC_WIDGETS[] = {