feat(ludic.telemetry): an event queue batched to a PostHog-shaped endpoint
Events are encoded once into lines held in memory; a batch is the first lines joined, with the player id put in where a mark stood, POSTed on Http's own thread every flush_ms or at flush_at events; a failure keeps its lines and doubles the wait up to backoff_max; the queue is on disk every save_ms and read back at the next start; off (the enabled port) drops the request in flight, empties the queue and deletes the file; and a run that may not send (can_send) keeps nothing. The player id is the machine's own id (MachineGuid, IOPlatformUUID, /etc/machine-id) hashed with the game's salt, else random; it lives in the game's id file beside whatever else the game keeps there. Ports: TelemetryWorld (can_send, enabled, now_ms, stamp) and TelemetryTransport (send, poll, drop; unbound: Http). Config is a record (host, key, path, lib, queue_file, id_file, id_salt, and the timings). Facts: TELEMETRY_SENT, _FAILED, _ID. Not a System: it runs from a launcher's first frame, before any world exists. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
parent
cac8c740fa
commit
e8c1d54a96
10 changed files with 711 additions and 0 deletions
77
packages/ludic.telemetry/README.md
Normal file
77
packages/ludic.telemetry/README.md
Normal file
|
|
@ -0,0 +1,77 @@
|
|||
# ludic.telemetry
|
||||
|
||||
What a game says about how it is played, sent to an analytics server without ever costing a
|
||||
frame. Each event is encoded once into a line held in memory; the lines go out in batches on
|
||||
Http's own thread; a failed send keeps its lines and waits twice as long next time; the queue is
|
||||
on disk every few seconds, so a crash or a quit loses little and the next start sends it; and a
|
||||
player who says no has nothing queued and the file deleted. Uses
|
||||
[`ludic.base`](../ludic.base/README.md) and nothing else.
|
||||
|
||||
```ludic
|
||||
import "ludic.telemetry"
|
||||
```
|
||||
|
||||
The wire format is PostHog's batch API: `POST <host><path>` with
|
||||
`{"api_key": <key>, "batch": [{"event", "distinct_id", "properties", "timestamp"}, ...]}`. Every
|
||||
event carries `$session_id` (a fresh one each start) and `$lib`. What an event is called and
|
||||
what else its properties say are the game's.
|
||||
|
||||
The player id is one per machine: the operating system's own id (Windows `MachineGuid` through
|
||||
PowerShell, macOS `IOPlatformUUID` through `ioreg`, Linux `/etc/machine-id`) hashed with the
|
||||
game's salt, so a reinstall comes back as the same player and the raw id never leaves the
|
||||
machine. With no salt, or where it cannot be read within ten seconds, it is random (`r-...`). It
|
||||
lives in the game's id file as `{"id": ...}`; any other key the game keeps there is left alone.
|
||||
An event queued before the id is known carries a mark that the batch replaces.
|
||||
|
||||
## Config and ports (`bind`)
|
||||
|
||||
```ludic
|
||||
property TelemetryConfig {
|
||||
host, key, path ("/batch/"), lib # an empty host or key sends nothing, ever
|
||||
queue_file, id_file, id_salt, id_scratch
|
||||
flush_ms (30000), flush_at (50), batch (200), queue_max (5000), backoff_max (6), save_ms (5000)
|
||||
}
|
||||
port TelemetryWorld {
|
||||
can_send: fn() -> bool # may this run send at all - a test, a headless run (unbound: yes)
|
||||
enabled: fn() -> bool # the player's switch (unbound: on)
|
||||
now_ms: fn() -> int # a clock (unbound: the wall clock, by the second)
|
||||
stamp: fn() -> string # an event's ISO time (unbound: now)
|
||||
}
|
||||
port TelemetryTransport { # unbound: Http
|
||||
send: fn(string, string) -> int # url, body -> a handle, < 0 if it could not start
|
||||
poll: fn(int) -> int # -1 pending, 0 failed, 1 taken
|
||||
drop: fn(int) -> void # abandon one
|
||||
}
|
||||
```
|
||||
|
||||
The game binds what it needs once, where it is put together:
|
||||
|
||||
```ludic
|
||||
bind TelemetryWorld { can_send: fn my_can_send, enabled: fn my_switch }
|
||||
```
|
||||
|
||||
## API
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| `telemetry_config(c)` | before the start |
|
||||
| `telemetry_start()` | once, at boot: the id, a session, and the queue an earlier start left (or none, if off) |
|
||||
| `telemetry_event(name, props)` | queue one (props may be null); nothing is kept while off or while the run may not send |
|
||||
| `telemetry_tick()` | every frame: the machine id's child, a batch when due (every `flush_ms`, or at `flush_at` events), its answer, the backoff, the queue on disk every `save_ms`; off forgets |
|
||||
| `telemetry_save()` | the queue on disk now (a quit, a child about to start) |
|
||||
| `telemetry_forget()` | nothing queued, nothing on disk, the request in flight dropped |
|
||||
| `telemetry_set_id(id)`, `telemetry_random_hex(n)` | an id outright; random hex that is distinct per start on Windows too (its `Crypto.random_hex` is zeros) |
|
||||
| `telemetry_on()`, `telemetry_get_config()`, `telemetry_id()`, `telemetry_session()`, `telemetry_ready()`, `telemetry_queued()`, `telemetry_line(i)`, `telemetry_batch_body(n)`, `telemetry_fails()`, `telemetry_next_ms()`, `telemetry_in_flight()` | what the game asks |
|
||||
| `telemetry_facts() -> Queue<TelemetryFact>` | `{ what, count, status, retry_ms }`: `TELEMETRY_SENT`, `TELEMETRY_FAILED`, `TELEMETRY_ID` |
|
||||
| `telemetry_reset()` | back to before the start (tests) |
|
||||
|
||||
It is not a `System`: it runs from the first frame of a launcher, before any world exists, so the
|
||||
game calls `telemetry_tick()` itself.
|
||||
|
||||
## Tests
|
||||
|
||||
```bash
|
||||
ludic build packages/ludic.telemetry/tests/telemetry_test.ludic --headless -o /tmp/telemetry_test && /tmp/telemetry_test
|
||||
```
|
||||
|
||||
A fake clock and a fake transport: nothing reaches a network.
|
||||
92
packages/ludic.telemetry/ident.ludic
Normal file
92
packages/ludic.telemetry/ident.ludic
Normal file
|
|
@ -0,0 +1,92 @@
|
|||
# ident.ludic - one player id per machine: the operating system's own id (Windows MachineGuid,
|
||||
# macOS IOPlatformUUID, Linux /etc/machine-id) hashed with the game's salt, else random
|
||||
var telemetry__proc: int = -1
|
||||
var telemetry__proc_out: string = ""
|
||||
var telemetry__proc_t0: int = 0
|
||||
var telemetry__rand_n: int = 0
|
||||
|
||||
# n random bytes as hex. Crypto.random_hex reads /dev/urandom, which Windows lacks (zeros there),
|
||||
# so it is mixed with the clock and the process and hashed: distinct per start everywhere
|
||||
export function telemetry_random_hex(n: int) -> string {
|
||||
telemetry__rand_n += 1
|
||||
let seed = Crypto.random_hex(n) + "|" + `{Time.now()}|{Crypto.random_u32()}|{Os.pid()}|{telemetry__rand_n}`
|
||||
return Crypto.sha256(seed)[0..n * 2]
|
||||
}
|
||||
|
||||
# the id outright (a game with accounts of its own, a test); written to the id file
|
||||
export function telemetry_set_id(id: string) -> void {
|
||||
telemetry__id = id
|
||||
let path = telemetry__conf().id_file
|
||||
if len(path) > 0 {
|
||||
var v = telemetry__read_obj(path)
|
||||
if v == null { v = Value.object() }
|
||||
Value.put(v, "id", Value.str(id))
|
||||
Fs.write_text(path, Json.encode(v))
|
||||
}
|
||||
telemetry__fact(TELEMETRY_ID, 0, 0, 0)
|
||||
}
|
||||
|
||||
function telemetry__random_id() -> void { telemetry_set_id("r-" + telemetry_random_hex(16)) }
|
||||
|
||||
function telemetry__from_machine(raw: string) -> void {
|
||||
let salt = telemetry__conf().id_salt
|
||||
telemetry_set_id("m-" + Crypto.sha256(salt + ":" + Text.lower(raw))[0..32])
|
||||
}
|
||||
|
||||
function telemetry__id_begin() -> void {
|
||||
if len(telemetry__conf().id_salt) == 0 {
|
||||
telemetry__random_id()
|
||||
return
|
||||
}
|
||||
let plat = Os.platform()
|
||||
if plat == "linux" {
|
||||
let m = Fs.read_text("/etc/machine-id")
|
||||
if m != null and len(Text.trim(m)) > 0 { telemetry__from_machine(Text.trim(m)) } else { telemetry__random_id() }
|
||||
return
|
||||
}
|
||||
telemetry__proc_out = telemetry__conf().id_scratch
|
||||
if len(telemetry__proc_out) == 0 { telemetry__proc_out = Os.temp_dir() + "/ludic-telemetry-machine.txt" }
|
||||
if Fs.exists(telemetry__proc_out) { Fs.remove(telemetry__proc_out) }
|
||||
telemetry__proc = telemetry__spawn_machine(plat)
|
||||
telemetry__proc_t0 = Time.now()
|
||||
if telemetry__proc < 0 { telemetry__random_id() }
|
||||
}
|
||||
|
||||
function telemetry__spawn_machine(plat: string) -> int {
|
||||
let args = new []string
|
||||
if plat == "windows" {
|
||||
var root = "C:\\Windows"
|
||||
if Os.has_env("SystemRoot") and len(Os.env("SystemRoot")) > 0 { root = Os.env("SystemRoot") }
|
||||
let out = "'" + Text.replace(telemetry__proc_out, "'", "''") + "'"
|
||||
push(args, "-NoProfile")
|
||||
push(args, "-NonInteractive")
|
||||
push(args, "-Command")
|
||||
push(args, "(Get-ItemProperty -LiteralPath 'HKLM:\\SOFTWARE\\Microsoft\\Cryptography').MachineGuid | Out-File -Encoding ascii -LiteralPath " + out)
|
||||
return Process.spawn(root + "\\System32\\WindowsPowerShell\\v1.0\\powershell.exe", args)
|
||||
}
|
||||
push(args, "-c")
|
||||
push(args, "ioreg -rd1 -c IOPlatformExpertDevice | awk -F'\"' '/IOPlatformUUID/{print $4}' > '" + telemetry__proc_out + "'")
|
||||
return Process.spawn("/bin/sh", args)
|
||||
}
|
||||
|
||||
# the machine id, once the child has written it (ten seconds at most)
|
||||
function telemetry__id_tick() -> void {
|
||||
if telemetry__proc < 0 { return }
|
||||
let code = Process.poll(telemetry__proc)
|
||||
if code < 0 {
|
||||
if Time.now() - telemetry__proc_t0 > 10 {
|
||||
Process.kill(telemetry__proc)
|
||||
Process.free(telemetry__proc)
|
||||
telemetry__proc = -1
|
||||
telemetry__random_id()
|
||||
}
|
||||
return
|
||||
}
|
||||
Process.free(telemetry__proc)
|
||||
telemetry__proc = -1
|
||||
var raw = ""
|
||||
let s = Fs.read_text(telemetry__proc_out)
|
||||
if s != null { raw = Text.trim(s) }
|
||||
if Fs.exists(telemetry__proc_out) { Fs.remove(telemetry__proc_out) }
|
||||
if code == 0 and len(raw) >= 8 { telemetry__from_machine(raw) } else { telemetry__random_id() }
|
||||
}
|
||||
11
packages/ludic.telemetry/index.ludic
Normal file
11
packages/ludic.telemetry/index.ludic
Normal file
|
|
@ -0,0 +1,11 @@
|
|||
# ludic.telemetry - what a game says about how it is played: each event is encoded once into a
|
||||
# line, the lines go out in batches on Http's own thread, and a failed send waits longer next time
|
||||
module ludic_telemetry uses ludic_base
|
||||
numbers float
|
||||
import "ludic.base"
|
||||
import "state.ludic"
|
||||
import "ports.ludic"
|
||||
import "queue.ludic"
|
||||
import "send.ludic"
|
||||
import "ident.ludic"
|
||||
import "queries.ludic"
|
||||
5
packages/ludic.telemetry/package.ludic
Normal file
5
packages/ludic.telemetry/package.ludic
Normal file
|
|
@ -0,0 +1,5 @@
|
|||
# ludic.telemetry - events queued in memory, batched to a PostHog-shaped endpoint, backed off when
|
||||
# offline, kept on disk between starts, and forgotten when the player says no. See README.md.
|
||||
package "ludic.telemetry"
|
||||
version "0.1.0"
|
||||
kind source
|
||||
51
packages/ludic.telemetry/ports.ludic
Normal file
51
packages/ludic.telemetry/ports.ludic
Normal file
|
|
@ -0,0 +1,51 @@
|
|||
# ports.ludic - the config, and what the ports answer when the game leaves them unbound
|
||||
export function telemetry_config(c: TelemetryConfig) -> void { telemetry__cfg = c }
|
||||
|
||||
function telemetry__conf() -> TelemetryConfig {
|
||||
if telemetry__cfg == null { telemetry__cfg = new TelemetryConfig }
|
||||
return telemetry__cfg
|
||||
}
|
||||
|
||||
function telemetry__yes() -> bool { return true }
|
||||
function telemetry__wall_ms() -> int { return (Time.now() - telemetry__t0) * 1000 }
|
||||
function telemetry__wall_stamp() -> string { return DateTime.format(Time.now(), "YYYY-MM-DDTHH:mm:ssZ") }
|
||||
|
||||
function telemetry__can_send() -> bool {
|
||||
let c = telemetry__conf()
|
||||
if len(c.host) == 0 or len(c.key) == 0 { return false }
|
||||
return TelemetryWorld.can_send()
|
||||
}
|
||||
|
||||
function telemetry__enabled() -> bool { return TelemetryWorld.enabled() }
|
||||
function telemetry__now() -> int { return TelemetryWorld.now_ms() }
|
||||
function telemetry__stamp() -> string { return TelemetryWorld.stamp() }
|
||||
|
||||
function telemetry__http_send(url: string, body: string) -> int {
|
||||
let h = Http.open("POST", url)
|
||||
if h < 0 { return h }
|
||||
Http.set(h, "Content-Type", "application/json")
|
||||
Http.body(h, body)
|
||||
Http.send(h)
|
||||
return h
|
||||
}
|
||||
|
||||
function telemetry__http_poll(h: int) -> int {
|
||||
let st = Http.poll(h)
|
||||
if st < 0 { return -1 }
|
||||
var ok = 0
|
||||
if st > 0 and Http.ok(h) { ok = 1 }
|
||||
Http.free(h)
|
||||
return ok
|
||||
}
|
||||
|
||||
function telemetry__http_drop(h: int) -> void { Http.free(h) }
|
||||
|
||||
function telemetry__send(body: string) -> int { return TelemetryTransport.send(telemetry__conf().host + telemetry__conf().path, body) }
|
||||
function telemetry__poll(h: int) -> int { return TelemetryTransport.poll(h) }
|
||||
|
||||
# a request abandoned (the player said no)
|
||||
function telemetry__drop_request() -> void {
|
||||
if telemetry__h < 0 { return }
|
||||
TelemetryTransport.drop(telemetry__h)
|
||||
telemetry__h = -1
|
||||
}
|
||||
33
packages/ludic.telemetry/queries.ludic
Normal file
33
packages/ludic.telemetry/queries.ludic
Normal file
|
|
@ -0,0 +1,33 @@
|
|||
# queries.ludic - what the game (a Settings row, a test) asks, and a reset back to before start
|
||||
export function telemetry_id() -> string { return telemetry__id }
|
||||
export function telemetry_session() -> string { return telemetry__session }
|
||||
export function telemetry_ready() -> bool { return telemetry__ready }
|
||||
export function telemetry_get_config() -> TelemetryConfig { return telemetry__conf() }
|
||||
export function telemetry_fails() -> int { return telemetry__fails }
|
||||
export function telemetry_next_ms() -> int { return telemetry__next_ms }
|
||||
export function telemetry_in_flight() -> bool { return telemetry__h >= 0 }
|
||||
|
||||
export function telemetry_queued() -> int {
|
||||
if telemetry__q == null { return 0 }
|
||||
return len(telemetry__q)
|
||||
}
|
||||
|
||||
# a queued line as it will be sent, player mark and all
|
||||
export function telemetry_line(i: int) -> string { return telemetry__q[i] }
|
||||
|
||||
# back to before telemetry_start, ports and config kept (tests; a start after a failed boot)
|
||||
export function telemetry_reset() -> void {
|
||||
telemetry__drop_request()
|
||||
telemetry__ready = false
|
||||
telemetry__id = ""
|
||||
telemetry__session = ""
|
||||
telemetry__q = null
|
||||
telemetry__dirty = false
|
||||
telemetry__sent_n = 0
|
||||
telemetry__next_ms = 0
|
||||
telemetry__saved_ms = 0
|
||||
telemetry__fails = 0
|
||||
telemetry__clock_on = false
|
||||
telemetry__proc = -1
|
||||
q_clear(telemetry_facts())
|
||||
}
|
||||
97
packages/ludic.telemetry/queue.ludic
Normal file
97
packages/ludic.telemetry/queue.ludic
Normal file
|
|
@ -0,0 +1,97 @@
|
|||
# queue.ludic - a start, an event encoded once into a line, the queue on disk, and forgetting
|
||||
# once, at boot: who this is (from the id file, or begun from the machine), a new session, and
|
||||
# whatever an earlier start left unsent - or nothing, if the player has it off
|
||||
export function telemetry_start() -> void {
|
||||
if telemetry__ready { return }
|
||||
telemetry__ready = true
|
||||
telemetry__t0 = Time.now()
|
||||
telemetry__session = telemetry_random_hex(16)
|
||||
telemetry__q = new []string
|
||||
telemetry__id = sv_str(telemetry__read_obj(telemetry__conf().id_file), "id", "")
|
||||
if len(telemetry__id) == 0 { telemetry__id_begin() }
|
||||
if telemetry__enabled() { telemetry__queue_load() } else { telemetry_forget() }
|
||||
}
|
||||
|
||||
# may an event be kept right now?
|
||||
export function telemetry_on() -> bool { return telemetry__ready and telemetry__enabled() and telemetry__can_send() }
|
||||
|
||||
# an event: the properties (null for none) with the session and the library added, and the player
|
||||
# as a mark put in when the batch goes, so an event queued before the id is known is not misfiled
|
||||
export function telemetry_event(name: string, props: Val) -> void {
|
||||
if not telemetry_on() { return }
|
||||
var p = props
|
||||
if p == null { p = Value.object() }
|
||||
Value.put(p, "$session_id", Value.str(telemetry__session))
|
||||
if len(telemetry__conf().lib) > 0 { Value.put(p, "$lib", Value.str(telemetry__conf().lib)) }
|
||||
let e = Value.object()
|
||||
Value.put(e, "event", Value.str(name))
|
||||
Value.put(e, "distinct_id", Value.str(TELEMETRY_MARK))
|
||||
Value.put(e, "properties", p)
|
||||
Value.put(e, "timestamp", Value.str(telemetry__stamp()))
|
||||
push(telemetry__q, Json.encode(e))
|
||||
telemetry__dirty = true
|
||||
let c = telemetry__conf()
|
||||
if len(telemetry__q) > c.queue_max + c.batch { telemetry__queue_drop(len(telemetry__q) - c.queue_max) }
|
||||
}
|
||||
|
||||
# the queue on disk now, if it changed (a quit, a crash about to happen, a child starting)
|
||||
export function telemetry_save() -> void {
|
||||
if not telemetry__dirty or telemetry__q == null { return }
|
||||
telemetry__dirty = false
|
||||
let f = telemetry__conf().queue_file
|
||||
if len(f) == 0 { return }
|
||||
if len(telemetry__q) == 0 {
|
||||
if Fs.exists(f) { Fs.remove(f) }
|
||||
return
|
||||
}
|
||||
Fs.write_text(f, Text.join(telemetry__q, "\n") + "\n")
|
||||
}
|
||||
|
||||
# the player turned it off: nothing more is sent, and nothing queued is kept, here or on disk
|
||||
export function telemetry_forget() -> void {
|
||||
telemetry__drop_request()
|
||||
telemetry__q = new []string
|
||||
telemetry__dirty = false
|
||||
let f = telemetry__conf().queue_file
|
||||
if len(f) > 0 and Fs.exists(f) { Fs.remove(f) }
|
||||
}
|
||||
|
||||
function telemetry__queue_load() -> void {
|
||||
telemetry__q = new []string
|
||||
let f = telemetry__conf().queue_file
|
||||
if len(f) == 0 { return }
|
||||
let s = Fs.read_text(f)
|
||||
if s == null { return }
|
||||
let parts = Text.split(s, "\n")
|
||||
for i in 0 .. len(parts) { if len(parts[i]) > 2 { push(telemetry__q, parts[i]) } }
|
||||
let max = telemetry__conf().queue_max
|
||||
if len(telemetry__q) > max { telemetry__queue_drop(len(telemetry__q) - max) }
|
||||
}
|
||||
|
||||
# drop the first n lines (sent, or the oldest past the cap)
|
||||
function telemetry__queue_drop(n: int) -> void {
|
||||
let rest = new []string
|
||||
for i in n .. len(telemetry__q) { push(rest, telemetry__q[i]) }
|
||||
telemetry__q = rest
|
||||
telemetry__dirty = true
|
||||
}
|
||||
|
||||
# the batch: the first n lines joined, with the player id put in - no parsing
|
||||
export function telemetry_batch_body(n: int) -> string {
|
||||
let part = new []string
|
||||
for i in 0 .. n { push(part, telemetry__q[i]) }
|
||||
let head = Value.object()
|
||||
Value.put(head, "api_key", Value.str(telemetry__conf().key))
|
||||
let h: string = Json.encode(head)
|
||||
let body = h[0..len(h) - 1] + ",\"batch\":[" + Text.join(part, ",") + "]}"
|
||||
return Text.replace(body, TELEMETRY_MARK, telemetry__id)
|
||||
}
|
||||
|
||||
function telemetry__read_obj(path: string) -> Val {
|
||||
if len(path) == 0 { return null }
|
||||
let s = Fs.read_text(path)
|
||||
if s == null { return null }
|
||||
let v = Json.parse(s)
|
||||
if v == null or Value.kind(v) != 6 { return null }
|
||||
return v
|
||||
}
|
||||
70
packages/ludic.telemetry/send.ludic
Normal file
70
packages/ludic.telemetry/send.ludic
Normal file
|
|
@ -0,0 +1,70 @@
|
|||
# send.ludic - every frame: a batch when one is due, its answer when it lands, the wait doubled
|
||||
# when it failed, the queue on disk every few seconds. One request at a time; nothing waits on it.
|
||||
var telemetry__clock_on: bool = false
|
||||
|
||||
export function telemetry_tick() -> void {
|
||||
if not telemetry__ready { return }
|
||||
telemetry__id_tick()
|
||||
if not telemetry__enabled() {
|
||||
if len(telemetry__q) > 0 or telemetry__h >= 0 { telemetry_forget() }
|
||||
return
|
||||
}
|
||||
if not telemetry__can_send() { return }
|
||||
let now = telemetry__now()
|
||||
let c = telemetry__conf()
|
||||
if not telemetry__clock_on {
|
||||
telemetry__clock_on = true
|
||||
telemetry__next_ms = now + c.flush_ms
|
||||
telemetry__saved_ms = now
|
||||
}
|
||||
if now - telemetry__saved_ms >= c.save_ms {
|
||||
telemetry_save()
|
||||
telemetry__saved_ms = now
|
||||
}
|
||||
if telemetry__h >= 0 {
|
||||
telemetry__answer(now)
|
||||
return
|
||||
}
|
||||
if len(telemetry__q) == 0 { return }
|
||||
let early = telemetry__fails == 0 and len(telemetry__q) >= c.flush_at
|
||||
if early or now >= telemetry__next_ms {
|
||||
telemetry_save()
|
||||
telemetry__flush(now)
|
||||
if telemetry__fails == 0 { telemetry__next_ms = now + c.flush_ms }
|
||||
}
|
||||
}
|
||||
|
||||
function telemetry__answer(now: int) -> void {
|
||||
let st = telemetry__poll(telemetry__h)
|
||||
if st < 0 { return }
|
||||
telemetry__h = -1
|
||||
if st > 0 {
|
||||
telemetry__queue_drop(telemetry__sent_n)
|
||||
telemetry__fails = 0
|
||||
telemetry__next_ms = now + telemetry__conf().flush_ms
|
||||
telemetry__fact(TELEMETRY_SENT, telemetry__sent_n, st, 0)
|
||||
} else {
|
||||
telemetry__backoff(now)
|
||||
telemetry__fact(TELEMETRY_FAILED, telemetry__sent_n, st, telemetry__next_ms - now)
|
||||
}
|
||||
telemetry_save()
|
||||
}
|
||||
|
||||
function telemetry__flush(now: int) -> void {
|
||||
if telemetry__h >= 0 or len(telemetry__id) == 0 or len(telemetry__q) == 0 { return }
|
||||
let n = min(telemetry__conf().batch, len(telemetry__q))
|
||||
let h = telemetry__send(telemetry_batch_body(n))
|
||||
if h < 0 {
|
||||
telemetry__backoff(now)
|
||||
telemetry__fact(TELEMETRY_FAILED, n, 0, telemetry__next_ms - now)
|
||||
return
|
||||
}
|
||||
telemetry__h = h
|
||||
telemetry__sent_n = n
|
||||
}
|
||||
|
||||
# a failed send: wait twice as long next time, up to backoff_max doublings
|
||||
function telemetry__backoff(now: int) -> void {
|
||||
if telemetry__fails < telemetry__conf().backoff_max { telemetry__fails += 1 }
|
||||
telemetry__next_ms = now + (telemetry__conf().flush_ms << telemetry__fails)
|
||||
}
|
||||
75
packages/ludic.telemetry/state.ludic
Normal file
75
packages/ludic.telemetry/state.ludic
Normal file
|
|
@ -0,0 +1,75 @@
|
|||
# state.ludic - the game's config, the ports, the facts, and what is queued
|
||||
# where the events go and how often. An empty host or key sends nothing, ever.
|
||||
export property TelemetryConfig {
|
||||
host: string = "" # "https://analytics.example.com"
|
||||
key: string = "" # the project's public token (PostHog's phc_...)
|
||||
path: string = "/batch/" # the batch endpoint under the host
|
||||
lib: string = "" # every event's $lib
|
||||
queue_file: string = "" # the unsent lines between starts; "" keeps them in memory only
|
||||
id_file: string = "" # the player id's JSON file ({"id": ...}; other keys are kept)
|
||||
id_salt: string = "" # hashed with the machine's own id; "" makes a random id instead
|
||||
id_scratch: string = "" # where the machine-id child writes (unset: the temp directory)
|
||||
flush_ms: int = 30000 # a batch this often
|
||||
flush_at: int = 50 # or sooner, at this many events
|
||||
batch: int = 200 # events in one request
|
||||
queue_max: int = 5000 # the oldest go past this
|
||||
backoff_max: int = 6 # the wait doubles at most this many times
|
||||
save_ms: int = 5000 # the queue on disk at most this far behind
|
||||
}
|
||||
|
||||
# what telemetry asks the game (unbound: yes, on, and the wall clock)
|
||||
export port TelemetryWorld {
|
||||
can_send: fn() -> bool = fn telemetry__yes # may this run send at all (a test, a dev build: no)
|
||||
enabled: fn() -> bool = fn telemetry__yes # the player's switch; off forgets everything queued
|
||||
now_ms: fn() -> int = fn telemetry__wall_ms # a clock in milliseconds
|
||||
stamp: fn() -> string = fn telemetry__wall_stamp # the ISO time an event happened
|
||||
}
|
||||
|
||||
# how a batch travels (unbound: Http). send returns a handle (< 0: could not start); poll
|
||||
# answers -1 while pending, 0 on failure, 1 when the server took it; drop abandons one.
|
||||
export port TelemetryTransport {
|
||||
send: fn(string, string) -> int = fn telemetry__http_send
|
||||
poll: fn(int) -> int = fn telemetry__http_poll
|
||||
drop: fn(int) -> void = fn telemetry__http_drop
|
||||
}
|
||||
|
||||
export const TELEMETRY_SENT: int = 0 # a batch went (count)
|
||||
export const TELEMETRY_FAILED: int = 1 # a batch did not (status, retry_ms)
|
||||
export const TELEMETRY_ID: int = 2 # the player id became known
|
||||
|
||||
export property TelemetryFact {
|
||||
what: int = 0
|
||||
count: int = 0
|
||||
status: int = 0
|
||||
retry_ms: int = 0
|
||||
}
|
||||
|
||||
const TELEMETRY_MARK: string = "@@TELEMETRY_PLAYER@@"
|
||||
|
||||
var telemetry__cfg: TelemetryConfig = null
|
||||
var telemetry__ready: bool = false
|
||||
var telemetry__id: string = ""
|
||||
var telemetry__session: string = ""
|
||||
var telemetry__q: []string = null # the queued lines, oldest first
|
||||
var telemetry__dirty: bool = false # telemetry__q differs from the file
|
||||
var telemetry__h: int = -1 # the batch in flight
|
||||
var telemetry__sent_n: int = 0 # how many lines it carries
|
||||
var telemetry__next_ms: int = 0 # when the next send may go
|
||||
var telemetry__saved_ms: int = 0 # when the queue was last put on disk
|
||||
var telemetry__fails: int = 0 # sends failed in a row
|
||||
var telemetry__t0: int = 0 # the default clock's zero, wall seconds
|
||||
var telemetry__facts: Queue<TelemetryFact> = null
|
||||
|
||||
export function telemetry_facts() -> Queue<TelemetryFact> {
|
||||
if telemetry__facts == null { telemetry__facts = queue_new("telemetry.facts") }
|
||||
return telemetry__facts
|
||||
}
|
||||
|
||||
function telemetry__fact(what: int, count: int, status: int, retry: int) -> void {
|
||||
let f = new TelemetryFact
|
||||
f.what = what
|
||||
f.count = count
|
||||
f.status = status
|
||||
f.retry_ms = retry
|
||||
q_push(telemetry_facts(), f)
|
||||
}
|
||||
200
packages/ludic.telemetry/tests/telemetry_test.ludic
Normal file
200
packages/ludic.telemetry/tests/telemetry_test.ludic
Normal file
|
|
@ -0,0 +1,200 @@
|
|||
# telemetry_test.ludic - events queue as lines, a batch goes when due and not before, a failure
|
||||
# waits longer, off forgets, a test run never sends, and the queue survives a restart
|
||||
import "ludic.telemetry"
|
||||
import "ludic.base"
|
||||
program TelemetryTest {
|
||||
numbers float
|
||||
var clock: int = 0
|
||||
var on: bool = true
|
||||
var allowed: bool = true
|
||||
var answer: int = -1
|
||||
var refuse: bool = false
|
||||
var sends: int = 0
|
||||
var last_url: string = ""
|
||||
var last_body: string = ""
|
||||
|
||||
function fake_now() -> int { return clock }
|
||||
function fake_enabled() -> bool { return on }
|
||||
function fake_can() -> bool { return allowed }
|
||||
function fake_stamp() -> string { return "2026-09-25T12:00:00Z" }
|
||||
function fake_send(url: string, body: string) -> int {
|
||||
if refuse { return -1 }
|
||||
sends += 1
|
||||
last_url = url
|
||||
last_body = body
|
||||
return 7
|
||||
}
|
||||
function fake_poll(h: int) -> int { return answer }
|
||||
function fake_drop(h: int) -> void { }
|
||||
|
||||
bind TelemetryWorld { can_send: fn fake_can, enabled: fn fake_enabled, now_ms: fn fake_now, stamp: fn fake_stamp }
|
||||
bind TelemetryTransport { send: fn fake_send, poll: fn fake_poll, drop: fn fake_drop }
|
||||
|
||||
function qfile() -> string { return Os.temp_dir() + "/ludic-telemetry-test-queue.txt" }
|
||||
function idfile() -> string { return Os.temp_dir() + "/ludic-telemetry-test-id.json" }
|
||||
|
||||
function fresh() -> void {
|
||||
clock = 0
|
||||
on = true
|
||||
allowed = true
|
||||
answer = -1
|
||||
refuse = false
|
||||
sends = 0
|
||||
last_url = ""
|
||||
last_body = ""
|
||||
telemetry_reset()
|
||||
if Fs.exists(qfile()) { Fs.remove(qfile()) }
|
||||
Fs.write_text(idfile(), "{\"id\":\"p-1\",\"notices\":2}")
|
||||
let c = new TelemetryConfig
|
||||
c.host = "http://fake"
|
||||
c.key = "k-1"
|
||||
c.lib = "test-game"
|
||||
c.queue_file = qfile()
|
||||
c.id_file = idfile()
|
||||
c.flush_ms = 1000
|
||||
c.flush_at = 5
|
||||
c.batch = 3
|
||||
c.queue_max = 8
|
||||
c.backoff_max = 2
|
||||
c.save_ms = 100000
|
||||
telemetry_config(c)
|
||||
telemetry_start()
|
||||
telemetry_tick()
|
||||
}
|
||||
|
||||
function events(n: int) -> void {
|
||||
for i in 0 .. n { telemetry_event(`e{i}`, null) }
|
||||
}
|
||||
|
||||
function at(ms: int) -> void {
|
||||
clock = ms
|
||||
telemetry_tick()
|
||||
}
|
||||
|
||||
function facts_of(what: int) -> int {
|
||||
let fs = q_drain(telemetry_facts())
|
||||
var n = 0
|
||||
for i in 0 .. len(fs) { if fs[i].what == what { n += 1 } }
|
||||
return n
|
||||
}
|
||||
|
||||
test "an event is one line with the player as a mark, and the batch puts the id in" {
|
||||
fresh()
|
||||
expect(telemetry_id() == "p-1")
|
||||
telemetry_event("hello", null)
|
||||
expect_eq(telemetry_queued(), 1)
|
||||
let line = telemetry_line(0)
|
||||
expect(Text.contains(line, "\"event\":\"hello\""))
|
||||
expect(Text.contains(line, "test-game"))
|
||||
expect(not Text.contains(line, "p-1"))
|
||||
let body = telemetry_batch_body(1)
|
||||
expect(Text.contains(body, "\"api_key\":\"k-1\""))
|
||||
expect(Text.contains(body, "\"distinct_id\":\"p-1\""))
|
||||
}
|
||||
|
||||
test "a batch goes when due, and only what the server took leaves the queue" {
|
||||
fresh()
|
||||
events(4)
|
||||
at(500)
|
||||
expect_eq(sends, 0)
|
||||
at(1000)
|
||||
expect_eq(sends, 1)
|
||||
expect(last_url == "http://fake/batch/")
|
||||
expect(telemetry_in_flight())
|
||||
events(1)
|
||||
answer = 1
|
||||
at(1100)
|
||||
expect_eq(telemetry_queued(), 2)
|
||||
expect_eq(facts_of(TELEMETRY_SENT), 1)
|
||||
}
|
||||
|
||||
test "enough events go early" {
|
||||
fresh()
|
||||
events(5)
|
||||
at(10)
|
||||
expect_eq(sends, 1)
|
||||
}
|
||||
|
||||
test "a failure keeps the lines and waits twice as long, up to the cap" {
|
||||
fresh()
|
||||
events(2)
|
||||
answer = 0
|
||||
at(1000)
|
||||
at(1001)
|
||||
expect_eq(telemetry_queued(), 2)
|
||||
expect_eq(telemetry_fails(), 1)
|
||||
expect_eq(telemetry_next_ms(), 1001 + 2000)
|
||||
expect_eq(facts_of(TELEMETRY_FAILED), 1)
|
||||
at(3001)
|
||||
at(3002)
|
||||
at(7002)
|
||||
at(7003)
|
||||
expect_eq(telemetry_fails(), 2)
|
||||
expect_eq(telemetry_next_ms(), 7003 + 4000)
|
||||
}
|
||||
|
||||
test "a request that cannot start backs off too" {
|
||||
fresh()
|
||||
refuse = true
|
||||
events(1)
|
||||
at(1000)
|
||||
expect_eq(telemetry_fails(), 1)
|
||||
expect(not telemetry_in_flight())
|
||||
}
|
||||
|
||||
test "off keeps nothing and deletes the file" {
|
||||
fresh()
|
||||
events(3)
|
||||
telemetry_save()
|
||||
expect(Fs.exists(qfile()))
|
||||
on = false
|
||||
at(10)
|
||||
expect_eq(telemetry_queued(), 0)
|
||||
expect(not Fs.exists(qfile()))
|
||||
events(3)
|
||||
expect_eq(telemetry_queued(), 0)
|
||||
}
|
||||
|
||||
test "a run that may not send queues nothing and sends nothing" {
|
||||
fresh()
|
||||
allowed = false
|
||||
events(9)
|
||||
at(100000)
|
||||
expect_eq(telemetry_queued(), 0)
|
||||
expect_eq(sends, 0)
|
||||
}
|
||||
|
||||
test "the queue is capped at its newest events" {
|
||||
fresh()
|
||||
allowed = true
|
||||
events(12)
|
||||
expect(telemetry_queued() <= 11)
|
||||
expect(Text.contains(telemetry_line(telemetry_queued() - 1), "e11"))
|
||||
}
|
||||
|
||||
test "what was not sent is there at the next start" {
|
||||
fresh()
|
||||
events(3)
|
||||
telemetry_save()
|
||||
telemetry_reset()
|
||||
telemetry_start()
|
||||
expect_eq(telemetry_queued(), 3)
|
||||
}
|
||||
|
||||
test "an id set outright is written beside the file's other keys" {
|
||||
fresh()
|
||||
telemetry_set_id("p-2")
|
||||
let v = Json.parse(Fs.read_text(idfile()))
|
||||
expect(sv_str(v, "id", "") == "p-2")
|
||||
expect_eq(sv_int(v, "notices", 0), 2)
|
||||
}
|
||||
|
||||
test "no salt and no id file makes a random id" {
|
||||
fresh()
|
||||
telemetry_reset()
|
||||
Fs.remove(idfile())
|
||||
telemetry_start()
|
||||
expect(Text.starts_with(telemetry_id(), "r-"))
|
||||
expect_eq(len(telemetry_session()), 32)
|
||||
}
|
||||
}
|
||||
Loading…
Add table
Add a link
Reference in a new issue