Tiled P0 — C-free parsing primitives: XML reader, base64 decode, gzip framing #67

New issue

Closed

opened 2026-09-01 12:28:47 +02:00 by orkun · 1 comment

orkun commented

2026-09-01 12:28:47 +02:00

Owner

Follow-up P0 of the Tiled design (#66). Design record: Design: Tiled maps.

Scope — C-free parsing primitives

Two of the four "foundations" the #66 body assumed are already in main and are reused, not rebuilt:

DEFLATE (RFC 1951) — z_inflate in runtime/native/inflate.ludic ✅
zlib wrapper (RFC 1950) — z_uncompress, same file ✅
JSON reader — Json.parse/Value.* in runtime/native/value.ludic ✅

So P0 is the remaining primitives only:

XML reader — a minimal pure-Ludic reader for the element/attribute/CDATA subset TMX/TSX/TX use (the one genuinely new parser).
base64 decode — we only have Crypto.base64 encode today; add decode (RFC 4648, all padding remainders).
gzip framing (RFC 1952) — the same DEFLATE body as zlib with a 10-byte header + CRC32/ISIZE trailer; wrap the existing inflate. (Validating the CRC32 vs. skipping it is an open call — see design §6.4.)

Golden fixtures

desert.tmx+.tsx (base64+zlib), sewers.tmx (base64+zlib), mapobject.tmx (base64+gzip), plus a hand-authored base64-uncompressed + CSV pair. Vendor SHA-pinned into assets/tiled-fixtures/ with attribution (design §4).

Acceptance

XML reader parses the sample .tsx/.tmx element tree with golden-file tests.
base64 decode passes RFC 4648 vectors.
gzip inflate round-trips mapobject.tmx's layer data against a known-good decompression.

Blocks P0.5/P1. No map model yet — that is P1.

Follow-up P0 of the Tiled design (#66). Design record: [Design: Tiled maps](https://git.workshopsoft.io/workshopsoft/ludic/wiki/Design%2FTiled). ## Scope — C-free parsing primitives Two of the four "foundations" the #66 body assumed are **already in `main`** and are reused, not rebuilt: - DEFLATE (RFC 1951) — `z_inflate` in `runtime/native/inflate.ludic` ✅ - zlib wrapper (RFC 1950) — `z_uncompress`, same file ✅ - JSON reader — `Json.parse`/`Value.*` in `runtime/native/value.ludic` ✅ So P0 is the **remaining** primitives only: 1. **XML reader** — a minimal pure-Ludic reader for the element/attribute/CDATA subset TMX/TSX/TX use (the one genuinely new parser). 2. **base64 decode** — we only have `Crypto.base64` *encode* today; add decode (RFC 4648, all padding remainders). 3. **gzip framing (RFC 1952)** — the same DEFLATE body as zlib with a 10-byte header + CRC32/ISIZE trailer; wrap the existing inflate. (Validating the CRC32 vs. skipping it is an open call — see design §6.4.) ## Golden fixtures `desert.tmx`+`.tsx` (base64+zlib), `sewers.tmx` (base64+zlib), `mapobject.tmx` (base64+gzip), plus a hand-authored base64-uncompressed + CSV pair. Vendor SHA-pinned into `assets/tiled-fixtures/` with attribution (design §4). ## Acceptance - [ ] XML reader parses the sample `.tsx`/`.tmx` element tree with golden-file tests. - [ ] base64 decode passes RFC 4648 vectors. - [ ] gzip inflate round-trips `mapobject.tmx`'s layer data against a known-good decompression. Blocks P0.5/P1. No map model yet — that is P1.

orkun added the

area:stdlib

label 2026-09-01 12:28:47 +02:00

orkun referenced this issue

2026-09-01 12:29:06 +02:00

Tiled P0.5 — TMX/TSX/TX reader over the P0 XML reader #68

orkun referenced this issue

2026-09-01 12:29:25 +02:00

Tiled P1 — core orthogonal load: rt_tmap model, GID resolver, render #69

orkun referenced this issue

2026-09-01 12:34:25 +02:00

Proposal: comprehensive Tiled (TMX/TMJ) map support — parse & consume every Tiled feature/output #66

orkun referenced this issue from a commit

2026-09-01 13:04:55 +02:00

feat(stdlib): Tiled P0 — XML reader + base64 decode + gzip framing (#67)

orkun commented

2026-09-01 13:05:20 +02:00

Author

Owner

P0 landed in main (79776d1).

The three parsing primitives the TMX path needed — the other three "foundations" (DEFLATE, zlib, JSON) were already in the tree and are reused:

XML reader — runtime/native/xml.ludic, a minimal deterministic pure-Ludic reader for the element/attribute/CDATA subset TMX/TSX/TX use, spliced on demand as Xml.*. Handles nested elements, single/double-quoted attributes, text + <![CDATA[…]]>, comments, the <?xml?> prolog + <!DOCTYPE>, the five predefined entities and &#NN;/&#xHH; character references.
base64 decode — runtime/native/base64.ludic (Base64.decode/Base64.encode); the decoder skips the whitespace Tiled wraps into <data>, passes the RFC 4648 vectors for every padding remainder. Splicing Base64.* also pulls in inflate.ludic, so a plain tool can run the whole base64 → zlib/gzip decode chain.
gzip framing — z_gunzip in runtime/native/inflate.ludic: parse the 10-byte header + optional FEXTRA/FNAME/FCOMMENT/FHCRC fields, inflate the body through the existing DEFLATE core, skip the CRC32/ISIZE trailer (decode doesn't need the CRC — §6.4 resolved: skip).

Golden corpus vendored under assets/tiled-fixtures/ (mapeditor/tiled examples/ — desert/sewers/orthogonal-outside/isometric/hexagonal/perspective_walls, attributed — plus hand-authored handmade.tmx/.tsx whose four layers carry the same GID array in CSV, base64-uncompressed, base64+gzip and base64+zlib).

Acceptance

XML reader parses the sample .tsx/.tmx element tree with golden tests — examples/library/tiled_p0.ludic parses a TSX tree (tag/attrs/entities/children/CDATA text).
base64 decode passes RFC 4648 vectors — all padding remainders, incl. whitespace-embedded.
gzip inflate round-trips against a known-good decompression — a gzip blob decodes byte-exactly; zlib regression too.

Proven by library/tiled_p0.ludic (21 assertions) in the regression suite (x test: 90 passed). Docs pages + inventory added; x check-impl/check-docs green. Unblocks P0.5/P1.

**P0 landed** in `main` (79776d1). The three parsing primitives the TMX path needed — the other three "foundations" (DEFLATE, zlib, JSON) were already in the tree and are reused: - **XML reader** — `runtime/native/xml.ludic`, a minimal deterministic pure-Ludic reader for the element/attribute/CDATA subset TMX/TSX/TX use, spliced on demand as `Xml.*`. Handles nested elements, single/double-quoted attributes, text + `<![CDATA[…]]>`, comments, the `<?xml?>` prolog + `<!DOCTYPE>`, the five predefined entities and `&#NN;`/`&#xHH;` character references. - **base64 decode** — `runtime/native/base64.ludic` (`Base64.decode`/`Base64.encode`); the decoder skips the whitespace Tiled wraps into `<data>`, passes the RFC 4648 vectors for every padding remainder. Splicing `Base64.*` also pulls in `inflate.ludic`, so a plain tool can run the whole base64 → zlib/gzip decode chain. - **gzip framing** — `z_gunzip` in `runtime/native/inflate.ludic`: parse the 10-byte header + optional FEXTRA/FNAME/FCOMMENT/FHCRC fields, inflate the body through the existing DEFLATE core, skip the CRC32/ISIZE trailer (decode doesn't need the CRC — §6.4 resolved: skip). **Golden corpus** vendored under `assets/tiled-fixtures/` (mapeditor/tiled `examples/` — desert/sewers/orthogonal-outside/isometric/hexagonal/perspective_walls, attributed — plus hand-authored `handmade.tmx`/`.tsx` whose four layers carry the same GID array in CSV, base64-uncompressed, base64+gzip and base64+zlib). ### Acceptance - [x] XML reader parses the sample `.tsx`/`.tmx` element tree with golden tests — `examples/library/tiled_p0.ludic` parses a TSX tree (tag/attrs/entities/children/CDATA text). - [x] base64 decode passes RFC 4648 vectors — all padding remainders, incl. whitespace-embedded. - [x] gzip inflate round-trips against a known-good decompression — a gzip blob decodes byte-exactly; zlib regression too. Proven by `library/tiled_p0.ludic` (21 assertions) in the regression suite (`x test`: 90 passed). Docs pages + inventory added; `x check-impl`/`check-docs` green. Unblocks P0.5/P1.

orkun closed this issue

2026-09-01 13:05:28 +02:00