Parlance · validation internals

Validation at a million words.

Parlance is a narrative-design tool for story-driven games. It re-checks the whole project on every save: a dialogue pointing at a character that doesn't exist, a flag that's read but never set, an ending nothing reaches. At the scale it targets, 500k to 2M words, that check costs seconds, and it ran on the one thread the editor uses for everything else. This is how a save's validation went from ~1.3 s of frozen editor to milliseconds that never block it, without ever showing you a different answer. The figures below run live in your browser.

scroll
The cost that grew with the project

Every save paid for the whole project.

The validator reads the entire project and reports mistakes. The editor ran it on every save, synchronously, and made the save wait for it to finish. That's fine at a few thousand words. But the cost is linear in project size (about 1.3 microseconds per word), so at a million words every save spent roughly 1.3 seconds on a full re-check, on the single thread that also answers the editor.

Linear is the good case, with no hidden quadratic blow-up, but it's still unbounded. Twice the game is twice the wait, and the games this tool is for only get bigger. Drag the slider and watch the full-check line climb while the project does.

Cost per save · full check vs incremental1,000,000 words
full check 1300 ms incremental 1.0 ms saved 1300×
Full-check cost is ~1.3 µs/word, measured. The incremental cost tracks the edit, not the project, so it stays flat as the game grows. Both lines are log-scaled, so the widening gap is what the incremental path avoids paying.

Two independent changes fix this, and they compose. Move the work off the shared thread so a save never waits on it, and stop doing the whole project when only one thing changed. The first makes saves instant; the second makes the background work itself cheap. The rest of this page is those two, plus the part that makes either one safe to trust.

Split local from global

Two kinds of rule, sorted before anything else.

The original validator was one 1,600-line function. Before it could be made incremental, its rules had to be sorted into two kinds, because they behave differently under a small edit.

Each entity's local pass emits its own issues plus a small contribution: the flags it reads and writes, the references it makes, the XP it grants. A single derive pass folds every contribution together to judge the global rules. Message text moved byte-for-byte, one rule family at a time, so a full re-check still produces exactly what it always did.

Two phaseslocal → derive
entities dialogue d1 character npc quest q1 flag f_a local pass per entity contribution flags · refs · xp local issues REF, DUP… derive pass whole project · FLAG, REACH…
The split is what lets an edit re-run one local check instead of all of them, and re-derive from cached contributions instead of re-reading the project. Everything downstream depends on this sort.
Re-check only what changed

Edit one thing, re-check that thing and what points at it.

When you edit one dialogue, almost nothing about the project's correctness can have changed, only that dialogue and anything that refers to it. The engine records a reverse dependency as it walks each entity's references. Normally A depends on B because A points at B, but the reverse edge runs B → A, which answers "who would I have to re-check if B changed?" instantly.

Click any entity below to edit it. The engine lights up exactly the entity you touched plus its direct referrers (the dirty set), and skips everything else. In a million-word project the skipped set is nearly the whole project.

Reverse-dependency graph · click to edit an entitynothing edited
Click an entity to see who gets re-checked.
 
Forward edges (referrer → target) are stored inverted, so "who points at f_a?" is a single lookup, not a project scan. Edit f_a and only the three entities that name it re-run. Edit a leaf like loc and only it re-runs, because nothing points at it.

Why one hop is enough.

Say A refers to B, and B refers to C. You edit C. The engine re-checks C, and re-checks B because it points at C, but it never re-checks A, even though A depends on B which depends on C. Skipping A is safe. Two guarantees collapse the chain to a single step:

The edit type matters too. Renaming a flag's description doesn't change whether a dialogue that reads it is valid; changing its kind from flag to counter does. So each entity carries a fingerprint of the fields referrers care about, and a referrer re-runs only if that fingerprint changed. Toggle the edit type and edit C:

 
A structural edit to C changes its fingerprint, so B re-checks (one hop), then it stops. A cosmetic edit leaves the fingerprint untouched, so nobody downstream re-runs at all. Either way the hop from B back to A never fires. Re-checking B re-reads B's own body, which your edit to C never touched.
Off the main thread

Run the pass where it can't freeze the editor.

Incremental or not, validation should never sit on the thread that answers the editor. So it moved to a worker thread, a second thread the host talks to by passing messages. The host sends a delta ("apply these few changes and re-derive"), and the worker sends issues back, keeping a warm copy of the project between passes so there's no reload and no re-clone. Edits are debounced ~40 ms so a burst of keystrokes is one pass, and while a pass runs, new edits coalesce into the next one.

The hard part is never losing validation when the worker misbehaves. Drive the state machine below. Send a delta, then crash the worker.

Worker lifecycle · fail safe, not fail silenthealthy
Worker is idle, waiting for a delta.
If the worker crashes or a pass runs past 30 seconds, the host validates that pass in-process so the answer is never lost, then respawns the worker once. Only after a second failure does it settle into synchronous mode, still incremental, back on the main thread. Two field escape hatches exist: PARLANCE_VALIDATE_WORKER=0 runs the incremental engine in-process, and PARLANCE_VALIDATE_INCREMENTAL=0 restores the exact old full-validate-per-save behavior.
Never crash, never refetch

Two supporting moves.

The pass runs inside a bare timer, so a thrown exception there would take down the whole host, and the bad file is still on disk, so it would die again on every relaunch. The rule is that validation is total. It always returns a result and never throws. It reports malformed data as an issue. The incremental engine holds the same contract one level deeper. Any internal inconsistency it detects falls back to a full re-check.

The other move is on the client. The browser used to refetch the entire project (up to 17 MB) after each save to refresh its view. Now it patches its cached copy with the single entity that changed, and only refetches on a genuine miss. The save path stops moving megabytes to redraw one node.

Prove the fast path equals the slow path

All of this is worthless if the answers ever disagree.

If an incremental result ever differs from a full check, you'd see different problems depending on whether an edit arrived as a delta or a reload. The invariant is exact. applyDelta(...) must produce a result byte-identical to a full validate() of the same project state, with the same issues in the same order every time. Dev builds run both on every pass and self-heal on any mismatch, so real authoring sessions become fuzz runs.

Four checks assert that invariant. But a green suite only proves the code works on the cases it checks. It says nothing about whether the tests themselves would catch a bug. A test can run a line of code and assert nothing about the result. It stays green whether that line is right or wrong. A suite full of tests like that looks identical to a real one until something breaks.

Mutation testing is the fifth check, and it works backwards from the other four. It checks the tests rather than the code. It breaks the code on purpose (one deliberate one-line change, a "mutant") and re-runs the whole suite. If the code is now wrong, some test should go red. So:

The widget below runs one real mutant from the battery. Set the code and the test, and watch the verdict. The goal is to kill the mutant, because a killed mutant means a test that catches the bug.

Mutation testing · does the test catch the bug?baseline
Step 1 · choose the code the suite runs against

  
Step 2 · choose which test in the suite runs against that code

  
   
The mutant flips the one line that pulls referrers into the dirty set, so the engine re-checks the edited entity but drops everyone pointing at it. The two tests above differ only in their last line. The strong one asserts the broken reference appears; the weak one only checks that validate() returned something. To read it the way the battery does: hold the strong test and flip the code correct → mutant, and green becomes KILLED, so that test guards this line. Do the same with the weak test and it stays green: SURVIVED, a test that asserts nothing.
Five checks defend the invarianteach catches what the others miss

equivalence oracle

Every dev-build pass compares delta vs full and self-heals on mismatch, so real authoring sessions become fuzz runs.

property fuzzer

Thousands of random edit sequences; delta must equal full after each op. Run with self-heal off, so drift fails loudly.

cold + warm conformance

Every shared validator case entered both as a fresh load and through the delta path. The seeded defect must appear and leave identically.

mechanical field sweep

A test perturbs every field of every entity and asserts delta still equals full, so a new rule reading an untracked field turns it red by construction.

mutation battery

Fifteen incremental-specific mutants (a flipped retraction sign, a constant fingerprint, a dropped sort), each verified to be caught by one of the four checks above. Two concurrency tests here once passed against a broken implementation; the mutation battery is what exposed that they asserted nothing. The rule that follows is to add a mutant whenever you add a rule.

What it bought

A save returns the moment bytes hit disk.

The problems panel catches up tens of milliseconds later over a WebSocket. Typing is never blocked, not even by the one case incremental validation can't beat, a single dialogue of many hundreds of nodes, whose re-check is dominated by re-parsing that one large file and still runs off the main thread.

~1 ms
save response at 1M words, down from ~2,085 ms
24–90×
faster revalidation on a realistic edit mix
0
difference from a full check, proven by the five checks above

The fast path and the slow path give the same answer.

Parlance is the git-native narrative tool this validator runs inside.

Parlance · Orbitope  ·  numbers are one machine, one day, a baseline to compare against