# The overlap input widget

One box. The writer draws a capital letter, then draws the next letter **on top
of it**, and keeps going — the pen never moves along the line. The widget works
out which strokes belong to which letter, greys a letter out when it looks
complete, erases it the moment it is sure the next letter has begun, and after
**every stroke** reports the words the ink could be and draws the handwritten
word so far, newest stroke included. Tap a candidate and the strokes are
re-split to spell it. The result is text (and ink) for whatever your page
does next: a search box, a form field, a game, a note.

Any of five character sets — **capitals** (the default), **lowercase**,
**digits**, **punctuation** (the marks a word or string can contain:
`, . : / ' - % $`), or **all** of them — or exactly the characters you name,
against a word list you control (or none: a digits box reads free). It runs
entirely on the visitor's device: a 29 KB wasm matcher, the set's templates
(27–219 KB, 6–45 KB gzipped) and a 48 KB script (19 KB gzipped). Nothing the
visitor writes leaves the page.

* Try it and edit word lists: `/overlap.html` on the millirec server.
* Lab page with every reading and knob exposed: `/overlaplab.html` (dev only).
* Source: `library/src/overlap.js` (the widget), `core/overlap/overlap.js` (the
  recognizer), `scripts/build-overlap.mjs` (`npm run build:overlap`),
  `scripts/verify-overlap.mjs` (`npm run verify:overlap`, the gate).

---

## 1. Embedding it

### Hosted (one script tag, assets served by the millirec server)

```html
<script async src="https://YOUR-MILLIREC-HOST/embed/millirec-overlap.js"></script>

<div data-millirec-overlap data-target="#answer"></div>
<input id="answer">
```

That is a working input: the default word list (the 10,000 most common English
words, uppercased), a 240 px box, candidates under it, the handwritten word,
and the recognized text mirrored into `#answer` as it is written. Use a word
list you edited on `/overlap.html` with `data-list`:

```html
<div data-millirec-overlap data-list="ol_a1b2c3d4" data-target="#answer"></div>
```

or inline:

```html
<div data-millirec-overlap data-words="RED GREEN BLUE YELLOW ORANGE PURPLE"></div>
```

Pick the characters the box reads with `data-set` (or exactly which ones with
`data-alphabet`):

```html
<div data-millirec-overlap data-set="lowercase"></div>              <!-- a–z, the list lowercased -->
<div data-millirec-overlap data-set="digits"></div>                 <!-- 0–9: no usable words in the default list, so it reads FREE -->
<div data-millirec-overlap data-set="all" data-list="ol_…"></div>   <!-- all 75; the list keeps its case -->
<div data-millirec-overlap data-alphabet="0123456789-"></div>       <!-- exactly these, e.g. phone numbers -->
```

The script scans the page for `[data-millirec-overlap]` when it loads. For
elements added later, call `MillirecOverlap.scan(root)` or create widgets in
JavaScript (below).

### Self-hosted (download, no dependency on the millirec server)

Download `millirec-overlap.zip` from `/overlap.html` (or take
`library/templates/overlap/`). Put the files in one directory and serve them
over HTTP (wasm does not load from `file://`):

| file | what it is | size |
|---|---|---|
| `millirec-overlap.min.js` | the widget: recognizer + matcher glue + UI | 48 KB (19 gz) |
| `millirec.wasm` | the matcher (`-DMILLIREC_LITE`) | 29 KB (13 gz) |
| `training.capitals.json` | templates for A–Z (full + partial letters, 25 points each) | 87 KB (18 gz) |
| `training.lowercase.json` | a–z | 72 KB (15 gz) |
| `training.digits.json` | 0–9 | 27 KB (6 gz) |
| `training.punctuation.json` | `, . : / ' -` (no drill cells yet for `%` and `$`, so those two are named but unreadable) | 13 KB (2 gz) |
| `training.all.json` = `training.json` | all 70 drilled classes (what `data-alphabet` loads and filters) | 201 KB (42 gz) |
| `words.json` / `words.js` | the default word list, two forms (lowercase; folded per set at load) | 94 KB (37 gz) |
| `index.html` | a demo page | |

A widget loads only its set's file; delete the files for sets you do not use.

```html
<script src="words.js"></script>                 <!-- optional: sets window.MILLIREC_WORDS -->
<script src="millirec-overlap.min.js"></script>  <!-- finds millirec.wasm + training.json next to itself -->
<div data-millirec-overlap></div>
```

Without `words.js` the widget fetches `words.json` from the same directory.
If the assets live elsewhere, say so before the script runs:
`window.MILLIREC_OVERLAP = { assetBase: 'https://cdn.example/overlap/' }` (or
`data-asset-base` / the `assetBase` option). Serve `.wasm` as
`application/wasm` and, if the assets are on another origin, with CORS.

The self-hosted build can still use lists hosted on a millirec server:
`window.MILLIREC_OVERLAP = { apiBase: 'https://YOUR-MILLIREC-HOST' }` and then
`data-list="ol_…"`.

---

## 2. What the writer sees

```
┌──────────────┐   THE WOR K            ← text: finished words, then the current word
│              │   ╭──────────────╮
│      R       │   │ W O R I      │     ← the handwritten word so far, newest stroke included
│              │   ╰──────────────╯
│              │   [WORK✓] [WORKING] [WORKS] [WORD✓] …   ← candidates, best first, ✓ = complete
└──────────────┘   [done] [undo] [clear]
```

* The letter being written is drawn in ink. When the best reading says a
  letter is **complete** (the strokes after it belong to a new letter) it turns
  **light grey**; when the recognizer is **sure** — the best reading beats the
  best reading that keeps those strokes together by a margin — the grey strokes
  fade out and the letter is appended to the text.
* The **candidates** are the words the ink could be, best first. A ✓ marks a
  word that is already complete; the others would continue. Tapping one
  **accepts it**: the strokes are re-split to spell that word (so the
  handwritten word shows the letters the writer actually drew, split the way
  the chosen word requires), the word is finished, and the box is cleared.
* **done** finishes the current word with its best reading (Space in the lab
  page); **undo** takes back the last stroke, including any erase it caused;
  **clear** drops the current word's ink.
* Ambiguity survives: if the erased strokes could be O or Q, both readings stay
  alive and the letters that follow settle it through the word list. The text
  shows the best.

---

## 3. Options

Every option is settable in JavaScript (`MillirecOverlap.create(el, { … })`)
or as a `data-*` attribute on the element (kebab-case: `data-idle-finish`).

| option | attribute | default | meaning |
|---|---|---|---|
| `set` | `data-set` | `capitals` | which characters the box reads: `capitals`, `lowercase`, `digits`, `punctuation` (`, . : / ' - % $`), `all` |
| `alphabet` | `data-alphabet` | — | exactly these characters instead (any of the sets' characters; loads the superset and uses only these) |
| `words` | `data-words` | — | the word list: an array, or a string of words separated by spaces/commas |
| `list` | `data-list` (or the value of `data-millirec-overlap`) | — | id of a list hosted on the millirec server (`ol_…`) |
| `wordsSrc` | `data-words-src` | — | URL of a text file (one word per line) or JSON (array or `{words}`) |
| `target` | `data-target` | — | selector or element that receives the text: `value` of an input/textarea (an `input` event is fired), else `textContent` |
| `targetMode` | `data-target-mode` | `live` | `live` = finished words plus the word being written; `words` = finished words only |
| `size` | `data-size` | `240` | the box, in CSS pixels |
| `layout` | `data-layout` | `column` | `column` puts the text/handwriting/candidates under the box, `row` beside it |
| `chips` | `data-chips` | `6` | how many candidate words to show |
| `history` | `data-history` | `1` | previous words' handwriting kept (dimmed) beside the current word |
| `idleFinish` | `data-idle-finish` | `0` | seconds of pen-up after which the word finishes itself (`0` = only on done/tap) |
| `theme` | `data-theme` | `auto` | `auto` follows the visitor's colour scheme; `light`; `dark` |
| `hide` | `data-hide` | — | any of `text handwriting words buttons`, to leave out parts |
| `placeholder` | `data-placeholder` | | text shown before anything is written |
| `engine` | `data-engine` (JSON) | — | recognizer knobs, see §7 |
| `assetBase` | `data-asset-base` | script dir / `BASE/overlap/` | where `millirec.wasm`, `training.json`, `words.json` are |
| `apiBase` | `data-api-base` | the millirec host | the server hosted lists are fetched from |
| `onReady` `onStroke` `onChange` `onWord` `onSelect` `onError` | — | — | callbacks; same detail as the events |

**Word list precedence**: `words` → `list` → `wordsSrc` → `window.MILLIREC_OVERLAP.words`
→ `window.MILLIREC_WORDS` → `words.json` next to the assets.

**The list is folded onto the set.** A capitals box uppercases every word, a
lowercase box lowercases it, the all set keeps case and adds each lowercase
word's Capitalized form (at a third of its weight) and UPPER form (a tenth). A
word with a character the set lacks is skipped. If nothing usable is left —
the default list has no digits — the box runs **free**: every character priced
alike (marks sharing 15% of the prior), no candidates beyond the reading
itself, nothing "off-list". That is the natural mode for numbers and codes.

**Marks inside and after words.** Any complete word may end with `,` `.` or
`:` without the list naming it (`markP`), so "hello," reads. An apostrophe is
read only where the list has it: the default list carries fifty contractions
(don't, it's, we're, o'clock …), and a hosted or inline list keeps whatever
apostrophe forms you give it — a comma and an apostrophe are the same size and
nearly the same shape, so the list is what tells "don't" from "don,t".

---

## 4. JavaScript API

```js
const w = await MillirecOverlap.create('#box', {
  words: ['APPLE', 'BANANA', 'CHERRY'],
  onWord: ({ word, canvas }) => { answers.append(word, canvas) },
})
```

`MillirecOverlap` (global, also the module's default export):

| | |
|---|---|
| `create(host, opts) → Promise<widget>` | mount in an element or selector; resolves when the matcher and word list are loaded |
| `scan(root = document)` | mount every `[data-millirec-overlap]` under `root` that is not mounted yet |
| `preload(assetBase?)` | start loading the matcher + templates early (optional) |
| `buildLexicon(words)`, `splitWords(text)`, `wordStrip(letters, capH, colors)`, `sentenceStrip(...)` | the pieces, for custom UIs |
| `Widget` | the class; `DEFAULTS` the engine knobs; `LETTERS` = `'A'..'Z'`; `version` |

A widget:

| method | |
|---|---|
| `getText()` | the finished words, space-separated |
| `getLiveText()` | finished words plus the best reading of the word being written |
| `getWords()` | finished word records: `{word, letters: [{ch, strokes}], inList, selected, alternatives}` |
| `getCandidates()` | the words the current ink could be: `[{word, score, complete, inList}]`, best first |
| `getHandwriting({scope, index, capHeight})` | a canvas: the current word (default), one finished word by `index`, or `scope: 'all'` for everything written |
| `getState()` | the recognizer's full state (every reading, the beam, the stroke → letter assignment) |
| `setWords(words)` | replace the word list (array or string); returns the number of usable words after folding (0 = free mode) |
| `setAlphabet(spec)` | switch to a set name or an explicit alphabet; reloads the templates; resolves when ready |
| `getLexiconSize()` | usable words in the current list (0 = free) |
| `loadList(id)` | fetch a hosted list and use it |
| `getWordList()` | the list in use |
| `finishWord(word?)` | finish the current word with its best reading, or force it to `word` |
| `selectWord(word, {finish = true})` | what a tap does: re-split the strokes to spell `word` and accept it; `finish: false` adopts the fitted prefix and keeps writing from it |
| `undo()` `clearWord()` `clear()` | last stroke back / drop the current word's ink / forget everything |
| `setOptions(engineOpts)` | recognizer knobs at runtime |
| `on(event, fn) → off` | subscribe; the same events are dispatched on the host element as `millirec-overlap:<event>` (bubbling `CustomEvent`s) |
| `destroy()` | remove the UI |
| `ready`, `el`, `rec` | the load promise, the host, the underlying `OverlapRecognizer` |

Strokes in the records are arrays of `{x, y}` in box coordinates (0..size).

### Events

| event | when | `detail` |
|---|---|---|
| `ready` | the matcher and the list are loaded | `{words, source, list, alphabet, set, free}` |
| `stroke` | after every stroke is read | `{text, live, current, committed, pending, candidates, strokes, erased, canvas}` |
| `change` | after anything that changes the reading (a stroke, undo, a list change, a finished word) | same as `stroke` |
| `word` | a word was finished (done, idle, a tap, or `finishWord`) | the `stroke` detail plus `{word, letters, inList, selected, alternatives, canvas}` |
| `select` | a candidate was chosen | `{word, finish}` |
| `error` | assets or the list could not be loaded | `{error}` |

`detail.canvas` is the handwritten current word (or, on `word`, the finished
word) — an ordinary `<canvas>`, so `canvas.toDataURL()` stores it.

```js
el.addEventListener('millirec-overlap:word', (e) => {
  console.log(e.detail.word)            // 'WORK'
  gallery.appendChild(e.detail.canvas)  // the handwriting, letters side by side
})
```

---

## 5. Word lists

The recognizer reads against the **beginning of every word** in the list as
the writer goes, and prefers a word that is already complete when the ink is
close enough. Order matters: the list is taken as **most common first** and
that rank is the frequency estimate (a word at rank 1 is weighted 1, at rank
100 weighted 1/100, …), so put the words the writer is most likely to mean
first. Anything but capital letters A–Z is dropped; case does not matter;
duplicates are removed.

Three ways to give the widget a list:

1. **On the millirec site**: `/overlap.html` has an editor and the character
   picker. Paste or type words, try them in the live box, then *save as a
   hosted list*. You get an
   id (`ol_…`) for `data-list` and an **edit token** — shown once — that lets
   you update the list later, from the same page or with the API below. Every
   page embedding that id picks up the change on its next load.
2. **In the page**: `data-words="…"`, `data-words-src="words.txt"`, or
   `window.MILLIREC_WORDS = [...]` before the script.
3. **In JavaScript, at any time**: `widget.setWords([...])` or
   `widget.loadList('ol_…')`. The word being written is re-read against the new
   list immediately.

### Hosted-list API

```
POST /api/overlap/lists            {name?, words: [...] | "a b c"}   → 201 {id, editToken, count}
GET  /api/overlap/lists/:id                                          → {id, name, words, updatedAt}
PUT  /api/overlap/lists/:id        {editToken, words?, name?}        → {ok, id, name, count, updatedAt}
```

Lists are capped at 50,000 words; entries keep their case and may use any of
the 75 characters (the widget folds them per set). GETs are open and
CORS-wide (that is what embedding is). In public mode a PUT needs the edit
token; on a dev server everything is editable.

---

## 6. Styling

The widget renders in a shadow root, so page CSS does not leak in. Set custom
properties on the host element:

```css
[data-millirec-overlap] { --mo-ink: #222; --mo-bg: #fff; --mo-line: #ccc; --mo-accent: #0a7;
                          --mo-muted: #999; --mo-suspect: #bbb; --mo-chip: #f4f4f4; --mo-radius: 12px; }
```

`--mo-suspect` is the grey a letter turns when it looks complete. Parts are
exposed for deeper styling: `::part(root)`, `::part(box)`, `::part(text)`,
`::part(handwriting)`, `::part(words)`, `::part(chip)`, `::part(buttons)`.
`data-theme="light|dark|auto"` on the host picks the palette.

---

## 7. Recognizer knobs

Pass as `engine: {…}` / `data-engine='{…}'` / `setOptions({…})`. Defaults are
in `core/overlap/overlap.js` `DEFAULTS`; the ones worth touching:

| knob | default | effect |
|---|---|---|
| `commitMargin` | 5 | nats the best reading must beat the best reading *without* that letter boundary by before the strokes are erased. Lower erases sooner and risks erasing under a wrong reading; measured 3 → 90.6%, 5 → 95.0%, 7 → 95.0% word accuracy at a longer lag |
| `lmW` | 1 | weight of the word list (log P(letter | prefix)); 0.6 → 90.6%, 1 → 95.0%, 1.5 → 92.8% |
| `beamMargin` / `beamK` | 12 / 32 | how many alternative readings of the erased strokes stay alive for later letters to settle |
| `oovPen` | 15 | cost of leaving the word list. Lower it (≈8) if the writer will often write words that are not in the list |
| `strokeCountW` | `auto` | weight of the stroke-count prior (§8): 1 in free mode, 0.25 with a list |
| `sizeW`, `sizeCap`, `sizeRef0` | 1, 8, 0.55 × box | the size channel (§8): weight, most nats it may charge, and the assumed height of a capital before anything is committed |
| `tinyPen`, `tinyRatio` | 8, 0.5 | the charge for a group holding a stroke smaller than half the class's smallest-ever stroke (a period merged into a letter) |
| `markP` | 0.3 | P(a complete word ends with `,` `.` `:`) |
| `freeMarkShare` | 0.15 | in free mode, the prior mass the marks share (a 7 is not "- then /") |
| `pruneClasses` | true | score only the characters the list allows next (what keeps the 75-class set fast) |
| `maxStrokes` | 5 | most strokes one letter may take |
| `costRef`, `temp`, `shMax`, `costEps` | 0.5, 0.4, 4, 0.02 | the matcher-cost → nats mapping; fitted, leave alone |

---

## 8. How it works

Strokes arrive in writing order, so a letter is always a **contiguous run of
strokes**. After every stroke the uncommitted strokes are segmented by a
dynamic program over stroke positions. Each candidate group is scored against
every letter by the wasm point-cloud matcher — against **full** letter
templates, and for the last (open) group also against **partial** templates,
the first *k* strokes of a letter drawn with more, so "an E in progress" is a
hypothesis of its own rather than a bad I plus a bar. Each letter is priced by
the word list: a trie whose nodes carry the frequency mass beneath them, so
P(letter | prefix) is one division. The best path is the reading.

A boundary is **committed** (the strokes before it erased) when the best
reading beats the best reading that has no boundary there by `commitMargin`.
That rival is computed exactly, by a second pass with the boundary forbidden.
What survives a commit is a small **beam** of prefixes at their trie nodes, so
the letters that follow can still repair an early misread.

Three things make the scoring work, each a measured failure first:

1. The matcher cost is a ratio scale (a true letter ≈ 0.1, a wrong letter or
   two letters merged ≈ 1.1), so the score is `(ln costRef − ln cost) / temp`,
   on the log.
2. That score saturates (`shMax`): costs 0.02 and 0.1 are both "excellent";
   unsaturated, two perfectly matched single strokes always beat one merely
   good two-stroke partial.
3. A group's score is multiplied by the strokes it explains, so every
   segmentation sums over the same strokes and only the word list prices the
   number of letters; without it an H's two stems read as "I, then the start
   of something".

Accepting a candidate runs a constrained version of the same program over
**all** the word's strokes (the erased ones are kept), with the letter sequence
fixed, and takes the best split.

Two more pieces arrived with the bigger alphabets. A **stroke-count prior**:
the same character written twice on top of itself leaves exactly one
character's ink, so the only evidence that "55" is not "5" is that this writer
never draws a 5 with two strokes — the templates record how many strokes each
class takes, and a count the class never shows is charged (weight 1 without a
list, where it took digits from 90.0% to 98.3%; 0.25 with a list, where the
list already carries most of that evidence and a full weight cost capitals
four points). And **lexicon pruning**: a group is scored only against the
characters the word list allows next from the current prefixes, and against
the whole alphabet only when none of those fits at all — what keeps a stroke
under ~50 ms with 75 classes.

**Size is a channel of its own**, because a box has no baseline: the writer's
scale is estimated from the characters already committed (each committed
character divided by its class's typical drill size, median of the last
twelve; `sizeRef0` before any), and every reading is charged by how far the
group's size sits from the class's — up to `sizeCap` nats. It is what finds a
**period**: in the drills a period is 0.11 of a capital's height, a comma 0.57
and an apostrophe 0.50, so size isolates the dot and cannot split the other
two (that is the list's job). A period's 25-point cloud is a random curl, so
the period is scored on size alone (`tinyShape`); and a group that holds a
stroke far smaller than its class ever draws — a D with a dot in it — pays
`tinyPen`, which is how a trailing period stops being absorbed into the last
letter ("CARRIED." read "CARRIED" without it).

Measured (`npm run verify:overlap --alphabet <set>`: 3-fold, held-out drill
cells overlaid into one box, 180 words per set):

| set | what is written | words right | characters | ms a stroke (node) |
|---|---|---|---|---|
| capitals | list words | 96.1% | 97.5% | 8 |
| lowercase | list words | 99.4% | 99.7% | 7 |
| digits (free) | random digit strings | 98.9% | 98.7% | 3 |
| all | words as written, Capitalized, UPPER | 95.0% (98.3% ignoring case) | 98.5% | 17 |
| `text` (capitals + `. , : ' -`) | list words, 25% with a trailing `,`/`.`, 20% contractions | 92.2% | 93.7% | 10 |
| `numbers` (digits + `. , : / -`, free) | 3.5, 12:30, 1/2, 1,000, 10-12 | 95.6% | 96.1% | 5 |

The last two are the gate's scenarios for the marks (`--alphabet text` /
`numbers`). Trailing periods and commas read; what remains in `text` is
letter-level (an M read as three I's), and in `numbers` a 7 occasionally
splits into `-` and `/`. The forced re-split finds every character's own
strokes on 97–100% of characters in every set. Browser times run about twice
node's.

---

## 9. Limits and what to expect

* **The templates are one writer's.** Most writers' letters are close enough; a
  writer who forms characters very differently (stroke order, shape) will see
  more corrections. Per-writer templates are the obvious extension —
  `scripts/build-overlap-training.mjs` builds the files from drill cells with
  stroke boundaries intact.
* **Case twins.** In one box there is no baseline or x-height, so C/c, O/o,
  S/s, V/v, W/w, X/x, Z/z, K/k, P/p, U/u are the same ink at different sizes,
  and only the word list can decide case. The all-75 set therefore reads
  "Send" as "send" about a tenth of the time (96.7% right ignoring case). A
  capitals-only or lowercase-only set never meets this; a size channel
  (letter height against the running median in the box) is the next step for
  mixed case.
* **Lookalikes across scripts** — I, l and 1; O and 0; S and 5; Z and 2; B and
  8 — do not collide inside one set. Someone who picks letters plus digits
  should expect the word list to do the work.
* **Punctuation is for mixing in**, not writing alone: `, . : / ' -` are there
  for "3.5", "12:30", "1/2", "don't", "e-mail", "hello,". `%` and `$` are in
  the set but have no drill cells yet, so they cannot be read until they are
  drilled (`/penlab.html`, twelve cells each).
* Some ambiguity is in the paradigm: a P followed by a diagonal leg *is* an R.
  The word list usually decides right; when it decides early and wrong, the
  candidates still offer the alternative and a tap fixes it.
* Words not in the list can be written (an off-list reading costs `oovPen`),
  but the list is what makes fast, confident erasing possible — give the widget
  the words it will actually see.
* Touch works (`touch-action: none` on the box; a palm beside a pen is
  ignored); a mouse works; a stylus is best.

---

## 10. Files

| | |
|---|---|
| `core/overlap/overlap.js` | the recognizer (DOM-free): lexicon trie, matcher wrapper, segmentation DP, commit, `fitToWord` |
| `library/src/overlap.js` | the widget: shadow-DOM UI, asset + list loading, events, API |
| `scripts/build-overlap.mjs` | `npm run build:overlap` — bundles the widget over the lite wasm glue, builds the superset and per-set `training.*.json` and `words.json`, the template dir and the zip |
| `scripts/build-overlap-training.mjs` | the templates (full + partial) from `training/pen/char-*`, any alphabet |
| `scripts/verify-overlap.mjs` | `npm run verify:overlap` — the gate; `--alphabet <set>`, `--stats`, `--word HIGHER`, `--set knob=value` |
| `scripts/build-charsets.mjs` | `npm run build:charsets` — the per-set SINGLE-character bundles (char-lite) and their 3-fold measurements for the home page's demo and downloads |
| `server/routes-overlap.js` | the hosted-list API |
| `server/public/overlap.html` | the product page: demo, list editor, snippets |
| `server/public/overlaplab.html` | the lab: every reading, every knob |
| `server/public/embed/millirec-overlap.js` | the hosted embed (served with the host injected) |
| `library/templates/overlap/` | the self-host download |