# Training characters from a zip of images

This describes the **zip + `manifest.json`** format the `/builder` "Characters"
product accepts when you upload photos/scans of **individual characters** (one
character per image) instead of drawing samples by hand. It builds real
image-mode templates (better than the rasterized-stroke fallback) and records
per-class **baseline geometry** so height/position information is preserved for
later scoring.

It follows the same convention as the internal `wordseg/collect` image-import
(`wordseg/collect/index.html`), narrowed to single characters and extended with
an optional envelope.

## What goes in the zip

```
my-charset.zip
├── manifest.json        ← required, at the zip root
├── a_01.png             ← one character per image
├── a_02.png
├── p_01.png
├── apos_01.png
└── ...
```

- **Images**: PNG or JPEG, one drawn character each, dark ink on a light
  background (the same threshold/Otsu path as photo input; color is discarded).
  Any size — they're normalized on import. Filenames are arbitrary; the
  manifest maps each file to its label.
- **`manifest.json`**: the labels + baseline data (below). It must sit at the
  root of the zip, not inside a subfolder.

## `manifest.json` format

The baseline for each image is **computed by you, elsewhere**, and supplied
here — the builder does not estimate it. Coordinates are in **image pixels**,
origin top-left, **y increasing downward** (standard image coordinates).

```jsonc
{
  "version": 1,
  "xHeight": 52,            // OPTIONAL global scale — see "Units" below
  "characters": [
    { "file": "a_01.png",   "char": "a", "baselineY": 140 },
    { "file": "a_02.png",   "char": "a", "baselineY": 138 },
    { "file": "p_01.png",   "char": "p", "baselineY": 96  },  // descender dips below
    { "file": "l_01.png",   "char": "l", "baselineY": 150 },  // ascender rises high
    { "file": "apos_01.png","char": "'", "baselineY": 150, "split": "test" }
  ]
}
```

A bare array is also accepted (no envelope) — identical to the `wordseg/collect`
shape, with `char` in place of `word`:

```json
[
  { "file": "a_01.png", "char": "a", "baselineY": 140, "xHeight": 52 }
]
```

### Per-entry fields

| field       | required | meaning |
|-------------|----------|---------|
| `file`      | yes      | image filename, relative to the zip root |
| `char`      | yes      | the character this image is (`"a"`, `"P"`, `"7"`, `"'"`, `"?"`). One codepoint. |
| `baselineY` | yes      | the writing **baseline** in this image, in pixels from the top edge. Ink *above* the baseline has a smaller y; descenders fall *below* (larger y). |
| `xHeight`   | no       | this image's x-height in pixels (overrides the global). See Units. |
| `split`     | no       | `"train"` (default) or `"test"`. Force this sample into the test set instead of the automatic split. |

### Envelope fields (object form only)

| field       | required | meaning |
|-------------|----------|---------|
| `version`   | no       | format version, currently `1`. |
| `xHeight`   | no       | global x-height in pixels, applied to every entry without its own. |
| `characters`| yes      | the array of per-entry objects above. |

## Units — how height becomes meaningful

The character matcher is scale-normalized, so to make **height and vertical
position** meaningful across characters they must be expressed in a **shared
unit**. That unit is the **x-height** (the height of a lowercase body letter
like `x`), exactly as in `core/segment/geometry.js`.

Because all your images are captured "as if in the same word" — one consistent
scale — a **single x-height in pixels** describes the whole set. Provide it as
the global `manifest.xHeight`. If you omit it, the builder derives it
automatically from the median height of the x-height letters present in your set
(`a c e m n o r s u v w x z`), matching `wordseg/collect/server.js`. Per-image
`xHeight` is available if your capture scale varies image to image.

From `baselineY` and `xHeight`, the builder computes each character's
baseline-relative geometry, in x-height units:

```
top    = (baselineY − yTopInk)    / xHeight     ← the "height" (see below)
bottom = (baselineY − yBottomInk) / xHeight     ← negative when ink descends below the baseline
width  = (xRightInk − xLeftInk)   / xHeight
```

where `yTopInk`/`yBottomInk` are the topmost/bottommost inked pixels and
`xLeftInk`/`xRightInk` the leftmost/rightmost.

**Height ignores descenders.** The character's *height* is `top` — the extent
**above** the baseline only. Anything descending below the baseline (the tail of
`p g y j q`, a comma) does **not** add to the height; it is recorded separately
in `bottom`. So a `p` and an `n` written at the same body size report the same
height even though the `p` has a descender, and `'` `.` `,` — one shape to the
matcher — are separated by where they sit: `'` reports a large `top`, `.` a tiny
one, `,` a negative `bottom`.

These per-sample values are averaged into a per-class Gaussian
(`{top:[mean,var], bottom:[mean,var], width:[mean,var]}`) and stored in the
`geometry` field of the downloaded training data
(`docs/training-format.md` → *geometry*).

> **Scoring:** the shape matcher is scale/translation-invariant, so on its own
> it cannot tell same-shape classes of different size/position apart (c/C, o/O,
> p/P, and `'` `.` `,` — one dot shape). When a query's baseline frame is known,
> the recognizer reranks the shape candidates by how well each class's stored
> geometry matches the query's measured height/position (a Gaussian prior on
> `top`/`bottom`, weight `geoGamma`, default 2.5). In the builder's accuracy
> test the frame comes from each held-out image's manifest baseline, so case
> pairs are separated automatically. At inference the library applies the same
> rerank when you pass a frame — `rec.recognizeCloud(cloud, geom)` or
> `rec.recognize(canvas, { geom })`, where `geom` is
> `MillirecChar.imageGeometry(bounds, baselineY, xHeight)`. Without a frame
> (an isolated character photo with no baseline reference) it falls back to
> shape only.

## Train / test split

Upload most of your samples for training and the builder holds out the last few
per character to measure accuracy:

- Any entry with `"split": "test"` goes to the test set explicitly.
- Otherwise, within each `char`, the **last N** entries (in manifest order) are
  held out as test (default: the final ~15%, at least 1 when a class has
  several samples). Everything else trains.

After the build the builder reports per-character and overall top-1/top-3
accuracy on the held-out test set so you can see how well single-character
recognition is doing before moving on to words.

## Minimal example

```
letters.zip
├── manifest.json
├── a1.png a2.png a3.png ... a20.png
├── b1.png ... b20.png
└── ...
```

```json
{
  "version": 1,
  "xHeight": 50,
  "characters": [
    { "file": "a1.png", "char": "a", "baselineY": 130 },
    { "file": "a2.png", "char": "a", "baselineY": 131 },
    { "file": "a20.png", "char": "a", "baselineY": 129, "split": "test" }
  ]
}
```

Drop it into `/builder` → Characters → **Import image zip**, review the accuracy
report, and download your library.