feat(trainer): phase-B body-type classifier — trainer job on the collector host + the classifier stage on the booth

apps/trainer (parking-trainer): inspect / train / evaluate / publish. Reads the wash
collector's SQLite + crops read-only off its volume; time split (validation = newest
slice); thin classes dropped; damped class weights; `features` mode (frozen ImageNet
backbone, on-disk feature cache, seconds to retrain) and `finetune` mode (light
augmentation). CPU-only torch from PyTorch's wheel index. ONNX export checked against
the torch model; NO model file below the validation floor (exit 3, report still written);
exit 2 = not enough labels. `evaluate` scores a shipped model on labels reviewed after
training + the unlabelled pile; `publish` PUTs a version folder to a Gitea generic package.
Light core deps; the `train` extra is heavy — CI syncs without it, torch tests skip.

apps/vision: BodyTypeClassifier (bodytype.onnx + sidecar = the preprocessing contract:
crop margin, input size, RGB 0-255, normalisation inside the graph) and
RefinedVehicleDetector over YOLOX — refines only `car` or a class the classifier trained
on, min-confidence, `detector_class` on the result; path set but no file = phase B off
without an error; a broken file is a health detail. models/bodytype.version (tracked,
empty) pins the published version the Dockerfile fetches at build (BuildKit secret;
a pin that cannot be fetched fails the build). Verified: a trainer model gives identical
probabilities inside the vision service; both images built and smoke-tested.

Delivery: parking-trainer image in build-images.yml, the `trainer` compose profile on the
collector stack (CPU, read-only data, TRAINER_OUT), commented TRAINER_OUT/PUBLISH_TOKEN in
the wash-collector stack, .dockerignore for both Python contexts, trainer deps synced in CI.

Wiki: bodytype-classifier-training rewritten as built (+ one fleet model not per site,
secrets/access, where the crops live), opencv-anpr-service §Phase B, vision-review-outbox,
vision-service-packaging, fleet-deployment-komodo, index, log.

Claude-Session: https://claude.ai/code/session_01FWncR69HgGPuei1dLrW3cU

wiki/decisions/bodytype-classifier-training.md

@@ -1,52 +1,116 @@
 ---
 title: Body-type classifier (phase B) — training path and hardware
 type: decision
-status: decided 2026-09-07; NOT built (user: "no build just yet")
-related: [vision-review-outbox, opencv-anpr-service, venue-modules, fleet-deployment-komodo, technology-stack]
+status: decided 2026-09-07; BUILT 2026-09-07 (trainer + vision stage); first real run waits for labels
+related: [vision-review-outbox, opencv-anpr-service, venue-modules, fleet-deployment-komodo, vision-service-packaging, technology-stack]
 ---
 
 # Body-type classifier (phase B) — training path and hardware
 
 The Car Wash category suggestion needs SUV vs sedan, which the phase-A COCO detector cannot give
 ([[opencv-anpr-service]] §Vehicle body type). Phase B is a **classifier over the detector's crop**,
-trained on the reviewer's labels gathered through the [[vision-review-outbox]]. This page records
-what the loop looks like, what it runs on, and what is deliberately not done. Discussed and decided
-with the user on 2026-09-07; **nothing here is built yet** — the user will say when.
+trained on the reviewer's labels gathered through the [[vision-review-outbox]]. Decided with the
+user on 2026-09-07 (morning), **built the same day** once the user said "build the trainer for the
+Xeon". This page is the loop as built; what is still outstanding is at the end.
 
-## The loop (as designed)
+## The loop (as built)
 
-Today the loop stops at the reviewer's verdict: the collector holds labels + crops and can export
-`labels.csv`. Nothing trains, nothing updates a booth. The rest of the path, each step a place
-where a person decides:
+Each step is a place where a person decides. Nothing here runs on its own.
 
-1. **Train** — a one-off job (`apps/trainer`, Python/uv like the vision service) on the
-   collector's host reads the export and the crops straight off the collector volume, splits by
-   TIME (validation = newer cars than training, so the number reflects tomorrow's traffic), and
-   fine-tunes a small **BSD-licensed torchvision backbone** (the licence rule applies to weights as
-   much as code; timm/ImageNet weights only if their terms are checked). Outputs three files: the
-   ONNX classifier, a sidecar (class list, preprocessing constants, version), and a metrics report
-   (accuracy per class + confusion matrix). It **refuses to write the model** below a validation
-   floor the owner sets — a bad model never becomes a file. Class imbalance (nine sedans in ten)
-   is weighted in the loss and reported; classes with too few labels are dropped from that run.
-2. **Evaluate before anything ships** — the owner reads the report. 85–95 % on frontal gate views
-   is the expectation once tuned; enough to *flag*, never to *bill* (the flag records, the site
-   threshold exists for exactly this).
-3. **Publish** — weights are not code and do not live in git: a versioned file in the Gitea
-   package registry / a release asset, fetched by URL like the YOLOX weights.
-4. **Bake and build** — the vision Dockerfile fetches that version at build time; a second stage in
-   `vehicle.py` runs the classifier on the detector's box and replaces `car` with the finer class
-   when confident, else keeps YOLOX's answer. One model path setting like the YOLOX one; off when
-   unset. The contract, the mapping chips and the flag do not change — the vocabulary already holds
-   sedan/hatchback/suv/minivan/pickup.
+1. **Train** — `apps/trainer` (`parking-trainer`, Python/uv like the vision service; its own
+   image `parking-trainer`, a one-off job on the collector's host — never a booth service).
+   `train` reads the collector's `collector.sqlite` and `crops/` **straight off the volume**
+   (read-only), takes only reviewed, usable rows (the operator's pick and the camera's class are
+   never labels), **splits by TIME** (validation = the newest 20 % by *time seen*, so the number
+   reflects tomorrow's traffic), drops classes with fewer than `--min-per-class` (20) labels from
+   that run and reports them, weighs the loss by damped inverse frequency (√, mean 1 — full
+   inverse over-corrects on small sets), trains, and exports. Two modes:
+   - `--mode features` (default): the ImageNet backbone is **frozen**; every crop's feature
+     vector is cached on disk (`<out>/cache/features-<backbone>-<size>.npz`, keyed by item id),
+     and only a linear head is trained — minutes for thousands of crops, **seconds** to retrain
+     when labels arrive (only new crops go through the backbone).
+   - `--mode finetune`: warm-starts the head the same way, then unfreezes everything with light
+     label-preserving augmentation (flip, mild zoom, brightness/contrast) — the step when the
+     cheap mode plateaus.
+   Backbones: `resnet18` (default), `mobilenet_v3_small`, `efficientnet_b0` — torchvision,
+   BSD-3, and the ImageNet weights ship under the same licence (the licence rule applies to
+   weights as much as code). CPU-only PyTorch from PyTorch's own wheel index (`tool.uv.index`).
+2. **Evaluate before anything ships** — every run writes `report.md` (accuracy, macro recall,
+   per-class recall/precision, confusion matrix, dropped classes, loss weights, agreement with the
+   detector's coarse class) and `metrics.json`. The job **refuses to write the model** below
+   `--min-accuracy` (default 0.85) — exit 3, report still written — and also withholds it if the
+   exported ONNX disagrees with the torch model on validation (< 99 % argmax agreement). Exit 2 =
+   not enough labels (fewer than two classes clear the minimum). Later, `evaluate --model …`
+   scores a shipped model against labels **reviewed after it was trained** (a clean held-out
+   check) and prints its class histogram + detector agreement over the **unlabelled** pile — the
+   ongoing drift check without labelling everything, which is why every entry is sent
+   ([[vision-review-outbox]] §The entry stream). 85–95 % on frontal gate views is the
+   expectation once tuned; enough to *flag*, never to *bill*.
+3. **Publish** — weights are not code and do not live in git. `publish <version dir> --url
+   https://git.infra.msai.al/api/packages/mca/generic/parking-bodytype` PUTs the four files as a
+   **Gitea generic package** version (token with `package:write`, `TRAINER_PUBLISH_TOKEN`).
+4. **Bake** — `apps/vision/models/bodytype.version` (tracked in git, empty today) **pins** the
+   version the vision image carries. The Dockerfile fetches `bodytype.onnx` + `bodytype.json`
+   from the package registry at build (auth via a BuildKit secret `bodytype_auth` = the
+   registry user's credentials, never a layer); **a pinned version that cannot be fetched fails
+   the build** — the image must carry what git says it carries; empty pin = no classifier, phase
+   B off, build passes. The pin is a normal commit: reviewable, revertible.
 5. **Deploy** — a TAG bump on the booth's stack. **A booth gets a model the way it gets code**: a
    pinned release you can see and roll back. No runtime model fetch (air-gapped appliance,
    read-only model path — [[vision-service-hardening]]).
 
 Retrain when the labels have grown meaningfully (every few hundred new verdicts at first). First
-run needs roughly **200 reviewed crops per class that matters** (Vetura and SUV at least). Once a
-model exists, its predictions on the *unlabelled* pile checked against a small reviewed sample are
-the ongoing accuracy check without labelling everything — which is why every entry is sent, not a
-sample ([[vision-review-outbox]] §The entry stream).
+run needs roughly **200 reviewed crops per class that matters** (Vetura and SUV at least) — until
+then `inspect` says `ready: false` and `train` exits 2.
+
+## The contract between trainer and booth
+
+The trainer and the vision service share **no code** (different packages, different images), so
+the preprocessing contract is **data**: the `bodytype.json` sidecar (`format:
+parking-bodytype/1`) carries version, the class list (a subset of the shared vocabulary, in
+vocabulary order), `input_size` (224), `crop_margin` (0.08 — the same as the outbox's
+`makeReviewCrop`), colour order, resize method, backbone, mode, label counts and the validation
+metrics. Both sides cut the detector's box + margin, blur the plate strip, squash-resize with
+OpenCV `INTER_AREA` (the crop *is* the vehicle; no centre-crop that loses a bumper), and feed raw
+RGB 0–255 float; **normalisation lives inside the ONNX graph**, so a consumer cannot get the
+constants wrong. Verified on 2026-09-07: a trainer-produced model loaded by the vision service's
+`BodyTypeClassifier` gives identical classes and probabilities (< 1e-4) to the trainer's own
+`OnnxClassifier` on the same crops.
+
+On the booth ([[opencv-anpr-service]] §Phase B): `RefinedVehicleDetector` runs the classifier
+only when the detector said `car` **or** a class the classifier trained on; a truck or bus it
+never saw is left alone (its softmax on an unknown thing means nothing). Below
+`VISION_VEHICLE_CLASSIFIER_MIN_CONFIDENCE` (0.6) the detector's class stands. `vehicle.
+detector_class` records the coarse class whenever the stage ran; `model_version` reads
+`…+yolox:…+bodytype:<version>`. The flag on the desk, the mapping chips, the threshold — nothing
+downstream changed: the vocabulary already held sedan/hatchback/suv/minivan/pickup.
+
+## One model for the fleet, not one per site (user asked, 2026-09-07)
+
+The trainer pools **every booth's** reviewed labels into one training set (no per-booth filter),
+and one `bodytype.version` pin bakes one model into the one vision image every booth runs. Body
+type is a property of the car, not the site; pooling is what makes 200 crops per class reachable;
+and a single pinned version is the whole "a booth gets a model the way it gets code" idea. What
+*is* per site stays in Setup: the class→category mapping and the flag threshold — the model says
+"suv", the site decides what an SUV costs and when a downgrade is worth flagging.
+
+Where a site can still differ is the **camera** (mount height, angle, lens), not the cars. Every
+crop carries its booth id, so the report can break accuracy down per booth — not in the report
+today; add it once a second site sends labels. A booth filter in the trainer and a second pin
+would only be built on evidence that a site's view needs its own model.
+
+## Secrets and access (2026-09-07)
+
+- **`TRAINER_PUBLISH_TOKEN`** — a Gitea access token with the `write:package` scope, used by the
+  `publish` command and nothing else, to PUT a passing model's files into the generic package
+  `mca/parking-bodytype`. Training, `inspect` and `evaluate` need no token; leave it unset until
+  the first model passes the floor. Create it under a user who can write packages in the `mca`
+  org, store it as the Komodo secret `gitea_package_write_token`, uncomment the line in the
+  `wash-collector` stack.
+- **Read side** — the CI build fetches the pinned version with the existing registry
+  credentials (`REGISTRY_USERNAME:PASSWORD` as the BuildKit secret `bodytype_auth`); if the
+  package is org-private that user needs package read, which the Docker-registry user already
+  has in Gitea.
 
 ## Hardware (decided 2026-09-07)
 
@@ -60,25 +124,49 @@ What the owner has: an **NVIDIA Quadro FX 3800** (in hand, not installed), and i
 - **HD P530 — not for training.** Usable for *inference* via OpenVINO, irrelevant here: inference
   runs on the booths' CPUs, which already do YOLOX in ~250 ms.
 - **The Xeon does the job.** The problem is small (a few thousand 224-px crops, ten classes, a
-  small pretrained backbone). Two modes the trainer should offer:
-  - *Feature extraction + a small head* — run every crop once through the frozen backbone, cache
-    the feature vectors, train a classifier on top: minutes for a few thousand crops, seconds to
-    retrain when labels arrive. Expected to carry most of the accuracy on frontal gate views.
-  - *Full fine-tune* — unfreeze and train end to end: roughly an hour per run on four cores with
-    a mobile-sized backbone. The step to take when the cheap mode plateaus.
-  Training is occasional and unattended, so an hour on a CPU is a non-issue; the data is already
-  on that host, so nothing moves.
-- **Consequences for the build:** the trainer image is **CPU-only PyTorch** (< 1 GB, not the 5 GB
-  CUDA build); the `trainer` seam in `docker-compose.collector.yml` drops the NVIDIA device
-  reservation (one-line change if a modern card ever lands in the host; the trainer should pick
-  up CUDA when present).
+  small pretrained backbone): features mode in minutes, a full fine-tune in roughly an hour with a
+  mobile-sized backbone. Training is occasional and unattended, and the data is already on that
+  host, so nothing moves.
+- **Consequences for the build (done):** the trainer image is **CPU-only PyTorch** (torch
+  2.14+cpu, ~200 MB of wheels, not the ~5 GB CUDA build); the `trainer` service in
+  `docker-compose.collector.yml` is real now — `profiles: ["train"]`, no device reservation
+  (one block to add if a modern card ever lands; the trainer would pick up CUDA), the collector
+  volume mounted read-only, output to `TRAINER_OUT` on the host (default `./models` beside the
+  compose file).
 - **If faster is ever wanted:** a used mid-range card of the last few generations (~€200) turns
   the hour into a minute, given a slot and a PSU. **Renting a cloud GPU is rejected**: the crops
   would leave the premises, and even scrubbed of plates and site that runs against the whole
   privacy design of the outbox.
 
-## Not built
+## Running it (on the collector host)
 
-`apps/trainer`, the classifier stage in `vehicle.py`, the publish step, the compose `trainer`
-profile (still commented, still with the GPU reservation to remove). First real run waits for
-the first few hundred reviewed labels on the collector.
+```
+docker compose -f docker-compose.collector.yml --profile train run --rm trainer inspect
+docker compose -f docker-compose.collector.yml --profile train run --rm trainer train --min-accuracy 0.85
+docker compose -f docker-compose.collector.yml --profile train run --rm trainer evaluate --model /out/<version>/bodytype.onnx
+docker compose -f docker-compose.collector.yml --profile train run --rm trainer publish /out/<version> --url https://git.infra.msai.al/api/packages/mca/generic/parking-bodytype
+```
+
+Then: write the version into `apps/vision/models/bodytype.version`, commit, let the build produce
+the image, bump the booth's `TAG`. The trainer is never started by a deploy (a profile), and the
+`wash-collector` stack's `TRAINER_OUT` / `TRAINER_PUBLISH_TOKEN` lines stay commented until the
+first publish.
+
+## Packaging rule (same as the vision service)
+
+Core deps are light (numpy, opencv-headless, onnxruntime): `inspect`, `evaluate`, the data and
+report code, and the tests run with `uv sync --frozen` alone — **CI syncs without the `train`
+extra** ([[vision-service-packaging]]); the torch tests `importorskip`. The image bakes
+`--extra train` and pre-warms the resnet18 + mobilenet_v3_small ImageNet weights so a run needs
+no network. `pnpm turbo run lint test` covers `@parking/trainer` through the same package.json
+shim pattern (workspace count 7→8).
+
+## Outstanding
+
+- **The first real run** — waits for ~200 reviewed crops per class on the collector (reviewing
+  is the bottleneck now, not code).
+- **Secrets on the reviewer's host** — a Gitea token with `package:write`
+  (`gitea_package_write_token`) for `publish`; the CI registry user must be able to *read* the
+  generic package (it passes its credentials as the build secret).
+- **Tuning knobs after the first report** — the floor, `--min-per-class`, whether finetune beats
+  features on this camera. The report decides, not a guess.

wiki/decisions/fleet-deployment-komodo.md

@@ -158,6 +158,11 @@ collector ([[vision-review-outbox]]) runs on the reviewer's GPU host as its own
 (`wash-collector`, `server = "art-docker-station"`, `file_paths = ["docker-compose.collector.yml"]`).
 Same repo, branch and pinned `TAG` promotion, its own secret references, and — because a stack
 names its compose files — nothing booth-side lands on that host and nothing of it on a booth.
+The same stack carries the phase-B **trainer** as a compose *profile* (`train`,
+[[bodytype-classifier-training]]): a deploy never starts it; the owner runs it by hand on the host
+with `docker compose … --profile train run --rm trainer …`. Its two env lines (`TRAINER_OUT`, the
+`TRAINER_PUBLISH_TOKEN` secret reference) stay commented in `resources.toml` until the first
+publish.
 
 ## Open / not yet done
 

wiki/decisions/vision-service-packaging.md

@@ -92,6 +92,14 @@ The skeleton is **built and wired** (no recognizer models yet):
   that has `alpr`. Rule (2026-09-07, after three red runs): pure post-processing tests get numpy
   from the **dev group**; anything needing OpenCV uses `pytest.importorskip("cv2")`; the service
   itself imports both lazily inside functions.
+- **The same pattern, second package (2026-09-07):** `apps/trainer` (`@parking/trainer`,
+  [[bodytype-classifier-training]]) — light core (numpy, opencv-headless, onnxruntime) + a
+  `train` extra (CPU-only torch/torchvision/onnx/onnxscript from PyTorch's wheel index via
+  `tool.uv.index`); CI syncs without it, torch tests `importorskip("torch")`, the module that
+  imports torch is imported lazily by the `train` command only. Its own image
+  (`parking-trainer`, context `apps/trainer`, uv base image, bakes `--extra train` + the
+  ImageNet backbone weights) is built by build-images.yml beside the other three; both Python
+  contexts now carry a `.dockerignore` (venv/caches/weights out). Workspace count 7→8.
 - **Light-core, heavy-optional:** core deps boot in **stub mode** (no model download) so `uv sync` +
   tests work offline; the real stack is the `alpr` extra (`uv sync --extra alpr` →
   fast-alpr + onnxruntime). `VISION_RECOGNIZER=fast_alpr` switches it on.