feat(trainer): phase-B body-type classifier — trainer job on the collector host + the classifier stage on the booth
Build & push images / images (push) Successful in 6m31s

apps/trainer (parking-trainer): inspect / train / evaluate / publish. Reads the wash
collector's SQLite + crops read-only off its volume; time split (validation = newest
slice); thin classes dropped; damped class weights; `features` mode (frozen ImageNet
backbone, on-disk feature cache, seconds to retrain) and `finetune` mode (light
augmentation). CPU-only torch from PyTorch's wheel index. ONNX export checked against
the torch model; NO model file below the validation floor (exit 3, report still written);
exit 2 = not enough labels. `evaluate` scores a shipped model on labels reviewed after
training + the unlabelled pile; `publish` PUTs a version folder to a Gitea generic package.
Light core deps; the `train` extra is heavy — CI syncs without it, torch tests skip.

apps/vision: BodyTypeClassifier (bodytype.onnx + sidecar = the preprocessing contract:
crop margin, input size, RGB 0-255, normalisation inside the graph) and
RefinedVehicleDetector over YOLOX — refines only `car` or a class the classifier trained
on, min-confidence, `detector_class` on the result; path set but no file = phase B off
without an error; a broken file is a health detail. models/bodytype.version (tracked,
empty) pins the published version the Dockerfile fetches at build (BuildKit secret;
a pin that cannot be fetched fails the build). Verified: a trainer model gives identical
probabilities inside the vision service; both images built and smoke-tested.

Delivery: parking-trainer image in build-images.yml, the `trainer` compose profile on the
collector stack (CPU, read-only data, TRAINER_OUT), commented TRAINER_OUT/PUBLISH_TOKEN in
the wash-collector stack, .dockerignore for both Python contexts, trainer deps synced in CI.

Wiki: bodytype-classifier-training rewritten as built (+ one fleet model not per site,
secrets/access, where the crops live), opencv-anpr-service §Phase B, vision-review-outbox,
vision-service-packaging, fleet-deployment-komodo, index, log.

Claude-Session: https://claude.ai/code/session_01FWncR69HgGPuei1dLrW3cU
This commit is contained in:
2026-09-07 11:14:50 +02:00
parent f9cb973fe9
commit f7a262ac9a
41 changed files with 3797 additions and 76 deletions
+7
View File
@@ -0,0 +1,7 @@
.venv/
**/__pycache__/
.pytest_cache/
.mypy_cache/
.ruff_cache/
out/
.env
+12
View File
@@ -0,0 +1,12 @@
# Python
__pycache__/
*.py[cod]
.venv/
.mypy_cache/
.pytest_cache/
.ruff_cache/
# Model weights (fetched at deploy / first run, never committed — can be large + license-scoped)
out/
*.onnx
+1
View File
@@ -0,0 +1 @@
3.12
+43
View File
@@ -0,0 +1,43 @@
# syntax=docker/dockerfile:1.7
# Parking TRAINER image: the phase-B body-type classifier job. Build CONTEXT is apps/trainer
# (self-contained Python package). A ONE-OFF JOB on the reviewer's host (art-docker-station),
# never a booth service: it reads the wash collector's volume (collector.sqlite + crops/)
# and writes a versioned model folder. CPU-only PyTorch — the host has no usable GPU and a
# few thousand crops train in minutes/an hour on four Xeon cores.
# See wiki/decisions/bodytype-classifier-training.md.
FROM ghcr.io/astral-sh/uv:python3.12-bookworm-slim AS base
WORKDIR /app
ENV UV_LINK_MODE=copy \
UV_COMPILE_BYTECODE=1 \
PYTHONUNBUFFERED=1
RUN apt-get update \
&& apt-get install -y --no-install-recommends libgl1 libglib2.0-0 \
&& rm -rf /var/lib/apt/lists/*
COPY pyproject.toml uv.lock .python-version ./
RUN --mount=type=cache,target=/root/.cache/uv \
uv sync --frozen --no-install-project --no-dev --extra train
COPY trainer/ ./trainer/
COPY README.md ./
RUN --mount=type=cache,target=/root/.cache/uv \
uv sync --frozen --no-dev --extra train
# Pre-warm the ImageNet backbone weights INTO the image so a run needs no network (the
# host has one, but a job that fetches at run time is a job that fails at 2 am). Best-effort:
# without network at build time torchvision fetches lazily on the first run.
ENV TORCH_HOME=/app/torch-home
RUN uv run python -c "import torchvision.models as m; m.resnet18(weights=m.ResNet18_Weights.IMAGENET1K_V1); m.mobilenet_v3_small(weights=m.MobileNet_V3_Small_Weights.IMAGENET1K_V1)" \
|| echo "[build] backbone weights not pre-warmed (no network) — fetched on first run"
RUN useradd --system --create-home --uid 999 trainer \
&& mkdir -p /data /out && chown -R trainer:trainer /app /out
USER trainer
ENV TRAINER_DATA_DIR=/data \
TRAINER_OUT_DIR=/out
VOLUME ["/out"]
ENTRYPOINT ["uv", "run", "--no-sync", "parking-trainer"]
CMD ["inspect"]
+36
View File
@@ -0,0 +1,36 @@
# parking-trainer
The phase-B **body-type classifier** job. Reads the wash collector's volume
(`collector.sqlite` + `crops/`), trains a classifier on the reviewer's labels, and writes a
versioned model folder the vision image bakes in — or refuses when validation is below the
floor. Design and decisions: `wiki/decisions/bodytype-classifier-training.md`.
```
parking-trainer inspect --data /data # what a run would train on
parking-trainer train --data /data --out /out # features mode (minutes)
parking-trainer train --mode finetune --epochs 12 ... # full fine-tune (about an hour on 4 cores)
parking-trainer evaluate --model /out/<version>/bodytype.onnx --data /data
parking-trainer publish /out/<version> --url https://git.infra.msai.al/api/packages/mca/generic/parking-bodytype
```
Exit codes: `0` model written · `2` not enough labels · `3` below the floor (report written,
no model) · `1` other.
A passing run writes `<out>/<version>/`:
| file | what |
| --- | --- |
| `bodytype.onnx` | the classifier; input `image` = RGB float32 0–255 `[N,3,S,S]`, output `logits` `[N,K]`; normalisation is inside the graph |
| `bodytype.json` | sidecar: version, class list (in vocabulary order), input size, crop margin, backbone, mode, label counts, validation metrics |
| `report.md` | the human report: accuracy, per-class recall/precision, confusion matrix, dropped classes, loss weights |
| `metrics.json` | the same numbers, machine-readable |
On the reviewer's host (the `wash-collector` stack):
```
docker compose -f docker-compose.collector.yml --profile train run --rm trainer inspect
docker compose -f docker-compose.collector.yml --profile train run --rm trainer train --min-accuracy 0.85
```
Local dev: `uv sync --extra train` (CPU torch, ~200 MB), `uv run pytest -q`. The test suite
runs without the extra (torch tests skip), matching CI.
+13
View File
@@ -0,0 +1,13 @@
{
"name": "@parking/trainer",
"version": "0.0.0",
"private": true,
"//": "Thin shim so this Python job is a node in the Turbo task graph (NOT a JS package — deps are managed by uv/pyproject.toml). It is a one-off job image, never a booth service: see wiki/decisions/bodytype-classifier-training.md.",
"scripts": {
"lint": "uv run ruff check .",
"format": "uv run ruff format .",
"typecheck": "uv run mypy trainer",
"test": "uv run pytest -q",
"build": "echo 'no build step (Python job; see Dockerfile)'"
}
}
+73
View File
@@ -0,0 +1,73 @@
[project]
name = "parking-trainer"
version = "0.0.0"
description = "Phase-B body-type classifier trainer: reviewer labels + crops off the wash collector's volume → an ONNX classifier the vision image bakes in."
requires-python = ">=3.10,<4.0"
# Core deps are LIGHT on purpose (same rule as the vision service): `inspect`, `evaluate`
# and the data/report code run with only these, so `uv sync` and the test suite work
# in CI without the PyTorch stack. Training itself needs the `train` extra.
# See wiki/decisions/bodytype-classifier-training.md.
dependencies = [
"numpy>=1.26",
# OpenCV does the decode + resize on BOTH sides (trainer and vision service): same
# library, same interpolation, same pixels — the preprocessing contract (preprocess.py).
"opencv-python-headless>=4.10",
"onnxruntime>=1.19",
]
[project.scripts]
parking-trainer = "trainer.cli:main"
[project.optional-dependencies]
# The training stack. CPU-only PyTorch (the reviewer's host has no usable GPU — the
# decision is recorded in the wiki page above): resolved from PyTorch's CPU wheel index,
# ~200 MB instead of the ~5 GB CUDA build. Install with: uv sync --extra train
# torch / torchvision are BSD-3; the ImageNet backbone weights ship under the same
# licence (the licence rule applies to weights as much as code).
train = [
"torch>=2.4",
"torchvision>=0.19",
"onnx>=1.16",
"onnxscript>=0.3", # the torch.export-based ONNX exporter (MIT)
]
[dependency-groups]
dev = [
"ruff>=0.8",
"pytest>=8.3",
"mypy>=1.13",
]
[tool.uv]
# Pick the CPU wheels for torch/torchvision from PyTorch's own index; everything else
# from PyPI. `explicit = true` keeps the index from shadowing PyPI for other packages.
[[tool.uv.index]]
name = "pytorch-cpu"
url = "https://download.pytorch.org/whl/cpu"
explicit = true
[tool.uv.sources]
torch = [{ index = "pytorch-cpu" }]
torchvision = [{ index = "pytorch-cpu" }]
[tool.ruff]
line-length = 110
target-version = "py310"
[tool.ruff.lint]
select = ["E", "F", "I", "B", "UP"]
[tool.pytest.ini_options]
testpaths = ["tests"]
[tool.mypy]
python_version = "3.12"
strict = true
ignore_missing_imports = true
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
[tool.hatch.build.targets.wheel]
packages = ["trainer"]
+102
View File
@@ -0,0 +1,102 @@
"""A synthetic collector volume: the collector's `items` table (same DDL as apps/collector
src/db.ts) + JPEG crops. Classes are told apart by COLOUR so even a random-init backbone's
features separate them — the tests check the plumbing (split, floor, export, sidecar),
not accuracy on real cars."""
from __future__ import annotations
import sqlite3
from datetime import datetime, timedelta, timezone
from pathlib import Path
import numpy as np
import pytest
DDL = """
CREATE TABLE IF NOT EXISTS items (
id TEXT PRIMARY KEY, booth TEXT NOT NULL, kind TEXT NOT NULL DEFAULT 'wash',
order_ref TEXT NOT NULL, at TEXT NOT NULL, operator_ref TEXT NOT NULL DEFAULT '',
operator_category_id TEXT NOT NULL DEFAULT '', operator_category_name TEXT NOT NULL DEFAULT '',
operator_classes TEXT NOT NULL DEFAULT '[]', service TEXT NOT NULL, vision_class TEXT NOT NULL,
vision_confidence REAL NOT NULL, vision_category_id TEXT, downgraded INTEGER NOT NULL DEFAULT 0,
image_width INTEGER NOT NULL, image_height INTEGER NOT NULL, plate_blurred INTEGER NOT NULL,
image_path TEXT NOT NULL, received_at TEXT NOT NULL, review_label TEXT, reviewed_at TEXT, reviewer TEXT
);
"""
COLOURS = {"sedan": (200, 40, 40), "suv": (40, 200, 40), "van": (40, 40, 200), "truck": (200, 200, 40)}
def write_jpeg(path: Path, colour: tuple[int, int, int], rng: np.random.Generator) -> None:
import cv2
path.parent.mkdir(parents=True, exist_ok=True)
h, w = int(rng.integers(120, 200)), int(rng.integers(160, 260))
img = np.empty((h, w, 3), np.uint8)
img[:] = colour[::-1] # BGR
noise = rng.integers(-20, 20, size=img.shape, dtype=np.int16)
img = np.clip(img.astype(np.int16) + noise, 0, 255).astype(np.uint8)
cv2.imwrite(str(path), img, [cv2.IMWRITE_JPEG_QUALITY, 85])
@pytest.fixture
def collector_dir(tmp_path: Path) -> Path:
"""40 labelled crops per class for sedan/suv/van, 5 for truck (below the minimum), a few
unusable, a few pending, one labelled row whose file is missing."""
rng = np.random.default_rng(1)
con = sqlite3.connect(tmp_path / "collector.sqlite")
con.executescript(DDL)
t0 = datetime(2026, 9, 1, tzinfo=timezone.utc)
n = 0
def add(label: str | None, reviewed: bool, kind: str = "wash", missing: bool = False) -> None:
nonlocal n
n += 1
item = f"item-{n:04d}"
rel = f"crops/booth-2/{item}.jpg"
colour = COLOURS.get(label or "sedan", (128, 128, 128))
if not missing:
write_jpeg(tmp_path / rel, colour, rng)
at = (t0 + timedelta(minutes=10 * n)).isoformat().replace("+00:00", "Z")
reviewed_at = (
(t0 + timedelta(days=1, minutes=n)).isoformat().replace("+00:00", "Z") if reviewed else None
)
con.execute(
"INSERT INTO items (id, booth, kind, order_ref, at, service, vision_class, vision_confidence, "
"image_width, image_height, plate_blurred, image_path, received_at, review_label, "
"reviewed_at, reviewer) "
"VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)",
(
item,
"booth-2",
kind,
"o",
at,
"wash",
"car" if label != "truck" else "truck",
0.9,
200,
150,
1,
rel,
at,
label if reviewed else None,
reviewed_at,
"reviewer" if reviewed else None,
),
)
# Interleaved in time so every class exists on both sides of the time split.
for i in range(40):
for label in ("sedan", "suv", "van"):
add(label, True)
if i % 8 == 0:
add("truck", True)
add("unusable", True)
add("unusable", True)
add("sedan", True, missing=True)
for _ in range(6):
add(None, False, kind="entry")
con.commit()
con.close()
return tmp_path
+98
View File
@@ -0,0 +1,98 @@
"""Data rules, torch-free: labels, the time split, thin classes, weights, the report."""
from __future__ import annotations
import json
from pathlib import Path
from trainer.cli import main
from trainer.data import (
class_weights,
load_labelled,
load_reviewed_since,
load_unlabelled,
make_split,
summarise,
)
from trainer.preprocess import CROP_MARGIN, Sidecar, load_input
from trainer.report import compute_metrics, render_report
def test_loads_only_reviewed_usable_rows_with_a_crop_on_disk(collector_dir: Path) -> None:
samples, missing = load_labelled(collector_dir)
assert missing == 1 # the labelled row whose file is gone
assert len(samples) == 125 # 3×40 + 5 trucks; unusable and pending excluded
assert all(s.path.is_file() for s in samples)
assert {s.label for s in samples} == {"sedan", "suv", "van", "truck"}
assert summarise(samples)["byClass"] == {"sedan": 40, "suv": 40, "van": 40, "truck": 5}
assert len(load_unlabelled(collector_dir)) == 6
assert len(load_reviewed_since(collector_dir, "2026-09-02T00:00:00Z")) == 125
assert load_reviewed_since(collector_dir, "2030-01-01T00:00:00Z") == []
def test_split_is_by_time_and_drops_thin_classes(collector_dir: Path) -> None:
samples, _ = load_labelled(collector_dir)
split = make_split(samples, val_fraction=0.2, min_per_class=20)
assert split.classes == ("sedan", "suv", "van") # canonical order, truck dropped
assert split.dropped == {"truck": 5}
assert len(split.train) + len(split.val) == 120
assert len(split.val) == 24
assert max(s.at for s in split.train) < min(s.at for s in split.val) # newest = validation
assert all(v > 0 for v in split.counts("val").values())
def test_class_weights_lean_against_imbalance_but_gently(collector_dir: Path) -> None:
samples, _ = load_labelled(collector_dir)
vans = [s for s in samples if s.label == "van"]
keep = set(vans[::10]) # 4 of 40 vans survive
split = make_split([s for s in samples if s.label != "van" or s in keep], 0.2, 3)
w = dict(zip(split.classes, class_weights(split), strict=True))
assert w["van"] > w["sedan"] > 0 # the rare class weighs more
assert w["van"] / w["sedan"] < 4 # but not the full inverse ratio (damped)
assert abs(sum(w.values()) / len(w) - 1.0) < 1e-9
def test_metrics_and_report() -> None:
classes = ("sedan", "suv")
m = compute_metrics(classes, [0, 0, 1, 1], [0, 1, 1, 1], camera=["car"] * 4)
assert m.accuracy == 0.75
assert m.per_class["sedan"].recall == 0.5 and m.per_class["suv"].precision == 2 / 3
assert m.confusion == [[1, 1], [0, 2]]
assert m.camera_agreement == 0.0
text = render_report(
version="v1",
trained_at="t",
mode="features",
backbone="resnet18",
epochs=3,
classes=classes,
train_counts={"sedan": 10, "suv": 8},
val_counts={"sedan": 2, "suv": 2},
dropped={"truck": 2},
missing_files=1,
weights=[0.9, 1.1],
metrics=m,
min_accuracy=0.85,
written=False,
)
assert "MODEL NOT WRITTEN" in text and "| **sedan** | 1 | 1 |" in text and "truck (2)" in text
def test_preprocess_contract(tmp_path: Path, collector_dir: Path) -> None:
samples, _ = load_labelled(collector_dir)
x = load_input(samples[0].path, 32)
assert x.shape == (3, 32, 32) and x.dtype.name == "float32" and 0 <= x.min() and x.max() <= 255
assert x[0].mean() > x[2].mean() # a sedan crop is red: RGB order, not BGR
assert load_input(tmp_path / "nope.jpg", 32) is None
side = Sidecar(version="v1", classes=["sedan", "suv"])
side.write(tmp_path / "s.json")
back = Sidecar.read(tmp_path / "s.json")
assert back == side and back.crop_margin == CROP_MARGIN == 0.08 and back.normalization == "in-graph"
def test_inspect_prints_the_run_shape(collector_dir: Path, capsys) -> None: # type: ignore[no-untyped-def]
assert main(["inspect", "--data", str(collector_dir)]) == 0
out = json.loads(capsys.readouterr().out)
assert out["ready"] is True and out["run"]["classes"] == ["sedan", "suv", "van"]
assert out["run"]["dropped"] == {"truck": 5} and out["missingCrops"] == 1
assert main(["inspect", "--data", str(collector_dir), "--min-per-class", "100"]) == 2
+155
View File
@@ -0,0 +1,155 @@
"""The training job end to end on the synthetic volume — needs the `train` extra (torch);
skipped where it is not installed (CI syncs without it, like the vision service)."""
from __future__ import annotations
import json
from pathlib import Path
import numpy as np
import pytest
torch = pytest.importorskip("torch")
from trainer.cli import main # noqa: E402
from trainer.infer import OnnxClassifier # noqa: E402
from trainer.preprocess import Sidecar # noqa: E402
COMMON = ["--no-pretrained", "--input-size", "64", "--no-cache", "--seed", "3"]
def test_features_run_writes_model_sidecar_report_and_evaluates(
collector_dir: Path, tmp_path: Path, capsys
) -> None: # type: ignore[no-untyped-def]
out = tmp_path / "out"
rc = main(
[
"train",
"--data",
str(collector_dir),
"--out",
str(out),
"--version",
"vtest",
"--mode",
"features",
"--epochs",
"150",
"--min-accuracy",
"0.0",
*COMMON,
]
)
assert rc == 0
d = out / "vtest"
assert {p.name for p in d.iterdir()} == {"bodytype.onnx", "bodytype.json", "report.md", "metrics.json"}
side = Sidecar.read(d / "bodytype.json")
assert side.classes == ["sedan", "suv", "van"] and side.input_size == 64 and side.mode == "features"
assert side.labels == {"train": 96, "val": 24} and side.metrics["floor"] == 0.0
metrics = json.loads((d / "metrics.json").read_text())
assert metrics["n"] == 24 and metrics["onnx_agreement"] == 1.0
# Colour-coded classes: even a random backbone's pooled features separate them.
assert metrics["accuracy"] >= 0.9
report = (d / "report.md").read_text()
assert (
"MODEL WRITTEN" in report
and "truck (5)" in report
and "crop is missing on disk (skipped): 1" in report
)
# The exported graph takes raw 0–255 RGB and answers by itself.
clf = OnnxClassifier(d / "bodytype.onnx")
probs, kept = clf.predict_files(
[s for s in sorted((collector_dir / "crops" / "booth-2").glob("*.jpg"))][:6]
)
assert probs.shape == (6, 3) and kept == [0, 1, 2, 3, 4, 5]
assert np.allclose(probs.sum(axis=1), 1.0, atol=1e-4)
# evaluate: labels reviewed after training (none — the fixture's reviews predate it) and the
# unlabelled pile (6 entry samples).
capsys.readouterr()
assert main(["evaluate", "--data", str(collector_dir), "--model", str(d / "bodytype.onnx")]) == 0
res = json.loads(capsys.readouterr().out)
assert res["model"] == "vtest" and res["reviewedSince"] is None
assert res["unlabelled"]["n"] == 6 and sum(res["unlabelled"]["predicted"].values()) == 6
assert (
main(
[
"evaluate",
"--data",
str(collector_dir),
"--model",
str(d / "bodytype.onnx"),
"--since",
"2026-09-01T00:00:00Z",
]
)
== 0
)
res2 = json.loads(capsys.readouterr().out)
assert res2["reviewedSince"]["n"] == 125 - 5 # trucks are not a class the model knows
def test_below_the_floor_writes_the_report_but_no_model(collector_dir: Path, tmp_path: Path) -> None:
out = tmp_path / "out"
rc = main(
[
"train",
"--data",
str(collector_dir),
"--out",
str(out),
"--version",
"vlow",
"--mode",
"features",
"--epochs",
"5",
"--min-accuracy",
"1.01",
*COMMON,
]
)
assert rc == 3
d = out / "vlow"
assert {p.name for p in d.iterdir()} == {"report.md", "metrics.json"}
assert "MODEL NOT WRITTEN" in (d / "report.md").read_text()
def test_not_enough_labels_is_exit_2(collector_dir: Path, tmp_path: Path) -> None:
out = tmp_path / "out"
rc = main(["train", "--data", str(collector_dir), "--out", str(out), "--min-per-class", "100", *COMMON])
assert rc == 2
assert not out.exists()
def test_finetune_runs_and_uses_the_feature_cache(collector_dir: Path, tmp_path: Path) -> None:
out = tmp_path / "out"
args = [
"train",
"--data",
str(collector_dir),
"--out",
str(out),
"--mode",
"finetune",
"--backbone",
"mobilenet_v3_small",
"--epochs",
"1",
"--batch",
"16",
"--min-accuracy",
"0.0",
"--no-pretrained",
"--input-size",
"64",
"--seed",
"3",
]
assert main([*args, "--version", "vft"]) == 0
cache = out / "cache" / "features-mobilenet_v3_small-64.npz"
assert cache.exists()
z = np.load(cache)
assert len(z["ids"]) == 96 and z["feats"].shape == (96, 576)
assert Sidecar.read(out / "vft" / "bodytype.json").mode == "finetune"
+7
View File
@@ -0,0 +1,7 @@
"""parking-trainer — the phase-B body-type classifier job.
Reads the wash collector's SQLite + crops straight off its volume, splits by TIME, trains a
small classifier on a pretrained backbone, and writes the ONNX model + sidecar + report —
or refuses to write the model when validation is below the owner's floor.
See wiki/decisions/bodytype-classifier-training.md.
"""
+388
View File
@@ -0,0 +1,388 @@
"""parking-trainer — inspect / train / evaluate / publish.
parking-trainer inspect --data /data
parking-trainer train --data /data --out /out [--mode features|finetune] [--min-accuracy 0.85]
parking-trainer evaluate --model /out/<version>/bodytype.onnx --data /data
parking-trainer publish /out/<version> --url https://<gitea>/api/packages/<owner>/generic/parking-bodytype
Exit codes: 0 ok · 2 not enough labels · 3 trained but below the floor (report written, model
NOT written) · 1 anything else.
"""
from __future__ import annotations
import argparse
import json
import os
import sys
import time
import urllib.request
from datetime import datetime, timezone
from pathlib import Path
from .data import (
VEHICLE_CLASSES,
class_weights,
load_labelled,
load_reviewed_since,
load_unlabelled,
make_split,
suggested_epochs,
summarise,
)
from .preprocess import CROP_MARGIN, Sidecar
from .report import compute_metrics, render_report
MODEL_FILE = "bodytype.onnx"
SIDECAR_FILE = "bodytype.json"
REPORT_FILE = "report.md"
METRICS_FILE = "metrics.json"
def _log(msg: str) -> None:
print(f"[trainer] {msg}", file=sys.stderr, flush=True)
def _now() -> str:
return datetime.now(timezone.utc).replace(microsecond=0).isoformat().replace("+00:00", "Z")
# ----------------------------------------------------------------------------------
# inspect
# ----------------------------------------------------------------------------------
def cmd_inspect(a: argparse.Namespace) -> int:
samples, missing = load_labelled(a.data)
split = make_split(samples, a.val_fraction, a.min_per_class)
out = {
"labelled": summarise(samples),
"missingCrops": missing,
"run": {
"classes": list(split.classes),
"train": split.counts("train"),
"val": split.counts("val"),
"dropped": split.dropped,
"minPerClass": a.min_per_class,
"valFraction": a.val_fraction,
},
"ready": len(split.classes) >= 2,
}
print(json.dumps(out, indent=2))
return 0 if out["ready"] else 2
# ----------------------------------------------------------------------------------
# train
# ----------------------------------------------------------------------------------
def cmd_train(a: argparse.Namespace) -> int:
try:
from . import model as M
except ImportError as exc: # torch missing
_log(f"the training stack is not installed ({exc}); install with: uv sync --extra train")
return 1
import numpy as np
samples, missing = load_labelled(a.data)
split = make_split(samples, a.val_fraction, a.min_per_class)
if len(split.classes) < 2:
_log(
f"not enough labels: {len(samples)} usable, classes with >= {a.min_per_class}: "
f"{list(split.classes)} (dropped {split.dropped}); nothing to train"
)
return 2
version = a.version or datetime.now(timezone.utc).strftime("v%Y%m%d-%H%M")
out_dir = a.out / version
epochs = a.epochs or suggested_epochs(len(split.train), a.mode)
weights = class_weights(split)
idx = split.class_index
_log(
f"{version}: {a.mode} on {a.backbone}, classes {list(split.classes)}, "
f"{len(split.train)} train / {len(split.val)} val, {epochs} epochs"
)
t0 = time.perf_counter()
x_train, train = M.load_images(split.train, a.input_size)
x_val, val = M.load_images(split.val, a.input_size)
y_train = [idx[s.label] for s in train]
y_val = [idx[s.label] for s in val]
_log(f"decoded {len(train)} + {len(val)} crops in {time.perf_counter() - t0:.1f}s")
if len(val) == 0 or len(set(y_train)) < 2:
_log("not enough decodable crops on both sides of the split")
return 2
backbone = M.build_backbone(a.backbone, pretrained=not a.no_pretrained)
cache = (
None if a.no_cache else M.FeatureCache(a.out / "cache" / f"features-{a.backbone}-{a.input_size}.npz")
)
t0 = time.perf_counter()
f_train = M.features_for(backbone, train, x_train, cache)
_log(
f"features for {len(train)} train crops in {time.perf_counter() - t0:.1f}s "
f"(cache: {cache.path if cache else 'off'})"
)
head_epochs = epochs if a.mode == "features" else max(30, epochs * 5)
head = M.train_head(f_train, y_train, len(split.classes), weights, head_epochs, seed=a.seed)
net = M.Classifier.make(backbone, head)
if a.mode == "finetune":
t0 = time.perf_counter()
net = M.finetune(
net, x_train, y_train, weights, epochs, batch=a.batch, lr=a.lr, seed=a.seed, log=_log
)
_log(f"fine-tuned in {(time.perf_counter() - t0) / 60:.1f} min")
logits = M.predict_logits(net, x_val)
y_pred = logits.argmax(axis=1).tolist()
metrics = compute_metrics(split.classes, y_val, y_pred, camera=[s.vision_class for s in val])
# Export and check the graph gives the same answers as the torch model.
out_dir.mkdir(parents=True, exist_ok=True)
tmp_model = out_dir / (MODEL_FILE + ".tmp")
M.export_onnx(net, a.input_size, tmp_model)
from .infer import OnnxClassifier
sidecar = Sidecar(
version=version,
classes=list(split.classes),
input_size=a.input_size,
backbone=a.backbone,
mode=a.mode,
trained_at=_now(),
labels={"train": len(train), "val": len(val)},
)
tmp_side = out_dir / (SIDECAR_FILE + ".tmp")
sidecar.write(tmp_side)
onnx_pred = OnnxClassifier(tmp_model, tmp_side).predict_inputs(x_val.astype(np.float32)).argmax(axis=1)
metrics.onnx_agreement = float((onnx_pred == np.array(y_pred)).mean()) if len(y_pred) else None
sidecar.metrics = {
"accuracy": metrics.accuracy,
"macroRecall": metrics.macro_recall,
"perClass": {c: m.__dict__ for c, m in metrics.per_class.items()},
"floor": a.min_accuracy,
}
written = metrics.accuracy >= a.min_accuracy and (metrics.onnx_agreement or 0.0) >= 0.99
notes = []
if metrics.onnx_agreement is not None and metrics.onnx_agreement < 0.99:
notes.append(
f"ONNX export disagrees with the torch model ({metrics.onnx_agreement:.3f}); model withheld"
)
report = render_report(
version=version,
trained_at=sidecar.trained_at,
mode=a.mode,
backbone=a.backbone,
epochs=epochs,
classes=split.classes,
train_counts=split.counts("train"),
val_counts=split.counts("val"),
dropped=split.dropped,
missing_files=missing,
weights=weights,
metrics=metrics,
min_accuracy=a.min_accuracy,
written=written,
notes=notes,
)
(out_dir / REPORT_FILE).write_text(report)
(out_dir / METRICS_FILE).write_text(json.dumps(metrics.to_dict(), indent=2) + "\n")
if written:
sidecar.write(out_dir / SIDECAR_FILE)
tmp_model.replace(out_dir / MODEL_FILE)
tmp_side.unlink(missing_ok=True)
_log(
f"MODEL WRITTEN: {out_dir / MODEL_FILE} "
f"(accuracy {metrics.accuracy:.3f} >= floor {a.min_accuracy})"
)
else:
tmp_model.unlink(missing_ok=True)
tmp_side.unlink(missing_ok=True)
_log(
f"MODEL NOT WRITTEN: accuracy {metrics.accuracy:.3f} < floor {a.min_accuracy}; "
f"see {out_dir / REPORT_FILE}"
)
print(report)
return 0 if written else 3
# ----------------------------------------------------------------------------------
# evaluate — an existing model against labels that arrived AFTER it was trained, and its
# view of the unlabelled pile (the ongoing accuracy check without labelling everything)
# ----------------------------------------------------------------------------------
def cmd_evaluate(a: argparse.Namespace) -> int:
from .infer import OnnxClassifier
clf = OnnxClassifier(a.model)
since = a.since or clf.sidecar.trained_at
classes = clf.classes
result: dict[str, object] = {"model": clf.sidecar.version, "classes": classes, "since": since}
reviewed = [s for s in load_reviewed_since(a.data, since) if s.label in classes]
if reviewed:
probs, kept = clf.predict_files([s.path for s in reviewed])
rows = [reviewed[i] for i in kept]
y_true = [classes.index(s.label) for s in rows]
y_pred = probs.argmax(axis=1).tolist()
m = compute_metrics(classes, y_true, y_pred, camera=[s.vision_class for s in rows])
result["reviewedSince"] = m.to_dict()
else:
result["reviewedSince"] = None
pending = load_unlabelled(a.data, a.limit)
if pending:
probs, kept = clf.predict_files([s.path for s in pending])
rows = [pending[i] for i in kept]
pred = probs.argmax(axis=1)
conf = probs.max(axis=1)
hist = {c: int((pred == i).sum()) for i, c in enumerate(classes)}
result["unlabelled"] = {
"n": len(rows),
"predicted": hist,
"meanConfidence": float(conf.mean()) if len(rows) else None,
"belowHalf": int((conf < 0.5).sum()),
"agreesWithDetector": float(
sum(1 for p, s in zip(pred, rows, strict=True) if classes[int(p)] == s.vision_class)
/ len(rows)
)
if rows
else None,
}
else:
result["unlabelled"] = None
print(json.dumps(result, indent=2))
return 0
# ----------------------------------------------------------------------------------
# publish — the versioned files to a Gitea generic package (weights are not code, they
# do not live in git; the vision image fetches them by URL at build)
# ----------------------------------------------------------------------------------
def cmd_publish(a: argparse.Namespace) -> int:
d: Path = a.dir
files = [d / MODEL_FILE, d / SIDECAR_FILE, d / REPORT_FILE, d / METRICS_FILE]
for f in files[:2]:
if not f.exists():
_log(f"{f} missing — nothing to publish (a run below the floor writes no model)")
return 1
version = Sidecar.read(d / SIDECAR_FILE).version
token = a.token or os.environ.get("TRAINER_PUBLISH_TOKEN", "")
if not token:
_log("no token: pass --token or set TRAINER_PUBLISH_TOKEN")
return 1
base = a.url.rstrip("/") + "/" + version
for f in files:
if not f.exists():
continue
req = urllib.request.Request(f"{base}/{f.name}", data=f.read_bytes(), method="PUT")
req.add_header("Authorization", f"token {token}")
req.add_header("Content-Type", "application/octet-stream")
with urllib.request.urlopen(req, timeout=120) as r:
_log(f"PUT {base}/{f.name} → {r.status}")
_log(f"published {version}; pin it in apps/vision/models/bodytype.version and rebuild the vision image")
return 0
# ----------------------------------------------------------------------------------
def build_parser() -> argparse.ArgumentParser:
p = argparse.ArgumentParser(
prog="parking-trainer", description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter
)
sub = p.add_subparsers(dest="cmd", required=True)
def data_args(sp: argparse.ArgumentParser) -> None:
sp.add_argument(
"--data",
type=Path,
default=Path(os.environ.get("TRAINER_DATA_DIR", "/data")),
help="collector volume (collector.sqlite + crops/)",
)
def split_args(sp: argparse.ArgumentParser) -> None:
sp.add_argument(
"--min-per-class",
type=int,
default=20,
help="classes with fewer reviewed crops are dropped from the run",
)
sp.add_argument(
"--val-fraction", type=float, default=0.2, help="newest fraction held out for validation"
)
i = sub.add_parser("inspect", help="what a run would train on")
data_args(i)
split_args(i)
i.set_defaults(fn=cmd_inspect)
t = sub.add_parser("train", help="train, evaluate, export (or refuse)")
data_args(t)
split_args(t)
t.add_argument(
"--out",
type=Path,
default=Path(os.environ.get("TRAINER_OUT_DIR", "/out")),
help="output root; a <version>/ folder is created under it",
)
t.add_argument("--mode", choices=["features", "finetune"], default="features")
t.add_argument(
"--backbone", choices=["resnet18", "mobilenet_v3_small", "efficientnet_b0"], default="resnet18"
)
t.add_argument("--epochs", type=int, default=0, help="0 = pick from the data size")
t.add_argument("--batch", type=int, default=32)
t.add_argument("--lr", type=float, default=1e-4, help="fine-tune learning rate")
t.add_argument("--input-size", type=int, default=224)
t.add_argument(
"--min-accuracy", type=float, default=0.85, help="validation floor below which NO model is written"
)
t.add_argument("--version", default="", help="model version (default v<date>-<time>)")
t.add_argument("--seed", type=int, default=7)
t.add_argument(
"--no-pretrained", action="store_true", help="random init (tests only — never for a real run)"
)
t.add_argument("--no-cache", action="store_true", help="do not read/write the feature cache")
t.set_defaults(fn=cmd_train)
e = sub.add_parser(
"evaluate", help="an existing model vs labels reviewed after it was trained + the unlabelled pile"
)
data_args(e)
e.add_argument(
"--model", type=Path, required=True, help="path to bodytype.onnx (sidecar .json beside it)"
)
e.add_argument("--since", default="", help="ISO time; default = the model's trained_at")
e.add_argument(
"--limit", type=int, default=2000, help="how many unlabelled crops to score (newest first)"
)
e.set_defaults(fn=cmd_evaluate)
u = sub.add_parser("publish", help="PUT a version folder to a Gitea generic package")
u.add_argument("dir", type=Path, help="the <version>/ folder a passing run wrote")
u.add_argument(
"--url", required=True, help="https://<gitea>/api/packages/<owner>/generic/parking-bodytype"
)
u.add_argument("--token", default="", help="Gitea token with package:write (or TRAINER_PUBLISH_TOKEN)")
u.set_defaults(fn=cmd_publish)
return p
def main(argv: list[str] | None = None) -> int:
a = build_parser().parse_args(argv)
try:
return int(a.fn(a))
except FileNotFoundError as exc:
_log(str(exc))
return 1
__all__ = ["main", "build_parser", "VEHICLE_CLASSES", "CROP_MARGIN"]
if __name__ == "__main__":
sys.exit(main())
+179
View File
@@ -0,0 +1,179 @@
"""The training set: reviewed, usable rows off the collector's SQLite, and how they are
split and weighed. Pure Python + sqlite3 — no torch, so `inspect` and the tests run light.
Rules (wiki/decisions/bodytype-classifier-training.md):
- Only rows a REVIEWER labelled count; the operator's pick and the camera's class are
never labels. `unusable` rows are dropped.
- Split by TIME (`at` = when the vehicle was seen): validation = the newest slice, so
the number reflects tomorrow's traffic rather than a random shuffle of the same days.
- Classes with too few labels are dropped from the run (and reported), never trained
on a handful of examples that would make the softmax confidently wrong.
- Class imbalance is weighed in the loss (inverse frequency, damped) and reported.
"""
from __future__ import annotations
import math
import sqlite3
from collections import Counter
from dataclasses import dataclass
from pathlib import Path
# The shared vocabulary (packages/shared VEHICLE_CLASSES) — the only labels a reviewer can
# give, and the only classes a model may emit. Order is the canonical one; the model's own
# class list (sidecar) is the subset it trained on, in this order.
VEHICLE_CLASSES: tuple[str, ...] = (
"car",
"sedan",
"hatchback",
"suv",
"minivan",
"pickup",
"van",
"truck",
"bus",
"motorcycle",
)
DB_FILE = "collector.sqlite"
@dataclass(frozen=True)
class Sample:
item: str
booth: str
kind: str # wash | entry
at: str # ISO-8601, when the vehicle was seen (the split key)
label: str # the reviewer's class
vision_class: str # what the detector said (for the report, never a label)
path: Path # the crop on disk
@dataclass(frozen=True)
class Split:
classes: tuple[str, ...]
train: list[Sample]
val: list[Sample]
dropped: dict[str, int] # class → count, below the per-class minimum
missing_files: int # labelled rows whose crop is not on disk
@property
def class_index(self) -> dict[str, int]:
return {c: i for i, c in enumerate(self.classes)}
def counts(self, part: str) -> dict[str, int]:
rows = self.train if part == "train" else self.val
c = Counter(s.label for s in rows)
return {k: c.get(k, 0) for k in self.classes}
_SELECT = "SELECT id, booth, kind, at, review_label, vision_class, image_path FROM items "
def _rows(
data_dir: Path, where: str, params: tuple[object, ...], db_file: Path | None = None
) -> list[Sample]:
db = db_file or data_dir / DB_FILE
if not db.exists():
raise FileNotFoundError(f"collector database not found: {db}")
con = sqlite3.connect(f"file:{db}?mode=ro", uri=True)
try:
rows = con.execute(_SELECT + where, params).fetchall()
finally:
con.close()
out: list[Sample] = []
for item, booth, kind, at, label, vision_class, image_path in rows:
out.append(Sample(item, booth, kind, at, label or "", vision_class, data_dir / image_path))
return out
def load_labelled(data_dir: Path, db_file: Path | None = None) -> tuple[list[Sample], int]:
"""Every reviewed, usable row with its crop path resolved. Returns (samples, missing)
where `missing` counts rows whose crop file is gone (pruned/moved) — skipped."""
rows = _rows(
data_dir,
"WHERE reviewed_at IS NOT NULL AND review_label != 'unusable' ORDER BY at, id",
(),
db_file,
)
out: list[Sample] = []
missing = 0
for s in rows:
if s.label not in VEHICLE_CLASSES:
continue # a label outside the vocabulary can only be a future/foreign row
if not s.path.is_file():
missing += 1
continue
out.append(s)
return out, missing
def load_reviewed_since(data_dir: Path, since: str, db_file: Path | None = None) -> list[Sample]:
"""Usable labels whose REVIEW happened after `since` (ISO) — a clean held-out check for a
model trained before then. Rows whose crop is gone are skipped."""
rows = _rows(
data_dir,
"WHERE reviewed_at IS NOT NULL AND review_label != 'unusable' AND reviewed_at > ? "
"ORDER BY reviewed_at, id",
(since,),
db_file,
)
return [s for s in rows if s.label in VEHICLE_CLASSES and s.path.is_file()]
def load_unlabelled(data_dir: Path, limit: int = 2000, db_file: Path | None = None) -> list[Sample]:
"""The pending pile, newest first — what the model would say about traffic nobody has
labelled (its class histogram and detector agreement are the cheap drift check)."""
rows = _rows(data_dir, "WHERE reviewed_at IS NULL ORDER BY received_at DESC LIMIT ?", (limit,), db_file)
return [s for s in rows if s.path.is_file()]
def make_split(samples: list[Sample], val_fraction: float = 0.2, min_per_class: int = 20) -> Split:
"""Drop thin classes, then cut by time: the newest `val_fraction` is validation."""
if not 0.0 < val_fraction < 1.0:
raise ValueError("val_fraction must be in (0, 1)")
counts = Counter(s.label for s in samples)
kept = tuple(c for c in VEHICLE_CLASSES if counts.get(c, 0) >= min_per_class)
dropped = {c: n for c, n in counts.items() if c not in kept}
rows = sorted((s for s in samples if s.label in kept), key=lambda s: (s.at, s.item))
n_val = int(round(len(rows) * val_fraction))
if rows and n_val == 0:
n_val = 1
cut = len(rows) - n_val
return Split(classes=kept, train=rows[:cut], val=rows[cut:], dropped=dropped, missing_files=0)
def class_weights(split: Split, damping: float = 0.5) -> list[float]:
"""Inverse-frequency weights for the loss, damped by `damping` (0.5 = square root, so a
1:9 imbalance becomes 1:3 rather than 1:9 — full inverse weights over-correct on small
sets). Normalised to mean 1 so the learning rate keeps its meaning."""
counts = split.counts("train")
total = sum(counts.values())
k = len(split.classes)
raw = [(total / (k * max(1, counts[c]))) ** damping for c in split.classes]
mean = sum(raw) / max(1, len(raw))
return [w / mean for w in raw]
def summarise(samples: list[Sample]) -> dict[str, object]:
"""What `inspect` prints: per-class counts, per-booth counts, time range."""
by_class = Counter(s.label for s in samples)
by_booth = Counter(s.booth for s in samples)
by_kind = Counter(s.kind for s in samples)
ats = sorted(s.at for s in samples)
return {
"total": len(samples),
"byClass": {c: by_class.get(c, 0) for c in VEHICLE_CLASSES if by_class.get(c, 0)},
"byBooth": dict(sorted(by_booth.items())),
"byKind": dict(sorted(by_kind.items())),
"from": ats[0] if ats else None,
"to": ats[-1] if ats else None,
}
def suggested_epochs(n_train: int, mode: str) -> int:
"""A sane default when the owner gives none: enough passes for a small set, fewer as
it grows. Feature-extraction heads converge fast; fine-tunes need more but cost more."""
if mode == "features":
return int(min(200, max(30, 4000 / max(1, n_train) * 10)))
return int(min(30, max(8, math.ceil(3000 / max(1, n_train)) * 4)))
+51
View File
@@ -0,0 +1,51 @@
"""Run an exported classifier (ONNX + sidecar) — torch-free. Used by `evaluate` and by
the tests; the vision service carries its own, equivalent, reader (vehicle.py)."""
from __future__ import annotations
from pathlib import Path
from typing import Any
from .preprocess import Sidecar, load_input, softmax
class OnnxClassifier:
def __init__(self, model_path: Path, sidecar_path: Path | None = None) -> None:
import onnxruntime as ort
self.model_path = Path(model_path)
self.sidecar = Sidecar.read(sidecar_path or self.model_path.with_suffix(".json"))
opts = ort.SessionOptions()
opts.intra_op_num_threads = 2
self._session = ort.InferenceSession(
str(self.model_path), sess_options=opts, providers=["CPUExecutionProvider"]
)
self._input = self._session.get_inputs()[0].name
@property
def classes(self) -> list[str]:
return list(self.sidecar.classes)
def predict_inputs(self, x: Any, batch: int = 64) -> Any:
"""[N,3,S,S] float32 → probabilities [N,K]."""
import numpy as np
outs = []
for i in range(0, len(x), batch):
logits = self._session.run(None, {self._input: x[i : i + batch]})[0]
outs.append(softmax(logits))
return np.concatenate(outs, axis=0) if outs else np.zeros((0, len(self.classes)), np.float32)
def predict_files(self, paths: list[Path], batch: int = 64) -> tuple[Any, list[int]]:
"""Decode + classify crop files. Returns (probs, indices of paths that decoded)."""
import numpy as np
xs, kept = [], []
for i, p in enumerate(paths):
x = load_input(p, self.sidecar.input_size)
if x is not None:
xs.append(x)
kept.append(i)
if not xs:
return np.zeros((0, len(self.classes)), np.float32), []
return self.predict_inputs(np.stack(xs), batch), kept
+305
View File
@@ -0,0 +1,305 @@
"""The network and the two ways of training it. Imports torch — only `train` reaches here;
everything else in the package stays torch-free (the `train` extra is heavy).
Backbone: a small ImageNet-pretrained torchvision model (BSD-3, weights included), used as
a feature extractor; head: one linear layer over its pooled features.
- "features" mode: the backbone is FROZEN. Every crop goes through it once, the vectors
are cached on disk (keyed by item id), and only the head is trained — minutes for a few
thousand crops, seconds to retrain when new labels arrive. Expected to carry most of
the accuracy on frontal gate views.
- "finetune" mode: warm-starts the head the same way, then unfreezes everything and trains
end to end with light augmentation. Roughly an hour on four Xeon cores for a few
thousand crops with a mobile-sized backbone; the step when the cheap mode plateaus.
The exported ONNX graph takes raw RGB 0–255 float pixels and normalises INSIDE
(see preprocess.py), so the vision service cannot get the constants wrong.
"""
from __future__ import annotations
import os
from dataclasses import dataclass
from pathlib import Path
from typing import Any
from .data import Sample
from .preprocess import IMAGENET_MEAN, IMAGENET_STD, load_input
BACKBONES: dict[str, int] = {"resnet18": 512, "mobilenet_v3_small": 576, "efficientnet_b0": 1280}
def _torch() -> Any:
import torch
torch.set_num_threads(max(1, os.cpu_count() or 1))
return torch
def build_backbone(name: str, pretrained: bool = True) -> Any:
"""torchvision model with its classifier removed → pooled feature vector."""
torch = _torch()
import torchvision.models as tvm
if name not in BACKBONES:
raise ValueError(f"unknown backbone {name!r} (choose from {', '.join(BACKBONES)})")
if name == "resnet18":
m = tvm.resnet18(weights=tvm.ResNet18_Weights.IMAGENET1K_V1 if pretrained else None)
m.fc = torch.nn.Identity()
elif name == "mobilenet_v3_small":
m = tvm.mobilenet_v3_small(
weights=tvm.MobileNet_V3_Small_Weights.IMAGENET1K_V1 if pretrained else None
)
m.classifier = torch.nn.Identity()
else:
m = tvm.efficientnet_b0(weights=tvm.EfficientNet_B0_Weights.IMAGENET1K_V1 if pretrained else None)
m.classifier = torch.nn.Identity()
return m
class Classifier: # a factory, not a Module subclass at import time (torch is lazy)
@staticmethod
def make(backbone: Any, head: Any) -> Any:
torch = _torch()
class _Net(torch.nn.Module): # type: ignore[misc,name-defined]
def __init__(self) -> None:
super().__init__()
self.backbone = backbone
self.head = head
self.register_buffer("mean", torch.tensor(IMAGENET_MEAN).view(1, 3, 1, 1))
self.register_buffer("std", torch.tensor(IMAGENET_STD).view(1, 3, 1, 1))
def forward(self, x: Any) -> Any:
x = (x / 255.0 - self.mean) / self.std
return self.head(self.backbone(x))
return _Net()
# ----------------------------------------------------------------------------------
# Images in memory (uint8 — a few thousand 224² crops is a few hundred MB; float32 would
# be four times that)
# ----------------------------------------------------------------------------------
def load_images(samples: list[Sample], input_size: int) -> tuple[Any, list[Sample]]:
"""Decode + resize every crop once. Returns (uint8 [N,3,S,S], the samples that decoded)."""
import numpy as np
xs, kept = [], []
for s in samples:
x = load_input(s.path, input_size)
if x is None:
continue
xs.append(x.astype(np.uint8))
kept.append(s)
if not xs:
return np.zeros((0, 3, input_size, input_size), np.uint8), []
return np.stack(xs), kept
def _batches(n: int, batch: int) -> list[slice]:
return [slice(i, min(n, i + batch)) for i in range(0, n, batch)]
# ----------------------------------------------------------------------------------
# Feature extraction (+ on-disk cache) and the head
# ----------------------------------------------------------------------------------
@dataclass
class FeatureCache:
"""`<cache_dir>/features-<backbone>-<size>.npz`: item ids + vectors. Retraining the head
after new labels arrive only runs the backbone on the NEW crops."""
path: Path
def load(self) -> dict[str, Any]:
import numpy as np
if not self.path.exists():
return {}
z = np.load(self.path, allow_pickle=False)
return dict(zip(z["ids"].tolist(), z["feats"], strict=True))
def save(self, table: dict[str, Any]) -> None:
import numpy as np
self.path.parent.mkdir(parents=True, exist_ok=True)
ids = np.array(list(table), dtype=str)
feats = np.stack(list(table.values())) if table else np.zeros((0, 0), np.float32)
np.savez(self.path, ids=ids, feats=feats)
def extract_features(backbone: Any, x_u8: Any, batch: int = 64) -> Any:
"""Frozen forward pass → [N,D] float32 (normalisation applied here, as in the graph)."""
torch = _torch()
import numpy as np
net = Classifier.make(backbone, torch.nn.Identity()).eval()
out = []
with torch.no_grad():
for sl in _batches(len(x_u8), batch):
xb = torch.from_numpy(x_u8[sl]).float()
out.append(net(xb).numpy())
return np.concatenate(out, axis=0) if out else np.zeros((0, 0), np.float32)
def features_for(
backbone: Any, samples: list[Sample], x_u8: Any, cache: FeatureCache | None, batch: int = 64
) -> Any:
"""Feature vectors for `samples` (aligned with x_u8), from the cache where present."""
import numpy as np
table = cache.load() if cache else {}
todo = [i for i, s in enumerate(samples) if s.item not in table]
if todo:
fresh = extract_features(backbone, x_u8[todo], batch)
for i, f in zip(todo, fresh, strict=True):
table[samples[i].item] = f.astype(np.float32)
if cache:
cache.save(table)
return np.stack([table[s.item] for s in samples]) if samples else np.zeros((0, 0), np.float32)
def train_head(
feats: Any,
y: list[int],
n_classes: int,
weights: list[float],
epochs: int,
lr: float = 1e-3,
seed: int = 7,
) -> Any:
"""Multinomial logistic regression on cached features (full batch, Adam, weighted CE)."""
torch = _torch()
torch.manual_seed(seed)
f = torch.from_numpy(feats).float()
t = torch.tensor(y, dtype=torch.long)
head = torch.nn.Linear(f.shape[1], n_classes)
opt = torch.optim.Adam(head.parameters(), lr=lr, weight_decay=1e-4)
loss_fn = torch.nn.CrossEntropyLoss(weight=torch.tensor(weights, dtype=torch.float32))
head.train()
for _ in range(epochs):
opt.zero_grad()
loss = loss_fn(head(f), t)
loss.backward()
opt.step()
return head.eval()
# ----------------------------------------------------------------------------------
# Full fine-tune
# ----------------------------------------------------------------------------------
def _augment(xb: Any) -> Any:
"""Light, label-preserving augmentation on a float batch [B,3,S,S] (0–255): horizontal
flip (a gate view mirrored is still the same body type), a mild random zoom, and
brightness/contrast jitter (dusk, headlights, wet tarmac)."""
torch = _torch()
b, _, s, _ = xb.shape
flip = torch.rand(b) < 0.5
xb = torch.where(flip.view(b, 1, 1, 1), xb.flip(-1), xb)
# zoom: crop a random 85–100 % window and resize back
out = torch.empty_like(xb)
for i in range(b):
frac = float(torch.empty(1).uniform_(0.85, 1.0))
w = max(8, int(s * frac))
x0 = int(torch.randint(0, s - w + 1, (1,)))
y0 = int(torch.randint(0, s - w + 1, (1,)))
crop = xb[i : i + 1, :, y0 : y0 + w, x0 : x0 + w]
out[i : i + 1] = torch.nn.functional.interpolate(
crop, size=(s, s), mode="bilinear", align_corners=False
)
bright = torch.empty(b, 1, 1, 1).uniform_(-25, 25)
contrast = torch.empty(b, 1, 1, 1).uniform_(0.8, 1.2)
mean = out.mean(dim=(1, 2, 3), keepdim=True)
out = (out - mean) * contrast + mean + bright
return out.clamp_(0, 255)
def finetune(
model: Any,
x_u8: Any,
y: list[int],
weights: list[float],
epochs: int,
batch: int = 32,
lr: float = 1e-4,
seed: int = 7,
log: Any = None,
) -> Any:
torch = _torch()
import numpy as np
torch.manual_seed(seed)
rng = np.random.default_rng(seed)
t = torch.tensor(y, dtype=torch.long)
opt = torch.optim.AdamW(model.parameters(), lr=lr, weight_decay=1e-2)
steps = epochs * max(1, (len(y) + batch - 1) // batch)
sched = torch.optim.lr_scheduler.OneCycleLR(opt, max_lr=lr, total_steps=max(1, steps), pct_start=0.15)
loss_fn = torch.nn.CrossEntropyLoss(weight=torch.tensor(weights, dtype=torch.float32))
for epoch in range(epochs):
model.train()
order = rng.permutation(len(y))
total = 0.0
for sl in _batches(len(y), batch):
idx = order[sl]
xb = _augment(torch.from_numpy(x_u8[idx]).float())
opt.zero_grad()
loss = loss_fn(model(xb), t[idx])
loss.backward()
opt.step()
sched.step()
total += float(loss.detach()) * len(idx)
if log:
log(f"epoch {epoch + 1}/{epochs} loss {total / max(1, len(y)):.4f}")
return model.eval()
def predict_logits(model: Any, x_u8: Any, batch: int = 64) -> Any:
torch = _torch()
import numpy as np
model.eval()
out = []
with torch.no_grad():
for sl in _batches(len(x_u8), batch):
out.append(model(torch.from_numpy(x_u8[sl]).float()).numpy())
return np.concatenate(out, axis=0) if out else np.zeros((0, 0), np.float32)
def export_onnx(model: Any, input_size: int, path: Path) -> None:
"""Export with the current (torch.export-based) exporter; fall back to the legacy
TorchScript one where the new path is unavailable or trips over an op. Whichever wrote
the graph, the job then checks it against the torch model (onnx_agreement) before the
file is kept."""
torch = _torch()
model.eval()
dummy = torch.zeros(1, 3, input_size, input_size)
names: dict[str, Any] = dict(input_names=["image"], output_names=["logits"])
try:
torch.onnx.export(
model,
(dummy,),
str(path),
dynamo=True,
dynamic_shapes={"x": {0: "batch"}},
opset_version=18,
external_data=False, # ONE file: the vision image bakes bodytype.onnx + .json, nothing else
verbose=False,
**names,
)
except Exception: # noqa: BLE001 - any failure → the legacy exporter
torch.onnx.export(
model,
dummy,
str(path),
dynamo=False,
dynamic_axes={"image": {0: "batch"}, "logits": {0: "batch"}},
opset_version=17,
**names,
)
+88
View File
@@ -0,0 +1,88 @@
"""Crop → model input. THE CONTRACT between the trainer and the vision service's classifier
stage: what the network sees at training time must be exactly what it sees on the booth.
The trainer does not share code with the vision service (different packages, different
images), so the contract is DATA: every constant here is written into the model's sidecar
(`bodytype.json`) and the vision side reads and applies them from there — nothing is
assumed on either side. Both use OpenCV with the same interpolation so the pixels match.
- input: the collector's crop (the detector's vehicle box + margin, plate blurred), or on
the booth the same cut made live from the frame (vehicle.py mirrors `makeReviewCrop`).
- resize: squash to input_size × input_size with INTER_AREA (the crop IS the vehicle; no
centre-crop that would lose a bumper or a roofline — the shape is the signal).
- colour: RGB, float32, 0–255. Normalisation (/255, ImageNet mean/std) lives INSIDE the
ONNX graph, so a consumer feeds raw pixels and cannot get the constants wrong.
"""
from __future__ import annotations
import json
from dataclasses import asdict, dataclass, field
from pathlib import Path
from typing import Any
SIDECAR_FORMAT = "parking-bodytype/1"
IMAGENET_MEAN = (0.485, 0.456, 0.406)
IMAGENET_STD = (0.229, 0.224, 0.225)
CROP_MARGIN = 0.08 # must equal CROP_MARGIN in apps/server review-outbox.ts
@dataclass
class Sidecar:
"""`bodytype.json` beside `bodytype.onnx`."""
version: str
classes: list[str]
input_size: int = 224
color: str = "rgb"
resize: str = "area"
crop_margin: float = CROP_MARGIN
normalization: str = "in-graph" # the ONNX divides by 255 and applies mean/std itself
mean: list[float] = field(default_factory=lambda: list(IMAGENET_MEAN))
std: list[float] = field(default_factory=lambda: list(IMAGENET_STD))
backbone: str = ""
mode: str = ""
trained_at: str = ""
labels: dict[str, int] = field(default_factory=dict) # train / val counts
metrics: dict[str, Any] = field(default_factory=dict) # accuracy, macro, per-class
format: str = SIDECAR_FORMAT
def write(self, path: Path) -> None:
path.write_text(json.dumps(asdict(self), indent=2) + "\n")
@classmethod
def read(cls, path: Path) -> Sidecar:
d = json.loads(path.read_text())
if d.get("format") != SIDECAR_FORMAT:
raise ValueError(f"{path}: unknown sidecar format {d.get('format')!r}")
known = {f for f in cls.__dataclass_fields__}
return cls(**{k: v for k, v in d.items() if k in known})
def load_input(path: Path, input_size: int) -> Any:
"""Decode a crop and produce the network input: RGB float32 CHW, 0–255, squashed to
input_size. Returns None when the file cannot be decoded."""
import cv2
img = cv2.imread(str(path), cv2.IMREAD_COLOR)
if img is None:
return None
return array_to_input(img, input_size)
def array_to_input(bgr: Any, input_size: int) -> Any:
"""BGR uint8 HWC (OpenCV's native) → RGB float32 CHW 0–255 at input_size."""
import cv2
import numpy as np
resized = cv2.resize(bgr, (input_size, input_size), interpolation=cv2.INTER_AREA)
rgb = cv2.cvtColor(resized, cv2.COLOR_BGR2RGB)
return np.ascontiguousarray(rgb.transpose(2, 0, 1).astype(np.float32))
def softmax(logits: Any) -> Any:
import numpy as np
z = logits - logits.max(axis=-1, keepdims=True)
e = np.exp(z)
return e / e.sum(axis=-1, keepdims=True)
+152
View File
@@ -0,0 +1,152 @@
"""Metrics + the human-readable report. The owner reads this BEFORE anything ships; the
floor decision is made on these numbers. Pure Python, no torch."""
from __future__ import annotations
from dataclasses import asdict, dataclass, field
from typing import Any
@dataclass
class ClassMetrics:
support: int
recall: float # of the true members, how many the model caught
precision: float # of the model's picks, how many were right
@dataclass
class Metrics:
n: int
accuracy: float
macro_recall: float
per_class: dict[str, ClassMetrics]
confusion: list[list[int]] # rows = true class, cols = predicted, in `classes` order
camera_agreement: float | None = None # how often the model equals the detector's class
onnx_agreement: float | None = None # exported graph vs the torch model, argmax
extra: dict[str, Any] = field(default_factory=dict)
def to_dict(self) -> dict[str, Any]:
return asdict(self)
def compute_metrics(
classes: tuple[str, ...] | list[str],
y_true: list[int],
y_pred: list[int],
camera: list[str] | None = None,
) -> Metrics:
k = len(classes)
conf = [[0] * k for _ in range(k)]
for t, p in zip(y_true, y_pred, strict=True):
conf[t][p] += 1
per: dict[str, ClassMetrics] = {}
recalls: list[float] = []
for i, c in enumerate(classes):
support = sum(conf[i])
tp = conf[i][i]
picked = sum(conf[r][i] for r in range(k))
recall = tp / support if support else 0.0
precision = tp / picked if picked else 0.0
per[c] = ClassMetrics(support=support, recall=recall, precision=precision)
if support:
recalls.append(recall)
n = len(y_true)
acc = sum(1 for t, p in zip(y_true, y_pred, strict=True) if t == p) / n if n else 0.0
agree = None
if camera is not None and n:
agree = sum(1 for p, cam in zip(y_pred, camera, strict=True) if classes[p] == cam) / n
return Metrics(
n=n,
accuracy=acc,
macro_recall=sum(recalls) / len(recalls) if recalls else 0.0,
per_class=per,
confusion=conf,
camera_agreement=agree,
)
def _pct(x: float | None) -> str:
return "—" if x is None else f"{x * 100:.1f} %"
def render_report(
*,
version: str,
trained_at: str,
mode: str,
backbone: str,
epochs: int,
classes: tuple[str, ...] | list[str],
train_counts: dict[str, int],
val_counts: dict[str, int],
dropped: dict[str, int],
missing_files: int,
weights: list[float],
metrics: Metrics,
min_accuracy: float,
written: bool,
notes: list[str] | None = None,
) -> str:
lines: list[str] = []
verdict = "MODEL WRITTEN" if written else "MODEL NOT WRITTEN — below the floor"
lines.append(f"# Body-type classifier {version}")
lines.append("")
lines.append(
f"**{verdict}** · validation accuracy {_pct(metrics.accuracy)} vs floor {_pct(min_accuracy)}"
)
lines.append("")
lines.append(f"- trained: {trained_at}")
lines.append(f"- mode: {mode} · backbone: {backbone} · epochs: {epochs}")
lines.append(f"- classes ({len(classes)}): {', '.join(classes)}")
lines.append(
f"- labels: {sum(train_counts.values())} train · {sum(val_counts.values())} validation "
"(validation = the NEWEST slice, by time seen)"
)
if dropped:
lines.append(
"- dropped (too few labels this run): "
+ ", ".join(f"{c} ({n})" for c, n in sorted(dropped.items()))
)
if missing_files:
lines.append(f"- labelled rows whose crop is missing on disk (skipped): {missing_files}")
lines.append("")
lines.append("## Validation")
lines.append("")
lines.append(f"- accuracy: {_pct(metrics.accuracy)} on {metrics.n} crops")
lines.append(f"- macro recall (each class counted equally): {_pct(metrics.macro_recall)}")
if metrics.camera_agreement is not None:
lines.append(
f"- agrees with the detector's coarse class: {_pct(metrics.camera_agreement)} "
"(informational — the detector only knows car/truck/bus/motorcycle)"
)
if metrics.onnx_agreement is not None:
lines.append(
f"- exported ONNX matches the trained model on validation: {_pct(metrics.onnx_agreement)}"
)
lines.append("")
lines.append("| class | train | val | recall | precision | loss weight |")
lines.append("|---|---:|---:|---:|---:|---:|")
for c, w in zip(classes, weights, strict=True):
m = metrics.per_class[c]
lines.append(
f"| {c} | {train_counts.get(c, 0)} | {m.support} | {_pct(m.recall) if m.support else '—'} | "
f"{_pct(m.precision) if m.support else '—'} | {w:.2f} |"
)
lines.append("")
lines.append("## Confusion (rows = reviewer's label, columns = model)")
lines.append("")
lines.append("| | " + " | ".join(classes) + " |")
lines.append("|---|" + "---:|" * len(classes))
for c, row in zip(classes, metrics.confusion, strict=True):
lines.append(f"| **{c}** | " + " | ".join(str(n) for n in row) + " |")
lines.append("")
if notes:
lines.append("## Notes")
lines.append("")
lines.extend(f"- {n}" for n in notes)
lines.append("")
lines.append(
"The flag on the booth records, it never bills: even a model that passes the floor is "
"advisory (the site threshold gates the flag)."
)
return "\n".join(lines) + "\n"
+9
View File
@@ -0,0 +1,9 @@
{
"$schema": "https://turbo.build/schema.json",
"extends": ["//"],
"tasks": {
"build": {
"outputs": []
}
}
}
+1444
View File
File diff suppressed because it is too large Load Diff
+9
View File
@@ -0,0 +1,9 @@
.venv/
**/__pycache__/
.pytest_cache/
.mypy_cache/
.ruff_cache/
.env
# weights are fetched inside the build (yolox) or pinned by models/bodytype.version
models/*
!models/bodytype.version
+8
View File
@@ -29,3 +29,11 @@ VISION_MIN_CONFIDENCE=0.5
# VISION_VEHICLE_MODEL_PATH=models/yolox_s.onnx
# VISION_VEHICLE_INPUT_SIZE=640
# VISION_VEHICLE_MIN_CONFIDENCE=0.4
# Phase B: the body-type classifier (sedan/hatchback/suv/… on the detector's crop), trained
# by apps/trainer on the reviewer's labels. The Docker image bakes it at
# /app/models/bodytype.onnx (+ .json sidecar) when models/bodytype.version pins a published
# version; locally copy a trainer output folder's two files into apps/vision/models/.
# Path set but no file = stage off (the normal state before the first model).
# VISION_VEHICLE_CLASSIFIER_PATH=models/bodytype.onnx
# VISION_VEHICLE_CLASSIFIER_MIN_CONFIDENCE=0.6
+2 -1
View File
@@ -7,5 +7,6 @@ __pycache__/
.ruff_cache/
# Model weights (fetched at deploy / first run, never committed — can be large + license-scoped)
models/
models/*
!models/bodytype.version
*.onnx
+20 -1
View File
@@ -33,6 +33,24 @@ ARG YOLOX_URL=https://github.com/Megvii-BaseDetection/YOLOX/releases/download/0.
RUN mkdir -p /app/models \
&& (curl -fsSL -o /app/models/yolox_s.onnx "$YOLOX_URL" \
|| (echo "[build] yolox weights not fetched (no network) — vehicle stage off" && rm -f /app/models/yolox_s.onnx))
# Phase B body-type classifier (apps/trainer output, published to the Gitea generic package
# registry — weights are not code, they never live in git). models/bodytype.version PINS the
# version this image carries: empty = no classifier (phase B off). A pinned version that
# cannot be fetched FAILS the build — the image must carry what git says it carries. The
# registry may need auth: pass a BuildKit secret `bodytype_auth` holding "user:token".
ARG BODYTYPE_BASE_URL=https://git.infra.msai.al/api/packages/mca/generic/parking-bodytype
COPY models/bodytype.version ./models/bodytype.version
RUN --mount=type=secret,id=bodytype_auth \
v="$(tr -d '[:space:]' < /app/models/bodytype.version)"; \
if [ -n "$v" ]; then \
cfg=/tmp/curl.cfg; : > "$cfg"; \
[ -f /run/secrets/bodytype_auth ] && printf 'user = "%s"\n' "$(cat /run/secrets/bodytype_auth)" > "$cfg"; \
curl -fsSL -K "$cfg" -o /app/models/bodytype.onnx "$BODYTYPE_BASE_URL/$v/bodytype.onnx" \
&& curl -fsSL -K "$cfg" -o /app/models/bodytype.json "$BODYTYPE_BASE_URL/$v/bodytype.json" \
&& echo "[build] bodytype classifier $v baked" \
|| { echo "[build] bodytype classifier $v could not be fetched"; rm -f "$cfg"; exit 1; }; \
rm -f "$cfg"; \
else echo "[build] no bodytype version pinned — phase B off"; fi
RUN --mount=type=cache,target=/root/.cache/uv \
uv sync --frozen --extra alpr
@@ -57,7 +75,8 @@ RUN uv run python -c "from fast_alpr import ALPR; ALPR()" \
ENV VISION_RECOGNIZER=stub \
VISION_HOST=0.0.0.0 \
VISION_PORT=8089 \
VISION_VEHICLE_MODEL_PATH=/app/models/yolox_s.onnx
VISION_VEHICLE_MODEL_PATH=/app/models/yolox_s.onnx \
VISION_VEHICLE_CLASSIFIER_PATH=/app/models/bodytype.onnx
EXPOSE 8089
HEALTHCHECK --interval=30s --timeout=5s --start-period=20s --retries=3 \
CMD python -c "import urllib.request,sys; sys.exit(0 if urllib.request.urlopen('http://localhost:8089/health').status==200 else 1)" || exit 1
View File
+132
View File
@@ -143,3 +143,135 @@ def test_app_reports_a_missing_model_file_and_keeps_serving() -> None:
assert res.json()["vehicle"] is None
finally:
os.environ.pop("VISION_VEHICLE_MODEL_PATH", None)
# ----------------------------------------------------------------------------------
# Phase B: the classifier stage over the detector
# ----------------------------------------------------------------------------------
def test_crop_vehicle_adds_the_margin_clamps_and_blurs_the_plate() -> None:
cv2 = pytest.importorskip("cv2")
from vision_service.vehicle import crop_vehicle
frame = np.zeros((100, 200, 3), dtype=np.uint8)
frame[40:50, 90:110] = (0, 255, 0) # a green "plate"
box = BBox(x1=50, y1=20, x2=150, y2=80) # 100×60 → 8 % margin = 8 / 5 px
crop = crop_vehicle(frame, box, None, 0.08)
assert crop.shape == (70, 116, 3)
edge = crop_vehicle(frame, BBox(x1=0, y1=0, x2=100, y2=60), None, 0.08)
assert edge.shape == (65, 108, 3) # clamped at the frame's top-left
assert crop_vehicle(frame, BBox(x1=10, y1=10, x2=12, y2=12), None, 0.08) is None
blurred = crop_vehicle(frame, box, BBox(x1=90, y1=40, x2=110, y2=50), 0.08)
strip = blurred[40 - 20 + 5 : 50 - 20 + 5, 90 - 50 + 8 : 110 - 50 + 8, 1] # plate strip, green channel
assert strip.mean() < 200 and crop[20 + 5 : 30 + 5, 40 + 8 : 60 + 8, 1].mean() == 255
assert cv2 is not None
class FakeClassifier:
ready = True
error = None
min_confidence = 0.6
model_version = "bodytype:vfake"
def __init__(self, classes: list[str], answer: tuple[str, float] | None) -> None:
self.classes = classes
self.answer = answer
self.calls = 0
def classify(self, frame, box, plate): # type: ignore[no-untyped-def]
self.calls += 1
if isinstance(self.answer, Exception):
raise self.answer
return self.answer
class FrameDetector:
"""A detector that answers on decoded frames (like YOLOX) with a fixed result."""
model_version = "det"
ready = True
error = None
def __init__(self, result: VehicleResult | None) -> None:
self.result = result
def detect_frame(self, frame, plate): # type: ignore[no-untyped-def]
return self.result
def detect(self, image_bytes: bytes, plate: BBox | None) -> VehicleResult | None:
raise AssertionError("the refined stage should share the decoded frame")
def _jpeg() -> bytes:
cv2 = pytest.importorskip("cv2")
ok, buf = cv2.imencode(".jpg", np.zeros((60, 80, 3), dtype=np.uint8))
assert ok
return bytes(buf)
def test_refined_detector_replaces_car_when_confident_else_keeps_the_detector() -> None:
from vision_service.vehicle import RefinedVehicleDetector
car = VehicleResult(body_type="car", confidence=0.85, bbox=BBox(x1=10, y1=10, x2=70, y2=50))
sure = FakeClassifier(["sedan", "suv"], ("suv", 0.91))
res = RefinedVehicleDetector(FrameDetector(car), sure).detect(_jpeg(), None)
assert (
res is not None and res.body_type == "suv" and res.confidence == 0.91 and res.detector_class == "car"
)
assert res.bbox == car.bbox
unsure = FakeClassifier(["sedan", "suv"], ("suv", 0.4))
res2 = RefinedVehicleDetector(FrameDetector(car), unsure).detect(_jpeg(), None)
assert (
res2 is not None
and res2.body_type == "car"
and res2.confidence == 0.85
and res2.detector_class == "car"
)
# A class the classifier never trained on is left alone (its softmax means nothing there).
bus = VehicleResult(body_type="bus", confidence=0.9, bbox=car.bbox)
skip = FakeClassifier(["sedan", "suv"], ("suv", 0.99))
res3 = RefinedVehicleDetector(FrameDetector(bus), skip).detect(_jpeg(), None)
assert res3 == bus and skip.calls == 0
# …unless it was: a classifier that knows trucks may override a truck.
knows = FakeClassifier(["sedan", "truck", "van"], ("van", 0.8))
truck = VehicleResult(body_type="truck", confidence=0.7, bbox=car.bbox)
res4 = RefinedVehicleDetector(FrameDetector(truck), knows).detect(_jpeg(), None)
assert res4 is not None and res4.body_type == "van" and res4.detector_class == "truck"
def test_refined_detector_survives_a_broken_classifier_and_reports_it() -> None:
from vision_service.vehicle import RefinedVehicleDetector
car = VehicleResult(body_type="car", confidence=0.85, bbox=BBox(x1=10, y1=10, x2=70, y2=50))
boom = FakeClassifier(["sedan"], RuntimeError("bad graph")) # type: ignore[arg-type]
ref = RefinedVehicleDetector(FrameDetector(car), boom)
assert ref.detect(_jpeg(), None) == car
assert ref.error == "classifier: RuntimeError: bad graph"
assert ref.ready is True and ref.model_version == "det+bodytype:vfake"
# No box, or a detector that found nothing → nothing to classify.
assert RefinedVehicleDetector(FrameDetector(None), boom).detect(_jpeg(), None) is None
boxless = VehicleResult(body_type="car", confidence=0.85)
assert RefinedVehicleDetector(FrameDetector(boxless), boom).detect(_jpeg(), None) == boxless
def test_classifier_without_files_is_not_ready_and_the_factory_skips_a_missing_model(tmp_path) -> None: # type: ignore[no-untyped-def]
from vision_service.recognizer import WithVehicle, build_recognizer
from vision_service.settings import Settings
from vision_service.vehicle import BodyTypeClassifier, RefinedVehicleDetector
clf = BodyTypeClassifier(str(tmp_path / "bodytype.onnx"))
assert clf.ready is False and "FileNotFoundError" in (clf.error or "")
(tmp_path / "bodytype.json").write_text('{"format": "other"}')
assert "unknown sidecar format" in (BodyTypeClassifier(str(tmp_path / "bodytype.onnx")).error or "")
# A path with no file = the normal pre-model state: phase A only, no error in health.
s = Settings(
vehicle_model_path="/nonexistent/yolox.onnx", vehicle_classifier_path=str(tmp_path / "none.onnx")
)
rec = build_recognizer(s)
assert isinstance(rec, WithVehicle)
assert not isinstance(rec._detector, RefinedVehicleDetector) # noqa: SLF001
assert "classifier" not in (rec.error or "")
+17 -2
View File
@@ -14,12 +14,16 @@ Adding a recognizer (e.g. a fine-tuned YOLO + PaddleOCR) = a new class here, no
from __future__ import annotations
import logging
import time
from pathlib import Path
from typing import Protocol
from .schemas import AnalyzeResponse, BBox, PlateResult
from .settings import Settings
from .vehicle import VehicleDetector, YoloxVehicleDetector
from .vehicle import BodyTypeClassifier, RefinedVehicleDetector, VehicleDetector, YoloxVehicleDetector
log = logging.getLogger("vision")
class Recognizer(Protocol):
@@ -215,10 +219,21 @@ def build_recognizer(settings: Settings) -> Recognizer:
else:
rec = StubRecognizer(settings)
if settings.vehicle_model_path:
detector = YoloxVehicleDetector(
detector: VehicleDetector = YoloxVehicleDetector(
settings.vehicle_model_path,
input_size=settings.vehicle_input_size,
min_confidence=settings.vehicle_min_confidence,
)
if settings.vehicle_classifier_path:
if Path(settings.vehicle_classifier_path).is_file():
detector = RefinedVehicleDetector(
detector,
BodyTypeClassifier(
settings.vehicle_classifier_path,
min_confidence=settings.vehicle_classifier_min_confidence,
),
)
else:
log.info("no body-type classifier at %s — phase B off", settings.vehicle_classifier_path)
return WithVehicle(rec, detector)
return rec
+3
View File
@@ -45,6 +45,9 @@ class VehicleResult(BaseModel):
confidence: float | None = Field(default=None, ge=0.0, le=1.0)
# The vehicle's box in frame pixels — the crop a reviewer sees / a classifier eats.
bbox: BBox | None = None
# Phase B: the detector's coarse class when the body-type classifier ran on this crop
# (body_type is then the classifier's answer if confident, else the detector's).
detector_class: str | None = None
make: str | None = None
model: str | None = None
+8
View File
@@ -42,6 +42,14 @@ class Settings(BaseSettings):
# site's own, stricter threshold before it FLAGS anything).
vehicle_min_confidence: float = 0.4
# Phase B — the body-type classifier on the detector's crop (bodytype.onnx + its .json
# sidecar, produced by apps/trainer, baked into the image when
# models/bodytype.version pins a published version). Path set but NO file = the normal
# state before the first model ships: the stage is simply off (logged, not an error).
vehicle_classifier_path: str | None = None
# Below this probability the classifier's answer is dropped and the detector's stands.
vehicle_classifier_min_confidence: float = 0.6
def get_settings() -> Settings:
return Settings()
+170
View File
@@ -229,6 +229,13 @@ class YoloxVehicleDetector:
frame = cv2.imdecode(np.frombuffer(image_bytes, dtype=np.uint8), cv2.IMREAD_COLOR)
if frame is None:
return None
return self.detect_frame(frame, plate)
def detect_frame(self, frame: Any, plate: BBox | None) -> VehicleResult | None:
"""Same as detect() on an already-decoded BGR frame (the classifier stage decodes
once and shares it)."""
if self._session is None:
return None
tensor, scale = letterbox(frame, self._size)
raw = self._session.run(None, {self._input_name: tensor})[0][0]
found = vehicles_from_output(raw, self._size, scale, self._min_confidence)
@@ -242,6 +249,169 @@ class YoloxVehicleDetector:
return VehicleResult(body_type=best.body_type, confidence=round(best.confidence, 4), bbox=box)
# ----------------------------------------------------------------------------------
# Phase B: the body-type classifier on the detector's crop
# ----------------------------------------------------------------------------------
SIDECAR_FORMAT = "parking-bodytype/1"
def crop_vehicle(frame: Any, box: BBox, plate: BBox | None, margin: float) -> Any:
"""The detector's box + margin, plate blurred — the SAME cut the collector stores
(apps/server review-outbox.ts makeReviewCrop), so the classifier sees at the booth
what it was trained on. Returns a BGR array, or None when the box is degenerate."""
import cv2
h, w = frame.shape[:2]
mw = round((box.x2 - box.x1) * margin)
mh = round((box.y2 - box.y1) * margin)
left, top = max(0, box.x1 - mw), max(0, box.y1 - mh)
right, bottom = min(w, box.x2 + mw), min(h, box.y2 + mh)
if right - left < 8 or bottom - top < 8:
return None
crop = frame[top:bottom, left:right].copy()
if plate is not None:
pad = round(max(plate.x2 - plate.x1, plate.y2 - plate.y1) * 0.25)
pl, pt = max(0, plate.x1 - pad - left), max(0, plate.y1 - pad - top)
pr, pb = min(right - left, plate.x2 + pad - left), min(bottom - top, plate.y2 + pad - top)
if pr - pl >= 2 and pb - pt >= 2:
sigma = max(6, round((pr - pl) / 6))
crop[pt:pb, pl:pr] = cv2.GaussianBlur(crop[pt:pb, pl:pr], (0, 0), sigma)
return crop
class BodyTypeClassifier:
"""`bodytype.onnx` + its `bodytype.json` sidecar (written by apps/trainer). The sidecar
carries the preprocessing contract — class list, input size, crop margin — and the graph
normalises internally, so this side only cuts, resizes (INTER_AREA, like the trainer)
and feeds raw RGB 0–255. Load failure → `error`, the stage yields nothing."""
def __init__(self, model_path: str, min_confidence: float = 0.6) -> None:
import json
self._path = Path(model_path)
self.min_confidence = min_confidence
self._session = None
self._input_name = "image"
self._error: str | None = None
self.classes: list[str] = []
self.version = "?"
self.input_size = 224
self.crop_margin = 0.08
try:
side = json.loads(self._path.with_suffix(".json").read_text())
if side.get("format") != SIDECAR_FORMAT:
raise ValueError(f"unknown sidecar format {side.get('format')!r}")
self.classes = [str(c) for c in side["classes"]]
self.version = str(side.get("version", "?"))
self.input_size = int(side.get("input_size", 224))
self.crop_margin = float(side.get("crop_margin", 0.08))
import onnxruntime as ort
opts = ort.SessionOptions()
opts.intra_op_num_threads = 2
self._session = ort.InferenceSession(
str(self._path), sess_options=opts, providers=["CPUExecutionProvider"]
)
self._input_name = self._session.get_inputs()[0].name
except Exception as exc: # noqa: BLE001 - not-ready, never fatal
self._error = f"{type(exc).__name__}: {exc}"
@property
def model_version(self) -> str:
return f"bodytype:{self.version}"
@property
def ready(self) -> bool:
return self._session is not None
@property
def error(self) -> str | None:
return self._error
def classify(self, frame: Any, box: BBox, plate: BBox | None) -> tuple[str, float] | None:
"""(class, probability) for the vehicle in `box`, or None when nothing could be cut."""
if self._session is None:
return None
import cv2
import numpy as np
crop = crop_vehicle(frame, box, plate, self.crop_margin)
if crop is None:
return None
resized = cv2.resize(crop, (self.input_size, self.input_size), interpolation=cv2.INTER_AREA)
rgb = cv2.cvtColor(resized, cv2.COLOR_BGR2RGB)
x = np.ascontiguousarray(rgb.transpose(2, 0, 1)[None].astype(np.float32))
logits = self._session.run(None, {self._input_name: x})[0][0]
z = logits - logits.max()
p = np.exp(z) / np.exp(z).sum()
i = int(p.argmax())
return self.classes[i], float(p[i])
class RefinedVehicleDetector:
"""Detector + classifier. The detector finds the vehicle (and picks WHICH one); when its
class is `car` — or one the classifier was trained on — the classifier's answer replaces
it if confident enough, else the detector's stands. A truck or bus the classifier has
never seen is left alone: its softmax on an unknown thing means nothing."""
def __init__(self, detector: Any, classifier: BodyTypeClassifier) -> None:
self._detector = detector
self._classifier = classifier
self.stage_error: str | None = None
@property
def model_version(self) -> str:
return f"{self._detector.model_version}+{self._classifier.model_version}"
@property
def ready(self) -> bool:
return bool(getattr(self._detector, "ready", True))
@property
def error(self) -> str | None:
parts = [
getattr(self._detector, "error", None),
f"classifier: {self._classifier.error}" if self._classifier.error else None,
f"classifier: {self.stage_error}" if self.stage_error else None,
]
kept = [p for p in parts if p]
return "; ".join(kept) if kept else None
def detect(self, image_bytes: bytes, plate: BBox | None) -> VehicleResult | None:
import cv2
import numpy as np
frame = cv2.imdecode(np.frombuffer(image_bytes, dtype=np.uint8), cv2.IMREAD_COLOR)
if frame is None:
return None
detect_frame = getattr(self._detector, "detect_frame", None)
base: VehicleResult | None = (
detect_frame(frame, plate) if detect_frame else self._detector.detect(image_bytes, plate)
)
if base is None or base.bbox is None or not self._classifier.ready:
return base
if not (base.body_type == "car" or base.body_type in self._classifier.classes):
return base
try:
out = self._classifier.classify(frame, base.bbox, plate)
except Exception as exc: # noqa: BLE001 - advisory stage, never fatal
self.stage_error = f"{type(exc).__name__}: {exc}"
return base
if out is None:
return base
body_type, confidence = out
if confidence < self._classifier.min_confidence:
return base.model_copy(update={"detector_class": base.body_type})
return base.model_copy(
update={
"body_type": body_type,
"confidence": round(confidence, 4),
"detector_class": base.body_type,
}
)
def time_detect(
detector: VehicleDetector, image_bytes: bytes, plate: BBox | None
) -> tuple[VehicleResult | None, float]: