feat(vision): vehicle stage, phase A — YOLOX-S (Apache-2.0 ONNX) beside the plate recognizer

Fills /analyze vehicle.body_type + confidence (car / motorcycle / bus / truck from COCO,
mapped to the shared vocabulary) for the Car Wash desk's category suggestion
(venue-modules.md §Vehicle category from vision). Advisory: the operator decides, a
confident downgrade is flagged, nothing is gated on it.

- vision_service/vehicle.py: pure numpy/cv2 letterbox (pad 114, raw BGR), stride-grid
  decode, class-agnostic NMS, one vehicle per frame (the box holding the plate's centre,
  else the largest); YoloxVehicleDetector on onnxruntime CPU, 2 intra-op threads.
- recognizer.py: WithVehicle composes the stage over any plate recognizer (stub included);
  a failing stage yields vehicle=null + a "vehicle: …" note in /health.detail — never
  costs the plate read. model_version reads "<plate>+yolox:yolox_s.onnx@640".
- settings: VISION_VEHICLE_MODEL_PATH (unset = off), _INPUT_SIZE (640), _MIN_CONFIDENCE
  (0.4, the detector's floor; the flag threshold is site config).
- Dockerfile bakes yolox_s.onnx (best-effort curl at build; no network → stage off) and
  sets the path; compose forwards it (empty = off); .env.example documents it.
- Measured on four real dev entry frames (DS-2CD1047G3H, 2560×1440): car at 0.83–0.88 in
  ~240–330 ms; empty lane with a person → none.
- tests/test_vehicle.py: decode/NMS/pick/letterbox on synthetic tensors, the composition,
  and a missing-model /health. Wiki: opencv-anpr-service, venue-modules, log.

Claude-Session: https://claude.ai/code/session_01FWncR69HgGPuei1dLrW3cU
This commit is contained in:
2026-09-06 19:53:11 +02:00
parent 5e1395db18
commit 20a3cb3e80
10 changed files with 521 additions and 15 deletions
+9
View File
@@ -20,3 +20,12 @@ VISION_OCR_MODEL=cct-xs-v2-global-model
# Confidence floor — a best plate below this is flagged low_confidence so the Node side # Confidence floor — a best plate below this is flagged low_confidence so the Node side
# treats it as advisory and falls back to the ticket path. Keep in sync with the server. # treats it as advisory and falls back to the ticket path. Keep in sync with the server.
VISION_MIN_CONFIDENCE=0.5 VISION_MIN_CONFIDENCE=0.5
# Vehicle stage (phase A): a YOLOX ONNX graph (Apache-2.0) run on the same frame after the
# plate read; fills /analyze `vehicle.body_type` (car/truck/bus/motorcycle) + confidence for
# the Car Wash desk's category suggestion. Unset = off. The Docker image bakes the weights
# at /app/models/yolox_s.onnx; locally: curl the release file into apps/vision/models/.
# https://github.com/Megvii-BaseDetection/YOLOX/releases/download/0.1.1rc0/yolox_s.onnx
# VISION_VEHICLE_MODEL_PATH=models/yolox_s.onnx
# VISION_VEHICLE_INPUT_SIZE=640
# VISION_VEHICLE_MIN_CONFIDENCE=0.4
+10 -2
View File
@@ -14,7 +14,7 @@ ENV UV_LINK_MODE=copy \
# System libs the recognizer stack needs (opencv/onnxruntime): GL + glib. Kept minimal. # System libs the recognizer stack needs (opencv/onnxruntime): GL + glib. Kept minimal.
RUN apt-get update \ RUN apt-get update \
&& apt-get install -y --no-install-recommends libgl1 libglib2.0-0 \ && apt-get install -y --no-install-recommends libgl1 libglib2.0-0 curl \
&& rm -rf /var/lib/apt/lists/* && rm -rf /var/lib/apt/lists/*
# ---- deps: resolve + install the venv from the lockfile (cache-friendly) ---- # ---- deps: resolve + install the venv from the lockfile (cache-friendly) ----
@@ -26,6 +26,13 @@ RUN --mount=type=cache,target=/root/.cache/uv \
# ---- project source ---- # ---- project source ----
COPY vision_service/ ./vision_service/ COPY vision_service/ ./vision_service/
COPY README.md ./ COPY README.md ./
# Vehicle stage weights (phase A): YOLOX-S, Apache-2.0, ~36 MB, baked into the image so the
# air-gapped appliance never fetches at runtime and no operator-writable path holds a model
# (vision-service-hardening.md). Best-effort at build: without network the stage stays off.
ARG YOLOX_URL=https://github.com/Megvii-BaseDetection/YOLOX/releases/download/0.1.1rc0/yolox_s.onnx
RUN mkdir -p /app/models \
&& (curl -fsSL -o /app/models/yolox_s.onnx "$YOLOX_URL" \
|| (echo "[build] yolox weights not fetched (no network) — vehicle stage off" && rm -f /app/models/yolox_s.onnx))
RUN --mount=type=cache,target=/root/.cache/uv \ RUN --mount=type=cache,target=/root/.cache/uv \
uv sync --frozen --extra alpr uv sync --frozen --extra alpr
@@ -49,7 +56,8 @@ RUN uv run python -c "from fast_alpr import ALPR; ALPR()" \
# Default to the stub recognizer (offline, no model load); override to fast_alpr in prod. # Default to the stub recognizer (offline, no model load); override to fast_alpr in prod.
ENV VISION_RECOGNIZER=stub \ ENV VISION_RECOGNIZER=stub \
VISION_HOST=0.0.0.0 \ VISION_HOST=0.0.0.0 \
VISION_PORT=8089 VISION_PORT=8089 \
VISION_VEHICLE_MODEL_PATH=/app/models/yolox_s.onnx
EXPOSE 8089 EXPOSE 8089
HEALTHCHECK --interval=30s --timeout=5s --start-period=20s --retries=3 \ HEALTHCHECK --interval=30s --timeout=5s --start-period=20s --retries=3 \
CMD python -c "import urllib.request,sys; sys.exit(0 if urllib.request.urlopen('http://localhost:8089/health').status==200 else 1)" || exit 1 CMD python -c "import urllib.request,sys; sys.exit(0 if urllib.request.urlopen('http://localhost:8089/health').status==200 else 1)" || exit 1
+143
View File
@@ -0,0 +1,143 @@
"""Vehicle stage (phase A) — pure post-processing on synthetic tensors, and the
recognizer composition over the stub with a fake detector. No weights needed."""
from __future__ import annotations
import os
import numpy as np
from fastapi.testclient import TestClient
from vision_service.schemas import BBox, VehicleResult
from vision_service.vehicle import (
COCO_VEHICLE_CLASSES,
Detection,
decode,
letterbox,
nms,
pick_vehicle,
vehicles_from_output,
)
SIZE = 64 # tiny "model" input: grids 8x8 + 4x4 + 2x2 = 84 rows
ROWS = (SIZE // 8) ** 2 + (SIZE // 16) ** 2 + (SIZE // 32) ** 2
def raw_output(hits: list[tuple[int, int, int, float, float, float, float]]) -> np.ndarray:
"""Build a YOLOX-style raw tensor [ROWS, 85] with the given (row, coco_class, _, obj,
cls_score, log_w, log_h) hits; everything else is background."""
raw = np.zeros((ROWS, 85), dtype=np.float32)
raw[:, 2:4] = -10.0 # exp → ~0 size for background rows
for row, cls, _, obj, score, lw, lh in hits:
raw[row, 0:2] = 0.5 # centre of its grid cell
raw[row, 2] = lw
raw[row, 3] = lh
raw[row, 4] = obj
raw[row, 5 + cls] = score
return raw
def test_decode_maps_grid_offsets_and_log_sizes_to_pixels() -> None:
raw = raw_output([(0, 2, 0, 1.0, 1.0, np.log(2.0), np.log(3.0))])
dec = decode(raw, SIZE)
# Row 0 = stride-8 grid cell (0,0): centre (0.5+0)*8 = 4, size exp(log 2)*8 = 16 / 24.
assert dec[0, :4].tolist() == [4.0, 4.0, 16.0, 24.0]
# Last row = stride-32 cell (1,1): centre (0.5+1)*32 = 48.
raw2 = raw_output([(ROWS - 1, 7, 0, 1.0, 1.0, 0.0, 0.0)])
dec2 = decode(raw2, SIZE)
assert dec2[ROWS - 1, :4].tolist() == [48.0, 48.0, 32.0, 32.0]
def test_vehicles_only_above_floor_mapped_to_vocabulary_and_scaled_back() -> None:
raw = raw_output(
[
(0, 2, 0, 0.9, 0.9, np.log(2.0), np.log(2.0)), # car, score .81
(1, 0, 0, 0.99, 0.99, np.log(2.0), np.log(2.0)), # person → ignored
(2, 7, 0, 0.5, 0.5, np.log(2.0), np.log(2.0)), # truck, score .25 → below floor
]
)
found = vehicles_from_output(raw, SIZE, scale=0.5, min_confidence=0.4)
assert [d.body_type for d in found] == ["car"]
assert round(found[0].confidence, 2) == 0.81
# Box 16px wide in the letterboxed input → 32px in the original (scale 0.5).
assert round(found[0].x2 - found[0].x1) == 32
assert set(COCO_VEHICLE_CLASSES.values()) == {"car", "motorcycle", "bus", "truck"}
def test_nms_keeps_the_best_of_overlapping_boxes() -> None:
boxes = np.array([[0, 0, 10, 10], [1, 1, 11, 11], [50, 50, 60, 60]], dtype=np.float32)
scores = np.array([0.5, 0.9, 0.7], dtype=np.float32)
assert sorted(nms(boxes, scores, 0.45)) == [1, 2]
def test_pick_prefers_the_box_holding_the_plate_else_the_largest() -> None:
near = Detection("car", 0.9, 0, 0, 100, 100)
far = Detection("truck", 0.8, 200, 200, 400, 400) # larger
inside = Detection("car", 0.7, 10, 10, 60, 60) # tighter box also holding the plate
assert pick_vehicle([near, far], None) is far
assert pick_vehicle([near, far], BBox(x1=20, y1=20, x2=30, y2=30)) is near
assert pick_vehicle([near, far, inside], BBox(x1=20, y1=20, x2=30, y2=30)) is inside
assert pick_vehicle([near, far], BBox(x1=900, y1=900, x2=910, y2=910)) is far # plate outside every box
assert pick_vehicle([], None) is None
def test_letterbox_keeps_aspect_and_pads_with_114() -> None:
frame = np.zeros((30, 60, 3), dtype=np.uint8)
tensor, scale = letterbox(frame, 64)
assert tensor.shape == (1, 3, 64, 64) and tensor.dtype == np.float32
assert abs(scale - 64 / 60) < 1e-9
assert tensor[0, 0, 63, 63] == 114.0 # padding
assert tensor[0, 0, 0, 0] == 0.0 # image
class FakeDetector:
model_version = "fake-vehicle"
def __init__(self, result: VehicleResult | None) -> None:
self.result = result
self.calls: list[BBox | None] = []
def detect(self, image_bytes: bytes, plate: BBox | None) -> VehicleResult | None:
self.calls.append(plate)
return self.result
def test_composition_fills_vehicle_over_the_stub_and_survives_a_failing_stage() -> None:
from vision_service.recognizer import StubRecognizer, WithVehicle
from vision_service.settings import Settings
det = FakeDetector(VehicleResult(body_type="truck", confidence=0.77))
rec = WithVehicle(StubRecognizer(Settings()), det)
res = rec.analyze(b"jpeg-bytes")
assert res.plate is None
assert res.vehicle == VehicleResult(body_type="truck", confidence=0.77)
assert res.model_version == "stub-0+fake-vehicle"
assert det.calls == [None]
class Boom:
model_version = "boom"
def detect(self, image_bytes: bytes, plate: BBox | None) -> VehicleResult | None:
raise RuntimeError("no model")
rec2 = WithVehicle(StubRecognizer(Settings()), Boom())
res2 = rec2.analyze(b"jpeg-bytes")
assert res2.vehicle is None
assert rec2.ready is True
assert "vehicle: RuntimeError: no model" in (rec2.error or "")
def test_app_reports_a_missing_model_file_and_keeps_serving() -> None:
from vision_service.app import app
os.environ["VISION_VEHICLE_MODEL_PATH"] = "/nonexistent/yolox.onnx"
try:
with TestClient(app) as client:
health = client.get("/health").json()
assert health["ready"] is True # the plate stage (stub) is fine
assert "vehicle:" in (health["detail"] or "")
res = client.post("/analyze", content=b"x", headers={"content-type": "application/octet-stream"})
assert res.status_code == 200
assert res.json()["vehicle"] is None
finally:
os.environ.pop("VISION_VEHICLE_MODEL_PATH", None)
+52 -2
View File
@@ -19,6 +19,7 @@ from typing import Protocol
from .schemas import AnalyzeResponse, BBox, PlateResult from .schemas import AnalyzeResponse, BBox, PlateResult
from .settings import Settings from .settings import Settings
from .vehicle import VehicleDetector, YoloxVehicleDetector
class Recognizer(Protocol): class Recognizer(Protocol):
@@ -165,10 +166,59 @@ class FastAlprRecognizer:
) )
class WithVehicle:
"""Composition: any plate recognizer + the vehicle stage. Runs the plate stage first
(its box picks WHICH vehicle), then fills `vehicle`. A failing vehicle stage is
logged into `error` and yields null — it must never cost the plate read."""
def __init__(self, inner: Recognizer, detector: VehicleDetector) -> None:
self._inner = inner
self._detector = detector
self.vehicle_error: str | None = None
@property
def model_version(self) -> str:
return f"{self._inner.model_version}+{self._detector.model_version}"
@property
def ready(self) -> bool:
return bool(self._inner.ready)
@property
def error(self) -> str | None:
inner = getattr(self._inner, "error", None)
det = getattr(self._detector, "error", None) or self.vehicle_error
parts = [p for p in (inner, f"vehicle: {det}" if det else None) if p]
return "; ".join(parts) if parts else None
def analyze(self, image_bytes: bytes) -> AnalyzeResponse:
started = time.perf_counter()
res = self._inner.analyze(image_bytes)
try:
vehicle = self._detector.detect(image_bytes, res.plate.bbox if res.plate else None)
except Exception as exc: # noqa: BLE001 - advisory stage, never fatal
self.vehicle_error = f"{type(exc).__name__}: {exc}"
vehicle = None
took_ms = (time.perf_counter() - started) * 1000.0
return res.model_copy(
update={"vehicle": vehicle, "model_version": self.model_version, "took_ms": took_ms}
)
def build_recognizer(settings: Settings) -> Recognizer: def build_recognizer(settings: Settings) -> Recognizer:
"""Factory: pick the recognizer from settings. Falls back to the stub if the real """Factory: pick the recognizer from settings. Falls back to the stub if the real
one can't load, so the service always comes up (with ready=False surfaced).""" one can't load, so the service always comes up (with ready=False surfaced). The
vehicle stage wraps whichever recognizer runs when a model path is configured."""
rec: Recognizer
if settings.recognizer == "fast_alpr": if settings.recognizer == "fast_alpr":
rec = FastAlprRecognizer(settings) rec = FastAlprRecognizer(settings)
else:
rec = StubRecognizer(settings)
if settings.vehicle_model_path:
detector = YoloxVehicleDetector(
settings.vehicle_model_path,
input_size=settings.vehicle_input_size,
min_confidence=settings.vehicle_min_confidence,
)
return WithVehicle(rec, detector)
return rec return rec
return StubRecognizer(settings)
+10
View File
@@ -32,6 +32,16 @@ class Settings(BaseSettings):
# Node side can fall back to the ticket path rather than trust it. # Node side can fall back to the ticket path rather than trust it.
min_confidence: float = 0.5 min_confidence: float = 0.5
# Vehicle stage (phase A — venue-modules.md §Vehicle category from vision): a YOLOX
# ONNX graph (Apache-2.0) run beside the plate recognizer. Unset = stage off (the
# response's `vehicle` stays null). Bake the file into the image (models/), never a
# path an operator can write (vision-service-hardening.md).
vehicle_model_path: str | None = None
vehicle_input_size: int = 640
# Detection score floor for a vehicle box to count at all (the Node side applies the
# site's own, stricter threshold before it FLAGS anything).
vehicle_min_confidence: float = 0.4
def get_settings() -> Settings: def get_settings() -> Settings:
return Settings() return Settings()
+247
View File
@@ -0,0 +1,247 @@
"""Vehicle stage: a COCO object detector beside the plate recognizer (Job 2, phase A).
Answers "what KIND of vehicle is in this entry frame?" for the Car Wash desk's category
suggestion (wiki/decisions/venue-modules.md §Vehicle category from vision). ADVISORY by
design: the Node server records it next to the plate, the desk pre-selects the site
category it maps to, the operator decides, a confident downgrade is flagged. Nothing is
ever gated on it, so a wrong or missing detection costs nothing but a suggestion.
Model: YOLOX (Megvii, Apache-2.0) as an ONNX graph on the ONNX Runtime the plate stage
already uses — the licence rule that keeps Ultralytics (AGPL) out. COCO's vehicle classes
are car / motorcycle / bus / truck: enough to tell a van or a truck from a car, NOT enough
for SUV vs sedan — that is phase B (a body-type classifier on the pilot's own frames).
The detector's vehicle box is also the crop phase B will classify.
Pure numpy/cv2 pre/post-processing, no torch: letterbox to the model's square input
(pad 114, no normalisation — YOLOX's exported graphs take raw 0–255 BGR), decode the
stride grids, class-agnostic NMS, map COCO ids to the shared vocabulary, pick ONE
vehicle: the one whose box holds the plate (when a plate was read), else the largest.
"""
from __future__ import annotations
import time
from dataclasses import dataclass
from pathlib import Path
from typing import Any, Protocol
from .schemas import BBox, VehicleResult
# COCO-80 class index → the shared VEHICLE_CLASSES vocabulary (packages/shared).
COCO_VEHICLE_CLASSES: dict[int, str] = {2: "car", 3: "motorcycle", 5: "bus", 7: "truck"}
# YOLOX feature strides; grids are input/stride per level (8400 anchors at 640).
_STRIDES = (8, 16, 32)
@dataclass(frozen=True)
class Detection:
body_type: str
confidence: float
x1: float
y1: float
x2: float
y2: float
@property
def area(self) -> float:
return max(0.0, self.x2 - self.x1) * max(0.0, self.y2 - self.y1)
def contains(self, x: float, y: float) -> bool:
return self.x1 <= x <= self.x2 and self.y1 <= y <= self.y2
class VehicleDetector(Protocol):
"""What the recognizer composition needs: frame bytes (+ the plate box) → a class."""
@property
def model_version(self) -> str: ...
def detect(self, image_bytes: bytes, plate: BBox | None) -> VehicleResult | None: ...
# ----------------------------------------------------------------------------------
# Pre/post-processing (pure functions — unit-tested on synthetic tensors)
# ----------------------------------------------------------------------------------
def letterbox(frame: Any, size: int) -> tuple[Any, float]:
"""Resize keeping aspect, pad bottom/right with 114 to size×size. Returns the CHW
float32 tensor (batch dim added) and the scale to map boxes back."""
import cv2
import numpy as np
h, w = frame.shape[:2]
r = min(size / h, size / w)
nh, nw = int(round(h * r)), int(round(w * r))
resized = cv2.resize(frame, (nw, nh), interpolation=cv2.INTER_LINEAR)
padded = np.full((size, size, 3), 114, dtype=np.uint8)
padded[:nh, :nw] = resized
tensor = padded.transpose(2, 0, 1)[None].astype(np.float32)
return np.ascontiguousarray(tensor), r
def decode(raw: Any, size: int) -> Any:
"""YOLOX raw output [N, 5+classes] (batch squeezed) → same shape with xywh decoded
into pixel units of the letterboxed input. Rows are ordered stride 8, 16, 32."""
import numpy as np
out = raw.astype(np.float32).copy()
grids = []
strides = []
for s in _STRIDES:
n = size // s
ys, xs = np.meshgrid(np.arange(n), np.arange(n), indexing="ij")
grids.append(np.stack((xs, ys), axis=-1).reshape(-1, 2))
strides.append(np.full((n * n, 1), s, dtype=np.float32))
grid = np.concatenate(grids, axis=0).astype(np.float32)
stride = np.concatenate(strides, axis=0)
if out.shape[0] != grid.shape[0]:
raise ValueError(f"unexpected output rows {out.shape[0]} for input {size} (want {grid.shape[0]})")
out[:, :2] = (out[:, :2] + grid) * stride
out[:, 2:4] = np.exp(out[:, 2:4]) * stride
return out
def nms(boxes: Any, scores: Any, iou_threshold: float) -> list[int]:
"""Greedy class-agnostic non-max suppression over xyxy boxes; returns kept indices."""
import numpy as np
if len(boxes) == 0:
return []
order = scores.argsort()[::-1]
x1, y1, x2, y2 = boxes[:, 0], boxes[:, 1], boxes[:, 2], boxes[:, 3]
areas = np.clip(x2 - x1, 0, None) * np.clip(y2 - y1, 0, None)
keep: list[int] = []
while order.size > 0:
i = int(order[0])
keep.append(i)
if order.size == 1:
break
rest = order[1:]
xx1 = np.maximum(x1[i], x1[rest])
yy1 = np.maximum(y1[i], y1[rest])
xx2 = np.minimum(x2[i], x2[rest])
yy2 = np.minimum(y2[i], y2[rest])
inter = np.clip(xx2 - xx1, 0, None) * np.clip(yy2 - yy1, 0, None)
iou = inter / (areas[i] + areas[rest] - inter + 1e-9)
order = rest[iou <= iou_threshold]
return keep
def vehicles_from_output(
raw: Any, size: int, scale: float, min_confidence: float, iou_threshold: float = 0.45
) -> list[Detection]:
"""Full post-processing: decode → vehicle classes only → confidence floor → NMS →
boxes in ORIGINAL frame pixels."""
import numpy as np
dec = decode(raw, size)
cls_scores = dec[:, 5:]
cls_idx = cls_scores.argmax(axis=1)
score = dec[:, 4] * cls_scores[np.arange(len(dec)), cls_idx]
wanted = np.isin(cls_idx, list(COCO_VEHICLE_CLASSES)) & (score >= min_confidence)
if not wanted.any():
return []
d = dec[wanted]
s = score[wanted]
c = cls_idx[wanted]
boxes = np.stack(
(d[:, 0] - d[:, 2] / 2, d[:, 1] - d[:, 3] / 2, d[:, 0] + d[:, 2] / 2, d[:, 1] + d[:, 3] / 2), axis=1
)
keep = nms(boxes, s, iou_threshold)
out: list[Detection] = []
for i in keep:
b = boxes[i] / scale
out.append(
Detection(
body_type=COCO_VEHICLE_CLASSES[int(c[i])],
confidence=float(s[i]),
x1=float(b[0]),
y1=float(b[1]),
x2=float(b[2]),
y2=float(b[3]),
)
)
return out
def pick_vehicle(detections: list[Detection], plate: BBox | None) -> Detection | None:
"""ONE vehicle per frame: the box holding the plate's centre (the car that was read —
a lane frame can show the car behind too), else the largest box (nearest the camera)."""
if not detections:
return None
if plate is not None:
cx = (plate.x1 + plate.x2) / 2
cy = (plate.y1 + plate.y2) / 2
holders = [d for d in detections if d.contains(cx, cy)]
if holders:
return min(holders, key=lambda d: d.area) # the tightest box around the plate
return max(detections, key=lambda d: d.area)
# ----------------------------------------------------------------------------------
# The ONNX Runtime detector
# ----------------------------------------------------------------------------------
class YoloxVehicleDetector:
"""YOLOX ONNX on onnxruntime (CPU). Loads once; a load failure is surfaced through
`error` and the stage simply yields no vehicle (never breaks the plate path)."""
def __init__(self, model_path: str, input_size: int = 640, min_confidence: float = 0.4) -> None:
self._path = Path(model_path)
self._size = input_size
self._min_confidence = min_confidence
self._session = None
self._input_name = "images"
self._error: str | None = None
try:
import onnxruntime as ort
opts = ort.SessionOptions()
opts.intra_op_num_threads = 2 # one frame per entry; leave cores to the lane
self._session = ort.InferenceSession(
str(self._path), sess_options=opts, providers=["CPUExecutionProvider"]
)
self._input_name = self._session.get_inputs()[0].name
except Exception as exc: # noqa: BLE001 - not-ready, never fatal
self._error = f"{type(exc).__name__}: {exc}"
@property
def model_version(self) -> str:
return f"yolox:{self._path.name}@{self._size}"
@property
def ready(self) -> bool:
return self._session is not None
@property
def error(self) -> str | None:
return self._error
def detect(self, image_bytes: bytes, plate: BBox | None) -> VehicleResult | None:
if self._session is None:
return None
import cv2
import numpy as np
frame = cv2.imdecode(np.frombuffer(image_bytes, dtype=np.uint8), cv2.IMREAD_COLOR)
if frame is None:
return None
tensor, scale = letterbox(frame, self._size)
raw = self._session.run(None, {self._input_name: tensor})[0][0]
found = vehicles_from_output(raw, self._size, scale, self._min_confidence)
best = pick_vehicle(found, plate)
if best is None:
return None
return VehicleResult(body_type=best.body_type, confidence=round(best.confidence, 4))
def time_detect(
detector: VehicleDetector, image_bytes: bytes, plate: BBox | None
) -> tuple[VehicleResult | None, float]:
"""detect() with wall time in ms (for logs/benchmarks)."""
started = time.perf_counter()
result = detector.detect(image_bytes, plate)
return result, (time.perf_counter() - started) * 1000.0
+3
View File
@@ -55,6 +55,9 @@ services:
environment: environment:
# Engine: stub (no models) by default; prod override sets fast_alpr. # Engine: stub (no models) by default; prod override sets fast_alpr.
VISION_RECOGNIZER: ${VISION_RECOGNIZER:-stub} VISION_RECOGNIZER: ${VISION_RECOGNIZER:-stub}
# Vehicle stage (Car Wash category suggestion): the image bakes YOLOX-S at this path.
# Set the var to an EMPTY value in the stack env to switch the stage off.
VISION_VEHICLE_MODEL_PATH: ${VISION_VEHICLE_MODEL_PATH-/app/models/yolox_s.onnx}
networks: networks:
- parking - parking
+7 -5
View File
@@ -188,11 +188,13 @@ onto the site's own categories ("car, sedan, hatchback → Vetura").
and the read is at or above the threshold → one `anomaly` (`carwash.categoryDowngrade`, both and the read is at or above the threshold → one `anomaly` (`carwash.categoryDowngrade`, both
categories, both prices, operator, snapshotId) and `downgrade_event_id` on the order. Equal, categories, both prices, operator, snapshotId) and `downgrade_event_id` on the order. Equal,
upgrade, unsure or unmapped reads flag nothing. The order is always created. upgrade, unsure or unmapped reads flag nothing. The order is always created.
- **Model — NOT built.** No bundled recognizer produces `body_type` yet, so today the desk shows - **Model — phase A built (same day).** YOLOX-S (Apache-2.0 ONNX) as a vehicle stage beside the
nothing and nothing is flagged. Phase A = a COCO detector on Apache-2.0 ONNX weights (car / plate recognizer: car / truck / bus / motorcycle + the vehicle crop, ~250 ms per entry frame on
truck / bus / motorcycle, plus the vehicle crop); Phase B = the body-type classifier trained on CPU, weights baked into the vision image. Details and measurements on [[opencv-anpr-service]]
the pilot's own frames — every wash order is a labelled frame (entry snapshot + the category a §Vehicle body type. Phase B = the body-type classifier trained on the pilot's own frames —
person chose), so the dataset builds itself on park-2. Reports (discrepancies per operator per every wash order is a labelled frame (entry snapshot + the category a person chose), so the
dataset builds itself on park-2. Until then a Vetura/SUV list sees every car as Vetura and no
downgrade fires; van/truck/bus/motorcycle do separate. Reports (discrepancies per operator per
shift) wait for the first real reads. shift) wait for the first real reads.
## Car Wash — the pilot module (settled 2026-09-05) ## Car Wash — the pilot module (settled 2026-09-05)
+30 -5
View File
@@ -249,11 +249,36 @@ service's `/health` each tick and shows a **"Vision" chip** in the booth footer
- Per-camera **opt-in** — ✅ **built**: `config.anpr === true` enables ANPR on a camera (set via the - Per-camera **opt-in** — ✅ **built**: `config.anpr === true` enables ANPR on a camera (set via the
SetupWizard checkbox); ANPR then runs on that camera's entry/exit snapshot. SetupWizard checkbox); ANPR then runs on that camera's entry/exit snapshot.
## Vehicle body type (advisory) — contract only, 2026-09-06 ## Vehicle body type (advisory) — the vehicle stage, phase A (2026-09-06)
`/analyze` may now populate `vehicle.body_type` + `vehicle.confidence` from the shared vocabulary `/analyze` populates `vehicle.body_type` + `vehicle.confidence` from the shared vocabulary
(car, sedan, hatchback, suv, minivan, pickup, van, truck, bus, motorcycle). Node records it beside (car, sedan, hatchback, suv, minivan, pickup, van, truck, bus, motorcycle). Node records it beside
the plate and the Car Wash desk pre-selects the category the site maps it to; the operator the plate and the Car Wash desk pre-selects the category the site maps it to; the operator
decides, a confident downgrade is flagged, nothing is gated on it. No bundled recognizer emits decides, a confident downgrade is flagged, nothing is gated on it ([[venue-modules]] §Vehicle
it yet — see [[venue-modules]] §Vehicle category from vision for the model plan (COCO detector category from vision).
first, body-type classifier on own frames second).
**Phase A = YOLOX-S (Megvii, Apache-2.0) as ONNX** on the same ONNX Runtime the plate stage uses
— the licence rule that keeps Ultralytics (AGPL) out. `vision_service/vehicle.py`: pure numpy/cv2
letterbox (pad 114, raw 0–255 BGR — YOLOX's exported graphs are not normalised), stride-grid
decode, class-agnostic NMS, COCO `car/motorcycle/bus/truck` → the vocabulary, and ONE vehicle per
frame: the box holding the plate's centre when a plate was read (the car that was read, not the
one behind), else the largest box. `WithVehicle` in `recognizer.py` wraps whichever plate
recognizer runs (stub included, so the stage is testable without fast-alpr); a failing stage
yields `vehicle: null` and a `vehicle: …` note in `/health.detail` — it never costs the plate
read. Composed `model_version` reads `<plate>+yolox:yolox_s.onnx@640`.
- **Config:** `VISION_VEHICLE_MODEL_PATH` (unset = stage off), `VISION_VEHICLE_INPUT_SIZE` (640),
`VISION_VEHICLE_MIN_CONFIDENCE` (0.4 — the detector's floor; the SITE threshold that decides a
flag lives in Setup → Car wash). The Docker image bakes the weights at `/app/models/yolox_s.onnx`
(best-effort curl at build; no network → stage off) and sets the path, so the air-gapped
appliance never fetches at runtime and no operator-writable path holds a model
([[vision-service-hardening]]). Compose forwards the var; set it EMPTY in the stack env to
switch the stage off. Locally: curl the release file into `apps/vision/models/` (gitignored).
- **Measured on dev (2026-09-06), four real 2560×1440 entry frames from the DS-2CD1047G3H:** three
with a car → `car` at 0.83–0.88, ~240–330 ms each on the dev CPU with 2 intra-op threads; the
empty-lane frame with a person at the camera → no vehicle (correct: a person is not a class we
keep). One frame per entry, so the cost is invisible to the lane.
- **What it cannot do:** SUV vs sedan — COCO has one `car`. For a Vetura/SUV price list every car
maps to Vetura and no downgrade fires; vans, trucks, buses and motorcycles do separate. Phase B
(a body-type classifier on the pilot's own frames — every wash order is a labelled frame) is
what closes that gap; the detector's box is the crop it will classify.
+9
View File
@@ -3091,3 +3091,12 @@ category + threshold; the desk pre-selects the mapped category and shows the rea
a confident, pricier-mapped read with a cheaper choice signs `anomaly carwash.categoryDowngrade` a confident, pricier-mapped read with a cheaper choice signs `anomaly carwash.categoryDowngrade`
(both categories/prices, operator, snapshot) — never blocks. No recognizer emits body_type yet. (both categories/prices, operator, snapshot) — never blocks. No recognizer emits body_type yet.
Tests in carwash.test.ts. Updated [[venue-modules]] (As built), [[opencv-anpr-service]]. Tests in carwash.test.ts. Updated [[venue-modules]] (As built), [[opencv-anpr-service]].
## [2026-09-06] ingest | Vision vehicle stage, phase A: YOLOX-S beside the plate recognizer
`vision_service/vehicle.py` (YOLOX ONNX on onnxruntime: letterbox, grid decode, NMS, COCO
car/motorcycle/bus/truck → vocabulary, one vehicle per frame — the box holding the plate, else the
largest) + `WithVehicle` composition over any plate recognizer; `VISION_VEHICLE_MODEL_PATH` (unset =
off), input size, detector floor; Dockerfile bakes yolox_s.onnx (best-effort curl) and sets the
path; compose forwards it (empty = off). Measured on four real dev entry frames: car at 0.83–0.88,
~240–330 ms, empty lane → none. Tests: tests/test_vehicle.py (pure post-processing + composition +
missing-model health). Updated [[opencv-anpr-service]], [[venue-modules]].