feat(vision): surface the recognized plate in the booth UI

The ANPR plate was saved (device_events kind:"read") but had no UI. Extend
GET /api/snapshots/by-identity/:identity to also return plates[] (plate, confidence,
region, direction, snapshotId, at) for that session, and render each as a cyan
"Plate: AA558EE 100%" chip in the SnapshotStrip — so it shows in both the booth
event-detail modal and the pay modal, beside the evidence photo, no separate screen.
Deduped by plate+direction; session:read gated; i18n sq+en.

Verified: by-identity returns plates[] for a seeded read (200, AA558EE 0.999 Albania
entry). Build + lint green.

Claude-Session: https://claude.ai/code/session_01Xcm6ikLgGoCxxHrxtjkk5V
This commit is contained in:
2026-06-19 17:23:32 +02:00
parent ecaaefd899
commit 9ec644811a
7 changed files with 100 additions and 5 deletions
+38 -1
View File
@@ -68,7 +68,44 @@ export async function snapshotRoutes(app: FastifyInstance, db: Db): Promise<void
}); });
} }
return { snapshots: rows, failures }; // Recognized PLATES for this session: kind="read" telemetry from the ANPR-on-
// snapshot path (snapshot.ts → recognizePlate). Advisory — a record of the plate
// observed for the session, shown beside the image. Newest first.
const plateRows = db
.select({ detail: deviceEvents.detail, occurredAt: deviceEvents.occurredAt })
.from(deviceEvents)
.where(and(eq(deviceEvents.category, "camera"), eq(deviceEvents.kind, "read")))
.orderBy(desc(deviceEvents.occurredAt))
.all();
const plates: {
plate: string;
confidence: number | null;
region: string | null;
direction: "entry" | "exit" | null;
snapshotId: string | null;
at: string;
}[] = [];
for (const row of plateRows) {
const d = (row.detail ?? {}) as {
identity?: string;
plate?: string;
confidence?: number;
region?: string | null;
direction?: string;
snapshotId?: string;
};
if (d.identity !== identity || !d.plate) continue;
plates.push({
plate: d.plate,
confidence: typeof d.confidence === "number" ? d.confidence : null,
region: d.region ?? null,
direction: d.direction === "entry" || d.direction === "exit" ? d.direction : null,
snapshotId: d.snapshotId ?? null,
at: row.occurredAt ?? "",
});
}
return { snapshots: rows, failures, plates };
}, },
); );
+13 -2
View File
@@ -789,11 +789,22 @@ export interface SnapshotFailure {
occurredAt: string; occurredAt: string;
} }
/** A licence plate recognized for this session by the ANPR-on-snapshot path (advisory
* record — see opencv-anpr-service.md). `snapshotId` links to the image it was read from. */
export interface PlateRead {
plate: string;
confidence: number | null;
region: string | null;
direction: "entry" | "exit" | null;
snapshotId: string | null;
at: string;
}
/** Snapshot metadata for a session identity (newest first) PLUS failed capture /** Snapshot metadata for a session identity (newest first) PLUS failed capture
* attempts. Image bytes are at `/api/snapshots/:id` — use that as an <img src>. */ * attempts PLUS any recognized plates. Image bytes are at `/api/snapshots/:id`. */
export function fetchSnapshots( export function fetchSnapshots(
identity: string, identity: string,
): Promise<{ snapshots: SnapshotMeta[]; failures?: SnapshotFailure[] }> { ): Promise<{ snapshots: SnapshotMeta[]; failures?: SnapshotFailure[]; plates?: PlateRead[] }> {
return apiFetch(`/api/snapshots/by-identity/${encodeURIComponent(identity)}`); return apiFetch(`/api/snapshots/by-identity/${encodeURIComponent(identity)}`);
} }
+1
View File
@@ -558,5 +558,6 @@ export const en: Catalog = {
snapEntry: "entry", snapEntry: "entry",
snapExit: "exit", snapExit: "exit",
snapFailed: "camera unreachable", snapFailed: "camera unreachable",
plate: "Plate",
}, },
}; };
+1
View File
@@ -573,6 +573,7 @@ export const sq = {
snapEntry: "hyrje", snapEntry: "hyrje",
snapExit: "dalje", snapExit: "dalje",
snapFailed: "kamera e paarritshme", snapFailed: "kamera e paarritshme",
plate: "Targa",
}, },
}; };
+38 -2
View File
@@ -1,7 +1,20 @@
import { useState } from "react"; import { useState } from "react";
import { useTranslation } from "react-i18next"; import { useTranslation } from "react-i18next";
import { useQuery } from "@tanstack/react-query"; import { useQuery } from "@tanstack/react-query";
import { fetchSnapshots, snapshotImageUrl } from "../api.js"; import { fetchSnapshots, snapshotImageUrl, type PlateRead } from "../api.js";
/** Keep one entry per (plate, direction) — newest wins (the list is newest-first). */
function dedupePlates(plates: PlateRead[]): PlateRead[] {
const seen = new Set<string>();
const out: PlateRead[] = [];
for (const p of plates) {
const key = `${p.plate}|${p.direction}`;
if (seen.has(key)) continue;
seen.add(key);
out.push(p);
}
return out;
}
// Entry/exit evidence images for a session. Lets the operator verify the car at the // Entry/exit evidence images for a session. Lets the operator verify the car at the
// booth against the ticket. Thumbnails load from /api/snapshots/:id (cookie-authed, // booth against the ticket. Thumbnails load from /api/snapshots/:id (cookie-authed,
@@ -18,17 +31,40 @@ export function SnapshotStrip({ identity }: { identity: string }) {
const shots = data?.snapshots ?? []; const shots = data?.snapshots ?? [];
const failures = data?.failures ?? []; const failures = data?.failures ?? [];
const plates = data?.plates ?? [];
/** Localized direction label for a snapshot/failure tile. */ /** Localized direction label for a snapshot/failure tile. */
const dirLabel = (dir: "entry" | "exit" | null): string => const dirLabel = (dir: "entry" | "exit" | null): string =>
dir === "entry" ? t("pay.snapEntry") : dir === "exit" ? t("pay.snapExit") : "—"; dir === "entry" ? t("pay.snapEntry") : dir === "exit" ? t("pay.snapExit") : "—";
if (isLoading) return <div className="text-[11px] text-term-muted">{t("pay.loadingSnapshots")}</div>; if (isLoading) return <div className="text-[11px] text-term-muted">{t("pay.loadingSnapshots")}</div>;
if (shots.length === 0 && failures.length === 0) if (shots.length === 0 && failures.length === 0 && plates.length === 0)
return <div className="text-[11px] text-term-muted">{t("pay.noSnapshots")}</div>; return <div className="text-[11px] text-term-muted">{t("pay.noSnapshots")}</div>;
return ( return (
<> <>
{/* Recognized plate(s) (ANPR) — advisory. Dedup by plate+direction so an
entry+exit read of the same plate shows once per direction. */}
{plates.length > 0 && (
<div className="mb-2 flex flex-wrap gap-1.5">
{dedupePlates(plates).map((p, i) => (
<span
key={`${p.plate}-${p.direction}-${i}`}
className="inline-flex items-center gap-1.5 rounded-term border border-term-cyan/40 bg-term-cyan/10 px-2 py-0.5 text-[11px]"
title={`${dirLabel(p.direction)}${p.region ? ` · ${p.region}` : ""}${
p.at ? ` · ${new Date(p.at).toLocaleString()}` : ""
}`}
>
<span className="text-[9px] uppercase tracking-wider text-term-muted">{t("pay.plate")}</span>
<span className="font-mono font-semibold text-term-cyan">{p.plate}</span>
{typeof p.confidence === "number" && (
<span className="text-[10px] text-term-muted">{(p.confidence * 100).toFixed(0)}%</span>
)}
</span>
))}
</div>
)}
<div className="flex flex-wrap gap-2"> <div className="flex flex-wrap gap-2">
{shots.map((s) => ( {shots.map((s) => (
<button <button
+5
View File
@@ -186,6 +186,11 @@ records nothing, a vision failure never delays or changes the open, and the plat
access decision (the flow already decided). *Verified end-to-end:* a simulated entry snapshot on an access decision (the flow already decided). *Verified end-to-end:* a simulated entry snapshot on an
`anpr` camera → stored the snapshot for the session AND recorded `{identity:"TICKET-…", plate:"AA558EE", `anpr` camera → stored the snapshot for the session AND recorded `{identity:"TICKET-…", plate:"AA558EE",
confidence:0.999, region:"Albania", snapshotId:…}`. confidence:0.999, region:"Albania", snapshotId:…}`.
**Viewing it:** `GET /api/snapshots/by-identity/:identity` now also returns `plates[]` (the
`kind:"read"` reads for that session), and the **`SnapshotStrip`** renders each as a cyan
"Plate: AA558EE 100%" chip above the images — so the recognized plate shows in the **booth
event-detail modal AND the pay modal** beside the evidence photo, with no separate screen.
(3) **field-accuracy** unknown — re-benchmark/tune (3) **field-accuracy** unknown — re-benchmark/tune
the threshold on real on-site captures (angle/night/dirt). (4) the **weight-provenance** check (open). the threshold on real on-site captures (angle/night/dirt). (4) the **weight-provenance** check (open).
**Bottom line: consume it as a gated advisory identity record off the entry/exit snapshot — not as sole **Bottom line: consume it as a gated advisory identity record off the entry/exit snapshot — not as sole
+4
View File
@@ -944,3 +944,7 @@ Made the vision service genuinely configurable (was env-only). THREE additions:
## [2026-06-19] refactor | ANPR rides the entry/exit snapshot (replaces polling VisionReader) ## [2026-06-19] refactor | ANPR rides the entry/exit snapshot (replaces polling VisionReader)
Reworked the ANPR TRIGGER per the real design goal: when a transient pushes the button or a subscriber passes QR/RFID, the entry/exit fires and takes the evidence snapshot — THAT is the moment to recognize the plate, off the SAME image, tied to the SAME session. So snapshotAsync now takes the VisionClient and, after storing each snapshot from an opt-in (config.anpr) camera, runs ANPR on shot.bytes and records the plate against the session identity (device_events kind:"read" with plate/confidence/region/snapshotId/source:"entry-exit-snapshot"). One image serves both evidence + plate extraction; recognition fires ONLY on a real entry/exit — NO POLLING. The flows (entry/exit/subscription) now take an optional VisionClient and pass it through; server.ts wires it. REMOVED the polling VisionReader (vision-reader.ts deleted) + VISION_POLL_MS/VISION_DEDUPE_MS env. Advisory + fire-and-forget: low-confidence/no-plate records nothing, a vision failure never delays/changes the open, the plate does NOT feed the access decision (the flow already decided) — it's a record ("session X entered on plate AA558EE"). VERIFIED e2e: a simulated entry snapshot on an anpr camera (live fast_alpr) stored the snapshot for session TICKET-SG-1 AND recorded {identity:TICKET-SG-1, plate:AA558EE, confidence:0.999, region:Albania, snapshotId:…}. Build+lint green. Updated [[opencv-anpr-service]] (trigger section + Configuration, polling refs removed) + both .env.example. Reworked the ANPR TRIGGER per the real design goal: when a transient pushes the button or a subscriber passes QR/RFID, the entry/exit fires and takes the evidence snapshot — THAT is the moment to recognize the plate, off the SAME image, tied to the SAME session. So snapshotAsync now takes the VisionClient and, after storing each snapshot from an opt-in (config.anpr) camera, runs ANPR on shot.bytes and records the plate against the session identity (device_events kind:"read" with plate/confidence/region/snapshotId/source:"entry-exit-snapshot"). One image serves both evidence + plate extraction; recognition fires ONLY on a real entry/exit — NO POLLING. The flows (entry/exit/subscription) now take an optional VisionClient and pass it through; server.ts wires it. REMOVED the polling VisionReader (vision-reader.ts deleted) + VISION_POLL_MS/VISION_DEDUPE_MS env. Advisory + fire-and-forget: low-confidence/no-plate records nothing, a vision failure never delays/changes the open, the plate does NOT feed the access decision (the flow already decided) — it's a record ("session X entered on plate AA558EE"). VERIFIED e2e: a simulated entry snapshot on an anpr camera (live fast_alpr) stored the snapshot for session TICKET-SG-1 AND recorded {identity:TICKET-SG-1, plate:AA558EE, confidence:0.999, region:Albania, snapshotId:…}. Build+lint green. Updated [[opencv-anpr-service]] (trigger section + Configuration, polling refs removed) + both .env.example.
## [2026-06-19] feat | Surface recognized plate in the booth UI (SnapshotStrip)
Made the ANPR plate VIEWABLE (it was saved but had no UI). Extended GET /api/snapshots/by-identity/:identity to also query device_events kind:"read" for that identity and return plates[] (plate, confidence, region, direction, snapshotId, at) alongside the existing snapshots + failures. The SnapshotStrip now renders each recognized plate as a cyan "Plate: AA558EE 100%" chip above the images (deduped by plate+direction; title shows region + time) — so it appears in BOTH the booth event-detail modal and the pay modal, beside the evidence photo, no separate screen. session:read gated (same as snapshots). i18n pay.plate sq+en. VERIFIED: by-identity returns plates[] for a seeded read (status 200, {plate:AA558EE, confidence:0.999, region:Albania, direction:entry, snapshotId}). Build+lint green. Updated [[opencv-anpr-service]].