mca/parking_solution
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 10 Wiki Activity

feat(vision): persist every recognized plate + snapshot as telemetry (non-blocking)

Browse Source 
Answers "is a recognized plate saved?" — now yes, for both transient and subscriber, as
an ANPR audit trail independent of whether it matched anything.

VisionReader now stores the snapshot bytes in `snapshots` keyed by identity=PLATE — the
same identity the flow signs its anomaly/event with — so GET /api/snapshots/by-identity/:plate
(the booth event-detail modal's snapshot strip) shows the car's photo against that
anomaly with no UI changes. It also records an unsigned device_events{kind:"read"}
breadcrumb (plate, confidence, region, model, snapshotId, and the dispatch outcome) as a
queryable recognition log. Switched from emitRead to calling ReadDispatcher.dispatch
directly (like qr-reader) to capture that outcome.

Non-blocking: a refused read (no session / unpaid / unknown plate) just returns
rejected — no barrier hold — and is logged with its snapshot for investigation. Plate
stays advisory (exit demands payment; subscription matches only a bound plate).

Verified e2e: a recognized AL plate with no open session signed exit.refused.noSession
(identity=plate), stored a 555KB snapshot under that plate, recorded the read breadcrumb
(accepted:false, reason "no open session"), and by-identity returned the image — the
refused read is fully investigable with its picture. Build + lint green.

Claude-Session: https://claude.ai/code/session_01Xcm6ikLgGoCxxHrxtjkk5V
This commit is contained in:
Julian Çuni
2026-06-19 16:29:10 +02:00
parent 7e086ff0d7
commit 540b333b06
4 changed files with 157 additions and 25 deletions
Download Patch File Download Diff File Expand all files Collapse all files
apps/server/src/server.ts
+1 -1
View File
@@ -177,7 +177,7 @@ export async function buildServer(opts: BuildOptions = {}): Promise<FastifyInsta
// reader uses → the dispatcher routes it to the subscription/exit flow unchanged. A
// plate stays advisory: the exit flow still demands a payment, the subscription flow
// only matches a BOUND plate. Idle when vision is disabled or no camera opts in.
const visionReader = new VisionReader(db, visionClient, app.log);
const visionReader = new VisionReader(db, visionClient, readDispatcher, app.log);
app.addHook("onReady", async () => visionReader.start());
app.addHook("onClose", async () => visionReader.stop());
apps/server/src/vision-reader.ts
+132 -16
View File
@@ -1,18 +1,37 @@
import { devices, eq, type Db, type DeviceRow } from "@parking/db";
import { randomUUID } from "node:crypto";
import {
deviceEvents as deviceEventsTable,
devices,
eq,
snapshots,
type Db,
type DeviceRow,
} from "@parking/db";
import { registry, type CameraDevice } from "@parking/devices";
import type { FastifyBaseLogger } from "fastify";
import { deviceEvents } from "./device-events.js";
import { directionOf } from "./device-resolve.js";
import { directionOf, type FlowDirection } from "./device-resolve.js";
import type { ReadDispatcher } from "./read-dispatch.js";
import type { VisionClient } from "./vision-client.js";
// VisionReader — turns an ANPR camera into a virtual plate READER. It polls each
// opt-in camera, sends a snapshot to the VisionClient (apps/vision), and on a CONFIDENT
// plate emits a `DeviceReadEvent{kind:"plate"}` onto the read bus — the SAME path a
// physical reader's scan takes, so the subscription/exit flows consume it unchanged.
// plate dispatches a `DeviceReadEvent{kind:"plate"}` through the SAME ReadDispatcher a
// physical reader's scan uses, so the subscription/exit flows consume it unchanged.
//
// EVERY confident read is PERSISTED for investigation (the ANPR audit trail):
// - the SNAPSHOT bytes are stored in `snapshots` keyed by `identity = plate` — the
// SAME identity the flow signs its anomaly/event with — so the booth event-detail
// modal's snapshot strip (GET /api/snapshots/by-identity/:identity) shows the car's
// photo against that anomaly with NO extra wiring. This is what makes a refused
// plate read investigable ("which car was this?").
// - a `device_events` breadcrumb (kind:"read") records plate/confidence/region +
// the dispatch OUTCOME (accepted + reason), so there's a queryable log of every
// recognition and whether it matched, independent of the signed ledger.
//
// Per the fitness assessment (wiki/entities/opencv-anpr-service.md): a plate read is an
// ADVISORY identity + evidence, never the sole authority to open a paid barrier. The
// guards that keep it advisory live below the recognition, in the flows it feeds:
// ADVISORY identity + evidence, never the sole authority to open a paid barrier, and it
// NEVER BLOCKS — recognition runs alongside the flow; a refused read is logged with its
// snapshot, not a barrier hold. The guards that keep it advisory live in the flows:
// - the EXIT flow still requires a covering `payment` (a plate can't bypass it);
// - the SUBSCRIPTION flow only matches a plate BOUND to a subscription (subscriptionPlates).
// So a recognized plate that owes money is refused exactly like a scanned ticket would be.
@@ -22,7 +41,7 @@ import type { VisionClient } from "./vision-client.js";
// default). The VisionClient itself is also opt-in (VISION_ENABLED) and fail-soft.
// - DEBOUNCE: a parked car sits in frame across many polls; the same plate from the
// same camera is NOT re-emitted within `dedupeMs` (avoids a storm of identical reads).
// - LOW-CONFIDENCE reads are dropped (not emitted) — a shaky read must not act as an
// - LOW-CONFIDENCE reads are dropped (not dispatched) — a shaky read must not act as an
// identity; the camera keeps polling until a confident frame (or the car leaves).
const POLL_MS = Number(process.env.VISION_POLL_MS ?? 2000);
@@ -36,6 +55,7 @@ interface CameraConfig {
export class VisionReader {
readonly #db: Db;
readonly #vision: VisionClient;
readonly #dispatcher: ReadDispatcher;
readonly #logger: FastifyBaseLogger;
readonly #pollMs: number;
readonly #dedupeMs: number;
@@ -45,9 +65,17 @@ export class VisionReader {
/** Last emitted plate + time per camera, for debounce. */
readonly #lastEmit = new Map<string, { value: string; at: number }>();
constructor(db: Db, vision: VisionClient, logger: FastifyBaseLogger, pollMs = POLL_MS, dedupeMs = DEDUPE_MS) {
constructor(
db: Db,
vision: VisionClient,
dispatcher: ReadDispatcher,
logger: FastifyBaseLogger,
pollMs = POLL_MS,
dedupeMs = DEDUPE_MS,
) {
this.#db = db;
this.#vision = vision;
this.#dispatcher = dispatcher;
this.#logger = logger;
this.#pollMs = pollMs;
this.#dedupeMs = dedupeMs;
@@ -104,7 +132,7 @@ export class VisionReader {
const direction = dir === "exit" ? "exit" : "entry"; // "both" → entry context
const shot = await camera.captureSnapshot({ direction });
const result = await this.#vision.analyze(shot.bytes, shot.contentType);
// Fail-soft: null (disabled/unreachable/timeout) or no plate ⇒ nothing to emit.
// Fail-soft: null (disabled/unreachable/timeout) or no plate ⇒ nothing to do.
if (!result || !result.plate) return;
// Advisory gate: a low-confidence read is NOT an identity — drop it.
if (result.lowConfidence) {
@@ -116,16 +144,36 @@ export class VisionReader {
if (this.#isDuplicate(row.id, plate)) return; // same car still in frame
this.#lastEmit.set(row.id, { value: plate, at: Date.now() });
// Emit onto the read bus — the SAME event a physical plate reader would send, so
// the ReadDispatcher routes it to the subscription/exit flow unchanged.
deviceEvents.emitRead({
// PERSIST the snapshot keyed by `identity = plate` — the same identity the flow
// will sign its anomaly/event with — so the image is investigable against it.
const snapshotId = this.#storeSnapshot(row.id, direction, plate, shot.bytes, shot.contentType);
this.#logger.info(`vision plate '${plate}' (${result.plate.confidence.toFixed(3)}) from camera ${row.id}`);
// Dispatch the read through the SAME path a physical reader uses (like qr-reader),
// capturing the OUTCOME. A refused read does NOT block — it just returns rejected;
// we log it (with its snapshot already stored) for investigation.
const read = {
driverId: row.driverId,
deviceId: row.id,
value: plate,
kind: "plate",
kind: "plate" as const,
at: new Date().toISOString(),
});
this.#logger.info(`vision plate '${plate}' (${result.plate.confidence.toFixed(3)}) from camera ${row.id}`);
};
let outcome: { accepted: boolean; direction?: string; reason?: string };
try {
outcome = await this.#dispatcher.dispatch(read);
} catch (err) {
outcome = { accepted: false, reason: (err as Error).message };
}
// Breadcrumb: a queryable record of the recognition + what the flow did with it.
this.#recordReadEvent(row, direction, plate, result, snapshotId, outcome);
if (!outcome.accepted) {
this.#logger.info(
`vision plate '${plate}' not accepted (${outcome.reason ?? "rejected"}) — logged with snapshot ${snapshotId ?? "none"}`,
);
}
} catch (err) {
// Never let a camera/recognition error break the poll loop.
this.#logger.warn(`vision reader ${row.id} failed: ${(err as Error).message}`);
@@ -134,6 +182,74 @@ export class VisionReader {
}
}
/** Store the recognition snapshot keyed by `identity = plate`. Returns the snapshot
* id, or null on failure (persistence is best-effort — never blocks the flow). */
#storeSnapshot(
cameraId: string,
direction: FlowDirection,
plate: string,
bytes: Buffer,
contentType: string,
): string | null {
try {
const id = randomUUID();
this.#db
.insert(snapshots)
.values({
id,
direction,
deviceId: cameraId,
identity: plate,
contentType,
bytes,
capturedAt: new Date().toISOString(),
})
.run();
return id;
} catch (err) {
this.#logger.error(`vision snapshot store failed (${plate}): ${(err as Error).message}`);
return null;
}
}
/** Record an unsigned `read` device_event: the ANPR audit trail (plate, confidence,
* region, the stored snapshot id, and the flow's outcome). Best-effort telemetry. */
#recordReadEvent(
row: DeviceRow,
direction: FlowDirection,
plate: string,
result: { plate: { confidence: number; region?: string | null } | null; modelVersion: string },
snapshotId: string | null,
outcome: { accepted: boolean; direction?: string; reason?: string },
): void {
try {
this.#db
.insert(deviceEventsTable)
.values({
id: randomUUID(),
deviceId: row.id,
category: "camera",
kind: "read",
detail: {
driverId: row.driverId,
plate,
confidence: result.plate?.confidence,
region: result.plate?.region ?? null,
modelVersion: result.modelVersion,
direction,
snapshotId,
accepted: outcome.accepted,
outcomeDirection: outcome.direction,
reason: outcome.reason,
},
occurredAt: new Date().toISOString(),
})
.run();
} catch (err) {
this.#logger.error(`vision read-event insert failed (${plate}): ${(err as Error).message}`);
}
}
/** Debounce: true if this same plate was emitted from this camera within dedupeMs. */
#isDuplicate(cameraId: string, plate: string): boolean {
const last = this.#lastEmit.get(cameraId);