refactor(vision): ANPR rides the entry/exit snapshot, drop polling reader
Rework the ANPR trigger to the real design: when a transient presses the button or a
subscriber passes QR/RFID, the entry/exit fires and takes its evidence snapshot — that
is the moment to recognize. snapshotAsync now takes the VisionClient and, after storing
each snapshot from an opt-in (config.anpr) camera, runs ANPR on the SAME image and
records the plate against the SAME session identity (device_events kind:"read" with
plate/confidence/region/snapshotId/source:"entry-exit-snapshot"). One image serves both
evidence and plate extraction; recognition fires only on a real entry/exit — no polling.
The entry/exit/subscription flows take an optional VisionClient and pass it through;
server.ts wires it. Removed the polling VisionReader and VISION_POLL_MS/VISION_DEDUPE_MS.
Advisory + fire-and-forget: a low-confidence/no-plate result records nothing, a vision
failure never delays or changes the open, and the plate does not feed the access
decision. Verified e2e: a simulated entry snapshot on an anpr camera (live fast_alpr)
stored the snapshot for the session and recorded {identity, plate:AA558EE, 0.999,
region:Albania, snapshotId}. Build + lint green.
Claude-Session: https://claude.ai/code/session_01Xcm6ikLgGoCxxHrxtjkk5V
This commit is contained in:
@@ -35,9 +35,8 @@ WS_ALLOWED_ORIGINS=http://localhost:5173
|
||||
# which runs as a separate process with its OWN apps/vision/.env. Both sides share the
|
||||
# VISION_ prefix but are different processes — keep the two .env files separate.
|
||||
# See wiki/entities/opencv-anpr-service.md "Configuration".
|
||||
# ANPR rides the entry/exit snapshot (button / QR / RFID triggers it) — no polling.
|
||||
# VISION_ENABLED=1 # master switch — nothing runs without it
|
||||
# VISION_URL=http://127.0.0.1:8089 # must match apps/vision VISION_HOST:VISION_PORT
|
||||
# VISION_TIMEOUT_MS=1500 # per-request cap so a slow call can't hang the lane
|
||||
# VISION_POLL_MS=2000 # how often each anpr camera is polled
|
||||
# VISION_DEDUPE_MS=15000 # suppress re-firing the same plate while a car sits in frame
|
||||
# VISION_MIN_CONFIDENCE=0.5 # confidence floor; keep in sync with the service
|
||||
|
||||
@@ -17,6 +17,7 @@ import { getOccupancy } from "./occupancy.js";
|
||||
import type { EventLog } from "./event-log.js";
|
||||
import { devicesByDirection, relayForButton, relayForPresence, type ResolvedRelay } from "./device-resolve.js";
|
||||
import { snapshotAsync } from "./snapshot.js";
|
||||
import type { VisionClient } from "./vision-client.js";
|
||||
|
||||
// The transient ENTRY flow: a button press → print a ticket → sign a vehicle_entry
|
||||
// → open the barrier. The button is wired into an access controller's input; the
|
||||
@@ -69,11 +70,14 @@ export class EntryFlow {
|
||||
readonly #inFlight = new Set<string>();
|
||||
/** Per-relay one-car-one-ticket state (presence + cooldown), keyed controllerId:relay. */
|
||||
readonly #guard = new Map<string, RelayGuardState>();
|
||||
/** Optional vision client — passed to snapshotAsync so ANPR runs on the entry image. */
|
||||
readonly #vision: VisionClient | null;
|
||||
|
||||
constructor(db: Db, log: EventLog, logger: FastifyBaseLogger) {
|
||||
constructor(db: Db, log: EventLog, logger: FastifyBaseLogger, vision: VisionClient | null = null) {
|
||||
this.#db = db;
|
||||
this.#log = log;
|
||||
this.#logger = logger;
|
||||
this.#vision = vision;
|
||||
}
|
||||
|
||||
/** Handle a device input edge. Two kinds of edge matter to this flow:
|
||||
@@ -308,8 +312,8 @@ export class EntryFlow {
|
||||
* Used on both the OPEN path and the refused/held anomaly paths — a turned-away or
|
||||
* held car is exactly when the operator wants the photo. */
|
||||
#fireSnapshot(direction: "entry", identity: string): void {
|
||||
void snapshotAsync({ db: this.#db, direction, identity, logger: this.#logger }).catch((err) =>
|
||||
this.#logger.error(`entry snapshot error: ${(err as Error).message}`),
|
||||
void snapshotAsync({ db: this.#db, direction, identity, logger: this.#logger, vision: this.#vision }).catch(
|
||||
(err) => this.#logger.error(`entry snapshot error: ${(err as Error).message}`),
|
||||
);
|
||||
}
|
||||
|
||||
|
||||
@@ -2,6 +2,7 @@ import { desc, eq, ledgerEvents, sessions, tariffVersions, tariffs, type Db, typ
|
||||
import { registry, type AccessControlDevice } from "@parking/devices";
|
||||
import { firstRelayByDirection, type ResolvedRelay } from "./device-resolve.js";
|
||||
import { snapshotAsync } from "./snapshot.js";
|
||||
import type { VisionClient } from "./vision-client.js";
|
||||
import { computeFee, reasonPayload, renderReasonEn, type LedgerPayload, type TariffStructure } from "@parking/shared";
|
||||
import type { FastifyBaseLogger } from "fastify";
|
||||
import type { DeviceReadEvent, ReadOutcome } from "./device-events.js";
|
||||
@@ -61,11 +62,14 @@ export class ExitFlow {
|
||||
readonly #log: EventLog;
|
||||
readonly #logger: FastifyBaseLogger;
|
||||
readonly #inFlight = new Set<string>();
|
||||
/** Optional vision client — passed to snapshotAsync so ANPR runs on the exit image. */
|
||||
readonly #vision: VisionClient | null;
|
||||
|
||||
constructor(db: Db, log: EventLog, logger: FastifyBaseLogger) {
|
||||
constructor(db: Db, log: EventLog, logger: FastifyBaseLogger, vision: VisionClient | null = null) {
|
||||
this.#db = db;
|
||||
this.#log = log;
|
||||
this.#logger = logger;
|
||||
this.#vision = vision;
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -364,6 +368,7 @@ export class ExitFlow {
|
||||
direction: "exit",
|
||||
identity,
|
||||
logger: this.#logger,
|
||||
vision: this.#vision,
|
||||
}).catch((err) => this.#logger.error(`exit snapshot error: ${(err as Error).message}`));
|
||||
}
|
||||
|
||||
|
||||
@@ -20,7 +20,6 @@ import { buildSigner, buildVerifier } from "./signer.js";
|
||||
import { LogService, pinoDbStream } from "./log-service.js";
|
||||
import { logRoutes } from "./routes/logs.js";
|
||||
import { VisionClient } from "./vision-client.js";
|
||||
import { VisionReader } from "./vision-reader.js";
|
||||
import { authRoutes } from "./routes/auth.js";
|
||||
import { userRoutes } from "./routes/users.js";
|
||||
import { roleRoutes } from "./routes/roles.js";
|
||||
@@ -151,7 +150,12 @@ export async function buildServer(opts: BuildOptions = {}): Promise<FastifyInsta
|
||||
// Subscribes to the SAME input bus as the telemetry writer below; the two are
|
||||
// independent (telemetry always records; the entry flow acts only on an access
|
||||
// device's rising edge). See wiki/concepts/device-input-flow.md + parking-session.md.
|
||||
const entryFlow = new EntryFlow(db, eventLog, app.log);
|
||||
// The flows take the vision client so ANPR rides their entry/exit SNAPSHOT: a button
|
||||
// press / QR / RFID triggers the open + snapshot, and the plate is recognized off that
|
||||
// same image and recorded against the session (advisory; never changes the decision).
|
||||
// No polling — recognition fires only on a real entry/exit. See snapshot.ts +
|
||||
// wiki/entities/opencv-anpr-service.md.
|
||||
const entryFlow = new EntryFlow(db, eventLog, app.log, visionClient);
|
||||
const unsubscribeEntry = deviceEvents.onInput((e) => {
|
||||
void entryFlow.onInput(e);
|
||||
});
|
||||
@@ -161,23 +165,14 @@ export async function buildServer(opts: BuildOptions = {}): Promise<FastifyInsta
|
||||
// dispatcher to either the SUBSCRIPTION flow (if it matches a subscription) or the
|
||||
// transient EXIT flow. See read-dispatch.ts, exit-flow.ts, subscription-flow.ts,
|
||||
// parking-session.md.
|
||||
const exitFlow = new ExitFlow(db, eventLog, app.log);
|
||||
const subscriptionFlow = new SubscriptionFlow(db, eventLog, app.log);
|
||||
const exitFlow = new ExitFlow(db, eventLog, app.log, visionClient);
|
||||
const subscriptionFlow = new SubscriptionFlow(db, eventLog, app.log, visionClient);
|
||||
const readDispatcher = new ReadDispatcher(db, exitFlow, subscriptionFlow, app.log);
|
||||
const unsubscribeRead = deviceEvents.onRead((e) => {
|
||||
void readDispatcher.dispatch(e);
|
||||
});
|
||||
app.addHook("onClose", async () => unsubscribeRead());
|
||||
|
||||
// Vision READER: polls opt-in (config.anpr) cameras, recognizes a plate via the
|
||||
// vision client, and emits a kind:"plate" read onto the SAME read bus a physical
|
||||
// reader uses → the dispatcher routes it to the subscription/exit flow unchanged. A
|
||||
// plate stays advisory: the exit flow still demands a payment, the subscription flow
|
||||
// only matches a BOUND plate. Idle when vision is disabled or no camera opts in.
|
||||
const visionReader = new VisionReader(db, visionClient, readDispatcher, app.log);
|
||||
app.addHook("onReady", async () => visionReader.start());
|
||||
app.addHook("onClose", async () => visionReader.stop());
|
||||
|
||||
// Credential capture ("enroll a card"): lets the operator present an RFID card to a
|
||||
// CHOSEN reader to populate a subscription credential, without blocking the other
|
||||
// reader's live flow. Single-shot + TTL. See credential-capture.ts.
|
||||
|
||||
@@ -3,6 +3,7 @@ import { deviceEvents as deviceEventsTable, snapshots, type Db } from "@parking/
|
||||
import { registry, type CameraDevice } from "@parking/devices";
|
||||
import type { FastifyBaseLogger } from "fastify";
|
||||
import { devicesByDirection, type FlowDirection } from "./device-resolve.js";
|
||||
import type { VisionClient } from "./vision-client.js";
|
||||
|
||||
// Camera snapshot capture, fired AFTER the barrier opens and never awaited on the
|
||||
// open path (decision 2026-06-16): a snapshot is EVIDENCE, not a gate. A camera
|
||||
@@ -15,6 +16,16 @@ import { devicesByDirection, type FlowDirection } from "./device-resolve.js";
|
||||
// → a `snapshots` row + a `kind:"snapshot"` telemetry device_event; a failure → a
|
||||
// telemetry device_event only. The caller passes the session `identity` so the image
|
||||
// links to the signed vehicle_entry/exit.
|
||||
//
|
||||
// ANPR rides this snapshot (2026-06-19). A transient button-press or a subscriber
|
||||
// QR/RFID read triggers the entry/exit, which fires THIS snapshot — that is exactly the
|
||||
// moment to recognize the plate, off the SAME image, tied to the SAME session identity.
|
||||
// So when a `vision` client is passed AND the camera opts in (config.anpr), each stored
|
||||
// snapshot is sent to the vision service and the extracted plate is RECORDED against the
|
||||
// session (a `kind:"read"` device_event with plate/confidence/snapshotId). ADVISORY +
|
||||
// fire-and-forget: it never blocks the open and never changes the entry/exit decision —
|
||||
// it's a record ("session X entered on plate AA558EE"). No polling; recognition only
|
||||
// happens on a real entry/exit. See wiki/entities/opencv-anpr-service.md.
|
||||
|
||||
interface SnapshotJob {
|
||||
readonly db: Db;
|
||||
@@ -22,6 +33,15 @@ interface SnapshotJob {
|
||||
/** Session/credential ref (ticket id, plate, subscription car key) — links to the ledger. */
|
||||
readonly identity: string;
|
||||
readonly logger: FastifyBaseLogger;
|
||||
/** Optional vision client — when present, ANPR runs on each captured image from an
|
||||
* `anpr`-enabled camera and records the plate against `identity`. Advisory only. */
|
||||
readonly vision?: VisionClient | null;
|
||||
}
|
||||
|
||||
/** Camera config flag opting it into snapshot-triggered ANPR. */
|
||||
interface CameraConfig {
|
||||
readonly anpr?: boolean;
|
||||
readonly [k: string]: unknown;
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -30,7 +50,7 @@ interface SnapshotJob {
|
||||
* The caller must NOT block its open path on this.
|
||||
*/
|
||||
export function snapshotAsync(job: SnapshotJob): Promise<string[]> {
|
||||
const { db, direction, identity, logger } = job;
|
||||
const { db, direction, identity, logger, vision } = job;
|
||||
const rows = devicesByDirection(db, "camera", direction);
|
||||
if (rows.length === 0) return Promise.resolve([]);
|
||||
|
||||
@@ -57,6 +77,12 @@ export function snapshotAsync(job: SnapshotJob): Promise<string[]> {
|
||||
.run();
|
||||
// Telemetry breadcrumb pointing at the stored image (NOT the bytes).
|
||||
recordEvent(db, direction, row.id, identity, { snapshotId: id, ok: true }, logger);
|
||||
|
||||
// ANPR off the SAME image, tied to the SAME session — when vision is enabled
|
||||
// and this camera opts in. Fire-and-forget: never delays the open path.
|
||||
if (vision?.enabled && (row.config as CameraConfig)?.anpr === true) {
|
||||
void recognizePlate(db, vision, row.id, direction, identity, id, shot, logger);
|
||||
}
|
||||
return id;
|
||||
} catch (err) {
|
||||
recordFailure(db, direction, row.id, identity, (err as Error).message, logger);
|
||||
@@ -66,6 +92,55 @@ export function snapshotAsync(job: SnapshotJob): Promise<string[]> {
|
||||
).then((ids) => ids.filter((id): id is string => id != null));
|
||||
}
|
||||
|
||||
/**
|
||||
* Recognize the plate off a captured entry/exit image and RECORD it against the session
|
||||
* `identity` — an unsigned `kind:"read"` device_event carrying the plate, confidence,
|
||||
* region, and the `snapshotId` it was read from. Advisory: this records the plate
|
||||
* observed for the session; it does NOT feed the access decision (the flow already
|
||||
* decided). A low-confidence/no-plate result records nothing (a shaky read isn't a fact).
|
||||
* Best-effort + fail-soft — a vision error never surfaces on the (already-open) path.
|
||||
*/
|
||||
async function recognizePlate(
|
||||
db: Db,
|
||||
vision: VisionClient,
|
||||
deviceId: string,
|
||||
direction: FlowDirection,
|
||||
identity: string,
|
||||
snapshotId: string,
|
||||
shot: { bytes: Buffer; contentType: string },
|
||||
logger: FastifyBaseLogger,
|
||||
): Promise<void> {
|
||||
try {
|
||||
const result = await vision.analyze(shot.bytes, shot.contentType);
|
||||
if (!result || !result.plate || result.lowConfidence) return; // nothing trustworthy to record
|
||||
const plate = result.plate.text.trim().toUpperCase();
|
||||
if (!plate) return;
|
||||
db.insert(deviceEventsTable)
|
||||
.values({
|
||||
id: randomUUID(),
|
||||
deviceId,
|
||||
category: "camera",
|
||||
kind: "read",
|
||||
// `identity` ties the plate to the session; `snapshotId` to the evidence image.
|
||||
detail: {
|
||||
identity,
|
||||
direction,
|
||||
plate,
|
||||
confidence: result.plate.confidence,
|
||||
region: result.plate.region ?? null,
|
||||
modelVersion: result.modelVersion,
|
||||
snapshotId,
|
||||
source: "entry-exit-snapshot",
|
||||
},
|
||||
occurredAt: new Date().toISOString(),
|
||||
})
|
||||
.run();
|
||||
logger.info(`anpr plate '${plate}' (${result.plate.confidence.toFixed(3)}) for ${identity}`);
|
||||
} catch (err) {
|
||||
logger.warn(`anpr recognize failed (${identity}): ${(err as Error).message}`);
|
||||
}
|
||||
}
|
||||
|
||||
/** Build a live camera adapter from a resolved devices row, or null. */
|
||||
function buildCamera(row: { driverId: string; config: unknown }): CameraDevice | null {
|
||||
const driver = registry.get(row.driverId);
|
||||
|
||||
@@ -16,6 +16,7 @@ import type { DeviceReadEvent, ReadOutcome } from "./device-events.js";
|
||||
import type { EventLog } from "./event-log.js";
|
||||
import { type FlowDirection, type ResolvedRelay } from "./device-resolve.js";
|
||||
import { snapshotAsync } from "./snapshot.js";
|
||||
import type { VisionClient } from "./vision-client.js";
|
||||
|
||||
// SUBSCRIPTION flow: a subscriber identified by card/QR/plate enters/exits without
|
||||
// paying per stay (they're on a recurring plan). Reached from the read dispatcher
|
||||
@@ -55,11 +56,14 @@ export class SubscriptionFlow {
|
||||
readonly #log: EventLog;
|
||||
readonly #logger: FastifyBaseLogger;
|
||||
readonly #inFlight = new Set<string>();
|
||||
/** Optional vision client — passed to snapshotAsync so ANPR runs on the subscriber image. */
|
||||
readonly #vision: VisionClient | null;
|
||||
|
||||
constructor(db: Db, log: EventLog, logger: FastifyBaseLogger) {
|
||||
constructor(db: Db, log: EventLog, logger: FastifyBaseLogger, vision: VisionClient | null = null) {
|
||||
this.#db = db;
|
||||
this.#log = log;
|
||||
this.#logger = logger;
|
||||
this.#vision = vision;
|
||||
}
|
||||
|
||||
/** Resolve a read to a subscription (by card/QR credential, or a bound plate), or null. */
|
||||
@@ -256,8 +260,8 @@ export class SubscriptionFlow {
|
||||
/** Fire the directional camera(s) for a refused-subscription event; never awaited
|
||||
* (evidence, not a gate). The accepted entry/exit paths snapshot inside #open. */
|
||||
#fireSnapshot(dir: FlowDirection, identity: string): void {
|
||||
void snapshotAsync({ db: this.#db, direction: dir, identity, logger: this.#logger }).catch((err) =>
|
||||
this.#logger.error(`subscription snapshot error: ${(err as Error).message}`),
|
||||
void snapshotAsync({ db: this.#db, direction: dir, identity, logger: this.#logger, vision: this.#vision }).catch(
|
||||
(err) => this.#logger.error(`subscription snapshot error: ${(err as Error).message}`),
|
||||
);
|
||||
}
|
||||
|
||||
|
||||
@@ -1,269 +0,0 @@
|
||||
import { randomUUID } from "node:crypto";
|
||||
import {
|
||||
deviceEvents as deviceEventsTable,
|
||||
devices,
|
||||
eq,
|
||||
snapshots,
|
||||
type Db,
|
||||
type DeviceRow,
|
||||
} from "@parking/db";
|
||||
import { registry, type CameraDevice } from "@parking/devices";
|
||||
import type { FastifyBaseLogger } from "fastify";
|
||||
import { directionOf, type FlowDirection } from "./device-resolve.js";
|
||||
import type { ReadDispatcher } from "./read-dispatch.js";
|
||||
import type { VisionClient } from "./vision-client.js";
|
||||
|
||||
// VisionReader — turns an ANPR camera into a virtual plate READER. It polls each
|
||||
// opt-in camera, sends a snapshot to the VisionClient (apps/vision), and on a CONFIDENT
|
||||
// plate dispatches a `DeviceReadEvent{kind:"plate"}` through the SAME ReadDispatcher a
|
||||
// physical reader's scan uses, so the subscription/exit flows consume it unchanged.
|
||||
//
|
||||
// EVERY confident read is PERSISTED for investigation (the ANPR audit trail):
|
||||
// - the SNAPSHOT bytes are stored in `snapshots` keyed by `identity = plate` — the
|
||||
// SAME identity the flow signs its anomaly/event with — so the booth event-detail
|
||||
// modal's snapshot strip (GET /api/snapshots/by-identity/:identity) shows the car's
|
||||
// photo against that anomaly with NO extra wiring. This is what makes a refused
|
||||
// plate read investigable ("which car was this?").
|
||||
// - a `device_events` breadcrumb (kind:"read") records plate/confidence/region +
|
||||
// the dispatch OUTCOME (accepted + reason), so there's a queryable log of every
|
||||
// recognition and whether it matched, independent of the signed ledger.
|
||||
//
|
||||
// Per the fitness assessment (wiki/entities/opencv-anpr-service.md): a plate read is an
|
||||
// ADVISORY identity + evidence, never the sole authority to open a paid barrier, and it
|
||||
// NEVER BLOCKS — recognition runs alongside the flow; a refused read is logged with its
|
||||
// snapshot, not a barrier hold. The guards that keep it advisory live in the flows:
|
||||
// - the EXIT flow still requires a covering `payment` (a plate can't bypass it);
|
||||
// - the SUBSCRIPTION flow only matches a plate BOUND to a subscription (subscriptionPlates).
|
||||
// So a recognized plate that owes money is refused exactly like a scanned ticket would be.
|
||||
//
|
||||
// Opt-in + safety:
|
||||
// - PER-CAMERA opt-in: only cameras whose config has `anpr: true` are polled (off by
|
||||
// default). The VisionClient itself is also opt-in (VISION_ENABLED) and fail-soft.
|
||||
// - DEBOUNCE: a parked car sits in frame across many polls; the same plate from the
|
||||
// same camera is NOT re-emitted within `dedupeMs` (avoids a storm of identical reads).
|
||||
// - LOW-CONFIDENCE reads are dropped (not dispatched) — a shaky read must not act as an
|
||||
// identity; the camera keeps polling until a confident frame (or the car leaves).
|
||||
|
||||
const POLL_MS = Number(process.env.VISION_POLL_MS ?? 2000);
|
||||
const DEDUPE_MS = Number(process.env.VISION_DEDUPE_MS ?? 15_000);
|
||||
|
||||
interface CameraConfig {
|
||||
readonly anpr?: boolean;
|
||||
readonly [k: string]: unknown;
|
||||
}
|
||||
|
||||
export class VisionReader {
|
||||
readonly #db: Db;
|
||||
readonly #vision: VisionClient;
|
||||
readonly #dispatcher: ReadDispatcher;
|
||||
readonly #logger: FastifyBaseLogger;
|
||||
readonly #pollMs: number;
|
||||
readonly #dedupeMs: number;
|
||||
#timer: ReturnType<typeof setInterval> | null = null;
|
||||
/** In-flight guard per camera so a slow recognize doesn't overlap its own next tick. */
|
||||
readonly #busy = new Set<string>();
|
||||
/** Last emitted plate + time per camera, for debounce. */
|
||||
readonly #lastEmit = new Map<string, { value: string; at: number }>();
|
||||
|
||||
constructor(
|
||||
db: Db,
|
||||
vision: VisionClient,
|
||||
dispatcher: ReadDispatcher,
|
||||
logger: FastifyBaseLogger,
|
||||
pollMs = POLL_MS,
|
||||
dedupeMs = DEDUPE_MS,
|
||||
) {
|
||||
this.#db = db;
|
||||
this.#vision = vision;
|
||||
this.#dispatcher = dispatcher;
|
||||
this.#logger = logger;
|
||||
this.#pollMs = pollMs;
|
||||
this.#dedupeMs = dedupeMs;
|
||||
}
|
||||
|
||||
start(): void {
|
||||
if (this.#timer) return;
|
||||
// Only run when vision is enabled AND at least one camera opts in — otherwise the
|
||||
// timer is pure overhead. We still re-check enabled per tick (config can change).
|
||||
if (!this.#vision.enabled) {
|
||||
this.#logger.info("vision reader idle (VISION_ENABLED off)");
|
||||
return;
|
||||
}
|
||||
this.#timer = setInterval(() => void this.#tick(), this.#pollMs);
|
||||
this.#timer.unref?.();
|
||||
this.#logger.info(`vision reader polling anpr cameras every ${this.#pollMs}ms`);
|
||||
}
|
||||
|
||||
stop(): void {
|
||||
if (this.#timer) {
|
||||
clearInterval(this.#timer);
|
||||
this.#timer = null;
|
||||
}
|
||||
}
|
||||
|
||||
/** One poll cycle: recognize on every opt-in camera, concurrently. Never throws. */
|
||||
async #tick(): Promise<void> {
|
||||
const cams = this.#anprCameras();
|
||||
if (cams.length === 0) return;
|
||||
await Promise.all(cams.map((c) => this.#recognizeOn(c)));
|
||||
}
|
||||
|
||||
/** Enabled cameras with `config.anpr === true`. */
|
||||
#anprCameras(): DeviceRow[] {
|
||||
return this.#db
|
||||
.select()
|
||||
.from(devices)
|
||||
.where(eq(devices.category, "camera"))
|
||||
.all()
|
||||
.filter((r) => r.enabled && (r.config as CameraConfig)?.anpr === true);
|
||||
}
|
||||
|
||||
/**
|
||||
* Capture from one camera → recognize → maybe emit a plate read. Public so a future
|
||||
* on-demand trigger (a loop edge, an API call) can call it directly, not just the poll.
|
||||
*/
|
||||
async #recognizeOn(row: DeviceRow): Promise<void> {
|
||||
if (this.#busy.has(row.id)) return; // skip if its previous recognize is still running
|
||||
this.#busy.add(row.id);
|
||||
try {
|
||||
const camera = this.#buildCamera(row);
|
||||
if (!camera) return;
|
||||
const dir = directionOf(this.#db, row);
|
||||
const direction = dir === "exit" ? "exit" : "entry"; // "both" → entry context
|
||||
const shot = await camera.captureSnapshot({ direction });
|
||||
const result = await this.#vision.analyze(shot.bytes, shot.contentType);
|
||||
// Fail-soft: null (disabled/unreachable/timeout) or no plate ⇒ nothing to do.
|
||||
if (!result || !result.plate) return;
|
||||
// Advisory gate: a low-confidence read is NOT an identity — drop it.
|
||||
if (result.lowConfidence) {
|
||||
this.#logger.debug(`vision low-confidence plate '${result.plate.text}' on ${row.id} — dropped`);
|
||||
return;
|
||||
}
|
||||
const plate = result.plate.text.trim().toUpperCase();
|
||||
if (!plate) return;
|
||||
if (this.#isDuplicate(row.id, plate)) return; // same car still in frame
|
||||
this.#lastEmit.set(row.id, { value: plate, at: Date.now() });
|
||||
|
||||
// PERSIST the snapshot keyed by `identity = plate` — the same identity the flow
|
||||
// will sign its anomaly/event with — so the image is investigable against it.
|
||||
const snapshotId = this.#storeSnapshot(row.id, direction, plate, shot.bytes, shot.contentType);
|
||||
|
||||
this.#logger.info(`vision plate '${plate}' (${result.plate.confidence.toFixed(3)}) from camera ${row.id}`);
|
||||
|
||||
// Dispatch the read through the SAME path a physical reader uses (like qr-reader),
|
||||
// capturing the OUTCOME. A refused read does NOT block — it just returns rejected;
|
||||
// we log it (with its snapshot already stored) for investigation.
|
||||
const read = {
|
||||
driverId: row.driverId,
|
||||
deviceId: row.id,
|
||||
value: plate,
|
||||
kind: "plate" as const,
|
||||
at: new Date().toISOString(),
|
||||
};
|
||||
let outcome: { accepted: boolean; direction?: string; reason?: string };
|
||||
try {
|
||||
outcome = await this.#dispatcher.dispatch(read);
|
||||
} catch (err) {
|
||||
outcome = { accepted: false, reason: (err as Error).message };
|
||||
}
|
||||
|
||||
// Breadcrumb: a queryable record of the recognition + what the flow did with it.
|
||||
this.#recordReadEvent(row, direction, plate, result, snapshotId, outcome);
|
||||
if (!outcome.accepted) {
|
||||
this.#logger.info(
|
||||
`vision plate '${plate}' not accepted (${outcome.reason ?? "rejected"}) — logged with snapshot ${snapshotId ?? "none"}`,
|
||||
);
|
||||
}
|
||||
} catch (err) {
|
||||
// Never let a camera/recognition error break the poll loop.
|
||||
this.#logger.warn(`vision reader ${row.id} failed: ${(err as Error).message}`);
|
||||
} finally {
|
||||
this.#busy.delete(row.id);
|
||||
}
|
||||
}
|
||||
|
||||
/** Store the recognition snapshot keyed by `identity = plate`. Returns the snapshot
|
||||
* id, or null on failure (persistence is best-effort — never blocks the flow). */
|
||||
#storeSnapshot(
|
||||
cameraId: string,
|
||||
direction: FlowDirection,
|
||||
plate: string,
|
||||
bytes: Buffer,
|
||||
contentType: string,
|
||||
): string | null {
|
||||
try {
|
||||
const id = randomUUID();
|
||||
this.#db
|
||||
.insert(snapshots)
|
||||
.values({
|
||||
id,
|
||||
direction,
|
||||
deviceId: cameraId,
|
||||
identity: plate,
|
||||
contentType,
|
||||
bytes,
|
||||
capturedAt: new Date().toISOString(),
|
||||
})
|
||||
.run();
|
||||
return id;
|
||||
} catch (err) {
|
||||
this.#logger.error(`vision snapshot store failed (${plate}): ${(err as Error).message}`);
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
/** Record an unsigned `read` device_event: the ANPR audit trail (plate, confidence,
|
||||
* region, the stored snapshot id, and the flow's outcome). Best-effort telemetry. */
|
||||
#recordReadEvent(
|
||||
row: DeviceRow,
|
||||
direction: FlowDirection,
|
||||
plate: string,
|
||||
result: { plate: { confidence: number; region?: string | null } | null; modelVersion: string },
|
||||
snapshotId: string | null,
|
||||
outcome: { accepted: boolean; direction?: string; reason?: string },
|
||||
): void {
|
||||
try {
|
||||
this.#db
|
||||
.insert(deviceEventsTable)
|
||||
.values({
|
||||
id: randomUUID(),
|
||||
deviceId: row.id,
|
||||
category: "camera",
|
||||
kind: "read",
|
||||
detail: {
|
||||
driverId: row.driverId,
|
||||
plate,
|
||||
confidence: result.plate?.confidence,
|
||||
region: result.plate?.region ?? null,
|
||||
modelVersion: result.modelVersion,
|
||||
direction,
|
||||
snapshotId,
|
||||
accepted: outcome.accepted,
|
||||
outcomeDirection: outcome.direction,
|
||||
reason: outcome.reason,
|
||||
},
|
||||
occurredAt: new Date().toISOString(),
|
||||
})
|
||||
.run();
|
||||
} catch (err) {
|
||||
this.#logger.error(`vision read-event insert failed (${plate}): ${(err as Error).message}`);
|
||||
}
|
||||
}
|
||||
|
||||
/** Debounce: true if this same plate was emitted from this camera within dedupeMs. */
|
||||
#isDuplicate(cameraId: string, plate: string): boolean {
|
||||
const last = this.#lastEmit.get(cameraId);
|
||||
return last != null && last.value === plate && Date.now() - last.at < this.#dedupeMs;
|
||||
}
|
||||
|
||||
/** Build a live camera adapter from a devices row, or null. */
|
||||
#buildCamera(row: DeviceRow): CameraDevice | null {
|
||||
const driver = registry.get(row.driverId);
|
||||
if (!driver) return null;
|
||||
try {
|
||||
return driver.create(row.config as never) as CameraDevice;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -171,32 +171,26 @@ now exists: **opt-in** (`VISION_ENABLED`, default off), **fail-soft** (any error
|
||||
`null`, never throws into the lane → ticket-path fallback), and **re-applies the confidence floor**
|
||||
(`VISION_MIN_CONFIDENCE`) so a low read is flagged advisory. Constructed in `server.ts`; verified
|
||||
end-to-end against the live service (Node → `AA558EE` 0.999, `region=Albania`). (2) ✅ **DONE —
|
||||
trigger wiring (`apps/server/src/vision-reader.ts`).** A **`VisionReader`** polls each **opt-in**
|
||||
camera (`config.anpr === true`, off by default) every `VISION_POLL_MS`, captures a snapshot →
|
||||
`VisionClient.analyze` → on a **confident** plate dispatches a `DeviceReadEvent{kind:"plate"}` through
|
||||
the **same `ReadDispatcher` a physical reader uses** (called directly to capture the outcome, like
|
||||
`qr-reader.ts`), so the subscription/exit flow consumes it **unchanged**. Guards: low-confidence reads
|
||||
are dropped (not an identity); a **debounce** (`VISION_DEDUPE_MS`) stops the same plate re-firing while
|
||||
a car sits in frame; an in-flight guard prevents overlapping recognizes; idle when vision is off or no
|
||||
camera opts in. Plate stays **advisory + non-blocking** — the exit flow still demands a `payment`, the
|
||||
subscription flow only matches a **bound** plate, and a refused read never holds a barrier.
|
||||
|
||||
**Every confident read is PERSISTED (the ANPR audit trail, so a read is investigable):** the
|
||||
**snapshot bytes** are stored in `snapshots` keyed by **`identity = plate`** — the SAME identity the
|
||||
flow signs its anomaly/event with — so `GET /api/snapshots/by-identity/:plate` (the booth event-detail
|
||||
modal's snapshot strip) shows the car's photo **against that anomaly with no extra wiring**; plus an
|
||||
unsigned **`device_events{kind:"read"}`** breadcrumb records plate / confidence / region / model /
|
||||
`snapshotId` / the **dispatch outcome** (`accepted` + `reason`) — a queryable log of every recognition
|
||||
and whether it matched, separate from the signed ledger. *Verified end-to-end:* a recognized AL plate
|
||||
with no open session was non-blocking → signed a `exit.refused.noSession` anomaly (identity=plate),
|
||||
stored a 555 KB snapshot under that plate, recorded the read breadcrumb with
|
||||
`accepted:false, reason:"…no open session…"`, and `by-identity` returned the image — i.e. the refused
|
||||
read is fully investigable with its picture. Debounce held a re-seen plate to 1 emit over 7 polls.
|
||||
trigger: ANPR rides the entry/exit SNAPSHOT (`snapshot.ts`).** The real-world trigger is a **transient
|
||||
button-press or a subscriber QR/RFID read** — which already fires the entry/exit and its evidence
|
||||
snapshot. That is exactly the moment to recognize: `snapshotAsync` now takes the `VisionClient`, and
|
||||
after storing each snapshot from an **opt-in** camera (`config.anpr === true`), it runs ANPR off the
|
||||
**SAME image** and **records the plate against the SAME session `identity`** — an unsigned
|
||||
`device_events{kind:"read"}` with plate / confidence / region / model / `snapshotId` /
|
||||
`source:"entry-exit-snapshot"`. So you can later answer *"session X entered on plate AA558EE"*, with the
|
||||
evidence image linked by `snapshotId`. **No polling — recognition fires only on a real entry/exit**,
|
||||
one image serving both evidence and plate extraction. *(Superseded the earlier polling `VisionReader`,
|
||||
now removed — `VISION_POLL_MS`/`VISION_DEDUPE_MS` gone.)* The flows pass the client (entry/exit/
|
||||
subscription constructors). It is **advisory + fire-and-forget**: a low-confidence/no-plate result
|
||||
records nothing, a vision failure never delays or changes the open, and the plate does **not** feed the
|
||||
access decision (the flow already decided). *Verified end-to-end:* a simulated entry snapshot on an
|
||||
`anpr` camera → stored the snapshot for the session AND recorded `{identity:"TICKET-…", plate:"AA558EE",
|
||||
confidence:0.999, region:"Albania", snapshotId:…}`.
|
||||
(3) **field-accuracy** unknown — re-benchmark/tune
|
||||
the threshold on real on-site captures (angle/night/dirt). (4) the **weight-provenance** check (open).
|
||||
**Bottom line: consume it as a gated advisory identity source feeding the existing `kind:"plate"` path
|
||||
— not as sole authority — and Job 2 is still required for the anti-spoofing value.** The
|
||||
adapter + the opt-in poll→read trigger are now **both built and verified end-to-end**; remaining is
|
||||
**Bottom line: consume it as a gated advisory identity record off the entry/exit snapshot — not as sole
|
||||
authority — and Job 2 is still required for the anti-spoofing value.** The
|
||||
adapter + the snapshot-triggered ANPR are now **both built and verified end-to-end**; remaining is
|
||||
field tuning (3), the provenance check (4), and Job 2.
|
||||
|
||||
## Configuration (2026-06-19)
|
||||
@@ -213,9 +207,9 @@ the AL-benchmark winners), `VISION_MIN_CONFIDENCE`. Install the models with `uv
|
||||
weights download on first run, so **cache them at build/deploy** for the air-gapped appliance.
|
||||
|
||||
**2. The Node server (`apps/server/.env`):** `VISION_ENABLED=1` is the **master switch** (off by
|
||||
default — nothing polls or shows without it); `VISION_URL` must match the service's host:port;
|
||||
`VISION_TIMEOUT_MS` (slow-call cap so a lane never hangs), `VISION_POLL_MS`, `VISION_DEDUPE_MS`,
|
||||
`VISION_MIN_CONFIDENCE` (re-applied client-side).
|
||||
default — nothing runs or shows without it); `VISION_URL` must match the service's host:port;
|
||||
`VISION_TIMEOUT_MS` (slow-call cap so a lane never hangs) and `VISION_MIN_CONFIDENCE` (re-applied
|
||||
client-side). ANPR fires on the entry/exit snapshot, so there are **no poll/dedupe knobs**.
|
||||
|
||||
**3. Per-camera opt-in (device config, not env):** a camera does ANPR only when its config has **both**
|
||||
`anpr: true` **and** a relay binding (`controllerId` + `relay`). The `anpr` flag is a **checkbox on the
|
||||
@@ -242,6 +236,5 @@ service's `/health` each tick and shows a **"Vision" chip** in the booth footer
|
||||
mismatch an anomaly without false-positiving on lighting/angle.
|
||||
- **Compute footprint** on the appliance (CPU-only vs. a small GPU/NPU) — procurement input
|
||||
([[bom]], [[open-questions]]).
|
||||
- Per-camera **opt-in** — ✅ **mechanism built**: `config.anpr === true` on a camera enables ANPR
|
||||
polling (the `VisionReader`). Remaining: expose the toggle in the **SetupWizard** (it's currently
|
||||
set in raw config) and decide sensible `VISION_POLL_MS`/`VISION_DEDUPE_MS` defaults per site.
|
||||
- Per-camera **opt-in** — ✅ **built**: `config.anpr === true` enables ANPR on a camera (set via the
|
||||
SetupWizard checkbox); ANPR then runs on that camera's entry/exit snapshot.
|
||||
|
||||
@@ -940,3 +940,7 @@ VisionReader now PERSISTS every confident plate read so a recognition is investi
|
||||
## [2026-06-19] feat | Vision service configuration — SetupWizard ANPR toggle, footer health chip, .env.example
|
||||
|
||||
Made the vision service genuinely configurable (was env-only). THREE additions: (1) SetupWizard CAMERA form now has an "ANPR / Njohja e targave" checkbox (writes config.anpr; only persisted when on; sq+en) — opt-in is no longer raw JSON. (2) DeviceMonitor now optionally takes the VisionClient and probes its /health each tick, emitting a "vision" pseudo-device status (id vision-service, category "vision" — widened the DeviceStatusEvent + frontend DeviceStatus category unions + the footer CATEGORY_KEY/ORDER maps + devices.catVision sq/en) → a "Vision · ready/degraded/offline" chip in the booth footer; NO chip when VISION_ENABLED off. Verified: emits ready/fast_alpr when up, 0 chips when disabled. (3) apps/vision/.env.example (the Python service env) + VISION_* block appended to apps/server/.env.example (the Node side) + a "Configuration" section in [[opencv-anpr-service]] documenting all FOUR layers (python env / node env / per-camera anpr+binding / footer health) and the caveats: the two processes SHARE the VISION_ prefix but need SEPARATE .env files; bind /analyze to 127.0.0.1 (Node is the only caller); models download on first run so cache at deploy; an unbound anpr camera recognizes but every read is refused. Build+lint green. Updated [[opencv-anpr-service]] (Configuration section; SetupWizard-toggle gap closed), vision README.
|
||||
|
||||
## [2026-06-19] refactor | ANPR rides the entry/exit snapshot (replaces polling VisionReader)
|
||||
|
||||
Reworked the ANPR TRIGGER per the real design goal: when a transient pushes the button or a subscriber passes QR/RFID, the entry/exit fires and takes the evidence snapshot — THAT is the moment to recognize the plate, off the SAME image, tied to the SAME session. So snapshotAsync now takes the VisionClient and, after storing each snapshot from an opt-in (config.anpr) camera, runs ANPR on shot.bytes and records the plate against the session identity (device_events kind:"read" with plate/confidence/region/snapshotId/source:"entry-exit-snapshot"). One image serves both evidence + plate extraction; recognition fires ONLY on a real entry/exit — NO POLLING. The flows (entry/exit/subscription) now take an optional VisionClient and pass it through; server.ts wires it. REMOVED the polling VisionReader (vision-reader.ts deleted) + VISION_POLL_MS/VISION_DEDUPE_MS env. Advisory + fire-and-forget: low-confidence/no-plate records nothing, a vision failure never delays/changes the open, the plate does NOT feed the access decision (the flow already decided) — it's a record ("session X entered on plate AA558EE"). VERIFIED e2e: a simulated entry snapshot on an anpr camera (live fast_alpr) stored the snapshot for session TICKET-SG-1 AND recorded {identity:TICKET-SG-1, plate:AA558EE, confidence:0.999, region:Albania, snapshotId:…}. Build+lint green. Updated [[opencv-anpr-service]] (trigger section + Configuration, polling refs removed) + both .env.example.
|
||||
|
||||
Reference in New Issue
Block a user