Files
parking_solution/wiki/entities/lpr-camera.md
T
julian 6ceaadfbf2
Build desktop / desktop (push) Successful in 4m18s
CI / check (push) Successful in 44s
Build & push images / images (push) Successful in 2m51s
feat(devices): camera clock sync via ISAPI — heal the 1970 power-cut reset
park-buzi field observation: after a power cut the Hikvision cameras
reboot at the 1970 epoch (no/dead RTC battery, no NTP) and stay there
until a human logs into the web UI (which silently pushes the browser
clock) — corrupting the snapshot OSD timestamps (the evidence trail) and
ANPR push times meanwhile.

The host is the site's time authority (offline-first, no NTP infra):

- Device monitor triggers a sync at each camera's offline→ready edge —
  exactly the power-restored moment — plus a 24h backstop; the attempt
  is stamped before the async call so a failing camera retries at
  backstop cadence, never every poll.
- HikvisionCamera.syncClock: GET /ISAPI/System/time; drift ≤60s → leave
  alone; beyond (or unparseable = infinite drift) → PUT timeMode=manual
  with the site wall-clock now WITH explicit utc offset
  (localIsoWithOffset), echoing the camera's timeZone verbatim — correct
  the clock, never fight its tz/DST config.
- Jumps >1h (the power-cut signature) log warn (persisted to app_logs);
  small corrections info. Capability-guarded (isClockSyncable) —
  hikvision only; dahua's CGI has no such endpoint.
- http-digest generalised to digestRequest (GET/PUT/POST + body); the
  handshake was already method-aware. digestGet delegates unchanged.

8 new tests: in-sync no-op, 1970 PUT shape (manual + host instant +
echoed tz), unparseable→sync, failed-set surfaces, dahua non-capability,
DST-both-sides pins on the offset formatter.

Claude-Session: https://claude.ai/code/session_01Xcm6ikLgGoCxxHrxtjkk5V
2026-07-07 12:56:51 +02:00

295 lines
22 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
---
type: entity
tags: [parking, hardware, readers, offline-first]
sources: [parking-system-architecture]
updated: 2026-07-07
---
# LPR Camera
License-plate-recognition camera. For **casual/transient** vehicles, the **plate acts as ticket +
an independent record**. (See [[parking-system-architecture]] §8, §9.)
> **Superseded direction (2026-06-15):** recognition now runs **host-side** on snapshots from
> ordinary Hikvision/Dahua cameras via the [[opencv-anpr-service]], **not** on a dedicated edge-AI
> LPR camera — see [[vision-service]]. The edge-AI camera below is kept as the original assumption /
> a fallback option, but is no longer the planned path. The host-side service also does **vehicle
> verification** (anti-plate-spoofing), which an edge-LPR camera does not.
- **Edge AI (original assumption)**: recognition runs **on-device**, so it keeps working with no
internet — fits [[offline-first]].
- It's a **host-side** identity source: only the host sees the read; the host decides and
commands the relay open (the [[uhppote-controller]] is demoted to a commanded relay for that
lane). See [[entry-exit-readers]].
- Being host-in-the-loop is **good for fraud detection** — you get two independent records (the
host's signed [[append-only-event-chain]] entry + the controller's remote-open event) that
should reconcile one-to-one; any mismatch is an anomaly.
- Mounting: within ~15° of vehicle travel at a controlled chokepoint for best reads.
## Snapshot driver (entry/exit fraud-control record)
Separate from edge-AI LPR: the camera driver (`packages/devices/src/drivers/camera.ts`) does
**snapshot-on-event** — the host pulls a still over HTTP when an entry/exit fires and stores it,
referenced from the signed [[append-only-event-chain]] entry as an independent record. The camera
**pulls, it does not push** — so it is NOT `pushesToBackend` and the setup wizard correctly hides
the "Backend push IP" field for it (gated on the driver's `pushesToBackend` flag; only
[[dingtian-relay]] sets it).
- **Hikvision** uses **ISAPI**: `GET /ISAPI/Streaming/channels/<id>/picture` (`101` = ch1 main
stream) with **HTTP Digest** auth. The "Enable Hikvision-CGI" toggle (Network → Advanced →
Integration Protocol) is a *different* legacy CGI surface — **not** needed for ISAPI.
- **Dahua** uses CGI: `GET /cgi-bin/snapshot.cgi?channel=<n>` (0-based channel; the wizard's
1-based channel is decremented).
**Driver / storage boundary:** the driver FETCHES the image bytes (client-side HTTP Digest in
`drivers/http-digest.ts`) and returns them on `Snapshot.bytes`; **storage is the caller's job**
(the future entry/exit flow stores the bytes + mints a durable `imageRef`). This keeps the device
adapter free of any filesystem/blob-store dependency. `healthCheck()` is honest — it actually pulls
a frame (exercising reachability + auth + path/channel in one shot), not a fake `ready/stub`.
### Verified on hardware (2026-06-15)
A **Hikvision** unit ("Camera 20", MAC `94:e1:ac:…`, Hikvision OUI) at `10.0.10.121`, creds
`admin` / `admin123` (Digest), TCP 80:
- Initial `curl` test confirmed the ISAPI path returns a 2688×1520 JPEG (~306 KB).
- The **real driver** (no longer a stub) was then run end to end against it:
`healthCheck()` → `ready` (pulled a frame), `captureSnapshot()` → valid `image/jpeg`, ~322 KB,
correct JPEG magic. Digest handshake works through `HttpCamera`.
- Reaching it from the WSL dev box required forcing the source address (`config.localAddress`,
threaded into the driver) — see [[wsl-dev-networking]] (multi-subnet source-selection trap).
### HTTP 503 "Device Busy" — can be PERSISTENT; the real fix is stream selection (2026-06-26)
The snapshot endpoint returns **HTTP 503** with the ISAPI body `statusCode 2` / `"Device Busy"` /
`subStatusCode deviceBusy` (occasionally **500**). It comes in two flavours, and they need different
fixes — **don't assume it's a momentary blip**:
- **Transient** — the encoder is briefly occupied (another snapshot in flight, a stream starting).
Clears on retry within a frame or two.
- **Persistent** — the **MAIN-stream encoder is saturated** and 503s on EVERY main-stream snapshot.
Confirmed on hardware (**DS-2CD1047G3H-LIU**, 2026-06-26): `channels/101/picture` → 503 on five
consecutive probes 800 ms apart, while **`channels/102/picture` (the SUB stream) → 200 every time**,
a clean ~15 KB JPEG. So the path/API was correct (the camera answered with a structured Hikvision
status); the main encoder was simply never free. A retry loop **cannot** fix this — it just delays
the failure.
> **Sharper finding (2026-06-27, same DS-2CD1047G3H-LIU).** Re-probed `10.0.10.13` directly after a
> camera reboot with every web/live-view connection closed. **Sub (102): 10/10 rapid back-to-back
> pulls → 200** (~50 ms, ~14.7 KB) — flawless even with NO delay, harder than the live ANPR cadence.
> **Main (101): 3/3 → 503 in ~20 ms** — an *instant* reject, not a timeout. So main isn't merely
> "saturated/busy" on this model — its snapshot endpoint is **structurally unavailable**; **sub (102)
> is mandatory**, not just preferable. SEPARATELY, the camera 503s on **any** stream when its
> **connection slots are exhausted** — a human holding the web UI / live-view, or parallel
> main-stream experiments, consume slots; a **reboot clears stuck slots**. This was the actual cause
> of the **2026-06-27 "subscribers auto-enter but don't auto-exit"** scare: manual main-stream
> testing held the exit camera's slots → the system's sub-stream snapshot pulls got "Device Busy" →
> the ANPR exit read never got a frame → no auto-exit. **NOT a code/flow bug** — the subscription
> exit logic, camera→relay binding, and `stream: "2"` config were all correct (auto-exit/entry pairs
> were clean before the test storm and after the reboot). (Also recorded as the LLM memory
> `g3h-main-stream-snapshot-503`.)
**The fix that actually works: snapshot from the SUB stream.** The Hikvision ISAPI channel id is
`<channel><stream>` (e.g. ch1 main = `101`, ch1 **sub = `102`**). The driver now has a **`stream`
config field** (`1` = main, default for back-compat; `2` = sub). Set the G3H camera to **Sub (02)** in
the setup form → its status flips `degraded → ready` (verified live: pulled a 14.7 KB JPEG in ~87 ms).
The sub-stream is also the better fit for snapshot/ANPR anyway (smaller/faster; doesn't contend with
live-view/recording for the main encoder).
Two more complementary mitigations (both BUILT, for the *transient* case):
1. **Don't cause concurrent busy.** On a vehicle entry two server paths used to snapshot the same
camera at once (the ANPR bridge + the advisory `snapshotAsync`); the 2nd concurrent GET drew a 503.
They now share ONE pull via `captureSnapshotShared` (deviceId-keyed, `apps/server/src/snapshot.ts`)
— the main cause of the slow 2026-06-25 subscriber entry. See [[lane-presence-and-anpr-entry]].
2. **Retry a transient one.** `HttpCamera.captureSnapshot` retries 503/500 with a short linear backoff
(250/500/750 ms, ≤4 attempts), then fails naming it `(device busy)`; it does NOT retry 401/404
(config errors won't self-heal). This recovers a momentary blip but, by design, still fails a
PERSISTENTLY-busy main stream — the cue to switch that camera to the sub-stream.
Covered by `packages/devices/src/drivers/camera.test.ts` (retry behaviour + the main/sub path
selection). `healthCheck()` deliberately reports a live 503 as `degraded` (it surfaces a genuinely
saturated main stream rather than hiding it behind a retry).
## Clock sync — the 1970 power-cut reset (built 2026-07-07)
Field observation (park-buzi): after a power cut these cameras come back with their clock at the
**1970 epoch** (no/dead RTC battery, no NTP) and stay there until a human logs into the web UI —
Hikvision's web login silently pushes the browser clock. A wrong camera clock corrupts the OSD
timestamp burned into every snapshot (the evidence trail) and the times on ANPR pushes.
Built: the HOST is the site's time authority (offline-first — no NTP infra dependency), and the
device monitor re-syncs each Hikvision camera over the same Digest-auth ISAPI used for snapshots:
- **Trigger:** the camera's **offline → ready transition** (exactly the power-restored moment) +
a **24h backstop**; the attempt timestamp is stamped BEFORE the async call, so a failing camera
retries at backstop cadence, never every 8s poll.
- **Mechanics** (`HikvisionCamera.syncClock`): `GET /ISAPI/System/time`, parse `localTime`; drift ≤
60s → leave alone. Beyond that → `PUT /ISAPI/System/time` with `timeMode=manual`, the site's
wall-clock now WITH explicit utc offset (`localIsoWithOffset(siteTz)`, e.g.
`2026-07-07T15:30:22+02:00` — the offset makes the instant unambiguous), and the camera's own
`timeZone` string **echoed back verbatim** (we correct the clock, never fight its tz/DST config).
An unparseable camera reply counts as infinite drift → sync.
- **Visibility:** a sync after a big jump (>1h — the power-cut signature) logs at **warn**
(persisted to app_logs); small corrections log info. Failures log warn.
- **Scope:** Hikvision only (`isClockSyncable` capability guard); the Dahua driver's CGI has no
ISAPI time endpoint — a Dahua clock sync would be its own driver work.
- Rejected alternative: camera-side **NTP against the booth** (chrony on the appliance). More
standard, but adds a provisioning dependency per booth and the camera polls NTP on ITS schedule
— a freshly power-cycled camera could still sit at 1970 for a while, which is precisely the
moment that matters.
## Camera PUSH — "Alarm Server" event notifications (2026-06-22)
Separate from the **pull** snapshot path above: newer Hikvision firmware can **push** an event to
us. Under **Event → Smart/VCA** (e.g. line crossing / intrusion / "Vehicle Detection") the unit
exposes **Detection Target: Human / Vehicle** — selecting **Vehicle** + **Notify Surveillance
Center**, then **Alarm Settings → Alarm Server**, makes the camera **HTTP-POST an
`EventNotificationAlert`** to a URL we host on each detection. Same machine-call shape as the
[[dingtian-relay]] Input Link push — no polling.
- **Ingress:** `POST /api/devices/hikvision/:deviceId/event` (`apps/server/src/routes/hikvision-alarm.ts`).
**Source-IP guarded** (must come from the device's configured `host`) + **optional HTTP Digest**
(some firmware can't authenticate the Alarm Server call → source-IP only). NOT behind the SPA
cookie/CSRF (it's a device call), exactly like the Dingtian push.
- **Config:** added to the `hikvision` driver — `alarmPushEnabled` (bool), `pushUser`/`pushPassword`
(optional Digest). The driver is now `pushesToBackend: true`, so first-run setup offers the backend
push IP. Point the camera's Alarm Server at `http://<backend-ip>:<port>/api/devices/hikvision/<deviceId>/event`.
- **Discovery-first:** the endpoint is **permissive** — accepts ANY content-type as raw bytes (event
XML, multipart-with-JPEG, or JSON; Hik's format varies by model/firmware), records the **verbatim
body** as a `kind:"alarm"` device_event, and best-effort extracts `eventType` / `target` / `plate`
/ `dateTime` / `channelID`. The point of this first cut is to **see exactly what a given camera
sends** (inspect via `GET /api/events` or the server log) before wiring it to the read bus.
- **Not yet a barrier trigger.** It records + breadcrumbs only; it does NOT emit a `DeviceReadEvent`
or open anything. A plate read is **advisory, never the sole reason** a barrier opens
([[append-only-event-chain]], [[opencv-anpr-service]]). Two consumers were since designed off this
same vehicle event — see **[[lane-presence-and-anpr-entry]]**: (a) BUILT — advisory lane busy/free
booth lights; (b) BUILT (2026-06-22) — the ANPR "bridge" (`anpr-entry.ts`) that snapshots → ANPR →
emits a `kind:"plate"` read for a SUBSCRIBER match through the existing gated flow (a small
`apps/server` handler class, not a service). If the camera ever emits its own `<plateNumber>` we'd
use it directly; this `DS-2CD1043G2`
does not, so the server pulls the frame and hands it to the [[opencv-anpr-service|vision service]].
### "Subscribers auto-enter but don't auto-exit" — a 4-layer CAMERA fault, NOT our code (2026-06-27)
A long debugging session on the **DS-2CD1047G3H-LIU** exit camera (`10.0.10.13`, exit-lane). The
symptom: subscribers (e.g. Caca) auto-entered via ANPR fine but **never auto-exited**. **Every
assumption about *our* code was wrong; all four real causes were camera-side.** Method that finally
cracked it: a **dumb HTTP sink** (`scratch-camera-sink.py`) the camera's Alarm Server was pointed
at, to see — verbatim — what the camera actually sends, independent of our app's parsing/acceptance.
The wrong turns, and what was actually true:
1. **Wrong assumption: "the exit camera 503s, so harden the snapshot retry / reduce load."** The 503
storm in the data was mostly **manual main-stream testing**: on this G3H, `channels/101/picture`
(MAIN) **503s instantly every time** — structurally unavailable, not "busy" — while `102` (SUB)
serves 10/10 rapid pulls cleanly. AND the camera **503s on *any* stream when its connection slots
are exhausted** (a held web UI / live-view, parallel experiments); a **reboot clears stuck slots**.
So the snapshot retry/flow code was fine. See [[#HTTP 503 "Device Busy"]] + the
`g3h-main-stream-snapshot-503` memory. (The exit/subscription FLOW logic, camera→relay binding,
and `stream:"2"` config were all correct the whole time — verified: clean entry/exit pairs before
the test storm, and after the reboot.)
2. **The actual blocker #1 — the exit camera never POSTed at all.** `alarmPushEnabled=true` in our
config, but `10.0.10.13` had sent **ZERO** alarms ever (entry cam `.12`: 1126). The sink received
nothing from `.13`; our `/event` endpoint logged no rejections either → the camera wasn't sending.
Cause found in the camera's own **Diagnose Information** dump: **`Main Db is broken` /
`db_restore failed` / `Going to reset cfg`** — the camera's internal config DB (`ipc_db`) was
**CORRUPT**, plus repeated reboots. A broken config DB means the event→linkage→push pipeline can't
reliably read its own config, so it silently never POSTs. **Fix: factory-reset the camera** (rebuilds
`ipc_db`), then reconfigure. (If corruption returns after a clean reset → failing flash → RMA.)
3. **Wrong assumption: "missing gateway/DNS blocks the push."** A documented Hikvision note says a
gateway is needed even same-subnet — but here it was **inverted**: the WORKING cam `.12` has NO
gateway/DNS; the broken `.13` HAD both. Red herring. Gateway/DNS was not the cause.
4. **The actual blocker #2 — after reset, the camera pushed PLAIN MOTION, not vehicle.** Post-reset
`.13` POSTed `<eventType>VMD</eventType>` with **no target tag**. The backend gates on
`target == "vehicle"` (`hikvision-alarm.ts` `isVehicleActive`, matches `<targetType>` /
`<detectionTarget>` / `<objectType>`), so a plain-motion push is **ignored** → bridge never fires.
The working `.12` sends `eventType=VMD` **with `target=vehicle`**. The difference is the AcuSense
**Detection-Target = Vehicle** filter ON the Motion event — defaulted OFF after factory reset.
**Fix: enable Vehicle target classification** on `.13`'s Motion Detection. Confirmed live: a real
drive-through then POSTed `<eventType>VMD</eventType> … <targetType>vehicle</targetType>` — exactly
what the backend needs. (So "VMD/Motion" IS the right event for this camera class; it's the *target
filter* that matters, not switching to a different event type.)
**Takeaways:** (a) a camera that's silently not-pushing looks identical to "fine" in our logs — the
sink-to-prove-it-sends method is the fastest disambiguator; flagged as an observability gap (a
`alarmPushEnabled=true` camera with 0 pushes ever should be a surfaced condition, like the
[[device-status-monitoring|reader-liveness]] fix). (b) For ANPR the camera must send a **vehicle
`targetType`** — verify the push body, not just the UI toggles. (c) Hikvision config-DB corruption
is real; factory reset is the cure. None of this was a code bug. See
[[lane-presence-and-anpr-entry]] for the push→bridge→exit path.
### Gotchas learned the hard way (2026-06-22 field session)
Several traps surfaced trying to get a real camera to push. In order of how long each cost:
- **WSL rewrites the inbound source IP.** On the dev host (WSL mirrored mode), an inbound LAN packet
arrives at our server with its **source rewritten to the host's own IP** (`10.0.10.203`), not the
camera's. The source-IP guard then rejects every push as a mismatch. Fix: a per-device
**`skipSourceIpCheck`** config flag (a Setup checkbox) that bypasses the IP guard — the signed
ledger + optional Digest remain the real guards. Leave OFF on a normal LAN.
- **The setup checkbox saved booleans as the STRING `"true"`.** The generic config-field form had no
boolean renderer, so a `type:"boolean"` field fell through to a text input. Fixed (checkbox
renderer); the server also coerces `"true"`/`1`/`yes`/`on` defensively.
- **The camera's "Test" button proves almost nothing.** It does a TCP/connectivity probe and reports
"service available" on ANY HTTP reply (even our 404) — it does **not** POST a real event to your
URL. Only a real detection (or the ISAPI `httpHosts/<id>/test`) actually exercises the path.
- **`httpBroken` latches.** Once the camera marks the host broken (from earlier failed deliveries),
it stays `true` across reboots and won't retry. Clear it by **re-PUTting** the httpHost config
(`PUT /ISAPI/Event/notification/httpHosts/1` with `<httpBroken>false</httpBroken>`).
- **"Notify Surveillance Center" ≠ the HTTP Alarm Server** on some firmware (separate upload
channels). Always confirm the **Arming Schedule** covers the test time, too (a silent killer).
- **⭐ THE ROOT CAUSE (2026-06-22): no detection AREA drawn.** This is what actually defeated us for
most of a day. On the motion/smart-detection page there's a **Draw Area** step — if **no region is
drawn on the frame, the camera detects nothing, generates NO event, and therefore posts nothing**
anywhere (httpHost, FTP, alarm stream all stay silent because there's no event upstream). Enabling
the detection + ticking Notify Surveillance Center is **not enough** — you must draw the region.
Once an area was drawn, the very first vehicle produced a clean POST. **Check this FIRST.**
### Confirmed real payload (DS-2CD1043G2-LIU, V5.8.10, 2026-06-22)
What this camera actually POSTs on a motion event with a target — captured end-to-end:
- **`Content-Type: multipart/form-data; boundary=boundary`**, one XML part named `MoveDetection.xml`
(`Content-Type: application/xml`). A real frame/JPEG *may* be attached as a second part on other
event types — our endpoint stores the readable head; splitting an image part to `snapshots` is a
forward step (not needed for plain motion).
- The XML is an `EventNotificationAlert` with the fields we care about:
- `<eventType>VMD</eventType>` (Video Motion Detection) + `<eventState>active</eventState>`
- **`<targetType>vehicle</targetType>`** — the camera classifies **vehicle vs human ON-DEVICE**.
(Field is `targetType`, NOT `detectionTarget`.) This means simple presence + class comes for
free, no vision model needed for that part.
- `<targetInfo><targetRect>` with normalized `X/Y/width/height` (0–1) — the **bounding box**.
- `<channelID>`, `<macAddress>` (provenance), `<dateTime>` — **but the dateTime is GARBAGE**
(`2032-…`) because this unit's **RTC is dead** (see below); we use our own server receive time,
never the camera's. (No `<plateNumber>` — this is a motion event, not an ANPR camera.)
### If a camera still won't push — diagnostics (read its OWN state)
Only after confirming the **detection area is drawn** + arming schedule covers now + Notify
Surveillance Center is on. These read the camera directly (no cooperation from our server):
1. **`netstat` on the camera (via SSH) while you trigger** — watch for an OUTBOUND line
`cam:port → server:3000`. It appearing = the camera fired and is delivering (then check our
`/api/devices/hikvision/alarms`). None = no event was generated (almost always: **no area drawn**).
2. **`GET /ISAPI/Event/notification/alertStream`** (Digest, needs a clean handshake) — the live event
bus. NB: a `curl --digest` tap that fails the handshake returns empty and looks like "no events"
— don't over-read silence here (this misled us); the netstat watch above is more reliable.
3. **SSH `showStatus` / `dmesg`** expose internal state. ⚠ **Caveat learned the hard way:** these
surface scary-looking strings that are **red herrings** — `EventScribe: except`, a `diskfull`
error on `Event/triggers` (on a camera with **no disk**), and `fh rtc get time error` / a 1970
clock. On our unit ALL of these were present **and the camera worked fine** once an area was
drawn. The dead RTC is real (hence the bogus `dateTime`) but **harmless** to event push. **Do NOT
conclude "dead camera / RMA" from these** — they are not proof of a broken event engine.
> **Correction (2026-06-22):** an earlier version of this page concluded this DS-2CD1043G2-LIU was a
> **defective unit needing RMA**, based on the silent alertStream + `diskfull`/`EventScribe:except` +
> dead RTC surviving a full factory reset. **That was WRONG.** The camera was healthy; the real cause
> was simply **no detection area drawn**, so no event was ever generated. The `diskfull`/RTC findings
> were unrelated quirks (RTC genuinely dead, but it doesn't block event push). Lesson: don't
> escalate to "hardware fault" while a basic config precondition (the drawn region) is unmet — and
> treat vendor status-API error strings as unreliable. The pull + [[opencv-anpr-service|vision]] path
> remains a valid fallback, but it was not needed here.