feat(logs): app log store — backend pino DB sink + frontend error collection
Add a third data stream (app_logs), distinct from the signed ledger and device telemetry, for operational/diagnostic logs — an offline appliance has no Sentry to ship to, so the host is the log store. Backend: a pino stream tees warn/error/fatal into app_logs (info/debug stay stdout-only) with no call-site change; the DB is built before Fastify so the logger has its sink. Frontend (lib/logger.ts): ships failed API requests (minus 401 churn), window.onerror, unhandledrejection, and a top-level React ErrorBoundary; console warn/error forwarded only at debug/trace. Batched/throttled POST, sendBeacon on pagehide, loop-safe (never logs the /api/logs call), best-effort everywhere. POST /api/logs (any signed-in user, CSRF, tolerant) + GET /api/logs gated by a new log:read permission (new `log` RBAC resource; admin holds it). Retention: pruned by age + row cap, hourly + at startup. UI: a Logs screen under /setup (filter level/source/since, expand to context+stack), sq+en. Migration 0009_app_logs. Verified end-to-end via app.inject: login -> POST 204 -> GET 200 with the record; backend warn/error persisted, info dropped; non-admin GET 403 / POST 204. Claude-Session: https://claude.ai/code/session_01Xcm6ikLgGoCxxHrxtjkk5V
This commit is contained in:
@@ -0,0 +1,101 @@
|
||||
---
|
||||
type: concept
|
||||
tags: [parking, observability, diagnostics, logging, frontend, backend]
|
||||
sources: []
|
||||
updated: 2026-06-19
|
||||
status: open
|
||||
---
|
||||
|
||||
# Application logs (diagnostics) — the third stream
|
||||
|
||||
A **third data stream**, deliberately distinct from the two in [[event-streams-split]]:
|
||||
|
||||
| Stream | Table | Signed? | Purpose |
|
||||
| --- | --- | --- | --- |
|
||||
| Business ledger | `ledger_events` | ✅ ATECC608 | money/accountability ([[append-only-event-chain]]) |
|
||||
| Device telemetry | `device_events` | ❌ | hardware chatter ([[device-events]]) |
|
||||
| **App logs** | **`app_logs`** | ❌ | **operational/diagnostic logs** (this page) |
|
||||
|
||||
App logs answer *"why did the booth misbehave?"* — a question neither of the other streams should
|
||||
absorb (logs are neither business facts nor hardware telemetry). On an **offline appliance** there's
|
||||
no Sentry/Datadog to ship to, so the host **is** the log store: backend warnings/errors AND frontend
|
||||
errors land in one queryable table, viewable at the booth. Built 2026-06-19.
|
||||
|
||||
## What's captured
|
||||
|
||||
- **Backend `warn` / `error` / `fatal`** — a pino stream tees these into `app_logs` (and still writes
|
||||
them to stdout, unchanged). `info`/`debug`/`trace` stay **stdout-only** — they'd bloat the DB. So
|
||||
every `app.log.warn/error(...)` already in the codebase is now persisted with **no call-site
|
||||
change**.
|
||||
- **Frontend errors** (always): every **failed API request** (`apiFetch`'s non-OK path → method,
|
||||
path, status, server error body — except `401`, which is normal pre-login churn), every **uncaught
|
||||
error** (`window.onerror`), every **unhandled promise rejection**, and a top-level **React
|
||||
ErrorBoundary** (a render crash is reported as `fatal` instead of a white screen).
|
||||
- **`console.warn` / `console.error`** — only forwarded when the **client log level is `debug`/`trace`**
|
||||
(off by default; they're noisy with third-party chatter). The high-signal sources above are always
|
||||
on. Toggle via `VITE_LOG_LEVEL` / `setClientLogLevel()`.
|
||||
|
||||
## Shape
|
||||
|
||||
`app_logs`: `level` (pino names), `source` (`frontend`|`backend`), `message`, `context` (one JSON
|
||||
column — the failed request, error name, component stack, anything), plus pulled-out `httpStatus` /
|
||||
`path` for cheap filtering, `stack`, `userId`, `userAgent`, `createdAt`. Indexed on `created_at` +
|
||||
`level`. Shared types: `AppLogRecord` / `ClientLogInput` / `LogLevel` in `@parking/shared`.
|
||||
|
||||
## The API + the access split
|
||||
|
||||
- **`POST /api/logs`** — the frontend ships errors here. **Any signed-in user** may write (it's their
|
||||
own browser's diagnostics) — `requireAuth`, not a permission. CSRF still applies (it's a mutation).
|
||||
Accepts one entry or a `{ entries: [...] }` batch (capped at 50). **Deliberately never 4xx's on a
|
||||
malformed entry** — a client erroring *while reporting an error* must not get a second error.
|
||||
- **`GET /api/logs`** — read the store (level/source/since filters), gated by the **new `log:read`
|
||||
permission** (a new `log` resource in the dynamic [[local-jwt-auth|RBAC]] grid). Admin holds it;
|
||||
it's grantable to a diagnostic role. *Verified: a cashier without `log:read` gets 403 on GET but
|
||||
204 on POST — the intended asymmetry.*
|
||||
|
||||
## Reliability invariants (a logger must never make things worse)
|
||||
|
||||
- **No infinite loop.** The frontend collector never logs the `/api/logs` request itself, and flushes
|
||||
via **raw `fetch`/`sendBeacon`**, not `apiFetch` (so a flush failure can't recurse into a new log).
|
||||
The backend `LogService` has a **reentrancy guard** — persisting a log can't emit a persisted log.
|
||||
- **Best-effort, never fatal.** Every write is wrapped; a DB/logging failure is swallowed (it can't be
|
||||
logged — that's the recursion we guard). Diagnostics must never break the path they observe.
|
||||
- **Bounded.** Frontend queue capped (drops oldest); message/stack/context clamped per row;
|
||||
ingest batch capped.
|
||||
|
||||
## Retention (offline appliance ⇒ must be bounded)
|
||||
|
||||
Pruned by **age AND a row cap** (a burst could blow past an age-only window): delete older than
|
||||
`LOG_RETENTION_DAYS` (default 30) **and** keep only the newest `LOG_RETENTION_MAX_ROWS` (default
|
||||
50 000). Runs **hourly** (unref'd timer) + once at startup. Both env-configurable. Same "prunable,
|
||||
not precious" durability class as `device_events` — the opposite of the append-only ledger.
|
||||
|
||||
## The booth viewer
|
||||
|
||||
A **Logs screen** under Setup (`/setup/logs`, gated by `log:read`, sq+en) — filter by
|
||||
level/source/since, newest first, each row expands to the structured `context` + stack. Read-only
|
||||
(logs are evidence, never edited). Polls every 15 s (no WS — diagnostics aren't latency-critical).
|
||||
Sits alongside the other admin tabs in [[booth-console]].
|
||||
|
||||
## As-built (2026-06-19)
|
||||
|
||||
- `packages/db`: `app_logs` table + migration `0009_app_logs.sql` (+ journal idx 9; seeds admin
|
||||
`log:read`). Applied to the live `apps/server/parking.sqlite`.
|
||||
- `@parking/shared`: `log` resource + `log:read` permission; `LogLevel`/`LogSource`/`AppLogRecord`/
|
||||
`ClientLogInput`/`LOG_LEVEL_ORDER`.
|
||||
- `apps/server`: `log-service.ts` (`LogService` + `pinoDbStream`), `routes/logs.ts`, wired in
|
||||
`server.ts` (DB built before Fastify so the pino stream has the sink; prune timer).
|
||||
- `apps/web`: `lib/logger.ts` (collector + global handlers), `lib/ErrorBoundary.tsx`, `apiFetch` hook,
|
||||
`LogsViewer.tsx` + route/nav, i18n.
|
||||
|
||||
## Open
|
||||
|
||||
- **No automated test** (the standing harness gap) — though the ingest/read/gate path was verified by
|
||||
in-process `app.inject` smoke (login → POST 204 → GET 200 with the record; non-admin 403/204 split).
|
||||
- **Server API error strings stay English** — unchanged here; this is about *persisting* logs, not
|
||||
localizing them. The localized-ledger-reason pattern ([[i18n]]) is the template if log *display*
|
||||
ever needs translation (currently the message is whatever the thrower wrote).
|
||||
- **Surfacing critical logs live** — a `fatal`/`error` count badge on the booth footer over the
|
||||
existing `/api/ws` could flag problems without opening the viewer. Deferred.
|
||||
- **Correlation id** — no request-id threads a frontend failed-request log to its backend log yet;
|
||||
add a `x-request-id` echo if cross-stream correlation is wanted.
|
||||
@@ -31,6 +31,11 @@ diagnostics, and live booth status — **not** anti-fraud.
|
||||
- **Device-keyed** — references the `devices` instance (raw device provenance). No `lane`
|
||||
(pool-of-spaces model — see [[entry-exit-points]]).
|
||||
|
||||
> **Not to be confused with [[app-logs]].** `device_events` is **hardware telemetry** (a relay fired,
|
||||
> a camera failed). Diagnostic/application logs (a failed API request, an uncaught frontend error,
|
||||
> a backend warning) are a **separate third stream** in `app_logs` — don't route app errors here, nor
|
||||
> hardware telemetry there. Both are unsigned + prunable; the distinction is *what produced it*.
|
||||
|
||||
## The boundary that matters
|
||||
|
||||
A device event is *evidence the host saw something happen*; it does **not** by itself authorize or
|
||||
|
||||
@@ -50,8 +50,16 @@ A raw button press is **telemetry** → `device_events`. The entry flow then min
|
||||
input-push handler to emit `device_events` (+ the entry flow signs `vehicle_entry`).
|
||||
- `ParkingEventType` in `packages/shared` splits into ledger types vs. a device-event type set.
|
||||
|
||||
## A third stream followed (2026-06-19)
|
||||
|
||||
The same separation logic produced a **third** stream: **`app_logs`** — operational/diagnostic logs
|
||||
(backend warn+ via a pino sink, plus frontend errors). They're neither business facts (ledger) nor
|
||||
hardware telemetry (device_events), so they get their own unsigned, prunable table. See
|
||||
[[app-logs]]. The principle generalizes: *one stream per durability/meaning class*.
|
||||
|
||||
## Open
|
||||
|
||||
- `device_events` retention/rotation policy.
|
||||
- `device_events` retention/rotation policy. (Resolved for `app_logs`: age + row cap — see
|
||||
[[app-logs]]; the same policy is a candidate for `device_events`.)
|
||||
- Which device facts (if any) are witness-grade enough to *also* warrant a signed ledger entry
|
||||
(e.g. `barrier_open_observed` from a loop sensor) — see [[append-only-event-chain]] witness gap.
|
||||
+2
-1
@@ -7,7 +7,7 @@ updated: 2026-06-19
|
||||
# Index
|
||||
|
||||
Content catalog for the wiki. Start at [[overview]]. Maintained on every ingest.
|
||||
Counts: 4 sources · 19 entities · 42 concepts · 5 decision records.
|
||||
Counts: 4 sources · 19 entities · 44 concepts · 5 decision records.
|
||||
|
||||
## Overview & navigation
|
||||
- [[overview]] — the top-level synthesis and entry point.
|
||||
@@ -93,6 +93,7 @@ Counts: 4 sources · 19 entities · 42 concepts · 5 decision records.
|
||||
- [[ticket-encoding]] — transient ticket id (11-digit numeric + Luhn) as Code128; printed at entry, scanned at pay station + exit; barcode geometry must fit paper width (KP-300H overflow); plate-as-ticket alt.
|
||||
- [[anti-passback]] — block/flag one id entering twice without an exit; fold over open sessions.
|
||||
- [[device-events]] — unsigned hardware telemetry (relay/printer/camera/reader/input); separate from the signed ledger.
|
||||
- [[app-logs]] — the third stream: diagnostic logs (backend warn+ pino sink + frontend errors) → app_logs; log:read viewer; pruned by age+row cap.
|
||||
- [[subscription]] — recurring plan (e.g. 10,000 ALL/month); RF/QR or plate identity, car-count + max-concurrent, host-in-loop; short-circuits payment. (Renamed from "permit"; time-of-day windows noted, deferred.)
|
||||
- [[opencv-anpr-service]] — host-side vision microservice: ANPR (plate identity) + vehicle verification (anti-plate-spoofing witness).
|
||||
- [[blocklist]] — barred plates/cards refused at entry (never at exit); signed, attributed.
|
||||
|
||||
+12
@@ -892,3 +892,15 @@ Dates were raw ISO on paper and time-only in the UI (a 2-day-old session showed
|
||||
## [2026-06-19] fix | KP-300H barcode line-overflow — ticket id 13→11 digits
|
||||
|
||||
The Cashino KP-300H entry dispenser printed entry tickets as RASTER GARBAGE (solid black bars/banding) while the Rongta printed the IDENTICAL byte stream fine. Diagnosed on hardware: plain-text-only prints were clean → isolated to the `GS k` Code128 barcode. ROOT CAUSE = barcode line-overflow, not corruption: a Code128-B symbol is (11·chars+35)·moduleWidth dots; the old 13-digit id at module width 3 = ~534 dots OVERRAN the KP-300H's 72mm line (512 usable dots @ 203 dpi). The Rongta runs 80mm (576 dots) and had just enough room — why only the Cashino failed. FIX: shorten the ticket id 13→11 digits (10 random + Luhn) → ~468 dots, fits 72mm; scanned the full value at the exit reader (verified). Length is driven by GUESS-RESISTANCE not volume (10^10 space, ~1-in-10^7 to hit a live open ticket vs the booth-operator threat); chose 11 over the requested 9 (10^8 → ~1-in-10^5, too weak). validateTicketCode made length-agnostic (\d{10,14}+Luhn) so legacy 13-digit tickets still validate. NB: module width must stay 3 — a width-2 test scanned but returned TRUNCATED values (partial reads logged as exit.refused.noSession anomalies). Also fixed a separate latent transport bug in sendRaw: write-then-destroy could RST mid-stream (the write callback ≠ peer-flushed) and truncate a job; now end(payload)+FIN, resolve on socket `close`, timeout-after-write = success. NOT the cause of the garbage but a real risk. Committed bbf61c4. Updated [[ticket-encoding]], [[rongta-printer]].
|
||||
|
||||
## [2026-06-19] feat | Snapshots on refused entry/exit + subscriber access medium in the activity log
|
||||
|
||||
Two booth-evidence gaps closed. (1) **Refused entry/exit now snapshot.** Originally only the OPEN paths fired the directional camera; refusal/hold anomalies didn't — yet a turned-away car is exactly the evidence an operator/auditor wants (fraud/dispute signal). Added `#fireSnapshot` to every refusal: entry refused-full + held-no-ticket (a refused entry has no ticket id, so mint a synthetic `REFUSED-…` ref to key the anomaly + photo together), exit refused closed/no-session/unpaid/grace-expired (BOTH booth `exitForBooth` and reader `#runExit` paths), and refused [[subscription]] (the lane the reader sits at — `resolved.direction`, "both"→entry — picks the camera). Same fire-and-forget contract: a refusal is never delayed/blocked by a camera; failed captures still surface as "⚠ camera unreachable" tiles. (2) **Subscriber access medium (`via`) surfaced.** The subscription flow already SIGNED `via` (`"qr"|"card"|"plate"`) into the entry/exit payload but the activity log never showed it. Added it as a typed `LedgerPayload.via` field, a cyan chip in the ticker, and an "Entry medium / Mënyra e hyrjes" row in the detail modal (QR code / RFID card·chip / plate, localized sq+en) — a lost-card investigation can now see which credential opened a barrier. Display-only, no re-signing. Refused-subscription anomalies also now carry `via`. Build+lint green. Updated [[entry-exit-points]], [[booth-console]].
|
||||
|
||||
## [2026-06-19] feat | One car = one ticket (entry anti-double-press) + refusal snapshots + subscriber via
|
||||
|
||||
FLAW found: the entry button could be pressed without limit — each press minted a fresh ticket + signed vehicle_entry, corrupting occupancy (one car counts as many) and letting a transient SHOP the cheapest ticket at exit. The old `#inFlight` guard only blocked OVERLAPPING presses (released in finally). FIX is per-relay config (`config.relays[]`), mode chosen by available barrier feedback: (1) PRESENCE (preferred) — `presenceInput` ties ticketing to a vehicle loop on a Dingtian input; a press prints only with a car present, and NO second ticket until the loop CLEARS (car drove in) and a new car re-occupies it → physical one-car-one-ticket; (2) COOLDOWN (fallback, no feedback) — `entryCooldownSec` suppresses repeat presses for N seconds (a timer, mitigation not guarantee). New `relayForPresence()` resolves a loop edge to its entry relay; `EntryFlow` keeps a per-relay `#guard` map (present/armed), disarms on PRINT success, re-arms on loop clear. A suppressed press = UNSIGNED device_events telemetry (entrySuppressed:true), NOT a signed anomaly (operator's call — it's a correct no-op, not fraud). SetupWizard relay editor exposes Presence-loop + Cooldown fields (sq+en). Fail-closed entry + barrier-is-not-a-door invariants untouched; guard state is in-memory/rebuildable, starts armed after restart (safe default). New page [[entry-double-press]]; updated [[entry-exit-points]], index. Build+lint green. (Bundled with this session's earlier refusal-snapshots + subscriber-`via` work.)
|
||||
|
||||
## [2026-06-19] feat | Application logs — backend pino DB sink + frontend error collection (app_logs)
|
||||
|
||||
Added a THIRD data stream (`app_logs`) alongside the signed ledger and device telemetry — operational/diagnostic logs, since an OFFLINE appliance has no Sentry to ship to. BACKEND: a pino stream tees warn/error/fatal into app_logs (info/debug stay stdout-only — no bloat) with NO call-site change; the DB is now built BEFORE Fastify so the logger stream has its sink. FRONTEND (lib/logger.ts): always ships failed API requests (apiFetch non-OK path, minus 401 pre-login churn), window.onerror, unhandledrejection, and a top-level React ErrorBoundary (render crash → fatal, not a white screen); console.warn/error forwarded ONLY at client debug/trace level (noisy otherwise). Batched/throttled POST, flush via raw fetch + sendBeacon on pagehide. Reliability invariants: never log the /api/logs call itself (loop guard), LogService reentrancy guard, all writes best-effort/swallowed, bounded queue + clamped rows. API: POST /api/logs (any signed-in user, CSRF, tolerant — never 4xx on a bad entry) + GET /api/logs gated by a NEW `log:read` permission (new `log` resource in the RBAC grid; admin holds it). Retention: pruned by age (LOG_RETENTION_DAYS=30) AND row cap (MAX_ROWS=50k), hourly + at startup. UI: a Logs screen under /setup (filter level/source/since, expand to context+stack, 15s poll), sq+en. DB migration 0009_app_logs (+journal idx 9, seeds admin log:read) applied to the live apps/server DB. Verified end-to-end via app.inject: login→POST 204→GET 200 with the record; backend warn/error persisted + info dropped; non-admin GET 403 / POST 204 (the intended split). Build+lint green. New page [[app-logs]]; updated [[event-streams-split]], [[device-events]], index.
|
||||
|
||||
Reference in New Issue
Block a user