julian
c21babf293
Operator asked for bounded container logs (~2 months of history), human- readable timestamps, and clarity on levels. Levels already existed (LOG_LEVEL env → pino, default info; warn+ teed into app_logs, queryable at /setup/logs) — the "level":30 / epoch-ms "time" in docker logs were pino defaults. - server.ts logger: stamp ISO-8601 UTC time (timestamp fn) and level NAMES (formatters.level) so `docker logs` reads human. - log-service.ts pinoDbStream: accept BOTH level encodings (name + numeric) — the label switch would otherwise have silently stopped warn+ persistence into app_logs. New log-service-stream.test.ts pins both encodings, the info-stays-stdout-only rule, and the never-throws fallback. - docker-compose.prod.yml: json-file caps resized from 10m×3 (≈30 MB — days, not months) to ≈2 months by volume: server 20m×30, vision 20m×10, proxy 10m×5. json-file rotates by SIZE; time-based isn't a driver feature — comment says to revisit if `docker logs` holds under ~60 days. - app_logs retention default aligned 30→60 days (LOG_RETENTION_DAYS still overrides). Wiki: app-logs.md gains the container-log store section (rotation, format, LOG_LEVEL knob) + retention update; log.md entry. Suite 282 green. Claude-Session: https://claude.ai/code/session_01Xcm6ikLgGoCxxHrxtjkk5V
7.2 KiB
type, tags, sources, updated, status
| type | tags | sources | updated | status | ||||||
|---|---|---|---|---|---|---|---|---|---|---|
| concept |
|
2026-07-04 | open |
Application logs (diagnostics) — the third stream
A third data stream, deliberately distinct from the two in event-streams-split:
| Stream | Table | Signed? | Purpose |
|---|---|---|---|
| Business ledger | ledger_events |
✅ ATECC608 | money/accountability (append-only-event-chain) |
| Device telemetry | device_events |
❌ | hardware chatter (device-events) |
| App logs | app_logs |
❌ | operational/diagnostic logs (this page) |
App logs answer "why did the booth misbehave?" — a question neither of the other streams should absorb (logs are neither business facts nor hardware telemetry). On an offline appliance there's no Sentry/Datadog to ship to, so the host is the log store: backend warnings/errors AND frontend errors land in one queryable table, viewable at the booth. Built 2026-06-19.
What's captured
- Backend
warn/error/fatal— a pino stream tees these intoapp_logs(and still writes them to stdout, unchanged).info/debug/tracestay stdout-only — they'd bloat the DB. So everyapp.log.warn/error(...)already in the codebase is now persisted with no call-site change. - Frontend errors (always): every failed API request (
apiFetch's non-OK path → method, path, status, server error body — except401, which is normal pre-login churn), every uncaught error (window.onerror), every unhandled promise rejection, and a top-level React ErrorBoundary (a render crash is reported asfatalinstead of a white screen). console.warn/console.error— only forwarded when the client log level isdebug/trace(off by default; they're noisy with third-party chatter). The high-signal sources above are always on. Toggle viaVITE_LOG_LEVEL/setClientLogLevel().
Shape
app_logs: level (pino names), source (frontend|backend), message, context (one JSON
column — the failed request, error name, component stack, anything), plus pulled-out httpStatus /
path for cheap filtering, stack, userId, userAgent, createdAt. Indexed on created_at +
level. Shared types: AppLogRecord / ClientLogInput / LogLevel in @parking/shared.
The API + the access split
POST /api/logs— the frontend ships errors here. Any signed-in user may write (it's their own browser's diagnostics) —requireAuth, not a permission. CSRF still applies (it's a mutation). Accepts one entry or a{ entries: [...] }batch (capped at 50). Deliberately never 4xx's on a malformed entry — a client erroring while reporting an error must not get a second error.GET /api/logs— read the store (level/source/since filters), gated by the newlog:readpermission (a newlogresource in the dynamic local-jwt-auth grid). Admin holds it; it's grantable to a diagnostic role. Verified: a cashier withoutlog:readgets 403 on GET but 204 on POST — the intended asymmetry.
Reliability invariants (a logger must never make things worse)
- No infinite loop. The frontend collector never logs the
/api/logsrequest itself, and flushes via rawfetch/sendBeacon, notapiFetch(so a flush failure can't recurse into a new log). The backendLogServicehas a reentrancy guard — persisting a log can't emit a persisted log. - Best-effort, never fatal. Every write is wrapped; a DB/logging failure is swallowed (it can't be logged — that's the recursion we guard). Diagnostics must never break the path they observe.
- Bounded. Frontend queue capped (drops oldest); message/stack/context clamped per row; ingest batch capped.
Retention (offline appliance ⇒ must be bounded)
Pruned by age AND a row cap (a burst could blow past an age-only window): delete older than
LOG_RETENTION_DAYS (default 60 — the operator's ≈2-month diagnostic window, 2026-07-04;
was 30) and keep only the newest LOG_RETENTION_MAX_ROWS (default 50 000). Runs hourly
(unref'd timer) + once at startup. Both env-configurable. Same "prunable, not precious"
durability class as device_events — the opposite of the append-only ledger.
Container (stdout) logs — the OTHER log store (2026-07-04)
docker logs is a separate, size-bounded store from app_logs — it holds everything
(info/debug too), while app_logs keeps only warn+. Three knobs, all set 2026-07-04:
- Rotation: Docker's json-file driver rotates by SIZE, not time; the caps in
docker-compose.prod.ymlare sized to hold ≈2 months at observed booth rates (server 20 MB × 30, vision 20 MB × 10, proxy 10 MB × 5).docker logsreaches back only that far — revisit the caps if it shows under ~60 days. (Dev compose is uncapped — laptop concern only.) - Human-readable lines: the pino logger stamps ISO-8601 UTC
time(was epoch-ms) and level NAMES ("warn", was40) viatimestamp+formatters.levelinserver.ts.pinoDbStreamaccepts BOTH level encodings, so the app_logs tee survives either config. - Level knob:
LOG_LEVELenv (trace|debug|info|warn|error|fatal; defaultinfo) — a booth under diagnosis runsLOG_LEVEL=debugwith no code change; warn+ persistence is unaffected (it filters independently in the tee).
The booth viewer
A Logs screen under Setup (/setup/logs, gated by log:read, sq+en) — filter by
level/source/since, newest first, each row expands to the structured context + stack. Read-only
(logs are evidence, never edited). Polls every 15 s (no WS — diagnostics aren't latency-critical).
Sits alongside the other admin tabs in booth-console.
As-built (2026-06-19)
packages/db:app_logstable + migration0009_app_logs.sql(+ journal idx 9; seeds adminlog:read). Applied to the liveapps/server/parking.sqlite.@parking/shared:logresource +log:readpermission;LogLevel/LogSource/AppLogRecord/ClientLogInput/LOG_LEVEL_ORDER.apps/server:log-service.ts(LogService+pinoDbStream),routes/logs.ts, wired inserver.ts(DB built before Fastify so the pino stream has the sink; prune timer).apps/web:lib/logger.ts(collector + global handlers),lib/ErrorBoundary.tsx,apiFetchhook,LogsViewer.tsx+ route/nav, i18n.
Open
- No automated test (the standing harness gap) — though the ingest/read/gate path was verified by
in-process
app.injectsmoke (login → POST 204 → GET 200 with the record; non-admin 403/204 split). - Server API error strings stay English — unchanged here; this is about persisting logs, not localizing them. The localized-ledger-reason pattern (i18n) is the template if log display ever needs translation (currently the message is whatever the thrower wrote).
- Surfacing critical logs live — a
fatal/errorcount badge on the booth footer over the existing/api/wscould flag problems without opening the viewer. Deferred. - Correlation id — no request-id threads a frontend failed-request log to its backend log yet;
add a
x-request-idecho if cross-stream correlation is wanted.