docs(wiki): USB printer cover-open field bug writeup; add art-docker-station lab box
Printer investigation (park-buzi): cover-open on the USB thermal printer wedges its status offline/faulty, surviving a full reboot, recoverable only via `docker restart server`. Traced sendRawUsb/ probeUsb end-to-end — no persistent handle in the app layer, so the leading theory is the container's /dev/usb directory bind-mount retaining a stale view across the printer's physical re-enumeration. Not yet confirmed on hardware; documented with repro/confirmation commands and ranked candidate fixes. Also registers a new lab bench box, "art-docker-station", as a Komodo Stack (dev tier, same shape as park-lab, its own isolated secret refs). Claude-Session: https://claude.ai/code/session_01FWncR69HgGPuei1dLrW3cU
This commit is contained in:
@@ -64,6 +64,39 @@ EVENT_SIGNING_KEY=[[park_lab_event_signing_key]]
|
|||||||
BACKUP_KEY=[[park_lab_backup_key]]
|
BACKUP_KEY=[[park_lab_backup_key]]
|
||||||
"""
|
"""
|
||||||
|
|
||||||
|
##############################################################################
|
||||||
|
# art-docker-station — second LAB bench box (hardware/dev testing, no real traffic). Same tier as
|
||||||
|
# park-lab: chases `dev` (compose files + MOVING image tag), own art_docker_station_* secret refs
|
||||||
|
# (never shared with park-lab or a real booth, even lab-to-lab — per-box blast radius).
|
||||||
|
##############################################################################
|
||||||
|
|
||||||
|
[[stack]]
|
||||||
|
name = "art-docker-station"
|
||||||
|
[stack.config]
|
||||||
|
server = "art-docker-station"
|
||||||
|
git_provider = "git.infra.msai.al"
|
||||||
|
git_account = "komodo"
|
||||||
|
repo = "mca/parking_solution"
|
||||||
|
branch = "dev"
|
||||||
|
file_paths = [
|
||||||
|
"docker-compose.yml",
|
||||||
|
"docker-compose.prod.yml"
|
||||||
|
]
|
||||||
|
registry_provider = "git.infra.msai.al"
|
||||||
|
registry_account = "komodo"
|
||||||
|
environment = """
|
||||||
|
REGISTRY=git.infra.msai.al/mca/parking_solution
|
||||||
|
# Lab tier: the MOVING dev tag — redeploy pulls the latest dev build. Pin to a
|
||||||
|
# dev-<sha> only when reproducing a specific state.
|
||||||
|
TAG=dev
|
||||||
|
COOKIE_SECURE=0
|
||||||
|
VISION_ENABLED=1
|
||||||
|
WS_ALLOWED_ORIGINS=
|
||||||
|
JWT_SECRET=[[art_docker_station_jwt_secret]]
|
||||||
|
EVENT_SIGNING_KEY=[[art_docker_station_event_signing_key]]
|
||||||
|
BACKUP_KEY=[[art_docker_station_backup_key]]
|
||||||
|
"""
|
||||||
|
|
||||||
##############################################################################
|
##############################################################################
|
||||||
|
|
||||||
[[stack]]
|
[[stack]]
|
||||||
|
|||||||
@@ -2,7 +2,7 @@
|
|||||||
type: concept
|
type: concept
|
||||||
tags: [parking, device, printer, transport, usb, escpos, provisioning]
|
tags: [parking, device, printer, transport, usb, escpos, provisioning]
|
||||||
sources: []
|
sources: []
|
||||||
updated: 2026-07-06
|
updated: 2026-08-30
|
||||||
status: settled
|
status: settled
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -140,5 +140,70 @@ hint. The transport option label no longer hardcodes lp0.
|
|||||||
> the monitor would mark a perfectly working printer offline/degraded. Over USB the two drivers
|
> the monitor would mark a perfectly working printer offline/degraded. Over USB the two drivers
|
||||||
> behave identically (reachability floor), so either works post-fix. See [[rongta-printer]].
|
> behave identically (reachability floor), so either works post-fix. See [[rongta-printer]].
|
||||||
|
|
||||||
|
## Field bug — cover-open re-enumeration wedges the container's `/dev/usb` view; only `docker restart`, not a host reboot, clears it (investigated 2026-08-30, unconfirmed root cause)
|
||||||
|
|
||||||
|
**Symptom (park-buzi, unknown/"Generic" USB printer, model not yet identified — see below):** every
|
||||||
|
time the booth operator opens the printer's paper-roll cover to reload paper, the printer's status
|
||||||
|
goes `offline`/faulty in the app and **never self-recovers** — not after the cover closes, not after
|
||||||
|
a full appliance reboot. The only fix found so far is SSH in and `docker restart server`.
|
||||||
|
|
||||||
|
**Ruled out at the application layer.** Traced `sendRawUsb`/`probeUsb` in `printer-escpos.ts`: every
|
||||||
|
print AND every poll tick (`device-monitor.ts` 8s / `printer-monitor.ts` 5s) does a fresh
|
||||||
|
`open()` → write/probe → `close()` against the configured `devicePath`. **No fd, socket, or driver
|
||||||
|
instance is held across calls** — `driver.create(config)` is a throwaway object with no persistent
|
||||||
|
handle. So a naive "stale Node file descriptor" explanation does not fit this codebase; the
|
||||||
|
app-layer retry-by-fresh-open-every-poll should self-heal within one poll cycle if the kernel's view
|
||||||
|
of the device node is current.
|
||||||
|
|
||||||
|
**Leading hypothesis: the container's bind-mount of `/dev/usb`, not the Node process, holds the
|
||||||
|
stale state.** Docker Compose wires the printer in as a **directory bind-mount**
|
||||||
|
(`docker-compose.prod.yml`, `volumes: - /dev/usb:/dev/usb`), chosen deliberately (per its own
|
||||||
|
comment) so the app survives the printer renumbering to a different `lpN`. But many USB thermal
|
||||||
|
printers cut power to their own USB interface board when the cover-open microswitch trips (a
|
||||||
|
hardware safety/power feature, not just a status flag) — the printer drops off the bus and
|
||||||
|
re-enumerates, potentially as a new device node, when the cover closes. The **host** kernel picks
|
||||||
|
this up fine; the **container's mount namespace**, once established, is a known Docker/OverlayFS
|
||||||
|
sharp edge for `/dev` subtree bind-mounts — it can keep resolving the old node until the mount
|
||||||
|
itself is redone.
|
||||||
|
|
||||||
|
- `docker restart server` recreates the container's mount namespace → the `/dev/usb` bind-mount is
|
||||||
|
redone against current host state → the new node is picked up → fixed.
|
||||||
|
- A full host reboot restarts the container too (`restart: always`), but as a boot-time race: if the
|
||||||
|
container starts before the USB subsystem finishes settling, or the printer re-enumerated some
|
||||||
|
time *before* the reboot and Docker doesn't necessarily redo an already-satisfied bind-mount
|
||||||
|
target on a policy-driven restart, the container can come back up still bound to the pre-incident
|
||||||
|
view. This matches the exact reported asymmetry (reboot doesn't fix it; explicit restart does).
|
||||||
|
|
||||||
|
**Not yet confirmed on hardware** — this is the leading theory, not a verified root cause. To
|
||||||
|
confirm at the next occurrence, BEFORE restarting anything:
|
||||||
|
```bash
|
||||||
|
# host:
|
||||||
|
ls -la /dev/usb/ && stat /dev/usb/lp1
|
||||||
|
# container:
|
||||||
|
docker exec server ls -la /dev/usb/ && docker exec server stat /dev/usb/lp1
|
||||||
|
```
|
||||||
|
A major:minor or inode mismatch between host and container is the smoking gun. Also worth
|
||||||
|
capturing on the lab RONGTA (different printer, but same cover-open mechanism is plausible):
|
||||||
|
`watch -n1 lsusb` + `sudo dmesg -w | grep -i -E 'usb|disconnect'` while cycling the cover, to see
|
||||||
|
whether the Bus/Device number changes.
|
||||||
|
|
||||||
|
**Candidate fixes, not yet implemented** (ranked cheapest-to-most-invasive):
|
||||||
|
1. A host-side watchdog/udev rule that detects re-enumeration of this printer (match vendor:product
|
||||||
|
ID) and runs `docker restart server` automatically — turns the manual SSH fix into a self-healing
|
||||||
|
one without touching app code.
|
||||||
|
2. Same idea but event-driven via a udev rule or systemd path unit watching `/dev/usb`, rather than
|
||||||
|
polling.
|
||||||
|
3. Switch the compose device wiring from the directory bind-mount to a specific `--device=` cgroup
|
||||||
|
passthrough + a udev rule pinning a stable symlink name — reintroduces the renumbering fragility
|
||||||
|
the directory bind-mount was chosen to avoid, so only worth doing alongside (1)/(2), not instead.
|
||||||
|
|
||||||
|
**Open sub-question — printer identity.** The park-buzi unit shows as "Generic (unknown)" in the
|
||||||
|
app; not yet identified by vendor/product ID. Lab reproduction uses a **RONGTA** unit instead (not
|
||||||
|
the same hardware), so the lab cannot currently reproduce the park-buzi symptom directly — only
|
||||||
|
validate the general re-enumeration mechanism. Commands to identify the real park-buzi printer next
|
||||||
|
time it's reachable via SSH: `lsusb`, `udevadm info -q property -n /dev/usb/lp1`, `udevadm info -a
|
||||||
|
-n /dev/usb/lp1`. This mirrors the same discovery gap already noted above under "Device discovery"
|
||||||
|
(sysfs `ieee1284_id` enrichment) — once identified, fold the model into that mechanism's coverage.
|
||||||
|
|
||||||
Related: [[rongta-printer]], [[printer-status-monitoring]], [[printer-roles-failover]],
|
Related: [[rongta-printer]], [[printer-status-monitoring]], [[printer-roles-failover]],
|
||||||
[[appliance-provisioning]], [[network-isolation]], [[technology-stack]].
|
[[appliance-provisioning]], [[network-isolation]], [[technology-stack]].
|
||||||
|
|||||||
+2
-2
@@ -58,7 +58,7 @@ Counts: 4 sources · 19 entities · 47 concepts · 8 decision records.
|
|||||||
- [[hardware-signer-options]] — where the ledger signing key should live (TPM interim → USB-HSM target; ATECC608 upcoming, not on-site) so a host-owner can't forge the chain.
|
- [[hardware-signer-options]] — where the ledger signing key should live (TPM interim → USB-HSM target; ATECC608 upcoming, not on-site) so a host-owner can't forge the chain.
|
||||||
- [[reconciliation]] — the real anti-fraud control; what remote sync actually is.
|
- [[reconciliation]] — the real anti-fraud control; what remote sync actually is.
|
||||||
- [[disk-os-hardening]] — the *why* of host hardening: LUKS FDE + TPM-sealed auto-unlock (PCR 7) + Secure Boot + GRUB edit-lock + unprivileged operator + firmware/dbx lockdown; secondary control (reconciliation is the main event). Commands → [[appliance-provisioning]].
|
- [[disk-os-hardening]] — the *why* of host hardening: LUKS FDE + TPM-sealed auto-unlock (PCR 7) + Secure Boot + GRUB edit-lock + unprivileged operator + firmware/dbx lockdown; secondary control (reconciliation is the main event). Commands → [[appliance-provisioning]].
|
||||||
- [[backup-recovery]] — admin-driven encrypted full-DB backup (local/SMB/SFTP) + DR; signing key escrowed & decoupled from TPM so the ledger survives total hardware loss; restore is admin-only.
|
- [[backup-recovery]] — admin-driven encrypted full-DB backup (local/SMB/SFTP) + DR; signing key escrowed & decoupled from TPM so the ledger survives total hardware loss; restore is admin-only; last-success/error status + schedule are now restart-durable (migration 0025, fixed a "shows Never despite valid backups" bug).
|
||||||
|
|
||||||
## Concepts — device architecture & safety
|
## Concepts — device architecture & safety
|
||||||
- [[device-adapter-pattern]] — business logic talks to interfaces; swap hardware → new adapter.
|
- [[device-adapter-pattern]] — business logic talks to interfaces; swap hardware → new adapter.
|
||||||
@@ -69,7 +69,7 @@ Counts: 4 sources · 19 entities · 47 concepts · 8 decision records.
|
|||||||
- [[barrier-not-a-door]] — never timed-close a barrier; safety lives in barrier firmware.
|
- [[barrier-not-a-door]] — never timed-close a barrier; safety lives in barrier firmware.
|
||||||
- [[printer-roles-failover]] — ≥2 printers by role; entry ticket falls back outside→booth.
|
- [[printer-roles-failover]] — ≥2 printers by role; entry ticket falls back outside→booth.
|
||||||
- [[printer-status-monitoring]] — live poll of paper/cover/cutter/offline via the device's status page; SSE to the booth UI.
|
- [[printer-status-monitoring]] — live poll of paper/cover/cutter/offline via the device's status page; SSE to the booth UI.
|
||||||
- [[printer-usb-transport]] — ESC/POS drivers drive TCP (9100) OR local USB (/dev/usb/lp0) behind one render layer; USB = usblp char device, reachability-only status; provisioning open (oq#14).
|
- [[printer-usb-transport]] — ESC/POS drivers drive TCP (9100) OR local USB (/dev/usb/lp0) behind one render layer; USB = usblp char device, reachability-only status; provisioning open (oq#14); park-buzi cover-open-wedges-USB-status bug (docker restart-only fix) under investigation.
|
||||||
- [[device-status-monitoring]] — unified live status across ALL device categories (healthCheck + printer readStatus) → the booth footer over /api/ws.
|
- [[device-status-monitoring]] — unified live status across ALL device categories (healthCheck + printer readStatus) → the booth footer over /api/ws.
|
||||||
- [[trust-boundary]] — the core fork: network vs. device; auditable vs. unforgeable.
|
- [[trust-boundary]] — the core fork: network vs. device; auditable vs. unforgeable.
|
||||||
- [[fail-state-safety]] — entry fails closed, exit fails open; manual override; watchdog.
|
- [[fail-state-safety]] — entry fails closed, exit fails open; manual override; watchdog.
|
||||||
|
|||||||
+37
@@ -2673,3 +2673,40 @@ the DS-2CD1047G3H-LIU units rather than carry an RTSP/ffmpeg workaround dependen
|
|||||||
DS-2CD1043G2-LIU (no such bug, ISAPI main-stream snapshot works natively) is the reference model
|
DS-2CD1043G2-LIU (no such bug, ISAPI main-stream snapshot works natively) is the reference model
|
||||||
going forward. RTSP main-stream capture remains documented as a proven, viable fallback if a G3H
|
going forward. RTSP main-stream capture remains documented as a proven, viable fallback if a G3H
|
||||||
camera is ever unavoidable, but is not being built. Full sweep table + reasoning on [[lpr-camera]].
|
camera is ever unavoidable, but is not being built. Full sweep table + reasoning on [[lpr-camera]].
|
||||||
|
|
||||||
|
## [2026-08-30] update | Booth USB printer cover-open bug: leading theory is a stale container bind-mount, not a stale app-layer handle
|
||||||
|
|
||||||
|
Live troubleshooting request (park-buzi): opening the printer's paper-roll cover reliably wedges its
|
||||||
|
status to offline/faulty, surviving a full appliance reboot; only `docker restart server` clears it.
|
||||||
|
Traced `sendRawUsb`/`probeUsb` end-to-end in `printer-escpos.ts` plus both poll loops
|
||||||
|
(`device-monitor.ts`, `printer-monitor.ts`): every print AND every poll does a fresh
|
||||||
|
open→write/probe→close with no persistent fd/socket/driver instance anywhere — ruling out a naive
|
||||||
|
"stale Node handle" explanation. Leading hypothesis instead: the cover-open microswitch cuts power
|
||||||
|
to the printer's USB interface board, causing a real bus re-enumeration; the container's directory
|
||||||
|
bind-mount of `/dev/usb` (chosen specifically to survive `lpN` renumbering) can retain a stale view
|
||||||
|
of the old device node until the container's mount namespace is recreated — which `docker restart`
|
||||||
|
does and a policy-driven reboot-time restart may not (boot-order race). Not yet confirmed on
|
||||||
|
hardware (host-vs-container `stat`/inode comparison at the next occurrence is the next step); lab
|
||||||
|
repro is blocked because the lab has a RONGTA, not the park-buzi unit's actual (still unidentified,
|
||||||
|
"Generic (unknown)") model. Full writeup, confirmation commands, and candidate fixes on
|
||||||
|
[[printer-usb-transport]].
|
||||||
|
|
||||||
|
## [2026-08-30] update | Backup status "Never" despite valid rotating backups — restart amnesia in BackupService, fixed
|
||||||
|
|
||||||
|
Admin noticed park-buzi's Backup screen showed "last successful backup: Never" despite 7 real,
|
||||||
|
correctly-rotating encrypted backup files on disk, plus a 2-day gap since the last file. Traced
|
||||||
|
both symptoms to the same cause: `BackupService` tracked last-success/last-error as PLAIN
|
||||||
|
IN-PROCESS FIELDS (never written to the DB), and the daily schedule was a `setInterval(...,24h)`
|
||||||
|
measured from PROCESS START, not wall-clock time since the last real backup — so any server
|
||||||
|
restart (routine under `restart: always`: deploy/crash/OOM/host reboot) simultaneously wiped the
|
||||||
|
visible status back to "Never" and reset the 24h countdown, independent of the actual
|
||||||
|
file-writing/retention engine (`backup.ts`), which was working correctly the whole time and
|
||||||
|
explains why files existed on disk despite the UI's contradictory-seeming status. Fix: four new
|
||||||
|
nullable `site_config` columns (migration `0025_backup_last_status.sql`) persist last-success/
|
||||||
|
error there instead of in memory; `BackupService.status()` reads them fresh each call so a new
|
||||||
|
instance (= a restart) sees the prior instance's outcome; a new `isDue()` method computes
|
||||||
|
schedule-due-ness from the persisted last-success timestamp; `server.ts`'s scheduler is now a
|
||||||
|
15-minute poll gated by `isDue()` instead of a 24h `setInterval`, making the real cadence immune
|
||||||
|
to restart timing. New test file `backup-service.test.ts` (6 tests) covers restart-durability and
|
||||||
|
`isDue()` directly; full existing suite (319 tests) still green. No API/UI contract change. Not
|
||||||
|
yet committed (holding per instruction). Full writeup on [[backup-recovery]].
|
||||||
|
|||||||
Reference in New Issue
Block a user