Legacy UI and Observatory
Overview
The legacy browser UI is a static ES-module app rooted at ui/index.html and started by ui/app.js. The HTML shell defines the dashboard, hardware, live demo, static architecture/performance/application pages, sensing, and training tab containers; it also links ui/style.css, ui/manifest.json, and navigation links to the standalone ui/pose-fusion.html and ui/observatory.html experiences. WiFiDensePoseApp in ui/app.js initializes shared services first, then constructs TabManager, DashboardTab, HardwareTab, LiveDemoTab, SensingTab, and lazy-loaded training/model panels. It also mounts app-wide utilities such as connection status, theme toggle, command palette, activity log, data export, fullscreen, hash routing, onboarding, idle handling, notifications, screenshot, uptime, quick settings, and the global data-source banner.
Configuration is centralized in ui/…/api.config.js. HTTP URLs are derived from window.location.origin; WebSocket URLs follow the page protocol (https: becomes wss:) and page host. The app talks to the backend through singleton services in ui/services/: api.service.js wraps fetch, injects a saved bearer from localStorage key ruview-api-token, applies interceptors, and delegates base-url selection to ui/…/backend-detector.js; health.service.js, pose.service.js, stream.service.js, training.service.js, and model.service.js provide endpoint-specific wrappers. backend-detector.js can still decide between real and mock backends, but API_CONFIG.MOCK_SERVER.AUTO_DETECT is disabled because sensing uses its own WebSocket path.
The main live sensing stream is owned by ui/…/sensing.service.js. It maps UI HTTP ports to sensing WebSocket ports (3000 -> 3001, 8080 -> 8765) and connects to /ws/sensing, using ui/…/ws-ticket.js to exchange the saved bearer for a short-lived single-use ?ticket= when the server supports ADR-272. It publishes frames through onData, connection transitions through onStateChange, and a separate dataSource label that distinguishes live, server-simulated, reconnecting, unreachable, simulated, and auth-required. Client-side simulation is deliberately opt-in via ?simulate=1 or localStorage['ruview-client-simulation']='true'; otherwise an outage keeps retrying and marks the display as unreachable instead of inventing measurements. ui/…/data-source-banner.js subscribes to this service and shows a fixed cross-tab warning whenever the data is not trustworthy.
Dashboard and tab organization are component-driven. ui/…/TabManager.js toggles .nav-tab/.tab-content state and ARIA attributes, while ui/…/router.js makes tabs bookmarkable through the URL hash. ui/…/DashboardTab.js reads API info, pose stats, and /health via health.service.js/pose.service.js, subscribes to sensingService to keep the dashboard data-source card current, and triggers an immediate health refresh when the sensing socket connects or drops so the streaming card does not wait for the next 30-second poll. ui/…/HardwareTab.js is a local interactive antenna/CSI demonstration; it simulates CSI amplitude and phase from active TX/RX antenna counts rather than consuming backend data.
The sensing visualization path starts in ui/…/SensingTab.js. The tab is lazy-initialized the first time it becomes active in ui/app.js, dynamically loads Three.js from CDN, creates ui/…/gaussian-splats.js, starts sensingService, and updates both a HUD and a Three.js scene from each sensing_update frame. GaussianSplatRenderer renders a room grid, router and node markers, a 20x20 signal field (signal_field.values), and a body disruption blob whose opacity/color follow classification.presence, classification.motion_level, and classification.confidence. It also supports multi-node markers from data.nodes. Older Three.js building blocks remain under ui/…/scene.js, ui/…/environment.js, ui/…/body-model.js, and ui/…/signal-viz.js: they build reusable room, body, AP/RX marker, heatmap, phase, Doppler, and motion visualizations for DensePose-style 3D scenes.
The live pose demo path is ui/…/LiveDemoTab.js → ui/…/PoseDetectionCanvas.js → ui/…/pose.service.js → ui/…/websocket.service.js. LiveDemoTab dynamically imports model.service.js and training.service.js, builds the model/training/debug/sidebar UI, auto-starts pose detection once sensingService.dataSource is live or server-simulated, and labels both provenance and estimation mode. PoseDetectionCanvas subscribes to poseService, starts the /api/v1/stream/pose WebSocket for a selected zone, and renders frames through ui/…/pose-renderer.js. pose.service.js validates messages, converts zone-shaped pose_data into the REST-like {persons, zone_summary, pose_source} shape, applies a lower confidence threshold for model inference, and publishes updates to subscribers. pose-renderer.js draws COCO keypoints/skeletons, heatmaps, dense body-part overlays, zones, confidence labels, trails, and client-side smoothing on a 2D canvas.
Training and model controls in the legacy UI are split between ui/…/TrainingPanel.js, ui/…/ModelPanel.js, ui/…/training.service.js, and ui/…/model.service.js. The training service manages /api/v1/train/*, /api/v1/recording/*, and a /ws/train/progress socket; the model service manages /api/v1/models*, active model state, and LoRA profile activation. LiveDemoTab uses these services for model selection, load/unload, quick recording, and training state indicators without requiring those modules to load before the rest of the app works.
ui/sw.js is the PWA/offline service worker. It caches a static shell allowlist cache-first, ignores non-GET, cross-origin, and WebSocket-upgrade requests, bypasses every /oauth/ path, serves /api/ and /health/ network-only with a JSON 503 offline response, and purges old API/health entries on /oauth/logout. ui/manifest.json defines the standalone RuView PWA metadata and icons. The service worker intentionally avoids caching credentialed or live data, because a cached signed-out OAuth status or stale pose reading can make auth and sensing state wrong.
The observatory is a separate Three.js experience launched from ui/observatory.html and orchestrated by ui/…/main.js. It builds a room scene, router, WiFi waves, signal field, dot-matrix body mist, particle trails, scenario props, a figure pool, post-processing, and HudController. ui/…/demo-data.js generates scenario-rich synthetic sensing_update frames for empty rooms, vital signs, walking, fall, sleep, intrusion, gesture, crowd, search/rescue, elderly care, fitness, and patrol views. On startup, main.js probes same-origin and common local sensing server ports with /health; if a server responds with {status:'ok'}, it opens /ws/sensing using withWsTicket, otherwise it remains in demo mode. The animation loop chooses _liveData only when the source is ws and a frame has arrived; otherwise it advances demo data. ui/…/hud-controller.js owns HUD DOM updates, settings persistence, scenario descriptions, edge-module badges, vital metrics, signal metrics, sparkline, presence/fall labels, presets, and quick scenario selection.
The pose-fusion demo is another standalone flow at ui/pose-fusion.html with orchestration in ui/…/main.js. It combines webcam video from video-capture.js, CSI frames from csi-simulator.js, two cnn-embedder.js instances, fusion-engine.js, pose-decoder.js, and canvas-renderer.js. Modes are dual, video-only, or CSI-only. The main loop captures a 56x56 video frame when available, derives video motion/brightness, feeds correlated motion into the CSI simulator, builds a CSI pseudo-image from amplitude/phase history, extracts embeddings, fuses them with confidence-weighted attention, decodes keypoints, and renders skeleton, CSI heatmap, embedding projection, RSSI, latency, cross-modal similarity, and RuVector attention stats. csi-simulator.js stays in demo mode after a socket opens and promotes to live only after it decodes a verified JSON amplitude frame or ADR-018 binary frame; this prevents an open but unverified socket from being labeled live.
The mobile UI under ui/mobile/src/** is a React Native/Expo shell that reuses the same sensing concepts. ui/…/RootNavigator.tsx delegates to MainTabs.tsx, which lazy-loads Live, Vitals, Zones, MAT, and Settings tabs and badges MAT with alert count from matStore. ui/…/usePoseStream.ts subscribes to ui/…/ws.service.ts, connects to the server URL from settings, and stores frames in ui/…/poseStore.ts. The mobile WebSocket service converts a configured HTTP(S) base URL into /ws/sensing, reconnects with bounded backoff, and starts local simulation after retry exhaustion; simulated frames come from ui/…/simulation.service.ts. LiveScreen renders a Gaussian-splat WebView on native or a web viewer on web, overlays LiveHUD, and derives LIVE/SIM/RSSI mode from connection status and frame source. MATScreen bridges sensing frames into ui/…/mat-dashboard.html through useMatBridge.ts, seeds a training event/zones after the WebView is ready, records survivors/alerts into matStore, and shows simulation warnings whenever the connection is not real.
Tests are split between browser-style HTML runners and Node regression tests. ui/…/test-runner.js is an in-browser test harness for API config, URL construction, services, tabs, and components. Executed Node tests cover fragile behavior: ui/sw.test.mjs runs the real service worker in a VM to prove OAuth bypass, API network-only behavior, and logout purging; ui/…/ws-ticket.test.mjs stubs storage/fetch to verify WebSocket ticket minting and auth-refusal detection; ui/…/sensing.service.test.mjs executes the sensing service against fake WebSockets to verify auth-required and source-label behavior; ui/…/DashboardTab.test.mjs pins the stream-health refresh timing. ui/TEST_REPORT.md is a historical manual assessment and should not be treated as the current exhaustive test source.
Coupling
- If
/ws/sensingauth behavior changes on the server, update ui/…/ws-ticket.js, ui/…/sensing.service.js, ui/…/websocket.service.js, ui/…/main.js, and ui/…/main.js together. Browser clients cannot setAuthorizationon WebSocket upgrades, so these paths rely on the same single-use ticket contract. - If the sensing frame schema changes, verify ui/…/sensing.service.js, ui/…/SensingTab.js, ui/…/gaussian-splats.js, ui/…/demo-data.js, ui/…/main.js, ui/…/csi-simulator.js, and ui/…/sensing.ts. These consumers read
features.mean_rssi,classification,signal_field.values,nodes,source, andsource_statedirectly, without a shared runtime decoder. - If data-source labels or source-state semantics change, update all provenance displays: ui/…/data-source-banner.js, ui/…/DashboardTab.js, ui/…/SensingTab.js, ui/…/LiveDemoTab.js, ui/…/csi-simulator.js, and ui/…/index.tsx. Several views intentionally distinguish live hardware, server simulation, client simulation, stale/unreachable, and auth-required states.
- If UI/backend ports or deployment mapping changes, check ui/…/sensing.service.js and ui/…/main.js. The legacy sensing service uses explicit HTTP-port mappings, while pose-fusion derives its default WebSocket port from the current HTTP port plus one; the two defaults must match deployment expectations.
- If
/api/v1/stream/posemessage format changes, update ui/…/pose.service.js, ui/…/PoseDetectionCanvas.js, ui/…/LiveDemoTab.js, and ui/…/pose-renderer.js. The service convertspayload/dataandzone_idinto a local REST-like frame shape expected by the renderer and the demo UI. - If offline/PWA caching rules change, rerun and update ui/sw.test.mjs with ui/sw.js. The service worker is intentionally allowlist-based for shell assets and network-only for
/api/and/health/; widening cache-first routing can silently freeze new credentialed endpoints. - If observatory settings or scenario keys change, keep ui/observatory.html, ui/…/hud-controller.js, ui/…/demo-data.js, and ui/…/scenario-props.js aligned. The dropdown values, saved
SETTINGS_VERSION, scenario descriptions, edge badges, props, and generator cases all share string keys. - If the mobile sensing connection adds authentication, ui/…/ws.service.ts does not currently mint WebSocket tickets like the browser legacy UI. Update it alongside mobile settings/API-token storage rather than assuming the web helper applies.
Risks
- ui/sw.js can break authentication or live-data correctness if request routing is broadened. Prior comments and ui/sw.test.mjs document a real defect where
/oauth/statuswas cached as signed-out forever, and the same cache class can leak stale/api/data across sessions or show old sensing readings as current. - ui/…/sensing.service.js is sensitive to concurrent connection attempts and auth-refusal handling. Its comments document prior duplicate-socket churn that incremented reconnect attempts despite a healthy socket, and ui/…/sensing.service.test.mjs pins the fix for token-required servers so the UI stops retrying and shows
auth-required. - ui/…/csi-simulator.js and ui/…/main.js can falsely imply live sensing if an open socket is treated as measurement. The code explicitly references issue #1557/ADR-295 and only promotes to live after a verified CSI frame is decoded; bypassing that guard would label synthetic data as live.
- ui/…/api.config.js and ui/…/websocket.service.js handle bearer tokens differently from WebSocket tickets. pose.service.js still includes a
tokenparam in stream params for compatibility, but websocket.service.js strips any long-livedtokenquery parameter before callingwithWsTicket; removing that strip would leak durable credentials into URLs. - ui/…/DashboardTab.js depends on a delayed health refresh after sensing state changes. Without the 500 ms refresh path tested in ui/…/DashboardTab.test.mjs, the dashboard can show streaming as idle for up to the normal 30-second health polling interval while the page's own socket is connected.