v2 WiFi DensePose core processing pipeline
Overview
The v2 WiFi DensePose sensing stack is a Rust workspace rooted at v2/Cargo.toml. The core sensing path is split across hardware ingestion, signal conditioning, calibration, multistatic fusion, pose/vitals inference surfaces, trust/provenance emission, and privacy egress. The workspace deliberately keeps real-time state in process rather than a database: v2/Cargo.toml notes that the old api/db/config crates were removed and that REST/WS live elsewhere while this area remains real-time.
Raw CSI enters through v2/crates/wifi-densepose-hardware. src/…/mod.rs binds a UDP socket, parses ADR-018 ESP32 packets with Esp32CsiParser::parse_frame, tracks per-node sequence gaps, and forwards parsed hardware::CsiFrames over an mpsc channel. The frame contract in src/csi_frame.rs stores node id, antenna/subcarrier counts, channel frequency, RSSI/noise floor, PPDU type, ADR-018 flags, and per-subcarrier I/Q samples; CsiFrame::to_amplitude_phase turns each (i, q) sample into amplitude and phase. src/bridge.rs provides a lighter CsiData bridge with antenna-major amplitude/phase vectors and SNR for consumers that do not need ndarray. Vendor and future-standard front ends are modeled in the same crate (src/mediatek_csi.rs, src/qualcomm_csi.rs, src/realtek_csi.rs, src/rtl8720f.rs, and src/ieee80211bf/**), but they feed the same CSI-like measurement shape.
The canonical cross-crate data model lives in v2/crates/wifi-densepose-core. src/types.rs defines CsiFrame as Array2<Complex64> plus derived amplitude/phase arrays and CsiMetadata with device, band/channel, antenna, RSSI/noise, sequence, calibration_id, model_id, and model_version. ComplexSample and CanonicalFrame (src/traits.rs) define deterministic little-endian frame bytes and BLAKE3 witness hashes; CsiFrame::from_canonical_bytes rejects malformed encodings so replayed captures round-trip to the same witness. The same crate defines output contracts: ProcessedSignal carries [time, stream, subcarrier] amplitude and phase tensors plus SignalFeatures; PoseEstimate/PersonPose carry COCO-17 keypoints; pose_observation_v2.rs defines the metric, provenance-rich PoseObservationV2 physics boundary; and pose_refinement_v1.rs defines additive PoseRefinementV1 results with disposition, residuals, optional corrected joints, provenance, and canonical result hash.
Signal conditioning is centered in v2/crates/wifi-densepose-signal. For single-node CSI, src/csi_processor.rs validates CsiData, removes low-amplitude noise in dB, applies a Hamming window, z-score normalizes amplitude, stores bounded history, and exposes temporal smoothing/statistics. src/phase_sanitizer.rs unwraps phase, removes outliers, smooths, and optionally filters noise. src/features.rs derives amplitude statistics, phase gradients/coherence, antenna correlation, FFT-based PSD, and optional Doppler from history, with FeatureExtractor caching the PSD FFT plan. src/motion.rs fuses variance, correlation, phase, and optional Doppler into MotionScore, MotionAnalysis, and HumanDetectionResult with fixed or adaptive thresholds.
Hardware normalization for multistatic sensing happens in v2/…/hardware_norm.rs: HardwareNormalizer detects common subcarrier layouts, cubic-resamples amplitudes/phases to a canonical 56-tone grid, z-score normalizes amplitude, and unwraps/detrends phase. The resample_to_canonical path intentionally resamples without z-scoring for live multistatic fusion so amplitude scale still carries person-score information.
The RuvSense multistatic path is under v2/crates/wifi-densepose-signal/src/ruvsense. mod.rs describes the 20 Hz TDMA cycle: multi-band fusion, phase alignment, multistatic fusion, coherence scoring/gating, and pose tracking. multiband.rs builds a per-node MultiBandCsiFrame from canonical per-channel CSI, sorts channels by frequency, rejects duplicate/mismatched channels, computes cross-channel amplitude coherence, and marks phase as NodeLocal unless a shared reference is explicitly supplied. phase_align.rs estimates per-channel LO offsets from low-variance static subcarriers and subtracts them. multistatic.rs fuses a cohort of MultiBandCsiFrames into a FusedSensingFrame: it enforces hard/soft timestamp guards, dimension equality, per-node positions, amplitude attention weights, optional CIR gating, and explicit PhaseFusion status. Cross-node phase is only emitted for a single node or when every node declares the same shared phase reference; otherwise fused_phase is empty with a reason such as IndependentClocks.
Empty-room calibration is implemented twice for two levels of the stack. In the signal crate, ruvsense/calibration.rs records sanitized CSI with Welford amplitude stats and circular phase stats, serializes deterministic baselines, computes deviation scores, subtracts baseline amplitude in place, and stamps CsiMetadata.calibration_id via BaselineCalibration::apply. It is the only stage that should set that provenance field. ruvsense/cir.rs then estimates a sparse Channel Impulse Response using ISTA/L1 recovery over HT/HE/canonical-56 subcarrier masks; it rejects unsanitized phase by circular variance and can feed CIR-domain coherence into multistatic.rs. At the higher room-specialization level, v2/crates/wifi-densepose-calibration trains a per-room SpecialistBank from guided anchors. src/bank.rs ties every bank to a baseline_id and optional transceiver geometry snapshot; src/runtime.rs runs a MixtureOfSpecialists that reports presence, posture, breathing, heartbeat, restlessness, and anomaly while marking stale banks and letting anomaly veto vitals/posture; src/specialist.rs implements the small statistical heads.
v2/…/lib.rs is the governed composition root for the multistatic trust cycle. StreamingEngine::process_cycle_calibrated takes MultiBandCsiFrames plus per-node calibration ids and a WorldGraph room, runs optional array coordination from registered node geometry, calls MultistaticFuser::fuse_scored_calibrated, tracks environment change-points, runs a mesh partition guard, demotes the active BFLD privacy class on contradictions or mesh risk, writes a provenance-bearing SemanticState into the WorldGraph, prunes live semantic state retention, computes a length-prefixed BLAKE3 witness over evidence/model/calibration/privacy, and returns TrustedOutput. TrustedOutput is the main governed output: it carries the WorldGraph node id, QualityScore, effective privacy class, demotion flag, provenance, optional directional/change-point/mesh reports, recalibration recommendation, witness, and PhaseFusion status.
Pose tracking and physics assessment are separate from the engine's occupancy/trust output. v2/…/pose_tracker.rs tracks COCO-17 metric keypoints with one 6D constant-velocity Kalman state per joint, lifecycle states (Tentative, Active, Lost, Terminated), AETHER-style 128-d re-id embeddings, and optional trajectory priors. v2/…/engine.rs consumes PoseObservationV2 and returns PoseRefinementV1: it validates schema/hash/staleness/coordinate frame/covariance, rejects replay and non-monotonic input per track, abstains on unknown trust, missing calibration/floor, unauthenticated correction, or too few known joints, audits 2D observations without correction, and only selects corrected 3D joints in correction modes with verified authorization and source context. Optional dynamics (feature = "dynamics") adds a Rapier audit but does not override the kinematic invariants.
Vital-sign extraction appears in two forms. v2/crates/wifi-densepose-vitals is the ESP32 CSI-grade path: src/preprocessor.rs suppresses static components with per-subcarrier EMA residuals; src/breathing.rs fuses weighted residuals, bandpasses 0.1-0.5 Hz, estimates respiratory rate by zero crossings, and resets stale locks on out-of-band rejection; src/heartrate.rs uses phase-coherence-weighted residuals, cascaded 0.8-2.0 Hz cardiac filters, autocorrelation, plausibility bands, and subcarrier-count confidence; src/anomaly.rs/src/store.rs add alerting and retention. v2/crates/wifi-densepose-wifiscan is a lower-rate multi-BSSID pseudo-CSI path: src/…/frame.rs converts scans into MultiApFrame RSSI/amplitude/phase histories; src/…/orchestrator.rs runs predictive gating, attention weighting, BSSID correlation, motion estimation, coarse breathing, quality gating, and optional fingerprint matching to produce EnhancedSensingResult with motion, breathing, posture, quality, BSSID count, and verdict.
Output and downstream geometry support are spread across leaf crates. v2/…/csi_pipeline.rs ingests raw ESP32 frames into a heuristic, non-ML amplitude-energy pose/vitals/tomography state; it explicitly does not execute WiFlow weights even if metadata exists. src/fusion.rs converts RF occupancy volumes into sparse point clouds and voxel-fuses WiFi/camera point clouds; src/pointcloud.rs exports colored points and Gaussian splats. v2/crates/wifi-densepose-sar is a synthetic-only wideband RF tomography leaf: src/measurement.rs simulates stepped-frequency aperture measurements, src/reconstruct.rs does delay-and-sum backprojection, and src/pointcloud.rs extracts local maxima from reflectivity images. Its harness under harness/wifi-densepose-sar is a development/router scaffold with illustrative routing data, not part of the live RF pipeline.
Privacy and egress are explicit pipeline stages rather than afterthoughts. v2/crates/wifi-densepose-bfld defines BFI privacy classes (Raw, Derived, Anonymous, Restricted), identity-risk/coherence gating, and BfldPipeline; src/pipeline.rs can emit privacy-gated events or serialized frames, and src/privacy_gate.rs monotonically demotes frames while zeroing payload sections forbidden by the target class. v2/crates/wifi-densepose-privshield is the VEIL countermeasure: src/protector.rs applies keyed Givens rotations (and optional DP dither) to a synthetic beamforming-feedback model so passive re-identification collapses while energy and legitimate throughput are preserved; it never emits RF. Its harness under harness/wifi-densepose-privshield is similarly a development/guidance/router scaffold. v2/…/bridge.rs converts owned SensingSnapshots into signed RuField FieldEvents without importing engine internals; src/lib.rs caps unattended network egress to P1/P2 and maps RuView Derived by information content, not numeric class value.
AETHER and RuVector support the sensing path as compute primitives. v2/…/embedding.rs provides deterministic pure-Rust projection heads, contrastive embeddings, augmentations, and fingerprint indexing; src/sona.rs adds LoRA and EWC-style online adaptation. v2/…/fusion.rs models cross-viewpoint embedding fusion with geometric bias, attention, and coherence gating, while src/signal/** and src/mat/** wrap subcarrier partitioning, spectrogram/BVP/Fresnel, and MAT triangulation helpers.
Coupling
-
If v2/…/types.rs changes the
CsiFramecanonical byte layout, also updateCsiFrame::from_canonical_bytesand any replay/witness consumers. v2/…/traits.rs requires deterministicCanonicalFrame::witness_hash, andruvsensestages rely on provenance fields (calibration_id,model_id,model_version) remaining append-only across stage boundaries. -
If calibration changes, verify both v2/…/calibration.rs and v2/…/lib.rs.
BaselineCalibration::applystamps the frame UUID provenance, whileStreamingEngine::process_cycle_calibratedindependently receivesCalibrationIds and turns agreement/mismatch intoQualityScore, privacy demotion, provenance, and witness fields. -
If the canonical subcarrier grid changes in v2/…/hardware_norm.rs, update v2/…/cir.rs, multiband.rs, and multistatic.rs. CIR has an explicit
CANONICAL56_ACTIVEmask, and multistatic fusion rejects dimension mismatches across node frames. -
If TDMA timing or slot scheduling changes, update v2/…/multistatic.rs and the engine setup in v2/…/lib.rs. The fuser's hard/soft guards drive fusion rejection or tolerated contradictions, and the engine turns contradictions into one-step BFLD privacy demotion.
-
If node geometry is configured outside the engine, make sure it is applied to the exact
StreamingEngineinstance viaset_node_positions/register_node_geometryin v2/…/lib.rs. The engine owns its ownMultistaticFuser; positions set on another fuser instance do not affect governed witnesses or provenance. -
If a pipeline starts combining cross-node phase, it must set
PhaseReference::Sharedcorrectly in v2/…/multiband.rs. v2/…/multistatic.rs deliberately emits amplitude-only output for node-local or mixed references so downstream consumers do not treat unrelated oscillator phases as measurements. -
If pose model output changes, keep v2/…/pose_observation_v2.rs, v2/…/engine.rs, and v2/…/pose_refinement_v1.rs aligned. Physics correction requires metric 3D coordinates, valid floor plane, calibrated uncertainty, trusted source verification, monotonic sequence/timestamps, and COCO-17 joint order; otherwise it abstains or rejects rather than silently correcting.
-
If privacy classes or egress policy change, update v2/…/lib.rs, v2/…/privacy_gate.rs, v2/…/lib.rs, and v2/…/lib.rs together. The engine's effective class and witness, BFLD payload zeroing, and RuField network egress ceiling must continue to agree.
-
If vitals sampling rates or frequency bands change, check both v2/…/breathing.rs/heartrate.rs and v2/…/breathing_extractor.rs. They share the same IIR/zero-crossing pattern but run at very different sample rates and confidence thresholds.
-
If AETHER embedding dimension changes, update v2/…/embedding.rs, v2/…/pose_tracker.rs, and v2/…/fusion.rs. The tracker documents 128-d re-id embeddings and the viewpoint fusion default
embed_dimis 128.
Risks
-
Timestamp guard drift can silently turn good multistatic cycles into privacy-demoted or rejected cycles. v2/…/multistatic.rs documents prior issue #1031: a 5 ms guard rejected real two-slot spreads around 18.2 ms, disabling fusion on hardware; if schedule-derived guards regress, v2/…/lib.rs will either fail fusion or emit lower-information
TrustedOutputs. -
Cross-node phase misuse produces false coherent measurements. v2/…/multiband.rs and multistatic.rs explicitly cite issue #1752 and treat independent ESP32 oscillators as amplitude-only; if a caller labels node-local phase as shared, downstream fused phase can encode non-physical relationships.
-
Provenance/witness changes are privacy-sensitive. v2/…/lib.rs notes a previous witness-framing bug where un-length-prefixed fields could collide across evidence/model/calibration/privacy boundaries; if witness inputs are concatenated or adapter ids are not content-derived, two distinct trust decisions can become indistinguishable.
-
Calibration tone-map assumptions are fragile across PHY tiers. v2/…/calibration.rs documents issue #1009 for HE20: the recorder uses all 256 delivered bins while v2/…/cir.rs uses an exact active-tone mask for Φ. Collapsing those conventions would corrupt baselines or reject valid CIR input.
-
Vitals filters can retain stale physiological locks if reset logic is removed. v2/…/breathing.rs documents issue #1422 and resets on out-of-band rejection; without that, a subject leaving can keep producing stale breathing estimates until the full analysis window flushes.
-
Heart-rate extraction is safety-sensitive to missing phase and out-of-band locks. v2/…/heartrate.rs documents issue #1423 (empty phase slices previously disabled extraction) and rejects estimates outside 40-180 BPM; relaxing either guard can turn missing data or breathing harmonics into confident HR readings.
-
Pointcloud pose in v2/…/csi_pipeline.rs is heuristic, not the trained DensePose/WiFlow path. The file explicitly says model metadata only enables an amplitude-energy heuristic; treating its
Skeletonas neural inference would overstate accuracy and provenance. -
RuField privacy mapping must not use RuView class byte ordering. v2/…/lib.rs warns that RuView
Derivedhas byte value1but carries identity information; mapping by numeric value would leak identity as low-privacy P1 instead of holding it at P4/P5 and off the unattended network surface.