mmHRI: Privacy-Preserving Human-Robot Interaction

What happened

To address privacy leakage risks of existing RGB camera-dependent human-robot interaction systems, researchers launched mmHRI, the first mmWave radar-guided privacy-preserving multimodal robotic manipulation framework. The framework adopts a dual-stream architecture to learn both raw radar tensors and radar point clouds, jointly estimating human actions and 3D poses. It introduces a memory-enabled state space model with historical radar feature storage to address issues of sparse radar signals and inconsistent temporal alignment. The estimated human state is ultimately converted into structured text instructions to drive a vision-language-action policy for closed-loop robotic manipulation. Tests show that, in privacy-sensitive scenarios where subjects are separated by curtains, its action recognition accuracy reaches 85.09%, outperforming existing radar-based solutions.

Key facts

Product Name

mmHRI

Core Technical Path

mmWave radar sensing + dual-stream architecture + memory-enabled state space model

Action Recognition Accuracy in Privacy Scenarios

85.09%

Covered Tasks

Action recognition, closed-loop object delivery and retrieval manipulation

Background

Assistive robots are now widely deployed in human-centric settings such as hospitals and restaurants to carry out interactive tasks like item delivery, but mainstream human-robot interaction solutions rely on continuous RGB camera capture of human imagery, creating compliance and usability barriers in privacy-sensitive scenarios where filming individuals is prohibited.

Why it matters

This solution addresses the core pain point of robotic interaction in privacy-sensitive settings, enabling service robots to complete contactless interactive operations in hospitals, restaurants, and other venues without capturing human imagery. This compliance breakthrough will significantly expand the deployment scenarios for service robots; developers can build more privacy-friendly robotic applications on top of this framework, while regular users can enjoy robotic services without the risk of their personal imagery being collected and leaked.

What to watch

Future work can focus on optimizing radar data sensing accuracy and expanding support for non-verbal interaction commands beyond gestures.