G. Vaidya
Please Note
5 records found
1
Don’t Pry Into My Personal Space
Exploiting mmWave Sensors to Provide Privacy for Areas With Glass Walls
Glass walls reduce illumination costs, enhance thermal comfort and improve social safety. However, they raise privacy concerns. People do not like passersby looking into personal areas, such as computer monitors. Our solution combines two upcoming technologies, mmWave sensors and transparent screens. The mmWave sensors detect passersby, and the transparent screen allows us to turn opaque only a small area instead of the entire wall. We build upon studies performing human pose estimation (HPE) from radar point clouds while addressing their two important shortcomings. They (i) focus on single static users, and (ii) estimate only the right pose, not the exact body location. We provide a solution that works for multiple mobile people, while simultaneously providing accurate location and pose estimation, enabling a robust identification of head coordinates. Our live evaluation shows that, for three passersby, dynamic opaque areas of 30 cm × 30 cm block the line of sight over 95% of the time. Furthermore, our method outperforms a competitive SoA alternative for HPE, reducing the RMSE error (worst-case error) for the head by a factor of 17. Our code and dataset will be publicly available to study human pose estimation in challenging mobile scenarios with multiple people.
Given the significant amount of time that people spend indoors, public spaces are reinventing themselves to deliver immersive experiences. One approach is to provide interactive gaming, such as the spaces created in big international airports like Singapore. The problem is that these gaming areas rely on either (i) wearable devices, which can easily spread germs, particularly in highly transited areas, such as those in airports, or (ii) cameras, which are increasingly raising privacy concerns and are being forbidden in some public areas. To provide an immersive gaming experience that is device-free and privacy-aware, we propose a system based on mmWave radar. In particular, our work provides three contributions. (1) For the first time, we compare various tracking and human pose estimation (HPE) models in the SoA under a unified framework. This approach allows us to identify the best methods in terms of position accuracy, latency, and smoothness-which are critical gaming metrics. (2) We build an integrated system with tracking and HPE capabilities. Our system includes the design of four games with different levels of complexity and a dataset collected specifically to train models for gaming applications. (3) Our evaluation, which includes a subjective users' survey and an objective comparison with a Kinect console, shows that today's mmWave is suitable for games where only the user's location is required or when coarse-grained gestures are needed. There are two key areas where mmWave research needs to improve to match camera-based consoles: sensors need a higher sampling rate, since a high frame rate is critical for seamless gaming, and HPE models need to master fine-grained gestures, in particular when arms end up in front of the body. All the code and datasets will be made available as a stepping stone for future work.
Artifact
Towards Device-Free Gaming with mmWave Radar
Developing millimeter-wave (mmWave) radar applications, particularly for human pose estimation (HPE), often requires substantial manual effort to build custom code for data access, visualization, and model training. In addition, the lack of shared dataset formats and module interfaces makes it difficult to reproduce results and to compare different systems under consistent preprocessing and evaluation settings. To address these challenges, we provide two toolkits, mwCore and mwPose3d, released alongside our paper [1] : mwCore [2] and mwPose3d [3]. Together, these artifacts provide a unified pipeline for processing radar point clouds, evaluating state-of-the-art tracking and HPE models, and deploying online applications via inference from streaming radar data, e.g., device-free gaming.