Hierarchical Sensor Fusion for Micro-Gesture Recognition with Pressure Sensor Array and Radar

Abstract

This paper presents a hierarchical sensor fusion approach for human micro-gesture recognition by combining an Ultra Wide Band (UWB) Doppler radar and wearable pressure sensors. First, the wrist-worn pressure sensor array (PSA) and Doppler radar are used to respectively identify static and dynamic gestures through a Quadratic-kernel SVM (Support Vector Machine) classifier. Then, a robust wrapper method is applied on the features from both sensors to search the optimal combination. Subsequently, two hierarchical approaches where one sensor acts as 'enhancer' of the other are explored. In the first case, scores from Doppler radar related to the confidence level of its classifier and the prediction label corresponding to the posterior probabilities are utilized to maximize the static hand gestures classification performance by hierarchical combination with PSA data. In the second case, the PSA acts as an 'enhancer' for radar to improve the dynamic gesture recognition. In this regard, different weights of the 'enhancer' sensor in the fusion process have been evaluated and compared in terms of classification accuracy. A realistic cross-validation method is chosen to test one unknown participant with the model trained by data from others, demonstrating that this hierarchical fusion approach for static and dynamic gestures yields approximately 15% improvement in classification accuracy in the best cases.