Multimodal Sensor-Fusion for Context-Aware Semi-Autonomous Control of a Multi Degree-of-Freedom Upper Limb Prosthesis

Abstract

Objective. Dexterous control of myoelectric upper limb prosthesis is still limited by the capabilities of the modern human-machine interfaces. The first goal of the current work was to develop a system that supplements the academic myoelectric state-of-the-art interface during the interaction with objects (e.g., grasping, manipulation) with the goal of increasing the overall performances and robustness of the prosthetic device. Additionally, the current study aims to define guidelines for a larger-scale experiment to be performed in the immediate future. Approach. I developed algorithms, which provide context- and user-awareness to the system by fusing multimodal sensory input data, and a control scheme that employs such context-awareness to estimate the user’s grasp intentions to automatically preshape the prosthesis for grasping in real time. The control scheme was compared against the major academic state-of-the-art myoelectric control scheme (i.e., pattern recognition) in two able-bodied subjects. The experimental tests consisted of grasping, reorienting, and relocating sets of common objects using a multi-degree-of-freedom prosthesis with two grip types and two degrees-of-freedom actuated wrist. Main Results. The proposed semi-autonomous system was able to function in realistic and time-varying cluttered environments. The obtained results illustrate better and more consistent performances (i.e., lower task completion time and standard deviation) of the developed control scheme with respect to the state-of-the-art counterpart. Improvements in control robustness during object manipulation (i.e., lower number of object drops) have also been obtained. The current study helped in defining guidelines for the future larger-scale experiment: more than one experimental session, data logging and subjective measurements recording. Significance. The proposed system improves multiple aspects involved in the control of myoelectric multi-degree-of-freedom upper limb prostheses. The guidelines defined in this work, are essential for evaluating, during the future larger-scale study, the impact of the proposed system on users’ experience (e.g., workload and ease of use).