Evaluating surgeon experience to improve AR in liver surgery

Abstract

Introduction - Accurate intrahepatic anatomical localization remains challenging for laparoscopic liver resection (LLR) of patients with colorectal liver metastases (CRLMs). Image-guided surgery (IGS) systems utilizing augmented reality (AR) visualizations have been developed to assist in these procedures by projecting virtual models onto the laparoscopic view. However, AR visualization, often presented as a comprehensive model, requires improvements to achieve an effective and positive experience for surgeons. A user study was performed to develop and evaluate multiple AR and VR visualizations and identify the most promising and effective visualizations based on feedback from surgeons. Additionally, adapters with EM sensors are required to track the 30° laparoscopes and enable AR. The secondary objective was to develop and calibrate these adapters.

Methods - Multiple visualizations, categorized into three classes, were developed and assessed on a phantom and patients. The user study consisted of two phases. Phase 1 focused on the development and evaluation of these visualizations. Feedback from three surgeons was gathered using a custom-made survey that assessed experience, effectiveness, and usability. Phase 2 involved evaluating clinical usage of the visualizations and their applications by showing videos of these tests to seven surgeons from two hospitals.

Results - In Phase 1, five patients were included for AR-guided laparoscopic resection. Improvements were made compared to the gold standard whole liver model in AR experience, effectiveness, depth perception, and information balance. In Phase 2, the most promising visualizations from class 1 were the depth shader and target
structures visualization, both preferred by six surgeons. Additionally, all seven surgeons wanted the ability to augment a predefined virtual resection plan. For class 2, all seven surgeons found the US view promising and only one surgeon indicated they would use the virtual painter. Five surgeons indicated they would use the third-person and target view of class 3 during liver procedures. Furthermore, adapters with EM sensors were developed and calibrated to track the 30° laparoscopes and enable AR.

Conclusion - AR is promising in liver surgery when surgeons can easily interpret the visualization and have a positive experience. To be effective, AR systems should accurately localize and define vessels and tumor margins, provide clear depth perception, and offer well-balanced information. Several aspects of AR visualization for the entire liver model have been improved. Future research should further improve these visualization aspects, implement non-rigid registration techniques and on-demand patient-specific activation of structures. These advancements can enable AR integration into the surgical workflow, enhancing precision and effectiveness.