Purpose: Image-guided surgery can decrease perioperative complication incidences. Augmented Reality (AR) with head-mounted displays (HMDs) provides an accessible method of visualizing 3D preoperative models intraoperatively. This could provide the surgeon with an easy to use intraoperative image-guided surgery system. Image-to-patient registration, the other key step of image-guided surgery, can still be intricate and time-consuming with traditional systems. This research explores the feasibility of using the depth sensors of the HoloLens 2, a state-of-the-art AR HMD, for depth-based image-to-patient registration. This research contributes to the advancement of less complex and more efficient image-guided surgical techniques.
Methods: To achieve these objectives, three experiments were conducted using a pilot system based on the HoloLens 2's depth sensors. The first experiment evaluated the accuracy of the depth sensors quantitatively. The second experiment compared four registration initialization methods, including manual and automated approaches. The accuracy and success rate of alignment were assessed using a multi-modal ground truth. Finally, a qualitative assessment of the pilot system was performed on various objects and materials. This experiment aimed to evaluate the system's performance and usability in real-world scenarios.
Results: The depth accuracy experiment showed that both the AHAT and LT sensors had mean overestimation errors of 5.7 and 9.0 mm, respectively. In the registration experiment, the two manual initialization methods consistently achieved successful registration (100%), while the two automatic methods had varying success rates (23.3% and 50%). Three out of four depth registration methods completed the registration within 5 seconds. The mean translation errors ranged from 12.6 to 14.7 mm, and rotation errors ranged from 1.5 to 1.8 degrees. The minimum observed translation and rotation errors were 6.9 mm and 0.5 degrees, respectively, while the maximum errors were 18.8 mm and 3.2 degrees, respectively.
Conclusion: The study's results suggest the potential for achieving sub-10 mm registration accuracy within 5 seconds with depth-based image-to-patient registration. This offers a fast and convenient alternative to other tracking systems that require invasive fiducial markers and time-consuming calibration steps. However, the current accuracy level of the system poses some limitations. Nonetheless, the developed system holds promise for a wide range of surgical procedures that currently do not utilize image guidance due to its complexity. By enabling faster and more accessible image guidance, depth-based registration has the potential to enhance surgical outcomes, such as improving tumor resection margins and avoidance of vulnerable tissues, making it highly beneficial for various procedures.

Introduction: Breast reconstruction after a mastectomy is crucial to improve a patient’s quality of life. The Deep Inferior Epigastric artery Perforator (DIEP) flap procedure is considered the golden standard for breast reconstruction. Three-dimensional (3D) visualization methods have shown promise in providing a better understanding of Computed Tomography Angiography (CTA) information and potentially reducing operative time. However, for these methods manual segmentation of the perforators is needed, which is time-consuming and prone to variability. Automated segmentation using Deep learning (DL) can potentially overcome these limitations and provide accurate and efficient segmentation of the DIEP flap perforators. This study aimed to evaluate the application of DL for automated perforator segmentation in DIEP flap breast reconstruction, improving the efficiency and objectivity of DIEP flap perforator segmentation.

Materials and methods: The dataset comprised 25 CTA scans for training and validation, and 5 CTA scans for testing. DL was employed for automated segmentation, and quantitative evaluation included metrics such as Dice coefficient score, recall, precision, surface distance, and centerline overlap. The qualitative evaluation involved grading the clinical acceptability of segmentations by four experienced plastic reconstructive surgeons.

Results: On the training set, a Dice score of 0.58 (± 0.08) and a true positive centerline overlap of 0.66 (± 0.10) were achieved for perforator segmentation. The DL model successfully segmented the intramuscular main branch, but some perforators were missed in the subcutaneous fat tissue. Combined grading by all surgeons showed no statistical difference between manual and automated segmentations and both segmentations were evaluated as clinically acceptable.

Conclusion: Automated DL segmentation holds promise for enhancing the efficiency and objectivity of identifying DIEP flap perforators in CTA images, providing an alternative to manual segmentation. Nonetheless, further research is needed to refine the automated segmentation results and to validate the generalizability and clinical applicability of the DL segmentation approach in larger patient cohorts and different clinical settings.

Surface Registration is a registration problem that handles the registration of two similar surfaces. In most research that utilises Deep Learning (DL) models to handle surface registration two theories are investigated; the first being whether surfaces sampled from the same origin can be registered together, and the second theory being whether the models can register Point Clouds with low overlapping data for utilisation in Simultaneous Localisation and Mapping (SLAM) applications. However, the surface registration to be utilised in the HoloNav Augmented Reality (AR) navigation system will utilise Point Clouds sampled from different origins with a high overlap ratio. This research, therefore, aims to determine the viability of DL methods for surface registration in HoloNav data. To determine the viability, rotation and translation errors in the match were used, with the aforementioned metrics later being evaluated manually with the utilisation of a visualiser. The results indicate that the models can generalise on the navigator data for an initial Euler angle difference of 45 degrees, but due to the difference in sampling density on the utilised point clouds can not provide accurate matches. Therefore, the utilisation of DL models can be considered to be viable if the navigator data has a sampling density similar to the pre-operative model.

In order to be able to use the Microsoft HoloLens for surgical navigation purposes, performing good patient alignment is of utmost importance. This paper will discuss how this patient alignment can be done using different point cloud registration algorithms.
A lot of research is being conducted on point cloud registration algorithms. However, most research assumes that the point clouds to be aligned are almost identical, while patient alignment aims at aligning a very detailed pre-operative scan with a very sparse point cloud obtained by the surgeon using an optical marker.
In order to get around this problem, the HoloLens' depth camera is used to obtain a detailed point cloud so that registration algorithms can be used. Then the performance of different point cloud registration algorithms is tested on this depth sensor data to see whether using the HoloLens' depth sensor is a viable option for patient alignment.
From the results, it appears that algorithmic approaches for performing patient alignment are feasible, but the performance of these algorithms is very dependent on the quality of the input data.