Cerebral X-ray digital subtraction angiography (DSA) is a widely used imaging technique in patients with neurovascular disease, allowing for vessel and flow visualization with high spatio-temporal resolution. Automatic artery–vein segmentation in DSA plays a fundamental role in vascular analysis with quantitative biomarker extraction, facilitating a wide range of clinical applications. The widely adopted U-Net applied on static DSA frames often struggles with disentangling vessels from subtraction artifacts. Further, it falls short in effectively separating arteries and veins as it disregards the temporal perspectives inherent in DSA. To address these limitations, we propose to simultaneously leverage spatial vasculature and temporal cerebral flow characteristics to segment arteries and veins in DSA. The proposed network, coined CAVE, encodes a 2D+time DSA series using spatial modules, aggregates all the features using temporal modules, and decodes it into 2D segmentation maps. On a large multi-center clinical dataset, CAVE achieves a vessel segmentation Dice of 0.84 (±0.04) and an artery–vein segmentation Dice of 0.79 (±0.06). CAVE surpasses traditional Frangi-based k-means clustering (P < 0.001) and U-Net (P < 0.001) by a significant margin, demonstrating the advantages of harvesting spatio-temporal features. This study represents the first investigation into automatic artery–vein segmentation in DSA using deep learning. The code is publicly available at https://github.com/RuishengSu/CAVE_DSA.

Rationale and Objectives: Distinguishing malignant from benign liver lesions based on magnetic resonance imaging (MRI) is an important but often challenging task, especially in noncirrhotic livers. We developed and externally validated a radiomics model to quantitatively assess T2-weighted MRI to distinguish the most common malignant and benign primary solid liver lesions in noncirrhotic livers. Materials and Methods: Data sets were retrospectively collected from three tertiary referral centers (A, B, and C) between 2002 and 2018. Patients with malignant (hepatocellular carcinoma and intrahepatic cholangiocarcinoma) and benign (hepatocellular adenoma and focal nodular hyperplasia) lesions were included. A radiomics model based on T2-weighted MRI was developed in data set A using a combination of machine learning approaches. The model was internally evaluated on data set A through cross-validation, externally validated on data sets B and C, and compared to visual scoring of two experienced abdominal radiologists on data set C. Results: The overall data set included 486 patients (A: 187, B: 98, and C: 201). The radiomics model had a mean area under the curve (AUC) of 0.78 upon internal validation on data set A and a similar AUC in external validation (B: 0.74 and C: 0.76). In data set C, the two radiologists showed moderate agreement (Cohen's κ: 0.61) and achieved AUCs of 0.86 and 0.82. Conclusion: Our T2-weighted MRI radiomics model shows potential for distinguishing malignant from benign primary solid liver lesions. External validation indicated that the model is generalizable despite substantial MRI acquisition protocol differences. Pending further optimization and generalization, this model may aid radiologists in improving the diagnostic workup of patients with liver lesions.

Integrating multi-omics data into predictive models has the potential to enhance accuracy, which is essential for precision medicine. In this study, we developed interpretable predictive models for multi-omics data by employing neural networks informed by prior biological knowledge, referred to as visible networks. These neural networks offer insights into the decision-making process and can unveil novel perspectives on the underlying biological mechanisms associated with traits and complex diseases. We tested the performance, interpretability and generalizability for inferring smoking status, subject age and LDL levels using genome-wide RNA expression and CpG methylation data from the blood of the BIOS consortium (four population cohorts, Ntotal = 2940). In a cohort-wise cross-validation setting, the consistency of the diagnostic performance and interpretation was assessed. Performance was consistently high for predicting smoking status with an overall mean AUC of 0.95 (95% CI: 0.90-1.00) and interpretation revealed the involvement of well-replicated genes such as AHRR, GPR15 and LRRN3. LDL-level predictions were only generalized in a single cohort with an R2 of 0.07 (95% CI: 0.05-0.08). Age was inferred with a mean error of 5.16 (95% CI: 3.97-6.35) years with the genes COL11A2, AFAP1, OTUD7A, PTPRN2, ADARB2 and CD34 consistently predictive. For both regression tasks, we found that using multi-omics networks improved performance, stability and generalizability compared to interpretable single omic networks. We believe that visible neural networks have great potential for multi-omics analysis; they combine multi-omic data elegantly, are interpretable, and generalize well to data from different cohorts.

Augmented reality (AR) has shown potential in computer-aided surgery. It allows for the visualization of hidden anatomical structures as well as assists in navigating and locating surgical instruments at the surgical site. Various modalities (devices and/or visualizations) have been used in the literature, but few studies investigated the adequacy/superiority of one modality over the other. For instance, the use of optical see-through (OST) HMDs has not always been scientifically justified. Our goal is to compare various visualization modalities for catheter insertion in external ventricular drain and ventricular shunt procedures. We investigate two AR approaches: (1) 2D approaches consisting of a smartphone and a 2D window visualized through an OST (Microsoft HoloLens 2), and (2) 3D approaches consisting of a fully aligned patient model and a model that is adjacent to the patient and is rotationally aligned using an OST. 32 participants joined this study. For each visualization approach, participants were asked to perform five insertions after which they filled NASA-TLX and SUS forms. Moreover, the position and orientation of the needle with respect to the planning during the insertion task were collected. The results show that participants achieved a better insertion performance significantly under 3D visualizations, and the NASA-TLX and SUS forms reflected the preference of participants for these approaches compared to 2D approaches.

STUDY QUESTION: Is there an association between low-to-moderate levels of prenatal alcohol exposure (PAE) and children's facial shape? SUMMARY ANSWER: PAE before and during pregnancy, even at low level (<12 g of alcohol per week), was found associated with the facial shape of children, and these associations were found attenuated as children grow older. WHAT IS KNOWN ALREADY: High levels of PAE during pregnancy can have significant adverse associations with a child's health development resulting in recognizably abnormal facial development. STUDY DESIGN, SIZE, DURATION: This study was based on the Generation R Study, a prospective cohort from fetal life onwards with maternal and offspring data. We analyzed children 3-dimensional (3D) facial images taken at ages 9 (n = 3149) and 13 years (n = 2477) together with the data of maternal alcohol consumption. PARTICIPANTS/MATERIALS, SETTING, METHODS: We defined six levels of PAE based on the frequency and dose of alcohol consumption and defined three tiers based on the timing of alcohol exposure of the unborn child. For the image analysis, we used 3D graph convolutional networks for non-linear dimensionality reduction, which compressed the high-dimensional images into 200 traits representing facial morphology. These 200 traits were used for statistical analysis to search for associations with PAE. Finally, we generated heatmaps to display the facial phenotypes associated with PAE. MAIN RESULTS AND THE ROLE OF CHANCE: The results of the linear regression in the 9-year-old children survived correction for multiple testing with false discovery rate (FDR). In Tier 1 where we examined PAE only before pregnancy (exposed N = 278, unexposed N = 760), we found three traits survived FDR correction. The lowest FDR-P is 1.7e-05 (beta = 0.021, SE = 0.0040) in Trait #29; In Tier 2b where we examine any PAE during first trimester (exposed N = 756; unexposed N = 760), we found eight traits survived FDR correction. The lowest FDR-P is 9.0e-03 (beta = -0.013, SE = 0.0033) in Trait #139. Moreover, more statistically significant facial traits were found in higher levels of PAE. No FDR-significant results were found in the 13-year-old children. We map these significant traits back to the face, and found the most common detected facial phenotypes included turned-up nose tip, shortened nose, turned-out chin, and turned-in lower-eyelid-related regions. LIMITATIONS, REASONS FOR CAUTION: We had no data for alcohol consumption more than three months prior to pregnancy and thus do not know if maternal drinking had chronic effects. The self-reported questionnaire might not reflect accurate alcohol measurements because mothers may have denied their alcohol consumption. WIDER IMPLICATIONS OF THE FINDINGS: Our results imply that facial morphology, such as quantified by the approach we proposed here, can be used as a biomarker in further investigations. Furthermore, our study suggests that for women who are pregnant or want to become pregnant soon, should quit alcohol consumption several months before conception and completely during pregnancy to avoid adverse health outcomes in the offspring. STUDY FUNDING/COMPETING INTEREST(S): This work was supported by Erasmus Medical Centre, Rotterdam, the Erasmus University Rotterdam, and the Netherlands Organization for Health Research. V.W.V.J. reports receipt of funding from the Netherlands Organization for Health Research (ZonMw 90700303). W.J.N. is a founder, a scientific lead, and a shareholder of Quantib BV. TRIAL REGISTRATION NUMBER: N/A.

Background & Aims: Impaired liver function affects brain health and therefore understanding potential mechanisms for subclinical liver disease is essential. We assessed the liver–brain associations using liver measures with brain imaging markers, and cognitive measures in the general population. Methods: Within the population-based Rotterdam Study, liver serum and imaging measures (ultrasound and transient elastography), metabolic dysfunction-associated fatty liver disease (MAFLD), non-alcoholic fatty liver disease (NAFLD) and fibrosis phenotypes, and brain structure were determined in 3493 non-demented and stroke-free participants in 2009–2014. This resulted in subgroups of n = 3493 for MAFLD (mean age 69 ± 9 years, 56% ♀), n = 2938 for NAFLD (mean age 70 ± 9 years, 56% ♀) and n = 2252 for fibrosis (mean age 65 ± 7 years, 54% ♀). Imaging markers of small vessel disease and neurodegeneration, cerebral blood flow (CBF) and brain perfusion (BP) were acquired from brain MRI (1.5-tesla). General cognitive function was assessed by Mini-Mental State Examination and the g-factor. Multiple linear and logistic regression models were used for liver-brain associations and adjusted for age, sex, intracranial volume, cardiovascular risk factors and alcohol use. Results: Higher gamma-glutamyltransferase (GGT) levels were significantly associated with smaller total brain volume (TBV, standardized mean difference (SMD), −0.02, 95% confidence interval (CI) (−0.03 to −0.01); p = 8.4·10⁻⁴), grey matter volumes, and lower CBF and BP. Liver serum measures were not related to small vessel disease markers, nor to white matter microstructural integrity or general cognition. Participants with ultrasound-based liver steatosis had a higher fractional anisotropy (FA, SMD 0.11, 95% CI (0.04 to 0.17), p = 1.5·10⁻³) and lower CBF and BP. MAFLD and NAFLD phenotypes were associated with alterations in white matter microstructural integrity (NAFLD ~ FA, SMD 0.14, 95% CI (0.07 to 0.22), p = 1.6·10⁻⁴; NAFLD ~ mean diffusivity, SMD −0.12, 95% CI (−0.18 to −0.05), p = 4.7·10⁻⁴) and also with lower CBF and BP (MAFLD ~ CBF, SMD −0.13, 95% CI (−0.20 to −0.06), p = 3.1·10⁻⁴; MAFLD ~ BP, SMD −0.12, 95% CI (−0.20 to −0.05), p = 1.6·10⁻³). Furthermore, fibrosis phenotypes were related to TBV, grey and white matter volumes. Conclusions: Presence of liver steatosis, fibrosis and elevated serum GGT are associated with structural and hemodynamic brain markers in a population-based cross-sectional setting. Understanding the hepatic role in brain changes can target modifiable factors and prevent brain dysfunction.

Background: X-ray digital subtraction angiography (DSA) is the imaging modality for peri-procedural guidance and treatment evaluation in (neuro-) vascular interventions. Perfusion image construction from DSA, as a means of quantitatively depicting cerebral hemodynamics, has been shown feasible. However, the quantitative property of perfusion DSA has not been well studied. Purpose: To comparatively study the independence of deconvolution-based perfusion DSA with respect to varying injection protocols, as well as its sensitivity to alterations in brain conditions. Methods: We developed a deconvolution-based algorithm to compute perfusion parametric images from DSA, including cerebral blood volume (CBV (Figure presented.)), cerebral blood flow (CBF (Figure presented.)), time to maximum (Tmax), and mean transit time (MTT (Figure presented.)) and applied it to DSA sequences obtained from two swine models. We also extracted the time intensity curve (TIC)-derived parameters, that is, area under the curve (AUC), peak concentration of the curve, and the time to peak (TTP) from these sequences. Deconvolution-based parameters were quantitatively compared to TIC-derived parameters in terms of consistency upon variations in injection profile and time resolution of DSA, as well as sensitivity to alterations of cerebral condition. Results: Comparing to TIC-derived parameters, the standard deviation (SD) of deconvolution-based parameters (normalized with respect to the mean) are two to five times smaller, indicating that they are more consistent across different injection protocols and time resolutions. Upon ischemic stroke induced in a swine model, the sensitivities of deconvolution-based parameters are equal to, if not higher than, those of TIC-derived parameters. Conclusions: In comparison to TIC-derived parameters, deconvolution-based perfusion imaging in DSA shows significantly higher quantitative reliability against variations in injection protocols across different time resolutions, and is sensitive to alterations in cerebral hemodynamics. The quantitative nature of perfusion angiography may allow for objective treatment assessment in neurovascular interventions.

Computer-aided methods have shown added value for diagnosing and predicting brain disorders and can thus support decision making in clinical care and treatment planning. This chapter will provide insight into the type of methods, their working, their input data –such as cognitive tests, imaging, and genetic data– and the types of output they provide. We will focus on specific use cases for diagnosis, i.e., estimating the current “condition” of the patient, such as early detection and diagnosis of dementia, differential diagnosis of brain tumors, and decision making in stroke. Regarding prediction, i.e., estimation of the future “condition” of the patient, we will zoom in on use cases such as predicting the disease course in multiple sclerosis and predicting patient outcomes after treatment in brain cancer. Furthermore, based on these use cases, we will assess the current state-of-the-art methodology and highlight current efforts on benchmarking of these methods and the importance of open science therein. Finally, we assess the current clinical impact of computer-aided methods and discuss the required next steps to increase clinical impact.

BACKGROUND: Accurate characterization of glioma is crucial for clinical decision making. A delineation of the tumor is also desirable in the initial decision stages but is time-consuming. Previously, deep learning methods have been developed that can either non-invasively predict the genetic or histological features of glioma, or that can automatically delineate the tumor, but not both tasks at the same time. Here, we present our method that can predict the molecular subtype and grade, while simultaneously providing a delineation of the tumor. METHODS: We developed a single multi-task convolutional neural network that uses the full 3D, structural, preoperative MRI scans to predict the IDH mutation status, the 1p/19q co-deletion status, and the grade of a tumor, while simultaneously segmenting the tumor. We trained our method using a patient cohort containing 1508 glioma patients from 16 institutes. We tested our method on an independent dataset of 240 patients from 13 different institutes. RESULTS: In the independent test set, we achieved an IDH-AUC of 0.90, an 1p/19q co-deletion AUC of 0.85, and a grade AUC of 0.81 (grade II/III/IV). For the tumor delineation, we achieved a mean whole tumor Dice score of 0.84. CONCLUSIONS: We developed a method that non-invasively predicts multiple, clinically relevant features of glioma. Evaluation in an independent dataset shows that the method achieves a high performance and that it generalizes well to the broader clinical population. This first-of-its-kind method opens the door to more generalizable, instead of hyper-specialized, AI methods.

Background: The aim of this study was to compare three surgical interventions for correction of sagittal synostosis-frontobiparietal remodeling (FBR), extended strip craniotomy (ESC), and spring-Assisted correction (SAC)-based on three-dimensional (3D) photogrammetry and operation characteristics. Methods: Patients who were born between 1991 and 2019 and diagnosed with nonsyndromic sagittal synostosis who underwent FBR, ESC, or SAC and had at least one postoperative 3D photogrammetry image taken during one of six follow-up appointments until age 6 were considered for this study. Operative characteristics, postoperative complications, reinterventions, and presence of intracranial hypertension were collected. To assess cranial growth, orthogonal cranial slices and 3D photocephalometric measurements were extracted automatically and evaluated from 3D photogrammetry images. Results: A total of 322 postoperative 3D images from 218 patients were included. After correcting for age and sex, no significant differences were observed in 3D photocephalometric measurements. Mean cranial shapes suggested that postoperative growth and shape gradually normalized with higher occipitofrontal head circumference and intracranial volume values compared with normal values, regardless of type of surgery. Flattening of the vertex seems to persist after surgical correction. The authors' cranial 3D mesh processing tool has been made publicly available as a part of this study. Conclusions: The findings suggest that until age 6, there are no significant differences among the FBR, ESC, and SAC in their ability to correct sagittal synostosis with regard to 3D photocephalometric measurements. Therefore, efforts should be made to ensure early diagnosis so that minimally invasive surgery is a viable treatment option. CLINICAL QUESTION/LEVEL OF EVIDENCE: Therapeutic, III.

Identification and detection of thin-cap fibroatheroma (TCFA) from intravascular optical coherence tomography (IVOCT) images is critical for treatment of coronary heart diseases. Recently, deep learning methods have shown promising successes in TCFA identification. However, most methods usually do not effectively utilize multi-view information or incorporate prior domain knowledge. In this paper, we propose a multi-view contour-constrained transformer network (MVCTN) for TCFA identification in IVOCT images. Inspired by the diagnosis process of cardiologists, we use contour constrained self-attention modules (CCSM) to emphasize features corresponding to salient regions (i.e., vessel walls) in an unsupervised manner and enhance the visual interpretability based on class activation mapping (CAM). Moreover, we exploit transformer modules (TM) to build global-range relations between two views (i.e., polar and Cartesian views) to effectively fuse features at multiple feature scales. Experimental results on a semi-public dataset and an in-house dataset demonstrate that the proposed MVCTN outperforms other single-view and multi-view methods. Lastly, the proposed MVCTN can also provide meaningful visualization for cardiologists via CAM.

Extracting the cerebral anterior vessel tree of patients with an intracranial large vessel occlusion (LVO) is relevant to investigate potential biomarkers that can contribute to treatment decision making. The purpose of our work is to develop a method that can achieve this from routinely acquired computed tomography angiography (CTA) and computed tomography perfusion (CTP) images. To this end, we regard the anterior vessel tree as a set of bifurcations and connected centerlines. The method consists of a proximal policy optimization (PPO) based deep reinforcement learning (DRL) approach for tracking centerlines, a convolutional neural network based bifurcation detector, and a breadth-first vessel tree construction approach taking the tracking and bifurcation detection results as input. We experimentally determine the added values of various components of the tracker. Both DRL vessel tracking and CNN bifurcation detection were assessed in a cross validation experiment using 115 subjects. The anterior vessel tree formation was evaluated on an independent test set of 25 subjects, and compared to interobserver variation on a small subset of images. The DRL tracking result achieves a median overlapping rate until the first error (1.8 mm off the reference standard) of 100, [46, 100] % on 8032 vessels over 115 subjects. The bifurcation detector reaches an average recall and precision of 76% and 87% respectively during the vessel tree formation process. The final vessel tree formation achieves a median recall of 68% and precision of 70%, which is in line with the interobserver agreement.

Purpose: The assessment of collateral status may depend on the timing of image acquisition. The purpose of this study is to investigate whether there are optimal time points in CT Perfusion (CTP) for collateral status assessment, and compare collaterals scores at these time points with collateral scores from multiphase CT angiography (mCTA). Methods: Patients with an acute intracranial occlusion who underwent baseline non-contrast CT, mCTA and CT perfusion were selected. Collateral status was assessed using an automatically computed Collateral Ratio (CR) score in mCTA, and predefined time points in CTP acquisition. CRs extracted from CTP were correlated with CRs from mCTA. In addition, all CRs were related to baseline National Institutes of Health Stroke Scale (NIHSS) and Alberta Stoke Program Early CT Score (ASPECTS) with linear regression analysis to find the optimal CR. Results: In total 58 subjects (median age 74 years; interquartile range 61–83 years; 33 male) were included. When comparing the CRs from the CTP vs. mCTA acquisition, the strongest correlations were found between CR from baseline mCTA and the CR at the maximal intensity projection of time-resolved CTP (r = 0.81) and the CR at the peak of arterial enhancement point (r = 0.78). Baseline mCTA-derived CR had the highest correlation with ASPECTS (β = 0.36 (95%CI 0.11, 0.61)) and NIHSS (β = − 0.48 (95%CI − 0.72, − 0.16)). Conclusion: Collateral status assessment strongly depends on the timing of acquisition. Collateral scores obtained from mCTA imaging is close to the optimal collateral score obtained from CTP imaging.

Augmented reality (AR) permits the visualization of pre-operative data in the surgical field of view of the surgeon. This requires the alignment of the AR device’s coordinate system with the used navigation/tracking system. We propose a multimodal marker approach to align an AR device with a tracking system: in our implementation, an electromagnetic tracking system (EMTS). The solution makes use of a calibration method which determines the relationship between a 2D pattern detected by an RGB camera and an electromagnetic sensor of the EMTS. This allowed the projection of a 3D skull model on its physical counterpart. This projection was evaluated using a monocular camera and an optical see-through device (HoloLens 2) (https://www.microsoft.com/en-us/hololens/) achieving an accuracy of less than 2.5 mm in the image plane of the HoloLens 2 (HL2). Additionally, 10 volunteers participated in a user study consisting of an alignment task of a pointer with 25 projections viewed through the HL2. The participants achieved a mean error of 2.7 1.3 mm and 2.9 2.9^∘ in positional and orientation error. This study showcases the feasibility of the approach, provides an evaluation of the alignment, and finally, discusses its advantages and limitations.

Intracranial vessel perforation is a peri-procedural complication during endovascular therapy (EVT). Prompt recognition is important as its occurrence is strongly associated with unfavorable treatment outcomes. However, perforations can be hard to detect because they are rare, can be subtle, and the interventionalist is working under time pressure and focused on treatment of vessel occlusions. Automatic detection holds potential to improve rapid identification of intracranial vessel perforation. In this work, we present the first study on automated perforation detection and localization on X-ray digital subtraction angiography (DSA) image series. We adapt several state-of-the-art single-frame detectors and further propose temporal modules to learn the progressive dynamics of contrast extravasation. Application-tailored loss function and post-processing techniques are designed. We train and validate various automated methods using two national multi-center datasets (i.e., MR CLEAN Registry and MR CLEAN-NoIV Trial), and one international multi-trial dataset (i.e., the HERMES collaboration). With ten-fold cross-validation, the proposed methods achieve an area under the curve (AUC) of the receiver operating characteristic of 0.93 in terms of series level perforation classification. Perforation localization precision and recall reach 0.83 and 0.70 respectively. Furthermore, we demonstrate that the proposed automatic solutions perform at similar level as an expert radiologist.

The computer-aided analysis of prostate multiparametric MRI (mpMRI) could improve significant-prostate-cancer (PCa) detection. Various deep-learning-and radiomics-based methods for significant-PCa segmentation or classification have been reported in the literature. To be able to assess the generalizability of the performance of these methods, using various external data sets is crucial. While both deep-learning and radiomics approaches have been compared based on the same data set of one center, the comparison of the performances of both approaches on various data sets from different centers and different scanners is lacking. The goal of this study was to compare the performance of a deep-learning model with the performance of a radiomics model for the significant-PCa diagnosis of the cohorts of various patients. We included the data from two consecutive patient cohorts from our own center (n = 371 patients), and two external sets of which one was a publicly available patient cohort (n = 195 patients) and the other contained data from patients from two hospitals (n = 79 patients). Using multiparametric MRI (mpMRI), the radiologist tumor delineations and pathology reports were collected for all patients. During training, one of our patient cohorts (n = 271 patients) was used for both the deep-learning-and radiomics-model development, and the three remaining cohorts (n = 374 patients) were kept as unseen test sets. The performances of the models were assessed in terms of their area under the receiver-operating-characteristic curve (AUC). Whereas the internal cross-validation showed a higher AUC for the deep-learning approach, the radiomics model obtained AUCs of 0.88, 0.91 and 0.65 on the independent test sets compared to AUCs of 0.70, 0.73 and 0.44 for the deep-learning model. Our radiomics model that was based on delineated regions resulted in a more accurate tool for significant-PCa classification in the three unseen test sets when compared to a fully automated deep-learning model.

Optical-based navigation systems are widely used in surgical interventions. However, despite their great utility and accuracy, they are expensive and require time and effort to setup for surgeries. Moreover, traditional navigation systems use 2D screens to display instrument positions causing the surgeons to look away from the operative field. Head mounted displays such as the Microsoft HoloLens may provide an attractive alternative for surgical navigation that also permits augmented reality visualization. The HoloLens is equipped with multiple sensors for tracking and scene understanding. Mono and stereo-vision in the HoloLens have been both reported to be used for marker tracking, but no extensive evaluation on accuracy has been done to compare the two approaches. The objective of our work is to investigate the tracking performance of various camera setups in the HoloLens, and to study the effect of the marker size, marker distance from camera, and camera resolution on marker locating accuracy. We also investigate the speed and stability of marker pose for each camera setup. The tracking approaches are evaluated using ArUco markers. Our results show that mono-vision is more accurate in marker locating than stereo-vision when high resolution is used. However, this comes at the expense of higher frame processing time. Alternatively, we propose a combined low-resolution mono-stereo tracking setup that outperforms each tracking approach individually and is comparable to high resolution mono tracking, with a mean translational error of 1.8 ± 0.6mm for 10cm marker size at 50cm distance. We further discuss our findings and their implications for navigation in surgical interventions.

Purpose: In minimally invasive spring-assisted craniectomy, surgeons plan the surgery by manually locating the cranial sutures. However, this approach is prone to error. Augmented reality (AR) could be used to visualize the cranial sutures and assist in the surgery planning. The purpose of our work is to develop an AR-based system to visualize cranial sutures, and to assess the accuracy and usability of using AR-based navigation for surgical guidance in minimally invasive spring-assisted craniectomy. Methods: An AR system was developed that consists of an electromagnetic tracking system linked with a Microsoft HoloLens. The system was used to conduct a study with two skull phantoms. For each phantom, five sutures were annotated and visualized on the skull surface. Twelve participants assessed the system. For each participant, model alignment using six anatomical landmarks was performed, followed by the participant delineation of the visualized sutures. At the end, the participants filled a system usability scale (SUS) questionnaire. For evaluation, an independent optical tracking system was used and the delineated sutures were digitized and compared to the CT-annotated sutures. Results: For a total of 120 delineated sutures, the distance of the annotated sutures to the planning reference was 2.4 ± 1.2 mm. The average delineation time per suture was 13 ± 5 s. For the system usability questionnaire, an average SUS score of 73 was obtained. Conclusion: The developed AR-system has good accuracy (average 2.4 mm distance) and could be used in the OR. The system can assist in the pre-planning of minimally invasive craniosynostosis surgeries to locate cranial sutures accurately instead of the traditional approach of manual palpation. Although the conducted phantom study was designed to closely reflect the clinical setup in the OR, further clinical validation of the developed system is needed and will be addressed in a future work.

Treatment planning of gastrointestinal stromal tumors (GISTs) includes distinguishing GISTs from other intra-abdominal tumors and GISTs’ molecular analysis. The aim of this study was to evaluate radiomics for distinguishing GISTs from other intra-abdominal tumors, and in GISTs, predict the c-KIT, PDGFRA, BRAF mutational status, and mitotic index (MI). Patients diagnosed at the Erasmus MC between 2004 and 2017, with GIST or non-GIST intra-abdominal tumors and a contrast-enhanced venous-phase CT, were retrospectively included. Tumors were segmented, from which 564 image features were extracted. Prediction models were constructed using a combination of machine learning approaches. The evaluation was performed in a 100 × random-split cross-validation. Model performance was compared to that of three radiologists. One hundred twenty-five GISTs and 122 non-GISTs were included. The GIST vs. non-GIST radiomics model had a mean area under the curve (AUC) of 0.77. Three radiologists had an AUC of 0.69, 0.76, and 0.84, respectively. The radiomics model had an AUC of 0.52 for c-KIT, 0.56 for c-KIT exon 11, and 0.52 for the MI. The numbers of PDGFRA, BRAF, and other c-KIT mutations were too low for analysis. Our radiomics model was able to distinguish GISTs from non-GISTs with a performance similar to three radiologists, but less observer dependent. Therefore, it may aid in the early diagnosis of GIST, facilitating rapid referral to specialized treatment centers. As the model was not able to predict any genetic or molecular features, it cannot aid in treatment planning yet.

Histopathological growth patterns (HGPs) are independent prognosticators for colorectal liver metastases (CRLM). Currently, HGPs are determined postoperatively. In this study, we evaluated radiomics for preoperative prediction of HGPs on computed tomography (CT), and its robustness to segmentation and acquisition variations. Patients with pure HGPs [i.e. 100% desmoplastic (dHGP) or 100% replacement (rHGP)] and a CT-scan who were surgically treated at the Erasmus MC between 2003–2015 were included retrospectively. Each lesion was segmented by three clinicians and a convolutional neural network (CNN). A prediction model was created using 564 radiomics features and a combination of machine learning approaches by training on the clinician’s and testing on the unseen CNN segmentations. The intra-class correlation coefficient (ICC) was used to select features robust to segmentation variations; ComBat was used to harmonize for acquisition variations. Evaluation was performed through a 100 × random-split cross-validation. The study included 93 CRLM in 76 patients (48% dHGP; 52% rHGP). Despite substantial differences between the segmentations of the three clinicians and the CNN, the radiomics model had a mean area under the curve of 0.69. ICC-based feature selection or ComBat yielded no improvement. Concluding, the combination of a CNN for segmentation and radiomics for classification has potential for automatically distinguishing dHGPs from rHGP, and is robust to segmentation and acquisition variations. Pending further optimization, including extension to mixed HGPs, our model may serve as a preoperative addition to postoperative HGP assessment, enabling further exploitation of HGPs as a biomarker.