Unsupervised multi-modal variational autoencoder for anomaly detection in coronary angiography
Emanuele Frassini (TU Delft - Mechanical Engineering)
Rick M. Butler (TU Delft - Mechanical Engineering)
Benno H.W. Hendriks (TU Delft - Mechanical Engineering, Philips Healthcare Nederland)
Maarten van der Elst (TU Delft - Mechanical Engineering, Reinier de Graaf Gasthuis)
John J. van den Dobbelsteen (TU Delft - Mechanical Engineering)
More Info
expand_more
Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.
Abstract
Background
Coronary angiography (CAG) is among the most common procedures in catheterization laboratories. Patient-specific vessel tortuosity may require additional catheter exchanges, increasing procedure duration and complication risk. Detecting such workflow disruptions automatically is challenging because anomalies are subtle and difficult to capture using a single data modality.
Methods
We propose a novel unsupervised multi-modal anomaly detection framework based on a variational autoencoder (VAE). Video and pose data from 191 real-world CAG procedures were segmented into 10-second clips. Features extracted with 3D convolutional networks were fused through modality-specific LSTM encoders. Reconstruction error (mean squared error) was used as the anomaly score, and thresholds were calibrated on a held-out dataset.
Results
Procedures requiring additional catheter exchanges accounted for 29.8% of the dataset and had a 34% longer mean duration than normal procedures (47.6 vs. 35.5 min). The multi-modal VAE outperformed unimodal models, achieving an F1-score of 0.83, AUC-ROC of 0.88, and AUC-PR of 0.90. Detection performance peaked during the period when anomalies most frequently occurred.
Conclusions
This study demonstrates the effectiveness of multi-modal fusion for unsupervised anomaly detection in complex surgical workflows. These findings support the development of intelligent workflow monitoring systems for catheterization laboratories.