Objective: Navigation through tortuous and deformable vessels using catheters with limited steering capability underscores the need for reliable path planning. State-of-the-art path planners do not fully account for the deformable nature of the environment. Methods: This work proposes a robust path planner via a learning from demonstrations method, named Curriculum Generative Adversarial Imitation Learning (C-GAIL). This path planning framework takes into account the interaction between steerable catheters and vessel walls and the deformable property of vessels. Results: In-silico comparative experiments show that the proposed network achieves a 38% higher success rate in static environments and 17% higher in dynamic environments compared to a state-of-the-art approach based on GAIL. In-vitro validation experiments indicate that the path generated by the proposed C-GAIL path planner achieves a targeting error of 1.26±0.55mm and a tracking error of 5.18±3.48mm. These results represent improvements of 41% and 40% over the conventional centerline-following technique for targeting error and tracking error, respectively. Conclusion: The proposed C-GAIL path planner outperforms the state-of-the-art GAIL approach. The in-vitro validation experiments demonstrate that the path generated by the proposed C-GAIL path planner aligns better with the actual steering capability of the pneumatic artificial muscle-driven catheter utilized in this study. Therefore, the proposed approach can provide enhanced support to the user in navigating the catheter towards the target with greater accuracy, effectively meeting clinical accuracy requirements. Significance: The proposed path planning framework exhibits superior performance in managing uncertainty associated with vessel deformation, thereby resulting in lower tracking errors.

Endovascular intervention is a minimally invasive method for treating cardiovascular diseases. Although fluoroscopy, known for real-time catheter visualization, is commonly used, it exposes patients and physicians to ionizing radiation and lacks depth perception due to its 2D nature. To address these limitations, a study was conducted using teleoperation and 3D visualization techniques. This <italic>in-vitro</italic> study involved the use of a robotic catheter system and aimed to evaluate user performance through both subjective and objective measures. The focus was on determining the most effective modes of interaction. Three interactive modes for guiding robotic catheters were compared in the study: 1) Mode GM, using a gamepad for control and a standard 2D monitor for visual feedback; 2) Mode GH, with a gamepad for control and HoloLens providing 3D visualization; and 3) Mode HH, where HoloLens serves as both control input and visualization device. Mode GH outperformed other modalities in subjective metrics, except for mental demand. It exhibited a median tracking error of 4.72 mm, a median targeting error of 1.01 mm, a median duration of 82.34 s, and a median natural logarithm of dimensionless squared jerk of 40.38 in the <italic>in-vitro</italic> study. Mode GH showed 8.5&#x0025;, 4.7&#x0025;, 6.5&#x0025;, and 3.9&#x0025; improvements over Mode GM and 1.5&#x0025;, 33.6&#x0025;, 34.9&#x0025;, and 8.1&#x0025; over Mode HH for tracking error, targeting error, duration, and dimensionless squared jerk, respectively. To sum up, the user study emphasizes the potential benefits of employing HoloLens for enhanced 3D visualization in catheterization. The user study also illustrates the advantages of using a gamepad for catheter teleoperation, including user-friendliness and passive haptic feedback, compared to HoloLens. To further gauge the potential of using a more traditional joystick as a control input device, an additional study utilizing the Haption Virtuose^TM robot was conducted. It reveals the potential for achieving smoother trajectories, with a 38.9&#x0025; reduction in total path length compared to a gamepad, potentially due to its larger range of motion and single-handed control.

While articulated surgical instruments have enabled the proliferation of minimally invasive interventions, procedures such as laparo-endoscopic single-site surgery are waning in popularity. One potential reason for this decline is a lack of sufficiently dexterous instruments. Although multi-steerable instruments exist, these are often complex and therefore expensive assemblies. Even when 3D printing was used to simplify the design of these instruments, the requirement for high-performance 3D printers limited the reduction in manufacturing costs. To tackle this issue, we propose six guidelines for converting a 3D printed compliant medical instrument from printing on a Digital Light Processing (DLP) printer to a Fused Filament Fabrication (FFF) printer. These guidelines provide a framework to manage and compensate for differences in the two processes to achieve comparable results at a reduced cost. The proposed guidelines were evaluated by assembling a FFF 3D printed prototype that shows equivalent performance to its DLP 3D printed counterpart.

Optical fiber-based shape sensing is gaining popularity in cardiac catheterization lately. Typically, these procedures are taking place under the guidance of fluoroscopy. However, fluoroscopy has several disadvantages. Thanks to fiber optic shape sensing and Electromagnetic Tracking (EMT), the 3D catheter shape can now be tracked in real-time without the need for fluoroscopy. Traditional optical fiber and EMT-based shape tracking methods have the drawback of the highest shape sensing error at the tip. The information offered by the EMT sensors is used mainly to localize the estimated shape in a fixed coordinate frame. In this letter, a novel approach for tracking the catheter is introduced to address the aforementioned problem. The catheter shape is directly reconstructed in the EMT coordinate frame by approximating the catheter shape by a number of Bézier curves while taking into account the curvatures measured by the optical fiber. Both 2D and 3D shape sensing experiments are conducted. The results of the 3D experiment show that the proposed method reduces the mean shape tracking error by approximately 38% (from 12.1 mm to 5.4 mm for a sensed length of 540 mm long) compared to the traditional method where the same number of sensors are used.

Minimally invasive catheter-based interventions normally take place under the guidance of fluoroscopy. However, fluoroscopy is harmful to both patients and clinicians. Moreover, it only offers 2-D shape visualization of flexible devices. To solve the problem of harmful radiation and offer 3-D pose and shape information, recent studies propose a combination of electromagnetic tracking (EMT) sensors and multicore fiber Bragg grating (FBG) fiber sensing. However, for robust localization, at least two EMT sensors are required to be attached to each multicore fiber. This may make the catheter overly complex and fragile. Furthermore, the inability of multicore FBG fibers to distinguish between twist-induced strain and bend-induced strain impacts shape sensing accuracy. This article proposes a new approach offering a precise shape sensing method that is robust against torsional twists and exploits symmetry and geometry to compensate for limited sensing information. The proposed approach originates from the observation that many interventional procedures employ a plurality of concentric instruments. By distributing sensors over these instruments, the complexity per instrument can be kept acceptable. The proposed sensor fusion approach ensures robust and superior shape reconstruction. Experiments in 3-D with ground truth generated by a stereo vision system have been done and yielded promising results. Compared to the state-of-the-art methods, the presented framework uses only half of the required EMT sensors per instrument resulting in significant spatial conservation while improving the catheter shape tracking accuracy by 57%.

In this article, a deep learning method for the shape sensing of continuum robots based on multicore fiber bragg grating (FBG) fiber is introduced. The proposed method, based on an artificial neural network (ANN), differs from traditional approaches, where accurate shape reconstruction requires a tedious characterization of many characteristic parameters. A further limitation of traditional approaches is that they require either multiple fibers, whose location relative to the centerline must be precisely known (calibrated), or a single multicore fiber whose position typically coincides with the neutral line. The proposed method addresses this limitation and, thus, allows shape sensing based on a single multicore fiber placed off-center. This helps in miniaturizing and leaves the central channel available for other purposes. The proposed approach was compared to a recent state-of-the-art model-based shape sensing approach. A two-degree-of-freedom benchtop fluidics-driven catheter system was built to validate the proposed ANN. The proposed ANN-based shape sensing approach was evaluated on a 40-mm-long steerable continuum robot in both 3-D free-space and 2-D constrained environments, yielding an average shape sensing error of 0.24 and 0.49 mm, respectively. With these results, the superiority of the proposed approach compared to the recent model-based shape sensing method was demonstrated.

Increased demand for less invasive procedures has accelerated the adoption of Intraluminal Procedures (IP) and Endovascular Interventions (EI) performed through body lumens and vessels. As navigation through lumens and vessels is quite complex, interest grows to establish autonomous navigation techniques for IP and EI for reaching the target area. Current research efforts are directed toward increasing the Level of Autonomy (LoA) during the navigation phase. One key ingredient for autonomous navigation is Motion Planning (MP) techniques. This paper provides an overview of MP techniques categorizing them based on LoA. Our analysis investigates advances for the different clinical scenarios. Through a systematic literature analysis using the PRISMA method, the study summarizes relevant works and investigates the clinical aim, LoA, adopted MP techniques, and validation types. We identify the limitations of the corresponding MP methods and provide directions to improve the robustness of the algorithms in dynamic intraluminal environments. MP for IP and EI can be classified into four subgroups: node, sampling, optimization, and learning-based techniques, with a notable rise in learning-based approaches in recent years. One of the review's contributions is the identification of the limiting factors in IP and EI robotic systems hindering higher levels of autonomous navigation. In the future, navigation is bound to become more autonomous, placing the clinician in a supervisory position to improve control precision and reduce workload.

In cardiovascular interventions, when steering catheters and especially robotic catheters, great care should be paid to prevent applying too large forces on the vessel walls as this could dislodge calcifications, induce scars or even cause perforation. To address this challenge, this paper presents a novel compliant motion control algorithm that relies solely on position sensing of the catheter tip and knowledge of the catheter's behavior. The proposed algorithm features a data-driven tip position controller. The controller is trained based on a so-called control Long Short-Term Memory Network (control-LSTM). Trajectory following experiments are conducted to validate the quality of the proposed control-LSTM. Results demonstrated superior positioning capability with sub-degree precision of the new approach in the presence of severe rate-dependent hysteresis. Experiments both in a simplified setup as well as in an aortic phantom further show that the proposed approach allows reducing the interaction forces with the environment by around 70%. This work shows how deep learning can be exploited advantageously to avoid tedious modeling that would be needed to precisely steer continuum robots in constrained environments such as the patient's vasculature.

Precise control of robotic catheters remains challenging in interventions. Inherent non-linearities such as hysteresis and external disturbances such as blood flow or contact with the vessel walls have a large impact on the reachable positioning precision. As inaccurate positioning of the catheter tip could lead to tissue damage, controllers that would perform adequately in the presence of hysteresis and environmental contacts would be highly desirable. This paper proposes a method based on multiple Long Short-Term Memory Networks (LSTMs). To this end, a so-called free-space-LSTM (f-LSTM) is trained in order to steer the catheter when it moves in free. Constrained-space-LSTMs (c-LSTMs) are trained to drive the catheter when it is in contact with an obstacle. Based on contact estimation methods, LSTMs are switched. The f-LSTM and c-LSTMs are first tested in free space motion and under constraint situations. The results reveal that LSTMs perform well (RMSE < 0.5 mm) for a steerable robot section with a total length of 77 mm when tested in the same situation where trained. However, when f-LSTM and c-LSTM were tested in an environment different from the one in which they were trained, errors tended to increase. The results highlight the need to exhaustively estimate the contact location and switch between different LSTMs accordingly. The effective working range of a c-LSTM was investigated as well. Experiments showed that a well-Trained single c-LSTM could be used effectively in a range of 8.8 mm among the entire length of a steerable catheter section, while maintaining the average tip positioning error below 2 mm in this range.

Continuum robots such as robotic catheters are increasingly being used in minimally invasive surgery. Compliance contributes to enhanced safety during e.g. catheter insertion, however, estimation of contact force and location may help clinicians avoiding exerting excessive force. Ultimately this could lead to faster and safer interventions. Researchers proposed force sensors integrated in the catheter tip in the past. However, such sensors add extra complexity to the catheter design. Also, tip force sensors do not provide insights on forces that act along the catheter length. This paper proposes a data-driven approach for localizing contact forces that appear over the length of the catheter. The proposed approach consists of a collision detection method and a contact localization method. The framework only requires the measurement of the catheter's shape which can be done by an embedded multi-core Fiber Bragg Grating fiber. The method was validated experimentally with a 3D-printed continuum robot with an integrated multi-core fiber. A second contact localization method which is based on identifying the discontinuity in the measured curvature, is also implemented and compared with the proposed method. The static and dynamic experiments show a mean average localization error of 2.3 mm and 4.3 mm which correspond to respectively 3.3% and 6.1% of a 70 mm long flexible robot. These findings demonstrate that the proposed framework outperforms the previous methods and yield promising results. The contact state estimation algorithm can detect collisions in at most approximately 1.08s.

Catheters are increasingly being used to tackle problems in the cardiovascular system. However, positioning precision of the catheter tip is negatively affected by hysteresis. To ensure tissue damage due to imprecise positioning is avoided, hysteresis is to be understood and compensated for. This work investigates the feasibility to model hysteresis with a Long Short-Term Memory (LSTM) network. A bench-top setup containing a catheter distal segment was developed for model evaluation.The LSTM was first tested using four groups of test datasets containing diverse patterns. To compare with the LSTM, a Deadband Rate-Dependent Prandtl-Ishlinskii (DRDPI) model and a Support Vector Regression (SVR) model were established. The results demonstrated that the LSTM is capable of predicting the tip bending angle with sub-degree precision. The LSTM outperformed the DRDPI model and the SVR model by 60.1% and 36.0%, respectively, in arbitrarily varying signals. Next, the LSTM was further validated in a 3D reconstruction experiment using Forward-Looking Optical Coherence Tomography (FL-OCT). The results revealed that the LSTM was able to accurately reconstruct the environment with a reconstruction error below 0.25 mm. Overall, the proposed LSTM enabled precise free-space control of a robotic catheter in the presence of severe hysteresis. The LSTM predicted the catheter tip response precisely based on proximal input pressure, minimizing the need to install sensors at the catheter tip for localization.

Optical ultrasound sensing is a promising technique for the emerging field of biomedical photoacoustic imaging. Previously at imec, micro-opto-mechanical sensors with integrated Mach-Zehnder interferometers were designed and demonstrated as highly sensitive for static pressure sensing. At quasi-static pressures, they are demonstrated to operate as a sensitive microphone. In this study, the application range is extended to include dynamic pressures targeting a proof of concept for a photoacoustic imaging application. To achieve this, the dynamic behavior of the pressure sensor is firstly characterized, showing its resonance frequency at 73.8 kHz. Since this is below the range desired in photoacoustic imaging, several approaches for resonance frequency increase are investigated, resulting positively for incomplete membrane release and new designs. These approaches are analyzed based on the evaluated sensor sensitivity, allowing the selection of optimal design. The mentioned evaluation of sensor sensitivity is possible due to a developed methodology based on measurement data, analytical and accurate acousto-mechanical finite element models. The developed methodology is demonstrated throughout a range of membrane sizes, frequencies and modes of vibration allowing the selection of maximum sensitivity design. In this study, a micro-opto-mechanical ultrasound sensor based on integrated Mach-Zehnder interferometer is designed for 1 MHz operation in a photoacoustic imaging environment and optimized for sensitivity.