Motion tracking software for assessing laparoscopic surgical proficiency has been proven to be effective in differentiating between expert and novice performances. However, with several indices that can be generated from the software, there is no set threshold that can be used to benchmark performances. The aim of this study was to identify the best possible algorithm that can be used to benchmark expert, intermediate and novice performances for objective evaluation of psychomotor skills. 12 video recordings of various surgeons were collected in a blinded fashion. Data from our previous study of 6 experts and 23 novices was also included in the analysis to determine thresholds for performance. Video recording were analyzed both by the Kinovea 0.8.15 software and a blinded expert observer using the CAT form. Multiple algorithms were tested to accurately identify expert and novice performances. ½ L + 13 A + 16 J scoring of path length, average movement and jerk index respectively resulted in identifying 23/24 performances. Comparing the algorithm to CAT assessment yielded in a linear regression coefficient R² of 0.844. The value of motion tracking software in providing objective clinical evaluation and retrospective analysis is evident. Given the prospective use of this tool the algorithm developed in this study proves to be effective in benchmarking performances for psychomotor skills evaluation.

Virtual reality (VR) training is widely used in several minimal invasive surgery (MIS) training curricula for procedural training. However, VR training in its current state lack immersive training environments, such as using head-mounted displays that is implemented in military or aviation training and even entertainment. The virtual operating room simulation setup (VORSS) is explored in this study to determine the effectiveness of immersive training in MIS. Twenty-eight surgeons and surgical trainees performed a laparoscopic cholecystectomy on the VORSS comprising of a head-mounted 360-degree realistic OR surrounding on a VR laparoscopic simulator. The VORSS replicated a full setup of instruments and surgical team-members as well as some of the distractions occurring during surgical procedures. Questionnaires were followed by semi-structured interviews to collect the data. Experts and novices found the VORSS to be intuitive and easy to use (p = 0.001). The outcome of the usability test, applying QUESI and NASA-TLX, reflected the usability of the VORSS (p < 0.05), at the cognitive level, which indicates a good sense of immersion and satisfaction, when performing the procedure within VORSS. The need for personalized experience within the setup was strongly noted from most of the participants. The VORSS for procedural training has the potential to become a useful tool to provide immersive training in MIS surgery. Further optimizing of the VORSS realism and introduction of distractors in the OR should result in an improvement of the system.

Background: The purpose of this study was to validate the eoSim, an affordable and mobile inanimate laparoscopic simulator with instrument tracking capabilities, regarding face, content and construct validity on complex suturing tasks. Methods: Participants recruited for this study were novices (no laparoscopic experience), target group for this training (surgical/gynaecologic/urologic residents, > 10 basic and < 20 advanced laparoscopic procedures) and experts (> 20 advanced laparoscopic procedures). Each participant performed the intracorporeal suturing exercise (Task 1), an upside down needle transfer (Task 2, developed for this study) and an anastomosis needle transfer (Task 3). Following, the participants completed a questionnaire regarding their demographics and opinion on the eoSim in terms of realism, didactic value and usability. Measured outcome parameters were time, distance, percentage of instrument tip off-screen, working area, speed, acceleration and smoothness. Results: In total, 104 participants completed the study, of which 60 novices, 31 residents and 13 experts. Face and content validity results showed a mean positive opinion on realism (3.9 Task 1, 3.6 Task 2 and 3.7 Task 3), didactic value (4.0, 3.4 and 3.7, respectively) and usability (4.2. 3.7 and 4.0, respectively). There were no significant differences in these outcomes between the specified expertise groups. Construct validity results showed significant differences between experts, target group or novices for Task 1 in terms of time (means 339, 607 and 1224 s, respectively, p < 0.001) and distance (means 8.1, 15.6 and 21.7 m, respectively, p < 0.001). Task 2 showed significant differences between groups regarding time (p < 0.001), distance (p 0.003), off-screen (p < 0.001) and working area (p < 0.001). Task 3 showed significant differences between groups, after subanalyses, on total number of stitches (p < 0.001), time per stitch (p < 0.001) and distance per stitch (p < 0.001). Conclusions: The results of this study indicate that the eoSim is a potential meaningful and valuable simulator in the training of suturing tasks.

Background: The eoSim^® laparoscopic augmented reality (AR) simulator has instrument tracking capabilities that may be suitable for implementation in laparoscopic training. The objective is to assess face, content, and construct validity of this simulator for basic laparoscopic skills training. Methods: Participants were divided into three groups: novices (no training), intermediates (<50 laparoscopic procedures), and experts (>50 laparoscopic procedures). Three basic tasks were completed on the simulator: thread transfer (1), cyst dissection (2), and tube ligation (3). A questionnaire was completed on realism, didactic value, and usability of the simulator. Measured outcome parameters were as follows: time, distance, time off screen, average speed, acceleration, and smoothness. Results: Mean ± standard deviation scores on realism were positive (Task 1 or T1; 3.9 ± 0.7, P = .13, T2; 3.7 ± 0.7, P = .07, T3; 3.7 ± 0.07), as well as didactic value (T1; 3.9 ± 0.8, P = .71, T2; 3.9 ± 0.8, P = .31, T3; 4.0 ± 0.8, P = .40). Usability was valued the highest, with mean scores between 3.9 and 4.3 (T1; P = .71, T2; P = .80, T3; P = .85). Scores did not differ significantly between groups. Experts were significantly faster (Task 1; P < .001, Task 2; P = .042, Task 3: P < .001) with higher handling speed for tasks 2 and 3 (Task 1; P = .20, task 2; P = .034, task 3; P = .049). Results for other outcome parameters also indicated experts had better instrument control and efficiency than novices, although these differences did not reach statistical significance. Conclusions: The eoSim laparoscopic AR simulator is regarded as a realistic, accessible, and useful tool for the training of basic laparoscopic skills, with good face validity. Construct validity of the eoSim AR simulator was demonstrated on several core variables, but not all.

Background: Laparoscopic suturing can be technically challenging and requires extensive training to achieve competency. To date no specific and objective assessment method for laparoscopic suturing and knot tying is available that can guide training and monitor performance in these complex surgical skills. In this study we aimed to develop a laparoscopic suturing competency assessment tool (LS-CAT) and assess its inter-observer reliability. Methods: We developed a bespoke CAT tool for laparoscopic suturing through a structured, mixed methodology approach, overseen by a steering committee with experience in developing surgical assessment tools. A wide Delphi consultation with over twelve experts in laparoscopic surgery guided the development stages of the tool. Following, subjects with different levels of laparoscopic expertise were included to evaluate this tool, using a simulated laparoscopic suturing task which involved placing of two surgical knots. A research assistant video recorded and anonymised each performance. Two blinded expert surgeons assessed the anonymised videos using the developed LS-CAT. The LS-CAT scores of the two experts were compared to assess the inter-observer reliability. Lastly, we compared the subjects’ LS-CAT performance scores at the beginning and end of their learning curve. Results: This study evaluated a novel LS-CAT performance tool, comprising of four tasks. Thirty-six complete videos were analysed and evaluated with the LS-CAT, of which the scores demonstrated excellent inter-observer reliability. Cohen’s Kappa analysis revealed good to excellent levels of agreement for almost all tasks of both instrument handling and tissue handling (0.87; 0.77; 0.75; 0.86; 0.85, all with p < 0.001). Subjects performed significantly better at the end of their learning curve compared to their first attempt for all LS-CAT items (all with p < 0.001). Conclusions: We developed the LS-CAT, which is a laparoscopic suturing grading matrix, with excellent inter-rater reliability and to discriminate between experience levels. This LS-CAT has a potential for wider use to objectively assess laparoscopic suturing skills.

Background: The use of motion tracking has been proved to provide an objective assessment in surgical skills training. Current systems, however, require the use of additional equipment or specialised laparoscopic instruments and cameras to extract the data. The aim of this study was to determine the possibility of using a software-based solution to extract the data. Methods: 6 expert and 23 novice participants performed a basic laparoscopic cholecystectomy procedure in the operating room. The recorded videos were analysed using Kinovea 0.8.15 and the following parameters calculated the path length, average instrument movement and number of sudden or extreme movements. Results: The analysed data showed that experts had significantly shorter path length (median 127 cm vs. 187 cm, p = 0.01), smaller average movements (median 0.40 cm vs. 0.32 cm, p = 0.002) and fewer sudden movements (median 14.00 vs. 21.61, p = 0.001) than their novice counterparts. Conclusion: The use of software-based video motion tracking of laparoscopic cholecystectomy is a simple and viable method enabling objective assessment of surgical performance. It provides clear discrimination between expert and novice performance.