Changes in functional neuroimaging measures as novices gain proficiency on the fundamentals of laparoscopic surgery suturing task.

Significance: As trainees practice fundamental surgical skills, they typically rely on performance measures such as time and errors, which are limited in their sensitivity. Aim: The goal of our study was to evaluate the use of portable neuroimaging measures to map the neural processes associated with learning basic surgical skills. Approach: Twenty-one subjects completed 15 sessions of training on the fundamentals of laparoscopic surgery (FLS) suture with intracorporeal knot-tying task in a box trainer. Functional near infrared spectroscopy data were recorded using an optode montage that covered the prefrontal and sensorimotor brain areas throughout the task. Average oxy-hemoglobin (HbO) changes were determined for repetitions performed during the first week of training compared with the third week of training. Statistical differences between the time periods were evaluated using a general linear model of the HbO changes. Results: Average performance scores across task repetitions increased significantly from the first day to the last day of training ( p < 0.01 ). During the first day of training, there was significant lateral prefrontal cortex (PFC) activation. On the final day, significant activation was observed in the PFC, as well as the sensorimotor areas. When comparing the two periods, significant differences in activation ( p < 0.05 ) were found for the right medial PFC and the right inferior parietal gyrus. While gaining proficiency, trainees activated the perception-action cycle to build a perceptual model and then apply the model to improve task execution. Conclusions: Learners engaged the sensorimotor areas more substantially as they developed skill on the FLS suturing task. These findings are consistent with findings for the FLS pattern cutting task and contribute to the development of objective metrics for skill evaluation.

Predicting Resident Competence for Otolaryngology Key Indicator Procedures.

OBJECTIVE: Competency-based surgical education requires practical assessments and meaningful benchmarks. In otolaryngology, key indicator procedure (KIP) minima are indicators of surgical exposure during training, yet it remains unknown how many times trainees must be evaluated on KIPs to ensure operative competence. Herein, we used Bayesian mixed effects models to compute predicted performance expectations for KIPs. METHODS: From November 2017 to September 2021, a smartphone application (SIMPL OR) was used by attendings at five otolaryngology training programs to rate resident operative performance after each case on a five-level scale. Bayesian mixed effects models were used to estimate the probability that postgraduate year (PGY) 3, 4, or 5 trainees would earn a "practice-ready" (PR) rating on a subsequent evaluation based on their previously earned PR ratings for each KIP. Probabilities of earning a subsequent PR rating were examined for interpretability, and cross-validation was used to assess predictive validity. RESULTS: A total of 842 assessments of KIPs were submitted by 72 attendings for 92 residents PGY 2-5. The predictive model had an average Area Under the Receiver Operating Curve of 0.77. The number of prior PR ratings that senior residents needed to attain a 95% probability of earning a PR rating on a subsequent evaluation was estimated for each KIP. For example, for mastoidectomies, PGY4 residents needed to earn 10 PR ratings whereas PGY5 residents needed 4 PR ratings on average to have a 95% probability of attaining a PR rating on a subsequent evaluation. CONCLUSION: Predictive modeling can inform assessment benchmarks for competency-based surgical education. LEVEL OF EVIDENCE: NA Laryngoscope, 133:3341-3345, 2023.

A Platform for Tracking Surgeon and Observer Gaze as a Surrogate for Attention in Ophthalmic Surgery.

Purpose: To develop and validate a platform that can extract eye gaze metrics from surgeons observing cataract and vitreoretinal procedures and to enable post hoc data analysis to assess potential discrepancies in eye movement behavior according to surgeon experience. Design: Experimental, prospective, single-center study. Participants: Eleven ophthalmic surgeons observing deidentified vitreoretinal and cataract surgical procedures performed at a single university-based medical center. Methods: An open-source platform was developed to extract gaze coordinates and metrics from ophthalmic surgeons via a computer vision algorithm in conjunction with a neural network to track and segment instruments and tissues, identifying areas of attention in the visual field of study subjects. Eleven surgeons provided validation data by watching videos of 6 heterogeneous vitreoretinal and cataract surgical phases. Main Outcome Measures: Accuracy and distance traveled by the eye gaze of participants and overlap of the participants' eye gaze with instruments and tissues while observing surgical procedures. Results: The platform demonstrated repeatability of > 94% when acquiring the eye gaze behavior of subjects. Attending ophthalmic surgeons and clinical fellows exhibited a lower overall cartesian distance traveled in comparison to resident physicians in ophthalmology (P < 0.02). Ophthalmology residents and clinical fellows exhibited more fixations to the display area where surgical device parameters were superimposed than attending surgeons (P < 0.05). There was a trend toward gaze overlap with the instrument tooltip location among resident physicians in comparison to attending surgeons and fellows (41.42% vs. 34.8%, P > 0.2). The number and duration of fixations did not vary substantially among groups (P > 0.3). Conclusions: The platform proved effective in extracting gaze metrics of ophthalmic surgeons. These preliminary data suggest that surgeon gaze behavior differs according to experience.

Deep neural network architecture for automated soft surgical skills evaluation using objective structured assessment of technical skills criteria.

PURPOSE: Classic methods of surgery skills evaluation tend to classify the surgeon performance in multi-categorical discrete classes. If this classification scheme has proven to be effective, it does not provide in-between evaluation levels. If these intermediate scoring levels were available, they would provide more accurate evaluation of the surgeon trainee. METHODS: We propose a novel approach to assess surgery skills on a continuous scale ranging from 1 to 5. We show that the proposed approach is flexible enough to be used either for scores of global performance or several sub-scores based on a surgical criteria set called Objective Structured Assessment of Technical Skills (OSATS). We established a combined CNN+BiLSTM architecture to take advantage of both temporal and spatial features of kinematic data. Our experimental validation relies on real-world data obtained from JIGSAWS database. The surgeons are evaluated on three tasks: Knot-Tying, Needle-Passing and Suturing. The proposed framework of neural networks takes as inputs a sequence of 76 kinematic variables and produces an output float score ranging from 1 to 5, reflecting the quality of the performed surgical task. RESULTS: Our proposed model achieves high-quality OSATS scores predictions with means of Spearman correlation coefficients between the predicted outputs and the ground-truth outputs of 0.82, 0.60 and 0.65 for Knot-Tying, Needle-Passing and Suturing, respectively. To our knowledge, we are the first to achieve this regression performance using the OSATS criteria and the JIGSAWS kinematic data. CONCLUSION: An effective deep learning tool was created for the purpose of surgical skills assessment. It was shown that our method could be a promising surgical skills evaluation tool for surgical training programs.

Multi-Modal Deep Learning for Assessing Surgeon Technical Skill.

This paper introduces a new dataset of a surgical knot-tying task, and a multi-modal deep learning model that achieves comparable performance to expert human raters on this skill assessment task. Seventy-two surgical trainees and faculty were recruited for the knot-tying task, and were recorded using video, kinematic, and image data. Three expert human raters conducted the skills assessment using the Objective Structured Assessment of Technical Skill (OSATS) Global Rating Scale (GRS). We also designed and developed three deep learning models: a ResNet-based image model, a ResNet-LSTM kinematic model, and a multi-modal model leveraging the image and time-series kinematic data. All three models demonstrate performance comparable to the expert human raters on most GRS domains. The multi-modal model demonstrates the best overall performance, as measured using the mean squared error (MSE) and intraclass correlation coefficient (ICC). This work is significant since it demonstrates that multi-modal deep learning has the potential to replicate human raters on a challenging human-performed knot-tying task. The study demonstrates an algorithm with state-of-the-art performance in surgical skill assessment. As objective assessment of technical skill continues to be a growing, but resource-heavy, element of surgical education, this study is an important step towards automated surgical skill assessment, ultimately leading to reduced burden on training faculty and institutes.

National Implementation of Simulator Training Improves Transurethral Resection of Bladder Tumours in Patients.

Background: Transurethral resection of bladder tumours (TURBT) is the initial diagnostic treatment for patients with bladder cancer. TURBT is not an easy procedure to master and simulator training may play a role in improving the learning curve. Objective: To implement a national training programme for simulation-based mastery learning in TURBT and explore operating theatre performance after training. Design setting and participants: From June 2019 to March 2021, 31 doctors at urology departments in Denmark performed two pretraining TURBT procedures on patients, followed by proficiency-based mastery learning on a virtual reality simulator and then two post-training TURBTs on patients. Outcome measurements and statistical analyses: Operating theatre performances were video-recorded and assessed by two independent, blinded raters using the Objective Structured Assessment for Transurethral Resection of Bladder Tumours Skills (OSATURBS) assessment tool. Paired-sample t tests were used to compare pretraining and post-training analyses and independent t tests for between-group comparisons. This trial is registered at ClinicalTrials.gov as NCT03864302. Results and limitations: Before training, novices had significantly lower performance scores in comparison to those with intermediate experience (p = 0.017) and experienced doctors (p < 0.001). After training, novices significantly improved their clinical performance score (from 11.4 to 17.1; p = 0.049, n = 10). Those with intermediate experience and experienced doctors did not benefit significantly from simulator training (p = 0.9 and p = 0.8, respectively). Conclusions: Novices improved their TURBT performance in the operating theatre after completing a proficiency-based training programme on a virtual reality simulator. Patient summary: We trained surgeons in an operation to remove bladder tumours using a virtual reality simulator. Novice doctors improved their performance significantly after the training, but the training effects for more experienced doctors were minimal. Therefore, we suggest the introduction of mandatory simulator training in the residency programme for urologists.

Validation of a Surgical Objective Structured Clinical Examination (S-OSCE) Using Convergent, Divergent, And Trainee-Based Assessments of Fidelity.

OBJECTIVE: Describe the validation of a surgical objective structured clinical examination (S-OSCE) for the purpose of competency assessment based on the Royal College of Canada's Can-MEDS framework. DESIGN: A surgical OSCE was developed to evaluate the management of common orthopedic surgical problems. The scores derived from this S-OSCE were compared to Ottawa Surgical Competency Operating Room Evaluation (O-SCORE), a validated entrustability assessment, to establish convergent validity. The S-OSCE scores were compared to Orthopedic In-Training Examination (OITE) scores to evaluate divergent validity. Resident evaluations of the clinical encounter with a standardized patient and the operative procedure were scored on a 10-point Likert scale for fidelity. SETTING: A tertiary level academic teaching hospital. PARTICIPANTS: 21 postgraduate year 2 to 5 trainees of a 5-year Canadian orthopedic residency program creating 160 operative case performances for review. RESULTS: There were 5 S-OSCE days, over a 4-year period (2016-2019) encompassing a variety of surgical procedures. Performance on the S-OSCE correlated strongly with the O-SCORE (Pearson correlation coefficient 0.88), and a linear regression analysis correlated moderately with year of training (R²â€¯= 0.5345). The Pearson correlation coefficient between the S-OSCE and OITE scores was 0.57. There was a significant increase in the average OITE score after the introduction of the surgical OSCE. Resident fidelity ratings were available from 16 residents encompassing 8 different surgical cases. The average score for the overall simulation (8.0±1.6) was significantly higher than the cadaveric surgical simulation (6.5 ± 0.8) (p < 0.001) CONCLUSIONS: The S-OSCE scores correlate strongly with an established form of assessment demonstrating convergent validity. The correlation between the S-OSCE and OITE scores was less, demonstrating divergent validity. Although residents rank the overall simulation highly, the fidelity of the cadaveric simulation may need improvement. Administration of a surgical OSCE can be used to evaluate preoperative and intraoperative decision making and complement other forms of assessment.

Developing a longitudinal database of surgical skills performance for practicing surgeons: A formal feasibility and acceptance inquiry.

BACKGROUND: We explored the feasibility and surgeons' perceptions of the utility of a longitudinal skills performance database. METHODS: A 10-station surgical skills assessment center was established at a national scientific meeting. Skills assessment volunteers (n = 189) completed a survey including opinions on practicing surgeons' skills evaluation, ethics, and interest in a longitudinal database. A subset (n = 23) participated in a survey-related interview. RESULTS: Nearly all participants reported interest in a longitudinal database and most believed there is an ethical obligation for such assessments to protect the public. Several interviewees specified a critical role for both formal and informal evaluation is to first create a safe and supportive environment. CONCLUSIONS: Participants support the construction of longitudinal skills databases that allow information sharing and establishment of professional standards. In a constructive environment, structured peer feedback was deemed acceptable to enhance and diversify surgeon skills. Large scale skills testing is feasible and scientific meetings may be the ideal location.

Reconstructing the nasal septum from instrument motion during septoplasty surgery.

Purpose: Surgery involves modifying anatomy to achieve a goal. Reconstructing anatomy can facilitate surgical care through surgical planning, real-time decision support, or anticipating outcomes. Tool motion is a rich source of data that can be used to quantify anatomy. Our work develops and validates a method for reconstructing the nasal septum from unstructured motion of the Cottle elevator during the elevation phase of septoplasty surgery, without need to explicitly delineate the surface of the septum. Approach: The proposed method uses iterative closest point registration to initially register a template septum to the tool motion. Subsequently, statistical shape modeling with iterative most likely oriented point registration is used to fit the reconstructed septum to Cottle tip position and orientation during flap elevation. Regularization of the shape model and transformation is incorporated. The proposed methods were validated on 10 septoplasty surgeries performed on cadavers by operators of varying experience level. Preoperative CT images of the cadaver septums were segmented as ground truth. Results: We estimated reconstruction error as the difference between the projections of the Cottle tip onto the surface of the reconstructed septum and the ground-truth septum segmented from the CT image. We found translational differences of 2.74 ( 2.06 - 2.81 ) mm and a rotational differences of 8.95 ( 7.11 - 10.55 ) deg between the reconstructed septum and the ground-truth septum [median (interquartile range)], given the optimal regularization parameters. Conclusions: Accurate reconstruction of the nasal septum can be achieved from tool tracking data during septoplasty surgery on cadavers. This enables understanding of the septal anatomy without need for traditional medical imaging. This result may be used to facilitate surgical planning, intraoperative care, or skills assessment.

Multi-institutional Validation of a Vaginal Hysterectomy Simulation Model for Resident Training.

STUDY OBJECTIVE: The purpose of the research was to both develop a vaginal hysterectomy model with surgically pertinent anatomic landmarks and assess its validity for simulation training. DESIGN: A low-cost, reproducible vaginal hysterectomy model with relevant anatomic landmarks for key surgical steps. SETTING: Nine academic and community-based obstetrics and gynecology residency programs. PARTICIPANTS: One hundred sixty-nine obstetrics and gynecology residents. INTERVENTIONS: A vaginal hysterectomy model with surgically pertinent anatomic landmarks was developed and tested for construct validity. MEASUREMENTS AND MAIN RESULTS: Of the 184 available residents, 169 (91%) participated in this study and performed a vaginal hysterectomy procedure on the described model. The validated objective 7-item global rating scale (GRS) and the 13-item task-specific checklist (TSC) were used as tools to assess performance. The median TSC and GRS scores correlated with year of training, prior experience, and trainee confidence. In addition, the TSC scores also correlated with the GRS scores (p <.001) with regard to performance and resident year of training. Receiver Operator Curves for identification of the residents meeting national residency accreditation minimum numbers for vaginal hysterectomy using the GRS and TSC scores had an area under the curve of 0.89 and 0.83, respectively. CONCLUSION: This reduced-cost vaginal hysterectomy model offers high construct validity and pertinence for simulation.

Surgical Games: A Simulation-Based Structured Assessment of Orthopedic Surgery Resident Technical Skill.

BACKGROUND: Simulation-based education and objective evaluation of surgical skill have been incorporated into many surgical training programs. We describe the development and implementation of a timed, multitask, station-based Surgical Games to evaluate orthopedic resident surgical skills. METHODS: Participants in the study were postgraduate-year 2 to 5 orthopedic surgery residents from a single institution. Residents completed 4-timed simulated tasks: cadaveric carpal tunnel release (CTR), Sawbones model of total knee arthroplasty (TKA), Sawbones model of ankle fracture open reduction internal fixation (ORIF), and knee arthroscopy simulator (KAS) of removal of loose body. Evaluations were performed using standardized score sheets by attending surgeons. Resident performance was analyzed by postgraduate-year and number of weeks of prior task-related residency training. RESULTS: A total of 32 residents were assessed at the 4 stations. Total scores were significantly different for CTR (p = 0.006), TKA (p = 0.05), and the KAS (p = 0.004) by year of training, but not for the ankle ORIF task. Residents with more task-specific experience performed significantly better on the KAS (p < 0.001), TKA (p = 0.002), and CTR (p = 0.02) tasks but not on the ankle ORIF task (p = 0.1). Overall, residents rated the exercise valuable to their education with mean scores of 3.9 ± 0.54 on a 5-point Likert Scale. CONCLUSIONS: This Surgical Games exercise provided an objective evaluation of surgical skill that correlated with year-in-training and prior experience in skill-specific rotations for the KAS, TKA, and CTR tasks. This surgical skills assessment provided an opportunity for effective structured feedback and identification of areas for improvement.

Developing and Assessing the Feasibility of Implementing a Surgical Objective Structured Clinical Skills Examination (S-OSCE).

OBJECTIVE: To1 describe the development and evaluate the feasibility of a surgical objective structured clinical examination (OSCE) for the purpose of competency assessment based on the Royal College of Canada's CanMEDS framework. DESIGN: A unique surgical OSCE was developed to evaluate the clinical and surgical management of common orthopaedic problems using simulated patients and cadaveric specimens. Cases were graded by degree of difficulty (less complex, complex, more complex) Developing an assessment tool with significant resource utilization and good correlation with traditional methods is challenging. The feasibility of an OSCE that evaluates independent clinical and surgical decision making was evaluated. In addition, as part of establishing construct validity, correlation of OSCE scores with previously validated O-scores was performed. SETTING: A tertiary level academic teaching hospital. PARTICIPANTS: Thirty-four Postgraduate year 3-5 trainees of a 5-year Canadian orthopedic residency program creating 96 operative case performances available for final review. RESULTS: The development of the OSCE cases involved a multistep process with attending surgeons, residents and a surgical education consultant. There were 4 different OSCE days, over a 3 year period (2016-2018) encompassing a variety of less complex and more complex procedures. Performance on the OSCE correlated strongly with the (O-SCORE, 0.89) and a linear regression analysis correlated moderately with year of training (r2 = 0.5737). The feasibility analysis demonstrated good financial practicality with solid programmatic integration. CONCLUSIONS: The unique surgical OSCE scores correlate strongly with an established entrustability scale. Administering this OSCE to evaluate preoperative and intraoperative decision making to complement other forms of assessment is feasible. The financial burden to training programs is modest in comparison to the insight gained by both residents and faculty.

Towards near real-time assessment of surgical skills: A comparison of feature extraction techniques.

BACKGROUND AND OBJECTIVE: Surgical skill assessment aims to objectively evaluate and provide constructive feedback for trainee surgeons. Conventional methods require direct observation with assessment from surgical experts which are both unscalable and subjective. The recent involvement of surgical robotic systems in the operating room has facilitated the ability of automated evaluation of the expertise level of trainees for certain representative maneuvers by using machine learning for motion analysis. The features extraction technique plays a critical role in such an automated surgical skill assessment system. METHODS: We present a direct comparison of nine well-known feature extraction techniques which are statistical features, principal component analysis, discrete Fourier/Cosine transform, codebook, deep learning models and auto-encoder for automated surgical skills evaluation. Towards near real-time evaluation, we also investigate the effect of time interval on the classification accuracy and efficiency. RESULTS: We validate the study on the benchmark robotic surgical training JIGSAWS dataset. An accuracy of 95.63, 90.17 and 90.26% by the Principal Component Analysis and 96.84, 92.75 and 95.36% by the deep Convolutional Neural Network for suturing, knot tying and needle passing, respectively, highlighted the effectiveness of these two techniques in extracting the most discriminative features among different surgical skill levels. CONCLUSIONS: This study contributes toward the development of an online automated and efficient surgical skills assessment technique.

What are the skills that represent expert-level laparoscopic suturing? A Delphi Study.

BACKGROUND: As the field of laparoscopic surgery grows, the need for a valid assessment of laparoscopic surgical skills becomes apparent. The objective of this study was to identify laparoscopic suturing skills reflective of performing at an expert level. METHODS: A Delphi Study was conducted with expert laparoscopic surgeons. Tasks were considered to meet criteria for consensus after 80% of survey participants answered "strongly agree." Results of the previous round of surveys were shared with participants at the start of the next round. The predefined endpoint for the Delphi was set as maximum of three rounds, reaching 80% consensus on each skill, or if > 50% of initial respondents fail to return for subsequent surveys. RESULTS: After three rounds of surveys completed by 13 advanced laparoscopic surgeons, 8 observed skills met consensus (> 80% selected "strongly agree"): forehand suturing, continuous suturing, avoiding tissue trauma, achieving a technically acceptable final product (tight closure), tying a secure knot at the end of suturing, maintaining/checking for appropriate tightness of running suture, accurate depth perception, and loading the needle in < 3 attempts. Skills that approached consensus (with > 50% ranking them as "strongly agree") included correct penetration of target tissue, ability to create a surgeon's knot, awareness of needle tip position, confined space suturing, following the curve of the needle, smooth movements of instruments, loading the needle correctly, adjusting for tissue thickness, and bimanual dexterity. CONCLUSIONS: Using Delphi methodology, we identified the skills that are representative of expert laparoscopic suturing. This information can be used to create an assessment tool for this domain. Validation of such a tool will help advance the surgical field towards true competency-based assessment of expertise.

Intraoperative Point of View Video Capture and Surgical Segmentation in Carpal Tunnel Release: A Feasibility Analysis.

OBJECTIVE: The purpose of this study was to (1) examine the feasibility of intraoperative point of view video while performing open and endoscopic carpal tunnel release (CTR), (2) define surgical segments of CTR, and (3) describe the duration of various surgical steps of open versus endoscopic CTR in a teaching setting. DESIGN: Fellowship trained hand surgeons reached consensus on surgical segments for CTR. Adult patients 18 and older previously indicated for CTR in clinic were eligible. Head-mounted point-of-view cameras were worn during endoscopic and open CTR by resident surgeons. Video was reviewed to determine segment duration. Independent sample t tests were used for comparison of duration by technique with statistical significance set as p < 0.05. SETTING: University of Iowa Hospitals and Clinics; 200 Hawkins Dr, Iowa City, IA 52242; Tertiary Academic Medical Center. PARTICIPANTS: Orthopedic Surgery Residents and Orthopedic Surgery Faculty. RESULTS: Surgical segments were defined as incision, dissection of superficial soft tissue structures, transection of the carpal ligament, and surgical incision closure. Twelve of 14 video capture events yielded data. In the teaching setting, the average duration of endoscopic CTR was 609.5 seconds (±111.07) versus 547.75 seconds (±82.06) for open with p value = 0.406. No surgical segments were significantly different. Transition time from dissection to ligament transection differed significantly (p = 0.004) between endoscopic (46.88 seconds ± 19.19) and open (9.0 seconds ± 7.90) CTR. Transition time between ligament transection and closure was significantly different (p = 0.029) among endoscopic (50.5 seconds ± 15.0) and open (26.25 seconds ± 2.99) CTR. CONCLUSIONS: Point-of-view video capture is feasible for the capture of video during a common hand surgery procedure. A method for managing device battery power is necessary for future applications. CTR can be defined as, and described in, individual procedure segments potentially useful for surgical education as well as efficiency improvements. Identification of surgical segments may aid the development of better objective tools for the assessment of surgeon skill and competency for common orthopedic procedures.

Detecting Mental Workload in Surgical Teams Using a Wearable Single-Channel Electroencephalographic Device.

OBJECTIVE: To assess the sensitivity of an electroencephalographic (EEG)-based index, the prefrontal beta power, to quantify the mental workload in surgeons in real scenarios. Such EEG-based index might offer unique and unbiased measures of overload, a crucial factor when designing learning and training surgical programs. DESIGN: The experiment followed a 2 × 2 × 2 within subjects design with 3 factors: (1) Surgical Role during the surgery (primary surgeon vs. assistant surgeon), (2) the Surgical Procedure (laparo-endoscopic single-site [LESS] surgery vs. multiport laparoscopic surgery [MPS]), and (3) the Suturing Techniques (interrupted vs. continuous suture). SETTING: The study was carried out at the Advanced Multi-Purpose Simulation and Technological Innovation Complex situated at IAVANTE (Granada, Spain). METHODS: Four surgical teams (primary surgeon and assistant surgeon, experts in MPS) performed 8 surgical exercises on porcine models, under different task complexities. They performed 2 suturing techniques (continuous and interrupted), employing a low complex procedure (MPS) and a high complex procedure (LESS). Surgeons acted as the primary surgeon during half of the exercises, and, as the assistant surgeon, during the rest of them. Simultaneously, we monitored EEG prefrontal EEG beta power spectra of both surgeons, using 2 synchronized wearable EEG devices. We also collected performance and subjective data. RESULTS: Surgical complexity modulated prefrontal beta power. LESS surgery caused significant higher prefrontal beta power for both suturing techniques for both surgical roles which indicates higher demands than MPS. Perceived task complexity, overall surgical evaluation, and laparoscopic execution time confirmed EEG-based results. Finally, subjective ratings of surgical complexity differentiated between surgical roles within the same exercise, even when prefrontal beta power did not. CONCLUSIONS: To detect mental overload when surgeons are engaged with complex surgeries, real or simulated, is still guesswork. EEG-based indices have great potential as objective and nonintrusive measures to assess mental overload in surgeons. Furthermore, EEG-based indices might play a relevant role in monitoring surgeons and residents' cognitive state during their training.

Use of a Novel Measure of Nontechnical Skills in Surgical Trainees: Is There an Association With Technical Skills Performance?

OBJECTIVE: To investigate whether scores on a psychological measure of concentration and interpersonal characteristics, The Attentional and Interpersonal Style Inventory (TAIS), are associated with performance of surgical skills. DESIGN: Postgraduate surgical trainees completed an operative surgical skills assessment in the simulation laboratory and the psychological measure (TAIS). The surgical skills assessment consisted of 6 tasks (3 per trainee): laceration suturing; lipoma excision; incision and closure of a laparotomy wound; bowel anastomosis; saphenofemoral junction ligation and basic laparoscopic skills. The association between operative surgical skill performance and TAIS factors was investigated. SETTING: The TAIS assessments and surgical skills assessments were conducted at the National Surgical Training Centre at the Royal College of Surgeons in Ireland (RCSI). PARTICIPANTS: One hundred and two surgical trainees in years one and two (PGY 2-3 equivalent) participated in the study. RESULTS: Performance on 2 of the 6 tasks assessed (bowel anastomosis and lipoma excision) were positively associated with multiple TAIS factors (energy, confidence, competitiveness, extroversion, self-criticism and performing under pressure). Another factor, focus over time, was significantly associated with scores on the lipoma excision task. CONCLUSIONS: Trainees with high levels of energy, confidence, competitiveness, extroversion, and focus over time and low levels of self-criticism demonstrated better performance on specific technical skills tasks.

Video and accelerometer-based motion analysis for automated surgical skills assessment.

PURPOSE: Basic surgical skills of suturing and knot tying are an essential part of medical training. Having an automated system for surgical skills assessment could help save experts time and improve training efficiency. There have been some recent attempts at automated surgical skills assessment using either video analysis or acceleration data. In this paper, we present a novel approach for automated assessment of OSATS-like surgical skills and provide an analysis of different features on multi-modal data (video and accelerometer data). METHODS: We conduct a large study for basic surgical skill assessment on a dataset that contained video and accelerometer data for suturing and knot-tying tasks. We introduce "entropy-based" features-approximate entropy and cross-approximate entropy, which quantify the amount of predictability and regularity of fluctuations in time series data. The proposed features are compared to existing methods of Sequential Motion Texture, Discrete Cosine Transform and Discrete Fourier Transform, for surgical skills assessment. RESULTS: We report average performance of different features across all applicable OSATS-like criteria for suturing and knot-tying tasks. Our analysis shows that the proposed entropy-based features outperform previous state-of-the-art methods using video data, achieving average classification accuracies of 95.1 and 92.2% for suturing and knot tying, respectively. For accelerometer data, our method performs better for suturing achieving 86.8% average accuracy. We also show that fusion of video and acceleration features can improve overall performance for skill assessment. CONCLUSION: Automated surgical skills assessment can be achieved with high accuracy using the proposed entropy features. Such a system can significantly improve the efficiency of surgical training in medical schools and teaching hospitals.

Energy-Based Metrics for Arthroscopic Skills Assessment.

Minimally invasive skills assessment methods are essential in developing efficient surgical simulators and implementing consistent skills evaluation. Although numerous methods have been investigated in the literature, there is still a need to further improve the accuracy of surgical skills assessment. Energy expenditure can be an indication of motor skills proficiency. The goals of this study are to develop objective metrics based on energy expenditure, normalize these metrics, and investigate classifying trainees using these metrics. To this end, different forms of energy consisting of mechanical energy and work were considered and their values were divided by the related value of an ideal performance to develop normalized metrics. These metrics were used as inputs for various machine learning algorithms including support vector machines (SVM) and neural networks (NNs) for classification. The accuracy of the combination of the normalized energy-based metrics with these classifiers was evaluated through a leave-one-subject-out cross-validation. The proposed method was validated using 26 subjects at two experience levels (novices and experts) in three arthroscopic tasks. The results showed that there are statistically significant differences between novices and experts for almost all of the normalized energy-based metrics. The accuracy of classification using SVM and NN methods was between 70% and 95% for the various tasks. The results show that the normalized energy-based metrics and their combination with SVM and NN classifiers are capable of providing accurate classification of trainees. The assessment method proposed in this study can enhance surgical training by providing appropriate feedback to trainees about their level of expertise and can be used in the evaluation of proficiency.

Quantifying the cognitive cost of laparo-endoscopic single-site surgeries: Gaze-based indices.

BACKGROUND: Despite the growing interest concerning the laparo-endoscopic single-site surgery (LESS) procedure, LESS presents multiple difficulties and challenges that are likely to increase the surgeon's cognitive cost, in terms of both cognitive load and performance. Nevertheless, there is currently no objective index capable of assessing the surgeon cognitive cost while performing LESS. We assessed if gaze-based indices might offer unique and unbiased measures to quantify LESS complexity and its cognitive cost. We expect that the assessment of surgeon's cognitive cost to improve patient safety by measuring fitness-for-duty and reducing surgeons overload. METHODS: Using a wearable eye tracker device, we measured gaze entropy and velocity of surgical trainees and attending surgeons during two surgical procedures (LESS vs. multiport laparoscopy surgery [MPS]). None of the participants had previous experience with LESS. They performed two exercises with different complexity levels (Low: Pattern Cut vs. High: Peg Transfer). We also collected performance and subjective data. RESULTS: LESS caused higher cognitive demand than MPS, as indicated by increased gaze entropy in both surgical trainees and attending surgeons (exploration pattern became more random). Furthermore, gaze velocity was higher (exploration pattern became more rapid) for the LESS procedure independently of the surgeon's expertise. Perceived task complexity and laparoscopic accuracy confirmed gaze-based results. CONCLUSION: Gaze-based indices have great potential as objective and non-intrusive measures to assess surgeons' cognitive cost and fitness-for-duty. Furthermore, gaze-based indices might play a relevant role in defining future guidelines on surgeons' examinations to mark their achievements during the entire training (e.g. analyzing surgical learning curves).