RESUMO
BACKGROUND: Video-based review is paramount for operative performance assessment but can be laborious when performed manually. Hierarchical Task Analysis (HTA) is a well-known method that divides any procedure into phases, steps, and tasks. HTA requires large datasets of videos with consistent definitions at each level. Our aim was to develop an AI model for automated segmentation of phases, steps, and tasks for laparoscopic cholecystectomy videos using a standardized HTA. METHODS: A total of 160 laparoscopic cholecystectomy videos were collected from a publicly available dataset known as cholec80 and from our own institution. All videos were annotated for the beginning and ending of a predefined set of phases, steps, and tasks. Deep learning models were then separately developed and trained for the three levels using a 3D Convolutional Neural Network architecture. RESULTS: Four phases, eight steps, and nineteen tasks were defined through expert consensus. The training set for our deep learning models contained 100 videos with an additional 20 videos for hyperparameter optimization and tuning. The remaining 40 videos were used for testing the performance. The overall accuracy for phases, steps, and tasks were 0.90, 0.81, and 0.65 with the average F1 score of 0.86, 0.76 and 0.48 respectively. Control of bleeding and bile spillage tasks were most variable in definition, operative management, and clinical relevance. CONCLUSION: The use of hierarchical task analysis for surgical video analysis has numerous applications in AI-based automated systems. Our results show that our tiered method of task analysis can successfully be used to train a DL model.
Assuntos
Colecistectomia Laparoscópica , Aprendizado Profundo , Humanos , Redes Neurais de Computação , ColecistectomiaRESUMO
BACKGROUND: Perineal proctectomy is a complex procedure that requires advanced skills. Currently, there are no simulators for training in this procedure. As part of our objective of developing a virtual reality simulator, our goal was to develop and validate task-specific metrics for the assessment of performance for this procedure. We conducted a three-phase study to establish task-specific metrics, obtain expert consensus on the appropriateness of the developed metrics, and establish the discriminant validity of the developed metrics. METHODS: In phase I, we utilized hierarchical task analysis to formulate the metrics. In phase II, a survey involving expert colorectal surgeons determined the significance of the developed metrics. Phase III was aimed at establishing the discriminant validity for novices (PGY1-3) and experts (PGY4-5 and faculty). They performed a perineal proctectomy on a rectal prolapse model. Video recordings were independently assessed by two raters using global ratings and task-specific metrics for the procedure. Total scores for both metrics were computed and analyzed using the Kruskal-Wallis test. A Mann-Whitney U test with Benjamini-Hochberg correction was used to evaluate between-group differences. Spearman's rank correlation coefficient was computed to assess the correlation between global and task-specific scores. RESULTS: In phase II, a total of 23 colorectal surgeons were recruited and consensus was obtained on all the task-specific metrics. In phase III, participants (n = 22) included novices (n = 15) and experts (n = 7). There was a strong positive correlation between the global and task-specific scores (rs = 0.86; P < 0.001). Significant between-group differences were detected for both global (χ2 = 15.38; P < 0.001; df = 2) and task-specific (χ2 = 11.38; P = 0.003; df = 2) scores. CONCLUSIONS: Using a biotissue rectal prolapse model, this study documented high IRR and significant discriminant validity evidence in support of video-based assessment using task-specific metrics.
RESUMO
INTRODUCTION: Approximately one-third of surgical patients exhibit low health literacy, and 39% of our patients are primary Spanish speakers. We first evaluated the current content of our arteriovenous fistula/graft discharge instruction (DCI) templates. Using the Plan-Do-Study-Act cycle quality improvement methodology, we then aimed to optimize the readability and formally translate new DCI and evaluate usage and inappropriate bouncebacks following implementation. METHODS: Current arteriovenous fistula/graft template content was reviewed by the literacy department for readability and vascular faculty for completeness and accuracy. The literacy department edits were categorized by word choice, added/removed content, format change, and grammatical errors. Two vascular surgeons rated completeness and accuracy on a Likert scale (1-5). Retrospective chart review was performed for telephone calls and emergency department bouncebacks for 3 mo flanking new DCI implementation. RESULTS: Of the 10 templates, all were in English and word count ranged from 192 to 990 words. Despite each template including all necessary subcategories, the median number of edits per 100 words was 9.2 [7.0-9.5]. Approximately half of the edits (5.4 [5.1-5.5]) were word choice edits. Overall, experts rated completeness at 3.9 [3.2-4.2] and accuracy at 4.0 [3.7-4.1]. Highest template utilization occurred during post-implementation months 1 (90%) and 3 (100%) with orientation sessions. There was a significant increase in concordant Spanish DCI use (P < 0.01) and no inappropriate bouncebacks after implementation. CONCLUSIONS: Our study demonstrated notable variability in the content and readability of our vascular access instruction templates. New DCI had strong usage and language concordance; continued use may decrease bouncebacks.