Surgical Sabermetrics: A Scoping Review of Technology-enhanced Assessment of Nontechnical Skills in the Operating Room.

OBJECTIVE: To evaluate the current evidence for surgical sabermetrics: digital methods of assessing surgical nontechnical skills and investigate the implications for enhancing surgical performance. BACKGROUND: Surgeons need high-quality, objective, and timely feedback to optimize performance and patient safety. Digital tools to assess nontechnical skills have the potential to reduce human bias and aid scalability. However, we do not fully understand which of the myriad of digital metrics of performance assessment have efficacy for surgeons. METHODS: A systematic review was conducted by searching PubMed, EMBASE, CINAHL, and PSYCINFO databases following PRISMA-ScR guidelines. MeSH terms and keywords included "Assessment," "Surgeons," and "Technology". Eligible studies included a digital assessment of nontechnical skills for surgeons, residents, and/or medical students within an operative context. RESULTS: From 19,229 articles screened, 81 articles met the inclusion criteria. The studies varied in surgical specialties, settings, and outcome measurements. A total of 122 distinct objective, digital metrics were utilized. Studies digitally measured at least 1 category of surgical nontechnical skill using a single (n=54) or multiple objective measures (n=27). The majority of studies utilized simulation (n=48) over live operative settings (n=32). Surgical Sabermetrics has been demonstrated to be beneficial in measuring cognitive load (n=57), situation awareness (n=24), communication (n=3), teamwork (n=13), and leadership (n=2). No studies measured intraoperative decision-making. CONCLUSIONS: The literature detailing the intersection between surgical data science and operative nontechnical skills is diverse and growing rapidly. Surgical Sabermetrics may provide a promising modifiable technique to achieve desirable outcomes for both the surgeon and the patient. This study identifies a diverse array of measurements possible with sensor devices and highlights research gaps, including the need for objective assessment of decision-making. Future studies may advance the integration of physiological sensors to provide a holistic assessment of surgical performance.

Application of generative language models to orthopaedic practice.

OBJECTIVE: To explore whether large language models (LLMs) Generated Pre-trained Transformer (GPT)-3 and ChatGPT can write clinical letters and predict management plans for common orthopaedic scenarios. DESIGN: Fifteen scenarios were generated and ChatGPT and GPT-3 prompted to write clinical letters and separately generate management plans for identical scenarios with plans removed. MAIN OUTCOME MEASURES: Letters were assessed for readability using the Readable Tool. Accuracy of letters and management plans were assessed by three independent orthopaedic surgery clinicians. RESULTS: Both models generated complete letters for all scenarios after single prompting. Readability was compared using Flesch-Kincade Grade Level (ChatGPT: 8.77 (SD 0.918); GPT-3: 8.47 (SD 0.982)), Flesch Readability Ease (ChatGPT: 58.2 (SD 4.00); GPT-3: 59.3 (SD 6.98)), Simple Measure of Gobbledygook (SMOG) Index (ChatGPT: 11.6 (SD 0.755); GPT-3: 11.4 (SD 1.01)), and reach (ChatGPT: 81.2%; GPT-3: 80.3%). ChatGPT produced more accurate letters (8.7/10 (SD 0.60) vs 7.3/10 (SD 1.41), p=0.024) and management plans (7.9/10 (SD 0.63) vs 6.8/10 (SD 1.06), p<0.001) than GPT-3. However, both LLMs sometimes omitted key information or added additional guidance which was at worst inaccurate. CONCLUSIONS: This study shows that LLMs are effective for generation of clinical letters. With little prompting, they are readable and mostly accurate. However, they are not consistent, and include inappropriate omissions or insertions. Furthermore, management plans produced by LLMs are generic but often accurate. In the future, a healthcare specific language model trained on accurate and secure data could provide an excellent tool for increasing the efficiency of clinicians through summarisation of large volumes of data into a single clinical letter.

Cognitive Load Management: An Invaluable Tool for Safe and Effective Surgical Training.

This article highlights the importance of considering Cognitive Load (CL) and Cognitive Load Theory (CLT) during surgical training, focusing on the acquisition of intra-operative skills. It describes the basis of CLT with the overarching aim of describing CLT-based techniques to enhance current training strategies and surgical performance, many of which are instinctively already employed in surgical practice. Currently, methods of feedback and assessment are imperfect - typically subjective, unsystematic, opportunistic, or retrospective, and at risk of human bias. Surgical Sabermetrics, the advanced analytics of surgical and audio-visual data, aims to enhance this feedback by providing objective, real-time, digital-based feedback. This article introduces the benefit of real-time measurement of CL to enhance feedback and its applications to surgical performance that follow the ethos of Surgical Sabermetrics.1 The 2022 theme for ICOSET was "Making it Better." Cognitive Load and Surgical Sabermetrics principles provide tools to make Surgical training better, with the goal of higher quality care for patients.

Coaching with audiovisual technology in acute-care hospital settings: systematic review.

BACKGROUND: Surgical coaching programmes are a means of improving surgeon performance. Embedded audiovisual technology has the potential to further enhance participant benefit and scalability of coaching. The objective of this systematic review was to evaluate how audiovisual technology has augmented coaching in the acute-care hospital setting and to characterize its impact on outcomes. METHODS: A systematic review was conducted, searching PubMed, Ovid MEDLINE, Embase, PsycInfo, and CINAHL databases using PRISMA. Eligible studies described a coaching programme that utilized audiovisual technology, involved at least one coach-coachee interaction, and included healthcare professionals from the acute-care hospital environment. The risk of bias 2 tool and grading of recommendations, assessment, development, and evaluations (GRADE) framework were used to evaluate studies. Synthesis without meta-analysis was performed, creating harvest plots of three coaching outcomes: technical skills, self-assessment/feedback, and non-technical skills. RESULTS: Of 10 458 abstracts screened, 135 full texts were reviewed, and 21 studies identified for inclusion. Seventeen studies were conducted within surgical specialties and six classes of audiovisual technology were utilized. An overall positive direction of effect was demonstrated for studies measuring improvement of either technical skills or non-technical skills. Direction of effect for self-assessment/feedback was weakly positive. CONCLUSION: Audiovisual technology has been used successfully in coaching programmes within acute-care hospital settings to facilitate or assess coaching, with a positive impact on outcome measures. Future studies may address the additive benefits of video over in-person observation and enhance the certainty of evidence that coaching impacts on surgeon performance, surgeon well-being, and patient outcomes.

Parenting a Child with Phenylketonuria: An Investigation into the Factors That Contribute to Parental Distress.

Phenylketonuria (PKU) is an inherited metabolic condition that can lead to the onset of intellectual disabilities if not strictly managed through a low-protein diet. Parents are responsible for supervising their child's treatment for PKU, which may impact on their experience of distress. This cross-sectional study aimed to identify the factors that contribute to distress in parents who care for a child with PKU, distinct from parents in the general population. Thirty-eight parents of children and adolescents with PKU and 32 parents in the general population completed the questionnaires measuring parental psychological resilience, child behaviour problems, perceived social support and distress. Parents of children with PKU also completed measures of their child's care dependency and behaviour related to developmental and intellectual disabilities. The findings revealed no statistically significant differences in distress between the groups, but parents of children with PKU reported more child behaviour problems. Multiple regression analysis identified that parental psychological resilience and child anxious behaviour explained 35% of the variance in distress for parents of children with PKU. By comparison, parental psychological resilience and generic child behaviour only accounted for 19% of the variance in distress for parents in the general population. This has implications for developing interventions in clinical settings that aim to reduce parents' distress by enhancing their psychological resilience and supporting them to manage child behaviour difficulties, particularly anxious behaviour. Future research should include larger, more diverse samples and use longitudinal study designs.