Assessment of the Non-Technical Skills for Surgeons (NOTSS) framework in the USA.

BACKGROUND: Surgeons' non-technical skills are important for patient safety. The Non-Technical Skills for Surgeons assessment tool was developed in the UK and recently adapted to the US surgical context (NOTSS-US). The aim of this study was to evaluate the reliability and distribution of non-technical skill ratings given by attending (consultant) surgeons who underwent brief online training. METHODS: Attending surgeons across six specialties at a large US academic medical centre underwent a 10-min online training, then rated 60-s standardized videos of simulated operations. Intraclass correlation coefficient (ICC), and mean(s.d.) values for NOTSS-US ratings were determined for each non-technical skill category (score range 1-5, where 1 indicates poor, 3 average and 5 excellent) and for total NOTSS-US score (range 4-20; sum of 4 category scores). Outcomes were adjusted for rater characteristics including sex, specialty and clinical experience. RESULTS: A total of 8889 ratings were submitted by 81 surgeon raters on 30 simulated intraoperative videos. The mean(s.d.) total NOTSS-US score for all videos was 9·5(4·8) of 20. The within-video ICC for total NOTSS-US score was 0·64 (95 per cent c.i. 0·57 to 0·70). For individual non-technical skill categories, the ICC was highest for social skills (communication/teamwork: 0·63, 95 per cent c.i. 0·56 to 0·71; leadership: 0·64, 0·55 to 0·72) and lowest for cognitive skills (situation awareness: 0·54, 0·45 to 0·62; decision-making: 0·50, 0·41 to 0·59). Women gave higher total NOTSS-US scores than men (adjusted mean difference 0·93, 95 per cent c.i. 0·44 to 1·43; P = 0·001). CONCLUSION: After brief online training, the inter-rater reliability of the NOTSS-US assessment tool achieved moderate strength among trained surgeons rating simulated intraoperative videos.

ANTECEDENTES: Las habilidades no técnicas de los cirujanos (Non-Technical Skills for Surgeons, NOTSS) son importantes para la seguridad del paciente. La herramienta de evaluación de habilidades no técnicas para cirujanos se desarrolló en el Reino Unido y se adaptó recientemente al contexto quirúrgico de los Estados Unidos (NOTSS-US.). El objetivo de este estudio fue evaluar la fiabilidad y distribución de las calificaciones de habilidades no técnicas obtenidas por cirujanos adjuntos de cirugía (consultores) que recibieron una breve formación online. MÉTODOS: Cirujanos adjuntos de 6 especialidades en un gran centro universitario de Estados Unidos recibieron una formación online de 10 minutos de duración y seguidamente puntuaron vídeos estandarizados de operaciones simuladas de 60 minutos de duración. Se calcularon el coeficiente de correlación intraclase (intraclass correlation coefficient, ICC), la media y la desviación estándar (standard deviation, SD) para la puntuación de cada categoría de habilidad no técnica del NOTSS-US (rango 1-5, siendo 1 = pobre, 3 = promedio, 5 = excelente) y para la puntuación global de NOTSS-US (rango 4-20, suma de las puntuaciones de las cuatro categorías). Los resultados se ajustaron de acuerdo con las características del evaluador, incluyendo sexo, especialidad, experiencia clínica. RESULTADOS: En 30 videos intraoperatorios simulados, 81 cirujanos evaluadores proporcionaron 8.889 puntaciones. La puntuación media global de NOTSS-US para todos los vídeos fue de 9,5 sobre 20 (SD 4,8). El ICC de los vídeos para la puntuación global de NOTSS-US fue 0,64 (i.c. del 95% 0,57-0,70). Para las categorías individuales de habilidades no técnicas, el ICC más alto fue para las habilidades sociales (comunicación / trabajo en equipo: 0,63, (i.c. del 95% 0,56-0,71); liderazgo, 0,64 (i.c. del 95% 0,55-0,72)) y el más bajo para las habilidades cognitivas (conciencia de la situación 0,54 (i.c. del 95% 0,45-0,62); toma de decisiones 0,50 (i.c. del 95% 0,41-0,59)). Las evaluadoras femeninas presentaron puntuaciones globales de NOTSS-US más altas que los evaluadores masculinos (diferencia 0,93, i.c. del 95% 0,44-1,43; P = 0,001)). CONCLUSIÓN: Después de una breve formación online, la fiabilidad de la herramienta de evaluación NOTSS-US mostró una correlación moderada entre los cirujanos que puntuaron vídeos de simulaciones de intervenciones quirúrgicas.

Construct and criterion validity testing of the Non-Technical Skills for Surgeons (NOTSS) behaviour assessment tool using videos of simulated operations.

BACKGROUND: Surgeons' non-technical skills are an important part of surgical performance and surgical education. The most widely adopted assessment tool is the Non-Technical Skills for Surgeons (NOTSS) behaviour rating system. Psychometric analysis of this tool to date has focused on inter-rater reliability and feasibility rather than validation. METHODS: NOTSS assessments were collected from two groups of consultant/attending surgeons in the UK and USA, who rated behaviours of the lead surgeon during a video-based simulated crisis scenario after either online or classroom instruction. The process of validation consisted of assessing construct validity, scale reliability and concurrent criterion validity, and undertaking a sensitivity analysis. Central to this was confirmatory factor analysis to evaluate the structure of the NOTSS taxonomy. RESULTS: Some 255 consultant surgeons participated in the study. The four-category NOTSS model was found to have robust construct validity evidence, and a superior fit compared with alternative models. Logistic regression and sensitivity analysis revealed that, after adjusting for technical skills, for every 1-point increase in NOTSS score of the lead surgeon, the odds of having a higher versus lower patient safety score was 2·29 times. The same pattern of results was obtained for a broad mix of surgical specialties (UK) as well as a single discipline (cardiothoracic, USA). CONCLUSION: The NOTSS tool can be applied in research and education settings to measure non-technical skills in a valid and efficient manner.

Non-technical skills of surgeons and anaesthetists in simulated operating theatre crises.

BACKGROUND: Deficiencies in non-technical skills (NTS) have been increasingly implicated in avoidable operating theatre errors. Accordingly, this study sought to characterize the impact of surgeon and anaesthetist non-technical skills on time to crisis resolution in a simulated operating theatre. METHODS: Non-technical skills were assessed during 26 simulated crises (haemorrhage and airway emergency) performed by surgical teams. Teams consisted of surgeons, anaesthetists and nurses. Behaviour was assessed by four trained raters using the Non-Technical Skills for Surgeons (NOTSS) and Anaesthetists' Non-Technical Skills (ANTS) rating scales before and during the crisis phase of each scenario. The primary endpoint was time to crisis resolution; secondary endpoints included NTS scores before and during the crisis. A cross-classified linear mixed-effects model was used for the final analysis. RESULTS: Thirteen different surgical teams were assessed. Higher NTS ratings resulted in significantly faster crisis resolution. For anaesthetists, every 1-point increase in ANTS score was associated with a decrease of 53·50 (95 per cent c.i. 31·13 to 75·87) s in time to crisis resolution (P < 0·001). Similarly, for surgeons, every 1-point increase in NOTSS score was associated with a decrease of 64·81 (26·01 to 103·60) s in time to crisis resolution in the haemorrhage scenario (P = 0·001); however, this did not apply to the difficult airway scenario. Non-technical skills scores were lower during the crisis phase of the scenarios than those measured before the crisis for both surgeons and anaesthetists. CONCLUSION: A higher level of NTS of surgeons and anaesthetists led to quicker crisis resolution in a simulated operating theatre environment.

Using extended reality (XR) for medical training and real-time clinical support during deep space missions.

Medical events can affect space crew health and compromise the success of deep space missions. To successfully manage such events, crew members must be sufficiently prepared to manage certain medical conditions for which they are not technically trained. Extended Reality (XR) can provide an immersive, realistic user experience that, when integrated with augmented clinical tools (ACT), can improve training outcomes and provide real-time guidance during non-routine tasks, diagnostic, and therapeutic procedures. The goal of this study was to develop a framework to guide XR platform development using astronaut medical training and guidance as the domain for illustration. We conducted a mixed-methods study-using video conference meetings (45 subject-matter experts), Delphi panel surveys, and a web-based card sorting application-to develop a standard taxonomy of essential XR capabilities. We augmented this by identifying additional models and taxonomies from related fields. Together, this "taxonomy of taxonomies," and the essential XR capabilities identified, serve as an initial framework to structure the development of XR-based medical training and guidance for use during deep space exploration missions. We provide a schematic approach, illustrated with a use case, for how this framework and materials generated through this study might be employed.

Leaders by example: Best practices and advice on establishing a state-of-the art surgical simulation center that optimizes available resources.

BACKGROUND: The role of simulation-based education continues to expand exponentially. To excel in this environment as a surgical simulation leader requires unique knowledge, skills, and abilities that are different from those used in traditional clinically-based education. METHODS: Leaders in surgical simulation were invited to participate as discussants in a pre-conference course offered by the Association for Surgical Education. Highlights from their discussions were recorded. RESULTS: Recommendations were provided on topics such as building a simulation team, preparing for accreditation requirements, what to ask for during early stages of development, identifying tools and resources needed to meet educational goals, expanding surgical simulation programming, and building educational curricula. CONCLUSION: These recommendations provide new leaders in simulation with a unique combination of up-to-date best practices in simulation-based education, as well as valuable advice gained from lessons learned from the personal experiences of national leaders in the field of surgical simulation and education.

Modeling the cost-effectiveness of granulocyte colony-stimulating factor use in early-stage breast cancer.

PURPOSE: To model the cost-effectiveness (CE) of granulocyte colony-stimulating factor (G-CSF) in early-stage breast cancer when its use is directed to those most in need of the medication. METHODS: A conditional CE model was developed for the use of G-CSF based on a ranking of patient need as determined by patient blood counts during the first cycle of chemotherapy. In the base case, no G-CSF was used. In the alternative case, G-CSF was used in the following manner. If the risk of a neutropenic event (as defined by a predictive model based on nadir absolute neutrophil count [ANC] and hemoglobin decrease in cycle 1) was equal to or exceeded a predetermined critical value "T," then patients would receive G-CSF in cycles 2 through 6 of chemotherapy. If the risk of an event was less than T, patients would not use G-CSF unless an event occurred, at which time G-CSF would be administered with every subsequent cycle. RESULTS: A decision rule (T) that would allow the most needy 50% of early-stage breast cancer patients to receive G-CSF after the first cycle of chemotherapy resulted in a CE ratio of $34,297 dollars per life-year saved (LYS). If only the most needy 10% of patients received G-CSF, then the associated CE ratio was $23,748/LYS; if 90% of patients could receive the medication, the CE ratio would be $76,487/LYS. These estimates were relatively insensitive to inpatient hospital cost estimates (inpatient costs for fever and neutropenia of $3,090 to $7,726 per admission produced dollar per LYS figures of $34,297 to $32,415, respectively). However, the model was sensitive to assumptions about the shape of the relationship between dose reduction and disease-free survival (DFS) at 3 years. CONCLUSION: Providing G-CSF to the neediest 50% of early-stage breast cancer patients (as defined by first-cycle blood counts) starting after the first cycle of chemotherapy is associated with a CE ratio of $34,297/LYS, which is well in the range of CE ratios for treatment of other common medical conditions. Furthermore, conditional CE studies, based on predictive models that incorporate individual patient risk, allow one to define populations for which therapy is, or is not, cost-effective. Limitations of our present understanding of the shape of the chemotherapy dose-response curve, especially at low levels of dose reductions, affect these results. Further work is required to define the shape of the dose-response curve in early-stage breast cancer.