Development and Evaluation of a New Formative Assessment of Surgical Performance.

BACKGROUND: Formative assessment of operative performance is a mandatory part of surgical training. Engagement with assessment is limited in part by the time-consuming nature and the high perceived stakes of current assessment tools. OBJECTIVES: Our aims were to develop and collect validity evidence for a new operative assessment tool that addresses barriers to assessment that current trainers and trainees experience. METHODS: We developed the Generic Operative Supervised Learning Event (GOSLE). Orthopedic trainees were invited to complete GOSLEs with their trainers after surgical procedures. Experienced consultants assessed videotaped operations performed by trainees using the GOSLE. Validity evidence for content, relationships to other scores, internal structure, response process, and consequences of testing were evaluated. RESULTS: A total of 250 GOSLEs were completed. A strong correlation was found between the GOSLE scores and the Procedure-Based Assessment ratings (r = 0.87, p < 0.001). Rasch analysis confirmed satisfactory internal structure of the rating scale, with sequential increases in rating as performance improved. The reproducibility coefficient was 0.88, with 10 assessments of the same trainee who has to achieve a reliability coefficient of 0.8. Over 90% of users found the GOSLE easy to use, with most preferring it to other assessment methods. Feedback quality was higher using the GOSLE than with current assessments. CONCLUSION: We have collected validity evidence across multiple domains in support of the GOSLE. Its psychometric performance is comparable to that of current assessments. It is preferred by trainers and trainees over existing assessments. It stimulates high-quality, actionable feedback which better supports formative assessment. By addressing issues experienced with existing assessments, we expect engagement among users to be high.

A decision tool for whole-body CT in major trauma that safely reduces unnecessary scanning and associated radiation risks: An initial exploratory analysis.

BACKGROUND: Whole-body CT (WBCT) has become routine practice in the assessment of major trauma patients. Whilst this may be associated with increased survival, several studies report high rates of negative scans. As no national guideline exists, selection criteria for WBCT vary widely. This study aims to (1) produce a scoring system that improves patient selection for WBCT (2) quantify patient radiation doses and their concomitant risk of malignancy. METHODS: Clinical notes were reviewed for all patients undergoing a WBCT for trauma over a 21-month period at a UK major trauma centre. Clinical and radiological findings were categorised according to body region. Univariate analysis was performed using Chi-squared testing, followed by multivariable logistic regression. Secondary regression analysis of patients with significant injuries that the model did not identify was performed. The model was optimised and used to develop a scoring system. Sensitivity and specificity were calculated using the same dataset as was used to derive the models. Radiation exposure was determined and the excess lifetime risk of malignancy calculated. RESULTS: 255 patients were included, with a mean age of 45 years. 16% of scans were positive for polytrauma, 42% demonstrated some injury and 42% showed no injury. The regression model identified independent predictors of polytrauma to be (1) clinical signs in more than one body region, (2) reduced Glasgow Coma Score, (3) haemodynamic abnormality, (4) respiratory abnormality, (5) mechanism of injury. The final model had a sensitivity of 95% (95% CI 86-99%) and specificity of 59% (95% CI 52-66%) for significant CT findings. Mean radiation exposure was 31.8 mSv, conferring a median excess malignancy risk of 1 in 474. CONCLUSION: After including neurological deficit, our scoring system had a sensitivity of 97% (95% CI 88-99%) and specificity of 56% (95% CI 49-64%) for significant injury. We propose this is used to stratify the use of trauma radiographs, focused CT and WBCT for major trauma patients. Although not intended to replace clinical judgement, our scoring system adds an objective component to decision-making. We believe this will safely reduce the number of unnecessary CT scans performed on a relatively young cohort of patients.

Measurement of time-dependent changes in the irregularity of neural spiking.

Irregularity of firing in spike trains has been associated with coding processes and information transfer or alternatively treated as noise. Previous studies of irregularity have mainly used the coefficient of variation (CV) of the interspike interval distribution. Proper estimation of CV requires a constant underlying firing rate, a condition that most experimental situations do not fulfill either within or across trials. Here we introduce a novel irregularity metric based on the ratio of adjacent intervals in the spike train. The new metric is not affected by firing rate and is very localized in time so that it can be used to examine the time course of irregularity relative to an alignment marker. We characterized properties of the new metric with simulated spike trains of known characteristics and then applied it to data recorded from 108 single neurons in the motor cortex of two monkeys during performance of a precision grip task. Fifty-six cells were antidromically identified as pyramidal tract neurons (PTNs). Sixty-one cells (30 PTNs) exhibited significant temporal modulation of their irregularity during task performance with the contralateral hand. The irregularity modulations generally differed in sign and latency from the modulations of firing rate. High irregularity tended to occur during the task phases requiring the most detailed control of movement, whereas neural firing became more regular during the steady hold phase. Such irregularity modulation could have important consequences for the response of downstream neurons and may provide insight into the nature of the cortical code.