The Effect of Critical Task Auto-failure Criteria on Medical Evaluation Methods in the Pararescue Schoolhouse.

ABSTRACT

BACKGROUND: Medical training and evaluation are important for mission readiness in the pararescue career field. Because evaluation methods are not standardized, evaluation methods must align with training objectives. We propose an alternative evaluation method and discuss relevant factors when designing military medical evaluation metrics. METHODS: We compared two evaluation methods, the traditional checklist (TC) method used in the pararescue apprentice course and an alternative weighted checklist (AWC) method like that used at the U.S. Army static line jumpmaster course. The AWC allows up to two minor errors, while critical task errors result in autofailure. We recorded 168 medical scenarios during two Apprentice course classes and retroactively compared the two evaluation methods. RESULTS: Despite the possibility of auto-failure with the AWC, there was no significant difference between the two evaluation methods, and both showed similar overall pass rates (TC=50% pass, AWC=48.8% pass, p=.41). The two evaluation methods yielded the same result for 147 out of 168 scenarios (87.5%). CONCLUSIONS: The AWC method strongly emphasizes critical tasks without significantly increasing failures. It may provide additional benefits by being more closely aligned with our training objectives while providing quantifiable data for a longitudinal review of student performance.