A comparative assessment and gap analysis of commonly used team rating scales.

BACKGROUND: The purpose of this article was to conduct a gap analysis of important team constructs that may be absent in widely used team assessments. METHODS AND MATERIALS: Two assessment tools with known validity evidence (1) Non-Technical Skills for Surgeons (NOTSS) and (2) the Cannon-Bowers Scale were used to evaluate 11 teams of surgical residents (n = 33) performing simulated laparoscopic hernia repairs. Faculty raters' scores were used to compare the surveys and assess validity and reliability. Raters' detailed observation notes were used to indicate important behavioral constructs that were missing from the team rating scales. RESULTS: When assessing inter-item correlations (reliability) four of five NOTSS' scale items had significant correlations (r = 0.9-1.0, P < 0.05) with the Cannon-Bowers items. While the correlations were only noted for three of six Cannon-Bowers items, in each instance the same four of five NOTSS items correlated with the three Cannon-Bowers items, thus providing further validity evidence for both scales. When evaluating the gap, key emerging themes included the need to focus on critical team errors, individual team member contributions, task performance, and overall team performance. These gaps, plus items from the NOTSS and Cannon-Bowers scales, were incorporated into a new rating scale. CONCLUSIONS: Despite continued evidence of validity and reliability, there were several behavioral constructs that were not represented when using the NOTSS and Cannon-Bowers scales. Critical team errors, individual team member contributions, task performance, and overall team performance appear important in our ability to understand teams and teamwork.

How do physicians decide to treat: an empirical evaluation of the threshold model.

BACKGROUND: According to the threshold model, when faced with a decision under diagnostic uncertainty, physicians should administer treatment if the probability of disease is above a specified threshold and withhold treatment otherwise. The objectives of the present study are to a) evaluate if physicians act according to a threshold model, b) examine which of the existing threshold models [expected utility theory model (EUT), regret-based threshold model, or dual-processing theory] explains the physicians' decision-making best. METHODS: A survey employing realistic clinical treatment vignettes for patients with pulmonary embolism and acute myeloid leukemia was administered to forty-one practicing physicians across different medical specialties. Participants were randomly assigned to the order of presentation of the case vignettes and re-randomized to the order of "high" versus "low" threshold case. The main outcome measure was the proportion of physicians who would or would not prescribe treatment in relation to perceived changes in threshold probability. RESULTS: Fewer physicians choose to treat as the benefit/harms ratio decreased (i.e. the threshold increased) and more physicians administered treatment as the benefit/harms ratio increased (and the threshold decreased). When compared to the actual treatment recommendations, we found that the regret model was marginally superior to the EUT model [Odds ratio (OR) = 1.49; 95% confidence interval (CI) 1.00 to 2.23; p = 0.056]. The dual-processing model was statistically significantly superior to both EUT model [OR = 1.75, 95% CI 1.67 to 4.08; p < 0.001] and regret model [OR = 2.61, 95% CI 1.11 to 2.77; p = 0.018]. CONCLUSIONS: We provide the first empirical evidence that physicians' decision-making can be explained by the threshold model. Of the threshold models tested, the dual-processing theory of decision-making provides the best explanation for the observed empirical results.