The script concordance test for clinical reasoning: re-examining its utility and potential weakness.

CONTEXT: The script concordance test (SCT) assesses clinical reasoning under conditions of uncertainty. Relatively little information exists on Z-score (standard deviation [SD]) cut-offs for distinguishing more experienced from less experienced trainees, and whether scores depend on factual knowledge. Additionally, a recent review highlighted the finding that the SCT is potentially weakened by the fact that the mere avoidance of extreme responses may greatly increase test scores. OBJECTIVES: This study was conducted in order to elucidate the best cut-off Z-scores, to correlate SCT scores with scores on a separate medical knowledge examination (MKE), and to investigate potential solutions to the weakness of the SCT. METHODS: An analysis of scores on pulmonary and critical care medicine tests undertaken during July and August 2013 was performed. Clinical reasoning was tested using 1-hour SCTs (Question Sets 1 or 2). Medical knowledge was tested using a 3-hour, computer-adapted, multiple-choice question examination. RESULTS: The expert panel was composed of 16 attending physicians. The SCTs were completed by 16 fellows and 10 residents. Fourteen fellows completed the MKE. Test reliability was acceptable for both Question Sets 1 and 2 (Cronbach's alphas of 0.79 and 0.89, respectively). Z-scores of - 2.91 and - 1.76 best separated the scores of residents from those of fellows, and the scores of fellows from those of attending physicians, respectively. Scores on the SCT and MKE were poorly correlated. Simply avoiding extreme answers boosted the Z-scores of the lowest 10 scorers on both Question Sets 1 and 2 by ≥ 1 SD. Increasing the proportion of questions with extreme modal answers to 50%, and using hypothetical question sets created from Question Set 1 overcame this problem, but consensus scoring did not. CONCLUSIONS: The SCT was able to differentiate between test subjects of varying levels of competence, and results were not associated with medical knowledge. However, the test was vulnerable to responses that intentionally avoided extreme values. Increasing the proportion of questions with extreme modal answers may attenuate the effect of candidates exploiting the test weakness related to extreme responses.

Identification of concurrent bacterial infection in adult patients with dengue.

We aim to construct a diagnostic model for bacterial coinfection in dengue patients (Dengue Dual Infection Score [DDIS]); 2,065 adult dengue patients (mean age = 41.9 ± 17.2 years, 58.4% male, 83 patients with bacterial coinfection) seen at a university hospital from January of 2005 to February of 2010 were studied. The DDIS was created by assigning one point to each of five risk factors for bacterial coinfection: pulse rate ≥ 90 beats/minute, total white cell count ≥ 6 × 10(9)/L, hematocrit < 40%, serum sodium < 135 mmol/L, and serum urea ≥ 5 mmol/L. The DDIS identified bacterial coinfection (derivation set area under the curve = 0.793, 95% confidence interval = 0.732-0.854; validation set area under the curve = 0.761, 95% confidence interval = 0.637-0.886). A DDIS of ≥ 4 had a specificity of 94.4%, whereas a DDIS of ≥ 1 had a sensitivity of 94.4% for bacterial coinfection. The DDIS can help to select dengue patients for early bacterial cultures and empirical antibiotics.