Interprofessional collaborative reasoning by residents and nurses in internal medicine: Evidence from a simulation study.

Clinical reasoning has been studied in residents or nurses, using interviews or patient-provider encounters. Despite a growing interest in interprofessional collaboration, the notion of collaborative reasoning has not been well studied in clinical settings. Our study aims at exploring resident-nurse collaborative reasoning in a simulation setting. We enrolled 14 resident-nurse teams from a general internal medicine division in a mixed methods study. Teams each managed one of four acute case scenarios, followed by a stimulated-recall session. A qualitative, inductive analysis of the transcripts identified five dimensions of collaborative reasoning: diagnostic reasoning, patient management, patient monitoring, communication with the patient, and team communication. Three investigators (two senior physicians, one nurse) assessed individual and team performances using a five-point Likert scale, and further extracted elements supporting the collaborative reasoning process. Global assessment of the resident-nurse team was not simply an average of individual performances. Qualitative results underlined the need to improve situational awareness, particularly for task overload. Team communication helped team members stay abreast of each other's thoughts and improve their efficiency. Residents and nurses differed in their reasoning processes, and awareness of this difference may contribute to improving interprofessional collaboration. Understanding collaborative reasoning can provide an additional dimension to interprofessional education.

Developing and evaluating the student assessment system in the preclinical problem-based curriculum at Sherbrooke.

BACKGROUND: Students' learning was used as an outcome measure in the first phases of the major curriculum reform started in 1987 by the Université de Sherbrooke Faculty of Medicine, which shifted from a traditional to a student-centered, problem-based learning (PBL) and community-oriented program. The system for evaluating preclinical students' learning is intended to reinforce the integration of basic and clinical sciences. METHOD: To discover whether the evaluation system was fulfilling its intended goals, the authors used data from the classes of 1991-1993 to assess the reliability and validity of three evaluation instruments. The three instruments were (1) written examinations composed of multiple-choice questions (MCQs), short-answer questions (SAQs), and problem-analysis questions (PAQs); (2) PBL tutor rating forms that evaluate students' reasoning skills, communication and group-interaction skills, and autonomy and humanism; and (3) clinical skills evaluations, including objective structured clinical examinations (OSCEs). The weights allocated to the instruments reflected how the faculty valued each evaluation dimension in each of the three phases of the preclinical curriculum. RESULTS: Reliability indexes improved throughout the system implementation. The written examinations proved to have content validity according to the PBL learning objectives. As evaluated by students, the PAQs were found to be at a taxonomic level that assessed ability to analyze information a third of the time in the first year of implementation of the PBL curriculum and 17% in the second year. Variations and correlations of students' mean performances across instructional units and between the evaluation instruments led to the development of a student longitudinal performance profile to be used before yearly promotion decisions are proposed. The profile was introduced in the fifth year of PBL implementation. CONCLUSION: The system allows students to learn higher-taxonomic-level content and fulfills the institution's social responsibility of judging program outcomes and promoting qualified students, although evaluation by PBL tutors is still psychometrically questionable and the measurement of students' reasoning and ability to analyze problems is still an unfinished evaluation task.

Effects of using two or more standardized patients to simulate the same case on case means and case failure rates.

The effects of using two or more standardized patients (multiple SPs) to simulate the same case in a performance-based examination were studied at the case level by comparing case means and case failure rates for multiple SPs simulating the same case, using data from the classes of 1988, 1989, and 1990 at the Southern Illinois University School of Medicine. For total scores and scores on the students' written answers, the effects on means and failure rates were negligible and could be explained as due to sampling error. For scores on the checklists completed by the SPs, there were more significant differences than would be expected by chance alone, even though the number of significant differences was relatively small. The results demonstrate a need for caution in the interpretation of scores obtained from a case checklist completed by multiple SPs, particularly in regard to making pass-fail decisions.

Screening test length for sequential testing with a standardized-patient examination: a receiver operating characteristic (ROC) analysis.

The feasibility of using sequential testing (i.e., using a screening test) to reduce the length and expense of a performance-based examination with standardized-patient cases was demonstrated previously. In the present study, quantitative criteria rather than practical considerations were used to determine optimal values for the length of the screening test (i.e., number of cases) and the location of the screening pass-fail cutoff (i.e., its relation to the mean of the pass levels for the different cases). Data were derived from five classes of senior students at the Southern Illinois University School of Medicine, 1987-1991. Specifically, receiver operating characteristic (ROC) curves were plotted for screening tests of varying lengths, with the points on each ROC curve corresponding to different pass-fail cutoffs on the screening test. The results showed that good accuracy can be attained with a screening test that is only one-third the length of the full examination and that the cutoff for this screen should be set slightly above the mean of the case pass levels to maximize sensitivity and specificity. The authors conclude that their study demonstrates the value of an ROC analysis in evaluating the psychometric properties of a screening test in sequential testing.

Effects of examinee gender, standardized-patient gender, and their interaction on standardized patients' ratings of examinees' interpersonal and communication skills.

PURPOSE: To assess the effects of examinee gender, standardized-patient (SP) gender, and, in particular, their interaction on ratings made by SPs of examinees' interpersonal and communication skills in a performance-based examination of clinical competence. METHOD: The examination was administered to four classes of senior medical students (about 70 per class) at Southern Illinois University School of Medicine, 1988-1991. The skill dimensions tested were clarity of communication, thoroughness of explanation, professional manner, personal manner, and overall patient satisfaction. Split-plot analyses of variance were used. RESULTS: There was no interaction of examinee gender and SP gender for any of the five rating scales. There was no main effect of examinee gender for four of the five scales; however, for personal manner, women students were rated slightly higher than men students. There was a main effect of SP gender, but the effect was not consistent from rating scale to rating scale or from class to class. Nevertheless, differences in ratings given by men and women SPs should not be of psychometric concern, since the ratings of men and women examinees are necessarily affected alike. CONCLUSIONS: Except for the women examinee's higher performance in personal manner, the men and women examinees generally performed equally well with respect to interpersonal and communication skills, and they performed equally well regardless of the gender of the SP.

Clinical performance-based test sensitivity and specificity in predicting first-year residency performance.

To assess the use of a standardized-patient-based postclerkship examination (PCX) to predict students' performances in their first year of residency, the authors used data from 202 students of the classes of 1987, 1988, and 1990 at the Southern Illinois University School of Medicine. The PCX was found to be more sensitive than specific in that it identified more correctly those students who received high ratings in their first year of residency than those who received low ratings. Consequently, while the rate of false negatives was low across the three classes, the rate of false positives was relatively high. Analyses of the supervisors' written comments on residents' performances indicated that more than half of the false positives and false negatives resulted mostly from problems found with the supervisors' ratings, rather than from the inaccuracy of the PCX in predicting students' residency performances. The supervisors' ratings did not always match their written comments and did not always represent pure assessments of the residents' cognitive performances. Finally, because of the large number of missing residency performance ratings among the graduates who had performed unsatisfactorily on the PCX, it is expected that the sensitivity of the PCX may be overestimated and its specificity underestimated. Overall, the results suggest that the standardized-patient-based PCX is a useful indicator of students' readiness for and performance in residency.

Test security in examinations that use standardized-patient cases at one medical school.

The use of performance-based examinations consisting of standardized-patient (SP) cases has increased greatly in recent years. These examinations are typically long and thus require the presentation of the same SP cases to several consecutive examinee groups. Consequently, concerns have arisen about the potential for violations of test security whereby students who were tested early in the examination period pass on information to students tested later. These concerns are addressed using data from the SP-based examinations administered to five classes (1986-1990) of senior medical students at Southern Illinois University School of Medicine. Because of the length of the examinations, each class was randomly divided into five groups and the examination was administered to one group at a time, requiring three days of testing time per group and three weeks of testing time per class. The results showed no consistent, systematic changes in case means across the five groups tested at different times throughout the examination period, and thus provide no evidence of serious, widespread violations of test security.

Comparing checklists and databases with physicians' ratings as measures of students' history and physical-examination skills.

PURPOSE: To compare two methods of rating students' performances on history and physical examination: (1) by using checklists completed by standardized patients (SPs) and databases completed by students, and (2) by using ratings of students by three physicians for each SP-student encounter. METHOD: Four cases were chosen for the study, and 30 students were examined per case. The students were all in their fourth year at the Southern Illinois University School of Medicine in the spring of 1991. Two of the cases had both checklists and databases, and the remaining two had databases only. Each SP-student encounter was videotaped and was viewed independently by three physicians unfamiliar with the contents of the checklists and databases. The physicians' pooled ratings were then compared with the checklist and database scores. Uncorrected and corrected correlations were obtained, with the generalizability coefficient used as the index of reliability. RESULTS: Interrater generalizability of physicians' ratings was very good, ranging from .65 to .93 for overall ratings. Generalizability of physicians' ratings pooled across the four cases was .85. Checklist scores tended to correlate higher with physicians' ratings than did database scores: across the cases, correlation coefficients between physicians' ratings and checklist scores and database scores were .65 and .39, respectively. CONCLUSION: The checklist scores correlated strongly with the physicians' ratings of history and physical-examination skills, providing some evidence of validity for their use. The checklist scores correlated much better with the physicians' ratings than did the database scores. Possible explanations for this finding are discussed.

Six years of comprehensive, clinical, performance-based assessment using standardized patients at the Southern Illinois University School of Medicine.

By the end of 1990-91, the Southern Illinois University School of Medicine had had six years of experience with comprehensive, performance-based examinations of senior medical students' levels of clinical competence; this report assesses the psychometric aspects of the six examinations given during that period. The examinations were aimed at determining the students' readiness for postgraduate training. Compared with other clinical performance-based assessments that use standardized patients (SPs), these examinations had two important and unique features: (1) the examinations assessed a comprehensive range of clinical skills and reasoning; and (2) they approximated the challenges of real clinical practice wherein a practitioner's skills need to be orchestrated and prioritized in order to meet the challenges of the case encountered. Each year, the performance-based assessment given was an intensive clinical examination requiring each student to work up 13 to 18 SP problems over a three-day period. To administer an examination to an entire class of students took three weeks. Because all students after the first year of administration (1986) were required to pass these examinations, the fairness of test design and scoring and the setting of performance standards for the examinations became important issues for the faculty. The results, accumulated over six years and based on a total of 6,804 student-patient encounters involving 405 students, indicate that this kind of clinical performance-based examination can discriminate a wide range of students' clinical performances. The results provide evidence for the examinations' test security, content validity, construct validity, and reliability.

Diagnostic single nucleotide polymorphisms for identifying westslope cutthroat trout (Oncorhynchus clarki lewisi), Yellowstone cutthroat trout (Oncorhynchus clarkii bouvieri) and rainbow trout (Oncorhynchus mykiss).

We describe 12 diagnostic single nucleotide polymorphism (SNP) assays for use in species identification among rainbow and cutthroat trout: five of these loci have alleles unique to rainbow trout (Oncorhynchus mykiss), three unique to westslope cutthroat trout (O. clarkii lewisi) and four unique to Yellowstone cutthroat trout (O. clarkii bouvieri). These diagnostic assays were identified using a total of 489 individuals from 26 populations and five fish hatchery strains.