Telemedicine in Anesthesiology: Using Simulation to Teach Remote Preoperative Assessment.

Background: The move toward telemedicine has markedly accelerated with the COVID-19 pandemic. Anesthesia residents must learn to provide preoperative assessments on a virtual platform. We created a pilot telemedicine curriculum for postgraduate year-2 (PGY2) anesthesiology. Methods: The curriculum included a virtual didactic session and a simulated virtual preoperative assessment with a standardized patient (SP). A faculty member and the SP provided feedback using a checklist based on the American Medical Association Telehealth Visit Etiquette Checklist and the American Board of Anesthesiology Applied Examination Objective Structured Clinical Examination content outline. Residents completed surveys assessing their perceptions of the effectiveness and helpfulness of the didactic session and simulated encounter, as well as the cognitive workload of the encounter. Results: A total of 12 PGY2 anesthesiology residents in their first month of clinical anesthesia residency training participated in this study. Whereas most (11/12) residents felt confident, very confident, or extremely confident in being able to conduct a telemedicine preoperative assessment after the didactic session, only 42% ensured adequate lighting and only 33% ensured patient privacy before conducting the visit. Postencounter survey comments indicated that the SP encounter was of greater value (more effective and helpful) than the didactic session. Residents perceived the encounter as demanding, but they felt successful in accomplishing it and did not feel rushed. Faculty and SP indicated that the checklist guided them in providing clear and useful formative feedback. Conclusions: A virtual SP encounter can augment didactics to help residents learn and practice essential telemedicine skills for virtual preoperative assessments.

Optimizing Clinical Reasoning Assessments With Analytic and Holistic Ratings: Examining the Validity, Reliability, and Cost of a Simplified Patient Note Scoring Procedure.

PURPOSE: Post-standardized patient (SP) encounter patient notes used to assess students' clinical reasoning represent a significant time burden for faculty who traditionally score them. To reduce this burden, the authors previously reported a complex faculty-developed scoring method to assess patient notes rated by nonclinicians. The current study explored whether a simplified scoring procedure for nonclinician raters could further optimize patient note assessments by reducing time, cost, and creating additional opportunities for formative feedback. METHOD: Ten nonclinician raters scored patient notes of 141 students across 5 SP cases by identifying case-specific patient note checklist items. The authors identified the bottom quintile of students using the proportion of correct items identified in the note (percent-scores) and case-specific faculty-generated scoring formulas (formula-scores). Five faculty raters scored a subset of notes from low, borderline, and high-performing students (n = 30 students) using a global rating scale. The authors performed analyses to gather validity evidence for percent-scores (i.e., relationship to other variables), investigate its reliability (i.e., generalizability study), and evaluate its costs (i.e., faculty time). RESULTS: Nonclinician percent- and formula-scores were highly correlated ( r = .88) and identified similar lists of low-performing students. Both methods demonstrated good agreement for pass-fail determinations with each other (Kappa = .68) and with faculty global ratings (Kappa percent =.61; Kappa formula =.66). The G-coefficient of percent-scores was .52, with 38% of variability attributed to checklist items nested in cases. Using percent-scores saved an estimated $746 per SP case (including 6 hours of faculty time) in development costs over formula-scores. CONCLUSIONS: Nonclinician percent-scores reliably identified low-performing students without the need for complex faculty-developed scoring formulas. Combining nonclinician analytic and faculty holistic ratings can reduce the time and cost of patient note scoring and afford faculty more time to coach at-risk students and provide targeted assessment input for high-stakes summative exams.

Response Process Validity Evidence for Video Commentary Assessment in Surgery: A Qualitative Study.

OBJECTIVES: Well-developed mental representations of a task are fundamental to proficient performance. 'Video Commentary' (VC) is a novel assessment intended to measure mental representations of surgical tasks that would reflect an important aspect of task proficiency. Whether examinees' actual response processes align with this intent remains unknown. As part of ongoing validation of the assessment, we sought to understand examinees' response processes in VC. DESIGN: Grounded theory qualitative study. In 2019, residents were interviewed about their understanding of and approach to VC. Using grounded theory, we created a theoretical model explaining relationships among factors that influence residents' response processes and performance. Residents' perceived purpose of VC was also explored using Likert-type questions. SETTING: Academic surgical residency program. PARTICIPANTS: Forty-eight surgical residents (PGY-1 to PGY-5). RESULTS: Analysis of narrative comments indicated that residents' perceived purposes of VC generally align with the educator's intent. Resident response processes are influenced by test characteristics, residents' perception and understanding of VC, and residents' personal characteristics. Four strategies seem to guide how residents respond, namely a focus on speed, points, logic, and relevance. Quantitative results indicated residents believe VC scores reflect their ability to speak quickly, ability to think quickly, and knowledge of anatomy (mean = 5.0, 4.5, and 4.4 respectively [1 = strongly disagree, 6 = strongly agree]). PGY-1 and PGY-2 residents tend to focus on naming facts whereas PGY-4 and PGY-5 residents focus on providing comprehensive descriptions. CONCLUSIONS: Residents generally have an accurate understanding of the purpose of VC. However, their use of different approaches could represent a threat to validity. The response strategies of speed, points, logic, and relevance may inform other clinical skills assessments.

Mastery skill assessment in hepato-pancreato-biliary surgical ultrasound: It's a Matter of Entrustment.

BACKGROUND: A formative hepato-pancreato-biliary (HPB) ultrasound (US) skills practicum is offered annually to graduating HPB fellows, using entrustment assessments for open (IOUS) and laparoscopic (LAPUS) US. It is hypothesized that validity evidence will support the use of these assessments to determine if graduating fellows are well prepared to perform HPB US independently. METHODS: Expert faculty were surveyed to set Mastery Entrustment standards for fellow performance. Standards were applied to fellow performances during two annual US skills practicums. RESULTS: 11 faculty questionnaires were included. Mean Entrustment cut scores across all items were 4.9/5.0 and 4.8/5.0 and Global Entrustment cut scores were 5.0/5.0 and 4.8/5.0 for IOUS and LAPUS, respectively. 78.5% (29/37) fellows agreed to have their de-identified data evaluated. Mean fellow Entrustments (across all skills) were 4.1 (SD 0.6; 2.6-4.9) and 3.9 (SD 0.7; 2.7-5), while the Global Entrustments were 3.6 (SD 0.8; 2-5) and 3.5 (SD 1.0; 2-5) for IOUS and LAPUS, respectively. CONCLUSIONS: Two cohorts of graduating HPB fellows are not meeting Mastery Standards for HPB US performance determined by a panel of expert faculty.

A Meaningful and Actionable Professionalism Assessment: Validity Evidence for the Professionalism Mini-Evaluation Exercise (P-MEX) Across 8 Years.

PURPOSE: With the growing importance of professionalism in medical education, it is imperative to develop professionalism assessments that demonstrate robust validity evidence. The Professionalism Mini-Evaluation Exercise (P-MEX) is an assessment that has demonstrated validity evidence in the authentic clinical setting. Identifying the factorial structure of professionalism assessments determines professionalism constructs that can be used to provide diagnostic and actionable feedback. This study examines validity evidence for the P-MEX, a focused and standardized assessment of professionalism, in a simulated patient setting. METHOD: The P-MEX was administered to 275 pediatric residency applicants as part of a 3-station standardized patient encounter, pooling data over an 8-year period (2012 to 2019 residency admission years). Reliability and construct validity for the P-MEX were evaluated using Cronbach's alpha, exploratory factor analysis (EFA), and confirmatory factor analysis (CFA). RESULTS: Cronbach's alpha for the P-MEX was 0.91. The EFA yielded 4 factors: doctor-patient relationship skills, interprofessional skills, professional demeanor, and reflective skills. The CFA demonstrated good model fit with a root-mean-square error of approximation of .058 and a comparative fit index of .92, confirming the reproducibility of the 4-factor structure of professionalism. CONCLUSIONS: The P-MEX demonstrates construct validity as an assessment of professionalism, with 4 underlying subdomains in doctor-patient relationship skills, interprofessional skills, professional demeanor, and reflective skills. These results yield new confidence in providing diagnostic and actionable subscores within the P-MEX assessment. Educators may wish to integrate the P-MEX assessment into their professionalism curricula.

Teaching and assessing intra-operative consultations in competency-based medical education: development of a workplace-based assessment instrument.

Competency-based medical education (CBME) is being implemented worldwide. In CMBE, residency training is designed around competencies required for unsupervised practice and use entrustable professional activities (EPAs) as workplace "units of assessment". Well-designed workplace-based assessment (WBA) tools are required to document competence of trainees in authentic clinical environments. In this study, we developed a WBA instrument to assess residents' performance of intra-operative pathology consultations and conducted a validity investigation. The entrustment-aligned pathology assessment instrument for intra-operative consultations (EPA-IC) was developed through a national iterative consultation and used clinical supervisors to assess residents' performance at an anatomical pathology program. Psychometric analyses and focus groups were conducted to explore the sources of evidence using modern validity theory: content, response process, internal structure, relations to other variables, and consequences of assessment. The content was considered appropriate, the assessment was feasible and acceptable by residents and supervisors, and it had a positive educational impact by improving performance of intra-operative consultations and feedback to learners. The results had low reliability, which seemed to be related to assessment biases, and supervisors were reluctant to fully entrust trainees due to cultural issues. With CBME implementation, new workplace-based assessment tools are needed in pathology. In this study, we showcased the development of the first instrument for assessing resident's performance of a prototypical entrustable professional activity in pathology using modern education principles and validity theory.

Validity Evidence for a Residency Admissions Standardized Assessment Letter for Pediatrics.

Construct: This study aims to provide validity evidence for the standardized Assessment Letter for Pediatrics as a measure of competencies expected of a 1st-year pediatrics resident as part of a pediatric residency admissions process. BACKGROUND: The Narrative Letter of Recommendation is a frequently used tool in the residency admissions process even though it has poor interrater reliability, lacks pertinent content, and does not correlate with residency performance. A newer tool, the Standardized Letter, has shown validity evidence for content and interrater reliability in other specialties. We sought to develop and provide validity evidence for the standardized Assessment Letter for Pediatrics. APPROACH: All 2012 and 2013 applicants invited to interview at the University of Geneva Pediatrics Residency Program provided 2 standardized Assessment Letters. Content for the letter was based on CanMEDS roles and ratings of 6 desired competencies and an overall assessment. Validity evidence was gathered for internal structure (Cronbach's alpha and generalizability), response process (interrater reliability with intraclass correlation), relations to other variables (Pearson's correlation coefficient), and consequences (logistic regression to predict admission). RESULTS: One hundred fourteen faculty completed 142 standardized Assessment Letters for 71 applicants. Average overall assessment was 3.0 of 4 (SD = 0.59). Cronbach's alpha was 0.93. The G-coefficient was 0.59. The decision study projected that four Assessment Letters are needed to attain a G-coefficient of 0.73. Applicant variance (28.5%) indicated high applicant differentiation. The Assessment Letter intraclass coefficient was 0.51, 95% confidence interval (CI) [0.43, 0.59]. Assessment Letter scores were correlated with the structured interview (r = .28), 95% CI [0.05, 0.51]; global rating (r = .36), 95% CI [0.13, 0.58]; and admissions decision (r = .25), 95% CI [0.02, 0.46]. Assessment Letter scores did not predict the admissions decision (odds ratio = 1.67, p = .37) after controlling for the unique contribution of the structured interview and global rating scores. CONCLUSION: Validity evidence supports use of the Assessment Letter for Pediatrics; future studies should refine items to improve predictive validity and explore how to best integrate the Assessment Letter into the residency admissions process.

Improving the residency admissions process by integrating a professionalism assessment: a validity and feasibility study.

The purpose of this study was to provide validity and feasibility evidence in measuring professionalism using the Professionalism Mini-Evaluation Exercise (P-MEX) scores as part of a residency admissions process. In 2012 and 2013, three standardized-patient-based P-MEX encounters were administered to applicants invited for an interview at the University of Geneva Pediatrics Residency Program. Validity evidence was gathered for P-MEX content (item analysis); response process (qualitative feedback); internal structure (inter-rater reliability with intraclass correlation and Generalizability); relations to other variables (correlations); and consequences (logistic regression to predict admission). To improve reliability, Kane's formula was used to create an applicant composite score using P-MEX, structured letter of recommendation (SLR), and structured interview (SI) scores. Applicant rank lists using composite scores versus faculty global ratings were compared using the Wilcoxon signed-rank test. Seventy applicants were assessed. Moderate associations were found between pairwise correlations of P-MEX scores and SLR (r = 0.25, P = .036), SI (r = 0.34, P = .004), and global ratings (r = 0.48, P < .001). Generalizability of the P-MEX using three cases was moderate (G-coefficient = 0.45). P-MEX scores had the greatest correlation with acceptance (r = 0.56, P < .001), were the strongest predictor of acceptance (OR 4.37, P < .001), and increased pseudo R-squared by 0.20 points. Including P-MEX scores increased composite score reliability from 0.51 to 0.74. Rank lists of applicants using composite score versus global rating differed significantly (z = 5.41, P < .001). Validity evidence supports the use of P-MEX scores to improve the reliability of the residency admissions process by improving applicant composite score reliability.

Characteristics and Core Curricular Elements of Medical Simulation Fellowships in North America.

Background In the past few years, there has been rapid growth in the number of simulation fellowships for physicians in the United States and Canada, with the objective of producing faculty with expertise and leadership training in medical simulation. Relatively little is known about the collective content and structure of these new fellowship opportunities. Objective We sought to identify a common set of core curricular elements among existing simulation fellowships and to obtain demographic background information on participants and leadership. Methods We designed a web-based survey and circulated it to simulation fellowship directors in the United States and Canada. The questions explored aspects of the fellowship curriculum. A grounded theory approach was used to qualitatively analyze fellowship goals and objectives. Results Of the 29 program directors surveyed, 23 responded (79%). The most commonly listed goals and objectives were to increase skills in simulation curriculum development, simulation operations and training environment setup, research, educational theory, administration, and debriefing. The majority of the responding fellowship directors (17 of 22, 77%) indicated that a set of consensus national guidelines would benefit their fellowship program. Conclusions Simulation fellowships are experiencing a period of rapid growth. Development of a common set of program guidelines is a widely shared objective among fellowship directors.

Unannounced standardized patient assessment of the roter interaction analysis system: the challenge of measuring patient-centered communication.

BACKGROUND: Despite wide-spread endorsement of patient-centered communication (PCC) in health care, there has been little evidence that it leads to positive change in health outcomes. The lack of correlation may be due either to an overestimation of the value of PCC or to a measurement problem. If PCC measures do not capture elements of the interaction that determine whether the resulting care plan is patient-centered, they will confound efforts to link PCC to outcomes. OBJECTIVE: To evaluate whether one widely used measure of PCC, the Roter Interaction Analysis System (RIAS), captures patient-centered care planning. DESIGN: RIAS was employed in the coding of unannounced standardized patient (USP) encounters that were scripted so that the failure to address patient contextual factors would result in an ineffective plan of care. The design enabled an assessment of whether RIAS can differentiate between communication behavior that does and does not result in a care plan that takes into account a patient's circumstances and needs. PARTICIPANTS: Eight actors role playing four scripted cases (one African American and one Caucasian for each case) in 399 visits to 111 internal medicine attending physicians. MAIN MEASURES: RIAS measures included composites for physician utterance types and (in separate models) two different previously applied RIAS patient-centeredness summary composites. The gold standard comparison measure was whether the physician's treatment plan, as abstracted from the visit note, successfully addressed the patient's problem. Mixed effects regression models were used to evaluate the relationship between RIAS measures and USP measured performance, controlling for a variety of design features. KEY RESULTS: None of the RIAS measures of PCC differentiated encounters in which care planning was patient-centered from care planning in which it was not. CONCLUSIONS: RIAS, which codes each utterance during a visit into mutually exclusive and exhaustive categories, does not differentiate between conversations leading to and not leading to care plans that accommodate patients' circumstances and needs.

Uncharted territory: measuring costs of diagnostic errors outside the medical record.

CONTEXT: In a past study using unannounced standardised patients (USPs), substantial rates of diagnostic and treatment errors were documented among internists. Because the authors know the correct disposition of these encounters and obtained the physicians' notes, they can identify necessary treatment that was not provided and unnecessary treatment. They can also discern which errors can be identified exclusively from a review of the medical records. OBJECTIVE: To estimate the avoidable direct costs incurred by physicians making errors in our previous study. DESIGN: In the study, USPs visited 111 internal medicine attending physicians. They presented variants of four previously validated cases that jointly manipulate the presence or absence of contextual and biomedical factors that could lead to errors in management if overlooked. For example, in a patient with worsening asthma symptoms, a complicating biomedical factor was the presence of reflux disease and a complicating contextual factor was inability to afford the currently prescribed inhaler. Costs of missed or unnecessary services were computed using Medicare cost-based reimbursement data. SETTING: Fourteen practice locations, including two academic clinics, two community-based primary care networks with multiple sites, a core safety net provider, and three Veteran Administration government facilities. MAIN OUTCOME MEASURES: Contribution of errors to costs of care. RESULTS: Overall, errors in care resulted in predicted costs of approximately $174,000 across 399 visits, of which only $8745 was discernible from a review of the medical records alone (without knowledge of the correct diagnoses). The median cost of error per visit with an incorrect care plan differed by case and by presentation variant within case. CONCLUSIONS: Chart reviews alone underestimate costs of care because they typically reflect appropriate treatment decisions conditional on (potentially erroneous) diagnoses. Important information about patient context is often entirely missing from medical records. Experimental methods, including the use of USPs, reveal the substantial costs of these errors.

A hypothesis-driven physical examination learning and assessment procedure for medical students: initial validity evidence.

CONTEXT: Diagnostic accuracy is maximised by having clinical signs and diagnostic hypotheses in mind during the physical examination (PE). This diagnostic reasoning approach contrasts with the rote, hypothesis-free screening PE learned by many medical students. A hypothesis-driven PE (HDPE) learning and assessment procedure was developed to provide targeted practice and assessment in anticipating, eliciting and interpreting critical aspects of the PE in the context of diagnostic challenges. OBJECTIVES: This study was designed to obtain initial content validity evidence, performance and reliability estimates, and impact data for the HDPE procedure. METHODS: Nineteen clinical scenarios were developed, covering 160 PE manoeuvres. A total of 66 Year 3 medical students prepared for and encountered three clinical scenarios during required formative assessments. For each case, students listed anticipated positive PE findings for two plausible diagnoses before examining the patient; examined a standardised patient (SP) simulating one of the diagnoses; received immediate feedback from the SP, and documented their findings and working diagnosis. The same students later encountered some of the scenarios during their Year 4 clinical skills examination. RESULTS: On average, Year 3 students anticipated 65% of the positive findings, correctly performed 88% of the PE manoeuvres and documented 61% of the findings. Year 4 students anticipated and elicited fewer findings overall, but achieved proportionally more discriminating findings, thereby more efficiently achieving a diagnostic accuracy equivalent to that of students in Year 3. Year 4 students performed better on cases on which they had received feedback as Year 3 students. Twelve cases would provide a reliability of 0.80, based on discriminating checklist items only. CONCLUSIONS: The HDPE provided medical students with a thoughtful, deliberate approach to learning and assessing PE skills in a valid and reliable manner.

Evaluating the effectiveness of rating instruments for a communication skills assessment of medical residents.

The investigators used evidence based on response processes to evaluate and improve the validity of scores on the Patient-Centered Communication and Interpersonal Skills (CIS) Scale for the assessment of residents' communication competence. The investigators retrospectively analyzed the communication skills ratings of 68 residents at the University of Illinois at Chicago (UIC). Each resident encountered six standardized patients (SPs) portraying six cases. SPs rated the performance of each resident using the CIS Scale--an 18-item rating instrument asking for level of agreement on a 5-category scale. A many-faceted Rasch measurement model was used to determine how effectively each item and scale on the rating instrument performed. The analyses revealed that items were too easy for the residents. The SPs underutilized the lowest rating category, making the scale function as a 4-category rating scale. Some SPs were inconsistent when assigning ratings in the middle categories. The investigators modified the rating instrument based on the findings, creating the Revised UIC Communication and Interpersonal Skills (RUCIS) Scale--a 13-item rating instrument that employs a 4-category behaviorally anchored rating scale for each item. The investigators implemented the RUCIS Scale in a subsequent communication skills OSCE for 85 residents. The analyses revealed that the RUCIS Scale functioned more effectively than the CIS Scale in several respects (e.g., a more uniform distribution of ratings across categories, and better fit of the items to the measurement model). However, SPs still rarely assigned ratings in the lowest rating category of each scale.

Exploring individual opinions of potential evaluators in a 360-degree assessment: four distinct viewpoints of a competent resident.

BACKGROUND: Despite the highly acclaimed psychometric features of a 360-degree assessment in the fields of economics, military, and education, there has been increased interest in developing 360-degree instruments to assess competencies in graduate medical education only in the past recent years. Most of the effort to date, however, has focused on developing instruments and testing their reliability and feasibility. Insufficient attention has gone into issues of construct validity and particularly understanding the underlying constructs on which the instruments are based as well as the phenomena that affect ratings. PURPOSE: In preparation for developing a 360-degree assessment instrument, we explored variations in evaluators' opinion type of a competent resident and offer observation about evaluator's professional background and opinions. METHOD: Evaluators from two residency programs ranked 36 opinion statements, using a relative-ranking model, based on their opinion of a competent resident. By-person factor analysis was used to structure opinion types. RESULTS: Factor analysis of 156 responses identified four factors interpreted as four different opinion types of a competent resident: (a) altruistic, compassionate healer (n = 42 evaluators), (b) scientifically grounded clinician (n = 30), (c) holistic, humanistic clinician (n = 62), and (d) patient-focused, health manager (n = 31). Although 72% of nurses/respiratory therapist evaluators expressed type C, 28% expressed other types just as often. Only 14% of evaluator physicians expressed type D, and the remainders were evenly split among the other types. CONCLUSIONS: Our evaluators in 360-degree system expressed four opinion types of a competent resident. The individual opinion and not professional background influences the characteristics an evaluator values in a competent resident. We propose that these values will have an impact on competency assessment and should be taken into account in a 360-degree assessment.

Rater errors in a clinical skills assessment of medical students.

The authors used a many-faceted Rasch measurement model to analyze rating data from a clinical skills assessment of 173 fourth-year medical students to investigate four types of rater errors: leniency, inconsistency, the halo effect, and restriction of range. Students performed six clinical tasks with 6 standardized patients (SPs) selected from a pool of 17 SPs. SPs rated the performance of each student in six skills: history taking, physical examination, interpersonal skills, communication technique, counseling skills, and physical examination etiquette. SPs showed statistically significant differences in their rating severity, indicating rater leniency error. Four SPs exhibited rating inconsistency. Four SPs restricted their ratings in high categories. Only 1 SP exhibited a halo effect. Administrators of objective structured clinical examinations should be vigilant for various types of rater errors and attempt to reduce or eliminate those errors to improve the validity of inferences based on objective structured clinical examination scores.

Developing an institution-based assessment of resident communication and interpersonal skills.

PURPOSE: The authors describe the development and validation of an institution-wide, cross-specialty assessment of residents' communication and interpersonal skills, including related components of patient care and professionalism. METHOD: Residency program faculty, the department of medical education, and the Clinical Performance Center at the University of Illinois at Chicago College of Medicine collaborated to develop six standardized patient-based clinical simulations. The standardized patients rated the residents' performance. The assessment was piloted in 2003 for internal medicine and family medicine and was subsequently adapted for other specialties, including surgery, pediatrics, obstetrics-gynecology, and neurology. We present validity evidence based on the content, internal structure, relationship to other variables, feasibility, acceptability, and impact of the 2003 assessment. RESULTS: Seventy-nine internal medicine and family medicine residents participated in the initial administration of the assessment. A factor analysis of the 18 communication scale items resulted in two factors interpretable as "communication" and "interpersonal skills." Median internal consistency of the scale (coefficient alpha) was 0.91. Generalizability of the assessment ranged from 0.57 to 0.82 across specialties. Case-specific items provided information about group-level deficiencies. Cost of the assessment was about $250 per resident. Once the initial cases had been developed and piloted, they could be adapted for other specialties with minimal additional effort, at a cost saving of about $1,000 per program. CONCLUSION: Centrally developed, institution-wide competency assessment uses resources efficiently to relieve individual programs of the need to "reinvent the wheel" and provides program directors and residents with useful information for individual and programmatic review.

Prior experiences associated with residents' scores on a communication and interpersonal skill OSCE.

OBJECTIVE: This exploratory study investigated whether prior task experience and comfort correlate with scores on an assessment of patient-centered communication. METHODS: A six-station standardized patient exam assessed patient-centered communication of 79 PGY2-3 residents in Internal Medicine and Family Medicine. A survey provided information on prior experiences. t-tests, correlations, and multi-factorial ANOVA explored relationship between scores and experiences. RESULTS: Experience with a task predicted comfort but did not predict communication scores. Comfort was moderately correlated with communication scores for some tasks; residents who were less comfortable were indeed less skilled, but greater comfort did not predict higher scores. Female gender and medical school experiences with standardized patients along with training in patient-centered interviewing were associated with higher scores. Residents without standardized patient experiences in medical school were almost five times more likely to be rejected by patients. CONCLUSIONS: Task experience alone does not guarantee better communication, and may instill a false sense of confidence. Experiences with standardized patients during medical school, especially in combination with interviewing courses, may provide an element of "deliberate practice" and have a long-term impact on communication skills. PRACTICE IMPLICATIONS: The combination of didactic courses and practice with standardized patients may promote a patient-centered approach.