Validity Evidence for the Physician Oral Language Observation Matrix as a Measure of Medical Spanish Proficiency.

BACKGROUND: The lack of a standardized language assessment process for medical students and physicians communicating in a non-English language threatens healthcare quality and safety. OBJECTIVE: To evaluate the validity of a new rating tool, the Physician Oral Language Observation Matrix (POLOM)™, in assessing medical students' oral communication with Spanish-speaking standardized patients (SPs). DESIGN: POLOM scores were compared to measures of student medical Spanish proficiency to examine convergent validity and to measures of clinical performance to examine concurrent/criterion validity. PARTICIPANTS: Forty-two students at two schools completed SP encounters between January 2021 and April 2022, and POLOM raters scored the videorecorded performances between January and June 2022. MAIN MEASURES: Two approaches to generating POLOM total scores were investigated: rater average and strict consensus. Convergent validity was examined via the POLOM's correlations with (1) the phone-based Clinician Cultural and Linguistic Assessment (CCLA) and (2) the self-rated Interagency Language Roundtable scale for healthcare (ILR-H). Concurrent/criterion validity was examined via correlations with (1) the Comunicación y Habilidades Interpersonales (CAI) scale, (2) a checklist completed by the SP, and (3) a faculty rating of the student's post-encounter clinical note. Pearson's correlations of r ≥ 0.5 and r ≥ 0.2 were considered evidence of convergent validity and concurrent/criterion validity, respectively. KEY RESULTS: Both rater average and strict consensus POLOM scores were strongly correlated with ILR-H (r = 0.72) and CCLA (r ≥ 0.60), providing evidence of convergent validity. The POLOM was substantially correlated with the CAI (r ≥ 0.29), the SP Checklist (r = 0.32), and the faculty scoring of the student's clinical note (r ≥ 0.24), providing concurrent/criterion validity evidence. CONCLUSIONS: The POLOM has demonstrated evidence of convergent and concurrent/criterion validity as a measure of medical students' Spanish proficiency during SP encounters. Additional research is needed to evaluate how the POLOM can be implemented with resident and practicing physicians, applied to other health professions, and adapted to other languages.

A Systematic Review of the Impact of Patient-Physician Non-English Language Concordance on Quality of Care and Outcomes.

BACKGROUND: Approximately 25 million people in the USA are limited English proficient (LEP). When LEP patients receive care from physicians who are truly language concordant, some evidence show that language disparities are reduced, but others demonstrate worse outcomes. We conducted a systematic review of the literature to compare the impact of language-concordant care for LEP patients with that of other interventions, including professional and ad hoc interpreters. METHODS: Data was collected through a systematic review of the literature using PubMed, PsycINFO, Web of Science, Cochrane Library, and EMBASE in October 2017. The literature search strategy had three main components, which were immigrant/minority status, language barrier/proficiency, and healthcare provider/patient relationship. The quality of the articles was appraised using the Downs and Black checklist. RESULTS: The 33 studies were grouped by the outcome measure studied, including quality of care (subdivided into primary care, diabetes, pain management, cancer, and inpatient), satisfaction with care/communication, medical understanding, and mental health. Of the 33, 4 (6.9%) were randomized controlled trials and the remaining 29 (87.9%) were cross-sectional studies. Seventy-six percent (25/33) of the studies demonstrated that at least one of the outcomes assessed was better for patients receiving language-concordant care, while 15% (5/33) of studies demonstrated no difference in outcomes, and 9% (3/33) studies demonstrated worse outcomes in patients receiving language-concordant care. DISCUSSION: The findings of this review indicate that, in the majority of situations, language-concordant care improves outcomes. Although most studies included were of good quality, none provided a standardized assessment of provider language skills. To systematically evaluate the impact of truly language-concordant care on outcomes and draw meaningful conclusions, future studies must include an assessment of clinician language proficiency. Language-concordant care offers an important way for physicians to meet the unique needs of their LEP patients.

Development of a Tool to Assess Medical Oral Language Proficiency.

PURPOSE: To communicate with linguistically diverse patients, medical students and physicians often use their non-English-language skills. However, there is no standard protocol to determine whether those skills are adequate before patient care. This causes many physicians, institutions, educators, and learners to forgo non-English-language proficiency assessment altogether. The purpose of this study is to report on the development, refinement, and interrater reliability of the Physician Oral Language Observation Matrix (POLOM), a rater-based tool assessing 6 language skill categories observed during clinical interactions: comprehension, fluency/fluidity, vocabulary, pronunciation, grammar, and communication. This study focused on the use of the POLOM in Spanish interactions. METHOD: The authors adapted an existing language observation tool for use in clinical settings, creating the preliminary POLOM. Next, they iteratively refined the tool from April to July 2021 using videorecorded medical student-standardized patient encounters from a U.S.-based medical Spanish program. In each refinement iteration, 4 bilingual raters (2 physicians and 2 linguists) independently rated 3 to 6 encounters and convened to discuss ratings with the goals of improving instrument instructions, descriptors, and subsequent rater agreement. Using the final POLOM, raters independently rated 50 videos in rotating interdisciplinary pairs. Generalizability theory was applied to estimate reliability via interrater agreement (dependability) coefficients (range 0-1) for each POLOM category and the total score. RESULTS: POLOM total score dependability equaled 0.927 (single rater) and 0.962 (averaged across 2 raters). The highest mean score was observed for the comprehension category (4.15; range 1-5) while the lowest was for communication (3.01; range 1-5). CONCLUSIONS: Raters achieved a high level of agreement on POLOM assessments of students' medical oral Spanish proficiency. The POLOM is the first assessment tool that provides examinees and instructors with both a holistic and detailed review of clinician non-English oral language skills as contextualized for patient care.

Current State and Proposal of Best Practices of Standardized Patients for Spanish-Language Objective Structured Clinical Encounters in US Medical Schools.

INTRODUCTION: Medical Spanish programs commonly engage Spanish-speaking standardized patients (SPs) for communication skills assessment, yet no studies address SP recruitment, selection, or training. METHODS: We sent questionnaires to medical Spanish faculty at 20 US medical schools to gauge their practices in recruiting and selecting Spanish-language SPs. We invited faculty to distribute a separate questionnaire to Spanish-language SPs to gather SP language abilities, training, and experience. We analyzed data using descriptive statistics and qualitative content analysis. When available, we reviewed SP video encounters to formally assess participating SPs' linguistic performance using the SP Oral Language Observation Matrix, a rating tool adapted from the Physician Oral Language Observation Matrix to assess oral medical Spanish proficiency. RESULTS: Eighty percent of faculty (16/20) responded. Standardized patient recruitment sources included institutional English-language SPs, Hispanic student groups and professional organizations, communities, and language professionals. Faculty-reported strategies to determine language readiness included interviewing SP candidates in Spanish and asking them to self-rate language skills using a validated scale. Fifteen SPs (54%, 15/28) from 5 schools responded to the SP questionnaire, and one third (5/15) reported that their Spanish was not assessed before being selected as an SP. In addition, one third (5/15) did not receive any initial training before performing a medical Spanish case. Raters assessed 11 different SPs using the SP Oral Language Observation Matrix, and 6 were rated as linguistically "ready" for the SP role. CONCLUSIONS: Current approaches to recruitment, training, and language assessment of SPs vary. We propose strategies to ensure that medical Spanish encounters authentically reflect Spanish-speaking patients.