Impact of an unannounced standardized veteran program on access to community-based services for veterans experiencing homelessness.

BACKGROUND: The United States Department of Veterans Affairs established a program in which actors incognito portray veterans experiencing homelessness with pre-determined needs to identify barriers to access and services at community-based organizations. METHODS: From 2017 to 2019, actors who varied in gender, skin color and age portrayed one of three scripts at all VA Community-Based Resource and Referral Centers (CRRCs) serving veterans experiencing homelessness in 30 cities and completed an evaluative survey. They carried authentic VA identification and were registered in a VA patient database for each identity. CRRCs were provided with reports annually and asked to implement corrective plans. Data from the survey were analysed for change over time. RESULTS: Access to food, counselling, PTSD treatment, and hypertension/prediabetes care services increased significantly from 68-77% in year 2 to 83-97% in year 3 (each P < 0.05 adjusted for script present). A significant disparity in access for African American actors resolved following more uniform adherence to pre-existing policies. CONCLUSIONS: The 'unannounced standardized veteran' (USV) can identify previously unrecognized barriers to needed services and care. Audit and feedback programs based on direct covert observation with systematic data collection and rapid feedback may be an effective strategy for improving services to highly vulnerable populations.

Evaluation of a Patient-Collected Audio Audit and Feedback Quality Improvement Program on Clinician Attention to Patient Life Context and Health Care Costs in the Veterans Affairs Health Care System.

Importance: Evidence-based care plans can fail when they do not consider relevant patient life circumstances, termed contextual factors, such as a loss of social support or financial hardship. Preventing these contextual errors can reduce obstacles to effective care. Objective: To evaluate the effectiveness of a quality improvement program in which clinicians receive ongoing feedback on their attention to patient contextual factors. Design, Setting, and Participants: In this quality improvement study, patients at 6 Department of Veterans Affairs outpatient facilities audio recorded their primary care visits from May 2017 to May 2019. Encounters were analyzed using the Content Coding for Contextualization of Care (4C) method. A feedback intervention based on the 4C coded analysis was introduced using a stepped wedge design. In the 4C coding schema, clues that patients are struggling with contextual factors are termed contextual red flags (eg, sudden loss of control of a chronic condition), and a positive outcome is prospectively defined for each encounter as a quantifiable improvement of the contextual red flag. Data analysis was performed from May to October 2019. Interventions: Clinicians received feedback at 2 intensity levels on their attention to patient contextual factors and on predefined patient outcomes at 4 to 6 months. Main Outcomes and Measures: Contextual error rates, patient outcomes, and hospitalization rates and costs were measured. Results: The patients (mean age, 62.0 years; 92% male) recorded 4496 encounters with 666 clinicians. At baseline, clinicians addressed 413 of 618 contextual factors in their care plans (67%). After either standard or enhanced feedback, they addressed 1707 of 2367 contextual factors (72%), a significant difference (odds ratio, 1.3; 95% CI, 1.1-1.6; P = .01). In a mixed-effects logistic regression model, contextualized care planning was associated with a greater likelihood of improved outcomes (adjusted odds ratio, 2.5; 95% CI, 1.5-4.1; P < .001). In a budget analysis, estimated savings from avoided hospitalizations were $25.2 million (95% CI, $23.9-$26.6 million), at a cost of $337 242 for the intervention. Conclusions and Relevance: These findings suggest that patient-collected audio recordings of the medical encounter with feedback may enhance clinician attention to contextual factors, improve outcomes, and reduce hospitalizations. In addition, the intervention is associated with substantial cost savings.

Alerting Doctors About Patient Life Challenges: A Randomized Control Trial of a Previsit Inventory of Contextual Factors.

Objective. Effective care attends to relevant patient life context. We tested whether a patient-completed inventory helps providers contextualize care and increases patients' perception of patient-centered care (PCC). Method. The inventory listed six red flags (e.g., emergency room visits) and if the patient checked any, prompted for related contextual factors (e.g., transportation difficulties). Patients were randomized to complete the inventory or watch health videos prior to their visit. Patients presented their inventory results to providers during audio-recorded encounters. Audios were coded for physician probing and incorporating context in care plans. Patients completed the Consultation and Relational Empathy (CARE) instrument after the encounter. Results. A total of 272 Veterans were randomized. Adjusting for covariates and clustering within providers, inventory patients rated visits as more patient-centered (44.5; standard error = 1.1) than controls (42.7, standard error = 1.1, P = 0.04, CARE range = 10-50). Providers were more likely to probe red flags (odds ratio = 1.54; confidence interval = 1.07-2.22; P = 0.02) when receiving the inventory, but not incorporating context into care planning. Conclusion. A previsit inventory of life context increased perceptions of PCC and providers' likelihood of exploring context but not contextualizing care. Information about patients' life challenges is not sufficient to assure that context informs provider decision making even when provided at the point of care by patients themselves.

Comparing announced with unannounced standardized patients in performance assessment.

BACKGROUND: Accurately assessing how physicians perform in practice remains an unresolved psychometric challenge. Neither chart reviews nor patient surveys indicate when physicians overlook important information, which can result in a missed opportunity for a correct diagnosis and appropriate plan of care. Standardized patient (SP) assessments provide an opportunity for direct observation of clinical behavior and are increasingly used in licensure examinations. (SPs who are sent incognito are termed unannounced standardized patients [USPs].) One study showed that physicians had particular difficulty adapting care to individual patient context ("contextual error"). In a subsequent study with the same actors, SP cases, and outcomes, an intervention was deployed to reduce contextual error among medical students. In an exploratory reanalysis of data from the two studies, clinicians' assessments of SPs and USPs were compared. METHODS: Participants in the first study were 65 board-certified internists visited by USPs; the 59 participants in the second were fourth-year medical students examining SPs in a clinical performance center. RESULTS: Attending physicians measured with USPs significantly underperformed medical students measured with SPs in the probing of biomedical red flags (odds ratio [OR] = 0.45 [0.30 to 0.67]) and contextual red flags (OR = 0.66 [0.45 to 0.99]) and in planning appropriate care (OR = 0.43 [0.27 to 0.67]). CONCLUSIONS: Across these two studies, attending physicians underperformed medical students on the same outcomes, measured with the same patient cases presented by the same actors. Studies that seek to assess elicitation and incorporation of patient information by physicians as measures of individualization of care should weigh the benefits and costs of direct observation by USPs.

Unannounced standardized patient assessment of the roter interaction analysis system: the challenge of measuring patient-centered communication.

BACKGROUND: Despite wide-spread endorsement of patient-centered communication (PCC) in health care, there has been little evidence that it leads to positive change in health outcomes. The lack of correlation may be due either to an overestimation of the value of PCC or to a measurement problem. If PCC measures do not capture elements of the interaction that determine whether the resulting care plan is patient-centered, they will confound efforts to link PCC to outcomes. OBJECTIVE: To evaluate whether one widely used measure of PCC, the Roter Interaction Analysis System (RIAS), captures patient-centered care planning. DESIGN: RIAS was employed in the coding of unannounced standardized patient (USP) encounters that were scripted so that the failure to address patient contextual factors would result in an ineffective plan of care. The design enabled an assessment of whether RIAS can differentiate between communication behavior that does and does not result in a care plan that takes into account a patient's circumstances and needs. PARTICIPANTS: Eight actors role playing four scripted cases (one African American and one Caucasian for each case) in 399 visits to 111 internal medicine attending physicians. MAIN MEASURES: RIAS measures included composites for physician utterance types and (in separate models) two different previously applied RIAS patient-centeredness summary composites. The gold standard comparison measure was whether the physician's treatment plan, as abstracted from the visit note, successfully addressed the patient's problem. Mixed effects regression models were used to evaluate the relationship between RIAS measures and USP measured performance, controlling for a variety of design features. KEY RESULTS: None of the RIAS measures of PCC differentiated encounters in which care planning was patient-centered from care planning in which it was not. CONCLUSIONS: RIAS, which codes each utterance during a visit into mutually exclusive and exhaustive categories, does not differentiate between conversations leading to and not leading to care plans that accommodate patients' circumstances and needs.

Uncharted territory: measuring costs of diagnostic errors outside the medical record.

CONTEXT: In a past study using unannounced standardised patients (USPs), substantial rates of diagnostic and treatment errors were documented among internists. Because the authors know the correct disposition of these encounters and obtained the physicians' notes, they can identify necessary treatment that was not provided and unnecessary treatment. They can also discern which errors can be identified exclusively from a review of the medical records. OBJECTIVE: To estimate the avoidable direct costs incurred by physicians making errors in our previous study. DESIGN: In the study, USPs visited 111 internal medicine attending physicians. They presented variants of four previously validated cases that jointly manipulate the presence or absence of contextual and biomedical factors that could lead to errors in management if overlooked. For example, in a patient with worsening asthma symptoms, a complicating biomedical factor was the presence of reflux disease and a complicating contextual factor was inability to afford the currently prescribed inhaler. Costs of missed or unnecessary services were computed using Medicare cost-based reimbursement data. SETTING: Fourteen practice locations, including two academic clinics, two community-based primary care networks with multiple sites, a core safety net provider, and three Veteran Administration government facilities. MAIN OUTCOME MEASURES: Contribution of errors to costs of care. RESULTS: Overall, errors in care resulted in predicted costs of approximately $174,000 across 399 visits, of which only $8745 was discernible from a review of the medical records alone (without knowledge of the correct diagnoses). The median cost of error per visit with an incorrect care plan differed by case and by presentation variant within case. CONCLUSIONS: Chart reviews alone underestimate costs of care because they typically reflect appropriate treatment decisions conditional on (potentially erroneous) diagnoses. Important information about patient context is often entirely missing from medical records. Experimental methods, including the use of USPs, reveal the substantial costs of these errors.