Directly Observed Care: Crossing the Chasm of Quality Measurement.

After more than two decades of national attention to quality improvement in US healthcare, significant gaps in quality remain. A fundamental problem is that current approaches to measure quality are indirect and therefore imprecise, focusing on clinical documentation of care rather than the actual delivery of care. The National Academy of Medicine (NAM) has identified six domains of quality that are essential to address to improve quality: patient-centeredness, equity, timeliness, efficiency, effectiveness, and safety. In this perspective, we describe how directly observed care-a recorded audit of clinical care delivery-may address problems with current quality measurement, providing a more holistic assessment of healthcare delivery. We further show how directly observed care has the potential to improve each NAM domain of quality.

Impact of an unannounced standardized veteran program on access to community-based services for veterans experiencing homelessness.

BACKGROUND: The United States Department of Veterans Affairs established a program in which actors incognito portray veterans experiencing homelessness with pre-determined needs to identify barriers to access and services at community-based organizations. METHODS: From 2017 to 2019, actors who varied in gender, skin color and age portrayed one of three scripts at all VA Community-Based Resource and Referral Centers (CRRCs) serving veterans experiencing homelessness in 30 cities and completed an evaluative survey. They carried authentic VA identification and were registered in a VA patient database for each identity. CRRCs were provided with reports annually and asked to implement corrective plans. Data from the survey were analysed for change over time. RESULTS: Access to food, counselling, PTSD treatment, and hypertension/prediabetes care services increased significantly from 68-77% in year 2 to 83-97% in year 3 (each P < 0.05 adjusted for script present). A significant disparity in access for African American actors resolved following more uniform adherence to pre-existing policies. CONCLUSIONS: The 'unannounced standardized veteran' (USV) can identify previously unrecognized barriers to needed services and care. Audit and feedback programs based on direct covert observation with systematic data collection and rapid feedback may be an effective strategy for improving services to highly vulnerable populations.

Patient-centered quality measurement for opioid use disorder: Development of a taxonomy to address gaps in research and practice.

Background: Evidence-based treatment is provided infrequently and inconsistently to patients with opioid use disorder (OUD). Treatment guidelines call for high-quality, patient-centered care that meets individual preferences and needs, but it is unclear whether current quality measures address individualized aspects of care and whether measures of patient-centered OUD care are supported by evidence. Methods: We conducted an environmental scan of OUD care quality to (1) evaluate patient-centeredness in current OUD quality measures endorsed by national agencies and in national OUD treatment guidelines; and (2) review literature evidence for patient-centered care in OUD diagnosis and management, including gaps in current guidelines, performance data, and quality measures. We then synthesized these findings to develop a new quality measurement taxonomy that incorporates patient-centered aspects of care and identifies priority areas for future research and quality measure development. Results: Across 31 endorsed OUD quality measures, only two measures of patient experience incorporated patient preferences and needs, while national guidelines emphasized providing patient-centered care. Among 689 articles reviewed, evidence varied for practices of patient-centered care. Many practices were supported by guidelines and substantial evidence, while others lacked evidence despite guideline support. Our synthesis of findings resulted in EQuIITable Care, a taxonomy comprised of six classifications: (1) patient Experience and engagement, (2) Quality of life; (3) Identification of patient risks; (4) Interventions to mitigate patient risks; (5) Treatment; and (6) Care coordination and navigation. Conclusions: Current quality measurement for OUD lacks patient-centeredness. EQuIITable Care for OUD provides a roadmap to develop measures of patient-centered care for OUD.

Impact of Unannounced Standardized Patient Audit and Feedback on Care, Documentation, and Costs: an Experiment and Claims Analysis.

BACKGROUND: Meaningful variations in physician performance are not always discernible from the medical record. OBJECTIVE: We used unannounced standardized patients to measure and provide feedback on care quality and fidelity of documentation, and examined downstream effects on reimbursement claims. DESIGN: Static group pre-post comparison study conducted between 2017 and 2019. SETTING: Fourteen New Jersey primary care practice groups (22 practices) enrolled in Horizon BCBS's value-based program received the intervention. For claims analyses, we identified 14 additional comparison practice groups matched on county, practice size, and claims activity. PARTICIPANTS: Fifty-nine of 64 providers volunteered to participate. INTERVENTION: Unannounced standardized patients (USPs) made 217 visits portraying patients with 1-2 focal conditions (diabetes, depression, back pain, smoking, or preventive cancer screening). After two baseline visits to a provider, we delivered feedback and conducted two follow-up visits. MEASUREMENTS: USP-completed checklists of guideline-based provider care behaviors, visit audio recordings, and provider notes were used to measure behaviors performed and documentation errors pre- and post-feedback. We also compared changes in 3-month office-based claims by actual patients between the intervention and comparison practice groups before and after feedback. RESULTS: Expected clinical behaviors increased from 46% to 56% (OR = 1.53, 95% CI 1.29-1.83, p < 0.0001), with significant improvements in smoking cessation, back pain, and depression screening. Providers were less likely to document unperformed tasks after (16%) than before feedback (18%; OR = 0.74, 95% CI 0.62 to 0.90, p = 0.002). Actual claim costs increased significantly less in the study than comparison group for diabetes and depression but significantly more for smoking cessation, cancer screening, and low back pain. LIMITATIONS: Self-selection of participating practices and lack of access to prescription claims. CONCLUSION: Direct observation of care identifies hidden deficits in practice and documentation, and with feedback can improve both, with concomitant effects on costs.

Implementation of a patient-collected audio recording audit & feedback quality improvement program to prevent contextual error: stakeholder perspective.

BACKGROUND: Using patient audio recordings of medical visits to provide clinicians with feedback on their attention to patient life context in care planning can improve health care delivery and outcomes, and reduce costs. However, such an initiative can raise concerns across stakeholders about surveillance, intrusiveness and merit. This study examined the perspectives of patients, physicians and other clinical staff, and facility leaders over 3 years at six sites during the implementation of a patient-collected audio quality improvement program designed to improve patient-centered care in a non-threatening manner and with minimal effort required of patients and clinicians. METHODS: Patients were invited during the first and third year to complete exit surveys when they returned their audio recorders following visits, and clinicians to complete surveys annually. Clinicians were invited to participate in focus groups in the first and third years. Facility leaders were interviewed individually during the last 6 months of the study. RESULTS: There were a total of 12 focus groups with 89 participants, and 30 leadership interviews. Two hundred fourteen clinicians and 800 patients completed surveys. In a qualitative analysis of focus group data employing NVivo, clinicians initially expressed concerns that the program could be disruptive and/or burdensome, but these diminished with program exposure and were substantially replaced by an appreciation for the value of low stakes constructive feedback. They were also significantly more confident in the value of the intervention in the final year (p = .008), more likely to agree that leadership supports continuous improvement of patient care and gives feedback on outcomes (p = .02), and at a time that is convenient (p = .04). Patients who volunteered sometimes expressed concerns they were "spying" on their doctors, but most saw it as an opportunity to improve care. Leaders were supportive of the program but not yet prepared to commit to funding it exclusively with facility resources. CONCLUSIONS: A patient-collected audio program can be implemented when it is perceived as safe, not disruptive or burdensome, and as contributing to better health care.

Directly observed care: can unannounced standardized patients address a gap in performance measurement?

There are three potential sources of information for evaluating a clinician's performance: documentation, patient report, and directly observed care. Current measures draw on just two of these: data recorded in the medical record and surveys of patients. Neither captures an array of performance characteristics, including clinician attention to symptoms and signs while taking a history or conducting a physical exam, accurate recording in the medical record of information obtained during the encounter, evidence based communication strategies for preventive care counseling, and effective communication behavior. Unannounced Standardized Patients (USPs) have been widely deployed as a research strategy for systematically uncovering significant performance deficits in each of these areas, but have not been adopted for quality improvement. Likely obstacles include concerns about the ethics of sending health professionals sham patients, the technical challenges of the subterfuge, and concerns about the relatively small sample sizes and substantial costs involved. However, the high frequency of significant and remediable performance deficits unmasked by USPs, and the potential to adapt registration and record keeping systems to accommodate their visits, suggest that their selective and purposeful deployment could be a cost effective and powerful strategy for addressing a gap in performance measurement.

Patient-centered decision making and health care outcomes: an observational study.

BACKGROUND: Patient-centered decision making (PCDM) is the process of identifying clinically relevant, patient-specific circumstances and behaviors to formulate a contextually appropriate care plan. OBJECTIVE: To ascertain whether encounters in which PCDM occurs are followed by improved health care outcomes compared with encounters where there is inattention to patient context. DESIGN: Patients surreptitiously audio-recorded encounters with their physicians. Medical records of these encounters were then screened for "contextual red flags," such as deteriorating self-management of a chronic condition, that could reflect such underlying contextual factors as competing responsibilities or loss of social support. When a contextual factor was identified, either as a result of physician questioning or because a patient volunteered information, physicians were scored on the basis of whether they adapted the care plan to it. SETTING: Internal medicine clinics at 2 Veterans Affairs facilities. PARTICIPANTS: 774 patients audio-recorded encounters with 139 resident physicians. MEASUREMENTS: Individualized outcome measures were based on the contextual red flag, such as improved blood pressure control in a patient presenting with hypertension and loss of medication coverage. Outcome coders were blinded to physician performance. RESULTS: Among 548 contextual red flags, 208 contextual factors were confirmed, either when physicians probed or patients volunteered information. Physician attention to contextual factors (both probing for them and addressing them in care plans) varied according to the presenting contextual red flags. Outcome data were available for 157 contextual factors, of which PCDM was found to address 96. Of these, health care outcomes improved in 68 (71%), compared with 28 (46%) of the 61 that were not addressed by PCDM (P = 0.002). LIMITATION: The extent to which the findings can be generalized to other clinical settings is unknown. CONCLUSION: Attention to patient needs and circumstances when planning care is associated with improved health care outcomes. PRIMARY FUNDING SOURCE: U.S. Department of Veterans Affairs, Health Services Research & Development Service.

Unannounced standardized patient assessment of the roter interaction analysis system: the challenge of measuring patient-centered communication.

BACKGROUND: Despite wide-spread endorsement of patient-centered communication (PCC) in health care, there has been little evidence that it leads to positive change in health outcomes. The lack of correlation may be due either to an overestimation of the value of PCC or to a measurement problem. If PCC measures do not capture elements of the interaction that determine whether the resulting care plan is patient-centered, they will confound efforts to link PCC to outcomes. OBJECTIVE: To evaluate whether one widely used measure of PCC, the Roter Interaction Analysis System (RIAS), captures patient-centered care planning. DESIGN: RIAS was employed in the coding of unannounced standardized patient (USP) encounters that were scripted so that the failure to address patient contextual factors would result in an ineffective plan of care. The design enabled an assessment of whether RIAS can differentiate between communication behavior that does and does not result in a care plan that takes into account a patient's circumstances and needs. PARTICIPANTS: Eight actors role playing four scripted cases (one African American and one Caucasian for each case) in 399 visits to 111 internal medicine attending physicians. MAIN MEASURES: RIAS measures included composites for physician utterance types and (in separate models) two different previously applied RIAS patient-centeredness summary composites. The gold standard comparison measure was whether the physician's treatment plan, as abstracted from the visit note, successfully addressed the patient's problem. Mixed effects regression models were used to evaluate the relationship between RIAS measures and USP measured performance, controlling for a variety of design features. KEY RESULTS: None of the RIAS measures of PCC differentiated encounters in which care planning was patient-centered from care planning in which it was not. CONCLUSIONS: RIAS, which codes each utterance during a visit into mutually exclusive and exhaustive categories, does not differentiate between conversations leading to and not leading to care plans that accommodate patients' circumstances and needs.

Comparing announced with unannounced standardized patients in performance assessment.

BACKGROUND: Accurately assessing how physicians perform in practice remains an unresolved psychometric challenge. Neither chart reviews nor patient surveys indicate when physicians overlook important information, which can result in a missed opportunity for a correct diagnosis and appropriate plan of care. Standardized patient (SP) assessments provide an opportunity for direct observation of clinical behavior and are increasingly used in licensure examinations. (SPs who are sent incognito are termed unannounced standardized patients [USPs].) One study showed that physicians had particular difficulty adapting care to individual patient context ("contextual error"). In a subsequent study with the same actors, SP cases, and outcomes, an intervention was deployed to reduce contextual error among medical students. In an exploratory reanalysis of data from the two studies, clinicians' assessments of SPs and USPs were compared. METHODS: Participants in the first study were 65 board-certified internists visited by USPs; the 59 participants in the second were fourth-year medical students examining SPs in a clinical performance center. RESULTS: Attending physicians measured with USPs significantly underperformed medical students measured with SPs in the probing of biomedical red flags (odds ratio [OR] = 0.45 [0.30 to 0.67]) and contextual red flags (OR = 0.66 [0.45 to 0.99]) and in planning appropriate care (OR = 0.43 [0.27 to 0.67]). CONCLUSIONS: Across these two studies, attending physicians underperformed medical students on the same outcomes, measured with the same patient cases presented by the same actors. Studies that seek to assess elicitation and incorporation of patient information by physicians as measures of individualization of care should weigh the benefits and costs of direct observation by USPs.

Contextualizing care: An essential and measurable clinical competency.

Contextualizing care is the process of adapting research evidence to patient life context. The failure to do so, when it results in a care plan that is not likely to achieve its intended aim, is a contextual error. There is substantial evidence that contextual errors are common, adversely affect patient outcomes and health care costs, and are preventable. This evidence comes from over 5000 mostly incognito recordings of physician-patient encounters over a range of practice settings that have been analyzed along with the medical records of each encounter utilizing a specialized coding algorithm. Educational and practice improvement interventions have been tested at the medical student, resident, and attending level, each with evidence of benefits and limitations. The author argues that contextualizing care is an essential clinician competency and proposes an evidence-informed strategy for building and reinforcing the requisite skills across the continuum of medical education and professional development.

Cross-Sectional Associations: Social Risks and Diabetes Care Quality, Outcomes.

INTRODUCTION: Social risks (e.g., food/transportation insecurity) can hamper type 2 diabetes mellitus (T2DM) self-management, leading to poor outcomes. To determine the extent to which high-quality care can overcome social risks' health impacts, this study assessed the associations between reported social risks, receipt of guideline-based T2DM care, and T2DM outcomes when care is up to date among community health center patients. METHODS: A cross-sectional study of adults aged ≥18 years (N=73,484) seen at 186 community health centers, with T2DM and ≥1 year of observation between July 2016 and February 2020. Measures of T2DM care included up-to-date HbA1c, microalbuminuria, low-density lipoprotein screening, and foot examination, and active statin prescription when indicated. Measures of T2DM outcomes among patients with up-to-date care included blood pressure, HbA1c, and low-density lipoprotein control on or within 6‒12 months of an index encounter. Analyses were conducted in 2021. RESULTS: Individuals reporting transportation or housing insecurity were less likely to have up-to-date low-density lipoprotein screening; no other associations were seen between social risks and clinical care quality. Among individuals with up-to-date care, food insecurity was associated with lower adjusted rates of controlled HbA1c (79% vs 75%, p<0.001), and transportation insecurity was associated with lower rates of controlled HbA1c (79% vs 74%, p=0.005), blood pressure (74% vs 72%, p=0.025), and low-density lipoprotein (61% vs 57%, p=0.009) than among those with no reported need. CONCLUSIONS: Community health center patients received similar care regardless of the presence of social risks. However, even among those up to date on care, social risks were associated with worse T2DM control. Future research should identify strategies for improving HbA1c control for individuals with social risks. TRIAL REGISTRATION: This study is registered at www. CLINICALTRIALS: gov NCT03607617.

Effect of Electronic Health Record Clinical Decision Support on Contextualization of Care: A Randomized Clinical Trial.

Importance: Contextualizing care is a process of incorporating information about the life circumstances and behavior of individual patients, termed contextual factors, into their plan of care. In 4 steps, clinicians recognize clues (termed contextual red flags), clinicians ask about them (probe for context), patients disclose contextual factors, and clinicians adapt care accordingly. The process is associated with a desired outcome resolution of the presenting contextual red flag. Objective: To determine whether contextualized clinical decision support (CDS) tools in the electronic health record (EHR) improve clinician contextual probing, attention to contextual factors in care planning, and the presentation of contextual red flags. Design, Setting, and Participants: This randomized clinical trial was performed at the primary care clinics of 2 academic medical centers with different EHR systems. Participants were adults 18 years or older consenting to audio record their visits and their physicians between September 6, 2018, and March 4, 2021. Patients were randomized to an intervention or a control group. Analyses were performed on an intention-to-treat basis. Interventions: Patients completed a previsit questionnaire that elicited contextual red flags and factors and appeared in the clinician's note template in a contextual care box. The EHR also culled red flags from the medical record, included them in the contextual care box, used passive and interruptive alerts, and proposed relevant orders. Main Outcomes and Measures: Proportion of contextual red flags noted at the index visit that resolved 6 months later (primary outcome), proportion of red flags probed (secondary outcome), and proportion of contextual factors addressed in the care plan by clinicians (secondary outcome), adjusted for study site and for multiple red flags and factors within a visit. Results: Four hundred fifty-two patients (291 women [65.1%]; mean [SD] age, 55.6 [15.1] years) completed encounters with 39 clinicians (23 women [59.0%]). Contextual red flags were not more likely to resolve in the intervention vs control group (adjusted odds ratio [aOR], 0.96 [95% CI, 0.57-1.63]). However, the intervention increased both contextual probing (aOR, 2.12 [95% CI, 1.14-3.93]) and contextualization of the care plan (aOR, 2.67 [95% CI, 1.32-5.41]), controlling for whether a factor was identified by probing or otherwise. Across study groups, contextualized care plans were more likely than noncontextualized plans to result in improvement in the presenting red flag (aOR, 2.13 [95% CI, 1.38-3.28]). Conclusions and Relevance: This randomized clinical trial found that contextualized CDS did not improve patients' outcomes but did increase contextualization of their care, suggesting that use of this technology could ultimately help improve outcomes. Trial Registration: ClinicalTrials.gov Identifier: NCT03244033.

Contextual errors and failures in individualizing patient care: a multicenter study.

BACKGROUND: A contextual error occurs when a physician overlooks elements of a patient's environment or behavior that are essential to planning appropriate care. In contrast to biomedical errors, which are not patient-specific, contextual errors represent a failure to individualize care. OBJECTIVE: To explore the frequency and circumstances under which physicians probe contextual and biomedical red flags and avoid treatment error by incorporating what they learn from these probes. DESIGN: An incomplete randomized block design in which unannounced, standardized patients visited 111 internal medicine attending physicians between April 2007 and April 2009 and presented variants of 4 scenarios. In all scenarios, patients presented both a contextual and a biomedical red flag. Responses to probing about flags varied in whether they revealed an underlying complicating biomedical or contextual factor (or both) that would lead to errors in management if overlooked. SETTING: 14 practices, including 2 academic clinics, 2 community-based primary care networks with multiple sites, a core safety net provider, and 3 U.S. Department of Veterans Affairs facilities. MEASUREMENTS: Primary outcomes were the proportion of visits in which physicians probed for contextual and biomedical factors in response to hints or red flags and the proportion of visits that resulted in error-free treatment plans. RESULTS: Physicians probed fewer contextual red flags (51%) than biomedical red flags (63%). Probing for contextual or biomedical information in response to red flags was usually necessary but not sufficient for an error-free plan of care. Physicians provided error-free care in 73% of the uncomplicated encounters, 38% of the biomedically complicated encounters, 22% of the contextually complicated encounters, and 9% of the combined biomedically and contextually complicated encounters. LIMITATIONS: Only 4 case scenarios were used. The study assessed physicians' propensity to make errors when every encounter provided an opportunity to do so and did not measure actual error rates that occur in primary care settings because of inattention to context. CONCLUSION: Inattention to contextual information, such as a patient's transportation needs, economic situation, or caretaker responsibilities, can lead to contextual error, which is not currently measured in assessments of physician performance. PRIMARY FUNDING SOURCE: U.S. Department of Veterans Affairs Health Services Research and Development Service

Advancing Health Equity by Avoiding Judgmentalism and Contextualizing Care.

This article examines the care of a Spanish-speaking woman with end-stage renal disease who returns repeatedly to the emergency department with complications related to missing hemodialysis. Her life circumstances suggest that she has been making difficult but rational decisions in an untenable situation, which is then readily resolved with the assistance of her care team. The case illustrates the pernicious effect of judgmentalism on patients from poor and marginalized communities, which exacerbates health inequity and illuminates the ethical importance of contextualizing patients' care.

Development of an unannounced standardized patient protocol to evaluate opioid use disorder treatment in pregnancy for American Indian and rural communities.

BACKGROUND: Opioid use disorder (OUD) disproportionately impacts rural and American Indian communities and has quadrupled among pregnant individuals nationwide in the past two decades. Yet, limited data are available about access and quality of care available to pregnant individuals in rural areas, particularly among American Indians (AIs). Unannounced standardized patients (USPs), or "secret shoppers" with standardized characteristics, have been used to assess healthcare access and quality when outcomes cannot be measured by conventional methods or when differences may exist between actual versus reported care. While the USP approach has shown benefit in evaluating primary care and select specialties, its use to date for OUD and pregnancy is very limited. METHODS: We used literature review, current practice guidelines for perinatal OUD management, and stakeholder engagement to design a novel USP protocol to assess healthcare access and quality for OUD in pregnancy. We developed two USP profiles-one white and one AI-to reflect our target study area consisting of three rural, predominantly white and AI US counties. We partnered with a local community health center network providing care to a large AI population to define six priority outcomes for evaluation: (1) OUD treatment knowledge among clinical staff answering telephones; (2) primary care clinic facilitation and provision of prenatal care and buprenorphine treatment; (3) appropriate completion of evidence-based screening, symptom assessment, and initial steps in management; (4) appropriate completion of risk factor screening/probing about individual circumstances that may affect care; (5) patient-directed tone, stigma, and professionalism by clinic staff; and (6) disparities in care between whites and American Indians. DISCUSSION: The development of this USP protocol tailored to a specific environment and high-risk patient population establishes an innovative approach to evaluate healthcare access and quality for pregnant individuals with OUD. It is intended to serve as a roadmap for our own study and for future related work within the context of substance use disorders and pregnancy.

Patients' perceptions of screening for health literacy: reactions to the newest vital sign.

Difficulties in caring for patients with limited health literacy have prompted interest in health literacy screening. Several prior studies, however, have suggested that health literacy testing can lead to feelings of shame and stigmatization. In this study, we examine patient reaction to the Newest Vital Sign (NVS), a screening instrument developed specifically for use in primary care. Data were collected in 2008 in the Morehouse School of Medicine, Department of Family Medicine Primary Care Clinics, where health literacy screening was implemented as part of routine intake procedures. Following the visit, patients completed a series of questions assessing their screening experiences. A total of 179 patients completed both the NVS and the reaction survey. Nearly all (> 99%) patients reported that the screening did not cause them to feel shameful. There were also no differences in the reported prevalence of shame (p

An educational intervention for contextualizing patient care and medical students' abilities to probe for contextual issues in simulated patients.

CONTEXT: A contextual error occurs when a physician does not identify elements of a patient's environment or behavior, such as access to care, that must be addressed to appropriately plan care. Research has demonstrated that contextual errors can be identified using standardized patients. OBJECTIVE: To evaluate an educational intervention designed to increase physicians' skills in incorporating the patient's context in assessment and management of care and to thereby decrease the rate of contextual errors. DESIGN, SETTING, AND PARTICIPANTS: Quasi-randomized controlled trial, with assessments by blinded observers. Fourth-year medical students (n = 124) in internal medicine subinternships at the University of Illinois at Chicago or Jesse Brown Veterans Administration Medical Center between July 2008 and April 2009 and between August 2009 and April 2010 participated and were assessed. INTERVENTION: A 4-hour course on contextualization. MAIN OUTCOME MEASURES: Probing for contextual issues in an encounter, probing for medical issues in an encounter, and developing an appropriate treatment plan. Outcomes were assessed using 4 previously validated standardized patient encounters performed by each participant and were adjusted for subinternship site, academic year, time of year, and case scenario. RESULTS: Students who participated in the contextualization workshops were significantly more likely to probe for contextual issues in the standardized patient encounters than students who did not (90% [95% confidence interval {CI}, 87%-94% ] vs 62% [95% CI, 54%-69%], respectively) and significantly more likely to develop appropriate treatment plans for standardized patients with contextual issues (69% [95% CI, 57%-81%] vs 22% [95% CI, 12%-32%]. There was no difference between the groups in the rate of probing for medical issues (80% [95% CI, 75%-85%] vs 81% [95% CI, 76%-86%]) or developing appropriate treatment plans for standardized patients with medical issues (54% [95% CI, 42%-67%] vs 66% [95% CI, 53%-79%]). CONCLUSION: Medical students who underwent an educational intervention were more likely to contextualize care for individual standardized patients.

Direct observation of depression screening: identifying diagnostic error and improving accuracy through unannounced standardized patients.

Background Depression is substantially underdiagnosed in primary care, despite recommendations for screening at every visit. We report a secondary analysis focused on depression of a recently completed study using unannounced standardized patients (USPs) to measure and improve provider behaviors, documentation, and subsequent claims for real patients. Methods Unannounced standardized patients presented incognito in 217 visits to 59 primary care providers in 22 New Jersey practices. We collected USP checklists, visit audio recordings, and provider notes after visits; provided feedback to practices and providers based on the first two visits per provider; and compared care and documentation behaviors in the visits before and after feedback. We obtained real patient claims from the study practices and a matched comparison group and compared the likelihood of visits including International Classification of Diseases, 10th Revision (ICD-10) codes for depression before and after feedback between the study and comparison groups. Results Providers significantly improved in their rate of depression screening following feedback [adjusted odds ratio (AOR), 3.41; 95% confidence interval (CI), 1.52-7.65; p = 0.003]. Sometimes expected behaviors were documented when not performed. The proportion of claims by actual patients with depression-related ICD-10 codes increased significantly more from prefeedback to postfeedback in the study group than in matched control group (interaction AOR, 1.41; 95% CI, 1.32-1.50; p < 0.001). Conclusions Using USPs, we found significant performance issues in diagnosis of depression, as well as discrepancies in documentation that may reduce future diagnostic accuracy. Providing feedback based on a small number of USP encounters led to some improvements in clinical performance observed both directly and indirectly via claims.

How accurate is the medical record? A comparison of the physician's note with a concealed audio recording in unannounced standardized patient encounters.

OBJECTIVES: Accurate documentation in the medical record is essential for quality care; extensive documentation is required for reimbursement. At times, these 2 imperatives conflict. We explored the concordance of information documented in the medical record with a gold standard measure. MATERIALS AND METHODS: We compared 105 encounter notes to audio recordings covertly collected by unannounced standardized patients from 36 physicians, to identify discrepancies and estimate the reimbursement implications of billing the visit based on the note vs the care actually delivered. RESULTS: There were 636 documentation errors, including 181 charted findings that did not take place, and 455 findings that were not charted. Ninety percent of notes contained at least 1 error. In 21 instances, the note justified a higher billing level than the gold standard audio recording, and in 4, it underrepresented the level of service (P = .005), resulting in 40 level 4 notes instead of the 23 justified based on the audio, a 74% inflated misrepresentation. DISCUSSION: While one cannot generalize about specific error rates based on a relatively small sample of physicians exclusively within the Department of Veterans Affairs Health System, the magnitude of the findings raise fundamental concerns about the integrity of the current medical record documentation process as an actual representation of care, with implications for determining both quality and resource utilization. CONCLUSION: The medical record should not be assumed to reflect care delivered. Furthermore, errors of commission-documentation of services not actually provided-may inflate estimates of resource utilization.