Translating ethical and quality principles for the effective, safe and fair development, deployment and use of artificial intelligence technologies in healthcare.

OBJECTIVE: The complexity and rapid pace of development of algorithmic technologies pose challenges for their regulation and oversight in healthcare settings. We sought to improve our institution's approach to evaluation and governance of algorithmic technologies used in clinical care and operations by creating an Implementation Guide that standardizes evaluation criteria so that local oversight is performed in an objective fashion. MATERIALS AND METHODS: Building on a framework that applies key ethical and quality principles (clinical value and safety, fairness and equity, usability and adoption, transparency and accountability, and regulatory compliance), we created concrete guidelines for evaluating algorithmic technologies at our institution. RESULTS: An Implementation Guide articulates evaluation criteria used during review of algorithmic technologies and details what evidence supports the implementation of ethical and quality principles for trustworthy health AI. Application of the processes described in the Implementation Guide can lead to algorithms that are safer as well as more effective, fair, and equitable upon implementation, as illustrated through 4 examples of technologies at different phases of the algorithmic lifecycle that underwent evaluation at our academic medical center. DISCUSSION: By providing clear descriptions/definitions of evaluation criteria and embedding them within standardized processes, we streamlined oversight processes and educated communities using and developing algorithmic technologies within our institution. CONCLUSIONS: We developed a scalable, adaptable framework for translating principles into evaluation criteria and specific requirements that support trustworthy implementation of algorithmic technologies in patient care and healthcare operations.

Effect of bar-code technology on the safety of medication administration.

BACKGROUND: Serious medication errors are common in hospitals and often occur during order transcription or administration of medication. To help prevent such errors, technology has been developed to verify medications by incorporating bar-code verification technology within an electronic medication-administration system (bar-code eMAR). METHODS: We conducted a before-and-after, quasi-experimental study in an academic medical center that was implementing the bar-code eMAR. We assessed rates of errors in order transcription and medication administration on units before and after implementation of the bar-code eMAR. Errors that involved early or late administration of medications were classified as timing errors and all others as nontiming errors. Two clinicians reviewed the errors to determine their potential to harm patients and classified those that could be harmful as potential adverse drug events. RESULTS: We observed 14,041 medication administrations and reviewed 3082 order transcriptions. Observers noted 776 nontiming errors in medication administration on units that did not use the bar-code eMAR (an 11.5% error rate) versus 495 such errors on units that did use it (a 6.8% error rate)--a 41.4% relative reduction in errors (P<0.001). The rate of potential adverse drug events (other than those associated with timing errors) fell from 3.1% without the use of the bar-code eMAR to 1.6% with its use, representing a 50.8% relative reduction (P<0.001). The rate of timing errors in medication administration fell by 27.3% (P<0.001), but the rate of potential adverse drug events associated with timing errors did not change significantly. Transcription errors occurred at a rate of 6.1% on units that did not use the bar-code eMAR but were completely eliminated on units that did use it. CONCLUSIONS: Use of the bar-code eMAR substantially reduced the rate of errors in order transcription and in medication administration as well as potential adverse drug events, although it did not eliminate such errors. Our data show that the bar-code eMAR is an important intervention to improve medication safety. (ClinicalTrials.gov number, NCT00243373.)

Impact of automated alerts on follow-up of post-discharge microbiology results: a cluster randomized controlled trial.

BACKGROUND: Failure to follow up microbiology results pending at the time of hospital discharge can delay diagnosis and treatment of important infections, harm patients, and increase the risk of litigation. Current systems to track pending tests are often inadequate. OBJECTIVE: To design, implement, and evaluate an automated system to improve follow-up of microbiology results that return after hospitalized patients are discharged. DESIGN: Cluster randomized controlled trial. SUBJECTS: Inpatient and outpatient physicians caring for adult patients hospitalized at a large academic hospital from February 2009 to June 2010 with positive and untreated or undertreated blood, urine, sputum, or cerebral spinal fluid cultures returning post-discharge. INTERVENTION: An automated e-mail-based system alerting inpatient and outpatient physicians to positive post-discharge culture results not adequately treated with an antibiotic at the time of discharge. MAIN MEASURES: Our primary outcome was documented follow-up of results within 3 days. Secondary outcomes included physician awareness and assessment of result urgency, impact on clinical assessments and plans, and preferred alerting scenarios. KEY RESULTS: We evaluated the follow-up of 157 post-discharge microbiology results from patients of 121 physicians. We found documented follow-up in 27/97 (28%) results in the intervention group and 8/60 (13%) in the control group [aOR 3.2, (95% CI 1.3-8.4); p=0.01]. Of all inpatient physician respondents, 32/82 (39%) were previously aware of the results, 45/77 (58%) felt the results changed their assessments and plans, 43/77 (56%) felt the results required urgent action, and 67/70 (96%) preferred alerts for current or broader scenarios. CONCLUSION: Our alerting system improved the proportion of important post-discharge microbiology results with documented follow-up, though the proportion remained low. The alerts were well received and may be expanded in the future.

Cognitive errors and logistical breakdowns contributing to missed and delayed diagnoses of breast and colorectal cancers: a process analysis of closed malpractice claims.

PURPOSE: To erform a process analysis of missed and delayed diagnoses of breast and colorectal cancers to identify: (1) the cognitive and logistical factors that lead to these diagnostic errors, and (2) prevention strategies. METHODS: Using 56 cases (43 breast, 13 colon) of missed and delayed diagnosis, we performed structured analyses to identify specific points in the diagnostic process in which errors occurred. Each error was classified as either a cognitive error or logistical breakdown. Finally, two physician-investigators identified strategies to prevent the errors in each case. RESULTS: Virtually all cases involved one or more cognitive errors (53/56, 95 %) and approximately half (31/56, 55 %) involved logistical breakdowns. The clinical activity most prone to cognitive error was the selection of the diagnostic strategy, both during the office visit (25/56, 45 %) and during interpretation of test results (22/50, 44 %). Arrangement of follow-up visits with a primary care physician (8/29, 28 %) or specialist physician (7/29, 26 %) were especially prone to logistical breakdowns. Adherence to current clinical guidelines could have prevented at least one error in 66 % of cases and assistance from a patient advocate could have prevented at least one error in 48 % of cases. CONCLUSIONS: Cognitive errors and logistical breakdowns are common among missed and delayed diagnoses of breast and colorectal cancers. Prevention strategies should focus on ensuring improving the effectiveness and use of clinical guidelines in the selection of diagnostic strategy, both during office visits and when interpreting test results. Tools to facilitate communication and to ensure that follow-up visits occur should also be considered.

Randomized controlled trial of health maintenance reminders provided directly to patients through an electronic PHR.

BACKGROUND: Provider and patient reminders can be effective in increasing rates of preventive screenings and vaccinations. However, the effect of patient-directed electronic reminders is understudied. OBJECTIVE: To determine whether providing reminders directly to patients via an electronic Personal Health Record (PHR) improved adherence to care recommendations. DESIGN: We conducted a cluster randomized trial without blinding from 2005 to 2007 at 11 primary care practices in the Partners HealthCare system. PARTICIPANTS: A total of 21,533 patients with access to a PHR were invited to the study, and 3,979 (18.5%) consented to enroll. INTERVENTIONS: Patients in the intervention arm received health maintenance (HM) reminders via a secure PHR "eJournal," which allowed them to review and update HM and family history information. Patients in the active control arm received access to an eJournal that allowed them to input and review information related to medications, allergies and diabetes management. MAIN MEASURES: The primary outcome measure was adherence to guideline-based care recommendations. KEY RESULTS: Intention-to-treat analysis showed that patients in the intervention arm were significantly more likely to receive mammography (48.6% vs 29.5%, p = 0.006) and influenza vaccinations (22.0% vs 14.0%, p = 0.018). No significant improvement was observed in rates of other screenings. Although Pap smear completion rates were higher in the intervention arm (41.0% vs 10.4%, p < 0.001), this finding was no longer significant after excluding women's health clinics. Additional on-treatment analysis showed significant increases in mammography (p = 0.019) and influenza vaccination (p = 0.015) for intervention arm patients who opened an eJournal compared to control arm patients, but no differences for any measure among patients who did not open an eJournal. CONCLUSIONS: Providing patients with HM reminders via a PHR may be effective in improving some elements of preventive care.

Curating a knowledge base for individuals with coinfection of HIV and SARS-CoV-2: a study protocol of EHR-based data mining and clinical implementation.

INTRODUCTION: Despite a higher risk of severe COVID-19 disease in individuals with HIV, the interactions between SARS-CoV-2 and HIV infections remain unclear. To delineate these interactions, multicentre Electronic Health Records (EHR) hold existing promise to provide full-spectrum and longitudinal clinical data, demographics and sociobehavioural data at individual level. Presently, a comprehensive EHR-based cohort for the HIV/SARS-CoV-2 coinfection has not been established; EHR integration and data mining methods tailored for studying the coinfection are urgently needed yet remain underdeveloped. METHODS AND ANALYSIS: The overarching goal of this exploratory/developmental study is to establish an EHR-based cohort for individuals with HIV/SARS-CoV-2 coinfection and perform large-scale EHR-based data mining to examine the interactions between HIV and SARS-CoV-2 infections and systematically identify and validate factors contributing to the severe clinical course of the coinfection. We will use a nationwide EHR database in the USA, namely, National COVID Cohort Collaborative (N3C). Ultimately, collected clinical evidence will be implemented and used to pilot test a clinical decision support prototype to assist providers in screening and referral of at-risk patients in real-world clinics. ETHICS AND DISSEMINATION: The study was approved by the institutional review boards at the University of South Carolina (Pro00121828) as non-human subject study. Study findings will be presented at academic conferences and published in peer-reviewed journals. This study will disseminate urgently needed clinical evidence for guiding clinical practice for individuals with the coinfection at Prisma Health, a healthcare system in collaboration.

A framework for the oversight and local deployment of safe and high-quality prediction models.

Artificial intelligence/machine learning models are being rapidly developed and used in clinical practice. However, many models are deployed without a clear understanding of clinical or operational impact and frequently lack monitoring plans that can detect potential safety signals. There is a lack of consensus in establishing governance to deploy, pilot, and monitor algorithms within operational healthcare delivery workflows. Here, we describe a governance framework that combines current regulatory best practices and lifecycle management of predictive models being used for clinical care. Since January 2021, we have successfully added models to our governance portfolio and are currently managing 52 models.

Relationship between use of electronic health record features and health care quality: results of a statewide survey.

BACKGROUND: Electronic health records (EHRs) are widely viewed as useful tools for supporting the provision of high quality healthcare. However, evidence regarding their effectiveness for this purpose is mixed, and existing studies have generally considered EHR usage a binary factor and have not considered the availability and use of specific EHR features. OBJECTIVE: To assess the relationship between the use of an EHR and the use of specific EHR features with quality of care. RESEARCH DESIGN: A statewide mail survey of physicians in Massachusetts conducted in 2005. The results of the survey were linked with Healthcare Effectiveness Data and Information Set (HEDIS) quality measures, and generalized linear regression models were estimated to examine the associations between the use of EHRs and specific EHR features with quality measures, adjusting for physician practice characteristics. SUBJECTS: A stratified random sample of 1884 licensed physicians in Massachusetts, 1345 of whom responded. Of these, 507 had HEDIS measures available and were included in the analysis (measures are only available for primary care providers). MEASURE: Performance on HEDIS quality measures. RESULTS: The survey had a response rate of 71%. There was no statistically significant association between use of an EHR as a binary factor and performance on any of the HEDIS measure groups. However, there were statistically significant associations between the use of many, but not all, specific EHR features and HEDIS measure group scores. The associations were strongest for the problem list, visit note and radiology test result EHR features and for quality measures relating to women's health, colon cancer screening, and cancer prevention. For example, users of problem list functionality performed better on women's health, depression, colon cancer screening, and cancer prevention measures, with problem list users outperforming nonusers by 3.3% to 9.6% points on HEDIS measure group scores (all significant at the P < 0.05 level). However, these associations were not universal. CONCLUSIONS: Consistent with past studies, there was no significant relationship between use of EHR as a binary factor and performance on quality measures. However, availability and use of specific EHR features by primary care physicians was associated with higher performance on certain quality measures. These results suggest that, to maximize health care quality, developers, implementers and certifiers of EHRs should focus on increasing the adoption of robust EHR systems and increasing the use of specific features rather than simply aiming to deploy an EHR regardless of functionality.

Overcoming barriers to the adoption and implementation of predictive modeling and machine learning in clinical care: what can we learn from US academic medical centers?

There is little known about how academic medical centers (AMCs) in the US develop, implement, and maintain predictive modeling and machine learning (PM and ML) models. We conducted semi-structured interviews with leaders from AMCs to assess their use of PM and ML in clinical care, understand associated challenges, and determine recommended best practices. Each transcribed interview was iteratively coded and reconciled by a minimum of 2 investigators to identify key barriers to and facilitators of PM and ML adoption and implementation in clinical care. Interviews were conducted with 33 individuals from 19 AMCs nationally. AMCs varied greatly in the use of PM and ML within clinical care, from some just beginning to explore their utility to others with multiple models integrated into clinical care. Informants identified 5 key barriers to the adoption and implementation of PM and ML in clinical care: (1) culture and personnel, (2) clinical utility of the PM and ML tool, (3) financing, (4) technology, and (5) data. Recommendation to the informatics community to overcome these barriers included: (1) development of robust evaluation methodologies, (2) partnership with vendors, and (3) development and dissemination of best practices. For institutions developing clinical PM and ML applications, they are advised to: (1) develop appropriate governance, (2) strengthen data access, integrity, and provenance, and (3) adhere to the 5 rights of clinical decision support. This article highlights key challenges of implementing PM and ML in clinical care at AMCs and suggests best practices for development, implementation, and maintenance at these institutions.

Implementation and Continuous Monitoring of an Electronic Health Record Embedded Readmissions Clinical Decision Support Tool.

Unplanned hospital readmissions represent a significant health care value problem with high costs and poor quality of care. A significant percentage of readmissions could be prevented if clinical inpatient teams were better able to predict which patients were at higher risk for readmission. Many of the current clinical decision support models that predict readmissions are not configured to integrate closely with the electronic health record or alert providers in real-time prior to discharge about a patient's risk for readmission. We report on the implementation and monitoring of the Epic electronic health record-"Unplanned readmission model version 1"-over 2 years from 1/1/2018-12/31/2019. For patients discharged during this time, the predictive capability to discern high risk discharges was reflected in an AUC/C-statistic at our three hospitals of 0.716-0.760 for all patients and 0.676-0.695 for general medicine patients. The model had a positive predictive value ranging from 0.217-0.248 for all patients. We also present our methods in monitoring the model over time for trend changes, as well as common readmissions reduction strategies triggered by the score.

Telehealth transformation: COVID-19 and the rise of virtual care.

The novel coronavirus disease-19 (COVID-19) pandemic has altered our economy, society, and healthcare system. While this crisis has presented the U.S. healthcare delivery system with unprecedented challenges, the pandemic has catalyzed rapid adoption of telehealth, or the entire spectrum of activities used to deliver care at a distance. Using examples reported by U.S. healthcare organizations, including ours, we describe the role that telehealth has played in transforming healthcare delivery during the 3 phases of the U.S. COVID-19 pandemic: (1) stay-at-home outpatient care, (2) initial COVID-19 hospital surge, and (3) postpandemic recovery. Within each of these 3 phases, we examine how people, process, and technology work together to support a successful telehealth transformation. Whether healthcare enterprises are ready or not, the new reality is that virtual care has arrived.

Identified themes of interactive visualizations overlayed onto EHR data: an example of improving birth center operating room efficiency.

OBJECTIVE: While electronic health record (EHR) systems store copious amounts of patient data, aggregating those data across patients can be challenging. Visual analytic tools that integrate with EHR systems allow clinicians to gain better insight and understanding into clinical care and management. We report on our experience building Tableau-based visualizations and integrating them into our EHR system. MATERIALS AND METHODS: Visual analytic tools were created as part of 12 clinician-initiated quality improvement projects. We built the visual analytic tools in Tableau and linked it within our EPIC environment. We identified 5 visual themes that spanned the various projects. To illustrate these themes, we choose 1 exemplary project which aimed to improve obstetric operating room efficiency. RESULTS: Across our 12 projects, we identified 5 visual themes that are integral to project success: scheduling & optimization (in 11/12 projects); provider assessment (10/12); executive assessment (8/12); patient outcomes (7/12); and control and goal charts (2/12). DISCUSSION: Many visualizations share common themes. Identification of these themes has allowed our internal team to be more efficient and directed in developing visualizations for future projects. CONCLUSION: Organizing visual analytics into themes can allow informatics teams to more efficiently provide visual products to clinical collaborators.

Trends in primary care clinician perceptions of a new electronic health record.

BACKGROUND: Clinician perceptions of a newly implemented electronic health record play an important role in its success or failure. OBJECTIVE: To measure changes in primary care clinician attitudes toward an electronic health record during the first year following implementation. DESIGN: Longitudinal survey. PARTICIPANTS: 86 primary care clinicians surveyed between December 2006 and January 2008. MEASUREMENTS: Perceived impact on overall quality of care, patient safety, communication, and efficiency at 1, 3, 6, and 12 months following implementation. RESULTS: Response rates for months 1, 3, 6, and 12 were 92%, 95%, 90%, and 82%, respectively. The proportion of clinicians agreeing that the EHR improved the overall quality of care (63% to 86%; p < 0.001), reduced medication-related errors (72% to 81%; p = 0.03), improved follow-up of test results (62% to 87%; p < 0.001), and improved communication among clinicians (72% to 93%; p < 0.001) increased from month 1 to month 12. During the same time period, a decreasing proportion of clinicians agreed that the EHR reduced the quality of patient interactions (49% to 33%; p = 0.001), resulted in longer patient visits (68% to 51%; p = 0.001), and increased time spent on medical documentation (78% to 68%; p = 0.006). Significant improvements in perceptions related to test result follow-up were first detected at 6 months, while those related to overall quality, efficiency, and communication were first identified at 12 months. CONCLUSIONS: Primary care clinicians report increasingly positive perceptions of a new electronic health record within 1 year of implementation across a spectrum of domains of care.

Physicians' use of key functions in electronic health records from 2005 to 2007: a statewide survey.

OBJECTIVE Electronic health records (EHRs) have potential to improve quality and safety, but many physicians do not use these systems to full capacity. The objective of this study was to determine whether this usage gap is narrowing over time. DESIGN Follow-up mail survey of 1,144 physicians in Massachusetts who completed a 2005 survey. MEASUREMENTS Adoption of EHRs and availability and use of 10 EHR functions. RESULTS The response rate was 79.4%. In 2007, 35% of practices had EHRs, up from 23% in 2005. Among practices with EHRs, there was little change between 2005 and 2007 in the availability of nine of ten EHR features; the notable exception was electronic prescribing, reported as available in 44.7% of practices with EHRs in 2005 and 70.8% in 2007. Use of EHR functions changed inconsequentially, with more than one out of five physicians not using each available function regularly in both 2005 and 2007. Only electronic prescribing increased substantially: in 2005, 19.9% of physicians with this function available used it most or all the time, compared with 42.6% in 2007 (p < 0.001). CONCLUSIONS By 2007, more than one third of practices in Massachusetts reported having EHRs; the availability and use of electronic prescribing within these systems has increased. In contrast, physicians reported little change in the availability and use of other EHR functions. System refinements, certification efforts, and health policies, including standards development, should address the gaps in both EHR adoption and the use of key functions.

Ability to generate patient registries among practices with and without electronic health records.

BACKGROUND: The ability to generate registries of patients with particular clinical attributes, such as diagnoses or medications taken, is central to measuring and improving the quality of health care. However, it is not known how many providers have the ability to generate such registries. OBJECTIVES: To assess the proportion of physician practices that can construct registries of patients with specific diagnoses, laboratory results, or medications, and to determine the relationship between electronic health record (EHR) usage and the ability to perform registry functions. METHODS: We conducted a mail survey of a stratified random sample of physician practices in Massachusetts in the northeastern United States (N = 1884). The survey included questions about the physicians' ability to generate diagnosis, laboratory result, and medication registries; the presence of EHR; and usage of specific EHR features. RESULTS: The response rate was 71% (1345/1884). Overall, 79.8% of physician practices reported being able to generate registries of patients by diagnosis; 56.1% by laboratory result; and 55.8% by medication usage. In logistic regression analyses, adjusting for urban/rural location, practice size and ownership, teaching status, hospital affiliation, and specialty, physician practices with an EHR were more likely to be able to construct diagnosis registries (adjusted odds ratio [OR] 1.53, 95% confidence interval [CI] 1.25 - 1.86), laboratory registries (OR 1.42, 95% CI 1.22 - 1.66), and medication registries (OR 2.30, 95% CI 1.96 - 2.70). CONCLUSIONS: Many physician practices were able to generate registries, but this capability is far from universal. Adoption of EHRs appears to be a useful step toward this end, and practices with EHRs are considerably more likely to be able to carry out registry functions. Because practices need registries to perform broad-based quality improvement, they should consider adopting EHRs that have built-in registry functionality.

Formative evaluation: a critical component in EHR implementation.

This Viewpoint paper has grown out of a presentation at the American College of Medical Informatics 2007 Winter Symposium, the resulting discussion, and several activities that have coalesced around an issue that most informaticians accept as true but is not commonly considered during the implementation of Electronic Health Records (EHR) outside of academia or research institutions. Successful EHR implementation is facilitated and sometimes determined by formative evaluation, usually focusing on process rather than outcomes. With greater federal funding for the implementation of electronic health record systems in health care organizations unfamiliar with research protocols, the need for formative evaluation assistance is growing. Such assistance, in the form of tools and protocols necessary to do formative evaluation and resulting in successful EHR implementations, should be provided by practicing medical informaticians.

Physicians and electronic health records: a statewide survey.

BACKGROUND: Electronic health records (EHRs) allow for a variety of functions, ranging from visit documentation to laboratory test ordering, but little is known about physicians' actual use of these functions. METHODS: We surveyed a random sample of 1884 physicians in Massachusetts by mail and assessed availability and use of EHR functions, predictors of use, and the relationships between EHR use and physicians' perceptions of medical practice. RESULTS: A total of 1345 physicians responded to the survey (71.4% response rate), and 387 (28.8%) reported that their practice had adopted EHRs. More than 80% of physicians with EHRs reported having the ability to view laboratory reports (84.8%) and document visits electronically (84.0%), but considerably fewer reported being able to order laboratory tests electronically (46.8%) or transmit prescriptions to a pharmacy electronically (44.7%). Fewer than half of the physicians who had systems with clinical decision support, transmittal of electronic prescriptions, and radiology order entry actually used these functions most or all of the time. Compared with physicians who had not adopted EHRs, EHR users reported more positive views of the effects of computers on health care; there were no significant differences in these attitudes between high and low users of EHRs. Overall, about 1 in 4 physicians reported dissatisfaction with medical practice; there was no difference in this measure by EHR adoption or use. CONCLUSIONS: There is considerable variability in the functions available in EHRs and in the extent to which physicians use them. Future work should emphasize factors that affect the use of available functions.

Cost-benefit analysis of a hospital pharmacy bar code solution.

BACKGROUND: Bar coding can reduce hospital pharmacy dispensing errors, but it is unclear if the benefits of this technology justify its costs. The purpose of this study was to assess the costs and benefits and determine the return on investment at the institutional level for implementing a pharmacy bar code system. METHODS: We performed a cost-benefit analysis of a bar code-assisted medication-dispensing system within a large, academic, nonprofit tertiary care hospital pharmacy. We took the implementing hospital's perspective for a 5-year horizon. The primary outcome was the net financial cost and benefit after 5 years. The secondary outcome was the time until total benefits equaled total costs. Single-variable, 2-variable, and multiple-variable Monte Carlo sensitivity analyses were performed to test the stability of the outcomes. RESULTS: In inflation- and time value-adjusted 2005 dollars, total costs during 5 years were $2.24 million ($1.31 million in 1-time costs during the initial 3.5 years and $342 000 per year in recurring costs starting in year 3). The primary benefit was a decrease in adverse drug events from dispensing errors (517 events annually), resulting in an annual savings of $2.20 million. The net benefit after 5 years was $3.49 million. The break-even point for the hospital's investment occurred within 1 year after becoming fully operational. A net benefit was achieved within 10 years under almost all sensitivity scenarios. In the Monte Carlo simulation, the net benefit during 5 years was $3.2 million (95% confidence interval, -$1.2 million to $12.1 million), and the break-even point for return on investment occurred after 51 months (95% confidence interval, 30 to 180 months). CONCLUSION: Implementation of a bar code-assisted medication-dispensing system in hospital pharmacies can result in a positive financial return on investment for the health care organization.

Impact of an automated test results management system on patients' satisfaction about test result communication.

BACKGROUND: Few reliable and efficient systems support the communication of test results to outpatients, and this may lead to patient dissatisfaction with test result communication. The objective of this study was to assess the impact of physicians' use of a test results management tool embedded in an electronic health record on patient satisfaction with test result communication. METHODS: A prospective, cluster-randomized, controlled trial of 570 patient encounters in 26 outpatient primary care practices was performed from December 1, 2002, to April 31, 2005. Physicians in the intervention practices were trained and given access to a physician test results management tool with imbedded patient notification functions to evaluate whether patient satisfaction with communication of test results ordered by the primary care provider was improved. Patient satisfaction surveys were conducted by telephone after the patient underwent the test and were administered before and after the intervention in both arms. RESULTS: The survey response rate after successful patient contact was 74.2% (570/768). After adjusting for patient age, sex, race, socioeconomic status, and insurance type, the intervention significantly increased patient satisfaction with test results communication (odds ratio, 2.35; 95% confidence interval, 1.05-5.25; P = .03). In addition, patients in the postintervention group were more satisfied with information given them for medical treatments and conditions regarding their results (odds ratio, 3.45; 95% confidence interval, 1.30-9.17; P = .02). CONCLUSION: An automated test results management system can improve patient satisfaction with communication of test results ordered by their primary care provider and can improve patient satisfaction with the communication of information regarding their condition and treatment plans.

Readiness for electronic health records: comparison of characteristics of practices in a collaborative with the remainder of Massachusetts.

OBJECTIVE: The Massachusetts e-Health Collaborative (MAeHC) is implementing electronic health records (EHRs) in physicians' offices throughout three diverse communities. This study's objective was to assess the degree to which these practices are representative of physicians' practices statewide. DESIGN: We surveyed all MAeHC physicians (n=464) and compared their responses to those of a contemporaneously surveyed statewide random sample (n=1884). MEASUREMENTS: The survey questionnaire assessed practice characteristics related to EHR adoption, prevailing office culture related to quality and safety, attitudes toward health information technology (HIT) and perceptions of medical practice. RESULTS: A total of 355 MAeHC physicians (77%) and 1345 physicians from the statewide sample (71%) completed the survey. MAeHC practices resembled practices throughout Massachusetts in terms of practice size, physician age and gender, prevailing financial incentives for quality performance and HIT adoption and available resources for practice expansion. MAeHC practices were more likely to be located in rural areas (9.5% vs 4.4%, P=0.004). Physicians in both samples responded similarly to six of seven self-assessments of the office practice environment for quality and safety. Internet connections were more prevalent among MAeHC practices than across the state (96% vs 83%, P<0.001), but similar proportions of MAeHC physicians (83%) and statewide physicians (86%) used the internet daily (P=0.19). CONCLUSION: MAeHC is implementing EHRs and health information exchange among communities with physicians and practices that appear generally representative of Massachusetts. The lessons learned from this pilot project should be applicable statewide and to other states with large numbers of physicians in small office practices.