Training the next generation of delivery science researchers: 10-year experience of a post-doctoral research fellowship program within an integrated care system.

Introduction: Learning health systems require a workforce of researchers trained in the methods of identifying and overcoming barriers to effective, evidence-based care. Most existing postdoctoral training programs, such as NIH-funded postdoctoral T32 awards, support basic and epidemiological science with very limited focus on rigorous delivery science methods for improving care. In this report, we present the 10-year experience of developing and implementing a Delivery Science postdoctoral fellowship embedded within an integrated health care delivery system. Methods: In 2012, the Kaiser Permanente Northern California Division of Research designed and implemented a 2-year postdoctoral Delivery Science Fellowship research training program to foster research expertise in identifying and addressing barriers to evidence-based care within health care delivery systems. Results: Since 2014, 20 fellows have completed the program. Ten fellows had PhD-level scientific training, and 10 fellows had clinical doctorates (eg, MD, RN/PhD, PharmD). Fellowship alumni have graduated to faculty research positions at academic institutions (9), and research or clinical organizations (4). Seven alumni now hold positions in Kaiser Permanente's clinical operations or medical group (7). Conclusions: This delivery science fellowship program has succeeded in training graduates to address delivery science problems from both research and operational perspectives. In the next 10 years, additional goals of the program will be to expand its reach (eg, by developing joint research training models in collaboration with clinical fellowships) and strengthen mechanisms to support transition from fellowship to the workforce, especially for researchers from underrepresented groups.

Changes in COVID-19 Vaccine Intent Among a Diverse Population of Older Adults, June 2021-February 2022.

Introduction COVID-19 vaccination rates remain suboptimal in the United States. Clinicians and policymakers need to better understand how likely vaccine-hesitant individuals are to ultimately accept vaccination and what is associated with such changes. This study's aims were to 1) describe changes between vaccine intentions and actual uptake from June 2021 through February 2022, and 2) identify modifiable factors associated with vaccine uptake among those with initial hesitancy. Methods This cohort study included a stratified random sample of adults aged 65 years and older in an integrated health care system. The survey, conducted June through August 2021, elicited intent and perceptions regarding COVID-19 vaccination. Subsequent vaccine uptake through February 2022 was analyzed using electronic health records. Results Of 1195 individuals surveyed, 66% responded; 213 reported not yet having received a COVID-19 vaccine and were further analyzed. At baseline, most individuals said they would definitely not (42%) or probably not (5%) get the COVID-19 vaccine or were not sure (26%). During follow-up, 61 individuals (29%) were vaccinated, including 19% of those who initially said they would definitely not be vaccinated. Among vaccine-hesitant individuals, the rate of vaccination was highest for those who initially considered COVID-19 less dangerous than the vaccine (46%) or named short-term side effects (36%) as their most important concern. Conclusions COVID-19 vaccine intent among older adults was malleable during the pandemic's second year, even among those who initially said they would definitely not be vaccinated. Vaccine uptake could be enhanced by increasing awareness of COVID-19 risks and by addressing vaccine side effects.

Strategies Facilitating Video Visit Implementation by a Medical Group Serving a Diverse Population.

IntroductionVideo visits have created new opportunities to enhance access to care, but limited information exists on strategies medical groups can employ to facilitate video visit use by higher-risk patients. Our objective was to identify generalizable strategies to facilitate successful delivery of video visits by systems serving highly diverse patient populations. MethodsThe authors conducted a qualitative study of physicians and staff members in a large group practice with 4.5 million patients with diverse race and ethnicity and socioeconomic status. Semi-structured interviews were conducted between January 2021 and April 2021, with key informants identified via purposive and snowball sampling. Video-recorded interviews were transcribed and analyzed using thematic analysis to identify major themes and subthemes. ResultsThe 42 key informants included regional and medical center leaders, primary care physicians, service managers, and medical assistants. Participants described clinical leadership in technology and multidisciplinary collaboration as crucial to sustained video care adoption. Strategies to facilitate real-time learning included local innovation, rapid communication channels, and psychological safety. The organization offered broad access to frequently updated data reports to help managers and practitioners understand processes, measure performance, and share best practices. Medical assistants and physicians developed new approaches to empathize, tailor interactions with patients, and overcome psychological and technical barriers to connecting via video. ConclusionsKey strategies for sustained video care adoption included clinical leadership articulating its purpose, multidisciplinary collaboration, local innovation, effective data use, empathy, and personalized care. These findings provide a model for how health care systems can foster robust adoption of technologies to serve diverse populations.

Effect of Electronic and Mail Outreach From Primary Care Physicians for COVID-19 Vaccination of Black and Latino Older Adults: A Randomized Clinical Trial.

Importance: COVID-19 morbidity is highest in Black and Latino older adults. These racial and ethnic groups initially had lower vaccination uptake than others, and rates in Black adults continue to lag. Objectives: To evaluate the effect of outreach via electronic secure messages and mailings from primary care physicians (PCPs) on COVID-19 vaccination uptake among Black and Latino older adults and to compare the effects of culturally tailored and standard PCP messages. Design, Setting, and Participants: This randomized clinical trial was conducted from March 29 to May 20, 2021, with follow-up surveys through July 31, 2021. Latino and Black individuals aged 65 years and older from 4 Kaiser Permanente Northern California (KPNC) service areas were included. Data were analyzed from May 27, 2021, to September 28, 2021. Interventions: Individuals who had not received COVID-19 vaccination after previous outreach were randomized to electronic secure message and/or mail outreach from their PCP, similar outreach with additional culturally tailored content, or usual care. Outreach groups were sent a secure message or letter in their PCP's name, followed by a postcard to those still unvaccinated after 4 weeks. Main Outcomes and Measures: The primary outcome was time to receipt of COVID-19 vaccination during the 8 weeks after initial study outreach. KPNC data were supplemented with state data from external sources. Intervention effects were evaluated via proportional hazards regression. Results: Of 8287 included individuals (mean [SD] age, 72.6 [7.0] years; 4665 [56.3%] women), 2434 (29.4%) were Black, 3782 (45.6%) were Latino and preferred English-language communications, and 2071 (25.0%) were Latino and preferred Spanish-language communications; 2847 participants (34.4%) had a neighborhood deprivation index at the 75th percentile or higher. A total of 2767 participants were randomized to culturally tailored PCP outreach, 2747 participants were randomized to standard PCP outreach, and 2773 participants were randomized to usual care. Culturally tailored PCP outreach led to higher COVID-19 vaccination rates during follow-up compared with usual care (664 participants [24.0%] vs 603 participants [21.7%]; adjusted hazard ratio (aHR), 1.22; 95% CI, 1.09-1.37), as did standard PCP outreach (635 participants [23.1%]; aHR, 1.17; 95% CI, 1.04-1.31). Individuals who were Black (aHR, 1.19; 95% CI, 1.06-1.33), had high neighborhood deprivation (aHR, 1.17; 95% CI, 1.03-1.33), and had medium to high comorbidity scores (aHR, 1.19; 95% CI, 1.09-1.31) were more likely to be vaccinated during follow-up. Conclusions and Relevance: This randomized clinical trial found that PCP outreach using electronic and mailed messages increased COVID-19 vaccination rates among Black and Latino older adults. Trial Registration: ClinicalTrials.gov Identifier: NCT05096026.

A prognostic information system for real-time personalized care: Lessons for embedded researchers.

Embedded researchers could play a central role in developing tools to personalize care using electronic medical records (EMRs). However, few studies have described the steps involved in developing such tools, or evaluated the key factors in success and failure. This case study describes how we used an EMR-derived data warehouse to develop a prototype informatics tool to help oncologists counsel patients with pancreatic cancer about their prognosis. The tool generated real-time prognostic information based on tumor type and stage, age, comorbidity status and lab tests. Our multidisciplinary team included embedded researchers, application developers, user experience experts, and an oncologist leader.This prototype succeeded in establishing proof of principle, but did not reach adoption into actual practice. In pilot testing, oncologists succeeded in generating prognostic information in real time. A few found it helpful in patient encounters, but all identified critical areas for further development before implementation. Generalizable lessons included the need to (1) include a wide range of potential use cases and stakeholders when selecting use cases for such tools; (2) develop talking points for clinicians to explain results from predictive tools to patients; (3) develop ways to reduce lag time between events and data availability; and (4) keep the options presented in the user interface very simple. This case demonstrates that embedded researchers can lead collaborations using EMR-derived data to create systems for real-time personalized patient counseling, and highlights challenges that such teams can anticipate.

Physician Stress During Electronic Health Record Inbox Work: In Situ Measurement With Wearable Sensors.

BACKGROUND: Increased work through electronic health record (EHR) messaging is frequently cited as a factor of physician burnout. However, studies to date have relied on anecdotal or self-reported measures, which limit the ability to match EHR use patterns with continuous stress patterns throughout the day. OBJECTIVE: The aim of this study is to collect EHR use and physiologic stress data through unobtrusive means that provide objective and continuous measures, cluster distinct patterns of EHR inbox work, identify physicians' daily physiologic stress patterns, and evaluate the association between EHR inbox work patterns and physician physiologic stress. METHODS: Physicians were recruited from 5 medical centers. Participants (N=47) were given wrist-worn devices (Garmin Vivosmart 3) with heart rate sensors to wear for 7 days. The devices measured physiological stress throughout the day based on heart rate variability (HRV). Perceived stress was also measured with self-reports through experience sampling and a one-time survey. From the EHR system logs, the time attributed to different activities was quantified. By using a clustering algorithm, distinct inbox work patterns were identified and their associated stress measures were compared. The effects of EHR use on physician stress were examined using a generalized linear mixed effects model. RESULTS: Physicians spent an average of 1.08 hours doing EHR inbox work out of an average total EHR time of 3.5 hours. Patient messages accounted for most of the inbox work time (mean 37%, SD 11%). A total of 3 patterns of inbox work emerged: inbox work mostly outside work hours, inbox work mostly during work hours, and inbox work extending after hours that were mostly contiguous to work hours. Across these 3 groups, physiologic stress patterns showed 3 periods in which stress increased: in the first hour of work, early in the afternoon, and in the evening. Physicians in group 1 had the longest average stress duration during work hours (80 out of 243 min of valid HRV data; P=.02), as measured by physiological sensors. Inbox work duration, the rate of EHR window switching (moving from one screen to another), the proportion of inbox work done outside of work hours, inbox work batching, and the day of the week were each independently associated with daily stress duration (marginal R2=15%). Individual-level random effects were significant and explained most of the variation in stress (conditional R2=98%). CONCLUSIONS: This study is among the first to demonstrate associations between electronic inbox work and physiological stress. We identified 3 potentially modifiable factors associated with stress: EHR window switching, inbox work duration, and inbox work outside work hours. Organizations seeking to reduce physician stress may consider system-based changes to reduce EHR window switching or inbox work duration or the incorporation of inbox management time into work hours.

Evaluation of Attention Switching and Duration of Electronic Inbox Work Among Primary Care Physicians.

Importance: Primary care physicians (PCPs) report multitasking during workdays while processing electronic inbox messages, but scant systematic information exists on attention switching and its correlates in the health care setting. Objectives: To describe PCPs' frequency of attention switching associated with electronic inbox work, identify potentially modifiable factors associated with attention switching and inbox work duration, and compare the relative association of attention switching and other factors with inbox work duration. Design, Setting, and Participants: This cross-sectional study of the work of 1275 PCPs in an integrated group serving 4.5 million patients used electronic health record (EHR) access logs from March 1 to 31, 2018, to evaluate PCPs' frequency of attention switching. Statistical analysis was performed from October 15, 2018, to August 28, 2020. Main Outcomes and Measures: Attention switching was defined as switching between the electronic inbox, other EHR work, and non-EHR periods. Inbox work duration included minutes spent on electronic inbox message views and related EHR tasks. Multivariable models controlled for the exposures. Results: The 1275 PCPs studied (721 women [56.5%]; mean [SD] age, 45.9 [8.5] years) had a mean (SD) of 9.0 (7.6) years of experience with the medical group and received a mean (SD) of 332.6 (148.3) (interquartile range, 252-418) new inbox messages weekly. On workdays, PCPs made a mean (SD) of 79.4 (21.8) attention switches associated with inbox work and did a mean (SD) 64.2 (18.7) minutes of inbox work over the course of 24 hours on workdays. In the model for attention switching, each additional patient secure message beyond the reference value was associated with 0.289 (95% CI, 0.217-0.362) additional switches, each additional results message was associated with 0.203 (95% CI, 0.127-0.278) additional switches, each additional request message was associated with 0.190 (95% CI, 0.124-0.257) additional switches, and each additional administrative message was associated with 0.262 (95% CI, 0.166-0.358) additional switches. Having a panel (a list of patients assigned to a primary care team) with more elderly patients (0.144 switches per percentage increase [95% CI, 0.009-0.278]) and higher inbox work duration (0.468 switches per additional minute of inbox work [95% CI, 0.411-0.524]) were also associated with higher attention switching involving the inbox. In the model for inbox work duration, each additional patient secure message beyond the reference value was associated with 0.151 (95% CI, 0.085-0.217) additional minutes, each additional results message was associated with 0.338 (95% CI, 0.272-0.404) additional minutes, each additional request message was associated with 0.101 (95% CI, 0.041-0.161) additional minutes, and each additional administrative message was associated with 0.179 (95% CI, 0.093-0.265) additional minutes. A higher percentage of the panel's patients initiating messages (0.386 minutes per percentage increase [95% CI, 0.026-0.745]) and attention switches (0.373 minutes per switch [95% CI, 0.328-0.419]) were also associated with higher inbox work duration. In addition, working at a medical center where all PCPs had high inbox work duration was independently associated with high or low inbox work duration. Conclusions and Relevance: This study suggests that PCPs make frequent attention switches during workdays while processing electronic inbox messages. Message quantity was associated with both attention switching and inbox work duration. Physician and patient panel characteristics had less association with attention switching and inbox work duration. Assisting PCPs with message quantity might help modulate both attention switching and inbox work duration.

Physicians' electronic inbox work patterns and factors associated with high inbox work duration.

OBJECTIVES: Electronic health record systems are increasingly used to send messages to physicians, but research on physicians' inbox use patterns is limited. This study's aims were to (1) quantify the time primary care physicians (PCPs) spend managing inboxes; (2) describe daily patterns of inbox use; (3) investigate which types of messages consume the most time; and (4) identify factors associated with inbox work duration. MATERIALS AND METHODS: We analyzed 1 month of electronic inbox data for 1275 PCPs in a large medical group and linked these data with physicians' demographic data. RESULTS: PCPs spent an average of 52 minutes on inbox management on workdays, including 19 minutes (37%) outside work hours. Temporal patterns of electronic inbox use differed from other EHR functions such as charting. Patient-initiated messages (28%) and results (29%) accounted for the most inbox work time. PCPs with higher inbox work duration were more likely to be female (P < .001), have more patient encounters (P < .001), have older patients (P < .001), spend proportionally more time on patient messages (P < .001), and spend more time per message (P < .001). Compared with PCPs with the lowest duration of time on inbox work, PCPs with the highest duration had more message views per workday (200 vs 109; P < .001) and spent more time on the inbox outside work hours (30 minutes vs 9.7 minutes; P < .001). CONCLUSIONS: Electronic inbox work by PCPs requires roughly an hour per workday, much of which occurs outside scheduled work hours. Interventions to assist PCPs in handling patient-initiated messages and results may help alleviate inbox workload.

Rethinking ethical oversight in the era of the learning health system.

Opportunities to advance science increasingly arise through investigations embedded within routine clinical practice in the form of learning health systems. Such activities challenge conventional approaches to research regulation that have not caught up with those opportunities, often imposing burdens generalized from riskier research. We analyze the rules and conventions in the US, demonstrating how even those rules are compatible with a much more flexible approach to participant risk, institutional oversight, participant consent, and disclosure for low-risk learning activities in all jurisdictions.

Feasibility of Measuring Preferences for Chemotherapy Among Early-Stage Breast Cancer Survivors Using a Direct Rank Ordering Multicriteria Decision Analysis Versus a Time Trade-Off.

OBJECTIVES: Chemotherapy is increasingly a preference-based choice among women diagnosed with early-stage breast cancer. Multicriteria decision analysis (MCDA) is a promising but underutilized method to facilitate shared decision making. We explored the feasibility of conducting an MCDA using direct rank ordering versus a time trade-off (TTO) to assess chemotherapy choice in a large population-based sample. METHODS: We surveyed 904 early-stage breast cancer survivors who were within 5 years of diagnosis and reported to the Western Washington State Cancer System and Kaiser Permanente Northern California registries. Direct rank ordering of 11 criteria and TTO surveys were conducted from September 2015 to July 2016; clinical data were obtained from registries or medical records. Multivariable regressions estimated post hoc associations between the MCDA, TTO, and self-reported chemotherapy receipt, considering covariates. RESULTS: Survivors ranged in age from 25 to 74 years and 73.9% had stage I tumors. The response rate for the rank ordering was 81.0%; TTO score was 94.2%. A one-standard deviation increase in the difference between the chemotherapy and no chemotherapy MCDA scores was associated with a 75.1% (95% confidence interval 43.9-109.7%; p < 0.001) increase in the adjusted odds of having received chemotherapy; no association was found between the TTO score and chemotherapy receipt. CONCLUSIONS: A rank-order-based MCDA was feasible and was associated with chemotherapy choice. Future research should consider developing and testing this MCDA for use in clinical encounters. Additional research is required to develop a TTO-based model and test its properties against a pragmatic MCDA to inform future shared decision-making tools.

Primary Care Physicians' Experiences With and Strategies for Managing Electronic Messages.

Importance: The increasing use of electronic communications has enhanced access to physicians for patients and clinical staff. Primary care physicians (PCPs) have anecdotally identified electronic inbox management as a new source of work-related stress. Objectives: To describe PCPs' experiences managing their electronic inboxes and to characterize the array of management strategies developed by individual physicians and practice groups. Design, Setting, and Participants: This qualitative study was conducted in 8 medical centers of a large group practice with more than 4 million patients in diverse settings and a mature electronic health record. The group encourages patients to use portal secure messaging to enhance access to their physicians and the care experience. Semistructured interviews were conducted with 24 internists and family medicine physicians identified via snowball sampling. Interviews were conducted July through November 2018. Data analysis was conducted between November 2018 and April 2019. Main Outcomes and Measures: Audio recorded interviews were transcribed and analyzed using thematic analysis to identify major themes and subthemes. Results: The 24 participants (12 women [50.0%]; mean [SD] age, 45.5 [6.5] years), including 9 department chiefs and 15 PCPs, had a mean (SD) of 16.8 (7.8) years since medical school graduation. Participants described substantial changes in medical practice due to electronic communication, including perceived patient expectations to receive rapid responses to portal secure messages. They described portal secure messaging as useful for building relationships with patients, but also reported that electronic message management has created new stressors, including erosion of work-life boundaries and anxiety associated with unlimited inbox volume. Individual PCPs used a diverse array of strategies, including multitasking during and outside work and delegating to medical assistants. Chiefs described group-level strategies, including reserving clinic time for inbox management, coverage systems for vacation and sick days, physician-to-physician training, and interdisciplinary teams to share messaging work. Conclusions and Relevance: Individual physicians and local practice groups have developed a wide array of strategies for electronic inbox management. The volume of electronic messages and PCPs' perceptions that patients expect rapid responses have created new stressors in primary care practice. Medical groups and health systems can support PCPs by facilitating knowledge transfer among physicians about inbox management strategies and further developing team structures for inbox coverage.

Developing a Prognostic Information System for Personalized Care in Real Time.

CONTEXT: Electronic medical records hold promise to transform clinical practice. However, technological and other barriers may preclude using them to guide care in real time. We used the Virtual Data Warehouse (VDW) to develop a tool that enables physicians to generate real-time, personalized prognostic information about survival after cancer. CASE DESCRIPTION: Patients with cancer often ask their oncologists, "Have you ever seen a patient like me?" To help oncologists answer this question, we developed a prototype Prognostic Information System (PRISM), a web-based tool that gathers data about the index patient from Kaiser Permanente's clinical information systems, selects a historical cohort of similar patients, and displays the survival curve of the similar patients relative to key points in their treatment course. FINDINGS AND MAJOR THEMES: The prototype was developed by a multidisciplinary team with expertise in oncology, research, and technology. We have completed two rounds of user testing and refinement. Successful development rested on: (1) executive support and a clinical champion; (2) collaboration among experts from multiple disciplines; (3) starting with simple cases rather than ambitious ones; (4) extensive research experience with the Virtual Data Warehouse, related databases, and an existing query tool; and (5) following agile software development principles, especially iterative user testing. CONCLUSION: Clinical data stored in health care systems' electronic medical records can be used to personalize clinical care in real time. Development of prognostic information systems can be accelerated by collaborations among researchers, technology specialists, and clinicians and by use of existing technology like the Virtual Data Warehouse.

Strategies for Building Delivery Science in an Integrated Health Care System.

Health systems today have increasing opportunities and imperatives to conduct delivery science, which is applied research that evaluates clinical or organizational practices that systems can implement or encourage. Examples include research on eliminating racial/ethnic disparities in hypertension management and on identifying the types of patients who can successfully use video visits. Clinical leaders and researchers often face barriers to delivery science, including limited funding, insufficient leadership support, lack of engagement between operational and research leaders, limited pools of research expertise, and lack of pathways to identify and develop ideas. We describe five key strategies we employed to address these barriers and develop a portfolio of delivery science programs in Kaiser Permanente Northern California. This portfolio now includes small and medium-sized grant programs, training programs for postdoctoral research fellows and experienced physician researchers, and a dedicated team that partners with clinicians to develop high-priority ideas and conduct small projects. Most of our approaches are consistent with frameworks used to develop delivery science by other health systems; some are innovative. Most of these strategies are adaptable by other health systems prepared to make long-range organizational commitments to mechanisms that foster partnerships between clinical leaders and researchers.

The impact of gene expression profile testing on confidence in chemotherapy decisions and prognostic expectations.

PURPOSE: Little is known about whether gene expression profile (GEP) testing and specific recurrence scores (e.g., medium risk) improve women's confidence in their chemotherapy decision or perceived recurrence risk. We evaluate the relationship between these outcomes and GEP testing. METHODS: We surveyed women eligible for GEP testing (stage I or II, Gr1-2, ER+, HER2-) identified through the Surveillance, Epidemiology, and End Results (SEER) Registry of Washington or Kaiser Permanente Northern California from 2012 to 2016, approximately 0-4 years from diagnosis (N = 904, RR = 45.4%). Confidence in chemotherapy was measured as confident (Very, completely) versus Not Confident (Somewhat, A little, Not At All); perceived risk recurrence was recorded numerically (0-100%). Women reported their GEP test receipt (Yes, No, Unknown) and risk recurrence score (High, Intermediate, Low, Unknown). In our analytic sample (N = 833), we propensity score weighted the three test receipt cohorts and used propensity weighted multivariable regressions to examine associations between the outcomes and the three test receipt cohorts, with receipt stratified by score. RESULTS: 29.5% reported an unknown GEP test receipt; 86% being confident. Compared to no test receipt, an intermediate score (aOR 0.34; 95% CI 0.20-0.58), unknown score (aOR 0.09; 95% CI 0.05-0.18), and unknown test receipt (aOR 0.37; 95% CI 0.24-0.57) were less likely to report confidence. Most women greatly overestimated their recurrence risk regardless of their test receipt or score. CONCLUSIONS: GEP testing was not associated with greater confidence in chemotherapy decisions. Better communication about GEP testing and the implications for recurrence risk may improve women's decisional confidence.

Cost Effectiveness of Gene Expression Profile Testing in Community Practice.

Purpose Gene expression profile (GEP) testing can support chemotherapy decision making for patients with early-stage, estrogen receptor-positive, human epidermal growth factor 2-negative breast cancers. This study evaluated the cost effectiveness of one GEP test, Onco type DX (Genomic Health, Redwood City, CA), in community practice with test-eligible patients age 40 to 79 years. Methods A simulation model compared 25-year societal incremental costs and quality-adjusted life-years (QALYs) of community Onco type DX use from 2005 to 2012 versus usual care in the pretesting era (2000 to 2004). Inputs included Onco type DX and chemotherapy data from an integrated health care system and national and published data on Onco type DX accuracy, chemotherapy effectiveness, utilities, survival and recurrence, and Medicare and patient costs. Sensitivity analyses varied individual parameters; results were also estimated for ideal conditions (ie, 100% testing and adherence to test-suggested treatment, perfect test accuracy, considering test effects on reassurance or worry, and lowest costs). Results Twenty-four percent of test-eligible patients had Onco type DX testing. Testing was higher in younger patients and patients with stage I disease ( v stage IIA), and 75.3% and 10.2% of patients with high and low recurrence risk scores received chemotherapy, respectively. The cost-effectiveness ratio for testing ( v usual care) was $188,125 per QALY. Considering test effects on worry versus reassurance decreased the cost-effectiveness ratio to $58,431 per QALY. With perfect test accuracy, the cost-effectiveness ratio was $28,947 per QALY, and under ideal conditions, it was $39,496 per QALY. Conclusion GEP testing is likely to have a high cost-effectiveness ratio on the basis of community practice patterns. However, realistic variations in assumptions about key variables could result in GEP testing having cost-effectiveness ratios in the range of other accepted interventions. The differences in cost-effectiveness ratios on the basis of community versus ideal conditions underscore the importance of considering real-world implementation when assessing the new technology.

Impact of Copayment Changes on Children's Albuterol Inhaler Use and Costs after the Clean Air Act Chlorofluorocarbon Ban.

OBJECTIVE: To examine changes in children's albuterol use and out-of-pocket (OOP) costs in response to increased copayments after the Food and Drug Administration banned inhalers with chlorofluorocarbon (CFC) propellants. SETTING: Four health maintenance organizations (HMOs), two that increased copayments for albuterol inhalers that went from generic CFC-containing to branded CFC-free versions, and two that retained generic copayments for CFC-free inhalers (controls). We included children with asthma aged 4-17 years with commercial coverage from 2007 to 2010. DESIGN: Interrupted time series with comparison series. DATA: We obtained enrollee and plan characteristics from enrollment files, and utilization data from pharmacy and medical claims; OOP expenditures were extracted from pharmacy claims for two HMOs with cost data available. FINDINGS: There were no significant differences in albuterol use between the group with increased cost-sharing and controls with respect to changes after the policy change. There was a postpolicy increase of $6.11 OOP per month per child using albuterol among those with increased cost-sharing versus $0.36 in controls; the difference between groups was significant (p < .01). CONCLUSIONS: Increased copayments for brand-name CFC-free albuterol after the CFC ban did not lead to a decrease in children's albuterol use, but it led to a modest increase in OOP costs.