Prospective Evaluation of the Kaiser Permanente Prostate Cancer Risk Calculator in Biopsy-Naïve Men With Mild PSA Elevations.

INTRODUCTION: Our goal was to assess the impact of providing prostate cancer risk estimates to patients and urologists among biopsy-naïve men with mild PSA elevations. METHODS: This prospective intervention study was conducted in urology departments in Kaiser Permanente Northern California among biopsy-naïve men with mild PSA elevations (<10 ng/mL) between March 2021 and March 2023. The intervention was providing prostate cancer risk estimates for intermediate-/high-grade cancer (Grade Groups 2-5) and any cancer to patients and urologists using our previously validated risk calculator. Biopsy outcomes were compared to a historical comparison group biopsied between January 2016 and December 2019. RESULTS: Over 24 months, the risk calculator was used to estimate risk for 1962 men, of whom 1455 (74%) accepted the biopsy. The men who accepted the biopsy had higher predicted intermediate-/high-grade cancer risks (median 24%, interquartile range [IQR] 17%-31%, vs 20%, IQR 14%-29%; P < .0001) and higher predicted risks for any cancer (median 48%, IQR 40%-56%, vs 45%, IQR 37%-53%, respectively; P < .0001) than those who declined the biopsy. Compared to the historical group, use of the risk calculator resulted in a biopsy population enriched with intermediate-/high-grade cancer: 29% vs 25%; similar rates of low-grade cancer: 24% vs 24%; and fewer negative biopsies: 47% vs 51%, respectively; overall P = .013. CONCLUSIONS: In biopsy-naïve men with mild PSA elevations, use of this risk calculator resulted in a biopsy population that had more intermediate-/high-grade cancer and fewer negative biopsies compared to a historical group.

The implementation checklist: A pragmatic instrument for accelerating research-to-implementation cycles.

Introduction: Learning health systems require rapid-cycle research and nimble implementation processes to maximize innovation across disparate specialties and operations. Existing detailed research-to-implementation frameworks require extensive time commitments and can be overwhelming for physician-researchers with clinical and operational responsibilities, inhibiting their widespread adoption. The creation of a short, pragmatic checklist to inform implementation processes may substantially improve uptake and implementation efficiency across a variety of health systems. Methods: We conducted a systematic review of existing implementation frameworks to identify core concepts. Utilizing comprehensive stakeholder engagement with 25 operational leaders, embedded physician-researchers, and delivery scientists, concepts were iteratively integrated to create and implement a final concise instrument. Results: A systematic review identified 894 publications describing implementation frameworks, which included 15 systematic reviews. Among these, domains were extracted from three commonly utilized instruments: the Quality Implementation Framework (QIF), the Consolidated Framework for Implementation Research (CFIR), and the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework. Iterative testing and stakeholder engagement revision of a four-page draft implementation document with five domains resulted in a concise, one-page implementation planning instrument to be used at project outset and periodically throughout project implementation planning. The instrument addresses end-user feasibility concerns while retaining the main goals of more complex tools. This instrument was then systematically integrated into projects within the Kaiser Permanente Northern California Delivery Science and Applied Research program to address stakeholder engagement, efficiency, project planning, and operational implementation of study results. Conclusion: A streamlined one-page implementation planning instrument, incorporating core concepts of existing frameworks, provides a pragmatic, robust framework for evidence-based healthcare innovation cycles that is being broadly implemented within a learning health system. These streamlined processes could inform other settings needing a best practice rapid-cycle research-to-implementation tool for large numbers of diverse projects.

Effectiveness of Bundled Hyperpolypharmacy Deprescribing Compared With Usual Care Among Older Adults: A Randomized Clinical Trial.

Importance: Older patients using many prescription drugs (hyperpolypharmacy) may be at increased risk of adverse drug effects. Objective: To test the effectiveness and safety of a quality intervention intended to reduce hyperpolypharmacy. Design, Setting, and Participants: This randomized clinical trial allocated patients 76 years or older who used 10 or more prescription medications to a deprescribing intervention or to usual care (1:1 ratio) at an integrated health system with multiple preexisting deprescribing workflows. Data were collected from October 15, 2020, to July 29, 2022. Intervention: Physician-pharmacist collaborative drug therapy management, standard-of-care practice recommendations, shared decision-making, and deprescribing protocols administered by telephone over multiple cycles for a maximum of 180 days after allocation. Main Outcomes and Measures: Primary end points were change in the number of medications and in the prevalence of geriatric syndrome (falls, cognition, urinary incontinence, and pain) from 181 to 365 days after allocation compared with before randomization. Secondary outcomes were use of medical services and adverse drug withdrawal effects. Results: Of a random sample of 2860 patients selected for potential enrollment, 2470 (86.4%) remained eligible after physician authorization, with 1237 randomized to the intervention and 1233 to usual care. A total of 1062 intervention patients (85.9%) were reached and agreed to enroll. Demographic variables were balanced. The median age of the 2470 patients was 80 (range, 76-104) years, and 1273 (51.5%) were women. In terms of race and ethnicity, 185 patients (7.5%) were African American, 234 (9.5%) were Asian or Pacific Islander, 220 (8.9%) were Hispanic, 1574 (63.7%) were White (63.7%), and 257 (10.4%) were of other (including American Indian or Alaska Native, Native Hawaiian, or >1 race or ethnicity) or unknown race or ethnicity. During follow-up, both the intervention and usual care groups had slight reductions in the number of medications dispensed (mean changes, -0.4 [95% CI, -0.6 to -0.2] and -0.4 [95% CI, -0.6 to -0.3], respectively), with no difference between the groups (P = .71). There were no significant changes in the prevalence of a geriatric condition in the usual care and intervention groups at the end of follow-up and no difference between the groups (baseline prevalence: 47.7% [95% CI, 44.9%-50.5%] vs 42.9% [95% CI, 40.1%-45.7%], respectively; difference-in-differences, 1.0 [95% CI, -3.5 to 5.6]; P = .65). No differences in use of medical services or adverse drug withdrawal effects were observed. Conclusions and Relevance: In this randomized clinical trial from an integrated care setting with various preexisting deprescribing workflows, a bundled hyperpolypharmacy deprescribing intervention was not associated with reduction in medication dispensing, prevalence of geriatric syndrome, utilization of medical services, or adverse drug withdrawal effects. Additional research is needed in less integrated settings and in more targeted populations. Trial Registration: ClinicalTrials.gov Identifier: NCT05616689.

Changes in COVID-19 Vaccine Intent Among a Diverse Population of Older Adults, June 2021-February 2022.

Introduction COVID-19 vaccination rates remain suboptimal in the United States. Clinicians and policymakers need to better understand how likely vaccine-hesitant individuals are to ultimately accept vaccination and what is associated with such changes. This study's aims were to 1) describe changes between vaccine intentions and actual uptake from June 2021 through February 2022, and 2) identify modifiable factors associated with vaccine uptake among those with initial hesitancy. Methods This cohort study included a stratified random sample of adults aged 65 years and older in an integrated health care system. The survey, conducted June through August 2021, elicited intent and perceptions regarding COVID-19 vaccination. Subsequent vaccine uptake through February 2022 was analyzed using electronic health records. Results Of 1195 individuals surveyed, 66% responded; 213 reported not yet having received a COVID-19 vaccine and were further analyzed. At baseline, most individuals said they would definitely not (42%) or probably not (5%) get the COVID-19 vaccine or were not sure (26%). During follow-up, 61 individuals (29%) were vaccinated, including 19% of those who initially said they would definitely not be vaccinated. Among vaccine-hesitant individuals, the rate of vaccination was highest for those who initially considered COVID-19 less dangerous than the vaccine (46%) or named short-term side effects (36%) as their most important concern. Conclusions COVID-19 vaccine intent among older adults was malleable during the pandemic's second year, even among those who initially said they would definitely not be vaccinated. Vaccine uptake could be enhanced by increasing awareness of COVID-19 risks and by addressing vaccine side effects.

Effect of Electronic and Mail Outreach From Primary Care Physicians for COVID-19 Vaccination of Black and Latino Older Adults: A Randomized Clinical Trial.

Importance: COVID-19 morbidity is highest in Black and Latino older adults. These racial and ethnic groups initially had lower vaccination uptake than others, and rates in Black adults continue to lag. Objectives: To evaluate the effect of outreach via electronic secure messages and mailings from primary care physicians (PCPs) on COVID-19 vaccination uptake among Black and Latino older adults and to compare the effects of culturally tailored and standard PCP messages. Design, Setting, and Participants: This randomized clinical trial was conducted from March 29 to May 20, 2021, with follow-up surveys through July 31, 2021. Latino and Black individuals aged 65 years and older from 4 Kaiser Permanente Northern California (KPNC) service areas were included. Data were analyzed from May 27, 2021, to September 28, 2021. Interventions: Individuals who had not received COVID-19 vaccination after previous outreach were randomized to electronic secure message and/or mail outreach from their PCP, similar outreach with additional culturally tailored content, or usual care. Outreach groups were sent a secure message or letter in their PCP's name, followed by a postcard to those still unvaccinated after 4 weeks. Main Outcomes and Measures: The primary outcome was time to receipt of COVID-19 vaccination during the 8 weeks after initial study outreach. KPNC data were supplemented with state data from external sources. Intervention effects were evaluated via proportional hazards regression. Results: Of 8287 included individuals (mean [SD] age, 72.6 [7.0] years; 4665 [56.3%] women), 2434 (29.4%) were Black, 3782 (45.6%) were Latino and preferred English-language communications, and 2071 (25.0%) were Latino and preferred Spanish-language communications; 2847 participants (34.4%) had a neighborhood deprivation index at the 75th percentile or higher. A total of 2767 participants were randomized to culturally tailored PCP outreach, 2747 participants were randomized to standard PCP outreach, and 2773 participants were randomized to usual care. Culturally tailored PCP outreach led to higher COVID-19 vaccination rates during follow-up compared with usual care (664 participants [24.0%] vs 603 participants [21.7%]; adjusted hazard ratio (aHR), 1.22; 95% CI, 1.09-1.37), as did standard PCP outreach (635 participants [23.1%]; aHR, 1.17; 95% CI, 1.04-1.31). Individuals who were Black (aHR, 1.19; 95% CI, 1.06-1.33), had high neighborhood deprivation (aHR, 1.17; 95% CI, 1.03-1.33), and had medium to high comorbidity scores (aHR, 1.19; 95% CI, 1.09-1.31) were more likely to be vaccinated during follow-up. Conclusions and Relevance: This randomized clinical trial found that PCP outreach using electronic and mailed messages increased COVID-19 vaccination rates among Black and Latino older adults. Trial Registration: ClinicalTrials.gov Identifier: NCT05096026.

Race-Specific Trends in Prostate Cancer Screening and Presentation before and after the 2012 United States Preventive Services Task Force Statement.

INTRODUCTION: Our goal was to assess temporal changes in the race-specific rates of prostate specific antigen (PSA) screening, prostate biopsy, overall prostate cancer detection and metastatic cancer at presentation among screen-eligible men in Kaiser Permanente Northern California before and after the 2012 United States Preventive Services Task Force Prostate Cancer Screening Statement. METHODS: This was a retrospective study spanning the years 2006 to 2017 in screen-eligible Kaiser Permanente Northern California members (Black men ages 45-69, all other men ages 50-69) with no history of prostate cancer. We compared the race-specific biennial rates of PSA testing, prostate biopsy, overall prostate cancer incidence and metastatic cancer at presentation. RESULTS: A total of 422,664 to 567,660 men per biennial period were evaluated (72% White, 8% Black, 20% Asian). Following the 2012 United States Preventive Services Task Force statement, all races experienced similar declines in screening (22%-25%), biopsy (47%-57%) and overall cancer detection (34%-48%) rates. We found an increase in metastatic rates (39%-105%). CONCLUSIONS: Following the 2012 United States Preventive Services Task Force Statement, in men under the age of 70, PSA screening, biopsy and overall prostate cancer detection rates significantly decreased in a similar fashion across all races, while rates of metastatic disease significantly increased in all races.

A prognostic information system for real-time personalized care: Lessons for embedded researchers.

Embedded researchers could play a central role in developing tools to personalize care using electronic medical records (EMRs). However, few studies have described the steps involved in developing such tools, or evaluated the key factors in success and failure. This case study describes how we used an EMR-derived data warehouse to develop a prototype informatics tool to help oncologists counsel patients with pancreatic cancer about their prognosis. The tool generated real-time prognostic information based on tumor type and stage, age, comorbidity status and lab tests. Our multidisciplinary team included embedded researchers, application developers, user experience experts, and an oncologist leader.This prototype succeeded in establishing proof of principle, but did not reach adoption into actual practice. In pilot testing, oncologists succeeded in generating prognostic information in real time. A few found it helpful in patient encounters, but all identified critical areas for further development before implementation. Generalizable lessons included the need to (1) include a wide range of potential use cases and stakeholders when selecting use cases for such tools; (2) develop talking points for clinicians to explain results from predictive tools to patients; (3) develop ways to reduce lag time between events and data availability; and (4) keep the options presented in the user interface very simple. This case demonstrates that embedded researchers can lead collaborations using EMR-derived data to create systems for real-time personalized patient counseling, and highlights challenges that such teams can anticipate.

Prospective validation of the Kaiser Permanente prostate cancer risk calculator in a contemporary, racially diverse, referral population.

PURPOSE: To prospectively validate a new prostate cancer risk calculator in a racially diverse population. MATERIALS AND METHODS: We recently developed, internally validated and published the Kaiser Permanente Prostate Cancer Risk Calculator. This study is a prospective validation of the calculator in a separate, referral population over a 21-month period. All patients were tested with a uniform PSA assay and a standardized systematic, ultrasound-guided biopsy scheme. We report on 3 calculator models: Model 1 included age, race, PSA, prior biopsy status, body mass index, and family history of prostate cancer; Model 2 added digital rectal exam to Model 1 variables; Model 3 added prostate volume to Model 2 variables. We considered three outcomes: high-grade disease (Gleason score ≥7), low-grade disease (Gleason score=6), and no cancer. Predictive discrimination and calibration were calculated. How each model might alter biopsy frequency and outcomes at various thresholds of risk was assessed. We compared the performance of our calculator with two other calculators. RESULTS: In 4178 patients (16.2% Asian, 11.3% African American, 13.5% Hispanic), cancer was found in 53%; 62% were Gleason score ≥7. Using a high-grade risk threshold for biopsy of ≥10%, Model 2 predictions would result in 9% of men avoiding a biopsy, while only missing 2% of high-grade cancers. At the same threshold, Model 3 predictions would result in 26% of men avoiding a biopsy, while only missing 5% of high-grade cancers. The c-statistics for Models 1, 2, and 3 to predict high-grade disease vs. low-grade or no cancer were 0.76, 0.79 and 0.85, respectively. The c-statistics for Models 1, 2, and 3 to predict any prostate cancer vs. no cancer were 0.70, 0.72 and 0.80, respectively. All models were well calibrated for all outcomes. Our Model 3 calculator had superior discrimination for high grade disease (c-statistic=0.85, 0.84-0.86) and any cancer (0.80, 0.79-0.82) compared to the PBCG calculator [(0.79, 0.78-0.80); 0.72 (0.70-0.73)] and the PCPT calculator [(0.75, 0.74-0.77); 0.69 (0.67-0.70)], respectively. In the high-grade cancer predicted risk range of 0-30%, our Model 2 was better calibrated than the PCPT and PBCG calculators. CONCLUSIONS: This validation of our calculator showed excellent performance characteristics.

Physician Stress During Electronic Health Record Inbox Work: In Situ Measurement With Wearable Sensors.

BACKGROUND: Increased work through electronic health record (EHR) messaging is frequently cited as a factor of physician burnout. However, studies to date have relied on anecdotal or self-reported measures, which limit the ability to match EHR use patterns with continuous stress patterns throughout the day. OBJECTIVE: The aim of this study is to collect EHR use and physiologic stress data through unobtrusive means that provide objective and continuous measures, cluster distinct patterns of EHR inbox work, identify physicians' daily physiologic stress patterns, and evaluate the association between EHR inbox work patterns and physician physiologic stress. METHODS: Physicians were recruited from 5 medical centers. Participants (N=47) were given wrist-worn devices (Garmin Vivosmart 3) with heart rate sensors to wear for 7 days. The devices measured physiological stress throughout the day based on heart rate variability (HRV). Perceived stress was also measured with self-reports through experience sampling and a one-time survey. From the EHR system logs, the time attributed to different activities was quantified. By using a clustering algorithm, distinct inbox work patterns were identified and their associated stress measures were compared. The effects of EHR use on physician stress were examined using a generalized linear mixed effects model. RESULTS: Physicians spent an average of 1.08 hours doing EHR inbox work out of an average total EHR time of 3.5 hours. Patient messages accounted for most of the inbox work time (mean 37%, SD 11%). A total of 3 patterns of inbox work emerged: inbox work mostly outside work hours, inbox work mostly during work hours, and inbox work extending after hours that were mostly contiguous to work hours. Across these 3 groups, physiologic stress patterns showed 3 periods in which stress increased: in the first hour of work, early in the afternoon, and in the evening. Physicians in group 1 had the longest average stress duration during work hours (80 out of 243 min of valid HRV data; P=.02), as measured by physiological sensors. Inbox work duration, the rate of EHR window switching (moving from one screen to another), the proportion of inbox work done outside of work hours, inbox work batching, and the day of the week were each independently associated with daily stress duration (marginal R2=15%). Individual-level random effects were significant and explained most of the variation in stress (conditional R2=98%). CONCLUSIONS: This study is among the first to demonstrate associations between electronic inbox work and physiological stress. We identified 3 potentially modifiable factors associated with stress: EHR window switching, inbox work duration, and inbox work outside work hours. Organizations seeking to reduce physician stress may consider system-based changes to reduce EHR window switching or inbox work duration or the incorporation of inbox management time into work hours.

Evaluation of Attention Switching and Duration of Electronic Inbox Work Among Primary Care Physicians.

Importance: Primary care physicians (PCPs) report multitasking during workdays while processing electronic inbox messages, but scant systematic information exists on attention switching and its correlates in the health care setting. Objectives: To describe PCPs' frequency of attention switching associated with electronic inbox work, identify potentially modifiable factors associated with attention switching and inbox work duration, and compare the relative association of attention switching and other factors with inbox work duration. Design, Setting, and Participants: This cross-sectional study of the work of 1275 PCPs in an integrated group serving 4.5 million patients used electronic health record (EHR) access logs from March 1 to 31, 2018, to evaluate PCPs' frequency of attention switching. Statistical analysis was performed from October 15, 2018, to August 28, 2020. Main Outcomes and Measures: Attention switching was defined as switching between the electronic inbox, other EHR work, and non-EHR periods. Inbox work duration included minutes spent on electronic inbox message views and related EHR tasks. Multivariable models controlled for the exposures. Results: The 1275 PCPs studied (721 women [56.5%]; mean [SD] age, 45.9 [8.5] years) had a mean (SD) of 9.0 (7.6) years of experience with the medical group and received a mean (SD) of 332.6 (148.3) (interquartile range, 252-418) new inbox messages weekly. On workdays, PCPs made a mean (SD) of 79.4 (21.8) attention switches associated with inbox work and did a mean (SD) 64.2 (18.7) minutes of inbox work over the course of 24 hours on workdays. In the model for attention switching, each additional patient secure message beyond the reference value was associated with 0.289 (95% CI, 0.217-0.362) additional switches, each additional results message was associated with 0.203 (95% CI, 0.127-0.278) additional switches, each additional request message was associated with 0.190 (95% CI, 0.124-0.257) additional switches, and each additional administrative message was associated with 0.262 (95% CI, 0.166-0.358) additional switches. Having a panel (a list of patients assigned to a primary care team) with more elderly patients (0.144 switches per percentage increase [95% CI, 0.009-0.278]) and higher inbox work duration (0.468 switches per additional minute of inbox work [95% CI, 0.411-0.524]) were also associated with higher attention switching involving the inbox. In the model for inbox work duration, each additional patient secure message beyond the reference value was associated with 0.151 (95% CI, 0.085-0.217) additional minutes, each additional results message was associated with 0.338 (95% CI, 0.272-0.404) additional minutes, each additional request message was associated with 0.101 (95% CI, 0.041-0.161) additional minutes, and each additional administrative message was associated with 0.179 (95% CI, 0.093-0.265) additional minutes. A higher percentage of the panel's patients initiating messages (0.386 minutes per percentage increase [95% CI, 0.026-0.745]) and attention switches (0.373 minutes per switch [95% CI, 0.328-0.419]) were also associated with higher inbox work duration. In addition, working at a medical center where all PCPs had high inbox work duration was independently associated with high or low inbox work duration. Conclusions and Relevance: This study suggests that PCPs make frequent attention switches during workdays while processing electronic inbox messages. Message quantity was associated with both attention switching and inbox work duration. Physician and patient panel characteristics had less association with attention switching and inbox work duration. Assisting PCPs with message quantity might help modulate both attention switching and inbox work duration.

Physicians' electronic inbox work patterns and factors associated with high inbox work duration.

OBJECTIVES: Electronic health record systems are increasingly used to send messages to physicians, but research on physicians' inbox use patterns is limited. This study's aims were to (1) quantify the time primary care physicians (PCPs) spend managing inboxes; (2) describe daily patterns of inbox use; (3) investigate which types of messages consume the most time; and (4) identify factors associated with inbox work duration. MATERIALS AND METHODS: We analyzed 1 month of electronic inbox data for 1275 PCPs in a large medical group and linked these data with physicians' demographic data. RESULTS: PCPs spent an average of 52 minutes on inbox management on workdays, including 19 minutes (37%) outside work hours. Temporal patterns of electronic inbox use differed from other EHR functions such as charting. Patient-initiated messages (28%) and results (29%) accounted for the most inbox work time. PCPs with higher inbox work duration were more likely to be female (P < .001), have more patient encounters (P < .001), have older patients (P < .001), spend proportionally more time on patient messages (P < .001), and spend more time per message (P < .001). Compared with PCPs with the lowest duration of time on inbox work, PCPs with the highest duration had more message views per workday (200 vs 109; P < .001) and spent more time on the inbox outside work hours (30 minutes vs 9.7 minutes; P < .001). CONCLUSIONS: Electronic inbox work by PCPs requires roughly an hour per workday, much of which occurs outside scheduled work hours. Interventions to assist PCPs in handling patient-initiated messages and results may help alleviate inbox workload.

Changing Provider PSA Screening Behavior Using Best Practice Advisories: Interventional Study in a Multispecialty Group Practice.

BACKGROUND: Most guidelines recommend against PSA-based screening for prostate cancer in men ≥ 70 years of age. Adherence to these guidelines is variable. OBJECTIVE: To determine whether the use of a "Best Practice Advisory" (BPA) intervention within the electronic medical record (EMR) system can alter the rate of PSA screening in men ≥ 70 years of age. DESIGN: This is an interventional study spanning the years 2013 through 2017, in men ≥ 70 years of age in Kaiser Permanente Northern California with no prior history of prostate cancer. The BPA intervention was activated in the EMR system on October 15, 2015, with no prior notice or education. SETTING: Integrated healthcare system including all Kaiser Permanente Northern California facilities. PARTICIPANTS: A population-based sample that included all male members ≥ 70 years of age without a history of prostate cancer. MAIN MEASURES: The main outcome was the rate of PSA testing in men ≥ 70 years of age. We compared the rates of PSA testing between the pre-BPA period (January 1, 2013-October 14, 2015) and the post-BPA period (October 15, 2015-December 31, 2017). An interrupted time series analysis of PSA ordering rates was performed. KEY RESULTS: Following the 2015 BPA intervention, screening rates substantially declined from 36.0 per 100 person-years to 14.9 per 100 person-years (rate ratio = 0.415; 95% CI: 0.410-0.419). The effect of the BPA was comparable among all patient races and ordering provider specialties. The interrupted time series analysis showed a rapid, large, and sustained drop in the rate of PSA ordering, and much less temporal variation in test ordering after activation of the BPA. CONCLUSION: Following activation of a BPA within the EMR, the rates of inappropriate PSA testing significantly declined by 58.5% in men ≥ 70 years of age and temporal variation was reduced.

Prospective development of a prostate cancer risk calculator in a racially diverse population: The Kaiser Permanente Prostate Cancer Risk Calculator.

PURPOSE: To prospectively develop a prostate cancer (CaP) risk calculator in a racially diverse population. MATERIALS AND METHODS: All patients referred for prostate biopsy due to an elevated prostate-specific antigen or abnormal digital rectal exam in a 19-months period at Kaiser Permanente Northern California underwent a standardized systematic, ultrasound-guided biopsy scheme (14-cores for initial biopsy, 18-20 cores for repeat biopsy). All pertinent clinical variables were prospectively collected. The highest Gleason score for each patient was recorded for all positive biopsies. We used a split sample design to develop and validate 3 multivariable prediction models using multinomial logistic regression with the least absolute shrinkage and selection operator. All models included these core variables: age, race, prostate-specific antigen, prior biopsy status, body mass index, and family history of CaP. Model 1 included only the core variables, Model 2 added digital rectal exam, and Model 3 added digital rectal exam and prostate volume. We considered 3 outcomes: high-grade disease (Gleason score ≥7), low-grade disease (Gleason score = 6), and no cancer. Predictive discrimination was quantified using the c-statistic. RESULTS: Complete data were available for 2,967 patients. Cancer was found in 50% of patients: of these, 58% were Gleason score ≥7 and 42% were low grade. Compared to Caucasians, African Americans were at a higher risk while Asians and Hispanics were at a lower risk for overall and high-grade cancer detection. The number of prior negative biopsies was also protective for these outcomes. The c-statistics for Model 1, 2, and 3 to predict high-grade disease vs. low-grade or no cancer were 0.76, 0.79, and 0.85, respectively. The c-statistics for Model 1, 2, and 3 to predict any CaP vs. no cancer were 0.69, 0.70, and 0.79, respectively. All models were well calibrated for all outcomes. CONCLUSIONS: In men with elevated PSA levels, our calculator provides useful information that may enhance the shared decision-making process regarding the need for biopsy.

Changes in Prostate Cancer Presentation Following the 2012 USPSTF Screening Statement: Observational Study in a Multispecialty Group Practice.

BACKGROUND: In 2012, the US Preventive Services Task Force (USPSTF) recommended against PSA-based screening for prostate cancer in men of all ages. Following this change, screening declined yet the complete impact on clinical presentation is not well defined in the screen-eligible population. OBJECTIVE: To determine if the rates of PSA screening, prostate biopsy, incident prostate cancer detection, and stage IV at presentation in screen-eligible men in Kaiser Permanente Northern California changed following the 2012 USPSTF Prostate Cancer Screening recommendations. DESIGN: Retrospective study spanning the years 2010 to 2015, in screen-eligible Kaiser Permanente Northern California members (African American men ages 45-69 and all other men ages 50-69) with no prior history of prostate cancer. Participants All screen-eligible, male members during 2010 (n = 403,931) to 2015 (n = 483,286) without a history of prostate cancer within all Kaiser Permanente Northern California facilities. MAIN MEASURES: Annual rates of PSA testing, prostate biopsy, incident prostate cancer detection, and stage IV cancer at presentation were compared between the pre-guideline period, 2010 and 2011, and the post-guideline period, 2014 and 2015, in men under the age of 70. KEY RESULTS: Following the 2012 USPSTF guideline change, screening rates declined 23.4% (95% CI 23.0-23.8%), biopsy rates declined 64.3% (95% CI 62.9-65.6%), and incident prostate cancer detection rates declined 53.5% (95% CI 50.1-56.7%) resulting in 1871 fewer incident cancers detected, and metastatic cancer rates increased 36.9% (95% CI 9.5-71.0%) resulting in 75 more stage IV cancers detected. CONCLUSION: Less screening resulted in a large decrease in cancer detection, some of which may be beneficial as many cancers may be indolent, yet this decrease occurred at the expense of an increase in metastatic cancer rates. For every 25 fewer cancers detected, one metastatic cancer was diagnosed. This information may be valuable in the shared decision-making process around prostate cancer screening.

Primary Care Physicians' Experiences With and Strategies for Managing Electronic Messages.

Importance: The increasing use of electronic communications has enhanced access to physicians for patients and clinical staff. Primary care physicians (PCPs) have anecdotally identified electronic inbox management as a new source of work-related stress. Objectives: To describe PCPs' experiences managing their electronic inboxes and to characterize the array of management strategies developed by individual physicians and practice groups. Design, Setting, and Participants: This qualitative study was conducted in 8 medical centers of a large group practice with more than 4 million patients in diverse settings and a mature electronic health record. The group encourages patients to use portal secure messaging to enhance access to their physicians and the care experience. Semistructured interviews were conducted with 24 internists and family medicine physicians identified via snowball sampling. Interviews were conducted July through November 2018. Data analysis was conducted between November 2018 and April 2019. Main Outcomes and Measures: Audio recorded interviews were transcribed and analyzed using thematic analysis to identify major themes and subthemes. Results: The 24 participants (12 women [50.0%]; mean [SD] age, 45.5 [6.5] years), including 9 department chiefs and 15 PCPs, had a mean (SD) of 16.8 (7.8) years since medical school graduation. Participants described substantial changes in medical practice due to electronic communication, including perceived patient expectations to receive rapid responses to portal secure messages. They described portal secure messaging as useful for building relationships with patients, but also reported that electronic message management has created new stressors, including erosion of work-life boundaries and anxiety associated with unlimited inbox volume. Individual PCPs used a diverse array of strategies, including multitasking during and outside work and delegating to medical assistants. Chiefs described group-level strategies, including reserving clinic time for inbox management, coverage systems for vacation and sick days, physician-to-physician training, and interdisciplinary teams to share messaging work. Conclusions and Relevance: Individual physicians and local practice groups have developed a wide array of strategies for electronic inbox management. The volume of electronic messages and PCPs' perceptions that patients expect rapid responses have created new stressors in primary care practice. Medical groups and health systems can support PCPs by facilitating knowledge transfer among physicians about inbox management strategies and further developing team structures for inbox coverage.

Developing a Prognostic Information System for Personalized Care in Real Time.

CONTEXT: Electronic medical records hold promise to transform clinical practice. However, technological and other barriers may preclude using them to guide care in real time. We used the Virtual Data Warehouse (VDW) to develop a tool that enables physicians to generate real-time, personalized prognostic information about survival after cancer. CASE DESCRIPTION: Patients with cancer often ask their oncologists, "Have you ever seen a patient like me?" To help oncologists answer this question, we developed a prototype Prognostic Information System (PRISM), a web-based tool that gathers data about the index patient from Kaiser Permanente's clinical information systems, selects a historical cohort of similar patients, and displays the survival curve of the similar patients relative to key points in their treatment course. FINDINGS AND MAJOR THEMES: The prototype was developed by a multidisciplinary team with expertise in oncology, research, and technology. We have completed two rounds of user testing and refinement. Successful development rested on: (1) executive support and a clinical champion; (2) collaboration among experts from multiple disciplines; (3) starting with simple cases rather than ambitious ones; (4) extensive research experience with the Virtual Data Warehouse, related databases, and an existing query tool; and (5) following agile software development principles, especially iterative user testing. CONCLUSION: Clinical data stored in health care systems' electronic medical records can be used to personalize clinical care in real time. Development of prognostic information systems can be accelerated by collaborations among researchers, technology specialists, and clinicians and by use of existing technology like the Virtual Data Warehouse.

Oncologist and organizational factors associated with variation in breast cancer multigene testing.

PURPOSE: Multigene testing for breast cancer recurrence risk became available in 2007, yet many eligible patients remain untested. This study evaluated variation in testing rates, and oncologist and organizational factors associated with variation, in a setting without financial influences on testing. METHODS: We conducted a retrospective cohort study using electronic data and oncologist surveys within Kaiser Permanente Northern California, a large integrated health care system. Analyses included all 2974 test eligible patients from 2013 to 2015, 113 oncologists, and 15 practice groups. Receipt of multigene testing was evaluated with generalized linear mixed models. RESULTS: Overall, 39% of eligible patients had multigene testing, but rates varied widely among practice groups, ranging from 24 to 48% after case mix adjustment. This 24% difference among practices was greater than the variation associated with most patient characteristics, including comorbidities and race/ethnicity, and similar to that associated with tumor size. Practice group and oncologist factors were statistically significant contributors to the variation in testing after adjusting for patient factors. Patients were more likely to be tested if they had a female oncologist (aOR 1.60, 95% CI 1.21-2.12) or were in a practice whose chief had a high testing rate (aOR 1.20, 95% CI 1.12-1.29 per 10% increase in the percent tested). CONCLUSIONS: Oncologist and leadership practices play a key role in the variation in genomic test use for cancer recurrence risk even in a healthcare system without financial barriers to testing and could be a leverage point for implementing desired practice changes for new genomic advances.

Building a Governance Strategy for CER: The Patient Outcomes Research to Advance Learning (PORTAL) Network Experience.

INTRODUCTION: The Patient Outcomes Research to Advance Learning (PORTAL) Network was established with funding from the Patient-Centered Outcomes Research Institute (PCORI) in 2014. The PORTAL team adapted governance structures and processes from past research network collaborations. We will review and outline the structures and processes of the PORTAL governance approach and describe how proactively focusing on priority areas helped us to facilitate an ambitious research agenda. BACKGROUND: For years a variety of funders have supported large-scale infrastructure grants to promote the use of clinical datasets to answer important comparative effectiveness research (CER) questions. These awards have provided the impetus for health care systems to join forces in creating clinical data research networks. Often, these scientific networks do not develop governance processes proactively or systematically, and address issues only as problems arise. Even if network leaders and collaborators foresee the need to develop governance approaches, they may underestimate the time and effort required to develop sound processes. The resulting delays can impede research progress. INNOVATION: Because the PORTAL sites had built trust and a foundation of collaboration by participating with one another in past research networks, essential elements of effective governance such as guiding principles, decision making processes, project governance, data governance, and stakeholders in governance were familiar to PORTAL investigators. This trust and familiarity enabled the network to rapidly prioritize areas that required sound governance approaches: responding to new research opportunities, creating a culture of trust and collaboration, conducting individual studies, within the broader network, assigning responsibility and credit to scientific investigators, sharing data while protecting privacy/security, and allocating resources. The PORTAL Governance Document, complete with a Toolkit of Appendices is included for reference and for adaptation by other networks. CREDIBILITY: As a result of identifying project-based governance priorities (IRB approval, subcontracting, selection of new research including lead PI and participating sites, and authorship) and data governance priorities (reciprocal data use agreement, analytic plan procedures, and other tools for data governance), PORTAL established most of its governance structure by Month 6 of the 18 month project. This allowed science to progress and collaborators to experience first-hand how the structures and procedures functioned in the remaining 12 months of the project, leaving ample time to refine them and to develop new structures or processes as necessary. DISCUSSION: The use of procedures and processes with which participating investigators and their home institutions were already familiar allowed project and regulatory requirements to be established quickly to protect patients, their data, and the health care systems that act as stewards for both. As the project progressed, PORTAL was able to test and adjust the structures it put place, and to make substantive revisions by Month 17. As a result, priority processes have been predictable, transparent and effective. CONCLUSION/NEXT STEPS: Strong governance practices are a stewardship responsibility of research networks to justify the trust of patients, health plan members, health care delivery organizations, and other stakeholders. Well-planned governance can reduce the time necessary to initiate the scientific activities of a network, a particular concern when the time frame to complete research is short. Effective network and data governance structures protect patient and institutional data as well as the interests of investigators and their institutions, and assures that the network has built an environment to meet the goals of the research.