Identifying erroneous height and weight values from adult electronic health records in the All of Us research program.

INTRODUCTION: Electronic Health Records (EHR) are a useful data source for research, but their usability is hindered by measurement errors. This study investigated an automatic error detection algorithm for adult height and weight measurements in EHR for the All of Us Research Program (All of Us). METHODS: We developed reference charts for adult heights and weights that were stratified on participant sex. Our analysis included 4,076,534 height and 5,207,328 wt measurements from âˆ¼ 150,000 participants. Errors were identified using modified standard deviation scores, differences from their expected values, and significant changes between consecutive measurements. We evaluated our method with chart-reviewed heights (8,092) and weights (9,039) from 250 randomly selected participants and compared it with the current cleaning algorithm in All of Us. RESULTS: The proposed algorithm classified 1.4 % of height and 1.5 % of weight errors in the full cohort. Sensitivity was 90.4 % (95 % CI: 79.0-96.8 %) for heights and 65.9 % (95 % CI: 56.9-74.1 %) for weights. Precision was 73.4 % (95 % CI: 60.9-83.7 %) for heights and 62.9 (95 % CI: 54.0-71.1 %) for weights. In comparison, the current cleaning algorithm has inferior performance in sensitivity (55.8 %) and precision (16.5 %) for height errors while having higher precision (94.0 %) and lower sensitivity (61.9 %) for weight errors. DISCUSSION: Our proposed algorithm outperformed in detecting height errors compared to weights. It can serve as a valuable addition to the current All of Us cleaning algorithm for identifying erroneous height values.

A Case Report on the Effectiveness of Virtual Monitoring of Postdischarge COVID-19 Positive Patients in a Rural Hospital Setting: A Retrospective Review.

Objective: The pandemic has pushed hospital system to re-evaluate the ways they provide care. West Tennessee Healthcare (WTH) developed a remote patient monitoring (RPM) program to monitor positive COVID-19 patients after being discharged from the hospital for any worsening symptomatology and preemptively mitigate the potential of readmission. Methods: We sought to compare the readmission rates of individuals placed on our remote monitoring protocol with individuals not included in the program. We selected remotely monitored individuals discharged from WTH from October 2020 to December 2020 and compared these data points with a control group. Results: We analyzed 1,351 patients with 241 patients receiving no RPM intervention, 969 patients receiving standard monitoring, and 141 patients enrolled in our 24-h remote monitoring. Our lowest all cause readmission rate was 4.96% (p = 0.37) in our 24-h remote monitoring group. We also collected 641 surveys from the monitored patients with two statistically significant answers. Discussion: The low readmission rate noted in our 24-h remotely monitored cohort signifies a potential opportunity that a program of this nature can create for a health care system struggling during a resource-limited time to continue to provide quality care. Conclusion: The program allowed the allocation of hospital resources for individuals with more acute states and monitored less critical patients without using personal protective equipment. The novel program was able to offer an avenue to improve resource utilization and provide care for a health system in a rural area. Further investigation is needed; however, significant opportunities can be seen with data obtained during the study.

Automating the Classification of Complexity of Medical Decision-Making in Patient-Provider Messaging in a Patient Portal.

BACKGROUND: Patient portals are consumer health applications that allow patients to view their health information. Portals facilitate the interactions between patients and their caregivers by offering secure messaging. Patients communicate different needs through portal messages. Medical needs contain requests for delivery of care (e.g. reporting new symptoms). Automating the classification of medical decision complexity in portal messages has not been investigated. MATERIALS AND METHODS: We trained two multiclass classifiers, multinomial Naïve Bayes and random forest on 500 message threads, to quantify and label the complexity of decision-making into four classes: no decision, straightforward, low, and moderate. We compared the performance of the models to using only the number of medical terms without training a machine learning model. RESULTS: Our analysis demonstrated that machine learning models have better performance than the model that did not use machine learning. Moreover, machine learning models could quantify the complexity of decision-making that the messages contained with 0.59, 0.45, and 0.58 for macro, micro, and weighted precision and 0.63,0.41, and 0.63 for macro, micro, and weighted recall. CONCLUSIONS: This study is one of the first to attempt to classify patient portal messages by whether they involve medical decision-making and the complexity of that decision-making. Machine learning classifiers trained on message content resulted in better message thread classification than classifiers that employed medical terms in the messages alone.

Cost-utility of osteoarticular allograft versus endoprosthetic reconstruction for primary bone sarcoma of the knee: A markov analysis.

BACKGROUND: The most cost-effective reconstruction after resection of bone sarcoma is unknown. The goal of this study was to compare the cost effectiveness of osteoarticular allograft to endoprosthetic reconstruction of the proximal tibia or distal femur. METHODS: A Markov model was used. Revision and complication rates were taken from existing studies. Costs were based on Medicare reimbursement rates and implant prices. Health-state utilities were derived from the Health Utilities Index 3 survey with additional assumptions. Incremental cost-effectiveness ratios (ICER) were used with less than $100 000 per quality-adjusted life year (QALY) considered cost-effective. Sensitivity analyses were performed for comparison over a range of costs, utilities, complication rates, and revisions rates. RESULTS: Osteoarticular allografts, and a 30% price-discounted endoprosthesis were cost-effective with ICERs of $92.59 and $6 114.77. One-way sensitivity analysis revealed discounted endoprostheses were favored if allografts cost over $21 900 or endoprostheses cost less than $51 900. Allograft reconstruction was favored over discounted endoprosthetic reconstruction if the allograft complication rate was less than 1.3%. Allografts were more cost-effective than full-price endoprostheses. CONCLUSIONS: Osteoarticular allografts and price-discounted endoprosthetic reconstructions are cost-effective. Sensitivity analysis, using plausible complication and revision rates, favored the use of discounted endoprostheses over allografts. Allografts are more cost-effective than full-price endoprostheses.

Classifying patient portal messages using Convolutional Neural Networks.

OBJECTIVE: Patients communicate with healthcare providers via secure messaging in patient portals. As patient portal adoption increases, growing messaging volumes may overwhelm providers. Prior research has demonstrated promise in automating classification of patient portal messages into communication types to support message triage or answering. This paper examines if using semantic features and word context improves portal message classification. MATERIALS AND METHODS: Portal messages were classified into the following categories: informational, medical, social, and logistical. We constructed features from portal messages including bag of words, bag of phrases, graph representations, and word embeddings. We trained one-versus-all random forest and logistic regression classifiers, and convolutional neural network (CNN) with a softmax output. We evaluated each classifier's performance using Area Under the Curve (AUC). RESULTS: Representing the messages using bag of words, the random forest detected informational, medical, social, and logistical communications in patient portal messages with AUCs: 0.803, 0.884, 0.828, and 0.928, respectively. Graph representations of messages outperformed simpler features with AUCs: 0.837, 0.914, 0.846, 0.884 for informational, medical, social, and logistical communication, respectively. Representing words with Word2Vec embeddings, and mapping features using a CNN had the best performance with AUCs: 0.908 for informational, 0.917 for medical, 0.935 for social, and 0.943 for logistical categories. DISCUSSION AND CONCLUSION: Word2Vec and graph representations improved the accuracy of classifying portal messages compared to features that lacked semantic information such as bag of words, and bag of phrases. Furthermore, using Word2Vec along with a CNN model, which provide a higher order representation, improved the classification of portal messages.

Illuminating the landscape of high-level clinical trial opportunities in the All of Us Research Program.

OBJECTIVE: With its size and diversity, the All of Us Research Program has the potential to power and improve representation in clinical trials through ancillary studies like Nutrition for Precision Health. We sought to characterize high-level trial opportunities for the diverse participants and sponsors of future trial investment. MATERIALS AND METHODS: We matched All of Us participants with available trials on ClinicalTrials.gov based on medical conditions, age, sex, and geographic location. Based on the number of matched trials, we (1) developed the Trial Opportunities Compass (TOC) to help sponsors assess trial investment portfolios, (2) characterized the landscape of trial opportunities in a phenome-wide association study (PheWAS), and (3) assessed the relationship between trial opportunities and social determinants of health (SDoH) to identify potential barriers to trial participation. RESULTS: Our study included 181 529 All of Us participants and 18 634 trials. The TOC identified opportunities for portfolio investment and gaps in currently available trials across federal, industrial, and academic sponsors. PheWAS results revealed an emphasis on mental disorder-related trials, with anxiety disorder having the highest adjusted increase in the number of matched trials (59% [95% CI, 57-62]; P < 1e-300). Participants from certain communities underrepresented in biomedical research, including self-reported racial and ethnic minorities, had more matched trials after adjusting for other factors. Living in a nonmetropolitan area was associated with up to 13.1 times fewer matched trials. DISCUSSION AND CONCLUSION: All of Us data are a valuable resource for identifying trial opportunities to inform trial portfolio planning. Characterizing these opportunities with consideration for SDoH can provide guidance on prioritizing the most pressing barriers to trial participation.

Effects of Cost Sharing on Ophthalmic Care Utilization in the Affordable Care Act Marketplace.

PURPOSE: To determine the distribution and quantity of ophthalmic care consumed on Affordable Care Act (ACA) plans, the demographics of the population utilizing these services, and the relationship between ACA insurance coverage plan tier, cost sharing, and total cost of ophthalmic care consumed. METHODS: This cross-sectional study analyzed ACA individual and small group market claims data from the Wakely Affordable Care Act (WACA) 2018 dataset, which contains detailed claims, enrollment, and premium data from Edge Servers for 3.9 million individual and small group market lives. We identified all enrollees with ophthalmology-specific billing, procedure, and national drug codes. We then analyzed the claims by plan type and calculated the total cost and out-of-pocket (OOP) cost. RESULTS: Among 3.9 million enrollees in the WACA 2018 dataset, 538,169 (13.7%) had claims related to ophthalmology procedures, medications, and/or diagnoses. A total of $203 million was generated in ophthalmology-related claims, with $54 million in general services, $42 million in medications, $20 million in diagnostics and imaging, and $86 million in procedures. Average annual OOP costs were $116 per member, or 30.9% of the total cost, and were lowest for members with platinum plans (16% OOP) and income-driven cost sharing reduction (ICSR) subsidies (17% OOP). Despite stable ocular disease distribution across plan types, beneficiaries with silver ICSR subsidies consumed more total care than any other plan, higher than platinum plan enrollees and almost 1.5× the cost of bronze plan enrollees. CONCLUSIONS: Ophthalmic care for enrollees on ACA plans generated substantial costs in 2018. Plans with higher OOP cost sharing may result in lower utilization of ophthalmic care.

Patient Cost Exposure And Use Of Preventive Care Among ACA-Compliant Individual Plans.

The Affordable Care Act (ACA) mandated coverage of common preventive services with zero patient cost sharing. However, patients may still experience high same-day costs when receiving these "zero-dollar" preventive services. Our analysis of on- and off-exchange individual-market health plans during 2016-18 revealed that 21-61 percent of enrollees experienced same-day cost exposure greater than $0 when accessing ACA-mandated free preventive services.

Importance of missingness in baseline variables: A case study of the All of Us Research Program.

OBJECTIVE: The All of Us Research Program collects data from multiple information sources, including health surveys, to build a national longitudinal research repository that researchers can use to advance precision medicine. Missing survey responses pose challenges to study conclusions. We describe missingness in All of Us baseline surveys. STUDY DESIGN AND SETTING: We extracted survey responses between May 31, 2017, to September 30, 2020. Missing percentages for groups historically underrepresented in biomedical research were compared to represented groups. Associations of missing percentages with age, health literacy score, and survey completion date were evaluated. We used negative binomial regression to evaluate participant characteristics on the number of missed questions out of the total eligible questions for each participant. RESULTS: The dataset analyzed contained data for 334,183 participants who submitted at least one baseline survey. Almost all (97.0%) of the participants completed all baseline surveys, and only 541 (0.2%) participants skipped all questions in at least one of the baseline surveys. The median skip rate was 5.0% of the questions, with an interquartile range (IQR) of 2.5% to 7.9%. Historically underrepresented groups were associated with higher missingness (incidence rate ratio (IRR) [95% CI]: 1.26 [1.25, 1.27] for Black/African American compared to White). Missing percentages were similar by survey completion date, participant age, and health literacy score. Skipping specific questions were associated with higher missingness (IRRs [95% CI]: 1.39 [1.38, 1.40] for skipping income, 1.92 [1.89, 1.95] for skipping education, 2.19 [2.09-2.30] for skipping sexual and gender questions). CONCLUSION: Surveys in the All of Us Research Program will form an essential component of the data researchers can use to perform their analyses. Missingness was low in All of Us baseline surveys, but group differences exist. Additional statistical methods and careful analysis of surveys could help mitigate challenges to the validity of conclusions.

Evaluating automated electronic case report form data entry from electronic health records.

Background: Many clinical trials leverage real-world data. Typically, these data are manually abstracted from electronic health records (EHRs) and entered into electronic case report forms (CRFs), a time and labor-intensive process that is also error-prone and may miss information. Automated transfer of data from EHRs to eCRFs has the potential to reduce data abstraction and entry burden as well as improve data quality and safety. Methods: We conducted a test of automated EHR-to-CRF data transfer for 40 participants in a clinical trial of hospitalized COVID-19 patients. We determined which coordinator-entered data could be automated from the EHR (coverage), and the frequency with which the values from the automated EHR feed and values entered by study personnel for the actual study matched exactly (concordance). Results: The automated EHR feed populated 10,081/11,952 (84%) coordinator-completed values. For fields where both the automation and study personnel provided data, the values matched exactly 89% of the time. Highest concordance was for daily lab results (94%), which also required the most personnel resources (30 minutes per participant). In a detailed analysis of 196 instances where personnel and automation entered values differed, both a study coordinator and a data analyst agreed that 152 (78%) instances were a result of data entry error. Conclusions: An automated EHR feed has the potential to significantly decrease study personnel effort while improving the accuracy of CRF data.

Systematic replication of smoking disease associations using survey responses and EHR data in the All of Us Research Program.

OBJECTIVE: The All of Us Research Program (All of Us) aims to recruit over a million participants to further precision medicine. Essential to the verification of biobanks is a replication of known associations to establish validity. Here, we evaluated how well All of Us data replicated known cigarette smoking associations. MATERIALS AND METHODS: We defined smoking exposure as follows: (1) an EHR Smoking exposure that used International Classification of Disease codes; (2) participant provided information (PPI) Ever Smoking; and, (3) PPI Current Smoking, both from the lifestyle survey. We performed a phenome-wide association study (PheWAS) for each smoking exposure measurement type. For each, we compared the effect sizes derived from the PheWAS to published meta-analyses that studied cigarette smoking from PubMed. We defined two levels of replication of meta-analyses: (1) nominally replicated: which required agreement of direction of effect size, and (2) fully replicated: which required overlap of confidence intervals. RESULTS: PheWASes with EHR Smoking, PPI Ever Smoking, and PPI Current Smoking revealed 736, 492, and 639 phenome-wide significant associations, respectively. We identified 165 meta-analyses representing 99 distinct phenotypes that could be matched to EHR phenotypes. At P < .05, 74 were nominally replicated and 55 were fully replicated. At P < 2.68 × 10-5 (Bonferroni threshold), 58 were nominally replicated and 40 were fully replicated. DISCUSSION: Most phenotypes found in published meta-analyses associated with smoking were nominally replicated in All of Us. Both survey and EHR definitions for smoking produced similar results. CONCLUSION: This study demonstrated the feasibility of studying common exposures using All of Us data.

The All of Us Data and Research Center: Creating a Secure, Scalable, and Sustainable Ecosystem for Biomedical Research.

The All of Us Research Program's Data and Research Center (DRC) was established to help acquire, curate, and provide access to one of the world's largest and most diverse datasets for precision medicine research. Already, over 500,000 participants are enrolled in All of Us, 80% of whom are underrepresented in biomedical research, and data are being analyzed by a community of over 2,300 researchers. The DRC created this thriving data ecosystem by collaborating with engaged participants, innovative program partners, and empowered researchers. In this review, we first describe how the DRC is organized to meet the needs of this broad group of stakeholders. We then outline guiding principles, common challenges, and innovative approaches used to build the All of Us data ecosystem. Finally, we share lessons learned to help others navigate important decisions and trade-offs in building a modern biomedical data platform.

EHR Data Quality Assessment Tools and Issue Reporting Workflows for the 'All of Us' Research Program Clinical Data Research Network.

The All of Us (AoU) Research Program aggregates electronic health records (EHR) data from 300,00+ participants spanning 50+ distinct data sites. The diversity and size of AoU's data network result in multifaceted obstacles to data integration that may undermine the usability of patient EHR. Consequently, the AoU team implemented data quality tools to regularly evaluate and communicate EHR data quality issues at scale. The use of systematic feedback and educational tools ultimately increased site engagement and led to quantitative improvements in EHR quality as measured by program- and externally-defined metrics. These improvements enabled the AoU team to save time on troubleshooting EHR and focus on the development of alternate mechanisms to improve the quality of future EHR submissions. While this framework has proven effective, further efforts to automate and centralize communication channels are needed to deepen the program's efforts while retaining its scalability.

Comparing medical history data derived from electronic health records and survey answers in the All of Us Research Program.

OBJECTIVE: A participant's medical history is important in clinical research and can be captured from electronic health records (EHRs) and self-reported surveys. Both can be incomplete, EHR due to documentation gaps or lack of interoperability and surveys due to recall bias or limited health literacy. This analysis compares medical history collected in the All of Us Research Program through both surveys and EHRs. MATERIALS AND METHODS: The All of Us medical history survey includes self-report questionnaire that asks about diagnoses to over 150 medical conditions organized into 12 disease categories. In each category, we identified the 3 most and least frequent self-reported diagnoses and retrieved their analogues from EHRs. We calculated agreement scores and extracted participant demographic characteristics for each comparison set. RESULTS: The 4th All of Us dataset release includes data from 314 994 participants; 28.3% of whom completed medical history surveys, and 65.5% of whom had EHR data. Hearing and vision category within the survey had the highest number of responses, but the second lowest positive agreement with the EHR (0.21). The Infectious disease category had the lowest positive agreement (0.12). Cancer conditions had the highest positive agreement (0.45) between the 2 data sources. DISCUSSION AND CONCLUSION: Our study quantified the agreement of medical history between 2 sources-EHRs and self-reported surveys. Conditions that are usually undocumented in EHRs had low agreement scores, demonstrating that survey data can supplement EHR data. Disagreement between EHR and survey can help identify possible missing records and guide researchers to adjust for biases.

Identifying Nonpatient Authors of Patient Portal Secure Messages in Oncology: A Proof-of-Concept Demonstration of Natural Language Processing Methods.

PURPOSE: Patient portal secure messages are not always authored by the patient account holder. Understanding who authored the message is particularly important in an oncology setting where symptom reporting is crucial to patient treatment. Natural language processing has the potential to detect messages not authored by the patient automatically. METHODS: Patient portal secure messages from the Memorial Sloan Kettering Cancer Center were retrieved and manually annotated as a predicted unregistered proxy (ie, not written by the patient) or a presumed patient. After randomly splitting the annotated messages into training and test sets in a 70:30 ratio, a bag-of-words approach was used to extract features and then a Least Absolute Shrinkage and Selection Operator (LASSO) model was trained and used for classification. RESULTS: Portal secure messages (n = 2,000) were randomly selected from unique patient accounts and manually annotated. We excluded 335 messages from the data set as the annotators could not determine if they were written by a patient or proxy. Using the remaining 1,665 messages, a LASSO model was developed that achieved an area under the curve of 0.932 and an area under the precision recall curve of 0.748. The sensitivity and specificity related to classifying true-positive cases (predicted unregistered proxy-authored messages) and true negatives (presumed patient-authored messages) were 0.681 and 0.960, respectively. CONCLUSION: Our work demonstrates the feasibility of using unstructured, heterogenous patient portal secure messages to determine portal secure message authorship. Identifying patient authorship in real time can improve patient portal account security and can be used to improve the quality of the information extracted from the patient portal, such as patient-reported outcomes.

The All of Us Research Program: Data quality, utility, and diversity.

The All of Us Research Program seeks to engage at least one million diverse participants to advance precision medicine and improve human health. We describe here the cloud-based Researcher Workbench that uses a data passport model to democratize access to analytical tools and participant information including survey, physical measurement, and electronic health record (EHR) data. We also present validation study findings for several common complex diseases to demonstrate use of this novel platform in 315,000 participants, 78% of whom are from groups historically underrepresented in biomedical research, including 49% self-reporting non-White races. Replication findings include medication usage pattern differences by race in depression and type 2 diabetes, validation of known cancer associations with smoking, and calculation of cardiovascular risk scores by reported race effects. The cloud-based Researcher Workbench represents an important advance in enabling secure access for a broad range of researchers to this large resource and analytical tools.

Classification and analysis of asynchronous communication content between care team members involved in breast cancer treatment.

OBJECTIVE: A growing research literature has highlighted the work of managing and triaging clinical messages as a major contributor to professional exhaustion and burnout. The goal of this study was to discover and quantify the distribution of message content sent among care team members treating patients with breast cancer. MATERIALS AND METHODS: We analyzed nearly two years of communication data from the electronic health record (EHR) between care team members at Vanderbilt University Medical Center. We applied natural language processing to perform sentence-level annotation into one of five information types: clinical, medical logistics, nonmedical logistics, social, and other. We combined sentence-level annotations for each respective message. We evaluated message content by team member role and clinic activity. RESULTS: Our dataset included 81 857 messages containing 613 877 sentences. Across all roles, 63.4% and 21.8% of messages contained logistical information and clinical information, respectively. Individuals in administrative or clinical staff roles sent 81% of all messages containing logistical information. There were 33.2% of messages sent by physicians containing clinical information-the most of any role. DISCUSSION AND CONCLUSION: Our results demonstrate that EHR-based asynchronous communication is integral to coordinate care for patients with breast cancer. By understanding the content of messages sent by care team members, we can devise informatics initiatives to improve physicians' clerical burden and reduce unnecessary interruptions.

Comparison of family health history in surveys vs electronic health record data mapped to the observational medical outcomes partnership data model in the All of Us Research Program.

OBJECTIVE: Family health history is important to clinical care and precision medicine. Prior studies show gaps in data collected from patient surveys and electronic health records (EHRs). The All of Us Research Program collects family history from participants via surveys and EHRs. This Demonstration Project aims to evaluate availability of family health history information within the publicly available data from All of Us and to characterize the data from both sources. MATERIALS AND METHODS: Surveys were completed by participants on an electronic portal. EHR data was mapped to the Observational Medical Outcomes Partnership data model. We used descriptive statistics to perform exploratory analysis of the data, including evaluating a list of medically actionable genetic disorders. We performed a subanalysis on participants who had both survey and EHR data. RESULTS: There were 54 872 participants with family history data. Of those, 26% had EHR data only, 63% had survey only, and 10.5% had data from both sources. There were 35 217 participants with reported family history of a medically actionable genetic disorder (9% from EHR only, 89% from surveys, and 2% from both). In the subanalysis, we found inconsistencies between the surveys and EHRs. More details came from surveys. When both mentioned a similar disease, the source of truth was unclear. CONCLUSIONS: Compiling data from both surveys and EHR can provide a more comprehensive source for family health history, but informatics challenges and opportunities exist. Access to more complete understanding of a person's family health history may provide opportunities for precision medicine.

Racial, ethnic, and gender differences in obesity and body fat distribution: An All of Us Research Program demonstration project.

Differences in obesity and body fat distribution across gender and race/ethnicity have been extensively described. We sought to replicate these differences and evaluate newly emerging data from the All of Us Research Program (AoU). We compared body mass index (BMI), waist circumference, and waist-to-hip ratio from the baseline physical examination, and alanine aminotransferase (ALT) from the electronic health record in up to 88,195 Non-Hispanic White (NHW), 40,770 Non-Hispanic Black (NHB), 35,640 Hispanic, and 5,648 Asian participants. We compared AoU sociodemographic variable distribution to National Health and Nutrition Examination Survey (NHANES) data and applied the pseudo-weighting method for adjusting selection biases of AoU recruitment. Our findings replicate previous observations with respect to gender differences in BMI. In particular, we replicate the large gender disparity in obesity rates among NHB participants, in which obesity and mean BMI are much higher in NHB women than NHB men (33.34 kg/m2 versus 28.40 kg/m2 respectively; p<2.22x10-308). The overall age-adjusted obesity prevalence in AoU participants is similar overall but lower than the prevalence found in NHANES for NHW participants. ALT was higher in men than women, and lower among NHB participants compared to other racial/ethnic groups, consistent with previous findings. Our data suggest consistency of AoU with national averages related to obesity and suggest this resource is likely to be a major source of scientific inquiry and discovery in diverse populations.

Analysis of Employee Patient Portal Use and Electronic Health Record Access at an Academic Medical Center.

BACKGROUND: Patient portals provide patients and their caregivers online access to limited health results. Health care employees with electronic health record (EHR) access may be able to view their health information not available in the patient portal by looking in the EHR. OBJECTIVE: In this study, we examine how employees use the patient portal when they also have access to the tethered EHR. METHODS: We obtained patient portal and EHR usage logs corresponding to all employees who viewed their health data at our institution between January 1, 2013 and November 1, 2017. We formed three cohorts based on the systems that employees used to view their health data: employees who used the patient portal only, employees who viewed health data in the EHR only, and employees who used both systems. We compared system accesses and usage patterns for each employee cohort. RESULTS: During the study period, 35,172 employees accessed the EHR as part of patients' treatment and 28,631 employees accessed their health data: 25,193 of them used the patient portal and 13,318 accessed their clinical data in EHR. All employees who accessed their records in the EHR viewed their clinical notes at least once. Among EHR accesses, clinical note accesses comprised more than 42% of all EHR accesses. Provider messaging and appointment scheduling were the most commonly used functions in the patient portal. Employees who had access to their health data in both systems were more likely to engage with providers through portal messages. CONCLUSION: Employees at a large medical center accessed clinical notes in the EHR to obtain information about their health. Employees also viewed other health data not readily available in the patient portal.