Systematic replication of smoking disease associations using survey responses and EHR data in the All of Us Research Program.

OBJECTIVE: The All of Us Research Program (All of Us) aims to recruit over a million participants to further precision medicine. Essential to the verification of biobanks is a replication of known associations to establish validity. Here, we evaluated how well All of Us data replicated known cigarette smoking associations. MATERIALS AND METHODS: We defined smoking exposure as follows: (1) an EHR Smoking exposure that used International Classification of Disease codes; (2) participant provided information (PPI) Ever Smoking; and, (3) PPI Current Smoking, both from the lifestyle survey. We performed a phenome-wide association study (PheWAS) for each smoking exposure measurement type. For each, we compared the effect sizes derived from the PheWAS to published meta-analyses that studied cigarette smoking from PubMed. We defined two levels of replication of meta-analyses: (1) nominally replicated: which required agreement of direction of effect size, and (2) fully replicated: which required overlap of confidence intervals. RESULTS: PheWASes with EHR Smoking, PPI Ever Smoking, and PPI Current Smoking revealed 736, 492, and 639 phenome-wide significant associations, respectively. We identified 165 meta-analyses representing 99 distinct phenotypes that could be matched to EHR phenotypes. At P < .05, 74 were nominally replicated and 55 were fully replicated. At P < 2.68 × 10-5 (Bonferroni threshold), 58 were nominally replicated and 40 were fully replicated. DISCUSSION: Most phenotypes found in published meta-analyses associated with smoking were nominally replicated in All of Us. Both survey and EHR definitions for smoking produced similar results. CONCLUSION: This study demonstrated the feasibility of studying common exposures using All of Us data.

Identifying Nonpatient Authors of Patient Portal Secure Messages in Oncology: A Proof-of-Concept Demonstration of Natural Language Processing Methods.

PURPOSE: Patient portal secure messages are not always authored by the patient account holder. Understanding who authored the message is particularly important in an oncology setting where symptom reporting is crucial to patient treatment. Natural language processing has the potential to detect messages not authored by the patient automatically. METHODS: Patient portal secure messages from the Memorial Sloan Kettering Cancer Center were retrieved and manually annotated as a predicted unregistered proxy (ie, not written by the patient) or a presumed patient. After randomly splitting the annotated messages into training and test sets in a 70:30 ratio, a bag-of-words approach was used to extract features and then a Least Absolute Shrinkage and Selection Operator (LASSO) model was trained and used for classification. RESULTS: Portal secure messages (n = 2,000) were randomly selected from unique patient accounts and manually annotated. We excluded 335 messages from the data set as the annotators could not determine if they were written by a patient or proxy. Using the remaining 1,665 messages, a LASSO model was developed that achieved an area under the curve of 0.932 and an area under the precision recall curve of 0.748. The sensitivity and specificity related to classifying true-positive cases (predicted unregistered proxy-authored messages) and true negatives (presumed patient-authored messages) were 0.681 and 0.960, respectively. CONCLUSION: Our work demonstrates the feasibility of using unstructured, heterogenous patient portal secure messages to determine portal secure message authorship. Identifying patient authorship in real time can improve patient portal account security and can be used to improve the quality of the information extracted from the patient portal, such as patient-reported outcomes.

The All of Us Research Program: Data quality, utility, and diversity.

The All of Us Research Program seeks to engage at least one million diverse participants to advance precision medicine and improve human health. We describe here the cloud-based Researcher Workbench that uses a data passport model to democratize access to analytical tools and participant information including survey, physical measurement, and electronic health record (EHR) data. We also present validation study findings for several common complex diseases to demonstrate use of this novel platform in 315,000 participants, 78% of whom are from groups historically underrepresented in biomedical research, including 49% self-reporting non-White races. Replication findings include medication usage pattern differences by race in depression and type 2 diabetes, validation of known cancer associations with smoking, and calculation of cardiovascular risk scores by reported race effects. The cloud-based Researcher Workbench represents an important advance in enabling secure access for a broad range of researchers to this large resource and analytical tools.

Comparing medical history data derived from electronic health records and survey answers in the All of Us Research Program.

OBJECTIVE: A participant's medical history is important in clinical research and can be captured from electronic health records (EHRs) and self-reported surveys. Both can be incomplete, EHR due to documentation gaps or lack of interoperability and surveys due to recall bias or limited health literacy. This analysis compares medical history collected in the All of Us Research Program through both surveys and EHRs. MATERIALS AND METHODS: The All of Us medical history survey includes self-report questionnaire that asks about diagnoses to over 150 medical conditions organized into 12 disease categories. In each category, we identified the 3 most and least frequent self-reported diagnoses and retrieved their analogues from EHRs. We calculated agreement scores and extracted participant demographic characteristics for each comparison set. RESULTS: The 4th All of Us dataset release includes data from 314 994 participants; 28.3% of whom completed medical history surveys, and 65.5% of whom had EHR data. Hearing and vision category within the survey had the highest number of responses, but the second lowest positive agreement with the EHR (0.21). The Infectious disease category had the lowest positive agreement (0.12). Cancer conditions had the highest positive agreement (0.45) between the 2 data sources. DISCUSSION AND CONCLUSION: Our study quantified the agreement of medical history between 2 sources-EHRs and self-reported surveys. Conditions that are usually undocumented in EHRs had low agreement scores, demonstrating that survey data can supplement EHR data. Disagreement between EHR and survey can help identify possible missing records and guide researchers to adjust for biases.

Classification and analysis of asynchronous communication content between care team members involved in breast cancer treatment.

OBJECTIVE: A growing research literature has highlighted the work of managing and triaging clinical messages as a major contributor to professional exhaustion and burnout. The goal of this study was to discover and quantify the distribution of message content sent among care team members treating patients with breast cancer. MATERIALS AND METHODS: We analyzed nearly two years of communication data from the electronic health record (EHR) between care team members at Vanderbilt University Medical Center. We applied natural language processing to perform sentence-level annotation into one of five information types: clinical, medical logistics, nonmedical logistics, social, and other. We combined sentence-level annotations for each respective message. We evaluated message content by team member role and clinic activity. RESULTS: Our dataset included 81 857 messages containing 613 877 sentences. Across all roles, 63.4% and 21.8% of messages contained logistical information and clinical information, respectively. Individuals in administrative or clinical staff roles sent 81% of all messages containing logistical information. There were 33.2% of messages sent by physicians containing clinical information-the most of any role. DISCUSSION AND CONCLUSION: Our results demonstrate that EHR-based asynchronous communication is integral to coordinate care for patients with breast cancer. By understanding the content of messages sent by care team members, we can devise informatics initiatives to improve physicians' clerical burden and reduce unnecessary interruptions.

Impact of High-Dose-Rate Brachytherapy Training via Telehealth in Low- and Middle-Income Countries.

PURPOSE: Our objective was to demonstrate the efficacy of a telehealth training course on high-dose-rate (HDR) brachytherapy for gynecologic cancer treatment for clinicians in low- and middle-income countries (LMICs). METHODS: A 12-week course consisting of 16 live video sessions was offered to 10 cancer centers in the Middle East, Africa, and Nepal. A total of 46 participants joined the course, and 22 participants, on average, attended each session. Radiation oncologists and medical physicists from 11 US and international institutions prepared and provided lectures for each topic covered in the course. Confidence surveys of 15 practical competencies were administered to participants before and after the course. Competencies focused on HDR commissioning, shielding, treatment planning, radiobiology, and applicators. Pre- and post-program surveys of provider confidence, measured by 5-point Likert scale, were administered and compared. RESULTS: Forty-six participants, including seven chief medical physicists, 16 senior medical physicists, five radiation oncologists, and three dosimetrists, representing nine countries attended education sessions. Reported confidence scores, both aggregate and paired, demonstrated increases in confidence in all 15 competencies. Post-curriculum score improvement was statistically significant (P < .05) for paired respondents in 11 of 15 domains. Absolute improvements were largest for confidence in applicator commissioning (2.3 to 3.8, P = .009), treatment planning system commissioning (2.2 to 3.9, P = .0055), and commissioning an HDR machine (2.2 to 4.0, P = .0031). Overall confidence in providing HDR brachytherapy services safely and teaching other providers increased from 3.1 to 3.8 and 3.0 to 3.5, respectively. CONCLUSION: A 12-week, low-cost telehealth training program on HDR brachytherapy improved confidence in treatment delivery and teaching for clinicians in 10 participating LMICs.

A systematic literature review of machine learning in online personal health data.

OBJECTIVE: User-generated content (UGC) in online environments provides opportunities to learn an individual's health status outside of clinical settings. However, the nature of UGC brings challenges in both data collecting and processing. The purpose of this study is to systematically review the effectiveness of applying machine learning (ML) methodologies to UGC for personal health investigations. MATERIALS AND METHODS: We searched PubMed, Web of Science, IEEE Library, ACM library, AAAI library, and the ACL anthology. We focused on research articles that were published in English and in peer-reviewed journals or conference proceedings between 2010 and 2018. Publications that applied ML to UGC with a focus on personal health were identified for further systematic review. RESULTS: We identified 103 eligible studies which we summarized with respect to 5 research categories, 3 data collection strategies, 3 gold standard dataset creation methods, and 4 types of features applied in ML models. Popular off-the-shelf ML models were logistic regression (n = 22), support vector machines (n = 18), naive Bayes (n = 17), ensemble learning (n = 12), and deep learning (n = 11). The most investigated problems were mental health (n = 39) and cancer (n = 15). Common health-related aspects extracted from UGC were treatment experience, sentiments and emotions, coping strategies, and social support. CONCLUSIONS: The systematic review indicated that ML can be effectively applied to UGC in facilitating the description and inference of personal health. Future research needs to focus on mitigating bias introduced when building study cohorts, creating features from free text, improving clinical creditability of UGC, and model interpretability.

Cost-utility of osteoarticular allograft versus endoprosthetic reconstruction for primary bone sarcoma of the knee: A markov analysis.

BACKGROUND: The most cost-effective reconstruction after resection of bone sarcoma is unknown. The goal of this study was to compare the cost effectiveness of osteoarticular allograft to endoprosthetic reconstruction of the proximal tibia or distal femur. METHODS: A Markov model was used. Revision and complication rates were taken from existing studies. Costs were based on Medicare reimbursement rates and implant prices. Health-state utilities were derived from the Health Utilities Index 3 survey with additional assumptions. Incremental cost-effectiveness ratios (ICER) were used with less than $100 000 per quality-adjusted life year (QALY) considered cost-effective. Sensitivity analyses were performed for comparison over a range of costs, utilities, complication rates, and revisions rates. RESULTS: Osteoarticular allografts, and a 30% price-discounted endoprosthesis were cost-effective with ICERs of $92.59 and $6 114.77. One-way sensitivity analysis revealed discounted endoprostheses were favored if allografts cost over $21 900 or endoprostheses cost less than $51 900. Allograft reconstruction was favored over discounted endoprosthetic reconstruction if the allograft complication rate was less than 1.3%. Allografts were more cost-effective than full-price endoprostheses. CONCLUSIONS: Osteoarticular allografts and price-discounted endoprosthetic reconstructions are cost-effective. Sensitivity analysis, using plausible complication and revision rates, favored the use of discounted endoprostheses over allografts. Allografts are more cost-effective than full-price endoprostheses.