Uncovering interpretable potential confounders in electronic medical records.

Randomized clinical trials (RCT) are the gold standard for informing treatment decisions. Observational studies are often plagued by selection bias, and expert-selected covariates may insufficiently adjust for confounding. We explore how unstructured clinical text can be used to reduce selection bias and improve medical practice. We develop a framework based on natural language processing to uncover interpretable potential confounders from text. We validate our method by comparing the estimated hazard ratio (HR) with and without the confounders against established RCTs. We apply our method to four cohorts built from localized prostate and lung cancer datasets from the Stanford Cancer Institute and show that our method shifts the HR estimate towards the RCT results. The uncovered terms can also be interpreted by oncologists for clinical insights. We present this proof-of-concept study to enable more credible causal inference using observational data, uncover meaningful insights from clinical text, and inform high-stakes medical decisions.

Natural Language Processing to Identify Cancer Treatments With Electronic Medical Records.

PURPOSE: Knowing the treatments administered to patients with cancer is important for treatment planning and correlating treatment patterns with outcomes for personalized medicine study. However, existing methods to identify treatments are often lacking. We develop a natural language processing approach with structured electronic medical records and unstructured clinical notes to identify the initial treatment administered to patients with cancer. METHODS: We used a total number of 4,412 patients with 483,782 clinical notes from the Stanford Cancer Institute Research Database containing patients with nonmetastatic prostate, oropharynx, and esophagus cancer. We trained treatment identification models for each cancer type separately and compared performance of using only structured, only unstructured (bag-of-words, doc2vec, fasttext), and combinations of both (structured + bow, structured + doc2vec, structured + fasttext). We optimized the identification model among five machine learning methods (logistic regression, multilayer perceptrons, random forest, support vector machines, and stochastic gradient boosting). The treatment information recorded in the cancer registry is the gold standard and compares our methods to an identification baseline with billing codes. RESULTS: For prostate cancer, we achieved an f1-score of 0.99 (95% CI, 0.97 to 1.00) for radiation and 1.00 (95% CI, 0.99 to 1.00) for surgery using structured + doc2vec. For oropharynx cancer, we achieved an f1-score of 0.78 (95% CI, 0.58 to 0.93) for chemoradiation and 0.83 (95% CI, 0.69 to 0.95) for surgery using doc2vec. For esophagus cancer, we achieved an f1-score of 1.0 (95% CI, 1.0 to 1.0) for both chemoradiation and surgery using all combinations of structured and unstructured data. We found that employing the free-text clinical notes outperforms using the billing codes or only structured data for all three cancer types. CONCLUSION: Our results show that treatment identification using free-text clinical notes greatly improves upon the performance using billing codes and simple structured data. The approach can be used for treatment cohort identification and adapted for longitudinal cancer treatment identification.

A Probabilistic Model to Support Radiologists' Classification Decisions in Mammography Practice.

We developed a probabilistic model to support the classification decisions made by radiologists in mammography practice. Using the feature observations and Breast Imaging Reporting and Data System (BI-RADS) classifications from radiologists examining diagnostic and screening mammograms, we modeled their decisions to understand their judgments. Our model could help improve the decisions made by radiologists using their own feature observations and classifications while maintaining their observed sensitivities. Based on 112,433 mammographic cases from 36,111 patients and 13 radiologists at 2 separate institutions with a 1.1% prevalence of malignancy, we trained a probabilistic Bayesian network (BN) to estimate the malignancy probabilities of lesions. For each radiologist, we learned an observed probabilistic threshold within the model. We compared the sensitivity and specificity of each radiologist against the BN model using either their observed threshold or the standard 2% threshold recommended by BI-RADS. We found significant variability among the radiologists' observed thresholds. By applying the observed thresholds, the BN model showed a 0.01% (1 case) increase in false negatives and a 28.9% (3612 cases) reduction in false positives. When using the standard 2% BI-RADS-recommended threshold, there was a 26.7% (47 cases) increase in false negatives and a 47.3% (5911 cases) reduction in false positives. Our results show that we can significantly reduce screening mammography false positives with a minimal increase in false negatives. We find that learning radiologists' observed thresholds provides valuable information regarding the conservativeness of clinical practice and allows us to quantify the variability in sensitivity across and within institutions. Our model could provide support to radiologists to improve their performance and consistency within mammography practice.

Dynamic strategy for personalized medicine: An application to metastatic breast cancer.

We compare methods to develop an adaptive strategy for therapy choice in a class of breast cancer patients, as an example of approaches to personalize therapies for individual characteristics and each patient's response to therapy. Our model maintains a Markov belief about the effectiveness of the different therapies and updates it as therapies are administered and tumor images are observed, reflecting tumor response. We compare three different approximate methods to solve our analytical model against standard medical practice and show significant potential benefit of the computed dynamic strategies to limit tumor growth and to reduce the number of time periods patients are given chemotherapy, with its attendant side effects.

Cost-effectiveness of a potential prophylactic Helicobacter pylori vaccine in the United States.

BACKGROUND: Helicobacter pylori vaccines are under development to prevent infection. We quantified the cost-effectiveness of such a vaccine in the United States, using a dynamic transmission model. METHODS: We compartmentalized the population by age, infection status, and clinical disease state and measured effectiveness in quality-adjusted life years (QALYs). We simulated no intervention, vaccination of infants, and vaccination of school-age children. Variables included costs of vaccine, vaccine administration, and gastric cancer treatment (in 2007 US dollars), vaccine efficacy, quality adjustment due to gastric cancer, and discount rate. We evaluated possible outcomes for periods of 10-75 years. RESULTS: H. pylori vaccination of infants would cost $2.9 billion over 10 years; savings from cancer prevention would be realized decades later. Over a long time horizon (75 years), incremental costs of H. pylori vaccination would be $1.8 billion, and incremental QALYs would be 0.5 million, yielding a cost-effectiveness ratio of $3871/QALY. With school-age vaccination, the cost-effectiveness ratio would be $22,137/QALY. With time limited to <40 years, the cost-effectiveness ratio exceeded $50,000/QALY. CONCLUSION: When evaluated with a time horizon beyond 40 years, the use of a prophylactic H. pylori vaccine was cost-effective in the United States, especially with infant vaccination.

Are patients getting the "gist" in risk communication? Patient understanding of prognosis in breast cancer treatment.

BACKGROUND: Many oncologists consult the Adjuvant! prognostic model to communicate risk with breast cancer patients; however, little is known about how effective that communication is. METHODS: The authors analyzed this small data set featuring 20 breast cancer patients' risk estimates, focusing on rankings or gist of the estimates. RESULTS: Overall, there was no gain in the accuracy of patient rankings. The number of patients with more accurate estimates was matched by the number of patients with less accurate estimates after consultation. CONCLUSIONS: The current methods used by oncologists to present Adjuvant! risks were not effective in helping patients to get the gist of their risks.

Bayesian network to predict breast cancer risk of mammographic microcalcifications and reduce number of benign biopsy results: initial experience.

PURPOSE: To retrospectively determine whether a Bayesian network (BN) computer model can accurately predict the probability of breast cancer on the basis of risk factors and mammographic appearance of microcalcifications, to improve the positive predictive value (PPV) of biopsy, with pathologic examination and follow-up as reference standards. MATERIALS AND METHODS: The institutional review board approved this HIPAA-compliant study; informed consent was not required. Results of 111 consecutive image-guided breast biopsies performed for microcalcifications deemed suspicious by radiologists were analyzed. Mammograms obtained before biopsy were analyzed in a blinded manner by a breast imager who recorded Breast Imaging Reporting and Data System (BI-RADS) descriptors and provided a probability of malignancy. The BN uses probabilistic relationships between breast disease and mammography findings to estimate the risk of malignancy. Probability estimates from the radiologist and the BN were used to create receiver operating characteristic (ROC) curves, and area under the ROC curve (A(z)) values were compared. PPV of biopsy was also evaluated on the basis of these probability estimates. RESULTS: The BN and the radiologist achieved A(z) values of 0.919 and 0.916, respectively, which were not significantly different. If the 34 patients estimated by the BN to have less than a 10% probability of malignancy had not undergone biopsy, the PPV of biopsy would have increased from 21.6% to 31.2% without missing a breast cancer (P < .001). At this level, the radiologist's probability estimation improved the PPV to 30.0% (P < .001). CONCLUSION: A probabilistic model that includes BI-RADS descriptors for microcalcifications can distinguish between benign and malignant abnormalities at mammography as well as a breast imaging specialist can and may be able to improve the PPV of image-guided breast biopsy.

Using a Bayesian network to predict the probability and type of breast cancer represented by microcalcifications on mammography.

Since the widespread adoption of mammographic screening in the 1980's there has been a significant increase in the detection and biopsy of both benign and malignant microcalcifications. Though current practice standards recommend that the positive predictive value (PPV) of breast biopsy should be in the range of 25-40%, there exists significant variability in practice. Microcalcifications, if malignant, can represent either a non-invasive or an invasive form of breast cancer. The distinction is critical because distinct surgical therapies are indicated. Unfortunately, this information is not always available at the time of surgery due to limited sampling at image-guided biopsy. For these reasons we conducted an experiment to determine whether a previously created Bayesian network for mammography could predict the significance of microcalcifications. In this experiment we aim to test whether the system is able to perform two related tasks in this domain: 1) to predict the likelihood that microcalcifications are malignant and 2) to predict the likelihood that a malignancy is invasive to help guide the choice of appropriate surgical therapy.

A probabilistic expert system that provides automated mammographic-histologic correlation: initial experience.

OBJECTIVE: We sought to determine whether a probabilistic expert system can provide accurate automated imaging-histologic correlations to aid radiologists in assessing the concordance of mammographic findings with the results of imaging-guided breast biopsies. MATERIALS AND METHODS: We created a Bayesian network in which Breast Imaging Reporting and Data System (BI-RADS) descriptors are used to convey the level of suspicion of mammographic abnormalities. Our system is a computer model that links BI-RADS descriptors with diseases of the breast using probabilities derived from the literature. Mammographic findings are used to update pretest probabilities (prevalence of disease) into posttest probabilities applying Bayes' theorem. We evaluated the histologic results of 92 consecutive imaging-guided breast biopsies for concordance with the mammographic findings during radiology-pathology review sessions. First, radiologists with no knowledge of the biopsy results chose BI-RADS descriptors for the mammographic findings. After the histologic diagnosis was revealed, the radiologists assessed concordance between the pathologic results and the mammographic findings. We then input the information gathered from these sessions into the Bayesian network to produce an automated mammographic-histologic correlation. RESULTS: We had a sampling error rate of 1.1% (1/92 biopsies). Our expert system was able to integrate pathologic diagnoses and mammographic findings to obtain probabilities of sampling error, thereby enabling us to identify the incorrect pathologic diagnosis with 100% sensitivity while maintaining a specificity of 91%. CONCLUSION: Our probabilistic expert system has the potential to help radiologists in identifying breast biopsy results that are discordant with mammographic findings and discovering cases in which biopsy sampling errors may have occurred.