Accuracy of Electronic Medical Record Follow-Up Data for Estimating the Survival Time of Patients With Cancer.

PURPOSE: For real-world evidence, it is convenient to use routinely collected data from the electronic medical record (EMR) to measure survival outcomes. However, patients can become lost to follow-up, causing incomplete data and biased survival time estimates. We quantified this issue for patients with metastatic cancer seen in an academic health system by comparing survival estimates from EMR data only and from EMR data combined with high-quality cancer registry data. MATERIALS AND METHODS: Patients diagnosed with metastatic cancer from 2008 to 2014 were included in this retrospective study. Patients who were diagnosed with cancer or received their initial treatment within our system were included in the institutional cancer registry and this study. Overall survival was calculated using the Kaplan-Meier method. Survival curves were generated in two ways: using EMR follow-up data alone and using EMR data supplemented with data from the Stanford Cancer Registry/California Cancer Registry. RESULTS: Four thousand seventy-seven patients were included. The median follow-up using EMR + Cancer Registry data was 19.9 months, and the median follow-up in surviving patients was 67.6 months. There were 1,301 deaths recorded in the EMR and 3,140 deaths recorded in the Cancer Registry. The median overall survival from the date of cancer diagnosis using EMR data was 58.7 months (95% CI, 54.2 to 63.2); using EMR + Cancer Registry data, it was 20.8 months (95% CI, 19.6 to 22.3). A similar pattern was seen using the date of first systemic therapy or date of first hospital admission as the baseline date. CONCLUSION: Using EMR data alone, survival time was overestimated compared with EMR + Cancer Registry data.

Natural Language Processing to Identify Cancer Treatments With Electronic Medical Records.

PURPOSE: Knowing the treatments administered to patients with cancer is important for treatment planning and correlating treatment patterns with outcomes for personalized medicine study. However, existing methods to identify treatments are often lacking. We develop a natural language processing approach with structured electronic medical records and unstructured clinical notes to identify the initial treatment administered to patients with cancer. METHODS: We used a total number of 4,412 patients with 483,782 clinical notes from the Stanford Cancer Institute Research Database containing patients with nonmetastatic prostate, oropharynx, and esophagus cancer. We trained treatment identification models for each cancer type separately and compared performance of using only structured, only unstructured (bag-of-words, doc2vec, fasttext), and combinations of both (structured + bow, structured + doc2vec, structured + fasttext). We optimized the identification model among five machine learning methods (logistic regression, multilayer perceptrons, random forest, support vector machines, and stochastic gradient boosting). The treatment information recorded in the cancer registry is the gold standard and compares our methods to an identification baseline with billing codes. RESULTS: For prostate cancer, we achieved an f1-score of 0.99 (95% CI, 0.97 to 1.00) for radiation and 1.00 (95% CI, 0.99 to 1.00) for surgery using structured + doc2vec. For oropharynx cancer, we achieved an f1-score of 0.78 (95% CI, 0.58 to 0.93) for chemoradiation and 0.83 (95% CI, 0.69 to 0.95) for surgery using doc2vec. For esophagus cancer, we achieved an f1-score of 1.0 (95% CI, 1.0 to 1.0) for both chemoradiation and surgery using all combinations of structured and unstructured data. We found that employing the free-text clinical notes outperforms using the billing codes or only structured data for all three cancer types. CONCLUSION: Our results show that treatment identification using free-text clinical notes greatly improves upon the performance using billing codes and simple structured data. The approach can be used for treatment cohort identification and adapted for longitudinal cancer treatment identification.

Automated model versus treating physician for predicting survival time of patients with metastatic cancer.

OBJECTIVE: Being able to predict a patient's life expectancy can help doctors and patients prioritize treatments and supportive care. For predicting life expectancy, physicians have been shown to outperform traditional models that use only a few predictor variables. It is possible that a machine learning model that uses many predictor variables and diverse data sources from the electronic medical record can improve on physicians' performance. For patients with metastatic cancer, we compared accuracy of life expectancy predictions by the treating physician, a machine learning model, and a traditional model. MATERIALS AND METHODS: A machine learning model was trained using 14 600 metastatic cancer patients' data to predict each patient's distribution of survival time. Data sources included note text, laboratory values, and vital signs. From 2015-2016, 899 patients receiving radiotherapy for metastatic cancer were enrolled in a study in which their radiation oncologist estimated life expectancy. Survival predictions were also made by the machine learning model and a traditional model using only performance status. Performance was assessed with area under the curve for 1-year survival and calibration plots. RESULTS: The radiotherapy study included 1190 treatment courses in 899 patients. A total of 879 treatment courses in 685 patients were included in this analysis. Median overall survival was 11.7 months. Physicians, machine learning model, and traditional model had area under the curve for 1-year survival of 0.72 (95% CI 0.63-0.81), 0.77 (0.73-0.81), and 0.68 (0.65-0.71), respectively. CONCLUSIONS: The machine learning model's predictions were more accurate than those of the treating physician or a traditional model.

Automated Survival Prediction in Metastatic Cancer Patients Using High-Dimensional Electronic Medical Record Data.

BACKGROUND: Oncologists use patients' life expectancy to guide decisions and may benefit from a tool that accurately predicts prognosis. Existing prognostic models generally use only a few predictor variables. We used an electronic medical record dataset to train a prognostic model for patients with metastatic cancer. METHODS: The model was trained and tested using 12 588 patients treated for metastatic cancer in the Stanford Health Care system from 2008 to 2017. Data sources included provider note text, labs, vital signs, procedures, medication orders, and diagnosis codes. Patients were divided randomly into a training set used to fit the model coefficients and a test set used to evaluate model performance (80%/20% split). A regularized Cox model with 4126 predictor variables was used. A landmarking approach was used due to the multiple observations per patient, with t0 set to the time of metastatic cancer diagnosis. Performance was also evaluated using 399 palliative radiation courses in test set patients. RESULTS: The C-index for overall survival was 0.786 in the test set (averaged across landmark times). For palliative radiation courses, the C-index was 0.745 (95% confidence interval [CI] = 0.715 to 0.775) compared with 0.635 (95% CI = 0.601 to 0.669) for a published model using performance status, primary tumor site, and treated site (two-sided P < .001). Our model's predictions were well-calibrated. CONCLUSIONS: The model showed high predictive performance, which will need to be validated using external data. Because it is fully automated, the model can be used to examine providers' practice patterns and could be deployed in a decision support tool to help improve quality of care.

Increased Risk of Cutaneous Squamous Cell Carcinoma After Vismodegib Therapy for Basal Cell Carcinoma.

IMPORTANCE: Smoothened inhibitors (SIs) are a new type of targeted therapy for advanced basal cell carcinoma (BCC), and their long-term effects, such as increased risk of subsequent malignancy, are still being explored. OBJECTIVE: To evaluate the risk of developing a non-BCC malignancy after SI exposure in patients with BCC. DESIGN, SETTING, AND PARTICIPANTS: A case-control study at Stanford Medical Center, an academic hospital. Participants were higher-risk patients with BCC diagnosed from January 1, 1998, to December 31, 2014. The dates of the analysis were January 1 to November 1, 2015. EXPOSURES: The exposed participants (cases) comprised patients who had confirmed prior vismodegib treatment, and the nonexposed participants (controls) comprised patients who had never received any SI. Because vismodegib was the first approved SI, only patients exposed to this SI were included. MAIN OUTCOMES AND MEASURES: Hazard ratio for non-BCC malignancies after vismodegib exposure, adjusting for covariates. RESULTS: The study cohort comprised 180 participants. Their mean (SD) age at BCC diagnosis was 56 (16) years, and 68.9% (n = 124) were male. Fifty-five cases were compared with 125 controls, accounting for age, sex, prior radiation therapy or cisplatin treatment, Charlson Comorbidity Index, clinical follow-up time, immunosuppression, and basal cell nevus syndrome status. Patients exposed to vismodegib had a hazard ratio of 6.37 (95% CI, 3.39-11.96; P < .001), indicating increased risk of developing a non-BCC malignancy. Most non-BCC malignancies were cutaneous squamous cell carcinomas, with a hazard ratio of 8.12 (95% CI, 3.89-16.97; P < .001), accounting for age and basal cell nevus syndrome status. There was no significant increase in other cancers. CONCLUSIONS AND RELEVANCE: Increased risk for cutaneous squamous cell carcinomas after vismodegib therapy highlights the importance of continued skin surveillance after initiation of this therapy.