Sequence Labeling for Disambiguating Medical Abbreviations.

Abbreviations are unavoidable yet critical parts of the medical text. Using abbreviations, especially in clinical patient notes, can save time and space, protect sensitive information, and help avoid repetitions. However, most abbreviations might have multiple senses, and the lack of a standardized mapping system makes disambiguating abbreviations a difficult and time-consuming task. The main objective of this study is to examine the feasibility of sequence labeling methods for medical abbreviation disambiguation. Specifically, we explore the capability of sequence labeling methods to deal with multiple unique abbreviations in a single text. We use two public datasets to compare and contrast the performance of several transformer models pre-trained on different scientific and medical corpora. Our proposed sequence labeling approach outperforms the more commonly used text classification models for the abbreviation disambiguation task. In particular, the SciBERT model shows a strong performance for both sequence labeling and text classification tasks over the two considered datasets. Furthermore, we find that abbreviation disambiguation performance for the text classification models becomes comparable to that of sequence labeling only when postprocessing is applied to their predictions, which involves filtering possible labels for an abbreviation based on the training data.

Active Learning for Multi-way Sensitivity Analysis with Application to Disease Screening Modeling.

Sensitivity analysis is an important aspect of model development as it can be used to assess the level of confidence that is associated with the outcomes of a study. In many practical problems, sensitivity analysis involves evaluating a large number of parameter combinations which may require an extensive amount of time and resources. However, such a computational burden can be avoided by identifying smaller subsets of parameter combinations that can be later used to generate the desired outcomes for other parameter combinations. In this study, we investigate machine learning-based approaches for speeding up the sensitivity analysis. Furthermore, we apply feature selection methods to identify the relative importance of quantitative model parameters in terms of their predictive ability on the outcomes. Finally, we highlight the effectiveness of active learning strategies in improving the sensitivity analysis processes by reducing the total number of quantitative model runs required to construct a high-performance prediction model. Our experiments on two datasets obtained from the sensitivity analysis performed for two disease screening modeling studies indicate that ensemble methods such as Random Forests and XGBoost consistently outperform other machine learning algorithms in the prediction task of the associated sensitivity analysis. In addition, we note that active learning can lead to significant speed-ups in sensitivity analysis by enabling the selection of more useful parameter combinations (i.e., instances) to be used for prediction models.

Auto Response Generation in Online Medical Chat Services.

Telehealth helps to facilitate access to medical professionals by enabling remote medical services for the patients. These services have become gradually popular over the years with the advent of necessary technological infrastructure. The benefits of telehealth have been even more apparent since the beginning of the COVID-19 crisis, as people have become less inclined to visit doctors in person during the pandemic. In this paper, we focus on facilitating chat sessions between a doctor and a patient. We note that the quality and efficiency of the chat experience can be critical as the demand for telehealth services increases. Accordingly, we develop a smart auto-response generation mechanism for medical conversations that helps doctors respond to consultation requests efficiently, particularly during busy sessions. We explore over 900,000 anonymous, historical online messages between doctors and patients collected over 9 months. We implement clustering algorithms to identify the most frequent responses by doctors and manually label the data accordingly. We then train machine learning algorithms using this preprocessed data to generate the responses. The considered algorithm has two steps: a filtering (i.e., triggering) model to filter out infeasible patient messages and a response generator to suggest the top-3 doctor responses for the ones that successfully pass the triggering phase. Among the models utilized, BERT provides an accuracy of 85.41% for precision@3 and shows robustness to its parameters.

Word-level text highlighting of medical texts for telehealth services.

The medical domain is often subject to information overload. The digitization of healthcare, constant updates to online medical repositories, and increasing availability of biomedical datasets make it challenging to effectively analyze the data. This creates additional workload for medical professionals who are heavily dependent on medical data to complete their research and consult their patients. This paper aims to show how different text highlighting techniques can capture relevant medical context. This would reduce the doctors' cognitive load and response time to patients by facilitating them in making faster decisions, thus improving the overall quality of online medical services. Three different word-level text highlighting methodologies are implemented and evaluated. The first method uses Term Frequency - Inverse Document Frequency (TF-IDF) scores directly to highlight important parts of the text. The second method is a combination of TF-IDF scores, Word2Vec and the application of Local Interpretable Model-Agnostic Explanations to classification models. The third method uses neural networks directly to make predictions on whether or not a word should be highlighted. Our numerical study shows that the neural network approach is successful in highlighting medically-relevant terms and its performance is improved as the size of the input segment increases.

Evaluation of interpretability methods for multivariate time series forecasting.

Being able to interpret a model's predictions is a crucial task in many machine learning applications. Specifically, local interpretability is important in determining why a model makes particular predictions. Despite the recent focus on interpretable Artificial Intelligence (AI), there have been few studies on local interpretability methods for time series forecasting, while existing approaches mainly focus on time series classification tasks. In this study, we propose two novel evaluation metrics for time series forecasting: Area Over the Perturbation Curve for Regression and Ablation Percentage Threshold. These two metrics can measure the local fidelity of local explanation methods. We extend the theoretical foundation to collect experimental results on four popular datasets. Both metrics enable a comprehensive comparison of numerous local explanation methods, and an intuitive approach to interpret model predictions. Lastly, we provide heuristical reasoning for this analysis through an extensive numerical study.

Comparing CISNET Breast Cancer Models Using the Maximum Clinical Incidence Reduction Methodology.

BACKGROUND: Collaborative modeling has been used to estimate the impact of potential cancer screening strategies worldwide. A necessary step in the interpretation of collaborative cancer screening model results is to understand how model structure and model assumptions influence cancer incidence and mortality predictions. In this study, we examined the relative contributions of the pre-clinical duration of breast cancer, the sensitivity of screening, and the improvement in prognosis associated with treatment of screen-detected cases to the breast cancer incidence and mortality predictions of 5 Cancer Intervention and Surveillance Modeling Network (CISNET) models. METHODS: To tease out the impact of model structure and assumptions on model predictions, the Maximum Clinical Incidence Reduction (MCLIR) method compares changes in the number of breast cancers diagnosed due to clinical symptoms and cancer mortality between 4 simplified scenarios: 1) no-screening; 2) one-time perfect screening exam, which detects all existing cancers and perfect treatment (i.e., cure) of all screen-detected cancers; 3) one-time digital mammogram and perfect treatment of all screen-detected cancers; and 4) one-time digital mammogram and current guideline-concordant treatment of all screen-detected cancers. RESULTS: The 5 models predicted a large range in maximum clinical incidence (19% to 71%) and in breast cancer mortality reduction (33% to 67%) from a one-time perfect screening test and perfect treatment. In this perfect scenario, the models with assumptions of tumor inception before it is first detectable by mammography predicted substantially higher incidence and mortality reductions than models with assumptions of tumor onset at the start of a cancer's screen-detectable phase. The range across models in breast cancer clinical incidence (11% to 24%) and mortality reduction (8% to 18%) from a one-time digital mammogram at age 62 y with observed sensitivity and current guideline-concordant treatment was considerably smaller than achievable under perfect conditions. CONCLUSIONS: The timing of tumor inception and its effect on the length of the pre-clinical phase of breast cancer had a substantial impact on the grouping of models based on their predictions for clinical incidence and breast cancer mortality reduction. This key finding about the timing of tumor inception will be included in future CISNET breast analyses to enhance model transparency. The MCLIR approach should aid in the interpretation of variations in model results and could be adopted in other disease screening settings to enhance model transparency.

The University of Wisconsin Breast Cancer Epidemiology Simulation Model: An Update.

The University of Wisconsin Breast Cancer Epidemiology Simulation Model (UWBCS), also referred to as Model W, is a discrete-event microsimulation model that uses a systems engineering approach to replicate breast cancer epidemiology in the US over time. This population-based model simulates the lifetimes of individual women through 4 main model components: breast cancer natural history, detection, treatment, and mortality. A key feature of the UWBCS is that, in addition to specifying a population distribution in tumor growth rates, the model allows for heterogeneity in tumor behavior, with some tumors having limited malignant potential (i.e., would never become fatal in a woman's lifetime if left untreated) and some tumors being very aggressive based on metastatic spread early in their onset. The model is calibrated to Surveillance, Epidemiology, and End Results (SEER) breast cancer incidence and mortality data from 1975 to 2010, and cross-validated against data from the Wisconsin cancer reporting system. The UWBCS model generates detailed outputs including underlying disease states and observed clinical outcomes by age and calendar year, as well as costs, resource usage, and quality of life associated with screening and treatment. The UWBCS has been recently updated to account for differences in breast cancer detection, treatment, and survival by molecular subtypes (defined by ER/HER2 status), to reflect the recent advances in screening and treatment, and to consider a range of breast cancer risk factors, including breast density, race, body-mass-index, and the use of postmenopausal hormone therapy. Therefore, the model can evaluate novel screening strategies, such as risk-based screening, and can assess breast cancer outcomes by breast cancer molecular subtype. In this article, we describe the most up-to-date version of the UWBCS.

Using Active Learning for Speeding up Calibration in Simulation Models.

BACKGROUND: Most cancer simulation models include unobservable parameters that determine disease onset and tumor growth. These parameters play an important role in matching key outcomes such as cancer incidence and mortality, and their values are typically estimated via a lengthy calibration procedure, which involves evaluating a large number of combinations of parameter values via simulation. The objective of this study is to demonstrate how machine learning approaches can be used to accelerate the calibration process by reducing the number of parameter combinations that are actually evaluated. METHODS: Active learning is a popular machine learning method that enables a learning algorithm such as artificial neural networks to interactively choose which parameter combinations to evaluate. We developed an active learning algorithm to expedite the calibration process. Our algorithm determines the parameter combinations that are more likely to produce desired outputs and therefore reduces the number of simulation runs performed during calibration. We demonstrate our method using the previously developed University of Wisconsin breast cancer simulation model (UWBCS). RESULTS: In a recent study, calibration of the UWBCS required the evaluation of 378 000 input parameter combinations to build a race-specific model, and only 69 of these combinations produced results that closely matched observed data. By using the active learning algorithm in conjunction with standard calibration methods, we identify all 69 parameter combinations by evaluating only 5620 of the 378 000 combinations. CONCLUSION: Machine learning methods hold potential in guiding model developers in the selection of more promising parameter combinations and hence speeding up the calibration process. Applying our machine learning algorithm to one model shows that evaluating only 1.49% of all parameter combinations would be sufficient for the calibration.

Comparative effectiveness of combined digital mammography and tomosynthesis screening for women with dense breasts.

PURPOSE: To evaluate the effectiveness of combined biennial digital mammography and tomosynthesis screening, compared with biennial digital mammography screening alone, among women with dense breasts. MATERIALS AND METHODS: An established, discrete-event breast cancer simulation model was used to estimate the comparative clinical effectiveness and cost-effectiveness of biennial screening with both digital mammography and tomosynthesis versus digital mammography alone among U.S. women aged 50-74 years with dense breasts from a federal payer perspective and a lifetime horizon. Input values were estimated for test performance, costs, and health state utilities from the National Cancer Institute Breast Cancer Surveillance Consortium, Medicare reimbursement rates, and medical literature. Sensitivity analyses were performed to determine the implications of varying key model parameters, including combined screening sensitivity and specificity, transient utility decrement of diagnostic work-up, and additional cost of tomosynthesis. RESULTS: For the base-case analysis, the incremental cost per quality-adjusted life year gained by adding tomosynthesis to digital mammography screening was $53 893. An additional 0.5 deaths were averted and 405 false-positive findings avoided per 1000 women after 12 rounds of screening. Combined screening remained cost-effective (less than $100 000 per quality-adjusted life year gained) over a wide range of incremental improvements in test performance. Overall, cost-effectiveness was most sensitive to the additional cost of tomosynthesis. CONCLUSION: Biennial combined digital mammography and tomosynthesis screening for U.S. women aged 50-74 years with dense breasts is likely to be cost-effective if priced appropriately (up to $226 for combined examinations vs $139 for digital mammography alone) and if reported interpretive performance metrics of improved specificity with tomosynthesis are met in routine practice.

Benefits, harms, and cost-effectiveness of supplemental ultrasonography screening for women with dense breasts.

BACKGROUND: Many states have laws requiring mammography facilities to tell women with dense breasts and negative results on screening mammography to discuss supplemental screening tests with their providers. The most readily available supplemental screening method is ultrasonography, but little is known about its effectiveness. OBJECTIVE: To evaluate the benefits, harms, and cost-effectiveness of supplemental ultrasonography screening for women with dense breasts. DESIGN: Comparative modeling with 3 validated simulation models. DATA SOURCES: Surveillance, Epidemiology, and End Results Program; Breast Cancer Surveillance Consortium; and medical literature. TARGET POPULATION: Contemporary cohort of women eligible for routine screening. TIME HORIZON: Lifetime. PERSPECTIVE: Payer. INTERVENTION: Supplemental ultrasonography screening for women with dense breasts after a negative screening mammography result. OUTCOME MEASURES: Breast cancer deaths averted, quality-adjusted life-years (QALYs) gained, biopsies recommended after a false-positive ultrasonography result, and costs. RESULTS OF BASE-CASE ANALYSIS: Supplemental ultrasonography screening after a negative mammography result for women aged 50 to 74 years with heterogeneously or extremely dense breasts averted 0.36 additional breast cancer deaths (range across models, 0.14 to 0.75), gained 1.7 QALYs (range, 0.9 to 4.7), and resulted in 354 biopsy recommendations after a false-positive ultrasonography result (range, 345 to 421) per 1000 women with dense breasts compared with biennial screening by mammography alone. The cost-effectiveness ratio was $325,000 per QALY gained (range, $112,000 to $766,000). Supplemental ultrasonography screening for only women with extremely dense breasts cost $246,000 per QALY gained (range, $74,000 to $535,000). RESULTS OF SENSITIVITY ANALYSIS: The conclusions were not sensitive to ultrasonography performance characteristics, screening frequency, or starting age. LIMITATION: Provider costs for coordinating supplemental ultrasonography were not considered. CONCLUSION: Supplemental ultrasonography screening for women with dense breasts would substantially increase costs while producing relatively small benefits. PRIMARY FUNDING SOURCE: National Cancer Institute.

Benefits, harms, and costs for breast cancer screening after US implementation of digital mammography.

BACKGROUND: Compared with film, digital mammography has superior sensitivity but lower specificity for women aged 40 to 49 years and women with dense breasts. Digital has replaced film in virtually all US facilities, but overall population health and cost from use of this technology are unclear. METHODS: Using five independent models, we compared digital screening strategies starting at age 40 or 50 years applied annually, biennially, or based on density with biennial film screening from ages 50 to 74 years and with no screening. Common data elements included cancer incidence and test performance, both modified by breast density. Lifetime outcomes included mortality, quality-adjusted life-years, and screening and treatment costs. RESULTS: For every 1000 women screened biennially from age 50 to 74 years, switching to digital from film yielded a median within-model improvement of 2 life-years, 0.27 additional deaths averted, 220 additional false-positive results, and $0.35 million more in costs. For an individual woman, this translates to a health gain of 0.73 days. Extending biennial digital screening to women ages 40 to 49 years was cost-effective, although results were sensitive to quality-of-life decrements related to screening and false positives. Targeting annual screening by density yielded similar outcomes to targeting by age. Annual screening approaches could increase costs to $5.26 million per 1000 women, in part because of higher numbers of screens and false positives, and were not efficient or cost-effective. CONCLUSIONS: The transition to digital breast cancer screening in the United States increased total costs for small added health benefits. The value of digital mammography screening among women aged 40 to 49 years depends on women's preferences regarding false positives.