From Bench-to-Bedside: How Artificial Intelligence Is Changing Thyroid Nodule Diagnostics.

CONTEXT: Use of artificial intelligence (AI) to predict clinical outcomes in thyroid nodule diagnostics has grown exponentially over the past decade. The greatest challenge is in understanding the best model to apply to one's own patient population, and how to operationalize such a model in practice. EVIDENCE ACQUISITION: A literature search of PubMed and IEEE Xplore was conducted for English language publications between January 1, 2015 and January 1, 2023 studying diagnostic tests on suspected thyroid nodules that utilized AI. We excluded articles without prospective or external validation, non-primary literature, duplicates, focused on non-nodular thyroid conditions, not using AI, and those incidentally utilizing AI in support of an experimental diagnostic outside standard clinical practice. Quality was graded by Oxford level of evidence. EVIDENCE SYNTHESIS: A total of 61 studies were identified; all performed external validation, sixteen studies were prospective, and 33 compared a model to physician prediction of ground truth. Statistical validation was reported in 50 papers. A diagnostic pipeline was abstracted, yielding five high-level outcomes: (1) nodule localization, (2) ultrasound risk score, (3) molecular status, (4) malignancy, and (5) long-term prognosis. Seven prospective studies validated a single commercial AI; strengths included automating nodule feature assessment from ultrasound and assisting the physician in predicting malignancy risk, while weaknesses included automated margin prediction and inter-observer variability. CONCLUSIONS: Models predominantly used ultrasound images to predict malignancy. Of four FDA-approved products, only S-Detect was extensively validated. Implementing an AI model locally requires data sanitization and re-validation to ensure appropriate clinical performance.

A multitask approach for automated detection and segmentation of thyroid nodules in ultrasound images.

An increase in the incidence and diagnosis of thyroid nodules and thyroid cancer underscores the need for a better approach to nodule detection and risk stratification in ultrasound (US) images that can reduce healthcare costs, patient discomfort, and unnecessary invasive procedures. However, variability in ultrasound technique and interpretation makes the diagnostic process partially subjective. Therefore, an automated approach that detects and segments nodules could improve performance on downstream tasks, such as risk stratification. Ultrasound studies were acquired from 280 patients at UCLA Health, totaling 9888 images, and annotated by collaborating radiologists. Current deep learning architectures for segmentation are typically semi-automated because they are evaluated solely on images known to have nodules and do not assess ability to identify suspicious images. However, the proposed multitask approach both detects suspicious images and segments potential nodules; this allows for a clinically translatable model that aptly parallels the workflow for thyroid nodule assessment. The multitask approach is centered on an anomaly detection (AD) module that can be integrated with any UNet architecture variant to improve image-level nodule detection. Of the evaluated multitask models, a UNet with a ImageNet pretrained encoder and AD achieved the highest F1 score of 0.839 and image-wide Dice similarity coefficient of 0.808 on the hold-out test set. Furthermore, models were evaluated on two external validations datasets to demonstrate generalizability and robustness to data variability. Ultimately, the proposed architecture is an automated multitask method that expands on previous methods by successfully both detecting and segmenting nodules in ultrasound.

Optimizing detection and deep learning-based classification of pathological high-frequency oscillations in epilepsy.

OBJECTIVE: This study aimed to explore sensitive detection methods for pathological high-frequency oscillations (HFOs) to improve seizure outcomes in epilepsy surgery. METHODS: We analyzed interictal HFOs (80-500 Hz) in 15 children with medication-resistant focal epilepsy who underwent chronic intracranial electroencephalogram via subdural grids. The HFOs were assessed using the short-term energy (STE) and Montreal Neurological Institute (MNI) detectors and examined for spike association and time-frequency plot characteristics. A deep learning (DL)-based classification was applied to purify pathological HFOs. Postoperative seizure outcomes were correlated with HFO-resection ratios to determine the optimal HFO detection method. RESULTS: The MNI detector identified a higher percentage of pathological HFOs than the STE detector, but some pathological HFOs were detected only by the STE detector. HFOs detected by both detectors had the highest spike association rate. The Union detector, which detects HFOs identified by either the MNI or STE detector, outperformed other detectors in predicting postoperative seizure outcomes using HFO-resection ratios before and after DL-based purification. CONCLUSIONS: HFOs detected by standard automated detectors displayed different signal and morphological characteristics. DL-based classification effectively purified pathological HFOs. SIGNIFICANCE: Enhancing the detection and classification methods of HFOs will improve their utility in predicting postoperative seizure outcomes.

A Multitask Approach for Automated Detection and Segmentation of Thyroid Nodules in Ultrasound Images.

An increase in the incidence and diagnosis of thyroid nodules and thyroid cancer underscores the need for a better approach to nodule detection and risk stratification in ultrasound (US) images that can reduce healthcare costs, patient discomfort, and unnecessary invasive procedures. However, variability in ultrasound technique and interpretation makes the diagnostic process partially subjective. Therefore, an automated approach that detects and segments nodules could improve performance on downstream tasks, such as risk stratification.Current deep learning architectures for segmentation are typically semi-automated because they are evaluated solely on images known to have nodules and do not assess ability to identify suspicious images. However, the proposed multitask approach both detects suspicious images and segments potential nodules; this allows for a clinically translatable model that aptly parallels the workflow for thyroid nodule assessment. The multitask approach is centered on an anomaly detection (AD) module that can be integrated with any U-Net architecture variant to improve image-level nodule detection. Ultrasound studies were acquired from 280 patients at UCLA Health, totaling 9,888 images, and annotated by collaborating radiologists. Of the evaluated models, a multi-scale UNet (MSUNet) with AD achieved the highest F1 score of 0.829 and image-wide Dice similarity coefficient of 0.782 on our hold-out test set. Furthermore, models were evaluated on two external validations datasets to demonstrate generalizability and robustness to data variability. Ultimately, the proposed architecture is an automated multitask method that expands on previous methods by successfully both detecting and segmenting nodules in ultrasound.

Synthesizing MR Image Contrast Enhancement Using 3D High-Resolution ConvNets.

OBJECTIVE: Gadolinium-based contrast agents (GBCAs) have been widely used to better visualize disease in brain magnetic resonance imaging (MRI). However, gadolinium deposition within the brain and body has raised safety concerns about the use of GBCAs. Therefore, the development of novel approaches that can decrease or even eliminate GBCA exposure while providing similar contrast information would be of significant use clinically. METHODS: In this work, we present a deep learning based approach for contrast-enhanced T1 synthesis on brain tumor patients. A 3D high-resolution fully convolutional network (FCN), which maintains high resolution information through processing and aggregates multi-scale information in parallel, is designed to map pre-contrast MRI sequences to contrast-enhanced MRI sequences. Specifically, three pre-contrast MRI sequences, T1, T2 and apparent diffusion coefficient map (ADC), are utilized as inputs and the post-contrast T1 sequences are utilized as target output. To alleviate the data imbalance problem between normal tissues and the tumor regions, we introduce a local loss to improve the contribution of the tumor regions, which leads to better enhancement results on tumors. RESULTS: Extensive quantitative and visual assessments are performed, with our proposed model achieving a PSNR of 28.24 dB in the brain and 21.2 dB in tumor regions. CONCLUSION AND SIGNIFICANCE: Our results suggest the potential of substituting GBCAs with synthetic contrast images generated via deep learning.

Patient-level thyroid cancer classification using attention multiple instance learning on fused multi-scale ultrasound image features.

For patients with thyroid nodules, the ability to detect and diagnose a malignant nodule is the key to creating an appropriate treatment plan. However, assessments of ultrasound images do not accurately represent malignancy, and often require a biopsy to confirm the diagnosis. Deep learning techniques can classify thyroid nodules from ultrasound images, but current methods depend on manually annotated nodule segmentations. Furthermore, the heterogeneity in the level of magnification across ultrasound images presents a significant obstacle to existing methods. We developed a multi-scale, attention-based multiple-instance learning model which fuses both global and local features of different ultrasound frames to achieve patient-level malignancy classification. Our model demonstrates improved performance with an AUROC of 0.785 (p<0.05) and AUPRC of 0.539, significantly surpassing the baseline model trained on clinical features with an AUROC of 0.667 and AUPRC of 0.444. Improved classification performance better triages the need for biopsy.

Visualization of Fluoroscopic Imaging in Orthopedic Surgery: Head-Mounted Display vs Conventional Monitor.

Purpose. See-through head-mounted displays (HMDs) can be used to view fluoroscopic imaging during orthopedic surgical procedures. The goals of this study were to determine whether HMDs reduce procedure time, number of fluoroscopic images required, or number of head turns by the surgeon compared with standard monitors. Methods. Sixteen orthopedic surgery residents each performed fluoroscopy-guided drilling of 8 holes for placement of tibial nail distal interlocking screws in an anatomical model, with 4 holes drilled while using HMD and 4 holes drilled while using a standard monitor. Procedure time, number of fluoroscopic images needed, and number of head turns by the resident during the procedure were compared between the 2 modalities. Statistical significance was set at P < .05. Results. Mean (SD) procedure time did not differ significantly between attempts using the standard monitor (55 [37] seconds) vs the HMD (56 [31] seconds) (P = .73). Neither did mean number of fluoroscopic images differ significantly between attempts using the standard monitor vs the HMD (9 [5] images for each) (P = .84). Residents turned their heads significantly more times when using the standard monitor (9 [5] times) vs the HMD (1 [2] times) (P < .001). Conclusions. Head-mounted displays lessened the need for residents to turn their heads away from the surgical field while drilling holes for tibial nail distal interlocking screws in an anatomical model; however, there was no difference in terms of procedure time or number of fluoroscopic images needed using the HMD compared with the standard monitor.

PathAL: An Active Learning Framework for Histopathology Image Analysis.

Deep neural networks, in particular convolutional networks, have rapidly become a popular choice for analyzing histopathology images. However, training these models relies heavily on a large number of samples manually annotated by experts, which is cumbersome and expensive. In addition, it is difficult to obtain a perfect set of labels due to the variability between expert annotations. This paper presents a novel active learning (AL) framework for histopathology image analysis, named PathAL. To reduce the required number of expert annotations, PathAL selects two groups of unlabeled data in each training iteration: one "informative" sample that requires additional expert annotation, and one "confident predictive" sample that is automatically added to the training set using the model's pseudo-labels. To reduce the impact of the noisy-labeled samples in the training set, PathAL systematically identifies noisy samples and excludes them to improve the generalization of the model. Our model advances the existing AL method for medical image analysis in two ways. First, we present a selection strategy to improve classification performance with fewer manual annotations. Unlike traditional methods focusing only on finding the most uncertain samples with low prediction confidence, we discover a large number of high confidence samples from the unlabeled set and automatically add them for training with assigned pseudo-labels. Second, we design a method to distinguish between noisy samples and hard samples using a heuristic approach. We exclude the noisy samples while preserving the hard samples to improve model performance. Extensive experiments demonstrate that our proposed PathAL framework achieves promising results on a prostate cancer Gleason grading task, obtaining similar performance with 40% fewer annotations compared to the fully supervised learning scenario. An ablation study is provided to analyze the effectiveness of each component in PathAL, and a pathologist reader study is conducted to validate our proposed algorithm.

Harnessing clinical annotations to improve deep learning performance in prostate segmentation.

PURPOSE: Developing large-scale datasets with research-quality annotations is challenging due to the high cost of refining clinically generated markup into high precision annotations. We evaluated the direct use of a large dataset with only clinically generated annotations in development of high-performance segmentation models for small research-quality challenge datasets. MATERIALS AND METHODS: We used a large retrospective dataset from our institution comprised of 1,620 clinically generated segmentations, and two challenge datasets (PROMISE12: 50 patients, ProstateX-2: 99 patients). We trained a 3D U-Net convolutional neural network (CNN) segmentation model using our entire dataset, and used that model as a template to train models on the challenge datasets. We also trained versions of the template model using ablated proportions of our dataset, and evaluated the relative benefit of those templates for the final models. Finally, we trained a version of the template model using an out-of-domain brain cancer dataset, and evaluated the relevant benefit of that template for the final models. We used five-fold cross-validation (CV) for all training and evaluation across our entire dataset. RESULTS: Our model achieves state-of-the-art performance on our large dataset (mean overall Dice 0.916, average Hausdorff distance 0.135 across CV folds). Using this model as a pre-trained template for refining on two external datasets significantly enhanced performance (30% and 49% enhancement in Dice scores respectively). Mean overall Dice and mean average Hausdorff distance were 0.912 and 0.15 for the ProstateX-2 dataset, and 0.852 and 0.581 for the PROMISE12 dataset. Using even small quantities of data to train the template enhanced performance, with significant improvements using 5% or more of the data. CONCLUSION: We trained a state-of-the-art model using unrefined clinical prostate annotations and found that its use as a template model significantly improved performance in other prostate segmentation tasks, even when trained with only 5% of the original dataset.

Optimizing Spatial Biopsy Sampling for the Detection of Prostate Cancer.

PURPOSE: The appropriate number of systematic biopsy cores to retrieve during magnetic resonance imaging (MRI)-targeted prostate biopsy is not well defined. We aimed to demonstrate a biopsy sampling approach that reduces required core count while maintaining diagnostic performance. MATERIALS AND METHODS: We collected data from a cohort of 971 men who underwent MRI-ultrasound fusion targeted biopsy for suspected prostate cancer. A regional targeted biopsy (RTB) was evaluated retrospectively; only cores within 2 cm of the margin of a radiologist-defined region of interest were considered part of the RTB. We compared detection rates for clinically significant prostate cancer (csPCa) and cancer upgrading rate on final whole mount pathology after prostatectomy between RTB, combined, MRI-targeted, and systematic biopsy. RESULTS: A total of 16,459 total cores from 971 men were included in the study data sets, of which 1,535 (9%) contained csPCa. The csPCa detection rates for systematic, MRI-targeted, combined, and RTB were 27.0% (262/971), 38.3% (372/971), 44.8% (435/971), and 44.0% (427/971), respectively. Combined biopsy detected significantly more csPCa than systematic and MRI-targeted biopsy (p <0.001 and p=0.004, respectively) but was similar to RTB (p=0.71), which used on average 3.8 (22%) fewer cores per patient. In 102 patients who underwent prostatectomy, there was no significant difference in upgrading rates between RTB and combined biopsy (p=0.84). CONCLUSIONS: A RTB approach can maintain state-of-the-art detection rates while requiring fewer retrieved cores. This result informs decision making about biopsy site selection and total retrieved core count.

A multi-resolution model for histopathology image classification and localization with multiple instance learning.

Large numbers of histopathological images have been digitized into high resolution whole slide images, opening opportunities in developing computational image analysis tools to reduce pathologists' workload and potentially improve inter- and intra-observer agreement. Most previous work on whole slide image analysis has focused on classification or segmentation of small pre-selected regions-of-interest, which requires fine-grained annotation and is non-trivial to extend for large-scale whole slide analysis. In this paper, we proposed a multi-resolution multiple instance learning model that leverages saliency maps to detect suspicious regions for fine-grained grade prediction. Instead of relying on expensive region- or pixel-level annotations, our model can be trained end-to-end with only slide-level labels. The model is developed on a large-scale prostate biopsy dataset containing 20,229 slides from 830 patients. The model achieved 92.7% accuracy, 81.8% Cohen's Kappa for benign, low grade (i.e. Grade group 1) and high grade (i.e. Grade group ≥ 2) prediction, an area under the receiver operating characteristic curve (AUROC) of 98.2% and an average precision (AP) of 97.4% for differentiating malignant and benign slides. The model obtained an AUROC of 99.4% and an AP of 99.8% for cancer detection on an external dataset.

Semi-automated PIRADS scoring via mpMRI analysis.

Purpose: Prostate cancer (PCa) is the most common solid organ cancer and second leading cause of death in men. Multiparametric magnetic resonance imaging (mpMRI) enables detection of the most aggressive, clinically significant PCa (csPCa) tumors that require further treatment. A suspicious region of interest (ROI) detected on mpMRI is now assigned a Prostate Imaging-Reporting and Data System (PIRADS) score to standardize interpretation of mpMRI for PCa detection. However, there is significant inter-reader variability among radiologists in PIRADS score assignment and a minimal input semi-automated artificial intelligence (AI) system is proposed to harmonize PIRADS scores with mpMRI data. Approach: The proposed deep learning model (the seed point model) uses a simulated single-click seed point as input to annotate the lesion on mpMRI. This approach is in contrast to typical medical AI-based approaches that require annotation of the complete lesion. The mpMRI data from 617 patients used in this study were prospectively collected at a major tertiary U.S. medical center. The model was trained and validated to classify whether an mpMRI image had a lesion with a PIRADS score greater than or equal to PIRADS 4. Results: The model yielded an average receiver-operator characteristic (ROC) area under the curve (ROC-AUC) of 0.704 over a 10-fold cross-validation, which is significantly higher than the previously published benchmark. Conclusions: The proposed model could aid in PIRADS scoring of mpMRI, providing second reads to promote quality as well as offering expertise in environments that lack a radiologist with training in prostate mpMRI interpretation. The model could help identify tumors with a higher PIRADS for better clinical management and treatment of PCa patients at an early stage.

Using Sequential Decision Making to Improve Lung Cancer Screening Performance.

Globally, lung cancer is responsible for nearly one in five cancer deaths. The National Lung Screening Trial (NLST) demonstrated the efficacy of low-dose computed tomography (LDCT) to identify early-stage disease, setting the basis for widespread implementation of lung cancer screening programs. However, the specificity of LDCT lung cancer screening is suboptimal, with a significant false positive rate. Representing this imaging-based screening process as a sequential decision making problem, we combined multiple machine learning-based methods to learn a partially-observable Markov decision process that simultaneously optimizes lung cancer detection while enhancing test specificity. Using NLST data, we trained a dynamic Bayesian network as an observational model and used inverse reinforcement learning to discover a rewards function based on experts' decisions. Our resultant predictive model decreased the false positive rate while maintaining a high true positive rate at a level comparable to human experts. Our model also detected a number of lung cancers earlier.

An EM-based semi-supervised deep learning approach for semantic segmentation of histopathological images from radical prostatectomies.

Automated Gleason grading is an important preliminary step for quantitative histopathological feature extraction. Different from the traditional task of classifying small pre-selected homogeneous regions, semantic segmentation provides pixel-wise Gleason predictions across an entire slide. Deep learning-based segmentation models can automatically learn visual semantics from data, which alleviates the need for feature engineering. However, performance of deep learning models is limited by the scarcity of large-scale fully annotated datasets, which can be both expensive and time-consuming to create. One way to address this problem is to leverage external weakly labeled datasets to augment models trained on the limited data. In this paper, we developed an expectation maximization-based approach constrained by an approximated prior distribution in order to extract useful representations from a large number of weakly labeled images generated from low-magnification annotations. This method was utilized to improve the performance of a model trained on a limited fully annotated dataset. Our semi-supervised approach trained with 135 fully annotated and 1800 weakly annotated tiles achieved a mean Jaccard Index of 49.5% on an independent test set, which was 14% higher than the initial model trained only on the fully annotated dataset.

Patients' Perceptions of Radiation Exposure Associated With Mammography.

OBJECTIVE: Radiation associated with breast imaging is a sensitive issue, particularly for women who undergo mammography as a screening measure to detect breast cancer. Misinformation and misunderstanding regarding the risks associated with ionizing radiation have created heightened public concern and fear, which may result in avoidance of diagnostic procedures. The objectives of this study were to ascertain patients' knowledge and opinion of ionizing radiation as a whole and specifically in mammography, as well as to determine common misunderstandings and points of view that may affect women's decisions about whether to have a mammogram. MATERIALS AND METHODS: Over a 9-month period, a total of 1725 patients presenting for a mammogram completed a 25-point questionnaire focused on the following: general knowledge of radiation dose in common imaging modalities, the amount of radiation associated with a mammogram relative to five radiation benchmarks, and patients' opinions of the involvement of radiation in their health care. RESULTS: Although 65% of the women receiving a mammogram responded that they had been informed of the risks and benefits of the examination, 60% overestimated the radiation in a mammogram. CONCLUSION: Efforts should be made to accurately inform women of the risks and benefits of mammography, specifically highlighting the low dose of mammographic ionizing radiation and providing objective facts to ensure that they are making an informed decision regarding screening.

Prevalence of coronary artery disease evaluated by coronary CT angiography in women with mammographically detected breast arterial calcifications.

To assess the correlation between breast arterial calcifications (BAC) on digital mammography and the extent of coronary artery disease (CAD) diagnosed with dual source coronary computed tomography angiography (CTA) in a population of women both symptomatic and asymptomatic for coronary artery disease. 100 consecutive women (aged 34 - 86 years) who underwent both coronary CTA and digital mammography were included in the study. Health records were reviewed to determine the presence of cardiovascular risk factors such as hypertension, hyperlipidemia, diabetes mellitus, and smoking. Digital mammograms were reviewed for the presence and degree of BAC, graded in terms of severity and extent. Coronary CTAs were reviewed for CAD, graded based on the extent of calcified and non-calcified plaque, and the degree of major vessel stenosis. A four point grading scale was used for both coronary CTA and mammography. The overall prevalence of positive BAC and CAD in the studied population were 12% and 29%, respectively. Ten of the 12 patients with moderate or advanced BAC on mammography demonstrated moderate to severe CAD as determined by coronary CTA. For all women, the positive predictive value of BAC for CAD was 0.83 and the negative predictive value was 0.78. The presence of BAC on mammography appears to correlate with CAD as determined by coronary CTA (Spearman's rank correlation coefficient = 0.48, p<.000001). Using logistic regression, the inclusion of BAC as a feature in CAD predication significantly increased classification results (p=0.04).

Motivating the additional use of external validity: examining transportability in a model of glioblastoma multiforme.

Despite the growing ubiquity of data in the medical domain, it remains difficult to apply results from experimental and observational studies to additional populations suffering from the same disease. Many methods are employed for testing internal validity; yet limited effort is made in testing generalizability, or external validity. The development of disease models often suffers from this lack of validity testing and trained models frequently have worse performance on different populations, rendering them ineffective. In this work, we discuss the use of transportability theory, a causal graphical model examination, as a mechanism for determining what elements of a data resource can be shared or moved between a source and target population. A simplified Bayesian model of glioblastoma multiforme serves as the example for discussion and preliminary analysis. Examination over data collection hospitals from the TCGA dataset demonstrated improvement of prediction in a transported model over a baseline model.

Automated extraction of reported statistical analyses: towards a logical representation of clinical trial literature.

Randomized controlled trials are an important source of evidence for guiding clinical decisions when treating a patient. However, given the large number of studies and their variability in quality, determining how to summarize reported results and formalize them as part of practice guidelines continues to be a challenge. We have developed a set of information extraction and annotation tools to automate the identification of key information from papers related to the hypothesis, sample size, statistical test, confidence interval, significance level, and conclusions. We adapted the Automated Sequence Annotation Pipeline to map extracted phrases to relevant knowledge sources. We trained and tested our system on a corpus of 42 full-text articles related to chemotherapy of non-small cell lung cancer. On our test set of 7 papers, we obtained an overall precision of 86%, recall of 78%, and an F-score of 0.82 for classifying sentences. This work represents our efforts towards utilizing this information for quality assessment, meta-analysis, and modeling.

Robust skull stripping of clinical glioblastoma multiforme data.

Skull stripping is the first step in many neuroimaging analyses and its success is critical to all subsequent processing. Methods exist to skull strip brain images without gross deformities, such as those affected by Alzheimer's and Huntington's disease. However, there are no techniques for extracting brains affected by diseases that significantly disturb normal anatomy. Glioblastoma multiforme (GBM) is such a disease, as afflicted individuals develop large tumors that often require surgical resection. In this paper, we extend the ROBEX skull stripping method to extract brains from GBM images. The proposed method uses a shape model trained on healthy brains to be relatively insensitive to lesions inside the brain. The brain boundary is then searched for potential resection cavities using adaptive thresholding and the Random Walker algorithm corrects for leakage into the ventricles. The results show significant improvement over three popular skull stripping algorithms (BET, BSE and HWA) in a dataset of 48 GBM cases.

Cost implications of new treatments for advanced colorectal cancer.

BACKGROUND: Since 1996, 6 new drugs have been introduced for the treatment of metastatic colorectal cancer. Although they are promising, these drugs frequently are given in the palliative and are much more expensive than older treatments. The objective of the current study was to measure the cost implications of treatment with sequential regimens that include chemotherapy and/or monoclonal antibodies. METHODS: A Markov model was used to evaluate a hypothetical cohort of 1000 patients with newly diagnosed, metastatic colorectal cancer. Patients supposedly received up to 3 lines of treatment before supportive care and subsequent death. Data were obtained from published, multicenter phase 2 and randomized phase 3 clinical trials. Sensitivity analyses were conducted on the efficacy, toxicity, and cost. RESULTS: Using drug costs alone, treatment that included new chemotherapeutic agents increased survival at an incremental cost-effectiveness ratio (ICER) of $100,000 per discounted life-year (DLY). The addition of monoclonal antibodies improved survival at an ICER of >$170,000 per DLY. The results were most sensitive to changes in the initial regimen. Even with significant improvements in clinical characteristics (efficacy and toxicity), treatment with the most effective regimens still had very high ICERs. CONCLUSIONS: Treatment of metastatic colorectal cancer with the most effective regimens came at very high incremental costs. The authors concluded that cost-effectiveness analyses should be a routine component of the drug-development process, so that physicians and patients are informed appropriately regarding the value of new innovations.