Artificial Intelligence-Assisted Amphiregulin and Epiregulin IHC Predicts Panitumumab Benefit in RAS Wild-Type Metastatic Colorectal Cancer.

PURPOSE: High tumor mRNA levels of the EGFR ligands amphiregulin (AREG) and epiregulin (EREG) are associated with anti-EGFR agent response in metastatic colorectal cancer (mCRC). However, ligand RNA assays have not been adopted into routine practice due to issues with analytic precision and practicality. We investigated whether AREG/EREG IHC could predict benefit from the anti-EGFR agent panitumumab. EXPERIMENTAL DESIGN: Artificial intelligence algorithms were developed to assess AREG/EREG IHC in 274 patients from the PICCOLO trial of irinotecan with or without panitumumab (Ir vs. IrPan) in RAS wild-type mCRC. The primary endpoint was progression-free survival (PFS). Secondary endpoints were RECIST response rate (RR) and overall survival (OS). Models were repeated adjusting separately for BRAF mutation status and primary tumor location (PTL). RESULTS: High ligand expression was associated with significant PFS benefit from IrPan compared with Ir [8.0 vs. 3.2 months; HR, 0.54; 95% confidence interval (CI), 0.37-0.79; P = 0.001]; whereas low ligand expression was not (3.4 vs. 4.4 months; HR, 1.05; 95% CI, 0.74-1.49; P = 0.78). The ligand-treatment interaction was significant (P interaction = 0.02) and remained significant after adjustment for BRAF-mutation status and PTL. Likewise, RECIST RR was significantly improved in patients with high ligand expression (IrPan vs. Ir: 48% vs. 6%; P < 0.0001) but not those with low ligand expression (25% vs. 14%; P = 0.10; P interaction = 0.01). The effect on OS was similar but not statistically significant. CONCLUSIONS: AREG/EREG IHC identified patients who benefitted from the addition of panitumumab to irinotecan chemotherapy. IHC is a practicable assay that may be of use in routine practice.

Joint Region and Nucleus Segmentation for Characterization of Tumor Infiltrating Lymphocytes in Breast Cancer.

Histologic assessment of stromal tumor infiltrating lymphocytes (sTIL) as a surrogate of the host immune response has been shown to be prognostic and potentially chemo-predictive in triple-negative and HER2-positive breast cancers. The current practice of manual assessment is prone to intra- and inter-observer variability. Furthermore, the inter-play of sTILs, tumor cells, other microenvironment mediators, their spatial relationships, quantity, and other image-based features have yet to be determined exhaustively and systemically. Towards analysis of these aspects, we developed a deep learning based method for joint region-level and nucleus-level segmentation and classification of breast cancer H&E tissue whole slide images. Our proposed method simultaneously identifies tumor, fibroblast, and lymphocyte nuclei, along with key histologic region compartments including tumor and stroma. We also show how the resultant segmentation masks can be combined with seeding approaches to yield accurate nucleus classifications. Furthermore, we outline a simple workflow for calibrating computational scores to human scores for consistency. The pipeline identifies key compartments with high accuracy (Dice= overall: 0.78, tumor: 0.83, and fibroblasts: 0.77). ROC AUC for nucleus classification is high at 0.89 (micro-average), 0.89 (lymphocytes), 0.90 (tumor), and 0.78 (fibroblasts). Spearman correlation between computational sTIL and pathologist consensus is high (R=0.73, p<0.001) and is higher than inter-pathologist correlation (R=0.66, p<0.001). Both manual and computational sTIL scores successfully stratify patients by clinical progression outcomes.

Automated left ventricular segmentation in cardiac MRI.

We present an automated left ventricular (LV) myocardial boundary extraction method. Automatic localization of the LV is achieved using a motion map and an expectation maximization algorithm. The myocardial region is then segmented using an intensity-based fuzzy affinity map and the myocardial contours are extracted by cost minimization through a dynamic programming approach. The results from the automated algorithm compared against the experienced radiologists using Bland and Altman analysis were found to have consistent mean bias of 7% and limits of agreement comparable to the inter-observer variability inherent in the manual method.

PDM-ENLOR for segmentation of mouse brain gene expression images.

Statistical shape models, such as Active Shape Models (ASMs), suffer from their inability to represent a large range of variations of a complex shape and to account for the large errors in detection of (point) landmarks. We propose a method, PDM-ENLOR (Point Distribution Model-based ENsemble of LOcal Regressors), that overcomes these limitations by locating each landmark individually using an ensemble of local regression models and appearance cues from selected landmarks. We first detect a set of reference landmarks which were selected based on their saliency during training. For each landmark, an ensemble of regressors is built. From the locations of the detected reference landmarks, each regressor infers a candidate location for that landmark using local geometric constraints, encoded by a point distribution model (PDM). The final location of that point is determined as a weighted linear combination, whose coefficients are learned from the training data, of candidates proposed by its ensemble's component regressors. We use multiple subsets of reference landmarks as explanatory variables for the component regressors to provide varying degrees of locality for the models in each ensemble. This helps our ensemble model to capture a larger range of shape variations as compared to a single PDM. We demonstrate the advantages of our method on the challenging problem of segmenting gene expression images of mouse brain. The overall mean and standard deviation of the Dice coefficient overlap over all 14 anatomical regions and all 100 test images were (88.1 ± 9.5)%.

3D dense local point descriptors for mouse brain gene expression images.

Anatomical landmarks play an important role in many biomedical image analysis applications (e.g., registration and segmentation). Landmark detection can be computationally very expensive, especially in 3D images, because every single voxel in a region of interest may need to be evaluated. In this paper, we introduce two 3D local image descriptors which can be computed simultaneously for every voxel in a volume. Both our proposed descriptors are extensions of the DAISY descriptor, a popular descriptor that is based on the histograms of oriented gradients and was named after its daisy-flower-like configuration. Our experiments on mouse brain gene expression images indicate that our descriptors are discriminative and are able to reduce the detection errors of landmark points more than 30% when compared with SIFT-3D, an extension in 3D of SIFT (scale-invariant feature transform). We also demonstrate that our descriptors are more computationally efficient than SIFT-3D and n-SIFT (an extension SIFT in n-dimensions) for densely sampled points. Therefore, our descriptors can be used in applications that require computation of the descriptors at densely sampled points (e.g., landmark point detection or feature-based registration).

Segmentation of the thoracic aorta in noncontrast cardiac CT images.

Studies have shown that aortic calcification is associated with cardiovascular disease. In this study, a method for localization, centerline extraction, and segmentation of the thoracic aorta in noncontrast cardiac-computed tomography (CT) images, toward the detection of aortic calcification, is presented. The localization of the right coronary artery ostium slice is formulated as a regression problem whose input variables are obtained from simple intensity features computed from a pyramid representation of the slice. The localization, centerline extraction, and segmentation of the aorta are formulated as optimal path detection problems. Dynamic programming is applied in the Hough space for localizing key center points in the aorta which guide the centerline tracing using a fast marching-based minimal path extraction framework. The input volume is then resampled into a stack of 2-D cross-sectional planes orthogonal to the obtained centerline. Dynamic programming is again applied for the segmentation of the aorta in each slice of the resampled volume. The obtained segmentation is finally mapped back to its original volume space. The performance of the proposed method was assessed on cardiac noncontrast CT scans and promising results were obtained.

Similarity-based appearance-prior for fitting a subdivision mesh in gene expression images.

Automated segmentation of multi-part anatomical objects in images is a challenging task. In this paper, we propose a similarity-based appearance-prior to fit a compartmental geometric atlas of the mouse brain in gene expression images. A subdivision mesh which is used to model the geometry is deformed using a Markov random field (MRF) framework. The proposed appearance-prior is computed as a function of the similarity between local patches at corresponding atlas locations from two images. In addition, we introduce a similarity-saliency score to select the mesh points that are relevant for the computation of the proposed prior. Our method significantly improves the accuracy of the atlas fitting, especially in the regions that are influenced by the selected similarity-salient points, and outperforms the previous subdivision mesh fitting methods for gene expression images.

Markov Random Field-based Fitting of a Subdivision-based Geometric Atlas.

An accurate labeling of a multi-part, complex anatomical structure (e.g., brain) is required in order to compare data across images for spatial analysis. It can be achieved by fitting an object-specific geometric atlas that is constructed using a partitioned, high-resolution deformable mesh and tagging each of its polygons with a region label. Subdivision meshes have been used to construct such an atlas because they can provide a compact representation of a partitioned, multi-resolution, object-specific mesh structure using only a few control points. However, automated fitting of a subdivision mesh-based geometric atlas to an anatomical structure in an image is a difficult problem and has not been sufficiently addressed. In this paper, we propose a novel Markov Random Field-based method for fitting a planar, multi-part subdivision mesh to anatomical data. The optimal fitting of the atlas is obtained by determining the optimal locations of the control points. We also tackle the problem of landmark matching in tandem with atlas fitting by constructing a single graphical model to impose pose-invariant, landmark-based geometric constraints on atlas deformation. The atlas deformation is also governed by additional constraints imposed by the mesh's geometric properties and the object boundary. We demonstrate the potential of the proposed method on the difficult problem of segmenting a mouse brain and its interior regions in gene expression images which exhibit large intensity and shape variability. We obtain promising results when compared with manual annotations and prior methods.

Landmark/Image-based Deformable Registration of Gene Expression Data.

Analysis of gene expression patterns in brain images obtained from high-throughput in situ hybridization requires accurate and consistent annotations of anatomical regions/subregions. Such annotations are obtained by mapping an anatomical atlas onto the gene expression images through intensity- and/or landmark-based registration methods or deformable model-based segmentation methods. Due to the complex appearance of the gene expression images, these approaches require a pre-processing step to determine landmark correspondences in order to incorporate landmark-based geometric constraints. In this paper, we propose a novel method for landmark-constrained, intensity-based registration without determining landmark correspondences a priori. The proposed method performs dense image registration and identifies the landmark correspondences, simultaneously, using a single higher-order Markov Random Field model. In addition, a machine learning technique is used to improve the discriminating properties of local descriptors for landmark matching by projecting them in a Hamming space of lower dimension. We qualitatively show that our method achieves promising results and also compares well, quantitatively, with the expert's annotations, outperforming previous methods.

Toward the automatic detection of coronary artery calcification in non-contrast computed tomography data.

Measurements related to coronary artery calcification (CAC) offer significant predictive value for coronary artery disease (CAD). In current medical practice CAC scoring is a labor-intensive task. The objective of this paper is the development and evaluation of a family of coronary artery region (CAR) models applied to the detection of CACs in coronary artery zones and sections. Thirty patients underwent non-contrast electron-beam computed tomography scanning. Coronary artery trajectory points as presented in the University of Houston heart-centered coordinate system were utilized to construct the CAR models which automatically detect coronary artery zones and sections. On a per-patient and per-zone basis the proposed CAR models detected CACs with a sensitivity, specificity and accuracy of 85.56 (± 15.80)%, 93.54 (± 1.98)%, and 85.27 (± 14.67)%, respectively while the corresponding values in the zones and segments based case were 77.94 (± 7.78)%, 96.57 (± 4.90)%, and 73.58 (± 8.96)%, respectively. The results of this study suggest that the family of CAR models provide an effective method to detect different regions of the coronaries. Further, the CAR classifiers are able to detect CACs with a mean sensitivity and specificity of 86.33 and 93.78%, respectively.

A supervised classification-based method for coronary calcium detection in non-contrast CT.

Accurate quantification of coronary artery calcium provides an opportunity to assess the extent of atherosclerosis disease. Coronary calcification burden has been reported to be associated with cardiovascular risk. Currently, an observer has to identify the coronary calcifications among a set of candidate regions, obtained by thresholding and connected component labeling, by clicking on them. To relieve the observer of such a labor-intensive task, an automated tool is needed that can detect and quantify the coronary calcifications. However, the diverse and heterogeneous nature of the candidate regions poses a significant challenge. In this paper, we investigate a supervised classification-based approach to distinguish the coronary calcifications from all the candidate regions and propose a two-stage, hierarchical classifier for automated coronary calcium detection. At each stage, we learn an ensemble of classifiers where each classifier is a cost-sensitive learner trained on a distinct asymmetrically sampled data subset. We compute the relative location of the calcifications with respect to a heart-centered coordinate system, and also use the neighboring regions of the calcifications to better characterize their properties for discrimination. Our method detected coronary calcifications with an accuracy, sensitivity and specificity of 98.27, 92.07 and 98.62%, respectively, for a testing dataset of non-contrast computed tomography scans from 105 subjects.

Localization and segmentation of left ventricle in cardiac cine-MR images.

Accurate delineation of the left ventricular myocardial boundaries on cardiac cine magnetic resonance (MR) images is essential for volumetric and functional cardiac analysis. Automated myocardial contour delineation often suffers from misalignment of slices, nonuniform coil sensitivity, blood-flow-related inter- and intraslice intensity inhomogeneities, blurring due to motion, partial voluming, and a need to circumscribe the papillary muscles and the trabeculae. In this paper, we propose a novel method for data-driven localization and segmentation of the left ventricle in the cine-MR images toward automated computation of ejection fraction (EF). Our hybrid segmentation method combines intensity- and texture-based fuzzy affinity maps obtained from a novel multiclass, multifeature fuzzy connectedness method with dynamic-programming-based boundary detection to delineate the myocardial contours. Bland-Altman analysis indicates that the mean biases of the end-diastolic volume, end-systolic volume, and EF estimates of our method are comparable to the interobserver variability when compared with the annotations from two experts.