Machine Learning Prediction of Lymph Node Metastasis in Breast Cancer: Performance of a Multi-institutional MRI-based 4D Convolutional Neural Network.

Purpose To develop a custom deep convolutional neural network (CNN) for noninvasive prediction of breast cancer nodal metastasis. Materials and Methods This retrospective study included patients with newly diagnosed primary invasive breast cancer with known pathologic (pN) and clinical nodal (cN) status who underwent dynamic contrast-enhanced (DCE) breast MRI at the authors' institution between July 2013 and July 2016. Clinicopathologic data (age, estrogen receptor and human epidermal growth factor 2 status, Ki-67 index, and tumor grade) and cN and pN status were collected. A four-dimensional (4D) CNN model integrating temporal information from dynamic image sets was developed. The convolutional layers learned prognostic image features, which were combined with clinicopathologic measures to predict cN0 versus cN+ and pN0 versus pN+ disease. Performance was assessed with the area under the receiver operating characteristic curve (AUC), with fivefold nested cross-validation. Results Data from 350 female patients (mean age, 51.7 years ± 11.9 [SD]) were analyzed. AUC, sensitivity, and specificity values of the 4D hybrid model were 0.87 (95% CI: 0.83, 0.91), 89% (95% CI: 79%, 93%), and 76% (95% CI: 68%, 88%) for differentiating pN0 versus pN+ and 0.79 (95% CI: 0.76, 0.82), 80% (95% CI: 77%, 84%), and 62% (95% CI: 58%, 67%), respectively, for differentiating cN0 versus cN+. Conclusion The proposed deep learning model using tumor DCE MR images demonstrated high sensitivity in identifying breast cancer lymph node metastasis and shows promise for potential use as a clinical decision support tool. Keywords: MR Imaging, Breast, Breast Cancer, Breast MRI, Machine Learning, Metastasis, Prognostic Prediction Supplemental material is available for this article. Published under a CC BY 4.0 license.

A Layered, Hybrid Machine Learning Analytic Workflow for Mouse Risk Assessment Behavior.

Accurate and efficient quantification of animal behavior facilitates the understanding of the brain. An emerging approach within machine learning (ML) field is to combine multiple ML-based algorithms to quantify animal behavior. These so-called hybrid models have emerged because of limitations associated with supervised [e.g., random forest (RF)] and unsupervised [e.g., hidden Markov model (HMM)] ML models. For example, RF models lack temporal information across video frames, and HMM latent states are often difficult to interpret. We sought to develop a hybrid model, and did so in the context of a study of mouse risk assessment behavior. We used DeepLabCut to estimate the positions of mouse body parts. Positional features were calculated using DeepLabCut outputs and were used to train RF and HMM models with equal number of states, separately. The per-frame predictions from RF and HMM models were then passed to a second HMM model layer ("reHMM"). The outputs of the reHMM layer showed improved interpretability over the initial HMM output. Finally, we combined predictions from RF and HMM models with selected positional features to train a third HMM model ("reHMM+"). This reHMM+ layered hybrid model unveiled distinctive temporal and human-interpretable behavioral patterns. We applied this workflow to investigate risk assessment to trimethylthiazoline and snake feces odor, finding unique behavioral patterns to each that were separable from attractive and neutral stimuli. We conclude that this layered, hybrid ML workflow represents a balanced approach for improving the depth and reliability of ML classifiers in chemosensory and other behavioral contexts.

A scalable open-source MATLAB toolbox for reconstruction and analysis of multispectral optoacoustic tomography data.

Multispectral photoacoustic tomography enables the resolution of spectral components of a tissue or sample at high spatiotemporal resolution. With the availability of commercial instruments, the acquisition of data using this modality has become consistent and standardized. However, the analysis of such data is often hampered by opaque processing algorithms, which are challenging to verify and validate from a user perspective. Furthermore, such tools are inflexible, often locking users into a restricted set of processing motifs, which may not be able to accommodate the demands of diverse experiments. To address these needs, we have developed a Reconstruction, Analysis, and Filtering Toolbox to support the analysis of photoacoustic imaging data. The toolbox includes several algorithms to improve the overall quantification of photoacoustic imaging, including non-negative constraints and multispectral filters. We demonstrate various use cases, including dynamic imaging challenges and quantification of drug effect, and describe the ability of the toolbox to be parallelized on a high performance computing cluster.

Particle-Tracking Proton Computed Tomography-Data Acquisition, Preprocessing, and Preconditioning.

Proton CT (pCT) is a promising new imaging technique that can reconstruct relative stopping power (RSP) more accurately than x-ray CT in each cubic millimeter voxel of the patient. This, in turn, will result in better proton range accuracy and, therefore, smaller planned tumor volumes (PTV). The hardware description and some reconstructed images have previously been reported. In a series of two contributions, we focus on presenting the software algorithms that convert pCT detector data to the final reconstructed pCT images for application in proton treatment planning. There were several options on how to accomplish this, and we will describe our solutions at each stage of the data processing chain. In the first paper of this series, we present the data acquisition with the pCT tracking and energy-range detectors and how the data are preprocessed, including the conversion to the well-formatted track information from tracking data and water-equivalent path length from the data of a calibrated multi-stage energy-range detector. These preprocessed data are then used for the initial image formation with an FDK cone-beam CT algorithm. The output of data acquisition, preprocessing, and FDK reconstruction is presented along with illustrative imaging results for two phantoms, including a pediatric head phantom. The second paper in this series will demonstrate the use of iterative solvers in conjunction with the superiorization methodology to further improve the images resulting from the upfront FDK image reconstruction and the implementation of these algorithms on a hybrid CPU/GPU computer cluster.

Preoperative Prediction of Lymph Node Metastasis from Clinical DCE MRI of the Primary Breast Tumor Using a 4D CNN.

In breast cancer, undetected lymph node metastases can spread to distal parts of the body for which the 5-year survival rate is only 27%, making accurate nodal metastases diagnosis fundamental to reducing the burden of breast cancer, when it is still early enough to intervene with surgery and adjuvant therapies. Currently, breast cancer management entails a time consuming and costly sequence of steps to clinically diagnose axillary nodal metastases status. The purpose of this study is to determine whether preoperative, clinical DCE MRI of the primary tumor alone may be used to predict clinical node status with a deep learning model. If possible then many costly steps could be eliminated or reserved for only those with uncertain or probable nodal metastases. This research develops a data-driven approach that predicts lymph node metastasis through the judicious integration of clinical and imaging features from preoperative 4D dynamic contrast enhanced (DCE) MRI of 357 patients from 2 hospitals. Innovative deep learning classifiers are trained from scratch, including 2D, 3D, 4D and 4D deep convolutional neural networks (CNNs) that integrate multiple data types and predict the nodal metastasis differentiating nodal stage N0 (non metastatic) against stages N1, N2 and N3. Appropriate methodologies for data preprocessing and network interpretation are presented, the later of which bolster radiologist confidence that the model has learned relevant features from the primary tumor. Rigorous nested 10-fold cross-validation provides an unbiased estimate of model performance. The best model achieves a high sensitivity of 72% and an AUROC of 71% on held out test data. Results are strongly supportive of the potential of the combination of DCE MRI and machine learning to inform diagnostics that could substantially reduce breast cancer burden.

An Improved Method of Total Variation Superiorization Applied to Reconstruction in Proton Computed Tomography.

Previous work has shown that total variation superiorization (TVS) improves reconstructed image quality in proton computed tomography (pCT). The structure of the TVS algorithm has evolved since then and this paper investigated if this new algorithmic structure provides additional benefits to pCT image quality. Structural and parametric changes introduced to the original TVS algorithm included: (1) inclusion or exclusion of TV reduction requirement, (2) a variable number, N , of TV perturbation steps per feasibility-seeking iteration, and (3) introduction of a perturbation kernel . The structural change of excluding the TV reduction requirement check tended to have a beneficial effect for 3 ≤ N ≤ 6 and allows full parallelization of the TVS algorithm. Repeated perturbations per feasibility-seeking iterations reduced total variation (TV) and material dependent standard deviations for 3 ≤ N ≤ 6 . The perturbation kernel α , effectively equal to α = 0.5 in the original TVS algorithm, reduced TV and standard deviations as α was increased beyond α = 0.5 , but negatively impacted reconstructed relative stopping power (RSP) values for . The reductions in TV and standard deviations allowed feasibility-seeking with a larger relaxation parameter λ than previously used, without the corresponding increases in standard deviations experienced with the original TVS algorithm. This paper demonstrates that the modifications related to the evolution of the original TVS algorithm provide benefits in terms of both pCT image quality and computational efficiency for appropriately chosen parameter values.

Robust Iterative Methods: Convergence and Applications to Proton Computed Tomography.

Robust methods, such as Tikhonov regularization and Bounded data uncertainty, have been used extensively in relatively small problems involving dense matrices for many decades, but have not been used in large-scale iterative methods for image reconstruction in particle imaging until recently. In this case, robust methods may allow more accurate reconstruction of images in the presence of errors of both the energy measurement of the protons and ions but also in the estimated path taken by the proton or ion through the object. Robust systems may also be used when entire blocks of data are missing, or in low-dose reconstructions using a very small number of particles without substantial loss of image quality. In this contribution, we demonstrate that robust methods show great promise in proton/ion (particle) computed tomography (pCT), and, for the first time, that they can be proven to converge. Thus, the convergence of robust methods as well as benefits for reconstruction in uncertain systems is shown to constitute the main advantage for pCT reconstruction.