Age Encoded Adversarial Learning for Pediatric CT Segmentation.

Organ segmentation from CT images is critical in the early diagnosis of diseases, progress monitoring, pre-operative planning, radiation therapy planning, and CT dose estimation. However, data limitation remains one of the main challenges in medical image segmentation tasks. This challenge is particularly huge in pediatric CT segmentation due to children's heightened sensitivity to radiation. In order to address this issue, we propose a novel segmentation framework with a built-in auxiliary classifier generative adversarial network (ACGAN) that conditions age, simultaneously generating additional features during training. The proposed conditional feature generation segmentation network (CFG-SegNet) was trained on a single loss function and used 2.5D segmentation batches. Our experiment was performed on a dataset with 359 subjects (180 male and 179 female) aged from 5 days to 16 years and a mean age of 7 years. CFG-SegNet achieved an average segmentation accuracy of 0.681 dice similarity coefficient (DSC) on the prostate, 0.619 DSC on the uterus, 0.912 DSC on the liver, and 0.832 DSC on the heart with four-fold cross-validation. We compared the segmentation accuracy of our proposed method with previously published U-Net results, and our network improved the segmentation accuracy by 2.7%, 2.6%, 2.8%, and 3.4% for the prostate, uterus, liver, and heart, respectively. The results indicate that our high-performing segmentation framework can more precisely segment organs when limited training images are available.

Deep learning classification of deep ultraviolet fluorescence images toward intra-operative margin assessment in breast cancer.

Background: Breast-conserving surgery is aimed at removing all cancerous cells while minimizing the loss of healthy tissue. To ensure a balance between complete resection of cancer and preservation of healthy tissue, it is necessary to assess themargins of the removed specimen during the operation. Deep ultraviolet (DUV) fluorescence scanning microscopy provides rapid whole-surface imaging (WSI) of resected tissues with significant contrast between malignant and normal/benign tissue. Intra-operative margin assessment with DUV images would benefit from an automated breast cancer classification method. Methods: Deep learning has shown promising results in breast cancer classification, but the limited DUV image dataset presents the challenge of overfitting to train a robust network. To overcome this challenge, the DUV-WSI images are split into small patches, and features are extracted using a pre-trained convolutional neural network-afterward, a gradient-boosting tree trains on these features for patch-level classification. An ensemble learning approach merges patch-level classification results and regional importance to determine the margin status. An explainable artificial intelligence method calculates the regional importance values. Results: The proposed method's ability to determine the DUV WSI was high with 95% accuracy. The 100% sensitivity shows that the method can detect malignant cases efficiently. The method could also accurately localize areas that contain malignant or normal/benign tissue. Conclusion: The proposed method outperforms the standard deep learning classification methods on the DUV breast surgical samples. The results suggest that it can be used to improve classification performance and identify cancerous regions more effectively.

Analysis of Deep Ultraviolet Fluorescence Images for Intraoperative Breast Tumor Margin Assessment.

Positive margin status after breast-conserving surgery (BCS) is a predictor of higher rates of local recurrence. Intraoperative margin assessment aims to achieve negative surgical margin status at the first operation, thus reducing the re-excision rates that are usually associated with potential surgical complications, increased medical costs, and mental pressure on patients. Microscopy with ultraviolet surface excitation (MUSE) can rapidly image tissue surfaces with subcellular resolution and sharp contrasts by utilizing the nature of the thin optical sectioning thickness of deep ultraviolet light. We have previously imaged 66 fresh human breast specimens that were topically stained with propidium iodide and eosin Y using a customized MUSE system. To achieve objective and automated assessment of MUSE images, a machine learning model is developed for binary (tumor vs. normal) classification of obtained MUSE images. Features extracted by texture analysis and pre-trained convolutional neural networks (CNN) have been investigated for sample descriptions. A sensitivity, specificity, and accuracy better than 90% have been achieved for detecting tumorous specimens. The result suggests the potential of MUSE with machine learning being utilized for intraoperative margin assessment during BCS.

High-Throughput Mass Spectrometry Imaging with Dynamic Sparse Sampling.

Mass spectrometry imaging (MSI) enables label-free mapping of hundreds of molecules in biological samples with high sensitivity and unprecedented specificity. Conventional MSI experiments are relatively slow, limiting their utility for applications requiring rapid data acquisition, such as intraoperative tissue analysis or 3D imaging. Recent advances in MSI technology focus on improving the spatial resolution and molecular coverage, further increasing the acquisition time. Herein, a deep learning approach for dynamic sampling (DLADS) was employed to reduce the number of required measurements, thereby improving the throughput of MSI experiments in comparison with conventional methods. DLADS trains a deep learning model to dynamically predict molecularly informative tissue locations for active mass spectra sampling and reconstructs high-fidelity molecular images using only the sparsely sampled information. Experimental hardware and software integration of DLADS with nanospray desorption electrospray ionization (nano-DESI) MSI is reported for the first time, which demonstrates a 2.3-fold improvement in throughput for a linewise acquisition mode. Meanwhile, simulations indicate that a 5-10-fold throughput improvement may be achieved using the pointwise acquisition mode.

Automated assessment of breast margins in deep ultraviolet fluorescence images using texture analysis.

Microscopy with ultraviolet surface excitation (MUSE) is increasingly studied for intraoperative assessment of tumor margins during breast-conserving surgery to reduce the re-excision rate. Here we report a two-step classification approach using texture analysis of MUSE images to automate the margin detection. A study dataset consisting of MUSE images from 66 human breast tissues was constructed for model training and validation. Features extracted using six texture analysis methods were investigated for tissue characterization, and a support vector machine was trained for binary classification of image patches within a full image based on selected feature subsets. A weighted majority voting strategy classified a sample as tumor or normal. Using the eight most predictive features ranked by the maximum relevance minimum redundancy and Laplacian scores methods has achieved a sample classification accuracy of 92.4% and 93.0%, respectively. Local binary pattern alone has achieved an accuracy of 90.3%.

Deep Learning for Breast Cancer Classification of Deep Ultraviolet Fluorescence Images toward Intra-Operative Margin Assessment.

Breast conserving surgery aims at the complete removal of malignant lesions while minimizing healthy tissue loss. To ensure the balance between complete resection of the cancer and conservation of healthy tissue, intra-operative margin assessment is necessary. Deep ultraviolet (DUV) fluorescence scanning microscope provides fast whole-surface-imaging (WSI) of excised tissue with contrast between malignant and normal tissues. Then, an automated breast cancer classification method on DUV images is required for intra-operative margin assessment. Deep learning shows the promising results in breast cancer classification, but limited DUV image dataset poses overfitting challenge to train the robust network. To tackle this challenge, we partition the DUV WSI image into small patches and extract pathological features for each patch from a pre-trained network using a transfer learning approach. We feed pathological features into a decision-tree-based classifier and fuse patch-level classification results based on regional importance to determine malignant or benign WSI. Experimental results on 60 DUV images show that our proposed method outperforms the standard deep learning classification in terms of improving the classification performance and identifying cancerous regions.

Multi-Domain Neumann Network with Sensitivity Maps for Parallel MRI Reconstruction.

MRI is an imaging technology that non-invasively obtains high-quality medical images for diagnosis. However, MRI has the major disadvantage of long scan times which cause patient discomfort and image artifacts. As one of the methods for reducing the long scan time of MRI, the parallel MRI method for reconstructing a high-fidelity MR image from under-sampled multi-coil k-space data is widely used. In this study, we propose a method to reconstruct a high-fidelity MR image from under-sampled multi-coil k-space data using deep-learning. The proposed multi-domain Neumann network with sensitivity maps (MDNNSM) is based on the Neumann network and uses a forward model including coil sensitivity maps for parallel MRI reconstruction. The MDNNSM consists of three main structures: the CNN-based sensitivity reconstruction block estimates coil sensitivity maps from multi-coil under-sampled k-space data; the recursive MR image reconstruction block reconstructs the MR image; and the skip connection accumulates each output and produces the final result. Experiments using the fastMRI T1-weighted brain image dataset were conducted at acceleration factors of 2, 4, and 8. Qualitative and quantitative experimental results show that the proposed MDNNSM method reconstructs MR images more accurately than other methods, including the generalized autocalibrating partially parallel acquisitions (GRAPPA) method and the original Neumann network.

Pediatric chest-abdomen-pelvis and abdomen-pelvis CT images with expert organ contours.

PURPOSE: Organ autosegmentation efforts to date have largely been focused on adult populations, due to limited availability of pediatric training data. Pediatric patients may present additional challenges for organ segmentation. This paper describes a dataset of 359 pediatric chest-abdomen-pelvis and abdomen-pelvis Computed Tomography (CT) images with expert contours of up to 29 anatomical organ structures to aid in the evaluation and development of autosegmentation algorithms for pediatric CT imaging. ACQUISITION AND VALIDATION METHODS: The dataset collection consists of axial CT images in Digital Imaging and Communications in Medicine (DICOM) format of 180 male and 179 female pediatric chest-abdomen-pelvis or abdomen-pelvis exams acquired from one of three CT scanners at Children's Wisconsin. The datasets represent random pediatric cases based upon routine clinical indications. Subjects ranged in age from 5 days to 16 years, with a mean age of 7 years. The CT acquisition, contrast, and reconstruction protocols varied across the scanner models and patients, with specifications available in the DICOM headers. Expert contours were manually labeled for up to 29 organ structures per subject. Not all contours are available for all subjects, due to limited field of view or unreliable contouring due to high noise. DATA FORMAT AND USAGE NOTES: The data are available on The Cancer Imaging Archive (TCIA_ (https://www.cancerimagingarchive.net/) under the collection Pediatric-CT-SEG. The axial CT image slices for each subject are available in DICOM format. The expert contours are stored in a single DICOM RTSTRUCT file for each subject. The contour names are listed in Table 2. POTENTIAL APPLICATIONS: This dataset will enable the evaluation and development of organ autosegmentation algorithms for pediatric populations, which exhibit variations in organ shape and size across age. Automated organ segmentation from CT images has numerous applications including radiation therapy, diagnostic tasks, surgical planning, and patient-specific organ dose estimation.

Breast Cancer Histopathological Image Classification with Adversarial Image Synthesis.

Data limitation is one of the major challenges in applying deep learning to medical images. Data augmentation is a critical step to train robust and accurate deep learning models for medical images. In this research, we increase the size of a small dataset by using an Auxiliary Classifier Generative Adversarial Network (ACGAN) which generates realistic images along with their class labels.We evaluate the effectiveness of our ACGAN augmentation method by performing breast cancer histopathological image classification with deep convolutional neural network (dCNN) classifiers trained on our enhanced dataset. For our classifier, we use a transfer learning approach where the convolutional features are extracted from a pertained model and subsequently fed into several extreme gradient boosting (XGBoost) classifiers. Our experimental results on Breast Cancer Histopathological (BreakHis) dataset show that ACGAN data augmentation, along with our XGBoost classifier increases the classification accuracy by 9.35% for binary classification (benign vs. malignant) and 8.88% for four-class tumor sub-type classification compared with standard transfer learning approach.

Deep Learning Approach for Dynamic Sparse Sampling for High-Throughput Mass Spectrometry Imaging.

A Supervised Learning Approach for Dynamic Sampling (SLADS) addresses traditional issues with the incorporation of stochastic processes into a compressed sensing method. Statistical features, extracted from a sample reconstruction, estimate entropy reduction with regression models, in order to dynamically determine optimal sampling locations. This work introduces an enhanced SLADS method, in the form of a Deep Learning Approach for Dynamic Sampling (DLADS), showing reductions in sample acquisition times for high-fidelity reconstructions between ~ 70-80% over traditional rectilinear scanning. These improvements are demonstrated for dimensionally asymmetric, high-resolution molecular images of mouse uterine and kidney tissues, as obtained using Nanospray Desorption ElectroSpray Ionization (nano-DESI) Mass Spectrometry Imaging (MSI). The methodology for training set creation is adjusted to mitigate stretching artifacts generated when using prior SLADS approaches. Transitioning to DLADS removes the need for feature extraction, further advanced with the employment of convolutional layers to leverage inter-pixel spatial relationships. Additionally, DLADS demonstrates effective generalization, despite dissimilar training and testing data. Overall, DLADS is shown to maximize potential experimental throughput for nano-DESI MSI.

Enhancing Reproductive Organ Segmentation in Pediatric CT via Adversarial Learning.

Accurately segmenting organs in abdominal computed tomography (CT) scans is crucial for clinical applications such as pre-operative planning and dose estimation. With the recent advent of deep learning algorithms, many robust frameworks have been proposed for organ segmentation in abdominal CT images. However, many of these frameworks require large amounts of training data in order to achieve high segmentation accuracy. Pediatric abdominal CT images containing reproductive organs are particularly hard to obtain since these organs are extremely sensitive to ionizing radiation. Hence, it is extremely challenging to train automatic segmentation algorithms on organs such as the uterus and the prostate. To address these issues, we propose a novel segmentation network with a built-in auxiliary classifier generative adversarial network (ACGAN) that conditionally generates additional features during training. The proposed CFG-SegNet (conditional feature generation segmentation network) is trained on a single loss function which combines adversarial loss, reconstruction loss, auxiliary classifier loss and segmentation loss. 2.5D segmentation experiments are performed on a custom data set containing 24 female CT volumes containing the uterus and 40 male CT volumes containing the prostate. CFG-SegNet achieves an average segmentation accuracy of 0.929 DSC (Dice Similarity Coefficient) on the prostate and 0.724 DSC on the uterus with 4-fold cross validation. The results show that our network is high-performing and has the potential to precisely segment difficult organs with few available training images.

Rapid assessment of breast tumor margins using deep ultraviolet fluorescence scanning microscopy.

SIGNIFICANCE: Re-excision rates for women with invasive breast cancer undergoing breast conserving surgery (or lumpectomy) have decreased in the past decade but remain substantial. This is mainly due to the inability to assess the entire surface of an excised lumpectomy specimen efficiently and accurately during surgery. AIM: The goal of this study was to develop a deep-ultraviolet scanning fluorescence microscope (DUV-FSM) that can be used to accurately and rapidly detect cancer cells on the surface of excised breast tissue. APPROACH: A DUV-FSM was used to image the surfaces of 47 (31 malignant and 16 normal/benign) fresh breast tissue samples stained in propidium iodide and eosin Y solutions. A set of fluorescence images were obtained from each sample using low magnification (4 × ) and fully automated scanning. The images were stitched to form a color image. Three nonmedical evaluators were trained to interpret and assess the fluorescence images. Nuclear-cytoplasm ratio (N/C) was calculated and used for tissue classification. RESULTS: DUV-FSM images a breast sample with subcellular resolution at a speed of 1.0 min / cm2. Fluorescence images show excellent visual contrast in color, tissue texture, cell density, and shape between invasive carcinomas and their normal counterparts. Visual interpretation of fluorescence images by nonmedical evaluators was able to distinguish invasive carcinoma from normal samples with high sensitivity (97.62%) and specificity (92.86%). Using N/C alone was able to differentiate patch-level invasive carcinoma from normal breast tissues with reasonable sensitivity (81.5%) and specificity (78.5%). CONCLUSIONS: DUV-FSM achieved a good balance between imaging speed and spatial resolution with excellent contrast, which allows either visual or quantitative detection of invasive cancer cells on the surfaces of a breast surgical specimen.

AGE-CONDITIONED SYNTHESIS OF PEDIATRIC COMPUTED TOMOGRAPHY WITH AUXILIARY CLASSIFIER GENERATIVE ADVERSARIAL NETWORKS.

Deep learning is a popular and powerful tool in computed tomography (CT) image processing such as organ segmentation, but its requirement of large training datasets remains a challenge. Even though there is a large anatomical variability for children during their growth, the training datasets for pediatric CT scans are especially hard to obtain due to risks of radiation to children. In this paper, we propose a method to conditionally synthesize realistic pediatric CT images using a new auxiliary classifier generative adversarial network (ACGAN) architecture by taking age information into account. The proposed network generated age-conditioned high-resolution CT images to enrich pediatric training datasets.

Prior-Guided Metal Artifact Reduction for Iterative X-Ray Computed Tomography.

High-attenuation materials pose significant challenges to computed tomographic imaging. Formed of high mass-density and high atomic number elements, they cause more severe beam hardening and scattering artifacts than do water-like materials. Pre-corrected line-integral density measurements are no longer linearly proportional to the path lengths, leading to reconstructed image suffering from streaking artifacts extending from metal, often along highest-density directions. In this paper, a novel prior-based iterative approach is proposed to reduce metal artifacts. It combines the superiority of statistical methods with the benefits of sinogram completion methods to estimate and correct metal-induced biases. Preliminary results show minimized residual artifacts and significantly improved image quality.

Dynamic Sparse Sampling for Confocal Raman Microscopy.

The total number of data points required for image generation in Raman microscopy was greatly reduced using sparse sampling strategies, in which the preceding set of measurements informed the next most information-rich sampling location. Using this approach, chemical images of pharmaceutical materials were obtained with >99% accuracy from 15.8% sampling, representing an ∼6-fold reduction in measurement time relative to full field of view rastering with comparable image quality. This supervised learning approach to dynamic sampling (SLADS) has the distinct advantage of being directly compatible with standard confocal Raman instrumentation. Furthermore, SLADS is not limited to Raman imaging, potentially providing time-savings in image reconstruction whenever the single-pixel measurement time is the limiting factor in image generation.

Second Harmonic Generation Guided Raman Spectroscopy for Sensitive Detection of Polymorph Transitions.

Second harmonic generation (SHG) was integrated with Raman spectroscopy for the analysis of pharmaceutical materials. Particulate formulations of clopidogrel bisulfate were prepared in two crystal forms (Form I and Form II). Image analysis approaches enable automated identification of particles by bright field imaging, followed by classification by SHG. Quantitative SHG microscopy enabled discrimination of crystal form on a per particle basis with 99.95% confidence in a total measurement time of ∼10 ms per particle. Complementary measurements by Raman and synchrotron XRD are in excellent agreement with the classifications made by SHG, with measurement times of ∼1 min and several seconds per particle, respectively. Coupling these capabilities with at-line monitoring may enable real-time feedback for reaction monitoring during pharmaceutical production to favor the more bioavailable but metastable Form I with limits of detection in the ppm regime.

Dynamic X-ray diffraction sampling for protein crystal positioning.

A sparse supervised learning approach for dynamic sampling (SLADS) is described for dose reduction in diffraction-based protein crystal positioning. Crystal centering is typically a prerequisite for macromolecular diffraction at synchrotron facilities, with X-ray diffraction mapping growing in popularity as a mechanism for localization. In X-ray raster scanning, diffraction is used to identify the crystal positions based on the detection of Bragg-like peaks in the scattering patterns; however, this additional X-ray exposure may result in detectable damage to the crystal prior to data collection. Dynamic sampling, in which preceding measurements inform the next most information-rich location to probe for image reconstruction, significantly reduced the X-ray dose experienced by protein crystals during positioning by diffraction raster scanning. The SLADS algorithm implemented herein is designed for single-pixel measurements and can select a new location to measure. In each step of SLADS, the algorithm selects the pixel, which, when measured, maximizes the expected reduction in distortion given previous measurements. Ground-truth diffraction data were obtained for a 5 µm-diameter beam and SLADS reconstructed the image sampling 31% of the total volume and only 9% of the interior of the crystal greatly reducing the X-ray dosage on the crystal. Using in situ two-photon-excited fluorescence microscopy measurements as a surrogate for diffraction imaging with a 1 µm-diameter beam, the SLADS algorithm enabled image reconstruction from a 7% sampling of the total volume and 12% sampling of the interior of the crystal. When implemented into the beamline at Argonne National Laboratory, without ground-truth images, an acceptable reconstruction was obtained with 3% of the image sampled and approximately 5% of the crystal. The incorporation of SLADS into X-ray diffraction acquisitions has the potential to significantly minimize the impact of X-ray exposure on the crystal by limiting the dose and area exposed for image reconstruction and crystal positioning using data collection hardware present in most macromolecular crystallography end-stations.

Synchrotron X-Ray Diffraction Dynamic Sampling for Protein Crystal Centering.

A supervised learning approach for dynamic sampling (SLADS) was developed to reduce X-ray exposure prior to data collection in protein structure determination. Implementation of this algorithm allowed reduction of the X-ray dose to the central core of the crystal by up to 20-fold compared to current raster scanning approaches. This dose reduction corresponds directly to a reduction on X-ray damage to the protein crystals prior to data collection for structure determination. Implementation at a beamline at Argonne National Laboratory suggests promise for the use of the SLADS approach to aid in the analysis of X-ray labile crystals. The potential benefits match a growing need for improvements in automated approaches for microcrystal positioning.

Regional manifold learning for disease classification.

While manifold learning from images itself has become widely used in medical image analysis, the accuracy of existing implementations suffers from viewing each image as a single data point. To address this issue, we parcellate images into regions and then separately learn the manifold for each region. We use the regional manifolds as low-dimensional descriptors of high-dimensional morphological image features, which are then fed into a classifier to identify regions affected by disease. We produce a single ensemble decision for each scan by the weighted combination of these regional classification results. Each weight is determined by the regional accuracy of detecting the disease. When applied to cardiac magnetic resonance imaging of 50 normal controls and 50 patients with reconstructive surgery of Tetralogy of Fallot, our method achieves significantly better classification accuracy than approaches learning a single manifold across the entire image domain.