DLSIA: Deep Learning for Scientific Image Analysis.

DLSIA (Deep Learning for Scientific Image Analysis) is a Python-based machine learning library that empowers scientists and researchers across diverse scientific domains with a range of customizable convolutional neural network (CNN) architectures for a wide variety of tasks in image analysis to be used in downstream data processing. DLSIA features easy-to-use architectures, such as autoencoders, tunable U-Nets and parameter-lean mixed-scale dense networks (MSDNets). Additionally, this article introduces sparse mixed-scale networks (SMSNets), generated using random graphs, sparse connections and dilated convolutions connecting different length scales. For verification, several DLSIA-instantiated networks and training scripts are employed in multiple applications, including inpainting for X-ray scattering data using U-Nets and MSDNets, segmenting 3D fibers in X-ray tomographic reconstructions of concrete using an ensemble of SMSNets, and leveraging autoencoder latent spaces for data compression and clustering. As experimental data continue to grow in scale and complexity, DLSIA provides accessible CNN construction and abstracts CNN complexities, allowing scientists to tailor their machine learning approaches, accelerate discoveries, foster interdisciplinary collaboration and advance research in scientific image analysis.

Deploying Machine Learning Based Segmentation for Scientific Imaging Analysis at Synchrotron Facilities.

Scientific user facilities present a unique set of challenges for image processing due to the large volume of data generated from experiments and simulations. Furthermore, developing and implementing algorithms for real-time processing and analysis while correcting for any artifacts or distortions in images remains a complex task, given the computational requirements of the processing algorithms. In a collaborative effort across multiple Department of Energy national laboratories, the "MLExchange" project is focused on addressing these challenges. MLExchange is a Machine Learning framework deploying interactive web interfaces to enhance and accelerate data analysis. The platform allows users to easily upload, visualize, label, and train networks. The resulting models can be deployed on real data while both results and models could be shared with the scientists. The MLExchange web-based application for image segmentation allows for training, testing, and evaluating multiple machine learning models on hand-labeled tomography data. This environment provides users with an intuitive interface for segmenting images using a variety of machine learning algorithms and deep-learning neural networks. Additionally, these tools have the potential to overcome limitations in traditional image segmentation techniques, particularly for complex and low-contrast images.

A comparison of deep-learning-based inpainting techniques for experimental X-ray scattering.

The implementation is proposed of image inpainting techniques for the reconstruction of gaps in experimental X-ray scattering data. The proposed methods use deep learning neural network architectures, such as convolutional autoencoders, tunable U-Nets, partial convolution neural networks and mixed-scale dense networks, to reconstruct the missing information in experimental scattering images. In particular, the recovered pixel intensities are evaluated against their corresponding ground-truth values using the mean absolute error and the correlation coefficient metrics. The results demonstrate that the proposed methods achieve better performance than traditional inpainting algorithms such as biharmonic functions. Overall, tunable U-Net and mixed-scale dense network architectures achieved the best reconstruction performance among all the tested algorithms, with correlation coefficient scores greater than 0.9980.

MLExchange: A web-based platform enabling exchangeable machine learning workflows for scientific studies.

Machine learning (ML) algorithms are showing a growing trend in helping the scientific communities across different disciplines and institutions to address large and diverse data problems. However, many available ML tools are programmatically demanding and computationally costly. The MLExchange project aims to build a collaborative platform equipped with enabling tools that allow scientists and facility users who do not have a profound ML background to use ML and computational resources in scientific discovery. At the high level, we are targeting a full user experience where managing and exchanging ML algorithms, workflows, and data are readily available through web applications. Since each component is an independent container, the whole platform or its individual service(s) can be easily deployed at servers of different scales, ranging from a personal device (laptop, smart phone, etc.) to high performance clusters (HPC) accessed (simultaneously) by many users. Thus, MLExchange renders flexible using scenarios-users could either access the services and resources from a remote server or run the whole platform or its individual service(s) within their local network.

Mammary tumors in Sprague Dawley rats induced by N-ethyl-N-nitrosourea for evaluating terahertz imaging of breast cancer.

Purpose: The objective of this study is to quantitatively evaluate terahertz (THz) imaging for differentiating cancerous from non-cancerous tissues in mammary tumors developed in response to injection of N-ethyl-N-nitrosourea (ENU) in Sprague Dawley rats. Approach: While previous studies have investigated the biology of mammary tumors of this model, the current work is the first study to employ an imaging modality to visualize these tumors. A pulsed THz imaging system is utilized to experimentally collect the time-domain reflection signals from each pixel of the rat's excised tumor. A statistical segmentation algorithm based on the expectation-maximization (EM) classification method is implemented to quantitatively assess the obtained THz images. The model classification of cancer is reported in terms of the receiver operating characteristic (ROC) curves and the areas under the curves. Results: The obtained low-power microscopic images of 17 ENU-rat tumor sections exhibited the presence of healthy connective tissue adjacent to cancerous tissue. The results also demonstrated that high reflection THz signals were received from cancerous compared with non-cancerous tissues. Decent tumor classification was achieved using the EM method with values ranging from 83% to 96% in fresh tissues and 89% to 96% in formalin-fixed paraffin-embedded tissues. Conclusions: The proposed ENU breast tumor model of Sprague Dawley rats showed a potential to obtain cancerous tissues, such as human breast tumors, adjacent to healthy tissues. The implemented EM classification algorithm quantitatively demonstrated the ability of THz imaging in differentiating cancerous from non-cancerous tissues.

Breast Cancer Detection with Low-dimension Ordered Orthogonal Projection in Terahertz Imaging.

This paper proposes a new dimension reduction algorithm based on low-dimension ordered orthogonal projection (LOOP), which is used for cancer detection with terahertz (THz) images of freshly excised human breast cancer tissues. A THz image can be represented by a data cube with each pixel containing a high dimension spectrum vector covering several THz frequencies, where each frequency represents a different dimension in the vector. The proposed algorithm projects the high-dimension spectrum vector of each pixel within the THz image into a low-dimension subspace that contains the majority of the unique features embedded in the image. The low-dimension subspace is constructed by sequentially identifying its orthonormal basis vectors, such that each newly chosen basis vector represents the most unique information not contained by existing basis vectors. A multivariate Gaussian mixture model is used to represent the statistical distributions of the low-dimension feature vectors obtained from the proposed dimension reduction algorithm. The model parameters are iteratively learned by using unsupervised learning methods such as Markov chain Monte Carlo or expectation maximization, and the results are used to classify the various regions within a tumor sample. Experiment results demonstrate that the proposed method achieves apparent performance improvement in human breast cancer tissue over existing approaches such as one-dimension Markov chain Monte Carlo. The results confirm that the dimension reduction algorithm presented in this paper is a promising technique for breast cancer detection with THz images, and the classification results present a good correlation with respect to the histopathology results of the analyzed samples.

Pulsed terahertz imaging of breast cancer in freshly excised murine tumors.

This paper investigates terahertz (THz) imaging and classification of freshly excised murine xenograft breast cancer tumors. These tumors are grown via injection of E0771 breast adenocarcinoma cells into the flank of mice maintained on high-fat diet. Within 1 h of excision, the tumor and adjacent tissues are imaged using a pulsed THz system in the reflection mode. The THz images are classified using a statistical Bayesian mixture model with unsupervised and supervised approaches. Correlation with digitized pathology images is conducted using classification images assigned by a modal class decision rule. The corresponding receiver operating characteristic curves are obtained based on the classification results. A total of 13 tumor samples obtained from 9 tumors are investigated. The results show good correlation of THz images with pathology results in all samples of cancer and fat tissues. For tumor samples of cancer, fat, and muscle tissues, THz images show reasonable correlation with pathology where the primary challenge lies in the overlapping dielectric properties of cancer and muscle tissues. The use of a supervised regression approach shows improvement in the classification images although not consistently in all tissue regions. Advancing THz imaging of breast tumors from mice and the development of accurate statistical models will ultimately progress the technique for the assessment of human breast tumor margins.

Assessment of Terahertz Imaging for Excised Breast Cancer Tumors with Image Morphing.

This paper presents an image morphing algorithm for quantitative evaluation methodology of terahertz (THz) images of excised breast cancer tumors. Most current studies on the assessment of THz imaging rely on qualitative evaluation, and there is no established benchmark or procedure to quantify the THz imaging performance. The proposed morphing algorithm provides a tool to quantitatively align the THz image with the histopathology image. Freshly excised xenograft murine breast cancer tumors are imaged using the pulsed THz imaging and spectroscopy system in the reflection mode. Upon fixing the tumor tissue in formalin and embedding in paraffin, an FFPE tissue block is produced. A thin slice of the block is prepared for the pathology image while another THz reflection image is produced directly from the block. We developed an algorithm of mesh morphing using homography mapping of the histopathology image to adjust the alignment, shape, and resolution to match the external contour of the tissue in the THz image. Unlike conventional image morphing algorithms that rely on internal features of the source and target images, only the external contour of the tissue is used to avoid bias. Unsupervised Bayesian learning algorithm is applied to THz images to classify the tissue regions of cancer, fat, and muscles present in xenograft breast tumors. The results demonstrate that the proposed mesh morphing algorithm can provide more effective and accurate evaluation of THz imaging compared with existing algorithms. The results also showed that while THz images of FFPE tissue are highly in agreement with pathology images, challenges remain in assessing THz imaging of fresh tissue.