An interpretable decision-support model for breast cancer diagnosis using histopathology images.

Microscopic examination of biopsy tissue slides is perceived as the gold-standard methodology for the confirmation of presence of cancer cells. Manual analysis of an overwhelming inflow of tissue slides is highly susceptible to misreading of tissue slides by pathologists. A computerized framework for histopathology image analysis is conceived as a diagnostic tool that greatly benefits pathologists, augmenting definitive diagnosis of cancer. Convolutional Neural Network (CNN) turned out to be the most adaptable and effective technique in the detection of abnormal pathologic histology. Despite their high sensitivity and predictive power, clinical translation is constrained by a lack of intelligible insights into the prediction. A computer-aided system that can offer a definitive diagnosis and interpretability is therefore highly desirable. Conventional visual explanatory techniques, Class Activation Mapping (CAM), combined with CNN models offers interpretable decision making. The major challenge in CAM is, it cannot be optimized to create the best visualization map. CAM also decreases the performance of the CNN models. To address this challenge, we introduce a novel interpretable decision-support model using CNN with a trainable attention mechanism using response-based feed-forward visual explanation. We introduce a variant of DarkNet19 CNN model for the classification of histopathology images. In order to achieve visual interpretation as well as boost the performance of the DarkNet19 model, an attention branch is integrated with DarkNet19 network forming Attention Branch Network (ABN). The attention branch uses a convolution layer of DarkNet19 and Global Average Pooling (GAP) to model the context of the visual features and generate a heatmap to identify the region of interest. Finally, the perception branch is constituted using a fully connected layer to classify images. We trained and validated our model using more than 7000 breast cancer biopsy slide images from an openly available dataset and achieved 98.7% accuracy in the binary classification of histopathology images. The observations substantiated the enhanced clinical interpretability of the DarkNet19 CNN model, supervened by the attention branch, besides delivering a 3%-4% performance boost of the baseline model. The cancer regions highlighted by the proposed model correlate well with the findings of an expert pathologist. The coalesced approach of unifying attention branch with the CNN model capacitates pathologists with augmented diagnostic interpretability of histological images with no detriment to state-of-art performance. The model's proficiency in pinpointing the region of interest is an added bonus that can lead to accurate clinical translation of deep learning models that underscore clinical decision support.

Undersampled single-shell to MSMT fODF reconstruction using CNN-based ODE solver.

BACKGROUND AND OBJECTIVE: Diffusion MRI (dMRI) has been considered one of the most popular non-invasive techniques for studying the human brain's white matter (WM). dMRI is used to delineate the brain's microstructure by approximating the WM region's fiber tracts. The achieved fiber tracts can be utilized to assess mental diseases like Multiple sclerosis, ADHD, Seizures, Intellectual disability, and others. New techniques such as high angular resolution diffusion-weighted imaging (HARDI) have been developed, providing precise fiber directions, and overcoming the limitation of traditional DTI. Unlike Single-shell, Multi-shell HARDI provides tissue fractions for white matter, gray matter, and cerebrospinal fluid, resulting in a Multi-shell Multi-tissue fiber orientation distribution function (MSMT fODF). This MSMT fODF comes up with more precise fiber directions than a Single-shell, which helps to get correct fiber tracts. In addition, various multi-compartment diffusion models, including as CHARMED and NODDI, have been developed to describe the brain tissue microstructural information. This type of model requires multi-shell data to obtain more specific tissue microstructural information. However, a major concern with multi-shell is that it takes a longer scanning time restricting its use in clinical applications. In addition, most of the existing dMRI scanners with low gradient strengths commonly acquire a single b-value (shell) upto b=1000s/mm2 due to SNR (Signal-to-noise ratio) reasons and severe imaging artifacts. METHODS: To address this issue, we propose a CNN-based ordinary differential equations solver for the reconstruction of MSMT fODF from under-sampled and fully sampled Single-shell (b=1000s/mm2) dMRI. The proposed architecture consists of CNN-based Adams-Bash-forth and Runge-Kutta modules along with two loss functions, including L1 and total variation. RESULTS: We have shown quantitative results and visualization of fODF, fiber tracts, and structural connectivity for several brain regions on the publicly available HCP dataset. In addition, the obtained angular correlation coefficients for white matter and full brain are high, showing the proposed network's utility.Finally, we have also demonstrated the effect of noise by adjusting SNR from 5 to 50 and observed the network robustness. CONCLUSION: We can conclude that our model can accurately predict MSMT fODF from under-sampled or fully sampled Single-shell dMRI volumes.

TrGANet: Transforming 3T to 7T dMRI using Trapezoidal Rule and Graph based Attention Modules.

Diffusion MRI (dMRI) is a non-invasive tool for assessing the white matter region of the brain by approximating the fiber streamlines, structural connectivity, and estimation of microstructure. This modality can yield useful information for diagnosing several mental diseases as well as for surgical planning. The higher angular resolution diffusion imaging (HARDI) technique is helpful in obtaining more robust fiber tracts by getting a good approximation of regions where fibers cross. Moreover, HARDI is more sensitive to tissue changes and can accurately represent anatomical details in the human brain at higher magnetic strengths. In other words, magnetic strengths affect the quality of the image, and hence high magnetic strength has good tissue contrast with better spatial resolution. However, a higher magnetic strength scanner (like 7T) is costly and unaffordable to most hospitals. Hence, in this work, we have proposed a novel CNN architecture for the transformation of 3T to 7T dMRI. Additionally, we have also reconstructed the multi-shell multi-tissue fiber orientation distribution function (MSMT fODF) at 7T from single-shell 3T. The proposed architecture consists of a CNN-based ODE solver utilizing the Trapezoidal rule and graph-based attention layer alongwith L1 and total variation loss. Finally, the model has been validated on the HCP data set quantitatively and qualitatively.

DiabeticSense: A Non-Invasive, Multi-Sensor, IoT-Based Pre-Diagnostic System for Diabetes Detection Using Breath.

Diabetes mellitus is a widespread chronic metabolic disorder that requires regular blood glucose level surveillance. Current invasive techniques, such as finger-prick tests, often result in discomfort, leading to infrequent monitoring and potential health complications. The primary objective of this study was to design a novel, portable, non-invasive system for diabetes detection using breath samples, named DiabeticSense, an affordable digital health device for early detection, to encourage immediate intervention. The device employed electrochemical sensors to assess volatile organic compounds in breath samples, whose concentrations differed between diabetic and non-diabetic individuals. The system merged vital signs with sensor voltages obtained by processing breath sample data to predict diabetic conditions. Our research used clinical breath samples from 100 patients at a nationally recognized hospital to form the dataset. Data were then processed using a gradient boosting classifier model, and the performance was cross-validated. The proposed system attained a promising accuracy of 86.6%, indicating an improvement of 20.72% over an existing regression technique. The developed device introduces a non-invasive, cost-effective, and user-friendly solution for preliminary diabetes detection. This has the potential to increase patient adherence to regular monitoring.

Detection of mitotic HEp-2 cell images: role of feature representation and classification framework under class skew.

We propose and analyze a framework to detect and identify the mitotic type staining patterns among different non-mitotic (interphase) patterns on HEp-2 cell substrate specimen images. This is considered as a principal task in computer-aided diagnosis (CAD) of the autoimmune disorders. Due to the rare appearance of mitotic patterns in whole slide/specimen images, the sample skew between mitotic and non-mitotic patterns is an important consideration.We suggest to apply some effective samples skew balancing strategies for the task of classification between mitotic v/s interphase patterns. Another aspect of this study is to consider the morphology and texture-based differences between both the classes that can be incorporated through effective morphology and texture-based descriptors, including the Gabor and LM (Leung-Malik) filter banks and also through some contemporary filter banks derived from convolutional neural networks (CNN).The proposed framework is evaluated on a public dataset and we demonstrate good performance (0.99 or 1 Matthews correlation coefficient (MCC) in many cases), across various experiments. The study also presents a comparison between hand-engineered and CNN-based feature representation, along with the comparisons with state-of-the-art approaches. Hence, the framework proves to be a good solution for the mentioned skewed classification problem.

Single-shell to multi-shell dMRI transformation using spatial and volumetric multilevel hierarchical reconstruction framework.

Single or Multi-shell high angular resolution diffusion imaging (HARDI) has become an important dMRI acquisition technique for studying brain white matter fibers. Existing single-shell HARDI makes it challenging to estimate the intravoxel structure up to the desired resolution. However, multi-shell acquisition (with multiple b-values) can provide higher resolution for the intravoxel structure, which further helps in getting accurate fiber tracts; But, this comes at the cost of larger acquisition time and larger setup. Hence, we propose a novel deep learning architecture for the reconstruction of diffusion MRI volumes for different b-values (degree of diffusion weighting) using acquisitions at a fixed b-value (termed as single-shell) acquisition. This reconstruction has been performed in the spherical harmonics space to better manage varying gradient directions. In this work, we have demonstrated such a reconstruction for b = 3000 s/mm2 and b = 2000 s/mm2 from b = 1000 s/mm2. The proposed Multilevel Hierarchical Spherical Harmonics Coefficients Reconstruction (MHSH) framework takes advantage of contextual information within each slice as well as across the slices by involving Slice Level ReconNet (SLRNet) network and a Volumetric ROI Level ReconNet (VPLRNet) network, respectively. Three-loss functions have been used to optimize network learning, i.e., L1, Adversarial, and Total Variation Loss. Finally, the network is trained and validated on the publicly available HCP data-set with standard qualitative and quantitative performance measures and achieves promising results.

MHATC: Autism Spectrum Disorder Identification Utilizing Multi-Head Attention Encoder Along with Temporal Consolidation Modules.

Resting-state fMRI is commonly used for diagnosing Autism Spectrum Disorder (ASD) by using network-based functional connectivity. It has been shown that ASD is associated with brain regions and their inter-connections. However, discriminating based on connectivity patterns among imaging data of the control population and that of ASD patients' brains is a non-trivial task. In order to tackle said classification task, we propose a novel deep learning architecture (MHATC) consisting of multi-head attention and temporal consolidation modules for classifying an individual as a patient of ASD. The devised architecture results from an in-depth analysis of the limitations of current deep neural network solutions for similar applications. Our approach is not only robust but computationally efficient, which can allow its adoption in a variety of other research and clinical settings.

VRfRNet: Volumetric ROI fODF reconstruction network for estimation of multi-tissue constrained spherical deconvolution with only single shell dMRI.

Diffusion MRI (dMRI) is one of the most popular techniques for studying the brain structure, mainly the white matter region. Among several sampling methods in dMRI, the high angular resolution diffusion imaging (HARDI) technique has attracted researchers due to its more accurate fiber orientation estimation. However, the current single-shell HARDI makes the intravoxel structure challenging to estimate accurately. While multi-shell acquisition can address this problem, it takes a longer scanning time, restricting its use in clinical applications. In addition, most existing dMRI scanners with low gradient-strengths often acquire single-shell up to b=1000s/mm2 because of signal-to-noise ratio issues and severe image artefacts. Hence, we propose a novel generative adversarial network, VRfRNet, for the reconstruction of multi-shell multi-tissue fiber orientation distribution function from single-shell HARDI volumes. Such a transformation learning is performed in the spherical harmonics (SH) space, as raw input HARDI volume is transformed to SH coefficients to soften gradient directions. The proposed VRfRNet consists of several modules, such as multi-context feature enrichment module, feature level attention, and softmax level attention. In addition, three loss functions have been used to optimize network learning, including L1, adversarial, and total variation. The network is trained and tested using standard qualitative and quantitative performance metrics on the publicly available HCP data-set.

Heterogeneous ensemble with information theoretic diversity measure for human epithelial cell image classification.

In this work, we propose a heterogeneous committee (ensemble) of diverse members (classification approaches) to solve the problem of human epithelial (HEp-2) cell image classification using indirect Immunofluorescence (IIF) imaging. We hypothesize that an ensemble involving different feature representations can enable higher performance if individual members in the ensemble are sufficiently varied. These members are of two types: (1) CNN-based members, (2) traditional members. For the CNN members, we have employed the well-established ResNet, DenseNet, and Inception models, which have distinctive salient aspects. For the traditional members, we incorporate class-specific features which are characterized depending on visual morphological attributes, and some standard texture features. To select the members which are discriminating and not redundant, we use an information theoretic measure which considers the trade-off between individual accuracies and diversity among the members. For all selected members, a compelling fusion required to combine their outputs to reach a final decision. Thus, we also investigate various fusion methods that combine the opinion of the committee at different levels: maximum voting, product, decision template, Bayes, Dempster-Shafer, etc. The proposed method is evaluated using ICPR-2014 data which consists of more images than some previous datasets ICPR-2012 and demonstrate state-of-the-art performance. To check the effectiveness of the proposed methodology for other related datasets, we test our methodology with newly compiled large-scale HEp-2 dataset with 63K cell images and demonstrate comparable performance even with less number of training samples. The proposed method produces 99.80% and 86.03% accuracy respectively when tested on ICPR-2014 and a new large-scale data containing 63K samples. Graphical Abstract Overview of the proposed methodology.

Effectiveness of GAN-based Synthetic Samples Generation of Minority Patterns in HEp-2 Cell Images.

In this paper, we present a framework to address the augmentation of images for the rare and minor appearance of mitotic type staining patterns, for Human Epithelium Type2 (HEp-2) cell images. The identification of mitotic patterns among non-mitotic/interphase patterns is important in the process of diagnosis of various autoimmune disorders. This task leads to a pattern classification problem between mitotic v/s interphase. However, among the two classes, typically, the number of mitotic cells are relatively very less. Thus, in this work, we propose to generate synthetic mitotic samples, which can be used to augment the number of mitotic samples and balance the samples of mitotic and interphase patterns in classification paradigm. An effective feature representation is used, to validate the usefulness of the synthetic samples in classification task, along with a subjective validation done by a medical expert. The results demonstrate that the approach of generating and mingling synthetic samples with existing training data works well and yields good performance, with 0.98 balanced class accuracy (BcA) in one case, over a public dataset, i.e., UQ-SNP I3A Task-3 mitotic cell identification dataset.

Multi-Shell D-MRI Reconstruction via Residual Learning utilizing Encoder-Decoder Network with Attention (MSR-Net).

Contemporary diffusion MRI based analysis with HARDI, which provides more accurate fiber orientation, can be performed using single or multiple b-values (single or multi-shell). Single shell HARDI cannot provide volume fraction for different tissue types, which can produce bias and noisier results in estimation of fiber ODF. Multi-shell acquisition can resolve this issue. However, it requires more scanning time and is therefore not very well suited in clinical setting. Considering this, we propose a novel deep learning architecture, MSR-Net, for reconstruction of diffusion MRI volumes for some b-value using acquisitions at another b-value. In this work, we demonstrate this for b = 2000 s/mm2 and b = 1000 s/mm2. We learn such a transformation in the space of spherical harmonic coefficients. The proposed network consists of encoder-decoder along-with an attention module and a feature module. We have considered L2 and Content loss for optimizing and improving the performance. We have trained and validated the network using the HCP data-set with standard qualitative and quantitative performance measures.

Localization of common carotid artery transverse section in B-mode ultrasound images using faster RCNN: a deep learning approach.

Cardiologists can acquire important information related to patients' cardiac health using carotid artery stiffness, its lumen diameter (LD), and its carotid intima-media thickness (cIMT). The sonographers primarily concern about the location of the artery in B-mode ultrasound images. Localization using manual methods is tedious and time-consuming and also may lead to some errors. On the other hand, automated approaches are more objective and can provide the localization of the artery at near real time. Above arterial parameters may be determined after localization of the artery in real time.A novel method of localization of common carotid artery (CCA) transverse section is presented in this work. The method is known as fast region convolutional neural network (FRCNN)-based localization method and is designed using a stack of three layers viz. convolutional layers, fully connected layers, and pooling layers. These organized layers constitute a region proposal network (RPN) and an object class detection network (OCDN). We obtain an outcome as a bounding box along with a score of prediction around the cross-section of the CCA.B-mode ultrasound image database of CCA is split into training and testing set, to accomplish this, three partition methods K = 2, 5, and 10 are used in our work. The training is extended for 30, 200, and 2000 epochs in order to achieve fine-tuned features from the convolutional neural network. After 2000 epochs, we obtain 95% validation accuracy; however, mean of the accuracies up to 2000 epochs is 89.36% for K = 10 partitions protocol (training 90%, testing 10%). Generated CNN model is tested on a different dataset of 433 images and the acquired accuracy is 87.99%. Thus, the proposed method including an advanced deep learning technique demonstrates promising localization for carotid artery transverse section. Graphical abstract.

Detecting mitotic cells in HEp-2 images as anomalies via one class classifier.

We propose a novel framework for classification of mitotic v/s non-mitotic cells in a Computer Aided Diagnosis (CAD) system for Anti-Nuclear Antibodies (ANA) detection. In the proposed work, due to unique characteristics (the rare occurrence) of the mitotic cells, their identification is posed as an anomaly detection approach. This will resolve the issue of data imbalance, which can arise in the traditional binary classification paradigm for mitotic v/s non-mitotic cell image classification. Here, the characteristics of only non-mitotic/interphase cells are captured using a well-defined feature representation to characterize the non-mitotic class distribution well, and the mitotic class is posed as an anomalous class. This framework requires training data only for the majority (non-mitotic) class, to build the classification model. The feature representation of the non-mitotic class includes morphology, texture, and Convolutional Neural Network (CNN) based feature representations, coupled with Bag-of-Words (BoW) and Spatial Pyramid Pooling (SPP) based summarization techniques. For classification, in this work, we employ the One-Class Support Vector Machines (OC-SVM). The proposed classification framework is validated on a publicly available dataset, and across various experiments, we demonstrate comparable or better performance over binary classification, attaining 0.99 (max.) F-Score in one case. The proposed framework proves to be an effective way to solve the mentioned problem statement, where there are less number of samples in one of the classes.

Depth Map Restoration From Undersampled Data.

Depth map sensed by low-cost active sensor is often limited in resolution, whereas depth information achieved from structure from motion or sparse depth scanning techniques may result in a sparse point cloud. Achieving a high-resolution (HR) depth map from a low resolution (LR) depth map or densely reconstructing a sparse non-uniformly sampled depth map are fundamentally similar problems with different types of upsampling requirements. The first problem involves upsampling in a uniform grid, whereas the second type of problem requires an upsampling in a non-uniform grid. In this paper, we propose a new approach to address such issues in a unified framework, based on sparse representation. Unlike, most of the approaches of depth map restoration, our approach does not require an HR intensity image. Based on example depth maps, sub-dictionaries of exemplars are constructed, and are used to restore HR/dense depth map. In the case of uniform upsampling of LR depth map, an edge preserving constraint is used for preserving the discontinuity present in the depth map, and a pyramidal reconstruction strategy is applied in order to deal with higher upsampling factors. For upsampling of non-uniformly sampled sparse depth map, we compute the missing information in local patches from that from similar exemplars. Furthermore, we also suggest an alternative method of reconstructing dense depth map from very sparse non-uniformly sampled depth data by sequential cascading of uniform and non-uniform upsampling techniques. We provide a variety of qualitative and quantitative results to demonstrate the efficacy of our approach for depth map restoration.

Contemplation on marking scheme for Type X multiple choice questions, and an illustration of a practically applicable scheme.

Ever since its inception 100 years back, multiple choice items have been widely used as a method of assessment. It has certain inherent limitations such as inability to test higher cognitive skills, element of guesswork while answering, and issues related with marking schemes. Various marking schemes have been proposed in the past but they are not balanced, skewed, and complex, which are based on mathematical calculations which are typically not within the grasp of medical personnel. Type X questions has many advantages being easy to construct, can test multiple concepts/application/facets of a topic, cognitive skill of various level of hierarchy can be tested, and unlike Type K items, they are free from complicated coding. In spite of these advantages, they are not in common use due to complicated marking schemes. This is the reason we explored the aspects of methods of evaluation of multiple correct options multiple choice questions and came up with the simple, practically applicable, nonstringent but logical scoring system for the same. The rationale of the illustrated marking scheme is that it takes into consideration the distracter recognition ability of the examinee rather than relying on the ability only to select the correct response. Thus, examinee's true knowledge is tested, and he is rewarded accordingly for selecting a correct answer and omitting a distracter. The scheme also penalizes for not recognizing a distracter thus controlling guessing behavior. It is emphasized that if the illustrated scoring scheme is adopted, then Type X questions would come in common practice.

Harnessing group-sparsity regularization for resolution enhancement of lung 4D-CT.

A critical concern with lung 4D-CT is the low superior-inferior resolution, due to the consideration of radiation dose. We propose a resolution enhancement approach that reconstructs missing intermediate slices by exploiting the idea that information lost in one respiratory phase can be found in others, according to the complimentary nature of inter-phase information. Our approach is based on a patch-based framework that explores the role of group-sparsity involving groups of similar neighbouring patches. We discuss the regularizing role of group-sparsity, which helps in reducing the effect of noise and enables better enhancement of anatomical structures. Our results positively demonstrate the potential of group-sparsity for 4D-CT resolution enhancement.

Resolution enhancement of lung 4D-CT via group-sparsity.

PURPOSE: 4D-CT typically delivers more accurate information about anatomical structures in the lung, over 3D-CT, due to its ability to capture visual information of the lung motion across different respiratory phases. This helps to better determine the dose during radiation therapy for lung cancer. However, a critical concern with 4D-CT that substantially compromises this advantage is the low superior-inferior resolution due to less number of acquired slices, in order to control the CT radiation dose. To address this limitation, the authors propose an approach to reconstruct missing intermediate slices, so as to improve the superior-inferior resolution. METHODS: In this method the authors exploit the observation that sampling information across respiratory phases in 4D-CT can be complimentary due to lung motion. The authors' approach uses this locally complimentary information across phases in a patch-based sparse-representation framework. Moreover, unlike some recent approaches that treat local patches independently, the authors' approach employs the group-sparsity framework that imposes neighborhood and similarity constraints between patches. This helps in mitigating the trade-off between noise robustness and structure preservation, which is an important consideration in resolution enhancement. The authors discuss the regularizing ability of group-sparsity, which helps in reducing the effect of noise and enables better structural localization and enhancement. RESULTS: The authors perform extensive experiments on the publicly available DIR-Lab Lung 4D-CT dataset [R. Castillo, E. Castillo, R. Guerra, V. Johnson, T. McPhail, A. Garg, and T. Guerrero, "A framework for evaluation of deformable image registration spatial accuracy using large landmark point sets," Phys. Med. Biol. 54, 1849-1870 (2009)]. First, the authors carry out empirical parametric analysis of some important parameters in their approach. The authors then demonstrate, qualitatively as well as quantitatively, the ability of their approach to achieve more accurate and better localized results over bicubic interpolation as well as a related state-of-the-art approach. The authors also show results on some datasets with tumor, to further emphasize the clinical importance of their method. CONCLUSIONS: The authors have proposed to improve the superior-inferior resolution of 4D-CT by estimating intermediate slices. The authors' approach exploits neighboring constraints in the group-sparsity framework, toward the goal of achieving better localization and noise robustness. The authors' results are encouraging, and positively demonstrate the role of group-sparsity for 4D-CT resolution enhancement.

Resolution enhancement in multi-image stereo.

Under stereo settings, the twin problems of image superresolution (SR) and high-resolution (HR) depth estimation are intertwined. The subpixel registration information required for image superresolution is tightly coupled to the 3D structure. The effects of parallax and pixel averaging (inherent in the downsampling process) preclude a priori estimation of pixel motion for superresolution. These factors also compound the correspondence problem at low resolution (LR), which in turn affects the quality of the LR depth estimates. In this paper, we propose an integrated approach to estimate the HR depth and the SR image from multiple LR stereo observations. Our results demonstrate the efficacy of the proposed method in not only being able to bring out image details but also in enhancing the HR depth over its LR counterpart.