Dilated Heterogeneous Convolution for Cell Detection and Segmentation Based on Mask R-CNN.

Owing to the variable shapes, large size difference, uneven grayscale, and dense distribution among biological cells in an image, it is very difficult to accurately detect and segment cells. Especially, it is a serious challenge for some microscope imaging devices with limited resources owing to a large number of learning parameters and computational burden when using the standard Mask R-CNN. In this work, we propose a mask R-DHCNN for cell detection and segmentation. More specifically, Dilation Heterogeneous Convolution (DHConv) is proposed by designing a novel convolutional kernel structure (i.e., DHConv), which integrates the strengths of the heterogeneous kernel structure and dilated convolution. Then, the traditional homogeneous convolution structure of the standard Mask R-CNN is replaced with the proposed DHConv module to it adapt to shape and size differences encountered in cell detection and segmentation tasks. Finally, a series of comparison and ablation experiments are conducted on various biological cell datasets (such as U373, GoTW1, SIM+, and T24) to verify the effectiveness of the proposed method. The results show that the proposed method can obtain better performance than some state-of-the-art methods in multiple metrics (including AP, Precision, Recall, Dice, and PQ) while maintaining competitive FLOPs and FPS.

Adaptively Customizing Activation Functions for Various Layers.

To enhance the nonlinearity of neural networks and increase their mapping abilities between the inputs and response variables, activation functions play a crucial role to model more complex relationships and patterns in the data. In this work, a novel methodology is proposed to adaptively customize activation functions only by adding very few parameters to the traditional activation functions such as Sigmoid, Tanh, and rectified linear unit (ReLU). To verify the effectiveness of the proposed methodology, some theoretical and experimental analysis on accelerating the convergence and improving the performance is presented, and a series of experiments are conducted based on various network models (such as AlexNet, VggNet, GoogLeNet, ResNet and DenseNet), and various datasets (such as CIFAR10, CIFAR100, miniImageNet, PASCAL VOC, and COCO). To further verify the validity and suitability in various optimization strategies and usage scenarios, some comparison experiments are also implemented among different optimization strategies (such as SGD, Momentum, AdaGrad, AdaDelta, and ADAM) and different recognition tasks such as classification and detection. The results show that the proposed methodology is very simple but with significant performance in convergence speed, precision, and generalization, and it can surpass other popular methods such as ReLU and adaptive functions such as Swish in almost all experiments in terms of overall performance.

A literature survey of MR-based brain tumor segmentation with missing modalities.

Multimodal MR brain tumor segmentation is one of the hottest issues in the community of medical image processing. However, acquiring the complete set of MR modalities is not always possible in clinical practice, due to the acquisition protocols, image corruption, scanner availability, scanning cost or allergies to certain contrast materials. The missing information can cause some restraints to brain tumor diagnosis, monitoring, treatment planning and prognosis. Thus, it is highly desirable to develop brain tumor segmentation methods to address the missing modalities problem. Based on the recent advancements, in this review, we provide a detailed analysis of the missing modality issue in MR-based brain tumor segmentation. First, we briefly introduce the biomedical background concerning brain tumor, MR imaging techniques, and the current challenges in brain tumor segmentation. Then, we provide a taxonomy of the state-of-the-art methods with five categories, namely, image synthesis-based method, latent feature space-based model, multi-source correlation-based method, knowledge distillation-based method, and domain adaptation-based method. In addition, the principles, architectures, benefits and limitations are elaborated in each method. Following that, the corresponding datasets and widely used evaluation metrics are described. Finally, we analyze the current challenges and provide a prospect for future development trends. This review aims to provide readers with a thorough knowledge of the recent contributions in the field of brain tumor segmentation with missing modalities and suggest potential future directions.

Medical image augmentation for lesion detection using a texture-constrained multichannel progressive GAN.

Lesion detectors based on deep learning can assist doctors in diagnosing diseases. However, the performance of current detectors is likely to be unsatisfactory due to the scarcity of training samples. Therefore, it is beneficial to use image generation to augment the training set of a detector. However, when the imaging texture of the medical image is relatively delicate, the synthesized image generated by an existing method may be too poor in quality to meet the training requirements of the detectors. In this regard, a medical image augmentation method, namely, a texture-constrained multichannel progressive generative adversarial network (TMP-GAN), is proposed in this work. TMP-GAN uses joint training of multiple channels to effectively avoid the typical shortcomings of the current generation methods. It also uses an adversarial learning-based texture discrimination loss to further improve the fidelity of the synthesized images. In addition, TMP-GAN employs a progressive generation mechanism to steadily improve the accuracy of the medical image synthesizer. Experiments on the publicly available dataset CBIS-DDMS and our pancreatic tumor dataset show that the precision/recall/F1-score of the detector trained on the TMP-GAN augmented dataset improves by 2.59%/2.70%/2.77% and 2.44%/2.06%/2.36%, respectively, compared to the optimal results of other data augmentation methods. The FROC curve of the detector is also better than the curve from the contrast-augmented trained dataset. Therefore, we believe the proposed TMP-GAN is a practical technique to efficiently implement lesion detection case studies.

Generative Adversarial Networks in Medical Image augmentation: A review.

OBJECT: With the development of deep learning, the number of training samples for medical image-based diagnosis and treatment models is increasing. Generative Adversarial Networks (GANs) have attracted attention in medical image processing due to their excellent image generation capabilities and have been widely used in data augmentation. In this paper, a comprehensive and systematic review and analysis of medical image augmentation work are carried out, and its research status and development prospects are reviewed. METHOD: This paper reviews 105 medical image augmentation related papers, which mainly collected by ELSEVIER, IEEE Xplore, and Springer from 2018 to 2021. We counted these papers according to the parts of the organs corresponding to the images, and sorted out the medical image datasets that appeared in them, the loss function in model training, and the quantitative evaluation metrics of image augmentation. At the same time, we briefly introduce the literature collected in three journals and three conferences that have received attention in medical image processing. RESULT: First, we summarize the advantages of various augmentation models, loss functions, and evaluation metrics. Researchers can use this information as a reference when designing augmentation tasks. Second, we explore the relationship between augmented models and the amount of the training set, and tease out the role that augmented models may play when the quality of the training set is limited. Third, the statistical number of papers shows that the development momentum of this research field remains strong. Furthermore, we discuss the existing limitations of this type of model and suggest possible research directions. CONCLUSION: We discuss GAN-based medical image augmentation work in detail. This method effectively alleviates the challenge of limited training samples for medical image diagnosis and treatment models. It is hoped that this review will benefit researchers interested in this field.

Deep co-supervision and attention fusion strategy for automatic COVID-19 lung infection segmentation on CT images.

Due to the irregular shapes,various sizes and indistinguishable boundaries between the normal and infected tissues, it is still a challenging task to accurately segment the infected lesions of COVID-19 on CT images. In this paper, a novel segmentation scheme is proposed for the infections of COVID-19 by enhancing supervised information and fusing multi-scale feature maps of different levels based on the encoder-decoder architecture. To this end, a deep collaborative supervision (Co-supervision) scheme is proposed to guide the network learning the features of edges and semantics. More specifically, an Edge Supervised Module (ESM) is firstly designed to highlight low-level boundary features by incorporating the edge supervised information into the initial stage of down-sampling. Meanwhile, an Auxiliary Semantic Supervised Module (ASSM) is proposed to strengthen high-level semantic information by integrating mask supervised information into the later stage. Then an Attention Fusion Module (AFM) is developed to fuse multiple scale feature maps of different levels by using an attention mechanism to reduce the semantic gaps between high-level and low-level feature maps. Finally, the effectiveness of the proposed scheme is demonstrated on four various COVID-19 CT datasets. The results show that the proposed three modules are all promising. Based on the baseline (ResUnet), using ESM, ASSM, or AFM alone can respectively increase Dice metric by 1.12%, 1.95%,1.63% in our dataset, while the integration by incorporating three models together can rise 3.97%. Compared with the existing approaches in various datasets, the proposed method can obtain better segmentation performance in some main metrics, and can achieve the best generalization and comprehensive performance.

CT classification model of pancreatic serous cystic neoplasms and mucinous cystic neoplasms based on a deep neural network.

BACKGROUND: At present, numerous challenges exist in the diagnosis of pancreatic SCNs and MCNs. After the emergence of artificial intelligence (AI), many radiomics research methods have been applied to the identification of pancreatic SCNs and MCNs. PURPOSE: A deep neural network (DNN) model termed Multi-channel-Multiclassifier-Random Forest-ResNet (MMRF-ResNet) was constructed to provide an objective CT imaging basis for differential diagnosis between pancreatic serous cystic neoplasms (SCNs) and mucinous cystic neoplasms (MCNs). MATERIALS AND METHODS: This study is a retrospective analysis of pancreatic unenhanced and enhanced CT images in 63 patients with pancreatic SCNs and 47 patients with MCNs (3 of which were mucinous cystadenocarcinoma) confirmed by pathology from December 2010 to August 2016. Different image segmented methods (single-channel manual outline ROI image and multi-channel image), feature extraction methods (wavelet, LBP, HOG, GLCM, Gabor, ResNet, and AlexNet) and classifiers (KNN, Softmax, Bayes, random forest classifier, and Majority Voting rule method) are used to classify the nature of the lesion in each CT image (SCNs/MCNs). Then, the comparisons of classification results were made based on sensitivity, specificity, precision, accuracy, F1 score, and area under the receiver operating characteristic curve (AUC), with pathological results serving as the gold standard. RESULTS: Multi-channel-ResNet (AUC 0.98) was superior to Manual-ResNet (AUC 0.91).CT image characteristics of lesions extracted by ResNet are more representative than wavelet, LBP, HOG, GLCM, Gabor, and AlexNet. Compared to the use of three classifiers alone and Majority Voting rule method, the use of the MMRF-ResNet model exhibits a better evaluation effect (AUC 0.96) for the classification of the pancreatic SCNs and MCNs. CONCLUSION: The CT image classification model MMRF-ResNet is an effective method to distinguish between pancreatic SCNs and MCNs.

An adaptive learning method of anchor shape priors for biological cells detection and segmentation.

BACKGROUND AND OBJECTIVE: Owing to the variable shapes, large size difference, uneven grayscale and dense distribution among biological cells in an image, it is still a challenging task for the standard Mask R-CNN to accurately detect and segment cells. Especially, the state-of-the-art anchor-based methods fail to generate the anchors of sufficient scales effectively according to the various sizes and shapes of cells, thereby hardly covering all scales of cells. METHODS: We propose an adaptive approach to learn the anchor shape priors from data samples, and the aspect ratios and the number of anchor boxes can be dynamically adjusted by using ISODATA clustering algorithm instead of human prior knowledge in this work. To solve the identification difficulties for small objects owing to the multiple down-samplings in a deep learning-based method, a densification strategy of candidate anchors is presented to enhance the effects of identifying tinny size cells. Finally, a series of comparative experiments are conducted on various datasets to select appropriate a network structure and verify the effectiveness of the proposed methods. RESULTS: The results show that the ResNet-50-FPN combining the ISODATA method and densification strategy can obtain better performance than other methods in multiple metrics (including AP, Precision, Recall, Dice and PQ) for various biological cell datasets, such as U373, GoTW1, SIM+ and T24. CONCLUSIONS: Our adaptive algorithm could effectively learn the anchor shape priors from the various sizes and shapes of cells. It is promising and encouraging for a real-world anchor-based detection and segmentation application of biomedical engineering in the future.

Detection and Recognition for Life State of Cell Cancer Using Two-Stage Cascade CNNs.

Cancer cell detection and its stages recognition of life cycle are an important step to analyze cellular dynamics in the automation of cell based-experiments. In this work, a two-stage hierarchical method is proposed to detect and recognize different life stages of bladder cells by using two cascade Convolutional Neural Networks (CNNs). Initially, a hybrid object proposal algorithm (called EdgeSelective) by combining EdgeBoxes and Selective Search is proposed to generate candidate object proposals instead of a single Selective Search method in Region-CNN (R-CNN), and it can exploit the advantages of different mechanisms for generating proposals so that each cell in the image can be fully contained by at least one proposed region during the detection process. Then, the obtained cells from the previous step are used to train and extract features by employing CNNs for the purpose of cell life stage recognition. Finally, a series of comparison experiments are implemented. The results show that the proposed method can obtain better performance than traditional methods either in the stage of cell detection or cell life stage recognition, and it encourages and suggests the application in the development of new anticancer drug and cytopathology analysis of cancer patients in the near future.

Detection and segmentation of lymphomas in 3D PET images via clustering with entropy-based optimization strategy.

PURPOSE: Lymphoma detection and segmentation from PET images are critical tasks for cancer staging and treatment monitoring. However, it is still a challenge owing to the complexities of lymphoma PET data themselves, and the huge computational burdens and memory requirements for 3D volume data. In this work, an entropy-based optimization strategy for clustering is proposed to detect and segment lymphomas in 3D PET images. METHODS: To reduce computational complexity and add more feature information, billions of voxels in 3D volume data are first aggregated into supervoxels. Then, such supervoxels serve as basic data units for further clustering by using DBSCAN algorithm, in which some new feature attributes based on physical spatial information and prior knowledge are proposed. In addition, more importantly, an entropy-based objective function is constructed to search the most appropriate parameters of DBSCAN to obtain the optimal clustering results by using a genetic algorithm. This step allows to automatically adapt the parameters to each patient. Finally, a series of comparison experiments among various feature attributes are performed. RESULTS: 48 patient data are conducted, showing the combination of three features, supervoxel intensity, geographic coordinates and organ distributions, can achieve good performance and the proposed entropy-based optimization scheme has more advantages than the existing methods. CONCLUSION: The proposed entropy-based optimization strategy for clustering by integrating physical spatial attributes and prior knowledge can achieve better performance than traditional methods.

Mathematical and Computational Modeling in Complex Biological Systems.

The biological process and molecular functions involved in the cancer progression remain difficult to understand for biologists and clinical doctors. Recent developments in high-throughput technologies urge the systems biology to achieve more precise models for complex diseases. Computational and mathematical models are gradually being used to help us understand the omics data produced by high-throughput experimental techniques. The use of computational models in systems biology allows us to explore the pathogenesis of complex diseases, improve our understanding of the latent molecular mechanisms, and promote treatment strategy optimization and new drug discovery. Currently, it is urgent to bridge the gap between the developments of high-throughput technologies and systemic modeling of the biological process in cancer research. In this review, we firstly studied several typical mathematical modeling approaches of biological systems in different scales and deeply analyzed their characteristics, advantages, applications, and limitations. Next, three potential research directions in systems modeling were summarized. To conclude, this review provides an update of important solutions using computational modeling approaches in systems biology.

Nonlinear adaptive PID control for greenhouse environment based on RBF network.

This paper presents a hybrid control strategy, combining Radial Basis Function (RBF) network with conventional proportional, integral, and derivative (PID) controllers, for the greenhouse climate control. A model of nonlinear conservation laws of enthalpy and matter between numerous system variables affecting the greenhouse climate is formulated. RBF network is used to tune and identify all PID gain parameters online and adaptively. The presented Neuro-PID control scheme is validated through simulations of set-point tracking and disturbance rejection. We compare the proposed adaptive online tuning method with the offline tuning scheme that employs Genetic Algorithm (GA) to search the optimal gain parameters. The results show that the proposed strategy has good adaptability, strong robustness and real-time performance while achieving satisfactory control performance for the complex and nonlinear greenhouse climate control system, and it may provide a valuable reference to formulate environmental control strategies for actual application in greenhouse production.

A compatible control algorithm for greenhouse environment control based on MOCC strategy.

Conventional methods used for solving greenhouse environment multi-objective conflict control problems lay excessive emphasis on control performance and have inadequate consideration for both energy consumption and special requirements for plant growth. The resulting solution will cause higher energy cost. However, during the long period of work and practice, we find that it may be more reasonable to adopt interval or region control objectives instead of point control objectives. In this paper, we propose a modified compatible control algorithm, and employ Multi-Objective Compatible Control (MOCC) strategy and an extant greenhouse model to achieve greenhouse climate control based on feedback control architecture. A series of simulation experiments through various comparative studies are presented to validate the feasibility of the proposed algorithm. The results are encouraging and suggest the energy-saving application to real-world engineering problems in greenhouse production. It may be valuable and helpful to formulate environmental control strategies, and to achieve high control precision and low energy cost for real-world engineering application in greenhouse production. Moreover, the proposed approach has also potential to be useful for other practical control optimization problems with the features like the greenhouse environment control system.

Multi-objective control optimization for greenhouse environment using evolutionary algorithms.

This paper investigates the issue of tuning the Proportional Integral and Derivative (PID) controller parameters for a greenhouse climate control system using an Evolutionary Algorithm (EA) based on multiple performance measures such as good static-dynamic performance specifications and the smooth process of control. A model of nonlinear thermodynamic laws between numerous system variables affecting the greenhouse climate is formulated. The proposed tuning scheme is tested for greenhouse climate control by minimizing the integrated time square error (ITSE) and the control increment or rate in a simulation experiment. The results show that by tuning the gain parameters the controllers can achieve good control performance through step responses such as small overshoot, fast settling time, and less rise time and steady state error. Besides, it can be applied to tuning the system with different properties, such as strong interactions among variables, nonlinearities and conflicting performance criteria. The results implicate that it is a quite effective and promising tuning method using multi-objective optimization algorithms in the complex greenhouse production.