An adaptive superpixel based hand gesture tracking and recognition system.

We propose an adaptive and robust superpixel based hand gesture tracking system, in which hand gestures drawn in free air are recognized from their motion trajectories. First we employed the motion detection of superpixels and unsupervised image segmentation to detect the moving target hand using the first few frames of the input video sequence. Then the hand appearance model is constructed from its surrounding superpixels. By incorporating the failure recovery and template matching in the tracking process, the target hand is tracked by an adaptive superpixel based tracking algorithm, where the problem of hand deformation, view-dependent appearance invariance, fast motion, and background confusion can be well handled to extract the correct hand motion trajectory. Finally, the hand gesture is recognized by the extracted motion trajectory with a trained SVM classifier. Experimental results show that our proposed system can achieve better performance compared to the existing state-of-the-art methods with the recognition accuracy 99.17% for easy set and 98.57 for hard set.

Digital image forgery detection using JPEG features and local noise discrepancies.

Wide availability of image processing software makes counterfeiting become an easy and low-cost way to distort or conceal facts. Driven by great needs for valid forensic technique, many methods have been proposed to expose such forgeries. In this paper, we proposed an integrated algorithm which was able to detect two commonly used fraud practices: copy-move and splicing forgery in digital picture. To achieve this target, a special descriptor for each block was created combining the feature from JPEG block artificial grid with that from noise estimation. And forehand image quality assessment procedure reconciled these different features by setting proper weights. Experimental results showed that, compared to existing algorithms, our proposed method is effective on detecting both copy-move and splicing forgery regardless of JPEG compression ratio of the input image.

CTNet: Contrastive Transformer Network for Polyp Segmentation.

Segmenting polyps from colonoscopy images is very important in clinical practice since it provides valuable information for colorectal cancer. However, polyp segmentation remains a challenging task as polyps have camouflage properties and vary greatly in size. Although many polyp segmentation methods have been recently proposed and produced remarkable results, most of them cannot yield stable results due to the lack of features with distinguishing properties and those with high-level semantic details. Therefore, we proposed a novel polyp segmentation framework called contrastive Transformer network (CTNet), with three key components of contrastive Transformer backbone, self-multiscale interaction module (SMIM), and collection information module (CIM), which has excellent learning and generalization abilities. The long-range dependence and highly structured feature map space obtained by CTNet through contrastive Transformer can effectively localize polyps with camouflage properties. CTNet benefits from the multiscale information and high-resolution feature maps with high-level semantic obtained by SMIM and CIM, respectively, and thus can obtain accurate segmentation results for polyps of different sizes. Without bells and whistles, CTNet yields significant gains of 2.3%, 3.7%, 3.7%, 18.2%, and 10.1% over classical method PraNet on Kvasir-SEG, CVC-ClinicDB, Endoscene, ETIS-LaribPolypDB, and CVC-ColonDB respectively. In addition, CTNet has advantages in camouflaged object detection and defect detection. The code is available at https://github.com/Fhujinwu/CTNet.

Learning From Incorrectness: Active Learning With Negative Pre-Training and Curriculum Querying for Histological Tissue Classification.

Patch-level histological tissue classification is an effective pre-processing method for histological slide analysis. However, the classification of tissue with deep learning requires expensive annotation costs. To alleviate the limitations of annotation budgets, the application of active learning (AL) to histological tissue classification is a promising solution. Nevertheless, there is a large imbalance in performance between categories during application, and the tissue corresponding to the categories with relatively insufficient performance are equally important for cancer diagnosis. In this paper, we propose an active learning framework called ICAL, which contains Incorrectness Negative Pre-training (INP) and Category-wise Curriculum Querying (CCQ) to address the above problem from the perspective of category-to-category and from the perspective of categories themselves, respectively. In particular, INP incorporates the unique mechanism of active learning to treat the incorrect prediction results that obtained from CCQ as complementary labels for negative pre-training, in order to better distinguish similar categories during the training process. CCQ adjusts the query weights based on the learning status on each category by the model trained by INP, and utilizes uncertainty to evaluate and compensate for query bias caused by inadequate category performance. Experimental results on two histological tissue classification datasets demonstrate that ICAL achieves performance approaching that of fully supervised learning with less than 16% of the labeled data. In comparison to the state-of-the-art active learning algorithms, ICAL achieved better and more balanced performance in all categories and maintained robustness with extremely low annotation budgets. The source code will be released at https://github.com/LactorHwt/ICAL.

Quaternion Cross-Modality Spatial Learning for Multi-Modal Medical Image Segmentation.

Recently, the Deep Neural Networks (DNNs) have had a large impact on imaging process including medical image segmentation, and the real-valued convolution of DNN has been extensively utilized in multi-modal medical image segmentation to accurately segment lesions via learning data information. However, the weighted summation operation in such convolution limits the ability to maintain spatial dependence that is crucial for identifying different lesion distributions. In this paper, we propose a novel Quaternion Cross-modality Spatial Learning (Q-CSL) which explores the spatial information while considering the linkage between multi-modal images. Specifically, we introduce to quaternion to represent data and coordinates that contain spatial information. Additionally, we propose Quaternion Spatial-association Convolution to learn the spatial information. Subsequently, the proposed De-level Quaternion Cross-modality Fusion (De-QCF) module excavates inner space features and fuses cross-modality spatial dependency. Our experimental results demonstrate that our approach compared to the competitive methods perform well with only 0.01061 M parameters and 9.95G FLOPs.

A Parkinson's Auxiliary Diagnosis Algorithm Based on a Hyperparameter Optimization Method of Deep Learning.

Parkinson's disease is a common mental disease in the world, especially in the middle-aged and elderly groups. Today, clinical diagnosis is the main diagnostic method of Parkinson's disease, but the diagnosis results are not ideal, especially in the early stage of the disease. In this paper, a Parkinson's auxiliary diagnosis algorithm based on a hyperparameter optimization method of deep learning is proposed for the Parkinson's diagnosis. The diagnosis system uses ResNet50 to achieve feature extraction and Parkinson's classification, mainly including speech signal processing part, algorithm improvement part based on Artificial Bee Colony algorithm (ABC) and optimizing the hyperparameters of ResNet50 part. The improved algorithm is called Gbest Dimension Artificial Bee Colony algorithm (GDABC), proposing "Range pruning strategy" which aims at narrowing the scope of search and "Dimension adjustment strategy" which is to adjust gbest dimension by dimension. The accuracy of the diagnosis system in the verification set of Mobile Device Voice Recordings at King's College London (MDVR-CKL) dataset can reach more than 96%. Compared with current Parkinson's sound diagnosis methods and other optimization algorithms, our auxiliary diagnosis system shows better classification performance on the dataset within limited time and resources.

QGD-Net: A Lightweight Model Utilizing Pixels of Affinity in Feature Layer for Dermoscopic Lesion Segmentation.

RESPONSE: Pixels with location affinity, which can be also called "pixels of affinity," have similar semantic information. Group convolution and dilated convolution can utilize them to improve the capability of the model. However, for group convolution, it does not utilize pixels of affinity between layers. For dilated convolution, after multiple convolutions with the same dilated rate, the pixels utilized within each layer do not possess location affinity with each other. To solve the problem of group convolution, our proposed quaternion group convolution uses the quaternion convolution, which promotes the communication between to promote utilizing pixels of affinity between channels. In quaternion group convolution, the feature layers are divided into 4 layers per group, ensuring the quaternion convolution can be performed. To solve the problem of dilated convolution, we propose the quaternion sawtooth wave-like dilated convolutions module (QS module). QS module utilizes quaternion convolution with sawtooth wave-like dilated rates to effectively leverage the pixels that share the location affinity both between and within layers. This allows for an expanded receptive field, ultimately enhancing the performance of the model. In particular, we perform our quaternion group convolution in QS module to design the quaternion group dilated neutral network (QGD-Net). Extensive experiments on Dermoscopic Lesion Segmentation based on ISIC 2016 and ISIC 2017 indicate that our method has significantly reduced the model parameters and highly promoted the precision of the model in Dermoscopic Lesion Segmentation. And our method also shows generalizability in retinal vessel segmentation.

Generative AI for brain image computing and brain network computing: a review.

Recent years have witnessed a significant advancement in brain imaging techniques that offer a non-invasive approach to mapping the structure and function of the brain. Concurrently, generative artificial intelligence (AI) has experienced substantial growth, involving using existing data to create new content with a similar underlying pattern to real-world data. The integration of these two domains, generative AI in neuroimaging, presents a promising avenue for exploring various fields of brain imaging and brain network computing, particularly in the areas of extracting spatiotemporal brain features and reconstructing the topological connectivity of brain networks. Therefore, this study reviewed the advanced models, tasks, challenges, and prospects of brain imaging and brain network computing techniques and intends to provide a comprehensive picture of current generative AI techniques in brain imaging. This review is focused on novel methodological approaches and applications of related new methods. It discussed fundamental theories and algorithms of four classic generative models and provided a systematic survey and categorization of tasks, including co-registration, super-resolution, enhancement, classification, segmentation, cross-modality, brain network analysis, and brain decoding. This paper also highlighted the challenges and future directions of the latest work with the expectation that future research can be beneficial.

QMLS: quaternion mutual learning strategy for multi-modal brain tumor segmentation.

Objective.Due to non-invasive imaging and the multimodality of magnetic resonance imaging (MRI) images, MRI-based multi-modal brain tumor segmentation (MBTS) studies have attracted more and more attention in recent years. With the great success of convolutional neural networks in various computer vision tasks, lots of MBTS models have been proposed to address the technical challenges of MBTS. However, the problem of limited data collection usually exists in MBTS tasks, making existing studies typically have difficulty in fully exploring the multi-modal MRI images to mine complementary information among different modalities.Approach.We propose a novel quaternion mutual learning strategy (QMLS), which consists of a voxel-wise lesion knowledge mutual learning mechanism (VLKML mechanism) and a quaternion multi-modal feature learning module (QMFL module). Specifically, the VLKML mechanism allows the networks to converge to a robust minimum so that aggressive data augmentation techniques can be applied to expand the limited data fully. In particular, the quaternion-valued QMFL module treats different modalities as components of quaternions to sufficiently learn complementary information among different modalities on the hypercomplex domain while significantly reducing the number of parameters by about 75%.Main results.Extensive experiments on the dataset BraTS 2020 and BraTS 2019 indicate that QMLS achieves superior results to current popular methods with less computational cost.Significance.We propose a novel algorithm for brain tumor segmentation task that achieves better performance with fewer parameters, which helps the clinical application of automatic brain tumor segmentation.

Improved Normalized Cut for Multi-View Clustering.

Spectral clustering (SC) algorithms have been successful in discovering meaningful patterns since they can group arbitrarily shaped data structures. Traditional SC approaches typically consist of two sequential stages, i.e., performing spectral decomposition of an affinity matrix and then rounding the relaxed continuous clustering result into a binary indicator matrix. However, such a two-stage process could make the obtained binary indicator matrix severely deviate from the ground true one. This is because the former step is not devoted to achieving an optimal clustering result. To alleviate this issue, this paper presents a general joint framework to simultaneously learn the optimal continuous and binary indicator matrices for multi-view clustering, which also has the ability to tackle the conventional single-view case. Specially, we provide theoretical proof for the proposed method. Furthermore, an effective alternate updating algorithm is developed to optimize the corresponding complex objective. A number of empirical results on different benchmark datasets demonstrate that the proposed method outperforms several state-of-the-arts in terms of six clustering metrics.

An Efficient Artificial Bee Colony Algorithm With an Improved Linkage Identification Method.

The artificial colony (ABC) algorithm shows a relatively powerful exploration search capability but is constrained by the curse of dimensionality, especially on nonseparable functions, where its convergence speed slows dramatically. In this article, based on an analysis of the difference between updating mechanisms that include both all-variable and one-variable updating mechanisms, we find that when equipped with the former strategy, the algorithm rapidly converges to an optimal region, while with the latter strategy, it searches the solution space thoroughly. To utilize multivariable and one-variable updating mechanisms on nonseparable and separable functions, respectively, we embed an improved linkage identification strategy into the ABC by detecting the linkage between variables more effectively. Then, we propose three common strategies for ABC to improve its performance. First, a new approach that considers the historic experiences of the population is proposed to balance exploration and exploitation. Second, a new strategy for initializing scout bees is used to reduce the number of function evaluations. Finally, the individual with the worst performance is updated with a defined probability on multiple dimensions instead of one dimension, causing it to follow the population steps on nonseparable functions. This article is the first to propose all these concepts, which could be adopted for other ABC variants. The effectiveness of our algorithm is validated through basic, CEC2010, CEC2013, and CEC2014 functions and real-world problems.

Multi-scale contextual semantic enhancement network for 3D medical image segmentation.

Objective. Accurate and automatic segmentation of medical images is crucial for improving the efficiency of disease diagnosis and making treatment plans. Although methods based on convolutional neural networks have achieved excellent results in numerous segmentation tasks of medical images, they still suffer from challenges including drastic scale variations of lesions, blurred boundaries of lesions and class imbalance. Our objective is to design a segmentation framework named multi-scale contextual semantic enhancement network (3D MCSE-Net) to address the above problems.Approach. The 3D MCSE-Net mainly consists of a multi-scale context pyramid fusion module (MCPFM), a triple feature adaptive enhancement module (TFAEM), and an asymmetric class correction loss (ACCL) function. Specifically, the MCPFM resolves the problem of unreliable predictions due to variable morphology and drastic scale variations of lesions by capturing the multi-scale global context of feature maps. Subsequently, the TFAEM overcomes the problem of blurred boundaries of lesions caused by the infiltrating growth and complex context of lesions by adaptively recalibrating and enhancing the multi-dimensional feature representation of suspicious regions. Moreover, the ACCL alleviates class imbalances by adjusting asy mmetric correction coefficient and weighting factor.Main results. Our method is evaluated on the nasopharyngeal cancer tumor segmentation (NPCTS) dataset, the public dataset of the MICCAI 2017 liver tumor segmentation (LiTS) challenge and the 3D image reconstruction for comparison of algorithm and DataBase (3Dircadb) dataset to verify its effectiveness and generalizability. The experimental results show the proposed components all have unique strengths and exhibit mutually reinforcing properties. More importantly, the proposed 3D MCSE-Net outperforms previous state-of-the-art methods for tumor segmentation on the NPCTS, LiTS and 3Dircadb dataset.Significance. Our method addresses the effects of drastic scale variations of lesions, blurred boundaries of lesions and class imbalance, and improves tumors segmentation accuracy, which facilitates clinical medical diagnosis and treatment planning.

Collaborative multi-feature extraction and scale-aware semantic information mining for medical image segmentation.

Objective.In recent years, methods based on U-shaped structure and skip connection have achieved remarkable results in many medical semantic segmentation tasks. However, the information integration capability of this structure is still limited due to the incompatibility of feature maps of encoding and decoding stages at corresponding levels and lack of extraction of valid information in the final stage of encoding. This structural defect is particularly obvious in segmentation tasks with non-obvious, small and blurred-edge targets. Our objective is to design a novel segmentation network to solve the above problems.Approach.The segmentation network named Global Context-Aware Network is mainly designed by inserting a Multi-feature Collaboration Adaptation (MCA) module, a Scale-Aware Mining (SAM) module and an Edge-enhanced Pixel Intensity Mapping (Edge-PIM) into the U-shaped structure. Firstly, the MCA module can integrate information from all encoding stages and then effectively acts on the decoding stages, solving the problem of information loss during downsampling and pooling. Secondly, the SAM module can further mine information from the encoded high-level features to enrich the information passed to the decoding stage. Thirdly, Edge-PIM can further refine the segmentation results by edge enhancement.Main results.We newly collect Magnetic Resonance Imaging of Colorectal Cancer Liver Metastases (MRI-CRLM) dataset in different imaging sequences with non-obvious, small and blurred-edge liver metastases. Our method performs well on the MRI-CRLM dataset and the publicly available ISIC-2018 dataset, outperforming state-of-the-art methods such as CPFNet on multiple metrics after boxplot analysis, indicating that it can perform well on a wide range of medical image segmentation tasks.Significance.The proposed method solves the problem mentioned above and improved segmentation accuracy for non-obvious, small and blurred-edge targets. Meanwhile, the proposed visualization method Edge-PIM can make the edge more prominent, which can assist medical radiologists in their research work well.

Shift-channel attention and weighted-region loss function for liver and dense tumor segmentation.

PURPOSE: To assist physicians in the diagnosis and treatment planning of tumor, a robust and automatic liver and tumor segmentation method is highly demanded in the clinical practice. Recently, numerous researchers have improved the segmentation accuracy of liver and tumor by introducing multiscale contextual information and attention mechanism. However, this tends to introduce more training parameters and suffer from a heavier computational burden. In addition, the tumor has various sizes, shapes, locations, and numbers, which is the main reason for the poor accuracy of automatic segmentation. Although current loss functions can improve the learning ability of the model for hard samples to a certain extent, these loss functions are difficult to optimize the segmentation effect of small tumor regions when the large tumor regions in the sample are in the majority. METHODS: We propose a Liver and Tumor Segmentation Network (LiTS-Net) framework. First, the Shift-Channel Attention Module (S-CAM) is designed to model the feature interdependencies in adjacent channels and does not require additional training parameters. Second, the Weighted-Region (WR) loss function is proposed to emphasize the weight of small tumors in dense tumor regions and reduce the weight of easily segmented samples. Moreover, the Multiple 3D Inception Encoder Units (MEU) is adopted to capture the multiscale contextual information for better segmentation of liver and tumor. RESULTS: Efficacy of the LiTS-Net is demonstrated through the public dataset of MICCAI 2017 Liver Tumor Segmentation (LiTS) challenge, with Dice per case of 96.9 % ${\bf \%}$ and 75.1 % ${\bf \%}$ , respectively. For the 3D Image Reconstruction for Comparison of Algorithm and DataBase (3Dircadb), Dices are 96.47 % ${\bf \%}$ for the liver and 74.54 % ${\bf \%}$ for tumor segmentation. The proposed LiTS-Net outperforms existing state-of-the-art networks. CONCLUSIONS: We demonstrated the effectiveness of LiTS-Net and its core components for liver and tumor segmentation. The S-CAM is designed to model the feature interdependencies in the adjacent channels, which is characterized by no need to add additional training parameters. Meanwhile, we conduct an in-depth study of the feature shift proportion of adjacent channels to determine the optimal shift proportion. In addition, the WR loss function can implicitly learn the weights among regions without the need to manually specify the weights. In dense tumor segmentation tasks, WR aims to enhance the weights of small tumor regions and alleviate the problem that small tumor segmentation is difficult to optimize further when large tumor regions occupy the majority. Last but not least, our proposed method outperforms other state-of-the-art methods on both the LiTS dataset and the 3Dircadb dataset.

Kinship Verification Based on Cross-Generation Feature Interaction Learning.

Kinship verification from facial images has been recognized as an emerging yet challenging technique in many potential computer vision applications. In this paper, we propose a novel cross-generation feature interaction learning (CFIL) framework for robust kinship verification. Particularly, an effective collaborative weighting strategy is constructed to explore the characteristics of cross-generation relations by corporately extracting features of both parents and children image pairs. Specifically, we take parents and children as a whole to extract the expressive local and non-local features. Different from the traditional works measuring similarity by distance, we interpolate the similarity calculations as the interior auxiliary weights into the deep CNN architecture to learn the whole and natural features. These similarity weights not only involve corresponding single points but also excavate the multiple relationships cross points, where local and non-local features are calculated by using these two kinds of distance measurements. Importantly, instead of separately conducting similarity computation and feature extraction, we integrate similarity learning and feature extraction into one unified learning process. The integrated representations deduced from local and non-local features can comprehensively express the informative semantics embedded in images and preserve abundant correlation knowledge from image pairs. Extensive experiments demonstrate the efficiency and superiority of the proposed model compared to some state-of-the-art kinship verification methods.

Improving the Harmony of the Composite Image by Spatial-Separated Attention Module.

Image composition is one of the most important applications in image processing. However, the inharmonious appearance between the spliced region and background degrade the quality of the image. Thus, we address the problem of Image Harmonization: Given a spliced image and the mask of the spliced region, we try to harmonize the "style" of the pasted region with the background (non-spliced region). Previous approaches have been focusing on learning directly by the neural network. In this work, we start from an empirical observation: the differences can only be found in the spliced region between the spliced image and the harmonized result while they share the same semantic information and the appearance in the nonspliced region. Thus, in order to learn the feature map in the masked region and the others individually, we propose a novel attention module named Spatial-Separated Attention Module (S2AM). Furthermore, we design a novel image harmonization framework by inserting the S2AM in the coarser low-level features of the Unet structure by two different ways. Besides image harmonization, we make a big step for harmonizing the composite image without the specific mask under previous observation. The experiments show that the proposed S2AM performs better than other state-of-the-art attention modules in our task. Moreover, we demonstrate the advantages of our model against other state-of-the-art image harmonization methods via criteria from multiple points of view.

Depth-Assisted Full Resolution Network for Single Image-Based View Synthesis.

Research works in novel viewpoint synthesis are based mainly on multiview input images. In this paper, we focus on a more challenging and ill-posed problem that is to synthesize surrounding novel viewpoints from a single image. To achieve this goal, we design a full resolution network to extract fine-scale image features, which contributes to prevent blurry artifacts. We also involve a pretrained relative depth estimation network, thus three-dimensional information is utilized to infer the flow field between the input and the target image. Since the depth network is trained by depth order between any pair of objects, large-scale image features are also involved in our system. Finally, a synthesis layer is used to not only warp the observed pixels to the desired positions but also hallucinate the missing pixels from other recorded pixels. Experiments show that our technique successfully synthesizes reasonable novel viewpoints surrounding the input, while other state-of-the-art techniques fail.

Post-boosting of classification boundary for imbalanced data using geometric mean.

In this paper, a novel imbalance learning method for binary classes is proposed, named as Post-Boosting of classification boundary for Imbalanced data (PBI), which can significantly improve the performance of any trained neural networks (NN) classification boundary. The procedure of PBI simply consists of two steps: an (imbalanced) NN learning method is first applied to produce a classification boundary, which is then adjusted by PBI under the geometric mean (G-mean). For data imbalance, the geometric mean of the accuracies of both minority and majority classes is considered, that is statistically more suitable than the common metric accuracy. PBI also has the following advantages over traditional imbalance methods: (i) PBI can significantly improve the classification accuracy on minority class while improving or keeping that on majority class as well; (ii) PBI is suitable for large data even with high imbalance ratio (up to 0.001). For evaluation of (i), a new metric called Majority loss/Minority advance ratio (MMR) is proposed that evaluates the loss ratio of majority class to minority class. Experiments have been conducted for PBI and several imbalance learning methods over benchmark datasets of different sizes, different imbalance ratios, and different dimensionalities. By analyzing the experimental results, PBI is shown to outperform other imbalance learning methods on almost all datasets.

Cascade Chaotic System With Applications.

Chaotic maps are widely used in different applications. Motivated by the cascade structure in electronic circuits, this paper introduces a general chaotic framework called the cascade chaotic system (CCS). Using two 1-D chaotic maps as seed maps, CCS is able to generate a huge number of new chaotic maps. Examples and evaluations show the CCS's robustness. Compared with corresponding seed maps, newly generated chaotic maps are more unpredictable and have better chaotic performance, more parameters, and complex chaotic properties. To investigate applications of CCS, we introduce a pseudo-random number generator (PRNG) and a data encryption system using a chaotic map generated by CCS. Simulation and analysis demonstrate that the proposed PRNG has high quality of randomness and that the data encryption system is able to protect different types of data with a high-security level.

Extraction of shift invariant wavelet features for classification of images with different sizes.

An effective shift invariant wavelet feature extraction method for classification of images with different sizes is proposed. The feature extraction process involves a normalization followed by an adaptive shift invariant wavelet packet transform. An energy signature is computed for each subband of these invariant wavelet coefficients. A reduced subset of energy signatures is selected as the feature vector for classification of images with different sizes. Experimental results show that the proposed method can achieve high classification accuracy of 98.5 percent and outperforms the other two image classification methods.