Textual emotion classification using MPNet and cascading broad learning.

As one of the most important tasks of natural language processing, textual emotion classification (TEC) aims to recognize and detect all emotions contained in texts. However, most existing methods are implemented using deep learning approaches, which may suffer from long training time and low convergence. Motivated by these challenges, in this paper, we provide a new solution for TEC by using cascading broad learning (CBL) and sentence embedding using a masked and permuted pre-trained language model (MPNet), named CBLMP. Texts are input into MPNet to generate sentence embedding containing emotional semantic information. CBL is adopted to improve the ability of feature extraction in texts and to enhance model performance for general broad learning, by cascading feature nodes and cascading enhancement nodes, respectively. The L-curve model is adopted to ensure the balance between under-regularization and over-regularization for regularization parameter optimization. Extensive experiments have been carried out on datasets of SMP2020-EWECT and SemEval-2019 Task 3, and the results show that CBLMP outperforms the baseline methods in TEC.

A novel radial basis neural network for the Zika virus spreading model.

The motive of current investigations is to design a novel radial basis neural network stochastic structure to present the numerical representations of the Zika virus spreading model (ZVSM). The mathematical ZVSM is categorized into humans and vectors based on the susceptible S(q), exposed E(q), infected I(q) and recovered R(q), i.e., SEIR. The stochastic performances are designed using the radial basis activation function, feed forward neural network, twenty-two numbers of neurons along with the optimization of Bayesian regularization in order to solve the ZVSM. A dataset is achieved using the explicit Runge-Kutta scheme, which is used to reduce the mean square error (MSE) based on the process of training for solving the nonlinear ZVSM. The division of the data is categorized into training, which is taken as 78 %, while 11 % for both authentication and testing. Three different cases of the nonlinear ZVSM have been taken, while the scheme's correctness is performed through the matching of the results. Furthermore, the reliability of the scheme is observed by applying different performances of regression, MSE, error histograms and state transition.

Delayed blow-up and enhanced diffusion by transport noise for systems of reaction-diffusion equations.

This paper is concerned with the problem of regularization by noise of systems of reaction-diffusion equations with mass control. It is known that strong solutions to such systems of PDEs may blow-up in finite time. Moreover, for many systems of practical interest, establishing whether the blow-up occurs or not is an open question. Here we prove that a suitable multiplicative noise of transport type has a regularizing effect. More precisely, for both a sufficiently noise intensity and a high spectrum, the blow-up of strong solutions is delayed up to an arbitrary large time. Global existence is shown for the case of exponentially decreasing mass. The proofs combine and extend recent developments in regularization by noise and in the L p ( L q ) -approach to stochastic PDEs, highlighting new connections between the two areas.

Bayesian Adaptive Lasso for Detecting Item-Trait Relationship and Differential Item Functioning in Multidimensional Item Response Theory Models.

In multidimensional tests, the identification of latent traits measured by each item is crucial. In addition to item-trait relationship, differential item functioning (DIF) is routinely evaluated to ensure valid comparison among different groups. The two problems are investigated separately in the literature. This paper uses a unified framework for detecting item-trait relationship and DIF in multidimensional item response theory (MIRT) models. By incorporating DIF effects in MIRT models, these problems can be considered as variable selection for latent/observed variables and their interactions. A Bayesian adaptive Lasso procedure is developed for variable selection, in which item-trait relationship and DIF effects can be obtained simultaneously. Simulation studies show the performance of our method for parameter estimation, the recovery of item-trait relationship and the detection of DIF effects. An application is presented using data from the Eysenck Personality Questionnaire.

Uncertainty Quantification for Scale-Space Blob Detection.

We consider the problem of blob detection for uncertain images, such as images that have to be inferred from noisy measurements. Extending recent work motivated by astronomical applications, we propose an approach that represents the uncertainty in the position and size of a blob by a region in a three-dimensional scale space. Motivated by classic tube methods such as the taut-string algorithm, these regions are obtained from level sets of the minimizer of a total variation functional within a high-dimensional tube. The resulting non-smooth optimization problem is challenging to solve, and we compare various numerical approaches for its solution and relate them to the literature on constrained total variation denoising. Finally, the proposed methodology is illustrated on numerical experiments for deconvolution and models related to astrophysics, where it is demonstrated that it allows to represent the uncertainty in the detected blobs in a precise and physically interpretable way.

Convergence analysis of sparse TSK fuzzy systems based on spectral Dai-Yuan conjugate gradient and application to high-dimensional feature selection.

Dealing with high-dimensional problems has always been a key and challenging issue in the field of fuzzy systems. Traditional Takagi-Sugeno-Kang (TSK) fuzzy systems face the challenges of the curse of dimensionality and computational complexity when applied to high-dimensional data. To overcome these challenges, this paper proposes a novel approach for optimizing TSK fuzzy systems by integrating the spectral Dai-Yuan conjugate gradient (SDYCG) algorithm and the smoothing group L0 regularization technique. This method aims to address the challenges faced by TSK fuzzy systems in handling high-dimensional problems. The smoothing group L0 regularization technique is employed to introduce sparsity, select relevant features, and improve the generalization ability of the model. The SDYCG algorithm effectively accelerates convergence and enhances the learning performance of the network. Furthermore, we prove the weak convergence and strong convergence of the new algorithm under the strong Wolfe criterion, which means that the gradient norm of the error function with respect to the weight vector converges to zero, and the weight sequence approaches a fixed point.

Deep learning of multimodal networks with topological regularization for drug repositioning.

MOTIVATION: Computational techniques for drug-disease prediction are essential in enhancing drug discovery and repositioning. While many methods utilize multimodal networks from various biological databases, few integrate comprehensive multi-omics data, including transcriptomes, proteomes, and metabolomes. We introduce STRGNN, a novel graph deep learning approach that predicts drug-disease relationships using extensive multimodal networks comprising proteins, RNAs, metabolites, and compounds. We have constructed a detailed dataset incorporating multi-omics data and developed a learning algorithm with topological regularization. This algorithm selectively leverages informative modalities while filtering out redundancies. RESULTS: STRGNN demonstrates superior accuracy compared to existing methods and has identified several novel drug effects, corroborating existing literature. STRGNN emerges as a powerful tool for drug prediction and discovery. The source code for STRGNN, along with the dataset for performance evaluation, is available at https://github.com/yuto-ohnuki/STRGNN.git .

Translational symmetry in convolutions with localized kernels causes an implicit bias toward high frequency adversarial examples.

Adversarial attacks are still a significant challenge for neural networks. Recent efforts have shown that adversarial perturbations typically contain high-frequency features, but the root cause of this phenomenon remains unknown. Inspired by theoretical work on linear convolutional models, we hypothesize that translational symmetry in convolutional operations together with localized kernels implicitly bias the learning of high-frequency features, and that this is one of the main causes of high frequency adversarial examples. To test this hypothesis, we analyzed the impact of different choices of linear and non-linear architectures on the implicit bias of the learned features and adversarial perturbations, in spatial and frequency domains. We find that, independently of the training dataset, convolutional operations have higher frequency adversarial attacks compared to other architectural parameterizations, and that this phenomenon is exacerbated with stronger locality of the kernel (kernel size) end depth of the model. The explanation for the kernel size dependence involves the Fourier Uncertainty Principle: a spatially-limited filter (local kernel in the space domain) cannot also be frequency-limited (local in the frequency domain). Using larger convolution kernel sizes or avoiding convolutions (e.g., by using Vision Transformers or MLP-style architectures) significantly reduces this high-frequency bias. Looking forward, our work strongly suggests that understanding and controlling the implicit bias of architectures will be essential for achieving adversarial robustness.

SABO-ILSTSVR: a genomic prediction method based on improved least squares twin support vector regression.

In modern breeding practices, genomic prediction (GP) uses high-density single nucleotide polymorphisms (SNPs) markers to predict genomic estimated breeding values (GEBVs) for crucial phenotypes, thereby speeding up selection breeding process and shortening generation intervals. However, due to the characteristic of genotype data typically having far fewer sample numbers than SNPs markers, overfitting commonly arise during model training. To address this, the present study builds upon the Least Squares Twin Support Vector Regression (LSTSVR) model by incorporating a Lasso regularization term named ILSTSVR. Because of the complexity of parameter tuning for different datasets, subtraction average based optimizer (SABO) is further introduced to optimize ILSTSVR, and then obtain the GP model named SABO-ILSTSVR. Experiments conducted on four different crop datasets demonstrate that SABO-ILSTSVR outperforms or is equivalent in efficiency to widely-used genomic prediction methods. Source codes and data are available at: https://github.com/MLBreeding/SABO-ILSTSVR.

Cube is a good form: Hyperspectral band selection via multi-dimensional and high-order structure preserved clustering.

As an effective strategy for reducing the noisy and redundant information for hyperspectral imagery (HSI), hyperspectral band selection intends to select a subset of original hyperspectral bands, which boosts the subsequent different tasks. In this paper, we introduce a multi-dimensional high-order structure preserved clustering method for hyperspectral band selection, referred to as MHSPC briefly. By regarding original hyperspectral images as a tensor cube, we apply the tensor CP (CANDECOMP/PARAFAC) decomposition on it to exploit the multi-dimensional structural information as well as generate a low-dimensional latent feature representation. In order to capture the local geometrical structure along the spectral dimension, a graph regularizer is imposed on the new feature representation in the lower dimensional space. In addition, since the low rankness of HSIs is an important global property, we utilize a nuclear norm constraint on the latent feature representation matrix to capture the global data structure information. Different to most of previous clustering based hyperspectral band selection methods which vectorize each band as a vector without considering the 2-D spatial information, the proposed MHSPC can effectively capture the spatial structure as well as the spectral correlation of original hyperspectral cube in both local and global perspectives. An efficient alternatively updating algorithm with theoretical convergence guarantee is designed to solve the resultant optimization problem, and extensive experimental results on four benchmark datasets validate the effectiveness of the proposed MHSPC over other state-of-the-arts.

Self-supervised tomographic image noise suppression via residual image prior network.

Computed tomography (CT) denoising is a challenging task in medical imaging that has garnered considerable attention. Supervised networks require a lot of noisy-clean image pairs, which are always unavailable in clinical settings. Existing self-supervised algorithms for suppressing noise with paired noisy images have limitations, such as ignoring the residual between similar image pairs during training and insufficiently learning the spectrum information of images. In this study, we propose a Residual Image Prior Network (RIP-Net) to sufficiently model the residual between the paired similar noisy images. Our approach offers new insights into the field by addressing the limitations of existing methods. We first establish a mathematical theorem clarifying the non-equivalence between similar-image-based self-supervised learning and supervised learning. It helps us better understand the strengths and limitations of self-supervised learning. Secondly, we introduce a novel regularization term to model a low-frequency residual image prior. This can improve the accuracy and robustness of our model. Finally, we design a well-structured denoising network capable of exploring spectrum information while simultaneously sensing context messages. The network has dual paths for modeling high and low-frequency compositions in the raw noisy image. Additionally, context perception modules capture local and global interactions to produce high-quality images. The comprehensive experiments on preclinical photon-counting CT, clinical brain CT, and low-dose CT datasets, demonstrate that our RIP-Net is superior to other unsupervised denoising methods.

Compressed-sensing-inspired reconstruction algorithms in low-dose computed tomography: A review.

BACKGROUND: Optimization of the dose the patient receives during scanning is an important problem in modern medical X-ray computed tomography (CT). One of the basic ways to its solution is to reduce the number of views. Compressed sensing theory helped promote the development of a new class of effective reconstruction algorithms for limited data CT. These compressed-sensing-inspired (CSI) algorithms optimize the Lp (0 ≤ p ≤ 1) norm of images and can accurately reconstruct CT tomograms from a very few views. The paper presents a review of the CSI algorithms and discusses prospects for their further use in commercial low-dose CT. METHODS: Many literature references with the CSI algorithms have been were searched. To structure the material collected the author gives a classification framework within which he describes Lp regularization methods, the basic CSI algorithms that are used most often in few-view CT, and some of their derivatives. Lots of examples are provided to illustrate the use of the CSI algorithms in few-view and low-dose CT. RESULTS: A list of the CSI algorithms is compiled from the literature search. For better demonstrativeness they are summarized in a table. The inference is done that already today some of the algorithms are capable of reconstruction from 20 to 30 views with acceptable quality and dose reduction by a factor of 10. DISCUSSION: In conclusion the author discusses how soon the CSI reconstruction algorithms can be introduced in the practice of medical diagnosis and used in commercial CT scanners.

Hierarchical multi-level dynamic hyperparameter deformable image registration with convolutional neural network.

Objective. To enable the registration network to be trained only once, achieving fast regularization hyperparameter selection during the inference phase, and to improve registration accuracy and deformation field regularity.Approach. Hyperparameter tuning is an essential process for deep learning deformable image registration (DLDIR). Most DLDIR methods usually perform a large number of independent experiments to select the appropriate regularization hyperparameters, which are time-consuming and resource-consuming. To address this issue, we propose a novel dynamic hyperparameter block, which comprises a distributed mapping network, dynamic convolution, attention feature extraction layer, and instance normalization layer. The dynamic hyperparameter block encodes the input feature vectors and regularization hyperparameters into learnable feature variables and dynamic convolution parameters which changes the feature statistics of the high-dimensional features layer feature variables, respectively. In addition, the proposed method replaced the single-level structure residual blocks in LapIRN with a hierarchical multi-level architecture for the dynamic hyperparameter block in order to improve registration performance.Main results. On the OASIS dataset, the proposed method reduced the percentage of|Jϕ|⩽0by 28.01%, 9.78%and improved Dice similarity coefficient by 1.17%, 1.17%, compared with LapIRN and CIR, respectively. On the DIR-Lab dataset, the proposed method reduced the percentage of|Jϕ|⩽0by 10.00%, 5.70%and reduced target registration error by 10.84%, 10.05%, compared with LapIRN and CIR, respectively.Significance. The proposed method can fast achieve the corresponding registration deformation field for arbitrary hyperparameter value during the inference phase. Extensive experiments demonstrate that the proposed method reduces training time compared to DLDIR with fixed regularization hyperparameters while outperforming the state-of-the-art registration methods concerning registration accuracy and deformation smoothness on brain dataset OASIS and lung dataset DIR-Lab.

CPSS: Fusing consistency regularization and pseudo-labeling techniques for semi-supervised deep cardiovascular disease detection using all unlabeled electrocardiograms.

BACKGROUND AND OBJECTIVE: Deep learning usually achieves good performance in the supervised way, which requires a large amount of labeled data. However, manual labeling of electrocardiograms (ECGs) is laborious that requires much medical knowledge. Semi-supervised learning (SSL) provides an effective way of leveraging unlabeled data to improve model performance, providing insight for solving this problem. The objective of this study is to improve the performance of cardiovascular disease (CVD) detection by fully utilizing unlabeled ECG. METHODS: A novel SSL algorithm fusing consistency regularization and pseudo-labeling techniques (CPSS) is proposed. CPSS consists of supervised learning and unsupervised learning. For supervised learning, the labeled ECGs are mapped into prediction vectors by the classifier. The cross-entropy loss function is used to optimize the classifier. For unsupervised learning, the unlabeled ECGs are weakly and strongly augmented, and a consistency loss is used to minimize the difference between the classifier's predictions for the two augmentations. Pseudo-labeling techniques include positive pseudo-labeling (PL) and ranking-based negative pseudo-labeling (RNL). PL introduces pseudo-labels for data with high prediction confidence. RNL assigns negative pseudo-labels to the lower-ranked categories in the prediction vectors to leverage data with low prediction confidence. In this study, VGGNet and ResNet are used as classifiers, which are jointly optimized by labeled and unlabeled ECGs. RESULTS: CPSS has been validated on several databases. With the same number of labeled ECGs (10%), it improves the accuracies over pure supervised learning by 13.59%, 4.60%, and 5.38% in the CPSC2018, PTB-XL, and Chapman databases, respectively. CPSS achieves comparable results to the fully supervised method with only 10% of labeled ECGs, which reduces the labeling workload by 90%. In addition, to verify the practicality of CPSS, a cardiovascular disease monitoring system is designed by heterogeneously deploying the trained classifiers on an SoC (system-on-a-chip), which can detect CVD in real time. CONCLUSION: The results of this study indicate that the proposed CPSS can significantly improve the performance of CVD detection using unlabeled ECG, which reduces the burden of ECG labeling in deep learning. In addition, the designed monitoring system makes the proposed CPSS promising for real-world applications.

FetalBrainAwareNet: Bridging GANs with anatomical insight for fetal ultrasound brain plane synthesis.

Over the past decade, deep-learning (DL) algorithms have become a promising tool to aid clinicians in identifying fetal head standard planes (FHSPs) during ultrasound (US) examination. However, the adoption of these algorithms in clinical settings is still hindered by the lack of large annotated datasets. To overcome this barrier, we introduce FetalBrainAwareNet, an innovative framework designed to synthesize anatomically accurate images of FHSPs. FetalBrainAwareNet introduces a cutting-edge approach that utilizes class activation maps as a prior in its conditional adversarial training process. This approach fosters the presence of the specific anatomical landmarks in the synthesized images. Additionally, we investigate specialized regularization terms within the adversarial training loss function to control the morphology of the fetal skull and foster the differentiation between the standard planes, ensuring that the synthetic images faithfully represent real US scans in both structure and overall appearance. The versatility of our FetalBrainAwareNet framework is highlighted by its ability to generate high-quality images of three predominant FHSPs using a singular, integrated framework. Quantitative (Fréchet inception distance of 88.52) and qualitative (t-SNE) results suggest that our framework generates US images with greater variability compared to state-of-the-art methods. By using the synthetic images generated with our framework, we increase the accuracy of FHSP classifiers by 3.2% compared to training the same classifiers solely with real acquisitions. These achievements suggest that using our synthetic images to increase the training set could provide benefits to enhance the performance of DL algorithms for FHSPs classification that could be integrated in real clinical scenarios.

Flexor Tendon Continuity and Negative X-ray: The "Combo" Negative Features in Finger Subamputation.

Background: The subamputation of fingers with vascular compromise presents a surgical challenge. Although tissue continuity may be considered a favourable prognostic element, in our experience, we noticed that there is not always a direct correlation between soft tissue involvement, radiographic appearance and final outcome. Methods: We included, in our study, all cases of vascular pedicle injury in which finger salvage was attempted with microsurgical revascularisation. Exclusion criteria were: integrity of both vascular pedicles, pedicle lesion without global circulatory compromise and patients treated immediately with amputation. Results: Between May 2018 and July 2023, 27 male patients with finger subamputation injuries were treated at our institution. In 11 cases of injured fingers, the only intact tissue was the flexor digitorum profundus (FDP) or flexor pollicis longus (FPL). Our global failure rate was 49%; whereas, in the subgroup of the 11 cases with continuity of the FDP or FPL, the failure rate rose to 73% and when the fingers showed flexor tendon integrity and radiographs demonstrated minimal bone damage, revascularisation failure was observed in all cases (100%). Conclusions: The results of the study show that subamputations with devascularisation, clinically presented with the combination of flexor tendon as the only element of tissue continuity and dislocation or minimal bone/articular injury, have a worse prognosis because of their trauma mechanism. We propose to add them to the Kay-Adani Classification as a subset of the poorest prognostic injuries group (III), to help surgeons to make decisions about the management of subamputation finger injuries.

Deep Transfer Learning Method Using Self-Pixel and Global Channel Attentive Regularization.

The purpose of this paper is to propose a novel transfer learning regularization method based on knowledge distillation. Recently, transfer learning methods have been used in various fields. However, problems such as knowledge loss still occur during the process of transfer learning to a new target dataset. To solve these problems, there are various regularization methods based on knowledge distillation techniques. In this paper, we propose a transfer learning regularization method based on feature map alignment used in the field of knowledge distillation. The proposed method is composed of two attention-based submodules: self-pixel attention (SPA) and global channel attention (GCA). The self-pixel attention submodule utilizes both the feature maps of the source and target models, so that it provides an opportunity to jointly consider the features of the target and the knowledge of the source. The global channel attention submodule determines the importance of channels through all layers, unlike the existing methods that calculate these only within a single layer. Accordingly, transfer learning regularization is performed by considering both the interior of each single layer and the depth of the entire layer. Consequently, the proposed method using both of these submodules showed overall improved classification accuracy than the existing methods in classification experiments on commonly used datasets.

Sparse-view X-ray CT based on a box-constrained nonlinear weighted anisotropic TV regularization.

Sparse-view computed tomography (CT) is an important way to reduce the negative effect of radiation exposure in medical imaging by skipping some X-ray projections. However, due to violating the Nyquist/Shannon sampling criterion, there are severe streaking artifacts in the reconstructed CT images that could mislead diagnosis. Noting the ill-posedness nature of the corresponding inverse problem in a sparse-view CT, minimizing an energy functional composed by an image fidelity term together with properly chosen regularization terms is widely used to reconstruct a medical meaningful attenuation image. In this paper, we propose a regularization, called the box-constrained nonlinear weighted anisotropic total variation (box-constrained NWATV), and minimize the regularization term accompanying the least square fitting using an alternative direction method of multipliers (ADMM) type method. The proposed method is validated through the Shepp-Logan phantom model, alongisde the actual walnut X-ray projections provided by Finnish Inverse Problems Society and the human lung images. The experimental results show that the reconstruction speed of the proposed method is significantly accelerated compared to the existing $ L_1/L_2 $ regularization method. Precisely, the central processing unit (CPU) time is reduced more than 8 times.

Tangent space functional reconfigurations in individuals at risk for alcohol use disorder.

Human brain function dynamically adjusts to ever-changing stimuli from the external environment. Studies characterizing brain functional reconfiguration are nevertheless scarce. Here we present a principled mathematical framework to quantify brain functional reconfiguration when engaging and disengaging from a stop signal task (SST). We apply tangent space projection (a Riemannian geometry mapping technique) to transform functional connectomes (FCs) and quantify functional reconfiguration using the correlation distance of the resulting tangent-FCs. Our goal was to compare functional reconfigurations in individuals at risk for alcohol use disorder (AUD). We hypothesized that functional reconfigurations when transitioning in/from a task would be influenced by family history of alcohol use disorder (FHA) and other AUD risk factors. Multilinear regression model results showed that engaging and disengaging functional reconfiguration were driven by different AUD risk factors. Functional reconfiguration when engaging in the SST was negatively associated with recent drinking. When disengaging from the SST, however, functional reconfiguration was negatively associated with FHA. In both models, several other factors contributed to the explanation of functional reconfiguration. This study demonstrates that tangent-FCs can characterize task-induced functional reconfiguration, and that it is related to AUD risk.

A few theoretical results for Laplace and arctan penalized ordinary least squares linear regression estimators.

Two new nonconvex penalty functions - Laplace and arctan - were recently introduced in the literature to obtain sparse models for high-dimensional statistical problems. In this paper, we study the theoretical properties of Laplace and arctan penalized ordinary least squares linear regression models. We first illustrate the near-unbiasedness of the nonzero regression weights obtained by the new penalty functions, in the orthonormal design case. In the general design case, we present theoretical results in two asymptotic settings: (a) the number of features, p fixed, but the sample size, n → ∞ , and (b) both n and p tend to infinity. The theoretical results shed light onto the differences between the solutions based on the new penalty functions and those based on existing convex and nonconvex Bridge penalty functions. Our theory also shows that both Laplace and arctan penalties satisfy the oracle property. Finally, we also present results from a brief simulations study illustrating the performance of Laplace and arctan penalties based on the gradient descent optimization algorithm.