Application of a hybrid algorithm of LSTM and Transformer based on random search optimization for improving rainfall-runoff simulation.

Flood forecasting using traditional physical hydrology models requires consideration of multiple complex physical processes including the spatio-temporal distribution of rainfall, the spatial heterogeneity of watershed sub-surface characteristics, and runoff generation and routing behaviours. Data-driven models offer novel solutions to these challenges, though they are hindered by difficulties in hyperparameter selection and a decline in prediction stability as the lead time extends. This study introduces a hybrid model, the RS-LSTM-Transformer, which combines Random Search (RS), Long Short-Term Memory networks (LSTM), and the Transformer architecture. Applied to the typical Jingle watershed in the middle reaches of the Yellow River, this model utilises rainfall and runoff data from basin sites to simulate flood processes, and its outcomes are compared against those from RS-LSTM, RS-Transformer, RS-BP, and RS-MLP models. It was evaluated against RS-LSTM, RS-Transformer, RS-BP, and RS-MLP models using the Nash-Sutcliffe Efficiency Coefficient (NSE), Root Mean Square Error (RMSE), Mean Absolute Error (MAE), and Bias percentage as metrics. At a 1-h lead time during calibration and validation, the RS-LSTM-Transformer model achieved NSE, RMSE, MAE, and Bias values of 0.970, 14.001m3/s, 5.304m3/s, 0.501% and 0.953, 14.124m3/s, 6.365m3/s, 0.523%, respectively. These results demonstrate the model's superior simulation capabilities and robustness, providing more accurate peak flow forecasts as the lead time increases. The study highlights the RS-LSTM-Transformer model's potential in flood forecasting and the advantages of integrating various data-driven approaches for innovative modelling.

Automatic fitting of multiple-field solid-state NMR spectra.

The NMR lineshapes produced by half-integer quadrupolar nuclei are sensitive to 11 distinct fit parameters per inequivalent site. To date, automatic fitting routines have failed to replace manual parameter insertion and evaluation due to the importance of local minima and the need for fitting multiple-field magic-angle spinning (MAS) and static spectra simultaneously. Herein we introduce a new tool, AMES-Fit (Automatic Multiple Experiment Simulation and Fitting), to automatically find the global best-fit simulation parameters for a series of multiple-field NMR lineshapes. AMES-Fit uses an adaptive step size random search algorithm to dynamically probe parameter space and requires minimal human input. The best fits are obtained in a few minutes of computation time that would otherwise have required several person-hours of work. The program is freely available and open-source.

Role of Optimization in RNA-Protein-Binding Prediction.

RNA-binding proteins (RBPs) play an important role in regulating biological processes, such as gene regulation. Understanding their behaviors, for example, their binding site, can be helpful in understanding RBP-related diseases. Studies have focused on predicting RNA binding by means of machine learning algorithms including deep convolutional neural network models. One of the integral parts of modeling deep learning is achieving optimal hyperparameter tuning and minimizing a loss function using optimization algorithms. In this paper, we investigate the role of optimization in the RBP classification problem using the CLIP-Seq 21 dataset. Three optimization methods are employed on the RNA-protein binding CNN prediction model; namely, grid search, random search, and Bayesian optimizer. The empirical results show an AUC of 94.42%, 93.78%, 93.23% and 92.68% on the ELAVL1C, ELAVL1B, ELAVL1A, and HNRNPC datasets, respectively, and a mean AUC of 85.30 on 24 datasets. This paper's findings provide evidence on the role of optimizers in improving the performance of RNA-protein binding prediction.

Generalization of navigation memory in honeybees.

Flying insects like the honeybee learn multiple features of the environment for efficient navigation. Here we introduce a novel paradigm in the natural habitat, and ask whether the memory of such features is generalized to novel test conditions. Foraging bees from colonies located in 5 different home areas were tested in a common area for their search flights. The home areas differed in the arrangements of rising natural objects or their lack, and in the existence or lack of elongated ground structures. The test area resembled partly or not at all the layout of landmarks in the respective home areas. In particular, the test area lacked rising objects. The search flights were tracked with harmonic radar and quantified by multiples procedures, extracting their differences on an individual basis. Random search as the only guide for searching was excluded by two model calculations. The frequencies of directions of flight sectors differed from both model calculations and between the home areas in a graded fashion. Densities of search flight fixes were used to create heat maps and classified by a partial least squares regression analysis. Classification was performed with a support vector machine in order to account for optimal hyperplanes. A rank order of well separated clusters was found that partly resemble the graded differences between the ground structures of the home areas and the test area. The guiding effect of elongated ground structures was quantified with respect to the sequence, angle and distance from these ground structures. We conclude that foragers generalize their specific landscape memory in a graded way to the landscape features in the test area, and argue that both the existence and absences of landmarks are taken into account. The conclusion is discussed in the context of the learning and generalization process in an insect, the honeybee, with an emphasis on exploratory learning in the context of navigation.

On a Population Sizing Model for Evolution Strategies Optimizing the Highly Multimodal Rastrigin Function.

A model is presented that allows for the calculation of the success probability by which a vanilla Evolution Strategy converges to the global optimizer of the Rastrigin test function. As a result a population size scaling formula will be derived that allows for an estimation of the population size needed to ensure a high convergence security depending on the search space dimensionality.

Erratum: OPRA-RS: A hearing-aid fitting method based on automatic speech recognition and random search.

[This corrects the article DOI: 10.3389/fnins.2022.779048.].

Time Series Forecasting of Motor Bearing Vibration Based on Informer.

Electric energy, as an economical and clean energy, plays a significant role in the development of science and technology and the economy. The motor is the core equipment of the power station; therefore, monitoring the motor vibration and predicting time series of the bearing vibration can effectively avoid hazards such as bearing heating and reduce energy consumption. Time series forecasting methods of motor bearing vibration based on sliding window forecasting, such as CNN, LSTM, etc., have the problem of error accumulation, and the longer the time-series forecasting, the larger the error. In order to solve the problem of error accumulation caused by the conventional methods of time series forecasting of motor bearing vibration, this paper innovatively introduces Informer into time series forecasting of motor bearing vibration. Based on Transformer, Informer introduces ProbSparse self-attention and self-attention distilling, and applies random search to optimize the model parameters to reduce the error accumulation in forecasting, achieve the optimization of time and space complexity and improve the model forecasting. Comparing the forecasting results of Informer and those of other forecasting models in three publicly available datasets, it is verified that Informer has excellent performance in time series forecasting of motor bearing vibration and the forecasting results reach 10-2∼10-6.

An Embodied Intelligence-Based Biologically Inspired Strategy for Searching a Moving Target.

Bacterial chemotaxis in unicellular Escherichia coli, the simplest biological creature, enables it to perform effective searching behaviour even with a single sensor, achieved via a sequence of "tumbling" and "swimming" behaviours guided by gradient information. Recent studies show that suitable random walk strategies may guide the behaviour in the absence of gradient information. This article presents a novel and minimalistic biologically inspired search strategy inspired by bacterial chemotaxis and embodied intelligence concept: a concept stating that intelligent behaviour is a result of the interaction among the "brain," body morphology including the sensory sensitivity tuned by the morphology, and the environment. Specifically, we present bacterial chemotaxis inspired searching behaviour with and without gradient information based on biological fluctuation framework: a mathematical framework that explains how biological creatures utilize noises in their behaviour. Via extensive simulation of a single sensor mobile robot that searches for a moving target, we will demonstrate how the effectiveness of the search depends on the sensory sensitivity and the inherent random walk strategies produced by the brain of the robot, comprising Ballistic, Levy, Brownian, and Stationary search. The result demonstrates the importance of embodied intelligence even in a behaviour inspired by the simplest creature.

Using Automatic Speech Recognition to Optimize Hearing-Aid Time Constants.

Automatic speech recognition (ASR), when combined with hearing-aid (HA) and hearing-loss (HL) simulations, can predict aided speech-identification performances of persons with age-related hearing loss. ASR can thus be used to evaluate different HA configurations, such as combinations of insertion-gain functions and compression thresholds, in order to optimize HA fitting for a given person. The present study investigated whether, after fixing compression thresholds and insertion gains, a random-search algorithm could be used to optimize time constants (i.e., attack and release times) for 12 audiometric profiles. The insertion gains were either those recommended by the CAM2 prescription rule or those optimized using ASR, while compression thresholds were always optimized using ASR. For each audiometric profile, the random-search algorithm was used to vary time constants with the aim to maximize ASR performance. A HA simulator and a HL simulator simulator were used, respectively, to amplify and to degrade speech stimuli according to the input audiogram. The resulting speech signals were fed to an ASR system for recognition. For each audiogram, 1,000 iterations of the random-search algorithm were used to find the time-constant configuration yielding the highest ASR score. To assess the reproducibility of the results, the random search algorithm was run twice. Optimizing the time constants significantly improved the ASR scores when CAM2 insertion gains were used, but not when using ASR-based gains. Repeating the random search yielded similar ASR scores, but different time-constant configurations.

OPRA-RS: A Hearing-Aid Fitting Method Based on Automatic Speech Recognition and Random Search.

Hearing-aid (HA) prescription rules (such as NAL-NL2, DSL-v5, and CAM2) are used by HA audiologists to define initial HA settings (e.g., insertion gains, IGs) for patients. This initial fitting is later individually adjusted for each patient to improve clinical outcomes in terms of speech intelligibility and listening comfort. During this fine-tuning stage, speech-intelligibility tests are often carried out with the patient to assess the benefits associated with different HA settings. As these tests tend to be time-consuming and performance on them depends on the patient's level of fatigue and familiarity with the test material, only a limited number of HA settings can be explored. Consequently, it is likely that a suboptimal fitting is used for the patient. Recent studies have shown that automatic speech recognition (ASR) can be used to predict the effects of IGs on speech intelligibility for patients with age-related hearing loss (ARHL). The aim of the present study was to extend this approach by optimizing, in addition to IGs, compression thresholds (CTs). However, increasing the number of parameters to be fitted increases exponentially the number of configurations to be assessed. To limit the number of HA settings to be tested, three random-search (RS) genetic algorithms were used. The resulting new HA fitting method, combining ASR and RS, is referred to as "objective prescription rule based on ASR and random search" (OPRA-RS). Optimal HA settings were computed for 12 audiograms, representing average and individual audiometric profiles typical for various levels of ARHL severity, and associated ASR performances were compared to those obtained with the settings recommended by CAM2. Each RS algorithm was run twice to assess its reliability. For all RS algorithms, ASR scores obtained with OPRA-RS were significantly higher than those associated with CAM2. Each RS algorithm converged on similar optimal HA settings across repetitions. However, significant differences were observed between RS algorithms in terms of maximum ASR performance and processing costs. These promising results open the way to the use of ASR and RS algorithms for the fine-tuning of HAs with potential speech-intelligibility benefits for the patient.

Parsimonious Optimization of Multitask Neural Network Hyperparameters.

Neural networks are rapidly gaining popularity in chemical modeling and Quantitative Structure-Activity Relationship (QSAR) thanks to their ability to handle multitask problems. However, outcomes of neural networks depend on the tuning of several hyperparameters, whose small variations can often strongly affect their performance. Hence, optimization is a fundamental step in training neural networks although, in many cases, it can be very expensive from a computational point of view. In this study, we compared four of the most widely used approaches for tuning hyperparameters, namely, grid search, random search, tree-structured Parzen estimator, and genetic algorithms on three multitask QSAR datasets. We mainly focused on parsimonious optimization and thus not only on the performance of neural networks, but also the computational time that was taken into account. Furthermore, since the optimization approaches do not directly provide information about the influence of hyperparameters, we applied experimental design strategies to determine their effects on the neural network performance. We found that genetic algorithms, tree-structured Parzen estimator, and random search require on average 0.08% of the hours required by grid search; in addition, tree-structured Parzen estimator and genetic algorithms provide better results than random search.

Improving the field homogeneity of fixed- and variable-diameter discrete Halbach magnet arrays for MRI via optimization of the angular magnetization distribution.

The aim of this work was to maximize the homogeneity of fixed- or variable-diameter Halbach array of discrete magnets by optimizing the angular rotation of individual magnets within each ring of the array. Numerical simulations have been performed for magnet arrays with various length:radius ratios (L/R) using a dipole-approximation model. These simulations used an uninformed random-search algorithm, with the initial state corresponding to the classical Halbach dipole configuration. Two different classes of systems were studied, one with magnet rings of constant radius, and the other in which the radius of the rings was allowed to vary to increase the homogeneity. Simulation results showed that for a fixed-diameter array optimization of the angular orientation of individual magnets increased the homogeneity by ~17% for very short magnets, with the improvement dropping to ~5% for L/R values greater than ~3:1, where the homogeneity was measured over a region-of-interest equal to one-half the diameter of the magnet array. An empirical formula was derived which allows easy estimation of the required magnetization angles for any L/R. For a 23-ring variable diameter magnet with L/R of ~4:1 the optimization procedure produces an increase in homogeneity of ~18%.

Random Search as a Neural Network Optimization Strategy for Convolutional-Neural-Network (CNN)-based Noise Reduction in CT.

In this study, we describe a systematic approach to optimize deep-learning-based image processing algorithms using random search. The optimization technique is demonstrated on a phantom-based noise reduction training framework; however, the techniques described can be applied generally for other deep learning image processing applications. The parameter space explored included number of convolutional layers, number of filters, kernel size, loss function, and network architecture (either U-Net or ResNet). A total of 100 network models were examined (50 random search, 50 ablation experiments). Following the random search, ablation experiments resulted in a very minor performance improvement indicating near optimal settings were found during the random search. The top performing network architecture was a U-Net with 4 pooling layers, 64 filters, 3×3 kernel size, ELU activation, and a weighted feature reconstruction loss (0.2×VGG + 0.8×MSE). Relative to the low-dose input image, the CNN reduced noise by 90%, reduced RMSE by 34%, and increased SSIM by 76% on six patient exams reserved for testing. The visualization of hepatic and bone lesions was greatly improved following noise reduction.

2DOF multi-objective optimal tuning of disturbance reject fractional order PIDA controllers according to improved consensus oriented random search method.

This study presents a Fractional Order Proportional Integral Derivative Acceleration (FOPIDA) controller design methodology to improve set point and disturbance reject control performance. The proposed controller tuning method performs a multi-objective optimal fine-tuning strategy that implements a Consensus Oriented Random Search (CORS) algorithm to evaluate transient simulation results of a set point filter type Two Degree of Freedom (2DOF) FOPIDA control system. Contributions of this study have three folds: Firstly, it addresses tuning problem of FOPIDA controllers for first order time delay systems. Secondly, the study aims fine-tuning of 2DOF FOPIDA control structure for improved set point and disturbance rejection control according to transient simulations of implementation models. This enhances practical performance of theoretical tuning method according to implementation requirements. Thirdly, the paper presents a hybrid controller tuning methodology that increases effectiveness of the CORS algorithm by using stabilizing controller coefficients as an initial configuration. Accordingly, the CORS algorithm performs the fine-tuning of 2DOF FOPIDA controllers to achieve an improved set point and disturbance rejection control performances. This fine-tuning is carried out by considering transient simulation results of 2DOF FOPIDA controller implementation model. Moreover, Reference to Disturbance Ratio (RDR) formulation of the FOPIDA controller is derived and used for measurement of disturbance rejection control performance. Illustrative design examples are presented to demonstrate effectiveness of the proposed method.

Functional advantages of Lévy walks emerging near a critical point.

A special class of random walks, so-called Lévy walks, has been observed in a variety of organisms ranging from cells, insects, fishes, and birds to mammals, including humans. Although their prevalence is considered to be a consequence of natural selection for higher search efficiency, some findings suggest that Lévy walks might also be epiphenomena that arise from interactions with the environment. Therefore, why they are common in biological movements remains an open question. Based on some evidence that Lévy walks are spontaneously generated in the brain and the fact that power-law distributions in Lévy walks can emerge at a critical point, we hypothesized that the advantages of Lévy walks might be enhanced by criticality. However, the functional advantages of Lévy walks are poorly understood. Here, we modeled nonlinear systems for the generation of locomotion and showed that Lévy walks emerging near a critical point had optimal dynamic ranges for coding information. This discovery suggested that Lévy walks could change movement trajectories based on the magnitude of environmental stimuli. We then showed that the high flexibility of Lévy walks enabled switching exploitation/exploration based on the nature of external cues. Finally, we analyzed the movement trajectories of freely moving Drosophila larvae and showed empirically that the Lévy walks may emerge near a critical point and have large dynamic range and high flexibility. Our results suggest that the commonly observed Lévy walks emerge near a critical point and could be explained on the basis of these functional advantages.

Termite males enhance mating encounters by changing speed according to density.

Search theory predicts that animals evolve efficient movement patterns to enhance encounter rates with specific targets. The optimal movements vary with the surrounding environments, which may explain the observation that animals often switch their movement patterns depending on conditions. However, the effectiveness of behavioural change during search is rarely evaluated because it is difficult to examine the actual encounter dynamics. Here we studied how partner-seeking termites update their search strategies depending on the local densities of potential mates. After a dispersal flight, termites drop their wings and walk to search for a mate; when a female and a male meet, they form a female-led tandem pair and search for a favourable nesting site. If a pair is separated, they have two search options-reunite with their stray partner, or seek a new partner. We hypothesized that the density of individuals affects separation-reunion dynamics and thus the optimal search strategy. We observed the searching process across different densities and found that termite pairs were often separated but obtained a new partner quickly at high mate density. After separation, while females consistently slowed down, males increased their speed according to the density. Under high mate density, separated males obtained a partner earlier than females, who do not change movement with density. Our data-based simulations confirmed that the observed behavioural change by males contributes to enhancing encounters. Males at very low mate densities did best to move slowly and thereby reduce the risk of missing their stray partner, who is the only available mate. On the other hand, males that experienced high mate densities did better in mating encounters by moving fast because the risk of isolation is low, and they must compete with other males to find a partner. These results demonstrate that termite males adaptively update their search strategy depending on conditions. Understanding the encounter dynamics experienced by animals is key to connecting the empirical work to the idealized search processes of theoretical studies.

Hybrid foraging in patchy environments using spatial memory.

Efficient random searches are essential to the survival of foragers searching for sparsely distributed targets. Lévy walks have been found to optimize the search over a wide range of constraints. When targets are distributed within patches, generating a spatial memory over the detected targets can be beneficial towards optimizing the search efficiency. Because foragers have limited memory, storing each target location separately is unrealistic. Instead, we propose incrementally learning a spatial distribution in favour of memorizing target locations. We demonstrate that an ensemble of Gaussian mixture models is a suitable candidate for such a spatial distribution. Using this, a hybrid foraging strategy is proposed, which interchanges random searches with informed movement. Informed movement results in displacements towards target locations, and is more likely to occur if the learned spatial distribution is correct. We show that, depending on the strength of the memory effects, foragers optimize search efficiencies by continuous revisitation of non-destructive targets. However, this negatively affects both the target and patch diversity, indicating that memory does not necessarily optimize multi-objective searches. Hence, the benefits of memory depend on the specific goals of the forager. Furthermore, through analysis of the distribution over walking distances of the forager, we show that memory changes the underlying walk characteristics. Specifically, the forager resorts to Brownian motion instead of Lévy walks, due to truncation of the long straight line displacements resulting from memory effects. This study provides a framework that opens up new avenues for investigating memory effects on foraging in sparse environments.

Efficiency improvement by navigated safety inspection involving visual clutter based on the random search model.

Navigated inspection seeks to improve hazard identification (HI) accuracy. With a tight inspection schedule, HI also requires efficiency. However, lacking quantification of HI efficiency, navigated inspection strategies cannot be comprehensively assessed. This work aims to determine inspection efficiency in navigated safety inspection, controlling for HI accuracy. Based on a cognitive method of the random search model (RSM), an experiment was conducted to observe the HI efficiency in navigation, for a variety of visual clutter (VC) scenarios, while using eye-tracking devices to record the search process and analyze the search performance. The results show that the RSM is an appropriate instrument, and VC serves as a hazard classifier for navigation inspection in improving inspection efficiency. This suggests a new and effective solution for addressing the low accuracy and efficiency of manual inspection through navigated inspection involving VC and the RSM. It also provides insights into the inspectors' safety inspection ability.

Deep Learning Convolutional Neural Networks for the Estimation of Liver Fibrosis Severity from Ultrasound Texture.

Diagnosis and staging of liver fibrosis is a vital prognostic marker in chronic liver diseases. Due to the inaccuracies and risk of complications associated with liver core needle biopsy, the current standard for diagnosis, other less invasive methods are sought for diagnosis. One such method that has been shown to correlate well with liver fibrosis is shear wave velocity measured by ultrasound (US) shear wave elastography; however, this technique requires specific software, hardware, and training. A current perspective in the radiology community is that the texture pattern from an US image may be predictive of the stage of liver fibrosis. We propose the use of convolutional neural networks (CNNs), a framework shown to be well suited for real world image interpretation, to test whether the texture pattern in gray scale elastography images (B-mode US with fixed, subject-agnostic acquisition settings) is predictive of the shear wave velocity (SWV). In this study, gray scale elastography images from over 300 patients including 3,500 images with corresponding SWV measurements were preprocessed and used as input to 100 different CNN architectures that were trained to regress shear wave velocity. In this study, even the best performing CNN explained only negligible variation in the shear wave velocity measures. These extensive test results suggest that the gray scale elastography image texture provides little predictive information about shear wave velocity and liver fibrosis.

Energy-Effective Data Gathering for UAV-Aided Wireless Sensor Networks.

Unmanned aerial vehicles (UAVs) are capable of serving as a data collector for wireless sensor networks (WSNs). In this paper, we investigate an energy-effective data gathering approach in UAV-aided WSNs, where each sensor node (SN) dynamically chooses the transmission modes, i.e., (1) waiting, (2) conventional sink node transmission, (3) uploading to UAV, to transmit sensory data within a given time. By jointly considering the SN's transmission policy and UAV trajectory optimization, we aim to minimize the transmission energy consumption of the SNs and ensure all sensory data completed collected within the given time. We take a two-step iterative approach and decouple the SN's transmission design and UAV trajectory optimization process. First, we design the optimal SNs transmission mode policy with preplanned UAV trajectory. A dynamic programming (DP) algorithm is proposed to obtain the optimal transmission policy. Then, with the fixed transmission policy, we optimize the UAV's trajectory from the preplanned trace with recursive random search (RRS) algorithm. Numerical results show that the proposed scheme achieves significant energy savings gain over the benchmark schemes.