Disorders of compulsivity: Deficits in arbitrating learning strategies.

While previous research has shown that compulsivity is related to an imbalance between goal-directed and habitual learning systems, very little is known about whether this effect is due to the impairment of a single system or the impairment of the arbitration mechanism that determines which system controls behaviour at any given moment; the current study aims to address this disagreement. Nineteen alcohol use disorder, 30 obsessive-compulsive disorder (OCD) and 20 major depressive disorder patients and corresponding sex- and age-matched controls performed two-choice, three-stage Markov decision-making paradigm. Model-based and mode-free reinforcement learning models were used to independently fitted their behavioural data. Alcohol use disorder and OCD patients showed less model-based strategy choice than healthy controls in task conditions where the model-based strategy was optimal. Only OCD patients showed higher behavioural control system switching in task conditions where model-free use was optimal. Major depressive disorder patients did not differ from the matched control in both. These findings suggest that dysfunction in arbitration control between dual systems may be the basis for diverse disorders involving compulsivity.

An optimal control algorithm toward unknown constrained nonlinear systems based on the sequential sampling and updating of surrogate model.

The application of optimal control theory in practical engineering is often limited by the modeling cost and complexity of the mathematical model of the controlled plant, and various constraints. To bridge the gap between the theory and practice, this paper proposes a model-free direct method based on the sequential sampling and updating of surrogate model, and extends the ability of direct method to solve model-free optimal control problems with general constraints. The algorithm selects sample points from the current actual trajectory data to update the surrogate model of controlled plant, and solve the optimal control problem of the constantly refined surrogate model until the result converges. The presented initial and subsequent sampling strategies eliminate the dependence on the model. Furthermore, the new stopping criteria ensure the overlap of final actual and planned trajectories. The several examples illustrate that the presented algorithm can obtain constrained solutions with greater accuracy and require fewer sample data.

Deuterium spin relaxation of fractionally deuterated ribonuclease H using paired 475 and 950 MHz NMR spectrometers.

Deuterium (2H) spin relaxation of 13CH2D methyl groups has been widely applied to investigate picosecond-to-nanosecond conformational dynamics in proteins by solution-state NMR spectroscopy. The B0 dependence of the 2H spin relaxation rates is represented by a linear relationship between the spectral density function at three discrete frequencies J(0), J(ωD) and J(2ωD). In this study, the linear relation between 2H relaxation rates at B0 fields separated by a factor of two and the interpolation of rates at intermediate frequencies are combined for a more robust approach for spectral density mapping. The general usefulness of the approach is demonstrated on a fractionally deuterated (55%) and alternate 13C-12C labeled sample of E. coli RNase H. Deuterium relaxation rate constants (R1, R1ρ, RQ, RAP) were measured for 57 well-resolved 13CH2D moieties in RNase H at 1H frequencies of 475 MHz, 500 MHz, 900 MHz, and 950 MHz. The spectral density mapping of the 475/950 MHz data combination was performed independently and jointly to validate the expected relationship between data recorded at B0 fields separated by a factor of two. The final analysis was performed by jointly analyzing 475/950 MHz rates with 700 MHz rates interpolated from 500/900 MHz data to yield six J(ωD) values for each methyl peak. The J(ω) profile for each peak was fit to the original (τM, Sf2, τf) or extended model-free function (τM, Sf2, Ss2, τf, τs) to obtain optimized dynamic parameters.

Kinetic Study of Anaerobic Adhesive Curing on Copper and Iron Base Substrates.

Anaerobic adhesives (AAs) cure at room temperature in oxygen-deprived spaces between metal substrates. The curing process is significantly influenced by the type of metal ions present. This study investigates the curing kinetics of a high-strength AA on iron and copper substrates using differential scanning calorimetry (DSC). The activation energy and kinetic parameters were determined with different empiric models, revealing that curing on copper is faster and more complete compared to iron. The findings suggest that copper ions lower the activation energy required for curing, enhancing the adhesive's performance. This research addresses the gap in understanding how metal ions affect AA curing kinetics, offering valuable insights for optimizing adhesive formulations for industrial applications.

The well-worn route revisited: Striatal and hippocampal system contributions to familiar route navigation.

Classic research has shown a division in the neuroanatomical structures that support flexible (e.g., short-cutting) and habitual (e.g., familiar route following) navigational behavior, with hippocampal-caudate systems associated with the former and putamen systems with the latter. There is, however, disagreement about whether the neural structures involved in navigation process particular forms of spatial information, such as associations between constellations of cues forming a cognitive map, versus single landmark-action associations, or alternatively, perform particular reinforcement learning algorithms that allow the use of different spatial strategies, so-called model-based (flexible) or model-free (habitual) forms of learning. We sought to test these theories by asking participants (N = 24) to navigate within a virtual environment through a previously learned, 9-junction route with distinctive landmarks at each junction while undergoing functional magnetic resonance imaging (fMRI). In a series of probe trials, we distinguished knowledge of individual landmark-action associations along the route versus knowledge of the correct sequence of landmark-action associations, either by having absent landmarks, or "out-of-sequence" landmarks. Under a map-based perspective, sequence knowledge would not require hippocampal systems, because there are no constellations of cues available for cognitive map formation. Within a learning-based model, however, responding based on knowledge of sequence would require hippocampal systems because prior context has to be utilized. We found that hippocampal-caudate systems were more active in probes requiring sequence knowledge, supporting the learning-based model. However, we also found greater putamen activation in probes where navigation based purely on sequence memory could be planned, supporting models of putamen function that emphasize its role in action sequencing.

Robust model-free adaptive iterative learning control for an autonomous bus trajectory tracking system.

A robust model-free adaptive iterative learning control (R-MFAILC) algorithm is proposed in this work to address the issue of laterally controlling an autonomous bus. First, according to the periodic repetitive working characteristics of autonomous buses, a novel dynamic linearized method used in the iterative domain is utilized, and a time-varying data model with a pseudo gradient (PG) is given. Then, the R-MFAILC controller is designed with a proposed adaptive attenuation factor. The proposed algorithm's advantage lies in the R-MFAILC controller, which solely utilizes the input and output data of the regulated entity. Moreover, the R-MFAILC controller has strong robustness and can handle the nonlinear measurement disturbances of the system. In simulations based on the Truck-Sim simulation platform, the effectiveness of the proposed algorithm is verified. A rigorous mathematical analysis is employed to demonstrate the stability and convergence of the proposed algorithm.

Explaining empirical dynamic modelling using verbal, graphical and mathematical approaches.

Empirical dynamic modelling (EDM) is becoming an increasingly popular method for understanding the dynamics of ecosystems. It has been applied to laboratory, terrestrial, freshwater and marine systems, used to forecast natural populations and has addressed fundamental ecological questions. Despite its increasing use, we have not found full explanations of EDM in the ecological literature, limiting understanding and reproducibility. Here we expand upon existing work by providing a detailed introduction to EDM. We use three progressively more complex approaches. A short verbal explanation of EDM is then explicitly demonstrated by graphically working through a simple example. We then introduce a full mathematical description of the steps involved. Conceptually, EDM translates a time series of data into a path through a multi-dimensional space, whose axes are lagged values of the time series. A time step is chosen from which to make a prediction. The state of the system at that time step corresponds to a 'focal point' in the multi-dimensional space. The set (called the library) of candidate nearest neighbours to the focal point is constructed, to determine the nearest neighbours that are then used to make the prediction. Our mathematical explanation explicitly documents which points in the multi-dimensional space should not be considered as focal points. We suggest a new option for excluding points from the library that may be useful for short-term time series that are often found in ecology. We focus on the core simplex and S-map algorithms of EDM. Our new R package, pbsEDM, enhances understanding (by outputting intermediate calculations), reproduces our results and can be applied to new data. Our work improves the clarity of the inner workings of EDM, a prerequisite for EDM to reach its full potential in ecology and have wide uptake in the provision of advice to managers of natural resources.

An Adaptive Control Scheme Based on Non-Interference Nonlinearity Approximation for a Class of Nonlinear Cascaded Systems and Its Application to Flexible Joint Manipulators.

Control design for the nonlinear cascaded system is challenging due to its complicated system dynamics and system uncertainty, both of which can be considered some kind of system nonlinearity. In this paper, we propose a novel nonlinearity approximation scheme with a simplified structure, where the system nonlinearity is approximated by a steady component and an alternating component using only local tracking errors. The nonlinearity of each subsystem is estimated independently. On this basis, a model-free adaptive control for a class of nonlinear cascaded systems is proposed. A squared-error correction procedure is introduced to regulate the weight coefficients of the approximation components, which makes the whole adaptive system stable even with the unmodeled uncertainties. The effectiveness of the proposed controller is validated on a flexible joint system through numerical simulations and experiments. Simulation and experimental results show that the proposed controller can achieve better control performance than the radial basis function network control. Due to its simplicity and robustness, this method is suitable for engineering applications.

Model-free robust motion control for biological optical microscopy using time-delay estimation with an adaptive RBFNN compensator.

The field of large numerical aperture microscopy has witnessed significant advancements in spatial and temporal resolution, as well as improvements in optical microscope imaging quality. However, these advancements have concurrently raised the demand for enhanced precision, extended range, and increased load-bearing capacity in objective motion carrier (OMC). To address this challenge, this study introduces an innovative OMC that employs a ball screw mechanism as its primary driving component. Furthermore, a robust nonlinear motion control strategy has been developed, which integrates fast nonsingular terminal sliding mode, experimental estimation techniques, and adaptive radial basis neural network, to mitigate the impact of nonlinear friction within the ball screw mechanism on motion precision. The stability of the closed-loop control system has been rigorously demonstrated through Lyapunov theory. Compared with other enhanced sliding mode control strategies, the maximum error and root mean square error of this controller are improved by 33% and 34% respectively. The implementation of the novel OMC has enabled the establishment of a high-resolution bio-optical microscope, which has proven its effectiveness in the microscopic imaging of retinal organoids.

Breaking down walls: Continuous potential models for internal motions in NMR spin relaxation.

Simple physical models for restricted diffusion in a potential, which provide important insights for NMR spin relaxation, usually are based on free diffusion within rigid boundaries or diffusion in relatively simple continuous potential energy surfaces. The diffusion-in-a-cone model is an example of the former and diffusion in an N-fold cosine potential is an example of the latter. The present work models restricted diffusion for arbitrary potential energy functions on the surface of a cone or a sphere, by expanding the potentials in Fourier or spherical harmonic series, respectively. The results exhibit simple relationships between generalized order parameters and effective correlation times, critical for analysis of experimental spin relaxation data, and illustrate the transition from diffusive-like to jump-like behavior in multi-well potentials.

Electrophysiological analysis of signal detection outcomes emphasizes the role of decisional factors in recognition memory.

Introduction: Event-related potential (ERP) studies have identified two time windows associated with recognition memory and interpreted them as reflecting two processes: familiarity and recollection. However, using relatively simple stimuli and achieving high recognition rates, most studies focused on hits and correct rejections. This leaves out some information (misses and false alarms) that according to Signal Detection Theory (SDT) is necessary to understand signal processing. Methods: We used a difficult visual recognition task with colored pictures of different categories to obtain enough of the four possible SDT outcomes and analyzed them with modern ERP methods. Results: Non-parametric analysis of these outcomes identified a single time window (470 to 670 ms) which reflected activity within fronto-central and posterior-left clusters of electrodes, indicating differential processing. The posterior-left cluster significantly distinguished all STD outcomes. The fronto-central cluster only distinguished ERPs according to the subject's response: yes vs. no. Additionally, only electrophysiological activity within the posterior-left cluster correlated with the discrimination index (d'). Discussion: We show that when all SDT outcomes are examined, ERPs of recognition memory reflect a single-time window that may reveal a bottom-up factor discriminating the history of items (i.e. memory strength), as well as a top-down factor indicating participants' decision.

Feedforward-feedback-enhanced model-free adaptive iterative learning control with measurement disturbance and data dropout for an autonomous bus trajectory tracking system.

This article presents an innovative enhanced model-free adaptive iterative learning control approach suited for autonomous bus trajectory tracking systems that may experience measurement disruptions and random data dropouts. Data loss can occur independently and randomly at different times and in different iterations with varying probabilities, leading to successive data dropouts on both the time and iteration axes. The proposed enhanced model-free adaptive iterative learning control controller incorporates a data compensation mechanism to compensate for missing data, ensuring excellent control performance. This data-driven control strategy requires only input/output data for controller design. The convergence and effectiveness of the proposed approach are verified through rigorous mathematical analysis and simulation outcomes.

Kinetics Study of PVA Polymer by Model-Free and Model-Fitting Methods Using TGA.

Thermogravimetric Analysis (TGA) serves a pivotal technique for evaluating the thermal behavior of Polyvinyl alcohol (PVA), a polymer extensively utilized in the production of fibers, films, and membranes. This paper targets the kinetics of PVA thermal degradation using high three heating rate range 20, 30, and 40 K min-1. The kinetic study was performed using six model-free methods: Freidman (FR), Flynn-Wall-Qzawa (FWO), Kissinger-Akahira-Sunose (KAS), Starink (STK), Kissinger (K), and Vyazovkin (VY) for the determination of the activation energy (Ea). TGA showed two reaction stages: the main one at 550-750 K and the second with 700-810 K. But only the first step has been considered in calculating Ea. The average activation energy values for the conversion range (0.1-0.7) are between minimum 104 kJ mol-1 by VY to maximum 199 kJ mol-1 by FR. Model-fitting has been applied by combing Coats-Redfern (CR) with the master plot (Criado's) to identify the most convenient reaction mechanism. Ea values gained by the above six models were very similar with the average value of (126 kJ mol-1) by CR. The reaction order models-Second order (F2) was recommended as the best mechanism reaction for PVA pyrolysis. Mechanisms were confirmed by the compensation effect. Finally, (∆H, ∆G, and ∆S) parameters were presented and proved that the reaction is endothermic.

Model-Free Control of a Soft Pneumatic Segment.

Soft robotics faces challenges in attaining control methods that ensure precision from hard-to-model actuators and sensors. This study focuses on closed-chain control of a segment of PAUL, a modular pneumatic soft arm, using elastomeric-based resistive sensors with negative piezoresistive behaviour irrespective of ambient temperature. PAUL's performance relies on bladder inflation and deflation times. The control approach employs two neural networks: the first translates position references into valve inflation times, and the second acts as a state observer to estimate bladder inflation times using sensor data. Following training, the system achieves position errors of 4.59 mm, surpassing the results of other soft robots presented in the literature. The study also explores system modularity by assessing performance under external loads from non-actuated segments.

Model-free finite-time robust control using fractional-order ultra-local model and prescribed performance sliding surface for upper-limb rehabilitation exoskeleton.

To address the trajectory tracking issue of upper-limb rehabilitation exoskeleton with uncertainties and external disturbances, this paper proposes a fractional-order ultra-local model-based model-free finite-time robust controller (FO-FTRC) using predefined performance sliding surface. Different from previous model-free control strategies, a novel multi-input multi-output (MIMO) fractional-order ultra-local model which is a virtual model is proposed to approximate the complex uncertain nonlinear exoskeleton dynamics in a short sliding time window. This allows the design of controller to be independent of any exoskeleton model information and reduces the difficulty of controller design. The developed robust model-free control method incorporates a fractional-order quasi-time delay estimator (FO-QTDE), unknown disturbance estimator (UDE) as well as prescribed performance sliding mode control (PPSMC). The FO-QTDE is utilized to estimate the unknown lumped uncertainties which employs short time delayed knowledge only about the control input. However, the low-pass filter is always added for FO-QTDE when disturbances change fast, which leads to unavoidable estimation error. Then, UDE is designed to further eliminate the estimation error of FO-QTDE to enhance control performance. The PPSMC is constructed to converge sliding surface to zero in a finite time. Besides, the sliding surface is always limited in performance boundaries. After that, the overall system stability and convergence analyses are demonstrated by using the Lyapunov theorem. Finally, with the comparison to other methods of α-variable adaptive model free control (α-AMFC), time-delay estimation-based continuous nonsingular fast terminal sliding mode controller (TDE-CNFTSMC), time delay estimation (TDE)-based model-free fractional-order nonsingular fast terminal sliding mode control (MFF-TSM) and fractional-order proportion-differential (PDß), the co-simulation results on 7-degree-of-freedom (DOF) iReHave upper-limb exoskeleton virtual prototype and experiment results on 2-DOF upper-limb exoskeleton are obtained to illustrate the effectiveness and superiority of the proposed FO-FTRC method.

Model-based and model-free mechanisms in methamphetamine use disorder.

People with methamphetamine use disorder (MUD) struggle to shift their behaviour from methamphetamine-orientated habits to goal-oriented choices. The model-based/model-free framework is well suited to understand this difficulty by unpacking the computational mechanisms that support experienced-based (model-free) and goal-directed (model-based) choices. We aimed to examine whether 1) participants with MUD differed from controls on behavioural proxies and/or computational mechanisms of model-based/model-free choices; 2) model-based/model-free decision-making correlated with MUD symptoms; and 3) model-based/model-free deficits improved over six weeks in the group with MUD. Participants with MUD and controls with similar age, IQ and socioeconomic status completed the Two-Step Task at treatment commencement (MUD n = 30, Controls n = 31) and six weeks later (MUD n = 23, Controls n = 26). We examined behavioural proxies of model-based/model-free decisions using mixed logistic regression, and their underlying mechanisms using computational modelling. At a behavioural level, participants with MUD were more likely to switch their choices following rewarded actions, although this pattern improved at follow up. At a computational level, groups were similar in their use of model-based mechanisms, but participants with MUD were less likely to apply model-free mechanisms and less likely to repeat rewarded actions. We did not find evidence that individual differences in model-based or model-free parameters were associated with greater severity of methamphetamine dependence, nor did we find that group differences in computational parameters changed between baseline and follow-up assessment. Decision-making challenges in people with MUD are likely related to difficulties in pursuing choices previously associated with positive outcomes.

Intelligent Navigation of a Magnetic Microrobot with Model-Free Deep Reinforcement Learning in a Real-World Environment.

Microrobotics has opened new horizons for various applications, especially in medicine. However, it also witnessed challenges in achieving maximum optimal performance. One key challenge is the intelligent, autonomous, and precise navigation control of microrobots in fluid environments. The intelligence and autonomy in microrobot control, without the need for prior knowledge of the entire system, can offer significant opportunities in scenarios where their models are unavailable. In this study, two control systems based on model-free deep reinforcement learning were implemented to control the movement of a disk-shaped magnetic microrobot in a real-world environment. The training and results of an off-policy SAC algorithm and an on-policy TRPO algorithm revealed that the microrobot successfully learned the optimal path to reach random target positions. During training, the TRPO exhibited a higher sample efficiency and greater stability. The TRPO and SAC showed 100% and 97.5% success rates in reaching the targets in the evaluation phase, respectively. These findings offer basic insights into achieving intelligent and autonomous navigation control for microrobots to advance their capabilities for various applications.

Large Range of a High-Precision, Independent, Sub-Mirror Three-Dimensional Co-Phase Error Sensing and Correction Method via a Mask and Population Algorithm.

The emergence of segmented mirrors is expected to solve the design, processing, manufacturing, testing, and launching of space telescopes of large apertures. However, with the increase in the number of sub-mirrors, the sensing and correction of co-phase errors in segmented mirrors will be very difficult. In this paper, an independent three-dimensional method for sub-mirror co-phase error sensing and correction method is proposed. The method is based on a wide spectral modulation transfer function (MTF), mask, population optimization algorithm, and online model-free correction. In this method, the sensing and correction process of each sub-mirror co-phase error is independent of each other, so the increase in the number of sub-mirrors will not increase the difficulty of the method. This method can sense and correct the co-phase errors of three dimensions of the sub-mirror, including piston, tip, and tilt, even without modeling the optical system, and has a wide detection range and high precision. And the efficiency is high because the sub-mirrors can be corrected simultaneously in parallel. Simulation results show that the proposed method can effectively sense and correct the co-phase errors of the sub-mirrors in the range [-50λ, 50λ] in three dimensions with high precision. The average RMSE value in 100 experiments of the true co-phase error values and the experimental co-phase error values of one of the six sub-mirrors is 2.358 × 10-7λ.

A comparison of model-free phase I dose escalation designs for dual-agent combination therapies.

It is increasingly common for therapies in oncology to be given in combination. In some cases, patients can benefit from the interaction between two drugs, although often at the risk of higher toxicity. A large number of designs to conduct phase I trials in this setting are available, where the objective is to select the maximum tolerated dose combination. Recently, a number of model-free (also called model-assisted) designs have provoked interest, providing several practical advantages over the more conventional approaches of rule-based or model-based designs. In this paper, we demonstrate a novel calibration procedure for model-free designs to determine their most desirable parameters. Under the calibration procedure, we compare the behaviour of model-free designs to model-based designs in a comprehensive simulation study, covering a number of clinically plausible scenarios. It is found that model-free designs are competitive with the model-based designs in terms of the proportion of correct selections of the maximum tolerated dose combination. However, there are a number of scenarios in which model-free designs offer a safer alternative. This is also illustrated in the application of the designs to a case study using data from a phase I oncology trial.

Investigation of the evolved pyrolytic products and energy potential of Bagasse: experimental, kinetic, thermodynamic and boosted regression trees analysis.

This study explored bagasse's energy potential grown using treated industrial wastewater through various analyses, experimental, kinetic, thermodynamic, and machine learning boosted regression tree methods. Thermogravimetry was employed to determine thermal degradation characteristics, varying the heating rate from 10 to 30 °C/min. The primary pyrolysis products from bagasse are H2, CH4, H2O, CO2, and hydrocarbons. Kinetic parameters were estimated using three model-free methods, yielding activation energies of approximately 245.98 kJ mol-1, 247.58 kJ mol-1, and 244.69 kJ mol-1. Thermodynamic parameters demonstrated the feasibility and reactivity of pyrolysis with ΔH ≈ 240.72 kJ mol-1, ΔG ≈ 162.87 kJ mol-1, and ΔS ≈ 165.35 J mol-1 K-1. The distribution of activation energy was analyzed using the multiple distributed activation energy model. Lastly, boosted regression trees predicted thermal degradation successfully, with an R2 of 0.9943. Therefore, bagasse's potential as an eco-friendly alternative to fossil fuels promotes waste utilization and carbon footprint reduction.