A proof of concept reinforcement learning based tool for non parametric population pharmacokinetics workflow optimization.

The building of population pharmacokinetic models can be described as an iterative process in which given a model and a dataset, the pharmacometrician introduces some changes to the model specification, then perform an evaluation and based on the predictions obtained performs further optimization. This process (perform an action, witness a result, optimize your knowledge) is a perfect scenario for the implementation of Reinforcement Learning algorithms. In this paper we present the conceptual background and a implementation of one of those algorithms aiming to show pharmacometricians how to automate (to a certain point) the iterative model building process.We present the selected discretization for the action and the state space. SARSA (State-Action-Reward-State-Action) was selected as the RL algorithm to use, configured with a window of 1000 episodes with and a limit of 30 actions per episode. SARSA was configured to control an interface to the Non-Parametric Optimal Design algorithm, that was actually performing the parameter optimization.The Reinforcement Learning (RL) based agent managed to obtain the same likelihood and number of support points, with a distribution similar to the reported in the original paper. The total amount of time used by the train the agent was 5.5 h although we think this time can be further improved. It is possible to automatically find the structural model that maximizes the final likelihood for an specific pharmacokinetic dataset by using RL algorithm. The framework provided could allow the integration of even more actions i.e: add/remove covariates, non-linear compartments or the execution of secondary analysis. Many limitations were found while performing this study but we hope to address them all in future studies.

Developing New Drugs for Mycobacterium tuberculosis Therapy: What Information Do We Get from Preclinical Animal Models?

Preclinical animal models of infection are employed to develop new agents but also to screen among molecules to rank them. There are often major differences between human pharmacokinetic (PK) profiles and those developed by animal models of infection, and these may lead to substantial differences in efficacy relative to that seen in humans. Linezolid is a repurposed agent employed to great effect for therapy of Mycobacterium tuberculosis In this study, we used the hollow-fiber infection model (HFIM) to evaluate the impact of different pharmacokinetic profiles of mice and nonhuman primates (NHP) versus humans on bacterial cell kill as well as resistance suppression. We examined both plasma and epithelial lining fluid (ELF) profiles. We examined simulated exposures equivalent to 600 mg and 900 mg daily of linezolid in humans. For both plasma and ELF exposures, the murine PK profile provided estimates of effect that were biased low relative to human and NHP PK profiles. Mathematical modeling identified a linkage between minimum concentrations (Cmin) and bacterial kill and peak concentrations (Cpeak) and resistance suppression, with the latter being supported by a prospective validation study. Finding new agents with novel mechanisms of action against M. tuberculosis is difficult. It would be a tragedy to discard a new agent because of a biased estimate of effect in a preclinical animal system. The HFIM provides a system to benchmark evaluation of new compounds in preclinical animal model systems against human PK effects (species scale-up estimates of PK), to safeguard against unwarranted rejection of promising new agents.

The Combination of Fosfomycin plus Meropenem Is Synergistic for Pseudomonas aeruginosa PAO1 in a Hollow-Fiber Infection Model.

Treating high-density bacterial infections is a challenging clinical problem. We have a paucity of new agents that can address this problem. Pseudomonas aeruginosa is a particularly difficult pathogen to treat effectively because of the plethora of resistance mechanisms it carries. Fosfomycin is an agent discovered circa 40 years ago. Recently, it has been resurrected in the United States and studied for intravenous therapy. We hypothesized that, to maximize its utility, it would require combination chemotherapy when used in a clinical circumstance in high-bacterial-burden infections. We chose to examine the combination of meropenem plus fosfomycin. These agents were studied in the hollow-fiber infection model. We utilized a fully factorial study design, looking at 2 doses of meropenem alone (1 and 2 g 8-hourly) and two doses of fosfomycin alone (6 and 8 g 8-hourly), as well as all possible combinations plus a no-treatment control. We used a high-dimensional model of 5 inhomogeneous differential equations with 5 system outputs to analyze all data simultaneously. Combination therapy outperformed all monotherapy regimens, with all combinations driving >6 log10 CFU/ml of bacterial killing. Combination therapy was able to counterselect resistance emergence (meropenem mutants being killed by the combination, as well as fosfomycin mutants being killed by the combination) in all regimens studied. The analysis demonstrated that the combination was significantly synergistic for bacterial cell killing and resistance suppression. Meropenem plus fosfomycin is a promising combination for therapy of high-burden Pseudomonas aeruginosa infections and requires further study.

Predicting the temporal responses of non-phase-locking bullfrog auditory units to complex acoustic waveforms.

Axons from the basilar papilla of the American bullfrog (Rana catesbeiana) do not phase lock to stimuli within an octave of their best frequencies. Nevertheless, they show consistent temporal patterns of instantaneous spike rate (as reflected in peristimulus time histograms) in response to repeated stimuli in that frequency range. We show that the second-order Wiener kernels for these axons, derived from the cross-correlation of continuous (non-repeating), broad-band noise stimulus with the spike train produced in response to that stimulus, can predict with considerable precision the temporal pattern of instantaneous spike rate in response to a novel, complex acoustic waveform (a repeated, 100-ms segment of noise, band-limited to cover the single octaves above and below best frequency). Furthermore, we show that most of this predictive power is retained when the second-order Wiener kernel is reduced to the highest-ranking pair of singular vectors derived from singular-value decomposition, that the retained pair of vectors corresponds to a single auditory filter followed by an envelope-detection process, and that the auditory filter itself predicts the characteristic frequency (CF) of the axon and the shape of the frequency-threshold tuning curve in the vicinity of CF.

Essential roles of noise in neural coding and in studies of neural coding.

We present examples of results from our studies of auditory primary afferent nerve fibers and populations of such fibers in the frog and gerbil. We take advantage of the natural dithering effect of internal noise, where it is sufficient, to construct highly predictive descriptive models (based on the Wiener series with kernels derived from white-noise analysis). Where the internal noise is insufficient, we enhance dithering by applying external acoustic noise together with our stimuli. Using acoustic noise as a background sound, orthogonal to the stimulus waveform, we show that under some circumstances such background sound can enhance the ability of individual fibers and populations of fibers to encode the stimulus waveform.

Time course of transmitter release calculated from simulations of a calcium diffusion model.

A three-dimensional presynaptic calcium diffusion model developed to account for characteristics of transmitter release was modified to provide for binding of calcium to a receptor and subsequent triggering of exocytosis. When low affinity (20 microM) and rapid kinetics were assumed for the calcium receptor triggering exocytosis, and stimulus parameters were selected to match those of experiments, the simulations predicted a virtual invariance of the time course of transmitter release to paired stimulation, stimulation with pulses of different amplitude, and stimulation in different calcium solutions. The large temperature sensitivity of experimental release time course was explained by a temperature sensitivity of the model's final rate limiting exocytotic process. Inclusion of calcium tail currents and a saturable buffer with finite binding kinetics resulted in high peak calcium transients near release sites, exceeding 100 microM. Models with a single class of calcium binding site to the secretory trigger molecule failed to produce sufficient synaptic facilitation under this condition. When at least one calcium ion binds to a different site having higher affinity and slow kinetics, facilitation again reaches levels similar to those seen experimentally. It is possible that the neurosecretory trigger molecule reacts with calcium at more than one class of binding site.