Reproduction Factor Based Latent Epidemic Model Inference: A Data-Driven Approach Using COVID-19 Datasets.

The mathematical modeling of infectious diseases aims to evaluate the transmissibility of the on-going spread of disease and guide the government's control strategies and interventions. In this paper, we propose a novel transmissibility indicator, reproduction factor, which evaluates the number of secondary infections from a single nonisolated infectious individual. In contrast to classic reproduction numbers, the reproduction factor explicitly considers the fraction of susceptible individuals (who are not immune to disease naturally or through vaccination) and the nonisolated population to evaluate near real-time transmissibility. Thus, it can be an effective indicator when the spread of disease has progressed and control strategies have been implemented. Other merits of the proposed reproduction factor include data-driven inference based on a Markov chain, which enables the inference of latent information, such as the number of nondetected infectious individuals and the number of daily new infections. We performed an extensive simulation using the COVID-19 datasets of Germany, Italy, South Korea, and California (the U.S.) to verify our model. We further compared the results with other transmissibility measures, including reproduction numbers, and the results of state-of-the-art epidemic models. Through the results, we confirmed that the proposed reproduction factor and corresponding inference model explained the COVID-19 datasets.

Inverse Rational Control with Partially Observable Continuous Nonlinear Dynamics.

A fundamental question in neuroscience is how the brain creates an internal model of the world to guide actions using sequences of ambiguous sensory information. This is naturally formulated as a reinforcement learning problem under partial observations, where an agent must estimate relevant latent variables in the world from its evidence, anticipate possible future states, and choose actions that optimize total expected reward. This problem can be solved by control theory, which allows us to find the optimal actions for a given system dynamics and objective function. However, animals often appear to behave suboptimally. Why? We hypothesize that animals have their own flawed internal model of the world, and choose actions with the highest expected subjective reward according to that flawed model. We describe this behavior as rational but not optimal. The problem of Inverse Rational Control (IRC) aims to identify which internal model would best explain an agent's actions. Our contribution here generalizes past work on Inverse Rational Control which solved this problem for discrete control in partially observable Markov decision processes. Here we accommodate continuous nonlinear dynamics and continuous actions, and impute sensory observations corrupted by unknown noise that is private to the animal. We first build an optimal Bayesian agent that learns an optimal policy generalized over the entire model space of dynamics and subjective rewards using deep reinforcement learning. Crucially, this allows us to compute a likelihood over models for experimentally observable action trajectories acquired from a suboptimal agent. We then find the model parameters that maximize the likelihood using gradient ascent. Our method successfully recovers the true model of rational agents. This approach provides a foundation for interpreting the behavioral and neural dynamics of animal brains during complex tasks.