Optimal STI controls for HIV patients based on an efficient deep Q learning method.

We investigate an efficient computational tool to suggest useful treatment regimens for people infected with the human immunodeficiency virus (HIV). Structured treatment interruption (STI) is a regimen in which therapeutic drugs are periodically administered and withdrawn to give patients relief from an arduous drug therapy. Numerous studies have been conducted to find better STI treatment strategies using various computational tools with mathematical models of HIV infection. In this paper, we leverage a modified version of the double deep Q network with prioritized experience replay to improve the performance of classic deep learning algorithms. Numerical simulation results show that our methodology produces significantly more optimal cost values for shorter treatment periods compared to other recent studies. Furthermore, our proposed algorithm performs well in one-day segment scenarios, whereas previous studies only reported results for five-day segment scenarios.

Reformulation of Trivers-Willard hypothesis for parental investment.

The Trivers-Willard hypothesis (TWH) plays a central role in understanding the optimal investment strategies to male and female offspring. Empirical studies of TWH, however, yielded conflicting results. Here, we present models to predict optimal comprehensive multi-element parental strategies composed of primary sex ratio, brood size, resource allocation among offspring, and the resultant secondary sex ratio. Our results reveal that the optimal strategy depends on sex differences in the shape of offspring fitness function rather than in fitness variance. Also, the slope of the tangent line (through the origin) to the offspring fitness function can be used to predict the preferred offspring sex. We also briefly discuss links between the model and the empirical research. This comprehensive reformulation of TWH will offer a thorough understanding of multi-element parental investment strategies beyond the classical TWH.

Estimation of the reproduction number of influenza A(H1N1)pdm09 in South Korea using heterogeneous models.

BACKGROUND: The reproduction number is one of the most crucial parameters in determining disease dynamics, providing a summary measure of the transmission potential. However, estimating this value is particularly challenging owing to the characteristics of epidemic data, including non-reproducibility and incompleteness. METHODS: In this study, we propose mathematical models with different population structures; each of these models can produce data on the number of cases of the influenza A(H1N1)pdm09 epidemic in South Korea. These structured models incorporating the heterogeneity of age and region are used to estimate the reproduction numbers at various terminal times. Subsequently, the age- and region-specific reproduction numbers are also computed to analyze the differences illustrated in the incidence data. RESULTS: Incorporation of the age-structure or region-structure allows for robust estimation of parameters, while the basic SIR model provides estimated values beyond the reasonable range with severe fluctuation. The estimated duration of infectious period using age-structured model is around 3.8 and the reproduction number was estimated to be 1.6. The estimated duration of infectious period using region-structured model is around 2.1 and the reproduction number was estimated to be 1.4. The estimated age- and region-specific reproduction numbers are consistent with cumulative incidence for corresponding groups. CONCLUSIONS: Numerical results reveal that the introduction of heterogeneity into the population to represent the general characteristics of dynamics is essential for the robust estimation of parameters.

The effect of control measures on COVID-19 transmission in South Korea.

Countries around the world have taken control measures to mitigate the spread of COVID-19, including Korea. Social distancing is considered an essential strategy to reduce transmission in the absence of vaccination or treatment. While interventions have been successful in controlling COVID-19 in Korea, maintaining the current restrictions incurs great social costs. Thus, it is important to analyze the impact of different polices on the spread of the epidemic. To model the COVID-19 outbreak, we use an extended age-structured SEIR model with quarantine and isolation compartments. The model is calibrated to age-specific cumulative confirmed cases provided by the Korea Disease Control and Prevention Agency (KDCA). Four control measures-school closure, social distancing, quarantine, and isolation-are investigated. Because the infectiousness of the exposed has been controversial, we study two major scenarios, considering contributions to infection of the exposed, the quarantined, and the isolated. Assuming the transmission rate would increase more than 1.7 times after the end of social distancing, a second outbreak is expected in the first scenario. The epidemic threshold for increase of contacts between teenagers after school reopening is 3.3 times, which brings the net reproduction number to 1. The threshold values are higher in the second scenario. If the average time taken until isolation and quarantine reduces from three days to two, cumulative cases are reduced by 60% and 47% in the first scenario, respectively. Meanwhile, the reduction is 33% and 41%, respectively, for rapid isolation and quarantine in the second scenario. Without social distancing, a second wave is possible, irrespective of whether we assume risk of infection by the exposed. In the non-infectivity of the exposed scenario, early detection and isolation are significantly more effective than quarantine. Furthermore, quarantining the exposed is as important as isolating the infectious when we assume that the exposed also contribute to infection.

Cost-effectiveness analysis of pre-exposure prophylaxis for the prevention of HIV in men who have sex with men in South Korea: a mathematical modelling study.

In February 2018, the Ministry of Food and Drug Safety in Korea approved tenofovir disoproxil fumarate and emtricitabine (TDF/FTC) co-formulate for use in pre-exposure prophylaxis (PrEP) for the prevention of human immunodeficiency virus (HIV) infection. This study aimed to estimate the cost-effectiveness of PrEP in men who have sex with men (MSM), a major risk group emerging in Korea. A dynamic compartmental model was developed for HIV transmission and progression in MSM aged 15-64 years. With a combined model including economic analysis, we estimated averted HIV infections, changes in HIV prevalence, discounted costs, quality-adjusted life-years (QALYs), and incremental cost-effectiveness ratios (ICERs). PrEP was evaluated in both the general MSM and high-risk MSM populations and was assumed to reduce infection risk by 80%. Implementing PrEP in all MSM would avert 75.2% HIV infections and facilitate a gain of 37,372 QALYs at a cost of $274,822 per QALY gained over 20 years relative to the status quo. Initiating PrEP in high-risk MSM with an average of eight partners per year (around 20% of MSM) would improve the cost-effectiveness, averting 78.0% HIV infections and add 29,242 QALYs at a cost of $51,597 per QALY gained, which is within the willingness-to-pay threshold for Korea of $56,000/QALY gained. This result was highly sensitive to annual PrEP costs, quality-of-life for people who are on PrEP, and initial HIV prevalence. Initiating PrEP in a larger proportion of MSM in Korea would prevent more HIV infections, but at an increasing cost per QALY gained. Focusing PrEP on higher risk MSM and any reduction in PrEP cost would improve cost-effectiveness.

Feedback control problem of an SIR epidemic model based on the Hamilton-Jacobi-Bellman equation.

We consider a feedback control problem of a susceptible-infective-recovered (SIR) model to design an efficient vaccination strategy for influenza outbreaks. We formulate an optimal control problem that minimizes the number of people who become infected, as well as the costs of vaccination. A feedback methodology based on the Hamilton-Jacobi-Bellman (HJB) equation is introduced to derive the control function. We describe the viscosity solution, which is an approximation solution of the HJB equation. A successive approximation method combined with the upwind finite difference method is discussed to find the viscosity solution. The numerical simulations show that feedback control can help determine the vaccine policy for any combination of susceptible individuals and infectious individuals. We also verify that feedback control can immediately reflect changes in the number of susceptible and infectious individuals.

Feedback control of an HBV model based on ensemble kalman filter and differential evolution.

In this paper, we derive efficient drug treatment strategies for hepatitis B virus (HBV) infection by formulating a feedback control problem. We introduce and analyze a dynamic mathematical model that describes the HBV infection during antiviral therapy. We determine the reproduction number and then conduct a qualitative analysis of the model using the number. A control problem is considered to minimize the viral load with consideration for the treatment costs. In order to reflect the status of patients at both the initial time and the follow-up visits, we consider the feedback control problem based on the ensemble Kalman filter (EnKF) and differential evolution (DE). EnKF is employed to estimate full information of the state from incomplete observation data. We derive a piecewise constant drug schedule by applying DE algorithm. Numerical simulations are performed using various weights in the objective functional to suggest optimal treatment strategies in different situations.

Constrained optimal control applied to vaccination for influenza.

The efficient time schedule and prioritization of vaccine supplies are important in mitigating impact of an influenza pandemic. In practice, there are restrictions associated with limited vaccination coverage and the maximum daily vaccine administration. We extend previous work on optimal control for influenza to reflect these realistic restrictions using mixed constraints on state and control variables. An optimal control problem is formulated with the aim of minimizing the number of infected individuals while considering intervention costs. Time-dependent vaccination is computed and analysed using a model incorporating heterogeneity in population structure under different settings of transmissibility levels, vaccine coverages, and time delays.

Mathematical modeling of HIV prevention measures including pre-exposure prophylaxis on HIV incidence in South Korea.

BACKGROUND: Multiple prevention measures have the possibility of impacting HIV incidence in South Korea, including early diagnosis, early treatment, and pre-exposure prophylaxis (PrEP). We investigated how each of these interventions could impact the local HIV epidemic, especially among men who have sex with men (MSM), who have become the major risk group in South Korea. A mathematical model was used to estimate the effects of each these interventions on the HIV epidemic in South Korea over the next 40 years, as compared to the current situation. METHODS: We constructed a mathematical model of HIV infection among MSM in South Korea, dividing the MSM population into seven groups, and simulated the effects of early antiretroviral therapy (ART), early diagnosis, PrEP, and combination interventions on the incidence and prevalence of HIV infection, as compared to the current situation that would be expected without any new prevention measures. RESULTS: Overall, the model suggested that the most effective prevention measure would be PrEP. Even though PrEP effectiveness could be lessened by increased unsafe sex behavior, PrEP use was still more beneficial than the current situation. In the model, early diagnosis of HIV infection was also effectively decreased HIV incidence. However, early ART did not show considerable effectiveness. As expected, it would be most effective if all interventions (PrEP, early diagnosis and early treatment) were implemented together. CONCLUSIONS: This model suggests that PrEP and early diagnosis could be a very effective way to reduce HIV incidence in South Korea among MSM.

Optimal control of an influenza model with seasonal forcing and age-dependent transmission rates.

This study considers an optimal intervention strategy for influenza outbreaks. Variations in the SEIAR model are considered to include seasonal forcing and age structure, and control strategies include vaccination, antiviral treatment, and social distancing such as school closures. We formulate an optimal control problem by minimizing the incidence of influenza outbreaks while considering intervention costs. We examine the effects of delays in vaccine production, seasonal forcing, and age-dependent transmission rates on the optimal control and suggest some optimal strategies through numerical simulations.

Mathematical modeling of triphasic viral dynamics in patients with HBeAg-positive chronic hepatitis B showing response to 24-week clevudine therapy.

BACKGROUND: Modeling of short-term viral dynamics of hepatitis B with traditional biphasic model might be insufficient to explain long-term viral dynamics. The aim was to develop a novel method of mathematical modeling to shed light on the dissociation between early and long-term dynamics in previous studies. METHODS: We investigated the viral decay pattern in 50 patients from the phase III clinical trial of 24-week clevudine therapy, who showed virological response and HBsAg decline. Immune effectors were added as a new compartment in the model equations. We determined some parameter values in the model using the non-linear least square minimization method. RESULTS: Median baseline viral load was 8.526 Log(10)copies/mL, and on-treatment viral load decline was 5.683 Log(10)copies/mL. The median half-life of free virus was 24.89 hours. The median half-life of infected hepatocytes was 7.39 days. The viral decay patterns were visualized as triphasic curves with decreasing slopes over time: fastest decay in the first phase; slowest in the third phase; the second phase in between. CONCLUSIONS: In the present study, mathematical modeling of hepatitis B in patients with virological response and HBsAg decline during 24-week antiviral therapy showed triphasic viral dynamics with direct introduction of immune effectors as a new compartment, which was thought to reflect the reduction of clearance rate of infected cells over time. This modeling method seems more appropriate to describe long-term viral dynamics compared to the biphasic model, and needs further validation.

Modelling and optimal control of immune response of renal transplant recipients.

We consider the increasingly important and highly complex immunological control problem: control of the dynamics of immunosuppression for organ transplant recipients. The goal in this problem is to maintain the delicate balance between over-suppression (where opportunistic latent viruses threaten the patient) and under-suppression (where rejection of the transplanted organ is probable). First, a mathematical model is formulated to describe the immune response to both viral infection and introduction of a donor kidney in a renal transplant recipient. Some numerical results are given to qualitatively validate and demonstrate that this initial model exhibits appropriate characteristics of primary infection and reactivation for immunosuppressed transplant recipients. In addition, we develop a computational framework for designing adaptive optimal treatment regimes with partial observations and low-frequency sampling, where the state estimates are obtained by solving a second deterministic optimal tracking problem. Numerical results are given to illustrate the feasibility of this method in obtaining optimal treatment regimes with a balance between under-suppression and over-suppression of the immune system.

Optimal control of chikungunya disease: larvae reduction, treatment and prevention.

Since the 1980s, there has been a worldwide re-emergence of vector-borne diseases including Malaria, Dengue, Yellow fever or, more recently, chikungunya. These viruses are arthropod-borne viruses (arboviruses) transmitted by arthropods like mosquitoes of Aedes genus. The nature of these arboviruses is complex since it conjugates human, environmental, biological and geographical factors. Recent researchs have suggested, in particular during the Reunion Island epidemic in 2006, that the transmission by Aedes albopictus (an Aedes genus specie) has been facilitated by genetic mutations of the virus and the vector capacity to adapt to non tropical regions. In this paper we formulate an optimal control problem, based on biological observations. Three main efforts are considered in order to limit the virus transmission. Indeed, there is no vaccine nor specific treatment against chikungunya, that is why the main measures to limit the impact of such epidemic have to be considered. Therefore, we look at time dependent breeding sites destruction, prevention and treatment efforts, for which optimal control theory is applied. Using analytical and numerical techniques, it is shown that there exist cost effective control efforts.

Free terminal time optimal control problem of an HIV model based on a conjugate gradient method.

The minimum duration of treatment periods and the optimal multidrug therapy for human immunodeficiency virus (HIV) type 1 infection are considered. We formulate an optimal tracking problem, attempting to drive the states of the model to a "healthy" steady state in which the viral load is low and the immune response is strong. We study an optimal time frame as well as HIV therapeutic strategies by analyzing the free terminal time optimal tracking control problem. The minimum duration of treatment periods and the optimal multidrug therapy are found by solving the corresponding optimality systems with the additional transversality condition for the terminal time. We demonstrate by numerical simulations that the optimal dynamic multidrug therapy can lead to the long-term control of HIV by the strong immune response after discontinuation of therapy.

Dynamic multidrug therapies for hiv: optimal and sti control approaches.

We formulate a dynamic mathematical model that describes the interaction of the immune system with the human immunodeficiency virus (HIV) and that permits drug "cocktail " therapies. We derive HIV therapeutic strategies by formulating and analyzing an optimal control problem using two types of dynamic treatments representing reverse transcriptase (RT) in hibitors and protease inhibitors (PIs). Continuous optimal therapies are found by solving the corresponding optimality systems. In addition, using ideas from dynamic programming, we formulate and derive suboptimal structured treatment interruptions (STI)in antiviral therapy that include drug-free periods of immune-mediated control of HIV. Our numerical results support a scenario in which STI therapies can lead to long-term control of HIV by the immune response system after discontinuation of therapy.