A reinforcement learning-based hybrid modeling framework for bioprocess kinetics identification.

Constructing predictive models to simulate complex bioprocess dynamics, particularly time-varying (i.e., parameters varying over time) and history-dependent (i.e., current kinetics dependent on historical culture conditions) behavior, has been a longstanding research challenge. Current advances in hybrid modeling offer a solution to this by integrating kinetic models with data-driven techniques. This article proposes a novel two-step framework: first (i) speculate and combine several possible kinetic model structures sourced from process and phenomenological knowledge, then (ii) identify the most likely kinetic model structure and its parameter values using model-free Reinforcement Learning (RL). Specifically, Step 1 collates feasible history-dependent model structures, then Step 2 uses RL to simultaneously identify the correct model structure and the time-varying parameter trajectories. To demonstrate the performance of this framework, a range of in-silico case studies were carried out. The results show that the proposed framework can efficiently construct high-fidelity models to quantify both time-varying and history-dependent kinetic behaviors while minimizing the risks of over-parametrization and over-fitting. Finally, the primary advantages of the proposed framework and its limitation were thoroughly discussed in comparison to other existing hybrid modeling and model structure identification techniques, highlighting the potential of this framework for general bioprocess modeling.

A transfer learning approach for predictive modeling of bioprocesses using small data.

Predictive modeling of new biochemical systems with small data is a great challenge. To fill this gap, transfer learning, a subdomain of machine learning that serves to transfer knowledge from a generalized model to a more domain-specific model, provides a promising solution. While transfer learning has been used in natural language processing, image analysis, and chemical engineering fault detection, its application within biochemical engineering has not been systematically explored. In this study, we demonstrated the benefits of transfer learning when applied to predict dynamic behaviors of new biochemical processes. Two different case studies were presented to investigate the accuracy, reliability, and advantage of this innovative modeling approach. We thoroughly discussed the different transfer learning strategies and the effects of topology on transfer learning, comparing the performance of the transfer learning models against benchmark kinetic and data-driven models. Furthermore, strong connections between the underlying process mechanism and the transfer learning model's optimal structure were highlighted, suggesting the interpretability of transfer learning to enable more accurate prediction than a naive data-driven modeling approach. Therefore, this study shows a novel approach to effectively combining data from different resources for bioprocess simulation.