Deep hybrid modeling of a HEK293 process: Combining long short-term memory networks with first principles equations.

The combination of physical equations with deep learning is becoming a promising methodology for bioprocess digitalization. In this paper, we investigate for the first time the combination of long short-term memory (LSTM) networks with first principles equations in a hybrid workflow to describe human embryonic kidney 293 (HEK293) culture dynamics. Experimental data of 27 extracellular state variables in 20 fed-batch HEK293 cultures were collected in a parallel high throughput 250 mL cultivation system in an industrial process development setting. The adaptive moment estimation method with stochastic regularization and cross-validation were employed for deep learning. A total of 784 hybrid models with varying deep neural network architectures, depths, layers sizes and node activation functions were compared. In most scenarios, hybrid LSTM models outperformed classical hybrid Feedforward Neural Network (FFNN) models in terms of training and testing error. Hybrid LSTM models revealed to be less sensitive to data resampling than FFNN hybrid models. As disadvantages, Hybrid LSTM models are in general more complex (higher number of parameters) and have a higher computation cost than FFNN hybrid models. The hybrid model with the highest prediction accuracy consisted in a LSTM network with seven internal states connected in series with dynamic material balance equations. This hybrid model correctly predicted the dynamics of the 27 state variables (R2 = 0.93 in the test data set), including biomass, key substrates, amino acids and metabolic by-products for around 10 cultivation days.

Hybrid deep modeling of a CHO-K1 fed-batch process: combining first-principles with deep neural networks.

Introduction: Hybrid modeling combining First-Principles with machine learning is becoming a pivotal methodology for Biopharma 4.0 enactment. Chinese Hamster Ovary (CHO) cells, being the workhorse for industrial glycoproteins production, have been the object of several hybrid modeling studies. Most previous studies pursued a shallow hybrid modeling approach based on three-layered Feedforward Neural Networks (FFNNs) combined with macroscopic material balance equations. Only recently, the hybrid modeling field is incorporating deep learning into its framework with significant gains in descriptive and predictive power. Methods: This study compares, for the first time, deep and shallow hybrid modeling in a CHO process development context. Data of 24 fed-batch cultivations of a CHO-K1 cell line expressing a target glycoprotein, comprising 30 measured state variables over time, were used to compare both methodologies. Hybrid models with varying FFNN depths (3-5 layers) were systematically compared using two training methodologies. The classical training is based on the Levenberg-Marquardt algorithm, indirect sensitivity equations and cross-validation. The deep learning is based on the Adaptive Moment Estimation Method (ADAM), stochastic regularization and semidirect sensitivity equations. Results and conclusion: The results point to a systematic generalization improvement of deep hybrid models over shallow hybrid models. Overall, the training and testing errors decreased by 14.0% and 23.6% respectively when applying the deep methodology. The Central Processing Unit (CPU) time for training the deep hybrid model increased by 31.6% mainly due to the higher FFNN complexity. The final deep hybrid model is shown to predict the dynamics of the 30 state variables within the error bounds in every test experiment. Notably, the deep hybrid model could predict the metabolic shifts in key metabolites (e.g., lactate, ammonium, glutamine and glutamate) in the test experiments. We expect deep hybrid modeling to accelerate the deployment of high-fidelity digital twins in the biopharma sector in the near future.

Genome-scale modeling of Chinese hamster ovary cells by hybrid semi-parametric flux balance analysis.

Flux balance analysis (FBA) is currently the standard method to compute metabolic fluxes in genome-scale networks. Several FBA extensions employing diverse objective functions and/or constraints have been published. Here we propose a hybrid semi-parametric FBA extension that combines mechanistic-level constraints (parametric) with empirical constraints (non-parametric) in the same linear program. A CHO dataset with 27 measured exchange fluxes obtained from 21 reactor experiments served to evaluate the method. The mechanistic constraints were deduced from a reduced CHO-K1 genome-scale network with 686 metabolites, 788 reactions and 210 degrees of freedom. The non-parametric constraints were obtained by principal component analysis of the flux dataset. The two types of constraints were integrated in the same linear program showing comparable computational cost to standard FBA. The hybrid FBA is shown to significantly improve the specific growth rate prediction under different constraints scenarios. A metabolically efficient cell growth feed targeting minimal byproducts accumulation was designed by hybrid FBA. It is concluded that integrating parametric and nonparametric constraints in the same linear program may be an efficient approach to reduce the solution space and to improve the predictive power of FBA methods when critical mechanistic information is missing.

Generic Chemometric Models for Metabolite Concentration Prediction Based on Raman Spectra.

Chemometric models for on-line process monitoring have become well established in pharmaceutical bioprocesses. The main drawback is the required calibration effort and the inflexibility regarding system or process changes. So, a recalibration is necessary whenever the process or the setup changes even slightly. With a large and diverse Raman dataset, however, it was possible to generate generic partial least squares regression models to reliably predict the concentrations of important metabolic compounds, such as glucose-, lactate-, and glutamine-indifferent CHO cell cultivations. The data for calibration were collected from various cell cultures from different sites in different companies using different Raman spectrophotometers. In testing, the developed "generic" models were capable of predicting the concentrations of said compounds from a dilution series in FMX-8 mod medium, as well as from an independent CHO cell culture. These spectra were taken with a completely different setup and with different Raman spectrometers, demonstrating the model flexibility. The prediction errors for the tests were mostly in an acceptable range (<10% relative error). This demonstrates that, under the right circumstances and by choosing the calibration data carefully, it is possible to create generic and reliable chemometric models that are transferrable from one process to another without recalibration.

Time Integrated Flux Analysis: Exploiting the Concentration Measurements Directly for Cost-Effective Metabolic Network Flux Analysis.

BACKGROUND: Flux analyses, such as Metabolic Flux Analysis (MFA), Flux Balance Analysis (FBA), Flux Variability Analysis (FVA) or similar methods, can provide insights into the cellular metabolism, especially in combination with experimental data. The most common integration of extracellular concentration data requires the estimation of the specific fluxes (/rates) from the measured concentrations. This is a time-consuming, mathematically ill-conditioned inverse problem, raising high requirements for the quality and quantity of data. METHOD: In this contribution, a time integrated flux analysis approach is proposed which avoids the error-prone estimation of specific flux values. The approach is adopted for a Metabolic time integrated Flux Analysis and (sparse) time integrated Flux Balance/Variability Analysis. The proposed approach is applied to three case studies: (1) a simulated bioprocess case studying the impact of the number of samples (experimental points) and measurements' noise on the performance; (2) a simulation case to understand the impact of network redundancies and reaction irreversibility; and (3) an experimental bioprocess case study, showing its relevance for practical applications. RESULTS: It is observed that this method can successfully estimate the time integrated flux values, even with relatively low numbers of samples and significant noise levels. In addition, the method allows the integration of additional constraints (e.g., bounds on the estimated concentrations) and since it eliminates the need for estimating fluxes from measured concentrations, it significantly reduces the workload while providing about the same level of insight into the metabolism as classic flux analysis methods.

Engineering the hard-soft tissue interface with random-to-aligned nanofiber scaffolds.

Tendon injuries can be difficult to heal and have high rates of relapse due to stress concentrations caused by scar formation and the sutures used in surgical repair. Regeneration of the tendon/ligament-to-bone interface is critical to provide functional graft integration after injury. The objective of this study is to recreate the tendon-to-bone interface using a gradient scaffold which is fabricated by a one-station electrospinning process. Two cell phenotypes were grown on a poly-ε-caprolactone nanofiber scaffold which possesses a gradual transition from random to aligned nanofiber patterns. We assessed the effects of the polymer concentration, tip-to-collector distance, and electrospinning time on the microfiber diameter and density. Osteosarcoma and fibroblast cells were seeded on the random and aligned sections of scaffolds, respectively. A random-to-aligned cocultured tissue interface which mimicked the native transition in composition of enthesis was created after 96 h culturing. The results showed that the microstructure gradient influenced the cell morphology, tissue topology, and promoted enthesis formation. This study demonstrates a heterogeneous nanofiber scaffold strategy for interfacial tissue regeneration. It provides a potential solution for mimicking transitional interface between distinct tissues, and can be further developed as a heterogeneous cellular composition platform to facilitate the formation of multi-tissue complex systems.

Cannabis use correlates with schizotypal personality traits in healthy students.

The literature suggests that cannabis use and schizotypal traits both constitute risk factors for the later development of schizophrenia. However, their interrelationships remain to be evaluated. The present study examined the association between cannabis use and schizotypal traits in 232 healthy students who ranged in age from 18 to 25 years. All the students had completed the Schizotypal Personality Questionnaire and four of the Chapman Psychosis Proneness Scales: the Magical Ideation Scale; the Perceptual Aberration Scale; the Revised Physical Anhedonia Scale; and the Revised Social Anhedonia Scale. Subjects were divided into three groups according to cannabis use typology: those who had never used cannabis, those who were past or occasional users, and those who were regular users. Higher scores on the Schizotypal Personality Questionnaire and the Magical Ideation Scale characterized the regular and past or occasional users compared with those who had never used cannabis. The co-occurrence of cannabis use and schizotypal traits appeared to be independent of anxiety and depression dimensions. These data suggest that cannabis use and schizotypal traits have to be jointly considered in further longitudinal studies of schizophrenia risk factors.