Generative data augmentation and automated optimization of convolutional neural networks for process monitoring.

Chemometric modeling for spectral data is considered a key technology in biopharmaceutical processing to realize real-time process control and release testing. Machine learning (ML) models have been shown to increase the accuracy of various spectral regression and classification tasks, remove challenging preprocessing steps for spectral data, and promise to improve the transferability of models when compared to commonly applied, linear methods. The training and optimization of ML models require large data sets which are not available in the context of biopharmaceutical processing. Generative methods to extend data sets with realistic in silico samples, so-called data augmentation, may provide the means to alleviate this challenge. In this study, we develop and implement a novel data augmentation method for generating in silico spectral data based on local estimation of pure component profiles for training convolutional neural network (CNN) models using four data sets. We simultaneously tune hyperparameters associated with data augmentation and the neural network architecture using Bayesian optimization. Finally, we compare the optimized CNN models with partial least-squares regression models (PLS) in terms of accuracy, robustness, and interpretability. The proposed data augmentation method is shown to produce highly realistic spectral data by adapting the estimates of the pure component profiles to the sampled concentration regimes. Augmenting CNNs with the in silico spectral data is shown to improve the prediction accuracy for the quantification of monoclonal antibody (mAb) size variants by up to 50% in comparison to single-response PLS models. Bayesian structure optimization suggests that multiple convolutional blocks are beneficial for model accuracy and enable transfer across different data sets. Model-agnostic feature importance methods and synthetic noise perturbation are used to directly compare the optimized CNNs with PLS models. This enables the identification of wavelength regions critical for model performance and suggests increased robustness against Gaussian white noise and wavelength shifts of the CNNs compared to the PLS models.

Monitoring of ultra- and diafiltration processes by Kalman-filtered Raman measurements.

Monitoring the protein concentration and buffer composition during the Ultrafiltration/Diafiltration (UF/DF) step enables the further automation of biopharmaceutical production and supports Real-time Release Testing (RTRT). Previously, in-line Ultraviolet (UV) and Infrared (IR) measurements have been used to successfully monitor the protein concentration over a large range. The progress of the diafiltration step has been monitored with density measurements and Infrared Spectroscopy (IR). Raman spectroscopy is capable of measuring both the protein and excipient concentration while being more robust and suitable for production measurements in comparison to Infrared Spectroscopy (IR). Regardless of the spectroscopic sensor used, the low concentration of excipients poses a challenge for the sensors. By combining sensor measurements with a semi-mechanistic model through an Extended Kalman Filter (EKF), the sensitivity to determine the progress of the diafiltration can be improved. In this study, Raman measurements are combined with an EKF for three case studies. The advantages of Kalman-filtered Raman measurements for excipient monitoring are shown in comparison to density measurements. Furthermore, Raman measurements showed a higher measurement speed in comparison to Variable Pathlength (VP) UV measurement at the trade-off of a slightly worse prediction accuracy for the protein concentration. However, the Raman-based protein concentration measurements relied mostly on an increase in the background signal during the process and not on proteinaceous features, which could pose a challenge due to the potential influence of batch variability on the background signal. Overall, the combination of Raman spectroscopy and EKF is a promising tool for monitoring the UF/DF step and enables process automation by using adaptive process control.

Comparison of UV- and Raman-based monitoring of the Protein A load phase and evaluation of data fusion by PLS models and CNNs.

A promising application of Process Analytical Technology to the downstream process of monoclonal antibodies (mAbs) is the monitoring of the Protein A load phase as its control promises economic benefits. Different spectroscopic techniques have been evaluated in literature with regard to the ability to quantify the mAb concentration in the column effluent. Raman and Ultraviolet (UV) spectroscopy are among the most promising techniques. In this study, both were investigated in an in-line setup and directly compared. The data of each sensor were analyzed independently with Partial-Least-Squares (PLS) models and Convolutional Neural Networks (CNNs) for regression. Furthermore, data fusion strategies were investigated by combining both sensors in hierarchical PLS models or in CNNs. Among the tested options, UV spectroscopy alone allowed for the most precise and accurate prediction of the mAb concentration. A Root Mean Square Error of Prediction (RMSEP) of 0.013 g L-1 was reached with the UV-based PLS model. The Raman-based PLS model reached an RMSEP of 0.232 g L-1 . The different data fusion techniques did not improve the prediction accuracy above the prediction accuracy of the UV-based PLS model. Data fusion by PLS models seems meritless when combining a very accurate sensor with a less accurate signal. Furthermore, the application of CNNs for UV and Raman spectra did not yield significant improvements in the prediction quality. For the presented application, linear regression techniques seem to be better suited compared with advanced nonlinear regression techniques, like, CNNs. In summary, the results support the application of UV spectroscopy and PLS modeling for future research and development activities aiming to implement spectroscopic real-time monitoring of the Protein A load phase.

A multisensor approach for improved protein A load phase monitoring by conductivity-based background subtraction of UV spectra.

Real-time monitoring and control of protein A capture steps by process analytical technologies (PATs) promises significant economic benefits due to the improved usage of the column's binding capacity, by eliminating time-consuming off-line analytics and costly resin lifetime studies, and enabling continuous production. The PAT method proposed in this study relies on ultraviolet (UV) spectroscopy with a dynamic background subtraction based on the leveling out of the conductivity signal. This point in time can be used to collect a reference spectrum for removing the majority of spectral contributions by process-related contaminants. The removal of the background spectrum facilitates chemometric model building and model accuracy. To demonstrate the benefits of this method, five different feedstocks from our industry partner were used to mix the load material for a case study. To our knowledge, such a large design space, which covers possible variations in upstream condition besides the product concentration, has not been disclosed yet. By applying the conductivity-based background subtraction, the root mean square error of prediction (RMSEP) of the partial least squares (PLS) model improved from 0.2080 to 0.0131 g L-1 . Finally, the potential of the background subtraction method was further evaluated for single wavelength-based predictions to facilitate implementation in production processes. An RMSEP of 0.0890 g L-1 with univariate linear regression was achieved, showing that by subtraction of the background better prediction accuracy is achieved then without subtraction and a PLS model. In summary, the developed background subtraction method is versatile, enables accurate prediction results, and is easily implemented into existing chromatography setups with typically already integrated sensors.

On the analysis of chromatographic biopharmaceutical data by curve resolution techniques in the framework of the area of feasible solutions.

Monitoring preparative protein chromatographic steps by in-line spectroscopic tools or fraction analytics results in medium or large sized data matrices. Multivariate Curve Resolution (MCR) serve to compute or to estimate the concentration values of the pure components only from these data matrices. However, MCR methods often suffer from an inherent solution ambiguity which underlies the factorization problem. The typical unimodality of the chromatographic profiles of pure components can support the chemometric analysis. Here we present the pure components estimation process within the framework of the area of feasible solutions, which is a systematic approach to represent the range of all possible solutions. The unimodality constraint in combination with Pareto optimization is shown to be an effective method for the pure component calculation. Applications are presented for chromatograms on a model protein mixture containing ribonuclease A, cytochrome c and lysozyme and on a two-dimensional chromatographic separation of a monoclonal antibody from its aggregate species. The root mean squared errors of the first case study are 0.0373, 0.0529 and 0.0380 g/L compared to traditional off-line analytics. The second case study illustrates the potential of recovering hidden components with MCR from off-line reference analytics.

A critical review of recent trends, and a future perspective of optical spectroscopy as PAT in biopharmaceutical downstream processing.

As competition in the biopharmaceutical market gets keener due to the market entry of biosimilars, process analytical technologies (PATs) play an important role for process automation and cost reduction. This article will give a general overview and address the recent innovations and applications of spectroscopic methods as PAT tools in the downstream processing of biologics. As data analysis strategies are a crucial part of PAT, the review discusses frequently used data analysis techniques and addresses data fusion methodologies as the combination of several sensors is moving forward in the field. The last chapter will give an outlook on the application of spectroscopic methods in combination with chemometrics and model predictive control (MPC) for downstream processes. Graphical abstract.

Multi-attribute PAT for UF/DF of Proteins-Monitoring Concentration, particle sizes, and Buffer Exchange.

Ultrafiltration/diafiltration (UF/DF) plays an important role in the manufacturing of biopharmaceuticals. Monitoring critical process parameters and quality attributes by process analytical technology (PAT) during those steps can facilitate process development and assure consistent quality in production processes. In this study, a lab-scale cross-flow filtration (CFF) device was equipped with a variable pathlength (VP) ultraviolet and visible (UV/Vis) spectrometer, a light scattering photometer, and a liquid density sensor (microLDS). Based on the measured signals, the protein concentration, buffer exchange, apparent molecular weight, and hydrodynamic radius were monitored. The setup was tested in three case studies. First, lysozyme was used in an UF/DF run to show the comparability of on-line and off-line measurements. The corresponding correlation coefficients exceeded 0.97. Next, urea-induced changes in protein size of glucose oxidase (GOx) were monitored during two DF steps. Here, correlation coefficients were ≥ 0.92 for static light scattering (SLS) and dynamic light scattering (DLS). The correlation coefficient for the protein concentration was 0.82, possibly due to time-dependent protein precipitation. Finally, a case study was conducted with a monoclonal antibody (mAb) to show the full potential of this setup. Again, off-line and on-line measurements were in good agreement with all correlation coefficients exceeding 0.92. The protein concentration could be monitored in-line in a large range from 3 to 120 g L- 1. A buffer-dependent increase in apparent molecular weight of the mAb was observed during DF, providing interesting supplemental information for process development and stability assessment. In summary, the developed setup provides a powerful testing system for evaluating different UF/DF processes and may be a good starting point to develop process control strategies. Graphical Abstract Piping and instrumentation diagram of the experimental setup and data generated by the different sensors. A VP UV/Vis spectrometer (FlowVPE, yellow) measures the protein concentration. From the data of the light scattering photometer (Zetasizer, green) in the on-line measurement loop, the apparant molecular weight and z-average are calculated. The density sensor (microLDS) measures density and viscosity of the fluid in the on-line loop.

Fourier-transform infrared spectroscopy as a process analytical technology for near real time in-line estimation of the degree of PEGylation in chromatography.

PEGylation of biological macromolecules is a well-established strategy to increase circulation half-life, decrease renal clearance and improve biocompatibility. PEGylation is a process in which polyethylene glycol (PEG) is covalently attached to a target molecule. The production of PEGylated biopharmaceuticals is usually executed by first producing and purifying the base molecule followed by the PEGylation reaction and purification of the modified molecule. Most PEGylated pharmaceuticals are produced by random PEGylation in batch mode and need to be purified as mainly the mono-PEGylated form is the desired drug product. In this work we propose a method to estimate the degree of PEGylation (DOP) of modified protein eluting from a chromatography column in near real-time. extended multiplicative signal correction (EMSC) is used in conjunction with asymmetric least squares (aaLS) to alleviate the influence of a salt gradient during ion exchange chromatography (IEX) on the spectral data. To convert the raw data obtained from spectral data to the actual DOP additional information obtained from off-line measurements is utilized. Once the signal correction is applied to in-line spectral data the DOP can be estimated without further use of off-line analytics. As the prerequisites for the application of this method are relatively easy to obtain it may also find use to speed up process development.

Process monitoring of virus-like particle reassembly by diafiltration with UV/Vis spectroscopy and light scattering.

Virus-like particles (VLPs) have shown great potential as biopharmaceuticals in the market and in clinics. Nonenveloped, in vivo assembled VLPs are typically disassembled and reassembled in vitro to improve particle stability, homogeneity, and immunogenicity. At the industrial scale, cross-flow filtration (CFF) is the method of choice for performing reassembly by diafiltration. Here, we developed an experimental CFF setup with an on-line measurement loop for the implementation of process analytical technology (PAT). The measurement loop included an ultraviolet and visible (UV/Vis) spectrometer as well as a light scattering photometer. These sensors allowed for monitoring protein concentration, protein tertiary structure, and protein quaternary structure. The experimental setup was tested with three Hepatitis B core Antigen (HBcAg) variants. With each variant, three reassembly processes were performed at different transmembrane pressures (TMPs). While light scattering provided information on the assembly progress, UV/Vis allowed for monitoring the protein concentration and the rate of VLP assembly based on the microenvironment of Tyrosine-132. VLP formation was verified by off-line dynamic light scattering (DLS) and transmission electron microscopy (TEM). Furthermore, the experimental results provided evidence of aggregate-related assembly inhibition and showed that off-line size-exclusion chromatography does not provide a complete picture of the particle content. Finally, a Partial-Least Squares (PLS) model was calibrated to predict VLP concentrations in the process solution. Q2 values of 0.947-0.984 were reached for the three HBcAg variants. In summary, the proposed experimental setup provides a powerful platform for developing and monitoring VLP reassembly steps by CFF.

Factorization of preparative protein chromatograms with hard-constraint multivariate curve resolution and second-derivative pretreatment.

Current biopharmaceutical production heavily relies on chromatography for protein purification. Recently, research has intensified towards finding suitable solutions to monitoring the chromatographic steps by multivariate spectroscopic sensors. Here, hard-constraint multivariate curve resolution (MCR) was investigated as a calibration-free method for factorizing bilinear preparative protein chromatograms into concentrations and spectra. Protein elutions were assumed to follow exponentially modified Gaussian (EMG) curves. In three case studies, MCR was applied to chromatograms of second-derivative ultraviolet and visible (UV-vis) spectra. The three case studies consisted of the separation of a ternary mixture (ribonuclease A, cytochrome c, and lysozyme), multiple binary chromatography runs of cytochrome c and lysozyme, and the separation of an antibody-drug conjugate (ADC) from unconjugated immunoglobulin G (IgG). In all case studies, good estimates of the elution curves were obtained. R2 values compared to off-line analytics exceeded 0.90. The estimated spectra allowed for protein identification based on a protein spectral library. In summary, MCR was shown to be well able to factorize protein chromatograms without prior calibration. The method may thus substantially simplify analysis of multivariate protein chromatograms with multiple co-eluting species. It may be especially useful in process development.

Monitoring of antibody-drug conjugation reactions with UV/Vis spectroscopy.

The conjugation reaction of monoclonal antibodies (mAbs) with small-molecule drugs is a central step during production of antibody-drug conjugates (ADCs). The ability to monitor this step in real time can be advantageous for process understanding and control. Here, we propose a method based on UV/Vis spectroscopy in conjunction with partial least squares (PLS) regression for non-invasive monitoring of conjugation reactions. In experiments, the method was applied to conjugation reactions with two surrogate drugs in microplate format as well as at 20 ml scale. All calibrated PLS models performed well in cross-validation (Q2>0.975 for all models). In microplate format, the PLS models were furthermore successfully validated with an independent prediction set (Rpred2=0.9770 resp. 0.8940). In summary, the proposed method provides a quick and easily implementable tool for reaction monitoring of ADC conjugation reactions and may in the future support the implementation of Process Analytical Technologies (PAT).

In-line Fourier-transform infrared spectroscopy as a versatile process analytical technology for preparative protein chromatography.

Fourier-transform infrared spectroscopy (FTIR) is a well-established spectroscopic method in the analysis of small molecules and protein secondary structure. However, FTIR is not commonly applied for in-line monitoring of protein chromatography. Here, the potential of in-line FTIR as a process analytical technology (PAT) in downstream processing was investigated in three case studies addressing the limits of currently applied spectroscopic PAT methods. A first case study exploited the secondary structural differences of monoclonal antibodies (mAbs) and lysozyme to selectively quantify the two proteins with partial least squares regression (PLS) giving root mean square errors of cross validation (RMSECV) of 2.42 g/l and 1.67 g/l, respectively. The corresponding Q2 values are 0.92 and, respectively, 0.99, indicating robust models in the calibration range. Second, a process separating lysozyme and PEGylated lysozyme species was monitored giving an estimate of the PEGylation degree of currently eluting species with RMSECV of 2.35 g/l for lysozyme and 1.24 g/l for PEG with Q2 of 0.96 and 0.94, respectively. Finally, Triton X-100 was added to a feed of lysozyme as a typical process-related impurity. It was shown that the species could be selectively quantified from the FTIR 3D field without PLS calibration. In summary, the proposed PAT tool has the potential to be used as a versatile option for monitoring protein chromatography. It may help to achieve a more complete implementation of the PAT initiative by mitigating limitations of currently used techniques.

Advances in downstream processing of biologics - Spectroscopy: An emerging process analytical technology.

Process analytical technologies (PAT) for the manufacturing of biologics have drawn increased interest in the last decade. Besides being encouraged by the Food and Drug Administration's (FDA's) PAT initiative, PAT promises to improve process understanding, reduce overall production costs and help to implement continuous manufacturing. This article focuses on spectroscopic tools for PAT in downstream processing (DSP). Recent advances and future perspectives will be reviewed. In order to exploit the full potential of gathered data, chemometric tools are widely used for the evaluation of complex spectroscopic information. Thus, an introduction into the field will be given.

Real-time monitoring and control of the load phase of a protein A capture step.

The load phase in preparative Protein A capture steps is commonly not controlled in real-time. The load volume is generally based on an offline quantification of the monoclonal antibody (mAb) prior to loading and on a conservative column capacity determined by resin-life time studies. While this results in a reduced productivity in batch mode, the bottleneck of suitable real-time analytics has to be overcome in order to enable continuous mAb purification. In this study, Partial Least Squares Regression (PLS) modeling on UV/Vis absorption spectra was applied to quantify mAb in the effluent of a Protein A capture step during the load phase. A PLS model based on several breakthrough curves with variable mAb titers in the HCCF was successfully calibrated. The PLS model predicted the mAb concentrations in the effluent of a validation experiment with a root mean square error (RMSE) of 0.06 mg/mL. The information was applied to automatically terminate the load phase, when a product breakthrough of 1.5 mg/mL was reached. In a second part of the study, the sensitivity of the method was further increased by only considering small mAb concentrations in the calibration and by subtracting an impurity background signal. The resulting PLS model exhibited a RMSE of prediction of 0.01 mg/mL and was successfully applied to terminate the load phase, when a product breakthrough of 0.15 mg/mL was achieved. The proposed method has hence potential for the real-time monitoring and control of capture steps at large scale production. This might enhance the resin capacity utilization, eliminate time-consuming offline analytics, and contribute to the realization of continuous processing. Biotechnol. Bioeng. 2017;114: 368-373. © 2016 The Authors. Biotechnology and Bioengineering published by Wiley Periodicals, Inc.

Combined Yamamoto approach for simultaneous estimation of adsorption isotherm and kinetic parameters in ion-exchange chromatography.

Application of model-based design is appealing to support the development of protein chromatography in the biopharmaceutical industry. However, the required efforts for parameter estimation are frequently perceived as time-consuming and expensive. In order to speed-up this work, a new parameter estimation approach for modelling ion-exchange chromatography in linear conditions was developed. It aims at reducing the time and protein demand for the model calibration. The method combines the estimation of kinetic and thermodynamic parameters based on the simultaneous variation of the gradient slope and the residence time in a set of five linear gradient elutions. The parameters are estimated from a Yamamoto plot and a gradient-adjusted Van Deemter plot. The combined approach increases the information extracted per experiment compared to the individual methods. As a proof of concept, the combined approach was successfully applied for a monoclonal antibody on a cation-exchanger and for a Fc-fusion protein on an anion-exchange resin. The individual parameter estimations for the mAb confirmed that the new approach maintained the accuracy of the usual Yamamoto and Van Deemter plots. In the second case, offline size-exclusion chromatography was performed in order to estimate the thermodynamic parameters of an impurity (high molecular weight species) simultaneously with the main product. Finally, the parameters obtained from the combined approach were used in a lumped kinetic model to simulate the chromatography runs. The simulated chromatograms obtained for a wide range of gradient lengths and residence times showed only small deviations compared to the experimental data.