In this paper, Long Short-Term Memory (LSTM) networks and multilayered feedforward neural networks (FFNNs) were combined with first principles equations in a hybrid workflow to describe human embryonic kidney 293 (HEK293) culture dynamics. Experimental data of 27 extracellular state variables in 20 fed-batch HEK293 cultures were collected in a parallel high throughput 250 mL cultivation system. The adaptive moment estimation method (ADAM) with stochastic regularization and cross-validation were employed for deep learning. A total of 784 hybrid models with varying deep neural network (DNN) architectures, depths, layers sizes and node activation functions were compared. In most scenarios, hybrid LSTM models outperformed hybrid FFNN models in terms of training and testing error. Hybrid LSTM models revealed to be less sensitive to data resampling than FFNN hybrid models. As disadvantages, Hybrid LSTM models are in general more complex (higher number of parameters) and have a higher computation cost than FFNN hybrid models. The hybrid model with the highest prediction accuracy consisted in a LSTM network with 7 internal states connected in series with dynamic material balance equations. This hybrid model correctly predicted the dynamics of the 27 state variables (R 2=0.93 in the test data set), including biomass, key substrates, amino acids and metabolic by-products for around 10 cultivation days.

Due to the complexity of biological processes, developing model-based strategies for monitoring, optimization and control is nontrivial. Hybrid neural models, combining mechanistic modeling with artificial neural networks, have been reported as powerful tools for bioprocess applications. In this paper, a systematic literature review is presented focused on the application of hybrid neural models to bioprocesses by Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) over the last 30 years. This analysis showed that hybrid neural modeling has covered a wide range of microbial processes, animal cells, mixed microbial cultures, and enzyme biocatalysis. Hybrid neural models have been mainly applied for predictive modeling/process analysis, process monitoring/software sensors, open- and closed-loop control, batch-to-batch control, model predictive control, intensified design of experiments, process analytical technology, quality-by-design, and more recently, digital twins. Hybrid modeling experienced a decline in the number of publications after a peak in 2004 and is now surging again. A “model scale” research gap was identified, which will likely narrow by a better integration with deep learning and systems biology in the near future. The biopharma sector is currently a major driver but applications to biologics quality attributes (e.g. glycosylation), new modalities and downstream unit operations are significant research gaps.

Due to the complexity of biological transformations, developing model-based strategies to optimize and control bioprocesses is nontrivial. Hybrid models combining a mechanistic description of known influential factors with machine learning to infer the missing influential factors from data have been reported as powerful tools for bioprocesses applications. The artificial neural network is one of the most popular machine learning methods in this case. This paper presents a systematic literature review by computerized search across two databases: Scopus and Web of Science, and backward citation. The PRISMA method was applied to selecting the publications and 159 research articles were categorized as hybrid model applications to bioprocesses problems. It was found that hybrid models were mainly applied in upstream operation steps with a predominance of bioreaction steps. In downstream processing, chromatography appeared as a more recent research topic, with a relatively small number of publications. Furthermore, holistic hybrid modeling applications that integrate data and knowledge from several bioprocess steps will likely emerge in the future, enabling better optimization and control of the bioprocess’s platform. The combination of other machine learning methods with the hybrid neural network model is another opportunity that could improve the output of the model.

Flux balance analysis is currently the standard method to compute metabolic fluxes in genome-scale networks. Several variations employing diverse objective functions and/or constraints have been published. Here we propose a hybrid semi-parametric version of flux balance analysis that combines mechanistic-level constraints (parametric) with empirical constraints (non-parametric), at the genome-scale. A CHO dataset with 27 measured exchange fluxes obtained from 21 reactor experiments served to evaluate the method. The reduced CHO genome-scale model comprehended 686 metabolites, 788 reactions and 210 degrees of freedom. The experimental flux dataset could be compressed to 6 principal components retaining 93.7% of explained variance. The conjugation of both types of constraints is coded as a linear program with comparable computational cost as standard flux balance analysis. The hybrid flux balance analysis showed a significant reduction in the specific growth rate prediction error in comparison to the non-hybrid version. The hybrid method was eventually used to design a metabolically efficient feed to extend cell expansion from 9.87 Mcell/ml to 22.48 Mcell/ml at the point of induction with minimal accumulation of byproducts. It is concluded that the predictive advantage of the hybrid method resulted from the statistical abstraction of regulatory mechanisms, which were absent in the standard flux balance analysis.