Microgels are cross-linked, colloidal polymer networks with great potential for stimuli-response release in drug-delivery applications, as their small size allows them to pass human cell boundaries. For applications with specified requirements regarding size, producing tailored microgels in a continuous flow reactor is advantageous because the microgel properties can be controlled tightly. However, no fully-specified mechanistic models are available for continuous microgel synthesis, as the physical properties of the included components are only studied partly. To address this gap and accelerate tailor-made microgel development, we propose a data-driven optimization in a hardware-in-the-loop approach to efficiently synthesize microgels with defined sizes. We optimize the synthesis regarding conflicting objectives (maximum production efficiency, minimum energy consumption, and the desired microgel radius) by applying Bayesian optimization via the solver “Thompson sampling efficient multi-objective optimization” (TS-EMO). We validate the optimization using the deterministic global solver “McCormick-based Algorithm for mixed-integer Nonlinear Global Optimization” (MAiNGO) and verify three computed Pareto optimal solutions via experiments. The proposed framework can be applied to other desired microgel properties and reactor setups and has the potential of efficient development by minimizing number of experiments and modeling effort needed.

Computational Fluid Dynamics (CFD) is a powerful tool which can help with the geometry optimization of continuous milli-scale reactors, which often are highly complex devices. Attempting to perform this optimization by manually modifying and testing geometry configurations can however be tedious and computationally inefficient. Addressing this problem, we present a framework in which the CFD software COMSOL Multiphysics is coupled with the multi-objective Bayesian Optimization algorithm TSEMO (Thompson sampling efficient multiobjective optimization), implemented in MATLAB. The mixing element geometry of a Miprowa Lab millireactor is parameterized, and the framework automatically executes CFD simulations to minimize areas of stagnating flow and maximize the mixing performance. The framework is able to find Pareto-optimal reactor variations, and can easily be adapted for other devices and objectives.

Ionic liquids (ILs) are important solvents for sustainable processes and predicting activity coefficients (ACs) of solutes in ILs is needed. Recently, matrix completion methods (MCMs), transformers, and graph neural networks (GNNs) have shown high accuracy in predicting ACs of binary mixtures, superior to well-established models, e.g., COSMO-RS and UNIFAC. GNNs are particularly promising here as they learn a molecular graph-to-property relationship without pretraining, typically required for transformers, and are, unlike MCMs, applicable to molecules not included in training. For ILs, however, GNN applications are currently missing. Herein, we present a GNN to predict temperature-dependent infinite dilution ACs of solutes in ILs. We train the GNN on a database including more than 40,000 AC values and compare it to a state-of-the-art MCM. The GNN and MCM achieve similar high prediction performance, with the GNN additionally enabling high-quality predictions for ACs of solutions that contain ILs and solutes not considered during training.

Graph neural networks (GNNs) are emerging in chemical engineering for the end-to-end learning of physicochemical properties based on molecular graphs. A key element of GNNs is the pooling function which combines atom feature vectors into molecular fingerprints. Most previous works use a standard pooling function to predict a variety of properties. However, unsuitable pooling functions can lead to unphysical GNNs that poorly generalize. We compare and select meaningful GNN pooling methods based on physical knowledge about the learned properties. The impact of physical pooling functions is demonstrated with molecular properties calculated from quantum mechanical computations. We also compare our results to the recent set2set pooling approach. We recommend using sum pooling for the prediction of properties that depend on molecular size and compare pooling functions for properties that are molecular size-independent. Overall, we show that the use of physical pooling functions significantly enhances generalization.

Fuels with high-knock resistance enable modern spark-ignition engines to achieve high efficiency and thus low CO₂ emissions. Identification of molecules with desired autoignition properties indicated by a high research octane number and a high octane sensitivity is therefore of great practical relevance and can be supported by computer-aided molecular design (CAMD). Recent developments in the field of graph machine learning (graph-ML) provide novel, promising tools for CAMD. We propose a modular graph-ML CAMD framework that integrates generative graph-ML models with graph neural networks and optimization, enabling the design of molecules with desired ignition properties in a continuous molecular space. In particular, we explore the potential of Bayesian optimization and genetic algorithms in combination with generative graph-ML models. The graph-ML CAMD framework successfully identifies well-established high-octane components. It also suggests new candidates, one of which we experimentally investigate and use to illustrate the need for further autoignition training data.

Hybrid modelling, i.e., the combination of data-driven modelling with mechanistic model components, reduces the data demand and enables extrapolation of data-driven models. However, building, training and evaluation of hybrid models is cumbersome with current frameworks. We developed HybridML, an open-source modeling platform, in which hybrid models can be trained, i.e., combinations of artificial neural networks, arithmetic expressions, and differential equations. We employ TensorFlow for artificial neural network training and Casadi to integrate ordinary differential equations and provide gradients of differential model equations enabling continuous time representations. HybridML provides also a JSON interface for the model development. We apply HybridML to an industrial case study, in which the trained model is used to predict drug concentrations over time, based on physiological information about the patients. To demonstrate its versatility, we also present a nonlinear application, where HybridML is used to model the spread of the COVID-19 pandemic in German federal states based on the state's socio-economic attributes.

Road transportation needs to abandon fossil fuels. One promising alternative are renewable fuels for internal combustion engines. We consider three competing types of spark-ignition engines, i.e., conventional spark-ignition engines (CSIEs), flexible fuel vehicle engines (FFVEs), and ultra-high efficiency engines (UHEEs), which all have different fuel requirements. To determine which engine-fuel combination is optimal regarding fuel production cost and global warming impact (GWI), we apply our integrated fuel and process design method [König, et al. 2020. Comput. Chem. Eng.]. Specifically, we consider 47 pre-screened fuel species, their selective production routes from renewable resources, and a surrogate for optional blending of fossil gasoline. The designed FFVE (UHEE) fuels reduce GWI by up to 87% (84%) compared to fossil gasoline. In contrast, optimal CSIE fuels only achieve up to 60% GWI reduction and only at higher cost. The superior production performance of selectively-produced UHEE and FFVE fuels motivates replacement of today's CSIE technology.

The transformation of the chemical industry to renewable energy and feedstock supply requires new paradigms for the design of flexible plants, (bio-)catalysts, and functional materials. Recent breakthroughs in machine learning (ML) provide unique opportunities, but only joint interdisciplinary research between the ML and chemical engineering (CE) communities will unfold the full potential. We identify six challenges that will open new methods for CE and formulate new types of problems for ML: (1) optimal decision making, (2) introducing and enforcing physics in ML, (3) information and knowledge representation, (4) heterogeneity of data, (5) safety and trust in ML applications, and (6) creativity. Under the umbrella of these challenges, we discuss perspectives for future interdisciplinary research that will enable the transformation of CE.

Data-driven models are becoming increasingly popular in engineering, on their own or in combination with mechanistic models. Commonly, the trained models are subsequently used in model-based optimization of design and/or operation of processes. Thus, it is critical to ensure that data-driven models are not evaluated outside their validity domain during process optimization. We propose a method to learn this validity domain and encode it as constraints in process optimization. We first perform a topological data analysis using persistent homology identifying potential holes or separated clusters in the training data. In case clusters or holes are identified, we train a one-class classifier, i.e., a one-class support vector machine, on the training data domain and encode it as constraints in the subsequent process optimization. Otherwise, we construct the convex hull of the data and encode it as constraints. We finally perform deterministic global process optimization with the data-driven models subject to their respective validity constraints. To ensure computational tractability, we develop a reduced-space formulation for trained one-class support vector machines and show that our formulation outperforms common full-space formulations by a factor of over 3000, making it a viable tool for engineering applications. The method is ready-to-use and available open-source as part of our MeLOn toolbox (https://git.rwth-aachen.de/avt.svt/public/MeLOn).

Gaussian processes (Kriging) are interpolating data-driven models that are frequently applied in various disciplines. Often, Gaussian processes are trained on datasets and are subsequently embedded as surrogate models in optimization problems. These optimization problems are nonconvex and global optimization is desired. However, previous literature observed computational burdens limiting deterministic global optimization to Gaussian processes trained on few data points. We propose a reduced-space formulation for deterministic global optimization with trained Gaussian processes embedded. For optimization, the branch-and-bound solver branches only on the free variables and McCormick relaxations are propagated through explicit Gaussian process models. The approach also leads to significantly smaller and computationally cheaper subproblems for lower and upper bounding. To further accelerate convergence, we derive envelopes of common covariance functions for GPs and tight relaxations of acquisition functions used in Bayesian optimization including expected improvement, probability of improvement, and lower confidence bound. In total, we reduce computational time by orders of magnitude compared to state-of-the-art methods, thus overcoming previous computational burdens. We demonstrate the performance and scaling of the proposed method and apply it to Bayesian optimization with global optimization of the acquisition function and chance-constrained programming. The Gaussian process models, acquisition functions, and training scripts are available open-source within the “MeLOn—MachineLearning Models for Optimization” toolbox (https://git.rwth-aachen.de/avt.svt/public/MeLOn).

We propose an algorithm for scheduling subject to time-variable electricity prices using nonlinear process models that enables long planning horizons with fine discretizations. The algorithm relies on a reduced-space formulation and enhances our previous work (Schäfer et al., Comput Chem Eng, 2020;132:106598) by a sensitivity-based refinement procedure. We therein expose the coefficients of the wavelet transform of the time series of independent process variables to the optimizer. The problem size is reduced by truncating the transform and iteratively adjusted using Lagrangian multipliers. We apply the algorithm to the scheduling of a multi-product air separation unit. The nonlinear power consumption characteristic is replaced by an artificial neural network trained on data from a rigorous model. We demonstrate that the proposed algorithm reduces the number of optimization variables by more than one order of magnitude, whilst furnishing feasible schedules with insignificant losses in objective values compared to solutions considering the full dimensionality.

The performance of an organic Rankine cycle (ORC) relies on process design and operation. Simultaneous optimization of design and operation for a range of working fluids (WFs) is therefore a promising approach for WF selection. For this, deterministic global process optimization can guarantee to identify a global optimum, in contrast to local or stochastic global solution approaches. However, providing accurate thermodynamic models for a large number of WFs while maintaining computational tractability of the resulting optimization problems are open research questions. We integrate accurate thermodynamic and transport properties via artificial neural networks (ANNs) and solve the design problems with MAiNGO in a reduced-space formulation. We illustrate the approach for an ORC process for waste heat recovery of a diesel truck. After an automated preselection of 122 WFs, ANNs are automatically trained for the 37 selected WFs based on data retrieved from the thermodynamic library CoolProp. Then, we perform deterministic global optimization of design and operation for every WF individually. Therein, the trade-off between net power generation and investment cost is investigated by multiobjective optimization. Further, a thermoeconomic optimization finds a compromise between both objectives. The results show that, for the given conditions, monoaromatic hydrocarbons are a promising group of WFs. In future work, the proposed method and the trained ANNs can be applied to the design of a variety of energy processes.

Numerical optimization is very useful for design and operation of energy processes. As the design has a major impact on the economics of the system, it is desirable to find a global optimum in the presence of local optima. So far, deterministic global optimization with detailed thermodynamic models incorporated has been limited to single fluid energy systems. We extend our previously presented approach [Schweidtmann et int Mitsos, COMPUT CHEM ENG (2019)] from single-species working fluids to binary mixtures with variable composition. First, we create accurate thermodynamic data for two selected binary mixtures using a thermodynamic library. Using this data, we train artificial neural networks, select them based on desired accuracy, and include them in the process model. The resulting hybrid model is then optimized with the open-source solver MAiNGO. We perform thermodynamic optimizations of geothermal power plants, considering both organic Rankine and Kalina cycle. For each cycle, we identify the globally optimal design, operation, and working fluid composition for the selected binary fluid mixtures within tractable CPU times. We show how a second mixture component enables improved ORC performance.

Innovative membrane technologies optimally integrated into large separation process plants are essential for economical water treatment and disposal. However, the mass transport through membranes is commonly described by nonlinear differential-algebraic mechanistic models at the nano-scale, while the process and its economics range up to large-scale. Thus, the optimal design of membranes in process plants requires decision making across multiple scales, which is not possible using standard tools. In this work, we embed artificial neural networks (ANNs) as surrogate models in the deterministic global optimization to bridge the gap of scales. This methodology allows for deterministic global optimization of membrane processes with accurate transport models – avoiding the utilization of inaccurate approximations through heuristics or short-cut models. The ANNs are trained based on data generated by a one-dimensional extended Nernst-Planck ion transport model and extended to a more accurate two-dimensional distribution of the membrane module, that captures the filtration-related decreasing retention of salt. We simultaneously design the membrane and plant layout yielding optimal membrane module synthesis properties along with the optimal plant design for multiple objectives, feed concentrations, filtration stages, and salt mixtures. The developed process models and the optimization solver are available open-source, enabling computational resource-efficient multi-scale optimization in membrane science.

Using nonlinear process models in discrete-time scheduling typically prohibits long planning horizons with fine temporal discretizations. Therefore, we propose an adaptive grid algorithm tailored for scheduling subject to time-variable electricity prices. The scheduling problem is formulated in a reduced space. In the algorithm, the number of degrees of freedom is reduced by linearly mapping one degree of freedom to multiple intervals with similar electricity prices. The mapping is iteratively refined using a wavelet-based analysis of the previous solution. We apply the algorithm to the scheduling of a compressed air energy storage. We model the efficiency characteristics of the turbo machinery using artificial neural networks. Using our in-house global solver MAiNGO, the algorithm identifies a feasible near-optimal solution with < 1% deviation in the objective value within < 5% of the computational time compared to a solution considering the full dimensionality.

Nonlinear model predictive control requires the solution of nonlinear programs with potentially multiple local solutions. Here, deterministic global optimization can guarantee to find a global optimum. However, its application is currently severely limited by computational cost and requires further developments in problem formulation, optimization solvers, and computing architectures. In this work, we propose a reduced-space formulation for the global optimization of problems with recurrent neural networks (RNN) embedded, based on our recent work on feed-forward artificial neural networks embedded. The method reduces the dimensionality of the optimization problem significantly, lowering the computational cost. We implement the NMPC problem in our open-source solver MAiNGO and solve it using parallel computing on 40 cores. We demonstrate real-time capability for the illustrative van de Vusse CSTR case study. We further propose two alternatives to reduce computational time: i) reformulate the RNN model by exposing a selected state variable to the optimizer; ii) replace the RNN with a neural multi-model. In our numerical case studies each proposal results in a reduction of computational time by an order of magnitude.

Global optimization is desirable for the design of chemical and energy processes as design decisions have a significant influence on the economics. A relevant challenge for global flowsheet optimization is the incorporation of accurate thermodynamic models. A promising alternative to conventional thermodynamic property models is the integration of data-driven surrogate models into mechanistic process models and deterministic global optimization in a reduced space. In our previous works, we trained artificial neural networks (ANNs) on thermodynamic data and included the surrogate models in the global flowsheet optimization of subcritical organic Rankine cycles (ORC). In this work, we extend the framework to the optimization of transcritical ORCs operating at a supercritical high pressure level and subcritical low pressure level. We train separate ANNs for supercritical and subcritical thermodynamic properties. ANNs with a small number of neurons can learn the thermodynamic properties to sufficient accuracy. We identify the optimal working fluid among 122 available fluids in the thermodynamic library CoolProp via a deterministic global optimization of the hybrid process model using the solver MAiNGO. The results show that the process can be optimized efficiently and that transcritical operation can enable high power generation.

Extensive literature has considered reduced, but still highly accurate, nonlinear dynamic process models, particularly for distillation columns. Nevertheless, there is a need for continuing research in this field. Herein, opportunities from the integration of machine learning into existing reduction approaches are discussed. First, key concepts for dynamic model reduction and their limitations are briefly reviewed. Afterwards, promising model structures for reduced hybrid mechanistic/data-driven models are outlined. Finally, crucial future challenges as well as promising research perspectives are presented.

Organic Rankine cycles (ORCs) offer a high structural design flexibility. The best process structure can be identified via the optimization of a superstructure, which considers design alternatives simultaneously. In this contribution, we apply deterministic global optimization to a geothermal ORC superstructure, thus guaranteeing to find the best solution. We implement a hybrid mechanistic data-driven model, employing artificial neural networks as thermodynamic surrogate models. This approach is beneficial as we optimize the problem in a reduced space using the optimization solver MAiNGO. We further introduce redundant constraints that are only considered for the lower-bounding problem of the branch-and-bound algorithm. We perform two separate optimizations, one maximizing power output and one minimizing levelized cost of electricity. The optimal solutions of both objectives differ from each other, but both have three pressure levels. Global optimization is necessary as there exist suboptimal local solutions for both flowsheet configuration and design with fixed configurations.

Prediction of combustion-related properties of (oxygenated) hydrocarbons is an important and challenging task for which quantitative structure-property relationship (QSPR) models are frequently employed. Recently, a machine learning method, graph neural networks (GNNs), has shown promising results for the prediction of structure-property relationships. GNNs utilize a graph representation of molecules, where atoms correspond to nodes and bonds to edges containing information about the molecular structure. More specifically, GNNs learn physicochemical properties as a function of the molecular graph in a supervised learning setup using a backpropagation algorithm. This end-to-end learning approach eliminates the need for selection of molecular descriptors or structural groups, as it learns optimal fingerprints through graph convolutions and maps the fingerprints to the physicochemical properties by deep learning. We develop GNN models for predicting three fuel ignition quality indicators, i.e., the derived cetane number (DCN), the research octane number (RON), and the motor octane number (MON), of oxygenated and nonoxygenated hydrocarbons. In light of limited experimental data in the order of hundreds, we propose a combination of multitask learning, transfer learning, and ensemble learning. The results show competitive performance of the proposed GNN approach compared to state-of-the-art QSPR models, making it a promising field for future research. The prediction tool is available via a web front-end at www.avt.rwth-aachen.de/gnn.