Chemie

Ingenieur Review 943

Technik

Recent Advances in Chemical Process

Optimization
Lorenz T. Biegler
DOI: 10.1002/cite.201400033

Systematic optimization strategies are essential in process design, operations and control. This study provides a concise
overview of recent advances in process optimization, with a particular focus on nonlinear programming. Here, derivative-
free, SQP, reduced gradient, and interior point methods are described. Moreover, state of the art problem formulations
and modeling environments are described, and contrasted with popular process simulation tools. Finally research direc-
tions are identified for future trends in multi-scale optimization within large-scale equation-oriented optimization environ-
ments.
Keywords: Complementarity problems, Multi-scale modeling, Nonlinear programming, Optimization, Process simulation
Received: February 13, 2014; revised: March 01, 2014; accepted: March 21, 2014

1 Introduction As shown in Fig. 1, optimization problems that arise in

chemical engineering can be classified in terms of continu-
Decision-making is an essential component of process engi- ous and discrete variables. For the former, nonlinear pro-
neering, whether in process design, operations, control or gramming (NLP) problems form the most general case, and
analysis. For this task, powerful predictive modeling tools widely applied specializations include linear programming
have been developed for a variety of time and length scales. (LP) and quadratic programming (QP). An important dis-
As shown in [1], multi-scale models and tasks arise in all tinction for NLP is whether the optimization problem is
applications of process systems engineering. Here, time convex or not. In the non-convex case, NLP problems may
and length scales may cover over 15 orders of magnitude, have multiple local optima, and an important question is
and reflect the behavior of electronic/atomistic/molecular/ whether global or local NLP solutions are needed, as the for-
molecular-cluster levels all the way to process units, plants mer requires far more expensive methods. Another impor-
and their integration into the enterprise. Moreover, these tant distinction is whether objective and constraint func-
tasks may be strongly coupled. Basic components at the low- tions in the optimization problem provide derivative
er scales serve as the building blocks for the scales above information or not. If not, the size and complexity of the op-
them, while the demands at higher scales impose the speci- timization model is much more limited by the available
fications at smaller levels, particularly with respect to mate- methods that can be applied.
rial properties. Optimization with discrete variables (e.g., binary variables
In addition to these modeling tasks, it is essential to de- that take on values of 0 and 1 only) are treated through
velop systematic decision-making methods that incorporate mixed-integer problems. These can be written as mixed-in-
quantitative predictive behavior, and find the best solutions teger nonlinear programs (MINLP), or as mixed-integer lin-
to satisfy constraints and extremize relevant criteria for ear programs (MILP) if all variables appear linearly in the
superior process systems. For this purpose, optimization constraint and objective functions. For the latter an impor-
models and algorithms form the core tools for (a) experi- tant case occurs when all the variables are integer. This gives
mental design, parameter estimation, model development rise to an integer programming (IP) problem. IPs can be
and statistical analysis, (b) process synthesis analysis, design further classified into particular problem types, such as as-
and retrofit, (c) model predictive control and real-time opti- signment, set covering, lot sizing, and traveling salesman.
mization, and (d) planning, scheduling and the integration Similarly, the MINLP problem gives rise to special problem
of process operations into the supply chain. classes, although here, the main distinction is whether its
relaxation is convex or non-convex.
Key elements in formulating optimization problems in-
clude a mathematical model of the system, an objective
– function that quantifies a criterion to be extremized, vari-
Prof. Lorenz T. Biegler (lb01@andrew.cmu.edu), Carnegie Mellon ables that can serve as decisions, and optionally, inequality
University, Chemical Engineering Department, Doherty Hall, 5000 constraints on the system. When represented in algebraic
Forbes Avenue, Pittsburgh, PA 15213, USA. form, the most general formulation of discrete/continuous

Chem. Ing. Tech. 2014, 86, No. 7, 943–952 © 2014 Wiley-VCH Verlag GmbH & Co. KGaA, Weinheim www.cit-journal.com
 15222640, 2014, 7, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/cite.201400033 by CAPES, Wiley Online Library on [02/12/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
Chemie
944 Review Ingenieur
Technik

operations, control, scheduling, and planning.

Optimization Process development and modeling is concerned
with transforming experimental data and funda-
mental chemical and physical relationships into
Mixed Integer
NLP
a predictive process model. Incorporating the
(Discrete) tasks of experimental design, parameter estima-
(Continuous)
tion, and discrimination of competing models
MINLP Differentiable Derivative-free gives rise to NLP and MINLP problems. Process
synthesis and design then incorporates these
MILP predictive models to devise a process that is both
Convex Non-convex SA/GA technically and economically feasible. The synth-
IP
DFO esis step is largely focused on establishing the
LP QP Local Global structure of the process flowsheet (usually with
simplified models) and systematic strategies
have been developed for particular subsystems
Figure 1. Classes of optimization problems. that involve reaction, separation, energy manage-
ment and waste reduction. In particular, popular
optimization problems can be written as the following synthesis models that are based on pinch concepts, such as
mixed-integer optimization problem: heat exchanger network synthesis (HENS), and mass ex-
changer network synthesis (MENS) have straightforward
min f x; y†
realizations as LPs (for developing targets) and MILPs (to
s:t: h x; y† ˆ 0 realize stream matches in the network). On the other hand,
(1)
g x; y† ≤ 0 the design step is concerned with establishing equipment
x ∈ℜn ; y ∈f0; 1gt parameters and nominal operating conditions for the flow-
sheet. As a result, synthesis problems can be addressed with
where f(x,y) is the objective function (e.g., cost, energy con- a wide range of optimization formulations, while the design
sumption, etc.), h(x,y) = 0 are the equations that describe step requires more detailed models that give rise to NLP
the performance of the system (e.g., material balances, pro- and MINLP problems.
duction rates), and the inequality constraints g(x,y) ≤ 0 de- In the operation of chemical processes, there is wide-
fine process and product specifications, or constraints for spread interest in improving the scheduling and planning
feasible designs and operating policies. Note that the opera- of these operations. These tasks are usually formulated as
tor max f(x) is equivalent to min –f(x). The real n-vector x is LP and MILP problems, which are less detailed as most op-
defined to represent the continuous variables while the erations are described through time requirements and activ-
t-vector y represents the discrete variables, which, without ities. On the other hand, the development of large-scale
loss of generality, are often restricted to take 0-1 values to de- NLP tools has led to the application of real-time optimiza-
fine logical or discrete decisions, such as assignment of tion, which interacts with the process and responds to
equipment and sequencing of tasks. The problem described changes in the production schedule. The results of the real-
in Eq. (1) corresponds to a MINLP when any of the func- time optimization also feed operating conditions or set-
tions involved are nonlinear. If all functions are linear, the points to the control system. Note that real-time optimiza-
problem corresponds to a MILP. If there are no 0-1 vari- tion describes steady state behavior while the control system
ables, then it becomes an NLP or LP, depending on whether responds essentially to the process dynamics. The control
or not the functions are linear. task addresses maintenance of optimal production levels
Optimization has found widespread use in chemical engi- and rejection of disturbances. For multivariable, constrained
neering applications, especially in the engineering of pro- systems, model predictive control (MPC) is now accepted as
cess systems. These systems often have many degrees of a widely used optimization-based strategy. MPC models
freedom as well as complex economic and performance in- allow various levels of detail to handle linear and nonlinear
teractions; so it is often difficult to identify the optimal solu- dynamics as well as discontinuities that occur in hybrid sys-
tion through intuitive reasoning. Moreover, the economics tems.
of the system often show that the optimum solution trans- Tab. 1 summarizes the model types that have been formu-
lates into large savings, along with a large economic penalty lated for process engineering applications. Note that pinch
for sticking to suboptimal solutions. Therefore, optimiza- synthesis models are dominated by LP and MILP formula-
tion has become a major technology that helps the chemical tions, while the remaining design problems are dominated
industry to remain competitive. by NLP and MINLP models. These require explicit handling
Optimization problems are encountered in all facets of of performance equations. Operations problems, in con-
chemical engineering, from model and process develop- trast, tend to be dominated by linear models, LP and MILP,
ment to process synthesis and design, and finally to process for planning, scheduling and supply chain problems. NLP,

www.cit-journal.com © 2014 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim Chem. Ing. Tech. 2014, 86, No. 7, 943–952
 15222640, 2014, 7, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/cite.201400033 by CAPES, Wiley Online Library on [02/12/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
Chemie
Ingenieur Review 945
Technik

Table 1. Optimization Problem Classes and Process Applications. Global Optimization Methods
These methods for NLPs have
MILP MINLP Global LP, QP NLP Derivative Free
been developed to converge to the
HENS x x x x x x global optimum. More specifi-
MENS x x x x x x cally, one can show under mild
conditions that they converge to
Separations x x x an e-distance to the global opti-
Reactors x x x x mum with a finite number of
steps. Because there are no crite-
Equipment x x x
Design ria analogous to Karush-Kuhn-
Tucker conditions (KKT) for
Flowsheeting x x
NLPs, global optimization meth-
Planning x x x ods are based on spatial branch
Scheduling x x x x
and bound searches over the fea-
sible region. These methods par-
Real-time x x tition the feasible region into sub-
Optimization
regions, where upper bounds on
Linear MPC x the objective function are com-
Nonlinear MPC x puted, and lower bounds are de-
rived from convex relaxations of
Hybrid MPC x the objective function and con-
straints for each subregion. The
however, plays a crucial role at the level of real time optimi- algorithm then proceeds to eliminate all subregions that
zation. Finally, process control has traditionally relied on LP have infeasible constraint relaxations or lower bounds that
and NLP models with special structures, although MILPs are greater than the least upper bound. After this, the
are being increasingly used for hybrid systems. remaining regions are further partitioned to create new sub-
In this survey the focus lies on process optimization with regions and the cycle continues until the upper and lower
NLP models (Eq. (1) without integer variables) where the ob- bounds converge. Comprehensive development of these
jective and constraint functions are differentiable and deri- methods can be found in [7 – 9].
vatives are available from the model. Applications of these
form the NLP column in Tab. 1. Optimization methods that Mixed-Integer Programming
are not included in this survey include the following. Both MILP and MINLP deal with discrete and continuous
decision variables in Eq. (1). As with global optimization
Optimization Methods Without Derivatives methods, there are no optimality conditions, like the KKT
Derivative-free optimization methods (DFO) include the conditions, that can be applied directly. Instead, a systematic
classical direct search methods based on simplex and pat- search of the solution space, coupled with upper and lower
tern searches [2, 3] and phenomenological methods, such as bounding information, is applied. For MILPs, very efficient
simulated annealing (SA) [4] and genetic algorithms (GA) algorithms have been developed. Lower bounds are provided
[5]. More recently, classical direct search approaches have by LP solutions at intermediate nodes in the tree; these are
been extended so that rigorous convergence to local solu- relatively easy to calculate with the simplex method. Once
tions can be proved. These include the multivariable DFO the solution is known at an intermediate node, efficient pi-
algorithm that uses surrogate models, such as quadratic in- voting operations from this solution update solutions for
terpolation models, for the objective function within a trust the nodes at the next level. Moreover, a number of branch-
region method. DFO methods are formulated, analyzed and ing strategies navigate the tree very efficiently. A detailed de-
reviewed in [6]. These methods are easy to apply to a wide scription and availability of these and other MILP solvers
variety of problem types and optimization models and may can be found in [10]. Solution methods for MINLPs operate
also favor the search for global rather than locally optimal on the same concepts, but are more challenging, as NLPs re-
solutions. However, they are usually restricted to uncon- sult when binary variables are fixed, and these need to be
strained or bound constrained problems. Otherwise, they of- solved while navigating the search tree of binary decisions.
ten have difficulties with constraints, as equality constraint Strategies for the solution of MINLPs include NLP branch
elimination and addition of penalty functions for inequality and bound (NLP/BB), outer approximation (OA) and gener-
constraints are the only options. Finally, derivative-free alized disjunctive programming (GDP) methods. Compre-
methods scale poorly with the number of decision variables hensive reviews of MINLP methods can be found in [11, 12].
and are rarely applied to problems with more than a few In the next section, differentiable NLPs are considered
dozen decision variables. where efficient large-scale methods (to local solutions) are
briefly described. Sect. 3 continues with a discussion of opti-

Chem. Ing. Tech. 2014, 86, No. 7, 943–952 © 2014 Wiley-VCH Verlag GmbH & Co. KGaA, Weinheim www.cit-journal.com
 15222640, 2014, 7, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/cite.201400033 by CAPES, Wiley Online Library on [02/12/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
Chemie
946 Review Ingenieur
Technik

mization modeling software, and their implementation on Complementarity

engineering models. In particular, the need for a tight inte- Inequality constraints are either strictly satisfied (active) or
gration between the engineering model and the optimiza- inactive, in which case they are irrelevant to the solution. In
tion algorithm is emphasized, as well as an exploitation of the latter case the corresponding KKT multiplier must be
problem structure and derivative information by the optimi- zero. This is written as:
zation strategy. Sect. 4 concludes the paper with a short, tar-
geted summary on future directions for multi-scale optimi- 0 ≤ m?g x† ≤ 0 (6)
zation strategies, particularly with complex, nonlinear
process models, and equation-oriented optimization formu- where the ? operator indicates the complementarity rela-
lations. tion, i.e., element i of vector v may be positive only if the cor-
responding element of g(x) is zero.
2 Overview of Nonlinear Programming
Constraint Qualification (CQ)
Strategies
For a local optimum to satisfy the KKT conditions, an addi-
tional regularity condition is required. The CQ can be
For continuous variable optimization, Eq. (1) without dis-
viewed as a condition on the relative influence of constraint
crete variables y is presented below:
curvature. In fact, linearly constrained problems with non-
min f x† empty feasible regions require no constraint qualification.
s:t: h x† ˆ 0 (2) For NLPs, a widely applied CQ is the linear independence
g x† ≤ 0 constraint qualification (LICQ), which requires the active
constraints at x* to be linearly independent, i.e., the matrix:
where it is assumed that the functions f(x), h(x) and g(x) ‰∇h x*†j∇gA x*†Š is full column rank, where gA is the vector
have continuous first and second derivatives. A key question of inequality constraints with elements that satisfy
is whether the problem described by Eq. (2) is convex, gA;i x*† ˆ 0. With LICQ, the KKT multipliers (k,m) are guar-
whether it has a convex objective function and a convex fea- anteed to be unique at the optimal solution.
sible region or not. A function f(x) of x in some domain X
is convex if, and only if, for all points x1, x2 ∈ X: Second Order Conditions
In addition to the relations of Eqs. (4) – (7) for the first order
f(ax1+ (1–a)x2) ≤ af(x1) + (1–a)f(x2) (3)
KKT conditions, nonnegative (positive) curvature is neces-
sary (sufficient) in all of the allowable, i.e., constrained, di-
holds for all a ∈ (0, 1). Convex feasible regions require g(x)
rections p. This condition can be stated in several ways. A
to be a convex function and h(x) to be linear. If Eq. (2) de-
typical second order condition requires a point x* that satis-
scribes a convex problem, then any local solution is guaran-
fies LICQ and Eqs. (4) – (6) with multipliers k and m to satisfy
teed to be a global solution to Eq. (2). On the other hand,
the necessary conditions given by:
non-convex problems may have multiple local solutions,
which minimize the objective function only within the
neighborhood of the solution. Since global solvers for non- pT ∇xx L x*; k; m†p ≥ 0 for all p≠0;∇h x*†T p ˆ 0; ∇gA x*†T p ˆ 0
convex problems are beyond the scope of this survey, only (7)
the local case is considered.
The corresponding sufficient conditions require that the
Local optimality conditions are generally derived from
inequality in Eq. (7) be strict. Additional derivation and dis-
gradient information from the objective and constraint
cussion can be found in [13, 14].
functions. This leads to the celebrated necessary KKT condi-
tions, stated as follows.

Stationarity Condition 2.1 Convex Cases of NLP

It is convenient to define the Lagrangian function L(x,k,m) =
f(x)+h(x)Tk+g(x)Tm, along with weights or multipliers k and Linear programs and quadratic programs are special cases
m for the constraints. These multipliers are also known as of Eq. (2) that allow for more efficient solutions, based on
dual variables and shadow prices. The stationarity condition the application of the KKT conditions (Eqs. (4) – (7)). Be-
is then given by: cause these are convex problems, any locally optimal solu-
tion is a global solution. If the objective and constraint func-
∇L x; k; m† ˆ ∇f x† ‡ ∇h x†k ‡ ∇g x†m ˆ 0 (4) tions in Eq. (2) are linear, then the following LP can be
solved in a finite number of steps and the optimal solution
Feasibility lies at a vertex of the polyhedron described by the linear con-
Both equality and inequality constraints must be satisfied: straints.

h x† ˆ 0; g x† ≤ 0 (5)

www.cit-journal.com © 2014 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim Chem. Ing. Tech. 2014, 86, No. 7, 943–952
 15222640, 2014, 7, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/cite.201400033 by CAPES, Wiley Online Library on [02/12/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
Chemie
Ingenieur Review 947
Technik

min cTx set at point xk is determined by solving the following QP at

s.t. Ax = b (8) iteration k:
Cx ≤ d
min ∇f(xk)Tp +1⁄2pT ∇xx L(xk, kk, vk) p
The standard method to solve Eq. (8) is the simplex meth- s.t. h(xk) + ∇h(xk)T p = 0 (15)
od, developed in the late 1940s, although, starting from Kar- g(xk) + ∇g(xk)T p + s = 0, s ≥ 0
markar’s discovery in 1984, interior point methods have be-
come quite advanced and competitive for highly constrained where the Hessian of the Lagrange function, ∇xxL(x,k,m) =
problems [14]. Quadratic programs represent a slight modi- ∇xx(f(x) + h(x)T k + g(x)T v), is calculated directly or through
fication of Eq. (8) and can be stated as: a quasi-Newton approximation created by differences of gra-
dient vectors. Details of this approach can be found in
min cTx + 1⁄2 xTQx [14, 15]. As alternatives that avoid the combinatorial prob-
s.t. Ax = b (9) lem of selecting the active set, interior point (or barrier)
Cx ≤ d methods modify the NLP (Eq. (2)) to form:
min f(xk) – l Ri ln si
If the matrix Q is positive semi-definite (positive definite)
s.t. h(xk) = 0 (16)
when projected into the null space of the active constraints,
g(xk) + s = 0
then Eq. (9) is (strictly) convex and the QP is a global and
unique minimum. Otherwise, local solutions exist for
where the solution has s > 0 for the penalty parameter l > 0,
Eq. (9) and more extensive global optimization methods are
and decreasing l to zero leads to the solution of the problem
needed to obtain the global solution. Like LPs, convex QPs
in Eq. (2). The KKT conditions for this problem can be writ-
can be solved in a finite number of steps. Popular QP sol-
ten as:
vers include null space algorithms, range space methods
and Schur complement methods. As with LPs, QP prob- ∇f(x*) + ∇h(x*) k + ∇g(x*) v = 0
lems can also be solved with interior point methods. h(x*) = 0 (17)
Detailed presentation and discussion of these methods can g(x*) + s = 0
be found in [14]. S V e = le

and at iteration k the Newton steps are applied to solve

2.2 Solving the General NLP Problem Eq. (17). Detailed descriptions of this algorithm can be
found in [14, 15].
Solution techniques for Eq. (2) deal with satisfaction of the Active set and interior point SQP methods have clear
KKT conditions (Eqs. (4) – (7)). Many NLP solvers are based trade-offs. Interior point methods may require more itera-
on applying an extension of the Newton-Raphson method to tions to solve Eq. (17) for various values of l, while active set
the KKT conditions. By adding slack variables s, the first or- methods require the solution of the more expensive QP sub-
der KKT conditions can be rewritten as: problem (Eq. (15)). Thus, when there are few inequality con-
straints or an active set is known, e.g., from a good starting
∇f(x) + ∇h(x) k + ∇g(x) v = 0 (10) guess, or a known QP solution from a previous iteration,
then solving Eq. (15) is not expensive and the active set
h(x) = 0 (11) method is favored. On the other hand, for problems with
many inequality constraints, interior point methods are of-
g(x) + s = 0 (12) ten faster as they avoid the combinatorial problem of select-
ing the active set, and more readily exploit sparsity and
SVe=0 (13) structure in the KKT conditions. This is especially the case
for large-scale problems and when a large number of
(s, v) ≥ 0 (14) bounds are active.
Finally, reduced gradient methods are active set strategies
where e = [1, 1, ... , 1]T, S = diag{s} and V = diag{m}. Succes- that rely on partitioning the variables and solving Eqs. (10) –
sive quadratic programming (SQP) methods find solutions (14) in a nested manner. By adding slack variables as in
that satisfy Eqs. (10) – (14) by generating Newton-like search Eq. (12) and redefining the optimization variables and con-
directions at iteration k. However, Eq. (13) and active bounds straints, the problem of Eq. (2) can be rewritten as:
(Eq. (14)) make the KKT system ill-conditioned near the so-
lution. Newton-type algorithms treat these conditions in two Min f(z)
ways. In the SQP strategy, discrete decisions are made re- s.t. c(z) = 0 (18)
garding the active constraint set, i ∈ I = {igi(x*) = 0} and a≤z≤b
Eq. (13) is replaced by si = 0, i ∈ I, and vi = 0, i ∉ I. The active

Chem. Ing. Tech. 2014, 86, No. 7, 943–952 © 2014 Wiley-VCH Verlag GmbH & Co. KGaA, Weinheim www.cit-journal.com
 15222640, 2014, 7, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/cite.201400033 by CAPES, Wiley Online Library on [02/12/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
Chemie
948 Review Ingenieur
Technik

The variables are then partitioned as nonbasic variables Complementarities also arise naturally in the solution of
fixed at their bounds, superbasic variables, between bounds, multi-level optimization problems. Such problems are
that drive the optimization, and basic variables that can members of a more general problem class called mathema-
be solved from the equality constraints; this leads to tical programs with equilbrium constraints (MPECs) [16].
zT = [zNT, zST, zBT]. This partition is derived from local infor- For bi-level optimization problems of the form
mation and may change over the course of the optimization
iterations. The corresponding KKT conditions can be written min f(x)
as: s.t. (x,y) ∈ Z, y = arg{miny h (x,y), y ∈ C (x)} (24)

∇N f z† ‡ ∇N c z†c ˆ b a bb (19) Z and C(x) are defined as feasible regions for the upper
and lower level problems, respectively, with f(x,y) and h(x,y)
∇S f z† ‡ ∇S c z†c ˆ 0 (20) as objective functions for the upper and lower level prob-
lems, respectively. In particular, bi-level optimization prob-
∇B f z† ‡ ∇B c z†c ˆ 0 (21) lems can be reformulated as mathematical programs with
complementarity constraints (MPCCs) by writing the opti-
c z† ˆ 0 (22) mality conditions of the inner optimization problem (espe-
cially if this problem is convex) as constraints on the outer
zN;j ˆ aj or bj ; b a;j ≥ 0; bb;j ˆ 0 or b b;j ≥ 0; ba;j ˆ 0 (23) problem. An MPCC takes the following general form:

min f(x)
where c and b are the KKT multipliers for the equality and s.t. h(x,y,z) = 0
bound constraints, respectively, and Eq. (23) replaces the g(x,y,z) ≤ 0 (25)
complementarity conditions from Eq. (6). Reduced gradient 0≤x⊥y≥0
methods work by nesting Eqs. (21) and (22) within Eqs. (19)
and (20). At iteration k, for fixed values of zNk and zSk, one where complementarity constraint in Eq. (25) implies xi = 0
can solve for zB using Eq. (22) and for c using Eq. (21). Fol- or yi = 0 (or both) for each vector element i. The complemen-
lowing this, bound multipliers b are calculated from tarity constraint may be written in several equivalent ways
Eq. (19). Over the course of the iterations, if the variables zB including:
or zS exceed their bounds or if some bound multipliers b
become negative, then the variable partition needs to be xT y = 0, x ≥ 0, y ≥ 0
changed and the Eqs. (19) – (23) are reconstructed. These re- xi yi = 0, x ≥ 0, y ≥ 0 (26)
duced gradient methods are embodied in the popular GRG2 xi yi ≤ 0, x ≥ 0, y ≥ 0
and CONOPT codes. In addition, reduced gradient concepts xi – max(0, xi – yi) = 0.
have been extended to develop large-scale reduced-space
SQP algorithms in SNOPT and rSQP. See [15] for further These alternate forms are particularly useful when apply-
details and references. ing existing NLP solution strategies to solve MPCCs.
On the other hand, MPCCs of the form of Eq. (25) cannot
be solved directly with most NLP solvers because the asso-
2.3 Handling Complementarity Constraints ciated optimality conditions of Eq. (25) do not satisfy con-
straint qualifications CQs. This leads to singular KKT con-
The MINLP (Eq. (1)) leads to the most general description of ditions, with unbounded constraint multipliers and
process optimization problems. To solve these problems, dependent constraints at every feasible point. Instead, the
NLP formulations are usually solved at a lower level for following MPCC reformulations allow standard NLP solvers
fixed values of the discrete variables. On the other hand, the to be applied.
ability to deal with some discrete decisions through continu-
ous variables, within an all-at-once NLP formulation may be min f(x)
more efficient than nested MINLP strategies. This is parti- s.t. h(x,y,z) = 0
cularly true if many discrete decisions must be handled. g(x,y,z) ≤ 0 (27)
One approach to modeling discrete decisions with contin- xiyi ≤ e
uous variables is through complementarity constraints. x,y ≥ 0
Complementarity problems arise in a large number of appli-
cations in engineering and economic systems. They include min f(x)
contact and friction problems in computational mechanics, s.t. h(x,y,z) = 0
equilibrium relations in economics, and a wide variety of g(x,y,z) ≤ 0 (28)
discrete events in process systems. However, these mathe- xTy ≤ e
matical programming formulations and related solution x,y ≥ 0
strategies are not yet fully developed in process engineering.

www.cit-journal.com © 2014 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim Chem. Ing. Tech. 2014, 86, No. 7, 943–952
 15222640, 2014, 7, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/cite.201400033 by CAPES, Wiley Online Library on [02/12/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
Chemie
Ingenieur Review 949
Technik

min f(x) 3 Development of Process Optimization

s.t. h(x,y,z) = 0 Models
g(x,y,z) ≤ 0 (29)
xiyi = e The most important aspect to a successful optimization
x,y ≥ 0 study is the formulation of the process optimization model.
These models must reflect the real-world problem so that
min f(x) + q(xTy) meaningful optimization results are obtained. They must
s.t. h(x,y,z) = 0 also satisfy the properties of the problem class. For instance,
g(x,y,z) ≤ 0 (30) NLPs addressed by gradient-based methods need to have
x,y ≥ 0 functions that are defined in the variable domain and have
bound and continuous first and second derivatives. In
For the first three, regularized formulations, the comple- mixed-integer problems, proper formulations are also
mentarity conditions are relaxed and the MPCC is reformu- needed to yield good lower bounds for efficient search. With
lated in Eqs. (27) – (29) with a positive relaxation parameter increased understanding of optimization methods and the
e. The solution of the MPCC, (x*, y*, z*), can be obtained by development of efficient and reliable optimization codes,
solving a series of relaxed solutions as e approaches zero. optimization practitioners now focus on the formulation of
Convergence properties of these NLP formulations have optimization models that are realistic, well-posed and inex-
been analyzed in [17]. In contrast to the regularized formu- pensive to solve. More importantly, efficient performance of
lations (Eqs. (27) – (29)), also the exact penalty function NLP, MILP and MINLP solvers requires accurate first, and
shown in Eq. (30) was considered. Here the complementar- often second derivatives from the optimization model. If
ity can be moved from the constraints to the objective func- these contain numerical errors, e.g., through finite differ-
tion and the resulting problem is solved for a particular ence approximations, then performance of these solvers de-
value of q. Moreover, if q is greater than some critical value, teriorates considerably.
then the complementarity constraints will be satisfied at the Optimization models arise frequently in process design
solution under certain regularity conditions. In this case, if and analysis studies. These require accurate simulation
the problem in Eq. (30) has a solution with xTy= 0, then this models to describe steady-state behavior of the chemical
corresponds to the solution of Eq. (25). Conversely, any solu- process. Because of the considerable detail required for
tion of Eq. (25) also is a solution of Eq. (30) if q is sufficiently these simulation models, particularly in the description of
large. thermodynamic equilibrium, models need to be structured
Eq. (30) allows any nonlinear programming solver to be with specialized solution procedures, along the lines of the
used to solve a complementarity problem, without modi- individual process units represented in the model. This col-
fying the algorithm to deal with the sequence for e. More- lection of unit models leads to flowsheet simulation prob-
over, using Eq. (30), the penalty parameter can also be chan- lems, typically executed in a modular simulation mode.
ged during the course of the optimization, for both active Careful attention to problem definition and implementation
set and interior point optimization algorithms. Finally, a is required in order to interface the simulation structure to
number of numerical comparisons conducted among efficient NLP solvers, particularly with respect to formulat-
Eqs. (27) – (30) indicate that MPCC reformulation of Eq. (30) ing well-defined objective and constraint functions with ac-
usually has the most efficient and robust performance [18]. curate derivatives. Nevertheless, for process flowsheet opti-
While binary variables are especially needed for structural mization, the NLP remains relatively small with only a few
decisions and many logical conditions, MPCCs deal effec- constraints and few degrees of freedom. Summaries of
tively with a number of process-specific switches including advances and guidelines for formulating optimization prob-
nonsmoothness in process correlations and piecewise lems in chemical engineering can be found in [15, 20].
smooth equations, check valves, overflow conditions, con- On the other hand, real-time optimization (RTO) is an
troller saturation and compressor kickbacks. Moreover, online activity that achieves optimal operation in petroleum
important MPCC applications for process synthesis [19] in- refineries, power plants, and chemical plants. Nonlinear
clude programs for RTO are based on process models similar to
– monitoring phase disappearance in vapor-liquid equili- those used for design and analysis. Recent RTO case studies
brium and distillation optimization, include [21 – 23]. Moreover, because RTO problems must be
– embedding equilibrium-based reactor models into pro- solved at regular intervals (at least every few hours), fitted
cess flowsheets, and correlations updated by process data replace a number of de-
– incorporating LP-based heat integration target models tailed simulation (usually thermodynamic) models. These
within process optimization problems. time-critical NLPs are formulated in the so-called equation-
oriented (EO) mode. As a result, the nonlinear program is
generally large, with many equality constraints from the
process model, but relatively few degrees of freedom for
optimization.

Chem. Ing. Tech. 2014, 86, No. 7, 943–952 © 2014 Wiley-VCH Verlag GmbH & Co. KGaA, Weinheim www.cit-journal.com
 15222640, 2014, 7, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/cite.201400033 by CAPES, Wiley Online Library on [02/12/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
Chemie
950 Review Ingenieur
Technik

Finally, for process models that comprise differential-alge- mization algorithms and remove the need for the user to in-
braic equations (DAEs) or partial differential-algebraic equa- terface with the solver directly. These platforms allow the
tions (PDAEs), as in dynamic and distributed systems, a si- general formulation for all problem classes discussed above
milar choice exists. These models can either be embedded with direct interfaces to state of the art optimization codes.
within a simulation module and interfaced to the NLP sol- Three representative platforms are GAMS (General Alge-
ver, or formulated with an EO strategy and incorporated braic Modeling Systems), AMPL (A Mathematical Program-
directly within the optimization model. For the EO model, ming Language) and AIMMS (Advanced Integrated Multidi-
the NLP is generally very large, with many equality con- mensional Modeling Software). All three require problem
straints. Moreover, discretization of control profiles often model input via a declarative modeling language and pro-
leads to many degrees of freedom for optimization. vide exact gradient and Hessian information through auto-
Additionally, evolution of these model formulations is clo- matic differentiation strategies. Although possible, these
sely tied to the choice of appropriate optimization algo- platforms were not designed to handle externally added pro-
rithms. Efficient NLP algorithms that assume open EO cedural models. As a result, these platforms are best applied
models are available with exact first and second derivatives on optimization models that can be developed entirely with-
for all of the constraint and objective functions. These algo- in their modeling framework. These platforms are widely
rithms work very efficiently with RTO, DAE and PDAE used for large-scale research and industrial applications.
models. On the other hand, for problems with embedded Simulation platforms with optimization are found in the
modules, where function evaluations are expensive, and gra- bottom three levels in Fig. 2. These are often dedicated,
dients and Hessians are difficult to obtain, it is clear that application-specific modeling tools to which optimization
large-scale NLP solvers should not be applied. Fig. 2 shows a solvers have been interfaced. These lead to very useful opti-
hierarchy of models paired with suitable NLP strategies. For mization studies, but because they were not originally de-
instance, large equation-based models are solved most effi- signed for optimization models, they need to be used with
ciently with structured barrier NLP solvers. On the other some caution. In particular, most of these platforms do not
hand, black-box process models with inexact, or approxi- provide exact derivatives to the optimization solver, often
mate, derivatives and few decision variables are poorly they are approximated through finite difference. In addi-
served by large-scale NLP solvers, and derivative-free optimi- tion, the models themselves are constructed and calculated
zation algorithms must be considered instead. The problem through numerical procedures, instead of through an open
formulations also include intermediate levels in Fig. 2, declarative language. Examples of these include widely used
where SQP and reduced gradient methods are expected to process simulators such as Aspen/Plus, PRO/II and Hysys.
perform well. Nevertheless, accurate derivatives can be obtained from
To exploit synergies due to model-solver interfaces, appro- simulation models through the application of efficient auto-
priate modeling platforms are essential for the optimization matic differentiation tools. Examples of these include ADIC,
task. These are classified into two broad areas: optimization ADOL-C, Open AD and Tapenade. When linked to modern
modeling platforms and simulation platforms with optimi- NLP and MINLP algorithms, these can be applied to existing
zation. procedural models to provide powerful optimization capabil-
Optimization modeling platforms represent the top level ities. It should also be noted that more recent platforms such
in Fig. 2. They provide general purpose interfaces for opti- as Aspen Custom Modeler and gPROMS include declarative
models and exact derivatives. The MATLAB platform allows
the flexible formulation of optimization models as well,
although it currently has only limited capabilities for auto-
matic differentiation and limited optimization solvers. More
information on these issues can be found in [15].

4 Conclusions and Future Outlook

Process systems engineering (PSE) tasks provide increasing

opportunities for superior process modeling and optimiza-
tion strategies. These opportunities also present a number
of challenges, particularly as chemical and energy processes
incorporate advanced technologies that need to be modeled,
integrated and optimized. To address these needs, state-of-
the-art nonlinear optimization algorithms can now solve
models with millions of decision variables and constraints.
Figure 2. Hierarchy of model-solver interfaces for process optimi- Correspondingly, the computational cost of solving discrete
zation. optimization problems has been reduced by several orders

www.cit-journal.com © 2014 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim Chem. Ing. Tech. 2014, 86, No. 7, 943–952
 15222640, 2014, 7, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/cite.201400033 by CAPES, Wiley Online Library on [02/12/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
Chemie
Ingenieur Review 951
Technik

of magnitude [24]. These algorithmic advances have been optimized product properties [27]. Because of the substan-
realized through software modeling frameworks that link tial complexity of these models, the computational costs for
optimization models to efficient nonlinear (NLP) and process optimization are prohibitive. While lumped para-
mixed-integer nonlinear programming (MINLP) solvers. meter flowsheet models take only a few CPU seconds to
Here it is important to note that these advances are enabled solve, CFD and multi-phase population balance models may
because these frameworks permit (indeed require) optimi- require many CPU hours or even days.
zation models to be formulated carefully as well-posed prob- Moreover, as shown in [28], multi-scale modeling chal-
lems with exact first and second derivatives. lenges need to be addressed within an optimization frame-
Unfortunately, many commercial simulation tools still work. Here, original detailed models comprising complex
struggle with full flowsheet optimization, thus restricting the differential-algebraic equations (DAEs) and partial differen-
size and scope of synthesis problems that can be practically tial-algebraic equations (PDAEs) are replaced by reduced
considered. Even with state of the art simulation packages models consisting of algebraic equations that can be
(e.g., AspenPlus, gPROMS, ProSIM, Pro/II) that offer gradi- handled within an EO optimization environment, and re-
ent-based optimization, the structure of the flowsheeting quire only a few CPU seconds to evaluate. Because such an
model typically limits the formulation of nonlinear optimiza- optimization strategy is allowed to evaluate and compare
tion problems to no more than 100 degrees of freedom. information from detailed models, this reduced model fra-
While integration of nonlinear optimization presents de- mework will capture complex multi-scale phenomena with-
manding challenges for simulation tools, many of the in the process optimization. However, these strategies need
capabilities for large-scale optimization are already available to consider the following basic questions:
in optimization modeling environments such as GAMS, – What properties are needed for optimization with re-
AMPL and AIMMS. Moreover, additional families of NLP duced models to converge to the optimum of the original
algorithms, such as interior point methods, have been devel- system models?
oped that are better suited for large optimization problems. – What properties govern the construction of reduced mod-
These have been tailored to exploit these characteristics and els in order to balance model accuracy with computa-
lead to very fast performance for NLPs with many thou- tional cost during the optimization?
sands of variables and degrees of freedom. In addition, the – Can an optimization strategy based on reduced models be
incorporation of complementarity conditions to treat non- performed efficiently without frequent recourse to the ori-
smooth elements, process switches and bi-level problem for- ginal models?
mulations leads to very flexible EO optimization models Based on extensions of trust region methods in [6, 29],
[19]. Unfortunately, these algorithms are not yet available in prototype algorithms have been developed, analyzed and de-
most commercial flowsheet simulators, as the optimization monstrated in [28]. These show tremendous potential in
models require fully open, equation-oriented (EO) models dealing with intractable multi-scale optimization models.
with exact first and second derivatives. Coupled with algorithmic and software breakthroughs in
In addition, truly multi-scale process optimization still the EO mode, it is clear that these superior process optimi-
needs effective problem formulation and modeling environ- zation strategies will continue to have tremendous impacts
ments. Most process simulation models consist of lumped in generating more efficient, more productive and more
parameter descriptions with a number of ideal assumptions, sustainable process engineering systems.
e.g., perfect mixing, plug flow, equilibrium behavior and
shortcut models. These models do not account for complex
transport effects, nor do they consider phenomena that take Lorenz T. Biegler is cur-
place at broader length scales. Consequently, they cannot in- rently the Bayer University
clude detailed interactions with material design, complex Professor and Head of
fluid flow and transport effects with multiphase interac- Chemical Engineering at
tions. These effects may also require integration between Carnegie Mellon Univer-
molecular, nanoscale and distributed, continuum levels. As sity, which he joined after
a result, there is a growing need for more detailed, distribu- receiving his Ph.D. from
ted parameter, multi-scale process models. the University of Wiscon-
For instance, optimization models for advanced power sin in 1981. His research
generation processes [25, 26] comprise a heterogeneous interests lie in computer
modeling environment with lumped parameter (algebraic) aided process engineering
models, such as heat exchangers, compressors and expan- (CAPE) and he was an
ders, dynamic models (e.g., for gas separation units) and institute fellow at the National Energy Technology Lab
multi-phase (partial differential-algebraic) CFD models, e.g., as well as a scientist-in-residence at Argonne and San-
for gasification and combustor-turbine units. Similarly, poly- dia National Labs. He is also a member of the National
mer processes require detailed population balance models Academy of Engineering and has authored over 350
to determine molecular weight distributions that provide publications including two books.

Chem. Ing. Tech. 2014, 86, No. 7, 943–952 © 2014 Wiley-VCH Verlag GmbH & Co. KGaA, Weinheim www.cit-journal.com
 15222640, 2014, 7, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/cite.201400033 by CAPES, Wiley Online Library on [02/12/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
Chemie
952 Review Ingenieur
Technik

Abbreviations [7] C. A. Floudas, Deterministic Global Optimization: Theory,

Methods and Applications, Kluwer Academic Publishers,
AMPL A Mathematical Programming Dordrecht 2000.
Language [8] R. Horst, P. M. Tuy, Global Optimization: Deterministic
Approaches, 3rd Ed., Springer-Verlag, Berlin 1996.
CQ constraint qualification
[9] M. Tawarmalani, N. Sahinidis, Convexification and Global Opti-
DAE differential-algebraic equations
mization in Continuous and Mixed-Integer Nonlinear Program-
DFO derivative-free optimization methods ming: Theory, Algorithms, Software, and Applications, Kluwer
EO equation-oriented Academic Publishers, Dordrecht 2002.
GA genetic algorithms [10] J. T. Linderoth, A. Lodi, In Wiley Encyclopedia of Operations
GAMS General Algebraic Modeling Systems Research and Management Science (Ed: J. J. Cochran), Vol. 5,
GDP generalized disjunctive programming John Wiley & Sons, Hoboken 2011, 3239 – 3248.
HENS heat exchanger network synthesis [11] I. E. Grossmann, Adv. Chem. Eng. 1996, 23, 171 – 246.
IP integer programming [12] I. Nowak, Relaxation and Decomposition Methods for Mixed Inte-
KKT Karush-Kuhn-Tucker ger Nonlinear Programming, Birkhaeuser, Boston, MA 2005.
LICQ linear independence constraint [13] O. L. Mangasarian, Nonlinear Programming, McGraw-Hill,
New York 1969.
qualification
[14] J. Nocedal, S. J. Wright, Numerical Optimization, Springer,
LP linear programming New York 2006.
MENS mass exchanger network synthesis [15] L. T. Biegler, Nonlinear Programming: Concepts, Algorithms and
MILP mixed-integer linear programs Applications to Chemical Engineering, SIAM, Philadelphia, PA
MINLP mixed-integer nonlinear programs 2010.
MPC model predictive control [16] Z.-Q. Luo, J.-S. Pang, D. Ralph, Mathematical Programs with
MPCC mathematical programs with Equilibrium Constraints, Cambridge University Press, Cam-
complementary constraints bridge 1996.
MPEC mathematical programs with [17] D. Ralph, S. J. Wright, Optim. Meth. Software 2004, 19 (5),
equilibrium constraints 527 – 556.
[18] B. T. Baumrucker, J. G. Renfro, L. T. Biegler, Comp. Chem. Eng.
NLP nonlinear programming
2008, 32, 2903 – 2913.
OA outer approximation
[19] A. W. Dowling, L. T. Biegler, Chem. Eng. Trans. 2013, 35, 1 – 12.
PDAE partial differential-algebraic equations [20] E. Esche, D. Müller, R. Kraus, G. Wozny, Chem. Eng. Sci., in
PSE process systems engineering press. DOI: 10.1016/j.ces.2013.11.041
QP quadratic programming [21] D. Müller, O. Kahrs, S. Höser, H. Arellano-Garcia, Chem. Ing.
RTO real-time optimization Tech. 2012, 84 (11), 1971 – 197.
SA simulated annealing [22] A. Ramdial, N. Hudson, R. Pike, J. Rodriguez, M. Stratman,
SQP successive quadratic programming S. Sama, in Proc. of SPE Annual Technical Conference and Exhi-
bition, Society of Petroleum Engineers, Houston, TX 2009.
[23] D. Sarabia, C. D. Prada, E. Gómez, G. Gutierrez, S. Cristea,
References J. Sola, R. Gonzalez, Control Eng. Pract. 2012, 20, 343 – 354.
[24] L. T. Biegler, I. E. Grossmann, Comp. Chem. Eng. 2004, 28 (8),
[1] W. Marquardt, L. Von Wedel, B. Bayer, AIChE Symp. Ser. 2000, 1169 – 1192.
96 (323), 192. [25] Y.-D. Lang, A. Malacina, L. T. Biegler, S. Munteanu, J. I. Mad-
[2] R. Hooke, T. A. Jeeves, J. ACM 1961, 8, 212. sen, S. E. Zitney, Energy Fuels 2009, 23, 1695 – 1706.
[3] J. A. Nelder, R. Mead, Comp. J. 1965, 7, 308. [26] Y.-D. Lang, S. E. Zitney, L. T. Biegler, Comp. Chem. Eng. 2011,
[4] P. J. M. van Laarhoven, E. H. L. Aarts, Simulated Annealing: 35, 1705 – 1717.
Theory and Applications, Reidel Publishing, Dordrecht 1987. [27] W. Lin, L. T. Biegler, A. M. Jacobson, Macromol. Theory Simul.
[5] J. H. Holland, Adaptations in Natural and Artificial Systems, 2011, 20 (2), 146 – 165.
University of Michigan Press, Ann Arbor 1975. [28] L. T. Biegler, Y.-D. Lang, W.-J. Lin, Comp. Chem. Eng. 2014, 60
[6] A. R. Conn, K. Scheinberg, L. Vicente, Introduction to Deri- (10), 17 – 30.
vative Free Optimization, SIAM, Philadelphia, PA 2009. [29] A. R. Conn, N. Gould, P. Toint, Trust Region Methods, SIAM,
Philadelphia, PA 2000.

www.cit-journal.com © 2014 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim Chem. Ing. Tech. 2014, 86, No. 7, 943–952