Optimization of Functions of Multiple Variables subject to
Equality Constraints
Objectives
Optimization of functions of multiple variables
subjected to equality constraints can be solved using the
method of constrained variation and the method of
Lagrange multipliers.
Constrained optimization
A function of multiple variables, f(x), is to be optimized
subject to one or more equality constraints of many
variables. These equality constraints, gj(x), may or may
not be linear.
The problem statement is as follows:
x1
Maximize (or minimize) f(X) x
subject to gj(X) = 0, j = 1, 2, …, m 2
where
X
x n
Constrained optimization (contd.)
With the condition that m ≤ n ; otherwise( if m > n) then
the problem becomes an over defined one and there will
be no solution.
There are several methods available for the solution of
this problem. The methods of direct substitution, the
method of constrained variation and the method of using
Lagrange multipliers are discussed.
Solution by Direct Substitution
For a problem with n variables and m equality constraints,
it is theoretically possible to solve simultaneously the m
equality constraints and express any set of m variables in
terms of remaining n-m variables.
When these expressions are substituted into the original
objective function, there results a new objective function
involving only n-m variables.
The new objective function is not subjected to any
constraint, and Hence its optimum solution can be found
using unconstraint optimization technique.
Solution by Direct Substitution
This method appears to be simple in theory, but not
convenient from practical point of view. As the constraint
equations will be non linear for most of practical problems
and often, it becomes impossible to solve them.
For simpler problems we can apply this method.
Let us understand How this method works by giving one
simple example.
Example
Find the dimension of a box of largest volume that can be
inscribed in a sphere of unit radius.
Solution
Consider Cartesian co - ordinate system x1, x2, x3
Let the origin of the Cartesian co ordinate system be at the
center of the sphere and the sides of the box be 2x1, 2x2, 2x3.
The volume of the box is given by f (x1, x2, x3) = 8x1x2x3
Since the corner of the box lie on the surface of the sphere
of unit radius, x1, x2, x3 have to satisfy the constraint
x x x
1
2 2
2
2
3 1
Example (contd…)
The problem has three design variables and one equality
constraint.
The equality constraint can be used to eliminate any one
of the design variables from the objective function.
If we choose to eliminate x3 ,
1
x3 1 x x
2
1
2 2
2
Thus, the objective function becomes,
f x1, x2 8x1 x2 1 x x
1
2 2 2
1 2
Which can be maximized as an unconstrained function in two variables.
The necessary conditions for the maximum of f give
f f 1 2x12 x22 0
0 0
x1 x 2
1 x12 2x22 0
As x2 ≠ 0, x1 ≠ 0 and
1 x x 0
2
1
2
2
As x1 , x2 and x3 are co ordinates, which can not be zero.
From which it follows that x1* = x2 * = √⅓
This solution gives the maximum volume of the box as
fmax = 8/(3√3)
To find whether the solution found corresponds to a
maximum or a minimum, we apply the sufficiency
condition to f(x1 ,x2)
The second order partial derivatives of f at (x1* , x2 *) are
given by
2 f 2 f 32 1 1
at
x12
x22
3 3 3
f2
16 1 1
at
x1 x 2 3 3 3
Since
2
f
2
f f f
2 2 2
0 and 0
x1
2
x1 x2 x1 x2
2 2
The Hessian matrix of f is negative definite at (x1* , x2 *)
Hence the point (x1* , x2 *) corresponds to the maximum
of f
Solution by method of Constrained Variation
For the optimization problem defined above,
let us consider a specific case with n= 2 and m= 1 before
we proceed to find the necessary and sufficient conditions
for a general problem using Lagrange multipliers.
The problem statement is as follows :
Minimize f(x1,x2),
subject to g(x1,x2) = 0
Solution by method of Constrained Variation
The problem statement is as follows :
Minimize f(x1,x2),
subject to g(x1,x2) = 0
The necessary condition in order to have (x1* , x2 *) as an
extreme point (minima or maxima) is
f g f g
at x1
x2 0
x1 x2 x2 x1
Example -1
A beam of uniform rectangular cross section is to be cut
from a log (a trunk of tree after removing branches)
having a circular cross section of diameter 2a.
The beam has to be used as a cantilever beam (i.e. one
end fixed) (the length is fixed) to carry a concentrated
load at the free end. Find the dimension of the beam that
corresponds to the maximum tensile (bending ) stress
carrying capacity.
Cross section of the log
2x
•
2y a
•
X
•
x2+y2=a2
•
• Y
•
• Y
Solution
From elementary strength of the materials, we know that
the tensile stress induced in a rectangular beam (σ) at any
fiber located a distance y from the X axis which is neutral
axis is given by M
y I
Where M is the bending moment acting and I is the
moment of inertia of the cross section about the X axis.
If the width and depth of the rectangular beam are 2x and
2y respectively, Then the maximum tensile stress induced
is given by M My 3 M
max y
I 1 3 4 xy 2
( 2 x )( 2 y )
12
Solution
As , for rectangle I =1/12 (width) (depth)3
Thus, for any specified bending moment , the beam is said
to have maximum tensile stress carrying capacity if the
maximum induced stress (σmax) is a minimum.
Hence, we need to minimize k/xy2 where k =3M/I,
subject to the constraint x2 +y2 =a2
This problem has two variables and one constraint.
Solution
Thus, we have to solve
Minimize f = kx-1y-2
Subject to the constraints,
g = x2 + y2 - a2
We have,
f f
kx 2 y 2 2 kx 1 y 3
x y
g g
2x 2y
x y
Solution
f g f g
at x y 0
x y y x
kx2 y2 (2y) 2kx1 y3 (2x) 0 at x y
y 2x
Thus, the beam of maximum tensile stress carrying capacity
has a depth of √2 times its breadth.
By substituting value of y in equation of constraint,
a a
x y 2
3 3
Solution by method of Lagrange multipliers
Continuing with the same specific case of the
optimization problem with n= 2 and m= 1 we define a
quantity λ, called the Lagrange multiplier as
f
x 2
g
at x1 x 2
x 2
Solution by method of Lagrange multipliers
Using this the necessary conditions for the point
(x1* , x2 *) to be an extreme point is given by
f g
x1
x1
at x 1
x 2 0
f g
x2
x2
at x 1
x 2 0
And g (x1* , x2 *) = 0
Solution by method of Lagrange multipliers
λ could be expressed in terms of ∂g/ ∂ x1 as well where
∂g/ ∂ x1 ≠ 0
Thus, these necessary conditions require that at least one
of the partial derivatives of g (x1* , x2 *) be non-zero.
This conditions can also be generated by constructing a
functions L, known as the Lagrangian function, as
L(x1, x2, λ) = f(x1, x2) + λ g(x1, x2)
Alternatively, treating Las a function of x1, x2 and λ, the
necessary conditions for its optimum are given by
L f g
x1
x1 , x 2 ,
x1
x1 , x 2
x1
x1 , x 2 0
L f g
x 1 , x 2 , x 1 , x 2 x1 , x 2 0
x 2 x 2 x 2
L
x1 , x 2 , g x 1 , x 2 0
Necessary conditions for a general problem
For a general problem with n variables and m equality
constraints the problem is defined as shown earlier
Maximize (or minimize) f(X) X1
subject to gj(X) = 0, j = 1, 2, …, m X
2
where
X
X n
Necessary conditions for a general problem
In this case the Lagrange function, L, will have one
Lagrange multiplier λj for each constraint as
L(x1, x2, ……..xn, λ1 λ2…… λm)
= f(X) + λ1 g1(X) + λ2 g2(X) + …… + λm gm(X)
Necessary conditions for a general problem…contd.
L is now a function of n + m unknowns, x1, x2, ……..xn, λ1
λ2…… λm , and the necessary conditions for the problem
defined above are given by
L f m g j
X j X 0
x i x i j 1 x i
L
gjX 0
j
Where i = 1,2, …… n and j = 1,2,….m
Necessary conditions for a general
problem…contd.
which represent n + m equations in terms of the n + m
unknowns, xi and λj.
The solution to this set of equations gives us
x 1 1
x2 2
X and
x n n
Sufficient conditions for a general problem
A sufficient condition for f(X) to have a relative minimum at
X*is that each root of the polynomial in , defined by the
following determinant equation be positive.
L11 L1 2 L1 n g 11 g 21 g m1
L21 L2 2 L2 n g 1 2 g 22 gm2
Ln1 Ln 2 Ln n g1n g 2n gmn
0
g 11 g12 g1n 0 0
g 21 g22 g2n
g m1 gm2 gmn 0 0
Sufficient conditions for a general problem…
contd.
Where
2L
Li j
x i x j
X , ,
g p
g pq
x q
X ,
For i =1,2,3,….n j =1,2,3, ……m
p =1,2,3, ……m q =1,2,3,….n
Sufficient conditions for a general problem…contd.
Similarly, a sufficient condition for f(X) to have a relative
maximum at X*is that each root of the polynomial in ,
defined by above equation be negative.
If above equation, on solving gives roots, some of which
are positive and others negative, then the point X*is
neither a maximum nor a minimum.
Example
Minimize
f x 3x 6x1x2 5x 7x1 5x2
2
1
2
2
Subject to x1+ x2 = 5
Solution
g1(X) = x1+ x2 -5 =0
L(x1, x2, ……..xn, λ1 λ2…… λm)
= f(X) + λ1 g1(X) + λ2 g2(X) + …… + λm gm(X)
With n=2 and m=1
Example
L 3 x12 6 x1 x 2 5 x 22 7 x1 5 x 2 1 x1 x 2 5
L
6 x1 6 x 2 7 1 0
x1
1
x1 x 2 7 1
6
1 23
Example
L
6 x 1 10 x 2 5 1 0
x 2
1
3 x1 5 x 2 5 1
2
1
3 x1 x 2 2 x 2 5 1
2
1 11
x2 x1
2 2
L11 L12 g11
L21 L22 g12
0
g11 g12 0
L2
L 2
L11 2 6 L22 2 10
x1 X , x2 X ,
L 2
L12 L21 6
x1 x2 X ,
g1 g1
g11 1, g12 1
x1 X , x2 X ,
6 6 1
6 10 1
0
1 1 0
6 1 6 1 1 6 10 0
2
Since is negative, (X )corresponding to a maximum.
Example
Find the dimensions of the cylindrical tin (with top and
bottom) made up of sheet metal to maximize its volume
such that the total surface area is equal to A0= 24π
Solution:
If x1 and x2 denote the radius of the base and length of the
tin respectively, Then the problem can be stated as:
Maximize f (x1, x2) = π x12 x2
(Volume of the cylinder = π r2 l)
Subject to, 2 π x12 + 2π x1x2 = A0= 24π
(Surface area of cylinder = 2 π r2 + 2π r l )
Solution
The Lagrange function is,
L x 12 x 2 ( 2 x 12 2 x 1 x 2 A 0 )
The necessary conditions for the maximum of f give,
L
2 x 1 x 2 4 x 1 2 x 2 0 1
x1
L
x 12 2 x 1 0 2
x2
L
2 x 12 2 x 1 x 2 A 0 0 3
Solution
From (1) x1 x 2
2 x1 x 2
x1
From (2)
2
x
By equating both λ we get, x 2
1 4
By substituting (4) in (3) we get, 2
1 1 1
A0 2 2 A0 2 A0 2
x1 x2
6 3 24
1
A0 3 2
f
54
Solution
If A0= 24π, the optimum solution becomes
x 2, x 4, 1, f 16
1 2
To see that this solution really corresponds to the
maximum of f, we apply the sufficiency condition .
L11 L12 g11
L21 L22 g12
0
g11 g12 0
Solution
2L 2L
L11 4 L 22 0
x1 X ,
2
x 2 X ,
2
2L
L12 L 2 1 2
x1 x 2 X ,
g1 g1
g11 16 , g12 4
x1 X , x2 X ,
4 2 16
2 0 4
0
16 4 0
272 2 192 3
0
12
17
Since the value of is negative, the point (x1* , x2* )
corresponds to the maximum of f.
THANK YOU
