Chapter #7: Optimization

Math Econ (MEF)

Raúl Bajo ([email protected])

Notation
• Let K be a subset of R and f : F → R. We write minK f = minx∈K f (x)
(or, more technically, inf K f = inf x∈K f (x) ) for the minimization program
but also its optimal value. And arg minK f = arg minx∈K f (x) is the
(possible empty) set of global minima, or simply solutions.

• A local minimum of the function f on K is a point a ∈ K s.t.

∃ε > 0 s.t. a ∈ arg minx∈K∩B(a,ε) f (x)
• Symmetric definitions apply to supK f (or, more technically, supK f ),
arg maxK f , local and global maximum.

Continuous functions
Assume: K is compact (closed and bounded) in the metric space (E, d) and
f is continuous on K. Then arg minK f is non-empty and compact (and so is
arg maxK f ).

• Maximum Theorem I: Assume K is compact in (E, d), Y is closed in

Rn , and f : K × Y → R is continuous. Then F (y) = inf x∈K f (x, y) is
continuous on Y . And ϕ(y) = arg minx∈K f (x, y) is compact valued and
upper hemicontinuous.
• Maximum Theorem II: Assume Y ⊆ Rn , µ is a correspondence Y →→
Rm , µ(y) is a compact for all y, and f : µ(Y ) → R a continuous function.
If µ is upper and lower hemicontinuous at y, then F (y) = inf x∈µ(y) f (x)
is continuous at y and ϕ(y) = arg minx∈µ(y) f (x) is compact valued and
upper hemicontinuous at y.

Convex functions
• Assume that K is a convex subset of R and f : K → R is convex. If
x ∈ int K (interior point) is a local minimum, then x is a global minimum
as well.
Similarly, if f is concave an (interior) local maximum is a global maximum.
• For any convex program (K convex, f convex) the set of solutions arg minK f
is convex (even a single point at most if f is strictly convex).

1
Differentiable functions
Some general ideas that are important in optimization.

• First order conditions

. Assume K ⊆ Rn , and a ∈ int K is an interior point, and f : K → Rn

is differentiable at a. If a is a local minimum (or maximum) of f on
K, then f 0 (a) = 0.

• Second order conditions

. In addition to the assumptions above, suppose that f is twice differ-

entiable at a. If a is a local minimum of f on K, then f 00 (a) is positive
(f 00 (a)(x, x) ≥ 0 for all x). Similarly, if a is a local maximum, then
f 00 (a) is negative.

However, these are necessary but not sufficient conditions. E.g. f (x) =
x3 at a = 0 shows that the combination of the first and second order
conditions are not sufficient to guarantee a local minimum.
• The following is a sufficient condition that guarantees a local minimum: if
f 0 (a) = 0 and f 00 (a) is definite positive (f 00 (a)(x, x) > 0 for x 6= 0), then
a is a local minimum of f .

Unconstrained optimization
Note: For this and the remaining sections, we will assume all functions are twice
continuously differentiable.

• We consider the problem of finding the set of values x1 , x2 , · · · , xn that

maximize the function f (x1 , x2 , · · · , xn ). We can write this as follows

max f (x)
x

where x is an n-dimensional vector. This is considered an unconstrained

optimization problem, since no restrictions are placed on the values of x.

• First order necessary conditions (FOC or FONC):

fx (x∗ ) = 0 i = 1, · · · , 0

or, more compactly

fx (x∗ ) = 0

f reaches a global maximum at x∗ if f (x∗ ) ≥ f (x). If this condition does

not hold, we say f has reached a local maximum at x∗ .
• Second order sufficient conditions (SOC): the Hessian matrix of f eval-
uated at x∗ must be negative semidefinite. That is, if fx (x∗ ) = 0 and
fxx (x∗ ) is negative semidefinite, then we can state that f (x) reaches a
maximum at x∗ .

2
 • Suppose we want to find the set if values x1 , x2 , · · · , xn that minimize the
function g(x1 , x2 , · · · , xn ). We can write this as follows

min g(x)
x

In this case, we can solve the equivalent problem

max −g(x)
x

and use the same FOC as above. SOC now requires the Hessian matrix
to be positive semidefinite.
• If the Hessian matrix is neither positive semidefinite nor negative semidef-
inite, then the solution obtained is a saddle point.

Constrained optimization with equality constraints

• We consider an objective function f of n variables with m (equality) con-
straints, where m < n.

max f (x1 , · · · xn )
x1 ,···xn

subject to g 1 (x1 , · · · xn ) = 0
g 2 (x1 , · · · xn ) = 0
..
.
g m (x1 , · · · xn ) = 0

Or, more compactly, in matrix notation

max f (x)
x
subject to g(x) = 0

• To solve this optimization problem, we can apply the Lagrangian method.

The Lagrangian is a function of n + m variables built as follows
m
X
L(x, Λ) = f (x) + λj g j (x)
i=1

By taking the FOC, we will have a system of n+m equations to determine

x∗ and Λ∗ through our FOC.
m
∂L ∂f (x)∗ X ∗ ∂g(x∗ )
= + λi =0 i = 1, · · · , n
∂xi ∂xi i=1
∂xi

∂L
= g j (x)∗ ) = 0 j = 1, · · · , m
∂λj

• Check SOC (if necessary).

3
 • The Lagrange multiplier(s) has/have an economic interpretation, and it/they
is/are usually called the shadow price.

• The Envelope Theorem (very important!)

Consider the problem:

max f (x, a) s.t. g(x, a), x ∈ Rn+

x

where x is a vector of choice variables, a >> 0 is a vector of parameters

that may enter the objective function, the constraint(s), or both.
Suppose that the objective function and the constraints are constinuously
differentiable at a. Let L(x, a, λ) be the associated Lagrangian with so-
lution (x∗ (a), λ∗ (a)). Define the associated maximum-value function as
follows:
M (a) ≡ f (x∗ , a)

Then,
∂M (a) ∂L(x∗ , a, λ∗ )
= for j = 1, · · · , m
∂aj ∂aj

Constrained optimization with inequality constraints

• We consider the most general case, in which there are both equality and
inequality constraints in the optimization problem.

max f (x)
x∈Rn |{z}
objective function

subject to g i (x) ≥ 0 i = 1, · · · , I
| {z }
inequality constraints

hj (x) = 0 j = 1, · · · , J
| {z }
equality constraints

• We can use the Lagrange multiplier as previously discussed for the equal-
ity constraints. But for the inequality constraints, we used the so-called
Kuhn-Tucker conditions.
• Let µi be the multiplier for the inequality constraints and λj be the mul-
tipliers for the equality constraints. The FOC for all constraints are as
follows
XI J
X
5f (x∗ ) + µi 5 g i (x∗ ) + λj 5 hj (x∗ ) = 0
i=1 j=1

With the following additional conditions for the inequality constraints

λ ≥ 0, µ ≥ 0, g i (x) ≥ 0, µi g i (x) = 0