Dynamic Optimization

Mohammed Bouaddi

American University in Cairo

October 20, 2022

MB (Business School) Econ 4061 - Mathematical Economics October 20, 2022 1/1
This part is based on chapter 25 of the book Mathematical for Economics
of Michael-Hoy et. al (second edition)
Outline
I. Static versus dynamic optimization
II. Optimal control theory
III. The maximum principal
IV. The Hamiltonian function
V. Economic examples

MB (Business School) Econ 4061 - Mathematical Economics October 20, 2022 1/1
I. Static versus dynamic optimization
We assume that the …rm’s output depends on the amount of capital only

Y = f (K )

The pro…t is given by

π (K ) = pf (K ) RK

where π is the pro…t function, p is the price per unit of output and R the
rent of capital per unite of capital. It is assumed that there is no contraint
on the quantity of capital.

MB (Business School) Econ 4061 - Mathematical Economics October 20, 2022 1/1
The quantity of capital that maximizes the current pro…t is geven by FOC:

π 0 (K ) = pf 0 (K ) R=0

This approach of modeling the behaviour of a …rm is very restrictive. Why

the …rm’s problem should focus on maximizing the curent pro…t while it
life time is longer? The more realistic strategy of the …rm is to miximize
discounted future pro…ts as well. Thus, the problem should be a maximum
of the discounted some of current and future pro…ts util a given time T in
the future: dynamic optimization problem.

II. Optimal control theory

The …rm’s problem is:
ZT
max V (K (t )) = exp ( ρt ) π (K (t )) dt
0

where ρ is the discount rate (degree of time preference) and exp ( ρt ) is

the discounting factor. High value ρ of means that the more recent
periods pro…ts are more valuable.
MB (Business School) Econ 4061 - Mathematical Economics October 20, 2022 1/1
The solution of this problem amounts of choosing the capital path K (t )
as function of time to maximize the V (K (t )).

Investment example
An optimization problem is dynamic if decisions taken in the current
period a¤ect current pro…t as well as pro…ts in future periods.
Instead of renting capital, let assume now that the …rm purchases the
capital. This means that the cost of capital should be split accros di¤erent
periods in the future since the …rm will use it not only in the current
period but also in the future until it is completely depreciated. The change
is capital accros time is given by:
.
K = I (t ) δK (t )
where I (t ) represents the current investment and δ is the depreciation
rate of capital.
The …rm’s pro…t function is given by
π (K (t ) , I (t )) = pf (K (t )) c (I (t ))
where c (I (t )) is the function that gives the cost of investing. This
problem is dynamic because current investment a¤ects the current
MB (Business School) Econ 4061 - Mathematical Economics October 20, 2022 1/1
production cost but it also a¤ects future pro…ts because it increases the
capital stock that will last several periods in the future.
The objective of the …rm is:

ZT
max V (I (t )) = exp ( ρt ) π (K (t ) , I (t )) dt
0

such that .
K = I (t ) δK (t )
K (0) = K0
where is the initial capital at time zero. The …rm will choose how much to
invest in order to maximize discounted fro…ts from time 0 to time T .
Onece the optimal investment path is determined, the capital path will be
completely determined. To solve this problem, we will use optimal
control theory.

MB (Business School) Econ 4061 - Mathematical Economics October 20, 2022 1/1
III. The maximum principal
De…nition
Dynamic optimization problem with a …nite time horizon and a free
endpoint in continuous-time models has the following general form:

ZT
max V (y (t )) = f (x (t ) , y (t ) , t ) dt
0

such that
.
x = g (x (t ) , y (t ) , t )
x (0) = x0 > 0. Given initial condition
where x (t ) is the state variable, y (t ) is the control variable.

The state and the control variable are related by the di¤erential equation.
The word control means that this variable is the optimazed one (choice
variable). Free endpoint means that x (T ) can be any value chosen
optimally.
MB (Business School) Econ 4061 - Mathematical Economics October 20, 2022 1/1
IV. The Hamiltonian function
Similar to constrained optimization problems where we use the Lagrangean
function to solve the problem, The necessary conditions to solve the
dynamic optimization problem are based on the Hamiltonian function.

De…nition
For dynamic optimization problem, the Hamiltonian function is given by:

H (x (t ) , y (t ) , λ (t ) , t ) = f (x (t ) , y (t ) , t ) + λ (t ) g (x (t ) , y (t ) , t )

where λ (t ) is the costate variable.

On the following theorem, we state the necessary conditions to solve the

problem. We assume that the Hamiltonian function is strictly concave in
y (t ) and some regularity assumptions about the functions f and g to
avoid second-order conditions.
MB (Business School) Econ 4061 - Mathematical Economics October 20, 2022 1/1
Theorem
For the dynamic optimization problem, the optimal solution path for the
control variable y (t ) must satisfy the following necessary conditions:
1 The control variable y (t ) is chosen to maximize H at each point in
time:
∂H
= 0.
∂y
2 The paths of x (t ) and λ(t ) are given by the solution of the system of
di¤erential equations:
. ∂H
λ=
∂x
.
x = g (x (t ) , y (t ) , t )
3 The boundary conditions used to solve the system of di¤erential
equations are: x (0) = x0 > 0 (Given initial condition) and
λ (T ) = 0 (Tranversality condition).

MB (Business School) Econ 4061 - Mathematical Economics October 20, 2022 1/1
Example:

Z2
max 5x (t ) 3y (t )2 dt
0

such that
.
x = 2y (t ) 4
and
x (0) = 0.
Answer:

H (x (t ) , y (t ) , λ (t ) , t ) = 5x (t ) 3y (t )2 + λ (t ) (2y (t ) 4)

The Hamiltonian is strictly concave in y (t ). Thus we have the following

necessary and su¢ cient condition:

∂H
= 6y (t ) + 2λ (t ) = 0.
∂y
MB (Business School) Econ 4061 - Mathematical Economics October 20, 2022 1/1
It follows that
λ (t ) = 3y (t )
. ∂H
λ= = 5
∂x
.
x = 2y (t ) 4

This is a free endpoint problem because the value for x (2) is not speci…ed.
Therefore, we obtain the following boundary conditions:

x (0) = 0 and λ (2) = 0.

Z . Z Z
∂H
λ (t ) = λdt = dt = 5dt = 5t + C
∂x
λ (2) = 5 2 + C = 0 =) C = 10

Thus,
λ (t ) = 5t + 10.
MB (Business School) Econ 4061 - Mathematical Economics October 20, 2022 1/1
In the other hand, we have
Z Z Z
. λ (t )
x (t ) = xdt = (2y (t ) 4) dt = 2 4 dt
3
Z Z
5t + 10 10 8 8 5 2
= 2 4 dt = t+ dt = t t +C
3 3 3 3 3
Using the initial condition we get
5 8
x (0) = 02 + 0+C = 0
3 3
It follows that C = 0 and
5 2 8
x (t ) = t + t.
3 3
We have the following path of the control variable
λ (t ) 5t + 10
y (t ) = = .
3 3
10
At time zero, y start at 3 and declines until it reaches the value of 0 and
the end.
MB (Business School) Econ 4061 - Mathematical Economics October 20, 2022 1/1
Optimization under discounting
Usually, in economics optimization involves discounting. The Hamiltonian
function is

H (x (t ) , y (t ) , λ (t ) , t ) = f (x (t ) , y (t ) , t ) + λ (t ) g (x (t ) , y (t ) , t )

and we assume that the objective function has the following form

f (x (t ) , y (t ) , t ) = exp ( ρt ) F (x (t ) , y (t )) .

where ρ is the discount rate. We notice that the time enters explicitely in
the problem via discounting only (exp ( ρt )). The optmization problem
becomes
ZT
max exp ( ρt ) F (x (t ) , y (t )) dt
0

such that
autonomous such that
.
x = g (x (t ) , y (t ) , t ) = g (x (t ) , y (t )) di¤erential equation is autonomous
MB (Business School) Econ 4061 - Mathematical Economics October 20, 2022 1/1
 x (0) = x0 > 0. Given initial condition
The Hamiltonian function is

H (x (t ) , y (t ) , λ (t ) , t ) = exp ( ρt ) F (x (t ) , y (t )) + λ (t ) g (x (t ) , y (t ))

The …rst order necessary condition is

∂H ∂F (x (t ) , y (t )) ∂g (x (t ) , y (t ))
= exp ( ρt ) + λ (t ) = 0. (1)
∂y ∂y (t ) ∂y (t )

Equivalently we have:

∂F (x (t ) , y (t )) ∂g (x (t ) , y (t ))
+ λ (t ) exp (ρt ) = 0. (2)
∂y (t ) ∂y (t )

The second necessary condition is

. ∂H ∂F (x (t ) , y (t )) ∂g (x (t ) , y (t ))
λ= = exp ( ρt ) λ (t ) (3)
∂x ∂x (t ) ∂x (t )

MB (Business School) Econ 4061 - Mathematical Economics October 20, 2022 1/1
Let µ (t ) = λ (t ) exp (ρt ) , we get
. .
µ = λ exp (ρt ) + ρλ (t ) exp (ρt )
.
= λ exp (ρt ) + ρµ (t ) . (4)

Equation 2 implicitly gives y (t ) as a function of x (t ) and λ (t ) exp (ρt )

which we can write as

y (t ) = h [x (t ) , λ (t ) exp (ρt )] .

From 3 we have
. ∂H ∂F (x (t ) , y (t )) ∂g (x (t ) , y (t ))
exp (ρt ) λ = = λ (t ) exp (ρt )
∂x ∂x (t ) ∂x (t )
(3)
and using equation 4 we get:
. ∂F (x (t ) , h [x (t ) , λ (t ) exp (ρt )])
µ ρµ (t ) =
∂x (t )
∂g (x (t ) , h [x (t ) , λ (t ) exp (ρt )])
µ (t )
∂x (t )
MB (Business School) Econ 4061 - Mathematical Economics October 20, 2022 1/1
It follows that
. ∂F (x (t ) , h [x (t ) , λ (t ) exp (ρt )])
µ = ρµ (t )
∂x (t )
∂g (x (t ) , h [x (t ) , λ (t ) exp (ρt )])
µ (t ) .
∂x (t )
For this problem the transformed Hamiltonian isThe Hamiltonian function
is

H (x (t ) , y (t ) , λ (t ) , t ) = exp (ρt ) H (x (t ) , y (t ) , λ (t ) , t )
= F (x (t ) , y (t )) + λ (t ) exp (ρt ) g (x (t ) , y (t ))
= F (x (t ) , y (t )) + µ (t ) g (x (t ) , y (t ))

The necessary conditions for solving the problem are:

1 The control variable y (t ) is chosen to maximize H at each point in
time:
∂H
= 0.
∂y
MB (Business School) Econ 4061 - Mathematical Economics October 20, 2022 1/1
 2 The paths of x (t ) and λ(t ) are given by the solution of the system of
di¤erential equations:
. ∂H
µ ρµ (t ) =
∂x
.
x = g (x (t ) , y (t ))
3 The boundary conditions used to solve the system of di¤erential
equations are: x (0) = x0 > 0 (Given initial condition) and
µ (T ) exp ( ρT ) = 0 (Tranversality condition).

Example with discounting:

Z2
max exp ( ρt ) 2x (t ) 3x (t )2 3y (t )2 dt
0
such that
.
x= 2y (t ) x (t )
and
x (0) = 10.
MB (Business School) Econ 4061 - Mathematical Economics October 20, 2022 1/1
Answer:

H (x (t ) , y (t ) , λ (t ) , t ) = 3x (t )2 3y (t )2 + µ (t ) (2y (t ) 4x (t ))

∂H
= 6y (t ) + 2µ (t ) = 0.
∂y
µ (t )
y (t ) =
3
.
µ ρµ (t ) = 6x (t ) + 4µ (t )
.
µ = 6x (t ) + (4 + ρ) µ (t )
. 2
x= 2y (t ) 4x (t ) = µ (t ) x (t )
3
System of di¤erential equations
.
µ (4 + ρ ) 6 µ (t )
. = 2
x 3 1 x (t )
MB (Business School) Econ 4061 - Mathematical Economics October 20, 2022 1/1
 x (0) = 10 and µ (2) = 0
Eigenvalus
4+ρ τ 6
2 = (4 + ρ τ) ( 1 τ) + 4
3 1 τ
= τ2 (3 + ρ ) τ ρ=0

∆=((3 + ρ))2 4 1 ( ρ) = ρ2 + 10ρ + 9 > 0

p p
3+ρ ρ2 + 10ρ + 9 3 + ρ + ρ2 + 10ρ + 9
τ1 = and τ 2 =
2 2
Eigenvectors p
3 +ρ ρ2 +10ρ+9
For τ 1 = 2
p
4 6 3+ρ ρ2 + 10ρ + 9
2 V1 = V1
3 1 2
0 p 1
3 +ρ ρ2 +10ρ+9
@ 4 2 6p A V1,1
=0
2 3 +ρ ρ2 +10ρ+9 V1,2
3 1 2
MB (Business School) Econ 4061 - Mathematical Economics October 20, 2022 1/1
 p
3 +ρ ρ2 +10ρ+9
4 6p 5 1
2 = ρ + ρ2
2 3 +ρ ρ2 +10ρ+9 2 2
3 1 2
q
1
ρ ρ2 + 10ρ + 9
2
The
0 determonant
p 2 of the matrix 1
3 +ρ ρ +10ρ+9
@ 4 2 6p A is zero only when
2 3 +ρ ρ2 +10ρ+9
3 1 2
ρ = 0.

Case 1: ρ 6= 0
It follows that
V1,1 = V1,2 = 0
0
therefore the vector V1 = as the eigenvector corresponding to
0
p2
3 +ρ ρ +10ρ+9
eigenvalue τ 1 = 2 .

MB (Business School) Econ 4061 - Mathematical Economics October 20, 2022 1/1
Case 2: ρ = 0
It follows that
0 p2 1
3 +ρ ρ +10ρ+9
@ 4 2 6p A V1,1
= 0
2 3 +ρ ρ2 +10ρ+9 V1,2
3 1 2
4 6 V1,1
2 = 0
3 1 V1,2
In this case, equation one of the system is exactely equal to equation 2
times -6. Therefore, V1,2 = 23 V1,1 by taking V1,1 = 1 we get
1
V1 = 2 as an eigenvector corresponding to τ 1 = 0.
3 p2
3 +ρ+ ρ +10ρ+9
For τ 2 = 2
p
4 6 3 + ρ + ρ2 + 10ρ + 9
2 V1 = V2
3 1 2
0 p 1
3 +ρ+ ρ2 +10ρ+9
@ 4 2 6p A V2,1 =0
2 3 + ρ + ρ 2 +10ρ +9
V2,2
3 1 2
MB (Business School) Econ 4061 - Mathematical Economics October 20, 2022 1/1
 p
3 +ρ+ ρ2 +10ρ+9
4 6p 5 1
2 = ρ + ρ2
2 3 +ρ+ ρ2 +10ρ+9 2 2
3 1 2
q
1
+ ρ ρ2 + 10ρ + 9
2
The
0 determonant
p of the matrix 1
3 +ρ+ ρ2 +10ρ+9
@ 4 2 6p A is zero only when
2 3 +ρ+ ρ2 +10ρ+9
3 1 2
ρ = 0.

Case 1: ρ 6= 0
It follows that
V2,1 = V2,2 = 0
0
therefore the vector V2 = as the eigenvector corresponding to
0
p2
3 +ρ+ ρ +10ρ+9
eigenvalue τ 1 = 2 .

MB (Business School) Econ 4061 - Mathematical Economics October 20, 2022 1/1
Case 2: ρ = 0
It follows that
0 p 1
3 +ρ+ ρ2 +10ρ+9
@ 4 2 6p A V1,1
= 0
2 3 +ρ+ ρ2 +10ρ+9 V1,2
3 1 2
1 6 V1,1
2 = 0
3 4 V1,2

In this case, equation one of the system is exactely equal to equation 2

times 32 . Therefore, V1,2 = 6V1,1 by taking V2,1 = 1 we get
1
V1 = as an eigenvector corresponding to τ 1 = 3.
6
Conclusion: The solution to the system is:
Case 1: if ρ 6= 0 then
p p
µ (t ) 3 +ρ ρ2 +10ρ+9 0 3 +ρ+ ρ2 +10ρ+9 0
t t
= c2 e 2 + c2 e 2
x (t ) 0 0
0
= .
0
MB (Business School) Econ 4061 - Mathematical Economics October 20, 2022 1/1
 µ (t ) = 0
x (t ) = 0
µ (t )
y (t ) = =0
3

Case 2: if ρ = 0 then

µ (t ) 1 3t 1
= c1 3 + c2 e
x (t ) 2 6

3 3t
µ (t ) = c1 6c2 e
2
x (t ) = c1 + c2 e 3t

3 3
µ (2) = c1 6c2 e 3 2 = c1 6e 6
c2 = 0
2 2
x (0) = c1 + c2 e 3 0 = c1 + c2 = 10
MB (Business School) Econ 4061 - Mathematical Economics October 20, 2022 1/1
 c2 = 10 c1
3 6
c1 6e (10 c1 ) = 0
2
e 6
c1 = 40 6
4e 1
e 6
c2 = 10 40
4e 6 1
10
=
4e 6 1
The solution to the system becomes
40e 6 3 10
µ (t ) 2 3t 6
= e
x (t ) 4e 6 1 1 4e 6 1 1

3 40e 6 60
µ (t ) = + e 3t
2 4e 6 1 4e 6 1
60e 6 60
= 6
+ 6
e 3t
MB (Business School)
4e 1 4e
Econ 4061 - Mathematical Economics
1 October 20, 2022 1/1
 40e 6 10 3t
x (t ) = e
4e 6 1 4e 6 1
60e 6
µ (t ) 4e 6 1
+ 4e 606 1 e 3t
20e 6 20 3t
y (t ) = = = + e
3 3 4e 6 1 4e 6 1
40e 6 10
At time zero, x starts at x (0) = 4e 6 1
= 10 and declines until it
40e 6 10 6
reaches x (2) = 4e 6 1 4e 6 1
e 3 2 = 4e30e6 1 = 0.07 510 7 at the
20e 6 20
end of the second period. While y starts at y (0) = 4e 6 1
= 20. 15
20e 6 20
and increases until it reaches y (2) = + 4e 6 1 e at the 3 2 =0
4e 6 1
end of the second period.
V. Economic example: An optimal consumption Model
Let x0 be the amount of money an an individual has in a bank account at
time zero. The bank gives an interest rate of r on the deposits. Let c (t )
be the amount of money used for consumption at time t which is the only
source of consumption. The instantanuous change of bank account is
.
x = rx (t ) c (t )

We assume that the individual chooses the consumtion path to maximize

MB (Business School) Econ 4061 - Mathematical Economics October 20, 2022 1/1
her life time utility function given by

ZT
max exp ( ρt ) U (c (t )) dt
0

where U (c (t )) is the instantanuous utility function. We assume that the

utility function has the following form

U (c (t )) = ln (c (t ))

The problem becomes:

ZT
max exp ( ρt ) ln (c (t )) dt
0

such that
x (0) = x0 > 0
x (T ) = b
MB (Business School) Econ 4061 - Mathematical Economics October 20, 2022 1/1
Answer:

H (x (t ) , y (t ) , λ (t ) , t ) = ln (c (t )) + µ (t ) (rx (t ) c (t ))

∂H 1
= µ (t ) = 0.
∂c c (t )
1
c (t ) =
µ (t )
.
µ ρµ (t ) = r µ (t )
The system of di¤erential equations is given by:
.
µ = (r ρ ) µ (t )
.
x = rx (t ) c (t )
which can be rewritten as
.
µ = (r ρ ) µ (t )
. 1
x = rx (t )
µ (t )
MB (Business School) Econ 4061 - Mathematical Economics October 20, 2022 1/1
The …rst equation is just a linear homogenous di¤erential equation for
which the solution is given by
µ (t ) = C1 exp ( (r ρ) t )
The second equation becomes
. 1
x = rx (t )
C1 exp ( (r ρ) t )
. 1
x rx (t ) =
exp [(r ρ) t ]
C1
This is a non-autonomous …rst-order di¤erential equation. Multiply both
sides of the last equation by exp ( rt ), we get
. 1
x exp ( rt ) rx (t ) exp ( rt ) = exp [(r ρ) t ] exp ( rt )
C1
1
= exp ( ρt )
C1
Observe that (Calculuc I)
. d [x (t ) exp ( rt )]
x exp ( rt ) rx (t ) exp ( rt ) = .
MB (Business School) Econ 4061 - Mathematical Economics
dt October 20, 2022 1/1
Thus, we have
Zt
1
x (t ) exp ( rt ) = exp ( ρs ) ds + C2
C1
0
1 exp ( ρs ) t
= + C2
C1 ρ 0
1
= (exp ( ρt ) 1) + C2 .
C1
It follows that
1 exp ( ρt ) 1
x (t ) = exp (rt ) + C2 exp (rt ) .
C1 ρ

Using initial conditions, we get

1 exp ( ρt ) 1
x (t ) = exp (rt ) + C2 exp (rt ) .
C1 ρ

x (0) = x0 = C2
MB (Business School) Econ 4061 - Mathematical Economics October 20, 2022 1/1
and

1 exp ( ρT ) 1
x (T ) = b = exp (rT ) + C2 exp (rT )
C1 ρ
1 exp ( ρT ) 1
= exp (rT ) + x0 exp (rT )
C1 ρ

(exp ( ρT ) 1) exp (rT )

C1 =
ρ (b x0 exp (rT ))

We get

µ (t ) = C1 exp ( (r ρ) t )
(exp ( ρT ) 1) exp (rT )
= exp ( (r ρ) t )
ρ (b x0 exp (rT ))
(exp ( ρT ) 1)
= exp (ρt )
ρ (b x0 exp (rT ))
MB (Business School) Econ 4061 - Mathematical Economics October 20, 2022 1/1
 1 exp ( ρt ) 1
x (t ) = (exp ( ρT ) 1 ) exp (rT )
exp (rt ) + x0 exp (rt )
ρ
ρ(b x0 exp (rT ))
(b x0 exp (rT )) (exp ( ρt ) 1)
= exp (rt ) + x0 exp (rt )
( exp ( ρT ) 1) exp (rT )

The consumption path is

1 1
c (t ) = = exp ((r ρ) t )
µ (t ) C1
1
= (exp ( ρT ) 1 ) exp (rT )
exp ((r ρ) t )
ρ(b x0 exp (rT ))
(b x0 exp (rT ))
= ρ exp ( ρt )
(exp ( ρT ) 1)
(x0 exp (rT ) b )
= ρ exp ( ρt )
(1 exp ( ρT ))
MB (Business School) Econ 4061 - Mathematical Economics October 20, 2022 1/1
In the case of ρ = r and x0 = b, we have

(exp ( rT ) 1) exp (rT )

µ ( t ) = C1 =
r (x0 x0 exp (rT ))
1
=
rx0

x (t ) = x0
1
c (t ) = = rx0
µ (t )
and as a results the bank acount will not change and the consumption
expense at any time will be just the interests earned.

MB (Business School) Econ 4061 - Mathematical Economics October 20, 2022 1/1