Model III : Additive separable return function and single additive constraint

Consider the problem :

Minimize Z ={ f 1 ⋅ ( d 1) + f 2 ( d 2 ) +… ⋅+ f n ( ⅆn )}

Subject to the constraint

a 1 d 1 +a 2 d 2 +a n d n >b

Proceed in the same manner as in model II. Defining state variable s1 , s 2 , s 3 , …… . sn such that

Sn=a1 d 1+ a2 d 2+ an d n Sn=a1 d 1 a 2 d 2 +a n d n ≥b

Sn =Sn−a n ⅆn
−1

s j−1=s j−a d ; j=1,2,3,…..n

j j

n
Let f n ¿¿ min ∑ f j ¿¿ ¿ ¿
df > 0 n=1

Such that sn ≥ b

The general recursive equation for obtaining the minimum value of Z , for all decision variable and for
feasible value of all decision variable is given by :

f j s j =min ¿ f j ¿( s j ¿ +f j ( sj ) ¿ }
dj>0

f 1 ( s1 ) =f 1 ( d 1 )

Where S j−1=t j ( s j ⅆj )

Example 22.7 Use dynamic programming to show that :

P1 log ❑ P1 + P2log p2 + ….. + pnlog pn

Subject to constraint

p1+ p2+……+ pnand pi ≥ 0 , for all I is minimum when

 1
p1= p2= ……= pn =
n

Solution: the problem is divide unity into n parts , p1, p2, ….. , pn such that the quantity

n
f n(1) = ∑ pi log pi
i=1

Is minimum where f n(1) denotes the minimum attainable sum of pi log pi (i= 1, 2, 3 ….,n ), where 1 is
divided into n parts

For n = 1 (stage 1 ) we have

f 1(1) = pi log pi = 1 log 1 , as unity is divided into p1 = 1 part

For n = 2 (stage 2) the unity is to be divided into two parts p1∧¿ p2 such that p1+ p2= 1 and p1=z

and p2= 1- z, then :

f 2(1) = min { pi log p i+ p 2 log p2 }

0< z ≤ 1

min ¿ ¿
= 0< z≤ 1

Similarly in general , for an n-stage problem ,where unity is to be divided nto n parts , the recursive
equation is ,

min { p log p + p log p +…+ p log p

f n(1) = 0< z≤ 1 i i 2 2 n n

min ¿ ¿
= 0< z≤ 1

Solution of Recursive equation the solution to the above recursive equation to get optimal policy can
be obtained by the differential calculus

The minimum value of recursive equation f z = z log z + ( 1-z ) is attained a z = ½

f 2(1) =
1
2
1 1
( ) ( ) ( )
1
log + 1− log 1− = 2 1−
2 2 2
1
2

Similarly , for stage 3 , the minimum value of recursive equaltion

min ¿ ¿
f 3 (1) = 0< z≤ 1

{
= min z log z +2
0< z ≤ 1
( 1−z
2 ) log (
2 )}
1−z
 1
is attained at z = 1/3 satisfying the condition 0<z<1 . Hence for stage 3 , the optimal policy is z 1 = z 3 =
3
and

( ) ( )
1 1
f 3 (1) =
1
3
log
1
3
+
1−
2
()
3 log
1−
2
3

=
1
3
log
1
3
+2
1
3() {
log
1
3
=3
1
3
log
1
3 } { }
Thus , in general , the optimal policy is

p1= p2= ……= pn =

1
n
and f n(1) = n
1
n { ( )}
log
1
n

The above result can be proved for any value of n by using mathematical induction n = m+1

The Recursive equation is

f m+1 = min { z log z+ f m (1− z) }

0< z ≤ 1

{
= min z log z +m
0< z ≤ 1
{ 1−z
m
log
m }}
1−z

Since the minimum of the function

F(z) = log z + {1−zm log 1−zm }

Is attained for z = 1 ( m + 1 ) , therefore using the concept of maximum and minimum we have

f m+1 =
1
m+1
log
1
m+1
+m
1
m+1( ) {
log
1
m+ 1 ( )}
=m+1 {m+11 log m+1
1
}
Hence , the result is also true for n = m + 1

Example 22.8 Use the dyanamic programming to solve the following problem
2 2 2
Minimize Z = y 1 + y 2 + y 3

Subject to the constrait

 y 1+ y 2 + y 3 = 10 and y 1 y 2 y 3 ≥ 0

Solution Let the state variable be s1 , s2, and s3 such that

s3 = y 1+ y 2 + y 3 = 15 ; s2 = y 1+ y 2= s3− y 3 s1= y 1= s2−¿ y 2

The recursive equation can now be written as :

f 3( s3 ) = min { y 3+ f 2 ( s 2 ) }
2

y3

f 2 ( s 2) = min { y2 + f 1 (s 1 ) }
2

y2

f 1 ( s1 ¿ = min { y1 } = y 21 ¿(s 2− y 2)2

2

y 1

The recursive equation f 2 ( s 2) can also be expressed as

f 2 ( s 2) =min { y 22 +(s 2− y 2 )2}
y2

Now , by using the concept of maximum and minimum in differential calculus , the minimum
value f 2 ( s 2) can be obtained as explained below

Differentiating f 2 ( s 2) with respect to y 2and equating to 0 , we get

2( y 2) - 2(s2 − y 2) = 0

( ) { }
22
s2 s2
Thus , f 2 ( s 2) = + s 2−
2 2

{ }
2
s2
f 3( s3) = min { y 3+ f 2 ( s 2 ) } = min
2
2
y y 3+
y 3 3
2

= min
y
y 23+
s2− y
2
3 { 3

}
Differentiating f 3( s3) with respect to y 3 and equating to zero to get its minimum value

2 y 3 - ( s3 – y 3) or y 3= f 3( s3 /3)

( ) ( )
2 2 3
s3 1 s3 s
Thus , f 3( s3) = + s 3− = 3
3 2 3 3

But for minimum value of f 3( s3) we must have y 1+ y 2 + y 3 = 10 . Therefore

 f 3( s3 ) = (10)2/3 = 100/3, and

y 3 = s3/ 3 = 10/3

y 2= s2/2 = ( s3 – y 3)/2 = {10-(10/3)}/2 = 10/3

y 1= s1/2 = ( s3 – y 3) – y 2= { 10 – {10/3}} – 10/3 = 10/3

2 2 2
Hence the minimum value of y 1 + y 2 + y 3 is 100/3 for y 1 = y 2 = y 3 = 10/3