Drawbacks of traditional Optimization Techniques:
• Final solution is dependant in the initially chosen random solution. No guarantee
that final solution is global optimum
• Discontinuous objective functions cannot be solved using gradient-based methods
• A particular optimization method is applicable to particular problem.
DIFFERENTIAL EVOLUTION
DE is an evolutionary algorithm
DE is population based stochastic method for global optimization. (by Storn and Price,
1995)
STEPS IN DIFFERENTIAL EVOLUTION
INITIALIZATION Xi,g= lb + (ub - lb).rand (0,1)
MUTATION Vi,g = Xr1,g + F.(Xr2,g - Xr3,g)
CROSSOVER {
𝑈 𝑖 , 𝑔= 𝑉 𝑖 , 𝑗 , 𝑔 𝑖𝑓 ( 𝑟𝑎𝑛𝑑 ( 0 , 1 ) ≤ 𝐶𝑟 ) 𝑂𝑅 𝑟𝑎𝑛𝑑𝑗= 𝑗
𝑋 𝑖 , 𝑗 , 𝑔 , 𝑂𝑡h𝑒𝑟𝑤𝑖𝑠𝑒 .
SELECTION 𝑋 𝑖 , 𝑔+1 =
{
𝑈 𝑖 ,𝑔 ∧𝑖𝑓 𝑓 (𝑈 𝑖 ,𝑔 ) ≤ 𝑓 ( 𝑋 𝑖 ,𝑔 )
𝑋 𝑖 ,𝑔 𝑂𝑡h𝑒𝑟𝑤𝑖𝑠𝑒
INITIALIZATION of algorithm parameters
In Initialization, the dimensions, number of constraints, scaling factor, upper bounds,
lower bounds and population is specified.
A sample of initialization from a DE program is given below:-
Fig.1 Mock illustration of Initial random
Population
MUTATION STRATEGIES (DE/a/b/c) & Minimum Np
Differential mutation adds a scaled, randomly sampled, vector difference to a third
random vector
DE/rand/1
• a = base vector
• b = no. of difference vectors added to base vector
• c = type of crossover scheme (binomial (bin) or exponential
(exp))
Types of Mutation Strategies
1) DE/rand/1 Vi,g = Xr1,g + F.(Xr2,g - Xr3,g)
2) DE/best/1 Vi,g = Xbest + F.(Xr1,g - Xr2,g)
3) DE/rand/2 Vi,g = Xr1,g + F.(Xr2,g - Xr3,g) + F.(Xr4,g - Xr5,g)
4) DE/best/2 Vi,g = Xbest + F.(Xr1,g - Xr2,g) + F.(Xr3,g - Xr4,g)
5) DE/target-to-best/2 Vi,g = Xi + F.(Xbest,g - Xi,g) + F.(Xr1,g - Xr2,g)
𝑉 𝑖, 𝑔= 𝑋 𝑟 1+ 𝐹 ( 𝑋 𝑟 2 − 𝑋 𝑟 3 )
𝑖=1 ,2 , …, 𝑁 𝑝
𝑉 𝑖 , 𝑔= 𝑋 𝑏𝑒𝑠𝑡 + 𝐹 ( 𝑋 𝑟 1 − 𝑋 𝑟 2 )
DE/
rand/1
DE/best/1
𝑉 𝑖. 𝑔= 𝑋 𝑟 1+ 𝐹 ( 𝑋 𝑟 2 − 𝑋 𝑟 3 ) + λ ( 𝑋 𝑟 4 − 𝑋 𝑟 5 )
DE / rand /2
x12
Xr2
F(Xr2 – Xr3)
Xr3
Xr1
x11
Mutation Vector Strategy for scale factor, F = 0.5, for index i = 1
CROSSOVER – Binomial
Crossover is a genetic operator used to vary the programming of a chromosome(vector) or
chromosomes from one generation to next. It is analogous to reproduction.
Crossover combines components from the current element and the mutant element,
according to the control parameter Cr = [0, 1]
It is done by following equation for j = 1 to D(no. of decision variables),
{ 𝑂𝑅
𝑈 𝑖 , 𝑔= 𝑣𝑖 , 𝑗 ,𝑔 𝑖𝑓 ( 𝑟𝑎𝑛 𝑑 ≤ 𝐶𝑟
𝑥𝑖 , 𝑗 ,𝑔 , 𝑂𝑡h𝑒𝑟𝑤𝑖𝑠𝑒 .
) 𝑟𝑎𝑛𝑑𝑗= 𝑗
Element from Mutant Vector (i.e.,
value from Mutant Vector)
Trial Vector Element from Target Vector (i.e.,
value from Target Vector)
Note: rand will generate random no.
Let’s understand using an example. between 0 and 1
𝑢𝑖 ,𝑔 =
{𝑣 𝑖 , 𝑗 , 𝑔 𝑖𝑓 ( 𝑟𝑎𝑛𝑑
𝑥𝑖 , 𝑗 ,𝑔 𝑂𝑡h𝑒𝑟𝑤𝑖𝑠𝑒 .
≤ 𝐶𝑟 𝑂𝑅 𝑟𝑎𝑛𝑑𝑗= 𝑗
)
Element from
Trial Vector Element from Mutant Vector
Target Vector
Let Cr (or pc) = 0.8 and randj = 3
j=1 j=2 j=3 j=4
i = 1, g =1 x1 x2 x3 x4
Target vector, X1 =[ 4 0 1 8
Mutant vector, 5 1 11 9
rand <= pc (0.8), 0.3 <= 0.8 0.9<=0.8 0.2<=0.8 0.6<=0.8
Trial, U1 = [ 5 0 11 9
SELECTION
DE uses one-to-one survivor selection, where the most fit objective value is
selected, i.e., “Survival of the Fittest”
It is done by,
Fig. 4 Main stages of Differential Evolution
DE PSEUDOCODE
Inputs: Objective function, lb, ub, Np, maxgen, F (scaling factor) and Cr (crossover prob.)
1. Initialize a random population, Xi
2. Evaluate fitness function f (Xi)
for g = 1 to maxgen %termination critertion but any other also possible
for i = 1 to Np
Generate the Mutant vector (Vi) using mutation Vi,g = Xr1i,g + F.(Xr2i,g - Xr3i,g)
Perform crossover to generate offspring (Ui)
end
for i = 1 to Np
Bound Ui
𝑈 𝑖,𝑔= {
𝑣 𝑟𝑎𝑛𝑑≤𝐶𝑟𝑂𝑅𝑟𝑎𝑛𝑑𝑗= 𝑗
𝑖, 𝑗,𝑔
¿
¿ 𝑖 , 𝑗 ,𝑔𝑥 𝑂𝑡h𝑒𝑟𝑤𝑖𝑠𝑒. ¿
Evaluate the fitness f(Ui) of Ui
Perform selection (greedy) between f(Ui) and f(Xi) and update Xi
end
end 𝑋 𝑖, 𝑔+1
{
= 𝑈 𝑖,𝑔 ∧𝑖𝑓 𝑓 (𝑈 𝑖 ,𝑔 )≤ 𝑓 ( 𝑋 𝑖 ,𝑔 )
𝑋 𝑖,𝑔 𝑂𝑡h𝑒𝑟𝑤𝑖𝑠𝑒
No. of functions evaluation? Np + maxgen*Np
Exponential crossover (example 1)
• Randomly choose an integer (randiD) between 1 and D. (Say randiD = 3, starting
element)
• Copy the randiDth variable from mutant vector as randiDth variable of trial vector.
• For subsequent variables, generate a random integer , L, between [1, D], say L = 2, two
consecutive elements will be exchanged from starting point
randiD = 3
j= 1 2 3 4 5
Target, X1 X11 X12 x13 x14 x15
Trail, U1 X11 X12 V13 V14 x15
randiD = 3
Mutant, V1 v11 v12 v13 v14 v15
Exponential crossover (example 2)
• Randomly choose an integer (randiD) between 1 and D.
• Copy the randiDth variable from mutant vector as randiDth variable of trial vector.
• For the subsequent variables, generate a random number between 0 and 1, till rand >= Cr.
• If rand <= Cr, copy the variable from mutant vector to trial vector.
• Else (rand > Cr) copy the remaining variables from target to trial vector.
randiD = 3 and Cr = 0.85
j= 1 2 3 4 5
Target, X1 x11 x12 x13 x14 x15
Trail, U1 x11 x12 v13 v14 v15
randiD = 3 rand <= Cr rand <= Cr
Mutant, V1 v11 v12 v13 v14 v15
DE Hand Calculation: Maximize z = x 1 + x2
Such that 1 ≤ x1 ≤ 10 and 1 ≤ x2 ≤ 18
Scaling (Mutation) factor F = 0.5
Crossover prob. Cr = 0.8
SOLUTION:
Step 1: Fix the inputs values: Np = 6, maxgen = 5, D = 2, lb, lb, F = 0.5, and Cr = 0.8
Step 2: Generate initial random population
Let the population size = 6
Generating random population:
Populati Solution/ Function
on Individual/ Value
Vector
x1 = (1, 1) + (0.3, 0.5) x[(10, 18) – (1, 1)] = (1, 1) + (0.3 x 9, 0.5x17) = (3.7, 8.5)
X1 (3.7, 8.5) 12.2
x2 = (1, 1) + (0.6, 0.4) x[(10, 18) – (1, 1)] = (1, 1) + (0.6 x 9, 0.4x17) = (5.4, 6.8)
X2 (5.4, 6.8) 12.2
x3 = (1, 1) + (0.4, 0.8) x[(10, 18) – (1, 1)] = (1, 1) + (0.4 x 9, 0.8x17) = (3.6, 13.6)
X3 (3.6, 13.8) 17.4
x4 = (1, 1) + (0.7, 0.9) x[(10, 18) – (1, 1)] = (1, 1) + (0.7 x 9, 0.9x17) = (6.3, 15.3) X4 (6.3, 15.3) 21.6
x5 = (1, 1) + (0.8, 0.2) x[(10, 18) – (1, 1)] = (1, 1) + (0.8 x 9, 0.2x17) = (7.2, 3.4) X5 (7.2, 3.4) 10.6
x6 = (1, 1) + (0.2, 0.4) x[(10, 18) – (1, 1)] = (1, 1) + (0.2 x 9, 0.4x17) = (1.8, 6.8) X6 (1.8, 6.8) 8.6
Step 3 : Mutation operation DE/rand/1 and F = 0.5, let r1 = a and r2 = b and r3 = c, a b c i (i = 1 to NP)
where, a, b and c are three individuals selected randomly from the population, such that a ≠ b ≠ c ≠ i.
X1 a = 2, b = 4 , c = 6 V1 = x2 + F(x4 – x6) V1 = (5.4, 6.8) + 0.5[(6.3, 15.3) – (1.8, 6.8)] V1 = (7.65, 11.05)
X2 a = 3, b = 4 , c = 5 V2 = x3 + F(x4 – x5) V2 = (3.6, 13.8) + 0.5[(6.3, 15.3) – (7.2, 3.4)] V2 = (3.15, 19.75)
X3 a = 1, b = 5 , c = 6 V3 = x1 + F(x5 – x6) V3 = (3.7, 8.5) + 0.5[(7.2, 3.4) – (1.8, 6.8)] V3 = (6.4, 6.8)
X4 a = 2, b = 3 , c = 6 V4 = x2 + F(x3 – x6) V4 = (5.4, 6.8) + 0.5[(3.6, 13.8) – (1.8, 6.8)] V4 = (7.2, 10.3)
X5 a = 6, b = 4 , c = 1 V5 = x2 + F(x4 – x1) V5 = (1.8, 6.8) + 0.5[(6.3, 15.3) – (3.7, 8.5)] V5 = (3.1, 10.2)
X6 a = 4, b = 3 , c = 1 V6 = x4 + F(x3 – x1) V6 = (6.3, 15.3) + 0.5[(3.6, 13.8) – (3.7, 8.5)] V6 = (6.25, 17.95)
Population Solution/Individual/Vector Function
Value
V1 (7.65, 11.05) 18.7
V2 (3.15, 19.75) 22.9
V3 (6.4, 6.8) 13.2
V4 (7.2, 10.3) 17.5
V5 (3.1, 10.2) 13.3
Step 4: Crossover & bound check, function evaluation 1 ≤ x1 ≤ 10 and 1 ≤ x2 ≤ 18
• A random number is generated for each variable in the vector and compared with the crossover probability.
• If ri ≤ Cr , then the variable is taken from mutant vector otherwise its taken from initial vector (target vector).
• The vector created by crossover is called as trial vector Population Solution/Individual/ Function Value
Vector
where, Cr = 0.8
U1 (7.65, 11.05) 18.7
For X1 let r1 = 0.3 and r2 = 0.7 Therefore, u1 = (7.65, 11.05)
U2 (5.4, 19.5) (5.4, 18) 23.4
For X2 let r1 = 0.9 and r2 = 0.2 Therefore, u2 = (5.4, 19.75)
U3 (6.4, 6.8) 13.2
For X3 let r1 = 0.6 and r2 = 0.5 Therefore, u3 = (6.4, 6.8) U4 (7.2, 10.3) 17.5
For X4 let r1 = 0.4 and r2 = 0.6 Therefore, u4 = (7.2, 10.3) U5 (3.1, 3.4) 6.5
For X5 let r1 = 0.8 and r2 = 0.9 Therefore, u5 = (3.1, 3.4) U6 (6.25, 17.95) 24.2
For X6 let r1 = 0.5 and r2 = 0.6 Therefore, u6 = (6.25, 17.95)
Step 5: U2 variable violates the bound,
corner bounding method is used.
U2 = (5.4, 19.5), then U2 = (5.4, 18)
Step 6 : Greedy selection
• The vector with better function value is chosen from trial vector and target vector.
where, g is the current generation
f(Ui) f(Xi) Selected vector Vector (Xg+1) f(xg+1)
18.7 12.2 U1 (7.65, 14.45) 22.1
23.4 12.2 U2 (5.4, 18) 23.4
13.2 17.4 X3 (3.6, 13.8) 17.6
17.5 21.6 X4 (6.3, 15.3) 21.6
6.5 10.6 X5 (7.2, 3.4) 10.6
24.2 8.6 U6 (6.25, 12.25) 18.5
• The selected population is considered as initial population for next generation and the complete procedure is repeated
until the desired criterion is achieved (i.e., no. of generation in our case).
