Iran University of Science and Technology
Automotive Engineering Department
Optimization
Lecture 4.4:
Genetic Algorithms
for Optimum
Design
The Continuous GA
Flowchart of a continuous GA
Lecture 7: Genetic Algorithms for Optimum Design Optimization 2
The Continuous GA
Example
Lecture 7: Genetic Algorithms for Optimum Design Optimization 3
The Continuous GA
Example
Solution Structure
Lecture 7: Genetic Algorithms for Optimum Design Optimization 4
The Continuous GA
Example
Solution Structure
Initial Population
Lecture 7: Genetic Algorithms for Optimum Design Optimization 5
The Continuous GA
Example
Solution Structure
Initial Population
Lecture 7: Genetic Algorithms for Optimum Design Optimization 6
The Continuous GA
Example
Solution Structure
Initial Population
Lecture 7: Genetic Algorithms for Optimum Design Optimization 7
The Continuous GA
Initial Population
Lecture 7: Genetic Algorithms for Optimum Design Optimization 8
The Continuous GA
Initial Population
Lecture 7: Genetic Algorithms for Optimum Design Optimization 9
The Continuous GA
Natural Selection Nkeep=50%
Lecture 7: Genetic Algorithms for Optimum Design Optimization 10
The Continuous GA
Natural Selection Nkeep=50%
Lecture 7: Genetic Algorithms for Optimum Design Optimization 11
The Continuous GA
Pairing: Rank Weighting
Random Number Generation
Lecture 7: Genetic Algorithms for Optimum Design Optimization 12
The Continuous GA
Mating:
Simplest Form
Lecture 7: Genetic Algorithms for Optimum Design Optimization 13
The Continuous GA
Mating:
Simplest Form
Lecture 7: Genetic Algorithms for Optimum Design Optimization 14
The Continuous GA
Mating:
Simplest Form
What is the problem?
Lecture 7: Genetic Algorithms for Optimum Design Optimization 15
The Continuous GA
Mating:
Simplest Form
What is the problem?
No new information is introduced: each continuous value that was randomly
initiated in the initial population is propagated to the next generation, only in
different combinations.
Lecture 7: Genetic Algorithms for Optimum Design Optimization 16
The Continuous GA
Mating:
What is the solution?
Lecture 7: Genetic Algorithms for Optimum Design Optimization 17
The Continuous GA
Mating:
What is the solution?
The blending methods remedy this problem by finding ways to combine variable
values from the two parents into new variable values in the offspring.
Lecture 7: Genetic Algorithms for Optimum Design Optimization 18
The Continuous GA
Mating:
What is the solution?
The blending methods remedy this problem by finding ways to combine variable
values from the two parents into new variable values in the offspring.
Lecture 7: Genetic Algorithms for Optimum Design Optimization 19
The Continuous GA
Mating:
What is the solution?
The blending methods remedy this problem by finding ways to combine variable
values from the two parents into new variable values in the offspring.
OR
Lecture 7: Genetic Algorithms for Optimum Design Optimization 20
The Continuous GA
Mating:
What is the solution?
The blending methods remedy this problem by finding ways to combine variable
values from the two parents into new variable values in the offspring.
OR
Crossover Points
Lecture 7: Genetic Algorithms for Optimum Design Optimization 21
The Continuous GA
Mating:
What is the solution?
The blending methods remedy this problem by finding ways to combine variable
values from the two parents into new variable values in the offspring.
OR
Crossover Points
Lecture 7: Genetic Algorithms for Optimum Design Optimization 22
The Continuous GA
Mating:
For our example problem, the first set of parents are given by:
Lecture 7: Genetic Algorithms for Optimum Design Optimization 23
The Continuous GA
Mating:
For our example problem, the first set of parents are given by:
Lecture 7: Genetic Algorithms for Optimum Design Optimization 24
The Continuous GA
Mating Result:
Lecture 7: Genetic Algorithms for Optimum Design Optimization 25
The Continuous GA
Mutation
Elitism
Lecture 7: Genetic Algorithms for Optimum Design Optimization 26
The Continuous GA
Mutation
Elitism
A mutated variable is replaced by a new random variable. The following
pairs were randomly selected:
Lecture 7: Genetic Algorithms for Optimum Design Optimization 27
The Continuous GA
Mutation
Elitism
A mutated variable is replaced by a new random variable. The following
pairs were randomly selected:
The value in row 4 and column 1 of the population matrix is replaced with a
uniform random number between one and ten:
Lecture 7: Genetic Algorithms for Optimum Design Optimization 28
The Continuous GA
Mutation
Elitism
A mutated variable is replaced by a new random variable. The following
pairs were randomly selected:
The value in row 4 and column 1 of the population matrix is replaced with a
uniform random number between one and ten:
Lecture 7: Genetic Algorithms for Optimum Design Optimization 29
The Continuous GA
Mutation
Lecture 7: Genetic Algorithms for Optimum Design Optimization 30
The Continuous GA
Contour plot of the cost function with the population after the first generation
Lecture 7: Genetic Algorithms for Optimum Design Optimization 31
The Continuous GA
Next Generation
New Ranked Population at the Start of the Second Generation
Lecture 7: Genetic Algorithms for Optimum Design Optimization 32
The Continuous GA
Next Generation
• Population after Crossover and Mutation in the Second Generation
Lecture 7: Genetic Algorithms for Optimum Design Optimization 33
The Continuous GA
Next Generation
Contour plot of the cost function with the population after the second generation
Lecture 7: Genetic Algorithms for Optimum Design Optimization 34
The Continuous GA
3rd Generation
New Ranked Population at the Start of the Third Generation
Lecture 7: Genetic Algorithms for Optimum Design Optimization 35
The Continuous GA
3rd Generation
Ranking of Generation 3 from Least to Most Cost
Lecture 7: Genetic Algorithms for Optimum Design Optimization 36
The Continuous GA
3rd Generation
Contour plot of the cost function with the population after the third and final generation
Lecture 7: Genetic Algorithms for Optimum Design Optimization 37
The Continuous GA
3rd Generation
Plot of the minimum and mean costs as a function of generation.
• The algorithm converged in three generations
• Nelder-Mead and BFGS algorithms cannot find the global minimum of this problem
Lecture 7: Genetic Algorithms for Optimum Design Optimization 38
Basic Concepts and Definitions
Fitness Function
Unconstrained Problems:
𝐶𝑜𝑠𝑡 = 𝑓 𝑥
Chromosome
Constrained Problems:
𝐶𝑜𝑠𝑡 = 𝑓 𝑥
Only Feasible Chromosomes
OR
𝐶𝑜𝑠𝑡 = 𝑓 𝑥 + 𝑣 ∗ 𝑃𝑒𝑛𝑎𝑙𝑡𝑦(𝑥)
Chromosome Constraints Violation
Lecture 7: Genetic Algorithms for Optimum Design Optimization 39
Basic Concepts and Definitions
Fitness Function
Unconstrained Problems:
𝐶𝑜𝑠𝑡 = 𝑓 𝑥
Chromosome
Constrained Problems:
𝐶𝑜𝑠𝑡 = 𝑓 𝑥
Only Feasible Chromosomes
OR
𝐶𝑜𝑠𝑡 = 𝑓 𝑥 + 𝑣 ∗ 𝑃𝑒𝑛𝑎𝑙𝑡𝑦(𝑥)
Chromosome Constraints Violation
Lecture 7: Genetic Algorithms for Optimum Design Optimization 40
Basic Concepts and Definitions
Fitness Function Example: Transportation Problem
• A chemical company has two
factories: F1 and F2
• Outlets are R1, R2, …., R12
• Fi can Produce ai tons of a certain
chemical product each week
• Outlet Rj has a known weekly
demand of bj tons of the product
• The cost of shipping one ton of • xij is the number of tons of
the product from factory Fi to the product shipped from Fi
retail outlet Rj is cij to retail outlet Rj
How much of the product to ship from each factory to each outlet
so as to satisfy all the requirements and minimize cost.
Lecture 7: Genetic Algorithms for Optimum Design Optimization 41
Basic Concepts and Definitions
Fitness Function Example: Transportation Problem
Lecture 7: Genetic Algorithms for Optimum Design Optimization 42
Basic Concepts and Definitions
Fitness Function Example: Transportation Problem
Defining a penalty for violating the constraints
Lecture 7: Genetic Algorithms for Optimum Design Optimization 43
Basic Concepts and Definitions
Fitness Function Example: Transportation Problem
Defining a penalty for violating the constraints
Constraint: 𝒈 ≥ 𝒈𝟎
Lecture 7: Genetic Algorithms for Optimum Design Optimization 44
Basic Concepts and Definitions
Fitness Function Example: Transportation Problem
Defining a penalty for violating the constraints
Constraint: 𝒈 ≥ 𝒈𝟎
𝟎; 𝒈 ≥ 𝒈𝟎
Violation: 𝒗 𝒈 ≥ 𝒈𝟎 =
+; 𝒈 < 𝒈𝟎
Lecture 7: Genetic Algorithms for Optimum Design Optimization 45
Basic Concepts and Definitions
Fitness Function Example: Transportation Problem
Defining a penalty for violating the constraints
Constraint: 𝒈 ≥ 𝒈𝟎
𝟎; 𝒈 ≥ 𝒈𝟎
Violation: 𝒗 𝒈 ≥ 𝒈𝟎 =
+; 𝒈 < 𝒈𝟎
Its value?
Lecture 7: Genetic Algorithms for Optimum Design Optimization 46
Basic Concepts and Definitions
Fitness Function Example: Transportation Problem
Defining a penalty for violating the constraints
Constraint: 𝒈 ≥ 𝒈𝟎
𝟎; 𝒈 ≥ 𝒈𝟎
Violation: 𝒗 𝒈 ≥ 𝒈𝟎 =
+; 𝒈 < 𝒈𝟎
Its value?
Since it’s a penalty, it is clear that it must be a positive value!
Lecture 7: Genetic Algorithms for Optimum Design Optimization 47
Basic Concepts and Definitions
Fitness Function Example: Transportation Problem
Defining a penalty for violating the constraints
Constraint: 𝒈 ≥ 𝒈𝟎
𝟎; 𝒈 ≥ 𝒈𝟎
Violation: 𝒗 𝒈 ≥ 𝒈𝟎 =
+; 𝒈 < 𝒈𝟎
It seems that |𝒈 − 𝒈𝟎 | is a good choice!
Lecture 7: Genetic Algorithms for Optimum Design Optimization 48
Basic Concepts and Definitions
Fitness Function Example: Transportation Problem
Defining a penalty for violating the constraints
Constraint: 𝒈 ≥ 𝒈𝟎
𝟎; 𝒈 ≥ 𝒈𝟎
Violation: 𝒗 𝒈 ≥ 𝒈𝟎 =
+; 𝒈 < 𝒈𝟎
It seems that |𝒈 − 𝒈𝟎 | is a good choice!
But it has a problem!
Lecture 7: Genetic Algorithms for Optimum Design Optimization 49
Basic Concepts and Definitions
Fitness Function Example: Transportation Problem
But it has a problem!
𝒈 𝒈𝟎 |𝒈 − 𝒈𝟎 | 𝒈
| − 𝟏|
𝒈𝟎
8 10 0.2
17 20 0.15
44 50 0.12
Lecture 7: Genetic Algorithms for Optimum Design Optimization 50
Basic Concepts and Definitions
Fitness Function Example: Transportation Problem
But it has a problem!
𝒈 𝒈𝟎 |𝒈 − 𝒈𝟎 | 𝒈
| − 𝟏|
𝒈𝟎
8 10 2 0.2
17 20 3 0.15
44 50 6 0.12
Lecture 7: Genetic Algorithms for Optimum Design Optimization 51
Basic Concepts and Definitions
Fitness Function Example: Transportation Problem
But it has a problem!
𝒈 𝒈𝟎 |𝒈 − 𝒈𝟎 | 𝒈
| − 𝟏|
𝒈𝟎
8 10 2 0.2
17 20 3 0.15
44 50 6 0.12
Greatest violation occurred!
Lecture 7: Genetic Algorithms for Optimum Design Optimization 52
Basic Concepts and Definitions
Fitness Function Example: Transportation Problem
But it has a problem!
𝒈 𝒈𝟎 |𝒈 − 𝒈𝟎 | 𝒈
| − 𝟏|
𝒈𝟎
8 10 2
17 20 3
44 50 6
Greatest violation occurred!
Lecture 7: Genetic Algorithms for Optimum Design Optimization 53
Basic Concepts and Definitions
Fitness Function Example: Transportation Problem
But it has a problem!
𝒈 𝒈𝟎 |𝒈 − 𝒈𝟎 | 𝒈
| − 𝟏|
𝒈𝟎
8 10 2 0.2
17 20 3 0.15
44 50 6 0.12
Greatest violation occurred!
Lecture 7: Genetic Algorithms for Optimum Design Optimization 54
Basic Concepts and Definitions
Fitness Function Example: Transportation Problem
But it has a problem!
Greatest violation occurred!
𝒈 𝒈𝟎 |𝒈 − 𝒈𝟎 | 𝒈
| − 𝟏|
𝒈𝟎
8 10 2 0.2
17 20 3 0.15
44 50 6 0.12
Greatest violation occurred!
Lecture 7: Genetic Algorithms for Optimum Design Optimization 55
Basic Concepts and Definitions
Fitness Function Example: Transportation Problem
Defining a penalty for violating the constraints
Constraint: 𝒈 ≥ 𝒈𝟎
𝟎; 𝒈 ≥ 𝒈𝟎
Violation: 𝒗 𝒈 ≥ 𝒈𝟎 = 𝒈
1− ; 𝒈 < 𝒈𝟎
𝒈𝟎
Lecture 7: Genetic Algorithms for Optimum Design Optimization 56
Basic Concepts and Definitions
Fitness Function Example: Transportation Problem
Defining a penalty for violating the constraints
Constraint: 𝒈 ≥ 𝒈𝟎
𝟎; 𝒈 ≥ 𝒈𝟎
Violation: 𝒗 𝒈 ≥ 𝒈𝟎 = 𝒈
1− ; 𝒈 < 𝒈𝟎
𝒈𝟎
𝒈
𝒗 𝒈 ≥ 𝒈𝟎 = 𝒎𝒂𝒙(1 − , 𝟎)
𝒈𝟎
Lecture 7: Genetic Algorithms for Optimum Design Optimization 57
Basic Concepts and Definitions
Fitness Function Example: Transportation Problem
Defining a penalty for violating the constraints
Constraint: 𝒈 ≤ 𝒈𝟎
𝟎; 𝒈 ≤ 𝒈𝟎
Violation: 𝒗 𝒈 ≤ 𝒈𝟎 = 𝒈
−1 ; 𝒈 > 𝒈𝟎
𝒈𝟎
𝒈
𝒗 𝒈 ≤ 𝒈𝟎 = 𝒎𝒂𝒙( − 1, 𝟎)
𝒈𝟎
Lecture 7: Genetic Algorithms for Optimum Design Optimization 58
Basic Concepts and Definitions
Fitness Function Example: Transportation Problem
Defining a penalty for violating the constraints
Constraint: 𝒈 = 𝒈𝟎
Lecture 7: Genetic Algorithms for Optimum Design Optimization 59
Basic Concepts and Definitions
Fitness Function Example: Transportation Problem
Defining a penalty for violating the constraints
Constraint: 𝒈 = 𝒈𝟎 𝒈 ≥ 𝒈𝟎 & 𝒈 ≤ 𝒈𝟎
Lecture 7: Genetic Algorithms for Optimum Design Optimization 60
Basic Concepts and Definitions
Fitness Function Example: Transportation Problem
Defining a penalty for violating the constraints
Constraint: 𝒈 = 𝒈𝟎 𝒈 ≥ 𝒈𝟎 & 𝒈 ≤ 𝒈𝟎
Violation:
𝒈 𝒈 𝒈 𝒈
|1 − |+1 − | −1|+ −1
𝒈𝟎 𝒈𝟎 𝒈𝟎 𝒈𝟎
( + )
𝒗 𝒈 = 𝒈𝟎 = 𝟐 𝟐
𝟐
Lecture 7: Genetic Algorithms for Optimum Design Optimization 61
Basic Concepts and Definitions
Fitness Function Example: Transportation Problem
Defining a penalty for violating the constraints
Constraint: 𝒈 = 𝒈𝟎 𝒈 ≥ 𝒈𝟎 & 𝒈 ≤ 𝒈𝟎
Violation:
𝒈 𝒈 𝒈 𝒈
|1 − |+1 − | −1|+ −1
𝒈𝟎 𝒈𝟎 𝒈𝟎 𝒈𝟎
( + )
𝒗 𝒈 = 𝒈𝟎 = 𝟐 𝟐
𝟐
𝒈
𝒗 𝒈 = 𝒈𝟎 =| −1|
𝒈𝟎
Lecture 7: Genetic Algorithms for Optimum Design Optimization 62
Basic Concepts and Definitions
Fitness Function Example: Transportation Problem
Lecture 7: Genetic Algorithms for Optimum Design Optimization 63
Basic Concepts and Definitions
Fitness Function Example: Transportation Problem
12
𝑗=1 𝑥𝑖𝑗
𝒗𝟏𝒊 = 𝒎𝒂𝒙 − 1, 𝟎 ; ∀𝒊
ai
Lecture 7: Genetic Algorithms for Optimum Design Optimization 64
Basic Concepts and Definitions
Fitness Function Example: Transportation Problem
12 2
𝑗=1 𝑥𝑖𝑗 𝟏
𝒗𝟏𝒊 = 𝒎𝒂𝒙 − 1, 𝟎 ; ∀𝒊 𝒗𝟏 = 𝒗𝟏𝒊
ai 𝟐
𝑖=1
Average of the 𝑎𝑖 violation
Lecture 7: Genetic Algorithms for Optimum Design Optimization 65
Basic Concepts and Definitions
Fitness Function Example: Transportation Problem
2
𝑖=1 𝑥𝑖𝑗
𝒗𝟐𝒋 = 𝒎𝒂𝒙 𝟏 − ,𝟎 ; ∀𝒋
bj
Lecture 7: Genetic Algorithms for Optimum Design Optimization 66
Basic Concepts and Definitions
Fitness Function Example: Transportation Problem
2 12
𝑖=1 𝑥𝑖𝑗 𝟏
𝒗𝟐𝒋 = 𝒎𝒂𝒙 𝟏 − ,𝟎 ; ∀𝒋 𝒗𝟐 = 𝒗𝟐𝒋
bj 𝟏𝟐
𝑗=1
Average of the 𝑏𝑗 violation
Lecture 7: Genetic Algorithms for Optimum Design Optimization 67
Basic Concepts and Definitions
Fitness Function Example: Transportation Problem
𝟏
Total Violation: 𝒗 = (𝒗𝟏 + 𝒗𝟐 )
𝟐
Lecture 7: Genetic Algorithms for Optimum Design Optimization 68
Basic Concepts and Definitions
Fitness Function Example: Transportation Problem
𝟏
Total Violation: 𝒗 = (𝒗𝟏 + 𝒗𝟐 ) Penalty Function
𝟐
New Objective Function: 𝒎𝒊𝒏 ( 𝒄𝒊𝒋 𝒙𝒊𝒋 + 𝑷 𝒗 )
𝒊𝒋
Lecture 7: Genetic Algorithms for Optimum Design Optimization 69
Basic Concepts and Definitions
Fitness Function Example: Transportation Problem
New Objective Function in General Form: 𝒎𝒊𝒏 (𝒇 + 𝑷 𝒗 )
Lecture 7: Genetic Algorithms for Optimum Design Optimization 70
Basic Concepts and Definitions
Fitness Function Example: Transportation Problem
New Objective Function in General Form: 𝒎𝒊𝒏 (𝒇 + 𝑷 𝒗 )
Options for P(v):
• Additive 𝒇 + 𝜶𝒗; 𝜶>𝟎
Lecture 7: Genetic Algorithms for Optimum Design Optimization 71
Basic Concepts and Definitions
Fitness Function Example: Transportation Problem
New Objective Function in General Form: 𝒎𝒊𝒏 (𝒇 + 𝑷 𝒗 )
Options for P(v):
• Additive 𝒇 + 𝜶𝒗; 𝜶>𝟎
• Multiplicative 𝒇(𝟏 + 𝜷𝒗); 𝜷>𝟎
Lecture 7: Genetic Algorithms for Optimum Design Optimization 72
Basic Concepts and Definitions
Fitness Function Example: Transportation Problem
New Objective Function in General Form: 𝒎𝒊𝒏 (𝒇 + 𝑷 𝒗 )
Options for P(v):
• Additive 𝒇 + 𝜶𝒗; 𝜶>𝟎
• Multiplicative 𝒇(𝟏 + 𝜷𝒗); 𝜷>𝟎
• Mixed (𝒇 + 𝜶𝒗)(𝟏 + 𝜷𝒗); 𝜷>𝟎
OR
𝒇 𝟏 + 𝜷𝒗 + 𝜶𝒗; 𝜷>𝟎
Lecture 7: Genetic Algorithms for Optimum Design Optimization 73
Basic Concepts and Definitions
Fitness Function Example: Transportation Problem
New Objective Function in General Form: 𝒎𝒊𝒏 (𝒇 + 𝑷 𝒗 )
Options for P(v):
• Additive 𝒇 + 𝜶𝒗; 𝜶>𝟎
• Multiplicative 𝒇(𝟏 + 𝜷𝒗); 𝜷>𝟎 = 𝒇 + 𝒇𝜷𝒗
• Mixed (𝒇 + 𝜶𝒗)(𝟏 + 𝜷𝒗); 𝜷>𝟎
OR
𝒇 𝟏 + 𝜷𝒗 + 𝜶𝒗; 𝜷>𝟎
Lecture 7: Genetic Algorithms for Optimum Design Optimization 74
Basic Concepts and Definitions
Fitness Function Example: Transportation Problem
New Objective Function in General Form: 𝒎𝒊𝒏 (𝒇 + 𝑷 𝒗 )
Options for P(v):
• Additive 𝒇 + 𝜶𝒗; 𝜶>𝟎
• Multiplicative 𝒇(𝟏 + 𝜷𝒗); 𝜷>𝟎 = 𝒇 + 𝒇𝜷𝒗
Since f may have a negative value: ∗ 𝒔𝒊𝒈𝒏(𝒇)
• Mixed (𝒇 + 𝜶𝒗)(𝟏 + 𝜷𝒗); 𝜷>𝟎
OR
𝒇 𝟏 + 𝜷𝒗 + 𝜶𝒗; 𝜷>𝟎
Lecture 7: Genetic Algorithms for Optimum Design Optimization 75
