Genetic Algorithms

Assoc. Prof. Ayman Ghoneim

Operations Research and Decision Support Department

Faculty of Computers and Artificial Intelligence, Cairo University
Contact Email: [email protected]

Computational Intelligence (DS313/DS351) - 2022

Ayman Ghoneim (FCAI - Cairo University) Genetic Algorithms Spring 2022 1 / 28

Genetic Algorithms

After the examples we had covered, the following are the points that must be
addressed when building any GA:
Design a representation (Encoding).
Decide how to initialize a population.
Design a way of evaluating an individual.
Decide how to select individuals to be parents.
Design suitable recombination operator(s).
Design suitable mutation operator(s).
Decide how offspring replace parents.
Decide when to stop the algorithm.

Ayman Ghoneim (FCAI - Cairo University) Genetic Algorithms Spring 2022 2 / 28

Genetic Algorithms - Representation (Encoding)

Representation:
We have to come up with a method of representing an individual as a
genotype.

Ayman Ghoneim (FCAI - Cairo University) Genetic Algorithms Spring 2022 3 / 28

Genetic Algorithms - Representation (Encoding)

Binary Representations:

the simplest and most commonly used representation.

the chromosome is a string of 0 or 1 bits (genes are 0 or 1).
can be used to represent integers or real values.

Ayman Ghoneim (FCAI - Cairo University) Genetic Algorithms Spring 2022 4 / 28

Genetic Algorithms - Representation (Encoding)

Binary Representations:

Ayman Ghoneim (FCAI - Cairo University) Genetic Algorithms Spring 2022 5 / 28

Genetic Algorithms - Representation (Encoding)
Binary Representation:
For encoding real values for variable x ∈ [xmin , xmax ], the standard decoding
function is:
PL−1
j=0 x[j] × 2L−j−1
x = xmin + × (xmax − xmin ),
2L
where x[j] is the gene value at position j.
The standard decoding function suffers from the Hamming cliff problem as an
disadvantage, which is the Hamming distance between two consecutive
integers may be greater than 1 bit.
Example: for 5 bit binary representation
14: 01110
15: 01111
16: 10000
Probability of changing 15 into 16 by independent bit flips (mutation) is not
same as changing it into 14!
Ayman Ghoneim (FCAI - Cairo University) Genetic Algorithms Spring 2022 6 / 28
Genetic Algorithms - Representation (Encoding)

Gray decoding solves the hamming cliff problem using the following formula:
PL−1 Pj L−j−1
j=0 (( k=0 x[k])mod2) × 2
x = xmin + × (xmax − xmin )
2L
The previous equation guarantees that the Hamming distance between any
two consecutive integers is 1 bit.

Ayman Ghoneim (FCAI - Cairo University) Genetic Algorithms Spring 2022 7 / 28

Genetic Algorithms - Representation (Encoding)
Example: 3-bit Gray Code

Integer 0 1 2 3 4 5 6 7
Standard decoding 000 001 010 011 100 101 110 111
Gray decoding 000 001 011 010 110 111 101 100

Ayman Ghoneim (FCAI - Cairo University) Genetic Algorithms Spring 2022 8 / 28

Genetic Algorithms - Representation (Encoding)

Integer Representation:
binary representations may not always be best choice.
another representation may be more natural for a specific problem e.g.
for optimization of a function with integer variables.
values may be unrestricted (all integers), or restricted to a finite set (e.g.,
{0, 1, 2, 3} which can represent categorical data such as
{North, East, South, West}).

Ayman Ghoneim (FCAI - Cairo University) Genetic Algorithms Spring 2022 9 / 28

Genetic Algorithms - Representation (Encoding)

Real-Valued / Floating Point Representations:

genes take values from a continuous interval.
genotype for solution becomes the vector of real values (floating point
number) < x1 , x2 , . . . , xk > with xi ∈ <.

Ayman Ghoneim (FCAI - Cairo University) Genetic Algorithms Spring 2022 10 / 28

Genetic Algorithms - Representation (Encoding)

Binary vs. Real Value Representation

Ayman Ghoneim (FCAI - Cairo University) Genetic Algorithms Spring 2022 11 / 28

Genetic Algorithms - Representation (Encoding)
It is worth mentioning that binary encoding in itself can be used to solve
problems (not necessarily required to encode integer or real values).
For example, scheduling jobs to start at different starting point.
Schedule Phenotype:

Ayman Ghoneim (FCAI - Cairo University) Genetic Algorithms Spring 2022 12 / 28

Genetic Algorithms - Representation (Encoding)

Permutation Representations:
deciding on sequence of events.
Most natural representation is permutation of a set of integers (number
may occur more than once in the chromosome).
May include invalid permutations.
Need special variation operators.

Two classes of problems:

based on order of events, e.g., Job Scheduling Problem.

based on adjacencies, e.g. Traveling Salesperson Problem (TSP) finding
a complete tour of minimal length between n cities, visiting each city
only once.

Ayman Ghoneim (FCAI - Cairo University) Genetic Algorithms Spring 2022 13 / 28

Genetic Algorithms - Representation (Encoding)

Two ways to encode a permutation:

ith element represents event that happens in that location in a sequence.

value of ith element denotes position in sequence in which ith event
occurs.
Example (TSP):
4 cities A,B,C,D and permutation [3,1,2,4] denotes the tours:
First encoding type: [C → A → B → D]
Second encoding type: [B → C → A → D]

Ayman Ghoneim (FCAI - Cairo University) Genetic Algorithms Spring 2022 14 / 28

Genetic Algorithms - Representation (Encoding)

Tree-based Representation:
Tree of Functions or Terminals:
Functions: sine, cosine, add, sub, and, If-Then-Else ... etc.
Terminals: X, Y, 0.456, true, false, π, ... etc

Ayman Ghoneim (FCAI - Cairo University) Genetic Algorithms Spring 2022 15 / 28

Genetic Algorithms - Explore/Exploit Trade-off

For all the next decisions, we need to bear in mind the search explore/exploit
trade-off, which is directly related to diversity and convergence.
Diversity Vs Convergence:
Diversity represents differences of genetic characteristics in the
population.
Convergence means that the search reaches (converges) to a good
solution at its last phases.
Loss of genetic diversity means all individuals in the population look
alike.
High Diversity → Slow Convergence.
Low Diversity → Premature Convergence.

Ayman Ghoneim (FCAI - Cairo University) Genetic Algorithms Spring 2022 16 / 28

Genetic Algorithms - Explore/Exploit Trade-off

Explore Vs Exploit
Exploration: is to visit unknown search space regions.
Too much Exploration → High diversity → Slow Convergence.
Exploitation: is to improve the best-so-far individuals.
Too much exploitation → Low Diversity → Premature Convergence.

Ayman Ghoneim (FCAI - Cairo University) Genetic Algorithms Spring 2022 17 / 28

Genetic Algorithms - Initialization

Decide how to initialize a population:

Ayman Ghoneim (FCAI - Cairo University) Genetic Algorithms Spring 2022 18 / 28

Genetic Algorithms - Fitness Evaluation

Design a way of evaluating an individual:

Ayman Ghoneim (FCAI - Cairo University) Genetic Algorithms Spring 2022 19 / 28

Genetic Algorithms - Fitness Evaluation
Evaluation is the most costly step in EA for real applications.
Design a way of evaluating an individual:

Ayman Ghoneim (FCAI - Cairo University) Genetic Algorithms Spring 2022 20 / 28

Genetic Algorithms - Selection

Selection scheme: process that selects an individual to go into the mating

pool.
Selection pressure: degree to which the better individuals are favored.
If higher selection pressure, better individuals favored more.
Selection pressure determines convergence rate:
if too high, possible premature convergence.
if too low, may take too long to find good solutions.
Two types of selection scheme:
proportionate.
ordinal based.

Ayman Ghoneim (FCAI - Cairo University) Genetic Algorithms Spring 2022 21 / 28

Genetic Algorithms - Selection

Roulette-Wheel Selection
Fitness proportionate: Expected number of representatives of each individual
is proportional to its fitness in comparison to the total fitness of all the
individuals in the population.

fitnessi
probi = Pj=popSize
j=0 fitnessi
The previous will compute a probability of selection for each individual, and
individuals will be selected randomly according to this probability.

Ayman Ghoneim (FCAI - Cairo University) Genetic Algorithms Spring 2022 22 / 28

Genetic Algorithms - Selection

Roulette-Wheel Selection

Ayman Ghoneim (FCAI - Cairo University) Genetic Algorithms Spring 2022 23 / 28

Genetic Algorithms - Selection

Linear Ranking Selection:

It rescales the fitness of individuals using a parameter SP = [1, 2] and the
following steps:
1 Each individual in the population is ranked in increasing order of fitness,
from 1 to N. I.e., the solution with the lowest fitness will be of rank 1,
while the solution with the highest fitness will be of rank N (N is the
population size).
2 The rank fitness value of the solution at rank i (Rank(i)) is computed as
rankFiti = (2 − SP) + 2(SP − 1) × Rank(i)−1
N−1 .
3 Given the rank fitness computed for all solutions, apply Roulette wheel
selection.

Ayman Ghoneim (FCAI - Cairo University) Genetic Algorithms Spring 2022 24 / 28

Genetic Algorithms - Selection

Roulette wheel vs Linear Ranking Selection

Ayman Ghoneim (FCAI - Cairo University) Genetic Algorithms Spring 2022 25 / 28

Genetic Algorithms - Selection

Tournament Selection
Select individuals by conducting a number of tournaments among them.
It has one parameter: the tournament size.
It selects randomly a number of individuals (equivalent to the tournament
size) from the population. Then conduct a tournament among them and select
the winner (the one with the best fitness among them).

Ayman Ghoneim (FCAI - Cairo University) Genetic Algorithms Spring 2022 26 / 28

Genetic Algorithms - Selection

Tournament Selection

Ayman Ghoneim (FCAI - Cairo University) Genetic Algorithms Spring 2022 27 / 28

Genetic Algorithms - Selection

Manual Selection:
User assign fitness to solutions based on subjectivity.
Used when it is difficult to design an appropriate fitness function.

Ayman Ghoneim (FCAI - Cairo University) Genetic Algorithms Spring 2022 28 / 28