Introduction to
Genetic Algorithms (GAs)
Evolutionary Computation
(EC)
EC is a computing paradigm based on the principles of biological
evolution.
Meta-heuristic optimization techniques
Population based nature
Mimics natural genetic systems
Components of EC:
Genetic Algorithms
Genetic Programming
Particle Swarm Optimization (PSO)
Ant Colony Algorithms
11/25/17 GA 2
Genetic Algorithms (GAs)
• Definition:
Randomized Search and Optimization
technique guided by the Principles of Natural
Genetics.
• Natural selection : Survival of the fittest
• Natural reproduction
11/25/17 GA 3
Randomized Search
– Not a simple random search
– Randomized search : random choice in a guided
path !
– Use the payoff function to guide search : fitness
11/25/17 GA 4
Classes of Search Techniques
Search techniques
Calculus-based techniques Guided random search techniques Enumerative techniques
Direct methods Indirect methods Evolutionary algorithms Simulated annealing Dynamic programming
Finonacci Newton Evolutionary strategies Genetic algorithms
Parallel Sequential
Centralized Distributed Steady-state Generational
11/25/17 GA 5
Different Search Techniques
Calculus based techniques – seek
local extrema by gradient descent –
hill climbing
–
Lack of robustness
• Single point search
• Restrictive requirements of
continuity and derivative
existence
• Multi peak function causes
dilemma : random restart
• Best in neighborhood of the
current point
11/25/17 GA 6
Search Techniques (contd.)
Enumerative Techniques:
look at objective function values at every point in
the search space
– dynamic programming – n points
If n very very large ?
Simple but inefficient (time consuming)
11/25/17 GA 7
GAs: A quick Overview
• Developed by John Holland, his colleagues, and
his students, K. DeJong, D. Goldberg, University of
Michigan (1970’s), USA
– to understand the adaptive processes of natural
systems
– to design artificial systems software that retains the
robustness of natural systems
11/25/17 GA 8
Why GAs
• Most of the real life problems are very complex and can not be
solved in polynomial time using a deterministic algorithm.
• Sometimes attainment to the best is less important for complex
problem. Rather near optimal solutions that can be generated
quickly are more desirable than optimal solutions which require
huge amount of time.
– Ex: Traveling Salesman Problem (TSP).
GAs strive to improve as close as possible to optimality.
• When the problem can be modeled as an optimization one.
11/25/17 GA 9
TSP example
Problem:
Given n cities
Find a complete tour with
minimal length/cost
Computation Time ??
Search space is BIG:
for 30 cities there are 30! ≈ 1032
possible tours
NP-Hard Problem
11/25/17 GA 10
Why GAs (contd.)
• Not limited by restrictive assumptions about the
search space
– Continuity
– Existence of derivative
– Uni-modality and local minima
• Robust search in complex space
– Balance between efficiency and efficacy (flexibility of
biological system necessary for survival in many different
situations)
11/25/17 GA 11
Why GAs (contd.)
• Efficiently exploit historical information to speculate on new search
points with expected improved performance : nature is full of
precedents
• Adaptive : Features of self-repair, self-guidance and reproduction are
the rules of the biological systems
• Costly redesign can be reduced
GAs work from a rich database of points simultaneously, climbing
many peaks in parallel; thus probability of finding a false peak (local
optima) is reduced.
11/25/17 GA 12
GAs
Distribution of Individuals in Generation 0
Distribution of Individuals in Generation N
11/25/17 GA 13
Why GAs (contd.)
• Always an answer/solution; answer gets better with time
empirically proved to be convergent
• Inherently parallel; easily distributed
• Supports multi-objective optimization
• Modular
• Finally, the power of GAs comes from simple concepts;
easy to understand
11/25/17 GA 14
Features of GAs
• Optimization Technique
• Darwin’s principle of evolution (survival of the fittest) has
made this optimization algorithm effective.
• Work with a coding of the parameter set
• Search from a population of points, not a single point
• Use payoff (objective function) information, not derivatives or
auxiliary knowledge
• Use probabilistic transition rules, not deterministic rules
11/25/17 GA 15
GAs vs. Nature
• A solution (phenotype) Individual
• Representation of a solution Chromosome
(genotype )
• Components of the representation Genes
• Set of solutions Population
• Selection:survival of Darwin’s Theory
the fittest
• Search operators Crossover and
Mutation
11/25/17 GA 16
Simple Genetic Algorithm (SGA)
1. Generate random population of chromosomes
2. Decode each chromosome to get an individual
3. Evaluate the fitness of each individual
4. Perform selection, crossover and mutation
5. Repeat steps 2, 3 and 4 until it reaches a
termination condition.
There is no overlapping between generations
11/25/17 GA 17
I–nitialize Population
GA Flowchart
– Evaluate Fitness
Yes
– Terminate? –
Output
solution
No
Perform selection,
–
crossover and mutation
– Evaluate Fitness
11/25/17 GA 18
Encoding and Population
• Encoding: A chromosome encodes a solution in the search space
– Usually binary string of 0’s and 1’s
– Each bit in the string can represent some characteristics of the
solution
– Chromosome size depends on parameter set to be encoded
• Population:
– A set of chromosomes in a population
– Population size is usually constant
– Common practice is to choose the initial population randomly.
11/25/17 GA 19
Chromosome Size
Chromosome: A set of alleles/genes
A chromosome is an encoded form of all parameters describing the
given problem
An encoded parameter – A subset of alleles
Chromosome size = ??
An allele parameter n
A Chromosome
11/25/17 GA 20
Population
string 1
string 2
A population –
string m
11/25/17 GA 21
Fitness Evaluation
• Fitness (objective function) associated with each
chromosome
• Indicates the degree of goodness of the encoded solution
(chromosome)
• only problem specific information (also known as the
payoff information) that GAs use
• for minimization problem
fitness ∝ 1/objective
11/25/17 GA 22
Selection (Reproduction)
• A process in which individual strings are copied according
to their objective function or fitness values : Darwin’s
survival of fittest
• Better individuals get higher chance
– more copies to good strings
– fewer copies to bad strings
• Mimics the natural selection procedure to some extent
• Implementation:
• Proportional selection :
• Roulette wheel selection
• Tournament selection etc.
11/25/17 GA 23
Roulette Wheel Selection
» Assign to each individual a part of the
roulette wheel
» Spin the wheel n times to select n individuals
1/6 = 17%
fitness(A) = 3
A B fitness(B) = 1
– C fitness(C) = 2
3/6 = 50% 2/6 = 33%
11/25/17 GA 24
Elitist Model of GAs
The best string up to the current generation is
preserved
11/25/17 GA 25
Crossover
• Partial exchange of information between two
randomly selected parent chromosome
• Single point crossover : most commonly
used
• Probabilistic operation, µc
11/25/17 GA 26
Single point Crossover (example)
Crossover point
before Crossover after
11/25/17 GA 27
n-point crossover
• Choose n random crossover points
• Split along those points
• Glue parts, alternating between parents
• Generalisation of 1 point (still some positional bias)
11/25/17 GA 28
Mutation
• Random alteration in the genetic structure
• Mutating a binary gene : simple negation of the
bit
• Probabilistic operation : alter each gene
independently with a probability pm
• Exploring the search area : new search points
• Introduce genetic diversity into the population
11/25/17 GA 29
Mutation (example)
Before mutation
After mutation
11/25/17 GA 30
Exploitation vs. Exploration
• Exploration: Discovering promising areas in the search space, i.e.
gaining information on the problem – new search points
• Exploitation: Optimizing within a promising area, i.e. using
information - Exploit historical information: survival of the fittest
co-operation AND competition between them
• Crossover is explorative, it makes a big jump to an area somewhere “in
between” two (parent) areas
• Mutation is exploitative as well as explorative, it creates random small
diversions, thereby staying near (in the area of ) the parent
• Selection is totally exploitive
11/25/17 GA 31
An example of SGA (Goldberg)
• Simple problem: max x2 over {0,1,…,31}
• GA approach:
– Representation: binary code, e.g. 01101 ↔ 13
– Random initialization
– Population size: 4
– Roulette wheel selection
– 1-point crossover, bit wise mutation
We show one generational cycle done by hand
11/25/17 GA 32
x2 example (contd.) : Selection
11/25/17 GA 33
x2 example (contd.) :Crossover
11/25/17 GA 34
x2 example (contd.) :Mutation
11/25/17 GA 35
Shortcomings of SGA
– Representation is too restrictive
– Mutation & crossovers only applicable for bit-
string & integer representations
– Selection mechanism sensitive for converging
populations with close fitness values
11/25/17 GA 36
Parameters
• Population size : usually fixed
• String/chromosome length : usually fixed
• Crossover probability : (µc)
• Mutation probability : (µm)
µc >> µm
• Termination criteria :
– desired fitness, if possible (known)
– generally a maximum number of iterations
11/25/17 GA 37
Termination Criterion
The cycle of GA-operators (selection, crossover and
mutation) is repeated a number of times till :
– A desired objective function value is attained in any string/
chromosome of the population
or
– The average fitness value of a population becomes more or less
constant over a specified number of generations
or
– The number of generations/iterations is greater than some
pre-specified value.
11/25/17 GA 38
TSP representation
Representation is an ordered list of
city numbers known as an order-
based GA.
• Encoding:
• Label the cities 1, 2, 3, …..,
n
• One complete tour is one
permutation, e.g. for n = 8
CityList1 (3 5 7 2 1 6 4 8)
CityList2 (2 5 7 6 8 1 3 4)
11/25/17 GA 39
Crossover
Crossover combines inversion and
recombination:
* *
Parent1 (3 5 – 7 2 1 6 4 8)
Parent2 (2 5 7 6 8 1 3 4)
Child (5 8 – 7 2 1 6 3 4)
This operator is called the Order1 crossover.
11/25/17 GA 40
Mutation
Mutation involves reordering of the list:
* *
Before: (5 8 7 2 1 6 3 4)
– –
After: (5 8 6 2 1 7 3 4)
11/25/17 GA 41
TSP30 - Overview of Performance
1800
1600
1400
1200
Distance
1000
800
600
400
200
0 Best
11
13
15
17
19
21
23
25
27
29
31
1
3
5
7
Worst
Generations (1000)
Average
11/25/17 GA 42
??
• Why not a bad chromosome is totally
eliminated ?
• Mutation is mandatory?
• Population size : Population size is a critical factor
to determine.
– Large : time vs. diversity
– Too small : converge to local optima
11/25/17 GA 43
??
• Total execution time T(approx) ?
I + n ??
given : n = no. of iterations (no. of generations)
p = population size (no. of chromosome)
d = decoding time of the parameters (from chromosome
representation to actual problem/function domain)
E = time to execute the function (problem to be optimized)
to calculate its output/result from the decoded parameters
F = time to calculate the fitness of the function/output/result
I = time to generate initial population
11/25/17 GA 44
Some GA Application Types
Domain Application Types
Control gas pipeline, pole balancing, missile evasion, pursuit
Design semiconductor layout, aircraft design, keyboard
configuration, communication networks
Scheduling manufacturing, facility scheduling, resource allocation
Robotics trajectory planning
Machine Learning designing neural networks, improving classification
algorithms, classifier systems
Signal Processing filter design
Game Playing poker, checkers, prisoner’s dilemma
Combinatorial set covering, travelling salesman, routing, bin packing,
graph colouring and partitioning
Optimization
11/25/17 GA 45
Conclusion
Question: ‘If GAs are so smart, why ain’t they rich?’
Answer: ‘Genetic algorithms are rich - rich in
application across a large and growing
number of disciplines.’
- David E. Goldberg, Genetic Algorithms in Search, Optimization and Machine Learning
11/25/17 GA 46
Discussion
??
11/25/17 GA 47
Thanks !
11/25/17 GA 48
