Course on Metaheuristics and Hybrids

c C. Blum Course on Metaheuristics and Hybrids

c C. Blum

Swarm Intelligence Outline

Christian Blum Topics:

Artifical Intelligence Research Institute (iiia) ◮ Swarm intelligence: Short introduction

Spanish National Research Council (csic)
◮ Topic 1: Ant colony optimization
⋆ Inspiration: Foraging behavior of ant colonies

◮ Topic 2: Particle swarm optimization

⋆ Inspiration: Social behavior of flocks of birds and fish schools

c Alex Wild (http://www.myrmecos.net)

Course on Metaheuristics and Hybrids

c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

Outline Swarm intelligence

Topics (continued):

◮ Topic 3: Self-synchronized duty cycling in sensor networks Swarm Intelligence

⋆ Inspiration: Self-synchronization in ant colonies
Short introduction
◮ Topic 4: Distributed graph coloring
⋆ Inspiration: Self-desynchronization of Japanese tree frogs

Course on Metaheuristics and Hybrids

c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

What is swarm intelligence Swarm intelligence

Properties:
In a nutshell: AI discipline whose goal is designing intelligent multi-agent
systems by taking inspiration from the collective behaviour of animal societies ◮ Consist of a set of simple entities
such as ant colonies, flocks of birds, or fish schools
◮ Distributedness: No global control

◮ Self-organization by:
⋆ Direct communication: visual, or chemical contact
⋆ Indirect communication: Stigmergy (Grassé, 1959)
R1 R2 Rn−1

E1 E2 En−1 En

c Ralf Müller By curtesy of www.aerospaceweb.org
c Su Neko Result: Complex tasks/behaviors can be accomplished/exhibited in cooperation
Course on Metaheuristics and Hybrids
c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

Swarm intelligence Current state: distributed optimization/control and robotics

Examples of social insects: Examples:

◮ Ants ◮ Cemetery formation (ants)

◮ Termites ◮ Division of labour / Task allocation (ants + bees)
◮ Some wasps and bees ◮ Self-synchronization of fireflies
◮ Nest construction (termites + ants)
◮ Animal-robot interaction
◮ Flocking (birds + fish)
◮ Foraging behavior of ants

c Velo Steve
c Alex Wild (http://www.myrmecos.net)
c Alex Wild

Course on Metaheuristics and Hybrids

c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

Current state: Cemetery formation (1) Current state: Cemetery formation (2)

Note: Models for cemetery formation (brood tending) are used for clustering

◮ E. D. Lumer and B. Faieta. Diversity and adaptation in populations of

clustering ants. 3rd International Conference on Simulation of Adaptive
Behaviour: From Animals to Animats 3 (SAB 94), 1994
◮ D. Merkle, M. Middendorf, A. Scheidler. Decentralized packet clustering
in router-based networks. Int. J. Found. Comput. Sci., 2005.
◮ R. Klazar, A. P. Engelbrecht. Dynamic Load Balancing Inspired by
Cemetery Formation in Ant Colonies. Swarm Intelligence, 2012.

c by the National Academy of Sciences (PNAS)

Course on Metaheuristics and Hybrids

c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

Current state: distributed optimization/control and robotics Current state: Division of Labour / Task Allocation (1)

◮ Problem: in any colony (ants, bees, etc) are a number of tasks to fulfill
Examples:
◮ Examples: brood tending, foraging for resources, maintaining the nest
◮ Cemetery formation (ants)
◮ Requires: dyanamic allocation of individuals to tasks
◮ Division of labour / Task allocation (ants + bees)
◮ Self-synchronization of fireflies ◮ Depends on: state of the environment, needs of the colony

◮ Nest construction (termites + ants) ◮ Requires: global assessment of the colonies current state

◮ Animal-robot interaction
However: Individuals are unable (as individuals) to make a global assessment
◮ Flocking (birds + fish)
◮ Foraging behavior of ants Solution: Response threshold models
Course on Metaheuristics and Hybrids
c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

Division of Labour / Task Allocation (2) Division of Labour / Task Allocation (3)
Assume that:
This means (continued):
◮ We have m tasks to fulfill
◮ If sj = δij : pij = 0.5
◮ We have n individuals in the colony
◮ An individual i with a low δij is likely to respond to a lower stimulus sj
◮ Each individual i has a response threshold δij for each task j

◮ Let sj ≥ 0 be the stimulus of task j Additional feature: response thresholds are dynamic
◮ An individual engages in task j with probability ◮ Let ∆t be a duration of time.
s2j
pij = ◮ Let xij ∆t be the fraction of time spent by i on task j within ∆t
s2j + δij
2
◮ Then: (1 − xij )∆t is the time spent by i on other tasks
This means:
Response threshold update:
◮ If sj << δij : pij is close to 0
◮ If sj >> δij : pij is close to 1 δij → δij − ξxij ∆t + ρ(1 − xij )∆t

Course on Metaheuristics and Hybrids

c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

Division of Labour / Task Allocation (4) Current state: Division of Labour / Task Allocation (4)

where: References:

◮ ξ is a reinforcement coefficient ◮ M. Campos, E. Bonabeau, G. Theraulaz, and J.-L. Deneubourg. Dynamic

scheduling and division of labor in social insects. Adaptive Behavior,
◮ ρ is a forgetting coefficient
2000.
◮ D. Merkle, M. Middendorf and A. Scheidler. Self-Organized Task
Effects:
Allocation for Service Tasks in Computing Systems with
◮ The more an individual engages in a task j, the lower becomes its threshold Reconfigurable Components, Journal of Mathematical Modelling and
◮ The less an individual engages in a task j, the higher becomes its threshold Algorithms, 2008.
◮ A. Brutschy, G. Pini, C. Pinciroli, M. Birattari, M. Dorigo. Self-organized
task allocation to sequentially interdependent tasks in swarm
robotics. Autonomous Agents and Multi-Agent Systems, 2014.

Course on Metaheuristics and Hybrids

c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

Current state: distributed optimization/control and robotics Current state: Self-synchronization of fireflies (1)

Examples:

◮ Cemetery formation (ants)

◮ Division of labour / Task allocation (ants + bees)
◮ Self-synchronization of fireflies
◮ Nest construction (termites + ants)
◮ Animal-robot interaction
◮ Flocking (birds + fish)
◮ Foraging behavior of ants

By curtesy of www.learner.org
Course on Metaheuristics and Hybrids
c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

Current state: Self-synchronization of fireflies (2) Current state: Self-synchronization of fireflies (3)

Pulse-coupled oscillator model References:

◮ A. Rowe, R. Mangharam and R. Rajkumar. FireFly: A Time

Synchronized Real-Time Sensor Networking Platform, Wireless Ad
Hoc Networking: Personal-Area, Local-Area, and the Sensory-Area Networks,
CRC Press Book Chapter (2006)
◮ N. Lipa, E. Mannes, A. Santos and M. Nogueira. Firefly-inspired and
robust time synchronization for cognitive radio ad hoc networks.
Computer Communications, 2015.
◮ D. Sutantyo and P. Levi. Decentralized underwater multi-robot
communication using bio-inspired approaches. Artificial Life and
Robotics, 2015.

c A. Tyrrell and G. Auer

Course on Metaheuristics and Hybrids

c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

Current state: distributed optimization/control and robotics Current state: Nest construction (1)
Termite mound Ant hill
Examples:

◮ Cemetery formation (ants)

◮ Division of labour / Task allocation (ants + bees)
◮ Self-synchronization of fireflies
◮ Nest construction (termites + ants)
◮ Animal-robot interaction
◮ Flocking (birds + fish)
◮ Foraging behavior of ants

Course on Metaheuristics and Hybrids

c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

Current state: Nest construction (2) Current state: distributed optimization/control and robotics
Automated construction Automated shape forming
Examples:

◮ Cemetery formation (ants)

◮ Division of labour / Task allocation (ants + bees)
◮ Self-synchronization of fireflies
◮ Nest construction (termites + ants)
◮ Animal-robot interaction

◮ J. Werfel, K. Petersen, R. Nagpal. Designing Collective Behavior in a ◮ Flocking (birds + fish)

Termite-Inspired Robot Construction Team, Science, 2014. ◮ Foraging behavior of ants
◮ M. Rubenstein, C. Ahler, N. Hoff, A. Cabrera, R. Nagpal. Kilobot: A Low Cost
Robot with Scalable Operations Designed for Collective Behaviors.
Robotics and Autonomous Systems Journal, 2013.
Course on Metaheuristics and Hybrids
c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

Current state: Animal-robot interaction (1) Current state: Animal-robot interaction (2)
Robot fish Influencing the fish school
Reasons for controlling animal swarms:

1. Studies of animal behaviour. For example:

◮ Studying swarm/herd/group formation
◮ Studying collective decision making

2. Guiding a swarm out of a danger zone

◮ F. Bonnet, Y. Kato, J. Halloy, F. Mondada. Infiltrating the Zebrafish Swarm:

Design, Implementation and Experimental Tests of a Miniature Robotic
Fish Lure for Fish-Robot Interaction Studies. SWARM 2015: The First
International Symposium on Swarm Behavior and Bio-Inspired Robotics, 2015.

Course on Metaheuristics and Hybrids

c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

Current state: Animal-robot interaction (3) Swarm intelligence: examples

Examples:

◮ Cemetery formation (ants)

◮ Division of labour / Task allocation (ants + bees)
◮ Self-synchronization of fireflies
◮ Nest construction (termites + ants)
◮ Animal-robot interaction
◮ Flocking (birds + fish)

◮ S. Gade, A. A. Paranjape, and S.-J. Chung. a Flock of Birds Approaching an ◮ Foraging behavior of ants
Airport Using an Unmanned Aerial Vehicle. AIAA Guidance, Navigation,
and Control Conference, 2015.

Course on Metaheuristics and Hybrids

c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

Flocking (1) Flocking (2)

References:
Definition: The collective motion of a large number of self-propolled entities
◮ J. Kennedy and R. Eberhart. Particle Swarm Optimization, Proceedings
Note: of IEEE International Conference on Neural Networks, pages 1942–1948, 1995
◮ G. Folino, A. Forestiero and G. Spezzano. An adaptive flocking algorithm
◮ Commonly used as a demonstration of emergence and self-organization
for performing approximate clustering, Information Sciences,
◮ Modelled/simulated for the first time by Craig Reynolds (Boids, 1986) 179(18):3059–3078, 2009
◮ X. Cui, J. Gao, and E. Potok. A Flocking based algorithm for document
Model: Basic rules clustering analysis, Journal of Systems Architecture, 52, 505–515, 2006

1. Separation: avoid crowding neighbours (short range repulsion)

2. Alignment: steer towards average heading of neighbours

3. Cohesion: steer towards average position of neighbours (long range attraction)

Course on Metaheuristics and Hybrids
c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

Swarm intelligence: examples Foraging behavior of ants (1)

Examples: Communication strategies:

◮ Cemetery formation (ants) ◮ Direct communication: For example, recruitment

◮ Division of labour / Task allocation (ants + bees) ◮ Indirect communication: via chemical pheromone trails

◮ Self-synchronization of fireflies
◮ Nest construction (termites + ants)
◮ Animal-robot interaction
◮ Flocking (birds + fish)
◮ Foraging behavior of ants

c Alex Wild (http://www.myrmecos.net)
c Christian Blum

Course on Metaheuristics and Hybrids

c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

Foraging behavior of ants (2) Foraging behavior of ants (3)

Communication strategies:

◮ Direct communication: For example, recruitment

◮ Indirect communication: via chemical pheromone trails

Basic behaviour:

c Alex Wild (http://www.myrmecos.net)

Course on Metaheuristics and Hybrids

c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

Foraging behavior of ants (4) Foraging behavior of ants (5)

Nest Food Nest Food

Nest Food Nest Food

Course on Metaheuristics and Hybrids
c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

Foraging behavior of ants (6) Bla

References:
Topic 1: Ant Colony Optimization
◮ M. Dorigo and T. Stützle. Ant Colony Optimization, MIT press, 2004.
Inspiration: Foraging behavior of ant colonies
◮ C. Blum. Ant colony optimization: introduction and recent trends,
Physics of Life Reviews, 2(4):353–373, 2005.
◮ P. Korosec, J. Silc and B. Filipic. The differential ant-stigmergy
algorithm, Information Sciences, 192, 82–97, 2012

Course on Metaheuristics and Hybrids

c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

The ant colony optimization metaheuristic Simulation of the foraging behaviour (1)

Technical simulation:
Outline (ACO part):

Nest e 1 , l1 = 1 Food
a b
◮ Simulation of the foraging behaviour
◮ The ACO metaheuristic
e 2 , l2 = 2
◮ Example: traveling salesman problem (TSP)
1. We introduce artificial pheromone parameters:
◮ Example: assembly line balancing
◮ A closer look at algorithm components T1 for e1 and T2 for e2
◮ ACO for continuous optimization 2. W initialize the phermomone values:

τ1 = τ2 = c > 0

Course on Metaheuristics and Hybrids

c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

Simulation of the foraging behaviour (2) Simulation of the foraging behaviour (3)

Algorithm: Simulation results:

Iterate: 1 1
% of ants using the short path

% of ants using the short path

1. Place na ants in node a. 0.8 0.8

2. Each of the na ants traverses from a to b either 0.6 0.6

◮ via e1 with probability p1 = τ1

τ1 +τ2 , 0.4 0.4

◮ or via e2 with probability p2 = 1 − p1 .

0.2 0.2

3. Evaporate the artificial pheromone: i = 1, 2

0 0
0 50 100 150 0 50 100 150
τi ← (1 − ρ)τi , ρ ∈ (0, 1] iteration iteration

4. Each ant leaves pheromone on its traversed edge ei : Colony size: 10 ants Colony size 100 ants

1
τi ← τi + Observation: Optimization capability is due to co-operation
li
Course on Metaheuristics and Hybrids
c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

Simulation of the foraging behaviour (4) The ant colony optimization metaheuristic

Main differences between model and reality: Outline (ACO part):

Real ants Simulated ants

◮ Simulation of the foraging behaviour
◮ The ACO metaheuristic
Ants’ movement asynchronous synchronized
◮ Example: traveling salesman problem (TSP)

Pheromone laying while moving after the trip ◮ Example: assembly line balancing
◮ A closer look at algorithm components
Solution evaluation implicitly explicit quality measure ◮ ACO for continuous optimization

Problem: In combinatorial optimization we want to find good solutions

Course on Metaheuristics and Hybrids

c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

The ACO framework The ACO pseudocode

input: An instance P of a combinatorial problem P.
InitializePheromoneValues(T )
while termination conditions not met do
ACO Siter ← ∅
solution
for j = 1, . . . , na do
components
s ← ConstructSolution(T )
probabilistic pheromone s ← LocalSearch(s) — optional —
CO problem solution value Siter ← Siter ∪ {s}
construction update end for
ApplyPheromoneUpdate(T )
pheromone
end while
model initialization output: The best solution found
of pheromone
values

Course on Metaheuristics and Hybrids

c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

Metaheuristics: Timeline of their introduction The ant colony optimization metaheuristic

Metaheuristics:
Outline (ACO part):
◮ Simulated Annealing (SA) [Kirkpatrick, 1983]
◮ Tabu Search (TS) [Glover, 1986] ◮ Simulation of the foraging behaviour
◮ Genetic and Evolutionary Computation (EC) [Goldberg, 1989] ◮ The ACO metaheuristic
◮ Ant Colony Optimization (ACO) [Dorigo, 1992] ◮ Example: traveling salesman problem (TSP)
◮ Greedy Randomized Adaptive Search Procedure (GRASP) [Resende, 1995] ◮ Example: assembly line balancing
◮ Particle Swarm Optimization (PSO) [Kennedy, 1995] ◮ A closer look at algorithm components
◮ Guided Local Search (GLS) [Voudouris, 1997] ◮ ACO for continuous optimization
◮ Iterated Local Search (ILS) [Stützle, 1999]
◮ Variable Neighborhood Search (VNS) [Mladenović, 1999]
Course on Metaheuristics and Hybrids
c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

TSP: definition (1) TSP definition (2)

Example: Traveling salesman problem (TSP) . Given a completely connected,

TSP in terms of a combinatorial optimization problem P = (S, f ):
undirected graph G = (V, E) with edge-weights.
◮ S consists of all possible Hamiltonian cycles in G.
2
1 2 ◮ Objetive function f : S 7→ IR+ : s ∈ S is defined as the sum of the edge-weights
1 5 of the edges that are in s.
2 2
2 2 2
1 2 1 2 1 2
3 4
2 1 5 1 5 1 5
2 2 2 2 2 2
Goal:
Find a tour (a Hamiltonian cycle) in G with minimal sum of edge weights.
3 4 3 4 3 4
2 2 2
obj. function value: 8 obj. function value: 10 obj. function value: 10

Course on Metaheuristics and Hybrids

c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

Applying ACO to the TSP TSP: solution construction

Preliminary step: Definition of the Tour construction:

◮ solution components Step 1 Step 2 Finished

◮ pheromone model τ1,2 τ1,2 τ1,2
1 2 1 2 1 2

τ1,4 τ2,3 τ1,4 τ2,3 τ1,4 τ2,3

example instance solution components pheromone model τ1,3 τ2,4 τ1,3 τ2,4 τ1,3 τ2,4

2 c1,2 T1,2 3 4 3 4 3 4
τ3,4 τ3,4 τ3,4
1 2 1 2 1 2

1 5 c1,4 c2,3 T1,4 T2,3

2 2 c1,3 c2,4 T1,3 T2,4 τi,j τi,j
p(ci,j ) = p(ci,j ) =
τ1,2 + τ1,3 + τ1,4 τ2,3 + τ2,4
3 4 3 4 3 4
2 c3,4 T3,4

Course on Metaheuristics and Hybrids

c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

TSP: pheromone update (1) TSP: pheromone update (2)

Pheromone update: For example with the Ant System (AS) update rule Pheromone update: For example with the Ant System (AS) update rule

Pheromone evaporation Reinforcement start p evaporation solution s1 solution s2

2 2
1 2 1 2
P
τi,j ← (1 − ρ) · τi,j τi,j ← τi,j + ρ · F (s) 1 5 1 5
{s∈Siter |ci,j ∈s}
2 2 2 2

where 3 4 3 4
2 2
◮ evaporation rate ρ ∈ (0, 1]
1 2 1 2 1 2 1 2
◮ Siter is the set of solutions generated in the current iteration
◮ quality function F : S 7→ IR+ . We use F (·) = 1
f (·)
3 4 3 4 3 4 3 4
Course on Metaheuristics and Hybrids
c C. Blum Course on Metaheuristics and Hybrids
c Christian Blum

The ant colony optimization metaheuristic The ant colony optimization metaheuristic

Assembly line balancing

Outline (ACO part):

◮ Simulation of the foraging behaviour

◮ The ACO metaheuristic
◮ Example: traveling salesman problem (TSP)
◮ Example: assembly line balancing
◮ A closer look at algorithm components
◮ ACO for continuous optimization
c BMW
c J. Bautista & NISSAN

Specific problem: Simple assembly line balancing (SALB) [Bautista,Pereira,2004]

Course on Metaheuristics and Hybrids

c Christian Blum Course on Metaheuristics and Hybrids
c Christian Blum

The ant colony optimization metaheuristic The ant colony optimization metaheuristic

Additionally given: The maximum number U B of possible work stations

Goal: Minimize the number of work stations needed!

1st step of applying ACO: Solution components and pheromone model

cycle time C = x seconds 1. Solution components: We consider each possible assignment of

◮ a task i
Tasks: Each task i has a time requirement ti
◮ to a work station j
9 8 10
3 5 7 to be a solution component ci,j
11 17
1 2 2. Pheromone model: We assign to each solution component ci,j a pheromone
5 12 3 trail parameter Ti,j with value τi,j
4 6 8

Course on Metaheuristics and Hybrids

c Christian Blum Course on Metaheuristics and Hybrids
c Christian Blum

The ant colony optimization metaheuristic The ant colony optimization metaheuristic

Assumption: Cycle time C = 30 seconds

Solution construction: Work stations are filled with tasks one after the other

Example situation 1:
At each iteration:
9 8 10
◮ j ∗ : The current work station to be filled 3 5 7
11 17
◮ T : The set of tasks 1 2 1 2
5 12 3
1. that are not yet assigned to a work station
4 6 8
2. whose predecessors are all assigned to work stations
3. whose time requirement is such that it fits into j ∗ Example situation 2:
9 8 10

If T is empty: Open a new work station 3 5 7

11 17
1 2 4 1 2
5 12 3
If all tasks assigned: Stop solution construction
4 6 8
Course on Metaheuristics and Hybrids
c Christian Blum Course on Metaheuristics and Hybrids
c Christian Blum

The ant colony optimization metaheuristic The ant colony optimization metaheuristic

At each iteration: How to choose a task from T ? Possible solution: The summation rule [Merkle et al., 2000]

P 
j∗
h=1 τi,h
τi,j ∗ p(ci,j ∗ ) = P P  ∀i∈T
p(ci,j ∗ ) = P ∀i∈T j∗
k∈T τk,j ∗ k∈T h=1 τk,h

Disadvantage in this case: Graphical example: Current work station: 6

τi,1 τi,2 τi,3 τi,4 τi,5 τi,6 τi,7 τi,8

τi,1 τi,2 τi,3 τi,4 τi,5 τi,6 τi,7 τi,8 P

Course on Metaheuristics and Hybrids

c Christian Blum Course on Metaheuristics and Hybrids
c C. Blum

The ant colony optimization metaheuristic The ant colony optimization metaheuristic

Pheromone update: For example with the iteration-best (IB) update rule
Outline (ACO part):

Pheromone evaporation Reinforcement

◮ Simulation of the foraging behaviour

τi,j ← (1 − ρ) · τi,j τi,j ← τi,j + ρ · F (sib ) ∀ ci,j ∈ sib ◮ The ACO metaheuristic
◮ Example: traveling salesman problem (TSP)
where
◮ Example: assembly line balancing
◮ evaporation rate ρ ∈ (0, 1]
◮ A closer look at algorithm components
◮ sib is the best solution constructed in the current iteration
◮ ACO for continuous optimization
◮ quality function F : S 7→ IR+ . We use F (·) = 1
f (·)

Course on Metaheuristics and Hybrids

c Christian Blum Course on Metaheuristics and Hybrids
c Christian Blum

The ant colony optimization metaheuristic The ant colony optimization metaheuristic
Definition of solution components and pheromone model Note: Different pheromone models can be used to solve a problem!

Example: 3 different pheromone models for group shop scheduling

ACO
solution
On the board!
components

probabilistic pheromone
CO problem solution value
construction update

pheromone
model initialization
of pheromone
values
Course on Metaheuristics and Hybrids
c Christian Blum Course on Metaheuristics and Hybrids
c C. Blum

The ant colony optimization metaheuristic Solution construction (1)

Result: Different pheromone models ⇒ different algorithm performance Solution construction: A closer look

0.0008 0.001
0.00075 (4) 0.0009 (2)
0.0007 (2) (4)
average iteration quality

average iteration quality

0.00065 0.0008 (3)

0.0006 0.0007
ACO
0.00055 (3)
0.0006
solution
0.0005
0.00045 AS_JSS_suc (1) (1)
components
AS_JSS_rel (2) 0.0005
0.0004 AS_JSS_pos (3) IB_JSS_suc (1)
IB_JSS_rel (2)
0.00035
AS_JSS_sum (4)
(1)
0.0004
IB_JSS_pos (3) probabilistic pheromone
IB_JSS_sum (4)
0.0003 0.0003
0 500 1000 1500 2000 0 500 1000 1500 2000
CO problem solution value
iteration iteration
construction update
AS-update IB-update
pheromone
model initialization
of pheromone
values

Course on Metaheuristics and Hybrids

c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

Solution construction (2) Solution construction (3)

A general constructive heuristic: Possibilities for implementing ChooseFrom(N (sp )):

◮ sp = hi ◮ Greedy algorithms:
◮ Determine N (sp )
c∗ = argmaxci,j ∈N (sp ) η(ci,j ) ,
◮ while N (sp ) 6= ∅
+
⋆ c ← ChooseFrom(N (sp )) where η : C 7→ IR is a Greedy function
⋆ sp ← extend sp by adding solution component c
⋆ Determine N (sp )
Examples for Greedy functions:
◮ end while
◮ TSP: Inverse distance between nodes (i.e., cities)
Problem: How to implement function ChooseFrom(N (sp ))?
◮ SALB: ti /C

Course on Metaheuristics and Hybrids

c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

Solution construction (4) Pheromone update (1)

Possibilities for implementing ChooseFrom(N (sp )): Pheromone update: A closer look

◮ Ant colony optimization:

α β
[τi,j ] · [η(ci,j )]
p(ci,j | sp ) = P α β
, ∀ ci,j ∈ N (sp ) , ACO
[τk,l ] · [η(ck,l )]
ck,l ∈N (sp )
solution
components
where α and β are positive values
probabilistic pheromone
Note: α and β balance between pheromone information and Greedy function CO problem solution value
construction update
Observations: pheromone
◮ ACO can be applied if a constructive heuristic exists! model initialization
of pheromone
◮ ACO can be seen as an iterative, adaptive Greedy algorithm values
Course on Metaheuristics and Hybrids
c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

Pheromone update (2) Pheromone update (3)

A general update rule: ACO update variants:

X AS-update Supd ← Siter

τi,j ← (1 − ρ) · τi,j + ρ · ws · F (s) ,
{s∈Supd |ci,j ∈s} weights: ws = 1 ∀ s ∈ Supd
elitist AS-update Supd ← Siter ∪ {sbs } (sbs is best found solution)
where
weights: ws = 1 ∀ s ∈ Siter , wsbs = e ≥ 1
◮ evaporation rate ρ ∈ (0, 1]
rank-based AS-update Supd ← best m − 1 solutions of Siter ∪ {sbs } (ranked)
◮ Supd is the set of solutions used for the update
weights: ws = m − r for solutions from Siter , wsbs = m
◮ quality function F : S 7→ IR+ . We use F (·) = 1
f (·) IB-update: Supd ← argmax{F (s) | s ∈ Siter }
◮ ws is the weight of solution s weight 1
BS-update: Supd ← {sbs }
Question: Which solutions should be used for updating?
weight 1

Course on Metaheuristics and Hybrids

c Christian Blum Course on Metaheuristics and Hybrids
c Christian Blum

The ant colony optimization metaheuristic The ant colony optimization metaheuristic

Successful ACO variant: Successful ACO variant:

◮ MAX –MIN Ant System(MMAS) [Stützle, Hoos, 2000] ◮ Ant Colony System(ACS) [Gambardella, Dorigo, 1996]

Characteristic properties: Characteristic properties:

◮ Deterministic construction steps with probability q

◮ Use of a pheromone lower bound τmin > 0
α β
c = argmaxci,j ∈N (sp ) [τi,j ] · [η(ci,j )]
◮ Application of restarts (by re-initializing the phermone values)

◮ Mix of IB-update and BS-update depending on a convergence measure ◮ Evaporation of pheromone during the construction of solution s:

τi,j ← γτi,j + (1 − γ)c , ∀ ci,j ∈ s ,

where c > 0 is the initial pheromone value, and γ ∈ (0, 1]

◮ Use of the BS-update (evaporation only for used solution components)

Course on Metaheuristics and Hybrids

c Christian Blum Course on Metaheuristics and Hybrids
c Christian Blum

The ant colony optimization metaheuristic The ant colony optimization metaheuristic

Successful ACO variant:

Rewriting the HCF update in vector form:
◮ The hyper-cube framework (HCF) for ACO [Blum, Dorigo, 2004]

Charactersitic properties: ~τ ← ~τ + ρ · (m
~ − ~τ ) ,

Limits the pheromone values to the interval [0, 1] by using the folling update:
~ is a |C|-dimensional vector with
where m

X F (s)
τi,j ← (1 − ρ) · τi,j + ρ · P X F (s)
{s∈Supd |ci,j ∈s} s′ ∈S upd
F (s′ ) m
~ = γs · ~s and γs = P .
s∈Supd
F (s′ )
s′ ∈Supd
Course on Metaheuristics and Hybrids
c Christian Blum Course on Metaheuristics and Hybrids
c Christian Blum

The ant colony optimization metaheuristic Theoretical studies of ant colony optimization

Search bias in ant colony optimization:

Example: Problem with 3 solutions, 2 ants per iteration
solution of ant 2
(0, 1, 1) (1, 1, 1)
◮ Positive (and wanted) bias: Choice of (in comparison) good solutions for
updating

◮ Negative bias:
(0, 0, 1) (1, 0, 1) 1. Modelling of the problem

2. Solution construction process

~τ
m
~ 3. Pheromone update
(1, 1, 0)
How to detect negative bias? Decreasing algorithm performance over time
solution of ant 1
(0, 0, 0) (1, 0, 0)

Course on Metaheuristics and Hybrids

c Christian Blum Course on Metaheuristics and Hybrids
c Christian Blum

Theoretical studies of ant colony optimization Theoretical studies of ant colony optimization
Implicit assumptions in ACO: Implicit assumptions in ACO:

Assumption 1: Assumption 1:

Good solutions are composed of good solution components. Good solutions are composed of good solution components.
(A solution component is regarded to be good, if the average quality of (A solution component is regarded to be good, if the average quality of
the solutions that contain it is high.) the solutions that contain it is high.)

Assumption 2: Assumption 2:

The pheromone update is such that good solution components on The pheromone update is such that good solution components on
average are stronger reinforced than others. average are stronger reinforced than others.

Course on Metaheuristics and Hybrids

c Christian Blum Course on Metaheuristics and Hybrids
c Christian Blum

Theoretical studies of ant colony optimization Theoretical studies of ant colony optimization
Example: 2-cardinality tree problem Average iteration quality of Ant System ρ = 0.01

1 2 2 1
v1 v2 v3 v4 v5
e1 e2 e3 e4 0.35
AS_KCT_fo
0.35

0.34 0.34
average iteration quality

average iteration quality

0.33 0.33
3 different solutions:
0.32 0.32

1 2 2 1 0.31 0.31
s1 : v1 v2 v3 v4 v5 f (s1 ) = 3 0.3 0.3

0.29 0.29

0.28 0.28
1 2 2 1 0.27 0.27
AS_KCT_fo
s2 : v1 v2 v3 v4 v5 f (s2 ) = 4 0 500 1000 1500 2000 0 500 1000 1500 2000
iteration iteration

na = 10 na = 1000
1 2 2 1
s3 : v1 v2 v3 v4 v5 f (s3 ) = 3
Course on Metaheuristics and Hybrids
c Christian Blum Course on Metaheuristics and Hybrids
c Christian Blum

Theoretical studies of ant colony optimization Theoretical studies of ant colony optimization
Benchmark instances: Ant System applied to an Internet-like instance Instance statistics:

32

0.00066
AS_KCT_fo 28

32 0.00065 48

44 64 35 37
average iteration quality

28 0.00064 1
36
26
15 18 23

48 62 6 17 40
43

44 64 35 37 0.00063 39 49 38
52
60 41 0 51

36 15 18 23 7 54 31
50 63 14 12
1 26
56 19
43
62 6 17 40
0.00062 10 46
55
42 24 61 29

49 60 41 0 51 3 58 47
39 38 11
52 2 22
50
56
63
19
14 12 7 54 31
0.00061 25 59 21 45

46 42 24 61 29 57 30 16 13 20
10
55

2
11
22
3 58 47
0.0006 5 8 34

25 59 21 45 27 9
4

57 30 16 13 20
0.00059 53

5 8 34 33

27 9
4
0.00058
53
0 500 1000 1500 2000
33

iteration

instance gd96c (65 nodes, 125 edges) 10 ants, ρ = 0.1, k = 30

Course on Metaheuristics and Hybrids

c Christian Blum Course on Metaheuristics and Hybrids
c Christian Blum

Theoretical studies of ant colony optimization Theoretical studies of ant colony optimization

Example: 2-cardinality tree problem

Defintion: Competition-balanced system (CBS)
1 2 2 1
v1 v2 v3 v4 v5
Given: e1 e2 e3 e4

1. a feasible partial solution sp ;

Search tree:
2. and the set of solution components N (sp ) that can be added to extend the
hi
partial solution sp

ce1 ce2 ce3 ce4

An ACO algorithm applied to P ∈ P is called a CBS , if each solution
component c ∈ N (sp ) is a component of the same number of feasible solutions. ce2 ce1 ce3 ce2 ce4 ce3

s1 s1 s2 s2 s3 s3

Therefore: This example is NOT a competition-balanced system

Course on Metaheuristics and Hybrids

c Christian Blum Course on Metaheuristics and Hybrids
c C. Blum

Theoretical studies of ant colony optimization The ant colony optimization metaheuristic

What do we know?
Outline (ACO part):
1. In case an ACO algorithm applied to a problem instance is NOT a
competition-balanced system → possibility of negative search bias
◮ Simulation of the foraging behaviour
2. Existing theoretical result: The Ant System algorithm applied to
unconstrained problems does not suffer from negative search bias ◮ The ACO metaheuristic
◮ Example: traveling salesman problem (TSP)
Open questions:
◮ Example: assembly line balancing
1. Can it be shown that a competition-balanced system does not suffer from
negative search bias? ◮ A closer look at algorithm components

2. ... ◮ ACO for continuous optimization

In general: Research on search bias might lead to better guidelines on how to

develop ACO algorithms
Course on Metaheuristics and Hybrids
c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

Ant colony optimization for continuous optimization Ant colony optimization for continuous optimization
Different approaches:
Continuous optimization
◮ N. Monmarché, G. Venturini and M. Slimane. On how Pachycondyla
Apicalis ants suggest a new search algorithm, Future Generation
Given:
Computer Systems, 16:937–946, 2000.
1. Function f : IRn 7→ IR
◮ K. Socha and M. Dorigo. Ant colony optimization for continuous
2. Constrains such as, for example, xi ∈ [li , ui ] domains, European Journal of Operational Research, 185(3):1155–1173, 2008.
◮ X. M. Hu, J. Zhang and Y. Li. Orthogonal methods based ant colony
Goal: Find search for solving continuous optimization problems, Journal of
Computer Science & Technology, 23:2–18, 2008).
X~ ∗ = (x∗1 , . . . , x∗n ) ∈ IRn ◮ P. Korosec, J. Silc and B. Filipic. The differential ant-stigmergy
such that algorithm, Information Sciences, 192:82–97, 2012.

◮ X~ ∗ fulfills all constraints ◮ T. Liao et al. A unified ant colony optimization algorithm for
continuous optimization, European Journal of Operational Research,
◮ f (X~ ∗ ) ≤ f (Y
~ ), ∀ Y
~ ∈ IRn
234(3):597–609, 2014.

Course on Metaheuristics and Hybrids

c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

Discrete ant colony optimization Continuous ant colony optimization

ACO ACO
solution
components population
Continuous problem
of solutions
probabilistic pheromone probabilistic population
CO problem solution value solution update
construction update construction

pheromone
model initialization Main conceptual difference: initialization
of pheromone of the
values
Population instead of pheromone model population

Course on Metaheuristics and Hybrids

c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

Continuous ACO: Probabilistic solution construction Continuous ACO: Probabilistic solution construction

A solution construction: Choose a value xi ∈ IR for each variable Xi , i = 1, . . . , n A Gaussian kernel PDF:

→ n solution construction steps

How to choose a value for variable Xi ?

Gaussian kernels
individual Gausian functions

→ by sampling the following Gaussian kernel probability density function (PDF):

!
k
X (x−µj )2
1 −
2σj 2
Gi (x) = ωj √ e
j=1
σj 2π
−4 −2 0 2 4

where k is the cardinality of the population P . z

Course on Metaheuristics and Hybrids
c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

Continuous ACO: Probabilistic solution construction Continuous ACO: Probabilistic solution construction

Choice of a Gaussian kernel:

Problem: It is quite difficult to sample a Gaussian kernel PDF
ωj
pj = P k , ∀ j = 1, . . . , k
Solution: Instead, at the start of each solution construction l=1 ωl
1. choose probabilistically one of the Gaussian kernels, denoted by j ∗
Definition of ωj ’s:
2. and sample—for all decision variables—the j ∗ -th Gaussian kernel
(rj −1)2
1 −
Methods for sampling: For example, the Box-Muller method ωj = √ ·e 2q 2 k2

qk 2π

Hereby:

◮ rj is the rank of solution j in population P

◮ q is a parameter of the algorithm: A small q favours high-ranked solutions

Course on Metaheuristics and Hybrids

c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

Continuous ACO: Probabilistic solution construction Continuous ACO: Probabilistic solution construction

Assumption: Gaussian kernel j ∗ is chosen for sampling Definition of µj ∗ :

∗
(x− µj ∗ )2 µj ∗ = xji ,
−
1 σj ∗ 2
j ∗ -th Gaussian kernel =
2
√ e
σj ∗ 2π ∗
where xji is the value of the i-th decision variable of solution j ∗ .

What remains? Definition of Definition of σj ∗ :

1. the mean µj ∗ s
2 
Pk ∗
xli − xji
2. and the standard deviation σj ∗ σj ∗ = ρ l=1 
k

where ρ is a parameter of the algorithm: high ρ means slow convergence speed

Course on Metaheuristics and Hybrids

c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

Continuous ACO Continuous ACO

Example: f (x) = x2 , population size 5, 3 ants, rho = 2.0

Different methods for constraint handling:

1. Repair function: Each unfeasible solution is transformed into a feasible one 1

2. Penalty function: Unfeasible solutions are penalized by high objective 0.8

function values
0.6

0.4

f(x)
0.2 g(x)
h(x)
q(x)
p(x)
0
-10-9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8 9 10
x

Iteration 1
Course on Metaheuristics and Hybrids
c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

Continuous ACO Continuous ACO

Example: f (x) = x2 , population size 5, 3 ants, rho = 2.0 Example: f (x) = x2 , population size 5, 3 ants, rho = 2.0

1 1

0.8 0.8

0.6 0.6

0.4 0.4

f(x) f(x)
0.2 g(x) 0.2 g(x)
h(x) h(x)
q(x) q(x)
p(x) p(x)
0 0
-10-9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8 9 10 -10-9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8 9 10
x x

Iteration 2 Iteration 3

Course on Metaheuristics and Hybrids

c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

Continuous ACO Continuous ACO

Example: f (x) = x2 , population size 5, 3 ants, rho = 2.0 Example: f (x) = x2 , population size 5, 3 ants, rho = 2.0

1 1

0.8 0.8

0.6 0.6

0.4 0.4

f(x) f(x)
0.2 g(x) 0.2 g(x)
h(x) h(x)
q(x) q(x)
p(x) p(x)
0 0
-10-9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8 9 10 -10-9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8 9 10
x x

Iteration 4 Iteration 5

Course on Metaheuristics and Hybrids

c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

Bla PSO: inspiration

Inspiration: Social behaviour observed in animals or insects

Topic 2: Particle Swarm Optimization
Inspiration: Social behavior of flocks of birds and fish schools

Examples:

◮ Bird flocking
◮ Fish schooling
◮ Animal herding
Course on Metaheuristics and Hybrids
c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

PSO: Facts PSO: Basic notations

Note: Notations: Each particle i = 1, . . . , m has a

◮ Invented by J. Kennedy and R. Eberhart in 1995 ◮ current position xi

◮ Initial intention: Modelling the movements of flocks of birds and fish schools ◮ velocity vi
◮ PSO deals with a swarm of particles at each iteration
◮ personal best position pi
◮ Particles move in the solution space in the search for good solutions
Furthermore:
◮ The term particles was used because the notion of velocity was adopted

◮ The initial algorithm is for continuous optimization ◮ Each particle i has (resp., belongs to) a neighborhood N (i) ⊆ {1, . . . , m}

◮ pg is called the neighborhood best position of i

Course on Metaheuristics and Hybrids

c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

PSO: Basic algorithm PSO: Velocity update

input: A continuous optimization problem in n dimensions
Generate for each particle i a random initial position pi , i = 1, . . . , m Basic update rule:
xi := pi vi := vi + ρ1 · (pi − xi ) + ρ2 · (pg − xi )
while termination conditions not met do
vi := vi + ρ1 · (pi − xi ) + ρ2 · (pg − xi )
Consists of:
xi := xi + vi
if f (xi ) > f (pi ) then pi := xi 1. Momentum term: vi
end while → reinforces the previous direction
output: The best solution found
2. Cognitive part: ρ1 · (pi − xi )
→ represents the influence of the best solution seen so far by i
Hereby:
3. Social part: ρ2 · (pg − xi )
◮ ρk = ck · rk , where
→ represents the influence of the best solution seen by the neighborhood of i
1. ck is a so-called acceleration coefficient
2. rk is a n-dimensional vector of random numbers from [0, 1]

Course on Metaheuristics and Hybrids

c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

PSO: Pictorial view of the update PSO: neighborhoods (1)

Basic division:

1. gbest PSO: The neighborhood of each particle is the whole swarm

2. lbest PSO: Neighborhoods are restricted

c X. Li

c University of Málaga
Course on Metaheuristics and Hybrids
c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

PSO: neighborhoods (2) PSO: neighborhoods (3)

More sophisticated neighborhoods: General rules:

◮ The gbest PSO converges fast, but might miss good solutions

◮ A lbest PSO converges slower, but usually performs better

◮ The choice of the right neighborhood is strongly problem dependent
◮ Dynamic neighborhoods perform usually well, but are computationally more
Examples: expensive

◮ Von Neumann Systematic evaluation: Two criteria

◮ Star
1. k: degree of connectivity
◮ Pyramid
2. C: amount of clustering
◮ Dynamic

c University of Málaga Result: higher k favours exploration , while lower k favours exploitation

Course on Metaheuristics and Hybrids

c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

PSO: velocity clampling and inertia weight PSO variants

Problem: Main problem: Finding a balance between exploration and exploitation

◮ Observation: Velocities have the tendency to explode to large values
◮ Result: Particles may leave the boundaries of the search space Some variants:

◮ Attractive and repulsive PSO (ARPSO)

Possible solutions:
Uses different phases of attraction and repulsion between the particles
◮ Velocity clamping: introducing a maximum velocity vmax ◮ Fitness-distance-ratio PSO (FDR-PSO)
◮ Inertia weight w: Encourages interaction between particles that are fit and close to each other

vi := w · vi + ρ1 · (pi − xi ) + ρ2 · (pg − xi ) ◮ Hierarchical PSO (H-PSO)

Consequences:
1. For w > 1: divergence behaviour, for w < 1: convergence behaviour ⋆ Organizes the particles in a dynamically changing tree structure

2. Recommendation from literature: Start with w = 0.9 and decrease to ⋆ Particles are only influenced by their current father
w = 0.4

Course on Metaheuristics and Hybrids

c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

Discrete PSO: binary problems (1) Discrete PSO: binary problems (2)

Note: First discrete PSO introduced in 1997 for binary problems

Extended versions of binary PSO:

Changes: with respect to the standard PSO ◮ L. Y. Chuang, H. W. Chang, C. J. Tu, et al. Improved binary PSO for
feature selection using gene expression data. Computational Biology and
◮ The position vectors xi are binary Chemistry, 32, pages 29–37, 2008
◮ The position update xi := xi + vi is re-interpreted:
◮ Y. Zhang et al. Binary PSO with mutation operator for feature
if (r < S(vid )) then xid = 1, otherwise xid = 0 selection using decision tree applied to spam detection,
Knowledge-Based Systems, 64, pages 22–31, 2014
where S() is a sigmoidal function, mapping all vid to [0, 1]
◮ J. C.-W. Lin et al. A binary PSO approach to mine high-utility
itemsets, Soft Computing, 2016, in press.
Note: The velocity update can now be seen as changing the probability that
bit xi will be 1, i = 1, . . . , n
Course on Metaheuristics and Hybrids
c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

Discrete PSO: TSP example TSP example: positions

Basic update rule: Definition:

vi := vi + ρ1 · (pi − xi ) + ρ2 · (pg − xi ) ◮ Given: TSP instance with n cities

xi := xi + vi
◮ Position: Permutation of the n cities + first city added to the end

Note: In order to apply PSO to any optimziation problem we need to define ... Example:
◮ Position of a particle
2
◮ Velocity of a particle 1 2
1 5
◮ Addition of position and velocity. Result: Position 2 2
◮ Substraction of positions . Result: Velocity
3 4
(1, 2, 4, 3, 1) 2
◮ Addition of velocities. Result: Velocity
Position Graphically
◮ Multiplying a velocity with a real. Result: Velocity

Course on Metaheuristics and Hybrids

c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

TSP example: velocity TSP example: addition of position and velocity

Definition: Definition:

◮ Given: A position x and a velocity v

◮ List of swaps of city identifiers
◮ Apply all swaps of city identifiers of v to position x

Example:
Example: x = (1, 2, 4, 3, 1), v = ((1, 4), (3, 4))
((1, 4), (3, 4))
First swap Second swap
2 2 2
Note: 1 2 1 2 1 2
1 5 1 5 1 5
◮ Null-velocity: empty list 2 2 2 2 2 2

◮ Oposite velocity: reversed list. Example: ((1, 4), (3, 4)) → ((3, 4), (1, 4)) 3 4 3 4 3 4
2 2 2
(1, 2, 4, 3, 1) (4, 2, 1, 3, 4) (3, 2, 1, 4, 3)

Course on Metaheuristics and Hybrids

c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

TSP example: substraction and addition TSP example: multiplication with a real-number

Substraction of positions: Results in a velocity Multiplication: Multiplying a velocity with a real-number

◮ Given: Two positions x1 and x2 . We want x1 − x2 ◮ Given: A velocity v and a real-number r ∈ [0, 1]

◮ Resulting velocity v: sequence of swaps that transforms x1 into x2 ◮ Result: Reduce v to the first 100 · r% of the swaps

◮ Example: x1 = (1, 2, 4, 3, 1), x2 = (1, 4, 2, 3, 1) → v = ((2, 4))

Note:

Addition of velocities: Results in a velocity ◮ This completes the necessary definitions

◮ Given: Two velocities v1 and v2 . We want v = v1 + v2 ◮ This is only an example . This algorithm will not work very well

◮ Resulting velocity v: merge lists v1 and v2 and remove all doubles

◮ Example: v1 = ((1, 3), (2, 4)), v2 = ((1, 4), (2, 4)) → v = ((1, 3), (2, 4), (1, 4))
Course on Metaheuristics and Hybrids
c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

Discrete PSO: Other examples Bla

Some references: Topic 3: Self-Synchronized Duty-Cycling in Sensor

◮ D. Y. Sha and C.-Y. Hsu. A new particle swarm optimization for the
Networks
open shop scheduling problem, Computers and Operations Research, Inspiration: Self-synchronized activity phases of ant colonies
35(10):3243–3261, 2008

◮ Y. Tian et al. A discrete PSO for two-stage assembly scheduling

problem, The International Journal of Advanced Manufacturing Technology,
66 (1-4), pages, 481–499, 2013

◮ M. Gómez-González, A. López and F. Jurado. Hybrid discrete PSO and

OPF approach for optimization of biomass fueled micro-scale energy
system, Energy Conversion and Management, 65, pages 539–545, 2013

Course on Metaheuristics and Hybrids

c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

Self-synchronized activity phases in ant colonies (1) Self-synchronized activity phases in ant colonies (2)
◮ Each ant is modelled as an automaton
Biologist discovered:
◮ The state of an automaton i is described by a continuous state variable:
◮ Colonies of ants show synchronized activity patterns
Si (t) ∈ R where t is the time step
◮ Synchronization is achieved in a self-organized way: self-synchronization
◮ Each automaton i can move on a LxL grid with periodic boundary conditions
◮ Synchronized activity ...
◮ At time step t, each automaton i is either active or inactive :
1. ... provides a mechanism for information propagation
ai (t) = Φ(Si (t) − θact ) , where
2. ... facilitates the sampling of information from other individuals
⋆ θact : activation threshold
Model of self-synchronization: ⋆ Φ(x) = 1 if x ≥ 0, and Φ(x) = 0 otherwise

J. Delgado and R.V. Solé. Self-synchronization and task fulfilment in ant

colonies, Journal of Theoretical Biology, 205, 433–441 (2000)

Course on Metaheuristics and Hybrids

c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

Self-synchronized activity phases in ant colonies (3) Self-synchronized activity phases in ant colonies (4)

What do we measure? Mean activity of the system at time t:

Simulation: At each iteration t

N
1. Activity calculation: 1 X
A(t) = ai (t)
◮ Calculate ai (t) N i=1

◮ If ai (t) = 0: Spontaniously activate i with probability pa (activity level Sa ) where N is the number of automata

2. Move: Each active automaton i moves (if possible) to one of the free places in
its 8-neighborhood
1.0
0.8

3. State variable update:

0.6
activity

0.4

X
0.2

Si (t + 1) = tanh(g · (Si (t) + Sj (t)))

0.0

j∈Ni
3700 3800 3900 4000 4100 4200 4300

where Ni is the 8-neighborhood of the position of i time steps

Course on Metaheuristics and Hybrids
c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

Design of a duty-cycling protocol: organization Design of a duty-cycling protocol: sensor nodes

Organization: Note: Automata are now mobile or static sensor nodes

Each sensor i ...

DC User application DC User application DC User application
0 ∆ 2∆ 3∆ ◮ has a battery with level 0 ≤ bi (t) ≥ 1

◮ maintains a special message queue Qi for incoming duty-cycling messages

Note: during the duty-cycling phase (DC) ...
◮ is equipped with a radio antenna that allows to choose from a set
◮ .... all nodes are awake
{T 1 , . . . , T n } of n different transmission power levels
◮ Each node executes one duty-cyling event at a randomly chosen time.
Includes sending of one message.
Contents of a message m ∈ Qi : the sender nodes state variable value

Course on Metaheuristics and Hybrids

c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

Design of a duty-cycling protocol: energy harvesting Design of a duty-cycling protocol: duty-cycling events
Characteristics:
1: Calculate ai
◮ Sensor nodes that are equipped with solar panels 2: if ai = 0 then
3: Draw a random number p ∈ [0, 1]
◮ Each sensor i harvests a certain amount of energy per time step
4: if p ≤ pa then Si := Sa and ai := 1 endif
Daily sun intensity: 5: end if
6: Determine transmission power level Ti
7: Compute new value for state variable Si
8: Send duty-cycling message m (containing value Si ) with transmission power Ti
1.0
0.8
sun intensity

0.6
0.4
0.2
0.0

0 200 400 600 800 1000 1200 1400

time steps

Course on Metaheuristics and Hybrids

c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

Choice of the transmission power level Experimental Results (1): Network simulator Shawn

Ideal transmission power level: Parameters: 120 static sensor nodes, no packet loss
1.0

Ti := Tmin · (1 − bi ) + Tmax · bi activity

battery
0.8

sun
where
0.6

◮ Tmin : minimum transmission power level

0.4

◮ Tmax : maximum transmission power level

0.2
0.0

13000 13400 13800 14200

time

Note: The average activity of the system is at about 0.6

Course on Metaheuristics and Hybrids
c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

Experimental Results (2): Packet Loss Experimental Results (3): Restricted Energy Harvesting

Parameters: 120 static sensor nodes, different packet loss rates Parameters: 120 static sensor nodes, different cloud densities
1.0

1
0.8

0.8
mean system activity

mean system activity

0.6

0.6
0.4

0.4
0.2

0.2
0.0

0
0 0.2 0.4 0.6 0.8 1.0 0 0.2 0.4 0.6 0.8 1
packet loss rate cloud density

Note: System is very robust up to a packet loss rate of about 0.3 Note: linear relationship between cloud density and system activity

Course on Metaheuristics and Hybrids

c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

Self-synchronized duty-cycling: literature Hybrid metaheuristics

Some Papers:
Topic 4: Distributed Graph Coloring
◮ H. Hernández, C. Blum, M. Middendorf, K. Ramsch and A. Scheidler.
Self-synchronized duty-cycling for mobile sensor networks with energy Inspiration: Self-desynchronization of Japanese tree frogs
harvesting capabilities: A swarm intelligence study. Proceedings of SIS 2009,
pages 153–159, IEEE press, 2009.

◮ H. Hernández and C. Blum. Foundations of ANTCYCLE: Self-synchronized

duty-cycling in mobile sensor networks. The Computer Journal, 54(9),
1427–1448, 2011.

Course on Metaheuristics and Hybrids

c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

De-synchronization in Japanese Tree Frogs (1) De-synchronization in Japanese Tree Frogs (2)

Biologist discovered: Model components:

◮ Male Japanese Tree Frogs de-couple their calls ◮ A set of pulse-coupled oscillators .

◮ WHY? ◮ Some oscillators are coupled, others are independent of each other
⋆ The purpose of the calls is to attract females ◮ Each oscillator i has a phase θi ∈ [0, 1) which changes over time
⋆ Female frogs cannot distinguish between too close calls
1 1
⋆ Result: females cannot determine the correct direction 2
1

Mathematical model:
I. Aihara, H. Kitahata, K. Yoshikawa and K. Aihara. Mathematical modeling
2 2
of frogs’ calling behavior and its possible applications to artificial life
and robotics. Artificial Life and Robotics, 12(1):29–32, 2008.
Course on Metaheuristics and Hybrids
c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

De-synchronization in Japanese Tree Frogs (3) Distributed graph coloring: organization of the algorithm
Topology Suboptimal de-synchronization Optimal de-synchronization
Implementation:
4 3 4 1 1, 3
◮ In static wireless ad-hoc networks (such as sensor networks)
◮ Algorithm works with communication rounds (length: 1 time unit)

1 2 3 2 2, 4 Each network node i maintains:

◮ θi ∈ [0, 1]: a graph coloring event is executed at time θi in each

4 3 5 1 communication round
2, 4 1, 3
◮ ci ∈ {0, 1, ...}: the current color of the node
5

1 2 3 2, 4
5

Course on Metaheuristics and Hybrids

c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

Distributed graph coloring: graph coloring event Distributed graph coloring: re-calculating θi

1: PHASE I
X
2: Recalculate θi θi := θi + inc(thetam − θi )
3: Choose a (new) color ci m∈Mi
4: Send a graph coloring event message m to the neighbors
5: Function inc():
6: PHASE II
7: Execute a kind of distributed local search 1

Messages:
0
◮ Graph coloring event messages are collected in a separate message queue Mi -1 0 1
◮ Each message m contains two values:
⋆ The θ-value thetam of the sender node -1

⋆ The current color colorm of the sender node

Course on Metaheuristics and Hybrids

c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

Distributed graph coloring: choosing a new color ci Experiments: quality of the coloring over time

Example: DIMACS graph DSJC1000.9.col

ci := min{c ∈ N |6 ∃m ∈ Mi with colorm = c}.

Use of the θ-values:

290 300 310 320 330 340

◮ They determine the order in which nodes choose a color

Number of colors

◮ This is in contrast to an exsiting attempt to use frogs’ behavior for graph

coloring

0 20 40 60 80
Communication rounds
Course on Metaheuristics and Hybrids
c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

Experiments: comparison Distributed graph coloring: literature

Avg. improvement over: Algorithm by Finocchi et al. Some Papers:

◮ H. Hernández and C. Blum. Distributed Graph Coloring: An Approach

Based on the Calling Behavior of Japanese Tree Frogs. Swarm
40

Intelligence, 6(2), 117–150, 2012

Improvement (in percent)
30

◮ H. Hernández and C. Blum. FrogSim: distributed graph coloring in

20

wireless ad hoc networks, Telecommunication Systems, 55(2), pages

211–223, 2014
10

◮ C. Blum, B. Calvo and M. J. Blesa. FrogCOL and FrogMIS: new

0

Random DIMACS 1 DIMACS 2 Grids Lee (2008)

Instances decentralized algorithms for finding large independent sets in
graphs, Swarm intelligence, 9(2-3), pages 205–227, 2015
◮ I. Finocchi, A. Panconesi and R. Silvestri. An experimental analysis of
simple distributed vertex coloring algorithms, Algorithmica, 41(1),
1–23, 2005

Course on Metaheuristics and Hybrids

c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

Extension: Finding Large Independent Sets Idea: Use Existing FroSim Algorithm

1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 Observe: relation between graph coloring and independent sets

Independent set Maximal independent set Maximum independent set
5 5

Complexity in centralized settings:

6 1 2 8 6 1 2 8
◮ Problem of finding a maximal independent set is in P
3 4 3 4
◮ Problem of finding a maximum independent set is N P -hard
7 7

Algorithms from the literature: (for the de-centralized case)

◮ Simple distributed greedy algorithms

◮ Iterative self-stabilizing algorithms

Course on Metaheuristics and Hybrids

c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

Experiments (1) Numerical Results: Random geometric graphs of 1000 nodes

radius (r) Greedy FruitFly FrogSim

Competitors:
avg. rounds avg. rounds convergence
◮ Centralized Greedy: Just for interest 0.049 244.88 225.39 66.66 229.76 416.14 734.08
0.0578 190.96 176.50 115.30 180.26 414.50 758.72
◮ FruitFly: Newest iterative self-stabilizing algorithm published in the
0.0666 152.35 142.98 236.02 144.82 325.74 775.16
literature
0.0754 124.53 118.74 480.64 118.82 272.97 770.88
0.0842 103.82 100.91 1114.90 99.55 248.03 754.76
Inspiration of Fruitfly: Development of the fly’s nervous system , when sensory
0.093 87.82 87.19 2562.90 84.93 279.20 755.75
organ precursor (SOP) cells are chosen
0.1018 75.42 76.72 7014.74 73.16 212.85 750.31
0.1106 65.61 67.91 22063.76 64.14 205.49 740.14
Paper:
0.1194 57.84 60.60 53523.54 56.55 215.69 684.79
Afek, Y., Alon, N., Barad, O., Hornstein, E., Barkai, N., Bar-Joseph, Z. A biological
0.1282 51.53 54.60 165323.04 50.16 165.77 700.42
solution to a fundamental distributed computing problem. Science, 331:183–185,
0.134 47.83 50.84 321192.92 46.95 160.96 678.45
2011.
Course on Metaheuristics and Hybrids
c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

Indication For Convergence To Good Solutions (1) Indication for Convergence To Good Solutions (2)

Example: Sparse graph on 1000 nodes Example: Dense graph on 1000 nodes
Average number of times used

Average number of times used

200 40

150 30

First round dist. First round dist.

100 Last round dist. 20 Last round dist.

50 10

0 0

0 5 10 15 0 10 20 30 40
Color index Color index

Course on Metaheuristics and Hybrids

c C. Blum Course on Metaheuristics and Hybrids
c C. Blum

Summary and Conclusions Swarm Intelligence: Quo vadis?

Presented Topics: ◮ Problem: Swarm intelligence has attracted too many people
◮ As a consequence:
◮ Ant colony optimization
1. Experienced researchers were overwhelmed with reviewing
◮ Particle swarm optimization 2. People who should have never been asked to do so did reviewing work

◮ Self-synchronized duty-cycling in sensor networks ◮ Therefore: nowadays we find numerous papers in the literature that are either
1. Non-sense, or
◮ Distributed graph coloring in wireless ad-hoc networks
2. Re-inventing the wheel

Observation: First steps against this trend:

◮ It is still very much possible to find swarm-intelligent behaviors in the ◮ Some journals ( J. of Heur. , Comp. & Oper. Res. ) ask for algorithms to be
biological literature, which are still not used for any technical applications. described in metahpor-free language
◮ Colleagues start to expose the problem ( G. Rudolph , K. Sörensen )

Course on Metaheuristics and Hybrids

c C. Blum

Outlook

Questions?