HEURISTIC ALGORITHMS (CH.

5, K&S)
 Backtracking algorithms are most useful for generating or enumerating all
solutions.
 If we only want one (possibly optimal) solution, then backtracking may not be
efficient:
 We may “waste” a lot of time before even one solution is found
 To verify that a solution is optimal may require looking at a lot of the search tree.
(Note: if we need to verify it’s optimal, then we don’t have a choice)
 Sometimes it’s sufficient to find a feasible solution that is “good” (nearly
optimal).
→ don’t want to waste time looking at the entire tree
→ heuristic algorithms may be more suitable here.
COSC 4P03 WEEK 7 1
 GENERIC OPTIMIZATION PROBLEM
Generally:
 Take a given solution or partial solution, then apply some
modification to it to obtain a new solution/partial solution.
Given:
 A finite set X
 An objective function (profit) P(x) for x in X
 Feasibility functions (constraints) gj(x) I ≤ j ≤ m
Find:
 The maximum value of P(x) such that gj(x) ≥ 0 for I ≤ j ≤ m
COSC 4P03 WEEK 7 2
 NEIGHBOURHOOD FUNCTIONS
In constructing a heuristic we use a neighbourhood function N(x) that defines
when an element is “close to” element x.
Example:
 Nd(x) = {y in X: dist(x,y) ≤ d}
𝑛
 Then the neighbourhood has size |Nd(x)| = σ𝑑
𝑖=0 𝑖
How can we find feasible solutions in the neighbourhood of a given feasible
solution?
 Exhaustive search
 Randomized search (usually faster)
Output:
 Another feasible solution, or
 “Fail”, indicating we did not find a feasible solution in the neighbourhood.
COSC 4P03 WEEK 7 3
 NEIGHBOURHOOD SEARCH
STRATEGIES – SEE CHAPTER 5, KS
Given a feasible solution x, with neighbourhood N(x), and a profit function P(x):
 Find a feasible solution y in N(x) such that P(y) is maximized; return “fail” if no
feasible solutions exist.
 Find a feasible solution y in N(x) such that P(y) is maximized; if P(y) > P(x)
then return y, else return “fail”. (“steepest ascent”)
 Find any feasible solution y in N(x)
 Find any feasible solution y in N(x); if P(y) > P(x) then return y, else return
“fail”.
 In general a heuristic hN for improving our current solution x could be:
 A single neighbourhood search, or
 A sequence of neighbourhood searches – each successive solution is obtained from the
previous solution by a neighbourhood search.
COSC 4P03 WEEK 7 4
GENERIC HEURISTIC SEARCH – SEE
SECTION 5.1, KS
c = 0;
select a feasible solution X;
Xbest = X;
while(c <= cmax)
{
Y = hN(X); //using chosen search strategy
if(Y != fail)
{
X = Y;
if(P(X) > P(Xbest))
Xbest = X;
}
c++;
}
return Xbest;
COSC 4P03 WEEK 7 5
UNIFORM GRAPH PARTITION (SECTION 5.1.1, KS)

Given:
 A weighted graph G = (V,E) on 2n vertices (for some n)
Find the minimum cost of a partition [X0, X1] of G, where
 V = X0 U X1,
 |X0| = |X1| = n,
 cost([X0, X1]) = Σ (weight(u,v)) where (u,v)  E and u  X0
and v  X1 (the cost of all edges “crossing” the partition)
COSC 4P03 WEEK 7 6
 UNIFORM GRAPH PARTITION (UGP)
EXAMPLE
X0 = {0,2,5,7}, X1 = {1,3,4,6} 0 9 2
Cost of this partition =
weight(1,2) + weight(2,4) + 8
8 2
weight(2,6) + weight(3,5) 2
= 8 + 7 + 2 + 4 = 21. 5 9
7 7 6 1
The set of all possible solutions is 4 6
{[X0, X1] such that |X0| = |X1| = n} 7
Neighbourhood of a given partition
[X0, X1]: set of partitions in which one 3 9 4
element in X0 has been swapped with
one element in X1.

COSC 4P03 WEEK 7 7

 UNIFORM GRAPH PARTITION (UGP)
EXAMPLE
Neighbourhood of [{0,2,5,7}, {1,3,4,6}] is: 0 9 2
[{1,2,5,7}, {0,3,4,6}] (swap 0 & 1),
[{2,3,5,7}, {0,1,4,6}] (swap 0 & 3), etc.
8 2 8
To work out gain from exchanging u in X0 with v 2
in X1, look at edges affected by swapping u and v: 5 7 7 6 9
1
4 6
Gain = cost([X0,X1])
– cost([X0 – {u} U {v}, X1 – {v} U {u}])
7
= σyX 𝑤𝑒𝑖𝑔ℎ𝑡 𝑢, 𝑦 + σx X 𝑤𝑒𝑖𝑔ℎ𝑡(𝑥, 𝑣)
1 0 9
– σyX 𝑤𝑒𝑖𝑔ℎ𝑡(𝑣, 𝑦) – σx X 𝑤𝑒𝑖𝑔ℎ𝑡(𝑥, 𝑢) 3 4
1 0

Positive gain: solution has improved.

Negative gain: solution has worsened.
COSC 4P03 WEEK 7 8
 UNIFORM GRAPH PARTITION (UGP)
EXAMPLE
u v [Y0,Y1] Gain Cost 0 9 2
0 1 1257, 0346 -27 48 8
8 2
2
0 3 2357, 0146 -24 45 5 9
7 7 6 1
0 4 2457, 0136 -34 55 6
4
7
0 6 2567, 0134 -32 53
3 9 4
2 1 0157, 2346 -16 37

2 3 0357, 1246 +3 18 X0 = 0257, X1 = 1346

… … … … … Cost: 8 + 7 + 2 + 4 = 21
COSC 4P03 WEEK 7 9
NEIGHBOURHOOD SEARCH – DECISIONS

 Any design strategy for a heuristic algorithm involves decisions about how to deal
with result of a neighbourhood search.
 We’re going to look at a simple hill-climber, just to give an overview.
 Others in the book: simulated annealing, tabu search and genetic algorithms
Hill-Climbing:
 Given initial solution X, perform an exhaustive neighbourhood search to find Y in
N(X).
 We must have P(Y)>P(X) for any Y in N(X) returned by the search algorithm.
 No such Y exists → search algorithm must return “fail”.
 This strategy tends to find local optimal solutions rather than a global optimal
solution. It’s very dependent on the initial solution.

COSC 4P03 WEEK 7 10

 UNIFORM GRAPH PARTITION – UGP

Algorithm to find initial solution:

SelectPartition()
{
r = random(0, (2n choose n) – 1);
X0 = KSubsetLexUnrank(r, n, 2n);
//note change from book
X1 = V – X0;
}
COSC 4P03 WEEK 7 11
 UGP NEIGHBOURHOOD SEARCH

Ascend([X0, X1])
{
g = 0; // gain
for each i in X0
{ for each j in X1
{ t = gain([X0, X1], i, j);
if(t > g) // current best gain
{ x = i; y = j; g = t; }
} }
if g > 0 // improved
{ Y0 = (X0 U {y}) – {x};
Y1 = (X1 U {x}) – {y};
fail = false;
return ([Y0, Y1]);
}
else // no improvement
{ fail = true;
return ([X0, X1]);
} }
COSC 4P03 WEEK 7 12
 HILL CLIMBING FOR UGP

UGP(cmax)
{
[X0, X1] = SelectPartition();
for(c = 0; c < cmax; c++)
{
[Y0, Y1] = Ascend([X0, X1]);
if(!fail)
// use new partition and try again
{
X0 = Y0;
X1 = Y1;
}
else return; // couldn’t improve
}
}
COSC 4P03 WEEK 7 13
 CRYPTOGRAPHY

 Main reference book: Stinson, “Cryptography,Theory and Practice”, 4th ed.

Main objective:
 Enable two people, A (usually called Alice) and B (usually called Bob) to
communicate over an insecure channel, ensuring that their opponent O
(usually called Oscar) cannot understand their communication.
Terminology:
 Plaintext: the message you wish to send
 Ciphertext: the encrypted plaintext
 Key: information used to encrypt the plaintext and decrypt the ciphertext.

COSC 4P03 WEEK 7 14

 ENCRYPTION AND DECRYPTION
For each key K, there is:
 An encryption rule eK(x) to encrypt the plaintext x
 A decryption rule dK(y) to decrypt the ciphertext y
 Each eK and dK are functions such that dK(eK(x)) = x for every plaintext x
 Each eK is one-to-one, i.e. there should be no y = eK(x1) = eK(x2) for x1 ≠ x2,
or we couldn’t decrypt y unambiguously.
Desirable properties of a cryptosystem:
1. eK and dK should be efficiently computable
2. An opponent should not be able to determine the key or the plaintext.
Cryptanalysis: the process of attempting to compute a key given a string of
ciphertext. Once Oscar has the key, he would simply be able to apply dK to
obtain the plaintext.
COSC 4P03 WEEK 7 15
SHIFT CIPHER

 Also called Caesar cipher since used by Julius Caesar.

 Shifts each letter in the plaintext by a fixed amount.
 There are 26 possible keys (assuming we wish to use the English alphabet).
 Each letter is assigned an integer value between 0 and 25, e.g. a = 0, b = 1, …
 Given a key K:
 eK(x) = x + K mod 26
 dK(y) = y – K mod 26
 Note: this cipher is not secure. In order to break it, we can just try each key
in turn, looking for a message that makes sense. On average we only need to
try 13 keys before breaking it.
COSC 4P03 WEEK 7 16
 SHIFT CIPHER EXAMPLE

K = 11, Plaintext: “meetatmidnight”

Translation:
12 4 4 19 0 19 12 8 3 13 8 6 7 19
Encryption:
23 15 15 4 11 4 23 19 14 24 19 17 18 4
Corresponding ciphertext: “XPPELEXTOYTRSE”
Translation:
23 15 15 4 11 4 23 19 14 24 19 17 18 4
Decryption:
12 4 4 19 0 19 12 8 3 13 8 6 7 19
Corresponding plaintext: “meetatmidnight”
COSC 4P03 WEEK 7 17
SUBSTITUTION CIPHER

 eK: apply a permutation to the alphabet. Each letter in the

plaintext is substituted with the appropriate letter from the
permutation.
 dK: use the inverse permutation.
 Note: the key is just one of the 26! > 4 x 1026 possible
permutations.
 It is impossible to do an exhaustive search to find it.
 However we can apply statistical methods (seen later).
COSC 4P03 WEEK 7 18
 SUBSTITUTION CIPHER EXAMPLE

Possible permutation for encryption:

a b c d e f g h i j k l m n o p q r s t u v w x y z
X N Y A H P O G Z Q WB T S F L R C V MU E K J D I

Corresponding permutation for decryption:

A B C D E F G H I J K L MN O P Q R S T U V WX Y Z
d l r y v o h e z x w p t b g f j q n m u s k a c i
Plaintext: meetatnoon
Ciphertext: THHMXMSFFS

COSC 4P03 WEEK 7 19

VIGENERE CIPHER
(NAMED FOR BLAISE DE VIGENERE, 16TH C.)
 The Shift Cipher and Substitution Cipher are monoalphabetic – a single alphabetic
character is encrypted at a time (mapped to a unique alphabetic character).
 The Vigenere cipher encrypts m characters of plaintext at a time, obtaining m
characters of ciphertext. This is called a polyalphabetic cipher.
 Each letter is assigned an integer value (a = 0, b = 1, etc.)
 Each possible key corresponds to a keyword – a string of length m.
 The plaintext is broken into pieces of length m, to which the key is added.
 Number of possible keywords of length m: 26m.
 For m = 5, the number of keywords = 265 > 1 x 107.
 A computer could break this using an exhaustive search, if m is known. Later, we
will see other cryptanalysis techniques: first find m, then try to discover keyword.
COSC 4P03 WEEK 7 20
 VIGENERE CIPHER EXAMPLE

Keyword: BROCK → key = (1,17,14,2,10)

Plaintext: “meetatmidnight”
Translation:
12 4 4 19 0 19 12 8 3 13 8 6 7 19
Encryption:
13 21 18 21 10 20 3 22 5 23 9 23 21 21
Corresponding ciphertext: “NVSVKUDWFXJXVV”
Translation:
13 21 18 21 10 20 3 22 5 23 9 23 21 21
Decryption:
12 4 4 19 0 19 12 8 3 13 8 6 7 19
Corresponding plaintext: “meetatmidnight”
COSC 4P03 WEEK 7 21
PERMUTATION CIPHER

 Also called Transposition Cipher

 The plaintext is broken into pieces of length m.
 The key is a permutation of length m.
 Each piece of plaintext is permuted according to the key, to obtain
the ciphertext.
 To decrypt, apply the inverse permutation to the ciphertext.
 For m = 5, there are 120 permutations, so exhaustive search is
easy.
 Obviously a long key should be used.

COSC 4P03 WEEK 7 22

 PERMUTATION CIPHER EXAMPLE

Key = 1 2 3 4 5
3 1 5 2 4
Inverse Permutation:
1 2 3 4 5
2 4 1 5 3

Plaintext: “meetatnoon”
Corresponding ciphertext: “EMAETOTNNO”

COSC 4P03 WEEK 7 23

STREAM CIPHERS
 Generate a key stream z1z2…
 A plaintext string x1x2… is encrypted as ciphertext y1y2… via a separate
encryption rule for each element:
y1 = ez1(x1), y2 = ez2(x2), …
 The keystream may repeat: a stream cipher is periodic with period d if zi+d = zi
for all integers i ≥ 1.
 Synchronous stream cipher: keystream is constructed from key, independent
of plaintext string
 Non-synchronous stream cipher: each keystream element depends on
previous plaintext or ciphertext elements.
 The Vigenere cipher is a synchronous stream cipher:
let key = (k1, k2, …, km) → keystream is k1k2…kmk1k2…km…
COSC 4P03 WEEK 7 24
STREAM CIPHERS (2)

 Stream ciphers are often described in terms of binary

alphabets
 If we don’t have a binary alphabet, then we translate what we have to
binary (e.g. use Unicode or ASCII code).
 Use binary addition for encryption and decryption:
 Encryption: ez(x) = (x+z) mod 2
 Decryption: dz(y) = (y+z) mod 2
 Note: this is just the exclusive-or operation → “cheap” to
implement in hardware
COSC 4P03 WEEK 7 25
 STREAM CIPHERS:
GENERATING A SYNCHRONOUS KEYSTREAM
 Let (k1,k2,…,km) be a binary m-tuple
 Let zi = ki for 1 ≤ i ≤ m

 Let zi+m = σ𝑚−1

𝑗=0 (cj∗zi+j) mod 2 for i ≥ 1 and where c0, …, cm-1 are given constants.

 The recurrence has degree m

 Note: we should never use (k1,k2,…,km) = (0,0,…,0) because then ciphertext =
plaintext!
 If the constants c0, …, cm-1 and initial keystream elements (k1,k2,…,km) are chosen
appropriately then we will have a keystream of period 2m-1.
Example: Let m = 4 and let the keystream by generated by the linear recurrence
zi+4 = (zi + zi+1) mod 2, i.e. c0 = 1, c1 = 1, c2 = 0, c3 = 0.
 Suppose keystream is initialized to (1,0,0,0). Then we have a keystream of period 15:
1,0,0,0,1,0,0,1,1,0,1,0,1,1,1,…
COSC 4P03 WEEK 7 26
 STREAM CIPHER: IMPLEMENTATION USING A
LINEAR FEEDBACK REGISTER WITH M STAGES
• Initialize shift register to (kl,k2, …, km)
• k1 is output as the next keystream bit
• k2, …, km are each shifted one position to left to become new k1, …, km-1
• New value of km is computed as σ𝑚−1 𝑗=0 (cj∗kj+1) (using old values of kj)

Example: Let m = 4 and zi+4 = (zi + zi+1) mod 2

COSC 4P03 WEEK 7 27

AUTOKEY CIPHER

 This is an example of a non-synchronous stream cipher (due

to Vigenere?)
 Given initial key K, we have z1 = K and zi = xi-1 if i > 1.
 ez(x) = (x+z) mod 26
 dz(y) = (y-z) mod 26
 Note this is insecure, since there are only 26 possible keys.

COSC 4P03 WEEK 7 28

AUTOKEY CIPHER EXAMPLE

K=8

Plaintext: r e n d e z v o u s
17 4 13 3 4 25 21 14 20 18
Keystream: 8 17 4 13 3 4 25 21 14 20

Addition: 25 21 17 16 7 3 20 9 8 12
Ciphertext: Z V R Q H D U J I M
COSC 4P03 WEEK 7 29
CRYPTANALYSIS

 Cryptanalysis is the process Oscar (the opponent) uses to find the key
 Kerckhoff’s Principle: Oscar knows the cryptosystem being used
 Allows us to evaluate the security of the cryptosystem itself, rather than
relying on Oscar not knowing which cryptosystem is used.
 Types of attacks on cryptosystems:
 Ciphertext-only: Oscar has a string of ciphertext
 Known plaintext: Oscar has a string of plaintext and its corresponding
ciphertext
 Chosen plaintext: Oscar chooses a string of plaintext and is able to construct
the corresponding ciphertext
 Chosen ciphertext: Oscar chooses a string of ciphertext, and is able to
construct the corresponding plaintext
COSC 4P03 WEEK 7 30
STATISTICAL INFORMATION

Relative frequencies of 26 letters in English language, in decreasing order:

 E (12%)
 T, A, O, I, N, S, H, R (6-9% each)
 D, L (4%)
 U, C, M, W, F, G, Y, P, B (1.5 – 2.8%)
 V, K, J, X, Q, Z (less than 1%)
Also look at common sequences of consecutive letters.
 Some common 2-letter sequences (digrams): TH, HE, IN, ER, AN
 Some common 3-letter sequences (trigrams): THE, ING, AND

COSC 4P03 WEEK 7 31

CRYPTANALYSIS OF SUBSTITUTION CIPHER
(CIPHERTEXT-ONLY)

General procedure followed:

 Look for most common letters. If one is far in advance, guess it’s E.
Other common ones are probably from T, A, O, I, N, S, H, R.
 Look at common digrams, especially those involving letters you have
guessed from above. Using these, guess the other letter in the digram.
Most common digrams (in order): TH, HE, IN, ER, AN, RE, … (longer list
in book)
 Try to fill in each of the most common letters one at a time.
 This can vary a lot depending on the ciphertext you have at hand – you
may have to make lots of guesses (many incorrect).

COSC 4P03 WEEK 7 32

EXAMPLE 2.11, STINSON: CIPHERTEXT OBTAINED
FROM A SUBSTITUTION CIPHER

YIFQFMZRWQFYVECFMDZPCVMRZWNMDZVEJBTXCDDUMJ
NDIFEFMDZCDMQZKCEYFCJMYRNCWJCSZREXCHZUNMXZ
NZUCDRJXYYSMRTMEYIFZWDYVZVYFZUMRZCRWNZDZJJ
XZWGCHSMRNMDHNCMFQCHZJMXJZWIEJYUCFWDJNZDIR

COSC 4P03 WEEK 7 33

TABLE 2.3, STINSON: FREQUENCY OF
OCCURRENCE OF 26 CIPHERTEXT LETTERS
Letter Frequency Letter Frequency
A 0 N 9
B 1 O 0
C 15 P 1
D 13 Q 4
E 7 R 10
F 11 S 3
G 1 T 2
H 4 U 5
I 5 V 5
J 11 W 8
K 1 X 6
L 0 Y 10
M 16 Z 20
COSC 4P03 WEEK 7 34
GUESS Z=E, ZW = ED, R = N

------end---------e----ned---e------------
YIFQFMZRWQFYVECFMDZPCVMRZWNMDZVEJBTXCDDUMJ
--------e----e---------n--d---en----e----e
NDIFEFMDZCDMQZKCEYFCJMYRNCWJCSZREXCHZUNMXZ
-e---n------n------ed---e---e--ne-nd-e-e--
NZUCDRJXYYSMRTMEYIFZWDYVZVYFZUMRZCRWNZDZJJ
-ed-----n-----------e----ed-------d---e--n
XZWGCHSMRNMDHNCMFQCHZJMXJZWIEJYUCFWDJNZDIR
COSC 4P03 WEEK 7 35
GUESS N=H, C=A

------end-----a---e-a--nedh--e------a-----
YIFQFMZRWQFYVECFMDZPCVMRZWNMDZVEJBTXCDDUMJ
h-------ea---e-a---a---nhad-a-en--a-e-h--e
NDIFEFMDZCDMQZKCEYFCJMYRNCWJCSZREXCHZUNMXZ
he-a-n------n------ed---e---e--neandhe-e--
NZUCDRJXYYSMRTMEYIFZWDYVZVYFZUMRZCRWNZDZJJ
-ed-a---nh---ha---a-e----ed-----a-d--he--n
XZWGCHSMRNMDHNCMFQCHZJMXJZWIEJYUCFWDJNZDIR
COSC 4P03 WEEK 7 36
GUESS M=I

-----iend-----a-i-e-a-inedhi-e------a---i-
YIFQFMZRWQFYVECFMDZPCVMRZWNMDZVEJBTXCDDUMJ
h-----i-ea-i-e-a---a-i-nhad-a-en--a-e-hi-e
NDIFEFMDZCDMQZKCEYFCJMYRNCWJCSZREXCHZUNMXZ
he-a-n-----in-i----ed---e---e-ineandhe-e--
NZUCDRJXYYSMRTMEYIFZWDYVZVYFZUMRZCRWNZDZJJ
-ed-a--inhi--hai--a-e-i--ed-----a-d--he--n
XZWGCHSMRNMDHNCMFQCHZJMXJZWIEJYUCFWDJNZDIR
COSC 4P03 WEEK 7 37
GUESS Y=O, D=S, F=R, H=C, J=T

o-r-riend-ro--arise-a-inedhise--t---ass-it
YIFQFMZRWQFYVECFMDZPCVMRZWNMDZVEJBTXCDDUMJ
hs-r-riseasi-e-a-orationhadta-en--ace-hi-e
NDIFEFMDZCDMQZKCEYFCJMYRNCWJCSZREXCHZUNMXZ
he-asnt-oo-in-i-o-redso-e-ore-ineandhesett
NZUCDRJXYYSMRTMEYIFZWDYVZVYFZUMRZCRWNZDZJJ
-ed-ac-inhischair-aceti-ted--to-ardsthes-n
XZWGCHSMRNMDHNCMFQCHZJMXJZWIEJYUCFWDJNZDIR
COSC 4P03 WEEK 7 38
SUBSTITUTION CIPHER – PLAINTEXT

“Our friend from Paris examined his empty glass with surprise, as if
evaporation had taken place while he wasn’t looking. I poured some
more wine and he settled back in his chair, face tilted up towards the
sun”

COSC 4P03 WEEK 7 39