1

Principles of Communications
By: Dang Quang Vinh
Faculty of Electronics and Telecommunications
Ho Chi Minh University of Natural Sciences
Convolutional codes
09/2008
2
Introduction
In block coding, the encoder accepts k-
bit message block and generates n-bit
codewordBlock-by-block basis
Encoder must buffer an entire message
block before generating the codeword
When the message bits come in serially
rather than in large blocks, using buffer
is undesirable
Convolutional coding
3
Definitions
An convolutional encoder: a finite-state machine that
consists of an M-stage shift register, n modulo-2 adders
L-bit message sequence produces an output sequence
with n(L+M) bits
Code rate:


L>>M, so


ol) (bits/symb
) ( M L n
L
r
+
=
ol) (bits/symb
1
n
r =
4
Definitions
Constraint length (K): the number of
shifts over which a single message bit
influence the output
M-stage shift register: needs M+1 shifts
for a message to enter the shift register
and come out
K=M+1
5
Example
Convolutional code (2,1,2)
n=2: 2 modulo-2 adders or 2 outputs
k=1: 1 input
M=2: 2 stages of shift register (K=M+1=2+1=3)

Path 1
Path 2
Output
Input
6
Example
Output
Input
Convolutional code (3,2,1)
n=3: 3 modulo-2 adders or 3 outputs
k=2: 2 input
M=1: 1 stages of each shift register (K=2 each)

7
Generations
Convolutional code is nonsystematic code
Each path connecting the output to the input can be
characterized by impulse response or generator
polynomial
denoting the impulse response of the
i
th
path
Generator polynomial of the i
th
path:

D denotes the unit-delay variabledifferent from X of
cyclic codes
A complete convilutional code described by a set of
polynomials { }
) , , ,..., (
) (
0
) (
1
) (
2
) ( i i i i
M
g g g g
) (
0
) (
1
2 ) (
2
) ( ) (
... ) (
i i i M i
M
i
g D g D g D g D g + + + + =
) ( ),..., ( ), (
) ( ) 2 ( ) 1 (
D g D g D g
n
8
Example(1/8)
Consider the case of (2,1,2)
Impulse response of path 1 is (1,1,1)
The corresponding generator polynomial is

Impulse response of path 2 is (1,0,1)
The corresponding generator polynomial is

Message sequence (11001)
Polynomial representation:
1 ) (
2 ) 1 (
+ + = D D D g
1 ) (
2 ) 2 (
+ = D D g
1 ) (
3 4
+ + = D D D m
9
Example(2/8)
Output polynomial of path 1:




Output sequence of path 1 (1001111)
Output polynomial of path 2:



Output sequence of path 2 (1111101)
1
1
) 1 )( 1 (
) ( ) ( ) (
2 3 6
2 3 4 5 4 5 6
2 3 4
) 1 ( ) 1 (
+ + + + =
+ + + + + + + + =
+ + + + =
=
D D D D
D D D D D D D D
D D D D
D g D m D c
1
) 1 )( 1 (
) ( ) ( ) (
2 3 5 4 6
2 3 4
) 2 ( ) 2 (
+ + + + + =
+ + + =
=
D D D D D
D D D
D g D m D c
10
Example(3/8)
m= (11001)
c
(1)
=(1001111)
c
(2)
=(1111101)
Encoded sequence c=(11,01,01,11,11,10,11)
Message length L=5bits
Output length n(L+K-1)=14bits
A terminating sequence of K-1=2 zeros is
appended to the last input bit for the shift
register to be restored to its zero initial state
11
Example(4/8)
Another way to calculate the output:
Path 1:
1 11 100 00
1 011 010 0
1 0011 001
1 1 001 001
0 011 0010
0 11 00100
1 1 001001
output 111 m
c
(1)
=(1001111)
12
Example(5/8)
Path 2
m 101 output
001001 1 1
00100 11 1
0010 011 1
001 001 1 1
00 100 11 1
0 010 011 0
001 0011 1
c
(2)
=(1111101)
13
Example(6/8)
Consider the case of (3,2,1)








denoting the impulse
response of the j
th
path corresponding to i
th

input


Output
Input
) , ,..., , (
) (
0 ,
) (
1 ,
) (
1 ,
) (
,
) ( j
i
j
i
j
M i
j
M i
j
i
g g g g g

=
14
Example(7/8)
Output
Input
D D g g
D D g g
D D g g
D g g
D g g
D D g g
= =
+ = =
= =
= =
= =
+ = =
) ( ) 10 (
1 ) ( ) 11 (
) ( ) 10 (
1 ) ( ) 01 (
1 ) ( ) 01 (
1 ) ( ) 11 (
) 1 (
1
) 3 (
2
) 1 (
1
) 3 (
1
) 2 (
2
) 2 (
2
) 2 (
1
) 2 (
1
) 1 (
1
) 1 (
2
) 1 (
1
) 1 (
1
15
Example(8/8)
Assume that:
m
(1)
=(101)m
(1)
(D)=D
2
+1
m
(2)
=(011)m
(1)
(D)=D+1
Outputs are:
c
(1)
=m
(1)
*g
1
(1)
+m
(2)
*g
2
(1)

= (D
2
+1)(D+1)+(D+1)(1)
=D
3
+D
2
+D+1+D+1=D
3
+D
2
c
(1)
=(1100)
c
(2)
=m
(1)
*g
1
(2)
+m
(2)
*g
2
(2)

= (D
2
+1)(1)+(D+1)(D)
=D
2
+1+D
2
+D=D+1 c
(2)
=(0011)
c
(3)
=m
(1)
*g
1
(3)
+m
(2)
*g
2
(3)

= (D
2
+1)(D+1)+(D+1)(D)
=D
3
+D
2
+D+1+D
2
+D=1=D
3
+1

c
(3)
=(1001)
Output c=(101,100,010,011)
16
State diagram
4 possible states
Each node has 2 incoming
branches, 2 outgoing branches
A transition from on state to
another in case of input 0 is
represented by a solid line and
of input 1 is represented by
dashed line
Output is labeled over the
transition line
state Binary
description
a 00
b 10
c 01
d 11
00
10 01
11
a
b
d
c
0/00
1/11
0/10
1/01
1/10
0/01
1/00
0/11
Consider convolutional code (2,1,2)
17
Example
Message 11001
Start at state a
Walk through the
state diagram in
accordance with
message sequence
00
10 01
11
a
b
d
c
0/00
1/11
0/10
1/01
1/10
0/01
1/00
0/11
00
a
10 01 00
b c a
State
Input 1 0 0
Output
10
b
1
11
01
c
0
01
00
a
0
11 11 10 11
11
d
01
1
18
Trellis(1/2)

a=00
b=10
c=01
d=11
0/00 0/00 0/00
1/10
0/00
1/10
0/00
1/10
0/00
1/10
0/00 0/00
Level j=0 1 5 4 3 2 L-1 L L+1 L+2
19
Trellis(1/2)
The trellis contains (L+K) levels
Labeled as j=0,1,,L,,L+K-1
The first (K-1) levels correspond to the
encoders departure from the initial state a
The last (K-1) levels correspond to the
encoders return to state a
For the level j lies in the range K-1sjsL, all
the states are reachable
20
Example
Message 11001
a=00
b=10
c=01
d=11
0/00 0/00 0/00
1/10
0/00
1/10
0/00
1/10
0/00 0/00
Level j=0 1 5 4 3 2
Input
Output
1 1 0 0 1 0 0
6 7
11 01 01 11 11 10 11
21
Maximum Likelihood Decoding
of Convolutional codes
m denotes a message vector
c denotes the corresponding code vector
r denotes the received vector
With a given r , decoder is required to make estimate
of message vector, equivalently produce an estimate
of the code vector
otherwise, a decoding error
happens
Decoding rule is said to be optimum when the
propability of decoding error is minimized
The maximum likelihood decoder or decision rule is
described as follows:
Choose the estimate for which the log-likelihood
function logp(r/c) is maximum
m

c c m m = =

if only

22
Maximum Likelihood Decoding
of Convolutional codes
Binary symmetric channel: both c and r are binary
sequences of length N






r differs from c in d positions, or d is the Hamming
distance between r and c
[
=
=
N
i
i i
c r p c r p
1
) | ( ) | (

=
=
N
i
i i
c r p c r p
1
) | ( log ) | ( log

=
=

=
i i
i i
i i
c r
c r
p
p
c r p
if
if

1
) | ( with
) 1 log(
1
log
) 1 log( ) ( log ) | ( log
p N
p
p
d
p d N p d c r p
+
|
|
.
|

\
|

=
+ =
23
Maximum Likelihood Decoding
of Convolutional codes
Decoding rule is restated as follows:
Choose the estimate that minimizes the
Hamming distance between the received vector r
and the transmitted vector c
The received vector r is compared with each
possible code vector c, and the one closest
to r is chosen as the correct transmitted
code vector
c

24
The Viterbi algorithm
Choose a path in the trellis whose
coded sequence differs from the
received sequence in the fewest
number of positions
25
The Viterbi algorithm
The algorithm operates by computing a
metric for every possible path in the trellis
Metric is Hamming distance between coded
sequence represented by that path and
received sequence
For each node, two paths enter the node, the
lower metric is survived. The other is
discarded
Computation is repeated every level j in the
range K-1s jsL
Number of survivors at each level s2
K-1
=4
26
The Viterbi algorithm
c=(11,01,01,11,11,10,11),r=(11,00,01,11,10,10,11)
a=00
b=10
c=01
d=11
0/00 0/00 0/00
1/10
0/00
1/10
0/00
1/10
0/00 0/00
Input 11 00 01 11 10 10 11
11 01 01 11 11 10 11
2
0
2
4
1
1
3 2
3 2
6
1
4 3
4 1
3 2
4
3
4
3
2 4
2 4
2 5
4
3
3
2 5
2
5
Code
Output 1 1 0 0 1 0 0
27
Free distance of a conv. code
Performance of a conv. code depends on
decoding algorithm and distance properties of
the code.
Free distance, denoted by d
free
, is a measure of
code s ability to combat channel noise
Free distance: minimum Hamming distance
between any two codewords in the code
d
free
>2t
Since a convolutional code doesn't use blocks,
processing instead a continuous bitstream, the
value of t applies to a quantity of errors located
relatively near to each other
28
Free distance of a conv. code
Conv. code has linear property
So, free distance also defined:


Calculate d
free
by a generating function
Generating function viewed the transfer
function of encoder
Relating input and output by convolution
Generation func relating initial and final state
by multiplication
Free distance
Decoding error probability

.... 000 )] ( [
min
= =
A
X X w d
free
29
Free distance of a conv. code
Modify state diagram
00
10 01
11
a
b
d
c
0/00
1/11
0/10
1/01
1/10
0/01
1/00
0/11
d
a
0
b
c a
1
DL
D
2
L
DL D
D
L
D
2
Signal-flow graph
Exponent of D: Hamming weight of encoder
output on that branch.
Exponent of L: number of nonzero message
bits
30
Free distance of a conv. code
State equations:
c D a
DLd DLb d
Dd Db c
Lc La D b
2
1
0
2
=
+ =
+ =
+ =
d
a
0
b
c a
1
DL
D
2
L
DL D
D
L
D
2
a
0
,b,c,d,a
1
: node signals of the graph
Solve the equation set for a
1
/a
0
. So
the generating func is:

=
=

= =
0
5
5
0
1
) 2 (
2 1
) , (
i
i
DL L D
DL
L D
a
a
L D T
4
5
5 3 7 2 6 5
2 ... 4 2 ) , (

= + + + =
d
d
d d
L D L D L D L D L D T
31
Free distance of a conv. code
T (D,L) represents all possible transmitted
sequences that terminate with c-e
transition
For any d >5, there are 2
d -5
paths with
weight w(X)=d that terminate with c-e
transition, those paths are generated by
messages containing d -4 nonzero bits
The free distance is the smallest of w(X),
so d
free
=5
4
5
5 3 7 2 6 5
2 ... 4 2 ) , (

= + + + =
d
d
d d
L D L D L D L D L D T
32
Systematic conv. code
The message elements appear explicitly in the
output sequence together with the redundant
elements
Path 1
Path 2
Output
Input
33
Systematic conv. code
Impulse response of path 1 is (1,0,0)
The corresponding generator polynomial is

Impulse response of path 2 is (1,0,1)
The corresponding generator polynomial is

Message sequence (11001)
2 ) 1 (
) ( D D g =
1 ) (
2 ) 2 (
+ = D D g
34
Systematic conv. code
Output sequence of path 1 (1100100)
Output sequence of path 2 (1111101)
m= (11001)
c
(1)
=(1100100)
c
(2)
=(1111101)
Encoded sequence
c=(11,11,01,01,11,00,01)

35
Systematic conv. code
Another example of systemmatic conv.
code
Path 1
Path 2
Output
Input
36
Systematic vs nonsystematic
Assumption: T(D,L) is convergent
When T(D,L) is nonconvergent, an finite number
of transmission errors can cause an infinite
number of decoding errors
The code is called catastrophic code
Systematic conv. code cannot be catastrophic
But, for the same constraint length, free distance
of systematic code is smaller than that of
nonsystematic code
Table 10.8
4
5
5 3 7 2 6 5
2 ... 4 2 ) , (

= + + + =
d
d
d d
L D L D L D L D L D T
37
Systematic vs nonsystematic
Maximum free distance with systematic and
nonsystematic conv codes of rate 1/2

K Systematic Nonsystematic
2 3 3
3 4 5
4 4 6
5 5 7
6 6 8
7 6 10
8 7 10