• Linear block
codes
• Rectangular codes
• Hamming codes
Single Link Communication Model
End-host
Original source computers Receiving app/user
Digitize Render/display,
(if needed) etc.
Source binary
digits (“message
bits”)
Source decoding
Source coding
Bit stream
Bit stream
Channel Channel
Coding Mapper Recv
Decoding
(bit error Bit + Signals samples Bit
s s (reducing or
correction) Xmit (Voltages) +
over Demapper removing
samples
physical link bit errors)
Embedding for Structural
Separation
Encode so that the codewords are “far enough” from
each other
Likely error patterns shouldn’t transform one codeword
to another
01
Code: nodes chosen in
“0” 00 11 “1” hypercube + mapping
of message bits to nodes
single-bit error may
cause 00 to be 10 10
(or 01)
101 111 “1”
If we choose 2k out of
100 110 2n nodes, it means
we can map all k-bit
message strings in a
space of n-bit
001 011 codewords.
The code rate is k/n.
“0”
000 001100
000
Minimum Hamming Distance of
Code vs.
Detection & Correction Capabilities
If d is the minimum Hamming distance between
codewords, we can:
• detect all patterns of up to t bit errors
if and only if d ≥ t+1
• correct all patterns of up to t bit errors
if and only if d ≥ 2t+1
• detect all patterns of up to tD bit errors
while correcting all patterns of tC (<tD) errors
if and only if d ≥ tC+tD+1
e.g.: d=4,
tC=1, tD=2
6.02 Fall 2012 Lecture 4, Slide
Linear Block Codes
Block code: k message bits encoded to n code bits
i.e., each of 2k messages encoded into a unique n-bit
codeword via a linear transformation.
Key property: Sum of any two codewords is also a
codeword necessary and sufficient for code to be
linear.
(n,k) code has rate k/n.
Sometime written as (n,k,d), where d is the
minimum Hamming Distance of the code.
Generator Matrix of Linear Block
Code
Linear transformation:
C=D.G
C is an n-element row vector containing the
codeword D is a k-element row vector containing the
message
G is the kxn generator matrix
Each codeword bit is a specified linear combination
of message bits.
Each codeword is a linear combination of rows of G.
(n,k) Systematic Linear Block
Codes
• Split data into k-bit blocks
• Add (n-k) parity bits to each block using (n-k)
linear equations, making each block n bits long
k n-k
Message bits Parity bits
n
• Every linear code can be represented by an equivalent
systematic form --- ordering is not significant, direct
inclusion of k message bits in n-bit codeword is.
• Corresponds to using invertible transformations
on rows and permutations on columns of G to get
• G = [I | A] --- identity matrix in the first k columns
Example: Rectangular Parity Codes
P1 is parity bit
Idea: start with rectangular for row #1
array of data bits, add parity
checks for each row and D1 D2 P1
column. Single-bit error in
data will show up as parity D3 D4 P2 (n,k,d)=?
errors in a particular row
and column, pinpointing the P4 is parity
P3 P4
bit that has the error. bit for
column #2
011 011 011
110 100 111
10 10 10
Parity for each row Parity check fails for Parity check only fails
and column is row #2 and column #2 for row #2
correct no errors bit D4 is incorrect bit P2 is incorrect
Rectangular Code Corrects Single
Errors
Claim: The min HD of the rectangular code with r
rows and c columns is 3. Hence, it is a single
error correction (SEC) code.
D1 D2 D3 D4 P1
Code rate = rc / (rc + r + c).
If we add an overall parity bit D5 D6 D7 D8 P2
P, we get a (rc+r+c+1, rc, 4)
code D9 D10 D11 D12 P3
Improves error detection but not
P4 P5 P6 P7 P
correction capability
Proof: Three cases.
(1) Msgs with HD 1 differ in 1 row and 1 col parity
(2) Msgs with HD 2 differ in either 2 rows OR 2 cols
or both HD ≥ 4
(3) Msgs with HD 3 or more HD ≥ 4
Matrix Notation
Task: given k-bit message, compute n-bit codeword. We can
use standard matrix arithmetic (modulo 2) to do the job. For
example, here’s how we would describe the (9,4,4) rectangular
code that includes an overall parity bit.
D1xk Gkxn C1xn
1 0 0 0 1 0 1 0 1
D1
D4 0 1 0 0 1 0 0 1 1
D2 D3 D 1 D 2 D3 D4 P1 P3 P5
0 0 1 0 0 1 1 0 1
0 0 0 1 0 1 0 1 1
P2 P4
1×k k×n 1×n
message generator code word
vector matrix vector
The generator matrix, Gkxn =
Ak(nk )
Ik
k
Decoding Rectangular Parity Codes
Receiver gets possibly corrupted word, w.
Calculates all the parity bits from the data bits.
If no parity errors, return rc bits of data.
Single row or column parity bit error rc data
bits are fine, return them
If parity of row x and parity of column y are in
error, then the data bit in the (x,y) position is
wrong; flip it and return the rc data bits
All other parity errors are uncorrectable.
Return
the data as-is, flag an “uncorrectable error”
Let’s do some rectangular parity
decoding D1 D2 P1
D3 D4 P2
Received codewords P3 P4
1 0 1
0 1 0
1. Decoder
0 action:
1
0 0 0
1 1 1
1 1 2. Decoder action:
0 0 1
0 1 0
0 0 3. Decoder action:
How Many Parity Bits Do Really We
Need?
• We have n-k parity bits, which collectively can
represent 2n-k possibilities
• For single-bit error correction, parity bits need to
represent two sets of cases:
– Case 1: No error has occurred (1 possibility)
– Case 2: Exactly one of the code word bits has an
error (n possibilities, not k)
• So we need n+1 ≤ 2n-k
n ≤ 2n-k – 1
• Rectangular codes satisfy this with big margin ---
inefficient
Hamming Codes
• Hamming codes correct single errors with
the minimum number of parity bits:
n = 2n-k – 1
• (7,4,3)
• (15,11,3)
• (2m –1,2m -1-m,3)
• --- “perfect codes” (but not best!)
Towards More Efficient Codes:
(7,4,3) Hamming Code Example
• Use minimum number of parity bits, each covering
a subset of the data bits.
• No two message bits belong to exactly the same
subsets, so a single-bit error will generate a unique
set of parity check errors.
Suppose we check the
parity and discover that
Modulo-2
D1 P1 and P3 indicate an
addition, P1 error?
aka XOR P2 bit D2 must have flipped
D4
What if only P2
P1 = D1+D2+D4 D2 indicates an error?
P2 itself had the error!
D3
P2 = D1+D3+D4
P
P3 = D2+D3+D4 3
Logic Behind Hamming Code
Construction
• Idea: Use parity bits to cover each axis of the
binary vector space
– That way, all message bits will be covered with a unique
combination of parity bits
Index 1 2 3 4 5 6 7
Binary 001 010 011 100 101 110 111
index
(7,4) P1 P2 D1 P3 D2 D3 D4
code
P1 with binary index 001 covers
P1 = D1+D2+D4
P2 = D1+D3+D4 D1 with binary index 011
P3 = D2+D3+D4 D2 with binary index 101
D4 with binary index 111
Syndrome Decoding: Idea
• After receiving the possibly corrupted message (use
’ to indicate possibly erroneous symbol), compute a
syndrome bit (Ei) for each parity bit
E1 = D’1 + D’2 + D’4 + P’1 0 = D1+D2+D4+P1
E2 = D’1 + D’3 + D’4 + P’2 0 = D1+D3+D4+P2
E3 = D’2 + D’3 + D’4 + P’3 0 = D2+D3+D4+P3
• If all the Ei are zero: no errors
• Otherwise use the particular combination of the Ei
to figure out correction
Index 1 2 3 4 5 6 7
Binar 001 010 011 100 101 110 111
y
index
(7,4) P1 P2 D1 P3 D2 D3 D4
code
6.02 Fall Lecture 4, Slide
Constraints for more than single-bit
errors
Code parity constraint inequality for single-bit errors
1+ n ≤ 2n-k
Write-out the inequality for t-bit errors
6.02 Fall Lecture 4, Slide
Elementary Combinatorics
• Given n objects, in how many ways can we choose
m of them?
If the ordering of the m selected objects matters, then
n(n-1)(n-2) … (n-m+1) = n!/(n-m)!
If the ordering of the m selected objects doesn’t
matter, then the above expression is too large by a
factor m!, so
n n!
“n choose m” =
(n m)!m!
m
6.02 Fall Lecture 4, Slide
Error-Correcting Codes occur
in many other
contexts too
• e.g., ISBN numbers for books,
0-691-12418-3
(Luenberger’s Information Science)
• 1D1+ 2D2+3D3+…+10D10 = 0 mod 11
Detects single-digit errors, and transpositions
6.02 Fall Lecture 4, Slide
MIT
OpenCourseWare
http://ocw.mit.edu
6.02 Introduction to EECS II: Digital Communication Systems
Fall 2012
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.
