Finite Word Length Effects
(Number representation in register)
Sugumar D
FINITE word length effect
The Digital Signal Processors have finite width
of the data bus.
The word-length after mathematical operations,
if exceeds the bus width, will have to be
omitted.
This is the source of Serious Errors.
We now discuss attributes that cause such
errors.
First we will discuss about
Number representation and
Quantization error.
1.Number representation in registers
The Binary Number System
In conventional digital computers - integers
represented as binary numbers of fixed length n
An ordered sequence
of binary digits
Each digit x i (bit) is 0 or 1
The above sequence represents the integer value X
Upper case letters represent numerical values or
sequences of digits
Lower case letters, usually indexed, represent
individual digits
Radix of a Number System
The weight of the digit xi is the i th power of 2
2 is the radix of the binary number system
Binary numbers are radix-2 numbers allowed digits are 0,1
Decimal numbers are radix-10 numbers allowed digits are 0,1,2,,9
Radix indicated in subscript as a decimal number
Example:
(101) 10 - decimal value 101
(101)2 - decimal value 5
Range of Representations
Operands and results are stored in registers of
fixed length n - finite number of distinct
values that can be represented within an
arithmetic unit
Xmin ; Xmax - smallest and largest
representable values
[Xmin,Xmax] - range of the representable
numbers
A result larger then Xmax or smaller than Xmin
- incorrectly represented
The arithmetic unit should indicate that the
generated result is in error an overflow indication
Signed-magnitude Representation
Uses the high-order bit to indicate the sign
0 for positive
1 for negative
remaining low-order bits indicate the magnitude of the
value
Signed magnitude representation of +41 and -41
0 0 1 0
32
+
1 0 0 1
+ 8 +
41
1 0 1 0
32
1
-
1 0 0 1
+ 8 +
41
Disadvantage of the Signed-Magnitude
Representation
Operation may depend on the signs of the operands
Example - adding a positive number X and a negative
number -Y :
X+(-Y)
If Y>X, final result is -(Y-X)
Calculation switch order of operands
perform subtraction rather than addition
attach the minus sign
A sequence of decisions must be made, costing
excess control logic and execution time
This is avoided in the complement representation
methods
Complement Representations of
Negative Numbers
Two alternatives -
Radix complement (called two's complement in the
binary system)
Diminished-radix complement (called one's complement
in the binary system)
In both complement methods - positive numbers
represented as in the signed-magnitude method
Advantage of Complement Representation
No decisions made before executing addition or
subtraction
No need to interchange the order of the two
operands
Ones Complement
Ones complement replaced signed magnitude
because the circuitry was too complicated.
Negative numbers are represented in ones
complement form by complementing each bit
even the sign
bit is
reversed
0 0 1 0
1 0 0 1
1 1 0 1
0 1 1 0
each 1 is
replaced
with a 0
each 0 is
replaced
with a 1
Twos Complement
The twos complement form of a negative integer
is created by adding one to the ones complement
representation.
0 0 1 0
1 0 0 1
0 0 1 0
1 0 0 1
1 1 0 1
0 1 1 0
+ 1 = 1 1 0 1
0 1 1 1
Twos complement representation has a single
(positive) value for zero.
The sign is represented by the most significant
bit.
The notation for positive integers is identical to
their signed-magnitude representations.
The Twos Complement
Representation
Representation of Mixed Numbers
A sequence of n digits in a register - not
necessarily representing an integer
Can represent a mixed number with a fractional
part and an integral part
The n digits are partitioned into two - k in the
integral part and m in the fractional part (k+m=n)
The value of an n-tuple with a radix point between
the k most significant digits and the m least
significant digits
is
Fractional Binary Numbers
2i
2i1
4
2
1
bi bi1
b2 b1 b0 . b1 b2 b3
1/2
1/4
1/8
bj
Representation
2j
Bits to right of binary point represent fractional powers
of 2
i
k
b
2
Represents rational number: k
k j
Fractional Binary Number Examples
Value
5.3/4
2.7/8
63/64
Observation
Representation
101.112
10.1112
0.1111112
Divide by 2 by shifting right
Numbers of form 0.1111112 just below 1.0
Use notation 1.0
Limitation
Can only exactly represent numbers of the form x/2k
Other numbers have repeating bit representations
Value
Representation
1/3
1/5
1/10
0.0101010101[01]2
0.001100110011[0011]2
0.0001100110011[0011]2
Fixed Point Representations
Radix point not stored in register - understood to
be in a fixed position between the k most
significant digits and the m least significant digits
These are called fixed-point representations
One bit is used for the sign and the remaining bits
for the magnitude.
Clearly there is a restriction to the numbers which
can be represented.
With 7 bits reserved for the magnitude, the
largest and smallest numbers represented are +127
and 127.
Sign bit (+ve number)
+127
-127
= 0 1 1 1 1 1 1 1
= 1 1 1 1 1 1 1 1
Sign bit (-ve number)
Fixed Point Representations
Things to note:
1. Fixed point numbers are represented as exact.
2. Arithmetic between fixed point numbers is also
exact provided the answer is within range.
3. Division is also exact if interpreted as
producing an integer and discarding any
remainder.
Floating Point
floating point representation consists of
A Sign Bit s
An Exponent e
Mantissa /fraction M or F
In floating point representation, numbers are represented by a sign
bit s, an integer component e, a positive integer mantissa M.
Eg of floating pt.
s M B e or (-1)s F Be
S (1 bit) Exponent (3 e bit) Fraction (4 M or F bit)
e-exact exponent.
B- base, usually 2 or 16.
-E bias : fixed int and machine dependent.
If mantissa is assumed to be 1.xxxxx (thus, one bit of the
mantissa is implied as 1)
This is called a normalized representation
8-bit floating point format (2)
sign
1 bit
0
exponent significand number number
3 bits
base 2
base 10
4 bits
001
1001
1.001x21 2.25
011
1100
1.1 x 23
111
1110
1.11 x 27 224.0
001
1110
1.11 x 2-1 0.875
12.0
Distribution of Floating Point Numbers
e = -1
1.00 X 2^(-1) =
1.01 X 2^(-1) =
1.10 X 2^(-1) =
1.11 X 2^(-1) =
1/2
5/8
3/4
7/8
e=0
1.00 X 2^0 =
1.01 X 2^0 =
1.10 X 2^0 =
1.11 X 2^0 =
1
5/4
3/2
7/4
e=1
1.00 X 2^1 = 2
1.01 X 2^1 = 5/2
1.10 X 2^1= 3
1.11 X 2^1 = 7/2
3 bit mantissa
Exponent 2 bit {-1,0,1}
Father of the Floating point standard
IEEE Standard
754 for Binary
Floating-Point
Arithmetic.
1989
ACM Turing
Award Winner!
Prof. Kahan
www.cs.berkeley.edu/~wkahan/
/ieee754status/754story.html
IEEE Floating Point
Defined by IEEE Std 754-1985
Developed in response to divergence of representations
(Established in 1985 as uniform standard for floating point
arithmetic >> Before that, many idiosyncratic formats )
Portability issues for scientific code
Supported by all major CPUs
Now almost universally adopted
Two representations
Single precision (32-bit)
Double precision (64-bit)
Driven by Numerical Concerns
Nice standards for rounding, overflow, underflow
Hard to make go fast
Numerical analysts predominated over hardware types in
defining standard
IEEE 754 Floating-Point Format
single precision
single: 8 bits
double: 11 bits
single: 23 bits
double: 52 bits
S Exponent Fraction
x (1)S (1 Fraction) 2( Exponent Bias)
31
Sign
30
Biased exponent
23
(-1)s F 2E-127
22
Normalized Mantissa (implicit 23rd bit = 1)
S: sign bit (0 non-negative, 1 negative)
Normalize significand: 1.0 |significand| < 2.0
Always has a leading pre-binary-point 1 bit, so no need to represent it
explicitly (hidden bit)
Significand is Fraction with the 1. restored
Exponent: excess representation: actual exponent + Bias
Ensures exponent is unsigned
Single: Bias = 127; Double: Bias = 1203
2.Quantization error
in
number representation
Quantization
1. Fixed-point: truncation
To truncate a fixed-point number from
(+1) bits to (b+1) bits, we just discard
the least significant (-b) bits. The
truncation error is denoted by
t Q( X ) X
Here Q(X) is the truncated version of the number X. For a positive X, the
error is equal to zero if all bits being discarded are zeros and is largest if all
discarded bits are ones.
(2b 2 ) t 0
Quantization
For a negative X, the truncation error will be different for three different
formats:
1) Sign-Magnitude:
0 2 b 2
t
2) Ones-complement:
0 t 2 b 2
3) Twos-complement:
2 b 2 t 0
Quantization
2. Fixed-point: rounding
In case of rounding, the number is quantized to the nearest quantization
level. The rounding error does not depend on the format used to represent
negative numbers:
1 b
1 b
2 2 r 2 2
2
2
In practice, >> b, therefore, 2- 0 in all expressions considered.
Quantization Noise
Quantization mechanisms: (Fixed Point)
Rounding
Truncation
2s Complement
All Positive Numbers
Truncation
Sign Magnitude
1s Complement
output
input
2b
probability
-2-b/2
2-b/2
2b
error
-2-b
2b/2
-2-b
2-b
Quantization Noise
Quantization mechanisms: (Floating Point)
Rounding
Truncation
Sign Magnitude
1s Complement
Truncation
2s Complement
All Positive Numbers
output
input
2b/2
probability
-2-b
2-b
2b/4
2b/2
error
-2.2-b
-2.2-b
2.2-b
Quantization
3. Floating-point
Considering a floating-point representation
Q X 2E Q M
X 2E M
of a number
Quantization is carried out on the mantissa only in case of floating-point
numbers. Therefore, it is more reasonable to consider the relative error.
Q X X QM M
X
M
In practice, a rounding quantizer can be modeled as follows:
Q X 2 B round X 2 B
