EC-372: Numerical Analysis

Topic 1:
Introduction to Numerical Methods and
Taylor Series
Sections 1-3:

1
Reference: Dr. Nabil Slides, Dr. Amena Slides
 Section 1
Introduction to Numerical
Analysis

 What are Numerical Analysis?

 Why do we need them?
 Topics covered in EC372.

Reading Assignment: Pages 3-10 of textbook

2
Reference book

 Numerical Methods for

Engineers, 6th edition by
Steven C. Chapra and
Raymond P. Canale

3
Numerical Analysis
Numerical Analysis:
Algorithms that are used to obtain numerical
solutions of a mathematical problem.
Why do we need them?
1. No analytical solution exists,
2. An analytical solution is difficult to obtain
or not practical.

4
What do we need?
Basic Needs in the Numerical Analysis:
 Practical:
Can be computed in a reasonable amount of time.
 Accurate:
 Good approximate to the true value,
 Information about the approximation error

(Bounds, error order,… ).

5
Outlines of the Course
 Taylor Theorem  Solution of linear
 Number Equations
Representation  Least Squares curve
 Solution of nonlinear fitting
Equations  Solution of ordinary
 Interpolation differential equations
 Numerical  Solution of Partial
Differentiation differential equations
 Numerical Integration

6
Solution of Nonlinear
Equations
 Some simple equations can be solved analytically:
x 2  4 x  3 0
 4  4 2  4(1)(3)
Analytic solution roots 
2(1)
x  1 and x  3
 Many other equations have no analytical solution:

x 9  2 x 2  5 0

 x  No analytic solution
x e 

7
Methods for Solving Nonlinear
Equations

o Bisection Method
o Method of False Position
o Fixed-Point Iteration
o Newton-Raphson Method
o Secant Method

8
Solution of Systems of Linear
Equations
x1  x2 3
x1  2 x2 5
We can solve it as :
x1 3  x2 , 3  x2  2 x2 5
 x2 2, x1 3  2 1
What to do if we have
1000 equations in 1000 unknowns.
9
Cramer’s Rule is Not
Practical
Cramer' s Rule can be used to solve the system :
3 1 1 3
5 2 1 5
x1  1, x2  2
1 1 1 1
1 2 1 2

But Cramer' s Rule is not practical for large problems.

To solve N equations with N unknowns, we need (N  1)(N  1)N!
multiplications.
To solve a 30 by 30 system, 2.3 1035 multiplications are needed.
A super computer needs more than 1020 years to compute this.
10
Methods for Solving Systems
of Linear Equations
o Naive Gaussian Elimination
o Gaussian Elimination with Scaled
Partial Pivoting
o LU-Decomposition
o Inverse of a Matrix
o Gauss-Seidel Methods

11
Curve Fitting
 Given a set of data:

x 0 1 2
y 0.5 10.3 21.3

 Select a curve that best fits the data. One

choice is to find the curve so that the sum
of the square of the error is minimized.

12
Interpolation
 Given a set of data:

xi 0 1 2
yi 0.5 10.3 15.3

 Find a polynomial P(x) whose graph passes

through all tabulated points.

yi P ( xi ) if xi is in the table
13
Methods for Curve Fitting
o Least Squares
o Linear Regression
o Nonlinear Least Squares Problems

o Interpolation
o Newton Polynomial Interpolation
o Lagrange Interpolation
o Quadratic Splines

14
Integration
 Some functions can be integrated
analytically:

3 3
1 2 9 1

1
xdx  x   4
2 1 2 2
But many functions have no analytical solutions :
a
 x2
e
0
dx ?

15
Methods for Numerical
Integration
o Trapezoid Method

o Simpson’s Rule

o Gauss Quadrature

16
Solution of Ordinary Differential
Equations
A solution to the differential equation :
x(t )  3 x (t )  3 x(t ) 0
x (0) 1; x(0) 0
is a function x(t) that satisfies the equations.

* Analytical solutions are available for

special cases only.
17
Solution of Partial Differential
Equations
Partial Differential Equations are more
difficult to solve than ordinary differential
equations:

2 2
 u  u
2
 2
 2 0
x t
u (0, t ) u (1, t ) 0, u ( x,0) sin(x)

18
Summary
Topics Covered in the Course
 Numerical Analysis:
Algorithms that are
 Solution of Nonlinear Equations
used to obtain  Solution of Linear Equations
numerical solution of a  Curve Fitting
mathematical problem.
 Least Squares
 We need them when
No analytical solution
 Interpolation
exists or it is difficult to  Numerical Integration
obtain it.  Numerical Differentiation
 Solution of Ordinary Differential
Equations
 Solution of Partial Differential
Equations

19
 Section 2
Number Representation and
Accuracy
 Number Representation
 Normalized Floating Point Representation
 Significant Digits
 Accuracy and Precision
 Rounding and Chopping

Reading Assignment: Chapter 3

20
Representing Real Numbers
 You are familiar with the decimal system:

312.45 3 10 2  1101  2 100  4 10  1  5 10  2

 Decimal System: Base = 10 , Digits (0,1,…,9)

 Standard Representations:

 3 1 2 . 4 5
sign integral fraction
part part
21
Normalized Floating Point
Representation
 Normalized Floating Point Representation:

 d . f1 f 2 f 3 f 4  10n
sign mantissa exponent

d 0, n : signed exponent

 Scientific Notation: Exactly one non-zero digit appears
before decimal point.
 Advantage: Efficient in representing very small or very
large numbers.

22
Binary System
 Binary System: Base = 2, Digits {0,1}

 1. f1 f 2 f 3 f 4  2 n
sign mantissa signed exponent

(1.101)2 (1  1 2  1  0 2  2  1 2  3 )10 (1.625)10

23
Fact
 Numbers that have a finite expansion in one numbering
system may have an infinite expansion in another
numbering system:

(1.1)10 (1.0001100110 01100 ...)2

 You can never represent 1.1 exactly in binary system.

24
IEEE 754 Floating-Point
Standard
 Single Precision (32-bit representation)
 1-bit Sign + 8-bit Exponent + 23-bit Fraction

S Exponent8 Fraction23

 Double Precision (64-bit representation)

 1-bit Sign + 11-bit Exponent + 52-bit Fraction

S Exponent11 Fraction52
(continued)

25
Significant Digits

 Significant digits are those digits that can be

used with confidence.
 Single-Precision: 7.2 Significant Digits

 Double-Precision: 15.9 Significant Digits

CISE301_Topic1 26
Calculator Example
 Suppose you want to compute:
3.578 * 2.139
using a calculator with two-digit fractions

3.57 * 2.13 = 7.60

True answer: 7.653342

27
Significant Digits - Example
48.9

28
Significant
Figures
• Number of significant figures indicates precision.
• Significant digits of a number are those that can be used with
confidence, e.g., the number of certain digits plus one
estimated digit.

CHAPTER (1):TRUNCATION ERRORS A 29

Identifying Significant Digits
• All non-zero digits are considered significant: For example, 91 has two significant
figures, while 123.45 has five significant figures.

• Zeros appearing anywhere between two non-zero digits are significant:

For example: 101.1002 has seven significant figures.

• Leading zeros are not significant:

For example: 0.00052 has two significant figures.

• Trailing zeros in a number containing a decimal point are significant.

For example: 12.2300 has six significant figures: 1, 2, 2, 3, 0 and 0. The number
0.000122300 still has only six significant figures. In addition, 120.00 has five significant
figures.

CHAPTER (1):TRUNCATION ERRORS A 30

• The significance of trailing zeros in a number not containing a decimal
point can be ambiguous. For example, it may not always be clear if a
number like 1300 is accurate to the nearest unit. Various conventions exist
to address this issue.

1.3 x 103 (2 significant digits).

1.30 x 103 (3 significant digits).
1.300 x 103 (4 significant digits).

CHAPTER (1):TRUNCATION ERRORS A 31

Example

How many significant digits of these numbers:

3.14159
0.00082
0.000820
3009
250
250.0
and 1000?

CHAPTER (1):TRUNCATION ERRORS A 32

Solution
Number Number of
significant
digits
3.14159 6 All digits are interesting
only the 8 and 2 is interesting; the zeroes are placeholders;
0.00082 2 they may have rounded something off to get this value.
only the 8, 2 and 0 tell us something; the other zeroes are
0.000820 3 placeholders
the 3 and 9 are interesting, and we have to count the zeroes,
3009 4 because they're between the two interesting numbers.
250 Ambiguous the last zero is just a placeholder.
the zero in the tenths place means that the measurement
was made accurate to the tenths place, and that there just
250.0 4 happen to be zero tenths; the 2 and 5 give useful information,
and the other zero is between significant digits, and must
therefore also be counted
Ambiguous(1 only the 1 is interesting (only it tells us anything specific); we
1000 or 2 or 3 or 4 don't know anything for sure about the hundreds, tens, or
SD) units places; the zeroes may just be placeholders

CHAPTER (1):TRUNCATION ERRORS A 33

Accuracy and Precision

 Accuracy is related to the closeness to the true

value.

 Precision is related to the closeness to other

estimated values.

34
35
Rounding and Chopping
 Rounding: Replace the number by the nearest
machine number.

 Chopping: Throw all extra digits.

36
Error Definitions – True Error

Can be computed if the true value is known:

Absolute True Error

Et  true value  approximation
Absolute Percent Relative Error
true value  approximation
t  *100
true value

37
Error Definitions – Estimated

Error

When the true value is not known:

Estimated Absolute Error

Ea  current estimate  previous estimate
Estimated Absolute Percent Relative Error
current estimate  previous estimate
a  *100
current estimate

38
Notation
We say that the estimate is correct to n
decimal digits if:
n
Error 10

We say that the estimate is correct to n

decimal digits rounded if:
1 n
Error  10
2

39
Summary
 Number Representation
Numbers that have a finite expansion in one numbering system
may have an infinite expansion in another numbering system.

 Normalized Floating Point Representation

 Efficient in representing very small or very large numbers,
 Representation error depends on the number of bits used in
the mantissa.

40