Advanced Neumerical Methods

Numerical Integration and Differentiation

 The Area Under the Curve
One interpretation of the definite integral is area under the curve

Graphical representation of the integral

f(x)
Area  
b
f(x)dx
a

a b
 Numerical Integration Methods

• Newton-Cotes Methods:
 Trapezoid Rule

 Simpson Rules
 Newton-Cotes Integration Formulas

 The Newton-Cotes formulas are the most common numerical

integration schemes.
 They are based on the strategy of replacing a complicated function or
tabulated data with an approximating function that is easy to integrate:
b b
I   f ( x )dx   f n ( x )dx
a a

f n ( x )  a0  a1 x    an1 x n1  an x n
 Newton-Cotes Methods
 In Newton-Cote Methods, the function is approximated by a
polynomial of order n
 Computing the integral of a polynomial is easy.

 f ( x )d x    
b b
a 0  a 1 x  ...  a n x n d x
a a

(b 2  a 2 ) (b n1  a n1 )

b
f ( x )d x a 0 ( b  a )  a 1  ...  a n
a 2 n1
 Newton-Cotes Methods

• Trapezoid Method (First Order Polynomial are used)

 f ( x )dx    a0  a1 x  dx
b b

a a

• Simpson 1/3 Rule (Second Order Polynomial),

 f ( x )dx    
b b
a0  a1 x  a2 x 2 dx
a a
 The Trapezoidal Rule

 The Trapezoidal rule is the first of the Newton-Cotes closed

integration formulas, corresponding to the case where the polynomial
is first order:
b b
I   f ( x)dx   f1 ( x)dx
a a
 The area under this first order polynomial is an estimate of the integral
of f(x) between the limits of a and b:
 Trapezoid Method

f (b)  f (a ) I   f ( x )dx
b
f ( x )  f (a )  ( x  a)
ba
a

 f (b)  f (a ) 
I    f (a ) 
b
( x  a )  dx
a
 ba 
f(x)  f (b)  f (a ) 
b

  f (a )  a  x
 ba  a
2 b
f (b)  f (a ) x

ba 2 a

f (b)  f (a )
 b  a
a b 2
 Trapezoid Method
f(x)
f (b)

f (a )
ba
Area   f (a)  f (b) 
2
a b
 Trapezoid Method
Multiple Application Rule
f ( x 2 )  f ( x1 )
f(x) Area  x 2  x1 
2

The interval [a, b] is

partitione d into n segments
a  x 0  x1  x 2  ...  x n  b
b

a
f ( x ) dx  sum of the areas
of the trapezoid s
x
x0 x1 x2 x3
a b
 The Multiple Application Trapezoidal Rule

 One way to improve the accuracy of the trapezoidal rule is to divide

the integration interval from a to b into n number of segments and
apply the method to each segment.

 The areas of individual segments can then be added to yield the

integral for the entire interval.

ba
h  a  x0 b  xn
n
x1 x2 xn

I  
x0
f ( x ) dx   f ( x ) dx   
x1

x n 1
f ( x ) dx
 Trapezoid Method
General Formula and special case
If the interval is divided into n segments(n ot necessaril y equal)
a  x 0  x 1  x 2  ...  x n  b
n 1
f ( x )dx    x i 1  x i f ( x i 1 )  f ( x i ) 
b 1
 a
i 0 2

Special Case ( Equally - spaced base points)

x i  1  x i  h for all i
1 n 1

 h  f ( x 0 )  f ( x n )   
b
 a
f ( x ) dx
2 i 1
f( xi ) 

 Example
Given a tabulated values of the velocity of an object.
Obtain an estimate of the distance traveled in the interval [0,3].

Time (s) 0.0 1.0 2.0 3.0

Velocity (m/s) 0.0 10 12 14

Distance = integral of the velocity

3
Distance   0
V (t ) dt
 Example
The interval is divided into 3 subinterva ls.
Base points are0,1,2,3 

Time (s) 0.0 1.0 2.0 3.0

Velocity 0.0 10 12 14
(m/s)

Trapezoid Method
h  xi1  xi  1
 n1 1 
T  h f ( xi )   f ( x0 )  f ( xn )
 i1 2 
 1 
Distance 1(1012)  (0 14)  29
 2 
 Example
x 1.0 1.5 2.0 2.5 3.0

f(x) 2.1 3.2 3.4 2.8 2.7

3  n 1 1 
1 f ( x ) dx  h   f ( x i )   f ( x 0 )  f ( x n ) 
 i 1 2 
 1 
 0 . 5  3 . 2  3 . 4  2 . 8  2 . 1  2 . 7 
 2 
 5 .9
 Simpson’s Rules

 More accurate estimate of an integral is obtained if a high-order

polynomial is used to connect the points. The formulas that result from
taking the integrals under such polynomials are called Simpson’s rules.

Simpson’s 1/3 Rule/

 Results when a second-order Lagrange interpolating polynomial is
used.
 Simpson’s 1/3 rule (Parabola)

Simpson’s 1/3 rule (parabola) Simpson’s 3/8 rule (cubic)

The Multiple-Application of Simpson’s 1/3 Rule

 Just as the trapezoidal rule, Simpson’s rule can be improved by

dividing the integration interval into a number of segments of
equal width.

 Yields accurate results and considered superior to trapezoidal

rule for most applications.

 However, it is limited to cases where values are equally spaced.

 Simpson 1/3 Rule
If there is an extra point between the integration limits a and b, then a
parabola can be used instead of the straight line to replace the function
to be integrated:
f(x)
f(x2)
f(x1)
b b
I   f ( x ).dx   f 2 ( x ).dx f(xo)
a a

x
xo=a x1 x2=b
 Simpson 1/3 Rule
b b b
I 
a
f ( x ).dx   f 2 ( x ).dx   [ Lo f o ( x )  L1 f1 ( x )  L2 f 2 ( x )].dx
a a

f(x)
x  x1 x  x2 f(x2)
f2 ( x)  f ( xo )
xo  x1 xo  x2 f(x1)
x  xo x  x 2 f(xo)
 f ( x1 )
x1  xo x1  x2
x  xo x  x1
 f ( x2 ) x
x2  xo x2  x1
xo=a x1 x2=b
 Simpson 1/3 Rule
 Substitute into the Integral for f2(x) and
integrate: f(x)
f(x2)
f(x1)
I    f ( xo )  4 f ( x1 )  f ( x2 )
h
3
f(xo)
ba
with h 
2

x
f ( xo )  4 f ( x1 )  f ( x2 ) xo=a x1 x2=b
I  (b  a )  h=(b-a)/2
6
Multiple Applications of Simpson 1/3 Rule

Instead of dividing the area into one segment we divided it into n segment
{h=(b-a)/n}:
f(x) 1 4 1
f(b)
1 4 1
b
I  a
f ( x ).dx

x2 x4
f(a)
 
xo
f ( x ).dx  
x2
f ( x ).dx 

xn

....   f ( x ).dx x
xn 2
a (b-a)/n b
Multiple Applications of Simpson 1/3 Rule

 Substitute into the Integral for f(x) by f2(x) in each segment and
integrate:

f ( xo )  4 f ( x1 )  f ( x2 ) f ( x 2 )  4 f ( x 3 )  f ( x4 )
I  2h  2h
6 6
f ( xn  2 )  4 f ( x n 1 )  f ( xn ) ba
..........  2h with h 
6 n
n 1 n 2
f ( xo )  f ( x n )  4 
i 1,3,5
f ( xi )  2 
j  2,4,6
f (xj )
I  (b  a )
3n
 Simpson 3/8 Rule
If there are 2 extra points between the integration limits a and b, then a
3rd degree polynomial can be used instead of the parabola to replace the
function to be integrated:
f(x)
f(x3)
f(x2)
f(x1)
b b f(xo)
I 
a
f ( x ).dx   f 3 ( x ).dx
a
b
[ Lo f o ( x )  L1 f1 ( x ) 
 L f 2 ( x )  L3 f 3 ( x )].dx
x
a 2 xo=a x1 x2 x3=b
 Simpson 3/8 Rule

b b
I  
a
f ( x ).dx   f 3 ( x ).dx
a
f(x)
x  x1 x  x2 x  x3
f(x3)
f3 ( x)  f ( xo ) f(x2)
xo  x1 xo  x2 xo  x3 f(x1)
x  xo x  x 2 x  x 3
 f ( x1 ) f(xo)
x1  xo x1  x2 x1  x3
x  xo x  x1 x  x3
 f ( x2 )
x2  xo x2  x1 x2  x3
x  xo x  x1 x  x2
 f ( x3 ) x
x3  xo x3  x1 x3  x2
xo=a x1 x2 x3=b
 Simpson 3/8 Rule

 Substitute into the Integral for f3(x) and integrate:

3h
I   f ( xo )  3 f ( x1 )  3 f ( x2 )  f ( x3 )
8
ba
with h 
3
f ( xo )  3 f ( x1 )  3 f ( x2 )  f ( x3 )
I  (b  a ) 
8
Newton Cotes Integration Technique: Example
Example 17.3: Find the integral of:
f(x) = 0.2 +25 x – 200 x2 + 675 x3 – 900 x4 + 400 x5
Between the limits 0 to 0.8 using:
1. The trapezoidal rule (ans. 0.1728)
2. The Simpson 1/3 rule (ans. 1.367467)
3. The Simpson 3/8 rule (ans. 1.519170)
4. Multiple application of trapezoidal rule (n=4) (ans. 1.4848)
5. Multiple application of Simpson 1/3 rule (n=4) (ans. 1.623467).
The exact answer to this integration is 1.640533
Numerical Integration and Differentiation
Section-II
Numerical Differentiation
Derivative Graphical Definition
 Introduction
 Calculus is the mathematics of change. Because engineers must
continuously deal with systems and processes that change,
calculus is an essential tool of engineering.

 Standing in the heart of calculus are the mathematical concepts

of differentiation and integration.
y f ( xi  x )  f ( xi )

x x
dy f ( xi  x )  f ( xi )
 x lim
 0
dx x
b
I  f ( x )dx
a
 Motivation
1. How do you evaluate the derivative of a tabulated function.
2. How do we determine the velocity and acceleration from tabulated
measurements.

Time Displacement
(second) (meters)
0 30.1
5 48.2
10 50.0
15 40.2
 Recall

df f ( x  h)  f ( x )
 lim
dx h 0 h

T a y lo r T h e o r e m :
(2)
f ( x )h 2 f (3)
( x )h 3
f ( x  h )  f ( x )  f '( x ) h    O (h4 )
2! 3!
n
E  O ( h n )   r e a l , fin ite C s u c h th a t E  C h
E is o f o r d e r h n  E is a p p r o a c h in g z e r o a t r a te s im ila r to h n
 Numerical Differentiation
Finite divided difference formulae
1) Forward divided difference formula f(x)
2) Backward divided difference formula f(xi+1)
f(x)
3) Central divided difference formula 1
3

f(xi-1) 2 f(xi)

f\(xi)
x
xi-1 xi xi+1
h h
Numerical Differentiation Equations

df ( x ) f ( x  h)  f ( x )
Forward Difference 
dx h

df ( x ) f ( x )  f ( x  h)
Backward Difference 
dx h

df ( x ) f ( x  h)  f ( x  h)
Central Difference 
dx 2h

Which method is better? How do we judge them?

Forward Difference Equation
Forward Difference f ( x  h)  f ( x )  f '( x )h  O( h2 )
 f '( x )h  f ( x  h)  f ( x )  O( h2 )
f ( x  h)  f ( x )
 f '( x )   O ( h)
h

Backward Difference Equation

Backward Difference: f ( x  h)  f ( x )  f '( x )h  O( h2 )
  f '( x )h  f ( x )  f ( x  h)  O( h2 )
f ( x )  f ( x  h)
 f '( x )   O ( h)
h
 Central Difference Equation

Central Difference
f ( 2) ( x )h2 f ( 3) ( x )h3 f (4) ( x )h4
f ( x  h)  f ( x )  f '( x )h     ...
2! 3! 4!
f ( 2) ( x )h2 f ( 3) ( x )h3 f (4) ( x )h4
f ( x  h)  f ( x )  f '( x )h     ...
2! 3! 4!

f ( 3) ( x )h3
f ( x  h)  f ( x  h)  2 f '( x )h  2  ...
3!
f ( x  h)  f ( x  h)
 f '( x )   O ( h2 )
2h
 Three Equations (revisited)

df ( x ) f ( x  h)  f ( x )
Forw ard Difference   O (h)
dx h

df ( x ) f ( x )  f ( x  h)
Backw ard Difference   O (h)
dx h

df ( x ) f ( x  h)  f ( x  h)
Central Difference   O (h2 )
dx 2h

Forw ard and backw ard difference form ulas are com parable in accurcy
Cent ral difference form ula is expected to give better answ er
 Higher Order Equations
f ( 2 ) ( x )h 2 f ( 3 ) ( x )h 3 f ( 4 ) ( x )h 4
f ( x  h )  f ( x )  f '( x ) h     ...
2! 3! 4!
f ( 2 ) ( x )h 2 f ( 3 ) ( x )h 3 f ( 4 ) ( x )h 4
f ( x  h )  f ( x )  f '( x ) h     ...
2! 3! 4!
f ( 2 ) ( x )h 2 f ( 4 ) ( x )h 4
f ( x  h)  f ( x  h)  2 f ( x )  2 2  ...
2! 4!
f ( x  h)  2 f ( x )  f ( x  h)
 f ( x) 
(2)
2
 O ( h 2
)
h

f ( 4 ) ( ) h 2
Error  
12
 Forward Divided Difference
f(x) f(xi+1)
f ( xi  1 )  f ( xi )
f ( x i ) 
1
h
f(xi)
f  ( x i  1 )  f ( x i )
f ( xi ) 
h f\(xi) h
f ( xi  2 )  2 f ( xi  1 )  f ( xi ) x

h2 xi xi+1
f ( xi  3 )  3 f ( xi  2 )  3 f ( xi  1 )  f ( xi )
f ( xi )  3
h
f ( xi  4 )  4 f ( xi  3 )  6 f ( xi  2 )  4 f ( xi  1 )  f ( xi )
f (4)
( xi ) 
h4
 Backward Divided Difference
f ( xi )  f ( xi 1 ) f(x)
f ( x i )  f(x)
h
f  ( x i )  f ( x i  1 ) 2
f ( xi )  f(xi-1) f(xi)
h
f\(xi)
f ( xi )  2 f ( xi 1 )  f ( xi  2 )

h2 h x
xi-1 xi
f ( xi )  3 f ( xi 1 )  3 f ( xi  2 )  f ( xi  3 )
f ( xi ) 
h3
f ( xi )  4 f ( xi 1 )  6 f ( xi  2 )  4 f ( xi  3 )  f ( xi  4 )
f ( 4)
( xi ) 
h4
 Central Divided Difference
f(x)
f(x) f(xi+1)
f ( xi  1 )  f ( xi 1 )
f ( x i ) 
2h 3

f(xi-1) f(xi)
f\(xi)
f ( xi  1 )  2 f ( xi )  f ( xi 1 ) xi+1
f ( xi ) 
h2 h h x
xi-1 xi
f ( xi  2 )  2 f ( xi  1 )  2 f ( xi 1 )  f ( xi  2 )
f ( xi ) 
2h3
f ( xi  2 )  4 f ( xi  1 )  6 f ( xi )  4 f ( xi 1 )  f ( xi  2 )
f (4)
( xi ) 
h4
 Numerical Differentiation: Example
• Find f/(x) at x=0.5 for :
f(x) = 1.2 -0.25 x – 0.5 x2 - 0.15 x3 – 0.1 x4
The exact value is f /(0.5) = -0.9125. Use step sizes h = 0.5 and 0.25
• Solution:
h = 0.5 h=0.25
xi+1 = 1.0 f(1.0)=0.2 xi+1 = 0.75 f(0.75)=0.63633
xi = 0.5 f(0.5)=0.925 xi = 0.5 f(0.5)=0.925
xi-1= 0.0 f(0.0)=1.2 xi-1= 0.25 f(0.0)=1.103516
Forward: f /(0.5) = -1.45 f /(0.5) = -1.155
Backward: f /(0.5) = -0.5 f /(0.5) = -0.714
Central: f /(0.5) = -1.0 f /(0.5) = -0.934
Higher Order Forward Divided Difference

 f ( xi  2 )  4 f ( xi  1 )  3 f ( xi )
f ( x i )   O ( h2 )
2h

 f ( xi  3 )  4 f ( xi  2 )  5 f ( xi  1 )  2 f ( xi )
f ( xi )  2
 O ( h 2
)
h

f ( xi ) 
3 f ( xi  4 )  14 f ( xi  3 )  24 f ( xi  2 )  18 f ( xi  1 )  5 f ( xi )
2h3
 O ( h2 )
 Higher Order Backward Divided Difference

3 f ( xi )  4 f ( xi 1 )  f ( xi  2 )
f ( x i )   O ( h2 )
2h

2 f ( xi )  5 f ( xi 1 )  4 f ( xi  2 )  f ( xi  3 )
f ( xi )  2
 O ( h 2
)
h

f ( xi )
5 f ( xi )  18 f ( xi 1 )  24 f ( xi  2 )  14 f ( xi  3 )  3 f ( xi  4 )
 3
 O ( h 2
)
2h
Higher Order Central Divided Difference

 f ( xi  2 )  8 f ( xi  1 )  8 f ( xi 1 )  f ( xi  2 )
f ( xi )   O( h4 )
12h

f ( xi )
 f ( xi  2 )  16 f ( xi 1 )  30 f ( xi )  16 f ( xi 1 )  f ( xi  2 )
 2
 O ( h 4
)
12h

f ( xi )
 f ( xi  3 )  8 f ( xi  2 )  13 f ( xi 1 )  13 f ( xi 1 )  8 f ( xi  2 )  f ( xi  3 )
 3
 O ( h4
)
8h