ENGG 407

Numerical Methods in Engineering

Lecture 3
NMs for Nonlinear Equations and Systems
Yingxu Wang, Prof., PhD, PEng, FWIF, FICIC, SMIEEE, SMACM
Visiting Professor: MIT (2012), Stanford Univ. (2008), UC Berkeley (2008), Oxford Univ. (1995)
President, International Institute of Cognitive Informatics and Cognitive Computing (ICIC)

Dept. of Electrical and Computer Engineering

S h li h School
Schulich
S h l off Engineering,
E i
i
University
U i
it off Calgary
C l
Office: ICT542, Tel: 403 220 6141
[email protected]
p
g y
http://www.ucalgary.ca/icic/

ENGG407 - Numerical Methods in Engineering, Dr. Y. Wang, 22/09/14 L3-1

1. Introduction

1. Introduction
2. The bisection method
3 The regula falsi method
3.
4. Newtons method
5. The secant method
6. The Newton-Raphson method for
f SNLEs
S
7. MATLAB applications in NLE analyses
y
8. Summary

ENGG407 - Numerical Methods in Engineering, Dr. Y. Wang, 22/09/14 L3-2

Nonlinear Equations and Systems

A nonlinear equation is a function in the form of
f ( x) 0

f ( x, y ) 0
...
f ( x1 , x2 ,,...,, xn ) 0
A system of nonlinear equations (SNLE) is a set of
multiple simultaneous nonlinear functions, m n, i.e.:

f1 ( x1 , x2 ,..., xn ) 0
f ( x , x ,,...,, x ) 0
2 1 2
n

...
f m ( x1 , x2 ,,...,, xn ) 0
ENGG407 - Numerical Methods in Engineering, Dr. Y. Wang, 22/09/14 L3-3

Solutions/Roots of Nonlinear Equations

SNLE

ENGG407 - Numerical Methods in Engineering, Dr. Y. Wang, 22/09/14 L3-4

Approaches to Solving NLEs

Bracketing
g methods

Open
p methods

- The bisection method

- The regula falsi method

- Newtons method
- The secant method

2. The Bisection Method

1. Introduction
2. The bisection method
3 Th
3.
The regula
l falsi
f l i method
th d
4. Newtons method
5. The secant method
6. The Newton-Raphson method for SNLEs
7. MATLAB applications in NLE analyses
8 Summary
8.

ENGG407 - Numerical Methods in Engineering, Dr. Y. Wang, 22/09/14 L3-6

Conceptual Model of the Bisection Method

The bisection method is a bracketing approximation for
iteratively finding the root for a function f(x) = 0 in [a, b]
determined by the midpoint in [a,
[a b].
b]
Existence of root
f(x) as continuous in [a,
[a b] f(a)f(b) < 0 x0, f(x0) = 0
Convergence
- Always exists
- Converges slowly

Stop condition
= (b - a)) / 2 <
tolerance or
f(x(i))
( ( )) < tolerance < tolerance
ENGG407 - Numerical Methods in Engineering, Dr. Y. Wang, 22/09/14 L3-7

Mathematical Model of the Bisection Method

Given f ( x ) 0 in [a , b ],
]
ai bi
The i +1th solution is determined by: xi 1
2
subject to: f ( ai ) f (bi ) 0
when f ( x ) is a parabola touching
tou ching x -axis,
axis, use open methods.
methods .

MATLAB: BisectionRoot (1/2)

function [xN] = BisectionRoot (Fun, a, b, imax, tol)
Fa = Fun(a);
Fb = Fun(b);
if Fa*Fb > 0
disp('Error: The function has the same sign at points a and b.')
else
disp ('iteration
( iteration a
b
xN
f(xN)
Tolerance')
Tolerance )
for i = 1 : imax
ai
xN = (a + b)/2;
x i1
toli = (b - a)/
to
a)/2;;
%C
Cf. to
toli = FxN - 0
2
FxN = Fun(xN);

bi

fprintf ('%3i %11.6f %11.6f %11.6f %11.6f %11.6f\n', i, a, b, xN, FxN, toli)

if FxN == 0
fprintf ('An exact solution x = %11.6f was found', xN)
break
end

ENGG407 - Numerical Methods in Engineering, Dr. Y. Wang, 22/09/14 L3-9

MATLAB: BisectionRoot (2/2)

if toli < tol

break
end
if i == imax
fprintf (('Solution
Solution was not obtained in %i iterations
iterations',, imax)
break
end
if Fa*FxN < 0
b = xN;
else
a = xN;
end
end
end

ENGG407 - Numerical Methods in Engineering, Dr. Y. Wang, 22/09/14 L3-10

Application of BisectionRoot (1/2)

See Program 3-1.

ENGG407 - Numerical Methods in Engineering, Dr. Y. Wang, 22/09/14 L3-11

Application of the BisectionRoot (2/2)

>> Fun = @ (x) 8-4
8 4.5
5*(x-sin(x));
(x sin(x));
>> a = 2; b = 3; imax = 20; tol = 0.001;
>> BisectionRoot (Fun, a, b, imax, tol)

ENGG407 - Numerical Methods in Engineering, Dr. Y. Wang, 22/09/14 L3-12

Solution for 1
1--D Functions with Multiple Roots
Iterative Solutions for Multiple Roots
function [[Roots]] = MultiRoots ((Fun,, Xmin,, Xmax))
i = 1;
Inc = 0.1;
A = Xmin;
B = A + Inc;
while B <= Xmax
if Fun(A)*Fun(B) < 0
% Check for a sign change
R t (i :)) = fzero
Roots(i,
f
(F
(Fun,
[A
[A, B])
B]); % Fi
Find
d the
th ith roott
i = i + 1;
end
A = B;
B = A + Inc;
end

ENGG407 - Numerical Methods in Engineering, Dr. Y. Wang, 22/09/14 L3-13

Application of MultiRoots
>> Fun
u = @ ((x)) cos(
cos(x)) * cos
cosh(x)
( ) + 1;;
>> Xmin = 0; Xmax = 5;
>> Roots = MultiRoots (Fun, Xmin, Xmax)
Roots =
1.8751
4.6841
>> Fun = @ (x) cos(x) * cosh(x) + 1;
>> Xmin = -8; Xmax = 8;
>> Roots = MultiRoots (Fun, Xmin, Xmax)
Roots =
-7.8548
-4.6941
-1.8751
1.8751
4.6941
7.8548

ENGG407 - Numerical Methods in Engineering, Dr. Y. Wang, 22/09/14 L3-14

3. The Regula Falsi Method

1. Introduction
2. The bisection method
3.
3 The
Th regula
l falsi
f l i method
th d
4. Newtons method
5. The secant method
6. The Newton-Raphson method for SNLEs
7. MATLAB applications in NLE analyses
8 Summary
8.

ENGG407 - Numerical Methods in Engineering, Dr. Y. Wang, 22/09/14 L3-15

Conceptual Model of the Regula Falsi Method

The
Th regula
l falsi
f l i method
th d is
i a bracketing
b k ti
approximation
i ti
for iteratively finding the root for a function f(x)=0 in
[a, b] determined by the straight line between (a, f(a))
and (b, f(b)).
- Also known as the false position and linear interpolation method

Existence
E istence of root
f(x) is continuous in [a, b] f(a)f(b) < 0 x0, f(x0) = 0
Convergence
- Always exists
- Converges relative faster
than the bisection method
method.

Stop condition
= ((b - a)) / 2 < tolerance
or f(x(i)) < tolerance

Mathematical Model of the Regula Falsi Method

Given f ( x) 0 in [a, b],
The secant through a and b is determined by:
f ( x) f (b) f (b) f (a)

x b
ba
The ith numerical solution is given when f (xi ) 0, i.e.:
xi

ai f (bi ) bi f (ai )
f (bi ) f (ai )

Then, updating: ai1 xi , bi1 bi

or:

The simple R.F. Method

if f (ai ) * f (bi ) 0 then bi1 xi The improved R.F. Method

else ai1 xi
ENGG407 - Numerical Methods in Engineering, Dr. Y. Wang, 22/09/14 L3-17

MATLAB: The Regula Falsi Method (1/2)

MATLAB: The Regula Falsi Method (2/2)

ai f (bi ) bi f (ai )
xi
f (bi ) f (ai )

Cf : The simple R.F.

Cf.:
R F Method:
a = Xs;
Fa = Fxs;

Application of RegulaRoot in Centroid Finding

>> Fun3 = @ (x) 2 * sin(x)^3 / (3 * (x - sin(x) * cos(x))) - 3/4;

>> Xs = RegulaRoot (Fun3, 0.1, 1.4, 0.00001)
Xs =
0 9 11
0.95511
ENGG407 - Numerical Methods in Engineering, Dr. Y. Wang, 22/09/14 L3-20

4. Newtons Method

1. Introduction
2. The bisection method
3 The regula falsi method
3.
4. Newtons method
5. The secant method
6. The Newton-Raphson method for SNLEs
7. MATLAB applications in NLE analyses
8 Summary
8.

ENGG407 - Numerical Methods in Engineering, Dr. Y. Wang, 22/09/14 L3-21

Conceptual Model of Newtons Method

The Newton
Newtons
s method is an open iterative approximation
for the root of a function f(x)=0 in [a, b] determined by its
derivative f(x) at the point x0.
Condition of root existence
f(x) is continuous f(x) 0 near x0 where f(x0) = 0.
Convergence
C
- Not always exists
- Converges faster
but may diverge.
diverge

Stop condition

xi 1 xi
i) Relative error:

xi
ii) Tolerance: f ( xi )
ENGG407 - Numerical Methods in Engineering, Dr. Y. Wang, 22/09/14 L3-22

Diverge Cases of Newtons Method

Mathematical Model of Newtons Method

G iv
i en f ( x ) 0 near xi ,
f ( xi ) 0
f '(( x i )
, x x i 1 x i 0
x i x i 1
o b ta in : x i 1

f ( xi )
xi
,
f '( x i )

x i 1 x i

xi

ENGG407 - Numerical Methods in Engineering, Dr. Y. Wang, 22/09/14 L3-24

E.g.1. Newtons Method for Solving a Function

ENGG407 - Numerical Methods in Engineering, Dr. Y. Wang, 22/09/14 L3-25

MATLAB: Newtons Method

f ( xi )
xi 1 xi
f '( xi )
x

i1

Applications of NewtonRoot

ENGG407 - Numerical Methods in Engineering, Dr. Y. Wang, 22/09/14 L3-27

5. The Secant Method

1. Introduction
2. The bisection method
3 The regula falsi method
3.
4. Newtons method
5. The secant method
6. The Newton-Raphson method for SNLEs
7. MATLAB applications in NLE analyses
8 Summary
8.

ENGG407 - Numerical Methods in Engineering, Dr. Y. Wang, 22/09/14 L3-28

Conceptual Model of the Secant Method

The Secant method is an open iterative approximation for
the root of a function f(x)=0 determined by a secant
between two points (x1, f(x1)) and (x2, f(x2)).
Condition of root existence
f(x) as continuous near x0 where f(x0) = 0.
Convergence
- Converges faster
but may diverge.

Stop conditions

xi1 xi
i) Relative
R l ti error:

xi
ii)) Tolerance: f (xi )
ENGG407 - Numerical Methods in Engineering, Dr. Y. Wang, 22/09/14 L3-29

Mathematical Model of the Secant Method

G iv e n f ( x ) 0 a t x i a d ja c e n t to x i 1 a n d x i 1 in b o th s id e s ,
w h e re

f ( x i 1 ) f ( x i )
f ( xi ) 0

,
x i 1 x i
x i x i 1

o b ta in : x i 1 x i

f ( x i ) ( x i 1 x i )
,
f ( x i 1 ) f ( x i )

x i 1 x i

xi

f ( x i 1 ) f ( x i )
R e la tio n to N e w to n ' s M e th o d : f '( x i ) lim
.
x 0
x i 1 x i

Advantage: Do not need

to calculate f(xi) as that
in Newtons method.

MATLAB: The Secant Method

f ( xi )( xi 1 xi )
xi 1 xi
f ( xi 1 ) f ( xi )
x i1 x i

xi

Applications of SecantRoot

ENGG407 - Numerical Methods in Engineering, Dr. Y. Wang, 22/09/14 L3-32

6. The Newton
Newton--Raphson Method for
Systems of Nonlinear Equations

1. Introduction
2. The bisection method
3 The
3.
Th regula
l falsi
f l i method
th d
4. Newtons method
5. The secant method
6. The Newton-Raphson method for SNLEs
7. MATLAB applications in NLE analyses
8 Summary
8.

ENGG407 - Numerical Methods in Engineering, Dr. Y. Wang, 22/09/14 L3-33

Math Model of NewtonNewton-Raphson Method for SNLEs

f1 ( x1 , x 2 ,..., x n ) 0
f1 ( x )

f (x )
f 2 ( x1 , x 2 ,..., x n ) 0

Given an SNLE:
, let x =[ x1 x 2 ... x n ]T , F = 2
...
mn
...

f m ( x1 , x 2 ,..., x n ) 0
x
f
(
)
m

Newton's method can be extended to n -D in the vector form , i.e.:

x i 1

i 1
i
x
x

x i (F i )T / J T x i J \ Fi ,

i
x

f1
x
1
f 2
F
x1
where J
x
...
f
m
x1

f1
x2

...

f 2
x2

...

...
f m
x2

...
...

f1
xn

f 2
xn

...
f m

x n

1-D
D Newton'
Newton s method
cf . 1

f ( xi )

x i 1 x i
f '(( x i )

MATLAB:: The NewtonMATLAB

Newton-Raphson Method for a 22-D SNLE
function [x,
[x y] = NewtonRaphson (xi,
(xi yi)
f1 = @(x1, x2) x2 - 0.5 * (exp(x1/2)+exp(-x1/2));
f2 = @(x1, x2) 9*x1^2 + 25*x2^2 - 225;
X = [xi; yi];
J = [-0.25*(exp(X(1)/2) + exp(-X(1)/2)), 1; 18*X(1), 50*X(2)];
% [Jf1x1,
Jf1x2; Jf2x1, Jf2x2]
for i = 1 : 10
i 1
i
f1i = f1(X(1), X(2));
f2i = f2(X(1), X(2));
F = [f1i; f2i];
X = X J \ F;
end
x = X(1);
y = X(2);

x J\F

>> [x, y] = NewtonRaphson (3, 2)

x = 3.0312, y = 2.3859

ENGG407 - Numerical Methods in Engineering, Dr. Y. Wang, 22/09/14 L3-35

A Case Study in Velocity Detection (1/2)

Displacement: d |p0 ( x0 , y0 , z0 ) p( x, y, z ) |

x0 x y0 y z0 z
2

A simplified
i lifi d case for
f
Lab 2.

Velocity: v
d d d
x
y
z
i.e. : v Jd
where
h the
h jacobian
j bi matrix
i iis J = [J x , J y , J z ]

J x d x0 x 2 y0 y 2 z0 z 2

x
x

y0 y
J

y
2
2
2
y
x

0 0 0

z0 z
J d
2
2
2
z z
x
x

0 0 0

x0 x

x0 x y0 y z0 z
2

ENGG407 - Numerical Methods in Engineering, Dr. Y. Wang, 22/09/14 L3-36

A Case Study in Velocity Detection (2/2)

ffunction
ti [] = VelocityTest
V l it T t ( )
p1 = [10 5 0];
p2 = [20 10 0];
J = zeros (1, 3);
D = sqrt ((p2(1)-p1(1))^2 + (p2(2)-p1(2))^2 + (p2(3)-p1(3))^2)
J(1, 1) = (p2(1)
(p2(1)-p1(1))
p1(1)) / D;
J(1, 2) = (p2(2)-p1(2)) / D;
J(1, 3) = (p2(3)-p1(3)) / D;
v = J*D
>> VelocityTest
D=
11.1803
v=
10

0
ENGG407 - Numerical Methods in Engineering, Dr. Y. Wang, 22/09/14 L3-37

A Special Solution for 22-D SNLEs by Newton

Newton--Raphson
f (x, y) 0
The textbook solution
G iv e n a n S N L E s : 1
- A special case of the
f2 ( x, y ) 0
generic solution
i1
i
x
x x
- Base on Cramers
T h e i 1t h s o lu t io n s a r e : i 1
rule (Eq.2.44)
(Eq 2 44)
yi y
y
w h e re
f 2 ( x1 , y1 )
f1 ( x1 , y1 )
f1 ( x1 , y1 )
f 2 ( x1 , y1 )
y
y
x
J ( f1 ( x1 , y1 ) , f 2 ( x1 , y1 ) )
y

f 2 ( x1 , y1 )

f1 ( x1 , y1 )
f 2 ( x1 , y1 )
f1 ( x1 , y1 )
x
x
J ( f1 ( x1 , y1 ) , f 2 ( x1 , y1 ) )

a n d th e J a c o b ia n : J ( f1 ( x1 , y1 ) , f 2 ( x1 , y1 ) ) d e t

f1
x
f2
x

ENGG407 - Numerical Methods in Engineering, Dr. Y. Wang, 22/09/14 L3-38

f1
y
f2
y

E.g.2 Implementation of the Special Case for 22-D SNLEs

xi 1 xi x
i 1
i
y

y
y

x = 3.0312, y = 2.3859
ENGG407 - Numerical Methods in Engineering, Dr. Y. Wang, 22/09/14 L3-39

7. MATLAB Applications in NLE Analyses

1. Introduction
2. The bisection method
3 The regula falsi method
3.
4. Newtons method
5. The secant method
6. The Newton-Raphson method for SNLEs
7. MATLAB applications in NLE analyses
8 Summary
8.

ENGG407 - Numerical Methods in Engineering, Dr. Y. Wang, 22/09/14 L3-40

7.1 MATLAB BuiltBuilt-in Functions

Function fzero single root
fzero solves a continuous function of one variable f(x) = 0
when an approximate x0 is known.
Function roots multiple roots of polynomial
roots solves a polynomial,
polynomial ii.e.,
e

for its multiple roots.

ENGG407 - Numerical Methods in Engineering, Dr. Y. Wang, 22/09/14 L3-41

The MATLAB BuiltBuilt-in Function fzero

Usage
fzero solves a continuous function of one variable f(x) = 0 when
an approximate x0 is known
known.

Syntax

Algorithm
A combination of bisection, secant, and inverse quadratic
interpolation methods.
Descriptions of the function
- An anonymous function
- A user-defined function
- A string of math expression

Application of Function fzero

ENGG407 - Numerical Methods in Engineering, Dr. Y. Wang, 22/09/14 L3-43

The MATLAB BuiltBuilt-in Function roots

Usage
roots
oots so
solve
e a po
polynomial
y o a for
o its
ts multiple
u t p e roots.
oots

Syntax

r = [rn rn-1 r1]T,

p = [an an-1 a1 a0]

ENGG407 - Numerical Methods in Engineering, Dr. Y. Wang, 22/09/14 L3-44

Applications of Function roots (1/2)

ENGG407 - Numerical Methods in Engineering, Dr. Y. Wang, 22/09/14 L3-45

Applications of Function roots (2/2)

>> Polyn = @ (x) x^4 5.5*x^3 7.2*x^2 + 43*x + 36;
>> p = [1 -5.5 -7.2 43 36];
>> RootsFound = roots (p)
RootsFound =
4.8242
4
8242
3.8736
-0.8052
-2.3926

ENGG407 - Numerical Methods in Engineering, Dr. Y. Wang, 22/09/14 L3-46

7.2 MATLAB Applications in Solving NLE & SNLE

>> Fun27 = @ (R) C1 + C2*log(R) +

C3*log(R).^3
g( )
1/(273.15+T);
(
)
>> C1 = 1.1e-3; C2 = 2.3e-4;
C3 = 0.82e-7; T = 50;
>> fplot (Fun27, [1000, 10000])
>> R = fzero (@Fun27, 4000)
R=
4 7047e+003
4.7047e+003

Resister Based Thermistors

Resister-Based

A Case Study on Max. Deflection of an II-Beam

>> Fun31 = @ (x) wo*(6*L^3*x 21*

L^2*x^2 + 20*L*x^3 5*x^4) / (120*L*E*I);
>> wo=15000; L=3; E=70e9; I =52.9e-6;
>> fplot (Fun31, [0, 3])
>> Xmax = fzero (@Fun31, 1.5)
Xmax =
1.4259

A Case Study on the Vibration Frequency of an RLC Circuit

>> Fun34 = @ (f) vm / sqrt(R^2 +

((2*pi*f*L
p
1 / ((2*pi*f*C))^2)
p
)) ) im;
>> R=140; L=260e-3; C=25e-6;
vm=24; im=0.15;
>> fplot (Fun34, [20. 60])
>> fo = fzero (@Fun34, 40)
fo =
43 0681
43.0681

A Case Study on the Vibration Frequency of Car Suspensions

>> Fun38 = @ (w) m*c*w^3 +

XY^2*w^2*(k*m
XY
2 w 2 (k m c
c^2)
2) XY
XY^2*k^2;
2 k 2;
>> m=2000; k=500e3; c=38000;
XY=0.2;
>> fplot (Fun38,
(Fun38 [0,
[0 10])
>> fwo = fzero (@Fun38, 4)
fwo =
5 1653
5.1653

8. Summary

1. Introduction
2. The bisection method
3 The regula falsi method
3.
4. Newtons method
5. The secant method
6 The Newton-Raphson
6.
Ne ton Raphson method for SNLEs
7. MATLAB applications in NLE analyses
8. Summaryy

ENGG407 - Numerical Methods in Engineering, Dr. Y. Wang, 22/09/14 L3-51

Summary of Lecture 3
Lecture 3. NMs for Nonlinear Equations and Systems
Approaches to solve NLEs/SNLEs
- Bracketing methods
- Open methods
- MATLAB functions

The secant method

- Conceptual model
- Mathematical model / Algorithm
- MATLAB implementations

The bisection method

- Conceptual model
- Mathematical model / Algorithm
- MATLAB implementations

The Newton-Raphson method for SNLEs

- Conceptual model
- Mathematical model / Algorithm
- MATLAB implementations

The regula falsi method

- Conceptual model
- Mathematical model / Algorithm
- MATLAB implementations

MATLAB applications in NLE&S

- MATLAB built-in functions
- fzero
- roots
- Engineering applications
- Iterative
te at e so
solutions
ut o s for
o multiple
u t p e roots
oots
- A thermistor
- An I-Beam
- An RLC circuit
- A car suspension system

Newtons method
- Conceptual model
- Mathematical model / Algorithm
- MATLAB implementations

ENGG407 - Numerical Methods in Engineering, Dr. Y. Wang, 22/09/14 L3-52

Further Reading

Numerical Methods for Engineers and Scientists

- Chapter 3 Solving Nonlinear Equations

ENGG407 - Numerical Methods in Engineering, Dr. Y. Wang, 22/09/14 L3-53