KING FAISAL UNIVERSITY

COLLEGE OF ENGINEERING

Engr310 – Numerical Methods for Engineers

Lecture 03: Roots of Equations – Open Methods

initial 2. ⇐
↳
& on
Ig iteration bbl s .

Majdi Adel
Value
converge
↓
or
diverge
faster than bracketing method
 Open Methods
They are based on formulas that require only a single starting value or two
starting values that do not necessarily bracket a root
9, #I

They may diverge as the computation *✗ a ÷

progresses, but when they do converge,

they usually do so much faster than
bracketing methods

 Fixed point iteration

 Newton-Raphson Method
 Secant Method

Engr310 – Numerical Methods for Engineers Majdi Adel 2

 Simple Fixed
Point
Iteration
Engr310 – Numerical Methods for Engineers Majdi Adel 3
 Hit @ I # *①
" "
•

✗ = . . . .

gcx ) @
bias # I @I
f- ( X)
i.
. sat
Eg : -

=
✗ 2- ✗ -2=0
Iginla
Fixed Point Iteration ↳*
%b1gxos.dz ,
121216*16*-1
-;
19
or
⇐ ✗ = ✗ 2- 2
É+2 → 9# =

f-(✗ 1=0 → 9. CX ) = ✗ or
¥ -2¥?0 931*1--1 -1¥
→

Rearrange the function f(x) = 0 so that x is on the left-hand side of the

equation to get a new function called g(x)

The new function g(x) is used to predict a new value of x. that is, xi+1 = g(xi)

[email protected]
④
The relative approximate error is given by:
𝑥 −𝑥
𝜀 = × 100%
𝑥
,Ea%iEt%
Et=×¥ 1×100%0 INN #
11
true
value
Engr310 – Numerical Methods for Engineers Majdi Adel 4
 Fixed Point Iteration Steps : .

.
gcxiojs A few W ①
of ☆1 It ⇐
"

f- ✗=
"

ébogcxi .
. _ . .

convergence / gixo) / ( I
• • 0.51-b≥ "

1€
"
,
89T Cow .
.
⇐

:& ggcx.io & ✗ is i ⇐ ◦

I
.

}I

:& JI w iterations JI T.nl ↳ Is ⑨

• 3 iterations .W § ⇐
Or Le 2 i 3 ⇐
.
: 0 A ⑨
gcx) & Agt - Xi j Sig Xo I ⇐
. ✗i + I V. ↳
. : I
& 'd gl . Xi 1 a ✗ ii. I ⇐

.ci?.?JlXi-i?W9cxj.%Ea.-1iibs A & ④ .

.
Xi +1 µ ✗i & £
657 I ✗
it , ⇐
 Example True root __
0.56714329

Use simple fixed point iteration to locate the root of the below function with an initial guess of x0 = 0.
Use 10 iterations ①
◦ → 10 𝑓 𝑥 =𝑒 −7-
𝑥 ◦
.gl/YJljs.&.-J
✗i # ¥81 % @ I •
Js # £5
>
⊖
e-
91×1 :X i
.

x.WWII:@ iwslg.AM?ijVoQ.Wl
=

Step 1: Rewrite the function as xi+1 = g(xi)

We can rewrite the function in several ways as we will see later. For this example we will use
this form
𝑥 =𝑒

Step 2: Start with the initial guess x0 = 0

Step 3: Continue until the specified tolerance εs or the number of iterations is reached

Engr310 – Numerical Methods for Engineers Majdi Adel 5

 ① ✗◦ ◦

✗ 0+1 = e- → × ,
= e- =L Ea and Et from
the laws .

②
¥
'
'

,=é✗2 ✗}= e-
e- ✗ 2- e- 0387870
✗
µ,
= _

= 0.367879 ③ ✗
z,
#
A 2911
iwslg ①&
↳ ✗it

'
d

" Note that each iteration brings the estimate xi

closer to the true value of the root: 0.56714329

In this case the solution is converging

↓
✗ i
94×1 = -
e-
◦

= -
e- = -121

converging

i-iwbgljljl.blgEs.i.be.gisdolsw.IT ' ' • *

Engr310 – Numerical Methods for Engineers Majdi Adel 6

 Convergence of Fixed Point Iteration
In the previous example the true relative error εt for each iteration is roughly
proportional (by a factor of about 0.5 to 0.6) to the error from the previous iteration
This property is called linear convergence and it is characteristic of fixed point iteration

But, dose fixed point iteration always converge?

Fixed-point methods may sometime “diverge”, the convergence depends on the
starting point (initial guess) and how the function behaves g(x)

Take for example the following function can be

written in different forms: 𝑔 𝑥 =𝑥 =𝑥 −2

𝑓 𝑥 =𝑥 −𝑥−2 𝑔 𝑥 =𝑥 = 𝑥−2
2
𝑔 𝑥 =𝑥 =1+
𝑥
Engr310 – Numerical Methods for Engineers Majdi Adel 7
 fixed-point iteration converges if, in the region of interest
if the magnitude of the slope of g(x) is less than the slope
of the line f(x) = x

Fixed-point iteration converges if

𝑔′ 𝑥 <1

+
NAI I'M .
0£09 # A#

iiEE
I

µ win
mix * * a ' ~ pig
& Xi
.

# .MX IN
g. j§ is
# I ✗ in wk is
w-t.WIII.ws .
True roof
Engr310 – Numerical Methods for Engineers Majdi Adel 8
 Convergence Divergence

Engr310 – Numerical Methods for Engineers Majdi Adel 9

Find the roots of fcx , = ✗ - tan 1×7 ? ✗◦ =

→ Ig soot , MAN
I -
91×1 iys A Howl :b ①

.
:b -61g MAJHI
Note : -
X -
tank ) -0

JI , >
* lit ✗
it ,
= tancxi )
tan (✗ i )
'
:# HIM :& ✗ in
-

jl.HN .
: Jg Epi ②
① ✗ it , = tan ( X ;) : -

ji { t.IO
true value ibso

4W .
Ea ji ↳ &

Ning Ñ
'

→ divergence

② ✗ it , = tan-1 ( × ;) : -

i• Eaji ↳ * &

→ convergence
Find the roots of fcx ,
"
= ✗ + ✗ -1 ? ✗◦ = 0

→ Ig shot , MAN ÷
> 1 .
91×1 iys A ↳I :b ①

.
:b -61g TAJ §) I
✗2-1×-1=0

✗ i -11 = ¥+1 ,
I
✗ it
,
=

: Jg # 1 Igi ②

¥5
-

① ✗ in = : -

ji { t.IO
true value ios 6

i• Eaji ↳ * &

→ convergence

② ✗ it , = TEXT : -

4W .
Ea ji ↳ &

Ning Ñ
'

→ divergence
 Newton-
Raphson
Method
is 1 ;
Is ↳ Is • WI Gil Lt is 1 if big
Mimi 6
.M
•

Engr310 – Numerical Methods for Engineers Majdi Adel 10

 Newton-Raphson Method
It is based on forming a tangent line to the
function f(x) curve at some guess x, then
following the tangent line to where it crosses
the x-axis
The formula is derived from first order Taylor series
expansion
𝑓 𝑥 =𝑓 𝑥 +𝑓 𝑥 𝑥 −𝑥 =0
Solving for xi+1 4-* 8N .

𝑓 𝑥
𝑥 =𝑥 − Newton-Raphson formula
𝑓 𝑥

Engr310 – Numerical Methods for Engineers Majdi Adel 11

 Newton -

Raph son formula steps : -

-
W ffxjo.gl rfcx, g. A Wijk .
①

:& JI w iterations JI I µ I ↳ Is ⑨
• 3 iterations .W § ⇐
Or Le 2 - 3 ⇐

- : 11 & if Is ⑨
.

i
/ / Xi
/
fcxi ) fixis / Ea
/ Et

↓ I
↳ in true sews so &
root
At
↳ éw ! Igloo WWI ✗◦ wig W' Is Xi ②
• ✗i-11 4 ↳
✗ i -110.6J is # § I 04 # Xia b & ⇐
. Ea 7 I IN @ 9 It
 Example
Solve the previous example using Newton-Raphson method with the same initial guess
𝑓 𝑥 =𝑒 −𝑥
jfcxi.at oh ↳ % It f- .

ji-wmjs.gs
Step 1: Find the derivative f’(x) [
𝑓= −𝑒 − 1 Ai 81
𝑥 =𝑥 −
𝑒 −𝑥
MA ' -

1-I
−𝑒 − 1
Step 2: Start with the initial guess x0 = 0

Step 3: Continue until the specified tolerance εs

a- As G- b w -7
or the number of iterations is reached
jowl & d- I I & 0.5 & &
o
j

Engr310 – Numerical Methods for Engineers Majdi Adel 12

 Convergence of Newton-Raphson Method
In the previous example the true relative error εt is roughly proportional to the square
of the previous error
Such behavior is referred to as quadratic convergence

However, Newton-Raphson has its limitations

Not convenient for functions whose derivatives cannot be evaluated analytically

Some functions show slow or poor convergence (functions with local minima/maxima,
asymptotes)

Overshooting with some functions

Engr310 – Numerical Methods for Engineers Majdi Adel 13

 Inflection point in the vicinity of a root leads to divergence Local maximum or minimum leads to oscillation then
divergence

Near-zero slopes leads to jump to a location several roots Zero slopes are disaster because it causes division by zero
away in the Newton-Raphson formula

Engr310 – Numerical Methods for Engineers Majdi Adel 14

Example :.

fcx )=✗2+X -
I ✗ 0=0 , Es=0 -01%

i.ly/.-j ①
,
FIX )= 2×+1
i. Earfcxi ) fali ) ( -
Xiii Us .
# • Is # £ 'M ⑨

i ✗i fcxi ) fail %Ea

0 0.000 -1.000 1.000 -

I 1. ooo 1.000 3.000 / ÷ / ✗ too =

100%
2 0.667 0.111 2.333 50%

1=0 : .

H1N1 I. ↳↳
gbq.X.im?-w.8--&!xi-.Ws
Wtem & @ INK
i=1 : -

- :
Xiii -6.21 jus -61 Ciws
✗ it =
Xi -
i ✗
,=✗o -
te
,
f 'CXi ) F'( ✗ ol
 = 0 -
= 0 + I = 1

I =3
j-w.si r I -2
-

d-

Convergence
:X true = I f. 6 His ④ WI isl b

↳ is 1

Convergence Et W
 The Secant
Method
8 > WI
#I
bing.b.ci
& @
-6%1

Engr310 – Numerical Methods for Engineers Majdi Adel 15

 FW '%j!
✗ is ✗ in -1
Secant Method
Used for functions whose derivatives are
difficult to evaluate. The derivative can be
approximated by a backward finite divided
difference
𝑓 𝑥 −𝑓 𝑥
𝑓 𝑥 =
𝑥 −𝑥
Substituting in Newton-Raphson formula
𝑓 𝑥 𝑥 −𝑥 ☆
𝑥 =𝑥 −
𝑓 𝑥 −𝑓 𝑥

This method requires two initial guesses xi–1 and xi, that are not necessarily
bracket the root. Therefore it is NOT a bracketing method
Engr310 – Numerical Methods for Engineers Majdi Adel 16
 The Secant Method Steps : .

.
✗i 9 ✗
i. , f. I kiss £ ①

g) I w iterations JI I µ I W Is "
• 3 iterations .W § ⇐
Or Le 2 i 3 ⇐

i. DWI I D) j
.
G -
J g D.) I ⑨

i
/ ✗ i.,
/ /fcxi.is/fcxii/Xi+iI.Ea /
✗
i Et

d d
do I
him As
root
.

µ true

-
ji Wt & ✗
i., I ✗it I & .
Xiao .
↳ ②
& ✗i ✗
g E. in 9 ✗ in AIX i 69 ⇐
E N D.) I ✗ i ↳ JIMI I
<
. . +, v.
 In case Es = 0 -10%
Example
Solve the previous example using secant method. Start with initial estimates of x−1 = 0 and x0 = 1.0
𝑓 𝑥 =𝑒 −𝑥
𝑒 −𝑥 𝑥 −𝑥
𝑥 =𝑥 −
𝑒 −𝑥 − 𝑒 −𝑥
o # MET a

i xi – 1 xi xi + 1 εt εa
1 giv@
0.00 1.000 O
0.613 8.0326% 63.212%
2 1.000
-0.613 ro
0.564 0.5827% 8.666%

-01
-
3 0.613 0.564 0.567 0.0048% 0.587%

jéÑwr .

Engr310 – Numerical Methods for Engineers Majdi Adel 17

 Example : -

f- (X ) = ✗ 21-11 -
I ✗ i. 1=0
✗ i = 1

Es = o -
o I %

i. Ear fcxij-fcxi.gr ✗ i
•
✗ it % jb.IS?Jt
go ✗ it 9 WI Xi -
II. &V1 -

ji Wtop I 5141
 Multiple
Roots

Engr310 – Numerical Methods for Engineers Majdi Adel 18

 Multiple Roots
A multiple root corresponds to a point where a function is
tangent to the x-axis
For example the function
𝑓 𝑥 = 𝑥 − 5𝑥 + 7𝑥 − 3
Can be decomposed as:
𝑓 𝑥 = 𝑥−3 𝑥−1 𝑥−1
It has double roots at x = 1. Graphically, this
corresponds to the curve touching the x axis
tangentially at this point

𝑓 𝑥 = 𝑥−3 𝑥−1 𝑥−1 𝑥−1

𝑓 𝑥 = 𝑥−3 𝑥−1 𝑥−1 𝑥−1 𝑥−1
Engr310 – Numerical Methods for Engineers Majdi Adel 19
 Difficulties with Multiple Roots
 The function does not change sign at multiple roots,
therefore we cannot use the reliable bracketing methods
 Not only f(x) but also f‘(x) goes to zero at the root. This poses
problems for both the Newton-Raphson and secant methods
(division by zero when the solution converges)
 Newton-Raphson and secant methods are linearly, rather
than quadratically, convergent for multiple roots

Engr310 – Numerical Methods for Engineers Majdi Adel 20

 Modified Newton-Raphson Method → faster than
the standard
Define a new function u(x) that is the ratio of The derivative of u(x):
the function to its derivative: 𝑓 𝑥 −𝑓 𝑥 𝑓 𝑥
𝑓 𝑥 𝑢 𝑥 =
𝑢 𝑥 = 𝑓 𝑥
𝑓 𝑥

This function has roots at all the same Substituting and rearranging:
locations as the original function f(x).
Therefore we can apply Newton-Raphson ☆ 𝑥 =𝑥 −
𝑓 𝑥 𝑓 𝑥
method to find its roots: 𝑓 𝑥 −𝑓 𝑥 𝑓 𝑥
𝑢 𝑥
𝑥 =𝑥 − Modified Newton-Raphson formula
𝑢 𝑥
idea ¥ .
Jg # if, jiwlii 4
f- Ex, D ↳ sat
"
Newton -

Raph
"
son

Engr310 – Numerical Methods for Engineers Majdi Adel 21

 Modified Newton Raph son Method steps . : .

- W fix, i fix) ? 91 ifeng.cm Wijk .

①

:& JI w iterations I T.nl ↳ Is ⑨

• 3 iterations .W § ⇐
Or Le 2 e 3 ⇐

- : Mt I & i § Is ⑨

i
/ /
✗i fail / /
fix it fix it / /
Ea Et

d I
aim true sins so &
root
At
↳ éw ! Igloo WWI ✗◦ wig WI > Xi ②
• ✗i-11 ? ↳

Xiao ↳ is #
.
§ I 04 # Xia b & ⇐
. Ea 7 I IN @ 9 Jj I
 In case : -

Example Es=0 -0170 ✗ true :O -01%

Use both the standard and modified Newton-Raphson methods to evaluate the multiple root of
function below, with an initial guess of x0 = 0 ex ) -3×2-10×+7
f' -

𝑓 𝑥 = 𝑥 − 5𝑥 + 7𝑥 − 3 f "( ✗ 1=6×-10

Convergence

Engr310 – Numerical Methods for Engineers Majdi Adel 22

 Standard Newton-Raphson methods
𝑥 − 5𝑥 + 7𝑥 − 3
𝑥 =𝑥 −
3𝑥 − 10𝑥 + 7

Modified Newton-Raphson methods

𝑥 − 5𝑥 + 7𝑥 − 3
3𝑥 − 10𝑥 + 7
𝑥 =𝑥 −
3𝑥 − 10𝑥 + 7 −
𝑥 − 5𝑥 + 7𝑥 − 3 6𝑥 − 10

Engr310 – Numerical Methods for Engineers Majdi Adel 23

 We can also use both methods to search for the single root at x = 3. Using an initial guess of x0 = 4

Engr310 – Numerical Methods for Engineers Majdi Adel 24

 Systems of
Nonlinear
Equations
Engr310 – Numerical Methods for Engineers Majdi Adel 25
 Systems of Nonlinear Equations [email protected]
> ↳ b. j -81

In this part we will see how to find the roots of a set of simultaneous
nonlinear equations in the form:
𝑓 𝑥 ,𝑥 ,…,𝑛 = 0
𝑓 𝑥 ,𝑥 ,…,𝑛 = 0
.
The solution of this system consists of a set of x
. values that simultaneously result in all the
. equations equaling ZERO
𝑓 𝑥 ,𝑥 ,…,𝑛 = 0 e.
g : -

system of nonlinear equ .

We will discuss two methods UCX , y ) = ✗ 2-1 ✗

y -10=0
 Fixed point iteration To find VCXIY ) =
4+3×42-57=0
the root
 Newton-Raphson method

Engr310 – Numerical Methods for Engineers Majdi Adel 26

 Fixed Point Iteration
The fixed point iteration approach that was discussed before can be modified
to solve two simultaneous, nonlinear equations. This approach will be
illustrated in the following example

I ↳I
:@ ↳ -
>
-

✗ = →
gcx,
£-181
- - . .

↳ gig
Y= . . .
→
gcyj

Two initial § 'S

ok
:# It
_

Values
. -

:-c ¥ -

1%1+1%1<1
1%-1+1%-1<1
Engr310 – Numerical Methods for Engineers Majdi Adel 27
 Example
+ Newton Raphson
method
Use fixed-point iteration(
to determine the roots of the following nonlinear set of equations. Initiate
the computation with guesses of x = 1.5 and y = 3.5
Note that a correct pair of roots is x = 2 and y = 3 Es = 0.10%

𝑢 𝑥, 𝑦 = 𝑥 + 𝑥𝑦 − 10 = 0
𝑣 𝑥, 𝑦 = 𝑦 + 3𝑥𝑦 − 57 = 0

Engr310 – Numerical Methods for Engineers Majdi Adel 28

* Fixed point iteration : .

. .
-
8 >↳ I 7 I ①

① U : -

equ ① : - ✗y = 10 -
✗ ✗
µ,
=

egu ② : . ✗2=10 - ✗
y ✗ it
,
= FÉ
② V : -

2
57 3 Xi Yi
equ ① Y
: - = -

i + I

equ ② : .
3×72=57 -

y → %, =
-53T¥
E. a # at if I
g ski:D > b N bl It F-
Cow @ .

8,41 # I 631 g # is I do gi
M im
'

For ✗ =
and 1=57-3 Xi Yi ? -

µ, i + I

HI & b- I j£ Hb NJ Ea ul 11 If I
 divergence
WE
"

✗ s Yi s Xi I - ✗ in on
.
# ;D I v.

HH g Xi -11 WI C- I I Yi+.-"
Yi Éñ

For ✗
ix. = FEI and Yin : -

convergence
* Newton : .

.
: 6--681 Nest ①

[email protected]
'

:& -

,
ZU / 211,24/04,24211 ,2V / 04 ,Ex , Ey

xi.si?i!H#.. j-bv.&A1w- I iii. At

 We need to rewrite the equations to have one for xi+1 and one for yi+1
There are many ways to do this. We must be careful in formulating the equations to have
convergence

𝑢 𝑥, 𝑦 = 𝑥 + 𝑥𝑦 − 10 = 0 → 𝑥 = 10 − 𝑥 𝑦

57 − 𝑦
𝑣 𝑥, 𝑦 = 𝑦 + 3𝑥𝑦 − 57 = 0 → 𝑦 =
3𝑥
Now, we use the first equation to find a new value of xi+1, using the initial guesses. Then we calculate
the value of yi+1 from the second equation using the new xi+1

Continue with the iterations until the relative approximate error in both of xi+1 and yi+1 reaches the
specified stopping criteria

Engr310 – Numerical Methods for Engineers Majdi Adel 29

 i xi+1 εa,x yi+1 εa,y
𝑥 = 10 − 𝑥 𝑦 0 1.5000 -- 3.500 --
57 − 𝑦
1 2.500 40.000% 2.696 7.259% 𝑦 =
2 1.806 38.445% 3.166 14.858%
3𝑥
3 2.069 12.743% 2.945 7.520%
𝑥 −𝑥 4 1.976 4.711% 3.019 2.476% 𝑦 −𝑦
𝜀 , = × 100% 𝜀 , = × 100%
𝑥 5 2.008 1.580% 2.993 0.869% 𝑦
6 1.997 0.544% 3.002 0.295%

Engr310 – Numerical Methods for Engineers Majdi Adel 30

 The convergence of fixed point iteration depends on:
 How the equations are formulated
 The initial guesses, if they are insufficiently close to the true solution it may lead to divergence

Conditions for convergence for the two-equation case are:

𝜕𝑢 𝜕𝑢
+ <1
𝜕𝑥 𝜕𝑦
𝜕𝑣 𝜕𝑣
+ <1
𝜕𝑥 𝜕𝑦

Engr310 – Numerical Methods for Engineers Majdi Adel 31

 Newton-Raphson Method
For a single equation, Newton-Raphson uses first order Taylor expansion. For
multiequation system we use multivariable Taylor series expansion
For the two-variable case, a first-order Taylor series can be written for each
nonlinear equation as
𝜕𝑢 𝜕𝑢
𝑢 =𝑢 + 𝑥 −𝑥 + 𝑦 −𝑦 =0
𝜕𝑥 𝜕𝑦
𝜕𝑣 𝜕𝑣
𝑣 =𝑣 + 𝑥 −𝑥 + 𝑦 −𝑦 =0
𝜕𝑥 𝜕𝑦

Engr310 – Numerical Methods for Engineers Majdi Adel 32

 Rearranging the equations
𝜕𝑢 𝜕𝑢 𝜕𝑢 𝜕𝑢
𝑥 + 𝑦 = −𝑢 + 𝑥 +𝑦
𝜕𝑥 𝜕𝑦 𝜕𝑥 𝜕𝑦
𝜕𝑣 𝜕𝑣 𝜕𝑣 𝜕𝑣
𝑥 + 𝑦 = −𝑣 + 𝑥 +𝑦
𝜕𝑥 𝜕𝑦 𝜕𝑥 𝜕𝑦

Or in matrix form
𝜕𝑢 𝜕𝑢 𝑥 𝜕𝑢 𝜕𝑢
−𝑢 + 𝑥 +𝑦
𝜕𝑥 𝜕𝑦 𝜕𝑥 𝜕𝑦
=
𝜕𝑣 𝜕𝑣 𝜕𝑣 𝜕𝑣
−𝑣 + 𝑥 +𝑦
𝜕𝑥 𝜕𝑦 𝑦 𝜕𝑥 𝜕𝑦

Jacobian matrix

Engr310 – Numerical Methods for Engineers Majdi Adel 33

 Solving the system using Cramer's rule of determinants
𝜕𝑣 𝜕𝑢
𝑢 −𝑣 7148
𝜕𝑦 𝜕𝑦
𝑥 =𝑥 −
𝜕𝑢 𝜕𝑣 𝜕𝑢 𝜕𝑣 6- €-81
−
𝜕𝑥 𝜕𝑦 𝜕𝑦 𝜕𝑥
At
𝜕𝑢 𝜕𝑣
𝑣 −𝑢
𝑦 =𝑦 − 𝜕𝑥 𝜕𝑥
𝜕𝑢 𝜕𝑣 𝜕𝑢 𝜕𝑣
−
𝜕𝑥 𝜕𝑦 𝜕𝑦 𝜕𝑥

We will solve the previous example in Excel

Engr310 – Numerical Methods for Engineers Majdi Adel 34

 Example
The Manning equation can be written for a rectangular open channel as below. Where:
Q: flow [m3/s] = 5
S: slope [m/m] = 0.0002
B: width [m] = 20 m 𝑆 𝐵𝐻
H: depth [m] 𝑄=
n: the Manning roughness coefficient = 0.03 𝑛 𝐵 + 2𝐻
Solve this equation for H using:
 Fixed point iteration
 Newton–Raphson
 Secant method
Stopping criteria εs = 1.0%

Engr310 – Numerical Methods for Engineers Majdi Adel 35