Numerical Methods

Lecture 06
7/17/2023 Md. Golam Moazzam, Dept. of CSE, JU 1
 Iterative Solution of Linear Equations

❑ Necessity
Direct methods pose some problems when the system grow larger or when most of
the coefficients are zero. They require large number of floating point operations.
Therefore, direct methods not only become time consuming but also severely
affect the accuracy of the solution due to round-off errors.

In such cases, iterative methods may apply.

Some iterative methods:

• Jacobi iteration method
• Gauss-Seidel iteration method
• Successive over relaxation method

7/17/2023 Md. Golam Moazzam, Dept. of CSE, JU 2

 Iterative Solution of Linear Equations

❑ Jacobi Iteration Method

– Jacobi method is one of the simple iterative methods.
– Let us consider a system of n equations in n unknowns.

a11x1 + a12x2 + …… + a1nxn = b1

a21x1 + a22x2 + …… + a2nxn = b2
.
.
.

an1x1 + an2x2 + ……. + annxn = bn

7/17/2023 Md. Golam Moazzam, Dept. of CSE, JU 3

 Iterative Solution of Linear Equations

❑ Jacobi Iteration Method

We rewrite the original system as

b1 − ( a12 x2 + a13 x3 + ..... + a1n xn )
x1 =
a11
b2 − ( a 21 x1 + a 23 x3 + ..... + a 2 n xn )
x2 =
a 22
.
.
.

bn − ( a n1 x1 + a n 2 x2 + ..... + a nn −1 xn −1 )
xn =
a nn

7/17/2023 Md. Golam Moazzam, Dept. of CSE, JU 4

 Iterative Solution of Linear Equations

❑ Jacobi Iteration Method

Now we can compute x1, x2, … , xn by using initial guesses for these values. These
new values are again used to compute the next set of x values. The process can
continue till we obtain a desired level of accuracy in the x values.

❑ Algorithm of Jacobi Iteration Method

7/17/2023 Md. Golam Moazzam, Dept. of CSE, JU 5

 Iterative Solution of Linear Equations

❑ Example: Obtain the solution of the following system using the Jacobi iteration
method:
2x1 + x2 + x3 = 5
3x1 + 5x2 + 2x3 = 15
2x1 + x2 + 4x3 = 8

Solution:
Solve the equations for unknowns on the diagonal. That is
x1 = (5 - x2 - x3)/2
x2 = (15 - 3x1 - 2x3)/5
x3 = (8 - 2x1 - x2)/4

7/17/2023 Md. Golam Moazzam, Dept. of CSE, JU 6

 Iterative Solution of Linear Equations

❑ Example: Obtain the solution of the following system using the Jacobi iteration
method:
2x1 + x2 + x3 = 5 x1 = (5 - x2 - x3)/2
3x1 + 5x2 + 2x3 = 15 x2 = (15 - 3x1 - 2x3)/5
2x1 + x2 + 4x3 = 8 x3 = (8 - 2x1 - x2)/4
Solution:
Assume the initial values of x1, x2 and x3 to be zero (i.e. x1 = x2 = x3 = 0).
After 1st iteration, we have
5
x1(1) = = 2.5
2
15
x 2(1) = =3
5
8
x3(1) = =2
4

7/17/2023 Md. Golam Moazzam, Dept. of CSE, JU 7

 Iterative Solution of Linear Equations

❑ Example: Obtain the solution of the following system using the Jacobi iteration
method:
2x1 + x2 + x3 = 5 x1 = (5 - x2 - x3)/2
3x1 + 5x2 + 2x3 = 15 x2 = (15 - 3x1 - 2x3)/5
2x1 + x2 + 4x3 = 8 x3 = (8 - 2x1 - x2)/4
Solution:
After 2nd iteration, we have
5−3− 2
x1( 2 ) = =0
2
15 − 3  2.5 − 2  2
x2( 2 ) = = 0.7
5
8 − 2  2.5 − 3
x3( 2 ) = =0
4

7/17/2023 Md. Golam Moazzam, Dept. of CSE, JU 8

 Iterative Solution of Linear Equations

❑ Example: Obtain the solution of the following system using the Jacobi iteration
method:
2x1 + x2 + x3 = 5 x1 = (5 - x2 - x3)/2
3x1 + 5x2 + 2x3 = 15 x2 = (15 - 3x1 - 2x3)/5
2x1 + x2 + 4x3 = 8 x3 = (8 - 2x1 - x2)/4
Solution:
After 3rd iteration, we have
5 − 0.7 − 0
x1( 3) = = 2.15
2
15 − 3  0 − 2  0
x2( 3) = =3
5
8 − 2  0 − 0.7
x3( 3) = = 1.825
4

7/17/2023 Md. Golam Moazzam, Dept. of CSE, JU 9

 Iterative Solution of Linear Equations

❑ Example: Obtain the solution of the following system using the Jacobi iteration
method:
2x1 + x2 + x3 = 5 x1 = (5 - x2 - x3)/2
3x1 + 5x2 + 2x3 = 15 x2 = (15 - 3x1 - 2x3)/5
2x1 + x2 + 4x3 = 8 x3 = (8 - 2x1 - x2)/4
Solution:
After 4th iteration, we have
5 − 3 − 1.825
x1( 4 ) = = 0.0875
2
15 − 3  2.15 − 2  1.825
x2( 4 ) = = 0.98
5
8 − 2  2.15 − 3
x3( 4 ) = = 0.175
4

The process can be continued till the values of x reach a desired level of accuracy.
7/17/2023 Md. Golam Moazzam, Dept. of CSE, JU 10
 Iterative Solution of Linear Equations

❑ Gauss-Seidel Method
– Gause seidel metod is an improved version of Jacobi iteration method.
– In Jacobi method, we begin with the initial values
x1(0), x2(0) , …. , xn(0)
and obtain next approximation
x1(1), x2(1) , …… , xn(1)

– In computing x2(1), we used x1(0) and not x1(1) which has just been computed.
Since at this point, both x1(0) and x1(1) are available, we can use x1(1) which is
better approximation for computing x2(1). Similarly, for computing x3(1), we can
use x1(1) and x2(1) along with x4(0), …. , xn(0). This idea can be extended to all
subsequent computations. This approach is called the Gauss-Seidel method.

7/17/2023 Md. Golam Moazzam, Dept. of CSE, JU 11

 Iterative Solution of Linear Equations

❑ Gauss-Seidel Method

x1 = (b1 − a12 x2 − a13 x3 ) / a11

x2 = (b2 − a 21 x1 − a 23 x3 ) / a 22 Iteration 1
x3 = (b3 − a31 x1 − a32 x2 ) / a33

x1 = (b1 − a12 x2 − a13 x3 ) / a11

x2 = (b2 − a 21 x1 − a 23 x3 ) / a 22 Iteration 2
x3 = (b3 − a31 x1 − a32 x2 ) / a33

Jacobi method

7/17/2023 Md. Golam Moazzam, Dept. of CSE, JU 12

 Iterative Solution of Linear Equations

❑ Gauss-Seidel Method

x1 = (b1 − a12 x2 − a13 x3 ) / a11

x2 = (b2 − a 21 x1 − a 23 x3 ) / a 22 Iteration 1

x3 = (b3 − a31 x1 − a32 x2 ) / a33

x1 = (b1 − a12 x2 − a13 x3 ) / a11

x2 = (b2 − a 21 x1 − a 23 x3 ) / a 22 Iteration 2

x3 = (b3 − a31 x1 − a32 x2 ) / a33

Gauss-Seidel method

7/17/2023 Md. Golam Moazzam, Dept. of CSE, JU 13

 Iterative Solution of Linear Equations

❑ Example: Obtain the solution of the following system using Gauss-Seidel iteration
method:
2x1 + x2 + x3 = 5
3x1 + 5x2 + 2x3 = 15
2x1 + x2 + 4x3 = 8
Solution:
Solve the equations for unknowns on the diagonal. That is
x1 = (5 - x2 - x3)/2
x2 = (15 - 3x1 - 2x3)/5
x3 = (8 - 2x1 - x2)/4

7/17/2023 Md. Golam Moazzam, Dept. of CSE, JU 14

 Iterative Solution of Linear Equations

❑ Example: Obtain the solution of the following system using Gauss-Seidel iteration
method:
2x1 + x2 + x3 = 5
3x1 + 5x2 + 2x3 = 15
2x1 + x2 + 4x3 = 8
Solution:
Assume the initial values of x1, x2 and x3 to be zero (i.e. x1 = x2 = x3 = 0).
Iteration 1:
x1 = (5 - 0 - 0)/2 = 2.5
x2 = (15 – 3x2.5 – 2x0)/5 = 7.5/5 = 1.5
x3 = (8 – 2x2.5 – 1.5)/4 = 1.5/4 = 0.4

7/17/2023 Md. Golam Moazzam, Dept. of CSE, JU 15

 Iterative Solution of Linear Equations

❑ Example: Obtain the solution of the following system using Gauss-Seidel iteration
method:
2x1 + x2 + x3 = 5
3x1 + 5x2 + 2x3 = 15
2x1 + x2 + 4x3 = 8
Solution:
Iteration 2:
x1 = (5 – 1.5 – 0.4)/2 = 3.1/2 = 1.6
x2 = (15 – 3x1.6 – 2x0.4)/5 = 9.4/5 = 1.9
x3 = (8 – 2x1.6 – 1.9)/4 = 2.9/4 = 0.7

We can continue this process until we get x1 = 1, x2 = 2 and x3 = 1.

7/17/2023 Md. Golam Moazzam, Dept. of CSE, JU 16