Numerical

Method
Chapter 2

- Er. Ganesh Ram Dhonju

2. Solution of system of linear algebraic equations (8 hours)

2.1 Direct methods

2.1.1 Gauss Jordan method
2.1.2 Gauss elimination method, pivoting strategies (Partial and complete)
2.1.3 Matrix inverse using Gauss Jordan and Gauss elimination methods
2.1.4 Factorization methods (Do-Little’s method and Crout’s method)
2.2 Iterative methods
2.2.1 Jacobi’s method
2.2.2 Gauss-Seidal method
2.3 Determination of largest and smallest Eigen values and corresponding vectors using the
power method
Linear Algebraic Equations
– An equation of the form ax+by+c=0 or equivalently ax+by=-c is
called a linear equation in x and y variables.
– ax+by+cz=d is a linear equation in three variables, x, y, and z.
– Thus, a linear equation in n variables is
a1x1+a2x2+ … +anxn = b
– A solution of such an equation consists of real numbers c1, c2, c3,
… , cn. If you need to work more than one linear equations, a
system of linear equations must be solved simultaneously.
 System of linear equation
– Metrix notation of linear equation
No solution Infinite solutions Ill-conditioned
(Slopes are too close)
Classiication of methods to solve a linear system of equations
Basic Gauss Elimination Method
– The equations were manipulated to eliminate one of the unknowns from the
equations. The result of this elimination step was that we has one equation
with one unknown.
– Consequently this equation could be solved directly and the result back
substituted into one of the original equations to solve for the remaining
unknown
 Forward Elimination of unknowns
– The first phase is designed to reduce the set of equations to an upper triangular system.
– The initial step will be to eliminate the first unknown x1 from the second through the nth
equations. To do this multiply equation by a21/a11 to give
Back subtitution
– Above equation can now be solved for xn as

– The result can be back substituted into the (n-1)th

equation to solve for xn-1. The procedure, which is
repeated to evaluate the remaining x’s can be represented
by the following formula:
 Problem
2x1+4x2-6x3 = -8 x+y+z=9 -3x1+2x2-x3 = -1
x1+3x2+x3 = 10 6x1-6x2+7x3 = -7
2x - 3y + 4z = 13
2x1 – 4x2-2x3 = -12 3x1 – 4x2+4x3 = -6
3x + 4y + 5z = 40

3x+2y+7z = 4 10x-7y+3z+5u=6
2x+3y+z = 5 -6x+8y-z-4u=5
3x+4y+z = 7 3x+y+4z+11u=2
5x-9y-2z+4u = 7
 Pitfalls of Elimination Methods
– Division by zero. It is possible that during both elimination and back-substitution
phases a division by zero can occur.
– Round-off errors.
– Ill-conditioned systems. Systems where small changes in coefficients result in
large changes in the solution. Alternatively, it happens when two or more
equations are nearly identical, resulting a wide ranges of answers to
approximately satisfy the equations. Since round off errors can induce small
changes in the coefficients, these changes can lead to large solution errors.
Slight change of – Singular systems. When two equations are identical, we would loose one degree
coefficient a21 from 1.1
of freedom and be dealing with the impossible case of n-1 equations for n
to 1.05, the result is
changed dramatically unknowns. For large sets of equations, it may not be obvious however. The fact
that the determinant of a singular system is zero can be used and tested by
computer algorithm after the elimination stage. If a zero diagonal element is
created, calculation is terminated.
Partial Pivoting / Complete Pivoting Method
– Solve using gauss elimination
0.0003 x +3.0000y = 2.0001
1.0000 x+1.0000y =1.0000
Techniques for Improving Solutions
– Use of more significant figures.
– Pivoting. If a pivot element is zero, normalization step leads to division by zero.
The same problem may arise, when the pivot element is close to zero. Problem
can be avoided:
– Partial pivoting. Switching the rows so that the largest element is the pivot element.
– Complete pivoting. Searching for the largest element in all rows and columns then
switching.
Gauss-Jordan Method
– We begin by writing the system as an augmented matrix

x = 1,
y = -1,
z=2
Solve the system
2x - 2 y + z = 13
2x + 3y + 4z = 20
3x - y + 3z = 10

2x + y + 4z = 12
8x - 3y + 2z = 20
4x + 11y - z = 33
Gauss-Jordan Method to find inverse metrix
– To find the inverse to an nxn matrix A:
– Adjoin the identity matrix I to the right side of A, thereby producing a matrix of the
form [A|I]
– Apply row operations to this matrix until the left side is reduced to I. If successful,
these operations will convert the right side to A-1, so that the final matrix will have
the form [I|A-1]
 Find the inverse matrix of following metrix using Gauss-Jordan Method.

– Do row operations to get upper triangle form – Continue doing row operations to get 0’s
in columns above the pivots:

– At this point the last matrix

on the left is the identity.
Thus, the right matrix must
be the inverse to A:
 Find the inverse matrix of following metrix using Gauss Elimination Method.
2 1 1 2 1 1 𝑥11 1
A= 3 2 3 2
0 1/2 3/2 𝑥21 = −3/2
1 4 9
0 0 −2 𝑥31 10
A I
2 1 1 𝑥12 0
2 1 1 1 0 0 0 1/2 3/2 𝑥22 = 1 3
3 2 3 0 1 0 0 0 −2 𝑥32 −7
1 4 9 0 0 1
2 1 1 𝑥13 0
0 1/2 3/2 𝑥23 = 0 4
0 0 −2 𝑥33 1
A I
2 1 1 1 0 0 −3 5/2 −1/2
0 1/2 3/2 −3/2 1 0 1 A-1= 12 −17/2 7/2
0 0 −2 10 −7 1
−1/2 3/2 1/2
𝑥11 𝑥12 𝑥13
is the inverse of A then eqn(i) is equivalent to
Now if 𝑥21 𝑥22 𝑥23
the following three system
𝑥31 𝑥32 𝑥33 AA-1=A-1A=I
Let’s try this also
Gauss Elimination Algorithm:
1. Start
2. Declare the variables and read the order of the matrix n.
3. Take the coefficients of the linear equation as:
Do for k=1 to n
Do for j=1 to n+1
Read a[k][j]
End for j
End for k
4. Do for k=1 to n-1
Do for i=k+1 to n
Do for j=1 to n+1
a[i][j] = a[i][j] – a[i][k] /a[k][k] * a[k][j]
End for j
End for i
End for k
5. Compute x[n] = a[n][n+1]/a[n][n]
6. Do for k=n-1 to 1
sum = 0
Do for j=k+1 to n
sum = sum + a[k][j] * x[j]
End for j
x[k] = 1/a[k][k] * (a[k][n+1] – sum)
End for k
7. Display the result x[k]
8. Stop
Gauss Jordan Method Algorithm:
1. Start

2. Read the order of the matrix ‘n’ and read the coefficients of the linear equations.

3. Do for k=1 to n

Do for l=k+1 to n+1

a[k][l] = a[k][l] / a[k][k]

End for l

Set a[k][k] = 1

Do for i=1 to n

if (i not equal to k) then,

Do for j=k+1 to n+1

a[i][j] = a[i][j] – (a[k][j] * a[i][k])

End for j

End for i

End for k

4. Do for m=1 to n

x[m] = a[m][n+1]

Display x[m]

End for m

5. Stop
Let’s take an example
Factorization Method
– LU Decomposition
– The LU decomposition is useful when one needs to solve Ax = b, when A is fixed
and there are many different b’s.
– First one determines L and U using Gaussian elimination. Then one writes
Ax = b
(LU)x = L(Ux) = b
we let
y = Ux,
And first solve, Ly = b
For y be forward substitution. We then solve, Ux = y
Solving Method
– Given
[A][X] = B

– Decompose [A] in to [L] and [U]

– Solve [L][Z] = B for [Z]

– Solve [U][X] = [Z] for [X]

Process
Numerical Problem
Iterative methods - Jacobi method
– Iterative or approximate methods provide and alternative to the elimination
methods.
– The system [A][X] = [B] is reshaped by solving the first equation for x1, the
second equation for x2, and third for x3, ….and nth equation for xn.
– Now we can start the solution process by choosing guesses for x’s.
– A simple way to obtain initial guesses is to assume that they are zero.
– These zeros can be substituted into x1 equation to calculate a new x1 = b1/a11.
– This technique includes computing a set of new x’s on the basis of a set of old
x’s.
– Update the values of all x’s simultaneously as a set using the set of x’s from the
previous iteration
– The procedure is repeated until the convergence criterion is satisfied:
Which one is Gauss-Seidel/Jacobi ?
20x + y – 2z = 17 3x – 0.1y – 0.2z = 7.85
3x + 20y – z = -18 0.1x + 7y – 0.3z = -19.3
2x – 3y + 20 z = 25 0.3x – 0.2y + 10z = 71.4
Gauss‐Seidel method
– Update the values of x’s one by one using the latest available set of x’s.

– New x1 is substituted to calculate x2 and x3, The procedure is repeated until the
convergence criterion is satisfied.
 1 − 3 x 2 + 5 x3 With an initial guess of
x1 =
12  x1  1
28 − x1 − 3x3  x  = 0 
x2 =  2  
5  x3  1
76 − 3x1 − 7 x 2
x3 =
After Iteration #1 13
 x1  0.5000 Substituting the x values into the After Iteration #2
 x  = 4.9000 equations  x1  0.14679
 2    x  =  3.7153 
 x3  3.0923 1 − 3(4.9000) + 5(3.0923)
x1 = = 0.14679  2  
12  x3   3.8118 
28 − (0.14679) − 3(3.0923)
x2 = = 3.7153
5

76 − 3(0.14679) − 7(4.900)
x3 = = 3.8118
13
 Repeating more iterations, the following values are obtained

 x1  0.99919  x1  1
The solution obtained  x  =  3.0001  is close to the exact solution of  x  =  3 .
 2    2  
 x3   4.0001   x3  4
Eigen value and Eigen vector
Eigen value and Eigen vector using Power method

– Power method is one of the most well suited iterative approach for machine
computations.
– With this, the numerically greatest eigen value and the corresponding eigen
vector can be computed to analyze different engineering problems.
– Solving polynomials, systems of equations and differential equations, with
applications to physics, geology, civil engineering, mechanical engineering,
machine learning etc
Power Method to find dominant eigen value

– Define initial eigen vector X

– Calculate Y = AX
– Find the largest element in magnitude of matrix Y and assign it to K.
– Calculate fresh value X = (1/K) * Y
– If [Kn – Kn-1] > e go to stem 3
Else print Kn, as the dominant eigen value and the corresponding eigen
vector.
– Find the Largest eigen value and the corresponding eigen vector of the matrix
using Power Method.
1 2 0
A= 2 1 0
0 0 −1
8 −3 8
– A= −3 8 −3
8 −3 8
2 −2 4
– A= 2 3 2
−1 1 1
6 −2 2
– A= −2 5 0 8.96
2 0 7
For Smallest Eigenvalue
– Write given matrix, A
1
– Do the inverse of A by 𝐴−1 = .Adj A
𝐴

– Do Power Method by using 𝐴−1 (instead of A)

– You will get the largest eigenvalue of 𝐴−1 (The inverse of which is the smallest
eigenvalue of A) and the corresponding eigenvector

Note: if λ is one of the eigenvalue of A. Then 1/ λ is also an eigenvalue of 𝐴−1 .

 Find out the largest and smallest eigenvalue
and eigenfunction using Power Method
6 −2 2
– 𝐴 = −2 5 0 = 6(35-0)-(-2)(-14-0)+2(0-10)=162
6 −2 2 2 0 7
– A= −2 5 0 35 14 10
2 0 7 Adj A= 14 38 −4
−10 −4 26
– Step 1: find A-1
1 1 35/162 14/162 10/162 1
𝐴−1 = .Adj A – Let X1 = 1 then 𝐴−1 𝑋1= 14/162 38/162 −4/162 1
𝐴
1 −10/162 −4/162 26/162 1
35/162 14/162 10/162
0.24074 0.8125
A-1= 14/162 38/162 −4/162 = 0.2962 = 0.2962 1
−10/162 −4/162 26/162 0.0740 0.25
Step 2: Apply Power method to A-1 to get
the smallest eigenvalue of A 0.3272
𝐴−1 𝑋𝑛 = 0.3309 Largest value=0.3309
−0.16149