MA 110

Linear Algebra and Differential Equations

Lecture 02

Prof. Sudhir R. Ghorpade

Department of Mathematics
IIT Bombay
http://www.math.iitb.ac.in/∼srg/

Spring 2025

Prof. S. R. Ghorpade, IIT Bombay Linear Algebra: Lecture 02

Linear System
Let m, n ∈ N. A linear system of m equations in the n
unknowns x1 , ..., xn is given by

a11 x1 + a12 x2 + · · · + a1n xn = b1 (1)

a21 x1 + a22 x2 + · · · + a2n xn = b2 (2)
.. .. .. .. ..
. . . . .
am1 x1 + am2 x2 + · · · + amn xn = bm , (m)

where ajk ∈ R for j = 1, . . . , m; k = 1, . . . , n and also bj ∈ R

for j = 1, . . . , m are given.
 T
Let A := [ajk ] ∈ Rm×n , x := x1 · · · xn ∈ Rn×1 and
 T
b := b1 · · · bm ∈ Rm×1 . Using matrix multiplication, we
write the linear system as
Ax = b.
Prof. S. R. Ghorpade, IIT Bombay Linear Algebra: Lecture 02
The m × n matrix A is known as the coefficient matrix of
the linear system.
A column vector x0 ∈ Rn×1 is called a solution of the above
linear system if it satisfies Ax0 = b.
Case (i) Homogeneous Linear System: b := 0, that is,
b1 = · · · = bm = 0.
A homogenous linear system always has a solution, namely the
 T
zero solution 0 := 0 · · · 0 since A0 = 0.
Also, if r ∈ N and x1 , . . . , xr are solutions of such a system,
then so is their linear combination α1 x1 + · · · + αr xr , since
A(α1 x1 + · · · + αr xr ) = α1 Ax1 + · · · + αr Axr = 0 + · · · + 0 = 0.
Case (ii) General Linear System: b ∈ Rm×1 is arbitrary.
A nonhomogenous linear system, that is, where b 6= 0, may
not have a solution, may have only one solution or may have
(infinitely) many solutions.
Prof. S. R. Ghorpade, IIT Bombay Linear Algebra: Lecture 02
Examples
(i) The linear system x1 + x2 = 1, 2x1 + 2x2 = 1 does not have
a solution.
(ii) The linear system x1 + x2 = 1, x1 − x2 = 0 has a unique
solution, namely x1 = 1/2 = x2 .
(iii) The linear system x1 + x2 = 1, 2x1 + 2x2 = 2 has
(infinitely) many solutions, namely x1 = α, x2 = 1 − α, α ∈ R.
Important Note:
Let S denote the set of all solutions of a homogeneous linear
system Ax = 0. If x0 is a particular solution of the general
system Ax = b, then the set of all solutions of the general
system Ax = b is given by {x0 + s : s ∈ S} since
s ∈ S =⇒ A(x0 + s) = Ax0 + As = b + 0 = b, and also
Ax = b =⇒ A(x − x0 ) = Ax − Ax0 = b − b = 0, so that
(x − x0 ) ∈ S, that is, x = x0 + s for some s ∈ S.
Prof. S. R. Ghorpade, IIT Bombay Linear Algebra: Lecture 02
We shall, therefore, address the problem of finding all solutions
of a homogeneous linear system Ax = 0, and one particular
solution of the corresponding general system Ax = b.
A Special Case
Suppose the coefficient matrix A is upper triangular and its
diagonal elements are nonzero. Then the linear system

a11 x1 + a12 x2 + · · · + · · · + · · · + · · · + a1n xn = b1 (1)

a22 x2 + a23 x3 + · · · + · · · + · · · + a2n xn = b2 (2)
... .. .. ..
. . .
a(n−1)(n−1) xn−1 + a(n−1)n xn = bn−1 (n − 1)
ann xn = bn (n)

can be solved by back substitution as follows.

Prof. S. R. Ghorpade, IIT Bombay Linear Algebra: Lecture 02

 xn = bn /ann
xn−1 = (bn−1 − a(n−1)n xn )/a(n−1)(n−1), where xn = bn /ann
.. .. ..
. . .
x2 = (b2 − a2n xn − · · · − a23 x3 )/a22 , where xn = · · ·, x3 = · · ·,
x1 = (b1 − a1n xn − · · · − · · · − a12 x2 )/a11 , where xn = · · ·, x2 = · · ·

Here the homogeneous system Ax = 0 has only the zero

solution and the general system Ax = b has a unique solution.
Taking a cue from this special case of an upper triangular
matrix, we shall attempt to transform any m × n matrix to an
upper triangular form. In this process, we successively attempt
to eliminate the unknown x1 from the equations (m), . . . , (2),
the unknown x2 from the equations (m), . . . , (3), and so on.

Prof. S. R. Ghorpade, IIT Bombay Linear Algebra: Lecture 02

Example
Consider the linear system

x1 − x2 + x3 = 0
−x1 + x2 − x3 = 0
10x2 + 25x3 = 90
20x1 + 10x2 = 80.

Eliminating x1 from the 4th, 3rd and 2nd equations,

x1 − x2 + x3 = 0
0 = 0
10x2 + 25x3 = 90
30x2 − 20x3 = 80.

Interchanging the 2nd and the 3rd equations,

Prof. S. R. Ghorpade, IIT Bombay Linear Algebra: Lecture 02
 x1 − x2 + x3 = 0
10x2 + 25x3 = 90
0 = 0
30x2 − 20x3 = 80.

Eliminating x2 from the 4th equation, and then interchanging

the 3rd and the 4th equations,

x1 − x2 + x3 = 0
10x2 + 25x3 = 90
−95x3 = −190
0 = 0.

Now back substitution gives x3 = 2, x2 = (90 − 25x3 )/10 = 4

 T
and x1 = −x3 + x2 = 2, that is, x = 2 4 2 .
Prof. S. R. Ghorpade, IIT Bombay Linear Algebra: Lecture 02
The above process can be carried out without writing down the
entire linear system by considering the augmented matrix
 
a11 a12 · · · a1n b1
 a12 a22 · · · a2n b2 
[A|b] :=  .. .. .. .. ..  .
 
 . . . . .
am1 am2 · · · amn bn

This m × (n + 1) matrix completely describes the linear system.

In the above example,
 
1 −1 1 0
−1 1 −1 0
[A|b] = 
 0 10 25
.
90
20 10 0 80

Subtracting 20 times the first row from the 4th row, and
adding the first row to the second row, we obtain
Prof. S. R. Ghorpade, IIT Bombay Linear Algebra: Lecture 02
  
1 −1 1 0
R4 −20R1 , R2 +R1 , 0 0 0 0
−−−−−−−−−→  0 10 25
.
90
0 30 −20 80
Interchanging the 2nd and the 3rd rows, we obtain
 
1 −1 1 0
R ←→R3 
−−2−−−→ 0 10 25 90.
0 0 0 0
0 30 −20 80
Finally, subtracting 3 times the 2nd row from the 4th row and
interchanging the  3rd and the 4th rows,  we arrive at
1 −1 1 0
R4 −3R2 , R3 ←→R4 0 10 25 90 
−−−−−−−−−−→  0 0 −95
.
−190
0 0 0 0
The upper triangular nature of the 3 × 3 matrix on the top left
enables back substitution.
Prof. S. R. Ghorpade, IIT Bombay Linear Algebra: Lecture 02
Row Echelon Form
We shall now consider a general form of a matrix for which the
method of back substitution works.
Let A ∈ Rm×n , that is, A is an m × n matrix with real entries.
A row of A is said to be zero if all its entries are zero.
If a row is not zero, then its first nonzero entry (from the left)
is called the pivot.Thus all entries to the left of a pivot equal 0.
Suppose A has r nonzero rows and m − r zero rows.
Then 0 ≤ r ≤ m. Clearly, r = 0 ⇐⇒ A = O.
Example 
0 1 4
If A =  0 0 0, then m = n = 3 and r = 2.
5 6 7
The pivot in the 1st row is 1 and the pivot in the 3rd row is 5.
Prof. S. R. Ghorpade, IIT Bombay Linear Algebra: Lecture 02
A matrix A is said to be in a row echelon form (REF)1 if the
following conditions are satisfied.
(i) The nonzero rows of A precede the zero rows of A.
(ii) If A has r nonzero rows, where r ∈ N, and the pivot in
row 1 appears in the column k1 , the pivot in row 2 appears in
the column k2 , and so on the pivot in row r appears in the
column kr , then k1 < k2 < · · · < kr .
Examples      
0 4 0 1 4
1 0 1 4
The matrices 0
   0
0 , 5 6 7 , 0
  5 7
5 7 6
0 0 0 0 0 0
 
5 6 7
are not in REF. The matrix 0 1 4 is in a
 REF.
0 0 0
1
In French, echelon means level.
Prof. S. R. Ghorpade, IIT Bombay Linear Algebra: Lecture 02
Pivotal Columns
(i) Suppose a matrix A ∈ Rm×n is in a REF. If A has exactly r
nonzero rows, then there are exactly r pivots. A column of A
containing a pivot, is called a pivotal column. Thus there are
exactly r pivotal columns, and so 0 ≤ r ≤ n.
(We have already noted that 0 ≤ r ≤ m.)
(ii) In a pivotal column, all entries below the pivot equal 0.
Here is a typical example of how back substitution works when
a matrix A is in a REF. Let
   
0 a12 a13 a14 a15 a16 b1
0 0 0 a24 a25 a26  b2  .
  
A := 0 0 and b := b3 
0 0 a35 a36 
0 0 0 0 0 0 b4

where a12 , a24 , a35 are nonzero. They are the pivots.
Prof. S. R. Ghorpade, IIT Bombay Linear Algebra: Lecture 02
Here m = 4, n = 6, r = 3, pivotal columns: 2 , 4 and 5 .
 T
Suppose there is x := x1 · · · x6 ∈ R6×1 such that Ax = b.
Then 0x1 + · · · + 0x6 = b4 , that is, b4 must be equal to 0.
Next, a35 x5 + a36 x6 = b3 , that is, x5 = (b3 − a36 x6 )/a35 , where
we can assign an arbitrary value to the unknown x6 .
Next, a24 x4 + a25 x5 + a26 x6 = b2 , that is,
x4 = (b2 − a25 x5 − a26 x6 )/a24 , where we back substitute the
values of x5 and x6 .
Finally, a12 x2 + a13 x3 + a14 x4 + a15 x5 + a16 x6 = b1 , that is,
x2 = (b1 − a13 x3 − a14 x4 − a15 x5 − a16 x6 )/a12 , where we can
assign an arbitrary value to the variable x3 , and back
substitute the values of x4 , x5 and x6 .
Also, we can assign an arbitrary value to the variable x1 .
The variables x1 , x3 and x6 to which we can assign arbitrary
values correspond to the nonpivotal columns 1 , 3 and 6 .
Prof. S. R. Ghorpade, IIT Bombay Linear Algebra: Lecture 02
Suppose an m×n matrix A is in a REF, and there are r
nonzero rows. Let the r pivots be in the columns k1 , . . . , kr
with k1 < · · · < kr , and let the columns `1 , . . . , `n−r be
nonpivotal. Then xk1 , . . . , xkr are called the pivotal
variables and x`1 , . . . , x`n−r are called the free variables.
Let b := [b1 · · · br br +1 · · · bm ]T , and consider the linear system
Ax = b.
Important Observations

1. The linear system has a solution ⇐⇒ br +1 = · · · = bm = 0.

This is known as the consistency condition.
2. Let the consistency condition br +1 = · · · = bm = 0 be
satisfied. Then we obtain a particular solution
 T
x0 := x1 · · · xn of the linear system by letting xk := 0
if k ∈ {`1 , . . . , `n−r }, and then by determining the pivotal
variables xk1 , . . . , xkr by back substitution.
Prof. S. R. Ghorpade, IIT Bombay Linear Algebra: Lecture 02
3. We obtain n − r basic solutions of the homogeneous
linear system Ax = 0 as follows. Fix ` ∈ {`1 , . . . , `n−r }.
 T
Define s` := x1 · · · xn by xk := 1 if k = `, while
xk := 0 if k ∈ {`1 , . . . , `n−r } but k 6= `. Then determine
the pivotal variables xk1 , . . . , xkr by back substitution.
 T
4. Let s := x1 · · · xn ∈ Rn×1 be any solution of the
homogeneous system, that is, As = 0. Then s is a linear
combination of the n − r basic solutions s`1 , . . . , s`n−r .
To see this, let y := s − x`1 s`1 − · · · − x`n−r s`n−r . Then
Ay = As − x`1 As`1 − · · · − x`n−r As`n−r = 0, and moreover,
the kth entry of y is 0 for each k ∈ {`1 , . . . , `n−r }. It then
follows that y = 0, that is, s = x`1 s`1 + · · · + x`n−r s`n−r .
Thus we find that the general solution of the
homogeneous system is given by
s = α1 s`1 + · · · + αn−r s`n−r , where α1 , . . . , αn−r ∈ R .

Prof. S. R. Ghorpade, IIT Bombay Linear Algebra: Lecture 02

 5. The general solution of Ax = b is given by

x = x0 + α1 s`1 + · · · + αn−r s`n−r , where α1 , . . . , αn−r ∈ R,

provided the consistency condition is satisfied.

Example
   
0 2 1 0 2 5 0
1 .
0 0 0 3 5 0  
Let A := 
0 and b := 2
0 0 0 1 2
0 0 0 0 0 0 0

As we have seen earlier, here m = 4, n = 6, r = 3, pivotal

columns: 2 , 4 and 5 , and nonpivotal columns: 1 , 3 , 6 .
Since b4 = 0, the linear system Ax = b is consistent.

Prof. S. R. Ghorpade, IIT Bombay Linear Algebra: Lecture 02

For a particular solution of Ax = b, let x1 = x3 = x6 = 0.
Then x5 + 2x6 = 2 =⇒ x5 = 2,
3x4 + 5x5 + 0x6 = 1 =⇒ x4 = −3,
2x2 + x3 + 0x
 4 + 2x5 + 5x6 = 0 =⇒T x2 = −2.
Thus x0 := 0 −2 0 −3 2 0 is a particular solution.
Basic solutions of Ax = 0:  T
x1 = 1, x3 = x6 = 0 gives s1 := 1 0 0 0 0 0 ,
 T
x3 = 1, x1 = x6 = 0 gives s3 := 0 −1/2 1 0 0 0 ,
 T
x6 = 1, x1 = x3 = 0 gives s6 := 0 −1/2 0 10/3 −2 1 .
The general solution of Ax = b is given by
x = x0 + α1 s1 + α3 s3 + α6 s6 , that is,
x1 = α1 , x2 = −2 − (α3 + α6 )/2, x3 = α3 , x4 = −3 + 10α6 /3,
x5 = 2(1 − α6 ), x6 = α6 , where α1 , α3 , α6 ∈ R.

Prof. S. R. Ghorpade, IIT Bombay Linear Algebra: Lecture 02

Conclusion
Suppose an m × n matrix A is in a REF, and let r be the
number of nonzero rows of A. If b ∈ Rm×1 , then the linear
system Ax = b has
(i) no solution if one of br +1 , . . . , bm is nonzero.
(ii) a unique solution if br +1 = · · · = bm = 0 and r = n.
(iii) infinitely many solutions if br +1 = · · · = bm = 0 and
r < n. In this case, there are n − r free variables which give


n − r degrees of freedom .
Considering the case b = 0 ∈ Rm×1 and recalling that r ≤ m,
we obtain the following important results.
Proposition
Let A ∈ Rm×n be in REF with r nonzero rows. Then the linear
system Ax = 0 has only the zero solution if and only if r = n.
In particular, if m < n, then Ax = 0 has a nonzero solution.
Prof. S. R. Ghorpade, IIT Bombay Linear Algebra: Lecture 02