LINEAR EQUATIONS AND MATRICES

We can write
_
2x
1
+ 3x
2
= 4
5x
1
+ 6x
2
= 7.
as
Ax = b
where A is the coefficient matrix
_
2 3
5 6
_
and x is the unknown vector
_
x
1
x
2
_
and
and b is the given vector
_
4
7
_
.
OR, as an augmented matrix
_
2 3
5 6

4
7
_
.
To solve this, we use elementary row operations, which are the following.
1) Swap 2 rows, R
i
R
j
.
2) Multiply a row by a non-zero number, R
i
k R
i
where k R and
k = 0 .
3) Add a multiple of one row to another row, R
i
R
i
+ k R
j
.
It can be shown that each of these operations gives a new system which has
the same set of solutions as the original system.
We first apply the Gaussian elimination algorithm to get the matrix
into row echelon form.
Row echelon form
In any matrix:
A leading entry is one which is the first nonzero entry in its row.
A leading column or pivot column is one containing the leading
entry for at least one row.
Example
A =
_
_
1 2 3
0 0 4
0 0 0
_
_
Leading entries are circled.
Columns 1 & 3 are leading columns. Column 2 is not.
1
Definition
A matrix is in row-echelon form (REF) if
1) any all-zero rows are at the bottom and
2) leading entry for each non-zero row is further right
than the leading entries in all rows which are higher up in the matrix.
Example The matrix A above is in row-ech form but
B =
_
_
1 2 3
0 4 0
0 5 6
_
_
is NOT.
Gaussian elimination
1)a) Select pivot entry
From the leftmost non-zero (i.e. not all zero) column,
choose a non-zero entry and call it the pivot.
(There are several versions of the algorithm with different rules for pivot
selection.)
b) Swap the pivot row with top row [If it is not already the top row]
c) Strip the column below the pivot entry
i.e. subtract multiples of the pivot row from lower rows
to make all entries 0 in the pivot column below the pivot entry.
2) Repeat 1) on the submatrix consisting of rows below the last pivot row
Repeat this till you get to row echelon form.
Example
_
_
0 3 1
3 5 7
1 1 1

8
5
1
_
_
From column 1 we choose 1 from R
3
as pivot.
Swap R
1
and R
3
, then strip below the pivot.
[Note that 3rd entry is already 0]
R
3
R
1

_
_
1 1 1
3 5 7
0 3 1

1
5
8
_
_R
2
R
2
3R
1

_
_
1 1 1
0 2 4
0 3 1

1
2
8
_
_
Step 1 is now finished.
Now consider the submatrix of rows below 1st.
The leftmost non-zero column is column 2.
Choose 2 as the pivot,
No swap is needed.
2
Strip below the pivot
R
3
R
3

3
2
R
2

_
_
1 1 1
0 2 4
0 0 5

1
2
5
_
_
This is in echelon form, so Gauss elimination is finished.
NB Echelon form for a matrix is not unique.
A different choice of pivots may give a different echelon form.
Deducing the number of solutions from row-echelon form
A system of linear equations may have no solutions
or exactly one solution
or infinitely many solutions.
From the row-echelon form for the augmented matrix we can tell
how many solutions there are.
Each column of the coefficient matrix consists of
the coefficients for one variable.
We say a variable is a
_
leading
dependent
_
variable if
its column of coefficients is a leading column.
Otherwise, we say it is a
_
_
_
non-leading
independent
free
_
_
_
variable.
Theorem on the number of solutions
i) A system has NO solns if and only if an echelon form for it has
an impossible row [0 0 . . . 0 | a] where a = 0
[which means 0x
1
+ 0x
2
+ . . . + 0x
n
= a where a = 0 ,
which is impossible.]
ii) If there is no impossible row, the system has
a) a unique soln if and only if every variable is a leading variable.
b) infinitely many solns if and only if at least one variable is non-leading,
in which case,
_ number of parameters
in the general solution
_
=
_
number of non-leading variables
in echelon form
_
Example
i)
_
2 1
0 2

5
2
_
ii)
_
2 1
0 0

1
1
_
iii)
_
2 2
0 0

1
0
_
In (i), no impossible rows and every variable is leading, so unique solution.
In (ii), 2nd row is impossible, so there is no solution.
In (iii), no impossible rows and x
2
is non-leading, so infinitely many
solutions.
3
Getting from echelon form to solutions.
Choice of two methods.
1) Back-substitution
Assign an arbitrary parameter (e.g. , , . . . )
as value for each nonleading variable.
Solve the last equation for the last leading variable
in terms of the values of any non leading variables.
Substitute in 2nd last equation and solve for 2nd last leading variable.
And so on.
Example
_
1 2 3
0 2 4

4
6
_
represents
x
1
+ 2x
2
+ 3x
3
= 4
2x
2
+ 4x
3
= 6
.
x
3
nonleading, so let x
3
= .
2nd equation gives x
2
= 3 2 .
Substitute in 1st and get x
1
= 2.
So solution is
_
_
x
1
x
2
x
3
_
_
=
_
_
2
3 2

_
_
=
_
_
2
3
0
_
_
+
_
_
1
2
1
_
_
, R.
with one arbitrary parameter .
2) Reduction to reduced ech form
Definition
A matrix is in reduced row-echelon form (RREF) if
it is in row-echelon form and
1) every leading entry is 1 and
2) every leading column contains only one non-zero entry.
Example The matrix
C =
_
_
1 0 2
0 1 3
0 0 0
_
_
is in reduced echelon form.
[Note: MAPLE has command rref for finding row reduced echelon form.]
To get from echelon form to reduced echelon form:
Mutiply lowest nonzero row by a constant so that leading entry is 1.
Use this row to strip entries above leading entry in this row.
Repeat with next row up, and so on.
When RREF is reached,
4
assign an arbitrary parameter value to each nonleading variable,
then read off solutions for leading variables in terms of parameters.
Example
_
1 2 3
0 2 4

4
6
_
R
2

1
2
R
2

_
1 2 3
0 1 2

4
3
_
R
1
R
1
2R
2

_
1 0 1
0 1 2

2
3
_
This is in reduced echelon form.
let x
3
= and read off x
1
= 2, x
2
= 3 2 .
SYSTEMS WITH VARIABLE RHS
Solubility without knowing rhs
You may need to solve Ax = b many times with same A but varying b .
The choice of row operations to solve Ax = b does not depend on rhs b .
We can reduce coefficient matrix A alone (not the augmented matrix [A|b] )
to echelon form U and make deductions about the existence of solutions.
Theorem on solubility for all b
If A is an mn matrix which can be transformed by row operations to
row-echelon form U then system Ax = b has
a) at least one solution for each b R
n
iff U has no all-zero rows
b) at most one solution for each b R
n
iff U has no non-leading vars
c) exactly one solution for each b R
n
iff U has
no all-zero rows and no non-leading variables.
Example 1)
U =
_
1 2
0 3
_
.
This has no all-zero rows and no non-leading columns, so there is exactly
one solution for every b .
Example 2)
U =
_
1 2
0 0
_
.
This has an all-zero row, so for some values of b there will be no solution.
It also has a non-leading variable, so for some values of b there will be
infinitely many solutions.
5
Solving systems with variable right hand side
By doing reduction with variable b , we can avoid redoing it all if b changes.
We then get a solution in terms of b .
It may show that b must satisfy some conditions for a solution to exist.
Example 1) Find all solutions to Ax = b if A =
_
1 1
2 3
_
.
_
1 1
2 3

b
1
b
2
_
R
2
R
2
2R
1

_
1 1
0 1

b
1
2b
1
+ b
2
_
(REF)
R
1
R
1
R
2

_
1 0
0 1

3b
1
b
2
2b
1
+ b
2
_
(RREF).
So for every b the system has unique soln
x =
_
3b
1
b
2
2b
1
+ b
2
_
.
Example 2)
If A =
_
1 2
2 4
_
, find conditions on b which are
necesssary and sufficient ensure that Ax = b has a soln.
_
1 2
2 4

b
1
b
2
_
R
2
R
2
2R
1

_
1 2
0 0

b
1
2b
1
+ b
2
_
So this system has solution if and only if 2b
1
+ b
2
= 0 .
(If this is satisfied, put nonleading x
2
as and get
infinitely many solutions given by
x =
_
b
1
0
_
+
_
2
1
_
R.)
6
MATRIX OPERATIONS
Matrix multiplication
If A is an mn matrix
and B is an n p matrix,
we can define a mp matrix AB so that
A(Bx) = (AB)x for any x R
p
.
So applying matrix B to vector x to get Bx R
n
then applying matrix A to vector Bx to get A(Bx) R
m
is the same as applying product matrix AB to vec x .
We define AB by
jth column of AB = result of applying A
to jth column of B
(considered as a column vector)
ij entry in AB = a
i1
b
1j
+ a
i2
b
2j
+ . . . + a
in
b
nj
i.e.
=
n

k=1
a
ik
b
kj
Example
_
a b
c d
__
p q
r s
_
=
_
ap + br aq + bs
cp + dr cq + ds
_
[NB: AB does not exist unless
number of columns in A = number of rows in B.]
Matrix multiplication is not commutative
i.e. in general AB = BA
Example A =
_
2 0
0 1
_
, B =
_
1 2
3 4
_
, AB =
_
2 4
3 4
_
, BA =
_
2 2
6 4
_
.
Properties of matrix multiplication
It can be shown that:
If A, B, C have the right number of rows and columns for the relevant sums
and products to exist then
1) Interaction with multiplication by a scalar
A(B) = (AB) for R.
2) Asociativity of multiplication
A(BC) = (AB)C
7
3) Distributive laws
A(B + C) = AB + AC and (B + C)A = BA + CA
4) Multiplication by Identity matrices
If A is mn then AI
n
= A and I
m
A = A.
INVERSE MATRICES
Say A is invertible if there exists a matrix B such that
AB = BA = I
We say B is an inverse for A.
Example B =
_
3 1
2 1
_
is an inverse for A =
_
1 1
2 3
_
.
(Check AB = BA = I .)
The Basic Facts about Inverses
1) A non-square matrix cannot have an inverse.
2) Some square matrices do not have inverses.
3) If A is invertible then it has a unique inverse denoted A
1
.
4) If A is invertible then A
1
is invertible and (A
1
)
1
= A
5) If A and B are invertible and AB is defined,
then AB is invertible and (AB)
1
= B
1
A
1
. (Note order)
6) If A is an n n square matrix then
A is invertible
if and only if for each b R
n
, Ax = b has exactly one soln,
namely x = A
1
b .
if and only if Ax = 0 has exactly one soln, namely x = 0
if and only if A can be reduced by row ops to I .
7) To show that a square matrix A is invertible and has inverse B ,
you only need to show that AB = I
(because BA = I folows automatically).
8) If A is invertible, we can find A
1
by using row ops to reduce [A | I] to
the form [I | C] . Then A
1
= C .
8
Example 1) A =
_
_
1 0 2
2 1 6
1 0 3
_
_
.
_
_
1 0 2
2 1 6
1 0 3

1 0 0
0 1 0
0 0 1
_
_
R
2
R
2
2R
1

R
3
R
3
R
1
_
_
1 0 2
0 1 2
0 0 1

1 0 0
2 1 0
1 0 1
_
_
R
1
R
1
2R
3

R
2
R
2
2R
3
_
_
1 0 0
0 1 0
0 0 1

3 0 2
0 1 2
1 0 1
_
_
So A is invertible and A
1
=
_
_
3 0 2
0 1 2
1 0 1
_
_
.
Example 2) If A =
_
_
1 0 3
2 1 6
1 0 3
_
_
then [A | I] reduces to ech form
_
_
1 0 2
0 1 2
0 0 0

1 0 0
2 1 0
1 0 1
_
_
3rd variable is non-leading,
so Ax = 0 has infinitely many solns
so A is not invertible.
All the Basic Facts except (7) are stated and proved in FYN
and (7) can be deduced from (6) as follows:
Suppose A is square and AB = I .
Then Ax = b has a solution for every b ,
in fact Bb is a solution because
A(Bb) = (AB)b = Ib = b,
so an echelon form U for A has no zero rows. ( )
Since U is square, ( ) implies that all columns are leading columns
so there cannot be more than one solution for any b
so A is invertible (by (6)) and then
B = IB = (A
1
A)B = A
1
(AB) = A
1
I = A
1
.
9
TRANSPOSES
The transpose of a matrix A is the matrix A
t
whose ij entry is a
ji
.
Note order ji . i.e. swap rows with columns.
Example 1)
A =
_
_
1 2
3 4
5 6
_
_
A
t
=
_
1 3 5
2 4 6
_
.
Example 2) [ 1 2 3 ]
t
=
_
_
1
2
3
_
_
.
Properties of taking transpose
1) It preserves addn and multiplication by a scalar, i.e.
(A + B)
t
= A
t
+ B
t
(A)
t
= (A
t
).
2) It does NOT preserve matrix multiplication, in fact
(AB)
t
= B
t
A
t
and not A
t
B
t
. Note order.
3) It it is an involution, i.e. (A
t
)
t
= A.
4) If A is invertible then A
t
is invertible and (A
t
)
1
=
_
A
1
_
t
.
Statements (1)-(3) are stated and proved in FYN. Exercise: prove (4).
Relations between transpose and dot product in R
n
If x, y R
n
and A is a real n n matrix then
1) x y = x
t
y
For example: x =
_
a
b
_
, y =
_
p
q
_
,
x
t
y = [ a b ]
_
p
q
_
= ap + bq = x y.
2) (Ax) y = x (A
t
y)
Proof: (Ax) y = (Ax)
t
y = x
t
A
t
y = x (A
t
y) .
10