An introduction to linear algebra

Lecture 1: Matrices and Linear Systems

by Dr. Pham Huu Anh Ngoc
13. August 2012
I. Matrix and operations
Denition:
An m x n matrix is a rectangular array of numbers arranged in m rows
(horizontal lines) and n columns (vertical lines).
Example: A matrix with 3 rows and 2 columns : a 3 x 2 matrix (read a 3
by 2 matrix)
_
_
0 1
3 1
0 0
_
_
A matrix with 3 rows and 3 columns : a 3 x 3 matrix
_
_
1 2 3
5 100 2
2 2 1
_
_
I. Matrix and operations
Denition:
An m x n matrix is a rectangular array of numbers arranged in m rows
(horizontal lines) and n columns (vertical lines).
Example: A matrix with 3 rows and 2 columns : a 3 x 2 matrix (read a 3
by 2 matrix)
_
_
0 1
3 1
0 0
_
_
A matrix with 3 rows and 3 columns : a 3 x 3 matrix
_
_
1 2 3
5 100 2
2 2 1
_
_
In general an m x n matrix A has the form
_
_
_
_
_
_
_
_
a
11
a
12
... ... ... a
1n
a
21
a
22
... ... ... a
2n
... ... ... ... ... ...
... ... ... ... ... ...
... ... ... ... ... ...
a
m1
a
m2
... ... ... a
mn
_
_
_
_
_
_
_
_
Another denotation for matrix A is A= [a
ij
] for 1 i m and
1 j n. We denote matrices by capital boldface letter A, B, C,... .
The order of a matrix having m rows and n columns is mn. Then
a
ij
(1 i m; 1 j n) are called entries of the matrix A.
If m = n, we call A an n n square matrix and its main diagonal entries
are: a
11
, a
22
, ..., a
nn
.
Example
Let
_
_
0 1 1
2 0 1
23 0 1
_
_
.
It is an 3 3 square matrix and its main diagonal entries are: 0, 0, 1.
The order of this matrix is 9.
The following
_
_
0 1 1 0
2 0 1 5
23 0 1 6
_
_
is not a square matrix.
Remarks
Let A = [a
ij
] and B = [b
ij
] be m n matrices. Then A = B if and only if
a
ij
= b
ij
for all i , j .
A vector is a matrix with only one row or one column. We denote
vectors by lowercase boldface letter a, b, c ....
A row vector is of the form
a =
_
a
1
a
2
... ... a
n1
a
n
_
.
A column vector is of the form
a =
_
_
_
_
_
_
a
1
a
2
.
.
a
n
_
_
_
_
_
_
Addition of two matrices
Only matrices of the same number of rows and same number of
columns may be added by adding corresponding elements.
By denition:
_
_
_
_
_
_
_
_
a
11
a
12
... ... ... a
1n
a
21
a
22
... ... ... a
2n
... ... ... ... ... ...
... ... ... ... ... ...
... ... ... ... ... ...
a
m1
a
m2
... ... ... a
mn
_
_
_
_
_
_
_
_
+
_
_
_
_
_
_
_
_
b
11
b
12
... ... ... b
1n
b
21
b
22
... ... ... b
2n
... ... ... ... ... ...
... ... ... ... ... ...
... ... ... ... ... ...
b
m1
b
m2
... ... ... b
mn
_
_
_
_
_
_
_
_
=
_
_
_
_
_
_
_
_
a
11
+ b
11
a
12
+ b
12
... ... ... a
1n
+ b
1n
a
21
+ b
21
a
22
+ b
22
... ... ... a
2n
+ b
2n
... ... ... ... ... ...
... ... ... ... ... ...
... ... ... ... ... ...
a
m1
+ b
m1
a
m2
+ b
m2
... ... ... a
mn
+ b
mn
_
_
_
_
_
_
_
_
Example:
Let
A =
_
0 1
1 1
_
B =
_
1 1
2 1
_
.
Then
A +B =
_
1 2
3 0
_
.
Let
C =
_
1 0 1
0 5 100
_
Note that A +C or B +C is NOT dened.
Example:
Let
A =
_
0 1
1 1
_
B =
_
1 1
2 1
_
.
Then
A +B =
_
1 2
3 0
_
.
Let
C =
_
1 0 1
0 5 100
_
Note that A +C or B +C is NOT dened.
Scalar Multiplication of a Matrix
To multiply matrix A of order m x n by a scalar k, we multiply each entry
of A by k to obtain another matrix of the same order.
That is,
k
_
_
_
_
_
_
_
_
a
11
a
12
... ... ... a
1n
a
21
a
22
... ... ... a
2n
... ... ... ... ... ...
... ... ... ... ... ...
... ... ... ... ... ...
a
m1
a
m2
... ... ... a
mn
_
_
_
_
_
_
_
_
=
_
_
_
_
_
_
_
_
ka
11
ka
12
... ... ... ka
1n
ka
21
ka
22
... ... ... ka
2n
... ... ... ... ... ...
... ... ... ... ... ...
... ... ... ... ... ...
ka
m1
ka
m2
... ... ... ka
mn
_
_
_
_
_
_
_
_
Example
Let
A =
_
1 1 0
1 2 1
_
.
Then
2A =
_
2 2 0
2 4 2
_
.
5A =
_
5 5 0
5 10 5
_
1
2
A =
_
1/2 1/2 0
1/2 1 1/2
_
.
Multiplication of two Matrices
If A = [a
ij
] is an m x n-matrix and B = [b
ij
] an n x p-matrix, then the
product C = AB of the two matrices is an m x p-matrix dened by
C = [c
ij
] where c
ij
is given by
c
ij
= a
i 1
b
1j
+ a
i 2
b
2j
+ ... + a
in
b
nj
,
(the inner product of (a
i 1
, a
i 2
, ..., a
in
) and (b
1j
, b
2j
, ..., b
nj
)).
Note: (m x n-matrix) (n x p-matrix)=(m x p-matrix)
Example:
Let
A =
_
1 2
0 1
_
B =
_
2 0
1 1
_
C =
_
1 1 0
1 1 1
_
.
Then
AB =
_
1 2
0 1
__
2 0
1 1
_
=
_
4 2
1 1
_
BA =
_
2 0
1 1
__
1 2
0 1
_
=
_
2 4
1 3
_
NOTE: AB = BA.
AC =
_
2 0
1 1
__
1 1 0
1 1 1
_
=
_
2 2 0
2 0 1
_
NOTE: We can not do: CA.
Ex:
32
..
_
_
1 0
2 1
3 1
_
_
24
..
_
1 2 2 1
2 3 1 0
_
=
34
..
_
_
1 2 2 1
4 7 5 2
5 9 7 3
_
_
Ex:
32
..
_
_
1 0
2 1
3 1
_
_
24
..
_
1 2 2 1
2 3 1 0
_
=
34
..
_
_
1 2 2 1
4 7 5 2
5 9 7 3
_
_
Ex:
32
..
_
_
1 0
2 1
3 1
_
_
24
..
_
1 2 2 1
2 3 1 0
_
=
34
..
_
_
1 2 2 1
4 7 5 2
5 9 7 3
_
_
Ex:
32
..
_
_
1 0
2 1
3 1
_
_
24
..
_
1 2 2 1
2 3 1 0
_
=
34
..
_
_
1 2 2 1
4 7 5 2
5 9 7 3
_
_
Properties
A +B = B +A
(A +B) +C = A + (B +C)
A +0 = A
A + (A) = 0
(AB)C = A(BC)
(A +B)C = AB +BC
C(A +B) = CA +CB
k(AB) = (kA)B = k(AB).
Identity matrix
The identity matrix or unit matrix of size n is the n-by-n square matrix
with ones on the main diagonal and zeros elsewhere. It is denoted by I
n
.
For example
I
1
= [1] I
2
=
_
1 0
0 1
_
I
3
=
_
_
1 0 0
0 1 0
0 0 1
_
_
I
4
=
_
_
_
_
1 0 0 0
0 1 0 0
0 0 1 0
0 0 0 1
_
_
_
_
, .....
I
n
is called the n n identity matrix.
Then, it is easy to see that
AI
n
= A, for any m n matrix A,
I
n
B = B for any n p matrix B.
II. Systems of Linear Equations
DEFINITION: (i) A linear system of m equations in n unknowns
x
1
, x
2
, ..., x
n
is a set of equations of the form
a
11
x
1
+ a
12
x
2
+ ... + a
1n
x
n
= b
1
a
21
x
1
+ a
22
x
2
+ ... + a
2n
x
n
= b
2
(1)
.................................. = ...
a
m1
x
1
+ a
m2
x
2
+ ... + a
mn
x
n
= b
m
.
(ii) A solution of the system (1) is a set of numbers x
1
, x
2
, ..., x
n
that
satises all m equations.
II. Systems of Linear Equations
DEFINITION: (i) A linear system of m equations in n unknowns
x
1
, x
2
, ..., x
n
is a set of equations of the form
a
11
x
1
+ a
12
x
2
+ ... + a
1n
x
n
= b
1
a
21
x
1
+ a
22
x
2
+ ... + a
2n
x
n
= b
2
(1)
.................................. = ...
a
m1
x
1
+ a
m2
x
2
+ ... + a
mn
x
n
= b
m
.
(ii) A solution of the system (1) is a set of numbers x
1
, x
2
, ..., x
n
that
satises all m equations.
Example
_
x + 2y = 0
2x + y = 1
and
_
x + 2y + z = 2
2x + y + z = 1
are linear systems.
The following is not a linear system
_
x + 2xy = 0
2x + y = 1
2. Systems of Linear Equations
DEFINITION:
The matrix form of the system (1) is
Ax = b,
where
A =
_
_
_
_
_
_
_
_
a
11
a
12
... ... a
1n
a
21
a
22
... ... a
2n
... ... ... ... ...
... ... ... ... ...
... ... ... ... ...
a
m1
a
m2
... ... a
mn
_
_
_
_
_
_
_
_
x =
_
_
_
_
_
_
_
_
x
1
x
2
.
.
.
x
n
_
_
_
_
_
_
_
_
b =
_
_
_
_
_
_
_
_
b
1
b
2
.
.
.
b
m
_
_
_
_
_
_
_
_
The matrix A is called the coecient matrix of the system (1).
The matrix

A =
_
_
_
_
_
_
_
_
a
11
a
12
... ... a
1n
b
1
a
21
a
22
... ... a
2n
b
2
... ... ... ... ... ...
... ... ... ... ... ...
... ... ... ... ... ...
a
m1
a
m2
... ... a
mn
b
m
_
_
_
_
_
_
_
_
is called the augmented matrix of the system (1).
Example
The matrix form of the system
_
x + 2y = 0
2x + y = 1
is
_
1 2
2 1
__
x
y
_
=
_
0
1
_
.
Furthermore, x =
2
3
; y =
1
3
is a solution of the given system.
DEFINITION:
We say that a matrix is in row echelon form if
- All nonzero rows (rows with at least one nonzero element) are above
any rows of all zeros, and
- The leading coecient (the rst nonzero number from the left, also
called the pivot) of a nonzero row is always strictly to the right of the
leading coecient of the row above it.
Example
The following matrices are in the row echelon form
_
_
_
_
1 3 1 4
0 1 3 5
0 0 18 36
0 0 0 0
_
_
_
_
_
_
1 3 1 4
0 1 3 5
0 0 0 36
_
_
However the matrix below is not in the row echelon form.
_
_
1 3 1 4
0 1 3 5
0 6 0 36
_
_
as the leading coecient of row 3 (that is 6) is not strictly to the right of
the leading coecient of row 2 (that is -1).
Example
The following matrices are in the row echelon form
_
_
_
_
1 3 1 4
0 1 3 5
0 0 18 36
0 0 0 0
_
_
_
_
_
_
1 3 1 4
0 1 3 5
0 0 0 36
_
_
However the matrix below is not in the row echelon form.
_
_
1 3 1 4
0 1 3 5
0 6 0 36
_
_
as the leading coecient of row 3 (that is 6) is not strictly to the right of
the leading coecient of row 2 (that is -1).
DEFINITION
A system of linear equations is said to be in row echelon form if its
augmented matrix is in the row echelon form.
Ex: The system
x
1
3x
2
+ x
3
= 4
x
2
+ 3x
3
= 5
2x
3
= 2
is in the row echelon form because its augmented matrix is
_
_
1 3 1 4
0 1 3 5
0 0 2 2
_
_
Remark: It is very easy to solve a linear system whose augmented matrix
is in the row echelon form.
Solving linear systems: Gaussian Elimination
Elementary Operations on a linear system
(a) Add a multiple of one equation to another
(b) Interchange two equations
(c) Multiply an equation by a nonzero constant.
Elementary operations on a linear system correspond to the
following
Elementary row operations on a matrix
(a) Add a multiple of one row to another
(b) Interchange two rows
(c) Multiply a row by a nonzero constant.
Example
Linear system Associated augmented matrix
_
x 3y = 4
2x 8y = 2
_
1 3 4
2 8 2
_
Adding -2 times the rst equation Adding -2 times the rst row to the
to the second equation second row
_
x 3y = 4
0x 2y = 10
_
1 3 4
0 2 10
_
The second equation gives y = 5 and replacing y with 5 into the rst
equation, we get x = 19.
Gaussian Elimination
Gaussian elimination is an algorithm for solving systems of linear
equations.
Algorithm overview:
The process of Gaussian elimination has two parts:
1. Reduce a given system to the row echelon form (using of elementary
row operations).
( Or equivalently, we reduce an augmented matrix to the row echelon
form using elementary row operations)
2. Use back substitution to nd solutions of the given system.
Solving linear systems
Linear system Associated augmented matrix
_
x 3y = 4
2x 8y = 2
_
1 3 4
2 8 2
_
Adding -2 times the rst equation Adding -2 times the rst row to the
to the second equation second row
_
x 3y = 4
0x 2y = 10
_
1 3 4
0 2 10
_
The second equation gives y = 5 and replacing y with 5 into the rst
equation, we get x = 19.