Gaussian Elimination in Systems of Equations
Topics covered
Gaussian Elimination in Systems of Equations
Topics covered
can be written as
Û Û
Ax = b
where
x
A = [ 2
5
3
6
4
7 ] ,
Û
x = y ,
z
Û
b = [ 12 ]
The system is abbreviated by writing
(1)
2
5
3
6
4
7 | 12
The matrix A is called the coefficient matrix. The 2 ≈ 4 matrix in (1) is called the
augmented matrix and is denoted A|b.
Gaussian elimination
Row ops on A|b amount to interchanging two equations or multiplying an equation by
a nonzero constant or adding a multiple of one equation to another. They do not
change the solution so they may be used to simplify the system. In particular,
performing row ops on A|b until A is in echelon form is called Gaussian elimination.
There are two possibilities (Fig 1).
1. The row ops produce a row of the form
(2) 0 0 0 0 |nonzero
1 0 2 4 2
0 1 3 5 0
0 0 0 0 6
and the variables are named x,y,z then the solution is x=2, y=5, z=6.
page 2 of Section 5.1
1 2 0 3 5
0 0 1 4 6
0 0 0 0 0
0 0 0 0 0
and the variables are named x,y,z,w then one way to write the solution is
x = 5 - 3w - 2y
z = 6 - 4w (w, y free)
x = 5 - 3s - 2t
y = t
z = 6 - 4s
w = s
→ →
Ax=b
example 1
Solve
2x1 + 4x2 - 2x3 + 8x4 + 4x5 = 6
3x1 + 6x2 + x3 + 12x4 - 2x5 = 1
9x1 + 18x2 + x3 + 36x4 + 38x5 = 0
solution Begin with
2 4 -2 8 4
6
3 6 1 12 -2 1
9 18 1 36 38
0
1
R1 = 2
R1
R2 = -3R1 + R2
R3 = -9R1 + R3
1
R2 = 4
R2
R1 = R2 + R1
R3 = -10R2 + R3
1
R3 = 40
R3
R2 = 2R3 + R2
to get
1 2 0 4 0 1
0 0 1 0 0 -94/40
0 0 0 0 1 -7/40
Choose x2 and x4, the variables corresponding to the cols without pivots, to be the
free variables (other choices are possible but I like these) and solve for
x1, x3, x5 in terms of them. A final sol is
7
x5 = - 40
94
x3 = - 40
x1 = 1 - 4x4 - 2x2
(x2, x4 free)
example 2
Suppose A is 4 ≈ 3 and Ax = b has infinitely many solutions. Consider the new
system Ax = c. Will it have infinitely many solutions also; i.e., what happens if
you change the righthand side of the system.
solution Since Ax = b has infinitely many sols, the echelon form of A must have at
least one col without a pivot and A|b must row op into something like this:
1 0 3 8 1 3 0 8 1 3 7 8
0 1 7 9 0 0 1 9 0 0 0 0
0 0 0 or 0 0 0 or 0 0 0
0 0 0
0 0 0 0 0 0 0 0 0 0 0 0
warning I'm not saying that the original system looks like this. But A|b must
row operate into something like this.
1 0 3 5
0 1 7 5
0 0 0 6
0 0 0 0
and have no solutions, and it's also possible for A|c to row op to
1 0 3 π
0 1 7 e
0 0 0 0
0 0 0 0
solving with unreduced echelon form and back substitution (much more efficient)
Row operate on the system so that the coeff matrix is in unreduced echelon form
(upper triangular form). Then starting with the last row, solve for the first
variable in each row and back substitute as you go along.
For example, if the row operations produce
1 2 3 5 7 2
0 0 4 6 8 6
0 0 0 0 2 6
0 0 0 0 0 0
then
x5 = 3 from the third row
6 - 8x5 - 6x4
x3 = from second row
4
6 - 8(3) - 6x4
= back substitute
4
-18 - 6x4
=
4
1
0
2
0
0
1
3
4
5
6
0
0
5
6
1 0
5
(a) 0 0 0 0 0 1 7 (b)
0
0
1
0
6
3
0 0 0 0 0 0 0 0 0
0
0 0 0 0 0 0 0
1 0 5
1 2 0 0 5
(c)
0
0
0
1
0
0
6
0
0
(d)
1
0
0
0
0
0
0
0 | 0
0 (e) 0
0
0
0
1
0
0
0
3
0
2. Let
u = (1,0,1,1)
v = (0,1,1,0)
w = (1,1,2,1)
y = (2,-1,1,2)
Decide if y is in the subspace spanned by u,v,w using
(a) ideas from Section 3.1
(b) ideas from this section
2x + 8y + 5z + 2u- 6v = 8
4y + 6z + 3u + 3v = 9 2x + y + z = 1
(b) 2x + 12y + 11z + 5u - 3v = 17 (c) 4x + y = -2
4y + 6z + 3u + 6v = 9 -2x + 2y + z = 7
page 6 of Section 5.1
1 2 0 3 5
0 0 1 4 6
0 0 0 0 0
0 0 0 0 0
Û Û
8. Suppose A is 4 ≈ 3. If Ax = b has one solution, what can you conclude about
Û Û
solutions to Ax = c.
Û Û
9. What can you conclude about sols to Ax = b if
(a) A is 3 ≈ 5 with rank 3
(b) A is 3 ≈ 5 with rank 2
(c) A is 5 ≈ 3 with rank 3
10. Let
1 2 1
M =
0 0 1
0 0 2
0 0 3
Û Û Û
For what vectors b is Mx = b consistent. In that consistent case, solve.
x1 y1 z1
=
x2 y2 z2
B
x3 y3 z3
x4 y4 z4
so that AB = I.
Can you find such a B. If so, how many.
Suggestion: Think about systems of equations and remember that A is fixed and B is
filled with unknowns.
page 1 of Section 5.2
x1 = 0, x2 = 0, ..., xn = 0.
A homog system will always have either one solution (the trivial one) or
infinitely many sols (the trivial one plus infinitely many others).
Û Û
Furthermore, the set of solutions to Mx = 0 is a subspace called the null space
of M.
If M has n cols (so that the system has n variables) and rank M = r then the null
space is an (n – r)-dim subspace of Rn; the dimension of the null space is the number of
free variables.
x1 = -5r - 2s + 7t
x2 = t
x3 = 6r + 3s
x4 = s
x5 = r
x6 = 0
In vector notation, the solution is
The set of sols is the set of all combinations of u,v,w so it is a subspace of R6.
Furthermore u,v,w are ind (look at their 2nd, 4th and 5th components to see that no
one can be a combination of the other). So the sols are a 3—dim subspace with basis
u,v,w. The dimension of the subspace matches the number of free variables.
Now that you know the sols are a subspace, here's another way to extract a basis
from the solution in (1) without rewriting it as (2). Assign values to the three
free variables so as to get 3 ind solutions. The easiest way to do this is to let
x2 = 1, x4 = 0, x5 = 0
x2 = 0, x4 = 1, x5 = 0
x2 = 0, x4 = 0, x5 = 1
example 1
Û Û
Suppose the system Mx = 0 row ops to
1 2 0 3 0
0 0 1 4 0
0 0 0 0 0
Then
x3 = -4x4
(3) x1 = -3x4 - 2x2
The sols are a 2—dim subspace of R4 since there are 2 free variables.
To pick a basis let x4 = 1, x2 = 0 in (3) to get solution
u = (-3,0,-4,1); and set x4 = 0, x2 = 1 in (3) to get solution v = (-2,1,0,0). Then u
and v are two ind solutions and are a basis for the subspace of solutions.
summary
Suppose A is 8 ≈ 10 (8 rows and 10 columns) with rank r.
The row space of A is an r—dim subspace of R10.
The col space of A is an r—dim subspace of R8.
Û Û
The null space of A (the set of sols to Ax = 0) is a (10-r)—dim subspace of R10.
example 2
page 3 of Section 5.2
Look at the set of vectors of the form (a,b,c,-a). In example 3 of Section 2.5, I
used two methods to show that the set is a subspace. And I found a basis for the
subspace. Here's a third way.
The set consists of all vectors (x1,x2,x3,x4) where x4 = -x1. So the set is the
solution of the homogeneous system
0 1 0 0
1 1 2 0
(b) M =
0 0 1 0
(a) M = 0 0 1 1 (c) M = [1 1 2]
1 1 2 0 0 0 0 1
0 0 0 0
3. Find an orthogonal basis for the subspace of solutions to
x + 2y + z - w = 0
2x + 5y + 3z - w = 0
2 0 0 0 2 3 4 0 0 0 0 0
(a) 0 0 0 0 (b) 4 6 8 0 (c) 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0
1 3 0 0 1 0 -2 0
(d) 0 0 0 0 (e) 0 1 -3 0
0 0 0 0 0 0 0 0
Û Û
5. Solve Ax = b where A is the 3 ≈ 5 zero matrix and
page 4 of Section 5.2
Û Û Û
(a) b = 0 (b) b ≠ 0
6. Suppose A has six cols, the cols are dep but the last five cols are ind. What can
you conclude about the number of solutions (and the number of free variables) to
Û Û Û Û Û Û
Ax = b where (a) b = 0 (b) b ≠ 0
7. Let u = (1,0,2), v = (1,1,4). Find all the vectors orthog to both u and v
(a) using geometry and calculus
(b) using the ideas of this section
9. Use ideas from this section to show that the set of points is a subspace and find
a basis.
(a) The set of points of the form (x1,..., x6) where x2 = -x3 + x5 and x4 = 2x3 + x5
(b) The set of points of the form (a,a,2a,2a,b)
0 2
11. Solve the system of equations 0 3 |00 (call the unknowns x and y) and find a
basis for the subspace of solutions.
page 1 of Section 5.3
perp
V V
point on •
the line point in
• the plane
→
• •0
point in
the plane
FIG 1
(2) The sum of the dimensions of V and VÚ is n (e.g., if V is a 6—dim sub space of
(5) The null space and the row space of a matrix are orthog complements.
proof of (4)
Û
Suppose V is spanned by u,v,w,p. To find VÚ, find all x orthog to the spanning
vectors by solving
Û Û Û Û Û Û Û Û
(•) u…x = 0, v…x = 0, w…x = 0, p…x = 0 .
Û Û
is written as a column So to find VÚ, solve Ax = 0, i.e., find the null space of A.
proof of (5)
This follows from (4) which shows that if a matrix A has rows u,v,w,p then the
null space of the matrix is the orthog complement of the space spanned by u,v,w,p.
example 1
Let
u = (0,1,0,1,0)
v = (0,0,1,0,2)
0
0
1
0
0
1
1
0
0
2 | 00
It's already in echelon form. The solution is
x3 = -2x5
x2 = -4x4
To find a basis for VÚ, set x1 = 1, x4 = 0, x5 = 0 to get
p = (1,0,0,0,0);
set x1 = 0, x4 = 1, x5 = 0 to get
q = (0,-1,0,1,0);
set x1 = 0, x4 = 0, x5 = 1 to get
r = (0,0,-2,0,1)
proof
The number of independent vectors orthog to u,v,w is the same as the dimension of
the orthog complement of the subspace spanned by u,v,w. The subspace spanned by u,v,w
has dimension 3 (since u,v,w are ind). By (2), the orthog complement has dimension
27. So that's the most independent vectors you can get which are orthog to u,v and w.
Similarly, the number of independent vectors orthog to u is the same as the
dimension of the orthog complement of the 1—dim subspace spanned by u. By (2), the
orthog complement has dimension 29.
mathematical catechism
question What does it mean to say that U is the orthogonal complement of a subspace
W of R8.
answer It means that U is the set of all vectors in R8 that are orthogonal to every
vector in W.
3. Let
1 2 0 3 0
A =
2 4 1 10 0
0 0 1 4 1
1 2 1 7 0
Find the dimension of and a basis for
2 4 1
4. Let A =
3 6 0
4 8 0
5 10 0
(a) Find a basis for the orthogonal complement of the column space.
(b) Continue from part (a) and find an orthogonal basis.
6. Let L1 and L2 be perpendicular lines through the origin in space. They are
subspaces since a line through the origin in R3 is a subspace. Are they orthogonal
complements.
8. No. the row space of a matrix and its null space are orthogonal complements. so a
vector in the null space has to be orthog to everything in the row space. But
(2,1,1) is not orthog to (3,1,2).
page 1 of Section 5.4
Throughout this section, matrices are square, and all systems of equations have the
same number of equations as unknowns.
Û Û
number of solutions to Mx = b
Û Û Û Û
(a) If M is invertible then Mx = b has one solution, namely x = M-1 b.
Û Û
(b) If M is not invertible then Mx = b has either no solutions or infinitely many
solutions.
proof of (b)
Suppose M is 3 ≈ 3 and is not invertible. Then M row ops into something like
1 2 0 1 0 2
0 0 1 or 0 1 3 etc.
0 0 0 0 0 0
1 0 2 5
0 1 3 6
0 0 0 7
1 0 2 8
0 1 3 9
0 0 0 0
in which case there are infinitely many solutions (because there is a free variable
corresponding to the column without a pivot). You can't get exactly one solution
because when the system is consistent there will always be at least one free
variable.
Û Û
number of solutions to Mx = 0
This is a special case of (a) and (b) above.
Û Û
Remember that a homogeneous system always has at least the trivial solution x = 0.
Û Û Û Û
(aa) If M is invertible then Mx = 0 has only the trivial solution x = 0.
Û Û
(bb) If M is not invertible then Mx = 0 has more (infinitely many more) than just
the trivial solution.
I'm going to add this to the invertible rule.
invertible rule.
Let M be n ≈ n.
The following are equivalent; i.e., either all are true or all are false.
Cramer's rule
Û Û
Look at Mx = b where M is square.
(1) If |M| = 0 then either the system is inconsistent or it has infinitely many
solutions (this is just a restatement of (b) from above).
(2) If |M| ≠ 0 then the system has one solution (this is a restatement of (a)).
Û Û
Furthermore, in case (2), the solution is x = M-1 b which boils down to
Û
replace col 1 of M with b and then take det
x1 =
det M
(•)
Û
replace col 2 of M with b and then take det
x2 =
det M
etc.
m11 b1 m13
m21 b2 m23
=
m31 b3 m33
det M
example 1
The system
2x + y + z = 0
x - y + 5z = 0
y - z = 4
2 1 1
1 -1 5 = -6 ≠ 0
0 1 -1
The solution is
page 3 of Section 5.4
0 1 1
0 -1 5
4 1 -1
x = = -4
-6
2 0 1
1 0 5
0 4 -1
y = = 6
-6
2 1 0
1 -1 0
0 1 4
z = = 2
-6
1 0
1. Suppose M is 3 ≈ 3 and M 2 = 0 .
3 0
2. Try Cramer's rule and if that doesn't help, solve some other way.
2x - 5y + 4z = 5 2y + 4z = 10 2y + 4z = 6
(a) x - 3y + 2z = 1 (b) x + y + 4z = 4 (c) x + y + 4z = 4
y + z = 7 x + 2z = 1 x + 2z = 1
3x + y = 3
2x - y = 5
Û 5 1
5. Let A be 3 ≈ 3 and let b = 9 , cÛ = 2
8 0
Û Û
Suppose Ax = b has no solutions.
What can you conclude about
Û Û
(a) the number of sols (and the number of free variables) to Ax = 0
Û Û Û Û
(b) the number of sols (and the number of free variables) to Ax = c where c ≠ 0
page 4 of Section 5.4
(c) |A|
(d) rank A
1/
√6 2/
√6 1/
√6
6. Let A = -2/
√5 1/
√5 0
1/
√30 2/
√30 -5/
√30
x 2
Without any agony, solve A y = 3 .
z 4