MAT02B2 / MAT2B20

6.3 Gram-Schmidt Process; QR-Decomposition

Orthogonal, Orthonormal Sets

A set of two or more vectors in a real inner product space is said to be

orthogonal if all pairs of distinct vectors in the set are orthogonal. An or-
thogonal set in which each vector has norm 1 is said to be orthonormal.

In other words, the set

S = {v 1 , v 2 , . . . , v n }
of vectors from a real inner product space V is orthogonal if hv i , v j i = 0 for
all i, j ∈ {1, . . . , n} with i 6= j. If S is an orthogonal set and (in addition)
kv j k = 1 for each j ∈ {1, . . . , n}, then S is an orthonormal set.

Example 1
Let P2 have the inner product
Z 1
hp, qi = p(x)q(x)dx.
−1

Show that the set x, x2 , 1 − 53 x2 is an orthogonal set. Normalize the vec-



tors to obtain an orthonormal set of vectors.

Solution:
We check directly that the given vectors are orthogonal. We have
Z 1
2
x, x = (x)(x2 ) dx
−1
Z 1
= x3 dx
−1
4 1
 
x
=
4 −1
= 0,

1
   Z 1  
5 2 5 2
x, 1 − x = (x) 1 − x dx
3 −1 3
Z 1
5
= x − x3 dx
−1 3
 2 1
x 5 4
= − x
2 12 −1
= 0,

and
  Z 1  
5 5 2
2
x , 1 − x2 = 2
(x ) 1 − x dx
3 −1 3
Z 1
5
= x2 − x4 dx
−1 3
 3 1
x 5 5
= − x
3 15 −1
= 0.

(Note that you can streamline the calculations above if you know about even
and odd functions.) Hence, we conclude that x, x , 1 − 35 x2 is indeed an
 2

orthogonal set. To normalize a vector we simply multiply it by the reciprocal

of its norm which results in a unit vector, i.e. we must calculate

x x2 1 − 53 x2
, and .
kxk kx2 k 1 − 53 x2
p
Recall that kpk = hp, pi for any vector p ∈ P2 . Now,
Z 1
hx, xi = (x)(x) dx
−1
Z 1
= x2 dx
−1
 3 1
x
=
3 −1
2
= ,
3

2
 Z 1
2 2
x ,x = (x2 )(x2 ) dx
−1
Z 1
= x4 dx
−1
5 1
 
x
=
5 −1
2
= ,
5
and
  Z 1  
5 5 5 2 5 2
1 − x2 , 1 − x2 = 1− x 1 − x dx
3 3 −1 3 3
Z 1
10 25
= 1 − x2 + x4 dx
−1 3 9
 1
10 3 25 5
= x− x + x
9 45 −1
8
= .
9
Hence,
r r r √
2 2 5 8 2 2
kxk = , kx2 k = and 1 − x2 = = .
3 5 3 9 3

Consequently, (r r  )
3 5 2 3 5
x, x, √ 1 − x2
2 2 2 2 3
is an orthonormal set (note the we had to divide by the norms).

Theorem 0.1 (Theorem 6.3.1). If S = {v 1 , v 2 , . . . , v n } is an orthogonal set

of nonzero vectors in an inner product space, then S is linearly independent.

Proof. Suppose that

c1 v 1 + c2 v 2 + · · · + cn v n = 0

for some scalars c1 , . . . , cn . In order to prove that S is linearly independent,

we must show that c1 = c2 = · · · = cn = 0. Now, for any j ∈ {1, . . . , n}, we

3
have
0 = 0, v j
= hc1 v 1 + c2 v 2 + · · · + cn v n , v j i
= c1 hv 1 , v j i + c2 hv 2 , v j i + · · · + cn hv n , v j i
= cj hv j , v j i (since hv i , v j i = 0 for i 6= j).
Since v j 6= 0 it follows that hv j , v j i =
6 0. Hence,
0 = cj hv j , v j i ⇒ cj = 0.
Since j ∈ {1, . . . , n} was arbitrary, we may therefore conclude that
c1 = c2 = · · · = cn = 0,
establishing the result.
In an inner product space, a basis consisting of orthonormal vectors is called
an orthonormal basis, and a basis consisting or orthogonal vectors is called
an orthogonal basis.

Example 2
Explain why the set x, x2 , 1 − 53 x2 from the previous example is an orthog-


onal basis of P2 with inner product

Z 1
hp, qi = p(x)q(x)dx.
−1

Solution: 
Since the set x, x2 , 1 − 35 x2 is orthogonal, it follows from Theorem 0.1 that
it is a linearly independent set. Since P2 is 3-dimensional, we may therefore
conclude that x, x2 , 1 − 53 x2 is a basis. For interest sake we mention that
the set (r r  )
3 5 2 3 5 2
x, x , √ 1− x
2 2 2 2 3
is an orthonormal basis for P2 .

Coordinates Relative to Orthonormal Bases

The next result gives us a reason why orthogonal (or orthonormal) bases
are useful. It turns out that it is quite straightforward to find the (unique)
representation of a vector u as a linear combination of the vectors in an
orthogonal (or orthonormal) basis.

4
Theorem 0.2 (Theorem 6.3.2).

(a) If S = {v 1 , v 2 , . . . , v n } is an orthogonal basis for an inner product space

V , and if u is any vector in V , then

hu, v 1 i hu, v 2 i hu, v n i

u= v 1 + v 2 + · · · + vn.
kv 1 k2 kv 2 k2 kv n k2

(b) If S = {v 1 , v 2 , . . . , v n } is an orthonormal basis for an inner product

space V , then

u = hu, v 1 i v 1 + hu, v 2 i v 2 + · · · + hu, v n i v n .

Proof. Suppose that S = {v 1 , v 2 , . . . , v n } is an orthogonal basis for an inner

product space V and let u ∈ V . Since S is a basis for V it follows that

u = c1 v 1 + c2 v 2 + · · · + cn v n

for some (unique) scalars c1 , c2 , . . . , cn . Consequently, for any j ∈ {1, . . . , n},

we have

hu, v j i = hc1 v 1 + c2 v 2 + · · · + cn v n , v j i
= c1 hv 1 , v j i + c2 hv 2 , v j i + · · · + cn hv n , v j i
= cj hv j , v j i (since hv i , v j i = 0 for i 6= j),

and so,
hu, v j i hu, v j i
hu, v j i = cj hv j , v j i ⇒ cj = = .
hv j , v j i kv j k2
Hence,
hu, v 1 i hu, v 2 i hu, v n i
u= 2
v1 + 2
v2 + · · · + vn.
kv 1 k kv 2 k kv n k2
This establishes part (a). Part (b) now follows from part (a) if we observe
that
kv j k2 = 1
for each v j in an orthonormal basis S = {v 1 , v 2 , . . . , v n }.
Recall that the coordinate vector (w)B of a vector w in a vector space with
basis B = {x1 , x2 , . . . , xn } is obtained by looking at the (unique) representa-
tion of w as a linear combination of the vectors in B. Indeed, if

w = c1 x 1 + c2 x 2 + · · · + cn x n

5
for some scalars c1 , c2 , . . . , cn , then

(w)B = (c1 , c2 , . . . , cn ).

Thus, from Theorem 0.2 it follows that the coordinate vector of a vector u
in V relative to an orthogonal basis S = {v 1 , v 2 , . . . , v n } is
 
hu, v 1 i hu, v 2 i hu, v n i
(u)S = , ,..., ;
kv 1 k2 kv 2 k2 kv n k2

and relative to an orthonormal basis S = {v 1 , v 2 , . . . , v n } is

(u)S = (hu, v 1 i , hu, v 2 i , . . . , hu, v n i) .

Example 3
Assume R3 has the Euclidean inner product. Let v 1 = (0, 1, 0), v 2 = (1, 0, 1)
and v 3 = (1, 0, −1).

(a) Show that S = {v 1 , v 2 , v 3 } is an orthogonal basis for R3 .

(b) Find the coordinate vector (u)S where u = (1, 2, 3).

Solution:

(a) Since

v 1 • v 2 = (0)(1) + (1)(0) + (0)(1) = 0,

v 1 • v 3 = (0)(1) + (1)(0) + (0)(−1) = 0, and
v 2 • v 3 = (1)(1) + (0)(0) + (1)(−1) = 0,

we conclude that S is an orthogonal set. Since S consists of nonzero

vectors, we apply Theorem 0.1 and infer that S is linearly independent.
Since R3 is 3-dimensional and S consists of 3 vectors, we conclude that
S is indeed a basis.

(b) Notice that

u • v1 = (1)(0) + (2)(1) + (3)(0) = 2,

u • v2 = (1)(1) + (2)(0) + (3)(1) = 4,
u • v3 = (1)(1) + (2)(0) + (3)(−1) = −2,
v1 • v1 = (0)(0) + (1)(1) + (0)(0) = 1,
v2 • v2 = (1)(1) + (0)(0) + (1)(1) = 2, and
v3 • v3 = (1)(1) + (0)(0) + (−1)(−1) = 2.

6
 Thus, from the paragraph preceding this example we have
 
u • v1 u • v2 u • v3
(u)S = , , = (2, 2, −1) .
kv 1 k2 kv 2 k2 kv 3 k2

Indeed, as you may verify,

u = 2v 1 + 2v 2 − v 3 = 2(0, 1, 0) + 2(1, 0, 1) − (1, 0, −1) = (1, 2, 3).

Orthogonal Projections

Theorem 0.3 (Projection Theorem). If W is a finite-dimensional subspace

of an inner product space V , then every vector u in V can be expressed in
exactly one way as
u = w1 + w2 ,
where w1 is in W and w2 is in W ⊥ .

Proof. The proof is left as an exercise and appears as a problem in the tutorial
on §6.3. (Please work through the entire section before you attempt the proof
though.)
Let W be a finite-dimensional subspace of an inner product space V . Then
the unique vectors w1 ∈ W and w2 ∈ W ⊥ such that u = w1 + w2 are called
the orthogonal projection of u on W and the orthogonal projection
of u on W ⊥ , respectively, and are denoted by

w1 = projW u and w2 = projW ⊥ u.

Alternatively, w2 is also called the component of u orthogonal to W .

Moreover, since projW ⊥ u = u − projW u we can express u as follows:

u = projW u + projW ⊥ u
= projW u + (u − projW u).

The expression projW ⊥ u = u − projW u is extremely useful for calculation

if you already know projW u. To find projW u we typically use the following
theorem:

Theorem 0.4 (Theorem 6.3.4). Let W be a finite-dimensional subspace of

an inner product space V .

7
 (a) If {v 1 , v 2 , . . . , v r } is an orthogonal basis for W , and u is any vector in
V , then
hu, v 1 i hu, v 2 i hu, v r i
projW u = 2
v1 + 2
v2 + · · · + vr
kv 1 k kv 2 k kv r k2

(b) If {v 1 , v 2 , . . . , v r } is an orthonormal basis for W , and u is any vector

in V , then

projW u = hu, v 1 i v 1 + hu, v 2 i v 2 + · · · + hu, v r i v r

Proof. Let u be a vector in V and let {v 1 , v 2 , . . . , v r } be an orthogonal basis

for the subspace W of V . From Theorem 0.3 (the Projection Theorem) it
follows that u = w1 + w2 , where projW u = w1 ∈ W and w2 ∈ W ⊥ . Since
w1 ∈ W and {v 1 , v 2 , . . . , v r } is an orthogonal basis for W , it follows from
Theorem 0.2 that
hw1 , v 1 i hw1 , v 2 i hw1 , v r i
w1 = v 1 + v 2 + · · · + vr .
kv 1 k2 kv 2 k2 kv r k2

Thus, since w1 = u − w2 and w2 ∈ W ⊥ , we have

projW u = w1
hw1 , v 1 i hw1 , v 2 i hw1 , v r i
= 2
v1 + 2
v2 + · · · + vr
kv 1 k kv 2 k kv r k2
hu − w2 , v 1 i hu − w2 , v 2 i hu − w2 , v r i
= 2
v1 + 2
v2 + · · · + vr
kv 1 k kv 2 k kv r k2
   
hu, v 1 i hw2 , v 1 i hu, v 2 i hw2 , v 2 i
= v1 − v1 + v2 − v2 + · · ·
kv 1 k2 kv 1 k2 kv 2 k2 kv 2 k2
 
hu, v r i hw2 , v r i
··· + vr − vr
kv r k2 kv r k2
hu, v 1 i hu, v 2 i hu, v r i
= v 1 + v 2 + · · · + v r (since hw2 , v j i = 0).
kv 1 k2 kv 2 k2 kv r k2
This establishes part (a). Part (b) follows from part (a) since the norm of
every vector in an orthonormal set is 1.
Notice that according to Theorem 0.4, we only need an orthogonal (or or-
thonormal) basis for a space W in order to find the orthogonal projection of
any vector onto W . Indeed, the given formula only depends on the vector u
and the orthogonal (or orthonormal) basis {v 1 , . . . , v r } of W .

8
Example 4
   
0 1 1 0
Let W = span(U1 , U2 ) where U1 = and U2 = and define
−1 −1 1 −2
hA, Bi = a1 b1 + 2a2 b2 + 2a3 b3 + a4 b4
   
a a b 1 b2
for A = 1 2 and B = .
 a3 a4 b3 b4
1 0
If V = find projW V and projW ⊥ V .
0 1
Solution:
Since
hU1 , U2 i = (0)(1) + 2(1)(0) + 2(−1)(1) + (−1)(−2) = 0,
it follows that the set {U1 , U2 } is an orthogonal and hence linearly indepen-
dent subset of W . Moreover, since {U1 , U2 } spans W , it follows that {U1 , U2 }
is an orthogonal basis for W . To find projW V we can therefore utilize The-
orem 0.4 with the orthogonal basis {U1 , U2 }. Indeed, we have
hV, U1 i hV, U2 i
projW V = 2
U1 + U2
kU1 k kU2 k2
hV, U1 i hV, U2 i
= U1 + U2
hU1 , U1 i hU2 , U2 i
   
−1 0 1 −1 1 0
= +
5 −1 −1 7 1 −2
 1
− 7 − 15

= 2 17 .
35 35

Moreover,
projW ⊥ V = V − projW V
  1
− 7 − 15
 
1 0
= − 2 17
0 1 35 35
 8 1

= 7 5
2 18 .
− 35 35

Theorem 0.5 (Theorem 6.3.5). Every nonzero finite-dimensional inner prod-

uct space has an orthonormal basis.
The result above follows from the fact that any basis for an inner product
space can be converted into an orthonormal basis by using the following al-
gorithm.

9
 The Gram-Schmidt process

To convert a basis {u1 , u2 , . . . , ur } into an orthogonal basis {v 1 , v 2 , . . . , v r }

perform the following computations:

Step 1 v 1 = u1
hu2 ,v 1 i
Step 2 v 2 = u2 − kv 1 k2 1
v = u2 − projW1 u2 , where W1 = span(v 1 ).

Step 3 v 3 = u3 − hukv31,vk12i v 1 − hukv32,vk22i v 2 = u3 − projW2 u3 , where W2 = span(v 1 , v 2 ).

..
.

Step r v r = ur − hukvr1,vk12i v 1 − hukvr2,vk22i v 2 − · · · − hur ,v r−1 i

v
kv r−1 k2 r−1
= ur − projWr−1 ur ,
where Wr−1 = span(v 1 , v 2 , . . . , v r−1 ).

Optional To convert the orthogonal basis into an orthonormal basis {q 1 , q 2 , . . . , q r },

normalize the orthogonal basis vectors, i.e. let
vj
qj = for each j ∈ {1, . . . , r} .
kv j k

Remarks. Notice that each v j in the algorithm above is a linear combination

of u1 , . . . , ur (start at step 1 and then substitute for all the v j ’s in terms of
the ui ’s). Moreover, each uj is a linear combination of v 1 , . . . , v r (simply
solve for uj in each step). Consequently,

span (u1 , . . . , ur ) = span (v 1 , . . . , v r ) .

Furthermore, to see that the set {v 1 , v 2 , . . . , v r } is indeed orthogonal, simply

observe that for any k ∈ {2, . . . , r} we have
⊥
v k ∈ Wk−1 , where Wk−1 = span (v 1 , . . . , v k−1 ) .

Hence, hv i , v j i = 0 for i 6= j. Lastly, notice that {v 1 , v 2 , . . . , v r } is a basis

whenever {u1 , u2 , . . . , ur } is a basis. Indeed, {v 1 , v 2 , . . . , v r } is a set of r vec-
tors with the same span as the basis {u1 , u2 , . . . , ur } (for the r-dimensional
space).

Example 5
Let P2 have the inner product defined as follows: for p, q ∈ P2 ,

hp, qi = p(0)q(0) + p(1)q(1) + p(2)q(2).

10
Apply the Gram-Schmidt process to the basis
{1, 1 + x, 3x2 }
to obtain an orthogonal basis for P2 relative to the given inner product.

Solution:
Let
u1 = 1, u2 = 1 + x and u3 = 3x2 .
We apply the Gram-Schmidt procedure to the basis {u1 , u2 , u3 }:
v 1 = u1 .
Let W1 = span(v 1 ).

v 2 = u2 − projW1 u2 = u2 − hukv21,vk12i v 1 .
hu2 , v 1 i = (1)(1) + (2)(1) + (3)(1) = 6
kv 1 k2 = hv 1 , v 1 i = (1)(1) + (1)(1) + (1)(1) = 3.
Hence, v 2 = (1 + x) − 63 (1) = −1 + x.
Let W2 = span(v 1 , v 2 ).

v 3 = u3 − projW2 u3 = u3 − hukv31,vk12i v 1 − hukv32,vk22i v 2 .

hu3 , v 1 i = (0)(1) + (3)(1) + (12)(1) = 15
hu3 , v 2 i = (0)(−1) + (3)(0) + (12)(1) = 12
kv 2 k2 = hv 2 , v 2 i = (−1)(−1) + (0)(0) + (1)(1) = 2.
Hence, v 3 = 3x2 − 15 3
(1) − 12
2
(−1 + x) = 3x2 − 6x + 1.
Thus, an orthogonal basis for P2 with the given inner product is
{v 1 , v 2 , v 3 } = 1, −1 + x, 3x2 − 6x + 1 .


Example 6
Let M22 have
 the inner product
 defined
 as follows:
u1 u2 v1 v2
For U = and V = ,
u3 u4 v3 v4
hU, V i = u1 v1 + 2u2 v2 + 2u3 v3 + u4 v4 .
Apply the Gram-Schmidt process to the basis
       
1 0 1 1 1 1 1 1
, , , .
0 0 0 0 1 0 1 1
Solution:
Let        
1 0 1 1 1 1 1 1
U1 = , U2 = , U3 = , U4 = .
0 0 0 0 1 0 1 1
We apply the Gram-Schmidt process:

11
  
1 0
V1 = U1 =
0 0
Let W1 = span(V1 ).
 
V2 = U2 − projW1 U2 = U2 − hUkV21,Vk12i V1 .
hU2 , V1 i = 1(1) + 2(0) + 2(0) + 1(0) = 1
hV1 , V1 i =
 1(1)+ 2(0)
 + 2(0)
+ 1(0)
 = 1
1 1 1 0 0 1
So V2 = − 11 = .
0 0 0 0 0 0
Let W2 = span(V1 , V2 ).
 
hU3 ,V1 i hU3 ,V2 i
V3 = U3 − projW2 U3 = U3 − +
kV1 k2 1
V . kV2 k2 2
V
hU3 , V1 i = 1, hU3 , V2 i = 2
hV1 , V1 i =
 1, hV 2i =
2 , V 2     
1 1 1 1 0 2 0 1 0 0
So V3 = − 1 +2 = .
1 0 0 0 0 0 1 0
 
0 0
Continuing, we also get V4 = .
0 1

Thus, {V1 , V2 , V3 , V4 } is an orthogonal basis.

Recall that every linearly independent subset of a vector space can be en-
larged to a basis for the space. Moreover, notice that if the vectors

u1 , u2 , . . . , uj (j ≤ r)

in a basis {u1 , u2 , . . . , ur } are already mutually orthogonal, then the Gram-

Schmidt process will yield ui = v i in the first j steps (i.e. these vectors will
remain unchanged). Thus, if we combine the above observations, then we
obtain the following result:

Theorem 0.6 (Theorem 6.3.6). If W is a finite-dimensional inner product

space, then:

(a) Every orthogonal set of nonzero vectors in W can be enlarged to an

orthogonal basis for W .

(b) Every orthonormal set in W can be enlarged to an orthonormal basis

for W .

12
Example 7
Let R3 have the Euclidean inner product.
Extend {(1, −1, 1), (−1, 1, 2)} to an orthogonal basis for R3 .

Solution:
We need to find a nonzero vector which is orthogonal to both (1, −1, 1) and
(−1, 1, 2). One way to do this is as follows: Observe that

(1, −1, 1) • (a, b, c) = a − b + c = 0

and
(−1, 1, 2) • (a, b, c) = −a + b + 2c = 0
if and only if c = 0 and a = b. Consequently, any nonzero vector of the
form (a, a, 0) will do. In particular, the vector (1, 1, 0) is orthogonal to both
(1, −1, 1) and (1, −1, 2). Hence, since R3 is 3-dimensional and orthogonal
sets of nonzero vectors are automatically linearly independent, we conclude
that
{(1, −1, 1), (−1, 1, 2), (1, 1, 0)}
is the desired orthogonal basis.

QR-Decomposition

We immediately state and prove the main result in this subsection and then
proceed to describe an algorithm for obtaining the desired decomposition.
Theorem 0.7 (QR-Decomposition). If A is an m × n matrix with linearly
independent column vectors, then A can be factored as

A = QR

where Q is an m × n matrix with orthonormal column vectors, and R is an

n × n invertible upper triangular matrix.
Proof. Suppose that A is an m × n matrix with linearly independent column
vectors, say
{a1 , a2 , . . . , an },
and let {q 1 , q 2 , . . . , q n } be the resulting set of orthonormal vectors after ap-
plication of the Gram-Schmidt process (using the dot product as the given
inner product). If we look closely at how each step of the Gram-Schmidt
process works, we see that

ai , q j = 0 for all i ∈ {1, . . . , j − 1} .

13
Indeed, this follows from the observations that
vj ⊥
qj = ∈ Wj−1 ,
kv j k

where Wj−1 = span(v 1 , v 2 , . . . , v j−1 ), and a1 , . . . , aj−1 ∈ Wj−1 . Thus,

   
ha1 , q 1 i ha2 , q 1 i · · · han , q 1 i ha1 , q 1 i ha2 , q 1 i · · · han , q 1 i
 ha1 , q i ha2 , q i · · · han , q i   0 ha2 , q 2 i · · · han , q 2 i 
2 2 2 
R =  .. =
  
.. ... ..   .. .. ... .. 
 . . .   . . . 
ha1 , q n i ha2 , q n i · · · han , q n i 0 0 · · · han , q n i

(where the inner product is the usual dot product). Hence, with

Q = [q 1 q 2 · · · q n ]

we therefore have
 
ha1 , q 1 i ha2 , q 1 i ··· han , q 1 i
 ha1 , q i ha2 , q i ··· han , q 2 i 
2 2
QR = [q 1 q 2 · · · q n ]  .. ..  ,
 
.. ...
 . . . 
ha1 , q n i ha2 , q n i ··· han , q n i

which, by the properties of matrix multiplication and Theorem 0.2, is an

m × n matrix with jth-column

haj , q 1 i q 1 + haj , q 2 i q 2 + · · · + haj , q n i q n = aj .

Thus, A = QR as desired. Lastly we note that R is invertible since det(R) 6=

0. The latter follows from the fact that R is an upper triangular matrix
with only nonzero main diagonal entries. To see this, assume, for the sake of
reaching a contradiction, that

aj , q j = 0 for some j ∈ {1, . . . , n} .

Certainly j ≥ 2 since
1
ha1 , q 1 i = ha1 , a1 i =
6 0
ka1 k

(recall a1 , . . . , an are linearly independent and therefore nonzero.)

So haj , v j i = 0 for j ≥ 2. Thus, since j ≥ 2, we have
⊥
v j = aj − projWj−1 aj ∈ Wj−1 .

14
Consequently,
D E
hv j , v j i =aj − projWj−1 aj , v j
D E
= haj , v j i − projWj−1 aj , v j
= 0−0
= 0.

However, this gives v j = 0 which is a contradiction since {v 1 , . . . , v n } is a

basis. The result now follows.
The procedure:
Recall that A is an m × n matrix with linearly independent column vectors,
say
{a1 , a2 , . . . , an }.
1. We apply the Gram-Schmidt process (with respect to the dot product) to
this set to produce an orthonormal set {q 1 , q 2 , . . . , q n }. Please note that it
is important to not mix up the order of the vectors you obtain.

2. Take Q = [q 1 q 2 · · · q n ]
(Q is the matrix consisting of orthonormal column vectors).

3. Calculate R = QT A.
Alternatively, R can also be calculated directly as
 
ha1 , q 1 i ha2 , q 1 i · · · han , q 1 i
 0 ha2 , q 2 i · · · han , q 2 i 
R =  ..
 
.. ... .. 
 . . . 
0 0 ··· han , q n i

(where the inner product is the usual dot product).

Justification for R = QT A:
If Q is an m × n matrix with orthonormal column vectors, then

QT Q = In ,

where In is the n × n identity matrix. To see why this is the case, suppose

15
that Q = [q 1 q 2 · · · q n ], where {q 1 , q 2 , . . . , q n } is an orthonormal set. Then
 
q T1
q T 
 2
QT Q =  ..  [q 1 q 2 · · · q n ]
.
q Tn
 
q T1 q 1 q T1 q 2 · · · q T1 q n
q T q q T q · · · q T q 
 2 1 2 2 2 n
=  .. .. . . .. 
 . . . . 
T T T
qn q1 qn q2 · · · qn qn
 
q1 • q1 q1 • q2 · · · q1 • qn
q • q q • q · · · q • q 
 2 1 2 2 2 n
=  .. .. . . .. 
 . . . . 
qn • q1 qn • q2 · · · qn • qn
 
1 0 ··· 0
0 1 · · · 0
=  .. .. . . .. 
 
. . . .
0 0 ··· 1
= In ,

where we have used the orthonormality of the set {q 1 , q 2 , . . . , q n }, i.e.


1 if i = j
qi • qj =
6 j.
0 if i =

Thus, if A = QR, then

QT A = QT (QR) = (QT Q)R = In R = R,

which justifies our formula to calculate R.

Example 8
Find the QR-decomposition of the matrix where possible:
 
0 1 3
(a) A = 1 2 1
1 0 1
 
1 3 1
(b) B =
2 4 3

16
Solution:

(a) Since    
0 1 3 1 0 0
A = 1 2 1 → 0 1 0 ,
1 0 1 0 0 1
it follows that
          
0 1 3 0 1 3 c1 0
c1 1 + c2 2 + c3 1 = 1 2 1 c2 = 0
          
1 0 1 1 0 1 c3 0

only has the trivial solution i.e. c1 = c2 = c3 = 0. Hence, we conclude

that the column vectors of A are linearly independent; so A has a
QR-decomposition. We proceed to find Q and R. Let

a1 = (0, 1, 1), a2 = (1, 2, 0) and a3 = (3, 1, 1)

be the column vectors of A. We apply the Gram-Schmidt procedure to

the set {a1 , a2 , a3 } with respect to the dot product on R3 :

v 1 = a1 .
Let W1 = span(v 1 ).
v 2 = a2 − projW1 a2 = a2 − hakv21,vk12i v 1 .
ha2 , v 1 i = (1)(0) + (2)(1) + (0)(1) = 2
kv 1 k2 = hv 1 , v 1 i = (0)(0) + (1)(1) + (1)(1) = 2.
Hence, v 2 = (1, 2, 0) − 22 (0, 1, 1) = (1, 1, −1).
Let W2 = span(v 1 , v 2 ).
v 3 = a3 − projW2 a3 = a3 − hakv31,vk12i v 1 − hakv32,vk22i v 2 .
ha3 , v 1 i = (3)(0) + (1)(1) + (1)(1) = 2
ha3 , v 2 i = (3)(1) + (1)(1) + (1)(−1) = 3
kv 2 k2 = hv 2 , v 2 i = (1)(1) + (1)(1) + (−1)(−1) = 3.
Hence, v 3 = (3, 1, 1) − 22 (0, 1, 1) − 33 (1, 1, −1) = (2, −1, 1).

Thus, we obtain the orthonormal set of vectors {q 1 , q 2 , q 3 }, where

   
v1 1 1 v2 1 1 1
q1 = = 0, √ , √ , q 2 = = √ , √ , −√
kv 1 k 2 2 kv 2 k 3 3 3
and  
v3 2 1 1
q3 = = √ , −√ , √ .
kv 3 k 6 6 6

17
 Consequently, A = QR with
 
0 √1 √2
3 6
Q =  √12 √1 − √16 

3 
√1 − √13 √1
2 6

and
 
0 √1 √1
 
2 2 0 1 3
R = QT A =  √13 √1 − √13 
 1 2 1
 
3
√2 − √16 √1 1 0 1
6 6
√ √ √ 
2 √2 √2
=  0 3 √3 .
0 0 6

(b) The column vectors of B forms a set of 3 vectors in R2 . Since R2

is 2-dimensional, we may therefore conclude that the column vectors
of B must be linearly dependent. Hence, B does not have a QR-
decomposition.

18