S Y B Sc Mathematics (SEC)

Linear Algebra for Data Science-II

Assignment

March 18, 2025

ne
Instructions:

Pu
1. Solve the assignment in a notebook and submit it on or before 12 April 2025.

ge
2. Oral examination will be conducted at the time of submission.

lle
Co
on
Part I: Moore-Penrose pseudo inverse and least square ap-
ss
gu

proximation
er
,F

1. Find the least squares solution of each of the following systems:

ics
at

(a) (c)
m
he
at

−x1 + x2 = 10
fM

x1 + x2 + x3 =4
2x1 + x2 = 5
to

−x1 + x2 + x3 =0
en

x1 − 2x2 = 20
−x2 + x3 =1
m

(b)
rt

x1 + x3 =2
pa

x1 + x2 = 3
De

2x1 − 3x2 = 1
0x1 + 0x2 = 2

2. For each of your solutions x̂ in Exercise 1:

(a) determine the projection p = Ax̂.
(b) calculate the residual r(x̂).


(c) verify that r(x̂) ∈ N AT .
3. For each of the following systems Ax = b, find all least squares solutions:
   
1 2 3
(a) A =  2 4 , b=
  2 
−1 −2 1

1
   

1 1 3 −2
(b) A =  −1 3 1  , b= 0 
1 2 4 8

4. For each of the systems in Exercise 3, determine the projection p of b onto R(A) and verify
that b − p is orthogonal to each of the column vectors of A.

5. (a) Find the best least squares fit by a linear function to the data
x -1 0 1 2
y 0 1 3 9
(b) Plot your linear function from part (a) along with the data on a coordinate system.

6. Find the best least squares fit to the data in Exercise 5 by a quadratic polynomial. Plot the
points x = −1, 0, 1, 2 for your function and sketch the graph.

ne
Pu
7. Given a collection of points (x1 , y1 ) , (x2 , y2 ) , . . ., (xn , yn ), let

ge
lle
T T
x = (xP1 , x2 , . . . , xn ) , y = (yP1 , y2 , . . . , yn )

Co
n n
x̄ = n1 i=1 xi , ȳ = n1 i=1 yi
on
and let y = c0 + c1 x be the linear function that gives the best least squares fit to the points.
ss
gu

Show that if x̄ = 0, then

er
,F

xT y
ics

c0 = ȳ and c1 =
at

xT x
m
he

8. The point (x̄, ȳ) is the center of mass for the collection of points in Exercise 7. Show that
at

the least squares line must pass through the center of mass. [Hint: Use a change of variables
fM

z = x − x̄ to translate the problem so that the new independent variable has mean 0.]
to

−1 T
9. Let A be an m × n matrix of rank n and let P = A AT A
en

A .
m

(a) Show that P b = b for every b ∈ R(A). Explain this property in terms of projections.
rt
pa

(b) If b ∈ R(A)⊥ , show that P b = 0.

De

(c) Give a geometric illustration of parts (a) and (b) if R(A) is a plane through the origin in
R3 .

Part II: Determinant

 
1. Let A = a1 a2 . . . aj−1 x aj+1 . . . an , where a1 , a2 , . . . , an , x ∈ Rn . Define a transforma-
tion T from Rn to R by
 
T (x) = det a1 a2 . . . aj−1 x aj+1 . . . an .

Then show that

(a) T (cx) = cT (x) for any c ∈ R and any x ∈ Rn .

2
 (b) T (x + x) = T (x) + T (y) for any x, y ∈ Rn .

2. Find the determinant of the following matrices:

1 5 4 3 2
0 8 5 9 0
(a) 0 7 0 0 0
3 9 6 5 4
0 8 0 6 0
5 5 6 6 7
4 0 4 4 0
(b) 3 0 0 3 0 .
0 0 0 2 0
5 6 7 1 8

ne
Pu
3. Show that the equation of the line in R2 through distinct points (x1 , y1 ) and (x2 , y2 ) can be

ge
written as

lle
Co
 
1 x y on
det  1 x1 y1  = 0
ss

1 x2 y 2
gu
er
,F

4. Find a 3×3 determinant equation similar to that in above Exercise that describes the equation
ics

of the line through (x1 , y1 ) with slope m.

at
m

5. Let J be the n × n matrix of all 1’s and consider A = bI + aJ; that is,
he
at

 
···
fM

a+b a a a
a a+b a ··· a
to

 
 
A= a a a+b ··· a
en

 

.. .. .. .. ..
m

 
. . . . .
rt

 
a a a ··· a + b
pa
De

Confirm that det A = (na + b)bn−1 as follows:

(a) Subtract row n from rows 1 to n−1 and explain why this does not change the determinant
of the matrix.
(b) With the resulting matrix from part (a), add column 1 to column n, then add column
2 to column n and so on, and explain why this does not change the determinant of the
matrix.
(c) Find the determinant of the resulting matrix from (b).

6. Let A be the original matrix given in Problem 5, and let

3
  
a a a ··· a

 a a+b a ··· a 

B=
 a a a+b ··· a 

 .. .. .. ... .. 
 . . . . 
a a a ··· a + b

Notice that A, B are nearly the same except for the (1, 1) entry.

(a) Compute the determinant of B.

(b) Use part (a) to prove the formula in Problem 5 by induction on the size of matrix A.

a b c
7. Find the determinants in the following, where d e f = 7.

ne
g h i

Pu
ge
a b c a b c

lle
(a) d e f (d) 8d 8e 8f

Co
3g 3h 3i ong h i
a b c a b c
ss
gu

(b) d + 3g e + 3h f + 3i (e) 2d + a 2e + b 2f + c
er

g h i g h i
,F

a+d b+e c+f g h i

ics

(c) d e f (f) a b c
at
m

g h i d e f
he
at

8. Use Cramer’s rule to compute the solutions of the following systems:

fM
to

(a) 5x1 + 7x2 = 3 3x1 − x2 = −4

en

2x1 + 4x2 = 1
m

(e) x1 + x2 = 2
rt

(b) 6x1 + x2 = 3 −5x1 + 4x3 = 0

pa
De

5x1 + 2x2 = 4 x2 − x3 = −1
(c) 3x1 − 2x2 = 3 (f) x1 + 3x2 + x3 = 8
−4x1 + 6x2 = −5 −x1 + 2x3 = 4
(d) −5x1 + 2x2 = 9 3x1 + x2 = 4

Part III: Eigenvalues and Eigenvectors

1. Find the eigenvalues and the corresponding eigenspaces for each of the following matrices:
     
3 2 6 −4 3 −1
(a) (b) (c)
4 1 3 −1 1 1

4
      
3 −8 1 2 1 2 0 0 0
(d)
2 3 (h)  0 3 1   0 2 0 0 
(k)  

1 1
 0 5 −1  0 0 3 0 
(e) 0 0 0 4
−2 3  
4 −5 1  
3 0 0 0
 
0 1 0 (i)  1 0 −1 
(f)  0 0 1   4 1 0 0 
0 1 −1 (l) 
 0

0 0 0 0 2 1 
    0 0 0 2
1 1 1 −2 0 1
(g)  0 2 1  (j)  1 0 −1 
0 0 1 0 1 −1

2. In each of the following, factor the matrix A into a product XDX −1 , where D is diagonal:
     

ne
0 1 2 −8 1 0 0
(a) A = (c) A =

Pu
1 0 1 −4 (e) A =  −2 1 3 
1 1 −1

ge
lle
   
2 2 1 1 2 −1

Co
 
5 6 (d) A =  0 1 2  (f) A =  2 4 −2 
(b) A =
on
−2 −2 0 0 −1 3 6 −3
ss
gu

3. For each of the matrices in Problem 1, use the XDX −1 factorization to compute A6 .
er
,F

4. For each of the nonsingular matrices in Problem 1, use the XDX −1 factorization to compute
ics

A−1 .
at
m
he

5. For each of the following, find a matrix B such that B 2 = A.

at

 
9 −5 3
fM

 
2 1
(a) A = (b) A =  0 4 3 
to

−2 −1
0 0 1
en
m
rt

Part IV: Symmatric Matrices

pa
De

1. Find the matrix of the quadratic form. Assume x is in R2 .

(a) 4x21 − 6x1 x2 + 5x22 (b) 5x21 + 4x1 x2
2. Find the matrix of the quadratic form. Assume x is in R2 .
(a) 7x21 + 18x1 x2 − 7x22 (b) 8x1 x2
 
1 2
3. Find the spectral decomposition of the matrix A = .
2 1
 
1 2 2
4. Find a real spectral decomposition of the matrix A =  2 1 2 .
2 2 1
5. Find the Cholesky decomposition of each of the following matrices.

5
      
4 3 20 4 5 9 −6 6
(a)
3 5 (d)  4 2 3  (f)  −6 5 −1 

2 −1
 5 3 5 6 −1 15
(b)
−1 1
     
12 4 3 4 2 0 9 −6 6
(c)  4 2 −1  (e) 2 10 −3 (g) A = −6 5 −1 .
3 −1 7 0 −3 5 6 −1 12

6. For each of the following matrices, compute the determinants of all the leading principal
submatrices and use them to determine whether the matrix is positive definite:
 
2 −1
(a)
−1 2
 
3 4

ne
(b)

Pu
4 2

ge
 
6 4 −2

lle
(c)  4 5 3 

Co
−2 3 6 on
 
4 2 1
ss
gu

(d)  2 3 −2 
er

1 −2 5
,F
ics

7. Let A be a 3×3 symmetric positive definite matrix and suppose that det (A1 ) = 3, det (A2 ) = 6,
at

and det (A3 ) = 8. What would the pivot elements be in the reduction of A to triangular form,
m

assuming that only row operation III is used in the reduction process?
he
at
fM

Part V: Singular Value Decomposition

to
en

1. Let
m
rt
pa

 
2 −1 0 0
De

 −1 2 −1 0 
A=
 0 −1

2 −1 
0 0 −1 2

(a) Compute the LU factorization of A.

(b) Explain why A must be positive definite.

2. For each of the following, factor the given matrix into a product LDLT , where L is lower
triangular with 1’s on the diagonal and D is a diagonal matrix:
 
4 2
(a)
2 10
 
9 −3
(b)
−3 2

6
  
16 8 4
(c)  8 6 0 
4 0 7
 
9 3 −6
(d)  3 4 1 
−6 1 9
3. In each case find the QR-factorization:
     
1 −1 1 1 1 1 1 0
(a) A =
−1 0  1 1 0   −1 0 1 
(c) A = 
 1 0
 (d) A =  .
0   0 1 1 
0 0 0 1 −1 0
 
1 2 0

ne
 

Pu
2 1 (e) 0 1
 1
(b) A =
1 1 1 0 1

ge
lle
Co
4. Find the solution x to the least squares problem Ax = b, given that A = QR in each of the
following:
on
ss
 1
√1
gu

1 0 0
  
√
2 2
1 1 
er

(a) Q =  √12 − √12 . 0 √


√ 
,F

0 0 (c) Q = 
 2 2
ics

0 √ 1 1 
√ 
at

2 2
 
m

0 0 0
 
1
he

 
1 1
at

R= , b= 1 
0 1
fM

   
1 1 1 √ 1
R =  0 1  , b =  √2 
to

0 0 − 2
en
m

(b)
rt

(d)
pa
De

 1
√1 1


1 0 0

2 2
0 2
 0 √1 − √1   1 0 √1 − 1 
2 2  Q =  21 2 2 

Q=  0 √1 √1 , 0 − 2 − 21 
√ 1 
2 2
  2
1
0 0 0 2
− √12 0 1
2
   
  1 1 1 0  
1 1 0  3   0 1 1  2
R =  0 1 1 . b =  R=  −2 
 0 0 1 , b =
 
 1 
0 0 1 0
2 0 0 0

5. Find the singular value decomposition of the following matrices:

7
      
1 −1 4 6 −3 1
−2 2  (e)
(a) 0 4 (h)  6 −2 
2 −1 6 −2
 

−2 0
 3 −3  
(b) 1 1
0 0 (f)  0 0 
(i)  0 1 
1 1

−3 0
 −1 1
(c)
0 −2 
7 1
 
3 2 2

(j) [Hint:
2 3 −2
 
2 −1 (g)  5 5 
(d) Work with AT .]
2 2 0 0

6. Suppose the factorization below is an SVD of a matrix A, with the entries in U and V rounded
to two decimal places.

ne
Pu
    
.40 −.78 .47 7.10 0 0 .30 −.51 −.81
A = .37 −.33 −.87   0 3.10 0   .76 .64 −.12 

ge
lle
−.84 −.52 −.16 0 0 0 .58 −.58 .58

Co
(a) What is the rank of A?
on
ss
(b) Use this decomposition of A, with no calculations, to write a basis for Col A and a basis
gu

for Nul A. [Hint: First write the columns of V .]

er
,F

7. Show that if A is square, then | det A| is the product of the singular values of A.
ics
at

8. Suppose A is square and invertible. Find a singular value decomposition of A−1 .

m
he
at
fM
to
en
m
rt
pa
De