A=LU factorization

Much earlier we looked at the idea of an elimination matrix as a matrix E

that, when multiplied by another matrix A, performs a row operation in A.
Now we want to use elimination matrices to rewrite A as the product of a
lower triangular matrix L and an upper triangle matrix U, such that A = LU.

No row exchanges

When we perform an LU decomposition, or when we factor A into LU, we

can’t use row exchanges on the matrix A.

If row exchanges are required to put A into reduced row-echelon form

(because A has a 0 in a pivot position), then we need to perform all those
row exchanges first. Once any necessary row exchanges are complete,
only then can we start the LU decomposition, which we’ll perform on the
new “row-exchanged” version of A.

How to change A into LU

Let’s imagine that we’re starting with a matrix A,

5 1 3
A= 4 2 5
8 6 4

399
and we want to perform row operations on A until we’ve turned A into an
upper triangular matrix. For instance, we would start to rewrite A as an
upper triangular matrix by first changing the value in the second row and
first column, A2,1. To do that, we’ll multiply A by an elimination matrix E2,1.

E2,1A = U

1 0 0 5 1 3 5 1 3
− 45 1 0 6
4 2 5 = 0 5
13
5
0 0 1 8 6 4 8 6 4

The second row of the elimination matrix will replace the second row of A
with R2 − (4/5)R1, which changes A2,1 from a 4 to a 0. Then the goal is to keep
applying elimination matrices to the left side of the equation, such that
each elimination matrix performs a row operation on A that moves A one
step closer to becoming the upper triangular matrix U.

Our next step would be to rewrite the 8 in U3,1 as a 0,

E3,1E2,1A = U

1 0 0 1 0 0 5 1 3
5 1 3 6 13
0 1 0 − 45 1 0 4 2 5 = 0 5 5
− 85 0 1 0 0 1 8 6 4 0 22
− 45
5

then to rewrite the 22/5 in U3,2 as a 0.

E3,2 E3,1E2,1A = U

400
 1 0 0 1 0 0 1 0 0 5 1 3
5 1 3 6 13
0 1 0 0 1 0 − 45 1 0 4 2 5 = 0 5 5
0 − 22
6
1 − 85 0 1 0 0 1 8 6 4 0 0 − 31
3

Now that U is an upper triangular matrix, we need to solve the equation

E3,2 E3,1E2,1A = U for A, which we can do by multiplying both sides by the
inverse of each elimination matrix.

−1 −1
E3,2 E3,2 E3,1E2,1A = E3,2 U

−1
E3,1E2,1A = E3,2 U

−1 −1 −1
E3,1 E3,1E2,1A = E3,1 E3,2 U

−1 −1
E2,1A = E3,1 E3,2 U

−1 −1 −1 −1
E2,1 E2,1A = E2,1 E3,1 E3,2 U

−1 −1 −1
A = E2,1 E3,1 E3,2 U

In other words, to solve E3,2 E3,1E2,1A = U for A, we move the entire group of
elimination matrices to the other side of the equation, in front of U,
reversing their order and inverting each one.

−1 −1 −1 5 1 3
5 1 3 1 0 0 1 0 0 1 0 0
0 65 13
4
4 2 5 = −5 1 0 0 1 0 0 1 0 5
8 6 4 0 0 1 − 85 0 1 0 − 22
6
1 0 0 − 31
3

401
With the equation in this form, let’s redefine the group of elimination
−1 −1 −1
matrices as L, so L = E2,1 E3,1 E3,2 . If we say that L is the product of these
−1 −1 −1
inverted elimination matrices, then we can rewrite A = E2,1 E3,1 E3,2 U as
A = LU, and all that’s left to do is calculate L.

Inverting the elimination matrices to find L

To calculate L, we don’t actually have to manually calculate the inverse of

each elimination matrix. Instead, we know that the inverse of each
elimination matrix can be found by changing the sign on the non-zero
entry below the main diagonal.

In other words, given

−1 −1 −1 5 1 3
5 1 3 1 0 0 1 0 0 1 0 0
0 65 13
4
4 2 5 = −5 1 0 0 1 0 0 1 0 5
8 6 4 0 0 1 − 85 0 1 0 − 22
6
1 0 0 − 31
3

we can get the inverses by changing −4/5 to 4/5, −8/5 to 8/5, and −22/6 to
22/6.

1 0 0 1 0 0 1 0 0 5 1 3
5 1 3
0 65 13
4 2 5 =
4
1 0 0 1 0 0 1 0 5
5 8 22
8 6 4 0 0 1 5
0 1 0 6
1 0 0 − 31
3

Then, to calculate the product of the three inverted elimination matrices,

we can simply combine the non-zero entries below the main diagonal into

402
one matrix. So we’ll take the 4/5 from the first matrix, the 8/5 from the
second matrix, and the 22/6 from the third matrix, and we’ll get

1 0 0 5 1 3
5 1 3 4
0 65 13
4 2 5 = 5
1 0 5
8 6 4 8 22
1 0 0 − 31
5 6 3

In this form, we see that L is a lower triangular matrix with all 1s along the
main diagonal, and that U is an upper triangular matrix, which is exactly
what we want. We’ve rewritten A as the product of the lower and upper
triangular matrices, decomposing A into LU.

Another method

We can also accomplish this LU decomposition in a different way. We

essentially set up the equation A = IA, where A is the matrix we’re
decomposing and I is the identity matrix, but with no entries filled in below
the main diagonal.

This half empty identity matrix will become L while the matrix A on the
right side of the equation becomes U. With this method, we’ll collect row
operations directly in L as we work to put A into reduced row-echelon
form.

A = LU

403
 5 1 3 1 0 0 5 1 3
[ ]
4 2 5 = 1 0 4 2 5
8 6 4 1 8 6 4

We only need to zero-out the entries in U that are below the main
diagonal. In this case, that means we’re working on the 4, 8, and 6. To zero-
out the 4 in U2,1, we need to subtract 4/5 of R1 from R2. Since that row
operation is R2 − (4/5)R1, 4/5 is the fraction we put into L2,1.

5 1 3 1 0 0 5 1 3
0 65
4 13
4 2 5 = 5
1 0 5
8 6 4 1 8 6 4

To zero-out the 8 in U3,1, we need to subtract 8/5 of R1 from R3. Since the
row operation is R3 − (8/5)R1, 8/5 is the fraction we put into L3,1.

1 0 0 5 1 3
5 1 3 4 6 13
4 2 5 = 5
1 0 0 5 5
8 6 4 8
1 0 22
− 45
5 5

Finally, to zero-out the 22/5 in U3,2, we need to subtract 22/6 of R2 from R3.
Since the row operation is R3 − (22/6)R2, 22/6 is the fraction we put into L3,2.

1 0 0 5 1 3
5 1 3 4
0 65 13
4 2 5 = 5
1 0 5
8 6 4 8 22
1 0 0
31
−3
5 6

404
With this method, it helps to always write the row operations with
subtraction as Ri − CRj. This way, the constant C is the value that goes
directly into L. If we write the row operation with addition as Ri + CRj, then
−C will be the value we put into L.

Let’s try an example with a larger matrix.

Example

Rewrite the matrix A in factored LU form.

1 3 4 0
9 9 0 7
A=
7 7 6 5
0 3 0 2

Let’s find the LU decomposition using both methods we outlined earlier.

We’ll start by finding the elimination matrices we need to apply to A in
order to put it into reduced row-echelon form.

E2,1A = U

1 0 0 0 1 3 4 0 1 3 4 0
−9 1 0 0 9 9 0 7 0 −18 −36 7
=
0 0 1 0 7 7 6 5 7 7 6 5
0 0 0 1 0 3 0 2 0 3 0 2

E3,1E2,1A = U

405
 1 0 0 0 1 0 0 0 1 3 4 0 1 3 4 0
0 1 0 0 −9 1 0 0 9 9 0 7 0 −18 −36 7
=
−7 0 1 0 0 0 1 0 7 7 6 5 0 −14 −22 5
0 0 0 1 0 0 0 1 0 3 0 2 0 3 0 2

E3,2 E3,1E2,1A = U

1 0 0 0 1 0 0 0 1 0 0 0 1 3 4 0
0 1 0 0 0 1 0 0 −9 1 0 0 9 9 0 7
0 − 14
18
1 0 −7 0 1 0 0 0 1 0 7 7 6 5
0 0 0 1 0 0 0 1 0 0 0 1 0 3 0 2

1 3 4 0
0 −18 −36 7
=
0 0 6 − 49
0 3 0 2

E4,2 E3,2 E3,1E2,1A = U

1 0 0 0 1 0 0 0 1 0 0 0 1 0 0 0
0 1 0 0 0 1 0 0 0 1 0 0 −9 1 0 0
0 0 1 0 0 − 14 1 0 −7 0 1 0 0 0 1 0
3 18
0 18
0 1 0 0 0 1 0 0 0 1 0 0 0 1

1 3 4 0
1 3 4 0 0 −18 −36 7
9 9 0 7
= 0 0 6 − 4
7 7 6 5 9
0 3 0 2 0 0 −6 196

E4,3E4,2 E3,2 E3,1E2,1A = U

406
 1 0 0 0 1 0 0 0 1 0 0 0 1 0 0 0
0 1 0 0 0 1 0 0 0 1 0 0 0 1 0 0
0 0 1 0 0 0 1 0 0 − 14 1 0 −7 0 1 0
3 18
0 0 1 1 0 18
0 1 0 0 0 1 0 0 0 1

1 3 4 0
1 0 0 0 1 3 4 0 0 −18 −36 7
−9 1 0 0 9 9 0 7
= 0 0 6 − 4
0 0 1 0 7 7 6 5 9
0 0 0 1 0 3 0 2 0 0 0 49
18

To solve this equation for A, we need to move all the elimination matrices
to the right side, reversing the order and inverting each one.
−1
−1 −1 1 0 0 0
1 3 4 0 1 0 0 0 1 0 0 0
9 9 0 7 −9 1 0 0 0 1 0 0 0 1 0 0
=
7 7 6 5 0 0 1 0 −7 0 1 0 0 − 14
18
1 0
0 3 0 2 0 0 0 1 0 0 0 1 0 0 0 1

1 0 0 0
−1
−1 1 3 4 0
1 0 0 0 0 −18 −36 7
0 1 0 0 0 1 0 0
0 0 1 0 0 0 1 0 0 0 6 − 49
3
0 18
0 1 0 0 1 1 0 0 0 49
18

To invert each elimination matrix, we’ll change the sign on the non-zero
entry below the main diagonal,

407
 1 3 4 0 1 0 0 0 1 0 0 0 1 0 0 0
9 9 0 7 9 1 0 0 0 1 0 0 0 1 0 0
=
7 7 6 5 0 0 1 0 7 0 1 0 0 79 1 0
0 3 0 2 0 0 0 1 0 0 0 1 0 0 0 1

1 0 0 0 1 3 4 0
1 0 0 0 0 −18 −36 7
0 1 0 0 0 1 0 0
0 0 1 0 0 0 1 0 0 0 6 − 49
0 − 16 0 1 0 0 −1 1 0 0 0 49
18

then we’ll consolidate all the non-zero entries into one matrix, L.

1 0 0 0 1 3 4 0
1 3 4 0 9 1 0 0 0 −18 −36 7
9 9 0 7
= 7 7 1 0 0 0 6 − 49
7 7 6 5 9
0 3 0 2 1
0 − 6 −1 1 0 0 0 49
18

This is the A = LU factorization, but let’s use the other method to double-
check our work, starting with setting up A = LU.

1 3 4 0 1 0 0 0 1 3 4 0
9 9 0 7 1 0 0 9 9 0 7
=
7 7 6 5 1 0 7 7 6 5
0 3 0 2 1 0 3 0 2

We only need to zero-out the entries in U that are below the main
diagonal. To zero-out the 9 in U2,1, we need to subtract 9 of R1 from R2.
Since that row operation is R2 − 9R1, we put 9 into L2,1.

408
 1 3 4 0 1 0 0 0 1 3 4 0
9 9 0 7 9 1 0 0 0 −18 −36 7
=
7 7 6 5 1 0 7 7 6 5
0 3 0 2 1 0 3 0 2

To zero-out the 7 in U3,1, we need to subtract 7 of R1 from R3. Since the row
operation is R3 − 7R1, we put 7 into L3,1.

1 3 4 0 1 0 0 0 1 3 4 0
9 9 0 7 9 1 0 0 0 −18 −36 7
=
7 7 6 5 7 1 0 0 −14 −22 5
0 3 0 2 1 0 3 0 2

To zero-out the −14 in U3,2, we need to subtract 14/18 = 7/9 of R2 from R3.
Since the row operation is R3 − (7/9)R2, we put 7/9 into L3,2.

1 3 4 0 1 0 0 0 1 3 4 0
9 9 0 7 9 1 0 0 0 −18 −36 7
=
7 7 6 5 7 79 1 0 0 0 6 − 49
0 3 0 2 1 0 3 0 2

We don’t need to perform any row operation to keep the 0 in L4,1, so

1 3 4 0 1 0 0 0 1 3 4 0
9 9 0 7 9 1 0 0 0 −18 −36 7
=
7 7 6 5 7 79 1 0 0 0 6 − 49
0 3 0 2 0 1 0 3 0 2

409
To zero-out the 3 in U4,2, we need to add 3/18 = 1/6 of R2 to R4. The row
operation is R4 + (1/6)R2, but we want to see subtraction in the row
operation, so we write it as R4 − (−1/6)R2, then we put −1/6 into L4,2.

1 0 0 0 1 3 4 0
1 3 4 0 9 1 0 0 0 −18 −36 7
9 9 0 7
= 7 7 1 0 0 0 6 − 49
7 7 6 5 9
0 3 0 2 0 − 16 1 0 0 −6 19
6

To zero-out the −6 in U4,3, we need to add 1 of R3 to R4. The row operation

is R4 + 1R3, but we want to see subtraction in the row operation, so we
write it as R4 − (−1)R3, then we put −1 into L4,3.

1 0 0 0 1 3 4 0
1 3 4 0 9 1 0 0 0 −18 −36 7
9 9 0 7
= 7 7 1 0 0 0 6 − 49
7 7 6 5 9
0 3 0 2 1
0 − 6 −1 1 0 0 0 49
18

Using both methods, we’ve arrived at the same A = LU factorization.

Factoring out the pivots

Decomposing A into LU gives us the lower triangular matrix L with all 1s

along the main diagonal, and the upper triangular matrix U. But sometimes
we’ll want to rewrite U so that it also has all 1s along the main diagonal.

410
Let’s take the 3 × 3 matrix we looked at earlier. We had already factored it
into A = LU as

1 0 0 5 1 3
5 1 3 4
0 65 13
4 2 5 = 5
1 0 5
8 6 4 8 22
1 0 0 − 31
5 6 3

To change the entries along the main diagonal of U, we start by inserting

an identity matrix into the equation, changing it to A = LIU. This doesn’t
change the value of the equation at all, because multiplying by the identity
matrix is like multiplying by 1.

1 0 0 5 1 3
5 1 3 1 0 0
[0 0 1]
4 13 6
4 2 5 = 5
1 0 0 1 0 5
0 5
8 6 4 8 22
1 0 0 − 31
5 6 3

We want to change the entry in U1,1 from a 5 to a 1, which we can do by

dividing through the first row of U by 5, and putting the 5 into the identity
matrix.

1 3
1 0 0 1 5 5
5 1 3 4 5 0 0
4 2 5 = 5
1 0 0 1 0 0 6 13
5 5
8 6 4 8 22
1 0 0 1
5 6 0 0 − 31
3

To change the entry in U2,2 from a 6/5 to a 1, we’ll divide through the
second row of U by 6/5, putting the 6/5 into the identity matrix.

411
 1 3
1 0 0 5 0 0 1 5 5
5 1 3 4
4 2 5 = 5
1 0 0 65 0 0 1 13
6
8 6 4 8 22
1 0 0 1
5 6 0 0 − 31
3

Then finally, to change the entry in U3,3 from a −31/3 to a 1, we’ll divide
through the third row of U by −31/3, putting the −31/3 into the identity
matrix.

1 0 0 5 0 0 1 1 3
5 1 3 4 5 5
4 2 5 = 5
1 0 0 65 0 13
0 1 6
8 6 4 8 22
1 0 0 − 31
5 6 3 0 0 1

Let’s continue with the 4 × 4 matrix example from earlier, so that we can
see one more time how to change the entries along the main diagonal of
U.

Example

Rewrite the A = LU factorization so that U has only 1s along its main

diagonal.

1 0 0 0 1 3 4 0
1 3 4 0 9 1 0 0 0 −18 −36 7
9 9 0 7
= 7 7 1 0 0 0 6 − 49
7 7 6 5 9
0 3 0 2 1
0 − 6 −1 1 0 0 0 49
18

412
Let’s start by inserting an identity matrix into the equation.

1 0 0 0 1 3 4 0
1 3 4 0 9 1 0 0 1 0 0 0 0 −18 −36 7
9 9 0 7 0 1 0 0
= 7 7 1 0 0 0 6 − 49
7 7 6 5 9 0 0 1 0
0 3 0 2 1
0 − 6 −1 1 0 0 0 1 0 0 0 49
18

We’ll first pull out the −18 from U2,2, dividing the second row of U by −18.

1 0 0 0 1 3 4 0
1 3 4 0 9 1 0 0 1 0 0 0 7
0 1 2 − 18
9 9 0 7 0 −18 0 0
= 7 7 1 0
7 7 6 5 9 0 0 1 0 0 0 6 − 49
0 3 0 2 0 − 16 −1 1 0 0 0 1 49
0 0 0 18

Then we’ll pull out the 6 from U3,3, dividing the third row of U by 6.

1 0 0 0 1 3 4 0
1 3 4 0 9 1 0 0 1 0 0 0 7
0 1 2 − 18
9 9 0 7 0 −18 0 0
= 7 7 1 0 2
7 7 6 5 9 0 0 6 0 0 0 1 − 27
0 3 0 2 0 − 16 −1 1 0 0 0 1 49
0 0 0 18

Finally, we’ll pull out the 49/18 from U4,4, dividing the fourth row of U by
49/18.

413
 1 0 0 0 1 0 0 0 1 3 4 0
1 3 4 0 9 1 0 0 7
9 9 0 7 0 −18 0 0 0 1 2 − 18
= 7 7 1 0 0 0 6 0
7 7 6 5 9 2
0 0 1 − 27
49
0 3 0 2 1
0 − 6 −1 1 0 0 0 18
0 0 0 1

Now L is a lower triangular matrix, U is an upper triangular matrix, and

both matrices have only 1s along their main diagonals.

414