5.

1
Chapter 5: Matrix approach to simple linear regression
analysis

You need to understand matrix algebra for multiple

regression! Fox’s Section 8.2 contains information about
how to use R for matrix algebra.

5.1 Matrices

What is a matrix?

“A matrix is a rectangular array of elements arranged in

rows and columns” (p. 176 of KNN)

Example:

Dimension – Size of matrix: # rows  # columns = rc

Example: 23

Symbolic representation of a matrix:

Example:

 2008 Christopher R. Bilder

 5.2
where aij is the row i and column j element of A

a11=1 from the above example

Notice that the matrix A is in bold. When bolding is not

possible (writing on a piece of paper or chalkboard), the
letter is underlined - A

a11 is often called the “(1,1) element” of A, a12 is called

the “(1,2) element” of A,…

Example: rc matrix

Example: Square matrix is rc where r=c

Example: HS and College GPA

 2008 Christopher R. Bilder

 5.3

The above 202 matrix contains the HS GPAs in

the second column.

Vector – a r1 (column vector) or 1c (row vector) matrix –

special case of a matrix

Example: Symbolic representation of a 31 column vector

Example: HS and College GPA

The above 201 vector contains the College GPAs.

 2008 Christopher R. Bilder
 5.4

Transpose: Interchange the rows and columns of a matrix or

vector

Example:

and

A is 23 and A is 32

Example: HS and College GPA

 2008 Christopher R. Bilder

 5.5
5.2 Matrix addition and subtraction

Add or subtract the corresponding elements of matrices

with the same dimension.

Example:

Suppose . Then

Example: Using R (basic_matrix_algebra.R)

> A<-matrix(data = c(1, 2, 3,
4, 5, 6), nrow = 2, ncol = 3, byrow =
TRUE)

> class(A)
[1] "matrix"

> B<-matrix(data = c(-1, 10, -1, 5, 5, 8), nrow = 2, ncol =

3, byrow = TRUE)

> A+B
[,1] [,2] [,3]
[1,] 0 12 2
[2,] 9 10 14

> A-B
[,1] [,2] [,3]
[1,] 2 -8 4
[2,] -1 0 -2

 2008 Christopher R. Bilder

 5.6
Notes:
1.Be careful with the byrow option. By default, this is set
to FALSE. Thus, the numbers would be entered into
the matrix by columns. For example,
> matrix(data = c(1, 2, 3, 4, 5, 6), nrow = 2, ncol = 3)
[,1] [,2] [,3]
[1,] 1 3 5
[2,] 2 4 6

2.The class of these objects is “matrix”.

3.A vector can be represented as a “matrix” class type or
a type of its own.
> y<-matrix(data = c(1,2,3), nrow = 3, ncol = 1, byrow
= TRUE)
> y
[,1]
[1,] 1
[2,] 2
[3,] 3
> class(y)
[1] "matrix"

> x<-c(1,2,3)
> x
[1] 1 2 3
> class(x)
[1] "numeric"
> is.vector(x)
[1] TRUE

This can present some confusion when vectors are

multiplied with other vectors or matrices because no
specific row or column dimensions are given. More on
this shortly.
 2008 Christopher R. Bilder
 5.7
4.A transpose of a matrix can be done using the t()
function. For example,
> t(A)
[,1] [,2]
[1,] 1 4
[2,] 2 5
[3,] 3 6

Example: Simple linear regression model

Yi=E(Yi) + i for i=1,…,n can be represented as

where

 2008 Christopher R. Bilder

 5.8
5.3 Matrix multiplication

Scalar - 11 matrix

Example: Matrix multiplied by a scalar

where c is a scalar

Let and c=2. Then .

Multiplying two matrices

Suppose you want to multiply the matrices A and B; i.e.,

AB or AB. In order to do this, you need the number of
columns of A to be the same as the number of rows as
B. For example, suppose A is 23 and B is 310. You
can multiply these matrices. However if B is 410
instead, these matrices could NOT be multiplied.

The resulting dimension of C=AB

1.The number of rows of A is the number of rows of C.
2.The number of columns of B is the number of rows of
C.
3.In other words, where the dimension of the
matrices are shown below them.

 2008 Christopher R. Bilder

 5.9
How to multiply two matrices – an example

Suppose . Notice that A

is 23 and B is 32 so C=AB can be done.

The “cross product” of the rows of A and the columns of

B are taken to form C

In the above example, D=BAAB where BA is:

 2008 Christopher R. Bilder

 5.10

In general for a 23 matrix times a 32 matrix:

Example: Using R (basic_matrix_algebra.R)

> A<-matrix(data = c(1, 2, 3, 4, 5, 6), nrow = 2, ncol =
3, byrow = TRUE)
> B<-matrix(data = c(3, 0, 1, 2, 0, 1), nrow = 3, ncol =
2, byrow = TRUE)

 2008 Christopher R. Bilder

 5.11
> C<-A%*%B
> D<-B%*%A

> C
[,1] [,2]
[1,] 5 7
[2,] 17 16

> D
[,1] [,2] [,3]
[1,] 3 6 9
[2,] 9 12 15
[3,] 4 5 6

> #What is A*B?

> A*B
Error in A * B : non-conformable arrays

Notes:
1.%*% is used for multiplying matrices and/or vectors
2.* means to perform elementwise multiplications. Here
is an example where this can be done:
> E<-A
> A*E
[,1] [,2] [,3]
[1,] 1 4 9
[2,] 16 25 36

The (i,i) elements of each matrix are multiplied

together.

3.Multiplying vectors with other vectors or matrices can

be confusing since no row or column dimensions are

 2008 Christopher R. Bilder

 5.12
given for a vector object. For example, suppose x =

, a 31 vector

> x<-c(1,2,3)
> x%*%x
[,1]
[1,] 14
> A%*%x
[,1]
[1,] 14
[2,] 32

How does R know that we want xx (11) instead of

xx (33) when we have not told R that x is 31?
Similarly, how does R know that Ax is 21? From the
R help for %*% in the Base package:

Multiplies two matrices, if they are conformable. If

one argument is a vector, it will be promoted to
either a row or column matrix to make the two
arguments conformable. If both are vectors it will
return the inner product.

An inner product produces a scalar value. If you

wanted xx (33), one can use the outer product %o%
> x%o%x #outer product
[,1] [,2] [,3]
[1,] 1 2 3
[2,] 2 4 6
[3,] 3 6 9
 2008 Christopher R. Bilder
 5.13

We will only need to use %*% in this class.

Example: HS and College GPA (HS_college_GPA_ch5.R)

and

Find XX, XY, and YY

> #Read in the data
> gpa<-read.table(file =
"C:\\chris\\UNL\\STAT870\\Chapter1\\gpa.txt",
header=TRUE, sep = "")
> head(gpa)
HS.GPA College.GPA
1 3.04 3.1
2 2.35 2.3
3 2.70 3.0
4 2.05 1.9
5 2.83 2.5
6 4.32 3.7

> X<-cbind(1, gpa$HS.GPA)

> Y<-gpa$College.GPA

> X
[,1] [,2]

 2008 Christopher R. Bilder

 5.14
[1,] 1 3.04
[2,] 1 2.35
[3,] 1 2.70
[4,] 1 2.05
[5,] 1 2.83
[6,] 1 4.32
[7,] 1 3.39
[8,] 1 2.32
[9,] 1 2.69
[10,] 1 0.83
[11,] 1 2.39
[12,] 1 3.65
[13,] 1 1.85
[14,] 1 3.83
[15,] 1 1.22
[16,] 1 1.48
[17,] 1 2.28
[18,] 1 4.00
[19,] 1 2.28
[20,] 1 1.88

> Y
[1] 3.1 2.3 3.0 1.9 2.5 3.7 3.4 2.6 2.8 1.6 2.0 2.9 2.3
3.2 1.8 1.4 2.0 3.8 2.2
[20] 1.6

> t(X)%*%X
[,1] [,2]
[1,] 20.00 51.3800
[2,] 51.38 148.4634

> t(X)%*%Y
[,1]
[1,] 50.100
[2,] 140.229

> t(Y)%*%Y
[,1]
[1,] 135.15

Notes:
 2008 Christopher R. Bilder
 5.15
1.The cbind() function combines items by “c”olumns.
Since 1 is only one element, it will replicate itself for
all elements that you are combining so that one full
matrix is formed. There is also a rbind() function that
combines by rows. Thus, rbind(a,b) forms a matrix
with a above b.

2.

3.

since x11=…=xn1=1
4.XX

5.Here’s another way to get the X matrix

 2008 Christopher R. Bilder
 5.16

> mod.fit<-lm(formula = College.GPA ~ HS.GPA, data =

gpa)
> model.matrix(object = mod.fit)
(Intercept) HS.GPA
1 1 3.04
2 1 2.35
3 1 2.70
4 1 2.05
5 1 2.83
6 1 4.32
7 1 3.39
8 1 2.32
9 1 2.69
10 1 0.83
11 1 2.39
12 1 3.65
13 1 1.85
14 1 3.83
15 1 1.22
16 1 1.48
17 1 2.28
18 1 4.00
19 1 2.28
20 1 1.88
attr(,"assign")
[1] 0 1

 2008 Christopher R. Bilder

 5.17
5.4 Special types of matrices

Symmetric matrix: If A=A, then A is symmetric.

Example:

Diagonal matrix: A square matrix whose “off-diagonal”

elements are 0.

Example:

Identity matrix: A diagonal matrix with 1’s on the diagonal.

Example:

Note that “I” (the letter I, not the number one) usually
denotes the identity matrix.

Vector and matrix of 1’s

A column vector of 1’s:

 2008 Christopher R. Bilder

 5.18

A matrix of 1’s:

Notes:
1.

2.

3.

Vector of 0’s:

 2008 Christopher R. Bilder

 5.19
5.5 Linear dependence and rank of matrix

Let . Think of each column of A as a

vector; i.e., A = [A1, A2, A3]. Note that 3A2=A3. This

means the columns of A are “linearly dependent.”

Formally, a set of column vectors are linearly dependent

if there exists constants 1, 2,…, n (not all zero) such
that 1A1+2A2+…+cAc=0. A set of column vectors are
linearly independent if 1A1+2A2+…+cAc=0 only for
1=2=…=c=0 .

The rank of a matrix is the maximum number of linearly

independent columns in the matrix.

rank(A)=2

 2008 Christopher R. Bilder

 5.20
5.6 Inverse of a matrix

Note that the inverse of a scalar, say b, is b-1. For

example, the inverse of b=3 is 3-1=1/3. Also, bb-1=1. In
matrix algebra, the inverse of a matrix is another matrix.
For example, the inverse of A is A-1, and AA-1=A-1A=I.
Note that A must be a square matrix.

Example:

Check:

Finding the inverse

For a general way, see a matrix algebra book (such as:

Kolman, 1988). For a 22 matrix, there is a simple

formula. Let . Then

Verify AA-1=I on your own.

Example: Using R (basic_matrix_algebra.R)

> A<-matrix(data = c(1, 2, 3, 4), nrow = 2, ncol = 2,
byrow = TRUE)
 2008 Christopher R. Bilder
 5.21
> solve(A)
[,1] [,2]
[1,] -2.0 1.0
[2,] 1.5 -0.5

> A%*%solve(A)
[,1] [,2]
[1,] 1 1.110223e-16
[2,] 0 1.000000e+00

> solve(A)%*%A
[,1] [,2]
[1,] 1.000000e+00 0
[2,] 1.110223e-16 1

> round(solve(A)%*%A, 2)
[,1] [,2]
[1,] 1 0
[2,] 0 1

The solve() function inverts a matrix in R. The

solve(A,b) function can also be used to “solve” for x in
Ax = b since A-1Ax = A-1b  Ix = A-1b  x = A-1b

Example: HS and College GPA (HS_college_GPA_ch5.R)

Remember that and

Find (XX)-1 and (XX)-1XY

 2008 Christopher R. Bilder
 5.22
> solve(t(X)%*%X)
[,1] [,2]
[1,] 0.4507584 -0.15599781
[2,] -0.1559978 0.06072316

> solve(t(X)%*%X) %*% t(X)%*%Y

[,1]
[1,] 0.7075776
[2,] 0.6996584

From previous output:

> mod.fit<-lm(formula = College.GPA ~ HS.GPA, data = gpa)
> mod.fit$coefficients
(Intercept) HS.GPA
0.7075776 0.6996584

Note that (XX)-1XY = !!!

> #Another way to get (X'X)^(-1)

> sum.fit<-summary(mod.fit)
> sum.fit$cov.unscaled
(Intercept) HS.GPA
(Intercept) 0.4507584 -0.15599781
HS.GPA -0.1559978 0.06072316

Read “Uses of Inverse Matrix” on p. 192 of KNN on your

own.

 2008 Christopher R. Bilder

 5.23
5.7 Some basic theorems of matrices

Read on your own.

5.8 Random vectors and matrices

A random vector or random matrix contains elements that

are random variables.

Example: Simple linear regression model

Yi=E(Yi) + i for i=1,…,n can be represented as

where

are random vectors.

Expectation of random vector or matrix: Find the expected

value of the individual elements

Example:

 2008 Christopher R. Bilder

 5.24

since we assume

i~N(0,2)

Variance-covariance matrix of a random vector

Let Z1 and Z2 be random variables. Remember that

Var(Z1)=E(Z1-1)2 where E(Z1)=1.

The covariance of Z1 and Z2 is defined as Cov(Z1,Z2) =

E[(Z1-1)(Z2-2)]. The covariance measures the
relationship between Z1 and Z2. See p. 4.26 of my
Chapter 4 STAT 380 notes for a more mathematical
explanation (http://www.chrisbilder.com/
stat380/schedule.htm). Note that the correlation
between Z1 and Z2 is

The “Pearson correlation coefficient” which is denoted by

r and estimates Corr(Z1, Z2)

 2008 Christopher R. Bilder

 5.25

Remember that

where X=Z1 and Y=Z2.

Notes:
 Cov(Z1,Z1)=E[(Z1-1)(Z1-1)]=E[(Z1-1)2]=Var(Z1)
 If Z1 and Z2 are independent, then Cov(Z1,Z2)=0.

A variance-covariance matrix (most often just called the

covariance matrix) is a matrix whose elements are the
variances and covariances of random variables.

Example: Let . The covariance matrix of Z is

Cov(Z) =

Note that Cov(Z1,Z2)=Cov(Z2,Z1) and Cov(Zi,Zi)=Var(Zi)

KNN denote Cov(Z) by 2{Z}. The notation that I am

using is much more prevalent and there is less chance
for confusion (like Z being multiplied by 2 in 2{Z}).

Example: Simple linear regression model

 2008 Christopher R. Bilder

 5.26

= 2I

since Cov(i,i)=0 (Remember that i ~ INDEPENDENT

N(0,2)).

What is Cov(Y)?

Note that all covariance matrices are symmetric!

Some basic results

Let W = AY where Y is a random vector and A is a

matrix of constants (no random variables in it).

The following results follow:

1.E(A) = A

Remember this is like saying E(3) = 3

 2008 Christopher R. Bilder
 5.27

2.E(W) = E(AY) = AE(Y)

Again, this is like saying E(3Y1)=3E(Y1)

3.Cov(W) = Cov(AY) = ACov(Y)A

You may have seen before that Var(aY1) = a2Var(Y1)

where a is a constant.

Example:

Let . Then .

2.

3.

 2008 Christopher R. Bilder

 5.28

 2008 Christopher R. Bilder

 5.29
5.9 Simple linear regression model in matrix terms

Yi=0+1Xi+i where i~ independent N(0,2) and i=1,…,n

The model can be rewritten as

Y1=0+1X1+1
Y2=0+1X2+2

Yn=0+1Xn+n

In matrix terms, let

Then Y = X + , which is

Note that E(Y) = E(X + ) = E(X) + E() = X since E() = 0

and X are constants.

 2008 Christopher R. Bilder

 5.30
5.10 Least squares estimation of regression parameters

From Chapter 1: The least squares method tries to find

the b0 and b1 such that SSE = (Y- )2 = (residual)2 is
minimized. Formulas for b0 and b1 are derived using
calculus.

It can be shown that b0 and b1 can be found from solving

the “normal equations”:

The normal equations can be rewritten as

where . In Section 5.3, it was shown that

and

XX .

Thus, can be written as

 2008 Christopher R. Bilder
 5.31

Suppose both sides of are multiplied by

(XX)-1. Then

Therefore, we have a way to find b using matrix algebra!

Note that

 2008 Christopher R. Bilder

 5.32

Example: HS and College GPA (HS_college_GPA_ch5.R)

Remember that and

Find . We already did this on p. 5.22.

> solve(t(X)%*%X) %*% t(X)%*%Y

[,1]
[1,] 0.7075776
[2,] 0.6996584
 2008 Christopher R. Bilder
 5.33

> #Using the formulation of solve for x in Ax = b

> solve(t(X)%*%X, t(X)%*%Y)
[,1]
[1,] 0.7075776
[2,] 0.6996584

From previous output:

> mod.fit<-lm(formula = College.GPA ~ HS.GPA, data = gpa)
> mod.fit$coefficients
(Intercept) HS.GPA
0.7075776 0.6996584

 2008 Christopher R. Bilder

 5.34
5.11 Fitted values and residuals

Let . Then since

Hat matrix: where H=X(XX)-1X

is the hat matrix

Why is this called the Hat matrix?

We will use this in Chapter 10 to measure the influence

of observations on the estimated regression line.

Residuals:

Let

Covariance matrix of the residuals:

Cov(e) = Cov(Y(I-H)) = (I-H)Cov(Y)(I-H)

 2008 Christopher R. Bilder
 5.35
Now, Cov(Y) = Cov(X + ) = Cov() since X are
constants. And, Cov() = 2I

Also, (I-H) = (I-H) and (I-H)(I-H) = (I-H). (I-H) is called a

symmetric “idempotent” matrix. You can use matrix
algebra to see this result on your own (replace H with
X(XX)-1X and multiply out).

Thus, Cov(e) = (I-H)2I(I-H) = 2(I-H).

The estimated covariance matrix of the residuals is then

. Note that KNN
would denote as s2{e}. The notation is
more predominantly used.

 2008 Christopher R. Bilder

 5.36
5.12 Analysis of variance results

Sums of Squares
From Chapters 1 and 2: ,

, and

These can be rewritten using matrices:

 2008 Christopher R. Bilder

 5.37

Example: HS and College GPA (HS_college_GPA_ch5.R)

Continuing the same program

> n<-length(Y) #Can not use rows here since R does not
know Y's dimension
> #Could use nrow(X) as well - X is a nx2
matrix
> b<-solve(t(X)%*%X) %*% t(X)%*%Y
> Y.hat<-X%*%b
> e<-Y-Y.hat
> H<-X%*%solve(t(X)%*%X)%*%t(X)

> J<-matrix(data = 1, nrow = n, ncol = n)

> #Notice that R will repeat 1 the correct number of
times to fill the matrix

> SSTO<-t(Y)%*%Y - 1/n%*%t(Y)%*%J%*%Y

> SSE<-t(e)%*%e
> MSE<-SSE/(n-nrow(b))

> #Notice how MSE uses * to do the multiplying since it is

a scalar. Also diag(n) creates an identity matrix.
Diag(c(1,2)) would create a 2x2 diagonal matrix with
elements 1 and 2 on the diagonal
> Cov.e<-MSE*(diag(n) - H) #Does not work!
Error in MSE * (diag(n) - H) : non-conformable arrays

> Cov.e<-as.numeric(MSE)*(diag(n) - H) #The as.numeric()

removes 1x1 matrix
meaning from MSE
> SSR = SSTO - SSE

> data.frame(X, Y, Y.hat, e)

X1 X2 Y Y.hat e
 2008 Christopher R. Bilder
 5.38
1 1 3.04 3.1 2.834539 0.26546091
2 1 2.35 2.3 2.351775 -0.05177482
3 1 2.70 3.0 2.596655 0.40334475
4 1 2.05 1.9 2.141877 -0.24187731
5 1 2.83 2.5 2.687611 -0.18761083
6 1 4.32 3.7 3.730102 -0.03010181
7 1 3.39 3.4 3.079420 0.32058048
8 1 2.32 2.6 2.330785 0.26921493
9 1 2.69 2.8 2.589659 0.21034134
10 1 0.83 1.6 1.288294 0.31170591
11 1 2.39 2.0 2.379761 -0.37976115
12 1 3.65 2.9 3.261331 -0.36133070
13 1 1.85 2.3 2.001946 0.29805437
14 1 3.83 3.2 3.387269 -0.18726921
15 1 1.22 1.8 1.561161 0.23883914
16 1 1.48 1.4 1.743072 -0.34307203
17 1 2.28 2.0 2.302799 -0.30279873
18 1 4.00 3.8 3.506211 0.29378887
19 1 2.28 2.2 2.302799 -0.10279873
20 1 1.88 1.6 2.022935 -0.42293538

> data.frame(n, SSTO, SSE, MSE, SSR)

n SSTO SSE MSE SSR
1 20 9.6495 1.587966 0.08822035 8.061534

> round(Cov.e[1:5, 1:5],6) #5x5 part of Cov(e)

[,1] [,2] [,3] [,4] [,5]
[1,] 0.082621 -0.003858 -0.004742 -0.003101 -0.005070
[2,] -0.003858 0.083552 -0.004257 -0.005020 -0.004105
[3,] -0.004742 -0.004257 0.083717 -0.004047 -0.004594
[4,] -0.003101 -0.005020 -0.004047 0.082366 -0.003685
[5,] -0.005070 -0.004105 -0.004594 -0.003685 0.083444

> #From past work

> data.frame(X = model.matrix(mod.fit), Y.hat =
mod.fit$fitted.values, e = mod.fit$residuals)
X..Intercept. X.HS.GPA Y.hat e
1 1 3.04 2.834539 0.26546091
2 1 2.35 2.351775 -0.05177482
3 1 2.70 2.596655 0.40334475
4 1 2.05 2.141877 -0.24187731
5 1 2.83 2.687611 -0.18761083
6 1 4.32 3.730102 -0.03010181
 2008 Christopher R. Bilder
 5.39
7 1 3.39 3.079420 0.32058048
8 1 2.32 2.330785 0.26921493
9 1 2.69 2.589659 0.21034134
10 1 0.83 1.288294 0.31170591
11 1 2.39 2.379761 -0.37976115
12 1 3.65 3.261331 -0.36133070
13 1 1.85 2.001946 0.29805437
14 1 3.83 3.387269 -0.18726921
15 1 1.22 1.561161 0.23883914
16 1 1.48 1.743072 -0.34307203
17 1 2.28 2.302799 -0.30279873
18 1 4.00 3.506211 0.29378887
19 1 2.28 2.302799 -0.10279873
20 1 1.88 2.022935 -0.42293538

> anova(mod.fit)
Analysis of Variance Table

Response: College.GPA
Df Sum Sq Mean Sq F value Pr(>F)
HS.GPA 1 8.0615 8.0615 91.38 1.779e-08 ***
Residuals 18 1.5880 0.0882
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' '
1

 2008 Christopher R. Bilder

 5.40
5.13 Inferences in Regression Analysis

Covariance matrix of b:

Why?
Cov(b) = Cov[(XX)-1XY]
= (XX)-1XCov(Y)[(XX)-1X]
= (XX)-1XCov(Y)X(XX)-1
= (XX)-1X2IX(XX)-1
= 2(XX)-1XX(XX)-1
= 2(XX)-1

Remember that for two matrices C and B, (CB) =

BC

Estimated covariance matrix of b:

 2008 Christopher R. Bilder

 5.41
Note that the is from Sections 2.1 and 2.2. The
is new.

Estimated variance used in the C.I. for E(Yh):

where Xh = [1, Xh]

Why?

Since b is a vector, I replaced Var() with Cov() (just

notation)

Estimated variance used in the P.I. for Yh(new):

Example: HS and College GPA (HS_college_GPA_ch5.R)

> cov.beta.hat<-as.numeric(MSE)*solve(t(X)%*%X)
> cov.beta.hat
[,1] [,2]
[1,] 0.03976606 -0.013762181
[2,] -0.01376218 0.005357019
 2008 Christopher R. Bilder
 5.42

> #From earlier

> sum.fit<-summary(mod.fit)
> sum.fit$cov.unscaled #Another way to get (X'X)^(-1)
(Intercept) HS.GPA
(Intercept) 0.4507584 -0.15599781
HS.GPA -0.1559978 0.06072316

> sum.fit$sigma^2 * sum.fit$cov.unscaled

(Intercept) HS.GPA
(Intercept) 0.03976606 -0.013762181
HS.GPA -0.01376218 0.005357019

> #One more way to get the covariance matrix

> vcov(mod.fit)
(Intercept) HS.GPA
(Intercept) 0.03976606 -0.013762181
HS.GPA -0.01376218 0.005357019

> #Find Var(Y^)

> X.h<-c(1,1.88)
> as.numeric(MSE)*X.h%*%solve(t(X)%*%X)%*%X.h
[,1]
[1,] 0.006954107

> #Find Var(Y-Y^)

> as.numeric(MSE)* (1+X.h%*%solve(t(X)%*%X)%*%X.h)
[,1]
[1,] 0.09517446

> #From HS_college_GPA_ch2.R

> n<-nrow(gpa)
> X.h<-1.88
> Y.hat.h<-as.numeric(mod.fit$coefficients[1] +
mod.fit$coefficients[2]*X.h)
#as.numeric just removes an unneeded name
> ssx<-var(gpa$HS.GPA)*(n-1) #SUM( (X_i - X_bar)^2 )
> X.bar<-mean(gpa$HS.GPA)
> MSE * (1/n + (X.h - X.bar)^2 / ssx) #Taken from the
C.I. formula
[,1]
[1,] 0.006954107
 2008 Christopher R. Bilder
 5.43
> MSE * (1 + 1/n + (X.h - X.bar)^2 / ssx) #Taken from
the P.I. formula
[,1]
[1,] 0.09517446

 2008 Christopher R. Bilder