Markov Chains, Linear Transformations, and

Eigenvectors
Minor in AI
February 20, 2025

The Mood of a City

Imagine a city where people are either happy or sad. Every day, some happy people
become sad, and some sad people become happy. Over time, if we track this transition, we
might see a steady pattern emerge—for example, 60% of people always remain happy,
and 40% become sad. Eventually, the city reaches a balance where the same proportion
of people are happy and sad every day, no matter how many times the process continues.
This idea of states transitioning over time is central to Markov Chains, which
are used to model real-world systems like weather predictions, stock market trends, and
even Google’s PageRank algorithm for ranking websites.

Markov Chains and Transition Matrices

A Markov Chain is a mathematical model where a system transitions between different

states according to fixed probabilities. It is defined by:

• States: Possible conditions (e.g., Happy, Sad).

• Transition Probabilities: The likelihood of moving from one state to another.

1
Minor in AI 2

• Markov Property: The future state depends only on the current state, not the
past.

Mathematically, a Markov chain is defined as:

P (Xn+1 |Xn , Xn−1 , . . . , X0 ) = P (Xn+1 |Xn )

Transition Matrix Representation

For a system with N states, the transition probabilities are stored in a transition matrix
M:
 
P11 P12 . . . P1N
 P21 P22 . . . P2N 
M =  ..
 
.. . . .. 
 . . . . 
PN 1 PN 2 . . . PN N
where:

• Pij is the probability of transitioning from state i to state j.

• Each row sums to 1, ensuring valid probability distributions:

N
X
Pij = 1
j=1

Consider a two-state system:

• S1 (Happy)

• S2 (Sad)

The transition probabilities are:

• P (S1 → S1 ) = 0.6, P (S1 → S2 ) = 0.4

• P (S2 → S1 ) = 0.3, P (S2 → S2 ) = 0.7

Thus, the transition matrix is:

 
0.6 0.4
M=
0.3 0.7
Each row represents a current state, and each column shows the probability of tran-
sitioning to another state.
Minor in AI 3

State Vector Representation

If the initial population has 50% happy and 50% sad people, we represent it as:
 
0.5
V0 =
0.5
After applying the transition matrix once:
  
0.6 0.4 0.5
V1 = M × V0 =
0.3 0.7 0.5
 
(0.6 × 0.5) + (0.4 × 0.5)
=
(0.3 × 0.5) + (0.7 × 0.5)
   
0.3 + 0.2 0.5
= =
0.15 + 0.35 0.5
Repeating this process multiple times will eventually lead to a steady state where
the values stop changing.

Python Implementation of Markov Chains

The following Python code demonstrates how a Markov chain evolves over time:
import numpy as np

# Define the Markov transition matrix

P = np . array ([[0.6 , 0.4] ,
[0.3 , 0.7]])

# Initial probability distribution

V = np . array ([0.5 , 0.5])

# Apply the transition matrix multiple times

for _ in range (5) :
V = np . dot (P , V )
print ( V ) # Print new state distribution

Observations from the Code

• The initial probability distribution V is defined as an equal split between happy
and sad people.

• The transition matrix P is applied to V multiple times using matrix multiplication.

• Each iteration updates V , bringing it closer to its steady-state distribution.

• The values in V stabilize after repeated iterations, indicating that the system has
reached a steady state.
Minor in AI 4

Real-World Applications
• Weather Prediction: Modeling the likelihood of sunny, rainy, or cloudy days.

• Stock Markets: Predicting future stock trends based on current performance.

• Google PageRank: Determining webpage importance based on link transitions.

Why Does the System Settle?

This happens because after repeated applications of M , the system reaches a fixed pro-
portion of happy and sad people. This final, unchanging state is called the stationary
distribution.

Convergence and Steady-State Distribution

A Markov chain reaches a steady-state distribution when:

P × V∞ = V∞
This means that after applying the transition matrix multiple times, the probabilities
remain constant. To find this steady state, we solve:
    
0.6 0.4 x x
=
0.3 0.7 y y
Expanding:

0.6x + 0.4y = x

0.3x + 0.7y = y
Rearranging both equations:

−0.4x + 0.4y = 0

0.3x − 0.3y = 0
Solving:

x=y
Since x + y = 1:

x = 0.4286, y = 0.5714
Thus, the steady-state distribution is:
 
0.4286
V∞ =
0.5714
Minor in AI 5

Python Code to Compute Steady-State Distribution

The following Python script calculates the steady-state vector using NumPy.
import numpy as np

# Define the transition matrix

P = np . array ([[0.6 , 0.4] ,
[0.3 , 0.7]])

# Solve for the steady - state distribution

eigvals , eigvecs = np . linalg . eig ( P . T ) # Transpose to solve
for right eigenvector

# Find the eigenvector corresponding to eigenvalue 1

steady_state_vector = eigvecs [: , np . isclose ( eigvals , 1) ].
flatten ()
steady_state_vector = steady_state_vector /
steady_state_vector . sum () # Normalize

print ( " Steady - State ␣ Distribution : " , steady_state_vector )

Observations from the Code

• We compute the eigenvalues and eigenvectors of P T (transpose of P ) since the
steady-state vector is the right eigenvector of the transition matrix corresponding
to λ = 1.

• The eigenvector is normalized so that its sum is 1, ensuring it represents a valid

probability distribution.

• The output confirms that the Markov chain converges to [0.4286, 0.5714], which
matches our earlier calculations.

Linear Transformations and Convergence

What is a Linear Transformation?
A linear transformation is a mathematical operation that changes a vector’s direction
or length while preserving certain properties.
Mathematically, it is written as:

Vnew = M × Vold
The steady-state distribution V∞ satisfies:

M × V∞ = V∞
which leads to solving:
Minor in AI 6

    
0.6 0.4 x x
=
0.3 0.7 y y
Expanding:

0.6x + 0.4y = x

0.3x + 0.7y = y
Rearranging both equations:

0.6x + 0.4y − x = 0 ⇒ −0.4x + 0.4y = 0

0.3x + 0.7y − y = 0 ⇒ 0.3x − 0.3y = 0

Solving:

−0.4x + 0.4y = 0 ⇒ x=y

Since x + y = 1, we get:

x = 0.4286, y = 0.5714
Thus, the steady-state distribution is:
 
0.4286
V∞ =
0.5714

Connection to Markov Chains

In our city’s mood example, each day’s emotional state is obtained by multiplying yes-
terday’s state vector with the transition matrix. Over time, this process leads to a fixed
vector, just like in Markov Chains.

Visualizing Convergence Using Python

The following Python script applies a transition matrix repeatedly to a vector and plots
the results.
import numpy as np
import matplotlib . pyplot as plt

# Define the transition matrix

M = np . array ([[0.6 , 0.3] , [0.4 , 0.7]])

# Define an initial vector

vector = np . array ([0.5 , 0.5])

# Store vectors for plotting

vectors = [ vector ]
Minor in AI 7

# Apply the matrix repeatedly

for _ in range (5) :
vector = np . dot (M , vector )
vectors . append ( vector )

# Plot the vectors

plt . figure ( figsize =(8 , 8) )
plt . axhline (0 , color = ’ gray ’ , linewidth =0.5)
plt . axvline (0 , color = ’ gray ’ , linewidth =0.5)
plt . grid ( color = ’ lightgray ’ , linestyle = ’ -- ’ , linewidth =0.5)
colors = [ ’ black ’ , ’ red ’ , ’ blue ’ , ’ green ’ , ’ purple ’ , ’ orange ’
]

for i , v in enumerate ( vectors ) :

plt . quiver (0 , 0 , v [0] , v [1] , angles = ’ xy ’ , scale_units = ’ xy
’ , scale =1 , color = colors [ i % len ( colors ) ] , label = f ’
Iteration ␣ { i } ’)

plt . xlim ( -1 , 1)
plt . ylim ( -1 , 1)
plt . legend ()
plt . title ( " Vector ␣ Convergence ␣ Under ␣ Matrix ␣ Multiplication " )
plt . xlabel ( "X - axis " )
plt . ylabel ( "Y - axis " )
plt . show ()
Minor in AI 8

Observations from the Code

• Each vector represents the probability distribution at each iteration.

• The colors in the graph show how the vectors evolve and converge.

• As more iterations occur, the vectors stabilize towards a final state.

Stretching and Rotating Shapes

- Think of a photo filter that changes colors in a predictable way. - Every time you
apply the filter, the colors adjust slightly until they reach a final tone. - This is similar
to how repeated matrix transformations can lead to a stable result over time.

Key Takeaways from the Class

Markov Property
The probability of transitioning to the next state depends only on the current state, not
past states.

Transition Matrix Representation

The system’s state transitions are represented using a matrix P , where each row represents
the probabilities of moving to different states. The sum of each row is always 1, ensuring
that all probabilities are valid:
 
P11 P12
P =
P21 P22

State Vector Representation

The probability distribution of the system at a given time is represented as a column
vector Vt . The new state is computed as:

Vt+1 = P × Vt
A Markov chain reaches a steady-state distribution when repeated applications of
the transition matrix no longer change the state vector:

P × V∞ = V∞

Solving for Steady-State

This requires solving a system of linear equations, ensuring the sum of probabilities equals
1:

P11 x + P12 y = x

P21 x + P22 y = y
Minor in AI 9

Importance of Steady-State Distributions

• The steady-state vector represents long-term probabilities of different states.

• It helps predict how a system behaves after a long time.

Linear Transformations and Their Effects

A linear transformation applies a matrix M to a vector V , modifying its direction and
magnitude:

V′ =M ×V

Effects of a Transformation
• Scaling: The vector’s length can increase or decrease.

• Rotation: The vector’s direction can change.

• Shear: The vector can be skewed.

A transition matrix in Markov chains is a special type of linear transformation,

where probabilities modify the state vector over time.

Repeated Transformations Lead to Stability

If a transformation is repeatedly applied, the vector eventually stabilizes into a dominant
direction.

Eigenvectors and Eigenvalues

An eigenvector of a matrix M is a vector that does not change direction when trans-
formed, only scaled by an eigenvalue λ:

M v = λv

Finding Eigenvalues and Eigenvectors

Eigenvalues λ are found by solving:

det(M − λI) = 0
Eigenvectors are obtained by solving:

(M − λI)v = 0

Connection to Markov Chains

• The steady-state vector of a Markov chain is the eigenvector corresponding
to eigenvalue λ = 1.

• This means the state vector remains unchanged after transformation.

Minor in AI 10

Key Concepts

Concept Key Insight

Markov Chains State transitions with probability matrices
Steady-State Distribution The system stabilizes over time
Linear Transformations Vectors change under matrix multiplica-
tion
Eigenvectors & Eigenvalues Special vectors remain in the same direc-
tion