Fun with Semirings
A functional pearl on the abuse of linear algebra
Stephen Dolan
Computer Laboratory
University of Cambridge
[email protected]September 25, 2013
Linear algebra is magic
If your problem can be expressed as vectors and matrices, it is
essentially already solved.
Linear algebra works with fields, like the real or complex numbers:
sets with a notion of addition, multiplication, subtraction and
division.
Stephen Dolan
Fun with Semirings
We dont have fields
CS has many structures with multiplication and addition:
conjunction and disjunction
intersection and union
sequencing and choice
product type and sum type
But very few with a sensible division or subtraction.
What we have are semirings, not fields.
Stephen Dolan
Fun with Semirings
Semirings
A closed semiring is a set with some notion of addition and
multiplication as well as a unary operation , where:
a+b =b+a
(+, 0) is a commutative monoid
a + (b + c) = (a + b) + c
a+0=a
a (b c) = (a b) c
(, 1) is a monoid, with zero
a1=1a=a
a0=0a=0
a (b + c) = a b + a c
distributes over +
(a + b) c = a c + b c
a = 1 + a a
closure operation
Daniel J. Lehmann, Algebraic Structures for Transitive Closure, 1977.
Stephen Dolan
Fun with Semirings
Closed semirings
A closed semiring has a closure operation , where
a = 1 + a a = 1 + a a
Intuitively, we can often think of closure as:
a = 1 + a + a2 + a3 + . . .
Stephen Dolan
Fun with Semirings
Closed semirings as a Haskell typeclass
i n f i x l 9 @.
i n f i x l 8 @+
class Semiring r where
zero , one : : r
closure : : r > r
(@+), (@. ) : : r > r > r
instance Semiring Bool where
zero = False
one = True
closure x = True
(@+) = ( | | )
(@. ) = (&&)
Stephen Dolan
Fun with Semirings
Adjacency matrices
Directed graphs are represented as matrices of Booleans.
G 2 gives the two-hop paths through G .
0 1 0
0 0 1
0 0 0
0 1 0
0 0 1
0 0 0
(AB)ij =
0 0 1
= 0 0 0
0 0 0
Aik Bkj
= k such that Aik Bkj
Stephen Dolan
Fun with Semirings
Closure of an adjacency matrix
The closure of an adjacency matrix gives us the reflexive transitive
closure of the graph.
0 1 0
0 0 1
0 0 0
1
=
2
1 1 1
= 0 1 1
0 0 1
A = 1 + A A
= 1 + A + A2 + A3 + . . .
Stephen Dolan
Fun with Semirings
A semiring of matrices
A matrix is represented by a list of lists of elements.
data Matrix a = Matrix [ [ a ] ]
instance Semiring a => Semiring (Matrix a) where
...
Matrix addition and multiplication is as normal, and Lehmann gives
an imperative algorithm for calculating the closure of a matrix.
Stephen Dolan
Fun with Semirings
Closure of a matrix
The correctness proof of the closure algorithm states:
A B
If M =
C D
A + B 0 C 0 B 0
then M =
C 0
where B 0 = A B, C 0 = C A and = D + C A B.
Stephen Dolan
Fun with Semirings
Block matrices
We can split a matrix into blocks, and join them back together.
type BlockMatrix a = (Matrix a , Matrix a ,
Matrix a , Matrix a)
msplit : : Matrix a > BlockMatrix a
mjoin : : BlockMatrix a > Matrix a
Stephen Dolan
Fun with Semirings
Closure of a matrix
The algorithm is imperative, but the correctness proof gives a
recursive functional implementation:
closure (Matrix [ [ x ] ] ) = Matrix [ [ closure x ] ]
closure m = mjoin
( f i r s t @+ top @. rest @. l e f t , top @. rest ,
rest @. l e f t ,
rest )
where
( f i r s t , top , l e f t , rest ) = msplit m
f i r s t = closure f i r s t
top = f i r s t @. top
l e f t = l e f t @. f i r s t
rest = closure ( rest @+ l e f t @. top)
Stephen Dolan
Fun with Semirings
Shortest distances in a graph
Distances form a semiring, with as addition and + as choosing
the shorter. The closure algorithm then finds shortest distances.
data ShortestDistance = Distance Int | Unreachable
instance Semiring ShortestDistance where
zero = Unreachable
one = Distance 0
closure x = one
x @+ Unreachable = x
Unreachable @+ x = x
Distance a @+ Distance b = Distance (min a b)
x @. Unreachable = Unreachable
Unreachable @. x = Unreachable
Distance a @. Distance b = Distance (a + b)
Stephen Dolan
Fun with Semirings
Shortest paths in a graph
We can also recover the actual path:
data ShortestPath n = Path Int [ ( n , n) ] | NoPath
instance Ord n => Semiring (ShortestPath n) where
zero = NoPath
one = Path 0 [ ]
closure x = one
x @+ NoPath = x
NoPath @+ x = x
Path a p @+ Path a p
| a < a
= Path a p
| a == a && p < p = Path a p
| otherwise
= Path a p
x @. NoPath = NoPath
NoPath @. x = NoPath
Path a p @. Path a p = Path (a + a ) (p ++ p )
Stephen Dolan
Fun with Semirings
Solving linear equations
If we have a linear equation like:
x =ax +b
then a b is a solution:
a b = (a a + 1) b
= a (a b) + b
If we have a system of linear equations like:
x1 = A11 x1 + A12 x2 + . . . A1n xn + B1
..
.
xn = An1 x1 + An2 x2 + . . . Ann xn + Bn
then A B is a solution (for a matrix A and vector B of
coefficients) which can be found using closure.
Stephen Dolan
Fun with Semirings
Regular expressions and state machines
A state machine can be described by a regular grammar:
qA
x
A xB
B yA + zC
qB
C 1
z
qC
The regular grammar is a system of linear equations, and the
regular expression describing it can be found by closure.
Stephen Dolan
Fun with Semirings
Solving linear equations
Reconstructing regular expressions
Solving equations in the free semiring rebuilds regular
expressions from a state machine.
Dataflow analysis
Solving equations in the semiring of sets of variables does live
variables analysis (among others).
Stephen Dolan
Fun with Semirings
Linear recurrences and power series
Suppose the next value in a sequence is a linear combination of
previous values:
F (0) = 0
F (1) = 1
F (n) = F (n 2) + F (n 1)
We represent these as formal power series:
F = x + x 2 + 2x 3 + 3x 4 + 5x 5 + 8x 6 . . .
Multiplying by x shifts the sequence one place, so:
F = 1 + (x 2 + x) F
Stephen Dolan
Fun with Semirings
Power series are a semiring
We represent power series as lists: a::p is a + px.
instance Semiring r => Semiring [ r ] where
zero = [ ]
one = [ one ]
Addition is pointwise:
[ ] @+ y = y
x @+ [ ] = x
(x : xs ) @+ (y : ys ) = (x @+ y ) : ( xs @+ ys )
Stephen Dolan
Fun with Semirings
Multiplying power series
Multiplying power series works like this:
(a + px)(b + qx) = ab + (aq + pb + pqx)x
In Haskell:
[ ] @. = [ ]
@. [ ] = [ ]
(a : p) @. (b : q) = (a @. b ) : (map (a @. ) q @+
map (@. b) p @+
( zero : ( p @. q)))
This is convolution, without needing indices.
M. Douglas McIlroy. Power series, power serious. Journal of Functional
Programming, 1999.
Stephen Dolan
Fun with Semirings
Closure of a power series
The closure of a + px must satisfy:
(a + px) = 1 + (a + px) (a + px)
This has a solution satisfying:
(a + px) = a (1 + px (a + px) )
which translates neatly into (lazy!) Haskell:
closure [ ] = one
closure (a : p) = r
where r = [ closure a ] @. (one : ( p @. r ))
Stephen Dolan
Fun with Semirings
Fibonacci, again
F = 1 + (x + x 2 )F
= (x + x 2 )
f i b = closure [0 ,1 ,1]
Any linear recurrence can be solved with closure.
Stephen Dolan
Fun with Semirings
Knapsacks
Suppose we are trying to fill our baggage allowance with:
Knuth books:
Haskell books:
Java books:
weight 10, value 100
weight 7, value 80
weight 9, value 3
The best value we can have with weight n is:
bestn = max(100 + bestn10 , 80 + bestn7 , 3 + bestn9 )
In the (max, +)-semiring, that reads:
bestn = 100 bestn10 + 80 bestn7 + 3 bestn9
which is a linear recurrence.
Stephen Dolan
Fun with Semirings
Thank you!
Questions?
Many problems are linear, for a suitable notion of linear.
[email protected]
Stephen Dolan
Fun with Semirings
Live variables analysis
Many dataflow analyses are just linear equations in a semiring.
This live variables analysis uses the semiring of sets of variables.
INA = OUTA {x}
INB = OUTB {x, y}
A x := 1
INC = OUTC {x}
B while x < y:
C
IND = OUTD {x}
OUTA = INB
x := x * 2
OUTB = INC IND
OUTC = INB
D return x
OUTD =
Stephen Dolan
Fun with Semirings
Petri nets
Timed event graphs (a form of Petri net with a notion of time) can
be viewed as linear systems, in the (max, +)-semiring
5
This transition fires for the
nth time after all of its
inputs have fired for the
nth time.
The nth token is available
from this place 5 time
units after then (n 3)th
token is available from its
input.
G. Cohen, P. Moller, J.P. Quadrat, M. Viot, Linear system theory for discrete
event systems, 1984.
Stephen Dolan
Fun with Semirings
