15.

053
z

February 15, 2007

The Geometry of Linear Programs

The simplex algorithm More properties of linear programs

Pentagonal prism

Overview of Lecture
z

Review of Geometry The Simplex Algorithm More on convexity RHS Sensitivity Analysis

Quotes of the Day

Geometry is not true, it is advantageous. Jules H. Poincare

I've always been passionate about geometry and the study of threedimensional forms. Erno Rubik
3

Review of Geometry
A set S is convex if for every two points in the set, the line segment joining the points is also in the set; that is, if p1, p2 S, then so is(1-)p1 + p2 for [0,1] Theorem. The feasible region of a linear program is convex.

y
3

p1

p2

Corner Points

A corner point of the feasible region is a point that is not the midpoint of two other points of the feasible region. All feasible LPs with non-negativity constraints have at least one corner point. If an LP is feasible, has nonnegativity constraints, and has an optimal solution, then there is a corner point that is optimal.

Solving for Corner Points

z z

In two dimensions, a corner point is the intersection of two equality constraints. In three dimensions, a corner point is the intersection of three constraints. (3 planes) 0 x 2
0 y 2

0z2 x -y +z3
x y
6

The red corner point is the intersection of three planes

x = 2
z = 2
x - y + z = 3
The unique solution is x = 2, y = 1, z = 2.

There are 3 Types of Linear Programs

3 2 1 1 2 3 4 Those with an optimal solution 3 2 1 1 2 3 4

Those with no feasible solution.

5 4 3 2 1 1 2 3 4 5 6
7

Isoprofit line

Those whose objective value is unbounded

The Simplex Method in Two Dimensions

Start at any feasible corner point. Move to an adjacent corner point with better objective value. Move along an edge of the feasible region. Continue until no adjacent corner point has a better objective value.
Max z = 3 x + 5 y

y
5 4 3 2 1

3 x + 5 y = 19

8 8

The Simplex Method Again

Start at any feasible corner point.

5
4
3
2
1

Move to an adjacent corner point with better

objective value. Move along an edge of the feasible region. Continue until no adjacent corner point has a better objective value.

The Simplex Method in 3 Dimensions

Start at any feasible corner point. Move to an adjacent corner point with better objective value. Move along an edge of the feasible region. Continue until no adjacent corner point has a better objective value.

Note: in two dimensions, the edges are the intersections of two constraints. The corner points are the intersection of three constraints.

Pentagonal prism

10

An Example of the Simplex Method in 3 Dimensions

y
y=2

Maximize y The number of iterations depends upon which edge is chosen at each iteration. x

y=1.7 y=1.5
y=1 y=0 y=.3 y=.1

y=.2

In practice, the simplex method is very efficient, even on very complex large scale LPs.

11

This is a twisted cube. Notice how the simplex method starting at the origin could move to the optimum in 1 step or pivot. It is also possible for the simplex method to take 7 pivots, thus visiting each corner point. Klee and Minty developed an example that is very similar that has n variables. It is possible that the simplex method would take 2^n 1 pivots on these examples, thus showing that the simplex method can take exponential time in the worst case. In practice, there may be many different edges that the simplex method can select at a given iteration. The speed in which the simplex method moves to the optimum depends on the choice of the edge.

The Simplex Method on Unbounded LPs

Maximize x

y
5 4 3 2 1 1 2 3 4 5 6

If the objective is unbounded from above, then the simplex method will move infinitely far along an edge.

12

y
3

Interior Point Algorithms for LP

3x + 5y = 16

x
13

There are a variety of algorithms that move within the interior of linear programs. These algorithms typically take far fewer iterations than the simplex algorithm and far more time per iteration for large problems. Sometimes interior point algorithms obtain answers quicker than the simplex algorithm. Often they are slower. Interior point algorithms were popularized by Karmarkar in 1984, who proved that the number of iterations is bounded by a polynomial in the dimension of the problem and in the number of bits needed to describe the coefficients. While interior point algorithms are ingenious and have practical import, they are also beyond the scope of 15.053, and will not be covered further.

Comments on Optimality Conditions

z

Linear programming produces both the optimal solution and the proof of optimality. (This is true for any number of variables, and even if many of the constraints are equality constraints.)
special among optimization problems very valuable The Gold Standard for optimization

For other optimization problems in the subject, we will settle for bounds from optimality
e.g., we will be happy if we can guarantee at most 10% from optimality

14

Convex Combinations
Suppose that p1, p2, , pk are all vectors (or points).

Let pk +1 = 1p1 + 2 p2 + ... + k pk .

We say that pk+1 is a convex combination of p1, , pk if the following are true:

1 + 2 + ... + k = 1 and i 0 for i = 1 to k .

Suppose k = 2. What points are convex combinations of p1 and p2? p1 p2
15

More on Convex Combinations

What points can be represented as the convex combination of (0,0), (0, 4), and (3, 0)?

5 4 3 2 1

1 (0, 0) + 2 (0, 4) + 3 (3, 0)

6
16

Convex Combinations and Convex Hulls

The convex hull of points p1, , pk is the smallest convex region containing all of the points. It is also the set of all points that can be expressed as convex combinations of p1 to pk.
5 4 3 2 1

Note that the convex hull of points in 2 dimensions looks like an LP feasible region.
1 2 3 4 5 6
17

Convex Hulls in 3 dimensions

y

18

In Slide show mode, the figure is revealed as a cube that is partially cut off.

Representation Theorem
z

Theorem. Every bounded polyhedra (linear programming feasible region) can be represented as a convex hull of its corner points. Theorem. The convex hull of a set of points is a bounded linear programming feasible region.

Usually, we prefer to represent a linear program in terms of constraints. But there are times when it is useful to represent it as the convex combination of corner points.

19

Mental Break

What are the odds?

20

Sensitivity Analysis
z

Sensitivity analysis: Determining the marginal effect on the optimal objective function if we make small changes in the data.

In LP, we focus on two types of sensitivity analysis that are very useful and very easy for an LP package to compute

21

The revised DTC example

z = 3x + 5y 2x + x + x + x 3y 2y y

10
y 6 5 4 3

Gathering time Smoothing time Delivery Time Demand: kits Demand: shields non-negativity
22

x, y 0

We could have used the original variable names of K and S, but it is simpler to use x and y since we usually think of the two axes as the x and y axis.

The optimal solution

In two dimensions, a corner point lies at the intersection of two lines. x + 2y = 6 2x + 3y = 10 smoothing time y 3 2 1 z = 3x + 5y 1 2 3 4 x +2y = 6
gathering time

2x + 4y = 12 2x + 3y = 10

2x +3y = 10 y=2

x = 2, z = 16

23

Its very useful that the corner point lies at the intersection of two lines. Then solving a system of equations with two variables and two equations will give the value of the corner point.

Varying the RHS

Suppose that we consider the problem in which gathering time is parameterized by G.

y
3 2 1

smoothing time

x +2y = 6
gathering time

2x +3y = 10 G

Let z(G) be the optimal objective value when gathering time is G, and all other data is unchanged. z(10) = 16

What is z(10)?
(derivative)
24

Its very useful that the corner point lies at the intersection of two lines. Then solving a system of equations with two variables and two equations will give the value of the corner point.

Computing the derivative

z(10 + ) z(10)

z '(10) = lim0

Key observation: if is small, then the optimum corner point of the problem will be the intersection of the smoothing time constraint and the gathering constraint.

That is, the constraints that define the corner point will not change.
25

On the new corner point

The solution value changes, but the structure of the solution does not change.

y
3 2 1

smoothing time

x +2y = 6
gathering time

2x +3y = G

26

Computing the New Corner Point

x + 2y = 6 2x + 3y = 10 +

y
3 2 1

smoothing time

x +2y = 6
gathering time

2x + 4y = 12 2x + 3y = 10 + 2x +3y = 10 + y=2-

z = 3x + 5y 1 2 3 4

x = 2 + 2 z = 16 +
27

The Shadow Price

z (10 + ) z (10) Note that z(10 + ) is linear in .

z '(10) = lim0

(16 + ) 16 =1 z '(10) = lim0

This is the shadow price of the gathering constraint. Note. We only needed that the corner point was the intersection of the gathering and smoothing constraint.
28

More on Shadow Prices

The shadow price of a constraint is the unit increase in the optimal objective value per unit increase in the RHS of the constraint. It is also a derivative.

Let p denote the shadow price. If the RHS of gathering increases from 10 to 10 + , then the objective value increases from 16 to 16 + p, that is, it increases by p.

29

Exercises
z

z(10) = 16; z(10) = 1 (The shadow price is 1)

Fact: z(11) = 16 + 1 = 17.
What is z(10.2)?
What is z(9.7)?
What is z(0)? (trick question)

30

Computing the Shadow Price of a Constraint

z

Step 1. Determine the binding constraints that determine the corner point. Step 2. Add to the RHS of the constraint whose shadow price we are computing. Step 3. Solve the system of equations. Step 4. Compute the increase in z when increases from 0 to 1 (i.e., compute the derivative)

x + 2y = 6 2x + 3y = 10

x + 2y = 6 2x + 3y = 10 +

x = 2 + 2 y=2- z(10 + ) = 16 + Shadow price = 1.

31

Bounds on RHS coefficients in Sensitivity Analysis

z

Recall that the optimum solution is a corner point, which in 2 dimensions is the solution of 2 equations in 2 variables, and the equations are the binding constraints. Compute the largest changes in the RHS coefficient so that all constraints remain satisfied.

32

What happens if the RHS changes by a lot?

y
3 2 1

2x +3y = G

33

More Sensitivity Analysis: Determining the Interval

x =2+2;
Maximize Profit Gathering time: Smoothing time: Delivery time: Slingshot demand: Shield demand: Non-negativity:

y= 2-
(in 10s) y K,S 0 So, -1 1 10 + 6 5 4 3

Constraint after substitution. z = 16 + 10 + 10 + 6 6 4+ 5 2 + 2 4 2- 3 2 + 2 0 2- 0

34

z = 3 x + 5 y 2 x + x + x + x 3 y 2 y y

Summary for changes in RHS coefficients

z z

Determine the binding constraints Determine the change in the corner point solution as a function of . Compute the largest and smallest values of so that the solution stays feasible. The shadow price is valid so long as the corner point solution remains optimal, which is so long as it is feasible.

35

And now, its time for ..

36

Who wants a piece of candy is not stored on the web.