LINEAR

PROGRAMMING

Dhish Kumar Saxena

Professor
Department of Mechanical & Industrial Engineering
(Joint Faculty, Department of Computer Science)
IIT Roorkee

1
 Di erent forms of a Linear Programming Problem

General
Form

Max Min
Form Form

Standard
Form
2
ff
 Standard form of a Linear Programming Problem

The standard form demands: that particular solution vector X ≥ 0 ⟹ xi ≥ 0 i = 1,…, n

which (I) fulfils the System of Linear equations

and which (II) fulfils the objective: Minimize Z = c1x1 + c2 x2 + … + cn xn − Z0

The term Z0

∙ is a constant that may represent fixed costs or guaranteed benefits

∙ is preceeded with a negative sign: just for future convenience
∙ could be of any nature by sign Z0 < 0; Z0 = 0; Z0 > 0;

3
 Why Focus on the Standard form?
Feasibility of Conversion

The generic form; the Max form; and the Min Form
…………could all easily be converted to the Standard Form

Equivalence of Solutions
The solution of the generic form; the Max form; and the Min Form
..…will be the same as that obtained by their corresponding Standard Form

Access to the tools to solve S.O.L.E: AX=b

AX=b

4
 Conversion of an LPP into Standard form: Ra onale
Feasibility of Conversion
Maximize Z = 3x1 − 2x2 − x3 + x4 − 87
4x1 − x2 + x4 ≤ 6
Fixation
−7x1 + 8x2 + x3 ≥ 7 route
x1 + x2 + 4x4 = 12
x1, x2, x3 ≥ 0; x4 unrestricted
I
First: always fix the sign of variables II Then: fix the inequalities

x4 = x4a − x4b, where, x4a, x4b ≥ 0 : followed by the objective

Max Z = 3x1 − 2x2 − x3 + x4a − x4b − 87 Min Z = − 3x1 + 2x2 + x3 − x4a + x4b + 87
4x1 − x2 + x4a − x4b ≤ 6 4x1 − x2 + x4a − x4b + x5 =6
−7x1 + 8x2 + x3 ≥ 7 −7x1 + 8x2 + x3 − x6 = 7
x1 + x2 + 4x4a − 4x4b = 12 x1 + x2 + 4x4a − 4x4b = 12

x1, x2, x3, x4a, x4b ≥ 0 x1, x2, x3, x4a, x4b, x5, x6 ≥ 0
5
ti
Conversion of an LPP into Standard form: Ra onale
Equivalence of Solutions: Original (General;Max; or Min) vs Standard

Minimize 3x1 + 2x2 + 4x3 Minimize 3x1 + 2x2 + 4x3

g1(X) : 30x1 + 100x2 + 85x3 ≤ 2500 g˜1(X) : 30x1 + 100x2 + 85x3 + x4 = 2500
g2(X) : 6x1 + 2x2 + 3x3 ≥ 90 g˜2(X) : 6x1 + 2x2 + 3x3 − x5 = 90
x1, x2, x3 ≥ 0 x1, x2, x3, x4, x5 ≥ 0
If solution here is : (x* , x*, x*)
1 2 3
Then solution here is : (x* , x*, x*) ∪ (x*
1 2 3
, x*)
4 5
where x*
4
≥ 0 ⟹ g1(X ) in Original is honoured
x*
5
≥ 0 ⟹ g2(X ) in Original is honoured

that is x* , x* ≥ 0 are nothing but expressions of

4 5
Original Constraints

Promises Equivalence with Original, since the

Objective function f = ψ(x1, x2, x3) ≠ ϕ(x4, x5)

6
ti
 Conversion of an LPP into Standard form: Real U lity

} Establishes that you can

use the Standard form

But should you do every-

thing that you can do?

Well you should if it bears

utility

AX=b solved using the tools for System of Linear Equations lead to 3 possibilities

Ax=b: has no solution Ax=b: has a unique solution Ax=b: has multiple solutions

These 3 cases cover all possible “feasible sets” over which the optimum is to be found

ti
Conversion of an LPP into Standard form: Real U lity
Solution to AX=b: the principle
⎡ a11 a12 ⎤ ⎡ x1 ⎤ ⎡b1 ⎤
A X =b⇒⎢ ⎥ ⎢ ⎥ =⎢ ⎥
⎣ a21 a22 ⎦ ⎣ x2 ⎦ ⎣b2 ⎦
a11 x1 + a12 x2 = b1 ⎡ a11 ⎤ ⎡ a12 ⎤
x1 ⎢ ⎥ + x2 ⎢ ⎥ ≡ x1V1 + x2V2 = b
a21 x1 + a22 x2 = b2 ⎣ a21 ⎦ ⎣ a22 ⎦

Perspective-I: AX=b Perspective-II: AX=b

X: vector of scalar multiples (xi) of

Columns of A, combining to give the RHS
X: a point of intersection of hypersurfaces
‘b’ vector hypersurfaces (possible only
when b lies in the cone formed by V1 & V2)

ti
Conversion of an LPP into Standard form: Real U lity
Solution to AX=b: the principle

Ar⋆n Xn⋆1 = br⋆1 r: rows/constraints; n: columns/variables

∙ If r > n ⟹ more equations than variables: in general - no solution

∙ If r ≤ n ⟹ more variables than equations ⟹ the extra 'n-r' variables can be assigned
arbitrary values and the 'r' variables can be determined from 'r' equations

Out of the n variables (Xn), let :

∙ the r variables to be determined from r eqns. be called Basic variables XB

} Xn ≡ XB ∪ XN
∙ the n-r variables to be assigned values be called Nonbasic variables XN

⎡ x1 ⎤
⎡ a11 a12 a13 ⎤ ⎢ ⎥ ⎡ a11 ⎤ ⎡ a12 ⎤ ⎡ a13 ⎤
⎢a a ⎥ ⎢ x2 ⎥ = b ⟹ x1 ⎢ ⎥ + x2 ⎢ ⎥ + x3 ⎢ ⎥ = b ⟹ each variable xi ∈ X has a
⎣ 21 22 a23 ⎦ ⎣ a21 ⎦ ⎣ a22 ⎦ ⎣ a23 ⎦ Column of A reserved for itself
⎢⎣ x3 ⎥⎦

ti
 Conversion of an LPP into Standard form: Real U lity
Solution to AX=b: the principle
XB = {x1, x2} XB = {x1, x3} XB = {x2, x3}
⎡ a11 a12 ⎤ ⎡ x1 ⎤ ⎡ a13 ⎤ ⎡b1 ⎤ ⎡ a11 a13 ⎤ ⎡ x1 ⎤ ⎡ a12 ⎤ ⎡b1 ⎤ ⎡ a12 a13 ⎤ ⎡ x2 ⎤ ⎡ a11 ⎤ ⎡b1 ⎤
⎢ a a ⎥ ⎢ x ⎥ + ⎢ a ⎥ [x3 ] = ⎢b ⎥ ⎢ a a ⎥ ⎢ x ⎥ + ⎢ a ⎥ [x2 ] = ⎢b ⎥ ⎢ a a ⎥ ⎢ x ⎥ + ⎢ a ⎥ [x1 ] = ⎢b ⎥
⎣ 21 22 ⎦ ⎣ 2 ⎦ ⎣ 23 ⎦ ⎣ 2⎦ ⎣ 21 23 ⎦ ⎣ 3 ⎦ ⎣ 22 ⎦ ⎣ 2⎦ ⎣ 22 23 ⎦ ⎣ 3 ⎦ ⎣ 21 ⎦ ⎣ 2⎦

Ar⋆n Xn⋆1 = br⋆1 where Xn can be split in to 2 subclasses, namely XB and XN

⟹
∙ the Columns in A ≡ XB |r⋆1 will constitute a submatrix, say Br⋆r
∙ the Columns in A ≡ XN |(n−r)⋆1 will constitute a submatrix, say Nr⋆(n−r)

Ar⋆n Xn⋆1 = br⋆1 ⟹ Br⋆r XB |r⋆1 + Nr⋆(n−r) XN |(n−r)⋆1 = br⋆1

⟹

B XB = b − N XN ⟹ XB = B −1[b − N XN ]
⟹

∙ General Solution to AX = b
∙ Infinitely many General Solutions possible
due to Infinitely many possible values for XN
10

ti
Conversion of an LPP into Standard form: Real U lity
….a precursor

Infinitely many General Solutions can be reduced to finitely many by enforcing XN=0
∙ Basic Solution to AX = b

(r)
n
∙ r eqns in n variables ⟹ Max sets of XB =

(r)
n
⟹ Max possible Basic Solutions (BS) =

A Basic solution where XB = B −1b ≥ 0 is called a Basic "Feasible" Solution (BFS)

(r)
n
Since | BFS | ⊆ | BS | ⟹ Max possible Basic Feasible Solutions (BFS) ≤

11

ti
 Conversion of an LPP into Standard form: Real U lity
….a precursor

r=2; n=3 implies XB can have 3 different compositions BS or BFS =3 guaranteed?

⎡ a11 a12 ⎤ ⎡ x1 ⎤ ⎡ a13 ⎤ ⎡b1 ⎤

XB = {x1, x2} ⟹ ⎢ a a ⎥ ⎢ x ⎥ + ⎢ a ⎥ [x3 ] = ⎢b ⎥
⎣ 21 22 ⎦ ⎣ 2 ⎦ ⎣ 23 ⎦ ⎣ 2⎦

⎡ a11 a13 ⎤ ⎡ x1 ⎤ ⎡ a12 ⎤ ⎡b1 ⎤

XB = {x1, x3} ⟹ ⎢ ⎥ ⎢ x ⎥ + ⎢ a ⎥ [x2 ] = ⎢b ⎥
⎣ 21 23 ⎦ ⎣ 3 ⎦ ⎣ 22 ⎦
a a ⎣ 2⎦

⟹ ⎡ a12 a13 ⎤ ⎡ x2 ⎤ ⎡ a11 ⎤ ⎡b1 ⎤

XB = {x2, x3} ⎢ a a ⎥ ⎢ x ⎥ + ⎢ a ⎥ [x1 ] = ⎢b ⎥
⎣ 22 23 ⎦ ⎣ 3 ⎦ ⎣ 21 ⎦ ⎣ 2⎦
12

ti
 Conversion of an LPP into Standard form: Real U lity
XB = B −1b; XN = 0 ….a precursor: through Examples BFS: BS ≥ 0
Standard Form XB x1 x2 x3 x4
⎡ x1 ⎤ B
x1
x1 + x2 − x3 =2 ⎢ ⎥
⎡1 1 − 1 0 ⎤ ⎢ x2 ⎥ ⎡ 2 ⎤ x2
⎢1 1 0 1 ⎥ ⎢ x ⎥ = ⎢1 ⎥
x1 + x2 + x4 = 1 ⎣ ⎦ 3 ⎣ ⎦ x3
⎢ ⎥
x1, x2, x3, x4 ≥ 0 ⎣ x4 ⎦ x4
−1
⎡ x1 ⎤ ⎡1 1⎤ ⎡ x1 ⎤ ⎡1 1⎤ ⎡ 2 ⎤ 1 ⎡ 1 − 1⎤ BS BFS
Case − 1 : X B = ⎢ ⎥ B = ⎢ XB = ⎢ ⎥ = ⎢ =
⎣ x2 ⎦ ⎣1 1⎥⎦ ⎣ x2 ⎦ ⎣1
⎥ ⎢ ⎥ ⎢ ⎥
1⎦ ⎣1 ⎦ 1− 1 ⎣ −1 1 ⎦
Inverse not possible

−1
⎡ x1 ⎤ ⎡1 − 1⎤ ⎡ x1 ⎤ ⎡1 − 1⎤ ⎡ 2 ⎤ 1 ⎡ 0 1⎤ ⎡ 2 ⎤ ⎡ 0 1⎤ ⎡ 2 ⎤ ⎡ 1 ⎤ BS BFS
Case − 2 : X B = ⎢ ⎥ B = ⎢ ⎥ XB = ⎢ ⎥ = ⎢ ⎥ ⎢1 ⎥ = 0 + 1 ⎢ −1 1⎥ ⎢1 ⎥ = ⎢ −1 1⎥ ⎢1 ⎥ = ⎢ −1⎥
⎣ x3 ⎦ ⎣1 0 ⎦ ⎣ 3⎦ ⎣
x 1 0 ⎦ ⎣ ⎦ ⎣ ⎦⎣ ⎦ ⎣ ⎦⎣ ⎦ ⎣ ⎦
−1
⎡ x1 ⎤ ⎡1 0 ⎤ ⎡ x1 ⎤ ⎡1 0 ⎤ ⎡2 ⎤ 1 ⎡ 1 0 ⎤ ⎡2 ⎤ ⎡ 1 0 ⎤ ⎡2 ⎤ ⎡ 2 ⎤ BS BFS
Case − 3 : X B = ⎢ ⎥ B = ⎢ ⎥ XB = ⎢ ⎥ = ⎢ = = =
⎣ 4⎦
x ⎣1 1 ⎦ ⎣ x4 ⎦ ⎣1 1 ⎥⎦ ⎢⎣1 ⎥⎦ 1− 0 ⎢⎣ −1 1⎥⎦ ⎢⎣1 ⎥⎦ ⎢⎣ −1 1⎥⎦ ⎢⎣1 ⎥⎦ ⎢⎣ −1⎥⎦
−1
⎡ x2 ⎤ ⎡1 − 1⎤ ⎡ x2 ⎤ ⎡1 − 1⎤ ⎡ 2 ⎤ 1 ⎡ 0 1⎤ ⎡ 2 ⎤ ⎡ 0 1⎤ ⎡ 2 ⎤ ⎡ 1 ⎤ BS BFS
Case − 4 : X B = ⎢ ⎥ B = ⎢ ⎥ XB = ⎢ ⎥ = ⎢ ⎥ ⎢1 ⎥ = 0 + 1 ⎢ −1 1⎥ ⎢1 ⎥ = ⎢ −1 1⎥ ⎢1 ⎥ = ⎢ −1⎥
⎣ x3 ⎦ ⎣1 0 ⎦ ⎣ 3⎦ ⎣
x 1 0 ⎦ ⎣ ⎦ ⎣ ⎦⎣ ⎦ ⎣ ⎦⎣ ⎦ ⎣ ⎦
−1
⎡ x2 ⎤ ⎡1 0 ⎤ ⎡ x2 ⎤ ⎡1 0 ⎤ ⎡2 ⎤ 1 ⎡ 1 0 ⎤ ⎡2 ⎤ ⎡ 1 0 ⎤ ⎡2 ⎤ ⎡ 2 ⎤ BS BFS
Case − 5 : X B = ⎢ ⎥ B = ⎢ ⎥ XB = ⎢ ⎥ = ⎢ ⎥ ⎢ ⎥ = ⎢ ⎥ ⎢ ⎥ =⎢ ⎥ ⎢ ⎥ =⎢ ⎥
⎣ 4⎦
x ⎣1 1 ⎦ ⎣ x4 ⎦ ⎣1 1 ⎦ ⎣1 ⎦ 1− 0 ⎣ −1 1⎦ ⎣1 ⎦ ⎣ −1 1⎦ ⎣1 ⎦ ⎣ −1⎦

−1
⎡ x3 ⎤ ⎡ −1 0⎤ ⎡ x3 ⎤ ⎡ −1 0 ⎤ ⎡ 2 ⎤ 1 ⎡1 0 ⎤ ⎡ 2 ⎤ ⎡ −1 0 ⎤ ⎡ 2 ⎤ ⎡ −2 ⎤ BS BFS
Case − 6 : X B = ⎢ ⎥ B = ⎢ XB = ⎢ ⎥ = ⎢ ⎥ ⎢1 ⎥ = −1− 0 ⎢ 0 − 1⎥ ⎢1 ⎥ = ⎢ 0 =
⎣ x4 ⎦ ⎣ 0 1 ⎥⎦ ⎣ 4⎦ ⎣
x 0 1 ⎦ ⎣ ⎦ ⎣ ⎦⎣ ⎦ ⎣ 1 ⎥⎦ ⎢⎣1 ⎥⎦ ⎢⎣1 ⎥⎦
ti
13
 Conversion of an LPP into Standard form: Real U lity
XB = B −1b; XN = 0 ….a precursor: through Examples BFS: BS ≥ 0
Standard Form
XB x1 x2

⎡ 1 1 ⎤ ⎡ x1 ⎤ ⎡ 2 ⎤ x1
⎢ −1 1⎥ ⎢ x ⎥ = ⎢1 ⎥
⎣ ⎦ ⎣ 2⎦ ⎣ ⎦
x2

−1
⎡ x1 ⎤ ⎡ 1 1 ⎤ ⎡ 2 ⎤ 1 ⎡1 − 1⎤ ⎡ 2 ⎤ 1 ⎡1 − 1⎤ ⎡ 2 ⎤ ⎡ 0.5 ⎤ BS BFS
XB = ⎢ ⎥ = ⎢ ⎥ ⎢ ⎥ = ⎢ ⎥ ⎢ ⎥ = ⎢ ⎥ ⎢ ⎥ =⎢ ⎥
⎣ x2 ⎦ ⎣ −1 1⎦ ⎣1 ⎦ 1+ 1 ⎣1 1 ⎦ ⎣1 ⎦ 2 ⎣1 1 ⎦ ⎣1 ⎦ ⎣1.5 ⎦

ti
14
 Conversion of an LPP into Standard form: Real U lity
XB = B −1b; XN = 0 ….a precursor: through Examples BFS: BS ≥ 0

Standard Form XB x1 x2 x3 x4
⎡ x1 ⎤ B
x1
⎢ ⎥
x1 + x2 + x3 =2 ⎡1 1 1 0 ⎤ ⎢ x2 ⎥ ⎡ 2 ⎤ x2
⎢1 0 0 1 ⎥ ⎢ x ⎥ = ⎢1 ⎥
x1 + x4 = 1 ⎣ ⎦
⎢ ⎥
3 ⎣ ⎦ x3
⎣ x4 ⎦
x1, x2, x3, x4 ≥ 0 x4
−1
⎡ x1 ⎤ ⎡1 1⎤ ⎡ x1 ⎤ ⎡1 1 ⎤ ⎡ 2 ⎤ ⎡1⎤ BS BFS
Case − 1 : X B = ⎢ ⎥ B = ⎢ XB = ⎢ ⎥ = ⎢ = ⟹ {x1, x2} ≡ {1,1} ⟹ Vertex-C
⎣ x2 ⎦ ⎣1 0 ⎥⎦ ⎣ x2 ⎦ ⎣1 0 ⎥⎦ ⎢⎣1 ⎥⎦ ⎢⎣1⎥⎦

⎡ x1 ⎤ ⎡1 1⎤
−1
⎡ x1 ⎤ ⎡1 1⎤ ⎡ 2 ⎤ ⎡1⎤ BS BFS
Case − 2 : X B = ⎢ ⎥ B = ⎢ ⎥ XB = ⎢ ⎥ = ⎢ ⎥ ⎢1 ⎥ = ⎢1⎥ ⟹ {x1, x2} ≡ {1,0} ⟹ Vertex-D
⎣ x3 ⎦ ⎣1 0 ⎦ ⎣ 3⎦ ⎣
x 1 0 ⎦ ⎣ ⎦ ⎣ ⎦

⎡ x1 ⎤ ⎡1 0 ⎤
−1
⎡ x1 ⎤ ⎡1 0 ⎤ ⎡ 2 ⎤ ⎡ 2 ⎤
BS BFS
Case − 3 : X B = ⎢ ⎥ B = ⎢ ⎥ XB = ⎢ ⎥ = ⎢ ⎥ ⎢1 ⎥ = ⎢ −1⎥ ⟹ Not feasible
⎣ x4 ⎦ ⎣1 1⎦ ⎣ 4⎦ ⎣
x 1 1 ⎦ ⎣ ⎦ ⎣ ⎦

⎡ x2 ⎤ ⎡1 1 ⎤
−1
⎡ x2 ⎤ ⎡1 1 ⎤ ⎡ 2 ⎤ BS BFS
Case − 4 : X B = ⎢ ⎥ B = ⎢ ⎥ XB = ⎢ ⎥ = ⎢ ⎥ ⎢ ⎥ ⟹ Inverse not possible
⎣ x3 ⎦ ⎣0 0 ⎦ ⎣ x3 ⎦ ⎣ 0 0 ⎦ ⎣1 ⎦

−1
⎡ x2 ⎤ ⎡1 0 ⎤ ⎡ x2 ⎤ ⎡1 0 ⎤ ⎡ 2 ⎤ ⎡ 2 ⎤ BS BFS
Case − 5 : X B = ⎢ ⎥ B = ⎢ ⎥ XB = ⎢ ⎥ = ⎢ ⎥ ⎢1 ⎥ = ⎢1 ⎥ ⟹ {x1, x2} ≡ {0,2} ⟹ Vertex-B
⎣ x4 ⎦ ⎣ 0 1⎦ ⎣ 4⎦ ⎣
x 0 1 ⎦ ⎣ ⎦ ⎣ ⎦

−1
⎡ x3 ⎤ ⎡1 0 ⎤ ⎡ x3 ⎤ ⎡1 0 ⎤ ⎡ 2 ⎤ ⎡ 2 ⎤ BS BFS
Case − 6 : X B = ⎢ ⎥ B = ⎢ ⎥ XB = ⎢ ⎥ = ⎢ ⎥ ⎢1 ⎥ = ⎢1 ⎥ ⟹ {x1, x2} ≡ {0,0} ⟹ Vertex-A
⎣ x4 ⎦ ⎣ 0 1⎦ ⎣ 4⎦ ⎣
x 0 1 ⎦ ⎣ ⎦ ⎣ ⎦

ti
15
 Conversion of an LPP into Standard form: Real U lity
XB = B −1b; XN = 0 ….a precursor: through Examples BFS: BS ≥ 0
Standard Form XB x1 x2 x3 x4
⎡ x1 ⎤ B
x1
x1 − x3 =1 ⎢ ⎥
⎡ 1 0 − 1 0 ⎤ ⎢ x2 ⎥ ⎡1 ⎤ x2
⎢ −1 1 0 − 1⎥ ⎢ x ⎥ = ⎢ 0 ⎥
−x1 + x2 − x4 = 0 ⎣ ⎦ 3
⎢ ⎥
⎣ ⎦ x3
⎣ x4 ⎦
x1, x2, x3, x4 ≥ 0 x4
−1
⎡ x1 ⎤ ⎡ 1 0⎤ ⎡ x1 ⎤ ⎡ 1 0 ⎤ ⎡1 ⎤ ⎡ 1 ⎤ BS BFS
Case − 1 : X B = ⎢ ⎥ B=⎢ XB = ⎢ ⎥ = ⎢ = ⟹ {x1, x2} ≡ {1,1} ⟹ Vertex-P
⎣ x2 ⎦ ⎣ −1 1 ⎥⎦ ⎣ x2 ⎦ ⎣ −1 1 ⎥⎦ ⎢⎣ 0 ⎥⎦ ⎢⎣ 1 ⎥⎦

⎡ x1 ⎤ ⎡ 1 − 1⎤ −1
⎡ x1 ⎤ ⎡ 1 − 1⎤ ⎡1 ⎤ ⎡ 0 ⎤ BS BFS
Case − 2 : X B = ⎢ ⎥ B = ⎢ ⎥ XB = ⎢ ⎥ = ⎢ ⎥ ⎢ ⎥ =⎢ ⎥⟹ Not feasible
⎣ x3 ⎦ ⎣ −1 0 ⎦ ⎣ x3 ⎦ ⎣ −1 0 ⎦ ⎣ 0 ⎦ ⎣ −1⎦

−1
⎡ x1 ⎤ ⎡ 1 0⎤ ⎡ x1 ⎤ ⎡ 1 0 ⎤ ⎡1 ⎤ ⎡ 1 ⎤ BS BFS
Case − 3 : X B = ⎢ ⎥ B = ⎢ ⎥ XB = ⎢ ⎥ = ⎢ ⎥ ⎢ 0 ⎥ = ⎢ −1⎥ ⟹ Not feasible
⎣ x4 ⎦ ⎣ −1 − 1⎦ ⎣ 4⎦ ⎣
x −1 − 1⎦ ⎣ ⎦ ⎣ ⎦

⎡ 0 − 1⎤
−1
⎡ x2 ⎤ ⎡ x2 ⎤ ⎡ 0 − 1⎤ ⎡1 ⎤ ⎡ 0 ⎤ BS BFS
Case − 4 : X B = ⎢ ⎥ B = ⎢ ⎥ XB = ⎢ ⎥ = ⎢ ⎥ ⎢ 0 ⎥ = ⎢ −1⎥ ⟹ Not feasible
⎣ x3 ⎦ ⎣1 0 ⎦ ⎣ 3⎦ ⎣
x 1 0 ⎦ ⎣ ⎦ ⎣ ⎦

−1
⎡ x2 ⎤ ⎡0 0 ⎤ ⎡ x2 ⎤ ⎡ 0 0 ⎤ ⎡1 ⎤ BS BFS
Case − 5 : X B = ⎢ ⎥ B = ⎢ ⎥ XB = ⎢ ⎥ = ⎢ ⎥ ⎢ ⎥ ⟹ Inverse not possible
⎣ x4 ⎦ ⎣1 − 1⎦ ⎣ x4 ⎦ ⎣1 − 1⎦ ⎣ 0 ⎦

⎡ x3 ⎤ ⎡ −1 0 ⎤
−1
⎡ x3 ⎤ ⎡ −1 0 ⎤ ⎡1 ⎤ ⎡ −1⎤
BS BFS
Case − 6 : X B = ⎢ ⎥ B = ⎢ ⎥ XB = ⎢ ⎥ = ⎢ ⎥ ⎢ ⎥ =⎢ ⎥ ⟹ Not feasible
⎣ x4 ⎦ ⎣ 0 − 1⎦ ⎣ x4 ⎦ ⎣ 0 − 1 ⎦ ⎣ 0 ⎦ ⎣ 0 ⎦
16
ti
 Standard form U lity: Re ec ng back on the Results
(III ) If S = {X : AX = b; X ≥ 0} is a "non-empty, Compact" constraint set of an LPP:
∙ An optimum solution to the LPP exists
∙ That optimum solution corresponds to "a" vertex

( r ) (2) ( r ) (2) ( r ) (2) ( r ) (2)

n 4 n 2 n 4 n 4
Max possible BS = =6 = =1 = =6 = =6

Actual |BS| 5 (In one: No Inverse) 1 5 (In one: No Inverse) 5 (In one: No Inverse)
Actual |BFS| 0 1 4 1

Ver ces 0 1 4 1

(II ) "If " X* is an optimal solution to an LPP, it must belong to the boundary of the
Set S = {X : AX = b; X ≥ 0}
(I ) X* is a vertex of S = {X : AX = b; X ≥ 0} iff X* is a Basic “Feasible” Solution
17
ti
ti
fl
ti
 Standard form U lity: Re ec ng back on the Results

⎡ x2 ⎤
{x1, x2} ≡ {0,2} X B = ⎢ ⎥ ⎡ x1 ⎤
⎣ x4 ⎦ X B = ⎢ ⎥ {x1, x2} ≡ {1,1}
⎣ x2 ⎦
⎡ x3 ⎤ ⎡ x1 ⎤ {x , x } ≡ {1,0}
{x1, x2} ≡ {0,0} XB = ⎢ ⎥ XB = ⎢ ⎥ 1 2
⎣ x4 ⎦ ⎣ x3 ⎦
As you move from one Vertex/BFS to another: one Basic V and Nonbasic V gets interchanged
∙ one Currently -Basic V becomes Nonbasic
∙ one Currently - Nonbasic V becomes Basic
ti
fl
ti
 Standard form U lity: Re ec ng back on the Results

Pros & Cons, when applied to the Original Constraints converted to AX=b

+ Simple to Apply; Addresses All cases: Null; Singleton; Multiple Solutions

- ∙ Computation of B −1 is not a trivial task, specially in large systems

∙ For each choice of XB ⟹ X ≡ BFS ≡ Vertex: f is to be computed independently
No knowledge from other Xs is used; No guarantee that current X is optimum or not
Example: Vertex-A Vertex-B Vertex-C Vertex-D
Problem-3
{x1, x2} ≡ {0,0} {x1, x2} ≡ {0,2} {x1, x2} ≡ {1,1} {x1, x2} ≡ {1,0}
Min f :
−3x1 − x2 f=0 f =−2 f =−4 f =−3
ti
fl
ti
 Standard form U lity: Re ec ng back on the Results

Original
X
Task

Overcoming Limitations: 3 pre-requisite concepts

Part-I ∙ Cannonical Form of an LPP. In that, pre-requisite concepts include:
(A) ∙ Elementary row operations
A in AX=b
(B) ∙ Row Reduced Echelon Form of a Matrix
(C) ∙ Cannonical Form of LPP Constraints: AX = b AX=b
Part-II ∙ KKT conditions for an LPP Z

Part-III ∙ Synergy between Cannonical form of an LPP and KKT conditions AX=b & Z
ti
fl
ti
 Conversion of an LPP into Standard form: Enhancing U lity
Part-I: Concepts Leading up to Canonical Form of an LPP

Part-I(A): Elementary row operations

A System of Linear Equations
2x + 3y = 12
3x + 2y = 13
Ri ↔ Rj Ri → αRi Ri → αRi + βRJ
3x + 2y = 13 4x + 6y = 24 x−y =1 α =−1
2x + 3y = 12 3x + 2y = 13 3x + 2y = 13 β = 1

Elementary row operations: Operations performed on rows that yield an equivalent S.O.L.E.
⟹ the solution set(s) of both the systems are the same

ti
 Conversion of an LPP into Standard form: Enhancing U lity
Part-I: Concepts Leading up to Canonical Form of an LPP
Part-I(b): Row Reduced Echelon Form (RREF) of Ar⋆n
S.O.L.E For r ≤ n, largest possible I : Ir⋆r
Ar⋆n X Am⋆n √
Given n columns: a maximum of
⟹

Rank(A) = m ⟹ RREF
(r)
n
combinations or Basis

are possible for Ir⋆r

I Ñ I I Ñ
⎡1 0 − 5 ⎤ ⎡1 0 0 ⎤ ⎡1 0 5 − 2 ⎤
Examples ⎢0 1 3 ⎥ ⎢0 1 0 ⎥ ⎢ 0 1 − 1 3 ⎥ For Ar⋆n :
(r)
⎢ ⎥ ⎢ ⎥ ⎢ ⎥ Maximum possible RREFs = n
⎢⎣ 0 0 0 ⎥⎦ ⎢⎣ 0 0 1⎥⎦ ⎢⎣ 0 0 0 0 ⎥⎦

Definition: A Matrix A, subjected to ELROPS, is said to be in Row Reduced Echelon Form, if:
∙ All 0 rows, if any, are clubbed together and stacked below the non − 0 rows
∙ For All non − 0 rows, which define the Rank(A), the leading (first non-zero) element:
∙=1
∙ is the only non − 0 element in the corresponding column
∙ The leading element of any row lies to the right of the leading element of the row above it
ti
 Conversion of an LPP into Standard form: Enhancing U lity
Part-I: Concepts Leading up to Canonical Form of an LPP

Part-I(b): Row Reduced Echelon Form (RREF) of Ar⋆n : Practice Example-I

(3)
⎡ 2 3 − 2 − 7 ⎤ A maximum of 4 = 4 RREFs are possible; Let us derive one w.r.t. the first 3 columns
⎢1 1 1 3 ⎥
⎢ ⎥
⎢⎣1 − 1 1 5 ⎥⎦ Enablers: Leading Element (LE) = 1; LE: the only non-zero element in its column

⎡2 3 − 2 − 7 ⎤ ⎡1 1.5 − 1 − 3.5 ⎤ ⎡1 1.5 − 1 − 3.5 ⎤

⎢1 1 1 3 ⎥ R1 ← R1 /2 ⎢1 1 ⎥ R2 ← R2 − R1 ⎢ 0 − 0.5 2 6.5 ⎥
Let p1 ≡ A11 ⎢ ⎥ 1 3 ⎢ ⎥
⎢ ⎥
⎢⎣1 − 1 1 5 ⎥⎦ ⎢⎣1 − 1 1 5 ⎥⎦ R3 ← R3 − R1 ⎢⎣ 0 − 2.5 2 8.5 ⎥⎦

⎡1 1.5 − 1 − 3.5 ⎤ ⎡1 1.5 − 1 − 3.5 ⎤ ⎡1 0 5 16 ⎤

R1 ← R1 − 1.5R2 ⎢
Let p2 ≡ A22 ⎢ 0 − 0.5 2 6.5 ⎥ R2 ← − 2R2 ⎢0 1 − 4 − 13 ⎥
⎢0 1 −4 − 13 ⎥⎥
⎢ ⎥ ⎢ ⎥
R3 ← R3 + 2.5R2 ⎢ 0
⎢⎣ 0 − 2.5 2 8.5 ⎥⎦ ⎢⎣ 0 − 2.5 2 8.5 ⎥⎦ ⎣ 0 −8 − 24 ⎥⎦
I3⋆3 Ñ
⎡1 0 5 16 ⎤ ⎡1 0 5 16 ⎤ ⎡1 0 0 1⎤
R ← − R3 /8 R1 ← R1 − 5R3 ⎢0
Let p3 ≡ A33 ⎢⎢ 0 1 −4 − 13 ⎥⎥ 3
⎢0 1 −4 − 13⎥⎥ ⎢ 1 0 − 1⎥⎥
⎢
⎢⎣ 0 0 −8 − 24 ⎥⎦ ⎢⎣ 0 0 1 3 ⎥⎦ R2 ← R2 + 4R3 ⎢⎣ 0 0 1 3 ⎥⎦

How to implement it through a code?

ti
 Conversion of an LPP into Standard form: Enhancing U lity
Part-I: Concepts Leading up to Canonical Form of an LPP
Part-I(b): Row Reduced Echelon Form (RREF) of Ar⋆n : Practice Example-I
⎡2 3 − 2 − 7 ⎤ ⎡1 1.5 − 1 − 3.5 ⎤ ⎡1 1.5 − 1 − 3.5 ⎤
⎢1 1 1 3 ⎥ R1 ← R1 /2 ⎢1 1 ⎥ R2 ← R2 − R1 ⎢ 0 − 0.5 2 6.5 ⎥
Let p1 ≡ A11 ⎢ ⎥ 1 3 ⎢ ⎥
⎢ ⎥
⎢⎣1 − 1 1 5 ⎥⎦ ⎢⎣1 − 1 1 5 ⎥⎦ R3 ← R3 − R1 ⎢⎣ 0 − 2.5 2 8.5 ⎥⎦

Automated Implementation
Step-I: Scale the pivot row such that the pivot element becomes 1
RC
Step-II: Any element element E in the matrix gets modified to Ẽ : Ẽ = E − where:
P
∙ P : Original (unscaled) pivot value
∙ R : Matrix element in the 'Row of E' which "lies in the pivotal Column"
∙ C : Matrix element in the 'Column of E' which "lies in the pivotal Row"

⎡2 3 − 2 − 7 ⎤ ⎡1 1.5 − 1 − 3.5 ⎤
⎢1 1 1 3 ⎥ P = A11 = 2 ⎢ ⎥
⎢ ⎥ ⎢ ⎥
⎢⎣1 − 1 1 5 ⎥⎦ R1 ← R1 /P ⎢⎣ ⎥⎦
Consder E ≡ a21
⎡2 ⎤ ⎡2 3 − 2 − 7 ⎤
⎢1 1 1 3⎥⎥ ⟹ R = 1 ⎢1 ⎥ RC 1⋆2
⎢ ⎢ ⎥⟹ C = 2 Ẽ = E −
P
=1−
2
=0
⎢⎣1 ⎥⎦ ⎢⎣1 ⎥⎦
{a21, a22, a23, a24} ∩ {a11, a21, a31} ≡ a21 {a11, a21, a31} ∩ {a11, a12, a13} ≡ a11
ti
 Conversion of an LPP into Standard form: Enhancing U lity
Part-I: Concepts Leading up to Canonical Form of an LPP
Part-I(c): Cannonical Form of LPP Constraints Ar⋆n Xn⋆1 = br⋆1

RREF Cannonical Form

I I Ñ
I Ñ
⎡1 0 0 ⎤ ⎡1 0 5 − 2 ⎤
⎡1 0 − 5 ⎤ ⎢0 1 0 ⎥ ⎢0 1 − 1 3 ⎥
⎢0 1 3 ⎥ ⎢ ⎥ ⎢ ⎥
⎢ ⎥ ⎢⎣ 0 0 1⎥⎦ ⎢⎣ 0 0 0 0 ⎥⎦
⎢⎣ 0 0 0 ⎥⎦

Definition: A System of m equations in n variables (m ≤ n) is said to be in cannonical form w.r.t. a

(m)
n
particular choice of an m-dimensional Basis ( possibilities), if:

∙ each basic variable has a coefficient of 1 in one equation and 0 in others

∙ each equation has exactly one basic variable with coefficient of 1 ti

 Conversion of an LPP into Standard form: Enhancing U lity
Part-I: Concepts Leading up to Canonical Form of an LPP
Part-I(c): Cannonical Form of LPP Constraints Ar⋆n Xn⋆1 = br⋆1
A We have already learnt to convert A to its RREF
⎡ x1 ⎤
⎡ 2 3 − 2 − 7 ⎤ ⎢ ⎥ ⎡1 ⎤
⎢1 1 1 3 ⎥ ⎢ x2 ⎥ = ⎢ 6 ⎥
⎢ ⎥ ⎢ x3 ⎥ ⎢ ⎥
⎢⎣1 − 1 1 5 ⎥⎦ ⎢ ⎥ ⎢⎣ 4 ⎥⎦
⎣ x4 ⎦

XB = {x1, x2, x3}

⎡2 3 − 2 − 7 ⎤
⎢1 1 1 3 ⎥ Structuraly
⎢ ⎥
⎢⎣1 − 1 1 5 ⎥⎦ ⟹
B N

Inference
ELROPs ⟹ Since I = [ELROPs][B] ⟹ [ELROPs] ≡ B −1 ⟹ Ñ ≡ B −1N

I Ñ Utility

⎡1 0 0 1⎤ ⟹ If the same ELROPs were performed on the Augmented Matrix: G ≡ [A ⋮ b]

B −1B B −1N
⎢0 1 0 − 1⎥⎥
⎢ ⎡2 3 − 2 − 7 ! 1 ⎤ ⎡1 0 0
ELROPs ⎢
1! ⎤ ⎡1 0 0 1! 2 ⎤
⎢⎣ 0 3 ⎥⎦ ⎢1 1 1 3 ! 6 ⎥ −1 ⎥ ⎢0 − 1! 1⎥⎥
0 1
−1 −1 ⎢ ⎥ ⎢ 0 1 0 − 1! B b⎥ ≡ ⎢ 1 0
B B B N ⎢⎣1 − 1 1 5 ! 4 ⎥⎦ ⎢⎣ 0 0 1 3!
ti
⎥⎦ ⎢⎣ 0 0 1 3! 3 ⎥⎦
 Conversion of an LPP into Standard form: Enhancing U lity
Part-I: Concepts Leading up to Canonical Form of an LPP
Part-I(c): Cannonical Form of LPP Constraints Ar⋆n Xn⋆1 = br⋆1

I Ñ
A
⎡ x1 ⎤ ⎡2 3 − 2 − 7 ⎤ ⎡ 2 3 − 2 − 7 ⎤ ELROPs ⎡1 0 0 1⎤
⎡ 2 3 − 2 − 7 ⎤ ⎢ ⎥ ⎡1 ⎤
⎢1 1 1 3 ⎥ ⎢ x2 ⎥ = ⎢ 6 ⎥ ⎢1 1 1 3 ⎥ XB = {x1, x2, x3} ⎢1 1 1 3 ⎥ ⎢0 1 0 − 1⎥⎥
⎢ ⎥ ⎢
⎢ ⎥ ⎢ x3 ⎥ ⎢ ⎥ ⎢ ⎥ ⎢⎣ 0 3 ⎥⎦
⎢⎣1 − 1 1 5 ⎥⎦ ⎢ ⎥ ⎢⎣ 4 ⎥⎦ ⎢⎣1 − 1 1 5 ⎥⎦ ⎢⎣1 − 1 1 5 ⎥⎦
0 1
⎣ x4 ⎦
B N B −1B B −1N

B −1B B −1N B −1b

⎡ x1 ⎤
⎡2 3 − 2 − 7 ! 1 ⎤ XB = {x1, x2, x3} ⎡1 0 0 1! 2 ⎤ ⎡1 0 0 1 ⎤ ⎢ ⎥ ⎡2 ⎤
⎢1 1 1 3 ! 6 ⎥ ELROPs ⎢ ⎥ x2
⎢0 − 1! 1⎥⎥ −1 ⎥ ⎢ ⎥ = ⎢1 ⎥
⎢ ⎥ ⎢ 1 0
⎢0 1 0
⎢ x3 ⎥ ⎢ ⎥
⎢⎣1 − 1 1 5 ! 4 ⎥⎦ ⎢⎣ 0 0 1 3! 3 ⎥⎦ ⎢⎣ 0 0 1 3 ⎥⎦ ⎢ ⎥ ⎢⎣ 3 ⎥⎦
⎣ x4 ⎦

Utility-I: Direct Results Utility-II: BV = ψ(NBV ) ⟹ Z = ϕ(NBV )

XN ≡ {x4} = {0} x1 + x4 = 2 ⟹ x1 = 2 − x4
XB ≡ {x1 = 2; x2 = 1; x3 = 3} x2 − x4 = 1 ⟹ x2 = 1 + x4
x3 + 3x4 = 3 ⟹ x3 = 3 − 3x4
Gauss-Elimination
Example: Z = x1 + 2x2 + 3x3 − x4
⟹ Z = 1 + 5x4 ⟹ Z = 1 (x4 ∈ XN = 0)
ti
Conversion of an LPP into Standard form: Enhancing U lity
Part-I: Concepts Leading up to Canonical Form of an LPP
Part-I(c): Cannonical Form of LPP Constraints Ar⋆n Xn⋆1 = br⋆1

ti
 Conversion of an LPP into Standard form: Enhancing U lity
Part-I: Concepts Leading up to Canonical Form of an LPP
Part-I(c): Cannonical Form of LPP Constraints Ar⋆n Xn⋆1 = br⋆1
Example: Inconsistent System - No Solution
x + y + z = 150
x + 2y + 3z = 100
2x + 3y + 4z = 200

p = A11 = 1 p = A22 = 1
⎡1 1 1! 150 ⎤ ⎡1 1 1! 150 ⎤ ⎡1 0 − 1! 50 ⎤
⎢1 2 3! 100 ⎥ R2 ← R2 − R1 ⎢ 0 1 2! − 50 ⎥ R1 ← R1 − R2 ⎢ 0 1 2! − 50 ⎥
⎢ ⎥ ⎢ ⎥ ⎢ ⎥
⎢⎣ 2 3 4! 200 ⎥⎦ R3 ← R3 − R1 ⎢⎣ 0 1 2! − 100 ⎥⎦ R3 ← R3 − R2 ⎢⎣ 0 0 0! − 50 ⎥⎦

x − z = 50
y + 2z = 100
0 = − 50

ti
 Conversion of an LPP into Standard form: Enhancing U lity
Part-I: Concepts Leading up to Canonical Form of an LPP
Part-I(c): Cannonical Form of LPP Constraints Ar⋆n Xn⋆1 = br⋆1
Example: Consistent System - Unique Solution

2x + 4y − 2z = 10 ⎡ 2 4 − 2! 10 ⎤
3x + 9y + 3z = 9 ⎢3 9 3! 09 ⎥
⎢ ⎥
3y + 9z = 18 ⎢⎣ 0 3 9! 18 ⎥⎦

p = A22 = 3 R1 ← R1 − 2R1
p = A11 = 2 R2 ← R2 − 3R1 R2 ← R2 /3 R3 ← R3 − 3R2

⎡1 2 − 1! 05 ⎤ ⎡1 2 − 1! 05 ⎤ ⎡1 2 − 1! 05 ⎤ ⎡1 0 − 5! 09 ⎤
⎢3 9 ⎥ ⎢ 0 3 6! − 6 ⎥ ⎢ 0 1 2! − 2 ⎥ ⎢ 0 1 2! − 2 ⎥
⎢ 3! 09 ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥
⎢⎣ 0 3 9! 18 ⎥⎦ ⎢⎣ 0 3 9! 18 ⎥⎦ ⎢⎣ 0 3 9! 18 ⎥⎦ ⎢⎣ 0 0 3! 24 ⎥⎦

p = A33 = 3 R1 ← R1 + 5R3
R3 ← R3 /3 R2 ← R2 − 2R3
⎡1 0 − 5! 09 ⎤ ⎡1 0 0! 49 ⎤
⎢ 0 1 2! − 2 ⎥ ⎢0 1
⎢ ⎥ ⎢ 0! − 18 ⎥⎥
⎢⎣ 0 0 1! 08 ⎥⎦ ⎢⎣ 0 0 1! 08 ⎥⎦
ti
 Conversion of an LPP into Standard form: Enhancing U lity
Part-I: Concepts Leading up to Canonical Form of an LPP
Part-I(c): Cannonical Form of LPP Constraints Ar⋆n Xn⋆1 = br⋆1
Example: Consistent System - Multiple Solutions

x + 2y + 3z = 150 ⎡1 2 3! 150 ⎤
4x + 5y + 6z = 300 ⎢ 4 5 6! 300 ⎥
⎢ ⎥
7x + 8y + 9z = 450 ⎢⎣ 7 8 9! 450 ⎥⎦

R2 ← R2 − 4R1 p = A22 = 1 R1 ← R1 − 2R2

R3 ← R3 − 7R1 R2 ← R2 / − 3 R3 ← R3 + 6R2

⎡1 2 3! 150 ⎤ ⎡1 2 3! 150 ⎤ ⎡1 0 − 1! − 50 ⎤ x − z = − 50
⎢ 0 − 3 − 6! − 300 ⎥ ⎢ 0 1 2! 100 ⎥ ⎢0 1 2! 100 ⎥⎥ y + 2z = 100
⎢ ⎥ ⎢ ⎥ ⎢
⎢⎣ 0 − 6 − 12! − 600 ⎥⎦ ⎢⎣ 0 − 6 − 12! − 600 ⎥⎦ ⎢⎣ 0 0 0! 0 ⎥⎦ 2 Eqns in 3 variables

ti
Thank You