Split and list technique for solving hard problems

SPLIT AND LIST
EXACT EXPONENTIAL ALGORITHMS
CHAPTER 9
Rohit Kumar Singh
Imai Lab Reading Circle
June 17th, 2015

OUTLINE:
•Introduction
•Split and List [Williams, 2007]
•Sort and Search
•2-TABLE Problem
•Subset Sum
•Exact Satisfiability (XSAT)
•Binary Knapsack
•K-TABLE problem
•Maximum Cut
EXACT EXPONENTIAL ALGORITHMS (CHAPTER 9) 2June 17, 2015

INTRODUCTION:
Split the input of a problem
into sub-problems.
List all possible solutions to
each sub-problem.
Combine solutions of all
sub-problems using fast
polynomial time algorithms
to obtain complete solution

SORT AND SEARCH:
Lexicographical Order (≺):
On a vector space ℚ 𝑚
over rational number set ℚ
Vectors 𝒙 = (𝑥1, 𝑥2, … … , 𝑥 𝑚), 𝒚 = (𝑦1, 𝑦2, … … , 𝑦 𝑚) ∈ ℚ 𝑚
, then we define that
x≺y
if and only if ∃ 𝑡 ∈ {1,2, … . . , 𝑚}
Such that 𝑥𝑖 = 𝑦𝑖 for all 𝑖 < 𝑡 𝑎𝑛𝑑 𝑥 𝑡 < 𝑦𝑡
Example:
1. (2,4,8,3) ≺ (2,7,2,4)
2. (2,4,8,3) ≼ (2,4,8,3)
So we can also write 𝑥 ≼ 𝑦 if 𝑥 ≺ 𝑦 or 𝑥 = 𝑦

2-TABLE PROBLEM:
Given:
Two tables 𝑇1 and 𝑇2 of size 𝑚 x 𝑘, and a vector 𝑠 ∈ ℚ 𝑚
Each table consists of 𝒌 columns of size 𝒎 from vector space ℚ 𝑚
Goal:
𝐶 𝑇1: A column from Table 𝑇1
𝐶 𝑇2: A column from Table 𝑇2
𝐶 𝑇1 + 𝐶 𝑇2 = 𝑠
?
An instance of 2-TABLE problem consists of
entries from ℚ3

2-TABLE PROBLEM…
Example:
Let vector 𝑠 =
Then there are two solutions
Running Time:
• Trivial solution is to enumerate all pairs of columns
• Each comparisons will take 𝑂(𝑚)
• Number of pairs of vectors is 𝑂(𝑘2
)
• Total running time required is 𝑂(𝑚𝑘2)

2-TABLE PROBLEM…
Lemma 9.1
The 2-TABLE problem for tables 𝑇1 and 𝑇2 of size 𝑚 x 𝑘 with entries from ℚ 𝑚
can be
solved in time 𝑂(𝑚𝑘log𝑘).
𝑃𝑟𝑜𝑜𝑓:
1. Sort table 𝑇1 vectors increasingly in lexicographic order.
2. Sort table 𝑇2 vectors decreasingly in lexicographic order.
3. Two vectors comparison will take time 𝑂(𝑚).
4. Consequently sorting can be done in time 𝑂(𝑚𝑘log𝑘).

LEMMA 9.1…
Given sorted vector sequences 𝒂 𝟏 ≼ 𝒂 𝟐 ≼ ⋯ ≼ 𝒂 𝒌 increasing and 𝒃 𝒌 ≼
𝒃 𝒌−𝟏 ≼ ⋯ ≼ 𝒃 𝟏 decreasing in lexicographical order respectively.

SUBSET SUM:
[HOROWITZ, SAHNI, 1974]
Given:
Positive integers 𝑎1, 𝑎2, … , 𝑎 𝑛, and 𝑆
Goal:
Find a subset 𝐼 ⊆ {1,2, … 𝑛} such that
𝑖∈I
𝑎𝑖 = 𝑆,
Example:
𝑎1 = 5, 𝑎2 = 5, 𝑎3 = 10, 𝑎4 = 60, 𝑎5 = 61, and 𝑆 = 70, the solution is 𝐼 =
{1,2,4}

SUBSET SUM…
[HOROWITZ, SAHNI, 1974]
Theorem 9.2
𝑇ℎ𝑒 𝑆𝑈𝐵𝑆𝐸𝑇 𝑆𝑈𝑀 𝑝𝑟𝑜𝑏𝑙𝑒𝑚 𝑐𝑎𝑛 𝑏𝑒 𝑠𝑜𝑙𝑣𝑒𝑑 𝑖𝑛 𝑡𝑖𝑚𝑒 𝑂(𝑛2 𝑛/2
).
1. Partition {𝑎1, 𝑎2, … , 𝑎 𝑛} into two sets X = {𝑎1, 𝑎2, … , 𝑎⌊𝑛/2⌋} and Y = {𝑎 𝑛/2 +1, … , 𝑎 𝑛}
2. Compute set of all possible subset sums for both sets 𝑋 and 𝑌 ≤ 2 𝑛/2+1
3. Let 𝐼 𝑋 = {𝑠1, 𝑠2, … , 𝑠 𝑘} and 𝐼 𝑌 = {𝑟1, 𝑟2, … , 𝑟𝑘} where 𝑠 𝑋 ∈ 𝐼 𝑋 and 𝑟𝑌 ∈ 𝐼 𝑌 such that 𝑠 𝑋 +
𝑟𝑌 = 𝑆
4. To find 𝑠 𝑋 and 𝑟𝑌 we reduce the problem into 2-TABLE problem.
5. Table 𝑇1 and 𝑇2 is formed by elements of 𝐼 𝑋 and 𝐼 𝑌
6. Both table sizes will be 𝑚 x 𝑘, where 𝑚 = 1 and 𝑘 ≤ 2 𝑛/2
7. Using Lemma 9.1, we can calculate two elements (𝑠 𝑋, 𝑟𝑌) in 𝑂(2 𝑛/2
𝑙𝑜𝑔2 𝑛/2
) =
𝑂(𝑛2 𝑛/2)

EXACT SATISFIABILITY (XSAT):
Given:
A CNF-formula (Conjunctive Normal Form) 𝐹 with 𝑛 variables and 𝑚
clauses.
Goal:
A satisfying assignment of 𝐹 such that each clause contain exactly
one true literal.Example:
For CNF Formula 𝐹 : 𝑥1 ∨ 𝑥2 ∨ 𝑥3 ∧ 𝑥1 ∨ 𝑥2 ∧ (𝑥1 ∨ 𝑥3)
Satisfied by truth assignment 𝑥1 = true, 𝑥2 =false, and 𝑥3 = true

EXACT SATISFIABILITY (XSAT)…
Theorem 9.3
𝑇ℎ𝑒 𝑝𝑟𝑜𝑏𝑙𝑒𝑚 𝑋𝑆𝐴𝑇 𝑖𝑠 𝑠𝑜𝑙𝑣𝑎𝑏𝑙𝑒 𝑖𝑛 𝑡𝑖𝑚𝑒 𝑂∗ 2 𝑛/2 .
1. Let 𝐹 be an input of 𝑋𝑆𝐴𝑇.
2. Set {𝑐1, 𝑐2, … , 𝑐 𝑚} and {𝑥1, 𝑥2, … , 𝑥 𝑛} are sets of clauses and variables respectively for
𝑋𝑆𝐴𝑇.
3. Split variable set into two sets 𝑋 = {𝑥1, 𝑥2, … , 𝑥⌊𝑛/2⌋ } and 𝑌 = {𝑥 𝑛/2 +1 , … , 𝑥 𝑛}
4. For every possible truth assignment f (true, false) to variables of 𝑋, calculate
characteristic vector 𝜒 𝑓, 𝑋 ∈ ℚ 𝑚. The 𝑖th coordinate of 𝜒 𝑓, 𝑋 is total count of
true for literals in clause 𝑐𝑖.
5. Repeat step four for variable set 𝑌 and calculate 𝜒 𝑔, 𝑌 ∈ ℚ 𝑚
. The 𝑗𝑡ℎ coordinate
of 𝜒 𝑔, 𝑌 is total count of true for literals in clause 𝑐𝑗.
6. 𝐹 is exactly satisfied if and only if ∃ 𝜒 𝑓, 𝑋 + 𝜒 𝑔, 𝑌 = (1,1, … , 1).
7. Form two tables 𝑇1 𝑎𝑛𝑑 𝑇2 from characteristic vectors of set 𝑋 and 𝑌.
⌈𝑛/2⌉ ∗ 𝑛/2

SOLVING SAT IN 𝑂∗
2 𝑛/2
?
Because:
•We required to find characteristic vector such that,
1,1, … , 1 ≼ 𝜒 𝑓, 𝑋 + 𝜒(𝑔, 𝑌)
•Above condition is real obstacle for using Lemma 9.1
Argument:
• 𝐼𝑓 𝑎𝑖 + 𝑏𝑗 ≺ 1,1, … , 1 , then for every 𝑙 ≥ 𝑖, 𝑎𝑖 + 𝑏𝑙 ≺ 1,1, … , 1 does not
hold anymore.
• So enumerating all possible pairs of vectors is the only solution in
worst case.

BINARY KNAPSACK:
[SCHROEPPEL, SHAMIR, 1981]
Given:
Positive integer 𝑊 and 𝑛 items 𝑆 = {𝑠1, 𝑠2, … , 𝑠 𝑛},
Each item has value 𝑎𝑖 and its weight 𝑤𝑖 which are +𝑣𝑒 integers.
Goal:
Find 𝑆′
⊆ 𝑆 such that,
𝑖∈𝑆′ 𝑤𝑖 ≤ 𝑊 and 𝑖∈𝑆′ 𝑎𝑖 is maximized

MODIFIED 2-TABLE PROBLEM:
Given:
Two tables 𝑇1 and 𝑇2 of size 1 x 𝑘 having +𝑣𝑒 integers and an integer
𝑊
Goal:
Find one number from each table whose sum is at most 𝑊
Example:
For 𝑊 = 14, the solution is the pair of integers (2,11)

MODIFIED 2-TABLE PROBLEM…
MODIFIED 2-TABLE problem can be solved in time 𝑂(𝑘𝑙𝑜𝑔𝑘) with algorithm similar
to 2-table
1. First sorts entries of 𝑇1 in increasing and 𝑇2 in decreasing order. Let 𝑥 ∈ 𝑇1 and
𝑦 ∈ 𝑇2.
2. If 𝑥 + 𝑦 ≤ 𝑊, then all 𝑧 < 𝑦 in 𝑇2 yields 𝑥 + 𝑧 ≤ 𝑥 + 𝑦. Therefor all such pairs 𝑥, 𝑧
can be discarded for consideration.
3. Similarly, if 𝑥 + 𝑦 ≥ 𝑊, then all 𝑧 > 𝑥 in 𝑇1 yields 𝑧 + 𝑦 ≥ 𝑥 + 𝑦. Therefor all such
pairs (𝑧, 𝑦) can be discarded for consideration.
4. Initial sorting requires 𝑂(𝑘𝑙𝑜𝑔𝑘) time and pair finding will take 𝑂(𝑘) steps.

BINARY KNAPSACK…
Theorem 9.4
𝑇ℎ𝑒 𝐵𝐼𝑁𝐴𝑅𝑌 𝐾𝑁𝐴𝑃𝑆𝐴𝐶𝐾 𝑝𝑟𝑜𝑏𝑒𝑙𝑚 𝑖𝑠 𝑠𝑜𝑙𝑣𝑎𝑏𝑙𝑒 𝑖𝑛 𝑡𝑖𝑚𝑒 𝑂∗
2 𝑛/2
.
𝑷𝒓𝒐𝒐𝒇:
1. Split the set of items in two subsets {𝑠1, 𝑠2, … , 𝑠⌊𝑛/2⌋} and 𝑠 𝑛/2 +1, … , 𝑠 𝑛 .
2. For each subset 𝐼 ⊆ {1,2, … , ⌊𝑛/2⌋ }, construct a couple 𝑥𝐼 = 𝐴𝐼, 𝑊𝐼 , where 𝐴𝐼 =
𝑖∈𝐼 𝑎𝑖, and 𝑊𝐼 = 𝑖∈𝐼 𝑤𝑖.
3. Put all couples in set 𝑋 whose cardinality is at most 2 𝑛/2
.
4. Repeat step two to construct 𝑦𝐽 = (𝐴𝐽, 𝑊𝐽) where 𝐽 ⊆ 𝑛/2 + 1, 𝑛/2 + 1 + 1, … , 𝑛
and step three to obtain couples set 𝑌.
5. Find couples 𝑥𝐼 ∈ 𝑋 and 𝑦𝐼 ∈ 𝑌 such that 𝐴𝐼 + 𝐴𝐽 is maximum and 𝑊𝐼 + 𝑊𝐽 ≤ 𝑊.
6. Remove couples dominated by other couples by preprocessing in order to convert
the instance into Modified 2-TABLE problem.

BINARY KNAPSACK…
𝑃𝑟𝑜𝑜𝑓…
Preprocessing Steps:
1. Sort items of 𝑋 in increasing order of their weight { 𝐴1, 𝑊1 , 𝐴2, 𝑊2 , … , (𝐴 𝑘, 𝑊𝑘)},
where 𝑘 ≤ 2 𝑛/2.
2. For every 1 ≤ 𝑖 < 𝑗 ≤ 𝑘, 𝑊𝑖 ≤ 𝑊𝑗, we define 𝐴 ≔ 𝐴1 and move into the list from 1 to
𝑘.
3. If 𝐴𝑖 > 𝐴, replace 𝐴 ≔ 𝐴𝑖 otherwise (𝐴𝑖 ≤ 𝐴), we remove (𝐴𝑖, 𝑊𝑖).
4. Step 1-3 is repeated for set 𝑌.
1. We have 𝐴𝐼, 𝑊𝐼 ∈ 𝑋 and 𝐴𝐽, 𝑊𝐽 ∈ 𝑌, and maximum sum 𝐴𝐼 + 𝐴𝐽 subject to 𝑊𝐼 +
𝑊𝐽 ≤ 𝑊.
2. Construct a table of size 2x2 𝑛/2
and 𝑇1 = 𝑠𝑒𝑡 𝑋 and 𝑇2 = 𝑠𝑒𝑡 𝑌 to implement
MODIFIED 2-TABLE.
3. Total running time = Sorting time O(𝑘𝑙𝑜𝑔𝑘) + finding set with no dominant couple
in time 𝑂(𝑘). EXACT EXPONENTIAL ALGORITHMS (CHAPTER 9) 18June 17, 2015

K-TABLE PROBLEM:
Given:
𝑘 tables 𝑇1, 𝑇2, … , 𝑇𝑘 of size 𝑚 x 𝑘 with entries from ℝ 𝑚
Goal:
For a given vector 𝑐, find the set of vectors 𝑐1, 𝑐2, … , 𝑐 𝑘 , 𝑐𝑖 ∈ 𝑇𝑖 such
that,
𝑐1 + 𝑐2 + ⋯ + 𝑐 𝑘 = 𝑐.
Time Complexity:
𝑘-TABLE problem can be solved in time 𝑂(𝑛 𝑘−1 + 𝑘𝑛𝑙𝑜𝑔𝑛)

K-TABLE PROBLEM…
•2-TABLE problem uses 𝑘 = 2 𝑛/2 and hence useful in reducing space
requirement by algorithms we discussed.
•𝑘-TABLE problem is used by [Schroeppel, Shamir, 81] to reduce the
space requirement to 2 𝑛/4
Example:
• Problem like 𝑋𝑆𝐴𝑇 can be split into 𝑘 subsets.
• Then construct 2 𝑛/𝑘
sets of vector for each table.
• By using 2-TABLE algorithm recursively, 𝑘-TABLE problem of 𝑋𝑆𝐴𝑇 can be solved
in time 𝑂∗
(2 𝑘−1 𝑛/𝑘
)

MAXIMUM CUT:
Product of 𝑛 x 𝑛 matrices in time 𝑂(𝑛 𝜔), where 𝜔 < 2.376 called square
matrix multiplication exponent.
Given:
Undirected graph 𝐺 = (𝑉, 𝐸)
Goal:
Find set 𝑋 ⊆ 𝑉 maximizing the value of 𝐶𝑈𝑇(𝑋, 𝑉X)
Example:
𝐺 = (4,4) and 𝐶𝑈𝑇 2,42 =
3 EXACT EXPONENTIAL ALGORITHMS (CHAPTER 9) 21June 17, 2015

MAXIMUM CUT…
Theorem 9.5 [Itai, Rodeh, 1978]
𝐴 𝑡𝑟𝑖𝑎𝑛𝑔𝑙𝑒 𝑖𝑛 𝑎 𝑔𝑟𝑎𝑝ℎ 𝑜𝑛 𝑎 𝑛 𝑣𝑒𝑟𝑡𝑖𝑐𝑒𝑠 𝑐𝑎𝑛 𝑏𝑒 𝑓𝑜𝑢𝑛𝑑 𝑖𝑛 𝑡𝑖𝑚𝑒 𝑂 𝑛 𝜔 𝑎𝑛𝑑 𝑖𝑛 𝑂 𝑛2 𝑠𝑝𝑎𝑐𝑒.
1. Let 𝐴(𝐺) be the adjacency matrix of G.
2. 𝑘𝑡ℎ power of 𝐴 𝐺
𝑘
of 𝐴(𝐺) the entry 𝐴 𝐺
𝑘
⌊𝑖, 𝑖⌋ on the main diagonal of 𝐴 𝐺
𝑘
is equal to number of walks of length 𝑘 which starts and ends at vertex 𝑖.
3. A walk of length 3 = triangle, so non-zero entries at the diagonals of 𝐴 𝐺
3
assures triangles in G.
4. Time required is 𝑂(𝑛 𝜔).
5. Space requirement is proportional to size of 𝐴(𝐺), which is 𝑛2
.

MAXIMUM CUT…
[WILLIAMS, 2007]
Theorem 9.6
𝑇ℎ𝑒 𝑀𝐴𝑋𝐼𝑀𝑈𝑀 𝐶𝑈𝑇 𝑝𝑟𝑜𝑏𝑙𝑒𝑚 𝑜𝑛 𝑛-𝑣𝑒𝑟𝑡𝑒𝑥 𝑔𝑟𝑎𝑝ℎ𝑠 𝑖𝑠 𝑠𝑜𝑙𝑣𝑎𝑏𝑙𝑒 𝑖𝑛 𝑡𝑖𝑚𝑒 𝑂∗ 2 𝜔𝑛/3 =
O 1.7315n
, where 𝜔 < 2.376 𝑖𝑠 𝑠𝑞𝑢𝑎𝑟𝑒 𝑚𝑎𝑡𝑟𝑖𝑥 𝑚𝑢𝑙𝑡𝑖𝑝𝑙𝑖𝑐𝑎𝑡𝑖𝑜𝑛 𝑒𝑥𝑝𝑜𝑛𝑒𝑛𝑡.
[Coppersmith, Winograd, 1990]
• Let 𝐺(𝑉, 𝐸) is a graph on 𝑛 vertices and
n
3
= 0. Let 𝑉0, 𝑉1, 𝑉2 are partitions of 𝑉 into
sets of sizes
𝑛
3
.
• Construct a auxiliary weighted directed graph 𝐴 𝐺 as follow:
1. ∀ subset 𝑋 ⊆ 𝑉𝑖, 0 ≤ 𝑖 ≤ 2, graph 𝐴(𝐺) has vertex 𝑋.
2. Total vertices in 𝐴(𝐺) is 3 x 2 𝑛/3.
3. 𝐴 𝐺 has all possible pairs of arcs between (𝑋, 𝑌) where 𝑋 ⊆ 𝑉𝑖, 𝑌 ⊆ 𝑉𝑗, and 𝑗 = 𝑖 +
1(𝑚𝑜𝑑3).
4. Total number of arcs in 𝐴 𝐺 = 3 x (22𝑛/3
).
5. Define weight for for every arc (𝑋, 𝑌), 𝑋 ⊆ 𝑉𝑖 and 𝑌 ⊆ 𝑉𝑗, 𝑖 ≠ 𝑗 asEXACT EXPONENTIAL ALGORITHMS (CHAPTER 9) 23June 17, 2015

MAXIMUM CUT…
[WILLIAMS, 2007]
𝑃𝑟𝑜𝑜𝑓 …
Claim: The following properties are equivalent
(𝑖) There is 𝑋 ⊆ 𝑉 such that 𝐶𝑈𝑇 𝑋, 𝑉X = 𝑡.
(𝑖𝑖) The 𝐴(𝐺) contains directed triangle 𝑋0, 𝑋1, 𝑋2, 𝑋𝑖 ⊆ 𝑉𝑖, 0 ≤ 𝑖 ≤ 2, such that
𝑡 = 𝑤 𝑋0, 𝑋1 + 𝑤 𝑋1, 𝑋2 + 𝑤(𝑋2, 𝑋0)
𝑃𝑟𝑜𝑜𝑓 𝑜𝑓 𝑐𝑙𝑎𝑖𝑚𝑠:
1. 𝑖 ⇒ 𝑖𝑖 , put 𝑋𝑖 = 𝑋 ∩ 𝑉𝑖, 0 ≤ 𝑖 ≤ 2
2. Every edge 𝑒 = 1 if 𝑒 is edge between 𝑋 and VX otherwise 0 to the sum
𝑤 𝑋0, 𝑋1 + 𝑤 𝑋1, 𝑋2 + 𝑤(𝑋2, 𝑋0)
1. 𝑖𝑖 ⇒ 𝑖 , put 𝑋 = 𝑋0 ∪ 𝑋1 ∪ 𝑋2,
2. Every edge e counted in 𝐶𝑈𝑇 𝑋, 𝑉X is same as counted in sum
𝑤 𝑋0, 𝑋1 + 𝑤 𝑋1, 𝑋2 + 𝑤(𝑋2, 𝑋0)

MAXIMUM CUT…
[WILLIAMS, 2007]
𝑃𝑟𝑜𝑜𝑓 …
1. To proof claim 𝑖𝑖 holds, we try all possible values of 𝑤 𝑋𝑖, 𝑋𝑗 , 𝑗 = 𝑖 + 1(𝑚𝑜𝑑3).
2. For every triple 𝑊 = (𝑤01, 𝑊12, 𝑊20) such that 𝑤 = 𝑤01 + 𝑤12 + 𝑤20, construct
subgraph 𝐴 𝐺, 𝑊 of 𝐴(𝐺) having only arcs of weight 𝑤𝑖𝑗 from 𝑋𝑖 ⊆ 𝑉𝑖 to 𝑋𝑗 ⊆ 𝑉𝑗.
3. For ∀ 𝑡, number of such triples is 𝑡3.
4. The subgraph 𝐴(𝐺, 𝑊) for all arcs of 𝐴(𝐺) can be constructed in time 𝑂∗
2 𝑛/3 2
.
5. 𝐴 𝐺, 𝑊 contains triangle can be verified in time 𝑂∗
2 𝑛/3 𝜔
using Theorem 9.5 .
6. The total running time is 𝑂∗(𝑡. t3 22𝑛/3 + 2 𝑛𝜔/3 ) = 𝑂∗(2 𝜔𝑛/3).
June 17, 2015 EXACT EXPONENTIAL ALGORITHMS (CHAPTER 9) 25

REFERENCES:
Williams, R.: Algorithms and resource requirements for fundamental problems. Ph.D. thesis, Carnegie Mellon
University (2007)
Horowitz, E., Sahni, S.: Computing partitions with applications to the knapsack problem. J. ACM 21, 277–292
(1974)
Schroeppel, R., Shamir, A.: A T = O(2n/2), S = O(2n/4) algorithm for certain NP-completeproblems. SIAM J.
Comput. 10(3), 456–464 (1981)
Itai, A., Rodeh, M.: Finding a minimum circuit in a graph. SIAM J. Comput. 7(4), 413–423 (1978)
Coppersmith, D., Winograd, S.: Matrix multiplication via arithmetic progressions. J. Symbolic Comput. 9(3), 251–
280 (1990)
June 17, 2015 EXACT EXPONENTIAL ALGORITHMS (CHAPTER 9) 26

Split and list technique for solving hard problems

More Related Content

What's hot

Similar to Split and list technique for solving hard problems

Recently uploaded

Split and list technique for solving hard problems