ese Re Bl 4 Guide 6 he Theory ol NPoCommeteness ee DEM aitary of Congres Catalan Pabiton Dat ‘Cees tec 1 lone dat compe Prgrmming 2. Aoi 3 Copan coma E ipso’ Baie sua Tie tn" Scorpions pare oe tee Bon oieroaer ISBN O7IeTB0SS ob, ‘AMS Cheon: ary Sa enya #1919 BT No to ibook ay be pede! by ay (Bide form af «phonograph: toring. or ay Stel na etieval tem rare. of nti cope or obo ae ws oak ‘ren permis fem te polar rane Labor, oot Pa inh Used Sts of Aer Contents Computers, Complexity, and Insractability ' LE rvroveton L Problems. Algrits, ana Corps 4 13 Polynomial Time Algorithms and Inraiabie Problems 6 4 ProveblyInrenble Problems it ES- NP-Comlete Probes B 1G An Outine of he Book m ‘The Theory of NP-Completeness 0 21. Decision Problems, Languages. and Encoding Schemes... 18 22. Determinisie Turing Machines ad te Css P 3 23. Nondeterminisie Computation andthe Css NP 2 24 The Reisionship Between P and NP. 3 23. Polyromal Transiormations and NP-Copicieness a7 26 Cook's Theorem 38 Proving NP-Completeness Results 4s AL Six Basie NP-Complee Problems 46 SILL SSATISRIABILATY 3 31.2 DIMENSIONAL MATCHING 0 313 VERTEX COVER ang CLIQUE 3 313 HAMILTONIAN cIRCUTT 56 o 8 66 a4 Using NP-Completeness to Analyze Problems 41. Anaaing Subprobloms 42 Number Probleme and Song WP Contos 421 Some Adcivonal Detalions 43. Time Complexity a Function of Natu NP-Hardness 1 Turing Reus and NP-Hard Proboms 8A terminate Bory (© Coping wien WE-Compiete Fropiems 2 Apelyng NP-Compleenas co Approximation Prosiems 663. Pastormance Gusramees and Gahsvion In Practice” 7 Beyond NP-Con TA. The Strate of NP 12 The Plyromiat Hiemchy 13. The Complexity of Enumeration Problems 14 Paloma Space Completeness TS. Loprithmie Space 1 Profs of Inacabilty tnd B NP Appendix: A List of NP-Complete Problems AL Graph TRON ooo ALT Covering and Prisoaing ‘ALE Sobprapne ana Sopereapns AL Vertex Orsring AL's Wsosana Omer Moris ALS Macelaneoos [Ad Neiwoek Des ALL Spaming Tees | ‘23 Routing Probie AALS Mow Problems ALS Miscelineous ALL Covering, iting, and Sptning (822 Weighed Set rebome As Storage snd Retinal Att Dar se : N42 Compression and Retention conrext 7 los 109 m Me 189 161 ui ‘ONTENTS AS. Sequnncing and Scheduling AST Sequencing on One Piosessor AS? Malupecesee Scheduling [AS3 Shop Schecing AS. Miecelineous [AG Mathematical Propensing [AT Algeba and Number That ALT Divs Proiens, [AL2 Solabiliy of Equations ‘As Games ane Puzzles AY Loge AST Propoitinal Lge AUD Automata and Langage Thaory Ali auorsis theory ‘A102 Pormat Largusses Att Prog Optica ATI Cole Generaton ‘AIL Propiams and Seem lz Miscetanebus Symbol Index Reference and Author Index Subject Index Update for the Current Printing -26 2s ae 2st 20 28 230, Ea 29 239 es 28 m 25 79 3Preface Few lechnial terms have stined such rapid notoriety as the appl ign “NPrompleie.” In the stor dime sive ity iaouctin in te ally 1970's, this term has come to symbolize the abyss of inherent inclu ina agora Weigneis Ineeasingly fae ay ley wek ro Solve Lager sl ‘more complex problems. A wide variety of commonly encountered prob lems trom mauremaues, compurr setenee, ana operations researcn are now known to be NP-complete andthe collection of such problems continucs (0 ‘tow almost diy. Indeed, the NP-complete problems are now so pervasive {hati 8 important for anyone concerned with the computation aspects of| ‘ase fields tobe familar with the meaning and implications ofthis concent "This book is intended a a detaled guide to the theory of NP. completeness, erplaszing those concepts and techniques that Seem 10 be ‘oct seul fie sping the theory to pctv pbleme ean he veut 5 consisting of thee par “The fat part, Chapters | through 5, cavers the basic theory af NP. completeness, Chapter T presents a rolaively low-level Intoduction 10 ome of the canal notions of computational complenity and docusees the Significance of NP-completeness in this context. Chapters 2 through § po: Lndersanding and applying the theory. "The seuord pul, Chaplets 6 aid 7, provides an overview of two al ternative directions for further study. Chapter § oaceatrates on the seach for ecient" approximation” algorithms for NP-complete problems an ares Whose development has seen considerable interplay with the theory of NP. compleieness. Chapler 7 surveys a large number of theoretical 1pies in ‘computational complexity, many of which have arisen as a consequence of previous work on NP-completeness. Both of these chapters (especially Chapter 7) ate intended solely as introductions to these areas wih Our ex peetation being that any reader wishing to pursue particular topics in more ‘etait will do 30 by consulting the cited references “The thd andra a ofthe book the Append, which conins fesuls in otal) of NP-complete and NP-hard problems. Anaotations tothe fain entree diacuse what iz Keown about the complenty of subprobleme tnd variants of the sated problems.x raceace The hook should be suitable for use as a supplementary text in courses on algorithm design, computational complexity, operations research, ‘orcombinatorial mathematics It len ran be wsnd ae = karting poi for seminars on approximation algorithms or computational complexity at the Breliminary daft asthe basis for a graduate seminar on approximation algo- ius, uveting Chapters 1 through 3 in abvut five weeks and hen pursu ing the topics in Chapter 6, supplementing them extensively with additional materal from the references. A seminar on computaional complexity might proced similariy, substituting Chapter 7 for Chapter 6 as the initial access point fo the Ierature. It isso possible fo cover both chapters in & combined seminar More generally, the book can serve both asa selfstudy text for any= fone imereted in learning about the subict of NP-completeness and a5 2 ‘ference book for researchers and practitioners who ae concerned with al gorithms and their complexity. The list of NP-complete problems in the Appendix cin be used by anyone familiar withthe central notions of NP- compoteness, even without having read the matoral in the min text. The rove can gain such familar by skimming the material in Chapters 1 ragh 5, concentrating on the informal discussione of dafnitions and fechaigues, and returning 10 the more formal material only as needed for Clarifeation. To id thove wing the bok as reference, we have included @ Substantial umber of trms in the Subject Index, and the extensive Refer~ fence and Author Index xivey the seins whore el referonee is men Uoned i the text We are inaeoted 10 a tage qumper of people wno nave neiped us sreatly in preparing this book. Hal Gabow, Larry Landweber, and Bob Tat Jan taught Irom preliminary versions ofthe book and provided us with alu fable suggestions based on their experience. The following people read pre liminary drafts ofall or pat of the book and made consirective comments Al Abo, Shimon Even, Ron Graham, Harry Hunt, Vicior Klee, Albert Meyer, Chisios Papadimitriou, Henry Bolla, Sart Sahni, Ravi Sethi, Lar ry Stnckmeyer, and Ie Ullman Alpe _quimber of ceararchers 100 humerous to mention here (but see the Reference and Author Index), Fespanded to out call for NP-complateness results and contributed foward ‘making our list of NP-complote problems as extensive as it is. Several of ‘ur colleaguos at Ball Laborstorien, expecially Brian Kernighan, provide in- valuable assistance wth computer typesetting on the UNIX® system. Final- ‘or handwritten hieroglyphies int faultless input to the typesetting system Fae the as of wating this bo0K so much ease. ‘sucray Hi, New Jersey MICHAEL K. GAREY October, 1978 ‘DAVID S. JOHNSON COMPUTERS AND INTRACTABILITY ‘A Guide tothe Theory of NP-Completeness1 Computers, Complexity, and Intractability and Intractability 1.1 Introduction ‘The subject matter ofthis book is perhaps bes introduced through the Suppose that you, tke the authors, are employed inthe alls of indus tex, One day your boss calle you into hie offes and confides thatthe com pny is about to enter the highly competitive “bundersnatch” market. For This naon, a good metiiod le needed for determining whether not any tiven set of specifications for a new bandersnatch component can be met fd, iso, for vonstiacing 4 design that sioets therm Sine you are the ‘company’s chief algorithm designer, your charge isto find an efficient algo- fh for eoing ts ‘Aer consulting with the bandersnich depattmert to determine exactly ‘what the problem is, ou eagerly hurry back to your ofc, pull down your feference books, and plunge into the cask with great enifusiasm. Some ‘weeks later, your afice filled with mountains of crumpled-up serach paper, ‘our enthusiasm has lessened considerably. So fat vou have not been able to come up with any algorithm substantial beter than searching through ll patible designs. This would not puriculaly endear yo to you howe Since it would involve years of computation sime for just one set of2 (COMPUTERS, COMPLEXITY, AND INTRACTABILITY scificatons, and the bandersnatch department is alcady 13 components behind schedule. You cerainiy don't want 10 return to his office and re- fe x t Tea’ tnd an efieient lg, {gues lm jst c0 dum” ‘To avoid serious damaye to your postion within the company, it would bbe much better if you could prove thet the bandersoateh problem str hereny intractable, that 00 algorithm could possibly solve i quickly. You (We could stride content iio the boss office and proclaim: —* iF “Lean fnd an eteien lgorithe, because no such algorithm posible!” Unfortunately, proving inherent intractabiiy can be just as hard as Aoing efficient algorthns, Even the teat theutetans have Veo stymied in thelr atiempss 10 obtain such proofs for commonly encountered hard problems. However, luvin read this book, you have discovered something 1 asrRopucTion 3 almost ag gsod. The theory of NP complet ‘rard techniques for proving that a gven problem is Ieige outer of other problems that are widely recognized a boing dificult land that have been confounding the experts for years. Armed with these teennigues, you might pe aoie to prove tna ine pandersnatcn prooiem is NP-complete and, hence, that it i equivalent to all these other hard prob- lems. Then you could march ino your bos'soice and announce: afin yy , it tc’ indo fe algrith ‘but mthrsan ll these fans poste [AC the very lest, this would inform your boss that it would do no good to fire you and hie another expert on alorithms ‘OF course, our own Dosses would fro¥n upon.out wrting this book if ite cole pee ane ta ponte! the oe af algeithm desionereIndad, di fovering that a problem is NP-complete is usually just the beginning of ‘wore on that problem. "The needs of the bandernalah department wo! Aisappear overnight simply” because theie problem is known to be NP- ‘omplte, Howover, the knowledge that itis NP complate doce provide valuable information about what lines of approsch have the potential of be- should be accorded iow priority. Is now more appropriate to concentrate fn other, less ambitious, approaches. For example, You might 100K for efficient algorithms that solve various special cases ofthe general probiem, ‘You might took for algoetnms tha, though not guaranteed to run quickly, seem kay t0 do so most of the time. Or you might even relax the prob. lem sontewnat, looking for fast algorithm that merely finds designs that4 (COMPUTERS, COMPLEXITY, AND INTRACTABILITY tion of the theory of NP-completeness isto asist algorithm designers in iectng thele problem-solving effovts toward those approacies that have the sreates likeinood of leading o useful algorites. {nthe fst chapter of tls ulde 10 Nr-completeness, we inroauce many of the underlying concepts, discuss thet applicability (as well a5 give some caution), and outline the remainder of the book. 1.2 Problems, Algorithms, and Complexity In erder tn alahorna an what meant hy “Sinheronslyintarthle’ problems and problems having “equivalent” dificult, i i important that tre rt agree om the meaning of several more baie tem Let us begin withthe notion of a problem. For our purpose, a problem wil bea general quoston to be snawored, wally postosting several goram ‘tr, oF fee vatibes, whose values are left unspecified A problem is Wesctibed by giving: (1) a genoa description of ally puameiets, and (2) 4 Statement of what properties the answer, or solution, 8 required Co sais. ‘An instance ofa problem Is obiainea by speciying particular values for all the problem parameters. ‘AS an eximple, consider the classical “traveling salesman problem, ‘The parameters of this problem consist of a finite set C = (eco. ‘of “cities” and, foreach pair of cities Gc, in C, the “distance” ddeg) between them. A solution is en ordering’ is solution fo this instance, ab the corresponding tour has the maiirsum possible tour length of 27 VAlorithms em ganna, stnhy.cien neneedes for calving penhlems For conereteness, we can think of them simply as being computer programs, ‘writen in soma precise computor language. An algorithm is sid fo soe 4 probiem Tif that algorithm can be applied to any instance 7 of TL and i fguarantzad alwaja Lo produze 2 solution for that insanes 7. We enmphasiee ‘thatthe ferm solution” is intended here sey in the sense introduced ove, 0 that im parclat, an algorithm does not “solve” the traveling 12 PROBLEMS, ALGORITHMS, AND COMPLEXITY 5 Figure L1 An instance ofthe saveling salesman problem and a tour of lng 27, stich the minim posi ns case salesman problem unless it always constructs an ofdering that gives & ‘inimum length tour In genera, we are interested in finding the most “efficient” algorithm for solving a problem. In iis broadest sense, the notion of efliciency in- volves all the various computing resources needed for executing an algo Tithe. However, bythe “most effiient” algorithm one normally means the Fastest Since time requirements are often 2 dominant factor determining whether oF not a particular algorithm is efficient enough to be useful in ‘rutin hall enncontratapfimacly om tie singin esetene ‘The time requiemen’s of an algorithm ate conveniently expressed in terms af a tingle visable, the “sine” of 4 peablem instance, whichis in tended to relet the amount of input data needed to describe the instance “This convenient because wo Would oxpeat the relative dificult of prod Jem instances to vary roughly with thet size. Often the sie of a problem instance Is measured in an informal way. For the traveling salesman prob Jem, for example, the number of cites is commonly used for this purpose. However, an m-city problem Instance iniudes, in adion (othe fbels oF the mr ees, a collection of m(m—1)/2 numbers defining the intr-ciy ds- tances, and ie sas of these numbers also conibute to the amount of in pur dat. If we ae 10 deal with time requirements ina precise, mathemal- fal manner, we must take care to define instance size in such way tha all ‘these factors ae taken into account “To do this, observe thatthe desrigtion of a problem instance that we provide as input to the computer can be viewed as a sinzle finite sting of symbols chosen from a finite input alphabet. Although there are many Aiferant waye in which instances nf given problem might he deseribad, at 1s assume that one particular way has been choson in advance and that cath Problem har associated with it a xed encoding scheme, which maps problem6 (COMPUTERS, COMPLEXITY, AND INTRACTABILITY Instances ine the singe desobing them, The input gt for an instance of problem ILis defined to be the number of symbols inthe description ‘OFT baie! from the encaing scheme for TL. Iti aumber the input length, thats used asthe formal measure of instance siz, For example, instances of tne traveling salesman prooiem might oe described using the alphabet eyo} /;0, 1,2, 3,4, 5,6, 7,89] with our pre- vious example of '2 problem ‘insiance’ being encoded by the string “eltle(2e(Le(4l//10/5/9//6/9//3." More complicated insances would be ‘encoded in analogous fashion, If this were the encoding Scheme associated withthe tavelina Salesman problem, then the input length For our example sould be 32. ‘ments by giving, foreach possible input length, the largest amount of time needed by tho algorithm to solve problem instance of that size. OF ‘course, ths function isnot well-defined until one ies the encoding scheme fo be used for determining input lengit and the computer or Sompulor ‘model 10 be used for determining execution time. However, as we shall ‘Ste, the particular choices mide for these wil ive tilly offovt on the broad distinctions made in the theory of NP-oompletenes, Hence, in what fo- tows, the reader advised metely (0 Tix mind a parucular encoding scheme for each problem and a particular computer or computer model, and {o think in terms of time complexly as determined trom the coresponding input lengths and execution times. 1.3 Polynomiat Time Algorithms and Intractable Problems Different algorithms possess a wide varity of eifferent time complexity functions, andthe charicteriztion of which ofthese are “ficient ena and which are “too inefficient” will avays depend on the situation at hand. Howevor, computer sintsts resize a simple distinction that offre con sierabl insight into these matters. This i the distinction between polyno- ‘ial Vine agin and exponesia Cane agus Let us siy that function /(n) Is O(g(n)) whenever there exists a constant esueh that [7D] % e}xCa)| forall values of 20. A popmom {al time algorithm i defined to be one whose time complexity function is ‘O(pCm) for some polynomial Furecion p, where n Is used 10 Gente te in ‘put length. Any algorithm whose time tomplexity function cannot be so bounded is clled an exponential tne algeridim (although it should be noted that this definition includes certain non-solyaomial time complexity func ‘ions, like n®¥", which are nol normally regarded as exponential function). “The distincion between these «wo tvpes of algorithms has particule significance when considering the Solution of large problem instances. Fig lute 1.2 illustrates che affrences in growth rates among several typical com plexity functions of each type, whee the functions express execution time 1.5 POLYNOMIAL TIME ALGORITHMS AND INTRACTAALE PROBLEMS 1 in tims of microsovonds, Notice the wich me explosive grows sates for the two exponential coraplexity functions. oe fie 4 See fe | 2 |» | « |» | @ | | 0001 | 00002 | 00003 | coos | 0000s | ooo | | semna | second | second | esond | seeand_| Second + | a | ome | 0009 } core | oms | sox 1 1 | or | om | om | ose | noe | ane er ee ee we fa ws [ar | sa | no » | orf so | aro | ar | sr | oe |_% _[estena | seta { minus | diye | yes_| cents | yw [09 | st | 6s | aus | pao! | 1300%| ‘gure 12 Companson of Several polynomial end exponent ume compesty Fuctons Even more revealing is an examination ofthe effects of improved com- puter technology on algorithms having these time complexty Tunctions Figure 13 shows how the largest problem instance solvable in one hour would cfange if we had a computer 100 or 1000 times faster than our resent machine. Observe that with the 2" leorthm a thousand-fold in ‘tease in computing speed only adds 10 to the sizeof the largest problem instance we can solve fa an hour. whores with the n° alaorithm this size a= ‘most quadruples. rithms are generally regarded being much more desirable than exponen- Tal mo algorithm. This view, and the distinction betwoen the 0 pes of algorithms, is contralto our fotion of inherent intractablity and tothe theory of NPcomplecancss, “The fundamental nature of this distinction was fist discussed in (Cob- ‘nan, 1968) and (Camonds, 1968). Edmonds, sa particule, equated poly=8 ‘COMPUTERS, COMPLEXITY, AND INFRACTABILITY Sie of Largest Problem Instance Solvable int Hour Time] Fanston | computer | 10 mes fasier | 100 times Fister i a {om | too, | oto wm, | som | | i 466m 10m 25M 3. Mique £3 CMst of inyeoved tstauiogy on seve pupil and exponential ime lech. ‘nomial time algorithms with “good” algorithms and conjectured that cetain Inieger programming problems might not Be solvable by such “good” algo- rithms. This reflects the viewpoint that exponential time aigorithms should not be considered "good" algorithms, and indeed this usually is the ease. Most exponential time algorithms ate merely variations on exhaustive ‘search, whereas polynomial time algorithms generally are made possible ‘only through the sain of some deeper insight into the structure of a arb. lem. ‘There is wide agreement that a problem has not been “well-solved™ Until polynomial ime algorithm ie known fori. Hanca, we shall oer a problem as Inractable if i$ 50 hard that no polynomial time algorithm fan posiby colve i (OF course, this formal use of intractable” should be viewed only as ough approximation to ils dictionary iesuing. ‘The dstioction between “efficient” polynomial time algorithms and “inefficient exponential time algorihns adits of many exeepuions when dhe probe instances OF ie terest have limited size. Even in Figure 1.2, the 2 algosthm faster than ne n aigorinm tor m'< 20. More extreme examples can be constructed casi FFurtnermore, there are some exponential cme algorithms that have been quite useful in practice. Time complexity as defined isa worsecase ‘measure, andthe fact that an algorithm has time complexity 2" means only that atleast one problem instance of size n requires that much time. Most problem instances might actully require fr less time than that, a situation 1 POLYNOMIAL TIME ALGORITHMS AND INTRACTABLE PROBLEMS 9 ths apyoars fo Sold for several well knows algorithms. Ths simlex a Fithm Tor linear programming has been shown to have exponential time Complenty [Kies tna Minty, 1972], (Zadeh, 1973], but I bas an impressive record of running quickly in pracice. Likewise, branch-and-bound algo ‘ms for she Knapsic prooiem nave been so successiu chat many eonsto er it to be a “wellsalved” problem, even though these algorithms, too, have exponential ume compexiy ‘Unforunately, examples like these are quite rare. Although exponen tial ime algorithms are known fee many problems, fe’ of them are resurd- fdas being very useful in practice. Even the suczsful exponential time ak. forthms mentioned above have not stopped researchers from continuing 10 the very success of these algorithms fas led 10 the suspicion that they somehow expture actual property of the problsme whoze refinement lead to stil better methods. So far, Title progress has been made tovard explaining this success, and no methods ere known for predicting in ad ‘ance that piven exponential time algorithm wil un quickly in practice (On the vier har the miuch more srngent bowls excetion ne satisfied by polynomial time algorithms often permit such predictions 10 be ‘mace. Even though an algorithm raving time complexity n! oF AU mt might not be considered Iikely 10 cun quickly in practice, the polynomially Eeivable problems that arise naturally tend to be solvable within polynomiat time bounds that have degree 2 o¢ 3 at worst and that do not involve ex- tremely large coefcien’s. Algorithms satisfying such bounds can be con sidered to he “orovably efficient.” and iti this much-desred oronertv that ‘makes polynomial inte algorithms the prefered way to solve problems, ‘Dir dainition of “intrartaho™ als proves theoretical Frm of considerable geerality and power. The intracabliy ofa problem turns ut to bo exsenially independent of the paticular encoding scheme and com. pier model used for determining time complet. Lotus fret consider encoding schemes Suppose for example tha we are dealing with a problem in which each instance is a graph © = (VE), trite "i Ue act of verlves wud Eis Un et of edges em ee bing at Unordered pair of veries, Such an instance might be described (eee Figure 14) by simply sung al the veraces and edges, oF oy isting he rows of te adacency matrix for the graph, or by listing for each vertex all the other vertioes sharing a common gage wth i (a "neighbor" Ist. Each of (hese encodings can give a diferent input length forthe same graph. However, it is easy 10 very (ee Figure 18) thatthe input lengths they determine siffer at most oolynomially from one enother so that any algorithm having polynomial time complexity under one of these encoding schemes aso wil Fave polynomial time complexity under all the others. Tn fic, the tandaed fencoding schemes used in practice for any particular problem always seem fednr at moe polynomially from one another Te wuld he efile 0 maging « “reasonable” encoding scheme fora problem that difers more0 ‘COMPUTERS, COMPLEXITY, AND INTRACTABILITY ta poiynnnily from dhe stndard ones. Alinough wat we mean Nere by “reasonable” cannot be formalized, the folowing (wo conditions capture rmueh of tne notion: © dea with unecessary information or symbols, and (2 wmbers occuring in F should be represented in binary (or de Cal oF otal, ein any Fea heen other than 1 If we restrict ourselves to encoding schemes satisfying these conditions, ten the parcular encoding seneme used should not affect the determi tion of whether 2 given problem is itracable Encoding Scions [— ‘Siang Tanah [eres ts, Eage i _| vilviawvislvialiviaviantviavish | 36 [eine ee (wepevnivenceDc) 2% Adaceney maui rows | 0100/1010/0010/0000 » | Figure 14 Descriptions of the graph = (VE) where V = (04.5%) and Ewith. hal? 5), under totes omarentencoingScnemes- Eneoding Sms] Lower Bound Ure Bound) vertex tt Bae tat | ay 4 10e | ay + 10e + (9426) Hoga {ear ao ste incnny mais | view <1 aes dy 4 Be + Yeflonl Figure 5 Gener! baande on input lengths for the tees eneoding schemes of Figute L4 for graphs G = (V6) wih [P |=». [él=e. Since <0 "hase chow that the np lenge flea mast paloma Tro each tee (fl denotes the est ntoger nts than) Similar comments can be made concerning the choice of computer ‘modes. All the realistic models of computers studied 50 fat, Such a5 on2~ ‘ape Turing machines, multitape Turin machines. and randomacoess machines (RAMS), ae equivalent with respect to polysomia ime complex: ity (for example, see Figure 1.6). One would expect any other “reason able” model to share inthis equivalence. "The notion of “reasonable” n= 1 PROVABLY INTRACTABLE PROBLEMS un tended hese is essentially that there is a potynomiat tound on the amount of Work that can be done ina single unit of ime. Thus, for example, a modet Fraving the eapbiioy of performing atbicaily vamp operations i pale ‘wouKS not be considered "reasonable." and indeed no existing (or planied) Computer nas tis capaouy. At any fate, so Long as we resirit urseives to the standard models of realistic computers, the clss of intractable problems will be uraffeced by the particular model used, and we can make oUt coige on the basis of convenience without sacifiing the applicability of fur resus | | Sirlaed machine B iw 1-Tape Tung Machine dT) |_| acre | ocronmerian | cies Tres ary | ocr |= | ocrnari | 007%) | (Tm) 7 | Random Access Machine (RAM) Figure 1.6 Time casted by machine A fo simulate the execution of an algorithm ttime compleniy T(n) on Mactive 8 (or exarale, see [opcatt, nd Ullman, 1969] and (Aho, Hope, and Utinan, 1973) 1.4 Provably Tntractable Problems Now that we have discussed the formal meaning of “intractable prob tem,” iti appropriate that we ify sry the curent ste of knowiedge about the existence of intractable problems, Tis ueful to begin by distinguishing between two diffrent causes of Intractability allowed by our definition. The fst, which isthe one we ust- ally have in ming, Is eat she problem i 50 dificult that an exponent amount of time is needed to discover a solution. The second is that che Soto soei/is tequited Wo be so extensive thai exo he described wih ‘an expression having length bounded by a polynomial function of the input engin This second cause occurs, for example, in the variant of the traveling salesman problem teat cludes a numer as an addtional parameter and that asks for aif tours having total length B or les. Its easy to consttuct instances of this problem in which expoaentally many cours are shorter than the kiven bound, so that no polynomial ime aleorithm could posibhy list tiem all, Tracability of this sorts by n0 means insianifiant, and its impor. tant fo recognize it when it occurs, However, in most cass its existence Isn (COMPUTERS, COMPLEXITY, AND INTRACTABILITY sparen ho the prem detnn, In fas, ths ype of sas can be regarded asa signal thatthe problem i not defined realistically, because ‘we are asking for more infoemasion than we could ever hope to wse. Thus, from now on we shall estrct our attention tothe fist type ofintretailiy ‘According. ony problems for which the solution lengih 1s bounded by & polynomial function of the input length wil be considered. ‘The eales intractabilty resus fr such problems ate the classical un ecidabiity cesults of Alan Turing. Over 40 Years no, Turing demonstrated that certain problems are 30 hard that they are "undecidable,” in the sense ‘that no algorithm at all canbe given for solving ther. He proved. for ex- ample, that it is impossible to specify any algorithm which given an arbi- Whether oF not the program will eventually halt when applied to tha input (Turing, 1996). A varity of cther problems ars now known o be undecid able, including the tcivality problem for frtely presented groups (Rabin. 1958], Thlberts tenth problem (solvabilty of polynomial equations in in togers) Matjasevie, 1970], and several peoblems of “tiling the plane Berger, 1966. Since these uneeidsbie problems cannot be solved by any tlgortim, much less polynomial time algorithm, they indeed are intact able in am especialy strong sense. “The fst examples of intractable “decidable” problems vere obtained in the early 1960'S, as part of work on complexity “hierarchies” by Hart- manis and Stearns (1965). However these results involved only “artic problems, speiiclly constructed to have the appropiate properties. It was Ft unit the earl 1970 thst Mever and Stnckmever 119721. Fischer and Rabin 1974], and others finally succeeded in proving some “natural” de~ Cidable problems to be intncable. Thess inside s variety oF previously Studied ‘problems from automata theory, formal language theory, and ‘mathemati! logic. In fat, the roots show that these problems cennot be solved in polynomial time using even a “nondeterminstic” computer rode which has the silty to pursue an unbounded numberof indcpen- ‘ent computtional sequences in parallel, We shal see that this ““unreason- fable” computer model plays am important role in he theory of NP- Completeness, and is capabilities wil be specified more fully in Chapter 2, ‘Al te prOvably intractable problems known {0 dae fall into tne (Wo categories we have just mentioned. They ate ether undecidable or “non- Seterminisially” unretabe, However, most oF the apparently intractable problems encountered in practice are decidable and can be solved in poly- Fomial time withthe aid-of a nondeterministc computer. Thus, none of the proof techniaues developed so far is powerful enough to verify the ao- parent intracibity ofthese problems. 1 NP-COMPLETE PROBLEMS B 1.5 NP-Complete Problems ‘As theoreticians wntinue 19 seek more powerful methods For prow problems intractable, parle efforts focus on learning more about the ways In whieh venous problems are snteretated with respect fo ther aificuty. ‘AS we suggested earler, the discovery of such relationships between probs lems often ean provi information useful to algorithm designers, “Tne princinal techniaue used for demonstrating that two problems are related is that of “reducing” one to the other, By giving a constructive lransformation that maps any instance of the fst problem into an feivalent instance of the second. Such a transformation provides the corresponding algorithm for solving the fis problem, Many simple examples of such reductions have been known fo time. For example, Dantzig [1960] reduced a numberof combinatril op- timation problems to the generat zoro-one Integer linear programming problem. Eamonds (1962) reduced the graph theoretic problems of "cover. Ing alleges with = minimum umber of verfees” and “finding « max imum independent set of verces” (othe general “set covering problem. Gimpel 989) teceed the general st covering problem fo tne prime Im- Plicant covering problem’ of loge design. Danzig, Biatiner, and. Rao [1966] described a "welkknown’™ reduction from the traveling salesmen problem tothe “shortest path problem” with negative edge lengths allowed “These early reductions, although rather isolated and limited in scope, ‘oceshadow the kind of resitsaroved in the theory of NPcomblateness "The foundations forthe theory of NP-completeness were ai ina paper ‘of Stephan Conk, presented in 1971 enilad “The Complesty of Theorem Proving Procedures” (Cook, 19TIai. Ip this brie but elepant paper Cook id naveral important thing Fist, he emphasized the significance of “polynomial time redusbility,” that ia, ridetions for which the required transformation can be executed by ® polynomial time algorithm. If we have a polynomial time reduction from fe problem so another, this ensues that any polyoma tie aloe for the second problem can be converted into a corresponding polynomial time lgoranm forthe nist prose. ‘Second, he focused attention on the class NP of decsion problems that can be solved in polynomial time by a nondeterminisie computer. (A deci ‘sion problem is one whose solution is either “yes” or “no") Most of the Apparently iatacable problems encountered in practice, When phrased as ‘decision oroblems. Delos to this ass. hid, he proved that one particular problem in NP. called the “catsicility™ problem, hoe the property that every other aber in NP can be polynomially reduced toh he satishabiliy problem can be solved tithe polynomial time algorithm, then sean every problem in NP. and if any problem in NP Is intactable, then the satisfability problem also must4 ‘COMPUTERS, COMPLEXITY, AND INTRACTABILITY te incest. Th, in ene, te sibility problem i Une “hands” problem in NP. Finally, Cook suggested that other problems in NP might share with the satsfabilty problem this property of being the “hardest” member of INP. He showed this 1 be the cae for the problem “Does a given graph G contain 2 complete subgraph ona given number & of vertices?” Subsequently, Richard Karp presented a collection of results [Karp 1oT2) proving shat ined the dedsion problem versions of many wel known ‘combinatorial problems, including the traveling salesman problem, arent ae "hae" ag the etieSabityprofler Since then a wide variety of dther problems have been proved equivalent in dificult to these problems, been given aname: the dass of NP-complete roblems, ‘Cook's oFigina ideas have tuvsed wut to be veratkably powerful. They hhave provided the means for combining many individual complexity ques Hons into the single question. Are the NP-complete problems itractebe? ‘The lists included inthe Appendix ofthis book conan literally hundreds of aierent problems now known to be NP-complete. AS mote and mote problems of independent imerest are shown to belong Co this equivalence 8s, importance is continually cenforced. “The question of whether or not the NP-complete problems are intract= able is now considered tobe one ofthe foremost open Guestons of contem- porary mathematics and computer science. Despite the wilingness of most researchers to coajecture that the NP-complete problems areal intractable, Title progres hac vot hoon mari inward atahishingeithor a nrnaa proof of this far-reaching conjecture, However, even without a proof that NP-compleeness implies intrstabily, the knowledge that » problems ie NP-complete suggess, a the very lest, that a major breakthrough wil be rnweded to wove it witha polynomial time algorithm. 1.6 An Outline of the Book Although this book i intended mainly as primer on how to determine ‘whether ot not any particular problem is NF-complete either by looking it Lup in the lisis we present or by proving it yoursei), we shall aso discuss Some ofthe options available for dealing with a problem that is known to be ‘NP-complete. A brief outline of subseauent chapters follows. In Chapter 2, we preset the formal underpinnings of NP-completeness and prove Cook's theorem. The central definitions involve certain theorti= cal concenis, such as “languages” and “Turing: machines,” which we develop ins srsipheorwart manner, relating them tothe notions nf poke lems and computer models alteady discussed. This chapter should give the reader a good understanding of the technieal meaning of NP.completeness 6 AN OUTLINE OF THE 800K Is Chapter 3 is devoted fo methods for proving » problem NP-complete ‘A umber of examples are presented to illustrate the usual structure of such proufs, and to indeste bow one goes about generating one. In essence, one proves a new problem to be NP-complete by polynomially feauoing a known Nr-compiete probiem Io 1. we survey the known NI ome problems that ase bcm mos useful for hs purpose nd een ‘sate their Use In Chapter 4, we examine the wars in which the theory of NP- completeness can be used for conducting 2 detailed analysis of the complex- ity of problem, seeking to determine the "boundary" between those cases fof the problem that are polynomially solvable and those that are NP- In Chapter $, we show how the techniques used for proving NP- completeness can be generalized 20 that probleme other than just decision problems can be proved 'o be as hard as" the NP-complete problems. AS [tn id to reading the published Iterture onthe theory of NF-completeness ‘we also provide a brit historical survey of the development of the maln| Teas and the varying terminology that has been used for ascussng them In Chapter 6, we discuss several approaches for deeting with intractable problems, especialy that of fining near-optimal solutions using fast ago Fithms. Examples of the successes and fulures of each approach are ‘escrbed, and we ilustrate how the thedry of NP-competeness can be a= plied even here Chapter 7 is intended to acquaint the reader with some ofthe theoet- cal igsoes and ideas that have arisen in taal wth the theory of NP- Completeness. Among other topics we discuss the polynomial hierarchy, {P-competeness,palynomiat space completeness, nd the “relaiviration”™ ofthe question af the itractbiiy of the NP-complete problems "The lat thied of the book congists of the Appendix, an sxtansive and annotated lst oF problems known to be NP-complete of harder. The lit Is divided into sections, each devoted to problems {roms particular subject fare, suchas graph theory, scheduling algebra and number theory, covering Sid patiloning, watiemmtiat progtanining, pram oplimication, sate ‘ata an language theory, end, ofcourse, miscellaneous topics. The list in ‘luge reterences to fisted problems krowa to be solvable in pasynomi lime and to problems whose status cemeins open in tht neither polynomial time algorithms nor NP-completeness proofs are krown for them2 The Theory of NP-Completeness In this chapter we present the formal details of the theory of NP- completeness. So that the theory can be defined in a mathematically rigorous way, i willbe necessary to iniroduce formal counterparts for many Of our informal notions, such as “problems” and “algorithms.” Indeed, fone of the main goals of this chapter is to make exolcit the connection between the formal terminology and tne more Intutve, informal shorthand that is commonly used in is place. Once we have this connection well in hand, it will be posible for us 1o- pursue our discussions primarily at the informal Tove in Inter chapters, reverting Io the formal level only when necessary for eatity and rigor. “The chapter begins by & tan a Tanguages” equating "sovine« ison problem with “recor ‘duced as our base model for computation and is used 10 define the clas P Of all biguiges revigniaable ueterminitelly I poiynomlal me. This model is then augmented with a hypothetical “guessing” ability, and the augmented models used 10 denne the ciss Not ell languages recogniz- able “nondeterministcall” in polynomial time. After discussing the rela- Uonship between P and NP, We define he notion of a polynomial tansfor- ration from one language to another and use it to define what will be our1B ‘THE THEORY OF NP-COMPLETENESS most important cis, she eass of NP-complete probiems. ‘The chapter con ludes with the statement and proof of Cook's fundamental theorem, which Drovides us wit our fst oona fide N-compiete problem, 2.1 Decision Problems, Laneuazes. and Encoding Schemes AS mater of enmuenianes the theoey of NP.completonnss i designed tp be np ory 0 dein problems Suen poblms, a mentined in pte I. have ony to possible solutions either the answer “jes” or the answer “no.” Abstaty, «decision problem I consists simply of set Dy problems of interes possess « considerable amount of edtonal structure, And we will desu thon in way tal erases thy state, The Standard format we will use for specifying problems consists of two pats, ihe fst part specifying genet nance ofthe problem In terms Of ¥arlous| components, which are sets, gtaphs, Tunctions, numbers, etc, and the second part stating 2 yes-no question aed in erm ofthe Beneic stance ‘The way in which this specfes Dy and Yp should be apparent. An insance belongs to Dy if and only if ean be obained from the generic instance By substituting particular objects of the specified types forall the generic com ponens, and the instance belongs to ¥, if and ony ifthe answer forthe slated qUestion, when paricularized to that instance is “ves.” roof exe, the folowing desribes 1 weleinown dion problem SUBGRAPH ISOMORPHISM INSTANCE: Two graphs, G,~ (PE) and Gy (VE) QUESTION: Does contain » ser fora 18, tat ie» Set GV; and a sobset E'GE; such tat |= [Vl [| =LEsh and there frns« one-one function J:Ver¥" eating [sys]€ Ey iad oly YW@SO) EE? [A decision problem elated 0 the traveling salesman problem cin be described falls ‘TRAVELING SALESMAN INSTANCE: (A finite set C=1e;,¢q,-. Gq} of “ches,” @ "distance" lex.) EZ" foe each rit of iin» 8. sl han RE 7 Chews 7> denotes the postive integers). OUESTION Te there 9 tor” ofall the ciie is C having total length no ‘more then B, thats, an ordering of C such that F alegnescein| + degree) < BP 21 DECISION PROBLEMS, LANGUAGES, AND ENCODING SCHEMES 9 ‘The tees will Bod many move eranpies of the use Of tis fun throughout the book. but these two should sulle for now to convey the basi tes. The seeond example eso serves to illetate an imporant point about how a decision problem cin be derived from an optimization prob- Jem. I! the optimization problem asks fora structure ota certain type that has minimum “cost” among all such sructores (for example, @ tour that has minimum length among all ours), we can associate with that problem the decision problem that includes 2 rumercal bound 3s an additional Daremeter und that asks whether there rissa structure ofthe required type having cost no more thin B (Tor example. & tour of length no more than 1B). Decision problems can be derived from maximization problems in an The key point to observe about this correspondence is that, so long as th cost fuetion is relatively easy to evaluete, the decision problem can be no harder than the corresponding optimization problem, Clearly, if we {ould find = minim length tout forthe traveling slesman problem in Bo= Iynomial time, then we could also solve the associated decsion problem in Polynomial me. All we nee dof id the minimum length tour, compte Its Tergth, and compare that length to the given found 8. Thus, if we could demonstrate that IKAVELING SALESMAN i Nt-compete (as Indeed its), we would know thatthe taveling salesman optimization prob- Jem is at Teast as hard. Ip this way, even though the theory of NP- completeness restricts attention to only decision problems, We can extend the implications of the theory to optimization problems as well. (We shall ‘seein Chanter 5 that decision oroblems and ootimization orablems are often even more closely tied: Many decsion probiems, including TRAVELING: SATFSMAN, cam alan he shown fo he “na exsor shan thelr coeresnanding oplimization problems.) “The reason for the restriction t0 decision problem is that they have @ very natura, formal counterpart, which isa Sulable object to study in & ‘mathematically precise theory of computation. This counterpart is alld “Tanguage™ and is defined inthe folowing way. For any ine set E uf ytibuisy we Uete by Z* dhe set of ai finite strings of symbols from E. For example, if £01], then £* consists of fe empty sing "the stings 0,1,00,01,10,11,000,001 and al ther Gite strings of O'S and I's. TEL is a subset of 2°, wo say that Lisa language over the siphaber 2. Thus {01,001,111,1101010} 6 a language over (0,1), ais the set of all binary representations of integers that are per- fect squares, a isthe st 0.1)" se The correspondence between decision problems and lansuages is bought about by the encoding schemes we use for specifying problem in tances whenever we intend 10 compute with them. Recall thal an encoding Scheme e fora problem Il provides a way of describing each instance of Tt hy an appenpriate sting of Simos over some Fixed alphabet Thus the problem Il and the encoding sthere efor TI partition E* into three classesFo ‘THe THEORY OF NP-cOMPLETENESS of stings: those hat are nut envodings of instaioes Of TT, thse Ut encode Instances of TT foe which the answer is “no,” and those that encode in Sarees of UL for Which the answer iS “yes.” "TIS Ua els of Sings is the language we associate with (Land ¢ setting is the alphaber used by e, and x i te TIMLel = LX" pein under € of a instance 18 Ty, ae formal shor i plied to by saying that if at oie forthe language Life, then it hols Tor the pobiem fl Under te fnosing heme Tn fit, we shal usualy follow standard practice and bea it more ine ‘onal wn ths. Each te we inoouee unex concep in erm OF Tanguages, Wl observe thatthe property ese encoding indepen- Gen #0 ong as we rein! ouelves 12 resonable" cocoa seme ‘That fi e-and "ate any (wo resonable encoding schemes for then the propery Ros ether for both Lilie} and Lille} oe for nether” Tas ‘wl aw us to sy, informally hat he property hols or doesnot hold forthe problem f, without actually spesiing ary encoding scheme. HOW ve, whenever we doo, the impli aero wil be tha we eau c= esta specify = paral resonable encoding scheme ¢ sh that the froneny brs for Ele ‘Noie that when we operate inthis encodingindepedent manne, we Jose conta with any precee nation of "input length” Sine we ee some parameter in terms of which time comple can be expresied, ti con independent function Length: Dj 2". which is {cael i tbe tpl lerglis we woth cbt hn a seasonal eroding Scheme. By pobmomaly relted we mean tha, for any razonable encoding Scheme e for th tere exis two poynomias pane Psueh tat 7 € De ‘and x is a string encoding the instance / under e, then Length [1] < p(|x|) and |x| p'(Length [/1), where |x| denotes the length of the string x, In the SUBGRAPH ISOMORPHISM problem, for example, we might tke engin] = [7 + where i= (Vali) and Gy (VE) ace the graphs making uP an ine Stance. In the TRAVELING SALESMAN decon problem we might ake Length 1 = m+ fog + max fora, 6)I-¢.6C) Sine any two reasonable encoding cheres fr aproblem T wl il ply Tay relates input engi Nie varey of Length funeons te poe bie for and all our resus wil ery through fr any such Function tna, ‘The usefulness of this informal, enchding-independent approach éepends, of course, on ere being some wgreemen Hi whl consiutes 111 DECISION PROBLEMS, LANGUAGES. AND ENCODING SCHEMES a sonable” includes both the notion of “eoneisenes,” es captured by the (wo onan mensoned in Chap 1, und whe notion of decoaubly."- The ient of “conciseness” iS that iasances ofa problem should be described withthe nalural brevity We Would use In scully spectying tnose nstances Fors computer, without any unnatural "pacing" of the input, Such pad ding could be used, for example, 10 expand the input length so much that ‘we allially convert an exponential ime algorithm into @ polynomial time fligrithm, The intent of "decodabity™ i thal, given any parueular com ponent of » generic instance. one should be able to specify e polynomial Time algorithm tht is capable of extracting a desription of that component (Of course, these elaborations donot provide a formal definition of rable encoding scheme,” aad we know of fo aitsfacton’ way of making such a definition. Even though most people would agree on wheth- ror not particular encoding scheme for a piven problem is reasonable the absence of a formal definition can be somewhat iscomforing, “One ‘way of resolving this difculy would be co require tha genere problem in- stances always be formed from a fied collection of basle pes of set- Theoretic obeexs We will not impose Such a eonstraintnete, Bu, as an in- cation of our intent when we refer to “teasonable encoding Schemes,” We ‘now give a brief description (which fst time feaders may wish 10 skip) OF how Such a standard encoding scheme could be defined Gur standerd encoding scheme wil map instances into “structured csrings” quar the alnhabet VoI0.1.-.01.0)..1. We define srucured strings recursively, 2 follows (1) The binary cepresonaton of an intoger Kas a string of 0's and 1s (preceded by a minus sign "if A ie negative) i a truce tured srng representing the integer k (2) tx isa structured sing representing the integer k, then Ux] is @ structured string that can be used 28 “name” (ot exemle, for ‘vertex in a graph, a set element, of acy In a traveling siles> tan inane) (©) xysys coc q ate structured stings representing the obiests Aykgee eed MeN AR, vg) 1S a suture string representing the sequence - To derive an encoding. scheme for 4 particular decision problem specified In our standard formal, we fst note that, once we have bull up & epeesentation for each object in an instance asa sructured string the representation of the enlirrinsance is determined using rule (3) above, ‘Thue we neal Gly specify how the representation for each typeof abject i constructed, For this we shall resect ourselves to integers, “unstructured