Algorithms and Combinatorics Volume 21
Editorial Board R. L. Graham, La Jolla B. Korte, Bonn L. Lovász, Budapest A. Wi...

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!

Algorithms and Combinatorics Volume 21

Editorial Board R. L. Graham, La Jolla B. Korte, Bonn L. Lovász, Budapest A. Wigderson, Princeton G. M. Ziegler, Berlin

Bernhard Korte · Jens Vygen

Combinatorial Optimization Theory and Algorithms

Fourth Edition

123

Bernhard Korte Jens Vygen Research Institute for Discrete Mathematics University of Bonn Lennéstraße 2 53113 Bonn Germany [email protected] [email protected]

ISBN 978-3-540-71843-7

e-ISBN 978-3-540-71844-4

DOI 10.1007/978-3-540-71844-4 Algorithms and Combinatorics ISSN 0937-5511 Library of Congress Control Number: 2007936375 Mathematics Subject Classification (2000): 90C27, 68R10, 05C85, 68Q25 © 2008 Springer-Verlag Berlin Heidelberg This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilm or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer. Violations are liable to prosecution under the German Copyright Law. The use of general descriptive names, registered names, trademarks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. Typesetting: by the authors using a Springer TEX macro package Production: LE-TEX Jelonek, Schmidt & Vöckler GbR, Leipzig Cover design: WMX Design GmbH, Heidelberg Printed on acid-free paper 987654321 springer.com

Preface to the Fourth Edition

With four English editions, and translations into four other languages forthcoming, we are very happy with the development of our book. Again, we have revised, updated, and significantly extended it for this fourth edition. We have added some classical material that may have been missed so far, in particular on linear programming, the network simplex algorithm, and the max-cut problem. We have also added a number of new exercises and up-to-date references. We hope that these changes serve to make our book an even better basis for teaching and research. We gratefully acknowledge the continuous support of the Union of the German Academies of Sciences and Humanities and the NRW Academy of Sciences via the long-term research project “Discrete Mathematics and Its Applications”. We also thank those who gave us feedback on the third edition, in particular Takao Asano, Christoph Bartoschek, Bert Besser, Ulrich Brenner, Jean Fonlupt, Satoru Fujishige, Marek Karpinski, Jens Maßberg, Denis Naddef, Sven Peyer, Klaus Radke, Rabe von Randow, Dieter Rautenbach, Martin Skutella, Markus Struzyna, J¨urgen Werber, Minyi Yue, and Guochuan Zhang, for their valuable comments. At http://www.or.uni-bonn.de/∼vygen/co.html we will continue to maintain updated information about this book. Bonn, August 2007

Bernhard Korte and Jens Vygen

Preface to the Third Edition

After five years it was time for a thoroughly revised and substantially extended edition. The most significant feature is a completely new chapter on facility location. No constant-factor approximation algorithms were known for this important class of NP-hard problems until eight years ago. Today there are several interesting and very different techniques that lead to good approximation guarantees, which makes this area particularly appealing, also for teaching. In fact, the chapter has arisen from a special course on facility location. Many of the other chapters have also been extended significantly. The new material includes Fibonacci heaps, Fujishige’s new maximum flow algorithm, flows over time, Schrijver’s algorithm for submodular function minimization, and the Robins-Zelikovsky Steiner tree approximation algorithm. Several proofs have been streamlined, and many new exercises and references have been added. We thank those who gave us feedback on the second edition, in particular Takao Asano, Yasuhito Asano, Ulrich Brenner, Stephan Held, Tomio Hirata, Dirk M¨uller, Kazuo Murota, Dieter Rautenbach, Martin Skutella, Markus Struzyna and J¨urgen Werber, for their valuable comments. Eminently, Takao Asano’s notes and J¨urgen Werber’s proofreading of Chapter 22 helped to improve the presentation at various places. Again we would like to mention the Union of the German Academies of Sciences and Humanities and the Northrhine-Westphalian Academy of Sciences. Their continuous support via the long-term project “Discrete Mathematics and Its Applications” funded by the German Ministry of Education and Research and the State of Northrhine-Westphalia is gratefully acknowledged. Bonn, May 2005

Bernhard Korte and Jens Vygen

Preface to the Second Edition

It was more than a surprise to us that the first edition of this book already went out of print about a year after its first appearance. We were flattered by the many positive and even enthusiastic comments and letters from colleagues and the general readership. Several of our colleagues helped us in finding typographical and other errors. In particular, we thank Ulrich Brenner, Andr´as Frank, Bernd G¨artner and Rolf M¨ohring. Of course, all errors detected so far have been corrected in this second edition, and references have been updated. Moreover, the first preface had a flaw. We listed all individuals who helped us in preparing this book. But we forgot to mention the institutional support, for which we make amends here. It is evident that a book project which took seven years benefited from many different grants. We would like to mention explicitly the bilateral HungarianGerman Research Project, sponsored by the Hungarian Academy of Sciences and the Deutsche Forschungsgemeinschaft, two Sonderforschungsbereiche (special research units) of the Deutsche Forschungsgemeinschaft, the Minist`ere Franc¸ais de la Recherche et de la Technologie and the Alexander von Humboldt Foundation for support via the Prix Alexandre de Humboldt, and the Commission of the European Communities for participation in two projects DONET. Our most sincere thanks go to the Union of the German Academies of Sciences and Humanities and to the Northrhine-Westphalian Academy of Sciences. Their long-term project “Discrete Mathematics and Its Applications” supported by the German Ministry of Education and Research (BMBF) and the State of Northrhine-Westphalia was of decisive importance for this book. Bonn, October 2001

Bernhard Korte and Jens Vygen

Preface to the First Edition

Combinatorial optimization is one of the youngest and most active areas of discrete mathematics, and is probably its driving force today. It became a subject in its own right about 50 years ago. This book describes the most important ideas, theoretical results, and algorithms in combinatorial optimization. We have conceived it as an advanced graduate text which can also be used as an up-to-date reference work for current research. The book includes the essential fundamentals of graph theory, linear and integer programming, and complexity theory. It covers classical topics in combinatorial optimization as well as very recent ones. The emphasis is on theoretical results and algorithms with provably good performance. Applications and heuristics are mentioned only occasionally. Combinatorial optimization has its roots in combinatorics, operations research, and theoretical computer science. A main motivation is that thousands of real-life problems can be formulated as abstract combinatorial optimization problems. We focus on the detailed study of classical problems which occur in many different contexts, together with the underlying theory. Most combinatorial optimization problems can be formulated naturally in terms of graphs and as (integer) linear programs. Therefore this book starts, after an introduction, by reviewing basic graph theory and proving those results in linear and integer programming which are most relevant for combinatorial optimization. Next, the classical topics in combinatorial optimization are studied: minimum spanning trees, shortest paths, network flows, matchings and matroids. Most of the problems discussed in Chapters 6–14 have polynomial-time (“efficient”) algorithms, while most of the problems studied in Chapters 15–21 are NP-hard, i.e. a polynomial-time algorithm is unlikely to exist. In many cases one can at least find approximation algorithms that have a certain performance guarantee. We also mention some other strategies for coping with such “hard” problems. This book goes beyond the scope of a normal textbook on combinatorial optimization in various aspects. For example we cover the equivalence of optimization and separation (for full-dimensional polytopes), O(n 3 )-implementations of matching algorithms based on ear-decompositions, Turing machines, the Perfect Graph Theorem, MAXSNP-hardness, the Karmarkar-Karp algorithm for bin packing, recent approximation algorithms for multicommodity flows, survivable network design

XII

Preface to the First Edition

and the Euclidean traveling salesman problem. All results are accompanied by detailed proofs. Of course, no book on combinatorial optimization can be absolutely comprehensive. Examples of topics which we mention only briefly or do not cover at all are tree-decompositions, separators, submodular flows, path-matchings, deltamatroids, the matroid parity problem, location and scheduling problems, nonlinear problems, semidefinite programming, average-case analysis of algorithms, advanced data structures, parallel and randomized algorithms, and the theory of probabilistically checkable proofs (we cite the PCP Theorem without proof). At the end of each chapter there are a number of exercises containing additional results and applications of the material in that chapter. Some exercises which might be more difficult are marked with an asterisk. Each chapter ends with a list of references, including texts recommended for further reading. This book arose from several courses on combinatorial optimization and from special classes on topics like polyhedral combinatorics or approximation algorithms. Thus, material for basic and advanced courses can be selected from this book. We have benefited from discussions and suggestions of many colleagues and friends and – of course – from other texts on this subject. Especially we owe sincere thanks to Andr´as Frank, L´aszl´o Lov´asz, Andr´as Recski, Alexander Schrijver and Zolt´an Szigeti. Our colleagues and students in Bonn, Christoph Albrecht, Ursula B¨unnagel, Thomas Emden-Weinert, Mathias Hauptmann, Sven Peyer, Rabe von Randow, Andr´e Rohe, Martin Thimm and J¨urgen Werber, have carefully read several versions of the manuscript and helped to improve it. Last, but not least we thank Springer Verlag for the most efficient cooperation. Bonn, January 2000

Bernhard Korte and Jens Vygen

Table of Contents

1

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.1 Enumeration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.2 Running Time of Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 1.3 Linear Optimization Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 1.4 Sorting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

2

Graphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1 Basic Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Trees, Circuits, and Cuts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3 Connectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4 Eulerian and Bipartite Graphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.5 Planarity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.6 Planar Duality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

13 13 17 24 30 33 40 43 46

3

Linear Programming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1 Polyhedra . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 The Simplex Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 Implementation of the Simplex Algorithm . . . . . . . . . . . . . . . . . . . . 3.4 Duality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5 Convex Hulls and Polytopes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

49 50 54 57 60 64 66 68

4

Linear Programming Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1 Size of Vertices and Faces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Continued Fractions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3 Gaussian Elimination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4 The Ellipsoid Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.5 Khachiyan’s Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.6 Separation and Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

71 71 74 77 80 86 88 95 96

XIV

Table of Contents

5

Integer Programming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 5.1 The Integer Hull of a Polyhedron . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 5.2 Unimodular Transformations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 5.3 Total Dual Integrality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 5.4 Totally Unimodular Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 5.5 Cutting Planes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 5.6 Lagrangean Relaxation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125

6

Spanning Trees and Arborescences . . . . . . . . . . . . . . . . . . . . . . . . . . . 127 6.1 Minimum Spanning Trees . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127 6.2 Minimum Weight Arborescences . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133 6.3 Polyhedral Descriptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137 6.4 Packing Spanning Trees and Arborescences . . . . . . . . . . . . . . . . . . . 140 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147

7

Shortest Paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151 7.1 Shortest Paths From One Source . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152 7.2 Shortest Paths Between All Pairs of Vertices . . . . . . . . . . . . . . . . . . 156 7.3 Minimum Mean Cycles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163

8

Network Flows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165 8.1 Max-Flow-Min-Cut Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166 8.2 Menger’s Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170 8.3 The Edmonds-Karp Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172 8.4 Blocking Flows and Fujishige’s Algorithm . . . . . . . . . . . . . . . . . . . . 174 8.5 The Goldberg-Tarjan Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176 8.6 Gomory-Hu Trees . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180 8.7 The Minimum Capacity of a Cut in an Undirected Graph . . . . . . . 186 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194

9

Minimum Cost Flows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199 9.1 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199 9.2 An Optimality Criterion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201 9.3 Minimum Mean Cycle-Cancelling Algorithm . . . . . . . . . . . . . . . . . . 203 9.4 Successive Shortest Path Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . 207 9.5 Orlin’s Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211 9.6 The Network Simplex Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214 9.7 Flows Over Time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224

Table of Contents

XV

10

Maximum Matchings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227 10.1 Bipartite Matching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228 10.2 The Tutte Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230 10.3 Tutte’s Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232 10.4 Ear-Decompositions of Factor-Critical Graphs . . . . . . . . . . . . . . . . . 235 10.5 Edmonds’ Matching Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254

11

Weighted Matching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257 11.1 The Assignment Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258 11.2 Outline of the Weighted Matching Algorithm . . . . . . . . . . . . . . . . . 259 11.3 Implementation of the Weighted Matching Algorithm . . . . . . . . . . 262 11.4 Postoptimality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276 11.5 The Matching Polytope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 280 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282

12

b-Matchings and T -Joins . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285 12.1 b-Matchings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285 12.2 Minimum Weight T -Joins . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289 12.3 T -Joins and T -Cuts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293 12.4 The Padberg-Rao Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 296 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302

13

Matroids . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305 13.1 Independence Systems and Matroids . . . . . . . . . . . . . . . . . . . . . . . . . 305 13.2 Other Matroid Axioms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309 13.3 Duality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313 13.4 The Greedy Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 317 13.5 Matroid Intersection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 322 13.6 Matroid Partitioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 327 13.7 Weighted Matroid Intersection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 329 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 332 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 334

14

Generalizations of Matroids . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 337 14.1 Greedoids . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 337 14.2 Polymatroids . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 341 14.3 Minimizing Submodular Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . 345 14.4 Schrijver’s Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 347 14.5 Symmetric Submodular Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . 351 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 355

XVI

Table of Contents

15

NP-Completeness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 359 15.1 Turing Machines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 359 15.2 Church’s Thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 362 15.3 P and NP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 367 15.4 Cook’s Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 371 15.5 Some Basic NP-Complete Problems . . . . . . . . . . . . . . . . . . . . . . . . . . 375 15.6 The Class coNP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 382 15.7 NP-Hard Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 387 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 391

16

Approximation Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393 16.1 Set Covering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 394 16.2 The Max-Cut Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 399 16.3 Colouring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 405 16.4 Approximation Schemes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 413 16.5 Maximum Satisfiability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 415 16.6 The PCP Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 420 16.7 L-Reductions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 424 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 430 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 434

17

The Knapsack Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 439 17.1 Fractional Knapsack and Weighted Median Problem . . . . . . . . . . . 439 17.2 A Pseudopolynomial Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 442 17.3 A Fully Polynomial Approximation Scheme . . . . . . . . . . . . . . . . . . 444 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 447 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 447

18

Bin-Packing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 449 18.1 Greedy Heuristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 449 18.2 An Asymptotic Approximation Scheme . . . . . . . . . . . . . . . . . . . . . . 455 18.3 The Karmarkar-Karp Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 459 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 462 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 464

19

Multicommodity Flows and Edge-Disjoint Paths . . . . . . . . . . . . . . . . 467 19.1 Multicommodity Flows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 468 19.2 Algorithms for Multicommodity Flows . . . . . . . . . . . . . . . . . . . . . . . 471 19.3 Directed Edge-Disjoint Paths Problem . . . . . . . . . . . . . . . . . . . . . . . . 476 19.4 Undirected Edge-Disjoint Paths Problem . . . . . . . . . . . . . . . . . . . . . . 479 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 485 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 487

Table of Contents

XVII

20

Network Design Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 491 20.1 Steiner Trees . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 492 20.2 The Robins-Zelikovsky Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 497 20.3 Survivable Network Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 502 20.4 A Primal-Dual Approximation Algorithm . . . . . . . . . . . . . . . . . . . . . 505 20.5 Jain’s Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 514 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 520 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 522

21

The Traveling Salesman Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 527 21.1 Approximation Algorithms for the TSP . . . . . . . . . . . . . . . . . . . . . . 527 21.2 Euclidean TSP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 532 21.3 Local Search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 539 21.4 The Traveling Salesman Polytope . . . . . . . . . . . . . . . . . . . . . . . . . . . . 545 21.5 Lower Bounds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 551 21.6 Branch-and-Bound . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 553 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 556 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 559

22

Facility Location . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 563 22.1 The Uncapacitated Facility Location Problem . . . . . . . . . . . . . . . . . 563 22.2 Rounding Linear Programming Solutions . . . . . . . . . . . . . . . . . . . . . 565 22.3 Primal-Dual Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 567 22.4 Scaling and Greedy Augmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 573 22.5 Bounding the Number of Facilities . . . . . . . . . . . . . . . . . . . . . . . . . . . 576 22.6 Local Search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 579 22.7 Capacitated Facility Location Problems . . . . . . . . . . . . . . . . . . . . . . . 585 22.8 Universal Facility Location . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 588 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 594 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 596

Notation Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 599 Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 603 Subject Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 613

1 Introduction

Let us start with two examples. A company has a machine which drills holes into printed circuit boards. Since it produces many of these boards it wants the machine to complete one board as fast as possible. We cannot optimize the drilling time but we can try to minimize the time the machine needs to move from one point to another. Usually drilling machines can move in two directions: the table moves horizontally while the drilling arm moves vertically. Since both movements can be done simultaneously, the time needed to adjust the machine from one position to another is proportional to the maximum of the horizontal and the vertical distance. This is often called the `∞ distance. (Older machines can only move either horizontally or vertically at a time; in this case the adjusting time is proportional to the `1 -distance, the sum of the horizontal and the vertical distance.) An optimum drilling Pn−1path is given by an ordering of the hole positions p1 , . . . , pn such that i=1 d( pi , pi+1 ) is minimum, where d is the L ∞ -distance: for two points p = (x, y) and p 0 = (x 0 , y 0 ) in the plane we write d( p, p 0 ) := max{|x − x 0 |, |y − y 0 |}. An order of the holes can be represented by a permutation, i.e. a bijection π : {1, . . . , n} → {1, . . . , n}. Which permutation is best of course depends on the hole positions; for each list of hole positions we have a different problem instance. We say that one instance of our problem is a list of points in the plane, i.e. the coordinates of the holes to be drilled. Then the problem can be stated formally as follows:

D RILLING P ROBLEM Instance:

A set of points p1 , . . . , pn ∈ R2 .

Task:

Find a permutation π : {1, . . . , n} → {1, . . . , n} such that P n−1 i=1 d( pπ(i) , pπ(i+1) ) is minimum.

We now explain our second example. We have a set of jobs to be done, each having a specified processing time. Each job can be done by a subset of the employees, and we assume that all employees who can do a job are equally efficient. Several employees can contribute to the same job at the same time, and one employee can contribute to several jobs (but not at the same time). The objective is to get all jobs done as early as possible.

2

1 Introduction

In this model it suffices to prescribe for each employee how long he or she should work on which job. The order in which the employees carry out their jobs is not important, since the time when all jobs are done obviously depends only on the maximum total working time we have assigned to one employee. Hence we have to solve the following problem:

J OB A SSIGNMENT P ROBLEM Instance:

Task:

A set of numbers t1 , . . . , tn ∈ R+ (the processing times for n jobs), a number m ∈ N of employees, and a nonempty subset Si ⊆ {1, . . . , m} of employees for each job i ∈ {1, . . . , n}. Find P numbers xi j ∈ R+ for all i = 1, . . . , n and j ∈PSi such that j∈Si x i j = ti for i = 1, . . . , n and max j∈{1,...,m} i: j∈Si x i j is minimum.

These are two typical problems arising in combinatorial optimization. How to model a practical problem as an abstract combinatorial optimization problem is not described in this book; indeed there is no general recipe for this task. Besides giving a precise formulation of the input and the desired output it is often important to ignore irrelevant components (e.g. the drilling time which cannot be optimized or the order in which the employees carry out their jobs). Of course we are not interested in a solution to a particular drilling problem or job assignment problem in some company, but rather we are looking for a way how to solve all problems of these types. We first consider the D RILLING P ROBLEM.

1.1 Enumeration How can a solution to the D RILLING P ROBLEM look like? There are infinitely many instances (finite sets of points in the plane), so we cannot list an optimum permutation for each instance. Instead, what we look for is an algorithm which, given an instance, computes an optimum solution. Such an algorithm exists: Given a set of n points, just try all possible n! orders, and for each compute the L ∞ -length of the corresponding path. There are different ways of formulating an algorithm, differing mostly in the level of detail and the formal language they use. We certainly would not accept the following as an algorithm: “Given a set of n points, find an optimum path and output it.” It is not specified at all how to find the optimum solution. The above suggestion to enumerate all possible n! orders is more useful, but still it is not clear how to enumerate all the orders. Here is one possible way: We enumerate all n-tuples of numbers 1, . . . , n, i.e. all n n vectors of {1, . . . , n n} . This can be done similarly to counting: we start with (1, . . . , 1, 1), (1, . . . , 1, 2) up to (1, . . . , 1, n) then switch to (1, . . . , 1, 2, 1), and so on. At each step we increment the last entry unless it is already n, in which case we go back to the last entry that is smaller than n, increment it and set all subsequent entries to 1.

1.1 Enumeration

3

This technique is sometimes called backtracking. The order in which the vectors of {1, . . . , n}n are enumerated is called the lexicographical order: Definition 1.1. Let x, y ∈ Rn be two vectors. We say that a vector x is lexicographically smaller than y if there exists an index j ∈ {1, . . . , n} such that xi = yi for i = 1, . . . , j − 1 and x j < y j . Knowing how to enumerate all vectors of {1, . . . , n}n we can simply check for each vector whether its entries are pairwise distinct and, if so, whether the path represented by this vector is shorter than the best path encountered so far. Since this algorithm enumerates n n vectors it will take at least n n steps (in fact, even more). This is not best possible. There are only n! permutations of {1,√. . . , n}, n and n! is significantly smaller than n n . (By Stirling’s formula n! ≈ 2π n nen (Stirling [1730]); see Exercise 1.) We shall show how to enumerate all paths in approximately n 2 · n! steps. Consider the following algorithm which enumerates all permutations in lexicographical order:

PATH E NUMERATION A LGORITHM Input:

A natural number n ≥ 3. A set { p1 , . . . , pn } of points in the plane.

Output:

∗ A permutation Pπn−1: {1, . . . , n} → {1, . . . , n} with ∗ cost (π ) := i=1 d( pπ ∗ (i) , pπ ∗ (i+1) ) minimum.

1

Set π(i) := i and π ∗ (i) := i for i = 1, . . . , n. Set i := n − 1.

2

Let k := min({π(i) + 1, . . . , n + 1} \ {π(1), . . . , π(i − 1)}).

3

If k ≤ n then: Set π(i) := k. If i = n and cost (π) < cost (π ∗ ) then set π ∗ := π . If i < n then set π(i + 1) := 0 and i := i + 1. If k = n + 1 then set i := i − 1. If i ≥ 1 then go to . 2

Starting with (π(i))i=1,...,n = (1, 2, 3, . . . , n−1, n) and i = n−1, the algorithm finds at each step the next possible value of π(i) (not using π(1), . . . , π(i − 1)). If there is no more possibility for π(i) (i.e. k = n + 1), then the algorithm decrements i (backtracking). Otherwise it sets π(i) to the new value. If i = n, the new permutation is evaluated, otherwise the algorithm will try all possible values for π(i + 1), . . . , π(n) and starts by setting π(i + 1) := 0 and incrementing i. So all permutation vectors (π(1), . . . , π(n)) are generated in lexicographical order. For example, the first iterations in the case n = 6 are shown below:

4

1 Introduction

k k k k k k k

:= 6, := 5, := 7, := 7, := 5, := 4, := 6,

π := (1, 2, 3, 4, 5, 6), π := (1, 2, 3, 4, 6, 0), π := (1, 2, 3, 4, 6, 5), π := (1, 2, 3, 5, 0, 5), π := (1, 2, 3, 5, 4, 0), π := (1, 2, 3, 5, 4, 6),

i := 5 i := 6 cost (π ) < cost (π ∗ )? i i i i

:= 5 := 4 := 5 := 6 cost (π) < cost (π ∗ )?

Since the algorithm compares the cost of each path to π ∗ , the best path encountered so far, it indeed outputs the optimum path. But how many steps will this algorithm perform? Of course, the answer depends on what we call a single step. Since we do not want the number of steps to depend on the actual implementation we ignore constant factors. In any reasonable computer, 1 will take at least 2n + 1 steps (this many variable assignments are done) and at most cn steps for some constant c. The following common notation is useful for ignoring constant factors: Definition 1.2. Let f, g : D → R+ be two functions. We say that f is O(g) (and sometimes write f = O(g)) if there exist constants α, β > 0 such that f (x) ≤ αg(x) + β for all x ∈ D. If f = O(g) and g = O( f ) we also say that f = 2(g) (and of course g = 2( f )). In this case, f and g have the same rate of growth. Note that the use of the equation sign in the O-notation is not symmetric. To illustrate this definition, let D = N, and let f (n) be the number of elementary steps in 1 and g(n) = n (n ∈ N). Clearly we have f = O(g) (in fact f = 2(g)) in this case; we say that 1 takes O(n) time (or linear time). A single execution takes a constant number of steps (we speak of O(1) time or constant time) of 3 except in the case k ≤ n and i = n; in this case the cost of two paths have to be compared, which takes O(n) time. What about ? 2 A naive implementation, checking for each j ∈ {π(i) + 1, . . . , n} and each h ∈ {1, . . . , i − 1} whether j = π(h), takes O((n − π(i))i) steps, which can be as big as 2(n 2 ). A better implementation of 2 uses an auxiliary array indexed by 1, . . . , n: For j := 1 to n do aux( j) := 0. For j := 1 to i − 1 do aux(π( j)) := 1. Set k := π(i) + 1. While k ≤ n and aux(k) = 1 do k := k + 1. Obviously with this implementation a single execution of 2 takes only O(n) time. Simple techniques like this are usually not elaborated in this book; we assume that the reader can find such implementations himself. Having computed the running time for each single step we now estimate the total amount of work. Since the number of permutations is n! we only have to estimate the amount of work which is done between two permutations. The counter i might move back from n to some index i 0 where a new value π(i 0 ) ≤ n is found. Then it moves forward again up to i = n. While the counter i is constant each

2

1.2 Running Time of Algorithms

5

of 2 and 3 is performed once, except in the case k ≤ n and i = n; in this case 2 and 3 are performed twice. So the total amount of work between two permutations consists of at most 4n times 2 and , 3 i.e. O(n 2 ). So the overall running time of the PATH E NUMERATION A LGORITHM is O(n 2 n!). One can do slightly better; a more careful analysis shows that the running time is only O(n · n!) (Exercise 4). Still the algorithm is too time-consuming if n is large. The problem with the enumeration of all paths is that the number of paths grows exponentially with the number of points; already for 20 points there are 20! = 2432902008176640000 ≈ 2.4 · 1018 different paths and even the fastest computer needs several years to evaluate all of them. So complete enumeration is impossible even for instances of moderate size. The main subject of combinatorial optimization is to find better algorithms for problems like this. Often one has to find the best element of some finite set of feasible solutions (in our example: drilling paths or permutations). This set is not listed explicitly but implicitly depends on the structure of the problem. Therefore an algorithm must exploit this structure. In the case of the D RILLING P ROBLEM all information of an instance with n points is given by 2n coordinates. While the naive algorithm enumerates all n! paths it might be possible that there is an algorithm which finds the optimum path much faster, say in n 2 computation steps. It is not known whether such an algorithm exists (though results of Chapter 15 suggest that it is unlikely). Nevertheless there are much better algorithms than the naive one.

1.2 Running Time of Algorithms One can give a formal definition of an algorithm, and we shall in fact give one in Section 15.1. However, such formal models lead to very long and tedious descriptions as soon as algorithms are a bit more complicated. This is quite similar to mathematical proofs: Although the concept of a proof can be formalized nobody uses such a formalism for writing down proofs since they would become very long and almost unreadable. Therefore all algorithms in this book are written in an informal language. Still the level of detail should allow a reader with a little experience to implement the algorithms on any computer without too much additional effort. Since we are not interested in constant factors when measuring running times we do not have to fix a concrete computing model. We count elementary steps, but we are not really interested in how elementary steps look like. Examples of elementary steps are variable assignments, random access to a variable whose index is stored in another variable, conditional jumps (if – then – go to), and simple arithmetic operations like addition, subtraction, multiplication, division and comparison of numbers. An algorithm consists of a set of valid inputs and a sequence of instructions each of which can be composed of elementary steps, such that for each valid input

6

1 Introduction

the computation of the algorithm is a uniquely defined finite series of elementary steps which produces a certain output. Usually we are not satisfied with finite computation but rather want a good upper bound on the number of elementary steps performed, depending on the input size. The input to an algorithm usually consists of a list of numbers. If all these numbers are integers, we can code them in binary representation, using O(log(|a|+ 2)) bits for storing an integer a. Rational numbers can be stored by coding the numerator and the denominator separately. The input size size(x) of an instance x with rational data is the total number of bits needed for the binary representation. Definition 1.3. Let A be an algorithm which accepts inputs from a set X , and let f : N → R+ . If there exists a constant α > 0 such that A terminates its computation after at most α f (size(x)) elementary steps (including arithmetic operations) for each input x ∈ X , then we say that A runs in O( f ) time. We also say that the running time (or the time complexity) of A is O( f ). Definition 1.4. An algorithm with rational input is said to run in polynomial time if there is an integer k such that it runs in O(n k ) time, where n is the input size, and all numbers in intermediate computations can be stored with O(n k ) bits. An algorithm with arbitrary input is said to run in strongly polynomial time if there is an integer k such that it runs in O(n k ) time for any input consisting of n numbers and it runs in polynomial time for rational input. In the case k = 1 we have a linear-time algorithm. Note that the running time might be different for several instances of the same size (this was not the case with the PATH E NUMERATION A LGORITHM). We consider the worst-case running time, i.e. the function f : N → N where f (n) is the maximum running time of an instance with input size n. For some algorithms we do not know the rate of growth of f but only have an upper bound. The worst-case running time might be a pessimistic measure if the worst case occurs rarely. In some cases an average-case running time with some probabilistic model might be appropriate, but we shall not consider this. If A is an algorithm which for each input x ∈ X computes the output f (x) ∈ Y , then we say that A computes f : X → Y . If a function is computed by some polynomial-time algorithm, it is said to be computable in polynomial time. Polynomial-time algorithms are sometimes called “good” or “efficient”. This concept was introduced by Cobham [1964] and Edmonds [1965]. Table 1.1 motivates this by showing hypothetical running times of algorithms with various time complexities. For various input sizes n we show the running time of algorithms that take 100n log n, 10n 2 , n 3.5 , n log n , 2n , and n! elementary steps; we assume that one elementary step takes one nanosecond. As always in this book, log denotes the logarithm with basis 2. As Table 1.1 shows, polynomial-time algorithms are faster for large enough instances. The table also illustrates that constant factors of moderate size are not very important when considering the asymptotic growth of the running time.

1.2 Running Time of Algorithms

7

Table 1.1. n 10 20 30 40 50 60 80 100 200 500 1000 104 105 106 107 108 1010 1012

100n log n

10n 2

n 3.5

n log n

2n

n!

3 µs 9 µs 15 µs 21 µs 28 µs 35 µs 50 µs 66 µs 153 µs 448 µs 1 ms 13 ms 166 ms 2s 23 s 266 s 9 hours 46 days

1 µs 4 µs 9 µs 16 µs 25 µs 36 µs 64 µs 100 µs 400 µs 2.5 ms 10 ms 1s 100 s 3 hours 12 days 3 years 3 · 104 y. 3 · 108 y.

3 µs 36 µs 148 µs 404 µs 884 µs 2 ms 5 ms 10 ms 113 ms 3s 32 s 28 hours 10 years 3169 y. 107 y. 3 · 1010 y.

2 µs 420 µs 20 ms 340 ms 4s 32 s 1075 s 5 hours 12 years 5 · 105 y. 3 · 1013 y.

1 µs 1 ms 1s 1100 s 13 days 37 years 4 · 107 y. 4 · 1013 y.

4 ms 76 years 8 · 1015 y.

Table 1.2 shows the maximum input sizes solvable within one hour with the above six hypothetical algorithms. In (a) we again assume that one elementary step takes one nanosecond, (b) shows the corresponding figures for a ten times faster machine. Polynomial-time algorithms can handle larger instances in reasonable time. Moreover, even a speedup by a factor of 10 of the computers does not increase the size of solvable instances significantly for exponential-time algorithms, but it does for polynomial-time algorithms. Table 1.2. 10n 2

n 3.5

(a)

1.19 · 109

60000

(b)

10.8 · 109

189737

100n log n

n log n

2n

n!

3868

87

41

15

7468

104

45

16

(Strongly) polynomial-time algorithms, if possible linear-time algorithms, are what we look for. There are some problems where it is known that no polynomialtime algorithm exists, and there are problems for which no algorithm exists at all. (For example, a problem which can be solved in finite time but not in polynomial

8

1 Introduction

time is to decide whether a so-called regular expression defines the empty set; see Aho, Hopcroft and Ullman [1974]. A problem for which there exists no algorithm at all, the H ALTING P ROBLEM, is discussed in Exercise 1 of Chapter 15.) However, almost all problems considered in this book belong to the following two classes. For the problems of the first class we have a polynomial-time algorithm. For each problem of the second class it is an open question whether a polynomialtime algorithm exists. However, we know that if one of these problems has a polynomial-time algorithm, then all problems of this class do. A precise formulation and a proof of this statement will be given in Chapter 15. The J OB A SSIGNMENT P ROBLEM belongs to the first class, the D RILLING P ROBLEM belongs to the second class. These two classes of problems divide this book roughly into two parts. We first deal with tractable problems for which polynomial-time algorithms are known. Then, starting with Chapter 15, we discuss hard problems. Although no polynomialtime algorithms are known, there are often much better methods than complete enumeration. Moreover, for many problems (including the D RILLING P ROBLEM), one can find approximate solutions within a certain percentage of the optimum in polynomial time.

1.3 Linear Optimization Problems We now consider our second example given initially, the J OB A SSIGNMENT P ROB LEM , and briefly address some central topics which will be discussed in later chapters. The J OB A SSIGNMENT P ROBLEM is quite different to the D RILLING P ROBLEM since there are infinitely many feasible solutions for each instance (except for trivial cases). We can reformulate the problem by introducing a variable T for the time when all jobs are done: min s.t.

T X

xi j

= ti

(i ∈ {1, . . . , n})

≥ 0 ≤ T

(i ∈ {1, . . . , n}, j ∈ Si ) ( j ∈ {1, . . . , m})

j∈Si

X

xi j xi j

(1.1)

i: j∈Si

The numbers ti and the sets Si (i = 1, . . . , n) are given, the variables xi j and T are what we look for. Such an optimization problem with a linear objective function and linear constraints is called a linear program. The set of feasible solutions of (1.1), a so-called polyhedron, is easily seen to be convex, and one can prove that there always exists an optimum solution which is one of the finitely many extreme points of this set. Therefore a linear program can, theoretically, also

1.4 Sorting

9

be solved by complete enumeration. But there are much better ways as we shall see later. Although there are several algorithms for solving linear programs in general, such general techniques are usually less efficient than special algorithms exploiting the structure of the problem. In our case it is convenient to model the sets Si , i = 1, . . . , n, by a graph. For each job i and for each employee j we have a point (called vertex), and we connect employee j with job i by an edge if he or she can contribute to this job (i.e. if j ∈ Si ). Graphs are a fundamental combinatorial structure; many combinatorial optimization problems are described most naturally in terms of graph theory. Suppose for a moment that the processing time of each job is one hour, and we ask whether we can finish all jobs within one hour. So we look Pfor numbers xi j (i ∈ {1, . . . , n}, j P ∈ Si ) such that 0 ≤ xi j ≤ 1 for all i and j, j∈Si xi j = 1 for i = 1, . . . , n, and i: j∈Si xi j ≤ 1 for j = 1, . . . , n. One can show that if such a solution exists, then in fact an integral solution exists, i.e. all xi j are either 0 or 1. This is equivalent to assigning each job to one employee, such that no employee has to do more than one job. In the language of graph theory we then look for a matching covering all jobs. The problem of finding optimal matchings is one of the best-known combinatorial optimization problems. We review the basics of graph theory and linear programming in Chapters 2 and 3. In Chapter 4 we prove that linear programs can be solved in polynomial time, and in Chapter 5 we discuss integral polyhedra. In the subsequent chapters we discuss some classical combinatorial optimization problems in detail.

1.4 Sorting Let us conclude this chapter by considering a special case of the D RILLING P ROB LEM where all holes to be drilled are on one horizontal line. So we are given just one coordinate for each point pi , i = 1, . . . , n. Then a solution to the drilling problem is easy, all we have to do is sort the points by their coordinates: the drill will just move from left to right. Although there are still n! permutations, it is clear that we do not have to consider all of them to find the optimum drilling path, i.e. the sorted list. It is very easy to sort n numbers in nondecreasing order in O(n 2 ) time. To sort n numbers in O(n log n) time requires a little more skill. There are several algorithms accomplishing this; we present the well-known M ERGE -S ORT A LGORITHM. It proceeds as follows. First the list is divided into two sublists of approximately equal size. Then each sublist is sorted (this is done recursively by the same algorithm). Finally the two sorted sublists are merged together. This general strategy, often called “divide and conquer”, can be used quite often. See e.g. Section 17.1 for another example. We did not discuss recursive algorithms so far. In fact, it is not necessary to discuss them, since any recursive algorithm can be transformed into a sequential algorithm without increasing the running time. But some algorithms are easier to

10

1 Introduction

formulate (and implement) using recursion, so we shall use recursion when it is convenient.

M ERGE -S ORT A LGORITHM Input:

A list a1 , . . . , an of real numbers.

Output:

A permutation π : {1, . . . , n} → {1, . . . , n} such that aπ(i) ≤ aπ(i+1) for all i = 1, . . . , n − 1.

1

2

3

If n = 1 then set π(1) := 1 and stop (return π ). Set m := n2 . Let ρ :=M ERGE -S ORT(a1 , . . . , am ). Let σ :=M ERGE -S ORT(am+1 , . . . , an ). Set k := 1, l := 1. While k ≤ m and l ≤ n − m do: If aρ(k) ≤ am+σ (l) then set π(k + l − 1) := ρ(k) and k := k + 1 else set π(k + l − 1) := m + σ (l) and l := l + 1. While k ≤ m do: Set π(k + l − 1) := ρ(k) and k := k + 1. While l ≤ n − m do: Set π(k + l − 1) := m + σ (l) and l := l + 1.

As an example, consider the list “69,32,56,75,43,99,28”. The algorithm first splits this list into two, “69,32,56” and “75,43,99,28” and recursively sorts each of the two sublists. We get the permutations ρ = (2, 3, 1) and σ = (4, 2, 1, 3) corresponding to the sorted lists “32,56,69” and “28,43,75,99”. Now these lists are merged as shown below: k := 1, l := 1 l := 2 ρ(1) = 2, σ (1) = 4, aρ(1) = 32, aσ (1) = 28, π(1) := 7, ρ(1) = 2, σ (2) = 2, aρ(1) = 32, aσ (2) = 43, π(2) := 2, k := 2 ρ(2) = 3, σ (2) = 2, aρ(2) = 56, aσ (2) = 43, π(3) := 5, l := 3 ρ(2) = 3, σ (3) = 1, aρ(2) = 56, aσ (3) = 75, π(4) := 3, k := 3 ρ(3) = 1, σ (3) = 1, aρ(3) = 69, aσ (3) = 75, π(5) := 1, k := 4 l := 4 σ (3) = 1, aσ (3) = 75, π(6) := 4, σ (4) = 3, aσ (4) = 99, π(7) := 6, l := 5 Theorem 1.5. The M ERGE -S ORT A LGORITHM works correctly and runs in O(n log n) time. Proof: The correctness is obvious. We denote by T (n) the running time (number of steps) needed for instances consisting of n numbers and observe that T (1) = 1 and T (n) = T (b n2 c) + T (d n2 e) + 3n + 6. (The constants in the term 3n + 6 depend on how exactly a computation step is defined; but they do not really matter.) We claim that this yields T (n) ≤ 12n log n + 1. Since this is trivial for n = 1 we proceed by induction. For n ≥ 2, assuming that the inequality is true for 1, . . . , n − 1, we get

Exercises

11

ln m 2 2 n + 1 + 12 n + 1 + 3n + 6 log 2 3 2 3 = 12n(log n + 1 − log 3) + 3n + 8 13 ≤ 12n log n − n + 3n + 8 ≤ 12n log n + 1, 2

T (n) ≤ 12

because log 3 ≥

jn k

log

37 24 .

Of course the algorithm works for sorting the elements of any totally ordered set, assuming that we can compare any two elements in constant time. Can there be a faster, a linear-time algorithm? Suppose that the only way we can get information on the unknown order is to compare two elements. Then we can show that any algorithm needs at least 2(n log n) comparisons in the worst case. The outcome of a comparison can be regarded as a zero or one; the outcome of all comparisons an algorithm does is a 0-1-string (a sequence of zeros and ones). Note that two different orders in the input of the algorithm must lead to two different 0-1-strings (otherwise the algorithm could not distinguish between the two orders). For an input of n elements there are n! possible orders, so there must be n! different 01-strings corresponding to the computation. Since the number of 0-1-strings with n n n n n length less than n2 log n2 is 2b 2 log 2 c − 1 < 2 2 log 2 = ( n2 ) 2 ≤ n! we conclude that the maximum length of the 0-1-strings, and hence of the computation, must be at least n2 log n2 = 2(n log n). In the above sense, the running time of the M ERGE -S ORT A LGORITHM is optimal up to a constant factor. However, there is an algorithm for sorting integers (or sorting strings lexicographically) whose running time is linear in the input size; see Exercise 7. An algorithm to sort n integers in O(n log log n) time was proposed by Han [2004]. Lower bounds like the one above are known only for very few problems (except trivial linear bounds). Often a restriction on the set of operations is necessary to derive a superlinear lower bound.

Exercises 1. Prove that for all n ∈ N: e

2. 3. 4. 5.

n n e

≤ n! ≤ en

n n e

.

Hint: Use 1 + x ≤ e x for all x ∈ R. Prove that log(n!) = 2(n log n). Prove that n log n = O(n 1+ ) for any > 0. Show that the running time of the PATH E NUMERATION A LGORITHM is O(n · n!). Suppose we have an algorithm whose running time is 2(n(t + n 1/t )), where n is the input length and t is a positive parameter we can choose arbitrarily.

12

1 Introduction

How should t be chosen (depending on n) such that the running time (as a function of n) has a minimum rate of growth? 6. Let s, t be binary strings, both of length m. We say that s is lexicographically smaller than t if there exists an index j ∈ {1, . . . , m} such that si = ti for i = 1, . . . , j − 1 and s j < t j . Now given n strings of length m, we want to sort them lexicographically. Prove that there is a linear-time algorithm for this problem (i.e. one with running time O(nm)). Hint: Group the strings according to the first bit and sort each group. 7. Describe an algorithm which sorts a list of natural numbers a1 , . . . , an in linear time; i.e. which finds a permutation π with aπ(i) ≤ aπ(i+1) (i = 1, . . . , n − 1) and runs in O(log(a1 + 1) + · · · + log(an + 1)) time. Hint: First sort the strings encoding the numbers according to their length. Then apply the algorithm of Exercise 6. Note: The algorithm discussed in this and the previous exercise is often called radix sorting.

References General Literature: Cormen, T.H., Leiserson, C.E., Rivest, R.L., and Stein, C. [2001]: Introduction to Algorithms. Second Edition. MIT Press, Cambridge 2001 Knuth, D.E. [1968]: The Art of Computer Programming; Vol. 1. Fundamental Algorithms. Addison-Wesley, Reading 1968 (third edition: 1997)

Cited References: Aho, A.V., Hopcroft, J.E., and Ullman, J.D. [1974]: The Design and Analysis of Computer Algorithms. Addison-Wesley, Reading 1974 Cobham, A. [1964]: The intrinsic computational difficulty of functions. Proceedings of the 1964 Congress for Logic Methodology and Philosophy of Science (Y. Bar-Hillel, ed.), North-Holland, Amsterdam 1964, pp. 24–30 Edmonds, J. [1965]: Paths, trees, and flowers. Canadian Journal of Mathematics 17 (1965), 449–467 Han, Y. [2004]: Deterministic sorting in O(n log log n) time and linear space. Journal of Algorithms 50 (2004), 96–105 Stirling, J. [1730]: Methodus Differentialis. London 1730

2 Graphs

Graphs are a fundamental combinatorial structure used throughout this book. In this chapter we not only review the standard definitions and notation, but also prove some basic theorems and mention some fundamental algorithms. After some basic definitions in Section 2.1 we consider fundamental objects occurring very often in this book: trees, circuits, and cuts. We prove some important properties and relations, and we also consider tree-like set systems in Section 2.2. The first graph algorithms, determining connected and strongly connected components, appear in Section 2.3. In Section 2.4 we prove Euler’s Theorem on closed walks using every edge exactly once. Finally, in Sections 2.5 and 2.6 we consider graphs that can be drawn in the plane without crossings.

2.1 Basic Definitions An undirected graph is a triple (V, E, 9), where V and E are finite sets and 9 : E → {X ⊆ V : |X | = 2}. A directed graph or digraph is a triple (V, E, 9), where V and E are finite sets and 9 : E → {(v, w) ∈ V × V : v 6= w}. By a graph we mean either an undirected graph or a digraph. The elements of V are called vertices, the elements of E are the edges. Two edges e, e0 with 9(e) = 9(e0 ) are called parallel. Graphs without parallel edges are called simple. For simple graphs we usually identify an edge e with its image 9(e) and write G = (V (G), E(G)), where E(G) ⊆ {X ⊆ V (G) : |X | = 2} or E(G) ⊆ V (G) × V (G). We often use this simpler notation even in the presence of parallel edges, then the “set” E(G) may contain several “identical” elements. |E(G)| denotes the number of edges, and for two edge sets E and F we always . have |E ∪ F| = |E| + |F| even if parallel edges arise. We say that an edge e = {v, w} or e = (v, w) joins v and w. In this case, v and w are adjacent. v is a neighbour of w (and vice versa). v and w are the endpoints of e. If v is an endpoint of an edge e, we say that v is incident with e. In the directed case we say that e = (v, w) leaves v (the tail of e) and enters w (the head of e). Two edges which share at least one endpoint are called adjacent. This terminology for graphs is not the only one. Sometimes vertices are called nodes or points, other names for edges are arcs (especially in the directed case) or lines. In some texts, a graph is what we call a simple undirected graph, in the presence of parallel edges they speak of multigraphs. Sometimes edges whose

14

2 Graphs

endpoints coincide, so-called loops, are considered. However, unless otherwise stated, we do not use them. For a digraph G we sometimes consider the underlying undirected graph, i.e. the undirected graph G 0 on the same vertex set which contains an edge {v, w} for each edge (v, w) of G. We also say that G is an orientation of G 0 . A subgraph of a graph G = (V (G), E(G)) is a graph H = (V (H ), E(H )) with V (H ) ⊆ V (G) and E(H ) ⊆ E(G). We also say that G contains H . H is an induced subgraph of G if it is a subgraph of G and E(H ) = {{x, y} or (x, y) ∈ E(G) : x, y ∈ V (H )}. Here H is the subgraph of G induced by V (H ). We also write H = G[V (H )]. A subgraph H of G is called spanning if V (H ) = V (G). If v ∈ V (G), we write G − v for the subgraph of G induced by V (G) \ {v}. If e ∈ E(G), we define G − e := (V (G), E(G) \ {e}). Furthermore, the addition . of a new edge e is abbreviated by G + e := (V (G), E(G) ∪ {e}). If G and H are two graphs, we denote by G + H the graph with V (G + H ) = V (G) ∪ V (H ) and E(G + H ) being the disjoint union of E(G) and E(H ) (parallel edges may arise). Two graphs G and H are called isomorphic if there are bijections 8V : V (G) → V (H ) and 8 E : E(G) → E(H ) such that 8 E ((v, w)) = (8V (v), 8V (w)) for all (v, w) ∈ E(G), or 8 E ({v, w}) = {8V (v), 8V (w)} for all {v, w} ∈ E(G) in the undirected case. We normally do not distinguish between isomorphic graphs; for example we say that G contains H if G has a subgraph isomorphic to H . Suppose we have an undirected graph G and X ⊆ V (G). By contracting (or shrinking) X we mean deleting the vertices in X and the edges in G[X ], adding a new vertex x and replacing each edge {v, w} with v ∈ X , w ∈ / X by an edge {x, w} (parallel edges may arise). Similarly for digraphs. We often call the result G/X . For a graph G and X, Y ⊆ V (G) we define E(X, Y ) := {{x, y} ∈ E(G) : x ∈ X \ Y, y ∈ Y \ X } if G is undirected and E + (X, Y ) := {(x, y) ∈ E(G) : x ∈ X \ Y, y ∈ Y \ X } if G is directed. For undirected graphs G and X ⊆ V (G) we define δ(X ) := E(X, V (G) \ X ). The set of neighbours of X is defined by 0(X ) := {v ∈ V (G) \ X : E(X, {v}) 6= ∅}. For digraphs G and X ⊆ V (G) we define δ + (X ) := E + (X, V (G) \ X ), δ − (X ) := δ + (V (G) \ X ) and δ(X ) := δ + (X )∪δ − (X ). We use subscripts (e.g. δG (X )) to specify the graph G if necessary. For singletons, i.e. one-element vertex sets {v} (v ∈ V (G)) we write δ(v) := δ({v}), 0(v) := 0({v}), δ + (v) := δ + ({v}) and δ − (v) := δ − ({v}). The degree of a vertex v is |δ(v)|, the number of edges incident to v. In the directed case, the in-degree is |δ − (v)|, the out-degree is |δ + (v)|, and the degree is |δ + (v)|+|δ − (v)|. A vertex v with zero degree is called isolated. A graph where all vertices have degree k is called P k-regular. For any graph, v∈V (G) |δ(v)| = 2|E(G)|. In particular, the number of vertices P P with odd degree is even. In a digraph, v∈V (G) |δ + (v)| = v∈V (G) |δ − (v)|. To prove these statements, please observe that each edge is counted twice on each

2.1 Basic Definitions

15

side of the first equation and once on each side of the second equation. With just a little more effort we get the following useful statements: Lemma 2.1. For a digraph G and any two sets X, Y ⊆ V (G): (a) |δ + (X )| + |δ + (Y )| = |δ + (X ∩ Y )| + |δ + (X ∪ Y )| + |E + (X, Y )| + |E + (Y, X )|; (b) |δ − (X )| + |δ − (Y )| = |δ − (X ∩ Y )| + |δ − (X ∪ Y )| + |E + (X, Y )| + |E + (Y, X )|. For an undirected graph G and any two sets X, Y ⊆ V (G): (c) |δ(X )| + |δ(Y )| = |δ(X ∩ Y )| + |δ(X ∪ Y )| + 2|E(X, Y )|; (d) |0(X )| + |0(Y )| ≥ |0(X ∩ Y )| + |0(X ∪ Y )|. Proof: All parts can be proved by simple counting arguments. Let Z := V (G) \ (X ∪ Y ). To prove (a), observe that |δ + (X )|+|δ + (Y )| = |E + (X, Z )|+|E + (X, Y \ X )|+ + |E (Y, Z )| + |E + (Y, X \ Y )| = |E + (X ∪ Y, Z )| + |E + (X ∩ Y, Z )| + |E + (X, Y \ X )| + |E + (Y, X \ Y )| = |δ + (X ∪ Y )| + |δ + (X ∩ Y )| + |E + (X, Y )| + |E + (Y, X )|. (b) follows from (a) by reversing each edge (replace (v, w) by (w, v)). (c) follows from (a) by replacing each edge {v, w} by a pair of oppositely directed edges (v, w) and (w, v). To show (d), observe that |0(X )| + |0(Y )| = |0(X ∪ Y )| + |0(X ) ∩ 0(Y )| + |0(X ) ∩ Y | + |0(Y ) ∩ X | ≥ |0(X ∪ Y )| + |0(X ∩ Y )|. A function f : 2U → R (where U is some finite set and 2U denotes its power set) is called • submodular if f (X ∩ Y ) + f (X ∪ Y ) ≤ f (X ) + f (Y ) for all X, Y ⊆ U ; • supermodular if f (X ∩ Y ) + f (X ∪ Y ) ≥ f (X ) + f (Y ) for all X, Y ⊆ U ; • modular if f (X ∩ Y ) + f (X ∪ Y ) = f (X ) + f (Y ) for all X, Y ⊆ U . So Lemma 2.1 implies that |δ + |, |δ − |, |δ| and |0| are submodular. This will be useful later. A complete graph is a simple undirected graph where each pair of vertices is adjacent. We denote the complete graph on n vertices by K n . The complement of a simple undirected graph G is the graph H for which G + H is a complete graph. A matching in an undirected graph G is a set of pairwise disjoint edges (i.e. the endpoints are all different). A vertex cover in G is a set S ⊆ V (G) of vertices such that every edge of G is incident to at least one vertex in S. An edge cover in G is a set F ⊆ E(G) of edges such that every vertex of G is incident to at least one edge in F. A stable set in G is a set of pairwise non-adjacent vertices. A graph containing no edges is called empty. A clique is a set of pairwise adjacent vertices. Proposition 2.2. Let G be a graph and X ⊆ V (G). Then the following three statements are equivalent: (a) X is a vertex cover in G,

16

2 Graphs

(b) V (G) \ X is a stable set in G, (c) V (G) \ X is a clique in the complement of G.

If F is a family of sets or graphs, we say that F is a minimal element of F if F contains F but no proper subset/subgraph of F. Similarly, F is maximal in F if F ∈ F and F is not a proper subset/subgraph of any element of F. When we speak of a minimum or maximum element, we mean one of minimum/maximum cardinality. For example, a minimal vertex cover is not necessarily a minimum vertex cover (see e.g. the graph in Figure 13.1), and a maximal matching is in general not maximum. The problems of finding a maximum matching, stable set or clique, or a minimum vertex cover or edge cover in an undirected graph will play important roles in later chapters. The line graph of a simple undirected graph G is the graph (E(G), F), where F = {{e1 , e2 } : e1 , e2 ∈ E(G), |e1 ∩ e2 | = 1}. Obviously, matchings in a graph G correspond to stable sets in the line graph of G. For the following notation, let G be a graph, directed or undirected. An edge progression W in G is a sequence v1 , e1 , v2 , . . . , vk , ek , vk+1 such that k ≥ 0, and ei = (vi , vi+1 ) ∈ E(G) or ei = {vi , vi+1 } ∈ E(G) for i = 1, . . . , k. If in addition ei 6= e j for all 1 ≤ i < j ≤ k, W is called a walk in G. W is closed if v1 = vk+1 . A path is a graph P = ({v1 , . . . , vk+1 }, {e1 , . . . , ek }) such that vi 6= v j for 1 ≤ i < j ≤ k +1 and the sequence v1 , e1 , v2 , . . . , vk , ek , vk+1 is a walk. P is also called a path from v1 to vk+1 or a v1 -vk+1 -path. v1 and vk+1 are the endpoints of P. By P[x,y] with x, y ∈ V (P) we mean the (unique) subgraph of P which is an x-y-path. Evidently, there is an edge progression from a vertex v to another vertex w if and only if there is a v-w-path. A circuit or a cycle is a graph ({v1 , . . . , vk }, {e1 , . . . , ek }) such that the sequence v1 , e1 , v2 , . . . , vk , ek , v1 is a (closed) walk and vi 6= v j for 1 ≤ i < j ≤ k. An easy induction argument shows that the edge set of a closed walk can be partitioned into edge sets of circuits. The length of a path or circuit is the number of its edges. If it is a subgraph of G, we speak of a path or circuit in G. A spanning path in G is called a Hamiltonian path while a spanning circuit in G is called a Hamiltonian circuit or a tour. A graph containing a Hamiltonian circuit is a Hamiltonian graph. For two vertices v and w we write dist(v, w) or distG (v, w) for the length of a shortest v-w-path (the distance from v to w) in G. If there is no v-w-path at all, i.e. w is not reachable from v, we set dist(v, w) := ∞. In the undirected case, dist(v, w) = dist(w, v) for all v, w ∈ V (G). We shall often P have a cost function c : E(G) → R. Then for F ⊆ E(G) we write c(F) := e∈F c(e) (and c(∅) = 0). This extends c to a modular function c : 2 E(G) → R. Moreover, dist(G,c) (v, w) denotes the minimum c(E(P)) over all v-w-paths P in G.

2.2 Trees, Circuits, and Cuts

17

2.2 Trees, Circuits, and Cuts Let G be some undirected graph. G is called connected if there is a v-w-path for all v, w ∈ V (G); otherwise G is disconnected. The maximal connected subgraphs of G are its connected components. Sometimes we identify the connected components with the vertex sets inducing them. A set of vertices X is called connected if the subgraph induced by X is connected. A vertex v with the property that G − v has more connected components than G is called an articulation vertex. An edge e is called a bridge if G − e has more connected components than G. An undirected graph without a circuit (as a subgraph) is called a forest. A connected forest is a tree. A vertex of degree 1 in a tree is called a leaf. A star is a tree where at most one vertex is not a leaf. In the following we shall give some equivalent characterizations of trees and their directed counterparts, arborescences. We need the following connectivity criterion: Proposition 2.3. (a) An undirected graph G is connected if and only if δ(X ) 6= ∅ for all ∅ 6= X ⊂ V (G). (b) Let G be a digraph and r ∈ V (G). Then there exists an r -v-path for every v ∈ V (G) if and only if δ + (X ) 6= ∅ for all X ⊂ V (G) with r ∈ X . Proof: (a): If there is a set X ⊂ V (G) with r ∈ X , v ∈ V (G) \ X , and δ(X ) = ∅, there can be no r -v-path, so G is not connected. On the other hand, if G is not connected, there is no r -v-path for some r and v. Let R be the set of vertices reachable from r . We have r ∈ R, v ∈ / R and δ(R) = ∅. (b) is proved analogously. Theorem 2.4. Let G be an undirected graph on n vertices. Then the following statements are equivalent: (a) (b) (c) (d) (e) (f)

G is a tree (i.e. is connected and has no circuits). G has n − 1 edges and no circuits. G has n − 1 edges and is connected. G is a minimal connected graph (i.e. every edge is a bridge). G is a minimal graph with δ(X ) 6= ∅ for all ∅ 6= X ⊂ V (G). G is a maximal circuit-free graph (i.e. the addition of any edge creates a circuit). (g) G contains a unique path between any pair of vertices. Proof: (a)⇒(g) follows from the fact that the union of two distinct paths with the same endpoints contains a circuit. (g)⇒(e)⇒(d) follows from Proposition 2.3(a). (d)⇒(f) is trivial. (f)⇒(b)⇒(c): This follows from the fact that for forests with n vertices, m edges and p connected components n = m + p holds. (The proof is a trivial induction on m.)

18

2 Graphs

(c)⇒(a): Let G be connected with n −1 edges. As long as there are any circuits in G, we destroy them by deleting an edge of the circuit. Suppose we have deleted k edges. The resulting graph G 0 is still connected and has no circuits. G 0 has m = n − 1 − k edges. So n = m + p = n − 1 − k + 1, implying k = 0. In particular, (d)⇒(a) implies that a graph is connected if and only if it contains a spanning tree (a spanning subgraph which is a tree). A digraph is called connected if the underlying undirected graph is connected. A digraph is a branching if the underlying undirected graph is a forest and each vertex v has at most one entering edge. A connected branching is an arborescence. By Theorem 2.4 an arborescence with n vertices has n − 1 edges, hence it has exactly one vertex r with δ − (r ) = ∅. This vertex is called its root; we also speak of an arborescence rooted at r . The vertices v with δ + (v) = ∅ are called leaves. Theorem 2.5. Let G be a digraph on n vertices. Then the following statements are equivalent: (a) (b) (c) (d) (e) (f) (g)

G is an arborescence rooted at r (i.e. a connected branching with δ − (r ) = ∅). G is a branching with n − 1 edges and δ − (r ) = ∅. G has n − 1 edges and every vertex is reachable from r . Every vertex is reachable from r , but deleting any edge destroys this property. G is a minimal graph with δ + (X ) 6= ∅ for all X ⊂ V (G) with r ∈ X . δ − (r ) = ∅ and there is a unique r -v-path for any v ∈ V (G) \ {r }. δ − (r ) = ∅, |δ − (v)| = 1 for all v ∈ V (G) \ {r }, and G contains no circuit.

Proof: (a)⇒(b) and (c)⇒(d) follow from Theorem 2.4. (b)⇒(c): We have that |δ − (v)| = 1 for all v ∈ V (G) \ {r }. So for any v we have an r -v-path (start at v and always follow the entering edge until r is reached). (d)⇒(e) is implied by Proposition 2.3(b). (e)⇒(f): The minimality in (e) implies δ − (r ) = ∅. Moreover, by Proposition 2.3(b) there is an r -v-path for all v. Suppose there are two r -v-paths P and Q for some v. Let e be the last edge of P that does not belong to Q. Then after deleting e, every vertex is still reachable from r . By Proposition 2.3(b) this contradicts the minimality in (e). (f)⇒(g)⇒(a): trivial A cut in an undirected graph G is an edge set of type δ(X ) for some ∅ 6= X ⊂ V (G). In a digraph G, δ + (X ) is a directed cut if ∅ 6= X ⊂ V (G) and δ − (X ) = ∅, i.e. no edge enters the set X . We say that an edge set F ⊆ E(G) separates two vertices s and t if t is reachable from s in G but not in (V (G), E(G) \ F). In a digraph, an edge set δ + (X ) with s ∈ X and t ∈ / X is called an s-t-cut. An s-t-cut in an undirected graph is a cut δ(X ) for some X ⊂ V (G) with s ∈ X and t ∈ / X . An r-cut in a digraph is an edge set δ + (X ) for some X ⊂ V (G) with r ∈ X . By an undirected path, an undirected circuit, and an undirected cut in a digraph, we mean a subgraph corresponding to a path, a circuit, and a cut, respectively, in the underlying undirected graph.

2.2 Trees, Circuits, and Cuts

19

Lemma 2.6. (Minty [1960]) Let G be a digraph and e ∈ E(G). Suppose e is coloured black, while all other edges are coloured red, black or green. Then exactly one of the following statements holds: (a) There is an undirected circuit containing e and only red and black edges such that all black edges have the same orientation. (b) There is an undirected cut containing e and only green and black edges such that all black edges have the same orientation. Proof: Let e = (x, y). We label the vertices of G by the following procedure. First label y. In case v is already labelled and w is not, we label w if there is a black edge (v, w), a red edge (v, w) or a red edge (w, v). In this case, we write pr ed(w) := v. When the labelling procedure stops, there are two possibilities: Case 1: x has been labelled. Then the vertices x, pr ed(x), pr ed( pr ed(x)), . . . , y form an undirected circuit with the properties (a). Case 2: x has not been labelled. Then let R consist of all labelled vertices. Obviously, the undirected cut δ + (R) ∪ δ − (R) has the properties (b). Suppose that an undirected circuit C as in (a) and an undirected cut δ + (X ) ∪ − δ (X ) as in (b) both exist. All edges in their (nonempty) intersection are black, they all have the same orientation with respect to C, and they all leave X or all enter X . This is a contradiction. A digraph is called strongly connected if there is a path from s to t and a path from t to s for all s, t ∈ V (G). The strongly connected components of a digraph are the maximal strongly connected subgraphs. Corollary 2.7. In a digraph G, each edge belongs either to a (directed) circuit or to a directed cut. Moreover the following statements are equivalent: (a) G is strongly connected. (b) G contains no directed cut. (c) G is connected and each edge of G belongs to a circuit. Proof: The first statement follows directly from Minty’s Lemma 2.6 by colouring all edges black. This also proves (b)⇒(c). (a)⇒(b) follows from Proposition 2.3(b). (c)⇒(a): Let r ∈ V (G) be an arbitrary vertex. We prove that there is an r -vpath for each v ∈ V (G). Suppose this is not true, then by Proposition 2.3(b) there is some X ⊂ V (G) with r ∈ X and δ + (X ) = ∅. Since G is connected, we have δ + (X ) ∪ δ − (X ) 6= ∅ (by Proposition 2.3(a)), so let e ∈ δ − (X ). But then e cannot belong to a circuit since no edge leaves X . Corollary 2.7 and Theorem 2.5 imply that a digraph is strongly connected if and only if it contains for each vertex v a spanning arborescence rooted at v. A digraph is called acyclic if it contains no (directed) circuit. So by Corollary 2.7 a digraph is acyclic if and only if each edge belongs to a directed cut. Moreover,

20

2 Graphs

a digraph is acyclic if and only if its strongly connected components are the singletons. The vertices of an acyclic digraph can be ordered in a nice way: Definition 2.8. Let G be a digraph. A topological order of G is an order of the vertices V (G) = {v1 , . . . , vn } such that for each edge (vi , v j ) ∈ E(G) we have i < j. Proposition 2.9. A digraph has a topological order if and only if it is acyclic. Proof: If a digraph has a circuit, it clearly cannot have a topological order. We show the converse by induction on the number of edges. If there are no edges, every order is topological. Otherwise let e ∈ E(G); by Corollary 2.7 e belongs to a directed cut δ + (X ). Then a topological order of G[X ] followed by a topological order of G − X (both exist by the induction hypothesis) is a topological order of G. Circuits and cuts also play an important role in algebraic graph theory. For a graph G we associate a vector space R E(G) whose elements are vectors (xe )e∈E(G) with |E(G)| real components. Following Berge [1985] we shall now briefly discuss two linear subspaces which are particularly important. Let G be a digraph. We associate a vector ζ (C) ∈ {−1, 0, 1} E(G) with each undirected circuit C in G by setting ζ (C)e = 0 for e ∈ / E(C), and setting ζ (C)e ∈ {−1, 1} for e ∈ E(C) such that reorienting all edges e with ζ (C)e = −1 results in a directed circuit. Similarly, we associate a vector ζ (D) ∈ {−1, 0, 1} E(G) with / D, ζ (D)e = −1 each undirected cut D = δ(X ) in G by setting ζ (D)e = 0 for e ∈ for e ∈ δ − (X ) and ζ (D)e = 1 for e ∈ δ + (X ). Note that these vectors are properly defined only up to multiplication by −1. However, the subspaces of the vector space R E(G) generated by the set of vectors associated with the undirected circuits and by the set of vectors associated with the undirected cuts in G are properly defined; they are called the cycle space and the cocycle space of G, respectively. Proposition 2.10. The cycle space and the cocycle space are orthogonal to each other. Proof: Let C be any undirected circuit and D = δ(X ) be any undirected cut. We claim that the scalar product of ζ (C) and ζ (D) is zero. Since reorienting any edge does not change the scalar product we may assume that D is a directed cut. But then the result follows from observing that any circuit enters a set X the same number of times as it leaves X . We shall now show that the sum of the dimensions of the cycle space and the cocycle space is |E(G)|, the dimension of the whole space. A set of undirected circuits (undirected cuts) is called a cycle basis (a cocycle basis) if the associated vectors form a basis of the cycle space (the cocycle space, respectively). Let G be a graph (directed or undirected) and T a maximal subgraph without an undirected circuit. For each e ∈ E(G) \ E(T ) we call the unique undirected circuit in T + e the fundamental circuit of e with respect to T . Moreover, for each e ∈ E(T )

2.2 Trees, Circuits, and Cuts

21

there is a set X ⊆ V (G) with δG (X ) ∩ E(T ) = {e} (consider a component of T − e); we call δG (X ) the fundamental cut of e with respect to T . Theorem 2.11. Let G be a digraph and T a maximal subgraph without an undirected circuit. The |E(G) \ E(T )| fundamental circuits with respect to T form a cycle basis of G, and the |E(T )| fundamental cuts with respect to T form a cocycle basis of G. Proof: The vectors associated with the fundamental circuits are linearly independent since each fundamental circuit contains an element not belonging to any other. The same holds for the fundamental cuts. Since the vector spaces are orthogonal to each other by Proposition 2.10, the sum of their dimensions cannot exceed |E(G)| = |E(G) \ E(T )| + |E(T )|. The fundamental cuts have a nice property which we shall exploit quite often and which we shall discuss now. Let T be a digraph whose underlying undirected graph is a tree. Consider the family F := {Ce : e ∈ E(T )}, where for e = (x, y) ∈ E(T ) we denote by Ce the connected component of T − e containing y (so δ(Ce ) is the fundamental cut of e with respect to T ). If T is an arborescence, then any two elements of F are either disjoint or one is a subset of the other. In general F is at least cross-free: Deﬁnition 2.12. A set system is a pair (U, F), where U is a nonempty ﬁnite set and F a family of subsets of U . (U, F) is cross-free if for any two sets X, Y ∈ F, at least one of the four sets X \ Y , Y \ X , X ∩ Y , U \ (X ∪ Y ) is empty. (U, F) is laminar if for any two sets X, Y ∈ F, at least one of the three sets X \ Y , Y \ X , X ∩ Y is empty. In the literature set systems are also known as hypergraphs. See Figure 2.1(a) for an illustration of the laminar family {{a}, {b, c}, {a, b, c}, {a, b, c, d}, { f }, { f, g}}. Another word used for laminar is nested. (a)

(b) e

g

d a

b c

d

e

f

g f

a Fig. 2.1.

b, c

22

2 Graphs

Whether a set system (U, F) is laminar does not depend on U , so we sometimes simply say that F is a laminar family. However, whether a set system is cross-free can depend on the ground set U . If U contains an element that does not belong to any set of F, then F is cross-free if and only if it is laminar. Let r ∈ U be arbitrary. It follows directly from the deﬁnition that a set system (U, F) is cross-free if and only if F := {X ∈ F : r ∈ X } ∪ {U \ X : X ∈ F, r ∈ X } is laminar. Hence cross-free families are sometimes depicted similarly to laminar families: for example, Figure 2.2(a) shows the cross-free family {{b, c, d, e, f }, {c}, {a, b, c}, {e}, {a, b, c, d, f }, {e, f }}; a square corresponds to the set containing all elements outside. (a)

(b) d

f

b a

b

c

d

e

f a

c

e Fig. 2.2.

While oriented trees lead to cross-free families the converse is also true: every cross-free family can be represented by a tree in the following sense: Deﬁnition 2.13. Let T be a digraph such that the underlying undirected graph is a tree. Let U be a ﬁnite set and ϕ : U → V (T ). Let F := {Se : e ∈ E(T )}, where for e = (x, y) we deﬁne Se := {s ∈ U : ϕ(s) is in the same connected component of T − e as y}. Then (T, ϕ) is called a tree-representation of (U, F). See Figures 2.1(b) and 2.2(b) for examples. Proposition 2.14. Let (U, F) be a set system with a tree-representation (T, ϕ). Then (U, F) is cross-free. If T is an arborescence, then (U, F) is laminar. Moreover, every cross-free family has a tree-representation, and for laminar families, an arborescence can be chosen as T .

2.2 Trees, Circuits, and Cuts

23

Proof: If (T, ϕ) is a tree-representation of (U, F) and e = (v, w), f = (x, y) ∈ E(T ), we have an undirected v-x-path P in T (ignoring the orientations). There are four cases: If w, y ∈ / V (P) then Se ∩ Sf = ∅ (since T contains no circuit). / V (P) and w ∈ V (P) then If w ∈ / V (P) and y ∈ V (P) then Se ⊆ Sf . If y ∈ Sf ⊆ Se . If w, y ∈ V (P) then Se ∪ Sf = U . Hence (U, F) is cross-free. If T is an arborescence, the last case cannot occur (otherwise at least one vertex of P would have two entering edges), so F is laminar. . To prove the converse, let F first be a laminar family. We define V (T ) := F ∪ {r } and E(T ) := {(X, Y ) ∈ F × F : X ⊃ Y = 6 ∅ and there is no Z ∈ F with X ⊃ Z ⊃ Y } ∪ {(r, X ) : X = ∅ ∈ F or X is a maximal element of F} . We set ϕ(x) := X , where X is the minimal set in F containing x, and ϕ(x) := r if no set in F contains x. Obviously, T is an arborescence rooted at r , and (T, ϕ) is a tree-representation of F. Now let F be a cross-free family of subsets of U . Let r ∈ U . As noted above, F 0 := {X ∈ F : r 6∈ X } ∪ {U \ X : X ∈ F, r ∈ X } is laminar, so let (T, ϕ) be a tree-representation of (U, F 0 ). Now for an edge e ∈ E(T ) there are three cases: If Se ∈ F and U \ Se ∈ F, we replace the edge e = (x, y) by two edges (x, z) and (y, z), where z is a new vertex. If Se 6∈ F and U \ Se ∈ F, we replace the edge e = (x, y) by (y, x). If Se ∈ F and U \ Se 6∈ F, we do nothing. Let T 0 be the resulting graph. Then (T 0 , ϕ) is a tree-representation of (U, F). The above result is mentioned by Edmonds and Giles [1977] but was probably known earlier. Corollary 2.15. A laminar family of distinct subsets of U has at most 2|U | elements. A cross-free family of distinct subsets of U has at most 4|U | − 2 elements. Proof: We first consider a laminar family F of distinct nonempty proper subsets of U . We prove that |F| ≤ 2|U | − 2. Let (T, ϕ) be a tree-representation, where T is an arborescence whose number of vertices is as small as possible. For every w ∈ V (T ) we have either |δ + (w)| ≥ 2 or there exists an x ∈ U with ϕ(x) = w or both. (For the root this follows from U ∈ / F, for the leaves from ∅ ∈ / F, for all other vertices from the minimality of T .) j There k can be at most |U | vertices w with ϕ(x) = w for some x ∈ U and at most |E(T )| )| vertices w with |δ + (w)| ≥ 2. So |E(T )| + 1 = |V (T )| ≤ |U | + |E(T 2 2 and thus |F| = |E(T )| ≤ 2|U | − 2. Now let (U, F) be a cross-free family with ∅, U ∈ / F, and let r ∈ U . Since F 0 := {X ∈ F : r 6∈ X } ∪ {U \ X : X ∈ F, r ∈ X } is laminar, we have |F 0 | ≤ 2|U | − 2. Hence |F| ≤ 2|F 0 | ≤ 4|U | − 4. The proof is concluded by taking ∅ and U as possible members of F into account.

24

2 Graphs

2.3 Connectivity Connectivity is a very important concept in graph theory. For many problems it suffices to consider connected graphs, since otherwise we can solve the problem for each connected component separately. So it is a fundamental task to detect the connected components of a graph. The following simple algorithm finds a path from a specified vertex s to all other vertices that are reachable from s. It works for both directed and undirected graphs. In the undirected case it builds a maximal tree containing s; in the directed case it constructs a maximal arborescence rooted at s.

G RAPH S CANNING A LGORITHM Input:

A graph G (directed or undirected) and some vertex s.

Output:

The set R of vertices reachable from s, and a set T ⊆ E(G) such that (R, T ) is an arborescence rooted at s, or a tree.

1

Set R := {s}, Q := {s} and T := ∅.

2

If Q = ∅ then stop, else choose a v ∈ Q. Choose a w ∈ V (G) \ R with e = (v, w) ∈ E(G) or e = {v, w} ∈ E(G). If there is no such w then set Q := Q \ {v} and go to . 2 Set R := R ∪ {w}, Q := Q ∪ {w} and T := T ∪ {e}. Go to . 2

3

4

Proposition 2.16. The G RAPH S CANNING A LGORITHM works correctly. Proof: At any time, (R, T ) is a tree or an arborescence rooted at s. Suppose at the end there is a vertex w ∈ V (G) \ R that is reachable from s. Let P be an s-w-path, and let {x, y} or (x, y) be an edge of P with x ∈ R and y ∈ / R. Since x has been added to R, it also has been added to Q at some time during the execution of the algorithm. The algorithm does not stop before removing x from Q. But this is done in / R. 3 only if there is no edge {x, y} or (x, y) with y ∈ Since this is the first graph algorithm in this book we discuss some implementation issues. The first question is how the graph is given. There are several natural ways. For example, one can think of a matrix with a row for each vertex and a column for each edge. The incidence matrix of an undirected graph G is the matrix A = (av,e )v∈V (G), e∈E(G) where ( 1 if v ∈ e av,e = . 0 if v 6∈ e The incidence matrix of a digraph G is the matrix A = (av,e )v∈V (G), e∈E(G) where −1 if v = x . av,(x,y) = 1 if v = y 0 if v 6∈ {x, y}

2.3 Connectivity

25

Of course this is not very efficient since each column contains only two nonzero entries. The space needed for storing an incidence matrix is obviously O(nm), where n := |V (G)| and m := |E(G)|. A better way seems to be having a matrix whose rows and columns are indexed by the vertex set. The adjacency matrix of a simple graph G is the 0-1-matrix A = (av,w )v,w∈V (G) with av,w = 1 iff {v, w} ∈ E(G) or (v, w) ∈ E(G). For graphs with parallel edges we can define av,w to be the number of edges from v to w. An adjacency matrix requires O(n 2 ) space for simple graphs. The adjacency matrix is appropriate if the graph is dense, i.e. has 2(n 2 ) edges (or more). For sparse graphs, say with O(n) edges only, one can do much better. Besides storing the number of vertices we can simply store a list of the edges, for each edge noting its endpoints. If we address each vertex by a number from 1 to n, the space needed for each edge is O(log n). Hence we need O(m log n) space altogether. Just storing the edges in an arbitrary order is not very convenient. Almost all graph algorithms require finding the edges incident to a given vertex. Thus one should have a list of incident edges for each vertex. In case of directed graphs, two lists, one for entering edges and one for leaving edges, are appropriate. This data structure is called adjacency list; it is the most customary one for graphs. For direct access to the list(s) of each vertex we have pointers to the heads of all lists; these can be stored with O(n log m) additional bits. Hence the total number of bits required for an adjacency list is O(n log m + m log n). Whenever a graph is part of the input of an algorithm in this book, we assume that the graph is given by an adjacency list. As for elementary operations on numbers (see Section 1.2), we assume that elementary operations on vertices and edges take constant time only. This includes scanning an edge, identifying its ends and accessing the head of the adjacency list for a vertex. The running time will be measured by the parameters n and m, and an algorithm running in O(m + n) time is called linear. We shall always use the letters n and m for the number of vertices and the number of edges. For many graph algorithms it causes no loss of generality to assume that the graph at hand is simple and connected; hence n − 1 ≤ m < n 2 . Among parallel edges we often have to consider only one, and different connected components can often be analyzed separately. The preprocessing can be done in linear time in advance; see Exercise 13 and the following. We can now analyze the running time of the G RAPH S CANNING A LGORITHM: Proposition 2.17. The G RAPH S CANNING A LGORITHM can be implemented to run in O(m + n) time. The connected components of a graph can be determined in linear time. Proof: We assume that G is given by an adjacency list. For each vertex x we introduce a pointer current(x), indicating the current edge in the list containing all edges in δ(x) or δ + (x) (this list is part of the input). Initially current(x) is set to the first element of the list. In , 3 the pointer moves forward. When the end of

26

2 Graphs

the list is reached, x is removed from Q and will never be inserted again. So the overall running time is proportional to the number of vertices plus the number of edges, i.e. O(n + m). To identify the connected components of a graph, we apply the algorithm once and check if R = V (G). If so, the graph is connected. Otherwise R is a connected component, and we apply the algorithm to (G, s 0 ) for an arbitrary vertex s 0 ∈ V (G) \ R (and iterate until all vertices have been scanned, i.e. added to R). Again, no edge is scanned twice, so the overall running time remains linear. An interesting question is in which order the vertices are chosen in . 3 Obviously we cannot say much about this order if we do not specify how to choose a v ∈ Q in . 2 Two methods are frequently used; they are called D EPTH -F IRST S EARCH (DFS) and B READTH -F IRST S EARCH (BFS). In DFS we choose the v ∈ Q that was the last to enter Q. In other words, Q is implemented as a LIFO-stack (last-in-first-out). In BFS we choose the v ∈ Q that was the first to enter Q. Here Q is implemented by a FIFO-queue (first-in-first-out). An algorithm similar to DFS has been described already before 1900 by Tr´emaux and Tarry; see K¨onig [1936]. BFS seems to have been mentioned first by Moore [1959]. Trees (in the directed case: arborescences) (R, T ) computed by DFS and BFS are called DFS-tree and BFS-tree, respectively. For BFS-trees we note the following important property: Proposition 2.18. A BFS-tree contains a shortest path from s to each vertex reachable from s. The values distG (s, v) for all v ∈ V (G) can be determined in linear time. Proof: We apply BFS to (G, s) and add two statements: initially (in 1 of the G RAPH S CANNING A LGORITHM) we set l(s) := 0, and in 4 we set l(w) := l(v) + 1. We obviously have that l(v) = dist(R,T ) (s, v) for all v ∈ R, at any stage of the algorithm. Moreover, if v is the currently scanned vertex (chosen in ), 2 at this time there is no vertex w ∈ R with l(w) > l(v) + 1 (because the vertices are scanned in an order with nondecreasing l-values). Suppose that when the algorithm terminates there is a vertex w ∈ V (G) with distG (s, w) < dist(R,T ) (s, w); let w have minimum distance from s in G with this property. Let P be a shortest s-w-path in G, and let e = (v, w) or e = {v, w} be the last edge in P. We have distG (s, v) = dist(R,T ) (s, v), but e does not belong to T . Moreover, l(w) = dist(R,T ) (s, w) > distG (s, w) = distG (s, v) + 1 = dist(R,T ) (s, v)+1 = l(v)+1. This inequality combined with the above observation proves that w did not belong to R when v was removed from Q. But this contradicts

3 because of edge e. This result will also follow from the correctness of D IJKSTRA ’ S A LGORITHM for the S HORTEST PATH P ROBLEM, which can be thought of as a generalization of BFS to the case where we have nonnegative weights on the edges (see Section 7.1).

2.3 Connectivity

27

We now show how to identify the strongly connected components of a digraph. Of course, this can easily be done by using n times DFS (or BFS). However, it is possible to find the strongly connected components by visiting every edge only twice:

S TRONGLY C ONNECTED C OMPONENT A LGORITHM Input:

A digraph G.

Output:

A function comp : V (G) → N indicating the membership of the strongly connected components.

1

Set R := ∅. Set N := 0.

2

For all v ∈ V (G) do: If v ∈ / R then V ISIT1(v).

3

Set R := ∅. Set K := 0.

4

For i := |V (G)| down to 1 do: / R then set K := K + 1 and V ISIT2(ψ −1 (i)). If ψ −1 (i) ∈

V ISIT1(v)

1

Set R := R ∪ {v}.

2

For all w ∈ V (G) \ R with (v, w) ∈ E(G) do V ISIT1(w).

3

Set N := N + 1, ψ(v) := N and ψ −1 (N ) := v.

V ISIT2(v)

1

Set R := R ∪ {v}.

2

For all w ∈ V (G) \ R with (w, v) ∈ E(G) do V ISIT2(w).

3

Set comp(v) := K .

Figure 2.3 shows an example: The first DFS scans the vertices in the order a, g, b, d, e, f and produces the arborescence shown in the middle; the numbers are the ψ-labels. Vertex c is the only one that is not reachable from a; it gets the highest label ψ(c) = 7. The second DFS starts with c but cannot reach any other vertex via a reverse edge. So it proceeds with vertex a because ψ(a) = 6. Now b, g and f can be reached. Finally e is reached from d. The strongly connected components are {c}, {a, b, f, g} and {d, e}. In summary, one DFS is needed to find an appropriate numbering, while in the second DFS the reverse graph is considered and the vertices are processed in decreasing order with respect to this numbering. Each connected component of the second DFS-forest is an anti-arborescence, a graph arising from an arborescence by reversing every edge. We show that these anti-arborescences identify the strongly connected components.

28

2 Graphs

Fig. 2.3.

Theorem 2.19. The S TRONGLY C ONNECTED C OMPONENT A LGORITHM identifies the strongly connected components correctly in linear time. Proof: The running time is obviously O(n + m). Of course, vertices of the same strongly connected component are always in the same component of any DFS-forest, so they get the same comp-value. We have to prove that two vertices u and v with comp(u) = comp(v) indeed lie in the same strongly connected component. Let r (u) and r (v) be the vertex reachable from u and v with the highest ψ-label, respectively. Since comp(u) = comp(v), i.e. u and v lie in the same anti-arborescence of the second DFS-forest, r := r (u) = r (v) is the root of this anti-arborescence. So r is reachable from both u and v. Since r is reachable from u and ψ(r ) ≥ ψ(u), r has not been added to R after u in the first DFS, and the first DFS-forest contains an r -u-path. In other words, u is reachable from r . Analogously, v is reachable from r . Altogether, u is reachable from v and vice versa, proving that indeed u and v belong to the same strongly connected component. It is interesting that this algorithm also solves another problem: finding a topological order of an acyclic digraph. Observe that contracting the strongly connected components of any digraph yields an acyclic digraph. By Proposition 2.9 this acyclic digraph has a topological order. In fact, such an order is given by the numbers comp(v) computed by the S TRONGLY C ONNECTED C OMPONENT A LGORITHM: Theorem 2.20. The S TRONGLY C ONNECTED C OMPONENT A LGORITHM determines a topological order of the digraph resulting from contracting each strongly connected component of G. In particular, we can for any given digraph either find a topological order or decide that none exists in linear time. Proof: Let X and Y be two strongly connected components of a digraph G, and suppose the S TRONGLY C ONNECTED C OMPONENT A LGORITHM computes comp(x) = k1 for x ∈ X and comp(y) = k2 for y ∈ Y with k1 < k2 . We claim + (Y, X ) = ∅. that E G Suppose that there is an edge (y, x) ∈ E(G) with y ∈ Y and x ∈ X . All vertices in X are added to R in the second DFS before the first vertex of Y is

2.3 Connectivity

29

added. In particular we have x ∈ R and y ∈ / R when the edge (y, x) is scanned in the second DFS. But this means that y is added to R before K is incremented, contradicting comp(y) 6= comp(x). Hence the comp-values computed by the S TRONGLY C ONNECTED C OMPO NENT A LGORITHM determine a topological order of the digraph resulting from contracting the strongly connected components. The second statement of the theorem now follows from Proposition 2.9 and the observation that a digraph is acyclic if and only if its strongly connected components are the singletons. The first linear-time algorithm that identifies the strongly connected components was given by Tarjan [1972]. The problem of finding a topological order (or deciding that none exists) was solved earlier (Kahn [1962], Knuth [1968]). Both BFS and DFS occur as subroutines in many other combinatorial algorithms. Some examples will appear in later chapters. Sometimes one is interested in higher connectivity. Let k ≥ 2. An undirected graph with more than k vertices and the property that it remains connected even if we delete any k − 1 vertices, is called k-connected. A graph with at least two vertices is k-edge-connected if it remains connected after deleting any k −1 edges. So a connected graph with at least three vertices is 2-connected (2-edge-connected) if and only if it has no articulation vertex (no bridge, respectively). The largest k and l such that a graph G is k-connected and l-edge-connected are called the vertex-connectivity and edge-connectivity of G. Here we say that a graph is 1-connected (and 1-edge-connected) if it is connected. A disconnected graph has vertex-connectivity and edge-connectivity zero. The blocks of an undirected graph are its maximal connected subgraphs without articulation vertex. Thus each block is either a maximal 2-connected subgraph, or consists of a bridge or an isolated vertex. Two blocks have at most one vertex in common, and a vertex belonging to more than one block is an articulation vertex. The blocks of an undirected graph can be determined in linear time quite similarly to the S TRONGLY C ONNECTED C OMPONENT A LGORITHM; see Exercise 16. Here we prove a nice structure theorem for 2-connected graphs. We construct graphs from a single vertex by sequentially adding ears: Definition 2.21. Let G be a graph (directed or undirected). An ear-decomposition of G is a sequence r, P1 , . . . , Pk with G = ({r }, ∅)+ P1 +· · ·+ Pk , such that each Pi is either a path where exactly the endpoints belong to {r } ∪ V (P1 ) ∪ · · · ∪ V (Pi−1 ), or a circuit where exactly one of its vertices belongs to {r } ∪ V (P1 ) ∪ · · · ∪ V (Pi−1 ) (i ∈ {1, . . . , k}). P1 , . . . , Pk are called ears. If k ≥ 1, P1 is a circuit of length at least three, and P2 , . . . , Pk are paths, then the ear-decomposition is called proper. Theorem 2.22. (Whitney [1932]) An undirected graph is 2-connected if and only if it has a proper ear-decomposition. Proof: Evidently a circuit of length at least three is 2-connected. Moreover, if G is 2-connected, then so is G + P, where P is an x-y-path, x, y ∈ V (G) and x 6= y:

30

2 Graphs

deleting any vertex does not destroy connectivity. We conclude that a graph with a proper ear-decomposition is 2-connected. To show the converse, let G be a 2-connected graph. Let G 0 be the maximal simple subgraph of G; evidently G 0 is also 2-connected. Hence G 0 cannot be a tree; i.e. it contains a circuit. Since it is simple, G 0 , and thus G, contains a circuit of length at least three. So let H be a maximal subgraph of G that has a proper ear-decomposition; H exists by the above consideration. Suppose H is not spanning. Since G is connected, we then know that there exists an edge e = {x, y} ∈ E(G) with x ∈ V (H ) and y ∈ / V (H ). Let z be a vertex in V (H ) \ {x}. Since G − x is connected, there exists a path P from y to z in G − x. Let z 0 be the first vertex on this path, when traversed from y, that belongs to V (H ). Then P[y,z 0 ] + e can be added as an ear, contradicting the maximality of H . Thus H is spanning. Since each edge of E(G) \ E(H ) can be added as an ear, we conclude that H = G. See Exercise 17 for similar characterizations of 2-edge-connected graphs and strongly connected digraphs.

2.4 Eulerian and Bipartite Graphs Euler’s work on the problem of traversing each of the seven bridges of K¨onigsberg exactly once was the origin of graph theory. He showed that the problem had no solution by defining a graph, asking for a walk containing all edges, and observing that more than two vertices had odd degree. Definition 2.23. An Eulerian walk in a graph G is a closed walk containing every edge. An undirected graph G is called Eulerian if the degree of each vertex is even. A digraph G is Eulerian if |δ − (v)| = |δ + (v)| for each v ∈ V (G). Although Euler neither proved sufficiency nor considered the case explicitly in which we ask for a closed walk, the following famous result is usually attributed to him: Theorem 2.24. (Euler [1736], Hierholzer [1873]) A connected graph has an Eulerian walk if and only if it is Eulerian. Proof: The necessity of the degree conditions is obvious, the sufficiency is proved by the following algorithm (Theorem 2.25). The algorithm accepts as input only connected Eulerian graphs. Note that one can check in linear time whether a given graph is connected (Theorem 2.17) and Eulerian (trivial). The algorithm first chooses an initial vertex, then calls a recursive procedure. We first describe it for undirected graphs:

2.4 Eulerian and Bipartite Graphs

31

E ULER ’ S A LGORITHM Input:

An undirected connected Eulerian graph G.

Output:

An Eulerian walk W in G.

1

Choose v1 ∈ V (G) arbitrarily. Return W := E ULER(G, v1 ).

E ULER (G, v1 )

1

Set W := v1 and x := v1 .

2

If δ(x) = ∅ then go to . 4 Else let e ∈ δ(x), say e = {x, y}. Set W := W, e, y and x := y. Set E(G) := E(G) \ {e} and go to . 2

3

4

5

Let v1 , e1 , v2 , e2 , . . . , vk , ek , vk+1 be the sequence W . For i := 1 to k do: Set Wi := E ULER(G, vi ). Set W := W1 , e1 , W2 , e2 , . . . , Wk , ek , vk+1 . Return W . For digraphs, 2 has to be replaced by:

2

If δ + (x) = ∅ then go to . 4 Else let e ∈ δ + (x), say e = (x, y). We can analyze both versions (undirected and directed) simultaneously:

Theorem 2.25. E ULER ’ S A LGORITHM works correctly. Its running time is O(m+ n), where n = |V (G)| and m = |E(G)|. Proof: We use induction on |E(G)|, the case E(G) = ∅ being trivial. Because of the degree conditions, vk+1 = x = v1 when 4 is executed. So at this stage W is a closed walk. Let G 0 be the graph G at this stage. G 0 also satisfies the degree constraints. For each edge e ∈ E(G 0 ) there exists a minimum i ∈ {1, . . . , k} such that e is in the same connected component of G 0 as vi . Then by the induction hypothesis e belongs to Wi . So the closed walk W composed in 5 is indeed Eulerian. The running time is linear, because each edge is deleted immediately after being examined. E ULER ’ S A LGORITHM will be used several times as a subroutine in later chapters. Sometimes one is interested in making a given graph Eulerian by adding or contracting edges. Let G be an undirected graph and F a family of unordered . pairs of V (G) (edges or not). F is called an odd join if (V (G), E(G) ∪ F) is Eulerian. F is called an odd cover if the graph which results from G by successively contracting each e ∈ F is Eulerian. Both concepts are equivalent in the following sense.

32

2 Graphs

Theorem 2.26. (Aoshima and Iri [1977]) For any undirected graph we have: (a) Every odd join is an odd cover. (b) Every minimal odd cover is an odd join. Proof: Let G be an undirected graph. To prove (a), let F be an odd join. We build a graph G 0 by contracting the connected components of (V (G), F) in G. Each of these connected components contains an even number of odd-degree vertices (with respect to F and thus with respect to G, because F is an odd join). So the resulting graph has even degrees only. Thus F is an odd cover. To prove (b), let F be a minimal odd cover. Because of the minimality, (V (G), F) is a forest. We have to show that |δ F (v)| ≡ |δG (v)| (mod 2) for each v ∈ V (G). So let v ∈ V (G). Let C1 , . . . , Ck be the connected components of (V (G), F) − v that contain a vertex w with {v, w} ∈ F. Since F is a forest, k = |δ F (v)|. As F is an odd cover, contracting X := V (C1 ) ∪ · · · ∪ V (Ck ) ∪ {v} in G yields a vertex of even degree, i.e. |δG (X )| is even. On the other hand, because of the minimality of F, F \ {{v, w}} is not an odd cover (for any w with {v, w} ∈ F), so |δG (V (Ci ))| is odd for i = 1, . . . , k. Since k X

|δG (V (Ci ))| = |δG (X )| + |δG (v)|

i=1

− 2|E G ({v}, V (G) \ X )| + 2

X

|E G (Ci , C j )|,

1≤i< j≤k

we conclude that k has the same parity as |δG (v)|.

We shall return to the problem of making a graph Eulerian in Section 12.2. A bipartition of an undirected graph G is a partition of the vertex set V (G) = . A ∪ B such that the subgraphs induced by A and B are both empty. A graph is called . bipartite if it has a bipartition. The simple bipartite graph G with V (G) = A ∪ B, |A| = n, |B| = m and E(G) = {{a, b} : a ∈ A, b ∈ B} is denoted by K n,m (the . complete bipartite graph). When we write G = (A ∪ B, E(G)), we mean that G[A] and G[B] are both empty. Proposition 2.27. (K¨onig [1916]) An undirected graph is bipartite if and only if it contains no circuit of odd length. There is a linear-time algorithm which, given an undirected graph G, either finds a bipartition or an odd circuit. .

Proof: Suppose G is bipartite with bipartition V (G) = A ∪ B, and the closed walk v1 , e1 , v2 , . . . , vk , ek , vk+1 defines some circuit in G. W.l.o.g. v1 ∈ A. But then v2 ∈ B, v3 ∈ A, and so on. We conclude that vi ∈ A if and only if i is odd. But vk+1 = v1 ∈ A, so k must be even. To prove the sufficiency, we may assume that G is connected, since a graph is bipartite iff each connected component is (and the connected components can

2.5 Planarity

33

be determined in linear time; Proposition 2.17). We choose an arbitrary vertex s ∈ V (G) and apply BFS to (G, s) in order to obtain the distances from s to v for all v ∈ V (G) (see Proposition 2.18). Let T be the resulting BFS-tree. Define A := {v ∈ V (G) : distG (s, v) is even} and B := V (G) \ A. If there is an edge e = {x, y} in G[A] or G[B], the x-y-path in T together with e forms an odd circuit in G. If there is no such edge, we have a bipartition.

2.5 Planarity We often draw graphs in the plane. A graph is called planar if it can be drawn such that no pair of edges intersect. To formalize this concept we need the following topological terms: Definition 2.28. A simple Jordan curve is the image of a continuous injective function ϕ : [0, 1] → R2 ; its endpoints are ϕ(0) and ϕ(1). A closed Jordan curve is the image of a continuous function ϕ : [0, 1] → R2 with ϕ(0) = ϕ(1) and ϕ(τ ) 6= ϕ(τ 0 ) for 0 ≤ τ < τ 0 < 1. A polygonal arc is a simple Jordan curve which is the union of finitely many intervals (straight line segments). A polygon is a closed Jordan curve which is the union of finitely many intervals. Let R = R2 \ J , where J is the union of finitely many intervals. We define the connected regions of R as equivalence classes where two points in R are equivalent if they can be joined by a polygonal arc within R. Definition 2.29. A planar embedding of a graph G consists of an injective mapping ψ : V (G) → R2 and for each e = {x, y} ∈ E(G) a polygonal arc Je with endpoints ψ(x) and ψ(y), such that for each e = {x, y} ∈ E(G): [ Je0 = ∅. (Je \ {ψ(x), ψ(y)}) ∩ {ψ(v) : v ∈ V (G)} ∪ e0 ∈E(G)\{e}

A graph is called planar if it has a planar embedding. Let G be a (planar) graph with some fixed planar embedding 8 = (ψ, (Je )e∈E(G) ). After removing the points and polygonal arcs from the plane, the remainder, [ Je , R := R2 \ {ψ(v) : v ∈ V (G)} ∪ e∈E(G)

splits into open connected regions, called faces of 8. For example, K 4 is obviously planar but it will turn out that K 5 is not planar. Exercise 23 shows that restricting ourselves to polygonal arcs instead of arbitrary Jordan curves makes no substantial difference. We will show later that for simple graphs it is indeed sufficient to consider straight line segments only. Our aim is to characterize planar graphs. Following Thomassen [1981], we first prove the following topological fact, a version of the Jordan curve theorem:

34

2 Graphs

Theorem 2.30. If J is a polygon, then R2 \ J splits into exactly two connected regions, each of which has J as its boundary. If J is a polygonal arc, then R2 \ J has only one connected region. Proof: Let J be a polygon, p ∈ R2 \ J and q ∈ J . Then there exists a polygonal arc in (R2 \ J ) ∪ {q} joining p and q: starting from p, one follows the straight line towards q until one gets close to J , then one proceeds within the vicinity of J . (We use the elementary topological fact that disjoint compact sets, in particular non-adjacent intervals of J , have a positive distance from each other.) We conclude that p is in the same connected region of R2 \ J as some points arbitrarily close to q. J is the union of ﬁnitely many intervals; one or two of these intervals contain q. Let > 0 such that the ball with center q and radius contains no other interval of J ; then clearly this ball intersects at most two connected regions. Since p ∈ R2 \ J and q ∈ J were chosen arbitrarily, we conclude that there are at most two regions and each region has J as its boundary. Since the above also holds if J is a polygonal arc and q is an endpoint of J , R2 \ J has only one connected region in this case. Returning to the case when J is a polygon, it remains to prove that R2 \ J has more than one region. For any p ∈ R2 \ J and any angle α we consider the ray lα starting at p with angle α. J ∩ lα is a set of points or closed intervals. Let cr ( p, lα ) be the number of these points or intervals that J enters from a different side of lα than to which it leaves (the number of times J “crosses” lα ; e.g. in Figure 2.4 we have cr ( p, lα ) = 2).

J

J

p

lα

J Fig. 2.4.

Note that for any angle α, lim cr ( p, lα+ ) − cr ( p, lα ) + lim cr ( p, lα+ ) − cr ( p, lα ) →0, 0 is twice the number of intervals of J ∩ lα that J enters from the same side as to which it leaves. Therefore g( p, α) := (cr ( p, lα ) mod 2) is a continuous function in α, so it is constant and we denote it by g( p). Clearly g( p) is constant for points

2.5 Planarity

35

p on each straight line not intersecting J , so it is constant within each region. However, g( p) 6= g(q) for points p, q such that the straight line segment joining p and q intersects J exactly once. Hence there are indeed two regions. Exactly one of the faces, the outer face, is unbounded. Proposition 2.31. Let G be a 2-connected graph with a planar embedding 8. Then every face is bounded by a circuit, and every edge is on the boundary of exactly two faces. Moreover, the number of faces is |E(G)| − |V (G)| + 2. Proof: By Theorem 2.30 both assertions are true if G is a circuit. For general 2-connected graphs we use induction on the number of edges, using Theorem 2.22. Consider a proper ear-decomposition of G, and let P be the last ear, a path with endpoints x and y, say. Let G 0 be the graph before adding the last ear, and let 80 be the restriction of 8 to G 0 . S Let 8 = (ψ, (Je )e∈E(G) ). Let F 0 be the face of 80 containing e∈E(P) Je \ {ψ(x), ψ(y)}. By induction, F 0 is bounded by a circuit C. C contains x and y, so C is the union of two x-y-paths Q 1 , Q 2 in G 0 . Now we apply Theorem 2.30 to each of the circuits Q 1 + P and Q 2 + P. We conclude that [ . . F 0 ∪ {ψ(x), ψ(y)} = F1 ∪ F2 ∪ Je e∈E(P)

and F1 and F2 are two faces of G bounded by the circuits Q 1 + P and Q 2 + P, respectively. Hence G has one more face than G 0 . Using |E(G)\ E(G 0 )| = |V (G)\ V (G 0 )| + 1, this completes the induction step. This proof is due to Tutte. It also implies easily that the circuits bounding the finite faces constitute a cycle basis (Exercise 24). The last statement of Proposition 2.31 is known as Euler’s formula; it holds for general connected graphs: Theorem 2.32. (Euler [1758], Legendre [1794]) For any planar connected graph G with any embedding, the number of faces is |E(G)| − |V (G)| + 2. Proof: We have already proved the statement for 2-connected graphs (Proposition 2.31). Moreover, the assertion is trivial if |V (G)| = 1 and follows from Theorem 2.30 if |E(G)| = 1. If |V (G)| = 2 and |E(G)| ≥ 2, then we can subdivide one edge e, thereby increasing the number of vertices and the number of edges by one and making the graph 2-connected, and apply Proposition 2.31. So we may now assume that G has an articulation vertex x; we proceed by induction on the number of vertices. Let 8 be an embedding of G. Let C1 , . . . , Ck be the connected components of G − x; and let 8i be the restriction of 8 to G i := G[V (Ci ) ∪ {x}] for i = 1, . . . , k. The set of inner (bounded) faces of 8 is the disjoint union of the sets of inner faces of 8i , i = 1, . . . , k. By applying the induction hypothesis to (G i , 8i ), i = 1, . . . , k, we get that the total number of inner faces of (G, 8) is

36 k i=1

2 Graphs

(|E(G i )|−|V (G i )|+1) = |E(G)|−

k

|V (G i )\{x}| = |E(G)|−|V (G)|+1.

i=1

Taking the outer face into account concludes the proof.

In particular, the number of faces is independent of the embedding. The average degree of a simple planar graph is less than 6: Corollary 2.33. Let G be a 2-connected simple planar graph whose minimum k circuit has length k (we also say that G has girth k). Then G has at most (n −2) k−2 edges. Any simple planar graph with n ≥ 3 vertices has at most 3n − 6 edges. Proof: First assume that G is 2-connected. Let some embedding of G be given, and let r be the number of faces. By Euler’s formula (Theorem 2.32), r = |E(G)| − |V (G)| + 2. By Proposition 2.31, each face is bounded by a circuit, i.e. by at least k edges, and each edge is on the boundary of exactly two faces. Hence kr ≤ 2|E(G)|. Combining the two results we get |E(G)| − |V (G)| + 2 ≤ k2 |E(G)|, k implying |E(G)| ≤ (n − 2) k−2 . If G is not 2-connected we add edges between non-adjacent vertices to make it 3 2-connected while preserving planarity. By the ﬁrst part we have at most (n −2) 3−2 edges, including the new ones. Now we show that certain graphs are non-planar: Corollary 2.34. Neither K 5 nor K 3,3 is planar. Proof: This follows directly from Corollary 2.33: K 5 has ﬁve vertices but 10 > 4 3·5−6 edges; K 3,3 is 2-connected, has girth 4 (as it is bipartite) and 9 > (6−2) 4−2 edges.

Fig. 2.5.

Figure 2.5 shows these two graphs, which are the smallest non-planar graphs. We shall prove that every non-planar graph contains, in a certain sense, K 5 or K 3,3 . To make this precise we need the following notion: Deﬁnition 2.35. Let G and H be two undirected graphs. G is .a minor of H if . there exists a subgraph H of H and a partition V (H ) = V1 ∪ · · · ∪ Vk of its

2.5 Planarity

37

vertex set into connected subsets such that contracting each of V1 , . . . , Vk yields a graph which is isomorphic to G. In other words, G is a minor of H if it can be obtained from H by a series of operations of the following type: delete a vertex, delete an edge or contract an edge. Since neither of these operations destroys planarity, any minor of a planar graph is planar. Hence a graph which contains K 5 or K 3,3 as a minor cannot be planar. Kuratowski’s Theorem says that the converse is also true. We first consider 3-connected graphs and start with the following lemma (which is the heart of Tutte’s so-called wheel theorem): Lemma 2.36. (Tutte [1961], Thomassen [1980]) Let G be a 3-connected graph with at least five vertices. Then there exists an edge e such that G/e is also 3-connected. Proof: Suppose there is no such edge. Then for each edge e = {v, w} there exists a vertex x such that G − {v, w, x} is disconnected, i.e. has a connected component C with |V (C)| < |V (G)| − 3. Choose e, x and C such that |V (C)| is minimum. x has a neighbour y in C, because otherwise C is a connected component of G − {v, w} (but G is 3-connected). By our assumption, G/{x, y} is not 3connected, i.e. there exists a vertex z such that G − {x, y, z} is disconnected. Since {v, w} ∈ E(G), there exists a connected component D of G − {x, y, z} which contains neither v nor w. But D contains a neighbour d of y, since otherwise D is a connected component of G − {x, z} (again contradicting the fact that G is 3-connected). So d ∈ V (D) ∩ V (C), and thus D is a subgraph of C. Since y ∈ V (C) \ V (D), we have a contradiction to the minimality of |V (C)|. Theorem 2.37. (Kuratowski [1930], Wagner [1937]) A 3-connected graph is planar if and only if it contains neither K 5 nor K 3,3 as a minor. Proof: As the necessity is evident (see above), we prove the sufficiency. Since K 4 is obviously planar, we proceed by induction on the number of vertices: let G be a 3-connected graph with more than four vertices but no K 5 or K 3,3 minor. By Lemma 2.36, there exists an edge e = {v, w} such that G/e is 3-connected. Let 8 = ψ, (Je )e∈E(G) be a planar embedding of G/e, which exists by induction. Let x be the vertex in G/e which arises by contracting e. Consider (G/e) − x with the restriction of 8 as a planar embedding. Since (G/e) − x is 2-connected, every face is bounded by a circuit (Proposition 2.31). In particular, the face containing the point ψ(x) is bounded by a circuit C. Let y1 , . . . , yk ∈ V (C) be the neighbours of v that are distinct from w, numbered in cyclic order, and partition C into edge-disjoint paths Pi , i = 1, . . . , k, such that Pi is a yi -yi+1 -path (yk+1 := y1 ). Suppose there exists an index i ∈ {1, . . . , k} such that 0(w) ⊆ {v} ∪ V (Pi ). Then a planar embedding of G can be constructed easily by modifying 8. We shall prove that all other cases are impossible. First, if w has three neighbours among y1 , . . . , yk , we have a K 5 minor (Figure 2.6(a)).

38

2 Graphs (b)

(a)

(c) yi

z

yi

yi+1

v C

v

w

w

C

v

w

C yj

z

Fig. 2.6.

Next, if (w) = {v, yi , y j } for some i < j, then we must have i + 1 < j and (i, j) = (1, k) (otherwise yi and y j would both lie on Pi or Pj ); see Figure 2.6(b). Otherwise there is a neighbour z of w in V (Pi ) \ {yi , yi+1 } for some i and another neighbour z ∈ / V (Pi ) (Figure 2.6(c)). In both cases, there are four vertices y, z, y , z on C, in this cyclic order, with y, y ∈ (v) and z, z ∈ (w). This implies that we have a K 3,3 minor. The proof implies quite directly that every 3-connected simple planar graph has a planar embedding where each edge is embedded by a straight line and each face, except the outer face, is convex (Exercise 27(a)). The general case of Kuratowski’s Theorem can be reduced to the 3-connected case by gluing together planar embeddings of the maximal 3-connected subgraphs, or by the following lemma: Lemma 2.38. (Thomassen [1980]) Let G be a graph with at least ﬁve vertices which is not 3-connected and which contains neither K 5 nor K 3,3 as a minor. Then there exist two non-adjacent vertices v, w ∈ V (G) such that G + e, where e = {v, w} is a new edge, does not contain a K 5 or K 3,3 minor either. Proof: We use induction on |V (G)|. Let G be as above. If G is disconnected, we can simply add an edge e joining two different connected components. So henceforth we assume that G is connected. Since G is not 3-connected, there exists a set X = {x, y} of two vertices such that G − X is disconnected. (If G is not even 2-connected we may choose x to be an articulation vertex and y a neighbour of x.) Let C be a connected component of G − X , G 1 := G[V (C) ∪ X ] and G 2 := G − V (C). We ﬁrst prove the following: Claim: Let v, w ∈ V (G 1 ) be two vertices such that adding an edge e = {v, w} to G creates a K 3,3 or K 5 minor. Then at least one of G 1 + e + f and G 2 + f contains a K 5 or K 3,3 minor, where f is a new edge joining x and y. To prove this claim, let v, w ∈ V (G 1 ), e = {v, w} and suppose that there are disjoint connected vertex sets Z 1 , . . . , Z t of G + e such that after contracting each of them we have a K 5 (t = 5) or K 3,3 (t = 6) subgraph. Note that it is impossible that Z i ⊆ V (G 1 ) \ X and Z j ⊆ V (G 2 ) \ X for some i, j ∈ {1, . . . , t}: in this case the set of those Z k with Z k ∩ X = ∅ (there are at

2.5 Planarity

39

most two of these) separate Z i and Z j , contradicting the fact that both K 5 and K 3,3 are 3-connected. Hence there are two cases: If none of Z 1 , . . . , Z t is a subset of V (G 2 ) \ X , then G 1 + e + f also contains a K 5 or K 3,3 minor: just consider Z i ∩ V (G 1 ) (i = 1, . . . , t). Analogously, if none of Z 1 , . . . , Z t is a subset of V (G 1 ) \ X , then G 2 + f contains a K 5 or K 3,3 minor (consider Z i ∩ V (G 2 ) (i = 1, . . . , t)). The claim is proved. Now we ﬁrst consider the case when G contains an articulation vertex x, and y is a neighbour of x. We choose a second neighbour z of x such that y and z are in different connected components of G − x. W.l.o.g. say that z ∈ V (G 1 ). Suppose that the addition of e = {y, z} creates a K 5 or K 3,3 minor. By the claim, at least one of G 1 + e and G 2 contains a K 5 or K 3,3 minor (an edge {x, y} is already present). But then G 1 or G 2 , and thus G, contains a K 5 or K 3,3 minor, contradicting our assumption. Hence we may assume that G is 2-connected. Recall that x, y ∈ V (G) were chosen such that G − {x, y} is disconnected. If {x, y} ∈ / E(G) we simply add an edge f = {x, y}. If this creates a K 5 or K 3,3 minor, the claim implies that G 1 + f or G 2 + f contains such a minor. Since there is an x-y-path in each of G 1 , G 2 (otherwise we would have an articulation vertex of G), this implies that there is a K 5 or K 3,3 minor in G which is again a contradiction. Thus we can assume that f = {x, y} ∈ E(G). Suppose now that at least one of the graphs G i (i ∈ {1, 2}) is not planar. Then this G i has at least ﬁve vertices. Since it does not contain a K 5 or K 3,3 minor (this would also be a minor of G), we conclude from Theorem 2.37 that G i is not 3-connected. So we can apply the induction hypothesis to G i . By the claim, if adding an edge within G i does not introduce a K 5 or K 3,3 minor in G i , it cannot introduce such a minor in G either. So we may assume that both G 1 and G 2 are planar; let 1 and 2 be planar embeddings. Let Fi be a face of i with f on its boundary, and let z i be another vertex on the boundary of Fi , z i ∈ / {x, y} (i = 1, 2). We claim that adding an edge {z 1 , z 2 } (cf. Figure 2.7) does not introduce a K 5 or K 3,3 minor.

z1

z2

x

G1

f

G2

y Fig. 2.7.

40

2 Graphs

Suppose, on the contrary, that adding {z 1 , z 2 } and contracting some disjoint connected vertex sets Z 1 , . . . , Z t would create a K 5 (t = 5) or K 3,3 (t = 6) subgraph. First suppose that at most one of the sets Z i is a subset of V (G 1 ) \ {x, y}. Then the graph G 02 , arising from G 2 by adding one vertex w and edges from w to x, y and z 2 , also contains a K 5 or K 3,3 minor. (Here w corresponds to the contracted set Z i ⊆ V (G 1 ) \ {x, y}.) This is a contradiction since there is a planar embedding of G 02 : just supplement 82 by placing w within F2 . So we may assume that Z 1 , Z 2 ⊆ V (G 1 )\{x, y}. Analogously, we may assume / Z 1 and z 2 ∈ / Z 3 . Then we that Z 3 , Z 4 ⊆ V (G 2 ) \ {x, y}. W.l.o.g. we have z 1 ∈ cannot have a K 5 , because Z 1 and Z 3 are not adjacent. Moreover, the only possible common neighbours of Z 1 and Z 3 are Z 5 and Z 6 . Since in K 3,3 two vertices are either adjacent or have three common neighbours, a K 3,3 minor is also impossible. Theorem 2.37 and Lemma 2.38 yield Kuratowski’s Theorem: Theorem 2.39. (Kuratowski [1930], Wagner [1937]) An undirected graph is planar if and only if it contains neither K 5 nor K 3,3 as a minor. Indeed, Kuratowski proved a stronger version (Exercise 28). The proof can be turned into a polynomial-time algorithm quite easily (Exercise 27(b)). In fact, a linear-time algorithm exists: Theorem 2.40. (Hopcroft and Tarjan [1974]) There is a linear-time algorithm for finding a planar embedding of a given graph or deciding that it is not planar.

2.6 Planar Duality We shall now introduce an important duality concept. This is the only place in this book where we need loops. So in this section loops, i.e. edges whose endpoints coincide, are allowed. In a planar embedding loops are of course represented by polygons instead of polygonal arcs. Note that Euler’s formula (Theorem 2.32) also holds for graphs with loops: this follows from the observation that subdividing a loop e (i.e. replacing e = {v, v} by two parallel edges {v, w}, {w, v} where w is a new vertex) and adjusting the embedding (replacing the polygon Je by two polygonal arcs whose union is Je ) increases the number of edges and vertices each by one but does not change the number of faces. Definition 2.41. Let G be a directed or undirected graph, possibly with loops, and let 8 = (ψ, (Je )e∈E(G) ) be a planar embedding of G. We define the planar dual G ∗ whose vertices are the faces of 8 and whose edge set is {e∗ : e ∈ E(G)}, where e∗ connects the faces that are adjacent to Je (if Je is adjacent to only one face, then e∗ is a loop). In the directed case, say for e = (v, w), we orient e∗ = (F1 , F2 ) in such a way that F1 is the face “to the right” when traversing Je from ψ(v) to ψ(w).

2.6 Planar Duality

41

G ∗ is again planar. In fact, there obviously exists a planar embedding of G ∗ such that ψ ∗ (F) ∈ F for all faces F of and, for each e ∈ E(G), |Je∗ ∩ Je | = 1 and ⎞ ⎛ Je∗ ∩ ⎝{ψ(v) : v ∈ V (G)} ∪ Jf ⎠ = ∅.

ψ ∗ , (Je∗ )e∗ ∈E(G ∗ )

f ∈E(G)\{e}

Such an embedding is called a standard embedding of G ∗ . (a)

(b)

Fig. 2.8.

The planar dual of a graph really depends on the embedding: consider the two embeddings of the same graph shown in Figure 2.8. The resulting planar duals are not isomorphic, since the second one has a vertex of degree four (corresponding to the outer face) while the ﬁrst one is 3-regular. Proposition 2.42. Let G be an undirected connected planar graph with a ﬁxed embedding. Let G ∗ be its planar dual with a standard embedding. Then (G ∗ )∗ = G. Proof: Let ψ, (Je )e∈E(G) be a ﬁxed embedding of G and ψ ∗ , (Je∗ )e∗ ∈E(G ∗ ) a standard embedding of G ∗ . Let F be a face of G ∗ . The boundary of F contains Je∗ for at least one edge e∗ , so F must contain ψ(v) for one endpoint v of e. So every face of G ∗ contains at least one vertex of G. By applying Euler’s formula (Theorem 2.32) to G ∗ and to G, we get that the number of faces of G ∗ is |E(G ∗ )| − |V (G ∗ )| + 2 = |E(G)| − (|E(G)| − |V (G)| + 2) + 2 = |V (G)|. Hence each face of G ∗ contains exactly one vertex of G. From this we conclude that the planar dual of G ∗ is isomorphic to G. The requirement that G is connected is essential here: note that G ∗ is always connected, even if G is disconnected. Theorem 2.43. Let G be a connected planar undirected graph with arbitrary embedding. The edge set of any circuit in G corresponds to a minimal cut in G ∗ , and any minimal cut in G corresponds to the edge set of a circuit in G ∗ .

42

2 Graphs

Proof: Let 8 = (ψ, (Je )e∈E(G) ) beS a fixed planar embedding of G. Let C be a circuit in G. By Theorem 2.30, R2 \ e∈E(C) Je splits into exactly two connected regions. Let A and B be the set of faces of 8 in the inner and outer region, . respectively. We have V (G ∗ ) = A ∪ B and E G ∗ (A, B) = {e∗ : e ∈ E(C)}. Since A and B form connected sets in G ∗ , this is indeed a minimal cut. Conversely, let δG (A) be a minimal cut in G. Let 8∗ = (ψ ∗ , (Je )e∈E(G ∗ ) ) be a standard embedding of G ∗ . Let a ∈ A and b ∈ V (G) \ A. Observe that there is no polygonal arc in [ R := R2 \ {ψ ∗ (v) : v ∈ V (G ∗ )} ∪ Je∗ e∈δG (A)

which connects ψ(a) and ψ(b): the sequence of faces of G ∗ passed by such a polygonal arc would define an edge progression from a to b in G not using any edge of δG (A). So R consists of at least two connected regions. Then, obviously, the boundary of each region must contain a circuit. Hence F := {e∗ : e ∈ δG (A)} contains the edge set of a circuit C in G ∗ . We have {e∗ : e ∈ E(C)} ⊆ {e∗ : e ∈ F} = δG (A), and, by the first part, {e∗ : e ∈ E(C)} is a minimal cut in (G ∗ )∗ = G (cf. Proposition 2.42). We conclude that {e∗ : e ∈ E(C)} = δG (A). In particular, e∗ is a loop if and only if e is a bridge, and vice versa. For digraphs the above proof yields: Corollary 2.44. Let G be a connected planar digraph with some fixed planar embedding. The edge set of any circuit in G corresponds to a minimal directed cut in G ∗ , and vice versa. Another interesting consequence of Theorem 2.43 is: Corollary 2.45. Let G be a connected undirected graph with arbitrary planar embedding. Then G is bipartite if and only if G ∗ is Eulerian, and G is Eulerian if and only if G ∗ is bipartite. Proof: Observe that a connected graph is Eulerian if and only if every minimal cut has even cardinality. By Theorem 2.43, G is bipartite if G ∗ is Eulerian, and G is Eulerian if G ∗ is bipartite. By Proposition 2.42, the converse is also true. An abstract dual of G is a graph G 0 for which there is a bijection χ : E(G) → E(G 0 ) such that F is the edge set of a circuit iff χ(F) is a minimal cut in G 0 and vice versa. Theorem 2.43 shows that any planar dual is also an abstract dual. The converse is not true. However, Whitney [1933] proved that a graph has an abstract dual if and only if it is planar (Exercise 34). We shall return to this duality relation when dealing with matroids in Section 13.3.

Exercises

43

Exercises 1. Let G be a simple undirected graph on n vertices which is isomorphic to its complement. Show that n mod 4 ∈ {0, 1}. 2. Prove that every simple undirected graph G with |δ(v)| ≥ 21 |V (G)| for all v ∈ V (G) is Hamiltonian. Hint: Consider a longest path in G and the neighbours of its endpoints. (Dirac [1952]) 3. Prove that any simple undirected graph G with |E(G)| > |V (G)|−1 is con2 nected. 4. Let G be a simple undirected graph. Show that G or its complement is connected. 5. Prove that every simple undirected graph with more than one vertex contains two vertices that have the same degree. Prove that every tree (except a single vertex) contains at least two leaves. 6. Let G be a connected undirected graph, and let (V (G), F) be a forest in G. Prove that there is a spanning tree (V (G), T ) with F ⊆ T ⊆ E(G). 7. Let (V, F1 ) and (V, F2 ) be two forests with |F1 | < |F2 |. Prove that there exists an edge e ∈ F2 \ F1 such that (V, F1 ∪ {e}) is a forest. 8. Prove that any cut in an undirected graph is the disjoint union of minimal cuts. 9. Let G be an undirected graph, C a circuit and D a cut. Show that |E(C) ∩ D| is even. 10. Show that any undirected graph has a cut containing at least half of the edges. 11. Let (U, F) be a cross-free set system with |U | ≥ 2. Prove that F contains at most 4|U | − 4 distinct elements. 12. Let G be a connected undirected graph. Show that there exists an orientation G 0 of G and a spanning arborescence T of G 0 such that the set of fundamental circuits with respect to T is precisely the set of directed circuits in G 0 . Hint: Consider a DFS-tree. (Camion [1968], Crestin [1969]) 13. Describe a linear-time algorithm for the following problem: Given an adjacency list of a graph G, compute an adjacency list of the maximal simple subgraph of G. Do not assume that parallel edges appear consecutively in the input. 14. Given a graph G (directed or undirected), show that there is a linear-time algorithm to find a circuit or decide that none exists. 15. Let G be a connected undirected graph, s ∈ V (G) and T a DFS-tree resulting from running DFS on (G, s). s is called the root of T . x is a predecessor of y in T if x lies on the (unique) s-y-path in T . x is a direct predecessor of y if the edge {x, y} lies on the s-y-path in T . y is a (direct) successor of x if x is a (direct) predecessor of y. Note that with this definition each vertex is a successor (and a predecessor) of itself. Every vertex except s has exactly one direct predecessor. Prove: (a) For any edge {v, w} ∈ E(G), v is a predecessor or a successor of w in T . (b) A vertex v is an articulation vertex of G if and only if • either v = s and |δT (v)| > 1

44

2 Graphs

or v 6= s and there is a direct successor w of v such that no edge in G connects a proper predecessor of v (that is, excluding v) with a successor of w. ∗ 16. Use Exercise 15 to design a linear-time algorithm which finds the blocks of an undirected graph. It will be useful to compute numbers •

α(x) := min{ f (w) : w = x or {w, y} ∈ E(G)\T for some successor y of x}

17.

18.

19. 20. 21. ∗ 22.

recursively during the DFS. Here (R, T ) is the DFS-tree (with root s), and the f -values represent the order in which the vertices are added to R (see the G RAPH S CANNING A LGORITHM). If for some vertex x ∈ R \ {s} we have α(x) ≥ f (w), where w is the direct predecessor of x, then w must be either the root or an articulation vertex. Prove: (a) An undirected graph is 2-edge-connected if and only if it has at least two vertices and an ear-decomposition. (b) A digraph is strongly connected if and only if it has an ear-decomposition. (c) The edges of an undirected graph G with at least two vertices can be oriented such that the resulting digraph is strongly connected if and only if G is 2-edge-connected. (Robbins [1939]) A tournament is a digraph such that the underlying undirected graph is a (simple) complete graph. Prove that every tournament contains a Hamiltonian path (R´edei [1934]). Prove that every strongly connected tournament is Hamiltonian (Camion [1959]). Prove that if a connected undirected simple graph is Eulerian then its line graph is Hamiltonian. What about the converse? Prove that any connected bipartite graph has a unique bipartition. Prove that any non-bipartite undirected graph contains an odd circuit as an induced subgraph. Prove that a strongly connected digraph whose underlying undirected graph is non-bipartite contains a (directed) circuit of odd length. Let G be an undirected graph. A tree-decomposition of G is a pair (T, ϕ), where T is a tree and ϕ : V (T ) → 2V (G) satisfies the following conditions: • for each e ∈ E(G) there exists a t ∈ V (T ) with e ⊆ ϕ(t); • for each v ∈ V (G) the set {t ∈ V (T ) : v ∈ ϕ(t)} is connected in T . We say that the width of (T, ϕ) is maxt∈V (T ) |ϕ(t)| − 1. The tree-width of a graph G is the minimum width of a tree-decomposition of G. This notion is due to Robertson and Seymour [1986]. Show that the simple graphs of tree-width at most 1 are the forests. Moreover, prove that the following statements are equivalent for an undirected graph G: (a) G has tree-width at most 2; (b) G does not contain K 4 as a minor; (c) G can be obtained from an empty graph by successively adding bridges and doubling and subdividing edges. (Doubling an edge e = {v, w} ∈ E(G) means adding another edge with endpoints v and w; subdividing an edge

Exercises

23.

24. 25. 26.

27.

∗ 28.

29.

30.

45

e = {v, w} ∈ E(G) means adding a vertex x and replacing e by two edges {v, x}, {x, w}.) Note: Because of the construction in (c) such graphs are called series-parallel. Show that if a graph G has a planar embedding where the edges are embedded by arbitrary Jordan curves, then it also has a planar embedding with polygonal arcs only. Let G be a 2-connected graph with a planar embedding. Show that the set of circuits bounding the finite faces constitutes a cycle basis of G. Can you generalize Euler’s formula (Theorem 2.32) to disconnected graphs? Show that there are exactly five Platonic graphs (corresponding to the Platonic solids; cf. Exercise 11 of Chapter 4), i.e. 3-connected planar regular graphs whose faces are all bounded by the same number of edges. Hint: Use Euler’s formula (Theorem 2.32). Deduce from the proof of Kuratowski’s Theorem 2.39: (a) Every 3-connected simple planar graph has a planar embedding where each edge is embedded by a straight line and each face, except the outer face, is convex. (b) There is a polynomial-time algorithm for checking whether a given graph is planar. Given a graph G and an edge e = {v, w} ∈ E(G), we say that H results from . G by subdividing e if V (H ) = V (G) ∪ {x} and E(H ) = (E(G) \ {e}) ∪ {{v, x}, {x, w}}. A graph resulting from G by successively subdividing edges is called a subdivision of G. (a) Trivially, if H contains a subdivision of G then G is a minor of H . Show that the converse is not true. (b) Prove that a graph containing a K 3,3 or K 5 minor also contains a subdivision of K 3,3 or K 5 . Hint: Consider what happens when contracting one edge. (c) Conclude that a graph is planar if and only if no subgraph is a subdivision of K 3,3 or K 5 . (Kuratowski [1930]) Prove that each of the following statements implies the other: (a) For every infinite sequence of graphs G 1 , G 2 , . . . there are two indices i < j such that G i is a minor of G j . (b) Let G be a class of graphs such that for each G ∈ G and each minor H of G we have H ∈ G (i.e. membership in G is a hereditary graph property). Then there exists a finite set X of graphs such that G consists of all graphs that do not contain any element of X as a minor. Note: The statements have been proved by Robertson and Seymour [2004]; they are a main result of their series of papers on graph minors. Theorem 2.39 and Exercise 22 give examples of forbidden minor characterizations as in (b). Let G be a planar graph with an embedding 8, and let C be a circuit of G bounding some face of 8. Prove that then there is an embedding 80 of G such that C bounds the outer face.

46

2 Graphs

31. (a) Let G be disconnected with an arbitrary planar embedding, and let G ∗ be the planar dual with a standard embedding. Prove that (G ∗ )∗ arises from G by successively applying the following operation, until the graph is connected: Choose two vertices x and y which belong to different connected components and which are adjacent to the same face; contract {x, y}. (b) Generalize Corollary 2.45 to arbitrary planar graphs. Hint: Use (a) and Theorem 2.26. 32. Let G be a connected digraph with a fixed planar embedding, and let G ∗ be the planar dual with a standard embedding. How are G and (G ∗ )∗ related? 33. Prove that if a planar digraph is acyclic (strongly connected), then its planar dual is strongly connected (acyclic). What about the converse? 34. (a) Show that if G has an abstract dual and H is a minor of G then H also has an abstract dual. ∗ (b) Show that neither K 5 nor K 3,3 has an abstract dual. (c) Conclude that a graph is planar if and only if it has an abstract dual. (Whitney [1933])

References General Literature: Berge, C. [1985]: Graphs. Second Edition. Elsevier, Amsterdam 1985 Bollob´as, B. [1998]: Modern Graph Theory. Springer, New York 1998 Bondy, J.A. [1995]: Basic graph theory: paths and circuits. In: Handbook of Combinatorics; Vol. 1 (R.L. Graham, M. Gr¨otschel, L. Lov´asz, eds.), Elsevier, Amsterdam 1995 Bondy, J.A., and Murty, U.S.R. [2008]: Graph Theory. Springer, New York 2008 Diestel, R. [2005]: Graph Theory. Third Edition. Springer, New York 2005 Wilson, R.J. [1996]: Introduction to Graph Theory. Fourth Edition. Addison-Wesley, Reading 1996

Cited References: Aoshima, K., and Iri, M. [1977]: Comments on F. Hadlock’s paper: finding a maximum cut of a Planar graph in polynomial time. SIAM Journal on Computing 6 (1977), 86–87 Camion, P. [1959]: Chemins et circuits hamiltoniens des graphes complets. Comptes Rendus Hebdomadaires des S´eances de l’Acad´emie des Sciences (Paris) 249 (1959), 2151–2152 Camion, P. [1968]: Modulaires unimodulaires. Journal of Combinatorial Theory A 4 (1968), 301–362 Dirac, G.A. [1952]: Some theorems on abstract graphs. Proceedings of the London Mathematical Society 2 (1952), 69–81 Edmonds, J., and Giles, R. [1977]: A min-max relation for submodular functions on graphs. In: Studies in Integer Programming; Annals of Discrete Mathematics 1 (P.L. Hammer, E.L. Johnson, B.H. Korte, G.L. Nemhauser, eds.), North-Holland, Amsterdam 1977, pp. 185–204 Euler, L. [1736]: Solutio Problematis ad Geometriam Situs Pertinentis. Commentarii Academiae Petropolitanae 8 (1736), 128–140

References

47

Euler, L. [1758]: Demonstratio nonnullarum insignium proprietatum quibus solida hedris planis inclusa sunt praedita. Novi Commentarii Academiae Petropolitanae 4 (1758), 140–160 ¨ Hierholzer, C. [1873]: Uber die M¨oglichkeit, einen Linienzug ohne Wiederholung und ohne Unterbrechung zu umfahren. Mathematische Annalen 6 (1873), 30–32 Hopcroft, J.E., and Tarjan, R.E. [1974]: Efficient planarity testing. Journal of the ACM 21 (1974), 549–568 Kahn, A.B. [1962]: Topological sorting of large networks. Communications of the ACM 5 (1962), 558–562 Knuth, D.E. [1968]: The Art of Computer Programming; Vol. 1. Fundamental Algorithms. Addison-Wesley, Reading 1968 (third edition: 1997) ¨ K¨onig, D. [1916]: Uber Graphen und Ihre Anwendung auf Determinantentheorie und Mengenlehre. Mathematische Annalen 77 (1916), 453–465 K¨onig, D. [1936]: Theorie der endlichen und unendlichen Graphen. Chelsea Publishing Co., Leipzig 1936, reprint New York 1950 Kuratowski, K. [1930]: Sur le probl`eme des courbes gauches en topologie. Fundamenta Mathematicae 15 (1930), 271–283 ´ ements de G´eom´etrie. Firmin Didot, Paris 1794 Legendre, A.M. [1794]: El´ Minty, G.J. [1960]: Monotone networks. Proceedings of the Royal Society of London A 257 (1960), 194–212 Moore, E.F. [1959]: The shortest path through a maze. Proceedings of the International Symposium on the Theory of Switching; Part II. Harvard University Press 1959, pp. 285–292 R´edei, L. [1934]: Ein kombinatorischer Satz. Acta Litt. Szeged 7 (1934), 39–43 Robbins, H.E. [1939]: A theorem on graphs with an application to a problem of traffic control. American Mathematical Monthly 46 (1939), 281–283 Robertson, N., and Seymour, P.D. [1986]: Graph minors II: algorithmic aspects of tree-width. Journal of Algorithms 7 (1986), 309–322 Robertson, N., and Seymour, P.D. [2004]: Graph minors XX: Wagner’s conjecture. Journal of Combinatorial Theory B 92 (2004), 325–357 Tarjan, R.E. [1972]: Depth first search and linear graph algorithms. SIAM Journal on Computing 1 (1972), 146–160 Thomassen, C. [1980]: Planarity and duality of finite and infinite graphs. Journal of Combinatorial Theory B 29 (1980), 244–271 Thomassen, C. [1981]: Kuratowski’s theorem. Journal of Graph Theory 5 (1981), 225–241 Tutte, W.T. [1961]: A theory of 3-connected graphs. Konink. Nederl. Akad. Wetensch. Proc. A 64 (1961), 441–455 ¨ Wagner, K. [1937]: Uber eine Eigenschaft der ebenen Komplexe. Mathematische Annalen 114 (1937), 570–590 Whitney, H. [1932]: Non-separable and planar graphs. Transactions of the American Mathematical Society 34 (1932), 339–362 Whitney, H. [1933]: Planar graphs. Fundamenta Mathematicae 21 (1933), 73–84

3 Linear Programming

In this chapter we review the most important facts about Linear Programming. Although this chapter is self-contained, it cannot be considered to be a comprehensive treatment of the field. The reader unfamiliar with Linear Programming is referred to the textbooks mentioned at the end of this chapter. The general problem reads as follows:

L INEAR P ROGRAMMING Instance:

A matrix A ∈ Rm×n and column vectors b ∈ Rm , c ∈ Rn .

Task:

Find a column vector x ∈ Rn such that Ax ≤ b and c> x is maximum, decide that {x ∈ Rn : Ax ≤ b} is empty, or decide that for all α ∈ R there is an x ∈ Rn with Ax ≤ b and c> x > α.

Here c> x denotes the scalar product of the vectors. The notion x ≤ y for vectors x and y (of equal size) means that the inequality holds in each component. If no sizes are specified, the matrices and vectors are always assumed to be compatible in size. We often omit indicating the transposition of column vectors and write e.g. cx for the scalar product. By 0 we denote the number zero as well as all-zero vectors and all-zero matrices (the order will always be clear from the context). A linear program (LP) is an instance of the above problem. We often write a linear program as max{cx : Ax ≤ b}. A feasible solution of an LP max{cx : Ax ≤ b} is a vector x with Ax ≤ b. A feasible solution attaining the maximum is called an optimum solution. As the problem formulation indicates, there are two possibilities when an LP has no solution: The problem can be infeasible (i.e. P := {x ∈ Rn : Ax ≤ b} = ∅) or unbounded (i.e. for all α ∈ R there is an x ∈ P with cx > α). If an LP is neither infeasible nor unbounded it has an optimum solution: Proposition 3.1. Let P = {x ∈ Rn : Ax ≤ b} 6= ∅ and c ∈ Rn with δ := sup{c> x : x ∈ P} < ∞. Then there exists a vector z ∈ P with c> z = δ. Proof: Let U be a matrix whose columns are an orthonormal basis of the kernel of A, i.e. U > U = I , AU = 0, and rank(A0 ) = n where A0 := UA> . Let b0 := b0 . We show that for every y ∈ P there exists a subsystem A00 x ≤ b00 of A0 x ≤ b0 such that A00 is nonsingular, y 0 := (A00 )−1 b00 ∈ P, and c> y 0 ≥ c> y. As there

50

3 Linear Programming

are only finitely many such subsystems, one of these y 0 attains the maximum (c> y 0 = δ), and the assertion follows. So let y ∈ P, and denote by k(y) the rank of A00 for the maximal subsystem 00 A x ≤ b00 of A0 x ≤ b0 with A00 y = b00 . Suppose that k(y) < n. We show how to find a y 0 ∈ P with c> y 0 ≥ c> y and k(y 0 ) > k(y). After at most n steps we have a vector y 0 with k(y 0 ) = n as required. If U > y 6= 0, we set y 0 := y − UU > y. Since y + λUU > c ∈ P for all λ ∈ R we have sup{c> (y + λUU > c) : λ ∈ R} ≤ δ < ∞ and hence c> U = 0 and c> y 0 = c> y. Moreover, Ay 0 = Ay − AUU > y = Ay and U > y 0 = U > y − U > UU > y = 0. Now suppose that U > y = 0. Let v 6= 0 with nA00 v = 0. Denote o by βi −ai y ai x ≤ βi the i-th row of Ax ≤ b. Let µ := min ai v : ai v > 0 and o n iy : a v < 0 , where min ∅ = ∞ and max ∅ = −∞. We have κ := max βi a−a i iv κ ≤ 0 ≤ µ, and at least one of κ and µ is finite. For λ ∈ R with κ ≤ λ ≤ µ we have A00 (y + λv) = A00 y + λA00 v = A00 y = b00 and A(y + λv) = Ay + λAv ≤ b, i.e. y + λv ∈ P. Thus, as sup{c> x : x ∈ P} < ∞, we have µ < ∞ if c> v > 0 and κ > −∞ if c> v < 0. Moreover, if c> v ≥ 0 and µ < ∞, we have ai (y + µv) = βi for some i. Analogously, if c> v ≤ 0 and κ > −∞, we have ai (y + κv) = βi for some i. Thus in each case we have found a vector y 0 ∈ P with c> y 0 ≥ c> y and 0 k(y ) ≥ k(y) + 1. This justifies the notation max{c> x : Ax ≤ b} instead of sup{c> x : Ax ≤ b}. Many combinatorial optimization problems can be formulated as LPs. To do this, we encode the feasible solutions as vectors in Rn for some n. In Section 3.5 we show that one can optimize a linear objective function over a finite set S of vectors by solving a linear program. Although the feasible set of this LP contains not only the vectors in S but also all their convex combinations, one can show that among the optimum solutions there is always an element of S. In Section 3.1 we compile some terminology and basic facts about polyhedra, the sets P = {x ∈ Rn : Ax ≤ b} of feasible solutions of LPs. In Sections 3.2 and 3.3 we present the S IMPLEX A LGORITHM, which we also use to derive the Duality Theorem and related results (Section 3.4). LP duality is a most important concept which explicitly or implicitly appears in almost all areas of combinatorial optimization; we shall often refer to the results in Sections 3.4 and 3.5.

3.1 Polyhedra Linear Programming deals with maximizing or minimizing a linear objective function of finitely many variables subject to finitely many linear inequalities. So the set of feasible solutions is the intersection of finitely many halfspaces. Such a set is called a polyhedron:

3.1 Polyhedra

51

Definition 3.2. A polyhedron in Rn is a set of type P = {x ∈ Rn : Ax ≤ b} for some matrix A ∈ Rm×n and some vector b ∈ Rm . If A and b are rational, then P is a rational polyhedron. A bounded polyhedron is also called a polytope. We denote by rank(A) the rank of a matrix A. The dimension dim X of a nonempty set X ⊆ Rn is defined to be n − max{rank(A) : A is an n × n-matrix with Ax = Ay for all x, y ∈ X }. A polyhedron P ⊆ Rn is called full-dimensional if dim P = n. Equivalently, a polyhedron is full-dimensional if and only if there is a point in its interior. For most of this chapter it makes no difference whether we are in the rational or real space. We need the following standard terminology: Definition 3.3. Let P := {x : Ax ≤ b} be a nonempty polyhedron. If c is a nonzero vector for which δ := max{cx : x ∈ P} is finite, then {x : cx = δ} is called a supporting hyperplane of P. A face of P is P itself or the intersection of P with a supporting hyperplane of P. A point x for which {x} is a face is called a vertex of P, and also a basic solution of the system Ax ≤ b. Proposition 3.4. Let P = {x : Ax ≤ b} be a polyhedron and F ⊆ P. Then the following statements are equivalent: (a) F is a face of P. (b) There exists a vector c such that δ := max{cx : x ∈ P} is finite and F = {x ∈ P : cx = δ}. (c) F = {x ∈ P : A0 x = b0 } 6= ∅ for some subsystem A0 x ≤ b0 of Ax ≤ b. Proof: (a) and (b) are obviously equivalent. (c)⇒(b): If F = {x ∈ P : A0 x = b0 } is nonempty, let c be the sum of the rows of A0 , and let δ be the sum of the components of b0 . Then obviously cx ≤ δ for all x ∈ P and F = {x ∈ P : cx = δ}. (b)⇒(c): Assume that c is a vector, δ := max{cx : x ∈ P} is finite and F = {x ∈ P : cx = δ}. Let A0 x ≤ b0 be the maximal subsystem of Ax ≤ b such that A0 x = b0 for all x ∈ F. Let A00 x ≤ b00 be the rest of the system Ax ≤ b. We first observe that for each inequality ai00 x ≤ βi00 of A00 x ≤ b00 (i = 1, . . . , k) Pk there is a point xi ∈ F such that ai00 xi < βi00 . Let x ∗ := k1 i=1 xi be the center of gravity of these points (if k = 0, we can choose an arbitrary x ∗ ∈ F); we have x ∗ ∈ F and ai00 x ∗ < βi00 for all i. We have to prove that A0 y = b0 cannot hold for any y ∈ P \ F. So let y ∈ P \ F. We have cy < δ. Now consider z := x ∗ + (x ∗ − y) for some small > 0; in β 00 −ai00 x ∗ particular let be smaller than a 00i (x ∗ −y) for all i ∈ {1, . . . , k} with ai00 x ∗ > ai00 y. i We have cz > δ and thus z ∈ / P. So there is an inequality ax ≤ β of Ax ≤ b such that az > β. Thus ax ∗ > ay. The inequality ax ≤ β cannot belong to A00 x ≤ b00 , β−ax ∗ ∗ since otherwise we have az = ax ∗ + a(x ∗ − y) < ax ∗ + a(x ∗ −y) a(x − y) = β

52

3 Linear Programming

(by the choice of ). Hence the inequality ax ≤ β belongs to A0 x ≤ b0 . Since ay = a(x ∗ + 1 (x ∗ − z)) < β, this completes the proof. As a trivial but important corollary we remark: Corollary 3.5. If max{cx : x ∈ P} is bounded for a nonempty polyhedron P and a vector c, then the set of points where the maximum is attained is a face of P. The relation “is a face of ” is transitive: Corollary 3.6. Let P be a polyhedron and F a face of P. Then F is again a polyhedron. Furthermore, a set F 0 ⊆ F is a face of P if and only if it is a face of F. The maximal faces distinct from P are particularly important: Definition 3.7. Let P be a polyhedron. A facet of P is a maximal face distinct from P. An inequality cx ≤ δ is facet-defining for P if cx ≤ δ for all x ∈ P and {x ∈ P : cx = δ} is a facet of P. Proposition 3.8. Let P ⊆ {x ∈ Rn : Ax = b} be a nonempty polyhedron of dimension n − rank(A). Let A0 x ≤ b0 be a minimal inequality system such that P = {x : Ax = b, A0 x ≤ b0 }. Then each inequality of A0 x ≤ b0 is facet-defining for P, and each facet of P is defined by an inequality of A0 x ≤ b0 . Proof: If P = {x ∈ Rn : Ax = b}, then there are no facets and the statement is trivial. So let A0 x ≤ b0 be a minimal inequality system with P = {x : Ax = b, A0 x ≤ b0 }, let a 0 x ≤ β 0 be one of its inequalities and A00 x ≤ b00 be the rest of the system A0 x ≤ b0 . Let y be a vector with Ay = b, A00 y ≤ b00 and a 0 y > β 0 (such a vector y exists as the inequality a 0 x ≤ β 0 is not redundant). Let x ∈ P such that a 0 x < β 0 (such a vector must exist because dim P = n − rank(A)). 0 −a 0 x β 0 −a 0 x 0 0 Consider z := x + aβ0 y−a 0 x (y−x). We have a z = β and, since 0 < a 0 y−a 0 x < 1, z ∈ P. Therefore F := {x ∈ P : a 0 x = β 0 } 6= 0 and F 6= P (as x ∈ P \ F). Thus F is a facet of P. By Proposition 3.4 each facet is defined by an inequality of A0 x ≤ b0 . The other important class of faces (beside facets) are minimal faces (i.e. faces not containing any other face). Here we have: Proposition 3.9. (Hoffman and Kruskal [1956]) Let P = {x : Ax ≤ b} be a polyhedron. A nonempty subset F ⊆ P is a minimal face of P if and only if F = {x : A0 x = b0 } for some subsystem A0 x ≤ b0 of Ax ≤ b. Proof: If F is a minimal face of P, by Proposition 3.4 there is a subsystem A0 x ≤ b0 of Ax ≤ b such that F = {x ∈ P : A0 x = b0 }. We choose A0 x ≤ b0 maximal. Let A00 x ≤ b00 be a minimal subsystem of Ax ≤ b such that F = {x : A0 x = b0 , A00 x ≤ b00 }. We claim that A00 x ≤ b00 does not contain any inequality.

3.1 Polyhedra

53

Suppose, on the contrary, that a 00 x ≤ β 00 is an inequality of A00 x ≤ b00 . Since it is not redundant for the description of F, Proposition 3.8 implies that F 0 := {x : A0 x = b0 , A00 x ≤ b00 , a 00 x = β 00 } is a facet of F. By Corollary 3.6 F 0 is also a face of P, contradicting the assumption that F is a minimal face of P. Now let ∅ 6= F = {x : A0 x = b0 } ⊆ P for some subsystem A0 x ≤ b0 of Ax ≤ b. Obviously F has no faces except itself. By Proposition 3.4, F is a face of P. It follows by Corollary 3.6 that F is a minimal face of P. Corollary 3.5 and Proposition 3.9 imply that L INEAR P ROGRAMMING can be solved in finite time by solving the linear equation system A0 x = b0 for each subsystem A0 x ≤ b0 of Ax ≤ b. A more intelligent way is the S IMPLEX A LGORITHM which is described in the next section. Another consequence of Proposition 3.9 is: Corollary 3.10. Let P = {x ∈ Rn : Ax ≤ b} be a polyhedron. Then all minimal faces of P have dimension n−rank(A). The minimal faces of polytopes are vertices. This is why polyhedra {x ∈ Rn : Ax ≤ b} with rank(A) = n are called pointed: their minimal faces are points. Let us close this section with some remarks on polyhedral cones. Definition 3.11. A (convex) cone is a set C ⊆ Rn for which x, y ∈ C and λ, µ ≥ 0 implies λx + µy ∈ C. A cone C is said to be generated by x1 , . . . , xk if x1 ,P . . . , xk ∈ C and for any x ∈ C there are numbers λ1 , . . . , λk ≥ 0 with k λi xi . A cone is called finitely generated if some finite set of vectors x = i=1 generates it. A polyhedral cone is a polyhedron of type {x : Ax ≤ 0}. It is immediately clear that polyhedral cones are indeed cones. We shall now show that polyhedral cones are finitely generated. I always denotes an identity matrix. Lemma 3.12. (Minkowski [1896]) Let C = {x ∈ Rn : Ax ≤ 0} be a polyhedral cone. Then C is generated by a subset of the set of solutions to the systems M y = b0 , where M consists of n linearly independent rows of AI and b0 = ±e j for some unit vector e j . 0 Proof: Let A be an m × n-matrix. Consider the systems M y = b where M consists of n linearly independent rows of AI and b0 = ±e j for some unit vector e j . Let y1 , . . . , yt be those solutions of these equality systems that belong to C. We claim that C is generated by y1 , . . . , yt . First suppose C = {x : Ax = 0}, i.e. C is a linear subspace. Write C = {x : A0 x = 0} where A0 consists of a maximal set oflinearly independent rows of A. 0 0 Let I consist of some rows of I such that AI 0 is a nonsingular square matrix. Then C is generated by the solutions of

54

3 Linear Programming

! ! A0 0 , x= I0 b

for b = ±e j , j = 1, . . . , dim C.

For the general case we use induction on the dimension of C. If C is not a linear subspace, choose a row a of A and a submatrix A0 of A such that the rows 0 A of a are linearly independent and {x : A0 x = 0, ax ≤ 0} ⊆ C. By construction there is an index s ∈ {1, . . . , t} such that A0 ys = 0 and ays = −1. Now let o Let a1 , . . . , am be the rows of A and n an arbitrary z ∈ C be given. µ := min aaiiyzs : i = 1, . . . , m, ai ys < 0 . We have µ ≥ 0. Let k be an index where

the minimum is attained. Consider z 0 := z − µys . By the definition of µ we have a j z 0 = a j z − aakkyzs a j ys for j = 1, . . . , m, and hence z 0 ∈ C 0 := {x ∈ C : ak x = 0}. C 0 is a cone whose dimension is one less than that of C (because ak yP s < 0 and t λi yi ys ∈ C). By induction, C 0 is generated by a subset of y1 , . . . , yt , so z 0 = i=1 0 for some λ1 , . . . , λt ≥ 0. By setting λs := λs +µ (observe that µ ≥ 0) and λi0 := λi Pt (i 6= s), we obtain z = z 0 + µys = i=1 λi0 yi . Thus any polyhedral cone is finitely generated. We shall show the converse at the end of Section 3.4.

3.2 The Simplex Algorithm The oldest and best-known algorithm for L INEAR P ROGRAMMING is Dantzig’s [1951] simplex method. We first assume that the polyhedron has a vertex, and that some vertex is given as input. Later we shall show how general LPs can be solved with this method. For a set J of row indices we write A J for the submatrix of A consisting of the rows in J only, and b J for the subvector of b consisting of the components with indices in J . We abbreviate ai := A{i} and βi := b{i} .

S IMPLEX A LGORITHM Input: Output:

A matrix A ∈ Rm×n and column vectors b ∈ Rm , c ∈ Rn . A vertex x of P := {x ∈ Rn : Ax ≤ b}. A vertex x of P attaining max{cx : x ∈ P} or a vector w ∈ Rn with Aw ≤ 0 and cw > 0 (i.e. the LP is unbounded).

1

Choose a set of n row indices J such that A J is nonsingular and A J x = b J .

2

Compute c (A J )−1 and add zeros in order to obtain a vector y with c = y A such that all entries of y outside J are zero. If y ≥ 0 then stop. Return x and y. Choose the minimum index i with yi < 0. Let w be the column of −(A J )−1 with index i, so A J \{i} w = 0 and ai w = −1. If Aw ≤ 0 then stop. Return w.

3

3.2 The Simplex Algorithm

4

5

55

βj − aj x : j ∈ {1, . . . , m}, a j w > 0 , aj w and let j be the smallest row index attaining this minimum. Set J := (J \ {i}) ∪ { j} and x := x + λw. Go to . 2 Let λ := min

Step 1 relies on Proposition 3.9 and can be implemented with G AUSSIAN E LIMINATION (Section 4.3). The selection rules for i and j in 3 and 4 (often called pivot rule) are due to Bland [1977]. If one just chose an arbitrary i with yi < 0 and an arbitrary j attaining the minimum in 4 the algorithm would run into cyclic repetitions for some instances. Bland’s pivot rule is not the only one that avoids cycling; another one (the so-called lexicographic rule) was proved to avoid cycling already by Dantzig, Orden and Wolfe [1955]. Before proving the correctness of the S IMPLEX A LGORITHM, let us make the following observation (sometimes known as “weak duality”): Proposition 3.13. Let x and y be feasible solutions of the LPs and

(3.1)

min{yb : y > A = c> , y ≥ 0},

max{cx : Ax ≤ b}

(3.2)

respectively. Then cx ≤ yb. Proof: cx = (y A)x = y(Ax) ≤ yb.

Theorem 3.14. (Dantzig [1951], Dantzig, Orden and Wolfe [1955], Bland [1977]) The S IMPLEX A LGORITHM terminates after at most mn iterations. If it returns x and y in , 2 these vectors are optimum solutions of the LPs (3.1) and (3.2), respectively, with cx = yb. If the algorithm returns w in 3 then cw > 0 and the LP (3.1) is unbounded. Proof: We first prove that the following conditions hold at any stage of the algorithm: (a) (b) (c) (d) (e)

x ∈ P; AJ x = bJ ; A J is nonsingular; cw > 0; λ ≥ 0.

(a) and (b) hold initially. 2 and 3 guarantee cw = y Aw = −yi > 0. By , 4 x ∈ P implies λ ≥ 0. (c) follows from the fact that A J \{i} w = 0 and a j w > 0. It remains to show that 5 preserves (a) and (b). We show that if x ∈ P, then also x + λw ∈ P. For a row index k we have two cases: If ak w ≤ 0 then (using λ ≥ 0) ak (x + λw) ≤ ak x ≤ βk . Otherwise kx kx and hence ak (x + λw) ≤ ak x + ak w βka−a = βk . (Indeed, λ is chosen λ ≤ βka−a kw kw in 4 to be the largest number such that x + λw ∈ P.)

56

3 Linear Programming β j −a j x a j w , so β j −a j x aj w aj w = βj .

To show (b), note that after 4 we have A J \{i} w = 0 and λ =

A J \{i} (x + λw) = A J \{i} x = b J \{i} and a j (x + λw) = a j x + Therefore after , 5 A J x = b J holds again. So we indeed have (a)–(e) at any stage. If the algorithm returns x and y in

, 2 x and y are feasible solutions of (3.1) and (3.2), respectively. x is a vertex of P by (a), (b) and (c). Moreover, cx = y Ax = yb since the components of y are zero outside J . This proves the optimality of x and y by Proposition 3.13. If the algorithm stops in , 3 the LP (3.1) is indeed unbounded because in this case x + µw ∈ P for all µ ≥ 0, and cw > 0 by (d). We finally show that the algorithm terminates. Let J (k) and x (k) be the set J and the vector x in iteration k of theS IMPLEX A LGORITHM, respectively. If the algorithm did not terminate after mn iterations, there are iterations k < l with J (k) = J (l) . By (b) and (c), x (k) = x (l) . By (d) and (e), cx never decreases, and it strictly increases if λ > 0. Hence λ is zero in all the iterations k, k + 1, . . . , l − 1, and x (k) = x (k+1) = · · · = x (l) . Let h be the highest index leaving J in one of the iterations k, . . . , l − 1, say in iteration p. Index h must also have been added to J in some iteration q ∈ {k, . . . , l − 1}. Now let y 0 be the vector y at iteration p, and let w0 be the vector w at iteration q. We have y 0 Aw 0 = cw 0 > 0. So let r be an index for which yr0 ar w 0 > 0. Since yr0 6= 0, index r belongs to J ( p) . If r > h, index r would also belong to J (q) and J (q+1) , implying ar w 0 = 0. So r ≤ h. But by the choice of i in iteration p we have yr0 < 0 iff r = h, and by the choice of j in iteration q we have ar w 0 > 0 iff r = h (recall that λ = 0 and ar x (q) = ar x ( p) = βr as r ∈ J ( p) ). This is a contradiction. Klee and Minty [1972] and Avis and Chv´atal [1978] found examples where the S IMPLEX A LGORITHM (with Bland’s rule) needs 2n iterations on LPs with n variables and 2n constraints, proving that it is not a polynomial-time algorithm. It is not known whether there is a pivot rule that leads to a polynomial-time algorithm. However, Borgwardt [1982] showed that the average running time (for random instances in a certain natural probabilistic model) can be bounded by a polynomial. Spielman and Teng [2004] introduced a so-called smoothed analysis: for each input they consider the expected running time with respect to small random perturbations of the input. The maximum of all these expectations is polynomially bounded. Kelner and Spielman [2006] proposed a randomized polynomial-time algorithm for L INEAR P ROGRAMMING that is similar to the S IMPLEX A LGORITHM. The S IMPLEX A LGORITHM is also quite fast in practice if implemented skilfully; see Section 3.3. We now show how to solve general linear programs with the S IMPLEX A L GORITHM . More precisely, we show how to find an initial vertex. Since there are polyhedra that do not have vertices at all, we put a given LP into a different form first. Let max{cx : Ax ≤ b} be an LP. We substitute x by y − z and write it equivalently in the form

3.3 Implementation of the Simplex Algorithm

( max

y c −c z

! :

A −A

y z

!

57

) ≤ b, y, z ≥ 0

.

So w.l.o.g. we assume that our LP has the form max{cx : A0 x ≤ b0 , A00 x ≤ b00 , x ≥ 0}

(3.3)

with b0 ≥ 0 and b00 < 0. We first run the S IMPLEX A LGORITHM on the instance (3.4) min{(1lA00 )x + 1ly : A0 x ≤ b0 , A00 x + y ≥ b00 , x, y ≥ 0}, where 1l denotes a vector whose entries are all 1. Since xy = 0 defines a vertex, this is possible. The LP is obviously not unbounded since the minimum must be x 00 x is an optimum solution at least 1lb00 . For any feasible solution x of (3.3), b00 −A of (3.4) of value 1lb00 . Hence if the minimum of (3.4) is greater than 1lb00 , then (3.3) is infeasible. In the contrary case, let xy be an optimum vertex of (3.4) of value 1lb00 . We claim that x is a vertex of the polyhedron defined by (3.3). To see this, first observe that A00 x + y = b00 . Let n and m be the dimensions of x and y, respectively; then by Proposition 3.9 there is a set S of n + m inequalities of (3.4) satisfied with equality, such that the submatrix corresponding to these n + m inequalities is nonsingular. Let S 0 be the inequalities of A0 x ≤ b0 and of x ≥ 0 that belong to S. Let S 00 consist of those inequalities of A00 x ≤ b00 for which the corresponding inequalities of A00 x + y ≥ b00 and y ≥ 0 both belong to S. Obviously |S 0 ∪ S 00 | ≥ |S| − m = n, and the inequalities of S 0 ∪ S 00 are linearly independent and satisfied by x with equality. Hence x satisfies n linearly independent inequalities of (3.3) with equality; thus x is indeed a vertex. Therefore we can start the S IMPLEX A LGORITHM with (3.3) and x.

3.3 Implementation of the Simplex Algorithm The previous description of the S IMPLEX A LGORITHM is simple but not suitable for an efficient implementation. As we will see, it is not necessary to solve a linear equation system in each iteration. To motivate the main idea, we start with a proposition (which is actually not needed later): for LPs of the form max{cx : Ax = b, x ≥ 0}, vertices can be represented not only by subsets of rows but also by subsets of columns. For a matrix A and a set J of column indices we denote by A J the submatrix consisting of the columns in J only. Consequently, A IJ denotes the submatrix of A with rows in I and columns in J . Sometimes the order of the rows and columns is important: if J = ( j1 , . . . , jk ) is a vector of row (column) indices, we denote by A J (A J ) the matrix whose i-th row (column) is the ji -th row (column) of A (i = 1, . . . , k).

58

3 Linear Programming

Proposition 3.15. Let P := {x : Ax = b, x ≥ 0}, where A is a matrix and b is a vector. Then x is a vertex of P if and only if x ∈ P and the columns of A corresponding to positive entries of x are linearly independent. 0 0 0 Proof: Let A be an m × n-matrix. Let X := −I A I and b := b . Let N := {1, . . . , n} and M := {n +1, . . . , n +m}. For an index set J ⊆ N ∪ M with |J | = n N ∩ J¯ is nonsingular iff X J¯ let J¯ := (N ∪ M) \ J . Then X JN is nonsingular iff X M∩J M is nonsingular. If x is a vertex of P, then – by Proposition 3.9 – there exists a set J ⊆ N ∪ M such that |J | = n, X JN is nonsingular, and X JN x = b0J . Then the components of J¯ is nonsingular, and hence the x corresponding to N ∩ J are zero. Moreover, X M ¯ columns of A N ∩ J are linearly independent. Conversely, let x ∈ P, and let the set of columns of A corresponding to positive entries of x be linearly independent. By adding suitable unit column vectors to B with x = 0 for i ∈ N \ B. these columns we obtain a nonsingular submatrix X M i N N 0 Then X B¯ is nonsingular and X B¯ x = b B¯ . Hence, by Proposition 3.9, x is a vertex of P. Corollary 3.16. Let xy ∈ P := xy : Ax + y = b, x ≥ 0, y ≥ 0 . Then xy is a vertex of P if and only if the columns of (A I ) corresponding to positive components of xy are linearlyindependent. Moreover, x is a vertex of {x : Ax ≤ x b, x ≥ 0} if and only if b−Ax is a vertex of P. We will now analyze the behaviour of the S IMPLEX A LGORITHM when applied to an LP of the form max{cx : Ax ≤ b, x ≥ 0}. 0 0 Theorem 3.17. Let A ∈ Rm×n , b ∈ Rm , and c ∈ Rn . Let A0 := −I A , b := b and c¯ := (c> , 0). Let B ∈ {1, . . . , n + m}m such that (A I ) B is nonsingular. Let J ⊆ {1, . . . , n + m} be the set of the remaining n indices. Let Q B := ((A I ) B )−1 . Then: (a) A0J is nonsingular. (b) (b0 − A0 x) J = 0 and (b0 − A0 x) B = Q B b and c> x = c¯ B Q B b, where x := (A0J )−1 b0J . (c) Let y be the vector with y B = 0 and y > A0 = c> . Then y > = c¯ B Q B (A I ) − c. ¯ (d) Let i ∈ J . Let w be the vector with Ai0 w = −1 and A0J \{i} w = 0. Then A0B w = Q B (A I )i . (e) Define ! Q B (A I ) QBb TB := . c¯ B Q B (A I ) − c¯ c> x Given B and TB , we can compute B 0 and TB 0 in O(m(n + m)) time, where B 0 arises from B by replacing j by i, and i and j are given as in – 2 4 of the S IMPLEX A LGORITHM (applied to A0 , b0 , c, and index set J ).

3.3 Implementation of the Simplex Algorithm

59

TB is called the simplex tableau with respect to the basis B. N \J

Proof: (a): Let N := {1, . . . , n}. As (A I ) B is nonsingular, also (A0 ) J \N is nonsingular, and thus A0J is nonsingular. (b): The first statement follows directly from A0J x = b0J . Then b = Ax + ¯ 0 − A0 x) = I (b − Ax) = (A I )(b0 − A0 x) = (A I ) B (b0 − A0 x) B and c> x = c(b B 0 0 B c¯ (b − A x) B = c¯ Q B b. ¯ B = c¯ B Q B (A I ) B − c¯ B = 0 and (c): This follows from (c¯ B Q B (A I ) − c) B 0 B 0 > ¯ = c¯ Q B (A I )A − c (−I ) = c> . (c¯ Q B (A I ) − c)A (d): This follows from 0 = (A I )A0 w = (A I ) B (A0B w) + (A I ) J \{i} (A0J \{i} w) + (A I )i (Ai0 w) = (A I ) B (A0B w) − (A I )i . (e): By (c), y as in 2 of the S IMPLEX A LGORITHM is given by the last row of TB . If y ≥ 0, we stop (x and y are optimal). Otherwise i is the first index with yi < 0, found in O(n + m) time. If the i-th column of TB has no positive entry, we stop (the LP is unbounded, and w is given by (d)). Otherwise, by (b) and (d), we have that λ in 4 of the S IMPLEX A LGORITHM is given by (Q B b) j i : j ∈ {1, . . . , m}, (Q (A I ) ) > 0 , λ = min B j (Q B (A I )i ) j and among the indices attaining this minimum, j is the one for which the j-th component of B is minimum. So we can compute j in O(m) time by considering the i-th and the last column of TB . This yields B 0 . We can compute the updated tableau TB 0 as follows: Divide the j-th row by the entry in row j and column i. Then add a suitable multiple of the j-th row to all other rows, such that the i-th column has zeros only outside row j. Note that these row operations do not destroy the property that the tableau has the form ! Q(A I ) Qb v(A I ) − c¯ vb for some nonsingular matrix Q and some vector v, and in addition we have 0 0 ¯ B = 0. Since there is only one choice for Q and v, Q(A I ) B = I and (v(A I ) − c) 0 namely Q = Q B 0 and v = c¯ B Q B 0 , the updated tableau TB 0 is computed correctly by the above operations in O(m(n + m)) time. To start the S IMPLEX A LGORITHM we consider an LP of the form max{cx : A0 x ≤ b0 , A00 x ≤ b00 , x ≥ 0} 0

00

with A0 ∈ Rm ×n , A00 ∈ Rm ×n , b0 ≥ 0 and b00 < 0. We first run the S IMPLEX A LGORITHM on the instance min{(1lA00 )x + 1ly : A0 x ≤ b0 , A00 x + y ≥ b00 , x, y ≥ 0}, starting with the tableau

60

3 Linear Programming

A0 0 00 −A −I 1lA00 1l

I 0 b0 0 I −b00 , 0 0 0

(3.5)

corresponding to the basic solution x = 0, y = 0. Then we run the iterations of the S IMPLEX A LGORITHM as in Theorem 3.17(e). If the algorithm terminates with optimum value 1lb, we modify the final simplex tableau as follows. Multiply some rows by −1 such that none of the columns n + m 00 +m 0 +1, . . . , n+m 00 +m 0 +m 00 (the fourth section in (3.5)) is a unit vector, delete the fourth section of the tableau (i.e. columns n+m 00 +m 0 +1, . . . , n+m 00 +m 0 +m 00 ), and replace the last row by (−c, 0, 0, 0). Then add suitable multiples of the other rows to the last row in order to get zeros at m 0 +m 00 places corresponding to columns with distinct unit vectors; these will form our basis. The result is the simplex tableau with respect to the original LP and this basis. Therefore we can continue running the iterations of the S IMPLEX A LGORITHM as in Theorem 3.17(e). In fact, one can often do even more efficiently. Suppose we want to solve an LP min{cx : Ax ≥ b, x ≥ 0} with a very large number of inequalities which are implicitly given in a way that allows us to solve the following problem efficiently: Given a vector x ≥ 0, decide if Ax ≥ b and find a violated inequality otherwise. We apply the S IMPLEX A LGORITHM to the dual LP max{yb : y A ≤ c, y ≥ 0} = max{by : A> y ≤ c, y ≥ 0}. Let b¯ := (b> , 0). For a basis B we set Q B := ((A> I ) B )−1 and store only the right-hand part of the simplex tableau ! QB QBc . b¯ B Q B b> x ¯ To perform an iteration, The last row of the full simplex tableau is b¯ B Q B (A> I )− b. B B > ¯ ¯ we must check if b Q B ≥ 0 and b Q B A −c ≥ 0, and find a negative component if one exists. This reduces to solving the above problem for x = (b¯ B Q B )> . Then we generate the corresponding column of the full simplex tableau, but only for the current iteration. After updating the reduced tableau we can delete it again. This technique is known under the names revised simplex and column generation. We will see applications later.

3.4 Duality Theorem 3.14 shows that the LPs (3.1) and (3.2) are related. This motivates the following definition: Definition 3.18. Given a linear program max{cx : Ax ≤ b}, we define the dual LP to be the linear program min{yb : y A = c, y ≥ 0}. In this case, the original LP max{cx : Ax ≤ b} is often called the primal LP.

3.4 Duality

61

Proposition 3.19. The dual of the dual of an LP is (equivalent to) the original LP. Proof: Let the primal LP max{cx : Ax ≤ b} be given. Its dual is min{yb : y A = c, y ≥ 0}, or equivalently A> c − max −by : −A> y ≤ −c . 0 −I (Each equality constraint has been split up into two inequality constraints.) So the dual of the dual is z 0 0 0 − min zc − z c : A −A −I z = −b, z, z , w ≥ 0 w which is equivalent to − min{−cx : −Ax − w = −b, w ≥ 0} (where we have substituted x for z 0 − z). By eliminating the slack variables w we see that this is equivalent to the primal LP. We now obtain the most important theorem in LP theory, the Duality Theorem: Theorem 3.20. (von Neumann [1947], Gale, Kuhn and Tucker [1951]) If the polyhedra P := {x : Ax ≤ b} and D := {y : y A = c, y ≥ 0} are both nonempty, then max{cx : x ∈ P} = min{yb : y ∈ D}. Proof: If D is nonempty, it has a vertex y. We run the S IMPLEX A LGORITHM for min{yb : y ∈ D} and y. By Proposition 3.13, the existence of some x ∈ P guarantees that min{yb : y ∈ D} is not unbounded. Thus by Theorem 3.14, the S IMPLEX A LGORITHM returns optimum solutions y and z of the LP min{yb : y ∈ D} and its dual. However, the dual is max{cx : x ∈ P} by Proposition 3.19. We have yb = cz, as required. We can say even more about the relation between the optimum solutions of the primal and dual LP: Corollary 3.21. Let max{cx : Ax ≤ b} and min{yb : y A = c, y ≥ 0} be a primal-dual pair of LPs. Let x and y be feasible solutions, i.e. Ax ≤ b, y A = c and y ≥ 0. Then the following statements are equivalent: (a) x and y are both optimum solutions. (b) cx = yb. (c) y(b − Ax) = 0. Proof: The Duality Theorem 3.20 immediately implies the equivalence of (a) and (b). The equivalence of (b) and (c) follows from y(b − Ax) = yb − y Ax = yb −cx.

62

3 Linear Programming

The property (c) of optimum solutions is often called complementary slackness. It can also be formulated as follows: a point x ∗ ∈ P = {x : Ax ≤ b} is an optimum solution of max{cx : x ∈ P} if and only if c is a nonnegative combination of those rows of A which correspond to inequalities of Ax ≤ b that are satisfied by x ∗ with equality. It also implies: Corollary 3.22. Let P = {x : Ax ≤ b} be a polyhedron and Z ⊆ P. Then the set of vectors c for which each z ∈ Z is an optimum solution of max{cx : x ∈ P} is the cone generated by the rows of A0 , where A0 x ≤ b0 is the maximal subsystem of Ax ≤ b with A0 z = b0 for all z ∈ Z . Proof: There is a z ∈ Z which satisfies all other inequalities of Ax ≤ b strictly. Let c be a vector for which z is an optimum solution of max{cx : x ∈ P}. Then by Corollary 3.21 there exists an y ≥ 0 with c = y A0 , i.e. c is a nonnegative linear combination of the rows of A0 . Conversely, for a row a 0 x ≤ β 0 of A0 ≤ b0 and z ∈ Z we have a 0 z = β 0 = max{a 0 x : x ∈ P}. Let us write Corollary 3.21 in another form: Corollary 3.23. Let min{cx : Ax ≥ b, x ≥ 0} and max{yb : y A ≤ c, y ≥ 0} be a primal-dual pair of LPs. Let x and y be feasible solutions, i.e. Ax ≥ b, y A ≤ c and x, y ≥ 0. Then the following statements are equivalent: (a) x and y are both optimum solutions. (b) cx = yb. (c) (c − y A)x = 0 and y(b − Ax) = 0. Proof: Theequivalence of (a) and (b) is obtained by applying the Duality Theorem −b . 3.20 to max (−c)x : −A −I x ≤ 0 To prove that (b) and (c) are equivalent, observe that we have y(b − Ax) ≤ 0 ≤ (c − y A)x for any feasible solutions x and y, and that y(b − Ax) = (c − y A)x iff yb = cx. The two conditions in (c) are sometimes called primal and dual complementary slackness conditions. The Duality Theorem has many applications in combinatorial optimization. One reason for its importance is that the optimality of a solution can be proved by giving a feasible solution of the dual LP with the same objective value. We shall show now how to prove that an LP is unbounded or infeasible: Theorem 3.24. There exists a vector x with Ax ≤ b if and only if yb ≥ 0 for each vector y ≥ 0 for which y A = 0. Proof: If there is a vector x with Ax ≤ b, then yb ≥ y Ax = 0 for each y ≥ 0 with y A = 0.

3.4 Duality

63

Consider the LP − min{1lw : Ax − w ≤ b, w ≥ 0}. Writing it in standard form we have ! ( x : max 0 −1l w

A −I 0 −I

!

x w

!

b ≤ 0

(3.6)

!) .

The dual of this LP is ! ( ! ) ! ! y A> 0 0 y : min b 0 , y, z ≥ 0 , = z −1l z −I −I or, equivalently, min{yb : y A = 0, 0 ≤ y ≤ 1l}.

(3.7)

Since both (3.6) and (3.7) have a solution (x = 0, w = |b|, y = 0), we can apply Theorem 3.20. So the optimum values of (3.6) and (3.7) are the same. Since the system Ax ≤ b has a solution iff the optimum value of (3.6) is zero, the proof is complete. So the fact that a linear inequality system Ax ≤ b has no solution can be proved by giving a vector y ≥ 0 with y A = 0 and yb < 0. We mention two equivalent formulations of Theorem 3.24: Corollary 3.25. There is a vector x ≥ 0 with Ax ≤ b if and only if yb ≥ 0 for each vector y ≥ 0 with y A ≥ 0. A x ≤ b . Proof: Apply Theorem 3.24 to the system −I 0 Corollary 3.26. (Farkas [1894]) There is a vector x ≥ 0 with Ax = b if and only if yb ≥ 0 for each vector y with y A ≥ 0. b A x ≤ Proof: Apply Corollary 3.25 to the system −A −b , x ≥ 0. Corollary 3.26 is usually known as Farkas’ Lemma. The above results in turn imply the Duality Theorem 3.20 which is interesting since they have quite easy direct proofs (in fact they were known before the S IMPLEX A LGORITHM); see Exercises 10 and 11. We have seen how to prove that an LP is infeasible. How can we prove that an LP is unbounded? The next theorem answers this question. Theorem 3.27. If an LP is unbounded, then its dual LP is infeasible. If an LP has an optimum solution, then its dual also has an optimum solution.

64

3 Linear Programming

Proof: The first statement follows immediately from Proposition 3.13. To prove the second statement, suppose that the (primal) LP max{cx : Ax ≤ b} has an optimum solution x ∗ , but the dual min{yb : y A = c, y ≥ 0} is infeasible (it cannot be unbounded due to the first statement). In other words, there is no y ≥ 0 with A> y = c, and we apply Farkas’ Lemma (Corollary 3.26) to get a vector z with z A> ≥ 0 and zc < 0. But then x ∗ − z is feasible for the primal, because A(x ∗ − z) = Ax ∗ − Az ≤ b. The observation c(x ∗ − z) > cx ∗ therefore contradicts the optimality of x ∗ . So there are four cases for a primal-dual pair of LPs: either both have an optimum solution (in which case the optimum values are the same), or one is infeasible and the other one is unbounded, or both are infeasible. We also note: Corollary 3.28. A feasible LP max{cx : Ax ≤ b} is bounded if and only if c belongs to the cone generated by the rows of A. Proof: The LP is bounded iff its dual is feasible, i.e. there is a y ≥ 0 with y A = c. Farkas’ Lemma also enables us to prove that each finitely generated cone is polyhedral: Theorem 3.29. (Minkowski [1896], Weyl [1935]) A cone is polyhedral if and only if it is finitely generated. Proof: The only-if direction is given by Lemma 3.12. So consider the cone C generated by a1 , . . . , at . We have to show that C is polyhedral. Let A be the matrix whose rows are a1 , . . . , at . By Lemma 3.12, the cone D := {x : Ax ≤ 0} is generated by some vectors b1 , . . . , bs . Let B be the matrix whose rows are b1 , . . . , bs . We prove that C = {x : Bx ≤ 0}. As b j ai = ai b j ≤ 0 for all i and j, we have C ⊆ {x : Bx ≤ 0}. Now suppose there is a vector w ∈ / C with Bw ≤ 0. w 6∈ C means that there is no v ≥ 0 such that A> v = w. By Farkas’ Lemma (Corollary 3.26) this means that there is a vector y with yw < 0 and Ay ≥ 0. So −y ∈ D. Since D is generated by b1 , . . . , bs we have −y = z B for some z ≥ 0. But then 0 < −yw = z Bw ≤ 0, a contradiction.

3.5 Convex Hulls and Polytopes In this section we collect some more facts on polytopes. In particular, we show that polytopes are precisely those sets that are the convex hull of a finite number of points. We start by recalling some basic definitions: Pk Definition 3.30. Given vectors x1 , . . . , xk ∈ Rn and λ1 , . . . , λk ≥ 0 with i=1 λi Pk = 1, we call x = i=1 λi xi a convex combination of x1 , . . . , xk . A set X ⊆ Rn

3.5 Convex Hulls and Polytopes

65

is convex if λx + (1 − λ)y ∈ X for all x, y ∈ X and λ ∈ [0, 1]. The convex hull conv(X ) of a set X is defined as the set of all convex combinations of points in X . An extreme point of a set X is an element x ∈ X with x ∈ / conv(X \ {x}). So a set X is convex if and only if all convex combinations of points in X are again in X . The convex hull of a set X is the smallest convex set containing X . Moreover, the intersection of convex sets is convex. Hence polyhedra are convex. Now we prove the “finite basis theorem for polytopes”, a fundamental result which seems to be obvious but is not trivial to prove directly: Theorem 3.31. (Minkowski [1896], Steinitz [1916], Weyl [1935]) A set P is a polytope if and only if it is the convex hull of a finite set of points. Proof: (Schrijver [1986]) Let P = {x ∈ Rn : Ax ≤ b} be a nonempty polytope. Obviously, ! ) ( ! ) ( x x n+1 ∈R ∈ C , where C = : λ ≥ 0, Ax − λb ≤ 0 . P= x: λ 1 C is a polyhedral cone, so by Theorem 3.29 it is generated by finitely many nonzero x x vectors, say by λ11 , . . . , λkk . Since P is bounded, all λi are nonzero; w.l.o.g. all λi are 1. So x ∈ P if and only if ! ! ! x1 xk x = µ1 + · · · + µk 1 1 1 for some µ1 , . . . , µk ≥ 0. In other words, P is the convex hull of x1 , . . . , xk . n x ∈ P if and only if Now let P be the convex hull of x1 , . . . , xx1k ∈ R . Then xk x . By Theorem 3.29, C ∈ C, where C is the cone generated by , . . . , 1 1 1 is polyhedral, so ) ( ! x : Ax + bλ ≤ 0 . C = λ We conclude that P = {x ∈ Rn : Ax + b ≤ 0}.

Corollary 3.32. A polytope is the convex hull of its vertices. Proof: Let P be a polytope. By Theorem 3.31, the convex hull of its vertices is a polytope Q. Obviously Q ⊆ P. Suppose there is a point z ∈ P \ Q. Then there is a vector c with cz > max{cx : x ∈ Q}. The supporting hyperplane {x : cx = max{cy : y ∈ P}} of P defines a face of P containing no vertex. This is impossible by Corollary 3.10. The previous two and the following result are the starting point of polyhedral combinatorics; they will be used very often in this book. For a given ground set E and a subset X ⊆ E, the incidence vector of X (with respect to E) is defined as the vector x ∈ {0, 1} E with xe = 1 for e ∈ X and xe = 0 for e ∈ E \ X .

66

3 Linear Programming

Corollary 3.33. Let (E, F) be a set system, P the convex hull of the incidence vectors of the elements of F, and c : E → R. Then max{cx : x ∈ P} = max{c(X ) : X ∈ F}. Proof: Since max{cx : x ∈ P} ≥ max{c(X ) : X ∈ F} is trivial, let x be an optimum solution of max{cx : x ∈ P} (note that P is a polytope by Theorem 3.31). By definition of P,P x is a convex combination of incidence vectors Pk y1 , . . . , yk k of elements of F: x = i=1 λi yi for some λ1 , . . . , λk ≥ 0 with i=1 λi = 1. Pk Since cx = i=1 λi cyi , we have cyi ≥ cx for at least one i ∈ {1, . . . , k}. This yi is the incidence vector of a set Y ∈ F with c(Y ) = cyi ≥ cx.

Exercises 1. Let H be a hypergraph, F ⊆ V (H P ), and x, y : F → R. The task is to find x, y : V (H ) \ F → R such that e∈E(H ) (maxv∈e x(v) − minv∈e x(v) + maxv∈e y(v) − minv∈e y(v)) is minimum. Show that this can be formulated as an LP. Note: This is a relaxation of a placement problem in VLSI design. Here H is called the netlist, and its vertices correspond to modules that need to placed on the chip. Some (those in F) are pre-placed in advance. The main difficulty (ignored in this relaxation) is that modules must not overlap. is called affinely independent if there is no λ ∈ 2. A set of vectors x1 , . . . , xk P k λi xi = 0. Let ∅ 6= X ⊆ Rn . Show that the Rk \ {0} with λ> 1l = 0 and i=1 maximum cardinality of an affinely independent set of elements of X equals dim X + 1. 3. Let P, Q ∈ Rn be polyhedra. Prove that the closure of conv(P ∪ Q) is a polyhedron. Show polyhedra P and Q for which conv(P ∪ Q) is not a polyhedron. 4. Show that the problem to compute the largest ball that is a subset of a given polyhedron can be formulated as a linear program. 5. Let P be a polyhedron. Prove that the dimension of any facet of P is one less than the dimension of P. 6. Let F be a minimal face of a polyhedron {x : Ax ≤ b}. Prove that then Ax = Ay for all x, y ∈ F. 7. Formulate the dual of the LP formulation (1.1) of the J OB A SSIGNMENT P ROBLEM. Show how to solve the primal and the dual LP in the case when there are only two jobs (by a simple algorithm). 8. Let G be a digraph, c : E(G) → R+ , E 1 , E 2 ⊆ E(G), and s, t ∈ V (G). Consider the following linear program

Exercises

X

min

67

c(e)ye

e∈E(G)

s.t.

ye zt − zs ye ye

≥ = ≥ ≤

zw − zv 1 0 0

(e = (v, w) ∈ E(G)) (e ∈ E 1 ) (e ∈ E 2 ).

Prove that there is an optimum solution (y, z) and s ∈ X ⊆ V (G) \ {t} with ye = 1 for e ∈ δ + (X ), ye = −1 for e ∈ δ − (X ) \ E 1 , and ye = 0 for all other edges e. Hint: Consider the complementary slackness conditions for the edges entering or leaving {v ∈ V (G) : z v ≤ z s }. 9. Let Ax ≤ b be a linear inequality system in n variables. By multiplying each row by a positive constant we may assume that the first column of A is a vector with entries 0, −1 and 1 only. So we can write Ax ≤ b equivalently as ai0 x 0 ≤ bi −x1 + a j0 x 0 x1 + ak0 x 0

(i = 1, . . . , m 1 ),

≤ bj

( j = m 1 + 1, . . . , m 2 ),

≤ bk

(k = m 2 + 1, . . . , m),

0 are the rows of A without the first where x 0 = (x2 , . . . , xn ) and a10 , . . . , am entry. Then one can eliminate x1 : Prove that Ax ≤ b has a solution if and only if the system

ai0 x 0 ≤ bi a j0 x 0

10. 11. 12.

∗ 13.

− bj

≤

bk − ak0 x 0

(i = 1, . . . , m 1 ), ( j = m 1 + 1, . . . , m 2 , k = m 2 + 1, . . . , m)

has a solution. Show that this technique, when iterated, leads to an algorithm for solving a linear inequality system Ax ≤ b (or proving infeasibility). Note: This method is known as Fourier-Motzkin elimination because it was proposed by Fourier and studied by Motzkin [1936]. One can prove that it is not a polynomial-time algorithm. Use Fourier-Motzkin elimination (Exercise 9) to prove Theorem 3.24 directly. (Kuhn [1956]) Show that Theorem 3.24 implies the Duality Theorem 3.20. Prove the decomposition theorem for polyhedra: Any polyhedron P can be written as P = {x + c : x ∈ X, c ∈ C}, where X is a polytope and C is a polyhedral cone. (Motzkin [1936]) Let P be a rational polyhedron and F a face of P. Show that {c : cz = max {cx : x ∈ P} for all z ∈ F} is a rational polyhedral cone.

68

3 Linear Programming

14. Prove Carath´eodory’s theorem: If X ⊆ Rn and y ∈ conv(X ), then there are x1 , . . . , xn+1 ∈ X such that y ∈ conv({x1 , . . . , xn+1 }). (Carath´eodory [1911]) 15. Prove the following extension of Carath´eodory’s theorem (Exercise 14): If X ⊆ Rn and y, z ∈ conv(X ), then there are x1 , . . . , xn ∈ X such that y ∈ conv({z, x1 , . . . , xn }). 16. Prove that the extreme points of a polyhedron are precisely its vertices. 17. Let P be a nonempty polytope. Consider the graph G(P) whose vertices are the vertices of P and whose edges correspond to the 1-dimensional faces of P. Let x be any vertex of P, and c a vector with c> x < max{c> z : z ∈ P}. Prove that then there is a neighbour y of x in G(P) with c> x < c> y. ∗ 18. Use Exercise 17 to prove that G(P) is n-connected for any n-dimensional polytope P (n ≥ 1). 19. Let P ⊆ Rn be a polytope (not necessarily rational) and y ∈ / P. Prove that there exists a rational vector c with max{cx : x ∈ P} < cy. Show that the statement does not hold for general polyhedra. / X . Prove: 20. Let X ⊂ Rn be a nonempty convex set, X¯ the closure of X , and y ∈ (a) There is a unique point in X¯ that has minimum distance to y. (b) There exists a vector a ∈ Rn with a > x < a > y for all x ∈ X . (c) If X is bounded and y ∈ / X¯ , then there exists a vector a ∈ Qn with > > a x < a y for all x ∈ X . (d) A closed convex set is the intersection of all closed half-spaces containing it.

References General Literature: Bertsimas, D., and Tsitsiklis, J.N. [1997]: Introduction to Linear Optimization. Athena Scientific, Belmont 1997 Chv´atal, V. [1983]: Linear Programming. Freeman, New York 1983 Matouˇsek, J., and G¨artner, B. [2007]: Understanding and Using Linear Programming. Springer, Berlin 2007 Padberg, M. [1999]: Linear Optimization and Extensions. Second Edition. Springer, Berlin 1999 Schrijver, A. [1986]: Theory of Linear and Integer Programming. Wiley, Chichester 1986

Cited References: Avis, D., and Chv´atal, V. [1978]: Notes on Bland’s pivoting rule. Mathematical Programming Study 8 (1978), 24–34 Bland, R.G. [1977]: New finite pivoting rules for the simplex method. Mathematics of Operations Research 2 (1977), 103–107 Borgwardt, K.-H. [1982]: The average number of pivot steps required by the simplex method is polynomial. Zeitschrift f¨ur Operations Research 26 (1982), 157–177

References

69

¨ Carath´eodory, C. [1911]: Uber den Variabilit¨atsbereich der Fourierschen Konstanten von positiven harmonischen Funktionen. Rendiconto del Circolo Matematico di Palermo 32 (1911), 193–217 Dantzig, G.B. [1951]: Maximization of a linear function of variables subject to linear inequalities. In: Activity Analysis of Production and Allocation (T.C. Koopmans, ed.), Wiley, New York 1951, pp. 359–373 Dantzig, G.B., Orden, A., and Wolfe, P. [1955]: The generalized simplex method for minimizing a linear form under linear inequality restraints. Pacific Journal of Mathematics 5 (1955), 183–195 Farkas, G. [1894]: A Fourier-f´ele mechanikai elv alkalmaz´asai. Mathematikai e´ s Term´esz´ ettudom´anyi Ertesit¨ o 12 (1894), 457–472 Gale, D., Kuhn, H.W., and Tucker, A.W. [1951]: Linear programming and the theory of games. In: Activity Analysis of Production and Allocation (T.C. Koopmans, ed.), Wiley, New York 1951, pp. 317–329 Hoffman, A.J., and Kruskal, J.B. [1956]: Integral boundary points of convex polyhedra. In: Linear Inequalities and Related Systems; Annals of Mathematical Study 38 (H.W. Kuhn, A.W. Tucker, eds.), Princeton University Press, Princeton 1956, pp. 223–246 Kelner, J.A., and Spielman, D.A. [2006]: A randomized polynomial-time simplex algorithm for linear programming. Proceedings of the 38th Annual ACM Symposium on Theory of Computing (2006), 51–60 Klee, V., and Minty, G.J. [1972]: How good is the simplex algorithm? In: Inequalities III (O. Shisha, ed.), Academic Press, New York 1972, pp. 159–175 Kuhn, H.W. [1956]: Solvability and consistency for linear equations and inequalities. The American Mathematical Monthly 63 (1956), 217–232 Minkowski, H. [1896]: Geometrie der Zahlen. Teubner, Leipzig 1896 Motzkin, T.S. [1936]: Beitr¨age zur Theorie der linearen Ungleichungen (Dissertation). Azriel, Jerusalem 1936 von Neumann, J. [1947]: Discussion of a maximum problem. Working paper. Published in: John von Neumann, Collected Works; Vol. VI (A.H. Taub, ed.), Pergamon Press, Oxford 1963, pp. 27–28 Spielman, D.A., and Teng, S.-H. [2004]: Smoothed analysis of algorithms: why the simplex algorithm usually takes polynomial time. Journal of the ACM 51 (2004), 385–463 Steinitz, E. [1916]: Bedingt konvergente Reihen und konvexe Systeme. Journal f¨ur die reine und angewandte Mathematik 146 (1916), 1–52 Weyl, H. [1935]: Elementare Theorie der konvexen Polyeder. Commentarii Mathematici Helvetici 7 (1935), 290–306

4 Linear Programming Algorithms

There are basically three types of algorithms for L INEAR P ROGRAMMING: the S IMPLEX A LGORITHM (see Section 3.2), interior point algorithms, and the E L LIPSOID M ETHOD . Each of these has a disadvantage: In contrast to the other two, so far no variant of the S IMPLEX A LGORITHM has been shown to have a polynomial running time. In Sections 4.4 and 4.5 we present the E LLIPSOID M ETHOD and prove that it leads to a polynomial-time algorithm for L INEAR P ROGRAMMING. However, the E LLIPSOID M ETHOD is too inefficient to be used in practice. Interior point algorithms and, despite its exponential worst-case running time, the S IMPLEX A LGORITHM are far more efficient, and they are both used in practice to solve LPs. In fact, both the E LLIPSOID M ETHOD and interior point algorithms can be used for more general convex optimization problems, e.g. for so-called semidefinite programs. An advantage of the S IMPLEX A LGORITHM and the E LLIPSOID M ETHOD is that they do not require the LP to be given explicitly. It suffices to have an oracle (a subroutine) which decides whether a given vector is feasible and, if not, returns a violated constraint. We shall discuss this in detail with respect to the E LLIPSOID M ETHOD in Section 4.6, because it implies that many combinatorial optimization problems can be solved in polynomial time; for some problems this is in fact the only known way to show polynomial solvability. This is the reason why we discuss the E LLIPSOID M ETHOD but not interior point algorithms in this book. A prerequisite for polynomial-time algorithms is that there exists an optimum solution that has a binary representation whose length is bounded by a polynomial in the input size. We prove in Section 4.1 that this condition holds for L INEAR P ROGRAMMING. In Sections 4.2 and 4.3 we review some basic algorithms needed later, including the well-known Gaussian elimination method for solving systems of equations.

4.1 Size of Vertices and Faces Instances of L INEAR P ROGRAMMING are vectors and matrices. Since no strongly polynomial-time algorithm for L INEAR P ROGRAMMING is known we have to restrict attention to rational instances when analyzing the running time of algorithms. We assume that all numbers are coded in binary. To estimate the size (number of bits) of this representation we define size(n) := 1 + dlog(|n| + 1)e for integers

72

4 Linear Programming Algorithms

n ∈ Z and size(r ) := size( p) + size(q) for rational numbers r = qp , where p, q are relatively prime integers (i.e. their greatest common divisor is 1). For vectors x = (x1 , . . . , xn ) ∈ Qn we store the components and have size(x) := . + size(xn ). For a matrix A ∈ Qm×n with entries ai j we have n + size(x1 ) + . . P size(A) := mn + i, j size(ai j ). Of course these precise values are a somewhat random choice, but remember that we are not really interested in constant factors. For polynomial-time algorithms it is important that the sizes of numbers do not increase too much by elementary arithmetic operations. We note: Proposition 4.1. If r1 , . . . , rn are rational numbers, then size(r1 · · · rn ) ≤ size(r1 ) + · · · + size(rn ); size(r1 + · · · + rn ) ≤ 2(size(r1 ) + · · · + size(rn )). Proof: For integers s1 , . . . , sn we obviously have size(s1 · · · sn ) ≤ size(s1 ) + · · · + size(sn ) and size(s1 + · · · + sn ) ≤ size(s1 ) + · · · + size(sn ). Let now ri = qpii , where pi and qi are nonzero integers (i = 1, . . . , n). Then size(r1 · · · rn ) ≤ size( p1 · · · pn ) + size(q1 · · · qn ) ≤ size(r1 ) + · · · + size(rn ). For the second statement, observe that the denominator q1 · · · qn has size at most size(q1 ) + · · · + size(qn ). The numerator is the sum of the numbers q1 · · · qi−1 pi qi+1 · · · qn (i = 1, . . . , n), so its absolute value is at most (| p1 | + · · · + | pn |)|q1 · · · qn |. Therefore the size of the numerator is at most size(r1 ) + · · · + size(rn ). The first part of this proposition also implies that we can often assume w.l.o.g. that all numbers in a problem instance are integers, since otherwise we can multiply each of them with the product of all denominators. For addition and inner product of vectors we have: Proposition 4.2. If x, y ∈ Qn are rational vectors, then size(x + y) ≤ 2(size(x) + size(y)); size(x > y) ≤ 2(size(x) + size(y)). Pn Proof: Pn 4.1 we have size(x + y) = n + i=1 size(xi +>yi ) ≤ Pn Using Proposition and size(x y) = n +2 P i=1 size(x i )+2 Pni=1 size(yi ) = 2(size(x)+size(y))−3n Pn P n n x y size(x y ) ≤ 2 size(x ) + 2 size ≤ 2 i i i i i i=1 i=1 i=1 i=1 size(yi ) = 2(size(x) + size(y)) − 4n. Even under more complicated operations the numbers involved do not grow fast. Recall that the determinant of a matrix A = (ai j )1≤i, j≤n is defined by det A :=

X π ∈Sn

sgn(π)

n Y i=1

ai,π(i) ,

(4.1)

4.1 Size of Vertices and Faces

73

where Sn is the set of all permutations of {1, . . . , n} and sgn(π) is the sign of the permutation π (defined to be 1 if π can be obtained from the identity map by an even number of transpositions, and −1 otherwise). Proposition 4.3. For any matrix A ∈ Qm×n we have size(det A) ≤ 2 size(A). pi j qi j

with relatively prime integers pi j , qi j . Now let det A = Q where p and q are relatively prime integers. Then |det A| ≤ i, j (| pi j | + 1) Q and Q |q| ≤ i, j |qi j |. We obtain size(q) ≤ size(A) and, using | p| = |det A||q| ≤ i, j (| pi j | + 1)|qi j |,

Proof: We write ai j = p q

size( p) ≤

X (size( pi j ) + 1 + size(qi j )) = size(A). i, j

With this observation we can prove: Theorem 4.4. Suppose the rational LP max{cx : Ax ≤ b} has an optimum solution. Then it also has an optimum solution x with size(x) ≤ 4n(size(A) + size(b)), with components of size at most 4(size(A)+size(b)). If b = ei or b = −ei for some unit vector ei , then there is a nonsingular submatrix A0 of A and an optimum solution x with size(x) ≤ 4n size(A0 ). Proof: By Corollary 3.5, the maximum is attained in a face F of {x : Ax ≤ b}. Let F 0 ⊆ F be a minimal face. By Proposition 3.9, F 0 = {x : A0 x = b0 } for some subsystem A0 x ≤ b0 of Ax ≤ b. W.l.o.g., we may assume that the rows of A0 are linearly independent. We then take a maximal set of linear independent columns (call this matrix A00 ) and set all other components to zero. Then x = (A00 )−1 b0 , filled up with zeros, is an optimum solution to our LP. By Cramer’s rule the entries of x A000 000 00 are given by x j = det det A00 , where A arises from A by replacing the j-th column by b0 . By Proposition 4.3 we obtain size(x) ≤ n + 2n(size(A000 ) + size(A00 )) ≤ 4n(size(A00 ) + size(b0 )). If b = ±ei then | det(A000 )| is the absolute value of a subdeterminant of A00 . The encoding length of the faces of a polytope given by its vertices can be estimated as follows: Lemma 4.5. Let P ⊆ Rn be a rational polytope and T ∈ N such that size(x) ≤ T for each vertex x. Then P = {x : Ax ≤ b} for some inequality system Ax ≤ b, each of whose inequalities ax ≤ β satisfies size(a) + size(β) ≤ 75n 2 T . Proof: First assume that P is full-dimensional. Let F = {x ∈ P : ax = β} be a facet of P, where P ⊆ {x : ax ≤ β}. Let y1 , . . . , yt be the vertices of F (by Proposition 3.6 they are also vertices of P). Let c be the solution of Mc = e1 , where M is a t × n-matrix whose i-th row is yi − y1 (i = 2, . . . , t) and whose first row is some unit vector that is linearly

74

4 Linear Programming Algorithms

independent of the other rows. Observe that rank(M) = n (because dim F = n −1). So we have c> = κa for some κ ∈ R \ {0}. By Theorem 4.4 size(c) ≤ 4n size(M 0 ), where M 0 is a nonsingular n × nsubmatrix of M. By Proposition 4.2 we have size(M 0 ) ≤ 4nT and size(c> y1 ) ≤ 2(size(c) + size(y1 )). So the inequality c> x ≤ δ (or c> x ≥ δ if κ < 0), where δ := c> y1 = κβ, satisfies size(c) + size(δ) ≤ 3 size(c) + 2T ≤ 48n 2 T + 2T ≤ 50n 2 T . Collecting these inequalities for all facets F yields a description of P. If P = ∅, the assertion is trivial, so we now assume that P is neither fulldimensional nor empty. Let V be the set of vertices of P. For s = (s1 , . . . , sn ) ∈ {−1, 1}n let Ps be the convex hull of V ∪ {x + si ei : x ∈ V, i = 1, . . . , n}. Each Ps is a full-dimensional polytope (Theorem 3.31), and the size of any of its vertices is at most T + n (cf. Corollary 3.32). By the above, Ps can be described by inequalities of size at most 50n 2 (T + n) ≤ 75n 2 T (note that T ≥ 2n). Since T P = s∈{−1,1}n Ps , this completes the proof.

4.2 Continued Fractions When we say that the numbers occurring in a certain algorithm do not grow too fast, we often assume that for each rational qp the numerator p and the denominator q are relatively prime. This assumption causes no problem if we can easily find the greatest common divisor of two natural numbers. This is accomplished by one of the oldest algorithms:

E UCLIDEAN A LGORITHM Input:

Two natural numbers p and q.

Output:

The greatest common divisor d of p and q, i.e. prime integers.

p d

and

q d

are relatively

1

While p > 0 and q > 0 do: If p < q then set q := q − b qp c p else set p := p − b qp cq.

2

Return d := max{ p, q}.

Theorem 4.6. The E UCLIDEAN A LGORITHM works correctly. The number of iterations is at most size( p) + size(q). Proof: The correctness follows from the fact that the set of common divisors of p and q does not change throughout the algorithm, until one of the numbers becomes zero. One of p or q is reduced by at least a factor of two in each iteration, hence there are at most log p + log q + 1 iterations. Since no number occurring in an intermediate step is greater than p and q, we have a polynomial-time algorithm. A similar algorithm is the so-called C ONTINUED F RACTION E XPANSION. This can be used to approximate any number by a rational number whose denominator is

4.2 Continued Fractions

not too large. For any positive real number x we define x0 := x and xi+1 := for i = 1, 2, . . ., until xk ∈ N for some k. Then we have x = x0 = bx0 c +

1 1 = bx0 c + x1 bx1 c +

1 x2

= bx0 c +

1 bx1 c +

1 bx2 c+ x1

75 1 xi −bxi c

= ···

3

We claim that this sequence is finite if and only if x is rational. One direction follows immediately from the observation that xi+1 is rational if and only if xi is rational. The other direction is also easy: If x = qp , the above procedure is equivalent to the E UCLIDEAN ALGORITHM applied to p and q. This also shows that for a given rational number qp with p, q > 0 the (finite) sequence x1 , x2 , . . . , xk as above can be computed in polynomial time. The following algorithm is almost identical for the computation of the numbers gi to the E UCLIDEAN A LGORITHM except and h i ; we shall prove that the sequence hgii converges to x. i∈N

C ONTINUED F RACTION E XPANSION Input: Output:

Natural numbers p and q (let x := qp ). with x0 = The sequence xi = qpii i=0,1,...

1

2

p q

and xi+1 :=

1 xi −bxi c .

Set i := 0, p0 := p and q0 := q. Set g−2 := 0, g−1 := 1, h −2 := 1, and h −1 := 0. While qi 6= 0 do: Set ai := b qpii c. Set gi := ai gi−1 + gi−2 . Set h i := ai h i−1 + h i−2 . Set qi+1 := pi − ai qi . Set pi+1 := qi . Set i := i + 1.

We claim that the sequence hgii yields good approximations of x. Before we can prove this, we need some preliminary observations: Proposition 4.7. The following statements hold for all iterations i in the above algorithm: (a) ai ≥ 1 (except possibly for i = 0) and h i ≥ h i−1 . (b) gi−1 h i − gi h i−1 = (−1)i . pi gi−1 + qi gi−2 (c) = x. pi h i−1 + qi h i−2 (d) hgii ≤ x if i is even and hgii ≥ x if i is odd. Proof: (a) is obvious. (b) is easily shown by induction: For i = 0 we have gi−1 h i − gi h i−1 = g−1 h 0 = 1, and for i ≥ 1 we have

76

4 Linear Programming Algorithms

gi−1 h i − gi h i−1 = gi−1 (ai h i−1 + h i−2 ) − h i−1 (ai gi−1 + gi−2 ) = gi−1 h i−2 − h i−1 gi−2 . (c) is also proved by induction: For i = 0 we have pi · 1 + 0 pi gi−1 + qi gi−2 = = x. pi h i−1 + qi h i−2 0 + qi · 1 For i ≥ 1 we have pi gi−1 + qi gi−2 pi h i−1 + qi h i−2

qi−1 (ai−1 gi−2 + gi−3 ) + ( pi−1 − ai−1 qi−1 )gi−2 qi−1 (ai−1 h i−2 + h i−3 ) + ( pi−1 − ai−1 qi−1 )h i−2 qi−1 gi−3 + pi−1 gi−2 . qi−1 h i−3 + pi−1 h i−2

= =

= 0 < x < ∞ = hg−1 and proceed by We finally prove (d). We note hg−2 −2 −1 induction. The induction step follows easily from the fact that the function f (α) := αgi−1 +gi−2 pi αh i−1 +h i−2 is monotone for α > 0, and f ( qi ) = x by (c). Theorem 4.8. (Khintchine [1956]) Given a rational number α and a natural number n, a rational number β with denominator at most n such that |α − β| is minimum can be found in polynomial time (polynomial in size(n) + size(α)). Proof: We run the C ONTINUED F RACTION E XPANSION with x := α. If the = α by algorithm stops with qi = 0 and h i−1 ≤ n, we can set β = hgi−1 i−1 Proposition 4.7(c). Otherwise let i be the last index with h i ≤ n, and let t be the maximum integer such that th i + h i−1 ≤ n (cf. Proposition 4.7(a)). Since ai+1 h i + h i−1 = h i+1 > n, we have t < ai+1 . We claim that y :=

gi hi

or

z :=

tgi + gi−1 th i + h i−1

is an optimum solution. Both numbers have denominators at most n. If i is even, then y ≤ x < z by Proposition 4.7(d). Similarly, if i is odd, we have y ≥ x > z. We show that any rational number qp between y and z has denominator greater than n. Observe that |z − y| =

1 |h i gi−1 − h i−1 gi | = h i (th i + h i−1 ) h i (th i + h i−1 )

(using Proposition 4.7(b)). On the other hand, 1 h i−1 + (t + 1)h i p p 1 |z − y| = z − + − y ≥ + = , q q (th i + h i−1 )q hi q qh i (th i + h i−1 ) so q ≥ h i−1 + (t + 1)h i > n.

The above proof is from the book of Gr¨otschel, Lov´asz and Schrijver [1988], which also contains important generalizations.

4.3 Gaussian Elimination

77

4.3 Gaussian Elimination The most important algorithm in Linear Algebra is the so-called Gaussian elimination. It has been applied by Gauss but was known much earlier (see Schrijver [1986] for historical notes). Gaussian elimination is used to determine the rank of a matrix, to compute the determinant and to solve a system of linear equations. It occurs very often as a subroutine in linear programming algorithms; e.g. in 1 of the S IMPLEX A LGORITHM. Given a matrix A ∈ Qm×n , our algorithm for Gaussian Elimination works with an extended matrix Z = (B C) ∈ Qm×(n+m) ; initially B = A and C = I . The algorithm transforms B to the form 0I R0 by the following elementary operations: permuting rows and columns, adding a multiple of one row to another row, and (in the final step) multiplying rows by nonzero constants. At each iteration C is modified accordingly, such that the property C A˜ = B is maintained throughout where A˜ results from A by permuting rows and columns. The first part of the algorithm, consisting of 2 and , 3 transforms B to an upper triangular matrix. Consider for example the matrix Z after two iterations; it has the form z 12 z 13 · · · z 1n 1 0 0 · · · 0 z 11 6= 0 0 z 22 6= 0 z 23 · · · z 2n z 2,n+1 1 0 · · · 0 0 0 z 33 · · · z 3n z 3,n+1 z 3,n+2 1 0 · · 0 · · · · · · 0 · . · · · I · · · · · · · · · · · · 0 0

0

z m3 · · · z mn

z m,n+1 z m,n+2 0

·

·

0 1

i3 times the third row If z 33 6= 0, then the next step just consists of subtracting zz33 from the i-th row, for i = 4, . . . , m. If z 33 = 0 we first exchange the third row and/or the third column with another one. Note that if we exchange two rows, we have to exchange also the two corresponding columns of C in order to maintain the property C A˜ = B. To have A˜ available at each point we store the permutations of the rows and columns in variables r ow(i), i = 1, . . . , m and col( j), j = 1, . . . , n. Then A˜ = (Ar ow(i),col( j) )i∈{1,...,m}, j∈{1,...,n} . The second part of the algorithm, consisting of 4 and , 5 is simpler since no rows or columns are exchanged anymore.

G AUSSIAN E LIMINATION Input:

A matrix A = (ai j ) ∈ Qm×n .

Output:

Its rank r , a maximal nonsingular submatrix A0 = (ar ow(i),col( j) )i, j∈{1,...,r } of A, its determinant d = det A0 , and its inverse (A0 )−1 = (z i,n+ j )i, j∈{1,...,r } .

78

4 Linear Programming Algorithms

1

Set r := 0 and d := 1. Set z i j := ai j , r ow(i) := i and col( j) := j (i = 1, . . . , m, j = 1, . . . , n). Set z i,n+ j := 0 and z i,n+i := 1 for 1 ≤ i, j ≤ m, i 6= j.

2

Let p ∈ {r + 1, . . . , m} and q ∈ {r + 1, . . . , n} with z pq 6= 0. If no such p and q exist, then go to . 4 Set r := r + 1. If p 6= r then exchange z pj and zr j ( j = 1, . . . , n + m), exchange z i,n+ p and z i,n+r (i = 1, . . . , m), and exchange r ow( p) and r ow(r ). If q 6= r then exchange z iq and z ir (i = 1, . . . , m), and exchange col(q) and col(r ). Set d := d · zrr . For i := r + 1 to m do: For j := r to n + r do: z i j := z i j − zzrrir zr j . Go to . 2 For k := r down to 1 do: For i := 1 to k − 1 do: ik For j := k to n + r do z i j := z i j − zzkk zk j .

3

4

5

For k := 1 to r do: For j := 1 to n + r do z k j :=

zk j z kk .

Theorem 4.9. G AUSSIAN E LIMINATION works correctly and terminates after O(mnr ) steps. Proof: First observe that each time before 2 we have z ii 6= 0 for i ∈ {1, . . . , r } and z i j = 0 for all j ∈ {1, . . . , r } and i ∈ { j + 1, . . . , m}. Hence det (z i j )i, j∈{1,2,...,r } = z 11 z 22 · · · zrr = d 6= 0. Since adding a multiple of one row to another row of a square matrix does not change the value of the determinant (this well-known fact follows directly from the definition (4.1)) we have det (z i j )i, j∈{1,2,...,r } = det (ar ow(i),col( j) )i, j∈{1,2,...,r } at any stage before , 5 and hence the determinant d is computed correctly. A0 is a nonsingular r × r -submatrix of A. Since (z i j )i∈{1,...,m}, j∈{1,...,n} has rank r at termination andPthe operations did not change the rank, A has also rank r . m Moreover, j=1 z i,n+ j ar ow( j),col(k) = z ik for all i ∈ {1, . . . , m} and k ∈ {1, . . . , n} (i.e. C A˜ = B in our above notation) holds throughout. (Note that for j = r + 1, . . . , m we have at any stage z j,n+ j = 1 and z i,n+ j = 0 for i 6= j.) Since (z i j )i, j∈{1,2,...,r } is the unit matrix at termination this implies that (A0 )−1 is also computed correctly. The number of steps is obviously O(r mn + r 2 (n + r )) = O(mnr ).

4.3 Gaussian Elimination

79

In order to prove that G AUSSIAN E LIMINATION is a polynomial-time algorithm we have to guarantee that all numbers that occur are polynomially bounded by the input size. This is not trivial but can be shown: Theorem 4.10. (Edmonds [1967]) G AUSSIAN E LIMINATION is a polynomialtime algorithm. Each number occurring in the course of the algorithm can be stored with O(m(m + n) size(A)) bits. Proof: We first show that in 2 and 3 all numbers are 0, 1, or quotients of subdeterminants of A. First observe that entries z i j with i ≤ r or j ≤ r are not modified anymore. Entries z i j with j > n +r are 0 (if j 6= n +i) or 1 (if j = n +i). Furthermore, we have for all s ∈ {r + 1, . . . , m} and t ∈ {r + 1, . . . , n + m} det (z i j )i∈{1,2,...,r,s}, j∈{1,2,...,r,t} z st = . det (z i j )i, j∈{1,2,...,r } (This follows from evaluating the determinant det (z i j )i∈{1,2,...,r,s}, j∈{1,2,...,r,t} along the last row because z s j = 0 for all s ∈ {r +1, . . . , m} and all j ∈ {1, . . . , r }.) We have already observed in the proof of Theorem 4.9 that det (z i j )i, j∈{1,2,...,r } = det (ar ow(i),col( j) )i, j∈{1,2,...,r } , because adding a multiple of one row to another row of a square matrix does not change the value of the determinant. By the same argument we have det (z i j )i∈{1,2,...,r,s}, j∈{1,2,...,r,t} = det (ar ow(i),col( j) )i∈{1,2,...,r,s}, j∈{1,2,...,r,t} for s ∈ {r + 1, . . . , m} and t ∈ {r + 1, . . . , n}. Furthermore, det (z i j )i∈{1,2,...,r,s}, j∈{1,2,...,r,n+t} = det (ar ow(i),col( j) )i∈{1,2,...,r,s}\{t}, j∈{1,2,...,r } for all s ∈ {r + 1, . . . , m} and t ∈ {1, . . . , r }, which is checked by evaluating the left-hand side determinant (after ) 1 along column n + t. We conclude that at any stage in 2 and 3 all numbers z i j are 0, 1, or quotients of subdeterminants of A. Hence, by Proposition 4.3, each number occurring in 2 and 3 can be stored with O(size(A)) bits. Finally observe that 4 is equivalent to applying 2 and 3 again, choosing p and q appropriately (reversing the order of the first r rows and columns). Hence each number occurring in 4 can be stored with O size (z i j )i∈{1,...,m}, j∈{1,...,m+n} bits, which is O(m(m + n) size(A)). The easiest way to keep the representations of the numbers z i j small enough is to guarantee that the numerator and denominator of each of these numbers are relatively prime at any stage. This can be accomplished by applying the E UCLIDEAN A LGORITHM after each computation. This gives an overall polynomial running time. In fact, we can easily implement G AUSSIAN E LIMINATION to be a strongly polynomial-time algorithm (Exercise 4).

80

4 Linear Programming Algorithms

So we can check in polynomial time whether a set of vectors is linearly independent, and we can compute the determinant and the inverse of a nonsingular matrix in polynomial time (exchanging two rows or columns changes just the sign of the determinant). Moreover we get: Corollary 4.11. Given a matrix A ∈ Qm×n and a vector b ∈ Qm we can in polynomial time find a vector x ∈ Qn with Ax = b or decide that no such vector exists. Proof: We compute a maximal nonsingular submatrix A0 = (ar ow(i),col( j) )i, j∈{1,...,r } of A and its inverse (A0P )−1 = (z i,n+ j )i, j∈{1,...,r } by G AUSSIAN E LIMINATION. r Then we set xcol( j) := k=1 z j,n+k br ow(k) for j = 1, . . . , r and x k := 0 for k∈ / {col(1), . . . , col(r )}. We obtain for i = 1, . . . r : n X

ar ow(i), j x j

=

j=1

=

=

r X j=1 r X j=1 r X

ar ow(i),col( j) xcol( j)

ar ow(i),col( j)

r X

z j,n+k br ow(k)

k=1

br ow(k)

k=1

r X

ar ow(i),col( j) z j,n+k

j=1

= br ow(i) . Since the other rows of A with indices not in {r ow(1), . . . , r ow(r )} are linear combinations of these, either x satisfies Ax = b or no vector satisfies this system of equations.

4.4 The Ellipsoid Method In this section we describe the so-called ellipsoid method, developed by Iudin and Nemirovskii [1976] and Shor [1977] for nonlinear optimization. Khachiyan [1979] observed that it can be modified in order to solve LPs in polynomial time. Most of our presentation is based on (Gr¨otschel, Lov´asz and Schrijver [1981]), (Bland, Goldfarb and Todd [1981]) and the book of Gr¨otschel, Lov´asz and Schrijver [1988], which is also recommended for further study. The idea of the ellipsoid method is very roughly the following. We look for either a feasible or an optimum solution of an LP. We start with an ellipsoid which we know a priori to contain the solutions (e.g. a large ball). At each iteration k, we check if the center xk of the current ellipsoid is a feasible solution. Otherwise, we take a hyperplane containing xk such that all the solutions lie on one side of this hyperplane. Now we have a half-ellipsoid which contains all solutions. We take the smallest ellipsoid completely containing this half-ellipsoid and continue.

4.4 The Ellipsoid Method

81

Deﬁnition 4.12. An ellipsoid is a set E(A, x) = {z ∈ Rn : (z − x) A−1 (z − x) ≤ 1} for some symmetric positive deﬁnite n × n-matrix A. Note that B(x, r ) := E(r 2 I, x) (with I being the n × n unit matrix) is the n-dimensional Euclidean ball with center x and radius r . The volume of an ellipsoid E(A, x) is known to be √ volume (E(A, x)) = det A volume (B(0, 1)) (see Exercise 7). Given an ellipsoid E(A, x) and a hyperplane {z : az = ax}, the smallest ellipsoid E(A , x ) containing the half-ellipsoid E = {z ∈ E(A, x) : az ≥ ax} is called the L¨owner-John ellipsoid of E (see Figure 4.1). It can be computed by the following formulas: n2 2 A = bb , A− n+1 n2 − 1 1 b, x = x + n+1 1 Aa. b = √ a Aa

{z : az = ax}

x

E(A, x) E(A , x )

Fig. 4.1.

82

4 Linear Programming Algorithms

One difficulty of the ellipsoid method is caused by the square root in the computation of b. Because we have to tolerate rounding errors, it is necessary to increase the radius of the next ellipsoid a little bit. Here is an algorithmic scheme that takes care of this problem:

E LLIPSOID M ETHOD Input: Output:

1

2

3

4

A number n ∈ N, n ≥ 2. A number N ∈ N. x0 ∈ Qn and R ∈ Q+ , R ≥ 2. An ellipsoid E(A N , x N ).

Set p := d6N + log(9n 3 )e. Set A0 := R 2 I , where I is the n × n unit matrix. Set k := 0. Choose any ak ∈ Qn \ {0}. 1 A k ak . Set bk := q > ak A k ak 1 ∗ := xk + Set xk+1 :≈ xk+1 bk . n + 1 2n 2 + 3 2 > ∗ bk bk . Ak − Set Ak+1 :≈ Ak+1 := n+1 2n 2 (Here :≈ means computing the entries up to p decimal places, taking care that Ak+1 is symmetric). Set k := k + 1. If k < N then go to 2 else stop.

So in each of the N iterations an approximation E(Ak+1 , xk+1 ) of the smallest ellipsoid containing E(Ak , xk ) ∩ {z : ak z ≥ ak xk } is computed. Two main issues, how to obtain the ak and how to choose N , will be addressed in the next section. But let us first prove some lemmas. Let ||x|| denote the Euclidean norm of vector x, while ||A|| := max{||Ax|| : ||x|| = 1} shall denote the norm of the matrix A. For symmetric matrices, ||A|| is the maximum absolute value of an eigenvalue and ||A|| = max{x > Ax : ||x|| = 1}. The first lemma says that each E k := E(Ak , xk ) is indeed an ellipsoid. Furthermore, the absolute values of the numbers involved remain smaller than R 2 2 N + 2size(x0 ) . Therefore each iteration of the E LLIPSOID M ETHOD consists of O(n 2 ) computational steps, each involving numbers with O( p + size(ak ) + size(R) + size(x0 )) bits. Lemma 4.13. (Gr¨otschel, Lov´asz and Schrijver [1981]) Let k ∈ {0, 1, . . . , N }. Then Ak is positive definite, and we have ||xk || ≤ ||x0 || + R2k ,

||Ak || ≤ R 2 2k ,

and

−2 k ||A−1 k || ≤ R 4 .

Proof: We use induction on k. For k = 0 all the statements are obvious. Assume that they are true for some k ≥ 0. By a straightforward computation one verifies

4.4 The Ellipsoid Method

that (A∗k+1 )−1

2n 2 = 2n 2 + 3

A−1 k

ak ak> 2 + > n − 1 ak A k ak

.

83

(4.2)

So (A∗k+1 )−1 is the sum of a positive definite and a positive semidefinite matrix; thus it is positive definite. Hence A∗k+1 is also positive definite. Note that for positive semidefinite matrices A and B we have ||A|| ≤ ||A + B||. Therefore 2 2n 2 + 3 11 2 k 2n 2 + 3 ∗ > − b R 2 . b ||Ak || ≤ ||Ak+1 || = ≤ A k k k 2 2 n+1 8 2n 2n Since the n × n all-one matrix has norm n, the matrix Ak+1 − A∗k+1 , each of whose entries has absolute value at most 2− p , has norm at most n2− p . We conclude ||Ak+1 || ≤ ||A∗k+1 || + ||Ak+1 − A∗k+1 || ≤

11 2 k R 2 + n2− p ≤ R 2 2k+1 8

(here we used the very rough estimate 2− p ≤ n1 ). It is well-known from linear algebra that for any symmetric positive definite n × n-matrix A there exists a symmetric positive definite matrix B with A = B B. Writing Ak = B B with B = B > we obtain s s p ak> A2k ak (Bak )> Ak (Bak ) ||Ak ak || ≤ ||Ak || ≤ R2k−1 . ||bk || = q = = > > ak A k ak (Bak ) (Bak ) a> A a k

k k

Using this (and again the induction hypothesis) we get 1 ∗ ||bk || + ||xk+1 − xk+1 || n+1 √ 1 ≤ ||x0 || + R2k + R2k−1 + n2− p ≤ ||x0 || + R2k+1 . n+1

||xk+1 || ≤ ||xk || +

Using (4.2) and ||ak ak> || = ak> ak we compute ak> ak 2n 2 2 −1 ∗ −1 (4.3) + (Ak+1 ) ≤ A k n − 1 ak> Ak ak 2n 2 + 3 ! 2n 2 2 ak> B A−1 −1 k Bak = Ak + n − 1 ak> B Bak 2n 2 + 3 2n 2 2 −1 n + 1 −1 −1 ≤ + < Ak A A k k n−1 n−1 2n 2 + 3 ≤ 3R −2 4k . Let λ be the smallest eigenvalue of Ak+1 , and let v be a corresponding eigenvector with ||v|| = 1. Then – writing A∗k+1 = CC for a symmetric matrix C – we have

84

4 Linear Programming Algorithms

λ = v > Ak+1 v = v > A∗k+1 v + v > (Ak+1 − A∗k+1 )v v > CCv + v > (Ak+1 − A∗k+1 )v −1 v > C A∗k+1 Cv −1 1 ≥ (A∗k+1 )−1 − ||Ak+1 − A∗k+1 || > R 2 4−k − n2− p ≥ R 2 4−(k+1) , 3

=

where we used 2− p ≤

1 −k 3n 4 .

Since λ > 0, Ak+1 is positive definite. Furthermore,

1 ≤ R −2 4k+1 . (Ak+1 )−1 = λ

Next we show that in each iteration the ellipsoid contains the intersection of E 0 and the previous half-ellipsoid: Lemma 4.14. For k = 0, . . . , N −1 we have E k+1 ⊇ {x ∈ E k ∩ E 0 : ak x ≥ ak xk }. Proof: Let x ∈ E k ∩ E 0 with ak x ≥ ak xk . We first compute (using (4.2)) ∗ ∗ )> (A∗k+1 )−1 (x − xk+1 ) (x − xk+1 > 2 ak ak> 2n 1 2 1 −1 = bk bk Ak + x − xk − x − xk − n +1 n −1 ak> Ak ak n +1 2n 2 +3 ak a > 2 2n 2 (x − xk )> A−1 (x − xk )> > k (x − xk ) = k (x − x k ) + 2 n−1 ak A k ak 2n + 3 > > 1 2 bk ak ak bk > −1 + bk Ak bk + n − 1 ak> Ak ak (n + 1)2 2 ak ak> bk 2(x − xk )> b + − A−1 k k n+1 n − 1 ak> Ak ak ak ak> 2 2n 2 > −1 > (x − x = ) A (x − x ) + ) (x − xk ) + (x − x k k k k n−1 ak> Ak ak 2n 2 + 3 2 2 (x − xk )> ak 2 1 q 1 + − . 1 + n−1 n+1 n−1 (n + 1)2 ak> Ak ak

Since x ∈ E k , we have (x −xk )> A−1 k (x −x k ) ≤ 1. By abbreviating t :=

ak> (x−xk ) q ak> Ak ak

we obtain ∗ ∗ (x − xk+1 )> (A∗k+1 )−1 (x − xk+1 ) ≤

2 2n 2 1 2 2 t t . − + 1 + n−1 2n 2 + 3 n2 − 1 n − 1

> −1 Since bk> A−1 k bk = 1 and bk Ak (x − x k ) = t, we have

4.4 The Ellipsoid Method

85

1 ≥ (x − xk )> A−1 k (x − x k ) 2 = (x − xk − tbk )> A−1 k (x − x k − tbk ) + t

≥ t 2, because A−1 k is positive definite. So (using ak x ≥ ak x k ) we have 0 ≤ t ≤ 1 and obtain 2n 4 ∗ ∗ (x − xk+1 )> (A∗k+1 )−1 (x − xk+1 ) ≤ . 2n 4 + n 2 − 3 It remains to estimate the rounding error ∗ ∗ Z := (x − xk+1 )> (Ak+1 )−1 (x − xk+1 ) − (x − xk+1 )> (A∗k+1 )−1 (x − xk+1 ) ∗ − xk+1 ) ≤ (x − xk+1 )> (Ak+1 )−1 (xk+1 ∗ ∗ − xk+1 )> (Ak+1 )−1 (x − xk+1 ) + (xk+1 ∗ ∗ )> (Ak+1 )−1 − (A∗k+1 )−1 (x − xk+1 ) + (x − xk+1 ≤

∗ ||x − xk+1 || ||(Ak+1 )−1 || ||xk+1 − xk+1 || ∗ ∗ +||xk+1 − xk+1 || ||(Ak+1 )−1 || ||x − xk+1 || ∗ +||x − xk+1 ||2 ||(Ak+1 )−1 || ||(A∗k+1 )−1 || ||A∗k+1 − Ak+1 ||.

Using Lemma 4.13 and x ∈ E 0 we get ||x −√ xk+1 || ≤ ||x − x0 || + ||xk+1 − x0 || ≤ ∗ || ≤ ||x − x − p ≤ R2 N +1 . We also use (4.3) R + R2 N and ||x − xk+1 k+1 || + n2 and obtain √ Z ≤ 2(R2 N +1 )(R −2 4 N )( n2− p ) + (R 2 4 N +1 )(R −2 4 N )(3R −2 4 N −1 )(n2− p ) √ = 4R −1 23N n2− p + 3R −2 26N n2− p ≤ 26N n2− p 1 ≤ , 9n 2 by definition of p. Altogether we have (x − xk+1 )> (Ak+1 )−1 (x − xk+1 ) ≤

1 2n 4 + ≤ 1. 2n 4 + n 2 − 3 9n 2

The volumes of the ellipsoids decrease by a constant factor in each iteration: Lemma 4.15. For k = 0, . . . , N − 1 we have

volume (E k+1 ) volume (E k )

1

< e− 5n .

Proof: (Gr¨otschel, Lov´asz and Schrijver [1988]) We write s s s det A∗k+1 det Ak+1 det Ak+1 volume (E k+1 ) = = volume (E k ) det Ak det Ak det A∗k+1

86

4 Linear Programming Algorithms

and estimate the two factors independently. First observe that !n det A∗k+1 2n 2 + 3 2 ak ak> Ak = det I − . det Ak n + 1 ak> Ak ak 2n 2 The matrix

ak ak> Ak ak> Ak ak

has rank one and 1 as its only nonzero eigenvalue (eigenvector

ak ). Since the determinant is the product of the eigenvalues, we conclude that !n det A∗k+1 3 2 1 2 2n 2 + 3 1 − < e 2n e− n = e− 2n , = det Ak n+1 2n 2 n where we used 1 + x ≤ e x for all x and n−1 < e−2 for n ≥ 2. n+1 For the second estimation we use (4.3) and the well-known fact that det B ≤ ||B||n for any matrix B: det Ak+1 ∗ −1 ∗ = det I + (A ) (A − A ) k+1 k+1 k+1 det A∗k+1 n ≤ I + (A∗k+1 )−1 (Ak+1 − A∗k+1 ) n ≤ ||I || + ||(A∗k+1 )−1 || ||Ak+1 − A∗k+1 || n ≤ 1 + (R −2 4k+1 )(n2− p ) n 1 ≤ 1+ 10n 2 1

≤ e 10n (we used 2− p ≤ 10n43 4 N ≤ We conclude that volume (E k+1 ) = volume (E k )

R2 ). 10n 3 4k+1

s

det A∗k+1 det Ak

s

1 1 1 det Ak+1 ≤ e− 4n e 20n = e− 5n . ∗ det Ak+1

4.5 Khachiyan’s Theorem In this section we shall prove Khachiyan’s theorem: the E LLIPSOID M ETHOD can be applied to L INEAR P ROGRAMMING in order to obtain a polynomial-time algorithm. Let us first prove that it suffices to have an algorithm for checking feasibility of linear inequality systems: Proposition 4.16. Suppose there is a polynomial-time algorithm for the following problem: “Given a matrix A ∈ Qm×n and a vector b ∈ Qm , decide if {x : Ax ≤ b} is empty.” Then there is a polynomial-time algorithm for L INEAR P ROGRAMMING which finds an optimum basic solution if there exists one.

4.5 Khachiyan’s Theorem

87

Proof: Let an LP max{cx : Ax ≤ b} be given. We first check if the primal and dual LPs are both feasible. If at least one of them is infeasible, we are done by Theorem 3.27. Otherwise, by Corollary 3.21, it is sufficient to find an element of {(x, y) : Ax ≤ b, y A = c, y ≥ 0, cx = yb}. We show (by induction on k) that a solution of a feasible system of k inequalities and l equalities can be found by k calls to the subroutine checking emptiness of polyhedra plus additional polynomial-time work. For k = 0 a solution can be found easily by G AUSSIAN E LIMINATION (Corollary 4.11). Now let k > 0. Let ax ≤ β be an inequality of the system. By a call to the subroutine we check whether the system becomes infeasible by replacing ax ≤ β by ax = β. If so, the inequality is redundant and can be removed (cf. Proposition 3.8). If not, we replace it by the equality. In both cases we reduced the number of inequalities by one, so we are done by induction. If there exists an optimum basic solution, the above procedure generates one, because the final equality system contains a maximal feasible subsystem of Ax = b. Before we can apply the E LLIPSOID M ETHOD, we have to take care that the polyhedron is bounded and full-dimensional: Proposition 4.17. (Khachiyan [1979], G´acs and Lov´asz [1981]) Let A ∈ Qm×n and b ∈ Qm . The system Ax ≤ b has a solution if and only if the system Ax ≤ b + 1l,

−R1l ≤ x ≤ R1l

has a solution, where 1l is the all-one vector, 1 = 2n24(size(A)+size(b)) and R = 1 + 24(size(A)+size(b)) . n If Ax ≤ b has a solution, then volume ({x ∈ R : Ax ≤ b + 1l, −R1l ≤ x ≤ R1l}) ≥

2 n2size(A)

n

.

Proof: The box constraints −R1l ≤ x ≤ R1l do not change the solvability by Theorem 4.4. Now suppose that Ax ≤ b has no solution. By Theorem 3.24 (a version of Farkas’ Lemma), there is a vector y ≥ 0 with y A = 0 and yb = −1. By applying Theorem 4.4 to min{1ly : y ≥ 0, A> y = 0, b> y = −1} we conclude that y can be chosen such that its components are of absolute value at most 24(size(A)+size(b)) . Therefore y(b + 1l) ≤ −1 + n24(size(A)+size(b)) ≤ − 21 . Again by Theorem 3.24, this proves that Ax ≤ b + 1l has no solution. For the second statement, if x ∈ Rn with Ax ≤ b has components of absolute value at most R−1 (cf. Theorem 4.4), then {x ∈ Rn : Ax ≤ b+1l, −R1l ≤ x ≤ R1l} . contains all points z with ||z − x||∞ ≤ n2size(A) Note that the construction of this proposition increases the size of the system of inequalities by at most a factor of O(m + n). Theorem 4.18. (Khachiyan [1979]) There exists a polynomial-time algorithm for L INEAR P ROGRAMMING (with rational input), and this algorithm finds an optimum basic solution if there exists one.

88

4 Linear Programming Algorithms

Proof: By Proposition 4.16 it suffices to check feasibility of a system Ax ≤ b. We transform the system as in Proposition 4.17 to obtain a polytope P in order which is either empty or has volume at least

2 n2size(A)

n

.

We run the E LLIPSOID M ETHOD with x0 = 0, R = n 1 + 24(size(A)+size(b)) , N = d10n 2 (2 log n + 5(size(A) + size(b)))e. Each time in 2 we check whether xk ∈ P. If yes, we are done. Otherwise we take a violated inequality ax ≤ β of the system Ax ≤ b and set ak := −a. We claim that if the algorithm does not find an xk ∈ P before iteration N , then P must be empty. To see this, we first observe that P ⊆ E k for all k: for k = 0 this is clear by the construction of P and R; the induction step is Lemma 4.14. So we have P ⊆ E N . By Lemma 4.15, we have, abbreviating s := size(A) + size(b), N

N

volume (E N ) ≤ volume (E 0 )e− 5n ≤ (2R)n e− 5n n < 2n 1 + 24s n −4n e−10ns < n −2n 2−5ns . On the other hand, P 6= ∅ implies n 1 2 n = volume (P) ≥ = n −2n 2−5ns , n2s n 2 25s which is a contradiction.

If we estimate the running time for solving an LP max{cx : Ax ≤ b} with the above method, we get the bound O((n+m)9 (size(A)+size(b)+size(c))2 ) (Exercise 9), which is polynomial but completely useless for practical purposes. In practice, either the S IMPLEX A LGORITHM or interior point algorithms are used. Karmarkar [1984] was the first to describe a polynomial-time interior point algorithm for L INEAR P ROGRAMMING. We shall not go into the details here. A strongly polynomial-time algorithm for L INEAR P ROGRAMMING is not known. However, Tardos [1986] showed that there is an algorithm for solving max{cx : Ax ≤ b} with a running time that polynomially depends on size(A) only. For many combinatorial optimization problems, where A is a 0-1-matrix, this gives a strongly polynomial-time algorithm. Tardos’ result was extended by Frank and Tardos [1987].

4.6 Separation and Optimization The above method (in particular Proposition 4.16) requires that the polyhedron be given explicitly by a list of inequalities. However, a closer look shows that this is not really necessary. It is sufficient to have a subroutine which – given a vector x – decides if x ∈ P or otherwise returns a separating hyperplane, i.e. a vector a such that ax > max{ay : y ∈ P}. We shall prove this for full-dimensional polytopes; for

4.6 Separation and Optimization

89

the general (more complicated) case we refer to Gr¨otschel, Lov´asz and Schrijver [1988] (or Padberg [1995]). The results in this section are due to Gr¨otschel, Lov´asz and Schrijver [1981] and independently to Karp and Papadimitriou [1982] and Padberg and Rao [1981]. With the results of this section one can solve certain linear programs in polynomial time although the polytope has an exponential number of facets. Many examples will be discussed later in this book; see e.g. Corollary 12.19 or Theorem 20.34. By considering the dual LP one can also deal with linear programs with a huge number of variables. Let P ⊆ Rn be a full-dimensional polytope, or more generally, a fulldimensional bounded convex set. We assume that we know the dimension n and two balls B(x0 , r ) and B(x0 , R) such that B(x0 , r ) ⊆ P ⊆ B(x0 , R). But we do not assume that we know a linear inequality system defining P. In fact, this would not make sense if we want to solve linear programs with an exponential number of constraints in polynomial time, or even optimize linear objective functions over nonlinearly constrained convex sets. Below we shall prove that, under some reasonable assumptions, we can optimize a linear function over a polyhedron P in polynomial time (independent of the number of constraints) if we have a so-called separation oracle: a subroutine for the following problem:

S EPARATION P ROBLEM Instance:

A convex set P ⊆ Rn . A vector y ∈ Qn .

Task:

Either decide that y ∈ P or find a vector d ∈ Qn such that d x < dy for all x ∈ P.

Note that such a vector d exists if P is a rational polyhedron or a compact convex set (cf. Exercise 20 of Chapter 3). Given a convex set P by such a separation oracle, we look for an oracle algorithm using this as a black box. In an oracle algorithm we may ask the oracle at any time and we get a correct answer in one step. We can regard this concept as a subroutine whose running time we do not take into account. (In Chapter 15 we shall give a formal definition.) Indeed, it often suffices to have an oracle which solves the S EPARATION P ROB LEM approximately. More precisely we assume an oracle for the following problem:

W EAK S EPARATION P ROBLEM Instance: Task:

A convex set P ⊆ Rn , a vector c ∈ Qn and a number > 0. A vector y ∈ Qn . Either find a vector y 0 ∈ P with cy ≤ cy 0 + or find a vector d ∈ Qn such that d x < dy for all x ∈ P.

Using a weak separation oracle we first solve linear programs approximately:

90

4 Linear Programming Algorithms

W EAK O PTIMIZATION P ROBLEM Instance:

A number n ∈ N. A vector c ∈ Qn . A number > 0. A convex set P ⊆ Rn given by an oracle for the W EAK S EPARATION P ROBLEM for P, c and 2 .

Task:

Find a vector y ∈ P with cy ≥ sup{cx : x ∈ P} − .

Note that the above two definitions differ from the ones given e.g. in Gr¨otschel, Lov´asz and Schrijver [1981]. However, they are basically equivalent, and we shall need the above form again in Section 18.3. The following variant of the E LLIPSOID M ETHOD solves the W EAK O PTI MIZATION P ROBLEM for bounded full-dimensional convex sets:

¨ ´ -S CHRIJVER A LGORITHM G R OTSCHEL -L OV ASZ Input:

Output:

A number n ∈ N, n ≥ 2. A vector c ∈ Qn . A number 0 < ≤ 1. A convex set P ⊆ Rn given by an oracle for the W EAK S EPARATION P ROBLEM for P, c and 2 . x0 ∈ Qn and r, R ∈ Q+ such that B(x0 , r ) ⊆ P ⊆ B(x0 , R). A vector y ∗ ∈ P with cy ∗ ≥ sup{cx : x ∈ P} − .

1

Set R := max{R, l 2},2 r m:= min{r, 1} and γ := max{||c||, 1}. 2 Set N := 5n ln 6Rr γ . Set y ∗ := x0 .

2

Run the E LLIPSOID M ETHOD, with ak in 2 being computed as follows: Run the oracle for the W EAK S EPARATION P ROBLEM with y = xk . If it returns a y 0 ∈ P with cy ≤ cy 0 + 2 then: If cy 0 > cy ∗ then set y ∗ := y 0 . Set ak := c. If it returns a d ∈ Qn with d x < dy for all x ∈ P then: Set ak := −d.

¨ ´ -S CHRIJVER A LGORITHM correctly Theorem 4.19. The G R OTSCHEL -L OV ASZ solves the W EAK O PTIMIZATION P ROBLEM for bounded full-dimensional convex sets. Its running time is bounded by O n 6 α 2 + n 4 α f (size(c), size(), n size(x0 ) + n 3 α) , where α = log Rr γ and f (size(c), size(), size(y)) is an upper bound of the running time of the oracle for the W EAK S EPARATION P ROBLEM for P with input c, , y. 2

Proof: (Gr¨otschel, Lov´asz and Schrijver [1981]) The running time in each of the N = O(n 2 α) iterations of the E LLIPSOID M ETHOD is O(n 2 (n 2 α + size(R) + size(x0 ) + q)) plus one oracle call, where q is the size of the output of the oracle. As size(y) ≤ n(size(x0 ) + size(R) + N ) by Lemma 4.13, the overall running time is O(n 4 α(n 2 α + size(x0 ) + f (size(c), size(), n size(x0 ) + n 3 α))), as stated.

4.6 Separation and Optimization

91

By Lemma 4.14, we have x ∈ P : cx ≥ cy ∗ + ⊆ EN . 2 Let z ∈ P with cz ≥ sup{cx : x ∈ P} − 6 . We may assume that cz > cy ∗ + 2 ; otherwise we are done.

r x0 U

r

z

{x : cx = cx0 }

{x : cx = cy ∗ + 2 } Fig. 4.2.

Consider the convex hull U of z and the (n −1)-dimensional ball B(x0 , r )∩{x : cx = cx0 } (see Figure 4.2). We have U ⊆ P and hence U := {x ∈ U : cx ≥ cy ∗ + 2 } is contained in E N . The volume of U is

cz − cy ∗ − 2 n volume (U ) = volume (U ) cz − cx0 n ∗ cz − cx 0 cz − cy − 2 = Vn−1r n−1 , n||c|| cz − cx0

where Vn denotes the volume of the n-dimensional unit ball. Since volume (U ) ≤ volume (E N ), and Lemma 4.15 yields N

N

volume (E N ) ≤ e− 5n E 0 = e− 5n Vn R n , we have − N ≤ e 5n2 R cz − cy − 2 ∗

Vn (cz − cx0 )n−1 n||c|| Vn−1r n−1

1 n

.

Since cz − cx0 ≤ ||c|| · ||z − x0 || ≤ ||c||R we obtain − N ≤ ||c||e 5n2 R cz − cy ∗ − 2

nVn R n−1 Vn−1r n−1

1 n

and hence cy ∗ ≥ cz − 56 ≥ sup{cx : x ∈ P} − .

< 2||c||e

−

N 5n 2

R2 ≤ r 3

92

4 Linear Programming Algorithms

Of course we are usually interested in the exact optimum. For this we restrict ourselves to rational full-dimensional polytopes. We need some assumption on the size of the vertices of the polytope. Lemma 4.20. Let n ∈ N, let P ⊆ Rn be a rational polytope, and let x0 ∈ Qn be a point in the interior of P. Let T ∈ N such that size(x0 ) ≤ log T and size(x) ≤ log T 2 for all vertices x of P. Then B(x0 , r ) ⊆ P ⊆ B(x0 , R), where r := n1 T −379n and R := 2nT . Moreover, let K := 4T 2n+1 . Let c ∈ Zn , and define c0 := K n c + (1, K , . . . , n−1 ). Then max{c0 x : x ∈ P} is attained by a unique vector x ∗ , for all other K vertices y of P we have c0 (x ∗ − y) > T −2n , and x ∗ is also an optimum solution of max{cx : x ∈ P}. Proof: For any vertex x of P we have ||x|| ≤ nT and ||x0 || ≤ nT , so ||x − x0 || ≤ 2nT and x ∈ B(x0 , R). To show that B(x0 , r ) ⊆ P, let F = {x ∈ P : ax = β} be a facet of P, where by Lemma 4.5 we may assume that size(a) + size(β) < 75n 2 log T . Suppose there is a point y ∈ F with ||y − x0 || < r . Then |ax0 − β| = |ax0 − ay| ≤ ||a|| · ||y − x0 || < n2size(a)r ≤ T −304n

2

But on the other hand the size of ax0 − β can by estimated by size(ax0 − β) ≤ 4(size(a) + size(x0 ) + size(β)) ≤ 300n 2 log T + 4 log T ≤ 304n 2 log T. 2

Since ax0 6= β (x0 is in the interior of P), this implies |ax0 − β| ≥ T −304n , a contradiction. To prove the last statements, let x ∗ be a vertex of P maximizing c0 x, and let y be another vertex of P. By the assumption on the size of the vertices of P we may write x ∗ − y = α1 z, where α ∈ {1, 2, . . . , T 2n − 1} and z is an integral vector whose components have absolute value at most K2 . Then ! n X 1 K n cz + K i−1 z i . 0 ≤ c0 (x ∗ − y) = α i=1

Pn

Since K n > i=1 K i−1 |z i |, we must have cz ≥ 0 and hence cx ∗ ≥ cy. So x ∗ indeed maximizes cx over P. Moreover, since z 6= 0, we obtain c0 (x ∗ − y) ≥ as required.

1 > T −2n , α

Theorem 4.21. Let n ∈ N and c ∈ Qn . Let P ⊆ Rn be a rational polytope, and let x0 ∈ Qn be a point in the interior of P. Let T ∈ N such that size(x0 ) ≤ log T and size(x) ≤ log T for all vertices x of P.

4.6 Separation and Optimization

93

Given n, c, x0 , T and a polynomial-time oracle for the S EPARATION P ROBLEM for P, a vertex x ∗ of P attaining max{c> x : x ∈ P} can be found in time polynomial in n, log T and size(c). ¨ Proof: (Gr¨otschel, Lov´asz and Schrijver [1981]) We first use the G R OTSCHEL ´ -S CHRIJVER A LGORITHM to solve the W EAK O PTIMIZATION P ROBLEM; L OV ASZ we set c0 , r and R according to Lemma 4.20 and := 8nT12n+3 . (We first have to make c integral by multiplying with the product of its denominators; this increases its size by at most a factor 2n.) ¨ ´ -S CHRIJVER A LGORITHM returns a vector y ∈ P -L OV ASZ The G R OTSCHEL with c0 y ≥ c0 x ∗ −, where x ∗ is an optimum solution of max{c0 x : x ∈ P}. By Theo rem 4.19 the running time is O n 6 α 2 + n 4 α f (size(c0 ), size(), n size(x0 ) + n 3 α) = 2 0 ||,1} ≤ O n 6 α 2 + n 4 α f (size(c0 ), 6n log T, n log T + n 3 α) , where α = log R max{||c r

log(16n 5 T 400n 2size(c ) ) = O(n 2 log T + size(c0 )) and f is a polynomial upper bound of the running time of the oracle for the S EPARATION P ROBLEM for P. Since size(c0 ) ≤ 6n 2 log T + 2 size(c), we have an overall running time that is polynomial in n, log T and size(c). We claim that ||x ∗ − y|| ≤ 2T1 2 . To see this, write y as a convex combination of the vertices x ∗ , x1 , . . . , xk of P: 2

0

y = λ0 x + ∗

k X

λi xi ,

λi ≥ 0,

i=1

k X

λi = 1.

i=0

Now – using Lemma 4.20 – ≥ c0 (x ∗ − y) =

k X

λi c0 x ∗ − xi

i=1

>

k X

λi T −2n = (1 − λ0 )T −2n ,

i=1

so 1 − λ0 < T 2n . We conclude that ||y − x ∗ || ≤

k X

λi ||xi − x ∗ || ≤ (1 − λ0 )2R < 4nT 2n+1 ≤

i=1

1 . 2T 2

So when rounding each entry of y to the next rational number with denominator at most T , we obtain x ∗ . The rounding can be done in polynomial time by Theorem 4.8. We have proved that, under certain assumptions, optimizing over a polytope can be done whenever there is a separation oracle. We close this chapter by noting that the converse is also true. We need the concept of polarity: If X ⊆ Rn , we define the polar of X to be the set X ◦ := {y ∈ Rn : y > x ≤ 1 for all x ∈ X }. When applied to full-dimensional polytopes, this operation has some nice properties:

94

4 Linear Programming Algorithms

Theorem 4.22. Let P be a polytope in Rn with 0 in the interior. Then: (a) P ◦ is a polytope with 0 in the interior; (b) (P ◦ )◦ = P; (c) x is a vertex of P if and only if x > y ≤ 1 is a facet-defining inequality of P ◦ . Proof: (a): Let P be the convex hull of x1 , . . . , xk (cf. Theorem 3.31). By definition, P ◦ = {y ∈ Rn : y > xi ≤ 1 for all i ∈ {1, . . . , k}}, i.e. P ◦ is a polyhedron and the facet-defining inequalities of P ◦ are given by vertices of P. Moreover, 0 is in the interior of P ◦ because 0 satisfies all of the finitely many inequalities strictly. Suppose P ◦ is unbounded, i.e. there exists a w ∈ Rn \ {0} with αw ∈ P ◦ for all α > 0. Then αwx ≤ 1 for all α > 0 and all x ∈ P, so wx ≤ 0 for all x ∈ P. But then 0 cannot be in the interior of P. (b): Trivially, P ⊆ (P ◦ )◦ . To show the converse, suppose that z ∈ (P ◦ )◦ \ P. Then there is an inequality c> x ≤ δ satisfied by all x ∈ P but not by z. We have δ > 0 since 0 is in the interior of P. Then 1δ c ∈ P ◦ but 1δ c> z > 1, contradicting the assumption that z ∈ (P ◦ )◦ . (c): We have already seen in (a) that the facet-defining inequalities of P ◦ are given by vertices of P. Conversely, if x1 , . . . , xk are the vertices of P, then ¯ Now (b) implies P¯ := conv({ 12 x1 , x2 , . . . , xk }) 6= P, and 0 is in the interior of P. ◦ ◦ n > > ¯ P 6= P . Hence {y ∈ R : y x1 ≤ 2, y xi ≤ 1(i = 2, . . . , k)} = P¯ ◦ 6= P ◦ = {y ∈ Rn : y > xi ≤ 1(i = 1, . . . , k)}. We conclude that x1> y ≤ 1 is a facet-defining inequality of P ◦ . Now we can prove: Theorem 4.23. Let n ∈ N and y ∈ Qn . Let P ⊆ Rn be a rational polytope, and let x0 ∈ Qn be a point in the interior of P. Let T ∈ N such that size(x0 ) ≤ log T and size(x) ≤ log T for all vertices x of P. Given n, y, x0 , T and an oracle which for any given c ∈ Qn returns a vertex x ∗ of P attaining max{c> x : x ∈ P}, we can solve the S EPARATION P ROBLEM for P and y in time polynomial in n, log T and size(y). Indeed, in the case y ∈ / P we can find a facet-defining inequality of P that is violated by y. Proof: Consider Q := {x − x0 : x ∈ P} and its polar Q ◦ . If x1 , . . . , xk are the vertices of P, we have Q ◦ = {z ∈ Rn : z > (xi − x0 ) ≤ 1 for all i ∈ {1, . . . , k}}. By Theorem 4.4 we have size(z) ≤ 4n(2n log T + 3n) ≤ 20n 2 log T for all vertices z of Q ◦ . Observe that the S EPARATION P ROBLEM for P and y is equivalent to the S EPARATION P ROBLEM for Q and y − x0 . Since by Theorem 4.22 Q = (Q ◦ )◦ = {x : zx ≤ 1 for all z ∈ Q ◦ }, the S EPARATION P ROBLEM for Q and y − x0 is equivalent to solving max{(y − x0 )> x : x ∈ Q ◦ }. Since each vertex of Q ◦ corresponds to a facet-defining inequality

Exercises

95

of Q (and thus of P), it remains to show how to find a vertex attaining max{(y − x0 )> x : x ∈ Q ◦ }. To do this, we apply Theorem 4.21 to Q ◦ . By Theorem 4.22, Q ◦ is fulldimensional with 0 in the interior. We have shown above that the size of the vertices of Q ◦ is at most 20n 2 log T . So it remains to show that we can solve the S EPARATION P ROBLEM for Q ◦ in polynomial time. However, this reduces to the optimization problem for Q which can be solved using the oracle for optimizing over P. We finally mention that a new algorithm which is faster than the E LLIPSOID M ETHOD and also implies the equivalence of optimization and separation has been proposed by Vaidya [1996]. However, this algorithm does not seem to be of practical use either.

Exercises

∗

∗

1. Let A be a nonsingular rational n × n-matrix. Prove that size(A−1 ) ≤ 4n 2 size(A). 2. Let n ≥ 2, c ∈ Rn and y1 , . . . , yk ∈ {−1, 0, 1}n such that 0 < c> yi+1 ≤ 12 c> yi for i = 1, . . . , k − 1. Prove that then k ≤ 3n log n. Hint: Consider the linear program max{y1> x : yk> x = 1, (yi − 2yi+1 )> x ≥ 0 (i = 1, . . . , k − 1)} and recall the proof of Theorem 4.4. (M. Goemans) 3. Consider the numbers h i in the C ONTINUED F RACTION E XPANSION. Prove that h i ≥ Fi+1 for all i, where Fi is the i-th Fibonacci number (F1 = F2 = 1 and Fn = Fn−1 + Fn−2 for n ≥ 3). Observe that √ !n √ !n ! 1 1+ 5 1− 5 . Fn = √ − 2 2 5 Conclude that the number of iterations of the C ONTINUED F RACTION E X PANSION is O(log q). (Gr¨otschel, Lov´asz and Schrijver [1988]) 4. Show that G AUSSIAN E LIMINATION can be made a strongly polynomial-time algorithm. Hint: First assume that A is integral. Recall the proof of Theorem 4.10 and observe that we can choose d as the common denominator of the entries. (Edmonds [1967]) l d := 1 + dim{x , . . . , x }, λ , . . . , λ ∈ R 5. Let k k + with 1 1 Pk x1 , . . . , xk ∈ R , P k λ = 1, and x := λ x . Show how to compute numbers µ , . . . , µk i i i 1 i=1 i=1 Pk ∈ R+ , at most d of which are nonzero, such that µ = 1 and i=1 i Pk x = i=1 µi xi (cf. Exercise 14 of Chapter 3). Show that all computations can be performed in O((k + l)3 ) time. Hint: Run G AUSSIAN E LIMINATION with the matrix A ∈ R(l+1)×k whose

96

4 Linear Programming Algorithms

i-th column is x1i . If d < k, let w ∈ Rk be the vector with wcol(i) := z i,d+1 (i = 1, . . . , d), wcol(d+1) := −1 and wcol(i) := 0 (i = d + 2, . . . , k); observe that Aw = 0. Add a multiple of w to λ, eliminate at least one vector and iterate. > 6. Let A = αb bC ∈ Rn×n be a symmetric positive semidefinite matrix with 1 α 0 α > 0 and b ∈ Rn−1 . Let A0 := 0 C− 1 bb> and U := 1 α b . Prove that α

∗

0 I

A = U > A0 U and C − α1 bb> is positive semidefinite. Iterate and conclude that for any positive semidefinite matrix A there is a matrix U with A = U > U , and such a matrix can be computed with arbitrary precision in O(n 3 ) steps (some of which consist of computing approximate square roots). Note: This is called Cholesky factorization. It cannot be computed exactly as U can be irrational. 7. Let A be a symmetric positive definite n ×n-matrix. Let v1 , . . . , vn be n orthogonal eigenvectors of A, with corresponding eigenvalues λ1 , . . . , λn . W.l.o.g. ||vi || = 1 for i = 1, . . . , n. Prove that then o n p p E(A, 0) = µ1 λ1 v1 + · · · + µn λn vn : µ ∈ Rn , ||µ|| ≤ 1 .

8. 9. 10. ∗ 11.

12.

(The eigenvectors correspond to the√axes of symmetry of the ellipsoid.) Conclude that volume (E(A, 0)) = det A volume (B(0, 1)). Let E(A, x) ⊆ Rn be an ellipsoid and a ∈ Rn , and let E(A0 , x 0 )) be as defined on page 81. Prove that {z ∈ E(A, x) : az ≥ ax} ⊆ E(A0 , x 0 ). Prove that the algorithm of Theorem 4.18 solves a linear program max{cx : Ax ≤ b} in O((n + m)9 (size(A) + size(b) + size(c))2 ) time. Show that the assumption that P is bounded can be omitted in Theorem 4.21. One can detect if the LP is unbounded and otherwise find an optimum solution. Let P ⊆ R3 be a 3-dimensional polytope with 0 in its interior. Consider again the graph G(P) whose vertices are the vertices of P and whose edges correspond to the 1-dimensional faces of P (cf. Exercises 17 and 18 of Chapter 3). Show that G(P ◦ ) is the planar dual of G(P). Note: Steinitz [1922] proved that for every simple 3-connected planar graph G there is a 3-dimensional polytope P with G = G(P). Prove that the polar of a polyhedron is always a polyhedron. For which polyhedra P is (P ◦ )◦ = P?

References General Literature: Gr¨otschel, M., Lov´asz, L., and Schrijver, A. [1988]: Geometric Algorithms and Combinatorial Optimization. Springer, Berlin 1988 Padberg, M. [1999]: Linear Optimization and Extensions. Second edition. Springer, Berlin 1999 Schrijver, A. [1986]: Theory of Linear and Integer Programming. Wiley, Chichester 1986

References

97

Cited References: Bland, R.G., Goldfarb, D., and Todd, M.J. [1981]: The ellipsoid method: a survey. Operations Research 29 (1981), 1039–1091 Edmonds, J. [1967]: Systems of distinct representatives and linear algebra. Journal of Research of the National Bureau of Standards B 71 (1967), 241–245 ´ [1987]: An application of simultaneous Diophantine approximation Frank, A., and Tardos, E. in combinatorial optimization. Combinatorica 7 (1987), 49–65 G´acs, P., and Lov´asz, L. [1981]: Khachiyan’s algorithm for linear programming. Mathematical Programming Study 14 (1981), 61–68 Gr¨otschel, M., Lov´asz, L., and Schrijver, A. [1981]: The ellipsoid method and its consequences in combinatorial optimization. Combinatorica 1 (1981), 169–197 Iudin, D.B., and Nemirovskii, A.S. [1976]: Informational complexity and effective methods of solution for convex extremal problems. Ekonomika i Matematicheskie Metody 12 (1976), 357–369 [in Russian] Karmarkar, N. [1984]: A new polynomial-time algorithm for linear programming. Combinatorica 4 (1984), 373–395 Karp, R.M., and Papadimitriou, C.H. [1982]: On linear characterizations of combinatorial optimization problems. SIAM Journal on Computing 11 (1982), 620–632 Khachiyan, L.G. [1979]: A polynomial algorithm in linear programming [in Russian]. Doklady Akademii Nauk SSSR 244 (1979) 1093–1096. English translation: Soviet Mathematics Doklady 20 (1979), 191–194 Khintchine, A. [1956]: Kettenbr¨uche. Teubner, Leipzig 1956 Padberg, M.W., and Rao, M.R. [1981]: The Russian method for linear programming III: Bounded integer programming. Research Report 81-39, New York University 1981 Shor, N.Z. [1977]: Cut-off method with space extension in convex programming problems. Cybernetics 13 (1977), 94–96 Steinitz, E. [1922]: Polyeder und Raumeinteilungen. Enzyklop¨adie der Mathematischen Wissenschaften, Band 3 (1922), 1–139 ´ [1986]: A strongly polynomial algorithm to solve combinatorial linear programs. Tardos, E. Operations Research 34 (1986), 250–256 Vaidya, P.M. [1996]: A new algorithm for minimizing convex functions over convex sets. Mathematical Programming 73 (1996), 291–341

5 Integer Programming

In this chapter, we consider linear programs with integrality constraints:

I NTEGER P ROGRAMMING Instance:

A matrix A ∈ Zm×n and vectors b ∈ Zm , c ∈ Zn .

Task:

Find a vector x ∈ Zn such that Ax ≤ b and cx is maximum, decide that {x ∈ Zn : Ax ≤ b} = ∅, or decide that sup{cx : x ∈ Zn , Ax ≤ b} = ∞.

We do not consider mixed integer programs, i.e. linear programs with integrality constraints for only a subset of the variables. Most of the theory of linear and integer programming can be extended to mixed integer programming in a natural way.

PI P

Fig. 5.1.

Virtually all combinatorial optimization problems can be formulated as integer programs. The set of feasible solutions can be written as {x : Ax ≤ b, x ∈ Zn } for some matrix A and some vector b. The set P := {x ∈ Rn : Ax ≤ b} is a polyhedron, so let us deﬁne by PI = {x : Ax ≤ b} I the convex hull of the integral vectors in P. We call PI the integer hull of P. Obviously PI ⊆ P.

100

5 Integer Programming

If P is bounded, then PI is also a polytope by Theorem 3.31 (see Figure 5.1). Meyer [1974] proved: Theorem 5.1. For any rational polyhedron P, its integer hull PI is a rational polyhedron. Proof: Let P = {x : Ax ≤ b}. By theorem 3.29 the rational polyhedral cone C := {(x, ξ ) : x ∈ Rn , ξ ≥ 0, Ax − ξ b ≤ 0} is finitely generated. We may assume that (x1 , 1), . . . , (xk , 1), (y1 , 0), . . . , (yl , 0) generate C, where x1 , . . . , xk are rational and y1 , . . . , yl are integral (by multiplying the elements of a finite set of generators by suitable positive scalars). Consider the polytope Q :=

X k

κi xi +

l X

i=1

λi yi

: κi ≥ 0 (i = 1, . . . , k),

i=1 k X

κi = 1,

i=1

0 ≤ λi ≤ 1 (i = 1, . . . , l) . Note that Q ⊆ P. Let z 1 , . . . , z m be the integral points in Q. By theorem 3.29 the cone C 0 generated by (y1 , 0), . . . , (yl , 0), (z 1 , 1), . . . , (z m , 1) is polyhedral, i.e. can be written as {(x, ξ ) : M x + ξ b ≤ 0} for a rational matrix M and a rational vector b. We claim that PI = {x : M x ≤ −b}. Pk κi xi + To show “⊆”, let x ∈ P ∩ Zn . We have (x, 1) ∈ C, i.e. x = i=1 Pl Pk λ y for some κ . . . , κ ≥ 0 with κ = 1 and λ . . . , λ ≥ 0. Then i i k l 1 1 i=1 P i=1 i l c := i cyi is integral, and hence x − c is integral. Moreover, x − c = i=1 bλP Pk l κ x + i i i=1 i=1 (λi − bλi c)yi ∈ Q, and hence x − c = z i for some i. Thus (x, 1) = (c, 0) + (x − c, 1) ∈ C 0 and hence M x + b ≤ 0. To show be a rational vector satisfying M x ≤ −b, i.e. (x, 1) ∈ C 0 . Pl“⊇”, let x P m ThenP x = i=1 λi yi + i=1 µi z j for rational numbers λ1 . . . , λl , µ1 , . . . , µm ≥ 0 m with i=1 µi = 1. W.l.o.g. let µ1 > 0. Let δ ∈ N such that δλi ∈ N for i = 1, . . . , l P and δ ≥ µ11 . Then (z 1 + li=1 δλi yi , 1) ∈ C and hence l

X 1 z1 + δλi yi x= δ i=1

!

m X 1 + µ1 − µi z i z1 + δ

is a convex combination of integral points in P.

i=2

This does in general not hold for irrational polyhedra; see Exercise 1. Due to Theorem 5.1 we can write an instance of I NTEGER P ROGRAMMING as max{c> x : x ∈ PI } where P = {x : Ax ≤ b}.

5.1 The Integer Hull of a Polyhedron

101

We prove a generalization of Meyer’s Theorem 5.1 in Section 5.1 (Theorem 5.8). After some preparation in Section 5.2 we study conditions under which polyhedra are integral (i.e. P = PI ) in Sections 5.3 and 5.4. Note that in this case the integer linear program is equivalent to its LP relaxation (arising by omitting the integrality constraints), and can hence be solved in polynomial time. We shall encounter this situation for several combinatorial optimization problems in later chapters. In general, however, I NTEGER P ROGRAMMING is much harder than L INEAR P ROGRAMMING, and polynomial-time algorithms are not known. This is indeed not surprising since we can formulate many apparently hard problems as integer programs. Nevertheless we discuss a general method for finding the integer hull by successively cutting off parts of P \ PI in Section 5.5. Although it does not yield a polynomial-time algorithm it is a useful technique in some cases. Finally Section 5.6 contains an efficient way of approximating the optimal value of an integer linear program.

5.1 The Integer Hull of a Polyhedron As linear programs, integer programs can be infeasible or unbounded. It is not easy to decide whether PI = ∅ for a polyhedron P. But if an integer program is feasible we can decide whether it is bounded by simply considering the LP relaxation. Proposition 5.2. Let P = {x : Ax ≤ b} be some rational polyhedron whose integer hull is nonempty, and let c be some vector (not necessarily rational). Then max {cx : x ∈ P} is bounded if and only if max {cx : x ∈ PI } is bounded. Proof: Suppose max {cx : x ∈ P} is unbounded. Then Corollary 3.28 says that the system y A = c, y ≥ 0 has no solution. By Corollary 3.26 there is a vector z with cz < 0 and Az ≥ 0. Then the LP min{cz : Az ≥ 0, −1l ≤ z ≤ 1l} is feasible. Let z ∗ be an optimum basic solution of this LP. z ∗ is rational as it is a vertex of a rational polytope. Multiply z ∗ by a suitable natural number to obtain an integral vector w with Aw ≥ 0 and cw < 0. Let v ∈ PI be some integral vector. Then v − kw ∈ PI for all k ∈ N, and thus max {cx : x ∈ PI } is unbounded. The other direction is trivial. Definition 5.3. Let A be an integral matrix. A subdeterminant of A is det B for some square submatrix B of A (defined by arbitrary row and column indices). We write 4(A) for the maximum absolute value of the subdeterminants of A. Lemma 5.4. Let C = {x : Ax ≥ 0} be a polyhedral cone, where A is an integral matrix. Then C is generated by a finite set of integral vectors, each having components with absolute value at most 4(A). Proof: By Lemma 3.12, C is generated by some of the vectors y1 , . . . , yt , such that for each i, yi is the solution to a system M y = b0 where M consists of

102

5 Integer Programming

n linearly independent rows of AI and b0 = ±e j for some unit vector e j . Set z i := | det M|yi . By Cramer’s rule, z i is integral with ||z i ||∞ ≤ 4(A). Since this holds for each i, the set {z 1 , . . . , z t } has the required properties. A similar lemma will be used in the next section: Lemma 5.5. Each rational polyhedral cone C is generated by a finite set of integral vectors {a1 , . . . , at } such that each integral vector in C is a nonnegative integral combination of a1 , . . . , at . (Such a set is called a Hilbert basis for C.) Proof: Let C be generated by the integral vectors b1 , . . . , bk . Let a1 , . . . , at be all integral vectors in the polytope {λ1 b1 + . . . + λk bk : 0 ≤ λi ≤ 1 (i = 1, . . . , k)} We show that {a1 , . . . , at } is a Hilbert basis for C. They indeed generate C, because b1 , . . . , bk occur among the a1 , . . . , at . For any integral vector x ∈ C there are µ1 , . . . , µk ≥ 0 with x = µ1 b1 + . . . + µk bk = bµ1 cb1 + . . . + bµk cbk + (µ1 − bµ1 c)b1 + . . . + (µk − bµk c)bk , so x is a nonnegative integral combination of a1 , . . . , at .

An important basic fact in integer programming is that optimum integral and fractional solutions are not too far away from each other: Theorem 5.6. (Cook et al. [1986]) Let A be an integral m × n-matrix and b ∈ Rm , c ∈ Rn arbitrary vectors. Let P := {x : Ax ≤ b} and suppose that PI 6= ∅. (a) Suppose y is an optimum solution of max {cx : x ∈ P}. Then there exists an optimum integral solution z of max {cx : x ∈ PI } with ||z − y||∞ ≤ n 4(A). (b) Suppose y is a feasible integral solution of max {cx : x ∈ PI }, but not an optimal one. Then there exists a feasible integral solution z ∈ PI with cz > cy and ||z − y||∞ ≤ n 4(A). Proof: The proof is almost the same for both parts. Let first y ∈ P arbitrary. Let z ∗ ∈ P ∩ Zn be (a) an optimum solution of max {cx : x ∈ PI } (note that PI = {x : Ax ≤ bbc} I is a polyhedron by Theorem 5.1, and thus the maximum is attained) or (b) a vector with cz ∗ > cy. We split Ax ≤ b into two subsystems A1 x ≤ b1 , A2 x ≤ b2 such that A1 z ∗ ≥ A1 y and A2 z ∗ < A2 y. Then z ∗ − y belongs to the polyhedral cone C := {x : A1 x ≥ 0, A2 x ≤ 0}. C is generated by some vectors xi (i = 1, . . . , s). By Lemma 5.4, we may assume that xi is integral and ||xi ||∞ ≤ 4(A) for all i. ∗ ∗ Ps Since z − y ∈ C, there are nonnegative numbers λ1 , . . . , λs with z − y = i=1 λi x i . We may assume that at most n of the λi are nonzero.

5.1 The Integer Hull of a Polyhedron

103

For µ = (µ1 , . . . , µs ) with 0 ≤ µi ≤ λi (i = 1, . . . , s) we define ∗

z µ := z −

s X

µi xi = y +

i=1

s X

(λi − µi )xi

i=1

and observe that z µ ∈ P: the first representation of z µ implies A1 z µ ≤ A1 z ∗ ≤ b1 ; the second one implies A2 z µ ≤ A2 y ≤ b2 . Case 1: There is some i ∈ {1, . . . , s} with λi ≥ 1 and cxi > 0. Let z := y + xi . We have cz > cy, showing that this case cannot occur in case (a). In case (b), when y is integral, z is an integral solution of Ax ≤ b such that cz > cy and ||z − y||∞ = ||xi ||∞ ≤ 4(A). Case 2: For all i ∈ {1, . . . , s}, λi ≥ 1 implies cxi ≤ 0. Let z := z bλc = z ∗ −

s X

bλi c xi .

i=1

z is an integral vector of P with cz ≥ ||z − y||∞ ≤

s X

cz ∗

and

(λi − bλi c) ||xi ||∞ ≤ n 4(A).

i=1

Hence in both (a) and (b) this vector z does the job.

As a corollary we can bound the size of optimum solutions of integer programming problems: Corollary 5.7. If P = {x ∈ Qn : Ax ≤ b} is a rational polyhedron and max{cx : x ∈ PI } has an optimum solution, then it also has an optimum integral solution x with size(x) ≤ 12n(size(A) + size(b)). Proof: By Proposition 5.2 and Theorem 4.4, max{cx : x ∈ P} has an optimum solution y with size(y) ≤ 4n(size(A) + size(b)). By Theorem 5.6(a) there is an optimum solution x of max{cx : x ∈ PI } with ||x −y||∞ ≤ n 4(A). By Propositions 4.1 and 4.3 we have size(x) ≤

2 size(y) + 2n size(n 4(A))

≤ 8n(size(A) + size(b)) + 2n log n + 4n size(A) ≤ 12n(size(A) + size(b)). Theorem 5.6(b) implies the following: given any feasible solution of an integer program, optimality of a vector x can be checked simply by testing x + y for a finite set of vectors y that depend on the matrix A only. Such a finite test set (whose existence has been proved first by Graver [1975]) enables us to prove a fundamental theorem on integer programming:

104

5 Integer Programming

Theorem 5.8. (Wolsey [1981], Cook et al. [1986]) For each integral m × nmatrix A there exists an integral matrix M whose entries have absolute value at most n 2n 4(A)n , such that for each vector b ∈ Qm there exists a rational vector d with {x : Ax ≤ b} I = {x : M x ≤ d}. Proof: We may assume A 6= 0. Let C be the cone generated by the rows of A. Let L := {z ∈ Zn : ||z||∞ ≤ n4(A)}. For each K ⊆ L, consider the cone C K := C ∩ {y : zy ≤ 0 for all z ∈ K }. By the proof of Theorem 3.29 and Lemma 5.4, C K = {y : U y ≤ 0} for some integral matrix U (whose rows are generators of {x : Ax ≤ 0} and elements of K ) whose entries have absolute value at most n4(A). Hence, again by Lemma 5.4, there is a finite set G(K ) of integral vectors generating C K , each having components with absolute value at S most 4(U ) ≤ n!(n4(A))n ≤ n 2n 4(A)n . Let M be the matrix with rows K ⊆L G(K ). Since C∅ = C, we may assume that the rows of A are also rows of M. Now let b be some fixed vector. If Ax ≤ b has no solution, we can complete b to a vector d arbitrarily and have {x : M x ≤ d} ⊆ {x : Ax ≤ b} = ∅. If Ax ≤ b contains a solution, but no integral solution, we set b0 := b − A0 1l, where A0 arises from A by taking the absolute value of each entry. Then Ax ≤ b0 has no solution, since any such solution yields an integral solution of Ax ≤ b by rounding. Again, we complete b0 to d arbitrarily. Now we may assume that Ax ≤ b has an integral solution. For y ∈ C we define δ y := max {yx : Ax ≤ b, x integral} (by Corollary 3.28 this maximum is bounded for y ∈ C). It suffices to show that [ {x : Ax ≤ b} I = x : yx ≤ δ y for each y ∈ G(K ) . (5.1) K ⊆L

Here “⊆” is trivial. To show the converse, let c be any vector for which max {cx : Ax ≤ b, x integral} is bounded, and let x ∗ be a vector attaining this maximum. We show that cx ≤ cx ∗ for all x satisfying the inequalities on the right-hand side of (5.1). By Proposition 5.2 the LP max {cx : Ax ≤ b} is bounded, so by Corollary 3.28 we have c ∈ C. Let K¯ := {z ∈ L : A(x ∗ + z) ≤ b}. By definition cz ≤ 0 for all z ∈ K¯ , so c ∈ C K¯ . Thus there are nonnegative numbers λ y (y ∈ G( K¯ )) such that

5.2 Unimodular Transformations

c =

X

105

λ y y.

y∈G( K¯ )

Next we claim that

x∗

is an optimum solution for max {yx : Ax ≤ b, x integral}

for each y ∈ G( K¯ ): the contrary assumption would, by Theorem 5.6(b), yield a vector z ∈ K¯ with yz > 0, which is impossible since y ∈ C K¯ . We conclude that X X X λy δy = λ y yx ∗ = λ y y x ∗ = cx ∗ . y∈G( K¯ )

y∈G( K¯ )

y∈G( K¯ )

Thus the inequality cx ≤ cx ∗ is a nonnegative linear combination of the inequalities yx ≤ δ y for y ∈ G( K¯ ). Hence (5.1) is proved. See Lasserre [2004] for a similar result.

5.2 Unimodular Transformations In this section we shall prove two lemmas for later use. A square matrix is called unimodular if it is integral and has determinant 1 or −1. Three types of unimodular matrices will be of particular interest: For n ∈ N, p ∈ {1, . . . , n} and q ∈ {1, . . . , n} \ { p} consider the matrices (ai j )i, j∈{1,...,n} defined in one of the following ways: if i = j 6= p / { p, q} 1 1 if i = j ∈ ai j = 1 if {i, j} = { p, q} ai j = −1 if i = j = p 0 otherwise 0 otherwise if i = j 1 ai j = −1 if (i, j) = ( p, q) 0 otherwise These matrices are evidently unimodular. If U is one of the above matrices, then replacing an arbitrary matrix A (with n columns) by AU is equivalent to applying one of the following elementary column operations to A: • • •

multiply a column by −1; exchange two columns; subtract one column from another column.

A series of the above operations is called a unimodular transformation. Obviously the product of unimodular matrices is unimodular. It can be shown that a matrix is unimodular if and only if it arises from an identity matrix by a unimodular transformation (equivalently, it is the product of matrices of the above three types); see Exercise 6. Here we do not need this fact.

106

5 Integer Programming

Proposition 5.9. The inverse of a unimodular matrix is also unimodular. For each unimodular matrix U the mappings x 7→ U x and x 7→ xU are bijections on Zn . Proof: Let U be a unimodular matrix. By Cramer’s rule the inverse of a unimodular matrix is integral. Since (det U )(det U −1 ) = det(UU −1 ) = det I = 1, U −1 is also unimodular. The second statement follows directly from this. Lemma 5.10. For each rational matrix A whose rows are linearly independent there exists a unimodular matrix U such that AU has the form (B 0), where B is a nonsingular square matrix. Proof: Suppose we have found a unimodular matrix U such that ! B 0 AU = C D for some nonsingular square matrix B. (Initially U = I , D = A, and the parts B, C and 0 have no entries.) Let (δ1 , . . . , δk ) be the first P row of D. Apply unimodular transformations such k δi is minimum. W.l.o.g. δ1 ≥ δ2 ≥ · · · ≥ δk . that all δi are nonnegative and i=1 Then δ1 > 0 since the rows of A (and hence those of AU ) are linearly independent. If Pδk2 > 0, then subtracting the second column of D from the first one would decrease i=1 δi . So δ2 = δ3 = . . . = δk = 0. We can expand B by one row and one column and continue. Note that the operations applied in the proof correspond to the E UCLIDEAN A LGORITHM. The matrix B we get is in fact a lower diagonal matrix. With a little more effort one can obtain the so-called Hermite normal form of A. The following lemma gives a criterion for integral solvability of equation systems, similar to Farkas’ Lemma. Lemma 5.11. Let A be a rational matrix and b a rational column vector. Then Ax = b has an integral solution if and only if yb is an integer for each rational vector y for which y A is integral. Proof: Necessity is obvious: if x and y A are integral vectors and Ax = b, then yb = y Ax is an integer. To prove sufficiency, suppose yb is an integer whenever y A is integral. We may assume that Ax = b contains no redundant equalities, i.e. y A = 0 implies yb 6= 0 for all y 6= 0. Let m be the number of rows of A. If rank(A) < m then {y : y A = 0} contains a nonzero vector y 0 and y 00 := 2y10 b y 0 satisfies y 00 A = 0 and / Z. So the rows of A are linearly independent. y 00 b = 12 ∈ By Lemma 5.10 there exists a unimodular matrix U with AU = (B 0), where B is a nonsingular m × m-matrix. Since B −1 AU = (I 0) is an integral matrix, we have for each row y of B −1 that y AU is integral and thus by Proposition 5.9 −1 −1 y A is integral. Hence yb is an integer for each row y of B , implying that B b −1 is an integral vector. So U B 0 b is an integral solution of Ax = b.

5.3 Total Dual Integrality

107

5.3 Total Dual Integrality In this and the next section we focus on integral polyhedra: Definition 5.12. A polyhedron P is integral if P = PI . Theorem 5.13. (Hoffman [1974], Edmonds and Giles [1977]) Let P be a rational polyhedron. Then the following statements are equivalent: (a) (b) (c) (d) (e) (f)

P is integral. Each face of P contains integral vectors. Each minimal face of P contains integral vectors. Each supporting hyperplane of P contains integral vectors. Each rational supporting hyperplane of P contains integral vectors. max {cx : x ∈ P} is attained by an integral vector for each c for which the maximum is finite. (g) max {cx : x ∈ P} is an integer for each integral c for which the maximum is finite. Proof: We first prove (a)⇒(b)⇒(f)⇒(a), then (b)⇒(d)⇒(e)⇒(c)⇒(b), and finally (f)⇒(g)⇒(e). (a)⇒(b): Let F be a face, say F = P ∩ H , where H is a supporting hyperplane, and let x ∈ F. If P = PI , then x is a convex combination of integral points in P, and these must belong to H and thus to F. (b)⇒(f) follows directly from Proposition 3.4, because {y ∈ P : cy = max {cx : x ∈ P}} is a face of P for each c for which the maximum is finite. (f)⇒(a): Suppose there is a vector y ∈ P\PI . Then (since PI is a polyhedron by Theorem 5.1) there is an inequality ax ≤ β valid for PI for which ay > β. Then clearly (f) is violated, since max {ax : x ∈ P} (which is finite by Proposition 5.2) is not attained by any integral vector. (b)⇒(d) is also trivial since the intersection of a supporting hyperplane with P is a face of P. (d)⇒(e) and (c)⇒(b) are trivial. (e)⇒(c): Let P = {x : Ax ≤ b}. We may assume that A and b are integral. Let F = {x : A0 x = b0 } be a minimal face of P, where A0 x ≤ b0 is a subsystem of Ax ≤ b (we use Proposition 3.9). If A0 x = b0 has no integral solution, then – by Lemma 5.11 – there exists a rational vector y such that c := y A0 is integral but δ := yb0 is not an integer. Adding integers to components of y does not destroy this property (A0 and b0 are integral), so we may assume that all components of y are positive. Observe that H := {x : cx = δ} is a rational hyperplane which contains no integral vectors. We finally show that H is a supporting hyperplane by proving that H ∩ P = F. Since F ⊆ H is trivial, it remains to show that H ∩ P ⊆ F. But for x ∈ H ∩ P we have y A0 x = cx = δ = yb0 , so y(A0 x − b0 ) = 0. Since y > 0 and A0 x ≤ b0 , this implies A0 x = b0 , so x ∈ F. (f)⇒(g) is trivial, so we finally show (g)⇒(e). Let H = {x : cx = δ} be a rational supporting hyperplane of P, so max{cx : x ∈ P} = δ. Suppose H contains

108

5 Integer Programming

no integral vectors. Then – by Lemma 5.11 – there exists a number γ such that γ c is integral but γ δ ∈ / Z. Then max{(|γ |c)x : x ∈ P} = |γ | max{cx : x ∈ P} = |γ |δ ∈ / Z,

contradicting our assumption.

See also Gomory [1963], Fulkerson [1971] and Chv´atal [1973] for earlier partial results. By (a)⇔(b) and Corollary 3.6 every face of an integral polyhedron is integral. The equivalence of (f) and (g) of Theorem 5.13 motivated Edmonds and Giles to define TDI-systems: Definition 5.14. (Edmonds and Giles [1977]) A system Ax ≤ b of linear inequalities is called totally dual integral (TDI) if the minimum in the LP duality equation max {cx : Ax ≤ b} = min {yb : y A = c, y ≥ 0} has an integral optimum solution y for each integral vector c for which the minimum is finite. With this definition we get an easy corollary of (g)⇒(a) of Theorem 5.13: Corollary 5.15. Let Ax ≤ b be a TDI-system where A is rational and b is integral. Then the polyhedron {x : Ax ≤ b} is integral. But total dual integrality is not a property of polyhedra (cf. Exercise 8). In general, a TDI-system contains more inequalities than necessary for describing the polyhedron. Adding valid inequalities does not destroy total dual integrality: Proposition 5.16. If Ax ≤ b is TDI and ax ≤ β is a valid inequality for {x : Ax ≤ b}, then the system Ax ≤ b, ax ≤ β is also TDI. Proof: Let c be an integral vector such that min {yb + γβ : y A + γ a = c, y ≥ 0, γ ≥ 0} is finite. Since ax ≤ β is valid for {x : Ax ≤ b}, min {yb : y A = c, y ≥ 0} = max {cx : Ax ≤ b} = max {cx : Ax ≤ b, ax ≤ β} = min {yb + γβ : y A + γ a = c, y ≥ 0, γ ≥ 0}. The first minimum is attained by some integral vector y ∗ , so y = y ∗ , γ = 0 is an integral optimum solution for the second minimum. Theorem 5.17. (Giles and Pulleyblank [1979]) For each rational polyhedron P there exists a rational TDI-system Ax ≤ b with A integral and P = {x : Ax ≤ b}. Here b can be chosen to be integral if and only if P is integral.

5.3 Total Dual Integrality

109

Proof: Let P = {x : C x ≤ d} with C and d integral. W.l.o.g., P 6= ∅. For each minimal face F of P let K F := {c : cz = max {cx : x ∈ P} for all z ∈ F}. By Corollary 3.22 and Theorem 3.29, K F is a rational polyhedral cone. By Lemma 5.5 there exists an integral Hilbert basis a1 , . . . , at generating K F . Let S F be the system of inequalities a1 x ≤ max {a1 x : x ∈ P} , . . . , at x ≤ max {at x : x ∈ P}. Let Ax ≤ b be the collection of all these systems S F (for all minimal faces F). Note that if P is integral then b is integral. Moreover, P ⊆ {x : Ax ≤ b}. Let c be an integral vector for which max {cx : x ∈ P} is finite. The set of vectors attaining this maximum is a face of P, so let F be a minimal face such that cz = max {cx : x ∈ P} for all z ∈ F. Let S F be the system a1 x ≤ β1 , . . . , at x ≤ βt . Then c = λ1 a1 + · · · + λt at for some nonnegative integers λ1 , . . . , λt . We add zero ¯ = c and components to λ1 , . . . , λt in order to get an integral vector λ¯ ≥ 0 with λA ¯ ¯ ¯ ¯ ¯ thus cx = (λA)x = λ(Ax) ≤ λb = λ(Az) = (λA)z = cz for all x with Ax ≤ b and all z ∈ F. Applying this for each row c of C yields C x ≤ d for all x with Ax ≤ b; and thus P = {x : Ax ≤ b}. Moreover, for general c we conclude that λ¯ is an optimum solution of the dual LP min {yb : y ≥ 0, y A = c}. Hence Ax ≤ b is TDI. If P is integral, we have chosen b to be integral. Conversely, if b can be chosen integral, by Corollary 5.15 P must be integral. Indeed, for full-dimensional rational polyhedra there is a unique minimal TDIsystem describing it (Schrijver [1981]). For later use, we prove that each “face” of a TDI-system is again TDI: Theorem 5.18. (Cook [1983]) Let Ax ≤ b, ax ≤ β be a TDI-system, where a is integral. Then the system Ax ≤ b, ax = β is also TDI. Proof: (Schrijver [1986]) Let c be an integral vector such that max {cx : Ax ≤ b, ax = β} = min {yb + (λ − µ)β : y, λ, µ ≥ 0, y A + (λ − µ)a = c}

(5.2)

is finite. Let x ∗ , y ∗ , λ∗ , µ∗ attain these optima. We set c0 := c +dµ∗ ea and observe that max {c0 x : Ax ≤ b, ax ≤ β} = min {yb + λβ : y, λ ≥ 0, y A + λa = c0 } (5.3) is finite, because x := x ∗ is feasible for the maximum and y := y ∗ , λ := λ∗ + dµ∗ e − µ∗ is feasible for the minimum. Since Ax ≤ b, ax ≤ β is TDI, the minimum in (5.3) has an integral optimum solution y˜ , λ˜ . We finally set y := y˜ , λ := λ˜ and µ := dµ∗ e and claim that (y, λ, µ) is an integral optimum solution for the minimum in (5.2).

110

5 Integer Programming

Obviously (y, λ, µ) is feasible for the minimum in (5.2). Furthermore, ˜ − dµ∗ eβ yb + (λ − µ)β = y˜ b + λβ ∗ ≤ y b + (λ∗ + dµ∗ e − µ∗ )β − dµ∗ eβ ˜ is an since (y ∗ , λ∗ + dµ∗ e − µ∗ ) is feasible for the minimum in (5.3), and ( y˜ , λ) optimum solution. We conclude that yb + (λ − µ)β ≤ y ∗ b + (λ∗ − µ∗ )β, proving that (y, λ, µ) is an integral optimum solution for the minimum in (5.2). The following statements are straightforward consequences of the definition of TDI-systems: A system Ax = b, x ≥ 0 is TDI if min {yb : y A ≥ c} has an integral optimum solution y for each integral vector c for which the minimum is finite. A system Ax ≤ b, x ≥ 0 is TDI if min {yb : y A ≥ c, y ≥ 0} has an integral optimum solution y for each integral vector c for which the minimum is finite. One may ask whether there are matrices A such that Ax ≤ b, x ≥ 0 is TDI for each integral vector b. It will turn out that these matrices are exactly the totally unimodular matrices.

5.4 Totally Unimodular Matrices Definition 5.19. A matrix A is totally unimodular if each subdeterminant of A is 0, +1, or −1. In particular, each entry of a totally unimodular matrix must be 0, +1, or −1. The main result of this section is: Theorem 5.20. (Hoffman and Kruskal [1956]) An integral matrix A is totally unimodular if and only if the polyhedron {x : Ax ≤ b, x ≥ 0} is integral for each integral vector b. Proof: Let A be an m × n-matrix and P := {x : Ax ≤ b, x ≥ 0}. Observe that the minimal faces of P are vertices. To prove necessity, suppose that A is totally unimodular. Let b be some integral vector and x a vertex of P. x is the solution of A0 x = b0 for some subsystem A x ≤ b , with A0 being a nonsingular n × n-matrix. Since A is A0 x ≤ b0 of −I 0 totally unimodular, | det A0 | = 1, so by Cramer’s rule x = (A0 )−1 b0 is integral. We now prove sufficiency. Suppose that the vertices of P are integral for each integral vector b. Let A0 be some nonsingular k × k-submatrix of A. We have to show | det A0 | = 1. W.l.o.g., A0 contains the elements of the first k rows and columns of A. Consider the integral m × m-matrix B consisting of the first k and the last m − k columns of (A I ) (see Figure 5.2). Obviously, | det B| = | det A0 |.

5.4 Totally Unimodular Matrices n−k

k A

k

k

m−k

I

0

111

(A I ) m−k

0

0

z

I

0

z

Fig. 5.2.

To prove | det B| = 1, we shall prove that B −1 is integral. Since det B det B −1 = 1, this implies that | det B| = 1, and we are done. Let i ∈ {1, . . . , m}; we prove that B −1 ei is integral. Choose an integral vector y such that z := y + B −1 ei ≥ 0. Then b := Bz = By + ei is integral. We add zero components to z in order to obtain z with A I z = Bz = b. Now z , consisting of the ﬁrst n components of z , belongs to P. Furthermore, n linearly independent constraints are satisﬁed with equality, namely the ﬁrst k and the last n − k inequalities of A b z ≤ . −I 0 Hence z is a vertex of P. By our assumption z is integral. But then z must also be integral: its ﬁrst n components are the components of z , and the last m components are the slack variables b − Az (and A and b are integral). So z is also integral, and hence B −1 ei = z − y is integral. The above proof is due to Veinott and Dantzig [1968]. Corollary 5.21. An integral matrix A is totally unimodular if and only if for all integral vectors b and c both optima in the LP duality equation max {cx : Ax ≤ b, x ≥ 0} = min {yb : y ≥ 0, y A ≥ c} are attained by integral vectors (if they are ﬁnite).

112

5 Integer Programming

Proof: This follows from the Hoffman-Kruskal Theorem 5.20 by using the fact that the transpose of a totally unimodular matrix is also totally unimodular. Let us reformulate these statements in terms of total dual integrality: Corollary 5.22. An integral matrix A is totally unimodular if and only if the system Ax ≤ b, x ≥ 0 is TDI for each vector b. Proof: If A (and thus A> ) is totally unimodular, then by the Hoffman-Kruskal Theorem min {yb : y A ≥ c, y ≥ 0} is attained by an integral vector for each vector b and each integral vector c for which the minimum is finite. In other words, the system Ax ≤ b, x ≥ 0 is TDI for each vector b. To show the converse, suppose Ax ≤ b, x ≥ 0 is TDI for each integral vector b. Then by Corollary 5.15, the polyhedron {x : Ax ≤ b, x ≥ 0} is integral for each integral vector b. By Theorem 5.20 this means that A is totally unimodular. This is not the only way how total unimodularity can be used to prove that a certain system is TDI. The following lemma contains another proof technique; this will be used several times later (Theorems 6.13, 19.10 and 14.12). Lemma 5.23. Let Ax ≤ b, x ≥ 0 be an inequality system, where A ∈ Rm×n and b ∈ Rm . Suppose that for each c ∈ Zn for which min{yb : y A ≥ c, y ≥ 0} has an optimum solution, it has one y ∗ such that the rows of A corresponding to nonzero components of y ∗ form a totally unimodular matrix. Then Ax ≤ b, x ≥ 0 is TDI. Proof: Let c ∈ Zn , and let y ∗ be an optimum solution of min{yb : y A ≥ c, y ≥ 0} such that the rows of A corresponding to nonzero components of y ∗ form a totally unimodular matrix A0 . We claim that min{yb : y A ≥ c, y ≥ 0} = min{yb0 : y A0 ≥ c, y ≥ 0},

(5.4)

where b0 consists of the components of b corresponding to the rows of A0 . To see the inequality “≤” of (5.4), observe that the LP on the right-hand side arises from the LP on the left-hand side by setting some variables to zero. The inequality “≥” follows from the fact that y ∗ without zero components is a feasible solution for the LP on the right-hand side. Since A0 is totally unimodular, the second minimum in (5.4) has an integral optimum solution (by the Hoffman-Kruskal Theorem 5.20). By filling this solution with zeros we obtain an integral optimum solution to the first minimum in (5.4), completing the proof. A very useful criterion for total unimodularity is the following: Theorem 5.24. (Ghouila-Houri [1962]) A matrix A = (ai j ) ∈ Zm×n is totally . unimodular if and only if for every R ⊆ {1, . . . , m} there is a partition R = R1 ∪ R2 such that

5.4 Totally Unimodular Matrices

X i∈R1

ai j −

X

113

ai j ∈ {−1, 0, 1}

i∈R2

for all j = 1, . . . , n. Proof: Let A be totally unimodular, and let R ⊆ {1, . . . , m}. Let dr := 1 for A> r ∈ R and dr := 0 for r ∈ {1, . . . , m} \ R. The matrix −A> is also totally I

unimodular, so by Theorem 5.20 the polytope 1 1 dA , xA ≥ d A , x ≤ d, x ≥ 0 x : xA ≤ 2 2 is integral. Moreover it is nonempty because it contains 12 d. So it has an integral vertex, say z. Setting R1 := {r ∈ R : zr = 0} and R2 := {r ∈ R : zr = 1} we obtain X X ai j − ai j = (d − 2z)A ∈ {−1, 0, 1}n , i∈R1

i∈R2

1≤ j≤n

as required. We now prove the converse. By induction on k we prove that every k × ksubmatrix has determinant 0, 1 or −1. For k = 1 this is directly implied by the criterion for |R| = 1. Now let k > 1, and let B = (bi j )i, j∈{1,...,k} be a nonsingular k × k-submatrix of B0 0 A. By Cramer’s rule, each entry of B −1 is det det B , where B arises from B by replacing a column by a unit vector. By the induction hypothesis, det B 0 ∈ {−1, 0, 1}. So B ∗ := (det B)B −1 is a matrix with entries −1, 0, 1 only. Let b1∗ be the first row of B ∗ . We have b1∗ B = (det B)e1 , where e1 is the first ∗ 6= 0}. Then for j = 2, . . . , k we have 0 = (b∗ B) = unit vector. Let R := {i : b1i j 1 P ∗ b b , so |{i ∈ R : b = 6 0}| is even. i j i j i∈R 1i . P ∪ R2 with Pi∈R1 bi j − By the hypothesis there is a partition R = R 1 P for all j. SoPfor j = 2, . . . , k we have i∈R1 bi j − Pi∈R2 bi j ∈ {−1, 0, 1} P b = 0. If also b − b = 0, then the sum of the rows i∈R2 i j i∈R1 i1 i∈R2 i1 in R1 equals the sum of the rows in R2 , contradicting the assumption that B is nonsingular P (becausePR 6= ∅). So i∈R1 bi1 − i∈R2 bi1 ∈ {−1, 1} and we have y B ∈ {e1 , −e1 }, where if i ∈ R1 1 yi := −1 if i ∈ R2 . 0 if i 6∈ R Since b1∗ B = (det B)e1 and B is nonsingular, we have b1∗ ∈ {(det B)y, −(det B)y}. Since both y and b1∗ are nonzero vectors with entries −1, 0, 1 only, this implies that | det B| = 1. We apply this criterion to the incidence matrices of graphs:

114

5 Integer Programming

Theorem 5.25. The incidence matrix of an undirected graph G is totally unimodular if and only if G is bipartite. Proof: By Theorem 5.24 the incidence matrix M of G is totally unimodular . if and only if for any X ⊆ V (G) there is a partition X = A ∪ B such that E(G[A]) = E(G[B]) = ∅. By definition, such a partition exists iff G[X ] is bipartite. Theorem 5.26. The incidence matrix of any digraph is totally unimodular. Proof: Using Theorem 5.24, it suffices to set R1 := R and R2 := ∅ for any R ⊆ V (G). Applications of Theorems 5.25 and 5.26 will be discussed in later chapters. Theorem 5.26 has an interesting generalization to cross-free families: Definition 5.27. Let G be a digraph and F a family of subsets of V (G). The one-way cut-incidence matrix of F is the matrix M = (m X,e ) X ∈F , e∈E(G) where ( 1 if e ∈ δ + (X ) m X,e = . 0 if e ∈ / δ + (X ) The two-way cut-incidence matrix of F where −1 m X,e = 1 0

is the matrix M = (m X,e ) X ∈F , e∈E(G) if e ∈ δ − (X ) if e ∈ δ + (X ) . otherwise

Theorem 5.28. Let G be a digraph and (V (G), F) a cross-free set system. Then the two-way cut-incidence matrix of F is totally unimodular. If F is laminar, then also the one-way cut-incidence matrix of F is totally unimodular. Proof: Let F be some cross-free family of subsets of V (G). We first consider the case when F is laminar. We use Theorem 5.24. To see that the criterion is satisfied, let R ⊆ F, and consider the tree-representation (T, ϕ) of R, where T is an arborescence rooted at r (Proposition 2.14). With the notation of Definition 2.13, R = {Se : e ∈ E(T )}. Set R1 := {S(v,w) ∈ R : distT (r, w) even} and R2 := R \ R1 . Now for any edge f ∈ E(G), the edges e ∈ E(T ) with f ∈ δ + (Se ) form a path Pf in T (possibly of zero length). So |{X ∈ R1 : f ∈ δ + (X )}| − |{X ∈ R2 : f ∈ δ + (X )}| ∈ {−1, 0, 1}, as required for the one-way cut-incidence matrix. Moreover, for any edge f the edges e ∈ E(T ) with f ∈ δ − (Se ) form a path Q f in T . Since Pf and Q f have a common endpoint, we have

5.5 Cutting Planes

115

|{X ∈ R1 : f ∈ δ + (X )}| − |{X ∈ R2 : f ∈ δ + (X )}| −|{X ∈ R1 : f ∈ δ − (X )}| + |{X ∈ R2 : f ∈ δ − (X )}| ∈ {−1, 0, 1}, as required for the two-way cut-incidence matrix. Now if (V (G), F) is a general cross-free set system, consider F 0 := {X ∈ F : r 6∈ X } ∪ {V (G) \ X : X ∈ F, r ∈ X } for some fixed r ∈ V (G). F 0is laminar. Since the two-way cut-incidence matrix M , where M is the two-way cut-incidence matrix of of F is a submatrix of −M 0 F , it is totally unimodular, too. For general cross-free families the one-way cut-incidence matrix is not totally unimodular; see Exercise 13. For a necessary and sufficient condition, see Schrijver [1983]. The two-way cut-incidence matrices of cross-free families are also known as network matrices (Exercise 14). Seymour [1980] showed that all totally unimodular matrices can be constructed in a certain way from these network matrices and two other totally unimodular matrices. This deep result implies a polynomial-time algorithm which decides whether a given matrix is totally unimodular (see Schrijver [1986]).

5.5 Cutting Planes In the previous sections we considered integral polyhedra. For general polyhedra P we have P ⊃ PI . If we want to solve an integer linear program max {cx : x ∈ PI }, it is a natural idea to cut off certain parts of P such that the resulting set is again a polyhedron P 0 and we have P ⊃ P 0 ⊃ PI . Hopefully max {cx : x ∈ P 0 } is attained by an integral vector; otherwise we can repeat this cutting-off procedure for P 0 in order to obtain P 00 and so on. This is the basic idea behind the cutting plane method, first proposed for a special problem (the TSP) by Dantzig, Fulkerson and Johnson [1954]. Gomory [1958, 1963] found an algorithm which solves general integer programs with the cutting plane method. In this section we restrict ourselves to the theoretical background. Gomory’s algorithm does not run in polynomial time and has little practical relevance in its original form. However, the general idea of cutting plane methods is used very often and successfully in practice. We shall discuss this in Section 21.6. The following presentation is mainly based on Schrijver [1986]. Definition 5.29. Let P = {x : Ax ≤ b} be a polyhedron. Then we define \ HI , P 0 := P⊆H

where the intersection ranges over all rational affine half-spaces H = {x : cx ≤ δ} 0 containing P. We set P (0) := P and P (i+1) := P (i) . P (i) is called the i-th Gomory-Chv´atal-truncation of P.

116

5 Integer Programming

For a rational polyhedron P we obviously have P ⊇ P 0 ⊇ P (2) ⊇ · · · ⊇ PI and PI = (P 0 ) I . Proposition 5.30. For any rational polyhedron P = {x : Ax ≤ b}, P 0 = {x : u Ax ≤ bubc for all u ≥ 0 with u A integral }. Proof: We first make two observations. For any rational affine half-space H = {x : cx ≤ δ} with c integral we obviously have H 0 = H I ⊆ {x : cx ≤ bδc}.

(5.5)

If in addition the components of c are relatively prime, we claim that H 0 = H I = {x : cx ≤ bδc}.

(5.6)

To prove (5.6), let c be an integral vector whose components are relatively prime. By Lemma 5.11 the hyperplane {x : cx = bδc} contains an integral vector y. For any rational vector x ∈ {x : cx ≤ bδc} let α ∈ N such that αx is integral. Then we can write x =

α−1 1 (αx − (α − 1)y) + y, α α

i.e. x is a convex combination of integral points in H . Hence x ∈ H I , implying (5.6). We now turn to the main proof. To see “⊆”, observe that for any u ≥ 0, {x : u Ax ≤ ub} is a half-space containing P, so by (5.5) P 0 ⊆ {x : u Ax ≤ bubc} if u A is integral. We now prove “⊇”. For P = ∅ this is easy, so we assume P 6= ∅. Let H = {x : cx ≤ δ} be some rational affine half-space containing P. W.l.o.g. c is integral and the components of c are relatively prime. We observe that δ ≥ max {cx : Ax ≤ b} = min {ub : u A = c, u ≥ 0}. Now let u ∗ be any optimum solution for the minimum. Then for any z ∈ {x : u Ax ≤ bubc for all u ≥ 0 with u A integral } ⊆ {x : u ∗ Ax ≤ bu ∗ bc} we have: cz = u ∗ Az ≤ bu ∗ bc ≤ bδc which, using (5.6), implies z ∈ H I .

Below we shall prove that for any rational polyhedron P there is a number t with PI = P (t) . So Gomory’s cutting plane method successively solves the linear programs over P, P 0 , P 00 , and so on, until the optimum is integral. At each step only a finite number of new inequalities have to be added, namely those corresponding to a TDI-system defining the current polyhedron (recall Theorem 5.17):

5.5 Cutting Planes

117

Theorem 5.31. (Schrijver [1980]) Let P = {x : Ax ≤ b} be a polyhedron with Ax ≤ b TDI, A integral and b rational. Then P 0 = {x : Ax ≤ bbc}. In particular, for any rational polyhedron P, P 0 is a polyhedron again. Proof: The statement is trivial if P is empty, so let P 6= ∅. Obviously P 0 ⊆ {x : Ax ≤ bbc}. To show the other inclusion, let u ≥ 0 be a vector with u A integral. By Proposition 5.30 it suffices to show that u Ax ≤ bubc for all x with Ax ≤ bbc. We know that ub ≥ max {u Ax : Ax ≤ b} = min {yb : y ≥ 0, y A = u A}. Since Ax ≤ b is TDI, the minimum is attained by some integral vector y ∗ . Now Ax ≤ bbc implies u Ax = y ∗ Ax ≤ y ∗ bbc ≤ by ∗ bc ≤ bubc. The second statement follows from Theorem 5.17.

To prove the main theorem of this section, we need two more lemmas: Lemma 5.32. If F is a face of a rational polyhedron P, then F 0 = P 0 ∩ F. More generally, F (i) = P (i) ∩ F for all i ∈ N. Proof: Let P = {x : Ax ≤ b} with A integral, b rational, and Ax ≤ b TDI (recall Theorem 5.17). Now let F = {x : Ax ≤ b, ax = β} be a face of P, where ax ≤ β is a valid inequality for P with a and β integral. By Proposition 5.16, Ax ≤ b, ax ≤ β is TDI, so by Theorem 5.18, Ax ≤ b, ax = β is also TDI. As β is an integer, P 0 ∩ F = {x : Ax ≤ bbc, ax = β} = {x : Ax ≤ bbc, ax ≤ bβc, ax ≥ dβe} = F 0. Here we have used Theorem 5.31 twice. Note that F 0 is either empty or a face of P 0 . Now the statement follows by induction on i: For each i, F (i) is either empty or a face of P (i) , and F (i) = P (i) ∩ F (i−1) = P (i) ∩ (P (i−1) ∩ F) = P (i) ∩ F. Lemma 5.33. Let P be a polyhedron in Rn and U a unimodular n × n-matrix. Write f (P) := {U x : x ∈ P}. Then f (P) is again a polyhedron. Moreover, if P is a rational polyhedron, then ( f (P))0 = f (P 0 ) and ( f (P)) I = f (PI ). Proof: Since f : Rn → Rn , x 7→ U x is a bijective linear function, the first statement is obviously true. Since also the restrictions of f and f −1 to Zn are bijections (by Proposition 5.9) we have

118

5 Integer Programming

( f (P)) I

= conv({y ∈ Zn : y = U x, x ∈ P}) = conv({y ∈ Rn : y = U x, x ∈ P, x ∈ Zn }) = conv({y ∈ Rn : y = U x, x ∈ PI }) =

f (PI ).

Let P = {x : Ax ≤ b} with Ax ≤ b TDI, A integral, b rational (cf. Theorem 5.17). Then by definition AU −1 x ≤ b is also TDI. Using Theorem 5.31 twice we get ( f (P))0 = {x : AU −1 x ≤ b}0 = {x : AU −1 x ≤ bbc} = f (P 0 ). Theorem 5.34. (Schrijver [1980]) For each rational polyhedron P there exists a number t such that P (t) = PI . Proof: Let P be a rational polyhedron in Rn . We prove the theorem by induction on n + dim P. The case P = ∅ is trivial, the case dim P = 0 is easy. First suppose that P is not full-dimensional. Then P ⊆ K for some rational hyperplane K . If K contains no integral vectors, K = {x : ax = β} for some integral vector a and some non-integer β (by Lemma 5.11). But then P 0 ⊆ {x : ax ≤ bβc, ax ≥ dβe} = ∅ = PI . If K contains integral vectors, say K = {x : ax = β} with a integral, β an integer, we may assume β = 0, because the theorem is invariant under translations by integral vectors. By Lemma 5.10 there exists a unimodular matrix U with aU = αe1 . Since the theorem is also invariant under the transformation x 7→ U −1 x (by Lemma 5.33), we may assume a = αe1 . Then the first component of each vector in P is zero, and thus we can reduce the dimension of the space by one and apply the induction hypothesis (observe that ({0} × Q) I = {0} × Q I and ({0} × Q)(t) = {0} × Q (t) for any polyhedron Q in Rn−1 and any t ∈ N). Let now P = {x : Ax ≤ b} be full-dimensional, and w.l.o.g. A integral. By Theorem 5.1 there is some integral matrix C and some vector d with PI = {x : C x ≤ d}. In the case PI = ∅ we set C := A and d := b− A0 1l, where A0 arises from A by taking the absolute value of each entry. (Note that {x : Ax ≤ b − A0 1l} = ∅.) Let cx ≤ δ be an inequality of C x ≤ d. We claim that P (s) ⊆ H := {x : cx ≤ δ} for some s ∈ N. This claim obviously implies the theorem. First observe that there is some β ≥ δ such that P ⊆ {x : cx ≤ β}: in the case PI = ∅ this follows from the choice of C and d; in the case PI 6= ∅ this follows from Proposition 5.2. Suppose our claim is false, i.e. there is an integer γ with δ < γ ≤ β for which there exists an s0 ∈ N with P (s0 ) ⊆ {x : cx ≤ γ }, but there is no s ∈ N with P (s) ⊆ {x : cx ≤ γ − 1}. Observe that max{cx : x ∈ P (s) } = γ for all s ≥ s0 , because if max{cx : x ∈ (s) P } < γ for some s, then P (s+1) ⊆ {x : cx ≤ γ − 1}.

5.6 Lagrangean Relaxation

119

Let F := P (s0 ) ∩ {x : cx = γ }. F is a face of P (s0 ) , and dim F < n = dim P. By the induction hypothesis, there is a number s1 such that F (s1 ) = FI ⊆ PI ∩ {x : cx = γ } = ∅. By applying Lemma 5.32 to F and P (s0 ) we obtain ∅ = F (s1 ) = P (s0 +s1 ) ∩ F = P (s0 +s1 ) ∩ {x : cx = γ }. Hence max{cx : x ∈ P (s0 +s1 ) } < γ , a contradiction.

This theorem also implies the following: Theorem 5.35. (Chv´atal [1973]) For each polytope P there is a number t such that P (t) = PI . Proof: As P is bounded, there exists some rational polytope Q ⊇ P with Q I = PI (take a hypercube containing P and intersect it with a rational half-space containing P but not z for each integral point z which belongs to the hypercube but not to P; cf. Exercise 19 of Chapter 3). By Theorem 5.34, Q (t) = Q I for some t. Hence PI ⊆ P (t) ⊆ Q (t) = Q I = PI , implying P (t) = PI . This number t is called the Chv´atal rank of P. If P is neither bounded nor rational, one cannot have an analogous theorem: see Exercises 1 and 17. A more efficient algorithm which computes the integer hull of a two-dimensional polyhedron has been found by Harvey [1999]. A version of the cutting plane method which, in polynomial time, approximates a linear objective function over an integral polytope given by a separation oracle was described by Boyd [1997]. Cook, Kannan and Schrijver [1990] generalized the Gomory-Chv´atal procedure to mixed integer programming. Eisenbrand [1999] showed that it is coNP-complete to decide whether a given rational vector is in P 0 for a given rational polyhedron P.

5.6 Lagrangean Relaxation Suppose we have an integer linear program max{cx : Ax ≤ b, A0 x ≤ b0 , x integral} that becomes substantially easier to solve when the constraints A0 x ≤ b0 are omitted. We write Q := {x ∈ Zn : Ax ≤ b} and assume that we can optimize linear objective functions over Q (for example if conv(Q) = {x : Ax ≤ b}). Lagrangean relaxation is a technique to get rid of some troublesome constraints (in our case A0 x ≤ b0 ). Instead of explicitly enforcing the constraints we modify the objective function in order to punish infeasible solutions. More precisely, instead of optimizing max{c> x : A0 x ≤ b0 , x ∈ Q} (5.7) we consider, for any vector λ ≥ 0,

120

5 Integer Programming

L R(λ) := max{c> x + λ> (b0 − A0 x) : x ∈ Q}.

(5.8)

For each λ ≥ 0, L R(λ) is an upper bound for (5.7) which is relatively easy to compute. (5.8) is called the Lagrangean relaxation of (5.7), and the components of λ are called Lagrange multipliers. Lagrangean relaxation is a useful technique in nonlinear programming; but here we restrict ourselves to (integer) linear programming. Of course one is interested in as good an upper bound as possible. Observe that λ 7→ L R(λ) is a convex function. The following procedure (called subgradient optimization) can be used to minimize L R(λ): Start with an arbitrary vector λ(0) ≥ 0. In iteration i, given λ(i) , find a vector (i) x maximizing c> x + (λ(i) )> (b0 − A0 x) over Q (i.e. compute L R(λ(i) )). Note that L R(λ) − L R(λ(i) ) ≥ (λ − λ(i) )> (b0 − A0 x (i) ) for all λ, i.e. b0 − A0 x (i) is a − A0 x (i) )} for some subgradient of L R at λ(i) . Set λ(i+1) := max{0, λ(i) − ti (b0 P ∞ ti = ∞, then ti > 0. Polyak [1967] showed that if limi→∞ ti = 0 and i=0 (i) limi→∞ L R(λ ) = min{L R(λ) : λ ≥ 0}. For more results on the convergence of subgradient optimization, see (Goffin [1977]). The problem to find the best such upper bound, i.e. min{L R(λ) : λ ≥ 0}, is sometimes called the Lagrangean dual of (5.7). We will show that the minimum is always attained unless {x : Ax ≤ b, A0 x ≤ b0 } = ∅. The second question that we address is how good this upper bound is. Of course this depends on the structure of the original problem. In Section 21.5 we shall meet an application to the TSP, where Lagrangean relaxation is very effective. The following theorem helps to estimate the quality of the upper bound: Theorem 5.36. (Geoffrion [1974]) Let c ∈ Rn , A0 ∈ Rm×n and b0 ∈ Rm . Let Q ⊆ Rn such that conv(Q) is a polyhedron. Suppose that max{c> x : A0 x ≤ b0 , x ∈ conv(Q)} has an optimum solution. Let L R(λ) := max{c> x + λ> (b0 − A0 x) : x ∈ Q}. Then inf{L R(λ) : λ ≥ 0} (the optimum value of the Lagrangean dual of max{c> x : A0 x ≤ b0 , x ∈ Q}) is attained by some λ, and the minimum is equal to max{c> x : A0 x ≤ b0 , x ∈ conv(Q)}. Proof: Let conv(Q) = {x : Ax ≤ b}. By using the LP Duality Theorem 3.20 twice and reformulating we get = = = = = = =

max{c> x : x ∈ conv(Q), A0 x ≤ b0 } max{c> x : Ax ≤ b, A0 x ≤ b0 } min λ> b0 + y > b : y > A + λ> A0 = c> , y ≥ 0, λ ≥ 0 min λ> b0 + min{y > b : y > A = c> − λ> A0 , y ≥ 0} : λ ≥ 0 min λ> b0 + max{(c> − λ> A0 )x : Ax ≤ b} : λ ≥ 0 min max{c> x + λ> (b0 − A0 x) : x ∈ conv(Q)} : λ ≥ 0 min max{c> x + λ> (b0 − A0 x) : x ∈ Q} : λ ≥ 0 min{L R(λ) : λ ≥ 0}.

Exercises

The third line, an LP, shows that there is a λ attaining the minimum.

121

In particular, if we have an integer linear program max{cx : A0 x ≤ b0 , Ax ≤ b, x integral} where {x : Ax ≤ b} is integral, then the Lagrangean dual (when relaxing A0 x ≤ b0 as above) yields the same upper bound as the standard LP relaxation max{cx : A0 x ≤ b0 , Ax ≤ b}. If {x : Ax ≤ b} is not integral, the upper bound is in general stronger (but can be difficult to compute). See Exercise 21 for an example. Lagrangean relaxation can also be used to approximate linear programs. For example, consider the J OB A SSIGNMENT P ROBLEM (see (1.1) in Section 1.3). The problem can be rewritten equivalently as X xi j ≥ ti (i = 1, . . . , n), (x, T ) ∈ P min T : (5.9) j∈Si

where P is the polytope (x, T ) : 0 ≤ xi j ≤ ti (i = 1, . . . , n, j ∈ Si ), X

xi j ≤ T ( j = 1, . . . , m),

i: j∈Si

T ≤

n X

ti

.

i=1

Now we apply Lagrangean relaxation and consider n X X λi ti − xi j : (x, T ) ∈ P . L R(λ) := min T + i=1

(5.10)

j∈Si

Because of its special structure this LP can be solved by a simple combinatorial algorithm (Exercise 23), for arbitrary λ. If we let Q be the set of vertices of P (cf. Corollary 3.32), then we can apply Theorem 5.36 and conclude that the optimum value of the Lagrangean dual max{L R(λ) : λ ≥ 0} equals the optimum of (5.9).

Exercises n √ o 1. Let P := (x, y) ∈ R2 : y ≤ 2x . Prove that PI is not a polyhedron. Show a polyhedron P where even the closure of PI is not a polyhedron. 2. Let P = {x ∈ Rk+l : Ax ≤ b} be a rational polyhedron. Show that conv(P ∩ (Zk × Rl )) is a polyhedron. Hint: Generalize the proof of Theorem 5.1. Note: This is the basis of mixed integer programming. See Schrijver [1986].

122

∗

∗

5 Integer Programming

3. Prove the following integer analogue of Carath´eodory’s theorem (Exercise 14 of Chapter 3): For each pointed polyhedral cone C = {x ∈ Qn : Ax ≤ 0}, each Hilbert basis {a1 , . . . , at } of C, and each integral point x ∈ C there are 2n − 1 vectors among a1 , . . . , at such that x is a nonnegative integer combination of those. Hint: Consider an optimum basic solution of the LP max{y1l : y A = x, y ≥ 0} and round the components down. Note: The number 2n − 1 was improved by Seb˝o [1990] to 2n − 2. It cannot be improved to less than b 76 nc (Bruns et al. [1999]). (Cook, Fonlupt and Schrijver [1986]) 4. Let C = {x : Ax ≥ 0} be a rational polyhedral cone and b some vector with bx > 0 for all x ∈ C \ {0}. Show that there exists a unique minimal integral Hilbert basis generating C. (Schrijver [1981]) 5. Let A be an integral m × n-matrix, and let b and c be vectors, and y an optimum solution of max {cx : Ax ≤ b, x integral}. Prove that there exists an optimum solution z of max {cx : Ax ≤ b} with ||y − z||∞ ≤ n4(A). (Cook et al. [1986]) 6. Prove that each unimodular matrix arises from an identity matrix by a unimodular transformation. Hint: Recall the proof of Lemma 5.10. 7. Prove that there is a polynomial-time algorithm which, given an integral matrix A and an integral vector b, finds an integral vector x with Ax = b or decides that none exists. Hint: See the proofs of Lemma 5.10 and Lemma 5.11. 8. Consider the two systems ! ! ! ! 0 1 1 x1 1 1 0 x1 . ≤ 0 and ≤ 1 0 x2 x2 0 1 −1 0 1 −1 They clearly define the same polyhedron. Prove that the first one is TDI but the second one is not. 9. Let a 6= 0 be an integral vector and β a rational number. Prove that the inequality ax ≤ β is TDI if and only if the components of a are relatively prime. 10. Let Ax ≤ b be TDI, k ∈ N and α > 0 rational. Show that k1 Ax ≤ αb is again TDI. Moreover, prove that α Ax ≤ αb is not necessarily TDI. 11. Use Theorem 5.25 in order to prove K¨onig’s Theorem 10.2 (cf. Exercise 2 of Chapter 11): The maximum cardinality of a matching in a bipartite graph equals the minimum cardinality of a vertex cover. 1 11

12. Show that A = −1 1 0 is not totally unimodular, but {x : Ax = b} is integral 1 00 for all integral vectors b. (Nemhauser and Wolsey [1988])

Exercises

123

13. Let G be the digraph ({1, 2, 3, 4}, {(1, 3), (2, 4), (2, 1), (4, 1), (4, 3)}), and let F := {{1, 2, 4}, {1, 2}, {2}, {2, 3, 4}, {4}}. Prove that (V (G), F) is cross-free but the one-way cut-incidence matrix of F is not totally unimodular. ∗ 14. Let G and T be digraphs such that V (G) = V (T ) and the undirected graph underlying T is a tree. For v, w ∈ V (G) let P(v, w) be the unique undirected path from v to w in T . Let M = (m f,e ) f ∈E(T ), e∈E(G) be the matrix defined by if (x, y) ∈ E(P(v, w)) and (x, y) ∈ E(P(v, y)) 1 m (x,y),(v,w) := −1 if (x, y) ∈ E(P(v, w)) and (x, y) ∈ E(P(v, x)) . 0 if (x, y) ∈ / E(P(v, w))

15.

16.

17. 18. ∗ 19.

20.

Matrices arising this way are called network matrices. Show that the network matrices are precisely the two-way cut-incidence matrices of cross-free set systems. An interval matrix is a 0-1-matrix such that in each row the 1-entries are consecutive. Prove that interval matrices are totally unimodular. Note: Hochbaum and Levin [2006] showed how to solve optimization problems with such matrices very efficiently. Consider the following interval packing problem: Given a list of intervals [ai , bi ], i = 1, . . . , n, with weights c1 , . . . , cn and a number k ∈ N, find a maximum weight subset of the intervals such that no point is contained in more than k of them. (a) Find an LP formulation (without integrality constraints) of this problem. (b) Consider the case k = 1. What combinatorial meaning has the dual LP? Show how to solve the dual LP by a simple combinatorial algorithm. (c) Use (b) to obtain an algorithm for the interval packing problem in the case k = 1 which runs in O(n log n) time. (d) Find a simple O(n log n)-algorithm for general k and unit weights. Note: See also Exercise 11 of Chapter 9. √ Let P := {(x, y) ∈ R2 : y = 2x, x ≥ 0} and Q := {(x, y) ∈ R2 : y = √ 2x}. Prove that P (t) = P 6= PI for all t ∈ N and Q 0 = R2 . Let P be the convex hull of the three points (0, 0), (0, 1) and (k, 12 ) in R2 , where k ∈ N. Show that P (2k−1) 6= PI but P (2k) = PI . Let P ⊆ [0, 1]n be a polytope in the unit hypercube with PI = ∅. Prove that then P (n) = ∅. 2 Note: Eisenbrand and Schulz [2003] proved that P (dn (1+log n)e) = PI for any polytope P ⊆ [0, 1]n . In this exercise we apply Lagrangean relaxation to linear equation systems. Let Q be a finite set of vectors in Rn , c ∈ Rn and A0 ∈ Rm×n and b0 ∈ Rm . Prove that min max{c> x + λ> (b0 − A0 x) : x ∈ Q} : λ ∈ Rm = max{c> y : y ∈ conv(Q), A0 y = b0 }.

124

5 Integer Programming

21. Consider the following facility location problem: Given a set of n customers with demands d1 , . . . , dn , and m optional facilities each of which can be opened or not. For each facility i = 1, . . . , m we have a cost f i for opening it, a capacity u i and a distance ci j to each customer j = 1, . . . , n. The task is to decide which facilities should be opened and to assign each customer to an open facility. The total demand of the customers assigned to one facility must not exceed its capacity. The objective is to minimize the facility opening costs plus the sum of the distances of each customer to its facility. In terms of I NTEGER P ROGRAMMING the problem can be formulated as X X X X min ci j xi j + f i yi : d j xi j ≤ u i yi , xi j = 1, xi j , yi ∈ {0, 1} . i, j

i

j

i

P Apply Lagrangean relaxation in two ways, once relaxing j d j xi j ≤ u i yi for P all i, then relaxing i xi j = 1 for all j. Which Lagrangean dual yields a tighter bound? Note: Both Lagrangean relaxations can be dealt with: see Exercise 7 of Chapter 17. ∗ 22. Consider the U NCAPACITATED FACILITY L OCATION P ROBLEM: given numbers n, m, f i and ci j (i = 1, . . . , m, j = 1, . . . , n), the problem can be formulated as X X X min ci j xi j + f i yi : xi j = 1, xi j ≤ yi , xi j , yi ∈ {0, 1} . i, j

i

i

For S ⊆ {1, . . . , n} we denote by c(S) the cost of supplying facilities for the customers in S, i.e. X X X ci j xi j + f i yi : xi j = 1 for j ∈ S, xi j ≤ yi , xi j , yi ∈ {0, 1} . min i, j

i

i

The cost allocation problem asks whether the total cost c({1, . . . , n}) can be distributed among the customers such that no subset SPpays more than c(S). In otherPwords: are there numbers p1 , . . . , pn such that nj=1 p j = c({1, . . . , n}) and j∈S p j ≤ c(S) for all S ⊆ {1, . . . , n}? Show that this is the case if and only if c({1, . . . , n}) equals X X X ci j xi j + f i yi : xi j = 1, xi j ≤ yi , xi j , yi ≥ 0 , min i, j

i

i

i.e. if the integrality conditions can be left out. Hint: Apply Lagrangean relaxation to the above LP. For each set of Lagrange multipliers decompose the resulting minimization problem to minimization problems over polyhedral cones. What are the vectors generating these cones? (Goemans and Skutella [2004])

References

125

23. Describe a combinatorial algorithm (without using L INEAR P ROGRAMMING) to solve (5.10) for arbitrary (but fixed) Lagrange multipliers λ. What running time can you achieve?

References General Literature: Bertsimas, D., and Weismantel, R. [2005]: Optimization Over Integers. Dynamic Ideas, Belmont 2005 Cook, W.J., Cunningham, W.H., Pulleyblank, W.R., and Schrijver, A. [1998]: Combinatorial Optimization. Wiley, New York 1998, Chapter 6 Nemhauser, G.L., and Wolsey, L.A. [1988]: Integer and Combinatorial Optimization. Wiley, New York 1988 Schrijver, A. [1986]: Theory of Linear and Integer Programming. Wiley, Chichester 1986 Wolsey, L.A. [1998]: Integer Programming. Wiley, New York 1998

Cited References: Boyd, E.A. [1997]: A fully polynomial epsilon approximation cutting plane algorithm for solving combinatorial linear programs containing a sufficiently large ball. Operations Research Letters 20 (1997), 59–63 Bruns, W., Gubeladze, J., Henk, M., Martin, A., and Weismantel, R. [1999]: A counterexample to an integral analogue of Carath´eodory’s theorem. Journal f¨ur die Reine und Angewandte Mathematik 510 (1999), 179–185 Chv´atal, V. [1973]: Edmonds’ polytopes and a hierarchy of combinatorial problems. Discrete Mathematics 4 (1973), 305–337 Cook, W. [1983]: Operations that preserve total dual integrality. Operations Research Letters 2 (1983), 31–35 Cook, W., Fonlupt, J., and Schrijver, A. [1986]: An integer analogue of Carath´eodory’s theorem. Journal of Combinatorial Theory B 40 (1986), 63–70 ´ [1986]: Sensitivity theorems in integer Cook, W., Gerards, A., Schrijver, A., and Tardos, E. linear programming. Mathematical Programming 34 (1986), 251–264 Cook, W., Kannan, R., and Schrijver, A. [1990]: Chv´atal closures for mixed integer programming problems. Mathematical Programming 47 (1990), 155–174 Dantzig, G., Fulkerson, R., and Johnson, S. [1954]: Solution of a large-scale travelingsalesman problem. Operations Research 2 (1954), 393–410 Edmonds, J., and Giles, R. [1977]: A min-max relation for submodular functions on graphs. In: Studies in Integer Programming; Annals of Discrete Mathematics 1 (P.L. Hammer, E.L. Johnson, B.H. Korte, G.L. Nemhauser, eds.), North-Holland, Amsterdam 1977, pp. 185–204 Eisenbrand, F. [1999]: On the membership problem for the elementary closure of a polyhedron. Combinatorica 19 (1999), 297–300 Eisenbrand, F., and Schulz, A.S. [2003]: Bounds on the Chv´atal rank of polytopes in the 0/1-cube. Combinatorica 23 (2003), 245–261 Fulkerson, D.R. [1971]: Blocking and anti-blocking pairs of polyhedra. Mathematical Programming 1 (1971), 168–194

126

5 Integer Programming

Geoffrion, A.M. [1974]: Lagrangean relaxation for integer programming. Mathematical Programming Study 2 (1974), 82–114 Giles, F.R., and Pulleyblank, W.R. [1979]: Total dual integrality and integer polyhedra. Linear Algebra and Its Applications 25 (1979), 191–196 Ghouila-Houri, A. [1962]: Caract´erisation des matrices totalement unimodulaires. Comptes Rendus Hebdomadaires des S´eances de l’Acad´emie des Sciences (Paris) 254 (1962), 1192–1194 Goemans, M.X., and Skutella, M. [2004]: Cooperative facility location games. Journal of Algorithms 50 (2004), 194–214 Goffin, J.L. [1977]: On convergence rates of subgradient optimization methods. Mathematical Programming 13 (1977), 329–347 Gomory, R.E. [1958]: Outline of an algorithm for integer solutions to linear programs. Bulletin of the American Mathematical Society 64 (1958), 275–278 Gomory, R.E. [1963]: An algorithm for integer solutions of linear programs. In: Recent Advances in Mathematical Programming (R.L. Graves, P. Wolfe, eds.), McGraw-Hill, New York, 1963, pp. 269–302 Graver, J.E. [1975]: On the foundations of linear and integer programming I. Mathematical Programming 9 (1975), 207–226 Harvey, W. [1999]: Computing two-dimensional integer hulls. SIAM Journal on Computing 28 (1999), 2285–2299 Hochbaum, D.S., and Levin, A. [2006]: Optimizing over consecutive 1’s and circular 1’s constraints. SIAM Journal on Optimization 17 (2006), 311–330 Hoffman, A.J. [1974]: A generalization of max flow-min cut. Mathematical Programming 6 (1974), 352–359 Hoffman, A.J., and Kruskal, J.B. [1956]: Integral boundary points of convex polyhedra. In: Linear Inequalities and Related Systems; Annals of Mathematical Study 38 (H.W. Kuhn, A.W. Tucker, eds.), Princeton University Press, Princeton 1956, 223–246 Lasserre, J.B. [2004]: The integer hull of a convex rational polytope. Discrete & Computational Geometry 32 (2004), 129–139 Meyer, R.R. [1974]: On the existence of optimal solutions to integer and mixed-integer programming problems. Mathematical Programming 7 (1974), 223–235 Polyak, B.T. [1967]: A general method for solving extremal problems. Doklady Akademii Nauk SSSR 174 (1967), 33–36 [in Russian]. English translation: Soviet Mathematics Doklady 8 (1967), 593–597 Schrijver, A. [1980]: On cutting planes. In: Combinatorics 79; Part II; Annals of Discrete Mathematics 9 (M. Deza, I.G. Rosenberg, eds.), North-Holland, Amsterdam 1980, pp. 291–296 Schrijver, A. [1981]: On total dual integrality. Linear Algebra and its Applications 38 (1981), 27–32 Schrijver, A. [1983]: Packing and covering of crossing families of cuts. Journal of Combinatorial Theory B 35 (1983), 104–128 Seb˝o, A. [1990]: Hilbert bases, Carath´eodory’s theorem and combinatorial optimization. In: Integer Programming and Combinatorial Optimization (R. Kannan and W.R. Pulleyblank, eds.), University of Waterloo Press, 1990 Seymour, P.D. [1980]: Decomposition of regular matroids. Journal of Combinatorial Theory B 28 (1980), 305–359 Veinott, A.F., Jr., and Dantzig, G.B. [1968]. Integral extreme points. SIAM Review 10 (1968), 371–372 Wolsey, L.A. [1981]: The b-hull of an integer program. Discrete Applied Mathematics 3 (1981), 193–201

6 Spanning Trees and Arborescences

Consider a telephone company that wants to rent a subset from an existing set of cables, each of which connects two cities. The rented cables should suffice to connect all cities and they should be as cheap as possible. It is natural to model the network by a graph: the vertices are the cities and the edges correspond to the cables. By Theorem 2.4 the minimal connected spanning subgraphs of a given graph are its spanning trees. So we look for a spanning tree of minimum weight, where we say that P a subgraph T of a graph G with weights c : E(G) → R has weight c(E(T )) = e∈E(T ) c(e). This is a simple but very important combinatorial optimization problem. It is also among the combinatorial optimization problems with the longest history; the first algorithm was given by Bor˚uvka [1926a,1926b]; see Neˇsetˇril, Milkov´a and Neˇsetˇrilov´a [2001]. Compared to the D RILLING P ROBLEM which asks for a shortest path containing all vertices of a complete graph, we now look for a shortest spanning tree. Although the number of spanning trees is even bigger than the number of paths (K n contains n! n−2 different 2 Hamiltonian paths, but, by a theorem of Cayley [1889], as many as n spanning trees; see Exercise 1), the problem turns out to be much easier. In fact, a simple greedy strategy works as we shall see in Section 6.1. Arborescences can be considered as the directed counterparts of trees; by Theorem 2.5 they are the minimal spanning subgraphs of a digraph such that all vertices are reachable from a root. The directed version of the M INIMUM S PANNING T REE P ROBLEM, the M INIMUM W EIGHT A RBORESCENCE P ROBLEM, is more difficult since a greedy strategy no longer works. In Section 6.2 we show how to solve this problem. Since there are very efficient combinatorial algorithms it is not recommended to solve these problems with L INEAR P ROGRAMMING. Nevertheless it is interesting that the corresponding polytopes (the convex hull of the incidence vectors of spanning trees or arborescences; cf. Corollary 3.33) can be described in a nice way, which we shall show in Section 6.3. In Section 6.4 we prove some classical results concerning the packing of spanning trees and arborescences.

6.1 Minimum Spanning Trees In this section, we consider the following two problems:

128

6 Spanning Trees and Arborescences

M AXIMUM W EIGHT F OREST P ROBLEM Instance:

An undirected graph G, weights c : E(G) → R.

Task:

Find a forest in G of maximum weight.

M INIMUM S PANNING T REE P ROBLEM Instance:

An undirected graph G, weights c : E(G) → R.

Task:

Find a spanning tree in G of minimum weight or decide that G is not connected.

We claim that both problems are equivalent. To make this precise, we say that a problem P linearly reduces to a problem Q if there are functions f and g, each computable in linear time, such that f transforms an instance x of P to an instance f (x) of Q, and g transforms a solution of f (x) to a solution of x. If P linearly reduces to Q and Q linearly reduces to P, then both problems are called equivalent. Proposition 6.1. The M AXIMUM W EIGHT F OREST P ROBLEM and the M INI MUM S PANNING T REE P ROBLEM are equivalent. Proof: Given an instance (G, c) of the M AXIMUM W EIGHT F OREST P ROBLEM, delete all edges of negative weight, let c0 (e) := −c(e) for all e ∈ E(G 0 ), and add a minimum set F of edges (with arbitrary weight) to make the graph connected; let us call the resulting graph G 0 . Then instance (G 0 , c0 ) of the M INIMUM S PANNING T REE P ROBLEM is equivalent in the following sense: Deleting the edges of F from a minimum weight spanning tree in (G 0 , c0 ) yields a maximum weight forest in (G, c). Conversely, given an instance (G, c) of the M INIMUM S PANNING T REE P ROB LEM , let c0 (e) := K − c(e) for all e ∈ E(G), where K = 1 + maxe∈E(G) c(e). Then the instance (G, c0 ) of the M AXIMUM W EIGHT F OREST P ROBLEM is equivalent, since all spanning trees have the same number of edges (Theorem 2.4). We shall return to different reductions of one problem to another in Chapter 15. In the rest of this section we consider the M INIMUM S PANNING T REE P ROBLEM only. We start by proving two optimality conditions: Theorem 6.2. Let (G, c) be an instance of the M INIMUM S PANNING T REE P ROBLEM, and let T be a spanning tree in G. Then the following statements are equivalent: (a) T is optimum. (b) For every e = {x, y} ∈ E(G) \ E(T ), no edge on the x-y-path in T has higher cost than e. (c) For every e ∈ E(T ), e is a minimum cost edge of δ(V (C)), where C is a connected component of T − e.

6.1 Minimum Spanning Trees

129

Proof: (a)⇒(b): Suppose (b) is violated: Let e = {x, y} ∈ E(G) \ E(T ) and let f be an edge on the x-y-path in T with c( f ) > c(e). Then (T − f ) + e is a spanning tree with lower cost. (b)⇒(c): Suppose (c) is violated: let e ∈ E(T ), C a connected component of T − e and f = {x, y} ∈ δ(V (C)) with c( f ) < c(e). Observe that the x-y-path in T must contain an edge of δ(V (C)), but the only such edge is e. So (b) is violated. (c)⇒(a): Suppose T satisfies (c), and let T ∗ be an optimum spanning tree with E(T ∗ ) ∩ E(T ) as large as possible. We show that T = T ∗ . Namely, suppose there is an edge e = {x, y} ∈ E(T ) \ E(T ∗ ). Let C be a connected component of T − e. T ∗ + e contains a circuit D. Since e ∈ E(D) ∩ δ(V (C)), at least one more edge f ( f 6= e) of D must belong to δ(V (C)) (see Exercise 9 of Chapter 2). Observe that (T ∗ + e) − f is a spanning tree. Since T ∗ is optimum, c(e) ≥ c( f ). But since (c) holds for T , we also have c( f ) ≥ c(e). So c( f ) = c(e), and (T ∗ + e) − f is another optimum spanning tree. This is a contradiction, because it has one edge more in common with T . The following “greedy” algorithm for the M INIMUM S PANNING T REE P ROB was proposed by Kruskal [1956]. It can be regarded as a special case of a quite general greedy algorithm which will be discussed in Section 13.4. In the following let n := |V (G)| and m := |E(G)|. LEM

K RUSKAL ’ S A LGORITHM Input:

A connected undirected graph G, weights c : E(G) → R.

Output:

A spanning tree T of minimum weight.

1

Sort the edges such that c(e1 ) ≤ c(e2 ) ≤ . . . ≤ c(em ).

2

Set T := (V (G), ∅).

3

For i := 1 to m do: If T + ei contains no circuit then set T := T + ei .

Theorem 6.3. K RUSKAL ’ S A LGORITHM works correctly. Proof: It is clear that the algorithm constructs a spanning tree T . It also guarantees condition (b) of Theorem 6.2, so T is optimum. The running time of K RUSKAL ’ S A LGORITHM is O(mn): the edges can be sorted in O(m log m) time (Theorem 1.5), and testing for a circuit in a graph with at most n edges can be implemented in O(n) time (just apply DFS (or BFS) and check if there is any edge not belonging to the DFS-tree). Since this is repeated m times, we get a total running time of O(m log m + mn) = O(mn). However, a more efficient implementation is possible: Theorem 6.4. K RUSKAL ’ S A LGORITHM can be implemented to run in O(m log n) time.

130

6 Spanning Trees and Arborescences

Proof: Parallel edges can be eliminated first: all but the cheapest edges are redundant. So we may assume that m = O(n 2 ). Since the running time of 1 is obviously O(m log m) = O(m log n) we concentrate on . 3 We study a data structure maintaining the connected components of T . In 3 we have to test whether the addition of an edge ei = {v, w} to T results in a circuit. This is equivalent to testing if v and w are in the same connected component. Our implementation maintains a branching B with V (B) = V (G). At any time the connected components of B will be induced by the same vertex sets as the connected components of T . (Note however that B is in general not an orientation of T .) When checking an edge ei = {v, w} in , 3 we find the root rv of the arborescence in B containing v and the root rw of the arborescence in B containing w. The time needed for this is proportional to the length of the rv -v-path plus the length of the rw -w-path in B. We shall show later that this length is always at most log n. Next we check if rv = rw . If rv 6= rw , we insert ei into T and we have to add an edge to B. Let h(r ) be the maximum length of a path from r in B. If h(rv ) ≥ h(rw ), then we add an edge (rv , rw ) to B, otherwise we add (rw , rv ) to B. If h(rv ) = h(rw ), this operation increases h(rv ) by one, otherwise the new root has the same h-value as before. So the h-values of the roots can be maintained easily. Of course initially B := (V (G), ∅) and h(v) := 0 for all v ∈ V (G). We claim that an arborescence of B with root r contains at least 2h(r ) vertices. This implies that h(r ) ≤ log n, concluding the proof. At the beginning, the claim is clearly true. We have to show that this property is maintained when adding an edge (x, y) to B. This is trivial if h(x) does not change. Otherwise we have h(x) = h(y) before the operation, implying that each of the two arborescences contains at least 2h(x) vertices. So the new arborescence rooted at x contains at least 2 · 2h(x) = 2h(x)+1 vertices, as required. The above implementation can be improved by another trick: whenever the root rv of the arborescence in B containing v has been determined, all the edges on the rv -v-path P are deleted and an edge (r x , x) is inserted for each x ∈ V (P) \ {rv }. A complicated analysis shows that this so-called path compression heuristic makes the running time of 3 almost linear: it is O(mα(m, n)), where α(m, n) is the functional inverse of Ackermann’s function (see Tarjan [1975,1983]). We now mention another well-known algorithm for the M INIMUM S PANNING T REE P ROBLEM, due to Jarn´ık [1930] (see Korte and Neˇsetˇril [2001]), Dijkstra [1959] and Prim [1957]:

P RIM ’ S A LGORITHM Input:

A connected undirected graph G, weights c : E(G) → R.

Output:

A spanning tree T of minimum weight.

1

Choose v ∈ V (G). Set T := ({v}, ∅).

6.1 Minimum Spanning Trees

2

131

While V (T ) 6= V (G) do: Choose an edge e ∈ δG (V (T )) of minimum weight. Set T := T + e.

Theorem 6.5. P RIM ’ S A LGORITHM works correctly. Its running time is O(n 2 ). Proof: The correctness follows from the fact that condition (c) of Theorem 6.2 is guaranteed. To obtain the O(n 2 ) running time, we maintain for each vertex v ∈ V (G)\V (T ) the cheapest edge e ∈ E(V (T ), {v}). Let us call these edges the candidates. The initialization of the candidates takes O(m) time. Each selection of the cheapest edge among the candidates takes O(n) time. The update of the candidates can be done by scanning the edges incident to the vertex which is added to V (T ) and thus also takes O(n) time. Since the while-loop of 2 has n − 1 iterations, the O(n 2 ) bound is proved. The running time can be improved by efficient data structures. Denote l T,v := min{c(e) : e ∈ E(V (T ), {v})}. We maintain the set {(v, l T,v ) : v ∈ V (G) \ V (T ), l T,v < ∞} in a data structure, called priority queue or heap, that allows inserting an element, finding and deleting an element (v, l) with minimum l, and decreasing the so-called key l of an element (v, l). Then P RIM ’ S A LGORITHM can be written as follows:

1

2

Choose v ∈ V (G). Set T := ({v}, ∅). Let lw := ∞ for w ∈ V (G) \ {v}. While V (T ) 6= V (G) do: For e = {v, w} ∈ E({v}, V (G) \ V (T )) do: If c(e) < lw < ∞ then set lw := c(e) and DECREASEKEY(w, lw ). If lw = ∞ then set lw := c(e) and INSERT(w, lw ). (v, lv ) := DELETEMIN. Let e ∈ E(V (T ), {v}) with c(e) = lv . Set T := T + e.

There are several possible ways to implement a heap. A very efficient way, the so-called Fibonacci heap, has been proposed by Fredman and Tarjan [1987]. Our presentation is based on Schrijver [2003]: Theorem 6.6. It is possible to maintain a data structure for a finite set (initially empty), where each element u is associated with a real number d(u), called its key, and perform any sequence of p INSERT-operations (adding an element u with key d(u)); n DELETEMIN-operations (finding and deleting an element u with d(u) minimum); • m DECREASEKEY-operations (decreasing d(u) to a specified value for an element u)

• •

in O(m + p + n log p) time.

132

6 Spanning Trees and Arborescences

Proof: The set, denoted by U , is stored in a Fibonacci heap, i.e. a branching (U, E) with a function ϕ : U → {0, 1} with the following properties: (i) If (u, v) ∈ E then d(u) ≤ d(v). (This is called the heap order.) (ii) For each u ∈ U the children of u can be numbered 1, . . . , |δ + (u)| such that the i-th child v satisfies |δ + (v)| + ϕ(v) ≥ i − 1. (iii) If u and v are distinct roots (δ − (u) = δ − (v) = ∅), then |δ + (u)| 6= |δ + (v)|. Condition (ii) implies: (iv) If a vertex u has out-degree at least k, then at least from u.

√ k 2 vertices are reachable

We prove (iv) by induction on k, the case k = 0 being trivial. So let u be a vertex with |δ + (u)| ≥ k ≥ 1, and let v be a child of u with |δ + (v)| ≥ k −2 (v exists due to (ii)). We apply the induction hypothesis to v in (U, E) and to u in (U, E \ {(u, v)}) √ k−2 √ k−1 and 2 vertices are reachable. (iv) follows and conclude that at least 2 √ k √ k−2 √ k−1 from observing that 2 ≤ 2 + 2 . In particular, (iv) implies that |δ + (u)| ≤ 2 log |U | for all u ∈ U . Thus, using (iii), we can store the roots of (U, E) by a function b : {0, 1, . . . , b2 log |U |c} → U with b(|δ + (u)|) = u for each root u. In addition to this, we keep track of a doubly-linked list of children (in arbitrary order), a pointer to the parent (if existent) and the out-degree of each vertex. We now show how the INSERT-, DELETEMIN- and DECREASEKEY-operations are implemented. INSERT (v, d(v)) is implemented by setting ϕ(v) := 0 and applying PLANT (v):

1

Set r := b(|δ + (v)|). if r is a root with r 6= v and |δ + (r )| = |δ + (v)| then: if d(r ) ≤ d(v) then add (r, v) to E and PLANT(r ). if d(v) < d(r ) then add (v, r ) to E and PLANT(v). else set b(|δ + (v)|) := v.

As (U, E) is always a branching, the recursion terminates. Note also that (i), (ii) and (iii) are maintained. DELETEMIN is implemented by scanning b(i) for i = 0, . . . , b2 log |U |c in order to find an element u with d(u) minimum, deleting u and its incident edges and successively applying PLANT(v) for each (former) child v of u. DECREASEKEY (v, (d(v)) is a bit more complicated. Let P be the longest path in (U, E) ending in v such that each internal vertex u satisfies ϕ(u) = 1. We set ϕ(u) := 1 − ϕ(u) for all u ∈ V (P) \ {v}, delete all edges of P from E and apply PLANT (z) for each deleted edge (y, z). To see that this maintains (ii) we only have to consider the parent of the start vertex x of P, if existent. But then x is not a root, and thus ϕ(x) changes from 0 to 1, making up for the lost child.

6.2 Minimum Weight Arborescences

133

We finally estimate the running time. As ϕ increases at most m times (at most once in each DECREASEKEY), ϕ decreases at most m times. Thus the sum of the length of the paths P in all DECREASEKEY-operations is at most m + m. So at most 2m + 2n log p edges are deleted overall (as each DELETEMIN-operation may delete up to 2 log p edges). Thus at most 2m + 2n log p + p − 1 edges are inserted in total. This proves the overall O(m + p + n log p) running time. Corollary 6.7. P RIM ’ S A LGORITHM implemented with Fibonacci heap solves the M INIMUM S PANNING T REE P ROBLEM in O(m + n log n) time. Proof: We have at most n−1 INSERT-, n−1 DELETEMIN-, and m DECREASEKEYoperations. With a more sophisticated implementation, the running time can be improved to O (m log β(n, m)), where β(n, m) = min i : log(i) n ≤ mn ; see Fredman and Tarjan [1987], Gabow, Galil and Spencer [1989], and Gabow et al. [1986]. The fastest known deterministic algorithm is due to Chazelle [2000] and has a running time of O(mα(m, n)), where α is the functional inverse of Ackermann’s function. On a different computational model Fredman and Willard [1994] achieved linear running time. Moreover, there is a randomized algorithm which finds a minimum weight spanning tree and has linear expected running time (Karger, Klein and Tarjan [1995]; such an algorithm which always finds an optimum solution is called a Las Vegas algorithm). This algorithm uses a (deterministic) procedure for testing whether a given spanning tree is optimum; a linear-time algorithm for this problem has been found by Dixon, Rauch and Tarjan [1992]; see also King [1995]. The M INIMUM S PANNING T REE P ROBLEM for planar graphs can be solved (deterministically) in linear time (Cheriton and Tarjan [1976]). The problem of finding a minimum spanning tree for a set of n points in the plane can be solved in O(n log n) time (Exercise 9). P RIM ’ S A LGORITHM can be quite efficient for such instances since one can use suitable data structures for finding nearest neighbours in the plane effectively.

6.2 Minimum Weight Arborescences Natural directed generalizations of the M AXIMUM W EIGHT F OREST P ROBLEM and the M INIMUM S PANNING T REE P ROBLEM read as follows:

M AXIMUM W EIGHT B RANCHING P ROBLEM Instance:

A digraph G, weights c : E(G) → R.

Task:

Find a maximum weight branching in G.

134

6 Spanning Trees and Arborescences

M INIMUM W EIGHT A RBORESCENCE P ROBLEM Instance:

A digraph G, weights c : E(G) → R.

Task:

Find a minimum weight spanning arborescence in G or decide that none exists.

Sometimes we want to specify the root in advance:

M INIMUM W EIGHT ROOTED A RBORESCENCE P ROBLEM Instance:

A digraph G, a vertex r ∈ V (G), weights c : E(G) → R.

Task:

Find a minimum weight spanning arborescence rooted at r in G or decide that none exists.

As for the undirected case, these three problems are equivalent: Proposition 6.8. The M AXIMUM W EIGHT B RANCHING P ROBLEM, the M INI MUM W EIGHT A RBORESCENCE P ROBLEM and the M INIMUM W EIGHT ROOTED A RBORESCENCE P ROBLEM are all equivalent. RBORESCENCE Proof: Given an instance (G, c) of the M INIMUM W EIGHT AP P ROBLEM, let c0 (e) := K − c(e) for all e ∈ E(G), where K = 2 e∈E(G) |c(e)|. Then the instance (G, c0 ) of the M AXIMUM W EIGHT B RANCHING P ROBLEM is equivalent, because for any two branchings B, B 0 with |E(B)| > |E(B 0 )| we have c0 (B) > c0 (B 0 ) (and branchings with n − 1 edges are exactly the spanning arborescences). Given an instance . (G, c) of the M AXIMUM W EIGHT B RANCHING P ROBLEM , let G 0 := (V (G) ∪ {r }, E(G) ∪ {(r, v) : v ∈ V (G)}). Let c0 (e) := −c(e) for e ∈ E(G) and c(e) := 0 for e ∈ E(G 0 ) \ E(G). Then the instance (G 0 , r, c0 ) of the M INIMUM W EIGHT ROOTED A RBORESCENCE P ROBLEM is equivalent. Finally, given an instance (G, r, c) of the M INIMUM W EIGHT ROOTED A R . BORESCENCE P ROBLEM , let G 0 := (V (G) ∪ {s}, E(G)∪{(s, r )}) and c((s, r )) := 0. Then the instance (G 0 , c) of the M INIMUM W EIGHT A RBORESCENCE P ROB LEM is equivalent.

In the rest of this section we shall deal with the M AXIMUM W EIGHT B RANCH P ROBLEM only. This problem is not as easy as its undirected version, the M AXIMUM W EIGHT F OREST P ROBLEM. For example any maximal forest is maximum, but the bold edges in Figure 6.1 form a maximal branching which is not maximum. Recall that a branching is a graph B with |δ − B (x)| ≤ 1 for all x ∈ V (B), such that the underlying undirected graph is a forest. Equivalently, a branching is an acyclic digraph B with |δ − B (x)| ≤ 1 for all x ∈ V (B); see Theorem 2.5(g): ING

Proposition 6.9. Let B be a digraph with |δ − B (x)| ≤ 1 for all x ∈ V (B). Then B contains a circuit if and only if the underlying undirected graph contains a circuit.

6.2 Minimum Weight Arborescences

135

Fig. 6.1.

Now let G be a digraph and c : E(G) → R+ . We can ignore negative weights since such edges will never appear in an optimum branching. A ﬁrst idea towards an algorithm could be to take the best entering edge for each vertex. Of course the resulting graph may contain circuits. Since a branching cannot contain circuits, we must delete at least one edge of each circuit. The following lemma says that one is enough. Lemma 6.10. (Karp [1972]) Let B0 be a maximum weight subgraph of G with |δ − B0 (v)| ≤ 1 for all v ∈ V (B0 ). Then there exists an optimum branching B of G such that for each circuit C in B0 , |E(C) \ E(B)| = 1.

a1

b1

C a2

b3

a3

b2 Fig. 6.2.

Proof: Let B be an optimum branching of G containing as many edges of B0 as possible. Let C be some circuit in B0 . Let E(C) \ E(B) = {(a1 , b1 ), . . . , (ak , bk )}; suppose that k ≥ 2 and a1 , b1 , a2 , b2 , a3 , . . . , bk lie in this order on C (see Figure 6.2). We claim that B contains a bi -bi−1 -path for each i = 1, . . . , k (b0 := bk ). This, however, is a contradiction because these paths form a closed edge progression in B, and a branching cannot have a closed edge progression. Let i ∈ {1, . . . , k}. It remains to show that B contains a bi -bi−1 -path. Consider B with V (B ) = V (G) and E(B ) := {(x, y) ∈ E(B) : y = bi } ∪ {(ai , bi )}. B cannot be a branching since it would be optimum and contain more edges of B0 than B. So (by Proposition 6.9) B contains a circuit, i.e. B contains a

136

6 Spanning Trees and Arborescences

bi -ai -path P. Since k ≥ 2, P is not completely on C, so let e be the last edge of P not belonging to C. Obviously e = (x, bi−1 ) for some x, so P (and thus B) contains a bi -bi−1 -path. The main idea of Edmonds’ [1967] algorithm is to find first B0 as above, and then contract every circuit of B0 in G. If we choose the weights of the resulting graph G 1 correctly, any optimum branching in G 1 will correspond to an optimum branching in G.

E DMONDS ’ B RANCHING A LGORITHM Input:

A digraph G, weights c : E(G) → R+ .

Output:

A maximum weight branching B of G.

1

Set i := 0, G 0 := G, and c0 := c.

2

Let Bi be a maximum weight subgraph of G i with |δ − Bi (v)| ≤ 1 for all v ∈ V (Bi ). If Bi contains no circuit then set B := Bi and go to . 5

3

4

5

6

Construct (G i+1 , ci+1 ) from (G i , ci ) by doing the following for each circuit C of Bi : Contract C to a single vertex vC in G i+1 For each edge e = (z, y) ∈ E(G i ) with z ∈ / V (C), y ∈ V (C) do: Let z 0 = vC 0 if z belongs to a circuit C 0 of Bi and z 0 = z otherwise. Let e0 := (z 0 , vC ) and 8(e0 ) := e. Set ci+1 (e0 ) := ci (e) − ci (α(e, C)) + ci (eC ), where α(e, C) = (x, y) ∈ E(C) and eC is some cheapest edge of C. Set i := i + 1 and go to . 2 If i = 0 then stop. For each circuit C of Bi−1 do: If there is an edge e0 = (z, vC ) ∈ E(B) then set E(B) := (E(B) \ {e0 }) ∪ 8(e0 ) ∪ (E(C) \ {α(8(e0 ), C)}) else set E(B) := E(B) ∪ (E(C) \ {eC }). Set V (B) := V (G i−1 ), i := i − 1 and go to . 5

This algorithm was also discovered independently by Chu and Liu [1965] and Bock [1971]. Theorem 6.11. (Edmonds [1967]) E DMONDS ’ B RANCHING A LGORITHM works correctly. Proof: We show that each time just before the execution of , 5 B is an optimum branching of G i . This is trivial for the first time we reach . 5 So we have to show that 6 transforms an optimum branching B of G i into an optimum branching B 0 of G i−1 .

6.3 Polyhedral Descriptions

137

∗ ∗ )| = 1 for each Let Bi−1 be any branching of G i−1 such that |E(C) \ E(Bi−1 ∗ ∗ circuit C of Bi−1 . Let Bi result from Bi−1 by contracting the circuits of Bi−1 . Bi∗ is a branching of G i . Furthermore we have X ∗ ci−1 (Bi−1 ) = ci (Bi∗ ) + (ci−1 (C) − ci−1 (eC )). C: circuit of Bi−1

By the induction hypothesis, B is an optimum branching of G i , so we have ci (B) ≥ ci (Bi∗ ). We conclude that X ∗ ci−1 (Bi−1 ) ≤ ci (B) + (ci−1 (C) − ci−1 (eC )) C: circuit of Bi−1

= ci−1 (B ). 0

This, together with Lemma 6.10, implies that B 0 is an optimum branching of G i−1 . This proof is due to Karp [1972]. Edmonds’ original proof was based on a linear programming formulation (see Corollary 6.14). The running time of E DMONDS ’ B RANCHING A LGORITHM is easily seen to be O(mn), where m = |E(G)| and n = |V (G)|: there are at most n iterations (i.e. i ≤ n at any stage of the algorithm), and each iteration can be implemented in O(m) time. The best known bound has been obtained by Gabow et al. [1986] using a Fibonacci heap: their branching algorithm runs in O(m + n log n) time.

6.3 Polyhedral Descriptions A polyhedral description of the M INIMUM S PANNING T REE P ROBLEM is as follows: Theorem 6.12. (Edmonds [1970]) Given a connected undirected graph G, n := |V (G)|, the polytope P := X X x ∈ [0, 1] E(G) : xe = n − 1, xe ≤ |X | − 1 for ∅ 6= X ⊂ V (G) e∈E(G)

e∈E(G[X ])

is integral. Its vertices are exactly the incidence vectors of spanning trees of G. (P is called the spanning tree polytope of G.) Proof: Let T be a spanning tree of G, and let x be the incidence vector of E(T ). Obviously (by Theorem 2.4), x ∈ P. Furthermore, since x ∈ {0, 1} E(G) , it must be a vertex of P. On the other hand let x be an integral vertex of P. Then x is the incidence vector of the edge set of some subgraph H with n − 1 edges and no circuit. Again by Theorem 2.4 this implies that H is a spanning tree.

138

6 Spanning Trees and Arborescences

So it suffices to show that P is integral (recall Theorem 5.13). Let c : E(G) → R, and let T be the tree produced by K RUSKAL ’ S A LGORITHM when applied to (G, c) (ties are broken arbitrarily when sorting the edges). Denote E(T ) = { f 1 , . . . , f n−1 }, where the f i were taken in this order by the algorithm. In particular, c( f 1 ) ≤ · · · ≤ c( f n−1 ). Let X k ⊆ V (G) be the connected component of (V (G), { f 1 , . . . , f k }) containing f k (k = 1, . . . , n − 1). Let x ∗ be the incidence vector of E(T ). We show that x ∗ is an optimum solution to the LP X c(e)xe min e∈E(G)

X

s.t.

e∈E(G) X

xe = n − 1 xe ≤ |X | − 1

(∅ 6= X ⊂ V (G))

xe ≥ 0

(e ∈ E(G)).

e∈E(G[X ])

We introduce a dual variable z X for each ∅ 6= X ⊂ V (G) and one additional dual variable z V (G) for the equality constraint. Then the dual LP is X max − (|X | − 1)z X ∅6= X ⊆V (G)

s.t.

−

X

zX

≤ c(e)

(e ∈ E(G))

zX

≥ 0

(∅ 6= X ⊂ V (G)).

e⊆X ⊆V (G)

Note that the dual variable z V (G) is not forced to be nonnegative. For k = 1, . . . , n − 2 let z ∗X k := c( fl ) − c( f k ), where l is the first index greater than k for which fl ∩ X k 6= ∅. Let z ∗V (G) := −c( f n−1 ), and let z ∗X := 0 for all X 6∈ {X 1 , . . . , X n−1 }. For each e = {v, w} we have that X z ∗X = c( f i ), − e⊆X ⊆V (G)

where i is the smallest index such that v, w ∈ X i . Moreover c( f i ) ≤ c(e) since v and w are in different connected components of (V (G), { f 1 , . . . , f i−1 }). Hence z ∗ is a feasible dual solution. Moreover xe∗ > 0, i.e. e ∈ E(T ), implies X − z ∗X = c(e), e⊆X ⊆V (G)

i.e. the corresponding dual constraint is satisfied with equality. Finally, z ∗X > 0 implies that T [X ] is connected, so the corresponding primal constraint is satisfied

6.3 Polyhedral Descriptions

139

with equality. In other words, the primal and dual complementary slackness conditions are satisfied, thus (by Corollary 3.23) x ∗ and z ∗ are optimum solutions for the primal and dual LP, respectively. Indeed, we have proved that the inequality system in Theorem 6.12 is TDI. We remark that the above is also an alternative proof of the correctness of K RUSKAL ’ S A LGORITHM (Theorem 6.3). Another description of the spanning tree polytope is the subject of Exercise 14. P P If we replace the constraint e∈E(G) xe = n − 1 by e∈E(G) xe ≤ n − 1, we obtain the convex hull of the incidence vectors of all forests in G (Exercise 15). A generalization of these results is Edmonds’ characterization of the matroid polytope (Theorem 13.21). We now turn to a polyhedral description of the M INIMUM W EIGHT ROOTED A RBORESCENCE P ROBLEM. First we prove a classical result of Fulkerson. Recall that an r -cut is a set of edges δ + (S) for some S ⊂ V (G) with r ∈ S. Theorem 6.13. (Fulkerson [1974]) Let G be a digraph with weights c : E(G) → Z+ , and r ∈ V (G) such that G contains a spanning arborescence rooted at r . Then the minimum weight of a spanning arborescence rooted at r equals the maximum number t of r -cuts C1 , . . . , Ct (repetitions allowed) such that no edge e is contained in more than c(e) of these cuts. Proof: Let A be the matrix whose columns are indexed by the edges and whose rows are all incidence vectors of r -cuts. Consider the LP min{cx : Ax ≥ 1l, x ≥ 0}, and its dual max{1ly : y A ≤ c, y ≥ 0}. Then (by part (e) of Theorem 2.5) we have to show that for any nonnegative integral c, both the primal and dual LP have integral optimum solutions. By Corollary 5.15 it suffices to show that the system Ax ≥ 1l, x ≥ 0 is TDI. We use Lemma 5.23. Since the dual LP is feasible if and only if c is nonnegative, let c : E(G) → Z+ . Let y be an optimum solution of max{1ly : y A ≤ c, y ≥ 0} for which X yδ − (X ) |X |2 (6.1) ∅6= X ⊆V (G)\{r }

is as large as possible. We claim that F := {X : yδ − (X ) > 0} is laminar. To see this, suppose X, Y ∈ F with X ∩ Y 6= ∅, X \ Y 6= ∅ and Y \ X 6= ∅ (Figure 6.3). Let := min{yδ − (X ) , yδ − (Y ) }. Set yδ0 − (X ) := yδ − (X ) − , yδ0 − (Y ) := yδ − (Y ) − , yδ0 − (X ∩Y ) := yδ − (X ∩Y ) + , yδ0 − (X ∪Y ) := yδ − (X ∪Y ) + , and y 0 (S) := y(S) for all other r -cuts S. Observe that y 0 A ≤ y A, so y 0 is a feasible dual solution. Since 1ly = 1ly 0 , it is also optimum and contradicts the choice of y, because (6.1) is larger for y 0 . (For any numbers a > b ≥ c > d > 0 with a + d = b + c we have a 2 + d 2 > b2 + c2 .)

140

6 Spanning Trees and Arborescences X

Y

r Fig. 6.3.

Now let A be the submatrix of A consisting of the rows corresponding to the elements of F. A is the one-way cut-incidence matrix of a laminar family (to be precise, we must consider the graph resulting from G by reversing each edge). So by Theorem 5.28 A is totally unimodular, as required. The above proof also yields the promised polyhedral description: Corollary 6.14. (Edmonds [1967]) Let G be a digraph with weights c : E(G) → R+ , and r ∈ V (G) such that G contains a spanning arborescence rooted at r . Then the LP ⎫ ⎧ ⎬ ⎨ xe ≥ 1 for all X ⊂ V (G) with r ∈ X min cx : x ≥ 0, ⎭ ⎩ + e∈δ (X )

has an integral optimum solution (which is the incidence vector of a minimum weight spanning arborescence rooted at r , plus possibly some edges of zero weight). For a description of the convex hull of the incidence vectors of all branchings or spanning arborescences rooted at r , see Exercises 16 and 17.

6.4 Packing Spanning Trees and Arborescences If we are looking for more than one spanning tree or arborescence, classical theorems of Tutte, Nash-Williams and Edmonds are of help. We ﬁrst give a proof of Tutte’s Theorem on packing spanning trees which is essentially due to Mader (see Diestel [1997]) and which uses the following lemma: Lemma 6.15. Let G be an undirected graph, and let F = (F1 , . . . , Fk ) be a k-tuple of edge-disjoint forests in G such that |E(F)| is maximum, where E(F) := k E(F i ). Let e ∈ E(G) \ E(F). Then there exists a set X ⊆ V (G) with e ⊆ X i=1 such that Fi [X ] is connected for each i ∈ {1, . . . , k}. Proof: For two k-tuples F = (F1 , . . . , Fk ) and F = (F1 , . . . , Fk ) we say that . F arises from F by exchanging e for e if Fj = (Fj \ e ) ∪ e for some j and

6.4 Packing Spanning Trees and Arborescences

141

Fi00 = Fi0 for all i 6= j. Let F be the set of all k-tuples of edge-disjoint forests arising T 0 from F by a sequence of such exchanges. Let E := E(G) \ F 0 ∈F E(F ) and G := (V (G), E). We have F ∈ F and thus e ∈ E. Let X be the vertex set of the connected component of G containing e. We shall prove that Fi [X ] is connected for each i. Claim: For any F 0 = (F10 , . . . , Fk0 ) ∈ F and any e¯ = {v, w} ∈ E(G[X ]) \ E(F 0 ) there exists a v-w-path in Fi0 [X ] for all i ∈ {1, . . . , k}. To prove this, let i ∈ {1, . . . , k} be fixed. Since F 0 ∈ F and |E(F 0 )| = |E(F)| is ¯ we have Fe00 ∈ F, maximum, Fi0 + e¯ contains a circuit C. Now for all e0 ∈ E(C)\{e} 0 0 0 ¯ This shows that E(C) ⊆ E, and where Fe0 arises from F by exchanging e for e. so C − e¯ is a v-w-path in Fi0 [X ]. This proves the claim. Since G[X ] is connected, it suffices to prove that for each e¯ = {v, w} ∈ E(G[X ]) and each i there is a v-w-path in Fi [X ]. So let e¯ = {v, w} ∈ E(G[X ]). Since e¯ ∈ E, there is some F 0 = (F10 , . . . , Fk0 ) ∈ F with e¯ 6∈ E(F 0 ). By the claim there is a v-w-path in Fi0 [X ] for each i. Now there is a sequence F = F (0) , F (1) . . . , F (s) = F 0 of elements of F such that F (r +1) arises from F (r ) by exchanging one edge (r = 0, . . . , s − 1). It suffices (r +1) [X ] implies the existence of a to show that the existence of a v-w-path in Fi (r ) v-w-path in Fi [X ] (r = 0, . . . , s − 1). (r +1) (r ) To see this, suppose that Fi [X ] arises from Fi [X ] by exchanging er for (r +1) er +1 , and let P be the v-w-path in Fi [X ]. If P does not contain er +1 = {x, y}, (r ) it is also a path in Fi [X ]. Otherwise er +1 ∈ E(G[X ]), and we consider the x-y(r ) path Q in Fi [X ] which exists by the claim. Since (E(P) \ {er +1 }) ∪ Q contains (r ) a v-w-path in Fi [X ], the proof is complete. Now we can prove Tutte’s theorem on disjoint spanning trees. A multicut in an undirected graph G is a set of edges δ(X 1 , . . . , X p ) := δ(X 1 ) ∪ · · · ∪ δ(X p ) . . . for some partition V (G) = X 1 ∪ X 2 ∪ · · · ∪ X p of the vertex set into nonempty subsets. For p = 3 we also speak of 3-cuts. Observe that cuts are multicuts with p = 2. Theorem 6.16. (Tutte [1961], Nash-Williams [1961]) An undirected graph G contains k edge-disjoint spanning trees if and only if |δ(X 1 , . . . , X p )| ≥ k( p − 1) for every multicut δ(X 1 , . . . , X p ). Proof: To prove necessity, let T1 , . . . , Tk be edge-disjoint spanning trees in G, and let δ(X 1 , . . . , X p ) be a multicut. Contracting each of the vertex subsets X 1 , . . . , X p yields a graph G 0 whose vertices are X 1 , . . . , X p and whose edges correspond to the edges of the multicut. T1 , . . . , Tk correspond to edge-disjoint connected subgraphs T10 , . . . , Tk0 in G 0 . Each of the T10 , . . . , Tk0 has at least p − 1 edges, so G 0 (and thus the multicut) has at least k( p − 1) edges.

142

6 Spanning Trees and Arborescences

To prove sufficiency we use induction on |V (G)|. For n := |V (G)| ≤ 2 the statement is true. Now assume n > 2, and suppose that |δ(X 1 , . . . , X p )| ≥ k( p −1) for every multicut δ(X 1 , . . . , X p ). In particular (consider the partition into singletons) G has at least k(n − 1) edges. Moreover, the condition is preserved when contracting vertex sets, so by the induction hypothesis G/ X contains k edge-disjoint spanning trees for each X ⊂ V (G) with |X | ≥ 2. edge-disjoint forests in G such that Let F = (F1 , . . . , Fk ) be a k-tuple of S k E(Fi ). We claim that each Fi is |E(F)| is maximum, where again E(F) := i=1 a spanning tree. Otherwise |E(F)| < k(n −1), so there is an edge e ∈ E(G)\ E(F). By Lemma 6.15 there is an X ⊆ V (G) with e ⊆ X such that Fi [X ] is connected for each i. Since |X | ≥ 2, G/ X contains k edge-disjoint spanning trees F10 , . . . , Fk0 . Now Fi0 together with Fi [X ] forms a spanning tree in G for each i, and all these k spanning trees are edge-disjoint. We now turn to the corresponding problem in digraphs, packing spanning arborescences: Theorem 6.17. (Edmonds [1973]) Let G be a digraph and r ∈ V (G). Then the maximum number of edge-disjoint spanning arborescences rooted at r equals the minimum cardinality of an r -cut. Proof: Let k be the minimum cardinality of an r -cut. Obviously there are at most k edge-disjoint spanning arborescences. We prove the existence of k edge-disjoint spanning arborescences by induction on k. The case k = 0 is trivial. If we can find one spanning arborescence A rooted at r such that + (S) \ E(A) ≥ k − 1, (6.2) min δG r ∈S⊂V (G)

then we are done by induction. Suppose we have already found some arborescence A rooted at r (but not necessarily spanning) such that (6.2) holds. Let R ⊆ V (G) be the set of vertices covered by A. Initially, R = {r }; if R = V (G), we are done. If R 6= V (G), we call a set X ⊆ V (G) critical if (a) r ∈ X ; (b) X ∪ R 6= V (G); + (X ) \ E(A)| = k − 1. (c) |δG If there is no critical vertex set, we can augment A by any edge leaving R. Otherwise let X be a maximal critical set, and let e = (x, y) be an edge such that x ∈ R \ X and y ∈ V (G) \ (R ∪ X ) (see Figure 6.4). Such an edge must exist because + + + (R ∪ X )| = |δG (R ∪ X )| ≥ k > k − 1 = |δG−E(A) (X )|. |δG−E(A)

We now add e to A. Obviously A + e is an arborescence rooted at r . We have to show that (6.2) continues to hold. + Suppose there is some Y such that r ∈ Y ⊂ V (G) and |δG (Y )\E(A+e)| < k−1. + Then x ∈ Y , y ∈ / Y , and |δG (Y ) \ E(A)| = k − 1. Now Lemma 2.1(a) implies

6.4 Packing Spanning Trees and Arborescences

143

R x

r e X

y

Fig. 6.4. + + k − 1 + k − 1 = |δG−E(A) (X )| + |δG−E(A) (Y )| + + (X ∪ Y )| + |δG−E(A) (X ∩ Y )| ≥ |δG−E(A) ≥ k−1+k−1,

because r ∈ X ∩ Y and y ∈ V (G) \ (X ∪ Y ). So equality must hold throughout, in + particular |δG−E(A) (X ∪ Y )| = k − 1. Since y ∈ V (G) \ (X ∪ Y ∪ R) we conclude that X ∪ Y is critical. But since x ∈ Y \ X , this contradicts the maximality of X . This proof is due to Lov´asz [1976]. A generalization of Theorems 6.16 and 6.17 was found by Frank [1978]. A good characterization of the problem of packing spanning arborescences with arbitrary roots is given by the following theorem, which we cite without proof: Theorem 6.18. (Frank [1979]) A digraph G contains k edge-disjoint spanning arborescences if and only if p

|δ − (X i )| ≥ k( p − 1)

i=1

for every collection of pairwise disjoint nonempty subsets X 1 , . . . , X p ⊆ V (G). Another question is how many forests are needed to cover a graph. This is answered by the following theorem: Theorem 6.19. (Nash-Williams [1964]) The edge set of an undirected graph G is the union of k forests if and only if |E(G[X ])| ≤ k(|X |−1) for all ∅ = X ⊆ V (G). Proof: The necessity is clear since no forest can contain more than |X | − 1 edges within a vertex set X . To prove the sufﬁciency, assume that |E(G[X ])| ≤ k(|X |−1) for all ∅ = X ⊆ V (G), and let F = (F1 , . . . , Fk ) be a k-tuple of disjoint forests in k E(Fi ) is maximum. We claim that E(F) = E(G). G such that |E(F)| = i=1 To see this, suppose there is an edge e ∈ E(G) \ E(F). By Lemma 6.15 there exists a set X ⊆ V (G) with e ⊆ X such that Fi [X ] is connected for each i. In particular,

144

6 Spanning Trees and Arborescences

k . [ |E(G[X ])| ≥ {e} ∪ E(Fi [X ]) ≥ 1 + k(|X | − 1), i=1

contradicting the assumption.

Exercise 22 gives a directed version. A generalization of Theorems 6.16 and 6.19 to matroids can be found in Exercise 18 of Chapter 13.

Exercises 1. Prove Cayley’s theorem, stating that K n has n n−2 spanning trees, by showing that the following defines a one-to-one correspondence between the spanning trees in K n and the vectors in {1, . . . , n}n−2 : For a tree T with V (T ) = {1, . . . , n}, n ≥ 3, let v be the leaf with the smallest index and let a1 be the neighbour of v. We recursively define a(T ) := (a1 , . . . , an−2 ), where (a2 , . . . , an−2 ) = a(T − v). (Cayley [1889], Pr¨ufer [1918]) 2. Let (V, T1 ) and (V, T2 ) be two trees on the same vertex set V . Prove that for any edge e ∈ T1 there is an edge f ∈ T2 such that both (V, (T1 \ {e}) ∪ { f }) and (V, (T2 \ { f }) ∪ {e}) are trees. 3. Given an undirected graph G with weights c : E(G) → R and a vertex v ∈ V (G), we ask for a minimum weight spanning tree in G where v is not a leaf. Can you solve this problem in polynomial time? 4. We want to determine the set of edges e in an undirected graph G with weights c : E(G) → R for which there exists a minimum weight spanning tree in G containing e (in other words, we are looking for the union of all minimum weight spanning trees in G). Show how this problem can be solved in O(mn) time. 5. Given an undirected graph G with arbitrary weights c : E(G) → R, we ask for a minimum weight connected spanning subgraph. Can you solve this problem efficiently? 6. Consider the following algorithm (sometimes called W ORST-O UT-G REEDY A LGORITHM, see Section 13.4). Examine the edges in order of non-increasing weights. Delete an edge unless it is a bridge. Does this algorithm solve the M INIMUM S PANNING T REE P ROBLEM? 7. Consider the following “colouring” algorithm. Initially all edges are uncoloured. Then apply the following rules in arbitrary order until all edges are coloured: Blue rule: Select a cut containing no blue edge. Among the uncoloured edges in the cut, select one of minimum cost and colour it blue. Red rule: Select a circuit containing no red edge. Among the uncoloured edges in the circuit, select one of maximum cost and colour it red. Show that one of the rules is always applicable as long as there are uncoloured edges left. Moreover, show that the algorithm maintains the “colour invariant”:

Exercises

145

there always exists an optimum spanning tree containing all blue edges but no red edge. (So the algorithm solves the M INIMUM S PANNING T REE P ROBLEM optimally.) Observe that K RUSKAL ’ S A LGORITHM and P RIM ’ S A LGORITHM are special cases. (Tarjan [1983]) 8. Suppose we wish to find a spanning tree T in an undirected graph such that the maximum weight of an edge in T is as small as possible. How can this be done? 9. For a finite set V ⊂ R2 , the Vorono¨ı diagram consists of the regions Pv := x ∈ R2 : ||x − v||2 = min ||x − w||2 w∈V

for v ∈ V . The Delaunay triangulation of V is the graph (V, {{v, w} ⊆ V, v 6= w, |Pv ∩ Pw | > 1}) .

10.

11.

12.

13.

A P minimum spanning tree for V is a tree T with V (T ) = V whose length {v,w}∈E(T ) ||v − w||2 is minimum. Prove that every minimum spanning tree is a subgraph of the Delaunay triangulation. Note: Using the fact that the Delaunay triangulation can be computed in O(n log n) time (where n = |V |; see e.g. Fortune [1987], Knuth [1992]), this implies an O(n log n) algorithm for the M INIMUM S PANNING T REE P ROB LEM for point sets in the plane. (Shamos and Hoey [1975]; see also Zhou, Shenoy and Nicholls [2002]) Can you decide in linear time whether a graph contains a spanning arborescence? Hint: To find a possible root, start at an arbitrary vertex and traverse edges backwards as long as possible. When encountering a circuit, contract it. Can you find a maximum cardinality branching in a given digraph in linear time? Hint: First find the strongly connected components. The M INIMUM W EIGHT ROOTED A RBORESCENCE P ROBLEM can be reduced to the M AXIMUM W EIGHT B RANCHING P ROBLEM by Proposition 6.8. However, it can also be solved directly by a modified version of E DMONDS ’ B RANCHING A LGORITHM. Show how. Prove that the spanning tree polytope of an undirected graph G (see Theorem 6.12) with n := |V (G)| is in general a proper subset of the polytope X X xe = n − 1, xe ≥ 1 for ∅ ⊂ X ⊂ V (G) . x ∈ [0, 1] E(G) : e∈E(G)

e∈δ(X )

Hint: To prove that this polytope is not integral, consider the graph shown in Figure 6.5 (the numbers are edge weights). (Magnanti and Wolsey [1995])

146

6 Spanning Trees and Arborescences 1

0 0

1

1

0

Fig. 6.5.

∗ 14. In Exercise 13 we saw that cut constraints do not sufﬁce to describe the spanning tree polytope. However, if we consider multicuts instead, we obtain a complete description: Prove that the spanning tree polytope of an undirected graph G with n := |V (G)| consists of all vectors x ∈ [0, 1] E(G) with xe = n − 1 and xe ≥ k − 1 for all multicuts C = δ(X 1 , . . . , X k ). e∈E(G)

e∈C

(Magnanti and Wolsey [1995]) 15. Prove that the convex hull of the incidence vectors of all forests in an undirected graph G is the polytope ⎧ ⎫ ⎨ ⎬ P := x ∈ [0, 1] E(G) : xe ≤ |X | − 1 for ∅ = X ⊆ V (G) . ⎩ ⎭ e∈E(G[X ])

Note: This statement implies Theorem 6.12 since e∈E(G[X ]) xe = |V (G)| −1 is a supporting hyperplane. Moreover, it is a special case of Theorem 13.21. ∗ 16. Prove that the convex hull of the incidence vectors of all branchings in a digraph G is the set of all vectors x ∈ [0, 1] E(G) with xe ≤ |X | − 1 for ∅ = X ⊆ V (G) and xe ≤ 1 for v ∈ V (G). e∈δ − (v)

e∈E(G[X ])

Note: This is a special case of Theorem 14.13. ∗ 17. Let G be a digraph and r ∈ V (G). Prove that the polytopes xe = 1 (v ∈ V (G) \ {r }), x ∈ [0, 1] E(G) : xe = 0 (e ∈ δ − (r )), e∈δ − (v)

e∈E(G[X ])

and

xe ≤ |X | − 1 for ∅ = X ⊆ V (G)

References

x ∈ [0, 1] E(G) : xe = 0 (e ∈ δ − (r )),

X

147

xe = 1 (v ∈ V (G) \ {r }),

e∈δ − (v)

X

xe ≥ 1 for r ∈ X ⊂ V (G)

e∈δ + (X )

18.

19.

∗ 20.

21.

22.

are both equal to the convex hull of the incidence vectors of all spanning arborescences rooted at r . Let G be a digraph and r ∈ V (G). Prove that G is the disjoint union of k spanning arborescences rooted at r if and only if the underlying undirected graph is the disjoint union of k spanning trees and |δ − (x)| = k for all x ∈ V (G) \ {r }. (Edmonds) Let G be a digraph and r ∈ V (G). Suppose that G contains k edge-disjoint paths from r to every other vertex, but removing any edge destroys this property. Prove that every vertex of G except r has exactly k entering edges. Hint: Use Theorem 6.17. Prove the statement of Exercise 19 without using Theorem 6.17. Formulate and prove a vertex-disjoint version. Hint: If a vertex v has more than k entering edges, take k edge-disjoint r -vpaths. Show that an edge entering v that is not used by these paths can be deleted. Give a polynomial-time algorithm for finding a maximum set of edge-disjoint spanning arborescences (rooted at r ) in a digraph G. Note: The most efficient algorithm is due to Gabow [1995]; see also (Gabow and Manu [1998]). Prove that the edges of a digraph G can be covered by k branchings if and only if the following two conditions hold: (a) |δ − (v)| ≤ k for all v ∈ V (G); (b) |E(G[X ])| ≤ k(|X | − 1) for all X ⊆ V (G). Hint: Use Theorem 6.17. (Frank [1979])

References General Literature: Ahuja, R.K., Magnanti, T.L., and Orlin, J.B. [1993]: Network Flows. Prentice-Hall, Englewood Cliffs 1993, Chapter 13 Balakrishnan, V.K. [1995]: Network Optimization. Chapman and Hall, London 1995, Chapter 1 Cormen, T.H., Leiserson, C.E., Rivest, R.L., and Stein, C. [2001]: Introduction to Algorithms. Second Edition. MIT Press, Cambridge 2001, Chapter 23 Gondran, M., and Minoux, M. [1984]: Graphs and Algorithms. Wiley, Chichester 1984, Chapter 4

148

6 Spanning Trees and Arborescences

Magnanti, T.L., and Wolsey, L.A. [1995]: Optimal trees. In: Handbooks in Operations Research and Management Science; Volume 7: Network Models (M.O. Ball, T.L. Magnanti, C.L. Monma, G.L. Nemhauser, eds.), Elsevier, Amsterdam 1995, pp. 503–616 Schrijver, A. [2003]: Combinatorial Optimization: Polyhedra and Efficiency. Springer, Berlin 2003, Chapters 50–53 Tarjan, R.E. [1983]: Data Structures and Network Algorithms. SIAM, Philadelphia 1983, Chapter 6 Wu, B.Y., and Chao, K.-M. [2004]: Spanning Trees and Optimization Problems. Chapman & Hall/CRC, Boca Raton 2004

Cited References: Bock, F.C. [1971]: An algorithm to construct a minimum directed spanning tree in a directed network. In: Avi-Itzak, B. (Ed.): Developments in Operations Research. Gordon and Breach, New York 1971, 29–44 Bor˚uvka, O. [1926a]: O jist´em probl´emu minim´aln´ım. Pr´aca Moravsk´e P˘r´ırodov˘edeck´e Spolne˘cnosti 3 (1926), 37–58 Bor˚uvka, O. [1926b]: P˘r´ıspev˘ek k ˘re˘sen´ı ot´azky ekonomick´e stavby. Elektrovodn´ıch s´ıt´ı. Elektrotechnicky Obzor 15 (1926), 153–154 Cayley, A. [1889]: A theorem on trees. Quarterly Journal on Mathematics 23 (1889), 376–378 Chazelle, B. [2000]: A minimum spanning tree algorithm with inverse-Ackermann type complexity. Journal of the ACM 47 (2000), 1028–1047 Cheriton, D., and Tarjan, R.E. [1976]: Finding minimum spanning trees. SIAM Journal on Computing 5 (1976), 724–742 Chu, Y., and Liu, T. [1965]: On the shortest arborescence of a directed graph. Scientia Sinica 4 (1965), 1396–1400; Mathematical Review 33, # 1245 Diestel, R. [1997]: Graph Theory. Springer, New York 1997 Dijkstra, E.W. [1959]: A note on two problems in connexion with graphs. Numerische Mathematik 1 (1959), 269–271 Dixon, B., Rauch, M., and Tarjan, R.E. [1992]: Verification and sensitivity analysis of minimum spanning trees in linear time. SIAM Journal on Computing 21 (1992), 1184– 1192 Edmonds, J. [1967]: Optimum branchings. Journal of Research of the National Bureau of Standards B 71 (1967), 233–240 Edmonds, J. [1970]: Submodular functions, matroids and certain polyhedra. In: Combinatorial Structures and Their Applications; Proceedings of the Calgary International Conference on Combinatorial Structures and Their Applications 1969 (R. Guy, H. Hanani, N. Sauer, J. Schonheim, eds.), Gordon and Breach, New York 1970, pp. 69–87 Edmonds, J. [1973]: Edge-disjoint branchings. In: Combinatorial Algorithms (R. Rustin, ed.), Algorithmic Press, New York 1973, pp. 91–96 Fortune, S. [1987]: A sweepline algorithm for Voronoi diagrams. Algorithmica 2 (1987), 153–174 Frank, A. [1978]: On disjoint trees and arborescences. In: Algebraic Methods in Graph Theory; Colloquia Mathematica; Soc. J. Bolyai 25 (L. Lov´asz, V.T. S´os, eds.), NorthHolland, Amsterdam 1978, pp. 159–169 Frank, A. [1979]: Covering branchings. Acta Scientiarum Mathematicarum (Szeged) 41 (1979), 77–82

References

149

Fredman, M.L., and Tarjan, R.E. [1987]: Fibonacci heaps and their uses in improved network optimization problems. Journal of the ACM 34 (1987), 596–615 Fredman, M.L., and Willard, D.E. [1994]: Trans-dichotomous algorithms for minimum spanning trees and shortest paths. Journal of Computer and System Sciences 48 (1994), 533–551 Fulkerson, D.R. [1974]: Packing rooted directed cuts in a weighted directed graph. Mathematical Programming 6 (1974), 1–13 Gabow, H.N. [1995]: A matroid approach to finding edge connectivity and packing arborescences. Journal of Computer and System Sciences 50 (1995), 259–273 Gabow, H.N., Galil, Z., and Spencer, T. [1989]: Efficient implementation of graph algorithms using contraction. Journal of the ACM 36 (1989), 540–572 Gabow, H.N., Galil, Z., Spencer, T., and Tarjan, R.E. [1986]: Efficient algorithms for finding minimum spanning trees in undirected and directed graphs. Combinatorica 6 (1986), 109–122 Gabow, H.N., and Manu, K.S. [1998]: Packing algorithms for arborescences (and spanning trees) in capacitated graphs. Mathematical Programming B 82 (1998), 83–109 Jarn´ık, V. [1930]: O jist´em probl´emu minim´aln´ım. Pr´aca Moravsk´e P˘r´ırodov˘edeck´e Spole˘cnosti 6 (1930), 57–63 Karger, D., Klein, P.N., and Tarjan, R.E. [1995]: A randomized linear-time algorithm to find minimum spanning trees. Journal of the ACM 42 (1995), 321–328 Karp, R.M. [1972]: A simple derivation of Edmonds’ algorithm for optimum branchings. Networks 1 (1972), 265–272 King, V. [1995]: A simpler minimum spanning tree verification algorithm. Algorithmica 18 (1997), 263–270 Knuth, D.E. [1992]: Axioms and hulls; LNCS 606. Springer, Berlin 1992 Korte, B., and Neˇsetˇril, J. [2001]: Vojt˘ech Jarn´ık’s work in combinatorial optimization. Discrete Mathematics 235 (2001), 1–17 Kruskal, J.B. [1956]: On the shortest spanning subtree of a graph and the travelling salesman problem. Proceedings of the AMS 7 (1956), 48–50 Lov´asz, L. [1976]: On two minimax theorems in graph. Journal of Combinatorial Theory B 21 (1976), 96–103 Nash-Williams, C.S.J.A. [1961]: Edge-disjoint spanning trees of finite graphs. Journal of the London Mathematical Society 36 (1961), 445–450 Nash-Williams, C.S.J.A. [1964]: Decompositions of finite graphs into forests. Journal of the London Mathematical Society 39 (1964), 12 Neˇsetˇril, J., Milkov´a, E., and Neˇsetˇrilov´a, H. [2001]: Otakar Bor˚uvka on minimum spanning tree problem. Translation of both the 1926 papers, comments, history. Discrete Mathematics 233 (2001), 3–36 Prim, R.C. [1957]: Shortest connection networks and some generalizations. Bell System Technical Journal 36 (1957), 1389–1401 Pr¨ufer, H. [1918]: Neuer Beweis eines Satzes u¨ ber Permutationen. Arch. Math. Phys. 27 (1918), 742–744 Shamos, M.I., and Hoey, D. [1975]: Closest-point problems. Proceedings of the 16th Annual IEEE Symposium on Foundations of Computer Science (1975), 151–162 Tarjan, R.E. [1975]: Efficiency of a good but not linear set union algorithm. Journal of the ACM 22 (1975), 215–225 Tutte, W.T. [1961]: On the problem of decomposing a graph into n connected factor. Journal of the London Mathematical Society 36 (1961), 221–230 Zhou, H., Shenoy, N., and Nicholls, W. [2002]: Efficient minimum spanning tree construction without Delaunay triangulation. Information Processing Letters 81 (2002), 271–276

7 Shortest Paths

One of the best-known combinatorial optimization problems is to find a shortest path between two specified vertices of a digraph:

S HORTEST PATH P ROBLEM Instance:

A digraph G, weights c : E(G) → R and two vertices s, t ∈ V (G).

Task:

Find a shortest s-t-path P, i.e. one of minimum weight c(E(P)), or decide that t is not reachable from s.

Obviously this problem has many practical applications. Like the M INIMUM S PANNING T REE P ROBLEM it also often appears as a subproblem when one deals with more difficult combinatorial optimization problems. In fact, the problem is not easy to solve if we allow arbitrary weights. For example, if all weights are −1 then the s-t-paths of weight 1−|V (G)| are precisely the Hamiltonian s-t-paths. Deciding whether such a path exists is a difficult problem (see Exercise 14(b) of Chapter 15). The problem becomes much easier if we restrict ourselves to nonnegative weights or at least exclude negative circuits: Definition 7.1. Let G be a (directed or undirected) graph with weights c : E(G) → R. c is called conservative if there is no circuit of negative total weight. We shall present algorithms for the S HORTEST PATH P ROBLEM in Section 7.1. The first one allows nonnegative weights only while the second algorithm can deal with arbitrary conservative weights. The algorithms of Section 7.1 in fact compute a shortest s-v-path for all v ∈ V (G) without using significantly more running time. Sometimes one is interested in the distance for every pair of vertices; Section 7.2 shows how to deal with this problem. Since negative circuits cause problems we also show how to detect them. If none exists, a circuit of minimum total weight can be computed quite easily. Another interesting problem asks for a circuit whose mean weight is minimum. As we shall see in Section 7.3 this problem can also be solved efficiently by similar techniques. Finding shortest paths in undirected graphs is more difficult unless the edge weights are nonnegative. Undirected edges of nonnegative weights can be replaced equivalently by a pair of oppositely directed edges of the same weight; this reduces

152

7 Shortest Paths

the undirected problem to a directed one. However, this construction does not work for edges of negative weight since it would introduce negative circuits. We shall return to the problem of finding shortest paths in undirected graphs with conservative weights in Section 12.2 (Corollary 12.12). Henceforth we work with a digraph G. Without loss of generality we may assume that G is connected and simple; among parallel edges we have to consider only the one with least weight.

7.1 Shortest Paths From One Source All shortest path algorithms we present are based on the following observation, sometimes called Bellman’s principle of optimality, which is indeed the core of dynamic programming: Proposition 7.2. Let G be a digraph with conservative weights c : E(G) → R, let k ∈ N, and let s and w be two vertices. Let P be a shortest one among all s-w-paths with at most k edges, and let e = (v, w) be its final edge. Then P[s,v] (i.e. P without the edge e) is a shortest one among all s-v-paths with at most k − 1 edges. Proof: Suppose Q is a shorter s-v-path than P[s,v] , and |E(Q)| ≤ k − 1. Then c(E(Q)) + c(e) < c(E(P)). If Q does not contain w, then Q + e is a shorter s-w-path than P, otherwise Q [s,w] has length c(E(Q [s,w] )) = c(E(Q)) + c(e) − c(E(Q [w,v] + e)) < c(E(P)) − c(E(Q [w,v] + e)) ≤ c(E(P)), because Q [w,v] + e is a circuit and c is conservative. In both cases we have a contradiction to the assumption that P is a shortest s-w-path with at most k edges. The same result holds for undirected graphs with nonnegative weights and also for acyclic digraphs with arbitrary weights. It yields the recursion formulas dist(s, s) = 0 and dist(s, w) = min{dist(s, v) + c((v, w)) : (v, w) ∈ E(G)} for w ∈ V (G) \ {s} which immediately solve the S HORTEST PATH P ROBLEM for acyclic digraphs (Exercise 6). Proposition 7.2 is also the reason why most algorithms compute the shortest paths from s to all other vertices. If one computes a shortest s-t-path P, one has already computed a shortest s-v-path for each vertex v on P. Since we cannot know in advance which vertices belong to P, it is only natural to compute shortest s-v-paths for all v. We can store these s-v-paths very efficiently by just storing the final edge of each path. We first consider nonnegative edge weights, i.e. c : E(G) → R+ . The S HORTEST PATH P ROBLEM can be solved by BFS if all weights are 1 (Proposition 2.18). For weights c : E(G) → N one could replace an edge e by a path of length c(e) and again use BFS. However, this might introduce an exponential number P of edges; recall that the input size is 2 n log m + m log n + e∈E(G) log c(e) , where n = |V (G)| and m = |E(G)|.

7.1 Shortest Paths From One Source

153

A much better idea is to use the following algorithm, due to Dijkstra [1959]. It is quite similar to P RIM ’ S A LGORITHM for the M INIMUM S PANNING T REE P ROBLEM (Section 6.1).

D IJKSTRA ’ S A LGORITHM Input:

A digraph G, weights c : E(G) → R+ and a vertex s ∈ V (G).

Output:

Shortest paths from s to all v ∈ V (G) and their lengths. More precisely, we get the outputs l(v) and p(v) for all v ∈ V (G). l(v) is the length of a shortest s-v-path, which consists of a shortest s- p(v)-path together with the edge ( p(v), v). If v is not reachable from s, then l(v) = ∞ and p(v) is undefined.

2

Set l(s) := 0. Set l(v) := ∞ for all v ∈ V (G) \ {s}. Set R := ∅. Find a vertex v ∈ V (G) \ R such that l(v) = min

3

Set R := R ∪ {v}.

4

For all w ∈ V (G) \ R such that (v, w) ∈ E(G) do: If l(w) > l(v) + c((v, w)) then set l(w) := l(v) + c((v, w)) and p(w) := v. If R 6= V (G) then go to . 2

1

5

w∈V (G)\R

l(w).

Theorem 7.3. (Dijkstra [1959]) D IJKSTRA ’ S A LGORITHM works correctly. Proof: We prove that the following statements hold at any stage of the algorithm: (a) For each v ∈ V (G) \ {s} with l(v) < ∞ we have p(v) ∈ R, l( p(v)) + c(( p(v), v)) = l(v), and the sequence v, p(v), p( p(v)), . . . contains s. (b) For all v ∈ R: l(v) = dist(G,c) (s, v). The statements trivially hold after . 1 l(w) is decreased to l(v) + c((v, w)) / R. As the sequence and p(w) is set to v in 4 only if v ∈ R and w ∈ v, p(v), p( p(v)), . . . contains s but no vertex outside R, in particular not w, (a) is preserved by . 4 (b) is trivial for v = s. Suppose that v ∈ V (G) \ {s} is added to R in , 3 and there is an s-v-path P in G that is shorter than l(v). Let y be the first vertex on P that belongs to (V (G) \ R) ∪ {v}, and let x be the predecessor of y on P. Since x ∈ R, we have by 4 and the induction hypothesis: l(y) ≤ l(x) + c((x, y)) = dist(G,c) (s, x) + c((x, y)) ≤ c(E(P[s,y] )) ≤ c(E(P)) < l(v), contradicting the choice of v in . 2

The running time is obviously O(n 2 ). Using a Fibonacci heap we can do better:

154

7 Shortest Paths

Theorem 7.4. (Fredman and Tarjan [1987]) D IJKSTRA ’ S A LGORITHM implemented with a Fibonacci heap runs in O(m + n log n) time, where n = |V (G)| and m = |E(G)|. Proof: We apply Theorem 6.6 to maintain the set {(v, l(v)) : v ∈ V (G) \ R, l(v) < ∞}. Then 2 and 3 are one DELETEMIN-operation, while the update of l(w) in 4 is an INSERT-operation if l(w) was infinite and a DECREASEKEYoperation otherwise. This is the best known strongly polynomial running time for the S HORTEST PATH P ROBLEM with nonnegative weights. (On different computational models, Fredman and Willard [1994], Thorup [2000] and Raman [1997] achieved slightly better running times.) If the weights are integers within a fixed range there is a simple linear-time algorithm (Exercise 2). In general, running times of O(m log log cmax ) (Johnson p [1982]) and O m + n log cmax (Ahuja et al. [1990]) are possible for weights c : E(G) → {0, . . . , cmax }. This has been improved by Thorup [2004] to O(m + n log log cmax ) and O(m + n log log n), but even the latter bound applies to integral edge weights only, and the algorithm is not strongly polynomial. For planar digraphs there is a linear-time algorithm due to Henzinger et al. [1997]. Finally we mention that Thorup [1999] found a linear-time algorithm for finding a shortest path in an undirected graph with nonnegative integral weights. See also Pettie and Ramachandran [2005]; this paper also contains more references. We now turn to an algorithm for general conservative weights:

M OORE -B ELLMAN -F ORD A LGORITHM Input: Output:

A digraph G, conservative weights c : E(G) → R, and a vertex s ∈ V (G). Shortest paths from s to all v ∈ V (G) and their lengths. More precisely, we get the outputs l(v) and p(v) for all v ∈ V (G). l(v) is the length of a shortest s-v-path which consists of a shortest s- p(v)-path together with the edge ( p(v), v). If v is not reachable from s, then l(v) = ∞ and p(v) is undefined.

1

Set l(s) := 0 and l(v) := ∞ for all v ∈ V (G) \ {s}.

2

For i := 1 to n − 1 do: For each edge (v, w) ∈ E(G) do: If l(w) > l(v) + c((v, w)) then set l(w) := l(v) + c((v, w)) and p(w) := v.

Theorem 7.5. (Moore [1959], Bellman [1958], Ford [1956]) The M OORE -B ELL MAN -F ORD A LGORITHM works correctly. Its running time is O(nm). Proof: The O(nm) running time is obvious. At any stage of the algorithm let R := {v ∈ V (G) : l(v) < ∞} and F := {(x, y) ∈ E(G) : x = p(y)}. We claim:

7.1 Shortest Paths From One Source

155

(a) l(y) ≥ l(x) + c((x, y)) for all (x, y) ∈ F; (b) If F contains a circuit C, then C has negative total weight; (c) If c is conservative, then (R, F) is an arborescence rooted at s. To prove (a), observe that l(y) = l(x) + c((x, y)) when p(y) is set to x and l(x) is never increased. To prove (b), suppose at some stage a circuit C in F was created by setting p(y) := x. Then before the insertion we had l(y) > l(x) + c((x, y)) as well as l(w) ≥ l(v) + c((v, w)) for all (v, w) ∈ E(C) \ {(x, y)} (by (a)). Summing these inequalities (the l-values cancel), we see that the total weight of C is negative. Since c is conservative, (b) implies that F is acyclic. Moreover, x ∈ R \ {s} implies p(x) ∈ R, so (R, F) is an arborescence rooted at s. Therefore l(x) is at least the length of the s-x-path in (R, F) for any x ∈ R (at any stage of the algorithm). We claim that after k iterations of the algorithm, l(x) is at most the length of a shortest s-x-path with at most k edges. This statement is easily proved by induction: Let P be a shortest s-x-path with at most k edges and let (w, x) be the last edge of P. Then, by Proposition 7.2, P[s,w] must be a shortest s-w-path with at most k − 1 edges, and by the induction hypothesis we have l(w) ≤ c(E(P[s,w] )) after k − 1 iterations. But in the k-th iteration edge (w, x) is also examined, after which l(x) ≤ l(w) + c((w, x)) ≤ c(E(P)). Since no path has more than n−1 edges, the above claim implies the correctness of the algorithm. This algorithm is still the fastest known strongly polynomial-time algorithm for the S HORTEST PATH P ROBLEM (with conservative algorithm √ weights). A scaling due to Goldberg [1995] has a running time of O nm log(|cmin | + 2) if the edge weights are integral and at least cmin . For planar graphs, Fakcharoenphol and Rao [2006] described an O(n log3 n)-algorithm. If G contains negative circuits, no polynomial-time algorithm is known (the problem becomes NP-hard; see Exercise 14(b) of Chapter 15). The main difficulty is that Proposition 7.2 does not hold for general weights. It is not clear how to construct a path instead of an arbitrary edge progression. If there are no negative circuits, any shortest edge progression is a path, plus possibly some circuits of zero weight that can be deleted. In view of this it is also an important question how to detect negative circuits. The following concept due to Edmonds and Karp [1972] is useful: Definition 7.6. Let G be a digraph with weights c : E(G) → R, and let π : V (G) → R. Then for any (x, y) ∈ E(G) we define the reduced cost of (x, y) with respect to π by cπ ((x, y)) := c((x, y)) + π(x) − π(y). If cπ (e) ≥ 0 for all e ∈ E(G), π is called a feasible potential. Theorem 7.7. Let G be a digraph with weights c : E(G) → R. There exists a feasible potential of (G, c) if and only if c is conservative.

156

7 Shortest Paths

Proof: If π is a feasible potential, we have for each circuit C: X X cπ (e) = (c(e) + π(x) − π(y)) = 0 ≤ e∈E(C)

e=(x,y)∈E(C)

X

c(e)

e∈E(C)

(the potentials cancel). So c is conservative. On the other hand, if c is conservative, we add a new vertex s and edges (s, v) of zero cost for all v ∈ V (G). We run the M OORE -B ELLMAN -F ORD A LGORITHM on this instance and obtain numbers l(v) for all v ∈ V (G). Since l(v) is the length of a shortest s-v-path for all v ∈ V (G), we have l(w) ≤ l(v) + c((v, w)) for all (v, w) ∈ E(G). Hence l is a feasible potential. This can be regarded as a special form of LP duality; see Exercise 8. Corollary 7.8. Given a digraph G with weights c : E(G) → R we can find in O(nm) time either a feasible potential or a negative circuit. Proof: As above, we add a new vertex s and edges (s, v) of zero cost for all v ∈ V (G). We run a modified version of the M OORE -B ELLMAN -F ORD A LGORITHM on this instance: Regardless of whether c is conservative or not, we run 1 and

2 as above. We obtain numbers l(v) for all v ∈ V (G). If l is a feasible potential, we are done. Otherwise let (v, w) be any edge with l(w) > l(v) + c((v, w)). We claim that the sequence w, v, p(v), p( p(v)), . . . contains a circuit. To see this, observe that l(v) must have been changed in the final iteration of . 2 Hence l( p(v)) has been changed during the last two iterations, l( p( p(v))) has been changed during the last three iterations, and so on. Since l(s) never changes, the first |V (G)| places of the sequence w, v, p(v), p( p(v)), . . . do not contain s, so a vertex must appear twice in the sequence. Thus we have found a circuit C in F := {(x, y) ∈ E(G) : x = p(y)}∪{(v, w)}. By (a) and (b) of the proof of Theorem 7.5, C has negative total weight. In practice there are more efficient methods to detect negative circuits; see Cherkassky and Goldberg [1999].

7.2 Shortest Paths Between All Pairs of Vertices Suppose we now want to find a shortest s-t-path for all ordered pairs of vertices (s, t) in a digraph:

A LL PAIRS S HORTEST PATHS P ROBLEM Instance:

A digraph G and conservative weights c : E(G) → R.

Task:

Find numbers lst and vertices pst for all s, t ∈ V (G) with s 6= t, such that lst is the length of a shortest s-t-path, and ( pst , t) is the final edge of such a path (if it exists).

7.2 Shortest Paths Between All Pairs of Vertices

157

Of course we could run the M OORE -B ELLMAN -F ORD A LGORITHM n times, once for each choice of s. This immediately gives us an O(n 2 m)-algorithm. However, one can do better: Theorem 7.9. The A LL PAIRS S HORTEST PATHS P ROBLEM can be solved in O(mn + n 2 log n) time, where n = |V (G)| and m = |E(G)|. Proof: Let (G, c) be an instance. First we compute a feasible potential π, which is possible in O(nm) time by Corollary 7.8. Then for each s ∈ V (G) we do a single-source shortest path computation from s using the reduced costs cπ instead of c. For any vertex t the resulting s-t-path is also a shortest path with respect to c, because the length of any s-t-path changes by π(s) − π(t), a constant. Since the reduced costs are nonnegative, we can use D IJKSTRA ’ S A LGORITHM each time. So, by Theorem 7.4, the total running time is O(mn + n(m + n log n)). The same idea will be used again in Chapter 9 (in the proof of Theorem 9.12). Pettie [2004] showed how to improve the running time to O(mn +n 2 log log n); this is the best known time bound. For dense graphs with nonnegative weights, Chan’s [2007] bound of O(n 3 log3 log n/ log2 n) is slightly better. If all edge weights are small positive integers, this can be improved using fast matrix multiplication; see e.g. Zwick [2002]. The solution of the A LL PAIRS S HORTEST PATHS P ROBLEM also enables us to compute the metric closure: Definition 7.10. Given a graph G (directed or undirected) with conservative ¯ c), weights c : E(G) → R. The metric closure of (G, c) is the pair (G, ¯ where G¯ is the simple graph on V (G) that, for x, y ∈ V (G) with x 6= y, contains an edge e = {x, y} (or e = (x, y) if G is directed) with weight c(e) ¯ = dist(G,c) (x, y) if and only if y is reachable from x in G. Corollary 7.11. Let G be a directed or undirected graph with conservative weights c : E(G) → R. Then the metric closure of (G, c) can be computed in O(mn + n 2 log n) time. Proof: If G is undirected, we replace each edge by a pair of oppositely directed edges. Then we solve the resulting instance of the A LL PAIRS S HORTEST PATHS P ROBLEM. The rest of the section is devoted to the F LOYD -WARSHALL A LGORITHM, another O(n 3 )-algorithm for the A LL PAIRS S HORTEST PATHS P ROBLEM. The main advantage of the F LOYD -WARSHALL A LGORITHM is its simplicity. We assume w.l.o.g. that the vertices are numbered 1, . . . , n.

158

7 Shortest Paths

F LOYD -WARSHALL A LGORITHM Input: Output:

A digraph G with V (G) = {1, . . . , n} and conservative weights c : E(G) → R. Matrices (li j )1≤i, j≤n and ( pi j )1≤i, j≤n where li j is the length of a shortest path from i to j, and ( pi j , j) is the final edge of such a path (if it exists). li j := c((i, j)) for all (i, j) ∈ E(G). li j := ∞ for all (i, j) ∈ (V (G) × V (G)) \ E(G) with i 6= j. lii := 0 for all i. pi j := i for all i, j ∈ V (G).

1

Set Set Set Set

2

For j := 1 to n do: For i := 1 to n do: If i 6= j then: For k := 1 to n do: If k 6= j then: If lik > li j + l jk then set lik := li j + l jk and pik := p jk .

Theorem 7.12. (Floyd [1962], Warshall [1962]) The F LOYD -WARSHALL A L GORITHM works correctly. Its running time is O(n 3 ). Proof: The running time is obvious. Claim: After the algorithm has run through the outer loop for j = 1, 2, . . . , j0 , the variable lik contains the length of a shortest i-k-path with intermediate vertices v ∈ {1, . . . , j0 } only (for all i and k), and ( pik , k) is the final edge of such a path. This statement will be shown by induction for j0 = 0, . . . , n. For j0 = 0 it is true by , 1 and for j0 = n it implies the correctness of the algorithm. Suppose the claim holds for some j0 ∈ {0, . . . , n − 1}. We have to show that it still holds for j0 + 1. For any i and k, during processing the outer loop for j = j0 + 1, lik (containing by the induction hypothesis the length of a shortest i-kpath with intermediate vertices v ∈ {1, . . . , j0 } only) is replaced by li, j0 +1 +l j0 +1,k if this value is smaller. It remains to show that the corresponding i-( j0 + 1)-path P and the ( j0 + 1)-k-path Q have no inner vertex in common. Suppose that there is an inner vertex belonging to both P and Q. By shortcutting the maximal closed walk in P + Q (which by our assumption has nonnegative weight because it is the union of circuits) we get an i-k-path R with intermediate vertices v ∈ {1, . . . , j0 } only. R is no longer than li, j0 +1 + l j0 +1,k (and in particular shorter than the lik before processing the outer loop for j = j0 + 1). This contradicts the induction hypothesis since R has intermediate vertices v ∈ {1, . . . , j0 } only. Like the M OORE -B ELLMAN -F ORD A LGORITHM, the F LOYD -WARSHALL A LGORITHM can also be used to detect the existence of negative circuits (Exercise 11). The A LL PAIRS S HORTEST PATHS P ROBLEM in undirected graphs with arbitrary conservative weights is more difficult; see Theorem 12.13.

7.3 Minimum Mean Cycles

159

7.3 Minimum Mean Cycles We can easily find a circuit of minimum total weight in a digraph with conservative weights, using the above shortest path algorithms (see Exercise 12). Another problem asks for a circuit whose mean weight is minimum:

M INIMUM M EAN C YCLE P ROBLEM Instance:

A digraph G, weights c : E(G) → R.

Task:

Find a circuit C whose mean weight that G is acyclic.

c(E(C)) |E(C)|

is minimum, or decide

In this section we show how to solve this problem with dynamic programming, quite similar to the shortest path algorithms. We may assume that G is strongly connected, since otherwise we can identify the strongly connected components in linear time (Theorem 2.19) and solve the problem for each strongly connected component separately. But for the following min-max theorem it suffices to assume that there is a vertex s from which all vertices are reachable. We consider not only paths, but arbitrary edge progressions (where vertices and edges may be repeated). Theorem 7.13. (Karp [1978]) Let G be a digraph with weights c : E(G) → R. Let s ∈ V (G) such that each vertex is reachable from s. For x ∈ V (G) and k ∈ Z+ let ( k ) X Fk (x) := min c((vi−1 , vi )) : v0 = s, vk = x, (vi−1 , vi ) ∈ E(G) for all i i=1

be the minimum weight of an edge progression of length k from s to x (and ∞ if there is none). Let µ(G, c) be the minimum mean weight of a circuit in G (and µ(G, c) = ∞ if G is acyclic). Then µ(G, c) =

min

max

x∈V (G) 0≤k≤n−1 Fk (x) 0, and therefore there must exist a vertex w ∈ R with ex f (w) < 0. Since f is an s-t-preflow, this vertex must be s. (b): Suppose there is a v-w-path in G f , say with vertices v = v0 , v1 , . . . , vk = w. Since there is a distance labeling ψ with respect to f , ψ(vi ) ≤ ψ(vi+1 ) + 1 for i = 0, . . . , k − 1. So ψ(v) ≤ ψ(w) + k. Note that k ≤ n − 1. (c): follows from (b) as ψ(s) = n and ψ(t) = 0. Part (c) helps us to prove the following: Theorem 8.23. When the algorithm terminates, f is a maximum s-t-flow. Proof: f is an s-t-flow because there are no active vertices. Lemma 8.22(c) implies that there is no augmenting path. Then by Theorem 8.5 we know that f is maximum. The question now is how many P USH and R ELABEL operations are performed. Lemma 8.24. (a) For each v ∈ V (G), ψ(v) is strictly increased by every R ELABEL(v), and is never decreased. (b) At any stage of the algorithm, ψ(v) ≤ 2n − 1 for all v ∈ V (G). (c) No vertex is relabelled more than 2n − 1 times. The total number of R ELABEL operations is at most 2n 2 − n.

8.5 The Goldberg-Tarjan Algorithm

179

+ Proof: (a): ψ is changed only in the R ELABEL procedure. If no e ∈ δG (v) f is admissible, then R ELABEL(v) strictly increases ψ(v) (because ψ is a distance labeling at any time). (b): We only change ψ(v) if v is active. By Lemma 8.22(a) and (b), ψ(v) ≤ ψ(s) + n − 1 = 2n − 1. (c): follows directly from (a) and (b).

We shall now analyse the number of P USH operations. We distinguish between saturating pushes (where u f (e) = 0 after the push) and nonsaturating pushes. Lemma 8.25. The number of saturating pushes is at most 2mn. Proof: After each saturating push from v to w, another such push cannot occur until ψ(w) increases by at least 2, a push from w to v occurs, and ψ(v) increases by at least 2. Together with Lemma 8.24(a) and (b), this proves that there are at ↔

most n saturating pushes on each edge (v, w) ∈ E(G).

The number of nonsaturating pushes can be in the order of n 2 m in general (Exercise 19). By choosing an active vertex v with ψ(v) maximum in 3 we can prove a better bound. As usual we denote n := |V (G)|, m := |E(G)| and may assume n ≤ m ≤ n 2 . Lemma 8.26. If we always choose v to be an active vertex with ψ(v) maximum in 3 of the P USH -R ELABEL A LGORITHM, the number of nonsaturating pushes √ is at most 8n 2 m. Proof: Call a phase the time between two consecutive changes of ψ ∗ := max{ψ(v) : v active}. As ψ ∗ can increase only by relabeling, its total increase is at most 2n 2 . As ψ ∗ = 0 initially, it can decrease at most 2n 2 times, and the number of phases is at most 4n 2 . √ m nonsaturating pushes and expensive Call a phase cheap if it contains at most √ otherwise. Clearly there are at most 4n 2 m nonsaturating pushes in cheap phases. Let X |{w ∈ V (G) : ψ(w) ≤ ψ(v)}|. 8 := v∈V (G):v active Initially 8 ≤ n 2 . A relabeling step may increase 8 by at most n. A saturating push may increase 8 by at most n. A nonsaturating push does not increase 8. Since 8 = 0 at termination, the total decrease of 8 is at most n 2 +n(2n 2 −n)+n(2mn) ≤ 4mn 2 . Now consider the nonsaturating pushes in an expensive phase. Each of them pushes flow along an edge (v, w) with ψ(v) = ψ ∗ = ψ(w) + 1, deactivating v and possibly activating w. As the phase ends by relabeling or by deactivating the last active vertex v with w with ψ(w) = ψ ∗ remains constant during the ψ(v) = ψ ∗ , the set of vertices √ phase, and it contains more than m vertices as the phase is expensive. Hence each

180

8 Network Flows

√ nonsaturating push in an expensive phase decreases 8 by at least m. Thus the total √ 2 √ number of nonsaturating pushes in expensive phases is at most 4mn = 4n 2 m. m This proof is due to Cheriyan and Mehlhorn [1999]. We finally get: Theorem 8.27. (Goldberg and Tarjan [1988], Cheriyan and Maheshwari [1989], Tunc¸el [1994]) The P USH -R ELABEL A LGORITHM solves the√M AXIMUM F LOW P ROBLEM correctly and can be implemented to run in O(n 2 m) time. Proof: The correctness follows from Proposition 8.21 and Theorem 8.23. As in Lemma 8.26 we always choose v in 3 to be an active vertex with ψ(v) maximum. To make this easy we keep track of doubly-linked lists L 0 , . . . , L 2n−1 , where L i contains the active vertices v with ψ(v) = i. These lists can be updated during each P USH and R ELABEL operation in constant time. We can then start by scanning L i for i = 0. When a vertex is relabelled, we increase i accordingly. When we find a list L i for the current i empty (after deactivating the last active vertex at that level), we decrease i until L i is nonempty. As we increase i at most 2n 2 times by Lemma 8.24(c), we also decrease i at most 2n 2 times. As a second data structure, we store a doubly-linked list Av containing the admissible edges leaving v for each vertex v. They can also be updated in each P USH operation in constant time, and in each R ELABEL operation in time proportional to the total number of edges incident to the relabelled vertex. So R ELABEL(v) takes a total of O(|δG (v)|) time, and by Lemma 8.24(c) the overall time for relabelling is O(mn). Each P USH takes constant√time, and by Lemma 8.25 and Lemma 8.26 the total number of pushes is O(n 2 m).

8.6 Gomory-Hu Trees Any algorithm for the M AXIMUM F LOW P ROBLEM also implies a solution to the following problem:

M INIMUM C APACITY C UT P ROBLEM Instance:

A network (G, u, s, t).

Task:

An s-t-cut in G with minimum capacity.

Proposition 8.28. The M INIMUM C APACITY C UT P ROBLEM can be solved in √the same running time as the M AXIMUM F LOW P ROBLEM, in particular in O(n 2 m) time. Proof: For a network (G, u, s, t) we compute a maximum s-t-flow f and define X to be the set of all vertices reachable from s in G f . X can be computed with the G RAPH S CANNING A LGORITHM in linear time (Proposition 2.17). By Lemma 8.3

8.6 Gomory-Hu Trees

181

√ + and Theorem 8.5, δG (X ) constitutes a minimum capacity s-t-cut. The O(n 2 m) running time follows from Theorem 8.27 (and is not best possible). In this section we consider the problem of finding a minimum capacity s-t-cut for each pair of vertices s, t in an undirected graph G with capacities u : E(G) → R+ . This problem can be reduced to the above one: For all pairs s, t ∈ V (G) we solve the M INIMUM C APACITY C UT P ROBLEM for (G 0 , u 0 , s, t), where (G 0 , u 0 ) arises from (G, u) by replacing each undirected edge {v, w} by two oppositely directed edges (v, w) and (w, v) with u 0 ((v, w)) = u 0((w, v)) = u({v, w}). In this way we obtain minimum s-t-cuts for all s, t after n2 flow computations. This section is devoted to the elegant method of Gomory and Hu [1961], which requires only n − 1 flow computations. We shall see some applications in Sections 12.3 and 20.3. Definition 8.29. Let G be an undirected graph and u : E(G) → R+ a capacity function. For two vertices s, t ∈ V (G) we denote by λst their local edgeconnectivity, i.e. the minimum capacity of a cut separating s and t. The edge-connectivity of a graph is obviously the minimum local edgeconnectivity with respect to unit capacities. Lemma 8.30. For all vertices i, j, k ∈ V (G) we have λik ≥ min(λi j , λ jk ). Proof: Let δ(A) be a cut with i ∈ A, k ∈ V (G) \ A and u(δ(A)) = λik . If j ∈ A then δ(A) separates j and k, so u(δ(A)) ≥ λ jk . If j ∈ V (G)\ A then δ(A) separates i and j, so u(δ(A)) ≥ λi j . We conclude that λik = u(δ(A)) ≥ min(λi j , λ jk ). Indeed, this condition is not only necessary but also sufficient for numbers (λi j )1≤i, j≤n with λi j = λ ji to be local edge-connectivities of some graph (Exercise 23). Definition 8.31. Let G be an undirected graph and u : E(G) → R+ a capacity function. A tree T is called a Gomory-Hu tree for (G, u) if V (T ) = V (G) and λst =

min u(δG (Ce ))

e∈E(Pst )

for all s, t ∈ V (G),

where Pst is the (unique) s-t-path in T and, for e ∈ E(T ), Ce and V (G) \ Ce are the connected components of T − e (i.e. δG (Ce ) is the fundamental cut of e with respect to T ). We shall see that every graph possesses a Gomory-Hu tree. This implies that for any undirected graph G there is a list of n − 1 cuts such that for each pair s, t ∈ V (G) a minimum s-t-cut belongs to the list. In general, a Gomory-Hu tree cannot be chosen as a subgraph of G. For example, consider G = K 3,3 and u ≡ 1. Here λst = 3 for all s, t ∈ V (G). It is easy to see that the Gomory-Hu trees for (G, u) are exactly the stars with five edges. The main idea of the algorithm for constructing a Gomory-Hu tree is as follows. First we choose any s, t ∈ V (G) and find some minimum s-t-cut, say δ(A). Let

182

8 Network Flows

B := V (G)\ A. Then we contract A (or B) to a single vertex, choose any s , t ∈ B (or s , t ∈ A, respectively) and look for a minimum s -t -cut in the contracted graph G . We continue this process, always choosing a pair s , t of vertices not separated by any cut obtained so far. At each step, we contract – for each cut E(A , B ) obtained so far – A or B , depending on which part does not contain s and t . Eventually each pair of vertices is separated. We have obtained a total of n − 1 cuts. The crucial observation is that a minimum s -t -cut in the contracted graph G is also a minimum s -t -cut in G. This is the subject of the following lemma. Note that when contracting a set A of vertices in (G, u), the capacity of each edge in G is the capacity of the corresponding edge in G. Lemma 8.32. Let G be an undirected graph and u : E(G) → R+ a capacity function. Let s, t ∈ V (G), and let δ(A) be a minimum s-t-cut in (G, u). Let now s , t ∈ V (G) \ A, and let (G , u ) arise from (G, u) by contracting A to a single vertex. Then for any minimum s -t -cut δ(K ∪ {A}) in (G , u ), δ(K ∪ A) is a minimum s -t -cut in (G, u). Proof: Let s, t, A, s , t , G , u be as above. W.l.o.g. s ∈ A. It sufﬁces to prove that there is a minimum s -t -cut δ(A ) in (G, u) such that A ⊂ A . So let δ(C) be any minimum s -t -cut in (G, u). W.l.o.g. s ∈ C. A V (G) \ A

t V (G) \ C C s

s

Fig. 8.3.

Since u(δ(·)) is submodular (cf. Lemma 2.1(c)), we have u(δ(A)) + u(δ(C)) ≥ u(δ(A ∩ C)) + u(δ(A ∪ C)). But δ(A ∩ C) is an s-t-cut, so u(δ(A ∩ C)) ≥ λst = u(δ(A)). Therefore u(δ(A ∪ C)) ≤ u(δ(C)) = λs t proving that δ(A ∪ C) is a minimum s -t -cut. (See Figure 8.3.) Now we describe the algorithm which constructs a Gomory-Hu tree. Note that the vertices of the intermediate trees T will be vertex sets of the original graph;

8.6 Gomory-Hu Trees

183

indeed they form a partition of V (G). At the beginning, the only vertex of T is V (G). In each iteration, a vertex of T containing at least two vertices of G is chosen and split into two.

G OMORY -H U A LGORITHM Input:

An undirected graph G and a capacity function u : E(G) → R+ .

Output:

A Gomory-Hu tree T for (G, u).

1

Set V (T ) := {V (G)} and E(T ) := ∅.

2

Choose some X ∈ V (T ) with |X | ≥ 2. If no such X exists then go to . 6

3

Choose s, t ∈ X with s 6= t. S For each connected component C of T − X do: Let SC := Y ∈V (C) Y . 0 0 Let (G , u ) arise from (G, u) by contracting SC to a single vertex vC for each connected component C of T − X . (So V (G 0 ) = X ∪ {vC : C is a connected component of T − X }.)

4

0 0 0 B 0 := V (G 0 )\ A0 . Find a minimum s-t-cut δ(A ) in (G , u ). Let [ [ SC ∪ (A0 ∩ X ) and B := SC ∪ (B 0 ∩ X ). Set A := vC ∈A0 \X

5

6

vC ∈B 0 \X

Set V (T ) := (V (T ) \ {X }) ∪ {A ∩ X, B ∩ X }. For each edge e = {X, Y } ∈ E(T ) incident to the vertex X do: If Y ⊆ A then set e0 := {A ∩ X, Y } else set e0 := {B ∩ X, Y }. Set E(T ) := (E(T ) \ {e}) ∪ {e0 } and w(e0 ) := w(e). Set E(T ) := E(T ) ∪ {{A ∩ X, B ∩ X }}. Set w({A ∩ X, B ∩ X }) := u 0 (δG 0 (A0 )). Go to . 2 Replace all {x} ∈ V (T ) by x and all {{x}, {y}} ∈ E(T ) by {x, y}. Stop.

Figure 8.4 illustrates the modification of T in . 5 To prove the correctness of this algorithm, we first show the following lemma: Lemma 8.33. Each time at the end of 4 we have .

(a) A ∪ B = V (G) (b) E(A, B) is a minimum s-t-cut in (G, u). Proof: The elements of V (T ) are always nonempty subsets of V (G), indeed V (T ) constitutes a partition of V (G). From this, (a) follows easily. We now prove (b). The claim is trivial for the first iteration (since here G 0 = G). We show that the property is preserved in each iteration. Let C1 , . . . , Ck be the connected components of T − X . Let us contract them one by one; for i = 0, . . . , k let (G i , u i ) arise from (G, u) by contracting each of SC1 , . . . , SCi to a single vertex. So (G k , u k ) is the graph which is denoted by (G 0 , u 0 ) in 3 of the algorithm.

184

8 Network Flows

(a)

X

(b)

A∩X

B∩X

Fig. 8.4.

Claim: For any minimum s-t-cut δ(Ai ) in (G i , u i ), δ(Ai−1 ) is a minimum s-t-cut in (G i−1 , u i−1 ), where if v Ci ∈ Ai (Ai \ {v Ci }) ∪ SCi Ai−1 := . Ai if v Ci ∈ / Ai Applying this claim successively for k, k − 1, . . . , 1 implies (b). To prove the claim, let δ(Ai ) be a minimum s-t-cut in (G i , u i ). By our assumption that (b) is true for the previous iterations, δ(SCi ) is a minimum si -ti -cut in (G, u) for some appropriate si , ti ∈ V (G). Furthermore, s, t ∈ V (G) \ SCi . So applying Lemma 8.32 completes the proof. Lemma 8.34. At any stage of the algorithm (until 6 is reached) for all e ∈ E(T ) ⎛ ⎛ ⎞⎞ w(e) = u ⎝δG ⎝ Z ⎠⎠ , Z ∈Ce

8.6 Gomory-Hu Trees

185

where Ce and V (T ) \ Ce are the connected components of T − e. Moreover for all e = {P, Q} ∈ E(T ) there are vertices p ∈ P and q ∈ Q with λ pq = w(e). Proof: Both statements are trivial at the beginning of the algorithm when T contains no edges; we show that they are never violated. So let X be the vertex of T chosen in 2 in some iteration of the algorithm. Let s, t, A , B , A, B be as determined in 3 and 4 next. W.l.o.g. assume s ∈ A . Edges of T not incident to X are not affected by . 5 For the new edge {A ∩ X, B ∩ X }, w(e) is clearly set correctly, and we have λst = w(e), s ∈ A ∩ X , t ∈ B ∩ X. So let us consider an edge e = {X, Y } that is replaced by e in . 5 We assume w.l.o.g. Y ⊆ A, so e = {A ∩ X, Y }. Assuming that the assertions were true for e we claim that they remain true for e . This is trivial for the ﬁrst assertion, because does not change. w(e) = w(e ) and u δG Z ∈Ce Z To show the second statement, we assume that there are p ∈ X, q ∈ Y with λ pq = w(e). If p ∈ A ∩ X then we are done. So henceforth assume that p ∈ B ∩ X (see Figure 8.5).

q Y

s

t

p B∩X

A∩X Fig. 8.5.

We claim that λsq = λ pq . Since λ pq = w(e) = w(e ) and s ∈ A ∩ X , this will conclude the proof. By Lemma 8.30, λsq ≥ min{λst , λt p , λ pq }. Since by Lemma 8.33(b) E(A, B) is a minimum s-t-cut, and since s, q ∈ A, we may conclude from Lemma 8.32 that λsq does not change if we contract B. Since t, p ∈ B, this means that adding an edge {t, p} with arbitrary high capacity does not change λsq . Hence λsq ≥ min{λst , λ pq }. Now observe that λst ≥ λ pq because the minimum s-t-cut E(A, B) also separates p and q. So we have

186

8 Network Flows

λsq ≥ λ pq . To prove equality, observe that w(e) is the capacity of a cut separating X and Y , and thus s and q. Hence λsq ≤ w(e) = λ pq .

This completes the proof.

Theorem 8.35. (Gomory and Hu [1961]) The G OMORY -H U A LGORITHM works correctly. Every√undirected graph possesses a Gomory-Hu tree, and such a tree is found in O(n 3 m) time. Proof: The complexity of the algorithm is clearly determined by n − 1 times the complexity of finding a minimum s-t-cut, since everything√else can be implemented in O(n 3 ) time. By Proposition 8.28 we obtain the O(n 3 m) bound. We prove that the output T of the algorithm is a Gomory-Hu tree for (G, u). It should be clear that T is a tree with V (T ) = V (G). Now let s, t ∈ V (G). Let Pst be the (unique) s-t-path in T and, for e ∈ E(T ), let Ce and V (G) \ Ce be the connected components of T − e. Since δ(Ce ) is an s-t-cut for each e ∈ E(Pst ), λst ≤

min u(δ(Ce )).

e∈E(Pst )

On the other hand, a repeated application of Lemma 8.30 yields λst ≥

min

{v,w}∈E(Pst )

λvw .

Hence applying Lemma 8.34 to the situation before execution of 6 (where each vertex X of T is a singleton) yields λst ≥

min u(δ(Ce )),

e∈E(Pst )

so equality holds.

A similar algorithm for the same task (which might be easier to implement) was suggested by Gusfield [1990].

8.7 The Minimum Capacity of a Cut in an Undirected Graph If we are only interested in a minimum capacity cut in an undirected graph G with capacities u : E(G) → R+ , there is a simpler method using n − 1 flow computations: just compute a minimum s-t-cut for some fixed vertex s and each t ∈ V (G) \ {s}. However, there are more efficient algorithms.

8.7 The Minimum Capacity of a Cut in an Undirected Graph

187

2

Hao and Orlin [1994] found an O(nm log nm )-algorithm for determining a minimum capacity cut. They use a modified version of the P USH -R ELABEL A L GORITHM . If we just want to compute the edge-connectivity of the graph (i.e. unit capacities), the currently fastest algorithm is due to Gabow [1995] with running time n ), where λ(G) is the edge-connectivity (observe that 2m ≥ λn). O(m +λ2 n log λ(G) Gabow’s algorithm uses matroid intersection techniques. We remark that the M AX IMUM F LOW P ROBLEM in undirected graphs with unit capacities can also be solved faster than in general (Karger and Levine [1998]). Nagamochi and Ibaraki [1992] found a completely different algorithm to determine a minimum capacity cut in an undirected graph. Their algorithm does not use max-flow computations at all. In this section we present this algorithm in a simplified form due to Stoer and Wagner [1997] and independently to Frank [1994]. We start with an easy definition. Definition 8.36. Given a graph G with capacities u : E(G) → R+ , we call an order v1 , . . . , vn of the vertices an MA (maximum adjacency) order if for all i ∈ {2, . . . , n}: X X u(e) = max u(e). e∈E({v1 ,...,vi−1 },{vi })

j∈{i,...,n}

e∈E({v1 ,...,vi−1 },{v j })

Proposition 8.37. Given a graph G with capacities u : E(G) → R+ , an MA order can be found in O(m + n log n) time. Proof: Consider the following algorithm. First set α(v) := 0 for all v ∈ V (G). Then for i := 1 to n do the following: choose vi from among V (G)\{v1 , . . . , vi−1 } such that P it has maximum α-value (breaking ties arbitrarily), and set α(v) := α(v) + e∈E({vi },{v}) u(e) for all v ∈ V (G) \ {v1 , . . . , vi }. The correctness of this algorithm is obvious. By implementing it with a Fibonacci heap, storing each vertex v with key −α(v) until it is selected, we get a running time of O(m + n log n) by Theorem 6.6 as there are n INSERT-, n DELETEMIN - and (at most) m DECREASEKEY -operations. Lemma 8.38. (Stoer and Wagner [1997], Frank [1994]) Let G be a graph with n := |V (G)| ≥ 2, capacities u : E(G) → R+ and an MA order v1 , . . . , vn . Then X λvn−1 vn = u(e). e∈E({vn },{v1 ,...,vn−1 })

Proof: Of course we only have to show “≥”. We shall use induction on |V (G)|+ |E(G)|. For |V (G)| < 3 the statement is trivial. We may assume that there is no edge e = {vn−1 , vn } ∈ E(G), because otherwise we would delete it (both left-hand side and right-hand side decrease by u(e)) and apply the induction hypothesis. Denote the right-hand side by R. Of course v1 , . . . , vn−1 is an MA order in G − vn . So by induction,

188

8 Network Flows n λvG−v = n−2 vn−1

X

X

u(e) ≥

e∈E({vn−1 },{v1 ,...,vn−2 })

u(e) = R.

e∈E({vn },{v1 ,...,vn−2 })

Here the inequality holds because v1 , . . . , vn was an MA order for G. The last n / E(G). So λvGn−2 vn−1 ≥ λvG−v equality is true because {vn−1 , vn } ∈ n−2 vn−1 ≥ R. On the other hand v1 , . . . , vn−2 , vn is an MA order in G −vn−1 . So by induction, X G−v u(e) = R, = λvn−2 vn−1 n e∈E({vn },{v1 ,...,vn−2 }) G−v

/ E(G). So λvGn−2 vn ≥ λvn−2 vn−1 again because {vn−1 , vn } ∈ n = R. Now by Lemma 8.30 λvn−1 vn ≥ min{λvn−1 vn−2 , λvn−2 vn } ≥ R. P Note that the existence of two vertices x, y with λx y = e∈δ(x) u(e) was already shown by Mader [1972], and follows easily from the existence of a Gomory-Hu tree (Exercise 25). Theorem 8.39. (Nagamochi and Ibaraki [1992], Stoer and Wagner [1997]) A minimum capacity cut in an undirected graph with nonnegative capacities can be found in O(mn + n 2 log n) time. Proof: We may assume that the given graph G is simple since we can unite parallel edges. Denote by λ(G) the minimum capacity of a cut in G. The algorithm proceeds as follows: Let G 0 := G. In the i-th step (i = 1, . . . , n−1) choose vertices x, y ∈ V (G i−1 ) with X G u(e). λx yi−1 = e∈δG i−1 (x)

By Proposition 8.37 and Lemma 8.38 this can be done in O(m + n log n) time. Set G γi := λx yi−1 , z i := x, and let G i result from G i−1 by contracting {x, y}. Observe that (8.1) λ(G i−1 ) = min{λ(G i ), γi }, because a minimum cut in G i−1 either separates x and y (in this case its capacity is γi ) or does not (in this case contracting {x, y} does not change anything). After arriving at G n−1 which has only one vertex, we choose an k ∈ {1, . . . , n − 1} for which γk is minimum. We claim that δ(X ) is a minimum capacity cut in G, where X is the vertex set in G whose contraction resulted in the vertex z k of G k−1 . But this is easy to see, since by (8.1) λ(G) = min{γ1 , . . . , γn−1 } = γk and γk is the capacity of the cut δ(X ). A randomized contraction algorithm for finding a minimum cut (with high probability) is discussed in Exercise 29. Moreover, we mention that the vertexconnectivity of a graph can be computed by O(n 2 ) flow computations (Exercise 30).

Exercises

189

In this section we have shown how to minimize f (X ) := u(δ(X )) over ∅ = X ⊂ V (G). Note that this f : 2V (G) → R+ is submodular and symmetric (i.e. f (A) = f (V (G)\ A) for all A). The algorithm presented here has been generalized by Queyranne [1998] to minimize general symmetric submodular functions; see Section 14.5. The problem of ﬁnding a maximum cut is much harder and will be discussed in Section 16.2.

Exercises 1. Let (G, u, s, t) be a network, and let δ + (X ) and δ + (Y ) be minimum s-t-cuts in (G, u). Show that δ + (X ∩ Y ) and δ + (X ∪ Y ) are also minimum s-t-cuts in (G, u). 2. Show that in case of irrational capacities, the F ORD -F ULKERSON A LGORITHM may not terminate at all. Hint: Consider the following network (Figure 8.6): x1

y1

x2

y2

s

t x3

y3

x4

y4 Fig. 8.6.

All lines represent edges in both directions. All edges have capacity S = except

1 1−σ

u((x1 , y1 )) = 1, u((x2 , y2 )) = σ, u((x3 , y3 )) = u((x4 , y4 )) = σ 2 √

∗

n n+1 + σ n+2 . where σ = 5−1 2 . Note that σ = σ (Ford and Fulkerson [1962]) 3. Let G be a digraph and M the incidence matrix of G. Prove that for all c, l, u ∈ Z E(G) with l ≤ u: max cx : x ∈ Z E(G) , l ≤ x ≤ u, M x = 0 =

E(G) min y u − y l : y , y ∈ Z+ , z M + y − y = c for some z ∈ ZV (G) . Show how this implies Theorem 8.6 and Corollary 8.7.

190

8 Network Flows

4. Prove Hoffman’s circulation theorem: Given a digraph G and lower and upper capacities l, u : E(G) → R+ with l(e) ≤ u(e) for all e ∈ E(G), there is circulation f with l(e) ≤ f (e) ≤ u(e) for all e ∈ E(G) if and only if X X l(e) ≤ u(e) for all X ⊆ V (G). e∈δ − (X )

∗

e∈δ + (X )

Note: Hoffman’s circulation theorem in turn quite easily implies the MaxFlow-Min-Cut Theorem. (Hoffman [1960]) 5. Consider a network (G, u, s, t), a maximum s-t-flow f and the residual graph G f . Form a digraph H from G f by contracting the set S of vertices reachable from s to a vertex v S , contracting the set T of vertices from which t is reachable to a vertex vT , and contracting each strongly connected component X of G f − (S ∪ T ) to a vertex v X . Observe that H is acyclic. Prove that there + is a one-to-one correspondence between the sets X ⊆ V (G) for which δG (X ) + is a minimum s-t-cut in (G, u) and the sets Y ⊆ V (H ) for which δ H (Y ) is a directed vT -v S -cut in H (i.e. a directed cut in H separating vT and v S ). Note: This statement also holds for G f without any contraction instead of H . However, we shall use the statement in the above form in Section 20.4. (Picard and Queyranne [1980]) 6. Let G be a digraph and c P : E(G) → R. We Plook for a set X ⊂ V (G) with s ∈ X and t ∈ / X such that e∈δ + (X ) c(e) − e∈δ − (X ) c(e) is minimum. Show how to reduce this problem to the M INIMUM C APACITY C UT P ROBLEM. Hint: Construct a network where all edges are incident to s or t. 7. Let G be an acyclic digraph with mappings σ, τ, c : E(G) → R+ , and a number x : E(G) → R+ such that σ (e) ≤ x(e) ≤ τ (e) C ∈ R+ . We look for a mapping P for all e ∈ E(G) and e∈E(G) (τ (e) − x(e))c(e) ≤ C. Among the feasible solutions we want to minimize the length (with respect to x) of the longest path in G. The meaning behind the above is the following. The edges correspond to jobs, σ (e) and τ (e) stand for the minimum and maximum completion time of job e, and c(e) is the cost of reducing the completion time of job e by one unit. If there are two jobs e = (i, j) and e0 = ( j, k), job e has to be finished before job e0 can be processed. We have a fixed budget C and want to minimize the total completion time. Show how to solve this problem using network flow techniques. (This application is known as PERT, program evaluation and review technique, or CPM, critical path method.) Hint: Introduce one source s and one sink t. Start with x = τ and successively reduce the length of the longest s-t-path (with respect to x) at the minimum possible cost. Use Exercise 7 of Chapter 7, Exercise 8 of Chapter 3, and Exercise 6. (Phillips and Dessouky [1977])

Exercises

191

8. Let (G, c, s, t) be a network such that G is planar even when an edge e = (s, t) is added. Consider the following algorithm. Start with the flow f ≡ 0 and let G 0 := G f . At each step consider the boundary B of a face of G 0 +e containing e (with respect to some fixed planar embedding). Augment f along B − e. Let G 0 consist of the forward edges of G f only and iterate as long as t is reachable from s in G 0 . Prove that this algorithm computes a maximum s-t-flow. Use Theorem 2.40 to show that it can be implemented to run in O(n 2 ) time. (Ford and Fulkerson [1956], Hu [1969]) Note: This problem can be solved in O(n) time. For general planar networks an O(n log n)-algorithm exists; see Weihe [1997] and Borradaile and Klein [2006]. 9. Show that the directed edge-disjoint version of Menger’s Theorem 8.9 also follows directly from Theorem 6.17. 10. Let G be a graph (directed or undirected), x, y, z three vertices, and α, β ∈ N with α ≤ λx y , β ≤ λx z and α + β ≤ max{λx y , λx z }. Prove that there are α x-y-paths and β x-z-paths such that these α+β paths are pairwise edge-disjoint. 11. Let G be a digraph that contains k edge-disjoint s-t-paths for any two vertices s and t (such a graph is called strongly k-edge-connected). Let H be any digraph with V (H ) = V (G) and |E(H )| = k. Prove that the instance (G, H ) of the D IRECTED E DGE -D ISJOINT PATHS P ROBLEM has a solution. (Mader [1981] and Shiloach [1979]) 12. Let G be a digraph with at least k edges. Prove: G contains k edge-disjoint s-t-paths for any two vertices s and t if and only if for any k distinct edges e1 = (x1 , y1 ), . . . , ek = (xk , yk ), G − {e1 , . . . , ek } contains k edge-disjoint spanning arborescences T1 , . . . , Tk such that Ti is rooted at yi (i = 1, . . . , k). Note: This generalizes Exercise 11. Hint: Use Theorem 6.17. (Su [1997]) 13. Let G be a digraph with capacities c : E(G) → R+ and r ∈ V (G). Can one determine an r -cut with minimum capacity in polynomial time? Can one determine a directed cut with minimum capacity in polynomial time (or decide that G is strongly connected)? Note: The answer to the first question solves the S EPARATION P ROBLEM for the M INIMUM W EIGHT ROOTED A RBORESCENCE P ROBLEM; see Corollary 6.14. 14. Show how to find a blocking flow in an acyclic network in O(nm) time. (Dinic [1970]) 15. Let (G, u, s, t) be a network such that G − t is an arborescence. Show how to find a maximum s-t-flow in linear time. Hint: Use DFS. ∗ 16. Let (G, u, s, t) be a network such that the underlying undirected graph of G − {s, t} is a forest. Show how to find a maximum s-t-flow in linear time. (Vygen [2002])

∗

192

8 Network Flows

17. Consider a modified version of F UJISHIGE ’ S A LGORITHM where in 5 we choose vi ∈ V (G) \ {v1 , . . . , vi−1 } such that b(vi ) is maximum, step 4 is replaced by stopping if b(v) = 0 for all v ∈ V (G) \ {v1 , . . . , vi }, and in the beginning of 6 we set β(t) := minij=2 b( j). Then X and α are not needed anymore. (a) Show that this variant of the algorithm works correctly. (b) Let αk be the number minij=2 b( j) in iteration k (or zero if the algorithm

18.

19. 20.

21.

22.

23.

24.

k+2n stops before iteration k). Show that minl=k+1 αl ≤ 12 αk for all k. Conclude that the number of iterations is O(n log u max ). (c) Show how to implement one iteration in O(m + n log n) time. Let us call a preflow f maximum if ex f (t) is maximum. (a) Show that for any maximum preflow f there exists a maximum flow f 0 with f 0 (e) ≤ f (e) for all e ∈ E(G). (b) Show how a maximum preflow can be converted into a maximum flow in O(nm) time. (Hint: Use a variant of the E DMONDS -K ARP A LGORITHM.) Prove that the P USH -R ELABEL A LGORITHM performs O(n 2 m) nonsaturating pushes, independent of the choice of v in . 3 Given an acyclic digraph G with weights c : E(G) → R+ , find a maximum weight directed cut in G. Show how this problem can be reduced to a minimum s-t-cut problem and be solved in O(n 3 ) time. Hint: Use Exercise 6. Let G be an acyclic digraph with weights c : E(G) → R+ . We look for the maximum weight edge set F ⊆ E(G) such that no path in G contains more than one edge of F. Show that this problem is equivalent to looking for the maximum weight directed cut in G (and thus can be solved in O(n 3 ) time by Exercise 20). Given an undirected graph G with capacities u : E(G) → R+ and a set T ⊆ V (G) with |T | ≥ 2.PWe look for a set X ⊂ V (G) with T ∩ X 6= ∅ and T \ X 6= ∅ such that e∈δ(X ) u(e) is minimum. Show how to solve this problem in O(n 4 ) time, where n = |V (G)|. Let λi j , 1 ≤ i, j ≤ n, be nonnegative numbers with λi j = λ ji and λik ≥ min(λi j , λ jk ) for any three distinct indices i, j, k ∈ {1, . . . , n}. Show that there exists a graph G with V (G) = {1, . . . , n} and capacities u : E(G) → R+ such that the local edge-connectivities are precisely the λi j . Hint: Consider a maximum weight spanning tree in (K n , c), where c({i, j}) := λi j . (Gomory and Hu [1961]) Let G be an undirected graph with capacities u : E(G) → R+ , and let T ⊆ V (G) with |T | even. A T -cut in G is a cut δ(X ) with |X ∩ T | odd. Construct a polynomial time algorithm for finding a T -cut of minimum capacity in (G, u). Hint: Use a Gomory-Hu tree. (A solution of this exercise can be found in Section 12.3.)

Exercises

193

25. Let G be a simple undirected graph with at least two vertices. Suppose the degree of each vertex of G is at least k. Prove that there are two vertices s and t such that at least k edge-disjoint s-t-paths exist. What if there is exactly one vertex with degree less than k? Hint: Consider a Gomory-Hu tree for G. 26. Consider the problem of determining the edge-connectivity λ(G) of an undirected graph (with unit capacities). Section 8.7 shows how to solve this problem in O(mn) time, provided that we can find an MA order of an undirected graph with unit capacities in O(m + n) time. How can this be done? G denote ∗ 27. Let G be an undirected graph with an MA order v1 , . . . , vn . Let κuv the maximum number of vertex-disjoint u-v-paths in G. Prove κvGn−1 vn = |E({vn }, {v1 , . . . , vn−1 })| (the vertex-disjoint counterpart of Lemma 8.38). G Hint: Prove by induction that κv j vi ji = |E({v j }, {v1 , . . . , vi })|, where G i j = / E(G), choose G[{v1 , . . . , vi }∪{v j }]. To do this, assume w.l.o.g. that {v j , vi } ∈ a minimal set Z ⊆ {v1 , . . . , vi−1 } separating v j and vi (Menger’s Theorem 8.10), and let h ≤ i be the maximum number such that vh ∈ / Z and vh is adjacent to vi or v j . (Frank [unpublished]) ∗ 28. An undirected graph is called chordal if it has no circuit of length at least four as an induced subgraph. An order v1 , . . . , vn of an undirected graph G is called simplicial if {vi , v j }, {vi , vk } ∈ E(G) implies {v j , vk } ∈ E(G) for i < j < k. (a) Prove that a graph with a simplicial order must be chordal. (b) Let G be a chordal graph, and let v1 , . . . , vn be an MA order. Prove that vn , vn−1 , . . . , v1 is a simplicial order. Hint: Use Exercise 27 and Menger’s Theorem 8.10. Note: The fact that a graph is chordal if and only if it has a simplicial order is due to Rose [1970]. 29. Let G an undirected graph with capacities u : E(G) → R+ . Let ∅ 6= A ⊂ V (G) such that δ(A) is a minimum capacity cut in G. (a) Show that u(δ(A)) ≤ n2 u(E(G)). (Hint: Consider the trivial cuts δ(x), x ∈ V (G).) (b) Consider the following procedure: We randomly choose an edge which u(e) we contract, each edge e is chosen with probability u(E(G)) . We repeat this operation until there are only two vertices. Prove that the probability 2 . that we never contract an edge of δ(A) is at least (n−1)n (c) Conclude that running the randomized algorithm in (b) kn 2 times yields δ(A) with probability at least 1 − e−2k . (Such an algorithm with a positive probability of a correct answer is called a Monte Carlo algorithm.) (Karger and Stein [1996]; see also Karger [2000]) 30. Show how the vertex-connectivity of an undirected graph can be determined in O(n 5 ) time. Hint: Recall the proof of Menger’s Theorem.

194

8 Network Flows

Note: There exists an O(n 4 )-algorithm; see (Henzinger, Rao and Gabow [2000]). 31. Let G be a connected undirected graph with capacities u : E(G) → R+ . We are looking for a minimum capacity 3-cut, i.e. an edge set whose deletion splits G into at least three connected components. Let n := |V (G)| ≥ 4. Let δ(X 1 ), δ(X 2 ), . . . be a list of the cuts ordered by nondecreasing capacities: u(δ(X 1 )) ≤ u(δ(X 2 )) ≤ · · · . Assume that we know the first 2n − 2 elements of this list (note: they can be computed in polynomial time by a method of Vazirani and Yannakakis [1992]). (a) Show that for some indices i, j ∈ {1, . . . , 2n − 2} all sets X i \ X j , X j \ X i , X i ∩ X j and V (G) \ (X i ∪ X j ) are nonempty. (b) Show that there is a 3-cut of capacity at most 32 u(δ(X 2n−2 ). (c) For each i = 1, . . . , 2n − 2 consider δ(X i ) plus a minimum capacity cut of G − X i , and also δ(X i ) plus a minimum capacity cut of G[X i ]. This yields a list of at most 4n − 4 3-cuts. Prove that one of them is optimum. (Nagamochi and Ibaraki [2000]) Note: The problem of finding the optimum 3-cut separating three given vertices is much harder; see Dahlhaus et al. [1994] and Cheung, Cunningham and Tang [2006].

References General Literature: Ahuja, R.K., Magnanti, T.L., and Orlin, J.B. [1993]: Network Flows. Prentice-Hall, Englewood Cliffs 1993 Cook, W.J., Cunningham, W.H., Pulleyblank, W.R., and Schrijver, A. [1998]: Combinatorial Optimization. Wiley, New York 1998, Chapter 3 Cormen, T.H., Leiserson, C.E., Rivest, R.L., and Stein, C. [2001]: Introduction to Algorithms. Second Edition. MIT Press, Cambridge 2001, Chapter 26 Ford, L.R., and Fulkerson, D.R. [1962]: Flows in Networks. Princeton University Press, Princeton 1962 Frank, A. [1995]: Connectivity and network flows. In: Handbook of Combinatorics; Vol. 1 (R.L. Graham, M. Gr¨otschel, L. Lov´asz, eds.), Elsevier, Amsterdam, 1995 ´ and Tarjan, R.E. [1990]: Network flow algorithms. In: Paths, Goldberg, A.V., Tardos, E., Flows, and VLSI-Layout (B. Korte, L. Lov´asz, H.J. Pr¨omel, A. Schrijver, eds.), Springer, Berlin 1990, pp. 101–164 Gondran, M., and Minoux, M. [1984]: Graphs and Algorithms. Wiley, Chichester 1984, Chapter 5 Jungnickel, D. [2007]: Graphs, Networks and Algorithms. Third Edition. Springer, Berlin 2007 Phillips, D.T., and Garcia-Diaz, A. [1981]: Fundamentals of Network Analysis. PrenticeHall, Englewood Cliffs 1981 Ruhe, G. [1991]: Algorithmic Aspects of Flows in Networks. Kluwer Academic Publishers, Dordrecht 1991 Schrijver, A. [2003]: Combinatorial Optimization: Polyhedra and Efficiency. Springer, Berlin 2003, Chapters 9,10,13–15

References

195

Tarjan, R.E. [1983]: Data Structures and Network Algorithms. SIAM, Philadelphia 1983, Chapter 8 Thulasiraman, K., and Swamy, M.N.S. [1992]: Graphs: Theory and Algorithms. Wiley, New York 1992, Chapter 12

Cited References: Ahuja, R.K., Orlin, J.B., and Tarjan, R.E. [1989]: Improved time bounds for the maximum flow problem. SIAM Journal on Computing 18 (1989), 939–954 Borradaile, G. and Klein, P. [2006]: An O(n log n) algorithm for maximum st-flow in a directed planar graph. Proceedings of the 17th Annual ACM-SIAM Symposium on Discrete Algorithms (2006), 524–533 Cheriyan, J., and Maheshwari, S.N. [1989]: Analysis of preflow push algorithms for maximum network flow. SIAM Journal on Computing 18 (1989), 1057–1086 Cheriyan, J., and Mehlhorn, K. [1999]: An analysis of the highest-level selection rule in the preflow-push max-flow algorithm. Information Processing Letters 69 (1999), 239–242 Cherkassky, B.V. [1977]: √ Algorithm of construction of maximal flow in networks with complexity of O(V 2 E) operations. Mathematical Methods of Solution of Economical Problems 7 (1977), 112–125 [in Russian] Cheung, K.K.H., Cunningham, W.H., and Tang, L. [2006]: Optimal 3-terminal cuts and linear programming. Mathematical Programming 106 (2006), 1–23 Dahlhaus, E., Johnson, D.S., Papadimitriou, C.H., Seymour, P.D., and Yannakakis, M. [1994]: The complexity of multiterminal cuts. SIAM Journal on Computing 23 (1994), 864–894 Dantzig, G.B., and Fulkerson, D.R. [1956]: On the max-flow min-cut theorem of networks. In: Linear Inequalities and Related Systems (H.W. Kuhn, A.W. Tucker, eds.), Princeton University Press, Princeton 1956, pp. 215–221 Dinic, E.A. [1970]: Algorithm for solution of a problem of maximum flow in a network with power estimation. Soviet Mathematics Doklady 11 (1970), 1277–1280 Edmonds, J., and Karp, R.M. [1972]: Theoretical improvements in algorithmic efficiency for network flow problems. Journal of the ACM 19 (1972), 248–264 Elias, P., Feinstein, A., and Shannon, C.E. [1956]: Note on maximum flow through a network. IRE Transactions on Information Theory, IT-2 (1956), 117–119 Ford, L.R., and Fulkerson, D.R. [1956]: Maximal Flow Through a Network. Canadian Journal of Mathematics 8 (1956), 399–404 Ford, L.R., and Fulkerson, D.R. [1957]: A simple algorithm for finding maximal network flows and an application to the Hitchcock problem. Canadian Journal of Mathematics 9 (1957), 210–218 Frank, A. [1994]: On the edge-connectivity algorithm of Nagamochi and Ibaraki. Laboratoire Artemis, IMAG, Universit´e J. Fourier, Grenoble, 1994 Fujishige, S. [2003]: A maximum flow algorithm using MA ordering. Operations Research Letters 31 (2003), 176–178 Gabow, H.N. [1995]: A matroid approach to finding edge connectivity and packing arborescences. Journal of Computer and System Sciences 50 (1995), 259–273 5

2

Galil, Z. [1980]: An O(V 3 E 3 ) algorithm for the maximal flow problem. Acta Informatica 14 (1980), 221–242 Galil, Z., and Namaad, A. [1980]: An O(E V log2 V ) algorithm for the maximal flow problem. Journal of Computer and System Sciences 21 (1980), 203–217

196

8 Network Flows

Gallai, T. [1958]: Maximum-minimum S¨atze u¨ ber Graphen. Acta Mathematica Academiae Scientiarum Hungaricae 9 (1958), 395–434 Goldberg, A.V., and Rao, S. [1998]: Beyond the flow decomposition barrier. Journal of the ACM 45 (1998), 783–797 Goldberg, A.V., and Tarjan, R.E. [1988]: A new approach to the maximum flow problem. Journal of the ACM 35 (1988), 921–940 Gomory, R.E., and Hu, T.C. [1961]: Multi-terminal network flows. Journal of SIAM 9 (1961), 551–570 Gusfield, D. [1990]: Very simple methods for all pairs network flow analysis. SIAM Journal on Computing 19 (1990), 143–155 Hao, J., and Orlin, J.B. [1994]: A faster algorithm for finding the minimum cut in a directed graph. Journal of Algorithms 17 (1994), 409–423 Henzinger, M.R., Rao, S., and Gabow, H.N. [2000]: Computing vertex connectivity: new bounds from old techniques. Journal of Algorithms 34 (2000), 222–250 Hoffman, A.J. [1960]: Some recent applications of the theory of linear inequalities to extremal combinatorial analysis. In: Combinatorial Analysis (R.E. Bellman, M. Hall, eds.), AMS, Providence 1960, pp. 113–128 Hu, T.C. [1969]: Integer Programming and Network Flows. Addison-Wesley, Reading 1969 Karger, D.R. [2000]: Minimum cuts in near-linear time. Journal of the ACM 47 (2000), 46–76 Karger, D.R., and Levine, M.S. [1998]: Finding maximum flows in undirected graphs seems easier than bipartite matching. Proceedings of the 30th Annual ACM Symposium on Theory of Computing (1998), 69–78 Karger, D.R., and Stein, C. [1996]: A new approach to the minimum cut problem. Journal of the ACM 43 (1996), 601–640 Karzanov, A.V. [1974]: Determining the maximal flow in a network by the method of preflows. Soviet Mathematics Doklady 15 (1974), 434–437 King, V., Rao, S., and Tarjan, R.E. [1994]: A faster deterministic maximum flow algorithm. Journal of Algorithms 17 (1994), 447–474 ¨ Mader, W. [1972]: Uber minimal n-fach zusammenh¨angende, unendliche Graphen und ein Extremalproblem. Arch. Math. 23 (1972), 553–560 Mader, W. [1981]: On a property of n edge-connected digraphs. Combinatorica 1 (1981), 385–386 Malhotra, V.M., Kumar, M.P., and Maheshwari, S.N. [1978]: An O(|V |3 ) algorithm for finding maximum flows in networks. Information Processing Letters 7 (1978), 277–278 Menger, K. [1927]: Zur allgemeinen Kurventheorie. Fundamenta Mathematicae 10 (1927), 96–115 Nagamochi, H., and Ibaraki, T. [1992]: Computing edge-connectivity in multigraphs and capacitated graphs. SIAM Journal on Discrete Mathematics 5 (1992), 54–66 Nagamochi, H., and Ibaraki, T. [2000]: A fast algorithm for computing minimum 3-way and 4-way cuts. Mathematical Programming 88 (2000), 507–520 Phillips, S., and Dessouky, M.I. [1977]: Solving the project time/cost tradeoff problem using the minimal cut concept. Management Science 24 (1977), 393–400 Picard, J., and Queyranne, M. [1980]: On the structure of all minimum cuts in a network and applications. Mathematical Programming Study 13 (1980), 8–16 Queyranne, M. [1998]: Minimizing symmetric submodular functions. Mathematical Programming B 82 (1998), 3–12 Rose, D.J. [1970]: Triangulated graphs and the elimination process. Journal of Mathematical Analysis and Applications 32 (1970), 597–609

References

197

Shiloach, Y. [1978]: An O(n I log2 I ) maximum-flow algorithm. Technical Report STANCS-78-802, Computer Science Department, Stanford University, 1978 Shiloach, Y. [1979]: Edge-disjoint branching in directed multigraphs. Information Processing Letters 8 (1979), 24–27 Shioura, A. [2004]: The MA ordering max-flow algorithm is not strongly polynomial for directed networks. Operations Research Letters 32 (2004), 31–35 Sleator, D.D. [1980]: An O(nm log n) algorithm for maximum network flow. Technical Report STAN-CS-80-831, Computer Science Department, Stanford University, 1978 Sleator, D.D., and Tarjan, R.E. [1983]: A data structure for dynamic trees. Journal of Computer and System Sciences 26 (1983), 362–391 Su, X.Y. [1997]: Some generalizations of Menger’s theorem concerning arc-connected digraphs. Discrete Mathematics 175 (1997), 293–296 Stoer, M., and Wagner, F. [1997]: A simple min cut algorithm. Journal of the ACM 44 (1997), 585–591 Tunc¸el, L. [1994]: On the complexity preflow-push algorithms for maximum flow problems. Algorithmica 11 (1994), 353–359 Vazirani, V.V., and Yannakakis, M. [1992]: Suboptimal cuts: their enumeration, weight, and number. In: Automata, Languages and Programming; Proceedings; LNCS 623 (W. Kuich, ed.), Springer, Berlin 1992, pp. 366–377 Vygen, J. [2002]: On dual minimum cost flow algorithms. Mathematical Methods of Operations Research 56 (2002), 101–126 Weihe, K. [1997]: Maximum (s, t)-flows in planar networks in O(|V | log |V |) time. Journal of Computer and System Sciences 55 (1997), 454–475 Whitney, H. [1932]: Congruent graphs and the connectivity of graphs. American Journal of Mathematics 54 (1932), 150–168

9 Minimum Cost Flows

In this chapter we show how we can take edge costs into account. For example, in our application of the M AXIMUM F LOW P ROBLEM to the J OB A SSIGNMENT P ROBLEM mentioned in the introduction of Chapter 8 one could introduce edge costs to model that the employees have different salaries; our goal is to meet a deadline when all jobs must be finished at a minimum cost. Of course, there are many more applications. A second generalization, allowing several sources and sinks, is more due to technical reasons. We introduce the general problem and an important special case in Section 9.1. In Section 9.2 we prove optimality criteria that are the basis of the minimum cost flow algorithms presented in Sections 9.3, 9.4, 9.5 and 9.6. Most of these use algorithms of Chapter 7 for finding a minimum mean cycle or a shortest path as a subroutine. Section 9.7 concludes this chapter with an application to time-dependent flows.

9.1 Problem Formulation We are again given a digraph G with capacities u : E(G) → R+ , but in addition numbers c : E(G) → R indicating the cost of each edge. Furthermore, we allow several sources and sinks: Definition 9.1. Given P a digraph G, capacities u : E(G) → R+ , and numbers b : function f : E(G) → V (G) → R with v∈V (G) b(v) = 0, a b-flow P in (G, u) is a P R+ with f (e) ≤ u(e) for all e ∈ E(G) and e∈δ + (v) f (e)− e∈δ − (v) f (e) = b(v) for all v ∈ V (G). Thus a b-flow with b ≡ 0 is a circulation. b(v) is called the balance of vertex v. |b(v)| is sometimes called the supply (if b(v) > 0) or the demand (if b(v) < 0) of v. Vertices v with b(v) > 0 are called sources, those with b(v) < 0 sinks. Note that a b-flow can be found by any algorithm for the M AXIMUM F LOW P ROBLEM: Just add two vertices s and t and edges (s, v), (v, t) with capacities u((s, v)) := max{0, b(v)} and P u((v, t)) := max{0, −b(v)} for all v ∈ V (G) to G. Then any s-t-flow of value v∈V (G) u((s, v)) in the resulting network corresponds to a b-flow in G. Thus a criterion for the existence of a b-flow can be derived from the Max-Flow-Min-Cut Theorem 8.6 (see Exercise 2). The problem is to find a minimum cost b-flow:

200

9 Minimum Cost Flows

M INIMUM C OST F LOW P ROBLEM Instance: Task:

A digraph P G, capacities u : E(G) → R+ , numbers b : V (G) → R with v∈V (G) b(v) = 0, and weights c : E(G) → R. P Find a b-flow f whose cost c( f ) := e∈E(G) f (e)c(e) is minimum (or decide that none exists).

Sometimes one also allows infinite capacities. In this case an instance can be unbounded, but this can be checked in advance easily; see Exercise 5. The M INIMUM C OST F LOW P ROBLEM is quite general and has a couple of interesting special cases. The uncapacitated case (u ≡ ∞) is sometimes called the transshipment problem. An even more restricted problem, also known as the transportation problem, has been formulated quite early by Hitchcock [1941] and others:

H ITCHCOCK P ROBLEM

.

Instance:

A digraph G with V (G) = A ∪ B and E(G) ⊆ A × B. Supplies b(v) P ≥ 0 for v ∈ A and demands −b(v) ≥ 0 for v ∈ B with v∈V (G) b(v) = 0. Weights c : E(G) → R.

Task:

Find a b-flow f in (G, ∞) of minimum cost (or decide that none exists).

In the H ITCHCOCK P ROBLEM it causes no loss of generality to assume that c is nonnegative: Adding a constant α to P each weight increases the cost of each b-flow by the same amount, namely by α v∈A b(v). Often only the special case where c is nonnegative and E(G) = A × B is considered. Obviously, any instance of the H ITCHCOCK P ROBLEM can be written as an instance of the M INIMUM C OST F LOW P ROBLEM on a bipartite graph with infinite capacities. It is less obvious that any instance of the M INIMUM C OST F LOW P ROB LEM can be transformed to an equivalent (but larger) instance of the H ITCHCOCK P ROBLEM: Lemma 9.2. (Orden [1956], Wagner [1959]) An instance of the M INIMUM C OST F LOW P ROBLEM with n vertices and m edges can be transformed to an equivalent instance of the H ITCHCOCK P ROBLEM with n + m vertices and 2m edges. Proof: Let (G, u, b, c) be an instance of the M INIMUM C OST F LOW P ROBLEM. We define an equivalent instance (G 0 , A0 , B 0 , b0 , c0 ) of the H ITCHCOCK P ROBLEM as follows: Let A0 := E(G), B 0 := V (G) and G 0 := (A0 ∪ B 0 , E 1 ∪ E 2 ), where E 1 := {((x, y), x) : (x, y) ∈ E(G)} and E 2 := {((x, y), y) : (x, y) ∈ E(G)}. Let c0 ((e, x)) := 0 for (e, x) ∈ E 1 and c0 ((e, y)) := c(e) for (e, y) ∈ E 2 . Finally let b0 (e) := u(e) for e ∈ E(G) and X u(e) for x ∈ V (G). b0 (x) := b(x) − + e∈δG (x)

9.2 An Optimality Criterion b (e1 ) = 5

b(x) = 4

e1

b (e2 ) = 4 e2

b(y) = −1

e3

b(z) = −3

b (e3 ) = 7

u(e1 ) = 5, u(e2 ) = 4, u(e3 ) = 7

0 c(e1 ) 0 c(e2 ) c(e3 ) 0

201

b (x) = −1

b (y) = −5

b (z) = −10

u ≡ ∞ Fig. 9.1.

For an example, see Figure 9.1. We prove that both instances are equivalent. Let f be a b-ﬂow in (G, u). Deﬁne f ((e, y)) := f (e) and f ((e, x)) := u(e) − f (e) for e = (x, y) ∈ E(G). Obviously f is a b -ﬂow in G with c ( f ) = c( f ). Conversely, if f is a b -ﬂow in G , then f ((x, y)) := f (((x, y), y)) deﬁnes a b-ﬂow in G with c( f ) = c ( f ). The above proof is due to Ford and Fulkerson [1962].

9.2 An Optimality Criterion In this section we prove some simple results, in particular an optimality criterion, which will be the basis for the algorithms in the subsequent sections. We again use the concepts of residual graphs and augmenting paths. We extend the weights ↔

←

c to G by deﬁning c( e ) := −c(e) for each edge e ∈ E(G). Our deﬁnition of a residual graph has the advantage that the weight of an edge in a residual graph G f is independent of the ﬂow f . Deﬁnition 9.3. Given a digraph G with capacities and a b-ﬂow f , an f-augmenting cycle is a circuit in G f . The following simple observation will prove useful: Proposition 9.4. Let G be a digraph with capacities u : E(G) → R+ . Let f and ↔

f be b-ﬂows in (G, u). Then g : E(G) → R+ deﬁned by g(e) := max{0, f (e) − ↔

←

f (e)} and g( e ) := max{0, f (e) − f (e)} for e ∈ E(G) is a circulation in G. Furthermore, g(e) = 0 for all e ∈ / E(G f ) and c(g) = c( f ) − c( f ). ↔

Proof: At each vertex v ∈ V (G) we have

202

9 Minimum Cost Flows

X

X

g(e) −

+ e∈δ↔ (v)

− e∈δ↔ (v)

G

X

g(e) =

( f 0 (e) − f (e)) −

+ e∈δG (v)

G

X

( f 0 (e) − f (e))

− e∈δG (v)

= b(v) − b(v) = 0, ↔

so g is a circulation in G. ↔

For e ∈ E(G) \ E(G f ) we consider two cases: If e ∈ E(G) then f (e) = u(e) ← and thus f 0 (e) ≤ f (e), implying g(e) = 0. If e = e0 for some e0 ∈ E(G) then ← f (e0 ) = 0 and thus g(e0 ) = 0. The last statement is easily verified: X X X c(g) = c(e)g(e) = c(e) f 0 (e) − c(e) f (e) = c( f 0 ) − c( f ). ↔

e∈E(G)

e∈E(G)

e∈E(G)

Just as Eulerian graphs can be partitioned into circuits, circulations can be decomposed into flows on single circuits: Proposition 9.5. (Ford and Fulkerson [1962]) For any circulation f in a digraph G there is a family C of atP most |E(G)| circuits in G and positive numbers h(C) (C ∈ C) such that f (e) = {h(C) : C ∈ C, e ∈ E(C)} for all e ∈ E(G). Proof: This is a special case of Theorem 8.8.

Now we can prove an optimality criterion: Theorem 9.6. (Klein [1967]) Let (G, u, b, c) be an instance of the M INIMUM C OST F LOW P ROBLEM. A b-flow f is of minimum cost if and only if there is no f -augmenting cycle with negative total weight. Proof: If there is an f -augmenting cycle C with weight γ < 0, we can augment f along C by some ε > 0 and get a b-flow f 0 with cost decreased by −γ ε. So f is not a minimum cost flow. If f is not a minimum cost b-flow, there is another b-flow f 0 with smaller cost. Consider g as defined in Proposition 9.4. Then g is a circulation with c(g) < 0. By Proposition 9.5, g can be decomposed into flows on single circuits. Since g(e) = 0 for all e ∈ / E(G f ), all these circuits are f -augmenting. At least one of them must have negative total weight, proving the theorem. This result goes back essentially to Tolsto˘ı [1930] and has been rediscovered several times in different forms. One equivalent formulation is the following: Corollary 9.7. (Ford and Fulkerson [1962]) Let (G, u, b, c) be an instance of the M INIMUM C OST F LOW P ROBLEM. A b-flow f is of minimum cost if and only if there exists a feasible potential for (G f , c).

9.3 Minimum Mean Cycle-Cancelling Algorithm

203

Proof: By Theorem 9.6 f is a minimum cost b-flow if and only if G f contains no negative circuit. By Theorem 7.7 there is no negative circuit in (G f , c) if and only if there exists a feasible potential. Feasible potentials can also be regarded as solutions of the linear programming dual of the M INIMUM C OST F LOW P ROBLEM. This is shown by the following different proof of the above optimality criterion: Second Proof of Corollary 9.7: We write the M INIMUM C OST F LOW P ROB LEM as a maximization problem and consider the LP X −c(e)xe max e∈E(G)

s.t.

X

xe −

e∈δ + (v)

X

xe = b(v)

(v ∈ V (G))

xe ≤ u(e) xe ≥ 0

(e ∈ E(G)) (e ∈ E(G))

e∈δ − (v)

(9.1)

and its dual min

X

b(v)yv +

v∈V (G)

s.t.

X

u(e)z e

e∈E(G)

yv − yw + z e ≥ −c(e) ze ≥ 0

(e = (v, w) ∈ E(G)) (e ∈ E(G))

(9.2)

Let x be any b-flow, i.e. any feasible solution of (9.1). By Corollary 3.23 x is optimum if and only if there exists a feasible dual solution (y, z) of (9.2) such that x and (y, z) satisfy the complementary slackness conditions z e (u(e)− xe ) = 0 and xe (c(e)+ z e + yv − yw ) = 0 for all e = (v, w) ∈ E(G). So x is optimum if and only if there exists a pair of vectors (y, z) with 0 = −z e ≤ c(e) + yv − yw c(e) + yv − yw = −z e ≤ 0

for e = (v, w) ∈ E(G) with xe < u(e)

and

for e = (v, w) ∈ E(G) with xe > 0.

This is equivalent to the existence of a vector y such that c(e) + yv − yw ≥ 0 for all residual edges e = (v, w) ∈ E(G x ), i.e. to the existence of a feasible potential y for (G x , c).

9.3 Minimum Mean Cycle-Cancelling Algorithm Note that Klein’s Theorem 9.6 already suggests an algorithm: first find any b-flow (using a max-flow algorithm as described above), and then successively augment along negative weight augmenting cycles until no more exist. We must however

204

9 Minimum Cost Flows

be careful in choosing the cycle if we want to have polynomial running time (see Exercise 7). A good strategy is to choose an augmenting cycle with minimum mean weight each time:

M INIMUM M EAN C YCLE -C ANCELLING A LGORITHM Input:

A digraph P G, capacities u : E(G) → R+ , numbers b : V (G) → R with v∈V (G) b(v) = 0, and weights c : E(G) → R.

Output:

A minimum cost b-flow f .

1

Find a b-flow f .

2

Find a circuit C in G f whose mean weight is minimum. If C has nonnegative total weight (or G f is acyclic) then stop.

3

Compute γ := min u f (e). Augment f along C by γ . e∈E(C)

Go to . 2 As described in Section 9.1, 1 can be implemented with any algorithm for the M AXIMUM F LOW P ROBLEM. 2 can be implemented with the algorithm presented in Section 7.3. We shall now prove that this algorithm terminates after a polynomial number of iterations. The proof will be similar to the one in Section 8.3. Let µ( f ) denote the minimum mean weight of a circuit in G f . Then Theorem 9.6 says that a b-flow f is optimum if and only if µ( f ) ≥ 0. We first show that µ( f ) is non-decreasing throughout the algorithm. Moreover, we can show that it is strictly increasing with every |E(G)| iterations. As usual we denote by n and m the number of vertices and edges of G, respectively. Lemma 9.8. Let f 1 , f 2 , . . . , f t be a sequence of b-flows such that for i = 1, . . . , t − 1 we have µ( f i ) < 0 and f i+1 results from f i by augmenting along Ci , where Ci is a circuit of minimum mean weight in G fi . Then (a) µ( f k ) ≤ µ( f k+1 ) for all k. n µ( fl ) for all k < l such that Ck ∪ Cl contains a pair of reverse (b) µ( f k ) ≤ n−2 edges. Proof: (a): Let f k , f k+1 be two consecutive flows in this sequence. Consider the . Eulerian graph H resulting from (V (G), E(Ck ) ∪ E(Ck+1 )) by deleting pairs of reverse edges. (Edges appearing both in Ck and Ck+1 are counted twice.) H is a subgraph of G fk because each edge in E(G fk+1 ) \ E(G fk ) must be the reverse of an edge in E(Ck ). Since H is Eulerian, it can be decomposed into circuits, and each of these circuits has mean weight at least µ( f k ). So c(E(H )) ≥ µ( f k )|E(H )|. Since the total weight of each pair of reverse edges is zero, c(E(H )) = c(E(Ck )) + c(E(Ck+1 )) = µ( f k )|E(Ck )| + µ( f k+1 )|E(Ck+1 )|. Since |E(H )| ≤ |E(Ck )| + |E(Ck+1 )|, we conclude

9.3 Minimum Mean Cycle-Cancelling Algorithm

205

µ( f k )(|E(Ck )| + |E(Ck+1 )|) ≤ µ( f k )|E(H )| ≤ c(E(H )) = µ( f k )|E(Ck )| + µ( f k+1 )|E(Ck+1 )|, implying µ( f k+1 ) ≥ µ( f k ). (b): By (a) it is enough to prove the statement for those k, l such that for k < i < l, Ci ∪ Cl contains no pair of reverse edges. As in the proof of (a), consider the Eulerian graph H resulting from (V (G), . E(Ck ) ∪ E(Cl )) by deleting pairs of reverse edges. H is a subgraph of G fk because any edge in E(Cl ) \ E(G fk ) must be the reverse of an edge in one of Ck , Ck+1 , . . . , Cl−1 . But – due to the choice of k and l – only Ck among these contains the reverse of an edge of Cl . So as in (a) we have c(E(H )) ≥ µ( f k )|E(H )| and c(E(H )) = µ( f k )|E(Ck )| + µ( fl )|E(Cl )|. Since |E(H )| ≤ |E(Ck )| + n−2 n |E(Cl )| (we deleted at least two edges) we get n−2 |E(Cl )| ≤ µ( f k )|E(H )| µ( f k ) |E(Ck )| + n ≤ c(E(H )) = µ( f k )|E(Ck )| + µ( fl )|E(Cl )|, implying µ( f k ) ≤

n n−2

µ( fl ).

Corollary 9.9. During the execution of the M INIMUM M EAN C YCLE -C ANCELLING A LGORITHM, |µ( f )| decreases by at least a factor of 12 with every mn iterations. Proof: Let Ck , Ck+1 , . . . , Ck+m be the augmenting cycles in consecutive iterations of the algorithm. Since each of these circuits contains one edge as a bottleneck edge (an edge removed afterwards from the residual graph), there must be two of these circuits, say Ci and C j (k ≤ i < j ≤ k + m) whose union contains a pair of reverse edges. By Lemma 9.8 we then have n n µ( f k ) ≤ µ( f i ) ≤ µ( f j ) ≤ µ( f k+m ). n−2 n−2 with every m iterations. The So |µ( f )| decreases by at least a factor of n−2 n n n−2 −2 corollary follows from this because of < e < 12 . n This already proves that the algorithm runs in polynomial time provided that all edge costs are integral: |µ( f )| is at most |cmin | at the beginning, where cmin is the minimum cost of any edge, and decreases by at least a factor of 21 with every mn iterations. So after O(mn log(n|cmin |)) iterations, µ( f ) is greater than − n1 . If the edge costs are integral, this implies µ( f ) ≥ 0 and the algorithm stops. So by Corollary 7.14, the running time is O m 2 n 2 log(n|cmin |) . Even better, we can also derive a strongly polynomial running time for the M INIMUM C OST F LOW P ROBLEM (first obtained by Tardos [1985]):

206

9 Minimum Cost Flows

Theorem 9.10. (Goldberg and Tarjan [1989]) The M INIMUM M EAN C YCLE C ANCELLING A LGORITHM runs in O m 3 n 2 log n time. Proof: We show that every mn(log n + 1) iterations at least one edge is ﬁxed, i.e. the ﬂow on this edge will not change anymore. Therefore there are at most O m 2 n log n iterations. Using Corollary 8.15 for 1 and Corollary 7.14 for 2 then proves the theorem. Let f be the ﬂow at some iteration, and let f be the ﬂow mn(log n + 1) iterations later. Deﬁne weights c by c (e) := c(e) − µ( f ) (e ∈ E(G f )). Let π be a feasible potential of (G f , c ) (which exists by Theorem 7.7). We have 0 ≤ cπ (e) = cπ (e) − µ( f ), so cπ (e) ≥ µ( f )

for all e ∈ E(G f ).

(9.3)

Now let C be the circuit of minimum mean weight in G f that is chosen in the algorithm to augment f . Since by Corollary 9.9 µ( f ) ≤ 2log n+1 µ( f ) ≤ 2nµ( f ) (see Figure 9.2), we have cπ (e) = c(e) = µ( f )|E(C)| ≤ 2nµ( f )|E(C)|. e∈E(C)

e∈E(C)

/ E(G f ). So let e0 = (x, y) ∈ E(C) with cπ (e0 ) ≤ 2nµ( f ). By (9.3) we have e0 ∈ µ( f )

2nµ( f )

µ( f )

0

Fig. 9.2.

Claim: For any b-ﬂow f with e0 ∈ E(G f ) we have µ( f ) < µ( f ). By Lemma 9.8(a) the claim implies that e0 will never be in the residual graph ← anymore, i.e. e0 and e0 are ﬁxed mn(log n + 1) iterations after e0 is used in C. This completes the proof. To prove the claim, let f be a b-ﬂow with e0 ∈ E(G f ). We apply Proposition 9.4 to f and f and obtain a circulation g with g(e) = 0 for all e ∈ / E(G f ) and ←

g(e0 ) > 0 (because e0 ∈ E(G f ) \ E(G f )). By Proposition 9.5, g can be written as the sum of ﬂows on f -augmenting ← ← cycles. One of these circuits, say W , must contain e0 . By using cπ (e0 ) = −cπ (e0 ) ≥ ← −2nµ( f ) and applying (9.3) to all e ∈ E(W ) \ {e0 } we obtain a lower bound for the total weight of W : cπ (e) ≥ −2nµ( f ) + (n − 1)µ( f ) > −nµ( f ). c(E(W )) = e∈E(W )

9.4 Successive Shortest Path Algorithm

207

But the reverse of W is an f 00 -augmenting cycle (this can be seen by exchanging the roles of f 0 and f 00 ), and its total weight is less than nµ( f 0 ). This means that G f 00 contains a circuit whose mean weight is less than µ( f 0 ), and so the claim is proved.

9.4 Successive Shortest Path Algorithm The following theorem gives rise to another algorithm: Theorem 9.11. (Jewell [1958], Iri [1960], Busacker and Gowen [1961]) Let (G, u, b, c) be an instance of the M INIMUM C OST F LOW P ROBLEM, and let f be a minimum cost b-flow. Let P be a shortest (with respect to c) s-t-path in G f (for some s and t). Let f 0 be a flow obtained when augmenting f along P by at most the minimum residual capacity on P. Then f 0 is a minimum cost b0 -flow (for some b0 ). Proof: f 0 is a b0 -flow for some b0 . Suppose f 0 is not a minimum cost b0 -flow. Then by Theorem 9.6 there is a circuit C in G f 0 with negative total weight. Consider . the graph H resulting from (V (G), E(C) ∪ E(P)) by deleting pairs of reverse edges. (Again, edges appearing both in C and P are taken twice.) For any edge e ∈ E(G f 0 ) \ E(G f ), the reverse of e must be in E(P). Therefore E(H ) ⊆ E(G f ). We have c(E(H )) = c(E(C)) + c(E(P)) < c(E(P)). Furthermore, H is the union of an s-t-path and some circuits. But since E(H ) ⊆ E(G f ), none of the circuits can have negative weight (otherwise f would not be a minimum cost b-flow). Therefore H , and thus G f , contains an s-t-path of less weight than P, contradicting the choice of P. If the weights are conservative, we can start with f ≡ 0 as an optimum circulation (b-flow with b ≡ 0). Otherwise we can initially saturate all edges of negative cost and bounded capacity. This changes the b-values but guarantees that there is no negative augmenting cycle (i.e. c is conservative for G f ) unless the instance is unbounded.

S UCCESSIVE S HORTEST PATH A LGORITHM Input:

A digraph P G, capacities u : E(G) → R+ , numbers b : V (G) → R with v∈V (G) b(v) = 0, and conservative weights c : E(G) → R.

Output:

A minimum cost b-flow f .

1

Set b0 := b and f (e) := 0 for all e ∈ E(G).

2

If b0 = 0 then stop, else: Choose a vertex s with b0 (s) > 0. Choose a vertex t with b0 (t) < 0 such that t is reachable from s in G f . If there is no such t then stop. (There exists no b-flow.)

208

3

4

9 Minimum Cost Flows

Find an s-t-path P in G f of minimum weight. Compute γ := min min u f (e), b0 (s), −b0 (t) . e∈E(P)

Set b0 (s) := b0 (s) − γ and b0 (t) := b0 (t) + γ . Augment f along P by γ . Go to . 2 If we allow arbitrary capacities, we have the same problems as with the F ORD F ULKERSON A LGORITHM (see Exercise 2 of Chapter 8; set all costs to zero). So henceforth we assume u and P b to be integral. Then it is clear that the algorithm stops after at most B := 12 v∈V (G) |b(v)| augmentations. By Theorem 9.11, the resulting flow is optimum if the initial zero flow is optimum. This is true if and only if c is conservative. We remark that if the algorithm decides that there is no b-flow, this decision is indeed correct. This is an easy observation, left as Exercise 13. Each augmentation requires a shortest path computation. Since negative weights occur, we have to use the M OORE -B ELLMAN -F ORD A LGORITHM whose running time is O(nm) (Theorem 7.5), so the overall running time will be O(Bnm). However, as in the proof of Theorem 7.9, it can be arranged that (except at the beginning) the shortest paths are computed in a graph with nonnegative weights: Theorem 9.12. (Tomizawa [1971], Edmonds and Karp [1972]) For integral capacities and supplies, the S UCCESSIVE S HORTEST PATH A LGORITHM can bePimplemented with a running time of O (nm + B(m + n log n)), where B = 1 v∈V (G) |b(v)|. 2 Proof: It is convenient to assume that there is only one source s. Otherwise we introduce a new vertex s and edges (s, v) with capacity max{0, b(v)} and zero cost for all v ∈ V (G). Then we can set b(s) := B and b(v) := 0 for each former source v. In this way we obtain an equivalent problem with only one source. Moreover, we may assume that every vertex is reachable from s (other vertices can be deleted). We introduce a potential πi : V (G) → R for each iteration i of the S UCCESSIVE S HORTEST PATH A LGORITHM. We start with any feasible potential π0 of (G, c). By Corollary 7.8, this exists and can be computed in O(mn) time. Now let f i−1 be the flow before iteration i. Then the shortest path computation in iteration i is done with the reduced costs cπi−1 instead of c. Let li (v) denote the length of a shortest s-v-path in G fi−1 with respect to the weights cπi−1 . Then we set πi (v) := πi−1 (v) + li (v). We prove by induction on i that πi is a feasible potential for (G fi , c). This is clear for i = 0. For i > 0 and any edge e = (x, y) ∈ E(G fi−1 ) we have (by definition of li and the induction hypothesis) li (y) ≤ li (x) + cπi−1 (e) = li (x) + c(e) + πi−1 (x) − πi−1 (y), so cπi (e) = c(e) + πi (x) − πi (y) = c(e) + πi−1 (x) + li (x) − πi−1 (y) − li (y) ≥ 0.

9.4 Successive Shortest Path Algorithm

209

For any edge e = (x, y) ∈ Pi (where Pi is the augmenting path in iteration i) we have li (y) = li (x) + cπi−1 (e) = li (x) + c(e) + πi−1 (x) − πi−1 (y), so cπi (e) = 0, and the reverse edge of e also has zero weight. Since each edge in E(G fi ) \ E(G fi−1 ) is the reverse of an edge in Pi , cπi is indeed a nonnegative weight function on E(G fi ). We observe that, for any i and any t, the shortest s-t-paths with respect to c are precisely the shortest s-t-paths with respect to cπi , because cπi (P) − c(P) = πi (s) − πi (t) for any s-t-path P. Hence we can use D IJKSTRA ’ S A LGORITHM – which runs in O (m + n log n) time when implemented with a Fibonacci heap by Theorem 7.4 – for all shortest path computations except the initial one. Since we have at most B iterations, we obtain an overall running time of O (nm + B(m + n log n)). Note that (in contrast to many other problems, e.g. the M AXIMUM F LOW P ROBLEM) we cannot assume without loss of generality that the input graph is simple when considering the M INIMUM C OST F LOW P ROBLEM. The running time of Theorem 9.12 is still exponential unless B is known to be small. If B = O(n), this is the fastest algorithm known. For an application, see Section 11.1. In the rest of this section we show how to modify the algorithm in order to reduce the number of shortest path computations. We only consider the case when all capacities are infinite. By Lemma 9.2 each instance of the M INIMUM C OST F LOW P ROBLEM can be transformed to an equivalent instance with infinite capacities. The basic idea – due to Edmonds and Karp [1972] – is the following. In early iterations we consider only augmenting paths where γ – the amount of flow that can be pushed – is large. We start with γ = 2blog bmax c and reduce γ by a factor of two if no more augmentations by γ can be done. After blog bmax c + 1 iterations we have γ = 1 and stop (we again assume b to be integral). Such a scaling technique has proved useful for many algorithms (see also Exercise 14). A detailed description of the first scaling algorithm reads as follows:

C APACITY S CALING A LGORITHM Input:

Output:

1

A digraph G with infinite P capacities u(e) = ∞ (e ∈ E(G)), numbers b : V (G) → Z with v∈V (G) b(v) = 0, and conservative weights c : E(G) → R. A minimum cost b-flow f .

Set b0 := b and f (e) := 0 for all e ∈ E(G). Set γ = 2blog bmax c , where bmax = max{b(v) : v ∈ V (G)}.

210

2

3

4

5

9 Minimum Cost Flows

If b0 = 0 then stop, else: Choose a vertex s with b0 (s) ≥ γ . Choose a vertex t with b0 (t) ≤ −γ such that t is reachable from s in G f . If there is no such s or t then go to . 5 Find an s-t-path P in G f of minimum weight. Set b0 (s) := b0 (s) − γ and b0 (t) := b0 (t) + γ . Augment f along P by γ . Go to . 2 If γ = 1 then stop. (There exists no b-flow.) Else set γ := γ2 and go to . 2

Theorem 9.13. (Edmonds and Karp [1972]) The C APACITY S CALING A LGO RITHM correctly solves the M INIMUM C OST F LOW P ROBLEM for integral b, infinite capacities and conservative weights. It can be implemented to run in O(n(m + n log n) log bmax ) time, where bmax = max{b(v) : v ∈ V (G)}. Proof: As above, the correctness follows directly from Theorem 9.11. Note that at any time, the residual capacity of any edge is either infinite or an integral multiple of γ . To establish the running time, we call the period in which γ remains constant a phase. We prove that there are less than 4n augmentations within each phase. Suppose this is not true. For some value of γ , let f and g be the flow at the beginning and at the end P of the γ -phase, respectively. g − f can be regarded as a b00 -flow in G f , where x∈V (G) |b00 (x)| ≥ 8nγ . Let S := {x ∈ V (G) : b00 (x) > 0}, S + := {x ∈ V (G) : b00 (x) ≥ 2γ }, T := {x ∈ V (G) : b00 (x) < 0}, T + := {x ∈ V (G) : b00 (x) ≤ −2γ }. If there had been a path from S + to T + in G f , the 2γ -phase would have continued. Therefore the total b00 -value of all sinks reachable from S + in G f is greater than n(−2γ ). Therefore (note that there exists a b00 -flow in G f ) P 00 x∈S + b (x) < 2nγ . Now we have X X X X |b00 (x)| = 2 b00 (x) = 2 b00 (x) + b00 (x) x∈V (G)

x∈S

x∈S +

x∈S\S +

< 2(2nγ + 2nγ ) = 8nγ , a contradiction. This means that the total number of shortest path computations is O(n log bmax ). Combining this with the technique of Theorem 9.12 we obtain the O(mn + n log bmax (m + n log n)) bound. This was the first polynomial-time algorithm for the M INIMUM C OST F LOW P ROBLEM. By some further modifications we can even obtain a strongly polynomial running time. This is the subject of the next section.

9.5 Orlin’s Algorithm

211

9.5 Orlin’s Algorithm The C APACITY S CALING A LGORITHM of the previous section can be improved further. A basic idea is that if an edge carries more than 8nγ units of flow at any stage of the C APACITY S CALING A LGORITHM, it may be contracted. Namely, observe that such an edge will always keep a positive flow (and thus zero reduced cost with respect to any feasible potential in the residual graph): there are at most 4n more augmentations by γ , another 4n by γ2 and so on; hence the total amount of flow moved in the rest of the algorithm is less than 8nγ . We shall describe O RLIN ’ S A LGORITHM without explicitly using contraction. This simplifies the description, especially from the point of view of implementing the algorithm. A set F keeps track of the edges (and their reverse edges) that can be contracted. A representative is chosen out of each connected component of (V (G), F). The algorithm maintains the property that the representative of a connected component is its only non-balanced vertex. For any vertex x, r (x) denotes the representative of the connected component of (V (G), F) containing x. O RLIN ’ S A LGORITHM does not require that b is integral. However, it can deal with uncapacitated problems only (but recall Lemma 9.2).

O RLIN ’ S A LGORITHM Input:

A digraph G with infinite P capacities u(e) = ∞ (e ∈ E(G)), numbers b : V (G) → R with v∈V (G) b(v) = 0, and conservative weights c : E(G) → R. A minimum cost b-flow f .

Output:

1

Set b0 := b and f (e) := 0 for all e ∈ E(G). Set r (v) := v for all v ∈ V (G). Set F := ∅. Set γ = max |b0 (v)|. v∈V (G)

b0

2

If

3

Choose a vertex s with b0 (s) > n−1 n γ. If there is no such s then go to . 4 Choose a vertex t with b0 (t) < − n1 γ such that t is reachable from s in G f . If there is no such t then stop. (There exists no b-flow.) Go to . 5

4

Choose a vertex t with b0 (t) < − n−1 n γ. If there is no such t then go to . 6 Choose a vertex s with b0 (s) > n1 γ such that t is reachable from s in G f . If there is no such s then stop. (There exists no b-flow.) Find an s-t-path P in G f of minimum weight. Set b0 (s) := b0 (s) − γ and b0 (t) := b0 (t) + γ . Augment f along P by γ . Go to . 2

5

= 0 then stop.

212

6

9 Minimum Cost Flows

If f (e) = 0 for all e ∈ E(G) \ F then set γ := min else set γ :=

7

8

γ 2.

γ , max |b0 (v)| , 2 v∈V (G)

For all e = (x, y) ∈ E(G) \ F with r (x) 6= r (y) and f (e) > 8nγ do: ← Set F := F ∪ {e, e }. Let x 0 := r (x) and y 0 := r (y). Let Q be the x 0 -y 0 -path in F. If b0 (x 0 ) > 0 then augment f along Q by b0 (x 0 ), else augment f along the reverse of Q by −b0 (x 0 ). Set b0 (y 0 ) := b0 (y 0 ) + b0 (x 0 ) and b0 (x 0 ) := 0. Set r (z) := y 0 for all vertices z reachable from y 0 in F. Go to . 2

This algorithm is due to Orlin [1993]. See also (Plotkin and Tardos [1990]). Let us first prove its correctness. Let us call the time between two changes of γ a phase. Lemma 9.14. O RLIN ’ S A LGORITHM solves the uncapacitated M INIMUM C OST F LOW P ROBLEM with conservative weights correctly. At any stage f is a minimumcost (b − b0 )-flow. Proof: We first prove that f is always a (b − b0 )-flow. In particular, we have to show that f is always nonnegative. To prove this, we first observe that at any time the residual capacity of any edge not in F is either infinite or an integral multiple of γ . Moreover we claim that every edge e ∈ F always has positive residual capacity. To see this, observe that any phase consists of at most n − 1 augmentations by less than 2 n−1 7 and at most 2n augmentations by γ in n γ in

; 5 hence the total amount of flow moved after e has become a member of F in the γ -phase is less than 8nγ . Hence f is always nonnegative and thus it is always a (b − b0 )-flow. We now claim that f is always a minimum cost (b − b0 )-flow and that each v-w-path in F is a shortest v-w-path in G f . Indeed, the first statement implies the second one, since by Theorem 9.6 for a minimum cost flow f there is no negative circuit in G f . Now the claim follows from Theorem 9.11: P in 5 and Q in 7 are both shortest paths. We finally show that if the algorithm stops in 3 or 4 with b0 6= 0, then there is indeed no b-flow. Suppose the algorithm stops in , 3 implying that there is a vertex s with b0 (s) > n−1 γ , but that no vertex t with b0 (t) < − n1 γ is reachable n from s in G f . ThenPlet R be the set of vertices reachable from s in G f . Since f is a (b − b0 )-flow, x∈R (b(x) − b0 (x)) = 0. Therefore we have X X X X X b(x) = (b(x)−b0 (x))+ b0 (x) = b0 (x) = b0 (s)+ b0 (x) > 0. x∈R

x∈R

x∈R

x∈R

x∈R\{s}

This proves that no b-flow exists. An analogous proof applies in the case that the algorithm stops in . 4

9.5 Orlin’s Algorithm

213

We now analyse the running time. Lemma 9.15. (Plotkin and Tardos [1990]) If at some stage of the algorithm |b0 (s)| > n−1 n γ for a vertex s, then the connected component of (V (G), F) containing s increases during the next d2 log n + log me + 4 phases. Proof: Let |b0 (s)| > n−1 n γ1 for a vertex s at the beginning of some phase of the algorithm where γ = γ1 . Let γ0 be the γ -value in the preceding phase, and γ2 the γ -value d2 log n + log me + 4 phases later. We have 12 γ0 ≥ γ1 ≥ 16n 2 mγ2 . Let b10 and f 1 be the b0 and f at the beginning of the γ1 -phase, respectively, and let b20 and f 2 be the b0 and f at the end of the γ2 -phase, respectively. Let S be the connected component of (V (G), F) containing s in the γ1 -phase, and suppose that this remains unchanged for the d2 log n + log me + 4 phases considered. Note that 7 guarantees b0 (v) = 0 for all vertices v with r (v) 6= v. 0 Hence b (v) = 0 for all v ∈ S \ {s} and X X X X b(x) − b10 (s) = (b(x) − b10 (x)) = f 1 (e) − f 1 (e). (9.4) x∈S

e∈δ + (S)

x∈S

We claim that

e∈δ − (S)

X 1 b(x) ≥ γ1 . n

(9.5)

x∈S

If γ1 < γ20 , then each edge not in F has zero flow, so the right-hand side of (9.4) P 1 is zero, implying x∈S b(x) = |b10 (s)| > n−1 n γ1 ≥ n γ1 . γ0 In the other case (γ1 = 2 ) we have 1 n−1 n−1 2 γ1 ≤ γ1 < |b10 (s)| ≤ γ0 = γ 0 − γ 1 . n n n n

(9.6)

Since the flow on any edge not in F is an integral multiple of γ0 , the right-hand side of (9.4) is also an integral multiple of γ0 . This together with (9.6) implies (9.5). S minus the total flow on Now consider the total f 2 -flow on edges leaving P edges entering S. Since f 2 is a (b − b20 )-flow, this is x∈S b(x) − b20 (s). Using (9.5) and |b20 (s)| ≤ n−1 n γ2 we obtain X X n−1 1 γ1 − γ2 | f 2 (e)| ≥ b(x) − |b20 (s)| ≥ n n + − e∈δ (S)∪δ (S)

x∈S

≥ (16nm − 1)γ2 > m(8nγ2 ). Thus there exists at least one edge e with exactly one end in S and f 2 (e) > 8nγ2 . By 7 of the algorithm, this means that S is increased. Theorem 9.16. (Orlin [1993]) O RLIN ’ S A LGORITHM solves the uncapacitated M INIMUM C OST F LOW P ROBLEM with conservative weights correctly in O(n log m (m + n log n)) time.

214

9 Minimum Cost Flows

Proof: The correctness has been proved above (Lemma 9.14). 7 takes O(mn) total time. Lemma 9.15 implies that the total number of phases is O(n log m). Moreover, it says the following: For a vertex s and a set S ⊆ V (G) there are at most d2 log n +log me+4 augmentations in 5 starting at s while S is the connected component of (V (G), F) containing s. Since all vertices v with r (v) 6= v have b0 (v) = 0 at any time, there are at most d2 log n + log me + 4 augmentations for each set S that is at some stage of the algorithm a connected component of F. Since the family of these sets is laminar, there are at most 2n − 1 such sets (Corollary 2.15) and thus O(n log m) augmentations in 5 altogether. Using the technique of Theorem 9.12, we obtain an overall running time of O (mn + (n log m)(m + n log n)). This is the best known running time for the uncapacitated M INIMUM C OST F LOW P ROBLEM. Theorem 9.17. (Orlin [1993]) The general M INIMUM C OST F LOW P ROBLEM can be solved in O (m log m(m + n log n)) time, where n = |V (G)| and m = |E(G)|. Proof: We apply the construction given in Lemma 9.2. Thus we have to solve an uncapacitated M INIMUM C OST F LOW P ROBLEM on a bipartite graph H with . V (H ) = A0 ∪ B 0 , where A0 = E(G) and B 0 = V (G). Since H is acyclic, an initial feasible potential can be computed in O(|E(H )|) = O(m) time. As shown above (Theorem 9.16), the overall running time is bounded by O(m log m) shortest ↔

path computations in a subgraph of H with nonnegative weights. Before we call D IJKSTRA ’ S A LGORITHM we apply the following operation to each vertex a ∈ A0 that is not an endpoint of the path we are looking for: add an edge (b, b0 ) for each pair of edges (b, a), (a, b0 ) and set its weight to the sum of the weights of (b, a) and (a, b0 ); finally delete a. Clearly the resulting instance of the S HORTEST PATH P ROBLEM is equivalent. Since each vertex in A0 has four ↔

incident edges in H , the resulting graph has O(m) edges and at most n +2 vertices. The preprocessing takes constant time per vertex, i.e. O(m). The same holds for ↔

the final computation of the path in H and of the distance labels of the deleted vertices. We get an overall running time of O ((m log m)(m + n log n)). This is the fastest known strongly polynomial algorithm for the general M IN C OST F LOW P ROBLEM. An algorithm which achieves the same running time but works directly on capacitated instances has been described by Vygen [2002]. IMUM

9.6 The Network Simplex Algorithm The M INIMUM C OST F LOW P ROBLEM is a special case of L INEAR P ROGRAM MING . By applying the S IMPLEX A LGORITHM and exploiting the special structure

9.6 The Network Simplex Algorithm

215

we arrive at what is called the N ETWORK S IMPLEX A LGORITHM. To make the connection clear, we first characterize the set of basic solutions (although we will not need this to prove correctness). Definition 9.18. Let (G, u, b, c) be an instance of the M INIMUM C OST F LOW P ROBLEM. A b-flow f in (G, u) is a spanning tree solution if (V (G), {e ∈ E(G) : 0 < f (e) < u(e)}) contains no undirected circuit. Proposition 9.19. An instance of the M INIMUM C OST F LOW P ROBLEM either has an optimum solution which is a spanning tree solution or has no optimum solution at all. Proof: Given an optimum solution f and an undirected circuit C in (V (G), {e ∈ E(G) : 0 < f (e) < u(e)}), we have two directed circuits C 0 and C 00 in G f with the same underlying undirected graph as C. Let be the minimum residual capacity in E(C 0 ) ∪ E(C 00 ). We can obtain two other feasible solutions f 0 and f 00 by augmenting f by along C 0 and C 00 , respectively. As 2c( f ) = c( f 0 ) + c( f 00 ), both f 0 and f 00 are also optimum solutions. At least one of them has fewer edges e with 0 < f (e) < u(e) than f , so after less than |E(G)| steps we arrive at an optimum spanning tree solution. Corollary 9.20. Let (G, u, b, c) be an instance of the M INIMUM C OST F LOW P ROBLEM. Then the basic solutions of x ∈ R E(G) : 0 ≤ xe ≤ u(e) (e ∈ E(G)), X e∈δ + (v)

xe −

X

xe = b(v) (v ∈ V (G))

e∈δ − (v)

are precisely the spanning tree solutions of (G, u, b, c). Proof: Proposition 9.19 shows that each basic solution is a spanning tree solution. For a spanning tree solution f we consider the inequalities xe ≥ 0P for e ∈ E(G) ≤ u(e) for e ∈ E(G) with f (e) = u(e), and with f (e) = 0, x e e∈δ + (v) x e − P x = b(v) for all v except one vertex of each connected component − e e∈δ (v) of (V (G), {e ∈ E(G) : 0 < f (e) < u(e)}). These |E(G)| inequalities are all satisfied by f with equality, and the submatrix corresponding to these inequalities is nonsingular. Thus f is a basic solution. In a spanning tree solution there are three types of edges: those with zero flow, those with saturated capacity, and those whose flow is positive but less than the capacity. Assuming that G is connected, we can extend the last set of edges to a connected spanning subgraph without undirected circuits (i.e., an oriented spanning tree; hence the name “spanning tree solution”). Definition 9.21. Let (G, u, b, c) be an instance of the M INIMUM C OST F LOW P ROBLEM where G is connected. A spanning tree structure is a quadruple

216

9 Minimum Cost Flows .

.

(r, T, L , U ) where r ∈ V (G), E(G) = T ∪ L ∪ U , |T | = |V (G)| − 1, and (V (G), T ) contains no undirected circuit. The b-flow associated with the spanning tree structure (r, T, L , U ) is defined by • • •

f (e) := 0 for e ∈ L, f (e) := u(e) , P P for e ∈ UP f (e) := v∈Ce b(v)+ e∈U ∩δ − (Ce ) u(e)− e∈U ∩δ + (Ce ) u(e) for e ∈ T , where for e = (v, w) we denote by Ce the connected component of (V (G), T \ {e}) containing v.

(r, T, L , U ) is called feasible if 0 ≤ f (e) ≤ u(e) for all e ∈ T . Call an edge (v, w) in T downward if v belongs to the undirected r -w-path in T , otherwise upward. (r, T, L , U ) is called strongly feasible if 0 < f (e) ≤ u(e) for each downward edge e ∈ T and 0 ≤ f (e) < u(e) for each upward edge e ∈ T . The unique function π : V (G) → R with π(r ) = 0 and cπ (e) = 0 for all e ∈ T is called the potential associated with the spanning tree structure (r, T, L , U ). b-flow f associated with a spanning tree structure satisfies P Evidently the P f (e) − + e∈δ (v) e∈δ − (v) f (e) = b(v) for all v ∈ V (G) (though it is not always a feasible b-flow). Moreover we note: Proposition 9.22. Given an instance (G, u, b, c) of the M INIMUM C OST F LOW P ROBLEM and a spanning tree structure (r, T, L , U ), the b-flow f and the potential π associated with it can be computed in O(m) and O(n) time, respectively. Moreover, f is integral whenever b and u are integral, and π is integral whenever c is integral. Proof: The potential associated with (r, T, L , U ) can be computed simply by applying the G RAPH S CANNING A LGORITHM to the edges of T and their reverse edges. The b-flow associated with (r, T, L , U ) can be computed in linear time by scanning the vertices in an order of nonincreasing distance from r . The integrality properties follow immediately from the definition. The N ETWORK S IMPLEX A LGORITHM maintains a strongly feasible spanning tree structure and works towards optimality. Note that the optimality criterion in Corollary 9.7 immediately implies: Proposition 9.23. Let (r, T, L , U ) be a feasible spanning tree structure and let π be the potential associated with it. Suppose that • •

cπ (e) ≥ 0 for all e ∈ L, and cπ (e) ≤ 0 for all e ∈ U .

Then (r, T, L , U ) is associated with an optimum b-flow.

↔

Note that π(v) is the length of the r -v-path in G that contains only edges of T ↔

or their reverse edges. For an edge e = (v, w) ∈ E(G) we define the fundamental

9.6 The Network Simplex Algorithm

217

circuit of e to consist of e and the w-v-path that contains only edges of T and their reverse edges. The vertex of C that is closest to r in T is called its peak. Hence for e = (v, w) ∈ / T , cπ (e) = c(e) + π(v) − π(w) is the cost of sending one unit of flow along the fundamental circuit of e. There are several ways to obtain an initial strongly feasible spanning tree structure. For example, one could compute any b-flow (by solving a M AXIMUM F LOW P ROBLEM), apply the procedure in the proof of Proposition 9.19, choose r arbitrarily, and define T, L , U according to the flow (adding suitable edges to T if necessary). Alternatively, one could apply “phase one” of the S IMPLEX M ETHOD. However, the simplest possibility is to introduce very expensive auxiliary edges with sufficient capacity between r and each other vertex: For each sink v ∈ V (G) \ {r } we introduce an edge (r, v) with capacity −b(v), and for each other vertex v ∈ V (G) \ {r } we introduce an edge (v, r ) with capacity b(v) + 1. The cost of each auxiliary edge should be high enough so that they never occur in an optimum solution, for example 1 + (|V (G)| − 1) maxe∈E(G) |c(e)| (Exercise 19). Then we can choose T as the set of all auxiliary edges, L as the set of all original edges, and U := ∅ to obtain an initial strongly feasible spanning tree structure.

N ETWORK S IMPLEX A LGORITHM Input: Output:

An instance (G, u, b, c) of the M INIMUM C OST F LOW P ROBLEM and a strongly feasible spanning tree structure (r, T, L , U ). An optimum solution f .

1

Compute the b-flow f and the potential π associated with (r, T, L , U ).

2

Let e ∈ L with cπ (e) < 0 or e ∈ U with cπ (e) > 0. If such an edge e does not exist then stop.

3

Let C be the fundamental circuit of e (if e ∈ L) or of e (if e ∈ U ). Let γ := cπ (e).

4

Let δ := mine0 ∈E(C) u f (e0 ), and let e0 be the last edge where the minimum is attained when traversing C in its orientation, starting at its peak. ← Let e0 ∈ E(G) such that e0 is e0 or e0 . Remove e from L or U . Set T := (T ∪ {e}) \ {e0 }. If e0 = e0 then insert e0 into U else insert e0 into L. Augment f by δ along C. Let X be the connected component of (V (G), T \ {e}) containing r . If e ∈ δ + (X ) then set π(v) := π(v) + γ for v ∈ V (G) \ X . If e ∈ δ − (X ) then set π(v) := π(v) − γ for v ∈ V (G) \ X . Go to . 2

5

6

←

We remark that 6 could be replaced simply by going back to , 1 as f and π computed in 6 are associated with the new spanning tree structure. Also note

218

9 Minimum Cost Flows

that e = e0 is possible; in this case X = V (G), and T , f and π do not change, but e moves from L to U or vice versa. Theorem 9.24. (Dantzig [1951], Cunningham [1976]) The N ETWORK S IMPLEX A LGORITHM stops after finitely many iterations and returns an optimum solution. Proof: First observe that 6 maintains the property that f and π are the b-flow and potential associated with (r, T, L , U ). Next we prove that the spanning tree structure is always strongly feasible. By the choice of δ we maintain the condition 0 ≤ f (e) ≤ u(e) for all e, and hence the spanning tree structure remains feasible. As the edges of the subpath of C from the head of e0 to the peak did not attain the minimum in , 4 they will continue to have positive residual capacity after the augmentation. For the edges of the subpath of C from the peak to the tail of e0 we have to make sure that their reverse edges have positive residual capacity after the augmentation. This is clear if δ > 0. Otherwise (if δ = 0) we have e 6= e0 , and the fact that the ← spanning tree structure was strongly feasible before implies that neither e nor e ← can belong to this subpath (i.e. e = e0 or δ − (X ) ∩ E(C) ∩ {e, e } 6= ∅) and that ← the reverse edges of the subpath of C from the peak to tail of e or e had positive residual capacity. By Proposition 9.23 the computed flow f is optimum when the algorithm terminates. We show that there are no two iterations with the same pair ( f, π), and hence each spanning tree structure occurs at most once. In each iteration the cost of the flow is reduced by |γ |δ. As γ 6= 0, we only need to consider iterations with δ = 0. Here the cost of the flow remains P constant. If e 6= e0 , then e ∈ L ∩ δ − (X ) or e ∈ U ∩ δ + (X ), and hence v∈V (G) π(v) strictly increases (by at least |γ |). Finally, if δ = 0 and e = e0 , then u(e) = 0, X = V (G), π remains constant, and |{e ∈ L : cπ (e) < 0}| + |{e ∈ U : cπ (e) > 0}| strictly decreases. This shows that no two iterations feature the same spanning tree structure. Although the N ETWORK S IMPLEX A LGORITHM is not a polynomial-time algorithm, it is quite efficient in practice. Orlin [1997] proposed a variant that runs in polynomial time. Polynomial-time dual network simplex algorithms were found by Orlin, Plotkin and Tardos [1993], and Armstrong and Jin [1997].

9.7 Flows Over Time We now consider flows over time (also sometimes called dynamic flows); i.e. the flow value on each edge may change over time, and flow entering an edge arrives at the endvertex after a specified delay:

9.7 Flows Over Time

219

Definition 9.25. Let (G, u, s, t) be a network with transit times l : E(G) → R+ and a time horizon T ∈ R+ . Then an s-t-flow over time f consists of a Lebesguemeasurable function f e : [0, T ] → R+ for each e ∈ E(G) with f e (τ ) ≤ u(e) for all τ ∈ [0, T ] and e ∈ E(G) and ex f (v, a) :=

X Z e∈δ − (v)

max{0,a−l(e)}

f e (τ )dτ −

0

X Z e∈δ + (v)

a

f e (τ )dτ ≥ 0

(9.7)

0

for all v ∈ V (G) \ {s} and a ∈ [0, T ]. f e (τ ) is called the rate of flow entering e at time τ (and leaving this edge l(e) time units later). (9.7) allows intermediate storage at vertices, like in s-t-preflows. It is natural to maximize the flow arriving at sink t:

M AXIMUM F LOW OVER T IME P ROBLEM Instance: Task:

A network (G, u, s, t). Transit times l : E(G) → R+ and a time horizon T ∈ R+ . Find an s-t-flow over time f such that value ( f ) := ex f (t, T ) is maximum.

Following Ford and Fulkerson [1958], we show that this problem can be reduced to the M INIMUM C OST F LOW P ROBLEM. Theorem 9.26. The M AXIMUM F LOW OVER T IME P ROBLEM can be solved in the same time as the M INIMUM C OST F LOW P ROBLEM. Proof: Given an instance (G, u, s, t, l, T ) as above, define a new edge e0 = (t, s) and G 0 := G + e0 . Set u(e0 ) := u(E(G)), c(e0 ) := −T and c(e) := l(e) for e ∈ E(G). Consider the instance (G 0 , u, 0, c) of the M INIMUM C OST F LOW P ROBLEM. Let f 0 be an optimum solution, i.e. a minimum cost (with respect to c) circulation in (G 0 , u). By Proposition 9.5, f 0 can be decomposed into flows on 0 circuits, i.e. there is Pa set C of circuits in G and positive numbers h : C → R+ 0 such that f (e) = {h(C) : C ∈ C, e ∈ E(C)}. We have c(C) ≤ 0 for all C ∈ C as f 0 is a minimum cost circulation. Let C ∈ C with c(C) < 0. C must contain e0 . For e = (v, w) ∈ E(C) \ {e0 }, let deC be the distance from s to v in (C, c). Set X f e∗ (τ ) := {h(C) : C ∈ C, c(C) < 0, e ∈ E(C), deC ≤ τ ≤ deC − c(C)} for e ∈ E(G) and τ ∈ [0, T ]. This defines an s-t-flow over time without intermediate storage (i.e. ex f (v, a) = 0 for all v ∈ V (G) \ {s, t} and all a ∈ [0, T ]). Moreover, X Z T −l(e) X ∗ value ( f ) = f e∗ (τ )dτ = − c(e) f 0 (e). e∈δ − (t)

0

e∈E(G 0 )

220

9 Minimum Cost Flows

We claim that f ∗ is optimum. To see this, let f be any s-t-flow over time, and set f e (τ ) := 0 for e ∈ E(G) and τ ∈ / [0, T ]. Let π(v) := dist(G 0 0 ,c) (s, v) for f

v ∈ V (G). As G 0f 0 contains no negative circuit (cf. Theorem 9.6), π is a feasible potential in (G 0f 0 , c). We have value ( f ) = ex f (t, T ) ≤

X

ex f (v, π(v))

v∈V (G)

because of (9.7), π(t) = T , π(s) = 0 and 0 ≤ π(v) ≤ T for all v ∈ V (G). Hence ! Z π(v) Z π(w)−l(e) X value ( f ) ≤ f e (τ )dτ − f e (τ )dτ 0

e=(v,w)∈E(G)

X

≤

0

(π(w) − l(e) − π(v))u(e)

e=(v,w)∈E(G):π(w)−l(e)>π(v)

X

=

(π(w) − l(e) − π(v)) f 0 (e)

e=(v,w)∈E(G)

X

=

(π(w) − c(e) − π(v)) f 0 (e)

e=(v,w)∈E(G 0 )

X

= −

c(e) f 0 (e)

e=(v,w)∈E(G 0 ) ∗

= value ( f )

Other flow over time problems are significantly more difficult. Hoppe and Tardos [2000] solved the so-called quickest transshipment problem (with several sources and sinks) with integral transit times using submodular function minimization (see Chapter 14). Finding minimum cost flows over time is NP-hard (Klinz and Woeginger [2004]). See Fleischer and Skutella [2007] for approximation algorithms and more information.

Exercises 1. Show that the M AXIMUM F LOW P ROBLEM can be regarded as a special case of the M INIMUM C OST F LOW P ROBLEM. 2. Let GPbe a digraph with capacities u : E(G) → R+ , and let b : V (G) → R with v∈V (G) b(v) = 0. Prove that there exists a b-flow if and only if X X u(e) ≥ b(v) for all X ⊆ V (G). e∈δ + (X )

(Gale [1957])

v∈X

Exercises

221

3. Let G be a digraph with lower and upper capacities l, u : E(G) → R+ , where l(e) ≤ u(e) for all e ∈ E(G), and let b1 , b2 : V (G) → R with b1 (v) ≤ b2 (v) for all v ∈ V (G). Prove that there exists a flow f with l(e) ≤ f (e) ≤ u(e) for all e ∈ E(G) and X X f (e) − f (e) ≤ b2 (v) for all v ∈ V (G) b1 (v) ≤ e∈δ + (v)

e∈δ − (v)

if and only if X

u(e) ≥ max

e∈δ + (X )

X

b1 (v), −

v∈X

X

b2 (v)

v∈V (G)\X

X

+

l(e)

e∈δ − (X )

for all X ⊆ V (G). (This is a generalization of Exercise 4 of Chapter 8 and Exercise 2 of this chapter.) (Hoffman [1960]) 4. Prove the following theorem of Ore [1956]. Given a digraph G and nonnegative integers a(x), b(x) for each x ∈ V (G), then G has a spanning subgraph H − with |δ + H (x)| = a(x) and |δ H (x)| = b(x) for all x ∈ V (G) if and only if X

a(x) =

x∈V (G)

X x∈X

∗

a(x) ≤

X

X

b(x)

and

x∈V (G)

+ min{b(y), |E G (X, {y})|}

for all X ⊆ V (G).

y∈V (G)

(Ford and Fulkerson [1962]) 5. Consider the M INIMUM C OST F LOW P ROBLEM where infinite capacities (u(e) = ∞ for some edges e) are allowed. (a) Show that an instance is unbounded if and only if it is feasible and there is a negative circuit all whose edges have infinite capacity. (b) Show how to decide in O(n 3 + m) time whether an instance is unbounded. (c) Show that for an instance that is not unbounded each infinite capacity can be equivalently replaced by a finite capacity. 6. Let (G, u, c, b) be an instance of the M INIMUM C OST F LOW P ROBLEM. We call a function π : V (G) → R an optimal potential if there exists a minimum cost b-flow f such that π is a feasible potential with respect to (G f , c). (a) Prove that a function π : V (G) → R is an optimal potential if and only if for all X ⊆ V (G): X X u(e) ≤ u(e). b(X ) + e∈δ − (X ):cπ (e) c(X ) or assert that no such Y exists. Suppose this algorithm has a running time which is polynomial in size(c). Prove that then there is an algorithm for finding a maximum weight set X ∈ F for a given (E, F) ∈ 9 and c : E → Z+ , whose running time is polynomial in size(c). (Gr¨otschel and Lov´asz [1995]; see also Schulz, Weismantel and Ziegler [1995], and Schulz and Weismantel [2002]) 15. Prove that O RLIN ’ S A LGORITHM always computes a spanning tree solution. 16. Prove that in 7 of O RLIN ’ S A LGORITHM one can replace the 8nγ -bound by 5nγ . 17. Consider the shortest path computations with nonnegative weights (using D IJKSTRA ’ S A LGORITHM) in the algorithms of Section 9.4 and 9.5. Show that even for graphs with parallel edges each of these computations can be performed in O(n 2 ) time, provided that we have the incidence list of G sorted by edge costs. Conclude that O RLIN ’ S A LGORITHM runs in O(mn 2 log m) time. ∗ 18. The P USH -R ELABEL A LGORITHM (Section 8.5) can be generalized to the M INIMUM C OST F LOW P ROBLEM. For an instance (G, u, b, c) with integral costs c, we look for a b-flow f and a feasible potential π in (G f , c). We start by setting π := 0 and saturating all edges e with negative cost. Then we apply 3 of the P USH -R ELABEL A LGORITHM with the following modifications. An edge e is admissible if e ∈ E(G f ) and cπ (e) < 0. A vertex v is active if b(v)+ex f (v) > 0. R ELABEL(v) consists of setting π(v) := max{π(w)−c(e)−1 : e = (v, w) ∈ E(G f )}. In P USH(e) for e ∈ δ + (v) we set γ := min{b(v) + ex f (v), u f (e)}. (a) Prove that the number of R ELABEL operations is O(n 2 |cmax |), where cmax = maxe∈E(G) c(e). Hint: Some vertex w with b(w) + ex f (w) < 0 must be reachable in G f from any active vertex v. Note that b(w) has never changed and recall the proofs of Lemmata 8.22 and 8.24. (b) Show that the overall running time is O(n 2 mcmax ). (c) Prove that the algorithm computes an optimum solution. (d) Apply scaling to obtain an O(n 2 m log cmax )-algorithm for the M INIMUM C OST F LOW P ROBLEM with integral costs c. (Goldberg and Tarjan [1990]) 19. Let (G, u, c, b) be an instance of the M INIMUM C OST F LOW P ROBLEM. Let e¯ ∈ E(G) with c(e) ¯ > (|V (G)| − 1) maxe∈E(G)\{e} ¯ |c(e)|. Prove: If there exists a b-flow f in (G, u) with f (e) ¯ = 0, then f (e) ¯ = 0 holds for every optimum solution f . 20. Given a network (G, u, s, t) with integral transit times l : E(G) → Z+ , a time horizon T ∈ N, a value V ∈ R+ , and costs c : E(G) → R+ . We look for an s-t-flow over time f with value ( f ) = V and minimum cost

224

9 Minimum Cost Flows

RT e∈E(G) c(e) 0

f e (τ )dτ . Show how to solve this in polynomial time if T is a constant. Hint: Consider a time-expanded network with a copy of G for each discrete time step.

P

References General Literature: Ahuja, R.K., Magnanti, T.L., and Orlin, J.B. [1993]: Network Flows. Prentice-Hall, Englewood Cliffs 1993 Cook, W.J., Cunningham, W.H., Pulleyblank, W.R., and Schrijver, A. [1998]: Combinatorial Optimization. Wiley, New York 1998, Chapter 4 ´ and Tarjan, R.E. [1990]: Network flow algorithms. In: Paths, Goldberg, A.V., Tardos, E., Flows, and VLSI-Layout (B. Korte, L. Lov´asz, H.J. Pr¨omel, A. Schrijver, eds.), Springer, Berlin 1990, pp. 101–164 Gondran, M., and Minoux, M. [1984]: Graphs and Algorithms. Wiley, Chichester 1984, Chapter 5 Jungnickel, D. [2007]: Graphs, Networks and Algorithms. Third Edition. Springer, Berlin 2007, Chapters 10 and 11 Lawler, E.L. [1976]: Combinatorial Optimization: Networks and Matroids. Holt, Rinehart and Winston, New York 1976, Chapter 4 Ruhe, G. [1991]: Algorithmic Aspects of Flows in Networks. Kluwer Academic Publishers, Dordrecht 1991

Cited References: Arkin, E.M., and Silverberg, E.B. [1987]: Scheduling jobs with fixed start and end times. Discrete Applied Mathematics 18 (1987), 1–8 Armstrong, R.D., and Jin, Z. [1997]: A new strongly polynomial dual network simplex algorithm. Mathematical Programming 78 (1997), 131–148 Busacker, R.G., and Gowen, P.J. [1961]: A procedure for determining a family of minimumcost network flow patterns. ORO Technical Paper 15, Operational Research Office, Johns Hopkins University, Baltimore 1961 Cunningham, W.H. [1976]: A network simplex method. Mathematical Programming 11 (1976), 105–116 Dantzig, G.B. [1951]: Application of the simplex method to a transportation problem. In: Activity Analysis and Production and Allocation (T.C. Koopmans, Ed.), Wiley, New York 1951, pp. 359–373 Edmonds, J., and Karp, R.M. [1972]: Theoretical improvements in algorithmic efficiency for network flow problems. Journal of the ACM 19 (1972), 248–264 Fleischer, L., and Skutella, M. [2007]: Quickest flows over time. SIAM Journal on Computing 36 (2007), 1600–1630 Ford, L.R., and Fulkerson, D.R. [1958]: Constructing maximal dynamic flows from static flows. Operations Research 6 (1958), 419–433 Ford, L.R., and Fulkerson, D.R. [1962]: Flows in Networks. Princeton University Press, Princeton 1962

References

225

Gale, D. [1957]: A theorem on flows in networks. Pacific Journal of Mathematics 7 (1957), 1073–1082 Goldberg, A.V., and Tarjan, R.E. [1989]: Finding minimum-cost circulations by cancelling negative cycles. Journal of the ACM 36 (1989), 873–886 Goldberg, A.V., and Tarjan, R.E. [1990]: Finding minimum-cost circulations by successive approximation. Mathematics of Operations Research 15 (1990), 430–466 Gr¨otschel, M., and Lov´asz, L. [1995]: Combinatorial optimization. In: Handbook of Combinatorics; Vol. 2 (R.L. Graham, M. Gr¨otschel, L. Lov´asz, eds.), Elsevier, Amsterdam 1995 Hassin, R. [1983]: The minimum cost flow problem: a unifying approach to dual algorithms and a new tree-search algorithm. Mathematical Programming 25 (1983), 228–239 Hitchcock, F.L. [1941]: The distribution of a product from several sources to numerous localities. Journal of Mathematical Physics 20 (1941), 224–230 Hoffman, A.J. [1960]: Some recent applications of the theory of linear inequalities to extremal combinatorial analysis. In: Combinatorial Analysis (R.E. Bellman, M. Hall, eds.), AMS, Providence 1960, pp. 113–128 ´ [2000]: The quickest transshipment problem. Mathematics of Hoppe, B., and Tardos, E. Operations Research 25 (2000), 36–62 Iri, M. [1960]: A new method for solving transportation-network problems. Journal of the Operations Research Society of Japan 3 (1960), 27–87 Jewell, W.S. [1958]: Optimal flow through networks. Interim Technical Report 8, MIT 1958 Klein, M. [1967]: A primal method for minimum cost flows, with applications to the assignment and transportation problems. Management Science 14 (1967), 205–220 Klinz, B., and Woeginger, G.J. [2004]: Minimum cost dynamic flows: the series-parallel case. Networks 43 (2004), 153–162 Orden, A. [1956]: The transshipment problem. Management Science 2 (1956), 276–285 Ore, O. [1956]: Studies on directed graphs I. Annals of Mathematics 63 (1956), 383–406 Orlin, J.B. [1993]: A faster strongly polynomial minimum cost flow algorithm. Operations Research 41 (1993), 338–350 Orlin, J.B. [1997]: A polynomial time primal network simplex algorithm for minimum cost flows. Mathematical Programming 78 (1997), 109–129 ´ [1993]: Polynomial dual network simplex algoOrlin, J.B., Plotkin, S.A., and Tardos, E. rithms. Mathematical Programming 60 (1993), 255–276 ´ [1990]: Improved dual network simplex. Proceedings of the Plotkin, S.A., and Tardos, E. 1st Annual ACM-SIAM Symposium on Discrete Algorithms (1990), 367–376 Schulz, A.S., Weismantel, R., and Ziegler, G.M. [1995]: 0/1-Integer Programming: optimization and augmentation are equivalent. In: Algorithms – ESA ’95; LNCS 979 (P. Spirakis, ed.), Springer, Berlin 1995, pp. 473–483 Schulz, A.S., and Weismantel, R. [2002]: The complexity of generic primal algorithms for solving general integer problems. Mathematics of Operations Research 27 (2002), 681–192 ´ [1985]: A strongly polynomial minimum cost circulation algorithm. CombinaTardos, E. torica 5 (1985), 247–255 Tolsto˘ı, A.N. [1930]: Metody nakhozhdeniya naimen’shego summovogo kilometrazha pri planirovanii perevozok v prostanstve. In: Planirovanie Perevozok, Sbornik pervy˘ı, Transpechat’ NKPS, Moskow 1930, pp. 23–55. (See A. Schrijver, On the history of the transportation and maximum flow problems, Mathematical Programming 91 (2002), 437–445)

226

9 Minimum Cost Flows

Tomizawa, N. [1971]: On some techniques useful for solution of transportation network problems. Networks 1 (1971), 173–194 Vygen, J. [2002]: On dual minimum cost flow algorithms. Mathematical Methods of Operations Research 56 (2002), 101–126 Wagner, H.M. [1959]: On a class of capacitated transportation problems. Management Science 5 (1959), 304–318

10 Maximum Matchings

Matching theory is one of the classical and most important topics in combinatorial theory and optimization. All the graphs in this chapter are undirected. Recall that a matching is a set of pairwise disjoint edges. Our main problem is:

C ARDINALITY M ATCHING P ROBLEM Instance:

An undirected graph G.

Task:

Find a maximum cardinality matching in G.

Since the weighted version of this problem is significantly more difficult we postpone it to Chapter 11. But already the above cardinality version has applications: Suppose in the J OB A SSIGNMENT P ROBLEM each job has the same processing time, say one hour, and we ask whether we can finish all the jobs within one hour. . In other words: given a bipartite graph GP with bipartition V (G) = A ∪ B, we look P for numbers x : E(G) → R+ with e∈δ(a) x(e) = 1 for each job a ∈ A and e∈δ(b) x(e) ≤ 1 for each employee b ∈ B. We can write this as a linear inequality system x ≥ 0, M x ≤ 1l, M 0 x ≥ 1l, where the rows of M and M 0 are rows of the node-edge incidence matrix of G. These matrices are totally unimodular by Theorem 5.25. From Theorem 5.20 we conclude that if there is any solution x, then there is also an integral solution. Now observe that the integral solutions to the above linear inequality system are precisely the incidence vectors of the matchings in G covering A. Definition 10.1. Let G be a graph and M a matching in G. We say that a vertex v is covered by M if v ∈ e for some e ∈ M. M is called a perfect matching if all vertices are covered by M. In Section 10.1 we consider matchings in bipartite graphs. Algorithmically this problem can be reduced to the M AXIMUM F LOW P ROBLEM. The Max-Flow-MinCut Theorem as well as the concept of augmenting paths have nice interpretations in our context. Matching in general, non-bipartite graphs does not reduce directly to network flows. We introduce two necessary and sufficient conditions for a general graph to have a perfect matching in Sections 10.2 and 10.3. In Section 10.4 we consider factor-critical graphs which have a matching covering all vertices but v, for each v ∈ V (G). These play an important role in Edmonds’ algorithm for the C ARDINALITY

228

10 Maximum Matchings

M ATCHING P ROBLEM, described in Section 10.5, and its weighted version which we postpone to Sections 11.2 and 11.3.

10.1 Bipartite Matching Since the C ARDINALITY M ATCHING P ROBLEM is easier if G is bipartite, we shall deal with this case first. In this section, a bipartite graph G is always assumed to . have the bipartition V (G) = A ∪ B. Since we may assume that G is connected, we can regard this bipartition as unique (Exercise 20 of Chapter 2). For a graph G, let ν(G) denote the maximum cardinality of a matching in G, while τ (G) is the minimum cardinality of a vertex cover in G. Theorem 10.2. (K¨onig [1931]) If G is bipartite, then ν(G) = τ (G). .

Proof: Consider the graph G 0 = (V (G) ∪ {s, t}, E(G) ∪ {{s, a} : a ∈ A} ∪ {{b, t} : b ∈ B}). Then ν(G) is the maximum number of vertex-disjoint s-t-paths, while τ (G) is the minimum number of vertices whose deletion makes t unreachable from s. The theorem now immediately follows from Menger’s Theorem 8.10. ν(G) ≤ τ (G) evidently holds for any graph (bipartite or not), but we do not have equality in general (as the triangle K 3 shows). Several statements are equivalent to K¨onig’s Theorem. Hall’s Theorem is probably the best-known version. Theorem 10.3. (Hall [1935]) Let G be a bipartite graph with bipartition V (G) = . A ∪ B. Then G has a matching covering A if and only if |0(X )| ≥ |X |

for all X ⊆ A.

(10.1)

Proof: The necessity of the condition is obvious. To prove the sufficiency, assume that G has no matching covering A, i.e. ν(G) < |A|. By Theorem 10.2 this implies τ (G) < |A|. Let A0 ⊆ A, B 0 ⊆ B such that A0 ∪ B 0 covers all the edges and |A0 ∪ B 0 | < |A|. Obviously 0(A \ A0 ) ⊆ B 0 . Therefore |0(A \ A0 )| ≤ |B 0 | < |A| − |A0 | = |A \ A0 |, and the Hall condition (10.1) is violated. It is worthwhile to mention that it is not too difficult to prove Hall’s Theorem directly. The following proof is due to Halmos and Vaughan [1950]: Second Proof of Theorem 10.3: We show that any G satisfying the Hall condition (10.1) has a matching covering A. We use induction on |A|, the cases |A| = 0 and |A| = 1 being trivial. If |A| ≥ 2, we consider two cases: If |0(X )| > |X | for every nonempty proper subset X of A, then we take any edge {a, b} (a ∈ A, b ∈ B), delete its two vertices and apply induction. The smaller graph satisfies the Hall condition because |0(X )| − |X | can have decreased by at most one for any X ⊆ A \ {a}.

10.1 Bipartite Matching

229

Now assume that there is a nonempty proper subset X of A with |0(X )| = |X |. By induction there is a matching covering X in G[X ∪ 0(X )]. We claim that we can extend this to a matching in G covering A. Again by the induction hypothesis, we have to show that G[(A \ X ) ∪ (B \ 0(X ))] satisfies the Hall condition. To check this, observe that for any Y ⊆ A \ X we have (in the original graph G): |0(Y ) \ 0(X )| = |0(X ∪ Y )| − |0(X )| ≥ |X ∪ Y | − |X | = |Y |. A special case of Hall’s Theorem is the so-called “Marriage Theorem”: Theorem 10.4. (Frobenius [1917]) Let G be a bipartite graph with bipartition . V (G) = A ∪ B. Then G has a perfect matching if and only if |A| = |B| and |0(X )| ≥ |X | for all X ⊆ A. The variety of applications of Hall’s Theorem is indicated by Exercises 4–7. The proof of K¨onig’s Theorem 10.2 shows how to solve the bipartite matching problem algorithmically: Theorem 10.5. The C ARDINALITY M ATCHING P ROBLEM for bipartite graphs G can be solved in O(nm) time, where n = |V (G)| and m = |E(G)|. .

Proof: Let G be a bipartite graph with bipartition V (G) = A ∪ B. Add a vertex s and connect it to all vertices of A, and add another vertex t connected to all vertices of B. Orient the edges from s to A, from A to B, and from B to t. Let the capacities be 1 everywhere. Then a maximum integral s-t-flow corresponds to a maximum cardinality matching (and vice versa). So we apply the F ORD -F ULKERSON A LGORITHM and find a maximum s-tflow (and thus a maximum matching) after at most n augmentations. Since each augmentation takes O(m) time, we are done. This result is essentially due to Kuhn [1955]. In fact, one can use the concept of shortest augmenting paths√again (cf. the E DMONDS -K ARP A LGORITHM). In this way one obtains the O n(m + n) -algorithm of Hopcroft and Karp [1973]. This algorithm will be discussed in Exercises 9 and 10. Slightimprovements of q mn the H OPCROFT -K ARP A LGORITHM yield running times of O n log n (Alt et √ 2 /m) al. [1991]) and O m n log(n (Feder and Motwani [1995]). The latter bound log n is the best known for dense graphs. Let us reformulate the augmenting path concept in our context. Definition 10.6. Let G be a graph (bipartite or not), and let M be some matching in G. A path P is an M-alternating path if E(P) \ M is a matching. An Malternating path is M-augmenting if its endpoints are not covered by M. One immediately checks that M-augmenting paths must have odd length.

230

10 Maximum Matchings

Theorem 10.7. (Berge [1957]) Let G be a graph (bipartite or not) with some matching M. Then M is maximum if and only if there is no M-augmenting path. Proof: If there is an M-augmenting path P, the symmetric difference M4E(P) is a matching and has greater cardinality than M, so M is not maximum. On the other hand, if there is a matching M 0 such that |M 0 | > |M|, the symmetric difference M4M 0 is the vertex-disjoint union of alternating circuits and paths, where at least one path must be M-augmenting. In the bipartite case Berge’s Theorem of course also follows from Theorem 8.5.

10.2 The Tutte Matrix We now consider maximum matchings from an algebraic point of view. Let G be a simple undirected graph, and let G 0 be the directed graph resulting from G by arbitrarily orienting the edges. For any vector x = (xe )e∈E(G) of variables, we define the Tutte matrix x )v,w∈V (G) TG (x) = (tvw

by x tvw

x{v,w} := −x{v,w} 0

if (v, w) ∈ E(G 0 ) if (w, v) ∈ E(G 0 ) . otherwise

(Such a matrix M, where M = −M > , is called skew-symmetric.) det TG (x) is a polynomial in the variables xe (e ∈ E(G)). Theorem 10.8. (Tutte [1947]) G has a perfect matching if and only if det TG (x) is not identically zero. Proof: Let V (G) = {v1 , . . . , vn }, and let Sn be the set of all permutations on {1, . . . , n}. By definition of the determinant, det TG (x) =

X π ∈Sn

sgn(π)

n Y

tvxi ,vπ(i) .

i=1

n o Qn x tvi ,vπ(i) 6= 0 . Each permutation π ∈ Sn corresponds Let Sn0 := π ∈ Sn : i=1 to a directed graph Hπ := (V (G), {(vi , vπ(i) ) : i = 1, . . . , n}) where each vertex ↔

+ 0 0 x has |δ − Hπ (x)| = |δ Hπ (x)| = 1. For permutations π ∈ Sn , Hπ is a subgraph of G . 0 If there exists a permutation π ∈ Sn such that Hπ consists of even circuits only, then by taking every second edge of each circuit (and ignoring the orientations) we obtain a perfect matching in G.

10.2 The Tutte Matrix

231

Otherwise, for each π ∈ Sn0 there is a permutation r (π) ∈ Sn0 such that Hr (π ) is obtained by reversing the first odd circuit in Hπ , i.e. the odd circuit containing the vertex with minimum index. Of course r (r (π)) = π . Observe that sgn(π ) = sgn(r (π)), i.e. the two permutations have the same sign: if the first odd circuit consists of the vertices w1 , . . . , w2k+1 with π(wi ) = wi+1 (i = 1, . . . , 2k) and π(w2k+1 ) = w1 , then we obtain r (π ) by 2k transpositions: for with π(w2k ) and then π(w2 j ) with π(w2k+1 ). j = 1, . . . , k exchange Qn x π(w2 j−1 )Q n tvi ,vπ(i) = − i=1 tvxi ,vr (π )(i) . So the two corresponding terms Moreover, i=1 in the sum n Y X sgn(π) tvxi ,vπ(i) det TG (x) = π ∈Sn0

i=1

cancel each other. Since this holds for all pairs π, r (π ) ∈ Sn0 , we conclude that det TG (x) is identically zero. So if G has no perfect matching, det TG (x) is identically zero. On the other hand, if G has a perfect matching M, consider the permutation definedQby π(i) := j n tvxi ,vπ(i) = and π( j) := i for all {vi , v j } ∈ M. The corresponding term i=1 Q 2 e∈M −x e cannot cancel out with any other term, so det TG (x) is not identically zero. Originally, Tutte used Theorem 10.8 to prove his main theorem on matchings, Theorem 10.13. Theorem 10.8 does not provide a good characterization of the property that a graph has a perfect matching. The problem is that the determinant is easy to compute if the entries are numbers (Theorem 4.10) but difficult to compute if the entries are variables. However, the theorem suggests a randomized algorithm for the C ARDINALITY M ATCHING P ROBLEM: Corollary 10.9. (Lov´asz [1979]) Let x = (xe )e∈E(G) be a random vector where each coordinate is equally distributed in [0, 1]. Then with probability 1 the rank of TG (x) is exactly twice the size of a maximum matching. Proof: Suppose the rank of TG (x) is k, say the first k rows are linearly independent. Since TG (x) is skew-symmetric, also the first k columns are linearly independent. So the principal submatrix (tvxi ,v j )1≤i, j≤k is nonsingular, and by Theorem 10.8 the subgraph G[{v1 , . . . , vk }] has a perfect matching. In particular, k is even and G has a matching of cardinality k2 . On the other hand, if G has a matching of cardinality k, the determinant of the principal submatrix T 0 whose rows and columns correspond to the 2k vertices covered by M is not identically zero by Theorem 10.8. The set of vectors x for which det T 0 (x) = 0 must then have measure zero. So with probability one, the rank of TG (x) is at least 2k. Of course it is not possible to choose random numbers from [0, 1] with a digital computer. However, it can be shown that it suffices to choose random integers from the finite set {1, 2, . . . , N }. For sufficiently large N , the probability of error will become arbitrarily small (see Lov´asz [1979]). Lov´asz’ algorithm

232

10 Maximum Matchings

can be used to determine a maximum matching (not only its cardinality). See Rabin and Vazirani [1989], Mulmuley, Vazirani and Vazirani [1987], and Mucha and Sankowski [2004] for further randomized algorithms for ﬁnding a maximum matching in a graph. Moreover we note that Geelen [2000] has shown how to derandomize Lov´asz’ algorithm. Although its running time is worse than that of Edmonds’ matching algorithm (see Section 10.5), it is important for some generalizations of the C ARDINALITY M ATCHING P ROBLEM (e.g., see Geelen and Iwata [2005]).

10.3 Tutte’s Theorem We now consider the C ARDINALITY M ATCHING P ROBLEM in general graphs. A necessary condition for a graph to have a perfect matching is that every connected component is even (i.e. has an even number of vertices). This condition is not sufﬁcient, as the graph K 1,3 (Figure 10.1(a)) shows. (a)

(b)

Fig. 10.1.

The reason that K 1,3 has no perfect matching is that there is one vertex (the black one) whose deletion produces three odd connected components. The graph shown in Figure 10.1(b) is more complicated. Does this graph have a perfect matching? If we delete the three black vertices, we get ﬁve odd connected components (and one even connected component). If there were a perfect matching, at least one vertex of each odd connected component would have to be connected to one of the black vertices. This is impossible because the number of odd connected components exceeds the number of black vertices. More generally, for X ⊆ V (G) let qG (X ) denote the number of odd connected components in G − X . Then a graph for which qG (X ) > |X | holds for some X ⊆ V (G) cannot have a perfect matching: otherwise there must be, for each

10.3 Tutte’s Theorem

233

odd connected component in G − X , at least one matching edge connecting this connected component with X , which is impossible if there are more odd connected components than elements of X . Tutte’s Theorem says that the above necessary condition is also sufficient: Definition 10.10. A graph G satisfies the Tutte condition if qG (X ) ≤ |X | for all X ⊆ V (G). A nonempty vertex set X ⊆ V (G) is a barrier if qG (X ) = |X |. To prove the sufficiency of the Tutte condition we shall need an easy observation and an important definition: Proposition 10.11. For any graph G and any X ⊆ V (G) we have qG (X ) − |X | ≡ |V (G)| (mod 2). Definition 10.12. A graph G is called factor-critical if G − v has a perfect matching for each v ∈ V (G). A matching is called near-perfect if it covers all vertices but one. Now we can prove Tutte’s Theorem: Theorem 10.13. (Tutte [1947]) A graph G has a perfect matching if and only if it satisfies the Tutte condition: qG (X ) ≤ |X |

for all X ⊆ V (G).

Proof: We have already seen the necessity of the Tutte condition. We now prove the sufficiency by induction on |V (G)| (the case |V (G)| ≤ 2 being trivial). Let G be a graph satisfying the Tutte condition. |V (G)| cannot be odd since otherwise the Tutte condition is violated because qG (∅) ≥ 1. So by Proposition 10.11, |X | − qG (X ) must be even for every X ⊆ V (G). Since |V (G)| is even and the Tutte condition holds, every singleton is a barrier. We choose a maximal barrier X . G − X has |X | odd connected components. G − X cannot have any even connected components because otherwise X ∪ {v}, where v is a vertex of some even connected component, is a barrier (G − (X ∪ {v}) has |X | + 1 odd connected components), contradicting the maximality of X . We now claim that each odd connected component of G − X is factor-critical. To prove this, let C be some odd connected component of G − X and v ∈ V (C). If C − v has no perfect matching, by the induction hypothesis there is some Y ⊆ V (C) \ {v} such that qC−v (Y ) > |Y |. By Proposition 10.11, qC−v (Y ) − |Y | must be even, so qC−v (Y ) ≥ |Y | + 2. Since X, Y and {v} are pairwise disjoint, we have

234

10 Maximum Matchings

qG (X ∪ Y ∪ {v}) = qG (X ) − 1 + qC (Y ∪ {v}) = |X | − 1 + qC−v (Y ) ≥ |X | − 1 + |Y | + 2 = |X ∪ Y ∪ {v}|. So X ∪ Y ∪ {v} is a barrier, contradicting the maximality of X . . We now consider the bipartite graph G 0 with bipartition V (G 0 ) = X ∪ Z which arises when we delete edges with both ends in X and contract the odd connected components of G − X to single vertices (forming the set Z ). It remains to show that G 0 has a perfect matching. If not, then by Frobenius’ Theorem 10.4 there is some A ⊆ Z such that |0G 0 (A)| < |A|. This implies qG (0G 0 (A)) ≥ |A| > |0G 0 (A)|, a contradiction. This proof is due to Anderson [1971]. The Tutte condition provides a good characterization of the perfect matching problem: either a graph has a perfect matching or it has a so-called Tutte set X proving that it has no perfect matching. An important consequence of Tutte’s Theorem is the so-called Berge-Tutte formula: Theorem 10.14. (Berge [1958]) 2ν(G) + max (qG (X ) − |X |) = |V (G)|. X ⊆V (G)

Proof: For any X ⊆ V (G), any matching must leave at least qG (X )−|X | vertices uncovered. Therefore 2ν(G) + qG (X ) − |X | ≤ |V (G)|. To prove the reverse inequality, let k :=

max (qG (X ) − |X |).

X ⊆V (G)

We construct a new graph H by adding k new vertices to G, each of which is connected to all the old vertices. If we can prove that H has a perfect matching, then 2ν(G) + k ≥ 2ν(H ) − k = |V (H )| − k = |V (G)|, and the theorem is proved. Suppose H has no perfect matching, then by Tutte’s Theorem there is a set Y ⊆ V (H ) such that q H (Y ) > |Y |. By Proposition 10.11, k has the same parity as |V (G)|, implying that |V (H )| is even. Therefore Y 6= ∅ and thus q H (Y ) > 1. But then Y contains all the new vertices, so qG (Y ∩ V (G)) = q H (Y ) > |Y | = |Y ∩ V (G)| + k, contradicting the definition of k. Let us close this section with a proposition for later use.

10.4 Ear-Decompositions of Factor-Critical Graphs

235

Proposition 10.15. Let G be a graph and X ⊆ V (G) with |V (G)| − 2ν(G) = qG (X ) − |X |. Then any maximum matching of G contains a perfect matching in each even connected component of G − X , a near-perfect matching in each odd connected component of G − X , and matches all the vertices in X to vertices of distinct odd connected components of G − X . Later we shall see (Theorem 10.32) that X can be chosen such that each odd connected component of G − X is factor-critical.

10.4 Ear-Decompositions of Factor-Critical Graphs This section contains some results on factor-critical graphs which we shall need later. In Exercise 17 of Chapter 2 we have seen that the graphs having an eardecomposition are exactly the 2-edge-connected graphs. Here we are interested in odd ear-decompositions only. Definition 10.16. An ear-decomposition is called odd if every ear has odd length. Theorem 10.17. (Lov´asz [1972]) A graph is factor-critical if and only if it has an odd ear-decomposition. Furthermore, the initial vertex of the ear-decomposition can be chosen arbitrarily. Proof: Let G be a graph with a fixed odd ear-decomposition. We prove that G is factor-critical by induction on the number of ears. Let P be the last ear in the odd ear-decomposition, say P goes from x to y, and let G 0 be the graph before adding P. We have to show for any vertex v ∈ V (G) that G − v contains a perfect matching. If v is not an inner vertex of P this is clear by induction (add every second edge of P to the perfect matching in G 0 − v). If v is an inner vertex of P, then exactly one of P[v,x] and P[v,y] must be even, say P[v,x] . By induction there is a perfect matching in G 0 − x. By adding every second edge of P[y,v] and of P[v,x] we obtain a perfect matching in G − v. We now prove the reverse direction. Choose the initial vertex z of the eardecomposition arbitrarily, and let M be a near-perfect matching in G covering V (G) \ {z}. Suppose we already have an odd ear-decomposition of a subgraph G 0 of G such that z ∈ V (G 0 ) and M ∩ E(G 0 ) is a near-perfect matching in G 0 . If G = G 0 , we are done. If not, then – since G is connected – there must be an edge e = {x, y} ∈ E(G) \ E(G 0 ) with x ∈ V (G 0 ). If y ∈ V (G 0 ), e is the next ear. Otherwise let N be a near-perfect matching in G covering V (G) \ {y}. M4N obviously contains the edges of a y-z-path P. Let w be the first vertex of P (when traversed from y) that belongs to V (G 0 ). The last edge of P 0 := P[y,w] cannot belong to M (because no edge of M leaves V (G 0 )), and the first edge cannot belong to N . Since P 0 is M-N -alternating, |E(P 0 )| must be even, so together with e it forms the next ear. In fact, we have constructed a special type of odd ear-decomposition:

236

10 Maximum Matchings

Definition 10.18. Given a factor-critical graph G and a near-perfect matching M, an M-alternating ear-decomposition of G is an odd ear-decomposition such that each ear is an M-alternating path or a circuit C with |E(C)∩ M|+1 = |E(C)\ M|. It is clear that the initial vertex of an M-alternating ear-decomposition must be the vertex not covered by M. The proof of Theorem 10.17 immediately yields: Corollary 10.19. For any factor-critical graph G and any near-perfect matching M in G there exists an M-alternating ear-decomposition. From now on, we shall only be interested in M-alternating ear-decompositions. An interesting way to store an M-alternating ear-decomposition efficiently is due to Lov´asz and Plummer [1986]: Definition 10.20. Let G be a factor-critical graph and M a near-perfect matching in G. Let r, P1 , . . . , Pk be an M-alternating ear-decomposition of G and µ, ϕ : V (G) → V (G) two functions. We say that µ and ϕ are associated with the ear-decomposition r, P1 , . . . , Pk if • µ(x) = y if {x, y} ∈ M, • ϕ(x) = y if {x, y} ∈ E(Pi ) \ M and x ∈ / {r } ∪ V (P1 ) ∪ · · · ∪ V (Pi−1 ), • µ(r ) = ϕ(r ) = r . If M is fixed, we also say that ϕ is associated with r, P1 , . . . , Pk . If M is some fixed near-perfect matching and µ, ϕ are associated with two M-alternating ear-decompositions, they are the same up to the order of the ears. Moreover, an explicit list of the ears can be obtained in linear time:

E AR -D ECOMPOSITION A LGORITHM Input: Output:

1

2

3

A factor-critical graph G, functions µ, ϕ associated with an Malternating ear-decomposition. An M-alternating ear-decomposition r, P1 , . . . , Pk .

Let initially be X := {r }, where r is the vertex with µ(r ) = r . Let k := 0, and let the stack be empty. If X = V (G) then go to . 5 If the stack is nonempty then let v ∈ V (G) \ X be an endpoint of the topmost element of the stack, else choose v ∈ V (G) \ X arbitrarily. Set x := v, y := µ(v) and P := ({x, y}, {{x, y}}). While ϕ(ϕ(x)) = x do: Set P := P + {x, ϕ(x)} + {ϕ(x), µ(ϕ(x))} and x := µ(ϕ(x)). While ϕ(ϕ(y)) = y do: Set P := P + {y, ϕ(y)} + {ϕ(y), µ(ϕ(y))} and y := µ(ϕ(y)). Set P := P + {x, ϕ(x)} + {y, ϕ(y)}. P is the ear containing y as an inner vertex. Put P on top of the stack.

10.4 Ear-Decompositions of Factor-Critical Graphs

4

5

237

While both endpoints of the topmost element P of the stack are in X do: Delete P from the stack, set k := k +1, Pk := P and X := X ∪ V (P). Go to . 2 For all {y, z} ∈ E(G) \ (E(P1 ) ∪ · · · ∪ E(Pk )) do: Set k := k + 1 and Pk := ({y, z}, {{y, z}}).

Proposition 10.21. Let G be a factor-critical graph and µ, ϕ functions associated with an M-alternating ear-decomposition. Then this ear-decomposition is unique up to the order of the ears. The E AR -D ECOMPOSITION A LGORITHM correctly determines an explicit list of these ears; it runs in linear time. Proof: Let D be an M-alternating ear-decomposition associated with µ and ϕ. The uniqueness of D as well as the correctness of the algorithm follows from the obvious fact that P as computed in 3 is indeed an ear of D. The running time of 1 – 4 is evidently O(|V (G)|), while 5 takes O(|E(G)|) time. The most important property of the functions associated with an alternating ear-decomposition is the following: Lemma 10.22. Let G be a factor-critical graph and µ, ϕ two functions associated with an M-alternating ear-decomposition. Let r be the vertex not covered by M. Then the maximal path given by an initial subsequence of x, µ(x), ϕ(µ(x)), µ(ϕ(µ(x))), ϕ(µ(ϕ(µ(x)))), . . . defines an M-alternating x-r -path of even length for all x ∈ V (G). Proof: Let x ∈ V (G) \ {r }, and let Pi be the first ear containing x. Clearly some initial subsequence of x, µ(x), ϕ(µ(x)), µ(ϕ(µ(x))), ϕ(µ(ϕ(µ(x)))), . . . must be a subpath Q of Pi from x to y, where y ∈ {r } ∪ V (P1 ) ∪ · · · ∪ V (Pi−1 ). Because we have an M-alternating ear-decomposition, the last edge of Q does not belong to M; hence Q has even length. If y = r , we are done, otherwise we apply induction on i. The converse of Lemma 10.22 is not true: In the counterexample in Figure 10.2 (bold edges are matching edges, edges directed from u to v indicate ϕ(u) = v), µ and ϕ also define alternating paths to the vertex not covered by the matching. However, µ and ϕ are not associated with any alternating ear-decomposition. For the W EIGHTED M ATCHING A LGORITHM (Section 11.3) we shall need a fast routine for updating an alternating ear-decomposition when the matching changes. Although the proof of Theorem 10.17 is algorithmic (provided that we can find a maximum matching in a graph), this is far too inefficient. We make use of the old ear-decomposition:

238

10 Maximum Matchings

Fig. 10.2.

Lemma 10.23. Given a factor-critical graph G, two near-perfect matchings M and M , and functions µ, ϕ associated with an M-alternating ear-decomposition. Then functions µ , ϕ associated with an M -alternating ear-decomposition can be found in O(|V (G)|) time. Proof: Let v be the vertex not covered by M, and let v be the vertex not covered by M . Let P be the v -v-path in MM , say P = x0 , x1 , . . . , xk with x0 = v and xk = v.

x p+1 xp

e

v X M v

M

P

Fig. 10.3.

An explicit list of the ears of the old ear-decomposition can be obtained from µ and ϕ by the E AR -D ECOMPOSITION A LGORITHM in linear time (Proposition 10.21). Indeed, since we do not have to consider ears of length one, we can omit : 5 then the total number of edges considered is at most 32 (|V (G)|−1) (cf. Exercise 19).

10.4 Ear-Decompositions of Factor-Critical Graphs

239

Suppose we have already constructed an M -alternating ear-decomposition of a spanning subgraph of G[X ] for some X ⊆ V (G) with v ∈ X (initially X := {v }). Of course no M -edge leaves X . Let p := max{i ∈ {0, . . . , k} : xi ∈ X } (illustrated in Figure 10.3). At each stage we keep track of p and of the edge set δ(X ) ∩ M. Their update when extending X is clearly possible in linear total time. Now we show how to extend the ear-decomposition. We shall add one or more ears in each step. The time needed for each step will be proportional to the total number of edges in new ears. Case 1: |δ(X ) ∩ M| ≥ 2. Let f ∈ δ(X ) ∩ M with x p ∈ / f . Evidently, f belongs to an M-M -alternating path which can be added as the next ear. The time needed to ﬁnd this ear is proportional to its length. Case 2: |δ(X ) ∩ M| = 1. Then v ∈ / X , and e = {x p , x p+1 } is the only edge in δ(X ) ∩ M. Let R be the x p+1 -v-path determined by µ and ϕ (cf. Lemma 10.22). The ﬁrst edge of R is e. Let q be the minimum index i ∈ {p + 2, p + 4, . . . , k} with xi ∈ V (R ) and V (R[x ) ∩ {xi+1 , . . . , xk } = ∅ (cf. Figure 10.4). Let p+1,x ] i

R := R[x . So R has vertices x p , ϕ(x p ), µ(ϕ(x p )), ϕ(µ(ϕ(x p ))), . . . , xq , and p ,x q ] can be traversed in time proportional to its length.

X x0 = v

xq xp

xk = v

x p+1

Fig. 10.4.

Let S := E(R) \ E(G[X ]), D := (MM ) \ (E(G[X ]) ∪ E(P[xq ,v] )), and Z := SD. S and D consist of M-alternating paths and circuits. Observe that every vertex outside X has degree 0 or 2 with respect to Z . Moreover, for every vertex outside X with two incident edges of Z , one of them belongs to M . (Here the choice of q is essential.) Hence all connected components C of (V (G), Z ) with E(C) ∩ δ(X ) = ∅ can be added as next ears, and after these ears have been added, S \ Z = S ∩ (MM ) is the vertex-disjoint union of paths each of which can then be added as an ear. Since e ∈ D \ S ⊆ Z , we have Z ∩ δ(X ) = ∅, so at least one ear is added.

240

10 Maximum Matchings

It remains to show that the time needed for the above construction is proportional to the total number of edges in new ears. Obviously, it sufﬁces to ﬁnd S in O(|E(S)|) time. This is difﬁcult because of the subpaths of R inside X . However, we do not really care what they look like. So we would like to shortcut these paths whenever possible. To achieve this, we modify the ϕ-variables. Namely, in each application of Case 2, let R[a,b] be a maximal subpath of R inside X with a = b. Let y := µ(b); y is the predecessor of b on R. We set ϕ(x) := y for all vertices x on R[a,y] where R[x,y] has odd length. It does not matter whether x and y are joined by an edge. See Figure 10.5 for an illustration.

y

X

R

xp

x p+1

x0 = v

Fig. 10.5.

The time required for updating the ϕ-variables is proportional to the number of edges examined. Note that these changes of ϕ do not destroy the property of Lemma 10.22, and the ϕ-variables are not used anymore except for ﬁnding M-alternating paths to v in Case 2. Now it is guaranteed that the time required for ﬁnding the subpaths of R inside X is proportional to the number of subpaths plus the number of edges examined for the ﬁrst time inside X . Since the number of subpaths inside X is less than or equal to the number of new ears in this step, we obtain an overall linear running time. Case 3: δ(X ) ∩ M = ∅. Then v ∈ X . We consider the ears of the (old) Malternating ear-decomposition in their order. Let R be the ﬁrst ear with V (R)\X = ∅. Similar to Case 2, let S := E(R) \ E(G[X ]), D := (MM ) \ E(G[X ]), and Z := SD. Again, all connected components C of (V (G), Z ) with E(C)∩δ(X ) =

10.5 Edmonds’ Matching Algorithm

241

∅ can be added as next ears, and after these ears have been added, S \ Z is the vertex-disjoint union of paths each of which can then be added as an ear. The total time needed for Case 3 is obviously linear.

10.5 Edmonds’ Matching Algorithm Recall Berge’s Theorem 10.7: A matching in a graph is maximum if and only if there is no augmenting path. Since this holds for non-bipartite graphs as well, our matching algorithm will again be based on augmenting paths. However, it is not at all clear how to ﬁnd an augmenting path (or decide that there is none). In the bipartite case (Theorem 10.5) it was sufﬁcient to mark the vertices that are reachable from a vertex not covered by the matching via an alternating edge progression. Since there were no odd circuits, vertices reachable by an alternating edge progression were also reachable by an alternating path. This is no longer the case when dealing with general graphs. v8

v1

v3

v4

v5

v2

v7

v6

Fig. 10.6.

Consider the example in Figure 10.6 (the bold edges constitute a matching M). When starting at v 1 , we have an alternating edge progression v 1 , v 2 , v 3 , v 4 , v 5 , v 6 , v 7 , v 5 , v 4 , v 8 , but this is not a path. We have run through an odd circuit, namely v 5 , v 6 , v 7 . Note that in our example there exists an augmenting path (v 1 , v 2 , v 3 , v 7 , v 6 , v 5 , v 4 , v 8 ) but it is not clear how to ﬁnd it. The question arises what to do if we encounter an odd circuit. Surprisingly, it sufﬁces to get rid of it by shrinking it to a single vertex. It turns out that the smaller graph has a perfect matching if and only if the original graph has one. This is the general idea of E DMONDS ’ C ARDINALITY M ATCHING A LGORITHM. We formulate this idea in Lemma 10.25 after giving the following deﬁnition: Deﬁnition 10.24. Let G be a graph and M a matching in G. A blossom in G with respect to M is a factor-critical subgraph C of G with |M ∩ E(C)| = |V (C)|−1 . 2 The vertex of C not covered by M ∩ E(C) is called the base of C. The blossom we have encountered in the above example (Figure 10.6) is induced by {v 5 , v 6 , v 7 }. Note that this example contains other blossoms. Any single vertex

242

10 Maximum Matchings

is also a blossom in terms of our definition. Now we can formulate the Blossom Shrinking Lemma: Lemma 10.25. Let G be a graph, M a matching in G, and C a blossom in G (with respect to M). Suppose there is an M-alternating v-r -path Q of even length from a vertex v not covered by M to the base r of C, where E(Q) ∩ E(C) = ∅. Let G 0 and M 0 result from G and M by shrinking V (C) to a single vertex. Then M is a maximum matching in G if and only if M 0 is a maximum matching in G 0 . Proof: Suppose that M is not a maximum matching in G. N := M4E(Q) is a matching of the same cardinality, so it is not maximum either. By Berge’s Theorem 10.7 there then exists an N -augmenting path P in G. Note that N does not cover r. At least one of the endpoints of P, say x, does not belong to C. If P and C are disjoint, let y be the other endpoint of P. Otherwise let y be the first vertex on P – when traversed from x – belonging to C. Let P 0 result from P[x,y] when shrinking V (C) in G. The endpoints of P 0 are not covered by N 0 (the matching in G 0 corresponding to N ). Hence P 0 is an N 0 -augmenting path in G 0 . So N 0 is not a maximum matching in G 0 , and nor is M 0 (which has the same cardinality). To prove the converse, suppose that M 0 is not a maximum matching in G 0 . Let N 0 be a larger matching in G 0 . N 0 corresponds to a matching N0 in G which covers at most one vertex of C in G. Since C is factor-critical, N0 can be extended edges to a matching N in G, where by k := |V (C)|−1 2 |N | = |N0 | + k = |N 0 | + k > |M 0 | + k = |M|, proving that M is not a maximum matching in G.

It is necessary to require that the base of the blossom is reachable from a vertex not covered by M by an M-alternating path of even length which is disjoint from the blossom. For example, the blossom induced by {v4 , v6 , v7 , v2 , v3 } in Figure 10.6 cannot be shrunk without destroying the only augmenting path. When looking for an augmenting path, we shall build up an alternating forest: Definition 10.26. Given a graph G and a matching M in G. An alternating forest with respect to M in G is a forest F in G with the following properties: (a) V (F) contains all the vertices not covered by M. Each connected component of F contains exactly one vertex not covered by M, its root. (b) We call a vertex v ∈ V (F) an outer (inner) vertex if it has even (odd) distance to the root of the connected component containing v. (In particular, the roots are outer vertices.) All inner vertices have degree 2 in F. (c) For any v ∈ V (F), the unique path from v to the root of the connected component containing v is M-alternating. Figure 10.7 shows an alternating forest. The bold edges belong to the matching. The black vertices are inner, the white vertices outer.

10.5 Edmonds’ Matching Algorithm

243

Fig. 10.7.

Proposition 10.27. In any alternating forest the number of outer vertices that are not a root equals the number of inner vertices. Proof: Each outer vertex that is not a root has exactly one neighbour which is an inner vertex and whose distance to the root is smaller. This is obviously a bijection between the outer vertices that are not a root and the inner vertices. Informally, E DMONDS ’ C ARDINALITY M ATCHING A LGORITHM works as follows. Given some matching M, we build up an M-alternating forest F. We start with the set S of vertices not covered by M, and no edges. At any stage of the algorithm we consider a neighbour y of an outer vertex x. Let P(x) denote the unique path in F from x to a root. There are three interesting cases, corresponding to three operations (“grow”, “augment”, and “shrink”): Case 1: y ∈ / V (F). Then the forest will grow when we add {x, y} and the matching edge covering y. Case 2: y is an outer vertex in a different connected component of F. Then we augment M along P(x) ∪ {x, y} ∪ P(y). Case 3: y is an outer vertex in the same connected component of F (with root q). Let r be the ﬁrst vertex of P(x) (starting at x) also belonging to P(y). (r can be one of x, y.) If r is not a root, it must have degree at least 3. So r is an outer vertex. Therefore C := P(x)[x,r ] ∪ {x, y} ∪ P(y)[y,r ] is a blossom with at least three vertices. We shrink C.

244

10 Maximum Matchings

If none of the cases applies, all the neighbours of outer vertices are inner. We claim that M is maximum. Let X be the set of inner vertices, s := |X |, and let t be the number of outer vertices. G − X has t odd components (each outer vertex is isolated in G − X ), so qG (X ) − |X | = t − s. Hence by the trivial part of the Berge-Tutte formula, any matching must leave at least t − s vertices uncovered. But on the other hand, the number of vertices not covered by M, i.e. the number of roots of F, is exactly t − s by Proposition 10.27. Hence M is indeed maximum. Since this is not at all a trivial task, we shall spend some time on implementation details. The difficult question is how to perform the shrinking efficiently so that the original graph can be recovered afterwards. Of course, several shrinking operations may involve the same vertex. Our presentation is based on the one given by Lov´asz and Plummer [1986]. Rather than actually performing the shrinking operation, we allow our forest to contain blossoms. Definition 10.28. Given a graph G and a matching M in G. A subgraph F of G is a general blossom forest (with respect to M) if there exists a partition . . . V (F) = V1 ∪ V2 ∪ · · · ∪ Vk of the vertex set such that Fi := F[Vi ] is a maximal factor-critical subgraph of F with |M ∩ E(Fi )| = |Vi 2|−1 (i = 1, . . . , k) and after contracting each of V1 , . . . , Vk we obtain an alternating forest F 0 . Fi is called an outer blossom (inner blossom) if Vi is an outer (inner) vertex in F 0 . All the vertices of an outer (inner) blossom are called outer (inner). A general blossom forest where each inner blossom is a single vertex is a special blossom forest. Figure 10.8 shows a connected component of a special blossom forest with five nontrivial outer blossoms. This corresponds to one of the connected components of the alternating forest in Figure 10.7. The orientations of the edges will be explained later. All vertices of G not belonging to the special blossom forest are called out-of-forest. Note that the Blossom Shrinking Lemma 10.25 applies to outer blossoms only. However, in this section we shall deal only with special blossom forests. General blossom forests will appear only in the W EIGHTED M ATCHING A LGORITHM in Chapter 11. To store a special blossom forest F, we introduce the following data structures. For each vertex x ∈ V (G) we have three variables µ(x), ϕ(x), and ρ(x) with the following properties:

10.5 Edmonds’ Matching Algorithm

245

y

x

Fig. 10.8.

x y ⎧ ⎪ x ⎪ ⎪ ⎪ ⎪ ⎪ ⎨y ϕ(x) = y ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎩ ⎧ ⎪ ⎨x ρ(x) = y ⎪ ⎩

µ(x) =

if x is not covered by M if {x, y} ∈ M

(10.2)

if x ∈ / V (F) or x is the base of an outer blossom for {x, y} ∈ E(F) \ M if x is an inner vertex (10.3) such that {x, y} ∈ E(F) \ M, and µ and ϕ are associated with an M-alternating ear-decomposition of the blossom containing x, if x is an outer vertex if x is not an outer vertex if x is an outer vertex and y is the base of the outer blossom in F containing x

(10.4)

For each outer vertex v we deﬁne P(v) to be the maximal path given by an initial subsequence of v, µ(v), ϕ(µ(v)), µ(ϕ(µ(v))), ϕ(µ(ϕ(µ(v)))), . . .

246

10 Maximum Matchings

We have the following properties: Proposition 10.29. Let F be a special blossom forest with respect to a matching M, and let µ, ϕ : V (G) → V (G) be functions satisfying (10.2) and (10.3). Then we have: (a) For each outer vertex v, P(v) is an alternating v-q-path, where q is the root of the tree of F containing v. (b) A vertex x is • outer iff either µ(x) = x or ϕ(µ(x)) 6= µ(x); • inner iff ϕ(µ(x)) = µ(x) and ϕ(x) 6= x; • out-of-forest iff µ(x) 6= x and ϕ(x) = x and ϕ(µ(x)) = µ(x). Proof: (a): By (10.3) and Lemma 10.22, some initial subsequence of v, µ(v), ϕ(µ(v)), µ(ϕ(µ(v))), ϕ(µ(ϕ(µ(v)))), . . . must be an M-alternating path of even length to the base r of the blossom containing v. If r is not the root of the tree containing v, then r is covered by M. Hence the above sequence continues with the matching edge {r, µ(r )} and also with {µ(r ), ϕ(µ(r ))}, because µ(r ) is an inner vertex. But ϕ(µ(r )) is an outer vertex again, and so we are done by induction. (b): If a vertex x is outer, then it is either a root (i.e. µ(x) = x) or P(x) is a path of length at least two, i.e. ϕ(µ(x)) 6= µ(x). If x is inner, then µ(x) is the base of an outer blossom, so by (10.3) ϕ(µ(x)) = µ(x). Furthermore, P(µ(x)) is a path of length at least 2, so ϕ(x) 6= x. If x is out-of-forest, then by definition x is covered by M, so by (10.2) µ(x) 6= x. Of course µ(x) is also out-of-forest, so by (10.3) we have ϕ(x) = x and ϕ(µ(x)) = µ(x). Since each vertex is either outer or inner or out-of-forest, and each vertex satisfies exactly one of the three right-hand side conditions, the proof is complete. In Figure 10.8, an edge is oriented from u to v if ϕ(u) = v. We are now ready for a detailed description of the algorithm.

E DMONDS ’ C ARDINALITY M ATCHING A LGORITHM Input:

A graph G.

Output:

A maximum matching in G given by the edges {x, µ(x)}.

1

2

Set µ(v) := v, ϕ(v) := v, ρ(v) := v and scanned(v) := false for all v ∈ V (G). If all outer vertices are scanned then stop, else let x be an outer vertex with scanned(x) = false.

10.5 Edmonds’ Matching Algorithm

3

4

5

6

247

Let y be a neighbour of x such that y is out-of-forest or (y is outer and ρ(y) 6= ρ(x)). If there is no such y then set scanned(x) := true and go to . 2 (“grow”) If y is out-of-forest then set ϕ(y) := x and go to . 3 (“augment”) If P(x) and P(y) are vertex-disjoint then Set µ(ϕ(v)) := v, µ(v) := ϕ(v) for all v ∈ V (P(x)) ∪ V (P(y)) with odd distance from x or y on P(x) or P(y), respectively. Set µ(x) := y. Set µ(y) := x. Set ϕ(v) := v, ρ(v) := v, scanned(v) := false for all v ∈ V (G). Go to . 2 (“shrink”) Let r be the first vertex on V (P(x)) ∩ V (P(y)) with ρ(r ) = r . For v ∈ V (P(x)[x,r ] ) ∪ V (P(y)[y,r ] ) with odd distance from x or y on P(x)[x,r ] or P(y)[y,r ] , respectively, and ρ(ϕ(v)) 6= r do: Set ϕ(ϕ(v)) := v. If ρ(x) 6= r then set ϕ(x) := y. If ρ(y) 6= r then set ϕ(y) := x. For all v ∈ V (G) with ρ(v) ∈ V (P(x)[x,r ] ) ∪ V (P(y)[y,r ] ) do: Set ρ(v) := r . Go to . 3

For an illustration of the effect of shrinking on the ϕ-values, see Figure 10.9, where 6 of the algorithm has been applied to x and y in Figure 10.8. Lemma 10.30. The following statements hold at any stage of E DMONDS ’ C AR DINALITY M ATCHING A LGORITHM : (a) The edges {x, µ(x)} form a matching M; (b) The edges {x, µ(x)} and {x, ϕ(x)} form a special blossom forest F with respect to M (plus some isolated matching edges); (c) The properties (10.2), (10.3) and (10.4) are satisfied with respect to F. Proof: (a): The only place where µ is changed is , 5 where the augmentation is obviously done correctly. (b): Since after 1 and 5 we trivially have a blossom forest without any edges and 4 correctly grows the blossom forest by two edges, we only have to check

. 6 r either is a root or must have degree at least three, so it must be outer. Let B := V (P(x)[x,r ] ) ∪ V (P(y)[y,r ] ). Consider an edge {u, v} of the blossom forest with u ∈ B and v ∈ / B. Since F[B] contains a near-perfect matching, {u, v} is a matching edge only if it is {r, µ(r )}. Moreover, u has been outer before applying

. 6 This implies that F continues to be a special blossom forest.

248

10 Maximum Matchings

y

x

r

Fig. 10.9.

(c): Here the only nontrivial fact is that, after shrinking, µ and ϕ are associated with an alternating ear-decomposition of the new blossom. So let x and y be two outer vertices in the same connected component of the special blossom forest, and let r be the ﬁrst vertex of V (P(x))∩V (P(y)) for which ρ(r ) = r . The new blossom consists of the vertices B := {v ∈ V (G) : ρ(v) ∈ V (P(x)[x,r ] ) ∪ V (P(y)[y,r ] )}. We note that ϕ(v) is not changed for any v ∈ B with ρ(v) = r . So the eardecomposition of the old blossom B := {v ∈ V (G) : ρ(v) = r } is the starting point of the ear-decomposition of B. The next ear consists of P(x)[x,x ] , P(y)[y,y ] , and the edge {x, y}, where x and y is the ﬁrst vertex on P(x) and P(y), respectively, that belongs to B . Finally, for each ear Q of an old outer blossom B ⊆ B, Q \ (E(P(x)) ∪ E(P(y))) is an ear of the new ear-decomposition of B. Theorem 10.31. (Edmonds [1965]) E DMONDS ’ C ARDINALITY M ATCHING A LGORITHM correctly determines a maximum matching in O(n 3 ) time, where n = |V (G)|. Proof: Lemma 10.30 and Proposition 10.29 show that the algorithm works correctly. Consider the situation when the algorithm terminates. Let M and F be the

10.5 Edmonds’ Matching Algorithm

249

matching and the special blossom forest according to Lemma 10.30(a) and (b). It is clear that any neighbour of an outer vertex x is either inner or a vertex y belonging to the same blossom (i.e. ρ(y) = ρ(x)). To show that M is a maximum matching, let X denote the set of inner vertices, while B is the set of vertices that are the base of some outer blossom in F. Then every unmatched vertex belongs to B, and the matched vertices of B are matched with elements of X : |B| = |X | + |V (G)| − 2|M|. (10.5) On the other hand, the outer blossoms in F are odd connected components in G − X . Therefore any matching must leave at least |B| − |X | vertices uncovered. By (10.5), M leaves exactly |B| − |X | vertices uncovered and thus is maximum. We now consider the running time. By Proposition 10.29(b), the status of each vertex (inner, outer, or out-of-forest) can be checked in constant time. Each of , 4 , 5 6 can be done in O(n) time. Between two augmentations, 4 or 6 are executed at most O(n) times, since the number of ﬁxed points of ϕ decreases each time. Moreover, between two augmentations no vertex is scanned twice. Thus the time spent between two augmentations is O(n 2 ), yielding an O(n 3 ) total running time. √ Micali and Vazirani [1980] improved the running time to O n m . They used the results of Exercise 9, but the existence of blossoms makes the search for a maximal set of disjoint minimum length augmenting paths more difﬁcult than in the bipartite case (which was solved earlier by Hopcroft and Karp [1973], see Exercise 10). See also Vazirani [1994]. The currently best √known2 time complexity /m) for the C ARDINALITY M ATCHING P ROBLEM is O m n log(n , just as in log n the bipartite case. This was obtained by Goldberg and Karzanov [2004] and by Fremuth-Paeger and Jungnickel [2003]. With the matching algorithm we can easily prove the Gallai-Edmonds Structure Theorem. This was ﬁrst proved by Gallai, but E DMONDS ’ C ARDINALITY M ATCHING A LGORITHM turns out to be a constructive proof thereof.

Y

X

W Fig. 10.10.

250

10 Maximum Matchings

Theorem 10.32. (Gallai [1964]) Let G be any graph. Denote by Y the set of vertices not covered by at least one maximum matching, by X the neighbours of Y in V (G) \ Y , and by W all other vertices. Then: (a) Any maximum matching in G contains a perfect matching of G[W ] and nearperfect matchings of the connected components of G[Y ], and matches all vertices in X to distinct connected components of G[Y ]; (b) The connected components of G[Y ] are factor-critical; (c) 2ν(G) = |V (G)| − qG (X ) + |X |. We call W, X, Y the Gallai-Edmonds decomposition of G (see Figure 10.10). Proof: We apply E DMONDS ’ C ARDINALITY M ATCHING A LGORITHM and consider the matching M and the special blossom forest F at termination. Let X 0 be the set of inner vertices, Y 0 the set of outer vertices, and W 0 the set of out-offorest vertices. We first prove that X 0 , Y 0 , W 0 satisfy (a)–(c), and then observe that X = X 0 , Y = Y 0 , and W = W 0 . The proof of Theorem 10.31 shows that 2ν(G) = |V (G)| − qG (X 0 ) + |X 0 |. We apply Proposition 10.15 to X 0 . Since the odd connected components of G − X 0 are exactly the outer blossoms in F, (a) holds for X 0 , Y 0 , W 0 . Since the outer blossoms are factor-critical, (b) also holds. Since part (a) holds for X 0 , Y 0 , and W 0 , we know that any maximum matching covers all the vertices in V (G) \ Y 0 . In other words, Y ⊆ Y 0 . We claim that Y 0 ⊆ Y also holds. Let v be an outer vertex in F. Then M4E(P(v)) is a maximum matching M 0 , and M 0 does not cover v. So v ∈ Y . Hence Y = Y 0 . This implies X = X 0 and W = W 0 , and the theorem is proved.

Exercises

∗

1. Let G be a graph and M1 , M2 two maximal matchings in G. Prove that |M1 | ≤ 2|M2 |. 2. Let α(G) denote the size of a maximum stable set in G, and ζ (G) the minimum cardinality of an edge cover. Prove: (a) α(G) + τ (G) = |V (G)| for any graph G. (b) ν(G) + ζ (G) = |V (G)| for any graph G with no isolated vertices. (c) ζ (G) = α(G) for any bipartite graph G with no isolated vertices. (K¨onig [1933], Gallai [1959]) 3. Prove that a k-regular bipartite graph has k disjoint perfect matchings. Deduce from this that the edge set of a bipartite graph of maximum degree k can be partitioned into k matchings. (K¨onig [1916]; see Rizzi [1998] or Theorem 16.16) 4. A partially ordered set (or poset) is defined to be a set S together with a partial order on S, i.e. a relation R ⊆ S × S that is reflexive ((x, x) ∈ R for all x ∈ S), symmetric (if (x, y) ∈ R and (y, x) ∈ R then x = y), and transitive (if

Exercises

251

(x, y) ∈ R and (y, z) ∈ R then (x, z) ∈ R). Two elements x, y ∈ S are called comparable if (x, y) ∈ R or (y, x) ∈ R, otherwise they are incomparable. A chain (an antichain) is a subset of pairwise comparable (incomparable) elements of S. Use K¨onig’s Theorem 10.2 to prove the following theorem of Dilworth [1950]: In a finite poset the maximum size of an antichain equals the minimum number of chains into which the poset can be partitioned. Hint: Take two copies v 0 and v 00 of each v ∈ S and consider the graph with an edge {v 0 , w00 } for each (v, w) ∈ R. (Fulkerson [1956]) 5. (a) Let S = {1, 2, . . . , n} and 0 ≤ k < n2 . Let A and B be the collection of all k-element and (k + 1)-element subsets of S, respectively. Construct a bipartite graph .

G = (A ∪ B, {{a, b} : a ∈ A, b ∈ B, a ⊆ b}). ∗

6.

7.

8.

9.

Prove that G has a matching covering A. (b) Prove Sperner’s Lemma: the maximum number of subsets of an n-element set such that none is contained in any other is b nn c . 2 (Sperner [1928]) Let (U, S) be a set system. An injective function 8 : S → U such that 8(S) ∈ S for all S ∈ S is called a system of distinct representatives of S. Prove: (a) S has a system of distinct representatives if and only if the union of any k of the sets in S has cardinality at least k. (Hall [1935]) (b) P For u ∈ U P let r (u) := |{S ∈ S : u ∈ S}|. Let n := |S| and N := N N |S| = S∈S u∈U r (u). Suppose |S| < n−1 for S ∈ S and r (u) < n−1 for u ∈ U . Then S has a system of distinct representatives. (Mendelsohn and Dulmage [1958]) . Let G be a bipartite graph with bipartition V (G) = A ∪ B. Suppose that S ⊆ A, T ⊆ B, and there is a matching covering S and a matching covering T . Prove that then there is a matching covering S ∪ T . (Mendelsohn and Dulmage [1958]) Show that any simple graph on n vertices with minimum degree k has a matching of cardinality min{k, b n2 c}. Hint: Use Berge’s Theorem 10.7. Let G be a graph and M a matching in G that is not maximum. (a) Show that there are ν(G) − |M| vertex-disjoint M-augmenting paths in G. Hint: Recall the proof of Berge’s Theorem 10.7. (b) Prove that there exists an M-augmenting path of length at most ν(G)+|M| ν(G)−|M| in G. (c) Let P be a shortest M-augmenting path in G, and P 0 an (M4E(P))augmenting path. Then |E(P 0 )| ≥ |E(P)| + |E(P ∩ P 0 )|.

252

10 Maximum Matchings

Consider the following generic algorithm. We start with the empty matching and in each iteration augment the matching along a shortest augmenting path. Let P1 , P2 , . . . be the sequence of augmenting paths chosen. By (c), |E(Pk )| ≤ |E(Pk+1 )| for all k. (d) Show that if |E(Pi )| = |E(Pj )| for i 6= j then Pi and Pj are vertex-disjoint. (e) Use √ (b) to prove that the sequence |E(P1 )|, |E(P2 )|, . . . contains at most 2 ν(G) + 2 different numbers. (Hopcroft and Karp [1973]) ∗ 10. Let G be a bipartite graph and consider the generic algorithm of Exercise 9. (a) Prove that – given a matching M – the union of all shortest M-augmenting paths in G can be found in O(n + m) time. Hint: Use a kind of breadth-first search with matching edges and nonmatching edges alternating. (b) Consider a sequence of iterations of the algorithm where the length of the augmenting path remains constant. Show that the time needed for the whole sequence is no more than O(n + m). Hint: First apply (a) and then find the paths successively by DFS. Mark vertices already visited. √ (c) Combine (b) with Exercise 9(e) to obtain an O n(m + n) -algorithm for the C ARDINALITY M ATCHING P ROBLEM in bipartite graphs. (Hopcroft and Karp [1973]) . 11. Let G be a bipartite graph with bipartition V (G) = A ∪ B, A = {a1 , . . . , ak }, B = {b1 , . . . , bk }. For any vector x = (xe )e∈E(G) we define a matrix MG (x) = (m ixj )1≤i, j≤k by ( xe if e = {ai , b j } ∈ E(G) x . m i j := 0 otherwise Its determinant det MG (x) is a polynomial in x = (xe )e∈E(G) . Prove that G has a perfect matching if and only if det MG (x) is not identically zero. 12. The permanent of a square matrix M = (m i j )1≤i, j≤n is defined by per(M) :=

k X Y

m i,π(i) ,

π ∈Sn i=1

where Sn is the set of permutations of {1, . . . , n}. Prove that a simple bipartite graph G has exactly per(MG (1l)) perfect matchings, where MG (x) is defined as in the previous exercise. 13. A doubly stochastic matrix is a nonnegative square matrix whose column sums and row sums are all 1. Integral doubly stochastic matrices are called permutation matrices. Falikman [1981] and Egoryˇcev [1980] proved that for a doubly stochastic n × n-matrix M, n! per(M) ≥ n , n

Exercises

253

and equality holds if and only if every entry of M is n1 . (This was a famous conjecture of van der Waerden; see also Schrijver [1998].) Br`egman [1973] proved that for a 0-1-matrix M with row sums r1 , . . . , rn , 1

1

per(M) ≤ (r1 !) r1 · . . . · (rn !) rn . Use these results and Exercise 12 to prove the following. Let G be a simple k-regular bipartite graph on 2n vertices, and let 8(G) be the number of perfect matchings in G. Then n n k ≤ 8(G) ≤ (k!) k . n! n 14. Prove that every 3-regular graph with at most two bridges has a perfect matching. Is there a 3-regular graph without a perfect matching? Hint: Use Tutte’s Theorem 10.13. (Petersen [1891]) ∗ 15. Let G be a graph, n := |V (G)| even, and for any set X ⊆ V (G) with |X | ≤ 34 n we have [ 4 0(x) ≥ |X |. 3 x∈X

16. 17. ∗ 18.

19.

20. 21. ∗ 22.

Prove that G has a perfect matching. Hint: Let S be a set violating the Tutte condition. Prove that the number of components in G − S with just one element is at most n connected o 4 1 max 0, 3 |S| − 3 n . Consider the cases |S| ≥ n4 and |S| < n4 separately. (Anderson [1971]) Prove that an undirected graph G is factor-critical if and only if G is connected and ν(G) = ν(G − v) for all v ∈ V (G). Prove that the number of ears in any two odd ear-decompositions of a factorcritical graph G is the same. For a 2-edge-connected graph G let ϕ(G) be the minimum number of even ears in an ear-decomposition of G (cf. Exercise 17(a) of Chapter 2). Show that for any edge e ∈ E(G) we have either ϕ(G/e) = ϕ(G)+1 or ϕ(G/e) = ϕ(G)−1. Note: The function ϕ(G) has been studied by Szigeti [1996] and Szegedy and Szegedy [2006]. Prove that a minimal factor-critical graph G (i.e. after the deletion of any edge the graph is no longer factor-critical) has at most 32 (|V (G)| − 1) edges. Show that this bound is tight. Show how E DMONDS ’ C ARDINALITY M ATCHING A LGORITHM finds a maximum matching in the graph shown in Figure 10.1(b). Given an undirected graph, can one find an edge cover of minimum cardinality in polynomial time? Given an undirected graph G, an edge is called unmatchable if it is not contained in any perfect matching. How can one determine the set of unmatchable edges

254

10 Maximum Matchings

in O(n 3 ) time? Hint: First determine a perfect matching in G. Then determine for each vertex v the set of unmatchable edges incident to v. 23. Let G be a graph, M a maximum matching in G, and F1 and F2 two special blossom forests with respect to M, each with the maximum possible number of edges. Show that the set of inner vertices in F1 and F2 is the same. 24. Let G be a k-connected graph with 2ν(G) < |V (G)| − 1. Prove: (a) ν(G) ≥ k; (b) τ (G) ≤ 2ν(G) − k. Hint: Use the Gallai-Edmonds Theorem 10.32. (Erd˝os and Gallai [1961])

References General Literature: Gerards, A.M.H. [1995]: Matching. In: Handbooks in Operations Research and Management Science; Volume 7: Network Models (M.O. Ball, T.L. Magnanti, C.L. Monma, G.L. Nemhauser, eds.), Elsevier, Amsterdam 1995, pp. 135–224 Lawler, E.L. [1976]: Combinatorial Optimization; Networks and Matroids. Holt, Rinehart and Winston, New York 1976, Chapters 5 and 6 Lov´asz, L., and Plummer, M.D. [1986]: Matching Theory. Akad´emiai Kiad´o, Budapest 1986, and North-Holland, Amsterdam 1986 Papadimitriou, C.H., and Steiglitz, K. [1982]: Combinatorial Optimization; Algorithms and Complexity. Prentice-Hall, Englewood Cliffs 1982, Chapter 10 Pulleyblank, W.R. [1995]: Matchings and extensions. In: Handbook of Combinatorics; Vol. 1 (R.L. Graham, M. Gr¨otschel, L. Lov´asz, eds.), Elsevier, Amsterdam 1995 Schrijver, A. [2003]: Combinatorial Optimization: Polyhedra and Efficiency. Springer, Berlin 2003, Chapters 16 and 24 Tarjan, R.E. [1983]: Data Structures and Network Algorithms. SIAM, Philadelphia 1983, Chapter 9

Cited References: Alt, H., Blum, N., Mehlhorn, K., and Paul, [1991]: Computing a maximum cardinality M. p matching in a bipartite graph in time O n 1.5 m/ log n . Information Processing Letters 37 (1991), 237–240 Anderson, I. [1971]: Perfect matchings of a graph. Journal of Combinatorial Theory B 10 (1971), 183–186 Berge, C. [1957]: Two theorems in graph theory. Proceedings of the National Academy of Science of the U.S. 43 (1957), 842–844 Berge, C. [1958]: Sur le couplage maximum d’un graphe. Comptes Rendus Hebdomadaires des S´eances de l’Acad´emie des Sciences (Paris) S´er. I Math. 247 (1958), 258–259 Br`egman, L.M. [1973]: Certain properties of nonnegative matrices and their permanents. Doklady Akademii Nauk SSSR 211 (1973), 27–30 [in Russian]. English translation: Soviet Mathematics Doklady 14 (1973), 945–949

References

255

Dilworth, R.P. [1950]: A decomposition theorem for partially ordered sets. Annals of Mathematics 51 (1950), 161–166 Edmonds, J. [1965]: Paths, trees, and flowers. Canadian Journal of Mathematics 17 (1965), 449–467 Egoryˇcev, G.P. [1980]: Solution of the van der Waerden problem for permanents. Soviet Mathematics Doklady 23 (1982), 619–622 Erd˝os, P., and Gallai, T. [1961]: On the minimal number of vertices representing the edges of a graph. Magyar Tudom´anyos Akad´emia; Matematikai Kutat´o Int´ezet´enek K¨ozlem´enyei 6 (1961), 181–203 Falikman, D.I. [1981]: A proof of the van der Waerden conjecture on the permanent of a doubly stochastic matrix. Matematicheskie Zametki 29 (1981), 931–938 [in Russian]. English translation: Math. Notes of the Acad. Sci. USSR 29 (1981), 475–479 Feder, T., and Motwani, R. [1995]: Clique partitions, graph compression and speeding-up algorithms. Journal of Computer and System Sciences 51 (1995), 261–272 Fremuth-Paeger, C., and Jungnickel, D. [2003]: Balanced network flows VIII: a revised √ theory of phase-ordered algorithms and the O( nm log(n 2 /m)/ log n) bound for the nonbipartite cardinality matching problem. Networks 41 (2003), 137–142 ¨ Frobenius, G. [1917]: Uber zerlegbare Determinanten. Sitzungsbericht der K¨oniglich Preussischen Akademie der Wissenschaften XVIII (1917), 274–277 Fulkerson, D.R. [1956]: Note on Dilworth’s decomposition theorem for partially ordered sets. Proceedings of the AMS 7 (1956), 701–702 ¨ Gallai, T. [1959]: Uber extreme Punkt- und Kantenmengen. Annales Universitatis Scientiarum Budapestinensis de Rolando E¨otv¨os Nominatae; Sectio Mathematica 2 (1959), 133–138 Gallai, T. [1964]: Maximale Systeme unabh¨angiger Kanten. Magyar Tudom´anyos Akad´emia; Matematikai Kutat´o Int´ezet´enek K¨ozlem´enyei 9 (1964), 401–413 Geelen, J.F. [2000]: An algebraic matching algorithm. Combinatorica 20 (2000), 61–70 Geelen, J. and Iwata, S. [2005]: Matroid matching via mixed skew-symmetric matrices. Combinatorica 25 (2005), 187–215 Goldberg, A.V., and Karzanov, A.V. [2004]: Maximum skew-symmetric flows and matchings. Mathematical Programming A 100 (2004), 537–568 Hall, P. [1935]: On representatives of subsets. Journal of the London Mathematical Society 10 (1935), 26–30 Halmos, P.R., and Vaughan, H.E. [1950]: The marriage problem. American Journal of Mathematics 72 (1950), 214–215 Hopcroft, J.E., and Karp, R.M. [1973]: An n 5/2 algorithm for maximum matchings in bipartite graphs. SIAM Journal on Computing 2 (1973), 225–231 ¨ K¨onig, D. [1916]: Uber Graphen und ihre Anwendung auf Determinantentheorie und Mengenlehre. Mathematische Annalen 77 (1916), 453–465 K¨onig, D. [1931]: Graphs and matrices. Matematikai´es Fizikai Lapok 38 (1931), 116–119 [in Hungarian] ¨ K¨onig, D. [1933]: Uber trennende Knotenpunkte in Graphen (nebst Anwendungen auf Determinanten und Matrizen). Acta Litteratum ac Scientiarum Regiae Universitatis Hungaricae Francisco-Josephinae (Szeged). Sectio Scientiarum Mathematicarum 6 (1933), 155–179 Kuhn, H.W. [1955]: The Hungarian method for the assignment problem. Naval Research Logistics Quarterly 2 (1955), 83–97 Lov´asz, L. [1972]: A note on factor-critical graphs. Studia Scientiarum Mathematicarum Hungarica 7 (1972), 279–280 Lov´asz, L. [1979]: On determinants, matchings and random algorithms. In: Fundamentals of Computation Theory (L. Budach, ed.), Akademie-Verlag, Berlin 1979, pp. 565–574

256

10 Maximum Matchings

Mendelsohn, N.S., and Dulmage, A.L. [1958]: Some generalizations of the problem of distinct representatives. Canadian Journal of Mathematics 10 (1958), 230–241 Micali, S., and Vazirani, V.V. [1980]: An O(V 1/2 E) algorithm for finding maximum matching in general graphs. Proceedings of the 21st Annual IEEE Symposium on Foundations of Computer Science (1980), 17–27 Mucha, M., and Sankowski, P. [2004]: Maximum matchings via Gaussian elimination. Proceedings of the 45th Annual IEEE Symposium on Foundations of Computer Science (2004), 248–255 Mulmuley, K., Vazirani, U.V., and Vazirani, V.V. [1987]: Matching is as easy as matrix inversion. Combinatorica 7 (1987), 105–113 Petersen, J. [1891]: Die Theorie der regul¨aren Graphen. Acta Mathematica 15 (1891), 193–220 Rabin, M.O., and Vazirani, V.V. [1989]: Maximum matchings in general graphs through randomization. Journal of Algorithms 10 (1989), 557–567 Rizzi, R. [1998]: K¨onig’s edge coloring theorem without augmenting paths. Journal of Graph Theory 29 (1998), 87 Schrijver, A. [1998]: Counting 1-factors in regular bipartite graphs. Journal of Combinatorial Theory B 72 (1998), 122–135 Sperner, E. [1928]: Ein Satz u¨ ber Untermengen einer endlichen Menge. Mathematische Zeitschrift 27 (1928), 544–548 Szegedy, B., and Szegedy, C. [2006]: Symplectic spaces and ear-decomposition of matroids. Combinatorica 26 (2006), 353–377 Szigeti, Z. [1996]: On a matroid defined by ear-decompositions. Combinatorica 16 (1996), 233–241 Tutte, W.T. [1947]: The factorization of linear graphs. Journal of the London Mathematical Society 22 (1947), 107–111 Vazirani, V.V. √ [1994]: A theory of alternating paths and blossoms for proving correctness of the O( V E) general graph maximum matching algorithm. Combinatorica 14 (1994), 71–109

11 Weighted Matching

Nonbipartite weighted matching appears to be one of the “hardest” combinatorial optimization problems that can be solved in polynomial time. We shall extend E DMONDS ’ C ARDINALITY M ATCHING A LGORITHM to the weighted case and shall again obtain an O(n 3 )-implementation. This algorithm has many applications, some of which are mentioned in the exercises and in Section 12.2. There are two basic formulations of the weighted matching problem:

M AXIMUM W EIGHT M ATCHING P ROBLEM Instance:

An undirected graph G and weights c : E(G) → R.

Task:

Find a maximum weight matching in G.

M INIMUM W EIGHT P ERFECT M ATCHING P ROBLEM Instance:

An undirected graph G and weights c : E(G) → R.

Task:

Find a minimum weight perfect matching in G or decide that G has no perfect matching.

It is easy to see that both problems are equivalent: Given an instance (G, c) of P ROBLEM, we set c0 (e) := K −c(e) the M INIMUM W EIGHT P ERFECT M ATCHING P for all e ∈ E(G), where K := 1 + e∈E(G) |c(e)|. Then any maximum weight matching in (G, c0 ) is a maximum cardinality matching, and hence gives a solution of the M INIMUM W EIGHT P ERFECT M ATCHING P ROBLEM (G, c). Conversely, let (G, c) be an instance of the M AXIMUM W EIGHT M ATCHING P ROBLEM. Then we add |V (G)| new vertices and all possible edges in order to obtain a complete graph G 0 on 2|V (G)| vertices. We set c0 (e) := −c(e) for all e ∈ E(G) and c0 (e) := 0 for all new edges e. Then a minimum weight perfect matching in (G 0 , c0 ) yields a maximum weight matching in (G, c), simply by deleting the edges not belonging to G. So in the following we consider only the M INIMUM W EIGHT P ERFECT M ATCHING P ROBLEM. As in the previous chapter, we start by considering bipartite graphs in Section 11.1. After an outline of the weighted matching algorithm in Section 11.2 we spend some effort on implementation details in Section 11.3 in order to obtain an O(n 3 ) running time. Sometimes one is interested in solving many matching problems that differ only on a few edges; in such a case it is not

258

11 Weighted Matching

necessary to solve the problem from scratch each time as is shown in Section 11.4. Finally, in Section 11.5 we discuss the matching polytope, i.e. the convex hull of the incidence vectors of matchings. We use a description of the related perfect matching polytope already for designing the weighted matching algorithm; in turn, this algorithm will directly imply that this description is complete.

11.1 The Assignment Problem The A SSIGNMENT P ROBLEM is just another name for the M INIMUM W EIGHT P ERFECT M ATCHING P ROBLEM in bipartite graphs. It is one of the classical combinatorial optimization problems; its history probably started with the work of Monge [1784]. As in the proof of Theorem 10.5, we can reduce the assignment problem to a network flow problem: Theorem 11.1. The A SSIGNMENT P ROBLEM can be solved in O(nm + n 2 log n) time. .

Proof: Let G be a bipartite graph with bipartition V (G) = A ∪ B. We assume |A| = |B| = n. Add a vertex s and connect it to all vertices of A, and add another vertex t connected to all vertices of B. Orient the edges from s to A, from A to B, and from B to t. Let the capacities be 1 everywhere, and let the new edges have zero cost. Then any integral s-t-flow of value n corresponds to a perfect matching with the same cost, and vice versa. So we have to solve a M INIMUM C OST F LOW P ROBLEM. We do this by applying the S UCCESSIVE S HORTEST PATH A LGORITHM (see Section 9.4). The total demand is n. So by Theorem 9.12, the running time is O(nm + n 2 log n). This is the fastest known algorithm. It is essentially equivalent to the “Hungarian method” by Kuhn [1955] and Munkres [1957], the oldest polynomial-time algorithm for the A SSIGNMENT P ROBLEM (cf. Exercise 9). It is worthwhile looking at the linear programming formulation of the A SSIGN MENT P ROBLEM . It turns out that in the integer programming formulation X X min c(e)xe : xe ∈ {0, 1} (e ∈ E(G)), xe = 1 (v ∈ V (G)) e∈E(G)

e∈δ(v)

the integrality constraints can be omitted (replace xe ∈ {0, 1} by xe ≥ 0): Theorem 11.2. Let G be a graph, and let

11.2 Outline of the Weighted Matching Algorithm

P :=

Q :=

E(G)

x ∈ R+

E(G)

x ∈ R+

259

X : xe ≤ 1 for all v ∈ V (G) and e∈δ(v) X : xe = 1 for all v ∈ V (G) e∈δ(v)

be the fractional matching polytope and the fractional perfect matching polytope of G. If G is bipartite, then P and Q are both integral. Proof: If G is bipartite, then the incidence matrix M of G is totally unimodular due to Theorem 5.25. Hence by the Hoffman-Kruskal Theorem 5.20, P is integral. Q is a face of P and thus it is also integral. There is a nice corollary concerning doubly-stochastic matrices. A doubly stochastic matrix is a nonnegative square matrix such that the sum of the entries in each row and each column is 1. Integral doubly stochastic matrices are called permutation matrices. Corollary 11.3. (Birkhoff [1946], von Neumann [1953]) Any doubly stochastic matrix M can be written as a convex combination of permutation matrices P1 , . . . , Pk (i.e. M = c1 P1 + . . . + ck Pk for nonnegative c1 , . . . , ck with c1 + . . . + ck = 1). Proof: Let M = (m i j )i, j∈{1,...,n} be a doubly stochastic n × n-matrix, and let . K n,n be the complete bipartite graph with bipartition {a1 , . . . , an } ∪ {b1 , . . . , bn }. For e = {ai , b j } ∈ E(K n,n ) let xe = m i j . Since M is doubly stochastic, x is in the fractional perfect matching polytope Q of K n,n . By Theorem 11.2 and Corollary 3.32, x can be written as a convex combination of integral vertices of Q. These obviously correspond to permutation matrices. This corollary can also be proved directly (Exercise 3).

11.2 Outline of the Weighted Matching Algorithm The purpose of this and the next section is to describe a polynomial-time algorithm for the general M INIMUM W EIGHT P ERFECT M ATCHING P ROBLEM. This algorithm was developed by Edmonds [1965] and uses the concepts of his algorithm for the C ARDINALITY M ATCHING P ROBLEM (Section 10.5). Let us first outline the main ideas without considering the implementation. Given a graph G with weights c : E(G) → R, the M INIMUM W EIGHT P ERFECT M ATCHING P ROBLEM can be formulated as the integer linear program X X min c(e)xe : xe ∈ {0, 1} (e ∈ E(G)), xe = 1 (v ∈ V (G)) . e∈E(G)

e∈δ(v)

260

11 Weighted Matching

If A is a subset of V (G) with odd cardinality, any perfect matching must contain an odd number of edges in δ(A), in particular at least one. So adding the constraint X xe ≥ 1 e∈δ(A)

does not change anything. Throughout this chapter we use the notation A := {A ⊆ V (G) : |A| odd}. Now consider the LP relaxation: X c(e)xe min e∈E(G)

s.t. X e∈δ(v) X

xe ≥ 0 xe = 1

(e ∈ E(G)) (v ∈ V (G))

xe ≥ 1

(A ∈ A, |A| > 1)

(11.1)

e∈δ(A)

We shall prove later that the polytope described by (11.1) is integral; hence this LP describes the M INIMUM W EIGHT P ERFECT M ATCHING P ROBLEM (this will be Theorem 11.13, a major result of this chapter). In the following we do not need this fact, but will rather use the LP formulation as a motivation. To formulate the dual of (11.1), we introduce a variable z A for each primal constraint, i.e. for each A ∈ A. The dual linear program is: X max zA A∈A

s.t. X

zA ≥ 0 z A ≤ c(e)

(A ∈ A, |A| > 1) (e ∈ E(G))

(11.2)

A∈A:e∈δ(A)

Note that the dual variables z {v} for v ∈ V (G) are not restricted to be nonnegative. Edmonds’ algorithm is a primal-dual algorithm. It starts with the empty matching (xe = 0 for all e ∈ E(G)) and the feasible dual solution 1 2 min{c(e) : e ∈ δ(A)} if |A| = 1 . z A := 0 otherwise At any stage of the algorithm, z will be a feasible dual solution, and we have X xe > 0 ⇒ z A = c(e); zA > 0

⇒

A∈ A:e∈δ(A) X

xe ≤ 1.

e∈δ(A)

(11.3)

11.2 Outline of the Weighted Matching Algorithm

261

The algorithm stops when x is the incidence vector of a perfect matching (i.e. we have primal feasibility). Due to the complementary slackness conditions (11.3) (Corollary 3.23) we then have the optimality of the primal and dual solutions. As x is optimal for (11.1) and integral, it is the incidence vector of a minimum weight perfect matching. Given a feasible dual solution z, we call an edge e tight if the corresponding dual constraint is satisﬁed with equality, i.e. if z A = c(e). A∈A:e∈δ(A)

At any stage, the current matching will consist of tight edges only. We work with a graph G z which results from G by deleting all edges that are not tight and contracting each set B with z B > 0 to a single vertex. The family B := {B ∈ A : z B > 0} will be laminar at any stage, and each element of B will induce a factor-critical subgraph consisting of tight edges only. Initially B consists of the singletons. One iteration of the algorithm roughly proceeds as follows. We ﬁrst ﬁnd a maximum cardinality matching M in G z , using E DMONDS ’ C ARDINALITY M ATCHING A LGORITHM. If M is a perfect matching, we are done: we can complete M to a perfect matching in G using tight edges only. Since the conditions (11.3) are satisﬁed, the matching is optimal.

+ε

+ε

−ε

+ε

−ε

+ε

−ε

Y

X

W Fig. 11.1.

Otherwise we consider the Gallai-Edmonds decomposition W, X, Y of G z (cf. Theorem 10.32). For each vertex v of G z let B(v) ∈ B be the vertex set whose contraction resulted in v. We modify the dual solution as follows (see Figure 11.1 for an illustration). For each v ∈ X we decrease z B(v) by some positive constant ε. For each connected component C of G z [Y ] we increase z A by ε, where A = v∈C B(v).

262

11 Weighted Matching

Note that tight matching edges remain tight, since by Theorem 10.32 all matching edges with one endpoint in X have the other endpoint in Y . (Indeed, all edges of the alternating forest we are working with remain tight). We choose ε maximum possible while preserving dual feasibility. Since the current graph contains no perfect matching, the number of connected components |X |. Hence the above dual change increases the dual of G z [Y ] is greater than P objective function value A∈A z A by at least ε. If ε can be chosen arbitrarily large, the dual LP (11.2) is unbounded, hence the primal LP (11.1) is infeasible (Theorem 3.27) and G has no perfect matching. Due to the change of the dual solution the graph G z will also change: new edges may become tight, new vertex sets may be contracted (corresponding to the components of Y that are not singletons), and some contracted sets may be “unpacked” (non-singletons whose dual variables become zero, corresponding to vertices of X ). The above is iterated until a perfect matching is found. We shall show later that this procedure is finite. This will follow from the fact that between two augmentations, each step (grow, shrink, unpack) increases the number of outer vertices.

11.3 Implementation of the Weighted Matching Algorithm After this informal description we now turn to the implementation details. As with E DMONDS ’ C ARDINALITY M ATCHING A LGORITHM we do not explicitly shrink blossoms but rather store their ear-decomposition. However, there are several difficulties. The “shrink”-step of E DMONDS ’ C ARDINALITY M ATCHING A LGORITHM produces an outer blossom. By the “augment”-step two connected components of the blossom forest become out-of-forest. Since the dual solution remains unchanged, we must retain the blossoms: we get so-called out-of-forest blossoms. The “grow”step may involve out-of-forest blossoms which then become either inner or outer blossoms. Hence we have to deal with general blossom forests. Another problem is that we must be able to recover nested blossoms one by one. Namely, if z A becomes zero for some inner blossom A, there may be subsets A0 ⊆ A with |A0 | > 1 and z A0 > 0. Then we have to unpack the blossom A, but not the smaller blossoms inside A (except if they remain inner and their dual variables are also zero). Throughout the algorithm we have a laminar family B ⊆ A, containing at least / B. The all singletons. All elements of B are blossoms. We have z A = 0 for all A ∈ set B is laminar and is stored by a tree-representation (cf. Proposition 2.14). For easy reference, a number is assigned to each blossom in B that is not a singleton. We store ear-decompositions of all blossoms in B at any stage of the algorithm. The variables µ(x) for x ∈ V (G) again encode the current matching M. We denote by b1 (x), . . . , bk x (x) the blossoms in B containing x, without the singleton. bk x (x) is the outermost blossom. We have variables ρ i (x) and ϕ i (x) for each x ∈ V (G)

11.3 Implementation of the Weighted Matching Algorithm

263

and i = 1, . . . , k x . ρ i (x) is the base of the blossom bi (x). µ(x) and ϕ j (x), for all x and j with b j (x) = i, are associated with an M-alternating ear-decomposition of blossom i. Of course, we must update the blossom structures (ϕ and ρ) after each augmentation. Updating ρ is easy. Updating ϕ can also be done in linear time by Lemma 10.23. For inner blossoms we need, in addition to the base, the vertex nearest to the root of the tree in the general blossom forest, and the neighbour in the next outer blossom. These two vertices are denoted by σ (x) and χ (σ (x)) for each base x of an inner blossom. See Figure 11.2 for an illustration. y0

x0

y1

x2

x1 = µ(x0 )

y2 = ρ(y0 )

x3

x4

y3 = µ(y2 )

x5 = σ (x1 )

y4

y5 = σ (y3 )

y6 = χ (y5 )

x6 = χ (x5 ) Fig. 11.2.

With these variables, the alternating paths to the root of the tree can be determined. Since the blossoms are retained after an augmentation, we must choose the augmenting path such that each blossom still contains a near-perfect matching afterwards. Figure 11.2 shows that we must be careful: There are two nested inner blossoms, induced by {x3 , x4 , x5 } and {x1 , x2 , x3 , x4 , x5 }. If we just consider the eardecomposition of the outermost blossom to ﬁnd an alternating path from x0 to the root x6 , we will end up with (x0 , x1 , x4 , x5 = σ (x1 ), x6 = χ (x5 )). After augmenting along (y6 , y5 , y4 , y3 , y2 , y1 , y0 , x0 , x1 , x4 , x5 , x6 ), the factor-critical subgraph induced by {x3 , x4 , x5 } no longer contains a near-perfect matching. Thus we must ﬁnd an alternating path within each blossom which contains an even number of edges within each sub-blossom. This is accomplished by the following procedure:

264

11 Weighted Matching

B LOSSOM PATH Input:

A vertex x0 .

Output:

An M-alternating path Q(x0 ) from x0 to ρ k x0 (x0 ).

1

Set h := 0 and B := {b j (x0 ) : j = 1, . . . , k x0 }.

2

While x2h 6= ρ k x0 (x0 ) do: Set x2h+1 := µ(x2h ) and x2h+2 := ϕ i (x2h+1 ), where i = min j ∈ {1, . . . , k x2h+1 } : b j (x2h+1 ) ∈ B . Add all blossoms of B to B that contain x2h+2 but not x2h+1 . Delete all blossoms from B whose base is x2h+2 . Set h := h + 1. Let Q(x0 ) be the path with vertices x0 , x1 , . . . , x2h .

3

Proposition 11.4. The procedure B LOSSOM PATH can be implemented in O(n) time. M4E(Q(x0 )) contains a near-perfect matching within each blossom. Proof: Let us first check that the procedure indeed computes a path. In fact, if a blossom of B is left, it is never entered again. This follows from the fact that contracting the maximal sub-blossoms of any blossom in B results in a circuit (a property which will be maintained). At the beginning of each iteration, B is the list of all blossoms that either contain x0 or have been entered via a non-matching edge and have not been left yet. The constructed path leaves any blossom in B via a matching edge. So the number of edges within each blossom is even, proving the second statement of the proposition. When implementing the procedure in O(n) time, the only nontrivial task is the update of B. We store B as a sorted list. Using the tree-representation of B and the fact that each blossom is entered and left at most once, we get a running time of O(n + |B|). Note that |B| = O(n), because B is laminar. Now determining an augmenting path consists of applying the procedure B LOS within blossoms, and using µ and χ between blossoms. When we find adjacent outer vertices x, y in different trees of the general blossom forest, we apply the following procedure to both x and y. The union of the two paths together with the edge {x, y} will be the augmenting path. SOM PATH

T REE PATH Input:

An outer vertex v.

Output:

An alternating path P(v) from v to the root of the tree in the blossom forest.

1

Let initially P(v) consist of v only. Let x := v.

2

Let y := ρ k x (x). Let Q(x) := B LOSSOM PATH(x). Append Q(x) to P(v). If µ(y) = y then stop.

11.3 Implementation of the Weighted Matching Algorithm

3

265

Set P(v) := P(v) + {y, µ(y)}. Let Q(σ (µ(y))) := B LOSSOM PATH(σ (µ(y))). Append the reverse of Q(σ (µ(y))) to P(v). Let P(v) := P(v) + {σ (µ(y)), χ (σ (µ(y)))}. Set x := χ (σ (µ(y))) and go to . 2

The second main problem is how to determine ε efficiently. The general blossom forest, after all possible grow-, shrink- and augment-steps are done, yields the GallaiEdmonds decomposition W, X, Y of G z . W contains the out-of-forest blossoms, X contains the inner blossoms, and Y consists of the outer blossoms. For a simpler notation, let us define c({v, w}) := ∞ if {v, w} ∈ / E(G). Moreover, we use the abbreviation X slack(v, w) := c({v, w}) − z A. A∈A, {v,w}∈δ(A)

So {v, w} is a tight edge if and only if slack(v, w) = 0. Then let ε1 := min{z A : A is a maximal inner blossom, |A| > 1}; ε2 := min {slack(x, y) : x outer, y out-of-forest} ; 1 min {slack(x, y) : x, y outer, belonging to different blossoms} ; ε3 := 2 ε := min{ε1 , ε2 , ε3 }. This ε is the maximum number such that the dual change by ε preserves dual feasibility. If ε = ∞, (11.2) is unbounded and so (11.1) is infeasible. In this case G has no perfect matching. Obviously, ε can be computed in finite time. However, in order to obtain an O(n 3 ) overall running time we must be able to compute ε in O(n) time. This is easy as far as ε1 is concerned, but requires additional data structures for ε2 and ε3 . For A ∈ B let X zB. ζ A := B∈B:A⊆B

We shall update these values whenever changing the dual solution; this can easily be done in linear time (using the tree-representation of B). Then ε2 = min c({x, y}) − ζ{x} − ζ{y} : x outer, y out-of-forest , 1 min c({x, y}) − ζ{x} − ζ{y} : x, y outer, belonging to different ε3 = 2 blossoms . To compute ε2 , we store for each out-of-forest vertex v the outer neighbour w for which slack(v, w) = c({v, w})−ζ{v} −ζ{w} is minimum. We call this neighbour

266

11 Weighted Matching

τv . These variables are updated whenever necessary. Then it is easy to compute ε2 = min{c({v, τv }) − ζ{v} − ζ{τv } : v out-of-forest}. To compute ε3 , we introduce variables tvA and τvA for each outer vertex v and each A ∈ B, unless A is outer but not maximal. τvA is the vertex in A minimizing slack(v, τvA ), and tvA = slack(v, τvA ) + 1 + ζ A , where 1 denotes the sum of the ε-values in all dual changes. Although when computing ε3 we are interested only in the values tvA for maximal outer blossoms of B, we update these variables also for inner and out-of-forest blossoms (even those that are not maximal), because they may become maximal outer later. Blossoms that are outer but not maximal will not become maximal outer before an augmentation takes place. After each augmentation, however, all these variables are recomputed. The variable tvA has the value slack(v, τvA ) + 1 + ζ A at any time. Observe that this value does not change as long as v remains outer, A ∈ B, and τvA is the vertex / A, v outer}. in A minimizing slack(v, τvA ). Finally, we write t A := min{tvA : v ∈ We conclude that ε3 =

1 1 1 slack(v, τvA ) = (tvA − 1 − ζ A ) = (t A − 1 − ζ A ), 2 2 2

where A is a maximal outer element of B for which t A − ζ A is minimum, and v is some outer vertex with v ∈ / A and tvA = t A . At certain stages we have to update τvA and tvA for a certain vertex v and all A ∈ B (except those that are outer but not maximal), for example if a new vertex becomes outer. The following procedure also updates the variables τw for out-of-forest vertices w if necessary.

U PDATE Input:

An outer vertex v.

Output:

Updated values of τvA , tvA and t A for all A ∈ B and τw for all out-offorest vertices w.

1

For each neighbour w of v that is out-of-forest do: If c({v, w}) − ζ{v} < c({w, τw }) − ζ{τw } then set τw := v.

2

For each x ∈ V (G) do: Set τv

3

For A ∈ B with |A| > 1 do: 0 0 Set inductively τvA := τvA and tvA := tvA − ζ A0 + ζ A , where A0 is the 0 one among the maximal proper subsets of A in B for which tvA − ζ A0 is minimum. For A ∈ B with v ∈ / A, except those that are outer but not maximal, do: Set t A := min{t A , tvA }.

4

{x}

{x}

:= x and tv

:= c({v, x}) − ζ{v} + 1.

Obviously this computation coincides with the above definition of τvA and tvA . It is important that this procedure runs in linear time:

11.3 Implementation of the Weighted Matching Algorithm

267

Lemma 11.5. If B is laminar, the procedure U PDATE can be implemented with O(n) time. Proof: By Proposition 2.15, a laminar family of subsets of V (G) has cardinality at most 2|V (G)| = O(n). If B is stored by its tree-representation, then a linear-time implementation is easy. We can now go ahead with the formal description of the algorithm. Instead of identifying inner and outer vertices by the µ-, φ- and ρ-values, we directly mark each vertex with its status (inner, outer or out-of-forest).

W EIGHTED M ATCHING A LGORITHM Input:

A graph G, weights c : E(G) → R.

Output:

A minimum weight perfect matching in G, given by the edges {x, µ(x)}, or the answer that G has no perfect matching.

1

2

3

4

5

6

Set B := {{v} : v ∈ V (G)} and K := 0. Set 1 := 0. Set z {v} := 21 min{c(e) : e ∈ δ(v)} and ζ{v} := z {v} for all v ∈ V (G). Set kv := 0, µ(v) := v, ρ 0 (v) := v, and ϕ 0 (v) := v for all v ∈ V (G). Mark all vertices as outer. For all v ∈ V (G) do: Set scanned(v) := false. For each out-of-forest vertex v do: Let τv be an arbitrary outer vertex. Set t A := ∞ for all A ∈ B. For all outer vertices v do: U PDATE(v). If all outer vertices are scanned then go to , 8 else let x be an outer vertex with scanned(x) = false. Let y be a neighbour of x such that {x, y} is tight and either y is out-of-forest or (y is outer and ρ k y (y) 6= ρ k x (x)). If there is no such y then set scanned(x) := true and go to . 3 If y is not out-of-forest then go to , 6 else: (“grow”) Set σ (ρ k y (y)) := y and χ (y) := x. Mark all vertices v with ρ kv (v) = ρ k y (y) as inner. Mark all vertices v with µ(ρ kv (v)) = ρ k y (y) as outer. For each new outer vertex v do: U PDATE(v). Go to . 4 Let P(x) := T REE PATH(x) be given by (x = x0 , x1 , x2 , . . . , x2h ). Let P(y) := T REE PATH(y) be given by (y = y0 , y1 , y2 , . . . , y2 j ).

268

7

8

11 Weighted Matching

If P(x) and P(y) are not vertex-disjoint then go to , 7 else: (“augment”) For i := 0 to h − 1 do: Set µ(x2i+1 ) := x2i+2 and µ(x2i+2 ) := x2i+1 . For i := 0 to j − 1 do: Set µ(y2i+1 ) := y2i+2 and µ(y2i+2 ) := y2i+1 . Set µ(x) := y and µ(y) := x. Mark all vertices v such that the endpoint of T REE PATH(v) is either x2h or y2 j as out-of-forest. Update all values ϕ i (v) and ρ i (v) for these vertices (using Lemma 10.23). If µ(v) 6= v for all v then stop, else go to . 2 (“shrink”) Let r = x2h 0 = y2 j 0 be the first outer vertex of V (P(x)) ∩ V (P(y)) with ρ kr (r ) = r . Let A := {v ∈ V (G) : ρ kv (v) ∈ V (P(x)[x,r ] ) ∪ V (P(y)[y,r ] )}. Set K := K + 1, B := B ∪ {A}, z A := 0 and ζ A := 0. For all v ∈ A do: Set kv := kv + 1, bkv (v) := K , ρ kv (v) := r , ϕ kv (v) := ϕ kv −1 (v) and mark v as outer. For i := 1 to h 0 do: If ρ k x2i (x2i ) 6= r then set ϕ k x2i (x2i ) := x2i−1 . If ρ k x2i−1 (x2i−1 ) 6= r then set ϕ k x2i−1 (x2i−1 ) := x2i . For i := 1 to j 0 do: If ρ k y2i (y2i ) 6= r then set ϕ k y2i (y2i ) := y2i−1 . If ρ k y2i−1 (y2i−1 ) 6= r then set ϕ k y2i−1 (y2i−1 ) := y2i . If ρ k x (x) 6= r then set ϕ k x (x) := y. If ρ k y (y) 6= r then set ϕ k y (y) := x. For each outer vertex v do: 0 0 Set tvA := tvA − ζ A0 and τvA := τvA , where A0 is the one among the 0 maximal proper subsets of A in B for which tvA − ζ A0 is minimum. ¯ Set t A := min{tvA : v outer, there is no A¯ ∈ B with A ∪ {v} ⊆ A}. For each new outer vertex v do: U PDATE(v). Go to . 4 (“dual change”) Set ε1 := min{z A : A maximal inner element of B, |A| > 1}. Set ε2 := min{c({v, τv }) − ζ{v} − ζ{τv } : v out-of-forest}. Set ε3 := min{ 21 (t A − 1 − ζ A ) : A maximal outer element of B}. Set ε := min{ε1 , ε2 , ε3 }. If ε = ∞, then stop (G has no perfect matching). If ε = ε2 = c({v, τv }) − ζ{v} − ζ{τv } , v outer then set scanned(τv ) := false. If ε = ε3 = 21 (tvA − 1 − ζ A ), A maximal outer element of B, v outer and v ∈ / A then set scanned(v) := false. For each maximal outer element A of B do: Set z A := z A + ε and ζ A0 := ζ A0 + ε for all A0 ∈ B with A0 ⊆ A. For each maximal inner element A of B do: Set z A := z A − ε and ζ A0 := ζ A0 − ε for all A0 ∈ B with A0 ⊆ A. Set 1 := 1 + ε.

11.3 Implementation of the Weighted Matching Algorithm

9

269

While there is a maximal inner A ∈ B with z A = 0 and |A| > 1 do: (“unpack”) Set B := B \ {A}. Let y := σ (ρ kv (v)) for some v ∈ A. Let Q(y) := B LOSSOM PATH(y) be given by (y = r0 , r1 , r2 , . . . , r2l−1 , r2l = ρ k y (y)). Mark all v ∈ A with ρ kv −1 (v) ∈ / V (Q(y)) as out-of-forest. Mark all v ∈ A with ρ kv −1 (v) = r2i−1 for some i as outer. For all v ∈ A with ρ kv −1 (v) = r2i for some i (v remains inner) do: Set σ (ρ kv (v)) := r j and χ (r j ) := r j−1 , where kr −1

j := min{j ∈ {0, . . . , 2l} : ρ j (r j ) = ρ kv −1 (v)}. For all v ∈ A do: Set kv := kv − 1. For each new out-of-forest vertex v do: Let τv be an outer vertex w for which c({v, w}) − ζ{v} − ζ{w} is minimum. For each new outer vertex v do: U PDATE(v). Go to . 3 Note that in contrast to our previous discussion, ε = 0 is possible. The variables τvA are not needed explicitly. The “unpack”-step 9 is illustrated in Figure 11.3, where a blossom with 19 vertices is unpacked. Two of the ﬁve sub-blossoms become out-of-forest, two become inner blossoms and one becomes an outer blossom. In , 6 the connected components of the blossom forest F have to be determined. This can be done in linear time by Proposition 2.17. (a)

(b) r10 r9

r8

r6

r7 r5 r3

r2

r4 y = r0

r1

Fig. 11.3.

270

11 Weighted Matching

(a) a 4

(b)

b

8

8

c

10 4

11

a 8

14 13

8

8

9 g

12

12

c

0

5

2 d

h

13

10

0 4

0 d

h

b

2

4 5

e

0

0

3 g

e

2

2

5

4

f

f Fig. 11.4.

Before analysing the algorithm, let us demonstrate its major steps by an example. Consider the graph in Figure 11.4(a). Initially, the algorithm sets z {a} = z {d} = z {h} = 2, z {b} = x{c} = z { f } = 4 and z {e} = z {g} = 6. In Figure 11.4(b) the slacks can be seen. So in the beginning the edges {a, d}, {a, h}, {b, c}, {b, f }, {c, f } are tight. We assume that the algorithm scans the vertices in alphabetical order. So the ﬁrst steps are augment(a, d),

augment(b, c),

grow( f, b).

Figure 11.5(a) shows the current general blossom forest. The next steps are shrink( f, c),

grow(h, a),

resulting in the general blossom forest shown in Figure 11.5(b). Now all the tight edges are used up, so the dual variables have to change. We perform 8 and obtain ε = ε3 = 1, say A = {b, c, f } and τvA = d. The new dual variables are z {b,c, f } = 1, z {a} = 1, z {d} = z {h} = 3, z {b} = z {c} = z { f } = 4, z {e} = z {g} = 7. The current slacks are shown in Figure 11.6(a). The next step is augment(d, c). (a)

(b) c

d

e

f

c

b

b

g

h

a

d Fig. 11.5.

e

f

a

g

h

11.3 Implementation of the Weighted Matching Algorithm (a) a 0

(b)

b

2

0

c

4 3

0

0

b

2 3

0

0

1 g

0

3

2

f g

e

e

0

c

d

d

h

271

a

h

f Fig. 11.6.

The blossom {b, c, f } becomes out-of-forest (Figure 11.6(b)). Now the edge {e, f } is tight, but in the previous dual change we have only set scanned(d) := false. So we need to do 8 with ε = ε3 = 0 twice to make the next steps grow(e, f ),

grow(d, a)

possible. We arrive at Figure 11.7(a). No more edges incident to outer vertices are tight, so we perform 8 once more. We have ε = ε1 = 1 and obtain the new dual solution z {b,c, f } = 0, z {a} = 0, z {d} = z {h} = z {b} = x{c} = z { f } = 4, z {e} = z {g} = 8. The new slacks are shown in Figure 11.7(b). Since the dual variable for the inner blossom {B, C, F} becomes zero, we have to unpack({b, c, f }). (a)

(b)

h

a

a 0 h

0 0

g Fig. 11.7.

0

1 0

0 f

e

0 d

2

g

f

c

6

1

b

0

3

d

c

b

4

1

2 e

272 (a)

11 Weighted Matching (b)

h

a

b

4

0

a d

0

c

0

f

e

d 2

0

1

1 g

0

0

2

h

b

c

7

0

0

0

2 e

f

g Fig. 11.8.

The general blossom forest we get is shown in Figure 11.8(a). After another dual variable change with ε = ε3 = 12 we obtain z {a} = −0.5, z {c} = z { f } = 3.5, z {b} = z {d} = z {h} = 4.5, z {e} = z {g} = 8.5 (the slacks are shown in Figure 11.8(b)). The ﬁnal steps are shrink(d, e), augment(g, h), and the algorithm terminates. The ﬁnal matching is M = {{e, f }, {b, c}, {a, d}, {g, h}}. We check that M has total weight 37, equal to the sum of the dual variables. Let us now check that the algorithm works correctly. Proposition 11.6. The following statements hold at any stage of the W EIGHTED M ATCHING A LGORITHM: (a) B is a laminar family. B = {v ∈ V (G) : bi (v) = j for some i} : j = 1, . . . , B . The sets Vρ kr (r ) := {v : ρ kv (v) = ρ kr (r )} are exactly the maximal elements of B. The vertices in each Vr are marked either all outer or all inner or all out-of-forest. Each (Vr , {{v, ϕ kv (v)} : v ∈ Vr \ {r }} ∪ {{v, μ(v)} : v ∈ Vr \ {r }}) is a blossom with base r . (b) The edges {x, μ(x)} form a matching M. M contains a near-perfect matching within each element of B. (c) For each b ∈ {1, . . . , K } let X (b) := {v ∈ V (G) : bi (v) = b for some i}. Then the variables μ(v) and ϕ i (v), for those v and i with bi (v) = b, are associated with an M-alternating ear-decomposition in G[X (b)]. (d) The edges {x, μ(x)} and {x, ϕ i (x)} for all x and i, and the edges {σ (x), χ (σ (x))} for all bases x of maximal inner blossoms, are all tight.

11.3 Implementation of the Weighted Matching Algorithm

273

(e) The edges {x, µ(x)}, {x, ϕ k x (x)} for all inner or outer x, together with the edges {σ (x), χ (σ (x))} for all bases x of maximal inner blossoms, form a general blossom forest F with respect to M. The vertex marks (inner, outer, out-of-forest) are consistent with F. (f) Contracting the maximal sub-blossoms of any blossom in B results in a circuit. (g) For each outer vertex v, the procedure T REE PATH gives an M-alternating v-r -path, where r is the root of the tree in F containing v. Proof: The properties clearly hold at the beginning (after 2 is executed the first time). We show that they are maintained throughout the algorithm. This is easily seen for (a) by considering 7 and . 9 For (b), this follows from Proposition 11.4 and the assumption that (f) and (g) hold before augmenting. The proof that (c) continues to hold after shrinking is the same as in the non-weighted case (see Lemma 10.30 (c)). The ϕ-values are recomputed after augmenting and not changed elsewhere. (d) is guaranteed by . 4 It is easy to see that (e) is maintained by : 5 The blossom containing y was outof-forest, and setting χ (y) := x and σ (v) := y for the base v of the blossom makes it inner. The blossom containing µ(ρ k y (y)) was also out-of-forest, and becomes outer. In , 6 two connected components of the general blossom forest clearly become out-of-forest, so (e) is maintained. In , 7 the vertices in the new blossom clearly become outer because r was outer before. In , 9 for the vertices v ∈ A with / V (Q(y)) we also have µ(ρ kv (v)) ∈ / V (Q(y)), so they become out-ofρ kv −1 (v) ∈ forest. For each other v ∈ A we have ρ kv −1 (v) = rk for some k. Since {ri , ri+1 } ∈ M iff i is even, v becomes outer iff k is odd. (f) holds for any blossom, as any new blossom arises from an odd circuit in

. 7 To see that (g) is maintained, it suffices to observe that σ (x) and χ (σ (x)) are set correctly for all bases x of maximal inner blossoms. This is easily checked for both 5 and . 9 Proposition 11.6(a) justifies calling the maximal elements of B inner, outer or out-of-forest in 8 and 9 of the algorithm. Next we show that the algorithm maintains a feasible dual solution. Lemma 11.7. At any stage of the algorithm, z is a feasible dual solution. If ε = ∞ then G has no perfect matching. Proof: We always have z A = 0 for all A ∈ A \ B. z A is decreased only for those A ∈ B that are maximal in B and inner. So the choice of ε1 guarantees that z A continues to be nonnegativePfor all A with |A| > 1. P How can the constraints A∈A:e∈δ(A) z A ≤ c(e) be violated? If A∈A:e∈δ(A) z A increases in , 8 e must either connect an outer and an out-of-forest vertex or two different outer blossoms. So the maximal ε such that the new z still satisfies P 1 z A∈A:e∈δ(A) A ≤ c(e) is slack(e) in the first case and 2 slack(e) in the second case.

274

11 Weighted Matching

We thus have to prove that ε2 and ε3 are computed correctly: ε2 = min{slack(v, w) : v outer, w out-of-forest} and

1 min{slack(v, w) : v, w outer, ρ kv (v) 6= ρ kw (w)}. 2 For ε2 this is easy to see, since for any out-of-forest vertex v we always have that τv is the outer vertex w minimizing slack(v, w) = c({v, w}) − ζ{v} − ζ{w} . Now consider ε3 . We claim that at any stage of the algorithm the following holds for any outer vertex v and any A ∈ B such that there is no A¯ ∈ B with ¯ A ∪ {v} ⊆ A: ε3 =

(a) (b) (c) (d) (e)

τvA ∈ A. A slack(v, P τv ) = min{slack(v, u) : u ∈ A}. ζ A = B∈B:A⊆B z B . 1 is the sum of the ε-values in all dual changes so far. slack(v, τvA ) = tvA − 1 − ζ A . ¯ t A = min{tvA : v outer and there is no A¯ ∈ B with A ∪ {v} ⊆ A}.

(a), (c), and (e) are easily seen to be true. (b) and (d) hold when τvA is defined (in 7 or in U PDATE(v)), and afterwards slack(v, u) decreases exactly by the amount that 1 + ζ A increases (due to (c)). Now (a), (b), (d), and (e) imply that ε3 is computed correctly. Now suppose ε = ∞, i.e. ε can be chosen arbitrarily large without destroying dual feasibility. Since the dual objective 1lz increases by at least ε in , 8 we conclude that the dual LP (11.2) is unbounded. Hence by Theorem 3.27 the primal LP (11.1) is infeasible. Now the correctness of the algorithm follows: Theorem 11.8. If the algorithm terminates in , 6 the edges {x, µ(x)} form a minimum weight perfect matching in G. Proof: Let x be the incidence vector of M (the matching consisting of the edges {x, µ(x)}). The complementary slackness conditions X z A = c(e) xe > 0 ⇒ A∈A:e∈δ(A)

zA > 0 ⇒

X

xe = 1

e∈δ(A)

are satisfied: the first one holds since all the matching edges are tight (Proposition 11.6(d)), and the second one follows from Proposition 11.6(b). Since we have feasible primal and dual solutions (Lemma 11.7), both must be optimal (Corollary 3.23). So x is optimal for the LP (11.1) and integral, proving that M is a minimum weight perfect matching. Until now we have not proved that the algorithm terminates.

11.3 Implementation of the Weighted Matching Algorithm

275

Theorem 11.9. The running time of the W EIGHTED M ATCHING A LGORITHM between two augmentations is O(n 2 ). The overall running time is O(n 3 ). Proof: By Lemma 11.5 and Proposition 11.6(a), the U PDATE procedure runs in linear time. Both 2 and 6 take O(n 2 ) time, once per augmentation. Each of , 5 , 7 and 9 can be done in O(nk) time, where k is the number of new outer vertices. (In , 7 the number of maximal proper subsets A0 of A to be considered is at most 2k + 1: every second sub-blossom of a new blossom must have been inner.) Since an outer vertex continues to be outer until the next augmentation, the total time spent by , 5 , 7 and 9 between two augmentations is O(n 2 ). It remains to estimate the running time of , 8 , 3 and . 4 Suppose in 8 we have ε 6= ε1 . Due to the variables tv and tvA we then obtain a new tight edge in . 8 We continue in 3 and , 4 where after at most O(n) time this edge is checked. Since it either connects an outer vertex with an out-of-forest vertex or two different outer connected components, we can apply one of , 5 , 6 . 7 If ε = ε1 we have to apply . 9 This consideration shows that the number of times 8 is executed is less than or equal to the number of times one of , 5 , 6 , 7 9 is executed. Since 8 takes only O(n) time, the O(n 2 ) bound between two augmentations is proved. Note that the case ε = 0 is not excluded. Since there are only n2 augmentations, the total running time is O(n 3 ). Corollary 11.10. The M INIMUM W EIGHT P ERFECT M ATCHING P ROBLEM can be solved in O(n 3 ) time. Proof:

This follows from Theorems 11.8 and 11.9.

The first O(n 3 )-implementation of Edmonds’ algorithm for the M INIMUM W EIGHT P ERFECT M ATCHING P ROBLEM was due to Gabow [1973] (see also Gabow [1976] and Lawler [1976]). The theoretically best running time, namely O(mn + n 2 log n), has also been obtained by Gabow [1990]. 3 For planar graphs a minimum weight perfect matching can be found in O n 2 log n time, as Lipton and Tarjan [1979,1980] showed by a divide and conquer approach, using the fact that planar graphs have small “separators”. For Euclidean instances (a set of points in the plane defining a complete graph whose edge weights are given by the 3 Euclidean distances) Varadarajan [1998] found an O n 2 log5 n algorithm. Probably the currently most efficient implementations are described by Mehlhorn and Sch¨afer [2000] and Cook and Rohe [1999]. They solve matching problems with millions of vertices optimally. A “primal” weighted matching algorithm – always maintaining a perfect matching and obtaining a feasible dual solution only at termination – was described by Cunningham and Marsh [1978].

276

11 Weighted Matching

11.4 Postoptimality In this section we prove two postoptimality results which we shall need in Section 12.2. Lemma 11.11. (Weber [1981], Ball and Derigs [1983]) Suppose we have run the W EIGHTED M ATCHING A LGORITHM for an instance (G, c). Let s ∈ V (G), and let c0 : E(G) → R with c0 (e) = c(e) for all e 6∈ δ(s). Then a minimum weight perfect matching with respect to (G, c0 ) can be determined in O(n 2 ) time. Proof: Let t := µ(s). If s is not contained in any nontrivial blossom, i.e. ks = 0, then the first step just consists of setting µ(s) := s and µ(t) := t. Otherwise we have to unpack all the blossoms P containing s. To accomplish this, we shall perform dual changes of total value A: s∈A, |A|>1 z A while s is inner all the time. Consider the following construction: .

E(G) ∪ {{a, s}, {b, t}}. Set V (G) := V (G) ∪ {a, b} and E(G) :=X z A + ζ{t} . Set c({a, s}) := ζ{s} and c({b, t}) := 2 A: s∈A, |A|>1

Set µ(a) := a and µ(b) := b. Mark a and b as outer. Set B := B ∪ {{a}, {b}}, z {a} := 0, z {b} := 0, ζ{a} := 0, ζ{b} := 0. Set ka := 0, kb := 0, ρ 0 (a) := a, ρ 0 (b) := b, ϕ 0 (a) := a, ϕ 0 (b) := b. U PDATE(a). U PDATE(b). The result is a possible status if the algorithm was applied to the modified instance (the graph extended by two vertices and two edges). In particular, the dual solution z is feasible. Moreover, the edge {a, s} is tight. Now we set scanned(a) := false and continue the algorithm starting with . 3 The algorithm will grow the forest by the edge {a, s} next, and s becomes inner. By Theorem 11.9 the algorithm terminates after O(n 2 ) steps with an augmentation. The only possible augmenting path is a, Ps, t, b. So the edge {b, t} must become tight. At the beginning, slack(b, t) = 2 A∈A, s∈A, |A|>1 z A . Vertex s will remain inner throughout. So ζ{s} will decrease at each dual change. Thus all blossoms A containing s are unpacked at the end. We finally delete the vertices a and b and the edges {a, s} and {b, t}, and set B := B \ {{a}, {b}} and µ(s) := s, µ(t) := t. Now s and t are outer, and there are no inner vertices. Furthermore, no edge incident to s belongs to the general blossom forest. So we can easily change weights of edges incident to s as well as z {s} , as long as we maintain the dual feasibility. This, however, is easily guaranteed by first computing the slacks according to the new edge weights and then increasing z {s} by mine∈δ(s) slack(e). We set scanned(s) := false and continue the algorithm starting with . 3 By Theorem 11.9, the algorithm will terminate after O(n 2 ) steps with a minimum weight perfect matching with respect to the new weights. The same result for a “primal” weighted matching algorithm was found by Cunningham and Marsh [1978]. The following lemma deals with the addition of two vertices to an instance that has already been solved.

11.5 The Matching Polytope

277

Lemma 11.12. Let (G, c) be an instance of the M INIMUM W EIGHT P ERFECT M ATCHING P ROBLEM, and let s, t ∈ V (G). Suppose we have run the W EIGHTED M ATCHING A LGORITHM for the instance (G − {s, t}, c). Then a minimum weight perfect matching with respect to (G, c) can be determined in O(n 2 ) time. Proof: The addition of two vertices requires the initialization of the data structures (as in the previous proof). The dual variable z v is set such that mine∈δ(v) slack(e) = 0 (for v ∈ {s, t}). Then setting scanned(s) := scanned(t) := false and starting the W EIGHTED M ATCHING A LGORITHM with 3 does the job.

11.5 The Matching Polytope The correctness of the W EIGHTED M ATCHING A LGORITHM also yields Edmonds’ characterization of the perfect matching polytope as a by-product. We again use the notation A := {A ⊆ V (G) : |A| odd}. Theorem 11.13. (Edmonds [1965]) Let G be an undirected graph. The perfect matching polytope of G, i.e. the convex hull of the incidence vectors of all perfect matchings in G, is the set of vectors x satisfying

X

xe ≥ 0

(e ∈ E(G))

xe = 1

(v ∈ V (G))

xe ≥ 1

(A ∈ A)

e∈δ(v)

X e∈δ(A)

Proof: By Corollary 3.32 it suffices to show that all vertices of the polytope described above are integral. By Theorem 5.13 this is true if the minimization problem has an integral optimum solution for any weight function. But our W EIGHTED M ATCHING A LGORITHM finds such a solution for any weight function (cf. the proof of Theorem 11.8). An alternative proof will be given in Section 12.3 (see the remark after Theorem 12.16). We can also describe the matching polytope, i.e. the convex hull of the incidence vectors of all matchings in an undirected graph G: Theorem 11.14. (Edmonds [1965]) Let G be a graph. The matching polytope E(G) satisfying of G is the set of vectors x ∈ R+ X e∈δ(v)

xe ≤ 1 for all v ∈ V (G)

and

X e∈E(G[A])

xe ≤

|A| − 1 for all A ∈ A. 2

278

11 Weighted Matching

Proof: Since the incidence vector of any matching obviously satisﬁes these inE(G) be a vector with equalities, we only have to prove one direction. Let x ∈ R+ |A|−1 x ≤ 1 for v ∈ V (G) and x ≤ for A ∈ A. We prove e∈δ(v) e e∈E(G[A]) e 2 that x is a convex combination of incidence vectors of matchings. Let H be the graph with V (H ) := {(v, i) : v ∈ V (G), i ∈ {1, 2}}, and E(H ) := {{(v, i), (w, i)} : {v, w} ∈ E(G), i ∈ {1, 2}} ∪ {{(v, 1), (v, 2)} : v ∈ V (G)}. So H consists of two copies of G, and there is an edge joining the two copies of each vertex. Let y{(v,i),(w,i)}:= xe for each e = {v, w} ∈ E(G) and i ∈ {1, 2}, and let y{(v,1),(v,2)} := 1 − e∈δG (v) xe for each v ∈ V (G). We claim that y belongs to the perfect matching polytope of H . Considering the subgraph induced by {(v, 1) : v ∈ V (G)}, which is isomorphic to G, we then get that x is a convex combination of incidence vectors of matchings in G. V (G)

{(v, 1) : v ∈ V (G)}

{(v, 2) : v ∈ V (G)}

A

A1

A2

B

B1

B2

C

:X Fig. 11.9.

E(H ) Obviously, y ∈ R+ and e∈δ H (v) ye = 1 for all v ∈ V (H ). To show that y belongs to the perfect matching polytope of H , we use Theorem 11.13. So let X ⊆ V (H ) with |X | odd. We prove that e∈δ H (X ) ye ≥ 1. Let A := {v ∈ V (G) : (v, 1) ∈ X, (v, 2) ∈ / X }, B := {v ∈ V (G) : (v, 1) ∈ X, (v, 2) ∈ X } and C := {v ∈ V (G) : (v, 1) ∈ / X, (v, 2) ∈ X }. Since |X | is odd, either A or C must have odd cardinality, w.l.o.g. |A| is odd. We write Ai := {(a, i) : a ∈ A} and Bi := {(b, i) : b ∈ B} for i = 1, 2 (see Figure 11.9). Then ye ≥ ye − 2 ye − ye + ye v∈A1 e∈δ H (v)

e∈δ H (X )

=

v∈A1 e∈δ H (v)

e∈E(H [A1 ])

ye − 2

e∈E H (A1 ,B1 )

e∈E H (B2 ,A2 )

xe

e∈E(G[A])

≥ |A1 | − (|A| − 1) = 1.

11.5 The Matching Polytope

279

Indeed, we can prove the following stronger result: Theorem 11.15. (Cunningham and Marsh [1978]) For any undirected graph G the linear inequality system

X e∈δ(v) X

(e ∈ E(G)) (v ∈ V (G))

xe ≥ 0 xe ≤ 1 |A|−1 2

xe ≤

(A ∈ A, |A| > 1)

e⊆A

is TDI. Proof: For c : E(G) → Z we consider the LP max the above constraints. The dual LP is: min

X v∈V (G)

s.t.

X v∈e

X

yv +

yv +

A∈A, |A|>1

X

P

e∈E(G) c(e)x e

subject to

|A| − 1 zA 2

z A ≥ c(e)

(e ∈ E(G))

yv ≥ 0 zA ≥ 0

(v ∈ V (G)) (A ∈ A, |A| > 1)

A∈A, e⊆A

Let (G, c) be the smallest counterexample, i.e. there is no integral optimum P dual solution and |V (G)| + |E(G)| + e∈E(G) |c(e)| is minimum. Then c(e) ≥ 1 for all e (otherwise we can delete any edge of nonpositive weight), and G has no isolated vertices (otherwise we can delete them). Moreover, for any optimum solution y, z we claim that y = 0. To prove this, suppose P yv > 0 for some v ∈ V (G). Then by complementary slackness (Corollary 3.23) e∈δ(v) xe = 1 for every primal optimum solution x. But then decreasing c(e) by one for each e ∈ δ(v) yields a smaller instance (G, c0 ), whose optimum LP value is one less (here we use primal integrality, i.e. Theorem 11.14). Since (G, c) is the smallest counterexample, there exists an integral optimum dual solution y 0 , z 0 for (G, c0 ). Increasing yv0 by one yields an integral optimum dual solution for (G, c), a contradiction. Now let y = 0 and z be an optimum dual solution for which X |A|2 z A (11.4) A∈A, |A|>1

is as large as possible. We claim that F := {A : z A > 0} is laminar. To see this, suppose there are sets X, Y ∈ F with X \ Y 6= ∅, Y \ X 6= ∅ and X ∩ Y 6= ∅. Let := min{z X , z Y } > 0. If |X ∩ Y | is odd, then |X ∪ Y | is also odd. Set z 0X := z X − , z Y0 := z Y − , 0 z X ∩Y := z X ∩Y + (unless |X ∩ Y | = 1), z 0X ∪Y := z X ∪Y + and z 0A := z A for all

280

11 Weighted Matching

other sets A. y, z 0 is also a feasible dual solution; moreover it is optimum as well. This is a contradiction since (11.4) is larger. If |X ∩ Y | is even, then |X \ Y | and |Y \ X | are odd. Set z 0X := z X − , 0 z Y := z Y − , z 0X \Y := z X \Y + (unless |X \ Y | = 1), z Y0 \X := z Y \X + (unless |Y \ X | = 1) and z 0A := z A for all other sets A. Set yv0 := yv + for v ∈ X ∩ Y and / X ∩ Y . Then y 0 , z 0 is a feasible dual solution that is also optimum. yv0 := yv for v ∈ This contradicts the fact that any optimum dual solution must have y = 0. / Z and A maximal. Set := z A − bz A c > 0. Now let A ∈ F with z A ∈ Let A1 , . . . , Ak be the maximal proper subsets of A in F; they must be disjoint because F is laminar. Setting z 0A := z A − and z 0Ai := z Ai + for i = 1, . . . , k (and z 0D := z D for all other D ∈ A) yields another feasible dual solution y = 0, z 0 (since c is integral). We have X X |B| − 1 0 |B| − 1 zB < zB, 2 2 B∈A, |B|>1

B∈A, |B|>1

contradicting the optimality of the original dual solution y = 0, z.

This proof is due to Schrijver [1983a]. For different proofs, see Lov´asz [1979] and PThe latter does not use Theorem 11.14. Moreover, replacing P Schrijver [1983b]. x ≤ 1 by e e∈δ(v) e∈δ(v) x e = 1 for v ∈ V (G) in Theorem 11.15 yields an alternative description of the perfect matching polytope, which is also TDI (by Theorem 5.18). Theorem 11.13 can easily be derived from this; however, the linear inequality system of Theorem 11.13 is not TDI in general (K 4 is a counterexample). Theorem 11.15 also implies the Berge-Tutte formula (Theorem 10.14; see Exercise 14). Generalizations will be discussed in Section 12.1.

Exercises 1. Use Theorem 11.2 to prove a weighted version of K¨onig’s Theorem 10.2. (Egerv´ary [1931]) 2. Describe the convex hull of the incidence vectors of all (a) vertex covers, (b) stable sets, (c) edge covers, in a bipartite graph G. Show how Theorem 10.2 and the statement of Exercise 2(c) of Chapter 10 follow. Hint: Use Theorem 5.25 and Corollary 5.21. 3. Prove the Birkhoff-von-Neumann Theorem 11.3 directly. 4. Let G be a graph and P the fractional perfect matching polytope of G. Prove that the vertices of P are exactly the vectors x with 1 2 if e ∈ E(C1 ) ∪ · · · ∪ E(Ck ) , xe = 1 if e ∈ M 0 otherwise

Exercises

5.

6. 7.

8.

9.

10. 11.

12.

13.

∗ 14.

281

where C1 , . . . , Ck are vertex-disjoint odd circuits and M is a perfect matching in G − (V (C1 ) ∪ · · · ∪ V (Ck )). (Balinski [1972]; see Lov´asz [1979]) . Let G be a bipartite graph with bipartition V = A ∪ B and A = {a1 , . . . , a p }, B = {b1 , . . . , bq }. Let c : E(G) → R be weights on the edges. We look for the maximum weight order-preserving matching M, i.e. for any two edges {ai , b j }, {ai 0 , b j 0 } ∈ M with i < i 0 we require j < j 0 . Solve this problem with an O(n 3 )-algorithm. Hint: Use dynamic programming. Prove that, at any stage of the W EIGHTED M ATCHING A LGORITHM, |B| ≤ 32 n. Let G be a graph with nonnegative weights c : E(G) → R+ . Let M be the matching at any intermediate stage of the W EIGHTED M ATCHING A LGO RITHM . Let X be the set of vertices covered by M. Show that any matching covering X is at least as expensive as M. (Ball and Derigs [1983]) A graph with integral weights on the edges is said to have the even circuit property if the total weight of every circuit is even. Show that the W EIGHTED M ATCHING A LGORITHM applied to a graph with the even circuit property maintains this property (with respect to the slacks) and also maintains a dual solution that is integral. Conclude that for any graph there exists an optimum dual solution z that is half-integral (i.e. 2z is integral). When the W EIGHTED M ATCHING A LGORITHM is restricted to bipartite graphs, it becomes much simpler. Show which parts are necessary even in the bipartite case and which are not. Note: One arrives at what is called the Hungarian method for the A SSIGN MENT P ROBLEM (Kuhn [1955]). This algorithm can also be regarded as an equivalent description of the procedure proposed in the proof of Theorem 11.1. How can the bottleneck matching problem (find a perfect matching M such that max{c(e) : e ∈ M} is minimum) be solved in O(n 3 ) time? Show how to solve the M INIMUM W EIGHT E DGE C OVER P ROBLEM in polynomial time: given an undirected graph G and weights c : E(G) → R, find a minimum weight edge cover. Given an undirected graph G with weights c : E(G) → R+ and two vertices s and t, we look for a shortest s-t-path with an even (or with an odd) number of edges. Reduce this to a M INIMUM W EIGHT P ERFECT M ATCHING P ROBLEM. Hint: Take two copies of G, connect each vertex with its copy by an edge of zero weight and delete s and t (or s and the copy of t). (Gr¨otschel and Pulleyblank [1981]) Let G be a k-regular and (k − 1)-edge-connected graph with an even number of vertices, and let c : E(G) → R+ . Prove that there exists a perfect matching M in G with c(M) ≥ k1 c(E(G)). Hint: Show that k1 1l is in the perfect matching polytope. Show that Theorem 11.15 implies: (a) the Berge-Tutte formula (Theorem 10.14);

282

11 Weighted Matching

(b) Theorem 11.13; (c) the existence of an optimum half-integral dual solution to the dual LP (11.2) (cf. Exercise 8). Hint: Use Theorem 5.18. 15. The fractional perfect matching polytope Q of G is identical to the perfect matching polytope if G is bipartite (Theorem 11.2). Consider the first GomoryChv´atal-truncation Q 0 of Q (Definition 5.29). Prove that Q 0 is always identical to the perfect matching polytope.

References General Literature: Gerards, A.M.H. [1995]: Matching. In: Handbooks in Operations Research and Management Science; Volume 7: Network Models (M.O. Ball, T.L. Magnanti, C.L. Monma, G.L. Nemhauser, eds.), Elsevier, Amsterdam 1995, pp. 135–224 Lawler, E.L. [1976]: Combinatorial Optimization; Networks and Matroids. Holt, Rinehart and Winston, New York 1976, Chapters 5 and 6 Papadimitriou, C.H., and Steiglitz, K. [1982]: Combinatorial Optimization; Algorithms and Complexity. Prentice-Hall, Englewood Cliffs 1982, Chapter 11 Pulleyblank, W.R. [1995]: Matchings and extensions. In: Handbook of Combinatorics; Vol. 1 (R.L. Graham, M. Gr¨otschel, L. Lov´asz, eds.), Elsevier, Amsterdam 1995

Cited References: Balinski, M.L. [1972]: Establishing the matching polytope. Journal of Combinatorial Theory 13 (1972), 1–13 Ball, M.O., and Derigs, U. [1983]: An analysis of alternative strategies for implementing matching algorithms. Networks 13 (1983), 517–549 Birkhoff, G. [1946]: Tres observaciones sobre el algebra lineal. Revista Universidad Nacional de Tucum´an, Series A 5 (1946), 147–151 Cook, W., and Rohe, A. [1999]: Computing minimum-weight perfect matchings. INFORMS Journal of Computing 11 (1999), 138–148 Cunningham, W.H., and Marsh, A.B. [1978]: A primal algorithm for optimum matching. Mathematical Programming Study 8 (1978), 50–72 Edmonds, J. [1965]: Maximum matching and a polyhedron with (0,1) vertices. Journal of Research of the National Bureau of Standards B 69 (1965), 125–130 Egerv´ary, E. [1931]: Matrixok kombinatorikus tulajdons´agairol. Matematikai e´ s Fizikai Lapok 38 (1931), 16–28 [in Hungarian] Gabow, H.N. [1973]: Implementation of algorithms for maximum matching on non-bipartite graphs. Ph.D. Thesis, Stanford University, Dept. of Computer Science, 1973 Gabow, H.N. [1976]: An efficient implementation of Edmonds’ algorithm for maximum matching on graphs. Journal of the ACM 23 (1976), 221–234 Gabow, H.N. [1990]: Data structures for weighted matching and nearest common ancestors with linking. Proceedings of the 1st Annual ACM-SIAM Symposium on Discrete Algorithms (1990), 434–443 Gr¨otschel, M., and Pulleyblank, W.R. [1981]: Weakly bipartite graphs and the max-cut problem. Operations Research Letters 1 (1981), 23–27

References

283

Kuhn, H.W. [1955]: The Hungarian method for the assignment problem. Naval Research Logistics Quarterly 2 (1955), 83–97 Lipton, R.J., and Tarjan, R.E. [1979]: A separator theorem for planar graphs. SIAM Journal on Applied Mathematics 36 (1979), 177–189 Lipton, R.J., and Tarjan, R.E. [1980]: Applications of a planar separator theorem. SIAM Journal on Computing 9 (1980), 615–627 Lov´asz, L. [1979]: Graph theory and integer programming. In: Discrete Optimization I; Annals of Discrete Mathematics 4 (P.L. Hammer, E.L. Johnson, B.H. Korte, eds.), North-Holland, Amsterdam 1979, pp. 141–158 Mehlhorn, K., and Sch¨afer, G. [2000]: Implementation of O(nm log n) weighted matchings in general graphs: the power of data structures. In: Algorithm Engineering; WAE-2000; LNCS 1982 (S. N¨aher, D. Wagner, eds.), pp. 23–38; also electronically in The ACM Journal of Experimental Algorithmics 7 (2002) Monge, G. [1784]: M´emoire sur la th´eorie des d´eblais et des remblais. Histoire de l’Acad´emie Royale des Sciences 2 (1784), 666–704 Munkres, J. [1957]: Algorithms for the assignment and transportation problems. Journal of the Society for Industrial and Applied Mathematics 5 (1957), 32–38 von Neumann, J. [1953]: A certain zero-sum two-person game equivalent to the optimal assignment problem. In: Contributions to the Theory of Games II; Ann. of Math. Stud. 28 (H.W. Kuhn, ed.), Princeton University Press, Princeton 1953, pp. 5–12 Schrijver, A. [1983a]: Short proofs on the matching polyhedron. Journal of Combinatorial Theory B 34 (1983), 104–108 Schrijver, A. [1983b]: Min-max results in combinatorial optimization. In: Mathematical Programming; The State of the Art – Bonn 1982 (A. Bachem, M. Gr¨otschel, B. Korte, eds.), Springer, Berlin 1983, pp. 439–500 Varadarajan, K.R. [1998]: A divide-and-conquer algorithm for min-cost perfect matching in the plane. Proceedings of the 39th Annual IEEE Symposium on Foundations of Computer Science (1998), 320–329 Weber, G.M. [1981]: Sensitivity analysis of optimal matchings. Networks 11 (1981), 41–56

12 b-Matchings and T-Joins

In this chapter we introduce two more combinatorial optimization problems, the M INIMUM W EIGHT b-M ATCHING P ROBLEM in Section 12.1 and the M INIMUM W EIGHT T -J OIN P ROBLEM in Section 12.2. Both can be regarded as generalizations of the M INIMUM W EIGHT P ERFECT M ATCHING P ROBLEM and also include other important problems. On the other hand, both problems can be reduced to the M INIMUM W EIGHT P ERFECT M ATCHING P ROBLEM. They have combinatorial polynomial-time algorithms as well as polyhedral descriptions. Since in both cases the S EPARATION P ROBLEM turns out to be solvable in polynomial time, we obtain another polynomial-time algorithm for these generalized matching problems (using the E LLIPSOID M ETHOD; see Section 4.6). In fact, the S EPARATION P ROBLEM can be reduced to finding a minimum capacity T -cut in both cases; see Sections 12.3 and 12.4. This problem, finding a minimum capacity cut δ(X ) such that |X ∩T | is odd for a specified vertex set T , can be solved with network flow techniques.

12.1 b-Matchings Definition 12.1. Let G be an undirected graph with integral edge capacities u : E(G) → N ∪ {∞} and numbers b : V (G) → N. Then a b-matching Pin (G, u) is a function f : E(G) → Z+ with f (e) ≤ u(e) for all e ∈ E(G) and e∈δ(v) f (e) ≤ b(v) for all v ∈ V (G). In the case u P ≡ 1 we speak of a simple b-matching in G. A b-matching f is called perfect if e∈δ(v) f (e) = b(v) for all v ∈ V (G). In the case b ≡ 1 the capacities are irrelevant, and we are back to ordinary matchings. A simple b-matching is sometimes also called a b-factor. It can be regarded as a subset of edges. In Chapter 21 we shall be interested in perfect simple 2-matchings, i.e. subsets of edges such that each vertex is incident to exactly two of them.

M AXIMUM W EIGHT b-M ATCHING P ROBLEM Instance: Task:

A graph G, capacities u : E(G) → N∪{∞}, weights c : E(G) → R, and numbers b : V (G) → N. P Find a b-matching f in (G, u) whose weight e∈E(G) c(e) f (e) is maximum.

286

12 b-Matchings and T -Joins

Edmonds’ W EIGHTED M ATCHING A LGORITHM can be extended to solve this problem (Marsh [1979]). We shall not describe this algorithm here, but shall rather give a polyhedral description and show that the S EPARATION P ROBLEM can be solved in polynomial time. This yields a polynomial-time algorithm via the E LLIPSOID M ETHOD (cf. Corollary 3.33). The b-matching polytope of (G, u) is defined to be the convex hull of the incidence vectors of all b-matchings in (G, u). We first consider the uncapacitated case (u ≡ ∞): Theorem 12.2. (Edmonds [1965]) Let G be an undirected graph and b : V (G) E(G) → N. The b-matching polytope of (G, ∞) is the set of vectors x ∈ R+ satisfying X xe ≤ b(v) (v ∈ V (G)); e∈δ(v)

%

$ X

xe ≤

e∈E(G[X ])

1 2

X

(X ⊆ V (G)).

b(v)

v∈X

Proof: Since any b-matching obviously satisfies these constraints, we only have P E(G) with e∈δ(v) xe ≤ b(v) for all v ∈ V (G) to show one direction. So let x ∈ R+ P P and e∈E(G[X ]) xe ≤ 12 v∈X b(v) for all X ⊆ V (G). We show that x is a convex combination of incidence vectors of b-matchings. We define a new graph H by splitting up each vertex v into S b(v) copies: we define X v := {(v, i) : i ∈ {1, . . . , b(v)}} for v ∈ V (G), V (H ) := v∈V (G) X v and 1 x{v,w} E(H ) := {{v 0 , w0 } : {v, w} ∈ E(G), v 0 ∈ X v , w 0 ∈ X w }. Let ye := b(v)b(w) 0 0 0 0 for each edge e = {v , w } ∈ E(H ), v ∈ X v , w ∈ X w . We claim that y is a convex combination of incidence vectors of matchings in H . By contracting the sets X v (v ∈ V (G)) in H we then return to G and x, and conclude that x is a convex combination of incidence vectors of b-matchings in G. P To prove that y is in the matching polytope of H we use Theorem 11.14. e∈δ(v) ye ≤ 1 obviously holds for each v ∈ V (H ). Let C ⊆ V (H ) with |C| odd. P We show that e∈E(H [C]) ye ≤ 21 (|C| − 1). If X v ⊆ C or X v ∩ C = ∅ for each v ∈ V (G), this follows directly from the inequalities assumed for x. Otherwise let a, b ∈ X v , a ∈ C, b 6∈ C. Then X X X X 2 ye = ye + ye c∈C\{a} e∈E({c},C\{c})

e∈E(H [C])

≤

X

X

X

X

ye −

c∈C\{a} e∈δ(c)

=

X

X

X

ye +

c∈C\{a} e∈δ(c)\{{c,b}}

=

e∈E({a},C\{a})

ye

e∈E({a},C\{a})

X e∈E({b},C\{a})

ye +

X

ye

e∈E({a},C\{a})

ye

c∈C\{a} e∈δ(c)

≤ |C| − 1.

12.1 b-Matchings

287

Note that this construction yields an algorithm which, however, P in general has an exponential running time. But we note that in the special case v∈V (G) b(v) = O(n) we can solve the uncapacitated M AXIMUM W EIGHT b-M ATCHING P ROB LEM in O(n 3 ) time (using the W EIGHTED M ATCHING A LGORITHM ; cf. Corollary 11.10). Pulleyblank [1973,1980] described the facets of this polytope and showed that the linear inequality system in Theorem 12.2 is TDI. The following generalization allows finite capacities: Theorem 12.3. (Edmonds and Johnson [1970]) Let G be an undirected graph, u : E(G) → N ∪ {∞} and b : V (G) → N. The b-matching polytope of (G, u) is E(G) satisfying the set of vectors x ∈ R+ X

(e ∈ E(G)); (v ∈ V (G));

xe ≤ u(e) xe ≤ b(v)

e∈δ(v)

$ X e∈E(G[X ])

xe +

X

xe ≤

e∈F

1 2

!% X

b(v) +

v∈X

X

u(e)

e∈F

(X ⊆ V (G), F ⊆ δ(X )).

Proof: First observe that the incidence vector of any b-matching f satisfies the constraints. This is clear except for the last one; here we argue as follows. Let X ⊆ V (G) and F ⊆ δ(X ). We have a budget of b(v) units at each vertex v ∈ X and a budget of u(e) units for each e ∈ F. Now for each e ∈ E(G[X ]) we take f (e) units from the budget at each vertex incident to e. For each e ∈ F, say e = {x, y} with x ∈ X , we take f (e) units from the budget at x and f (e) units from thePbudget at e. It is clear that the budgets are not exceeded, and we have taken 2 e∈E(G[X ])∪F f (e) units. So X e∈E(G[X ])

xe +

X e∈F

xe

1 ≤ 2

! X v∈X

b(v) +

X

u(e) .

e∈F

Since the left-hand side is an integer, so is the right-hand side; thus we may round down. P E(G) be a vector with xe ≤ u(e) for all e ∈ E(G), e∈δ(v) xe ≤ Now let x ∈ R+ b(v) for all v ∈ V (G) and $ !% X X X 1 X xe + xe ≤ b(v) + u(e) 2 e∈E(G[X ])

e∈F

v∈X

e∈F

for all X ⊆ V (G) and F ⊆ δ(X ). We show that x is a convex combination of incidence vectors of b-matchings in (G, u). Let H be the graph resulting from G by subdividing each edge e = {v, w} with u(e) 6= ∞ by means of two new vertices (e, v), (e, w). (Instead of e, H now contains the edges {v, (e, v)}, {(e, v), (e, w)} and {(e, w), w}.) Set b((e, v)) := b((e, w)) := u(e) for the new vertices.

288

12 b-Matchings and T -Joins

For each subdivided edge e = {v, w} set y{v,(e,v)} := y{(e,w),w} := xe and y{(e,v),(e,w)} := u(e) − xe . For each original edge e with u(e) = ∞ set ye := xe . We claim that y is in the b-matching polytope P of (H, ∞). E(H ) We use Theorem 12.2. Obviously y ∈ R+ and e∈δ(v) ye ≤ b(v) for all v ∈ V (H ). Suppose there is a set A ⊆ V (H ) with 1 ye > b(a) . (12.1) 2 a∈A

e∈E(H [A])

Let B := A ∩ V (G). For each e = {v, w} ∈ E(G[B]) we may assume (e, v), (e, w) ∈ A, for otherwise the addition of (e, v) and (e, w) does not destroy (12.1). On the other hand, we may assume that (e, v) ∈ A implies v ∈ A: If (e, v), (e, w) ∈ A but v ∈ / A, we can delete (e, v) and (e, w) from A without destroying (12.1). If (e, v) ∈ A but v, (e, w) ∈ / A, we can just delete (e, v) from A. Figure 12.1 shows the remaining possible edge types. A

Fig. 12.1.

Let F := {e = {v, w} ∈ E(G) : |A ∩ {(e, v), (e, w)}| = 1}. We have xe + xe = ye − u(e) e∈E(G[B])

e∈F

e∈E(H [A])

>

e∈E(G[B]), u(e) c(J 0 ∩ E(C)) = c(E(C) \ J ). This proposition can be regarded as a special case of Theorem 9.6. We now solve the M INIMUM W EIGHT T -J OIN P ROBLEM with nonnegative weights by reducing it to the M INIMUM W EIGHT P ERFECT M ATCHING P ROBLEM. The main idea is contained in the following lemma: Lemma 12.8. Let G be a graph, c : E(G) → R+ , and T ⊆ V (G) with |T | even. Every optimum T -join in G is the disjoint union of the edge sets of |T2 | paths whose ends are distinct and in T , and possibly some zero-weight circuits. Proof: By induction on |T |. The case T = ∅ is trivial since the minimum weight of an ∅-join is zero. Let J be any optimum T -join in G; w.l.o.g. J contains no zero-weight circuit. By Proposition 12.7 J contains no circuit of positive weight. As c is nonnegative, J thus forms a forest. Let x, y be two leaves of the same connected component, i.e. |J ∩ δ(x)| = |J ∩ δ(y)| = 1, and let P be the x-y-path in J . We have x, y ∈ T , and J \ E(P) is a minimum cost (T \ {x, y})-join (a cheaper (T \ {x, y})-join J 0 would imply a T -join J 0 4E(P) that is cheaper than J ). The assertion now follows from the induction hypothesis. Theorem 12.9. (Edmonds and Johnson [1973]) In the case of nonnegative weights, the M INIMUM W EIGHT T -J OIN P ROBLEM can be solved in O(n 3 ) time. Proof: Let (G, c, T ) be an instance. We first solve an A LL PAIRS S HORTEST PATHS P ROBLEM in (G, c); more precisely: in the graph resulting by replacing each edge by a pair of oppositely directed edges with the same weight. By Theorem 7.9 this takes O(mn + n 2 log n) time. In particular, we obtain the metric closure ¯ c) (G, ¯ of (G, c) (cf. Corollary 7.11). ¯ ], c). Now we find a minimum weight perfect matching M in (G[T ¯ By Corollary 3 ¯ is at most the minimum weight 11.10, this takes O(n ) time. By Lemma 12.8, c(M) of a T -join. We consider the shortest x-y-path in G for each {x, y} ∈ M (which we have already computed). Let J be the symmetric difference of the edge sets of all these paths. Evidently, J is a T -join in G. Moreover, c(J ) ≤ c(M), ¯ so J is optimum. This method no longer works if we allow negative weights, because we would introduce negative circuits. However, we can reduce the M INIMUM W EIGHT T J OIN P ROBLEM with arbitrary weights to that with nonnegative weights:

292

12 b-Matchings and T -Joins

Theorem 12.10. Let G be a graph with weights c : E(G) → R, and T ⊆ V (G) a vertex set of even cardinality. Let E − be the set of edges with negative weight, T − the set of vertices that are incident with an odd number of negative edges, and d : E(G) → R+ with d(e) := |c(e)|. Then J is a minimum c-weight T -join if and only if J 4E − is a minimum d-weight (T 4T − )-join. Proof: For any subset J of E(G) we have c(J ) = c(J \ E − ) + c(J ∩ E − ) = c(J \ E − ) + c(J ∩ E − ) + c(E − \ J ) + d(E − \ J ) = d(J \ E − ) + c(J ∩ E − ) + c(E − \ J ) + d(E − \ J ) = d(J 4E − ) + c(E − ) . Now J is a T -join if and only if J 4E − is a (T 4T − )-join, which together with the above equality proves the theorem (since c(E − ) is constant). Corollary 12.11. The M INIMUM W EIGHT T -J OIN P ROBLEM can be solved in O(n 3 ) time. Proof: This follows directly from Theorems 12.9 and 12.10.

In fact, using the fastest known implementation of the W EIGHTED M ATCHING A LGORITHM, a minimum weight T -join can be computed in O(nm + n 2 log n) time. We are finally able to solve the S HORTEST PATH P ROBLEM in undirected graphs: Corollary 12.12. The problem of finding a shortest path between two specified vertices in an undirected graph with conservative weights can be solved in O(n 3 ) time. Proof: Let s and t be the two specified vertices. Set T := {s, t} and apply Corollary 12.11. After deleting zero-weight circuits, the resulting T -join is a shortest s-t-path. Of course this also implies an O(mn 3 )-algorithm for finding a circuit of minimum total weight in an undirected graph with conservative weights (and in particular S HORTEST PATHS to compute the girth). If we are interested in the A LL PAIRS P ROBLEM in undirected graphs, we do not have to do n2 independent weighted matching computations (which would give a running time of O(n 5 )). Using the postoptimality results of Section 11.4 we can prove: Theorem 12.13. The problem of finding shortest paths for all pairs of vertices in an undirected graph G with conservative weights c : E(G) → R can be solved in O(n 4 ) time.

12.3 T -Joins and T -Cuts

293

Proof: By Theorem 12.10 and the proof of Corollary 12.12 we have to compute an optimum {s, t}4T − -join with respect to the weights d(e) := |c(e)| for all s, t ∈ V (G), where T − is the set of vertices incident to an odd number of negative ¯ edges. Let d({x, y}) := dist(G,d) (x, y) for x, y ∈ V (G), and let H X be the complete graph on X 4T − (X ⊆ V (G)). By the proof of Theorem 12.9 it is sufficient to compute a minimum weight perfect matching in H{s,t} , d¯ for all s and t. Our O(n 4 )-algorithm proceeds as follows. We first compute d¯ (cf. Corollary ¯ 7.11) and run the W EIGHTED M ATCHING A LGORITHM for the instance (H∅ , d). 3 Up to now we have spent O(n ) time. We show that we can now compute a minimum weight perfect matching of H{s,t} , d¯ in O(n 2 ) time, for any s and t. P ¯ Let K := e∈E(G) d(e), and let s, t ∈ V (G). There are four cases: − Case 1: s, t ∈ T . Then all we have to do is reduce the cost of the edge {s, t} to −K . After reoptimizing (using Lemma 11.11), {s, t} must belong to the optimum matching M, and M \ {{s, t}} is a minimum weight perfect matching of H{s,t} , d¯ . ¯ Case 2: s ∈ T − and t 6∈ T − . Then the cost of the edge {s, v} is set to d({t, v}) for all v ∈ T − \ {s}. Now s plays the role of t, and reoptimizing (using Lemma 11.11) does the job. Case 3: s 6∈ T − and t ∈ T − . Symmetric to Case 2. Case 4: s, t 6∈ T − . Then we add these two vertices and apply Lemma 11.12.

12.3 T-Joins and T-Cuts In this section we shall derive a polyhedral description of the M INIMUM W EIGHT T -J OIN P ROBLEM. In contrast to the description of the perfect matching polytope (Theorem 11.13), where we had a constraint for each cut δ(X ) with |X | odd, we now need a constraint for each T -cut. A T-cut is a cut δ(X ) with |X ∩ T | odd. The following simple observation is very useful: Proposition 12.14. Let G be an undirected graph and T ⊆ V (G) with |T | even. Then for any T -join J and any T -cut C we have J ∩ C 6= ∅. Proof: Suppose C = δ(X ), then |X ∩ T | is odd. So the number of edges in J ∩ C must be odd, in particular nonzero. A stronger statement can be found in Exercise 11. Proposition 12.14 implies that the minimum cardinality of a T -join is not less than the maximum number of edge-disjoint T -cuts. In general, we do not have equality: consider G = K 4 and T = V (G). However, for bipartite graphs equality holds: Theorem 12.15. (Seymour [1981]) Let G be a connected bipartite graph and T ⊆ V (G) with |T | even. Then the minimum cardinality of a T -join equals the maximum number of edge-disjoint T -cuts.

294

12 b-Matchings and T -Joins

Proof: (Seb˝o [1987]) We only have to prove “≤”. We use induction on |V (G)|. If T = ∅ (in particular if |V (G)| = 1), the statement is trivial. So we assume |V (G)| ≥ |T | ≥ 2. Denote by τ (G, T ) the minimum cardinality of a T -join in G. Choose a, b ∈ V (G), a 6= b, such that τ (G, T 4{a, b}) is minimum. Let T 0 := T 4{a, b}. Since we may assume T 6= ∅, τ (G, T 0 ) < τ (G, T ). Claim: For any minimum T -join J in G we have |J ∩ δ(a)| = |J ∩ δ(b)| = 1. To prove this claim, let J 0 be a minimum T 0 -join. J 4J 0 is the edge-disjoint union of an a-b-path P and some circuits C1 , . . . , Ck . We have |Ci ∩ J | = |Ci ∩ J 0 | for each i, because both J and J 0 are minimum. So |J 4P| = |J 0 |, and J 00 := J 4P is also a minimum T 0 -join. Now J 00 ∩ δ(a) = J 00 ∩ δ(b) = ∅, because if, say, {b, b0 } ∈ J 00 , J 00 \ {{b, b0 }} is a (T 4{a}4{b0 })-join, and we have τ (G, T 4{a}4{b0 }) < |J 00 | = |J 0 | = τ (G, T 0 ), contradicting the choice of a and b. We conclude that |J ∩ δ(a)| = |J ∩ δ(b)| = 1, and the claim is proved. In particular, a, b ∈ T . Now let J be a minimum T -join in G. Contract B := {b} ∪ 0(b) to a single vertex v B , and let the resulting graph be G ∗ . G ∗ is also bipartite. Let T ∗ := T \ B if |T ∩ B| is even and T ∗ := (T \ B) ∪ {v B } otherwise. The set J ∗ , resulting from J by the contraction of B, is obviously a T ∗ -join in G ∗ . Since 0(b) is a stable set in G (as G is bipartite), the claim implies that |J | = |J ∗ | + 1. It suffices to prove that J ∗ is a minimum T ∗ -join in G ∗ , because then we have τ (G, T ) = |J | = |J ∗ | + 1 = τ (G ∗ , T ∗ ) + 1, and the theorem follows by induction (observe that δ(b) is a T -cut in G disjoint from E(G ∗ )). So suppose that J ∗ is not a minimum T ∗ -join in G ∗ . Then by Proposition 12.7 there is a circuit C ∗ in G ∗ with |J ∗ ∩ E(C ∗ )| > |E(C ∗ ) \ J ∗ |. Since G ∗ is bipartite, |J ∗ ∩ E(C ∗ )| ≥ |E(C ∗ ) \ J ∗ | + 2. E(C ∗ ) corresponds to an edge set Q in G. Q cannot be a circuit, because |J ∩ Q| > |Q \ J | and J is a minimum T -join. Hence Q is an x-y-path in G for some x, y ∈ 0(b) with x 6= y. Let C be the circuit in G formed by Q together with {x, b} and {b, y}. Since J is a minimum T -join in G, |J ∩ E(C)| ≤ |E(C) \ J | ≤ |E(C ∗ ) \ J ∗ | + 2 ≤ |J ∗ ∩ E(C ∗ )| ≤ |J ∩ E(C)|. Thus we must have equality throughout, in particular {x, b}, {b, y} ∈ / J and |J ∩ E(C)| = |E(C)\ J |. So J¯ := J 4E(C) is also a minimum T -join and | J¯∩δ(b)| = 3. But this is impossible by the claim. T -cuts are also essential in the following description of the T -join polyhedron: Theorem 12.16. (Edmonds and Johnson [1973]) Let G be an undirected graph, c : E(G) → R+ , and T ⊆ V (G) with |T | even. Then the incidence vector of a minimum weight T -join is an optimum solution of the LP ( ) X min cx : x ≥ 0, xe ≥ 1 for all T -cuts C . e∈C

(This polyhedron is called the T-join polyhedron of G.)

12.3 T -Joins and T -Cuts

295

Proof: By Proposition 12.14, the incidence vector of a T -join satisfies the constraints. Let c : E(G) → R+ be given; we may assume that c(e) is an even integer for each e ∈ E(G). Let k be the minimum weight (with respect to c) of a T -join in G. We show that the optimum value of the above LP is k. We replace each edge e by a path of length c(e) (if c(e) = 0 we contract e and add the contracted vertex to T iff |e ∩ T | = 1). The resulting graph G 0 is bipartite. Moreover, the minimum cardinality of a T -join in G 0 is k. By Theorem 12.15, there is a family C 0 of k edge-disjoint T -cuts in G 0 . Back in G, this yields a family C of k T -cuts in G such that every edge e is contained in at most c(e) of these. So for any feasible solution x of the above LP we have XX X cx ≥ xe ≥ 1 = k, C∈C e∈C

C∈C

proving that the optimum value is k.

This implies Theorem 11.13: let G be a graph with a perfect matching and T := V (G). Then Theorem 12.16 implies that ) ( X xe ≥ 1 for all T -cuts C min cx : x ≥ 0, e∈C

is an integer for each c ∈ Z E(G) for which the minimum is finite. By Theorem 5.13, the polyhedron is integral, and so is its face X X E(G) x ∈ R+ : xe ≥ 1 for all T -cuts C, xe = 1 for all v ∈ V (G) . e∈C

e∈δ(v)

One can also derive a description of the convex hull of the incidence vectors of all T -joins (Exercise 14). Theorems 12.16 and 4.21 (along with Corollary 3.33) imply another polynomial-time algorithm for the M INIMUM W EIGHT T -J OIN P ROBLEM if we can solve the S EPARATION P ROBLEM for the above description. This is obviously equivalent to checking whether there exists a T -cut with capacity less than one (here x serves as capacity vector). So it suffices to solve the following problem:

M INIMUM C APACITY T -C UT P ROBLEM Instance: Task:

A graph G, capacities u : E(G) → R+ , and a set T ⊆ V (G) of even cardinality. Find a minimum capacity T -cut in G.

Note that the M INIMUM C APACITY T -C UT P ROBLEM also solves the S EPARA P ROBLEM for the perfect matching polytope (Theorem 11.13; T := V (G)). The following theorem solves the M INIMUM C APACITY T -C UT P ROBLEM: it suffices to consider the fundamental cuts of a Gomory-Hu tree. Recall that we TION

296

12 b-Matchings and T -Joins

can find a Gomory-Hu tree for an undirected graph with capacities in O(n 4 ) time (Theorem 8.35). Theorem 12.17. (Padberg and Rao [1982]) Let G be an undirected graph with capacities u : E(G) → R+ . Let H be a Gomory-Hu tree for (G, u). Let T ⊆ V (G) with |T | even. Then there is a minimum capacity T -cut among the fundamental cuts of H . Hence the minimum capacity T -cut can be found in O(n 4 ) time. Proof: We consider the pair (G + H, u 0 ) with u 0 (e) = u(e) for e ∈ E(G) and u 0 (e) = 0 for e ∈ E(H ). Let A ⊆ E(G)∪E(H ) be a minimum T -cut in (G+H, u 0 ). Obviously u 0 (A) = u(A ∩ E(G)) and A ∩ E(G) is a minimum T -cut in (G, u). Let now J be the set of edges e of H for which δG (Ce ) is a T -cut. It is easy to see that J is a T -join (in G + H ). By Proposition 12.14, there exists an edge e = {v, w} ∈ A ∩ J . We have X u(A ∩ E(G)) ≥ λvw = u({x, y}), {x,y}∈δG (Ce )

showing that δG (Ce ) is a minimum T -cut.

12.4 The Padberg-Rao Theorem The solution of the M INIMUM C APACITY T -C UT P ROBLEM also helps us to solve the S EPARATION P ROBLEM for the b-matching polytope (Theorem 12.3): Theorem 12.18. (Padberg and Rao [1982]) For undirected graphs G, u : E(G) → N ∪ {∞} and b : V (G) → N, the S EPARATION P ROBLEM for the b-matching polytope of (G, u) can be solved in polynomial time. Proof: We may assume u(e) < ∞ for all edges e (we may replace infinite capacities by a large enough number, e.g. max{b(v) : v ∈ V (G)}). We choose an arbitrary but fixed orientation of G; we will sometimes use the resulting directed edges and sometimes the original undirected edges. P E(G) with xe ≤ u(e) for all e ∈ E(G) and e∈δG (v) xe ≤ Given a vector x ∈ R+ b(v) for all v ∈ V (G) (these trivial inequalities can be checked in linear time), we define a new bipartite graph H with edge capacities t : E(H ) → R+ as follows: .

.

V (H ) := V (G) ∪ E(G) ∪ {S}, E(H ) := {{v, e} : v ∈ e ∈ E(G)} ∪ {{v, S} : v ∈ V (G)}, (e ∈ E(G), where v is the tail of e), t ({v, e}) := u(e) − xe (e ∈ E(G), where v is the head of e), t ({v, e}) := xe X t ({v, S}) := b(v) − xe (v ∈ V (G)). e∈δG (v)

Define T ⊆ V (H ) to consist of

12.4 The Padberg-Rao Theorem

297

• the vertices v ∈ V (G) for which b(v) + e∈δ + (v) u(e) is odd, G • the vertices e ∈ E(G) for which u(e) is odd, and • the vertex S if v∈V (G) b(v) is odd. Observe that |T | is even. We shall prove that there exists a T -cut in H with capacity less than one if and only if x is not in the convex hull of the b-matchings in (G, u).

E3

E1

∈ / F

∈F

X

E4

E2 Fig. 12.2.

We need some preparation. Let X ⊆ V (G) and F ⊆ δG (X ). Deﬁne + (X ) ∩ F}, E 1 := {e ∈ δG

− E 2 := {e ∈ δG (X ) ∩ F}, + E 3 := {e ∈ δG (X ) \ F}, − E 4 := {e ∈ δG (X ) \ F},

(see Figure 12.2) and W := X ∪ E(G[X ]) ∪ E 2 ∪ E 3 ⊆ V (H ). Claim:

(a) |W v∈X b(v) + e∈F u(e) is odd. ∩ T | is odd if and only if (b) e∈δ H (W ) t (e) < 1 if and only if e∈E(G[X ])

xe +

xe

e∈F

1 > 2

v∈X

b(v) +

u(e) − 1 .

e∈F

To prove (a), observe that by deﬁnition |W ∩ T | is odd if and only if ⎛ ⎞ ⎜ ⎟ u(e)⎠ + u(e) ⎝b(v) + v∈X

+ e∈δG (v)

e∈E(G[X ])∪E 2 ∪E 3

298

12 b-Matchings and T -Joins

is odd. But this number is equal to X X X b(v) + 2 u(e) + u(e) + v∈X

=

X

+ e∈δG (X )

e∈E(G[X ])

X

b(v) + 2

v∈X

u(e) − 2

+ e∈δG (X )

e∈E(G[X ])

u(e)

e∈E 2 ∪E 3

X

u(e) + 2

X

X

X

u(e) +

e∈E 1

u(e),

e∈E 1 ∪E 2

proving (a), because E 1 ∪ E 2 = F. Moreover, X X X X t (e) = t ({x, e}) + t ({y, e}) + t ({x, S}) e∈δ H (W )

e∈E 1 ∪E 4 x∈e∩X

e∈E 2 ∪E 3 y∈e\X

x∈X

X

=

(u(e) − xe ) +

e∈E 1 ∪E 2

=

X

X

xe +

u(e) +

b(v) − 2

v∈X

e∈F

X

X

b(v) −

X

xe − 2

e∈F

v∈X

e∈E 3 ∪E 4

X

X

xe

e∈δG (v)

xe ,

e∈E(G[X ])

proving (b). Now we can prove that there exists a T -cut in H with capacity less than one if and only if x is not in the convex hull of the b-matchings in (G, u). First suppose that there are X ⊆ V (G) and F ⊆ δG (X ) with $ !% X X X 1 X xe + xe > b(v) + u(e) . 2 Then

v∈X

b(v) +

X e∈E(G[X ])

v∈X

e∈F

e∈E(G[X ])

P

P

e∈F

xe +

e∈F

u(e) must be odd and

X e∈F

xe

1 > 2

! X v∈X

b(v) +

X

u(e) − 1 .

e∈F

By (a) and (b), this implies that δ H (W ) is a T -cut with capacity less than one. To prove the converse, let δ H (W ) now be any T -cut in H with capacity less than one. We show how to construct a violated inequality of the b-matching polytope. W.l.o.g. assume S 6∈ W (otherwise exchange W and V (H ) \ W ). Define X := W ∩ V (G). Observe that {v, {v, w}} ∈ δ H (W ) implies {v, w} ∈ δG (X ): If {v, w} ∈ / W for some v, w ∈ X , the two edges {v, {v, w}} and {w, {v, w}} (with total capacity u({v, w})) would belong to δ H (W ), contradicting the assumption that this cut has capacity less than one. The assumption {v, w} ∈ W for some v, w ∈ / X leads to the same contradiction. Define F := {(v, w) ∈ E(G) : {v, {v, w}} ∈ δ H (W )}. By the above observation we have F ⊆ δG (X ). We define E 1 , E 2 , E 3 , E 4 as above and claim that

Exercises

W = X ∪ E(G[X ]) ∪ E 2 ∪ E 3

299

(12.2)

holds. Again by the above observation, we only have to prove W ∩δG (X ) = E 2 ∪E 3 . + But e = (v, w) ∈ E 1 = δG (X )∩ F implies e ∈ / W by the definition of F. Similarly, − + e = (v, w) ∈ E 2 = δG (X ) ∩ F implies e ∈ W , e = (v, w) ∈ E 3 = δG (X ) \ F − / W . Thus (12.2) is implies e ∈ W , and e = (v, w) ∈ E 4 = δG (X ) \ F implies e ∈ proved. P P So (a) and (b) again hold. Since |W ∩ T | is odd, (a) implies P that v∈X b(v) + e∈F u(e) is odd. Then by (b) and the assumption that e∈δ H (W ) t (e) < 1, we get $ !% X X X 1 X xe + xe > b(v) + u(e) , 2 e∈E(G[X ])

e∈F

v∈X

e∈F

i.e. a violated inequality of the b-matching polytope. Let us summarize: We have shown that the minimum capacity of a T -cut in H is less than one if and only if x violates some inequality of the b-matching polytope. Furthermore, given some T -cut in H with capacity less than one, we can easily construct a violated inequality. So the problem reduces to the M INIMUM C APACITY T -C UT P ROBLEM with nonnegative weights. By Theorem 12.17, the latter can be solved in O(n 4 ) time, where n = |V (H )|. A generalization of this result has been found by Caprara and Fischetti [1996]. Letchford, Reinelt and Theis [2004] showed that it suffices to consider the GomoryHu tree for (G, u). They reduce the S EPARATION P ROBLEM for b-matching (and more general) inequalities to |V (G)| maximum flow computations on the original graph and thus solve it in O(|V (G)|4 ) time. The Padberg-Rao Theorem implies: Corollary 12.19. The M AXIMUM W EIGHT b-M ATCHING P ROBLEM can be solved in polynomial time. Proof: By Corollary 3.33 we have to solve the LP given in Theorem 12.3. By Theorem 4.21 it suffices to have a polynomial-time algorithm for the S EPARATION P ROBLEM. Such an algorithm is provided by Theorem 12.18. Marsh [1979] extended Edmonds’ W EIGHTED M ATCHING A LGORITHM to the M AXIMUM W EIGHT b-M ATCHING P ROBLEM. This combinatorial algorithm is of course more practical than using the E LLIPSOID M ETHOD. But Theorem 12.18 is also interesting for other purposes (see e.g. Section 21.4). For a combinatorial algorithm with a strongly polynomial running time, see Anstee [1987] or Gerards [1995].

Exercises 1. Show that a minimum weight perfect simple 2-matching in an undirected graph G can be found in O(n 6 ) time.

300

∗

12 b-Matchings and T -Joins

2. Let G be an undirected graph and b1 , b2 : V (G)P → N. Describe the convex hull of functions f : E(G) → Z+ with b1 (v) ≤ e∈δ(v) f (e) ≤ b2 (v). Hint: For X, Y ⊆ V (G) with X ∩ Y = ∅ consider the constraint X X X 1 X b2 (x) − b1 (y) , f (e) − f (e) ≤ 2 e∈E(G[X ])

∗

3.

∗

4.

5.

∗

6.

7.

8.

e∈E(G[Y ])∪E(Y,Z )

x∈X

y∈Y

where Z := V (G) \ (X ∪ Y ). Use Theorem 12.3. (Schrijver [1983]) Can one generalize the result of Exercise 2 further by introducing lower and upper capacities on the edges? Note: This can be regarded as an undirected version of the problem in Exercise 3 of Chapter 9. For a common generalization of both problems and also the M INIMUM W EIGHT T -J OIN P ROBLEM see the papers of Edmonds and Johnson [1973], and Schrijver [1983]. Even here a description of the polytope that is TDI is known. Prove Theorem 12.4. Hint: For the sufficiency, use Tutte’s Theorem 10.13 and the constructions in the proofs of Theorems 12.2 and 12.3. The subgraph degree polytope of a graph G is defined to be the convex hull of V (G) such that G has a perfect simple b-matching. Prove that all vectors b ∈ Z+ its dimension is |V (G)| − k, where k is the number of connected components of G that are bipartite. Given an undirected graph, an odd cycle cover is defined to be a subset of edges containing at least one edge of each odd circuit. Show how to find in polynomial time a minimum weight odd cycle cover in a planar graph with nonnegative weights on the edges. Can you also solve the problem for general weights? Hint: Consider the U NDIRECTED C HINESE P OSTMAN P ROBLEM in the planar dual graph and use Theorem 2.26 and Corollary 2.45. Consider the M AXIMUM W EIGHT C UT P ROBLEM in planar graphs: Given an undirected planar graph G with weights c : E(G) → R+ , we look for a maximum weight cut. Can one solve this problem in polynomial time? Hint: Use Exercise 6. Note: For general graphs this problem is NP-hard; see Theorem 16.6. (Hadlock [1975]) Given a graph G with weights c : E(G) → R+ and a set T ⊆ V (G) with |T | even. We construct a new graph G 0 by setting V (G 0 ) := {(v, e) : v ∈ e ∈ E(G)} ∪ {v¯ : v ∈ V (G), |δG (v)| + |{v} ∩ T | odd}, E(G 0 ) := {{(v, e), (w, e)} : e = {v, w} ∈ E(G)} ∪ {{(v, e), (v, f )} : v ∈ V (G), e, f ∈ δG (v), e 6= f } ∪ {{v, ¯ (v, e)} : v ∈ e ∈ E(G), v¯ ∈ V (G 0 )},

Exercises

∗

9.

10.

∗

11.

∗ 12.

13. 14.

301

and define c0 ({(v, e), (w, e)}) := c(e) for e = {v, w} ∈ E(G) and c0 (e0 ) = 0 for all other edges in G 0 . Show that a minimum weight perfect matching in G 0 corresponds to a minimum weight T -join in G. Is this reduction preferable to the one used in the proof of Theorem 12.9? The following problem combines simple perfect b-matchings and T -joins. We are given an undirected graph G with weights c : E(G) → R, a partition of . . the vertex set V (G) = R ∪ S ∪ T , and a function b : R → Z+ . We ask for a subset of edges J ⊆ E(G) such that J ∩ δ(v) = b(v) for v ∈ R, |J ∩ δ(v)| is even for v ∈ S, and |J ∩ δ(v)| is odd for v ∈ T . Show how to reduce this problem to a M INIMUM W EIGHT P ERFECT M ATCHING P ROBLEM. Hint: Consider the constructions in Section 12.1 and Exercise 8. Consider the U NDIRECTED M INIMUM M EAN C YCLE P ROBLEM: Given an undirected graph G and weights c : E(G) → R, find a circuit C in G whose mean weight c(E(C)) |E(C)| is minimum. (a) Show that the M INIMUM M EAN C YCLE A LGORITHM of Section 7.3 cannot be applied to the undirected case. (b) Find a strongly polynomial algorithm for the U NDIRECTED M INIMUM M EAN C YCLE P ROBLEM. Hint: Use Exercise 9. Let G be an undirected graph, T ⊆ V (G) with |T | even, and F ⊆ E(G). Prove: F has nonzero intersection with every T -join if and only if F contains a T -cut. F has nonzero intersection with every T -cut if and only if F contains a T -join. Let G be a planar 2-connected graph with a fixed embedding, let C be the circuit bounding the outer face, and let T be an even cardinality subset of V (C). Prove that the minimum cardinality of a T -join equals the maximum number of edge-disjoint T -cuts. Hint: Colour the edges of C red and blue such that, when traversing C, colours change precisely at the vertices in T . Consider the planar dual graph, split the vertex representing the outer face into a red and a blue vertex, and apply Menger’s Theorem 8.9. Prove Theorem 12.16 using Theorem 11.13 and the construction of Exercise 8. (Edmonds and Johnson [1973]) Let G be an undirected graph and T ⊆ V (G) with |T | even. Prove that the convex hull of the incidence vectors of all T -joins in G is the set of all vectors x ∈ [0, 1] E(G) satisfying X X xe + (1 − xe ) ≥ 1 e∈δG (X )\F

e∈F

for all X ⊆ V (G) and F ⊆ δG (X ) with |X ∩ T | + |F| odd. Hint: Use Theorems 12.16 and 12.10. 15. Let G be an undirected graph and T ⊆ V (G) with |T | = 2k even. Prove that k the minimum cardinality of a T -cut in G equals the maximum of mini=1 λsi ,ti

302

12 b-Matchings and T -Joins

over all pairings T = {s1 , t1 , s2 , t2 , . . . , sk , tk }. (λs,t denotes the maximum number of edge-disjoint s-t-paths.) Can you think of a weighted version of this min-max formula? Hint: Use Theorem 12.17. (Rizzi [2002]) 16. This exercise gives an algorithm for the M INIMUM C APACITY T -C UT P ROB LEM without using Gomory-Hu trees. The algorithm is recursive and – given G, u and T – proceeds as follows: 1. First we P find a set X ⊆ V (G) with T ∩ X 6= ∅ and T \ X 6= ∅, such that u(X ) := e∈δG (X ) u(e) is minimum (cf. Exercise 22 of Chapter 8). If |T ∩ X | happens to be odd, we are done (return X ). 2. Otherwise we apply the algorithm recursively first to G, u and T ∩ X , and then to G, u and T \ X . We obtain a set Y ⊆ V (G) with |(T ∩ X ) ∩ Y | odd and u(Y ) minimum and a set Z ⊆ V (G) with |(T \ X ) ∩ Z | odd and u(Z ) minimum. W.l.o.g. T \ X 6⊆ Y and X ∩ T 6⊆ Z (otherwise replace Y by V (G) \ Y and/or Z by V (G) \ Z ). 3. If u(X ∩ Y ) < u(Z \ X ) then return X ∩ Y else return Z \ X . Show that this algorithm works correctly and that its running time is O(n 5 ), where n = |V (G)|. 17. Show how to solve the M AXIMUM W EIGHT b-M ATCHING P ROBLEM for the special case when b(v) is even for all v ∈ V (G) in strongly polynomial time. Hint: Reduction to a M INIMUM C OST F LOW P ROBLEM as in Exercise 10 of Chapter 9.

References General Literature: Cook, W.J., Cunningham, W.H., Pulleyblank, W.R., and Schrijver, A. [1998]: Combinatorial Optimization. Wiley, New York 1998, Sections 5.4 and 5.5 Frank, A. [1996]: A survey on T -joins, T -cuts, and conservative weightings. In: Combinatorics, Paul Erd˝os is Eighty; Volume 2 (D. Mikl´os, V.T. S´os, T. Sz˝onyi, eds.), Bolyai Society, Budapest 1996, pp. 213–252 Gerards, A.M.H. [1995]: Matching. In: Handbooks in Operations Research and Management Science; Volume 7: Network Models (M.O. Ball, T.L. Magnanti, C.L. Monma, G.L. Nemhauser, eds.), Elsevier, Amsterdam 1995, pp. 135–224 Lov´asz, L., and Plummer, M.D. [1986]: Matching Theory. Akad´emiai Kiad´o, Budapest 1986, and North-Holland, Amsterdam 1986 Schrijver, A. [1983]: Min-max results in combinatorial optimization; Section 6. In: Mathematical Programming; The State of the Art – Bonn 1982 (A. Bachem, M. Gr¨otschel, B. Korte, eds.), Springer, Berlin 1983, pp. 439–500 Schrijver, A. [2003]: Combinatorial Optimization: Polyhedra and Efficiency. Springer, Berlin 2003, Chapters 29–33

References

303

Cited References: Anstee, R.P. [1987]: A polynomial algorithm for b-matchings: an alternative approach. Information Processing Letters 24 (1987), 153–157 Caprara, A., and Fischetti, M. [1996]: {0, 12 }-Chv´atal-Gomory cuts. Mathematical Programming 74 (1996), 221–235 Edmonds, J. [1965]: Maximum matching and a polyhedron with (0,1) vertices. Journal of Research of the National Bureau of Standards B 69 (1965), 125–130 Edmonds, J., and Johnson, E.L. [1970]: Matching: A well-solved class of integer linear programs. In: Combinatorial Structures and Their Applications; Proceedings of the Calgary International Conference on Combinatorial Structures and Their Applications 1969 (R. Guy, H. Hanani, N. Sauer, J. Schonheim, eds.), Gordon and Breach, New York 1970, pp. 69–87 Edmonds, J., and Johnson, E.L. [1973]: Matching, Euler tours and the Chinese postman problem. Mathematical Programming 5 (1973), 88–124 Guan, M. [1962]: Graphic programming using odd and even points. Chinese Mathematics 1 (1962), 273–277 Hadlock, F. [1975]: Finding a maximum cut of a planar graph in polynomial time. SIAM Journal on Computing 4 (1975), 221–225 Letchford, A.N., Reinelt, G., and Theis, D.O. [2004]: A faster exact separation algorithm for blossom inequalities. Proceedings of the 10th Conference on Integer Programming and Combinatorial Optimization; LNCS 3064 (D. Bienstock, G. Nemhauser, eds.), Springer, Berlin 2004, pp. 196–205 Marsh, A.B. [1979]: Matching algorithms. Ph.D. thesis, Johns Hopkins University, Baltimore 1979 Padberg, M.W., and Rao, M.R. [1982]: Odd minimum cut-sets and b-matchings. Mathematics of Operations Research 7 (1982), 67–80 Pulleyblank, W.R. [1973]: Faces of matching polyhedra. Ph.D. thesis, University of Waterloo, 1973 Pulleyblank, W.R. [1980]: Dual integrality in b-matching problems. Mathematical Programming Study 12 (1980), 176–196 Rizzi, R. [2002]: Minimum T -cuts and optimal T -pairings. Discrete Mathematics 257 (2002), 177–181 Seb˝o, A. [1987]: A quick proof of Seymour’s theorem on T -joins. Discrete Mathematics 64 (1987), 101–103 Seymour, P.D. [1981]: On odd cuts and multicommodity flows. Proceedings of the London Mathematical Society (3) 42 (1981), 178–192 Tutte, W.T. [1952]: The factors of graphs. Canadian Journal of Mathematics 4 (1952), 314–328 Tutte, W.T. [1954]: A short proof of the factor theorem for finite graphs. Canadian Journal of Mathematics 6 (1954), 347–352

13 Matroids

Many combinatorial optimization problems can be formulated as follows. Given a set system (E, F), i.e. a finite set E and some F ⊆ 2 E , and a cost function c : F → R, find an element of F whose cost is minimum or maximum.PIn the following we consider modular functions c, i.e. assume that c(X ) = c(∅)+ x∈X (c({x})−c(∅)) for all XP ⊆ E; equivalently we are given a function c : E → R and write c(X ) = e∈X c(e). In this chapter we restrict ourselves to those combinatorial optimization problems where F describes an independence system (i.e. is closed under subsets) or even a matroid. The results of this chapter generalize several results obtained in previous chapters. In Section 13.1 we introduce independence systems and matroids and show that many combinatorial optimization problems can be described in this context. There are several equivalent axiom systems for matroids (Section 13.2) and an interesting duality relation discussed in Section 13.3. The main reason why matroids are important is that a simple greedy algorithm can be used for optimization over matroids. We analyze greedy algorithms in Section 13.4 before turning to the problem of optimizing over the intersection of two matroids. As shown in Sections 13.5 and 13.7 this problem can be solved in polynomial time. This also solves the problem of covering a matroid by independent sets as discussed in Section 13.6.

13.1 Independence Systems and Matroids Definition 13.1. A set system (E, F) is an independence system if (M1) ∅ ∈ F; (M2) If X ⊆ Y ∈ F then X ∈ F. The elements of F are called independent, the elements of 2 E \ F dependent. Minimal dependent sets are called circuits, maximal independent sets are called bases. For X ⊆ E, the maximal independent subsets of X are called bases of X . Definition 13.2. Let (E, F) be an independence system. For X ⊆ E we define the rank of X by r (X ) := max{|Y | : Y ⊆ X, Y ∈ F}. Moreover, we define the closure of X by σ (X ) := {y ∈ E : r (X ∪ {y}) = r (X )}.

306

13 Matroids

Throughout this chapter, (E, F) will be an independence system, and c : E → R will be a cost function. We shall concentrate on the following two problems:

M AXIMIZATION P ROBLEM F OR I NDEPENDENCE S YSTEMS Instance: Task:

An independence system (E, F) and c : E → R. P Find an X ∈ F such that c(X ) := e∈X c(e) is maximum.

M INIMIZATION P ROBLEM F OR I NDEPENDENCE S YSTEMS Instance:

An independence system (E, F) and c : E → R.

Task:

Find a basis B such that c(B) is minimum.

The instance specification is somewhat vague. The set E and the cost function c are given explicitly as usual. However, the set F is usually not given by an explicit list of its elements. Rather one assumes an oracle which – given a subset F ⊆ E – decides whether F ∈ F. We shall return to this question in Section 13.4. The following list shows that many combinatorial optimization problems actually have one of the above two forms: (1) M AXIMUM W EIGHT S TABLE S ET P ROBLEM Given a graph G and weights c : V (G) → R, find a stable set X in G of maximum weight. Here E = V (G) and F = {F ⊆ E : F is stable in G}. (2) TSP Given a complete undirected graph G and weights c : E(G) → R+ , find a minimum weight Hamiltonian circuit in G. Here E = E(G) and F = {F ⊆ E : F is a subset of a Hamiltonian circuit in G}. (3) S HORTEST PATH P ROBLEM Given a digraph G, c : E(G) → R and s, t ∈ V (G) such that t is reachable from s, find a shortest s-t-path in G with respect to c. Here E = E(G) and F = {F ⊆ E : F is a subset of an s-t-path}. (4) K NAPSACK P ROBLEM Given nonnegative numbers i ≤ n), and W , find a subset P n, ci , wi (1 ≤ P S ⊆ {1, . . . , n} such that j∈S w j ≤ W and j∈S c j is maximum. o n P Here E = {1, . . . , n} and F = F ⊆ E : j∈F w j ≤ W . (5) M INIMUM S PANNING T REE P ROBLEM Given a connected undirected graph G and weights c : E(G) → R, find a minimum weight spanning tree in G. Here E = E(G) and F is the set of forests in G. (6) M AXIMUM W EIGHT F OREST P ROBLEM Given an undirected graph G and weights c : E(G) → R, find a maximum weight forest in G. Here again E = E(G) and F is the set of forests in G.

13.1 Independence Systems and Matroids

307

(7) S TEINER T REE P ROBLEM Given a connected undirected graph G, weights c : E(G) → R+ , and a set T ⊆ V (G) of terminals, find a Steiner tree for T , i.e. a tree S with T ⊆ V (S) and E(S) ⊆ E(G), such that c(E(S)) is minimum. Here E = E(G) and F = {F ⊆ E : F is a subset of a Steiner tree for T }. (8) M AXIMUM W EIGHT B RANCHING P ROBLEM Given a digraph G and weights c : E(G) → R, find a maximum weight branching in G. Here E = E(G) and F is the set of branchings in G. (9) M AXIMUM W EIGHT M ATCHING P ROBLEM Given an undirected graph G and weights c : E(G) → R, find a maximum weight matching in G. Here E = E(G) and F is the set of matchings in G. This list contains NP-hard problems ((1),(2),(4),(7)) as well as polynomially solvable problems ((5),(6),(8),(9)). Problem (3) is NP-hard in the above form but polynomially solvable for nonnegative weights. (NP-hardness is introduced in Chapter 15.) Definition 13.3. An independence system is a matroid if (M3) If X, Y ∈ F and |X | > |Y |, then there is an x ∈ X \ Y with Y ∪ {x} ∈ F. The name matroid points out that the structure is a generalization of matrices. This will become clear by our first example: Proposition 13.4. The following independence systems (E, F) are matroids: (a) E F (b) E F (c) E (d) E (s (e) E F

is a set of columns of a matrix A over some field, and := {F ⊆ E : The columns in F are linearly independent over that field}. is a set of edges of some undirected graph G and := {F ⊆ E : (V (G), F) is a forest}. is a finite set, k an integer and F := {F ⊆ E : |F| ≤ k}. is a set of edges of some undirected graph G, S a stable set in G, ks integers ∈ S) and F := {F ⊆ E : |δ F (s)| ≤ ks for all s ∈ S}. is a set of edges of some digraph G, S ⊆ V (G), ks integers (s ∈ S) and := {F ⊆ E : |δ − F (s)| ≤ ks for all s ∈ S}.

Proof: In all cases it is obvious that (E, F) is indeed an independence system. So it remains to show that (M3) holds. For (a) this is well-known from Linear Algebra, for (c) it is trivial. To prove (M3) for (b), let X, Y ∈ F and suppose Y ∪ {x} 6∈ F for all x ∈ X \ Y . We show that |X | ≤ |Y |. For each edge x = {v, w} ∈ X , v and w are in the same connected component of (V (G), Y ). Hence each connected component Z ⊆ V (G) of (V (G), X ) is a subset of a connected component of (V (G), Y ). So the number p of connected components of the forest (V (G), X ) is greater than or equal to the number q of connected components of the forest (V (G), Y ). But then |V (G)| − |X | = p ≥ q = |V (G)| − |Y |, implying |X | ≤ |Y |.

308

13 Matroids

To verify (M3) for (d), let X, Y ∈ F with |X | > |Y |. Let S 0 := {s ∈ S : |δY (s)| = ks }. As |X | > |Y | and |δ X (s)| ≤ ks for all s ∈ S 0 , there exists an e ∈ X \ Y with e ∈ / δ(s) for s ∈ S 0 . Then Y ∪ {e} ∈ F. For (e) the proof is identical except for replacing δ by δ − . Some of these matroids have special names: The matroid in (a) is called the vector matroid of A. Let M be a matroid. If there is a matrix A over the field F such that M is the vector matroid of A, then M is called representable over F. There are matroids that are not representable over any field. The matroid in (b) is called the cycle matroid of G and will sometimes be denoted by M(G). A matroid which is the cycle matroid of some graph is called a graphic matroid. The matroids in (c) are called uniform matroids. In our list of independence systems at the beginning of this section, the only matroids are the graphic matroids in (5) and (6). To check that all the other independence systems in the above list are not matroids in general is easily proved with the help of the following theorem (Exercise 1): Theorem 13.5. Let (E, F) be an independence system. Then the following statements are equivalent: (M3) If X, Y ∈ F and |X | > |Y |, then there is an x ∈ X \ Y with Y ∪ {x} ∈ F. (M30 ) If X, Y ∈ F and |X | = |Y |+1, then there is an x ∈ X \Y with Y ∪{x} ∈ F. (M300 ) For each X ⊆ E, all bases of X have the same cardinality. Proof: Trivially, (M3)⇔(M30 ) and (M3)⇒(M300 ). To prove (M300 )⇒(M3), let X, Y ∈ F and |X | > |Y |. By (M300 ), Y cannot be a basis of X ∪ Y . So there must be an x ∈ (X ∪ Y ) \ Y = X \ Y such that Y ∪ {x} ∈ F. Sometimes it is useful to have a second rank function: Definition 13.6. Let (E, F) be an independence system. For X ⊆ E we define the lower rank by ρ(X ) := min{|Y | : Y ⊆ X, Y ∈ F and Y ∪ {x} ∈ / F for all x ∈ X \ Y }. The rank quotient of (E, F) is defined by q(E, F) := min

F⊆E

ρ(F) . r (F)

Proposition 13.7. Let (E, F) be an independence system. Then q(E, F) ≤ 1. Furthermore, (E, F) is a matroid if and only if q(E, F) = 1. Proof: q(E, F) ≤ 1 follows from the definition. q(E, F) = 1 is obviously equivalent to (M300 ). To estimate the rank quotient, the following statement can be used:

13.2 Other Matroid Axioms

309

Theorem 13.8. (Hausmann, Jenkyns and Korte [1980]) Let (E, F) be an independence system. If, for any A ∈ F and e ∈ E, A ∪ {e} contains at most p circuits, then q(E, F) ≥ 1p . |J | 1 Proof: Let F ⊆ E and J, K two bases of F. We show |K | ≥ p. Let J \ K = {e1 , . . . , et }. We construct a sequence K = K 0 , K 1 , . . . , K t of independent subsets of J ∪ K such that J ∩ K ⊆ K i , K i ∩{e1 , . . . , et } = {e1 , . . . , ei } and |K i−1 \ K i | ≤ p for i = 1, . . . , t. Since K i ∪ {ei+1 } contains at most p circuits and each such circuit must meet K i \ J (because J is independent), there is an X ⊆ K i \ J such that |X | ≤ p and (K i \ X ) ∪ {ei+1 } ∈ F. We set K i+1 := (K i \ X ) ∪ {ei+1 }. Now J ⊆ K t ∈ F. Since J is a basis of F, J = K t . We conclude that

|K \ J | =

t X

|K i−1 \ K i | ≤ pt = p |J \ K |,

i=1

proving |K | ≤ p |J |.

This shows that in example (9) we have q(E, F) ≥ 12 (see also Exercise 1 of Chapter 10). In fact q(E, F) = 12 iff G contains a path of length 3 as a subgraph (otherwise q(E, F) = 1). For the independence system in example (1) of our list, the rank quotient can become arbitrarily small (choose G to be a star). In Exercise 5, the rank quotients for other independence systems will be discussed.

13.2 Other Matroid Axioms In this section we consider other axiom systems defining matroids. They characterize fundamental properties of the family of bases, the rank function, the closure operator and the family of circuits of a matroid. Theorem 13.9. Let E be a finite set and B ⊆ 2 E . B is the set of bases of some matroid (E, F) if and only if the following holds: (B1) B 6= ∅; (B2) For any B1 , B2 ∈ B and x ∈ B1 \ B2 there exists a y ∈ B2 \ B1 with (B1 \ {x}) ∪ {y} ∈ B. Proof: The set of bases of a matroid satisfies (B1) (by (M1)) and (B2): For bases B1 , B2 and x ∈ B1 \ B2 we have that B1 \ {x} is independent. By (M3) there is some y ∈ B2 \ B1 such that (B1 \ {x}) ∪ {y} is independent. Indeed, it must be a basis, because all bases of a matroid have the same cardinality. On the other hand, let B satisfy (B1) and (B2). We first show that all elements of B have the same cardinality: Otherwise let B1 , B2 ∈ B with |B1 | > |B2 | such that |B1 ∩ B2 | is maximum. Let x ∈ B1 \ B2 . By (B2) there is a y ∈ B2 \ B1 with (B1 \ {x}) ∪ {y} ∈ B, contradicting the maximality of |B1 ∩ B2 |.

310

13 Matroids

Now let F := {F ⊆ E : there exists a B ∈ B with F ⊆ B}. (E, F) is an independence system, and B is the family of its bases. To show that (E, F) satisfies (M3), let X, Y ∈ F with |X | > |Y |. Let X ⊆ B1 ∈ B and Y ⊆ B2 ∈ B, where B1 and B2 are chosen such that |B1 ∩ B2 | is maximum. If B2 ∩ (X \ Y ) 6= ∅, we are done because we can augment Y . We claim that the other case, B2 ∩ (X \ Y ) = ∅, is impossible. Namely, with this assumption we get |B1 ∩ B2 | + |Y \ B1 | + |(B2 \ B1 ) \ Y | = |B2 | = |B1 | ≥ |B1 ∩ B2 | + |X \ Y |. Since |X \ Y | > |Y \ X | ≥ |Y \ B1 |, this implies (B2 \ B1 ) \ Y 6= ∅. So let y ∈ (B2 \ B1 ) \ Y . By (B2) there exists an x ∈ B1 \ B2 with (B2 \ {y}) ∪ {x} ∈ B, contradicting the maximality of |B1 ∩ B2 |. See Exercise 7 for a similar statement. A very important property of matroids is that the rank function is submodular: Theorem 13.10. Let E be a finite set and r : 2 E → Z+ . Then the following statements are equivalent: (a) r is the rank function of a matroid (E, F) (and F = {F ⊆ E : r (F) = |F|}). (b) For all X, Y ⊆ E: (R1) r (X ) ≤ |X |; (R2) If X ⊆ Y then r (X ) ≤ r (Y ); (R3) r (X ∪ Y ) + r (X ∩ Y ) ≤ r (X ) + r (Y ). (c) For all X ⊆ E and x, y ∈ E: (R10 ) r (∅) = 0; (R20 ) r (X ) ≤ r (X ∪ {y}) ≤ r (X ) + 1; (R30 ) If r (X ∪ {x}) = r (X ∪ {y}) = r (X ) then r (X ∪ {x, y}) = r (X ). Proof: (a)⇒(b): If r is a rank function of an independence system (E, F), (R1) and (R2) evidently hold. If (E, F) is a matroid, we can also show (R3): Let X, Y ⊆ E, and let A be a basis of X ∩ Y . By (M3), A can be extended . . to a basis A ∪ B of X and to a basis (A ∪ B) ∪ C of X ∪ Y . Then A ∪ C is an independent subset of Y , so r (X ) + r (Y ) ≥ |A ∪ B| + |A ∪ C| = 2|A| + |B| + |C| = |A ∪ B ∪ C| + |A| = r (X ∪ Y ) + r (X ∩ Y ). (b)⇒(c): (R10 ) is implied by (R1). r (X ) ≤ r (X ∪ {y}) follows from (R2). By (R3) and (R1),

13.2 Other Matroid Axioms

311

r (X ∪ {y}) ≤ r (X ) + r ({y}) − r (X ∩ {y}) ≤ r (X ) + r ({y}) ≤ r (X ) + 1, proving (R20 ). (R30 ) is trivial for x = y. For x 6= y we have, by (R2) and (R3), 2r (X ) ≤ r (X ) + r (X ∪ {x, y}) ≤ r (X ∪ {x}) + r (X ∪ {y}), implying (R30 ). (c)⇒(a): Let r : 2 E → Z+ be a function satisfying (R10 )–(R30 ). Let F := {F ⊆ E : r (F) = |F|}. We claim that (E, F) is a matroid. (M1) follows from (R10 ). (R20 ) implies r (X ) ≤ |X | for all X ⊆ E. If Y ∈ F, y ∈ Y and X := Y \ {y}, we have |X | + 1 = |Y | = r (Y ) = r (X ∪ {y}) ≤ r (X ) + 1 ≤ |X | + 1, so X ∈ F. This implies (M2). Now let X, Y ∈ F and |X | = |Y | + 1. Let X \ Y = {x1 , . . . , xk }. Suppose that (M30 ) is violated, i.e. r (Y ∪ {xi }) = |Y | for i = 1, . . . , k. Then by (R30 ) r (Y ∪ {x1 , xi }) = r (Y ) for i = 2, . . . , k. Repeated application of this argument yields r (Y ) = r (Y ∪ {x1 , . . . , xk }) = r (X ∪ Y ) ≥ r (X ), a contradiction. So (E, F) is indeed a matroid. To show that r is the rank function of this matroid, we have to prove that r (X ) = max{|Y | : Y ⊆ X, r (Y ) = |Y |} for all X ⊆ E. So let X ⊆ E, and let Y a maximum subset of X with r (Y ) = |Y |. For all x ∈ X \ Y we have r (Y ∪ {x}) < |Y | + 1, so by (R20 ) r (Y ∪ {x}) = |Y |. Repeated application of (R30 ) implies r (X ) = |Y |. Theorem 13.11. Let E be a finite set and σ : 2 E → 2 E a function. σ is the closure operator of a matroid (E, F) if and only if the following conditions hold for all X, Y ⊆ E and x, y ∈ E: (S1) (S2) (S3) (S4)

X ⊆ σ (X ); X ⊆ Y ⊆ E implies σ (X ) ⊆ σ (Y ); σ (X ) = σ (σ (X )); If y ∈ / σ (X ) and y ∈ σ (X ∪ {x}) then x ∈ σ (X ∪ {y}).

Proof: If σ is the closure operator of a matroid, then (S1) holds trivially. For X ⊆ Y and z ∈ σ (X ) we have by (R3) and (R2) r (X ) + r (Y ) = r (X ∪ {z}) + r (Y ) ≥ r ((X ∪ {z}) ∩ Y ) + r (X ∪ {z} ∪ Y ) ≥ r (X ) + r (Y ∪ {z}), implying z ∈ σ (Y ) and thus proving (S2). By repeated application of (R30 ) we have r (σ (X )) = r (X ) for all X , which implies (S3).

312

13 Matroids

To prove (S4), suppose that there are X, x, y with y ∈ / σ (X ), y ∈ σ (X ∪ {x}) and x ∈ / σ (X ∪ {y}). Then r (X ∪ {y}) = r (X ) + 1, r (X ∪ {x, y}) = r (X ∪ {x}) and r (X ∪ {x, y}) = r (X ∪ {y}) + 1. Thus r (X ∪ {x}) = r (X ) + 2, contradicting (R20 ). To show the converse, let σ : 2 E → 2 E be a function satisfying (S1)–(S4). Let F := {X ⊆ E : x ∈ / σ (X \ {x}) for all x ∈ X }. We claim that (E, F) is a matroid. (M1) is trivial. For X ⊆ Y ∈ F and x ∈ X we have x ∈ / σ (Y \{x}) ⊇ σ (X \{x}), so X ∈ F and (M2) holds. To prove (M3) we need the following statement: Claim: For X ∈ F and Y ⊆ E with |X | > |Y | we have X 6⊆ σ (Y ). We prove the claim by induction on |Y \ X |. If Y ⊂ X , then let x ∈ X \ Y . Since X ∈ F we have x ∈ / σ (X \ {x}) ⊇ σ (Y ) by (S2). Hence x ∈ X \ σ (Y ) as required. If |Y \ X | > 0, then let y ∈ Y \ X . By the induction hypothesis there exists an x ∈ X \ σ (Y \ {y}). If x 6∈ σ (Y ), then we are done. Otherwise x ∈ / σ (Y \ {y}) but x ∈ σ (Y ) = σ ((Y \ {y}) ∪ {y}), so by (S4) y ∈ σ ((Y \ {y}) ∪ {x}). By (S1) we get Y ⊆ σ ((Y \ {y}) ∪ {x}) and thus σ (Y ) ⊆ σ ((Y \ {y}) ∪ {x}) by (S2) and (S3). Applying the induction hypothesis to X and (Y \ {y}) ∪ {x} (note that x 6= y) yields X 6⊆ σ ((Y \ {y}) ∪ {x}), so X 6⊆ σ (Y ) as required. Having proved the claim we can easily verify (M3). Let X, Y ∈ F with |X | > |Y |. By the claim there exists an x ∈ X \ σ (Y ). Now for each z ∈ Y ∪ {x} we have z ∈ / σ (Y \ {z}), because Y ∈ F and x ∈ / σ (Y ) = σ (Y \ {x}). By (S4) z∈ / σ (Y \ {z}) and x ∈ / σ (Y ) imply z ∈ / σ ((Y \ {z}) ∪ {x}) ⊇ σ ((Y ∪ {x}) \ {z}). Hence Y ∪ {x} ∈ F. So (M3) indeed holds and (E, F) is a matroid, say with rank function r and closure operator σ 0 . It remains to prove that σ = σ 0 . By definition, σ 0 (X ) = {y ∈ E : r (X ∪ {y}) = r (X )} and r (X ) = max{|Y | : Y ⊆ X, y ∈ / σ (Y \ {y}) for all y ∈ Y } for all X ⊆ E. Let X ⊆ E. To show σ 0 (X ) ⊆ σ (X ), let z ∈ σ 0 (X ) \ X . Let Y be a basis of X . Since r (Y ∪ {z}) ≤ r (X ∪ {z}) = r (X ) = |Y | < |Y ∪ {z}| we have y ∈ σ ((Y ∪{z})\{y}) for some y ∈ Y ∪{z}. If y = z, then we have z ∈ σ (Y ). Otherwise (S4) and y ∈ / σ (Y \ {y}) also yield z ∈ σ (Y ). Hence by (S2) z ∈ σ (X ). Together with (S1) this implies σ 0 (X ) ⊆ σ (X ). Now let z ∈ / σ 0 (X ), i.e. r (X ∪ {z}) > r (X ). Let now Y be a basis of X ∪ {z}. Then z ∈ Y and |Y \ {z}| = |Y | − 1 = r (X ∪ {z}) − 1 = r (X ). Therefore Y \ {z} is a basis of X , implying X ⊆ σ 0 (Y \ {z}) ⊆ σ (Y \ {z}), and thus σ (X ) ⊆ σ (Y \ {z}). As z ∈ / σ (Y \ {z}), we conclude that z 6∈ σ (X ). Theorem 13.12. Let E be a finite set and C ⊆ 2 E . C is the set of circuits of an independence system (E, F), where F = {F ⊂ E : there exists no C ∈ C with C ⊆ F}, if and only if the following conditions hold: (C1) ∅ ∈ / C;

13.3 Duality

313

(C2) For any C1 , C2 ∈ C, C1 ⊆ C2 implies C1 = C2 . Moreover, if C is the set of circuits of an independence system (E, F), then the following statements are equivalent: (a) (E, F) is a matroid. (b) For any X ∈ F and e ∈ E, X ∪ {e} contains at most one circuit. (C3) For any C1 , C2 ∈ C with C1 6= C2 and e ∈ C1 ∩ C2 there exists a C3 ∈ C with C3 ⊆ (C1 ∪ C2 ) \ {e}. (C30 ) For any C1 , C2 ∈ C, e ∈ C1 ∩ C2 and f ∈ C1 \ C2 there exists a C3 ∈ C with f ∈ C3 ⊆ (C1 ∪ C2 ) \ {e}. Proof: By definition, the family of circuits of any independence system satisfies (C1) and (C2). If C satisfies (C1), then (E, F) is an independence system. If C also satisfies (C2), it is the set of circuits of this independence system. (a)⇒(C30 ): Let C be the family of circuits of a matroid, and let C1 , C2 ∈ C, e ∈ C1 ∩ C2 and f ∈ C1 \ C2 . By applying (R3) twice we have |C1 | − 1 + r ((C1 ∪ C2 ) \ {e, f }) + |C2 | − 1 = r (C1 ) + r ((C1 ∪ C2 ) \ {e, f }) + r (C2 ) ≥ r (C1 ) + r ((C1 ∪ C2 ) \ { f }) + r (C2 \ {e}) ≥ r (C1 \ { f }) + r (C1 ∪ C2 ) + r (C2 \ {e}) = |C1 | − 1 + r (C1 ∪ C2 ) + |C2 | − 1. So r ((C1 ∪ C2 ) \ {e, f }) = r (C1 ∪ C2 ). Let B be a basis of (C1 ∪ C2 ) \ {e, f }. Then B ∪ { f } contains a circuit C3 , with f ∈ C3 ⊆ (C1 ∪ C2 ) \ {e} as required. (C30 )⇒(C3): trivial. (C3)⇒(b): If X ∈ F and X ∪ {e} contains two circuits C1 , C2 , (C3) implies (C1 ∪ C2 ) \ {e} ∈ / F. However, (C1 ∪ C2 ) \ {e} is a subset of X . (b)⇒(a): Follows from Theorem 13.8 and Proposition 13.7. Especially property (b) will be used often. For X ∈ F and e ∈ E such that X ∪ {e} 6∈ F we write C(X, e) for the unique circuit in X ∪ {e}. If X ∪ {e} ∈ F we write C(X, e) := ∅.

13.3 Duality Another basic concept in matroid theory is duality. Definition 13.13. Let (E, F) be an independence system. We define the dual of (E, F) by (E, F ∗ ), where F ∗ = {F ⊆ E : there is a basis B of (E, F) such that F ∩ B = ∅}. It is obvious that the dual of an independence system is again an independence system.

314

13 Matroids

Proposition 13.14. (E, F ∗∗ ) = (E, F). Proof: F ∈ F ∗∗ ⇔ there is a basis B ∗ of (E, F ∗ ) such that F ∩ B ∗ = ∅ ⇔ there is a basis B of (E, F) such that F ∩ (E \ B) = ∅ ⇔ F ∈ F. Theorem 13.15. Let (E, F) be an independence system, (E, F ∗ ) its dual, and let r and r ∗ be the corresponding rank functions. (a) (E, F) is a matroid if and only if (E, F ∗ ) is a matroid. (Whitney [1935]) (b) If (E, F) is a matroid, then r ∗ (F) = |F| + r (E \ F) − r (E) for F ⊆ E. Proof: Due to Proposition 13.14 we have to show only one direction of (a). So let (E, F) be a matroid. We define q : 2 E → Z+ by q(F) := |F| + r (E \ F) − r (E). We claim that q satisfies (R1), (R2) and (R3). By this claim and Theorem 13.10, q is the rank function of a matroid. Since obviously q(F) = |F| if and only if F ∈ F ∗ , we conclude that q = r ∗ , and (a) and (b) are proved. Now we prove the above claim: q satisfies (R1) because r satisfies (R2). To check that q satisfies (R2), let X ⊆ Y ⊆ E. Since (E, F) is a matroid, (R3) holds for r , so r (E \ X ) + 0 = r ((E \ Y ) ∪ (Y \ X )) + r (∅) ≤ r (E \ Y ) + r (Y \ X ). We conclude that r (E \ X ) − r (E \ Y ) ≤ r (Y \ X ) ≤ |Y \ X | = |Y | − |X | (note that r satisfies (R1)), so q(X ) ≤ q(Y ). It remains to show that q satisfies (R3). Let X, Y ⊆ E. Using the fact that r satisfies (R3) we have q(X ∪ Y ) + q(X ∩ Y ) = |X ∪ Y | + |X ∩ Y | + r (E \ (X ∪ Y )) + r (E \ (X ∩ Y )) − 2r (E) = |X | + |Y | + r ((E \ X ) ∩ (E \ Y )) + r ((E \ X ) ∪ (E \ Y )) − 2r (E) ≤ |X | + |Y | + r (E \ X ) + r (E \ Y ) − 2r (E) = q(X ) + q(Y ). For any graph G we have introduced the cycle matroid M(G) which of course has a dual. For an embedded planar graph G there is also a planar dual G ∗ (which in general depends on the embedding of G). It is interesting that the two concepts of duality coincide: Theorem 13.16. Let G be a connected planar graph with an arbitrary planar embedding, and G ∗ the planar dual. Then M(G ∗ ) = (M(G))∗ .

13.3 Duality

315

∗

Proof: For T ⊆ E(G) we write T := {e∗ : e ∈ E(G) \ T }, where e∗ is the dual of edge e. We have to prove the following: ∗ Claim: T is the edge set of a spanning tree in G iff T is the edge set of a ∗ spanning tree in G . ∗ ∗

Since (G ∗ )∗ = G (by Proposition 2.42) and (T ) = T it suffices to prove one direction of the claim. ∗ So let T ⊆ E(G), where T is the edge set of a spanning tree in G ∗ . (V (G), T ) must be connected, for otherwise a connected component would define a cut, the ∗ dual of which contains a circuit in T (Theorem 2.43). On the other hand, if ∗ (V (G), T ) contains a circuit, then the dual edge set is a cut and (V (G ∗ ), T ) is disconnected. Hence (V (G), T ) is indeed a spanning tree in G. This implies that if G is planar then (M(G))∗ is a graphic matroid. If, for any graph G, (M(G))∗ is a graphic matroid, say (M(G))∗ = M(G 0 ), then G 0 is evidently an abstract dual of G. By Exercise 34 of Chapter 2, the converse is also true: G is planar if and only if G has an abstract dual (Whitney [1933]). This implies that (M(G))∗ is graphic if and only if G is planar. Note that Theorem 13.16 quite directly implies Euler’s formula (Theorem 2.32): Let G be a connected planar graph with a planar embedding, and let M(G) be the cycle matroid of G. By Theorem 13.15 (b), r (E(G)) + r ∗ (E(G)) = |E(G)|. Since r (E(G)) = |V (G)| − 1 (the number of edges in a spanning tree) and r ∗ (E(G)) = |V (G ∗ )| − 1 (by Theorem 13.16), we obtain that the number of faces of G is |V (G ∗ )| = |E(G)| − |V (G)| + 2, Euler’s formula. Duality of independence systems has also some nice applications in polyhedral combinatorics. A set system (E, F) is called a clutter if X 6⊂ Y for all X, Y ∈ F. If (E, F) is a clutter, then we define its blocking clutter by B L(E, F) := (E, {X ⊆ E : X ∩ Y 6= ∅ for all Y ∈ F, X minimal with this property}). For an independence system (E, F) and its dual (E, F ∗ ) let B and B ∗ be the family of bases, and C and C ∗ the family of circuits, respectively. (Every clutter arises in both of these ways except for F = ∅ or F = {∅}.) It follows immediately from the definitions that (E, B ∗ ) = B L(E, C) and (E, C ∗ ) = B L(E, B). Together with Proposition 13.14 this implies B L(B L(E, F)) = (E, F) for every clutter (E, F). We give some examples for clutters (E, F) and their blocking clutters (E, F 0 ). In each case E = E(G) for some graph G: (1) F is the set of spanning trees, F 0 is the set of minimal cuts; (2) F is the set of arborescences rooted at r , F 0 is the set of minimal r -cuts; (3) F is the set of s-t-paths, F 0 is the set of minimal cuts separating s and t (this example works in undirected graphs and in digraphs); (4) F is the set of circuits in an undirected graph, F 0 is the set of complements of maximal forests;

316

13 Matroids

(5) F is the set of circuits in a digraph, F 0 is the set of minimal feedback edge sets (a feedback edge set is a set of edges whose removal makes the digraph acyclic); (6) F is the set of minimal edge sets whose contraction makes the digraph strongly connected, F 0 is the set of minimal directed cuts; (7) F is the set of minimal T -joins, F 0 is the set of minimal T -cuts. All these blocking relations can be verified easily: (1) and (2) follow directly from Theorems 2.4 and 2.5, (3), (4) and (5) are trivial, (6) follows from Corollary 2.7, and (7) from Proposition 12.6. In some cases, the blocking clutter gives a polyhedral characterization of the M INIMIZATION P ROBLEM F OR I NDEPENDENCE S YSTEMS for nonnegative cost functions: Definition 13.17. Let (E, F) be a clutter, (E, F 0 ) its blocking clutter and P the convex hull of the incidence vectors of the elements of F. We say that (E, F) has the Max-Flow-Min-Cut property if ) ( o n X E E 0 xe ≥ 1 for all B ∈ F . x + y : x ∈ P, y ∈ R+ = x ∈ R+ : e∈B

Examples are (2) and (7) of our list above (by Theorems 6.14 and 12.16), but also (3) and (6) (see Exercise 10). The following theorem relates the above covering-type formulation to a packing formulation of the dual problem and allows to derive certain min-max theorems from others: Theorem 13.18. (Fulkerson [1971], Lehman [1979]) Let (E, F) be a clutter and (E, F 0 ) its blocking clutter. Then the following statements are equivalent: (a) (E, F) has the Max-Flow-Min-Cut property; property; (b) (E, F 0 ) has the Max-Flow-Min-Cut 0 P (c) min{c(A) : A ∈ F} = max 1ly : y ∈ RF + , B∈F 0 :e∈B y B ≤ c(e) for all e ∈ E for every c : E → R+ . Proof: Since B L(E, F 0 ) = B L(B L(E, F)) = (E, F) it suffices to prove (a)⇒(c)⇒(b). The other implication (b)⇒(a) then follows by exchanging the roles of F and F 0 . (a)⇒(c): By Corollary 3.33 we have for every c : E → R+ o n E , min{c(A) : A ∈ F} = min{cx : x ∈ P} = min c(x + y) : x ∈ P, y ∈ R+ where P is the convex hull of the incidence vectors of elements of F. From this, the Max-Flow-Min-Cut property and the LP Duality Theorem 3.20 we get (c). (c)⇒(b): Let P 0 denote the convex hull of the incidence vectors of the elements of F 0 . We have to show that ( ) n o X E E : xe ≥ 1 for all A ∈ F . x + y : x ∈ P 0 , y ∈ R+ = x ∈ R+ e∈A

13.4 The Greedy Algorithm

317

Since “⊆” is trivial from the definition of blocking clutters we only show the other P E be a vector with inclusion. So let c ∈ R+ c e∈A e ≥ 1 for all A ∈ F. By (c) we have 1 ≤ min{c(A) : A ∈ F} ( F0

= max 1ly : y ∈ R+ ,

) X

y B ≤ c(e) for all e ∈ E ,

B∈F 0 :e∈B

F 0 be a vector with 1ly = 1 and P so let y ∈ RP + B∈F 0 :e∈B y B ≤ c(e) for all e ∈ E. defines a vector x ∈ P 0 with x ≤ c, proving Then xe := B∈F 0 :e∈B y B (e ∈ E) 0 E that c ∈ x + y : x ∈ P , y ∈ R+ .

For example, this theorem implies the Max-Flow-Min-Cut Theorem 8.6 quite directly: Let (G, u, s, t) be a network. By Exercise 1 of Chapter 7 the minimum length of an s-t-path in (G, u) equals the maximum number of s-t-cuts such that each edge e is contained in at most u(e) of them. Hence the clutter of s-t-paths (example (3) in the above list) has the Max-Flow-Min-Cut Property, and so has its blocking clutter. Now (c) applied to the clutter of minimal s-t-cuts implies the Max-Flow-Min-Cut Theorem. Note however that Theorem 13.18 does not guarantee an integral vector attaining the maximum in (c), even if c is integral. The clutter of T -joins for G = K 4 and T = V (G) shows that this does not exist in general.

13.4 The Greedy Algorithm Again, let (E, F) be an independence system and c : E → R+ . We consider the M AXIMIZATION P ROBLEM for (E, F, c) and formulate two “greedy algorithms”. We do not have to consider negative weights since elements with negative weight never appear in an optimum solution. We assume that (E, F) is given by an oracle. For the first algorithm we simply assume an independence oracle, i.e. an oracle which, given a set F ⊆ E, decides whether F ∈ F or not.

B EST-I N -G REEDY A LGORITHM Input: Output:

An independence system (E, F), given by an independence oracle. Weights c : E → R+ . A set F ∈ F.

1

Sort E = {e1 , e2 , . . . , en } such that c(e1 ) ≥ c(e2 ) ≥ · · · ≥ c(en ).

2

Set F := ∅.

3

For i := 1 to n do: If F ∪ {ei } ∈ F then set F := F ∪ {ei }.

318

13 Matroids

The second algorithm requires a more complicated oracle. Given a set F ⊆ E, this oracle decides whether F contains a basis. Let us call such an oracle a basissuperset oracle.

W ORST-O UT-G REEDY A LGORITHM Input: Output:

An independence system (E, F), given by a basis-superset oracle. Weights c : E → R+ . A basis F of (E, F).

1

Sort E = {e1 , e2 , . . . , en } such that c(e1 ) ≤ c(e2 ) ≤ · · · ≤ c(en ).

2

Set F := E.

3

For i := 1 to n do: If F \ {ei } contains a basis then set F := F \ {ei }.

Before we analyse these algorithms, let us take a closer look at the oracles required. It is an interesting question whether such oracles are polynomially equivalent, i.e. whether one can be simulated by a polynomial-time oracle algorithm using the other. The independence oracle and the basis-superset oracle do not seem to be polynomially equivalent: If we consider the independence system for the TSP (example (2) of the list in Section 13.1), it is easy (and the subject of Exercise 13) to decide whether a set of edges is independent, i.e. the subset of a Hamiltonian circuit (recall that we are working with a complete graph). On the other hand, it is a difficult problem to decide whether a set of edges contains a Hamiltonian circuit (this is NP-complete; cf. Theorem 15.25). Conversely, in the independence system for the S HORTEST PATH P ROBLEM (example (3)), it is easy to decide whether a set of edges contains an s-t-path. Here it is not known how to decide whether a given set is independent (i.e. subset of an s-t-path) in polynomial time (Korte and Monma [1979] proved NP-completeness). For matroids, both oracles are polynomially equivalent. Other equivalent oracles are the rank oracle and closure oracle, which return the rank and the closure of a given subset of E, respectively (Exercise 16). However, even for matroids there are other natural oracles that are not polynomially equivalent. For example, the oracle deciding whether a given set is a basis is weaker than the independence oracle. The oracle which for a given F ⊆ E returns the minimum cardinality of a dependent subset of F is stronger than the independence oracle (Hausmann and Korte [1981]). One can analogously formulate both greedy algorithms for the M INIMIZA TION P ROBLEM . It is easy to see that the B EST-I N -G REEDY for the M AXIMIZA TION P ROBLEM for (E, F, c) corresponds to the W ORST-O UT-G REEDY for the M INIMIZATION P ROBLEM for (E, F ∗ , c): adding an element to F in the B ESTI N -G REEDY corresponds to removing an element from F in the W ORST-O UTG REEDY. Observe that K RUSKAL ’ S A LGORITHM (see Section 6.1) is a B EST-I N G REEDY algorithm for the M INIMIZATION P ROBLEM in a cycle matroid.

13.4 The Greedy Algorithm

319

The rest of this section contains some results concerning the quality of a solution found by the greedy algorithms. Theorem 13.19. (Jenkyns [1976], Korte and Hausmann [1978]) Let (E, F) be an independence system. For c : E → R+ we denote by G(E, F, c) the cost of some solution found by the B EST-I N -G REEDY for the M AXIMIZATION P ROBLEM, and by OPT(E, F, c) the cost of an optimum solution. Then q(E, F) ≤

G(E, F, c) ≤ 1 OPT(E, F, c)

for all c : E → R+ . There is a cost function where the lower bound is attained. Proof: Let E = {e1 , e2 , . . . , en }, c : E → R+ , and c(e1 ) ≥ c(e2 ) ≥ . . . ≥ c(en ). Let G n be the solution found by the B EST-I N -G REEDY (when sorting E like this), while On is an optimum solution. We define E j := {e1 , . . . , e j }, G j := G n ∩ E j and O j := On ∩ E j ( j = 0, . . . , n). Set dn := c(en ) and d j := c(e j ) − c(e j+1 ) for j = 1, . . . , n − 1. Since O j ∈ F, we have |O j | ≤ r (E j ). Since G j is a basis of E j , we have |G j | ≥ ρ(E j ). With these two inequalities we conclude that c(G n ) =

=

≥

n X j=1 n X j=1 n X

(|G j | − |G j−1 |) c(e j )

|G j | d j ρ(E j ) d j

j=1

≥ q(E, F)

≥ q(E, F)

= q(E, F)

n X j=1 n X j=1 n X

r (E j ) d j

(13.1)

|O j | d j (|O j | − |O j−1 |) c(e j )

j=1

= q(E, F) c(On ). Finally we show that the lower bound is tight. Choose F ⊆ E and bases B1 , B2 of F such that |B1 | = q(E, F). |B2 | Define

320

13 Matroids

( c(e) :=

1 0

for e ∈ F for e ∈ E \ F

and sort e1 , . . . , en such that c(e1 ) ≥ c(e2 ) ≥ . . . ≥ c(en ) and B1 = {e1 , . . . , e|B1 | }. Then G(E, F, c) = |B1 | and OPT(E, F, c) = |B2 |, and the lower bound is attained. In particular we have the so-called Edmonds-Rado Theorem: Theorem 13.20. (Rado [1957], Edmonds [1971]) An independence system (E, F) is a matroid if and only if the B EST-I N -G REEDY finds an optimum solution for the M AXIMIZATION P ROBLEM for (E, F, c) for all cost functions c : E → R+ . Proof: By Theorem 13.19 we have q(E, F) < 1 if and only if there exists a cost function c : E → R+ for which the B EST-I N -G REEDY does not find an optimum solution. By Proposition 13.7 we have q(E, F) < 1 if and only if (E, F) is not a matroid. This is one of the rare cases where we can define a structure by its algorithmic behaviour. We also obtain a polyhedral description: Theorem 13.21. (Edmonds [1970]) Let (E, F) be a matroid and r : E → Z+ its rank function. Then the matroid polytope of (E, F), i.e. the convex hull of the incidence vectors of all elements of F, is equal to ) ( X E xe ≤ r (A) for all A ⊆ E . x ∈ R : x ≥ 0, e∈A

Proof: Obviously, this polytope contains all incidence vectors of independent sets. By Corollary 3.32 it remains to show that all vertices of this polytope are integral. By Theorem 5.13 this is equivalent to showing that ) ( X xe ≤ r (A) for all A ⊆ E (13.2) max cx : x ≥ 0, e∈A

has an integral optimum solution for any c : E → R. W.l.o.g. c(e) ≥ 0 for all e, = 0. since for e ∈ E with c(e) < 0 any optimum solution x of (13.2) has xe P Let x be an optimum solution of (13.2). In (13.1) we replace |O j | by e∈E j xe P ( j = 0, . . . , n). We obtain c(G n ) ≥ e∈E c(e)xe . So the B EST-I N -G REEDY produces a solution whose incidence vector is another optimum solution of (13.2). When applied to graphic matroids, this also yields Theorem 6.12. As in this special case, we also have total dual integrality in general. A generalization of this result will be proved in Section 14.2. The above observation that the B EST-I N -G REEDY for the M AXIMIZATION P ROBLEM for (E, F, c) corresponds to the W ORST-O UT-G REEDY for the M IN IMIZATION P ROBLEM for (E, F ∗ , c) suggests the following dual counterpart of Theorem 13.19:

13.4 The Greedy Algorithm

321

Theorem 13.22. (Korte and Monma [1979]) Let (E, F) be an independence system. For c : E → R+ let G(E, F, c) denote a solution found by the W ORSTO UT-G REEDY for the M INIMIZATION P ROBLEM. Then G(E, F, c) |F| − ρ ∗ (F) ≤ max F⊆E |F| − r ∗ (F) OPT(E, F, c)

1 ≤

(13.3)

for all c : E → R+ , where ρ ∗ and r ∗ are the rank functions of the dual independence system (E, F ∗ ). There is a cost function where the upper bound is attained. Proof: We use the same notation as in the proof of Theorem 13.19. By construction, G j ∪ (E \ E j ) contains a basis of E, but (G j ∪ (E \ E j )) \ {e} does not contain a basis of E for any e ∈ G j ( j = 1, . . . , n). In other words, E j \ G j is a basis of E j with respect to (E, F ∗ ), so |E j | − |G j | ≥ ρ ∗ (E j ). Since On ⊆ E \ (E j \ O j ) and On is a basis, E j \ O j is independent in (E, F ∗ ), so |E j | − |O j | ≤ r ∗ (E j ). We conclude that |G j | ≤ |E j | − ρ ∗ (E j ) |O j | ≥ |E j | − r ∗ (E j ).

and

Now the same calculation as (13.1) provides the upper bound. To see that this bound is tight, consider 1 for e ∈ F c(e) := , 0 for e ∈ E \ F where F ⊆ E is a set where the maximum in (13.3) is attained. Let B1 be a basis of F with respect to (E, F ∗ ), with |B1 | = ρ ∗ (F). If we sort e1 , . . . , en such that c(e1 ) ≥ c(e2 ) ≥ . . . ≥ c(en ) and B1 = {e1 , . . . , e|B1 | }, we have G(E, F, c) = |F| − |B1 | and OPT(E, F, c) = |F| − r ∗ (F).

1

2

M >> 2

Fig. 13.1.

If we apply the W ORST-O UT-G REEDY to the M AXIMIZATION P ROBLEM or the B EST-I N -G REEDY to the M INIMIZATION P ROBLEM, there is no positive G(E,F ,c) lower/ﬁnite upper bound for OPT(E, F ,c) . To see this, consider the problem of ﬁnding a minimal vertex cover of maximum weight or a maximal stable set of minimum weight in the simple graph shown in Figure 13.1. However in the case of matroids, it does not matter whether we use the B EST-I N G REEDY or the W ORST-O UT-G REEDY: since all bases have the same cardinality, the M INIMIZATION P ROBLEM for (E, F, c) is equivalent to the M AXIMIZATION

322

13 Matroids

P ROBLEM for (E, F, c0 ), where c0 (e) := M − c(e) for all e ∈ E and M := 1 + max{c(e) : e ∈ E}. Therefore K RUSKAL ’ S A LGORITHM (Section 6.1) solves the M INIMUM S PANNING T REE P ROBLEM optimally. The Edmonds-Rado Theorem 13.20 also yields the following characterization of optimum k-element solutions of the M AXIMIZATION P ROBLEM. Theorem 13.23. Let (E, F) be a matroid, c : E → R, k ∈ N and X ∈ F with |X | = k. Then c(X ) = max{c(Y ) : Y ∈ F, |Y | = k} if and only if the following two conditions hold: (a) For all y ∈ E \ X with X ∪ {y} ∈ / F and all x ∈ C(X, y) we have c(x) ≥ c(y); (b) For all y ∈ E \ X with X ∪ {y} ∈ F and all x ∈ X we have c(x) ≥ c(y). Proof: The necessity is trivial: if one of the conditions is violated for some y and x, the k-element set X 0 := (X ∪ {y}) \ {x} ∈ F has greater cost than X . To see the sufficiency, let F 0 := {F ∈ F : |F| ≤ k} and c0 (e) := c(e) + M for all e ∈ E, where M = max{|c(e)| : e ∈ E}. Sort E = {e1 , . . . , en } such that c0 (e1 ) ≥ · · · ≥ c0 (en ) and, for any i, c0 (ei ) = c0 (ei+1 ) and ei+1 ∈ X imply ei ∈ X (i.e. elements of X come first among those of equal weight). Let X 0 be the solution found by the B EST-I N -G REEDY for the instance (E, F 0 , c0 ) (sorted like this). Since (E, F 0 ) is a matroid, the Edmonds-Rado Theorem 13.20 implies: c(X 0 ) + k M = c0 (X 0 ) = max{c0 (Y ) : Y ∈ F 0 } = max{c(Y ) : Y ∈ F, |Y | = k} + k M. We conclude the proof by showing that X = X 0 . We know that |X | = k = |X 0 |. So suppose X 6= X 0 , and let ei ∈ X 0 \ X with i minimum. Then X ∩ {e1 , . . . , ei−1 } = / F, then (a) implies C(X, ei ) ⊆ X 0 , a X 0 ∩ {e1 , . . . , ei−1 }. Now if X ∪ {ei } ∈ contradiction. If X ∪ {ei } ∈ F, then (b) implies X ⊆ X 0 which is also impossible. We shall need this theorem in Section 13.7. The special case that (E, F) is a graphic matroid and k = r (E) is part of Theorem 6.2.

13.5 Matroid Intersection Definition 13.24. Given two independence systems (E, F1 ) and (E, F2 ), we define their intersection by (E, F1 ∩ F2 ). The intersection of a finite number of independence systems is defined analogously. It is clear that the result is again an independence system. Proposition 13.25. Any independence system (E, F) is the intersection of a finite number of matroids.

13.5 Matroid Intersection

323

Proof: Each circuit C of (E, F) defines a matroid (E, {F ⊆ E : C \ F 6= ∅}) by Theorem 13.12. The intersection of all these matroids is of course (E, F). Since the intersection of matroids is not a matroid in general, we cannot hope to get an optimum common independent set by a greedy algorithm. However, the following result, together with Theorem 13.19, implies a bound for the solution found by the B EST-I N -G REEDY: Proposition 13.26. If (E, F) is the intersection of p matroids, then q(E, F) ≥

1 p.

Proof: By Theorem 13.12(b), X ∪ {e} contains at most p circuits for any X ∈ F and e ∈ E. The statement now follows from Theorem 13.8. Of particular interest are independence systems that are the intersection of two matroids. The prime example here is the matching problem in a bipartite graph . G = (A ∪ B, E). If F := {F ⊆ E : F is a matching in G}, then (E, F) is the intersection of two matroids. Namely, let F1 := {F ⊆ E : |δ F (x)| ≤ 1 for all x ∈ A}

and

F2 := {F ⊆ E : |δ F (x)| ≤ 1 for all x ∈ B}. (E, F1 ), (E, F2 ) are matroids by Proposition 13.4(d). Clearly, F = F1 ∩ F2 . A second example is the independence system consisting of all branchings in a digraph G (Example 8 of the list at the beginning of Section 13.1). Here one matroid contains all sets of edges such that each vertex has at most one entering edge (see Proposition 13.4(e)), while the second matroid is the cycle matroid M(G) of the underlying undirected graph. We shall now describe Edmonds’ algorithm for the following problem:

M ATROID I NTERSECTION P ROBLEM Instance:

Two matroids (E, F1 ), (E, F2 ), given by independence oracles.

Task:

Find a set F ∈ F1 ∩ F2 such that |F| is maximum.

We start with the following lemma. Recall that, for X ∈ F and e ∈ E, C(X, e) denotes the unique circuit in X ∪ {e} if X ∪ {e} ∈ / F, and C(X, e) = ∅ otherwise. Lemma 13.27. (Frank [1981]) Let (E, F) be a matroid and X ∈ F. Let / X with x1 , . . . , xs ∈ X and y1 , . . . , ys ∈ (a) xk ∈ C(X, yk ) for k = 1, . . . , s and / C(X, yk ) for 1 ≤ j < k ≤ s. (b) x j ∈ Then (X \ {x1 , . . . , xs }) ∪ {y1 , . . . , ys } ∈ F. Proof: Let X r := (X \ {x1 , . . . , xr }) ∪ {y1 , . . . , yr }. We show that X r ∈ F for all r by induction. For r = 0 this is trivial. Let us assume that X r −1 ∈ F for some r ∈ {1, . . . , s}. If X r −1 ∪ {yr } ∈ F then we immediately have X r ∈ F.

324

13 Matroids

Otherwise X r −1 ∪ {yr } contains a unique circuit C (by Theorem 13.12(b)). Since C(X, yr ) ⊆ X r −1 ∪ {yr } (by (b)), we must have C = C(X, yr ). But then by (a) xr ∈ C(X, yr ) = C, so X r = (X r −1 ∪ {yr }) \ {xr } ∈ F. The idea behind E DMONDS ’ M ATROID I NTERSECTION A LGORITHM is the following. Starting with X = ∅, we augment X by one element in each iteration. Since in general we cannot hope for an element e such that X ∪ {e} ∈ F1 ∩ F2 , we shall look for “alternating paths”. To make this convenient, we deﬁne an auxiliary graph. We apply the notion C(X, e) to (E, Fi ) and write Ci (X, e) (i = 1, 2). E\X

X

SX A(2) X

A(1) X TX Fig. 13.2.

Given a set X ∈ F1 ∩ F2 , we deﬁne a directed auxiliary graph G X by AX

(1)

:= { (x, y) : y ∈ E \ X, x ∈ C1 (X, y) \ {y} },

(2) AX

:= { (y, x) : y ∈ E \ X, x ∈ C2 (X, y) \ {y} },

GX

:= (E, A X ∪ A X ).

(1)

(2)

We set SX

:= {y ∈ E \ X : X ∪ {y} ∈ F1 },

TX

:= {y ∈ E \ X : X ∪ {y} ∈ F2 }

(see Figure 13.2) and look for a shortest path from S X to TX . Such a path will enable us to augment the set X . (If S X ∩ TX = ∅, we have a path of length zero and we can augment X by any element in S X ∩ TX .)

13.5 Matroid Intersection

325

Lemma 13.28. Let X ∈ F1 ∩ F2 . Let y0 , x1 , y1 , . . . , xs , ys be the vertices of a shortest y0 -ys -path in G X (in this order), with y0 ∈ S X and ys ∈ TX . Then X 0 := (X ∪ {y0 , . . . , ys }) \ {x1 , . . . , xs } ∈ F1 ∩ F2 . Proof: First we show that X ∪ {y0 }, x1 , . . . , xs and y1 , . . . , ys satisfy the requirements of Lemma 13.27 with respect to F1 . Observe that X ∪ {y0 } ∈ F1 because (1) y0 ∈ S X . (a) is satisfied because (x j , y j ) ∈ A X for all j, and (b) is satisfied because otherwise the path could be shortcut. We conclude that X 0 ∈ F1 . Secondly, we show that X ∪ {ys }, xs , xs−1 , . . . , x1 and ys−1 , . . . , y1 , y0 satisfy the requirements of Lemma 13.27 with respect to F2 . Observe that X ∪ {ys } ∈ F2 (2) because ys ∈ TX . (a) is satisfied because (y j−1 , x j ) ∈ A X for all j, and (b) is satisfied because otherwise the path could be shortcut. We conclude that X 0 ∈ F2 . We shall now prove that if there exists no S X -TX -path in G X , then X is already maximum. We need the following simple fact: Proposition 13.29. Let (E, F1 ) and (E, F2 ) be two matroids with rank functions r1 and r2 . Then for any F ∈ F1 ∩ F2 and any Q ⊆ E we have |F| ≤ r1 (Q) + r2 (E \ Q). Proof: F ∩ Q ∈ F1 implies |F ∩ Q| ≤ r1 (Q). Similarly F \ Q ∈ F2 implies |F \ Q| ≤ r2 (E \ Q). Adding the two inequalities completes the proof. Lemma 13.30. X ∈ F1 ∩ F2 is maximum if and only if there is no S X -TX -path in G X . Proof: If there is an S X -TX -path, there is also a shortest one. We apply Lemma 13.28 and obtain a set X 0 ∈ F1 ∩ F2 of greater cardinality. Otherwise let R be the set of vertices reachable from S X in G X (see Figure 13.3). We have R ∩ TX = ∅. Let r1 and r2 be the rank function of F1 and F2 , respectively. We claim that r2 (R) = |X ∩ R|. If not, there would be a y ∈ R \ X with / F2 (because y ∈ / TX ), the circuit C2 (X, y) (X ∩ R) ∪ {y} ∈ F2 . Since X ∪ {y} ∈ (2) must contain an element x ∈ X \ R. But then (y, x) ∈ A X means that there is an edge leaving R. This contradicts the definition of R. Next we prove that r1 (E \ R) = |X \ R|. If not, there would be a y ∈ (E \ R)\ X / F1 (because y ∈ / S X ), the circuit C1 (X, y) with (X \ R)∪{y} ∈ F1 . Since X ∪{y} ∈ (1) must contain an element x ∈ X ∩ R. But then (x, y) ∈ A X means that there is an edge leaving R. This contradicts the definition of R. Altogether we have |X | = r2 (R)+r1 (E \ R). By Proposition 13.29, this implies optimality. The last paragraph of this proof yields the following min-max-equality:

326

13 Matroids E\X

X

SX A(2) X

R E\R A(1) X TX Fig. 13.3.

Theorem 13.31. (Edmonds [1970]) Let (E, F1 ) and (E, F2 ) be two matroids with rank functions r1 and r2 . Then max {|X | : X ∈ F1 ∩ F2 } = min {r1 (Q) + r2 (E \ Q) : Q ⊆ E} . We are now ready for a detailed description of the algorithm.

E DMONDS ’ M ATROID I NTERSECTION A LGORITHM Input:

Two matroids (E, F1 ) and (E, F2 ), given by independence oracles.

Output:

A set X ∈ F1 ∩ F2 of maximum cardinality.

1

Set X := ∅.

2

For each y ∈ E \ X and i ∈ {1, 2} do: Compute / Fi , (X ∪ {y}) \ {x} ∈ Fi }. Ci (X, y) := {x ∈ X ∪ {y} : X ∪ {y} ∈ Compute S X , TX , and G X as deﬁned above.

3 4 5

Apply BFS to ﬁnd a shortest S X -TX -path P in G X . If none exists then stop. Set X := X V (P) and go to . 2

Theorem 13.32. E DMONDS ’ M ATROID I NTERSECTION A LGORITHM correctly solves the M ATROID I NTERSECTION P ROBLEM in O(|E|3 θ ) time, where θ is the maximum complexity of the two independence oracles.

13.6 Matroid Partitioning

327

Proof: The correctness follows from Lemmata 13.28 and 13.30. 2 and 3 can be done in O(|E|2 θ ), 4 in O(|E|) time. Since there are at most |E| augmentations, the overall complexity is O(|E|3 θ). Faster matroid intersection algorithms are discussed by Cunningham [1986] and Gabow and Xu [1996]. We remark that the problem of finding a maximum cardinality set in the intersection of three matroids is an NP-hard problem; see Exercise 14(c) of Chapter 15.

13.6 Matroid Partitioning Instead of the intersection of matroids we now consider their union which is defined as follows: Definition 13.33. Let (E, F1 ), . . . , (E, Fk ) be k matroids. . A set X ⊆ E is called . partitionable if there exists a partition X = X 1 ∪ · · · ∪ X k with X i ∈ Fi for i = 1, . . . , k. Let F be the family of partitionable subsets of E. Then (E, F) is called the union or sum of (E, F1 ), . . . , (E, Fk ). We shall prove that the union of matroids is a matroid again. Moreover, we solve the following problem via matroid intersection:

M ATROID PARTITIONING P ROBLEM Instance: Task:

A number k ∈ N, k matroids (E, F1 ), . . . , (E, Fk ), given by independence oracles. Find a partitionable set X ⊆ E of maximum cardinality.

The main theorem with respect to matroid partitioning is: Theorem 13.34. (Nash-Williams [1967]) Let (E, F1 ), . . . , (E, Fk ) be matroids F) is a mawith rank functions r1 , . . . , rk , and let (E, F) be their union. Then (E, Pk ri (A) . troid, and its rank function r is given by r (X ) = min A⊆X |X \ A| + i=1 Proof: (E, F) is an independence system. Let X ⊆ E. We first prove obviouslyP k ri (A) . r (X ) = min A⊆X |X \ A| + i=1 .

.

For any Y ⊆ X such that Y is partitionable, i.e. Y = Y1 ∪ · · · ∪ Yk with Yi ∈ Fi (i = 1, . . . , k), and any A ⊆ X we have |Y | = |Y \ A| + |Y ∩ A| ≤ |X \ A| +

k X i=1

|Yi ∩ A| ≤ |X \ A| +

k X

ri (A),

i=1

Pk ri (A) . so r (X ) ≤ min A⊆X |X \ A| + i=1 On the other hand, let X 0 := X × {1, . . . , k}. We define two matroids on X 0 . For Q ⊆ X 0 and i ∈ {1, . . . , k} we write Q i := {e ∈ X : (e, i) ∈ Q}. Let

328

13 Matroids

I1 := {Q ⊆ X 0 : Q i ∈ Fi for all i = 1, . . . , k} and I2 := {Q ⊆ X 0 : Q i ∩ Q j = ∅ for all i 6= j}. Evidently, both (X 0 , I1 ) and (X 0 , I2 ) are matroids, S and their rank functions are Pk k Q i for Q ⊆ X 0 . given by s1 (Q) := i=1 ri (Q i ) and s2 (Q) := i=1 Now the family of partitionable subsets of X can be written as {A ⊆ X : there is a function f : A → {1, . . . , k} with {(e, f (e)) : e ∈ A} ∈ I1 ∩ I2 }. So the maximum cardinality of a partitionable set is the maximum cardinality of a common independent 13.31 this maximum set in I1 and I2 . By Theorem cardinality equals min s1 (Q) + s2 (X 0 \ Q) : Q ⊆ X 0 . If Q ⊆ X 0 attains this minimum, then for A := Q 1 ∩ · · · ∩ Q k we have k k k X \ X 0 r (X ) = s1 (Q) + s2 (X \ Q) = ri (Q i ) + X \ Qi ≥ ri (A) + |X \ A|. i=1

i=1

i=1

Pk So we have found a set A ⊆ X with i=1 ri (A) + |X \ A| ≤ r (X ). Having proved the formula for the rank function r , we finally show that r is submodular. By Theorem 13.10, this implies that (E, F) is a matroid. To show the Pk let A ⊆ X , B ⊆ Y with r (X ) = |X \ A| + Pk submodularity, let X, Y ⊆ E, and i=1 ri (A) and r (Y ) = |Y \ B| + i=1 ri (B). Then r (X ) + r (Y ) = |X \ A| + |Y \ B| +

k X

(ri (A) + ri (B))

i=1

≥ |(X ∪ Y ) \ (A ∪ B)| + |(X ∩ Y ) \ (A ∩ B)| +

k X

(ri (A ∪ B) + ri (A ∩ B))

i=1

≥ r (X ∪ Y ) + r (X ∩ Y ). The construction in the above proof (Edmonds [1970]) reduces the M ATROID PARTITIONING P ROBLEM to the M ATROID I NTERSECTION P ROBLEM. A reduction in the other direction is also possible (Exercise 20), so both problems can be regarded as equivalent. Note that we can find a maximum independent set in the union of an arbitrary number of matroids efficiently, while the intersection of more than two matroids is intractable.

13.7 Weighted Matroid Intersection

329

13.7 Weighted Matroid Intersection We now consider a generalization of the above algorithm to the weighted case.

W EIGHTED M ATROID I NTERSECTION P ROBLEM Instance: Task:

Two matroids (E, F1 ) and (E, F2 ), given by independence oracles. Weights c : E → R. Find a set X ∈ F1 ∩ F2 whose weight c(X ) is maximum.

We shall describe a primal-dual algorithm due to Frank [1981] for this problem. It generalizes E DMONDS ’ M ATROID I NTERSECTION A LGORITHM. Again we start with X := X 0 = ∅ and increase the cardinality in each iteration by one. We obtain sets X 0 , . . . , X m ∈ F1 ∩ F2 with |X k | = k (k = 0, . . . , m) and m = max{|X | : X ∈ F1 ∩ F2 }. Each X k will be optimum, i.e. c(X k ) = max{c(X ) : X ∈ F1 ∩ F2 , |X | = k}.

(13.4)

Hence at the end we just choose the optimum set among X 0 , . . . , X m . The main idea is to split up the weight function. At any stage we have two functions c1 , c2 : E → R with c1 (e) + c2 (e) = c(e) for all e ∈ E. For each k we shall guarantee ci (X k ) = max{ci (X ) : X ∈ Fi , |X | = k}

(i = 1, 2).

(13.5)

This condition obviously implies (13.4). To obtain (13.5) we use the optimality criterion of Theorem 13.23. Instead of G X , S X and TX only a subgraph G¯ and ¯ T¯ are considered. subsets S,

W EIGHTED M ATROID I NTERSECTION A LGORITHM Input: Output:

Two matroids (E, F1 ) and (E, F2 ), given by independence oracles. Weights c : E → R. A set X ∈ F1 ∩ F2 of maximum weight.

1

Set k := 0 and X 0 := ∅. Set c1 (e) := c(e) and c2 (e) := 0 for all e ∈ E.

2

For each y ∈ E \ X k and i ∈ {1, 2} do: Compute / Fi , (X k ∪ {y}) \ {x} ∈ Fi }. Ci (X k , y) := {x ∈ X k ∪ {y} : X k ∪ {y} ∈ Compute

3

A(1) := { (x, y) : y ∈ E \ X k , x ∈ C1 (X k , y) \ {y} }, A(2) := { (y, x) : y ∈ E \ X k , x ∈ C2 (X k , y) \ {y} }, S := { y ∈ E \ X k : X k ∪ {y} ∈ F1 }, T

:= { y ∈ E \ X k : X k ∪ {y} ∈ F2 }.

330

4

13 Matroids

Compute m 1 := max{c1 (y) : y ∈ S} m 2 := max{c2 (y) : y ∈ T } S¯ := { y ∈ S : c1 (y) = m 1 } T¯ := { y ∈ T : c2 (y) = m 2 } A¯ (1) := { (x, y) ∈ A(1) : c1 (x) = c1 (y) }, A¯ (2) := { (y, x) ∈ A(2) : c2 (x) = c2 (y) }, G¯ := (E, A¯ (1) ∪ A¯ (2) ).

5

6

7

¯ Apply BFS to compute the set R of vertices reachable from S¯ in G. ¯ T¯ -path P in G¯ with a minimum number If R ∩ T¯ 6= ∅ then: Find an Sof edges, set X k+1 := X k 4V (P) and k := k + 1 and go to . 2 Compute ε1 := min{c1 (x) − c1 (y) : (x, y) ∈ A(1) ∩ δ + (R)}; ε2 := min{c2 (x) − c2 (y) : (y, x) ∈ A(2) ∩ δ + (R)}; ε3 := min{m 1 − c1 (y) : y ∈ S \ R}; ε4 := min{m 2 − c2 (y) : y ∈ T ∩ R}; ε := min{ε1 , ε2 , ε3 , ε4 }

8

(where min ∅ := ∞). If ε < ∞ then: Set c1 (x) := c1 (x) − ε and c2 (x) := c2 (x) + ε for all x ∈ R. Go to . 4 If ε = ∞ then: Among X 0 , X 1 , . . . , X k , let X be the one with maximum weight. Stop. See Edmonds [1979] and Lawler [1976] for earlier versions of this algorithm.

Theorem 13.35. (Frank [1981]) The W EIGHTED M ATROID I NTERSECTION A LGORITHM correctly solves the W EIGHTED M ATROID I NTERSECTION P ROB LEM in O(|E|4 + |E|3 θ ) time, where θ is the maximum complexity of the two independence oracles. Proof: Let m be the final value of k. The algorithm computes sets X 0 , X 1 , . . . , X m . We first prove that X k ∈ F1 ∩ F2 for k = 0, . . . , m, by induction on k. This is trivial for k = 0. If we are working with X k ∈ F1 ∩ F2 for some k, G¯ is a subgraph of (E, A(1) ∪ A(2) ) = G X k . So if a path P is found in , 5 Lemma 13.28 ensures that X k+1 ∈ F1 ∩ F2 . When the algorithm stops, we have ε1 = ε2 = ε3 = ε4 = ∞, so T is not reachable from S in G X m . Then by Lemma 13.30 m = |X m | = max{|X | : X ∈ F1 ∩ F2 }.

13.7 Weighted Matroid Intersection

331

To prove correctness, we show that for k = 0, . . . , m, c(X k ) = max{c(X ) : X ∈ F1 ∩ F2 , |X | = k}. Since we always have c = c1 + c2 , it suffices to prove that at any stage of the algorithm (13.5) holds. This is clearly true when the algorithm starts (for k = 0); we show that (13.5) is never violated. We use Theorem 13.23. When we set X k+1 := X k 4V (P) in 6 we have to check that (13.5) holds. Let P ¯ t ∈ T¯ . By definition of G¯ we have c1 (X k+1 ) = c1 (X k )+c1 (s) be an s-t-path, s ∈ S, and c2 (X k+1 ) = c2 (X k ) + c2 (t). Since X k satisfies (13.5), conditions (a) and (b) of Theorem 13.23 must hold with respect to X k and each of F1 and F2 . By definition of S¯ both conditions continue to hold for X k ∪{s} and F1 . Therefore c1 (X k+1 ) = c1 (X k ∪ {s}) = max{c1 (Y ) : Y ∈ F1 , |Y | = k + 1}. Moreover, by definition of T¯ , (a) and (b) of Theorem 13.23 continue to hold for X k ∪ {t} and F2 , implying c2 (X k+1 ) = c2 (X k ∪ {t}) = max{c2 (Y ) : Y ∈ F2 , |Y | = k + 1}. In other words, (13.5) indeed holds for X k+1 . Now suppose we change c1 and c2 in . 8 We first show that ε > 0. By (13.5) and Theorem 13.23 we have c1 (x) ≥ c1 (y) for all y ∈ E \ X k and x ∈ C1 (X k , y) \ {y}. So for any (x, y) ∈ A(1) we have c1 (x) ≥ c1 (y). Moreover, by the definition of R no edge (x, y) ∈ δ + (R) belongs to A¯ (1) . This implies ε1 > 0. ε2 > 0 is proved analogously. m 1 ≥ c1 (y) holds for all y ∈ S. If in addition ¯ so m 1 > c1 (y). Therefore ε3 > 0. Similarly, ε4 > 0 (using y ∈ / R then y ∈ / S, T¯ ∩ R = ∅). We conclude that ε > 0. We can now prove that 8 preserves (13.5). Let c10 be the modified c1 , i.e. ( c1 (x) − ε if x ∈ R 0 . c1 (x) := if x ∈ / R c1 (x) We prove that X k and c10 satisfy the conditions of Theorem 13.23 with respect to F1 . To prove (a), let y ∈ E \ X k and x ∈ C1 (X k , y) \ {y}. Suppose c10 (x) < c10 (y). / R. Since also Since c1 (x) ≥ c1 (y) and ε > 0, we must have x ∈ R and y ∈ (x, y) ∈ A(1) , we have ε ≤ ε1 ≤ c1 (x)−c1 (y) = (c10 (x)+ε)−c10 (y), a contradiction. To prove (b), let x ∈ X k and y ∈ E \ X k with X k ∪ {y} ∈ F1 . Now suppose / R. Since c10 (y) > c10 (x). Since c1 (y) ≤ m 1 ≤ c1 (x), we must have x ∈ R and y ∈ y ∈ S we have ε ≤ ε3 ≤ m 1 − c1 (y) ≤ c1 (x) − c1 (y) = (c10 (x) + ε) − c10 (y), a contradiction. Let c20 be the modified c2 , i.e. ( c2 (x) + ε if x ∈ R . c20 (x) := if x ∈ / R c2 (x) We show that X k and c20 satisfy the conditions of Theorem 13.23 with respect to F2 . To prove (a), let y ∈ E \ X k and x ∈ C2 (X k , y) \ {y}. Suppose c20 (x) < c20 (y). Since c2 (x) ≥ c2 (y), we must have y ∈ R and x ∈ / R. Since also (y, x) ∈ A(2) , 0 0 we have ε ≤ ε2 ≤ c2 (x) − c2 (y) = c2 (x) − (c2 (y) − ε), a contradiction.

332

13 Matroids

To prove (b), let x ∈ X k and y ∈ E \ X k with X k ∪ {y} ∈ F2 . Now suppose c20 (y) > c20 (x). Since c2 (y) ≤ m 2 ≤ c2 (x), we must have y ∈ R and x ∈ / R. Since y ∈ T we have ε ≤ ε4 ≤ m 2 − c2 (y) ≤ c2 (x) − c2 (y) = c20 (x) − (c20 (y) − ε), a contradiction. So we have proved that (13.5) is not violated during , 8 and thus the algorithm works correctly. We now consider the running time. Observe that after an update of the weights ¯ T¯ , and R, as computed subsequently in in , 8 the new sets S, 4 and , 5 are ¯ T¯ , and R, respectively. If ε = ε4 < ∞, an augmentation supersets of the old S, (increase of k) follows. Otherwise the cardinality of R increases immediately (in

) 5 by at least one. So 4 – 8 are repeated less than |E| times between two augmentations. Since the running time of 4 – 8 is O(|E|2 ), the total running time between 3 two augmentations is O(|E| ) plus O(|E|2 ) oracle calls (in ). 2 Since there are m ≤ |E| augmentations, the stated overall running time follows. The running time can easily be improved to O(|E|3 θ) (Exercise 22).

Exercises 1. Prove that all the independence systems apart from (5) and (6) in the list at the beginning of Section 13.1 are – in general – not matroids. 2. Show that the uniform matroid with four elements and rank 2 is not a graphic matroid. 3. Prove that every graphic matroid is representable over every field. 4. Let G be an undirected graph, K ∈ N, and let F contain those subsets of E(G) that are the union of K forests. Prove that (E(G), F) is a matroid. 5. Compute tight lower bounds for the rank quotients of the independence systems listed at the beginning of Section 13.1. 6. Let S be a family of sets. A set T is a transversal of S if there is a bijection 8 : T → S with t ∈ 8(t) for all t ∈ T . (For a necessary and sufficient condition for the existence of a transversal, see Exercise 6 of Chapter 10.) Assume that S has a transversal. Prove that the family of transversals of S is the family of bases of a matroid. 7. Let E be a finite set and B ⊆ 2 E . Show that B is the set of bases of some matroid (E, F) if and only if the following holds: (B1) B 6= ∅; (B2) For any B1 , B2 ∈ B and y ∈ B2 \ B1 there exists an x ∈ B1 \ B2 with (B1 \ {x}) ∪ {y} ∈ B. 8. Let G be a graph. Let F be the family of sets X ⊆ V (G), for which a maximum matching exists that covers no vertex in X . Prove that (V (G), F) is a matroid. What is the dual matroid?

Exercises

333

9. Show that M(G ∗ ) = (M(G))∗ also holds for disconnected graphs G, extending Theorem 13.16. Hint: Use Exercise 31(a) of Chapter 2. 10. Show that the clutters in (3) and (6) in the list of Section 13.3 have the MaxFlow-Min-Cut property. (Use Theorem 19.10.) Show that the clutters in (1), (4) and (5) do not have the Max-Flow-Min-Cut property in general. ∗ 11. A clutter (E, F) is called binary if for all X 1 , . . . , X k ∈ F with k odd there exists a Y ∈ F with Y ⊆ X 1 4 · · · 4X k . Prove that the clutter of minimal T -joins and the clutter of minimal T -cuts (example (7) of the list in Section 13.3) are binary. Prove that a clutter is binary if and only if |A ∩ B| is odd for all A ∈ F and all B ∈ F 0 , where (E, F 0 ) is the blocking clutter. Conclude that a clutter is binary if and only if its blocking clutter is binary. Note: Seymour [1977] classified the binary clutters with the Max-Flow-MinCut property. ∗ 12. Let P be a polyhedron of blocking type, i.e. we have x + y ∈ P for all x ∈ P and y ≥ 0. The blocking polyhedron of P is defined to be B(P) := {z : z > x ≥ 1 for all x ∈ P}. Prove that B(P) is again a polyhedron of blocking type and that B(B(P)) = P. Note: Compare this with Theorem 4.22. 13. How can one check (in polynomial time) whether a given set of edges of a complete graph G is a subset of some Hamiltonian circuit in G? 14. Prove that if (E, F) is a matroid, then the B EST-I N -G REEDY maximizes any bottleneck function c(F) = min{ce : e ∈ F} over the bases. 15. Let (E, F) be a matroid and c : E → R such that c(e) 6= c(e0 ) for all e 6= e0 and c(e) 6= 0 for all e. Prove that both the M AXIMIZATION and the M INIMIZATION P ROBLEM for (E, F, c) have a unique optimum solution. ∗ 16. Prove that for matroids the independence, basis-superset, closure and rank oracles are polynomially equivalent. Hint: To show that the rank oracle reduces to the independence oracle, use the B EST-I N -G REEDY. To show that the independence oracle reduces to the basis-superset oracle, use the W ORST-O UT-G REEDY. (Hausmann and Korte [1981]) 17. Given an undirected graph G, we wish to colour the edges with a minimum number of colours such that for any circuit C of G, the edges of C do not all have the same colour. Show that there is a polynomial-time algorithm for this problem. 18. Let (E, F1 ), . . . , (E, Fk ) be matroids with rank functionsPr1 , . . . , rk . Prove k ri (A) for all that a set X ⊆ E is partitionable if and only if |A| ≤ i=1 A ⊆ X . Show that Theorem 6.19 is a special case. (Edmonds and Fulkerson [1965]) 19. Let (E, F) be a matroid with rank function r . Prove (using Theorem 13.34): (a) (E, F) has k pairwise disjoint bases if and only if kr (A)+|E \ A| ≥ kr (E) for all A ⊆ E.

334

20.

21.

22. 23.

13 Matroids

(b) (E, F) has k independent sets whose union is E if and only if kr (A) ≥ |A| for all A ⊆ E. Show that Theorem 6.19 and Theorem 6.16 are special cases. Let (E, F1 ) and (E, F2 ) be two matroids. Let X be a maximal partitionable . subset with respect to (E, F1 ) and (E, F2∗ ): X = X 1 ∪ X 2 with X 1 ∈ F1 and X 2 ∈ F2∗ . Let B2 ⊇ X 2 be a basis of F2∗ . Prove that then X \ B2 is a maximum-cardinality set in F1 ∩ F2 . (Edmonds [1970]) Let (E, S) be a set system, and let (E, F) be a matroid with rank function r. S Show that S has a transversal that is independent in (E, F) if and only if r B∈B B ≥ |B| for all B ⊆ S. Hint: First describe the rank function of the matroid whose independent sets are all transversals (Exercise 6), using Theorem 13.34. Then apply Theorem 13.31. (Rado [1942]) Show that the running time of the W EIGHTED M ATROID I NTERSECTION A LGORITHM (cf. Theorem 13.35) can be improved to O(|E|3 θ ). Let (E, F1 ) and (E, F2 ) be two matroids, and c : E → R. Let X 0 , . . . , X m ∈ F1 ∩ F2 with |X k | = k and c(X k ) = max{c(X ) : X ∈ F1 ∩ F2 , |X | = k} for all k. Prove that for k = 1, . . . , m − 2 c(X k+1 ) − c(X k ) ≤ c(X k ) − c(X k−1 ).

(Krogdahl [unpublished]) 24. Consider the following problem. Given a digraph G with edge weights, a vertex s ∈ V (G), and a number K , find a minimum weight subgraph H of G containing K edge-disjoint paths from s to each other vertex. Show that this reduces to the W EIGHTED M ATROID I NTERSECTION P ROBLEM. Hint: See Exercise 19 of Chapter 6 and Exercise 4 of this chapter. (Edmonds [1970]; Frank and Tardos [1989]; Gabow [1995]) . 25. Let A and B be two finite sets of cardinality n ∈ N, G = (A ∪ B, {{a, b} : a ∈ A, b ∈ B}) the complete bipartite graph, a¯ ∈ A, and c : E(G) → R a cost function. Let T be the family of edge sets of all spanning trees T in G with |δT (a)| = 2 for all a ∈ A \ {a}. ¯ Show that a minimum cost element of ¯ T can be computed in O(n 7 ) time. How many edges will be incident to a?

References General Literature: Bixby, R.E., and Cunningham, W.H. [1995]: Matroid optimization and algorithms. In: Handbook of Combinatorics; Vol. 1 (R.L. Graham, M. Gr¨otschel, L. Lov´asz, eds.), Elsevier, Amsterdam, 1995 Cook, W.J., Cunningham, W.H., Pulleyblank, W.R., and Schrijver, A. [1998]: Combinatorial Optimization. Wiley, New York 1998, Chapter 8

References

335

Faigle, U. [1987]: Matroids in combinatorial optimization. In: Combinatorial Geometries (N. White, ed.), Cambridge University Press, 1987 Gondran, M., and Minoux, M. [1984]: Graphs and Algorithms. Wiley, Chichester 1984, Chapter 9 Lawler, E.L. [1976]: Combinatorial Optimization; Networks and Matroids. Holt, Rinehart and Winston, New York 1976, Chapters 7 and 8 Oxley, J.G. [1992]: Matroid Theory. Oxford University Press, Oxford 1992 von Randow, R. [1975]: Introduction to the Theory of Matroids. Springer, Berlin 1975 Recski, A. [1989]: Matroid Theory and its Applications. Springer, Berlin, 1989 Schrijver, A. [2003]: Combinatorial Optimization: Polyhedra and Efficiency. Springer, Berlin 2003, Chapters 39–42 Welsh, D.J.A. [1976]: Matroid Theory. Academic Press, London 1976

Cited References: Cunningham, W.H. [1986] : Improved bounds for matroid partition and intersection algorithms. SIAM Journal on Computing 15 (1986), 948–957 Edmonds, J. [1970]: Submodular functions, matroids and certain polyhedra. In: Combinatorial Structures and Their Applications; Proceedings of the Calgary International Conference on Combinatorial Structures and Their Applications 1969 (R. Guy, H. Hanani, N. Sauer, J. Schonheim, eds.), Gordon and Breach, New York 1970, pp. 69–87 Edmonds, J. [1971]: Matroids and the greedy algorithm. Mathematical Programming 1 (1971), 127–136 Edmonds, J. [1979]: Matroid intersection. In: Discrete Optimization I; Annals of Discrete Mathematics 4 (P.L. Hammer, E.L. Johnson, B.H. Korte, eds.), North-Holland, Amsterdam 1979, pp. 39–49 Edmonds, J., and Fulkerson, D.R. [1965]: Transversals and matroid partition. Journal of Research of the National Bureau of Standards B 69 (1965), 67–72 Frank, A. [1981]: A weighted matroid intersection algorithm. Journal of Algorithms 2 (1981), 328–336 ´ [1989]: An application of submodular flows. Linear Algebra and Frank, A., and Tardos, E. Its Applications 114/115 (1989), 329–348 Fulkerson, D.R. [1971]: Blocking and anti-blocking pairs of polyhedra. Mathematical Programming 1 (1971), 168–194 Gabow, H.N. [1995]: A matroid approach to finding edge connectivity and packing arborescences. Journal of Computer and System Sciences 50 (1995), 259–273 Gabow, H.N., and Xu, Y. [1996]: Efficient theoretic and practical algorithms for linear matroid intersection problems. Journal of Computer and System Sciences 53 (1996), 129–147 Hausmann, D., Jenkyns, T.A., and Korte, B. [1980]: Worst case analysis of greedy type algorithms for independence systems. Mathematical Programming Study 12 (1980), 120–131 Hausmann, D., and Korte, B. [1981]: Algorithmic versus axiomatic definitions of matroids. Mathematical Programming Study 14 (1981), 98–111 Jenkyns, T.A. [1976]: The efficiency of the greedy algorithm. Proceedings of the 7th S-E Conference on Combinatorics, Graph Theory, and Computing, Utilitas Mathematica, Winnipeg 1976, pp. 341–350 Korte, B., and Hausmann, D. [1978]: An analysis of the greedy algorithm for independence systems. In: Algorithmic Aspects of Combinatorics; Annals of Discrete Mathematics 2 (B. Alspach, P. Hell, D.J. Miller, eds.), North-Holland, Amsterdam 1978, pp. 65–74

336

13 Matroids

Korte, B., and Monma, C.L. [1979]: Some remarks on a classification of oracle-type algorithms. In: Numerische Methoden bei graphentheoretischen und kombinatorischen Problemen; Band 2 (L. Collatz, G. Meinardus, W. Wetterling, eds.), Birkh¨auser, Basel 1979, pp. 195–215 Lehman, A. [1979]: On the width-length inequality. Mathematical Programming 17 (1979), 403–417 Nash-Williams, C.S.J.A. [1967]: An application of matroids to graph theory. In: Theory of Graphs; Proceedings of an International Symposium in Rome 1966 (P. Rosenstiehl, ed.), Gordon and Breach, New York, 1967, pp. 263–265 Rado, R. [1942]: A theorem on independence relations. Quarterly Journal of Math. Oxford 13 (1942), 83–89 Rado, R. [1957]: Note on independence functions. Proceedings of the London Mathematical Society 7 (1957), 300–320 Seymour, P.D. [1977]: The matroids with the Max-Flow Min-Cut property. Journal of Combinatorial Theory B 23 (1977), 189–222 Whitney, H. [1933]: Planar graphs. Fundamenta Mathematicae 21 (1933), 73–84 Whitney, H. [1935]: On the abstract properties of linear dependence. American Journal of Mathematics 57 (1935), 509–533

14 Generalizations of Matroids

There are several interesting generalizations of matroids. We have already seen independence systems in Section 13.1, which arose from dropping the axiom (M3). In Section 14.1 we consider greedoids, arising by dropping (M2) instead. Moreover, certain polytopes related to matroids and to submodular functions, called polymatroids, lead to strong generalizations of important theorems; we shall discuss them in Section 14.2. In Sections 14.3 and 14.4 we consider two approaches to the problem of minimizing an arbitrary submodular function: one using the E LLIPSOID M ETHOD, and one with a combinatorial algorithm. For the important special case of symmetric submodular functions we mention a simpler algorithm in Section 14.5.

14.1 Greedoids By definition, set systems (E, F) are matroids if and only if they satisfy (M1) ∅ ∈ F; (M2) If X ⊆ Y ∈ F then X ∈ F; (M3) If X, Y ∈ F and |X | > |Y |, then there is an x ∈ X \ Y with Y ∪ {x} ∈ F. If we drop (M3), we obtain independence systems, discussed in Sections 13.1 and 13.4. Now we drop (M2) instead: Definition 14.1. A greedoid is a set system (E, F) satisfying (M1) and (M3). Instead of the subclusiveness (M2) we have accessibility: We call a set system (E, F) accessible if ∅ ∈ F and for any X ∈ F \ {∅} there exists an x ∈ X with X \ {x} ∈ F. Greedoids are accessible (accessibility follows directly from (M1) and (M3)). Though more general than matroids, they comprise a rich structure and, on the other hand, generalize many different, seemingly unrelated concepts. We start with the following result: Theorem 14.2. Let (E, F) be an accessible set system. The following statements are equivalent: (a) For any X ⊆ Y ⊂ E and z ∈ E \ Y with X ∪ {z} ∈ F and Y ∈ F we have Y ∪ {z} ∈ F;

338

14 Generalizations of Matroids

(b) F is closed under union. Proof: (a) ⇒(b): Let X, Y ∈ F; we show that X ∪ Y ∈ F. Let Z be a maximal set with Z ∈ F and X ⊆ Z ⊆ X ∪ Y . Suppose Y \ Z 6= ∅. By repeatedly applying accessibility to Y we get a set Y 0 ∈ F with Y 0 ⊆ Z and an element y ∈ Y \ Z with Y 0 ∪ {y} ∈ F. We apply (a) to Z , Y 0 and y and obtain Z ∪ {y} ∈ F, contradicting the choice of Z . (b) ⇒(a) is trivial. If the conditions in Theorem 14.2 hold, then (E, F) is called an antimatroid. Proposition 14.3. Every antimatroid is a greedoid. Proof: Let (E, F) be an antimatroid, i.e. accessible and closed under union. To prove (M3), let X, Y ∈ F with |X | > |Y |. Since (E, F) is accessible there is an order X = {x1 , . . . , xn } with {x1 , . . . , xi } ∈ F for i = 0, . . . , n. Let i ∈ {1, . . . , n} / Y ; then Y ∪ {xi } = Y ∪ {x1 , . . . , xi } ∈ F (since be the minimum index with xi ∈ F is closed under union). Another equivalent definition of antimatroids is by a closure operator: Proposition 14.4. Let (E, F) be a set system such that F is closed under union and ∅ ∈ F. Define \ τ (A) := {X ⊆ E : A ⊆ X, E \ X ∈ F} Then τ is a closure operator, i.e. satisfies (S1)–(S3) of Theorem 13.11. Proof: Let X ⊆ Y ⊆ E. X ⊆ τ (X ) ⊆ τ (Y ) is trivial. To prove (S3), assume that there exists a y ∈ τ (τ (X )) \ τ (X ). Then y ∈ Y for all Y ⊆ E with τ (X ) ⊆ Y and E \ Y ∈ F, but there exists a Z ⊆ E \ {y} with X ⊆ Z and E \ Z ∈ F. This implies τ (X ) 6⊆ Z , a contradiction. Theorem 14.5. Let (E, F) be a set system such that F is closed under union and ∅ ∈ F. Then (E, F) is accessible if and only if the closure operator τ of Proposition 14.4 satisfies the anti-exchange property: if X ⊆ E, y, z ∈ E \ τ (X ), y 6= z and z ∈ τ (X ∪ {y}), then y ∈ / τ (X ∪ {z}). Proof: If (E, F) is accessible, then (M3) holds by Proposition 14.3. To show the anti-exchange property, let X ⊆ E, B := E \ τ (X ), and y, z ∈ B with z∈ / A := E \ τ (X ∪ {y}). Observe that A ∈ F, B ∈ F and A ⊆ B \ {y, z}. By applying (M3) to A and B we get an element b ∈ B \ A ⊆ E \ (X ∪ A) with A ∪ {b} ∈ F. A ∪ {b} cannot be a subset of E \ (X ∪ {y}) (otherwise τ (X ∪ {y}) ⊆ E \ (A ∪ {b}), contradicting τ (X ∪ {y}) = E \ A). Hence b = y. So we have A ∪ {y} ∈ F and thus τ (X ∪ {z}) ⊆ E \ (A ∪ {y}). We have proved y∈ / τ (X ∪ {z}). To show the converse, let A ∈ F \ {∅} and let X := E \ A. We have τ (X ) = X . Let a ∈ A such that |τ (X ∪ {a})| is minimum. We claim that τ (X ∪ {a}) = X ∪ {a}, i.e. A \ {a} ∈ F.

14.1 Greedoids

339

Suppose, on the contrary, that b ∈ τ (X ∪ {a}) \ (X ∪ {a}). By (c) we have a∈ / τ (X ∪ {b}). Moreover, τ (X ∪ {b}) ⊆ τ (τ (X ∪ {a}) ∪ {b}) = τ (τ (X ∪ {a})) = τ (X ∪ {a}). Hence τ (X ∪ {b}) is a proper subset of τ (X ∪ {a}), contradicting the choice of a. The anti-exchange property of Theorem 14.5 is different from (S4). While (S4) of Theorem 13.11 is a property of linear hulls in Rn , this is a property of convex hulls / conv(X ∪{z}). in Rn : if y 6= z, z 6∈ conv(X ) and z ∈ conv(X ∪{y}), then clearly y ∈ So for any finite set E ⊂ Rn , (E, {X ⊆ E : X ∩conv(E \ X ) = ∅}) is an antimatroid. Greedoids generalize matroids and antimatroids, but they also contain other interesting structures. One example is the blossom structure we used in E DMONDS ’ C ARDINALITY M ATCHING A LGORITHM (Exercise 1). Another basic example is: Proposition 14.6. Let G be a graph (directed or undirected) and r ∈ V (G). Let F be the family of all edge sets of arborescences in G rooted at r , or trees in G containing r (not necessarily spanning). Then (E(G), F) is a greedoid. Proof: (M1) is trivial. We prove (M3) for the directed case; the same argument applies to the undirected case. Let (X 1 , F1 ) and (X 2 , F2 ) be two arborescences in G rooted at r with |F1 | > |F2 |. Then |X 1 | = |F1 | + 1 > |F2 | + 1 = |X 2 |, so let x ∈ X 1 \ X 2 . The r -x-path in (X 1 , F1 ) contains an edge (v, w) with v ∈ X 2 and w∈ / X 2 . This edge can be added to (X 2 , F2 ), proving that F2 ∪ {(v, w)} ∈ F. This greedoid is called the directed (undirected) branching greedoid of G. The problem of finding a maximum weight spanning tree in a connected graph G with nonnegative weights is the M AXIMIZATION P ROBLEM for the cycle matroid M(G). The B EST-I N -G REEDY A LGORITHM is in this case nothing but K RUSKAL ’ S A LGORITHM. Now we have a second formulation of the same problem: we are looking for a maximum weight set F with F ∈ F, where (E(G), F) is the undirected branching greedoid of G. We now formulate a general greedy algorithm for greedoids. In the special case of matroids, it is exactly the B EST-I N -G REEDY A LGORITHM discussed in Section 13.4. If we have an undirected branching greedoid with a modular cost function c, it is P RIM ’ S A LGORITHM:

G REEDY A LGORITHM F OR G REEDOIDS Input: Output:

A greedoid (E, F) and a function c : 2 E → R, given by an oracle which for any given X ⊆ E says whether X ∈ F and returns c(X ). A set F ∈ F.

1

Set F := ∅.

2

Let e ∈ E \ F such that F ∪ {e} ∈ F and c(F ∪ {e}) is maximum; if no such e exists then stop.

340

3

14 Generalizations of Matroids

Set F := F ∪ {e} and go to . 2

Even for modular cost functions c this algorithm does not always provide an optimal solution. At least we can characterize those greedoids where it works: Theorem 14.7. Let (E, F) be a greedoid. The G REEDY A LGORITHM F OR G REE DOIDS finds a set F ∈ F of maximum weight for each modular weight function c : 2 E → R+ if and only if (E, F) has the so-called strong exchange property: For all A ∈ F, B maximal in F, A ⊆ B and x ∈ E \ B with A ∪ {x} ∈ F there exists a y ∈ B \ A such that A ∪ {y} ∈ F and (B \ y) ∪ {x} ∈ F. Proof: Suppose (E, F) is a greedoid with the strong exchange property. Let c : E → R+ , and let A = {a1 , . . . , al } be the solution found by the G REEDY A L GORITHM F OR G REEDOIDS . , where the elements are chosen in the order a1 , . . . , al . Let B = {a1 , . . . , ak } ∪ B 0 be an optimum solution such that k is maximum, and suppose that k < l. Then we apply the strong exchange property to {a1 , . . . , ak }, B and ak+1 . We conclude that there exists a y ∈ B 0 with {a1 , . . . , ak , y} ∈ F and (B \ y) ∪ {ak+1 } ∈ F. By the choice of ak+1 in 2 of the G REEDY A LGORITHM F OR G REEDOIDS we have c(ak+1 ) ≥ c(y) and thus c((B \ y) ∪ {ak+1 }) ≥ c(B), contradicting the choice of B. Conversely, let (E, F) be a greedoid that does not have the strong exchange property. Let A ∈ F, B maximal in F, A ⊆ B and x ∈ E \ B with A ∪ {x} ∈ F such that for all y ∈ B \ A with A ∪ {y} ∈ F we have (B \ y) ∪ {x} ∈ / F. Let Y := {y ∈ B \ A : A ∪ {y} ∈ F}. We set c(e) := 2 for e ∈ B \ Y , and c(e) := 1 for e ∈ Y ∪ {x} and c(e) := 0 for e ∈ E \ (B ∪ {x}). Then the G REEDY A LGORITHM F OR G REEDOIDS might choose the elements of A first (they have weight 2) and then might choose x. It will eventually end up with a set F ∈ F that cannot be optimal, since c(F) ≤ c(B ∪ {x}) − 2 < c(B ∪ {x}) − 1 = c(B) and B ∈ F. Indeed, optimizing modular functions over general greedoids is NP-hard. This follows from the following observation (together with Corollary 15.24): Proposition 14.8. The problem of deciding, given an undirected graph G and k ∈ N, whether G has a vertex cover of cardinality k, linearly reduces to the following problem: Given a greedoid (E, F) (by a membership oracle) and a function c : E → R+ , find an F ∈ F with c(F) maximum. .

Proof: Let G be any undirected graph and k ∈ N. Let D := V (G) ∪ E(G) and F := {X ⊆ D : for all e = {v, w} ∈ E(G) ∩ X we have v ∈ X or w ∈ X }. (D, F) is an antimatroid: it is accessible and closed under union. In particular, by Proposition 14.3, it is a greedoid. Now consider F 0 := {X ∈ F : |X | ≤ |E(G)| + k}. Since (M1) and (M3) are preserved, (D, F 0 ) is also a greedoid. Set c(e) := 1 for e ∈ E(G) and c(v) := 0

14.2 Polymatroids

341

for v ∈ V (G). Then there exists a set F ∈ F 0 with c(F) = |E(G)| if and only if G contains a vertex cover of size k. On the other hand, there are interesting functions that can be maximized over arbitrary greedoids, for example bottleneck functions c(F) := min{c0 (e) : e ∈ F} for some c0 : E → R+ (Exercise 2). See the book by Korte, Lov´asz and Schrader [1991] for more results in this area.

14.2 Polymatroids From Theorem 13.10 we know the tight connection between matroids and submodular functions. Submodular functions define the following interesting class of polyhedra: Definition 14.9. A polymatroid is a polytope of type ) ( X xe ≤ f (A) for all A ⊆ E P( f ) := x ∈ R E : x ≥ 0, e∈A

where E is a finite set and f : 2 E → R+ is a submodular function. It is not hard to see that for any polymatroid f can be chosen such that f (∅) = 0 and f is monotone (Exercise 5; a function f : 2 E → R is called monotone if f (X ) ≤ f (Y ) for X ⊆ Y ⊆ E). Edmonds’ original definition was different; see Exercise 6. Moreover, we mention that the term polymatroid is sometimes not used for the polytope but for the pair (E, f ). If f is the rank function of a matroid, P( f ) is the convex hull of the incidence vectors of the independent sets of this matroid (Theorem 13.21). We know that the B EST-I N -G REEDY optimizes any linear function over a matroid polytope. A similar greedy algorithm also works for general polymatroids. We assume that f is monotone:

P OLYMATROID G REEDY A LGORITHM Input: Output:

1

2

A finite set E and a submodular, monotone function f : 2 E → R+ (given by an oracle). A vector c ∈ R E . A vector x ∈ P( f ) with cx maximum.

Sort E = {e1 , . . . , en } such that c(e1 ) ≥ · · · ≥ c(ek ) > 0 ≥ c(ek+1 ) ≥ · · · ≥ c(en ). If k ≥ 1 then set x(e1 ) := f ({e1 }). Set x(ei ) := f ({e1 , . . . , ei }) − f ({e1 , . . . , ei−1 }) for i = 2, . . . , k. Set x(ei ) := 0 for i = k + 1, . . . , n.

342

14 Generalizations of Matroids

Proposition 14.10. Let E = {e1 , . . . , en } and f : 2 E → R be a submodular function with f (∅) ≥ 0. Let b : E → R with b(e1 ) ≤ fP ({e1 }) and b(ei ) ≤ f ({e1 , . . . , ei }) − f ({e1 , . . . , ei−1 }) for i = 2, . . . , n. Then a∈A b(a) ≤ f (A) for all A ⊆ E. Proof: Induction on i = max{ Pj : e j ∈ A}. The P assertion is trivial for A = ∅ and A = {e1 }. If i ≥ 2, then a∈A b(a) = a∈A\{ei } b(a) + b(ei ) ≤ f (A \ {ei }) + b(ei ) ≤ f (A \ {ei }) + f ({e1 , . . . , ei }) − f ({e1 , . . . , ei−1 }) ≤ f (A), where the first inequality follows from the induction hypothesis and the third one from submodularity. Theorem 14.11. The P OLYMATROID G REEDY A LGORITHM correctly finds an x ∈ P( f ) with cx maximum. If f is integral, then x is also integral. Proof: Let x ∈ R E be the output of the P OLYMATROID G REEDY A LGORITHM for E, f and c. By definition, if f is integral, then x is also integral. We have x ≥ 0 since f is monotone, and thus x ∈ P( f ) by Proposition 14.10. E with cy > cx. Similarly as in the proof of Theorem 13.19 Now let y ∈ R+ we set d j := c(e j ) − c(e j+1 ) ( j = 1, . . . , k − 1) and dk := c(ek ), and we have k X j=1

dj

j X

x(ei ) = cx < cy ≤

i=1

k X

c(e j )y(e j ) =

j=1

k X j=1

dj

j X

y(ei ).

i=1

Pj Since d j ≥ 0 for all j there is an index j ∈ {1, . . . , k} with i=1 y(ei ) > Pj Pj / i=1 x(ei ); however, since i=1 x(ei ) = f ({e1 , . . . , e j }) this means that y ∈ P( f ). As with matroids, we can also handle the intersection of two polymatroids. The following polymatroid intersection theorem has many implications: Theorem 14.12. (Edmonds [1970,1979]) Let E be a finite set, and let f, g : 2 E → R+ be submodular functions. Then the system X e∈A X

x ≥ 0 xe ≤ f (A)

(A ⊆ E)

xe ≤ g(A)

(A ⊆ E)

e∈A

is TDI. Proof: Consider the primal-dual pair of LPs ( ) X X max cx : x ≥ 0, xe ≤ f (A) and xe ≤ g(A) for all A ⊆ E e∈A

e∈A

14.2 Polymatroids

343

and min

X

( f (A)y A + g(A)z A ) : y, z ≥ 0,

A⊆E

X A⊆E, e∈A

(y A + z A ) ≥ ce for all e ∈ E

To show total dual integrality, we use Lemma 5.23. Let c : E(G) → Z, and let y, z be an optimum dual solution for which X (y A + z A )|A||E \ A| (14.1) A⊆E

is as small as possible. We claim that F := {A ⊆ E : y A > 0} is a chain, i.e. for any A, B ∈ F either A ⊆ B or B ⊆ A. To see this, suppose A, B ∈ F with A ∩ B 6= A and A ∩ B 6= B. Let := min{y A , y B }. Set y 0A := y A − , y B0 := y B − , y 0A∩B := y A∩B + , y 0A∪B := y A∪B + , and y 0 (S) := y(S) for all other S ⊆ E. Since y 0 , z is a feasible dual solution, it is also optimum ( f is submodular) and contradicts the choice of y, because (14.1) is smaller for y 0 , z. By the same argument, F 0 := {A ⊆ E : z A > 0} is a chain. Now let M and M 0 be the matrices whose columns are indexed with the elements of E and whose rows are the incidence vectors of the elements of F and F 0 , respectively. M By Lemma 5.23, it suffices to show that M 0 is totally unimodular. Here we use Ghouila-Houri’s Theorem 5.24. Let R be a set of rows, say R = {A1 , . . . , A p , B1 , . . . , Bq } with A1 ⊇ · · · ⊇ A p and B1 ⊇ · · · ⊇ Bq . Let R1 := {Ai : i odd} ∪ {Bi : i even} and R2 := R \ R1 . Since for any e ∈ E we have {R ∈ R : e ∈ R} = {A1 , . . . , A pe } ∪ {B1 , . . . , Bqe } for some pe ∈ {0, . . . , p} and qe ∈ {0, . . . , q}, the sum of the rows in R1 minus the sum of the rows in R2 is a vector with entries −1, 0, 1 only. So the criterion of Theorem 5.24 is satisfied. One can optimize linear functions over the intersection of two polymatroids. However, this is not as easy as with a single polymatroid. But we can use the E LLIP SOID M ETHOD if we can solve the S EPARATION P ROBLEM for each polymatroid. We return to this question in Section 14.3. Corollary 14.13. (Edmonds [1970]) Let (E, M1 ) and (E, M2 ) be two matroids with rank functions r1 and r2 . Then the convex hull of the incidence vectors of the elements of M1 ∩ M2 is the polytope ( ) X E x ∈ R+ : xe ≤ min{r1 (A), r2 (A)} for all A ⊆ E . e∈A

Proof: As r1 and r2 are nonnegative and submodular (by Theorem 13.10), the above inequality system is TDI (by Theorem 14.12). Since r1 and r2 are integral, the polytope is integral (by Corollary 5.15). Since r1 (A) ≤ |A| for all A ⊆ E,

.

344

14 Generalizations of Matroids

the vertices (the convex hull of which the polytope is by Corollary 3.32) are 01-vectors, and thus incidence vectors of common independent sets (elements of M1 ∩ M2 ). On the other hand, each such incidence vector satisfies the inequalities (by definition of the rank function). Of course, the description of the matroid polytope (Theorem 13.21) follows from this by setting M1 = M2 . Theorem 14.12 has some further consequences: Corollary 14.14. (Edmonds [1970]) Let E be a finite set, and let f, g : 2 E → R+ be submodular and monotone functions. Then max{1lx : x ∈ P( f ) ∩ P(g)} = min ( f (A) + g(E \ A)). A⊆E

Moreover, if f and g are integral, there exists an integral x attaining the maximum. Proof: By Theorem 14.12, the dual to max{1lx : x ∈ P( f ) ∩ P(g)}, which is X min ( f (A)y A + g(A)z A ) : y, z ≥ 0, A⊆E

X

(y A + z A ) ≥ 1 for all e ∈ E

A⊆E, e∈A

,

S S has an integral optimum solution y, z. Let B := A:y A ≥1 A and C := A:z A ≥1 A. Let y B0 := 1, z C0 := 1 and let all other components of y 0 and z 0 be zero. We have B ∪ C = E and y 0 , z 0 is a feasible dual solution. Since f and g are submodular and nonnegative, X ( f (A)y A + g(A)z A ) ≥ f (B) + g(C). A⊆E

Since E \ B ⊆ C and g is monotone, this is at least f (B) + g(E \ B), proving “≥”. The other inequality “≤” is trivial, because for any A ⊆ E we obtain a feasible dual solution y, z by setting y A := 1, z E\A := 1 and all other components to zero. The integrality follows directly from Theorem 14.12 and Corollary 5.15. Theorem 13.31 is a special case. Moreover we obtain: Corollary 14.15. (Frank [1982]) Let E be a finite set and f, g : 2 E → R such that f is supermodular, g is submodular and f ≤ g. Then there exists a modular function h : 2 E → R with f ≤ h ≤ g. If f and g are integral, h can be chosen integral. Proof: Let M := 2 max{| f (A)| + |g(A)| : A ⊆ E}. Let f 0 (A) := g(E) − f (E \ A) + M|A| and g 0 (A) := g(A) − f (∅) + M|A| for all A ⊆ E. f 0 and g 0 are nonnegative, submodular and monotone. An application of Corollary 14.14 yields

14.3 Minimizing Submodular Functions

345

max{1lx : x ∈ P( f 0 ) ∩ P(g 0 )} = min ( f 0 (A) + g 0 (E \ A)) A⊆E

= min (g(E) − f (E \ A) + M|A| + g(E \ A) − f (∅) + M|E \ A|) A⊆E

≥ g(E) − f (∅) + M|E|. f (∅)+ M|E|. If f and g are integral, x So let x ∈ P( f 0 )∩ P(g 0 ) with 1lx = g(E)− P can be chosen integral. Let h 0 (A) := e∈A xe and h(A) := h 0 (A) + f (∅) − M|A| for all A ⊆ E. The function h is modular. Moreover, for all A ⊆ E we have h(A) ≤ g 0 (A)+ f (∅)− M|A| = g(A) and h(A) = 1lx −h 0 (E \ A)+ f (∅)− M|A| ≥ g(E) + M|E| − M|A| − f 0 (E \ A) = f (A). The analogy to convex and concave functions is obvious; see also Exercise 9.

14.3 Minimizing Submodular Functions The S EPARATION PP ROBLEM for a polymatroid P( f ) and a vector x asks for a set this problem reduces to finding a set A minimizing A with f (A) < e∈A x(e). SoP g(A), where g(A) := f (A) − e∈A x(e). Note that if f is submodular, then g is also submodular. Therefore it is an interesting problem to minimize submodular functions. Another motivation might be that submodular functions can be regarded as the discrete analogue of convex functions (Corollary 14.15 and Exercise 9). We have already solved a special case in Section 8.7: finding the minimum cut in an undirected graph can be regarded as minimizing a certain symmetric submodular function f : 2U → R+ over 2U \ {∅, U }. Before returning to this special case we first show how to minimize general submodular functions. For simplicity we restrict ourselves to integer-valued submodular functions:

S UBMODULAR F UNCTION M INIMIZATION P ROBLEM Instance: Task:

A finite set U . A submodular function f : 2U → Z (given by an oracle). Find a subset X ⊆ U with f (X ) minimum.

Gr¨otschel, Lov´asz and Schrijver [1981] showed how this problem can be solved with the help of the E LLIPSOID M ETHOD. The idea is to determine the minimum by binary search; this will reduce the problem to the S EPARATION P ROBLEM for a polymatroid. Using the equivalence of separation and optimization (Section 4.6), it thus suffices to optimize linear functions over polymatroids. However, this can be done easily by the P OLYMATROID G REEDY A LGORITHM. We first need an upper bound on | f (S)| for S ⊆ U : Proposition 14.16. For any submodular function f : 2U → Z and any S ⊆ U we have

346

14 Generalizations of Matroids

f (U )−

X

max{0, f ({u})− f (∅)} ≤ f (S) ≤ f (∅)+

u∈U

X

max{0, f ({u})− f (∅)}.

u∈U

In particular, a number B with | f (S)| ≤ B for all S ⊆ U can be computed in linear time, with |U | + 2 oracle calls to f . Proof: By repeated application of submodularity we get for ∅ 6= S ⊆ U (let x ∈ S): X f (S) ≤ − f (∅) + f (S \ {x}) + f ({x}) ≤ · · · ≤ −|S| f (∅) + f (∅) + f ({x}), x∈S

and for S ⊂ U (let y ∈ U \ S): f (S) ≥ − f ({y}) + f (S ∪ {y}) + f (∅) ≥ · · · X f ({y}) + f (U ) + |U \ S| f (∅). ≥ − y∈U \S

Proposition 14.17. The following problem can be solved in polynomial time: Given a finite set U , a submodular and monotone function f : 2U → Z+ (by an oracle) with f (S) > 0 for S 6= ∅, a number B ∈ N with f (S) ≤ B for all S ⊆ U , and a vector x ∈ ZU + , decide if x ∈ P( f ) and otherwise return a set S ⊆ U with P x(v) > f (S). v∈S Proof: This is the S EPARATION P ROBLEM for the polymatroid P( f ). We will use Theorem 4.23, because we have already solved the optimization problem for P( f ): the P OLYMATROID G REEDY A LGORITHM maximizes any linear function over P( f ) (Theorem 14.11). We have to check the prerequisites of Theorem 4.23. Since the zero vector and the unit vectors are all in P( f ), we can take x0 := 1l as a point in the interior, where = |U 1|+1 . We have size(x0 ) = O(|U | log |U |). Moreover, each vertex of P( f ) is produced by the P OLYMATROID G REEDY A LGORITHM (for some objective function; cf. Theorem 14.11) and thus has size O(|U |(2 + log B)). We conclude that the S EPARATION P ROBLEM can be solved in polynomial time. By Theorem 4.23, we get a facet-defining inequality P of P( f ) violated by x if x∈ / P( f ). This corresponds to a set S ⊆ U with v∈S x(v) > f (S). If f is not monotone, we cannot apply this result directly. Instead we consider a different function: Proposition 14.18. Let f : 2U → R be a submodular function and β ∈ R. Then g : 2U → R, defined by X ( f (U \ {e}) − f (U )), g(X ) := f (X ) − β + e∈X

is submodular and monotone.

14.4 Schrijver’s Algorithm

347

Proof: The submodularity of g follows directly from the submodularity of f . To show that g is monotone, let X ⊂ U and e ∈ U \ X . We have g(X ∪ {e}) − g(X ) = f (X ∪ {e}) − f (X ) + f (U \ {e}) − f (U ) ≥ 0 since f is submodular. Theorem 14.19. The S UBMODULAR F UNCTION M INIMIZATION P ROBLEM can be solved in time polynomial in |U | + log max{| f (S)| : S ⊆ U }. Proof: Let U be a finite set; suppose we are given f by an oracle. First compute a number B ∈ N with | f (S)| ≤ B for all S ⊆ U (cf. Proposition 14.16). Since f is submodular, we have for each e ∈ U and for each X ⊆ U \ {e}: f ({e}) − f (∅) ≥ f (X ∪ {e}) − f (X ) ≥ f (U ) − f (U \ {e}).

(14.2)

If, for some e ∈ U , f ({e}) − f (∅) ≤ 0, then by (14.2) there is an optimum set S containing e. In this case we consider the instance (U 0 , B, f 0 ) defined by U 0 := U \ {e} and f 0 (X ) := f (X ∪ {e}) for X ⊆ U \ {e}, find a set S 0 ⊆ U 0 with f 0 (S 0 ) minimum and output S := S 0 ∪ {e}. Similarly, if f (U ) − f (U \ {e}) ≥ 0, then by (14.2) there is an optimum set S not containing e. In this case we simply minimize f restricted to U \ {e}. In both cases we have reduced the size of the ground set. So we may assume that f ({e}) − f (∅) > 0 and f (U \ {e}) − f (U ) > 0 for all e ∈ U . Let x(e) := f (U \ {e}) − f (UP ). For each integer β with −B ≤ β ≤ f (∅) we define g(X ) := f (X ) − β + e∈X x(e). By Proposition 14.18, g is submodular and monotone. Furthermore we have g(∅) = f (∅) − β ≥ 0 and g({e}) = f ({e}) − β + x(e) > 0 for all e ∈ U , and thus g(X ) > 0 for all ∅ 6= X ⊆ U . Now we apply Proposition 14.17 and check if x ∈ P(g). If yes, we have f (X ) ≥ β for all X ⊆ U and we are done. Otherwise we get a set S with f (S) < β. Now we apply binary search: by choosing β appropriately each time, we need O(log(2B)) iterations to find the number β ∗ ∈ {−B, −B + 1, . . . , f (∅)} for which f (X ) ≥ β ∗ for all X ⊆ U but f (S) < β ∗ +1 for some S ⊆ U . This set S minimizes f. The first strongly polynomial-time algorithm has been designed by Gr¨otschel, Lov´asz and Schrijver [1988], also based on the ellipsoid method. Combinatorial algorithms to solve the S UBMODULAR F UNCTION M INIMIZATION P ROBLEM in strongly polynomial time have been found by Schrijver [2000] and independently by Iwata, Fleischer and Fujishige [2001]. In the next section we describe Schrijver’s algorithm.

14.4 Schrijver’s Algorithm For a finite set U and a submodular function f : 2U → Z, assume w.l.o.g. that U = {1, . . . , n} and f (∅) = 0. At each stage, Schrijver’s [2000] algorithm maintains a point x in the so-called base polyhedron of f , defined by

348

14 Generalizations of Matroids

(

) U

x ∈R :

X

x(u) ≤ f (A) for all A ⊆ U,

u∈A

X

x(u) = f (U ) .

u∈U

We mention that the set of vertices of this base polyhedron is precisely the set of vectors b≺ for all total orders ≺ of U , where we define b≺ (u) := f ({v ∈ U : v u}) − f ({v ∈ U : v ≺ u}) (u ∈ U ). This fact, which we will not need here, can be proved in a similar way as Theorem 14.11 (Exercise 13). The point x is always written as an explicit convex combination x = λ1 b≺1 + · · · + λk b≺k of these vertices. Initially, one can choose k = 1 and any total order. For a total order ≺ and s, u ∈ U , we denote by ≺s,u the total order that results from ≺ by moving u just before s. Moreover, let χ u denote the incidence vector of u (u ∈ U ).

S CHRIJVER ’ S A LGORITHM Input: Output:

A finite set U = {1, . . . , n}. A submodular function f : 2U → Z with f (∅) = 0 (given by an oracle). A subset X ⊆ U with f (X ) minimum.

1

Set k := 1, let ≺1 be any total order on U , and set x := b≺1 .

2

Set D := (U, A), where A = {(u, v) : u ≺i v for some i ∈ {1, . . . , k}}.

3

Let P := {v ∈ U : x(v) > 0} and N := {v ∈ U : x(v) < 0}, and let X be the set of vertices not reachable from P in the digraph D. If N ⊆ X , then stop else let d(v) denote the distance from P to v in D. Choose the vertex t ∈ N reachable from P with (d(t), t) lexicographically maximum. Choose the maximal vertex s with (s, t) ∈ A and d(s) = d(t) − 1. Let i ∈ {1, . . . , k} such that α := |{v : s ≺i v i t}| is maximum (the number of indices attaining this maximum will be denoted by β).

4

5

Compute a number with 0 ≤ ≤ −x(t) and write x 0 := x + (χ t − χ s ) as an explicit convex combination of at most n vectors, chosen among s,u b≺1 , . . . , b≺k and b≺i for all u ∈ U with s ≺i u i t, with the additional property that b≺i does not occur if x 0 (t) < 0.

6

Set x := x 0 , rename the vectors in the convex combination of x as b≺1 , . . . , b≺k 0 , set k := k 0 , and go to . 2

Theorem 14.20. (Schrijver [2000]) S CHRIJVER ’ S A LGORITHM works correctly. Proof: The algorithm terminates if D contains no path from P to N and outputs of vertices not reachable from P. Clearly N ⊆ X ⊆ U \ P, so P the set XP x(u) ≤ u∈X u∈W x(u) for each W ⊆ U . Moreover, no edge enters X , so either X = ∅ or for each j ∈ {1, . . . , k} there exists a v ∈ X with X = {u ∈ U : u j v}.

14.4 Schrijver’s Algorithm

349

P ≺j Hence, by definition, P u∈X≺b (u) = f (X ) for all j ∈ {1, . . . , k}. Moreover, by Proposition 14.10, u∈W b j (u) ≤ f (W ) for all W ⊆ U and j ∈ {1, . . . , k}. Therefore, for each W ⊆ U , f (W ) ≥

k X

λj

≥

b≺ j (u) =

u∈W

j=1

X

X

x(u) =

u∈X

k XX

λ j b≺ j (u) =

u∈W j=1 k XX

λ j b (u) = ≺j

u∈X j=1

X

x(u)

u∈W k X j=1

λj

X

b≺ j (u) = f (X ),

u∈X

proving that X is an optimum solution.

Lemma 14.21. (Schrijver [2000]) Each iteration can be performed in O(n 3 + γ n 2 ) time, where γ is the time for an oracle call. Proof: It suffices to show that 5 can be done in O(n 3 + γ n 2 ) time. Let x = ≺ ≺ λ1 b 1 + · · · + λk b k and s ≺i t. We first show: Claim: δ(χ t − χ s ), for some δ ≥ 0, can be written as convex combination of s,v the vectors b≺i − b≺i for s ≺i v i t in O(γ n 2 ) time. To prove this, we need some preliminaries. Let s ≺i v i t. By definition, ≺is,v b (u) = b≺i (u) for u ≺i s or u i v. As f is submodular, we have for s i u ≺i v: s,v

f ({w ∈ U : w is,v u}) − f ({w ∈ U : w ≺is,v u})

b≺i (u) =

f ({w ∈ U : w i u}) − f ({w ∈ U : w ≺i u}) = b≺i (u).

≤

Moreover, for u = v we have: s,v

b≺i (v) =

f ({w ∈ U : w is,v v}) − f ({w ∈ U : w ≺is,v v})

=

f ({w ∈ U : w ≺i s} ∪ {v}) − f ({w ∈ U : w ≺i s})

≥

f ({w ∈ U : w i v}) − f ({w ∈ U : w ≺i v})

= b≺i (v). s,v P P Finally, observe that u∈U b≺i (u) = f (U ) = u∈U b≺i (u). s,v As the claim is trivial if b≺i = b≺i for some s ≺i v i t, we may assume s,v b≺i (v) > b≺i (v) for all s ≺i v i t. We recursively set κv :=

χvt −

P

s,w

v≺i wi t ≺is,v

b

κw (b≺i (v) − b≺i (v)) (v) − b≺i (v)

≥ 0

P ≺is,v − b≺i ) = χ t − χ s , because for s ≺i v i t, and obtain s≺i vi t κv (b s,v s,v P P ≺i (u) − b≺i (u)) = ui vi t κv (b≺i (u) − b≺i (u)) = χut for all s≺i vi t κv (b s ≺i u i t, and the sum over all components is zero.

350

14 Generalizations of Matroids

By setting δ :=

1 P

s≺i vi t

κv

and multiplying each κu by δ, we see that the claim

follows. Now consider := min{λi δ, −x(t)} and x 0 := x + (χ t − χ s ). If = λi δ ≤ Pk P ≺is,v ≺j + λ − b≺i ), i.e. we −x(t), then we have x 0 = i j=1 λ j b s≺i vi t κv (b s,v

have written x 0 as a convex combination of b≺ j ( j ∈ {1, . . . , k} \ {i}) and b≺i (s ≺i v i t). If = −x(t), we may additionally use b≺i in the convex combination. We finally reduce this convex combination to at most n vectors in O(n 3 ) time, as shown in Exercise 5 of Chapter 4. Lemma 14.22. (Vygen [2003]) O(n 5 ) iterations.

S CHRIJVER ’ S A LGORITHM terminates after s,v

Proof: If an edge (v, w) is introduced after a new vector b≺i has been added in

5 of an iteration, then s i w ≺i v i t in this iteration. Thus d(w) ≤ d(s) + 1 = d(t) ≤ d(v) + 1 in this iteration, and the introduction of the new edge cannot make the distance from P to any v ∈ U smaller. As 5 makes sure that no element is ever added to P, the distance d(v) never decreases for any v ∈ U . Call a block a sequence of iterations where the pair (t, s) remains constant. Note that each block has O(n 2 ) iterations, because (α, β) decreases lexicographically in each iteration within each block. It remains to prove that there are O(n 3 ) blocks. A block can end only because of at least one of the following reasons (by the choice of t and s, since an iteration with t = t ∗ does not add any edge whose head is t ∗ , and since a vertex v can enter N only if v = s and hence d(v) < d(t)): (a) the distance d(v) increases for some v ∈ U . (b) t is removed from N . (c) (s, t) is removed from A. We now count the number of blocks of these three types. Clearly there are O(n 2 ) blocks of type (a). Now consider type (b). We claim that for each t ∗ ∈ U there are O(n 2 ) iterations with t = t ∗ and x 0 (t) = 0. This is easy to see: between every two such iterations, d(v) must change for some v ∈ U , and this can happen O(n 2 ) times as d-values can only increase. Thus there are O(n 3 ) blocks of type (b). We finally show that there are O(n 3 ) blocks of type (c). It suffices to show that d(t) will change before the next such block with the pair (s, t). For s, t ∈ U , we say that s is t-boring if (s, t) ∈ / A or d(t) ≤ d(s). Let s ∗ , t ∗ ∈ U , and consider the time period after a block with s = s ∗ and t = t ∗ ending because (s ∗ , t ∗ ) is removed from A, until the subsequent change of d(t ∗ ). We prove that each v ∈ {s ∗ , . . . , n} is t ∗ -boring throughout this period. Applying this for v = s ∗ concludes the proof. At the beginning of the period, each v ∈ {s ∗ + 1, . . . , n} is t ∗ -boring due to the choice of s = s ∗ in the iteration immediately preceding the period. s ∗ is also t ∗ -boring as (s ∗ , t ∗ ) is removed from A. As d(t ∗ ) remains constant within the considered time period and d(v) never decreases for any v, we only have to check the introduction of new edges.

14.5 Symmetric Submodular Functions

351

Suppose that, for some v ∈ {s ∗ , . . . , n}, the edge (v, t ∗ ) is added to A after an iteration that chooses the pair (s, t). Then, by the initial remarks of this proof, s i t ∗ ≺i v i t in this iteration, and thus d(t ∗ ) ≤ d(s) + 1 = d(t) ≤ d(v) + 1. Now we distinguish two cases: If s > v, then we have d(t ∗ ) ≤ d(s): either because t ∗ = s, or as s was t ∗ -boring and (s, t ∗ ) ∈ A. If s < v, then we have d(t) ≤ d(v): either because t = v, or by the choice of s and since (v, t) ∈ A. In both cases we conclude that d(t ∗ ) ≤ d(v), and v remains t ∗ -boring. Theorem 14.20, Lemma 14.21 and Lemma 14.22 imply: Theorem 14.23. The S UBMODULAR F UNCTION M INIMIZATION P ROBLEM can be solved in O(n 8 + γ n 7 ) time, where γ is the time for an oracle call. Iwata [2002] described a fully combinatorial algorithm (using only additions, subtractions, comparisons and oracle calls, but no multiplication or division). He also improved the running time (Iwata [2003]). The currently fastest strongly polynomial-time algorithm was found by Orlin [2007]; it runs in O(n 6 + γ n 5 ) time.

14.5 Symmetric Submodular Functions A submodular function f : 2U → R is called symmetric if f (A) = f (U \ A) for all A ⊆ U . In this special case the S UBMODULAR F UNCTION M INIMIZATION P ROBLEM is trivial, since 2 f (∅) = f (∅) + f (U ) ≤ f (A) + f (U \ A) = 2 f (A) for all A ⊆ U , implying that the empty set is optimal. Hence the problem is interesting only if this trivial case is excluded: one looks for a nonempty proper subset A of U such that f (A) is minimum. Generalizing the algorithm of Section 8.7, Queyranne [1998] has found a relatively simple combinatorial algorithm for this problem using only O(n 3 ) oracle calls. The following lemma is a generalization of Lemma 8.38 (Exercise 14): Lemma 14.24. Given a symmetric submodular function f : 2U → R with n := |U | ≥ 2, we can find two elements x, y ∈ U with x 6= y and f ({x}) = min{ f (X ) : x ∈ X ⊆ U \ {y}} in O(n 2 θ ) time, where θ is the time bound of the oracle for f . Proof: We construct an order U = {u 1 , . . . , u n } by doing the following for k = 1, . . . , n − 1. Suppose that u 1 , . . . , u k−1 are already constructed; let Uk−1 := {u 1 , . . . , u k−1 }. For C ⊆ U we define 1 wk (C) := f (C) − ( f (C \ Uk−1 ) + f (C ∪ Uk−1 ) − f (Uk−1 )). 2 Note that wk is also symmetric. Let u k be an element of U \ Uk−1 that maximizes wk ({u k }). Finally, let u n be the only element in U \ {u 1 , . . . , u n−1 }. Obviously the construction of the order u 1 , . . . , u n can be done in O(n 2 θ ) time.

352

14 Generalizations of Matroids

Claim: For all k = 1, . . . , n − 1 and all x, y ∈ U \ Uk−1 with x 6= y and wk ({x}) ≤ wk ({y}) we have wk ({x}) = min{wk (C) : x ∈ C ⊆ U \ {y}}. We prove the claim by induction on k. For k = 1 the assertion is trivial since w1 (C) = 21 f (∅) for all C ⊆ U . Let now k > 1 and x, y ∈ U \ Uk−1 with x 6= y and wk ({x}) ≤ wk ({y}). / Z , and let z ∈ Z \ Uk−1 . By the choice of Moreover, let Z ⊆ U with u k−1 ∈ u k−1 we have wk−1 ({z}) ≤ wk−1 ({u k−1 }); thus by the induction hypothesis we get wk−1 ({z}) ≤ wk−1 (Z ). Furthermore, the submodularity of f implies (wk (Z ) − wk−1 (Z )) − (wk ({z}) − wk−1 ({z})) 1 = ( f (Z ∪ Uk−2 ) − f (Z ∪ Uk−1 ) − f (Uk−2 ) + f (Uk−1 )) 2 1 − ( f ({z} ∪ Uk−2 ) − f ({z} ∪ Uk−1 ) − f (Uk−2 ) + f (Uk−1 )) 2 1 ( f (Z ∪ Uk−2 ) + f ({z} ∪ Uk−1 ) − f (Z ∪ Uk−1 ) − f ({z} ∪ Uk−2 )) = 2 ≥ 0. Hence wk (Z ) − wk ({z}) ≥ wk−1 (Z ) − wk−1 ({z}) ≥ 0. To conclude the proof of the claim, let C ⊆ U with x ∈ C and y ∈ / C. There are two cases: / C. Then the above result for Z = C and z = x yields wk (C) ≥ Case 1: u k−1 ∈ wk ({x}) as required. Case 2: u k−1 ∈ C. Then we apply the above to Z = U \ C and z = y and get wk (C) = wk (U \ C) ≥ wk ({y}) ≥ wk ({x}). This completes the proof of the claim. Applying it to k = n − 1, x = u n and y = u n−1 we get wn−1 ({u n }) = min{wn−1 (C) : u n ∈ C ⊆ U \ {u n−1 }}. Since wn−1 (C) = f (C) − 21 ( f ({u n }) + f (U \ {u n−1 }) − f (Un−2 )) for all C ⊆ U / C, the lemma follows (set x := u n and y := u n−1 ). with u n ∈ C and u n−1 ∈ The above proof is due to Fujishige [1998]. Now we can proceed analogously to the proof of Theorem 8.39: Theorem 14.25. (Queyranne [1998]) Given a symmetric submodular function f : 2U → R, a nonempty proper subset A of U such that f (A) is minimum can be found in O(n 3 θ ) time where θ is the time bound of the oracle for f . Proof: If |U | = 1, the problem is trivial. Otherwise we apply Lemma 14.24 and find two elements x, y ∈ U with f ({x}) = min{ f (X ) : x ∈ X ⊆ U \ {y}} in O(n 2 θ ) time. Next we recursively find a nonempty proper subset of U \ {x} / X minimizing the function f 0 : 2U \{x} → R, defined by f 0 (X ) := f (X ) if y ∈

Exercises

353

and f 0 (X ) := f (X ∪ {x}) if y ∈ X . One readily observes that f 0 is symmetric and submodular. Let ∅ 6= Y ⊂ U \{x} be a set minimizing f 0 ; w.l.o.g. y ∈ Y (as f 0 is symmetric). We claim that either {x} or Y ∪ {x} minimizes f (over all nonempty proper subsets of U ). To see this, consider any C ⊂ U with x ∈ C. If y ∈ / C, then we have f ({x}) ≤ f (C) by the choice of x and y. If y ∈ C, then f (C) = f 0 (C \ {x}) ≥ f 0 (Y ) = f (Y ∪ {x}). Hence f (C) ≥ min{ f ({x}), f (Y ∪ {x})} for all nonempty proper subsets C of U . To achieve the asserted running time we of course cannot compute f 0 explicitly. Rather we store a partition of U , initially consisting of the singletons. At each step of the recursion we build the union of those two sets of the partition that contain x and y. In this way f 0 can be computed efficiently (using the oracle for f ). This result has been further generalized by Nagamochi and Ibaraki [1998] and by Rizzi [2000].

Exercises 1. Let G be an undirected graph and M a maximum matching in G. Let F be the family of those subsets X ⊆ E(G) for which there exists a special blossom forest F with respect to M with E(F) \ M = X . Prove that (E(G) \ M, F) is a greedoid. Hint: Use Exercise 23 of Chapter 10. 2. Let (E, F) be a greedoid and c0 : E → R+ . We consider the bottleneck function c(F) := min{c0 (e) : e ∈ F} for F ⊆ E. Show that the G REEDY A LGORITHM F OR G REEDOIDS, when applied to (E, F) and c, finds an F ∈ F with c(F) maximum. 3. This exercise shows that greedoids can also be defined as languages (cf. Definition 15.1). Let E be a finite set. A language L over the alphabet E is called a greedoid language if (a) L contains the empty string; (b) xi 6= x j for all (x1 , . . . , xn ) ∈ L and 1 ≤ i < j ≤ n; (c) (x1 , . . . , xn−1 ) ∈ L for all (x1 , . . . , xn ) ∈ L; (d) If (x1 , . . . , xn ), (y1 , . . . , ym ) ∈ L with m < n, then there exists an i ∈ {1, . . . , n} such that (y1 , . . . , ym , xi ) ∈ L. L is called an antimatroid language if it satisfies (a), (b), (c) and (d0 ) If (x1 , . . . , xn ), (y1 , . . . , ym ) ∈ L with {x1 , . . . , xn } 6⊆ {y1 , . . . , ym }, then there exists an i ∈ {1, . . . , n} such that (y1 , . . . , ym , xi ) ∈ L. Prove: A language L over the alphabet E is a greedoid language (an antimatroid language) if and only if the set system (E, F) is a greedoid (antimatroid), where F := {{x1 , . . . , xn } : (x1 , . . . , xn ) ∈ L}. 4. Let U be a finite set and f : 2U → R. Prove that f is submodular if and only if f (X ∪ {y, z}) − f (X ∪ {y}) ≤ f (X ∪ {z}) − f (X ) for all X ⊆ U and y, z ∈ U .

354

∗

14 Generalizations of Matroids

5. Let P be a nonempty polymatroid. Show that then there is a submodular and monotone function f with f (∅) = 0 and P = P( f ). 6. Prove that a nonempty compact set P ⊆ Rn+ is a polymatroid if and only if (a) For all 0 ≤ x ≤ y ∈ P we have x ∈ P. (b) For all x ∈ Rn+ and all y, z ≤ x with y, z ∈ P that are maximal with this property (i.e. y ≤ w ≤ x and w ∈ P implies w = y, and z ≤ w ≤ x and w ∈ P implies w = z) we have 1ly = 1lz. Note: This is the original definition of Edmonds [1970]. 7. Prove that the P OLYMATROID G REEDY A LGORITHM, when applied to a vector E and a submodular but not necessarily monotone function f : 2 E → R c ∈ R+ with f (∅) ≥ 0, solves ( ) X max cx : xe ≤ f (A) for all A ⊆ E . e∈A

8. Prove Theorem 14.12 for the special case that f and g are rank functions of matroids by constructing an integral optimum dual solution from c1 and c2 as generated by the W EIGHTED M ATROID I NTERSECTION A LGORITHM. (Frank [1981]) S → R as follows. For ∗ 9. Let S be a finite set and f : 2 S → R. Define f 0 : R+ S any x ∈ R+ there are unique k ∈ Z+ , λ1 , . . . , λk > 0 and ∅ ⊂ T1 ⊂ T2 ⊂ Pk λi χ Ti , where χ Ti is the incidence vector · · · ⊂ Tk ⊆ S such that x = i=1 P k λi f (Ti ). of Ti . Then f 0 (x) := i=1 Prove that f is submodular if and only if f 0 is convex. (Lov´asz [1983]) 10. Let E be a finite set and f : 2 E → R+ a submodular function with f ({e}) ≤ 2 for all e ∈ E. (The pair (E, f ) is sometimes called a 2-polymatroid.) The P OLYMATROID M ATCHING P ROBLEM asks for a maximum cardinality set X ⊆ E with f (X ) = 2|X |. ( f is of course given by an oracle.) Let E 1 , . . . , E k be pairwise disjoint unordered pairs and let (E, F) be a matroid (given by an independence oracle), where E = E 1 ∪ · · · ∪ E k . The M ATROID PARITY P ROBLEM asks for a maximum cardinality set I ⊆ {1, . . . , k} with S i∈I E i ∈ F. (a) Show that the M ATROID PARITY P ROBLEM polynomially reduces to the P OLYMATROID M ATCHING P ROBLEM. ∗ (b) Show that the P OLYMATROID M ATCHING P ROBLEM polynomially reduces to the M ATROID PARITY P ROBLEM. Hint: Use an algorithm for the S UBMODULAR F UNCTION M INIMIZATION P ROBLEM. ∗ (c) Show that there is no algorithm for the P OLYMATROID M ATCHING P ROB LEM whose running time is polynomial in |E|. (Jensen and Korte [1982], Lov´asz [1981]) (A problem polynomially reduces to another one if the former can be solved with a polynomial-time oracle algorithm using an oracle for the latter; see Chapter 15.)

References

355

Note: A polynomial-time algorithm for an important special case was given by Lov´asz [1980,1981]. 11. A function f : 2 S → R∪{∞} is called crossing submodular if f (X )+ f (Y ) ≥ f (X ∪Y )+ f (X ∩Y ) for any two sets X, Y ⊆ S with X ∩Y 6= ∅ and X ∪Y 6= S. The S UBMODULAR F LOW P ROBLEM is as follows: Given a digraph G, functions l : E(G) → R ∪ {−∞}, u : E(G) → R ∪ {∞}, c : E(G) → R, and a crossing submodular function b : 2V (G) → R ∪ {∞}. Then a feasible submodular flow is a function f : E(G) → R with l(e) ≤ f (e) ≤ u(e) for all e ∈ E(G) and X X f (e) − f (e) ≤ b(X ) e∈δ − (X )

e∈δ + (X )

for all X ⊆ V (G). The task is Pto decide whether a feasible flow exists and, if yes, to find one whose cost e∈E(G) c(e) f (e) is minimum possible. Show that this problem generalizes the M INIMUM C OST F LOW P ROBLEM and the problem of optimizing a linear function over the intersection of two polymatroids. Note: The S UBMODULAR F LOW P ROBLEM, introduced by Edmonds and Giles [1977], can be solved in strongly polynomial time; see Fujishige, R¨ock and Zimmermann [1989]. See also Fleischer and Iwata [2000]. ∗ 12. Show that the inequality system describing a feasible submodular flow (Exercise 11) is TDI. Show that this implies Theorems 14.12 and 19.10. (Edmonds and Giles [1977]) 13. Prove that the set of vertices of the base polyhedron is precisely the set of vectors b≺ for all total orders ≺ of U , where b≺ (u) := f ({v ∈ U : v u}) − f ({v ∈ U : v ≺ u}) (u ∈ U ). Hint: See the proof of Theorem 14.11. 14. Show that Lemma 8.38 is a special case of Lemma 14.24.

References General Literature: Bixby, R.E., and Cunningham, W.H. [1995]: Matroid optimization and algorithms. In: Handbook of Combinatorics; Vol. 1 (R.L. Graham, M. Gr¨otschel, L. Lov´asz, eds.), Elsevier, Amsterdam, 1995 Bj¨orner, A., and Ziegler, G.M. [1992]: Introduction to greedoids. In: Matroid Applications (N. White, ed.), Cambridge University Press, Cambridge 1992 Fujishige, S. [2005]: Submodular Functions and Optimization. Second Edition. Elsevier, Amsterdam 2005 Korte, B., Lov´asz, L., and Schrader, R. [1991]: Greedoids. Springer, Berlin 1991 McCormick, S.T. [2004]: Submodular function minimization. In: Discrete Optimization (K. Aardal, G.L. Nemhauser, R. Weismantel, eds.), Elsevier, Amsterdam 2005 Schrijver, A. [2003]: Combinatorial Optimization: Polyhedra and Efficiency. Springer, Berlin 2003, Chapters 44–49

356

14 Generalizations of Matroids

Cited References: Edmonds, J. [1970]: Submodular functions, matroids and certain polyhedra. In: Combinatorial Structures and Their Applications; Proceedings of the Calgary International Conference on Combinatorial Structures and Their Applications 1969 (R. Guy, H. Hanani, N. Sauer, J. Schonheim, eds.), Gordon and Breach, New York 1970, pp. 69–87 Edmonds, J. [1979]: Matroid intersection. In: Discrete Optimization I; Annals of Discrete Mathematics 4 (P.L. Hammer, E.L. Johnson, B.H. Korte, eds.), North-Holland, Amsterdam 1979, pp. 39–49 Edmonds, J., and Giles, R. [1977]: A min-max relation for submodular functions on graphs. In: Studies in Integer Programming; Annals of Discrete Mathematics 1 (P.L. Hammer, E.L. Johnson, B.H. Korte, G.L. Nemhauser, eds.), North-Holland, Amsterdam 1977, pp. 185–204 Fleischer, L., and Iwata, S. [2000]: Improved algorithms for submodular function minimization and submodular flow. Proceedings of the 32nd Annual ACM Symposium on Theory of Computing (2000), 107–116 Frank, A. [1981]: A weighted matroid intersection algorithm. Journal of Algorithms 2 (1981), 328–336 Frank, A. [1982]: An algorithm for submodular functions on graphs. In: Bonn Workshop on Combinatorial Optimization; Annals of Discrete Mathematics 16 (A. Bachem, M. Gr¨otschel, B. Korte, eds.), North-Holland, Amsterdam 1982, pp. 97–120 Fujishige, S. [1998]: Another simple proof of the validity of Nagamochi and Ibaraki’s min-cut algorithm and Queyranne’s extension to symmetric submodular function minimization. Journal of the Operations Research Society of Japan 41 (1998), 626–628 Fujishige, S., R¨ock, H., and Zimmermann, U. [1989]: A strongly polynomial algorithm for minimum cost submodular flow problems. Mathematics of Operations Research 14 (1989), 60–69 Gr¨otschel, M., Lov´asz, L., and Schrijver, A. [1981]: The ellipsoid method and its consequences in combinatorial optimization. Combinatorica 1 (1981), 169–197 Gr¨otschel, M., Lov´asz, L., and Schrijver, A. [1988]: Geometric Algorithms and Combinatorial Optimization. Springer, Berlin 1988 Iwata, S. [2002]: A fully combinatorial algorithm for submodular function minimization. Journal of Combinatorial Theory B 84 (2002), 203–212 Iwata, S. [2003]: A faster scaling algorithm for minimizing submodular functions. SIAM Journal on Computing 32 (2003), 833–840 Iwata, S., Fleischer, L., and Fujishige, S. [2001]: A combinatorial, strongly polynomialtime algorithm for minimizing submodular functions. Journal of the ACM 48 (2001), 761–777 Jensen, P.M., and Korte, B. [1982]: Complexity of matroid property algorithms. SIAM Journal on Computing 11 (1982), 184–190 Lov´asz, L. [1980]: Matroid matching and some applications. Journal of Combinatorial Theory B 28 (1980), 208–236 Lov´asz, L. [1981]: The matroid matching problem. In: Algebraic Methods in Graph Theory; Vol. II (L. Lov´asz, V.T. S´os, eds.), North-Holland, Amsterdam 1981, 495–517 Lov´asz, L. [1983]: Submodular functions and convexity. In: Mathematical Programming: The State of the Art – Bonn 1982 (A. Bachem, M. Gr¨otschel, B. Korte, eds.), Springer, Berlin 1983 Nagamochi, H., and Ibaraki, T. [1998]: A note on minimizing submodular functions. Information Processing Letters 67 (1998), 239–244

References

357

Orlin, J.B. [2007]: A faster strongly polynomial time algorithm for submodular function minimization. In: Integer Programming and Combinatorial Optimization; Proceedings of the 12th International IPCO Conference; LNCS 4513 (M. Fischetti, D.P. Williamson, eds.), Springer, Berlin 2007, pp. 240–251 Queyranne, M. [1998]: Minimizing symmetric submodular functions. Mathematical Programming B 82 (1998), 3–12 Rizzi, R. [2000]: On minimizing symmetric set functions. Combinatorica 20 (2000), 445– 450 Schrijver, A. [2000]: A combinatorial algorithm minimizing submodular functions in strongly polynomial time. Journal of Combinatorial Theory B 80 (2000), 346–355 Vygen, J. [2003]: A note on Schrijver’s submodular function minimization algorithm. Journal of Combinatorial Theory B 88 (2003), 399–402

15 NP-Completeness

For many combinatorial optimization problems a polynomial-time algorithm is known; the most important ones are presented in this book. However, there are also many important problems for which no polynomial-time algorithm is known. Although we cannot prove that none exists we can show that a polynomial-time algorithm for one “hard” (more precisely: NP-hard) problem would imply a polynomialtime algorithm for almost all problems discussed in this book (more precisely: all NP-easy problems). To formalize this concept and prove the above statement we need a machine model, i.e. a precise definition of a polynomial-time algorithm. Therefore we discuss Turing machines in Section 15.1. This theoretical model is not suitable to describe more complicated algorithms. However we shall argue that it is equivalent to our informal notion of algorithms: every algorithm in this book can, theoretically, be written as a Turing machine, with a loss in efficiency that is polynomially bounded. We indicate this in Section 15.2. In Section 15.3 we introduce decision problems, and in particular the classes P and NP. While NP contains most decision problems appearing in this book, P contains only those for which there are polynomial-time algorithms. It is an open question whether P = NP. Although we shall discuss many problems in NP for which no polynomial-time algorithm is known, nobody can (so far) prove that none exists. We specify what it means that one problem reduces to another, or that one problem is at least as hard as another one. In this notion, the hardest problems in NP are the NP-complete problems; they can be solved in polynomial time if and only if P = NP. In Section 15.4 we exhibit the first NP-complete problem, S ATISFIABILITY. In Section 15.5 some more decision problems, more closely related to combinatorial optimization, are proved to be NP-complete. In Sections 15.6 and 15.7 we shall discuss related concepts, also extending to optimization problems.

15.1 Turing Machines In this section we present a very simple model for computation: the Turing machine. It can be regarded as a sequence of simple instructions working on a string. The input and the output will be a binary string:

360

15 NP-Completeness

Definition 15.1. An alphabet is a finite set with at least two elements, not containing the special symbol t (which we shall use for blanks). For an alphabet A we denote by a string over A a finiteSsequence of elements of A, by An the set of strings of length n, and by A∗ := n∈Z+ An the set of all strings over A. We use the convention that A0 contains exactly one element, the empty string. A language over A is a subset of A∗ . The elements of a language are often called words. If x ∈ An we write size(x) := n for the length of the string. We shall often work with the alphabet A = {0, 1} and the set {0, 1}∗ of all 0-1-strings (or binary strings). The components of a 0-1-string are sometimes called its bits. So there is exactly one 0-1-string of zero length, the empty string. A Turing machine gets as input a string x ∈ A∗ for some fixed alphabet A. The input is completed by blank symbols (denoted by t) to a two-way infinite string s ∈ (A ∪ {t})Z . This string s can be regarded as a tape with a read-write head; only a single position can be read and modified at each step, and the read-write head can be moved by one position in each step. A Turing machine consists of a set of N + 1 instructions numbered 0, . . . , N . In the beginning instruction 0 is executed and the current position of the string is position 1. Now each instruction is of the following type. Read the bit at the current position, and depending on its value do the following: overwrite the current bit by some element of A ∪ {t}, possibly move the current position by one to the left or to the right, and go to a instruction which will be executed next. There is a special instruction denoted by −1 which marks the end of the computation. The components of our infinite string s indexed by 1, 2, 3, . . . up to the first t then yield the output string. Formally we define a Turing machine as follows: Definition 15.2. (Turing [1936]) Let A be an alphabet and A¯ := A ∪ {t}. A Turing machine (with alphabet A) is defined by a function 8 : {0, . . . , N } × A¯ → {−1, . . . , N } × A¯ × {−1, 0, 1} for some N ∈ Z+ . The computation of 8 on input x, where x ∈ A∗ , is the finite or infinite sequence of triples (n (i) , s (i) , π (i) ) with n (i) ∈ {−1, . . . , N }, s (i) ∈ A¯ Z and π (i) ∈ Z (i = 0, 1, 2, . . .) defined recursively as follows (n (i) denotes the current instruction, s (i) represents the string, and π (i) is the current position): (0) (0) n (0) := 0. s j := x j for 1 ≤ j ≤ size(x), and s j := t for all j ≤ 0 and j > size(x). π (0) := 1. If (n (i) , s (i) , π (i) ) is already defined, we distinguish two cases. If n (i) 6= −1, (i) (i+1) (i+1) (i) := s j then let (m, σ, δ) := 8 n (i) , sπ (i) and set n (i+1) := m, sπ (i) := σ , s j for j ∈ Z \ {π (i) }, and π (i+1) := π (i) + δ. If n (i) = −1, then this is the end of the sequence. We then define time(8, x) := i (i) and output(8, x) ∈ Ak , where k := min j ∈ N : s j = t − 1, by output(8, x) j (i)

:= s j

for j = 1, . . . , k.

15.1 Turing Machines

361

If this sequence is infinite (i.e. n (i) 6= −1 for all i), then we set time(8, x) := ∞. In this case output(8, x) is undefined. Of course we are interested mostly in Turing machines whose computation is finite or even polynomially bounded: Definition 15.3. Let A be an alphabet, S, T ⊆ A∗ two languages, and f : S → T a function. Let 8 be a Turing machine with alphabet A such that time(8, s) < ∞ and output(8, s) = f (s) for each s ∈ S. Then we say that 8 computes f . If there exists a polynomial p such that for all s ∈ S we have time(8, s) ≤ p(size(s)), then 8 is a polynomial-time Turing machine. In the case S = A∗ and T = {0, 1} we say that 8 decides the language L := {s ∈ S : f (s) = 1}. If there exists some polynomial-time Turing machine computing a function f (or deciding a language L), then we say that f is computable in polynomial time (or L is decidable in polynomial time, respectively). To make these definitions clear we give an example. The following Turing machine 8 : {0, . . . , 4}×{0, 1, t} → {−1, . . . , 4}×{0, 1, t}×{−1, 0, 1} computes the successor function n 7→ n + 1 (n ∈ N), where the numbers are coded by their usual binary representation. 8(0, 0) 8(0, 1) 8(0, t) 8(1, 1) 8(1, 0) 8(1, t) 8(2, 0) 8(2, t) 8(3, 0) 8(3, t) 8(4, 0)

= = = = = = = = = = =

(0, 0, 1) (0, 1, 1) (1, t, −1) (1, 0, −1) (−1, 1, 0) (2, t, 1) (2, 0, 1) (3, 0, −1) (3, 0, −1) (4, t, 1) (−1, 1, 0)

0 While sπ 6= t do π := π + 1. Set π := π − 1.

1 While sπ = 1 do sπ := 0 and π := π − 1. If sπ = 0 then sπ := 1 and stop. Set π := π + 1.

2 While sπ = 0 do π := π + 1. Set sπ := 0 and π := π − 1.

3 While sπ = 0 do π := π − 1. Set π := π + 1.

4 Set sπ := 1 and stop.

Note that several values of 8 are not specified as they are never used in any computation. The comments on the right-hand side illustrate the computation. Instructions , 2 3 and 4 are used only if the input consists of 1’s only, i.e. n = 2k − 1 for some k ∈ Z+ . We have time(8, s) ≤ 4 size(s) + 5 for all inputs s, so 8 is a polynomial-time Turing machine. In the next section we shall show that the above definition is consistent with our informal definition of a polynomial-time algorithm in Section 1.2: each polynomialtime algorithm in this book can be simulated by a polynomial-time Turing machine.

362

15 NP-Completeness

15.2 Church’s Thesis The Turing machine is the most customary theoretical model for algorithms. Although it seems to be very restricted, it is as powerful as any other reasonable model: the set of computable functions (sometimes also called recursive functions) is always the same. This statement, known as Church’s thesis, is of course too imprecise to be proved. However, there are strong results supporting this claim. For example, each program in a common programming language like C can be modelled by a Turing machine. In particular, all algorithms in this book can be rewritten as Turing machines. This is usually very inconvenient (thus we shall never do it), but theoretically it is possible. Moreover, any function computable in polynomial time by a C program is also computable in polynomial time by a Turing machine (and vice versa). Since it is not a trivial task to implement more complicated programs on a Turing machine we consider as an intermediate step a Turing machine with two strings (tapes) and two independent read-write heads, one for each tape: Definition 15.4. Let A be an alphabet and A¯ := A ∪ {t}. A two-tape Turing machine is defined by a function 8 : {0, . . . , N } × A¯ 2 → {−1, . . . , N } × A¯ 2 × {−1, 0, 1}2 for some N ∈ Z+ . The computation of 8 on input x, where x ∈ A∗ , is the finite or infinite sequence of 5-tuples (n (i) , s (i) , t (i) , π (i) , ρ (i) ) with n (i) ∈ {−1, . . . , N }, s (i) , t (i) ∈ A¯ Z and π (i) , ρ (i) ∈ Z (i = 0, 1, 2, . . .) defined recursively as follows: (0) (0) n (0) := 0. s j := x j for 1 ≤ j ≤ size(x), and s j := t for all j ≤ 0 and (0)

j > size(x). t j := t for all j ∈ Z. π (0) := 1 and ρ (0) := 1. two cases. If If (n (i) , s (i) , t (i) , π (i) , ρ (i) ) is alreadydefined, we distinguish (i) (i) (i) (i) n 6= −1, then let (m, σ, τ, δ, ) := 8 n , sπ (i) , tρ (i) and set n (i+1) := m, (i+1)

sπ (i)

(i+1)

:= σ , s j

Z \ {ρ (i) },

(i)

:= s j

π (i+1)

π (i)

(i+1)

for j ∈ Z \ {π (i) }, tρ (i) ρ (i+1)

(i+1)

:= τ , t j

(i)

:= t j

for

ρ (i)

:= + δ, and := + . If n (i) = −1, then this is the end of the sequence. time(8, x) and output(8, x) are defined as with the one-tape Turing machine. j∈

Turing machines with more than two tapes can be defined analogously, but we shall not need them. Before we show how to perform standard operations with a two-tape Turing machine, let us note that a two-tape Turing machine can be simulated by an ordinary (one-tape) Turing machine. Theorem 15.5. Let A be an alphabet, and let 8 : {0, . . . , N } × (A ∪ {t})2 → {−1, . . . , N } × (A ∪ {t})2 × {−1, 0, 1}2 be a two-tape Turing machine. Then there exists an alphabet B ⊇ A and a (onetape) Turing machine

15.2 Church’s Thesis

363

80 : {0, . . . , N 0 } × (B ∪ {t}) → {−1, . . . , N 0 } × (B ∪ {t}) × {−1, 0, 1} such that output(80 , x) = output(8, x) and time(80 , x) = O(time(8, x))2 for x ∈ A∗ . Proof: We use the letters s and t for the two strings of 8, and denote by π and ρ the positions of the read-write heads, as in Definition 15.4. The string of 80 will be denoted by u and its read-write head position by ψ. We have to encode both strings s, t and both read-write head positions π, ρ in one string u. To make this possible each symbol u j of u is a 4-tuple (s j , p j , t j , r j ), where s j and t j are the corresponding symbols of s and t, and p j , r j ∈ {0, 1} indicate whether the read-write heads of the first and second string currently scans position j; i.e. we have p j = 1 iff π = j, and r j = 1 iff ρ = j. So we define B¯ := ( A¯ × {0, 1} × A¯ × {0, 1}); then we identify a ∈ A¯ with (a, 0, t, 0) to allow inputs from A∗ . The first step of 80 consists in initializing the marks p1 and r1 to 1: 80 (0, (., 0, ., 0)) = (1, (., 1, ., 1), 0)

0 Set π := ψ and ρ := ψ.

Here a dot stands for an arbitrary value (which is not modified). Now we show how to implement a general instruction 8(m, σ, τ ) = (m 0 , σ 0 , τ 0 , δ, ). We first have to find the positions π and ρ. It is convenient to assume that our single read-write head ψ is already at the leftmost of the two positions π and ρ; i.e. ψ = min{π, ρ}. We have to find the other position by scanning the string u to the right, we have to check whether sπ = σ and tρ = τ and, if so, perform the operation required (write new symbols to s and t, move π and ρ, jump to the next instruction). The following block implements one instruction 8(m, σ, τ ) = (m 0 , σ 0 , τ 0 , δ, ) ¯ 2 such blocks, one for each choice of σ and τ . for m = 0; for each m we have | A| 13 the first block for m 0 with , M where The second block for m = 0 starts with , 2 2 0 0 ¯ ¯ M := 12| A| m + 1. All in all we get N = 12(N + 1)| A| . A dot again stands for an arbitrary value which is not modified. Similarly, ζ and ξ stand for an arbitrary element of A¯ \{σ } and A¯ \{τ }, respectively. We assume 10 11 and 12 guarantee that this property that ψ = min{π, ρ} initially; note that , also holds at the end.

364

15 NP-Completeness

80 (1, (ζ, 1, ., .)) 80 (1, (., ., ξ, 1)) 80 (1, (σ, 1, τ, 1)) 80 (1, (σ, 1, ., 0)) 80 (1, (., 0, τ, 1)) 80 (2, (., ., ., 0)) 80 (2, (., ., ξ, 1))

= = = = = = =

80 (2, (., ., τ, 1)) 80 (3, (., ., ., 0)) 80 (4, (., 0, ., .)) 80 (4, (σ, 1, ., .))

= = = =

80 (5, (., 0, ., .))

= (10, (., 1, ., .), −1)

80 (6, (., 0, ., .)) = 80 (6, (ζ, 1, ., .)) = 80 (6, (σ, 1, ., ., )) 80 (7, (., 0, ., .)) 80 (8, (., ., ., 0)) 80 (8, (., ., τ, 1)) 80 (9, (., ., ., 0)) 80 (10, (., ., ., .)) 80 (11, (., 0, ., 0)) 80 (11, (., 1, ., .)) 80 (11, (., 0, ., 1)) 80 (12, (., 0, ., 0)) 80 (12, (., 1, ., .)) 80 (12, (., ., ., 1))

= = = = = = = = = = = =

13 (13, (ζ, 1, ., .), 0)

1 If ψ = π and sψ 6= σ then go to . 13 (13, (., ., ξ, 1), 0) If ψ = ρ and tψ 6= τ then go to . (2, (σ, 1, τ, 1), 0) If ψ = π then go to . 2 (2, (σ, 1, ., 0), 0) (6, (., 0, τ, 1), 0) If ψ = ρ then go to . 6 (2, (., ., ., 0), 1)

2 While ψ 6= ρ do ψ := ψ + 1. (12, (., ., ξ, 1), −1) If tψ 6= τ then set ψ := ψ − 1 12 and go to . 0 0 (3, (., ., τ , 0), ) Set tψ := τ and ψ := ψ + . (4, (., ., ., 1), 1)

3 Set ρ := ψ and ψ := ψ + 1. (4, (., 0, ., .), −1)

4 While ψ 6= π do ψ := ψ − 1. (5, (σ 0 , 0, ., .), δ) Set sψ := σ 0 and ψ := ψ + δ.

5 Set π := ψ and ψ := ψ − 1. 10 Go to . (6, (., 0, ., .), 1)

6 While ψ 6= π do ψ := ψ + 1. (12, (ζ, 1, ., .), −1) If sψ 6= σ then set ψ := ψ − 1 12 and go to . 0 0 (7, (σ , 0, ., .), δ) Set sψ := σ and ψ := ψ + δ. (8, (., 1, ., .), 1)

7 Set π := ψ and ψ := ψ + 1. (8, (., ., ., 0), −1)

8 While ψ 6= ρ do ψ := ψ − 1. 0 (9, (., ., τ , 0), ) Set tψ := τ 0 and ψ := ψ + . (10, (., ., ., 1), −1) 9 Set ρ := ψ and ψ := ψ − 1. 10 Set ψ := ψ − 1. (11, (., ., ., .), −1) 11 While ψ 6∈ {π, ρ} do ψ := ψ + 1. (11, (., 0, ., 0), 1)

M (M, (., 1, ., .), 0) Go to . (M, (., 0, ., 1), 0) 12 While ψ 6∈ {π, ρ} do ψ := ψ − 1. (12, (., 0, ., 0), −1) (13, (., 1, ., .), 0) (13, (., ., ., 1), 0) 2

¯ blocks like the above for Any computation of 80 passes through at most | A| each computation step of 8. The number of computation steps within each block ¯ is a constant and |π − ρ| is bounded by is at most 2|π − ρ| + 10. Since | A| time(8, x) we conclude that the whole computation of 8 is simulated by 80 with 2 O (time(8, x)) steps. Finally we have to clean up the output: replace each symbol (σ, ., ., .) by (σ, 0, t, 0). Obviously this at most doubles the total number of steps.

15.2 Church’s Thesis

365

With a two-tape Turing machine it is not too difficult to implement more complicated instructions, and thus arbitrary algorithms: We use the alphabet A = {0, 1, #} and model an arbitrary number of variables by the string x0 ##1#x1 ##10#x2 ##11#x3 ##100#x4 ##101#x5 ## . . .

(15.1)

which we store on the first tape. Each group (except the first one) contains a binary representation of the index i followed by the value of xi , which we assume to be a binary string. The first variable x0 and the second tape are used only as registers for intermediate results of computation steps. Random access to variables is not possible in constant time with a Turing machine, no matter how many tapes we have. If we simulate an arbitrary algorithm by a two-tape Turing machine, we will have to scan the first tape quite often. Moreover, if the length of the string in one variable changes, the substring to the right has to be shifted. Nevertheless each standard operation (i.e. each elementary step of an algorithm) can be simulated with O(l 2 ) computation steps of a two-tape Turing machine, where l is the current length of the string (15.1). We try to make this clearer with a concrete example. Consider the following instruction: Add to x5 the value of the variable whose index is given by x2 . To get the value of x5 we scan the first tape for the substring ##101#. We copy the substring following this up to #, exclusively, to the second tape. This is easy since we have two separate read-write heads. Then we copy the string from the second tape to x0 . If the new value of x0 is shorter or longer than the old one, we have to shift the rest of the string (15.1) to the left or to the right appropriately. Next we have to search for the variable index that is given by x2 . To do this, we first copy x2 to the second tape. Then we scan the first tape, checking each variable index (comparing it with the string on the second tape bitwise). When we have found the correct variable index, we copy the value of this variable to the second tape. Now we add the number stored in x0 to that on the second tape. A Turing machine for this task, using the standard method, is not hard to design. We can overwrite the number on the second tape by the result while computing it. Finally we have the result on the second tape and copy it back to x5 . If necessary we shift the substring to the right of x5 appropriately. All the above can be done by a two-tape Turing machine in O(l 2 ) computation steps (in fact all but shifting the string (15.1) can be done in O(l) steps). It should be clear that the same holds for all other standard operations, including multiplication and division. By Definition 1.4 an algorithm is said to run in polynomial time if there is a k ∈ N such that the number of elementary steps is bounded by O(n k ) and any number in intermediate computation can be stored with O(n k ) bits, where n is the input size. Moreover, we store at most O(n k ) numbers at any time. Hence we can bound the length of each of the two strings in a two-tape Turing machine simulating such an algorithm by l = O(n k · n k ) = O(n 2k ), and hence its running time by O(n k (n 2k )2 ) = O(n 5k ). This is still polynomial in the input size.

366

15 NP-Completeness

Recalling Theorem 15.5 we may conclude that for any function f there is a polynomial-time algorithm computing f if and only if there is a polynomial-time Turing machine computing f . Hopcroft and Ullman [1979], Lewis and Papadimitriou [1981], and van Emde Boas [1990] provide more details about the equivalence of different machine models. Another common model (which is close to our informal model of Section 1.2) is the RAM machine (cf. Exercise 3) which allows arithmetic operations on integers in constant time. Other models allow only operations on bits (or integers of fixed length) which is more realistic when dealing with large numbers. Obviously, addition and comparison of natural numbers with n bits can be done with O(n) bit operations. For multiplication (and division) the obvious method takes O(n 2 ) bit operations, but the algorithm of Sch¨onhage and Strassen [1971] needs only O(n log n log log n) bit operations for multiplying two n-bit integers, and this was improved further by F¨urer [2007]. This of course implies algorithms for the addition and comparison of rational numbers with the same time complexity. As far as polynomial-time computability is concerned all models are equivalent, but of course the running time measures are quite different. The model of encoding the entire input by 0-1-strings (or strings over any fixed alphabet) does not in principle exclude certain types of real numbers, e.g. algebraic numbers (if x ∈ R is the k-th smallest root of a polynomial p, then x can be coded by listing k and the degree and the coefficients of p). However, there is no way of representing arbitrary real numbers in a digital computer since there are uncountably many real numbers but only countably many 0-1-strings. We take the classical approach and restrict ourselves to rational input in this chapter. We close this section by giving a formal definition of oracle algorithms, based on two-tape Turing machines. We may call an oracle at any stage of the computation; we use the second tape for writing the oracle’s input and reading its output. We introduce a special instruction −2 for oracle calls: Definition 15.6. Let A be an alphabet and A¯ := A ∪ {t}. Let X ⊆ A∗ , and let f (x) ⊆ A∗ be a nonempty language for each x ∈ X . An oracle Turing machine using f is a function 8 : {0, . . . , N } × A¯ 2 → {−2, . . . , N } × A¯ 2 × {−1, 0, 1}2 for some N ∈ Z+ . Its computation is defined as for a two-tape Turing machine, (i) (i) but with one difference: if, for some computation step i, 8 n (i) , sπ (i) , tρ (i) = (−2, σ, τ, δ, ) forn some σ, τ, δ, , othen consider the string on the second tape (i) (i) x ∈ Ak , k := min j ∈ N : t j = t − 1, given by x j := t j for j = 1, . . . , k. If (i+1)

x ∈ X , then the second tape is overwritten by t j

= y j for j = 1, . . . , size(y)

(i+1) tsize(y)+1

and = t for some y ∈ f (x). The rest remains unchanged, and the computation continues with n (i+1) := n (i) + 1 (and stops if n (i) = −1). All definitions with respect to Turing machines can be extended to oracle Turing machines. The output of an oracle is not necessarily unique; hence there can be

15.3 P and NP

367

several possible computations for the same input. When proving the correctness or estimating the running time of an oracle algorithm we have to consider all possible computations, i.e. all choices of the oracle. By the results of this section the existence of a polynomial-time (oracle) algorithm is equivalent to the existence of a polynomial-time (oracle) Turing machine.

15.3 P and NP Most of complexity theory is based on decision problems. Any language L ⊆ {0, 1}∗ can be interpreted as decision problem: given a 0-1-string, decide whether it belongs to L. However, we are more interested in problems like the following:

H AMILTONIAN C IRCUIT Instance:

An undirected graph G.

Question: Has G a Hamiltonian circuit? We will always assume a fixed efficient encoding of the input as a binary string; occasionally we extend our alphabet by other symbols. For example we assume that a graph is given by an adjacency list, and such a list can be coded as a binary string of length O(n log m + m log n), where n and m denote the number of vertices and edges. We always assume an efficient encoding, i.e. one whose length is polynomially bounded by the minimum possible encoding length. Not all binary strings are instances of H AMILTONIAN C IRCUIT but only those representing an undirected graph. For most interesting decision problems the instances are a proper subset of the 0-1-strings. We require that we can decide in polynomial time whether an arbitrary string is an instance or not: Definition 15.7. A decision problem is a pair P = (X, Y ), where X is a language decidable in polynomial time and Y ⊆ X . The elements of X are called instances of P; the elements of Y are yes-instances, those of X \ Y are no-instances. An algorithm for a decision problem (X, Y ) is an algorithm computing the function f : X → {0, 1}, defined by f (x) = 1 for x ∈ Y and f (x) = 0 for x ∈ X \ Y. We give two more examples, the decision problems corresponding to L INEAR P ROGRAMMING and I NTEGER P ROGRAMMING:

L INEAR I NEQUALITIES Instance:

A matrix A ∈ Zm×n and a vector b ∈ Zm .

Question: Is there a vector x ∈ Qn such that Ax ≤ b?

368

15 NP-Completeness

I NTEGER L INEAR I NEQUALITIES A matrix A ∈ Zm×n and a vector b ∈ Zm .

Instance:

Question: Is there a vector x ∈ Zn such that Ax ≤ b? Definition 15.8. The class of all decision problems for which there is a polynomial-time algorithm is denoted by P. In other words, a member of P is a pair (X, Y ) with Y ⊆ X ⊆ {0, 1}∗ where both X and Y are languages decidable in polynomial time. To prove that a problem is in P one usually describes a polynomial-time algorithm. By the results of Section 15.2 there is a polynomial-time Turing machine for each problem in P. By Khachiyan’s Theorem 4.18, L INEAR I NEQUALITIES belongs to P. It is not known whether I NTEGER L INEAR I NEQUALITIES or H AMILTONIAN C IRCUIT belong to P. We shall now introduce another class, called NP, which contains these problems, and in fact most decision problems discussed in this book. We do not insist on a polynomial-time algorithm, but we require that for each yes-instance there is a certificate which can be checked in polynomial time. For example, for the H AMILTONIAN C IRCUIT problem a possible certificate is simply a Hamiltonian circuit. It is easy to check whether a given string is the binary encoding of a Hamiltonian circuit. Note that we do not require a certificate for no-instances. Formally we define: Definition 15.9. A decision problem P = (X, Y ) belongs to NP if there is a polynomial p and a decision problem P 0 = (X 0 , Y 0 ) in P, where o n X 0 := x#c : x ∈ X, c ∈ {0, 1}b p(size(x))c , such that Y =

n

o y ∈ X : there exists a string c ∈ {0, 1}b p(size(y))c with y#c ∈ Y 0 .

Here x#c denotes the concatenation of the string x, the symbol # and the string c. A string c with y#c ∈ Y 0 is called a certificate for y (since c proves that y ∈ Y ). An algorithm for P 0 is called a certificate-checking algorithm. Proposition 15.10. P ⊆ NP. Proof: One can choose p to be identically zero. An algorithm for P 0 just deletes the last symbol of the input “x#” and then applies an algorithm for P. It is not known whether P = NP. In fact, this is the most important open problem in complexity theory. As an example for problems in NP that are not known to be in P we have: Proposition 15.11. H AMILTONIAN C IRCUIT belongs to NP.

15.3 P and NP

369

Proof: For each yes-instance G we take any Hamiltonian circuit of G as a certificate. To check whether a given edge set is in fact a Hamiltonian circuit of a given graph is obviously possible in polynomial time. Proposition 15.12. I NTEGER L INEAR I NEQUALITIES belongs to NP. Proof: As a certificate we just take a solution vector. If there exists a solution, there exists one of polynomial size by Corollary 5.7. The name NP stands for “nondeterministic polynomial”. To explain this we have to define what a nondeterministic algorithm is. This is a good opportunity to define randomized algorithms in general, a concept which has already been mentioned before. The common feature of randomized algorithms is that their computation does not only depend on the input but also on some random bits. Definition 15.13. A randomized algorithm for computing a function f : S → T is an algorithm computing a function g : {s#r : s ∈ S, r ∈ {0, 1}k(s) } → T , where k : S → Z+ . So for each instance s ∈ S the algorithm can use k(s) ∈ Z+ random bits. We measure the running time dependency on size(s) only; thus randomized algorithms running in polynomial time can read only a polynomial number of random bits. Naturally we are interested in such a randomized algorithm only if f and g are related. In the ideal case, if g(s#r ) = f (s) for all s ∈ S and all r ∈ {0, 1}k(s) , we speak of a Las Vegas algorithm. A Las Vegas algorithm always computes the correct result, but the running time may vary for different runs on the same input s. Sometimes even less reliable algorithms are interesting: if there is at least a positive probability p of a correct answer, independent of the instance, i.e. p := inf

s∈S

|{r ∈ {0, 1}k(s) : g(s#r ) = f (s)}| > 0, 2k(s)

then we have a Monte Carlo algorithm. If T = {0, 1}, and for each s ∈ S with f (s) = 0 we have g(s#r ) = 0 for all r ∈ {0, 1}k(s) , then we have a randomized algorithm with one-sided error. If in addition for each s ∈ S with f (s) = 1 there is at least one r ∈ {0, 1}k(s) with g(s#r ) = 1, then the algorithm is called a nondeterministic algorithm. Alternatively a randomized algorithm can be regarded as an oracle algorithm where the oracle produces a random bit (0 or 1) whenever called. A nondeterministic algorithm for a decision problem always answers “no” for a no-instance, and for each yes-instance there is a chance that it answers “yes”. The following observation is easy: Proposition 15.14. A decision problem belongs to NP if and only if it has a polynomial-time nondeterministic algorithm.

370

15 NP-Completeness

Proof: Let P = (X, Y ) be a decision problem in NP, and let P 0 = (X 0 , Y 0 ) be defined as in Definition 15.9. Then a polynomial-time algorithm for P 0 is in fact also a nondeterministic algorithm for P: the unknown certificate is simply replaced by random bits. Since the number of random bits is bounded by a polynomial in size(x), x ∈ X , so is the running time of the algorithm. Conversely, if P = (X, Y ) has a polynomial-time nondeterministic algorithm using k(x) random bits for instance x, then thereis a polynomial p such that k(x) ≤ p(size(x)) for each instance x. We define X 0 := x#c : x ∈ X, c ∈ {0, 1}b p(size(x))c and Y 0 := x#c ∈ X 0 : g(x#r ) = 1, r consists of the first k(x) bits of c . Then by the definition of nondeterministic algorithms we have (X 0 , Y 0 ) ∈ P and o n Y = y ∈ X : there exists a string c ∈ {0, 1}b p(size(x))c with y#c ∈ Y 0 . Most decision problems encountered in combinatorial optimization belong to NP. For many of them it is not known whether they have a polynomial-time algorithm. However, one can say that certain problems are not easier than others. To make this precise we introduce the important concept of polynomial reductions. Definition 15.15. Let P1 and P2 = (X, Y ) be decision problems. Let f : X → {0, 1} with f (x) = 1 for x ∈ Y and f (x) = 0 for x ∈ X \ Y . We say that P1 polynomially reduces to P2 if there exists a polynomial-time oracle algorithm for P1 using f . The following observation is the main reason for this concept: Proposition 15.16. If P1 polynomially reduces to P2 and there is a polynomialtime algorithm for P2 , then there is a polynomial-time algorithm for P1 . Proof: Let A2 be an algorithm for P2 with time(A2 , y) ≤ p2 (size(y)) for all instances y of P2 , and let f (y) := output(A2 , y). Let A1 be an oracle algorithm for P1 using f with time(A1 , x) ≤ p1 (size(x)) for all instances x of P1 . Then replacing the oracle calls in A1 by subroutines equivalent to A2 yields an algorithm A3 for P1 . For any instance x of P1 with size(x) = n we have time(A3 , x) ≤ p1 (n)· p2 ( p1 (n)): there can be at most p1 (n) oracle calls in A1 , and none of the instances of P2 produced by A1 can be longer than p1 (n). Since we can choose p1 and p2 to be polynomials we conclude that A3 is a polynomial-time algorithm. The theory of NP-completeness is based on a special kind of polynomial-time reduction: Definition 15.17. Let P1 = (X 1 , Y1 ) and P2 = (X 2 , Y2 ) be decision problems. We say that P1 polynomially transforms to P2 if there is a function f : X 1 → X 2 computable in polynomial time such that f (x1 ) ∈ Y2 for all x1 ∈ Y1 and f (x1 ) ∈ X 2 \ Y2 for all x1 ∈ X 1 \ Y1 .

15.4 Cook’s Theorem

371

In other words, yes-instances are transformed to yes-instances, and no-instances are transformed to no-instances. Obviously, if a problem P1 polynomially transforms to P2 , then P1 also polynomially reduces to P2 . Polynomial transformations are sometimes called Karp reductions, while general polynomial reductions are also known as Turing reductions. Both are easily seen to be transitive. Definition 15.18. A decision problem P ∈ NP is called NP-complete if all other problems in NP polynomially transform to P. By Proposition 15.16 we know that if there is a polynomial-time algorithm for any NP-complete problem, then P = NP. Of course, the above definition would be meaningless if no NP-complete problems existed. The next section consists of a proof that there is an NP-complete problem.

15.4 Cook’s Theorem In his pioneering work, Cook [1971] proved that a certain decision problem, called S ATISFIABILITY, is in fact NP-complete. We need some definitions: Definition 15.19. Let X be a finite set of Boolean variables. A truth assignment . for X is a function T : X → {true, false}. We extend T to the set L := X ∪ {x : x ∈ X } by setting T (x) := true if T (x) := false and vice versa (x can be regarded as the negation of x). The elements of L are called the literals over X . A clause over X is a set of literals over X . A clause represents the disjunction of those literals and is satisfied by a truth assignment iff at least one of its members is true. A family of clauses over X is satisfiable iff there is some truth assignment simultaneously satisfying all of its clauses. Since we consider the conjunction of disjunctions of literals, we also speak of Boolean formulas in conjunctive normal form. For example, the family {{x1 , x2 }, {x2 , x3 }, {x1 , x2 , x3 }, {x1 , x3 }} corresponds to the Boolean formula (x1 ∨ x2 ) ∧ (x2 ∨ x3 ) ∧ (x1 ∨ x2 ∨ x3 ) ∧ (x1 ∨ x3 ). It is satisfiable as the truth assignment T (x1 ) := true, T (x2 ) := false and T (x3 ) := true shows. We are now ready to specify the satisfiability problem:

S ATISFIABILITY Instance:

A set X of variables and a family Z of clauses over X .

Question: Is Z satisfiable? Theorem 15.20. (Cook [1971]) S ATISFIABILITY is NP-complete. Proof: S ATISFIABILITY belongs to NP because a satisfying truth assignment serves as a certificate for any yes-instance, which of course can be checked in polynomial time.

372

15 NP-Completeness

Let now P = (X, Y ) be any other problem in NP. We have to show that P polynomially transforms to S ATISFIABILITY. By Definition 15.9 and a decision problem P 0 = (X 0 , Y 0 ) there is a polynomial pb p(size(x))c 0 in P, where X := x#c : x ∈ X, c ∈ {0, 1} and o n Y = y ∈ X : there exists a string c ∈ {0, 1}b p(size(x))c with y#c ∈ Y 0 . Let 8 : {0, . . . , N } × A¯ → {−1, . . . , N } × A¯ × {−1, 0, 1} be a polynomial-time Turing machine for P 0 with alphabet A; let A¯ := A ∪ {t}. Let q be a polynomial such that time(8, x#c) ≤ q(size(x#c)) for all instances x#c ∈ X 0 . Note that size(x#c) = size(x) + 1 + b p(size(x))c. We will now construct a collection Z(x) of clauses over some set V (x) of Boolean variables for each x ∈ X , such that Z(x) is satisfiable if and only if x ∈ Y. We abbreviate Q := q(size(x) + 1 + b p(size(x))c). Q is an upper bound on the length of any computation of 8 on input x#c, for any c ∈ {0, 1}b p(size(x))c . V (x) contains the following Boolean variables: • •

¯ a variable vi jσ for all 0 ≤ i ≤ Q, −Q ≤ j ≤ Q and σ ∈ A; a variable wi jn for all 0 ≤ i ≤ Q, −Q ≤ j ≤ Q and −1 ≤ n ≤ N .

The intended meaning is: vi jσ indicates whether at time i (i.e. after i steps of the computation) the j-th position of the string contains the symbol σ . wi jn indicates whether at time i the j-th position of the string is scanned and the n-th instruction is executed. So if (n (i) , s (i) , π (i) )i=0,1,... is a computation of 8 then we intend to set vi jσ (i) to true iff s j = σ and wi jn to true iff π (i) = j and n (i) = n. The collection Z(x) of clauses to be constructed will be satisfiable if and only if there is a string c with output(8, x#c) = 1. Z(x) contains the following clauses to model the following conditions: At any time each position of the string contains a unique symbol: ¯ • {vi jσ : σ ∈ A} for 0 ≤ i ≤ Q and −Q ≤ j ≤ Q; • {vi jσ , vi jτ } for 0 ≤ i ≤ Q, −Q ≤ j ≤ Q and σ, τ ∈ A¯ with σ 6= τ . At any time a unique position of the string is scanned and a single instruction is executed: • {wi jn : −Q ≤ j ≤ Q, −1 ≤ n ≤ N } for 0 ≤ i ≤ Q; • {wi jn , wi j 0 n 0 } for 0 ≤ i ≤ Q, −Q ≤ j, j 0 ≤ Q and −1 ≤ n, n 0 ≤ N with ( j, n) 6= ( j 0 , n 0 ). The algorithm starts correctly with input x#c for some c ∈ {0, 1}b p(size(x))c : • {v0, j,x j } for 1 ≤ j ≤ size(x); • {v0,size(x)+1,# };

15.4 Cook’s Theorem

373

• {v0,size(x)+1+ j,0 , v0,size(x)+1+ j,1 } for 1 ≤ j ≤ b p(size(x))c; • {v0, j,t } for −Q ≤ j ≤ 0 and size(x) + 2 + b p(size(x))c ≤ j ≤ Q; • {w010 }. The algorithm works correctly: •

{vi jσ , wi jn , vi+1, j,τ }, {vi jσ , wi jn , wi+1, j+δ,m } for 0 ≤ i < Q, −Q ≤ j ≤ Q, σ ∈ A¯ and 0 ≤ n ≤ N , where 8(n, σ ) = (m, τ, δ). When the algorithm reaches instruction −1, it stops:

•

{wi, j,−1 , wi+1, j,−1 }, {wi, j,−1 , vi, j,σ , vi+1, j,σ } ¯ for 0 ≤ i < Q, −Q ≤ j ≤ Q and σ ∈ A. Positions not being scanned remain unchanged:

•

¯ −1 ≤ n ≤ N and {vi jσ , wi j 0 n , vi+1, j,σ } for 0 ≤ i ≤ Q, σ ∈ A, 0 0 −Q ≤ j, j ≤ Q with j 6= j . The output of the algorithm is 1:

•

{v Q,1,1 }, {v Q,2,t }.

The encoding length of Z(x) is O(Q 3 log Q): there are O(Q 3 ) occurrences of literals, whose indices require O(log Q) space. Since Q depends polynomially on size(x) we conclude that there is a polynomial-time algorithm which, given x, constructs Z(x). Note that p, 8 and q are fixed and not part of the input of this algorithm. It remains to show that Z(x) is satisfiable if and only if x ∈ Y . If Z(x) is satisfiable, consider a truth assignment T satisfying all clauses. Let c ∈ {0, 1}b p(size(x))c with c j = 1 for all j with T (v0,size(x)+1+ j,1 ) = true and c j = 0 otherwise. By the above construction the variables reflect the computation of 8 on input x#c. Hence we may conclude that output(8, x#c) = 1. Since 8 is a certificate-checking algorithm, this implies that x is a yes-instance. Conversely, if x ∈ Y , let c be any certificate for x. Let (n (i) , s (i) , π (i) )i=0,1,...,m (i) be the computation of 8 on input x#c. Then we define T (vi, j,σ ) := true iff s j = σ and T (wi, j,n ) = true iff π (i) = j and n (i) = n. For i := m + 1, . . . , Q we set T (vi, j,σ ) := T (vi−1, j,σ ) and T (wi, j,n ) := T (wi−1, j,n ) for all j, n and σ . Then T is a truth assignment satisfying Z(x), completing the proof. S ATISFIABILITY is not the only NP-complete problem; we will encounter many others in this book. Now that we already have one NP-complete problem at hand, it is much easier to prove NP-completeness for another problem. To show that a certain decision problem P is NP-complete, we shall just prove that P ∈ NP and that S ATISFIABILITY (or any other problem which we know already to be NP-complete) polynomially transforms to P. Since polynomial transformability is transitive, this will be sufficient. The following restriction of S ATISFIABILITY will prove very useful for several NP-completeness proofs:

374

15 NP-Completeness

3S AT A set X of variables and a collection Z of clauses over X , each containing exactly three literals. Question: Is Z satisfiable?

Instance:

To show NP-completeness of 3S AT we observe that any clause can be replaced equivalently by a set of 3S AT-clauses: Proposition 15.21. Let X be a set of variables and Z a clause over X with k literals. Then there is a set Y of at most max{k .− 3, 2} new variables and a family Z 0 of at most max{k − 2, 4} clauses over X ∪ Y such that each element of Z 0 has exactly three literals, and for each family W of clauses over X we have that W ∪ {Z } is satisfiable if and only if W ∪ Z 0 is satisfiable. Moreover, such a family Z 0 can be computed in O(k) time. Proof: If Z has three literals, we set Z 0 := {Z }. If Z has more than three literals, say Z = {λ1 , . . . , λk }, we choose a set Y = {y1 , . . . , yk−3 } of k − 3 new variables and set Z 0 := {λ1 , λ2 , y1 }{y1 , λ3 , y2 }, {y2 , λ4 , y3 }, . . . , {yk−4 , λk−2 , yk−3 }, {yk−3 , λk−1 , λk } . If Z = {λ1 , λ2 }, we choose a new variable y1 (Y := {y1 }) and set Z 0 := {{λ1 , λ2 , y1 }, {λ1 , λ2 , y1 }} . If Z = {λ1 }, we choose a set Y = {y1 , y2 } of two new variables and set Z 0 := {{λ1 , y1 , y2 }, {λ1 , y1 , y2 }, {λ1 , y1 , y2 }, {λ1 , y1 , y2 }}. Observe that in each case Z can be equivalently replaced by the clauses in Z 0 in any instance of S ATISFIABILITY. Theorem 15.22. (Cook [1971]) 3S AT is NP-complete. Proof: As a restriction of S ATISFIABILITY, 3S AT is certainly in NP. We now show that S ATISFIABILITY polynomially transforms to 3S AT. Consider any collection Z of clauses Z 1 , . . . , Z m . We shall construct a new collection Z 0 of clauses with three literals per clause such that Z is satisfiable if and only if Z 0 is satisfiable. To do this, we replace each clause Z i by an equivalent set of clauses, each with three literals. This is possible in linear time by Proposition 15.21. If we restrict each clause to consist of just two literals, the problem (called 2S AT) can be solved in linear time (Exercise 7).

15.5 Some Basic NP-Complete Problems

375

15.5 Some Basic NP-Complete Problems Karp [1972] discovered the wealth of consequences of Cook’s work for combinatorial optimization problems. As a start, we consider the following problem:

S TABLE S ET Instance:

A graph G and an integer k.

Question: Is there a stable set of k vertices? Theorem 15.23. (Karp [1972]) S TABLE S ET is NP-complete. Proof: Obviously, S TABLE S ET ∈ NP. We show that S ATISFIABILITY polynomially transforms to S TABLE S ET. Let Z be a collection of clauses Z 1 , . . . , Z m with Z i = {λi1 , . . . , λiki } (i = 1, . . . , m), where the λi j are literals over some set X of variables. We shall construct a graph G such that G has a stable set of size m if and only if there is a truth assignment satisfying all m clauses. x1

x3

x1

x1

x2

x2

x3

x3

x2

x3

Fig. 15.1.

For each clause Z i , we introduce a clique of ki vertices according to the literals in this clause. Vertices corresponding to different clauses are connected by an edge if and only if the literals contradict each other. Formally, let V (G) := {v i j : 1 ≤ i ≤ m, 1 ≤ j ≤ ki } and E(G) := {v i j , v kl } : (i = k and j = l) or (λi j = x and λkl = x for some x ∈ X ) . See Figure 15.1 for an example (m = 4, Z 1 = {x1 , x2 , x3 }, Z 2 = {x1 , x3 }, Z 3 = {x2 , x3 } and Z 4 = {x1 , x2 , x3 }).

376

15 NP-Completeness

Suppose G has a stable set of size m. Then its vertices specify pairwise compatible literals belonging to different clauses. Setting each of these literals to be true (and setting variables not occurring there arbitrarily) we obtain a truth assignment satisfying all m clauses. Conversely, if some truth assignment satisfies all m clauses, then we choose a literal which is true out of each clause. The set of corresponding vertices then defines a stable set of size m in G. It is essential that k is part of the input: for each fixed k it can be decided in O(n k ) time whether a given graph with n vertices has a stable set of size k (simply by testing all vertex sets with k elements). Two interesting related problems are the following:

V ERTEX C OVER Instance:

A graph G and an integer k.

Question: Is there a vertex cover of cardinality k?

C LIQUE Instance:

A graph G and an integer k.

Question: Has G a clique of cardinality k? Corollary 15.24. (Karp [1972]) V ERTEX C OVER and C LIQUE are NP-complete. Proof: By Proposition 2.2, S TABLE S ET polynomially transforms to both V ER TEX C OVER and C LIQUE . We now turn to the famous Hamiltonian circuit problem (already defined in Section 15.3). Theorem 15.25. (Karp [1972]) H AMILTONIAN C IRCUIT is NP-complete. Proof: Membership in NP is obvious. We prove that 3S AT polynomially transforms to H AMILTONIAN C IRCUIT. Given a collection Z of clauses Z 1 , . . . , Z m over X = {x1 , . . . , xn }, each clause containing three literals, we shall construct a graph G such that G is Hamiltonian iff Z is satisfiable. We first define two gadgets which will appear several times in G. Consider the graph shown in Figure 15.2(a), which we call A. We assume that it is a subgraph of G and no vertex of A except u, u 0 , v, v 0 is incident to any other edge of G. Then any Hamiltonian circuit of G must traverse A in one of the ways shown in Figure 15.3(a) and (b). So we can replace A by two edges with the additional restriction that any Hamiltonian circuit of G must contain exactly one of them (Figure 15.2(b)). Now consider the graph B shown in Figure 15.4(a). We assume that it is a subgraph of G, and no vertex of B except u and u 0 is incident to any other edge

15.5 Some Basic NP-Complete Problems (a)

377

(b)

u

u

u

u

A

v

v

v

v

Fig. 15.2. (a)

(b)

u

u

u

u

v

v

v

v

Fig. 15.3. (a)

(b) u

u

e1

e2

B

e3 u

u

Fig. 15.4.

of G. Then no Hamiltonian circuit of G traverses all of e1 , e2 , e3 . Moreover, one easily checks that for any S ⊂ {e1 , e2 , e3 } there is a Hamiltonian path from u to u in B that contains S but none of {e1 , e2 , e3 } \ S. We represent B by the symbol shown in Figure 15.4(b).

378

15 NP-Completeness

B

A

A A A B

A A A A

B

A

Fig. 15.5.

We are now able to construct G. For each clause, we introduce a copy of B, joined one after another. Between the ﬁrst and the last copy of B, we insert two vertices for each variable, all joined one after another. We then double the edges between the two vertices of each variable x; these two edges will correspond to x and x, respectively. The edges e1 , e2 , and e3 in each copy of B are now connected via a copy of A to the edges corresponding to the ﬁrst, second, and third literal of the corresponding clause, respectively. These constructions are done consecutively: when introducing a copy of subgraph A at an edge e = {u, v} corresponding to a literal, the edge incident to u in Figure 15.2(a) takes the role of e: it is now the edge corresponding to that literal. The overall construction is illustrated by Figure 15.5 with the example {{x1 , x2 , x3 }, {x1 , x2 , x3 }, {x1 , x2 , x3 }}. Now we claim that G is Hamiltonian if and only if Z is satisﬁable. Let C be a Hamiltonian circuit. We deﬁne a truth assignment by setting a literal true iff C

15.5 Some Basic NP-Complete Problems

379

contains the corresponding edge. By the properties of the gadgets A and B each clause contains a literal that is true. Conversely, any satisfying truth assignment defines a set of edges corresponding to literals that are true. Since each clause contains a literal that is true this set of edges can be completed to a tour in G. This proof is essentially due to Papadimitriou and Steiglitz [1982]. The problem of deciding whether a given graph contains a Hamiltonian path is also NP-complete (Exercise 14(a)). Moreover, one can easily transform the undirected versions to the directed Hamiltonian circuit or Hamiltonian path problem by replacing each undirected edge by a pair of oppositely directed edges. Thus the directed versions are also NP-complete. There is another fundamental NP-complete problem:

3-D IMENSIONAL M ATCHING (3DM) Disjoint sets U, V, W of equal cardinality and T ⊆ U × V × W .

Instance:

Question: Is there a subset M of T with |M| = |U | such that for distinct (u, v, w), (u 0 , v 0 , w0 ) ∈ M one has u 6= u 0 , v 6= v 0 and w 6= w 0 ? Theorem 15.26. (Karp [1972]) 3DM is NP-complete. Proof: Membership in NP is obvious. We shall polynomially transform S ATISFIABILITY to 3DM. Given a collection Z of clauses Z 1 , . . . , Z m over X = {x1 , . . . , xn }, we construct an instance (U, V, W, T ) of 3DM which is a yesinstance if and only if Z is satisfiable. We define: j

U

:= {xi , xi j : i = 1, . . . , n; j = 1, . . . , m}

V

:= {ai : i = 1, . . . , n; j = 1, . . . , m} ∪ {v j : j = 1, . . . , m}

j

j

∪ {ck : k = 1, . . . , n − 1; j = 1, . . . , m} W

j

:= {bi : i = 1, . . . , n; j = 1, . . . , m} ∪ {w j : j = 1, . . . , m} j

∪ {dk : k = 1, . . . , n − 1; j = 1, . . . , m} j

j

j

j+1

T1 := {(xi , ai , bi ), (xi j , ai

j

, bi ) : i = 1, . . . , n; j = 1, . . . , m},

where aim+1 := ai1 j

T2 := {(xi , v j , w j ) : i = 1, . . . , n; j = 1, . . . , m; xi ∈ Z j } ∪ {(xi j , v j , w j ) : i = 1, . . . , n; j = 1, . . . , m; xi ∈ Z j } j

j

j

j

j

T3 := {(xi , ck , dk ), (xi j , ck , dk ) : i = 1, . . . , n; j = 1, . . . , m; k = 1, . . . , n −1} T := T1 ∪ T2 ∪ T3 . For an illustration of this construction, see Figure 15.6. Here m = 2, Z 1 = {x1 , x2 }, Z 2 = {x1 , x2 }. Each triangle corresponds to an element of T1 ∪ T2 . The j j elements ck , dk and the triples in T3 are not shown.

380

15 NP-Completeness w1

v1

x1 1 a12

x2 1 b11

x12

a22 x11

b12

b21

x22

a11

x21 b22

x1 2

a21 x2 2

w2

v2

Fig. 15.6.

Suppose (U, V, W, T ) is a yes-instance, so let M ⊆ T be a solution. Since the j j ai ’s and bi ’s appear only in elements of T1 , for each i we have either M ∩ T1 ⊇ j j j j+1 j {(xi , ai , bi ) : j = 1, . . . , m} or M ∩ T1 ⊇ {(xi j , ai , bi ) : j = 1, . . . , m}. In the ﬁrst case we set xi to false, in the second case to true. Furthermore, for each clause Z j we have (λ j , v j , w j ) ∈ M for some literal λ ∈ Z j . Since λ j does not appear in any element of M ∩ T1 this literal is true; hence we have a satisfying truth assignment. Conversely, a satisfying truth assignment suggests a set M1 ⊆ T1 of cardinality nm and a set M2 ⊆ T2 of cardinality m such that for distinct (u, v, w), (u , v , w ) ∈ M1 ∪ M2 we have u = u , v = v and w = w . It is easy to complete M1 ∪ M2 by (n − 1)m elements of T3 to a solution of the 3DM instance. A problem which looks simple but is not known to be solvable in polynomial time is the following:

S UBSET-S UM Instance:

Natural numbers c1 , . . . , cn , K .

Question: Is there a subset S ⊆ {1, . . . , n} such that

j∈S c j

=K ?

Corollary 15.27. (Karp [1972]) S UBSET-S UM is NP-complete. Proof: It is obvious that S UBSET-S UM is in NP. We prove that 3DM polynomially transforms to S UBSET-S UM. So let (U, V, W, T ) be an instance of 3DM. W.l.o.g. let U ∪ V ∪ W = {u 1 , . . . , u 3m }. We write S := {{a, b, c} : (a, b, c) ∈ T } and S = {s1 , . . . , sn }.

15.5 Some Basic NP-Complete Problems

381

Define cj

:=

X

(n + 1)i−1

( j = 1, . . . , n), and

u i ∈s j

K

:=

3m X

(n + 1)i−1 .

i=1

Written in (n + 1)-ary form, the number c j can be regarded as the incidence vector of s j ( j = 1, . . . , n), and K consists of 1’s only. Therefore P each solution to the 3DM instance corresponds to a subset R of S such that s j ∈R c j = K , and vice versa. Moreover, size(c j ) ≤ size(K ) = O(m log n), so the above is indeed a polynomial transformation. An important special case is the following problem:

PARTITION Instance:

Natural numbers c1 , . . . , cn .

Question: Is there a subset S ⊆ {1, . . . , n} such that

P

j∈S c j

=

P

j ∈S / cj

?

Corollary 15.28. (Karp [1972]) PARTITION is NP-complete. Proof: We show that S UBSET-S UM polynomially transforms to PARTITION. So be an instance of S UBSET-S UM. We add an element cn+1 := let P c1 , . . . , c n , K n ci − 2K (unless this number is zero) and have an instance c1 , . . . , cn+1 of i=1 PARTITION. Pn Case 1: 2K ≤ i=1 ci . Then for any I ⊆ {1, . . . , n} we have X X X ci = K if and only if ci = ci . i∈I

Case 2:

i∈I ∪{n+1}

i∈{1,...,n}\I

Pn

2K > i=1 ci . Then for any I ⊆ {1, . . . , n} we have X X X ci = K if and only if ci = ci . i∈I

i∈I

i∈{1,...,n+1}\I

In both cases we have constructed a yes-instance of PARTITION if and only if the original instance of S UBSET-S UM is a yes-instance. We finally note: Theorem 15.29. I NTEGER L INEAR I NEQUALITIES is NP-complete. Proof: We already mentioned the membership in NP in Proposition 15.12. Any of the above problems can easily be formulated as an instance of I NTEGER L INEAR I NEQUALITIES. For example a PARTITION instance c1 , . . . , cn is a yes-instance if and only if {x ∈ Zn : 0 ≤ x ≤ 1l, 2c> x = c> 1l} is nonempty.

382

15 NP-Completeness

15.6 The Class coNP The definition of NP is not symmetric with respect to yes-instances and no-instances. For example, it is an open question whether the following problem belongs to NP: given a graph G, is it true that G is not Hamiltonian? We introduce the following definitions: Definition 15.30. For a decision problem P = (X, Y ) we define its complement to be the decision problem (X, X \Y ). The class coNP consists of all problems whose complements are in NP. A decision problem P ∈ coNP is called coNP-complete if all other problems in coNP polynomially transform to P. Trivially, the complement of a problem in P is also in P. On the other hand, NP 6= coNP is commonly conjectured (though not proved). For this conjecture the NP-complete problems play a special role: Theorem 15.31. A decision problem is coNP-complete if and only if its complement is NP-complete. Unless NP = coNP, no coNP-complete problem is in NP. Proof: The first statement follows directly from the definition. Suppose P = (X, Y ) ∈ NP is a coNP-complete problem. Let Q = (V, W ) be an arbitrary problem in coNP. We show that Q ∈ NP. Since P is coNP-complete, Q polynomially transforms to P. So there is a polynomial-time algorithm which transforms any instance v of Q to an instance x = f (v) of P such that x ∈ Y if and only if v ∈ W . Note that size(x) ≤ p(size(v)) for some fixed polynomial p. Since P ∈ NP, there exists a polynomial q and a decision problem P 0 = (X 0 , Y 0 ) 0 bq(size(x))c , such that in P, where X := x#c : x ∈ X, c ∈ {0, 1} o n Y = y ∈ X : there exists a string c ∈ {0, 1}bq(size(y))c with y#c ∈ Y 0 (cf. Definition 15.9). We define a decision problem (V 0 , W 0 ) by V 0 := v#c : v ∈ V, c ∈ {0, 1}bq( p(size(v)))c , and v#c ∈ W 0 if and only if f (v)#c0 ∈ Y 0 where c0 consists of the first bq(size( f (v)))c components of c. Observe that (V 0 , W 0 ) ∈ P. Therefore, by definition, Q ∈ NP. We conclude coNP ⊆ NP and hence, by symmetry, NP = coNP. If one can show that a problem is in NP ∩ coNP, we say that the problem has a good characterization (Edmonds [1965]). This means that for yes-instances as well as for no-instances there are certificates that can be checked in polynomial time. Theorem 15.31 indicates that a problem with a good characterization is probably not NP-complete. To give examples, Proposition 2.9, Theorem 2.24, and Proposition 2.27 provide good characterizations for the problems of deciding whether a given graph is acyclic, whether it has an Eulerian walk, and whether it is bipartite, respectively. Of course, this is not very interesting since all these problems can be solved easily in polynomial time. But consider the decision version of L INEAR P ROGRAMMING:

15.6 The Class coNP

383

Theorem 15.32. L INEAR I NEQUALITIES is in NP ∩ coNP. Proof: This immediately follows from Theorem 4.4 and Corollary 3.24.

Of course, this theorem also follows from any polynomial-time algorithm for L INEAR P ROGRAMMING, e.g. Theorem 4.18. However, before the E LLIPSOID M ETHOD had been discovered, Theorem 15.32 was the only theoretical evidence that L INEAR I NEQUALITIES is probably not NP-complete. This gave hope to ﬁnd a polynomial-time algorithm for L INEAR P ROGRAMMING (which can be reduced to L INEAR I NEQUALITIES by Proposition 4.16); a justiﬁed hope as we know today. The following famous problem has a similar history:

P RIME Instance:

A number n ∈ N (in its binary representation).

Question: Is n a prime? It is obvious that P RIME belongs to coNP. Pratt [1975] proved that P RIME also belongs to NP. Finally, Agrawal, Kayal and Saxena [2004] proved that P RIME ∈ P by ﬁnding a surprisingly simple O(log7.5+ n)-algorithm (for any > 0). Before, the best known deterministic algorithm for P RIME was due to Adleman, Pomerance and Rumely [1983], running in O (log n)c log log log n time for some constant c. Since the input size is O(log n), this is not polynomial.

NP-complete

coNP-complete

NP ∩ coNP NP

coNP

P

Fig. 15.7.

We close this section by sketching the inclusions of NP and coNP (Figure 15.7). Ladner [1975] showed that, unless P = NP, there are problems in NP \ P that are not NP-complete. However, until the P = NP conjecture is resolved, it is still possible that all regions drawn in Figure 15.7 collapse to one.

384

15 NP-Completeness

15.7 NP-Hard Problems Now we extend our results to optimization problems. We start by formally defining the type of optimization problems we are interested in: Definition 15.33. A (discrete) optimization problem is a quadruple P = (X, (Sx )x∈X , c, goal), where X is a language over {0, 1} decidable in polynomial time; Sx is a subset of {0, 1}∗ for each x ∈ X ; there exists a polynomial p with size(y) ≤ p(size(x)) for all y ∈ Sx and all x ∈ X , and the languages {(x, y) : x ∈ X, y ∈ Sx } and {x ∈ X : Sx = ∅} are decidable in polynomial time; • c : {(x, y) : x ∈ X, y ∈ Sx } → Q is a function computable in polynomial time; and • goal ∈ {max, min}. • •

The elements of X are called instances of P. For each instance x, the elements of Sx are called feasible solutions of x. We write OPT(x) := goal{c(x, y) : y ∈ Sx }. An optimum solution of x is a feasible solution y of x with c(x, y) = OPT(x). An algorithm for an optimization problem (X, (Sx )x∈X , c, goal) is an algorithm A which computes for each input x ∈ X with Sx 6= ∅ a feasible solution y ∈ Sx . We sometimes write A(x) := c(x, y). If A(x) = OPT(x) for all x ∈ X with Sx 6= ∅, then A is an exact algorithm. Depending on the context, c(x, y) is often called the cost, the weight, the profit or the length of y. If c is nonnegative, then we say that the optimization problem has nonnegative weights. The values of c are rational numbers; we assume an encoding into binary strings as usual. The concept of polynomial reductions easily extends to optimization problems: a (decision or optimization) problem polynomially reduces to an optimization problem P = (X, (Sx )x∈X , c, goal) if it has an exact polynomial-time oracle algorithm using the function f defined by f (x) = {y ∈ Sx : c(x, y) = OPT(x)} for all x ∈ X with Sx 6= ∅. Now we can define: Definition 15.34. An optimization problem or decision problem P is called NPhard if all problems in NP polynomially reduce to P. Note that the definition is symmetric: a decision problem is NP-hard if and only if its complement is. NP-hard problems are at least as hard as the hardest problems in NP. But some may be harder than any problem in NP. A problem which polynomially reduces to some problem in NP is called NP-easy. A problem which is both NP-hard and NP-easy is NP-equivalent. In other words, a problem is NPequivalent if and only if it is polynomially equivalent to S ATISFIABILITY, where two problems P and Q are called polynomially equivalent if P polynomially reduces to Q, and Q polynomially reduces to P. We note: Proposition 15.35. Let P be an NP-equivalent problem. Then P has an exact polynomial-time algorithm if and only if P = NP.

15.7 NP-Hard Problems

385

Of course, all NP-complete problems and all coNP-complete problems are NP-equivalent. Almost all problems discussed in this book are NP-easy since they polynomially reduce to I NTEGER P ROGRAMMING; this is usually a trivial observation which we do not even mention. On the other hand, most problems we discuss from now on are also NP-hard, and we shall usually prove this by describing a polynomial reduction from an NP-complete problem. As a first example we consider M AX -2S AT: given an instance of S ATISFIABILITY with exactly two literals per clause, find a truth assignment that maximizes the number of satisfied clauses. Theorem 15.36. (Garey, Johnson and Stockmeyer [1976]) M AX -2S AT is NPhard. Proof: By reduction from 3S AT. Given an instance I of 3S AT with clauses C1 , . . . , Cm , we construct an instance I 0 of M AX -2S AT by adding new variables y1 , z 1 , . . . , ym , z m and replacing each clause Ci = {λ1 , λ2 , λ3 } by the fourteen clauses {λ1 , z i }, {λ1 , z¯ i }, {λ2 , z i }, {λ2 , z¯ i }, {λ3 , z i }, {λ3 , z¯ i }, {yi , z i }, {yi , z¯ i }, {λ1 , y¯i }, {λ2 , y¯i }, {λ3 , y¯i }, {λ¯ 1 , λ¯ 2 }, {λ¯ 1 , λ¯ 3 }, {λ¯ 2 , λ¯ 3 }. Note that no truth assignment satisfies more than 11 of these 14 clauses. Moreover, if 11 of these clauses are satisfied, then at least one of λ1 , λ2 , λ3 must be true. On the other hand, if one of λ1 , λ2 , λ3 is true we can set yi := λ1 ∧ λ2 ∧ λ3 and z i := true in order to satisfy 11 of these clauses. We conclude that I has a truth assignment satisfying all m clauses if and only if I 0 has a truth assignment that satisfies 11m clauses. It is an open question whether each NP-hard decision problem P ∈ NP is NP-complete (recall the difference between polynomial reduction and polynomial transformation; Definitions 15.15 and 15.17). Exercises 17 and 18 discuss two NP-hard decision problems that appear not to be in NP. Unless P = NP there is no exact polynomial-time algorithm for any NP-hard problem. There might, however, be a pseudopolynomial algorithm: Definition 15.37. Let P be a decision problem or an optimization problem such that each instance x consists of a list of nonnegative integers. We denote by largest(x) the largest of these integers. An algorithm for P is called pseudopolynomial if its running time is bounded by a polynomial in size(x) and largest(x). For example there is a trivial pseudopolynomial algorithm for P RIME which divides the natural number n to be tested for primality by each integer from 2 to √ b nc. Another example is: Theorem 15.38. There is a pseudopolynomial algorithm for S UBSET-S UM.

386

15 NP-Completeness

Proof: Given an instance c1 , . . . , cn , K of S UBSET-S UM, we construct a digraph G with vertex set {0, . . . , n} × {0, 1, 2, . . . , K }. For each j ∈ {1, . . . , n} we add edges (( j − 1, i), ( j, i)) (i = 0, 1, . . . , K ) and (( j − 1, i), ( j, i + c j )) (i = 0, 1, . . . , K − c j ). Observe that any path from (0, 0) to ( j, i) corresponds to a subset S ⊆ {1, . . . , j} P c with k∈S k = i, and vice versa. Therefore we can solve our S UBSET-S UM instance by checking whether G contains a path from (0, 0) to (n, K ). With the G RAPH S CANNING A LGORITHM this can be done in O(n K ) time, so we have a pseudopolynomial algorithm. above is also a pseudopolynomial algorithm for PARTITION because PThe n n c i=1 i ≤ 2 largest(c1 , . . . , cn ). We shall discuss an extension of this algorithm in Section 17.2. If the numbers are not too large, a pseudopolynomial algorithm can be quite efficient. Therefore the following definition is useful: 1 2

Definition 15.39. For a decision problem P = (X, Y ) or an optimization problem P = (X, (Sx )x∈X , c, goal), and a subset X 0 ⊆ X of instances we define the restriction of P to X 0 by P 0 = (X 0 , X 0 ∩ Y ) or P 0 = (X 0 , (Sx )x∈X 0 , c, goal), respectively. Let P be a decision or optimization problem such that each instance consists of a list of numbers. For a polynomial p let P p be the restriction of P to instances x consisting of nonnegative integers with largest(x) ≤ p(size(x)). P is called strongly NP-hard if there is a polynomial p such that P p is NP-hard. P is called strongly NP-complete if P ∈ NP and there is a polynomial p such that P p is NP-complete. Proposition 15.40. Unless P = NP there is no exact pseudopolynomial algorithm for any strongly NP-hard problem. We give some famous examples: Theorem 15.41. I NTEGER P ROGRAMMING is strongly NP-hard. Proof: For an undirected graph G the integer program max{1lx : x ∈ ZV (G) , 0 ≤ x ≤ 1l, xv + xw ≤ 1 for {v, w} ∈ E(G)} has optimum value at least k if and only if G contains a stable set of cardinality k. Since k ≤ |V (G)| for all nontrivial instances (G, k) of S TABLE S ET, the result follows from Theorem 15.23.

T RAVELING S ALESMAN P ROBLEM (TSP) Instance: Task:

A complete graph K n (n ≥ 3) and weights c : E(K n ) → R+ . P Find a Hamiltonian circuit T whose weight e∈E(T ) c(e) is minimum.

The vertices of a TSP-instance are often called cities, the weights are also referred to as distances. Theorem 15.42. The TSP is strongly NP-hard.

Exercises

387

Proof: We show that the TSP is NP-hard even when restricted to instances where all distances are 1 or 2. We describe a polynomial transformation from the H AMILTONIAN C IRCUIT problem. Given a graph G on n ≥ 3 vertices, we construct the following instance of TSP: Take one city for each vertex of G, and let the distances be 1 whenever the edge is in E(G) and 2 otherwise. It is then obvious that G is Hamiltonian if and only if the optimum TSP tour has length n. The proof also shows that the following decision problem is not easier than the TSP itself: Given an instance of the TSP and an integer k, is there a tour of length k or less? A similar statement is true for a large class of discrete optimization problems: Proposition 15.43. Let F and F 0 be (infinite) families of finite sets, and let P be the following optimization problem: given a set E ∈ F and a function c : E → Z, find a set F ⊆ E with F ∈ F 0 and c(F) minimum (or decide that no such F exists). Then P can be solved in polynomial time if and only if the following decision problem can be solved in polynomial time: given an instance (E, c) of P and an integer k, is OPT((E, c)) ≤ k? If the optimization problem is NP-hard, then so is this decision problem. Proof: It suffices to show that there is an oracle algorithm for the optimization problem using the decision problem (the converse is trivial). Let (E, c) be an instance of P. P We first determine OPT((E, c)) by binary search. Since there are at most 1 + e∈E |c(e)| ≤ 2size(c) possible values we can do this with O(size(c)) iterations, each including one oracle call. Then we successively check for each element of E whether there exists an optimum solution without this element. This can be done by increasing its weight (say by one) and checking whether this also increases the value of an optimum solution. If so, we keep the old weight, otherwise we indeed increase the weight. After checking all elements of E, those elements whose weight we did not change constitute an optimum solution. Examples where this result applies are the TSP, the M AXIMUM W EIGHT C LIQUE P ROBLEM, the S HORTEST PATH P ROBLEM, the K NAPSACK P ROBLEM, and many others.

Exercises 1. Observe that there are more languages than Turing machines. Conclude that there are languages that cannot be decided by a Turing machine. Turing machines can also be encoded by binary strings. Consider the famous H ALTING P ROBLEM: given two binary strings x and y, where x encodes a Turing machine 8, is time(8, y) < ∞?

388

15 NP-Completeness

Prove that the H ALTING P ROBLEM is undecidable (i.e. there is no algorithm for it). Hint: Assuming that there is such an algorithm A, construct a Turing machine which, on input x, first runs the algorithm A on input (x, x) and then terminates if and only if output(A, (x, x)) = 0. 2. Describe a Turing machine which compares two strings: it should accept as input a string a#b with a, b ∈ {0, 1}∗ and output 1 if a = b and 0 if a 6= b. 3. A well-known machine model is the RAM machine. It works with an infinite sequence of registers x1 , x2 , . . . and one special register, the accumulator Acc. Each register can store an arbitrarily large integer, possibly negative. A RAM program is a sequence of instructions. There are ten types of instructions (the meaning is illustrated on the right-hand side): WRITE LOAD LOADI STORE STOREI ADD SUBTR HALF IFPOS HALT

k k k k k k k i

Acc := k. Acc := xk . Acc := x xk . xk := Acc. x xk := Acc. Acc := Acc + xk . Acc := Acc − xk . Acc := bAcc/2c. If Acc > 0 then go to . i Stop.

A RAM program is a sequence of m instructions; each is one of the above, where k ∈ Z and i ∈ {1, . . . , m}. The computation starts with instruction 1; it then proceeds as one would expect; we do not give a formal definition. The above list of instructions may be extended. We say that a command can be simulated by a RAM program in time n if it can be substituted by RAM commands so that the total number of steps in any computation increases by at most a factor of n. (a) Show that the following commands can be simulated by small RAM programs in constant time: IFNEG i IFZERO i ∗

∗

If Acc < 0 then go to . i If Acc = 0 then go to . i

(b) Show that the SUBTR and HALF commands can be simulated by RAM programs using only the other eight commands in O(size(xk )) time and O(size(Acc)) time, respectively. (c) Show that the following commands can be simulated by RAM programs in O(n) time, where n = max{size(xk ), size(Acc)}:

Exercises

MULT k DIV k MOD k ∗

389

Acc := Acc · xk . Acc := bAcc/xk c. Acc := Acc mod xk .

4. Let f : {0, 1}∗ → {0, 1}∗ be a mapping. Show that if there is a Turing machine 8 computing f , then there is a RAM program (cf. Exercise 3) such that the computation on input x (in Acc) terminates after O(size(x)+time(8, x)) steps with Acc = f (x). Show that if there is a RAM machine which, given x in Acc, computes f (x) in Acc in at most g(size(x)) steps, then there is a Turing machine computing f with time(8, x) = O(g(size(x))3 ). 5. Prove that the following two decision problems are in NP: (a) Given two graphs G and H , is G isomorphic to a subgraph of H ? (b) Given a natural number n (in binary encoding), is there a prime number p with n = p p ? 6. Prove: If P ∈ NP, then there exists a polynomial p such that P can be solved by a (deterministic) algorithm having time complexity O 2 p(n) . 7. Let Z be a 2S AT instance, i.e. a collection of clauses over X with two literals each. Consider a digraph G(Z) as follows: V (G)is the set of literals over X . There is an edge (λ1 , λ2 ) ∈ E(G) iff the clause λ1 , λ2 is a member of Z. (a) Show that if, for some variable x, x and x are in the same strongly connected component of G(Z), then Z is not satisfiable. (b) Show the converse of (a). (c) Give a linear-time algorithm for 2S AT. 8. Describe a linear-time algorithm which for any instance of S ATISFIABILITY finds a truth assignment satisfying at least half of the clauses. 9. Consider 3-O CCURRENCE S AT, which is S ATISFIABILITY restricted to instances where each clause contains at most three literals and each variable occurs in at most three clauses. Prove that even this restricted version is NPcomplete. 10. Let κ : {0, 1}m → {0, 1}m be a (not necessarily bijective) mapping, m ≥ 2. For x = x1 × · · · × xn ∈ {0, 1}m × · · · × {0, 1}m = {0, 1}nm let κ(x) S := κ(x1 ) × · · · × κ(xn ), and for a decision problem P = (X, Y ) with X ⊆ n∈Z+ {0, 1}nm let κ(P) := ({κ(x) : x ∈ X }, {κ(x) : x ∈ Y }). Prove: (a) For all codings κ and all P ∈ NP we have also κ(P) ∈ NP. (b) If κ(P) ∈ P for all codings κ and all P ∈ P, then P = NP. (Papadimitriou [1994]) 11. Prove that S TABLE S ET is NP-complete even if restricted to graphs whose maximum degree is 4. Hint: Use Exercise 9. 12. Prove that the following problem, called D OMINATING S ET, is NP-complete. Given an undirected graph G and a number k ∈ N, is there a set X ⊆ V (G) with |X | ≤ k such that X ∪ 0(X ) = V (G) ? Hint: Transformation from V ERTEX C OVER.

390

15 NP-Completeness

13. The decision problem C LIQUE is NP-complete. Is it still NP-complete (provided that P 6= NP) if restricted to (a) bipartite graphs, (b) planar graphs, (c) 2-connected graphs? 14. Prove that the following problems are NP-complete: (a) H AMILTONIAN PATH and D IRECTED H AMILTONIAN PATH Given a graph G (directed or undirected), does G contain a Hamiltonian path? (b) S HORTEST PATH Given a graph G, weights c : E(G) → Z, two vertices s, t ∈ V (G), and an integer k. Is there an s-t-path of weight at most k? (c) 3-M ATROID I NTERSECTION Given three matroids (E, F1 ), (E, F2 ), (E, F3 ) (by independence oracles) and a number k ∈ N, decide whether there is a set F ∈ F1 ∩ F2 ∩ F3 with |F| ≥ k. (d) C HINESE P OSTMAN P ROBLEM Given graphs G and H with V (G) = V (H ), weights c : E(H ) → Z+ and an integer k. Is there a subset F ⊆ E(H ) with c(F) ≤ k such that . (V (G), E(G) ∪ F) is connected and Eulerian? 15. Either find a polynomial-time algorithm or prove NP-completeness for the following decision problems: (a) Given an undirected graph G and some T ⊆ V (G), is there a spanning tree in G such that all vertices in T are leaves? (b) Given an undirected graph G and some T ⊆ V (G), is there a spanning tree in G such that all leaves are elements of T ? (c) Given a digraph G, weights c : E(G) → R, a set T ⊆ V (G) and a number k, is there a branching B with |δ + B (x)| ≤ 1 for all x ∈ T and c(B) ≥ k? 16. Prove that the following decision problem belongs to coNP: given a matrix A ∈ Qm×n and a vector b ∈ Qn , is the polyhedron {x : Ax ≤ b} integral? Hint: Use Proposition 3.9, Lemma 5.11, and Theorem 5.13. Note: The problem is not known to be in NP. 17. Show that the following problem is NP-hard (it is not known to be in NP): given an instance of S ATISFIABILITY, does the majority of all truth assignments satisfy all the clauses? 18. Show that PARTITION polynomially transforms to the following problem (which is thus NP-hard; it is not known to be in NP):

K - TH H EAVIEST S UBSET Instance:

Integers c1 , . . . , cn , K , L.

Question: Are P there K distinct subsets S1 , . . . , S K ⊆ {1, . . . , n} such that j∈Si c j ≥ L for i = 1, . . . , K ?

References

391

19. Prove that the following problem belongs to coNP: given a matrix A ∈ Zm×n and a vector b ∈ Zm , decide whether the polyhedron P = {x ∈ Rn : Ax ≤ b} is integral. Note: In fact the problem is coNP-complete, as was shown by Papadimitriou and Yannakakis [1990].

References General Literature: Aho, A.V., Hopcroft, J.E., and Ullman, J.D. [1974]: The Design and Analysis of Computer Algorithms. Addison-Wesley, Reading 1974 Ausiello, G., Crescenzi, P., Gambosi, G., Kann, V., Marchetti-Spaccamela, A., and Protasi, M. [1999]: Complexity and Approximation: Combinatorial Optimization Problems and Their Approximability Properties. Springer, Berlin 1999 Bovet, D.B., and Crescenzi, P. [1994]: Introduction to the Theory of Complexity. PrenticeHall, New York 1994 Garey, M.R., and Johnson, D.S. [1979]: Computers and Intractability: A Guide to the Theory of NP-Completeness. Freeman, San Francisco 1979, Chapters 1–3, 5, and 7 Horowitz, E., and Sahni, S. [1978]: Fundamentals of Computer Algorithms. Computer Science Press, Potomac 1978, Chapter 11 Johnson, D.S. [1981]: The NP-completeness column: an ongoing guide. Journal of Algorithms starting with Vol. 4 (1981) Karp, R.M. [1975]: On the complexity of combinatorial problems. Networks 5 (1975), 45–68 Papadimitriou, C.H. [1994]: Computational Complexity. Addison-Wesley, Reading 1994 Papadimitriou, C.H., and Steiglitz, K. [1982]: Combinatorial Optimization: Algorithms and Complexity. Prentice-Hall, Englewood Cliffs 1982, Chapters 15 and 16 Wegener, I. [2005]: Complexity Theory: Exploring the Limits of Efficient Algorithms. Springer, Berlin 2005

Cited References: Adleman, L.M., Pomerance, C., and Rumely, R.S. [1983]: On distinguishing prime numbers from composite numbers. Annals of Mathematics 117 (1983), 173–206 Agrawal, M., Kayal, N., and Saxena, N. [2004]: PRIMES is in P. Annals of Mathematics 160 (2004), 781–793 Cook, S.A. [1971]: The complexity of theorem proving procedures. Proceedings of the 3rd Annual ACM Symposium on the Theory of Computing (1971), 151–158 Edmonds, J. [1965]: Minimum partition of a matroid into independent subsets. Journal of Research of the National Bureau of Standards B 69 (1965), 67–72 van Emde Boas, P. [1990]: Machine models and simulations. In: Handbook of Theoretical Computer Science; Volume A; Algorithms and Complexity (J. van Leeuwen, ed.), Elsevier, Amsterdam 1990, pp. 1–66 F¨urer, M. [2007]: Faster integer mulitplication. Proceedings of the 39th ACM Symposium on Theory of Computing (2007), 57–66 Garey, M.R., Johnson, D.S., and Stockmeyer, L. [1976]: Some simplified NP-complete graph problems. Theoretical Computer Science 1 (1976), 237–267

392

15 NP-Completeness

Hopcroft, J.E., and Ullman, J.D. [1979]: Introduction to Automata Theory, Languages, and Computation. Addison-Wesley, Reading 1979 Karp, R.M. [1972]: Reducibility among combinatorial problems. In: Complexity of Computer Computations (R.E. Miller, J.W. Thatcher, eds.), Plenum Press, New York 1972, pp. 85–103 Ladner, R.E. [1975]: On the structure of polynomial time reducibility. Journal of the ACM 22 (1975), 155–171 Lewis, H.R., and Papadimitriou, C.H. [1981]: Elements of the Theory of Computation. Prentice-Hall, Englewood Cliffs 1981 Papadimitriou, C.H., and Yannakakis, M. [1990]: On recognizing integer polyhedra. Combinatorica 10 (1990), 107–109 Pratt, V. [1975]: Every prime has a succinct certificate. SIAM Journal on Computing 4 (1975), 214–220 Sch¨onhage, A., and Strassen, V. [1971]: Schnelle Multiplikation großer Zahlen. Computing 7 (1971), 281–292 Turing, A.M. [1936]: On computable numbers, with an application to the Entscheidungsproblem. Proceedings of the London Mathematical Society (2) 42 (1936), 230–265 and 43 (1937), 544–546

16 Approximation Algorithms

In this chapter we introduce the important concept of approximation algorithms. So far we have dealt mostly with polynomially solvable problems. In the remaining chapters we shall indicate some strategies to cope with NP-hard combinatorial optimization problems. Here approximation algorithms must be mentioned in the first place. The ideal case is when the solution is guaranteed to differ from the optimum solution by a constant only: Definition 16.1. An absolute approximation algorithm for an optimization problem P is a polynomial-time algorithm A for P for which there exists a constant k such that |A(I ) − OPT(I )| ≤ k for all instances I of P. Unfortunately, an absolute approximation algorithm is known for very few classical NP-hard optimization problems. We shall discuss two major examples, the E DGE -C OLOURING P ROBLEM and the V ERTEX -C OLOURING P ROBLEM in planar graphs in Section 16.3. In most cases we must be satisfied with relative performance guarantees. Here we have to restrict ourselves to problems with nonnegative weights. Definition 16.2. Let P be an optimization problem with nonnegative weights and k ≥ 1. A k-factor approximation algorithm for P is a polynomial-time algorithm A for P such that 1 OPT(I ) ≤ A(I ) ≤ k OPT(I ) k for all instances I of P. We also say that A has performance ratio (or performance guarantee) k. The first inequality applies to maximization problems, the second one to minimization problems. Note that for instances I with OPT(I ) = 0 we require an exact solution. The 1-factor approximation algorithms are precisely the exact polynomialtime algorithms. Sometimes the above definition is extended to the case where k is a function of the instance I , rather than a constant. We will see an example in the next section.

394

16 Approximation Algorithms

In Section 13.4 we saw that the B EST-I N -G REEDY A LGORITHM for the M AX IMIZATION P ROBLEM for an independence system (E, F) has performance ratio 1 q(E,F ) (Theorem 13.19). In the following sections and chapters we shall illustrate

the above definitions and analyse the approximability of various NP-hard problems. We start with covering problems.

16.1 Set Covering In this section we focus on the following quite general problem:

M INIMUM W EIGHT S ET C OVER P ROBLEM Instance:

A set system (U, S) with

Task:

Find a minimum weight set cover of (U, S), i.e. a subfamily R ⊆ S S such that R∈R R = U .

S

S∈S

S = U , weights c : S → R+ .

For c ≡ 1, the problem is called M INIMUM S ET C OVER P ROBLEM. Another interesting special case arises if |{S ∈ S : x ∈ S}| = 2 for all x ∈ U ; this is the M INIMUM W EIGHT V ERTEX C OVER P ROBLEM: given a graph G and c : V (G) → R+ , the corresponding set covering instance is defined by U := E(G), S := {δ(v) : v ∈ V (G)} and c(δ(v)) := c(v) for all v ∈ V (G). As the M INIMUM W EIGHT V ERTEX C OVER P ROBLEM is NP-hard even for unit weights (Theorem 15.24), so is the M INIMUM S ET C OVER P ROBLEM. Johnson [1974] and Lov´asz [1975] proposed a simple greedy algorithm for the M INIMUM S ET C OVER P ROBLEM: in each iteration, pick a set which covers a maximum number of elements not already covered. Chv´atal [1979] generalized this algorithm to the weighted case:

G REEDY A LGORITHM F OR S ET C OVER Input:

A set system (U, S) with

Output:

A set cover R of (U, S).

S

S∈S

S = U , weights c : S → R+ .

1

Set R := ∅ and W := ∅.

2

While W 6= U do: Choose a set R ∈ S \ R for which R \ W 6= ∅ and Set R := R ∪ {R} and W := W ∪ R.

c(R) |R\W |

is minimum.

The running time is obviously O(|U ||S|). The following performance guarantee can be proved: Theorem 16.3. (Chv´atal [1979]) For any instance (U, S, c) of the M INIMUM W EIGHT S ET C OVER P ROBLEM, the G REEDY A LGORITHM F OR S ET C OVER finds a set cover whose weight is at most H (r ) OPT(U, S, c), where r := max S∈S |S| and H (r ) = 1 + 21 + · · · + r1 .

16.1 Set Covering

395

Proof: Let (U, S, c) be an instance of the M INIMUM W EIGHT S ET C OVER P ROBLEM, and let R = {R1 , . . . , Rk } be the solution found by the above algorithm, Sj where Ri is the set chosen in the i-th iteration. For j = 0, . . . , k let W j := i=1 Ri . For each e ∈ U let j (e) := min{ j ∈ {1, . . . , k} : e ∈ R j } be the iteration where e is covered. Let c(R j (e) ) . y(e) := |R j (e) \ W j (e)−1 | Let S ∈ S be fixed, and let k 0 := max{ j (e) : e ∈ S}. We have 0

X

y(e) =

k X

X

y(e)

i=1 e∈S: j (e)=i

e∈S

0

=

k X

c(Ri ) |S ∩ (Wi \ Wi−1 )| |Ri \ Wi−1 |

i=1 0

=

k X

c(Ri ) (|S \ Wi−1 | − |S \ Wi |) |Ri \ Wi−1 |

i=1 0

≤

k X

c(S) (|S \ Wi−1 | − |S \ Wi |) |S \ Wi−1 |

i=1

by the choice of the Ri in 2 (observe that S \ Wi−1 6= ∅ for i = 1, . . . , k 0 ). By writing si := |S \ Wi−1 | we get 0

X e∈S

k X si − si+1 y(e) ≤ c(S) si i=1 0

≤ c(S)

k X 1 i=1

1 1 + ··· + + si si − 1 si+1 + 1

0

= c(S)

k X

(H (si ) − H (si+1 ))

i=1

= c(S)(H (s1 ) − H (sk 0 +1 )) ≤ c(S)H (s1 ). Since s1 = |S| ≤ r , we conclude that X y(e) ≤ c(S)H (r ). e∈S

We sum over all S ∈ O for an optimum set cover O and obtain

396

16 Approximation Algorithms

c(O)H (r ) ≥

XX

y(e)

S∈O e∈S

≥

X

y(e)

e∈U

=

k X

X

y(e)

i=1 e∈U : j (e)=i

=

k X

c(Ri ) = c(R).

i=1

For a slightly tighter analysis of the non-weighted case, see Slav´ık [1997]. Raz and Safra [1997] discovered that there exists a constant c > 0 such that, unless P = NP, no performance ratio of c ln |U | can be achieved. Indeed, a performance ratio of c ln |U | cannot be achieved for any c < 1 unless each problem in NP can be solved in O n O(log log n) time (Feige [1998]). The M INIMUM W EIGHT E DGE C OVER P ROBLEM is obviously a special case of the M INIMUM W EIGHT S ET C OVER P ROBLEM. Here we have r = 2 in Theorem 16.3, hence the above algorithm is a 32 -factor approximation algorithm in this special case. However, the problem can also be solved optimally in polynomial time; cf. Exercise 11 of Chapter 11. For the M INIMUM V ERTEX C OVER P ROBLEM, the above algorithm reads as follows:

G REEDY A LGORITHM F OR V ERTEX C OVER Input:

A graph G.

Output:

A vertex cover R of G.

1

Set R := ∅.

2

While E(G) 6= ∅ do: Choose a vertex v ∈ V (G) \ R with maximum degree. Set R := R ∪ {v} and delete all edges incident to v.

This algorithm looks reasonable, so one might ask for which k it is a k-factor approximation algorithm. It may be surprising that there is no such k. Indeed, the bound given in Theorem 16.3 is almost best possible: Theorem 16.4. (Johnson [1974], Papadimitriou and Steiglitz [1982]) For all n ≥ 3 there is an instance G of the M INIMUM V ERTEX C OVER P ROBLEM such that n H (n − 1) + 2 ≤ |V (G)| ≤ n H (n − 1) + n, the maximum degree of G is n − 1, OPT(G) = n, and the above algorithm can find a vertex cover containing all but n vertices. j k P Proof: For each n ≥ 3 and i ≤ n we define Ain := ij=2 nj and

16.1 Set Covering

397

V (G n ) := a1 , . . . , a An−1 , b1 , . . . , bn , c1 , . . . , cn . n

E(G n ) := {{bi , ci } : i = 1, . . . , n} ∪ n−1

i

An

i−1 − 1)i + 1 ≤ k ≤ ( j − A )i . {a j , bk } : ( j − Ai−1 n n

i=2 j=Ai−1 n +1 n−1 ≤ n H (n − 1) − n and An−1 ≥ n H (n − Observe that |V (G n )| = 2n + An−1 n , An n 1) − n − (n − 2). Figure 16.1 shows G 6 .

c1

c2

c3

c4

c5

c6

b1

b2

b3

b4

b5

b6

a1

a2

a3

a4

a5

a6

a7

Fig. 16.1.

If we apply our algorithm to G n , it may ﬁrst choose vertex a An−1 (because n it has maximum degree), and subsequently the vertices a An−1 −1 , a An−1 −2 , . . . , a1 . n n After this there are n disjoint edges left, so n more vertices are needed. Hence the + n vertices, while the optimum vertex constructed vertex cover consists of An−1 n cover {b1 , . . . , bn } has size n. There are, however, 2-factor approximation algorithms for the M INIMUM V ER C OVER P ROBLEM. The simplest one is due to Gavril (see Garey and Johnson [1979]): just ﬁnd any maximal matching M and take the ends of all edges in M. This is obviously a vertex cover and contains 2|M| vertices. Since any vertex cover must contain |M| vertices (no vertex covers two edges of M), this is a 2-factor approximation algorithm. This performance guarantee is tight: simply think of a graph consisting of many disjoint edges. It may be surprising that the above is the best known approximation algorithm for the M INIMUM V ERTEX C OVER P ROBLEM. Later we shall show that there is a number k > 1 such that no k-factor approximation algorithm exists unless P = NP (Theorem 16.46). Indeed, a 1.36-factor approximation algorithm does not exist unless P = NP (Dinur and Safra [2002]). TEX

398

16 Approximation Algorithms

At least Gavril’s algorithm can be extended to the weighted case. We present the algorithm of Bar-Yehuda and Even [1981], which is applicable to the general M INIMUM W EIGHT S ET C OVER P ROBLEM:

BAR -Y EHUDA -E VEN A LGORITHM Input:

A set system (U, S) with

Output:

A set cover R of (U, S).

1

2

S

S∈S

S = U , weights c : S → R+ .

Set R := ∅ and W := ∅. Set y(e) := 0 for all e ∈ U . Set c0 (S) := c(S) for all S ∈ S. While W 6= U do: Choose an element e ∈ U \ W . Let R ∈ S with e ∈ R and c0 (R) minimum. Set y(e) := c0 (R). Set c0 (S) := c0 (S) − y(e) for all S ∈ S with e ∈ S. Set R := R ∪ {R} and W := W ∪ R.

Theorem 16.5. (Bar-Yehuda and Even [1981]) For any instance (U, S, c) of the M INIMUM W EIGHT S ET C OVER P ROBLEM, the BAR -Y EHUDA -E VEN A L GORITHM finds a set cover whose weight is at most p OPT(U, S, c), where p := maxe∈U |{S ∈ S : e ∈ S}|. Proof: The M INIMUM W EIGHT S ET C OVER P ROBLEM can be written as the integer linear program o n min cx : Ax ≥ 1l, x ∈ {0, 1}S , where A is the matrix whose rows correspond to the elements of U and whose columns are the incidence vectors of the sets in S. The optimum of the LP relaxation min {cx : Ax ≥ 1l, x ≥ 0} is a lower bound for OPT(U, S, c) (the omission of the constraints x ≤ 1l does not change the optimum value of this LP). Hence, by Proposition 3.13, the optimum of the dual LP max{y1l : y A ≤ c, y ≥ 0} is also a lower bound for OPT(U, S, c). Now observe that c0 (S) ≥ 0 for all S ∈ S at any stage P of the algorithm. Let y¯ be the vector y at termination. We have y¯ ≥ 0 and e∈S y¯ (e) ≤ c(S) for all S ∈ S, i.e. y¯ is a feasible solution of the dual LP and y¯ 1l ≤ max{y1l : y A ≤ c, y ≥ 0} ≤ OPT(U, S, c).

16.2 The Max-Cut Problem

399

Finally observe that c(R) =

X

c(R)

R∈R

=

XX

y¯ (e)

R∈R e∈R

≤

X

p y¯ (e)

e∈U

=

p y¯ 1l

≤

p OPT(U, S, c).

Since we have p = 2 in the vertex cover case, this is a 2-factor approximation algorithm for the M INIMUM W EIGHT V ERTEX C OVER P ROBLEM. The first 2factor approximation algorithm was due to Hochbaum [1982]. She proposed finding an optimum solution y of the dual LP in the above proof and taking all sets S P with e∈S y(e) = c(S). Alternatively, one could find an optimum solution x of the primal LP and take all sets S with x S ≥ 1p . The advantage of the BAR -Y EHUDA -E VEN A LGORITHM is that it does not explicitly use linear programming. In fact it can easily be implemented with P O |S| time. It is our first example of a primal-dual approximation alS∈S gorithm; more complicated examples will follow in Sections 20.4 and 22.3.

16.2 The Max-Cut Problem In this section we consider another basic problem:

M AXIMUM W EIGHT C UT P ROBLEM Instance:

An undirected graph G and weights c : E(G) → R+ .

Task:

Find a cut in G with maximum total weight.

This problem is often called M AX -C UT for short. In contrast to minimum weight cuts, discussed in Section 8.7, this is a difficult problem. It is strongly NP-hard; even the special case where c ≡ 1 (the M AXIMUM C UT P ROBLEM) is hard: Theorem 16.6. (Garey, Johnson and Stockmeyer [1976]) The M AXIMUM C UT P ROBLEM is NP-hard. Proof: By reduction from M AX -2S AT (cf. Theorem 15.36). Given an instance of M AX -2S AT with n variables and m clauses, we construct a graph G whose vertices are the literals plus an extra vertex z. For each variable x we add 3m parallel edges between x and x. ¯ For each clause {λ, λ0 } we add three edges {λ, λ0 }, 0 {λ, z} and {λ , z}. So G has 2n + 1 vertices and 3m(n + 1) edges.

400

16 Approximation Algorithms

We claim that the maximum cardinality of a cut in G is 3mn + 2t, where t is the maximum number of clauses satisfied by any truth assignment. Indeed, given a truth assignment satisfying t clauses, let X be the set of true literals. Then |δG (X )| = 3mn + 2t. Conversely, if there is a set X ⊆ V (G) with |δG (X )| ≥ 3mn + a, then w.l.o.g. z ∈ / X (otherwise replace X by V (G) \ X ), and for each variable x we have |X ∩ {x, x}| ¯ = 1 (otherwise replace X by X 1{x} and increase the cut). Thus we can set all literals in X to be true and get a truth assignment satisfying at least a2 clauses. It is very easy to find a 2-factor approximation algorithm for the M AXIMUM W EIGHT C UT P ROBLEM:P If V (G) = {v1 , . . . , vn }, start with P X := {v1 }, and for i = 3, . . . , n add vi to X if e∈E(vi ,{v1 ,...,vi−1 }∩X ) c(e) < e∈E(vi ,{v1 ,...,vi−1 }\X ) c(e). (The simple analysis of this algorithm is left as Exercise 9.) For a long time no better approximation algorithm was known. Then Goemans and Williamson [1995] found a much better one using semidefinite programming; the rest of this section is based on their paper. Let G be an undirected graph and c : E(G) → R+ . W.l.o.g., V (G) = {1, . . . , n}. For 1 ≤ i, j ≤ n let ci j := c({i, j}) if {i, j} ∈ E(G) and ci j := 0 othW EIGHT C UT P ROBLEM consists of finding a subset erwise. Then the M AXIMUM P S ⊆ {1, . . . , n} maximizing i∈S, j∈{1,...,n}\S ci j . Expressing S by y ∈ {−1, 1}n with yi = 1 iff i ∈ S, we can formulate the problem as follows:

max s.t.

1 2

X

ci j (1 − yi y j )

1≤i< j≤n

∈ {−1, 1}

yi

(i = 1, . . . , n)

The variables yi can be regarded as one-dimensional vectors of unit norm. By relaxing them to multidimensional vectors of unit Euclidean norm we get a very interesting relaxation:

max s.t.

1 2 yi

X

ci j (1 − yi> y j ) (16.1)

1≤i< j≤n

∈ Sm

(i = 1, . . . , n)

where m ∈ N and Sm = {x ∈ Rm : ||x||2 = 1} denotes the unit sphere in For example, for the triangle (n = 3, c12 = c13 = c23 = 1) the optimum is obtained by points on the unit sphere in R2 that are√vertices of an equilateral √ triangle, e.g. y1 = (0, −1), y2 = (− 23 , 12 ), and y3 = ( 23 , 12 ), giving an optimum value of 94 , in contrast to the maximum weight of a cut, which is 2. However, the interesting fact is that we can solve (16.1) almost optimally in polynomial time. The trick is not to consider the variables yi directly, not even their dimension. Instead, we consider the n × n-matrix (yi> y j )i, j=1,...,n . As a matrix X is symmetric Rm .

16.2 The Max-Cut Problem

401

and positive semidefinite if and only if it can be written as B > B for some matrix B, we can write equivalently max s.t.

1 2

X

ci j (1 − xi j )

1≤i< j≤n

xii = 1 (i = 1, . . . , n) X = (xi j )1≤i, j≤n symmetric and positive semidefinite

(16.2)

From a solution to (16.2) we can obtain a solution to (16.1) with m ≤ n and almost the same objective function value by Cholesky factorization in O(n 3 ) time (we have to accept an arbitrarily small rounding error; cf. Exercise 6 of Chapter 4). Problem (16.2) is called a semidefinite program. It can be solved approximately in polynomial time by the E LLIPSOID M ETHOD, applying Theorem 4.19, as we shall show now. First observe that we optimize a linear objective function over the convex set P := X = (xi j )1≤i, j≤n ∈ Rn×n : X symmetric and positive semidefinite, xii = 1 (i = 1, . . . , n) . 2

By projecting P to the n 2−n free variables we obtain P 0 := (xi j )1≤i< j≤n : (xi j )1≤i, j≤n ∈ P with xii := 1 and x ji := xi j for i < j . Note that neither P nor P 0 is a polyhedron. However, P 0 is convex, bounded, and full-dimensional: Proposition 16.7. P 0 is convex. Moreover, B(0, n1 ) ⊆ P 0 ⊆ B(0, n). Proof: Convexity follows from the simple fact that convex combinations of positive semidefinite matrices are positive semidefinite. For the first inclusion, observe that for a symmetric n × n-matrix X whose diagonal entries are 1 and whose off-diagonal entries have absolute value at most 1 n n we have, for any d ∈ R , d> X d =

n X

xi j di d j

i, j=1

≥

≥

=

1 X (xii di2 + x j j d j2 − (2n − 2)|xi j ||di d j |) 2n − 2 1 2n − 2 1 2n − 2

≥ 0,

i6= j n X i6= j n X i6= j

(di2 + d j2 − 2|di d j |) (|di | − |d j |)2

402

16 Approximation Algorithms

i.e. X is positive semidefinite. For the second inclusion, note that all off-diagonal entries of a matrix in P have absolute value at most 1, and thus the Euclidean norm of the vector of upper diagonal entries is at most n. It remains to show that the S EPARATION P ROBLEM for P 0 can be solved in polynomial time. This is achieved by G AUSSIAN E LIMINATION: Theorem 16.8. Given a symmetric matrix X ∈ Qn×n , we can decide in polynomial time whether X is positive semidefinite, and find a vector d ∈ Qn with d > X d < 0 if one exists. Proof: If xnn < 0, then we set d = (0, . . . , 0, 1) and have d > X d < 0. If xnn = 0 x and xn j 6= 0 for some j < n, then we can define d by d j := −1, dn := 2xjnj j + xn j , x

and di := 0 for i ∈ {1, . . . , n − 1} \ { j}, and have d > X d = x j j − 2xn j ( 2xjnj j + xn j ) = −2(xn j )2 < 0, again proving that X is not positive semidefinite. In the other cases we reduce the dimension. If xn j = 0 for all j, then the last row and column can be deleted: X is positive semidefinite if and only if X 0 := (xi j )i, j=1,...,n−1 is positive semidefinite. Moreover, if c ∈ Qn−1 satisfies c> X 0 c < 0, we set d := ( 0c ) and have d > X d < 0. Hence let us now assume that xnn > 0. Then we consider X 0 := (xi j − xni xn j xnn )i, j=1,...,n−1 ; this corresponds to one iteration of G AUSSIAN E LIMINATION . Note that X 0 is positive semidefinite if and only if X is positive semidefinite. c n−1 For a vector c ∈ Qn−1 with c> X 0 c < 0 we set d := ( − x 1 Pi=1 ci xni ). Then nn

d> X d =

n−1 X i, j=1

n−1 X xni di xi0 j + xn j d j + 2 dn xn j d j + dn2 xnn xnn

= c> X 0 c +

j=1

n−1 X i, j=1

ci

xni xn j c j (1 − 2 + 1) xnn

= c> X 0 c < 0. This defines a polynomial-time algorithm. To see that the numbers involved in the computation of d are not too large, let X (n) , X (n−1) , . . . , X (k) be the considered matrices (X (i) ∈ Qi×i ), and assume that we observe in iteration n + 1 − k that the matrix X (k) = (yi j )i, j=1,...,k is not positive semidefinite (i.e. ykk < 0 or ykk = 0 and yk j 6= 0 for some j < k). We have a vector c ∈ Qk with c> X (k) c < 0 and size(c) ≤ 2 size(X (k) ). Now a vector d ∈ Qn with d > X d < 0 can be constructed as above; note that d is a solution of the linear equation system Md = ( 0c ), where the j-th row of M is • the j-th unit vector if j ≤ k, • the j-th unit vector if j > k and the j-th row of X ( j) is zero,

16.2 The Max-Cut Problem

•

403

the j-th row of X ( j) , followed by zeros, otherwise.

Hence, with Theorem 4.4, we have size(d) ≤ 4n(size(M) + size(c)), which is polynomial by Theorem 4.10. Corollary 16.9. The S EPARATION P ROBLEM for P 0 can be solved in polynomial time. Proof: Let (yi j )1≤i< j≤n be given, and let Y = (yi j )1≤i, j≤n be the symmetric matrix defined by yii = 1 for all i and y ji := yi j for i < j. Apply Theorem 16.8. If Y is positive semidefinite, we are done. Pn di2 > Otherwise we find a vector d ∈ Qn with d > Y d < 0. Then − i=1 P P P P n n 2 > d Y d − i=1 di = 1≤i< j≤n 2di d j yi j , but 1≤i< j≤n 2di d j z i j ≥ − i=1 di2 for all z ∈ P 0 . Thus (di d j )1≤i< j≤n constitutes a separating hyperplane. We can now conclude: Theorem 16.10. For any instance of the M AXIMUM W EIGHT C UT P ROBLEM, we can find a matrix Y = (yi j )1≤i, j≤n ∈ P with X X ci j (1 − yi j ) ≥ max ci j (1 − xi j ) : (xi j )1≤i, j≤n ∈ P − 1≤i< j≤n

1≤i< j≤n

in time polynomial in n, size((ci j )1≤i< j≤n ), and size(). Proof: We apply Theorem 4.19, using Proposition 16.7 and Corollary 16.9. Semidefinite programs such as (16.2) can also be solved approximately by interior point algorithms, which are more efficient than the E LLIPSOID M ETHOD. See Alizadeh [1995] for details. As mentioned above, from an almost optimum solution to (16.2) we can derive a solution to (16.1) with almost the same objective function value by Cholesky factorization. This solution consists of a set of vectors yi ∈ Rm (i = 1, . . . , n) for some m ≤ n. As (16.1) Pis a relaxation of our original problem, we have that the optimum is at most 12 1≤i< j≤n ci j (1 − yi> y j ) + . The vectors yi lie on a unit sphere. The idea is now to take a random hyperplane through the origin, and define S to be the set of indices i for which yi is on one side of this hyperplane. A random hyperplane through the origin is given by a random point on the (m − 1)-dimensional sphere. This can be chosen by drawing m real numbers independently from the standard normal distribution, which in turn can be done using independent random numbers uniformly distributed in [0, 1]. See Knuth [1969] (Section 3.4.1) for details. The algorithm by Goemans and Williamson now reads as follows.

G OEMANS -W ILLIAMSON M AX -C UT-A LGORITHM Input:

A number n ∈ N, numbers ci j ≥ 0 for 1 ≤ i < j ≤ n.

Output:

A set S ⊆ {1, . . . , n}.

404

16 Approximation Algorithms

1

Solve (16.2) approximately; i.e. find a symmetric positive semidefinite matrix X = (xi j )1≤i, j≤n with xii = 1 for i = 1, . . . , n, such that P 1≤i< j≤n ci j (1 − x i j ) ≥ 0.9995 · OPT(16.2).

2

Apply Cholesky factorization to X in order to obtain vectors y1 , . . . , yn ∈ Rm with m ≤ n and yi> y j ≈ xi j for all i, j ∈ {1, . . . , n}.

3

Choose a random point a on the unit sphere {x ∈ Rm : ||x||2 = 1}.

4

Set S := {i ∈ {1, . . . , n} : a > yi ≥ 0}.

Theorem 16.11. The G OEMANS -W ILLIAMSON M AX -C UT-A LGORITHM runs in polynomial time. Proof: See the preceding discussion. The most difficult step, , 1 can Pbe solved in polynomial time by Theorem 16.10. Here we can choose = 0.00025 1≤i< j≤n ci j P as 12 1≤i< j≤n ci j is a lower bound on the optimum objective value (attained by randomly choosing S ⊆ {1, . . . , n}) and thus on the optimum value of (16.2). We now prove the performance guarantee: Theorem 16.12. (Goemans and Williamson [1995]) The G OEMANS -W P ILLIAMSON M AX -C UT-A LGORITHM returns a set S for which the expected value of i∈S, j ∈S / ci j is at least 0.878 times the maximum possible value. Proof: Let Sm again denote the unit sphere in Rm , and let H (y) := {x ∈ Sm : x > y ≥ 0} be the hemisphere with pole y, for y ∈ Sm . For a subset A ⊆ volume (A) Sm let µ(A) := volume (Sm ) ; this defines a probability measure on Sm . We have |S ∩{i, j}| = 1 with probability µ(H (yi )4H (y j )), where 4 denotes the symmetric difference. Note that H (yi )4H (y j ) is the union of two spherical digons, each with angle arccos(yi> y j ). As the volume is proportional to the angle, we have µ(H (yi )4H (y j )) = π1 arccos(yi> y j ). Claim: π1 arccos β ≥ 0.8785 1−β 2 for all β ∈ [−1, 1]. For β = 1 we have equality. Moreover, elementary calculus yields min

−1≤β 1.38. As 1.38 > 0.8785 2 , this proves the claim. Pπ c Thus the expected value of i∈S, j ∈S i j is /

16.3 Colouring

X

X

ci j µ(H (yi )4H (y j )) =

1≤i< j≤n

ci j

1≤i< j≤n

≥ 0.8785 ·

1 2

1 ≈ 0.8785 · 2

405

1 arccos(yi> y j ) π X

ci j (1 − yi> y j )

1≤i< j≤n

X

ci j (1 − xi j )

1≤i< j≤n

≥ 0.8785 · 0.9995 · OPT(16.2) > 0.878 · OPT(16.2) X ci j : S ⊆ {1, . . . , n} . ≥ 0.878 · max i∈S, j ∈S /

Therefore we have a randomized approximation algorithm with performance 1 < 1.139. Mahajan and Ramesh [1999] showed how to derandomize this ratio 0.878 algorithm, thus obtaining a deterministic 1.139-factor approximation algorithm. However, there is no 1.062-factor approximation algorithm unless P = NP (H˚astad [2001], Papadimitriou and Yannakakis [1991]). See Lov´asz [2003] for other interesting connections of semidefinite programming and combinatorial optimization.

16.3 Colouring In this section we briefly discuss two more well-known special cases of the M IN IMUM S ET C OVER P ROBLEM : we want to partition the vertex set of a graph into stable sets, or the edge set of a graph into matchings: Definition 16.13. Let G be an undirected graph. A vertex-colouring of G is a mapping f : V (G) → N with f (v) 6= f (w) for all {v, w} ∈ E(G). An edgecolouring of G is a mapping f : E(G) → N with f (e) 6= f (e0 ) for all e, e0 ∈ E(G) with e 6= e0 and e ∩ e0 6= ∅. The number f (v) or f (e) is called the colour of v or e. In other words, the set of vertices or edges with the same colour ( f -value) must be a stable set, or a matching, respectively. Of course we are interested in using as few colours as possible:

V ERTEX -C OLOURING P ROBLEM Instance:

An undirected graph G.

Task:

Find a vertex-colouring f : V (G) → {1, . . . , k} of G with minimum k.

406

16 Approximation Algorithms

E DGE -C OLOURING P ROBLEM Instance:

An undirected graph G.

Task:

Find an edge-colouring f : E(G) → {1, . . . , k} of G with minimum k.

Reducing these problems to the M INIMUM S ET C OVER P ROBLEM is not very useful: for the V ERTEX -C OLOURING P ROBLEM we would have to list the maximal stable sets (an NP-hard problem), while for the E DGE -C OLOURING P ROBLEM we would have to reckon with exponentially many maximal matchings. The optimum value of the V ERTEX -C OLOURING P ROBLEM (i.e. the minimum number of colours) is called the chromatic number of the graph. The optimum value of the E DGE -C OLOURING P ROBLEM is called the edge-chromatic number or sometimes the chromatic index. Both colouring problems are NP-hard: Theorem 16.14. The following decision problems are NP-complete: (a) (Holyer [1981]) Decide whether a given simple graph has edge-chromatic number 3. (b) (Stockmeyer [1973]) Decide whether a given planar graph has chromatic number 3. The problems remain NP-hard even when the graph has maximum degree three in (a), and maximum degree four in (b). Proposition 16.15. For any given graph we can decide in linear time whether the chromatic number is less than 3, and if so, find an optimum colouring. The same holds for the edge-chromatic number. Proof: A graph has chromatic number 1 iff it has no edges. By definition, the graphs with chromatic number at most 2 are precisely the bipartite graphs. By Proposition 2.27 we can check in linear time whether a graph is bipartite and in the positive case find a bipartition, i.e. a vertex-colouring with two colours. To check whether the edge-chromatic number of a graph G is less than 3 (and, if so, find an optimum edge-colouring) we simply consider the V ERTEX -C OLOURING P ROBLEM in the line graph of G. This is obviously equivalent. For bipartite graphs, the E DGE -C OLOURING P ROBLEM can be solved, too: Theorem 16.16. (K¨onig [1916]) The edge-chromatic number of a bipartite graph G equals the maximum degree of a vertex in G. Proof: By induction on |E(G)|. Let G be a graph with maximum degree k, and let e = {v, w} be an edge. By the induction hypothesis, G −e has an edge-colouring f with k colours. There are colours i, j ∈ {1, . . . , k} such that f (e0 ) 6= i for all e0 ∈ δ(v) and f (e0 ) 6= j for all e0 ∈ δ(w). If i = j, we are done since we can extend f to G by giving e colour i.

16.3 Colouring

407

The graph H = (V (G), {e0 ∈ E(G) \ {e} : f (e0 ) ∈ {i, j}}) has maximum degree 2, and v has degree at most 1 in H . Consider the maximal path P in H with endpoint v. The colours alternate on P; hence the other endpoint of P cannot be w. Exchange the colours i and j on P and extend the edge-colouring to G by giving e colour j. The maximum degree of a vertex is an obvious lower bound on the edgechromatic number of any graph. It is not always attained as the triangle K 3 shows. The following theorem shows how to find an edge-colouring of a given simple graph which needs at most one more colour than necessary: Theorem 16.17. (Vizing [1964]) Let G be an undirected simple graph with maximum degree k. Then G has an edge-colouring with at most k + 1 colours, and such a colouring can be found in polynomial time. Proof: By induction on |E(G)|. If G has no edges, the assertion is trivial. Otherwise let e = {x, y0 } be any edge; by the induction hypothesis there exists an edge-colouring f of G − e with k + 1 colours. For each vertex v choose a colour n(v) ∈ {1, . . . , k + 1} \ { f (w) : w ∈ δG−e (v)} missing at v. Starting from y0 , construct a maximal sequence y0 , y1 , . . . , yt of distinct neighbours of x such that n(yi−1 ) = f ({x, yi }) for i = 1, . . . , t. If no edge incident to x is coloured n(yt ), then we construct an edge-colouring f 0 of G from f by setting f 0 ({x, yi−1 }) := f ({x, yi }) (i = 1, . . . , t) and f 0 ({x, yt }) := n(yt ). So we assume that there is an edge incident to x with colour n(yt ); by the maximality of t we have f ({x, ys }) = n(yt ) for some s ∈ {1, . . . , t − 1}. ys−1

y2

n(ys−2 )

n(y1 )

y1

n(ys−1 ) = n(yt )

n(y0 ) y0

ys

x n(yt−1 )

yt n(x)

P

n(yt )

n(x)

n(yt )

Fig. 16.2.

Consider the maximum path P starting at yt in the graph (V (G), {e0 ∈ E(G − e) : f (e0 ) ∈ {n(x), n(yt )}}) (this graph has maximum degree 2; cf. Figure 16.2). We distinguish three cases. In each case we construct an edge-colouring f 0 of G.

408

16 Approximation Algorithms

If P ends in x, then {ys , x} is the last edge of P. We construct f 0 from f by exchanging colours n(x) and n(yt ) on P, and setting f 0 ({x, yi−1 }) := f ({x, yi }) (i = 1, . . . , s). If P ends in ys−1 , then the last edge of P has colour n(x), since colour n(yt ) = f ({x, ys }) = n(ys−1 ) is missing at ys−1 . We construct f 0 from f as follows: exchange colours n(x) and n(yt ) on P, set f 0 ({x, yi−1 }) := f ({x, yi }) (i = 1, . . . , s − 1) and f 0 ({x, ys−1 }) := n(x). If P ends neither in x nor in ys−1 , then we can construct f 0 from f by exchanging colours n(x) and n(yt ) on P, and setting f 0 ({x, yi−1 }) := f ({x, yi }) (i = 1, . . . , t) and f 0 ({x, yt }) := n(x). Vizing’s Theorem implies an absolute approximation algorithm for the E DGE C OLOURING P ROBLEM in simple graphs. If we allow parallel edges the statement of Vizing’s Theorem is no longer true: by replacing each edge of the triangle K 3 by r parallel edges we obtain a 2r -regular graph with edge-chromatic number 3r . We now turn to the V ERTEX -C OLOURING P ROBLEM. The maximum degree also gives an upper bound on the chromatic number: Theorem 16.18. Let G be an undirected graph with maximum degree k. Then G has an vertex-colouring with at most k + 1 colours, and such a colouring can be found in linear time. Proof: The following G REEDY C OLOURING A LGORITHM obviously finds such a colouring.

G REEDY C OLOURING A LGORITHM Input:

An undirected graph G.

Output:

A vertex-colouring of G.

1

Let V (G) = {v1 , . . . , vn }.

2

For i := 1 to n do: Set f (vi ) := min{k ∈ N : k 6= f (v j ) for all j < i with v j ∈ 0(vi )}.

For complete graphs and for odd circuits one evidently needs k + 1 colours, where k is the maximum degree. For all other connected graphs k colours suffice, as Brooks [1941] showed. However, the maximum degree is not a lower bound on the chromatic number: any star K 1,n (n ∈ N) has chromatic number 2. Therefore these results do not lead to an approximation algorithm. In fact, no algorithms for the V ERTEX -C OLOURING P ROBLEM with a reasonable performance guarantee for general graphs are known; see Khanna, Linial and Safra [2000]. Zuckerman [2006] showed that, unless P = NP, no polynomial-time algorithm computes the chromatic number of any graph with n vertices up to a factor of n 1− for any fixed > 0. Since the maximum degree is not a lower bound for the chromatic number one can consider the maximum size of a clique. Obviously, if a graph G contains a

16.3 Colouring

409

clique of size k, then the chromatic number of G is at least k. As the pentagon (circuit of length five) shows, the chromatic number can exceed the maximum clique size. Indeed, there are graphs with arbitrarily large chromatic number that contain no K 3 . This motivates the following definition, which is due to Berge [1961,1962]: Definition 16.19. A graph G is perfect if χ (H ) = ω(H ) for every induced subgraph H of G, where χ (H ) is the chromatic number and ω(H ) is the maximum cardinality of a clique in H . It follows immediately that the decision problem whether a given perfect graph has chromatic number k has a good characterization (belongs to NP ∩ coNP). Some examples of perfect graphs can be found in Exercise 15. A polynomial-time algorithm for recognizing perfect graphs has been found by Chudnovsky et al. [2005]. Berge [1961] conjectured that a graph is perfect if and only if it contains neither an odd circuit of length at least five nor the complement of such a circuit as an induced subgraph. This so-called strong perfect graph theorem has been proved by Chudnovsky et al. [2006]. Long before, Lov´asz [1972] proved the weaker assertion that a graph is perfect iff its complement is perfect. This is known as the weak perfect graph theorem; to prove it we need a lemma: Lemma. 16.20. Let G be a perfect graph and x ∈ V (G). Then the graph G 0 := . (V (G) ∪ {y}, E(G) ∪ {{y, v} : v ∈ {x} ∪ 0(x)}), resulting from G by adding a new vertex y which is joined to x and to all neighbours of x, is perfect. Proof: By induction on |V (G)|. The case |V (G)| = 1 is trivial since K 2 is perfect. Now let G be a perfect graph with at least two vertices. Let x ∈ V (G), and let G 0 arise by adding a new vertex y adjacent to x and all its neighbours. It suffices to prove that ω(G 0 ) = χ (G 0 ), since for proper subgraphs H of G 0 this follows from the induction hypothesis: either H is a subgraph of G and thus perfect, or it arises from a proper subgraph of G by adding a vertex y as above. Since we can colour G 0 with χ (G) + 1 colours easily, we may assume that ω(G 0 ) = ω(G). Then x is not contained in any maximum clique of G. Let f be a vertex-colouring of G with χ (G) colours, and let X := {v ∈ V (G) : f (v) = f (x)}. We have ω(G − X ) = χ (G − X ) = χ (G) − 1 = ω(G) − 1 and thus ω(G − (X \ {x})) = ω(G) − 1 (as x does not belong to any maximum clique of G). Since (X \ {x}) ∪ {y} = V (G 0 ) \ V (G − (X \ {x})) is a stable set, we have χ (G 0 ) = χ (G − (X \ {x})) + 1 = ω(G − (X \ {x})) + 1 = ω(G) = ω(G 0 ). Theorem 16.21. (Lov´asz [1972], Fulkerson [1972], Chv´atal [1975]) For a simple graph G the following statements are equivalent: (a) G is perfect.

410

16 Approximation Algorithms

(b) The complement of G is perfect. (c) The stable set polytope, i.e. the convex hull of the incidence vectors of the stable sets of G, is given by: ( ) X V (G) : xv ≤ 1 for all cliques S in G . (16.3) x ∈ R+ v∈S

Proof: We prove (a)⇒(c)⇒(b). This suffices, since applying (a)⇒(b) to the complement of G yields (b)⇒(a). (a)⇒(c): Evidently the stable set polytope is contained in (16.3). To prove the other inclusion, let x be a rational vector in the polytope (16.3); we may write xv = pqv , where q ∈ N and pv ∈ Z+ for v ∈ V (G). Replace each vertex v by a clique of size pv ; i.e. consider G 0 defined by V (G 0 ) := {(v, i) : v ∈ V (G), 1 ≤ i ≤ pv }, E(G 0 ) := {{(v, i), (v, j)} : v ∈ V (G), 1 ≤ i < j ≤ pv } ∪ {{(v, i), (w, j)} : {v, w} ∈ E(G), 1 ≤ i ≤ pv , 1 ≤ j ≤ pw }. Lemma 16.20 implies that G 0 is perfect. For an arbitrary clique X 0 in G 0 let X := {v ∈ V (G) : (v, i) ∈ X 0 for some i} be its projection to G (also a clique); we have X X pv = q xv ≤ q. |X 0 | ≤ v∈X

ω(G 0 )

v∈X

G0

≤ q. Since is perfect, it thus has a vertex-colouring f with at most So q colours. For v ∈ V (G) and i =P 1, . . . , q let ai,v := 1 if f ((v, j)) = i for some q j and ai,v := 0 otherwise. Then i=1 ai,v = pv for all v ∈ V (G) and hence x =

pv q

v∈V (G)

=

q 1X ai q i=1

is a convex combination of incidence vectors of stable sets, where ai = (ai,v )v∈V (G) . (c)⇒(b): We show by induction on |V (G)| that if (16.3) is integral then the complement of G is perfect. Since graphs with less than three vertices are perfect, let G be a graph with |V (G)| ≥ 3 where (16.3) is integral. We have to show that the vertex set of any induced subgraph H of G can be partitioned into α(H ) cliques, where α(H ) is the size of a maximum stable set in H . For proper subgraphs H this follows from the induction hypothesis, since (by Theorem 5.13) every face of the integral polytope (16.3) is integral, in particular the face defined by the supporting hyperplanes xv = 0 (v ∈ V (G) \ V (H )). So it remains to prove that V (G) can be partitioned into α(G) cliques. The equation 1lx = α(G) defines a supporting hyperplane of (16.3), so X X V (G) : xv ≤ 1 for all cliques S in G, xv = α(G) (16.4) x ∈ R+ v∈S

v∈V (G)

16.3 Colouring

411

is a face of (16.3). This face is contained in some facets, which cannot all be of the form {x ∈ (16.3) : xv = 0} for some v (otherwise the origin would belong P to the intersection). Hence there is some clique S in G such that v∈S xv = 1 for all x in (16.4). Hence this clique S intersects each maximum stable set of G. Now by the induction hypothesis, the vertex set of G − S can partitioned into α(G − S) = α(G) − 1 cliques. Adding S concludes the proof. This proof is due to Lov´asz [1979b]. Indeed, the inequality system defining (16.3) is TDI for perfect graphs (Exercise 16). With some more work one can prove that for perfect graphs the V ERTEX -C OLOURING P ROBLEM, the M AXIMUM W EIGHT S TABLE S ET P ROBLEM and the M AXIMUM W EIGHT C LIQUE P ROBLEM can be solved in strongly polynomial time. Although these problems are all NPhard for general graphs (Theorem 15.23, Corollary 15.24, Theorem 16.14(b)), there is a number (the so-called theta-function of the complement graph, introduced by Lov´asz [1979a]) which is always between the maximum clique size and the chromatic number, and which can be computed in polynomial time for general graphs using the E LLIPSOID M ETHOD. The details are a bit involved; see Gr¨otschel, Lov´asz and Schrijver [1988]. One of the best-known problems in graph theory has been the Four Colour Problem: is it true that every planar map can be coloured with four colours such that no two countries with a common border have the same colour? If we consider the countries as regions and switch to the planar dual graph, this is equivalent to asking whether every planar graph has a vertex-colouring with four colours. Appel and Haken [1977] and Appel, Haken and Koch [1977] proved that this is indeed true: every planar graph has chromatic number at most 4. For a simpler proof of the Four Colour Theorem (which nevertheless is based on a case checking by a computer) see Robertson et al. [1997]. We prove the following weaker result, known as the Five Colour Theorem: Theorem 16.22. (Heawood [1890]) Any planar graph has a vertex-colouring with at most five colours, and such a colouring can be found in polynomial time. Proof: By induction on |V (G)|. We may assume that G is simple, and we fix an arbitrary planar embedding 8 = ψ, (Je )e∈E(G) of G. By Corollary 2.33, G has a vertex v of degree five or less. By the induction hypothesis, G − v has a vertex-colouring f with at most 5 colours. We may assume that v has degree 5 and all neighbours have different colours; otherwise we can easily extend the colouring to G. Let w1 , w2 , w3 , w4 , w5 be the neighbours of v in the cyclic order in which the polygonal arcs J{v,wi } leave v. We first claim that there are no vertex-disjoint paths P from w1 to w3 and Q from w2 to w4 in G − v. To prove this, let P be a w1 -w3 -path, and let C be the circuit S in G consisting of P and the edges {v, w1 }, {v, w3 }. By Theorem 2.30 R2 \ e∈E(C) Je splits into two connected regions, and v is on the boundary of both regions. Hence w2 and w4 belong to different regions of that set, implying that every w2 -w4 -path in G − v must contain a vertex of C.

412

16 Approximation Algorithms

Let X be the connected component of the graph G[{x ∈ V (G) \ {v} : f (x) ∈ { f (w1 ), f (w3 )}}] which contains w1 . If X does not contain w3 , we can exchange the colours in X and afterwards extend the colouring to G by colouring v with the old colour of w1 . So we may assume that there is a w1 -w3 -path P containing only vertices coloured with f (w1 ) or f (w3 ). Analogously, we are done if there is no w2 -w4 -path Q containing only vertices coloured with f (w2 ) or f (w4 ). But the contrary assumption means that there are vertex-disjoint paths P from w1 to w3 and Q from w2 to w4 in G − v, a contradiction. Hence this is a second NP-hard problem which has an absolute approximation algorithm. Indeed, the Four Colour Theorem implies that the chromatic number of a non-bipartite planar graph can only be 3 or 4. Using the polynomial-time algorithm of Robertson et al. [1996], which colours any given planar graph with four colours, one obtains an absolute approximation algorithm which uses at most one colour more than necessary. F¨urer and Raghavachari [1994] detected a third natural problem which can be approximated up to an additive error of one: Given an undirected graph, they look for a spanning tree whose maximum degree is minimum among all the spanning trees (the problem is a generalization of the H AMILTONIAN PATH P ROBLEM and thus NP-hard). Their algorithm also extends to a general case corresponding to the S TEINER T REE P ROBLEM: given a set T ⊆ V (G), find a tree S in G with V (T ) ⊆ V (S) such that the maximum degree of S is minimum. Singh and Lau [2007] found an extension to minimum weight spanning trees with bounded degrees. On the other hand, the following theorem tells that many problems do not have absolute approximation algorithms unless P = NP: Proposition 16.23. Let F and F 0 be (infinite) families of finite sets, and let P be the following optimization problem: Given a set E ∈ F and a function c : E → Z, find a set F ⊆ E with F ∈ F 0 and c(F) minimum (or decide that no such F exists). Then P has an absolute approximation algorithm if and only if P can be solved in polynomial time. Proof: Suppose there is a polynomial-time algorithm A and an integer k such that |A((E, c)) − OPT((E, c))| ≤ k for all instances (E, c) of P. We show how to solve P exactly in polynomial time. Given an instance (E, c) of P, we construct a new instance (E, c0 ), where 0 c (e) := (k + 1)c(e) for all e ∈ E. Obviously the optimum solutions remain the same. But if we now apply A to the new instance, |A((E, c0 )) − OPT((E, c0 ))| ≤ k and thus A((E, c0 )) = OPT((E, c0 )).

16.4 Approximation Schemes

413

Examples are the M INIMIZATION P ROBLEM F OR I NDEPENDENCE S YSTEMS and the M AXIMIZATION P ROBLEM F OR I NDEPENDENCE S YSTEMS (multiply c by −1), and thus all problems in the list of Section 13.1.

16.4 Approximation Schemes Recall the absolute approximation algorithm for the E DGE -C OLOURING P ROBLEM discussed in the previous section. This also implies a relative performance guarantee: Since one can easily decide if the edge-chromatic number is 1 or 2 (Proposition 16.15), Vizing’s Theorem yields a 43 -factor approximation algorithm. On the other hand, Theorem 16.14(a) implies that no k-factor approximation algorithm exists for any k < 43 (unless P = NP). Hence the existence of an absolute approximation algorithm does not imply the existence of a k-factor approximation algorithm for all k > 1. We shall meet a similar situation with the B IN -PACKING P ROBLEM in Chapter 18. This consideration suggests the following definition: Definition 16.24. Let P be an optimization problem with nonnegative weights. An asymptotic k-factor approximation algorithm for P is a polynomial-time algorithm A for P for which there exists a constant c such that 1 OPT(I ) − c ≤ A(I ) ≤ k OPT(I ) + c k for all instances I of P. We also say that A has asymptotic performance ratio k. The (asymptotic) approximation ratio of an optimization problem P with nonnegative weights is defined to be the infimum of all numbers k for which there exists an (asymptotic) k-factor approximation algorithm for P, or ∞ if there is no (asymptotic) approximation algorithm at all. For example, the above-mentioned E DGE -C OLOURING P ROBLEM has approximation ratio 43 (unless P = NP), but asymptotic approximation ratio 1 (not only in simple graphs; see Sanders and Steurer [2005]). Optimization problems with (asymptotic) approximation ratio 1 are of particular interest. For these problems we introduce the following notion: Definition 16.25. Let P be an optimization problem with nonnegative weights. An approximation scheme for P is an algorithm A accepting as input an instance I of P and an > 0 such that, for each fixed , A is a (1 + )-factor approximation algorithm for P. An asymptotic approximation scheme for P is a pair of algorithms (A, A0 ) with the following properties: A0 is a polynomial-time algorithm accepting a number > 0 as input and computing a number c . A accepts an instance I of P and an > 0 as input, and its output consists of a feasible solution for I satisfying

414

16 Approximation Algorithms

1 OPT(I ) − c ≤ A(I, ) ≤ (1 + ) OPT(I ) + c . 1+ For each fixed , the running time of A is polynomially bounded in size(I ). An (asymptotic) approximation scheme is called a fully polynomial (asymptotic) approximation scheme if the running time as well as the maximum size of any number occurring in the computation is bounded by a polynomial in size(I ) + size() + 1 . In some other texts one finds the abbreviations PTAS for (polynomial-time) approximation scheme and FPAS or FPTAS for fully polynomial approximation scheme. Apart from absolute approximation algorithms, a fully polynomial approximation scheme can be considered the best we may hope for when faced with an NP-hard optimization problem, at least if the cost of any feasible solution is a nonnegative integer (which can be assumed in many cases without loss of generality): Proposition 16.26. Let P = (X, (Sx )x∈X , c, goal) be an optimization problem where the values of c are nonnegative integers. Let A be an algorithm which, given an instance I of P and a number > 0, computes a feasible solution of I with 1 OPT(I ) ≤ A(I, ) ≤ (1 + ) OPT(I ) 1+ and whose running time is bounded by a polynomial in size(I ) + size(). Then P can be solved exactly in polynomial time. 1 Proof: Given an instance I , we first run A on (I, 1). We set := 1+2A(I,1) and observe that OPT(I ) < 1. Now we run A on (I, ). Since size() is polynomially bounded in size(I ), this procedure constitutes a polynomial-time algorithm. If P is a minimization problem, we have

A(I, ) ≤ (1 + ) OPT(I ) < OPT(I ) + 1, which, since c is integral, implies optimality. Similarly, if P is a maximization problem, we have A(I, ) ≥

1 OPT(I ) > (1 − ) OPT(I ) > OPT(I ) − 1. 1+

Unfortunately, a fully polynomial approximation scheme exists only for very few problems (see Theorem 17.11). Moreover we note that even the existence of a fully polynomial approximation scheme does not imply an absolute approximation algorithm; the K NAPSACK P ROBLEM is an example. In Chapters 17 and 18 we shall discuss two problems (K NAPSACK and B IN PACKING) which have a fully polynomial approximation scheme and a fully polynomial asymptotic approximation scheme, respectively. For many problems the two types of approximation schemes coincide:

16.5 Maximum Satisfiability

415

Theorem 16.27. (Papadimitriou and Yannakakis [1993]) Let P be an optimization problem with nonnegative weights. Suppose that for each constant k there is a polynomial-time algorithm which decides whether a given instance has optimum value at most k, and, if so, finds an optimum solution. Then P has an approximation scheme if and only if P has an asymptotic approximation scheme. Proof: The only-if-part is trivial, so suppose that P has an asymptotic approximation scheme (A, A0 ). We describe an approximation scheme for P. − 2 Let a fixed > 0 be given; we may assume < 1. We set 0 := 2++ 2 < 2 and first run A0 on the input 0 , yielding a constant c 0 . 2c 0 For a given instance I we next test whether OPT(I ) is at most . This is a constant for each fixed , so we can decide this in polynomial time and find an 2c 0 optimum solution if OPT(I ) ≤ . Otherwise we apply A to I and 0 and obtain a solution of value V , with 1 OPT(I ) − c 0 ≤ V ≤ (1 + 0 ) OPT(I ) + c 0 . 1 + 0 We claim that this solution is good enough. Indeed, we have c 0 < 2 OPT(I ) which implies OPT(I ) + OPT(I ) = (1 + ) OPT(I ) V ≤ (1 + 0 ) OPT(I ) + c 0 < 1 + 2 2 and V

1 OPT(I ) − OPT(I ) 0 (1 + ) 2 2 2++ = OPT(I ) − OPT(I ) 2 + 2 2 1 = + OPT(I ) − OPT(I ) 1+ 2 2 1 OPT(I ). = 1+ ≥

So the definition of an asymptotic approximation scheme is meaningful only for problems (such as bin-packing or colouring problems) whose restriction to a constant optimum value is still difficult. For many problems this restriction can be solved in polynomial time by some kind of complete enumeration.

16.5 Maximum Satisfiability The S ATISFIABILITY Problem was our first NP-complete problem. In this section we analyse the corresponding optimization problem:

416

16 Approximation Algorithms

M AXIMUM S ATISFIABILITY (M AX -S AT ) Instance: Task:

A set X of variables, a family Z of clauses over X , and a weight function c : Z → R+ . Find a truth assignment T of X such that the total weight of the clauses in Z that are satisfied by T is maximum.

As we shall see, approximating M AX -S AT is a nice example (and historically one of the first) for the algorithmic use of the probabilistic method. Let us first consider the following trivial randomized algorithm: set each variable independently true with probability 12 . Obviously this algorithm satisfies each clause Z with probability 1 − 2−|Z | . Let us write r for random variables which are true with probability 12 and false otherwise, and let R = (r, r, . . . , r ) be the random variable uniformly distributed over all truth assignments (several copies of r are independent of each other). If we write c(T ) for the total weight of the clauses satisfied by the truth assignment T , the expected total weight of the clauses satisfied by R is X c(Z ) Prob(R satisfies Z ) Exp (c(R)) = Z ∈Z

=

X

c(Z ) 1 − 2−|Z |

(16.5)

Z ∈Z

≥

1 − 2− p

X

c(Z ),

Z ∈Z

where p := min Z ∈Z |Z |; Exp and Prob P denote expectation and probability. Since the optimum cannot exceed Z ∈Z c(Z ), R is expected to yield a solution within a factor 1−21 − p of the optimum. But what we would really like to have is a deterministic approximation algorithm. In fact, we can turn our (trivial) randomized algorithm into a deterministic algorithm while preserving the performance guarantee. This step is often called derandomization. Let us fix the truth assignment step by step. Suppose X = {x1 , . . . , xn }, and we have already fixed a truth assignment T for x1 , . . . , xk (0 ≤ k < n). If we now set xk+1 , . . . , xn randomly, setting each variable independently true with probability 12 , we will satisfy clauses of expected total weight e0 = Exp(c(T (x1 ), . . . , T (xk ), r, . . . , r )). If we set xk+1 true (false), and then set xk+2 , . . . , xn randomly, the satisfied clauses will have some expected total weight e1 (e2 , respectively). e1 and e2 can be thought of as conditional expectations. 2 Trivially e0 = e1 +e 2 , so at least one of e1 , e2 must be at least e0 . We set x k+1 to be true if e1 ≥ e2 and false otherwise. This is sometimes called the method of conditional probabilities.

16.5 Maximum Satisfiability

417

J OHNSON ’ S A LGORITHM F OR M AX -S AT Input: Output:

A set X = {x1 , . . . , xn } of variables, a family Z of clauses over X , and a weight function c : Z → R+ . A truth assignment T : X → {true, false}.

For k := 1 to n do: If Exp(c(T (x1 ), . . . , T (xk−1 ), true, r, . . . , r )) ≥ Exp(c(T (x1 ), . . . , T (xk−1 ), false, r, . . . , r )) then set T (xk ) := true else set T (xk ) := false.

1

The expectations can be easily computed with (16.5). Theorem 16.28. (Johnson [1974]) J OHNSON ’ S A LGORITHM F OR M AX -S AT is a 1−21 − p -factor approximation algorithm for M AX -S AT, where p is the minimum cardinality of a clause. Proof: Let us define the conditional expectation sk := Exp(c(T (x1 ), . . . , T (xk ), r, . . . , r )) for k = 0, . . . , n. Observe that sn = c(T ) is the total weight P of the clauses satisfied by our algorithm, while s0 = Exp(c(R)) ≥ 1 − 2− p Z ∈Z c(Z ) by (16.5). = 1, . . . , n). So Furthermore, si ≥Psi−1 by the choice of T (xi ) in 1 (for i P c(Z ). Since the optimum is at most s n ≥ s0 ≥ 1 − 2 − p Z ∈Z Z ∈Z c(Z ), the proof is complete. Since p ≥ 1, we have a 2-factor approximation algorithm. However, this is not too interesting as there is a much simpler 2-factor approximation algorithm: either set all variables true or all false, whichever is better. However, Chen, Friesen and Zheng [1999] showed that J OHNSON ’ S A LGORITHM F OR M AX -S AT is indeed a 3 2 -factor approximation algorithm. If there are no one-element clauses ( p ≥ 2), it is a 43 -factor approximation algorithm (by Theorem 16.28), for p ≥ 3 it is an 78 -factor approximation algorithm. Yannakakis [1994] found a 43 -factor approximation algorithm for the general case using network flow techniques. We shall describe a simpler 43 -factor approximation algorithm due to Goemans and Williamson [1994]. It is straightforward to translate M AX -S AT into an integer linear program: If we have variables X = {x1 , . . . , xn }, clauses Z = {Z 1 , . . . , Z m }, and weights c1 , . . . , cm , we can write max

m X

cj z j

j=1

s.t.

zj

≤

X i:xi ∈Z j

yi , z j

∈ {0, 1}

yi +

X

(1 − yi )

( j = 1, . . . , m)

i:xi ∈Z j

(i = 1, . . . , n, j = 1, . . . , m).

418

16 Approximation Algorithms

Here yi = 1 means that variable xi is true, and z j = 1 means that clause Z j is satisfied. Now consider the LP relaxation: max

m X

cj z j

j=1

s.t.

zj

X

≤

i:xi ∈Z j

yi yi zj zj

≤ ≥ ≤ ≥

yi +

X

(1 − yi )

( j = 1, . . . , m)

i:xi ∈Z j

(i = 1, . . . , n) (i = 1, . . . , n) ( j = 1, . . . , m) ( j = 1, . . . , m).

1 0 1 0

(16.6)

Let (y ∗ , z ∗ ) be an optimum solution of (16.6). Now independently set each variable xi true with probability yi∗ . This step is known as randomized rounding, a technique which has been introduced by Raghavan and Thompson [1987]. The above method constitutes another randomized algorithm for M AX -S AT, which can be derandomized as above. Let r p be the random variable which is true with probability p and false otherwise.

G OEMANS -W ILLIAMSON A LGORITHM F OR M AX -S AT Input: Output:

A set X = {x1 , . . . , xn } of variables, a family Z of clauses over X , and a weight function c : Z → R+ . A truth assignment T : X → {true, false}.

1

Solve the linear program (16.6); let (y ∗ , z ∗ ) be an optimum solution.

2

For k := 1 to n do: ∗ , . . . , r y∗ ) If Exp(c(T (x1 ), . . . , T (xk−1 ), true, r yk+1 n ∗ , . . . , r y∗ ) ≥ Exp(c(T (x1 ), . . . , T (xk−1 ), false, r yk+1 n then set T (xk ) := true else set T (xk ) := false.

Theorem 16.29. (Goemans and Williamson [1994]) The G OEMANS -W ILLIAM 1 q -factor approximation algorithm, SON A LGORITHM F OR M AX -S AT is a 1 1− 1− q

where q is the maximum cardinality of a clause. Proof: Let us write ∗ , . . . , r y ∗ )) sk := Exp(c(T (x1 ), . . . , T (xk ), r yk+1 n

for k = 0, . . . , n. We again have si ≥ si−1 for i = 1, . . . , n and sn = c(T ) is the total weight of clauses satisfied by our algorithm. So it remains to estimate s0 = Exp(c(R y ∗ )), where R y ∗ = (r y1∗ , . . . , r yn∗ ). For j = 1, . . . , m, the probability that the clause Z j is satisfied by R y ∗ is

16.5 Maximum Satisfiability

1−

Y

(1 − yi∗ ) ·

i:xi ∈Z j

419

Y

yi∗ .

i:xi ∈Z j

Since the geometric mean is always less than or equal to the arithmetic mean, this probability is at least |Z j | X X 1 1− (1 − yi∗ ) + yi∗ |Z j | i:xi ∈Z j

i:xi ∈Z j

|Z j |

= 1 − 1 −

1 |Z j |

X

yi∗ +

i:xi ∈Z j

≥ 1− 1− ≥

z ∗j

X

(1 − yi∗ )

i:xi ∈Z j

!|Z j |

|Z j | ! 1 |Z j | ∗ 1− 1− zj . |Z j |

To prove the last inequality, observe that for any 0 ≤ a ≤ 1 and any k ∈ N ! a k 1 k 1− 1− ≥ a 1− 1− k k holds: both sides of the inequality are equal for a ∈ {0, 1}, and the left-hand side (as a function of a) is concave, while the right-hand side is linear. So we have s0 = Exp(c(R y ∗ )) =

m X

c j Prob(R y ∗ satisfies Z j )

j=1

≥

m X

cj

j=1

1 1− 1− |Z j |

|Z j | !

z ∗j

m 1 q X ∗ ≥ 1− 1− cj z j q j=1

(observe that the sequence

1−

1 k

k

is monotonously increasing and conP ∗ verges to 1e ). Since the optimum is less than or equal to m j=1 z j c j , the optimum value of the LP relaxation, the proof is complete. q e e -factor approximation algorithm ( e−1 is Since 1 − q1 < 1e , we have an e−1 about 1.582). k∈N

420

16 Approximation Algorithms

We now have two similar algorithms that behave differently: the first one is better for long clauses, while the second is better for short clauses. Hence it is natural to combine them: Theorem 16.30. (Goemans and Williamson [1994]) The following is a 43 -factor approximation algorithm for M AX -S AT: run both J OHNSON ’ S A LGORITHM F OR M AX -S AT and the G OEMANS -W ILLIAMSON A LGORITHM F OR M AX -S AT and choose the better of the two solutions. Proof: We use the notation of the above proofs. The algorithm returns a truth assignment satisfying clauses of total weight at least max{Exp(c(R)), Exp(c(R y ∗ ))} 1 Exp(c(R)) + Exp(c(R y ∗ )) ≥ 2 ! |Z j | ! m 1X 1 ≥ 1 − 2−|Z j | c j + 1 − 1 − z ∗j c j 2 |Z j | j=1 |Z j | ! m X 1 1 2 − 2−|Z j | − 1 − ≥ z ∗j c j 2 |Z j | j=1

≥

m 3X

4

z ∗j c j .

j=1

k For the last inequality observe that 2 − 2−k − 1 − k1 ≥ 32 for all k ∈ N: for k k ∈ {1, 2} we have equality; for k ≥ 3 we have 2−2−k − 1 − k1 ≥ 2− 18 − 1e > 32 . P ∗ Since the optimum is at least m j=1 z j c j , the theorem is proved. Slightly better approximation algorithms for M AX -S AT (using semidefinite programming) have been found; see Goemans and Williamson [1995], Mahajan and Ramesh [1999], and Feige and Goemans [1995]. The currently best known algorithm achieves a performance ratio of 1.270 (Asano [2006]). Indeed, Bellare and Sudan [1994] showed that approximating M AX -S AT to 74 within a factor of 73 is NP-hard. Even for M AX -3S AT (which is M AX -S AT restricted to instances where each clause has exactly three literals) no approximation scheme exists (unless P = NP), as we shall show in the next section.

16.6 The PCP Theorem Many non-approximability results are based on a deep theorem which gives a new characterization of the class NP. Recall that a decision problem belongs to NP if and only if there is a polynomial-time certificate-checking algorithm. Now

16.6 The PCP Theorem

421

we consider randomized certificate-checking algorithms that read the complete instance but only a small part of the certificate to be checked. They always accept yes-instances with correct certificates but sometimes also accept no-instances. Which bits of the certificate are read is decided randomly in advance; more precisely this decision depends on the instance x and on O(log(size(x))) random bits. We now formalize this concept. If s is a string and t ∈ Nk , then st denotes the string of length k whose i-th component is the ti -th component of s (i = 1, . . . , k). Definition 16.31. A decision problem (X, Y ) belongs to the class PCP(log n,1) if there is a polynomial p and a constant k ∈ N, a function n o f : (x, r ) : x ∈ X, r ∈ {0, 1}blog( p(size(x)))c → Nk computable in polynomial time, with f (x, r ) ∈ {1, . . . , b p(size(x))c}k for all x and r , and a decision problem (X 0 , Y 0 ) in P, where X 0 := {(x, π, γ ) : x ∈ X, π ∈ {1, . . . , b p(size(x))c}k , γ ∈ {0, 1}k }, such that for any instance x ∈ X : If x ∈ Y then there exists a c ∈ {0, 1}b p(size(x))c with Prob (x, f (x, r ), c f (x,r ) ) ∈ Y 0 = 1. If x ∈ / Y then Prob (x, f (x, r ), c f (x,r ) ) ∈ Y 0 < 12 for all c ∈ {0, 1}b p(size(x))c . Here the probability is taken over the uniform distribution of random strings r ∈ {0, 1}blog( p(size(x)))c . The letters “PCP” stand for “probabilistically checkable proof ”. The parameters log n and 1 reflect that, for an instance of size n, O(log n) random bits are used and O(1) bits of the certificate are read. For any yes-instance there is a certificate which is always accepted; while for no-instances no string is accepted as a certificate with probability 12 or more. Note that this error probability 12 can be replaced equivalently by any number between zero and one (Exercise 19). Proposition 16.32. PCP(log n, 1) ⊆ NP. Proof: Let (X, Y ) ∈ PCP(log n, 1), and let p, k, f, (X 0 , Y 0 ) be given as in Def 00 inition 16.31. Let X := (x, c) : x ∈ X, c ∈ {0, 1}b p(size(x))c , and let Y 00 := (x, c) ∈ X 00 : Prob (x, f (x, r ), c f (x,r ) ) ∈ Y 0 = 1 . To show that (X, Y ) ∈ NP it suffices to show that (X 00 , Y 00 ) ∈ P. But since there are only 2blog( p(size(x)))c , i.e. at most p(size(x)) many strings r ∈ {0, 1}blog( p(size(x)))c , we can try them all. For each one we compute f (x, r ) and test whether (x, f (x, r ), c f (x,r ) ) ∈ Y 0 (we use that (X 0 , Y 0 ) ∈ P). The overall running time is polynomial in size(x). Now the surprising result is that these randomized verifiers, which read only a constant number of bits of the certificate, are as powerful as the standard (deterministic) certificate-checking algorithms which have the full information. This is the so-called PCP Theorem:

422

16 Approximation Algorithms

Theorem 16.33. (Arora et al. [1998]) NP = PCP(log n, 1).

The proof of NP ⊆ PCP(log n, 1) is very difficult and beyond the scope of this book. It is based on earlier (and weaker) results of Feige et al. [1996] and Arora and Safra [1998]. For a self-contained proof of the PCP Theorem 16.33, see also Arora [1994], Hougardy, Pr¨omel and Steger [1994], or Ausiello et al. [1999]. Stronger results were found subsequently by Bellare, Goldreich and Sudan [1998] and H˚astad [2001]. For example, the number k in Definition 16.31 can be chosen to be 9. A new proof of the PCP Theorem was proposed by Dinur [2007]. We show some of its consequences for the non-approximability of combinatorial optimization problems. We start with the M AXIMUM C LIQUE P ROBLEM and the M AXIMUM S TABLE S ET P ROBLEM: given an undirected graph G, find a clique, or a stable set, of maximum cardinality in G. Recall Proposition 2.2 (and Corollary 15.24): The problems of finding a maximum clique, a maximum stable set, or a minimum vertex cover are all equivalent. However, the 2-factor approximation algorithm for the M INIMUM V ERTEX C OVER P ROBLEM (Section 16.1) does not imply an approximation algorithm for the M AX IMUM S TABLE S ET P ROBLEM or the M AXIMUM C LIQUE P ROBLEM . Namely, it can happen that the algorithm returns a vertex cover C of size n − 2, while the optimum is n2 −1 (where n = |V (G)|). The complement V (G)\C is then a stable set of cardinality 2, but the maximum stable set has cardinality n2 + 1. This example shows that transferring an algorithm to another problem via a polynomial transformation does not in general preserve its performance guarantee. We shall consider a restricted type of transformation in the next section. Here we deduce a non-approximability result for the M AXIMUM C LIQUE P ROBLEM from the PCP Theorem: Theorem 16.34. (Arora and Safra [1998]) Unless P = NP there is no 2-factor approximation algorithm for the M AXIMUM C LIQUE P ROBLEM. Proof: Let P = (X, Y ) be some NP-complete problem. By the PCP Theorem 16.33, P ∈ PCP(log n, 1), so let p, k, f , P 0 := (X 0 , Y 0 ) be as in Definition 16.31. For any given x ∈ X we construct a graph G x as follows. Let n o V (G x ) := (r, a) : r ∈ {0, 1}blog( p(size(x)))c , a ∈ {0, 1}k , (x, f (x, r ), a) ∈ Y 0 (representing all “accepting runs” of the randomized certificate checking algorithm). Two vertices (r, a) and (r 0 , a 0 ) are joined by an edge if ai = a j0 whenever the i-th component of f (x, r ) equals the j-th component of f (x, r 0 ). Since P 0 ∈ P and there are only a polynomial number of random strings, G x can be computed in polynomial time (and has polynomial size).

16.6 The PCP Theorem

423

If x ∈ Y then by definition there exists a certificate c ∈ {0, 1}b p(size(x))c such that (x, f (x, r ), c f (x,r ) ) ∈ Y 0 for all r ∈ {0, 1}blog( p(size(x)))c . Hence there is a clique of size 2blog( p(size(x)))c in G x . On the other hand, if x ∈ / Y then there is no clique of size 12 2blog( p(size(x)))c in G x : Suppose (r (1) , a (1) ), . . . , (r (t) , a (t) ) are the vertices of a clique. Then r (1) , . . . , r (t) ( j) are pairwise different. We set ci := ak whenever the k-th component of f (x, r ( j) ) equals i, and set the remaining components of c (if any) arbitrarily. This way we obtain a certificate c with (x, f (x, r (i) ), c f (x,r (i) ) ) ∈ Y 0 for all i = 1, . . . , t. If x∈ / Y we have t < 12 2blog( p(size(x)))c . So any 2-factor approximation algorithm for the M AXIMUM C LIQUE P ROBLEM is able to decide if x ∈ Y , i.e. to solve P. Since P is NP-complete, this is possible only if P = NP. The reduction in the above proof is due to Feige et al. [1996]. Since the error probability 12 in Definition 16.31 can be replaced by any number between 0 and 1 (Exercise 19), we get that there is no ρ-factor approximation algorithm for the M AXIMUM C LIQUE P ROBLEM for any ρ ≥ 1 (unless P = NP). With more effort Zuckerman [2006] showed that, unless P = NP, no polynomialtime algorithm computes the maximum size of a clique in any graph with n vertices up to a factor of n 1− , for any fixed > 0. The best known algorithm guarantees to k log3 n in this case (Feige [2004]). Of course, all this also find a clique of size n(log log n)2 holds for the M AXIMUM S TABLE S ET P ROBLEM (by considering the complement of the given graph). Now we turn to the following restriction of M AX -S AT:

M AX -3S AT Instance: Task:

A set X of variables and a family Z of clauses over X , each with exactly three literals. Find a truth assignment T of X such that the number of clauses in Z that are satisfied by T is maximum.

In Section 16.5 we had a simple 87 -factor approximation algorithm for M AX 3S AT, even for the weighted form (Theorem 16.28). H˚astad [2001] showed that this is best possible: no ρ-factor approximation algorithm for M AX -3S AT can exist for any ρ < 78 unless P = NP. Here we prove the following weaker result: Theorem 16.35. (Arora et al. [1998]) Unless P = NP there is no approximation scheme for M AX -3S AT. Proof: Let P = (X, Y ) be some NP-complete problem. By the PCP Theorem 16.33, P ∈ PCP(log n, 1), so let p, k, f , P 0 := (X 0 , Y 0 ) be as in Definition 16.31. For any given x ∈ X we construct a 3S AT-instance Jx . Namely, for each random string r ∈ {0, 1}blog( p(size(x)))c we define a family Zr of 3S AT-clauses (the union of all these families will be Jx ). We first construct a family Zr0 of clauses with an arbitrary number of literals and then apply Proposition 15.21.

424

16 Approximation Algorithms

So let r ∈ {0, 1}blog( p(size(x)))c and f (x, r ) = (t1 , . . . , tk ). Let {a (1) , . . . , a (sr ) } be the set of strings a ∈ {0, 1}k for which (x, f (x, r ), a) ∈ Y 0 . If sr = 0 then we simply set Z 0 := {{y}, { y¯ }}, where y is some variable not used anywhere else. Otherwise let c ∈ {0, 1}b p(size(x))c . We have that (x, f (x, r ), c f (x,r ) ) ∈ Y 0 if and only if ! sr k _ ^ ( j) cti = ai . j=1

i=1

This is equivalent to ^

( j)

cti j = ai j

(i 1 ,...,i sr )∈{1,...,k}sr

sr _

.

j=1

This conjunction of clauses can be constructed in polynomial time because P 0 ∈ P and k is a constant. By introducing Boolean variables π1 , . . . , πb p(size(x))c representing the bits c1 , . . . , cb p(size(x))c we obtain a family Zr0 of k sr clauses (each with sr literals) such that Zr0 is satisfied if and only if (x, f (x, r ), c f (x,r ) ) ∈ Y 0 . By Proposition 15.21, we can rewrite each Zr0 equivalently as a conjunction of 3S AT-clauses, where the number of clauses increases by at most a factor of max{sr − 2, 4}. Let this family of clauses be Zr . Since sr ≤ 2k , each Zr consists k of at most l := k 2 max{2k − 2, 4} 3S AT-clauses. Our 3S AT-instance Jx is the union of all the families Zr for all r . Note that Jx can be computed in polynomial time. Now if x is a yes-instance, then there exists a certificate c as in Definition 16.31. This c immediately defines a truth assignment satisfying Jx . On the other hand, if x is a no-instance, then only 21 of the formulas Zr are simultaneously satisfiable. So in this case any truth assignment leaves at least a fraction of 2l1 of the clauses unsatisfied. 2l So any k-factor approximation algorithm for M AX -3S AT with k < 2l−1 satisfies 2l−1 1 more than a fraction of 2l = 1 − 2l of the clauses of any satisfiable instance. Hence such an algorithm can decide whether x ∈ Y or not. Since P is NP-complete, such an algorithm cannot exist unless P = NP.

16.7 L-Reductions Our goal is to show, for other problems than M AX -3S AT, that they have no approximation scheme unless P = NP. As with the NP-completeness proofs (Section 15.5), it is not necessary to have a direct proof using the definition of PCP(log n, 1) for each problem. Rather we use a certain type of reduction which preserves approximability (general polynomial transformations do not): Definition 16.36. Let P = (X, (Sx )x∈X , c, goal) and P 0 = (X 0 , (Sx0 )x∈X 0 , c0 , goal0 ) be two optimization problems with nonnegative weights. An L-reduction from P

16.7 L-Reductions

425

to P 0 is a pair of functions f and g, both computable in polynomial time, and two constants α, β > 0 such that for any instance x of P: (a) f (x) is an instance of P 0 with OPT( f (x)) ≤ α OPT(x); (b) For any feasible solution y 0 of f (x), g(x, y 0 ) is a feasible solution of x such that |c(x, g(x, y 0 )) − OPT(x)| ≤ β|c0 ( f (x), y 0 ) − OPT( f (x))|. We say that P is L-reducible to P 0 if there is an L-reduction from P to P 0 . The letter “L” in the term L-reduction stands for “linear”. L-reductions were introduced by Papadimitriou and Yannakakis [1991]. The definition immediately implies that L-reductions can be composed: Proposition 16.37. Let P, P 0 , P 00 be optimization problems with nonnegative weights. If ( f, g, α, β) is an L-reduction from P to P 0 and ( f 0 , g 0 , α 0 , β 0 ) is an Lreduction from P 0 to P 00 , then their composition ( f 00 , g 00 , αα 0 , ββ 0 ) is an L-reduction from P to P 00 , where f 00 (x) = f 0 ( f (x)) and g 00 (x, y 00 ) = g(x, g 0 (x 0 , y 00 )). The decisive property of L-reductions is that they preserve approximability: Theorem 16.38. (Papadimitriou and Yannakakis [1991]) Let P and P 0 be two optimization problems with nonnegative weights. Let ( f, g, α, β) be an L-reduction from P to P 0 . If there is an approximation scheme for P 0 , then there is an approximation scheme for P. Proof: Given an instance x of P and a number 0 < < 1, we apply the approximation scheme for P 0 to f (x) and 0 := 2αβ . We obtain a feasible solution 0 0 y of f (x) and finally return y := g(x, y ), a feasible solution of x. Since |c(x, y) − OPT(x)| ≤ β|c0 ( f (x), y 0 ) − OPT( f (x))| n ≤ β max (1 + 0 ) OPT( f (x)) − OPT( f (x)), o 1 OPT( f (x)) OPT( f (x)) − 0 1+ ≤ β 0 OPT( f (x)) ≤ αβ 0 OPT(x) OPT(x) = 2 we get c(x, y) ≤ OPT(x) + |c(x, y) − OPT(x)| ≤

1+ OPT(x) 2

and c(x, y) ≥ OPT(x)−| OPT(x)−c(x, y)| ≥

1−

1 OPT(x) > OPT(x), 2 1+

so this constitutes an approximation scheme for P.

This theorem together with Theorem 16.35 motivates the following definition:

426

16 Approximation Algorithms

Definition 16.39. An optimization problem P with nonnegative weights is called MAXSNP-hard if M AX -3S AT is L-reducible to P. The name MAXSNP refers to a class of optimization problems introduced by Papadimitriou and Yannakakis [1991]. Here we do not need this class, so we omit its (nontrivial) definition. Corollary 16.40. Unless P = NP there is no approximation scheme for any MAXSNP-hard problem. Proof:

Directly from Theorems 16.35 and 16.38.

We shall show MAXSNP-hardness for several problems by describing Lreductions. We start with a restricted version of M AX -3S AT:

3-O CCURRENCE M AX -S AT P ROBLEM Instance:

Task:

A set X of variables and a family Z of clauses over X , each with at most three literals, such that no variable occurs in more than three clauses. Find a truth assignment T of X such that the number of clauses in Z that are satisfied by T is maximum.

That this problem is NP-hard can be proved by a simple transformation from 3S AT (or M AX -3S AT), cf. Exercise 9 of Chapter 15. Since this transformation is not an L-reduction, it does not imply MAXSNP-hardness. We need a more complicated construction, using so-called expander graphs: Definition 16.41. Let G be an undirected graph and γ > 0 a constant. G is a we have |0(A)| ≥ γ |A|. γ -expander if for each A ⊆ V (G) with |A| ≤ |V (G)| 2 For example, a complete graph is a 1-expander. However, one is interested in expanders with a small number of edges. We cite the following theorem without its quite complicated proof: Theorem 16.42. (Ajtai [1994]) There exists a positive constant γ such that for any given even integer n ≥ 4, a 3-regular γ -expander with n vertices can be constructed in O(n 3 log3 n) time. The following corollary was mentioned (and used) by Papadimitriou [1994], and a correct proof was given by Fern´andez-Baca and Lagergren [1998]: Corollary 16.43. For any given integer n ≥ 3, a digraph G with O(n) vertices and a set S ⊆ V (G) of cardinality n with the following properties can be constructed in O(n 3 log3 n) time: |δ − (v)| + |δ + (v)| ≤ 3 for each v ∈ V (G); |δ − (v)| + |δ + (v)| = 2 for each v ∈ S; and |δ + (A)| ≥ min{|S ∩ A|, |S \ A|} for each A ⊆ V (G).

16.7 L-Reductions

427

l m

Proof: Let γ > 0 be the constant of Theorem 16.42, and let k := γ1 . We first construct a 3-regular γ -expander H with n or n + 1 vertices, using Theorem 16.42. We replace each edge {v, w} by k parallel edges (v, w) and k parallel edges (w, v). Let the resulting digraph be H 0 . Note that for any A ⊆ V (H 0 ) with 0 )| |A| ≤ |V (H we have 2 |δ + H 0 (A)| = k|δ H (A)| ≥ k|0 H (A)| ≥ kγ |A| ≥ |A|. Similarly we have for any A ⊆ V (H 0 ) with |A| >

|V (H 0 )| : 2

0 0 |δ + H 0 (A)| = k|δ H (V (H ) \ A)| ≥ k|0 H (V (H ) \ A)|

≥ kγ |V (H 0 ) \ A| ≥ |V (H 0 ) \ A|. 0 So in both cases we have |δ + H 0 (A)| ≥ min{|A|, |V (H ) \ A|}. 0 Now we split up each vertex v ∈ V (H ) into 6k +1 vertices xv,i (i = 0, . . . , 6k) such that each vertex except xv,0 has degree 1. For each vertex xv,i we now add vertices wv,i, j and yv,i, j ( j = 0, . . . , 6k) connected by a path of length 12k +2 with vertices wv,i,0 , wv,i,1 , . . . , wv,i,6k , xv,i , yv,i,0 , . . . , yv,i,6k in this order. Finally we add edges (yv,i, j , wv, j,i ) for all v ∈ V (H 0 ), all i ∈ {0, . . . , 6k} and all j ∈ {0, . . . , 6k} \ {i}. Altogether we have a vertex set Z v of cardinality (6k + 1)(12k + 3) for each v ∈ V (H 0 ). The overall resulting graph G has |V (H 0 )|(6k + 1)(12k + 3) = O(n) vertices, each of degree two or three. By the construction, G[Z v ] contains min{|X 1 |, |X 2 |} vertex-disjoint paths from X 1 to X 2 for any pair of disjoint subsets X 1 , X 2 of {xv,i : i = 0, . . . , 6k}. We choose S to be an n-element subset of {xv,0 : v ∈ V (H 0 )}; note that each of these vertices has one entering and one leaving edge. It remains to prove that |δ + (A)| ≥ min{|S ∩ A|, |S \ A|} for each A ⊆ V (G). We prove this byS induction on |{v ∈ V (H 0 ) : ∅ 6= A ∩ Z v 6= Z v }|. If this number is zero, i.e. A = v∈B Z v for some B ⊆ V (H 0 ), then we have + 0 |δG (A)| = |δ + H 0 (B)| ≥ min{|B|, |V (H ) \ B|} ≥ min{|S ∩ A|, |S \ A|}.

Otherwise let v ∈ V (H 0 ) with ∅ 6= A ∩ Z v 6= Z v . Let P := {xv,i : i = 0, . . . , 6k} ∩ A and Q := {xv,i : i = 0, . . . , 6k} \ A. If |P| ≤ 3k, then by the property of G[Z v ] we have + |E G (Z v ∩ A, Z v \ A)| ≥ |P| = |P \ S| + |P ∩ S| + ≥ |E G (A \ Z v , A ∩ Z v )| + |P ∩ S|.

By applying the induction hypothesis to A \ Z v we therefore get + + |δG (A)| ≥ |δG (A \ Z v )| + |P ∩ S|

≥ min{|S ∩ (A \ Z v )|, |S \ (A \ Z v )|} + |P ∩ S| ≥ min{|S ∩ A|, |S \ A|}.

428

16 Approximation Algorithms

Similarly, if |P| ≥ 3k + 1, then |Q| ≤ 3k and by the property of G[Z v ] we have + |E G (Z v ∩ A, Z v \ A)| ≥ |Q| = |Q \ S| + |Q ∩ S| + (Z v \ A, V (G) \ (A ∪ Z v ))| + |Q ∩ S|. ≥ |E G

By applying the induction hypothesis to A ∪ Z v we therefore get + + (A)| ≥ |δG (A ∪ Z v )| + |Q ∩ S| |δG

≥ min{|S ∩ (A ∪ Z v )|, |S \ (A ∪ Z v )|} + |Q ∩ S| ≥ min{|S ∩ A|, |S \ A|}. Now we can prove: Theorem 16.44. (Papadimitriou and Yannakakis [1991], Papadimitriou [1994], Fern´andez-Baca and Lagergren [1998]) The 3-O CCURRENCE M AX -S AT P ROB LEM is MAXSNP-hard. Proof: We describe an L-reduction ( f, g, α, β) from M AX -3S AT. To define f , let (X, Z) be an instance of M AX -3S AT. For each variable x ∈ X which occurs in more than three, say in k clauses, we modify the instance as follows. We replace x by a new and distinct variable in each clause. This way we introduce new variables x1 , . . . , xk . We introduce additional constraints (and further variables) which ensure, roughly speaking, that it is favourable to assign the same truth value to all the variables x1 , . . . , xk . We construct G and S as in Corollary 16.43 and rename the vertices such that S = {1, . . . , k}. Now for each vertex v ∈ V (G) \ S we introduce a new variable xv , and for each edge (v, w) ∈ E(G) we introduce a clause {xv , xw }. In total we have added at most 2 1 1 1 3 (k + 1) 6 k +1 12 + 3 ≤ 315 2 γ γ γ new clauses, where γ is again the constant of Theorem 16.42. Applying the above substitution for each variable we obtain an instance (X 0 , Z 0 ) = f (X, Z) of the 3-O CCURRENCE M AX -S AT P ROBLEM with |Z 0 | ≤ |Z| + 315

2 2 1 1 3|Z| ≤ 946 |Z|. γ γ

Hence 2 2 1 1 |Z| ≤ 1892 OPT(X, Z), OPT(X , Z ) ≤ |Z | ≤ 946 γ γ 0

0

0

16.7 L-Reductions

429

because at least half of the clauses of a M AX -S AT-instance can be satisfied (either l m2 by setting all variables true or all false). So we can set α := 1892 γ1 . To describe g, let T 0 be a truth assignment of X 0 . We first construct a truth assignment T 00 of X 0 satisfying at least as many clauses of Z 0 as T 0 , and satisfying all new clauses (corresponding to edges of the graphs G above). Namely, for any variable x occurring more than three times in (X, Z), let G be the graph constructed above, and let A := {v ∈ V (G) : T 0 (xv ) = true}. If |S ∩ A| ≥ |S \ A| then we set T 00 (xv ) := true for all v ∈ V (G), otherwise we set T 00 (xv ) := false for all v ∈ V (G). It is clear that all new clauses (corresponding to edges) are satisfied. There are at most min{|S ∩ A|, |S \ A|} old clauses satisfied by T 0 but not by T 00 . On the other hand, T 0 does not satisfy any of the clauses {xv , xw } for + (A). By the properties of G, the number of these clauses is at least (v, w) ∈ δG min{|S ∩ A|, |S \ A|}. Now T 00 yields a truth assignment T = g(X, Z, T 0 ) of X in the obvious way: Set T (x) := T 00 (x) = T 0 (x) for x ∈ X ∩ X 0 and T (x) := T 00 (xi ) if xi is any variable replacing x in the construction from (X, Z) to (X 0 , Z 0 ). T violates as many clauses as T 00 . So if c(X, Z, T ) denotes the number of clauses in instance (X, Z) that are satisfied by T , we conclude |Z| − c(X, Z, T ) = |Z 0 | − c(X 0 , Z 0 , T 00 ) ≤ |Z 0 | − c(X 0 , Z 0 , T 0 ).

(16.7)

On the other hand, any truth assignment T of X leads to a truth assignment T 0 of X 0 violating the same number of clauses (by setting the variables xv (v ∈ V (G)) uniformly to T (x) for each variable x and the corresponding graph G in the above construction). Hence |Z| − OPT(X, Z) ≥ |Z 0 | − OPT(X 0 , Z 0 ).

(16.8)

Combining (16.7) and (16.8) we get | OPT(X, Z) − c(X, Z, T )| = (|Z| − c(X, Z, T )) − (|Z| − OPT(X, Z)) ≤ OPT(X 0 , Z 0 ) − c(X 0 , Z 0 , T 0 ) = | OPT(X 0 , Z 0 ) − c(X 0 , Z 0 , T 0 )|, where T = g(X, Z, T 0 ). So ( f, g, α, 1) is indeed an L-reduction.

This result is the starting point of several MAXSNP-hardness proofs. For example: Corollary 16.45. (Papadimitriou and Yannakakis [1991]) The M AXIMUM S TA BLE S ET P ROBLEM restricted to graphs with maximum degree 4 is MAXSNP-hard. Proof: The construction of the proof of Theorem 15.23 defines an L-reduction from the 3-O CCURRENCE M AX -S AT P ROBLEM to the M AXIMUM S TABLE S ET P ROBLEM restricted to graphs with maximum degree 4: for each instance (X, Z)

430

16 Approximation Algorithms

a graph G is constructed such that from each truth assignment satisfying k clauses one easily obtains a stable set of cardinality k, and vice versa. Indeed, the M AXIMUM S TABLE S ET P ROBLEM is MAXSNP-hard even when restricted to 3-regular graphs (Berman and Fujito [1999]). On the other hand, a simple greedy algorithm, which in each step chooses a vertex v of minimum degree and deletes v and all its neighbours, is a (k+2) 3 -factor approximation algorithm for the M AXIMUM S TABLE S ET P ROBLEM in graphs with maximum degree k (Halld´orsson and Radhakrishnan [1997]). For k = 4 this gives a performance ratio of 2 which is better than the ratio 8 we get from the following proof (using the 2-factor approximation algorithm for the M INIMUM V ERTEX C OVER P ROBLEM). Theorem 16.46. (Papadimitriou and Yannakakis [1991]) The M INIMUM V ER TEX C OVER P ROBLEM restricted to graphs with maximum degree 4 is MAXSNPhard. Proof: Consider the trivial transformation from the M AXIMUM S TABLE S ET P ROBLEM (Proposition 2.2) with f (G) := G and g(G, X ) := V (G) \ X for all graphs G and all X ⊆ V (G). Although this is not an L-reduction in general, it is an L-reduction if restricted to graphs with maximum degree 4, as we shall show. If G has maximum degree 4, there exists a stable set of cardinality at least |V (G)| 5 . So if we denote by α(G) the maximum cardinality of a stable set and by τ (G) the minimum cardinality of a vertex cover we have 1 1 (|V (G)| − α(G)) = τ (G) 4 4 and α(G) − |X | = |V (G) \ X | − τ (G) for any stable set X ⊆ V (G). Hence ( f, g, 4, 1) is an L-reduction. α(G) ≥

See Clementi and Trevisan [1999] and Chleb´ık and Chleb´ıkov´a [2006] for stronger statements. In particular, there is no approximation scheme for the M IN IMUM V ERTEX C OVER P ROBLEM (unless P = NP). We shall prove MAXSNPhardness of other problems in later chapters; see also Exercise 22.

Exercises 1. Formulate a 2-factor approximation algorithm for the following problem. Given a digraph with edge weights, find an acyclic subgraph of maximum weight. Note: No k-factor approximation algorithm for this problem is known for k < 2. 2. The k-C ENTER P ROBLEM is defined as follows: given an undirected graph G, weights c : E(G) → R+ , and a number k ∈ N with k ≤ |V (G)|, find a set X ⊆ V (G) of cardinality k such that max min dist(v, x)

v∈V (G) x∈X

is minimum. As usual we denote the optimum value by OPT(G, c, k).

Exercises

∗

3. 4. 5.

6.

∗

7.

431

(a) Let S be a maximal stable set in (V (G), {{v, w} : dist(v, w) ≤ 2R}). Show that then OPT(G, c, |S| − 1) > R. (b) Use (a) to describe a 2-factor approximation algorithm for the k-C ENTER P ROBLEM. (Hochbaum and Shmoys [1985]) (c) Show that there is no r -factor approximation algorithm for the k-C ENTER P ROBLEM for any r < 2. Hint: Use Exercise 12 of Chapter 15. (Hsu and Nemhauser [1979]) Can one find a minimum vertex cover (or a maximum stable set) in a bipartite graph in polynomial time? Show that the performance guarantee in Theorem 16.5 is tight. Show that the following is a 2-factor approximation algorithm for the M INIMUM V ERTEX C OVER P ROBLEM: compute a DFS-tree and output all its vertices with nonzero out-degree. (Bar-Yehuda [unpublished]) Show that the LP relaxation min{cx : M > x ≥ 1l, x ≥ 0} of the M INIMUM W EIGHT V ERTEX C OVER P ROBLEM, where M is the incidence matrix of an V (G) undirected graph and c ∈ R+ , always has a half-integral optimum solution 1 (i.e. one with entries 0, 2 , 1 only). Derive another 2-factor approximation algorithm from this fact. Consider the M INIMUM W EIGHT F EEDBACK V ERTEX S ET P ROBLEM: given an undirected graph G and weights c : V (G) → R+ , find a vertex set X ⊆ V (G) of minimum weight such that G − X is a forest. Consider the following recursive algorithm A: If E(G) = ∅, then return A(G, c) := ∅. If |δG (x)| ≤ 1 for some x ∈ V (G), then return A(G, c) := A(G − x, c). If c(x) = 0 for some x ∈ V (G), then return A(G, c) := {x} ∪ A(G − x, c). Otherwise let :=

c(v) x∈V (G) |δ(v)| min

and c0 (v) := c(v) − |δ(v)| (v ∈ V (G)). Let X := A(G, c0 ). For each x ∈ X do: If G − (X \ {x}) is a forest, then set X := X \ {x}. Return A(G, c) := x. Prove that this is a 2-factor approximation algorithm for the M INIMUM W EIGHT F EEDBACK V ERTEX S ET P ROBLEM. (Becker and Geiger [1996]) 8. Show that the M AXIMUM C UT P ROBLEM is NP-hard even for simple graphs. 9. Prove that the simple greedy algorithm for M AX -C UT described at the beginning of Section 16.2 is a 2-factor approximation algorithm. 10. Consider the following local search algorithm for the M AXIMUM C UT P ROB LEM . Start with any nonempty proper subset S of V (G). Now check iteratively if some vertex can be added to S or deleted from S such that |δ(S)| increases. Stop if no such improvement is possible. (a) Prove that the above is a 2-factor approximation algorithm. (Recall Exercise 10 of Chapter 2.)

432

16 Approximation Algorithms

(b) Can the algorithm be extended to the M AXIMUM W EIGHT C UT P ROBLEM, where we have nonnegative edge weights? (c) Does the algorithm always find an optimum solution for planar graphs, or for bipartite graphs? For both classes there is a polynomial-time algorithm (Exercise 7 of Chapter 12 and Proposition 2.27). 11. In the D IRECTED M AXIMUM W EIGHT C UT P ROBLEM we are given a digraph G with P weights c : E(G) → R+ , and we look for a set X ⊆ V (G) such that e∈δ + (X ) c(e) is maximum. Show that there is a 4-factor approximation algorithm for this problem. Hint: Use Exercise 10. Note: There is a 1.165-factor but no 1.09-factor approximation algorithm unless P = NP (Feige and Goemans [1995], H˚astad [2001]). 12. Show that ( π1 arccos(yi> y j ))1≤i, j≤n is a convex combination of cut semimetrics δ R , R ⊆ {1, . . . , n}, where δi,R j = 1 if |R ∩ {i, j}| = 1 and δi,R j = 0 otherwise. Hint: Write ! X [ \ µ H (yi ) \ H (yi ) δ R . (µ(H (yi )4H (y j )))1≤i, j≤n = R⊆{1,...,n}

i∈R

i ∈R /

Note: See Deza and Laurent [1997] for a lot of related information. 13. Show that for each n ∈ N there is a bipartite graph on 2n vertices for which the G REEDY C OLOURING A LGORITHM needs n colours. So the algorithm may give arbitrarily bad results. However, show that there always exists an order of the vertices for which the algorithm finds an optimum colouring. √ 14. Show that one can colour any 3-colourable graph G with at most 2 2n colours in polynomial time, where n := |V (G)|. √ Hint: As long as there is a vertex v of degree at least 2n, colour 0(v) optimally with at most two colours (not to be used again), and delete these vertices. Finally use the G REEDY C OLOURING A LGORITHM. (Wigderson [1983]) 15. Show that the following graphs are perfect: (a) bipartite graphs; (b) interval graphs: ({v1 , . . . , vn }, {{vi , v j } : i 6= j, [ai , bi ] ∩ [a j , b j ] 6= ∅}), where [a1 , b1 ], . . . , [an , bn ] is a set of closed intervals; (c) chordal graphs (see Exercise 28 of Chapter 8). ∗ 16. Let G be an undirected graph. Prove that the following statements are equivalent: (a) G is perfect. (b) For any weight function c : V (G) → Z+ the maximum weight of a clique in G equals the minimum number of stable sets such that each vertex v is contained in c(v) of them. (c) For any weight function c : V (G) → Z+ the maximum weight of a stable set in G equals the minimum number of cliques such that each vertex v is contained in c(v) of them. (d) The inequality system defining (16.3) is TDI.

Exercises

433

(e) The clique polytope of G, i.e. the convex hull of the incidence vectors of all cliques in G, is given by ( ) X V (G) : xv ≤ 1 for all stable sets S in G . (16.9) x ∈ R+ v∈S

(f) The inequality system defining (16.9) is TDI. Note: The polytope (16.9) is called the antiblocker of the polytope (16.3). 17. An instance of M AX -S AT is called k-satisfiable if any k of its clauses can be simultaneously satisfied. Let rk be the infimum of the fraction of clauses that one can satisfy in any k-satisfiable instance. (a) Prove that r1 = 21 . √

18.

19.

20.

21. 22.

(b) Prove that r2 = 5−1 2 . (Hint: Some variables occur in one-element clauses (w.l.o.g. all oneelement clauses are positive), set them true with probability a (for some 1 1 2 < a < 1), and set the other variables true with probability 2 . Apply the derandomization technique and choose a appropriately.) (c) Prove that r3 ≥ 23 . (Lieberherr and Specker [1981]) Erd˝os [1967] showed the following: for each constant k ∈ N, the (asymptotically) best fraction of the edges that we can guarantee to be in a maximum cut is 12 , even if we restrict attention to graphs without odd circuits of length k or less. (Compare Exercise 10(a).) (a) What about k = ∞? (b) Show how the M AXIMUM C UT P ROBLEM can be reduced to M AX -S AT. Hint: Use a variable for each vertex and two clauses {x, y}, {x, ¯ y¯ } for each edge {x, y}. (c) Use (b) and Erd˝os’ Theorem in order to prove that rk ≤ 34 for all k. (For a definition of rk , see Exercise 17.) Note: Trevisan [2004] proved that limk→∞ rk = 34 . Prove that the error probability 12 in Definition 16.31 can be replaced equivalently by any number between 0 and 1. Deduce from this (and the proof of Theorem 16.34) that there is no ρ-factor approximation algorithm for the M AXIMUM C LIQUE P ROBLEM for any ρ ≥ 1 (unless P = NP). Prove that the M AXIMUM C LIQUE P ROBLEM is L-reducible to the S ET PACK ING P ROBLEM : given a set system (U, S), find a maximum cardinality subfamily R ⊆ S whose elements are pairwise disjoint. Prove that the M INIMUM V ERTEX C OVER P ROBLEM has no absolute approximation algorithm (unless P = NP). Prove that M AX -2S AT is MAXSNP-hard. Hint: Use Corollary 16.45. (Papadimitriou and Yannakakis [1991])

434

16 Approximation Algorithms

References General Literature: Asano, T., Iwama, K., Takada, H., and Yamashita, Y. [2000]: Designing high-quality approximation algorithms for combinatorial optimization problems. IEICE Transactions on Communications/Electronics/Information and Systems E83-D (2000), 462–478 Ausiello, G., Crescenzi, P., Gambosi, G., Kann, V., Marchetti-Spaccamela, A., and Protasi, M. [1999]: Complexity and Approximation: Combinatorial Optimization Problems and Their Approximability Properties. Springer, Berlin 1999 Garey, M.R., and Johnson, D.S. [1979]: Computers and Intractability; A Guide to the Theory of NP-Completeness. Freeman, San Francisco 1979, Chapter 4 Hochbaum, D.S. [1996]: Approximation Algorithms for NP-Hard Problems. PWS, Boston, 1996 Horowitz, E., and Sahni, S. [1978]: Fundamentals of Computer Algorithms. Computer Science Press, Potomac 1978, Chapter 12 Shmoys, D.B. [1995]: Computing near-optimal solutions to combinatorial optimization problems. In: Combinatorial Optimization; DIMACS Series in Discrete Mathematics and Theoretical Computer Science 20 (W. Cook, L. Lov´asz, P. Seymour, eds.), AMS, Providence 1995 Papadimitriou, C.H. [1994]: Computational Complexity, Addison-Wesley, Reading 1994, Chapter 13 Vazirani, V.V. [2001]: Approximation Algorithms. Springer, Berlin, 2001

Cited References: Ajtai, M. [1994]: Recursive construction for 3-regular expanders. Combinatorica 14 (1994), 379–416 Alizadeh, F. [1995]: Interior point methods in semidefinite programming with applications to combinatorial optimization. SIAM Journal on Optimization 5 (1995), 13–51 Appel, K., and Haken, W. [1977]: Every planar map is four colorable; Part I; Discharging. Illinois Journal of Mathematics 21 (1977), 429–490 Appel, K., Haken, W., and Koch, J. [1977]: Every planar map is four colorable; Part II; Reducibility. Illinois Journal of Mathematics 21 (1977), 491–567 Arora, S. [1994]: Probabilistic checking of proofs and the hardness of approximation problems, Ph.D. thesis, U.C. Berkeley, 1994 Arora, S., Lund, C., Motwani, R., Sudan, M., and Szegedy, M. [1998]: Proof verification and hardness of approximation problems. Journal of the ACM 45 (1998), 501–555 Arora, S., and Safra, S. [1998]: Probabilistic checking of proofs. Journal of the ACM 45 (1998), 70–122 Asano, T. [2006]: An improved analysis of Goemans and Williamson’s LP-relaxation for MAX SAT. Theoretical Computer Science 354 (2006), 339–353 Bar-Yehuda, R., and Even, S. [1981]: A linear-time approximation algorithm for the weighted vertex cover problem. Journal of Algorithms 2 (1981), 198–203 Becker, A., and Geiger, D. [1996]: Optimization of Pearl’s method of conditioning and greedy-like approximation algorithms for the vertex feedback set problem. Artificial Intelligence Journal 83 (1996), 1–22 Bellare, M., and Sudan, M. [1994]: Improved non-approximability results. Proceedings of the 26th Annual ACM Symposium on the Theory of Computing (1994), 184–193

References

435

Bellare, M., Goldreich, O., and Sudan, M. [1998]: Free bits, PCPs and nonapproximability – towards tight results. SIAM Journal on Computing 27 (1998), 804–915 Berge, C. [1961]: F¨arbung von Graphen, deren s¨amtliche bzw. deren ungerade Kreise starr sind. Wissenschaftliche Zeitschrift, Martin Luther Universit¨at Halle-Wittenberg, Mathematisch-Naturwissenschaftliche Reihe (1961), 114–115 Berge, C. [1962]: Sur une conjecture relative au probl`eme des codes optimaux. Communication, 13`eme assembl´ee g´en´erale de l’URSI, Tokyo 1962 Berman, P., and Fujito, T. [1999]: On approximation properties of the independent set problem for low degree graphs. Theory of Computing Systems 32 (1999), 115–132 Brooks, R.L. [1941]: On colouring the nodes of a network. Proceedings of the Cambridge Philosophical Society 37 (1941), 194–197 Chen, J., Friesen, D.K., and Zheng, H. [1999]: Tight bound on Johnson’s algorithm for maximum satisfiability. Journal of Computer and System Sciences 58 (1999), 622–640 Chleb´ık, M. and Chleb´ıkov´a, J. [2006]: Complexity of approximating bounded variants of optimization problems. Theoretical Computer Science 354 (2006), 320-338 Chudnovsky, M., Cornu´ejols, G., Liu, X., Seymour, P., and Vu˘skovi´c, K. [2005]: Recognizing Berge graphs. Combinatorica 25 (2005), 143–186 Chudnovsky, M., Robertson, N., Seymour, P., and Thomas, R. [2006]: The strong perfect graph theorem. Annals of Mathematics 164 (2006), 51–229 Chv´atal, V. [1975]: On certain polytopes associated with graphs. Journal of Combinatorial Theory B 18 (1975), 138–154 Chv´atal, V. [1979]: A greedy heuristic for the set cover problem. Mathematics of Operations Research 4 (1979), 233–235 Clementi, A.E.F., and Trevisan, L. [1999]: Improved non-approximability results for minimum vertex cover with density constraints. Theoretical Computer Science 225 (1999), 113–128 Deza, M.M., and Laurent, M. [1997]: Geometry of Cuts and Metrics. Springer, Berlin 1997 Dinur, I. [2007]: The PCP theorem by gap amplification. Journal of the ACM 54 (2007), Article 12 Dinur, I., and Safra, S. [2002]: On the hardness of approximating minimum vertex cover. Annals of Mathematics 162 (2005), 439–485 Erd˝os, P. [1967]: On bipartite subgraphs of graphs. Mat. Lapok. 18 (1967), 283–288 Feige, U. [1998]: A threshold of ln n for the approximating set cover. Journal of the ACM 45 (1998), 634–652 Feige, U. [2004]: Approximating maximum clique by removing subgraphs. SIAM Journal on Discrete Mathematics 18 (2004), 219–225 Feige, U., and Goemans, M.X. [1995]: Approximating the value of two prover proof systems, with applications to MAX 2SAT and MAX DICUT. Proceedings of the 3rd Israel Symposium on Theory of Computing and Systems (1995), 182–189 Feige, U., Goldwasser, S., Lov´asz, L., Safra, S., and Szegedy, M. [1996]: Interactive proofs and the hardness of approximating cliques. Journal of the ACM 43 (1996), 268–292 Fern´andez-Baca, D., and Lagergren, J. [1998]: On the approximability of the Steiner tree problem in phylogeny. Discrete Applied Mathematics 88 (1998), 129–145 Fulkerson, D.R. [1972]: Anti-blocking polyhedra. Journal of Combinatorial Theory B 12 (1972), 50–71 F¨urer, M., and Raghavachari, B. [1994]: Approximating the minimum-degree Steiner tree to within one of optimal. Journal of Algorithms 17 (1994), 409–423 Garey, M.R., and Johnson, D.S. [1976]: The complexity of near-optimal graph coloring. Journal of the ACM 23 (1976), 43–49

436

16 Approximation Algorithms

Garey, M.R., Johnson, D.S., and Stockmeyer, L. [1976]: Some simplified NP-complete graph problems. Theoretical Computer Science 1 (1976), 237–267 Goemans, M.X., and Williamson, D.P. [1994]: New 3/4-approximation algorithms for the maximum satisfiability problem. SIAM Journal on Discrete Mathematics 7 (1994), 656–666 Goemans, M.X., and Williamson, D.P. [1995]: Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming. Journal of the ACM 42 (1995), 1115–1145 Gr¨otschel, M., Lov´asz, L., and Schrijver, A. [1988]: Geometric Algorithms and Combinatorial Optimization. Springer, Berlin 1988 Halld´orsson, M.M., and Radhakrishnan, J. [1997]: Greed is good: approximating independent sets in sparse and bounded degree graphs. Algorithmica 18 (1997), 145–163 H˚astad, J. [2001]: Some optimal inapproximability results. Journal of the ACM 48 (2001), 798–859 Heawood, P.J. [1890]: Map colour theorem. Quarterly Journal of Pure Mathematics 24 (1890), 332–338 Hochbaum, D.S. [1982]: Approximation algorithms for the set covering and vertex cover problems. SIAM Journal on Computing 11 (1982), 555–556 Hochbaum, D.S., and Shmoys, D.B. [1985]: A best possible heuristic for the k-center problem. Mathematics of Operations Research 10 (1985), 180–184 Holyer, I. [1981]: The NP-completeness of edge-coloring. SIAM Journal on Computing 10 (1981), 718–720 Hougardy, S., Pr¨omel, H.J., and Steger, A. [1994]: Probabilistically checkable proofs and their consequences for approximation algorithms. Discrete Mathematics 136 (1994), 175–223 Hsu, W.L., and Nemhauser, G.L. [1979]: Easy and hard bottleneck location problems. Discrete Applied Mathematics 1 (1979), 209–216 Johnson, D.S. [1974]: Approximation algorithms for combinatorial problems. Journal of Computer and System Sciences 9 (1974), 256–278 Khanna, S., Linial, N., and Safra, S. [2000]: On the hardness of approximating the chromatic number. Combinatorica 20 (2000), 393–415 Knuth, D.E. [1969]: The Art of Computer Programming; Vol. 2. Seminumerical Algorithms. Addison-Wesley, Reading 1969 (third edition: 1997) ¨ K¨onig, D. [1916]: Uber Graphen und ihre Anwendung auf Determinantentheorie und Mengenlehre. Mathematische Annalen 77 (1916), 453–465 Lieberherr, K., and Specker, E. [1981]: Complexity of partial satisfaction. Journal of the ACM 28 (1981), 411–421 Lov´asz, L. [1972]: Normal hypergraphs and the perfect graph conjecture. Discrete Mathematics 2 (1972), 253–267 Lov´asz, L. [1975]: On the ratio of optimal integral and fractional covers. Discrete Mathematics 13 (1975), 383–390 Lov´asz, L. [1979a]: On the Shannon capacity of a graph. IEEE Transactions on Information Theory 25 (1979), 1–7 Lov´asz, L. [1979b]: Graph theory and integer programming. In: Discrete Optimization I; Annals of Discrete Mathematics 4 (P.L. Hammer, E.L. Johnson, B.H. Korte, eds.), North-Holland, Amsterdam 1979, pp. 141–158 Lov´asz, L. [2003]: Semidefinite programs and combinatorial optimization. In: Recent Advances in Algorithms and Combinatorics (B.A. Reed, C.L. Sales, eds.), Springer, New York (2003), pp. 137–194

References

437

Mahajan, S., and Ramesh, H. [1999]: Derandomizing approximation algorithms based on semidefinite programming. SIAM Journal on Computing 28 (1999), 1641–1663 Papadimitriou, C.H., and Steiglitz, K. [1982]: Combinatorial Optimization; Algorithms and Complexity. Prentice-Hall, Englewood Cliffs 1982, pp. 406–408 Papadimitriou, C.H., and Yannakakis, M. [1991]: Optimization, approximation, and complexity classes. Journal of Computer and System Sciences 43 (1991), 425–440 Papadimitriou, C.H., and Yannakakis, M. [1993]: The traveling salesman problem with distances one and two. Mathematics of Operations Research 18 (1993), 1–12 Raghavan, P., and Thompson, C.D. [1987]: Randomized rounding: a technique for provably good algorithms and algorithmic proofs. Combinatorica 7 (1987), 365–374 Raz, R., and Safra, S. [1997]: A sub constant error probability low degree test, and a sub constant error probability PCP characterization of NP. Proceedings of the 29th Annual ACM Symposium on Theory of Computing (1997), 475–484 Robertson, N., Sanders, D.P., Seymour, P., and Thomas, R. [1996]: Efficiently four-coloring planar graphs. Proceedings of the 28th Annual ACM Symposium on the Theory of Computing (1996), 571–575 Robertson, N., Sanders, D.P., Seymour, P., and Thomas, R. [1997]: The four colour theorem. Journal of Combinatorial Theory B 70 (1997), 2–44 Sanders, P., and Steurer, D. [2005]: An asymptotic approximation scheme for multigraph edge coloring. Proceedings of the 16th Annual ACM-SIAM Symposium on Discrete Algorithms (2005), 897–906 Singh, M. and Lau, L.C. [2007]: Approximating minimum bounded degree spanning trees to within one of optimal. Proceedings of the 39th Annual ACM Symposium on Theory of Computing (2007), 661–670 Slav´ık, P. [1997]: A tight analysis of the greedy algorithm for set cover. Journal of Algorithms 25 (1997), 237–254 Stockmeyer, L.J. [1973]: Planar 3-colorability is polynomial complete. ACM SIGACT News 5 (1973), 19–25 Trevisan, L. [2004]: On local versus global satisfiability. SIAM Journal on Discrete Mathematics 17 (2004), 541–547 Vizing, V.G. [1964]: On an estimate of the chromatic class of a p-graph. Diskret. Analiz 3 (1964), 23–30 [in Russian] Wigderson, A. [1983]: Improving the performance guarantee for approximate graph coloring. Journal of the ACM 30 (1983), 729–735 Yannakakis, M. [1994]: On the approximation of maximum satisfiability. Journal of Algorithms 17 (1994), 475–502 Zuckerman, D. [2006]: Linear degree extractors and the inapproximability of Max Clique and Chromatic Number. Proceedings of the 38th Annual ACM Symposium on Theory of Computing (2006), 681–690

17 The Knapsack Problem

The M INIMUM W EIGHT P ERFECT M ATCHING P ROBLEM and the W EIGHTED M ATROID I NTERSECTION P ROBLEM discussed in earlier chapters are among the “hardest” problems for which a polynomial-time algorithm is known. In this chapter we deal with the following problem which turns out to be, in a sense, the “easiest” NP-hard problem:

K NAPSACK P ROBLEM Instance: Task:

Nonnegative integers n, c1 , . . . , cn , w1 , . . . , wn and W . P P Find a subset S ⊆ {1, . . . , n} such that j∈S w j ≤ W and j∈S c j is maximum.

Applications arise whenever we want to select an optimum subset of bounded weight from a set of elements each of which has a weight and a profit. We start by considering the fractional version in Section 17.1, which turns out to be solvable in linear time. The integral knapsack problem is NP-hard as shown in Section 17.2, but a pseudopolynomial algorithm solves it optimally. Combined with a rounding technique this can be used to design a fully polynomial approximation scheme, which is the subject of Section 17.3.

17.1 Fractional Knapsack and Weighted Median Problem We consider the following problem:

F RACTIONAL K NAPSACK P ROBLEM Instance: Task:

Nonnegative integers n, c1 , . . . , cn , w1 , . . . , wn and W . P Find numbers x1 , . . . , xn ∈ [0, 1] such that nj=1 x j w j ≤ W and Pn j=1 x j c j is maximum.

The following observation suggests a simple algorithm which requires sorting the elements appropriately: Proposition 17.1. (Dantzig Pn [1957]) Let c1 , . . . , cn , w1 , . . . , wn and W be nonwi > W and negative integers with i=1

440

17 The Knapsack Problem

c2 cn c1 ≥ ≥ ··· ≥ , w1 w2 wn and let k := min

j ∈ {1, . . . , n} :

j X

wi > W

i=1

.

Then an optimum solution of the given instance of the F RACTIONAL K NAPSACK P ROBLEM is defined by xj

xk := xj

for j = 1, . . . , k − 1,

:= 1 W−

Pk−1 j=1

wj

wk

:= 0

, for j = k + 1, . . . , n.

Sorting the elements takes O(n log n) time (Theorem 1.5), and computing k can be done in O(n) time by simple linear scanning. Although this algorithm is quite fast, one can do even better. Observe that the problem reduces to a weighted median search: Definition P 17.2. Let n ∈ N, z 1 , . . . , z n ∈ R, w1 , . . . , wn ∈ R+ and W ∈ R with n 0 < W ≤ i=1 wi . Then the (w1 , . . . , wn ; W )-weighted median with respect to (z 1 , . . . , z n ) is defined to be the unique number z ∗ for which X X wi < W ≤ wi . i:z i z m for i = l + 1, . . . , n.

4

If

k X i=1

If

l X

wi < W ≤

l X

wi then stop (z ∗ := z m ).

i=1 l X wi < W then find recursively the wl+1 , . . . , wn ; W − wi -

i=1

i=1

weighted median with respect to (zl+1 , . . . , z n ). Stop. k X wi ≥ W then find recursively the w1 , . . . , wk ; W -weighted If i=1

median with respect to (z 1 , . . . , z k ). Stop. Theorem 17.3. The W EIGHTED M EDIAN A LGORITHM works correctly and takes O(n) time only. Proof: The correctness is easily checked. Let us denote the worst-case running time for n elements by f (n). We obtain l m l n m 1 n 1 ln m f (n) = O(n) + f + O(n) + f 5+ 2 , 5 2 5 2 5 because the recursive call in 4 omits at least three elements out of at least half of the five-element blocks. The above recursion formula yields as f (n) = O(n): n 9 9 7 9 5 ≤ 41 n for all n ≥ 37, one obtains f (n) ≤ cn + f 41 n + f 2 41 n for a suitable c and n ≥ 37. Given this, f (n) ≤ (82c + f (36))n can be verified easily by induction. So indeed the overall running time is linear. We immediately obtain the following corollaries: Corollary 17.4. (Blum et al. [1973]) The S ELECTION P ROBLEM can be solved in O(n) time. Proof: Set wi := 1 for i = 1, . . . , n and W := k and apply Theorem 17.3.

Corollary 17.5. The F RACTIONAL K NAPSACK P ROBLEM can be solved in linear time.

442

17 The Knapsack Problem

Proof: Setting z i := − wcii (i = 1, . . . , n) reduces the F RACTIONAL K NAPSACK P ROBLEM to the W

Editorial Board R. L. Graham, La Jolla B. Korte, Bonn L. Lovász, Budapest A. Wigderson, Princeton G. M. Ziegler, Berlin

Bernhard Korte · Jens Vygen

Combinatorial Optimization Theory and Algorithms

Fourth Edition

123

Bernhard Korte Jens Vygen Research Institute for Discrete Mathematics University of Bonn Lennéstraße 2 53113 Bonn Germany [email protected] [email protected]

ISBN 978-3-540-71843-7

e-ISBN 978-3-540-71844-4

DOI 10.1007/978-3-540-71844-4 Algorithms and Combinatorics ISSN 0937-5511 Library of Congress Control Number: 2007936375 Mathematics Subject Classification (2000): 90C27, 68R10, 05C85, 68Q25 © 2008 Springer-Verlag Berlin Heidelberg This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilm or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer. Violations are liable to prosecution under the German Copyright Law. The use of general descriptive names, registered names, trademarks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. Typesetting: by the authors using a Springer TEX macro package Production: LE-TEX Jelonek, Schmidt & Vöckler GbR, Leipzig Cover design: WMX Design GmbH, Heidelberg Printed on acid-free paper 987654321 springer.com

Preface to the Fourth Edition

With four English editions, and translations into four other languages forthcoming, we are very happy with the development of our book. Again, we have revised, updated, and significantly extended it for this fourth edition. We have added some classical material that may have been missed so far, in particular on linear programming, the network simplex algorithm, and the max-cut problem. We have also added a number of new exercises and up-to-date references. We hope that these changes serve to make our book an even better basis for teaching and research. We gratefully acknowledge the continuous support of the Union of the German Academies of Sciences and Humanities and the NRW Academy of Sciences via the long-term research project “Discrete Mathematics and Its Applications”. We also thank those who gave us feedback on the third edition, in particular Takao Asano, Christoph Bartoschek, Bert Besser, Ulrich Brenner, Jean Fonlupt, Satoru Fujishige, Marek Karpinski, Jens Maßberg, Denis Naddef, Sven Peyer, Klaus Radke, Rabe von Randow, Dieter Rautenbach, Martin Skutella, Markus Struzyna, J¨urgen Werber, Minyi Yue, and Guochuan Zhang, for their valuable comments. At http://www.or.uni-bonn.de/∼vygen/co.html we will continue to maintain updated information about this book. Bonn, August 2007

Bernhard Korte and Jens Vygen

Preface to the Third Edition

After five years it was time for a thoroughly revised and substantially extended edition. The most significant feature is a completely new chapter on facility location. No constant-factor approximation algorithms were known for this important class of NP-hard problems until eight years ago. Today there are several interesting and very different techniques that lead to good approximation guarantees, which makes this area particularly appealing, also for teaching. In fact, the chapter has arisen from a special course on facility location. Many of the other chapters have also been extended significantly. The new material includes Fibonacci heaps, Fujishige’s new maximum flow algorithm, flows over time, Schrijver’s algorithm for submodular function minimization, and the Robins-Zelikovsky Steiner tree approximation algorithm. Several proofs have been streamlined, and many new exercises and references have been added. We thank those who gave us feedback on the second edition, in particular Takao Asano, Yasuhito Asano, Ulrich Brenner, Stephan Held, Tomio Hirata, Dirk M¨uller, Kazuo Murota, Dieter Rautenbach, Martin Skutella, Markus Struzyna and J¨urgen Werber, for their valuable comments. Eminently, Takao Asano’s notes and J¨urgen Werber’s proofreading of Chapter 22 helped to improve the presentation at various places. Again we would like to mention the Union of the German Academies of Sciences and Humanities and the Northrhine-Westphalian Academy of Sciences. Their continuous support via the long-term project “Discrete Mathematics and Its Applications” funded by the German Ministry of Education and Research and the State of Northrhine-Westphalia is gratefully acknowledged. Bonn, May 2005

Bernhard Korte and Jens Vygen

Preface to the Second Edition

It was more than a surprise to us that the first edition of this book already went out of print about a year after its first appearance. We were flattered by the many positive and even enthusiastic comments and letters from colleagues and the general readership. Several of our colleagues helped us in finding typographical and other errors. In particular, we thank Ulrich Brenner, Andr´as Frank, Bernd G¨artner and Rolf M¨ohring. Of course, all errors detected so far have been corrected in this second edition, and references have been updated. Moreover, the first preface had a flaw. We listed all individuals who helped us in preparing this book. But we forgot to mention the institutional support, for which we make amends here. It is evident that a book project which took seven years benefited from many different grants. We would like to mention explicitly the bilateral HungarianGerman Research Project, sponsored by the Hungarian Academy of Sciences and the Deutsche Forschungsgemeinschaft, two Sonderforschungsbereiche (special research units) of the Deutsche Forschungsgemeinschaft, the Minist`ere Franc¸ais de la Recherche et de la Technologie and the Alexander von Humboldt Foundation for support via the Prix Alexandre de Humboldt, and the Commission of the European Communities for participation in two projects DONET. Our most sincere thanks go to the Union of the German Academies of Sciences and Humanities and to the Northrhine-Westphalian Academy of Sciences. Their long-term project “Discrete Mathematics and Its Applications” supported by the German Ministry of Education and Research (BMBF) and the State of Northrhine-Westphalia was of decisive importance for this book. Bonn, October 2001

Bernhard Korte and Jens Vygen

Preface to the First Edition

Combinatorial optimization is one of the youngest and most active areas of discrete mathematics, and is probably its driving force today. It became a subject in its own right about 50 years ago. This book describes the most important ideas, theoretical results, and algorithms in combinatorial optimization. We have conceived it as an advanced graduate text which can also be used as an up-to-date reference work for current research. The book includes the essential fundamentals of graph theory, linear and integer programming, and complexity theory. It covers classical topics in combinatorial optimization as well as very recent ones. The emphasis is on theoretical results and algorithms with provably good performance. Applications and heuristics are mentioned only occasionally. Combinatorial optimization has its roots in combinatorics, operations research, and theoretical computer science. A main motivation is that thousands of real-life problems can be formulated as abstract combinatorial optimization problems. We focus on the detailed study of classical problems which occur in many different contexts, together with the underlying theory. Most combinatorial optimization problems can be formulated naturally in terms of graphs and as (integer) linear programs. Therefore this book starts, after an introduction, by reviewing basic graph theory and proving those results in linear and integer programming which are most relevant for combinatorial optimization. Next, the classical topics in combinatorial optimization are studied: minimum spanning trees, shortest paths, network flows, matchings and matroids. Most of the problems discussed in Chapters 6–14 have polynomial-time (“efficient”) algorithms, while most of the problems studied in Chapters 15–21 are NP-hard, i.e. a polynomial-time algorithm is unlikely to exist. In many cases one can at least find approximation algorithms that have a certain performance guarantee. We also mention some other strategies for coping with such “hard” problems. This book goes beyond the scope of a normal textbook on combinatorial optimization in various aspects. For example we cover the equivalence of optimization and separation (for full-dimensional polytopes), O(n 3 )-implementations of matching algorithms based on ear-decompositions, Turing machines, the Perfect Graph Theorem, MAXSNP-hardness, the Karmarkar-Karp algorithm for bin packing, recent approximation algorithms for multicommodity flows, survivable network design

XII

Preface to the First Edition

and the Euclidean traveling salesman problem. All results are accompanied by detailed proofs. Of course, no book on combinatorial optimization can be absolutely comprehensive. Examples of topics which we mention only briefly or do not cover at all are tree-decompositions, separators, submodular flows, path-matchings, deltamatroids, the matroid parity problem, location and scheduling problems, nonlinear problems, semidefinite programming, average-case analysis of algorithms, advanced data structures, parallel and randomized algorithms, and the theory of probabilistically checkable proofs (we cite the PCP Theorem without proof). At the end of each chapter there are a number of exercises containing additional results and applications of the material in that chapter. Some exercises which might be more difficult are marked with an asterisk. Each chapter ends with a list of references, including texts recommended for further reading. This book arose from several courses on combinatorial optimization and from special classes on topics like polyhedral combinatorics or approximation algorithms. Thus, material for basic and advanced courses can be selected from this book. We have benefited from discussions and suggestions of many colleagues and friends and – of course – from other texts on this subject. Especially we owe sincere thanks to Andr´as Frank, L´aszl´o Lov´asz, Andr´as Recski, Alexander Schrijver and Zolt´an Szigeti. Our colleagues and students in Bonn, Christoph Albrecht, Ursula B¨unnagel, Thomas Emden-Weinert, Mathias Hauptmann, Sven Peyer, Rabe von Randow, Andr´e Rohe, Martin Thimm and J¨urgen Werber, have carefully read several versions of the manuscript and helped to improve it. Last, but not least we thank Springer Verlag for the most efficient cooperation. Bonn, January 2000

Bernhard Korte and Jens Vygen

Table of Contents

1

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.1 Enumeration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.2 Running Time of Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 1.3 Linear Optimization Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 1.4 Sorting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

2

Graphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1 Basic Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Trees, Circuits, and Cuts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3 Connectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4 Eulerian and Bipartite Graphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.5 Planarity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.6 Planar Duality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

13 13 17 24 30 33 40 43 46

3

Linear Programming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1 Polyhedra . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 The Simplex Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 Implementation of the Simplex Algorithm . . . . . . . . . . . . . . . . . . . . 3.4 Duality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5 Convex Hulls and Polytopes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

49 50 54 57 60 64 66 68

4

Linear Programming Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1 Size of Vertices and Faces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Continued Fractions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3 Gaussian Elimination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4 The Ellipsoid Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.5 Khachiyan’s Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.6 Separation and Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

71 71 74 77 80 86 88 95 96

XIV

Table of Contents

5

Integer Programming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 5.1 The Integer Hull of a Polyhedron . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 5.2 Unimodular Transformations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 5.3 Total Dual Integrality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 5.4 Totally Unimodular Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 5.5 Cutting Planes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 5.6 Lagrangean Relaxation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125

6

Spanning Trees and Arborescences . . . . . . . . . . . . . . . . . . . . . . . . . . . 127 6.1 Minimum Spanning Trees . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127 6.2 Minimum Weight Arborescences . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133 6.3 Polyhedral Descriptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137 6.4 Packing Spanning Trees and Arborescences . . . . . . . . . . . . . . . . . . . 140 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147

7

Shortest Paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151 7.1 Shortest Paths From One Source . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152 7.2 Shortest Paths Between All Pairs of Vertices . . . . . . . . . . . . . . . . . . 156 7.3 Minimum Mean Cycles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163

8

Network Flows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165 8.1 Max-Flow-Min-Cut Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166 8.2 Menger’s Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170 8.3 The Edmonds-Karp Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172 8.4 Blocking Flows and Fujishige’s Algorithm . . . . . . . . . . . . . . . . . . . . 174 8.5 The Goldberg-Tarjan Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176 8.6 Gomory-Hu Trees . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180 8.7 The Minimum Capacity of a Cut in an Undirected Graph . . . . . . . 186 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194

9

Minimum Cost Flows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199 9.1 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199 9.2 An Optimality Criterion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201 9.3 Minimum Mean Cycle-Cancelling Algorithm . . . . . . . . . . . . . . . . . . 203 9.4 Successive Shortest Path Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . 207 9.5 Orlin’s Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211 9.6 The Network Simplex Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214 9.7 Flows Over Time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224

Table of Contents

XV

10

Maximum Matchings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227 10.1 Bipartite Matching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228 10.2 The Tutte Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230 10.3 Tutte’s Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232 10.4 Ear-Decompositions of Factor-Critical Graphs . . . . . . . . . . . . . . . . . 235 10.5 Edmonds’ Matching Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254

11

Weighted Matching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257 11.1 The Assignment Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258 11.2 Outline of the Weighted Matching Algorithm . . . . . . . . . . . . . . . . . 259 11.3 Implementation of the Weighted Matching Algorithm . . . . . . . . . . 262 11.4 Postoptimality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276 11.5 The Matching Polytope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 280 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282

12

b-Matchings and T -Joins . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285 12.1 b-Matchings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285 12.2 Minimum Weight T -Joins . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289 12.3 T -Joins and T -Cuts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293 12.4 The Padberg-Rao Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 296 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302

13

Matroids . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305 13.1 Independence Systems and Matroids . . . . . . . . . . . . . . . . . . . . . . . . . 305 13.2 Other Matroid Axioms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309 13.3 Duality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313 13.4 The Greedy Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 317 13.5 Matroid Intersection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 322 13.6 Matroid Partitioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 327 13.7 Weighted Matroid Intersection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 329 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 332 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 334

14

Generalizations of Matroids . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 337 14.1 Greedoids . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 337 14.2 Polymatroids . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 341 14.3 Minimizing Submodular Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . 345 14.4 Schrijver’s Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 347 14.5 Symmetric Submodular Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . 351 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 355

XVI

Table of Contents

15

NP-Completeness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 359 15.1 Turing Machines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 359 15.2 Church’s Thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 362 15.3 P and NP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 367 15.4 Cook’s Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 371 15.5 Some Basic NP-Complete Problems . . . . . . . . . . . . . . . . . . . . . . . . . . 375 15.6 The Class coNP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 382 15.7 NP-Hard Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 387 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 391

16

Approximation Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393 16.1 Set Covering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 394 16.2 The Max-Cut Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 399 16.3 Colouring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 405 16.4 Approximation Schemes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 413 16.5 Maximum Satisfiability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 415 16.6 The PCP Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 420 16.7 L-Reductions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 424 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 430 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 434

17

The Knapsack Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 439 17.1 Fractional Knapsack and Weighted Median Problem . . . . . . . . . . . 439 17.2 A Pseudopolynomial Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 442 17.3 A Fully Polynomial Approximation Scheme . . . . . . . . . . . . . . . . . . 444 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 447 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 447

18

Bin-Packing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 449 18.1 Greedy Heuristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 449 18.2 An Asymptotic Approximation Scheme . . . . . . . . . . . . . . . . . . . . . . 455 18.3 The Karmarkar-Karp Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 459 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 462 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 464

19

Multicommodity Flows and Edge-Disjoint Paths . . . . . . . . . . . . . . . . 467 19.1 Multicommodity Flows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 468 19.2 Algorithms for Multicommodity Flows . . . . . . . . . . . . . . . . . . . . . . . 471 19.3 Directed Edge-Disjoint Paths Problem . . . . . . . . . . . . . . . . . . . . . . . . 476 19.4 Undirected Edge-Disjoint Paths Problem . . . . . . . . . . . . . . . . . . . . . . 479 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 485 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 487

Table of Contents

XVII

20

Network Design Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 491 20.1 Steiner Trees . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 492 20.2 The Robins-Zelikovsky Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 497 20.3 Survivable Network Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 502 20.4 A Primal-Dual Approximation Algorithm . . . . . . . . . . . . . . . . . . . . . 505 20.5 Jain’s Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 514 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 520 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 522

21

The Traveling Salesman Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 527 21.1 Approximation Algorithms for the TSP . . . . . . . . . . . . . . . . . . . . . . 527 21.2 Euclidean TSP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 532 21.3 Local Search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 539 21.4 The Traveling Salesman Polytope . . . . . . . . . . . . . . . . . . . . . . . . . . . . 545 21.5 Lower Bounds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 551 21.6 Branch-and-Bound . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 553 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 556 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 559

22

Facility Location . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 563 22.1 The Uncapacitated Facility Location Problem . . . . . . . . . . . . . . . . . 563 22.2 Rounding Linear Programming Solutions . . . . . . . . . . . . . . . . . . . . . 565 22.3 Primal-Dual Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 567 22.4 Scaling and Greedy Augmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 573 22.5 Bounding the Number of Facilities . . . . . . . . . . . . . . . . . . . . . . . . . . . 576 22.6 Local Search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 579 22.7 Capacitated Facility Location Problems . . . . . . . . . . . . . . . . . . . . . . . 585 22.8 Universal Facility Location . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 588 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 594 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 596

Notation Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 599 Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 603 Subject Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 613

1 Introduction

Let us start with two examples. A company has a machine which drills holes into printed circuit boards. Since it produces many of these boards it wants the machine to complete one board as fast as possible. We cannot optimize the drilling time but we can try to minimize the time the machine needs to move from one point to another. Usually drilling machines can move in two directions: the table moves horizontally while the drilling arm moves vertically. Since both movements can be done simultaneously, the time needed to adjust the machine from one position to another is proportional to the maximum of the horizontal and the vertical distance. This is often called the `∞ distance. (Older machines can only move either horizontally or vertically at a time; in this case the adjusting time is proportional to the `1 -distance, the sum of the horizontal and the vertical distance.) An optimum drilling Pn−1path is given by an ordering of the hole positions p1 , . . . , pn such that i=1 d( pi , pi+1 ) is minimum, where d is the L ∞ -distance: for two points p = (x, y) and p 0 = (x 0 , y 0 ) in the plane we write d( p, p 0 ) := max{|x − x 0 |, |y − y 0 |}. An order of the holes can be represented by a permutation, i.e. a bijection π : {1, . . . , n} → {1, . . . , n}. Which permutation is best of course depends on the hole positions; for each list of hole positions we have a different problem instance. We say that one instance of our problem is a list of points in the plane, i.e. the coordinates of the holes to be drilled. Then the problem can be stated formally as follows:

D RILLING P ROBLEM Instance:

A set of points p1 , . . . , pn ∈ R2 .

Task:

Find a permutation π : {1, . . . , n} → {1, . . . , n} such that P n−1 i=1 d( pπ(i) , pπ(i+1) ) is minimum.

We now explain our second example. We have a set of jobs to be done, each having a specified processing time. Each job can be done by a subset of the employees, and we assume that all employees who can do a job are equally efficient. Several employees can contribute to the same job at the same time, and one employee can contribute to several jobs (but not at the same time). The objective is to get all jobs done as early as possible.

2

1 Introduction

In this model it suffices to prescribe for each employee how long he or she should work on which job. The order in which the employees carry out their jobs is not important, since the time when all jobs are done obviously depends only on the maximum total working time we have assigned to one employee. Hence we have to solve the following problem:

J OB A SSIGNMENT P ROBLEM Instance:

Task:

A set of numbers t1 , . . . , tn ∈ R+ (the processing times for n jobs), a number m ∈ N of employees, and a nonempty subset Si ⊆ {1, . . . , m} of employees for each job i ∈ {1, . . . , n}. Find P numbers xi j ∈ R+ for all i = 1, . . . , n and j ∈PSi such that j∈Si x i j = ti for i = 1, . . . , n and max j∈{1,...,m} i: j∈Si x i j is minimum.

These are two typical problems arising in combinatorial optimization. How to model a practical problem as an abstract combinatorial optimization problem is not described in this book; indeed there is no general recipe for this task. Besides giving a precise formulation of the input and the desired output it is often important to ignore irrelevant components (e.g. the drilling time which cannot be optimized or the order in which the employees carry out their jobs). Of course we are not interested in a solution to a particular drilling problem or job assignment problem in some company, but rather we are looking for a way how to solve all problems of these types. We first consider the D RILLING P ROBLEM.

1.1 Enumeration How can a solution to the D RILLING P ROBLEM look like? There are infinitely many instances (finite sets of points in the plane), so we cannot list an optimum permutation for each instance. Instead, what we look for is an algorithm which, given an instance, computes an optimum solution. Such an algorithm exists: Given a set of n points, just try all possible n! orders, and for each compute the L ∞ -length of the corresponding path. There are different ways of formulating an algorithm, differing mostly in the level of detail and the formal language they use. We certainly would not accept the following as an algorithm: “Given a set of n points, find an optimum path and output it.” It is not specified at all how to find the optimum solution. The above suggestion to enumerate all possible n! orders is more useful, but still it is not clear how to enumerate all the orders. Here is one possible way: We enumerate all n-tuples of numbers 1, . . . , n, i.e. all n n vectors of {1, . . . , n n} . This can be done similarly to counting: we start with (1, . . . , 1, 1), (1, . . . , 1, 2) up to (1, . . . , 1, n) then switch to (1, . . . , 1, 2, 1), and so on. At each step we increment the last entry unless it is already n, in which case we go back to the last entry that is smaller than n, increment it and set all subsequent entries to 1.

1.1 Enumeration

3

This technique is sometimes called backtracking. The order in which the vectors of {1, . . . , n}n are enumerated is called the lexicographical order: Definition 1.1. Let x, y ∈ Rn be two vectors. We say that a vector x is lexicographically smaller than y if there exists an index j ∈ {1, . . . , n} such that xi = yi for i = 1, . . . , j − 1 and x j < y j . Knowing how to enumerate all vectors of {1, . . . , n}n we can simply check for each vector whether its entries are pairwise distinct and, if so, whether the path represented by this vector is shorter than the best path encountered so far. Since this algorithm enumerates n n vectors it will take at least n n steps (in fact, even more). This is not best possible. There are only n! permutations of {1,√. . . , n}, n and n! is significantly smaller than n n . (By Stirling’s formula n! ≈ 2π n nen (Stirling [1730]); see Exercise 1.) We shall show how to enumerate all paths in approximately n 2 · n! steps. Consider the following algorithm which enumerates all permutations in lexicographical order:

PATH E NUMERATION A LGORITHM Input:

A natural number n ≥ 3. A set { p1 , . . . , pn } of points in the plane.

Output:

∗ A permutation Pπn−1: {1, . . . , n} → {1, . . . , n} with ∗ cost (π ) := i=1 d( pπ ∗ (i) , pπ ∗ (i+1) ) minimum.

1

Set π(i) := i and π ∗ (i) := i for i = 1, . . . , n. Set i := n − 1.

2

Let k := min({π(i) + 1, . . . , n + 1} \ {π(1), . . . , π(i − 1)}).

3

If k ≤ n then: Set π(i) := k. If i = n and cost (π) < cost (π ∗ ) then set π ∗ := π . If i < n then set π(i + 1) := 0 and i := i + 1. If k = n + 1 then set i := i − 1. If i ≥ 1 then go to . 2

Starting with (π(i))i=1,...,n = (1, 2, 3, . . . , n−1, n) and i = n−1, the algorithm finds at each step the next possible value of π(i) (not using π(1), . . . , π(i − 1)). If there is no more possibility for π(i) (i.e. k = n + 1), then the algorithm decrements i (backtracking). Otherwise it sets π(i) to the new value. If i = n, the new permutation is evaluated, otherwise the algorithm will try all possible values for π(i + 1), . . . , π(n) and starts by setting π(i + 1) := 0 and incrementing i. So all permutation vectors (π(1), . . . , π(n)) are generated in lexicographical order. For example, the first iterations in the case n = 6 are shown below:

4

1 Introduction

k k k k k k k

:= 6, := 5, := 7, := 7, := 5, := 4, := 6,

π := (1, 2, 3, 4, 5, 6), π := (1, 2, 3, 4, 6, 0), π := (1, 2, 3, 4, 6, 5), π := (1, 2, 3, 5, 0, 5), π := (1, 2, 3, 5, 4, 0), π := (1, 2, 3, 5, 4, 6),

i := 5 i := 6 cost (π ) < cost (π ∗ )? i i i i

:= 5 := 4 := 5 := 6 cost (π) < cost (π ∗ )?

Since the algorithm compares the cost of each path to π ∗ , the best path encountered so far, it indeed outputs the optimum path. But how many steps will this algorithm perform? Of course, the answer depends on what we call a single step. Since we do not want the number of steps to depend on the actual implementation we ignore constant factors. In any reasonable computer, 1 will take at least 2n + 1 steps (this many variable assignments are done) and at most cn steps for some constant c. The following common notation is useful for ignoring constant factors: Definition 1.2. Let f, g : D → R+ be two functions. We say that f is O(g) (and sometimes write f = O(g)) if there exist constants α, β > 0 such that f (x) ≤ αg(x) + β for all x ∈ D. If f = O(g) and g = O( f ) we also say that f = 2(g) (and of course g = 2( f )). In this case, f and g have the same rate of growth. Note that the use of the equation sign in the O-notation is not symmetric. To illustrate this definition, let D = N, and let f (n) be the number of elementary steps in 1 and g(n) = n (n ∈ N). Clearly we have f = O(g) (in fact f = 2(g)) in this case; we say that 1 takes O(n) time (or linear time). A single execution takes a constant number of steps (we speak of O(1) time or constant time) of 3 except in the case k ≤ n and i = n; in this case the cost of two paths have to be compared, which takes O(n) time. What about ? 2 A naive implementation, checking for each j ∈ {π(i) + 1, . . . , n} and each h ∈ {1, . . . , i − 1} whether j = π(h), takes O((n − π(i))i) steps, which can be as big as 2(n 2 ). A better implementation of 2 uses an auxiliary array indexed by 1, . . . , n: For j := 1 to n do aux( j) := 0. For j := 1 to i − 1 do aux(π( j)) := 1. Set k := π(i) + 1. While k ≤ n and aux(k) = 1 do k := k + 1. Obviously with this implementation a single execution of 2 takes only O(n) time. Simple techniques like this are usually not elaborated in this book; we assume that the reader can find such implementations himself. Having computed the running time for each single step we now estimate the total amount of work. Since the number of permutations is n! we only have to estimate the amount of work which is done between two permutations. The counter i might move back from n to some index i 0 where a new value π(i 0 ) ≤ n is found. Then it moves forward again up to i = n. While the counter i is constant each

2

1.2 Running Time of Algorithms

5

of 2 and 3 is performed once, except in the case k ≤ n and i = n; in this case 2 and 3 are performed twice. So the total amount of work between two permutations consists of at most 4n times 2 and , 3 i.e. O(n 2 ). So the overall running time of the PATH E NUMERATION A LGORITHM is O(n 2 n!). One can do slightly better; a more careful analysis shows that the running time is only O(n · n!) (Exercise 4). Still the algorithm is too time-consuming if n is large. The problem with the enumeration of all paths is that the number of paths grows exponentially with the number of points; already for 20 points there are 20! = 2432902008176640000 ≈ 2.4 · 1018 different paths and even the fastest computer needs several years to evaluate all of them. So complete enumeration is impossible even for instances of moderate size. The main subject of combinatorial optimization is to find better algorithms for problems like this. Often one has to find the best element of some finite set of feasible solutions (in our example: drilling paths or permutations). This set is not listed explicitly but implicitly depends on the structure of the problem. Therefore an algorithm must exploit this structure. In the case of the D RILLING P ROBLEM all information of an instance with n points is given by 2n coordinates. While the naive algorithm enumerates all n! paths it might be possible that there is an algorithm which finds the optimum path much faster, say in n 2 computation steps. It is not known whether such an algorithm exists (though results of Chapter 15 suggest that it is unlikely). Nevertheless there are much better algorithms than the naive one.

1.2 Running Time of Algorithms One can give a formal definition of an algorithm, and we shall in fact give one in Section 15.1. However, such formal models lead to very long and tedious descriptions as soon as algorithms are a bit more complicated. This is quite similar to mathematical proofs: Although the concept of a proof can be formalized nobody uses such a formalism for writing down proofs since they would become very long and almost unreadable. Therefore all algorithms in this book are written in an informal language. Still the level of detail should allow a reader with a little experience to implement the algorithms on any computer without too much additional effort. Since we are not interested in constant factors when measuring running times we do not have to fix a concrete computing model. We count elementary steps, but we are not really interested in how elementary steps look like. Examples of elementary steps are variable assignments, random access to a variable whose index is stored in another variable, conditional jumps (if – then – go to), and simple arithmetic operations like addition, subtraction, multiplication, division and comparison of numbers. An algorithm consists of a set of valid inputs and a sequence of instructions each of which can be composed of elementary steps, such that for each valid input

6

1 Introduction

the computation of the algorithm is a uniquely defined finite series of elementary steps which produces a certain output. Usually we are not satisfied with finite computation but rather want a good upper bound on the number of elementary steps performed, depending on the input size. The input to an algorithm usually consists of a list of numbers. If all these numbers are integers, we can code them in binary representation, using O(log(|a|+ 2)) bits for storing an integer a. Rational numbers can be stored by coding the numerator and the denominator separately. The input size size(x) of an instance x with rational data is the total number of bits needed for the binary representation. Definition 1.3. Let A be an algorithm which accepts inputs from a set X , and let f : N → R+ . If there exists a constant α > 0 such that A terminates its computation after at most α f (size(x)) elementary steps (including arithmetic operations) for each input x ∈ X , then we say that A runs in O( f ) time. We also say that the running time (or the time complexity) of A is O( f ). Definition 1.4. An algorithm with rational input is said to run in polynomial time if there is an integer k such that it runs in O(n k ) time, where n is the input size, and all numbers in intermediate computations can be stored with O(n k ) bits. An algorithm with arbitrary input is said to run in strongly polynomial time if there is an integer k such that it runs in O(n k ) time for any input consisting of n numbers and it runs in polynomial time for rational input. In the case k = 1 we have a linear-time algorithm. Note that the running time might be different for several instances of the same size (this was not the case with the PATH E NUMERATION A LGORITHM). We consider the worst-case running time, i.e. the function f : N → N where f (n) is the maximum running time of an instance with input size n. For some algorithms we do not know the rate of growth of f but only have an upper bound. The worst-case running time might be a pessimistic measure if the worst case occurs rarely. In some cases an average-case running time with some probabilistic model might be appropriate, but we shall not consider this. If A is an algorithm which for each input x ∈ X computes the output f (x) ∈ Y , then we say that A computes f : X → Y . If a function is computed by some polynomial-time algorithm, it is said to be computable in polynomial time. Polynomial-time algorithms are sometimes called “good” or “efficient”. This concept was introduced by Cobham [1964] and Edmonds [1965]. Table 1.1 motivates this by showing hypothetical running times of algorithms with various time complexities. For various input sizes n we show the running time of algorithms that take 100n log n, 10n 2 , n 3.5 , n log n , 2n , and n! elementary steps; we assume that one elementary step takes one nanosecond. As always in this book, log denotes the logarithm with basis 2. As Table 1.1 shows, polynomial-time algorithms are faster for large enough instances. The table also illustrates that constant factors of moderate size are not very important when considering the asymptotic growth of the running time.

1.2 Running Time of Algorithms

7

Table 1.1. n 10 20 30 40 50 60 80 100 200 500 1000 104 105 106 107 108 1010 1012

100n log n

10n 2

n 3.5

n log n

2n

n!

3 µs 9 µs 15 µs 21 µs 28 µs 35 µs 50 µs 66 µs 153 µs 448 µs 1 ms 13 ms 166 ms 2s 23 s 266 s 9 hours 46 days

1 µs 4 µs 9 µs 16 µs 25 µs 36 µs 64 µs 100 µs 400 µs 2.5 ms 10 ms 1s 100 s 3 hours 12 days 3 years 3 · 104 y. 3 · 108 y.

3 µs 36 µs 148 µs 404 µs 884 µs 2 ms 5 ms 10 ms 113 ms 3s 32 s 28 hours 10 years 3169 y. 107 y. 3 · 1010 y.

2 µs 420 µs 20 ms 340 ms 4s 32 s 1075 s 5 hours 12 years 5 · 105 y. 3 · 1013 y.

1 µs 1 ms 1s 1100 s 13 days 37 years 4 · 107 y. 4 · 1013 y.

4 ms 76 years 8 · 1015 y.

Table 1.2 shows the maximum input sizes solvable within one hour with the above six hypothetical algorithms. In (a) we again assume that one elementary step takes one nanosecond, (b) shows the corresponding figures for a ten times faster machine. Polynomial-time algorithms can handle larger instances in reasonable time. Moreover, even a speedup by a factor of 10 of the computers does not increase the size of solvable instances significantly for exponential-time algorithms, but it does for polynomial-time algorithms. Table 1.2. 10n 2

n 3.5

(a)

1.19 · 109

60000

(b)

10.8 · 109

189737

100n log n

n log n

2n

n!

3868

87

41

15

7468

104

45

16

(Strongly) polynomial-time algorithms, if possible linear-time algorithms, are what we look for. There are some problems where it is known that no polynomialtime algorithm exists, and there are problems for which no algorithm exists at all. (For example, a problem which can be solved in finite time but not in polynomial

8

1 Introduction

time is to decide whether a so-called regular expression defines the empty set; see Aho, Hopcroft and Ullman [1974]. A problem for which there exists no algorithm at all, the H ALTING P ROBLEM, is discussed in Exercise 1 of Chapter 15.) However, almost all problems considered in this book belong to the following two classes. For the problems of the first class we have a polynomial-time algorithm. For each problem of the second class it is an open question whether a polynomialtime algorithm exists. However, we know that if one of these problems has a polynomial-time algorithm, then all problems of this class do. A precise formulation and a proof of this statement will be given in Chapter 15. The J OB A SSIGNMENT P ROBLEM belongs to the first class, the D RILLING P ROBLEM belongs to the second class. These two classes of problems divide this book roughly into two parts. We first deal with tractable problems for which polynomial-time algorithms are known. Then, starting with Chapter 15, we discuss hard problems. Although no polynomialtime algorithms are known, there are often much better methods than complete enumeration. Moreover, for many problems (including the D RILLING P ROBLEM), one can find approximate solutions within a certain percentage of the optimum in polynomial time.

1.3 Linear Optimization Problems We now consider our second example given initially, the J OB A SSIGNMENT P ROB LEM , and briefly address some central topics which will be discussed in later chapters. The J OB A SSIGNMENT P ROBLEM is quite different to the D RILLING P ROBLEM since there are infinitely many feasible solutions for each instance (except for trivial cases). We can reformulate the problem by introducing a variable T for the time when all jobs are done: min s.t.

T X

xi j

= ti

(i ∈ {1, . . . , n})

≥ 0 ≤ T

(i ∈ {1, . . . , n}, j ∈ Si ) ( j ∈ {1, . . . , m})

j∈Si

X

xi j xi j

(1.1)

i: j∈Si

The numbers ti and the sets Si (i = 1, . . . , n) are given, the variables xi j and T are what we look for. Such an optimization problem with a linear objective function and linear constraints is called a linear program. The set of feasible solutions of (1.1), a so-called polyhedron, is easily seen to be convex, and one can prove that there always exists an optimum solution which is one of the finitely many extreme points of this set. Therefore a linear program can, theoretically, also

1.4 Sorting

9

be solved by complete enumeration. But there are much better ways as we shall see later. Although there are several algorithms for solving linear programs in general, such general techniques are usually less efficient than special algorithms exploiting the structure of the problem. In our case it is convenient to model the sets Si , i = 1, . . . , n, by a graph. For each job i and for each employee j we have a point (called vertex), and we connect employee j with job i by an edge if he or she can contribute to this job (i.e. if j ∈ Si ). Graphs are a fundamental combinatorial structure; many combinatorial optimization problems are described most naturally in terms of graph theory. Suppose for a moment that the processing time of each job is one hour, and we ask whether we can finish all jobs within one hour. So we look Pfor numbers xi j (i ∈ {1, . . . , n}, j P ∈ Si ) such that 0 ≤ xi j ≤ 1 for all i and j, j∈Si xi j = 1 for i = 1, . . . , n, and i: j∈Si xi j ≤ 1 for j = 1, . . . , n. One can show that if such a solution exists, then in fact an integral solution exists, i.e. all xi j are either 0 or 1. This is equivalent to assigning each job to one employee, such that no employee has to do more than one job. In the language of graph theory we then look for a matching covering all jobs. The problem of finding optimal matchings is one of the best-known combinatorial optimization problems. We review the basics of graph theory and linear programming in Chapters 2 and 3. In Chapter 4 we prove that linear programs can be solved in polynomial time, and in Chapter 5 we discuss integral polyhedra. In the subsequent chapters we discuss some classical combinatorial optimization problems in detail.

1.4 Sorting Let us conclude this chapter by considering a special case of the D RILLING P ROB LEM where all holes to be drilled are on one horizontal line. So we are given just one coordinate for each point pi , i = 1, . . . , n. Then a solution to the drilling problem is easy, all we have to do is sort the points by their coordinates: the drill will just move from left to right. Although there are still n! permutations, it is clear that we do not have to consider all of them to find the optimum drilling path, i.e. the sorted list. It is very easy to sort n numbers in nondecreasing order in O(n 2 ) time. To sort n numbers in O(n log n) time requires a little more skill. There are several algorithms accomplishing this; we present the well-known M ERGE -S ORT A LGORITHM. It proceeds as follows. First the list is divided into two sublists of approximately equal size. Then each sublist is sorted (this is done recursively by the same algorithm). Finally the two sorted sublists are merged together. This general strategy, often called “divide and conquer”, can be used quite often. See e.g. Section 17.1 for another example. We did not discuss recursive algorithms so far. In fact, it is not necessary to discuss them, since any recursive algorithm can be transformed into a sequential algorithm without increasing the running time. But some algorithms are easier to

10

1 Introduction

formulate (and implement) using recursion, so we shall use recursion when it is convenient.

M ERGE -S ORT A LGORITHM Input:

A list a1 , . . . , an of real numbers.

Output:

A permutation π : {1, . . . , n} → {1, . . . , n} such that aπ(i) ≤ aπ(i+1) for all i = 1, . . . , n − 1.

1

2

3

If n = 1 then set π(1) := 1 and stop (return π ). Set m := n2 . Let ρ :=M ERGE -S ORT(a1 , . . . , am ). Let σ :=M ERGE -S ORT(am+1 , . . . , an ). Set k := 1, l := 1. While k ≤ m and l ≤ n − m do: If aρ(k) ≤ am+σ (l) then set π(k + l − 1) := ρ(k) and k := k + 1 else set π(k + l − 1) := m + σ (l) and l := l + 1. While k ≤ m do: Set π(k + l − 1) := ρ(k) and k := k + 1. While l ≤ n − m do: Set π(k + l − 1) := m + σ (l) and l := l + 1.

As an example, consider the list “69,32,56,75,43,99,28”. The algorithm first splits this list into two, “69,32,56” and “75,43,99,28” and recursively sorts each of the two sublists. We get the permutations ρ = (2, 3, 1) and σ = (4, 2, 1, 3) corresponding to the sorted lists “32,56,69” and “28,43,75,99”. Now these lists are merged as shown below: k := 1, l := 1 l := 2 ρ(1) = 2, σ (1) = 4, aρ(1) = 32, aσ (1) = 28, π(1) := 7, ρ(1) = 2, σ (2) = 2, aρ(1) = 32, aσ (2) = 43, π(2) := 2, k := 2 ρ(2) = 3, σ (2) = 2, aρ(2) = 56, aσ (2) = 43, π(3) := 5, l := 3 ρ(2) = 3, σ (3) = 1, aρ(2) = 56, aσ (3) = 75, π(4) := 3, k := 3 ρ(3) = 1, σ (3) = 1, aρ(3) = 69, aσ (3) = 75, π(5) := 1, k := 4 l := 4 σ (3) = 1, aσ (3) = 75, π(6) := 4, σ (4) = 3, aσ (4) = 99, π(7) := 6, l := 5 Theorem 1.5. The M ERGE -S ORT A LGORITHM works correctly and runs in O(n log n) time. Proof: The correctness is obvious. We denote by T (n) the running time (number of steps) needed for instances consisting of n numbers and observe that T (1) = 1 and T (n) = T (b n2 c) + T (d n2 e) + 3n + 6. (The constants in the term 3n + 6 depend on how exactly a computation step is defined; but they do not really matter.) We claim that this yields T (n) ≤ 12n log n + 1. Since this is trivial for n = 1 we proceed by induction. For n ≥ 2, assuming that the inequality is true for 1, . . . , n − 1, we get

Exercises

11

ln m 2 2 n + 1 + 12 n + 1 + 3n + 6 log 2 3 2 3 = 12n(log n + 1 − log 3) + 3n + 8 13 ≤ 12n log n − n + 3n + 8 ≤ 12n log n + 1, 2

T (n) ≤ 12

because log 3 ≥

jn k

log

37 24 .

Of course the algorithm works for sorting the elements of any totally ordered set, assuming that we can compare any two elements in constant time. Can there be a faster, a linear-time algorithm? Suppose that the only way we can get information on the unknown order is to compare two elements. Then we can show that any algorithm needs at least 2(n log n) comparisons in the worst case. The outcome of a comparison can be regarded as a zero or one; the outcome of all comparisons an algorithm does is a 0-1-string (a sequence of zeros and ones). Note that two different orders in the input of the algorithm must lead to two different 0-1-strings (otherwise the algorithm could not distinguish between the two orders). For an input of n elements there are n! possible orders, so there must be n! different 01-strings corresponding to the computation. Since the number of 0-1-strings with n n n n n length less than n2 log n2 is 2b 2 log 2 c − 1 < 2 2 log 2 = ( n2 ) 2 ≤ n! we conclude that the maximum length of the 0-1-strings, and hence of the computation, must be at least n2 log n2 = 2(n log n). In the above sense, the running time of the M ERGE -S ORT A LGORITHM is optimal up to a constant factor. However, there is an algorithm for sorting integers (or sorting strings lexicographically) whose running time is linear in the input size; see Exercise 7. An algorithm to sort n integers in O(n log log n) time was proposed by Han [2004]. Lower bounds like the one above are known only for very few problems (except trivial linear bounds). Often a restriction on the set of operations is necessary to derive a superlinear lower bound.

Exercises 1. Prove that for all n ∈ N: e

2. 3. 4. 5.

n n e

≤ n! ≤ en

n n e

.

Hint: Use 1 + x ≤ e x for all x ∈ R. Prove that log(n!) = 2(n log n). Prove that n log n = O(n 1+ ) for any > 0. Show that the running time of the PATH E NUMERATION A LGORITHM is O(n · n!). Suppose we have an algorithm whose running time is 2(n(t + n 1/t )), where n is the input length and t is a positive parameter we can choose arbitrarily.

12

1 Introduction

How should t be chosen (depending on n) such that the running time (as a function of n) has a minimum rate of growth? 6. Let s, t be binary strings, both of length m. We say that s is lexicographically smaller than t if there exists an index j ∈ {1, . . . , m} such that si = ti for i = 1, . . . , j − 1 and s j < t j . Now given n strings of length m, we want to sort them lexicographically. Prove that there is a linear-time algorithm for this problem (i.e. one with running time O(nm)). Hint: Group the strings according to the first bit and sort each group. 7. Describe an algorithm which sorts a list of natural numbers a1 , . . . , an in linear time; i.e. which finds a permutation π with aπ(i) ≤ aπ(i+1) (i = 1, . . . , n − 1) and runs in O(log(a1 + 1) + · · · + log(an + 1)) time. Hint: First sort the strings encoding the numbers according to their length. Then apply the algorithm of Exercise 6. Note: The algorithm discussed in this and the previous exercise is often called radix sorting.

References General Literature: Cormen, T.H., Leiserson, C.E., Rivest, R.L., and Stein, C. [2001]: Introduction to Algorithms. Second Edition. MIT Press, Cambridge 2001 Knuth, D.E. [1968]: The Art of Computer Programming; Vol. 1. Fundamental Algorithms. Addison-Wesley, Reading 1968 (third edition: 1997)

Cited References: Aho, A.V., Hopcroft, J.E., and Ullman, J.D. [1974]: The Design and Analysis of Computer Algorithms. Addison-Wesley, Reading 1974 Cobham, A. [1964]: The intrinsic computational difficulty of functions. Proceedings of the 1964 Congress for Logic Methodology and Philosophy of Science (Y. Bar-Hillel, ed.), North-Holland, Amsterdam 1964, pp. 24–30 Edmonds, J. [1965]: Paths, trees, and flowers. Canadian Journal of Mathematics 17 (1965), 449–467 Han, Y. [2004]: Deterministic sorting in O(n log log n) time and linear space. Journal of Algorithms 50 (2004), 96–105 Stirling, J. [1730]: Methodus Differentialis. London 1730

2 Graphs

Graphs are a fundamental combinatorial structure used throughout this book. In this chapter we not only review the standard definitions and notation, but also prove some basic theorems and mention some fundamental algorithms. After some basic definitions in Section 2.1 we consider fundamental objects occurring very often in this book: trees, circuits, and cuts. We prove some important properties and relations, and we also consider tree-like set systems in Section 2.2. The first graph algorithms, determining connected and strongly connected components, appear in Section 2.3. In Section 2.4 we prove Euler’s Theorem on closed walks using every edge exactly once. Finally, in Sections 2.5 and 2.6 we consider graphs that can be drawn in the plane without crossings.

2.1 Basic Definitions An undirected graph is a triple (V, E, 9), where V and E are finite sets and 9 : E → {X ⊆ V : |X | = 2}. A directed graph or digraph is a triple (V, E, 9), where V and E are finite sets and 9 : E → {(v, w) ∈ V × V : v 6= w}. By a graph we mean either an undirected graph or a digraph. The elements of V are called vertices, the elements of E are the edges. Two edges e, e0 with 9(e) = 9(e0 ) are called parallel. Graphs without parallel edges are called simple. For simple graphs we usually identify an edge e with its image 9(e) and write G = (V (G), E(G)), where E(G) ⊆ {X ⊆ V (G) : |X | = 2} or E(G) ⊆ V (G) × V (G). We often use this simpler notation even in the presence of parallel edges, then the “set” E(G) may contain several “identical” elements. |E(G)| denotes the number of edges, and for two edge sets E and F we always . have |E ∪ F| = |E| + |F| even if parallel edges arise. We say that an edge e = {v, w} or e = (v, w) joins v and w. In this case, v and w are adjacent. v is a neighbour of w (and vice versa). v and w are the endpoints of e. If v is an endpoint of an edge e, we say that v is incident with e. In the directed case we say that e = (v, w) leaves v (the tail of e) and enters w (the head of e). Two edges which share at least one endpoint are called adjacent. This terminology for graphs is not the only one. Sometimes vertices are called nodes or points, other names for edges are arcs (especially in the directed case) or lines. In some texts, a graph is what we call a simple undirected graph, in the presence of parallel edges they speak of multigraphs. Sometimes edges whose

14

2 Graphs

endpoints coincide, so-called loops, are considered. However, unless otherwise stated, we do not use them. For a digraph G we sometimes consider the underlying undirected graph, i.e. the undirected graph G 0 on the same vertex set which contains an edge {v, w} for each edge (v, w) of G. We also say that G is an orientation of G 0 . A subgraph of a graph G = (V (G), E(G)) is a graph H = (V (H ), E(H )) with V (H ) ⊆ V (G) and E(H ) ⊆ E(G). We also say that G contains H . H is an induced subgraph of G if it is a subgraph of G and E(H ) = {{x, y} or (x, y) ∈ E(G) : x, y ∈ V (H )}. Here H is the subgraph of G induced by V (H ). We also write H = G[V (H )]. A subgraph H of G is called spanning if V (H ) = V (G). If v ∈ V (G), we write G − v for the subgraph of G induced by V (G) \ {v}. If e ∈ E(G), we define G − e := (V (G), E(G) \ {e}). Furthermore, the addition . of a new edge e is abbreviated by G + e := (V (G), E(G) ∪ {e}). If G and H are two graphs, we denote by G + H the graph with V (G + H ) = V (G) ∪ V (H ) and E(G + H ) being the disjoint union of E(G) and E(H ) (parallel edges may arise). Two graphs G and H are called isomorphic if there are bijections 8V : V (G) → V (H ) and 8 E : E(G) → E(H ) such that 8 E ((v, w)) = (8V (v), 8V (w)) for all (v, w) ∈ E(G), or 8 E ({v, w}) = {8V (v), 8V (w)} for all {v, w} ∈ E(G) in the undirected case. We normally do not distinguish between isomorphic graphs; for example we say that G contains H if G has a subgraph isomorphic to H . Suppose we have an undirected graph G and X ⊆ V (G). By contracting (or shrinking) X we mean deleting the vertices in X and the edges in G[X ], adding a new vertex x and replacing each edge {v, w} with v ∈ X , w ∈ / X by an edge {x, w} (parallel edges may arise). Similarly for digraphs. We often call the result G/X . For a graph G and X, Y ⊆ V (G) we define E(X, Y ) := {{x, y} ∈ E(G) : x ∈ X \ Y, y ∈ Y \ X } if G is undirected and E + (X, Y ) := {(x, y) ∈ E(G) : x ∈ X \ Y, y ∈ Y \ X } if G is directed. For undirected graphs G and X ⊆ V (G) we define δ(X ) := E(X, V (G) \ X ). The set of neighbours of X is defined by 0(X ) := {v ∈ V (G) \ X : E(X, {v}) 6= ∅}. For digraphs G and X ⊆ V (G) we define δ + (X ) := E + (X, V (G) \ X ), δ − (X ) := δ + (V (G) \ X ) and δ(X ) := δ + (X )∪δ − (X ). We use subscripts (e.g. δG (X )) to specify the graph G if necessary. For singletons, i.e. one-element vertex sets {v} (v ∈ V (G)) we write δ(v) := δ({v}), 0(v) := 0({v}), δ + (v) := δ + ({v}) and δ − (v) := δ − ({v}). The degree of a vertex v is |δ(v)|, the number of edges incident to v. In the directed case, the in-degree is |δ − (v)|, the out-degree is |δ + (v)|, and the degree is |δ + (v)|+|δ − (v)|. A vertex v with zero degree is called isolated. A graph where all vertices have degree k is called P k-regular. For any graph, v∈V (G) |δ(v)| = 2|E(G)|. In particular, the number of vertices P P with odd degree is even. In a digraph, v∈V (G) |δ + (v)| = v∈V (G) |δ − (v)|. To prove these statements, please observe that each edge is counted twice on each

2.1 Basic Definitions

15

side of the first equation and once on each side of the second equation. With just a little more effort we get the following useful statements: Lemma 2.1. For a digraph G and any two sets X, Y ⊆ V (G): (a) |δ + (X )| + |δ + (Y )| = |δ + (X ∩ Y )| + |δ + (X ∪ Y )| + |E + (X, Y )| + |E + (Y, X )|; (b) |δ − (X )| + |δ − (Y )| = |δ − (X ∩ Y )| + |δ − (X ∪ Y )| + |E + (X, Y )| + |E + (Y, X )|. For an undirected graph G and any two sets X, Y ⊆ V (G): (c) |δ(X )| + |δ(Y )| = |δ(X ∩ Y )| + |δ(X ∪ Y )| + 2|E(X, Y )|; (d) |0(X )| + |0(Y )| ≥ |0(X ∩ Y )| + |0(X ∪ Y )|. Proof: All parts can be proved by simple counting arguments. Let Z := V (G) \ (X ∪ Y ). To prove (a), observe that |δ + (X )|+|δ + (Y )| = |E + (X, Z )|+|E + (X, Y \ X )|+ + |E (Y, Z )| + |E + (Y, X \ Y )| = |E + (X ∪ Y, Z )| + |E + (X ∩ Y, Z )| + |E + (X, Y \ X )| + |E + (Y, X \ Y )| = |δ + (X ∪ Y )| + |δ + (X ∩ Y )| + |E + (X, Y )| + |E + (Y, X )|. (b) follows from (a) by reversing each edge (replace (v, w) by (w, v)). (c) follows from (a) by replacing each edge {v, w} by a pair of oppositely directed edges (v, w) and (w, v). To show (d), observe that |0(X )| + |0(Y )| = |0(X ∪ Y )| + |0(X ) ∩ 0(Y )| + |0(X ) ∩ Y | + |0(Y ) ∩ X | ≥ |0(X ∪ Y )| + |0(X ∩ Y )|. A function f : 2U → R (where U is some finite set and 2U denotes its power set) is called • submodular if f (X ∩ Y ) + f (X ∪ Y ) ≤ f (X ) + f (Y ) for all X, Y ⊆ U ; • supermodular if f (X ∩ Y ) + f (X ∪ Y ) ≥ f (X ) + f (Y ) for all X, Y ⊆ U ; • modular if f (X ∩ Y ) + f (X ∪ Y ) = f (X ) + f (Y ) for all X, Y ⊆ U . So Lemma 2.1 implies that |δ + |, |δ − |, |δ| and |0| are submodular. This will be useful later. A complete graph is a simple undirected graph where each pair of vertices is adjacent. We denote the complete graph on n vertices by K n . The complement of a simple undirected graph G is the graph H for which G + H is a complete graph. A matching in an undirected graph G is a set of pairwise disjoint edges (i.e. the endpoints are all different). A vertex cover in G is a set S ⊆ V (G) of vertices such that every edge of G is incident to at least one vertex in S. An edge cover in G is a set F ⊆ E(G) of edges such that every vertex of G is incident to at least one edge in F. A stable set in G is a set of pairwise non-adjacent vertices. A graph containing no edges is called empty. A clique is a set of pairwise adjacent vertices. Proposition 2.2. Let G be a graph and X ⊆ V (G). Then the following three statements are equivalent: (a) X is a vertex cover in G,

16

2 Graphs

(b) V (G) \ X is a stable set in G, (c) V (G) \ X is a clique in the complement of G.

If F is a family of sets or graphs, we say that F is a minimal element of F if F contains F but no proper subset/subgraph of F. Similarly, F is maximal in F if F ∈ F and F is not a proper subset/subgraph of any element of F. When we speak of a minimum or maximum element, we mean one of minimum/maximum cardinality. For example, a minimal vertex cover is not necessarily a minimum vertex cover (see e.g. the graph in Figure 13.1), and a maximal matching is in general not maximum. The problems of finding a maximum matching, stable set or clique, or a minimum vertex cover or edge cover in an undirected graph will play important roles in later chapters. The line graph of a simple undirected graph G is the graph (E(G), F), where F = {{e1 , e2 } : e1 , e2 ∈ E(G), |e1 ∩ e2 | = 1}. Obviously, matchings in a graph G correspond to stable sets in the line graph of G. For the following notation, let G be a graph, directed or undirected. An edge progression W in G is a sequence v1 , e1 , v2 , . . . , vk , ek , vk+1 such that k ≥ 0, and ei = (vi , vi+1 ) ∈ E(G) or ei = {vi , vi+1 } ∈ E(G) for i = 1, . . . , k. If in addition ei 6= e j for all 1 ≤ i < j ≤ k, W is called a walk in G. W is closed if v1 = vk+1 . A path is a graph P = ({v1 , . . . , vk+1 }, {e1 , . . . , ek }) such that vi 6= v j for 1 ≤ i < j ≤ k +1 and the sequence v1 , e1 , v2 , . . . , vk , ek , vk+1 is a walk. P is also called a path from v1 to vk+1 or a v1 -vk+1 -path. v1 and vk+1 are the endpoints of P. By P[x,y] with x, y ∈ V (P) we mean the (unique) subgraph of P which is an x-y-path. Evidently, there is an edge progression from a vertex v to another vertex w if and only if there is a v-w-path. A circuit or a cycle is a graph ({v1 , . . . , vk }, {e1 , . . . , ek }) such that the sequence v1 , e1 , v2 , . . . , vk , ek , v1 is a (closed) walk and vi 6= v j for 1 ≤ i < j ≤ k. An easy induction argument shows that the edge set of a closed walk can be partitioned into edge sets of circuits. The length of a path or circuit is the number of its edges. If it is a subgraph of G, we speak of a path or circuit in G. A spanning path in G is called a Hamiltonian path while a spanning circuit in G is called a Hamiltonian circuit or a tour. A graph containing a Hamiltonian circuit is a Hamiltonian graph. For two vertices v and w we write dist(v, w) or distG (v, w) for the length of a shortest v-w-path (the distance from v to w) in G. If there is no v-w-path at all, i.e. w is not reachable from v, we set dist(v, w) := ∞. In the undirected case, dist(v, w) = dist(w, v) for all v, w ∈ V (G). We shall often P have a cost function c : E(G) → R. Then for F ⊆ E(G) we write c(F) := e∈F c(e) (and c(∅) = 0). This extends c to a modular function c : 2 E(G) → R. Moreover, dist(G,c) (v, w) denotes the minimum c(E(P)) over all v-w-paths P in G.

2.2 Trees, Circuits, and Cuts

17

2.2 Trees, Circuits, and Cuts Let G be some undirected graph. G is called connected if there is a v-w-path for all v, w ∈ V (G); otherwise G is disconnected. The maximal connected subgraphs of G are its connected components. Sometimes we identify the connected components with the vertex sets inducing them. A set of vertices X is called connected if the subgraph induced by X is connected. A vertex v with the property that G − v has more connected components than G is called an articulation vertex. An edge e is called a bridge if G − e has more connected components than G. An undirected graph without a circuit (as a subgraph) is called a forest. A connected forest is a tree. A vertex of degree 1 in a tree is called a leaf. A star is a tree where at most one vertex is not a leaf. In the following we shall give some equivalent characterizations of trees and their directed counterparts, arborescences. We need the following connectivity criterion: Proposition 2.3. (a) An undirected graph G is connected if and only if δ(X ) 6= ∅ for all ∅ 6= X ⊂ V (G). (b) Let G be a digraph and r ∈ V (G). Then there exists an r -v-path for every v ∈ V (G) if and only if δ + (X ) 6= ∅ for all X ⊂ V (G) with r ∈ X . Proof: (a): If there is a set X ⊂ V (G) with r ∈ X , v ∈ V (G) \ X , and δ(X ) = ∅, there can be no r -v-path, so G is not connected. On the other hand, if G is not connected, there is no r -v-path for some r and v. Let R be the set of vertices reachable from r . We have r ∈ R, v ∈ / R and δ(R) = ∅. (b) is proved analogously. Theorem 2.4. Let G be an undirected graph on n vertices. Then the following statements are equivalent: (a) (b) (c) (d) (e) (f)

G is a tree (i.e. is connected and has no circuits). G has n − 1 edges and no circuits. G has n − 1 edges and is connected. G is a minimal connected graph (i.e. every edge is a bridge). G is a minimal graph with δ(X ) 6= ∅ for all ∅ 6= X ⊂ V (G). G is a maximal circuit-free graph (i.e. the addition of any edge creates a circuit). (g) G contains a unique path between any pair of vertices. Proof: (a)⇒(g) follows from the fact that the union of two distinct paths with the same endpoints contains a circuit. (g)⇒(e)⇒(d) follows from Proposition 2.3(a). (d)⇒(f) is trivial. (f)⇒(b)⇒(c): This follows from the fact that for forests with n vertices, m edges and p connected components n = m + p holds. (The proof is a trivial induction on m.)

18

2 Graphs

(c)⇒(a): Let G be connected with n −1 edges. As long as there are any circuits in G, we destroy them by deleting an edge of the circuit. Suppose we have deleted k edges. The resulting graph G 0 is still connected and has no circuits. G 0 has m = n − 1 − k edges. So n = m + p = n − 1 − k + 1, implying k = 0. In particular, (d)⇒(a) implies that a graph is connected if and only if it contains a spanning tree (a spanning subgraph which is a tree). A digraph is called connected if the underlying undirected graph is connected. A digraph is a branching if the underlying undirected graph is a forest and each vertex v has at most one entering edge. A connected branching is an arborescence. By Theorem 2.4 an arborescence with n vertices has n − 1 edges, hence it has exactly one vertex r with δ − (r ) = ∅. This vertex is called its root; we also speak of an arborescence rooted at r . The vertices v with δ + (v) = ∅ are called leaves. Theorem 2.5. Let G be a digraph on n vertices. Then the following statements are equivalent: (a) (b) (c) (d) (e) (f) (g)

G is an arborescence rooted at r (i.e. a connected branching with δ − (r ) = ∅). G is a branching with n − 1 edges and δ − (r ) = ∅. G has n − 1 edges and every vertex is reachable from r . Every vertex is reachable from r , but deleting any edge destroys this property. G is a minimal graph with δ + (X ) 6= ∅ for all X ⊂ V (G) with r ∈ X . δ − (r ) = ∅ and there is a unique r -v-path for any v ∈ V (G) \ {r }. δ − (r ) = ∅, |δ − (v)| = 1 for all v ∈ V (G) \ {r }, and G contains no circuit.

Proof: (a)⇒(b) and (c)⇒(d) follow from Theorem 2.4. (b)⇒(c): We have that |δ − (v)| = 1 for all v ∈ V (G) \ {r }. So for any v we have an r -v-path (start at v and always follow the entering edge until r is reached). (d)⇒(e) is implied by Proposition 2.3(b). (e)⇒(f): The minimality in (e) implies δ − (r ) = ∅. Moreover, by Proposition 2.3(b) there is an r -v-path for all v. Suppose there are two r -v-paths P and Q for some v. Let e be the last edge of P that does not belong to Q. Then after deleting e, every vertex is still reachable from r . By Proposition 2.3(b) this contradicts the minimality in (e). (f)⇒(g)⇒(a): trivial A cut in an undirected graph G is an edge set of type δ(X ) for some ∅ 6= X ⊂ V (G). In a digraph G, δ + (X ) is a directed cut if ∅ 6= X ⊂ V (G) and δ − (X ) = ∅, i.e. no edge enters the set X . We say that an edge set F ⊆ E(G) separates two vertices s and t if t is reachable from s in G but not in (V (G), E(G) \ F). In a digraph, an edge set δ + (X ) with s ∈ X and t ∈ / X is called an s-t-cut. An s-t-cut in an undirected graph is a cut δ(X ) for some X ⊂ V (G) with s ∈ X and t ∈ / X . An r-cut in a digraph is an edge set δ + (X ) for some X ⊂ V (G) with r ∈ X . By an undirected path, an undirected circuit, and an undirected cut in a digraph, we mean a subgraph corresponding to a path, a circuit, and a cut, respectively, in the underlying undirected graph.

2.2 Trees, Circuits, and Cuts

19

Lemma 2.6. (Minty [1960]) Let G be a digraph and e ∈ E(G). Suppose e is coloured black, while all other edges are coloured red, black or green. Then exactly one of the following statements holds: (a) There is an undirected circuit containing e and only red and black edges such that all black edges have the same orientation. (b) There is an undirected cut containing e and only green and black edges such that all black edges have the same orientation. Proof: Let e = (x, y). We label the vertices of G by the following procedure. First label y. In case v is already labelled and w is not, we label w if there is a black edge (v, w), a red edge (v, w) or a red edge (w, v). In this case, we write pr ed(w) := v. When the labelling procedure stops, there are two possibilities: Case 1: x has been labelled. Then the vertices x, pr ed(x), pr ed( pr ed(x)), . . . , y form an undirected circuit with the properties (a). Case 2: x has not been labelled. Then let R consist of all labelled vertices. Obviously, the undirected cut δ + (R) ∪ δ − (R) has the properties (b). Suppose that an undirected circuit C as in (a) and an undirected cut δ + (X ) ∪ − δ (X ) as in (b) both exist. All edges in their (nonempty) intersection are black, they all have the same orientation with respect to C, and they all leave X or all enter X . This is a contradiction. A digraph is called strongly connected if there is a path from s to t and a path from t to s for all s, t ∈ V (G). The strongly connected components of a digraph are the maximal strongly connected subgraphs. Corollary 2.7. In a digraph G, each edge belongs either to a (directed) circuit or to a directed cut. Moreover the following statements are equivalent: (a) G is strongly connected. (b) G contains no directed cut. (c) G is connected and each edge of G belongs to a circuit. Proof: The first statement follows directly from Minty’s Lemma 2.6 by colouring all edges black. This also proves (b)⇒(c). (a)⇒(b) follows from Proposition 2.3(b). (c)⇒(a): Let r ∈ V (G) be an arbitrary vertex. We prove that there is an r -vpath for each v ∈ V (G). Suppose this is not true, then by Proposition 2.3(b) there is some X ⊂ V (G) with r ∈ X and δ + (X ) = ∅. Since G is connected, we have δ + (X ) ∪ δ − (X ) 6= ∅ (by Proposition 2.3(a)), so let e ∈ δ − (X ). But then e cannot belong to a circuit since no edge leaves X . Corollary 2.7 and Theorem 2.5 imply that a digraph is strongly connected if and only if it contains for each vertex v a spanning arborescence rooted at v. A digraph is called acyclic if it contains no (directed) circuit. So by Corollary 2.7 a digraph is acyclic if and only if each edge belongs to a directed cut. Moreover,

20

2 Graphs

a digraph is acyclic if and only if its strongly connected components are the singletons. The vertices of an acyclic digraph can be ordered in a nice way: Definition 2.8. Let G be a digraph. A topological order of G is an order of the vertices V (G) = {v1 , . . . , vn } such that for each edge (vi , v j ) ∈ E(G) we have i < j. Proposition 2.9. A digraph has a topological order if and only if it is acyclic. Proof: If a digraph has a circuit, it clearly cannot have a topological order. We show the converse by induction on the number of edges. If there are no edges, every order is topological. Otherwise let e ∈ E(G); by Corollary 2.7 e belongs to a directed cut δ + (X ). Then a topological order of G[X ] followed by a topological order of G − X (both exist by the induction hypothesis) is a topological order of G. Circuits and cuts also play an important role in algebraic graph theory. For a graph G we associate a vector space R E(G) whose elements are vectors (xe )e∈E(G) with |E(G)| real components. Following Berge [1985] we shall now briefly discuss two linear subspaces which are particularly important. Let G be a digraph. We associate a vector ζ (C) ∈ {−1, 0, 1} E(G) with each undirected circuit C in G by setting ζ (C)e = 0 for e ∈ / E(C), and setting ζ (C)e ∈ {−1, 1} for e ∈ E(C) such that reorienting all edges e with ζ (C)e = −1 results in a directed circuit. Similarly, we associate a vector ζ (D) ∈ {−1, 0, 1} E(G) with / D, ζ (D)e = −1 each undirected cut D = δ(X ) in G by setting ζ (D)e = 0 for e ∈ for e ∈ δ − (X ) and ζ (D)e = 1 for e ∈ δ + (X ). Note that these vectors are properly defined only up to multiplication by −1. However, the subspaces of the vector space R E(G) generated by the set of vectors associated with the undirected circuits and by the set of vectors associated with the undirected cuts in G are properly defined; they are called the cycle space and the cocycle space of G, respectively. Proposition 2.10. The cycle space and the cocycle space are orthogonal to each other. Proof: Let C be any undirected circuit and D = δ(X ) be any undirected cut. We claim that the scalar product of ζ (C) and ζ (D) is zero. Since reorienting any edge does not change the scalar product we may assume that D is a directed cut. But then the result follows from observing that any circuit enters a set X the same number of times as it leaves X . We shall now show that the sum of the dimensions of the cycle space and the cocycle space is |E(G)|, the dimension of the whole space. A set of undirected circuits (undirected cuts) is called a cycle basis (a cocycle basis) if the associated vectors form a basis of the cycle space (the cocycle space, respectively). Let G be a graph (directed or undirected) and T a maximal subgraph without an undirected circuit. For each e ∈ E(G) \ E(T ) we call the unique undirected circuit in T + e the fundamental circuit of e with respect to T . Moreover, for each e ∈ E(T )

2.2 Trees, Circuits, and Cuts

21

there is a set X ⊆ V (G) with δG (X ) ∩ E(T ) = {e} (consider a component of T − e); we call δG (X ) the fundamental cut of e with respect to T . Theorem 2.11. Let G be a digraph and T a maximal subgraph without an undirected circuit. The |E(G) \ E(T )| fundamental circuits with respect to T form a cycle basis of G, and the |E(T )| fundamental cuts with respect to T form a cocycle basis of G. Proof: The vectors associated with the fundamental circuits are linearly independent since each fundamental circuit contains an element not belonging to any other. The same holds for the fundamental cuts. Since the vector spaces are orthogonal to each other by Proposition 2.10, the sum of their dimensions cannot exceed |E(G)| = |E(G) \ E(T )| + |E(T )|. The fundamental cuts have a nice property which we shall exploit quite often and which we shall discuss now. Let T be a digraph whose underlying undirected graph is a tree. Consider the family F := {Ce : e ∈ E(T )}, where for e = (x, y) ∈ E(T ) we denote by Ce the connected component of T − e containing y (so δ(Ce ) is the fundamental cut of e with respect to T ). If T is an arborescence, then any two elements of F are either disjoint or one is a subset of the other. In general F is at least cross-free: Deﬁnition 2.12. A set system is a pair (U, F), where U is a nonempty ﬁnite set and F a family of subsets of U . (U, F) is cross-free if for any two sets X, Y ∈ F, at least one of the four sets X \ Y , Y \ X , X ∩ Y , U \ (X ∪ Y ) is empty. (U, F) is laminar if for any two sets X, Y ∈ F, at least one of the three sets X \ Y , Y \ X , X ∩ Y is empty. In the literature set systems are also known as hypergraphs. See Figure 2.1(a) for an illustration of the laminar family {{a}, {b, c}, {a, b, c}, {a, b, c, d}, { f }, { f, g}}. Another word used for laminar is nested. (a)

(b) e

g

d a

b c

d

e

f

g f

a Fig. 2.1.

b, c

22

2 Graphs

Whether a set system (U, F) is laminar does not depend on U , so we sometimes simply say that F is a laminar family. However, whether a set system is cross-free can depend on the ground set U . If U contains an element that does not belong to any set of F, then F is cross-free if and only if it is laminar. Let r ∈ U be arbitrary. It follows directly from the deﬁnition that a set system (U, F) is cross-free if and only if F := {X ∈ F : r ∈ X } ∪ {U \ X : X ∈ F, r ∈ X } is laminar. Hence cross-free families are sometimes depicted similarly to laminar families: for example, Figure 2.2(a) shows the cross-free family {{b, c, d, e, f }, {c}, {a, b, c}, {e}, {a, b, c, d, f }, {e, f }}; a square corresponds to the set containing all elements outside. (a)

(b) d

f

b a

b

c

d

e

f a

c

e Fig. 2.2.

While oriented trees lead to cross-free families the converse is also true: every cross-free family can be represented by a tree in the following sense: Deﬁnition 2.13. Let T be a digraph such that the underlying undirected graph is a tree. Let U be a ﬁnite set and ϕ : U → V (T ). Let F := {Se : e ∈ E(T )}, where for e = (x, y) we deﬁne Se := {s ∈ U : ϕ(s) is in the same connected component of T − e as y}. Then (T, ϕ) is called a tree-representation of (U, F). See Figures 2.1(b) and 2.2(b) for examples. Proposition 2.14. Let (U, F) be a set system with a tree-representation (T, ϕ). Then (U, F) is cross-free. If T is an arborescence, then (U, F) is laminar. Moreover, every cross-free family has a tree-representation, and for laminar families, an arborescence can be chosen as T .

2.2 Trees, Circuits, and Cuts

23

Proof: If (T, ϕ) is a tree-representation of (U, F) and e = (v, w), f = (x, y) ∈ E(T ), we have an undirected v-x-path P in T (ignoring the orientations). There are four cases: If w, y ∈ / V (P) then Se ∩ Sf = ∅ (since T contains no circuit). / V (P) and w ∈ V (P) then If w ∈ / V (P) and y ∈ V (P) then Se ⊆ Sf . If y ∈ Sf ⊆ Se . If w, y ∈ V (P) then Se ∪ Sf = U . Hence (U, F) is cross-free. If T is an arborescence, the last case cannot occur (otherwise at least one vertex of P would have two entering edges), so F is laminar. . To prove the converse, let F first be a laminar family. We define V (T ) := F ∪ {r } and E(T ) := {(X, Y ) ∈ F × F : X ⊃ Y = 6 ∅ and there is no Z ∈ F with X ⊃ Z ⊃ Y } ∪ {(r, X ) : X = ∅ ∈ F or X is a maximal element of F} . We set ϕ(x) := X , where X is the minimal set in F containing x, and ϕ(x) := r if no set in F contains x. Obviously, T is an arborescence rooted at r , and (T, ϕ) is a tree-representation of F. Now let F be a cross-free family of subsets of U . Let r ∈ U . As noted above, F 0 := {X ∈ F : r 6∈ X } ∪ {U \ X : X ∈ F, r ∈ X } is laminar, so let (T, ϕ) be a tree-representation of (U, F 0 ). Now for an edge e ∈ E(T ) there are three cases: If Se ∈ F and U \ Se ∈ F, we replace the edge e = (x, y) by two edges (x, z) and (y, z), where z is a new vertex. If Se 6∈ F and U \ Se ∈ F, we replace the edge e = (x, y) by (y, x). If Se ∈ F and U \ Se 6∈ F, we do nothing. Let T 0 be the resulting graph. Then (T 0 , ϕ) is a tree-representation of (U, F). The above result is mentioned by Edmonds and Giles [1977] but was probably known earlier. Corollary 2.15. A laminar family of distinct subsets of U has at most 2|U | elements. A cross-free family of distinct subsets of U has at most 4|U | − 2 elements. Proof: We first consider a laminar family F of distinct nonempty proper subsets of U . We prove that |F| ≤ 2|U | − 2. Let (T, ϕ) be a tree-representation, where T is an arborescence whose number of vertices is as small as possible. For every w ∈ V (T ) we have either |δ + (w)| ≥ 2 or there exists an x ∈ U with ϕ(x) = w or both. (For the root this follows from U ∈ / F, for the leaves from ∅ ∈ / F, for all other vertices from the minimality of T .) j There k can be at most |U | vertices w with ϕ(x) = w for some x ∈ U and at most |E(T )| )| vertices w with |δ + (w)| ≥ 2. So |E(T )| + 1 = |V (T )| ≤ |U | + |E(T 2 2 and thus |F| = |E(T )| ≤ 2|U | − 2. Now let (U, F) be a cross-free family with ∅, U ∈ / F, and let r ∈ U . Since F 0 := {X ∈ F : r 6∈ X } ∪ {U \ X : X ∈ F, r ∈ X } is laminar, we have |F 0 | ≤ 2|U | − 2. Hence |F| ≤ 2|F 0 | ≤ 4|U | − 4. The proof is concluded by taking ∅ and U as possible members of F into account.

24

2 Graphs

2.3 Connectivity Connectivity is a very important concept in graph theory. For many problems it suffices to consider connected graphs, since otherwise we can solve the problem for each connected component separately. So it is a fundamental task to detect the connected components of a graph. The following simple algorithm finds a path from a specified vertex s to all other vertices that are reachable from s. It works for both directed and undirected graphs. In the undirected case it builds a maximal tree containing s; in the directed case it constructs a maximal arborescence rooted at s.

G RAPH S CANNING A LGORITHM Input:

A graph G (directed or undirected) and some vertex s.

Output:

The set R of vertices reachable from s, and a set T ⊆ E(G) such that (R, T ) is an arborescence rooted at s, or a tree.

1

Set R := {s}, Q := {s} and T := ∅.

2

If Q = ∅ then stop, else choose a v ∈ Q. Choose a w ∈ V (G) \ R with e = (v, w) ∈ E(G) or e = {v, w} ∈ E(G). If there is no such w then set Q := Q \ {v} and go to . 2 Set R := R ∪ {w}, Q := Q ∪ {w} and T := T ∪ {e}. Go to . 2

3

4

Proposition 2.16. The G RAPH S CANNING A LGORITHM works correctly. Proof: At any time, (R, T ) is a tree or an arborescence rooted at s. Suppose at the end there is a vertex w ∈ V (G) \ R that is reachable from s. Let P be an s-w-path, and let {x, y} or (x, y) be an edge of P with x ∈ R and y ∈ / R. Since x has been added to R, it also has been added to Q at some time during the execution of the algorithm. The algorithm does not stop before removing x from Q. But this is done in / R. 3 only if there is no edge {x, y} or (x, y) with y ∈ Since this is the first graph algorithm in this book we discuss some implementation issues. The first question is how the graph is given. There are several natural ways. For example, one can think of a matrix with a row for each vertex and a column for each edge. The incidence matrix of an undirected graph G is the matrix A = (av,e )v∈V (G), e∈E(G) where ( 1 if v ∈ e av,e = . 0 if v 6∈ e The incidence matrix of a digraph G is the matrix A = (av,e )v∈V (G), e∈E(G) where −1 if v = x . av,(x,y) = 1 if v = y 0 if v 6∈ {x, y}

2.3 Connectivity

25

Of course this is not very efficient since each column contains only two nonzero entries. The space needed for storing an incidence matrix is obviously O(nm), where n := |V (G)| and m := |E(G)|. A better way seems to be having a matrix whose rows and columns are indexed by the vertex set. The adjacency matrix of a simple graph G is the 0-1-matrix A = (av,w )v,w∈V (G) with av,w = 1 iff {v, w} ∈ E(G) or (v, w) ∈ E(G). For graphs with parallel edges we can define av,w to be the number of edges from v to w. An adjacency matrix requires O(n 2 ) space for simple graphs. The adjacency matrix is appropriate if the graph is dense, i.e. has 2(n 2 ) edges (or more). For sparse graphs, say with O(n) edges only, one can do much better. Besides storing the number of vertices we can simply store a list of the edges, for each edge noting its endpoints. If we address each vertex by a number from 1 to n, the space needed for each edge is O(log n). Hence we need O(m log n) space altogether. Just storing the edges in an arbitrary order is not very convenient. Almost all graph algorithms require finding the edges incident to a given vertex. Thus one should have a list of incident edges for each vertex. In case of directed graphs, two lists, one for entering edges and one for leaving edges, are appropriate. This data structure is called adjacency list; it is the most customary one for graphs. For direct access to the list(s) of each vertex we have pointers to the heads of all lists; these can be stored with O(n log m) additional bits. Hence the total number of bits required for an adjacency list is O(n log m + m log n). Whenever a graph is part of the input of an algorithm in this book, we assume that the graph is given by an adjacency list. As for elementary operations on numbers (see Section 1.2), we assume that elementary operations on vertices and edges take constant time only. This includes scanning an edge, identifying its ends and accessing the head of the adjacency list for a vertex. The running time will be measured by the parameters n and m, and an algorithm running in O(m + n) time is called linear. We shall always use the letters n and m for the number of vertices and the number of edges. For many graph algorithms it causes no loss of generality to assume that the graph at hand is simple and connected; hence n − 1 ≤ m < n 2 . Among parallel edges we often have to consider only one, and different connected components can often be analyzed separately. The preprocessing can be done in linear time in advance; see Exercise 13 and the following. We can now analyze the running time of the G RAPH S CANNING A LGORITHM: Proposition 2.17. The G RAPH S CANNING A LGORITHM can be implemented to run in O(m + n) time. The connected components of a graph can be determined in linear time. Proof: We assume that G is given by an adjacency list. For each vertex x we introduce a pointer current(x), indicating the current edge in the list containing all edges in δ(x) or δ + (x) (this list is part of the input). Initially current(x) is set to the first element of the list. In , 3 the pointer moves forward. When the end of

26

2 Graphs

the list is reached, x is removed from Q and will never be inserted again. So the overall running time is proportional to the number of vertices plus the number of edges, i.e. O(n + m). To identify the connected components of a graph, we apply the algorithm once and check if R = V (G). If so, the graph is connected. Otherwise R is a connected component, and we apply the algorithm to (G, s 0 ) for an arbitrary vertex s 0 ∈ V (G) \ R (and iterate until all vertices have been scanned, i.e. added to R). Again, no edge is scanned twice, so the overall running time remains linear. An interesting question is in which order the vertices are chosen in . 3 Obviously we cannot say much about this order if we do not specify how to choose a v ∈ Q in . 2 Two methods are frequently used; they are called D EPTH -F IRST S EARCH (DFS) and B READTH -F IRST S EARCH (BFS). In DFS we choose the v ∈ Q that was the last to enter Q. In other words, Q is implemented as a LIFO-stack (last-in-first-out). In BFS we choose the v ∈ Q that was the first to enter Q. Here Q is implemented by a FIFO-queue (first-in-first-out). An algorithm similar to DFS has been described already before 1900 by Tr´emaux and Tarry; see K¨onig [1936]. BFS seems to have been mentioned first by Moore [1959]. Trees (in the directed case: arborescences) (R, T ) computed by DFS and BFS are called DFS-tree and BFS-tree, respectively. For BFS-trees we note the following important property: Proposition 2.18. A BFS-tree contains a shortest path from s to each vertex reachable from s. The values distG (s, v) for all v ∈ V (G) can be determined in linear time. Proof: We apply BFS to (G, s) and add two statements: initially (in 1 of the G RAPH S CANNING A LGORITHM) we set l(s) := 0, and in 4 we set l(w) := l(v) + 1. We obviously have that l(v) = dist(R,T ) (s, v) for all v ∈ R, at any stage of the algorithm. Moreover, if v is the currently scanned vertex (chosen in ), 2 at this time there is no vertex w ∈ R with l(w) > l(v) + 1 (because the vertices are scanned in an order with nondecreasing l-values). Suppose that when the algorithm terminates there is a vertex w ∈ V (G) with distG (s, w) < dist(R,T ) (s, w); let w have minimum distance from s in G with this property. Let P be a shortest s-w-path in G, and let e = (v, w) or e = {v, w} be the last edge in P. We have distG (s, v) = dist(R,T ) (s, v), but e does not belong to T . Moreover, l(w) = dist(R,T ) (s, w) > distG (s, w) = distG (s, v) + 1 = dist(R,T ) (s, v)+1 = l(v)+1. This inequality combined with the above observation proves that w did not belong to R when v was removed from Q. But this contradicts

3 because of edge e. This result will also follow from the correctness of D IJKSTRA ’ S A LGORITHM for the S HORTEST PATH P ROBLEM, which can be thought of as a generalization of BFS to the case where we have nonnegative weights on the edges (see Section 7.1).

2.3 Connectivity

27

We now show how to identify the strongly connected components of a digraph. Of course, this can easily be done by using n times DFS (or BFS). However, it is possible to find the strongly connected components by visiting every edge only twice:

S TRONGLY C ONNECTED C OMPONENT A LGORITHM Input:

A digraph G.

Output:

A function comp : V (G) → N indicating the membership of the strongly connected components.

1

Set R := ∅. Set N := 0.

2

For all v ∈ V (G) do: If v ∈ / R then V ISIT1(v).

3

Set R := ∅. Set K := 0.

4

For i := |V (G)| down to 1 do: / R then set K := K + 1 and V ISIT2(ψ −1 (i)). If ψ −1 (i) ∈

V ISIT1(v)

1

Set R := R ∪ {v}.

2

For all w ∈ V (G) \ R with (v, w) ∈ E(G) do V ISIT1(w).

3

Set N := N + 1, ψ(v) := N and ψ −1 (N ) := v.

V ISIT2(v)

1

Set R := R ∪ {v}.

2

For all w ∈ V (G) \ R with (w, v) ∈ E(G) do V ISIT2(w).

3

Set comp(v) := K .

Figure 2.3 shows an example: The first DFS scans the vertices in the order a, g, b, d, e, f and produces the arborescence shown in the middle; the numbers are the ψ-labels. Vertex c is the only one that is not reachable from a; it gets the highest label ψ(c) = 7. The second DFS starts with c but cannot reach any other vertex via a reverse edge. So it proceeds with vertex a because ψ(a) = 6. Now b, g and f can be reached. Finally e is reached from d. The strongly connected components are {c}, {a, b, f, g} and {d, e}. In summary, one DFS is needed to find an appropriate numbering, while in the second DFS the reverse graph is considered and the vertices are processed in decreasing order with respect to this numbering. Each connected component of the second DFS-forest is an anti-arborescence, a graph arising from an arborescence by reversing every edge. We show that these anti-arborescences identify the strongly connected components.

28

2 Graphs

Fig. 2.3.

Theorem 2.19. The S TRONGLY C ONNECTED C OMPONENT A LGORITHM identifies the strongly connected components correctly in linear time. Proof: The running time is obviously O(n + m). Of course, vertices of the same strongly connected component are always in the same component of any DFS-forest, so they get the same comp-value. We have to prove that two vertices u and v with comp(u) = comp(v) indeed lie in the same strongly connected component. Let r (u) and r (v) be the vertex reachable from u and v with the highest ψ-label, respectively. Since comp(u) = comp(v), i.e. u and v lie in the same anti-arborescence of the second DFS-forest, r := r (u) = r (v) is the root of this anti-arborescence. So r is reachable from both u and v. Since r is reachable from u and ψ(r ) ≥ ψ(u), r has not been added to R after u in the first DFS, and the first DFS-forest contains an r -u-path. In other words, u is reachable from r . Analogously, v is reachable from r . Altogether, u is reachable from v and vice versa, proving that indeed u and v belong to the same strongly connected component. It is interesting that this algorithm also solves another problem: finding a topological order of an acyclic digraph. Observe that contracting the strongly connected components of any digraph yields an acyclic digraph. By Proposition 2.9 this acyclic digraph has a topological order. In fact, such an order is given by the numbers comp(v) computed by the S TRONGLY C ONNECTED C OMPONENT A LGORITHM: Theorem 2.20. The S TRONGLY C ONNECTED C OMPONENT A LGORITHM determines a topological order of the digraph resulting from contracting each strongly connected component of G. In particular, we can for any given digraph either find a topological order or decide that none exists in linear time. Proof: Let X and Y be two strongly connected components of a digraph G, and suppose the S TRONGLY C ONNECTED C OMPONENT A LGORITHM computes comp(x) = k1 for x ∈ X and comp(y) = k2 for y ∈ Y with k1 < k2 . We claim + (Y, X ) = ∅. that E G Suppose that there is an edge (y, x) ∈ E(G) with y ∈ Y and x ∈ X . All vertices in X are added to R in the second DFS before the first vertex of Y is

2.3 Connectivity

29

added. In particular we have x ∈ R and y ∈ / R when the edge (y, x) is scanned in the second DFS. But this means that y is added to R before K is incremented, contradicting comp(y) 6= comp(x). Hence the comp-values computed by the S TRONGLY C ONNECTED C OMPO NENT A LGORITHM determine a topological order of the digraph resulting from contracting the strongly connected components. The second statement of the theorem now follows from Proposition 2.9 and the observation that a digraph is acyclic if and only if its strongly connected components are the singletons. The first linear-time algorithm that identifies the strongly connected components was given by Tarjan [1972]. The problem of finding a topological order (or deciding that none exists) was solved earlier (Kahn [1962], Knuth [1968]). Both BFS and DFS occur as subroutines in many other combinatorial algorithms. Some examples will appear in later chapters. Sometimes one is interested in higher connectivity. Let k ≥ 2. An undirected graph with more than k vertices and the property that it remains connected even if we delete any k − 1 vertices, is called k-connected. A graph with at least two vertices is k-edge-connected if it remains connected after deleting any k −1 edges. So a connected graph with at least three vertices is 2-connected (2-edge-connected) if and only if it has no articulation vertex (no bridge, respectively). The largest k and l such that a graph G is k-connected and l-edge-connected are called the vertex-connectivity and edge-connectivity of G. Here we say that a graph is 1-connected (and 1-edge-connected) if it is connected. A disconnected graph has vertex-connectivity and edge-connectivity zero. The blocks of an undirected graph are its maximal connected subgraphs without articulation vertex. Thus each block is either a maximal 2-connected subgraph, or consists of a bridge or an isolated vertex. Two blocks have at most one vertex in common, and a vertex belonging to more than one block is an articulation vertex. The blocks of an undirected graph can be determined in linear time quite similarly to the S TRONGLY C ONNECTED C OMPONENT A LGORITHM; see Exercise 16. Here we prove a nice structure theorem for 2-connected graphs. We construct graphs from a single vertex by sequentially adding ears: Definition 2.21. Let G be a graph (directed or undirected). An ear-decomposition of G is a sequence r, P1 , . . . , Pk with G = ({r }, ∅)+ P1 +· · ·+ Pk , such that each Pi is either a path where exactly the endpoints belong to {r } ∪ V (P1 ) ∪ · · · ∪ V (Pi−1 ), or a circuit where exactly one of its vertices belongs to {r } ∪ V (P1 ) ∪ · · · ∪ V (Pi−1 ) (i ∈ {1, . . . , k}). P1 , . . . , Pk are called ears. If k ≥ 1, P1 is a circuit of length at least three, and P2 , . . . , Pk are paths, then the ear-decomposition is called proper. Theorem 2.22. (Whitney [1932]) An undirected graph is 2-connected if and only if it has a proper ear-decomposition. Proof: Evidently a circuit of length at least three is 2-connected. Moreover, if G is 2-connected, then so is G + P, where P is an x-y-path, x, y ∈ V (G) and x 6= y:

30

2 Graphs

deleting any vertex does not destroy connectivity. We conclude that a graph with a proper ear-decomposition is 2-connected. To show the converse, let G be a 2-connected graph. Let G 0 be the maximal simple subgraph of G; evidently G 0 is also 2-connected. Hence G 0 cannot be a tree; i.e. it contains a circuit. Since it is simple, G 0 , and thus G, contains a circuit of length at least three. So let H be a maximal subgraph of G that has a proper ear-decomposition; H exists by the above consideration. Suppose H is not spanning. Since G is connected, we then know that there exists an edge e = {x, y} ∈ E(G) with x ∈ V (H ) and y ∈ / V (H ). Let z be a vertex in V (H ) \ {x}. Since G − x is connected, there exists a path P from y to z in G − x. Let z 0 be the first vertex on this path, when traversed from y, that belongs to V (H ). Then P[y,z 0 ] + e can be added as an ear, contradicting the maximality of H . Thus H is spanning. Since each edge of E(G) \ E(H ) can be added as an ear, we conclude that H = G. See Exercise 17 for similar characterizations of 2-edge-connected graphs and strongly connected digraphs.

2.4 Eulerian and Bipartite Graphs Euler’s work on the problem of traversing each of the seven bridges of K¨onigsberg exactly once was the origin of graph theory. He showed that the problem had no solution by defining a graph, asking for a walk containing all edges, and observing that more than two vertices had odd degree. Definition 2.23. An Eulerian walk in a graph G is a closed walk containing every edge. An undirected graph G is called Eulerian if the degree of each vertex is even. A digraph G is Eulerian if |δ − (v)| = |δ + (v)| for each v ∈ V (G). Although Euler neither proved sufficiency nor considered the case explicitly in which we ask for a closed walk, the following famous result is usually attributed to him: Theorem 2.24. (Euler [1736], Hierholzer [1873]) A connected graph has an Eulerian walk if and only if it is Eulerian. Proof: The necessity of the degree conditions is obvious, the sufficiency is proved by the following algorithm (Theorem 2.25). The algorithm accepts as input only connected Eulerian graphs. Note that one can check in linear time whether a given graph is connected (Theorem 2.17) and Eulerian (trivial). The algorithm first chooses an initial vertex, then calls a recursive procedure. We first describe it for undirected graphs:

2.4 Eulerian and Bipartite Graphs

31

E ULER ’ S A LGORITHM Input:

An undirected connected Eulerian graph G.

Output:

An Eulerian walk W in G.

1

Choose v1 ∈ V (G) arbitrarily. Return W := E ULER(G, v1 ).

E ULER (G, v1 )

1

Set W := v1 and x := v1 .

2

If δ(x) = ∅ then go to . 4 Else let e ∈ δ(x), say e = {x, y}. Set W := W, e, y and x := y. Set E(G) := E(G) \ {e} and go to . 2

3

4

5

Let v1 , e1 , v2 , e2 , . . . , vk , ek , vk+1 be the sequence W . For i := 1 to k do: Set Wi := E ULER(G, vi ). Set W := W1 , e1 , W2 , e2 , . . . , Wk , ek , vk+1 . Return W . For digraphs, 2 has to be replaced by:

2

If δ + (x) = ∅ then go to . 4 Else let e ∈ δ + (x), say e = (x, y). We can analyze both versions (undirected and directed) simultaneously:

Theorem 2.25. E ULER ’ S A LGORITHM works correctly. Its running time is O(m+ n), where n = |V (G)| and m = |E(G)|. Proof: We use induction on |E(G)|, the case E(G) = ∅ being trivial. Because of the degree conditions, vk+1 = x = v1 when 4 is executed. So at this stage W is a closed walk. Let G 0 be the graph G at this stage. G 0 also satisfies the degree constraints. For each edge e ∈ E(G 0 ) there exists a minimum i ∈ {1, . . . , k} such that e is in the same connected component of G 0 as vi . Then by the induction hypothesis e belongs to Wi . So the closed walk W composed in 5 is indeed Eulerian. The running time is linear, because each edge is deleted immediately after being examined. E ULER ’ S A LGORITHM will be used several times as a subroutine in later chapters. Sometimes one is interested in making a given graph Eulerian by adding or contracting edges. Let G be an undirected graph and F a family of unordered . pairs of V (G) (edges or not). F is called an odd join if (V (G), E(G) ∪ F) is Eulerian. F is called an odd cover if the graph which results from G by successively contracting each e ∈ F is Eulerian. Both concepts are equivalent in the following sense.

32

2 Graphs

Theorem 2.26. (Aoshima and Iri [1977]) For any undirected graph we have: (a) Every odd join is an odd cover. (b) Every minimal odd cover is an odd join. Proof: Let G be an undirected graph. To prove (a), let F be an odd join. We build a graph G 0 by contracting the connected components of (V (G), F) in G. Each of these connected components contains an even number of odd-degree vertices (with respect to F and thus with respect to G, because F is an odd join). So the resulting graph has even degrees only. Thus F is an odd cover. To prove (b), let F be a minimal odd cover. Because of the minimality, (V (G), F) is a forest. We have to show that |δ F (v)| ≡ |δG (v)| (mod 2) for each v ∈ V (G). So let v ∈ V (G). Let C1 , . . . , Ck be the connected components of (V (G), F) − v that contain a vertex w with {v, w} ∈ F. Since F is a forest, k = |δ F (v)|. As F is an odd cover, contracting X := V (C1 ) ∪ · · · ∪ V (Ck ) ∪ {v} in G yields a vertex of even degree, i.e. |δG (X )| is even. On the other hand, because of the minimality of F, F \ {{v, w}} is not an odd cover (for any w with {v, w} ∈ F), so |δG (V (Ci ))| is odd for i = 1, . . . , k. Since k X

|δG (V (Ci ))| = |δG (X )| + |δG (v)|

i=1

− 2|E G ({v}, V (G) \ X )| + 2

X

|E G (Ci , C j )|,

1≤i< j≤k

we conclude that k has the same parity as |δG (v)|.

We shall return to the problem of making a graph Eulerian in Section 12.2. A bipartition of an undirected graph G is a partition of the vertex set V (G) = . A ∪ B such that the subgraphs induced by A and B are both empty. A graph is called . bipartite if it has a bipartition. The simple bipartite graph G with V (G) = A ∪ B, |A| = n, |B| = m and E(G) = {{a, b} : a ∈ A, b ∈ B} is denoted by K n,m (the . complete bipartite graph). When we write G = (A ∪ B, E(G)), we mean that G[A] and G[B] are both empty. Proposition 2.27. (K¨onig [1916]) An undirected graph is bipartite if and only if it contains no circuit of odd length. There is a linear-time algorithm which, given an undirected graph G, either finds a bipartition or an odd circuit. .

Proof: Suppose G is bipartite with bipartition V (G) = A ∪ B, and the closed walk v1 , e1 , v2 , . . . , vk , ek , vk+1 defines some circuit in G. W.l.o.g. v1 ∈ A. But then v2 ∈ B, v3 ∈ A, and so on. We conclude that vi ∈ A if and only if i is odd. But vk+1 = v1 ∈ A, so k must be even. To prove the sufficiency, we may assume that G is connected, since a graph is bipartite iff each connected component is (and the connected components can

2.5 Planarity

33

be determined in linear time; Proposition 2.17). We choose an arbitrary vertex s ∈ V (G) and apply BFS to (G, s) in order to obtain the distances from s to v for all v ∈ V (G) (see Proposition 2.18). Let T be the resulting BFS-tree. Define A := {v ∈ V (G) : distG (s, v) is even} and B := V (G) \ A. If there is an edge e = {x, y} in G[A] or G[B], the x-y-path in T together with e forms an odd circuit in G. If there is no such edge, we have a bipartition.

2.5 Planarity We often draw graphs in the plane. A graph is called planar if it can be drawn such that no pair of edges intersect. To formalize this concept we need the following topological terms: Definition 2.28. A simple Jordan curve is the image of a continuous injective function ϕ : [0, 1] → R2 ; its endpoints are ϕ(0) and ϕ(1). A closed Jordan curve is the image of a continuous function ϕ : [0, 1] → R2 with ϕ(0) = ϕ(1) and ϕ(τ ) 6= ϕ(τ 0 ) for 0 ≤ τ < τ 0 < 1. A polygonal arc is a simple Jordan curve which is the union of finitely many intervals (straight line segments). A polygon is a closed Jordan curve which is the union of finitely many intervals. Let R = R2 \ J , where J is the union of finitely many intervals. We define the connected regions of R as equivalence classes where two points in R are equivalent if they can be joined by a polygonal arc within R. Definition 2.29. A planar embedding of a graph G consists of an injective mapping ψ : V (G) → R2 and for each e = {x, y} ∈ E(G) a polygonal arc Je with endpoints ψ(x) and ψ(y), such that for each e = {x, y} ∈ E(G): [ Je0 = ∅. (Je \ {ψ(x), ψ(y)}) ∩ {ψ(v) : v ∈ V (G)} ∪ e0 ∈E(G)\{e}

A graph is called planar if it has a planar embedding. Let G be a (planar) graph with some fixed planar embedding 8 = (ψ, (Je )e∈E(G) ). After removing the points and polygonal arcs from the plane, the remainder, [ Je , R := R2 \ {ψ(v) : v ∈ V (G)} ∪ e∈E(G)

splits into open connected regions, called faces of 8. For example, K 4 is obviously planar but it will turn out that K 5 is not planar. Exercise 23 shows that restricting ourselves to polygonal arcs instead of arbitrary Jordan curves makes no substantial difference. We will show later that for simple graphs it is indeed sufficient to consider straight line segments only. Our aim is to characterize planar graphs. Following Thomassen [1981], we first prove the following topological fact, a version of the Jordan curve theorem:

34

2 Graphs

Theorem 2.30. If J is a polygon, then R2 \ J splits into exactly two connected regions, each of which has J as its boundary. If J is a polygonal arc, then R2 \ J has only one connected region. Proof: Let J be a polygon, p ∈ R2 \ J and q ∈ J . Then there exists a polygonal arc in (R2 \ J ) ∪ {q} joining p and q: starting from p, one follows the straight line towards q until one gets close to J , then one proceeds within the vicinity of J . (We use the elementary topological fact that disjoint compact sets, in particular non-adjacent intervals of J , have a positive distance from each other.) We conclude that p is in the same connected region of R2 \ J as some points arbitrarily close to q. J is the union of ﬁnitely many intervals; one or two of these intervals contain q. Let > 0 such that the ball with center q and radius contains no other interval of J ; then clearly this ball intersects at most two connected regions. Since p ∈ R2 \ J and q ∈ J were chosen arbitrarily, we conclude that there are at most two regions and each region has J as its boundary. Since the above also holds if J is a polygonal arc and q is an endpoint of J , R2 \ J has only one connected region in this case. Returning to the case when J is a polygon, it remains to prove that R2 \ J has more than one region. For any p ∈ R2 \ J and any angle α we consider the ray lα starting at p with angle α. J ∩ lα is a set of points or closed intervals. Let cr ( p, lα ) be the number of these points or intervals that J enters from a different side of lα than to which it leaves (the number of times J “crosses” lα ; e.g. in Figure 2.4 we have cr ( p, lα ) = 2).

J

J

p

lα

J Fig. 2.4.

Note that for any angle α, lim cr ( p, lα+ ) − cr ( p, lα ) + lim cr ( p, lα+ ) − cr ( p, lα ) →0, 0 is twice the number of intervals of J ∩ lα that J enters from the same side as to which it leaves. Therefore g( p, α) := (cr ( p, lα ) mod 2) is a continuous function in α, so it is constant and we denote it by g( p). Clearly g( p) is constant for points

2.5 Planarity

35

p on each straight line not intersecting J , so it is constant within each region. However, g( p) 6= g(q) for points p, q such that the straight line segment joining p and q intersects J exactly once. Hence there are indeed two regions. Exactly one of the faces, the outer face, is unbounded. Proposition 2.31. Let G be a 2-connected graph with a planar embedding 8. Then every face is bounded by a circuit, and every edge is on the boundary of exactly two faces. Moreover, the number of faces is |E(G)| − |V (G)| + 2. Proof: By Theorem 2.30 both assertions are true if G is a circuit. For general 2-connected graphs we use induction on the number of edges, using Theorem 2.22. Consider a proper ear-decomposition of G, and let P be the last ear, a path with endpoints x and y, say. Let G 0 be the graph before adding the last ear, and let 80 be the restriction of 8 to G 0 . S Let 8 = (ψ, (Je )e∈E(G) ). Let F 0 be the face of 80 containing e∈E(P) Je \ {ψ(x), ψ(y)}. By induction, F 0 is bounded by a circuit C. C contains x and y, so C is the union of two x-y-paths Q 1 , Q 2 in G 0 . Now we apply Theorem 2.30 to each of the circuits Q 1 + P and Q 2 + P. We conclude that [ . . F 0 ∪ {ψ(x), ψ(y)} = F1 ∪ F2 ∪ Je e∈E(P)

and F1 and F2 are two faces of G bounded by the circuits Q 1 + P and Q 2 + P, respectively. Hence G has one more face than G 0 . Using |E(G)\ E(G 0 )| = |V (G)\ V (G 0 )| + 1, this completes the induction step. This proof is due to Tutte. It also implies easily that the circuits bounding the finite faces constitute a cycle basis (Exercise 24). The last statement of Proposition 2.31 is known as Euler’s formula; it holds for general connected graphs: Theorem 2.32. (Euler [1758], Legendre [1794]) For any planar connected graph G with any embedding, the number of faces is |E(G)| − |V (G)| + 2. Proof: We have already proved the statement for 2-connected graphs (Proposition 2.31). Moreover, the assertion is trivial if |V (G)| = 1 and follows from Theorem 2.30 if |E(G)| = 1. If |V (G)| = 2 and |E(G)| ≥ 2, then we can subdivide one edge e, thereby increasing the number of vertices and the number of edges by one and making the graph 2-connected, and apply Proposition 2.31. So we may now assume that G has an articulation vertex x; we proceed by induction on the number of vertices. Let 8 be an embedding of G. Let C1 , . . . , Ck be the connected components of G − x; and let 8i be the restriction of 8 to G i := G[V (Ci ) ∪ {x}] for i = 1, . . . , k. The set of inner (bounded) faces of 8 is the disjoint union of the sets of inner faces of 8i , i = 1, . . . , k. By applying the induction hypothesis to (G i , 8i ), i = 1, . . . , k, we get that the total number of inner faces of (G, 8) is

36 k i=1

2 Graphs

(|E(G i )|−|V (G i )|+1) = |E(G)|−

k

|V (G i )\{x}| = |E(G)|−|V (G)|+1.

i=1

Taking the outer face into account concludes the proof.

In particular, the number of faces is independent of the embedding. The average degree of a simple planar graph is less than 6: Corollary 2.33. Let G be a 2-connected simple planar graph whose minimum k circuit has length k (we also say that G has girth k). Then G has at most (n −2) k−2 edges. Any simple planar graph with n ≥ 3 vertices has at most 3n − 6 edges. Proof: First assume that G is 2-connected. Let some embedding of G be given, and let r be the number of faces. By Euler’s formula (Theorem 2.32), r = |E(G)| − |V (G)| + 2. By Proposition 2.31, each face is bounded by a circuit, i.e. by at least k edges, and each edge is on the boundary of exactly two faces. Hence kr ≤ 2|E(G)|. Combining the two results we get |E(G)| − |V (G)| + 2 ≤ k2 |E(G)|, k implying |E(G)| ≤ (n − 2) k−2 . If G is not 2-connected we add edges between non-adjacent vertices to make it 3 2-connected while preserving planarity. By the ﬁrst part we have at most (n −2) 3−2 edges, including the new ones. Now we show that certain graphs are non-planar: Corollary 2.34. Neither K 5 nor K 3,3 is planar. Proof: This follows directly from Corollary 2.33: K 5 has ﬁve vertices but 10 > 4 3·5−6 edges; K 3,3 is 2-connected, has girth 4 (as it is bipartite) and 9 > (6−2) 4−2 edges.

Fig. 2.5.

Figure 2.5 shows these two graphs, which are the smallest non-planar graphs. We shall prove that every non-planar graph contains, in a certain sense, K 5 or K 3,3 . To make this precise we need the following notion: Deﬁnition 2.35. Let G and H be two undirected graphs. G is .a minor of H if . there exists a subgraph H of H and a partition V (H ) = V1 ∪ · · · ∪ Vk of its

2.5 Planarity

37

vertex set into connected subsets such that contracting each of V1 , . . . , Vk yields a graph which is isomorphic to G. In other words, G is a minor of H if it can be obtained from H by a series of operations of the following type: delete a vertex, delete an edge or contract an edge. Since neither of these operations destroys planarity, any minor of a planar graph is planar. Hence a graph which contains K 5 or K 3,3 as a minor cannot be planar. Kuratowski’s Theorem says that the converse is also true. We first consider 3-connected graphs and start with the following lemma (which is the heart of Tutte’s so-called wheel theorem): Lemma 2.36. (Tutte [1961], Thomassen [1980]) Let G be a 3-connected graph with at least five vertices. Then there exists an edge e such that G/e is also 3-connected. Proof: Suppose there is no such edge. Then for each edge e = {v, w} there exists a vertex x such that G − {v, w, x} is disconnected, i.e. has a connected component C with |V (C)| < |V (G)| − 3. Choose e, x and C such that |V (C)| is minimum. x has a neighbour y in C, because otherwise C is a connected component of G − {v, w} (but G is 3-connected). By our assumption, G/{x, y} is not 3connected, i.e. there exists a vertex z such that G − {x, y, z} is disconnected. Since {v, w} ∈ E(G), there exists a connected component D of G − {x, y, z} which contains neither v nor w. But D contains a neighbour d of y, since otherwise D is a connected component of G − {x, z} (again contradicting the fact that G is 3-connected). So d ∈ V (D) ∩ V (C), and thus D is a subgraph of C. Since y ∈ V (C) \ V (D), we have a contradiction to the minimality of |V (C)|. Theorem 2.37. (Kuratowski [1930], Wagner [1937]) A 3-connected graph is planar if and only if it contains neither K 5 nor K 3,3 as a minor. Proof: As the necessity is evident (see above), we prove the sufficiency. Since K 4 is obviously planar, we proceed by induction on the number of vertices: let G be a 3-connected graph with more than four vertices but no K 5 or K 3,3 minor. By Lemma 2.36, there exists an edge e = {v, w} such that G/e is 3-connected. Let 8 = ψ, (Je )e∈E(G) be a planar embedding of G/e, which exists by induction. Let x be the vertex in G/e which arises by contracting e. Consider (G/e) − x with the restriction of 8 as a planar embedding. Since (G/e) − x is 2-connected, every face is bounded by a circuit (Proposition 2.31). In particular, the face containing the point ψ(x) is bounded by a circuit C. Let y1 , . . . , yk ∈ V (C) be the neighbours of v that are distinct from w, numbered in cyclic order, and partition C into edge-disjoint paths Pi , i = 1, . . . , k, such that Pi is a yi -yi+1 -path (yk+1 := y1 ). Suppose there exists an index i ∈ {1, . . . , k} such that 0(w) ⊆ {v} ∪ V (Pi ). Then a planar embedding of G can be constructed easily by modifying 8. We shall prove that all other cases are impossible. First, if w has three neighbours among y1 , . . . , yk , we have a K 5 minor (Figure 2.6(a)).

38

2 Graphs (b)

(a)

(c) yi

z

yi

yi+1

v C

v

w

w

C

v

w

C yj

z

Fig. 2.6.

Next, if (w) = {v, yi , y j } for some i < j, then we must have i + 1 < j and (i, j) = (1, k) (otherwise yi and y j would both lie on Pi or Pj ); see Figure 2.6(b). Otherwise there is a neighbour z of w in V (Pi ) \ {yi , yi+1 } for some i and another neighbour z ∈ / V (Pi ) (Figure 2.6(c)). In both cases, there are four vertices y, z, y , z on C, in this cyclic order, with y, y ∈ (v) and z, z ∈ (w). This implies that we have a K 3,3 minor. The proof implies quite directly that every 3-connected simple planar graph has a planar embedding where each edge is embedded by a straight line and each face, except the outer face, is convex (Exercise 27(a)). The general case of Kuratowski’s Theorem can be reduced to the 3-connected case by gluing together planar embeddings of the maximal 3-connected subgraphs, or by the following lemma: Lemma 2.38. (Thomassen [1980]) Let G be a graph with at least ﬁve vertices which is not 3-connected and which contains neither K 5 nor K 3,3 as a minor. Then there exist two non-adjacent vertices v, w ∈ V (G) such that G + e, where e = {v, w} is a new edge, does not contain a K 5 or K 3,3 minor either. Proof: We use induction on |V (G)|. Let G be as above. If G is disconnected, we can simply add an edge e joining two different connected components. So henceforth we assume that G is connected. Since G is not 3-connected, there exists a set X = {x, y} of two vertices such that G − X is disconnected. (If G is not even 2-connected we may choose x to be an articulation vertex and y a neighbour of x.) Let C be a connected component of G − X , G 1 := G[V (C) ∪ X ] and G 2 := G − V (C). We ﬁrst prove the following: Claim: Let v, w ∈ V (G 1 ) be two vertices such that adding an edge e = {v, w} to G creates a K 3,3 or K 5 minor. Then at least one of G 1 + e + f and G 2 + f contains a K 5 or K 3,3 minor, where f is a new edge joining x and y. To prove this claim, let v, w ∈ V (G 1 ), e = {v, w} and suppose that there are disjoint connected vertex sets Z 1 , . . . , Z t of G + e such that after contracting each of them we have a K 5 (t = 5) or K 3,3 (t = 6) subgraph. Note that it is impossible that Z i ⊆ V (G 1 ) \ X and Z j ⊆ V (G 2 ) \ X for some i, j ∈ {1, . . . , t}: in this case the set of those Z k with Z k ∩ X = ∅ (there are at

2.5 Planarity

39

most two of these) separate Z i and Z j , contradicting the fact that both K 5 and K 3,3 are 3-connected. Hence there are two cases: If none of Z 1 , . . . , Z t is a subset of V (G 2 ) \ X , then G 1 + e + f also contains a K 5 or K 3,3 minor: just consider Z i ∩ V (G 1 ) (i = 1, . . . , t). Analogously, if none of Z 1 , . . . , Z t is a subset of V (G 1 ) \ X , then G 2 + f contains a K 5 or K 3,3 minor (consider Z i ∩ V (G 2 ) (i = 1, . . . , t)). The claim is proved. Now we ﬁrst consider the case when G contains an articulation vertex x, and y is a neighbour of x. We choose a second neighbour z of x such that y and z are in different connected components of G − x. W.l.o.g. say that z ∈ V (G 1 ). Suppose that the addition of e = {y, z} creates a K 5 or K 3,3 minor. By the claim, at least one of G 1 + e and G 2 contains a K 5 or K 3,3 minor (an edge {x, y} is already present). But then G 1 or G 2 , and thus G, contains a K 5 or K 3,3 minor, contradicting our assumption. Hence we may assume that G is 2-connected. Recall that x, y ∈ V (G) were chosen such that G − {x, y} is disconnected. If {x, y} ∈ / E(G) we simply add an edge f = {x, y}. If this creates a K 5 or K 3,3 minor, the claim implies that G 1 + f or G 2 + f contains such a minor. Since there is an x-y-path in each of G 1 , G 2 (otherwise we would have an articulation vertex of G), this implies that there is a K 5 or K 3,3 minor in G which is again a contradiction. Thus we can assume that f = {x, y} ∈ E(G). Suppose now that at least one of the graphs G i (i ∈ {1, 2}) is not planar. Then this G i has at least ﬁve vertices. Since it does not contain a K 5 or K 3,3 minor (this would also be a minor of G), we conclude from Theorem 2.37 that G i is not 3-connected. So we can apply the induction hypothesis to G i . By the claim, if adding an edge within G i does not introduce a K 5 or K 3,3 minor in G i , it cannot introduce such a minor in G either. So we may assume that both G 1 and G 2 are planar; let 1 and 2 be planar embeddings. Let Fi be a face of i with f on its boundary, and let z i be another vertex on the boundary of Fi , z i ∈ / {x, y} (i = 1, 2). We claim that adding an edge {z 1 , z 2 } (cf. Figure 2.7) does not introduce a K 5 or K 3,3 minor.

z1

z2

x

G1

f

G2

y Fig. 2.7.

40

2 Graphs

Suppose, on the contrary, that adding {z 1 , z 2 } and contracting some disjoint connected vertex sets Z 1 , . . . , Z t would create a K 5 (t = 5) or K 3,3 (t = 6) subgraph. First suppose that at most one of the sets Z i is a subset of V (G 1 ) \ {x, y}. Then the graph G 02 , arising from G 2 by adding one vertex w and edges from w to x, y and z 2 , also contains a K 5 or K 3,3 minor. (Here w corresponds to the contracted set Z i ⊆ V (G 1 ) \ {x, y}.) This is a contradiction since there is a planar embedding of G 02 : just supplement 82 by placing w within F2 . So we may assume that Z 1 , Z 2 ⊆ V (G 1 )\{x, y}. Analogously, we may assume / Z 1 and z 2 ∈ / Z 3 . Then we that Z 3 , Z 4 ⊆ V (G 2 ) \ {x, y}. W.l.o.g. we have z 1 ∈ cannot have a K 5 , because Z 1 and Z 3 are not adjacent. Moreover, the only possible common neighbours of Z 1 and Z 3 are Z 5 and Z 6 . Since in K 3,3 two vertices are either adjacent or have three common neighbours, a K 3,3 minor is also impossible. Theorem 2.37 and Lemma 2.38 yield Kuratowski’s Theorem: Theorem 2.39. (Kuratowski [1930], Wagner [1937]) An undirected graph is planar if and only if it contains neither K 5 nor K 3,3 as a minor. Indeed, Kuratowski proved a stronger version (Exercise 28). The proof can be turned into a polynomial-time algorithm quite easily (Exercise 27(b)). In fact, a linear-time algorithm exists: Theorem 2.40. (Hopcroft and Tarjan [1974]) There is a linear-time algorithm for finding a planar embedding of a given graph or deciding that it is not planar.

2.6 Planar Duality We shall now introduce an important duality concept. This is the only place in this book where we need loops. So in this section loops, i.e. edges whose endpoints coincide, are allowed. In a planar embedding loops are of course represented by polygons instead of polygonal arcs. Note that Euler’s formula (Theorem 2.32) also holds for graphs with loops: this follows from the observation that subdividing a loop e (i.e. replacing e = {v, v} by two parallel edges {v, w}, {w, v} where w is a new vertex) and adjusting the embedding (replacing the polygon Je by two polygonal arcs whose union is Je ) increases the number of edges and vertices each by one but does not change the number of faces. Definition 2.41. Let G be a directed or undirected graph, possibly with loops, and let 8 = (ψ, (Je )e∈E(G) ) be a planar embedding of G. We define the planar dual G ∗ whose vertices are the faces of 8 and whose edge set is {e∗ : e ∈ E(G)}, where e∗ connects the faces that are adjacent to Je (if Je is adjacent to only one face, then e∗ is a loop). In the directed case, say for e = (v, w), we orient e∗ = (F1 , F2 ) in such a way that F1 is the face “to the right” when traversing Je from ψ(v) to ψ(w).

2.6 Planar Duality

41

G ∗ is again planar. In fact, there obviously exists a planar embedding of G ∗ such that ψ ∗ (F) ∈ F for all faces F of and, for each e ∈ E(G), |Je∗ ∩ Je | = 1 and ⎞ ⎛ Je∗ ∩ ⎝{ψ(v) : v ∈ V (G)} ∪ Jf ⎠ = ∅.

ψ ∗ , (Je∗ )e∗ ∈E(G ∗ )

f ∈E(G)\{e}

Such an embedding is called a standard embedding of G ∗ . (a)

(b)

Fig. 2.8.

The planar dual of a graph really depends on the embedding: consider the two embeddings of the same graph shown in Figure 2.8. The resulting planar duals are not isomorphic, since the second one has a vertex of degree four (corresponding to the outer face) while the ﬁrst one is 3-regular. Proposition 2.42. Let G be an undirected connected planar graph with a ﬁxed embedding. Let G ∗ be its planar dual with a standard embedding. Then (G ∗ )∗ = G. Proof: Let ψ, (Je )e∈E(G) be a ﬁxed embedding of G and ψ ∗ , (Je∗ )e∗ ∈E(G ∗ ) a standard embedding of G ∗ . Let F be a face of G ∗ . The boundary of F contains Je∗ for at least one edge e∗ , so F must contain ψ(v) for one endpoint v of e. So every face of G ∗ contains at least one vertex of G. By applying Euler’s formula (Theorem 2.32) to G ∗ and to G, we get that the number of faces of G ∗ is |E(G ∗ )| − |V (G ∗ )| + 2 = |E(G)| − (|E(G)| − |V (G)| + 2) + 2 = |V (G)|. Hence each face of G ∗ contains exactly one vertex of G. From this we conclude that the planar dual of G ∗ is isomorphic to G. The requirement that G is connected is essential here: note that G ∗ is always connected, even if G is disconnected. Theorem 2.43. Let G be a connected planar undirected graph with arbitrary embedding. The edge set of any circuit in G corresponds to a minimal cut in G ∗ , and any minimal cut in G corresponds to the edge set of a circuit in G ∗ .

42

2 Graphs

Proof: Let 8 = (ψ, (Je )e∈E(G) ) beS a fixed planar embedding of G. Let C be a circuit in G. By Theorem 2.30, R2 \ e∈E(C) Je splits into exactly two connected regions. Let A and B be the set of faces of 8 in the inner and outer region, . respectively. We have V (G ∗ ) = A ∪ B and E G ∗ (A, B) = {e∗ : e ∈ E(C)}. Since A and B form connected sets in G ∗ , this is indeed a minimal cut. Conversely, let δG (A) be a minimal cut in G. Let 8∗ = (ψ ∗ , (Je )e∈E(G ∗ ) ) be a standard embedding of G ∗ . Let a ∈ A and b ∈ V (G) \ A. Observe that there is no polygonal arc in [ R := R2 \ {ψ ∗ (v) : v ∈ V (G ∗ )} ∪ Je∗ e∈δG (A)

which connects ψ(a) and ψ(b): the sequence of faces of G ∗ passed by such a polygonal arc would define an edge progression from a to b in G not using any edge of δG (A). So R consists of at least two connected regions. Then, obviously, the boundary of each region must contain a circuit. Hence F := {e∗ : e ∈ δG (A)} contains the edge set of a circuit C in G ∗ . We have {e∗ : e ∈ E(C)} ⊆ {e∗ : e ∈ F} = δG (A), and, by the first part, {e∗ : e ∈ E(C)} is a minimal cut in (G ∗ )∗ = G (cf. Proposition 2.42). We conclude that {e∗ : e ∈ E(C)} = δG (A). In particular, e∗ is a loop if and only if e is a bridge, and vice versa. For digraphs the above proof yields: Corollary 2.44. Let G be a connected planar digraph with some fixed planar embedding. The edge set of any circuit in G corresponds to a minimal directed cut in G ∗ , and vice versa. Another interesting consequence of Theorem 2.43 is: Corollary 2.45. Let G be a connected undirected graph with arbitrary planar embedding. Then G is bipartite if and only if G ∗ is Eulerian, and G is Eulerian if and only if G ∗ is bipartite. Proof: Observe that a connected graph is Eulerian if and only if every minimal cut has even cardinality. By Theorem 2.43, G is bipartite if G ∗ is Eulerian, and G is Eulerian if G ∗ is bipartite. By Proposition 2.42, the converse is also true. An abstract dual of G is a graph G 0 for which there is a bijection χ : E(G) → E(G 0 ) such that F is the edge set of a circuit iff χ(F) is a minimal cut in G 0 and vice versa. Theorem 2.43 shows that any planar dual is also an abstract dual. The converse is not true. However, Whitney [1933] proved that a graph has an abstract dual if and only if it is planar (Exercise 34). We shall return to this duality relation when dealing with matroids in Section 13.3.

Exercises

43

Exercises 1. Let G be a simple undirected graph on n vertices which is isomorphic to its complement. Show that n mod 4 ∈ {0, 1}. 2. Prove that every simple undirected graph G with |δ(v)| ≥ 21 |V (G)| for all v ∈ V (G) is Hamiltonian. Hint: Consider a longest path in G and the neighbours of its endpoints. (Dirac [1952]) 3. Prove that any simple undirected graph G with |E(G)| > |V (G)|−1 is con2 nected. 4. Let G be a simple undirected graph. Show that G or its complement is connected. 5. Prove that every simple undirected graph with more than one vertex contains two vertices that have the same degree. Prove that every tree (except a single vertex) contains at least two leaves. 6. Let G be a connected undirected graph, and let (V (G), F) be a forest in G. Prove that there is a spanning tree (V (G), T ) with F ⊆ T ⊆ E(G). 7. Let (V, F1 ) and (V, F2 ) be two forests with |F1 | < |F2 |. Prove that there exists an edge e ∈ F2 \ F1 such that (V, F1 ∪ {e}) is a forest. 8. Prove that any cut in an undirected graph is the disjoint union of minimal cuts. 9. Let G be an undirected graph, C a circuit and D a cut. Show that |E(C) ∩ D| is even. 10. Show that any undirected graph has a cut containing at least half of the edges. 11. Let (U, F) be a cross-free set system with |U | ≥ 2. Prove that F contains at most 4|U | − 4 distinct elements. 12. Let G be a connected undirected graph. Show that there exists an orientation G 0 of G and a spanning arborescence T of G 0 such that the set of fundamental circuits with respect to T is precisely the set of directed circuits in G 0 . Hint: Consider a DFS-tree. (Camion [1968], Crestin [1969]) 13. Describe a linear-time algorithm for the following problem: Given an adjacency list of a graph G, compute an adjacency list of the maximal simple subgraph of G. Do not assume that parallel edges appear consecutively in the input. 14. Given a graph G (directed or undirected), show that there is a linear-time algorithm to find a circuit or decide that none exists. 15. Let G be a connected undirected graph, s ∈ V (G) and T a DFS-tree resulting from running DFS on (G, s). s is called the root of T . x is a predecessor of y in T if x lies on the (unique) s-y-path in T . x is a direct predecessor of y if the edge {x, y} lies on the s-y-path in T . y is a (direct) successor of x if x is a (direct) predecessor of y. Note that with this definition each vertex is a successor (and a predecessor) of itself. Every vertex except s has exactly one direct predecessor. Prove: (a) For any edge {v, w} ∈ E(G), v is a predecessor or a successor of w in T . (b) A vertex v is an articulation vertex of G if and only if • either v = s and |δT (v)| > 1

44

2 Graphs

or v 6= s and there is a direct successor w of v such that no edge in G connects a proper predecessor of v (that is, excluding v) with a successor of w. ∗ 16. Use Exercise 15 to design a linear-time algorithm which finds the blocks of an undirected graph. It will be useful to compute numbers •

α(x) := min{ f (w) : w = x or {w, y} ∈ E(G)\T for some successor y of x}

17.

18.

19. 20. 21. ∗ 22.

recursively during the DFS. Here (R, T ) is the DFS-tree (with root s), and the f -values represent the order in which the vertices are added to R (see the G RAPH S CANNING A LGORITHM). If for some vertex x ∈ R \ {s} we have α(x) ≥ f (w), where w is the direct predecessor of x, then w must be either the root or an articulation vertex. Prove: (a) An undirected graph is 2-edge-connected if and only if it has at least two vertices and an ear-decomposition. (b) A digraph is strongly connected if and only if it has an ear-decomposition. (c) The edges of an undirected graph G with at least two vertices can be oriented such that the resulting digraph is strongly connected if and only if G is 2-edge-connected. (Robbins [1939]) A tournament is a digraph such that the underlying undirected graph is a (simple) complete graph. Prove that every tournament contains a Hamiltonian path (R´edei [1934]). Prove that every strongly connected tournament is Hamiltonian (Camion [1959]). Prove that if a connected undirected simple graph is Eulerian then its line graph is Hamiltonian. What about the converse? Prove that any connected bipartite graph has a unique bipartition. Prove that any non-bipartite undirected graph contains an odd circuit as an induced subgraph. Prove that a strongly connected digraph whose underlying undirected graph is non-bipartite contains a (directed) circuit of odd length. Let G be an undirected graph. A tree-decomposition of G is a pair (T, ϕ), where T is a tree and ϕ : V (T ) → 2V (G) satisfies the following conditions: • for each e ∈ E(G) there exists a t ∈ V (T ) with e ⊆ ϕ(t); • for each v ∈ V (G) the set {t ∈ V (T ) : v ∈ ϕ(t)} is connected in T . We say that the width of (T, ϕ) is maxt∈V (T ) |ϕ(t)| − 1. The tree-width of a graph G is the minimum width of a tree-decomposition of G. This notion is due to Robertson and Seymour [1986]. Show that the simple graphs of tree-width at most 1 are the forests. Moreover, prove that the following statements are equivalent for an undirected graph G: (a) G has tree-width at most 2; (b) G does not contain K 4 as a minor; (c) G can be obtained from an empty graph by successively adding bridges and doubling and subdividing edges. (Doubling an edge e = {v, w} ∈ E(G) means adding another edge with endpoints v and w; subdividing an edge

Exercises

23.

24. 25. 26.

27.

∗ 28.

29.

30.

45

e = {v, w} ∈ E(G) means adding a vertex x and replacing e by two edges {v, x}, {x, w}.) Note: Because of the construction in (c) such graphs are called series-parallel. Show that if a graph G has a planar embedding where the edges are embedded by arbitrary Jordan curves, then it also has a planar embedding with polygonal arcs only. Let G be a 2-connected graph with a planar embedding. Show that the set of circuits bounding the finite faces constitutes a cycle basis of G. Can you generalize Euler’s formula (Theorem 2.32) to disconnected graphs? Show that there are exactly five Platonic graphs (corresponding to the Platonic solids; cf. Exercise 11 of Chapter 4), i.e. 3-connected planar regular graphs whose faces are all bounded by the same number of edges. Hint: Use Euler’s formula (Theorem 2.32). Deduce from the proof of Kuratowski’s Theorem 2.39: (a) Every 3-connected simple planar graph has a planar embedding where each edge is embedded by a straight line and each face, except the outer face, is convex. (b) There is a polynomial-time algorithm for checking whether a given graph is planar. Given a graph G and an edge e = {v, w} ∈ E(G), we say that H results from . G by subdividing e if V (H ) = V (G) ∪ {x} and E(H ) = (E(G) \ {e}) ∪ {{v, x}, {x, w}}. A graph resulting from G by successively subdividing edges is called a subdivision of G. (a) Trivially, if H contains a subdivision of G then G is a minor of H . Show that the converse is not true. (b) Prove that a graph containing a K 3,3 or K 5 minor also contains a subdivision of K 3,3 or K 5 . Hint: Consider what happens when contracting one edge. (c) Conclude that a graph is planar if and only if no subgraph is a subdivision of K 3,3 or K 5 . (Kuratowski [1930]) Prove that each of the following statements implies the other: (a) For every infinite sequence of graphs G 1 , G 2 , . . . there are two indices i < j such that G i is a minor of G j . (b) Let G be a class of graphs such that for each G ∈ G and each minor H of G we have H ∈ G (i.e. membership in G is a hereditary graph property). Then there exists a finite set X of graphs such that G consists of all graphs that do not contain any element of X as a minor. Note: The statements have been proved by Robertson and Seymour [2004]; they are a main result of their series of papers on graph minors. Theorem 2.39 and Exercise 22 give examples of forbidden minor characterizations as in (b). Let G be a planar graph with an embedding 8, and let C be a circuit of G bounding some face of 8. Prove that then there is an embedding 80 of G such that C bounds the outer face.

46

2 Graphs

31. (a) Let G be disconnected with an arbitrary planar embedding, and let G ∗ be the planar dual with a standard embedding. Prove that (G ∗ )∗ arises from G by successively applying the following operation, until the graph is connected: Choose two vertices x and y which belong to different connected components and which are adjacent to the same face; contract {x, y}. (b) Generalize Corollary 2.45 to arbitrary planar graphs. Hint: Use (a) and Theorem 2.26. 32. Let G be a connected digraph with a fixed planar embedding, and let G ∗ be the planar dual with a standard embedding. How are G and (G ∗ )∗ related? 33. Prove that if a planar digraph is acyclic (strongly connected), then its planar dual is strongly connected (acyclic). What about the converse? 34. (a) Show that if G has an abstract dual and H is a minor of G then H also has an abstract dual. ∗ (b) Show that neither K 5 nor K 3,3 has an abstract dual. (c) Conclude that a graph is planar if and only if it has an abstract dual. (Whitney [1933])

References General Literature: Berge, C. [1985]: Graphs. Second Edition. Elsevier, Amsterdam 1985 Bollob´as, B. [1998]: Modern Graph Theory. Springer, New York 1998 Bondy, J.A. [1995]: Basic graph theory: paths and circuits. In: Handbook of Combinatorics; Vol. 1 (R.L. Graham, M. Gr¨otschel, L. Lov´asz, eds.), Elsevier, Amsterdam 1995 Bondy, J.A., and Murty, U.S.R. [2008]: Graph Theory. Springer, New York 2008 Diestel, R. [2005]: Graph Theory. Third Edition. Springer, New York 2005 Wilson, R.J. [1996]: Introduction to Graph Theory. Fourth Edition. Addison-Wesley, Reading 1996

Cited References: Aoshima, K., and Iri, M. [1977]: Comments on F. Hadlock’s paper: finding a maximum cut of a Planar graph in polynomial time. SIAM Journal on Computing 6 (1977), 86–87 Camion, P. [1959]: Chemins et circuits hamiltoniens des graphes complets. Comptes Rendus Hebdomadaires des S´eances de l’Acad´emie des Sciences (Paris) 249 (1959), 2151–2152 Camion, P. [1968]: Modulaires unimodulaires. Journal of Combinatorial Theory A 4 (1968), 301–362 Dirac, G.A. [1952]: Some theorems on abstract graphs. Proceedings of the London Mathematical Society 2 (1952), 69–81 Edmonds, J., and Giles, R. [1977]: A min-max relation for submodular functions on graphs. In: Studies in Integer Programming; Annals of Discrete Mathematics 1 (P.L. Hammer, E.L. Johnson, B.H. Korte, G.L. Nemhauser, eds.), North-Holland, Amsterdam 1977, pp. 185–204 Euler, L. [1736]: Solutio Problematis ad Geometriam Situs Pertinentis. Commentarii Academiae Petropolitanae 8 (1736), 128–140

References

47

Euler, L. [1758]: Demonstratio nonnullarum insignium proprietatum quibus solida hedris planis inclusa sunt praedita. Novi Commentarii Academiae Petropolitanae 4 (1758), 140–160 ¨ Hierholzer, C. [1873]: Uber die M¨oglichkeit, einen Linienzug ohne Wiederholung und ohne Unterbrechung zu umfahren. Mathematische Annalen 6 (1873), 30–32 Hopcroft, J.E., and Tarjan, R.E. [1974]: Efficient planarity testing. Journal of the ACM 21 (1974), 549–568 Kahn, A.B. [1962]: Topological sorting of large networks. Communications of the ACM 5 (1962), 558–562 Knuth, D.E. [1968]: The Art of Computer Programming; Vol. 1. Fundamental Algorithms. Addison-Wesley, Reading 1968 (third edition: 1997) ¨ K¨onig, D. [1916]: Uber Graphen und Ihre Anwendung auf Determinantentheorie und Mengenlehre. Mathematische Annalen 77 (1916), 453–465 K¨onig, D. [1936]: Theorie der endlichen und unendlichen Graphen. Chelsea Publishing Co., Leipzig 1936, reprint New York 1950 Kuratowski, K. [1930]: Sur le probl`eme des courbes gauches en topologie. Fundamenta Mathematicae 15 (1930), 271–283 ´ ements de G´eom´etrie. Firmin Didot, Paris 1794 Legendre, A.M. [1794]: El´ Minty, G.J. [1960]: Monotone networks. Proceedings of the Royal Society of London A 257 (1960), 194–212 Moore, E.F. [1959]: The shortest path through a maze. Proceedings of the International Symposium on the Theory of Switching; Part II. Harvard University Press 1959, pp. 285–292 R´edei, L. [1934]: Ein kombinatorischer Satz. Acta Litt. Szeged 7 (1934), 39–43 Robbins, H.E. [1939]: A theorem on graphs with an application to a problem of traffic control. American Mathematical Monthly 46 (1939), 281–283 Robertson, N., and Seymour, P.D. [1986]: Graph minors II: algorithmic aspects of tree-width. Journal of Algorithms 7 (1986), 309–322 Robertson, N., and Seymour, P.D. [2004]: Graph minors XX: Wagner’s conjecture. Journal of Combinatorial Theory B 92 (2004), 325–357 Tarjan, R.E. [1972]: Depth first search and linear graph algorithms. SIAM Journal on Computing 1 (1972), 146–160 Thomassen, C. [1980]: Planarity and duality of finite and infinite graphs. Journal of Combinatorial Theory B 29 (1980), 244–271 Thomassen, C. [1981]: Kuratowski’s theorem. Journal of Graph Theory 5 (1981), 225–241 Tutte, W.T. [1961]: A theory of 3-connected graphs. Konink. Nederl. Akad. Wetensch. Proc. A 64 (1961), 441–455 ¨ Wagner, K. [1937]: Uber eine Eigenschaft der ebenen Komplexe. Mathematische Annalen 114 (1937), 570–590 Whitney, H. [1932]: Non-separable and planar graphs. Transactions of the American Mathematical Society 34 (1932), 339–362 Whitney, H. [1933]: Planar graphs. Fundamenta Mathematicae 21 (1933), 73–84

3 Linear Programming

In this chapter we review the most important facts about Linear Programming. Although this chapter is self-contained, it cannot be considered to be a comprehensive treatment of the field. The reader unfamiliar with Linear Programming is referred to the textbooks mentioned at the end of this chapter. The general problem reads as follows:

L INEAR P ROGRAMMING Instance:

A matrix A ∈ Rm×n and column vectors b ∈ Rm , c ∈ Rn .

Task:

Find a column vector x ∈ Rn such that Ax ≤ b and c> x is maximum, decide that {x ∈ Rn : Ax ≤ b} is empty, or decide that for all α ∈ R there is an x ∈ Rn with Ax ≤ b and c> x > α.

Here c> x denotes the scalar product of the vectors. The notion x ≤ y for vectors x and y (of equal size) means that the inequality holds in each component. If no sizes are specified, the matrices and vectors are always assumed to be compatible in size. We often omit indicating the transposition of column vectors and write e.g. cx for the scalar product. By 0 we denote the number zero as well as all-zero vectors and all-zero matrices (the order will always be clear from the context). A linear program (LP) is an instance of the above problem. We often write a linear program as max{cx : Ax ≤ b}. A feasible solution of an LP max{cx : Ax ≤ b} is a vector x with Ax ≤ b. A feasible solution attaining the maximum is called an optimum solution. As the problem formulation indicates, there are two possibilities when an LP has no solution: The problem can be infeasible (i.e. P := {x ∈ Rn : Ax ≤ b} = ∅) or unbounded (i.e. for all α ∈ R there is an x ∈ P with cx > α). If an LP is neither infeasible nor unbounded it has an optimum solution: Proposition 3.1. Let P = {x ∈ Rn : Ax ≤ b} 6= ∅ and c ∈ Rn with δ := sup{c> x : x ∈ P} < ∞. Then there exists a vector z ∈ P with c> z = δ. Proof: Let U be a matrix whose columns are an orthonormal basis of the kernel of A, i.e. U > U = I , AU = 0, and rank(A0 ) = n where A0 := UA> . Let b0 := b0 . We show that for every y ∈ P there exists a subsystem A00 x ≤ b00 of A0 x ≤ b0 such that A00 is nonsingular, y 0 := (A00 )−1 b00 ∈ P, and c> y 0 ≥ c> y. As there

50

3 Linear Programming

are only finitely many such subsystems, one of these y 0 attains the maximum (c> y 0 = δ), and the assertion follows. So let y ∈ P, and denote by k(y) the rank of A00 for the maximal subsystem 00 A x ≤ b00 of A0 x ≤ b0 with A00 y = b00 . Suppose that k(y) < n. We show how to find a y 0 ∈ P with c> y 0 ≥ c> y and k(y 0 ) > k(y). After at most n steps we have a vector y 0 with k(y 0 ) = n as required. If U > y 6= 0, we set y 0 := y − UU > y. Since y + λUU > c ∈ P for all λ ∈ R we have sup{c> (y + λUU > c) : λ ∈ R} ≤ δ < ∞ and hence c> U = 0 and c> y 0 = c> y. Moreover, Ay 0 = Ay − AUU > y = Ay and U > y 0 = U > y − U > UU > y = 0. Now suppose that U > y = 0. Let v 6= 0 with nA00 v = 0. Denote o by βi −ai y ai x ≤ βi the i-th row of Ax ≤ b. Let µ := min ai v : ai v > 0 and o n iy : a v < 0 , where min ∅ = ∞ and max ∅ = −∞. We have κ := max βi a−a i iv κ ≤ 0 ≤ µ, and at least one of κ and µ is finite. For λ ∈ R with κ ≤ λ ≤ µ we have A00 (y + λv) = A00 y + λA00 v = A00 y = b00 and A(y + λv) = Ay + λAv ≤ b, i.e. y + λv ∈ P. Thus, as sup{c> x : x ∈ P} < ∞, we have µ < ∞ if c> v > 0 and κ > −∞ if c> v < 0. Moreover, if c> v ≥ 0 and µ < ∞, we have ai (y + µv) = βi for some i. Analogously, if c> v ≤ 0 and κ > −∞, we have ai (y + κv) = βi for some i. Thus in each case we have found a vector y 0 ∈ P with c> y 0 ≥ c> y and 0 k(y ) ≥ k(y) + 1. This justifies the notation max{c> x : Ax ≤ b} instead of sup{c> x : Ax ≤ b}. Many combinatorial optimization problems can be formulated as LPs. To do this, we encode the feasible solutions as vectors in Rn for some n. In Section 3.5 we show that one can optimize a linear objective function over a finite set S of vectors by solving a linear program. Although the feasible set of this LP contains not only the vectors in S but also all their convex combinations, one can show that among the optimum solutions there is always an element of S. In Section 3.1 we compile some terminology and basic facts about polyhedra, the sets P = {x ∈ Rn : Ax ≤ b} of feasible solutions of LPs. In Sections 3.2 and 3.3 we present the S IMPLEX A LGORITHM, which we also use to derive the Duality Theorem and related results (Section 3.4). LP duality is a most important concept which explicitly or implicitly appears in almost all areas of combinatorial optimization; we shall often refer to the results in Sections 3.4 and 3.5.

3.1 Polyhedra Linear Programming deals with maximizing or minimizing a linear objective function of finitely many variables subject to finitely many linear inequalities. So the set of feasible solutions is the intersection of finitely many halfspaces. Such a set is called a polyhedron:

3.1 Polyhedra

51

Definition 3.2. A polyhedron in Rn is a set of type P = {x ∈ Rn : Ax ≤ b} for some matrix A ∈ Rm×n and some vector b ∈ Rm . If A and b are rational, then P is a rational polyhedron. A bounded polyhedron is also called a polytope. We denote by rank(A) the rank of a matrix A. The dimension dim X of a nonempty set X ⊆ Rn is defined to be n − max{rank(A) : A is an n × n-matrix with Ax = Ay for all x, y ∈ X }. A polyhedron P ⊆ Rn is called full-dimensional if dim P = n. Equivalently, a polyhedron is full-dimensional if and only if there is a point in its interior. For most of this chapter it makes no difference whether we are in the rational or real space. We need the following standard terminology: Definition 3.3. Let P := {x : Ax ≤ b} be a nonempty polyhedron. If c is a nonzero vector for which δ := max{cx : x ∈ P} is finite, then {x : cx = δ} is called a supporting hyperplane of P. A face of P is P itself or the intersection of P with a supporting hyperplane of P. A point x for which {x} is a face is called a vertex of P, and also a basic solution of the system Ax ≤ b. Proposition 3.4. Let P = {x : Ax ≤ b} be a polyhedron and F ⊆ P. Then the following statements are equivalent: (a) F is a face of P. (b) There exists a vector c such that δ := max{cx : x ∈ P} is finite and F = {x ∈ P : cx = δ}. (c) F = {x ∈ P : A0 x = b0 } 6= ∅ for some subsystem A0 x ≤ b0 of Ax ≤ b. Proof: (a) and (b) are obviously equivalent. (c)⇒(b): If F = {x ∈ P : A0 x = b0 } is nonempty, let c be the sum of the rows of A0 , and let δ be the sum of the components of b0 . Then obviously cx ≤ δ for all x ∈ P and F = {x ∈ P : cx = δ}. (b)⇒(c): Assume that c is a vector, δ := max{cx : x ∈ P} is finite and F = {x ∈ P : cx = δ}. Let A0 x ≤ b0 be the maximal subsystem of Ax ≤ b such that A0 x = b0 for all x ∈ F. Let A00 x ≤ b00 be the rest of the system Ax ≤ b. We first observe that for each inequality ai00 x ≤ βi00 of A00 x ≤ b00 (i = 1, . . . , k) Pk there is a point xi ∈ F such that ai00 xi < βi00 . Let x ∗ := k1 i=1 xi be the center of gravity of these points (if k = 0, we can choose an arbitrary x ∗ ∈ F); we have x ∗ ∈ F and ai00 x ∗ < βi00 for all i. We have to prove that A0 y = b0 cannot hold for any y ∈ P \ F. So let y ∈ P \ F. We have cy < δ. Now consider z := x ∗ + (x ∗ − y) for some small > 0; in β 00 −ai00 x ∗ particular let be smaller than a 00i (x ∗ −y) for all i ∈ {1, . . . , k} with ai00 x ∗ > ai00 y. i We have cz > δ and thus z ∈ / P. So there is an inequality ax ≤ β of Ax ≤ b such that az > β. Thus ax ∗ > ay. The inequality ax ≤ β cannot belong to A00 x ≤ b00 , β−ax ∗ ∗ since otherwise we have az = ax ∗ + a(x ∗ − y) < ax ∗ + a(x ∗ −y) a(x − y) = β

52

3 Linear Programming

(by the choice of ). Hence the inequality ax ≤ β belongs to A0 x ≤ b0 . Since ay = a(x ∗ + 1 (x ∗ − z)) < β, this completes the proof. As a trivial but important corollary we remark: Corollary 3.5. If max{cx : x ∈ P} is bounded for a nonempty polyhedron P and a vector c, then the set of points where the maximum is attained is a face of P. The relation “is a face of ” is transitive: Corollary 3.6. Let P be a polyhedron and F a face of P. Then F is again a polyhedron. Furthermore, a set F 0 ⊆ F is a face of P if and only if it is a face of F. The maximal faces distinct from P are particularly important: Definition 3.7. Let P be a polyhedron. A facet of P is a maximal face distinct from P. An inequality cx ≤ δ is facet-defining for P if cx ≤ δ for all x ∈ P and {x ∈ P : cx = δ} is a facet of P. Proposition 3.8. Let P ⊆ {x ∈ Rn : Ax = b} be a nonempty polyhedron of dimension n − rank(A). Let A0 x ≤ b0 be a minimal inequality system such that P = {x : Ax = b, A0 x ≤ b0 }. Then each inequality of A0 x ≤ b0 is facet-defining for P, and each facet of P is defined by an inequality of A0 x ≤ b0 . Proof: If P = {x ∈ Rn : Ax = b}, then there are no facets and the statement is trivial. So let A0 x ≤ b0 be a minimal inequality system with P = {x : Ax = b, A0 x ≤ b0 }, let a 0 x ≤ β 0 be one of its inequalities and A00 x ≤ b00 be the rest of the system A0 x ≤ b0 . Let y be a vector with Ay = b, A00 y ≤ b00 and a 0 y > β 0 (such a vector y exists as the inequality a 0 x ≤ β 0 is not redundant). Let x ∈ P such that a 0 x < β 0 (such a vector must exist because dim P = n − rank(A)). 0 −a 0 x β 0 −a 0 x 0 0 Consider z := x + aβ0 y−a 0 x (y−x). We have a z = β and, since 0 < a 0 y−a 0 x < 1, z ∈ P. Therefore F := {x ∈ P : a 0 x = β 0 } 6= 0 and F 6= P (as x ∈ P \ F). Thus F is a facet of P. By Proposition 3.4 each facet is defined by an inequality of A0 x ≤ b0 . The other important class of faces (beside facets) are minimal faces (i.e. faces not containing any other face). Here we have: Proposition 3.9. (Hoffman and Kruskal [1956]) Let P = {x : Ax ≤ b} be a polyhedron. A nonempty subset F ⊆ P is a minimal face of P if and only if F = {x : A0 x = b0 } for some subsystem A0 x ≤ b0 of Ax ≤ b. Proof: If F is a minimal face of P, by Proposition 3.4 there is a subsystem A0 x ≤ b0 of Ax ≤ b such that F = {x ∈ P : A0 x = b0 }. We choose A0 x ≤ b0 maximal. Let A00 x ≤ b00 be a minimal subsystem of Ax ≤ b such that F = {x : A0 x = b0 , A00 x ≤ b00 }. We claim that A00 x ≤ b00 does not contain any inequality.

3.1 Polyhedra

53

Suppose, on the contrary, that a 00 x ≤ β 00 is an inequality of A00 x ≤ b00 . Since it is not redundant for the description of F, Proposition 3.8 implies that F 0 := {x : A0 x = b0 , A00 x ≤ b00 , a 00 x = β 00 } is a facet of F. By Corollary 3.6 F 0 is also a face of P, contradicting the assumption that F is a minimal face of P. Now let ∅ 6= F = {x : A0 x = b0 } ⊆ P for some subsystem A0 x ≤ b0 of Ax ≤ b. Obviously F has no faces except itself. By Proposition 3.4, F is a face of P. It follows by Corollary 3.6 that F is a minimal face of P. Corollary 3.5 and Proposition 3.9 imply that L INEAR P ROGRAMMING can be solved in finite time by solving the linear equation system A0 x = b0 for each subsystem A0 x ≤ b0 of Ax ≤ b. A more intelligent way is the S IMPLEX A LGORITHM which is described in the next section. Another consequence of Proposition 3.9 is: Corollary 3.10. Let P = {x ∈ Rn : Ax ≤ b} be a polyhedron. Then all minimal faces of P have dimension n−rank(A). The minimal faces of polytopes are vertices. This is why polyhedra {x ∈ Rn : Ax ≤ b} with rank(A) = n are called pointed: their minimal faces are points. Let us close this section with some remarks on polyhedral cones. Definition 3.11. A (convex) cone is a set C ⊆ Rn for which x, y ∈ C and λ, µ ≥ 0 implies λx + µy ∈ C. A cone C is said to be generated by x1 , . . . , xk if x1 ,P . . . , xk ∈ C and for any x ∈ C there are numbers λ1 , . . . , λk ≥ 0 with k λi xi . A cone is called finitely generated if some finite set of vectors x = i=1 generates it. A polyhedral cone is a polyhedron of type {x : Ax ≤ 0}. It is immediately clear that polyhedral cones are indeed cones. We shall now show that polyhedral cones are finitely generated. I always denotes an identity matrix. Lemma 3.12. (Minkowski [1896]) Let C = {x ∈ Rn : Ax ≤ 0} be a polyhedral cone. Then C is generated by a subset of the set of solutions to the systems M y = b0 , where M consists of n linearly independent rows of AI and b0 = ±e j for some unit vector e j . 0 Proof: Let A be an m × n-matrix. Consider the systems M y = b where M consists of n linearly independent rows of AI and b0 = ±e j for some unit vector e j . Let y1 , . . . , yt be those solutions of these equality systems that belong to C. We claim that C is generated by y1 , . . . , yt . First suppose C = {x : Ax = 0}, i.e. C is a linear subspace. Write C = {x : A0 x = 0} where A0 consists of a maximal set oflinearly independent rows of A. 0 0 Let I consist of some rows of I such that AI 0 is a nonsingular square matrix. Then C is generated by the solutions of

54

3 Linear Programming

! ! A0 0 , x= I0 b

for b = ±e j , j = 1, . . . , dim C.

For the general case we use induction on the dimension of C. If C is not a linear subspace, choose a row a of A and a submatrix A0 of A such that the rows 0 A of a are linearly independent and {x : A0 x = 0, ax ≤ 0} ⊆ C. By construction there is an index s ∈ {1, . . . , t} such that A0 ys = 0 and ays = −1. Now let o Let a1 , . . . , am be the rows of A and n an arbitrary z ∈ C be given. µ := min aaiiyzs : i = 1, . . . , m, ai ys < 0 . We have µ ≥ 0. Let k be an index where

the minimum is attained. Consider z 0 := z − µys . By the definition of µ we have a j z 0 = a j z − aakkyzs a j ys for j = 1, . . . , m, and hence z 0 ∈ C 0 := {x ∈ C : ak x = 0}. C 0 is a cone whose dimension is one less than that of C (because ak yP s < 0 and t λi yi ys ∈ C). By induction, C 0 is generated by a subset of y1 , . . . , yt , so z 0 = i=1 0 for some λ1 , . . . , λt ≥ 0. By setting λs := λs +µ (observe that µ ≥ 0) and λi0 := λi Pt (i 6= s), we obtain z = z 0 + µys = i=1 λi0 yi . Thus any polyhedral cone is finitely generated. We shall show the converse at the end of Section 3.4.

3.2 The Simplex Algorithm The oldest and best-known algorithm for L INEAR P ROGRAMMING is Dantzig’s [1951] simplex method. We first assume that the polyhedron has a vertex, and that some vertex is given as input. Later we shall show how general LPs can be solved with this method. For a set J of row indices we write A J for the submatrix of A consisting of the rows in J only, and b J for the subvector of b consisting of the components with indices in J . We abbreviate ai := A{i} and βi := b{i} .

S IMPLEX A LGORITHM Input: Output:

A matrix A ∈ Rm×n and column vectors b ∈ Rm , c ∈ Rn . A vertex x of P := {x ∈ Rn : Ax ≤ b}. A vertex x of P attaining max{cx : x ∈ P} or a vector w ∈ Rn with Aw ≤ 0 and cw > 0 (i.e. the LP is unbounded).

1

Choose a set of n row indices J such that A J is nonsingular and A J x = b J .

2

Compute c (A J )−1 and add zeros in order to obtain a vector y with c = y A such that all entries of y outside J are zero. If y ≥ 0 then stop. Return x and y. Choose the minimum index i with yi < 0. Let w be the column of −(A J )−1 with index i, so A J \{i} w = 0 and ai w = −1. If Aw ≤ 0 then stop. Return w.

3

3.2 The Simplex Algorithm

4

5

55

βj − aj x : j ∈ {1, . . . , m}, a j w > 0 , aj w and let j be the smallest row index attaining this minimum. Set J := (J \ {i}) ∪ { j} and x := x + λw. Go to . 2 Let λ := min

Step 1 relies on Proposition 3.9 and can be implemented with G AUSSIAN E LIMINATION (Section 4.3). The selection rules for i and j in 3 and 4 (often called pivot rule) are due to Bland [1977]. If one just chose an arbitrary i with yi < 0 and an arbitrary j attaining the minimum in 4 the algorithm would run into cyclic repetitions for some instances. Bland’s pivot rule is not the only one that avoids cycling; another one (the so-called lexicographic rule) was proved to avoid cycling already by Dantzig, Orden and Wolfe [1955]. Before proving the correctness of the S IMPLEX A LGORITHM, let us make the following observation (sometimes known as “weak duality”): Proposition 3.13. Let x and y be feasible solutions of the LPs and

(3.1)

min{yb : y > A = c> , y ≥ 0},

max{cx : Ax ≤ b}

(3.2)

respectively. Then cx ≤ yb. Proof: cx = (y A)x = y(Ax) ≤ yb.

Theorem 3.14. (Dantzig [1951], Dantzig, Orden and Wolfe [1955], Bland [1977]) The S IMPLEX A LGORITHM terminates after at most mn iterations. If it returns x and y in , 2 these vectors are optimum solutions of the LPs (3.1) and (3.2), respectively, with cx = yb. If the algorithm returns w in 3 then cw > 0 and the LP (3.1) is unbounded. Proof: We first prove that the following conditions hold at any stage of the algorithm: (a) (b) (c) (d) (e)

x ∈ P; AJ x = bJ ; A J is nonsingular; cw > 0; λ ≥ 0.

(a) and (b) hold initially. 2 and 3 guarantee cw = y Aw = −yi > 0. By , 4 x ∈ P implies λ ≥ 0. (c) follows from the fact that A J \{i} w = 0 and a j w > 0. It remains to show that 5 preserves (a) and (b). We show that if x ∈ P, then also x + λw ∈ P. For a row index k we have two cases: If ak w ≤ 0 then (using λ ≥ 0) ak (x + λw) ≤ ak x ≤ βk . Otherwise kx kx and hence ak (x + λw) ≤ ak x + ak w βka−a = βk . (Indeed, λ is chosen λ ≤ βka−a kw kw in 4 to be the largest number such that x + λw ∈ P.)

56

3 Linear Programming β j −a j x a j w , so β j −a j x aj w aj w = βj .

To show (b), note that after 4 we have A J \{i} w = 0 and λ =

A J \{i} (x + λw) = A J \{i} x = b J \{i} and a j (x + λw) = a j x + Therefore after , 5 A J x = b J holds again. So we indeed have (a)–(e) at any stage. If the algorithm returns x and y in

, 2 x and y are feasible solutions of (3.1) and (3.2), respectively. x is a vertex of P by (a), (b) and (c). Moreover, cx = y Ax = yb since the components of y are zero outside J . This proves the optimality of x and y by Proposition 3.13. If the algorithm stops in , 3 the LP (3.1) is indeed unbounded because in this case x + µw ∈ P for all µ ≥ 0, and cw > 0 by (d). We finally show that the algorithm terminates. Let J (k) and x (k) be the set J and the vector x in iteration k of theS IMPLEX A LGORITHM, respectively. If the algorithm did not terminate after mn iterations, there are iterations k < l with J (k) = J (l) . By (b) and (c), x (k) = x (l) . By (d) and (e), cx never decreases, and it strictly increases if λ > 0. Hence λ is zero in all the iterations k, k + 1, . . . , l − 1, and x (k) = x (k+1) = · · · = x (l) . Let h be the highest index leaving J in one of the iterations k, . . . , l − 1, say in iteration p. Index h must also have been added to J in some iteration q ∈ {k, . . . , l − 1}. Now let y 0 be the vector y at iteration p, and let w0 be the vector w at iteration q. We have y 0 Aw 0 = cw 0 > 0. So let r be an index for which yr0 ar w 0 > 0. Since yr0 6= 0, index r belongs to J ( p) . If r > h, index r would also belong to J (q) and J (q+1) , implying ar w 0 = 0. So r ≤ h. But by the choice of i in iteration p we have yr0 < 0 iff r = h, and by the choice of j in iteration q we have ar w 0 > 0 iff r = h (recall that λ = 0 and ar x (q) = ar x ( p) = βr as r ∈ J ( p) ). This is a contradiction. Klee and Minty [1972] and Avis and Chv´atal [1978] found examples where the S IMPLEX A LGORITHM (with Bland’s rule) needs 2n iterations on LPs with n variables and 2n constraints, proving that it is not a polynomial-time algorithm. It is not known whether there is a pivot rule that leads to a polynomial-time algorithm. However, Borgwardt [1982] showed that the average running time (for random instances in a certain natural probabilistic model) can be bounded by a polynomial. Spielman and Teng [2004] introduced a so-called smoothed analysis: for each input they consider the expected running time with respect to small random perturbations of the input. The maximum of all these expectations is polynomially bounded. Kelner and Spielman [2006] proposed a randomized polynomial-time algorithm for L INEAR P ROGRAMMING that is similar to the S IMPLEX A LGORITHM. The S IMPLEX A LGORITHM is also quite fast in practice if implemented skilfully; see Section 3.3. We now show how to solve general linear programs with the S IMPLEX A L GORITHM . More precisely, we show how to find an initial vertex. Since there are polyhedra that do not have vertices at all, we put a given LP into a different form first. Let max{cx : Ax ≤ b} be an LP. We substitute x by y − z and write it equivalently in the form

3.3 Implementation of the Simplex Algorithm

( max

y c −c z

! :

A −A

y z

!

57

) ≤ b, y, z ≥ 0

.

So w.l.o.g. we assume that our LP has the form max{cx : A0 x ≤ b0 , A00 x ≤ b00 , x ≥ 0}

(3.3)

with b0 ≥ 0 and b00 < 0. We first run the S IMPLEX A LGORITHM on the instance (3.4) min{(1lA00 )x + 1ly : A0 x ≤ b0 , A00 x + y ≥ b00 , x, y ≥ 0}, where 1l denotes a vector whose entries are all 1. Since xy = 0 defines a vertex, this is possible. The LP is obviously not unbounded since the minimum must be x 00 x is an optimum solution at least 1lb00 . For any feasible solution x of (3.3), b00 −A of (3.4) of value 1lb00 . Hence if the minimum of (3.4) is greater than 1lb00 , then (3.3) is infeasible. In the contrary case, let xy be an optimum vertex of (3.4) of value 1lb00 . We claim that x is a vertex of the polyhedron defined by (3.3). To see this, first observe that A00 x + y = b00 . Let n and m be the dimensions of x and y, respectively; then by Proposition 3.9 there is a set S of n + m inequalities of (3.4) satisfied with equality, such that the submatrix corresponding to these n + m inequalities is nonsingular. Let S 0 be the inequalities of A0 x ≤ b0 and of x ≥ 0 that belong to S. Let S 00 consist of those inequalities of A00 x ≤ b00 for which the corresponding inequalities of A00 x + y ≥ b00 and y ≥ 0 both belong to S. Obviously |S 0 ∪ S 00 | ≥ |S| − m = n, and the inequalities of S 0 ∪ S 00 are linearly independent and satisfied by x with equality. Hence x satisfies n linearly independent inequalities of (3.3) with equality; thus x is indeed a vertex. Therefore we can start the S IMPLEX A LGORITHM with (3.3) and x.

3.3 Implementation of the Simplex Algorithm The previous description of the S IMPLEX A LGORITHM is simple but not suitable for an efficient implementation. As we will see, it is not necessary to solve a linear equation system in each iteration. To motivate the main idea, we start with a proposition (which is actually not needed later): for LPs of the form max{cx : Ax = b, x ≥ 0}, vertices can be represented not only by subsets of rows but also by subsets of columns. For a matrix A and a set J of column indices we denote by A J the submatrix consisting of the columns in J only. Consequently, A IJ denotes the submatrix of A with rows in I and columns in J . Sometimes the order of the rows and columns is important: if J = ( j1 , . . . , jk ) is a vector of row (column) indices, we denote by A J (A J ) the matrix whose i-th row (column) is the ji -th row (column) of A (i = 1, . . . , k).

58

3 Linear Programming

Proposition 3.15. Let P := {x : Ax = b, x ≥ 0}, where A is a matrix and b is a vector. Then x is a vertex of P if and only if x ∈ P and the columns of A corresponding to positive entries of x are linearly independent. 0 0 0 Proof: Let A be an m × n-matrix. Let X := −I A I and b := b . Let N := {1, . . . , n} and M := {n +1, . . . , n +m}. For an index set J ⊆ N ∪ M with |J | = n N ∩ J¯ is nonsingular iff X J¯ let J¯ := (N ∪ M) \ J . Then X JN is nonsingular iff X M∩J M is nonsingular. If x is a vertex of P, then – by Proposition 3.9 – there exists a set J ⊆ N ∪ M such that |J | = n, X JN is nonsingular, and X JN x = b0J . Then the components of J¯ is nonsingular, and hence the x corresponding to N ∩ J are zero. Moreover, X M ¯ columns of A N ∩ J are linearly independent. Conversely, let x ∈ P, and let the set of columns of A corresponding to positive entries of x be linearly independent. By adding suitable unit column vectors to B with x = 0 for i ∈ N \ B. these columns we obtain a nonsingular submatrix X M i N N 0 Then X B¯ is nonsingular and X B¯ x = b B¯ . Hence, by Proposition 3.9, x is a vertex of P. Corollary 3.16. Let xy ∈ P := xy : Ax + y = b, x ≥ 0, y ≥ 0 . Then xy is a vertex of P if and only if the columns of (A I ) corresponding to positive components of xy are linearlyindependent. Moreover, x is a vertex of {x : Ax ≤ x b, x ≥ 0} if and only if b−Ax is a vertex of P. We will now analyze the behaviour of the S IMPLEX A LGORITHM when applied to an LP of the form max{cx : Ax ≤ b, x ≥ 0}. 0 0 Theorem 3.17. Let A ∈ Rm×n , b ∈ Rm , and c ∈ Rn . Let A0 := −I A , b := b and c¯ := (c> , 0). Let B ∈ {1, . . . , n + m}m such that (A I ) B is nonsingular. Let J ⊆ {1, . . . , n + m} be the set of the remaining n indices. Let Q B := ((A I ) B )−1 . Then: (a) A0J is nonsingular. (b) (b0 − A0 x) J = 0 and (b0 − A0 x) B = Q B b and c> x = c¯ B Q B b, where x := (A0J )−1 b0J . (c) Let y be the vector with y B = 0 and y > A0 = c> . Then y > = c¯ B Q B (A I ) − c. ¯ (d) Let i ∈ J . Let w be the vector with Ai0 w = −1 and A0J \{i} w = 0. Then A0B w = Q B (A I )i . (e) Define ! Q B (A I ) QBb TB := . c¯ B Q B (A I ) − c¯ c> x Given B and TB , we can compute B 0 and TB 0 in O(m(n + m)) time, where B 0 arises from B by replacing j by i, and i and j are given as in – 2 4 of the S IMPLEX A LGORITHM (applied to A0 , b0 , c, and index set J ).

3.3 Implementation of the Simplex Algorithm

59

TB is called the simplex tableau with respect to the basis B. N \J

Proof: (a): Let N := {1, . . . , n}. As (A I ) B is nonsingular, also (A0 ) J \N is nonsingular, and thus A0J is nonsingular. (b): The first statement follows directly from A0J x = b0J . Then b = Ax + ¯ 0 − A0 x) = I (b − Ax) = (A I )(b0 − A0 x) = (A I ) B (b0 − A0 x) B and c> x = c(b B 0 0 B c¯ (b − A x) B = c¯ Q B b. ¯ B = c¯ B Q B (A I ) B − c¯ B = 0 and (c): This follows from (c¯ B Q B (A I ) − c) B 0 B 0 > ¯ = c¯ Q B (A I )A − c (−I ) = c> . (c¯ Q B (A I ) − c)A (d): This follows from 0 = (A I )A0 w = (A I ) B (A0B w) + (A I ) J \{i} (A0J \{i} w) + (A I )i (Ai0 w) = (A I ) B (A0B w) − (A I )i . (e): By (c), y as in 2 of the S IMPLEX A LGORITHM is given by the last row of TB . If y ≥ 0, we stop (x and y are optimal). Otherwise i is the first index with yi < 0, found in O(n + m) time. If the i-th column of TB has no positive entry, we stop (the LP is unbounded, and w is given by (d)). Otherwise, by (b) and (d), we have that λ in 4 of the S IMPLEX A LGORITHM is given by (Q B b) j i : j ∈ {1, . . . , m}, (Q (A I ) ) > 0 , λ = min B j (Q B (A I )i ) j and among the indices attaining this minimum, j is the one for which the j-th component of B is minimum. So we can compute j in O(m) time by considering the i-th and the last column of TB . This yields B 0 . We can compute the updated tableau TB 0 as follows: Divide the j-th row by the entry in row j and column i. Then add a suitable multiple of the j-th row to all other rows, such that the i-th column has zeros only outside row j. Note that these row operations do not destroy the property that the tableau has the form ! Q(A I ) Qb v(A I ) − c¯ vb for some nonsingular matrix Q and some vector v, and in addition we have 0 0 ¯ B = 0. Since there is only one choice for Q and v, Q(A I ) B = I and (v(A I ) − c) 0 namely Q = Q B 0 and v = c¯ B Q B 0 , the updated tableau TB 0 is computed correctly by the above operations in O(m(n + m)) time. To start the S IMPLEX A LGORITHM we consider an LP of the form max{cx : A0 x ≤ b0 , A00 x ≤ b00 , x ≥ 0} 0

00

with A0 ∈ Rm ×n , A00 ∈ Rm ×n , b0 ≥ 0 and b00 < 0. We first run the S IMPLEX A LGORITHM on the instance min{(1lA00 )x + 1ly : A0 x ≤ b0 , A00 x + y ≥ b00 , x, y ≥ 0}, starting with the tableau

60

3 Linear Programming

A0 0 00 −A −I 1lA00 1l

I 0 b0 0 I −b00 , 0 0 0

(3.5)

corresponding to the basic solution x = 0, y = 0. Then we run the iterations of the S IMPLEX A LGORITHM as in Theorem 3.17(e). If the algorithm terminates with optimum value 1lb, we modify the final simplex tableau as follows. Multiply some rows by −1 such that none of the columns n + m 00 +m 0 +1, . . . , n+m 00 +m 0 +m 00 (the fourth section in (3.5)) is a unit vector, delete the fourth section of the tableau (i.e. columns n+m 00 +m 0 +1, . . . , n+m 00 +m 0 +m 00 ), and replace the last row by (−c, 0, 0, 0). Then add suitable multiples of the other rows to the last row in order to get zeros at m 0 +m 00 places corresponding to columns with distinct unit vectors; these will form our basis. The result is the simplex tableau with respect to the original LP and this basis. Therefore we can continue running the iterations of the S IMPLEX A LGORITHM as in Theorem 3.17(e). In fact, one can often do even more efficiently. Suppose we want to solve an LP min{cx : Ax ≥ b, x ≥ 0} with a very large number of inequalities which are implicitly given in a way that allows us to solve the following problem efficiently: Given a vector x ≥ 0, decide if Ax ≥ b and find a violated inequality otherwise. We apply the S IMPLEX A LGORITHM to the dual LP max{yb : y A ≤ c, y ≥ 0} = max{by : A> y ≤ c, y ≥ 0}. Let b¯ := (b> , 0). For a basis B we set Q B := ((A> I ) B )−1 and store only the right-hand part of the simplex tableau ! QB QBc . b¯ B Q B b> x ¯ To perform an iteration, The last row of the full simplex tableau is b¯ B Q B (A> I )− b. B B > ¯ ¯ we must check if b Q B ≥ 0 and b Q B A −c ≥ 0, and find a negative component if one exists. This reduces to solving the above problem for x = (b¯ B Q B )> . Then we generate the corresponding column of the full simplex tableau, but only for the current iteration. After updating the reduced tableau we can delete it again. This technique is known under the names revised simplex and column generation. We will see applications later.

3.4 Duality Theorem 3.14 shows that the LPs (3.1) and (3.2) are related. This motivates the following definition: Definition 3.18. Given a linear program max{cx : Ax ≤ b}, we define the dual LP to be the linear program min{yb : y A = c, y ≥ 0}. In this case, the original LP max{cx : Ax ≤ b} is often called the primal LP.

3.4 Duality

61

Proposition 3.19. The dual of the dual of an LP is (equivalent to) the original LP. Proof: Let the primal LP max{cx : Ax ≤ b} be given. Its dual is min{yb : y A = c, y ≥ 0}, or equivalently A> c − max −by : −A> y ≤ −c . 0 −I (Each equality constraint has been split up into two inequality constraints.) So the dual of the dual is z 0 0 0 − min zc − z c : A −A −I z = −b, z, z , w ≥ 0 w which is equivalent to − min{−cx : −Ax − w = −b, w ≥ 0} (where we have substituted x for z 0 − z). By eliminating the slack variables w we see that this is equivalent to the primal LP. We now obtain the most important theorem in LP theory, the Duality Theorem: Theorem 3.20. (von Neumann [1947], Gale, Kuhn and Tucker [1951]) If the polyhedra P := {x : Ax ≤ b} and D := {y : y A = c, y ≥ 0} are both nonempty, then max{cx : x ∈ P} = min{yb : y ∈ D}. Proof: If D is nonempty, it has a vertex y. We run the S IMPLEX A LGORITHM for min{yb : y ∈ D} and y. By Proposition 3.13, the existence of some x ∈ P guarantees that min{yb : y ∈ D} is not unbounded. Thus by Theorem 3.14, the S IMPLEX A LGORITHM returns optimum solutions y and z of the LP min{yb : y ∈ D} and its dual. However, the dual is max{cx : x ∈ P} by Proposition 3.19. We have yb = cz, as required. We can say even more about the relation between the optimum solutions of the primal and dual LP: Corollary 3.21. Let max{cx : Ax ≤ b} and min{yb : y A = c, y ≥ 0} be a primal-dual pair of LPs. Let x and y be feasible solutions, i.e. Ax ≤ b, y A = c and y ≥ 0. Then the following statements are equivalent: (a) x and y are both optimum solutions. (b) cx = yb. (c) y(b − Ax) = 0. Proof: The Duality Theorem 3.20 immediately implies the equivalence of (a) and (b). The equivalence of (b) and (c) follows from y(b − Ax) = yb − y Ax = yb −cx.

62

3 Linear Programming

The property (c) of optimum solutions is often called complementary slackness. It can also be formulated as follows: a point x ∗ ∈ P = {x : Ax ≤ b} is an optimum solution of max{cx : x ∈ P} if and only if c is a nonnegative combination of those rows of A which correspond to inequalities of Ax ≤ b that are satisfied by x ∗ with equality. It also implies: Corollary 3.22. Let P = {x : Ax ≤ b} be a polyhedron and Z ⊆ P. Then the set of vectors c for which each z ∈ Z is an optimum solution of max{cx : x ∈ P} is the cone generated by the rows of A0 , where A0 x ≤ b0 is the maximal subsystem of Ax ≤ b with A0 z = b0 for all z ∈ Z . Proof: There is a z ∈ Z which satisfies all other inequalities of Ax ≤ b strictly. Let c be a vector for which z is an optimum solution of max{cx : x ∈ P}. Then by Corollary 3.21 there exists an y ≥ 0 with c = y A0 , i.e. c is a nonnegative linear combination of the rows of A0 . Conversely, for a row a 0 x ≤ β 0 of A0 ≤ b0 and z ∈ Z we have a 0 z = β 0 = max{a 0 x : x ∈ P}. Let us write Corollary 3.21 in another form: Corollary 3.23. Let min{cx : Ax ≥ b, x ≥ 0} and max{yb : y A ≤ c, y ≥ 0} be a primal-dual pair of LPs. Let x and y be feasible solutions, i.e. Ax ≥ b, y A ≤ c and x, y ≥ 0. Then the following statements are equivalent: (a) x and y are both optimum solutions. (b) cx = yb. (c) (c − y A)x = 0 and y(b − Ax) = 0. Proof: Theequivalence of (a) and (b) is obtained by applying the Duality Theorem −b . 3.20 to max (−c)x : −A −I x ≤ 0 To prove that (b) and (c) are equivalent, observe that we have y(b − Ax) ≤ 0 ≤ (c − y A)x for any feasible solutions x and y, and that y(b − Ax) = (c − y A)x iff yb = cx. The two conditions in (c) are sometimes called primal and dual complementary slackness conditions. The Duality Theorem has many applications in combinatorial optimization. One reason for its importance is that the optimality of a solution can be proved by giving a feasible solution of the dual LP with the same objective value. We shall show now how to prove that an LP is unbounded or infeasible: Theorem 3.24. There exists a vector x with Ax ≤ b if and only if yb ≥ 0 for each vector y ≥ 0 for which y A = 0. Proof: If there is a vector x with Ax ≤ b, then yb ≥ y Ax = 0 for each y ≥ 0 with y A = 0.

3.4 Duality

63

Consider the LP − min{1lw : Ax − w ≤ b, w ≥ 0}. Writing it in standard form we have ! ( x : max 0 −1l w

A −I 0 −I

!

x w

!

b ≤ 0

(3.6)

!) .

The dual of this LP is ! ( ! ) ! ! y A> 0 0 y : min b 0 , y, z ≥ 0 , = z −1l z −I −I or, equivalently, min{yb : y A = 0, 0 ≤ y ≤ 1l}.

(3.7)

Since both (3.6) and (3.7) have a solution (x = 0, w = |b|, y = 0), we can apply Theorem 3.20. So the optimum values of (3.6) and (3.7) are the same. Since the system Ax ≤ b has a solution iff the optimum value of (3.6) is zero, the proof is complete. So the fact that a linear inequality system Ax ≤ b has no solution can be proved by giving a vector y ≥ 0 with y A = 0 and yb < 0. We mention two equivalent formulations of Theorem 3.24: Corollary 3.25. There is a vector x ≥ 0 with Ax ≤ b if and only if yb ≥ 0 for each vector y ≥ 0 with y A ≥ 0. A x ≤ b . Proof: Apply Theorem 3.24 to the system −I 0 Corollary 3.26. (Farkas [1894]) There is a vector x ≥ 0 with Ax = b if and only if yb ≥ 0 for each vector y with y A ≥ 0. b A x ≤ Proof: Apply Corollary 3.25 to the system −A −b , x ≥ 0. Corollary 3.26 is usually known as Farkas’ Lemma. The above results in turn imply the Duality Theorem 3.20 which is interesting since they have quite easy direct proofs (in fact they were known before the S IMPLEX A LGORITHM); see Exercises 10 and 11. We have seen how to prove that an LP is infeasible. How can we prove that an LP is unbounded? The next theorem answers this question. Theorem 3.27. If an LP is unbounded, then its dual LP is infeasible. If an LP has an optimum solution, then its dual also has an optimum solution.

64

3 Linear Programming

Proof: The first statement follows immediately from Proposition 3.13. To prove the second statement, suppose that the (primal) LP max{cx : Ax ≤ b} has an optimum solution x ∗ , but the dual min{yb : y A = c, y ≥ 0} is infeasible (it cannot be unbounded due to the first statement). In other words, there is no y ≥ 0 with A> y = c, and we apply Farkas’ Lemma (Corollary 3.26) to get a vector z with z A> ≥ 0 and zc < 0. But then x ∗ − z is feasible for the primal, because A(x ∗ − z) = Ax ∗ − Az ≤ b. The observation c(x ∗ − z) > cx ∗ therefore contradicts the optimality of x ∗ . So there are four cases for a primal-dual pair of LPs: either both have an optimum solution (in which case the optimum values are the same), or one is infeasible and the other one is unbounded, or both are infeasible. We also note: Corollary 3.28. A feasible LP max{cx : Ax ≤ b} is bounded if and only if c belongs to the cone generated by the rows of A. Proof: The LP is bounded iff its dual is feasible, i.e. there is a y ≥ 0 with y A = c. Farkas’ Lemma also enables us to prove that each finitely generated cone is polyhedral: Theorem 3.29. (Minkowski [1896], Weyl [1935]) A cone is polyhedral if and only if it is finitely generated. Proof: The only-if direction is given by Lemma 3.12. So consider the cone C generated by a1 , . . . , at . We have to show that C is polyhedral. Let A be the matrix whose rows are a1 , . . . , at . By Lemma 3.12, the cone D := {x : Ax ≤ 0} is generated by some vectors b1 , . . . , bs . Let B be the matrix whose rows are b1 , . . . , bs . We prove that C = {x : Bx ≤ 0}. As b j ai = ai b j ≤ 0 for all i and j, we have C ⊆ {x : Bx ≤ 0}. Now suppose there is a vector w ∈ / C with Bw ≤ 0. w 6∈ C means that there is no v ≥ 0 such that A> v = w. By Farkas’ Lemma (Corollary 3.26) this means that there is a vector y with yw < 0 and Ay ≥ 0. So −y ∈ D. Since D is generated by b1 , . . . , bs we have −y = z B for some z ≥ 0. But then 0 < −yw = z Bw ≤ 0, a contradiction.

3.5 Convex Hulls and Polytopes In this section we collect some more facts on polytopes. In particular, we show that polytopes are precisely those sets that are the convex hull of a finite number of points. We start by recalling some basic definitions: Pk Definition 3.30. Given vectors x1 , . . . , xk ∈ Rn and λ1 , . . . , λk ≥ 0 with i=1 λi Pk = 1, we call x = i=1 λi xi a convex combination of x1 , . . . , xk . A set X ⊆ Rn

3.5 Convex Hulls and Polytopes

65

is convex if λx + (1 − λ)y ∈ X for all x, y ∈ X and λ ∈ [0, 1]. The convex hull conv(X ) of a set X is defined as the set of all convex combinations of points in X . An extreme point of a set X is an element x ∈ X with x ∈ / conv(X \ {x}). So a set X is convex if and only if all convex combinations of points in X are again in X . The convex hull of a set X is the smallest convex set containing X . Moreover, the intersection of convex sets is convex. Hence polyhedra are convex. Now we prove the “finite basis theorem for polytopes”, a fundamental result which seems to be obvious but is not trivial to prove directly: Theorem 3.31. (Minkowski [1896], Steinitz [1916], Weyl [1935]) A set P is a polytope if and only if it is the convex hull of a finite set of points. Proof: (Schrijver [1986]) Let P = {x ∈ Rn : Ax ≤ b} be a nonempty polytope. Obviously, ! ) ( ! ) ( x x n+1 ∈R ∈ C , where C = : λ ≥ 0, Ax − λb ≤ 0 . P= x: λ 1 C is a polyhedral cone, so by Theorem 3.29 it is generated by finitely many nonzero x x vectors, say by λ11 , . . . , λkk . Since P is bounded, all λi are nonzero; w.l.o.g. all λi are 1. So x ∈ P if and only if ! ! ! x1 xk x = µ1 + · · · + µk 1 1 1 for some µ1 , . . . , µk ≥ 0. In other words, P is the convex hull of x1 , . . . , xk . n x ∈ P if and only if Now let P be the convex hull of x1 , . . . , xx1k ∈ R . Then xk x . By Theorem 3.29, C ∈ C, where C is the cone generated by , . . . , 1 1 1 is polyhedral, so ) ( ! x : Ax + bλ ≤ 0 . C = λ We conclude that P = {x ∈ Rn : Ax + b ≤ 0}.

Corollary 3.32. A polytope is the convex hull of its vertices. Proof: Let P be a polytope. By Theorem 3.31, the convex hull of its vertices is a polytope Q. Obviously Q ⊆ P. Suppose there is a point z ∈ P \ Q. Then there is a vector c with cz > max{cx : x ∈ Q}. The supporting hyperplane {x : cx = max{cy : y ∈ P}} of P defines a face of P containing no vertex. This is impossible by Corollary 3.10. The previous two and the following result are the starting point of polyhedral combinatorics; they will be used very often in this book. For a given ground set E and a subset X ⊆ E, the incidence vector of X (with respect to E) is defined as the vector x ∈ {0, 1} E with xe = 1 for e ∈ X and xe = 0 for e ∈ E \ X .

66

3 Linear Programming

Corollary 3.33. Let (E, F) be a set system, P the convex hull of the incidence vectors of the elements of F, and c : E → R. Then max{cx : x ∈ P} = max{c(X ) : X ∈ F}. Proof: Since max{cx : x ∈ P} ≥ max{c(X ) : X ∈ F} is trivial, let x be an optimum solution of max{cx : x ∈ P} (note that P is a polytope by Theorem 3.31). By definition of P,P x is a convex combination of incidence vectors Pk y1 , . . . , yk k of elements of F: x = i=1 λi yi for some λ1 , . . . , λk ≥ 0 with i=1 λi = 1. Pk Since cx = i=1 λi cyi , we have cyi ≥ cx for at least one i ∈ {1, . . . , k}. This yi is the incidence vector of a set Y ∈ F with c(Y ) = cyi ≥ cx.

Exercises 1. Let H be a hypergraph, F ⊆ V (H P ), and x, y : F → R. The task is to find x, y : V (H ) \ F → R such that e∈E(H ) (maxv∈e x(v) − minv∈e x(v) + maxv∈e y(v) − minv∈e y(v)) is minimum. Show that this can be formulated as an LP. Note: This is a relaxation of a placement problem in VLSI design. Here H is called the netlist, and its vertices correspond to modules that need to placed on the chip. Some (those in F) are pre-placed in advance. The main difficulty (ignored in this relaxation) is that modules must not overlap. is called affinely independent if there is no λ ∈ 2. A set of vectors x1 , . . . , xk P k λi xi = 0. Let ∅ 6= X ⊆ Rn . Show that the Rk \ {0} with λ> 1l = 0 and i=1 maximum cardinality of an affinely independent set of elements of X equals dim X + 1. 3. Let P, Q ∈ Rn be polyhedra. Prove that the closure of conv(P ∪ Q) is a polyhedron. Show polyhedra P and Q for which conv(P ∪ Q) is not a polyhedron. 4. Show that the problem to compute the largest ball that is a subset of a given polyhedron can be formulated as a linear program. 5. Let P be a polyhedron. Prove that the dimension of any facet of P is one less than the dimension of P. 6. Let F be a minimal face of a polyhedron {x : Ax ≤ b}. Prove that then Ax = Ay for all x, y ∈ F. 7. Formulate the dual of the LP formulation (1.1) of the J OB A SSIGNMENT P ROBLEM. Show how to solve the primal and the dual LP in the case when there are only two jobs (by a simple algorithm). 8. Let G be a digraph, c : E(G) → R+ , E 1 , E 2 ⊆ E(G), and s, t ∈ V (G). Consider the following linear program

Exercises

X

min

67

c(e)ye

e∈E(G)

s.t.

ye zt − zs ye ye

≥ = ≥ ≤

zw − zv 1 0 0

(e = (v, w) ∈ E(G)) (e ∈ E 1 ) (e ∈ E 2 ).

Prove that there is an optimum solution (y, z) and s ∈ X ⊆ V (G) \ {t} with ye = 1 for e ∈ δ + (X ), ye = −1 for e ∈ δ − (X ) \ E 1 , and ye = 0 for all other edges e. Hint: Consider the complementary slackness conditions for the edges entering or leaving {v ∈ V (G) : z v ≤ z s }. 9. Let Ax ≤ b be a linear inequality system in n variables. By multiplying each row by a positive constant we may assume that the first column of A is a vector with entries 0, −1 and 1 only. So we can write Ax ≤ b equivalently as ai0 x 0 ≤ bi −x1 + a j0 x 0 x1 + ak0 x 0

(i = 1, . . . , m 1 ),

≤ bj

( j = m 1 + 1, . . . , m 2 ),

≤ bk

(k = m 2 + 1, . . . , m),

0 are the rows of A without the first where x 0 = (x2 , . . . , xn ) and a10 , . . . , am entry. Then one can eliminate x1 : Prove that Ax ≤ b has a solution if and only if the system

ai0 x 0 ≤ bi a j0 x 0

10. 11. 12.

∗ 13.

− bj

≤

bk − ak0 x 0

(i = 1, . . . , m 1 ), ( j = m 1 + 1, . . . , m 2 , k = m 2 + 1, . . . , m)

has a solution. Show that this technique, when iterated, leads to an algorithm for solving a linear inequality system Ax ≤ b (or proving infeasibility). Note: This method is known as Fourier-Motzkin elimination because it was proposed by Fourier and studied by Motzkin [1936]. One can prove that it is not a polynomial-time algorithm. Use Fourier-Motzkin elimination (Exercise 9) to prove Theorem 3.24 directly. (Kuhn [1956]) Show that Theorem 3.24 implies the Duality Theorem 3.20. Prove the decomposition theorem for polyhedra: Any polyhedron P can be written as P = {x + c : x ∈ X, c ∈ C}, where X is a polytope and C is a polyhedral cone. (Motzkin [1936]) Let P be a rational polyhedron and F a face of P. Show that {c : cz = max {cx : x ∈ P} for all z ∈ F} is a rational polyhedral cone.

68

3 Linear Programming

14. Prove Carath´eodory’s theorem: If X ⊆ Rn and y ∈ conv(X ), then there are x1 , . . . , xn+1 ∈ X such that y ∈ conv({x1 , . . . , xn+1 }). (Carath´eodory [1911]) 15. Prove the following extension of Carath´eodory’s theorem (Exercise 14): If X ⊆ Rn and y, z ∈ conv(X ), then there are x1 , . . . , xn ∈ X such that y ∈ conv({z, x1 , . . . , xn }). 16. Prove that the extreme points of a polyhedron are precisely its vertices. 17. Let P be a nonempty polytope. Consider the graph G(P) whose vertices are the vertices of P and whose edges correspond to the 1-dimensional faces of P. Let x be any vertex of P, and c a vector with c> x < max{c> z : z ∈ P}. Prove that then there is a neighbour y of x in G(P) with c> x < c> y. ∗ 18. Use Exercise 17 to prove that G(P) is n-connected for any n-dimensional polytope P (n ≥ 1). 19. Let P ⊆ Rn be a polytope (not necessarily rational) and y ∈ / P. Prove that there exists a rational vector c with max{cx : x ∈ P} < cy. Show that the statement does not hold for general polyhedra. / X . Prove: 20. Let X ⊂ Rn be a nonempty convex set, X¯ the closure of X , and y ∈ (a) There is a unique point in X¯ that has minimum distance to y. (b) There exists a vector a ∈ Rn with a > x < a > y for all x ∈ X . (c) If X is bounded and y ∈ / X¯ , then there exists a vector a ∈ Qn with > > a x < a y for all x ∈ X . (d) A closed convex set is the intersection of all closed half-spaces containing it.

References General Literature: Bertsimas, D., and Tsitsiklis, J.N. [1997]: Introduction to Linear Optimization. Athena Scientific, Belmont 1997 Chv´atal, V. [1983]: Linear Programming. Freeman, New York 1983 Matouˇsek, J., and G¨artner, B. [2007]: Understanding and Using Linear Programming. Springer, Berlin 2007 Padberg, M. [1999]: Linear Optimization and Extensions. Second Edition. Springer, Berlin 1999 Schrijver, A. [1986]: Theory of Linear and Integer Programming. Wiley, Chichester 1986

Cited References: Avis, D., and Chv´atal, V. [1978]: Notes on Bland’s pivoting rule. Mathematical Programming Study 8 (1978), 24–34 Bland, R.G. [1977]: New finite pivoting rules for the simplex method. Mathematics of Operations Research 2 (1977), 103–107 Borgwardt, K.-H. [1982]: The average number of pivot steps required by the simplex method is polynomial. Zeitschrift f¨ur Operations Research 26 (1982), 157–177

References

69

¨ Carath´eodory, C. [1911]: Uber den Variabilit¨atsbereich der Fourierschen Konstanten von positiven harmonischen Funktionen. Rendiconto del Circolo Matematico di Palermo 32 (1911), 193–217 Dantzig, G.B. [1951]: Maximization of a linear function of variables subject to linear inequalities. In: Activity Analysis of Production and Allocation (T.C. Koopmans, ed.), Wiley, New York 1951, pp. 359–373 Dantzig, G.B., Orden, A., and Wolfe, P. [1955]: The generalized simplex method for minimizing a linear form under linear inequality restraints. Pacific Journal of Mathematics 5 (1955), 183–195 Farkas, G. [1894]: A Fourier-f´ele mechanikai elv alkalmaz´asai. Mathematikai e´ s Term´esz´ ettudom´anyi Ertesit¨ o 12 (1894), 457–472 Gale, D., Kuhn, H.W., and Tucker, A.W. [1951]: Linear programming and the theory of games. In: Activity Analysis of Production and Allocation (T.C. Koopmans, ed.), Wiley, New York 1951, pp. 317–329 Hoffman, A.J., and Kruskal, J.B. [1956]: Integral boundary points of convex polyhedra. In: Linear Inequalities and Related Systems; Annals of Mathematical Study 38 (H.W. Kuhn, A.W. Tucker, eds.), Princeton University Press, Princeton 1956, pp. 223–246 Kelner, J.A., and Spielman, D.A. [2006]: A randomized polynomial-time simplex algorithm for linear programming. Proceedings of the 38th Annual ACM Symposium on Theory of Computing (2006), 51–60 Klee, V., and Minty, G.J. [1972]: How good is the simplex algorithm? In: Inequalities III (O. Shisha, ed.), Academic Press, New York 1972, pp. 159–175 Kuhn, H.W. [1956]: Solvability and consistency for linear equations and inequalities. The American Mathematical Monthly 63 (1956), 217–232 Minkowski, H. [1896]: Geometrie der Zahlen. Teubner, Leipzig 1896 Motzkin, T.S. [1936]: Beitr¨age zur Theorie der linearen Ungleichungen (Dissertation). Azriel, Jerusalem 1936 von Neumann, J. [1947]: Discussion of a maximum problem. Working paper. Published in: John von Neumann, Collected Works; Vol. VI (A.H. Taub, ed.), Pergamon Press, Oxford 1963, pp. 27–28 Spielman, D.A., and Teng, S.-H. [2004]: Smoothed analysis of algorithms: why the simplex algorithm usually takes polynomial time. Journal of the ACM 51 (2004), 385–463 Steinitz, E. [1916]: Bedingt konvergente Reihen und konvexe Systeme. Journal f¨ur die reine und angewandte Mathematik 146 (1916), 1–52 Weyl, H. [1935]: Elementare Theorie der konvexen Polyeder. Commentarii Mathematici Helvetici 7 (1935), 290–306

4 Linear Programming Algorithms

There are basically three types of algorithms for L INEAR P ROGRAMMING: the S IMPLEX A LGORITHM (see Section 3.2), interior point algorithms, and the E L LIPSOID M ETHOD . Each of these has a disadvantage: In contrast to the other two, so far no variant of the S IMPLEX A LGORITHM has been shown to have a polynomial running time. In Sections 4.4 and 4.5 we present the E LLIPSOID M ETHOD and prove that it leads to a polynomial-time algorithm for L INEAR P ROGRAMMING. However, the E LLIPSOID M ETHOD is too inefficient to be used in practice. Interior point algorithms and, despite its exponential worst-case running time, the S IMPLEX A LGORITHM are far more efficient, and they are both used in practice to solve LPs. In fact, both the E LLIPSOID M ETHOD and interior point algorithms can be used for more general convex optimization problems, e.g. for so-called semidefinite programs. An advantage of the S IMPLEX A LGORITHM and the E LLIPSOID M ETHOD is that they do not require the LP to be given explicitly. It suffices to have an oracle (a subroutine) which decides whether a given vector is feasible and, if not, returns a violated constraint. We shall discuss this in detail with respect to the E LLIPSOID M ETHOD in Section 4.6, because it implies that many combinatorial optimization problems can be solved in polynomial time; for some problems this is in fact the only known way to show polynomial solvability. This is the reason why we discuss the E LLIPSOID M ETHOD but not interior point algorithms in this book. A prerequisite for polynomial-time algorithms is that there exists an optimum solution that has a binary representation whose length is bounded by a polynomial in the input size. We prove in Section 4.1 that this condition holds for L INEAR P ROGRAMMING. In Sections 4.2 and 4.3 we review some basic algorithms needed later, including the well-known Gaussian elimination method for solving systems of equations.

4.1 Size of Vertices and Faces Instances of L INEAR P ROGRAMMING are vectors and matrices. Since no strongly polynomial-time algorithm for L INEAR P ROGRAMMING is known we have to restrict attention to rational instances when analyzing the running time of algorithms. We assume that all numbers are coded in binary. To estimate the size (number of bits) of this representation we define size(n) := 1 + dlog(|n| + 1)e for integers

72

4 Linear Programming Algorithms

n ∈ Z and size(r ) := size( p) + size(q) for rational numbers r = qp , where p, q are relatively prime integers (i.e. their greatest common divisor is 1). For vectors x = (x1 , . . . , xn ) ∈ Qn we store the components and have size(x) := . + size(xn ). For a matrix A ∈ Qm×n with entries ai j we have n + size(x1 ) + . . P size(A) := mn + i, j size(ai j ). Of course these precise values are a somewhat random choice, but remember that we are not really interested in constant factors. For polynomial-time algorithms it is important that the sizes of numbers do not increase too much by elementary arithmetic operations. We note: Proposition 4.1. If r1 , . . . , rn are rational numbers, then size(r1 · · · rn ) ≤ size(r1 ) + · · · + size(rn ); size(r1 + · · · + rn ) ≤ 2(size(r1 ) + · · · + size(rn )). Proof: For integers s1 , . . . , sn we obviously have size(s1 · · · sn ) ≤ size(s1 ) + · · · + size(sn ) and size(s1 + · · · + sn ) ≤ size(s1 ) + · · · + size(sn ). Let now ri = qpii , where pi and qi are nonzero integers (i = 1, . . . , n). Then size(r1 · · · rn ) ≤ size( p1 · · · pn ) + size(q1 · · · qn ) ≤ size(r1 ) + · · · + size(rn ). For the second statement, observe that the denominator q1 · · · qn has size at most size(q1 ) + · · · + size(qn ). The numerator is the sum of the numbers q1 · · · qi−1 pi qi+1 · · · qn (i = 1, . . . , n), so its absolute value is at most (| p1 | + · · · + | pn |)|q1 · · · qn |. Therefore the size of the numerator is at most size(r1 ) + · · · + size(rn ). The first part of this proposition also implies that we can often assume w.l.o.g. that all numbers in a problem instance are integers, since otherwise we can multiply each of them with the product of all denominators. For addition and inner product of vectors we have: Proposition 4.2. If x, y ∈ Qn are rational vectors, then size(x + y) ≤ 2(size(x) + size(y)); size(x > y) ≤ 2(size(x) + size(y)). Pn Proof: Pn 4.1 we have size(x + y) = n + i=1 size(xi +>yi ) ≤ Pn Using Proposition and size(x y) = n +2 P i=1 size(x i )+2 Pni=1 size(yi ) = 2(size(x)+size(y))−3n Pn P n n x y size(x y ) ≤ 2 size(x ) + 2 size ≤ 2 i i i i i i=1 i=1 i=1 i=1 size(yi ) = 2(size(x) + size(y)) − 4n. Even under more complicated operations the numbers involved do not grow fast. Recall that the determinant of a matrix A = (ai j )1≤i, j≤n is defined by det A :=

X π ∈Sn

sgn(π)

n Y i=1

ai,π(i) ,

(4.1)

4.1 Size of Vertices and Faces

73

where Sn is the set of all permutations of {1, . . . , n} and sgn(π) is the sign of the permutation π (defined to be 1 if π can be obtained from the identity map by an even number of transpositions, and −1 otherwise). Proposition 4.3. For any matrix A ∈ Qm×n we have size(det A) ≤ 2 size(A). pi j qi j

with relatively prime integers pi j , qi j . Now let det A = Q where p and q are relatively prime integers. Then |det A| ≤ i, j (| pi j | + 1) Q and Q |q| ≤ i, j |qi j |. We obtain size(q) ≤ size(A) and, using | p| = |det A||q| ≤ i, j (| pi j | + 1)|qi j |,

Proof: We write ai j = p q

size( p) ≤

X (size( pi j ) + 1 + size(qi j )) = size(A). i, j

With this observation we can prove: Theorem 4.4. Suppose the rational LP max{cx : Ax ≤ b} has an optimum solution. Then it also has an optimum solution x with size(x) ≤ 4n(size(A) + size(b)), with components of size at most 4(size(A)+size(b)). If b = ei or b = −ei for some unit vector ei , then there is a nonsingular submatrix A0 of A and an optimum solution x with size(x) ≤ 4n size(A0 ). Proof: By Corollary 3.5, the maximum is attained in a face F of {x : Ax ≤ b}. Let F 0 ⊆ F be a minimal face. By Proposition 3.9, F 0 = {x : A0 x = b0 } for some subsystem A0 x ≤ b0 of Ax ≤ b. W.l.o.g., we may assume that the rows of A0 are linearly independent. We then take a maximal set of linear independent columns (call this matrix A00 ) and set all other components to zero. Then x = (A00 )−1 b0 , filled up with zeros, is an optimum solution to our LP. By Cramer’s rule the entries of x A000 000 00 are given by x j = det det A00 , where A arises from A by replacing the j-th column by b0 . By Proposition 4.3 we obtain size(x) ≤ n + 2n(size(A000 ) + size(A00 )) ≤ 4n(size(A00 ) + size(b0 )). If b = ±ei then | det(A000 )| is the absolute value of a subdeterminant of A00 . The encoding length of the faces of a polytope given by its vertices can be estimated as follows: Lemma 4.5. Let P ⊆ Rn be a rational polytope and T ∈ N such that size(x) ≤ T for each vertex x. Then P = {x : Ax ≤ b} for some inequality system Ax ≤ b, each of whose inequalities ax ≤ β satisfies size(a) + size(β) ≤ 75n 2 T . Proof: First assume that P is full-dimensional. Let F = {x ∈ P : ax = β} be a facet of P, where P ⊆ {x : ax ≤ β}. Let y1 , . . . , yt be the vertices of F (by Proposition 3.6 they are also vertices of P). Let c be the solution of Mc = e1 , where M is a t × n-matrix whose i-th row is yi − y1 (i = 2, . . . , t) and whose first row is some unit vector that is linearly

74

4 Linear Programming Algorithms

independent of the other rows. Observe that rank(M) = n (because dim F = n −1). So we have c> = κa for some κ ∈ R \ {0}. By Theorem 4.4 size(c) ≤ 4n size(M 0 ), where M 0 is a nonsingular n × nsubmatrix of M. By Proposition 4.2 we have size(M 0 ) ≤ 4nT and size(c> y1 ) ≤ 2(size(c) + size(y1 )). So the inequality c> x ≤ δ (or c> x ≥ δ if κ < 0), where δ := c> y1 = κβ, satisfies size(c) + size(δ) ≤ 3 size(c) + 2T ≤ 48n 2 T + 2T ≤ 50n 2 T . Collecting these inequalities for all facets F yields a description of P. If P = ∅, the assertion is trivial, so we now assume that P is neither fulldimensional nor empty. Let V be the set of vertices of P. For s = (s1 , . . . , sn ) ∈ {−1, 1}n let Ps be the convex hull of V ∪ {x + si ei : x ∈ V, i = 1, . . . , n}. Each Ps is a full-dimensional polytope (Theorem 3.31), and the size of any of its vertices is at most T + n (cf. Corollary 3.32). By the above, Ps can be described by inequalities of size at most 50n 2 (T + n) ≤ 75n 2 T (note that T ≥ 2n). Since T P = s∈{−1,1}n Ps , this completes the proof.

4.2 Continued Fractions When we say that the numbers occurring in a certain algorithm do not grow too fast, we often assume that for each rational qp the numerator p and the denominator q are relatively prime. This assumption causes no problem if we can easily find the greatest common divisor of two natural numbers. This is accomplished by one of the oldest algorithms:

E UCLIDEAN A LGORITHM Input:

Two natural numbers p and q.

Output:

The greatest common divisor d of p and q, i.e. prime integers.

p d

and

q d

are relatively

1

While p > 0 and q > 0 do: If p < q then set q := q − b qp c p else set p := p − b qp cq.

2

Return d := max{ p, q}.

Theorem 4.6. The E UCLIDEAN A LGORITHM works correctly. The number of iterations is at most size( p) + size(q). Proof: The correctness follows from the fact that the set of common divisors of p and q does not change throughout the algorithm, until one of the numbers becomes zero. One of p or q is reduced by at least a factor of two in each iteration, hence there are at most log p + log q + 1 iterations. Since no number occurring in an intermediate step is greater than p and q, we have a polynomial-time algorithm. A similar algorithm is the so-called C ONTINUED F RACTION E XPANSION. This can be used to approximate any number by a rational number whose denominator is

4.2 Continued Fractions

not too large. For any positive real number x we define x0 := x and xi+1 := for i = 1, 2, . . ., until xk ∈ N for some k. Then we have x = x0 = bx0 c +

1 1 = bx0 c + x1 bx1 c +

1 x2

= bx0 c +

1 bx1 c +

1 bx2 c+ x1

75 1 xi −bxi c

= ···

3

We claim that this sequence is finite if and only if x is rational. One direction follows immediately from the observation that xi+1 is rational if and only if xi is rational. The other direction is also easy: If x = qp , the above procedure is equivalent to the E UCLIDEAN ALGORITHM applied to p and q. This also shows that for a given rational number qp with p, q > 0 the (finite) sequence x1 , x2 , . . . , xk as above can be computed in polynomial time. The following algorithm is almost identical for the computation of the numbers gi to the E UCLIDEAN A LGORITHM except and h i ; we shall prove that the sequence hgii converges to x. i∈N

C ONTINUED F RACTION E XPANSION Input: Output:

Natural numbers p and q (let x := qp ). with x0 = The sequence xi = qpii i=0,1,...

1

2

p q

and xi+1 :=

1 xi −bxi c .

Set i := 0, p0 := p and q0 := q. Set g−2 := 0, g−1 := 1, h −2 := 1, and h −1 := 0. While qi 6= 0 do: Set ai := b qpii c. Set gi := ai gi−1 + gi−2 . Set h i := ai h i−1 + h i−2 . Set qi+1 := pi − ai qi . Set pi+1 := qi . Set i := i + 1.

We claim that the sequence hgii yields good approximations of x. Before we can prove this, we need some preliminary observations: Proposition 4.7. The following statements hold for all iterations i in the above algorithm: (a) ai ≥ 1 (except possibly for i = 0) and h i ≥ h i−1 . (b) gi−1 h i − gi h i−1 = (−1)i . pi gi−1 + qi gi−2 (c) = x. pi h i−1 + qi h i−2 (d) hgii ≤ x if i is even and hgii ≥ x if i is odd. Proof: (a) is obvious. (b) is easily shown by induction: For i = 0 we have gi−1 h i − gi h i−1 = g−1 h 0 = 1, and for i ≥ 1 we have

76

4 Linear Programming Algorithms

gi−1 h i − gi h i−1 = gi−1 (ai h i−1 + h i−2 ) − h i−1 (ai gi−1 + gi−2 ) = gi−1 h i−2 − h i−1 gi−2 . (c) is also proved by induction: For i = 0 we have pi · 1 + 0 pi gi−1 + qi gi−2 = = x. pi h i−1 + qi h i−2 0 + qi · 1 For i ≥ 1 we have pi gi−1 + qi gi−2 pi h i−1 + qi h i−2

qi−1 (ai−1 gi−2 + gi−3 ) + ( pi−1 − ai−1 qi−1 )gi−2 qi−1 (ai−1 h i−2 + h i−3 ) + ( pi−1 − ai−1 qi−1 )h i−2 qi−1 gi−3 + pi−1 gi−2 . qi−1 h i−3 + pi−1 h i−2

= =

= 0 < x < ∞ = hg−1 and proceed by We finally prove (d). We note hg−2 −2 −1 induction. The induction step follows easily from the fact that the function f (α) := αgi−1 +gi−2 pi αh i−1 +h i−2 is monotone for α > 0, and f ( qi ) = x by (c). Theorem 4.8. (Khintchine [1956]) Given a rational number α and a natural number n, a rational number β with denominator at most n such that |α − β| is minimum can be found in polynomial time (polynomial in size(n) + size(α)). Proof: We run the C ONTINUED F RACTION E XPANSION with x := α. If the = α by algorithm stops with qi = 0 and h i−1 ≤ n, we can set β = hgi−1 i−1 Proposition 4.7(c). Otherwise let i be the last index with h i ≤ n, and let t be the maximum integer such that th i + h i−1 ≤ n (cf. Proposition 4.7(a)). Since ai+1 h i + h i−1 = h i+1 > n, we have t < ai+1 . We claim that y :=

gi hi

or

z :=

tgi + gi−1 th i + h i−1

is an optimum solution. Both numbers have denominators at most n. If i is even, then y ≤ x < z by Proposition 4.7(d). Similarly, if i is odd, we have y ≥ x > z. We show that any rational number qp between y and z has denominator greater than n. Observe that |z − y| =

1 |h i gi−1 − h i−1 gi | = h i (th i + h i−1 ) h i (th i + h i−1 )

(using Proposition 4.7(b)). On the other hand, 1 h i−1 + (t + 1)h i p p 1 |z − y| = z − + − y ≥ + = , q q (th i + h i−1 )q hi q qh i (th i + h i−1 ) so q ≥ h i−1 + (t + 1)h i > n.

The above proof is from the book of Gr¨otschel, Lov´asz and Schrijver [1988], which also contains important generalizations.

4.3 Gaussian Elimination

77

4.3 Gaussian Elimination The most important algorithm in Linear Algebra is the so-called Gaussian elimination. It has been applied by Gauss but was known much earlier (see Schrijver [1986] for historical notes). Gaussian elimination is used to determine the rank of a matrix, to compute the determinant and to solve a system of linear equations. It occurs very often as a subroutine in linear programming algorithms; e.g. in 1 of the S IMPLEX A LGORITHM. Given a matrix A ∈ Qm×n , our algorithm for Gaussian Elimination works with an extended matrix Z = (B C) ∈ Qm×(n+m) ; initially B = A and C = I . The algorithm transforms B to the form 0I R0 by the following elementary operations: permuting rows and columns, adding a multiple of one row to another row, and (in the final step) multiplying rows by nonzero constants. At each iteration C is modified accordingly, such that the property C A˜ = B is maintained throughout where A˜ results from A by permuting rows and columns. The first part of the algorithm, consisting of 2 and , 3 transforms B to an upper triangular matrix. Consider for example the matrix Z after two iterations; it has the form z 12 z 13 · · · z 1n 1 0 0 · · · 0 z 11 6= 0 0 z 22 6= 0 z 23 · · · z 2n z 2,n+1 1 0 · · · 0 0 0 z 33 · · · z 3n z 3,n+1 z 3,n+2 1 0 · · 0 · · · · · · 0 · . · · · I · · · · · · · · · · · · 0 0

0

z m3 · · · z mn

z m,n+1 z m,n+2 0

·

·

0 1

i3 times the third row If z 33 6= 0, then the next step just consists of subtracting zz33 from the i-th row, for i = 4, . . . , m. If z 33 = 0 we first exchange the third row and/or the third column with another one. Note that if we exchange two rows, we have to exchange also the two corresponding columns of C in order to maintain the property C A˜ = B. To have A˜ available at each point we store the permutations of the rows and columns in variables r ow(i), i = 1, . . . , m and col( j), j = 1, . . . , n. Then A˜ = (Ar ow(i),col( j) )i∈{1,...,m}, j∈{1,...,n} . The second part of the algorithm, consisting of 4 and , 5 is simpler since no rows or columns are exchanged anymore.

G AUSSIAN E LIMINATION Input:

A matrix A = (ai j ) ∈ Qm×n .

Output:

Its rank r , a maximal nonsingular submatrix A0 = (ar ow(i),col( j) )i, j∈{1,...,r } of A, its determinant d = det A0 , and its inverse (A0 )−1 = (z i,n+ j )i, j∈{1,...,r } .

78

4 Linear Programming Algorithms

1

Set r := 0 and d := 1. Set z i j := ai j , r ow(i) := i and col( j) := j (i = 1, . . . , m, j = 1, . . . , n). Set z i,n+ j := 0 and z i,n+i := 1 for 1 ≤ i, j ≤ m, i 6= j.

2

Let p ∈ {r + 1, . . . , m} and q ∈ {r + 1, . . . , n} with z pq 6= 0. If no such p and q exist, then go to . 4 Set r := r + 1. If p 6= r then exchange z pj and zr j ( j = 1, . . . , n + m), exchange z i,n+ p and z i,n+r (i = 1, . . . , m), and exchange r ow( p) and r ow(r ). If q 6= r then exchange z iq and z ir (i = 1, . . . , m), and exchange col(q) and col(r ). Set d := d · zrr . For i := r + 1 to m do: For j := r to n + r do: z i j := z i j − zzrrir zr j . Go to . 2 For k := r down to 1 do: For i := 1 to k − 1 do: ik For j := k to n + r do z i j := z i j − zzkk zk j .

3

4

5

For k := 1 to r do: For j := 1 to n + r do z k j :=

zk j z kk .

Theorem 4.9. G AUSSIAN E LIMINATION works correctly and terminates after O(mnr ) steps. Proof: First observe that each time before 2 we have z ii 6= 0 for i ∈ {1, . . . , r } and z i j = 0 for all j ∈ {1, . . . , r } and i ∈ { j + 1, . . . , m}. Hence det (z i j )i, j∈{1,2,...,r } = z 11 z 22 · · · zrr = d 6= 0. Since adding a multiple of one row to another row of a square matrix does not change the value of the determinant (this well-known fact follows directly from the definition (4.1)) we have det (z i j )i, j∈{1,2,...,r } = det (ar ow(i),col( j) )i, j∈{1,2,...,r } at any stage before , 5 and hence the determinant d is computed correctly. A0 is a nonsingular r × r -submatrix of A. Since (z i j )i∈{1,...,m}, j∈{1,...,n} has rank r at termination andPthe operations did not change the rank, A has also rank r . m Moreover, j=1 z i,n+ j ar ow( j),col(k) = z ik for all i ∈ {1, . . . , m} and k ∈ {1, . . . , n} (i.e. C A˜ = B in our above notation) holds throughout. (Note that for j = r + 1, . . . , m we have at any stage z j,n+ j = 1 and z i,n+ j = 0 for i 6= j.) Since (z i j )i, j∈{1,2,...,r } is the unit matrix at termination this implies that (A0 )−1 is also computed correctly. The number of steps is obviously O(r mn + r 2 (n + r )) = O(mnr ).

4.3 Gaussian Elimination

79

In order to prove that G AUSSIAN E LIMINATION is a polynomial-time algorithm we have to guarantee that all numbers that occur are polynomially bounded by the input size. This is not trivial but can be shown: Theorem 4.10. (Edmonds [1967]) G AUSSIAN E LIMINATION is a polynomialtime algorithm. Each number occurring in the course of the algorithm can be stored with O(m(m + n) size(A)) bits. Proof: We first show that in 2 and 3 all numbers are 0, 1, or quotients of subdeterminants of A. First observe that entries z i j with i ≤ r or j ≤ r are not modified anymore. Entries z i j with j > n +r are 0 (if j 6= n +i) or 1 (if j = n +i). Furthermore, we have for all s ∈ {r + 1, . . . , m} and t ∈ {r + 1, . . . , n + m} det (z i j )i∈{1,2,...,r,s}, j∈{1,2,...,r,t} z st = . det (z i j )i, j∈{1,2,...,r } (This follows from evaluating the determinant det (z i j )i∈{1,2,...,r,s}, j∈{1,2,...,r,t} along the last row because z s j = 0 for all s ∈ {r +1, . . . , m} and all j ∈ {1, . . . , r }.) We have already observed in the proof of Theorem 4.9 that det (z i j )i, j∈{1,2,...,r } = det (ar ow(i),col( j) )i, j∈{1,2,...,r } , because adding a multiple of one row to another row of a square matrix does not change the value of the determinant. By the same argument we have det (z i j )i∈{1,2,...,r,s}, j∈{1,2,...,r,t} = det (ar ow(i),col( j) )i∈{1,2,...,r,s}, j∈{1,2,...,r,t} for s ∈ {r + 1, . . . , m} and t ∈ {r + 1, . . . , n}. Furthermore, det (z i j )i∈{1,2,...,r,s}, j∈{1,2,...,r,n+t} = det (ar ow(i),col( j) )i∈{1,2,...,r,s}\{t}, j∈{1,2,...,r } for all s ∈ {r + 1, . . . , m} and t ∈ {1, . . . , r }, which is checked by evaluating the left-hand side determinant (after ) 1 along column n + t. We conclude that at any stage in 2 and 3 all numbers z i j are 0, 1, or quotients of subdeterminants of A. Hence, by Proposition 4.3, each number occurring in 2 and 3 can be stored with O(size(A)) bits. Finally observe that 4 is equivalent to applying 2 and 3 again, choosing p and q appropriately (reversing the order of the first r rows and columns). Hence each number occurring in 4 can be stored with O size (z i j )i∈{1,...,m}, j∈{1,...,m+n} bits, which is O(m(m + n) size(A)). The easiest way to keep the representations of the numbers z i j small enough is to guarantee that the numerator and denominator of each of these numbers are relatively prime at any stage. This can be accomplished by applying the E UCLIDEAN A LGORITHM after each computation. This gives an overall polynomial running time. In fact, we can easily implement G AUSSIAN E LIMINATION to be a strongly polynomial-time algorithm (Exercise 4).

80

4 Linear Programming Algorithms

So we can check in polynomial time whether a set of vectors is linearly independent, and we can compute the determinant and the inverse of a nonsingular matrix in polynomial time (exchanging two rows or columns changes just the sign of the determinant). Moreover we get: Corollary 4.11. Given a matrix A ∈ Qm×n and a vector b ∈ Qm we can in polynomial time find a vector x ∈ Qn with Ax = b or decide that no such vector exists. Proof: We compute a maximal nonsingular submatrix A0 = (ar ow(i),col( j) )i, j∈{1,...,r } of A and its inverse (A0P )−1 = (z i,n+ j )i, j∈{1,...,r } by G AUSSIAN E LIMINATION. r Then we set xcol( j) := k=1 z j,n+k br ow(k) for j = 1, . . . , r and x k := 0 for k∈ / {col(1), . . . , col(r )}. We obtain for i = 1, . . . r : n X

ar ow(i), j x j

=

j=1

=

=

r X j=1 r X j=1 r X

ar ow(i),col( j) xcol( j)

ar ow(i),col( j)

r X

z j,n+k br ow(k)

k=1

br ow(k)

k=1

r X

ar ow(i),col( j) z j,n+k

j=1

= br ow(i) . Since the other rows of A with indices not in {r ow(1), . . . , r ow(r )} are linear combinations of these, either x satisfies Ax = b or no vector satisfies this system of equations.

4.4 The Ellipsoid Method In this section we describe the so-called ellipsoid method, developed by Iudin and Nemirovskii [1976] and Shor [1977] for nonlinear optimization. Khachiyan [1979] observed that it can be modified in order to solve LPs in polynomial time. Most of our presentation is based on (Gr¨otschel, Lov´asz and Schrijver [1981]), (Bland, Goldfarb and Todd [1981]) and the book of Gr¨otschel, Lov´asz and Schrijver [1988], which is also recommended for further study. The idea of the ellipsoid method is very roughly the following. We look for either a feasible or an optimum solution of an LP. We start with an ellipsoid which we know a priori to contain the solutions (e.g. a large ball). At each iteration k, we check if the center xk of the current ellipsoid is a feasible solution. Otherwise, we take a hyperplane containing xk such that all the solutions lie on one side of this hyperplane. Now we have a half-ellipsoid which contains all solutions. We take the smallest ellipsoid completely containing this half-ellipsoid and continue.

4.4 The Ellipsoid Method

81

Deﬁnition 4.12. An ellipsoid is a set E(A, x) = {z ∈ Rn : (z − x) A−1 (z − x) ≤ 1} for some symmetric positive deﬁnite n × n-matrix A. Note that B(x, r ) := E(r 2 I, x) (with I being the n × n unit matrix) is the n-dimensional Euclidean ball with center x and radius r . The volume of an ellipsoid E(A, x) is known to be √ volume (E(A, x)) = det A volume (B(0, 1)) (see Exercise 7). Given an ellipsoid E(A, x) and a hyperplane {z : az = ax}, the smallest ellipsoid E(A , x ) containing the half-ellipsoid E = {z ∈ E(A, x) : az ≥ ax} is called the L¨owner-John ellipsoid of E (see Figure 4.1). It can be computed by the following formulas: n2 2 A = bb , A− n+1 n2 − 1 1 b, x = x + n+1 1 Aa. b = √ a Aa

{z : az = ax}

x

E(A, x) E(A , x )

Fig. 4.1.

82

4 Linear Programming Algorithms

One difficulty of the ellipsoid method is caused by the square root in the computation of b. Because we have to tolerate rounding errors, it is necessary to increase the radius of the next ellipsoid a little bit. Here is an algorithmic scheme that takes care of this problem:

E LLIPSOID M ETHOD Input: Output:

1

2

3

4

A number n ∈ N, n ≥ 2. A number N ∈ N. x0 ∈ Qn and R ∈ Q+ , R ≥ 2. An ellipsoid E(A N , x N ).

Set p := d6N + log(9n 3 )e. Set A0 := R 2 I , where I is the n × n unit matrix. Set k := 0. Choose any ak ∈ Qn \ {0}. 1 A k ak . Set bk := q > ak A k ak 1 ∗ := xk + Set xk+1 :≈ xk+1 bk . n + 1 2n 2 + 3 2 > ∗ bk bk . Ak − Set Ak+1 :≈ Ak+1 := n+1 2n 2 (Here :≈ means computing the entries up to p decimal places, taking care that Ak+1 is symmetric). Set k := k + 1. If k < N then go to 2 else stop.

So in each of the N iterations an approximation E(Ak+1 , xk+1 ) of the smallest ellipsoid containing E(Ak , xk ) ∩ {z : ak z ≥ ak xk } is computed. Two main issues, how to obtain the ak and how to choose N , will be addressed in the next section. But let us first prove some lemmas. Let ||x|| denote the Euclidean norm of vector x, while ||A|| := max{||Ax|| : ||x|| = 1} shall denote the norm of the matrix A. For symmetric matrices, ||A|| is the maximum absolute value of an eigenvalue and ||A|| = max{x > Ax : ||x|| = 1}. The first lemma says that each E k := E(Ak , xk ) is indeed an ellipsoid. Furthermore, the absolute values of the numbers involved remain smaller than R 2 2 N + 2size(x0 ) . Therefore each iteration of the E LLIPSOID M ETHOD consists of O(n 2 ) computational steps, each involving numbers with O( p + size(ak ) + size(R) + size(x0 )) bits. Lemma 4.13. (Gr¨otschel, Lov´asz and Schrijver [1981]) Let k ∈ {0, 1, . . . , N }. Then Ak is positive definite, and we have ||xk || ≤ ||x0 || + R2k ,

||Ak || ≤ R 2 2k ,

and

−2 k ||A−1 k || ≤ R 4 .

Proof: We use induction on k. For k = 0 all the statements are obvious. Assume that they are true for some k ≥ 0. By a straightforward computation one verifies

4.4 The Ellipsoid Method

that (A∗k+1 )−1

2n 2 = 2n 2 + 3

A−1 k

ak ak> 2 + > n − 1 ak A k ak

.

83

(4.2)

So (A∗k+1 )−1 is the sum of a positive definite and a positive semidefinite matrix; thus it is positive definite. Hence A∗k+1 is also positive definite. Note that for positive semidefinite matrices A and B we have ||A|| ≤ ||A + B||. Therefore 2 2n 2 + 3 11 2 k 2n 2 + 3 ∗ > − b R 2 . b ||Ak || ≤ ||Ak+1 || = ≤ A k k k 2 2 n+1 8 2n 2n Since the n × n all-one matrix has norm n, the matrix Ak+1 − A∗k+1 , each of whose entries has absolute value at most 2− p , has norm at most n2− p . We conclude ||Ak+1 || ≤ ||A∗k+1 || + ||Ak+1 − A∗k+1 || ≤

11 2 k R 2 + n2− p ≤ R 2 2k+1 8

(here we used the very rough estimate 2− p ≤ n1 ). It is well-known from linear algebra that for any symmetric positive definite n × n-matrix A there exists a symmetric positive definite matrix B with A = B B. Writing Ak = B B with B = B > we obtain s s p ak> A2k ak (Bak )> Ak (Bak ) ||Ak ak || ≤ ||Ak || ≤ R2k−1 . ||bk || = q = = > > ak A k ak (Bak ) (Bak ) a> A a k

k k

Using this (and again the induction hypothesis) we get 1 ∗ ||bk || + ||xk+1 − xk+1 || n+1 √ 1 ≤ ||x0 || + R2k + R2k−1 + n2− p ≤ ||x0 || + R2k+1 . n+1

||xk+1 || ≤ ||xk || +

Using (4.2) and ||ak ak> || = ak> ak we compute ak> ak 2n 2 2 −1 ∗ −1 (4.3) + (Ak+1 ) ≤ A k n − 1 ak> Ak ak 2n 2 + 3 ! 2n 2 2 ak> B A−1 −1 k Bak = Ak + n − 1 ak> B Bak 2n 2 + 3 2n 2 2 −1 n + 1 −1 −1 ≤ + < Ak A A k k n−1 n−1 2n 2 + 3 ≤ 3R −2 4k . Let λ be the smallest eigenvalue of Ak+1 , and let v be a corresponding eigenvector with ||v|| = 1. Then – writing A∗k+1 = CC for a symmetric matrix C – we have

84

4 Linear Programming Algorithms

λ = v > Ak+1 v = v > A∗k+1 v + v > (Ak+1 − A∗k+1 )v v > CCv + v > (Ak+1 − A∗k+1 )v −1 v > C A∗k+1 Cv −1 1 ≥ (A∗k+1 )−1 − ||Ak+1 − A∗k+1 || > R 2 4−k − n2− p ≥ R 2 4−(k+1) , 3

=

where we used 2− p ≤

1 −k 3n 4 .

Since λ > 0, Ak+1 is positive definite. Furthermore,

1 ≤ R −2 4k+1 . (Ak+1 )−1 = λ

Next we show that in each iteration the ellipsoid contains the intersection of E 0 and the previous half-ellipsoid: Lemma 4.14. For k = 0, . . . , N −1 we have E k+1 ⊇ {x ∈ E k ∩ E 0 : ak x ≥ ak xk }. Proof: Let x ∈ E k ∩ E 0 with ak x ≥ ak xk . We first compute (using (4.2)) ∗ ∗ )> (A∗k+1 )−1 (x − xk+1 ) (x − xk+1 > 2 ak ak> 2n 1 2 1 −1 = bk bk Ak + x − xk − x − xk − n +1 n −1 ak> Ak ak n +1 2n 2 +3 ak a > 2 2n 2 (x − xk )> A−1 (x − xk )> > k (x − xk ) = k (x − x k ) + 2 n−1 ak A k ak 2n + 3 > > 1 2 bk ak ak bk > −1 + bk Ak bk + n − 1 ak> Ak ak (n + 1)2 2 ak ak> bk 2(x − xk )> b + − A−1 k k n+1 n − 1 ak> Ak ak ak ak> 2 2n 2 > −1 > (x − x = ) A (x − x ) + ) (x − xk ) + (x − x k k k k n−1 ak> Ak ak 2n 2 + 3 2 2 (x − xk )> ak 2 1 q 1 + − . 1 + n−1 n+1 n−1 (n + 1)2 ak> Ak ak

Since x ∈ E k , we have (x −xk )> A−1 k (x −x k ) ≤ 1. By abbreviating t :=

ak> (x−xk ) q ak> Ak ak

we obtain ∗ ∗ (x − xk+1 )> (A∗k+1 )−1 (x − xk+1 ) ≤

2 2n 2 1 2 2 t t . − + 1 + n−1 2n 2 + 3 n2 − 1 n − 1

> −1 Since bk> A−1 k bk = 1 and bk Ak (x − x k ) = t, we have

4.4 The Ellipsoid Method

85

1 ≥ (x − xk )> A−1 k (x − x k ) 2 = (x − xk − tbk )> A−1 k (x − x k − tbk ) + t

≥ t 2, because A−1 k is positive definite. So (using ak x ≥ ak x k ) we have 0 ≤ t ≤ 1 and obtain 2n 4 ∗ ∗ (x − xk+1 )> (A∗k+1 )−1 (x − xk+1 ) ≤ . 2n 4 + n 2 − 3 It remains to estimate the rounding error ∗ ∗ Z := (x − xk+1 )> (Ak+1 )−1 (x − xk+1 ) − (x − xk+1 )> (A∗k+1 )−1 (x − xk+1 ) ∗ − xk+1 ) ≤ (x − xk+1 )> (Ak+1 )−1 (xk+1 ∗ ∗ − xk+1 )> (Ak+1 )−1 (x − xk+1 ) + (xk+1 ∗ ∗ )> (Ak+1 )−1 − (A∗k+1 )−1 (x − xk+1 ) + (x − xk+1 ≤

∗ ||x − xk+1 || ||(Ak+1 )−1 || ||xk+1 − xk+1 || ∗ ∗ +||xk+1 − xk+1 || ||(Ak+1 )−1 || ||x − xk+1 || ∗ +||x − xk+1 ||2 ||(Ak+1 )−1 || ||(A∗k+1 )−1 || ||A∗k+1 − Ak+1 ||.

Using Lemma 4.13 and x ∈ E 0 we get ||x −√ xk+1 || ≤ ||x − x0 || + ||xk+1 − x0 || ≤ ∗ || ≤ ||x − x − p ≤ R2 N +1 . We also use (4.3) R + R2 N and ||x − xk+1 k+1 || + n2 and obtain √ Z ≤ 2(R2 N +1 )(R −2 4 N )( n2− p ) + (R 2 4 N +1 )(R −2 4 N )(3R −2 4 N −1 )(n2− p ) √ = 4R −1 23N n2− p + 3R −2 26N n2− p ≤ 26N n2− p 1 ≤ , 9n 2 by definition of p. Altogether we have (x − xk+1 )> (Ak+1 )−1 (x − xk+1 ) ≤

1 2n 4 + ≤ 1. 2n 4 + n 2 − 3 9n 2

The volumes of the ellipsoids decrease by a constant factor in each iteration: Lemma 4.15. For k = 0, . . . , N − 1 we have

volume (E k+1 ) volume (E k )

1

< e− 5n .

Proof: (Gr¨otschel, Lov´asz and Schrijver [1988]) We write s s s det A∗k+1 det Ak+1 det Ak+1 volume (E k+1 ) = = volume (E k ) det Ak det Ak det A∗k+1

86

4 Linear Programming Algorithms

and estimate the two factors independently. First observe that !n det A∗k+1 2n 2 + 3 2 ak ak> Ak = det I − . det Ak n + 1 ak> Ak ak 2n 2 The matrix

ak ak> Ak ak> Ak ak

has rank one and 1 as its only nonzero eigenvalue (eigenvector

ak ). Since the determinant is the product of the eigenvalues, we conclude that !n det A∗k+1 3 2 1 2 2n 2 + 3 1 − < e 2n e− n = e− 2n , = det Ak n+1 2n 2 n where we used 1 + x ≤ e x for all x and n−1 < e−2 for n ≥ 2. n+1 For the second estimation we use (4.3) and the well-known fact that det B ≤ ||B||n for any matrix B: det Ak+1 ∗ −1 ∗ = det I + (A ) (A − A ) k+1 k+1 k+1 det A∗k+1 n ≤ I + (A∗k+1 )−1 (Ak+1 − A∗k+1 ) n ≤ ||I || + ||(A∗k+1 )−1 || ||Ak+1 − A∗k+1 || n ≤ 1 + (R −2 4k+1 )(n2− p ) n 1 ≤ 1+ 10n 2 1

≤ e 10n (we used 2− p ≤ 10n43 4 N ≤ We conclude that volume (E k+1 ) = volume (E k )

R2 ). 10n 3 4k+1

s

det A∗k+1 det Ak

s

1 1 1 det Ak+1 ≤ e− 4n e 20n = e− 5n . ∗ det Ak+1

4.5 Khachiyan’s Theorem In this section we shall prove Khachiyan’s theorem: the E LLIPSOID M ETHOD can be applied to L INEAR P ROGRAMMING in order to obtain a polynomial-time algorithm. Let us first prove that it suffices to have an algorithm for checking feasibility of linear inequality systems: Proposition 4.16. Suppose there is a polynomial-time algorithm for the following problem: “Given a matrix A ∈ Qm×n and a vector b ∈ Qm , decide if {x : Ax ≤ b} is empty.” Then there is a polynomial-time algorithm for L INEAR P ROGRAMMING which finds an optimum basic solution if there exists one.

4.5 Khachiyan’s Theorem

87

Proof: Let an LP max{cx : Ax ≤ b} be given. We first check if the primal and dual LPs are both feasible. If at least one of them is infeasible, we are done by Theorem 3.27. Otherwise, by Corollary 3.21, it is sufficient to find an element of {(x, y) : Ax ≤ b, y A = c, y ≥ 0, cx = yb}. We show (by induction on k) that a solution of a feasible system of k inequalities and l equalities can be found by k calls to the subroutine checking emptiness of polyhedra plus additional polynomial-time work. For k = 0 a solution can be found easily by G AUSSIAN E LIMINATION (Corollary 4.11). Now let k > 0. Let ax ≤ β be an inequality of the system. By a call to the subroutine we check whether the system becomes infeasible by replacing ax ≤ β by ax = β. If so, the inequality is redundant and can be removed (cf. Proposition 3.8). If not, we replace it by the equality. In both cases we reduced the number of inequalities by one, so we are done by induction. If there exists an optimum basic solution, the above procedure generates one, because the final equality system contains a maximal feasible subsystem of Ax = b. Before we can apply the E LLIPSOID M ETHOD, we have to take care that the polyhedron is bounded and full-dimensional: Proposition 4.17. (Khachiyan [1979], G´acs and Lov´asz [1981]) Let A ∈ Qm×n and b ∈ Qm . The system Ax ≤ b has a solution if and only if the system Ax ≤ b + 1l,

−R1l ≤ x ≤ R1l

has a solution, where 1l is the all-one vector, 1 = 2n24(size(A)+size(b)) and R = 1 + 24(size(A)+size(b)) . n If Ax ≤ b has a solution, then volume ({x ∈ R : Ax ≤ b + 1l, −R1l ≤ x ≤ R1l}) ≥

2 n2size(A)

n

.

Proof: The box constraints −R1l ≤ x ≤ R1l do not change the solvability by Theorem 4.4. Now suppose that Ax ≤ b has no solution. By Theorem 3.24 (a version of Farkas’ Lemma), there is a vector y ≥ 0 with y A = 0 and yb = −1. By applying Theorem 4.4 to min{1ly : y ≥ 0, A> y = 0, b> y = −1} we conclude that y can be chosen such that its components are of absolute value at most 24(size(A)+size(b)) . Therefore y(b + 1l) ≤ −1 + n24(size(A)+size(b)) ≤ − 21 . Again by Theorem 3.24, this proves that Ax ≤ b + 1l has no solution. For the second statement, if x ∈ Rn with Ax ≤ b has components of absolute value at most R−1 (cf. Theorem 4.4), then {x ∈ Rn : Ax ≤ b+1l, −R1l ≤ x ≤ R1l} . contains all points z with ||z − x||∞ ≤ n2size(A) Note that the construction of this proposition increases the size of the system of inequalities by at most a factor of O(m + n). Theorem 4.18. (Khachiyan [1979]) There exists a polynomial-time algorithm for L INEAR P ROGRAMMING (with rational input), and this algorithm finds an optimum basic solution if there exists one.

88

4 Linear Programming Algorithms

Proof: By Proposition 4.16 it suffices to check feasibility of a system Ax ≤ b. We transform the system as in Proposition 4.17 to obtain a polytope P in order which is either empty or has volume at least

2 n2size(A)

n

.

We run the E LLIPSOID M ETHOD with x0 = 0, R = n 1 + 24(size(A)+size(b)) , N = d10n 2 (2 log n + 5(size(A) + size(b)))e. Each time in 2 we check whether xk ∈ P. If yes, we are done. Otherwise we take a violated inequality ax ≤ β of the system Ax ≤ b and set ak := −a. We claim that if the algorithm does not find an xk ∈ P before iteration N , then P must be empty. To see this, we first observe that P ⊆ E k for all k: for k = 0 this is clear by the construction of P and R; the induction step is Lemma 4.14. So we have P ⊆ E N . By Lemma 4.15, we have, abbreviating s := size(A) + size(b), N

N

volume (E N ) ≤ volume (E 0 )e− 5n ≤ (2R)n e− 5n n < 2n 1 + 24s n −4n e−10ns < n −2n 2−5ns . On the other hand, P 6= ∅ implies n 1 2 n = volume (P) ≥ = n −2n 2−5ns , n2s n 2 25s which is a contradiction.

If we estimate the running time for solving an LP max{cx : Ax ≤ b} with the above method, we get the bound O((n+m)9 (size(A)+size(b)+size(c))2 ) (Exercise 9), which is polynomial but completely useless for practical purposes. In practice, either the S IMPLEX A LGORITHM or interior point algorithms are used. Karmarkar [1984] was the first to describe a polynomial-time interior point algorithm for L INEAR P ROGRAMMING. We shall not go into the details here. A strongly polynomial-time algorithm for L INEAR P ROGRAMMING is not known. However, Tardos [1986] showed that there is an algorithm for solving max{cx : Ax ≤ b} with a running time that polynomially depends on size(A) only. For many combinatorial optimization problems, where A is a 0-1-matrix, this gives a strongly polynomial-time algorithm. Tardos’ result was extended by Frank and Tardos [1987].

4.6 Separation and Optimization The above method (in particular Proposition 4.16) requires that the polyhedron be given explicitly by a list of inequalities. However, a closer look shows that this is not really necessary. It is sufficient to have a subroutine which – given a vector x – decides if x ∈ P or otherwise returns a separating hyperplane, i.e. a vector a such that ax > max{ay : y ∈ P}. We shall prove this for full-dimensional polytopes; for

4.6 Separation and Optimization

89

the general (more complicated) case we refer to Gr¨otschel, Lov´asz and Schrijver [1988] (or Padberg [1995]). The results in this section are due to Gr¨otschel, Lov´asz and Schrijver [1981] and independently to Karp and Papadimitriou [1982] and Padberg and Rao [1981]. With the results of this section one can solve certain linear programs in polynomial time although the polytope has an exponential number of facets. Many examples will be discussed later in this book; see e.g. Corollary 12.19 or Theorem 20.34. By considering the dual LP one can also deal with linear programs with a huge number of variables. Let P ⊆ Rn be a full-dimensional polytope, or more generally, a fulldimensional bounded convex set. We assume that we know the dimension n and two balls B(x0 , r ) and B(x0 , R) such that B(x0 , r ) ⊆ P ⊆ B(x0 , R). But we do not assume that we know a linear inequality system defining P. In fact, this would not make sense if we want to solve linear programs with an exponential number of constraints in polynomial time, or even optimize linear objective functions over nonlinearly constrained convex sets. Below we shall prove that, under some reasonable assumptions, we can optimize a linear function over a polyhedron P in polynomial time (independent of the number of constraints) if we have a so-called separation oracle: a subroutine for the following problem:

S EPARATION P ROBLEM Instance:

A convex set P ⊆ Rn . A vector y ∈ Qn .

Task:

Either decide that y ∈ P or find a vector d ∈ Qn such that d x < dy for all x ∈ P.

Note that such a vector d exists if P is a rational polyhedron or a compact convex set (cf. Exercise 20 of Chapter 3). Given a convex set P by such a separation oracle, we look for an oracle algorithm using this as a black box. In an oracle algorithm we may ask the oracle at any time and we get a correct answer in one step. We can regard this concept as a subroutine whose running time we do not take into account. (In Chapter 15 we shall give a formal definition.) Indeed, it often suffices to have an oracle which solves the S EPARATION P ROB LEM approximately. More precisely we assume an oracle for the following problem:

W EAK S EPARATION P ROBLEM Instance: Task:

A convex set P ⊆ Rn , a vector c ∈ Qn and a number > 0. A vector y ∈ Qn . Either find a vector y 0 ∈ P with cy ≤ cy 0 + or find a vector d ∈ Qn such that d x < dy for all x ∈ P.

Using a weak separation oracle we first solve linear programs approximately:

90

4 Linear Programming Algorithms

W EAK O PTIMIZATION P ROBLEM Instance:

A number n ∈ N. A vector c ∈ Qn . A number > 0. A convex set P ⊆ Rn given by an oracle for the W EAK S EPARATION P ROBLEM for P, c and 2 .

Task:

Find a vector y ∈ P with cy ≥ sup{cx : x ∈ P} − .

Note that the above two definitions differ from the ones given e.g. in Gr¨otschel, Lov´asz and Schrijver [1981]. However, they are basically equivalent, and we shall need the above form again in Section 18.3. The following variant of the E LLIPSOID M ETHOD solves the W EAK O PTI MIZATION P ROBLEM for bounded full-dimensional convex sets:

¨ ´ -S CHRIJVER A LGORITHM G R OTSCHEL -L OV ASZ Input:

Output:

A number n ∈ N, n ≥ 2. A vector c ∈ Qn . A number 0 < ≤ 1. A convex set P ⊆ Rn given by an oracle for the W EAK S EPARATION P ROBLEM for P, c and 2 . x0 ∈ Qn and r, R ∈ Q+ such that B(x0 , r ) ⊆ P ⊆ B(x0 , R). A vector y ∗ ∈ P with cy ∗ ≥ sup{cx : x ∈ P} − .

1

Set R := max{R, l 2},2 r m:= min{r, 1} and γ := max{||c||, 1}. 2 Set N := 5n ln 6Rr γ . Set y ∗ := x0 .

2

Run the E LLIPSOID M ETHOD, with ak in 2 being computed as follows: Run the oracle for the W EAK S EPARATION P ROBLEM with y = xk . If it returns a y 0 ∈ P with cy ≤ cy 0 + 2 then: If cy 0 > cy ∗ then set y ∗ := y 0 . Set ak := c. If it returns a d ∈ Qn with d x < dy for all x ∈ P then: Set ak := −d.

¨ ´ -S CHRIJVER A LGORITHM correctly Theorem 4.19. The G R OTSCHEL -L OV ASZ solves the W EAK O PTIMIZATION P ROBLEM for bounded full-dimensional convex sets. Its running time is bounded by O n 6 α 2 + n 4 α f (size(c), size(), n size(x0 ) + n 3 α) , where α = log Rr γ and f (size(c), size(), size(y)) is an upper bound of the running time of the oracle for the W EAK S EPARATION P ROBLEM for P with input c, , y. 2

Proof: (Gr¨otschel, Lov´asz and Schrijver [1981]) The running time in each of the N = O(n 2 α) iterations of the E LLIPSOID M ETHOD is O(n 2 (n 2 α + size(R) + size(x0 ) + q)) plus one oracle call, where q is the size of the output of the oracle. As size(y) ≤ n(size(x0 ) + size(R) + N ) by Lemma 4.13, the overall running time is O(n 4 α(n 2 α + size(x0 ) + f (size(c), size(), n size(x0 ) + n 3 α))), as stated.

4.6 Separation and Optimization

91

By Lemma 4.14, we have x ∈ P : cx ≥ cy ∗ + ⊆ EN . 2 Let z ∈ P with cz ≥ sup{cx : x ∈ P} − 6 . We may assume that cz > cy ∗ + 2 ; otherwise we are done.

r x0 U

r

z

{x : cx = cx0 }

{x : cx = cy ∗ + 2 } Fig. 4.2.

Consider the convex hull U of z and the (n −1)-dimensional ball B(x0 , r )∩{x : cx = cx0 } (see Figure 4.2). We have U ⊆ P and hence U := {x ∈ U : cx ≥ cy ∗ + 2 } is contained in E N . The volume of U is

cz − cy ∗ − 2 n volume (U ) = volume (U ) cz − cx0 n ∗ cz − cx 0 cz − cy − 2 = Vn−1r n−1 , n||c|| cz − cx0

where Vn denotes the volume of the n-dimensional unit ball. Since volume (U ) ≤ volume (E N ), and Lemma 4.15 yields N

N

volume (E N ) ≤ e− 5n E 0 = e− 5n Vn R n , we have − N ≤ e 5n2 R cz − cy − 2 ∗

Vn (cz − cx0 )n−1 n||c|| Vn−1r n−1

1 n

.

Since cz − cx0 ≤ ||c|| · ||z − x0 || ≤ ||c||R we obtain − N ≤ ||c||e 5n2 R cz − cy ∗ − 2

nVn R n−1 Vn−1r n−1

1 n

and hence cy ∗ ≥ cz − 56 ≥ sup{cx : x ∈ P} − .

< 2||c||e

−

N 5n 2

R2 ≤ r 3

92

4 Linear Programming Algorithms

Of course we are usually interested in the exact optimum. For this we restrict ourselves to rational full-dimensional polytopes. We need some assumption on the size of the vertices of the polytope. Lemma 4.20. Let n ∈ N, let P ⊆ Rn be a rational polytope, and let x0 ∈ Qn be a point in the interior of P. Let T ∈ N such that size(x0 ) ≤ log T and size(x) ≤ log T 2 for all vertices x of P. Then B(x0 , r ) ⊆ P ⊆ B(x0 , R), where r := n1 T −379n and R := 2nT . Moreover, let K := 4T 2n+1 . Let c ∈ Zn , and define c0 := K n c + (1, K , . . . , n−1 ). Then max{c0 x : x ∈ P} is attained by a unique vector x ∗ , for all other K vertices y of P we have c0 (x ∗ − y) > T −2n , and x ∗ is also an optimum solution of max{cx : x ∈ P}. Proof: For any vertex x of P we have ||x|| ≤ nT and ||x0 || ≤ nT , so ||x − x0 || ≤ 2nT and x ∈ B(x0 , R). To show that B(x0 , r ) ⊆ P, let F = {x ∈ P : ax = β} be a facet of P, where by Lemma 4.5 we may assume that size(a) + size(β) < 75n 2 log T . Suppose there is a point y ∈ F with ||y − x0 || < r . Then |ax0 − β| = |ax0 − ay| ≤ ||a|| · ||y − x0 || < n2size(a)r ≤ T −304n

2

But on the other hand the size of ax0 − β can by estimated by size(ax0 − β) ≤ 4(size(a) + size(x0 ) + size(β)) ≤ 300n 2 log T + 4 log T ≤ 304n 2 log T. 2

Since ax0 6= β (x0 is in the interior of P), this implies |ax0 − β| ≥ T −304n , a contradiction. To prove the last statements, let x ∗ be a vertex of P maximizing c0 x, and let y be another vertex of P. By the assumption on the size of the vertices of P we may write x ∗ − y = α1 z, where α ∈ {1, 2, . . . , T 2n − 1} and z is an integral vector whose components have absolute value at most K2 . Then ! n X 1 K n cz + K i−1 z i . 0 ≤ c0 (x ∗ − y) = α i=1

Pn

Since K n > i=1 K i−1 |z i |, we must have cz ≥ 0 and hence cx ∗ ≥ cy. So x ∗ indeed maximizes cx over P. Moreover, since z 6= 0, we obtain c0 (x ∗ − y) ≥ as required.

1 > T −2n , α

Theorem 4.21. Let n ∈ N and c ∈ Qn . Let P ⊆ Rn be a rational polytope, and let x0 ∈ Qn be a point in the interior of P. Let T ∈ N such that size(x0 ) ≤ log T and size(x) ≤ log T for all vertices x of P.

4.6 Separation and Optimization

93

Given n, c, x0 , T and a polynomial-time oracle for the S EPARATION P ROBLEM for P, a vertex x ∗ of P attaining max{c> x : x ∈ P} can be found in time polynomial in n, log T and size(c). ¨ Proof: (Gr¨otschel, Lov´asz and Schrijver [1981]) We first use the G R OTSCHEL ´ -S CHRIJVER A LGORITHM to solve the W EAK O PTIMIZATION P ROBLEM; L OV ASZ we set c0 , r and R according to Lemma 4.20 and := 8nT12n+3 . (We first have to make c integral by multiplying with the product of its denominators; this increases its size by at most a factor 2n.) ¨ ´ -S CHRIJVER A LGORITHM returns a vector y ∈ P -L OV ASZ The G R OTSCHEL with c0 y ≥ c0 x ∗ −, where x ∗ is an optimum solution of max{c0 x : x ∈ P}. By Theo rem 4.19 the running time is O n 6 α 2 + n 4 α f (size(c0 ), size(), n size(x0 ) + n 3 α) = 2 0 ||,1} ≤ O n 6 α 2 + n 4 α f (size(c0 ), 6n log T, n log T + n 3 α) , where α = log R max{||c r

log(16n 5 T 400n 2size(c ) ) = O(n 2 log T + size(c0 )) and f is a polynomial upper bound of the running time of the oracle for the S EPARATION P ROBLEM for P. Since size(c0 ) ≤ 6n 2 log T + 2 size(c), we have an overall running time that is polynomial in n, log T and size(c). We claim that ||x ∗ − y|| ≤ 2T1 2 . To see this, write y as a convex combination of the vertices x ∗ , x1 , . . . , xk of P: 2

0

y = λ0 x + ∗

k X

λi xi ,

λi ≥ 0,

i=1

k X

λi = 1.

i=0

Now – using Lemma 4.20 – ≥ c0 (x ∗ − y) =

k X

λi c0 x ∗ − xi

i=1

>

k X

λi T −2n = (1 − λ0 )T −2n ,

i=1

so 1 − λ0 < T 2n . We conclude that ||y − x ∗ || ≤

k X

λi ||xi − x ∗ || ≤ (1 − λ0 )2R < 4nT 2n+1 ≤

i=1

1 . 2T 2

So when rounding each entry of y to the next rational number with denominator at most T , we obtain x ∗ . The rounding can be done in polynomial time by Theorem 4.8. We have proved that, under certain assumptions, optimizing over a polytope can be done whenever there is a separation oracle. We close this chapter by noting that the converse is also true. We need the concept of polarity: If X ⊆ Rn , we define the polar of X to be the set X ◦ := {y ∈ Rn : y > x ≤ 1 for all x ∈ X }. When applied to full-dimensional polytopes, this operation has some nice properties:

94

4 Linear Programming Algorithms

Theorem 4.22. Let P be a polytope in Rn with 0 in the interior. Then: (a) P ◦ is a polytope with 0 in the interior; (b) (P ◦ )◦ = P; (c) x is a vertex of P if and only if x > y ≤ 1 is a facet-defining inequality of P ◦ . Proof: (a): Let P be the convex hull of x1 , . . . , xk (cf. Theorem 3.31). By definition, P ◦ = {y ∈ Rn : y > xi ≤ 1 for all i ∈ {1, . . . , k}}, i.e. P ◦ is a polyhedron and the facet-defining inequalities of P ◦ are given by vertices of P. Moreover, 0 is in the interior of P ◦ because 0 satisfies all of the finitely many inequalities strictly. Suppose P ◦ is unbounded, i.e. there exists a w ∈ Rn \ {0} with αw ∈ P ◦ for all α > 0. Then αwx ≤ 1 for all α > 0 and all x ∈ P, so wx ≤ 0 for all x ∈ P. But then 0 cannot be in the interior of P. (b): Trivially, P ⊆ (P ◦ )◦ . To show the converse, suppose that z ∈ (P ◦ )◦ \ P. Then there is an inequality c> x ≤ δ satisfied by all x ∈ P but not by z. We have δ > 0 since 0 is in the interior of P. Then 1δ c ∈ P ◦ but 1δ c> z > 1, contradicting the assumption that z ∈ (P ◦ )◦ . (c): We have already seen in (a) that the facet-defining inequalities of P ◦ are given by vertices of P. Conversely, if x1 , . . . , xk are the vertices of P, then ¯ Now (b) implies P¯ := conv({ 12 x1 , x2 , . . . , xk }) 6= P, and 0 is in the interior of P. ◦ ◦ n > > ¯ P 6= P . Hence {y ∈ R : y x1 ≤ 2, y xi ≤ 1(i = 2, . . . , k)} = P¯ ◦ 6= P ◦ = {y ∈ Rn : y > xi ≤ 1(i = 1, . . . , k)}. We conclude that x1> y ≤ 1 is a facet-defining inequality of P ◦ . Now we can prove: Theorem 4.23. Let n ∈ N and y ∈ Qn . Let P ⊆ Rn be a rational polytope, and let x0 ∈ Qn be a point in the interior of P. Let T ∈ N such that size(x0 ) ≤ log T and size(x) ≤ log T for all vertices x of P. Given n, y, x0 , T and an oracle which for any given c ∈ Qn returns a vertex x ∗ of P attaining max{c> x : x ∈ P}, we can solve the S EPARATION P ROBLEM for P and y in time polynomial in n, log T and size(y). Indeed, in the case y ∈ / P we can find a facet-defining inequality of P that is violated by y. Proof: Consider Q := {x − x0 : x ∈ P} and its polar Q ◦ . If x1 , . . . , xk are the vertices of P, we have Q ◦ = {z ∈ Rn : z > (xi − x0 ) ≤ 1 for all i ∈ {1, . . . , k}}. By Theorem 4.4 we have size(z) ≤ 4n(2n log T + 3n) ≤ 20n 2 log T for all vertices z of Q ◦ . Observe that the S EPARATION P ROBLEM for P and y is equivalent to the S EPARATION P ROBLEM for Q and y − x0 . Since by Theorem 4.22 Q = (Q ◦ )◦ = {x : zx ≤ 1 for all z ∈ Q ◦ }, the S EPARATION P ROBLEM for Q and y − x0 is equivalent to solving max{(y − x0 )> x : x ∈ Q ◦ }. Since each vertex of Q ◦ corresponds to a facet-defining inequality

Exercises

95

of Q (and thus of P), it remains to show how to find a vertex attaining max{(y − x0 )> x : x ∈ Q ◦ }. To do this, we apply Theorem 4.21 to Q ◦ . By Theorem 4.22, Q ◦ is fulldimensional with 0 in the interior. We have shown above that the size of the vertices of Q ◦ is at most 20n 2 log T . So it remains to show that we can solve the S EPARATION P ROBLEM for Q ◦ in polynomial time. However, this reduces to the optimization problem for Q which can be solved using the oracle for optimizing over P. We finally mention that a new algorithm which is faster than the E LLIPSOID M ETHOD and also implies the equivalence of optimization and separation has been proposed by Vaidya [1996]. However, this algorithm does not seem to be of practical use either.

Exercises

∗

∗

1. Let A be a nonsingular rational n × n-matrix. Prove that size(A−1 ) ≤ 4n 2 size(A). 2. Let n ≥ 2, c ∈ Rn and y1 , . . . , yk ∈ {−1, 0, 1}n such that 0 < c> yi+1 ≤ 12 c> yi for i = 1, . . . , k − 1. Prove that then k ≤ 3n log n. Hint: Consider the linear program max{y1> x : yk> x = 1, (yi − 2yi+1 )> x ≥ 0 (i = 1, . . . , k − 1)} and recall the proof of Theorem 4.4. (M. Goemans) 3. Consider the numbers h i in the C ONTINUED F RACTION E XPANSION. Prove that h i ≥ Fi+1 for all i, where Fi is the i-th Fibonacci number (F1 = F2 = 1 and Fn = Fn−1 + Fn−2 for n ≥ 3). Observe that √ !n √ !n ! 1 1+ 5 1− 5 . Fn = √ − 2 2 5 Conclude that the number of iterations of the C ONTINUED F RACTION E X PANSION is O(log q). (Gr¨otschel, Lov´asz and Schrijver [1988]) 4. Show that G AUSSIAN E LIMINATION can be made a strongly polynomial-time algorithm. Hint: First assume that A is integral. Recall the proof of Theorem 4.10 and observe that we can choose d as the common denominator of the entries. (Edmonds [1967]) l d := 1 + dim{x , . . . , x }, λ , . . . , λ ∈ R 5. Let k k + with 1 1 Pk x1 , . . . , xk ∈ R , P k λ = 1, and x := λ x . Show how to compute numbers µ , . . . , µk i i i 1 i=1 i=1 Pk ∈ R+ , at most d of which are nonzero, such that µ = 1 and i=1 i Pk x = i=1 µi xi (cf. Exercise 14 of Chapter 3). Show that all computations can be performed in O((k + l)3 ) time. Hint: Run G AUSSIAN E LIMINATION with the matrix A ∈ R(l+1)×k whose

96

4 Linear Programming Algorithms

i-th column is x1i . If d < k, let w ∈ Rk be the vector with wcol(i) := z i,d+1 (i = 1, . . . , d), wcol(d+1) := −1 and wcol(i) := 0 (i = d + 2, . . . , k); observe that Aw = 0. Add a multiple of w to λ, eliminate at least one vector and iterate. > 6. Let A = αb bC ∈ Rn×n be a symmetric positive semidefinite matrix with 1 α 0 α > 0 and b ∈ Rn−1 . Let A0 := 0 C− 1 bb> and U := 1 α b . Prove that α

∗

0 I

A = U > A0 U and C − α1 bb> is positive semidefinite. Iterate and conclude that for any positive semidefinite matrix A there is a matrix U with A = U > U , and such a matrix can be computed with arbitrary precision in O(n 3 ) steps (some of which consist of computing approximate square roots). Note: This is called Cholesky factorization. It cannot be computed exactly as U can be irrational. 7. Let A be a symmetric positive definite n ×n-matrix. Let v1 , . . . , vn be n orthogonal eigenvectors of A, with corresponding eigenvalues λ1 , . . . , λn . W.l.o.g. ||vi || = 1 for i = 1, . . . , n. Prove that then o n p p E(A, 0) = µ1 λ1 v1 + · · · + µn λn vn : µ ∈ Rn , ||µ|| ≤ 1 .

8. 9. 10. ∗ 11.

12.

(The eigenvectors correspond to the√axes of symmetry of the ellipsoid.) Conclude that volume (E(A, 0)) = det A volume (B(0, 1)). Let E(A, x) ⊆ Rn be an ellipsoid and a ∈ Rn , and let E(A0 , x 0 )) be as defined on page 81. Prove that {z ∈ E(A, x) : az ≥ ax} ⊆ E(A0 , x 0 ). Prove that the algorithm of Theorem 4.18 solves a linear program max{cx : Ax ≤ b} in O((n + m)9 (size(A) + size(b) + size(c))2 ) time. Show that the assumption that P is bounded can be omitted in Theorem 4.21. One can detect if the LP is unbounded and otherwise find an optimum solution. Let P ⊆ R3 be a 3-dimensional polytope with 0 in its interior. Consider again the graph G(P) whose vertices are the vertices of P and whose edges correspond to the 1-dimensional faces of P (cf. Exercises 17 and 18 of Chapter 3). Show that G(P ◦ ) is the planar dual of G(P). Note: Steinitz [1922] proved that for every simple 3-connected planar graph G there is a 3-dimensional polytope P with G = G(P). Prove that the polar of a polyhedron is always a polyhedron. For which polyhedra P is (P ◦ )◦ = P?

References General Literature: Gr¨otschel, M., Lov´asz, L., and Schrijver, A. [1988]: Geometric Algorithms and Combinatorial Optimization. Springer, Berlin 1988 Padberg, M. [1999]: Linear Optimization and Extensions. Second edition. Springer, Berlin 1999 Schrijver, A. [1986]: Theory of Linear and Integer Programming. Wiley, Chichester 1986

References

97

Cited References: Bland, R.G., Goldfarb, D., and Todd, M.J. [1981]: The ellipsoid method: a survey. Operations Research 29 (1981), 1039–1091 Edmonds, J. [1967]: Systems of distinct representatives and linear algebra. Journal of Research of the National Bureau of Standards B 71 (1967), 241–245 ´ [1987]: An application of simultaneous Diophantine approximation Frank, A., and Tardos, E. in combinatorial optimization. Combinatorica 7 (1987), 49–65 G´acs, P., and Lov´asz, L. [1981]: Khachiyan’s algorithm for linear programming. Mathematical Programming Study 14 (1981), 61–68 Gr¨otschel, M., Lov´asz, L., and Schrijver, A. [1981]: The ellipsoid method and its consequences in combinatorial optimization. Combinatorica 1 (1981), 169–197 Iudin, D.B., and Nemirovskii, A.S. [1976]: Informational complexity and effective methods of solution for convex extremal problems. Ekonomika i Matematicheskie Metody 12 (1976), 357–369 [in Russian] Karmarkar, N. [1984]: A new polynomial-time algorithm for linear programming. Combinatorica 4 (1984), 373–395 Karp, R.M., and Papadimitriou, C.H. [1982]: On linear characterizations of combinatorial optimization problems. SIAM Journal on Computing 11 (1982), 620–632 Khachiyan, L.G. [1979]: A polynomial algorithm in linear programming [in Russian]. Doklady Akademii Nauk SSSR 244 (1979) 1093–1096. English translation: Soviet Mathematics Doklady 20 (1979), 191–194 Khintchine, A. [1956]: Kettenbr¨uche. Teubner, Leipzig 1956 Padberg, M.W., and Rao, M.R. [1981]: The Russian method for linear programming III: Bounded integer programming. Research Report 81-39, New York University 1981 Shor, N.Z. [1977]: Cut-off method with space extension in convex programming problems. Cybernetics 13 (1977), 94–96 Steinitz, E. [1922]: Polyeder und Raumeinteilungen. Enzyklop¨adie der Mathematischen Wissenschaften, Band 3 (1922), 1–139 ´ [1986]: A strongly polynomial algorithm to solve combinatorial linear programs. Tardos, E. Operations Research 34 (1986), 250–256 Vaidya, P.M. [1996]: A new algorithm for minimizing convex functions over convex sets. Mathematical Programming 73 (1996), 291–341

5 Integer Programming

In this chapter, we consider linear programs with integrality constraints:

I NTEGER P ROGRAMMING Instance:

A matrix A ∈ Zm×n and vectors b ∈ Zm , c ∈ Zn .

Task:

Find a vector x ∈ Zn such that Ax ≤ b and cx is maximum, decide that {x ∈ Zn : Ax ≤ b} = ∅, or decide that sup{cx : x ∈ Zn , Ax ≤ b} = ∞.

We do not consider mixed integer programs, i.e. linear programs with integrality constraints for only a subset of the variables. Most of the theory of linear and integer programming can be extended to mixed integer programming in a natural way.

PI P

Fig. 5.1.

Virtually all combinatorial optimization problems can be formulated as integer programs. The set of feasible solutions can be written as {x : Ax ≤ b, x ∈ Zn } for some matrix A and some vector b. The set P := {x ∈ Rn : Ax ≤ b} is a polyhedron, so let us deﬁne by PI = {x : Ax ≤ b} I the convex hull of the integral vectors in P. We call PI the integer hull of P. Obviously PI ⊆ P.

100

5 Integer Programming

If P is bounded, then PI is also a polytope by Theorem 3.31 (see Figure 5.1). Meyer [1974] proved: Theorem 5.1. For any rational polyhedron P, its integer hull PI is a rational polyhedron. Proof: Let P = {x : Ax ≤ b}. By theorem 3.29 the rational polyhedral cone C := {(x, ξ ) : x ∈ Rn , ξ ≥ 0, Ax − ξ b ≤ 0} is finitely generated. We may assume that (x1 , 1), . . . , (xk , 1), (y1 , 0), . . . , (yl , 0) generate C, where x1 , . . . , xk are rational and y1 , . . . , yl are integral (by multiplying the elements of a finite set of generators by suitable positive scalars). Consider the polytope Q :=

X k

κi xi +

l X

i=1

λi yi

: κi ≥ 0 (i = 1, . . . , k),

i=1 k X

κi = 1,

i=1

0 ≤ λi ≤ 1 (i = 1, . . . , l) . Note that Q ⊆ P. Let z 1 , . . . , z m be the integral points in Q. By theorem 3.29 the cone C 0 generated by (y1 , 0), . . . , (yl , 0), (z 1 , 1), . . . , (z m , 1) is polyhedral, i.e. can be written as {(x, ξ ) : M x + ξ b ≤ 0} for a rational matrix M and a rational vector b. We claim that PI = {x : M x ≤ −b}. Pk κi xi + To show “⊆”, let x ∈ P ∩ Zn . We have (x, 1) ∈ C, i.e. x = i=1 Pl Pk λ y for some κ . . . , κ ≥ 0 with κ = 1 and λ . . . , λ ≥ 0. Then i i k l 1 1 i=1 P i=1 i l c := i cyi is integral, and hence x − c is integral. Moreover, x − c = i=1 bλP Pk l κ x + i i i=1 i=1 (λi − bλi c)yi ∈ Q, and hence x − c = z i for some i. Thus (x, 1) = (c, 0) + (x − c, 1) ∈ C 0 and hence M x + b ≤ 0. To show be a rational vector satisfying M x ≤ −b, i.e. (x, 1) ∈ C 0 . Pl“⊇”, let x P m ThenP x = i=1 λi yi + i=1 µi z j for rational numbers λ1 . . . , λl , µ1 , . . . , µm ≥ 0 m with i=1 µi = 1. W.l.o.g. let µ1 > 0. Let δ ∈ N such that δλi ∈ N for i = 1, . . . , l P and δ ≥ µ11 . Then (z 1 + li=1 δλi yi , 1) ∈ C and hence l

X 1 z1 + δλi yi x= δ i=1

!

m X 1 + µ1 − µi z i z1 + δ

is a convex combination of integral points in P.

i=2

This does in general not hold for irrational polyhedra; see Exercise 1. Due to Theorem 5.1 we can write an instance of I NTEGER P ROGRAMMING as max{c> x : x ∈ PI } where P = {x : Ax ≤ b}.

5.1 The Integer Hull of a Polyhedron

101

We prove a generalization of Meyer’s Theorem 5.1 in Section 5.1 (Theorem 5.8). After some preparation in Section 5.2 we study conditions under which polyhedra are integral (i.e. P = PI ) in Sections 5.3 and 5.4. Note that in this case the integer linear program is equivalent to its LP relaxation (arising by omitting the integrality constraints), and can hence be solved in polynomial time. We shall encounter this situation for several combinatorial optimization problems in later chapters. In general, however, I NTEGER P ROGRAMMING is much harder than L INEAR P ROGRAMMING, and polynomial-time algorithms are not known. This is indeed not surprising since we can formulate many apparently hard problems as integer programs. Nevertheless we discuss a general method for finding the integer hull by successively cutting off parts of P \ PI in Section 5.5. Although it does not yield a polynomial-time algorithm it is a useful technique in some cases. Finally Section 5.6 contains an efficient way of approximating the optimal value of an integer linear program.

5.1 The Integer Hull of a Polyhedron As linear programs, integer programs can be infeasible or unbounded. It is not easy to decide whether PI = ∅ for a polyhedron P. But if an integer program is feasible we can decide whether it is bounded by simply considering the LP relaxation. Proposition 5.2. Let P = {x : Ax ≤ b} be some rational polyhedron whose integer hull is nonempty, and let c be some vector (not necessarily rational). Then max {cx : x ∈ P} is bounded if and only if max {cx : x ∈ PI } is bounded. Proof: Suppose max {cx : x ∈ P} is unbounded. Then Corollary 3.28 says that the system y A = c, y ≥ 0 has no solution. By Corollary 3.26 there is a vector z with cz < 0 and Az ≥ 0. Then the LP min{cz : Az ≥ 0, −1l ≤ z ≤ 1l} is feasible. Let z ∗ be an optimum basic solution of this LP. z ∗ is rational as it is a vertex of a rational polytope. Multiply z ∗ by a suitable natural number to obtain an integral vector w with Aw ≥ 0 and cw < 0. Let v ∈ PI be some integral vector. Then v − kw ∈ PI for all k ∈ N, and thus max {cx : x ∈ PI } is unbounded. The other direction is trivial. Definition 5.3. Let A be an integral matrix. A subdeterminant of A is det B for some square submatrix B of A (defined by arbitrary row and column indices). We write 4(A) for the maximum absolute value of the subdeterminants of A. Lemma 5.4. Let C = {x : Ax ≥ 0} be a polyhedral cone, where A is an integral matrix. Then C is generated by a finite set of integral vectors, each having components with absolute value at most 4(A). Proof: By Lemma 3.12, C is generated by some of the vectors y1 , . . . , yt , such that for each i, yi is the solution to a system M y = b0 where M consists of

102

5 Integer Programming

n linearly independent rows of AI and b0 = ±e j for some unit vector e j . Set z i := | det M|yi . By Cramer’s rule, z i is integral with ||z i ||∞ ≤ 4(A). Since this holds for each i, the set {z 1 , . . . , z t } has the required properties. A similar lemma will be used in the next section: Lemma 5.5. Each rational polyhedral cone C is generated by a finite set of integral vectors {a1 , . . . , at } such that each integral vector in C is a nonnegative integral combination of a1 , . . . , at . (Such a set is called a Hilbert basis for C.) Proof: Let C be generated by the integral vectors b1 , . . . , bk . Let a1 , . . . , at be all integral vectors in the polytope {λ1 b1 + . . . + λk bk : 0 ≤ λi ≤ 1 (i = 1, . . . , k)} We show that {a1 , . . . , at } is a Hilbert basis for C. They indeed generate C, because b1 , . . . , bk occur among the a1 , . . . , at . For any integral vector x ∈ C there are µ1 , . . . , µk ≥ 0 with x = µ1 b1 + . . . + µk bk = bµ1 cb1 + . . . + bµk cbk + (µ1 − bµ1 c)b1 + . . . + (µk − bµk c)bk , so x is a nonnegative integral combination of a1 , . . . , at .

An important basic fact in integer programming is that optimum integral and fractional solutions are not too far away from each other: Theorem 5.6. (Cook et al. [1986]) Let A be an integral m × n-matrix and b ∈ Rm , c ∈ Rn arbitrary vectors. Let P := {x : Ax ≤ b} and suppose that PI 6= ∅. (a) Suppose y is an optimum solution of max {cx : x ∈ P}. Then there exists an optimum integral solution z of max {cx : x ∈ PI } with ||z − y||∞ ≤ n 4(A). (b) Suppose y is a feasible integral solution of max {cx : x ∈ PI }, but not an optimal one. Then there exists a feasible integral solution z ∈ PI with cz > cy and ||z − y||∞ ≤ n 4(A). Proof: The proof is almost the same for both parts. Let first y ∈ P arbitrary. Let z ∗ ∈ P ∩ Zn be (a) an optimum solution of max {cx : x ∈ PI } (note that PI = {x : Ax ≤ bbc} I is a polyhedron by Theorem 5.1, and thus the maximum is attained) or (b) a vector with cz ∗ > cy. We split Ax ≤ b into two subsystems A1 x ≤ b1 , A2 x ≤ b2 such that A1 z ∗ ≥ A1 y and A2 z ∗ < A2 y. Then z ∗ − y belongs to the polyhedral cone C := {x : A1 x ≥ 0, A2 x ≤ 0}. C is generated by some vectors xi (i = 1, . . . , s). By Lemma 5.4, we may assume that xi is integral and ||xi ||∞ ≤ 4(A) for all i. ∗ ∗ Ps Since z − y ∈ C, there are nonnegative numbers λ1 , . . . , λs with z − y = i=1 λi x i . We may assume that at most n of the λi are nonzero.

5.1 The Integer Hull of a Polyhedron

103

For µ = (µ1 , . . . , µs ) with 0 ≤ µi ≤ λi (i = 1, . . . , s) we define ∗

z µ := z −

s X

µi xi = y +

i=1

s X

(λi − µi )xi

i=1

and observe that z µ ∈ P: the first representation of z µ implies A1 z µ ≤ A1 z ∗ ≤ b1 ; the second one implies A2 z µ ≤ A2 y ≤ b2 . Case 1: There is some i ∈ {1, . . . , s} with λi ≥ 1 and cxi > 0. Let z := y + xi . We have cz > cy, showing that this case cannot occur in case (a). In case (b), when y is integral, z is an integral solution of Ax ≤ b such that cz > cy and ||z − y||∞ = ||xi ||∞ ≤ 4(A). Case 2: For all i ∈ {1, . . . , s}, λi ≥ 1 implies cxi ≤ 0. Let z := z bλc = z ∗ −

s X

bλi c xi .

i=1

z is an integral vector of P with cz ≥ ||z − y||∞ ≤

s X

cz ∗

and

(λi − bλi c) ||xi ||∞ ≤ n 4(A).

i=1

Hence in both (a) and (b) this vector z does the job.

As a corollary we can bound the size of optimum solutions of integer programming problems: Corollary 5.7. If P = {x ∈ Qn : Ax ≤ b} is a rational polyhedron and max{cx : x ∈ PI } has an optimum solution, then it also has an optimum integral solution x with size(x) ≤ 12n(size(A) + size(b)). Proof: By Proposition 5.2 and Theorem 4.4, max{cx : x ∈ P} has an optimum solution y with size(y) ≤ 4n(size(A) + size(b)). By Theorem 5.6(a) there is an optimum solution x of max{cx : x ∈ PI } with ||x −y||∞ ≤ n 4(A). By Propositions 4.1 and 4.3 we have size(x) ≤

2 size(y) + 2n size(n 4(A))

≤ 8n(size(A) + size(b)) + 2n log n + 4n size(A) ≤ 12n(size(A) + size(b)). Theorem 5.6(b) implies the following: given any feasible solution of an integer program, optimality of a vector x can be checked simply by testing x + y for a finite set of vectors y that depend on the matrix A only. Such a finite test set (whose existence has been proved first by Graver [1975]) enables us to prove a fundamental theorem on integer programming:

104

5 Integer Programming

Theorem 5.8. (Wolsey [1981], Cook et al. [1986]) For each integral m × nmatrix A there exists an integral matrix M whose entries have absolute value at most n 2n 4(A)n , such that for each vector b ∈ Qm there exists a rational vector d with {x : Ax ≤ b} I = {x : M x ≤ d}. Proof: We may assume A 6= 0. Let C be the cone generated by the rows of A. Let L := {z ∈ Zn : ||z||∞ ≤ n4(A)}. For each K ⊆ L, consider the cone C K := C ∩ {y : zy ≤ 0 for all z ∈ K }. By the proof of Theorem 3.29 and Lemma 5.4, C K = {y : U y ≤ 0} for some integral matrix U (whose rows are generators of {x : Ax ≤ 0} and elements of K ) whose entries have absolute value at most n4(A). Hence, again by Lemma 5.4, there is a finite set G(K ) of integral vectors generating C K , each having components with absolute value at S most 4(U ) ≤ n!(n4(A))n ≤ n 2n 4(A)n . Let M be the matrix with rows K ⊆L G(K ). Since C∅ = C, we may assume that the rows of A are also rows of M. Now let b be some fixed vector. If Ax ≤ b has no solution, we can complete b to a vector d arbitrarily and have {x : M x ≤ d} ⊆ {x : Ax ≤ b} = ∅. If Ax ≤ b contains a solution, but no integral solution, we set b0 := b − A0 1l, where A0 arises from A by taking the absolute value of each entry. Then Ax ≤ b0 has no solution, since any such solution yields an integral solution of Ax ≤ b by rounding. Again, we complete b0 to d arbitrarily. Now we may assume that Ax ≤ b has an integral solution. For y ∈ C we define δ y := max {yx : Ax ≤ b, x integral} (by Corollary 3.28 this maximum is bounded for y ∈ C). It suffices to show that [ {x : Ax ≤ b} I = x : yx ≤ δ y for each y ∈ G(K ) . (5.1) K ⊆L

Here “⊆” is trivial. To show the converse, let c be any vector for which max {cx : Ax ≤ b, x integral} is bounded, and let x ∗ be a vector attaining this maximum. We show that cx ≤ cx ∗ for all x satisfying the inequalities on the right-hand side of (5.1). By Proposition 5.2 the LP max {cx : Ax ≤ b} is bounded, so by Corollary 3.28 we have c ∈ C. Let K¯ := {z ∈ L : A(x ∗ + z) ≤ b}. By definition cz ≤ 0 for all z ∈ K¯ , so c ∈ C K¯ . Thus there are nonnegative numbers λ y (y ∈ G( K¯ )) such that

5.2 Unimodular Transformations

c =

X

105

λ y y.

y∈G( K¯ )

Next we claim that

x∗

is an optimum solution for max {yx : Ax ≤ b, x integral}

for each y ∈ G( K¯ ): the contrary assumption would, by Theorem 5.6(b), yield a vector z ∈ K¯ with yz > 0, which is impossible since y ∈ C K¯ . We conclude that X X X λy δy = λ y yx ∗ = λ y y x ∗ = cx ∗ . y∈G( K¯ )

y∈G( K¯ )

y∈G( K¯ )

Thus the inequality cx ≤ cx ∗ is a nonnegative linear combination of the inequalities yx ≤ δ y for y ∈ G( K¯ ). Hence (5.1) is proved. See Lasserre [2004] for a similar result.

5.2 Unimodular Transformations In this section we shall prove two lemmas for later use. A square matrix is called unimodular if it is integral and has determinant 1 or −1. Three types of unimodular matrices will be of particular interest: For n ∈ N, p ∈ {1, . . . , n} and q ∈ {1, . . . , n} \ { p} consider the matrices (ai j )i, j∈{1,...,n} defined in one of the following ways: if i = j 6= p / { p, q} 1 1 if i = j ∈ ai j = 1 if {i, j} = { p, q} ai j = −1 if i = j = p 0 otherwise 0 otherwise if i = j 1 ai j = −1 if (i, j) = ( p, q) 0 otherwise These matrices are evidently unimodular. If U is one of the above matrices, then replacing an arbitrary matrix A (with n columns) by AU is equivalent to applying one of the following elementary column operations to A: • • •

multiply a column by −1; exchange two columns; subtract one column from another column.

A series of the above operations is called a unimodular transformation. Obviously the product of unimodular matrices is unimodular. It can be shown that a matrix is unimodular if and only if it arises from an identity matrix by a unimodular transformation (equivalently, it is the product of matrices of the above three types); see Exercise 6. Here we do not need this fact.

106

5 Integer Programming

Proposition 5.9. The inverse of a unimodular matrix is also unimodular. For each unimodular matrix U the mappings x 7→ U x and x 7→ xU are bijections on Zn . Proof: Let U be a unimodular matrix. By Cramer’s rule the inverse of a unimodular matrix is integral. Since (det U )(det U −1 ) = det(UU −1 ) = det I = 1, U −1 is also unimodular. The second statement follows directly from this. Lemma 5.10. For each rational matrix A whose rows are linearly independent there exists a unimodular matrix U such that AU has the form (B 0), where B is a nonsingular square matrix. Proof: Suppose we have found a unimodular matrix U such that ! B 0 AU = C D for some nonsingular square matrix B. (Initially U = I , D = A, and the parts B, C and 0 have no entries.) Let (δ1 , . . . , δk ) be the first P row of D. Apply unimodular transformations such k δi is minimum. W.l.o.g. δ1 ≥ δ2 ≥ · · · ≥ δk . that all δi are nonnegative and i=1 Then δ1 > 0 since the rows of A (and hence those of AU ) are linearly independent. If Pδk2 > 0, then subtracting the second column of D from the first one would decrease i=1 δi . So δ2 = δ3 = . . . = δk = 0. We can expand B by one row and one column and continue. Note that the operations applied in the proof correspond to the E UCLIDEAN A LGORITHM. The matrix B we get is in fact a lower diagonal matrix. With a little more effort one can obtain the so-called Hermite normal form of A. The following lemma gives a criterion for integral solvability of equation systems, similar to Farkas’ Lemma. Lemma 5.11. Let A be a rational matrix and b a rational column vector. Then Ax = b has an integral solution if and only if yb is an integer for each rational vector y for which y A is integral. Proof: Necessity is obvious: if x and y A are integral vectors and Ax = b, then yb = y Ax is an integer. To prove sufficiency, suppose yb is an integer whenever y A is integral. We may assume that Ax = b contains no redundant equalities, i.e. y A = 0 implies yb 6= 0 for all y 6= 0. Let m be the number of rows of A. If rank(A) < m then {y : y A = 0} contains a nonzero vector y 0 and y 00 := 2y10 b y 0 satisfies y 00 A = 0 and / Z. So the rows of A are linearly independent. y 00 b = 12 ∈ By Lemma 5.10 there exists a unimodular matrix U with AU = (B 0), where B is a nonsingular m × m-matrix. Since B −1 AU = (I 0) is an integral matrix, we have for each row y of B −1 that y AU is integral and thus by Proposition 5.9 −1 −1 y A is integral. Hence yb is an integer for each row y of B , implying that B b −1 is an integral vector. So U B 0 b is an integral solution of Ax = b.

5.3 Total Dual Integrality

107

5.3 Total Dual Integrality In this and the next section we focus on integral polyhedra: Definition 5.12. A polyhedron P is integral if P = PI . Theorem 5.13. (Hoffman [1974], Edmonds and Giles [1977]) Let P be a rational polyhedron. Then the following statements are equivalent: (a) (b) (c) (d) (e) (f)

P is integral. Each face of P contains integral vectors. Each minimal face of P contains integral vectors. Each supporting hyperplane of P contains integral vectors. Each rational supporting hyperplane of P contains integral vectors. max {cx : x ∈ P} is attained by an integral vector for each c for which the maximum is finite. (g) max {cx : x ∈ P} is an integer for each integral c for which the maximum is finite. Proof: We first prove (a)⇒(b)⇒(f)⇒(a), then (b)⇒(d)⇒(e)⇒(c)⇒(b), and finally (f)⇒(g)⇒(e). (a)⇒(b): Let F be a face, say F = P ∩ H , where H is a supporting hyperplane, and let x ∈ F. If P = PI , then x is a convex combination of integral points in P, and these must belong to H and thus to F. (b)⇒(f) follows directly from Proposition 3.4, because {y ∈ P : cy = max {cx : x ∈ P}} is a face of P for each c for which the maximum is finite. (f)⇒(a): Suppose there is a vector y ∈ P\PI . Then (since PI is a polyhedron by Theorem 5.1) there is an inequality ax ≤ β valid for PI for which ay > β. Then clearly (f) is violated, since max {ax : x ∈ P} (which is finite by Proposition 5.2) is not attained by any integral vector. (b)⇒(d) is also trivial since the intersection of a supporting hyperplane with P is a face of P. (d)⇒(e) and (c)⇒(b) are trivial. (e)⇒(c): Let P = {x : Ax ≤ b}. We may assume that A and b are integral. Let F = {x : A0 x = b0 } be a minimal face of P, where A0 x ≤ b0 is a subsystem of Ax ≤ b (we use Proposition 3.9). If A0 x = b0 has no integral solution, then – by Lemma 5.11 – there exists a rational vector y such that c := y A0 is integral but δ := yb0 is not an integer. Adding integers to components of y does not destroy this property (A0 and b0 are integral), so we may assume that all components of y are positive. Observe that H := {x : cx = δ} is a rational hyperplane which contains no integral vectors. We finally show that H is a supporting hyperplane by proving that H ∩ P = F. Since F ⊆ H is trivial, it remains to show that H ∩ P ⊆ F. But for x ∈ H ∩ P we have y A0 x = cx = δ = yb0 , so y(A0 x − b0 ) = 0. Since y > 0 and A0 x ≤ b0 , this implies A0 x = b0 , so x ∈ F. (f)⇒(g) is trivial, so we finally show (g)⇒(e). Let H = {x : cx = δ} be a rational supporting hyperplane of P, so max{cx : x ∈ P} = δ. Suppose H contains

108

5 Integer Programming

no integral vectors. Then – by Lemma 5.11 – there exists a number γ such that γ c is integral but γ δ ∈ / Z. Then max{(|γ |c)x : x ∈ P} = |γ | max{cx : x ∈ P} = |γ |δ ∈ / Z,

contradicting our assumption.

See also Gomory [1963], Fulkerson [1971] and Chv´atal [1973] for earlier partial results. By (a)⇔(b) and Corollary 3.6 every face of an integral polyhedron is integral. The equivalence of (f) and (g) of Theorem 5.13 motivated Edmonds and Giles to define TDI-systems: Definition 5.14. (Edmonds and Giles [1977]) A system Ax ≤ b of linear inequalities is called totally dual integral (TDI) if the minimum in the LP duality equation max {cx : Ax ≤ b} = min {yb : y A = c, y ≥ 0} has an integral optimum solution y for each integral vector c for which the minimum is finite. With this definition we get an easy corollary of (g)⇒(a) of Theorem 5.13: Corollary 5.15. Let Ax ≤ b be a TDI-system where A is rational and b is integral. Then the polyhedron {x : Ax ≤ b} is integral. But total dual integrality is not a property of polyhedra (cf. Exercise 8). In general, a TDI-system contains more inequalities than necessary for describing the polyhedron. Adding valid inequalities does not destroy total dual integrality: Proposition 5.16. If Ax ≤ b is TDI and ax ≤ β is a valid inequality for {x : Ax ≤ b}, then the system Ax ≤ b, ax ≤ β is also TDI. Proof: Let c be an integral vector such that min {yb + γβ : y A + γ a = c, y ≥ 0, γ ≥ 0} is finite. Since ax ≤ β is valid for {x : Ax ≤ b}, min {yb : y A = c, y ≥ 0} = max {cx : Ax ≤ b} = max {cx : Ax ≤ b, ax ≤ β} = min {yb + γβ : y A + γ a = c, y ≥ 0, γ ≥ 0}. The first minimum is attained by some integral vector y ∗ , so y = y ∗ , γ = 0 is an integral optimum solution for the second minimum. Theorem 5.17. (Giles and Pulleyblank [1979]) For each rational polyhedron P there exists a rational TDI-system Ax ≤ b with A integral and P = {x : Ax ≤ b}. Here b can be chosen to be integral if and only if P is integral.

5.3 Total Dual Integrality

109

Proof: Let P = {x : C x ≤ d} with C and d integral. W.l.o.g., P 6= ∅. For each minimal face F of P let K F := {c : cz = max {cx : x ∈ P} for all z ∈ F}. By Corollary 3.22 and Theorem 3.29, K F is a rational polyhedral cone. By Lemma 5.5 there exists an integral Hilbert basis a1 , . . . , at generating K F . Let S F be the system of inequalities a1 x ≤ max {a1 x : x ∈ P} , . . . , at x ≤ max {at x : x ∈ P}. Let Ax ≤ b be the collection of all these systems S F (for all minimal faces F). Note that if P is integral then b is integral. Moreover, P ⊆ {x : Ax ≤ b}. Let c be an integral vector for which max {cx : x ∈ P} is finite. The set of vectors attaining this maximum is a face of P, so let F be a minimal face such that cz = max {cx : x ∈ P} for all z ∈ F. Let S F be the system a1 x ≤ β1 , . . . , at x ≤ βt . Then c = λ1 a1 + · · · + λt at for some nonnegative integers λ1 , . . . , λt . We add zero ¯ = c and components to λ1 , . . . , λt in order to get an integral vector λ¯ ≥ 0 with λA ¯ ¯ ¯ ¯ ¯ thus cx = (λA)x = λ(Ax) ≤ λb = λ(Az) = (λA)z = cz for all x with Ax ≤ b and all z ∈ F. Applying this for each row c of C yields C x ≤ d for all x with Ax ≤ b; and thus P = {x : Ax ≤ b}. Moreover, for general c we conclude that λ¯ is an optimum solution of the dual LP min {yb : y ≥ 0, y A = c}. Hence Ax ≤ b is TDI. If P is integral, we have chosen b to be integral. Conversely, if b can be chosen integral, by Corollary 5.15 P must be integral. Indeed, for full-dimensional rational polyhedra there is a unique minimal TDIsystem describing it (Schrijver [1981]). For later use, we prove that each “face” of a TDI-system is again TDI: Theorem 5.18. (Cook [1983]) Let Ax ≤ b, ax ≤ β be a TDI-system, where a is integral. Then the system Ax ≤ b, ax = β is also TDI. Proof: (Schrijver [1986]) Let c be an integral vector such that max {cx : Ax ≤ b, ax = β} = min {yb + (λ − µ)β : y, λ, µ ≥ 0, y A + (λ − µ)a = c}

(5.2)

is finite. Let x ∗ , y ∗ , λ∗ , µ∗ attain these optima. We set c0 := c +dµ∗ ea and observe that max {c0 x : Ax ≤ b, ax ≤ β} = min {yb + λβ : y, λ ≥ 0, y A + λa = c0 } (5.3) is finite, because x := x ∗ is feasible for the maximum and y := y ∗ , λ := λ∗ + dµ∗ e − µ∗ is feasible for the minimum. Since Ax ≤ b, ax ≤ β is TDI, the minimum in (5.3) has an integral optimum solution y˜ , λ˜ . We finally set y := y˜ , λ := λ˜ and µ := dµ∗ e and claim that (y, λ, µ) is an integral optimum solution for the minimum in (5.2).

110

5 Integer Programming

Obviously (y, λ, µ) is feasible for the minimum in (5.2). Furthermore, ˜ − dµ∗ eβ yb + (λ − µ)β = y˜ b + λβ ∗ ≤ y b + (λ∗ + dµ∗ e − µ∗ )β − dµ∗ eβ ˜ is an since (y ∗ , λ∗ + dµ∗ e − µ∗ ) is feasible for the minimum in (5.3), and ( y˜ , λ) optimum solution. We conclude that yb + (λ − µ)β ≤ y ∗ b + (λ∗ − µ∗ )β, proving that (y, λ, µ) is an integral optimum solution for the minimum in (5.2). The following statements are straightforward consequences of the definition of TDI-systems: A system Ax = b, x ≥ 0 is TDI if min {yb : y A ≥ c} has an integral optimum solution y for each integral vector c for which the minimum is finite. A system Ax ≤ b, x ≥ 0 is TDI if min {yb : y A ≥ c, y ≥ 0} has an integral optimum solution y for each integral vector c for which the minimum is finite. One may ask whether there are matrices A such that Ax ≤ b, x ≥ 0 is TDI for each integral vector b. It will turn out that these matrices are exactly the totally unimodular matrices.

5.4 Totally Unimodular Matrices Definition 5.19. A matrix A is totally unimodular if each subdeterminant of A is 0, +1, or −1. In particular, each entry of a totally unimodular matrix must be 0, +1, or −1. The main result of this section is: Theorem 5.20. (Hoffman and Kruskal [1956]) An integral matrix A is totally unimodular if and only if the polyhedron {x : Ax ≤ b, x ≥ 0} is integral for each integral vector b. Proof: Let A be an m × n-matrix and P := {x : Ax ≤ b, x ≥ 0}. Observe that the minimal faces of P are vertices. To prove necessity, suppose that A is totally unimodular. Let b be some integral vector and x a vertex of P. x is the solution of A0 x = b0 for some subsystem A x ≤ b , with A0 being a nonsingular n × n-matrix. Since A is A0 x ≤ b0 of −I 0 totally unimodular, | det A0 | = 1, so by Cramer’s rule x = (A0 )−1 b0 is integral. We now prove sufficiency. Suppose that the vertices of P are integral for each integral vector b. Let A0 be some nonsingular k × k-submatrix of A. We have to show | det A0 | = 1. W.l.o.g., A0 contains the elements of the first k rows and columns of A. Consider the integral m × m-matrix B consisting of the first k and the last m − k columns of (A I ) (see Figure 5.2). Obviously, | det B| = | det A0 |.

5.4 Totally Unimodular Matrices n−k

k A

k

k

m−k

I

0

111

(A I ) m−k

0

0

z

I

0

z

Fig. 5.2.

To prove | det B| = 1, we shall prove that B −1 is integral. Since det B det B −1 = 1, this implies that | det B| = 1, and we are done. Let i ∈ {1, . . . , m}; we prove that B −1 ei is integral. Choose an integral vector y such that z := y + B −1 ei ≥ 0. Then b := Bz = By + ei is integral. We add zero components to z in order to obtain z with A I z = Bz = b. Now z , consisting of the ﬁrst n components of z , belongs to P. Furthermore, n linearly independent constraints are satisﬁed with equality, namely the ﬁrst k and the last n − k inequalities of A b z ≤ . −I 0 Hence z is a vertex of P. By our assumption z is integral. But then z must also be integral: its ﬁrst n components are the components of z , and the last m components are the slack variables b − Az (and A and b are integral). So z is also integral, and hence B −1 ei = z − y is integral. The above proof is due to Veinott and Dantzig [1968]. Corollary 5.21. An integral matrix A is totally unimodular if and only if for all integral vectors b and c both optima in the LP duality equation max {cx : Ax ≤ b, x ≥ 0} = min {yb : y ≥ 0, y A ≥ c} are attained by integral vectors (if they are ﬁnite).

112

5 Integer Programming

Proof: This follows from the Hoffman-Kruskal Theorem 5.20 by using the fact that the transpose of a totally unimodular matrix is also totally unimodular. Let us reformulate these statements in terms of total dual integrality: Corollary 5.22. An integral matrix A is totally unimodular if and only if the system Ax ≤ b, x ≥ 0 is TDI for each vector b. Proof: If A (and thus A> ) is totally unimodular, then by the Hoffman-Kruskal Theorem min {yb : y A ≥ c, y ≥ 0} is attained by an integral vector for each vector b and each integral vector c for which the minimum is finite. In other words, the system Ax ≤ b, x ≥ 0 is TDI for each vector b. To show the converse, suppose Ax ≤ b, x ≥ 0 is TDI for each integral vector b. Then by Corollary 5.15, the polyhedron {x : Ax ≤ b, x ≥ 0} is integral for each integral vector b. By Theorem 5.20 this means that A is totally unimodular. This is not the only way how total unimodularity can be used to prove that a certain system is TDI. The following lemma contains another proof technique; this will be used several times later (Theorems 6.13, 19.10 and 14.12). Lemma 5.23. Let Ax ≤ b, x ≥ 0 be an inequality system, where A ∈ Rm×n and b ∈ Rm . Suppose that for each c ∈ Zn for which min{yb : y A ≥ c, y ≥ 0} has an optimum solution, it has one y ∗ such that the rows of A corresponding to nonzero components of y ∗ form a totally unimodular matrix. Then Ax ≤ b, x ≥ 0 is TDI. Proof: Let c ∈ Zn , and let y ∗ be an optimum solution of min{yb : y A ≥ c, y ≥ 0} such that the rows of A corresponding to nonzero components of y ∗ form a totally unimodular matrix A0 . We claim that min{yb : y A ≥ c, y ≥ 0} = min{yb0 : y A0 ≥ c, y ≥ 0},

(5.4)

where b0 consists of the components of b corresponding to the rows of A0 . To see the inequality “≤” of (5.4), observe that the LP on the right-hand side arises from the LP on the left-hand side by setting some variables to zero. The inequality “≥” follows from the fact that y ∗ without zero components is a feasible solution for the LP on the right-hand side. Since A0 is totally unimodular, the second minimum in (5.4) has an integral optimum solution (by the Hoffman-Kruskal Theorem 5.20). By filling this solution with zeros we obtain an integral optimum solution to the first minimum in (5.4), completing the proof. A very useful criterion for total unimodularity is the following: Theorem 5.24. (Ghouila-Houri [1962]) A matrix A = (ai j ) ∈ Zm×n is totally . unimodular if and only if for every R ⊆ {1, . . . , m} there is a partition R = R1 ∪ R2 such that

5.4 Totally Unimodular Matrices

X i∈R1

ai j −

X

113

ai j ∈ {−1, 0, 1}

i∈R2

for all j = 1, . . . , n. Proof: Let A be totally unimodular, and let R ⊆ {1, . . . , m}. Let dr := 1 for A> r ∈ R and dr := 0 for r ∈ {1, . . . , m} \ R. The matrix −A> is also totally I

unimodular, so by Theorem 5.20 the polytope 1 1 dA , xA ≥ d A , x ≤ d, x ≥ 0 x : xA ≤ 2 2 is integral. Moreover it is nonempty because it contains 12 d. So it has an integral vertex, say z. Setting R1 := {r ∈ R : zr = 0} and R2 := {r ∈ R : zr = 1} we obtain X X ai j − ai j = (d − 2z)A ∈ {−1, 0, 1}n , i∈R1

i∈R2

1≤ j≤n

as required. We now prove the converse. By induction on k we prove that every k × ksubmatrix has determinant 0, 1 or −1. For k = 1 this is directly implied by the criterion for |R| = 1. Now let k > 1, and let B = (bi j )i, j∈{1,...,k} be a nonsingular k × k-submatrix of B0 0 A. By Cramer’s rule, each entry of B −1 is det det B , where B arises from B by replacing a column by a unit vector. By the induction hypothesis, det B 0 ∈ {−1, 0, 1}. So B ∗ := (det B)B −1 is a matrix with entries −1, 0, 1 only. Let b1∗ be the first row of B ∗ . We have b1∗ B = (det B)e1 , where e1 is the first ∗ 6= 0}. Then for j = 2, . . . , k we have 0 = (b∗ B) = unit vector. Let R := {i : b1i j 1 P ∗ b b , so |{i ∈ R : b = 6 0}| is even. i j i j i∈R 1i . P ∪ R2 with Pi∈R1 bi j − By the hypothesis there is a partition R = R 1 P for all j. SoPfor j = 2, . . . , k we have i∈R1 bi j − Pi∈R2 bi j ∈ {−1, 0, 1} P b = 0. If also b − b = 0, then the sum of the rows i∈R2 i j i∈R1 i1 i∈R2 i1 in R1 equals the sum of the rows in R2 , contradicting the assumption that B is nonsingular P (becausePR 6= ∅). So i∈R1 bi1 − i∈R2 bi1 ∈ {−1, 1} and we have y B ∈ {e1 , −e1 }, where if i ∈ R1 1 yi := −1 if i ∈ R2 . 0 if i 6∈ R Since b1∗ B = (det B)e1 and B is nonsingular, we have b1∗ ∈ {(det B)y, −(det B)y}. Since both y and b1∗ are nonzero vectors with entries −1, 0, 1 only, this implies that | det B| = 1. We apply this criterion to the incidence matrices of graphs:

114

5 Integer Programming

Theorem 5.25. The incidence matrix of an undirected graph G is totally unimodular if and only if G is bipartite. Proof: By Theorem 5.24 the incidence matrix M of G is totally unimodular . if and only if for any X ⊆ V (G) there is a partition X = A ∪ B such that E(G[A]) = E(G[B]) = ∅. By definition, such a partition exists iff G[X ] is bipartite. Theorem 5.26. The incidence matrix of any digraph is totally unimodular. Proof: Using Theorem 5.24, it suffices to set R1 := R and R2 := ∅ for any R ⊆ V (G). Applications of Theorems 5.25 and 5.26 will be discussed in later chapters. Theorem 5.26 has an interesting generalization to cross-free families: Definition 5.27. Let G be a digraph and F a family of subsets of V (G). The one-way cut-incidence matrix of F is the matrix M = (m X,e ) X ∈F , e∈E(G) where ( 1 if e ∈ δ + (X ) m X,e = . 0 if e ∈ / δ + (X ) The two-way cut-incidence matrix of F where −1 m X,e = 1 0

is the matrix M = (m X,e ) X ∈F , e∈E(G) if e ∈ δ − (X ) if e ∈ δ + (X ) . otherwise

Theorem 5.28. Let G be a digraph and (V (G), F) a cross-free set system. Then the two-way cut-incidence matrix of F is totally unimodular. If F is laminar, then also the one-way cut-incidence matrix of F is totally unimodular. Proof: Let F be some cross-free family of subsets of V (G). We first consider the case when F is laminar. We use Theorem 5.24. To see that the criterion is satisfied, let R ⊆ F, and consider the tree-representation (T, ϕ) of R, where T is an arborescence rooted at r (Proposition 2.14). With the notation of Definition 2.13, R = {Se : e ∈ E(T )}. Set R1 := {S(v,w) ∈ R : distT (r, w) even} and R2 := R \ R1 . Now for any edge f ∈ E(G), the edges e ∈ E(T ) with f ∈ δ + (Se ) form a path Pf in T (possibly of zero length). So |{X ∈ R1 : f ∈ δ + (X )}| − |{X ∈ R2 : f ∈ δ + (X )}| ∈ {−1, 0, 1}, as required for the one-way cut-incidence matrix. Moreover, for any edge f the edges e ∈ E(T ) with f ∈ δ − (Se ) form a path Q f in T . Since Pf and Q f have a common endpoint, we have

5.5 Cutting Planes

115

|{X ∈ R1 : f ∈ δ + (X )}| − |{X ∈ R2 : f ∈ δ + (X )}| −|{X ∈ R1 : f ∈ δ − (X )}| + |{X ∈ R2 : f ∈ δ − (X )}| ∈ {−1, 0, 1}, as required for the two-way cut-incidence matrix. Now if (V (G), F) is a general cross-free set system, consider F 0 := {X ∈ F : r 6∈ X } ∪ {V (G) \ X : X ∈ F, r ∈ X } for some fixed r ∈ V (G). F 0is laminar. Since the two-way cut-incidence matrix M , where M is the two-way cut-incidence matrix of of F is a submatrix of −M 0 F , it is totally unimodular, too. For general cross-free families the one-way cut-incidence matrix is not totally unimodular; see Exercise 13. For a necessary and sufficient condition, see Schrijver [1983]. The two-way cut-incidence matrices of cross-free families are also known as network matrices (Exercise 14). Seymour [1980] showed that all totally unimodular matrices can be constructed in a certain way from these network matrices and two other totally unimodular matrices. This deep result implies a polynomial-time algorithm which decides whether a given matrix is totally unimodular (see Schrijver [1986]).

5.5 Cutting Planes In the previous sections we considered integral polyhedra. For general polyhedra P we have P ⊃ PI . If we want to solve an integer linear program max {cx : x ∈ PI }, it is a natural idea to cut off certain parts of P such that the resulting set is again a polyhedron P 0 and we have P ⊃ P 0 ⊃ PI . Hopefully max {cx : x ∈ P 0 } is attained by an integral vector; otherwise we can repeat this cutting-off procedure for P 0 in order to obtain P 00 and so on. This is the basic idea behind the cutting plane method, first proposed for a special problem (the TSP) by Dantzig, Fulkerson and Johnson [1954]. Gomory [1958, 1963] found an algorithm which solves general integer programs with the cutting plane method. In this section we restrict ourselves to the theoretical background. Gomory’s algorithm does not run in polynomial time and has little practical relevance in its original form. However, the general idea of cutting plane methods is used very often and successfully in practice. We shall discuss this in Section 21.6. The following presentation is mainly based on Schrijver [1986]. Definition 5.29. Let P = {x : Ax ≤ b} be a polyhedron. Then we define \ HI , P 0 := P⊆H

where the intersection ranges over all rational affine half-spaces H = {x : cx ≤ δ} 0 containing P. We set P (0) := P and P (i+1) := P (i) . P (i) is called the i-th Gomory-Chv´atal-truncation of P.

116

5 Integer Programming

For a rational polyhedron P we obviously have P ⊇ P 0 ⊇ P (2) ⊇ · · · ⊇ PI and PI = (P 0 ) I . Proposition 5.30. For any rational polyhedron P = {x : Ax ≤ b}, P 0 = {x : u Ax ≤ bubc for all u ≥ 0 with u A integral }. Proof: We first make two observations. For any rational affine half-space H = {x : cx ≤ δ} with c integral we obviously have H 0 = H I ⊆ {x : cx ≤ bδc}.

(5.5)

If in addition the components of c are relatively prime, we claim that H 0 = H I = {x : cx ≤ bδc}.

(5.6)

To prove (5.6), let c be an integral vector whose components are relatively prime. By Lemma 5.11 the hyperplane {x : cx = bδc} contains an integral vector y. For any rational vector x ∈ {x : cx ≤ bδc} let α ∈ N such that αx is integral. Then we can write x =

α−1 1 (αx − (α − 1)y) + y, α α

i.e. x is a convex combination of integral points in H . Hence x ∈ H I , implying (5.6). We now turn to the main proof. To see “⊆”, observe that for any u ≥ 0, {x : u Ax ≤ ub} is a half-space containing P, so by (5.5) P 0 ⊆ {x : u Ax ≤ bubc} if u A is integral. We now prove “⊇”. For P = ∅ this is easy, so we assume P 6= ∅. Let H = {x : cx ≤ δ} be some rational affine half-space containing P. W.l.o.g. c is integral and the components of c are relatively prime. We observe that δ ≥ max {cx : Ax ≤ b} = min {ub : u A = c, u ≥ 0}. Now let u ∗ be any optimum solution for the minimum. Then for any z ∈ {x : u Ax ≤ bubc for all u ≥ 0 with u A integral } ⊆ {x : u ∗ Ax ≤ bu ∗ bc} we have: cz = u ∗ Az ≤ bu ∗ bc ≤ bδc which, using (5.6), implies z ∈ H I .

Below we shall prove that for any rational polyhedron P there is a number t with PI = P (t) . So Gomory’s cutting plane method successively solves the linear programs over P, P 0 , P 00 , and so on, until the optimum is integral. At each step only a finite number of new inequalities have to be added, namely those corresponding to a TDI-system defining the current polyhedron (recall Theorem 5.17):

5.5 Cutting Planes

117

Theorem 5.31. (Schrijver [1980]) Let P = {x : Ax ≤ b} be a polyhedron with Ax ≤ b TDI, A integral and b rational. Then P 0 = {x : Ax ≤ bbc}. In particular, for any rational polyhedron P, P 0 is a polyhedron again. Proof: The statement is trivial if P is empty, so let P 6= ∅. Obviously P 0 ⊆ {x : Ax ≤ bbc}. To show the other inclusion, let u ≥ 0 be a vector with u A integral. By Proposition 5.30 it suffices to show that u Ax ≤ bubc for all x with Ax ≤ bbc. We know that ub ≥ max {u Ax : Ax ≤ b} = min {yb : y ≥ 0, y A = u A}. Since Ax ≤ b is TDI, the minimum is attained by some integral vector y ∗ . Now Ax ≤ bbc implies u Ax = y ∗ Ax ≤ y ∗ bbc ≤ by ∗ bc ≤ bubc. The second statement follows from Theorem 5.17.

To prove the main theorem of this section, we need two more lemmas: Lemma 5.32. If F is a face of a rational polyhedron P, then F 0 = P 0 ∩ F. More generally, F (i) = P (i) ∩ F for all i ∈ N. Proof: Let P = {x : Ax ≤ b} with A integral, b rational, and Ax ≤ b TDI (recall Theorem 5.17). Now let F = {x : Ax ≤ b, ax = β} be a face of P, where ax ≤ β is a valid inequality for P with a and β integral. By Proposition 5.16, Ax ≤ b, ax ≤ β is TDI, so by Theorem 5.18, Ax ≤ b, ax = β is also TDI. As β is an integer, P 0 ∩ F = {x : Ax ≤ bbc, ax = β} = {x : Ax ≤ bbc, ax ≤ bβc, ax ≥ dβe} = F 0. Here we have used Theorem 5.31 twice. Note that F 0 is either empty or a face of P 0 . Now the statement follows by induction on i: For each i, F (i) is either empty or a face of P (i) , and F (i) = P (i) ∩ F (i−1) = P (i) ∩ (P (i−1) ∩ F) = P (i) ∩ F. Lemma 5.33. Let P be a polyhedron in Rn and U a unimodular n × n-matrix. Write f (P) := {U x : x ∈ P}. Then f (P) is again a polyhedron. Moreover, if P is a rational polyhedron, then ( f (P))0 = f (P 0 ) and ( f (P)) I = f (PI ). Proof: Since f : Rn → Rn , x 7→ U x is a bijective linear function, the first statement is obviously true. Since also the restrictions of f and f −1 to Zn are bijections (by Proposition 5.9) we have

118

5 Integer Programming

( f (P)) I

= conv({y ∈ Zn : y = U x, x ∈ P}) = conv({y ∈ Rn : y = U x, x ∈ P, x ∈ Zn }) = conv({y ∈ Rn : y = U x, x ∈ PI }) =

f (PI ).

Let P = {x : Ax ≤ b} with Ax ≤ b TDI, A integral, b rational (cf. Theorem 5.17). Then by definition AU −1 x ≤ b is also TDI. Using Theorem 5.31 twice we get ( f (P))0 = {x : AU −1 x ≤ b}0 = {x : AU −1 x ≤ bbc} = f (P 0 ). Theorem 5.34. (Schrijver [1980]) For each rational polyhedron P there exists a number t such that P (t) = PI . Proof: Let P be a rational polyhedron in Rn . We prove the theorem by induction on n + dim P. The case P = ∅ is trivial, the case dim P = 0 is easy. First suppose that P is not full-dimensional. Then P ⊆ K for some rational hyperplane K . If K contains no integral vectors, K = {x : ax = β} for some integral vector a and some non-integer β (by Lemma 5.11). But then P 0 ⊆ {x : ax ≤ bβc, ax ≥ dβe} = ∅ = PI . If K contains integral vectors, say K = {x : ax = β} with a integral, β an integer, we may assume β = 0, because the theorem is invariant under translations by integral vectors. By Lemma 5.10 there exists a unimodular matrix U with aU = αe1 . Since the theorem is also invariant under the transformation x 7→ U −1 x (by Lemma 5.33), we may assume a = αe1 . Then the first component of each vector in P is zero, and thus we can reduce the dimension of the space by one and apply the induction hypothesis (observe that ({0} × Q) I = {0} × Q I and ({0} × Q)(t) = {0} × Q (t) for any polyhedron Q in Rn−1 and any t ∈ N). Let now P = {x : Ax ≤ b} be full-dimensional, and w.l.o.g. A integral. By Theorem 5.1 there is some integral matrix C and some vector d with PI = {x : C x ≤ d}. In the case PI = ∅ we set C := A and d := b− A0 1l, where A0 arises from A by taking the absolute value of each entry. (Note that {x : Ax ≤ b − A0 1l} = ∅.) Let cx ≤ δ be an inequality of C x ≤ d. We claim that P (s) ⊆ H := {x : cx ≤ δ} for some s ∈ N. This claim obviously implies the theorem. First observe that there is some β ≥ δ such that P ⊆ {x : cx ≤ β}: in the case PI = ∅ this follows from the choice of C and d; in the case PI 6= ∅ this follows from Proposition 5.2. Suppose our claim is false, i.e. there is an integer γ with δ < γ ≤ β for which there exists an s0 ∈ N with P (s0 ) ⊆ {x : cx ≤ γ }, but there is no s ∈ N with P (s) ⊆ {x : cx ≤ γ − 1}. Observe that max{cx : x ∈ P (s) } = γ for all s ≥ s0 , because if max{cx : x ∈ (s) P } < γ for some s, then P (s+1) ⊆ {x : cx ≤ γ − 1}.

5.6 Lagrangean Relaxation

119

Let F := P (s0 ) ∩ {x : cx = γ }. F is a face of P (s0 ) , and dim F < n = dim P. By the induction hypothesis, there is a number s1 such that F (s1 ) = FI ⊆ PI ∩ {x : cx = γ } = ∅. By applying Lemma 5.32 to F and P (s0 ) we obtain ∅ = F (s1 ) = P (s0 +s1 ) ∩ F = P (s0 +s1 ) ∩ {x : cx = γ }. Hence max{cx : x ∈ P (s0 +s1 ) } < γ , a contradiction.

This theorem also implies the following: Theorem 5.35. (Chv´atal [1973]) For each polytope P there is a number t such that P (t) = PI . Proof: As P is bounded, there exists some rational polytope Q ⊇ P with Q I = PI (take a hypercube containing P and intersect it with a rational half-space containing P but not z for each integral point z which belongs to the hypercube but not to P; cf. Exercise 19 of Chapter 3). By Theorem 5.34, Q (t) = Q I for some t. Hence PI ⊆ P (t) ⊆ Q (t) = Q I = PI , implying P (t) = PI . This number t is called the Chv´atal rank of P. If P is neither bounded nor rational, one cannot have an analogous theorem: see Exercises 1 and 17. A more efficient algorithm which computes the integer hull of a two-dimensional polyhedron has been found by Harvey [1999]. A version of the cutting plane method which, in polynomial time, approximates a linear objective function over an integral polytope given by a separation oracle was described by Boyd [1997]. Cook, Kannan and Schrijver [1990] generalized the Gomory-Chv´atal procedure to mixed integer programming. Eisenbrand [1999] showed that it is coNP-complete to decide whether a given rational vector is in P 0 for a given rational polyhedron P.

5.6 Lagrangean Relaxation Suppose we have an integer linear program max{cx : Ax ≤ b, A0 x ≤ b0 , x integral} that becomes substantially easier to solve when the constraints A0 x ≤ b0 are omitted. We write Q := {x ∈ Zn : Ax ≤ b} and assume that we can optimize linear objective functions over Q (for example if conv(Q) = {x : Ax ≤ b}). Lagrangean relaxation is a technique to get rid of some troublesome constraints (in our case A0 x ≤ b0 ). Instead of explicitly enforcing the constraints we modify the objective function in order to punish infeasible solutions. More precisely, instead of optimizing max{c> x : A0 x ≤ b0 , x ∈ Q} (5.7) we consider, for any vector λ ≥ 0,

120

5 Integer Programming

L R(λ) := max{c> x + λ> (b0 − A0 x) : x ∈ Q}.

(5.8)

For each λ ≥ 0, L R(λ) is an upper bound for (5.7) which is relatively easy to compute. (5.8) is called the Lagrangean relaxation of (5.7), and the components of λ are called Lagrange multipliers. Lagrangean relaxation is a useful technique in nonlinear programming; but here we restrict ourselves to (integer) linear programming. Of course one is interested in as good an upper bound as possible. Observe that λ 7→ L R(λ) is a convex function. The following procedure (called subgradient optimization) can be used to minimize L R(λ): Start with an arbitrary vector λ(0) ≥ 0. In iteration i, given λ(i) , find a vector (i) x maximizing c> x + (λ(i) )> (b0 − A0 x) over Q (i.e. compute L R(λ(i) )). Note that L R(λ) − L R(λ(i) ) ≥ (λ − λ(i) )> (b0 − A0 x (i) ) for all λ, i.e. b0 − A0 x (i) is a − A0 x (i) )} for some subgradient of L R at λ(i) . Set λ(i+1) := max{0, λ(i) − ti (b0 P ∞ ti = ∞, then ti > 0. Polyak [1967] showed that if limi→∞ ti = 0 and i=0 (i) limi→∞ L R(λ ) = min{L R(λ) : λ ≥ 0}. For more results on the convergence of subgradient optimization, see (Goffin [1977]). The problem to find the best such upper bound, i.e. min{L R(λ) : λ ≥ 0}, is sometimes called the Lagrangean dual of (5.7). We will show that the minimum is always attained unless {x : Ax ≤ b, A0 x ≤ b0 } = ∅. The second question that we address is how good this upper bound is. Of course this depends on the structure of the original problem. In Section 21.5 we shall meet an application to the TSP, where Lagrangean relaxation is very effective. The following theorem helps to estimate the quality of the upper bound: Theorem 5.36. (Geoffrion [1974]) Let c ∈ Rn , A0 ∈ Rm×n and b0 ∈ Rm . Let Q ⊆ Rn such that conv(Q) is a polyhedron. Suppose that max{c> x : A0 x ≤ b0 , x ∈ conv(Q)} has an optimum solution. Let L R(λ) := max{c> x + λ> (b0 − A0 x) : x ∈ Q}. Then inf{L R(λ) : λ ≥ 0} (the optimum value of the Lagrangean dual of max{c> x : A0 x ≤ b0 , x ∈ Q}) is attained by some λ, and the minimum is equal to max{c> x : A0 x ≤ b0 , x ∈ conv(Q)}. Proof: Let conv(Q) = {x : Ax ≤ b}. By using the LP Duality Theorem 3.20 twice and reformulating we get = = = = = = =

max{c> x : x ∈ conv(Q), A0 x ≤ b0 } max{c> x : Ax ≤ b, A0 x ≤ b0 } min λ> b0 + y > b : y > A + λ> A0 = c> , y ≥ 0, λ ≥ 0 min λ> b0 + min{y > b : y > A = c> − λ> A0 , y ≥ 0} : λ ≥ 0 min λ> b0 + max{(c> − λ> A0 )x : Ax ≤ b} : λ ≥ 0 min max{c> x + λ> (b0 − A0 x) : x ∈ conv(Q)} : λ ≥ 0 min max{c> x + λ> (b0 − A0 x) : x ∈ Q} : λ ≥ 0 min{L R(λ) : λ ≥ 0}.

Exercises

The third line, an LP, shows that there is a λ attaining the minimum.

121

In particular, if we have an integer linear program max{cx : A0 x ≤ b0 , Ax ≤ b, x integral} where {x : Ax ≤ b} is integral, then the Lagrangean dual (when relaxing A0 x ≤ b0 as above) yields the same upper bound as the standard LP relaxation max{cx : A0 x ≤ b0 , Ax ≤ b}. If {x : Ax ≤ b} is not integral, the upper bound is in general stronger (but can be difficult to compute). See Exercise 21 for an example. Lagrangean relaxation can also be used to approximate linear programs. For example, consider the J OB A SSIGNMENT P ROBLEM (see (1.1) in Section 1.3). The problem can be rewritten equivalently as X xi j ≥ ti (i = 1, . . . , n), (x, T ) ∈ P min T : (5.9) j∈Si

where P is the polytope (x, T ) : 0 ≤ xi j ≤ ti (i = 1, . . . , n, j ∈ Si ), X

xi j ≤ T ( j = 1, . . . , m),

i: j∈Si

T ≤

n X

ti

.

i=1

Now we apply Lagrangean relaxation and consider n X X λi ti − xi j : (x, T ) ∈ P . L R(λ) := min T + i=1

(5.10)

j∈Si

Because of its special structure this LP can be solved by a simple combinatorial algorithm (Exercise 23), for arbitrary λ. If we let Q be the set of vertices of P (cf. Corollary 3.32), then we can apply Theorem 5.36 and conclude that the optimum value of the Lagrangean dual max{L R(λ) : λ ≥ 0} equals the optimum of (5.9).

Exercises n √ o 1. Let P := (x, y) ∈ R2 : y ≤ 2x . Prove that PI is not a polyhedron. Show a polyhedron P where even the closure of PI is not a polyhedron. 2. Let P = {x ∈ Rk+l : Ax ≤ b} be a rational polyhedron. Show that conv(P ∩ (Zk × Rl )) is a polyhedron. Hint: Generalize the proof of Theorem 5.1. Note: This is the basis of mixed integer programming. See Schrijver [1986].

122

∗

∗

5 Integer Programming

3. Prove the following integer analogue of Carath´eodory’s theorem (Exercise 14 of Chapter 3): For each pointed polyhedral cone C = {x ∈ Qn : Ax ≤ 0}, each Hilbert basis {a1 , . . . , at } of C, and each integral point x ∈ C there are 2n − 1 vectors among a1 , . . . , at such that x is a nonnegative integer combination of those. Hint: Consider an optimum basic solution of the LP max{y1l : y A = x, y ≥ 0} and round the components down. Note: The number 2n − 1 was improved by Seb˝o [1990] to 2n − 2. It cannot be improved to less than b 76 nc (Bruns et al. [1999]). (Cook, Fonlupt and Schrijver [1986]) 4. Let C = {x : Ax ≥ 0} be a rational polyhedral cone and b some vector with bx > 0 for all x ∈ C \ {0}. Show that there exists a unique minimal integral Hilbert basis generating C. (Schrijver [1981]) 5. Let A be an integral m × n-matrix, and let b and c be vectors, and y an optimum solution of max {cx : Ax ≤ b, x integral}. Prove that there exists an optimum solution z of max {cx : Ax ≤ b} with ||y − z||∞ ≤ n4(A). (Cook et al. [1986]) 6. Prove that each unimodular matrix arises from an identity matrix by a unimodular transformation. Hint: Recall the proof of Lemma 5.10. 7. Prove that there is a polynomial-time algorithm which, given an integral matrix A and an integral vector b, finds an integral vector x with Ax = b or decides that none exists. Hint: See the proofs of Lemma 5.10 and Lemma 5.11. 8. Consider the two systems ! ! ! ! 0 1 1 x1 1 1 0 x1 . ≤ 0 and ≤ 1 0 x2 x2 0 1 −1 0 1 −1 They clearly define the same polyhedron. Prove that the first one is TDI but the second one is not. 9. Let a 6= 0 be an integral vector and β a rational number. Prove that the inequality ax ≤ β is TDI if and only if the components of a are relatively prime. 10. Let Ax ≤ b be TDI, k ∈ N and α > 0 rational. Show that k1 Ax ≤ αb is again TDI. Moreover, prove that α Ax ≤ αb is not necessarily TDI. 11. Use Theorem 5.25 in order to prove K¨onig’s Theorem 10.2 (cf. Exercise 2 of Chapter 11): The maximum cardinality of a matching in a bipartite graph equals the minimum cardinality of a vertex cover. 1 11

12. Show that A = −1 1 0 is not totally unimodular, but {x : Ax = b} is integral 1 00 for all integral vectors b. (Nemhauser and Wolsey [1988])

Exercises

123

13. Let G be the digraph ({1, 2, 3, 4}, {(1, 3), (2, 4), (2, 1), (4, 1), (4, 3)}), and let F := {{1, 2, 4}, {1, 2}, {2}, {2, 3, 4}, {4}}. Prove that (V (G), F) is cross-free but the one-way cut-incidence matrix of F is not totally unimodular. ∗ 14. Let G and T be digraphs such that V (G) = V (T ) and the undirected graph underlying T is a tree. For v, w ∈ V (G) let P(v, w) be the unique undirected path from v to w in T . Let M = (m f,e ) f ∈E(T ), e∈E(G) be the matrix defined by if (x, y) ∈ E(P(v, w)) and (x, y) ∈ E(P(v, y)) 1 m (x,y),(v,w) := −1 if (x, y) ∈ E(P(v, w)) and (x, y) ∈ E(P(v, x)) . 0 if (x, y) ∈ / E(P(v, w))

15.

16.

17. 18. ∗ 19.

20.

Matrices arising this way are called network matrices. Show that the network matrices are precisely the two-way cut-incidence matrices of cross-free set systems. An interval matrix is a 0-1-matrix such that in each row the 1-entries are consecutive. Prove that interval matrices are totally unimodular. Note: Hochbaum and Levin [2006] showed how to solve optimization problems with such matrices very efficiently. Consider the following interval packing problem: Given a list of intervals [ai , bi ], i = 1, . . . , n, with weights c1 , . . . , cn and a number k ∈ N, find a maximum weight subset of the intervals such that no point is contained in more than k of them. (a) Find an LP formulation (without integrality constraints) of this problem. (b) Consider the case k = 1. What combinatorial meaning has the dual LP? Show how to solve the dual LP by a simple combinatorial algorithm. (c) Use (b) to obtain an algorithm for the interval packing problem in the case k = 1 which runs in O(n log n) time. (d) Find a simple O(n log n)-algorithm for general k and unit weights. Note: See also Exercise 11 of Chapter 9. √ Let P := {(x, y) ∈ R2 : y = 2x, x ≥ 0} and Q := {(x, y) ∈ R2 : y = √ 2x}. Prove that P (t) = P 6= PI for all t ∈ N and Q 0 = R2 . Let P be the convex hull of the three points (0, 0), (0, 1) and (k, 12 ) in R2 , where k ∈ N. Show that P (2k−1) 6= PI but P (2k) = PI . Let P ⊆ [0, 1]n be a polytope in the unit hypercube with PI = ∅. Prove that then P (n) = ∅. 2 Note: Eisenbrand and Schulz [2003] proved that P (dn (1+log n)e) = PI for any polytope P ⊆ [0, 1]n . In this exercise we apply Lagrangean relaxation to linear equation systems. Let Q be a finite set of vectors in Rn , c ∈ Rn and A0 ∈ Rm×n and b0 ∈ Rm . Prove that min max{c> x + λ> (b0 − A0 x) : x ∈ Q} : λ ∈ Rm = max{c> y : y ∈ conv(Q), A0 y = b0 }.

124

5 Integer Programming

21. Consider the following facility location problem: Given a set of n customers with demands d1 , . . . , dn , and m optional facilities each of which can be opened or not. For each facility i = 1, . . . , m we have a cost f i for opening it, a capacity u i and a distance ci j to each customer j = 1, . . . , n. The task is to decide which facilities should be opened and to assign each customer to an open facility. The total demand of the customers assigned to one facility must not exceed its capacity. The objective is to minimize the facility opening costs plus the sum of the distances of each customer to its facility. In terms of I NTEGER P ROGRAMMING the problem can be formulated as X X X X min ci j xi j + f i yi : d j xi j ≤ u i yi , xi j = 1, xi j , yi ∈ {0, 1} . i, j

i

j

i

P Apply Lagrangean relaxation in two ways, once relaxing j d j xi j ≤ u i yi for P all i, then relaxing i xi j = 1 for all j. Which Lagrangean dual yields a tighter bound? Note: Both Lagrangean relaxations can be dealt with: see Exercise 7 of Chapter 17. ∗ 22. Consider the U NCAPACITATED FACILITY L OCATION P ROBLEM: given numbers n, m, f i and ci j (i = 1, . . . , m, j = 1, . . . , n), the problem can be formulated as X X X min ci j xi j + f i yi : xi j = 1, xi j ≤ yi , xi j , yi ∈ {0, 1} . i, j

i

i

For S ⊆ {1, . . . , n} we denote by c(S) the cost of supplying facilities for the customers in S, i.e. X X X ci j xi j + f i yi : xi j = 1 for j ∈ S, xi j ≤ yi , xi j , yi ∈ {0, 1} . min i, j

i

i

The cost allocation problem asks whether the total cost c({1, . . . , n}) can be distributed among the customers such that no subset SPpays more than c(S). In otherPwords: are there numbers p1 , . . . , pn such that nj=1 p j = c({1, . . . , n}) and j∈S p j ≤ c(S) for all S ⊆ {1, . . . , n}? Show that this is the case if and only if c({1, . . . , n}) equals X X X ci j xi j + f i yi : xi j = 1, xi j ≤ yi , xi j , yi ≥ 0 , min i, j

i

i

i.e. if the integrality conditions can be left out. Hint: Apply Lagrangean relaxation to the above LP. For each set of Lagrange multipliers decompose the resulting minimization problem to minimization problems over polyhedral cones. What are the vectors generating these cones? (Goemans and Skutella [2004])

References

125

23. Describe a combinatorial algorithm (without using L INEAR P ROGRAMMING) to solve (5.10) for arbitrary (but fixed) Lagrange multipliers λ. What running time can you achieve?

References General Literature: Bertsimas, D., and Weismantel, R. [2005]: Optimization Over Integers. Dynamic Ideas, Belmont 2005 Cook, W.J., Cunningham, W.H., Pulleyblank, W.R., and Schrijver, A. [1998]: Combinatorial Optimization. Wiley, New York 1998, Chapter 6 Nemhauser, G.L., and Wolsey, L.A. [1988]: Integer and Combinatorial Optimization. Wiley, New York 1988 Schrijver, A. [1986]: Theory of Linear and Integer Programming. Wiley, Chichester 1986 Wolsey, L.A. [1998]: Integer Programming. Wiley, New York 1998

Cited References: Boyd, E.A. [1997]: A fully polynomial epsilon approximation cutting plane algorithm for solving combinatorial linear programs containing a sufficiently large ball. Operations Research Letters 20 (1997), 59–63 Bruns, W., Gubeladze, J., Henk, M., Martin, A., and Weismantel, R. [1999]: A counterexample to an integral analogue of Carath´eodory’s theorem. Journal f¨ur die Reine und Angewandte Mathematik 510 (1999), 179–185 Chv´atal, V. [1973]: Edmonds’ polytopes and a hierarchy of combinatorial problems. Discrete Mathematics 4 (1973), 305–337 Cook, W. [1983]: Operations that preserve total dual integrality. Operations Research Letters 2 (1983), 31–35 Cook, W., Fonlupt, J., and Schrijver, A. [1986]: An integer analogue of Carath´eodory’s theorem. Journal of Combinatorial Theory B 40 (1986), 63–70 ´ [1986]: Sensitivity theorems in integer Cook, W., Gerards, A., Schrijver, A., and Tardos, E. linear programming. Mathematical Programming 34 (1986), 251–264 Cook, W., Kannan, R., and Schrijver, A. [1990]: Chv´atal closures for mixed integer programming problems. Mathematical Programming 47 (1990), 155–174 Dantzig, G., Fulkerson, R., and Johnson, S. [1954]: Solution of a large-scale travelingsalesman problem. Operations Research 2 (1954), 393–410 Edmonds, J., and Giles, R. [1977]: A min-max relation for submodular functions on graphs. In: Studies in Integer Programming; Annals of Discrete Mathematics 1 (P.L. Hammer, E.L. Johnson, B.H. Korte, G.L. Nemhauser, eds.), North-Holland, Amsterdam 1977, pp. 185–204 Eisenbrand, F. [1999]: On the membership problem for the elementary closure of a polyhedron. Combinatorica 19 (1999), 297–300 Eisenbrand, F., and Schulz, A.S. [2003]: Bounds on the Chv´atal rank of polytopes in the 0/1-cube. Combinatorica 23 (2003), 245–261 Fulkerson, D.R. [1971]: Blocking and anti-blocking pairs of polyhedra. Mathematical Programming 1 (1971), 168–194

126

5 Integer Programming

Geoffrion, A.M. [1974]: Lagrangean relaxation for integer programming. Mathematical Programming Study 2 (1974), 82–114 Giles, F.R., and Pulleyblank, W.R. [1979]: Total dual integrality and integer polyhedra. Linear Algebra and Its Applications 25 (1979), 191–196 Ghouila-Houri, A. [1962]: Caract´erisation des matrices totalement unimodulaires. Comptes Rendus Hebdomadaires des S´eances de l’Acad´emie des Sciences (Paris) 254 (1962), 1192–1194 Goemans, M.X., and Skutella, M. [2004]: Cooperative facility location games. Journal of Algorithms 50 (2004), 194–214 Goffin, J.L. [1977]: On convergence rates of subgradient optimization methods. Mathematical Programming 13 (1977), 329–347 Gomory, R.E. [1958]: Outline of an algorithm for integer solutions to linear programs. Bulletin of the American Mathematical Society 64 (1958), 275–278 Gomory, R.E. [1963]: An algorithm for integer solutions of linear programs. In: Recent Advances in Mathematical Programming (R.L. Graves, P. Wolfe, eds.), McGraw-Hill, New York, 1963, pp. 269–302 Graver, J.E. [1975]: On the foundations of linear and integer programming I. Mathematical Programming 9 (1975), 207–226 Harvey, W. [1999]: Computing two-dimensional integer hulls. SIAM Journal on Computing 28 (1999), 2285–2299 Hochbaum, D.S., and Levin, A. [2006]: Optimizing over consecutive 1’s and circular 1’s constraints. SIAM Journal on Optimization 17 (2006), 311–330 Hoffman, A.J. [1974]: A generalization of max flow-min cut. Mathematical Programming 6 (1974), 352–359 Hoffman, A.J., and Kruskal, J.B. [1956]: Integral boundary points of convex polyhedra. In: Linear Inequalities and Related Systems; Annals of Mathematical Study 38 (H.W. Kuhn, A.W. Tucker, eds.), Princeton University Press, Princeton 1956, 223–246 Lasserre, J.B. [2004]: The integer hull of a convex rational polytope. Discrete & Computational Geometry 32 (2004), 129–139 Meyer, R.R. [1974]: On the existence of optimal solutions to integer and mixed-integer programming problems. Mathematical Programming 7 (1974), 223–235 Polyak, B.T. [1967]: A general method for solving extremal problems. Doklady Akademii Nauk SSSR 174 (1967), 33–36 [in Russian]. English translation: Soviet Mathematics Doklady 8 (1967), 593–597 Schrijver, A. [1980]: On cutting planes. In: Combinatorics 79; Part II; Annals of Discrete Mathematics 9 (M. Deza, I.G. Rosenberg, eds.), North-Holland, Amsterdam 1980, pp. 291–296 Schrijver, A. [1981]: On total dual integrality. Linear Algebra and its Applications 38 (1981), 27–32 Schrijver, A. [1983]: Packing and covering of crossing families of cuts. Journal of Combinatorial Theory B 35 (1983), 104–128 Seb˝o, A. [1990]: Hilbert bases, Carath´eodory’s theorem and combinatorial optimization. In: Integer Programming and Combinatorial Optimization (R. Kannan and W.R. Pulleyblank, eds.), University of Waterloo Press, 1990 Seymour, P.D. [1980]: Decomposition of regular matroids. Journal of Combinatorial Theory B 28 (1980), 305–359 Veinott, A.F., Jr., and Dantzig, G.B. [1968]. Integral extreme points. SIAM Review 10 (1968), 371–372 Wolsey, L.A. [1981]: The b-hull of an integer program. Discrete Applied Mathematics 3 (1981), 193–201

6 Spanning Trees and Arborescences

Consider a telephone company that wants to rent a subset from an existing set of cables, each of which connects two cities. The rented cables should suffice to connect all cities and they should be as cheap as possible. It is natural to model the network by a graph: the vertices are the cities and the edges correspond to the cables. By Theorem 2.4 the minimal connected spanning subgraphs of a given graph are its spanning trees. So we look for a spanning tree of minimum weight, where we say that P a subgraph T of a graph G with weights c : E(G) → R has weight c(E(T )) = e∈E(T ) c(e). This is a simple but very important combinatorial optimization problem. It is also among the combinatorial optimization problems with the longest history; the first algorithm was given by Bor˚uvka [1926a,1926b]; see Neˇsetˇril, Milkov´a and Neˇsetˇrilov´a [2001]. Compared to the D RILLING P ROBLEM which asks for a shortest path containing all vertices of a complete graph, we now look for a shortest spanning tree. Although the number of spanning trees is even bigger than the number of paths (K n contains n! n−2 different 2 Hamiltonian paths, but, by a theorem of Cayley [1889], as many as n spanning trees; see Exercise 1), the problem turns out to be much easier. In fact, a simple greedy strategy works as we shall see in Section 6.1. Arborescences can be considered as the directed counterparts of trees; by Theorem 2.5 they are the minimal spanning subgraphs of a digraph such that all vertices are reachable from a root. The directed version of the M INIMUM S PANNING T REE P ROBLEM, the M INIMUM W EIGHT A RBORESCENCE P ROBLEM, is more difficult since a greedy strategy no longer works. In Section 6.2 we show how to solve this problem. Since there are very efficient combinatorial algorithms it is not recommended to solve these problems with L INEAR P ROGRAMMING. Nevertheless it is interesting that the corresponding polytopes (the convex hull of the incidence vectors of spanning trees or arborescences; cf. Corollary 3.33) can be described in a nice way, which we shall show in Section 6.3. In Section 6.4 we prove some classical results concerning the packing of spanning trees and arborescences.

6.1 Minimum Spanning Trees In this section, we consider the following two problems:

128

6 Spanning Trees and Arborescences

M AXIMUM W EIGHT F OREST P ROBLEM Instance:

An undirected graph G, weights c : E(G) → R.

Task:

Find a forest in G of maximum weight.

M INIMUM S PANNING T REE P ROBLEM Instance:

An undirected graph G, weights c : E(G) → R.

Task:

Find a spanning tree in G of minimum weight or decide that G is not connected.

We claim that both problems are equivalent. To make this precise, we say that a problem P linearly reduces to a problem Q if there are functions f and g, each computable in linear time, such that f transforms an instance x of P to an instance f (x) of Q, and g transforms a solution of f (x) to a solution of x. If P linearly reduces to Q and Q linearly reduces to P, then both problems are called equivalent. Proposition 6.1. The M AXIMUM W EIGHT F OREST P ROBLEM and the M INI MUM S PANNING T REE P ROBLEM are equivalent. Proof: Given an instance (G, c) of the M AXIMUM W EIGHT F OREST P ROBLEM, delete all edges of negative weight, let c0 (e) := −c(e) for all e ∈ E(G 0 ), and add a minimum set F of edges (with arbitrary weight) to make the graph connected; let us call the resulting graph G 0 . Then instance (G 0 , c0 ) of the M INIMUM S PANNING T REE P ROBLEM is equivalent in the following sense: Deleting the edges of F from a minimum weight spanning tree in (G 0 , c0 ) yields a maximum weight forest in (G, c). Conversely, given an instance (G, c) of the M INIMUM S PANNING T REE P ROB LEM , let c0 (e) := K − c(e) for all e ∈ E(G), where K = 1 + maxe∈E(G) c(e). Then the instance (G, c0 ) of the M AXIMUM W EIGHT F OREST P ROBLEM is equivalent, since all spanning trees have the same number of edges (Theorem 2.4). We shall return to different reductions of one problem to another in Chapter 15. In the rest of this section we consider the M INIMUM S PANNING T REE P ROBLEM only. We start by proving two optimality conditions: Theorem 6.2. Let (G, c) be an instance of the M INIMUM S PANNING T REE P ROBLEM, and let T be a spanning tree in G. Then the following statements are equivalent: (a) T is optimum. (b) For every e = {x, y} ∈ E(G) \ E(T ), no edge on the x-y-path in T has higher cost than e. (c) For every e ∈ E(T ), e is a minimum cost edge of δ(V (C)), where C is a connected component of T − e.

6.1 Minimum Spanning Trees

129

Proof: (a)⇒(b): Suppose (b) is violated: Let e = {x, y} ∈ E(G) \ E(T ) and let f be an edge on the x-y-path in T with c( f ) > c(e). Then (T − f ) + e is a spanning tree with lower cost. (b)⇒(c): Suppose (c) is violated: let e ∈ E(T ), C a connected component of T − e and f = {x, y} ∈ δ(V (C)) with c( f ) < c(e). Observe that the x-y-path in T must contain an edge of δ(V (C)), but the only such edge is e. So (b) is violated. (c)⇒(a): Suppose T satisfies (c), and let T ∗ be an optimum spanning tree with E(T ∗ ) ∩ E(T ) as large as possible. We show that T = T ∗ . Namely, suppose there is an edge e = {x, y} ∈ E(T ) \ E(T ∗ ). Let C be a connected component of T − e. T ∗ + e contains a circuit D. Since e ∈ E(D) ∩ δ(V (C)), at least one more edge f ( f 6= e) of D must belong to δ(V (C)) (see Exercise 9 of Chapter 2). Observe that (T ∗ + e) − f is a spanning tree. Since T ∗ is optimum, c(e) ≥ c( f ). But since (c) holds for T , we also have c( f ) ≥ c(e). So c( f ) = c(e), and (T ∗ + e) − f is another optimum spanning tree. This is a contradiction, because it has one edge more in common with T . The following “greedy” algorithm for the M INIMUM S PANNING T REE P ROB was proposed by Kruskal [1956]. It can be regarded as a special case of a quite general greedy algorithm which will be discussed in Section 13.4. In the following let n := |V (G)| and m := |E(G)|. LEM

K RUSKAL ’ S A LGORITHM Input:

A connected undirected graph G, weights c : E(G) → R.

Output:

A spanning tree T of minimum weight.

1

Sort the edges such that c(e1 ) ≤ c(e2 ) ≤ . . . ≤ c(em ).

2

Set T := (V (G), ∅).

3

For i := 1 to m do: If T + ei contains no circuit then set T := T + ei .

Theorem 6.3. K RUSKAL ’ S A LGORITHM works correctly. Proof: It is clear that the algorithm constructs a spanning tree T . It also guarantees condition (b) of Theorem 6.2, so T is optimum. The running time of K RUSKAL ’ S A LGORITHM is O(mn): the edges can be sorted in O(m log m) time (Theorem 1.5), and testing for a circuit in a graph with at most n edges can be implemented in O(n) time (just apply DFS (or BFS) and check if there is any edge not belonging to the DFS-tree). Since this is repeated m times, we get a total running time of O(m log m + mn) = O(mn). However, a more efficient implementation is possible: Theorem 6.4. K RUSKAL ’ S A LGORITHM can be implemented to run in O(m log n) time.

130

6 Spanning Trees and Arborescences

Proof: Parallel edges can be eliminated first: all but the cheapest edges are redundant. So we may assume that m = O(n 2 ). Since the running time of 1 is obviously O(m log m) = O(m log n) we concentrate on . 3 We study a data structure maintaining the connected components of T . In 3 we have to test whether the addition of an edge ei = {v, w} to T results in a circuit. This is equivalent to testing if v and w are in the same connected component. Our implementation maintains a branching B with V (B) = V (G). At any time the connected components of B will be induced by the same vertex sets as the connected components of T . (Note however that B is in general not an orientation of T .) When checking an edge ei = {v, w} in , 3 we find the root rv of the arborescence in B containing v and the root rw of the arborescence in B containing w. The time needed for this is proportional to the length of the rv -v-path plus the length of the rw -w-path in B. We shall show later that this length is always at most log n. Next we check if rv = rw . If rv 6= rw , we insert ei into T and we have to add an edge to B. Let h(r ) be the maximum length of a path from r in B. If h(rv ) ≥ h(rw ), then we add an edge (rv , rw ) to B, otherwise we add (rw , rv ) to B. If h(rv ) = h(rw ), this operation increases h(rv ) by one, otherwise the new root has the same h-value as before. So the h-values of the roots can be maintained easily. Of course initially B := (V (G), ∅) and h(v) := 0 for all v ∈ V (G). We claim that an arborescence of B with root r contains at least 2h(r ) vertices. This implies that h(r ) ≤ log n, concluding the proof. At the beginning, the claim is clearly true. We have to show that this property is maintained when adding an edge (x, y) to B. This is trivial if h(x) does not change. Otherwise we have h(x) = h(y) before the operation, implying that each of the two arborescences contains at least 2h(x) vertices. So the new arborescence rooted at x contains at least 2 · 2h(x) = 2h(x)+1 vertices, as required. The above implementation can be improved by another trick: whenever the root rv of the arborescence in B containing v has been determined, all the edges on the rv -v-path P are deleted and an edge (r x , x) is inserted for each x ∈ V (P) \ {rv }. A complicated analysis shows that this so-called path compression heuristic makes the running time of 3 almost linear: it is O(mα(m, n)), where α(m, n) is the functional inverse of Ackermann’s function (see Tarjan [1975,1983]). We now mention another well-known algorithm for the M INIMUM S PANNING T REE P ROBLEM, due to Jarn´ık [1930] (see Korte and Neˇsetˇril [2001]), Dijkstra [1959] and Prim [1957]:

P RIM ’ S A LGORITHM Input:

A connected undirected graph G, weights c : E(G) → R.

Output:

A spanning tree T of minimum weight.

1

Choose v ∈ V (G). Set T := ({v}, ∅).

6.1 Minimum Spanning Trees

2

131

While V (T ) 6= V (G) do: Choose an edge e ∈ δG (V (T )) of minimum weight. Set T := T + e.

Theorem 6.5. P RIM ’ S A LGORITHM works correctly. Its running time is O(n 2 ). Proof: The correctness follows from the fact that condition (c) of Theorem 6.2 is guaranteed. To obtain the O(n 2 ) running time, we maintain for each vertex v ∈ V (G)\V (T ) the cheapest edge e ∈ E(V (T ), {v}). Let us call these edges the candidates. The initialization of the candidates takes O(m) time. Each selection of the cheapest edge among the candidates takes O(n) time. The update of the candidates can be done by scanning the edges incident to the vertex which is added to V (T ) and thus also takes O(n) time. Since the while-loop of 2 has n − 1 iterations, the O(n 2 ) bound is proved. The running time can be improved by efficient data structures. Denote l T,v := min{c(e) : e ∈ E(V (T ), {v})}. We maintain the set {(v, l T,v ) : v ∈ V (G) \ V (T ), l T,v < ∞} in a data structure, called priority queue or heap, that allows inserting an element, finding and deleting an element (v, l) with minimum l, and decreasing the so-called key l of an element (v, l). Then P RIM ’ S A LGORITHM can be written as follows:

1

2

Choose v ∈ V (G). Set T := ({v}, ∅). Let lw := ∞ for w ∈ V (G) \ {v}. While V (T ) 6= V (G) do: For e = {v, w} ∈ E({v}, V (G) \ V (T )) do: If c(e) < lw < ∞ then set lw := c(e) and DECREASEKEY(w, lw ). If lw = ∞ then set lw := c(e) and INSERT(w, lw ). (v, lv ) := DELETEMIN. Let e ∈ E(V (T ), {v}) with c(e) = lv . Set T := T + e.

There are several possible ways to implement a heap. A very efficient way, the so-called Fibonacci heap, has been proposed by Fredman and Tarjan [1987]. Our presentation is based on Schrijver [2003]: Theorem 6.6. It is possible to maintain a data structure for a finite set (initially empty), where each element u is associated with a real number d(u), called its key, and perform any sequence of p INSERT-operations (adding an element u with key d(u)); n DELETEMIN-operations (finding and deleting an element u with d(u) minimum); • m DECREASEKEY-operations (decreasing d(u) to a specified value for an element u)

• •

in O(m + p + n log p) time.

132

6 Spanning Trees and Arborescences

Proof: The set, denoted by U , is stored in a Fibonacci heap, i.e. a branching (U, E) with a function ϕ : U → {0, 1} with the following properties: (i) If (u, v) ∈ E then d(u) ≤ d(v). (This is called the heap order.) (ii) For each u ∈ U the children of u can be numbered 1, . . . , |δ + (u)| such that the i-th child v satisfies |δ + (v)| + ϕ(v) ≥ i − 1. (iii) If u and v are distinct roots (δ − (u) = δ − (v) = ∅), then |δ + (u)| 6= |δ + (v)|. Condition (ii) implies: (iv) If a vertex u has out-degree at least k, then at least from u.

√ k 2 vertices are reachable

We prove (iv) by induction on k, the case k = 0 being trivial. So let u be a vertex with |δ + (u)| ≥ k ≥ 1, and let v be a child of u with |δ + (v)| ≥ k −2 (v exists due to (ii)). We apply the induction hypothesis to v in (U, E) and to u in (U, E \ {(u, v)}) √ k−2 √ k−1 and 2 vertices are reachable. (iv) follows and conclude that at least 2 √ k √ k−2 √ k−1 from observing that 2 ≤ 2 + 2 . In particular, (iv) implies that |δ + (u)| ≤ 2 log |U | for all u ∈ U . Thus, using (iii), we can store the roots of (U, E) by a function b : {0, 1, . . . , b2 log |U |c} → U with b(|δ + (u)|) = u for each root u. In addition to this, we keep track of a doubly-linked list of children (in arbitrary order), a pointer to the parent (if existent) and the out-degree of each vertex. We now show how the INSERT-, DELETEMIN- and DECREASEKEY-operations are implemented. INSERT (v, d(v)) is implemented by setting ϕ(v) := 0 and applying PLANT (v):

1

Set r := b(|δ + (v)|). if r is a root with r 6= v and |δ + (r )| = |δ + (v)| then: if d(r ) ≤ d(v) then add (r, v) to E and PLANT(r ). if d(v) < d(r ) then add (v, r ) to E and PLANT(v). else set b(|δ + (v)|) := v.

As (U, E) is always a branching, the recursion terminates. Note also that (i), (ii) and (iii) are maintained. DELETEMIN is implemented by scanning b(i) for i = 0, . . . , b2 log |U |c in order to find an element u with d(u) minimum, deleting u and its incident edges and successively applying PLANT(v) for each (former) child v of u. DECREASEKEY (v, (d(v)) is a bit more complicated. Let P be the longest path in (U, E) ending in v such that each internal vertex u satisfies ϕ(u) = 1. We set ϕ(u) := 1 − ϕ(u) for all u ∈ V (P) \ {v}, delete all edges of P from E and apply PLANT (z) for each deleted edge (y, z). To see that this maintains (ii) we only have to consider the parent of the start vertex x of P, if existent. But then x is not a root, and thus ϕ(x) changes from 0 to 1, making up for the lost child.

6.2 Minimum Weight Arborescences

133

We finally estimate the running time. As ϕ increases at most m times (at most once in each DECREASEKEY), ϕ decreases at most m times. Thus the sum of the length of the paths P in all DECREASEKEY-operations is at most m + m. So at most 2m + 2n log p edges are deleted overall (as each DELETEMIN-operation may delete up to 2 log p edges). Thus at most 2m + 2n log p + p − 1 edges are inserted in total. This proves the overall O(m + p + n log p) running time. Corollary 6.7. P RIM ’ S A LGORITHM implemented with Fibonacci heap solves the M INIMUM S PANNING T REE P ROBLEM in O(m + n log n) time. Proof: We have at most n−1 INSERT-, n−1 DELETEMIN-, and m DECREASEKEYoperations. With a more sophisticated implementation, the running time can be improved to O (m log β(n, m)), where β(n, m) = min i : log(i) n ≤ mn ; see Fredman and Tarjan [1987], Gabow, Galil and Spencer [1989], and Gabow et al. [1986]. The fastest known deterministic algorithm is due to Chazelle [2000] and has a running time of O(mα(m, n)), where α is the functional inverse of Ackermann’s function. On a different computational model Fredman and Willard [1994] achieved linear running time. Moreover, there is a randomized algorithm which finds a minimum weight spanning tree and has linear expected running time (Karger, Klein and Tarjan [1995]; such an algorithm which always finds an optimum solution is called a Las Vegas algorithm). This algorithm uses a (deterministic) procedure for testing whether a given spanning tree is optimum; a linear-time algorithm for this problem has been found by Dixon, Rauch and Tarjan [1992]; see also King [1995]. The M INIMUM S PANNING T REE P ROBLEM for planar graphs can be solved (deterministically) in linear time (Cheriton and Tarjan [1976]). The problem of finding a minimum spanning tree for a set of n points in the plane can be solved in O(n log n) time (Exercise 9). P RIM ’ S A LGORITHM can be quite efficient for such instances since one can use suitable data structures for finding nearest neighbours in the plane effectively.

6.2 Minimum Weight Arborescences Natural directed generalizations of the M AXIMUM W EIGHT F OREST P ROBLEM and the M INIMUM S PANNING T REE P ROBLEM read as follows:

M AXIMUM W EIGHT B RANCHING P ROBLEM Instance:

A digraph G, weights c : E(G) → R.

Task:

Find a maximum weight branching in G.

134

6 Spanning Trees and Arborescences

M INIMUM W EIGHT A RBORESCENCE P ROBLEM Instance:

A digraph G, weights c : E(G) → R.

Task:

Find a minimum weight spanning arborescence in G or decide that none exists.

Sometimes we want to specify the root in advance:

M INIMUM W EIGHT ROOTED A RBORESCENCE P ROBLEM Instance:

A digraph G, a vertex r ∈ V (G), weights c : E(G) → R.

Task:

Find a minimum weight spanning arborescence rooted at r in G or decide that none exists.

As for the undirected case, these three problems are equivalent: Proposition 6.8. The M AXIMUM W EIGHT B RANCHING P ROBLEM, the M INI MUM W EIGHT A RBORESCENCE P ROBLEM and the M INIMUM W EIGHT ROOTED A RBORESCENCE P ROBLEM are all equivalent. RBORESCENCE Proof: Given an instance (G, c) of the M INIMUM W EIGHT AP P ROBLEM, let c0 (e) := K − c(e) for all e ∈ E(G), where K = 2 e∈E(G) |c(e)|. Then the instance (G, c0 ) of the M AXIMUM W EIGHT B RANCHING P ROBLEM is equivalent, because for any two branchings B, B 0 with |E(B)| > |E(B 0 )| we have c0 (B) > c0 (B 0 ) (and branchings with n − 1 edges are exactly the spanning arborescences). Given an instance . (G, c) of the M AXIMUM W EIGHT B RANCHING P ROBLEM , let G 0 := (V (G) ∪ {r }, E(G) ∪ {(r, v) : v ∈ V (G)}). Let c0 (e) := −c(e) for e ∈ E(G) and c(e) := 0 for e ∈ E(G 0 ) \ E(G). Then the instance (G 0 , r, c0 ) of the M INIMUM W EIGHT ROOTED A RBORESCENCE P ROBLEM is equivalent. Finally, given an instance (G, r, c) of the M INIMUM W EIGHT ROOTED A R . BORESCENCE P ROBLEM , let G 0 := (V (G) ∪ {s}, E(G)∪{(s, r )}) and c((s, r )) := 0. Then the instance (G 0 , c) of the M INIMUM W EIGHT A RBORESCENCE P ROB LEM is equivalent.

In the rest of this section we shall deal with the M AXIMUM W EIGHT B RANCH P ROBLEM only. This problem is not as easy as its undirected version, the M AXIMUM W EIGHT F OREST P ROBLEM. For example any maximal forest is maximum, but the bold edges in Figure 6.1 form a maximal branching which is not maximum. Recall that a branching is a graph B with |δ − B (x)| ≤ 1 for all x ∈ V (B), such that the underlying undirected graph is a forest. Equivalently, a branching is an acyclic digraph B with |δ − B (x)| ≤ 1 for all x ∈ V (B); see Theorem 2.5(g): ING

Proposition 6.9. Let B be a digraph with |δ − B (x)| ≤ 1 for all x ∈ V (B). Then B contains a circuit if and only if the underlying undirected graph contains a circuit.

6.2 Minimum Weight Arborescences

135

Fig. 6.1.

Now let G be a digraph and c : E(G) → R+ . We can ignore negative weights since such edges will never appear in an optimum branching. A ﬁrst idea towards an algorithm could be to take the best entering edge for each vertex. Of course the resulting graph may contain circuits. Since a branching cannot contain circuits, we must delete at least one edge of each circuit. The following lemma says that one is enough. Lemma 6.10. (Karp [1972]) Let B0 be a maximum weight subgraph of G with |δ − B0 (v)| ≤ 1 for all v ∈ V (B0 ). Then there exists an optimum branching B of G such that for each circuit C in B0 , |E(C) \ E(B)| = 1.

a1

b1

C a2

b3

a3

b2 Fig. 6.2.

Proof: Let B be an optimum branching of G containing as many edges of B0 as possible. Let C be some circuit in B0 . Let E(C) \ E(B) = {(a1 , b1 ), . . . , (ak , bk )}; suppose that k ≥ 2 and a1 , b1 , a2 , b2 , a3 , . . . , bk lie in this order on C (see Figure 6.2). We claim that B contains a bi -bi−1 -path for each i = 1, . . . , k (b0 := bk ). This, however, is a contradiction because these paths form a closed edge progression in B, and a branching cannot have a closed edge progression. Let i ∈ {1, . . . , k}. It remains to show that B contains a bi -bi−1 -path. Consider B with V (B ) = V (G) and E(B ) := {(x, y) ∈ E(B) : y = bi } ∪ {(ai , bi )}. B cannot be a branching since it would be optimum and contain more edges of B0 than B. So (by Proposition 6.9) B contains a circuit, i.e. B contains a

136

6 Spanning Trees and Arborescences

bi -ai -path P. Since k ≥ 2, P is not completely on C, so let e be the last edge of P not belonging to C. Obviously e = (x, bi−1 ) for some x, so P (and thus B) contains a bi -bi−1 -path. The main idea of Edmonds’ [1967] algorithm is to find first B0 as above, and then contract every circuit of B0 in G. If we choose the weights of the resulting graph G 1 correctly, any optimum branching in G 1 will correspond to an optimum branching in G.

E DMONDS ’ B RANCHING A LGORITHM Input:

A digraph G, weights c : E(G) → R+ .

Output:

A maximum weight branching B of G.

1

Set i := 0, G 0 := G, and c0 := c.

2

Let Bi be a maximum weight subgraph of G i with |δ − Bi (v)| ≤ 1 for all v ∈ V (Bi ). If Bi contains no circuit then set B := Bi and go to . 5

3

4

5

6

Construct (G i+1 , ci+1 ) from (G i , ci ) by doing the following for each circuit C of Bi : Contract C to a single vertex vC in G i+1 For each edge e = (z, y) ∈ E(G i ) with z ∈ / V (C), y ∈ V (C) do: Let z 0 = vC 0 if z belongs to a circuit C 0 of Bi and z 0 = z otherwise. Let e0 := (z 0 , vC ) and 8(e0 ) := e. Set ci+1 (e0 ) := ci (e) − ci (α(e, C)) + ci (eC ), where α(e, C) = (x, y) ∈ E(C) and eC is some cheapest edge of C. Set i := i + 1 and go to . 2 If i = 0 then stop. For each circuit C of Bi−1 do: If there is an edge e0 = (z, vC ) ∈ E(B) then set E(B) := (E(B) \ {e0 }) ∪ 8(e0 ) ∪ (E(C) \ {α(8(e0 ), C)}) else set E(B) := E(B) ∪ (E(C) \ {eC }). Set V (B) := V (G i−1 ), i := i − 1 and go to . 5

This algorithm was also discovered independently by Chu and Liu [1965] and Bock [1971]. Theorem 6.11. (Edmonds [1967]) E DMONDS ’ B RANCHING A LGORITHM works correctly. Proof: We show that each time just before the execution of , 5 B is an optimum branching of G i . This is trivial for the first time we reach . 5 So we have to show that 6 transforms an optimum branching B of G i into an optimum branching B 0 of G i−1 .

6.3 Polyhedral Descriptions

137

∗ ∗ )| = 1 for each Let Bi−1 be any branching of G i−1 such that |E(C) \ E(Bi−1 ∗ ∗ circuit C of Bi−1 . Let Bi result from Bi−1 by contracting the circuits of Bi−1 . Bi∗ is a branching of G i . Furthermore we have X ∗ ci−1 (Bi−1 ) = ci (Bi∗ ) + (ci−1 (C) − ci−1 (eC )). C: circuit of Bi−1

By the induction hypothesis, B is an optimum branching of G i , so we have ci (B) ≥ ci (Bi∗ ). We conclude that X ∗ ci−1 (Bi−1 ) ≤ ci (B) + (ci−1 (C) − ci−1 (eC )) C: circuit of Bi−1

= ci−1 (B ). 0

This, together with Lemma 6.10, implies that B 0 is an optimum branching of G i−1 . This proof is due to Karp [1972]. Edmonds’ original proof was based on a linear programming formulation (see Corollary 6.14). The running time of E DMONDS ’ B RANCHING A LGORITHM is easily seen to be O(mn), where m = |E(G)| and n = |V (G)|: there are at most n iterations (i.e. i ≤ n at any stage of the algorithm), and each iteration can be implemented in O(m) time. The best known bound has been obtained by Gabow et al. [1986] using a Fibonacci heap: their branching algorithm runs in O(m + n log n) time.

6.3 Polyhedral Descriptions A polyhedral description of the M INIMUM S PANNING T REE P ROBLEM is as follows: Theorem 6.12. (Edmonds [1970]) Given a connected undirected graph G, n := |V (G)|, the polytope P := X X x ∈ [0, 1] E(G) : xe = n − 1, xe ≤ |X | − 1 for ∅ 6= X ⊂ V (G) e∈E(G)

e∈E(G[X ])

is integral. Its vertices are exactly the incidence vectors of spanning trees of G. (P is called the spanning tree polytope of G.) Proof: Let T be a spanning tree of G, and let x be the incidence vector of E(T ). Obviously (by Theorem 2.4), x ∈ P. Furthermore, since x ∈ {0, 1} E(G) , it must be a vertex of P. On the other hand let x be an integral vertex of P. Then x is the incidence vector of the edge set of some subgraph H with n − 1 edges and no circuit. Again by Theorem 2.4 this implies that H is a spanning tree.

138

6 Spanning Trees and Arborescences

So it suffices to show that P is integral (recall Theorem 5.13). Let c : E(G) → R, and let T be the tree produced by K RUSKAL ’ S A LGORITHM when applied to (G, c) (ties are broken arbitrarily when sorting the edges). Denote E(T ) = { f 1 , . . . , f n−1 }, where the f i were taken in this order by the algorithm. In particular, c( f 1 ) ≤ · · · ≤ c( f n−1 ). Let X k ⊆ V (G) be the connected component of (V (G), { f 1 , . . . , f k }) containing f k (k = 1, . . . , n − 1). Let x ∗ be the incidence vector of E(T ). We show that x ∗ is an optimum solution to the LP X c(e)xe min e∈E(G)

X

s.t.

e∈E(G) X

xe = n − 1 xe ≤ |X | − 1

(∅ 6= X ⊂ V (G))

xe ≥ 0

(e ∈ E(G)).

e∈E(G[X ])

We introduce a dual variable z X for each ∅ 6= X ⊂ V (G) and one additional dual variable z V (G) for the equality constraint. Then the dual LP is X max − (|X | − 1)z X ∅6= X ⊆V (G)

s.t.

−

X

zX

≤ c(e)

(e ∈ E(G))

zX

≥ 0

(∅ 6= X ⊂ V (G)).

e⊆X ⊆V (G)

Note that the dual variable z V (G) is not forced to be nonnegative. For k = 1, . . . , n − 2 let z ∗X k := c( fl ) − c( f k ), where l is the first index greater than k for which fl ∩ X k 6= ∅. Let z ∗V (G) := −c( f n−1 ), and let z ∗X := 0 for all X 6∈ {X 1 , . . . , X n−1 }. For each e = {v, w} we have that X z ∗X = c( f i ), − e⊆X ⊆V (G)

where i is the smallest index such that v, w ∈ X i . Moreover c( f i ) ≤ c(e) since v and w are in different connected components of (V (G), { f 1 , . . . , f i−1 }). Hence z ∗ is a feasible dual solution. Moreover xe∗ > 0, i.e. e ∈ E(T ), implies X − z ∗X = c(e), e⊆X ⊆V (G)

i.e. the corresponding dual constraint is satisfied with equality. Finally, z ∗X > 0 implies that T [X ] is connected, so the corresponding primal constraint is satisfied

6.3 Polyhedral Descriptions

139

with equality. In other words, the primal and dual complementary slackness conditions are satisfied, thus (by Corollary 3.23) x ∗ and z ∗ are optimum solutions for the primal and dual LP, respectively. Indeed, we have proved that the inequality system in Theorem 6.12 is TDI. We remark that the above is also an alternative proof of the correctness of K RUSKAL ’ S A LGORITHM (Theorem 6.3). Another description of the spanning tree polytope is the subject of Exercise 14. P P If we replace the constraint e∈E(G) xe = n − 1 by e∈E(G) xe ≤ n − 1, we obtain the convex hull of the incidence vectors of all forests in G (Exercise 15). A generalization of these results is Edmonds’ characterization of the matroid polytope (Theorem 13.21). We now turn to a polyhedral description of the M INIMUM W EIGHT ROOTED A RBORESCENCE P ROBLEM. First we prove a classical result of Fulkerson. Recall that an r -cut is a set of edges δ + (S) for some S ⊂ V (G) with r ∈ S. Theorem 6.13. (Fulkerson [1974]) Let G be a digraph with weights c : E(G) → Z+ , and r ∈ V (G) such that G contains a spanning arborescence rooted at r . Then the minimum weight of a spanning arborescence rooted at r equals the maximum number t of r -cuts C1 , . . . , Ct (repetitions allowed) such that no edge e is contained in more than c(e) of these cuts. Proof: Let A be the matrix whose columns are indexed by the edges and whose rows are all incidence vectors of r -cuts. Consider the LP min{cx : Ax ≥ 1l, x ≥ 0}, and its dual max{1ly : y A ≤ c, y ≥ 0}. Then (by part (e) of Theorem 2.5) we have to show that for any nonnegative integral c, both the primal and dual LP have integral optimum solutions. By Corollary 5.15 it suffices to show that the system Ax ≥ 1l, x ≥ 0 is TDI. We use Lemma 5.23. Since the dual LP is feasible if and only if c is nonnegative, let c : E(G) → Z+ . Let y be an optimum solution of max{1ly : y A ≤ c, y ≥ 0} for which X yδ − (X ) |X |2 (6.1) ∅6= X ⊆V (G)\{r }

is as large as possible. We claim that F := {X : yδ − (X ) > 0} is laminar. To see this, suppose X, Y ∈ F with X ∩ Y 6= ∅, X \ Y 6= ∅ and Y \ X 6= ∅ (Figure 6.3). Let := min{yδ − (X ) , yδ − (Y ) }. Set yδ0 − (X ) := yδ − (X ) − , yδ0 − (Y ) := yδ − (Y ) − , yδ0 − (X ∩Y ) := yδ − (X ∩Y ) + , yδ0 − (X ∪Y ) := yδ − (X ∪Y ) + , and y 0 (S) := y(S) for all other r -cuts S. Observe that y 0 A ≤ y A, so y 0 is a feasible dual solution. Since 1ly = 1ly 0 , it is also optimum and contradicts the choice of y, because (6.1) is larger for y 0 . (For any numbers a > b ≥ c > d > 0 with a + d = b + c we have a 2 + d 2 > b2 + c2 .)

140

6 Spanning Trees and Arborescences X

Y

r Fig. 6.3.

Now let A be the submatrix of A consisting of the rows corresponding to the elements of F. A is the one-way cut-incidence matrix of a laminar family (to be precise, we must consider the graph resulting from G by reversing each edge). So by Theorem 5.28 A is totally unimodular, as required. The above proof also yields the promised polyhedral description: Corollary 6.14. (Edmonds [1967]) Let G be a digraph with weights c : E(G) → R+ , and r ∈ V (G) such that G contains a spanning arborescence rooted at r . Then the LP ⎫ ⎧ ⎬ ⎨ xe ≥ 1 for all X ⊂ V (G) with r ∈ X min cx : x ≥ 0, ⎭ ⎩ + e∈δ (X )

has an integral optimum solution (which is the incidence vector of a minimum weight spanning arborescence rooted at r , plus possibly some edges of zero weight). For a description of the convex hull of the incidence vectors of all branchings or spanning arborescences rooted at r , see Exercises 16 and 17.

6.4 Packing Spanning Trees and Arborescences If we are looking for more than one spanning tree or arborescence, classical theorems of Tutte, Nash-Williams and Edmonds are of help. We ﬁrst give a proof of Tutte’s Theorem on packing spanning trees which is essentially due to Mader (see Diestel [1997]) and which uses the following lemma: Lemma 6.15. Let G be an undirected graph, and let F = (F1 , . . . , Fk ) be a k-tuple of edge-disjoint forests in G such that |E(F)| is maximum, where E(F) := k E(F i ). Let e ∈ E(G) \ E(F). Then there exists a set X ⊆ V (G) with e ⊆ X i=1 such that Fi [X ] is connected for each i ∈ {1, . . . , k}. Proof: For two k-tuples F = (F1 , . . . , Fk ) and F = (F1 , . . . , Fk ) we say that . F arises from F by exchanging e for e if Fj = (Fj \ e ) ∪ e for some j and

6.4 Packing Spanning Trees and Arborescences

141

Fi00 = Fi0 for all i 6= j. Let F be the set of all k-tuples of edge-disjoint forests arising T 0 from F by a sequence of such exchanges. Let E := E(G) \ F 0 ∈F E(F ) and G := (V (G), E). We have F ∈ F and thus e ∈ E. Let X be the vertex set of the connected component of G containing e. We shall prove that Fi [X ] is connected for each i. Claim: For any F 0 = (F10 , . . . , Fk0 ) ∈ F and any e¯ = {v, w} ∈ E(G[X ]) \ E(F 0 ) there exists a v-w-path in Fi0 [X ] for all i ∈ {1, . . . , k}. To prove this, let i ∈ {1, . . . , k} be fixed. Since F 0 ∈ F and |E(F 0 )| = |E(F)| is ¯ we have Fe00 ∈ F, maximum, Fi0 + e¯ contains a circuit C. Now for all e0 ∈ E(C)\{e} 0 0 0 ¯ This shows that E(C) ⊆ E, and where Fe0 arises from F by exchanging e for e. so C − e¯ is a v-w-path in Fi0 [X ]. This proves the claim. Since G[X ] is connected, it suffices to prove that for each e¯ = {v, w} ∈ E(G[X ]) and each i there is a v-w-path in Fi [X ]. So let e¯ = {v, w} ∈ E(G[X ]). Since e¯ ∈ E, there is some F 0 = (F10 , . . . , Fk0 ) ∈ F with e¯ 6∈ E(F 0 ). By the claim there is a v-w-path in Fi0 [X ] for each i. Now there is a sequence F = F (0) , F (1) . . . , F (s) = F 0 of elements of F such that F (r +1) arises from F (r ) by exchanging one edge (r = 0, . . . , s − 1). It suffices (r +1) [X ] implies the existence of a to show that the existence of a v-w-path in Fi (r ) v-w-path in Fi [X ] (r = 0, . . . , s − 1). (r +1) (r ) To see this, suppose that Fi [X ] arises from Fi [X ] by exchanging er for (r +1) er +1 , and let P be the v-w-path in Fi [X ]. If P does not contain er +1 = {x, y}, (r ) it is also a path in Fi [X ]. Otherwise er +1 ∈ E(G[X ]), and we consider the x-y(r ) path Q in Fi [X ] which exists by the claim. Since (E(P) \ {er +1 }) ∪ Q contains (r ) a v-w-path in Fi [X ], the proof is complete. Now we can prove Tutte’s theorem on disjoint spanning trees. A multicut in an undirected graph G is a set of edges δ(X 1 , . . . , X p ) := δ(X 1 ) ∪ · · · ∪ δ(X p ) . . . for some partition V (G) = X 1 ∪ X 2 ∪ · · · ∪ X p of the vertex set into nonempty subsets. For p = 3 we also speak of 3-cuts. Observe that cuts are multicuts with p = 2. Theorem 6.16. (Tutte [1961], Nash-Williams [1961]) An undirected graph G contains k edge-disjoint spanning trees if and only if |δ(X 1 , . . . , X p )| ≥ k( p − 1) for every multicut δ(X 1 , . . . , X p ). Proof: To prove necessity, let T1 , . . . , Tk be edge-disjoint spanning trees in G, and let δ(X 1 , . . . , X p ) be a multicut. Contracting each of the vertex subsets X 1 , . . . , X p yields a graph G 0 whose vertices are X 1 , . . . , X p and whose edges correspond to the edges of the multicut. T1 , . . . , Tk correspond to edge-disjoint connected subgraphs T10 , . . . , Tk0 in G 0 . Each of the T10 , . . . , Tk0 has at least p − 1 edges, so G 0 (and thus the multicut) has at least k( p − 1) edges.

142

6 Spanning Trees and Arborescences

To prove sufficiency we use induction on |V (G)|. For n := |V (G)| ≤ 2 the statement is true. Now assume n > 2, and suppose that |δ(X 1 , . . . , X p )| ≥ k( p −1) for every multicut δ(X 1 , . . . , X p ). In particular (consider the partition into singletons) G has at least k(n − 1) edges. Moreover, the condition is preserved when contracting vertex sets, so by the induction hypothesis G/ X contains k edge-disjoint spanning trees for each X ⊂ V (G) with |X | ≥ 2. edge-disjoint forests in G such that Let F = (F1 , . . . , Fk ) be a k-tuple of S k E(Fi ). We claim that each Fi is |E(F)| is maximum, where again E(F) := i=1 a spanning tree. Otherwise |E(F)| < k(n −1), so there is an edge e ∈ E(G)\ E(F). By Lemma 6.15 there is an X ⊆ V (G) with e ⊆ X such that Fi [X ] is connected for each i. Since |X | ≥ 2, G/ X contains k edge-disjoint spanning trees F10 , . . . , Fk0 . Now Fi0 together with Fi [X ] forms a spanning tree in G for each i, and all these k spanning trees are edge-disjoint. We now turn to the corresponding problem in digraphs, packing spanning arborescences: Theorem 6.17. (Edmonds [1973]) Let G be a digraph and r ∈ V (G). Then the maximum number of edge-disjoint spanning arborescences rooted at r equals the minimum cardinality of an r -cut. Proof: Let k be the minimum cardinality of an r -cut. Obviously there are at most k edge-disjoint spanning arborescences. We prove the existence of k edge-disjoint spanning arborescences by induction on k. The case k = 0 is trivial. If we can find one spanning arborescence A rooted at r such that + (S) \ E(A) ≥ k − 1, (6.2) min δG r ∈S⊂V (G)

then we are done by induction. Suppose we have already found some arborescence A rooted at r (but not necessarily spanning) such that (6.2) holds. Let R ⊆ V (G) be the set of vertices covered by A. Initially, R = {r }; if R = V (G), we are done. If R 6= V (G), we call a set X ⊆ V (G) critical if (a) r ∈ X ; (b) X ∪ R 6= V (G); + (X ) \ E(A)| = k − 1. (c) |δG If there is no critical vertex set, we can augment A by any edge leaving R. Otherwise let X be a maximal critical set, and let e = (x, y) be an edge such that x ∈ R \ X and y ∈ V (G) \ (R ∪ X ) (see Figure 6.4). Such an edge must exist because + + + (R ∪ X )| = |δG (R ∪ X )| ≥ k > k − 1 = |δG−E(A) (X )|. |δG−E(A)

We now add e to A. Obviously A + e is an arborescence rooted at r . We have to show that (6.2) continues to hold. + Suppose there is some Y such that r ∈ Y ⊂ V (G) and |δG (Y )\E(A+e)| < k−1. + Then x ∈ Y , y ∈ / Y , and |δG (Y ) \ E(A)| = k − 1. Now Lemma 2.1(a) implies

6.4 Packing Spanning Trees and Arborescences

143

R x

r e X

y

Fig. 6.4. + + k − 1 + k − 1 = |δG−E(A) (X )| + |δG−E(A) (Y )| + + (X ∪ Y )| + |δG−E(A) (X ∩ Y )| ≥ |δG−E(A) ≥ k−1+k−1,

because r ∈ X ∩ Y and y ∈ V (G) \ (X ∪ Y ). So equality must hold throughout, in + particular |δG−E(A) (X ∪ Y )| = k − 1. Since y ∈ V (G) \ (X ∪ Y ∪ R) we conclude that X ∪ Y is critical. But since x ∈ Y \ X , this contradicts the maximality of X . This proof is due to Lov´asz [1976]. A generalization of Theorems 6.16 and 6.17 was found by Frank [1978]. A good characterization of the problem of packing spanning arborescences with arbitrary roots is given by the following theorem, which we cite without proof: Theorem 6.18. (Frank [1979]) A digraph G contains k edge-disjoint spanning arborescences if and only if p

|δ − (X i )| ≥ k( p − 1)

i=1

for every collection of pairwise disjoint nonempty subsets X 1 , . . . , X p ⊆ V (G). Another question is how many forests are needed to cover a graph. This is answered by the following theorem: Theorem 6.19. (Nash-Williams [1964]) The edge set of an undirected graph G is the union of k forests if and only if |E(G[X ])| ≤ k(|X |−1) for all ∅ = X ⊆ V (G). Proof: The necessity is clear since no forest can contain more than |X | − 1 edges within a vertex set X . To prove the sufﬁciency, assume that |E(G[X ])| ≤ k(|X |−1) for all ∅ = X ⊆ V (G), and let F = (F1 , . . . , Fk ) be a k-tuple of disjoint forests in k E(Fi ) is maximum. We claim that E(F) = E(G). G such that |E(F)| = i=1 To see this, suppose there is an edge e ∈ E(G) \ E(F). By Lemma 6.15 there exists a set X ⊆ V (G) with e ⊆ X such that Fi [X ] is connected for each i. In particular,

144

6 Spanning Trees and Arborescences

k . [ |E(G[X ])| ≥ {e} ∪ E(Fi [X ]) ≥ 1 + k(|X | − 1), i=1

contradicting the assumption.

Exercise 22 gives a directed version. A generalization of Theorems 6.16 and 6.19 to matroids can be found in Exercise 18 of Chapter 13.

Exercises 1. Prove Cayley’s theorem, stating that K n has n n−2 spanning trees, by showing that the following defines a one-to-one correspondence between the spanning trees in K n and the vectors in {1, . . . , n}n−2 : For a tree T with V (T ) = {1, . . . , n}, n ≥ 3, let v be the leaf with the smallest index and let a1 be the neighbour of v. We recursively define a(T ) := (a1 , . . . , an−2 ), where (a2 , . . . , an−2 ) = a(T − v). (Cayley [1889], Pr¨ufer [1918]) 2. Let (V, T1 ) and (V, T2 ) be two trees on the same vertex set V . Prove that for any edge e ∈ T1 there is an edge f ∈ T2 such that both (V, (T1 \ {e}) ∪ { f }) and (V, (T2 \ { f }) ∪ {e}) are trees. 3. Given an undirected graph G with weights c : E(G) → R and a vertex v ∈ V (G), we ask for a minimum weight spanning tree in G where v is not a leaf. Can you solve this problem in polynomial time? 4. We want to determine the set of edges e in an undirected graph G with weights c : E(G) → R for which there exists a minimum weight spanning tree in G containing e (in other words, we are looking for the union of all minimum weight spanning trees in G). Show how this problem can be solved in O(mn) time. 5. Given an undirected graph G with arbitrary weights c : E(G) → R, we ask for a minimum weight connected spanning subgraph. Can you solve this problem efficiently? 6. Consider the following algorithm (sometimes called W ORST-O UT-G REEDY A LGORITHM, see Section 13.4). Examine the edges in order of non-increasing weights. Delete an edge unless it is a bridge. Does this algorithm solve the M INIMUM S PANNING T REE P ROBLEM? 7. Consider the following “colouring” algorithm. Initially all edges are uncoloured. Then apply the following rules in arbitrary order until all edges are coloured: Blue rule: Select a cut containing no blue edge. Among the uncoloured edges in the cut, select one of minimum cost and colour it blue. Red rule: Select a circuit containing no red edge. Among the uncoloured edges in the circuit, select one of maximum cost and colour it red. Show that one of the rules is always applicable as long as there are uncoloured edges left. Moreover, show that the algorithm maintains the “colour invariant”:

Exercises

145

there always exists an optimum spanning tree containing all blue edges but no red edge. (So the algorithm solves the M INIMUM S PANNING T REE P ROBLEM optimally.) Observe that K RUSKAL ’ S A LGORITHM and P RIM ’ S A LGORITHM are special cases. (Tarjan [1983]) 8. Suppose we wish to find a spanning tree T in an undirected graph such that the maximum weight of an edge in T is as small as possible. How can this be done? 9. For a finite set V ⊂ R2 , the Vorono¨ı diagram consists of the regions Pv := x ∈ R2 : ||x − v||2 = min ||x − w||2 w∈V

for v ∈ V . The Delaunay triangulation of V is the graph (V, {{v, w} ⊆ V, v 6= w, |Pv ∩ Pw | > 1}) .

10.

11.

12.

13.

A P minimum spanning tree for V is a tree T with V (T ) = V whose length {v,w}∈E(T ) ||v − w||2 is minimum. Prove that every minimum spanning tree is a subgraph of the Delaunay triangulation. Note: Using the fact that the Delaunay triangulation can be computed in O(n log n) time (where n = |V |; see e.g. Fortune [1987], Knuth [1992]), this implies an O(n log n) algorithm for the M INIMUM S PANNING T REE P ROB LEM for point sets in the plane. (Shamos and Hoey [1975]; see also Zhou, Shenoy and Nicholls [2002]) Can you decide in linear time whether a graph contains a spanning arborescence? Hint: To find a possible root, start at an arbitrary vertex and traverse edges backwards as long as possible. When encountering a circuit, contract it. Can you find a maximum cardinality branching in a given digraph in linear time? Hint: First find the strongly connected components. The M INIMUM W EIGHT ROOTED A RBORESCENCE P ROBLEM can be reduced to the M AXIMUM W EIGHT B RANCHING P ROBLEM by Proposition 6.8. However, it can also be solved directly by a modified version of E DMONDS ’ B RANCHING A LGORITHM. Show how. Prove that the spanning tree polytope of an undirected graph G (see Theorem 6.12) with n := |V (G)| is in general a proper subset of the polytope X X xe = n − 1, xe ≥ 1 for ∅ ⊂ X ⊂ V (G) . x ∈ [0, 1] E(G) : e∈E(G)

e∈δ(X )

Hint: To prove that this polytope is not integral, consider the graph shown in Figure 6.5 (the numbers are edge weights). (Magnanti and Wolsey [1995])

146

6 Spanning Trees and Arborescences 1

0 0

1

1

0

Fig. 6.5.

∗ 14. In Exercise 13 we saw that cut constraints do not sufﬁce to describe the spanning tree polytope. However, if we consider multicuts instead, we obtain a complete description: Prove that the spanning tree polytope of an undirected graph G with n := |V (G)| consists of all vectors x ∈ [0, 1] E(G) with xe = n − 1 and xe ≥ k − 1 for all multicuts C = δ(X 1 , . . . , X k ). e∈E(G)

e∈C

(Magnanti and Wolsey [1995]) 15. Prove that the convex hull of the incidence vectors of all forests in an undirected graph G is the polytope ⎧ ⎫ ⎨ ⎬ P := x ∈ [0, 1] E(G) : xe ≤ |X | − 1 for ∅ = X ⊆ V (G) . ⎩ ⎭ e∈E(G[X ])

Note: This statement implies Theorem 6.12 since e∈E(G[X ]) xe = |V (G)| −1 is a supporting hyperplane. Moreover, it is a special case of Theorem 13.21. ∗ 16. Prove that the convex hull of the incidence vectors of all branchings in a digraph G is the set of all vectors x ∈ [0, 1] E(G) with xe ≤ |X | − 1 for ∅ = X ⊆ V (G) and xe ≤ 1 for v ∈ V (G). e∈δ − (v)

e∈E(G[X ])

Note: This is a special case of Theorem 14.13. ∗ 17. Let G be a digraph and r ∈ V (G). Prove that the polytopes xe = 1 (v ∈ V (G) \ {r }), x ∈ [0, 1] E(G) : xe = 0 (e ∈ δ − (r )), e∈δ − (v)

e∈E(G[X ])

and

xe ≤ |X | − 1 for ∅ = X ⊆ V (G)

References

x ∈ [0, 1] E(G) : xe = 0 (e ∈ δ − (r )),

X

147

xe = 1 (v ∈ V (G) \ {r }),

e∈δ − (v)

X

xe ≥ 1 for r ∈ X ⊂ V (G)

e∈δ + (X )

18.

19.

∗ 20.

21.

22.

are both equal to the convex hull of the incidence vectors of all spanning arborescences rooted at r . Let G be a digraph and r ∈ V (G). Prove that G is the disjoint union of k spanning arborescences rooted at r if and only if the underlying undirected graph is the disjoint union of k spanning trees and |δ − (x)| = k for all x ∈ V (G) \ {r }. (Edmonds) Let G be a digraph and r ∈ V (G). Suppose that G contains k edge-disjoint paths from r to every other vertex, but removing any edge destroys this property. Prove that every vertex of G except r has exactly k entering edges. Hint: Use Theorem 6.17. Prove the statement of Exercise 19 without using Theorem 6.17. Formulate and prove a vertex-disjoint version. Hint: If a vertex v has more than k entering edges, take k edge-disjoint r -vpaths. Show that an edge entering v that is not used by these paths can be deleted. Give a polynomial-time algorithm for finding a maximum set of edge-disjoint spanning arborescences (rooted at r ) in a digraph G. Note: The most efficient algorithm is due to Gabow [1995]; see also (Gabow and Manu [1998]). Prove that the edges of a digraph G can be covered by k branchings if and only if the following two conditions hold: (a) |δ − (v)| ≤ k for all v ∈ V (G); (b) |E(G[X ])| ≤ k(|X | − 1) for all X ⊆ V (G). Hint: Use Theorem 6.17. (Frank [1979])

References General Literature: Ahuja, R.K., Magnanti, T.L., and Orlin, J.B. [1993]: Network Flows. Prentice-Hall, Englewood Cliffs 1993, Chapter 13 Balakrishnan, V.K. [1995]: Network Optimization. Chapman and Hall, London 1995, Chapter 1 Cormen, T.H., Leiserson, C.E., Rivest, R.L., and Stein, C. [2001]: Introduction to Algorithms. Second Edition. MIT Press, Cambridge 2001, Chapter 23 Gondran, M., and Minoux, M. [1984]: Graphs and Algorithms. Wiley, Chichester 1984, Chapter 4

148

6 Spanning Trees and Arborescences

Magnanti, T.L., and Wolsey, L.A. [1995]: Optimal trees. In: Handbooks in Operations Research and Management Science; Volume 7: Network Models (M.O. Ball, T.L. Magnanti, C.L. Monma, G.L. Nemhauser, eds.), Elsevier, Amsterdam 1995, pp. 503–616 Schrijver, A. [2003]: Combinatorial Optimization: Polyhedra and Efficiency. Springer, Berlin 2003, Chapters 50–53 Tarjan, R.E. [1983]: Data Structures and Network Algorithms. SIAM, Philadelphia 1983, Chapter 6 Wu, B.Y., and Chao, K.-M. [2004]: Spanning Trees and Optimization Problems. Chapman & Hall/CRC, Boca Raton 2004

Cited References: Bock, F.C. [1971]: An algorithm to construct a minimum directed spanning tree in a directed network. In: Avi-Itzak, B. (Ed.): Developments in Operations Research. Gordon and Breach, New York 1971, 29–44 Bor˚uvka, O. [1926a]: O jist´em probl´emu minim´aln´ım. Pr´aca Moravsk´e P˘r´ırodov˘edeck´e Spolne˘cnosti 3 (1926), 37–58 Bor˚uvka, O. [1926b]: P˘r´ıspev˘ek k ˘re˘sen´ı ot´azky ekonomick´e stavby. Elektrovodn´ıch s´ıt´ı. Elektrotechnicky Obzor 15 (1926), 153–154 Cayley, A. [1889]: A theorem on trees. Quarterly Journal on Mathematics 23 (1889), 376–378 Chazelle, B. [2000]: A minimum spanning tree algorithm with inverse-Ackermann type complexity. Journal of the ACM 47 (2000), 1028–1047 Cheriton, D., and Tarjan, R.E. [1976]: Finding minimum spanning trees. SIAM Journal on Computing 5 (1976), 724–742 Chu, Y., and Liu, T. [1965]: On the shortest arborescence of a directed graph. Scientia Sinica 4 (1965), 1396–1400; Mathematical Review 33, # 1245 Diestel, R. [1997]: Graph Theory. Springer, New York 1997 Dijkstra, E.W. [1959]: A note on two problems in connexion with graphs. Numerische Mathematik 1 (1959), 269–271 Dixon, B., Rauch, M., and Tarjan, R.E. [1992]: Verification and sensitivity analysis of minimum spanning trees in linear time. SIAM Journal on Computing 21 (1992), 1184– 1192 Edmonds, J. [1967]: Optimum branchings. Journal of Research of the National Bureau of Standards B 71 (1967), 233–240 Edmonds, J. [1970]: Submodular functions, matroids and certain polyhedra. In: Combinatorial Structures and Their Applications; Proceedings of the Calgary International Conference on Combinatorial Structures and Their Applications 1969 (R. Guy, H. Hanani, N. Sauer, J. Schonheim, eds.), Gordon and Breach, New York 1970, pp. 69–87 Edmonds, J. [1973]: Edge-disjoint branchings. In: Combinatorial Algorithms (R. Rustin, ed.), Algorithmic Press, New York 1973, pp. 91–96 Fortune, S. [1987]: A sweepline algorithm for Voronoi diagrams. Algorithmica 2 (1987), 153–174 Frank, A. [1978]: On disjoint trees and arborescences. In: Algebraic Methods in Graph Theory; Colloquia Mathematica; Soc. J. Bolyai 25 (L. Lov´asz, V.T. S´os, eds.), NorthHolland, Amsterdam 1978, pp. 159–169 Frank, A. [1979]: Covering branchings. Acta Scientiarum Mathematicarum (Szeged) 41 (1979), 77–82

References

149

Fredman, M.L., and Tarjan, R.E. [1987]: Fibonacci heaps and their uses in improved network optimization problems. Journal of the ACM 34 (1987), 596–615 Fredman, M.L., and Willard, D.E. [1994]: Trans-dichotomous algorithms for minimum spanning trees and shortest paths. Journal of Computer and System Sciences 48 (1994), 533–551 Fulkerson, D.R. [1974]: Packing rooted directed cuts in a weighted directed graph. Mathematical Programming 6 (1974), 1–13 Gabow, H.N. [1995]: A matroid approach to finding edge connectivity and packing arborescences. Journal of Computer and System Sciences 50 (1995), 259–273 Gabow, H.N., Galil, Z., and Spencer, T. [1989]: Efficient implementation of graph algorithms using contraction. Journal of the ACM 36 (1989), 540–572 Gabow, H.N., Galil, Z., Spencer, T., and Tarjan, R.E. [1986]: Efficient algorithms for finding minimum spanning trees in undirected and directed graphs. Combinatorica 6 (1986), 109–122 Gabow, H.N., and Manu, K.S. [1998]: Packing algorithms for arborescences (and spanning trees) in capacitated graphs. Mathematical Programming B 82 (1998), 83–109 Jarn´ık, V. [1930]: O jist´em probl´emu minim´aln´ım. Pr´aca Moravsk´e P˘r´ırodov˘edeck´e Spole˘cnosti 6 (1930), 57–63 Karger, D., Klein, P.N., and Tarjan, R.E. [1995]: A randomized linear-time algorithm to find minimum spanning trees. Journal of the ACM 42 (1995), 321–328 Karp, R.M. [1972]: A simple derivation of Edmonds’ algorithm for optimum branchings. Networks 1 (1972), 265–272 King, V. [1995]: A simpler minimum spanning tree verification algorithm. Algorithmica 18 (1997), 263–270 Knuth, D.E. [1992]: Axioms and hulls; LNCS 606. Springer, Berlin 1992 Korte, B., and Neˇsetˇril, J. [2001]: Vojt˘ech Jarn´ık’s work in combinatorial optimization. Discrete Mathematics 235 (2001), 1–17 Kruskal, J.B. [1956]: On the shortest spanning subtree of a graph and the travelling salesman problem. Proceedings of the AMS 7 (1956), 48–50 Lov´asz, L. [1976]: On two minimax theorems in graph. Journal of Combinatorial Theory B 21 (1976), 96–103 Nash-Williams, C.S.J.A. [1961]: Edge-disjoint spanning trees of finite graphs. Journal of the London Mathematical Society 36 (1961), 445–450 Nash-Williams, C.S.J.A. [1964]: Decompositions of finite graphs into forests. Journal of the London Mathematical Society 39 (1964), 12 Neˇsetˇril, J., Milkov´a, E., and Neˇsetˇrilov´a, H. [2001]: Otakar Bor˚uvka on minimum spanning tree problem. Translation of both the 1926 papers, comments, history. Discrete Mathematics 233 (2001), 3–36 Prim, R.C. [1957]: Shortest connection networks and some generalizations. Bell System Technical Journal 36 (1957), 1389–1401 Pr¨ufer, H. [1918]: Neuer Beweis eines Satzes u¨ ber Permutationen. Arch. Math. Phys. 27 (1918), 742–744 Shamos, M.I., and Hoey, D. [1975]: Closest-point problems. Proceedings of the 16th Annual IEEE Symposium on Foundations of Computer Science (1975), 151–162 Tarjan, R.E. [1975]: Efficiency of a good but not linear set union algorithm. Journal of the ACM 22 (1975), 215–225 Tutte, W.T. [1961]: On the problem of decomposing a graph into n connected factor. Journal of the London Mathematical Society 36 (1961), 221–230 Zhou, H., Shenoy, N., and Nicholls, W. [2002]: Efficient minimum spanning tree construction without Delaunay triangulation. Information Processing Letters 81 (2002), 271–276

7 Shortest Paths

One of the best-known combinatorial optimization problems is to find a shortest path between two specified vertices of a digraph:

S HORTEST PATH P ROBLEM Instance:

A digraph G, weights c : E(G) → R and two vertices s, t ∈ V (G).

Task:

Find a shortest s-t-path P, i.e. one of minimum weight c(E(P)), or decide that t is not reachable from s.

Obviously this problem has many practical applications. Like the M INIMUM S PANNING T REE P ROBLEM it also often appears as a subproblem when one deals with more difficult combinatorial optimization problems. In fact, the problem is not easy to solve if we allow arbitrary weights. For example, if all weights are −1 then the s-t-paths of weight 1−|V (G)| are precisely the Hamiltonian s-t-paths. Deciding whether such a path exists is a difficult problem (see Exercise 14(b) of Chapter 15). The problem becomes much easier if we restrict ourselves to nonnegative weights or at least exclude negative circuits: Definition 7.1. Let G be a (directed or undirected) graph with weights c : E(G) → R. c is called conservative if there is no circuit of negative total weight. We shall present algorithms for the S HORTEST PATH P ROBLEM in Section 7.1. The first one allows nonnegative weights only while the second algorithm can deal with arbitrary conservative weights. The algorithms of Section 7.1 in fact compute a shortest s-v-path for all v ∈ V (G) without using significantly more running time. Sometimes one is interested in the distance for every pair of vertices; Section 7.2 shows how to deal with this problem. Since negative circuits cause problems we also show how to detect them. If none exists, a circuit of minimum total weight can be computed quite easily. Another interesting problem asks for a circuit whose mean weight is minimum. As we shall see in Section 7.3 this problem can also be solved efficiently by similar techniques. Finding shortest paths in undirected graphs is more difficult unless the edge weights are nonnegative. Undirected edges of nonnegative weights can be replaced equivalently by a pair of oppositely directed edges of the same weight; this reduces

152

7 Shortest Paths

the undirected problem to a directed one. However, this construction does not work for edges of negative weight since it would introduce negative circuits. We shall return to the problem of finding shortest paths in undirected graphs with conservative weights in Section 12.2 (Corollary 12.12). Henceforth we work with a digraph G. Without loss of generality we may assume that G is connected and simple; among parallel edges we have to consider only the one with least weight.

7.1 Shortest Paths From One Source All shortest path algorithms we present are based on the following observation, sometimes called Bellman’s principle of optimality, which is indeed the core of dynamic programming: Proposition 7.2. Let G be a digraph with conservative weights c : E(G) → R, let k ∈ N, and let s and w be two vertices. Let P be a shortest one among all s-w-paths with at most k edges, and let e = (v, w) be its final edge. Then P[s,v] (i.e. P without the edge e) is a shortest one among all s-v-paths with at most k − 1 edges. Proof: Suppose Q is a shorter s-v-path than P[s,v] , and |E(Q)| ≤ k − 1. Then c(E(Q)) + c(e) < c(E(P)). If Q does not contain w, then Q + e is a shorter s-w-path than P, otherwise Q [s,w] has length c(E(Q [s,w] )) = c(E(Q)) + c(e) − c(E(Q [w,v] + e)) < c(E(P)) − c(E(Q [w,v] + e)) ≤ c(E(P)), because Q [w,v] + e is a circuit and c is conservative. In both cases we have a contradiction to the assumption that P is a shortest s-w-path with at most k edges. The same result holds for undirected graphs with nonnegative weights and also for acyclic digraphs with arbitrary weights. It yields the recursion formulas dist(s, s) = 0 and dist(s, w) = min{dist(s, v) + c((v, w)) : (v, w) ∈ E(G)} for w ∈ V (G) \ {s} which immediately solve the S HORTEST PATH P ROBLEM for acyclic digraphs (Exercise 6). Proposition 7.2 is also the reason why most algorithms compute the shortest paths from s to all other vertices. If one computes a shortest s-t-path P, one has already computed a shortest s-v-path for each vertex v on P. Since we cannot know in advance which vertices belong to P, it is only natural to compute shortest s-v-paths for all v. We can store these s-v-paths very efficiently by just storing the final edge of each path. We first consider nonnegative edge weights, i.e. c : E(G) → R+ . The S HORTEST PATH P ROBLEM can be solved by BFS if all weights are 1 (Proposition 2.18). For weights c : E(G) → N one could replace an edge e by a path of length c(e) and again use BFS. However, this might introduce an exponential number P of edges; recall that the input size is 2 n log m + m log n + e∈E(G) log c(e) , where n = |V (G)| and m = |E(G)|.

7.1 Shortest Paths From One Source

153

A much better idea is to use the following algorithm, due to Dijkstra [1959]. It is quite similar to P RIM ’ S A LGORITHM for the M INIMUM S PANNING T REE P ROBLEM (Section 6.1).

D IJKSTRA ’ S A LGORITHM Input:

A digraph G, weights c : E(G) → R+ and a vertex s ∈ V (G).

Output:

Shortest paths from s to all v ∈ V (G) and their lengths. More precisely, we get the outputs l(v) and p(v) for all v ∈ V (G). l(v) is the length of a shortest s-v-path, which consists of a shortest s- p(v)-path together with the edge ( p(v), v). If v is not reachable from s, then l(v) = ∞ and p(v) is undefined.

2

Set l(s) := 0. Set l(v) := ∞ for all v ∈ V (G) \ {s}. Set R := ∅. Find a vertex v ∈ V (G) \ R such that l(v) = min

3

Set R := R ∪ {v}.

4

For all w ∈ V (G) \ R such that (v, w) ∈ E(G) do: If l(w) > l(v) + c((v, w)) then set l(w) := l(v) + c((v, w)) and p(w) := v. If R 6= V (G) then go to . 2

1

5

w∈V (G)\R

l(w).

Theorem 7.3. (Dijkstra [1959]) D IJKSTRA ’ S A LGORITHM works correctly. Proof: We prove that the following statements hold at any stage of the algorithm: (a) For each v ∈ V (G) \ {s} with l(v) < ∞ we have p(v) ∈ R, l( p(v)) + c(( p(v), v)) = l(v), and the sequence v, p(v), p( p(v)), . . . contains s. (b) For all v ∈ R: l(v) = dist(G,c) (s, v). The statements trivially hold after . 1 l(w) is decreased to l(v) + c((v, w)) / R. As the sequence and p(w) is set to v in 4 only if v ∈ R and w ∈ v, p(v), p( p(v)), . . . contains s but no vertex outside R, in particular not w, (a) is preserved by . 4 (b) is trivial for v = s. Suppose that v ∈ V (G) \ {s} is added to R in , 3 and there is an s-v-path P in G that is shorter than l(v). Let y be the first vertex on P that belongs to (V (G) \ R) ∪ {v}, and let x be the predecessor of y on P. Since x ∈ R, we have by 4 and the induction hypothesis: l(y) ≤ l(x) + c((x, y)) = dist(G,c) (s, x) + c((x, y)) ≤ c(E(P[s,y] )) ≤ c(E(P)) < l(v), contradicting the choice of v in . 2

The running time is obviously O(n 2 ). Using a Fibonacci heap we can do better:

154

7 Shortest Paths

Theorem 7.4. (Fredman and Tarjan [1987]) D IJKSTRA ’ S A LGORITHM implemented with a Fibonacci heap runs in O(m + n log n) time, where n = |V (G)| and m = |E(G)|. Proof: We apply Theorem 6.6 to maintain the set {(v, l(v)) : v ∈ V (G) \ R, l(v) < ∞}. Then 2 and 3 are one DELETEMIN-operation, while the update of l(w) in 4 is an INSERT-operation if l(w) was infinite and a DECREASEKEYoperation otherwise. This is the best known strongly polynomial running time for the S HORTEST PATH P ROBLEM with nonnegative weights. (On different computational models, Fredman and Willard [1994], Thorup [2000] and Raman [1997] achieved slightly better running times.) If the weights are integers within a fixed range there is a simple linear-time algorithm (Exercise 2). In general, running times of O(m log log cmax ) (Johnson p [1982]) and O m + n log cmax (Ahuja et al. [1990]) are possible for weights c : E(G) → {0, . . . , cmax }. This has been improved by Thorup [2004] to O(m + n log log cmax ) and O(m + n log log n), but even the latter bound applies to integral edge weights only, and the algorithm is not strongly polynomial. For planar digraphs there is a linear-time algorithm due to Henzinger et al. [1997]. Finally we mention that Thorup [1999] found a linear-time algorithm for finding a shortest path in an undirected graph with nonnegative integral weights. See also Pettie and Ramachandran [2005]; this paper also contains more references. We now turn to an algorithm for general conservative weights:

M OORE -B ELLMAN -F ORD A LGORITHM Input: Output:

A digraph G, conservative weights c : E(G) → R, and a vertex s ∈ V (G). Shortest paths from s to all v ∈ V (G) and their lengths. More precisely, we get the outputs l(v) and p(v) for all v ∈ V (G). l(v) is the length of a shortest s-v-path which consists of a shortest s- p(v)-path together with the edge ( p(v), v). If v is not reachable from s, then l(v) = ∞ and p(v) is undefined.

1

Set l(s) := 0 and l(v) := ∞ for all v ∈ V (G) \ {s}.

2

For i := 1 to n − 1 do: For each edge (v, w) ∈ E(G) do: If l(w) > l(v) + c((v, w)) then set l(w) := l(v) + c((v, w)) and p(w) := v.

Theorem 7.5. (Moore [1959], Bellman [1958], Ford [1956]) The M OORE -B ELL MAN -F ORD A LGORITHM works correctly. Its running time is O(nm). Proof: The O(nm) running time is obvious. At any stage of the algorithm let R := {v ∈ V (G) : l(v) < ∞} and F := {(x, y) ∈ E(G) : x = p(y)}. We claim:

7.1 Shortest Paths From One Source

155

(a) l(y) ≥ l(x) + c((x, y)) for all (x, y) ∈ F; (b) If F contains a circuit C, then C has negative total weight; (c) If c is conservative, then (R, F) is an arborescence rooted at s. To prove (a), observe that l(y) = l(x) + c((x, y)) when p(y) is set to x and l(x) is never increased. To prove (b), suppose at some stage a circuit C in F was created by setting p(y) := x. Then before the insertion we had l(y) > l(x) + c((x, y)) as well as l(w) ≥ l(v) + c((v, w)) for all (v, w) ∈ E(C) \ {(x, y)} (by (a)). Summing these inequalities (the l-values cancel), we see that the total weight of C is negative. Since c is conservative, (b) implies that F is acyclic. Moreover, x ∈ R \ {s} implies p(x) ∈ R, so (R, F) is an arborescence rooted at s. Therefore l(x) is at least the length of the s-x-path in (R, F) for any x ∈ R (at any stage of the algorithm). We claim that after k iterations of the algorithm, l(x) is at most the length of a shortest s-x-path with at most k edges. This statement is easily proved by induction: Let P be a shortest s-x-path with at most k edges and let (w, x) be the last edge of P. Then, by Proposition 7.2, P[s,w] must be a shortest s-w-path with at most k − 1 edges, and by the induction hypothesis we have l(w) ≤ c(E(P[s,w] )) after k − 1 iterations. But in the k-th iteration edge (w, x) is also examined, after which l(x) ≤ l(w) + c((w, x)) ≤ c(E(P)). Since no path has more than n−1 edges, the above claim implies the correctness of the algorithm. This algorithm is still the fastest known strongly polynomial-time algorithm for the S HORTEST PATH P ROBLEM (with conservative algorithm √ weights). A scaling due to Goldberg [1995] has a running time of O nm log(|cmin | + 2) if the edge weights are integral and at least cmin . For planar graphs, Fakcharoenphol and Rao [2006] described an O(n log3 n)-algorithm. If G contains negative circuits, no polynomial-time algorithm is known (the problem becomes NP-hard; see Exercise 14(b) of Chapter 15). The main difficulty is that Proposition 7.2 does not hold for general weights. It is not clear how to construct a path instead of an arbitrary edge progression. If there are no negative circuits, any shortest edge progression is a path, plus possibly some circuits of zero weight that can be deleted. In view of this it is also an important question how to detect negative circuits. The following concept due to Edmonds and Karp [1972] is useful: Definition 7.6. Let G be a digraph with weights c : E(G) → R, and let π : V (G) → R. Then for any (x, y) ∈ E(G) we define the reduced cost of (x, y) with respect to π by cπ ((x, y)) := c((x, y)) + π(x) − π(y). If cπ (e) ≥ 0 for all e ∈ E(G), π is called a feasible potential. Theorem 7.7. Let G be a digraph with weights c : E(G) → R. There exists a feasible potential of (G, c) if and only if c is conservative.

156

7 Shortest Paths

Proof: If π is a feasible potential, we have for each circuit C: X X cπ (e) = (c(e) + π(x) − π(y)) = 0 ≤ e∈E(C)

e=(x,y)∈E(C)

X

c(e)

e∈E(C)

(the potentials cancel). So c is conservative. On the other hand, if c is conservative, we add a new vertex s and edges (s, v) of zero cost for all v ∈ V (G). We run the M OORE -B ELLMAN -F ORD A LGORITHM on this instance and obtain numbers l(v) for all v ∈ V (G). Since l(v) is the length of a shortest s-v-path for all v ∈ V (G), we have l(w) ≤ l(v) + c((v, w)) for all (v, w) ∈ E(G). Hence l is a feasible potential. This can be regarded as a special form of LP duality; see Exercise 8. Corollary 7.8. Given a digraph G with weights c : E(G) → R we can find in O(nm) time either a feasible potential or a negative circuit. Proof: As above, we add a new vertex s and edges (s, v) of zero cost for all v ∈ V (G). We run a modified version of the M OORE -B ELLMAN -F ORD A LGORITHM on this instance: Regardless of whether c is conservative or not, we run 1 and

2 as above. We obtain numbers l(v) for all v ∈ V (G). If l is a feasible potential, we are done. Otherwise let (v, w) be any edge with l(w) > l(v) + c((v, w)). We claim that the sequence w, v, p(v), p( p(v)), . . . contains a circuit. To see this, observe that l(v) must have been changed in the final iteration of . 2 Hence l( p(v)) has been changed during the last two iterations, l( p( p(v))) has been changed during the last three iterations, and so on. Since l(s) never changes, the first |V (G)| places of the sequence w, v, p(v), p( p(v)), . . . do not contain s, so a vertex must appear twice in the sequence. Thus we have found a circuit C in F := {(x, y) ∈ E(G) : x = p(y)}∪{(v, w)}. By (a) and (b) of the proof of Theorem 7.5, C has negative total weight. In practice there are more efficient methods to detect negative circuits; see Cherkassky and Goldberg [1999].

7.2 Shortest Paths Between All Pairs of Vertices Suppose we now want to find a shortest s-t-path for all ordered pairs of vertices (s, t) in a digraph:

A LL PAIRS S HORTEST PATHS P ROBLEM Instance:

A digraph G and conservative weights c : E(G) → R.

Task:

Find numbers lst and vertices pst for all s, t ∈ V (G) with s 6= t, such that lst is the length of a shortest s-t-path, and ( pst , t) is the final edge of such a path (if it exists).

7.2 Shortest Paths Between All Pairs of Vertices

157

Of course we could run the M OORE -B ELLMAN -F ORD A LGORITHM n times, once for each choice of s. This immediately gives us an O(n 2 m)-algorithm. However, one can do better: Theorem 7.9. The A LL PAIRS S HORTEST PATHS P ROBLEM can be solved in O(mn + n 2 log n) time, where n = |V (G)| and m = |E(G)|. Proof: Let (G, c) be an instance. First we compute a feasible potential π, which is possible in O(nm) time by Corollary 7.8. Then for each s ∈ V (G) we do a single-source shortest path computation from s using the reduced costs cπ instead of c. For any vertex t the resulting s-t-path is also a shortest path with respect to c, because the length of any s-t-path changes by π(s) − π(t), a constant. Since the reduced costs are nonnegative, we can use D IJKSTRA ’ S A LGORITHM each time. So, by Theorem 7.4, the total running time is O(mn + n(m + n log n)). The same idea will be used again in Chapter 9 (in the proof of Theorem 9.12). Pettie [2004] showed how to improve the running time to O(mn +n 2 log log n); this is the best known time bound. For dense graphs with nonnegative weights, Chan’s [2007] bound of O(n 3 log3 log n/ log2 n) is slightly better. If all edge weights are small positive integers, this can be improved using fast matrix multiplication; see e.g. Zwick [2002]. The solution of the A LL PAIRS S HORTEST PATHS P ROBLEM also enables us to compute the metric closure: Definition 7.10. Given a graph G (directed or undirected) with conservative ¯ c), weights c : E(G) → R. The metric closure of (G, c) is the pair (G, ¯ where G¯ is the simple graph on V (G) that, for x, y ∈ V (G) with x 6= y, contains an edge e = {x, y} (or e = (x, y) if G is directed) with weight c(e) ¯ = dist(G,c) (x, y) if and only if y is reachable from x in G. Corollary 7.11. Let G be a directed or undirected graph with conservative weights c : E(G) → R. Then the metric closure of (G, c) can be computed in O(mn + n 2 log n) time. Proof: If G is undirected, we replace each edge by a pair of oppositely directed edges. Then we solve the resulting instance of the A LL PAIRS S HORTEST PATHS P ROBLEM. The rest of the section is devoted to the F LOYD -WARSHALL A LGORITHM, another O(n 3 )-algorithm for the A LL PAIRS S HORTEST PATHS P ROBLEM. The main advantage of the F LOYD -WARSHALL A LGORITHM is its simplicity. We assume w.l.o.g. that the vertices are numbered 1, . . . , n.

158

7 Shortest Paths

F LOYD -WARSHALL A LGORITHM Input: Output:

A digraph G with V (G) = {1, . . . , n} and conservative weights c : E(G) → R. Matrices (li j )1≤i, j≤n and ( pi j )1≤i, j≤n where li j is the length of a shortest path from i to j, and ( pi j , j) is the final edge of such a path (if it exists). li j := c((i, j)) for all (i, j) ∈ E(G). li j := ∞ for all (i, j) ∈ (V (G) × V (G)) \ E(G) with i 6= j. lii := 0 for all i. pi j := i for all i, j ∈ V (G).

1

Set Set Set Set

2

For j := 1 to n do: For i := 1 to n do: If i 6= j then: For k := 1 to n do: If k 6= j then: If lik > li j + l jk then set lik := li j + l jk and pik := p jk .

Theorem 7.12. (Floyd [1962], Warshall [1962]) The F LOYD -WARSHALL A L GORITHM works correctly. Its running time is O(n 3 ). Proof: The running time is obvious. Claim: After the algorithm has run through the outer loop for j = 1, 2, . . . , j0 , the variable lik contains the length of a shortest i-k-path with intermediate vertices v ∈ {1, . . . , j0 } only (for all i and k), and ( pik , k) is the final edge of such a path. This statement will be shown by induction for j0 = 0, . . . , n. For j0 = 0 it is true by , 1 and for j0 = n it implies the correctness of the algorithm. Suppose the claim holds for some j0 ∈ {0, . . . , n − 1}. We have to show that it still holds for j0 + 1. For any i and k, during processing the outer loop for j = j0 + 1, lik (containing by the induction hypothesis the length of a shortest i-kpath with intermediate vertices v ∈ {1, . . . , j0 } only) is replaced by li, j0 +1 +l j0 +1,k if this value is smaller. It remains to show that the corresponding i-( j0 + 1)-path P and the ( j0 + 1)-k-path Q have no inner vertex in common. Suppose that there is an inner vertex belonging to both P and Q. By shortcutting the maximal closed walk in P + Q (which by our assumption has nonnegative weight because it is the union of circuits) we get an i-k-path R with intermediate vertices v ∈ {1, . . . , j0 } only. R is no longer than li, j0 +1 + l j0 +1,k (and in particular shorter than the lik before processing the outer loop for j = j0 + 1). This contradicts the induction hypothesis since R has intermediate vertices v ∈ {1, . . . , j0 } only. Like the M OORE -B ELLMAN -F ORD A LGORITHM, the F LOYD -WARSHALL A LGORITHM can also be used to detect the existence of negative circuits (Exercise 11). The A LL PAIRS S HORTEST PATHS P ROBLEM in undirected graphs with arbitrary conservative weights is more difficult; see Theorem 12.13.

7.3 Minimum Mean Cycles

159

7.3 Minimum Mean Cycles We can easily find a circuit of minimum total weight in a digraph with conservative weights, using the above shortest path algorithms (see Exercise 12). Another problem asks for a circuit whose mean weight is minimum:

M INIMUM M EAN C YCLE P ROBLEM Instance:

A digraph G, weights c : E(G) → R.

Task:

Find a circuit C whose mean weight that G is acyclic.

c(E(C)) |E(C)|

is minimum, or decide

In this section we show how to solve this problem with dynamic programming, quite similar to the shortest path algorithms. We may assume that G is strongly connected, since otherwise we can identify the strongly connected components in linear time (Theorem 2.19) and solve the problem for each strongly connected component separately. But for the following min-max theorem it suffices to assume that there is a vertex s from which all vertices are reachable. We consider not only paths, but arbitrary edge progressions (where vertices and edges may be repeated). Theorem 7.13. (Karp [1978]) Let G be a digraph with weights c : E(G) → R. Let s ∈ V (G) such that each vertex is reachable from s. For x ∈ V (G) and k ∈ Z+ let ( k ) X Fk (x) := min c((vi−1 , vi )) : v0 = s, vk = x, (vi−1 , vi ) ∈ E(G) for all i i=1

be the minimum weight of an edge progression of length k from s to x (and ∞ if there is none). Let µ(G, c) be the minimum mean weight of a circuit in G (and µ(G, c) = ∞ if G is acyclic). Then µ(G, c) =

min

max

x∈V (G) 0≤k≤n−1 Fk (x) 0, and therefore there must exist a vertex w ∈ R with ex f (w) < 0. Since f is an s-t-preflow, this vertex must be s. (b): Suppose there is a v-w-path in G f , say with vertices v = v0 , v1 , . . . , vk = w. Since there is a distance labeling ψ with respect to f , ψ(vi ) ≤ ψ(vi+1 ) + 1 for i = 0, . . . , k − 1. So ψ(v) ≤ ψ(w) + k. Note that k ≤ n − 1. (c): follows from (b) as ψ(s) = n and ψ(t) = 0. Part (c) helps us to prove the following: Theorem 8.23. When the algorithm terminates, f is a maximum s-t-flow. Proof: f is an s-t-flow because there are no active vertices. Lemma 8.22(c) implies that there is no augmenting path. Then by Theorem 8.5 we know that f is maximum. The question now is how many P USH and R ELABEL operations are performed. Lemma 8.24. (a) For each v ∈ V (G), ψ(v) is strictly increased by every R ELABEL(v), and is never decreased. (b) At any stage of the algorithm, ψ(v) ≤ 2n − 1 for all v ∈ V (G). (c) No vertex is relabelled more than 2n − 1 times. The total number of R ELABEL operations is at most 2n 2 − n.

8.5 The Goldberg-Tarjan Algorithm

179

+ Proof: (a): ψ is changed only in the R ELABEL procedure. If no e ∈ δG (v) f is admissible, then R ELABEL(v) strictly increases ψ(v) (because ψ is a distance labeling at any time). (b): We only change ψ(v) if v is active. By Lemma 8.22(a) and (b), ψ(v) ≤ ψ(s) + n − 1 = 2n − 1. (c): follows directly from (a) and (b).

We shall now analyse the number of P USH operations. We distinguish between saturating pushes (where u f (e) = 0 after the push) and nonsaturating pushes. Lemma 8.25. The number of saturating pushes is at most 2mn. Proof: After each saturating push from v to w, another such push cannot occur until ψ(w) increases by at least 2, a push from w to v occurs, and ψ(v) increases by at least 2. Together with Lemma 8.24(a) and (b), this proves that there are at ↔

most n saturating pushes on each edge (v, w) ∈ E(G).

The number of nonsaturating pushes can be in the order of n 2 m in general (Exercise 19). By choosing an active vertex v with ψ(v) maximum in 3 we can prove a better bound. As usual we denote n := |V (G)|, m := |E(G)| and may assume n ≤ m ≤ n 2 . Lemma 8.26. If we always choose v to be an active vertex with ψ(v) maximum in 3 of the P USH -R ELABEL A LGORITHM, the number of nonsaturating pushes √ is at most 8n 2 m. Proof: Call a phase the time between two consecutive changes of ψ ∗ := max{ψ(v) : v active}. As ψ ∗ can increase only by relabeling, its total increase is at most 2n 2 . As ψ ∗ = 0 initially, it can decrease at most 2n 2 times, and the number of phases is at most 4n 2 . √ m nonsaturating pushes and expensive Call a phase cheap if it contains at most √ otherwise. Clearly there are at most 4n 2 m nonsaturating pushes in cheap phases. Let X |{w ∈ V (G) : ψ(w) ≤ ψ(v)}|. 8 := v∈V (G):v active Initially 8 ≤ n 2 . A relabeling step may increase 8 by at most n. A saturating push may increase 8 by at most n. A nonsaturating push does not increase 8. Since 8 = 0 at termination, the total decrease of 8 is at most n 2 +n(2n 2 −n)+n(2mn) ≤ 4mn 2 . Now consider the nonsaturating pushes in an expensive phase. Each of them pushes flow along an edge (v, w) with ψ(v) = ψ ∗ = ψ(w) + 1, deactivating v and possibly activating w. As the phase ends by relabeling or by deactivating the last active vertex v with w with ψ(w) = ψ ∗ remains constant during the ψ(v) = ψ ∗ , the set of vertices √ phase, and it contains more than m vertices as the phase is expensive. Hence each

180

8 Network Flows

√ nonsaturating push in an expensive phase decreases 8 by at least m. Thus the total √ 2 √ number of nonsaturating pushes in expensive phases is at most 4mn = 4n 2 m. m This proof is due to Cheriyan and Mehlhorn [1999]. We finally get: Theorem 8.27. (Goldberg and Tarjan [1988], Cheriyan and Maheshwari [1989], Tunc¸el [1994]) The P USH -R ELABEL A LGORITHM solves the√M AXIMUM F LOW P ROBLEM correctly and can be implemented to run in O(n 2 m) time. Proof: The correctness follows from Proposition 8.21 and Theorem 8.23. As in Lemma 8.26 we always choose v in 3 to be an active vertex with ψ(v) maximum. To make this easy we keep track of doubly-linked lists L 0 , . . . , L 2n−1 , where L i contains the active vertices v with ψ(v) = i. These lists can be updated during each P USH and R ELABEL operation in constant time. We can then start by scanning L i for i = 0. When a vertex is relabelled, we increase i accordingly. When we find a list L i for the current i empty (after deactivating the last active vertex at that level), we decrease i until L i is nonempty. As we increase i at most 2n 2 times by Lemma 8.24(c), we also decrease i at most 2n 2 times. As a second data structure, we store a doubly-linked list Av containing the admissible edges leaving v for each vertex v. They can also be updated in each P USH operation in constant time, and in each R ELABEL operation in time proportional to the total number of edges incident to the relabelled vertex. So R ELABEL(v) takes a total of O(|δG (v)|) time, and by Lemma 8.24(c) the overall time for relabelling is O(mn). Each P USH takes constant√time, and by Lemma 8.25 and Lemma 8.26 the total number of pushes is O(n 2 m).

8.6 Gomory-Hu Trees Any algorithm for the M AXIMUM F LOW P ROBLEM also implies a solution to the following problem:

M INIMUM C APACITY C UT P ROBLEM Instance:

A network (G, u, s, t).

Task:

An s-t-cut in G with minimum capacity.

Proposition 8.28. The M INIMUM C APACITY C UT P ROBLEM can be solved in √the same running time as the M AXIMUM F LOW P ROBLEM, in particular in O(n 2 m) time. Proof: For a network (G, u, s, t) we compute a maximum s-t-flow f and define X to be the set of all vertices reachable from s in G f . X can be computed with the G RAPH S CANNING A LGORITHM in linear time (Proposition 2.17). By Lemma 8.3

8.6 Gomory-Hu Trees

181

√ + and Theorem 8.5, δG (X ) constitutes a minimum capacity s-t-cut. The O(n 2 m) running time follows from Theorem 8.27 (and is not best possible). In this section we consider the problem of finding a minimum capacity s-t-cut for each pair of vertices s, t in an undirected graph G with capacities u : E(G) → R+ . This problem can be reduced to the above one: For all pairs s, t ∈ V (G) we solve the M INIMUM C APACITY C UT P ROBLEM for (G 0 , u 0 , s, t), where (G 0 , u 0 ) arises from (G, u) by replacing each undirected edge {v, w} by two oppositely directed edges (v, w) and (w, v) with u 0 ((v, w)) = u 0((w, v)) = u({v, w}). In this way we obtain minimum s-t-cuts for all s, t after n2 flow computations. This section is devoted to the elegant method of Gomory and Hu [1961], which requires only n − 1 flow computations. We shall see some applications in Sections 12.3 and 20.3. Definition 8.29. Let G be an undirected graph and u : E(G) → R+ a capacity function. For two vertices s, t ∈ V (G) we denote by λst their local edgeconnectivity, i.e. the minimum capacity of a cut separating s and t. The edge-connectivity of a graph is obviously the minimum local edgeconnectivity with respect to unit capacities. Lemma 8.30. For all vertices i, j, k ∈ V (G) we have λik ≥ min(λi j , λ jk ). Proof: Let δ(A) be a cut with i ∈ A, k ∈ V (G) \ A and u(δ(A)) = λik . If j ∈ A then δ(A) separates j and k, so u(δ(A)) ≥ λ jk . If j ∈ V (G)\ A then δ(A) separates i and j, so u(δ(A)) ≥ λi j . We conclude that λik = u(δ(A)) ≥ min(λi j , λ jk ). Indeed, this condition is not only necessary but also sufficient for numbers (λi j )1≤i, j≤n with λi j = λ ji to be local edge-connectivities of some graph (Exercise 23). Definition 8.31. Let G be an undirected graph and u : E(G) → R+ a capacity function. A tree T is called a Gomory-Hu tree for (G, u) if V (T ) = V (G) and λst =

min u(δG (Ce ))

e∈E(Pst )

for all s, t ∈ V (G),

where Pst is the (unique) s-t-path in T and, for e ∈ E(T ), Ce and V (G) \ Ce are the connected components of T − e (i.e. δG (Ce ) is the fundamental cut of e with respect to T ). We shall see that every graph possesses a Gomory-Hu tree. This implies that for any undirected graph G there is a list of n − 1 cuts such that for each pair s, t ∈ V (G) a minimum s-t-cut belongs to the list. In general, a Gomory-Hu tree cannot be chosen as a subgraph of G. For example, consider G = K 3,3 and u ≡ 1. Here λst = 3 for all s, t ∈ V (G). It is easy to see that the Gomory-Hu trees for (G, u) are exactly the stars with five edges. The main idea of the algorithm for constructing a Gomory-Hu tree is as follows. First we choose any s, t ∈ V (G) and find some minimum s-t-cut, say δ(A). Let

182

8 Network Flows

B := V (G)\ A. Then we contract A (or B) to a single vertex, choose any s , t ∈ B (or s , t ∈ A, respectively) and look for a minimum s -t -cut in the contracted graph G . We continue this process, always choosing a pair s , t of vertices not separated by any cut obtained so far. At each step, we contract – for each cut E(A , B ) obtained so far – A or B , depending on which part does not contain s and t . Eventually each pair of vertices is separated. We have obtained a total of n − 1 cuts. The crucial observation is that a minimum s -t -cut in the contracted graph G is also a minimum s -t -cut in G. This is the subject of the following lemma. Note that when contracting a set A of vertices in (G, u), the capacity of each edge in G is the capacity of the corresponding edge in G. Lemma 8.32. Let G be an undirected graph and u : E(G) → R+ a capacity function. Let s, t ∈ V (G), and let δ(A) be a minimum s-t-cut in (G, u). Let now s , t ∈ V (G) \ A, and let (G , u ) arise from (G, u) by contracting A to a single vertex. Then for any minimum s -t -cut δ(K ∪ {A}) in (G , u ), δ(K ∪ A) is a minimum s -t -cut in (G, u). Proof: Let s, t, A, s , t , G , u be as above. W.l.o.g. s ∈ A. It sufﬁces to prove that there is a minimum s -t -cut δ(A ) in (G, u) such that A ⊂ A . So let δ(C) be any minimum s -t -cut in (G, u). W.l.o.g. s ∈ C. A V (G) \ A

t V (G) \ C C s

s

Fig. 8.3.

Since u(δ(·)) is submodular (cf. Lemma 2.1(c)), we have u(δ(A)) + u(δ(C)) ≥ u(δ(A ∩ C)) + u(δ(A ∪ C)). But δ(A ∩ C) is an s-t-cut, so u(δ(A ∩ C)) ≥ λst = u(δ(A)). Therefore u(δ(A ∪ C)) ≤ u(δ(C)) = λs t proving that δ(A ∪ C) is a minimum s -t -cut. (See Figure 8.3.) Now we describe the algorithm which constructs a Gomory-Hu tree. Note that the vertices of the intermediate trees T will be vertex sets of the original graph;

8.6 Gomory-Hu Trees

183

indeed they form a partition of V (G). At the beginning, the only vertex of T is V (G). In each iteration, a vertex of T containing at least two vertices of G is chosen and split into two.

G OMORY -H U A LGORITHM Input:

An undirected graph G and a capacity function u : E(G) → R+ .

Output:

A Gomory-Hu tree T for (G, u).

1

Set V (T ) := {V (G)} and E(T ) := ∅.

2

Choose some X ∈ V (T ) with |X | ≥ 2. If no such X exists then go to . 6

3

Choose s, t ∈ X with s 6= t. S For each connected component C of T − X do: Let SC := Y ∈V (C) Y . 0 0 Let (G , u ) arise from (G, u) by contracting SC to a single vertex vC for each connected component C of T − X . (So V (G 0 ) = X ∪ {vC : C is a connected component of T − X }.)

4

0 0 0 B 0 := V (G 0 )\ A0 . Find a minimum s-t-cut δ(A ) in (G , u ). Let [ [ SC ∪ (A0 ∩ X ) and B := SC ∪ (B 0 ∩ X ). Set A := vC ∈A0 \X

5

6

vC ∈B 0 \X

Set V (T ) := (V (T ) \ {X }) ∪ {A ∩ X, B ∩ X }. For each edge e = {X, Y } ∈ E(T ) incident to the vertex X do: If Y ⊆ A then set e0 := {A ∩ X, Y } else set e0 := {B ∩ X, Y }. Set E(T ) := (E(T ) \ {e}) ∪ {e0 } and w(e0 ) := w(e). Set E(T ) := E(T ) ∪ {{A ∩ X, B ∩ X }}. Set w({A ∩ X, B ∩ X }) := u 0 (δG 0 (A0 )). Go to . 2 Replace all {x} ∈ V (T ) by x and all {{x}, {y}} ∈ E(T ) by {x, y}. Stop.

Figure 8.4 illustrates the modification of T in . 5 To prove the correctness of this algorithm, we first show the following lemma: Lemma 8.33. Each time at the end of 4 we have .

(a) A ∪ B = V (G) (b) E(A, B) is a minimum s-t-cut in (G, u). Proof: The elements of V (T ) are always nonempty subsets of V (G), indeed V (T ) constitutes a partition of V (G). From this, (a) follows easily. We now prove (b). The claim is trivial for the first iteration (since here G 0 = G). We show that the property is preserved in each iteration. Let C1 , . . . , Ck be the connected components of T − X . Let us contract them one by one; for i = 0, . . . , k let (G i , u i ) arise from (G, u) by contracting each of SC1 , . . . , SCi to a single vertex. So (G k , u k ) is the graph which is denoted by (G 0 , u 0 ) in 3 of the algorithm.

184

8 Network Flows

(a)

X

(b)

A∩X

B∩X

Fig. 8.4.

Claim: For any minimum s-t-cut δ(Ai ) in (G i , u i ), δ(Ai−1 ) is a minimum s-t-cut in (G i−1 , u i−1 ), where if v Ci ∈ Ai (Ai \ {v Ci }) ∪ SCi Ai−1 := . Ai if v Ci ∈ / Ai Applying this claim successively for k, k − 1, . . . , 1 implies (b). To prove the claim, let δ(Ai ) be a minimum s-t-cut in (G i , u i ). By our assumption that (b) is true for the previous iterations, δ(SCi ) is a minimum si -ti -cut in (G, u) for some appropriate si , ti ∈ V (G). Furthermore, s, t ∈ V (G) \ SCi . So applying Lemma 8.32 completes the proof. Lemma 8.34. At any stage of the algorithm (until 6 is reached) for all e ∈ E(T ) ⎛ ⎛ ⎞⎞ w(e) = u ⎝δG ⎝ Z ⎠⎠ , Z ∈Ce

8.6 Gomory-Hu Trees

185

where Ce and V (T ) \ Ce are the connected components of T − e. Moreover for all e = {P, Q} ∈ E(T ) there are vertices p ∈ P and q ∈ Q with λ pq = w(e). Proof: Both statements are trivial at the beginning of the algorithm when T contains no edges; we show that they are never violated. So let X be the vertex of T chosen in 2 in some iteration of the algorithm. Let s, t, A , B , A, B be as determined in 3 and 4 next. W.l.o.g. assume s ∈ A . Edges of T not incident to X are not affected by . 5 For the new edge {A ∩ X, B ∩ X }, w(e) is clearly set correctly, and we have λst = w(e), s ∈ A ∩ X , t ∈ B ∩ X. So let us consider an edge e = {X, Y } that is replaced by e in . 5 We assume w.l.o.g. Y ⊆ A, so e = {A ∩ X, Y }. Assuming that the assertions were true for e we claim that they remain true for e . This is trivial for the ﬁrst assertion, because does not change. w(e) = w(e ) and u δG Z ∈Ce Z To show the second statement, we assume that there are p ∈ X, q ∈ Y with λ pq = w(e). If p ∈ A ∩ X then we are done. So henceforth assume that p ∈ B ∩ X (see Figure 8.5).

q Y

s

t

p B∩X

A∩X Fig. 8.5.

We claim that λsq = λ pq . Since λ pq = w(e) = w(e ) and s ∈ A ∩ X , this will conclude the proof. By Lemma 8.30, λsq ≥ min{λst , λt p , λ pq }. Since by Lemma 8.33(b) E(A, B) is a minimum s-t-cut, and since s, q ∈ A, we may conclude from Lemma 8.32 that λsq does not change if we contract B. Since t, p ∈ B, this means that adding an edge {t, p} with arbitrary high capacity does not change λsq . Hence λsq ≥ min{λst , λ pq }. Now observe that λst ≥ λ pq because the minimum s-t-cut E(A, B) also separates p and q. So we have

186

8 Network Flows

λsq ≥ λ pq . To prove equality, observe that w(e) is the capacity of a cut separating X and Y , and thus s and q. Hence λsq ≤ w(e) = λ pq .

This completes the proof.

Theorem 8.35. (Gomory and Hu [1961]) The G OMORY -H U A LGORITHM works correctly. Every√undirected graph possesses a Gomory-Hu tree, and such a tree is found in O(n 3 m) time. Proof: The complexity of the algorithm is clearly determined by n − 1 times the complexity of finding a minimum s-t-cut, since everything√else can be implemented in O(n 3 ) time. By Proposition 8.28 we obtain the O(n 3 m) bound. We prove that the output T of the algorithm is a Gomory-Hu tree for (G, u). It should be clear that T is a tree with V (T ) = V (G). Now let s, t ∈ V (G). Let Pst be the (unique) s-t-path in T and, for e ∈ E(T ), let Ce and V (G) \ Ce be the connected components of T − e. Since δ(Ce ) is an s-t-cut for each e ∈ E(Pst ), λst ≤

min u(δ(Ce )).

e∈E(Pst )

On the other hand, a repeated application of Lemma 8.30 yields λst ≥

min

{v,w}∈E(Pst )

λvw .

Hence applying Lemma 8.34 to the situation before execution of 6 (where each vertex X of T is a singleton) yields λst ≥

min u(δ(Ce )),

e∈E(Pst )

so equality holds.

A similar algorithm for the same task (which might be easier to implement) was suggested by Gusfield [1990].

8.7 The Minimum Capacity of a Cut in an Undirected Graph If we are only interested in a minimum capacity cut in an undirected graph G with capacities u : E(G) → R+ , there is a simpler method using n − 1 flow computations: just compute a minimum s-t-cut for some fixed vertex s and each t ∈ V (G) \ {s}. However, there are more efficient algorithms.

8.7 The Minimum Capacity of a Cut in an Undirected Graph

187

2

Hao and Orlin [1994] found an O(nm log nm )-algorithm for determining a minimum capacity cut. They use a modified version of the P USH -R ELABEL A L GORITHM . If we just want to compute the edge-connectivity of the graph (i.e. unit capacities), the currently fastest algorithm is due to Gabow [1995] with running time n ), where λ(G) is the edge-connectivity (observe that 2m ≥ λn). O(m +λ2 n log λ(G) Gabow’s algorithm uses matroid intersection techniques. We remark that the M AX IMUM F LOW P ROBLEM in undirected graphs with unit capacities can also be solved faster than in general (Karger and Levine [1998]). Nagamochi and Ibaraki [1992] found a completely different algorithm to determine a minimum capacity cut in an undirected graph. Their algorithm does not use max-flow computations at all. In this section we present this algorithm in a simplified form due to Stoer and Wagner [1997] and independently to Frank [1994]. We start with an easy definition. Definition 8.36. Given a graph G with capacities u : E(G) → R+ , we call an order v1 , . . . , vn of the vertices an MA (maximum adjacency) order if for all i ∈ {2, . . . , n}: X X u(e) = max u(e). e∈E({v1 ,...,vi−1 },{vi })

j∈{i,...,n}

e∈E({v1 ,...,vi−1 },{v j })

Proposition 8.37. Given a graph G with capacities u : E(G) → R+ , an MA order can be found in O(m + n log n) time. Proof: Consider the following algorithm. First set α(v) := 0 for all v ∈ V (G). Then for i := 1 to n do the following: choose vi from among V (G)\{v1 , . . . , vi−1 } such that P it has maximum α-value (breaking ties arbitrarily), and set α(v) := α(v) + e∈E({vi },{v}) u(e) for all v ∈ V (G) \ {v1 , . . . , vi }. The correctness of this algorithm is obvious. By implementing it with a Fibonacci heap, storing each vertex v with key −α(v) until it is selected, we get a running time of O(m + n log n) by Theorem 6.6 as there are n INSERT-, n DELETEMIN - and (at most) m DECREASEKEY -operations. Lemma 8.38. (Stoer and Wagner [1997], Frank [1994]) Let G be a graph with n := |V (G)| ≥ 2, capacities u : E(G) → R+ and an MA order v1 , . . . , vn . Then X λvn−1 vn = u(e). e∈E({vn },{v1 ,...,vn−1 })

Proof: Of course we only have to show “≥”. We shall use induction on |V (G)|+ |E(G)|. For |V (G)| < 3 the statement is trivial. We may assume that there is no edge e = {vn−1 , vn } ∈ E(G), because otherwise we would delete it (both left-hand side and right-hand side decrease by u(e)) and apply the induction hypothesis. Denote the right-hand side by R. Of course v1 , . . . , vn−1 is an MA order in G − vn . So by induction,

188

8 Network Flows n λvG−v = n−2 vn−1

X

X

u(e) ≥

e∈E({vn−1 },{v1 ,...,vn−2 })

u(e) = R.

e∈E({vn },{v1 ,...,vn−2 })

Here the inequality holds because v1 , . . . , vn was an MA order for G. The last n / E(G). So λvGn−2 vn−1 ≥ λvG−v equality is true because {vn−1 , vn } ∈ n−2 vn−1 ≥ R. On the other hand v1 , . . . , vn−2 , vn is an MA order in G −vn−1 . So by induction, X G−v u(e) = R, = λvn−2 vn−1 n e∈E({vn },{v1 ,...,vn−2 }) G−v

/ E(G). So λvGn−2 vn ≥ λvn−2 vn−1 again because {vn−1 , vn } ∈ n = R. Now by Lemma 8.30 λvn−1 vn ≥ min{λvn−1 vn−2 , λvn−2 vn } ≥ R. P Note that the existence of two vertices x, y with λx y = e∈δ(x) u(e) was already shown by Mader [1972], and follows easily from the existence of a Gomory-Hu tree (Exercise 25). Theorem 8.39. (Nagamochi and Ibaraki [1992], Stoer and Wagner [1997]) A minimum capacity cut in an undirected graph with nonnegative capacities can be found in O(mn + n 2 log n) time. Proof: We may assume that the given graph G is simple since we can unite parallel edges. Denote by λ(G) the minimum capacity of a cut in G. The algorithm proceeds as follows: Let G 0 := G. In the i-th step (i = 1, . . . , n−1) choose vertices x, y ∈ V (G i−1 ) with X G u(e). λx yi−1 = e∈δG i−1 (x)

By Proposition 8.37 and Lemma 8.38 this can be done in O(m + n log n) time. Set G γi := λx yi−1 , z i := x, and let G i result from G i−1 by contracting {x, y}. Observe that (8.1) λ(G i−1 ) = min{λ(G i ), γi }, because a minimum cut in G i−1 either separates x and y (in this case its capacity is γi ) or does not (in this case contracting {x, y} does not change anything). After arriving at G n−1 which has only one vertex, we choose an k ∈ {1, . . . , n − 1} for which γk is minimum. We claim that δ(X ) is a minimum capacity cut in G, where X is the vertex set in G whose contraction resulted in the vertex z k of G k−1 . But this is easy to see, since by (8.1) λ(G) = min{γ1 , . . . , γn−1 } = γk and γk is the capacity of the cut δ(X ). A randomized contraction algorithm for finding a minimum cut (with high probability) is discussed in Exercise 29. Moreover, we mention that the vertexconnectivity of a graph can be computed by O(n 2 ) flow computations (Exercise 30).

Exercises

189

In this section we have shown how to minimize f (X ) := u(δ(X )) over ∅ = X ⊂ V (G). Note that this f : 2V (G) → R+ is submodular and symmetric (i.e. f (A) = f (V (G)\ A) for all A). The algorithm presented here has been generalized by Queyranne [1998] to minimize general symmetric submodular functions; see Section 14.5. The problem of ﬁnding a maximum cut is much harder and will be discussed in Section 16.2.

Exercises 1. Let (G, u, s, t) be a network, and let δ + (X ) and δ + (Y ) be minimum s-t-cuts in (G, u). Show that δ + (X ∩ Y ) and δ + (X ∪ Y ) are also minimum s-t-cuts in (G, u). 2. Show that in case of irrational capacities, the F ORD -F ULKERSON A LGORITHM may not terminate at all. Hint: Consider the following network (Figure 8.6): x1

y1

x2

y2

s

t x3

y3

x4

y4 Fig. 8.6.

All lines represent edges in both directions. All edges have capacity S = except

1 1−σ

u((x1 , y1 )) = 1, u((x2 , y2 )) = σ, u((x3 , y3 )) = u((x4 , y4 )) = σ 2 √

∗

n n+1 + σ n+2 . where σ = 5−1 2 . Note that σ = σ (Ford and Fulkerson [1962]) 3. Let G be a digraph and M the incidence matrix of G. Prove that for all c, l, u ∈ Z E(G) with l ≤ u: max cx : x ∈ Z E(G) , l ≤ x ≤ u, M x = 0 =

E(G) min y u − y l : y , y ∈ Z+ , z M + y − y = c for some z ∈ ZV (G) . Show how this implies Theorem 8.6 and Corollary 8.7.

190

8 Network Flows

4. Prove Hoffman’s circulation theorem: Given a digraph G and lower and upper capacities l, u : E(G) → R+ with l(e) ≤ u(e) for all e ∈ E(G), there is circulation f with l(e) ≤ f (e) ≤ u(e) for all e ∈ E(G) if and only if X X l(e) ≤ u(e) for all X ⊆ V (G). e∈δ − (X )

∗

e∈δ + (X )

Note: Hoffman’s circulation theorem in turn quite easily implies the MaxFlow-Min-Cut Theorem. (Hoffman [1960]) 5. Consider a network (G, u, s, t), a maximum s-t-flow f and the residual graph G f . Form a digraph H from G f by contracting the set S of vertices reachable from s to a vertex v S , contracting the set T of vertices from which t is reachable to a vertex vT , and contracting each strongly connected component X of G f − (S ∪ T ) to a vertex v X . Observe that H is acyclic. Prove that there + is a one-to-one correspondence between the sets X ⊆ V (G) for which δG (X ) + is a minimum s-t-cut in (G, u) and the sets Y ⊆ V (H ) for which δ H (Y ) is a directed vT -v S -cut in H (i.e. a directed cut in H separating vT and v S ). Note: This statement also holds for G f without any contraction instead of H . However, we shall use the statement in the above form in Section 20.4. (Picard and Queyranne [1980]) 6. Let G be a digraph and c P : E(G) → R. We Plook for a set X ⊂ V (G) with s ∈ X and t ∈ / X such that e∈δ + (X ) c(e) − e∈δ − (X ) c(e) is minimum. Show how to reduce this problem to the M INIMUM C APACITY C UT P ROBLEM. Hint: Construct a network where all edges are incident to s or t. 7. Let G be an acyclic digraph with mappings σ, τ, c : E(G) → R+ , and a number x : E(G) → R+ such that σ (e) ≤ x(e) ≤ τ (e) C ∈ R+ . We look for a mapping P for all e ∈ E(G) and e∈E(G) (τ (e) − x(e))c(e) ≤ C. Among the feasible solutions we want to minimize the length (with respect to x) of the longest path in G. The meaning behind the above is the following. The edges correspond to jobs, σ (e) and τ (e) stand for the minimum and maximum completion time of job e, and c(e) is the cost of reducing the completion time of job e by one unit. If there are two jobs e = (i, j) and e0 = ( j, k), job e has to be finished before job e0 can be processed. We have a fixed budget C and want to minimize the total completion time. Show how to solve this problem using network flow techniques. (This application is known as PERT, program evaluation and review technique, or CPM, critical path method.) Hint: Introduce one source s and one sink t. Start with x = τ and successively reduce the length of the longest s-t-path (with respect to x) at the minimum possible cost. Use Exercise 7 of Chapter 7, Exercise 8 of Chapter 3, and Exercise 6. (Phillips and Dessouky [1977])

Exercises

191

8. Let (G, c, s, t) be a network such that G is planar even when an edge e = (s, t) is added. Consider the following algorithm. Start with the flow f ≡ 0 and let G 0 := G f . At each step consider the boundary B of a face of G 0 +e containing e (with respect to some fixed planar embedding). Augment f along B − e. Let G 0 consist of the forward edges of G f only and iterate as long as t is reachable from s in G 0 . Prove that this algorithm computes a maximum s-t-flow. Use Theorem 2.40 to show that it can be implemented to run in O(n 2 ) time. (Ford and Fulkerson [1956], Hu [1969]) Note: This problem can be solved in O(n) time. For general planar networks an O(n log n)-algorithm exists; see Weihe [1997] and Borradaile and Klein [2006]. 9. Show that the directed edge-disjoint version of Menger’s Theorem 8.9 also follows directly from Theorem 6.17. 10. Let G be a graph (directed or undirected), x, y, z three vertices, and α, β ∈ N with α ≤ λx y , β ≤ λx z and α + β ≤ max{λx y , λx z }. Prove that there are α x-y-paths and β x-z-paths such that these α+β paths are pairwise edge-disjoint. 11. Let G be a digraph that contains k edge-disjoint s-t-paths for any two vertices s and t (such a graph is called strongly k-edge-connected). Let H be any digraph with V (H ) = V (G) and |E(H )| = k. Prove that the instance (G, H ) of the D IRECTED E DGE -D ISJOINT PATHS P ROBLEM has a solution. (Mader [1981] and Shiloach [1979]) 12. Let G be a digraph with at least k edges. Prove: G contains k edge-disjoint s-t-paths for any two vertices s and t if and only if for any k distinct edges e1 = (x1 , y1 ), . . . , ek = (xk , yk ), G − {e1 , . . . , ek } contains k edge-disjoint spanning arborescences T1 , . . . , Tk such that Ti is rooted at yi (i = 1, . . . , k). Note: This generalizes Exercise 11. Hint: Use Theorem 6.17. (Su [1997]) 13. Let G be a digraph with capacities c : E(G) → R+ and r ∈ V (G). Can one determine an r -cut with minimum capacity in polynomial time? Can one determine a directed cut with minimum capacity in polynomial time (or decide that G is strongly connected)? Note: The answer to the first question solves the S EPARATION P ROBLEM for the M INIMUM W EIGHT ROOTED A RBORESCENCE P ROBLEM; see Corollary 6.14. 14. Show how to find a blocking flow in an acyclic network in O(nm) time. (Dinic [1970]) 15. Let (G, u, s, t) be a network such that G − t is an arborescence. Show how to find a maximum s-t-flow in linear time. Hint: Use DFS. ∗ 16. Let (G, u, s, t) be a network such that the underlying undirected graph of G − {s, t} is a forest. Show how to find a maximum s-t-flow in linear time. (Vygen [2002])

∗

192

8 Network Flows

17. Consider a modified version of F UJISHIGE ’ S A LGORITHM where in 5 we choose vi ∈ V (G) \ {v1 , . . . , vi−1 } such that b(vi ) is maximum, step 4 is replaced by stopping if b(v) = 0 for all v ∈ V (G) \ {v1 , . . . , vi }, and in the beginning of 6 we set β(t) := minij=2 b( j). Then X and α are not needed anymore. (a) Show that this variant of the algorithm works correctly. (b) Let αk be the number minij=2 b( j) in iteration k (or zero if the algorithm

18.

19. 20.

21.

22.

23.

24.

k+2n stops before iteration k). Show that minl=k+1 αl ≤ 12 αk for all k. Conclude that the number of iterations is O(n log u max ). (c) Show how to implement one iteration in O(m + n log n) time. Let us call a preflow f maximum if ex f (t) is maximum. (a) Show that for any maximum preflow f there exists a maximum flow f 0 with f 0 (e) ≤ f (e) for all e ∈ E(G). (b) Show how a maximum preflow can be converted into a maximum flow in O(nm) time. (Hint: Use a variant of the E DMONDS -K ARP A LGORITHM.) Prove that the P USH -R ELABEL A LGORITHM performs O(n 2 m) nonsaturating pushes, independent of the choice of v in . 3 Given an acyclic digraph G with weights c : E(G) → R+ , find a maximum weight directed cut in G. Show how this problem can be reduced to a minimum s-t-cut problem and be solved in O(n 3 ) time. Hint: Use Exercise 6. Let G be an acyclic digraph with weights c : E(G) → R+ . We look for the maximum weight edge set F ⊆ E(G) such that no path in G contains more than one edge of F. Show that this problem is equivalent to looking for the maximum weight directed cut in G (and thus can be solved in O(n 3 ) time by Exercise 20). Given an undirected graph G with capacities u : E(G) → R+ and a set T ⊆ V (G) with |T | ≥ 2.PWe look for a set X ⊂ V (G) with T ∩ X 6= ∅ and T \ X 6= ∅ such that e∈δ(X ) u(e) is minimum. Show how to solve this problem in O(n 4 ) time, where n = |V (G)|. Let λi j , 1 ≤ i, j ≤ n, be nonnegative numbers with λi j = λ ji and λik ≥ min(λi j , λ jk ) for any three distinct indices i, j, k ∈ {1, . . . , n}. Show that there exists a graph G with V (G) = {1, . . . , n} and capacities u : E(G) → R+ such that the local edge-connectivities are precisely the λi j . Hint: Consider a maximum weight spanning tree in (K n , c), where c({i, j}) := λi j . (Gomory and Hu [1961]) Let G be an undirected graph with capacities u : E(G) → R+ , and let T ⊆ V (G) with |T | even. A T -cut in G is a cut δ(X ) with |X ∩ T | odd. Construct a polynomial time algorithm for finding a T -cut of minimum capacity in (G, u). Hint: Use a Gomory-Hu tree. (A solution of this exercise can be found in Section 12.3.)

Exercises

193

25. Let G be a simple undirected graph with at least two vertices. Suppose the degree of each vertex of G is at least k. Prove that there are two vertices s and t such that at least k edge-disjoint s-t-paths exist. What if there is exactly one vertex with degree less than k? Hint: Consider a Gomory-Hu tree for G. 26. Consider the problem of determining the edge-connectivity λ(G) of an undirected graph (with unit capacities). Section 8.7 shows how to solve this problem in O(mn) time, provided that we can find an MA order of an undirected graph with unit capacities in O(m + n) time. How can this be done? G denote ∗ 27. Let G be an undirected graph with an MA order v1 , . . . , vn . Let κuv the maximum number of vertex-disjoint u-v-paths in G. Prove κvGn−1 vn = |E({vn }, {v1 , . . . , vn−1 })| (the vertex-disjoint counterpart of Lemma 8.38). G Hint: Prove by induction that κv j vi ji = |E({v j }, {v1 , . . . , vi })|, where G i j = / E(G), choose G[{v1 , . . . , vi }∪{v j }]. To do this, assume w.l.o.g. that {v j , vi } ∈ a minimal set Z ⊆ {v1 , . . . , vi−1 } separating v j and vi (Menger’s Theorem 8.10), and let h ≤ i be the maximum number such that vh ∈ / Z and vh is adjacent to vi or v j . (Frank [unpublished]) ∗ 28. An undirected graph is called chordal if it has no circuit of length at least four as an induced subgraph. An order v1 , . . . , vn of an undirected graph G is called simplicial if {vi , v j }, {vi , vk } ∈ E(G) implies {v j , vk } ∈ E(G) for i < j < k. (a) Prove that a graph with a simplicial order must be chordal. (b) Let G be a chordal graph, and let v1 , . . . , vn be an MA order. Prove that vn , vn−1 , . . . , v1 is a simplicial order. Hint: Use Exercise 27 and Menger’s Theorem 8.10. Note: The fact that a graph is chordal if and only if it has a simplicial order is due to Rose [1970]. 29. Let G an undirected graph with capacities u : E(G) → R+ . Let ∅ 6= A ⊂ V (G) such that δ(A) is a minimum capacity cut in G. (a) Show that u(δ(A)) ≤ n2 u(E(G)). (Hint: Consider the trivial cuts δ(x), x ∈ V (G).) (b) Consider the following procedure: We randomly choose an edge which u(e) we contract, each edge e is chosen with probability u(E(G)) . We repeat this operation until there are only two vertices. Prove that the probability 2 . that we never contract an edge of δ(A) is at least (n−1)n (c) Conclude that running the randomized algorithm in (b) kn 2 times yields δ(A) with probability at least 1 − e−2k . (Such an algorithm with a positive probability of a correct answer is called a Monte Carlo algorithm.) (Karger and Stein [1996]; see also Karger [2000]) 30. Show how the vertex-connectivity of an undirected graph can be determined in O(n 5 ) time. Hint: Recall the proof of Menger’s Theorem.

194

8 Network Flows

Note: There exists an O(n 4 )-algorithm; see (Henzinger, Rao and Gabow [2000]). 31. Let G be a connected undirected graph with capacities u : E(G) → R+ . We are looking for a minimum capacity 3-cut, i.e. an edge set whose deletion splits G into at least three connected components. Let n := |V (G)| ≥ 4. Let δ(X 1 ), δ(X 2 ), . . . be a list of the cuts ordered by nondecreasing capacities: u(δ(X 1 )) ≤ u(δ(X 2 )) ≤ · · · . Assume that we know the first 2n − 2 elements of this list (note: they can be computed in polynomial time by a method of Vazirani and Yannakakis [1992]). (a) Show that for some indices i, j ∈ {1, . . . , 2n − 2} all sets X i \ X j , X j \ X i , X i ∩ X j and V (G) \ (X i ∪ X j ) are nonempty. (b) Show that there is a 3-cut of capacity at most 32 u(δ(X 2n−2 ). (c) For each i = 1, . . . , 2n − 2 consider δ(X i ) plus a minimum capacity cut of G − X i , and also δ(X i ) plus a minimum capacity cut of G[X i ]. This yields a list of at most 4n − 4 3-cuts. Prove that one of them is optimum. (Nagamochi and Ibaraki [2000]) Note: The problem of finding the optimum 3-cut separating three given vertices is much harder; see Dahlhaus et al. [1994] and Cheung, Cunningham and Tang [2006].

References General Literature: Ahuja, R.K., Magnanti, T.L., and Orlin, J.B. [1993]: Network Flows. Prentice-Hall, Englewood Cliffs 1993 Cook, W.J., Cunningham, W.H., Pulleyblank, W.R., and Schrijver, A. [1998]: Combinatorial Optimization. Wiley, New York 1998, Chapter 3 Cormen, T.H., Leiserson, C.E., Rivest, R.L., and Stein, C. [2001]: Introduction to Algorithms. Second Edition. MIT Press, Cambridge 2001, Chapter 26 Ford, L.R., and Fulkerson, D.R. [1962]: Flows in Networks. Princeton University Press, Princeton 1962 Frank, A. [1995]: Connectivity and network flows. In: Handbook of Combinatorics; Vol. 1 (R.L. Graham, M. Gr¨otschel, L. Lov´asz, eds.), Elsevier, Amsterdam, 1995 ´ and Tarjan, R.E. [1990]: Network flow algorithms. In: Paths, Goldberg, A.V., Tardos, E., Flows, and VLSI-Layout (B. Korte, L. Lov´asz, H.J. Pr¨omel, A. Schrijver, eds.), Springer, Berlin 1990, pp. 101–164 Gondran, M., and Minoux, M. [1984]: Graphs and Algorithms. Wiley, Chichester 1984, Chapter 5 Jungnickel, D. [2007]: Graphs, Networks and Algorithms. Third Edition. Springer, Berlin 2007 Phillips, D.T., and Garcia-Diaz, A. [1981]: Fundamentals of Network Analysis. PrenticeHall, Englewood Cliffs 1981 Ruhe, G. [1991]: Algorithmic Aspects of Flows in Networks. Kluwer Academic Publishers, Dordrecht 1991 Schrijver, A. [2003]: Combinatorial Optimization: Polyhedra and Efficiency. Springer, Berlin 2003, Chapters 9,10,13–15

References

195

Tarjan, R.E. [1983]: Data Structures and Network Algorithms. SIAM, Philadelphia 1983, Chapter 8 Thulasiraman, K., and Swamy, M.N.S. [1992]: Graphs: Theory and Algorithms. Wiley, New York 1992, Chapter 12

Cited References: Ahuja, R.K., Orlin, J.B., and Tarjan, R.E. [1989]: Improved time bounds for the maximum flow problem. SIAM Journal on Computing 18 (1989), 939–954 Borradaile, G. and Klein, P. [2006]: An O(n log n) algorithm for maximum st-flow in a directed planar graph. Proceedings of the 17th Annual ACM-SIAM Symposium on Discrete Algorithms (2006), 524–533 Cheriyan, J., and Maheshwari, S.N. [1989]: Analysis of preflow push algorithms for maximum network flow. SIAM Journal on Computing 18 (1989), 1057–1086 Cheriyan, J., and Mehlhorn, K. [1999]: An analysis of the highest-level selection rule in the preflow-push max-flow algorithm. Information Processing Letters 69 (1999), 239–242 Cherkassky, B.V. [1977]: √ Algorithm of construction of maximal flow in networks with complexity of O(V 2 E) operations. Mathematical Methods of Solution of Economical Problems 7 (1977), 112–125 [in Russian] Cheung, K.K.H., Cunningham, W.H., and Tang, L. [2006]: Optimal 3-terminal cuts and linear programming. Mathematical Programming 106 (2006), 1–23 Dahlhaus, E., Johnson, D.S., Papadimitriou, C.H., Seymour, P.D., and Yannakakis, M. [1994]: The complexity of multiterminal cuts. SIAM Journal on Computing 23 (1994), 864–894 Dantzig, G.B., and Fulkerson, D.R. [1956]: On the max-flow min-cut theorem of networks. In: Linear Inequalities and Related Systems (H.W. Kuhn, A.W. Tucker, eds.), Princeton University Press, Princeton 1956, pp. 215–221 Dinic, E.A. [1970]: Algorithm for solution of a problem of maximum flow in a network with power estimation. Soviet Mathematics Doklady 11 (1970), 1277–1280 Edmonds, J., and Karp, R.M. [1972]: Theoretical improvements in algorithmic efficiency for network flow problems. Journal of the ACM 19 (1972), 248–264 Elias, P., Feinstein, A., and Shannon, C.E. [1956]: Note on maximum flow through a network. IRE Transactions on Information Theory, IT-2 (1956), 117–119 Ford, L.R., and Fulkerson, D.R. [1956]: Maximal Flow Through a Network. Canadian Journal of Mathematics 8 (1956), 399–404 Ford, L.R., and Fulkerson, D.R. [1957]: A simple algorithm for finding maximal network flows and an application to the Hitchcock problem. Canadian Journal of Mathematics 9 (1957), 210–218 Frank, A. [1994]: On the edge-connectivity algorithm of Nagamochi and Ibaraki. Laboratoire Artemis, IMAG, Universit´e J. Fourier, Grenoble, 1994 Fujishige, S. [2003]: A maximum flow algorithm using MA ordering. Operations Research Letters 31 (2003), 176–178 Gabow, H.N. [1995]: A matroid approach to finding edge connectivity and packing arborescences. Journal of Computer and System Sciences 50 (1995), 259–273 5

2

Galil, Z. [1980]: An O(V 3 E 3 ) algorithm for the maximal flow problem. Acta Informatica 14 (1980), 221–242 Galil, Z., and Namaad, A. [1980]: An O(E V log2 V ) algorithm for the maximal flow problem. Journal of Computer and System Sciences 21 (1980), 203–217

196

8 Network Flows

Gallai, T. [1958]: Maximum-minimum S¨atze u¨ ber Graphen. Acta Mathematica Academiae Scientiarum Hungaricae 9 (1958), 395–434 Goldberg, A.V., and Rao, S. [1998]: Beyond the flow decomposition barrier. Journal of the ACM 45 (1998), 783–797 Goldberg, A.V., and Tarjan, R.E. [1988]: A new approach to the maximum flow problem. Journal of the ACM 35 (1988), 921–940 Gomory, R.E., and Hu, T.C. [1961]: Multi-terminal network flows. Journal of SIAM 9 (1961), 551–570 Gusfield, D. [1990]: Very simple methods for all pairs network flow analysis. SIAM Journal on Computing 19 (1990), 143–155 Hao, J., and Orlin, J.B. [1994]: A faster algorithm for finding the minimum cut in a directed graph. Journal of Algorithms 17 (1994), 409–423 Henzinger, M.R., Rao, S., and Gabow, H.N. [2000]: Computing vertex connectivity: new bounds from old techniques. Journal of Algorithms 34 (2000), 222–250 Hoffman, A.J. [1960]: Some recent applications of the theory of linear inequalities to extremal combinatorial analysis. In: Combinatorial Analysis (R.E. Bellman, M. Hall, eds.), AMS, Providence 1960, pp. 113–128 Hu, T.C. [1969]: Integer Programming and Network Flows. Addison-Wesley, Reading 1969 Karger, D.R. [2000]: Minimum cuts in near-linear time. Journal of the ACM 47 (2000), 46–76 Karger, D.R., and Levine, M.S. [1998]: Finding maximum flows in undirected graphs seems easier than bipartite matching. Proceedings of the 30th Annual ACM Symposium on Theory of Computing (1998), 69–78 Karger, D.R., and Stein, C. [1996]: A new approach to the minimum cut problem. Journal of the ACM 43 (1996), 601–640 Karzanov, A.V. [1974]: Determining the maximal flow in a network by the method of preflows. Soviet Mathematics Doklady 15 (1974), 434–437 King, V., Rao, S., and Tarjan, R.E. [1994]: A faster deterministic maximum flow algorithm. Journal of Algorithms 17 (1994), 447–474 ¨ Mader, W. [1972]: Uber minimal n-fach zusammenh¨angende, unendliche Graphen und ein Extremalproblem. Arch. Math. 23 (1972), 553–560 Mader, W. [1981]: On a property of n edge-connected digraphs. Combinatorica 1 (1981), 385–386 Malhotra, V.M., Kumar, M.P., and Maheshwari, S.N. [1978]: An O(|V |3 ) algorithm for finding maximum flows in networks. Information Processing Letters 7 (1978), 277–278 Menger, K. [1927]: Zur allgemeinen Kurventheorie. Fundamenta Mathematicae 10 (1927), 96–115 Nagamochi, H., and Ibaraki, T. [1992]: Computing edge-connectivity in multigraphs and capacitated graphs. SIAM Journal on Discrete Mathematics 5 (1992), 54–66 Nagamochi, H., and Ibaraki, T. [2000]: A fast algorithm for computing minimum 3-way and 4-way cuts. Mathematical Programming 88 (2000), 507–520 Phillips, S., and Dessouky, M.I. [1977]: Solving the project time/cost tradeoff problem using the minimal cut concept. Management Science 24 (1977), 393–400 Picard, J., and Queyranne, M. [1980]: On the structure of all minimum cuts in a network and applications. Mathematical Programming Study 13 (1980), 8–16 Queyranne, M. [1998]: Minimizing symmetric submodular functions. Mathematical Programming B 82 (1998), 3–12 Rose, D.J. [1970]: Triangulated graphs and the elimination process. Journal of Mathematical Analysis and Applications 32 (1970), 597–609

References

197

Shiloach, Y. [1978]: An O(n I log2 I ) maximum-flow algorithm. Technical Report STANCS-78-802, Computer Science Department, Stanford University, 1978 Shiloach, Y. [1979]: Edge-disjoint branching in directed multigraphs. Information Processing Letters 8 (1979), 24–27 Shioura, A. [2004]: The MA ordering max-flow algorithm is not strongly polynomial for directed networks. Operations Research Letters 32 (2004), 31–35 Sleator, D.D. [1980]: An O(nm log n) algorithm for maximum network flow. Technical Report STAN-CS-80-831, Computer Science Department, Stanford University, 1978 Sleator, D.D., and Tarjan, R.E. [1983]: A data structure for dynamic trees. Journal of Computer and System Sciences 26 (1983), 362–391 Su, X.Y. [1997]: Some generalizations of Menger’s theorem concerning arc-connected digraphs. Discrete Mathematics 175 (1997), 293–296 Stoer, M., and Wagner, F. [1997]: A simple min cut algorithm. Journal of the ACM 44 (1997), 585–591 Tunc¸el, L. [1994]: On the complexity preflow-push algorithms for maximum flow problems. Algorithmica 11 (1994), 353–359 Vazirani, V.V., and Yannakakis, M. [1992]: Suboptimal cuts: their enumeration, weight, and number. In: Automata, Languages and Programming; Proceedings; LNCS 623 (W. Kuich, ed.), Springer, Berlin 1992, pp. 366–377 Vygen, J. [2002]: On dual minimum cost flow algorithms. Mathematical Methods of Operations Research 56 (2002), 101–126 Weihe, K. [1997]: Maximum (s, t)-flows in planar networks in O(|V | log |V |) time. Journal of Computer and System Sciences 55 (1997), 454–475 Whitney, H. [1932]: Congruent graphs and the connectivity of graphs. American Journal of Mathematics 54 (1932), 150–168

9 Minimum Cost Flows

In this chapter we show how we can take edge costs into account. For example, in our application of the M AXIMUM F LOW P ROBLEM to the J OB A SSIGNMENT P ROBLEM mentioned in the introduction of Chapter 8 one could introduce edge costs to model that the employees have different salaries; our goal is to meet a deadline when all jobs must be finished at a minimum cost. Of course, there are many more applications. A second generalization, allowing several sources and sinks, is more due to technical reasons. We introduce the general problem and an important special case in Section 9.1. In Section 9.2 we prove optimality criteria that are the basis of the minimum cost flow algorithms presented in Sections 9.3, 9.4, 9.5 and 9.6. Most of these use algorithms of Chapter 7 for finding a minimum mean cycle or a shortest path as a subroutine. Section 9.7 concludes this chapter with an application to time-dependent flows.

9.1 Problem Formulation We are again given a digraph G with capacities u : E(G) → R+ , but in addition numbers c : E(G) → R indicating the cost of each edge. Furthermore, we allow several sources and sinks: Definition 9.1. Given P a digraph G, capacities u : E(G) → R+ , and numbers b : function f : E(G) → V (G) → R with v∈V (G) b(v) = 0, a b-flow P in (G, u) is a P R+ with f (e) ≤ u(e) for all e ∈ E(G) and e∈δ + (v) f (e)− e∈δ − (v) f (e) = b(v) for all v ∈ V (G). Thus a b-flow with b ≡ 0 is a circulation. b(v) is called the balance of vertex v. |b(v)| is sometimes called the supply (if b(v) > 0) or the demand (if b(v) < 0) of v. Vertices v with b(v) > 0 are called sources, those with b(v) < 0 sinks. Note that a b-flow can be found by any algorithm for the M AXIMUM F LOW P ROBLEM: Just add two vertices s and t and edges (s, v), (v, t) with capacities u((s, v)) := max{0, b(v)} and P u((v, t)) := max{0, −b(v)} for all v ∈ V (G) to G. Then any s-t-flow of value v∈V (G) u((s, v)) in the resulting network corresponds to a b-flow in G. Thus a criterion for the existence of a b-flow can be derived from the Max-Flow-Min-Cut Theorem 8.6 (see Exercise 2). The problem is to find a minimum cost b-flow:

200

9 Minimum Cost Flows

M INIMUM C OST F LOW P ROBLEM Instance: Task:

A digraph P G, capacities u : E(G) → R+ , numbers b : V (G) → R with v∈V (G) b(v) = 0, and weights c : E(G) → R. P Find a b-flow f whose cost c( f ) := e∈E(G) f (e)c(e) is minimum (or decide that none exists).

Sometimes one also allows infinite capacities. In this case an instance can be unbounded, but this can be checked in advance easily; see Exercise 5. The M INIMUM C OST F LOW P ROBLEM is quite general and has a couple of interesting special cases. The uncapacitated case (u ≡ ∞) is sometimes called the transshipment problem. An even more restricted problem, also known as the transportation problem, has been formulated quite early by Hitchcock [1941] and others:

H ITCHCOCK P ROBLEM

.

Instance:

A digraph G with V (G) = A ∪ B and E(G) ⊆ A × B. Supplies b(v) P ≥ 0 for v ∈ A and demands −b(v) ≥ 0 for v ∈ B with v∈V (G) b(v) = 0. Weights c : E(G) → R.

Task:

Find a b-flow f in (G, ∞) of minimum cost (or decide that none exists).

In the H ITCHCOCK P ROBLEM it causes no loss of generality to assume that c is nonnegative: Adding a constant α to P each weight increases the cost of each b-flow by the same amount, namely by α v∈A b(v). Often only the special case where c is nonnegative and E(G) = A × B is considered. Obviously, any instance of the H ITCHCOCK P ROBLEM can be written as an instance of the M INIMUM C OST F LOW P ROBLEM on a bipartite graph with infinite capacities. It is less obvious that any instance of the M INIMUM C OST F LOW P ROB LEM can be transformed to an equivalent (but larger) instance of the H ITCHCOCK P ROBLEM: Lemma 9.2. (Orden [1956], Wagner [1959]) An instance of the M INIMUM C OST F LOW P ROBLEM with n vertices and m edges can be transformed to an equivalent instance of the H ITCHCOCK P ROBLEM with n + m vertices and 2m edges. Proof: Let (G, u, b, c) be an instance of the M INIMUM C OST F LOW P ROBLEM. We define an equivalent instance (G 0 , A0 , B 0 , b0 , c0 ) of the H ITCHCOCK P ROBLEM as follows: Let A0 := E(G), B 0 := V (G) and G 0 := (A0 ∪ B 0 , E 1 ∪ E 2 ), where E 1 := {((x, y), x) : (x, y) ∈ E(G)} and E 2 := {((x, y), y) : (x, y) ∈ E(G)}. Let c0 ((e, x)) := 0 for (e, x) ∈ E 1 and c0 ((e, y)) := c(e) for (e, y) ∈ E 2 . Finally let b0 (e) := u(e) for e ∈ E(G) and X u(e) for x ∈ V (G). b0 (x) := b(x) − + e∈δG (x)

9.2 An Optimality Criterion b (e1 ) = 5

b(x) = 4

e1

b (e2 ) = 4 e2

b(y) = −1

e3

b(z) = −3

b (e3 ) = 7

u(e1 ) = 5, u(e2 ) = 4, u(e3 ) = 7

0 c(e1 ) 0 c(e2 ) c(e3 ) 0

201

b (x) = −1

b (y) = −5

b (z) = −10

u ≡ ∞ Fig. 9.1.

For an example, see Figure 9.1. We prove that both instances are equivalent. Let f be a b-ﬂow in (G, u). Deﬁne f ((e, y)) := f (e) and f ((e, x)) := u(e) − f (e) for e = (x, y) ∈ E(G). Obviously f is a b -ﬂow in G with c ( f ) = c( f ). Conversely, if f is a b -ﬂow in G , then f ((x, y)) := f (((x, y), y)) deﬁnes a b-ﬂow in G with c( f ) = c ( f ). The above proof is due to Ford and Fulkerson [1962].

9.2 An Optimality Criterion In this section we prove some simple results, in particular an optimality criterion, which will be the basis for the algorithms in the subsequent sections. We again use the concepts of residual graphs and augmenting paths. We extend the weights ↔

←

c to G by deﬁning c( e ) := −c(e) for each edge e ∈ E(G). Our deﬁnition of a residual graph has the advantage that the weight of an edge in a residual graph G f is independent of the ﬂow f . Deﬁnition 9.3. Given a digraph G with capacities and a b-ﬂow f , an f-augmenting cycle is a circuit in G f . The following simple observation will prove useful: Proposition 9.4. Let G be a digraph with capacities u : E(G) → R+ . Let f and ↔

f be b-ﬂows in (G, u). Then g : E(G) → R+ deﬁned by g(e) := max{0, f (e) − ↔

←

f (e)} and g( e ) := max{0, f (e) − f (e)} for e ∈ E(G) is a circulation in G. Furthermore, g(e) = 0 for all e ∈ / E(G f ) and c(g) = c( f ) − c( f ). ↔

Proof: At each vertex v ∈ V (G) we have

202

9 Minimum Cost Flows

X

X

g(e) −

+ e∈δ↔ (v)

− e∈δ↔ (v)

G

X

g(e) =

( f 0 (e) − f (e)) −

+ e∈δG (v)

G

X

( f 0 (e) − f (e))

− e∈δG (v)

= b(v) − b(v) = 0, ↔

so g is a circulation in G. ↔

For e ∈ E(G) \ E(G f ) we consider two cases: If e ∈ E(G) then f (e) = u(e) ← and thus f 0 (e) ≤ f (e), implying g(e) = 0. If e = e0 for some e0 ∈ E(G) then ← f (e0 ) = 0 and thus g(e0 ) = 0. The last statement is easily verified: X X X c(g) = c(e)g(e) = c(e) f 0 (e) − c(e) f (e) = c( f 0 ) − c( f ). ↔

e∈E(G)

e∈E(G)

e∈E(G)

Just as Eulerian graphs can be partitioned into circuits, circulations can be decomposed into flows on single circuits: Proposition 9.5. (Ford and Fulkerson [1962]) For any circulation f in a digraph G there is a family C of atP most |E(G)| circuits in G and positive numbers h(C) (C ∈ C) such that f (e) = {h(C) : C ∈ C, e ∈ E(C)} for all e ∈ E(G). Proof: This is a special case of Theorem 8.8.

Now we can prove an optimality criterion: Theorem 9.6. (Klein [1967]) Let (G, u, b, c) be an instance of the M INIMUM C OST F LOW P ROBLEM. A b-flow f is of minimum cost if and only if there is no f -augmenting cycle with negative total weight. Proof: If there is an f -augmenting cycle C with weight γ < 0, we can augment f along C by some ε > 0 and get a b-flow f 0 with cost decreased by −γ ε. So f is not a minimum cost flow. If f is not a minimum cost b-flow, there is another b-flow f 0 with smaller cost. Consider g as defined in Proposition 9.4. Then g is a circulation with c(g) < 0. By Proposition 9.5, g can be decomposed into flows on single circuits. Since g(e) = 0 for all e ∈ / E(G f ), all these circuits are f -augmenting. At least one of them must have negative total weight, proving the theorem. This result goes back essentially to Tolsto˘ı [1930] and has been rediscovered several times in different forms. One equivalent formulation is the following: Corollary 9.7. (Ford and Fulkerson [1962]) Let (G, u, b, c) be an instance of the M INIMUM C OST F LOW P ROBLEM. A b-flow f is of minimum cost if and only if there exists a feasible potential for (G f , c).

9.3 Minimum Mean Cycle-Cancelling Algorithm

203

Proof: By Theorem 9.6 f is a minimum cost b-flow if and only if G f contains no negative circuit. By Theorem 7.7 there is no negative circuit in (G f , c) if and only if there exists a feasible potential. Feasible potentials can also be regarded as solutions of the linear programming dual of the M INIMUM C OST F LOW P ROBLEM. This is shown by the following different proof of the above optimality criterion: Second Proof of Corollary 9.7: We write the M INIMUM C OST F LOW P ROB LEM as a maximization problem and consider the LP X −c(e)xe max e∈E(G)

s.t.

X

xe −

e∈δ + (v)

X

xe = b(v)

(v ∈ V (G))

xe ≤ u(e) xe ≥ 0

(e ∈ E(G)) (e ∈ E(G))

e∈δ − (v)

(9.1)

and its dual min

X

b(v)yv +

v∈V (G)

s.t.

X

u(e)z e

e∈E(G)

yv − yw + z e ≥ −c(e) ze ≥ 0

(e = (v, w) ∈ E(G)) (e ∈ E(G))

(9.2)

Let x be any b-flow, i.e. any feasible solution of (9.1). By Corollary 3.23 x is optimum if and only if there exists a feasible dual solution (y, z) of (9.2) such that x and (y, z) satisfy the complementary slackness conditions z e (u(e)− xe ) = 0 and xe (c(e)+ z e + yv − yw ) = 0 for all e = (v, w) ∈ E(G). So x is optimum if and only if there exists a pair of vectors (y, z) with 0 = −z e ≤ c(e) + yv − yw c(e) + yv − yw = −z e ≤ 0

for e = (v, w) ∈ E(G) with xe < u(e)

and

for e = (v, w) ∈ E(G) with xe > 0.

This is equivalent to the existence of a vector y such that c(e) + yv − yw ≥ 0 for all residual edges e = (v, w) ∈ E(G x ), i.e. to the existence of a feasible potential y for (G x , c).

9.3 Minimum Mean Cycle-Cancelling Algorithm Note that Klein’s Theorem 9.6 already suggests an algorithm: first find any b-flow (using a max-flow algorithm as described above), and then successively augment along negative weight augmenting cycles until no more exist. We must however

204

9 Minimum Cost Flows

be careful in choosing the cycle if we want to have polynomial running time (see Exercise 7). A good strategy is to choose an augmenting cycle with minimum mean weight each time:

M INIMUM M EAN C YCLE -C ANCELLING A LGORITHM Input:

A digraph P G, capacities u : E(G) → R+ , numbers b : V (G) → R with v∈V (G) b(v) = 0, and weights c : E(G) → R.

Output:

A minimum cost b-flow f .

1

Find a b-flow f .

2

Find a circuit C in G f whose mean weight is minimum. If C has nonnegative total weight (or G f is acyclic) then stop.

3

Compute γ := min u f (e). Augment f along C by γ . e∈E(C)

Go to . 2 As described in Section 9.1, 1 can be implemented with any algorithm for the M AXIMUM F LOW P ROBLEM. 2 can be implemented with the algorithm presented in Section 7.3. We shall now prove that this algorithm terminates after a polynomial number of iterations. The proof will be similar to the one in Section 8.3. Let µ( f ) denote the minimum mean weight of a circuit in G f . Then Theorem 9.6 says that a b-flow f is optimum if and only if µ( f ) ≥ 0. We first show that µ( f ) is non-decreasing throughout the algorithm. Moreover, we can show that it is strictly increasing with every |E(G)| iterations. As usual we denote by n and m the number of vertices and edges of G, respectively. Lemma 9.8. Let f 1 , f 2 , . . . , f t be a sequence of b-flows such that for i = 1, . . . , t − 1 we have µ( f i ) < 0 and f i+1 results from f i by augmenting along Ci , where Ci is a circuit of minimum mean weight in G fi . Then (a) µ( f k ) ≤ µ( f k+1 ) for all k. n µ( fl ) for all k < l such that Ck ∪ Cl contains a pair of reverse (b) µ( f k ) ≤ n−2 edges. Proof: (a): Let f k , f k+1 be two consecutive flows in this sequence. Consider the . Eulerian graph H resulting from (V (G), E(Ck ) ∪ E(Ck+1 )) by deleting pairs of reverse edges. (Edges appearing both in Ck and Ck+1 are counted twice.) H is a subgraph of G fk because each edge in E(G fk+1 ) \ E(G fk ) must be the reverse of an edge in E(Ck ). Since H is Eulerian, it can be decomposed into circuits, and each of these circuits has mean weight at least µ( f k ). So c(E(H )) ≥ µ( f k )|E(H )|. Since the total weight of each pair of reverse edges is zero, c(E(H )) = c(E(Ck )) + c(E(Ck+1 )) = µ( f k )|E(Ck )| + µ( f k+1 )|E(Ck+1 )|. Since |E(H )| ≤ |E(Ck )| + |E(Ck+1 )|, we conclude

9.3 Minimum Mean Cycle-Cancelling Algorithm

205

µ( f k )(|E(Ck )| + |E(Ck+1 )|) ≤ µ( f k )|E(H )| ≤ c(E(H )) = µ( f k )|E(Ck )| + µ( f k+1 )|E(Ck+1 )|, implying µ( f k+1 ) ≥ µ( f k ). (b): By (a) it is enough to prove the statement for those k, l such that for k < i < l, Ci ∪ Cl contains no pair of reverse edges. As in the proof of (a), consider the Eulerian graph H resulting from (V (G), . E(Ck ) ∪ E(Cl )) by deleting pairs of reverse edges. H is a subgraph of G fk because any edge in E(Cl ) \ E(G fk ) must be the reverse of an edge in one of Ck , Ck+1 , . . . , Cl−1 . But – due to the choice of k and l – only Ck among these contains the reverse of an edge of Cl . So as in (a) we have c(E(H )) ≥ µ( f k )|E(H )| and c(E(H )) = µ( f k )|E(Ck )| + µ( fl )|E(Cl )|. Since |E(H )| ≤ |E(Ck )| + n−2 n |E(Cl )| (we deleted at least two edges) we get n−2 |E(Cl )| ≤ µ( f k )|E(H )| µ( f k ) |E(Ck )| + n ≤ c(E(H )) = µ( f k )|E(Ck )| + µ( fl )|E(Cl )|, implying µ( f k ) ≤

n n−2

µ( fl ).

Corollary 9.9. During the execution of the M INIMUM M EAN C YCLE -C ANCELLING A LGORITHM, |µ( f )| decreases by at least a factor of 12 with every mn iterations. Proof: Let Ck , Ck+1 , . . . , Ck+m be the augmenting cycles in consecutive iterations of the algorithm. Since each of these circuits contains one edge as a bottleneck edge (an edge removed afterwards from the residual graph), there must be two of these circuits, say Ci and C j (k ≤ i < j ≤ k + m) whose union contains a pair of reverse edges. By Lemma 9.8 we then have n n µ( f k ) ≤ µ( f i ) ≤ µ( f j ) ≤ µ( f k+m ). n−2 n−2 with every m iterations. The So |µ( f )| decreases by at least a factor of n−2 n n n−2 −2 corollary follows from this because of < e < 12 . n This already proves that the algorithm runs in polynomial time provided that all edge costs are integral: |µ( f )| is at most |cmin | at the beginning, where cmin is the minimum cost of any edge, and decreases by at least a factor of 21 with every mn iterations. So after O(mn log(n|cmin |)) iterations, µ( f ) is greater than − n1 . If the edge costs are integral, this implies µ( f ) ≥ 0 and the algorithm stops. So by Corollary 7.14, the running time is O m 2 n 2 log(n|cmin |) . Even better, we can also derive a strongly polynomial running time for the M INIMUM C OST F LOW P ROBLEM (first obtained by Tardos [1985]):

206

9 Minimum Cost Flows

Theorem 9.10. (Goldberg and Tarjan [1989]) The M INIMUM M EAN C YCLE C ANCELLING A LGORITHM runs in O m 3 n 2 log n time. Proof: We show that every mn(log n + 1) iterations at least one edge is ﬁxed, i.e. the ﬂow on this edge will not change anymore. Therefore there are at most O m 2 n log n iterations. Using Corollary 8.15 for 1 and Corollary 7.14 for 2 then proves the theorem. Let f be the ﬂow at some iteration, and let f be the ﬂow mn(log n + 1) iterations later. Deﬁne weights c by c (e) := c(e) − µ( f ) (e ∈ E(G f )). Let π be a feasible potential of (G f , c ) (which exists by Theorem 7.7). We have 0 ≤ cπ (e) = cπ (e) − µ( f ), so cπ (e) ≥ µ( f )

for all e ∈ E(G f ).

(9.3)

Now let C be the circuit of minimum mean weight in G f that is chosen in the algorithm to augment f . Since by Corollary 9.9 µ( f ) ≤ 2log n+1 µ( f ) ≤ 2nµ( f ) (see Figure 9.2), we have cπ (e) = c(e) = µ( f )|E(C)| ≤ 2nµ( f )|E(C)|. e∈E(C)

e∈E(C)

/ E(G f ). So let e0 = (x, y) ∈ E(C) with cπ (e0 ) ≤ 2nµ( f ). By (9.3) we have e0 ∈ µ( f )

2nµ( f )

µ( f )

0

Fig. 9.2.

Claim: For any b-ﬂow f with e0 ∈ E(G f ) we have µ( f ) < µ( f ). By Lemma 9.8(a) the claim implies that e0 will never be in the residual graph ← anymore, i.e. e0 and e0 are ﬁxed mn(log n + 1) iterations after e0 is used in C. This completes the proof. To prove the claim, let f be a b-ﬂow with e0 ∈ E(G f ). We apply Proposition 9.4 to f and f and obtain a circulation g with g(e) = 0 for all e ∈ / E(G f ) and ←

g(e0 ) > 0 (because e0 ∈ E(G f ) \ E(G f )). By Proposition 9.5, g can be written as the sum of ﬂows on f -augmenting ← ← cycles. One of these circuits, say W , must contain e0 . By using cπ (e0 ) = −cπ (e0 ) ≥ ← −2nµ( f ) and applying (9.3) to all e ∈ E(W ) \ {e0 } we obtain a lower bound for the total weight of W : cπ (e) ≥ −2nµ( f ) + (n − 1)µ( f ) > −nµ( f ). c(E(W )) = e∈E(W )

9.4 Successive Shortest Path Algorithm

207

But the reverse of W is an f 00 -augmenting cycle (this can be seen by exchanging the roles of f 0 and f 00 ), and its total weight is less than nµ( f 0 ). This means that G f 00 contains a circuit whose mean weight is less than µ( f 0 ), and so the claim is proved.

9.4 Successive Shortest Path Algorithm The following theorem gives rise to another algorithm: Theorem 9.11. (Jewell [1958], Iri [1960], Busacker and Gowen [1961]) Let (G, u, b, c) be an instance of the M INIMUM C OST F LOW P ROBLEM, and let f be a minimum cost b-flow. Let P be a shortest (with respect to c) s-t-path in G f (for some s and t). Let f 0 be a flow obtained when augmenting f along P by at most the minimum residual capacity on P. Then f 0 is a minimum cost b0 -flow (for some b0 ). Proof: f 0 is a b0 -flow for some b0 . Suppose f 0 is not a minimum cost b0 -flow. Then by Theorem 9.6 there is a circuit C in G f 0 with negative total weight. Consider . the graph H resulting from (V (G), E(C) ∪ E(P)) by deleting pairs of reverse edges. (Again, edges appearing both in C and P are taken twice.) For any edge e ∈ E(G f 0 ) \ E(G f ), the reverse of e must be in E(P). Therefore E(H ) ⊆ E(G f ). We have c(E(H )) = c(E(C)) + c(E(P)) < c(E(P)). Furthermore, H is the union of an s-t-path and some circuits. But since E(H ) ⊆ E(G f ), none of the circuits can have negative weight (otherwise f would not be a minimum cost b-flow). Therefore H , and thus G f , contains an s-t-path of less weight than P, contradicting the choice of P. If the weights are conservative, we can start with f ≡ 0 as an optimum circulation (b-flow with b ≡ 0). Otherwise we can initially saturate all edges of negative cost and bounded capacity. This changes the b-values but guarantees that there is no negative augmenting cycle (i.e. c is conservative for G f ) unless the instance is unbounded.

S UCCESSIVE S HORTEST PATH A LGORITHM Input:

A digraph P G, capacities u : E(G) → R+ , numbers b : V (G) → R with v∈V (G) b(v) = 0, and conservative weights c : E(G) → R.

Output:

A minimum cost b-flow f .

1

Set b0 := b and f (e) := 0 for all e ∈ E(G).

2

If b0 = 0 then stop, else: Choose a vertex s with b0 (s) > 0. Choose a vertex t with b0 (t) < 0 such that t is reachable from s in G f . If there is no such t then stop. (There exists no b-flow.)

208

3

4

9 Minimum Cost Flows

Find an s-t-path P in G f of minimum weight. Compute γ := min min u f (e), b0 (s), −b0 (t) . e∈E(P)

Set b0 (s) := b0 (s) − γ and b0 (t) := b0 (t) + γ . Augment f along P by γ . Go to . 2 If we allow arbitrary capacities, we have the same problems as with the F ORD F ULKERSON A LGORITHM (see Exercise 2 of Chapter 8; set all costs to zero). So henceforth we assume u and P b to be integral. Then it is clear that the algorithm stops after at most B := 12 v∈V (G) |b(v)| augmentations. By Theorem 9.11, the resulting flow is optimum if the initial zero flow is optimum. This is true if and only if c is conservative. We remark that if the algorithm decides that there is no b-flow, this decision is indeed correct. This is an easy observation, left as Exercise 13. Each augmentation requires a shortest path computation. Since negative weights occur, we have to use the M OORE -B ELLMAN -F ORD A LGORITHM whose running time is O(nm) (Theorem 7.5), so the overall running time will be O(Bnm). However, as in the proof of Theorem 7.9, it can be arranged that (except at the beginning) the shortest paths are computed in a graph with nonnegative weights: Theorem 9.12. (Tomizawa [1971], Edmonds and Karp [1972]) For integral capacities and supplies, the S UCCESSIVE S HORTEST PATH A LGORITHM can bePimplemented with a running time of O (nm + B(m + n log n)), where B = 1 v∈V (G) |b(v)|. 2 Proof: It is convenient to assume that there is only one source s. Otherwise we introduce a new vertex s and edges (s, v) with capacity max{0, b(v)} and zero cost for all v ∈ V (G). Then we can set b(s) := B and b(v) := 0 for each former source v. In this way we obtain an equivalent problem with only one source. Moreover, we may assume that every vertex is reachable from s (other vertices can be deleted). We introduce a potential πi : V (G) → R for each iteration i of the S UCCESSIVE S HORTEST PATH A LGORITHM. We start with any feasible potential π0 of (G, c). By Corollary 7.8, this exists and can be computed in O(mn) time. Now let f i−1 be the flow before iteration i. Then the shortest path computation in iteration i is done with the reduced costs cπi−1 instead of c. Let li (v) denote the length of a shortest s-v-path in G fi−1 with respect to the weights cπi−1 . Then we set πi (v) := πi−1 (v) + li (v). We prove by induction on i that πi is a feasible potential for (G fi , c). This is clear for i = 0. For i > 0 and any edge e = (x, y) ∈ E(G fi−1 ) we have (by definition of li and the induction hypothesis) li (y) ≤ li (x) + cπi−1 (e) = li (x) + c(e) + πi−1 (x) − πi−1 (y), so cπi (e) = c(e) + πi (x) − πi (y) = c(e) + πi−1 (x) + li (x) − πi−1 (y) − li (y) ≥ 0.

9.4 Successive Shortest Path Algorithm

209

For any edge e = (x, y) ∈ Pi (where Pi is the augmenting path in iteration i) we have li (y) = li (x) + cπi−1 (e) = li (x) + c(e) + πi−1 (x) − πi−1 (y), so cπi (e) = 0, and the reverse edge of e also has zero weight. Since each edge in E(G fi ) \ E(G fi−1 ) is the reverse of an edge in Pi , cπi is indeed a nonnegative weight function on E(G fi ). We observe that, for any i and any t, the shortest s-t-paths with respect to c are precisely the shortest s-t-paths with respect to cπi , because cπi (P) − c(P) = πi (s) − πi (t) for any s-t-path P. Hence we can use D IJKSTRA ’ S A LGORITHM – which runs in O (m + n log n) time when implemented with a Fibonacci heap by Theorem 7.4 – for all shortest path computations except the initial one. Since we have at most B iterations, we obtain an overall running time of O (nm + B(m + n log n)). Note that (in contrast to many other problems, e.g. the M AXIMUM F LOW P ROBLEM) we cannot assume without loss of generality that the input graph is simple when considering the M INIMUM C OST F LOW P ROBLEM. The running time of Theorem 9.12 is still exponential unless B is known to be small. If B = O(n), this is the fastest algorithm known. For an application, see Section 11.1. In the rest of this section we show how to modify the algorithm in order to reduce the number of shortest path computations. We only consider the case when all capacities are infinite. By Lemma 9.2 each instance of the M INIMUM C OST F LOW P ROBLEM can be transformed to an equivalent instance with infinite capacities. The basic idea – due to Edmonds and Karp [1972] – is the following. In early iterations we consider only augmenting paths where γ – the amount of flow that can be pushed – is large. We start with γ = 2blog bmax c and reduce γ by a factor of two if no more augmentations by γ can be done. After blog bmax c + 1 iterations we have γ = 1 and stop (we again assume b to be integral). Such a scaling technique has proved useful for many algorithms (see also Exercise 14). A detailed description of the first scaling algorithm reads as follows:

C APACITY S CALING A LGORITHM Input:

Output:

1

A digraph G with infinite P capacities u(e) = ∞ (e ∈ E(G)), numbers b : V (G) → Z with v∈V (G) b(v) = 0, and conservative weights c : E(G) → R. A minimum cost b-flow f .

Set b0 := b and f (e) := 0 for all e ∈ E(G). Set γ = 2blog bmax c , where bmax = max{b(v) : v ∈ V (G)}.

210

2

3

4

5

9 Minimum Cost Flows

If b0 = 0 then stop, else: Choose a vertex s with b0 (s) ≥ γ . Choose a vertex t with b0 (t) ≤ −γ such that t is reachable from s in G f . If there is no such s or t then go to . 5 Find an s-t-path P in G f of minimum weight. Set b0 (s) := b0 (s) − γ and b0 (t) := b0 (t) + γ . Augment f along P by γ . Go to . 2 If γ = 1 then stop. (There exists no b-flow.) Else set γ := γ2 and go to . 2

Theorem 9.13. (Edmonds and Karp [1972]) The C APACITY S CALING A LGO RITHM correctly solves the M INIMUM C OST F LOW P ROBLEM for integral b, infinite capacities and conservative weights. It can be implemented to run in O(n(m + n log n) log bmax ) time, where bmax = max{b(v) : v ∈ V (G)}. Proof: As above, the correctness follows directly from Theorem 9.11. Note that at any time, the residual capacity of any edge is either infinite or an integral multiple of γ . To establish the running time, we call the period in which γ remains constant a phase. We prove that there are less than 4n augmentations within each phase. Suppose this is not true. For some value of γ , let f and g be the flow at the beginning and at the end P of the γ -phase, respectively. g − f can be regarded as a b00 -flow in G f , where x∈V (G) |b00 (x)| ≥ 8nγ . Let S := {x ∈ V (G) : b00 (x) > 0}, S + := {x ∈ V (G) : b00 (x) ≥ 2γ }, T := {x ∈ V (G) : b00 (x) < 0}, T + := {x ∈ V (G) : b00 (x) ≤ −2γ }. If there had been a path from S + to T + in G f , the 2γ -phase would have continued. Therefore the total b00 -value of all sinks reachable from S + in G f is greater than n(−2γ ). Therefore (note that there exists a b00 -flow in G f ) P 00 x∈S + b (x) < 2nγ . Now we have X X X X |b00 (x)| = 2 b00 (x) = 2 b00 (x) + b00 (x) x∈V (G)

x∈S

x∈S +

x∈S\S +

< 2(2nγ + 2nγ ) = 8nγ , a contradiction. This means that the total number of shortest path computations is O(n log bmax ). Combining this with the technique of Theorem 9.12 we obtain the O(mn + n log bmax (m + n log n)) bound. This was the first polynomial-time algorithm for the M INIMUM C OST F LOW P ROBLEM. By some further modifications we can even obtain a strongly polynomial running time. This is the subject of the next section.

9.5 Orlin’s Algorithm

211

9.5 Orlin’s Algorithm The C APACITY S CALING A LGORITHM of the previous section can be improved further. A basic idea is that if an edge carries more than 8nγ units of flow at any stage of the C APACITY S CALING A LGORITHM, it may be contracted. Namely, observe that such an edge will always keep a positive flow (and thus zero reduced cost with respect to any feasible potential in the residual graph): there are at most 4n more augmentations by γ , another 4n by γ2 and so on; hence the total amount of flow moved in the rest of the algorithm is less than 8nγ . We shall describe O RLIN ’ S A LGORITHM without explicitly using contraction. This simplifies the description, especially from the point of view of implementing the algorithm. A set F keeps track of the edges (and their reverse edges) that can be contracted. A representative is chosen out of each connected component of (V (G), F). The algorithm maintains the property that the representative of a connected component is its only non-balanced vertex. For any vertex x, r (x) denotes the representative of the connected component of (V (G), F) containing x. O RLIN ’ S A LGORITHM does not require that b is integral. However, it can deal with uncapacitated problems only (but recall Lemma 9.2).

O RLIN ’ S A LGORITHM Input:

A digraph G with infinite P capacities u(e) = ∞ (e ∈ E(G)), numbers b : V (G) → R with v∈V (G) b(v) = 0, and conservative weights c : E(G) → R. A minimum cost b-flow f .

Output:

1

Set b0 := b and f (e) := 0 for all e ∈ E(G). Set r (v) := v for all v ∈ V (G). Set F := ∅. Set γ = max |b0 (v)|. v∈V (G)

b0

2

If

3

Choose a vertex s with b0 (s) > n−1 n γ. If there is no such s then go to . 4 Choose a vertex t with b0 (t) < − n1 γ such that t is reachable from s in G f . If there is no such t then stop. (There exists no b-flow.) Go to . 5

4

Choose a vertex t with b0 (t) < − n−1 n γ. If there is no such t then go to . 6 Choose a vertex s with b0 (s) > n1 γ such that t is reachable from s in G f . If there is no such s then stop. (There exists no b-flow.) Find an s-t-path P in G f of minimum weight. Set b0 (s) := b0 (s) − γ and b0 (t) := b0 (t) + γ . Augment f along P by γ . Go to . 2

5

= 0 then stop.

212

6

9 Minimum Cost Flows

If f (e) = 0 for all e ∈ E(G) \ F then set γ := min else set γ :=

7

8

γ 2.

γ , max |b0 (v)| , 2 v∈V (G)

For all e = (x, y) ∈ E(G) \ F with r (x) 6= r (y) and f (e) > 8nγ do: ← Set F := F ∪ {e, e }. Let x 0 := r (x) and y 0 := r (y). Let Q be the x 0 -y 0 -path in F. If b0 (x 0 ) > 0 then augment f along Q by b0 (x 0 ), else augment f along the reverse of Q by −b0 (x 0 ). Set b0 (y 0 ) := b0 (y 0 ) + b0 (x 0 ) and b0 (x 0 ) := 0. Set r (z) := y 0 for all vertices z reachable from y 0 in F. Go to . 2

This algorithm is due to Orlin [1993]. See also (Plotkin and Tardos [1990]). Let us first prove its correctness. Let us call the time between two changes of γ a phase. Lemma 9.14. O RLIN ’ S A LGORITHM solves the uncapacitated M INIMUM C OST F LOW P ROBLEM with conservative weights correctly. At any stage f is a minimumcost (b − b0 )-flow. Proof: We first prove that f is always a (b − b0 )-flow. In particular, we have to show that f is always nonnegative. To prove this, we first observe that at any time the residual capacity of any edge not in F is either infinite or an integral multiple of γ . Moreover we claim that every edge e ∈ F always has positive residual capacity. To see this, observe that any phase consists of at most n − 1 augmentations by less than 2 n−1 7 and at most 2n augmentations by γ in n γ in

; 5 hence the total amount of flow moved after e has become a member of F in the γ -phase is less than 8nγ . Hence f is always nonnegative and thus it is always a (b − b0 )-flow. We now claim that f is always a minimum cost (b − b0 )-flow and that each v-w-path in F is a shortest v-w-path in G f . Indeed, the first statement implies the second one, since by Theorem 9.6 for a minimum cost flow f there is no negative circuit in G f . Now the claim follows from Theorem 9.11: P in 5 and Q in 7 are both shortest paths. We finally show that if the algorithm stops in 3 or 4 with b0 6= 0, then there is indeed no b-flow. Suppose the algorithm stops in , 3 implying that there is a vertex s with b0 (s) > n−1 γ , but that no vertex t with b0 (t) < − n1 γ is reachable n from s in G f . ThenPlet R be the set of vertices reachable from s in G f . Since f is a (b − b0 )-flow, x∈R (b(x) − b0 (x)) = 0. Therefore we have X X X X X b(x) = (b(x)−b0 (x))+ b0 (x) = b0 (x) = b0 (s)+ b0 (x) > 0. x∈R

x∈R

x∈R

x∈R

x∈R\{s}

This proves that no b-flow exists. An analogous proof applies in the case that the algorithm stops in . 4

9.5 Orlin’s Algorithm

213

We now analyse the running time. Lemma 9.15. (Plotkin and Tardos [1990]) If at some stage of the algorithm |b0 (s)| > n−1 n γ for a vertex s, then the connected component of (V (G), F) containing s increases during the next d2 log n + log me + 4 phases. Proof: Let |b0 (s)| > n−1 n γ1 for a vertex s at the beginning of some phase of the algorithm where γ = γ1 . Let γ0 be the γ -value in the preceding phase, and γ2 the γ -value d2 log n + log me + 4 phases later. We have 12 γ0 ≥ γ1 ≥ 16n 2 mγ2 . Let b10 and f 1 be the b0 and f at the beginning of the γ1 -phase, respectively, and let b20 and f 2 be the b0 and f at the end of the γ2 -phase, respectively. Let S be the connected component of (V (G), F) containing s in the γ1 -phase, and suppose that this remains unchanged for the d2 log n + log me + 4 phases considered. Note that 7 guarantees b0 (v) = 0 for all vertices v with r (v) 6= v. 0 Hence b (v) = 0 for all v ∈ S \ {s} and X X X X b(x) − b10 (s) = (b(x) − b10 (x)) = f 1 (e) − f 1 (e). (9.4) x∈S

e∈δ + (S)

x∈S

We claim that

e∈δ − (S)

X 1 b(x) ≥ γ1 . n

(9.5)

x∈S

If γ1 < γ20 , then each edge not in F has zero flow, so the right-hand side of (9.4) P 1 is zero, implying x∈S b(x) = |b10 (s)| > n−1 n γ1 ≥ n γ1 . γ0 In the other case (γ1 = 2 ) we have 1 n−1 n−1 2 γ1 ≤ γ1 < |b10 (s)| ≤ γ0 = γ 0 − γ 1 . n n n n

(9.6)

Since the flow on any edge not in F is an integral multiple of γ0 , the right-hand side of (9.4) is also an integral multiple of γ0 . This together with (9.6) implies (9.5). S minus the total flow on Now consider the total f 2 -flow on edges leaving P edges entering S. Since f 2 is a (b − b20 )-flow, this is x∈S b(x) − b20 (s). Using (9.5) and |b20 (s)| ≤ n−1 n γ2 we obtain X X n−1 1 γ1 − γ2 | f 2 (e)| ≥ b(x) − |b20 (s)| ≥ n n + − e∈δ (S)∪δ (S)

x∈S

≥ (16nm − 1)γ2 > m(8nγ2 ). Thus there exists at least one edge e with exactly one end in S and f 2 (e) > 8nγ2 . By 7 of the algorithm, this means that S is increased. Theorem 9.16. (Orlin [1993]) O RLIN ’ S A LGORITHM solves the uncapacitated M INIMUM C OST F LOW P ROBLEM with conservative weights correctly in O(n log m (m + n log n)) time.

214

9 Minimum Cost Flows

Proof: The correctness has been proved above (Lemma 9.14). 7 takes O(mn) total time. Lemma 9.15 implies that the total number of phases is O(n log m). Moreover, it says the following: For a vertex s and a set S ⊆ V (G) there are at most d2 log n +log me+4 augmentations in 5 starting at s while S is the connected component of (V (G), F) containing s. Since all vertices v with r (v) 6= v have b0 (v) = 0 at any time, there are at most d2 log n + log me + 4 augmentations for each set S that is at some stage of the algorithm a connected component of F. Since the family of these sets is laminar, there are at most 2n − 1 such sets (Corollary 2.15) and thus O(n log m) augmentations in 5 altogether. Using the technique of Theorem 9.12, we obtain an overall running time of O (mn + (n log m)(m + n log n)). This is the best known running time for the uncapacitated M INIMUM C OST F LOW P ROBLEM. Theorem 9.17. (Orlin [1993]) The general M INIMUM C OST F LOW P ROBLEM can be solved in O (m log m(m + n log n)) time, where n = |V (G)| and m = |E(G)|. Proof: We apply the construction given in Lemma 9.2. Thus we have to solve an uncapacitated M INIMUM C OST F LOW P ROBLEM on a bipartite graph H with . V (H ) = A0 ∪ B 0 , where A0 = E(G) and B 0 = V (G). Since H is acyclic, an initial feasible potential can be computed in O(|E(H )|) = O(m) time. As shown above (Theorem 9.16), the overall running time is bounded by O(m log m) shortest ↔

path computations in a subgraph of H with nonnegative weights. Before we call D IJKSTRA ’ S A LGORITHM we apply the following operation to each vertex a ∈ A0 that is not an endpoint of the path we are looking for: add an edge (b, b0 ) for each pair of edges (b, a), (a, b0 ) and set its weight to the sum of the weights of (b, a) and (a, b0 ); finally delete a. Clearly the resulting instance of the S HORTEST PATH P ROBLEM is equivalent. Since each vertex in A0 has four ↔

incident edges in H , the resulting graph has O(m) edges and at most n +2 vertices. The preprocessing takes constant time per vertex, i.e. O(m). The same holds for ↔

the final computation of the path in H and of the distance labels of the deleted vertices. We get an overall running time of O ((m log m)(m + n log n)). This is the fastest known strongly polynomial algorithm for the general M IN C OST F LOW P ROBLEM. An algorithm which achieves the same running time but works directly on capacitated instances has been described by Vygen [2002]. IMUM

9.6 The Network Simplex Algorithm The M INIMUM C OST F LOW P ROBLEM is a special case of L INEAR P ROGRAM MING . By applying the S IMPLEX A LGORITHM and exploiting the special structure

9.6 The Network Simplex Algorithm

215

we arrive at what is called the N ETWORK S IMPLEX A LGORITHM. To make the connection clear, we first characterize the set of basic solutions (although we will not need this to prove correctness). Definition 9.18. Let (G, u, b, c) be an instance of the M INIMUM C OST F LOW P ROBLEM. A b-flow f in (G, u) is a spanning tree solution if (V (G), {e ∈ E(G) : 0 < f (e) < u(e)}) contains no undirected circuit. Proposition 9.19. An instance of the M INIMUM C OST F LOW P ROBLEM either has an optimum solution which is a spanning tree solution or has no optimum solution at all. Proof: Given an optimum solution f and an undirected circuit C in (V (G), {e ∈ E(G) : 0 < f (e) < u(e)}), we have two directed circuits C 0 and C 00 in G f with the same underlying undirected graph as C. Let be the minimum residual capacity in E(C 0 ) ∪ E(C 00 ). We can obtain two other feasible solutions f 0 and f 00 by augmenting f by along C 0 and C 00 , respectively. As 2c( f ) = c( f 0 ) + c( f 00 ), both f 0 and f 00 are also optimum solutions. At least one of them has fewer edges e with 0 < f (e) < u(e) than f , so after less than |E(G)| steps we arrive at an optimum spanning tree solution. Corollary 9.20. Let (G, u, b, c) be an instance of the M INIMUM C OST F LOW P ROBLEM. Then the basic solutions of x ∈ R E(G) : 0 ≤ xe ≤ u(e) (e ∈ E(G)), X e∈δ + (v)

xe −

X

xe = b(v) (v ∈ V (G))

e∈δ − (v)

are precisely the spanning tree solutions of (G, u, b, c). Proof: Proposition 9.19 shows that each basic solution is a spanning tree solution. For a spanning tree solution f we consider the inequalities xe ≥ 0P for e ∈ E(G) ≤ u(e) for e ∈ E(G) with f (e) = u(e), and with f (e) = 0, x e e∈δ + (v) x e − P x = b(v) for all v except one vertex of each connected component − e e∈δ (v) of (V (G), {e ∈ E(G) : 0 < f (e) < u(e)}). These |E(G)| inequalities are all satisfied by f with equality, and the submatrix corresponding to these inequalities is nonsingular. Thus f is a basic solution. In a spanning tree solution there are three types of edges: those with zero flow, those with saturated capacity, and those whose flow is positive but less than the capacity. Assuming that G is connected, we can extend the last set of edges to a connected spanning subgraph without undirected circuits (i.e., an oriented spanning tree; hence the name “spanning tree solution”). Definition 9.21. Let (G, u, b, c) be an instance of the M INIMUM C OST F LOW P ROBLEM where G is connected. A spanning tree structure is a quadruple

216

9 Minimum Cost Flows .

.

(r, T, L , U ) where r ∈ V (G), E(G) = T ∪ L ∪ U , |T | = |V (G)| − 1, and (V (G), T ) contains no undirected circuit. The b-flow associated with the spanning tree structure (r, T, L , U ) is defined by • • •

f (e) := 0 for e ∈ L, f (e) := u(e) , P P for e ∈ UP f (e) := v∈Ce b(v)+ e∈U ∩δ − (Ce ) u(e)− e∈U ∩δ + (Ce ) u(e) for e ∈ T , where for e = (v, w) we denote by Ce the connected component of (V (G), T \ {e}) containing v.

(r, T, L , U ) is called feasible if 0 ≤ f (e) ≤ u(e) for all e ∈ T . Call an edge (v, w) in T downward if v belongs to the undirected r -w-path in T , otherwise upward. (r, T, L , U ) is called strongly feasible if 0 < f (e) ≤ u(e) for each downward edge e ∈ T and 0 ≤ f (e) < u(e) for each upward edge e ∈ T . The unique function π : V (G) → R with π(r ) = 0 and cπ (e) = 0 for all e ∈ T is called the potential associated with the spanning tree structure (r, T, L , U ). b-flow f associated with a spanning tree structure satisfies P Evidently the P f (e) − + e∈δ (v) e∈δ − (v) f (e) = b(v) for all v ∈ V (G) (though it is not always a feasible b-flow). Moreover we note: Proposition 9.22. Given an instance (G, u, b, c) of the M INIMUM C OST F LOW P ROBLEM and a spanning tree structure (r, T, L , U ), the b-flow f and the potential π associated with it can be computed in O(m) and O(n) time, respectively. Moreover, f is integral whenever b and u are integral, and π is integral whenever c is integral. Proof: The potential associated with (r, T, L , U ) can be computed simply by applying the G RAPH S CANNING A LGORITHM to the edges of T and their reverse edges. The b-flow associated with (r, T, L , U ) can be computed in linear time by scanning the vertices in an order of nonincreasing distance from r . The integrality properties follow immediately from the definition. The N ETWORK S IMPLEX A LGORITHM maintains a strongly feasible spanning tree structure and works towards optimality. Note that the optimality criterion in Corollary 9.7 immediately implies: Proposition 9.23. Let (r, T, L , U ) be a feasible spanning tree structure and let π be the potential associated with it. Suppose that • •

cπ (e) ≥ 0 for all e ∈ L, and cπ (e) ≤ 0 for all e ∈ U .

Then (r, T, L , U ) is associated with an optimum b-flow.

↔

Note that π(v) is the length of the r -v-path in G that contains only edges of T ↔

or their reverse edges. For an edge e = (v, w) ∈ E(G) we define the fundamental

9.6 The Network Simplex Algorithm

217

circuit of e to consist of e and the w-v-path that contains only edges of T and their reverse edges. The vertex of C that is closest to r in T is called its peak. Hence for e = (v, w) ∈ / T , cπ (e) = c(e) + π(v) − π(w) is the cost of sending one unit of flow along the fundamental circuit of e. There are several ways to obtain an initial strongly feasible spanning tree structure. For example, one could compute any b-flow (by solving a M AXIMUM F LOW P ROBLEM), apply the procedure in the proof of Proposition 9.19, choose r arbitrarily, and define T, L , U according to the flow (adding suitable edges to T if necessary). Alternatively, one could apply “phase one” of the S IMPLEX M ETHOD. However, the simplest possibility is to introduce very expensive auxiliary edges with sufficient capacity between r and each other vertex: For each sink v ∈ V (G) \ {r } we introduce an edge (r, v) with capacity −b(v), and for each other vertex v ∈ V (G) \ {r } we introduce an edge (v, r ) with capacity b(v) + 1. The cost of each auxiliary edge should be high enough so that they never occur in an optimum solution, for example 1 + (|V (G)| − 1) maxe∈E(G) |c(e)| (Exercise 19). Then we can choose T as the set of all auxiliary edges, L as the set of all original edges, and U := ∅ to obtain an initial strongly feasible spanning tree structure.

N ETWORK S IMPLEX A LGORITHM Input: Output:

An instance (G, u, b, c) of the M INIMUM C OST F LOW P ROBLEM and a strongly feasible spanning tree structure (r, T, L , U ). An optimum solution f .

1

Compute the b-flow f and the potential π associated with (r, T, L , U ).

2

Let e ∈ L with cπ (e) < 0 or e ∈ U with cπ (e) > 0. If such an edge e does not exist then stop.

3

Let C be the fundamental circuit of e (if e ∈ L) or of e (if e ∈ U ). Let γ := cπ (e).

4

Let δ := mine0 ∈E(C) u f (e0 ), and let e0 be the last edge where the minimum is attained when traversing C in its orientation, starting at its peak. ← Let e0 ∈ E(G) such that e0 is e0 or e0 . Remove e from L or U . Set T := (T ∪ {e}) \ {e0 }. If e0 = e0 then insert e0 into U else insert e0 into L. Augment f by δ along C. Let X be the connected component of (V (G), T \ {e}) containing r . If e ∈ δ + (X ) then set π(v) := π(v) + γ for v ∈ V (G) \ X . If e ∈ δ − (X ) then set π(v) := π(v) − γ for v ∈ V (G) \ X . Go to . 2

5

6

←

We remark that 6 could be replaced simply by going back to , 1 as f and π computed in 6 are associated with the new spanning tree structure. Also note

218

9 Minimum Cost Flows

that e = e0 is possible; in this case X = V (G), and T , f and π do not change, but e moves from L to U or vice versa. Theorem 9.24. (Dantzig [1951], Cunningham [1976]) The N ETWORK S IMPLEX A LGORITHM stops after finitely many iterations and returns an optimum solution. Proof: First observe that 6 maintains the property that f and π are the b-flow and potential associated with (r, T, L , U ). Next we prove that the spanning tree structure is always strongly feasible. By the choice of δ we maintain the condition 0 ≤ f (e) ≤ u(e) for all e, and hence the spanning tree structure remains feasible. As the edges of the subpath of C from the head of e0 to the peak did not attain the minimum in , 4 they will continue to have positive residual capacity after the augmentation. For the edges of the subpath of C from the peak to the tail of e0 we have to make sure that their reverse edges have positive residual capacity after the augmentation. This is clear if δ > 0. Otherwise (if δ = 0) we have e 6= e0 , and the fact that the ← spanning tree structure was strongly feasible before implies that neither e nor e ← can belong to this subpath (i.e. e = e0 or δ − (X ) ∩ E(C) ∩ {e, e } 6= ∅) and that ← the reverse edges of the subpath of C from the peak to tail of e or e had positive residual capacity. By Proposition 9.23 the computed flow f is optimum when the algorithm terminates. We show that there are no two iterations with the same pair ( f, π), and hence each spanning tree structure occurs at most once. In each iteration the cost of the flow is reduced by |γ |δ. As γ 6= 0, we only need to consider iterations with δ = 0. Here the cost of the flow remains P constant. If e 6= e0 , then e ∈ L ∩ δ − (X ) or e ∈ U ∩ δ + (X ), and hence v∈V (G) π(v) strictly increases (by at least |γ |). Finally, if δ = 0 and e = e0 , then u(e) = 0, X = V (G), π remains constant, and |{e ∈ L : cπ (e) < 0}| + |{e ∈ U : cπ (e) > 0}| strictly decreases. This shows that no two iterations feature the same spanning tree structure. Although the N ETWORK S IMPLEX A LGORITHM is not a polynomial-time algorithm, it is quite efficient in practice. Orlin [1997] proposed a variant that runs in polynomial time. Polynomial-time dual network simplex algorithms were found by Orlin, Plotkin and Tardos [1993], and Armstrong and Jin [1997].

9.7 Flows Over Time We now consider flows over time (also sometimes called dynamic flows); i.e. the flow value on each edge may change over time, and flow entering an edge arrives at the endvertex after a specified delay:

9.7 Flows Over Time

219

Definition 9.25. Let (G, u, s, t) be a network with transit times l : E(G) → R+ and a time horizon T ∈ R+ . Then an s-t-flow over time f consists of a Lebesguemeasurable function f e : [0, T ] → R+ for each e ∈ E(G) with f e (τ ) ≤ u(e) for all τ ∈ [0, T ] and e ∈ E(G) and ex f (v, a) :=

X Z e∈δ − (v)

max{0,a−l(e)}

f e (τ )dτ −

0

X Z e∈δ + (v)

a

f e (τ )dτ ≥ 0

(9.7)

0

for all v ∈ V (G) \ {s} and a ∈ [0, T ]. f e (τ ) is called the rate of flow entering e at time τ (and leaving this edge l(e) time units later). (9.7) allows intermediate storage at vertices, like in s-t-preflows. It is natural to maximize the flow arriving at sink t:

M AXIMUM F LOW OVER T IME P ROBLEM Instance: Task:

A network (G, u, s, t). Transit times l : E(G) → R+ and a time horizon T ∈ R+ . Find an s-t-flow over time f such that value ( f ) := ex f (t, T ) is maximum.

Following Ford and Fulkerson [1958], we show that this problem can be reduced to the M INIMUM C OST F LOW P ROBLEM. Theorem 9.26. The M AXIMUM F LOW OVER T IME P ROBLEM can be solved in the same time as the M INIMUM C OST F LOW P ROBLEM. Proof: Given an instance (G, u, s, t, l, T ) as above, define a new edge e0 = (t, s) and G 0 := G + e0 . Set u(e0 ) := u(E(G)), c(e0 ) := −T and c(e) := l(e) for e ∈ E(G). Consider the instance (G 0 , u, 0, c) of the M INIMUM C OST F LOW P ROBLEM. Let f 0 be an optimum solution, i.e. a minimum cost (with respect to c) circulation in (G 0 , u). By Proposition 9.5, f 0 can be decomposed into flows on 0 circuits, i.e. there is Pa set C of circuits in G and positive numbers h : C → R+ 0 such that f (e) = {h(C) : C ∈ C, e ∈ E(C)}. We have c(C) ≤ 0 for all C ∈ C as f 0 is a minimum cost circulation. Let C ∈ C with c(C) < 0. C must contain e0 . For e = (v, w) ∈ E(C) \ {e0 }, let deC be the distance from s to v in (C, c). Set X f e∗ (τ ) := {h(C) : C ∈ C, c(C) < 0, e ∈ E(C), deC ≤ τ ≤ deC − c(C)} for e ∈ E(G) and τ ∈ [0, T ]. This defines an s-t-flow over time without intermediate storage (i.e. ex f (v, a) = 0 for all v ∈ V (G) \ {s, t} and all a ∈ [0, T ]). Moreover, X Z T −l(e) X ∗ value ( f ) = f e∗ (τ )dτ = − c(e) f 0 (e). e∈δ − (t)

0

e∈E(G 0 )

220

9 Minimum Cost Flows

We claim that f ∗ is optimum. To see this, let f be any s-t-flow over time, and set f e (τ ) := 0 for e ∈ E(G) and τ ∈ / [0, T ]. Let π(v) := dist(G 0 0 ,c) (s, v) for f

v ∈ V (G). As G 0f 0 contains no negative circuit (cf. Theorem 9.6), π is a feasible potential in (G 0f 0 , c). We have value ( f ) = ex f (t, T ) ≤

X

ex f (v, π(v))

v∈V (G)

because of (9.7), π(t) = T , π(s) = 0 and 0 ≤ π(v) ≤ T for all v ∈ V (G). Hence ! Z π(v) Z π(w)−l(e) X value ( f ) ≤ f e (τ )dτ − f e (τ )dτ 0

e=(v,w)∈E(G)

X

≤

0

(π(w) − l(e) − π(v))u(e)

e=(v,w)∈E(G):π(w)−l(e)>π(v)

X

=

(π(w) − l(e) − π(v)) f 0 (e)

e=(v,w)∈E(G)

X

=

(π(w) − c(e) − π(v)) f 0 (e)

e=(v,w)∈E(G 0 )

X

= −

c(e) f 0 (e)

e=(v,w)∈E(G 0 ) ∗

= value ( f )

Other flow over time problems are significantly more difficult. Hoppe and Tardos [2000] solved the so-called quickest transshipment problem (with several sources and sinks) with integral transit times using submodular function minimization (see Chapter 14). Finding minimum cost flows over time is NP-hard (Klinz and Woeginger [2004]). See Fleischer and Skutella [2007] for approximation algorithms and more information.

Exercises 1. Show that the M AXIMUM F LOW P ROBLEM can be regarded as a special case of the M INIMUM C OST F LOW P ROBLEM. 2. Let GPbe a digraph with capacities u : E(G) → R+ , and let b : V (G) → R with v∈V (G) b(v) = 0. Prove that there exists a b-flow if and only if X X u(e) ≥ b(v) for all X ⊆ V (G). e∈δ + (X )

(Gale [1957])

v∈X

Exercises

221

3. Let G be a digraph with lower and upper capacities l, u : E(G) → R+ , where l(e) ≤ u(e) for all e ∈ E(G), and let b1 , b2 : V (G) → R with b1 (v) ≤ b2 (v) for all v ∈ V (G). Prove that there exists a flow f with l(e) ≤ f (e) ≤ u(e) for all e ∈ E(G) and X X f (e) − f (e) ≤ b2 (v) for all v ∈ V (G) b1 (v) ≤ e∈δ + (v)

e∈δ − (v)

if and only if X

u(e) ≥ max

e∈δ + (X )

X

b1 (v), −

v∈X

X

b2 (v)

v∈V (G)\X

X

+

l(e)

e∈δ − (X )

for all X ⊆ V (G). (This is a generalization of Exercise 4 of Chapter 8 and Exercise 2 of this chapter.) (Hoffman [1960]) 4. Prove the following theorem of Ore [1956]. Given a digraph G and nonnegative integers a(x), b(x) for each x ∈ V (G), then G has a spanning subgraph H − with |δ + H (x)| = a(x) and |δ H (x)| = b(x) for all x ∈ V (G) if and only if X

a(x) =

x∈V (G)

X x∈X

∗

a(x) ≤

X

X

b(x)

and

x∈V (G)

+ min{b(y), |E G (X, {y})|}

for all X ⊆ V (G).

y∈V (G)

(Ford and Fulkerson [1962]) 5. Consider the M INIMUM C OST F LOW P ROBLEM where infinite capacities (u(e) = ∞ for some edges e) are allowed. (a) Show that an instance is unbounded if and only if it is feasible and there is a negative circuit all whose edges have infinite capacity. (b) Show how to decide in O(n 3 + m) time whether an instance is unbounded. (c) Show that for an instance that is not unbounded each infinite capacity can be equivalently replaced by a finite capacity. 6. Let (G, u, c, b) be an instance of the M INIMUM C OST F LOW P ROBLEM. We call a function π : V (G) → R an optimal potential if there exists a minimum cost b-flow f such that π is a feasible potential with respect to (G f , c). (a) Prove that a function π : V (G) → R is an optimal potential if and only if for all X ⊆ V (G): X X u(e) ≤ u(e). b(X ) + e∈δ − (X ):cπ (e) c(X ) or assert that no such Y exists. Suppose this algorithm has a running time which is polynomial in size(c). Prove that then there is an algorithm for finding a maximum weight set X ∈ F for a given (E, F) ∈ 9 and c : E → Z+ , whose running time is polynomial in size(c). (Gr¨otschel and Lov´asz [1995]; see also Schulz, Weismantel and Ziegler [1995], and Schulz and Weismantel [2002]) 15. Prove that O RLIN ’ S A LGORITHM always computes a spanning tree solution. 16. Prove that in 7 of O RLIN ’ S A LGORITHM one can replace the 8nγ -bound by 5nγ . 17. Consider the shortest path computations with nonnegative weights (using D IJKSTRA ’ S A LGORITHM) in the algorithms of Section 9.4 and 9.5. Show that even for graphs with parallel edges each of these computations can be performed in O(n 2 ) time, provided that we have the incidence list of G sorted by edge costs. Conclude that O RLIN ’ S A LGORITHM runs in O(mn 2 log m) time. ∗ 18. The P USH -R ELABEL A LGORITHM (Section 8.5) can be generalized to the M INIMUM C OST F LOW P ROBLEM. For an instance (G, u, b, c) with integral costs c, we look for a b-flow f and a feasible potential π in (G f , c). We start by setting π := 0 and saturating all edges e with negative cost. Then we apply 3 of the P USH -R ELABEL A LGORITHM with the following modifications. An edge e is admissible if e ∈ E(G f ) and cπ (e) < 0. A vertex v is active if b(v)+ex f (v) > 0. R ELABEL(v) consists of setting π(v) := max{π(w)−c(e)−1 : e = (v, w) ∈ E(G f )}. In P USH(e) for e ∈ δ + (v) we set γ := min{b(v) + ex f (v), u f (e)}. (a) Prove that the number of R ELABEL operations is O(n 2 |cmax |), where cmax = maxe∈E(G) c(e). Hint: Some vertex w with b(w) + ex f (w) < 0 must be reachable in G f from any active vertex v. Note that b(w) has never changed and recall the proofs of Lemmata 8.22 and 8.24. (b) Show that the overall running time is O(n 2 mcmax ). (c) Prove that the algorithm computes an optimum solution. (d) Apply scaling to obtain an O(n 2 m log cmax )-algorithm for the M INIMUM C OST F LOW P ROBLEM with integral costs c. (Goldberg and Tarjan [1990]) 19. Let (G, u, c, b) be an instance of the M INIMUM C OST F LOW P ROBLEM. Let e¯ ∈ E(G) with c(e) ¯ > (|V (G)| − 1) maxe∈E(G)\{e} ¯ |c(e)|. Prove: If there exists a b-flow f in (G, u) with f (e) ¯ = 0, then f (e) ¯ = 0 holds for every optimum solution f . 20. Given a network (G, u, s, t) with integral transit times l : E(G) → Z+ , a time horizon T ∈ N, a value V ∈ R+ , and costs c : E(G) → R+ . We look for an s-t-flow over time f with value ( f ) = V and minimum cost

224

9 Minimum Cost Flows

RT e∈E(G) c(e) 0

f e (τ )dτ . Show how to solve this in polynomial time if T is a constant. Hint: Consider a time-expanded network with a copy of G for each discrete time step.

P

References General Literature: Ahuja, R.K., Magnanti, T.L., and Orlin, J.B. [1993]: Network Flows. Prentice-Hall, Englewood Cliffs 1993 Cook, W.J., Cunningham, W.H., Pulleyblank, W.R., and Schrijver, A. [1998]: Combinatorial Optimization. Wiley, New York 1998, Chapter 4 ´ and Tarjan, R.E. [1990]: Network flow algorithms. In: Paths, Goldberg, A.V., Tardos, E., Flows, and VLSI-Layout (B. Korte, L. Lov´asz, H.J. Pr¨omel, A. Schrijver, eds.), Springer, Berlin 1990, pp. 101–164 Gondran, M., and Minoux, M. [1984]: Graphs and Algorithms. Wiley, Chichester 1984, Chapter 5 Jungnickel, D. [2007]: Graphs, Networks and Algorithms. Third Edition. Springer, Berlin 2007, Chapters 10 and 11 Lawler, E.L. [1976]: Combinatorial Optimization: Networks and Matroids. Holt, Rinehart and Winston, New York 1976, Chapter 4 Ruhe, G. [1991]: Algorithmic Aspects of Flows in Networks. Kluwer Academic Publishers, Dordrecht 1991

Cited References: Arkin, E.M., and Silverberg, E.B. [1987]: Scheduling jobs with fixed start and end times. Discrete Applied Mathematics 18 (1987), 1–8 Armstrong, R.D., and Jin, Z. [1997]: A new strongly polynomial dual network simplex algorithm. Mathematical Programming 78 (1997), 131–148 Busacker, R.G., and Gowen, P.J. [1961]: A procedure for determining a family of minimumcost network flow patterns. ORO Technical Paper 15, Operational Research Office, Johns Hopkins University, Baltimore 1961 Cunningham, W.H. [1976]: A network simplex method. Mathematical Programming 11 (1976), 105–116 Dantzig, G.B. [1951]: Application of the simplex method to a transportation problem. In: Activity Analysis and Production and Allocation (T.C. Koopmans, Ed.), Wiley, New York 1951, pp. 359–373 Edmonds, J., and Karp, R.M. [1972]: Theoretical improvements in algorithmic efficiency for network flow problems. Journal of the ACM 19 (1972), 248–264 Fleischer, L., and Skutella, M. [2007]: Quickest flows over time. SIAM Journal on Computing 36 (2007), 1600–1630 Ford, L.R., and Fulkerson, D.R. [1958]: Constructing maximal dynamic flows from static flows. Operations Research 6 (1958), 419–433 Ford, L.R., and Fulkerson, D.R. [1962]: Flows in Networks. Princeton University Press, Princeton 1962

References

225

Gale, D. [1957]: A theorem on flows in networks. Pacific Journal of Mathematics 7 (1957), 1073–1082 Goldberg, A.V., and Tarjan, R.E. [1989]: Finding minimum-cost circulations by cancelling negative cycles. Journal of the ACM 36 (1989), 873–886 Goldberg, A.V., and Tarjan, R.E. [1990]: Finding minimum-cost circulations by successive approximation. Mathematics of Operations Research 15 (1990), 430–466 Gr¨otschel, M., and Lov´asz, L. [1995]: Combinatorial optimization. In: Handbook of Combinatorics; Vol. 2 (R.L. Graham, M. Gr¨otschel, L. Lov´asz, eds.), Elsevier, Amsterdam 1995 Hassin, R. [1983]: The minimum cost flow problem: a unifying approach to dual algorithms and a new tree-search algorithm. Mathematical Programming 25 (1983), 228–239 Hitchcock, F.L. [1941]: The distribution of a product from several sources to numerous localities. Journal of Mathematical Physics 20 (1941), 224–230 Hoffman, A.J. [1960]: Some recent applications of the theory of linear inequalities to extremal combinatorial analysis. In: Combinatorial Analysis (R.E. Bellman, M. Hall, eds.), AMS, Providence 1960, pp. 113–128 ´ [2000]: The quickest transshipment problem. Mathematics of Hoppe, B., and Tardos, E. Operations Research 25 (2000), 36–62 Iri, M. [1960]: A new method for solving transportation-network problems. Journal of the Operations Research Society of Japan 3 (1960), 27–87 Jewell, W.S. [1958]: Optimal flow through networks. Interim Technical Report 8, MIT 1958 Klein, M. [1967]: A primal method for minimum cost flows, with applications to the assignment and transportation problems. Management Science 14 (1967), 205–220 Klinz, B., and Woeginger, G.J. [2004]: Minimum cost dynamic flows: the series-parallel case. Networks 43 (2004), 153–162 Orden, A. [1956]: The transshipment problem. Management Science 2 (1956), 276–285 Ore, O. [1956]: Studies on directed graphs I. Annals of Mathematics 63 (1956), 383–406 Orlin, J.B. [1993]: A faster strongly polynomial minimum cost flow algorithm. Operations Research 41 (1993), 338–350 Orlin, J.B. [1997]: A polynomial time primal network simplex algorithm for minimum cost flows. Mathematical Programming 78 (1997), 109–129 ´ [1993]: Polynomial dual network simplex algoOrlin, J.B., Plotkin, S.A., and Tardos, E. rithms. Mathematical Programming 60 (1993), 255–276 ´ [1990]: Improved dual network simplex. Proceedings of the Plotkin, S.A., and Tardos, E. 1st Annual ACM-SIAM Symposium on Discrete Algorithms (1990), 367–376 Schulz, A.S., Weismantel, R., and Ziegler, G.M. [1995]: 0/1-Integer Programming: optimization and augmentation are equivalent. In: Algorithms – ESA ’95; LNCS 979 (P. Spirakis, ed.), Springer, Berlin 1995, pp. 473–483 Schulz, A.S., and Weismantel, R. [2002]: The complexity of generic primal algorithms for solving general integer problems. Mathematics of Operations Research 27 (2002), 681–192 ´ [1985]: A strongly polynomial minimum cost circulation algorithm. CombinaTardos, E. torica 5 (1985), 247–255 Tolsto˘ı, A.N. [1930]: Metody nakhozhdeniya naimen’shego summovogo kilometrazha pri planirovanii perevozok v prostanstve. In: Planirovanie Perevozok, Sbornik pervy˘ı, Transpechat’ NKPS, Moskow 1930, pp. 23–55. (See A. Schrijver, On the history of the transportation and maximum flow problems, Mathematical Programming 91 (2002), 437–445)

226

9 Minimum Cost Flows

Tomizawa, N. [1971]: On some techniques useful for solution of transportation network problems. Networks 1 (1971), 173–194 Vygen, J. [2002]: On dual minimum cost flow algorithms. Mathematical Methods of Operations Research 56 (2002), 101–126 Wagner, H.M. [1959]: On a class of capacitated transportation problems. Management Science 5 (1959), 304–318

10 Maximum Matchings

Matching theory is one of the classical and most important topics in combinatorial theory and optimization. All the graphs in this chapter are undirected. Recall that a matching is a set of pairwise disjoint edges. Our main problem is:

C ARDINALITY M ATCHING P ROBLEM Instance:

An undirected graph G.

Task:

Find a maximum cardinality matching in G.

Since the weighted version of this problem is significantly more difficult we postpone it to Chapter 11. But already the above cardinality version has applications: Suppose in the J OB A SSIGNMENT P ROBLEM each job has the same processing time, say one hour, and we ask whether we can finish all the jobs within one hour. . In other words: given a bipartite graph GP with bipartition V (G) = A ∪ B, we look P for numbers x : E(G) → R+ with e∈δ(a) x(e) = 1 for each job a ∈ A and e∈δ(b) x(e) ≤ 1 for each employee b ∈ B. We can write this as a linear inequality system x ≥ 0, M x ≤ 1l, M 0 x ≥ 1l, where the rows of M and M 0 are rows of the node-edge incidence matrix of G. These matrices are totally unimodular by Theorem 5.25. From Theorem 5.20 we conclude that if there is any solution x, then there is also an integral solution. Now observe that the integral solutions to the above linear inequality system are precisely the incidence vectors of the matchings in G covering A. Definition 10.1. Let G be a graph and M a matching in G. We say that a vertex v is covered by M if v ∈ e for some e ∈ M. M is called a perfect matching if all vertices are covered by M. In Section 10.1 we consider matchings in bipartite graphs. Algorithmically this problem can be reduced to the M AXIMUM F LOW P ROBLEM. The Max-Flow-MinCut Theorem as well as the concept of augmenting paths have nice interpretations in our context. Matching in general, non-bipartite graphs does not reduce directly to network flows. We introduce two necessary and sufficient conditions for a general graph to have a perfect matching in Sections 10.2 and 10.3. In Section 10.4 we consider factor-critical graphs which have a matching covering all vertices but v, for each v ∈ V (G). These play an important role in Edmonds’ algorithm for the C ARDINALITY

228

10 Maximum Matchings

M ATCHING P ROBLEM, described in Section 10.5, and its weighted version which we postpone to Sections 11.2 and 11.3.

10.1 Bipartite Matching Since the C ARDINALITY M ATCHING P ROBLEM is easier if G is bipartite, we shall deal with this case first. In this section, a bipartite graph G is always assumed to . have the bipartition V (G) = A ∪ B. Since we may assume that G is connected, we can regard this bipartition as unique (Exercise 20 of Chapter 2). For a graph G, let ν(G) denote the maximum cardinality of a matching in G, while τ (G) is the minimum cardinality of a vertex cover in G. Theorem 10.2. (K¨onig [1931]) If G is bipartite, then ν(G) = τ (G). .

Proof: Consider the graph G 0 = (V (G) ∪ {s, t}, E(G) ∪ {{s, a} : a ∈ A} ∪ {{b, t} : b ∈ B}). Then ν(G) is the maximum number of vertex-disjoint s-t-paths, while τ (G) is the minimum number of vertices whose deletion makes t unreachable from s. The theorem now immediately follows from Menger’s Theorem 8.10. ν(G) ≤ τ (G) evidently holds for any graph (bipartite or not), but we do not have equality in general (as the triangle K 3 shows). Several statements are equivalent to K¨onig’s Theorem. Hall’s Theorem is probably the best-known version. Theorem 10.3. (Hall [1935]) Let G be a bipartite graph with bipartition V (G) = . A ∪ B. Then G has a matching covering A if and only if |0(X )| ≥ |X |

for all X ⊆ A.

(10.1)

Proof: The necessity of the condition is obvious. To prove the sufficiency, assume that G has no matching covering A, i.e. ν(G) < |A|. By Theorem 10.2 this implies τ (G) < |A|. Let A0 ⊆ A, B 0 ⊆ B such that A0 ∪ B 0 covers all the edges and |A0 ∪ B 0 | < |A|. Obviously 0(A \ A0 ) ⊆ B 0 . Therefore |0(A \ A0 )| ≤ |B 0 | < |A| − |A0 | = |A \ A0 |, and the Hall condition (10.1) is violated. It is worthwhile to mention that it is not too difficult to prove Hall’s Theorem directly. The following proof is due to Halmos and Vaughan [1950]: Second Proof of Theorem 10.3: We show that any G satisfying the Hall condition (10.1) has a matching covering A. We use induction on |A|, the cases |A| = 0 and |A| = 1 being trivial. If |A| ≥ 2, we consider two cases: If |0(X )| > |X | for every nonempty proper subset X of A, then we take any edge {a, b} (a ∈ A, b ∈ B), delete its two vertices and apply induction. The smaller graph satisfies the Hall condition because |0(X )| − |X | can have decreased by at most one for any X ⊆ A \ {a}.

10.1 Bipartite Matching

229

Now assume that there is a nonempty proper subset X of A with |0(X )| = |X |. By induction there is a matching covering X in G[X ∪ 0(X )]. We claim that we can extend this to a matching in G covering A. Again by the induction hypothesis, we have to show that G[(A \ X ) ∪ (B \ 0(X ))] satisfies the Hall condition. To check this, observe that for any Y ⊆ A \ X we have (in the original graph G): |0(Y ) \ 0(X )| = |0(X ∪ Y )| − |0(X )| ≥ |X ∪ Y | − |X | = |Y |. A special case of Hall’s Theorem is the so-called “Marriage Theorem”: Theorem 10.4. (Frobenius [1917]) Let G be a bipartite graph with bipartition . V (G) = A ∪ B. Then G has a perfect matching if and only if |A| = |B| and |0(X )| ≥ |X | for all X ⊆ A. The variety of applications of Hall’s Theorem is indicated by Exercises 4–7. The proof of K¨onig’s Theorem 10.2 shows how to solve the bipartite matching problem algorithmically: Theorem 10.5. The C ARDINALITY M ATCHING P ROBLEM for bipartite graphs G can be solved in O(nm) time, where n = |V (G)| and m = |E(G)|. .

Proof: Let G be a bipartite graph with bipartition V (G) = A ∪ B. Add a vertex s and connect it to all vertices of A, and add another vertex t connected to all vertices of B. Orient the edges from s to A, from A to B, and from B to t. Let the capacities be 1 everywhere. Then a maximum integral s-t-flow corresponds to a maximum cardinality matching (and vice versa). So we apply the F ORD -F ULKERSON A LGORITHM and find a maximum s-tflow (and thus a maximum matching) after at most n augmentations. Since each augmentation takes O(m) time, we are done. This result is essentially due to Kuhn [1955]. In fact, one can use the concept of shortest augmenting paths√again (cf. the E DMONDS -K ARP A LGORITHM). In this way one obtains the O n(m + n) -algorithm of Hopcroft and Karp [1973]. This algorithm will be discussed in Exercises 9 and 10. Slightimprovements of q mn the H OPCROFT -K ARP A LGORITHM yield running times of O n log n (Alt et √ 2 /m) al. [1991]) and O m n log(n (Feder and Motwani [1995]). The latter bound log n is the best known for dense graphs. Let us reformulate the augmenting path concept in our context. Definition 10.6. Let G be a graph (bipartite or not), and let M be some matching in G. A path P is an M-alternating path if E(P) \ M is a matching. An Malternating path is M-augmenting if its endpoints are not covered by M. One immediately checks that M-augmenting paths must have odd length.

230

10 Maximum Matchings

Theorem 10.7. (Berge [1957]) Let G be a graph (bipartite or not) with some matching M. Then M is maximum if and only if there is no M-augmenting path. Proof: If there is an M-augmenting path P, the symmetric difference M4E(P) is a matching and has greater cardinality than M, so M is not maximum. On the other hand, if there is a matching M 0 such that |M 0 | > |M|, the symmetric difference M4M 0 is the vertex-disjoint union of alternating circuits and paths, where at least one path must be M-augmenting. In the bipartite case Berge’s Theorem of course also follows from Theorem 8.5.

10.2 The Tutte Matrix We now consider maximum matchings from an algebraic point of view. Let G be a simple undirected graph, and let G 0 be the directed graph resulting from G by arbitrarily orienting the edges. For any vector x = (xe )e∈E(G) of variables, we define the Tutte matrix x )v,w∈V (G) TG (x) = (tvw

by x tvw

x{v,w} := −x{v,w} 0

if (v, w) ∈ E(G 0 ) if (w, v) ∈ E(G 0 ) . otherwise

(Such a matrix M, where M = −M > , is called skew-symmetric.) det TG (x) is a polynomial in the variables xe (e ∈ E(G)). Theorem 10.8. (Tutte [1947]) G has a perfect matching if and only if det TG (x) is not identically zero. Proof: Let V (G) = {v1 , . . . , vn }, and let Sn be the set of all permutations on {1, . . . , n}. By definition of the determinant, det TG (x) =

X π ∈Sn

sgn(π)

n Y

tvxi ,vπ(i) .

i=1

n o Qn x tvi ,vπ(i) 6= 0 . Each permutation π ∈ Sn corresponds Let Sn0 := π ∈ Sn : i=1 to a directed graph Hπ := (V (G), {(vi , vπ(i) ) : i = 1, . . . , n}) where each vertex ↔

+ 0 0 x has |δ − Hπ (x)| = |δ Hπ (x)| = 1. For permutations π ∈ Sn , Hπ is a subgraph of G . 0 If there exists a permutation π ∈ Sn such that Hπ consists of even circuits only, then by taking every second edge of each circuit (and ignoring the orientations) we obtain a perfect matching in G.

10.2 The Tutte Matrix

231

Otherwise, for each π ∈ Sn0 there is a permutation r (π) ∈ Sn0 such that Hr (π ) is obtained by reversing the first odd circuit in Hπ , i.e. the odd circuit containing the vertex with minimum index. Of course r (r (π)) = π . Observe that sgn(π ) = sgn(r (π)), i.e. the two permutations have the same sign: if the first odd circuit consists of the vertices w1 , . . . , w2k+1 with π(wi ) = wi+1 (i = 1, . . . , 2k) and π(w2k+1 ) = w1 , then we obtain r (π ) by 2k transpositions: for with π(w2k ) and then π(w2 j ) with π(w2k+1 ). j = 1, . . . , k exchange Qn x π(w2 j−1 )Q n tvi ,vπ(i) = − i=1 tvxi ,vr (π )(i) . So the two corresponding terms Moreover, i=1 in the sum n Y X sgn(π) tvxi ,vπ(i) det TG (x) = π ∈Sn0

i=1

cancel each other. Since this holds for all pairs π, r (π ) ∈ Sn0 , we conclude that det TG (x) is identically zero. So if G has no perfect matching, det TG (x) is identically zero. On the other hand, if G has a perfect matching M, consider the permutation definedQby π(i) := j n tvxi ,vπ(i) = and π( j) := i for all {vi , v j } ∈ M. The corresponding term i=1 Q 2 e∈M −x e cannot cancel out with any other term, so det TG (x) is not identically zero. Originally, Tutte used Theorem 10.8 to prove his main theorem on matchings, Theorem 10.13. Theorem 10.8 does not provide a good characterization of the property that a graph has a perfect matching. The problem is that the determinant is easy to compute if the entries are numbers (Theorem 4.10) but difficult to compute if the entries are variables. However, the theorem suggests a randomized algorithm for the C ARDINALITY M ATCHING P ROBLEM: Corollary 10.9. (Lov´asz [1979]) Let x = (xe )e∈E(G) be a random vector where each coordinate is equally distributed in [0, 1]. Then with probability 1 the rank of TG (x) is exactly twice the size of a maximum matching. Proof: Suppose the rank of TG (x) is k, say the first k rows are linearly independent. Since TG (x) is skew-symmetric, also the first k columns are linearly independent. So the principal submatrix (tvxi ,v j )1≤i, j≤k is nonsingular, and by Theorem 10.8 the subgraph G[{v1 , . . . , vk }] has a perfect matching. In particular, k is even and G has a matching of cardinality k2 . On the other hand, if G has a matching of cardinality k, the determinant of the principal submatrix T 0 whose rows and columns correspond to the 2k vertices covered by M is not identically zero by Theorem 10.8. The set of vectors x for which det T 0 (x) = 0 must then have measure zero. So with probability one, the rank of TG (x) is at least 2k. Of course it is not possible to choose random numbers from [0, 1] with a digital computer. However, it can be shown that it suffices to choose random integers from the finite set {1, 2, . . . , N }. For sufficiently large N , the probability of error will become arbitrarily small (see Lov´asz [1979]). Lov´asz’ algorithm

232

10 Maximum Matchings

can be used to determine a maximum matching (not only its cardinality). See Rabin and Vazirani [1989], Mulmuley, Vazirani and Vazirani [1987], and Mucha and Sankowski [2004] for further randomized algorithms for ﬁnding a maximum matching in a graph. Moreover we note that Geelen [2000] has shown how to derandomize Lov´asz’ algorithm. Although its running time is worse than that of Edmonds’ matching algorithm (see Section 10.5), it is important for some generalizations of the C ARDINALITY M ATCHING P ROBLEM (e.g., see Geelen and Iwata [2005]).

10.3 Tutte’s Theorem We now consider the C ARDINALITY M ATCHING P ROBLEM in general graphs. A necessary condition for a graph to have a perfect matching is that every connected component is even (i.e. has an even number of vertices). This condition is not sufﬁcient, as the graph K 1,3 (Figure 10.1(a)) shows. (a)

(b)

Fig. 10.1.

The reason that K 1,3 has no perfect matching is that there is one vertex (the black one) whose deletion produces three odd connected components. The graph shown in Figure 10.1(b) is more complicated. Does this graph have a perfect matching? If we delete the three black vertices, we get ﬁve odd connected components (and one even connected component). If there were a perfect matching, at least one vertex of each odd connected component would have to be connected to one of the black vertices. This is impossible because the number of odd connected components exceeds the number of black vertices. More generally, for X ⊆ V (G) let qG (X ) denote the number of odd connected components in G − X . Then a graph for which qG (X ) > |X | holds for some X ⊆ V (G) cannot have a perfect matching: otherwise there must be, for each

10.3 Tutte’s Theorem

233

odd connected component in G − X , at least one matching edge connecting this connected component with X , which is impossible if there are more odd connected components than elements of X . Tutte’s Theorem says that the above necessary condition is also sufficient: Definition 10.10. A graph G satisfies the Tutte condition if qG (X ) ≤ |X | for all X ⊆ V (G). A nonempty vertex set X ⊆ V (G) is a barrier if qG (X ) = |X |. To prove the sufficiency of the Tutte condition we shall need an easy observation and an important definition: Proposition 10.11. For any graph G and any X ⊆ V (G) we have qG (X ) − |X | ≡ |V (G)| (mod 2). Definition 10.12. A graph G is called factor-critical if G − v has a perfect matching for each v ∈ V (G). A matching is called near-perfect if it covers all vertices but one. Now we can prove Tutte’s Theorem: Theorem 10.13. (Tutte [1947]) A graph G has a perfect matching if and only if it satisfies the Tutte condition: qG (X ) ≤ |X |

for all X ⊆ V (G).

Proof: We have already seen the necessity of the Tutte condition. We now prove the sufficiency by induction on |V (G)| (the case |V (G)| ≤ 2 being trivial). Let G be a graph satisfying the Tutte condition. |V (G)| cannot be odd since otherwise the Tutte condition is violated because qG (∅) ≥ 1. So by Proposition 10.11, |X | − qG (X ) must be even for every X ⊆ V (G). Since |V (G)| is even and the Tutte condition holds, every singleton is a barrier. We choose a maximal barrier X . G − X has |X | odd connected components. G − X cannot have any even connected components because otherwise X ∪ {v}, where v is a vertex of some even connected component, is a barrier (G − (X ∪ {v}) has |X | + 1 odd connected components), contradicting the maximality of X . We now claim that each odd connected component of G − X is factor-critical. To prove this, let C be some odd connected component of G − X and v ∈ V (C). If C − v has no perfect matching, by the induction hypothesis there is some Y ⊆ V (C) \ {v} such that qC−v (Y ) > |Y |. By Proposition 10.11, qC−v (Y ) − |Y | must be even, so qC−v (Y ) ≥ |Y | + 2. Since X, Y and {v} are pairwise disjoint, we have

234

10 Maximum Matchings

qG (X ∪ Y ∪ {v}) = qG (X ) − 1 + qC (Y ∪ {v}) = |X | − 1 + qC−v (Y ) ≥ |X | − 1 + |Y | + 2 = |X ∪ Y ∪ {v}|. So X ∪ Y ∪ {v} is a barrier, contradicting the maximality of X . . We now consider the bipartite graph G 0 with bipartition V (G 0 ) = X ∪ Z which arises when we delete edges with both ends in X and contract the odd connected components of G − X to single vertices (forming the set Z ). It remains to show that G 0 has a perfect matching. If not, then by Frobenius’ Theorem 10.4 there is some A ⊆ Z such that |0G 0 (A)| < |A|. This implies qG (0G 0 (A)) ≥ |A| > |0G 0 (A)|, a contradiction. This proof is due to Anderson [1971]. The Tutte condition provides a good characterization of the perfect matching problem: either a graph has a perfect matching or it has a so-called Tutte set X proving that it has no perfect matching. An important consequence of Tutte’s Theorem is the so-called Berge-Tutte formula: Theorem 10.14. (Berge [1958]) 2ν(G) + max (qG (X ) − |X |) = |V (G)|. X ⊆V (G)

Proof: For any X ⊆ V (G), any matching must leave at least qG (X )−|X | vertices uncovered. Therefore 2ν(G) + qG (X ) − |X | ≤ |V (G)|. To prove the reverse inequality, let k :=

max (qG (X ) − |X |).

X ⊆V (G)

We construct a new graph H by adding k new vertices to G, each of which is connected to all the old vertices. If we can prove that H has a perfect matching, then 2ν(G) + k ≥ 2ν(H ) − k = |V (H )| − k = |V (G)|, and the theorem is proved. Suppose H has no perfect matching, then by Tutte’s Theorem there is a set Y ⊆ V (H ) such that q H (Y ) > |Y |. By Proposition 10.11, k has the same parity as |V (G)|, implying that |V (H )| is even. Therefore Y 6= ∅ and thus q H (Y ) > 1. But then Y contains all the new vertices, so qG (Y ∩ V (G)) = q H (Y ) > |Y | = |Y ∩ V (G)| + k, contradicting the definition of k. Let us close this section with a proposition for later use.

10.4 Ear-Decompositions of Factor-Critical Graphs

235

Proposition 10.15. Let G be a graph and X ⊆ V (G) with |V (G)| − 2ν(G) = qG (X ) − |X |. Then any maximum matching of G contains a perfect matching in each even connected component of G − X , a near-perfect matching in each odd connected component of G − X , and matches all the vertices in X to vertices of distinct odd connected components of G − X . Later we shall see (Theorem 10.32) that X can be chosen such that each odd connected component of G − X is factor-critical.

10.4 Ear-Decompositions of Factor-Critical Graphs This section contains some results on factor-critical graphs which we shall need later. In Exercise 17 of Chapter 2 we have seen that the graphs having an eardecomposition are exactly the 2-edge-connected graphs. Here we are interested in odd ear-decompositions only. Definition 10.16. An ear-decomposition is called odd if every ear has odd length. Theorem 10.17. (Lov´asz [1972]) A graph is factor-critical if and only if it has an odd ear-decomposition. Furthermore, the initial vertex of the ear-decomposition can be chosen arbitrarily. Proof: Let G be a graph with a fixed odd ear-decomposition. We prove that G is factor-critical by induction on the number of ears. Let P be the last ear in the odd ear-decomposition, say P goes from x to y, and let G 0 be the graph before adding P. We have to show for any vertex v ∈ V (G) that G − v contains a perfect matching. If v is not an inner vertex of P this is clear by induction (add every second edge of P to the perfect matching in G 0 − v). If v is an inner vertex of P, then exactly one of P[v,x] and P[v,y] must be even, say P[v,x] . By induction there is a perfect matching in G 0 − x. By adding every second edge of P[y,v] and of P[v,x] we obtain a perfect matching in G − v. We now prove the reverse direction. Choose the initial vertex z of the eardecomposition arbitrarily, and let M be a near-perfect matching in G covering V (G) \ {z}. Suppose we already have an odd ear-decomposition of a subgraph G 0 of G such that z ∈ V (G 0 ) and M ∩ E(G 0 ) is a near-perfect matching in G 0 . If G = G 0 , we are done. If not, then – since G is connected – there must be an edge e = {x, y} ∈ E(G) \ E(G 0 ) with x ∈ V (G 0 ). If y ∈ V (G 0 ), e is the next ear. Otherwise let N be a near-perfect matching in G covering V (G) \ {y}. M4N obviously contains the edges of a y-z-path P. Let w be the first vertex of P (when traversed from y) that belongs to V (G 0 ). The last edge of P 0 := P[y,w] cannot belong to M (because no edge of M leaves V (G 0 )), and the first edge cannot belong to N . Since P 0 is M-N -alternating, |E(P 0 )| must be even, so together with e it forms the next ear. In fact, we have constructed a special type of odd ear-decomposition:

236

10 Maximum Matchings

Definition 10.18. Given a factor-critical graph G and a near-perfect matching M, an M-alternating ear-decomposition of G is an odd ear-decomposition such that each ear is an M-alternating path or a circuit C with |E(C)∩ M|+1 = |E(C)\ M|. It is clear that the initial vertex of an M-alternating ear-decomposition must be the vertex not covered by M. The proof of Theorem 10.17 immediately yields: Corollary 10.19. For any factor-critical graph G and any near-perfect matching M in G there exists an M-alternating ear-decomposition. From now on, we shall only be interested in M-alternating ear-decompositions. An interesting way to store an M-alternating ear-decomposition efficiently is due to Lov´asz and Plummer [1986]: Definition 10.20. Let G be a factor-critical graph and M a near-perfect matching in G. Let r, P1 , . . . , Pk be an M-alternating ear-decomposition of G and µ, ϕ : V (G) → V (G) two functions. We say that µ and ϕ are associated with the ear-decomposition r, P1 , . . . , Pk if • µ(x) = y if {x, y} ∈ M, • ϕ(x) = y if {x, y} ∈ E(Pi ) \ M and x ∈ / {r } ∪ V (P1 ) ∪ · · · ∪ V (Pi−1 ), • µ(r ) = ϕ(r ) = r . If M is fixed, we also say that ϕ is associated with r, P1 , . . . , Pk . If M is some fixed near-perfect matching and µ, ϕ are associated with two M-alternating ear-decompositions, they are the same up to the order of the ears. Moreover, an explicit list of the ears can be obtained in linear time:

E AR -D ECOMPOSITION A LGORITHM Input: Output:

1

2

3

A factor-critical graph G, functions µ, ϕ associated with an Malternating ear-decomposition. An M-alternating ear-decomposition r, P1 , . . . , Pk .

Let initially be X := {r }, where r is the vertex with µ(r ) = r . Let k := 0, and let the stack be empty. If X = V (G) then go to . 5 If the stack is nonempty then let v ∈ V (G) \ X be an endpoint of the topmost element of the stack, else choose v ∈ V (G) \ X arbitrarily. Set x := v, y := µ(v) and P := ({x, y}, {{x, y}}). While ϕ(ϕ(x)) = x do: Set P := P + {x, ϕ(x)} + {ϕ(x), µ(ϕ(x))} and x := µ(ϕ(x)). While ϕ(ϕ(y)) = y do: Set P := P + {y, ϕ(y)} + {ϕ(y), µ(ϕ(y))} and y := µ(ϕ(y)). Set P := P + {x, ϕ(x)} + {y, ϕ(y)}. P is the ear containing y as an inner vertex. Put P on top of the stack.

10.4 Ear-Decompositions of Factor-Critical Graphs

4

5

237

While both endpoints of the topmost element P of the stack are in X do: Delete P from the stack, set k := k +1, Pk := P and X := X ∪ V (P). Go to . 2 For all {y, z} ∈ E(G) \ (E(P1 ) ∪ · · · ∪ E(Pk )) do: Set k := k + 1 and Pk := ({y, z}, {{y, z}}).

Proposition 10.21. Let G be a factor-critical graph and µ, ϕ functions associated with an M-alternating ear-decomposition. Then this ear-decomposition is unique up to the order of the ears. The E AR -D ECOMPOSITION A LGORITHM correctly determines an explicit list of these ears; it runs in linear time. Proof: Let D be an M-alternating ear-decomposition associated with µ and ϕ. The uniqueness of D as well as the correctness of the algorithm follows from the obvious fact that P as computed in 3 is indeed an ear of D. The running time of 1 – 4 is evidently O(|V (G)|), while 5 takes O(|E(G)|) time. The most important property of the functions associated with an alternating ear-decomposition is the following: Lemma 10.22. Let G be a factor-critical graph and µ, ϕ two functions associated with an M-alternating ear-decomposition. Let r be the vertex not covered by M. Then the maximal path given by an initial subsequence of x, µ(x), ϕ(µ(x)), µ(ϕ(µ(x))), ϕ(µ(ϕ(µ(x)))), . . . defines an M-alternating x-r -path of even length for all x ∈ V (G). Proof: Let x ∈ V (G) \ {r }, and let Pi be the first ear containing x. Clearly some initial subsequence of x, µ(x), ϕ(µ(x)), µ(ϕ(µ(x))), ϕ(µ(ϕ(µ(x)))), . . . must be a subpath Q of Pi from x to y, where y ∈ {r } ∪ V (P1 ) ∪ · · · ∪ V (Pi−1 ). Because we have an M-alternating ear-decomposition, the last edge of Q does not belong to M; hence Q has even length. If y = r , we are done, otherwise we apply induction on i. The converse of Lemma 10.22 is not true: In the counterexample in Figure 10.2 (bold edges are matching edges, edges directed from u to v indicate ϕ(u) = v), µ and ϕ also define alternating paths to the vertex not covered by the matching. However, µ and ϕ are not associated with any alternating ear-decomposition. For the W EIGHTED M ATCHING A LGORITHM (Section 11.3) we shall need a fast routine for updating an alternating ear-decomposition when the matching changes. Although the proof of Theorem 10.17 is algorithmic (provided that we can find a maximum matching in a graph), this is far too inefficient. We make use of the old ear-decomposition:

238

10 Maximum Matchings

Fig. 10.2.

Lemma 10.23. Given a factor-critical graph G, two near-perfect matchings M and M , and functions µ, ϕ associated with an M-alternating ear-decomposition. Then functions µ , ϕ associated with an M -alternating ear-decomposition can be found in O(|V (G)|) time. Proof: Let v be the vertex not covered by M, and let v be the vertex not covered by M . Let P be the v -v-path in MM , say P = x0 , x1 , . . . , xk with x0 = v and xk = v.

x p+1 xp

e

v X M v

M

P

Fig. 10.3.

An explicit list of the ears of the old ear-decomposition can be obtained from µ and ϕ by the E AR -D ECOMPOSITION A LGORITHM in linear time (Proposition 10.21). Indeed, since we do not have to consider ears of length one, we can omit : 5 then the total number of edges considered is at most 32 (|V (G)|−1) (cf. Exercise 19).

10.4 Ear-Decompositions of Factor-Critical Graphs

239

Suppose we have already constructed an M -alternating ear-decomposition of a spanning subgraph of G[X ] for some X ⊆ V (G) with v ∈ X (initially X := {v }). Of course no M -edge leaves X . Let p := max{i ∈ {0, . . . , k} : xi ∈ X } (illustrated in Figure 10.3). At each stage we keep track of p and of the edge set δ(X ) ∩ M. Their update when extending X is clearly possible in linear total time. Now we show how to extend the ear-decomposition. We shall add one or more ears in each step. The time needed for each step will be proportional to the total number of edges in new ears. Case 1: |δ(X ) ∩ M| ≥ 2. Let f ∈ δ(X ) ∩ M with x p ∈ / f . Evidently, f belongs to an M-M -alternating path which can be added as the next ear. The time needed to ﬁnd this ear is proportional to its length. Case 2: |δ(X ) ∩ M| = 1. Then v ∈ / X , and e = {x p , x p+1 } is the only edge in δ(X ) ∩ M. Let R be the x p+1 -v-path determined by µ and ϕ (cf. Lemma 10.22). The ﬁrst edge of R is e. Let q be the minimum index i ∈ {p + 2, p + 4, . . . , k} with xi ∈ V (R ) and V (R[x ) ∩ {xi+1 , . . . , xk } = ∅ (cf. Figure 10.4). Let p+1,x ] i

R := R[x . So R has vertices x p , ϕ(x p ), µ(ϕ(x p )), ϕ(µ(ϕ(x p ))), . . . , xq , and p ,x q ] can be traversed in time proportional to its length.

X x0 = v

xq xp

xk = v

x p+1

Fig. 10.4.

Let S := E(R) \ E(G[X ]), D := (MM ) \ (E(G[X ]) ∪ E(P[xq ,v] )), and Z := SD. S and D consist of M-alternating paths and circuits. Observe that every vertex outside X has degree 0 or 2 with respect to Z . Moreover, for every vertex outside X with two incident edges of Z , one of them belongs to M . (Here the choice of q is essential.) Hence all connected components C of (V (G), Z ) with E(C) ∩ δ(X ) = ∅ can be added as next ears, and after these ears have been added, S \ Z = S ∩ (MM ) is the vertex-disjoint union of paths each of which can then be added as an ear. Since e ∈ D \ S ⊆ Z , we have Z ∩ δ(X ) = ∅, so at least one ear is added.

240

10 Maximum Matchings

It remains to show that the time needed for the above construction is proportional to the total number of edges in new ears. Obviously, it sufﬁces to ﬁnd S in O(|E(S)|) time. This is difﬁcult because of the subpaths of R inside X . However, we do not really care what they look like. So we would like to shortcut these paths whenever possible. To achieve this, we modify the ϕ-variables. Namely, in each application of Case 2, let R[a,b] be a maximal subpath of R inside X with a = b. Let y := µ(b); y is the predecessor of b on R. We set ϕ(x) := y for all vertices x on R[a,y] where R[x,y] has odd length. It does not matter whether x and y are joined by an edge. See Figure 10.5 for an illustration.

y

X

R

xp

x p+1

x0 = v

Fig. 10.5.

The time required for updating the ϕ-variables is proportional to the number of edges examined. Note that these changes of ϕ do not destroy the property of Lemma 10.22, and the ϕ-variables are not used anymore except for ﬁnding M-alternating paths to v in Case 2. Now it is guaranteed that the time required for ﬁnding the subpaths of R inside X is proportional to the number of subpaths plus the number of edges examined for the ﬁrst time inside X . Since the number of subpaths inside X is less than or equal to the number of new ears in this step, we obtain an overall linear running time. Case 3: δ(X ) ∩ M = ∅. Then v ∈ X . We consider the ears of the (old) Malternating ear-decomposition in their order. Let R be the ﬁrst ear with V (R)\X = ∅. Similar to Case 2, let S := E(R) \ E(G[X ]), D := (MM ) \ E(G[X ]), and Z := SD. Again, all connected components C of (V (G), Z ) with E(C)∩δ(X ) =

10.5 Edmonds’ Matching Algorithm

241

∅ can be added as next ears, and after these ears have been added, S \ Z is the vertex-disjoint union of paths each of which can then be added as an ear. The total time needed for Case 3 is obviously linear.

10.5 Edmonds’ Matching Algorithm Recall Berge’s Theorem 10.7: A matching in a graph is maximum if and only if there is no augmenting path. Since this holds for non-bipartite graphs as well, our matching algorithm will again be based on augmenting paths. However, it is not at all clear how to ﬁnd an augmenting path (or decide that there is none). In the bipartite case (Theorem 10.5) it was sufﬁcient to mark the vertices that are reachable from a vertex not covered by the matching via an alternating edge progression. Since there were no odd circuits, vertices reachable by an alternating edge progression were also reachable by an alternating path. This is no longer the case when dealing with general graphs. v8

v1

v3

v4

v5

v2

v7

v6

Fig. 10.6.

Consider the example in Figure 10.6 (the bold edges constitute a matching M). When starting at v 1 , we have an alternating edge progression v 1 , v 2 , v 3 , v 4 , v 5 , v 6 , v 7 , v 5 , v 4 , v 8 , but this is not a path. We have run through an odd circuit, namely v 5 , v 6 , v 7 . Note that in our example there exists an augmenting path (v 1 , v 2 , v 3 , v 7 , v 6 , v 5 , v 4 , v 8 ) but it is not clear how to ﬁnd it. The question arises what to do if we encounter an odd circuit. Surprisingly, it sufﬁces to get rid of it by shrinking it to a single vertex. It turns out that the smaller graph has a perfect matching if and only if the original graph has one. This is the general idea of E DMONDS ’ C ARDINALITY M ATCHING A LGORITHM. We formulate this idea in Lemma 10.25 after giving the following deﬁnition: Deﬁnition 10.24. Let G be a graph and M a matching in G. A blossom in G with respect to M is a factor-critical subgraph C of G with |M ∩ E(C)| = |V (C)|−1 . 2 The vertex of C not covered by M ∩ E(C) is called the base of C. The blossom we have encountered in the above example (Figure 10.6) is induced by {v 5 , v 6 , v 7 }. Note that this example contains other blossoms. Any single vertex

242

10 Maximum Matchings

is also a blossom in terms of our definition. Now we can formulate the Blossom Shrinking Lemma: Lemma 10.25. Let G be a graph, M a matching in G, and C a blossom in G (with respect to M). Suppose there is an M-alternating v-r -path Q of even length from a vertex v not covered by M to the base r of C, where E(Q) ∩ E(C) = ∅. Let G 0 and M 0 result from G and M by shrinking V (C) to a single vertex. Then M is a maximum matching in G if and only if M 0 is a maximum matching in G 0 . Proof: Suppose that M is not a maximum matching in G. N := M4E(Q) is a matching of the same cardinality, so it is not maximum either. By Berge’s Theorem 10.7 there then exists an N -augmenting path P in G. Note that N does not cover r. At least one of the endpoints of P, say x, does not belong to C. If P and C are disjoint, let y be the other endpoint of P. Otherwise let y be the first vertex on P – when traversed from x – belonging to C. Let P 0 result from P[x,y] when shrinking V (C) in G. The endpoints of P 0 are not covered by N 0 (the matching in G 0 corresponding to N ). Hence P 0 is an N 0 -augmenting path in G 0 . So N 0 is not a maximum matching in G 0 , and nor is M 0 (which has the same cardinality). To prove the converse, suppose that M 0 is not a maximum matching in G 0 . Let N 0 be a larger matching in G 0 . N 0 corresponds to a matching N0 in G which covers at most one vertex of C in G. Since C is factor-critical, N0 can be extended edges to a matching N in G, where by k := |V (C)|−1 2 |N | = |N0 | + k = |N 0 | + k > |M 0 | + k = |M|, proving that M is not a maximum matching in G.

It is necessary to require that the base of the blossom is reachable from a vertex not covered by M by an M-alternating path of even length which is disjoint from the blossom. For example, the blossom induced by {v4 , v6 , v7 , v2 , v3 } in Figure 10.6 cannot be shrunk without destroying the only augmenting path. When looking for an augmenting path, we shall build up an alternating forest: Definition 10.26. Given a graph G and a matching M in G. An alternating forest with respect to M in G is a forest F in G with the following properties: (a) V (F) contains all the vertices not covered by M. Each connected component of F contains exactly one vertex not covered by M, its root. (b) We call a vertex v ∈ V (F) an outer (inner) vertex if it has even (odd) distance to the root of the connected component containing v. (In particular, the roots are outer vertices.) All inner vertices have degree 2 in F. (c) For any v ∈ V (F), the unique path from v to the root of the connected component containing v is M-alternating. Figure 10.7 shows an alternating forest. The bold edges belong to the matching. The black vertices are inner, the white vertices outer.

10.5 Edmonds’ Matching Algorithm

243

Fig. 10.7.

Proposition 10.27. In any alternating forest the number of outer vertices that are not a root equals the number of inner vertices. Proof: Each outer vertex that is not a root has exactly one neighbour which is an inner vertex and whose distance to the root is smaller. This is obviously a bijection between the outer vertices that are not a root and the inner vertices. Informally, E DMONDS ’ C ARDINALITY M ATCHING A LGORITHM works as follows. Given some matching M, we build up an M-alternating forest F. We start with the set S of vertices not covered by M, and no edges. At any stage of the algorithm we consider a neighbour y of an outer vertex x. Let P(x) denote the unique path in F from x to a root. There are three interesting cases, corresponding to three operations (“grow”, “augment”, and “shrink”): Case 1: y ∈ / V (F). Then the forest will grow when we add {x, y} and the matching edge covering y. Case 2: y is an outer vertex in a different connected component of F. Then we augment M along P(x) ∪ {x, y} ∪ P(y). Case 3: y is an outer vertex in the same connected component of F (with root q). Let r be the ﬁrst vertex of P(x) (starting at x) also belonging to P(y). (r can be one of x, y.) If r is not a root, it must have degree at least 3. So r is an outer vertex. Therefore C := P(x)[x,r ] ∪ {x, y} ∪ P(y)[y,r ] is a blossom with at least three vertices. We shrink C.

244

10 Maximum Matchings

If none of the cases applies, all the neighbours of outer vertices are inner. We claim that M is maximum. Let X be the set of inner vertices, s := |X |, and let t be the number of outer vertices. G − X has t odd components (each outer vertex is isolated in G − X ), so qG (X ) − |X | = t − s. Hence by the trivial part of the Berge-Tutte formula, any matching must leave at least t − s vertices uncovered. But on the other hand, the number of vertices not covered by M, i.e. the number of roots of F, is exactly t − s by Proposition 10.27. Hence M is indeed maximum. Since this is not at all a trivial task, we shall spend some time on implementation details. The difficult question is how to perform the shrinking efficiently so that the original graph can be recovered afterwards. Of course, several shrinking operations may involve the same vertex. Our presentation is based on the one given by Lov´asz and Plummer [1986]. Rather than actually performing the shrinking operation, we allow our forest to contain blossoms. Definition 10.28. Given a graph G and a matching M in G. A subgraph F of G is a general blossom forest (with respect to M) if there exists a partition . . . V (F) = V1 ∪ V2 ∪ · · · ∪ Vk of the vertex set such that Fi := F[Vi ] is a maximal factor-critical subgraph of F with |M ∩ E(Fi )| = |Vi 2|−1 (i = 1, . . . , k) and after contracting each of V1 , . . . , Vk we obtain an alternating forest F 0 . Fi is called an outer blossom (inner blossom) if Vi is an outer (inner) vertex in F 0 . All the vertices of an outer (inner) blossom are called outer (inner). A general blossom forest where each inner blossom is a single vertex is a special blossom forest. Figure 10.8 shows a connected component of a special blossom forest with five nontrivial outer blossoms. This corresponds to one of the connected components of the alternating forest in Figure 10.7. The orientations of the edges will be explained later. All vertices of G not belonging to the special blossom forest are called out-of-forest. Note that the Blossom Shrinking Lemma 10.25 applies to outer blossoms only. However, in this section we shall deal only with special blossom forests. General blossom forests will appear only in the W EIGHTED M ATCHING A LGORITHM in Chapter 11. To store a special blossom forest F, we introduce the following data structures. For each vertex x ∈ V (G) we have three variables µ(x), ϕ(x), and ρ(x) with the following properties:

10.5 Edmonds’ Matching Algorithm

245

y

x

Fig. 10.8.

x y ⎧ ⎪ x ⎪ ⎪ ⎪ ⎪ ⎪ ⎨y ϕ(x) = y ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎩ ⎧ ⎪ ⎨x ρ(x) = y ⎪ ⎩

µ(x) =

if x is not covered by M if {x, y} ∈ M

(10.2)

if x ∈ / V (F) or x is the base of an outer blossom for {x, y} ∈ E(F) \ M if x is an inner vertex (10.3) such that {x, y} ∈ E(F) \ M, and µ and ϕ are associated with an M-alternating ear-decomposition of the blossom containing x, if x is an outer vertex if x is not an outer vertex if x is an outer vertex and y is the base of the outer blossom in F containing x

(10.4)

For each outer vertex v we deﬁne P(v) to be the maximal path given by an initial subsequence of v, µ(v), ϕ(µ(v)), µ(ϕ(µ(v))), ϕ(µ(ϕ(µ(v)))), . . .

246

10 Maximum Matchings

We have the following properties: Proposition 10.29. Let F be a special blossom forest with respect to a matching M, and let µ, ϕ : V (G) → V (G) be functions satisfying (10.2) and (10.3). Then we have: (a) For each outer vertex v, P(v) is an alternating v-q-path, where q is the root of the tree of F containing v. (b) A vertex x is • outer iff either µ(x) = x or ϕ(µ(x)) 6= µ(x); • inner iff ϕ(µ(x)) = µ(x) and ϕ(x) 6= x; • out-of-forest iff µ(x) 6= x and ϕ(x) = x and ϕ(µ(x)) = µ(x). Proof: (a): By (10.3) and Lemma 10.22, some initial subsequence of v, µ(v), ϕ(µ(v)), µ(ϕ(µ(v))), ϕ(µ(ϕ(µ(v)))), . . . must be an M-alternating path of even length to the base r of the blossom containing v. If r is not the root of the tree containing v, then r is covered by M. Hence the above sequence continues with the matching edge {r, µ(r )} and also with {µ(r ), ϕ(µ(r ))}, because µ(r ) is an inner vertex. But ϕ(µ(r )) is an outer vertex again, and so we are done by induction. (b): If a vertex x is outer, then it is either a root (i.e. µ(x) = x) or P(x) is a path of length at least two, i.e. ϕ(µ(x)) 6= µ(x). If x is inner, then µ(x) is the base of an outer blossom, so by (10.3) ϕ(µ(x)) = µ(x). Furthermore, P(µ(x)) is a path of length at least 2, so ϕ(x) 6= x. If x is out-of-forest, then by definition x is covered by M, so by (10.2) µ(x) 6= x. Of course µ(x) is also out-of-forest, so by (10.3) we have ϕ(x) = x and ϕ(µ(x)) = µ(x). Since each vertex is either outer or inner or out-of-forest, and each vertex satisfies exactly one of the three right-hand side conditions, the proof is complete. In Figure 10.8, an edge is oriented from u to v if ϕ(u) = v. We are now ready for a detailed description of the algorithm.

E DMONDS ’ C ARDINALITY M ATCHING A LGORITHM Input:

A graph G.

Output:

A maximum matching in G given by the edges {x, µ(x)}.

1

2

Set µ(v) := v, ϕ(v) := v, ρ(v) := v and scanned(v) := false for all v ∈ V (G). If all outer vertices are scanned then stop, else let x be an outer vertex with scanned(x) = false.

10.5 Edmonds’ Matching Algorithm

3

4

5

6

247

Let y be a neighbour of x such that y is out-of-forest or (y is outer and ρ(y) 6= ρ(x)). If there is no such y then set scanned(x) := true and go to . 2 (“grow”) If y is out-of-forest then set ϕ(y) := x and go to . 3 (“augment”) If P(x) and P(y) are vertex-disjoint then Set µ(ϕ(v)) := v, µ(v) := ϕ(v) for all v ∈ V (P(x)) ∪ V (P(y)) with odd distance from x or y on P(x) or P(y), respectively. Set µ(x) := y. Set µ(y) := x. Set ϕ(v) := v, ρ(v) := v, scanned(v) := false for all v ∈ V (G). Go to . 2 (“shrink”) Let r be the first vertex on V (P(x)) ∩ V (P(y)) with ρ(r ) = r . For v ∈ V (P(x)[x,r ] ) ∪ V (P(y)[y,r ] ) with odd distance from x or y on P(x)[x,r ] or P(y)[y,r ] , respectively, and ρ(ϕ(v)) 6= r do: Set ϕ(ϕ(v)) := v. If ρ(x) 6= r then set ϕ(x) := y. If ρ(y) 6= r then set ϕ(y) := x. For all v ∈ V (G) with ρ(v) ∈ V (P(x)[x,r ] ) ∪ V (P(y)[y,r ] ) do: Set ρ(v) := r . Go to . 3

For an illustration of the effect of shrinking on the ϕ-values, see Figure 10.9, where 6 of the algorithm has been applied to x and y in Figure 10.8. Lemma 10.30. The following statements hold at any stage of E DMONDS ’ C AR DINALITY M ATCHING A LGORITHM : (a) The edges {x, µ(x)} form a matching M; (b) The edges {x, µ(x)} and {x, ϕ(x)} form a special blossom forest F with respect to M (plus some isolated matching edges); (c) The properties (10.2), (10.3) and (10.4) are satisfied with respect to F. Proof: (a): The only place where µ is changed is , 5 where the augmentation is obviously done correctly. (b): Since after 1 and 5 we trivially have a blossom forest without any edges and 4 correctly grows the blossom forest by two edges, we only have to check

. 6 r either is a root or must have degree at least three, so it must be outer. Let B := V (P(x)[x,r ] ) ∪ V (P(y)[y,r ] ). Consider an edge {u, v} of the blossom forest with u ∈ B and v ∈ / B. Since F[B] contains a near-perfect matching, {u, v} is a matching edge only if it is {r, µ(r )}. Moreover, u has been outer before applying

. 6 This implies that F continues to be a special blossom forest.

248

10 Maximum Matchings

y

x

r

Fig. 10.9.

(c): Here the only nontrivial fact is that, after shrinking, µ and ϕ are associated with an alternating ear-decomposition of the new blossom. So let x and y be two outer vertices in the same connected component of the special blossom forest, and let r be the ﬁrst vertex of V (P(x))∩V (P(y)) for which ρ(r ) = r . The new blossom consists of the vertices B := {v ∈ V (G) : ρ(v) ∈ V (P(x)[x,r ] ) ∪ V (P(y)[y,r ] )}. We note that ϕ(v) is not changed for any v ∈ B with ρ(v) = r . So the eardecomposition of the old blossom B := {v ∈ V (G) : ρ(v) = r } is the starting point of the ear-decomposition of B. The next ear consists of P(x)[x,x ] , P(y)[y,y ] , and the edge {x, y}, where x and y is the ﬁrst vertex on P(x) and P(y), respectively, that belongs to B . Finally, for each ear Q of an old outer blossom B ⊆ B, Q \ (E(P(x)) ∪ E(P(y))) is an ear of the new ear-decomposition of B. Theorem 10.31. (Edmonds [1965]) E DMONDS ’ C ARDINALITY M ATCHING A LGORITHM correctly determines a maximum matching in O(n 3 ) time, where n = |V (G)|. Proof: Lemma 10.30 and Proposition 10.29 show that the algorithm works correctly. Consider the situation when the algorithm terminates. Let M and F be the

10.5 Edmonds’ Matching Algorithm

249

matching and the special blossom forest according to Lemma 10.30(a) and (b). It is clear that any neighbour of an outer vertex x is either inner or a vertex y belonging to the same blossom (i.e. ρ(y) = ρ(x)). To show that M is a maximum matching, let X denote the set of inner vertices, while B is the set of vertices that are the base of some outer blossom in F. Then every unmatched vertex belongs to B, and the matched vertices of B are matched with elements of X : |B| = |X | + |V (G)| − 2|M|. (10.5) On the other hand, the outer blossoms in F are odd connected components in G − X . Therefore any matching must leave at least |B| − |X | vertices uncovered. By (10.5), M leaves exactly |B| − |X | vertices uncovered and thus is maximum. We now consider the running time. By Proposition 10.29(b), the status of each vertex (inner, outer, or out-of-forest) can be checked in constant time. Each of , 4 , 5 6 can be done in O(n) time. Between two augmentations, 4 or 6 are executed at most O(n) times, since the number of ﬁxed points of ϕ decreases each time. Moreover, between two augmentations no vertex is scanned twice. Thus the time spent between two augmentations is O(n 2 ), yielding an O(n 3 ) total running time. √ Micali and Vazirani [1980] improved the running time to O n m . They used the results of Exercise 9, but the existence of blossoms makes the search for a maximal set of disjoint minimum length augmenting paths more difﬁcult than in the bipartite case (which was solved earlier by Hopcroft and Karp [1973], see Exercise 10). See also Vazirani [1994]. The currently best √known2 time complexity /m) for the C ARDINALITY M ATCHING P ROBLEM is O m n log(n , just as in log n the bipartite case. This was obtained by Goldberg and Karzanov [2004] and by Fremuth-Paeger and Jungnickel [2003]. With the matching algorithm we can easily prove the Gallai-Edmonds Structure Theorem. This was ﬁrst proved by Gallai, but E DMONDS ’ C ARDINALITY M ATCHING A LGORITHM turns out to be a constructive proof thereof.

Y

X

W Fig. 10.10.

250

10 Maximum Matchings

Theorem 10.32. (Gallai [1964]) Let G be any graph. Denote by Y the set of vertices not covered by at least one maximum matching, by X the neighbours of Y in V (G) \ Y , and by W all other vertices. Then: (a) Any maximum matching in G contains a perfect matching of G[W ] and nearperfect matchings of the connected components of G[Y ], and matches all vertices in X to distinct connected components of G[Y ]; (b) The connected components of G[Y ] are factor-critical; (c) 2ν(G) = |V (G)| − qG (X ) + |X |. We call W, X, Y the Gallai-Edmonds decomposition of G (see Figure 10.10). Proof: We apply E DMONDS ’ C ARDINALITY M ATCHING A LGORITHM and consider the matching M and the special blossom forest F at termination. Let X 0 be the set of inner vertices, Y 0 the set of outer vertices, and W 0 the set of out-offorest vertices. We first prove that X 0 , Y 0 , W 0 satisfy (a)–(c), and then observe that X = X 0 , Y = Y 0 , and W = W 0 . The proof of Theorem 10.31 shows that 2ν(G) = |V (G)| − qG (X 0 ) + |X 0 |. We apply Proposition 10.15 to X 0 . Since the odd connected components of G − X 0 are exactly the outer blossoms in F, (a) holds for X 0 , Y 0 , W 0 . Since the outer blossoms are factor-critical, (b) also holds. Since part (a) holds for X 0 , Y 0 , and W 0 , we know that any maximum matching covers all the vertices in V (G) \ Y 0 . In other words, Y ⊆ Y 0 . We claim that Y 0 ⊆ Y also holds. Let v be an outer vertex in F. Then M4E(P(v)) is a maximum matching M 0 , and M 0 does not cover v. So v ∈ Y . Hence Y = Y 0 . This implies X = X 0 and W = W 0 , and the theorem is proved.

Exercises

∗

1. Let G be a graph and M1 , M2 two maximal matchings in G. Prove that |M1 | ≤ 2|M2 |. 2. Let α(G) denote the size of a maximum stable set in G, and ζ (G) the minimum cardinality of an edge cover. Prove: (a) α(G) + τ (G) = |V (G)| for any graph G. (b) ν(G) + ζ (G) = |V (G)| for any graph G with no isolated vertices. (c) ζ (G) = α(G) for any bipartite graph G with no isolated vertices. (K¨onig [1933], Gallai [1959]) 3. Prove that a k-regular bipartite graph has k disjoint perfect matchings. Deduce from this that the edge set of a bipartite graph of maximum degree k can be partitioned into k matchings. (K¨onig [1916]; see Rizzi [1998] or Theorem 16.16) 4. A partially ordered set (or poset) is defined to be a set S together with a partial order on S, i.e. a relation R ⊆ S × S that is reflexive ((x, x) ∈ R for all x ∈ S), symmetric (if (x, y) ∈ R and (y, x) ∈ R then x = y), and transitive (if

Exercises

251

(x, y) ∈ R and (y, z) ∈ R then (x, z) ∈ R). Two elements x, y ∈ S are called comparable if (x, y) ∈ R or (y, x) ∈ R, otherwise they are incomparable. A chain (an antichain) is a subset of pairwise comparable (incomparable) elements of S. Use K¨onig’s Theorem 10.2 to prove the following theorem of Dilworth [1950]: In a finite poset the maximum size of an antichain equals the minimum number of chains into which the poset can be partitioned. Hint: Take two copies v 0 and v 00 of each v ∈ S and consider the graph with an edge {v 0 , w00 } for each (v, w) ∈ R. (Fulkerson [1956]) 5. (a) Let S = {1, 2, . . . , n} and 0 ≤ k < n2 . Let A and B be the collection of all k-element and (k + 1)-element subsets of S, respectively. Construct a bipartite graph .

G = (A ∪ B, {{a, b} : a ∈ A, b ∈ B, a ⊆ b}). ∗

6.

7.

8.

9.

Prove that G has a matching covering A. (b) Prove Sperner’s Lemma: the maximum number of subsets of an n-element set such that none is contained in any other is b nn c . 2 (Sperner [1928]) Let (U, S) be a set system. An injective function 8 : S → U such that 8(S) ∈ S for all S ∈ S is called a system of distinct representatives of S. Prove: (a) S has a system of distinct representatives if and only if the union of any k of the sets in S has cardinality at least k. (Hall [1935]) (b) P For u ∈ U P let r (u) := |{S ∈ S : u ∈ S}|. Let n := |S| and N := N N |S| = S∈S u∈U r (u). Suppose |S| < n−1 for S ∈ S and r (u) < n−1 for u ∈ U . Then S has a system of distinct representatives. (Mendelsohn and Dulmage [1958]) . Let G be a bipartite graph with bipartition V (G) = A ∪ B. Suppose that S ⊆ A, T ⊆ B, and there is a matching covering S and a matching covering T . Prove that then there is a matching covering S ∪ T . (Mendelsohn and Dulmage [1958]) Show that any simple graph on n vertices with minimum degree k has a matching of cardinality min{k, b n2 c}. Hint: Use Berge’s Theorem 10.7. Let G be a graph and M a matching in G that is not maximum. (a) Show that there are ν(G) − |M| vertex-disjoint M-augmenting paths in G. Hint: Recall the proof of Berge’s Theorem 10.7. (b) Prove that there exists an M-augmenting path of length at most ν(G)+|M| ν(G)−|M| in G. (c) Let P be a shortest M-augmenting path in G, and P 0 an (M4E(P))augmenting path. Then |E(P 0 )| ≥ |E(P)| + |E(P ∩ P 0 )|.

252

10 Maximum Matchings

Consider the following generic algorithm. We start with the empty matching and in each iteration augment the matching along a shortest augmenting path. Let P1 , P2 , . . . be the sequence of augmenting paths chosen. By (c), |E(Pk )| ≤ |E(Pk+1 )| for all k. (d) Show that if |E(Pi )| = |E(Pj )| for i 6= j then Pi and Pj are vertex-disjoint. (e) Use √ (b) to prove that the sequence |E(P1 )|, |E(P2 )|, . . . contains at most 2 ν(G) + 2 different numbers. (Hopcroft and Karp [1973]) ∗ 10. Let G be a bipartite graph and consider the generic algorithm of Exercise 9. (a) Prove that – given a matching M – the union of all shortest M-augmenting paths in G can be found in O(n + m) time. Hint: Use a kind of breadth-first search with matching edges and nonmatching edges alternating. (b) Consider a sequence of iterations of the algorithm where the length of the augmenting path remains constant. Show that the time needed for the whole sequence is no more than O(n + m). Hint: First apply (a) and then find the paths successively by DFS. Mark vertices already visited. √ (c) Combine (b) with Exercise 9(e) to obtain an O n(m + n) -algorithm for the C ARDINALITY M ATCHING P ROBLEM in bipartite graphs. (Hopcroft and Karp [1973]) . 11. Let G be a bipartite graph with bipartition V (G) = A ∪ B, A = {a1 , . . . , ak }, B = {b1 , . . . , bk }. For any vector x = (xe )e∈E(G) we define a matrix MG (x) = (m ixj )1≤i, j≤k by ( xe if e = {ai , b j } ∈ E(G) x . m i j := 0 otherwise Its determinant det MG (x) is a polynomial in x = (xe )e∈E(G) . Prove that G has a perfect matching if and only if det MG (x) is not identically zero. 12. The permanent of a square matrix M = (m i j )1≤i, j≤n is defined by per(M) :=

k X Y

m i,π(i) ,

π ∈Sn i=1

where Sn is the set of permutations of {1, . . . , n}. Prove that a simple bipartite graph G has exactly per(MG (1l)) perfect matchings, where MG (x) is defined as in the previous exercise. 13. A doubly stochastic matrix is a nonnegative square matrix whose column sums and row sums are all 1. Integral doubly stochastic matrices are called permutation matrices. Falikman [1981] and Egoryˇcev [1980] proved that for a doubly stochastic n × n-matrix M, n! per(M) ≥ n , n

Exercises

253

and equality holds if and only if every entry of M is n1 . (This was a famous conjecture of van der Waerden; see also Schrijver [1998].) Br`egman [1973] proved that for a 0-1-matrix M with row sums r1 , . . . , rn , 1

1

per(M) ≤ (r1 !) r1 · . . . · (rn !) rn . Use these results and Exercise 12 to prove the following. Let G be a simple k-regular bipartite graph on 2n vertices, and let 8(G) be the number of perfect matchings in G. Then n n k ≤ 8(G) ≤ (k!) k . n! n 14. Prove that every 3-regular graph with at most two bridges has a perfect matching. Is there a 3-regular graph without a perfect matching? Hint: Use Tutte’s Theorem 10.13. (Petersen [1891]) ∗ 15. Let G be a graph, n := |V (G)| even, and for any set X ⊆ V (G) with |X | ≤ 34 n we have [ 4 0(x) ≥ |X |. 3 x∈X

16. 17. ∗ 18.

19.

20. 21. ∗ 22.

Prove that G has a perfect matching. Hint: Let S be a set violating the Tutte condition. Prove that the number of components in G − S with just one element is at most n connected o 4 1 max 0, 3 |S| − 3 n . Consider the cases |S| ≥ n4 and |S| < n4 separately. (Anderson [1971]) Prove that an undirected graph G is factor-critical if and only if G is connected and ν(G) = ν(G − v) for all v ∈ V (G). Prove that the number of ears in any two odd ear-decompositions of a factorcritical graph G is the same. For a 2-edge-connected graph G let ϕ(G) be the minimum number of even ears in an ear-decomposition of G (cf. Exercise 17(a) of Chapter 2). Show that for any edge e ∈ E(G) we have either ϕ(G/e) = ϕ(G)+1 or ϕ(G/e) = ϕ(G)−1. Note: The function ϕ(G) has been studied by Szigeti [1996] and Szegedy and Szegedy [2006]. Prove that a minimal factor-critical graph G (i.e. after the deletion of any edge the graph is no longer factor-critical) has at most 32 (|V (G)| − 1) edges. Show that this bound is tight. Show how E DMONDS ’ C ARDINALITY M ATCHING A LGORITHM finds a maximum matching in the graph shown in Figure 10.1(b). Given an undirected graph, can one find an edge cover of minimum cardinality in polynomial time? Given an undirected graph G, an edge is called unmatchable if it is not contained in any perfect matching. How can one determine the set of unmatchable edges

254

10 Maximum Matchings

in O(n 3 ) time? Hint: First determine a perfect matching in G. Then determine for each vertex v the set of unmatchable edges incident to v. 23. Let G be a graph, M a maximum matching in G, and F1 and F2 two special blossom forests with respect to M, each with the maximum possible number of edges. Show that the set of inner vertices in F1 and F2 is the same. 24. Let G be a k-connected graph with 2ν(G) < |V (G)| − 1. Prove: (a) ν(G) ≥ k; (b) τ (G) ≤ 2ν(G) − k. Hint: Use the Gallai-Edmonds Theorem 10.32. (Erd˝os and Gallai [1961])

References General Literature: Gerards, A.M.H. [1995]: Matching. In: Handbooks in Operations Research and Management Science; Volume 7: Network Models (M.O. Ball, T.L. Magnanti, C.L. Monma, G.L. Nemhauser, eds.), Elsevier, Amsterdam 1995, pp. 135–224 Lawler, E.L. [1976]: Combinatorial Optimization; Networks and Matroids. Holt, Rinehart and Winston, New York 1976, Chapters 5 and 6 Lov´asz, L., and Plummer, M.D. [1986]: Matching Theory. Akad´emiai Kiad´o, Budapest 1986, and North-Holland, Amsterdam 1986 Papadimitriou, C.H., and Steiglitz, K. [1982]: Combinatorial Optimization; Algorithms and Complexity. Prentice-Hall, Englewood Cliffs 1982, Chapter 10 Pulleyblank, W.R. [1995]: Matchings and extensions. In: Handbook of Combinatorics; Vol. 1 (R.L. Graham, M. Gr¨otschel, L. Lov´asz, eds.), Elsevier, Amsterdam 1995 Schrijver, A. [2003]: Combinatorial Optimization: Polyhedra and Efficiency. Springer, Berlin 2003, Chapters 16 and 24 Tarjan, R.E. [1983]: Data Structures and Network Algorithms. SIAM, Philadelphia 1983, Chapter 9

Cited References: Alt, H., Blum, N., Mehlhorn, K., and Paul, [1991]: Computing a maximum cardinality M. p matching in a bipartite graph in time O n 1.5 m/ log n . Information Processing Letters 37 (1991), 237–240 Anderson, I. [1971]: Perfect matchings of a graph. Journal of Combinatorial Theory B 10 (1971), 183–186 Berge, C. [1957]: Two theorems in graph theory. Proceedings of the National Academy of Science of the U.S. 43 (1957), 842–844 Berge, C. [1958]: Sur le couplage maximum d’un graphe. Comptes Rendus Hebdomadaires des S´eances de l’Acad´emie des Sciences (Paris) S´er. I Math. 247 (1958), 258–259 Br`egman, L.M. [1973]: Certain properties of nonnegative matrices and their permanents. Doklady Akademii Nauk SSSR 211 (1973), 27–30 [in Russian]. English translation: Soviet Mathematics Doklady 14 (1973), 945–949

References

255

Dilworth, R.P. [1950]: A decomposition theorem for partially ordered sets. Annals of Mathematics 51 (1950), 161–166 Edmonds, J. [1965]: Paths, trees, and flowers. Canadian Journal of Mathematics 17 (1965), 449–467 Egoryˇcev, G.P. [1980]: Solution of the van der Waerden problem for permanents. Soviet Mathematics Doklady 23 (1982), 619–622 Erd˝os, P., and Gallai, T. [1961]: On the minimal number of vertices representing the edges of a graph. Magyar Tudom´anyos Akad´emia; Matematikai Kutat´o Int´ezet´enek K¨ozlem´enyei 6 (1961), 181–203 Falikman, D.I. [1981]: A proof of the van der Waerden conjecture on the permanent of a doubly stochastic matrix. Matematicheskie Zametki 29 (1981), 931–938 [in Russian]. English translation: Math. Notes of the Acad. Sci. USSR 29 (1981), 475–479 Feder, T., and Motwani, R. [1995]: Clique partitions, graph compression and speeding-up algorithms. Journal of Computer and System Sciences 51 (1995), 261–272 Fremuth-Paeger, C., and Jungnickel, D. [2003]: Balanced network flows VIII: a revised √ theory of phase-ordered algorithms and the O( nm log(n 2 /m)/ log n) bound for the nonbipartite cardinality matching problem. Networks 41 (2003), 137–142 ¨ Frobenius, G. [1917]: Uber zerlegbare Determinanten. Sitzungsbericht der K¨oniglich Preussischen Akademie der Wissenschaften XVIII (1917), 274–277 Fulkerson, D.R. [1956]: Note on Dilworth’s decomposition theorem for partially ordered sets. Proceedings of the AMS 7 (1956), 701–702 ¨ Gallai, T. [1959]: Uber extreme Punkt- und Kantenmengen. Annales Universitatis Scientiarum Budapestinensis de Rolando E¨otv¨os Nominatae; Sectio Mathematica 2 (1959), 133–138 Gallai, T. [1964]: Maximale Systeme unabh¨angiger Kanten. Magyar Tudom´anyos Akad´emia; Matematikai Kutat´o Int´ezet´enek K¨ozlem´enyei 9 (1964), 401–413 Geelen, J.F. [2000]: An algebraic matching algorithm. Combinatorica 20 (2000), 61–70 Geelen, J. and Iwata, S. [2005]: Matroid matching via mixed skew-symmetric matrices. Combinatorica 25 (2005), 187–215 Goldberg, A.V., and Karzanov, A.V. [2004]: Maximum skew-symmetric flows and matchings. Mathematical Programming A 100 (2004), 537–568 Hall, P. [1935]: On representatives of subsets. Journal of the London Mathematical Society 10 (1935), 26–30 Halmos, P.R., and Vaughan, H.E. [1950]: The marriage problem. American Journal of Mathematics 72 (1950), 214–215 Hopcroft, J.E., and Karp, R.M. [1973]: An n 5/2 algorithm for maximum matchings in bipartite graphs. SIAM Journal on Computing 2 (1973), 225–231 ¨ K¨onig, D. [1916]: Uber Graphen und ihre Anwendung auf Determinantentheorie und Mengenlehre. Mathematische Annalen 77 (1916), 453–465 K¨onig, D. [1931]: Graphs and matrices. Matematikai´es Fizikai Lapok 38 (1931), 116–119 [in Hungarian] ¨ K¨onig, D. [1933]: Uber trennende Knotenpunkte in Graphen (nebst Anwendungen auf Determinanten und Matrizen). Acta Litteratum ac Scientiarum Regiae Universitatis Hungaricae Francisco-Josephinae (Szeged). Sectio Scientiarum Mathematicarum 6 (1933), 155–179 Kuhn, H.W. [1955]: The Hungarian method for the assignment problem. Naval Research Logistics Quarterly 2 (1955), 83–97 Lov´asz, L. [1972]: A note on factor-critical graphs. Studia Scientiarum Mathematicarum Hungarica 7 (1972), 279–280 Lov´asz, L. [1979]: On determinants, matchings and random algorithms. In: Fundamentals of Computation Theory (L. Budach, ed.), Akademie-Verlag, Berlin 1979, pp. 565–574

256

10 Maximum Matchings

Mendelsohn, N.S., and Dulmage, A.L. [1958]: Some generalizations of the problem of distinct representatives. Canadian Journal of Mathematics 10 (1958), 230–241 Micali, S., and Vazirani, V.V. [1980]: An O(V 1/2 E) algorithm for finding maximum matching in general graphs. Proceedings of the 21st Annual IEEE Symposium on Foundations of Computer Science (1980), 17–27 Mucha, M., and Sankowski, P. [2004]: Maximum matchings via Gaussian elimination. Proceedings of the 45th Annual IEEE Symposium on Foundations of Computer Science (2004), 248–255 Mulmuley, K., Vazirani, U.V., and Vazirani, V.V. [1987]: Matching is as easy as matrix inversion. Combinatorica 7 (1987), 105–113 Petersen, J. [1891]: Die Theorie der regul¨aren Graphen. Acta Mathematica 15 (1891), 193–220 Rabin, M.O., and Vazirani, V.V. [1989]: Maximum matchings in general graphs through randomization. Journal of Algorithms 10 (1989), 557–567 Rizzi, R. [1998]: K¨onig’s edge coloring theorem without augmenting paths. Journal of Graph Theory 29 (1998), 87 Schrijver, A. [1998]: Counting 1-factors in regular bipartite graphs. Journal of Combinatorial Theory B 72 (1998), 122–135 Sperner, E. [1928]: Ein Satz u¨ ber Untermengen einer endlichen Menge. Mathematische Zeitschrift 27 (1928), 544–548 Szegedy, B., and Szegedy, C. [2006]: Symplectic spaces and ear-decomposition of matroids. Combinatorica 26 (2006), 353–377 Szigeti, Z. [1996]: On a matroid defined by ear-decompositions. Combinatorica 16 (1996), 233–241 Tutte, W.T. [1947]: The factorization of linear graphs. Journal of the London Mathematical Society 22 (1947), 107–111 Vazirani, V.V. √ [1994]: A theory of alternating paths and blossoms for proving correctness of the O( V E) general graph maximum matching algorithm. Combinatorica 14 (1994), 71–109

11 Weighted Matching

Nonbipartite weighted matching appears to be one of the “hardest” combinatorial optimization problems that can be solved in polynomial time. We shall extend E DMONDS ’ C ARDINALITY M ATCHING A LGORITHM to the weighted case and shall again obtain an O(n 3 )-implementation. This algorithm has many applications, some of which are mentioned in the exercises and in Section 12.2. There are two basic formulations of the weighted matching problem:

M AXIMUM W EIGHT M ATCHING P ROBLEM Instance:

An undirected graph G and weights c : E(G) → R.

Task:

Find a maximum weight matching in G.

M INIMUM W EIGHT P ERFECT M ATCHING P ROBLEM Instance:

An undirected graph G and weights c : E(G) → R.

Task:

Find a minimum weight perfect matching in G or decide that G has no perfect matching.

It is easy to see that both problems are equivalent: Given an instance (G, c) of P ROBLEM, we set c0 (e) := K −c(e) the M INIMUM W EIGHT P ERFECT M ATCHING P for all e ∈ E(G), where K := 1 + e∈E(G) |c(e)|. Then any maximum weight matching in (G, c0 ) is a maximum cardinality matching, and hence gives a solution of the M INIMUM W EIGHT P ERFECT M ATCHING P ROBLEM (G, c). Conversely, let (G, c) be an instance of the M AXIMUM W EIGHT M ATCHING P ROBLEM. Then we add |V (G)| new vertices and all possible edges in order to obtain a complete graph G 0 on 2|V (G)| vertices. We set c0 (e) := −c(e) for all e ∈ E(G) and c0 (e) := 0 for all new edges e. Then a minimum weight perfect matching in (G 0 , c0 ) yields a maximum weight matching in (G, c), simply by deleting the edges not belonging to G. So in the following we consider only the M INIMUM W EIGHT P ERFECT M ATCHING P ROBLEM. As in the previous chapter, we start by considering bipartite graphs in Section 11.1. After an outline of the weighted matching algorithm in Section 11.2 we spend some effort on implementation details in Section 11.3 in order to obtain an O(n 3 ) running time. Sometimes one is interested in solving many matching problems that differ only on a few edges; in such a case it is not

258

11 Weighted Matching

necessary to solve the problem from scratch each time as is shown in Section 11.4. Finally, in Section 11.5 we discuss the matching polytope, i.e. the convex hull of the incidence vectors of matchings. We use a description of the related perfect matching polytope already for designing the weighted matching algorithm; in turn, this algorithm will directly imply that this description is complete.

11.1 The Assignment Problem The A SSIGNMENT P ROBLEM is just another name for the M INIMUM W EIGHT P ERFECT M ATCHING P ROBLEM in bipartite graphs. It is one of the classical combinatorial optimization problems; its history probably started with the work of Monge [1784]. As in the proof of Theorem 10.5, we can reduce the assignment problem to a network flow problem: Theorem 11.1. The A SSIGNMENT P ROBLEM can be solved in O(nm + n 2 log n) time. .

Proof: Let G be a bipartite graph with bipartition V (G) = A ∪ B. We assume |A| = |B| = n. Add a vertex s and connect it to all vertices of A, and add another vertex t connected to all vertices of B. Orient the edges from s to A, from A to B, and from B to t. Let the capacities be 1 everywhere, and let the new edges have zero cost. Then any integral s-t-flow of value n corresponds to a perfect matching with the same cost, and vice versa. So we have to solve a M INIMUM C OST F LOW P ROBLEM. We do this by applying the S UCCESSIVE S HORTEST PATH A LGORITHM (see Section 9.4). The total demand is n. So by Theorem 9.12, the running time is O(nm + n 2 log n). This is the fastest known algorithm. It is essentially equivalent to the “Hungarian method” by Kuhn [1955] and Munkres [1957], the oldest polynomial-time algorithm for the A SSIGNMENT P ROBLEM (cf. Exercise 9). It is worthwhile looking at the linear programming formulation of the A SSIGN MENT P ROBLEM . It turns out that in the integer programming formulation X X min c(e)xe : xe ∈ {0, 1} (e ∈ E(G)), xe = 1 (v ∈ V (G)) e∈E(G)

e∈δ(v)

the integrality constraints can be omitted (replace xe ∈ {0, 1} by xe ≥ 0): Theorem 11.2. Let G be a graph, and let

11.2 Outline of the Weighted Matching Algorithm

P :=

Q :=

E(G)

x ∈ R+

E(G)

x ∈ R+

259

X : xe ≤ 1 for all v ∈ V (G) and e∈δ(v) X : xe = 1 for all v ∈ V (G) e∈δ(v)

be the fractional matching polytope and the fractional perfect matching polytope of G. If G is bipartite, then P and Q are both integral. Proof: If G is bipartite, then the incidence matrix M of G is totally unimodular due to Theorem 5.25. Hence by the Hoffman-Kruskal Theorem 5.20, P is integral. Q is a face of P and thus it is also integral. There is a nice corollary concerning doubly-stochastic matrices. A doubly stochastic matrix is a nonnegative square matrix such that the sum of the entries in each row and each column is 1. Integral doubly stochastic matrices are called permutation matrices. Corollary 11.3. (Birkhoff [1946], von Neumann [1953]) Any doubly stochastic matrix M can be written as a convex combination of permutation matrices P1 , . . . , Pk (i.e. M = c1 P1 + . . . + ck Pk for nonnegative c1 , . . . , ck with c1 + . . . + ck = 1). Proof: Let M = (m i j )i, j∈{1,...,n} be a doubly stochastic n × n-matrix, and let . K n,n be the complete bipartite graph with bipartition {a1 , . . . , an } ∪ {b1 , . . . , bn }. For e = {ai , b j } ∈ E(K n,n ) let xe = m i j . Since M is doubly stochastic, x is in the fractional perfect matching polytope Q of K n,n . By Theorem 11.2 and Corollary 3.32, x can be written as a convex combination of integral vertices of Q. These obviously correspond to permutation matrices. This corollary can also be proved directly (Exercise 3).

11.2 Outline of the Weighted Matching Algorithm The purpose of this and the next section is to describe a polynomial-time algorithm for the general M INIMUM W EIGHT P ERFECT M ATCHING P ROBLEM. This algorithm was developed by Edmonds [1965] and uses the concepts of his algorithm for the C ARDINALITY M ATCHING P ROBLEM (Section 10.5). Let us first outline the main ideas without considering the implementation. Given a graph G with weights c : E(G) → R, the M INIMUM W EIGHT P ERFECT M ATCHING P ROBLEM can be formulated as the integer linear program X X min c(e)xe : xe ∈ {0, 1} (e ∈ E(G)), xe = 1 (v ∈ V (G)) . e∈E(G)

e∈δ(v)

260

11 Weighted Matching

If A is a subset of V (G) with odd cardinality, any perfect matching must contain an odd number of edges in δ(A), in particular at least one. So adding the constraint X xe ≥ 1 e∈δ(A)

does not change anything. Throughout this chapter we use the notation A := {A ⊆ V (G) : |A| odd}. Now consider the LP relaxation: X c(e)xe min e∈E(G)

s.t. X e∈δ(v) X

xe ≥ 0 xe = 1

(e ∈ E(G)) (v ∈ V (G))

xe ≥ 1

(A ∈ A, |A| > 1)

(11.1)

e∈δ(A)

We shall prove later that the polytope described by (11.1) is integral; hence this LP describes the M INIMUM W EIGHT P ERFECT M ATCHING P ROBLEM (this will be Theorem 11.13, a major result of this chapter). In the following we do not need this fact, but will rather use the LP formulation as a motivation. To formulate the dual of (11.1), we introduce a variable z A for each primal constraint, i.e. for each A ∈ A. The dual linear program is: X max zA A∈A

s.t. X

zA ≥ 0 z A ≤ c(e)

(A ∈ A, |A| > 1) (e ∈ E(G))

(11.2)

A∈A:e∈δ(A)

Note that the dual variables z {v} for v ∈ V (G) are not restricted to be nonnegative. Edmonds’ algorithm is a primal-dual algorithm. It starts with the empty matching (xe = 0 for all e ∈ E(G)) and the feasible dual solution 1 2 min{c(e) : e ∈ δ(A)} if |A| = 1 . z A := 0 otherwise At any stage of the algorithm, z will be a feasible dual solution, and we have X xe > 0 ⇒ z A = c(e); zA > 0

⇒

A∈ A:e∈δ(A) X

xe ≤ 1.

e∈δ(A)

(11.3)

11.2 Outline of the Weighted Matching Algorithm

261

The algorithm stops when x is the incidence vector of a perfect matching (i.e. we have primal feasibility). Due to the complementary slackness conditions (11.3) (Corollary 3.23) we then have the optimality of the primal and dual solutions. As x is optimal for (11.1) and integral, it is the incidence vector of a minimum weight perfect matching. Given a feasible dual solution z, we call an edge e tight if the corresponding dual constraint is satisﬁed with equality, i.e. if z A = c(e). A∈A:e∈δ(A)

At any stage, the current matching will consist of tight edges only. We work with a graph G z which results from G by deleting all edges that are not tight and contracting each set B with z B > 0 to a single vertex. The family B := {B ∈ A : z B > 0} will be laminar at any stage, and each element of B will induce a factor-critical subgraph consisting of tight edges only. Initially B consists of the singletons. One iteration of the algorithm roughly proceeds as follows. We ﬁrst ﬁnd a maximum cardinality matching M in G z , using E DMONDS ’ C ARDINALITY M ATCHING A LGORITHM. If M is a perfect matching, we are done: we can complete M to a perfect matching in G using tight edges only. Since the conditions (11.3) are satisﬁed, the matching is optimal.

+ε

+ε

−ε

+ε

−ε

+ε

−ε

Y

X

W Fig. 11.1.

Otherwise we consider the Gallai-Edmonds decomposition W, X, Y of G z (cf. Theorem 10.32). For each vertex v of G z let B(v) ∈ B be the vertex set whose contraction resulted in v. We modify the dual solution as follows (see Figure 11.1 for an illustration). For each v ∈ X we decrease z B(v) by some positive constant ε. For each connected component C of G z [Y ] we increase z A by ε, where A = v∈C B(v).

262

11 Weighted Matching

Note that tight matching edges remain tight, since by Theorem 10.32 all matching edges with one endpoint in X have the other endpoint in Y . (Indeed, all edges of the alternating forest we are working with remain tight). We choose ε maximum possible while preserving dual feasibility. Since the current graph contains no perfect matching, the number of connected components |X |. Hence the above dual change increases the dual of G z [Y ] is greater than P objective function value A∈A z A by at least ε. If ε can be chosen arbitrarily large, the dual LP (11.2) is unbounded, hence the primal LP (11.1) is infeasible (Theorem 3.27) and G has no perfect matching. Due to the change of the dual solution the graph G z will also change: new edges may become tight, new vertex sets may be contracted (corresponding to the components of Y that are not singletons), and some contracted sets may be “unpacked” (non-singletons whose dual variables become zero, corresponding to vertices of X ). The above is iterated until a perfect matching is found. We shall show later that this procedure is finite. This will follow from the fact that between two augmentations, each step (grow, shrink, unpack) increases the number of outer vertices.

11.3 Implementation of the Weighted Matching Algorithm After this informal description we now turn to the implementation details. As with E DMONDS ’ C ARDINALITY M ATCHING A LGORITHM we do not explicitly shrink blossoms but rather store their ear-decomposition. However, there are several difficulties. The “shrink”-step of E DMONDS ’ C ARDINALITY M ATCHING A LGORITHM produces an outer blossom. By the “augment”-step two connected components of the blossom forest become out-of-forest. Since the dual solution remains unchanged, we must retain the blossoms: we get so-called out-of-forest blossoms. The “grow”step may involve out-of-forest blossoms which then become either inner or outer blossoms. Hence we have to deal with general blossom forests. Another problem is that we must be able to recover nested blossoms one by one. Namely, if z A becomes zero for some inner blossom A, there may be subsets A0 ⊆ A with |A0 | > 1 and z A0 > 0. Then we have to unpack the blossom A, but not the smaller blossoms inside A (except if they remain inner and their dual variables are also zero). Throughout the algorithm we have a laminar family B ⊆ A, containing at least / B. The all singletons. All elements of B are blossoms. We have z A = 0 for all A ∈ set B is laminar and is stored by a tree-representation (cf. Proposition 2.14). For easy reference, a number is assigned to each blossom in B that is not a singleton. We store ear-decompositions of all blossoms in B at any stage of the algorithm. The variables µ(x) for x ∈ V (G) again encode the current matching M. We denote by b1 (x), . . . , bk x (x) the blossoms in B containing x, without the singleton. bk x (x) is the outermost blossom. We have variables ρ i (x) and ϕ i (x) for each x ∈ V (G)

11.3 Implementation of the Weighted Matching Algorithm

263

and i = 1, . . . , k x . ρ i (x) is the base of the blossom bi (x). µ(x) and ϕ j (x), for all x and j with b j (x) = i, are associated with an M-alternating ear-decomposition of blossom i. Of course, we must update the blossom structures (ϕ and ρ) after each augmentation. Updating ρ is easy. Updating ϕ can also be done in linear time by Lemma 10.23. For inner blossoms we need, in addition to the base, the vertex nearest to the root of the tree in the general blossom forest, and the neighbour in the next outer blossom. These two vertices are denoted by σ (x) and χ (σ (x)) for each base x of an inner blossom. See Figure 11.2 for an illustration. y0

x0

y1

x2

x1 = µ(x0 )

y2 = ρ(y0 )

x3

x4

y3 = µ(y2 )

x5 = σ (x1 )

y4

y5 = σ (y3 )

y6 = χ (y5 )

x6 = χ (x5 ) Fig. 11.2.

With these variables, the alternating paths to the root of the tree can be determined. Since the blossoms are retained after an augmentation, we must choose the augmenting path such that each blossom still contains a near-perfect matching afterwards. Figure 11.2 shows that we must be careful: There are two nested inner blossoms, induced by {x3 , x4 , x5 } and {x1 , x2 , x3 , x4 , x5 }. If we just consider the eardecomposition of the outermost blossom to ﬁnd an alternating path from x0 to the root x6 , we will end up with (x0 , x1 , x4 , x5 = σ (x1 ), x6 = χ (x5 )). After augmenting along (y6 , y5 , y4 , y3 , y2 , y1 , y0 , x0 , x1 , x4 , x5 , x6 ), the factor-critical subgraph induced by {x3 , x4 , x5 } no longer contains a near-perfect matching. Thus we must ﬁnd an alternating path within each blossom which contains an even number of edges within each sub-blossom. This is accomplished by the following procedure:

264

11 Weighted Matching

B LOSSOM PATH Input:

A vertex x0 .

Output:

An M-alternating path Q(x0 ) from x0 to ρ k x0 (x0 ).

1

Set h := 0 and B := {b j (x0 ) : j = 1, . . . , k x0 }.

2

While x2h 6= ρ k x0 (x0 ) do: Set x2h+1 := µ(x2h ) and x2h+2 := ϕ i (x2h+1 ), where i = min j ∈ {1, . . . , k x2h+1 } : b j (x2h+1 ) ∈ B . Add all blossoms of B to B that contain x2h+2 but not x2h+1 . Delete all blossoms from B whose base is x2h+2 . Set h := h + 1. Let Q(x0 ) be the path with vertices x0 , x1 , . . . , x2h .

3

Proposition 11.4. The procedure B LOSSOM PATH can be implemented in O(n) time. M4E(Q(x0 )) contains a near-perfect matching within each blossom. Proof: Let us first check that the procedure indeed computes a path. In fact, if a blossom of B is left, it is never entered again. This follows from the fact that contracting the maximal sub-blossoms of any blossom in B results in a circuit (a property which will be maintained). At the beginning of each iteration, B is the list of all blossoms that either contain x0 or have been entered via a non-matching edge and have not been left yet. The constructed path leaves any blossom in B via a matching edge. So the number of edges within each blossom is even, proving the second statement of the proposition. When implementing the procedure in O(n) time, the only nontrivial task is the update of B. We store B as a sorted list. Using the tree-representation of B and the fact that each blossom is entered and left at most once, we get a running time of O(n + |B|). Note that |B| = O(n), because B is laminar. Now determining an augmenting path consists of applying the procedure B LOS within blossoms, and using µ and χ between blossoms. When we find adjacent outer vertices x, y in different trees of the general blossom forest, we apply the following procedure to both x and y. The union of the two paths together with the edge {x, y} will be the augmenting path. SOM PATH

T REE PATH Input:

An outer vertex v.

Output:

An alternating path P(v) from v to the root of the tree in the blossom forest.

1

Let initially P(v) consist of v only. Let x := v.

2

Let y := ρ k x (x). Let Q(x) := B LOSSOM PATH(x). Append Q(x) to P(v). If µ(y) = y then stop.

11.3 Implementation of the Weighted Matching Algorithm

3

265

Set P(v) := P(v) + {y, µ(y)}. Let Q(σ (µ(y))) := B LOSSOM PATH(σ (µ(y))). Append the reverse of Q(σ (µ(y))) to P(v). Let P(v) := P(v) + {σ (µ(y)), χ (σ (µ(y)))}. Set x := χ (σ (µ(y))) and go to . 2

The second main problem is how to determine ε efficiently. The general blossom forest, after all possible grow-, shrink- and augment-steps are done, yields the GallaiEdmonds decomposition W, X, Y of G z . W contains the out-of-forest blossoms, X contains the inner blossoms, and Y consists of the outer blossoms. For a simpler notation, let us define c({v, w}) := ∞ if {v, w} ∈ / E(G). Moreover, we use the abbreviation X slack(v, w) := c({v, w}) − z A. A∈A, {v,w}∈δ(A)

So {v, w} is a tight edge if and only if slack(v, w) = 0. Then let ε1 := min{z A : A is a maximal inner blossom, |A| > 1}; ε2 := min {slack(x, y) : x outer, y out-of-forest} ; 1 min {slack(x, y) : x, y outer, belonging to different blossoms} ; ε3 := 2 ε := min{ε1 , ε2 , ε3 }. This ε is the maximum number such that the dual change by ε preserves dual feasibility. If ε = ∞, (11.2) is unbounded and so (11.1) is infeasible. In this case G has no perfect matching. Obviously, ε can be computed in finite time. However, in order to obtain an O(n 3 ) overall running time we must be able to compute ε in O(n) time. This is easy as far as ε1 is concerned, but requires additional data structures for ε2 and ε3 . For A ∈ B let X zB. ζ A := B∈B:A⊆B

We shall update these values whenever changing the dual solution; this can easily be done in linear time (using the tree-representation of B). Then ε2 = min c({x, y}) − ζ{x} − ζ{y} : x outer, y out-of-forest , 1 min c({x, y}) − ζ{x} − ζ{y} : x, y outer, belonging to different ε3 = 2 blossoms . To compute ε2 , we store for each out-of-forest vertex v the outer neighbour w for which slack(v, w) = c({v, w})−ζ{v} −ζ{w} is minimum. We call this neighbour

266

11 Weighted Matching

τv . These variables are updated whenever necessary. Then it is easy to compute ε2 = min{c({v, τv }) − ζ{v} − ζ{τv } : v out-of-forest}. To compute ε3 , we introduce variables tvA and τvA for each outer vertex v and each A ∈ B, unless A is outer but not maximal. τvA is the vertex in A minimizing slack(v, τvA ), and tvA = slack(v, τvA ) + 1 + ζ A , where 1 denotes the sum of the ε-values in all dual changes. Although when computing ε3 we are interested only in the values tvA for maximal outer blossoms of B, we update these variables also for inner and out-of-forest blossoms (even those that are not maximal), because they may become maximal outer later. Blossoms that are outer but not maximal will not become maximal outer before an augmentation takes place. After each augmentation, however, all these variables are recomputed. The variable tvA has the value slack(v, τvA ) + 1 + ζ A at any time. Observe that this value does not change as long as v remains outer, A ∈ B, and τvA is the vertex / A, v outer}. in A minimizing slack(v, τvA ). Finally, we write t A := min{tvA : v ∈ We conclude that ε3 =

1 1 1 slack(v, τvA ) = (tvA − 1 − ζ A ) = (t A − 1 − ζ A ), 2 2 2

where A is a maximal outer element of B for which t A − ζ A is minimum, and v is some outer vertex with v ∈ / A and tvA = t A . At certain stages we have to update τvA and tvA for a certain vertex v and all A ∈ B (except those that are outer but not maximal), for example if a new vertex becomes outer. The following procedure also updates the variables τw for out-of-forest vertices w if necessary.

U PDATE Input:

An outer vertex v.

Output:

Updated values of τvA , tvA and t A for all A ∈ B and τw for all out-offorest vertices w.

1

For each neighbour w of v that is out-of-forest do: If c({v, w}) − ζ{v} < c({w, τw }) − ζ{τw } then set τw := v.

2

For each x ∈ V (G) do: Set τv

3

For A ∈ B with |A| > 1 do: 0 0 Set inductively τvA := τvA and tvA := tvA − ζ A0 + ζ A , where A0 is the 0 one among the maximal proper subsets of A in B for which tvA − ζ A0 is minimum. For A ∈ B with v ∈ / A, except those that are outer but not maximal, do: Set t A := min{t A , tvA }.

4

{x}

{x}

:= x and tv

:= c({v, x}) − ζ{v} + 1.

Obviously this computation coincides with the above definition of τvA and tvA . It is important that this procedure runs in linear time:

11.3 Implementation of the Weighted Matching Algorithm

267

Lemma 11.5. If B is laminar, the procedure U PDATE can be implemented with O(n) time. Proof: By Proposition 2.15, a laminar family of subsets of V (G) has cardinality at most 2|V (G)| = O(n). If B is stored by its tree-representation, then a linear-time implementation is easy. We can now go ahead with the formal description of the algorithm. Instead of identifying inner and outer vertices by the µ-, φ- and ρ-values, we directly mark each vertex with its status (inner, outer or out-of-forest).

W EIGHTED M ATCHING A LGORITHM Input:

A graph G, weights c : E(G) → R.

Output:

A minimum weight perfect matching in G, given by the edges {x, µ(x)}, or the answer that G has no perfect matching.

1

2

3

4

5

6

Set B := {{v} : v ∈ V (G)} and K := 0. Set 1 := 0. Set z {v} := 21 min{c(e) : e ∈ δ(v)} and ζ{v} := z {v} for all v ∈ V (G). Set kv := 0, µ(v) := v, ρ 0 (v) := v, and ϕ 0 (v) := v for all v ∈ V (G). Mark all vertices as outer. For all v ∈ V (G) do: Set scanned(v) := false. For each out-of-forest vertex v do: Let τv be an arbitrary outer vertex. Set t A := ∞ for all A ∈ B. For all outer vertices v do: U PDATE(v). If all outer vertices are scanned then go to , 8 else let x be an outer vertex with scanned(x) = false. Let y be a neighbour of x such that {x, y} is tight and either y is out-of-forest or (y is outer and ρ k y (y) 6= ρ k x (x)). If there is no such y then set scanned(x) := true and go to . 3 If y is not out-of-forest then go to , 6 else: (“grow”) Set σ (ρ k y (y)) := y and χ (y) := x. Mark all vertices v with ρ kv (v) = ρ k y (y) as inner. Mark all vertices v with µ(ρ kv (v)) = ρ k y (y) as outer. For each new outer vertex v do: U PDATE(v). Go to . 4 Let P(x) := T REE PATH(x) be given by (x = x0 , x1 , x2 , . . . , x2h ). Let P(y) := T REE PATH(y) be given by (y = y0 , y1 , y2 , . . . , y2 j ).

268

7

8

11 Weighted Matching

If P(x) and P(y) are not vertex-disjoint then go to , 7 else: (“augment”) For i := 0 to h − 1 do: Set µ(x2i+1 ) := x2i+2 and µ(x2i+2 ) := x2i+1 . For i := 0 to j − 1 do: Set µ(y2i+1 ) := y2i+2 and µ(y2i+2 ) := y2i+1 . Set µ(x) := y and µ(y) := x. Mark all vertices v such that the endpoint of T REE PATH(v) is either x2h or y2 j as out-of-forest. Update all values ϕ i (v) and ρ i (v) for these vertices (using Lemma 10.23). If µ(v) 6= v for all v then stop, else go to . 2 (“shrink”) Let r = x2h 0 = y2 j 0 be the first outer vertex of V (P(x)) ∩ V (P(y)) with ρ kr (r ) = r . Let A := {v ∈ V (G) : ρ kv (v) ∈ V (P(x)[x,r ] ) ∪ V (P(y)[y,r ] )}. Set K := K + 1, B := B ∪ {A}, z A := 0 and ζ A := 0. For all v ∈ A do: Set kv := kv + 1, bkv (v) := K , ρ kv (v) := r , ϕ kv (v) := ϕ kv −1 (v) and mark v as outer. For i := 1 to h 0 do: If ρ k x2i (x2i ) 6= r then set ϕ k x2i (x2i ) := x2i−1 . If ρ k x2i−1 (x2i−1 ) 6= r then set ϕ k x2i−1 (x2i−1 ) := x2i . For i := 1 to j 0 do: If ρ k y2i (y2i ) 6= r then set ϕ k y2i (y2i ) := y2i−1 . If ρ k y2i−1 (y2i−1 ) 6= r then set ϕ k y2i−1 (y2i−1 ) := y2i . If ρ k x (x) 6= r then set ϕ k x (x) := y. If ρ k y (y) 6= r then set ϕ k y (y) := x. For each outer vertex v do: 0 0 Set tvA := tvA − ζ A0 and τvA := τvA , where A0 is the one among the 0 maximal proper subsets of A in B for which tvA − ζ A0 is minimum. ¯ Set t A := min{tvA : v outer, there is no A¯ ∈ B with A ∪ {v} ⊆ A}. For each new outer vertex v do: U PDATE(v). Go to . 4 (“dual change”) Set ε1 := min{z A : A maximal inner element of B, |A| > 1}. Set ε2 := min{c({v, τv }) − ζ{v} − ζ{τv } : v out-of-forest}. Set ε3 := min{ 21 (t A − 1 − ζ A ) : A maximal outer element of B}. Set ε := min{ε1 , ε2 , ε3 }. If ε = ∞, then stop (G has no perfect matching). If ε = ε2 = c({v, τv }) − ζ{v} − ζ{τv } , v outer then set scanned(τv ) := false. If ε = ε3 = 21 (tvA − 1 − ζ A ), A maximal outer element of B, v outer and v ∈ / A then set scanned(v) := false. For each maximal outer element A of B do: Set z A := z A + ε and ζ A0 := ζ A0 + ε for all A0 ∈ B with A0 ⊆ A. For each maximal inner element A of B do: Set z A := z A − ε and ζ A0 := ζ A0 − ε for all A0 ∈ B with A0 ⊆ A. Set 1 := 1 + ε.

11.3 Implementation of the Weighted Matching Algorithm

9

269

While there is a maximal inner A ∈ B with z A = 0 and |A| > 1 do: (“unpack”) Set B := B \ {A}. Let y := σ (ρ kv (v)) for some v ∈ A. Let Q(y) := B LOSSOM PATH(y) be given by (y = r0 , r1 , r2 , . . . , r2l−1 , r2l = ρ k y (y)). Mark all v ∈ A with ρ kv −1 (v) ∈ / V (Q(y)) as out-of-forest. Mark all v ∈ A with ρ kv −1 (v) = r2i−1 for some i as outer. For all v ∈ A with ρ kv −1 (v) = r2i for some i (v remains inner) do: Set σ (ρ kv (v)) := r j and χ (r j ) := r j−1 , where kr −1

j := min{j ∈ {0, . . . , 2l} : ρ j (r j ) = ρ kv −1 (v)}. For all v ∈ A do: Set kv := kv − 1. For each new out-of-forest vertex v do: Let τv be an outer vertex w for which c({v, w}) − ζ{v} − ζ{w} is minimum. For each new outer vertex v do: U PDATE(v). Go to . 3 Note that in contrast to our previous discussion, ε = 0 is possible. The variables τvA are not needed explicitly. The “unpack”-step 9 is illustrated in Figure 11.3, where a blossom with 19 vertices is unpacked. Two of the ﬁve sub-blossoms become out-of-forest, two become inner blossoms and one becomes an outer blossom. In , 6 the connected components of the blossom forest F have to be determined. This can be done in linear time by Proposition 2.17. (a)

(b) r10 r9

r8

r6

r7 r5 r3

r2

r4 y = r0

r1

Fig. 11.3.

270

11 Weighted Matching

(a) a 4

(b)

b

8

8

c

10 4

11

a 8

14 13

8

8

9 g

12

12

c

0

5

2 d

h

13

10

0 4

0 d

h

b

2

4 5

e

0

0

3 g

e

2

2

5

4

f

f Fig. 11.4.

Before analysing the algorithm, let us demonstrate its major steps by an example. Consider the graph in Figure 11.4(a). Initially, the algorithm sets z {a} = z {d} = z {h} = 2, z {b} = x{c} = z { f } = 4 and z {e} = z {g} = 6. In Figure 11.4(b) the slacks can be seen. So in the beginning the edges {a, d}, {a, h}, {b, c}, {b, f }, {c, f } are tight. We assume that the algorithm scans the vertices in alphabetical order. So the ﬁrst steps are augment(a, d),

augment(b, c),

grow( f, b).

Figure 11.5(a) shows the current general blossom forest. The next steps are shrink( f, c),

grow(h, a),

resulting in the general blossom forest shown in Figure 11.5(b). Now all the tight edges are used up, so the dual variables have to change. We perform 8 and obtain ε = ε3 = 1, say A = {b, c, f } and τvA = d. The new dual variables are z {b,c, f } = 1, z {a} = 1, z {d} = z {h} = 3, z {b} = z {c} = z { f } = 4, z {e} = z {g} = 7. The current slacks are shown in Figure 11.6(a). The next step is augment(d, c). (a)

(b) c

d

e

f

c

b

b

g

h

a

d Fig. 11.5.

e

f

a

g

h

11.3 Implementation of the Weighted Matching Algorithm (a) a 0

(b)

b

2

0

c

4 3

0

0

b

2 3

0

0

1 g

0

3

2

f g

e

e

0

c

d

d

h

271

a

h

f Fig. 11.6.

The blossom {b, c, f } becomes out-of-forest (Figure 11.6(b)). Now the edge {e, f } is tight, but in the previous dual change we have only set scanned(d) := false. So we need to do 8 with ε = ε3 = 0 twice to make the next steps grow(e, f ),

grow(d, a)

possible. We arrive at Figure 11.7(a). No more edges incident to outer vertices are tight, so we perform 8 once more. We have ε = ε1 = 1 and obtain the new dual solution z {b,c, f } = 0, z {a} = 0, z {d} = z {h} = z {b} = x{c} = z { f } = 4, z {e} = z {g} = 8. The new slacks are shown in Figure 11.7(b). Since the dual variable for the inner blossom {B, C, F} becomes zero, we have to unpack({b, c, f }). (a)

(b)

h

a

a 0 h

0 0

g Fig. 11.7.

0

1 0

0 f

e

0 d

2

g

f

c

6

1

b

0

3

d

c

b

4

1

2 e

272 (a)

11 Weighted Matching (b)

h

a

b

4

0

a d

0

c

0

f

e

d 2

0

1

1 g

0

0

2

h

b

c

7

0

0

0

2 e

f

g Fig. 11.8.

The general blossom forest we get is shown in Figure 11.8(a). After another dual variable change with ε = ε3 = 12 we obtain z {a} = −0.5, z {c} = z { f } = 3.5, z {b} = z {d} = z {h} = 4.5, z {e} = z {g} = 8.5 (the slacks are shown in Figure 11.8(b)). The ﬁnal steps are shrink(d, e), augment(g, h), and the algorithm terminates. The ﬁnal matching is M = {{e, f }, {b, c}, {a, d}, {g, h}}. We check that M has total weight 37, equal to the sum of the dual variables. Let us now check that the algorithm works correctly. Proposition 11.6. The following statements hold at any stage of the W EIGHTED M ATCHING A LGORITHM: (a) B is a laminar family. B = {v ∈ V (G) : bi (v) = j for some i} : j = 1, . . . , B . The sets Vρ kr (r ) := {v : ρ kv (v) = ρ kr (r )} are exactly the maximal elements of B. The vertices in each Vr are marked either all outer or all inner or all out-of-forest. Each (Vr , {{v, ϕ kv (v)} : v ∈ Vr \ {r }} ∪ {{v, μ(v)} : v ∈ Vr \ {r }}) is a blossom with base r . (b) The edges {x, μ(x)} form a matching M. M contains a near-perfect matching within each element of B. (c) For each b ∈ {1, . . . , K } let X (b) := {v ∈ V (G) : bi (v) = b for some i}. Then the variables μ(v) and ϕ i (v), for those v and i with bi (v) = b, are associated with an M-alternating ear-decomposition in G[X (b)]. (d) The edges {x, μ(x)} and {x, ϕ i (x)} for all x and i, and the edges {σ (x), χ (σ (x))} for all bases x of maximal inner blossoms, are all tight.

11.3 Implementation of the Weighted Matching Algorithm

273

(e) The edges {x, µ(x)}, {x, ϕ k x (x)} for all inner or outer x, together with the edges {σ (x), χ (σ (x))} for all bases x of maximal inner blossoms, form a general blossom forest F with respect to M. The vertex marks (inner, outer, out-of-forest) are consistent with F. (f) Contracting the maximal sub-blossoms of any blossom in B results in a circuit. (g) For each outer vertex v, the procedure T REE PATH gives an M-alternating v-r -path, where r is the root of the tree in F containing v. Proof: The properties clearly hold at the beginning (after 2 is executed the first time). We show that they are maintained throughout the algorithm. This is easily seen for (a) by considering 7 and . 9 For (b), this follows from Proposition 11.4 and the assumption that (f) and (g) hold before augmenting. The proof that (c) continues to hold after shrinking is the same as in the non-weighted case (see Lemma 10.30 (c)). The ϕ-values are recomputed after augmenting and not changed elsewhere. (d) is guaranteed by . 4 It is easy to see that (e) is maintained by : 5 The blossom containing y was outof-forest, and setting χ (y) := x and σ (v) := y for the base v of the blossom makes it inner. The blossom containing µ(ρ k y (y)) was also out-of-forest, and becomes outer. In , 6 two connected components of the general blossom forest clearly become out-of-forest, so (e) is maintained. In , 7 the vertices in the new blossom clearly become outer because r was outer before. In , 9 for the vertices v ∈ A with / V (Q(y)) we also have µ(ρ kv (v)) ∈ / V (Q(y)), so they become out-ofρ kv −1 (v) ∈ forest. For each other v ∈ A we have ρ kv −1 (v) = rk for some k. Since {ri , ri+1 } ∈ M iff i is even, v becomes outer iff k is odd. (f) holds for any blossom, as any new blossom arises from an odd circuit in

. 7 To see that (g) is maintained, it suffices to observe that σ (x) and χ (σ (x)) are set correctly for all bases x of maximal inner blossoms. This is easily checked for both 5 and . 9 Proposition 11.6(a) justifies calling the maximal elements of B inner, outer or out-of-forest in 8 and 9 of the algorithm. Next we show that the algorithm maintains a feasible dual solution. Lemma 11.7. At any stage of the algorithm, z is a feasible dual solution. If ε = ∞ then G has no perfect matching. Proof: We always have z A = 0 for all A ∈ A \ B. z A is decreased only for those A ∈ B that are maximal in B and inner. So the choice of ε1 guarantees that z A continues to be nonnegativePfor all A with |A| > 1. P How can the constraints A∈A:e∈δ(A) z A ≤ c(e) be violated? If A∈A:e∈δ(A) z A increases in , 8 e must either connect an outer and an out-of-forest vertex or two different outer blossoms. So the maximal ε such that the new z still satisfies P 1 z A∈A:e∈δ(A) A ≤ c(e) is slack(e) in the first case and 2 slack(e) in the second case.

274

11 Weighted Matching

We thus have to prove that ε2 and ε3 are computed correctly: ε2 = min{slack(v, w) : v outer, w out-of-forest} and

1 min{slack(v, w) : v, w outer, ρ kv (v) 6= ρ kw (w)}. 2 For ε2 this is easy to see, since for any out-of-forest vertex v we always have that τv is the outer vertex w minimizing slack(v, w) = c({v, w}) − ζ{v} − ζ{w} . Now consider ε3 . We claim that at any stage of the algorithm the following holds for any outer vertex v and any A ∈ B such that there is no A¯ ∈ B with ¯ A ∪ {v} ⊆ A: ε3 =

(a) (b) (c) (d) (e)

τvA ∈ A. A slack(v, P τv ) = min{slack(v, u) : u ∈ A}. ζ A = B∈B:A⊆B z B . 1 is the sum of the ε-values in all dual changes so far. slack(v, τvA ) = tvA − 1 − ζ A . ¯ t A = min{tvA : v outer and there is no A¯ ∈ B with A ∪ {v} ⊆ A}.

(a), (c), and (e) are easily seen to be true. (b) and (d) hold when τvA is defined (in 7 or in U PDATE(v)), and afterwards slack(v, u) decreases exactly by the amount that 1 + ζ A increases (due to (c)). Now (a), (b), (d), and (e) imply that ε3 is computed correctly. Now suppose ε = ∞, i.e. ε can be chosen arbitrarily large without destroying dual feasibility. Since the dual objective 1lz increases by at least ε in , 8 we conclude that the dual LP (11.2) is unbounded. Hence by Theorem 3.27 the primal LP (11.1) is infeasible. Now the correctness of the algorithm follows: Theorem 11.8. If the algorithm terminates in , 6 the edges {x, µ(x)} form a minimum weight perfect matching in G. Proof: Let x be the incidence vector of M (the matching consisting of the edges {x, µ(x)}). The complementary slackness conditions X z A = c(e) xe > 0 ⇒ A∈A:e∈δ(A)

zA > 0 ⇒

X

xe = 1

e∈δ(A)

are satisfied: the first one holds since all the matching edges are tight (Proposition 11.6(d)), and the second one follows from Proposition 11.6(b). Since we have feasible primal and dual solutions (Lemma 11.7), both must be optimal (Corollary 3.23). So x is optimal for the LP (11.1) and integral, proving that M is a minimum weight perfect matching. Until now we have not proved that the algorithm terminates.

11.3 Implementation of the Weighted Matching Algorithm

275

Theorem 11.9. The running time of the W EIGHTED M ATCHING A LGORITHM between two augmentations is O(n 2 ). The overall running time is O(n 3 ). Proof: By Lemma 11.5 and Proposition 11.6(a), the U PDATE procedure runs in linear time. Both 2 and 6 take O(n 2 ) time, once per augmentation. Each of , 5 , 7 and 9 can be done in O(nk) time, where k is the number of new outer vertices. (In , 7 the number of maximal proper subsets A0 of A to be considered is at most 2k + 1: every second sub-blossom of a new blossom must have been inner.) Since an outer vertex continues to be outer until the next augmentation, the total time spent by , 5 , 7 and 9 between two augmentations is O(n 2 ). It remains to estimate the running time of , 8 , 3 and . 4 Suppose in 8 we have ε 6= ε1 . Due to the variables tv and tvA we then obtain a new tight edge in . 8 We continue in 3 and , 4 where after at most O(n) time this edge is checked. Since it either connects an outer vertex with an out-of-forest vertex or two different outer connected components, we can apply one of , 5 , 6 . 7 If ε = ε1 we have to apply . 9 This consideration shows that the number of times 8 is executed is less than or equal to the number of times one of , 5 , 6 , 7 9 is executed. Since 8 takes only O(n) time, the O(n 2 ) bound between two augmentations is proved. Note that the case ε = 0 is not excluded. Since there are only n2 augmentations, the total running time is O(n 3 ). Corollary 11.10. The M INIMUM W EIGHT P ERFECT M ATCHING P ROBLEM can be solved in O(n 3 ) time. Proof:

This follows from Theorems 11.8 and 11.9.

The first O(n 3 )-implementation of Edmonds’ algorithm for the M INIMUM W EIGHT P ERFECT M ATCHING P ROBLEM was due to Gabow [1973] (see also Gabow [1976] and Lawler [1976]). The theoretically best running time, namely O(mn + n 2 log n), has also been obtained by Gabow [1990]. 3 For planar graphs a minimum weight perfect matching can be found in O n 2 log n time, as Lipton and Tarjan [1979,1980] showed by a divide and conquer approach, using the fact that planar graphs have small “separators”. For Euclidean instances (a set of points in the plane defining a complete graph whose edge weights are given by the 3 Euclidean distances) Varadarajan [1998] found an O n 2 log5 n algorithm. Probably the currently most efficient implementations are described by Mehlhorn and Sch¨afer [2000] and Cook and Rohe [1999]. They solve matching problems with millions of vertices optimally. A “primal” weighted matching algorithm – always maintaining a perfect matching and obtaining a feasible dual solution only at termination – was described by Cunningham and Marsh [1978].

276

11 Weighted Matching

11.4 Postoptimality In this section we prove two postoptimality results which we shall need in Section 12.2. Lemma 11.11. (Weber [1981], Ball and Derigs [1983]) Suppose we have run the W EIGHTED M ATCHING A LGORITHM for an instance (G, c). Let s ∈ V (G), and let c0 : E(G) → R with c0 (e) = c(e) for all e 6∈ δ(s). Then a minimum weight perfect matching with respect to (G, c0 ) can be determined in O(n 2 ) time. Proof: Let t := µ(s). If s is not contained in any nontrivial blossom, i.e. ks = 0, then the first step just consists of setting µ(s) := s and µ(t) := t. Otherwise we have to unpack all the blossoms P containing s. To accomplish this, we shall perform dual changes of total value A: s∈A, |A|>1 z A while s is inner all the time. Consider the following construction: .

E(G) ∪ {{a, s}, {b, t}}. Set V (G) := V (G) ∪ {a, b} and E(G) :=X z A + ζ{t} . Set c({a, s}) := ζ{s} and c({b, t}) := 2 A: s∈A, |A|>1

Set µ(a) := a and µ(b) := b. Mark a and b as outer. Set B := B ∪ {{a}, {b}}, z {a} := 0, z {b} := 0, ζ{a} := 0, ζ{b} := 0. Set ka := 0, kb := 0, ρ 0 (a) := a, ρ 0 (b) := b, ϕ 0 (a) := a, ϕ 0 (b) := b. U PDATE(a). U PDATE(b). The result is a possible status if the algorithm was applied to the modified instance (the graph extended by two vertices and two edges). In particular, the dual solution z is feasible. Moreover, the edge {a, s} is tight. Now we set scanned(a) := false and continue the algorithm starting with . 3 The algorithm will grow the forest by the edge {a, s} next, and s becomes inner. By Theorem 11.9 the algorithm terminates after O(n 2 ) steps with an augmentation. The only possible augmenting path is a, Ps, t, b. So the edge {b, t} must become tight. At the beginning, slack(b, t) = 2 A∈A, s∈A, |A|>1 z A . Vertex s will remain inner throughout. So ζ{s} will decrease at each dual change. Thus all blossoms A containing s are unpacked at the end. We finally delete the vertices a and b and the edges {a, s} and {b, t}, and set B := B \ {{a}, {b}} and µ(s) := s, µ(t) := t. Now s and t are outer, and there are no inner vertices. Furthermore, no edge incident to s belongs to the general blossom forest. So we can easily change weights of edges incident to s as well as z {s} , as long as we maintain the dual feasibility. This, however, is easily guaranteed by first computing the slacks according to the new edge weights and then increasing z {s} by mine∈δ(s) slack(e). We set scanned(s) := false and continue the algorithm starting with . 3 By Theorem 11.9, the algorithm will terminate after O(n 2 ) steps with a minimum weight perfect matching with respect to the new weights. The same result for a “primal” weighted matching algorithm was found by Cunningham and Marsh [1978]. The following lemma deals with the addition of two vertices to an instance that has already been solved.

11.5 The Matching Polytope

277

Lemma 11.12. Let (G, c) be an instance of the M INIMUM W EIGHT P ERFECT M ATCHING P ROBLEM, and let s, t ∈ V (G). Suppose we have run the W EIGHTED M ATCHING A LGORITHM for the instance (G − {s, t}, c). Then a minimum weight perfect matching with respect to (G, c) can be determined in O(n 2 ) time. Proof: The addition of two vertices requires the initialization of the data structures (as in the previous proof). The dual variable z v is set such that mine∈δ(v) slack(e) = 0 (for v ∈ {s, t}). Then setting scanned(s) := scanned(t) := false and starting the W EIGHTED M ATCHING A LGORITHM with 3 does the job.

11.5 The Matching Polytope The correctness of the W EIGHTED M ATCHING A LGORITHM also yields Edmonds’ characterization of the perfect matching polytope as a by-product. We again use the notation A := {A ⊆ V (G) : |A| odd}. Theorem 11.13. (Edmonds [1965]) Let G be an undirected graph. The perfect matching polytope of G, i.e. the convex hull of the incidence vectors of all perfect matchings in G, is the set of vectors x satisfying

X

xe ≥ 0

(e ∈ E(G))

xe = 1

(v ∈ V (G))

xe ≥ 1

(A ∈ A)

e∈δ(v)

X e∈δ(A)

Proof: By Corollary 3.32 it suffices to show that all vertices of the polytope described above are integral. By Theorem 5.13 this is true if the minimization problem has an integral optimum solution for any weight function. But our W EIGHTED M ATCHING A LGORITHM finds such a solution for any weight function (cf. the proof of Theorem 11.8). An alternative proof will be given in Section 12.3 (see the remark after Theorem 12.16). We can also describe the matching polytope, i.e. the convex hull of the incidence vectors of all matchings in an undirected graph G: Theorem 11.14. (Edmonds [1965]) Let G be a graph. The matching polytope E(G) satisfying of G is the set of vectors x ∈ R+ X e∈δ(v)

xe ≤ 1 for all v ∈ V (G)

and

X e∈E(G[A])

xe ≤

|A| − 1 for all A ∈ A. 2

278

11 Weighted Matching

Proof: Since the incidence vector of any matching obviously satisﬁes these inE(G) be a vector with equalities, we only have to prove one direction. Let x ∈ R+ |A|−1 x ≤ 1 for v ∈ V (G) and x ≤ for A ∈ A. We prove e∈δ(v) e e∈E(G[A]) e 2 that x is a convex combination of incidence vectors of matchings. Let H be the graph with V (H ) := {(v, i) : v ∈ V (G), i ∈ {1, 2}}, and E(H ) := {{(v, i), (w, i)} : {v, w} ∈ E(G), i ∈ {1, 2}} ∪ {{(v, 1), (v, 2)} : v ∈ V (G)}. So H consists of two copies of G, and there is an edge joining the two copies of each vertex. Let y{(v,i),(w,i)}:= xe for each e = {v, w} ∈ E(G) and i ∈ {1, 2}, and let y{(v,1),(v,2)} := 1 − e∈δG (v) xe for each v ∈ V (G). We claim that y belongs to the perfect matching polytope of H . Considering the subgraph induced by {(v, 1) : v ∈ V (G)}, which is isomorphic to G, we then get that x is a convex combination of incidence vectors of matchings in G. V (G)

{(v, 1) : v ∈ V (G)}

{(v, 2) : v ∈ V (G)}

A

A1

A2

B

B1

B2

C

:X Fig. 11.9.

E(H ) Obviously, y ∈ R+ and e∈δ H (v) ye = 1 for all v ∈ V (H ). To show that y belongs to the perfect matching polytope of H , we use Theorem 11.13. So let X ⊆ V (H ) with |X | odd. We prove that e∈δ H (X ) ye ≥ 1. Let A := {v ∈ V (G) : (v, 1) ∈ X, (v, 2) ∈ / X }, B := {v ∈ V (G) : (v, 1) ∈ X, (v, 2) ∈ X } and C := {v ∈ V (G) : (v, 1) ∈ / X, (v, 2) ∈ X }. Since |X | is odd, either A or C must have odd cardinality, w.l.o.g. |A| is odd. We write Ai := {(a, i) : a ∈ A} and Bi := {(b, i) : b ∈ B} for i = 1, 2 (see Figure 11.9). Then ye ≥ ye − 2 ye − ye + ye v∈A1 e∈δ H (v)

e∈δ H (X )

=

v∈A1 e∈δ H (v)

e∈E(H [A1 ])

ye − 2

e∈E H (A1 ,B1 )

e∈E H (B2 ,A2 )

xe

e∈E(G[A])

≥ |A1 | − (|A| − 1) = 1.

11.5 The Matching Polytope

279

Indeed, we can prove the following stronger result: Theorem 11.15. (Cunningham and Marsh [1978]) For any undirected graph G the linear inequality system

X e∈δ(v) X

(e ∈ E(G)) (v ∈ V (G))

xe ≥ 0 xe ≤ 1 |A|−1 2

xe ≤

(A ∈ A, |A| > 1)

e⊆A

is TDI. Proof: For c : E(G) → Z we consider the LP max the above constraints. The dual LP is: min

X v∈V (G)

s.t.

X v∈e

X

yv +

yv +

A∈A, |A|>1

X

P

e∈E(G) c(e)x e

subject to

|A| − 1 zA 2

z A ≥ c(e)

(e ∈ E(G))

yv ≥ 0 zA ≥ 0

(v ∈ V (G)) (A ∈ A, |A| > 1)

A∈A, e⊆A

Let (G, c) be the smallest counterexample, i.e. there is no integral optimum P dual solution and |V (G)| + |E(G)| + e∈E(G) |c(e)| is minimum. Then c(e) ≥ 1 for all e (otherwise we can delete any edge of nonpositive weight), and G has no isolated vertices (otherwise we can delete them). Moreover, for any optimum solution y, z we claim that y = 0. To prove this, suppose P yv > 0 for some v ∈ V (G). Then by complementary slackness (Corollary 3.23) e∈δ(v) xe = 1 for every primal optimum solution x. But then decreasing c(e) by one for each e ∈ δ(v) yields a smaller instance (G, c0 ), whose optimum LP value is one less (here we use primal integrality, i.e. Theorem 11.14). Since (G, c) is the smallest counterexample, there exists an integral optimum dual solution y 0 , z 0 for (G, c0 ). Increasing yv0 by one yields an integral optimum dual solution for (G, c), a contradiction. Now let y = 0 and z be an optimum dual solution for which X |A|2 z A (11.4) A∈A, |A|>1

is as large as possible. We claim that F := {A : z A > 0} is laminar. To see this, suppose there are sets X, Y ∈ F with X \ Y 6= ∅, Y \ X 6= ∅ and X ∩ Y 6= ∅. Let := min{z X , z Y } > 0. If |X ∩ Y | is odd, then |X ∪ Y | is also odd. Set z 0X := z X − , z Y0 := z Y − , 0 z X ∩Y := z X ∩Y + (unless |X ∩ Y | = 1), z 0X ∪Y := z X ∪Y + and z 0A := z A for all

280

11 Weighted Matching

other sets A. y, z 0 is also a feasible dual solution; moreover it is optimum as well. This is a contradiction since (11.4) is larger. If |X ∩ Y | is even, then |X \ Y | and |Y \ X | are odd. Set z 0X := z X − , 0 z Y := z Y − , z 0X \Y := z X \Y + (unless |X \ Y | = 1), z Y0 \X := z Y \X + (unless |Y \ X | = 1) and z 0A := z A for all other sets A. Set yv0 := yv + for v ∈ X ∩ Y and / X ∩ Y . Then y 0 , z 0 is a feasible dual solution that is also optimum. yv0 := yv for v ∈ This contradicts the fact that any optimum dual solution must have y = 0. / Z and A maximal. Set := z A − bz A c > 0. Now let A ∈ F with z A ∈ Let A1 , . . . , Ak be the maximal proper subsets of A in F; they must be disjoint because F is laminar. Setting z 0A := z A − and z 0Ai := z Ai + for i = 1, . . . , k (and z 0D := z D for all other D ∈ A) yields another feasible dual solution y = 0, z 0 (since c is integral). We have X X |B| − 1 0 |B| − 1 zB < zB, 2 2 B∈A, |B|>1

B∈A, |B|>1

contradicting the optimality of the original dual solution y = 0, z.

This proof is due to Schrijver [1983a]. For different proofs, see Lov´asz [1979] and PThe latter does not use Theorem 11.14. Moreover, replacing P Schrijver [1983b]. x ≤ 1 by e e∈δ(v) e∈δ(v) x e = 1 for v ∈ V (G) in Theorem 11.15 yields an alternative description of the perfect matching polytope, which is also TDI (by Theorem 5.18). Theorem 11.13 can easily be derived from this; however, the linear inequality system of Theorem 11.13 is not TDI in general (K 4 is a counterexample). Theorem 11.15 also implies the Berge-Tutte formula (Theorem 10.14; see Exercise 14). Generalizations will be discussed in Section 12.1.

Exercises 1. Use Theorem 11.2 to prove a weighted version of K¨onig’s Theorem 10.2. (Egerv´ary [1931]) 2. Describe the convex hull of the incidence vectors of all (a) vertex covers, (b) stable sets, (c) edge covers, in a bipartite graph G. Show how Theorem 10.2 and the statement of Exercise 2(c) of Chapter 10 follow. Hint: Use Theorem 5.25 and Corollary 5.21. 3. Prove the Birkhoff-von-Neumann Theorem 11.3 directly. 4. Let G be a graph and P the fractional perfect matching polytope of G. Prove that the vertices of P are exactly the vectors x with 1 2 if e ∈ E(C1 ) ∪ · · · ∪ E(Ck ) , xe = 1 if e ∈ M 0 otherwise

Exercises

5.

6. 7.

8.

9.

10. 11.

12.

13.

∗ 14.

281

where C1 , . . . , Ck are vertex-disjoint odd circuits and M is a perfect matching in G − (V (C1 ) ∪ · · · ∪ V (Ck )). (Balinski [1972]; see Lov´asz [1979]) . Let G be a bipartite graph with bipartition V = A ∪ B and A = {a1 , . . . , a p }, B = {b1 , . . . , bq }. Let c : E(G) → R be weights on the edges. We look for the maximum weight order-preserving matching M, i.e. for any two edges {ai , b j }, {ai 0 , b j 0 } ∈ M with i < i 0 we require j < j 0 . Solve this problem with an O(n 3 )-algorithm. Hint: Use dynamic programming. Prove that, at any stage of the W EIGHTED M ATCHING A LGORITHM, |B| ≤ 32 n. Let G be a graph with nonnegative weights c : E(G) → R+ . Let M be the matching at any intermediate stage of the W EIGHTED M ATCHING A LGO RITHM . Let X be the set of vertices covered by M. Show that any matching covering X is at least as expensive as M. (Ball and Derigs [1983]) A graph with integral weights on the edges is said to have the even circuit property if the total weight of every circuit is even. Show that the W EIGHTED M ATCHING A LGORITHM applied to a graph with the even circuit property maintains this property (with respect to the slacks) and also maintains a dual solution that is integral. Conclude that for any graph there exists an optimum dual solution z that is half-integral (i.e. 2z is integral). When the W EIGHTED M ATCHING A LGORITHM is restricted to bipartite graphs, it becomes much simpler. Show which parts are necessary even in the bipartite case and which are not. Note: One arrives at what is called the Hungarian method for the A SSIGN MENT P ROBLEM (Kuhn [1955]). This algorithm can also be regarded as an equivalent description of the procedure proposed in the proof of Theorem 11.1. How can the bottleneck matching problem (find a perfect matching M such that max{c(e) : e ∈ M} is minimum) be solved in O(n 3 ) time? Show how to solve the M INIMUM W EIGHT E DGE C OVER P ROBLEM in polynomial time: given an undirected graph G and weights c : E(G) → R, find a minimum weight edge cover. Given an undirected graph G with weights c : E(G) → R+ and two vertices s and t, we look for a shortest s-t-path with an even (or with an odd) number of edges. Reduce this to a M INIMUM W EIGHT P ERFECT M ATCHING P ROBLEM. Hint: Take two copies of G, connect each vertex with its copy by an edge of zero weight and delete s and t (or s and the copy of t). (Gr¨otschel and Pulleyblank [1981]) Let G be a k-regular and (k − 1)-edge-connected graph with an even number of vertices, and let c : E(G) → R+ . Prove that there exists a perfect matching M in G with c(M) ≥ k1 c(E(G)). Hint: Show that k1 1l is in the perfect matching polytope. Show that Theorem 11.15 implies: (a) the Berge-Tutte formula (Theorem 10.14);

282

11 Weighted Matching

(b) Theorem 11.13; (c) the existence of an optimum half-integral dual solution to the dual LP (11.2) (cf. Exercise 8). Hint: Use Theorem 5.18. 15. The fractional perfect matching polytope Q of G is identical to the perfect matching polytope if G is bipartite (Theorem 11.2). Consider the first GomoryChv´atal-truncation Q 0 of Q (Definition 5.29). Prove that Q 0 is always identical to the perfect matching polytope.

References General Literature: Gerards, A.M.H. [1995]: Matching. In: Handbooks in Operations Research and Management Science; Volume 7: Network Models (M.O. Ball, T.L. Magnanti, C.L. Monma, G.L. Nemhauser, eds.), Elsevier, Amsterdam 1995, pp. 135–224 Lawler, E.L. [1976]: Combinatorial Optimization; Networks and Matroids. Holt, Rinehart and Winston, New York 1976, Chapters 5 and 6 Papadimitriou, C.H., and Steiglitz, K. [1982]: Combinatorial Optimization; Algorithms and Complexity. Prentice-Hall, Englewood Cliffs 1982, Chapter 11 Pulleyblank, W.R. [1995]: Matchings and extensions. In: Handbook of Combinatorics; Vol. 1 (R.L. Graham, M. Gr¨otschel, L. Lov´asz, eds.), Elsevier, Amsterdam 1995

Cited References: Balinski, M.L. [1972]: Establishing the matching polytope. Journal of Combinatorial Theory 13 (1972), 1–13 Ball, M.O., and Derigs, U. [1983]: An analysis of alternative strategies for implementing matching algorithms. Networks 13 (1983), 517–549 Birkhoff, G. [1946]: Tres observaciones sobre el algebra lineal. Revista Universidad Nacional de Tucum´an, Series A 5 (1946), 147–151 Cook, W., and Rohe, A. [1999]: Computing minimum-weight perfect matchings. INFORMS Journal of Computing 11 (1999), 138–148 Cunningham, W.H., and Marsh, A.B. [1978]: A primal algorithm for optimum matching. Mathematical Programming Study 8 (1978), 50–72 Edmonds, J. [1965]: Maximum matching and a polyhedron with (0,1) vertices. Journal of Research of the National Bureau of Standards B 69 (1965), 125–130 Egerv´ary, E. [1931]: Matrixok kombinatorikus tulajdons´agairol. Matematikai e´ s Fizikai Lapok 38 (1931), 16–28 [in Hungarian] Gabow, H.N. [1973]: Implementation of algorithms for maximum matching on non-bipartite graphs. Ph.D. Thesis, Stanford University, Dept. of Computer Science, 1973 Gabow, H.N. [1976]: An efficient implementation of Edmonds’ algorithm for maximum matching on graphs. Journal of the ACM 23 (1976), 221–234 Gabow, H.N. [1990]: Data structures for weighted matching and nearest common ancestors with linking. Proceedings of the 1st Annual ACM-SIAM Symposium on Discrete Algorithms (1990), 434–443 Gr¨otschel, M., and Pulleyblank, W.R. [1981]: Weakly bipartite graphs and the max-cut problem. Operations Research Letters 1 (1981), 23–27

References

283

Kuhn, H.W. [1955]: The Hungarian method for the assignment problem. Naval Research Logistics Quarterly 2 (1955), 83–97 Lipton, R.J., and Tarjan, R.E. [1979]: A separator theorem for planar graphs. SIAM Journal on Applied Mathematics 36 (1979), 177–189 Lipton, R.J., and Tarjan, R.E. [1980]: Applications of a planar separator theorem. SIAM Journal on Computing 9 (1980), 615–627 Lov´asz, L. [1979]: Graph theory and integer programming. In: Discrete Optimization I; Annals of Discrete Mathematics 4 (P.L. Hammer, E.L. Johnson, B.H. Korte, eds.), North-Holland, Amsterdam 1979, pp. 141–158 Mehlhorn, K., and Sch¨afer, G. [2000]: Implementation of O(nm log n) weighted matchings in general graphs: the power of data structures. In: Algorithm Engineering; WAE-2000; LNCS 1982 (S. N¨aher, D. Wagner, eds.), pp. 23–38; also electronically in The ACM Journal of Experimental Algorithmics 7 (2002) Monge, G. [1784]: M´emoire sur la th´eorie des d´eblais et des remblais. Histoire de l’Acad´emie Royale des Sciences 2 (1784), 666–704 Munkres, J. [1957]: Algorithms for the assignment and transportation problems. Journal of the Society for Industrial and Applied Mathematics 5 (1957), 32–38 von Neumann, J. [1953]: A certain zero-sum two-person game equivalent to the optimal assignment problem. In: Contributions to the Theory of Games II; Ann. of Math. Stud. 28 (H.W. Kuhn, ed.), Princeton University Press, Princeton 1953, pp. 5–12 Schrijver, A. [1983a]: Short proofs on the matching polyhedron. Journal of Combinatorial Theory B 34 (1983), 104–108 Schrijver, A. [1983b]: Min-max results in combinatorial optimization. In: Mathematical Programming; The State of the Art – Bonn 1982 (A. Bachem, M. Gr¨otschel, B. Korte, eds.), Springer, Berlin 1983, pp. 439–500 Varadarajan, K.R. [1998]: A divide-and-conquer algorithm for min-cost perfect matching in the plane. Proceedings of the 39th Annual IEEE Symposium on Foundations of Computer Science (1998), 320–329 Weber, G.M. [1981]: Sensitivity analysis of optimal matchings. Networks 11 (1981), 41–56

12 b-Matchings and T-Joins

In this chapter we introduce two more combinatorial optimization problems, the M INIMUM W EIGHT b-M ATCHING P ROBLEM in Section 12.1 and the M INIMUM W EIGHT T -J OIN P ROBLEM in Section 12.2. Both can be regarded as generalizations of the M INIMUM W EIGHT P ERFECT M ATCHING P ROBLEM and also include other important problems. On the other hand, both problems can be reduced to the M INIMUM W EIGHT P ERFECT M ATCHING P ROBLEM. They have combinatorial polynomial-time algorithms as well as polyhedral descriptions. Since in both cases the S EPARATION P ROBLEM turns out to be solvable in polynomial time, we obtain another polynomial-time algorithm for these generalized matching problems (using the E LLIPSOID M ETHOD; see Section 4.6). In fact, the S EPARATION P ROBLEM can be reduced to finding a minimum capacity T -cut in both cases; see Sections 12.3 and 12.4. This problem, finding a minimum capacity cut δ(X ) such that |X ∩T | is odd for a specified vertex set T , can be solved with network flow techniques.

12.1 b-Matchings Definition 12.1. Let G be an undirected graph with integral edge capacities u : E(G) → N ∪ {∞} and numbers b : V (G) → N. Then a b-matching Pin (G, u) is a function f : E(G) → Z+ with f (e) ≤ u(e) for all e ∈ E(G) and e∈δ(v) f (e) ≤ b(v) for all v ∈ V (G). In the case u P ≡ 1 we speak of a simple b-matching in G. A b-matching f is called perfect if e∈δ(v) f (e) = b(v) for all v ∈ V (G). In the case b ≡ 1 the capacities are irrelevant, and we are back to ordinary matchings. A simple b-matching is sometimes also called a b-factor. It can be regarded as a subset of edges. In Chapter 21 we shall be interested in perfect simple 2-matchings, i.e. subsets of edges such that each vertex is incident to exactly two of them.

M AXIMUM W EIGHT b-M ATCHING P ROBLEM Instance: Task:

A graph G, capacities u : E(G) → N∪{∞}, weights c : E(G) → R, and numbers b : V (G) → N. P Find a b-matching f in (G, u) whose weight e∈E(G) c(e) f (e) is maximum.

286

12 b-Matchings and T -Joins

Edmonds’ W EIGHTED M ATCHING A LGORITHM can be extended to solve this problem (Marsh [1979]). We shall not describe this algorithm here, but shall rather give a polyhedral description and show that the S EPARATION P ROBLEM can be solved in polynomial time. This yields a polynomial-time algorithm via the E LLIPSOID M ETHOD (cf. Corollary 3.33). The b-matching polytope of (G, u) is defined to be the convex hull of the incidence vectors of all b-matchings in (G, u). We first consider the uncapacitated case (u ≡ ∞): Theorem 12.2. (Edmonds [1965]) Let G be an undirected graph and b : V (G) E(G) → N. The b-matching polytope of (G, ∞) is the set of vectors x ∈ R+ satisfying X xe ≤ b(v) (v ∈ V (G)); e∈δ(v)

%

$ X

xe ≤

e∈E(G[X ])

1 2

X

(X ⊆ V (G)).

b(v)

v∈X

Proof: Since any b-matching obviously satisfies these constraints, we only have P E(G) with e∈δ(v) xe ≤ b(v) for all v ∈ V (G) to show one direction. So let x ∈ R+ P P and e∈E(G[X ]) xe ≤ 12 v∈X b(v) for all X ⊆ V (G). We show that x is a convex combination of incidence vectors of b-matchings. We define a new graph H by splitting up each vertex v into S b(v) copies: we define X v := {(v, i) : i ∈ {1, . . . , b(v)}} for v ∈ V (G), V (H ) := v∈V (G) X v and 1 x{v,w} E(H ) := {{v 0 , w0 } : {v, w} ∈ E(G), v 0 ∈ X v , w 0 ∈ X w }. Let ye := b(v)b(w) 0 0 0 0 for each edge e = {v , w } ∈ E(H ), v ∈ X v , w ∈ X w . We claim that y is a convex combination of incidence vectors of matchings in H . By contracting the sets X v (v ∈ V (G)) in H we then return to G and x, and conclude that x is a convex combination of incidence vectors of b-matchings in G. P To prove that y is in the matching polytope of H we use Theorem 11.14. e∈δ(v) ye ≤ 1 obviously holds for each v ∈ V (H ). Let C ⊆ V (H ) with |C| odd. P We show that e∈E(H [C]) ye ≤ 21 (|C| − 1). If X v ⊆ C or X v ∩ C = ∅ for each v ∈ V (G), this follows directly from the inequalities assumed for x. Otherwise let a, b ∈ X v , a ∈ C, b 6∈ C. Then X X X X 2 ye = ye + ye c∈C\{a} e∈E({c},C\{c})

e∈E(H [C])

≤

X

X

X

X

ye −

c∈C\{a} e∈δ(c)

=

X

X

X

ye +

c∈C\{a} e∈δ(c)\{{c,b}}

=

e∈E({a},C\{a})

ye

e∈E({a},C\{a})

X e∈E({b},C\{a})

ye +

X

ye

e∈E({a},C\{a})

ye

c∈C\{a} e∈δ(c)

≤ |C| − 1.

12.1 b-Matchings

287

Note that this construction yields an algorithm which, however, P in general has an exponential running time. But we note that in the special case v∈V (G) b(v) = O(n) we can solve the uncapacitated M AXIMUM W EIGHT b-M ATCHING P ROB LEM in O(n 3 ) time (using the W EIGHTED M ATCHING A LGORITHM ; cf. Corollary 11.10). Pulleyblank [1973,1980] described the facets of this polytope and showed that the linear inequality system in Theorem 12.2 is TDI. The following generalization allows finite capacities: Theorem 12.3. (Edmonds and Johnson [1970]) Let G be an undirected graph, u : E(G) → N ∪ {∞} and b : V (G) → N. The b-matching polytope of (G, u) is E(G) satisfying the set of vectors x ∈ R+ X

(e ∈ E(G)); (v ∈ V (G));

xe ≤ u(e) xe ≤ b(v)

e∈δ(v)

$ X e∈E(G[X ])

xe +

X

xe ≤

e∈F

1 2

!% X

b(v) +

v∈X

X

u(e)

e∈F

(X ⊆ V (G), F ⊆ δ(X )).

Proof: First observe that the incidence vector of any b-matching f satisfies the constraints. This is clear except for the last one; here we argue as follows. Let X ⊆ V (G) and F ⊆ δ(X ). We have a budget of b(v) units at each vertex v ∈ X and a budget of u(e) units for each e ∈ F. Now for each e ∈ E(G[X ]) we take f (e) units from the budget at each vertex incident to e. For each e ∈ F, say e = {x, y} with x ∈ X , we take f (e) units from the budget at x and f (e) units from thePbudget at e. It is clear that the budgets are not exceeded, and we have taken 2 e∈E(G[X ])∪F f (e) units. So X e∈E(G[X ])

xe +

X e∈F

xe

1 ≤ 2

! X v∈X

b(v) +

X

u(e) .

e∈F

Since the left-hand side is an integer, so is the right-hand side; thus we may round down. P E(G) be a vector with xe ≤ u(e) for all e ∈ E(G), e∈δ(v) xe ≤ Now let x ∈ R+ b(v) for all v ∈ V (G) and $ !% X X X 1 X xe + xe ≤ b(v) + u(e) 2 e∈E(G[X ])

e∈F

v∈X

e∈F

for all X ⊆ V (G) and F ⊆ δ(X ). We show that x is a convex combination of incidence vectors of b-matchings in (G, u). Let H be the graph resulting from G by subdividing each edge e = {v, w} with u(e) 6= ∞ by means of two new vertices (e, v), (e, w). (Instead of e, H now contains the edges {v, (e, v)}, {(e, v), (e, w)} and {(e, w), w}.) Set b((e, v)) := b((e, w)) := u(e) for the new vertices.

288

12 b-Matchings and T -Joins

For each subdivided edge e = {v, w} set y{v,(e,v)} := y{(e,w),w} := xe and y{(e,v),(e,w)} := u(e) − xe . For each original edge e with u(e) = ∞ set ye := xe . We claim that y is in the b-matching polytope P of (H, ∞). E(H ) We use Theorem 12.2. Obviously y ∈ R+ and e∈δ(v) ye ≤ b(v) for all v ∈ V (H ). Suppose there is a set A ⊆ V (H ) with 1 ye > b(a) . (12.1) 2 a∈A

e∈E(H [A])

Let B := A ∩ V (G). For each e = {v, w} ∈ E(G[B]) we may assume (e, v), (e, w) ∈ A, for otherwise the addition of (e, v) and (e, w) does not destroy (12.1). On the other hand, we may assume that (e, v) ∈ A implies v ∈ A: If (e, v), (e, w) ∈ A but v ∈ / A, we can delete (e, v) and (e, w) from A without destroying (12.1). If (e, v) ∈ A but v, (e, w) ∈ / A, we can just delete (e, v) from A. Figure 12.1 shows the remaining possible edge types. A

Fig. 12.1.

Let F := {e = {v, w} ∈ E(G) : |A ∩ {(e, v), (e, w)}| = 1}. We have xe + xe = ye − u(e) e∈E(G[B])

e∈F

e∈E(H [A])

>

e∈E(G[B]), u(e) c(J 0 ∩ E(C)) = c(E(C) \ J ). This proposition can be regarded as a special case of Theorem 9.6. We now solve the M INIMUM W EIGHT T -J OIN P ROBLEM with nonnegative weights by reducing it to the M INIMUM W EIGHT P ERFECT M ATCHING P ROBLEM. The main idea is contained in the following lemma: Lemma 12.8. Let G be a graph, c : E(G) → R+ , and T ⊆ V (G) with |T | even. Every optimum T -join in G is the disjoint union of the edge sets of |T2 | paths whose ends are distinct and in T , and possibly some zero-weight circuits. Proof: By induction on |T |. The case T = ∅ is trivial since the minimum weight of an ∅-join is zero. Let J be any optimum T -join in G; w.l.o.g. J contains no zero-weight circuit. By Proposition 12.7 J contains no circuit of positive weight. As c is nonnegative, J thus forms a forest. Let x, y be two leaves of the same connected component, i.e. |J ∩ δ(x)| = |J ∩ δ(y)| = 1, and let P be the x-y-path in J . We have x, y ∈ T , and J \ E(P) is a minimum cost (T \ {x, y})-join (a cheaper (T \ {x, y})-join J 0 would imply a T -join J 0 4E(P) that is cheaper than J ). The assertion now follows from the induction hypothesis. Theorem 12.9. (Edmonds and Johnson [1973]) In the case of nonnegative weights, the M INIMUM W EIGHT T -J OIN P ROBLEM can be solved in O(n 3 ) time. Proof: Let (G, c, T ) be an instance. We first solve an A LL PAIRS S HORTEST PATHS P ROBLEM in (G, c); more precisely: in the graph resulting by replacing each edge by a pair of oppositely directed edges with the same weight. By Theorem 7.9 this takes O(mn + n 2 log n) time. In particular, we obtain the metric closure ¯ c) (G, ¯ of (G, c) (cf. Corollary 7.11). ¯ ], c). Now we find a minimum weight perfect matching M in (G[T ¯ By Corollary 3 ¯ is at most the minimum weight 11.10, this takes O(n ) time. By Lemma 12.8, c(M) of a T -join. We consider the shortest x-y-path in G for each {x, y} ∈ M (which we have already computed). Let J be the symmetric difference of the edge sets of all these paths. Evidently, J is a T -join in G. Moreover, c(J ) ≤ c(M), ¯ so J is optimum. This method no longer works if we allow negative weights, because we would introduce negative circuits. However, we can reduce the M INIMUM W EIGHT T J OIN P ROBLEM with arbitrary weights to that with nonnegative weights:

292

12 b-Matchings and T -Joins

Theorem 12.10. Let G be a graph with weights c : E(G) → R, and T ⊆ V (G) a vertex set of even cardinality. Let E − be the set of edges with negative weight, T − the set of vertices that are incident with an odd number of negative edges, and d : E(G) → R+ with d(e) := |c(e)|. Then J is a minimum c-weight T -join if and only if J 4E − is a minimum d-weight (T 4T − )-join. Proof: For any subset J of E(G) we have c(J ) = c(J \ E − ) + c(J ∩ E − ) = c(J \ E − ) + c(J ∩ E − ) + c(E − \ J ) + d(E − \ J ) = d(J \ E − ) + c(J ∩ E − ) + c(E − \ J ) + d(E − \ J ) = d(J 4E − ) + c(E − ) . Now J is a T -join if and only if J 4E − is a (T 4T − )-join, which together with the above equality proves the theorem (since c(E − ) is constant). Corollary 12.11. The M INIMUM W EIGHT T -J OIN P ROBLEM can be solved in O(n 3 ) time. Proof: This follows directly from Theorems 12.9 and 12.10.

In fact, using the fastest known implementation of the W EIGHTED M ATCHING A LGORITHM, a minimum weight T -join can be computed in O(nm + n 2 log n) time. We are finally able to solve the S HORTEST PATH P ROBLEM in undirected graphs: Corollary 12.12. The problem of finding a shortest path between two specified vertices in an undirected graph with conservative weights can be solved in O(n 3 ) time. Proof: Let s and t be the two specified vertices. Set T := {s, t} and apply Corollary 12.11. After deleting zero-weight circuits, the resulting T -join is a shortest s-t-path. Of course this also implies an O(mn 3 )-algorithm for finding a circuit of minimum total weight in an undirected graph with conservative weights (and in particular S HORTEST PATHS to compute the girth). If we are interested in the A LL PAIRS P ROBLEM in undirected graphs, we do not have to do n2 independent weighted matching computations (which would give a running time of O(n 5 )). Using the postoptimality results of Section 11.4 we can prove: Theorem 12.13. The problem of finding shortest paths for all pairs of vertices in an undirected graph G with conservative weights c : E(G) → R can be solved in O(n 4 ) time.

12.3 T -Joins and T -Cuts

293

Proof: By Theorem 12.10 and the proof of Corollary 12.12 we have to compute an optimum {s, t}4T − -join with respect to the weights d(e) := |c(e)| for all s, t ∈ V (G), where T − is the set of vertices incident to an odd number of negative ¯ edges. Let d({x, y}) := dist(G,d) (x, y) for x, y ∈ V (G), and let H X be the complete graph on X 4T − (X ⊆ V (G)). By the proof of Theorem 12.9 it is sufficient to compute a minimum weight perfect matching in H{s,t} , d¯ for all s and t. Our O(n 4 )-algorithm proceeds as follows. We first compute d¯ (cf. Corollary ¯ 7.11) and run the W EIGHTED M ATCHING A LGORITHM for the instance (H∅ , d). 3 Up to now we have spent O(n ) time. We show that we can now compute a minimum weight perfect matching of H{s,t} , d¯ in O(n 2 ) time, for any s and t. P ¯ Let K := e∈E(G) d(e), and let s, t ∈ V (G). There are four cases: − Case 1: s, t ∈ T . Then all we have to do is reduce the cost of the edge {s, t} to −K . After reoptimizing (using Lemma 11.11), {s, t} must belong to the optimum matching M, and M \ {{s, t}} is a minimum weight perfect matching of H{s,t} , d¯ . ¯ Case 2: s ∈ T − and t 6∈ T − . Then the cost of the edge {s, v} is set to d({t, v}) for all v ∈ T − \ {s}. Now s plays the role of t, and reoptimizing (using Lemma 11.11) does the job. Case 3: s 6∈ T − and t ∈ T − . Symmetric to Case 2. Case 4: s, t 6∈ T − . Then we add these two vertices and apply Lemma 11.12.

12.3 T-Joins and T-Cuts In this section we shall derive a polyhedral description of the M INIMUM W EIGHT T -J OIN P ROBLEM. In contrast to the description of the perfect matching polytope (Theorem 11.13), where we had a constraint for each cut δ(X ) with |X | odd, we now need a constraint for each T -cut. A T-cut is a cut δ(X ) with |X ∩ T | odd. The following simple observation is very useful: Proposition 12.14. Let G be an undirected graph and T ⊆ V (G) with |T | even. Then for any T -join J and any T -cut C we have J ∩ C 6= ∅. Proof: Suppose C = δ(X ), then |X ∩ T | is odd. So the number of edges in J ∩ C must be odd, in particular nonzero. A stronger statement can be found in Exercise 11. Proposition 12.14 implies that the minimum cardinality of a T -join is not less than the maximum number of edge-disjoint T -cuts. In general, we do not have equality: consider G = K 4 and T = V (G). However, for bipartite graphs equality holds: Theorem 12.15. (Seymour [1981]) Let G be a connected bipartite graph and T ⊆ V (G) with |T | even. Then the minimum cardinality of a T -join equals the maximum number of edge-disjoint T -cuts.

294

12 b-Matchings and T -Joins

Proof: (Seb˝o [1987]) We only have to prove “≤”. We use induction on |V (G)|. If T = ∅ (in particular if |V (G)| = 1), the statement is trivial. So we assume |V (G)| ≥ |T | ≥ 2. Denote by τ (G, T ) the minimum cardinality of a T -join in G. Choose a, b ∈ V (G), a 6= b, such that τ (G, T 4{a, b}) is minimum. Let T 0 := T 4{a, b}. Since we may assume T 6= ∅, τ (G, T 0 ) < τ (G, T ). Claim: For any minimum T -join J in G we have |J ∩ δ(a)| = |J ∩ δ(b)| = 1. To prove this claim, let J 0 be a minimum T 0 -join. J 4J 0 is the edge-disjoint union of an a-b-path P and some circuits C1 , . . . , Ck . We have |Ci ∩ J | = |Ci ∩ J 0 | for each i, because both J and J 0 are minimum. So |J 4P| = |J 0 |, and J 00 := J 4P is also a minimum T 0 -join. Now J 00 ∩ δ(a) = J 00 ∩ δ(b) = ∅, because if, say, {b, b0 } ∈ J 00 , J 00 \ {{b, b0 }} is a (T 4{a}4{b0 })-join, and we have τ (G, T 4{a}4{b0 }) < |J 00 | = |J 0 | = τ (G, T 0 ), contradicting the choice of a and b. We conclude that |J ∩ δ(a)| = |J ∩ δ(b)| = 1, and the claim is proved. In particular, a, b ∈ T . Now let J be a minimum T -join in G. Contract B := {b} ∪ 0(b) to a single vertex v B , and let the resulting graph be G ∗ . G ∗ is also bipartite. Let T ∗ := T \ B if |T ∩ B| is even and T ∗ := (T \ B) ∪ {v B } otherwise. The set J ∗ , resulting from J by the contraction of B, is obviously a T ∗ -join in G ∗ . Since 0(b) is a stable set in G (as G is bipartite), the claim implies that |J | = |J ∗ | + 1. It suffices to prove that J ∗ is a minimum T ∗ -join in G ∗ , because then we have τ (G, T ) = |J | = |J ∗ | + 1 = τ (G ∗ , T ∗ ) + 1, and the theorem follows by induction (observe that δ(b) is a T -cut in G disjoint from E(G ∗ )). So suppose that J ∗ is not a minimum T ∗ -join in G ∗ . Then by Proposition 12.7 there is a circuit C ∗ in G ∗ with |J ∗ ∩ E(C ∗ )| > |E(C ∗ ) \ J ∗ |. Since G ∗ is bipartite, |J ∗ ∩ E(C ∗ )| ≥ |E(C ∗ ) \ J ∗ | + 2. E(C ∗ ) corresponds to an edge set Q in G. Q cannot be a circuit, because |J ∩ Q| > |Q \ J | and J is a minimum T -join. Hence Q is an x-y-path in G for some x, y ∈ 0(b) with x 6= y. Let C be the circuit in G formed by Q together with {x, b} and {b, y}. Since J is a minimum T -join in G, |J ∩ E(C)| ≤ |E(C) \ J | ≤ |E(C ∗ ) \ J ∗ | + 2 ≤ |J ∗ ∩ E(C ∗ )| ≤ |J ∩ E(C)|. Thus we must have equality throughout, in particular {x, b}, {b, y} ∈ / J and |J ∩ E(C)| = |E(C)\ J |. So J¯ := J 4E(C) is also a minimum T -join and | J¯∩δ(b)| = 3. But this is impossible by the claim. T -cuts are also essential in the following description of the T -join polyhedron: Theorem 12.16. (Edmonds and Johnson [1973]) Let G be an undirected graph, c : E(G) → R+ , and T ⊆ V (G) with |T | even. Then the incidence vector of a minimum weight T -join is an optimum solution of the LP ( ) X min cx : x ≥ 0, xe ≥ 1 for all T -cuts C . e∈C

(This polyhedron is called the T-join polyhedron of G.)

12.3 T -Joins and T -Cuts

295

Proof: By Proposition 12.14, the incidence vector of a T -join satisfies the constraints. Let c : E(G) → R+ be given; we may assume that c(e) is an even integer for each e ∈ E(G). Let k be the minimum weight (with respect to c) of a T -join in G. We show that the optimum value of the above LP is k. We replace each edge e by a path of length c(e) (if c(e) = 0 we contract e and add the contracted vertex to T iff |e ∩ T | = 1). The resulting graph G 0 is bipartite. Moreover, the minimum cardinality of a T -join in G 0 is k. By Theorem 12.15, there is a family C 0 of k edge-disjoint T -cuts in G 0 . Back in G, this yields a family C of k T -cuts in G such that every edge e is contained in at most c(e) of these. So for any feasible solution x of the above LP we have XX X cx ≥ xe ≥ 1 = k, C∈C e∈C

C∈C

proving that the optimum value is k.

This implies Theorem 11.13: let G be a graph with a perfect matching and T := V (G). Then Theorem 12.16 implies that ) ( X xe ≥ 1 for all T -cuts C min cx : x ≥ 0, e∈C

is an integer for each c ∈ Z E(G) for which the minimum is finite. By Theorem 5.13, the polyhedron is integral, and so is its face X X E(G) x ∈ R+ : xe ≥ 1 for all T -cuts C, xe = 1 for all v ∈ V (G) . e∈C

e∈δ(v)

One can also derive a description of the convex hull of the incidence vectors of all T -joins (Exercise 14). Theorems 12.16 and 4.21 (along with Corollary 3.33) imply another polynomial-time algorithm for the M INIMUM W EIGHT T -J OIN P ROBLEM if we can solve the S EPARATION P ROBLEM for the above description. This is obviously equivalent to checking whether there exists a T -cut with capacity less than one (here x serves as capacity vector). So it suffices to solve the following problem:

M INIMUM C APACITY T -C UT P ROBLEM Instance: Task:

A graph G, capacities u : E(G) → R+ , and a set T ⊆ V (G) of even cardinality. Find a minimum capacity T -cut in G.

Note that the M INIMUM C APACITY T -C UT P ROBLEM also solves the S EPARA P ROBLEM for the perfect matching polytope (Theorem 11.13; T := V (G)). The following theorem solves the M INIMUM C APACITY T -C UT P ROBLEM: it suffices to consider the fundamental cuts of a Gomory-Hu tree. Recall that we TION

296

12 b-Matchings and T -Joins

can find a Gomory-Hu tree for an undirected graph with capacities in O(n 4 ) time (Theorem 8.35). Theorem 12.17. (Padberg and Rao [1982]) Let G be an undirected graph with capacities u : E(G) → R+ . Let H be a Gomory-Hu tree for (G, u). Let T ⊆ V (G) with |T | even. Then there is a minimum capacity T -cut among the fundamental cuts of H . Hence the minimum capacity T -cut can be found in O(n 4 ) time. Proof: We consider the pair (G + H, u 0 ) with u 0 (e) = u(e) for e ∈ E(G) and u 0 (e) = 0 for e ∈ E(H ). Let A ⊆ E(G)∪E(H ) be a minimum T -cut in (G+H, u 0 ). Obviously u 0 (A) = u(A ∩ E(G)) and A ∩ E(G) is a minimum T -cut in (G, u). Let now J be the set of edges e of H for which δG (Ce ) is a T -cut. It is easy to see that J is a T -join (in G + H ). By Proposition 12.14, there exists an edge e = {v, w} ∈ A ∩ J . We have X u(A ∩ E(G)) ≥ λvw = u({x, y}), {x,y}∈δG (Ce )

showing that δG (Ce ) is a minimum T -cut.

12.4 The Padberg-Rao Theorem The solution of the M INIMUM C APACITY T -C UT P ROBLEM also helps us to solve the S EPARATION P ROBLEM for the b-matching polytope (Theorem 12.3): Theorem 12.18. (Padberg and Rao [1982]) For undirected graphs G, u : E(G) → N ∪ {∞} and b : V (G) → N, the S EPARATION P ROBLEM for the b-matching polytope of (G, u) can be solved in polynomial time. Proof: We may assume u(e) < ∞ for all edges e (we may replace infinite capacities by a large enough number, e.g. max{b(v) : v ∈ V (G)}). We choose an arbitrary but fixed orientation of G; we will sometimes use the resulting directed edges and sometimes the original undirected edges. P E(G) with xe ≤ u(e) for all e ∈ E(G) and e∈δG (v) xe ≤ Given a vector x ∈ R+ b(v) for all v ∈ V (G) (these trivial inequalities can be checked in linear time), we define a new bipartite graph H with edge capacities t : E(H ) → R+ as follows: .

.

V (H ) := V (G) ∪ E(G) ∪ {S}, E(H ) := {{v, e} : v ∈ e ∈ E(G)} ∪ {{v, S} : v ∈ V (G)}, (e ∈ E(G), where v is the tail of e), t ({v, e}) := u(e) − xe (e ∈ E(G), where v is the head of e), t ({v, e}) := xe X t ({v, S}) := b(v) − xe (v ∈ V (G)). e∈δG (v)

Define T ⊆ V (H ) to consist of

12.4 The Padberg-Rao Theorem

297

• the vertices v ∈ V (G) for which b(v) + e∈δ + (v) u(e) is odd, G • the vertices e ∈ E(G) for which u(e) is odd, and • the vertex S if v∈V (G) b(v) is odd. Observe that |T | is even. We shall prove that there exists a T -cut in H with capacity less than one if and only if x is not in the convex hull of the b-matchings in (G, u).

E3

E1

∈ / F

∈F

X

E4

E2 Fig. 12.2.

We need some preparation. Let X ⊆ V (G) and F ⊆ δG (X ). Deﬁne + (X ) ∩ F}, E 1 := {e ∈ δG

− E 2 := {e ∈ δG (X ) ∩ F}, + E 3 := {e ∈ δG (X ) \ F}, − E 4 := {e ∈ δG (X ) \ F},

(see Figure 12.2) and W := X ∪ E(G[X ]) ∪ E 2 ∪ E 3 ⊆ V (H ). Claim:

(a) |W v∈X b(v) + e∈F u(e) is odd. ∩ T | is odd if and only if (b) e∈δ H (W ) t (e) < 1 if and only if e∈E(G[X ])

xe +

xe

e∈F

1 > 2

v∈X

b(v) +

u(e) − 1 .

e∈F

To prove (a), observe that by deﬁnition |W ∩ T | is odd if and only if ⎛ ⎞ ⎜ ⎟ u(e)⎠ + u(e) ⎝b(v) + v∈X

+ e∈δG (v)

e∈E(G[X ])∪E 2 ∪E 3

298

12 b-Matchings and T -Joins

is odd. But this number is equal to X X X b(v) + 2 u(e) + u(e) + v∈X

=

X

+ e∈δG (X )

e∈E(G[X ])

X

b(v) + 2

v∈X

u(e) − 2

+ e∈δG (X )

e∈E(G[X ])

u(e)

e∈E 2 ∪E 3

X

u(e) + 2

X

X

X

u(e) +

e∈E 1

u(e),

e∈E 1 ∪E 2

proving (a), because E 1 ∪ E 2 = F. Moreover, X X X X t (e) = t ({x, e}) + t ({y, e}) + t ({x, S}) e∈δ H (W )

e∈E 1 ∪E 4 x∈e∩X

e∈E 2 ∪E 3 y∈e\X

x∈X

X

=

(u(e) − xe ) +

e∈E 1 ∪E 2

=

X

X

xe +

u(e) +

b(v) − 2

v∈X

e∈F

X

X

b(v) −

X

xe − 2

e∈F

v∈X

e∈E 3 ∪E 4

X

X

xe

e∈δG (v)

xe ,

e∈E(G[X ])

proving (b). Now we can prove that there exists a T -cut in H with capacity less than one if and only if x is not in the convex hull of the b-matchings in (G, u). First suppose that there are X ⊆ V (G) and F ⊆ δG (X ) with $ !% X X X 1 X xe + xe > b(v) + u(e) . 2 Then

v∈X

b(v) +

X e∈E(G[X ])

v∈X

e∈F

e∈E(G[X ])

P

P

e∈F

xe +

e∈F

u(e) must be odd and

X e∈F

xe

1 > 2

! X v∈X

b(v) +

X

u(e) − 1 .

e∈F

By (a) and (b), this implies that δ H (W ) is a T -cut with capacity less than one. To prove the converse, let δ H (W ) now be any T -cut in H with capacity less than one. We show how to construct a violated inequality of the b-matching polytope. W.l.o.g. assume S 6∈ W (otherwise exchange W and V (H ) \ W ). Define X := W ∩ V (G). Observe that {v, {v, w}} ∈ δ H (W ) implies {v, w} ∈ δG (X ): If {v, w} ∈ / W for some v, w ∈ X , the two edges {v, {v, w}} and {w, {v, w}} (with total capacity u({v, w})) would belong to δ H (W ), contradicting the assumption that this cut has capacity less than one. The assumption {v, w} ∈ W for some v, w ∈ / X leads to the same contradiction. Define F := {(v, w) ∈ E(G) : {v, {v, w}} ∈ δ H (W )}. By the above observation we have F ⊆ δG (X ). We define E 1 , E 2 , E 3 , E 4 as above and claim that

Exercises

W = X ∪ E(G[X ]) ∪ E 2 ∪ E 3

299

(12.2)

holds. Again by the above observation, we only have to prove W ∩δG (X ) = E 2 ∪E 3 . + But e = (v, w) ∈ E 1 = δG (X )∩ F implies e ∈ / W by the definition of F. Similarly, − + e = (v, w) ∈ E 2 = δG (X ) ∩ F implies e ∈ W , e = (v, w) ∈ E 3 = δG (X ) \ F − / W . Thus (12.2) is implies e ∈ W , and e = (v, w) ∈ E 4 = δG (X ) \ F implies e ∈ proved. P P So (a) and (b) again hold. Since |W ∩ T | is odd, (a) implies P that v∈X b(v) + e∈F u(e) is odd. Then by (b) and the assumption that e∈δ H (W ) t (e) < 1, we get $ !% X X X 1 X xe + xe > b(v) + u(e) , 2 e∈E(G[X ])

e∈F

v∈X

e∈F

i.e. a violated inequality of the b-matching polytope. Let us summarize: We have shown that the minimum capacity of a T -cut in H is less than one if and only if x violates some inequality of the b-matching polytope. Furthermore, given some T -cut in H with capacity less than one, we can easily construct a violated inequality. So the problem reduces to the M INIMUM C APACITY T -C UT P ROBLEM with nonnegative weights. By Theorem 12.17, the latter can be solved in O(n 4 ) time, where n = |V (H )|. A generalization of this result has been found by Caprara and Fischetti [1996]. Letchford, Reinelt and Theis [2004] showed that it suffices to consider the GomoryHu tree for (G, u). They reduce the S EPARATION P ROBLEM for b-matching (and more general) inequalities to |V (G)| maximum flow computations on the original graph and thus solve it in O(|V (G)|4 ) time. The Padberg-Rao Theorem implies: Corollary 12.19. The M AXIMUM W EIGHT b-M ATCHING P ROBLEM can be solved in polynomial time. Proof: By Corollary 3.33 we have to solve the LP given in Theorem 12.3. By Theorem 4.21 it suffices to have a polynomial-time algorithm for the S EPARATION P ROBLEM. Such an algorithm is provided by Theorem 12.18. Marsh [1979] extended Edmonds’ W EIGHTED M ATCHING A LGORITHM to the M AXIMUM W EIGHT b-M ATCHING P ROBLEM. This combinatorial algorithm is of course more practical than using the E LLIPSOID M ETHOD. But Theorem 12.18 is also interesting for other purposes (see e.g. Section 21.4). For a combinatorial algorithm with a strongly polynomial running time, see Anstee [1987] or Gerards [1995].

Exercises 1. Show that a minimum weight perfect simple 2-matching in an undirected graph G can be found in O(n 6 ) time.

300

∗

12 b-Matchings and T -Joins

2. Let G be an undirected graph and b1 , b2 : V (G)P → N. Describe the convex hull of functions f : E(G) → Z+ with b1 (v) ≤ e∈δ(v) f (e) ≤ b2 (v). Hint: For X, Y ⊆ V (G) with X ∩ Y = ∅ consider the constraint X X X 1 X b2 (x) − b1 (y) , f (e) − f (e) ≤ 2 e∈E(G[X ])

∗

3.

∗

4.

5.

∗

6.

7.

8.

e∈E(G[Y ])∪E(Y,Z )

x∈X

y∈Y

where Z := V (G) \ (X ∪ Y ). Use Theorem 12.3. (Schrijver [1983]) Can one generalize the result of Exercise 2 further by introducing lower and upper capacities on the edges? Note: This can be regarded as an undirected version of the problem in Exercise 3 of Chapter 9. For a common generalization of both problems and also the M INIMUM W EIGHT T -J OIN P ROBLEM see the papers of Edmonds and Johnson [1973], and Schrijver [1983]. Even here a description of the polytope that is TDI is known. Prove Theorem 12.4. Hint: For the sufficiency, use Tutte’s Theorem 10.13 and the constructions in the proofs of Theorems 12.2 and 12.3. The subgraph degree polytope of a graph G is defined to be the convex hull of V (G) such that G has a perfect simple b-matching. Prove that all vectors b ∈ Z+ its dimension is |V (G)| − k, where k is the number of connected components of G that are bipartite. Given an undirected graph, an odd cycle cover is defined to be a subset of edges containing at least one edge of each odd circuit. Show how to find in polynomial time a minimum weight odd cycle cover in a planar graph with nonnegative weights on the edges. Can you also solve the problem for general weights? Hint: Consider the U NDIRECTED C HINESE P OSTMAN P ROBLEM in the planar dual graph and use Theorem 2.26 and Corollary 2.45. Consider the M AXIMUM W EIGHT C UT P ROBLEM in planar graphs: Given an undirected planar graph G with weights c : E(G) → R+ , we look for a maximum weight cut. Can one solve this problem in polynomial time? Hint: Use Exercise 6. Note: For general graphs this problem is NP-hard; see Theorem 16.6. (Hadlock [1975]) Given a graph G with weights c : E(G) → R+ and a set T ⊆ V (G) with |T | even. We construct a new graph G 0 by setting V (G 0 ) := {(v, e) : v ∈ e ∈ E(G)} ∪ {v¯ : v ∈ V (G), |δG (v)| + |{v} ∩ T | odd}, E(G 0 ) := {{(v, e), (w, e)} : e = {v, w} ∈ E(G)} ∪ {{(v, e), (v, f )} : v ∈ V (G), e, f ∈ δG (v), e 6= f } ∪ {{v, ¯ (v, e)} : v ∈ e ∈ E(G), v¯ ∈ V (G 0 )},

Exercises

∗

9.

10.

∗

11.

∗ 12.

13. 14.

301

and define c0 ({(v, e), (w, e)}) := c(e) for e = {v, w} ∈ E(G) and c0 (e0 ) = 0 for all other edges in G 0 . Show that a minimum weight perfect matching in G 0 corresponds to a minimum weight T -join in G. Is this reduction preferable to the one used in the proof of Theorem 12.9? The following problem combines simple perfect b-matchings and T -joins. We are given an undirected graph G with weights c : E(G) → R, a partition of . . the vertex set V (G) = R ∪ S ∪ T , and a function b : R → Z+ . We ask for a subset of edges J ⊆ E(G) such that J ∩ δ(v) = b(v) for v ∈ R, |J ∩ δ(v)| is even for v ∈ S, and |J ∩ δ(v)| is odd for v ∈ T . Show how to reduce this problem to a M INIMUM W EIGHT P ERFECT M ATCHING P ROBLEM. Hint: Consider the constructions in Section 12.1 and Exercise 8. Consider the U NDIRECTED M INIMUM M EAN C YCLE P ROBLEM: Given an undirected graph G and weights c : E(G) → R, find a circuit C in G whose mean weight c(E(C)) |E(C)| is minimum. (a) Show that the M INIMUM M EAN C YCLE A LGORITHM of Section 7.3 cannot be applied to the undirected case. (b) Find a strongly polynomial algorithm for the U NDIRECTED M INIMUM M EAN C YCLE P ROBLEM. Hint: Use Exercise 9. Let G be an undirected graph, T ⊆ V (G) with |T | even, and F ⊆ E(G). Prove: F has nonzero intersection with every T -join if and only if F contains a T -cut. F has nonzero intersection with every T -cut if and only if F contains a T -join. Let G be a planar 2-connected graph with a fixed embedding, let C be the circuit bounding the outer face, and let T be an even cardinality subset of V (C). Prove that the minimum cardinality of a T -join equals the maximum number of edge-disjoint T -cuts. Hint: Colour the edges of C red and blue such that, when traversing C, colours change precisely at the vertices in T . Consider the planar dual graph, split the vertex representing the outer face into a red and a blue vertex, and apply Menger’s Theorem 8.9. Prove Theorem 12.16 using Theorem 11.13 and the construction of Exercise 8. (Edmonds and Johnson [1973]) Let G be an undirected graph and T ⊆ V (G) with |T | even. Prove that the convex hull of the incidence vectors of all T -joins in G is the set of all vectors x ∈ [0, 1] E(G) satisfying X X xe + (1 − xe ) ≥ 1 e∈δG (X )\F

e∈F

for all X ⊆ V (G) and F ⊆ δG (X ) with |X ∩ T | + |F| odd. Hint: Use Theorems 12.16 and 12.10. 15. Let G be an undirected graph and T ⊆ V (G) with |T | = 2k even. Prove that k the minimum cardinality of a T -cut in G equals the maximum of mini=1 λsi ,ti

302

12 b-Matchings and T -Joins

over all pairings T = {s1 , t1 , s2 , t2 , . . . , sk , tk }. (λs,t denotes the maximum number of edge-disjoint s-t-paths.) Can you think of a weighted version of this min-max formula? Hint: Use Theorem 12.17. (Rizzi [2002]) 16. This exercise gives an algorithm for the M INIMUM C APACITY T -C UT P ROB LEM without using Gomory-Hu trees. The algorithm is recursive and – given G, u and T – proceeds as follows: 1. First we P find a set X ⊆ V (G) with T ∩ X 6= ∅ and T \ X 6= ∅, such that u(X ) := e∈δG (X ) u(e) is minimum (cf. Exercise 22 of Chapter 8). If |T ∩ X | happens to be odd, we are done (return X ). 2. Otherwise we apply the algorithm recursively first to G, u and T ∩ X , and then to G, u and T \ X . We obtain a set Y ⊆ V (G) with |(T ∩ X ) ∩ Y | odd and u(Y ) minimum and a set Z ⊆ V (G) with |(T \ X ) ∩ Z | odd and u(Z ) minimum. W.l.o.g. T \ X 6⊆ Y and X ∩ T 6⊆ Z (otherwise replace Y by V (G) \ Y and/or Z by V (G) \ Z ). 3. If u(X ∩ Y ) < u(Z \ X ) then return X ∩ Y else return Z \ X . Show that this algorithm works correctly and that its running time is O(n 5 ), where n = |V (G)|. 17. Show how to solve the M AXIMUM W EIGHT b-M ATCHING P ROBLEM for the special case when b(v) is even for all v ∈ V (G) in strongly polynomial time. Hint: Reduction to a M INIMUM C OST F LOW P ROBLEM as in Exercise 10 of Chapter 9.

References General Literature: Cook, W.J., Cunningham, W.H., Pulleyblank, W.R., and Schrijver, A. [1998]: Combinatorial Optimization. Wiley, New York 1998, Sections 5.4 and 5.5 Frank, A. [1996]: A survey on T -joins, T -cuts, and conservative weightings. In: Combinatorics, Paul Erd˝os is Eighty; Volume 2 (D. Mikl´os, V.T. S´os, T. Sz˝onyi, eds.), Bolyai Society, Budapest 1996, pp. 213–252 Gerards, A.M.H. [1995]: Matching. In: Handbooks in Operations Research and Management Science; Volume 7: Network Models (M.O. Ball, T.L. Magnanti, C.L. Monma, G.L. Nemhauser, eds.), Elsevier, Amsterdam 1995, pp. 135–224 Lov´asz, L., and Plummer, M.D. [1986]: Matching Theory. Akad´emiai Kiad´o, Budapest 1986, and North-Holland, Amsterdam 1986 Schrijver, A. [1983]: Min-max results in combinatorial optimization; Section 6. In: Mathematical Programming; The State of the Art – Bonn 1982 (A. Bachem, M. Gr¨otschel, B. Korte, eds.), Springer, Berlin 1983, pp. 439–500 Schrijver, A. [2003]: Combinatorial Optimization: Polyhedra and Efficiency. Springer, Berlin 2003, Chapters 29–33

References

303

Cited References: Anstee, R.P. [1987]: A polynomial algorithm for b-matchings: an alternative approach. Information Processing Letters 24 (1987), 153–157 Caprara, A., and Fischetti, M. [1996]: {0, 12 }-Chv´atal-Gomory cuts. Mathematical Programming 74 (1996), 221–235 Edmonds, J. [1965]: Maximum matching and a polyhedron with (0,1) vertices. Journal of Research of the National Bureau of Standards B 69 (1965), 125–130 Edmonds, J., and Johnson, E.L. [1970]: Matching: A well-solved class of integer linear programs. In: Combinatorial Structures and Their Applications; Proceedings of the Calgary International Conference on Combinatorial Structures and Their Applications 1969 (R. Guy, H. Hanani, N. Sauer, J. Schonheim, eds.), Gordon and Breach, New York 1970, pp. 69–87 Edmonds, J., and Johnson, E.L. [1973]: Matching, Euler tours and the Chinese postman problem. Mathematical Programming 5 (1973), 88–124 Guan, M. [1962]: Graphic programming using odd and even points. Chinese Mathematics 1 (1962), 273–277 Hadlock, F. [1975]: Finding a maximum cut of a planar graph in polynomial time. SIAM Journal on Computing 4 (1975), 221–225 Letchford, A.N., Reinelt, G., and Theis, D.O. [2004]: A faster exact separation algorithm for blossom inequalities. Proceedings of the 10th Conference on Integer Programming and Combinatorial Optimization; LNCS 3064 (D. Bienstock, G. Nemhauser, eds.), Springer, Berlin 2004, pp. 196–205 Marsh, A.B. [1979]: Matching algorithms. Ph.D. thesis, Johns Hopkins University, Baltimore 1979 Padberg, M.W., and Rao, M.R. [1982]: Odd minimum cut-sets and b-matchings. Mathematics of Operations Research 7 (1982), 67–80 Pulleyblank, W.R. [1973]: Faces of matching polyhedra. Ph.D. thesis, University of Waterloo, 1973 Pulleyblank, W.R. [1980]: Dual integrality in b-matching problems. Mathematical Programming Study 12 (1980), 176–196 Rizzi, R. [2002]: Minimum T -cuts and optimal T -pairings. Discrete Mathematics 257 (2002), 177–181 Seb˝o, A. [1987]: A quick proof of Seymour’s theorem on T -joins. Discrete Mathematics 64 (1987), 101–103 Seymour, P.D. [1981]: On odd cuts and multicommodity flows. Proceedings of the London Mathematical Society (3) 42 (1981), 178–192 Tutte, W.T. [1952]: The factors of graphs. Canadian Journal of Mathematics 4 (1952), 314–328 Tutte, W.T. [1954]: A short proof of the factor theorem for finite graphs. Canadian Journal of Mathematics 6 (1954), 347–352

13 Matroids

Many combinatorial optimization problems can be formulated as follows. Given a set system (E, F), i.e. a finite set E and some F ⊆ 2 E , and a cost function c : F → R, find an element of F whose cost is minimum or maximum.PIn the following we consider modular functions c, i.e. assume that c(X ) = c(∅)+ x∈X (c({x})−c(∅)) for all XP ⊆ E; equivalently we are given a function c : E → R and write c(X ) = e∈X c(e). In this chapter we restrict ourselves to those combinatorial optimization problems where F describes an independence system (i.e. is closed under subsets) or even a matroid. The results of this chapter generalize several results obtained in previous chapters. In Section 13.1 we introduce independence systems and matroids and show that many combinatorial optimization problems can be described in this context. There are several equivalent axiom systems for matroids (Section 13.2) and an interesting duality relation discussed in Section 13.3. The main reason why matroids are important is that a simple greedy algorithm can be used for optimization over matroids. We analyze greedy algorithms in Section 13.4 before turning to the problem of optimizing over the intersection of two matroids. As shown in Sections 13.5 and 13.7 this problem can be solved in polynomial time. This also solves the problem of covering a matroid by independent sets as discussed in Section 13.6.

13.1 Independence Systems and Matroids Definition 13.1. A set system (E, F) is an independence system if (M1) ∅ ∈ F; (M2) If X ⊆ Y ∈ F then X ∈ F. The elements of F are called independent, the elements of 2 E \ F dependent. Minimal dependent sets are called circuits, maximal independent sets are called bases. For X ⊆ E, the maximal independent subsets of X are called bases of X . Definition 13.2. Let (E, F) be an independence system. For X ⊆ E we define the rank of X by r (X ) := max{|Y | : Y ⊆ X, Y ∈ F}. Moreover, we define the closure of X by σ (X ) := {y ∈ E : r (X ∪ {y}) = r (X )}.

306

13 Matroids

Throughout this chapter, (E, F) will be an independence system, and c : E → R will be a cost function. We shall concentrate on the following two problems:

M AXIMIZATION P ROBLEM F OR I NDEPENDENCE S YSTEMS Instance: Task:

An independence system (E, F) and c : E → R. P Find an X ∈ F such that c(X ) := e∈X c(e) is maximum.

M INIMIZATION P ROBLEM F OR I NDEPENDENCE S YSTEMS Instance:

An independence system (E, F) and c : E → R.

Task:

Find a basis B such that c(B) is minimum.

The instance specification is somewhat vague. The set E and the cost function c are given explicitly as usual. However, the set F is usually not given by an explicit list of its elements. Rather one assumes an oracle which – given a subset F ⊆ E – decides whether F ∈ F. We shall return to this question in Section 13.4. The following list shows that many combinatorial optimization problems actually have one of the above two forms: (1) M AXIMUM W EIGHT S TABLE S ET P ROBLEM Given a graph G and weights c : V (G) → R, find a stable set X in G of maximum weight. Here E = V (G) and F = {F ⊆ E : F is stable in G}. (2) TSP Given a complete undirected graph G and weights c : E(G) → R+ , find a minimum weight Hamiltonian circuit in G. Here E = E(G) and F = {F ⊆ E : F is a subset of a Hamiltonian circuit in G}. (3) S HORTEST PATH P ROBLEM Given a digraph G, c : E(G) → R and s, t ∈ V (G) such that t is reachable from s, find a shortest s-t-path in G with respect to c. Here E = E(G) and F = {F ⊆ E : F is a subset of an s-t-path}. (4) K NAPSACK P ROBLEM Given nonnegative numbers i ≤ n), and W , find a subset P n, ci , wi (1 ≤ P S ⊆ {1, . . . , n} such that j∈S w j ≤ W and j∈S c j is maximum. o n P Here E = {1, . . . , n} and F = F ⊆ E : j∈F w j ≤ W . (5) M INIMUM S PANNING T REE P ROBLEM Given a connected undirected graph G and weights c : E(G) → R, find a minimum weight spanning tree in G. Here E = E(G) and F is the set of forests in G. (6) M AXIMUM W EIGHT F OREST P ROBLEM Given an undirected graph G and weights c : E(G) → R, find a maximum weight forest in G. Here again E = E(G) and F is the set of forests in G.

13.1 Independence Systems and Matroids

307

(7) S TEINER T REE P ROBLEM Given a connected undirected graph G, weights c : E(G) → R+ , and a set T ⊆ V (G) of terminals, find a Steiner tree for T , i.e. a tree S with T ⊆ V (S) and E(S) ⊆ E(G), such that c(E(S)) is minimum. Here E = E(G) and F = {F ⊆ E : F is a subset of a Steiner tree for T }. (8) M AXIMUM W EIGHT B RANCHING P ROBLEM Given a digraph G and weights c : E(G) → R, find a maximum weight branching in G. Here E = E(G) and F is the set of branchings in G. (9) M AXIMUM W EIGHT M ATCHING P ROBLEM Given an undirected graph G and weights c : E(G) → R, find a maximum weight matching in G. Here E = E(G) and F is the set of matchings in G. This list contains NP-hard problems ((1),(2),(4),(7)) as well as polynomially solvable problems ((5),(6),(8),(9)). Problem (3) is NP-hard in the above form but polynomially solvable for nonnegative weights. (NP-hardness is introduced in Chapter 15.) Definition 13.3. An independence system is a matroid if (M3) If X, Y ∈ F and |X | > |Y |, then there is an x ∈ X \ Y with Y ∪ {x} ∈ F. The name matroid points out that the structure is a generalization of matrices. This will become clear by our first example: Proposition 13.4. The following independence systems (E, F) are matroids: (a) E F (b) E F (c) E (d) E (s (e) E F

is a set of columns of a matrix A over some field, and := {F ⊆ E : The columns in F are linearly independent over that field}. is a set of edges of some undirected graph G and := {F ⊆ E : (V (G), F) is a forest}. is a finite set, k an integer and F := {F ⊆ E : |F| ≤ k}. is a set of edges of some undirected graph G, S a stable set in G, ks integers ∈ S) and F := {F ⊆ E : |δ F (s)| ≤ ks for all s ∈ S}. is a set of edges of some digraph G, S ⊆ V (G), ks integers (s ∈ S) and := {F ⊆ E : |δ − F (s)| ≤ ks for all s ∈ S}.

Proof: In all cases it is obvious that (E, F) is indeed an independence system. So it remains to show that (M3) holds. For (a) this is well-known from Linear Algebra, for (c) it is trivial. To prove (M3) for (b), let X, Y ∈ F and suppose Y ∪ {x} 6∈ F for all x ∈ X \ Y . We show that |X | ≤ |Y |. For each edge x = {v, w} ∈ X , v and w are in the same connected component of (V (G), Y ). Hence each connected component Z ⊆ V (G) of (V (G), X ) is a subset of a connected component of (V (G), Y ). So the number p of connected components of the forest (V (G), X ) is greater than or equal to the number q of connected components of the forest (V (G), Y ). But then |V (G)| − |X | = p ≥ q = |V (G)| − |Y |, implying |X | ≤ |Y |.

308

13 Matroids

To verify (M3) for (d), let X, Y ∈ F with |X | > |Y |. Let S 0 := {s ∈ S : |δY (s)| = ks }. As |X | > |Y | and |δ X (s)| ≤ ks for all s ∈ S 0 , there exists an e ∈ X \ Y with e ∈ / δ(s) for s ∈ S 0 . Then Y ∪ {e} ∈ F. For (e) the proof is identical except for replacing δ by δ − . Some of these matroids have special names: The matroid in (a) is called the vector matroid of A. Let M be a matroid. If there is a matrix A over the field F such that M is the vector matroid of A, then M is called representable over F. There are matroids that are not representable over any field. The matroid in (b) is called the cycle matroid of G and will sometimes be denoted by M(G). A matroid which is the cycle matroid of some graph is called a graphic matroid. The matroids in (c) are called uniform matroids. In our list of independence systems at the beginning of this section, the only matroids are the graphic matroids in (5) and (6). To check that all the other independence systems in the above list are not matroids in general is easily proved with the help of the following theorem (Exercise 1): Theorem 13.5. Let (E, F) be an independence system. Then the following statements are equivalent: (M3) If X, Y ∈ F and |X | > |Y |, then there is an x ∈ X \ Y with Y ∪ {x} ∈ F. (M30 ) If X, Y ∈ F and |X | = |Y |+1, then there is an x ∈ X \Y with Y ∪{x} ∈ F. (M300 ) For each X ⊆ E, all bases of X have the same cardinality. Proof: Trivially, (M3)⇔(M30 ) and (M3)⇒(M300 ). To prove (M300 )⇒(M3), let X, Y ∈ F and |X | > |Y |. By (M300 ), Y cannot be a basis of X ∪ Y . So there must be an x ∈ (X ∪ Y ) \ Y = X \ Y such that Y ∪ {x} ∈ F. Sometimes it is useful to have a second rank function: Definition 13.6. Let (E, F) be an independence system. For X ⊆ E we define the lower rank by ρ(X ) := min{|Y | : Y ⊆ X, Y ∈ F and Y ∪ {x} ∈ / F for all x ∈ X \ Y }. The rank quotient of (E, F) is defined by q(E, F) := min

F⊆E

ρ(F) . r (F)

Proposition 13.7. Let (E, F) be an independence system. Then q(E, F) ≤ 1. Furthermore, (E, F) is a matroid if and only if q(E, F) = 1. Proof: q(E, F) ≤ 1 follows from the definition. q(E, F) = 1 is obviously equivalent to (M300 ). To estimate the rank quotient, the following statement can be used:

13.2 Other Matroid Axioms

309

Theorem 13.8. (Hausmann, Jenkyns and Korte [1980]) Let (E, F) be an independence system. If, for any A ∈ F and e ∈ E, A ∪ {e} contains at most p circuits, then q(E, F) ≥ 1p . |J | 1 Proof: Let F ⊆ E and J, K two bases of F. We show |K | ≥ p. Let J \ K = {e1 , . . . , et }. We construct a sequence K = K 0 , K 1 , . . . , K t of independent subsets of J ∪ K such that J ∩ K ⊆ K i , K i ∩{e1 , . . . , et } = {e1 , . . . , ei } and |K i−1 \ K i | ≤ p for i = 1, . . . , t. Since K i ∪ {ei+1 } contains at most p circuits and each such circuit must meet K i \ J (because J is independent), there is an X ⊆ K i \ J such that |X | ≤ p and (K i \ X ) ∪ {ei+1 } ∈ F. We set K i+1 := (K i \ X ) ∪ {ei+1 }. Now J ⊆ K t ∈ F. Since J is a basis of F, J = K t . We conclude that

|K \ J | =

t X

|K i−1 \ K i | ≤ pt = p |J \ K |,

i=1

proving |K | ≤ p |J |.

This shows that in example (9) we have q(E, F) ≥ 12 (see also Exercise 1 of Chapter 10). In fact q(E, F) = 12 iff G contains a path of length 3 as a subgraph (otherwise q(E, F) = 1). For the independence system in example (1) of our list, the rank quotient can become arbitrarily small (choose G to be a star). In Exercise 5, the rank quotients for other independence systems will be discussed.

13.2 Other Matroid Axioms In this section we consider other axiom systems defining matroids. They characterize fundamental properties of the family of bases, the rank function, the closure operator and the family of circuits of a matroid. Theorem 13.9. Let E be a finite set and B ⊆ 2 E . B is the set of bases of some matroid (E, F) if and only if the following holds: (B1) B 6= ∅; (B2) For any B1 , B2 ∈ B and x ∈ B1 \ B2 there exists a y ∈ B2 \ B1 with (B1 \ {x}) ∪ {y} ∈ B. Proof: The set of bases of a matroid satisfies (B1) (by (M1)) and (B2): For bases B1 , B2 and x ∈ B1 \ B2 we have that B1 \ {x} is independent. By (M3) there is some y ∈ B2 \ B1 such that (B1 \ {x}) ∪ {y} is independent. Indeed, it must be a basis, because all bases of a matroid have the same cardinality. On the other hand, let B satisfy (B1) and (B2). We first show that all elements of B have the same cardinality: Otherwise let B1 , B2 ∈ B with |B1 | > |B2 | such that |B1 ∩ B2 | is maximum. Let x ∈ B1 \ B2 . By (B2) there is a y ∈ B2 \ B1 with (B1 \ {x}) ∪ {y} ∈ B, contradicting the maximality of |B1 ∩ B2 |.

310

13 Matroids

Now let F := {F ⊆ E : there exists a B ∈ B with F ⊆ B}. (E, F) is an independence system, and B is the family of its bases. To show that (E, F) satisfies (M3), let X, Y ∈ F with |X | > |Y |. Let X ⊆ B1 ∈ B and Y ⊆ B2 ∈ B, where B1 and B2 are chosen such that |B1 ∩ B2 | is maximum. If B2 ∩ (X \ Y ) 6= ∅, we are done because we can augment Y . We claim that the other case, B2 ∩ (X \ Y ) = ∅, is impossible. Namely, with this assumption we get |B1 ∩ B2 | + |Y \ B1 | + |(B2 \ B1 ) \ Y | = |B2 | = |B1 | ≥ |B1 ∩ B2 | + |X \ Y |. Since |X \ Y | > |Y \ X | ≥ |Y \ B1 |, this implies (B2 \ B1 ) \ Y 6= ∅. So let y ∈ (B2 \ B1 ) \ Y . By (B2) there exists an x ∈ B1 \ B2 with (B2 \ {y}) ∪ {x} ∈ B, contradicting the maximality of |B1 ∩ B2 |. See Exercise 7 for a similar statement. A very important property of matroids is that the rank function is submodular: Theorem 13.10. Let E be a finite set and r : 2 E → Z+ . Then the following statements are equivalent: (a) r is the rank function of a matroid (E, F) (and F = {F ⊆ E : r (F) = |F|}). (b) For all X, Y ⊆ E: (R1) r (X ) ≤ |X |; (R2) If X ⊆ Y then r (X ) ≤ r (Y ); (R3) r (X ∪ Y ) + r (X ∩ Y ) ≤ r (X ) + r (Y ). (c) For all X ⊆ E and x, y ∈ E: (R10 ) r (∅) = 0; (R20 ) r (X ) ≤ r (X ∪ {y}) ≤ r (X ) + 1; (R30 ) If r (X ∪ {x}) = r (X ∪ {y}) = r (X ) then r (X ∪ {x, y}) = r (X ). Proof: (a)⇒(b): If r is a rank function of an independence system (E, F), (R1) and (R2) evidently hold. If (E, F) is a matroid, we can also show (R3): Let X, Y ⊆ E, and let A be a basis of X ∩ Y . By (M3), A can be extended . . to a basis A ∪ B of X and to a basis (A ∪ B) ∪ C of X ∪ Y . Then A ∪ C is an independent subset of Y , so r (X ) + r (Y ) ≥ |A ∪ B| + |A ∪ C| = 2|A| + |B| + |C| = |A ∪ B ∪ C| + |A| = r (X ∪ Y ) + r (X ∩ Y ). (b)⇒(c): (R10 ) is implied by (R1). r (X ) ≤ r (X ∪ {y}) follows from (R2). By (R3) and (R1),

13.2 Other Matroid Axioms

311

r (X ∪ {y}) ≤ r (X ) + r ({y}) − r (X ∩ {y}) ≤ r (X ) + r ({y}) ≤ r (X ) + 1, proving (R20 ). (R30 ) is trivial for x = y. For x 6= y we have, by (R2) and (R3), 2r (X ) ≤ r (X ) + r (X ∪ {x, y}) ≤ r (X ∪ {x}) + r (X ∪ {y}), implying (R30 ). (c)⇒(a): Let r : 2 E → Z+ be a function satisfying (R10 )–(R30 ). Let F := {F ⊆ E : r (F) = |F|}. We claim that (E, F) is a matroid. (M1) follows from (R10 ). (R20 ) implies r (X ) ≤ |X | for all X ⊆ E. If Y ∈ F, y ∈ Y and X := Y \ {y}, we have |X | + 1 = |Y | = r (Y ) = r (X ∪ {y}) ≤ r (X ) + 1 ≤ |X | + 1, so X ∈ F. This implies (M2). Now let X, Y ∈ F and |X | = |Y | + 1. Let X \ Y = {x1 , . . . , xk }. Suppose that (M30 ) is violated, i.e. r (Y ∪ {xi }) = |Y | for i = 1, . . . , k. Then by (R30 ) r (Y ∪ {x1 , xi }) = r (Y ) for i = 2, . . . , k. Repeated application of this argument yields r (Y ) = r (Y ∪ {x1 , . . . , xk }) = r (X ∪ Y ) ≥ r (X ), a contradiction. So (E, F) is indeed a matroid. To show that r is the rank function of this matroid, we have to prove that r (X ) = max{|Y | : Y ⊆ X, r (Y ) = |Y |} for all X ⊆ E. So let X ⊆ E, and let Y a maximum subset of X with r (Y ) = |Y |. For all x ∈ X \ Y we have r (Y ∪ {x}) < |Y | + 1, so by (R20 ) r (Y ∪ {x}) = |Y |. Repeated application of (R30 ) implies r (X ) = |Y |. Theorem 13.11. Let E be a finite set and σ : 2 E → 2 E a function. σ is the closure operator of a matroid (E, F) if and only if the following conditions hold for all X, Y ⊆ E and x, y ∈ E: (S1) (S2) (S3) (S4)

X ⊆ σ (X ); X ⊆ Y ⊆ E implies σ (X ) ⊆ σ (Y ); σ (X ) = σ (σ (X )); If y ∈ / σ (X ) and y ∈ σ (X ∪ {x}) then x ∈ σ (X ∪ {y}).

Proof: If σ is the closure operator of a matroid, then (S1) holds trivially. For X ⊆ Y and z ∈ σ (X ) we have by (R3) and (R2) r (X ) + r (Y ) = r (X ∪ {z}) + r (Y ) ≥ r ((X ∪ {z}) ∩ Y ) + r (X ∪ {z} ∪ Y ) ≥ r (X ) + r (Y ∪ {z}), implying z ∈ σ (Y ) and thus proving (S2). By repeated application of (R30 ) we have r (σ (X )) = r (X ) for all X , which implies (S3).

312

13 Matroids

To prove (S4), suppose that there are X, x, y with y ∈ / σ (X ), y ∈ σ (X ∪ {x}) and x ∈ / σ (X ∪ {y}). Then r (X ∪ {y}) = r (X ) + 1, r (X ∪ {x, y}) = r (X ∪ {x}) and r (X ∪ {x, y}) = r (X ∪ {y}) + 1. Thus r (X ∪ {x}) = r (X ) + 2, contradicting (R20 ). To show the converse, let σ : 2 E → 2 E be a function satisfying (S1)–(S4). Let F := {X ⊆ E : x ∈ / σ (X \ {x}) for all x ∈ X }. We claim that (E, F) is a matroid. (M1) is trivial. For X ⊆ Y ∈ F and x ∈ X we have x ∈ / σ (Y \{x}) ⊇ σ (X \{x}), so X ∈ F and (M2) holds. To prove (M3) we need the following statement: Claim: For X ∈ F and Y ⊆ E with |X | > |Y | we have X 6⊆ σ (Y ). We prove the claim by induction on |Y \ X |. If Y ⊂ X , then let x ∈ X \ Y . Since X ∈ F we have x ∈ / σ (X \ {x}) ⊇ σ (Y ) by (S2). Hence x ∈ X \ σ (Y ) as required. If |Y \ X | > 0, then let y ∈ Y \ X . By the induction hypothesis there exists an x ∈ X \ σ (Y \ {y}). If x 6∈ σ (Y ), then we are done. Otherwise x ∈ / σ (Y \ {y}) but x ∈ σ (Y ) = σ ((Y \ {y}) ∪ {y}), so by (S4) y ∈ σ ((Y \ {y}) ∪ {x}). By (S1) we get Y ⊆ σ ((Y \ {y}) ∪ {x}) and thus σ (Y ) ⊆ σ ((Y \ {y}) ∪ {x}) by (S2) and (S3). Applying the induction hypothesis to X and (Y \ {y}) ∪ {x} (note that x 6= y) yields X 6⊆ σ ((Y \ {y}) ∪ {x}), so X 6⊆ σ (Y ) as required. Having proved the claim we can easily verify (M3). Let X, Y ∈ F with |X | > |Y |. By the claim there exists an x ∈ X \ σ (Y ). Now for each z ∈ Y ∪ {x} we have z ∈ / σ (Y \ {z}), because Y ∈ F and x ∈ / σ (Y ) = σ (Y \ {x}). By (S4) z∈ / σ (Y \ {z}) and x ∈ / σ (Y ) imply z ∈ / σ ((Y \ {z}) ∪ {x}) ⊇ σ ((Y ∪ {x}) \ {z}). Hence Y ∪ {x} ∈ F. So (M3) indeed holds and (E, F) is a matroid, say with rank function r and closure operator σ 0 . It remains to prove that σ = σ 0 . By definition, σ 0 (X ) = {y ∈ E : r (X ∪ {y}) = r (X )} and r (X ) = max{|Y | : Y ⊆ X, y ∈ / σ (Y \ {y}) for all y ∈ Y } for all X ⊆ E. Let X ⊆ E. To show σ 0 (X ) ⊆ σ (X ), let z ∈ σ 0 (X ) \ X . Let Y be a basis of X . Since r (Y ∪ {z}) ≤ r (X ∪ {z}) = r (X ) = |Y | < |Y ∪ {z}| we have y ∈ σ ((Y ∪{z})\{y}) for some y ∈ Y ∪{z}. If y = z, then we have z ∈ σ (Y ). Otherwise (S4) and y ∈ / σ (Y \ {y}) also yield z ∈ σ (Y ). Hence by (S2) z ∈ σ (X ). Together with (S1) this implies σ 0 (X ) ⊆ σ (X ). Now let z ∈ / σ 0 (X ), i.e. r (X ∪ {z}) > r (X ). Let now Y be a basis of X ∪ {z}. Then z ∈ Y and |Y \ {z}| = |Y | − 1 = r (X ∪ {z}) − 1 = r (X ). Therefore Y \ {z} is a basis of X , implying X ⊆ σ 0 (Y \ {z}) ⊆ σ (Y \ {z}), and thus σ (X ) ⊆ σ (Y \ {z}). As z ∈ / σ (Y \ {z}), we conclude that z 6∈ σ (X ). Theorem 13.12. Let E be a finite set and C ⊆ 2 E . C is the set of circuits of an independence system (E, F), where F = {F ⊂ E : there exists no C ∈ C with C ⊆ F}, if and only if the following conditions hold: (C1) ∅ ∈ / C;

13.3 Duality

313

(C2) For any C1 , C2 ∈ C, C1 ⊆ C2 implies C1 = C2 . Moreover, if C is the set of circuits of an independence system (E, F), then the following statements are equivalent: (a) (E, F) is a matroid. (b) For any X ∈ F and e ∈ E, X ∪ {e} contains at most one circuit. (C3) For any C1 , C2 ∈ C with C1 6= C2 and e ∈ C1 ∩ C2 there exists a C3 ∈ C with C3 ⊆ (C1 ∪ C2 ) \ {e}. (C30 ) For any C1 , C2 ∈ C, e ∈ C1 ∩ C2 and f ∈ C1 \ C2 there exists a C3 ∈ C with f ∈ C3 ⊆ (C1 ∪ C2 ) \ {e}. Proof: By definition, the family of circuits of any independence system satisfies (C1) and (C2). If C satisfies (C1), then (E, F) is an independence system. If C also satisfies (C2), it is the set of circuits of this independence system. (a)⇒(C30 ): Let C be the family of circuits of a matroid, and let C1 , C2 ∈ C, e ∈ C1 ∩ C2 and f ∈ C1 \ C2 . By applying (R3) twice we have |C1 | − 1 + r ((C1 ∪ C2 ) \ {e, f }) + |C2 | − 1 = r (C1 ) + r ((C1 ∪ C2 ) \ {e, f }) + r (C2 ) ≥ r (C1 ) + r ((C1 ∪ C2 ) \ { f }) + r (C2 \ {e}) ≥ r (C1 \ { f }) + r (C1 ∪ C2 ) + r (C2 \ {e}) = |C1 | − 1 + r (C1 ∪ C2 ) + |C2 | − 1. So r ((C1 ∪ C2 ) \ {e, f }) = r (C1 ∪ C2 ). Let B be a basis of (C1 ∪ C2 ) \ {e, f }. Then B ∪ { f } contains a circuit C3 , with f ∈ C3 ⊆ (C1 ∪ C2 ) \ {e} as required. (C30 )⇒(C3): trivial. (C3)⇒(b): If X ∈ F and X ∪ {e} contains two circuits C1 , C2 , (C3) implies (C1 ∪ C2 ) \ {e} ∈ / F. However, (C1 ∪ C2 ) \ {e} is a subset of X . (b)⇒(a): Follows from Theorem 13.8 and Proposition 13.7. Especially property (b) will be used often. For X ∈ F and e ∈ E such that X ∪ {e} 6∈ F we write C(X, e) for the unique circuit in X ∪ {e}. If X ∪ {e} ∈ F we write C(X, e) := ∅.

13.3 Duality Another basic concept in matroid theory is duality. Definition 13.13. Let (E, F) be an independence system. We define the dual of (E, F) by (E, F ∗ ), where F ∗ = {F ⊆ E : there is a basis B of (E, F) such that F ∩ B = ∅}. It is obvious that the dual of an independence system is again an independence system.

314

13 Matroids

Proposition 13.14. (E, F ∗∗ ) = (E, F). Proof: F ∈ F ∗∗ ⇔ there is a basis B ∗ of (E, F ∗ ) such that F ∩ B ∗ = ∅ ⇔ there is a basis B of (E, F) such that F ∩ (E \ B) = ∅ ⇔ F ∈ F. Theorem 13.15. Let (E, F) be an independence system, (E, F ∗ ) its dual, and let r and r ∗ be the corresponding rank functions. (a) (E, F) is a matroid if and only if (E, F ∗ ) is a matroid. (Whitney [1935]) (b) If (E, F) is a matroid, then r ∗ (F) = |F| + r (E \ F) − r (E) for F ⊆ E. Proof: Due to Proposition 13.14 we have to show only one direction of (a). So let (E, F) be a matroid. We define q : 2 E → Z+ by q(F) := |F| + r (E \ F) − r (E). We claim that q satisfies (R1), (R2) and (R3). By this claim and Theorem 13.10, q is the rank function of a matroid. Since obviously q(F) = |F| if and only if F ∈ F ∗ , we conclude that q = r ∗ , and (a) and (b) are proved. Now we prove the above claim: q satisfies (R1) because r satisfies (R2). To check that q satisfies (R2), let X ⊆ Y ⊆ E. Since (E, F) is a matroid, (R3) holds for r , so r (E \ X ) + 0 = r ((E \ Y ) ∪ (Y \ X )) + r (∅) ≤ r (E \ Y ) + r (Y \ X ). We conclude that r (E \ X ) − r (E \ Y ) ≤ r (Y \ X ) ≤ |Y \ X | = |Y | − |X | (note that r satisfies (R1)), so q(X ) ≤ q(Y ). It remains to show that q satisfies (R3). Let X, Y ⊆ E. Using the fact that r satisfies (R3) we have q(X ∪ Y ) + q(X ∩ Y ) = |X ∪ Y | + |X ∩ Y | + r (E \ (X ∪ Y )) + r (E \ (X ∩ Y )) − 2r (E) = |X | + |Y | + r ((E \ X ) ∩ (E \ Y )) + r ((E \ X ) ∪ (E \ Y )) − 2r (E) ≤ |X | + |Y | + r (E \ X ) + r (E \ Y ) − 2r (E) = q(X ) + q(Y ). For any graph G we have introduced the cycle matroid M(G) which of course has a dual. For an embedded planar graph G there is also a planar dual G ∗ (which in general depends on the embedding of G). It is interesting that the two concepts of duality coincide: Theorem 13.16. Let G be a connected planar graph with an arbitrary planar embedding, and G ∗ the planar dual. Then M(G ∗ ) = (M(G))∗ .

13.3 Duality

315

∗

Proof: For T ⊆ E(G) we write T := {e∗ : e ∈ E(G) \ T }, where e∗ is the dual of edge e. We have to prove the following: ∗ Claim: T is the edge set of a spanning tree in G iff T is the edge set of a ∗ spanning tree in G . ∗ ∗

Since (G ∗ )∗ = G (by Proposition 2.42) and (T ) = T it suffices to prove one direction of the claim. ∗ So let T ⊆ E(G), where T is the edge set of a spanning tree in G ∗ . (V (G), T ) must be connected, for otherwise a connected component would define a cut, the ∗ dual of which contains a circuit in T (Theorem 2.43). On the other hand, if ∗ (V (G), T ) contains a circuit, then the dual edge set is a cut and (V (G ∗ ), T ) is disconnected. Hence (V (G), T ) is indeed a spanning tree in G. This implies that if G is planar then (M(G))∗ is a graphic matroid. If, for any graph G, (M(G))∗ is a graphic matroid, say (M(G))∗ = M(G 0 ), then G 0 is evidently an abstract dual of G. By Exercise 34 of Chapter 2, the converse is also true: G is planar if and only if G has an abstract dual (Whitney [1933]). This implies that (M(G))∗ is graphic if and only if G is planar. Note that Theorem 13.16 quite directly implies Euler’s formula (Theorem 2.32): Let G be a connected planar graph with a planar embedding, and let M(G) be the cycle matroid of G. By Theorem 13.15 (b), r (E(G)) + r ∗ (E(G)) = |E(G)|. Since r (E(G)) = |V (G)| − 1 (the number of edges in a spanning tree) and r ∗ (E(G)) = |V (G ∗ )| − 1 (by Theorem 13.16), we obtain that the number of faces of G is |V (G ∗ )| = |E(G)| − |V (G)| + 2, Euler’s formula. Duality of independence systems has also some nice applications in polyhedral combinatorics. A set system (E, F) is called a clutter if X 6⊂ Y for all X, Y ∈ F. If (E, F) is a clutter, then we define its blocking clutter by B L(E, F) := (E, {X ⊆ E : X ∩ Y 6= ∅ for all Y ∈ F, X minimal with this property}). For an independence system (E, F) and its dual (E, F ∗ ) let B and B ∗ be the family of bases, and C and C ∗ the family of circuits, respectively. (Every clutter arises in both of these ways except for F = ∅ or F = {∅}.) It follows immediately from the definitions that (E, B ∗ ) = B L(E, C) and (E, C ∗ ) = B L(E, B). Together with Proposition 13.14 this implies B L(B L(E, F)) = (E, F) for every clutter (E, F). We give some examples for clutters (E, F) and their blocking clutters (E, F 0 ). In each case E = E(G) for some graph G: (1) F is the set of spanning trees, F 0 is the set of minimal cuts; (2) F is the set of arborescences rooted at r , F 0 is the set of minimal r -cuts; (3) F is the set of s-t-paths, F 0 is the set of minimal cuts separating s and t (this example works in undirected graphs and in digraphs); (4) F is the set of circuits in an undirected graph, F 0 is the set of complements of maximal forests;

316

13 Matroids

(5) F is the set of circuits in a digraph, F 0 is the set of minimal feedback edge sets (a feedback edge set is a set of edges whose removal makes the digraph acyclic); (6) F is the set of minimal edge sets whose contraction makes the digraph strongly connected, F 0 is the set of minimal directed cuts; (7) F is the set of minimal T -joins, F 0 is the set of minimal T -cuts. All these blocking relations can be verified easily: (1) and (2) follow directly from Theorems 2.4 and 2.5, (3), (4) and (5) are trivial, (6) follows from Corollary 2.7, and (7) from Proposition 12.6. In some cases, the blocking clutter gives a polyhedral characterization of the M INIMIZATION P ROBLEM F OR I NDEPENDENCE S YSTEMS for nonnegative cost functions: Definition 13.17. Let (E, F) be a clutter, (E, F 0 ) its blocking clutter and P the convex hull of the incidence vectors of the elements of F. We say that (E, F) has the Max-Flow-Min-Cut property if ) ( o n X E E 0 xe ≥ 1 for all B ∈ F . x + y : x ∈ P, y ∈ R+ = x ∈ R+ : e∈B

Examples are (2) and (7) of our list above (by Theorems 6.14 and 12.16), but also (3) and (6) (see Exercise 10). The following theorem relates the above covering-type formulation to a packing formulation of the dual problem and allows to derive certain min-max theorems from others: Theorem 13.18. (Fulkerson [1971], Lehman [1979]) Let (E, F) be a clutter and (E, F 0 ) its blocking clutter. Then the following statements are equivalent: (a) (E, F) has the Max-Flow-Min-Cut property; property; (b) (E, F 0 ) has the Max-Flow-Min-Cut 0 P (c) min{c(A) : A ∈ F} = max 1ly : y ∈ RF + , B∈F 0 :e∈B y B ≤ c(e) for all e ∈ E for every c : E → R+ . Proof: Since B L(E, F 0 ) = B L(B L(E, F)) = (E, F) it suffices to prove (a)⇒(c)⇒(b). The other implication (b)⇒(a) then follows by exchanging the roles of F and F 0 . (a)⇒(c): By Corollary 3.33 we have for every c : E → R+ o n E , min{c(A) : A ∈ F} = min{cx : x ∈ P} = min c(x + y) : x ∈ P, y ∈ R+ where P is the convex hull of the incidence vectors of elements of F. From this, the Max-Flow-Min-Cut property and the LP Duality Theorem 3.20 we get (c). (c)⇒(b): Let P 0 denote the convex hull of the incidence vectors of the elements of F 0 . We have to show that ( ) n o X E E : xe ≥ 1 for all A ∈ F . x + y : x ∈ P 0 , y ∈ R+ = x ∈ R+ e∈A

13.4 The Greedy Algorithm

317

Since “⊆” is trivial from the definition of blocking clutters we only show the other P E be a vector with inclusion. So let c ∈ R+ c e∈A e ≥ 1 for all A ∈ F. By (c) we have 1 ≤ min{c(A) : A ∈ F} ( F0

= max 1ly : y ∈ R+ ,

) X

y B ≤ c(e) for all e ∈ E ,

B∈F 0 :e∈B

F 0 be a vector with 1ly = 1 and P so let y ∈ RP + B∈F 0 :e∈B y B ≤ c(e) for all e ∈ E. defines a vector x ∈ P 0 with x ≤ c, proving Then xe := B∈F 0 :e∈B y B (e ∈ E) 0 E that c ∈ x + y : x ∈ P , y ∈ R+ .

For example, this theorem implies the Max-Flow-Min-Cut Theorem 8.6 quite directly: Let (G, u, s, t) be a network. By Exercise 1 of Chapter 7 the minimum length of an s-t-path in (G, u) equals the maximum number of s-t-cuts such that each edge e is contained in at most u(e) of them. Hence the clutter of s-t-paths (example (3) in the above list) has the Max-Flow-Min-Cut Property, and so has its blocking clutter. Now (c) applied to the clutter of minimal s-t-cuts implies the Max-Flow-Min-Cut Theorem. Note however that Theorem 13.18 does not guarantee an integral vector attaining the maximum in (c), even if c is integral. The clutter of T -joins for G = K 4 and T = V (G) shows that this does not exist in general.

13.4 The Greedy Algorithm Again, let (E, F) be an independence system and c : E → R+ . We consider the M AXIMIZATION P ROBLEM for (E, F, c) and formulate two “greedy algorithms”. We do not have to consider negative weights since elements with negative weight never appear in an optimum solution. We assume that (E, F) is given by an oracle. For the first algorithm we simply assume an independence oracle, i.e. an oracle which, given a set F ⊆ E, decides whether F ∈ F or not.

B EST-I N -G REEDY A LGORITHM Input: Output:

An independence system (E, F), given by an independence oracle. Weights c : E → R+ . A set F ∈ F.

1

Sort E = {e1 , e2 , . . . , en } such that c(e1 ) ≥ c(e2 ) ≥ · · · ≥ c(en ).

2

Set F := ∅.

3

For i := 1 to n do: If F ∪ {ei } ∈ F then set F := F ∪ {ei }.

318

13 Matroids

The second algorithm requires a more complicated oracle. Given a set F ⊆ E, this oracle decides whether F contains a basis. Let us call such an oracle a basissuperset oracle.

W ORST-O UT-G REEDY A LGORITHM Input: Output:

An independence system (E, F), given by a basis-superset oracle. Weights c : E → R+ . A basis F of (E, F).

1

Sort E = {e1 , e2 , . . . , en } such that c(e1 ) ≤ c(e2 ) ≤ · · · ≤ c(en ).

2

Set F := E.

3

For i := 1 to n do: If F \ {ei } contains a basis then set F := F \ {ei }.

Before we analyse these algorithms, let us take a closer look at the oracles required. It is an interesting question whether such oracles are polynomially equivalent, i.e. whether one can be simulated by a polynomial-time oracle algorithm using the other. The independence oracle and the basis-superset oracle do not seem to be polynomially equivalent: If we consider the independence system for the TSP (example (2) of the list in Section 13.1), it is easy (and the subject of Exercise 13) to decide whether a set of edges is independent, i.e. the subset of a Hamiltonian circuit (recall that we are working with a complete graph). On the other hand, it is a difficult problem to decide whether a set of edges contains a Hamiltonian circuit (this is NP-complete; cf. Theorem 15.25). Conversely, in the independence system for the S HORTEST PATH P ROBLEM (example (3)), it is easy to decide whether a set of edges contains an s-t-path. Here it is not known how to decide whether a given set is independent (i.e. subset of an s-t-path) in polynomial time (Korte and Monma [1979] proved NP-completeness). For matroids, both oracles are polynomially equivalent. Other equivalent oracles are the rank oracle and closure oracle, which return the rank and the closure of a given subset of E, respectively (Exercise 16). However, even for matroids there are other natural oracles that are not polynomially equivalent. For example, the oracle deciding whether a given set is a basis is weaker than the independence oracle. The oracle which for a given F ⊆ E returns the minimum cardinality of a dependent subset of F is stronger than the independence oracle (Hausmann and Korte [1981]). One can analogously formulate both greedy algorithms for the M INIMIZA TION P ROBLEM . It is easy to see that the B EST-I N -G REEDY for the M AXIMIZA TION P ROBLEM for (E, F, c) corresponds to the W ORST-O UT-G REEDY for the M INIMIZATION P ROBLEM for (E, F ∗ , c): adding an element to F in the B ESTI N -G REEDY corresponds to removing an element from F in the W ORST-O UTG REEDY. Observe that K RUSKAL ’ S A LGORITHM (see Section 6.1) is a B EST-I N G REEDY algorithm for the M INIMIZATION P ROBLEM in a cycle matroid.

13.4 The Greedy Algorithm

319

The rest of this section contains some results concerning the quality of a solution found by the greedy algorithms. Theorem 13.19. (Jenkyns [1976], Korte and Hausmann [1978]) Let (E, F) be an independence system. For c : E → R+ we denote by G(E, F, c) the cost of some solution found by the B EST-I N -G REEDY for the M AXIMIZATION P ROBLEM, and by OPT(E, F, c) the cost of an optimum solution. Then q(E, F) ≤

G(E, F, c) ≤ 1 OPT(E, F, c)

for all c : E → R+ . There is a cost function where the lower bound is attained. Proof: Let E = {e1 , e2 , . . . , en }, c : E → R+ , and c(e1 ) ≥ c(e2 ) ≥ . . . ≥ c(en ). Let G n be the solution found by the B EST-I N -G REEDY (when sorting E like this), while On is an optimum solution. We define E j := {e1 , . . . , e j }, G j := G n ∩ E j and O j := On ∩ E j ( j = 0, . . . , n). Set dn := c(en ) and d j := c(e j ) − c(e j+1 ) for j = 1, . . . , n − 1. Since O j ∈ F, we have |O j | ≤ r (E j ). Since G j is a basis of E j , we have |G j | ≥ ρ(E j ). With these two inequalities we conclude that c(G n ) =

=

≥

n X j=1 n X j=1 n X

(|G j | − |G j−1 |) c(e j )

|G j | d j ρ(E j ) d j

j=1

≥ q(E, F)

≥ q(E, F)

= q(E, F)

n X j=1 n X j=1 n X

r (E j ) d j

(13.1)

|O j | d j (|O j | − |O j−1 |) c(e j )

j=1

= q(E, F) c(On ). Finally we show that the lower bound is tight. Choose F ⊆ E and bases B1 , B2 of F such that |B1 | = q(E, F). |B2 | Define

320

13 Matroids

( c(e) :=

1 0

for e ∈ F for e ∈ E \ F

and sort e1 , . . . , en such that c(e1 ) ≥ c(e2 ) ≥ . . . ≥ c(en ) and B1 = {e1 , . . . , e|B1 | }. Then G(E, F, c) = |B1 | and OPT(E, F, c) = |B2 |, and the lower bound is attained. In particular we have the so-called Edmonds-Rado Theorem: Theorem 13.20. (Rado [1957], Edmonds [1971]) An independence system (E, F) is a matroid if and only if the B EST-I N -G REEDY finds an optimum solution for the M AXIMIZATION P ROBLEM for (E, F, c) for all cost functions c : E → R+ . Proof: By Theorem 13.19 we have q(E, F) < 1 if and only if there exists a cost function c : E → R+ for which the B EST-I N -G REEDY does not find an optimum solution. By Proposition 13.7 we have q(E, F) < 1 if and only if (E, F) is not a matroid. This is one of the rare cases where we can define a structure by its algorithmic behaviour. We also obtain a polyhedral description: Theorem 13.21. (Edmonds [1970]) Let (E, F) be a matroid and r : E → Z+ its rank function. Then the matroid polytope of (E, F), i.e. the convex hull of the incidence vectors of all elements of F, is equal to ) ( X E xe ≤ r (A) for all A ⊆ E . x ∈ R : x ≥ 0, e∈A

Proof: Obviously, this polytope contains all incidence vectors of independent sets. By Corollary 3.32 it remains to show that all vertices of this polytope are integral. By Theorem 5.13 this is equivalent to showing that ) ( X xe ≤ r (A) for all A ⊆ E (13.2) max cx : x ≥ 0, e∈A

has an integral optimum solution for any c : E → R. W.l.o.g. c(e) ≥ 0 for all e, = 0. since for e ∈ E with c(e) < 0 any optimum solution x of (13.2) has xe P Let x be an optimum solution of (13.2). In (13.1) we replace |O j | by e∈E j xe P ( j = 0, . . . , n). We obtain c(G n ) ≥ e∈E c(e)xe . So the B EST-I N -G REEDY produces a solution whose incidence vector is another optimum solution of (13.2). When applied to graphic matroids, this also yields Theorem 6.12. As in this special case, we also have total dual integrality in general. A generalization of this result will be proved in Section 14.2. The above observation that the B EST-I N -G REEDY for the M AXIMIZATION P ROBLEM for (E, F, c) corresponds to the W ORST-O UT-G REEDY for the M IN IMIZATION P ROBLEM for (E, F ∗ , c) suggests the following dual counterpart of Theorem 13.19:

13.4 The Greedy Algorithm

321

Theorem 13.22. (Korte and Monma [1979]) Let (E, F) be an independence system. For c : E → R+ let G(E, F, c) denote a solution found by the W ORSTO UT-G REEDY for the M INIMIZATION P ROBLEM. Then G(E, F, c) |F| − ρ ∗ (F) ≤ max F⊆E |F| − r ∗ (F) OPT(E, F, c)

1 ≤

(13.3)

for all c : E → R+ , where ρ ∗ and r ∗ are the rank functions of the dual independence system (E, F ∗ ). There is a cost function where the upper bound is attained. Proof: We use the same notation as in the proof of Theorem 13.19. By construction, G j ∪ (E \ E j ) contains a basis of E, but (G j ∪ (E \ E j )) \ {e} does not contain a basis of E for any e ∈ G j ( j = 1, . . . , n). In other words, E j \ G j is a basis of E j with respect to (E, F ∗ ), so |E j | − |G j | ≥ ρ ∗ (E j ). Since On ⊆ E \ (E j \ O j ) and On is a basis, E j \ O j is independent in (E, F ∗ ), so |E j | − |O j | ≤ r ∗ (E j ). We conclude that |G j | ≤ |E j | − ρ ∗ (E j ) |O j | ≥ |E j | − r ∗ (E j ).

and

Now the same calculation as (13.1) provides the upper bound. To see that this bound is tight, consider 1 for e ∈ F c(e) := , 0 for e ∈ E \ F where F ⊆ E is a set where the maximum in (13.3) is attained. Let B1 be a basis of F with respect to (E, F ∗ ), with |B1 | = ρ ∗ (F). If we sort e1 , . . . , en such that c(e1 ) ≥ c(e2 ) ≥ . . . ≥ c(en ) and B1 = {e1 , . . . , e|B1 | }, we have G(E, F, c) = |F| − |B1 | and OPT(E, F, c) = |F| − r ∗ (F).

1

2

M >> 2

Fig. 13.1.

If we apply the W ORST-O UT-G REEDY to the M AXIMIZATION P ROBLEM or the B EST-I N -G REEDY to the M INIMIZATION P ROBLEM, there is no positive G(E,F ,c) lower/ﬁnite upper bound for OPT(E, F ,c) . To see this, consider the problem of ﬁnding a minimal vertex cover of maximum weight or a maximal stable set of minimum weight in the simple graph shown in Figure 13.1. However in the case of matroids, it does not matter whether we use the B EST-I N G REEDY or the W ORST-O UT-G REEDY: since all bases have the same cardinality, the M INIMIZATION P ROBLEM for (E, F, c) is equivalent to the M AXIMIZATION

322

13 Matroids

P ROBLEM for (E, F, c0 ), where c0 (e) := M − c(e) for all e ∈ E and M := 1 + max{c(e) : e ∈ E}. Therefore K RUSKAL ’ S A LGORITHM (Section 6.1) solves the M INIMUM S PANNING T REE P ROBLEM optimally. The Edmonds-Rado Theorem 13.20 also yields the following characterization of optimum k-element solutions of the M AXIMIZATION P ROBLEM. Theorem 13.23. Let (E, F) be a matroid, c : E → R, k ∈ N and X ∈ F with |X | = k. Then c(X ) = max{c(Y ) : Y ∈ F, |Y | = k} if and only if the following two conditions hold: (a) For all y ∈ E \ X with X ∪ {y} ∈ / F and all x ∈ C(X, y) we have c(x) ≥ c(y); (b) For all y ∈ E \ X with X ∪ {y} ∈ F and all x ∈ X we have c(x) ≥ c(y). Proof: The necessity is trivial: if one of the conditions is violated for some y and x, the k-element set X 0 := (X ∪ {y}) \ {x} ∈ F has greater cost than X . To see the sufficiency, let F 0 := {F ∈ F : |F| ≤ k} and c0 (e) := c(e) + M for all e ∈ E, where M = max{|c(e)| : e ∈ E}. Sort E = {e1 , . . . , en } such that c0 (e1 ) ≥ · · · ≥ c0 (en ) and, for any i, c0 (ei ) = c0 (ei+1 ) and ei+1 ∈ X imply ei ∈ X (i.e. elements of X come first among those of equal weight). Let X 0 be the solution found by the B EST-I N -G REEDY for the instance (E, F 0 , c0 ) (sorted like this). Since (E, F 0 ) is a matroid, the Edmonds-Rado Theorem 13.20 implies: c(X 0 ) + k M = c0 (X 0 ) = max{c0 (Y ) : Y ∈ F 0 } = max{c(Y ) : Y ∈ F, |Y | = k} + k M. We conclude the proof by showing that X = X 0 . We know that |X | = k = |X 0 |. So suppose X 6= X 0 , and let ei ∈ X 0 \ X with i minimum. Then X ∩ {e1 , . . . , ei−1 } = / F, then (a) implies C(X, ei ) ⊆ X 0 , a X 0 ∩ {e1 , . . . , ei−1 }. Now if X ∪ {ei } ∈ contradiction. If X ∪ {ei } ∈ F, then (b) implies X ⊆ X 0 which is also impossible. We shall need this theorem in Section 13.7. The special case that (E, F) is a graphic matroid and k = r (E) is part of Theorem 6.2.

13.5 Matroid Intersection Definition 13.24. Given two independence systems (E, F1 ) and (E, F2 ), we define their intersection by (E, F1 ∩ F2 ). The intersection of a finite number of independence systems is defined analogously. It is clear that the result is again an independence system. Proposition 13.25. Any independence system (E, F) is the intersection of a finite number of matroids.

13.5 Matroid Intersection

323

Proof: Each circuit C of (E, F) defines a matroid (E, {F ⊆ E : C \ F 6= ∅}) by Theorem 13.12. The intersection of all these matroids is of course (E, F). Since the intersection of matroids is not a matroid in general, we cannot hope to get an optimum common independent set by a greedy algorithm. However, the following result, together with Theorem 13.19, implies a bound for the solution found by the B EST-I N -G REEDY: Proposition 13.26. If (E, F) is the intersection of p matroids, then q(E, F) ≥

1 p.

Proof: By Theorem 13.12(b), X ∪ {e} contains at most p circuits for any X ∈ F and e ∈ E. The statement now follows from Theorem 13.8. Of particular interest are independence systems that are the intersection of two matroids. The prime example here is the matching problem in a bipartite graph . G = (A ∪ B, E). If F := {F ⊆ E : F is a matching in G}, then (E, F) is the intersection of two matroids. Namely, let F1 := {F ⊆ E : |δ F (x)| ≤ 1 for all x ∈ A}

and

F2 := {F ⊆ E : |δ F (x)| ≤ 1 for all x ∈ B}. (E, F1 ), (E, F2 ) are matroids by Proposition 13.4(d). Clearly, F = F1 ∩ F2 . A second example is the independence system consisting of all branchings in a digraph G (Example 8 of the list at the beginning of Section 13.1). Here one matroid contains all sets of edges such that each vertex has at most one entering edge (see Proposition 13.4(e)), while the second matroid is the cycle matroid M(G) of the underlying undirected graph. We shall now describe Edmonds’ algorithm for the following problem:

M ATROID I NTERSECTION P ROBLEM Instance:

Two matroids (E, F1 ), (E, F2 ), given by independence oracles.

Task:

Find a set F ∈ F1 ∩ F2 such that |F| is maximum.

We start with the following lemma. Recall that, for X ∈ F and e ∈ E, C(X, e) denotes the unique circuit in X ∪ {e} if X ∪ {e} ∈ / F, and C(X, e) = ∅ otherwise. Lemma 13.27. (Frank [1981]) Let (E, F) be a matroid and X ∈ F. Let / X with x1 , . . . , xs ∈ X and y1 , . . . , ys ∈ (a) xk ∈ C(X, yk ) for k = 1, . . . , s and / C(X, yk ) for 1 ≤ j < k ≤ s. (b) x j ∈ Then (X \ {x1 , . . . , xs }) ∪ {y1 , . . . , ys } ∈ F. Proof: Let X r := (X \ {x1 , . . . , xr }) ∪ {y1 , . . . , yr }. We show that X r ∈ F for all r by induction. For r = 0 this is trivial. Let us assume that X r −1 ∈ F for some r ∈ {1, . . . , s}. If X r −1 ∪ {yr } ∈ F then we immediately have X r ∈ F.

324

13 Matroids

Otherwise X r −1 ∪ {yr } contains a unique circuit C (by Theorem 13.12(b)). Since C(X, yr ) ⊆ X r −1 ∪ {yr } (by (b)), we must have C = C(X, yr ). But then by (a) xr ∈ C(X, yr ) = C, so X r = (X r −1 ∪ {yr }) \ {xr } ∈ F. The idea behind E DMONDS ’ M ATROID I NTERSECTION A LGORITHM is the following. Starting with X = ∅, we augment X by one element in each iteration. Since in general we cannot hope for an element e such that X ∪ {e} ∈ F1 ∩ F2 , we shall look for “alternating paths”. To make this convenient, we deﬁne an auxiliary graph. We apply the notion C(X, e) to (E, Fi ) and write Ci (X, e) (i = 1, 2). E\X

X

SX A(2) X

A(1) X TX Fig. 13.2.

Given a set X ∈ F1 ∩ F2 , we deﬁne a directed auxiliary graph G X by AX

(1)

:= { (x, y) : y ∈ E \ X, x ∈ C1 (X, y) \ {y} },

(2) AX

:= { (y, x) : y ∈ E \ X, x ∈ C2 (X, y) \ {y} },

GX

:= (E, A X ∪ A X ).

(1)

(2)

We set SX

:= {y ∈ E \ X : X ∪ {y} ∈ F1 },

TX

:= {y ∈ E \ X : X ∪ {y} ∈ F2 }

(see Figure 13.2) and look for a shortest path from S X to TX . Such a path will enable us to augment the set X . (If S X ∩ TX = ∅, we have a path of length zero and we can augment X by any element in S X ∩ TX .)

13.5 Matroid Intersection

325

Lemma 13.28. Let X ∈ F1 ∩ F2 . Let y0 , x1 , y1 , . . . , xs , ys be the vertices of a shortest y0 -ys -path in G X (in this order), with y0 ∈ S X and ys ∈ TX . Then X 0 := (X ∪ {y0 , . . . , ys }) \ {x1 , . . . , xs } ∈ F1 ∩ F2 . Proof: First we show that X ∪ {y0 }, x1 , . . . , xs and y1 , . . . , ys satisfy the requirements of Lemma 13.27 with respect to F1 . Observe that X ∪ {y0 } ∈ F1 because (1) y0 ∈ S X . (a) is satisfied because (x j , y j ) ∈ A X for all j, and (b) is satisfied because otherwise the path could be shortcut. We conclude that X 0 ∈ F1 . Secondly, we show that X ∪ {ys }, xs , xs−1 , . . . , x1 and ys−1 , . . . , y1 , y0 satisfy the requirements of Lemma 13.27 with respect to F2 . Observe that X ∪ {ys } ∈ F2 (2) because ys ∈ TX . (a) is satisfied because (y j−1 , x j ) ∈ A X for all j, and (b) is satisfied because otherwise the path could be shortcut. We conclude that X 0 ∈ F2 . We shall now prove that if there exists no S X -TX -path in G X , then X is already maximum. We need the following simple fact: Proposition 13.29. Let (E, F1 ) and (E, F2 ) be two matroids with rank functions r1 and r2 . Then for any F ∈ F1 ∩ F2 and any Q ⊆ E we have |F| ≤ r1 (Q) + r2 (E \ Q). Proof: F ∩ Q ∈ F1 implies |F ∩ Q| ≤ r1 (Q). Similarly F \ Q ∈ F2 implies |F \ Q| ≤ r2 (E \ Q). Adding the two inequalities completes the proof. Lemma 13.30. X ∈ F1 ∩ F2 is maximum if and only if there is no S X -TX -path in G X . Proof: If there is an S X -TX -path, there is also a shortest one. We apply Lemma 13.28 and obtain a set X 0 ∈ F1 ∩ F2 of greater cardinality. Otherwise let R be the set of vertices reachable from S X in G X (see Figure 13.3). We have R ∩ TX = ∅. Let r1 and r2 be the rank function of F1 and F2 , respectively. We claim that r2 (R) = |X ∩ R|. If not, there would be a y ∈ R \ X with / F2 (because y ∈ / TX ), the circuit C2 (X, y) (X ∩ R) ∪ {y} ∈ F2 . Since X ∪ {y} ∈ (2) must contain an element x ∈ X \ R. But then (y, x) ∈ A X means that there is an edge leaving R. This contradicts the definition of R. Next we prove that r1 (E \ R) = |X \ R|. If not, there would be a y ∈ (E \ R)\ X / F1 (because y ∈ / S X ), the circuit C1 (X, y) with (X \ R)∪{y} ∈ F1 . Since X ∪{y} ∈ (1) must contain an element x ∈ X ∩ R. But then (x, y) ∈ A X means that there is an edge leaving R. This contradicts the definition of R. Altogether we have |X | = r2 (R)+r1 (E \ R). By Proposition 13.29, this implies optimality. The last paragraph of this proof yields the following min-max-equality:

326

13 Matroids E\X

X

SX A(2) X

R E\R A(1) X TX Fig. 13.3.

Theorem 13.31. (Edmonds [1970]) Let (E, F1 ) and (E, F2 ) be two matroids with rank functions r1 and r2 . Then max {|X | : X ∈ F1 ∩ F2 } = min {r1 (Q) + r2 (E \ Q) : Q ⊆ E} . We are now ready for a detailed description of the algorithm.

E DMONDS ’ M ATROID I NTERSECTION A LGORITHM Input:

Two matroids (E, F1 ) and (E, F2 ), given by independence oracles.

Output:

A set X ∈ F1 ∩ F2 of maximum cardinality.

1

Set X := ∅.

2

For each y ∈ E \ X and i ∈ {1, 2} do: Compute / Fi , (X ∪ {y}) \ {x} ∈ Fi }. Ci (X, y) := {x ∈ X ∪ {y} : X ∪ {y} ∈ Compute S X , TX , and G X as deﬁned above.

3 4 5

Apply BFS to ﬁnd a shortest S X -TX -path P in G X . If none exists then stop. Set X := X V (P) and go to . 2

Theorem 13.32. E DMONDS ’ M ATROID I NTERSECTION A LGORITHM correctly solves the M ATROID I NTERSECTION P ROBLEM in O(|E|3 θ ) time, where θ is the maximum complexity of the two independence oracles.

13.6 Matroid Partitioning

327

Proof: The correctness follows from Lemmata 13.28 and 13.30. 2 and 3 can be done in O(|E|2 θ ), 4 in O(|E|) time. Since there are at most |E| augmentations, the overall complexity is O(|E|3 θ). Faster matroid intersection algorithms are discussed by Cunningham [1986] and Gabow and Xu [1996]. We remark that the problem of finding a maximum cardinality set in the intersection of three matroids is an NP-hard problem; see Exercise 14(c) of Chapter 15.

13.6 Matroid Partitioning Instead of the intersection of matroids we now consider their union which is defined as follows: Definition 13.33. Let (E, F1 ), . . . , (E, Fk ) be k matroids. . A set X ⊆ E is called . partitionable if there exists a partition X = X 1 ∪ · · · ∪ X k with X i ∈ Fi for i = 1, . . . , k. Let F be the family of partitionable subsets of E. Then (E, F) is called the union or sum of (E, F1 ), . . . , (E, Fk ). We shall prove that the union of matroids is a matroid again. Moreover, we solve the following problem via matroid intersection:

M ATROID PARTITIONING P ROBLEM Instance: Task:

A number k ∈ N, k matroids (E, F1 ), . . . , (E, Fk ), given by independence oracles. Find a partitionable set X ⊆ E of maximum cardinality.

The main theorem with respect to matroid partitioning is: Theorem 13.34. (Nash-Williams [1967]) Let (E, F1 ), . . . , (E, Fk ) be matroids F) is a mawith rank functions r1 , . . . , rk , and let (E, F) be their union. Then (E, Pk ri (A) . troid, and its rank function r is given by r (X ) = min A⊆X |X \ A| + i=1 Proof: (E, F) is an independence system. Let X ⊆ E. We first prove obviouslyP k ri (A) . r (X ) = min A⊆X |X \ A| + i=1 .

.

For any Y ⊆ X such that Y is partitionable, i.e. Y = Y1 ∪ · · · ∪ Yk with Yi ∈ Fi (i = 1, . . . , k), and any A ⊆ X we have |Y | = |Y \ A| + |Y ∩ A| ≤ |X \ A| +

k X i=1

|Yi ∩ A| ≤ |X \ A| +

k X

ri (A),

i=1

Pk ri (A) . so r (X ) ≤ min A⊆X |X \ A| + i=1 On the other hand, let X 0 := X × {1, . . . , k}. We define two matroids on X 0 . For Q ⊆ X 0 and i ∈ {1, . . . , k} we write Q i := {e ∈ X : (e, i) ∈ Q}. Let

328

13 Matroids

I1 := {Q ⊆ X 0 : Q i ∈ Fi for all i = 1, . . . , k} and I2 := {Q ⊆ X 0 : Q i ∩ Q j = ∅ for all i 6= j}. Evidently, both (X 0 , I1 ) and (X 0 , I2 ) are matroids, S and their rank functions are Pk k Q i for Q ⊆ X 0 . given by s1 (Q) := i=1 ri (Q i ) and s2 (Q) := i=1 Now the family of partitionable subsets of X can be written as {A ⊆ X : there is a function f : A → {1, . . . , k} with {(e, f (e)) : e ∈ A} ∈ I1 ∩ I2 }. So the maximum cardinality of a partitionable set is the maximum cardinality of a common independent 13.31 this maximum set in I1 and I2 . By Theorem cardinality equals min s1 (Q) + s2 (X 0 \ Q) : Q ⊆ X 0 . If Q ⊆ X 0 attains this minimum, then for A := Q 1 ∩ · · · ∩ Q k we have k k k X \ X 0 r (X ) = s1 (Q) + s2 (X \ Q) = ri (Q i ) + X \ Qi ≥ ri (A) + |X \ A|. i=1

i=1

i=1

Pk So we have found a set A ⊆ X with i=1 ri (A) + |X \ A| ≤ r (X ). Having proved the formula for the rank function r , we finally show that r is submodular. By Theorem 13.10, this implies that (E, F) is a matroid. To show the Pk let A ⊆ X , B ⊆ Y with r (X ) = |X \ A| + Pk submodularity, let X, Y ⊆ E, and i=1 ri (A) and r (Y ) = |Y \ B| + i=1 ri (B). Then r (X ) + r (Y ) = |X \ A| + |Y \ B| +

k X

(ri (A) + ri (B))

i=1

≥ |(X ∪ Y ) \ (A ∪ B)| + |(X ∩ Y ) \ (A ∩ B)| +

k X

(ri (A ∪ B) + ri (A ∩ B))

i=1

≥ r (X ∪ Y ) + r (X ∩ Y ). The construction in the above proof (Edmonds [1970]) reduces the M ATROID PARTITIONING P ROBLEM to the M ATROID I NTERSECTION P ROBLEM. A reduction in the other direction is also possible (Exercise 20), so both problems can be regarded as equivalent. Note that we can find a maximum independent set in the union of an arbitrary number of matroids efficiently, while the intersection of more than two matroids is intractable.

13.7 Weighted Matroid Intersection

329

13.7 Weighted Matroid Intersection We now consider a generalization of the above algorithm to the weighted case.

W EIGHTED M ATROID I NTERSECTION P ROBLEM Instance: Task:

Two matroids (E, F1 ) and (E, F2 ), given by independence oracles. Weights c : E → R. Find a set X ∈ F1 ∩ F2 whose weight c(X ) is maximum.

We shall describe a primal-dual algorithm due to Frank [1981] for this problem. It generalizes E DMONDS ’ M ATROID I NTERSECTION A LGORITHM. Again we start with X := X 0 = ∅ and increase the cardinality in each iteration by one. We obtain sets X 0 , . . . , X m ∈ F1 ∩ F2 with |X k | = k (k = 0, . . . , m) and m = max{|X | : X ∈ F1 ∩ F2 }. Each X k will be optimum, i.e. c(X k ) = max{c(X ) : X ∈ F1 ∩ F2 , |X | = k}.

(13.4)

Hence at the end we just choose the optimum set among X 0 , . . . , X m . The main idea is to split up the weight function. At any stage we have two functions c1 , c2 : E → R with c1 (e) + c2 (e) = c(e) for all e ∈ E. For each k we shall guarantee ci (X k ) = max{ci (X ) : X ∈ Fi , |X | = k}

(i = 1, 2).

(13.5)

This condition obviously implies (13.4). To obtain (13.5) we use the optimality criterion of Theorem 13.23. Instead of G X , S X and TX only a subgraph G¯ and ¯ T¯ are considered. subsets S,

W EIGHTED M ATROID I NTERSECTION A LGORITHM Input: Output:

Two matroids (E, F1 ) and (E, F2 ), given by independence oracles. Weights c : E → R. A set X ∈ F1 ∩ F2 of maximum weight.

1

Set k := 0 and X 0 := ∅. Set c1 (e) := c(e) and c2 (e) := 0 for all e ∈ E.

2

For each y ∈ E \ X k and i ∈ {1, 2} do: Compute / Fi , (X k ∪ {y}) \ {x} ∈ Fi }. Ci (X k , y) := {x ∈ X k ∪ {y} : X k ∪ {y} ∈ Compute

3

A(1) := { (x, y) : y ∈ E \ X k , x ∈ C1 (X k , y) \ {y} }, A(2) := { (y, x) : y ∈ E \ X k , x ∈ C2 (X k , y) \ {y} }, S := { y ∈ E \ X k : X k ∪ {y} ∈ F1 }, T

:= { y ∈ E \ X k : X k ∪ {y} ∈ F2 }.

330

4

13 Matroids

Compute m 1 := max{c1 (y) : y ∈ S} m 2 := max{c2 (y) : y ∈ T } S¯ := { y ∈ S : c1 (y) = m 1 } T¯ := { y ∈ T : c2 (y) = m 2 } A¯ (1) := { (x, y) ∈ A(1) : c1 (x) = c1 (y) }, A¯ (2) := { (y, x) ∈ A(2) : c2 (x) = c2 (y) }, G¯ := (E, A¯ (1) ∪ A¯ (2) ).

5

6

7

¯ Apply BFS to compute the set R of vertices reachable from S¯ in G. ¯ T¯ -path P in G¯ with a minimum number If R ∩ T¯ 6= ∅ then: Find an Sof edges, set X k+1 := X k 4V (P) and k := k + 1 and go to . 2 Compute ε1 := min{c1 (x) − c1 (y) : (x, y) ∈ A(1) ∩ δ + (R)}; ε2 := min{c2 (x) − c2 (y) : (y, x) ∈ A(2) ∩ δ + (R)}; ε3 := min{m 1 − c1 (y) : y ∈ S \ R}; ε4 := min{m 2 − c2 (y) : y ∈ T ∩ R}; ε := min{ε1 , ε2 , ε3 , ε4 }

8

(where min ∅ := ∞). If ε < ∞ then: Set c1 (x) := c1 (x) − ε and c2 (x) := c2 (x) + ε for all x ∈ R. Go to . 4 If ε = ∞ then: Among X 0 , X 1 , . . . , X k , let X be the one with maximum weight. Stop. See Edmonds [1979] and Lawler [1976] for earlier versions of this algorithm.

Theorem 13.35. (Frank [1981]) The W EIGHTED M ATROID I NTERSECTION A LGORITHM correctly solves the W EIGHTED M ATROID I NTERSECTION P ROB LEM in O(|E|4 + |E|3 θ ) time, where θ is the maximum complexity of the two independence oracles. Proof: Let m be the final value of k. The algorithm computes sets X 0 , X 1 , . . . , X m . We first prove that X k ∈ F1 ∩ F2 for k = 0, . . . , m, by induction on k. This is trivial for k = 0. If we are working with X k ∈ F1 ∩ F2 for some k, G¯ is a subgraph of (E, A(1) ∪ A(2) ) = G X k . So if a path P is found in , 5 Lemma 13.28 ensures that X k+1 ∈ F1 ∩ F2 . When the algorithm stops, we have ε1 = ε2 = ε3 = ε4 = ∞, so T is not reachable from S in G X m . Then by Lemma 13.30 m = |X m | = max{|X | : X ∈ F1 ∩ F2 }.

13.7 Weighted Matroid Intersection

331

To prove correctness, we show that for k = 0, . . . , m, c(X k ) = max{c(X ) : X ∈ F1 ∩ F2 , |X | = k}. Since we always have c = c1 + c2 , it suffices to prove that at any stage of the algorithm (13.5) holds. This is clearly true when the algorithm starts (for k = 0); we show that (13.5) is never violated. We use Theorem 13.23. When we set X k+1 := X k 4V (P) in 6 we have to check that (13.5) holds. Let P ¯ t ∈ T¯ . By definition of G¯ we have c1 (X k+1 ) = c1 (X k )+c1 (s) be an s-t-path, s ∈ S, and c2 (X k+1 ) = c2 (X k ) + c2 (t). Since X k satisfies (13.5), conditions (a) and (b) of Theorem 13.23 must hold with respect to X k and each of F1 and F2 . By definition of S¯ both conditions continue to hold for X k ∪{s} and F1 . Therefore c1 (X k+1 ) = c1 (X k ∪ {s}) = max{c1 (Y ) : Y ∈ F1 , |Y | = k + 1}. Moreover, by definition of T¯ , (a) and (b) of Theorem 13.23 continue to hold for X k ∪ {t} and F2 , implying c2 (X k+1 ) = c2 (X k ∪ {t}) = max{c2 (Y ) : Y ∈ F2 , |Y | = k + 1}. In other words, (13.5) indeed holds for X k+1 . Now suppose we change c1 and c2 in . 8 We first show that ε > 0. By (13.5) and Theorem 13.23 we have c1 (x) ≥ c1 (y) for all y ∈ E \ X k and x ∈ C1 (X k , y) \ {y}. So for any (x, y) ∈ A(1) we have c1 (x) ≥ c1 (y). Moreover, by the definition of R no edge (x, y) ∈ δ + (R) belongs to A¯ (1) . This implies ε1 > 0. ε2 > 0 is proved analogously. m 1 ≥ c1 (y) holds for all y ∈ S. If in addition ¯ so m 1 > c1 (y). Therefore ε3 > 0. Similarly, ε4 > 0 (using y ∈ / R then y ∈ / S, T¯ ∩ R = ∅). We conclude that ε > 0. We can now prove that 8 preserves (13.5). Let c10 be the modified c1 , i.e. ( c1 (x) − ε if x ∈ R 0 . c1 (x) := if x ∈ / R c1 (x) We prove that X k and c10 satisfy the conditions of Theorem 13.23 with respect to F1 . To prove (a), let y ∈ E \ X k and x ∈ C1 (X k , y) \ {y}. Suppose c10 (x) < c10 (y). / R. Since also Since c1 (x) ≥ c1 (y) and ε > 0, we must have x ∈ R and y ∈ (x, y) ∈ A(1) , we have ε ≤ ε1 ≤ c1 (x)−c1 (y) = (c10 (x)+ε)−c10 (y), a contradiction. To prove (b), let x ∈ X k and y ∈ E \ X k with X k ∪ {y} ∈ F1 . Now suppose / R. Since c10 (y) > c10 (x). Since c1 (y) ≤ m 1 ≤ c1 (x), we must have x ∈ R and y ∈ y ∈ S we have ε ≤ ε3 ≤ m 1 − c1 (y) ≤ c1 (x) − c1 (y) = (c10 (x) + ε) − c10 (y), a contradiction. Let c20 be the modified c2 , i.e. ( c2 (x) + ε if x ∈ R . c20 (x) := if x ∈ / R c2 (x) We show that X k and c20 satisfy the conditions of Theorem 13.23 with respect to F2 . To prove (a), let y ∈ E \ X k and x ∈ C2 (X k , y) \ {y}. Suppose c20 (x) < c20 (y). Since c2 (x) ≥ c2 (y), we must have y ∈ R and x ∈ / R. Since also (y, x) ∈ A(2) , 0 0 we have ε ≤ ε2 ≤ c2 (x) − c2 (y) = c2 (x) − (c2 (y) − ε), a contradiction.

332

13 Matroids

To prove (b), let x ∈ X k and y ∈ E \ X k with X k ∪ {y} ∈ F2 . Now suppose c20 (y) > c20 (x). Since c2 (y) ≤ m 2 ≤ c2 (x), we must have y ∈ R and x ∈ / R. Since y ∈ T we have ε ≤ ε4 ≤ m 2 − c2 (y) ≤ c2 (x) − c2 (y) = c20 (x) − (c20 (y) − ε), a contradiction. So we have proved that (13.5) is not violated during , 8 and thus the algorithm works correctly. We now consider the running time. Observe that after an update of the weights ¯ T¯ , and R, as computed subsequently in in , 8 the new sets S, 4 and , 5 are ¯ T¯ , and R, respectively. If ε = ε4 < ∞, an augmentation supersets of the old S, (increase of k) follows. Otherwise the cardinality of R increases immediately (in

) 5 by at least one. So 4 – 8 are repeated less than |E| times between two augmentations. Since the running time of 4 – 8 is O(|E|2 ), the total running time between 3 two augmentations is O(|E| ) plus O(|E|2 ) oracle calls (in ). 2 Since there are m ≤ |E| augmentations, the stated overall running time follows. The running time can easily be improved to O(|E|3 θ) (Exercise 22).

Exercises 1. Prove that all the independence systems apart from (5) and (6) in the list at the beginning of Section 13.1 are – in general – not matroids. 2. Show that the uniform matroid with four elements and rank 2 is not a graphic matroid. 3. Prove that every graphic matroid is representable over every field. 4. Let G be an undirected graph, K ∈ N, and let F contain those subsets of E(G) that are the union of K forests. Prove that (E(G), F) is a matroid. 5. Compute tight lower bounds for the rank quotients of the independence systems listed at the beginning of Section 13.1. 6. Let S be a family of sets. A set T is a transversal of S if there is a bijection 8 : T → S with t ∈ 8(t) for all t ∈ T . (For a necessary and sufficient condition for the existence of a transversal, see Exercise 6 of Chapter 10.) Assume that S has a transversal. Prove that the family of transversals of S is the family of bases of a matroid. 7. Let E be a finite set and B ⊆ 2 E . Show that B is the set of bases of some matroid (E, F) if and only if the following holds: (B1) B 6= ∅; (B2) For any B1 , B2 ∈ B and y ∈ B2 \ B1 there exists an x ∈ B1 \ B2 with (B1 \ {x}) ∪ {y} ∈ B. 8. Let G be a graph. Let F be the family of sets X ⊆ V (G), for which a maximum matching exists that covers no vertex in X . Prove that (V (G), F) is a matroid. What is the dual matroid?

Exercises

333

9. Show that M(G ∗ ) = (M(G))∗ also holds for disconnected graphs G, extending Theorem 13.16. Hint: Use Exercise 31(a) of Chapter 2. 10. Show that the clutters in (3) and (6) in the list of Section 13.3 have the MaxFlow-Min-Cut property. (Use Theorem 19.10.) Show that the clutters in (1), (4) and (5) do not have the Max-Flow-Min-Cut property in general. ∗ 11. A clutter (E, F) is called binary if for all X 1 , . . . , X k ∈ F with k odd there exists a Y ∈ F with Y ⊆ X 1 4 · · · 4X k . Prove that the clutter of minimal T -joins and the clutter of minimal T -cuts (example (7) of the list in Section 13.3) are binary. Prove that a clutter is binary if and only if |A ∩ B| is odd for all A ∈ F and all B ∈ F 0 , where (E, F 0 ) is the blocking clutter. Conclude that a clutter is binary if and only if its blocking clutter is binary. Note: Seymour [1977] classified the binary clutters with the Max-Flow-MinCut property. ∗ 12. Let P be a polyhedron of blocking type, i.e. we have x + y ∈ P for all x ∈ P and y ≥ 0. The blocking polyhedron of P is defined to be B(P) := {z : z > x ≥ 1 for all x ∈ P}. Prove that B(P) is again a polyhedron of blocking type and that B(B(P)) = P. Note: Compare this with Theorem 4.22. 13. How can one check (in polynomial time) whether a given set of edges of a complete graph G is a subset of some Hamiltonian circuit in G? 14. Prove that if (E, F) is a matroid, then the B EST-I N -G REEDY maximizes any bottleneck function c(F) = min{ce : e ∈ F} over the bases. 15. Let (E, F) be a matroid and c : E → R such that c(e) 6= c(e0 ) for all e 6= e0 and c(e) 6= 0 for all e. Prove that both the M AXIMIZATION and the M INIMIZATION P ROBLEM for (E, F, c) have a unique optimum solution. ∗ 16. Prove that for matroids the independence, basis-superset, closure and rank oracles are polynomially equivalent. Hint: To show that the rank oracle reduces to the independence oracle, use the B EST-I N -G REEDY. To show that the independence oracle reduces to the basis-superset oracle, use the W ORST-O UT-G REEDY. (Hausmann and Korte [1981]) 17. Given an undirected graph G, we wish to colour the edges with a minimum number of colours such that for any circuit C of G, the edges of C do not all have the same colour. Show that there is a polynomial-time algorithm for this problem. 18. Let (E, F1 ), . . . , (E, Fk ) be matroids with rank functionsPr1 , . . . , rk . Prove k ri (A) for all that a set X ⊆ E is partitionable if and only if |A| ≤ i=1 A ⊆ X . Show that Theorem 6.19 is a special case. (Edmonds and Fulkerson [1965]) 19. Let (E, F) be a matroid with rank function r . Prove (using Theorem 13.34): (a) (E, F) has k pairwise disjoint bases if and only if kr (A)+|E \ A| ≥ kr (E) for all A ⊆ E.

334

20.

21.

22. 23.

13 Matroids

(b) (E, F) has k independent sets whose union is E if and only if kr (A) ≥ |A| for all A ⊆ E. Show that Theorem 6.19 and Theorem 6.16 are special cases. Let (E, F1 ) and (E, F2 ) be two matroids. Let X be a maximal partitionable . subset with respect to (E, F1 ) and (E, F2∗ ): X = X 1 ∪ X 2 with X 1 ∈ F1 and X 2 ∈ F2∗ . Let B2 ⊇ X 2 be a basis of F2∗ . Prove that then X \ B2 is a maximum-cardinality set in F1 ∩ F2 . (Edmonds [1970]) Let (E, S) be a set system, and let (E, F) be a matroid with rank function r. S Show that S has a transversal that is independent in (E, F) if and only if r B∈B B ≥ |B| for all B ⊆ S. Hint: First describe the rank function of the matroid whose independent sets are all transversals (Exercise 6), using Theorem 13.34. Then apply Theorem 13.31. (Rado [1942]) Show that the running time of the W EIGHTED M ATROID I NTERSECTION A LGORITHM (cf. Theorem 13.35) can be improved to O(|E|3 θ ). Let (E, F1 ) and (E, F2 ) be two matroids, and c : E → R. Let X 0 , . . . , X m ∈ F1 ∩ F2 with |X k | = k and c(X k ) = max{c(X ) : X ∈ F1 ∩ F2 , |X | = k} for all k. Prove that for k = 1, . . . , m − 2 c(X k+1 ) − c(X k ) ≤ c(X k ) − c(X k−1 ).

(Krogdahl [unpublished]) 24. Consider the following problem. Given a digraph G with edge weights, a vertex s ∈ V (G), and a number K , find a minimum weight subgraph H of G containing K edge-disjoint paths from s to each other vertex. Show that this reduces to the W EIGHTED M ATROID I NTERSECTION P ROBLEM. Hint: See Exercise 19 of Chapter 6 and Exercise 4 of this chapter. (Edmonds [1970]; Frank and Tardos [1989]; Gabow [1995]) . 25. Let A and B be two finite sets of cardinality n ∈ N, G = (A ∪ B, {{a, b} : a ∈ A, b ∈ B}) the complete bipartite graph, a¯ ∈ A, and c : E(G) → R a cost function. Let T be the family of edge sets of all spanning trees T in G with |δT (a)| = 2 for all a ∈ A \ {a}. ¯ Show that a minimum cost element of ¯ T can be computed in O(n 7 ) time. How many edges will be incident to a?

References General Literature: Bixby, R.E., and Cunningham, W.H. [1995]: Matroid optimization and algorithms. In: Handbook of Combinatorics; Vol. 1 (R.L. Graham, M. Gr¨otschel, L. Lov´asz, eds.), Elsevier, Amsterdam, 1995 Cook, W.J., Cunningham, W.H., Pulleyblank, W.R., and Schrijver, A. [1998]: Combinatorial Optimization. Wiley, New York 1998, Chapter 8

References

335

Faigle, U. [1987]: Matroids in combinatorial optimization. In: Combinatorial Geometries (N. White, ed.), Cambridge University Press, 1987 Gondran, M., and Minoux, M. [1984]: Graphs and Algorithms. Wiley, Chichester 1984, Chapter 9 Lawler, E.L. [1976]: Combinatorial Optimization; Networks and Matroids. Holt, Rinehart and Winston, New York 1976, Chapters 7 and 8 Oxley, J.G. [1992]: Matroid Theory. Oxford University Press, Oxford 1992 von Randow, R. [1975]: Introduction to the Theory of Matroids. Springer, Berlin 1975 Recski, A. [1989]: Matroid Theory and its Applications. Springer, Berlin, 1989 Schrijver, A. [2003]: Combinatorial Optimization: Polyhedra and Efficiency. Springer, Berlin 2003, Chapters 39–42 Welsh, D.J.A. [1976]: Matroid Theory. Academic Press, London 1976

Cited References: Cunningham, W.H. [1986] : Improved bounds for matroid partition and intersection algorithms. SIAM Journal on Computing 15 (1986), 948–957 Edmonds, J. [1970]: Submodular functions, matroids and certain polyhedra. In: Combinatorial Structures and Their Applications; Proceedings of the Calgary International Conference on Combinatorial Structures and Their Applications 1969 (R. Guy, H. Hanani, N. Sauer, J. Schonheim, eds.), Gordon and Breach, New York 1970, pp. 69–87 Edmonds, J. [1971]: Matroids and the greedy algorithm. Mathematical Programming 1 (1971), 127–136 Edmonds, J. [1979]: Matroid intersection. In: Discrete Optimization I; Annals of Discrete Mathematics 4 (P.L. Hammer, E.L. Johnson, B.H. Korte, eds.), North-Holland, Amsterdam 1979, pp. 39–49 Edmonds, J., and Fulkerson, D.R. [1965]: Transversals and matroid partition. Journal of Research of the National Bureau of Standards B 69 (1965), 67–72 Frank, A. [1981]: A weighted matroid intersection algorithm. Journal of Algorithms 2 (1981), 328–336 ´ [1989]: An application of submodular flows. Linear Algebra and Frank, A., and Tardos, E. Its Applications 114/115 (1989), 329–348 Fulkerson, D.R. [1971]: Blocking and anti-blocking pairs of polyhedra. Mathematical Programming 1 (1971), 168–194 Gabow, H.N. [1995]: A matroid approach to finding edge connectivity and packing arborescences. Journal of Computer and System Sciences 50 (1995), 259–273 Gabow, H.N., and Xu, Y. [1996]: Efficient theoretic and practical algorithms for linear matroid intersection problems. Journal of Computer and System Sciences 53 (1996), 129–147 Hausmann, D., Jenkyns, T.A., and Korte, B. [1980]: Worst case analysis of greedy type algorithms for independence systems. Mathematical Programming Study 12 (1980), 120–131 Hausmann, D., and Korte, B. [1981]: Algorithmic versus axiomatic definitions of matroids. Mathematical Programming Study 14 (1981), 98–111 Jenkyns, T.A. [1976]: The efficiency of the greedy algorithm. Proceedings of the 7th S-E Conference on Combinatorics, Graph Theory, and Computing, Utilitas Mathematica, Winnipeg 1976, pp. 341–350 Korte, B., and Hausmann, D. [1978]: An analysis of the greedy algorithm for independence systems. In: Algorithmic Aspects of Combinatorics; Annals of Discrete Mathematics 2 (B. Alspach, P. Hell, D.J. Miller, eds.), North-Holland, Amsterdam 1978, pp. 65–74

336

13 Matroids

Korte, B., and Monma, C.L. [1979]: Some remarks on a classification of oracle-type algorithms. In: Numerische Methoden bei graphentheoretischen und kombinatorischen Problemen; Band 2 (L. Collatz, G. Meinardus, W. Wetterling, eds.), Birkh¨auser, Basel 1979, pp. 195–215 Lehman, A. [1979]: On the width-length inequality. Mathematical Programming 17 (1979), 403–417 Nash-Williams, C.S.J.A. [1967]: An application of matroids to graph theory. In: Theory of Graphs; Proceedings of an International Symposium in Rome 1966 (P. Rosenstiehl, ed.), Gordon and Breach, New York, 1967, pp. 263–265 Rado, R. [1942]: A theorem on independence relations. Quarterly Journal of Math. Oxford 13 (1942), 83–89 Rado, R. [1957]: Note on independence functions. Proceedings of the London Mathematical Society 7 (1957), 300–320 Seymour, P.D. [1977]: The matroids with the Max-Flow Min-Cut property. Journal of Combinatorial Theory B 23 (1977), 189–222 Whitney, H. [1933]: Planar graphs. Fundamenta Mathematicae 21 (1933), 73–84 Whitney, H. [1935]: On the abstract properties of linear dependence. American Journal of Mathematics 57 (1935), 509–533

14 Generalizations of Matroids

There are several interesting generalizations of matroids. We have already seen independence systems in Section 13.1, which arose from dropping the axiom (M3). In Section 14.1 we consider greedoids, arising by dropping (M2) instead. Moreover, certain polytopes related to matroids and to submodular functions, called polymatroids, lead to strong generalizations of important theorems; we shall discuss them in Section 14.2. In Sections 14.3 and 14.4 we consider two approaches to the problem of minimizing an arbitrary submodular function: one using the E LLIPSOID M ETHOD, and one with a combinatorial algorithm. For the important special case of symmetric submodular functions we mention a simpler algorithm in Section 14.5.

14.1 Greedoids By definition, set systems (E, F) are matroids if and only if they satisfy (M1) ∅ ∈ F; (M2) If X ⊆ Y ∈ F then X ∈ F; (M3) If X, Y ∈ F and |X | > |Y |, then there is an x ∈ X \ Y with Y ∪ {x} ∈ F. If we drop (M3), we obtain independence systems, discussed in Sections 13.1 and 13.4. Now we drop (M2) instead: Definition 14.1. A greedoid is a set system (E, F) satisfying (M1) and (M3). Instead of the subclusiveness (M2) we have accessibility: We call a set system (E, F) accessible if ∅ ∈ F and for any X ∈ F \ {∅} there exists an x ∈ X with X \ {x} ∈ F. Greedoids are accessible (accessibility follows directly from (M1) and (M3)). Though more general than matroids, they comprise a rich structure and, on the other hand, generalize many different, seemingly unrelated concepts. We start with the following result: Theorem 14.2. Let (E, F) be an accessible set system. The following statements are equivalent: (a) For any X ⊆ Y ⊂ E and z ∈ E \ Y with X ∪ {z} ∈ F and Y ∈ F we have Y ∪ {z} ∈ F;

338

14 Generalizations of Matroids

(b) F is closed under union. Proof: (a) ⇒(b): Let X, Y ∈ F; we show that X ∪ Y ∈ F. Let Z be a maximal set with Z ∈ F and X ⊆ Z ⊆ X ∪ Y . Suppose Y \ Z 6= ∅. By repeatedly applying accessibility to Y we get a set Y 0 ∈ F with Y 0 ⊆ Z and an element y ∈ Y \ Z with Y 0 ∪ {y} ∈ F. We apply (a) to Z , Y 0 and y and obtain Z ∪ {y} ∈ F, contradicting the choice of Z . (b) ⇒(a) is trivial. If the conditions in Theorem 14.2 hold, then (E, F) is called an antimatroid. Proposition 14.3. Every antimatroid is a greedoid. Proof: Let (E, F) be an antimatroid, i.e. accessible and closed under union. To prove (M3), let X, Y ∈ F with |X | > |Y |. Since (E, F) is accessible there is an order X = {x1 , . . . , xn } with {x1 , . . . , xi } ∈ F for i = 0, . . . , n. Let i ∈ {1, . . . , n} / Y ; then Y ∪ {xi } = Y ∪ {x1 , . . . , xi } ∈ F (since be the minimum index with xi ∈ F is closed under union). Another equivalent definition of antimatroids is by a closure operator: Proposition 14.4. Let (E, F) be a set system such that F is closed under union and ∅ ∈ F. Define \ τ (A) := {X ⊆ E : A ⊆ X, E \ X ∈ F} Then τ is a closure operator, i.e. satisfies (S1)–(S3) of Theorem 13.11. Proof: Let X ⊆ Y ⊆ E. X ⊆ τ (X ) ⊆ τ (Y ) is trivial. To prove (S3), assume that there exists a y ∈ τ (τ (X )) \ τ (X ). Then y ∈ Y for all Y ⊆ E with τ (X ) ⊆ Y and E \ Y ∈ F, but there exists a Z ⊆ E \ {y} with X ⊆ Z and E \ Z ∈ F. This implies τ (X ) 6⊆ Z , a contradiction. Theorem 14.5. Let (E, F) be a set system such that F is closed under union and ∅ ∈ F. Then (E, F) is accessible if and only if the closure operator τ of Proposition 14.4 satisfies the anti-exchange property: if X ⊆ E, y, z ∈ E \ τ (X ), y 6= z and z ∈ τ (X ∪ {y}), then y ∈ / τ (X ∪ {z}). Proof: If (E, F) is accessible, then (M3) holds by Proposition 14.3. To show the anti-exchange property, let X ⊆ E, B := E \ τ (X ), and y, z ∈ B with z∈ / A := E \ τ (X ∪ {y}). Observe that A ∈ F, B ∈ F and A ⊆ B \ {y, z}. By applying (M3) to A and B we get an element b ∈ B \ A ⊆ E \ (X ∪ A) with A ∪ {b} ∈ F. A ∪ {b} cannot be a subset of E \ (X ∪ {y}) (otherwise τ (X ∪ {y}) ⊆ E \ (A ∪ {b}), contradicting τ (X ∪ {y}) = E \ A). Hence b = y. So we have A ∪ {y} ∈ F and thus τ (X ∪ {z}) ⊆ E \ (A ∪ {y}). We have proved y∈ / τ (X ∪ {z}). To show the converse, let A ∈ F \ {∅} and let X := E \ A. We have τ (X ) = X . Let a ∈ A such that |τ (X ∪ {a})| is minimum. We claim that τ (X ∪ {a}) = X ∪ {a}, i.e. A \ {a} ∈ F.

14.1 Greedoids

339

Suppose, on the contrary, that b ∈ τ (X ∪ {a}) \ (X ∪ {a}). By (c) we have a∈ / τ (X ∪ {b}). Moreover, τ (X ∪ {b}) ⊆ τ (τ (X ∪ {a}) ∪ {b}) = τ (τ (X ∪ {a})) = τ (X ∪ {a}). Hence τ (X ∪ {b}) is a proper subset of τ (X ∪ {a}), contradicting the choice of a. The anti-exchange property of Theorem 14.5 is different from (S4). While (S4) of Theorem 13.11 is a property of linear hulls in Rn , this is a property of convex hulls / conv(X ∪{z}). in Rn : if y 6= z, z 6∈ conv(X ) and z ∈ conv(X ∪{y}), then clearly y ∈ So for any finite set E ⊂ Rn , (E, {X ⊆ E : X ∩conv(E \ X ) = ∅}) is an antimatroid. Greedoids generalize matroids and antimatroids, but they also contain other interesting structures. One example is the blossom structure we used in E DMONDS ’ C ARDINALITY M ATCHING A LGORITHM (Exercise 1). Another basic example is: Proposition 14.6. Let G be a graph (directed or undirected) and r ∈ V (G). Let F be the family of all edge sets of arborescences in G rooted at r , or trees in G containing r (not necessarily spanning). Then (E(G), F) is a greedoid. Proof: (M1) is trivial. We prove (M3) for the directed case; the same argument applies to the undirected case. Let (X 1 , F1 ) and (X 2 , F2 ) be two arborescences in G rooted at r with |F1 | > |F2 |. Then |X 1 | = |F1 | + 1 > |F2 | + 1 = |X 2 |, so let x ∈ X 1 \ X 2 . The r -x-path in (X 1 , F1 ) contains an edge (v, w) with v ∈ X 2 and w∈ / X 2 . This edge can be added to (X 2 , F2 ), proving that F2 ∪ {(v, w)} ∈ F. This greedoid is called the directed (undirected) branching greedoid of G. The problem of finding a maximum weight spanning tree in a connected graph G with nonnegative weights is the M AXIMIZATION P ROBLEM for the cycle matroid M(G). The B EST-I N -G REEDY A LGORITHM is in this case nothing but K RUSKAL ’ S A LGORITHM. Now we have a second formulation of the same problem: we are looking for a maximum weight set F with F ∈ F, where (E(G), F) is the undirected branching greedoid of G. We now formulate a general greedy algorithm for greedoids. In the special case of matroids, it is exactly the B EST-I N -G REEDY A LGORITHM discussed in Section 13.4. If we have an undirected branching greedoid with a modular cost function c, it is P RIM ’ S A LGORITHM:

G REEDY A LGORITHM F OR G REEDOIDS Input: Output:

A greedoid (E, F) and a function c : 2 E → R, given by an oracle which for any given X ⊆ E says whether X ∈ F and returns c(X ). A set F ∈ F.

1

Set F := ∅.

2

Let e ∈ E \ F such that F ∪ {e} ∈ F and c(F ∪ {e}) is maximum; if no such e exists then stop.

340

3

14 Generalizations of Matroids

Set F := F ∪ {e} and go to . 2

Even for modular cost functions c this algorithm does not always provide an optimal solution. At least we can characterize those greedoids where it works: Theorem 14.7. Let (E, F) be a greedoid. The G REEDY A LGORITHM F OR G REE DOIDS finds a set F ∈ F of maximum weight for each modular weight function c : 2 E → R+ if and only if (E, F) has the so-called strong exchange property: For all A ∈ F, B maximal in F, A ⊆ B and x ∈ E \ B with A ∪ {x} ∈ F there exists a y ∈ B \ A such that A ∪ {y} ∈ F and (B \ y) ∪ {x} ∈ F. Proof: Suppose (E, F) is a greedoid with the strong exchange property. Let c : E → R+ , and let A = {a1 , . . . , al } be the solution found by the G REEDY A L GORITHM F OR G REEDOIDS . , where the elements are chosen in the order a1 , . . . , al . Let B = {a1 , . . . , ak } ∪ B 0 be an optimum solution such that k is maximum, and suppose that k < l. Then we apply the strong exchange property to {a1 , . . . , ak }, B and ak+1 . We conclude that there exists a y ∈ B 0 with {a1 , . . . , ak , y} ∈ F and (B \ y) ∪ {ak+1 } ∈ F. By the choice of ak+1 in 2 of the G REEDY A LGORITHM F OR G REEDOIDS we have c(ak+1 ) ≥ c(y) and thus c((B \ y) ∪ {ak+1 }) ≥ c(B), contradicting the choice of B. Conversely, let (E, F) be a greedoid that does not have the strong exchange property. Let A ∈ F, B maximal in F, A ⊆ B and x ∈ E \ B with A ∪ {x} ∈ F such that for all y ∈ B \ A with A ∪ {y} ∈ F we have (B \ y) ∪ {x} ∈ / F. Let Y := {y ∈ B \ A : A ∪ {y} ∈ F}. We set c(e) := 2 for e ∈ B \ Y , and c(e) := 1 for e ∈ Y ∪ {x} and c(e) := 0 for e ∈ E \ (B ∪ {x}). Then the G REEDY A LGORITHM F OR G REEDOIDS might choose the elements of A first (they have weight 2) and then might choose x. It will eventually end up with a set F ∈ F that cannot be optimal, since c(F) ≤ c(B ∪ {x}) − 2 < c(B ∪ {x}) − 1 = c(B) and B ∈ F. Indeed, optimizing modular functions over general greedoids is NP-hard. This follows from the following observation (together with Corollary 15.24): Proposition 14.8. The problem of deciding, given an undirected graph G and k ∈ N, whether G has a vertex cover of cardinality k, linearly reduces to the following problem: Given a greedoid (E, F) (by a membership oracle) and a function c : E → R+ , find an F ∈ F with c(F) maximum. .

Proof: Let G be any undirected graph and k ∈ N. Let D := V (G) ∪ E(G) and F := {X ⊆ D : for all e = {v, w} ∈ E(G) ∩ X we have v ∈ X or w ∈ X }. (D, F) is an antimatroid: it is accessible and closed under union. In particular, by Proposition 14.3, it is a greedoid. Now consider F 0 := {X ∈ F : |X | ≤ |E(G)| + k}. Since (M1) and (M3) are preserved, (D, F 0 ) is also a greedoid. Set c(e) := 1 for e ∈ E(G) and c(v) := 0

14.2 Polymatroids

341

for v ∈ V (G). Then there exists a set F ∈ F 0 with c(F) = |E(G)| if and only if G contains a vertex cover of size k. On the other hand, there are interesting functions that can be maximized over arbitrary greedoids, for example bottleneck functions c(F) := min{c0 (e) : e ∈ F} for some c0 : E → R+ (Exercise 2). See the book by Korte, Lov´asz and Schrader [1991] for more results in this area.

14.2 Polymatroids From Theorem 13.10 we know the tight connection between matroids and submodular functions. Submodular functions define the following interesting class of polyhedra: Definition 14.9. A polymatroid is a polytope of type ) ( X xe ≤ f (A) for all A ⊆ E P( f ) := x ∈ R E : x ≥ 0, e∈A

where E is a finite set and f : 2 E → R+ is a submodular function. It is not hard to see that for any polymatroid f can be chosen such that f (∅) = 0 and f is monotone (Exercise 5; a function f : 2 E → R is called monotone if f (X ) ≤ f (Y ) for X ⊆ Y ⊆ E). Edmonds’ original definition was different; see Exercise 6. Moreover, we mention that the term polymatroid is sometimes not used for the polytope but for the pair (E, f ). If f is the rank function of a matroid, P( f ) is the convex hull of the incidence vectors of the independent sets of this matroid (Theorem 13.21). We know that the B EST-I N -G REEDY optimizes any linear function over a matroid polytope. A similar greedy algorithm also works for general polymatroids. We assume that f is monotone:

P OLYMATROID G REEDY A LGORITHM Input: Output:

1

2

A finite set E and a submodular, monotone function f : 2 E → R+ (given by an oracle). A vector c ∈ R E . A vector x ∈ P( f ) with cx maximum.

Sort E = {e1 , . . . , en } such that c(e1 ) ≥ · · · ≥ c(ek ) > 0 ≥ c(ek+1 ) ≥ · · · ≥ c(en ). If k ≥ 1 then set x(e1 ) := f ({e1 }). Set x(ei ) := f ({e1 , . . . , ei }) − f ({e1 , . . . , ei−1 }) for i = 2, . . . , k. Set x(ei ) := 0 for i = k + 1, . . . , n.

342

14 Generalizations of Matroids

Proposition 14.10. Let E = {e1 , . . . , en } and f : 2 E → R be a submodular function with f (∅) ≥ 0. Let b : E → R with b(e1 ) ≤ fP ({e1 }) and b(ei ) ≤ f ({e1 , . . . , ei }) − f ({e1 , . . . , ei−1 }) for i = 2, . . . , n. Then a∈A b(a) ≤ f (A) for all A ⊆ E. Proof: Induction on i = max{ Pj : e j ∈ A}. The P assertion is trivial for A = ∅ and A = {e1 }. If i ≥ 2, then a∈A b(a) = a∈A\{ei } b(a) + b(ei ) ≤ f (A \ {ei }) + b(ei ) ≤ f (A \ {ei }) + f ({e1 , . . . , ei }) − f ({e1 , . . . , ei−1 }) ≤ f (A), where the first inequality follows from the induction hypothesis and the third one from submodularity. Theorem 14.11. The P OLYMATROID G REEDY A LGORITHM correctly finds an x ∈ P( f ) with cx maximum. If f is integral, then x is also integral. Proof: Let x ∈ R E be the output of the P OLYMATROID G REEDY A LGORITHM for E, f and c. By definition, if f is integral, then x is also integral. We have x ≥ 0 since f is monotone, and thus x ∈ P( f ) by Proposition 14.10. E with cy > cx. Similarly as in the proof of Theorem 13.19 Now let y ∈ R+ we set d j := c(e j ) − c(e j+1 ) ( j = 1, . . . , k − 1) and dk := c(ek ), and we have k X j=1

dj

j X

x(ei ) = cx < cy ≤

i=1

k X

c(e j )y(e j ) =

j=1

k X j=1

dj

j X

y(ei ).

i=1

Pj Since d j ≥ 0 for all j there is an index j ∈ {1, . . . , k} with i=1 y(ei ) > Pj Pj / i=1 x(ei ); however, since i=1 x(ei ) = f ({e1 , . . . , e j }) this means that y ∈ P( f ). As with matroids, we can also handle the intersection of two polymatroids. The following polymatroid intersection theorem has many implications: Theorem 14.12. (Edmonds [1970,1979]) Let E be a finite set, and let f, g : 2 E → R+ be submodular functions. Then the system X e∈A X

x ≥ 0 xe ≤ f (A)

(A ⊆ E)

xe ≤ g(A)

(A ⊆ E)

e∈A

is TDI. Proof: Consider the primal-dual pair of LPs ( ) X X max cx : x ≥ 0, xe ≤ f (A) and xe ≤ g(A) for all A ⊆ E e∈A

e∈A

14.2 Polymatroids

343

and min

X

( f (A)y A + g(A)z A ) : y, z ≥ 0,

A⊆E

X A⊆E, e∈A

(y A + z A ) ≥ ce for all e ∈ E

To show total dual integrality, we use Lemma 5.23. Let c : E(G) → Z, and let y, z be an optimum dual solution for which X (y A + z A )|A||E \ A| (14.1) A⊆E

is as small as possible. We claim that F := {A ⊆ E : y A > 0} is a chain, i.e. for any A, B ∈ F either A ⊆ B or B ⊆ A. To see this, suppose A, B ∈ F with A ∩ B 6= A and A ∩ B 6= B. Let := min{y A , y B }. Set y 0A := y A − , y B0 := y B − , y 0A∩B := y A∩B + , y 0A∪B := y A∪B + , and y 0 (S) := y(S) for all other S ⊆ E. Since y 0 , z is a feasible dual solution, it is also optimum ( f is submodular) and contradicts the choice of y, because (14.1) is smaller for y 0 , z. By the same argument, F 0 := {A ⊆ E : z A > 0} is a chain. Now let M and M 0 be the matrices whose columns are indexed with the elements of E and whose rows are the incidence vectors of the elements of F and F 0 , respectively. M By Lemma 5.23, it suffices to show that M 0 is totally unimodular. Here we use Ghouila-Houri’s Theorem 5.24. Let R be a set of rows, say R = {A1 , . . . , A p , B1 , . . . , Bq } with A1 ⊇ · · · ⊇ A p and B1 ⊇ · · · ⊇ Bq . Let R1 := {Ai : i odd} ∪ {Bi : i even} and R2 := R \ R1 . Since for any e ∈ E we have {R ∈ R : e ∈ R} = {A1 , . . . , A pe } ∪ {B1 , . . . , Bqe } for some pe ∈ {0, . . . , p} and qe ∈ {0, . . . , q}, the sum of the rows in R1 minus the sum of the rows in R2 is a vector with entries −1, 0, 1 only. So the criterion of Theorem 5.24 is satisfied. One can optimize linear functions over the intersection of two polymatroids. However, this is not as easy as with a single polymatroid. But we can use the E LLIP SOID M ETHOD if we can solve the S EPARATION P ROBLEM for each polymatroid. We return to this question in Section 14.3. Corollary 14.13. (Edmonds [1970]) Let (E, M1 ) and (E, M2 ) be two matroids with rank functions r1 and r2 . Then the convex hull of the incidence vectors of the elements of M1 ∩ M2 is the polytope ( ) X E x ∈ R+ : xe ≤ min{r1 (A), r2 (A)} for all A ⊆ E . e∈A

Proof: As r1 and r2 are nonnegative and submodular (by Theorem 13.10), the above inequality system is TDI (by Theorem 14.12). Since r1 and r2 are integral, the polytope is integral (by Corollary 5.15). Since r1 (A) ≤ |A| for all A ⊆ E,

.

344

14 Generalizations of Matroids

the vertices (the convex hull of which the polytope is by Corollary 3.32) are 01-vectors, and thus incidence vectors of common independent sets (elements of M1 ∩ M2 ). On the other hand, each such incidence vector satisfies the inequalities (by definition of the rank function). Of course, the description of the matroid polytope (Theorem 13.21) follows from this by setting M1 = M2 . Theorem 14.12 has some further consequences: Corollary 14.14. (Edmonds [1970]) Let E be a finite set, and let f, g : 2 E → R+ be submodular and monotone functions. Then max{1lx : x ∈ P( f ) ∩ P(g)} = min ( f (A) + g(E \ A)). A⊆E

Moreover, if f and g are integral, there exists an integral x attaining the maximum. Proof: By Theorem 14.12, the dual to max{1lx : x ∈ P( f ) ∩ P(g)}, which is X min ( f (A)y A + g(A)z A ) : y, z ≥ 0, A⊆E

X

(y A + z A ) ≥ 1 for all e ∈ E

A⊆E, e∈A

,

S S has an integral optimum solution y, z. Let B := A:y A ≥1 A and C := A:z A ≥1 A. Let y B0 := 1, z C0 := 1 and let all other components of y 0 and z 0 be zero. We have B ∪ C = E and y 0 , z 0 is a feasible dual solution. Since f and g are submodular and nonnegative, X ( f (A)y A + g(A)z A ) ≥ f (B) + g(C). A⊆E

Since E \ B ⊆ C and g is monotone, this is at least f (B) + g(E \ B), proving “≥”. The other inequality “≤” is trivial, because for any A ⊆ E we obtain a feasible dual solution y, z by setting y A := 1, z E\A := 1 and all other components to zero. The integrality follows directly from Theorem 14.12 and Corollary 5.15. Theorem 13.31 is a special case. Moreover we obtain: Corollary 14.15. (Frank [1982]) Let E be a finite set and f, g : 2 E → R such that f is supermodular, g is submodular and f ≤ g. Then there exists a modular function h : 2 E → R with f ≤ h ≤ g. If f and g are integral, h can be chosen integral. Proof: Let M := 2 max{| f (A)| + |g(A)| : A ⊆ E}. Let f 0 (A) := g(E) − f (E \ A) + M|A| and g 0 (A) := g(A) − f (∅) + M|A| for all A ⊆ E. f 0 and g 0 are nonnegative, submodular and monotone. An application of Corollary 14.14 yields

14.3 Minimizing Submodular Functions

345

max{1lx : x ∈ P( f 0 ) ∩ P(g 0 )} = min ( f 0 (A) + g 0 (E \ A)) A⊆E

= min (g(E) − f (E \ A) + M|A| + g(E \ A) − f (∅) + M|E \ A|) A⊆E

≥ g(E) − f (∅) + M|E|. f (∅)+ M|E|. If f and g are integral, x So let x ∈ P( f 0 )∩ P(g 0 ) with 1lx = g(E)− P can be chosen integral. Let h 0 (A) := e∈A xe and h(A) := h 0 (A) + f (∅) − M|A| for all A ⊆ E. The function h is modular. Moreover, for all A ⊆ E we have h(A) ≤ g 0 (A)+ f (∅)− M|A| = g(A) and h(A) = 1lx −h 0 (E \ A)+ f (∅)− M|A| ≥ g(E) + M|E| − M|A| − f 0 (E \ A) = f (A). The analogy to convex and concave functions is obvious; see also Exercise 9.

14.3 Minimizing Submodular Functions The S EPARATION PP ROBLEM for a polymatroid P( f ) and a vector x asks for a set this problem reduces to finding a set A minimizing A with f (A) < e∈A x(e). SoP g(A), where g(A) := f (A) − e∈A x(e). Note that if f is submodular, then g is also submodular. Therefore it is an interesting problem to minimize submodular functions. Another motivation might be that submodular functions can be regarded as the discrete analogue of convex functions (Corollary 14.15 and Exercise 9). We have already solved a special case in Section 8.7: finding the minimum cut in an undirected graph can be regarded as minimizing a certain symmetric submodular function f : 2U → R+ over 2U \ {∅, U }. Before returning to this special case we first show how to minimize general submodular functions. For simplicity we restrict ourselves to integer-valued submodular functions:

S UBMODULAR F UNCTION M INIMIZATION P ROBLEM Instance: Task:

A finite set U . A submodular function f : 2U → Z (given by an oracle). Find a subset X ⊆ U with f (X ) minimum.

Gr¨otschel, Lov´asz and Schrijver [1981] showed how this problem can be solved with the help of the E LLIPSOID M ETHOD. The idea is to determine the minimum by binary search; this will reduce the problem to the S EPARATION P ROBLEM for a polymatroid. Using the equivalence of separation and optimization (Section 4.6), it thus suffices to optimize linear functions over polymatroids. However, this can be done easily by the P OLYMATROID G REEDY A LGORITHM. We first need an upper bound on | f (S)| for S ⊆ U : Proposition 14.16. For any submodular function f : 2U → Z and any S ⊆ U we have

346

14 Generalizations of Matroids

f (U )−

X

max{0, f ({u})− f (∅)} ≤ f (S) ≤ f (∅)+

u∈U

X

max{0, f ({u})− f (∅)}.

u∈U

In particular, a number B with | f (S)| ≤ B for all S ⊆ U can be computed in linear time, with |U | + 2 oracle calls to f . Proof: By repeated application of submodularity we get for ∅ 6= S ⊆ U (let x ∈ S): X f (S) ≤ − f (∅) + f (S \ {x}) + f ({x}) ≤ · · · ≤ −|S| f (∅) + f (∅) + f ({x}), x∈S

and for S ⊂ U (let y ∈ U \ S): f (S) ≥ − f ({y}) + f (S ∪ {y}) + f (∅) ≥ · · · X f ({y}) + f (U ) + |U \ S| f (∅). ≥ − y∈U \S

Proposition 14.17. The following problem can be solved in polynomial time: Given a finite set U , a submodular and monotone function f : 2U → Z+ (by an oracle) with f (S) > 0 for S 6= ∅, a number B ∈ N with f (S) ≤ B for all S ⊆ U , and a vector x ∈ ZU + , decide if x ∈ P( f ) and otherwise return a set S ⊆ U with P x(v) > f (S). v∈S Proof: This is the S EPARATION P ROBLEM for the polymatroid P( f ). We will use Theorem 4.23, because we have already solved the optimization problem for P( f ): the P OLYMATROID G REEDY A LGORITHM maximizes any linear function over P( f ) (Theorem 14.11). We have to check the prerequisites of Theorem 4.23. Since the zero vector and the unit vectors are all in P( f ), we can take x0 := 1l as a point in the interior, where = |U 1|+1 . We have size(x0 ) = O(|U | log |U |). Moreover, each vertex of P( f ) is produced by the P OLYMATROID G REEDY A LGORITHM (for some objective function; cf. Theorem 14.11) and thus has size O(|U |(2 + log B)). We conclude that the S EPARATION P ROBLEM can be solved in polynomial time. By Theorem 4.23, we get a facet-defining inequality P of P( f ) violated by x if x∈ / P( f ). This corresponds to a set S ⊆ U with v∈S x(v) > f (S). If f is not monotone, we cannot apply this result directly. Instead we consider a different function: Proposition 14.18. Let f : 2U → R be a submodular function and β ∈ R. Then g : 2U → R, defined by X ( f (U \ {e}) − f (U )), g(X ) := f (X ) − β + e∈X

is submodular and monotone.

14.4 Schrijver’s Algorithm

347

Proof: The submodularity of g follows directly from the submodularity of f . To show that g is monotone, let X ⊂ U and e ∈ U \ X . We have g(X ∪ {e}) − g(X ) = f (X ∪ {e}) − f (X ) + f (U \ {e}) − f (U ) ≥ 0 since f is submodular. Theorem 14.19. The S UBMODULAR F UNCTION M INIMIZATION P ROBLEM can be solved in time polynomial in |U | + log max{| f (S)| : S ⊆ U }. Proof: Let U be a finite set; suppose we are given f by an oracle. First compute a number B ∈ N with | f (S)| ≤ B for all S ⊆ U (cf. Proposition 14.16). Since f is submodular, we have for each e ∈ U and for each X ⊆ U \ {e}: f ({e}) − f (∅) ≥ f (X ∪ {e}) − f (X ) ≥ f (U ) − f (U \ {e}).

(14.2)

If, for some e ∈ U , f ({e}) − f (∅) ≤ 0, then by (14.2) there is an optimum set S containing e. In this case we consider the instance (U 0 , B, f 0 ) defined by U 0 := U \ {e} and f 0 (X ) := f (X ∪ {e}) for X ⊆ U \ {e}, find a set S 0 ⊆ U 0 with f 0 (S 0 ) minimum and output S := S 0 ∪ {e}. Similarly, if f (U ) − f (U \ {e}) ≥ 0, then by (14.2) there is an optimum set S not containing e. In this case we simply minimize f restricted to U \ {e}. In both cases we have reduced the size of the ground set. So we may assume that f ({e}) − f (∅) > 0 and f (U \ {e}) − f (U ) > 0 for all e ∈ U . Let x(e) := f (U \ {e}) − f (UP ). For each integer β with −B ≤ β ≤ f (∅) we define g(X ) := f (X ) − β + e∈X x(e). By Proposition 14.18, g is submodular and monotone. Furthermore we have g(∅) = f (∅) − β ≥ 0 and g({e}) = f ({e}) − β + x(e) > 0 for all e ∈ U , and thus g(X ) > 0 for all ∅ 6= X ⊆ U . Now we apply Proposition 14.17 and check if x ∈ P(g). If yes, we have f (X ) ≥ β for all X ⊆ U and we are done. Otherwise we get a set S with f (S) < β. Now we apply binary search: by choosing β appropriately each time, we need O(log(2B)) iterations to find the number β ∗ ∈ {−B, −B + 1, . . . , f (∅)} for which f (X ) ≥ β ∗ for all X ⊆ U but f (S) < β ∗ +1 for some S ⊆ U . This set S minimizes f. The first strongly polynomial-time algorithm has been designed by Gr¨otschel, Lov´asz and Schrijver [1988], also based on the ellipsoid method. Combinatorial algorithms to solve the S UBMODULAR F UNCTION M INIMIZATION P ROBLEM in strongly polynomial time have been found by Schrijver [2000] and independently by Iwata, Fleischer and Fujishige [2001]. In the next section we describe Schrijver’s algorithm.

14.4 Schrijver’s Algorithm For a finite set U and a submodular function f : 2U → Z, assume w.l.o.g. that U = {1, . . . , n} and f (∅) = 0. At each stage, Schrijver’s [2000] algorithm maintains a point x in the so-called base polyhedron of f , defined by

348

14 Generalizations of Matroids

(

) U

x ∈R :

X

x(u) ≤ f (A) for all A ⊆ U,

u∈A

X

x(u) = f (U ) .

u∈U

We mention that the set of vertices of this base polyhedron is precisely the set of vectors b≺ for all total orders ≺ of U , where we define b≺ (u) := f ({v ∈ U : v u}) − f ({v ∈ U : v ≺ u}) (u ∈ U ). This fact, which we will not need here, can be proved in a similar way as Theorem 14.11 (Exercise 13). The point x is always written as an explicit convex combination x = λ1 b≺1 + · · · + λk b≺k of these vertices. Initially, one can choose k = 1 and any total order. For a total order ≺ and s, u ∈ U , we denote by ≺s,u the total order that results from ≺ by moving u just before s. Moreover, let χ u denote the incidence vector of u (u ∈ U ).

S CHRIJVER ’ S A LGORITHM Input: Output:

A finite set U = {1, . . . , n}. A submodular function f : 2U → Z with f (∅) = 0 (given by an oracle). A subset X ⊆ U with f (X ) minimum.

1

Set k := 1, let ≺1 be any total order on U , and set x := b≺1 .

2

Set D := (U, A), where A = {(u, v) : u ≺i v for some i ∈ {1, . . . , k}}.

3

Let P := {v ∈ U : x(v) > 0} and N := {v ∈ U : x(v) < 0}, and let X be the set of vertices not reachable from P in the digraph D. If N ⊆ X , then stop else let d(v) denote the distance from P to v in D. Choose the vertex t ∈ N reachable from P with (d(t), t) lexicographically maximum. Choose the maximal vertex s with (s, t) ∈ A and d(s) = d(t) − 1. Let i ∈ {1, . . . , k} such that α := |{v : s ≺i v i t}| is maximum (the number of indices attaining this maximum will be denoted by β).

4

5

Compute a number with 0 ≤ ≤ −x(t) and write x 0 := x + (χ t − χ s ) as an explicit convex combination of at most n vectors, chosen among s,u b≺1 , . . . , b≺k and b≺i for all u ∈ U with s ≺i u i t, with the additional property that b≺i does not occur if x 0 (t) < 0.

6

Set x := x 0 , rename the vectors in the convex combination of x as b≺1 , . . . , b≺k 0 , set k := k 0 , and go to . 2

Theorem 14.20. (Schrijver [2000]) S CHRIJVER ’ S A LGORITHM works correctly. Proof: The algorithm terminates if D contains no path from P to N and outputs of vertices not reachable from P. Clearly N ⊆ X ⊆ U \ P, so P the set XP x(u) ≤ u∈X u∈W x(u) for each W ⊆ U . Moreover, no edge enters X , so either X = ∅ or for each j ∈ {1, . . . , k} there exists a v ∈ X with X = {u ∈ U : u j v}.

14.4 Schrijver’s Algorithm

349

P ≺j Hence, by definition, P u∈X≺b (u) = f (X ) for all j ∈ {1, . . . , k}. Moreover, by Proposition 14.10, u∈W b j (u) ≤ f (W ) for all W ⊆ U and j ∈ {1, . . . , k}. Therefore, for each W ⊆ U , f (W ) ≥

k X

λj

≥

b≺ j (u) =

u∈W

j=1

X

X

x(u) =

u∈X

k XX

λ j b≺ j (u) =

u∈W j=1 k XX

λ j b (u) = ≺j

u∈X j=1

X

x(u)

u∈W k X j=1

λj

X

b≺ j (u) = f (X ),

u∈X

proving that X is an optimum solution.

Lemma 14.21. (Schrijver [2000]) Each iteration can be performed in O(n 3 + γ n 2 ) time, where γ is the time for an oracle call. Proof: It suffices to show that 5 can be done in O(n 3 + γ n 2 ) time. Let x = ≺ ≺ λ1 b 1 + · · · + λk b k and s ≺i t. We first show: Claim: δ(χ t − χ s ), for some δ ≥ 0, can be written as convex combination of s,v the vectors b≺i − b≺i for s ≺i v i t in O(γ n 2 ) time. To prove this, we need some preliminaries. Let s ≺i v i t. By definition, ≺is,v b (u) = b≺i (u) for u ≺i s or u i v. As f is submodular, we have for s i u ≺i v: s,v

f ({w ∈ U : w is,v u}) − f ({w ∈ U : w ≺is,v u})

b≺i (u) =

f ({w ∈ U : w i u}) − f ({w ∈ U : w ≺i u}) = b≺i (u).

≤

Moreover, for u = v we have: s,v

b≺i (v) =

f ({w ∈ U : w is,v v}) − f ({w ∈ U : w ≺is,v v})

=

f ({w ∈ U : w ≺i s} ∪ {v}) − f ({w ∈ U : w ≺i s})

≥

f ({w ∈ U : w i v}) − f ({w ∈ U : w ≺i v})

= b≺i (v). s,v P P Finally, observe that u∈U b≺i (u) = f (U ) = u∈U b≺i (u). s,v As the claim is trivial if b≺i = b≺i for some s ≺i v i t, we may assume s,v b≺i (v) > b≺i (v) for all s ≺i v i t. We recursively set κv :=

χvt −

P

s,w

v≺i wi t ≺is,v

b

κw (b≺i (v) − b≺i (v)) (v) − b≺i (v)

≥ 0

P ≺is,v − b≺i ) = χ t − χ s , because for s ≺i v i t, and obtain s≺i vi t κv (b s,v s,v P P ≺i (u) − b≺i (u)) = ui vi t κv (b≺i (u) − b≺i (u)) = χut for all s≺i vi t κv (b s ≺i u i t, and the sum over all components is zero.

350

14 Generalizations of Matroids

By setting δ :=

1 P

s≺i vi t

κv

and multiplying each κu by δ, we see that the claim

follows. Now consider := min{λi δ, −x(t)} and x 0 := x + (χ t − χ s ). If = λi δ ≤ Pk P ≺is,v ≺j + λ − b≺i ), i.e. we −x(t), then we have x 0 = i j=1 λ j b s≺i vi t κv (b s,v

have written x 0 as a convex combination of b≺ j ( j ∈ {1, . . . , k} \ {i}) and b≺i (s ≺i v i t). If = −x(t), we may additionally use b≺i in the convex combination. We finally reduce this convex combination to at most n vectors in O(n 3 ) time, as shown in Exercise 5 of Chapter 4. Lemma 14.22. (Vygen [2003]) O(n 5 ) iterations.

S CHRIJVER ’ S A LGORITHM terminates after s,v

Proof: If an edge (v, w) is introduced after a new vector b≺i has been added in

5 of an iteration, then s i w ≺i v i t in this iteration. Thus d(w) ≤ d(s) + 1 = d(t) ≤ d(v) + 1 in this iteration, and the introduction of the new edge cannot make the distance from P to any v ∈ U smaller. As 5 makes sure that no element is ever added to P, the distance d(v) never decreases for any v ∈ U . Call a block a sequence of iterations where the pair (t, s) remains constant. Note that each block has O(n 2 ) iterations, because (α, β) decreases lexicographically in each iteration within each block. It remains to prove that there are O(n 3 ) blocks. A block can end only because of at least one of the following reasons (by the choice of t and s, since an iteration with t = t ∗ does not add any edge whose head is t ∗ , and since a vertex v can enter N only if v = s and hence d(v) < d(t)): (a) the distance d(v) increases for some v ∈ U . (b) t is removed from N . (c) (s, t) is removed from A. We now count the number of blocks of these three types. Clearly there are O(n 2 ) blocks of type (a). Now consider type (b). We claim that for each t ∗ ∈ U there are O(n 2 ) iterations with t = t ∗ and x 0 (t) = 0. This is easy to see: between every two such iterations, d(v) must change for some v ∈ U , and this can happen O(n 2 ) times as d-values can only increase. Thus there are O(n 3 ) blocks of type (b). We finally show that there are O(n 3 ) blocks of type (c). It suffices to show that d(t) will change before the next such block with the pair (s, t). For s, t ∈ U , we say that s is t-boring if (s, t) ∈ / A or d(t) ≤ d(s). Let s ∗ , t ∗ ∈ U , and consider the time period after a block with s = s ∗ and t = t ∗ ending because (s ∗ , t ∗ ) is removed from A, until the subsequent change of d(t ∗ ). We prove that each v ∈ {s ∗ , . . . , n} is t ∗ -boring throughout this period. Applying this for v = s ∗ concludes the proof. At the beginning of the period, each v ∈ {s ∗ + 1, . . . , n} is t ∗ -boring due to the choice of s = s ∗ in the iteration immediately preceding the period. s ∗ is also t ∗ -boring as (s ∗ , t ∗ ) is removed from A. As d(t ∗ ) remains constant within the considered time period and d(v) never decreases for any v, we only have to check the introduction of new edges.

14.5 Symmetric Submodular Functions

351

Suppose that, for some v ∈ {s ∗ , . . . , n}, the edge (v, t ∗ ) is added to A after an iteration that chooses the pair (s, t). Then, by the initial remarks of this proof, s i t ∗ ≺i v i t in this iteration, and thus d(t ∗ ) ≤ d(s) + 1 = d(t) ≤ d(v) + 1. Now we distinguish two cases: If s > v, then we have d(t ∗ ) ≤ d(s): either because t ∗ = s, or as s was t ∗ -boring and (s, t ∗ ) ∈ A. If s < v, then we have d(t) ≤ d(v): either because t = v, or by the choice of s and since (v, t) ∈ A. In both cases we conclude that d(t ∗ ) ≤ d(v), and v remains t ∗ -boring. Theorem 14.20, Lemma 14.21 and Lemma 14.22 imply: Theorem 14.23. The S UBMODULAR F UNCTION M INIMIZATION P ROBLEM can be solved in O(n 8 + γ n 7 ) time, where γ is the time for an oracle call. Iwata [2002] described a fully combinatorial algorithm (using only additions, subtractions, comparisons and oracle calls, but no multiplication or division). He also improved the running time (Iwata [2003]). The currently fastest strongly polynomial-time algorithm was found by Orlin [2007]; it runs in O(n 6 + γ n 5 ) time.

14.5 Symmetric Submodular Functions A submodular function f : 2U → R is called symmetric if f (A) = f (U \ A) for all A ⊆ U . In this special case the S UBMODULAR F UNCTION M INIMIZATION P ROBLEM is trivial, since 2 f (∅) = f (∅) + f (U ) ≤ f (A) + f (U \ A) = 2 f (A) for all A ⊆ U , implying that the empty set is optimal. Hence the problem is interesting only if this trivial case is excluded: one looks for a nonempty proper subset A of U such that f (A) is minimum. Generalizing the algorithm of Section 8.7, Queyranne [1998] has found a relatively simple combinatorial algorithm for this problem using only O(n 3 ) oracle calls. The following lemma is a generalization of Lemma 8.38 (Exercise 14): Lemma 14.24. Given a symmetric submodular function f : 2U → R with n := |U | ≥ 2, we can find two elements x, y ∈ U with x 6= y and f ({x}) = min{ f (X ) : x ∈ X ⊆ U \ {y}} in O(n 2 θ ) time, where θ is the time bound of the oracle for f . Proof: We construct an order U = {u 1 , . . . , u n } by doing the following for k = 1, . . . , n − 1. Suppose that u 1 , . . . , u k−1 are already constructed; let Uk−1 := {u 1 , . . . , u k−1 }. For C ⊆ U we define 1 wk (C) := f (C) − ( f (C \ Uk−1 ) + f (C ∪ Uk−1 ) − f (Uk−1 )). 2 Note that wk is also symmetric. Let u k be an element of U \ Uk−1 that maximizes wk ({u k }). Finally, let u n be the only element in U \ {u 1 , . . . , u n−1 }. Obviously the construction of the order u 1 , . . . , u n can be done in O(n 2 θ ) time.

352

14 Generalizations of Matroids

Claim: For all k = 1, . . . , n − 1 and all x, y ∈ U \ Uk−1 with x 6= y and wk ({x}) ≤ wk ({y}) we have wk ({x}) = min{wk (C) : x ∈ C ⊆ U \ {y}}. We prove the claim by induction on k. For k = 1 the assertion is trivial since w1 (C) = 21 f (∅) for all C ⊆ U . Let now k > 1 and x, y ∈ U \ Uk−1 with x 6= y and wk ({x}) ≤ wk ({y}). / Z , and let z ∈ Z \ Uk−1 . By the choice of Moreover, let Z ⊆ U with u k−1 ∈ u k−1 we have wk−1 ({z}) ≤ wk−1 ({u k−1 }); thus by the induction hypothesis we get wk−1 ({z}) ≤ wk−1 (Z ). Furthermore, the submodularity of f implies (wk (Z ) − wk−1 (Z )) − (wk ({z}) − wk−1 ({z})) 1 = ( f (Z ∪ Uk−2 ) − f (Z ∪ Uk−1 ) − f (Uk−2 ) + f (Uk−1 )) 2 1 − ( f ({z} ∪ Uk−2 ) − f ({z} ∪ Uk−1 ) − f (Uk−2 ) + f (Uk−1 )) 2 1 ( f (Z ∪ Uk−2 ) + f ({z} ∪ Uk−1 ) − f (Z ∪ Uk−1 ) − f ({z} ∪ Uk−2 )) = 2 ≥ 0. Hence wk (Z ) − wk ({z}) ≥ wk−1 (Z ) − wk−1 ({z}) ≥ 0. To conclude the proof of the claim, let C ⊆ U with x ∈ C and y ∈ / C. There are two cases: / C. Then the above result for Z = C and z = x yields wk (C) ≥ Case 1: u k−1 ∈ wk ({x}) as required. Case 2: u k−1 ∈ C. Then we apply the above to Z = U \ C and z = y and get wk (C) = wk (U \ C) ≥ wk ({y}) ≥ wk ({x}). This completes the proof of the claim. Applying it to k = n − 1, x = u n and y = u n−1 we get wn−1 ({u n }) = min{wn−1 (C) : u n ∈ C ⊆ U \ {u n−1 }}. Since wn−1 (C) = f (C) − 21 ( f ({u n }) + f (U \ {u n−1 }) − f (Un−2 )) for all C ⊆ U / C, the lemma follows (set x := u n and y := u n−1 ). with u n ∈ C and u n−1 ∈ The above proof is due to Fujishige [1998]. Now we can proceed analogously to the proof of Theorem 8.39: Theorem 14.25. (Queyranne [1998]) Given a symmetric submodular function f : 2U → R, a nonempty proper subset A of U such that f (A) is minimum can be found in O(n 3 θ ) time where θ is the time bound of the oracle for f . Proof: If |U | = 1, the problem is trivial. Otherwise we apply Lemma 14.24 and find two elements x, y ∈ U with f ({x}) = min{ f (X ) : x ∈ X ⊆ U \ {y}} in O(n 2 θ ) time. Next we recursively find a nonempty proper subset of U \ {x} / X minimizing the function f 0 : 2U \{x} → R, defined by f 0 (X ) := f (X ) if y ∈

Exercises

353

and f 0 (X ) := f (X ∪ {x}) if y ∈ X . One readily observes that f 0 is symmetric and submodular. Let ∅ 6= Y ⊂ U \{x} be a set minimizing f 0 ; w.l.o.g. y ∈ Y (as f 0 is symmetric). We claim that either {x} or Y ∪ {x} minimizes f (over all nonempty proper subsets of U ). To see this, consider any C ⊂ U with x ∈ C. If y ∈ / C, then we have f ({x}) ≤ f (C) by the choice of x and y. If y ∈ C, then f (C) = f 0 (C \ {x}) ≥ f 0 (Y ) = f (Y ∪ {x}). Hence f (C) ≥ min{ f ({x}), f (Y ∪ {x})} for all nonempty proper subsets C of U . To achieve the asserted running time we of course cannot compute f 0 explicitly. Rather we store a partition of U , initially consisting of the singletons. At each step of the recursion we build the union of those two sets of the partition that contain x and y. In this way f 0 can be computed efficiently (using the oracle for f ). This result has been further generalized by Nagamochi and Ibaraki [1998] and by Rizzi [2000].

Exercises 1. Let G be an undirected graph and M a maximum matching in G. Let F be the family of those subsets X ⊆ E(G) for which there exists a special blossom forest F with respect to M with E(F) \ M = X . Prove that (E(G) \ M, F) is a greedoid. Hint: Use Exercise 23 of Chapter 10. 2. Let (E, F) be a greedoid and c0 : E → R+ . We consider the bottleneck function c(F) := min{c0 (e) : e ∈ F} for F ⊆ E. Show that the G REEDY A LGORITHM F OR G REEDOIDS, when applied to (E, F) and c, finds an F ∈ F with c(F) maximum. 3. This exercise shows that greedoids can also be defined as languages (cf. Definition 15.1). Let E be a finite set. A language L over the alphabet E is called a greedoid language if (a) L contains the empty string; (b) xi 6= x j for all (x1 , . . . , xn ) ∈ L and 1 ≤ i < j ≤ n; (c) (x1 , . . . , xn−1 ) ∈ L for all (x1 , . . . , xn ) ∈ L; (d) If (x1 , . . . , xn ), (y1 , . . . , ym ) ∈ L with m < n, then there exists an i ∈ {1, . . . , n} such that (y1 , . . . , ym , xi ) ∈ L. L is called an antimatroid language if it satisfies (a), (b), (c) and (d0 ) If (x1 , . . . , xn ), (y1 , . . . , ym ) ∈ L with {x1 , . . . , xn } 6⊆ {y1 , . . . , ym }, then there exists an i ∈ {1, . . . , n} such that (y1 , . . . , ym , xi ) ∈ L. Prove: A language L over the alphabet E is a greedoid language (an antimatroid language) if and only if the set system (E, F) is a greedoid (antimatroid), where F := {{x1 , . . . , xn } : (x1 , . . . , xn ) ∈ L}. 4. Let U be a finite set and f : 2U → R. Prove that f is submodular if and only if f (X ∪ {y, z}) − f (X ∪ {y}) ≤ f (X ∪ {z}) − f (X ) for all X ⊆ U and y, z ∈ U .

354

∗

14 Generalizations of Matroids

5. Let P be a nonempty polymatroid. Show that then there is a submodular and monotone function f with f (∅) = 0 and P = P( f ). 6. Prove that a nonempty compact set P ⊆ Rn+ is a polymatroid if and only if (a) For all 0 ≤ x ≤ y ∈ P we have x ∈ P. (b) For all x ∈ Rn+ and all y, z ≤ x with y, z ∈ P that are maximal with this property (i.e. y ≤ w ≤ x and w ∈ P implies w = y, and z ≤ w ≤ x and w ∈ P implies w = z) we have 1ly = 1lz. Note: This is the original definition of Edmonds [1970]. 7. Prove that the P OLYMATROID G REEDY A LGORITHM, when applied to a vector E and a submodular but not necessarily monotone function f : 2 E → R c ∈ R+ with f (∅) ≥ 0, solves ( ) X max cx : xe ≤ f (A) for all A ⊆ E . e∈A

8. Prove Theorem 14.12 for the special case that f and g are rank functions of matroids by constructing an integral optimum dual solution from c1 and c2 as generated by the W EIGHTED M ATROID I NTERSECTION A LGORITHM. (Frank [1981]) S → R as follows. For ∗ 9. Let S be a finite set and f : 2 S → R. Define f 0 : R+ S any x ∈ R+ there are unique k ∈ Z+ , λ1 , . . . , λk > 0 and ∅ ⊂ T1 ⊂ T2 ⊂ Pk λi χ Ti , where χ Ti is the incidence vector · · · ⊂ Tk ⊆ S such that x = i=1 P k λi f (Ti ). of Ti . Then f 0 (x) := i=1 Prove that f is submodular if and only if f 0 is convex. (Lov´asz [1983]) 10. Let E be a finite set and f : 2 E → R+ a submodular function with f ({e}) ≤ 2 for all e ∈ E. (The pair (E, f ) is sometimes called a 2-polymatroid.) The P OLYMATROID M ATCHING P ROBLEM asks for a maximum cardinality set X ⊆ E with f (X ) = 2|X |. ( f is of course given by an oracle.) Let E 1 , . . . , E k be pairwise disjoint unordered pairs and let (E, F) be a matroid (given by an independence oracle), where E = E 1 ∪ · · · ∪ E k . The M ATROID PARITY P ROBLEM asks for a maximum cardinality set I ⊆ {1, . . . , k} with S i∈I E i ∈ F. (a) Show that the M ATROID PARITY P ROBLEM polynomially reduces to the P OLYMATROID M ATCHING P ROBLEM. ∗ (b) Show that the P OLYMATROID M ATCHING P ROBLEM polynomially reduces to the M ATROID PARITY P ROBLEM. Hint: Use an algorithm for the S UBMODULAR F UNCTION M INIMIZATION P ROBLEM. ∗ (c) Show that there is no algorithm for the P OLYMATROID M ATCHING P ROB LEM whose running time is polynomial in |E|. (Jensen and Korte [1982], Lov´asz [1981]) (A problem polynomially reduces to another one if the former can be solved with a polynomial-time oracle algorithm using an oracle for the latter; see Chapter 15.)

References

355

Note: A polynomial-time algorithm for an important special case was given by Lov´asz [1980,1981]. 11. A function f : 2 S → R∪{∞} is called crossing submodular if f (X )+ f (Y ) ≥ f (X ∪Y )+ f (X ∩Y ) for any two sets X, Y ⊆ S with X ∩Y 6= ∅ and X ∪Y 6= S. The S UBMODULAR F LOW P ROBLEM is as follows: Given a digraph G, functions l : E(G) → R ∪ {−∞}, u : E(G) → R ∪ {∞}, c : E(G) → R, and a crossing submodular function b : 2V (G) → R ∪ {∞}. Then a feasible submodular flow is a function f : E(G) → R with l(e) ≤ f (e) ≤ u(e) for all e ∈ E(G) and X X f (e) − f (e) ≤ b(X ) e∈δ − (X )

e∈δ + (X )

for all X ⊆ V (G). The task is Pto decide whether a feasible flow exists and, if yes, to find one whose cost e∈E(G) c(e) f (e) is minimum possible. Show that this problem generalizes the M INIMUM C OST F LOW P ROBLEM and the problem of optimizing a linear function over the intersection of two polymatroids. Note: The S UBMODULAR F LOW P ROBLEM, introduced by Edmonds and Giles [1977], can be solved in strongly polynomial time; see Fujishige, R¨ock and Zimmermann [1989]. See also Fleischer and Iwata [2000]. ∗ 12. Show that the inequality system describing a feasible submodular flow (Exercise 11) is TDI. Show that this implies Theorems 14.12 and 19.10. (Edmonds and Giles [1977]) 13. Prove that the set of vertices of the base polyhedron is precisely the set of vectors b≺ for all total orders ≺ of U , where b≺ (u) := f ({v ∈ U : v u}) − f ({v ∈ U : v ≺ u}) (u ∈ U ). Hint: See the proof of Theorem 14.11. 14. Show that Lemma 8.38 is a special case of Lemma 14.24.

References General Literature: Bixby, R.E., and Cunningham, W.H. [1995]: Matroid optimization and algorithms. In: Handbook of Combinatorics; Vol. 1 (R.L. Graham, M. Gr¨otschel, L. Lov´asz, eds.), Elsevier, Amsterdam, 1995 Bj¨orner, A., and Ziegler, G.M. [1992]: Introduction to greedoids. In: Matroid Applications (N. White, ed.), Cambridge University Press, Cambridge 1992 Fujishige, S. [2005]: Submodular Functions and Optimization. Second Edition. Elsevier, Amsterdam 2005 Korte, B., Lov´asz, L., and Schrader, R. [1991]: Greedoids. Springer, Berlin 1991 McCormick, S.T. [2004]: Submodular function minimization. In: Discrete Optimization (K. Aardal, G.L. Nemhauser, R. Weismantel, eds.), Elsevier, Amsterdam 2005 Schrijver, A. [2003]: Combinatorial Optimization: Polyhedra and Efficiency. Springer, Berlin 2003, Chapters 44–49

356

14 Generalizations of Matroids

Cited References: Edmonds, J. [1970]: Submodular functions, matroids and certain polyhedra. In: Combinatorial Structures and Their Applications; Proceedings of the Calgary International Conference on Combinatorial Structures and Their Applications 1969 (R. Guy, H. Hanani, N. Sauer, J. Schonheim, eds.), Gordon and Breach, New York 1970, pp. 69–87 Edmonds, J. [1979]: Matroid intersection. In: Discrete Optimization I; Annals of Discrete Mathematics 4 (P.L. Hammer, E.L. Johnson, B.H. Korte, eds.), North-Holland, Amsterdam 1979, pp. 39–49 Edmonds, J., and Giles, R. [1977]: A min-max relation for submodular functions on graphs. In: Studies in Integer Programming; Annals of Discrete Mathematics 1 (P.L. Hammer, E.L. Johnson, B.H. Korte, G.L. Nemhauser, eds.), North-Holland, Amsterdam 1977, pp. 185–204 Fleischer, L., and Iwata, S. [2000]: Improved algorithms for submodular function minimization and submodular flow. Proceedings of the 32nd Annual ACM Symposium on Theory of Computing (2000), 107–116 Frank, A. [1981]: A weighted matroid intersection algorithm. Journal of Algorithms 2 (1981), 328–336 Frank, A. [1982]: An algorithm for submodular functions on graphs. In: Bonn Workshop on Combinatorial Optimization; Annals of Discrete Mathematics 16 (A. Bachem, M. Gr¨otschel, B. Korte, eds.), North-Holland, Amsterdam 1982, pp. 97–120 Fujishige, S. [1998]: Another simple proof of the validity of Nagamochi and Ibaraki’s min-cut algorithm and Queyranne’s extension to symmetric submodular function minimization. Journal of the Operations Research Society of Japan 41 (1998), 626–628 Fujishige, S., R¨ock, H., and Zimmermann, U. [1989]: A strongly polynomial algorithm for minimum cost submodular flow problems. Mathematics of Operations Research 14 (1989), 60–69 Gr¨otschel, M., Lov´asz, L., and Schrijver, A. [1981]: The ellipsoid method and its consequences in combinatorial optimization. Combinatorica 1 (1981), 169–197 Gr¨otschel, M., Lov´asz, L., and Schrijver, A. [1988]: Geometric Algorithms and Combinatorial Optimization. Springer, Berlin 1988 Iwata, S. [2002]: A fully combinatorial algorithm for submodular function minimization. Journal of Combinatorial Theory B 84 (2002), 203–212 Iwata, S. [2003]: A faster scaling algorithm for minimizing submodular functions. SIAM Journal on Computing 32 (2003), 833–840 Iwata, S., Fleischer, L., and Fujishige, S. [2001]: A combinatorial, strongly polynomialtime algorithm for minimizing submodular functions. Journal of the ACM 48 (2001), 761–777 Jensen, P.M., and Korte, B. [1982]: Complexity of matroid property algorithms. SIAM Journal on Computing 11 (1982), 184–190 Lov´asz, L. [1980]: Matroid matching and some applications. Journal of Combinatorial Theory B 28 (1980), 208–236 Lov´asz, L. [1981]: The matroid matching problem. In: Algebraic Methods in Graph Theory; Vol. II (L. Lov´asz, V.T. S´os, eds.), North-Holland, Amsterdam 1981, 495–517 Lov´asz, L. [1983]: Submodular functions and convexity. In: Mathematical Programming: The State of the Art – Bonn 1982 (A. Bachem, M. Gr¨otschel, B. Korte, eds.), Springer, Berlin 1983 Nagamochi, H., and Ibaraki, T. [1998]: A note on minimizing submodular functions. Information Processing Letters 67 (1998), 239–244

References

357

Orlin, J.B. [2007]: A faster strongly polynomial time algorithm for submodular function minimization. In: Integer Programming and Combinatorial Optimization; Proceedings of the 12th International IPCO Conference; LNCS 4513 (M. Fischetti, D.P. Williamson, eds.), Springer, Berlin 2007, pp. 240–251 Queyranne, M. [1998]: Minimizing symmetric submodular functions. Mathematical Programming B 82 (1998), 3–12 Rizzi, R. [2000]: On minimizing symmetric set functions. Combinatorica 20 (2000), 445– 450 Schrijver, A. [2000]: A combinatorial algorithm minimizing submodular functions in strongly polynomial time. Journal of Combinatorial Theory B 80 (2000), 346–355 Vygen, J. [2003]: A note on Schrijver’s submodular function minimization algorithm. Journal of Combinatorial Theory B 88 (2003), 399–402

15 NP-Completeness

For many combinatorial optimization problems a polynomial-time algorithm is known; the most important ones are presented in this book. However, there are also many important problems for which no polynomial-time algorithm is known. Although we cannot prove that none exists we can show that a polynomial-time algorithm for one “hard” (more precisely: NP-hard) problem would imply a polynomialtime algorithm for almost all problems discussed in this book (more precisely: all NP-easy problems). To formalize this concept and prove the above statement we need a machine model, i.e. a precise definition of a polynomial-time algorithm. Therefore we discuss Turing machines in Section 15.1. This theoretical model is not suitable to describe more complicated algorithms. However we shall argue that it is equivalent to our informal notion of algorithms: every algorithm in this book can, theoretically, be written as a Turing machine, with a loss in efficiency that is polynomially bounded. We indicate this in Section 15.2. In Section 15.3 we introduce decision problems, and in particular the classes P and NP. While NP contains most decision problems appearing in this book, P contains only those for which there are polynomial-time algorithms. It is an open question whether P = NP. Although we shall discuss many problems in NP for which no polynomial-time algorithm is known, nobody can (so far) prove that none exists. We specify what it means that one problem reduces to another, or that one problem is at least as hard as another one. In this notion, the hardest problems in NP are the NP-complete problems; they can be solved in polynomial time if and only if P = NP. In Section 15.4 we exhibit the first NP-complete problem, S ATISFIABILITY. In Section 15.5 some more decision problems, more closely related to combinatorial optimization, are proved to be NP-complete. In Sections 15.6 and 15.7 we shall discuss related concepts, also extending to optimization problems.

15.1 Turing Machines In this section we present a very simple model for computation: the Turing machine. It can be regarded as a sequence of simple instructions working on a string. The input and the output will be a binary string:

360

15 NP-Completeness

Definition 15.1. An alphabet is a finite set with at least two elements, not containing the special symbol t (which we shall use for blanks). For an alphabet A we denote by a string over A a finiteSsequence of elements of A, by An the set of strings of length n, and by A∗ := n∈Z+ An the set of all strings over A. We use the convention that A0 contains exactly one element, the empty string. A language over A is a subset of A∗ . The elements of a language are often called words. If x ∈ An we write size(x) := n for the length of the string. We shall often work with the alphabet A = {0, 1} and the set {0, 1}∗ of all 0-1-strings (or binary strings). The components of a 0-1-string are sometimes called its bits. So there is exactly one 0-1-string of zero length, the empty string. A Turing machine gets as input a string x ∈ A∗ for some fixed alphabet A. The input is completed by blank symbols (denoted by t) to a two-way infinite string s ∈ (A ∪ {t})Z . This string s can be regarded as a tape with a read-write head; only a single position can be read and modified at each step, and the read-write head can be moved by one position in each step. A Turing machine consists of a set of N + 1 instructions numbered 0, . . . , N . In the beginning instruction 0 is executed and the current position of the string is position 1. Now each instruction is of the following type. Read the bit at the current position, and depending on its value do the following: overwrite the current bit by some element of A ∪ {t}, possibly move the current position by one to the left or to the right, and go to a instruction which will be executed next. There is a special instruction denoted by −1 which marks the end of the computation. The components of our infinite string s indexed by 1, 2, 3, . . . up to the first t then yield the output string. Formally we define a Turing machine as follows: Definition 15.2. (Turing [1936]) Let A be an alphabet and A¯ := A ∪ {t}. A Turing machine (with alphabet A) is defined by a function 8 : {0, . . . , N } × A¯ → {−1, . . . , N } × A¯ × {−1, 0, 1} for some N ∈ Z+ . The computation of 8 on input x, where x ∈ A∗ , is the finite or infinite sequence of triples (n (i) , s (i) , π (i) ) with n (i) ∈ {−1, . . . , N }, s (i) ∈ A¯ Z and π (i) ∈ Z (i = 0, 1, 2, . . .) defined recursively as follows (n (i) denotes the current instruction, s (i) represents the string, and π (i) is the current position): (0) (0) n (0) := 0. s j := x j for 1 ≤ j ≤ size(x), and s j := t for all j ≤ 0 and j > size(x). π (0) := 1. If (n (i) , s (i) , π (i) ) is already defined, we distinguish two cases. If n (i) 6= −1, (i) (i+1) (i+1) (i) := s j then let (m, σ, δ) := 8 n (i) , sπ (i) and set n (i+1) := m, sπ (i) := σ , s j for j ∈ Z \ {π (i) }, and π (i+1) := π (i) + δ. If n (i) = −1, then this is the end of the sequence. We then define time(8, x) := i (i) and output(8, x) ∈ Ak , where k := min j ∈ N : s j = t − 1, by output(8, x) j (i)

:= s j

for j = 1, . . . , k.

15.1 Turing Machines

361

If this sequence is infinite (i.e. n (i) 6= −1 for all i), then we set time(8, x) := ∞. In this case output(8, x) is undefined. Of course we are interested mostly in Turing machines whose computation is finite or even polynomially bounded: Definition 15.3. Let A be an alphabet, S, T ⊆ A∗ two languages, and f : S → T a function. Let 8 be a Turing machine with alphabet A such that time(8, s) < ∞ and output(8, s) = f (s) for each s ∈ S. Then we say that 8 computes f . If there exists a polynomial p such that for all s ∈ S we have time(8, s) ≤ p(size(s)), then 8 is a polynomial-time Turing machine. In the case S = A∗ and T = {0, 1} we say that 8 decides the language L := {s ∈ S : f (s) = 1}. If there exists some polynomial-time Turing machine computing a function f (or deciding a language L), then we say that f is computable in polynomial time (or L is decidable in polynomial time, respectively). To make these definitions clear we give an example. The following Turing machine 8 : {0, . . . , 4}×{0, 1, t} → {−1, . . . , 4}×{0, 1, t}×{−1, 0, 1} computes the successor function n 7→ n + 1 (n ∈ N), where the numbers are coded by their usual binary representation. 8(0, 0) 8(0, 1) 8(0, t) 8(1, 1) 8(1, 0) 8(1, t) 8(2, 0) 8(2, t) 8(3, 0) 8(3, t) 8(4, 0)

= = = = = = = = = = =

(0, 0, 1) (0, 1, 1) (1, t, −1) (1, 0, −1) (−1, 1, 0) (2, t, 1) (2, 0, 1) (3, 0, −1) (3, 0, −1) (4, t, 1) (−1, 1, 0)

0 While sπ 6= t do π := π + 1. Set π := π − 1.

1 While sπ = 1 do sπ := 0 and π := π − 1. If sπ = 0 then sπ := 1 and stop. Set π := π + 1.

2 While sπ = 0 do π := π + 1. Set sπ := 0 and π := π − 1.

3 While sπ = 0 do π := π − 1. Set π := π + 1.

4 Set sπ := 1 and stop.

Note that several values of 8 are not specified as they are never used in any computation. The comments on the right-hand side illustrate the computation. Instructions , 2 3 and 4 are used only if the input consists of 1’s only, i.e. n = 2k − 1 for some k ∈ Z+ . We have time(8, s) ≤ 4 size(s) + 5 for all inputs s, so 8 is a polynomial-time Turing machine. In the next section we shall show that the above definition is consistent with our informal definition of a polynomial-time algorithm in Section 1.2: each polynomialtime algorithm in this book can be simulated by a polynomial-time Turing machine.

362

15 NP-Completeness

15.2 Church’s Thesis The Turing machine is the most customary theoretical model for algorithms. Although it seems to be very restricted, it is as powerful as any other reasonable model: the set of computable functions (sometimes also called recursive functions) is always the same. This statement, known as Church’s thesis, is of course too imprecise to be proved. However, there are strong results supporting this claim. For example, each program in a common programming language like C can be modelled by a Turing machine. In particular, all algorithms in this book can be rewritten as Turing machines. This is usually very inconvenient (thus we shall never do it), but theoretically it is possible. Moreover, any function computable in polynomial time by a C program is also computable in polynomial time by a Turing machine (and vice versa). Since it is not a trivial task to implement more complicated programs on a Turing machine we consider as an intermediate step a Turing machine with two strings (tapes) and two independent read-write heads, one for each tape: Definition 15.4. Let A be an alphabet and A¯ := A ∪ {t}. A two-tape Turing machine is defined by a function 8 : {0, . . . , N } × A¯ 2 → {−1, . . . , N } × A¯ 2 × {−1, 0, 1}2 for some N ∈ Z+ . The computation of 8 on input x, where x ∈ A∗ , is the finite or infinite sequence of 5-tuples (n (i) , s (i) , t (i) , π (i) , ρ (i) ) with n (i) ∈ {−1, . . . , N }, s (i) , t (i) ∈ A¯ Z and π (i) , ρ (i) ∈ Z (i = 0, 1, 2, . . .) defined recursively as follows: (0) (0) n (0) := 0. s j := x j for 1 ≤ j ≤ size(x), and s j := t for all j ≤ 0 and (0)

j > size(x). t j := t for all j ∈ Z. π (0) := 1 and ρ (0) := 1. two cases. If If (n (i) , s (i) , t (i) , π (i) , ρ (i) ) is alreadydefined, we distinguish (i) (i) (i) (i) n 6= −1, then let (m, σ, τ, δ, ) := 8 n , sπ (i) , tρ (i) and set n (i+1) := m, (i+1)

sπ (i)

(i+1)

:= σ , s j

Z \ {ρ (i) },

(i)

:= s j

π (i+1)

π (i)

(i+1)

for j ∈ Z \ {π (i) }, tρ (i) ρ (i+1)

(i+1)

:= τ , t j

(i)

:= t j

for

ρ (i)

:= + δ, and := + . If n (i) = −1, then this is the end of the sequence. time(8, x) and output(8, x) are defined as with the one-tape Turing machine. j∈

Turing machines with more than two tapes can be defined analogously, but we shall not need them. Before we show how to perform standard operations with a two-tape Turing machine, let us note that a two-tape Turing machine can be simulated by an ordinary (one-tape) Turing machine. Theorem 15.5. Let A be an alphabet, and let 8 : {0, . . . , N } × (A ∪ {t})2 → {−1, . . . , N } × (A ∪ {t})2 × {−1, 0, 1}2 be a two-tape Turing machine. Then there exists an alphabet B ⊇ A and a (onetape) Turing machine

15.2 Church’s Thesis

363

80 : {0, . . . , N 0 } × (B ∪ {t}) → {−1, . . . , N 0 } × (B ∪ {t}) × {−1, 0, 1} such that output(80 , x) = output(8, x) and time(80 , x) = O(time(8, x))2 for x ∈ A∗ . Proof: We use the letters s and t for the two strings of 8, and denote by π and ρ the positions of the read-write heads, as in Definition 15.4. The string of 80 will be denoted by u and its read-write head position by ψ. We have to encode both strings s, t and both read-write head positions π, ρ in one string u. To make this possible each symbol u j of u is a 4-tuple (s j , p j , t j , r j ), where s j and t j are the corresponding symbols of s and t, and p j , r j ∈ {0, 1} indicate whether the read-write heads of the first and second string currently scans position j; i.e. we have p j = 1 iff π = j, and r j = 1 iff ρ = j. So we define B¯ := ( A¯ × {0, 1} × A¯ × {0, 1}); then we identify a ∈ A¯ with (a, 0, t, 0) to allow inputs from A∗ . The first step of 80 consists in initializing the marks p1 and r1 to 1: 80 (0, (., 0, ., 0)) = (1, (., 1, ., 1), 0)

0 Set π := ψ and ρ := ψ.

Here a dot stands for an arbitrary value (which is not modified). Now we show how to implement a general instruction 8(m, σ, τ ) = (m 0 , σ 0 , τ 0 , δ, ). We first have to find the positions π and ρ. It is convenient to assume that our single read-write head ψ is already at the leftmost of the two positions π and ρ; i.e. ψ = min{π, ρ}. We have to find the other position by scanning the string u to the right, we have to check whether sπ = σ and tρ = τ and, if so, perform the operation required (write new symbols to s and t, move π and ρ, jump to the next instruction). The following block implements one instruction 8(m, σ, τ ) = (m 0 , σ 0 , τ 0 , δ, ) ¯ 2 such blocks, one for each choice of σ and τ . for m = 0; for each m we have | A| 13 the first block for m 0 with , M where The second block for m = 0 starts with , 2 2 0 0 ¯ ¯ M := 12| A| m + 1. All in all we get N = 12(N + 1)| A| . A dot again stands for an arbitrary value which is not modified. Similarly, ζ and ξ stand for an arbitrary element of A¯ \{σ } and A¯ \{τ }, respectively. We assume 10 11 and 12 guarantee that this property that ψ = min{π, ρ} initially; note that , also holds at the end.

364

15 NP-Completeness

80 (1, (ζ, 1, ., .)) 80 (1, (., ., ξ, 1)) 80 (1, (σ, 1, τ, 1)) 80 (1, (σ, 1, ., 0)) 80 (1, (., 0, τ, 1)) 80 (2, (., ., ., 0)) 80 (2, (., ., ξ, 1))

= = = = = = =

80 (2, (., ., τ, 1)) 80 (3, (., ., ., 0)) 80 (4, (., 0, ., .)) 80 (4, (σ, 1, ., .))

= = = =

80 (5, (., 0, ., .))

= (10, (., 1, ., .), −1)

80 (6, (., 0, ., .)) = 80 (6, (ζ, 1, ., .)) = 80 (6, (σ, 1, ., ., )) 80 (7, (., 0, ., .)) 80 (8, (., ., ., 0)) 80 (8, (., ., τ, 1)) 80 (9, (., ., ., 0)) 80 (10, (., ., ., .)) 80 (11, (., 0, ., 0)) 80 (11, (., 1, ., .)) 80 (11, (., 0, ., 1)) 80 (12, (., 0, ., 0)) 80 (12, (., 1, ., .)) 80 (12, (., ., ., 1))

= = = = = = = = = = = =

13 (13, (ζ, 1, ., .), 0)

1 If ψ = π and sψ 6= σ then go to . 13 (13, (., ., ξ, 1), 0) If ψ = ρ and tψ 6= τ then go to . (2, (σ, 1, τ, 1), 0) If ψ = π then go to . 2 (2, (σ, 1, ., 0), 0) (6, (., 0, τ, 1), 0) If ψ = ρ then go to . 6 (2, (., ., ., 0), 1)

2 While ψ 6= ρ do ψ := ψ + 1. (12, (., ., ξ, 1), −1) If tψ 6= τ then set ψ := ψ − 1 12 and go to . 0 0 (3, (., ., τ , 0), ) Set tψ := τ and ψ := ψ + . (4, (., ., ., 1), 1)

3 Set ρ := ψ and ψ := ψ + 1. (4, (., 0, ., .), −1)

4 While ψ 6= π do ψ := ψ − 1. (5, (σ 0 , 0, ., .), δ) Set sψ := σ 0 and ψ := ψ + δ.

5 Set π := ψ and ψ := ψ − 1. 10 Go to . (6, (., 0, ., .), 1)

6 While ψ 6= π do ψ := ψ + 1. (12, (ζ, 1, ., .), −1) If sψ 6= σ then set ψ := ψ − 1 12 and go to . 0 0 (7, (σ , 0, ., .), δ) Set sψ := σ and ψ := ψ + δ. (8, (., 1, ., .), 1)

7 Set π := ψ and ψ := ψ + 1. (8, (., ., ., 0), −1)

8 While ψ 6= ρ do ψ := ψ − 1. 0 (9, (., ., τ , 0), ) Set tψ := τ 0 and ψ := ψ + . (10, (., ., ., 1), −1) 9 Set ρ := ψ and ψ := ψ − 1. 10 Set ψ := ψ − 1. (11, (., ., ., .), −1) 11 While ψ 6∈ {π, ρ} do ψ := ψ + 1. (11, (., 0, ., 0), 1)

M (M, (., 1, ., .), 0) Go to . (M, (., 0, ., 1), 0) 12 While ψ 6∈ {π, ρ} do ψ := ψ − 1. (12, (., 0, ., 0), −1) (13, (., 1, ., .), 0) (13, (., ., ., 1), 0) 2

¯ blocks like the above for Any computation of 80 passes through at most | A| each computation step of 8. The number of computation steps within each block ¯ is a constant and |π − ρ| is bounded by is at most 2|π − ρ| + 10. Since | A| time(8, x) we conclude that the whole computation of 8 is simulated by 80 with 2 O (time(8, x)) steps. Finally we have to clean up the output: replace each symbol (σ, ., ., .) by (σ, 0, t, 0). Obviously this at most doubles the total number of steps.

15.2 Church’s Thesis

365

With a two-tape Turing machine it is not too difficult to implement more complicated instructions, and thus arbitrary algorithms: We use the alphabet A = {0, 1, #} and model an arbitrary number of variables by the string x0 ##1#x1 ##10#x2 ##11#x3 ##100#x4 ##101#x5 ## . . .

(15.1)

which we store on the first tape. Each group (except the first one) contains a binary representation of the index i followed by the value of xi , which we assume to be a binary string. The first variable x0 and the second tape are used only as registers for intermediate results of computation steps. Random access to variables is not possible in constant time with a Turing machine, no matter how many tapes we have. If we simulate an arbitrary algorithm by a two-tape Turing machine, we will have to scan the first tape quite often. Moreover, if the length of the string in one variable changes, the substring to the right has to be shifted. Nevertheless each standard operation (i.e. each elementary step of an algorithm) can be simulated with O(l 2 ) computation steps of a two-tape Turing machine, where l is the current length of the string (15.1). We try to make this clearer with a concrete example. Consider the following instruction: Add to x5 the value of the variable whose index is given by x2 . To get the value of x5 we scan the first tape for the substring ##101#. We copy the substring following this up to #, exclusively, to the second tape. This is easy since we have two separate read-write heads. Then we copy the string from the second tape to x0 . If the new value of x0 is shorter or longer than the old one, we have to shift the rest of the string (15.1) to the left or to the right appropriately. Next we have to search for the variable index that is given by x2 . To do this, we first copy x2 to the second tape. Then we scan the first tape, checking each variable index (comparing it with the string on the second tape bitwise). When we have found the correct variable index, we copy the value of this variable to the second tape. Now we add the number stored in x0 to that on the second tape. A Turing machine for this task, using the standard method, is not hard to design. We can overwrite the number on the second tape by the result while computing it. Finally we have the result on the second tape and copy it back to x5 . If necessary we shift the substring to the right of x5 appropriately. All the above can be done by a two-tape Turing machine in O(l 2 ) computation steps (in fact all but shifting the string (15.1) can be done in O(l) steps). It should be clear that the same holds for all other standard operations, including multiplication and division. By Definition 1.4 an algorithm is said to run in polynomial time if there is a k ∈ N such that the number of elementary steps is bounded by O(n k ) and any number in intermediate computation can be stored with O(n k ) bits, where n is the input size. Moreover, we store at most O(n k ) numbers at any time. Hence we can bound the length of each of the two strings in a two-tape Turing machine simulating such an algorithm by l = O(n k · n k ) = O(n 2k ), and hence its running time by O(n k (n 2k )2 ) = O(n 5k ). This is still polynomial in the input size.

366

15 NP-Completeness

Recalling Theorem 15.5 we may conclude that for any function f there is a polynomial-time algorithm computing f if and only if there is a polynomial-time Turing machine computing f . Hopcroft and Ullman [1979], Lewis and Papadimitriou [1981], and van Emde Boas [1990] provide more details about the equivalence of different machine models. Another common model (which is close to our informal model of Section 1.2) is the RAM machine (cf. Exercise 3) which allows arithmetic operations on integers in constant time. Other models allow only operations on bits (or integers of fixed length) which is more realistic when dealing with large numbers. Obviously, addition and comparison of natural numbers with n bits can be done with O(n) bit operations. For multiplication (and division) the obvious method takes O(n 2 ) bit operations, but the algorithm of Sch¨onhage and Strassen [1971] needs only O(n log n log log n) bit operations for multiplying two n-bit integers, and this was improved further by F¨urer [2007]. This of course implies algorithms for the addition and comparison of rational numbers with the same time complexity. As far as polynomial-time computability is concerned all models are equivalent, but of course the running time measures are quite different. The model of encoding the entire input by 0-1-strings (or strings over any fixed alphabet) does not in principle exclude certain types of real numbers, e.g. algebraic numbers (if x ∈ R is the k-th smallest root of a polynomial p, then x can be coded by listing k and the degree and the coefficients of p). However, there is no way of representing arbitrary real numbers in a digital computer since there are uncountably many real numbers but only countably many 0-1-strings. We take the classical approach and restrict ourselves to rational input in this chapter. We close this section by giving a formal definition of oracle algorithms, based on two-tape Turing machines. We may call an oracle at any stage of the computation; we use the second tape for writing the oracle’s input and reading its output. We introduce a special instruction −2 for oracle calls: Definition 15.6. Let A be an alphabet and A¯ := A ∪ {t}. Let X ⊆ A∗ , and let f (x) ⊆ A∗ be a nonempty language for each x ∈ X . An oracle Turing machine using f is a function 8 : {0, . . . , N } × A¯ 2 → {−2, . . . , N } × A¯ 2 × {−1, 0, 1}2 for some N ∈ Z+ . Its computation is defined as for a two-tape Turing machine, (i) (i) but with one difference: if, for some computation step i, 8 n (i) , sπ (i) , tρ (i) = (−2, σ, τ, δ, ) forn some σ, τ, δ, , othen consider the string on the second tape (i) (i) x ∈ Ak , k := min j ∈ N : t j = t − 1, given by x j := t j for j = 1, . . . , k. If (i+1)

x ∈ X , then the second tape is overwritten by t j

= y j for j = 1, . . . , size(y)

(i+1) tsize(y)+1

and = t for some y ∈ f (x). The rest remains unchanged, and the computation continues with n (i+1) := n (i) + 1 (and stops if n (i) = −1). All definitions with respect to Turing machines can be extended to oracle Turing machines. The output of an oracle is not necessarily unique; hence there can be

15.3 P and NP

367

several possible computations for the same input. When proving the correctness or estimating the running time of an oracle algorithm we have to consider all possible computations, i.e. all choices of the oracle. By the results of this section the existence of a polynomial-time (oracle) algorithm is equivalent to the existence of a polynomial-time (oracle) Turing machine.

15.3 P and NP Most of complexity theory is based on decision problems. Any language L ⊆ {0, 1}∗ can be interpreted as decision problem: given a 0-1-string, decide whether it belongs to L. However, we are more interested in problems like the following:

H AMILTONIAN C IRCUIT Instance:

An undirected graph G.

Question: Has G a Hamiltonian circuit? We will always assume a fixed efficient encoding of the input as a binary string; occasionally we extend our alphabet by other symbols. For example we assume that a graph is given by an adjacency list, and such a list can be coded as a binary string of length O(n log m + m log n), where n and m denote the number of vertices and edges. We always assume an efficient encoding, i.e. one whose length is polynomially bounded by the minimum possible encoding length. Not all binary strings are instances of H AMILTONIAN C IRCUIT but only those representing an undirected graph. For most interesting decision problems the instances are a proper subset of the 0-1-strings. We require that we can decide in polynomial time whether an arbitrary string is an instance or not: Definition 15.7. A decision problem is a pair P = (X, Y ), where X is a language decidable in polynomial time and Y ⊆ X . The elements of X are called instances of P; the elements of Y are yes-instances, those of X \ Y are no-instances. An algorithm for a decision problem (X, Y ) is an algorithm computing the function f : X → {0, 1}, defined by f (x) = 1 for x ∈ Y and f (x) = 0 for x ∈ X \ Y. We give two more examples, the decision problems corresponding to L INEAR P ROGRAMMING and I NTEGER P ROGRAMMING:

L INEAR I NEQUALITIES Instance:

A matrix A ∈ Zm×n and a vector b ∈ Zm .

Question: Is there a vector x ∈ Qn such that Ax ≤ b?

368

15 NP-Completeness

I NTEGER L INEAR I NEQUALITIES A matrix A ∈ Zm×n and a vector b ∈ Zm .

Instance:

Question: Is there a vector x ∈ Zn such that Ax ≤ b? Definition 15.8. The class of all decision problems for which there is a polynomial-time algorithm is denoted by P. In other words, a member of P is a pair (X, Y ) with Y ⊆ X ⊆ {0, 1}∗ where both X and Y are languages decidable in polynomial time. To prove that a problem is in P one usually describes a polynomial-time algorithm. By the results of Section 15.2 there is a polynomial-time Turing machine for each problem in P. By Khachiyan’s Theorem 4.18, L INEAR I NEQUALITIES belongs to P. It is not known whether I NTEGER L INEAR I NEQUALITIES or H AMILTONIAN C IRCUIT belong to P. We shall now introduce another class, called NP, which contains these problems, and in fact most decision problems discussed in this book. We do not insist on a polynomial-time algorithm, but we require that for each yes-instance there is a certificate which can be checked in polynomial time. For example, for the H AMILTONIAN C IRCUIT problem a possible certificate is simply a Hamiltonian circuit. It is easy to check whether a given string is the binary encoding of a Hamiltonian circuit. Note that we do not require a certificate for no-instances. Formally we define: Definition 15.9. A decision problem P = (X, Y ) belongs to NP if there is a polynomial p and a decision problem P 0 = (X 0 , Y 0 ) in P, where o n X 0 := x#c : x ∈ X, c ∈ {0, 1}b p(size(x))c , such that Y =

n

o y ∈ X : there exists a string c ∈ {0, 1}b p(size(y))c with y#c ∈ Y 0 .

Here x#c denotes the concatenation of the string x, the symbol # and the string c. A string c with y#c ∈ Y 0 is called a certificate for y (since c proves that y ∈ Y ). An algorithm for P 0 is called a certificate-checking algorithm. Proposition 15.10. P ⊆ NP. Proof: One can choose p to be identically zero. An algorithm for P 0 just deletes the last symbol of the input “x#” and then applies an algorithm for P. It is not known whether P = NP. In fact, this is the most important open problem in complexity theory. As an example for problems in NP that are not known to be in P we have: Proposition 15.11. H AMILTONIAN C IRCUIT belongs to NP.

15.3 P and NP

369

Proof: For each yes-instance G we take any Hamiltonian circuit of G as a certificate. To check whether a given edge set is in fact a Hamiltonian circuit of a given graph is obviously possible in polynomial time. Proposition 15.12. I NTEGER L INEAR I NEQUALITIES belongs to NP. Proof: As a certificate we just take a solution vector. If there exists a solution, there exists one of polynomial size by Corollary 5.7. The name NP stands for “nondeterministic polynomial”. To explain this we have to define what a nondeterministic algorithm is. This is a good opportunity to define randomized algorithms in general, a concept which has already been mentioned before. The common feature of randomized algorithms is that their computation does not only depend on the input but also on some random bits. Definition 15.13. A randomized algorithm for computing a function f : S → T is an algorithm computing a function g : {s#r : s ∈ S, r ∈ {0, 1}k(s) } → T , where k : S → Z+ . So for each instance s ∈ S the algorithm can use k(s) ∈ Z+ random bits. We measure the running time dependency on size(s) only; thus randomized algorithms running in polynomial time can read only a polynomial number of random bits. Naturally we are interested in such a randomized algorithm only if f and g are related. In the ideal case, if g(s#r ) = f (s) for all s ∈ S and all r ∈ {0, 1}k(s) , we speak of a Las Vegas algorithm. A Las Vegas algorithm always computes the correct result, but the running time may vary for different runs on the same input s. Sometimes even less reliable algorithms are interesting: if there is at least a positive probability p of a correct answer, independent of the instance, i.e. p := inf

s∈S

|{r ∈ {0, 1}k(s) : g(s#r ) = f (s)}| > 0, 2k(s)

then we have a Monte Carlo algorithm. If T = {0, 1}, and for each s ∈ S with f (s) = 0 we have g(s#r ) = 0 for all r ∈ {0, 1}k(s) , then we have a randomized algorithm with one-sided error. If in addition for each s ∈ S with f (s) = 1 there is at least one r ∈ {0, 1}k(s) with g(s#r ) = 1, then the algorithm is called a nondeterministic algorithm. Alternatively a randomized algorithm can be regarded as an oracle algorithm where the oracle produces a random bit (0 or 1) whenever called. A nondeterministic algorithm for a decision problem always answers “no” for a no-instance, and for each yes-instance there is a chance that it answers “yes”. The following observation is easy: Proposition 15.14. A decision problem belongs to NP if and only if it has a polynomial-time nondeterministic algorithm.

370

15 NP-Completeness

Proof: Let P = (X, Y ) be a decision problem in NP, and let P 0 = (X 0 , Y 0 ) be defined as in Definition 15.9. Then a polynomial-time algorithm for P 0 is in fact also a nondeterministic algorithm for P: the unknown certificate is simply replaced by random bits. Since the number of random bits is bounded by a polynomial in size(x), x ∈ X , so is the running time of the algorithm. Conversely, if P = (X, Y ) has a polynomial-time nondeterministic algorithm using k(x) random bits for instance x, then thereis a polynomial p such that k(x) ≤ p(size(x)) for each instance x. We define X 0 := x#c : x ∈ X, c ∈ {0, 1}b p(size(x))c and Y 0 := x#c ∈ X 0 : g(x#r ) = 1, r consists of the first k(x) bits of c . Then by the definition of nondeterministic algorithms we have (X 0 , Y 0 ) ∈ P and o n Y = y ∈ X : there exists a string c ∈ {0, 1}b p(size(x))c with y#c ∈ Y 0 . Most decision problems encountered in combinatorial optimization belong to NP. For many of them it is not known whether they have a polynomial-time algorithm. However, one can say that certain problems are not easier than others. To make this precise we introduce the important concept of polynomial reductions. Definition 15.15. Let P1 and P2 = (X, Y ) be decision problems. Let f : X → {0, 1} with f (x) = 1 for x ∈ Y and f (x) = 0 for x ∈ X \ Y . We say that P1 polynomially reduces to P2 if there exists a polynomial-time oracle algorithm for P1 using f . The following observation is the main reason for this concept: Proposition 15.16. If P1 polynomially reduces to P2 and there is a polynomialtime algorithm for P2 , then there is a polynomial-time algorithm for P1 . Proof: Let A2 be an algorithm for P2 with time(A2 , y) ≤ p2 (size(y)) for all instances y of P2 , and let f (y) := output(A2 , y). Let A1 be an oracle algorithm for P1 using f with time(A1 , x) ≤ p1 (size(x)) for all instances x of P1 . Then replacing the oracle calls in A1 by subroutines equivalent to A2 yields an algorithm A3 for P1 . For any instance x of P1 with size(x) = n we have time(A3 , x) ≤ p1 (n)· p2 ( p1 (n)): there can be at most p1 (n) oracle calls in A1 , and none of the instances of P2 produced by A1 can be longer than p1 (n). Since we can choose p1 and p2 to be polynomials we conclude that A3 is a polynomial-time algorithm. The theory of NP-completeness is based on a special kind of polynomial-time reduction: Definition 15.17. Let P1 = (X 1 , Y1 ) and P2 = (X 2 , Y2 ) be decision problems. We say that P1 polynomially transforms to P2 if there is a function f : X 1 → X 2 computable in polynomial time such that f (x1 ) ∈ Y2 for all x1 ∈ Y1 and f (x1 ) ∈ X 2 \ Y2 for all x1 ∈ X 1 \ Y1 .

15.4 Cook’s Theorem

371

In other words, yes-instances are transformed to yes-instances, and no-instances are transformed to no-instances. Obviously, if a problem P1 polynomially transforms to P2 , then P1 also polynomially reduces to P2 . Polynomial transformations are sometimes called Karp reductions, while general polynomial reductions are also known as Turing reductions. Both are easily seen to be transitive. Definition 15.18. A decision problem P ∈ NP is called NP-complete if all other problems in NP polynomially transform to P. By Proposition 15.16 we know that if there is a polynomial-time algorithm for any NP-complete problem, then P = NP. Of course, the above definition would be meaningless if no NP-complete problems existed. The next section consists of a proof that there is an NP-complete problem.

15.4 Cook’s Theorem In his pioneering work, Cook [1971] proved that a certain decision problem, called S ATISFIABILITY, is in fact NP-complete. We need some definitions: Definition 15.19. Let X be a finite set of Boolean variables. A truth assignment . for X is a function T : X → {true, false}. We extend T to the set L := X ∪ {x : x ∈ X } by setting T (x) := true if T (x) := false and vice versa (x can be regarded as the negation of x). The elements of L are called the literals over X . A clause over X is a set of literals over X . A clause represents the disjunction of those literals and is satisfied by a truth assignment iff at least one of its members is true. A family of clauses over X is satisfiable iff there is some truth assignment simultaneously satisfying all of its clauses. Since we consider the conjunction of disjunctions of literals, we also speak of Boolean formulas in conjunctive normal form. For example, the family {{x1 , x2 }, {x2 , x3 }, {x1 , x2 , x3 }, {x1 , x3 }} corresponds to the Boolean formula (x1 ∨ x2 ) ∧ (x2 ∨ x3 ) ∧ (x1 ∨ x2 ∨ x3 ) ∧ (x1 ∨ x3 ). It is satisfiable as the truth assignment T (x1 ) := true, T (x2 ) := false and T (x3 ) := true shows. We are now ready to specify the satisfiability problem:

S ATISFIABILITY Instance:

A set X of variables and a family Z of clauses over X .

Question: Is Z satisfiable? Theorem 15.20. (Cook [1971]) S ATISFIABILITY is NP-complete. Proof: S ATISFIABILITY belongs to NP because a satisfying truth assignment serves as a certificate for any yes-instance, which of course can be checked in polynomial time.

372

15 NP-Completeness

Let now P = (X, Y ) be any other problem in NP. We have to show that P polynomially transforms to S ATISFIABILITY. By Definition 15.9 and a decision problem P 0 = (X 0 , Y 0 ) there is a polynomial pb p(size(x))c 0 in P, where X := x#c : x ∈ X, c ∈ {0, 1} and o n Y = y ∈ X : there exists a string c ∈ {0, 1}b p(size(x))c with y#c ∈ Y 0 . Let 8 : {0, . . . , N } × A¯ → {−1, . . . , N } × A¯ × {−1, 0, 1} be a polynomial-time Turing machine for P 0 with alphabet A; let A¯ := A ∪ {t}. Let q be a polynomial such that time(8, x#c) ≤ q(size(x#c)) for all instances x#c ∈ X 0 . Note that size(x#c) = size(x) + 1 + b p(size(x))c. We will now construct a collection Z(x) of clauses over some set V (x) of Boolean variables for each x ∈ X , such that Z(x) is satisfiable if and only if x ∈ Y. We abbreviate Q := q(size(x) + 1 + b p(size(x))c). Q is an upper bound on the length of any computation of 8 on input x#c, for any c ∈ {0, 1}b p(size(x))c . V (x) contains the following Boolean variables: • •

¯ a variable vi jσ for all 0 ≤ i ≤ Q, −Q ≤ j ≤ Q and σ ∈ A; a variable wi jn for all 0 ≤ i ≤ Q, −Q ≤ j ≤ Q and −1 ≤ n ≤ N .

The intended meaning is: vi jσ indicates whether at time i (i.e. after i steps of the computation) the j-th position of the string contains the symbol σ . wi jn indicates whether at time i the j-th position of the string is scanned and the n-th instruction is executed. So if (n (i) , s (i) , π (i) )i=0,1,... is a computation of 8 then we intend to set vi jσ (i) to true iff s j = σ and wi jn to true iff π (i) = j and n (i) = n. The collection Z(x) of clauses to be constructed will be satisfiable if and only if there is a string c with output(8, x#c) = 1. Z(x) contains the following clauses to model the following conditions: At any time each position of the string contains a unique symbol: ¯ • {vi jσ : σ ∈ A} for 0 ≤ i ≤ Q and −Q ≤ j ≤ Q; • {vi jσ , vi jτ } for 0 ≤ i ≤ Q, −Q ≤ j ≤ Q and σ, τ ∈ A¯ with σ 6= τ . At any time a unique position of the string is scanned and a single instruction is executed: • {wi jn : −Q ≤ j ≤ Q, −1 ≤ n ≤ N } for 0 ≤ i ≤ Q; • {wi jn , wi j 0 n 0 } for 0 ≤ i ≤ Q, −Q ≤ j, j 0 ≤ Q and −1 ≤ n, n 0 ≤ N with ( j, n) 6= ( j 0 , n 0 ). The algorithm starts correctly with input x#c for some c ∈ {0, 1}b p(size(x))c : • {v0, j,x j } for 1 ≤ j ≤ size(x); • {v0,size(x)+1,# };

15.4 Cook’s Theorem

373

• {v0,size(x)+1+ j,0 , v0,size(x)+1+ j,1 } for 1 ≤ j ≤ b p(size(x))c; • {v0, j,t } for −Q ≤ j ≤ 0 and size(x) + 2 + b p(size(x))c ≤ j ≤ Q; • {w010 }. The algorithm works correctly: •

{vi jσ , wi jn , vi+1, j,τ }, {vi jσ , wi jn , wi+1, j+δ,m } for 0 ≤ i < Q, −Q ≤ j ≤ Q, σ ∈ A¯ and 0 ≤ n ≤ N , where 8(n, σ ) = (m, τ, δ). When the algorithm reaches instruction −1, it stops:

•

{wi, j,−1 , wi+1, j,−1 }, {wi, j,−1 , vi, j,σ , vi+1, j,σ } ¯ for 0 ≤ i < Q, −Q ≤ j ≤ Q and σ ∈ A. Positions not being scanned remain unchanged:

•

¯ −1 ≤ n ≤ N and {vi jσ , wi j 0 n , vi+1, j,σ } for 0 ≤ i ≤ Q, σ ∈ A, 0 0 −Q ≤ j, j ≤ Q with j 6= j . The output of the algorithm is 1:

•

{v Q,1,1 }, {v Q,2,t }.

The encoding length of Z(x) is O(Q 3 log Q): there are O(Q 3 ) occurrences of literals, whose indices require O(log Q) space. Since Q depends polynomially on size(x) we conclude that there is a polynomial-time algorithm which, given x, constructs Z(x). Note that p, 8 and q are fixed and not part of the input of this algorithm. It remains to show that Z(x) is satisfiable if and only if x ∈ Y . If Z(x) is satisfiable, consider a truth assignment T satisfying all clauses. Let c ∈ {0, 1}b p(size(x))c with c j = 1 for all j with T (v0,size(x)+1+ j,1 ) = true and c j = 0 otherwise. By the above construction the variables reflect the computation of 8 on input x#c. Hence we may conclude that output(8, x#c) = 1. Since 8 is a certificate-checking algorithm, this implies that x is a yes-instance. Conversely, if x ∈ Y , let c be any certificate for x. Let (n (i) , s (i) , π (i) )i=0,1,...,m (i) be the computation of 8 on input x#c. Then we define T (vi, j,σ ) := true iff s j = σ and T (wi, j,n ) = true iff π (i) = j and n (i) = n. For i := m + 1, . . . , Q we set T (vi, j,σ ) := T (vi−1, j,σ ) and T (wi, j,n ) := T (wi−1, j,n ) for all j, n and σ . Then T is a truth assignment satisfying Z(x), completing the proof. S ATISFIABILITY is not the only NP-complete problem; we will encounter many others in this book. Now that we already have one NP-complete problem at hand, it is much easier to prove NP-completeness for another problem. To show that a certain decision problem P is NP-complete, we shall just prove that P ∈ NP and that S ATISFIABILITY (or any other problem which we know already to be NP-complete) polynomially transforms to P. Since polynomial transformability is transitive, this will be sufficient. The following restriction of S ATISFIABILITY will prove very useful for several NP-completeness proofs:

374

15 NP-Completeness

3S AT A set X of variables and a collection Z of clauses over X , each containing exactly three literals. Question: Is Z satisfiable?

Instance:

To show NP-completeness of 3S AT we observe that any clause can be replaced equivalently by a set of 3S AT-clauses: Proposition 15.21. Let X be a set of variables and Z a clause over X with k literals. Then there is a set Y of at most max{k .− 3, 2} new variables and a family Z 0 of at most max{k − 2, 4} clauses over X ∪ Y such that each element of Z 0 has exactly three literals, and for each family W of clauses over X we have that W ∪ {Z } is satisfiable if and only if W ∪ Z 0 is satisfiable. Moreover, such a family Z 0 can be computed in O(k) time. Proof: If Z has three literals, we set Z 0 := {Z }. If Z has more than three literals, say Z = {λ1 , . . . , λk }, we choose a set Y = {y1 , . . . , yk−3 } of k − 3 new variables and set Z 0 := {λ1 , λ2 , y1 }{y1 , λ3 , y2 }, {y2 , λ4 , y3 }, . . . , {yk−4 , λk−2 , yk−3 }, {yk−3 , λk−1 , λk } . If Z = {λ1 , λ2 }, we choose a new variable y1 (Y := {y1 }) and set Z 0 := {{λ1 , λ2 , y1 }, {λ1 , λ2 , y1 }} . If Z = {λ1 }, we choose a set Y = {y1 , y2 } of two new variables and set Z 0 := {{λ1 , y1 , y2 }, {λ1 , y1 , y2 }, {λ1 , y1 , y2 }, {λ1 , y1 , y2 }}. Observe that in each case Z can be equivalently replaced by the clauses in Z 0 in any instance of S ATISFIABILITY. Theorem 15.22. (Cook [1971]) 3S AT is NP-complete. Proof: As a restriction of S ATISFIABILITY, 3S AT is certainly in NP. We now show that S ATISFIABILITY polynomially transforms to 3S AT. Consider any collection Z of clauses Z 1 , . . . , Z m . We shall construct a new collection Z 0 of clauses with three literals per clause such that Z is satisfiable if and only if Z 0 is satisfiable. To do this, we replace each clause Z i by an equivalent set of clauses, each with three literals. This is possible in linear time by Proposition 15.21. If we restrict each clause to consist of just two literals, the problem (called 2S AT) can be solved in linear time (Exercise 7).

15.5 Some Basic NP-Complete Problems

375

15.5 Some Basic NP-Complete Problems Karp [1972] discovered the wealth of consequences of Cook’s work for combinatorial optimization problems. As a start, we consider the following problem:

S TABLE S ET Instance:

A graph G and an integer k.

Question: Is there a stable set of k vertices? Theorem 15.23. (Karp [1972]) S TABLE S ET is NP-complete. Proof: Obviously, S TABLE S ET ∈ NP. We show that S ATISFIABILITY polynomially transforms to S TABLE S ET. Let Z be a collection of clauses Z 1 , . . . , Z m with Z i = {λi1 , . . . , λiki } (i = 1, . . . , m), where the λi j are literals over some set X of variables. We shall construct a graph G such that G has a stable set of size m if and only if there is a truth assignment satisfying all m clauses. x1

x3

x1

x1

x2

x2

x3

x3

x2

x3

Fig. 15.1.

For each clause Z i , we introduce a clique of ki vertices according to the literals in this clause. Vertices corresponding to different clauses are connected by an edge if and only if the literals contradict each other. Formally, let V (G) := {v i j : 1 ≤ i ≤ m, 1 ≤ j ≤ ki } and E(G) := {v i j , v kl } : (i = k and j = l) or (λi j = x and λkl = x for some x ∈ X ) . See Figure 15.1 for an example (m = 4, Z 1 = {x1 , x2 , x3 }, Z 2 = {x1 , x3 }, Z 3 = {x2 , x3 } and Z 4 = {x1 , x2 , x3 }).

376

15 NP-Completeness

Suppose G has a stable set of size m. Then its vertices specify pairwise compatible literals belonging to different clauses. Setting each of these literals to be true (and setting variables not occurring there arbitrarily) we obtain a truth assignment satisfying all m clauses. Conversely, if some truth assignment satisfies all m clauses, then we choose a literal which is true out of each clause. The set of corresponding vertices then defines a stable set of size m in G. It is essential that k is part of the input: for each fixed k it can be decided in O(n k ) time whether a given graph with n vertices has a stable set of size k (simply by testing all vertex sets with k elements). Two interesting related problems are the following:

V ERTEX C OVER Instance:

A graph G and an integer k.

Question: Is there a vertex cover of cardinality k?

C LIQUE Instance:

A graph G and an integer k.

Question: Has G a clique of cardinality k? Corollary 15.24. (Karp [1972]) V ERTEX C OVER and C LIQUE are NP-complete. Proof: By Proposition 2.2, S TABLE S ET polynomially transforms to both V ER TEX C OVER and C LIQUE . We now turn to the famous Hamiltonian circuit problem (already defined in Section 15.3). Theorem 15.25. (Karp [1972]) H AMILTONIAN C IRCUIT is NP-complete. Proof: Membership in NP is obvious. We prove that 3S AT polynomially transforms to H AMILTONIAN C IRCUIT. Given a collection Z of clauses Z 1 , . . . , Z m over X = {x1 , . . . , xn }, each clause containing three literals, we shall construct a graph G such that G is Hamiltonian iff Z is satisfiable. We first define two gadgets which will appear several times in G. Consider the graph shown in Figure 15.2(a), which we call A. We assume that it is a subgraph of G and no vertex of A except u, u 0 , v, v 0 is incident to any other edge of G. Then any Hamiltonian circuit of G must traverse A in one of the ways shown in Figure 15.3(a) and (b). So we can replace A by two edges with the additional restriction that any Hamiltonian circuit of G must contain exactly one of them (Figure 15.2(b)). Now consider the graph B shown in Figure 15.4(a). We assume that it is a subgraph of G, and no vertex of B except u and u 0 is incident to any other edge

15.5 Some Basic NP-Complete Problems (a)

377

(b)

u

u

u

u

A

v

v

v

v

Fig. 15.2. (a)

(b)

u

u

u

u

v

v

v

v

Fig. 15.3. (a)

(b) u

u

e1

e2

B

e3 u

u

Fig. 15.4.

of G. Then no Hamiltonian circuit of G traverses all of e1 , e2 , e3 . Moreover, one easily checks that for any S ⊂ {e1 , e2 , e3 } there is a Hamiltonian path from u to u in B that contains S but none of {e1 , e2 , e3 } \ S. We represent B by the symbol shown in Figure 15.4(b).

378

15 NP-Completeness

B

A

A A A B

A A A A

B

A

Fig. 15.5.

We are now able to construct G. For each clause, we introduce a copy of B, joined one after another. Between the ﬁrst and the last copy of B, we insert two vertices for each variable, all joined one after another. We then double the edges between the two vertices of each variable x; these two edges will correspond to x and x, respectively. The edges e1 , e2 , and e3 in each copy of B are now connected via a copy of A to the edges corresponding to the ﬁrst, second, and third literal of the corresponding clause, respectively. These constructions are done consecutively: when introducing a copy of subgraph A at an edge e = {u, v} corresponding to a literal, the edge incident to u in Figure 15.2(a) takes the role of e: it is now the edge corresponding to that literal. The overall construction is illustrated by Figure 15.5 with the example {{x1 , x2 , x3 }, {x1 , x2 , x3 }, {x1 , x2 , x3 }}. Now we claim that G is Hamiltonian if and only if Z is satisﬁable. Let C be a Hamiltonian circuit. We deﬁne a truth assignment by setting a literal true iff C

15.5 Some Basic NP-Complete Problems

379

contains the corresponding edge. By the properties of the gadgets A and B each clause contains a literal that is true. Conversely, any satisfying truth assignment defines a set of edges corresponding to literals that are true. Since each clause contains a literal that is true this set of edges can be completed to a tour in G. This proof is essentially due to Papadimitriou and Steiglitz [1982]. The problem of deciding whether a given graph contains a Hamiltonian path is also NP-complete (Exercise 14(a)). Moreover, one can easily transform the undirected versions to the directed Hamiltonian circuit or Hamiltonian path problem by replacing each undirected edge by a pair of oppositely directed edges. Thus the directed versions are also NP-complete. There is another fundamental NP-complete problem:

3-D IMENSIONAL M ATCHING (3DM) Disjoint sets U, V, W of equal cardinality and T ⊆ U × V × W .

Instance:

Question: Is there a subset M of T with |M| = |U | such that for distinct (u, v, w), (u 0 , v 0 , w0 ) ∈ M one has u 6= u 0 , v 6= v 0 and w 6= w 0 ? Theorem 15.26. (Karp [1972]) 3DM is NP-complete. Proof: Membership in NP is obvious. We shall polynomially transform S ATISFIABILITY to 3DM. Given a collection Z of clauses Z 1 , . . . , Z m over X = {x1 , . . . , xn }, we construct an instance (U, V, W, T ) of 3DM which is a yesinstance if and only if Z is satisfiable. We define: j

U

:= {xi , xi j : i = 1, . . . , n; j = 1, . . . , m}

V

:= {ai : i = 1, . . . , n; j = 1, . . . , m} ∪ {v j : j = 1, . . . , m}

j

j

∪ {ck : k = 1, . . . , n − 1; j = 1, . . . , m} W

j

:= {bi : i = 1, . . . , n; j = 1, . . . , m} ∪ {w j : j = 1, . . . , m} j

∪ {dk : k = 1, . . . , n − 1; j = 1, . . . , m} j

j

j

j+1

T1 := {(xi , ai , bi ), (xi j , ai

j

, bi ) : i = 1, . . . , n; j = 1, . . . , m},

where aim+1 := ai1 j

T2 := {(xi , v j , w j ) : i = 1, . . . , n; j = 1, . . . , m; xi ∈ Z j } ∪ {(xi j , v j , w j ) : i = 1, . . . , n; j = 1, . . . , m; xi ∈ Z j } j

j

j

j

j

T3 := {(xi , ck , dk ), (xi j , ck , dk ) : i = 1, . . . , n; j = 1, . . . , m; k = 1, . . . , n −1} T := T1 ∪ T2 ∪ T3 . For an illustration of this construction, see Figure 15.6. Here m = 2, Z 1 = {x1 , x2 }, Z 2 = {x1 , x2 }. Each triangle corresponds to an element of T1 ∪ T2 . The j j elements ck , dk and the triples in T3 are not shown.

380

15 NP-Completeness w1

v1

x1 1 a12

x2 1 b11

x12

a22 x11

b12

b21

x22

a11

x21 b22

x1 2

a21 x2 2

w2

v2

Fig. 15.6.

Suppose (U, V, W, T ) is a yes-instance, so let M ⊆ T be a solution. Since the j j ai ’s and bi ’s appear only in elements of T1 , for each i we have either M ∩ T1 ⊇ j j j j+1 j {(xi , ai , bi ) : j = 1, . . . , m} or M ∩ T1 ⊇ {(xi j , ai , bi ) : j = 1, . . . , m}. In the ﬁrst case we set xi to false, in the second case to true. Furthermore, for each clause Z j we have (λ j , v j , w j ) ∈ M for some literal λ ∈ Z j . Since λ j does not appear in any element of M ∩ T1 this literal is true; hence we have a satisfying truth assignment. Conversely, a satisfying truth assignment suggests a set M1 ⊆ T1 of cardinality nm and a set M2 ⊆ T2 of cardinality m such that for distinct (u, v, w), (u , v , w ) ∈ M1 ∪ M2 we have u = u , v = v and w = w . It is easy to complete M1 ∪ M2 by (n − 1)m elements of T3 to a solution of the 3DM instance. A problem which looks simple but is not known to be solvable in polynomial time is the following:

S UBSET-S UM Instance:

Natural numbers c1 , . . . , cn , K .

Question: Is there a subset S ⊆ {1, . . . , n} such that

j∈S c j

=K ?

Corollary 15.27. (Karp [1972]) S UBSET-S UM is NP-complete. Proof: It is obvious that S UBSET-S UM is in NP. We prove that 3DM polynomially transforms to S UBSET-S UM. So let (U, V, W, T ) be an instance of 3DM. W.l.o.g. let U ∪ V ∪ W = {u 1 , . . . , u 3m }. We write S := {{a, b, c} : (a, b, c) ∈ T } and S = {s1 , . . . , sn }.

15.5 Some Basic NP-Complete Problems

381

Define cj

:=

X

(n + 1)i−1

( j = 1, . . . , n), and

u i ∈s j

K

:=

3m X

(n + 1)i−1 .

i=1

Written in (n + 1)-ary form, the number c j can be regarded as the incidence vector of s j ( j = 1, . . . , n), and K consists of 1’s only. Therefore P each solution to the 3DM instance corresponds to a subset R of S such that s j ∈R c j = K , and vice versa. Moreover, size(c j ) ≤ size(K ) = O(m log n), so the above is indeed a polynomial transformation. An important special case is the following problem:

PARTITION Instance:

Natural numbers c1 , . . . , cn .

Question: Is there a subset S ⊆ {1, . . . , n} such that

P

j∈S c j

=

P

j ∈S / cj

?

Corollary 15.28. (Karp [1972]) PARTITION is NP-complete. Proof: We show that S UBSET-S UM polynomially transforms to PARTITION. So be an instance of S UBSET-S UM. We add an element cn+1 := let P c1 , . . . , c n , K n ci − 2K (unless this number is zero) and have an instance c1 , . . . , cn+1 of i=1 PARTITION. Pn Case 1: 2K ≤ i=1 ci . Then for any I ⊆ {1, . . . , n} we have X X X ci = K if and only if ci = ci . i∈I

Case 2:

i∈I ∪{n+1}

i∈{1,...,n}\I

Pn

2K > i=1 ci . Then for any I ⊆ {1, . . . , n} we have X X X ci = K if and only if ci = ci . i∈I

i∈I

i∈{1,...,n+1}\I

In both cases we have constructed a yes-instance of PARTITION if and only if the original instance of S UBSET-S UM is a yes-instance. We finally note: Theorem 15.29. I NTEGER L INEAR I NEQUALITIES is NP-complete. Proof: We already mentioned the membership in NP in Proposition 15.12. Any of the above problems can easily be formulated as an instance of I NTEGER L INEAR I NEQUALITIES. For example a PARTITION instance c1 , . . . , cn is a yes-instance if and only if {x ∈ Zn : 0 ≤ x ≤ 1l, 2c> x = c> 1l} is nonempty.

382

15 NP-Completeness

15.6 The Class coNP The definition of NP is not symmetric with respect to yes-instances and no-instances. For example, it is an open question whether the following problem belongs to NP: given a graph G, is it true that G is not Hamiltonian? We introduce the following definitions: Definition 15.30. For a decision problem P = (X, Y ) we define its complement to be the decision problem (X, X \Y ). The class coNP consists of all problems whose complements are in NP. A decision problem P ∈ coNP is called coNP-complete if all other problems in coNP polynomially transform to P. Trivially, the complement of a problem in P is also in P. On the other hand, NP 6= coNP is commonly conjectured (though not proved). For this conjecture the NP-complete problems play a special role: Theorem 15.31. A decision problem is coNP-complete if and only if its complement is NP-complete. Unless NP = coNP, no coNP-complete problem is in NP. Proof: The first statement follows directly from the definition. Suppose P = (X, Y ) ∈ NP is a coNP-complete problem. Let Q = (V, W ) be an arbitrary problem in coNP. We show that Q ∈ NP. Since P is coNP-complete, Q polynomially transforms to P. So there is a polynomial-time algorithm which transforms any instance v of Q to an instance x = f (v) of P such that x ∈ Y if and only if v ∈ W . Note that size(x) ≤ p(size(v)) for some fixed polynomial p. Since P ∈ NP, there exists a polynomial q and a decision problem P 0 = (X 0 , Y 0 ) 0 bq(size(x))c , such that in P, where X := x#c : x ∈ X, c ∈ {0, 1} o n Y = y ∈ X : there exists a string c ∈ {0, 1}bq(size(y))c with y#c ∈ Y 0 (cf. Definition 15.9). We define a decision problem (V 0 , W 0 ) by V 0 := v#c : v ∈ V, c ∈ {0, 1}bq( p(size(v)))c , and v#c ∈ W 0 if and only if f (v)#c0 ∈ Y 0 where c0 consists of the first bq(size( f (v)))c components of c. Observe that (V 0 , W 0 ) ∈ P. Therefore, by definition, Q ∈ NP. We conclude coNP ⊆ NP and hence, by symmetry, NP = coNP. If one can show that a problem is in NP ∩ coNP, we say that the problem has a good characterization (Edmonds [1965]). This means that for yes-instances as well as for no-instances there are certificates that can be checked in polynomial time. Theorem 15.31 indicates that a problem with a good characterization is probably not NP-complete. To give examples, Proposition 2.9, Theorem 2.24, and Proposition 2.27 provide good characterizations for the problems of deciding whether a given graph is acyclic, whether it has an Eulerian walk, and whether it is bipartite, respectively. Of course, this is not very interesting since all these problems can be solved easily in polynomial time. But consider the decision version of L INEAR P ROGRAMMING:

15.6 The Class coNP

383

Theorem 15.32. L INEAR I NEQUALITIES is in NP ∩ coNP. Proof: This immediately follows from Theorem 4.4 and Corollary 3.24.

Of course, this theorem also follows from any polynomial-time algorithm for L INEAR P ROGRAMMING, e.g. Theorem 4.18. However, before the E LLIPSOID M ETHOD had been discovered, Theorem 15.32 was the only theoretical evidence that L INEAR I NEQUALITIES is probably not NP-complete. This gave hope to ﬁnd a polynomial-time algorithm for L INEAR P ROGRAMMING (which can be reduced to L INEAR I NEQUALITIES by Proposition 4.16); a justiﬁed hope as we know today. The following famous problem has a similar history:

P RIME Instance:

A number n ∈ N (in its binary representation).

Question: Is n a prime? It is obvious that P RIME belongs to coNP. Pratt [1975] proved that P RIME also belongs to NP. Finally, Agrawal, Kayal and Saxena [2004] proved that P RIME ∈ P by ﬁnding a surprisingly simple O(log7.5+ n)-algorithm (for any > 0). Before, the best known deterministic algorithm for P RIME was due to Adleman, Pomerance and Rumely [1983], running in O (log n)c log log log n time for some constant c. Since the input size is O(log n), this is not polynomial.

NP-complete

coNP-complete

NP ∩ coNP NP

coNP

P

Fig. 15.7.

We close this section by sketching the inclusions of NP and coNP (Figure 15.7). Ladner [1975] showed that, unless P = NP, there are problems in NP \ P that are not NP-complete. However, until the P = NP conjecture is resolved, it is still possible that all regions drawn in Figure 15.7 collapse to one.

384

15 NP-Completeness

15.7 NP-Hard Problems Now we extend our results to optimization problems. We start by formally defining the type of optimization problems we are interested in: Definition 15.33. A (discrete) optimization problem is a quadruple P = (X, (Sx )x∈X , c, goal), where X is a language over {0, 1} decidable in polynomial time; Sx is a subset of {0, 1}∗ for each x ∈ X ; there exists a polynomial p with size(y) ≤ p(size(x)) for all y ∈ Sx and all x ∈ X , and the languages {(x, y) : x ∈ X, y ∈ Sx } and {x ∈ X : Sx = ∅} are decidable in polynomial time; • c : {(x, y) : x ∈ X, y ∈ Sx } → Q is a function computable in polynomial time; and • goal ∈ {max, min}. • •

The elements of X are called instances of P. For each instance x, the elements of Sx are called feasible solutions of x. We write OPT(x) := goal{c(x, y) : y ∈ Sx }. An optimum solution of x is a feasible solution y of x with c(x, y) = OPT(x). An algorithm for an optimization problem (X, (Sx )x∈X , c, goal) is an algorithm A which computes for each input x ∈ X with Sx 6= ∅ a feasible solution y ∈ Sx . We sometimes write A(x) := c(x, y). If A(x) = OPT(x) for all x ∈ X with Sx 6= ∅, then A is an exact algorithm. Depending on the context, c(x, y) is often called the cost, the weight, the profit or the length of y. If c is nonnegative, then we say that the optimization problem has nonnegative weights. The values of c are rational numbers; we assume an encoding into binary strings as usual. The concept of polynomial reductions easily extends to optimization problems: a (decision or optimization) problem polynomially reduces to an optimization problem P = (X, (Sx )x∈X , c, goal) if it has an exact polynomial-time oracle algorithm using the function f defined by f (x) = {y ∈ Sx : c(x, y) = OPT(x)} for all x ∈ X with Sx 6= ∅. Now we can define: Definition 15.34. An optimization problem or decision problem P is called NPhard if all problems in NP polynomially reduce to P. Note that the definition is symmetric: a decision problem is NP-hard if and only if its complement is. NP-hard problems are at least as hard as the hardest problems in NP. But some may be harder than any problem in NP. A problem which polynomially reduces to some problem in NP is called NP-easy. A problem which is both NP-hard and NP-easy is NP-equivalent. In other words, a problem is NPequivalent if and only if it is polynomially equivalent to S ATISFIABILITY, where two problems P and Q are called polynomially equivalent if P polynomially reduces to Q, and Q polynomially reduces to P. We note: Proposition 15.35. Let P be an NP-equivalent problem. Then P has an exact polynomial-time algorithm if and only if P = NP.

15.7 NP-Hard Problems

385

Of course, all NP-complete problems and all coNP-complete problems are NP-equivalent. Almost all problems discussed in this book are NP-easy since they polynomially reduce to I NTEGER P ROGRAMMING; this is usually a trivial observation which we do not even mention. On the other hand, most problems we discuss from now on are also NP-hard, and we shall usually prove this by describing a polynomial reduction from an NP-complete problem. As a first example we consider M AX -2S AT: given an instance of S ATISFIABILITY with exactly two literals per clause, find a truth assignment that maximizes the number of satisfied clauses. Theorem 15.36. (Garey, Johnson and Stockmeyer [1976]) M AX -2S AT is NPhard. Proof: By reduction from 3S AT. Given an instance I of 3S AT with clauses C1 , . . . , Cm , we construct an instance I 0 of M AX -2S AT by adding new variables y1 , z 1 , . . . , ym , z m and replacing each clause Ci = {λ1 , λ2 , λ3 } by the fourteen clauses {λ1 , z i }, {λ1 , z¯ i }, {λ2 , z i }, {λ2 , z¯ i }, {λ3 , z i }, {λ3 , z¯ i }, {yi , z i }, {yi , z¯ i }, {λ1 , y¯i }, {λ2 , y¯i }, {λ3 , y¯i }, {λ¯ 1 , λ¯ 2 }, {λ¯ 1 , λ¯ 3 }, {λ¯ 2 , λ¯ 3 }. Note that no truth assignment satisfies more than 11 of these 14 clauses. Moreover, if 11 of these clauses are satisfied, then at least one of λ1 , λ2 , λ3 must be true. On the other hand, if one of λ1 , λ2 , λ3 is true we can set yi := λ1 ∧ λ2 ∧ λ3 and z i := true in order to satisfy 11 of these clauses. We conclude that I has a truth assignment satisfying all m clauses if and only if I 0 has a truth assignment that satisfies 11m clauses. It is an open question whether each NP-hard decision problem P ∈ NP is NP-complete (recall the difference between polynomial reduction and polynomial transformation; Definitions 15.15 and 15.17). Exercises 17 and 18 discuss two NP-hard decision problems that appear not to be in NP. Unless P = NP there is no exact polynomial-time algorithm for any NP-hard problem. There might, however, be a pseudopolynomial algorithm: Definition 15.37. Let P be a decision problem or an optimization problem such that each instance x consists of a list of nonnegative integers. We denote by largest(x) the largest of these integers. An algorithm for P is called pseudopolynomial if its running time is bounded by a polynomial in size(x) and largest(x). For example there is a trivial pseudopolynomial algorithm for P RIME which divides the natural number n to be tested for primality by each integer from 2 to √ b nc. Another example is: Theorem 15.38. There is a pseudopolynomial algorithm for S UBSET-S UM.

386

15 NP-Completeness

Proof: Given an instance c1 , . . . , cn , K of S UBSET-S UM, we construct a digraph G with vertex set {0, . . . , n} × {0, 1, 2, . . . , K }. For each j ∈ {1, . . . , n} we add edges (( j − 1, i), ( j, i)) (i = 0, 1, . . . , K ) and (( j − 1, i), ( j, i + c j )) (i = 0, 1, . . . , K − c j ). Observe that any path from (0, 0) to ( j, i) corresponds to a subset S ⊆ {1, . . . , j} P c with k∈S k = i, and vice versa. Therefore we can solve our S UBSET-S UM instance by checking whether G contains a path from (0, 0) to (n, K ). With the G RAPH S CANNING A LGORITHM this can be done in O(n K ) time, so we have a pseudopolynomial algorithm. above is also a pseudopolynomial algorithm for PARTITION because PThe n n c i=1 i ≤ 2 largest(c1 , . . . , cn ). We shall discuss an extension of this algorithm in Section 17.2. If the numbers are not too large, a pseudopolynomial algorithm can be quite efficient. Therefore the following definition is useful: 1 2

Definition 15.39. For a decision problem P = (X, Y ) or an optimization problem P = (X, (Sx )x∈X , c, goal), and a subset X 0 ⊆ X of instances we define the restriction of P to X 0 by P 0 = (X 0 , X 0 ∩ Y ) or P 0 = (X 0 , (Sx )x∈X 0 , c, goal), respectively. Let P be a decision or optimization problem such that each instance consists of a list of numbers. For a polynomial p let P p be the restriction of P to instances x consisting of nonnegative integers with largest(x) ≤ p(size(x)). P is called strongly NP-hard if there is a polynomial p such that P p is NP-hard. P is called strongly NP-complete if P ∈ NP and there is a polynomial p such that P p is NP-complete. Proposition 15.40. Unless P = NP there is no exact pseudopolynomial algorithm for any strongly NP-hard problem. We give some famous examples: Theorem 15.41. I NTEGER P ROGRAMMING is strongly NP-hard. Proof: For an undirected graph G the integer program max{1lx : x ∈ ZV (G) , 0 ≤ x ≤ 1l, xv + xw ≤ 1 for {v, w} ∈ E(G)} has optimum value at least k if and only if G contains a stable set of cardinality k. Since k ≤ |V (G)| for all nontrivial instances (G, k) of S TABLE S ET, the result follows from Theorem 15.23.

T RAVELING S ALESMAN P ROBLEM (TSP) Instance: Task:

A complete graph K n (n ≥ 3) and weights c : E(K n ) → R+ . P Find a Hamiltonian circuit T whose weight e∈E(T ) c(e) is minimum.

The vertices of a TSP-instance are often called cities, the weights are also referred to as distances. Theorem 15.42. The TSP is strongly NP-hard.

Exercises

387

Proof: We show that the TSP is NP-hard even when restricted to instances where all distances are 1 or 2. We describe a polynomial transformation from the H AMILTONIAN C IRCUIT problem. Given a graph G on n ≥ 3 vertices, we construct the following instance of TSP: Take one city for each vertex of G, and let the distances be 1 whenever the edge is in E(G) and 2 otherwise. It is then obvious that G is Hamiltonian if and only if the optimum TSP tour has length n. The proof also shows that the following decision problem is not easier than the TSP itself: Given an instance of the TSP and an integer k, is there a tour of length k or less? A similar statement is true for a large class of discrete optimization problems: Proposition 15.43. Let F and F 0 be (infinite) families of finite sets, and let P be the following optimization problem: given a set E ∈ F and a function c : E → Z, find a set F ⊆ E with F ∈ F 0 and c(F) minimum (or decide that no such F exists). Then P can be solved in polynomial time if and only if the following decision problem can be solved in polynomial time: given an instance (E, c) of P and an integer k, is OPT((E, c)) ≤ k? If the optimization problem is NP-hard, then so is this decision problem. Proof: It suffices to show that there is an oracle algorithm for the optimization problem using the decision problem (the converse is trivial). Let (E, c) be an instance of P. P We first determine OPT((E, c)) by binary search. Since there are at most 1 + e∈E |c(e)| ≤ 2size(c) possible values we can do this with O(size(c)) iterations, each including one oracle call. Then we successively check for each element of E whether there exists an optimum solution without this element. This can be done by increasing its weight (say by one) and checking whether this also increases the value of an optimum solution. If so, we keep the old weight, otherwise we indeed increase the weight. After checking all elements of E, those elements whose weight we did not change constitute an optimum solution. Examples where this result applies are the TSP, the M AXIMUM W EIGHT C LIQUE P ROBLEM, the S HORTEST PATH P ROBLEM, the K NAPSACK P ROBLEM, and many others.

Exercises 1. Observe that there are more languages than Turing machines. Conclude that there are languages that cannot be decided by a Turing machine. Turing machines can also be encoded by binary strings. Consider the famous H ALTING P ROBLEM: given two binary strings x and y, where x encodes a Turing machine 8, is time(8, y) < ∞?

388

15 NP-Completeness

Prove that the H ALTING P ROBLEM is undecidable (i.e. there is no algorithm for it). Hint: Assuming that there is such an algorithm A, construct a Turing machine which, on input x, first runs the algorithm A on input (x, x) and then terminates if and only if output(A, (x, x)) = 0. 2. Describe a Turing machine which compares two strings: it should accept as input a string a#b with a, b ∈ {0, 1}∗ and output 1 if a = b and 0 if a 6= b. 3. A well-known machine model is the RAM machine. It works with an infinite sequence of registers x1 , x2 , . . . and one special register, the accumulator Acc. Each register can store an arbitrarily large integer, possibly negative. A RAM program is a sequence of instructions. There are ten types of instructions (the meaning is illustrated on the right-hand side): WRITE LOAD LOADI STORE STOREI ADD SUBTR HALF IFPOS HALT

k k k k k k k i

Acc := k. Acc := xk . Acc := x xk . xk := Acc. x xk := Acc. Acc := Acc + xk . Acc := Acc − xk . Acc := bAcc/2c. If Acc > 0 then go to . i Stop.

A RAM program is a sequence of m instructions; each is one of the above, where k ∈ Z and i ∈ {1, . . . , m}. The computation starts with instruction 1; it then proceeds as one would expect; we do not give a formal definition. The above list of instructions may be extended. We say that a command can be simulated by a RAM program in time n if it can be substituted by RAM commands so that the total number of steps in any computation increases by at most a factor of n. (a) Show that the following commands can be simulated by small RAM programs in constant time: IFNEG i IFZERO i ∗

∗

If Acc < 0 then go to . i If Acc = 0 then go to . i

(b) Show that the SUBTR and HALF commands can be simulated by RAM programs using only the other eight commands in O(size(xk )) time and O(size(Acc)) time, respectively. (c) Show that the following commands can be simulated by RAM programs in O(n) time, where n = max{size(xk ), size(Acc)}:

Exercises

MULT k DIV k MOD k ∗

389

Acc := Acc · xk . Acc := bAcc/xk c. Acc := Acc mod xk .

4. Let f : {0, 1}∗ → {0, 1}∗ be a mapping. Show that if there is a Turing machine 8 computing f , then there is a RAM program (cf. Exercise 3) such that the computation on input x (in Acc) terminates after O(size(x)+time(8, x)) steps with Acc = f (x). Show that if there is a RAM machine which, given x in Acc, computes f (x) in Acc in at most g(size(x)) steps, then there is a Turing machine computing f with time(8, x) = O(g(size(x))3 ). 5. Prove that the following two decision problems are in NP: (a) Given two graphs G and H , is G isomorphic to a subgraph of H ? (b) Given a natural number n (in binary encoding), is there a prime number p with n = p p ? 6. Prove: If P ∈ NP, then there exists a polynomial p such that P can be solved by a (deterministic) algorithm having time complexity O 2 p(n) . 7. Let Z be a 2S AT instance, i.e. a collection of clauses over X with two literals each. Consider a digraph G(Z) as follows: V (G)is the set of literals over X . There is an edge (λ1 , λ2 ) ∈ E(G) iff the clause λ1 , λ2 is a member of Z. (a) Show that if, for some variable x, x and x are in the same strongly connected component of G(Z), then Z is not satisfiable. (b) Show the converse of (a). (c) Give a linear-time algorithm for 2S AT. 8. Describe a linear-time algorithm which for any instance of S ATISFIABILITY finds a truth assignment satisfying at least half of the clauses. 9. Consider 3-O CCURRENCE S AT, which is S ATISFIABILITY restricted to instances where each clause contains at most three literals and each variable occurs in at most three clauses. Prove that even this restricted version is NPcomplete. 10. Let κ : {0, 1}m → {0, 1}m be a (not necessarily bijective) mapping, m ≥ 2. For x = x1 × · · · × xn ∈ {0, 1}m × · · · × {0, 1}m = {0, 1}nm let κ(x) S := κ(x1 ) × · · · × κ(xn ), and for a decision problem P = (X, Y ) with X ⊆ n∈Z+ {0, 1}nm let κ(P) := ({κ(x) : x ∈ X }, {κ(x) : x ∈ Y }). Prove: (a) For all codings κ and all P ∈ NP we have also κ(P) ∈ NP. (b) If κ(P) ∈ P for all codings κ and all P ∈ P, then P = NP. (Papadimitriou [1994]) 11. Prove that S TABLE S ET is NP-complete even if restricted to graphs whose maximum degree is 4. Hint: Use Exercise 9. 12. Prove that the following problem, called D OMINATING S ET, is NP-complete. Given an undirected graph G and a number k ∈ N, is there a set X ⊆ V (G) with |X | ≤ k such that X ∪ 0(X ) = V (G) ? Hint: Transformation from V ERTEX C OVER.

390

15 NP-Completeness

13. The decision problem C LIQUE is NP-complete. Is it still NP-complete (provided that P 6= NP) if restricted to (a) bipartite graphs, (b) planar graphs, (c) 2-connected graphs? 14. Prove that the following problems are NP-complete: (a) H AMILTONIAN PATH and D IRECTED H AMILTONIAN PATH Given a graph G (directed or undirected), does G contain a Hamiltonian path? (b) S HORTEST PATH Given a graph G, weights c : E(G) → Z, two vertices s, t ∈ V (G), and an integer k. Is there an s-t-path of weight at most k? (c) 3-M ATROID I NTERSECTION Given three matroids (E, F1 ), (E, F2 ), (E, F3 ) (by independence oracles) and a number k ∈ N, decide whether there is a set F ∈ F1 ∩ F2 ∩ F3 with |F| ≥ k. (d) C HINESE P OSTMAN P ROBLEM Given graphs G and H with V (G) = V (H ), weights c : E(H ) → Z+ and an integer k. Is there a subset F ⊆ E(H ) with c(F) ≤ k such that . (V (G), E(G) ∪ F) is connected and Eulerian? 15. Either find a polynomial-time algorithm or prove NP-completeness for the following decision problems: (a) Given an undirected graph G and some T ⊆ V (G), is there a spanning tree in G such that all vertices in T are leaves? (b) Given an undirected graph G and some T ⊆ V (G), is there a spanning tree in G such that all leaves are elements of T ? (c) Given a digraph G, weights c : E(G) → R, a set T ⊆ V (G) and a number k, is there a branching B with |δ + B (x)| ≤ 1 for all x ∈ T and c(B) ≥ k? 16. Prove that the following decision problem belongs to coNP: given a matrix A ∈ Qm×n and a vector b ∈ Qn , is the polyhedron {x : Ax ≤ b} integral? Hint: Use Proposition 3.9, Lemma 5.11, and Theorem 5.13. Note: The problem is not known to be in NP. 17. Show that the following problem is NP-hard (it is not known to be in NP): given an instance of S ATISFIABILITY, does the majority of all truth assignments satisfy all the clauses? 18. Show that PARTITION polynomially transforms to the following problem (which is thus NP-hard; it is not known to be in NP):

K - TH H EAVIEST S UBSET Instance:

Integers c1 , . . . , cn , K , L.

Question: Are P there K distinct subsets S1 , . . . , S K ⊆ {1, . . . , n} such that j∈Si c j ≥ L for i = 1, . . . , K ?

References

391

19. Prove that the following problem belongs to coNP: given a matrix A ∈ Zm×n and a vector b ∈ Zm , decide whether the polyhedron P = {x ∈ Rn : Ax ≤ b} is integral. Note: In fact the problem is coNP-complete, as was shown by Papadimitriou and Yannakakis [1990].

References General Literature: Aho, A.V., Hopcroft, J.E., and Ullman, J.D. [1974]: The Design and Analysis of Computer Algorithms. Addison-Wesley, Reading 1974 Ausiello, G., Crescenzi, P., Gambosi, G., Kann, V., Marchetti-Spaccamela, A., and Protasi, M. [1999]: Complexity and Approximation: Combinatorial Optimization Problems and Their Approximability Properties. Springer, Berlin 1999 Bovet, D.B., and Crescenzi, P. [1994]: Introduction to the Theory of Complexity. PrenticeHall, New York 1994 Garey, M.R., and Johnson, D.S. [1979]: Computers and Intractability: A Guide to the Theory of NP-Completeness. Freeman, San Francisco 1979, Chapters 1–3, 5, and 7 Horowitz, E., and Sahni, S. [1978]: Fundamentals of Computer Algorithms. Computer Science Press, Potomac 1978, Chapter 11 Johnson, D.S. [1981]: The NP-completeness column: an ongoing guide. Journal of Algorithms starting with Vol. 4 (1981) Karp, R.M. [1975]: On the complexity of combinatorial problems. Networks 5 (1975), 45–68 Papadimitriou, C.H. [1994]: Computational Complexity. Addison-Wesley, Reading 1994 Papadimitriou, C.H., and Steiglitz, K. [1982]: Combinatorial Optimization: Algorithms and Complexity. Prentice-Hall, Englewood Cliffs 1982, Chapters 15 and 16 Wegener, I. [2005]: Complexity Theory: Exploring the Limits of Efficient Algorithms. Springer, Berlin 2005

Cited References: Adleman, L.M., Pomerance, C., and Rumely, R.S. [1983]: On distinguishing prime numbers from composite numbers. Annals of Mathematics 117 (1983), 173–206 Agrawal, M., Kayal, N., and Saxena, N. [2004]: PRIMES is in P. Annals of Mathematics 160 (2004), 781–793 Cook, S.A. [1971]: The complexity of theorem proving procedures. Proceedings of the 3rd Annual ACM Symposium on the Theory of Computing (1971), 151–158 Edmonds, J. [1965]: Minimum partition of a matroid into independent subsets. Journal of Research of the National Bureau of Standards B 69 (1965), 67–72 van Emde Boas, P. [1990]: Machine models and simulations. In: Handbook of Theoretical Computer Science; Volume A; Algorithms and Complexity (J. van Leeuwen, ed.), Elsevier, Amsterdam 1990, pp. 1–66 F¨urer, M. [2007]: Faster integer mulitplication. Proceedings of the 39th ACM Symposium on Theory of Computing (2007), 57–66 Garey, M.R., Johnson, D.S., and Stockmeyer, L. [1976]: Some simplified NP-complete graph problems. Theoretical Computer Science 1 (1976), 237–267

392

15 NP-Completeness

Hopcroft, J.E., and Ullman, J.D. [1979]: Introduction to Automata Theory, Languages, and Computation. Addison-Wesley, Reading 1979 Karp, R.M. [1972]: Reducibility among combinatorial problems. In: Complexity of Computer Computations (R.E. Miller, J.W. Thatcher, eds.), Plenum Press, New York 1972, pp. 85–103 Ladner, R.E. [1975]: On the structure of polynomial time reducibility. Journal of the ACM 22 (1975), 155–171 Lewis, H.R., and Papadimitriou, C.H. [1981]: Elements of the Theory of Computation. Prentice-Hall, Englewood Cliffs 1981 Papadimitriou, C.H., and Yannakakis, M. [1990]: On recognizing integer polyhedra. Combinatorica 10 (1990), 107–109 Pratt, V. [1975]: Every prime has a succinct certificate. SIAM Journal on Computing 4 (1975), 214–220 Sch¨onhage, A., and Strassen, V. [1971]: Schnelle Multiplikation großer Zahlen. Computing 7 (1971), 281–292 Turing, A.M. [1936]: On computable numbers, with an application to the Entscheidungsproblem. Proceedings of the London Mathematical Society (2) 42 (1936), 230–265 and 43 (1937), 544–546

16 Approximation Algorithms

In this chapter we introduce the important concept of approximation algorithms. So far we have dealt mostly with polynomially solvable problems. In the remaining chapters we shall indicate some strategies to cope with NP-hard combinatorial optimization problems. Here approximation algorithms must be mentioned in the first place. The ideal case is when the solution is guaranteed to differ from the optimum solution by a constant only: Definition 16.1. An absolute approximation algorithm for an optimization problem P is a polynomial-time algorithm A for P for which there exists a constant k such that |A(I ) − OPT(I )| ≤ k for all instances I of P. Unfortunately, an absolute approximation algorithm is known for very few classical NP-hard optimization problems. We shall discuss two major examples, the E DGE -C OLOURING P ROBLEM and the V ERTEX -C OLOURING P ROBLEM in planar graphs in Section 16.3. In most cases we must be satisfied with relative performance guarantees. Here we have to restrict ourselves to problems with nonnegative weights. Definition 16.2. Let P be an optimization problem with nonnegative weights and k ≥ 1. A k-factor approximation algorithm for P is a polynomial-time algorithm A for P such that 1 OPT(I ) ≤ A(I ) ≤ k OPT(I ) k for all instances I of P. We also say that A has performance ratio (or performance guarantee) k. The first inequality applies to maximization problems, the second one to minimization problems. Note that for instances I with OPT(I ) = 0 we require an exact solution. The 1-factor approximation algorithms are precisely the exact polynomialtime algorithms. Sometimes the above definition is extended to the case where k is a function of the instance I , rather than a constant. We will see an example in the next section.

394

16 Approximation Algorithms

In Section 13.4 we saw that the B EST-I N -G REEDY A LGORITHM for the M AX IMIZATION P ROBLEM for an independence system (E, F) has performance ratio 1 q(E,F ) (Theorem 13.19). In the following sections and chapters we shall illustrate

the above definitions and analyse the approximability of various NP-hard problems. We start with covering problems.

16.1 Set Covering In this section we focus on the following quite general problem:

M INIMUM W EIGHT S ET C OVER P ROBLEM Instance:

A set system (U, S) with

Task:

Find a minimum weight set cover of (U, S), i.e. a subfamily R ⊆ S S such that R∈R R = U .

S

S∈S

S = U , weights c : S → R+ .

For c ≡ 1, the problem is called M INIMUM S ET C OVER P ROBLEM. Another interesting special case arises if |{S ∈ S : x ∈ S}| = 2 for all x ∈ U ; this is the M INIMUM W EIGHT V ERTEX C OVER P ROBLEM: given a graph G and c : V (G) → R+ , the corresponding set covering instance is defined by U := E(G), S := {δ(v) : v ∈ V (G)} and c(δ(v)) := c(v) for all v ∈ V (G). As the M INIMUM W EIGHT V ERTEX C OVER P ROBLEM is NP-hard even for unit weights (Theorem 15.24), so is the M INIMUM S ET C OVER P ROBLEM. Johnson [1974] and Lov´asz [1975] proposed a simple greedy algorithm for the M INIMUM S ET C OVER P ROBLEM: in each iteration, pick a set which covers a maximum number of elements not already covered. Chv´atal [1979] generalized this algorithm to the weighted case:

G REEDY A LGORITHM F OR S ET C OVER Input:

A set system (U, S) with

Output:

A set cover R of (U, S).

S

S∈S

S = U , weights c : S → R+ .

1

Set R := ∅ and W := ∅.

2

While W 6= U do: Choose a set R ∈ S \ R for which R \ W 6= ∅ and Set R := R ∪ {R} and W := W ∪ R.

c(R) |R\W |

is minimum.

The running time is obviously O(|U ||S|). The following performance guarantee can be proved: Theorem 16.3. (Chv´atal [1979]) For any instance (U, S, c) of the M INIMUM W EIGHT S ET C OVER P ROBLEM, the G REEDY A LGORITHM F OR S ET C OVER finds a set cover whose weight is at most H (r ) OPT(U, S, c), where r := max S∈S |S| and H (r ) = 1 + 21 + · · · + r1 .

16.1 Set Covering

395

Proof: Let (U, S, c) be an instance of the M INIMUM W EIGHT S ET C OVER P ROBLEM, and let R = {R1 , . . . , Rk } be the solution found by the above algorithm, Sj where Ri is the set chosen in the i-th iteration. For j = 0, . . . , k let W j := i=1 Ri . For each e ∈ U let j (e) := min{ j ∈ {1, . . . , k} : e ∈ R j } be the iteration where e is covered. Let c(R j (e) ) . y(e) := |R j (e) \ W j (e)−1 | Let S ∈ S be fixed, and let k 0 := max{ j (e) : e ∈ S}. We have 0

X

y(e) =

k X

X

y(e)

i=1 e∈S: j (e)=i

e∈S

0

=

k X

c(Ri ) |S ∩ (Wi \ Wi−1 )| |Ri \ Wi−1 |

i=1 0

=

k X

c(Ri ) (|S \ Wi−1 | − |S \ Wi |) |Ri \ Wi−1 |

i=1 0

≤

k X

c(S) (|S \ Wi−1 | − |S \ Wi |) |S \ Wi−1 |

i=1

by the choice of the Ri in 2 (observe that S \ Wi−1 6= ∅ for i = 1, . . . , k 0 ). By writing si := |S \ Wi−1 | we get 0

X e∈S

k X si − si+1 y(e) ≤ c(S) si i=1 0

≤ c(S)

k X 1 i=1

1 1 + ··· + + si si − 1 si+1 + 1

0

= c(S)

k X

(H (si ) − H (si+1 ))

i=1

= c(S)(H (s1 ) − H (sk 0 +1 )) ≤ c(S)H (s1 ). Since s1 = |S| ≤ r , we conclude that X y(e) ≤ c(S)H (r ). e∈S

We sum over all S ∈ O for an optimum set cover O and obtain

396

16 Approximation Algorithms

c(O)H (r ) ≥

XX

y(e)

S∈O e∈S

≥

X

y(e)

e∈U

=

k X

X

y(e)

i=1 e∈U : j (e)=i

=

k X

c(Ri ) = c(R).

i=1

For a slightly tighter analysis of the non-weighted case, see Slav´ık [1997]. Raz and Safra [1997] discovered that there exists a constant c > 0 such that, unless P = NP, no performance ratio of c ln |U | can be achieved. Indeed, a performance ratio of c ln |U | cannot be achieved for any c < 1 unless each problem in NP can be solved in O n O(log log n) time (Feige [1998]). The M INIMUM W EIGHT E DGE C OVER P ROBLEM is obviously a special case of the M INIMUM W EIGHT S ET C OVER P ROBLEM. Here we have r = 2 in Theorem 16.3, hence the above algorithm is a 32 -factor approximation algorithm in this special case. However, the problem can also be solved optimally in polynomial time; cf. Exercise 11 of Chapter 11. For the M INIMUM V ERTEX C OVER P ROBLEM, the above algorithm reads as follows:

G REEDY A LGORITHM F OR V ERTEX C OVER Input:

A graph G.

Output:

A vertex cover R of G.

1

Set R := ∅.

2

While E(G) 6= ∅ do: Choose a vertex v ∈ V (G) \ R with maximum degree. Set R := R ∪ {v} and delete all edges incident to v.

This algorithm looks reasonable, so one might ask for which k it is a k-factor approximation algorithm. It may be surprising that there is no such k. Indeed, the bound given in Theorem 16.3 is almost best possible: Theorem 16.4. (Johnson [1974], Papadimitriou and Steiglitz [1982]) For all n ≥ 3 there is an instance G of the M INIMUM V ERTEX C OVER P ROBLEM such that n H (n − 1) + 2 ≤ |V (G)| ≤ n H (n − 1) + n, the maximum degree of G is n − 1, OPT(G) = n, and the above algorithm can find a vertex cover containing all but n vertices. j k P Proof: For each n ≥ 3 and i ≤ n we define Ain := ij=2 nj and

16.1 Set Covering

397

V (G n ) := a1 , . . . , a An−1 , b1 , . . . , bn , c1 , . . . , cn . n

E(G n ) := {{bi , ci } : i = 1, . . . , n} ∪ n−1

i

An

i−1 − 1)i + 1 ≤ k ≤ ( j − A )i . {a j , bk } : ( j − Ai−1 n n

i=2 j=Ai−1 n +1 n−1 ≤ n H (n − 1) − n and An−1 ≥ n H (n − Observe that |V (G n )| = 2n + An−1 n , An n 1) − n − (n − 2). Figure 16.1 shows G 6 .

c1

c2

c3

c4

c5

c6

b1

b2

b3

b4

b5

b6

a1

a2

a3

a4

a5

a6

a7

Fig. 16.1.

If we apply our algorithm to G n , it may ﬁrst choose vertex a An−1 (because n it has maximum degree), and subsequently the vertices a An−1 −1 , a An−1 −2 , . . . , a1 . n n After this there are n disjoint edges left, so n more vertices are needed. Hence the + n vertices, while the optimum vertex constructed vertex cover consists of An−1 n cover {b1 , . . . , bn } has size n. There are, however, 2-factor approximation algorithms for the M INIMUM V ER C OVER P ROBLEM. The simplest one is due to Gavril (see Garey and Johnson [1979]): just ﬁnd any maximal matching M and take the ends of all edges in M. This is obviously a vertex cover and contains 2|M| vertices. Since any vertex cover must contain |M| vertices (no vertex covers two edges of M), this is a 2-factor approximation algorithm. This performance guarantee is tight: simply think of a graph consisting of many disjoint edges. It may be surprising that the above is the best known approximation algorithm for the M INIMUM V ERTEX C OVER P ROBLEM. Later we shall show that there is a number k > 1 such that no k-factor approximation algorithm exists unless P = NP (Theorem 16.46). Indeed, a 1.36-factor approximation algorithm does not exist unless P = NP (Dinur and Safra [2002]). TEX

398

16 Approximation Algorithms

At least Gavril’s algorithm can be extended to the weighted case. We present the algorithm of Bar-Yehuda and Even [1981], which is applicable to the general M INIMUM W EIGHT S ET C OVER P ROBLEM:

BAR -Y EHUDA -E VEN A LGORITHM Input:

A set system (U, S) with

Output:

A set cover R of (U, S).

1

2

S

S∈S

S = U , weights c : S → R+ .

Set R := ∅ and W := ∅. Set y(e) := 0 for all e ∈ U . Set c0 (S) := c(S) for all S ∈ S. While W 6= U do: Choose an element e ∈ U \ W . Let R ∈ S with e ∈ R and c0 (R) minimum. Set y(e) := c0 (R). Set c0 (S) := c0 (S) − y(e) for all S ∈ S with e ∈ S. Set R := R ∪ {R} and W := W ∪ R.

Theorem 16.5. (Bar-Yehuda and Even [1981]) For any instance (U, S, c) of the M INIMUM W EIGHT S ET C OVER P ROBLEM, the BAR -Y EHUDA -E VEN A L GORITHM finds a set cover whose weight is at most p OPT(U, S, c), where p := maxe∈U |{S ∈ S : e ∈ S}|. Proof: The M INIMUM W EIGHT S ET C OVER P ROBLEM can be written as the integer linear program o n min cx : Ax ≥ 1l, x ∈ {0, 1}S , where A is the matrix whose rows correspond to the elements of U and whose columns are the incidence vectors of the sets in S. The optimum of the LP relaxation min {cx : Ax ≥ 1l, x ≥ 0} is a lower bound for OPT(U, S, c) (the omission of the constraints x ≤ 1l does not change the optimum value of this LP). Hence, by Proposition 3.13, the optimum of the dual LP max{y1l : y A ≤ c, y ≥ 0} is also a lower bound for OPT(U, S, c). Now observe that c0 (S) ≥ 0 for all S ∈ S at any stage P of the algorithm. Let y¯ be the vector y at termination. We have y¯ ≥ 0 and e∈S y¯ (e) ≤ c(S) for all S ∈ S, i.e. y¯ is a feasible solution of the dual LP and y¯ 1l ≤ max{y1l : y A ≤ c, y ≥ 0} ≤ OPT(U, S, c).

16.2 The Max-Cut Problem

399

Finally observe that c(R) =

X

c(R)

R∈R

=

XX

y¯ (e)

R∈R e∈R

≤

X

p y¯ (e)

e∈U

=

p y¯ 1l

≤

p OPT(U, S, c).

Since we have p = 2 in the vertex cover case, this is a 2-factor approximation algorithm for the M INIMUM W EIGHT V ERTEX C OVER P ROBLEM. The first 2factor approximation algorithm was due to Hochbaum [1982]. She proposed finding an optimum solution y of the dual LP in the above proof and taking all sets S P with e∈S y(e) = c(S). Alternatively, one could find an optimum solution x of the primal LP and take all sets S with x S ≥ 1p . The advantage of the BAR -Y EHUDA -E VEN A LGORITHM is that it does not explicitly use linear programming. In fact it can easily be implemented with P O |S| time. It is our first example of a primal-dual approximation alS∈S gorithm; more complicated examples will follow in Sections 20.4 and 22.3.

16.2 The Max-Cut Problem In this section we consider another basic problem:

M AXIMUM W EIGHT C UT P ROBLEM Instance:

An undirected graph G and weights c : E(G) → R+ .

Task:

Find a cut in G with maximum total weight.

This problem is often called M AX -C UT for short. In contrast to minimum weight cuts, discussed in Section 8.7, this is a difficult problem. It is strongly NP-hard; even the special case where c ≡ 1 (the M AXIMUM C UT P ROBLEM) is hard: Theorem 16.6. (Garey, Johnson and Stockmeyer [1976]) The M AXIMUM C UT P ROBLEM is NP-hard. Proof: By reduction from M AX -2S AT (cf. Theorem 15.36). Given an instance of M AX -2S AT with n variables and m clauses, we construct a graph G whose vertices are the literals plus an extra vertex z. For each variable x we add 3m parallel edges between x and x. ¯ For each clause {λ, λ0 } we add three edges {λ, λ0 }, 0 {λ, z} and {λ , z}. So G has 2n + 1 vertices and 3m(n + 1) edges.

400

16 Approximation Algorithms

We claim that the maximum cardinality of a cut in G is 3mn + 2t, where t is the maximum number of clauses satisfied by any truth assignment. Indeed, given a truth assignment satisfying t clauses, let X be the set of true literals. Then |δG (X )| = 3mn + 2t. Conversely, if there is a set X ⊆ V (G) with |δG (X )| ≥ 3mn + a, then w.l.o.g. z ∈ / X (otherwise replace X by V (G) \ X ), and for each variable x we have |X ∩ {x, x}| ¯ = 1 (otherwise replace X by X 1{x} and increase the cut). Thus we can set all literals in X to be true and get a truth assignment satisfying at least a2 clauses. It is very easy to find a 2-factor approximation algorithm for the M AXIMUM W EIGHT C UT P ROBLEM:P If V (G) = {v1 , . . . , vn }, start with P X := {v1 }, and for i = 3, . . . , n add vi to X if e∈E(vi ,{v1 ,...,vi−1 }∩X ) c(e) < e∈E(vi ,{v1 ,...,vi−1 }\X ) c(e). (The simple analysis of this algorithm is left as Exercise 9.) For a long time no better approximation algorithm was known. Then Goemans and Williamson [1995] found a much better one using semidefinite programming; the rest of this section is based on their paper. Let G be an undirected graph and c : E(G) → R+ . W.l.o.g., V (G) = {1, . . . , n}. For 1 ≤ i, j ≤ n let ci j := c({i, j}) if {i, j} ∈ E(G) and ci j := 0 othW EIGHT C UT P ROBLEM consists of finding a subset erwise. Then the M AXIMUM P S ⊆ {1, . . . , n} maximizing i∈S, j∈{1,...,n}\S ci j . Expressing S by y ∈ {−1, 1}n with yi = 1 iff i ∈ S, we can formulate the problem as follows:

max s.t.

1 2

X

ci j (1 − yi y j )

1≤i< j≤n

∈ {−1, 1}

yi

(i = 1, . . . , n)

The variables yi can be regarded as one-dimensional vectors of unit norm. By relaxing them to multidimensional vectors of unit Euclidean norm we get a very interesting relaxation:

max s.t.

1 2 yi

X

ci j (1 − yi> y j ) (16.1)

1≤i< j≤n

∈ Sm

(i = 1, . . . , n)

where m ∈ N and Sm = {x ∈ Rm : ||x||2 = 1} denotes the unit sphere in For example, for the triangle (n = 3, c12 = c13 = c23 = 1) the optimum is obtained by points on the unit sphere in R2 that are√vertices of an equilateral √ triangle, e.g. y1 = (0, −1), y2 = (− 23 , 12 ), and y3 = ( 23 , 12 ), giving an optimum value of 94 , in contrast to the maximum weight of a cut, which is 2. However, the interesting fact is that we can solve (16.1) almost optimally in polynomial time. The trick is not to consider the variables yi directly, not even their dimension. Instead, we consider the n × n-matrix (yi> y j )i, j=1,...,n . As a matrix X is symmetric Rm .

16.2 The Max-Cut Problem

401

and positive semidefinite if and only if it can be written as B > B for some matrix B, we can write equivalently max s.t.

1 2

X

ci j (1 − xi j )

1≤i< j≤n

xii = 1 (i = 1, . . . , n) X = (xi j )1≤i, j≤n symmetric and positive semidefinite

(16.2)

From a solution to (16.2) we can obtain a solution to (16.1) with m ≤ n and almost the same objective function value by Cholesky factorization in O(n 3 ) time (we have to accept an arbitrarily small rounding error; cf. Exercise 6 of Chapter 4). Problem (16.2) is called a semidefinite program. It can be solved approximately in polynomial time by the E LLIPSOID M ETHOD, applying Theorem 4.19, as we shall show now. First observe that we optimize a linear objective function over the convex set P := X = (xi j )1≤i, j≤n ∈ Rn×n : X symmetric and positive semidefinite, xii = 1 (i = 1, . . . , n) . 2

By projecting P to the n 2−n free variables we obtain P 0 := (xi j )1≤i< j≤n : (xi j )1≤i, j≤n ∈ P with xii := 1 and x ji := xi j for i < j . Note that neither P nor P 0 is a polyhedron. However, P 0 is convex, bounded, and full-dimensional: Proposition 16.7. P 0 is convex. Moreover, B(0, n1 ) ⊆ P 0 ⊆ B(0, n). Proof: Convexity follows from the simple fact that convex combinations of positive semidefinite matrices are positive semidefinite. For the first inclusion, observe that for a symmetric n × n-matrix X whose diagonal entries are 1 and whose off-diagonal entries have absolute value at most 1 n n we have, for any d ∈ R , d> X d =

n X

xi j di d j

i, j=1

≥

≥

=

1 X (xii di2 + x j j d j2 − (2n − 2)|xi j ||di d j |) 2n − 2 1 2n − 2 1 2n − 2

≥ 0,

i6= j n X i6= j n X i6= j

(di2 + d j2 − 2|di d j |) (|di | − |d j |)2

402

16 Approximation Algorithms

i.e. X is positive semidefinite. For the second inclusion, note that all off-diagonal entries of a matrix in P have absolute value at most 1, and thus the Euclidean norm of the vector of upper diagonal entries is at most n. It remains to show that the S EPARATION P ROBLEM for P 0 can be solved in polynomial time. This is achieved by G AUSSIAN E LIMINATION: Theorem 16.8. Given a symmetric matrix X ∈ Qn×n , we can decide in polynomial time whether X is positive semidefinite, and find a vector d ∈ Qn with d > X d < 0 if one exists. Proof: If xnn < 0, then we set d = (0, . . . , 0, 1) and have d > X d < 0. If xnn = 0 x and xn j 6= 0 for some j < n, then we can define d by d j := −1, dn := 2xjnj j + xn j , x

and di := 0 for i ∈ {1, . . . , n − 1} \ { j}, and have d > X d = x j j − 2xn j ( 2xjnj j + xn j ) = −2(xn j )2 < 0, again proving that X is not positive semidefinite. In the other cases we reduce the dimension. If xn j = 0 for all j, then the last row and column can be deleted: X is positive semidefinite if and only if X 0 := (xi j )i, j=1,...,n−1 is positive semidefinite. Moreover, if c ∈ Qn−1 satisfies c> X 0 c < 0, we set d := ( 0c ) and have d > X d < 0. Hence let us now assume that xnn > 0. Then we consider X 0 := (xi j − xni xn j xnn )i, j=1,...,n−1 ; this corresponds to one iteration of G AUSSIAN E LIMINATION . Note that X 0 is positive semidefinite if and only if X is positive semidefinite. c n−1 For a vector c ∈ Qn−1 with c> X 0 c < 0 we set d := ( − x 1 Pi=1 ci xni ). Then nn

d> X d =

n−1 X i, j=1

n−1 X xni di xi0 j + xn j d j + 2 dn xn j d j + dn2 xnn xnn

= c> X 0 c +

j=1

n−1 X i, j=1

ci

xni xn j c j (1 − 2 + 1) xnn

= c> X 0 c < 0. This defines a polynomial-time algorithm. To see that the numbers involved in the computation of d are not too large, let X (n) , X (n−1) , . . . , X (k) be the considered matrices (X (i) ∈ Qi×i ), and assume that we observe in iteration n + 1 − k that the matrix X (k) = (yi j )i, j=1,...,k is not positive semidefinite (i.e. ykk < 0 or ykk = 0 and yk j 6= 0 for some j < k). We have a vector c ∈ Qk with c> X (k) c < 0 and size(c) ≤ 2 size(X (k) ). Now a vector d ∈ Qn with d > X d < 0 can be constructed as above; note that d is a solution of the linear equation system Md = ( 0c ), where the j-th row of M is • the j-th unit vector if j ≤ k, • the j-th unit vector if j > k and the j-th row of X ( j) is zero,

16.2 The Max-Cut Problem

•

403

the j-th row of X ( j) , followed by zeros, otherwise.

Hence, with Theorem 4.4, we have size(d) ≤ 4n(size(M) + size(c)), which is polynomial by Theorem 4.10. Corollary 16.9. The S EPARATION P ROBLEM for P 0 can be solved in polynomial time. Proof: Let (yi j )1≤i< j≤n be given, and let Y = (yi j )1≤i, j≤n be the symmetric matrix defined by yii = 1 for all i and y ji := yi j for i < j. Apply Theorem 16.8. If Y is positive semidefinite, we are done. Pn di2 > Otherwise we find a vector d ∈ Qn with d > Y d < 0. Then − i=1 P P P P n n 2 > d Y d − i=1 di = 1≤i< j≤n 2di d j yi j , but 1≤i< j≤n 2di d j z i j ≥ − i=1 di2 for all z ∈ P 0 . Thus (di d j )1≤i< j≤n constitutes a separating hyperplane. We can now conclude: Theorem 16.10. For any instance of the M AXIMUM W EIGHT C UT P ROBLEM, we can find a matrix Y = (yi j )1≤i, j≤n ∈ P with X X ci j (1 − yi j ) ≥ max ci j (1 − xi j ) : (xi j )1≤i, j≤n ∈ P − 1≤i< j≤n

1≤i< j≤n

in time polynomial in n, size((ci j )1≤i< j≤n ), and size(). Proof: We apply Theorem 4.19, using Proposition 16.7 and Corollary 16.9. Semidefinite programs such as (16.2) can also be solved approximately by interior point algorithms, which are more efficient than the E LLIPSOID M ETHOD. See Alizadeh [1995] for details. As mentioned above, from an almost optimum solution to (16.2) we can derive a solution to (16.1) with almost the same objective function value by Cholesky factorization. This solution consists of a set of vectors yi ∈ Rm (i = 1, . . . , n) for some m ≤ n. As (16.1) Pis a relaxation of our original problem, we have that the optimum is at most 12 1≤i< j≤n ci j (1 − yi> y j ) + . The vectors yi lie on a unit sphere. The idea is now to take a random hyperplane through the origin, and define S to be the set of indices i for which yi is on one side of this hyperplane. A random hyperplane through the origin is given by a random point on the (m − 1)-dimensional sphere. This can be chosen by drawing m real numbers independently from the standard normal distribution, which in turn can be done using independent random numbers uniformly distributed in [0, 1]. See Knuth [1969] (Section 3.4.1) for details. The algorithm by Goemans and Williamson now reads as follows.

G OEMANS -W ILLIAMSON M AX -C UT-A LGORITHM Input:

A number n ∈ N, numbers ci j ≥ 0 for 1 ≤ i < j ≤ n.

Output:

A set S ⊆ {1, . . . , n}.

404

16 Approximation Algorithms

1

Solve (16.2) approximately; i.e. find a symmetric positive semidefinite matrix X = (xi j )1≤i, j≤n with xii = 1 for i = 1, . . . , n, such that P 1≤i< j≤n ci j (1 − x i j ) ≥ 0.9995 · OPT(16.2).

2

Apply Cholesky factorization to X in order to obtain vectors y1 , . . . , yn ∈ Rm with m ≤ n and yi> y j ≈ xi j for all i, j ∈ {1, . . . , n}.

3

Choose a random point a on the unit sphere {x ∈ Rm : ||x||2 = 1}.

4

Set S := {i ∈ {1, . . . , n} : a > yi ≥ 0}.

Theorem 16.11. The G OEMANS -W ILLIAMSON M AX -C UT-A LGORITHM runs in polynomial time. Proof: See the preceding discussion. The most difficult step, , 1 can Pbe solved in polynomial time by Theorem 16.10. Here we can choose = 0.00025 1≤i< j≤n ci j P as 12 1≤i< j≤n ci j is a lower bound on the optimum objective value (attained by randomly choosing S ⊆ {1, . . . , n}) and thus on the optimum value of (16.2). We now prove the performance guarantee: Theorem 16.12. (Goemans and Williamson [1995]) The G OEMANS -W P ILLIAMSON M AX -C UT-A LGORITHM returns a set S for which the expected value of i∈S, j ∈S / ci j is at least 0.878 times the maximum possible value. Proof: Let Sm again denote the unit sphere in Rm , and let H (y) := {x ∈ Sm : x > y ≥ 0} be the hemisphere with pole y, for y ∈ Sm . For a subset A ⊆ volume (A) Sm let µ(A) := volume (Sm ) ; this defines a probability measure on Sm . We have |S ∩{i, j}| = 1 with probability µ(H (yi )4H (y j )), where 4 denotes the symmetric difference. Note that H (yi )4H (y j ) is the union of two spherical digons, each with angle arccos(yi> y j ). As the volume is proportional to the angle, we have µ(H (yi )4H (y j )) = π1 arccos(yi> y j ). Claim: π1 arccos β ≥ 0.8785 1−β 2 for all β ∈ [−1, 1]. For β = 1 we have equality. Moreover, elementary calculus yields min

−1≤β 1.38. As 1.38 > 0.8785 2 , this proves the claim. Pπ c Thus the expected value of i∈S, j ∈S i j is /

16.3 Colouring

X

X

ci j µ(H (yi )4H (y j )) =

1≤i< j≤n

ci j

1≤i< j≤n

≥ 0.8785 ·

1 2

1 ≈ 0.8785 · 2

405

1 arccos(yi> y j ) π X

ci j (1 − yi> y j )

1≤i< j≤n

X

ci j (1 − xi j )

1≤i< j≤n

≥ 0.8785 · 0.9995 · OPT(16.2) > 0.878 · OPT(16.2) X ci j : S ⊆ {1, . . . , n} . ≥ 0.878 · max i∈S, j ∈S /

Therefore we have a randomized approximation algorithm with performance 1 < 1.139. Mahajan and Ramesh [1999] showed how to derandomize this ratio 0.878 algorithm, thus obtaining a deterministic 1.139-factor approximation algorithm. However, there is no 1.062-factor approximation algorithm unless P = NP (H˚astad [2001], Papadimitriou and Yannakakis [1991]). See Lov´asz [2003] for other interesting connections of semidefinite programming and combinatorial optimization.

16.3 Colouring In this section we briefly discuss two more well-known special cases of the M IN IMUM S ET C OVER P ROBLEM : we want to partition the vertex set of a graph into stable sets, or the edge set of a graph into matchings: Definition 16.13. Let G be an undirected graph. A vertex-colouring of G is a mapping f : V (G) → N with f (v) 6= f (w) for all {v, w} ∈ E(G). An edgecolouring of G is a mapping f : E(G) → N with f (e) 6= f (e0 ) for all e, e0 ∈ E(G) with e 6= e0 and e ∩ e0 6= ∅. The number f (v) or f (e) is called the colour of v or e. In other words, the set of vertices or edges with the same colour ( f -value) must be a stable set, or a matching, respectively. Of course we are interested in using as few colours as possible:

V ERTEX -C OLOURING P ROBLEM Instance:

An undirected graph G.

Task:

Find a vertex-colouring f : V (G) → {1, . . . , k} of G with minimum k.

406

16 Approximation Algorithms

E DGE -C OLOURING P ROBLEM Instance:

An undirected graph G.

Task:

Find an edge-colouring f : E(G) → {1, . . . , k} of G with minimum k.

Reducing these problems to the M INIMUM S ET C OVER P ROBLEM is not very useful: for the V ERTEX -C OLOURING P ROBLEM we would have to list the maximal stable sets (an NP-hard problem), while for the E DGE -C OLOURING P ROBLEM we would have to reckon with exponentially many maximal matchings. The optimum value of the V ERTEX -C OLOURING P ROBLEM (i.e. the minimum number of colours) is called the chromatic number of the graph. The optimum value of the E DGE -C OLOURING P ROBLEM is called the edge-chromatic number or sometimes the chromatic index. Both colouring problems are NP-hard: Theorem 16.14. The following decision problems are NP-complete: (a) (Holyer [1981]) Decide whether a given simple graph has edge-chromatic number 3. (b) (Stockmeyer [1973]) Decide whether a given planar graph has chromatic number 3. The problems remain NP-hard even when the graph has maximum degree three in (a), and maximum degree four in (b). Proposition 16.15. For any given graph we can decide in linear time whether the chromatic number is less than 3, and if so, find an optimum colouring. The same holds for the edge-chromatic number. Proof: A graph has chromatic number 1 iff it has no edges. By definition, the graphs with chromatic number at most 2 are precisely the bipartite graphs. By Proposition 2.27 we can check in linear time whether a graph is bipartite and in the positive case find a bipartition, i.e. a vertex-colouring with two colours. To check whether the edge-chromatic number of a graph G is less than 3 (and, if so, find an optimum edge-colouring) we simply consider the V ERTEX -C OLOURING P ROBLEM in the line graph of G. This is obviously equivalent. For bipartite graphs, the E DGE -C OLOURING P ROBLEM can be solved, too: Theorem 16.16. (K¨onig [1916]) The edge-chromatic number of a bipartite graph G equals the maximum degree of a vertex in G. Proof: By induction on |E(G)|. Let G be a graph with maximum degree k, and let e = {v, w} be an edge. By the induction hypothesis, G −e has an edge-colouring f with k colours. There are colours i, j ∈ {1, . . . , k} such that f (e0 ) 6= i for all e0 ∈ δ(v) and f (e0 ) 6= j for all e0 ∈ δ(w). If i = j, we are done since we can extend f to G by giving e colour i.

16.3 Colouring

407

The graph H = (V (G), {e0 ∈ E(G) \ {e} : f (e0 ) ∈ {i, j}}) has maximum degree 2, and v has degree at most 1 in H . Consider the maximal path P in H with endpoint v. The colours alternate on P; hence the other endpoint of P cannot be w. Exchange the colours i and j on P and extend the edge-colouring to G by giving e colour j. The maximum degree of a vertex is an obvious lower bound on the edgechromatic number of any graph. It is not always attained as the triangle K 3 shows. The following theorem shows how to find an edge-colouring of a given simple graph which needs at most one more colour than necessary: Theorem 16.17. (Vizing [1964]) Let G be an undirected simple graph with maximum degree k. Then G has an edge-colouring with at most k + 1 colours, and such a colouring can be found in polynomial time. Proof: By induction on |E(G)|. If G has no edges, the assertion is trivial. Otherwise let e = {x, y0 } be any edge; by the induction hypothesis there exists an edge-colouring f of G − e with k + 1 colours. For each vertex v choose a colour n(v) ∈ {1, . . . , k + 1} \ { f (w) : w ∈ δG−e (v)} missing at v. Starting from y0 , construct a maximal sequence y0 , y1 , . . . , yt of distinct neighbours of x such that n(yi−1 ) = f ({x, yi }) for i = 1, . . . , t. If no edge incident to x is coloured n(yt ), then we construct an edge-colouring f 0 of G from f by setting f 0 ({x, yi−1 }) := f ({x, yi }) (i = 1, . . . , t) and f 0 ({x, yt }) := n(yt ). So we assume that there is an edge incident to x with colour n(yt ); by the maximality of t we have f ({x, ys }) = n(yt ) for some s ∈ {1, . . . , t − 1}. ys−1

y2

n(ys−2 )

n(y1 )

y1

n(ys−1 ) = n(yt )

n(y0 ) y0

ys

x n(yt−1 )

yt n(x)

P

n(yt )

n(x)

n(yt )

Fig. 16.2.

Consider the maximum path P starting at yt in the graph (V (G), {e0 ∈ E(G − e) : f (e0 ) ∈ {n(x), n(yt )}}) (this graph has maximum degree 2; cf. Figure 16.2). We distinguish three cases. In each case we construct an edge-colouring f 0 of G.

408

16 Approximation Algorithms

If P ends in x, then {ys , x} is the last edge of P. We construct f 0 from f by exchanging colours n(x) and n(yt ) on P, and setting f 0 ({x, yi−1 }) := f ({x, yi }) (i = 1, . . . , s). If P ends in ys−1 , then the last edge of P has colour n(x), since colour n(yt ) = f ({x, ys }) = n(ys−1 ) is missing at ys−1 . We construct f 0 from f as follows: exchange colours n(x) and n(yt ) on P, set f 0 ({x, yi−1 }) := f ({x, yi }) (i = 1, . . . , s − 1) and f 0 ({x, ys−1 }) := n(x). If P ends neither in x nor in ys−1 , then we can construct f 0 from f by exchanging colours n(x) and n(yt ) on P, and setting f 0 ({x, yi−1 }) := f ({x, yi }) (i = 1, . . . , t) and f 0 ({x, yt }) := n(x). Vizing’s Theorem implies an absolute approximation algorithm for the E DGE C OLOURING P ROBLEM in simple graphs. If we allow parallel edges the statement of Vizing’s Theorem is no longer true: by replacing each edge of the triangle K 3 by r parallel edges we obtain a 2r -regular graph with edge-chromatic number 3r . We now turn to the V ERTEX -C OLOURING P ROBLEM. The maximum degree also gives an upper bound on the chromatic number: Theorem 16.18. Let G be an undirected graph with maximum degree k. Then G has an vertex-colouring with at most k + 1 colours, and such a colouring can be found in linear time. Proof: The following G REEDY C OLOURING A LGORITHM obviously finds such a colouring.

G REEDY C OLOURING A LGORITHM Input:

An undirected graph G.

Output:

A vertex-colouring of G.

1

Let V (G) = {v1 , . . . , vn }.

2

For i := 1 to n do: Set f (vi ) := min{k ∈ N : k 6= f (v j ) for all j < i with v j ∈ 0(vi )}.

For complete graphs and for odd circuits one evidently needs k + 1 colours, where k is the maximum degree. For all other connected graphs k colours suffice, as Brooks [1941] showed. However, the maximum degree is not a lower bound on the chromatic number: any star K 1,n (n ∈ N) has chromatic number 2. Therefore these results do not lead to an approximation algorithm. In fact, no algorithms for the V ERTEX -C OLOURING P ROBLEM with a reasonable performance guarantee for general graphs are known; see Khanna, Linial and Safra [2000]. Zuckerman [2006] showed that, unless P = NP, no polynomial-time algorithm computes the chromatic number of any graph with n vertices up to a factor of n 1− for any fixed > 0. Since the maximum degree is not a lower bound for the chromatic number one can consider the maximum size of a clique. Obviously, if a graph G contains a

16.3 Colouring

409

clique of size k, then the chromatic number of G is at least k. As the pentagon (circuit of length five) shows, the chromatic number can exceed the maximum clique size. Indeed, there are graphs with arbitrarily large chromatic number that contain no K 3 . This motivates the following definition, which is due to Berge [1961,1962]: Definition 16.19. A graph G is perfect if χ (H ) = ω(H ) for every induced subgraph H of G, where χ (H ) is the chromatic number and ω(H ) is the maximum cardinality of a clique in H . It follows immediately that the decision problem whether a given perfect graph has chromatic number k has a good characterization (belongs to NP ∩ coNP). Some examples of perfect graphs can be found in Exercise 15. A polynomial-time algorithm for recognizing perfect graphs has been found by Chudnovsky et al. [2005]. Berge [1961] conjectured that a graph is perfect if and only if it contains neither an odd circuit of length at least five nor the complement of such a circuit as an induced subgraph. This so-called strong perfect graph theorem has been proved by Chudnovsky et al. [2006]. Long before, Lov´asz [1972] proved the weaker assertion that a graph is perfect iff its complement is perfect. This is known as the weak perfect graph theorem; to prove it we need a lemma: Lemma. 16.20. Let G be a perfect graph and x ∈ V (G). Then the graph G 0 := . (V (G) ∪ {y}, E(G) ∪ {{y, v} : v ∈ {x} ∪ 0(x)}), resulting from G by adding a new vertex y which is joined to x and to all neighbours of x, is perfect. Proof: By induction on |V (G)|. The case |V (G)| = 1 is trivial since K 2 is perfect. Now let G be a perfect graph with at least two vertices. Let x ∈ V (G), and let G 0 arise by adding a new vertex y adjacent to x and all its neighbours. It suffices to prove that ω(G 0 ) = χ (G 0 ), since for proper subgraphs H of G 0 this follows from the induction hypothesis: either H is a subgraph of G and thus perfect, or it arises from a proper subgraph of G by adding a vertex y as above. Since we can colour G 0 with χ (G) + 1 colours easily, we may assume that ω(G 0 ) = ω(G). Then x is not contained in any maximum clique of G. Let f be a vertex-colouring of G with χ (G) colours, and let X := {v ∈ V (G) : f (v) = f (x)}. We have ω(G − X ) = χ (G − X ) = χ (G) − 1 = ω(G) − 1 and thus ω(G − (X \ {x})) = ω(G) − 1 (as x does not belong to any maximum clique of G). Since (X \ {x}) ∪ {y} = V (G 0 ) \ V (G − (X \ {x})) is a stable set, we have χ (G 0 ) = χ (G − (X \ {x})) + 1 = ω(G − (X \ {x})) + 1 = ω(G) = ω(G 0 ). Theorem 16.21. (Lov´asz [1972], Fulkerson [1972], Chv´atal [1975]) For a simple graph G the following statements are equivalent: (a) G is perfect.

410

16 Approximation Algorithms

(b) The complement of G is perfect. (c) The stable set polytope, i.e. the convex hull of the incidence vectors of the stable sets of G, is given by: ( ) X V (G) : xv ≤ 1 for all cliques S in G . (16.3) x ∈ R+ v∈S

Proof: We prove (a)⇒(c)⇒(b). This suffices, since applying (a)⇒(b) to the complement of G yields (b)⇒(a). (a)⇒(c): Evidently the stable set polytope is contained in (16.3). To prove the other inclusion, let x be a rational vector in the polytope (16.3); we may write xv = pqv , where q ∈ N and pv ∈ Z+ for v ∈ V (G). Replace each vertex v by a clique of size pv ; i.e. consider G 0 defined by V (G 0 ) := {(v, i) : v ∈ V (G), 1 ≤ i ≤ pv }, E(G 0 ) := {{(v, i), (v, j)} : v ∈ V (G), 1 ≤ i < j ≤ pv } ∪ {{(v, i), (w, j)} : {v, w} ∈ E(G), 1 ≤ i ≤ pv , 1 ≤ j ≤ pw }. Lemma 16.20 implies that G 0 is perfect. For an arbitrary clique X 0 in G 0 let X := {v ∈ V (G) : (v, i) ∈ X 0 for some i} be its projection to G (also a clique); we have X X pv = q xv ≤ q. |X 0 | ≤ v∈X

ω(G 0 )

v∈X

G0

≤ q. Since is perfect, it thus has a vertex-colouring f with at most So q colours. For v ∈ V (G) and i =P 1, . . . , q let ai,v := 1 if f ((v, j)) = i for some q j and ai,v := 0 otherwise. Then i=1 ai,v = pv for all v ∈ V (G) and hence x =

pv q

v∈V (G)

=

q 1X ai q i=1

is a convex combination of incidence vectors of stable sets, where ai = (ai,v )v∈V (G) . (c)⇒(b): We show by induction on |V (G)| that if (16.3) is integral then the complement of G is perfect. Since graphs with less than three vertices are perfect, let G be a graph with |V (G)| ≥ 3 where (16.3) is integral. We have to show that the vertex set of any induced subgraph H of G can be partitioned into α(H ) cliques, where α(H ) is the size of a maximum stable set in H . For proper subgraphs H this follows from the induction hypothesis, since (by Theorem 5.13) every face of the integral polytope (16.3) is integral, in particular the face defined by the supporting hyperplanes xv = 0 (v ∈ V (G) \ V (H )). So it remains to prove that V (G) can be partitioned into α(G) cliques. The equation 1lx = α(G) defines a supporting hyperplane of (16.3), so X X V (G) : xv ≤ 1 for all cliques S in G, xv = α(G) (16.4) x ∈ R+ v∈S

v∈V (G)

16.3 Colouring

411

is a face of (16.3). This face is contained in some facets, which cannot all be of the form {x ∈ (16.3) : xv = 0} for some v (otherwise the origin would belong P to the intersection). Hence there is some clique S in G such that v∈S xv = 1 for all x in (16.4). Hence this clique S intersects each maximum stable set of G. Now by the induction hypothesis, the vertex set of G − S can partitioned into α(G − S) = α(G) − 1 cliques. Adding S concludes the proof. This proof is due to Lov´asz [1979b]. Indeed, the inequality system defining (16.3) is TDI for perfect graphs (Exercise 16). With some more work one can prove that for perfect graphs the V ERTEX -C OLOURING P ROBLEM, the M AXIMUM W EIGHT S TABLE S ET P ROBLEM and the M AXIMUM W EIGHT C LIQUE P ROBLEM can be solved in strongly polynomial time. Although these problems are all NPhard for general graphs (Theorem 15.23, Corollary 15.24, Theorem 16.14(b)), there is a number (the so-called theta-function of the complement graph, introduced by Lov´asz [1979a]) which is always between the maximum clique size and the chromatic number, and which can be computed in polynomial time for general graphs using the E LLIPSOID M ETHOD. The details are a bit involved; see Gr¨otschel, Lov´asz and Schrijver [1988]. One of the best-known problems in graph theory has been the Four Colour Problem: is it true that every planar map can be coloured with four colours such that no two countries with a common border have the same colour? If we consider the countries as regions and switch to the planar dual graph, this is equivalent to asking whether every planar graph has a vertex-colouring with four colours. Appel and Haken [1977] and Appel, Haken and Koch [1977] proved that this is indeed true: every planar graph has chromatic number at most 4. For a simpler proof of the Four Colour Theorem (which nevertheless is based on a case checking by a computer) see Robertson et al. [1997]. We prove the following weaker result, known as the Five Colour Theorem: Theorem 16.22. (Heawood [1890]) Any planar graph has a vertex-colouring with at most five colours, and such a colouring can be found in polynomial time. Proof: By induction on |V (G)|. We may assume that G is simple, and we fix an arbitrary planar embedding 8 = ψ, (Je )e∈E(G) of G. By Corollary 2.33, G has a vertex v of degree five or less. By the induction hypothesis, G − v has a vertex-colouring f with at most 5 colours. We may assume that v has degree 5 and all neighbours have different colours; otherwise we can easily extend the colouring to G. Let w1 , w2 , w3 , w4 , w5 be the neighbours of v in the cyclic order in which the polygonal arcs J{v,wi } leave v. We first claim that there are no vertex-disjoint paths P from w1 to w3 and Q from w2 to w4 in G − v. To prove this, let P be a w1 -w3 -path, and let C be the circuit S in G consisting of P and the edges {v, w1 }, {v, w3 }. By Theorem 2.30 R2 \ e∈E(C) Je splits into two connected regions, and v is on the boundary of both regions. Hence w2 and w4 belong to different regions of that set, implying that every w2 -w4 -path in G − v must contain a vertex of C.

412

16 Approximation Algorithms

Let X be the connected component of the graph G[{x ∈ V (G) \ {v} : f (x) ∈ { f (w1 ), f (w3 )}}] which contains w1 . If X does not contain w3 , we can exchange the colours in X and afterwards extend the colouring to G by colouring v with the old colour of w1 . So we may assume that there is a w1 -w3 -path P containing only vertices coloured with f (w1 ) or f (w3 ). Analogously, we are done if there is no w2 -w4 -path Q containing only vertices coloured with f (w2 ) or f (w4 ). But the contrary assumption means that there are vertex-disjoint paths P from w1 to w3 and Q from w2 to w4 in G − v, a contradiction. Hence this is a second NP-hard problem which has an absolute approximation algorithm. Indeed, the Four Colour Theorem implies that the chromatic number of a non-bipartite planar graph can only be 3 or 4. Using the polynomial-time algorithm of Robertson et al. [1996], which colours any given planar graph with four colours, one obtains an absolute approximation algorithm which uses at most one colour more than necessary. F¨urer and Raghavachari [1994] detected a third natural problem which can be approximated up to an additive error of one: Given an undirected graph, they look for a spanning tree whose maximum degree is minimum among all the spanning trees (the problem is a generalization of the H AMILTONIAN PATH P ROBLEM and thus NP-hard). Their algorithm also extends to a general case corresponding to the S TEINER T REE P ROBLEM: given a set T ⊆ V (G), find a tree S in G with V (T ) ⊆ V (S) such that the maximum degree of S is minimum. Singh and Lau [2007] found an extension to minimum weight spanning trees with bounded degrees. On the other hand, the following theorem tells that many problems do not have absolute approximation algorithms unless P = NP: Proposition 16.23. Let F and F 0 be (infinite) families of finite sets, and let P be the following optimization problem: Given a set E ∈ F and a function c : E → Z, find a set F ⊆ E with F ∈ F 0 and c(F) minimum (or decide that no such F exists). Then P has an absolute approximation algorithm if and only if P can be solved in polynomial time. Proof: Suppose there is a polynomial-time algorithm A and an integer k such that |A((E, c)) − OPT((E, c))| ≤ k for all instances (E, c) of P. We show how to solve P exactly in polynomial time. Given an instance (E, c) of P, we construct a new instance (E, c0 ), where 0 c (e) := (k + 1)c(e) for all e ∈ E. Obviously the optimum solutions remain the same. But if we now apply A to the new instance, |A((E, c0 )) − OPT((E, c0 ))| ≤ k and thus A((E, c0 )) = OPT((E, c0 )).

16.4 Approximation Schemes

413

Examples are the M INIMIZATION P ROBLEM F OR I NDEPENDENCE S YSTEMS and the M AXIMIZATION P ROBLEM F OR I NDEPENDENCE S YSTEMS (multiply c by −1), and thus all problems in the list of Section 13.1.

16.4 Approximation Schemes Recall the absolute approximation algorithm for the E DGE -C OLOURING P ROBLEM discussed in the previous section. This also implies a relative performance guarantee: Since one can easily decide if the edge-chromatic number is 1 or 2 (Proposition 16.15), Vizing’s Theorem yields a 43 -factor approximation algorithm. On the other hand, Theorem 16.14(a) implies that no k-factor approximation algorithm exists for any k < 43 (unless P = NP). Hence the existence of an absolute approximation algorithm does not imply the existence of a k-factor approximation algorithm for all k > 1. We shall meet a similar situation with the B IN -PACKING P ROBLEM in Chapter 18. This consideration suggests the following definition: Definition 16.24. Let P be an optimization problem with nonnegative weights. An asymptotic k-factor approximation algorithm for P is a polynomial-time algorithm A for P for which there exists a constant c such that 1 OPT(I ) − c ≤ A(I ) ≤ k OPT(I ) + c k for all instances I of P. We also say that A has asymptotic performance ratio k. The (asymptotic) approximation ratio of an optimization problem P with nonnegative weights is defined to be the infimum of all numbers k for which there exists an (asymptotic) k-factor approximation algorithm for P, or ∞ if there is no (asymptotic) approximation algorithm at all. For example, the above-mentioned E DGE -C OLOURING P ROBLEM has approximation ratio 43 (unless P = NP), but asymptotic approximation ratio 1 (not only in simple graphs; see Sanders and Steurer [2005]). Optimization problems with (asymptotic) approximation ratio 1 are of particular interest. For these problems we introduce the following notion: Definition 16.25. Let P be an optimization problem with nonnegative weights. An approximation scheme for P is an algorithm A accepting as input an instance I of P and an > 0 such that, for each fixed , A is a (1 + )-factor approximation algorithm for P. An asymptotic approximation scheme for P is a pair of algorithms (A, A0 ) with the following properties: A0 is a polynomial-time algorithm accepting a number > 0 as input and computing a number c . A accepts an instance I of P and an > 0 as input, and its output consists of a feasible solution for I satisfying

414

16 Approximation Algorithms

1 OPT(I ) − c ≤ A(I, ) ≤ (1 + ) OPT(I ) + c . 1+ For each fixed , the running time of A is polynomially bounded in size(I ). An (asymptotic) approximation scheme is called a fully polynomial (asymptotic) approximation scheme if the running time as well as the maximum size of any number occurring in the computation is bounded by a polynomial in size(I ) + size() + 1 . In some other texts one finds the abbreviations PTAS for (polynomial-time) approximation scheme and FPAS or FPTAS for fully polynomial approximation scheme. Apart from absolute approximation algorithms, a fully polynomial approximation scheme can be considered the best we may hope for when faced with an NP-hard optimization problem, at least if the cost of any feasible solution is a nonnegative integer (which can be assumed in many cases without loss of generality): Proposition 16.26. Let P = (X, (Sx )x∈X , c, goal) be an optimization problem where the values of c are nonnegative integers. Let A be an algorithm which, given an instance I of P and a number > 0, computes a feasible solution of I with 1 OPT(I ) ≤ A(I, ) ≤ (1 + ) OPT(I ) 1+ and whose running time is bounded by a polynomial in size(I ) + size(). Then P can be solved exactly in polynomial time. 1 Proof: Given an instance I , we first run A on (I, 1). We set := 1+2A(I,1) and observe that OPT(I ) < 1. Now we run A on (I, ). Since size() is polynomially bounded in size(I ), this procedure constitutes a polynomial-time algorithm. If P is a minimization problem, we have

A(I, ) ≤ (1 + ) OPT(I ) < OPT(I ) + 1, which, since c is integral, implies optimality. Similarly, if P is a maximization problem, we have A(I, ) ≥

1 OPT(I ) > (1 − ) OPT(I ) > OPT(I ) − 1. 1+

Unfortunately, a fully polynomial approximation scheme exists only for very few problems (see Theorem 17.11). Moreover we note that even the existence of a fully polynomial approximation scheme does not imply an absolute approximation algorithm; the K NAPSACK P ROBLEM is an example. In Chapters 17 and 18 we shall discuss two problems (K NAPSACK and B IN PACKING) which have a fully polynomial approximation scheme and a fully polynomial asymptotic approximation scheme, respectively. For many problems the two types of approximation schemes coincide:

16.5 Maximum Satisfiability

415

Theorem 16.27. (Papadimitriou and Yannakakis [1993]) Let P be an optimization problem with nonnegative weights. Suppose that for each constant k there is a polynomial-time algorithm which decides whether a given instance has optimum value at most k, and, if so, finds an optimum solution. Then P has an approximation scheme if and only if P has an asymptotic approximation scheme. Proof: The only-if-part is trivial, so suppose that P has an asymptotic approximation scheme (A, A0 ). We describe an approximation scheme for P. − 2 Let a fixed > 0 be given; we may assume < 1. We set 0 := 2++ 2 < 2 and first run A0 on the input 0 , yielding a constant c 0 . 2c 0 For a given instance I we next test whether OPT(I ) is at most . This is a constant for each fixed , so we can decide this in polynomial time and find an 2c 0 optimum solution if OPT(I ) ≤ . Otherwise we apply A to I and 0 and obtain a solution of value V , with 1 OPT(I ) − c 0 ≤ V ≤ (1 + 0 ) OPT(I ) + c 0 . 1 + 0 We claim that this solution is good enough. Indeed, we have c 0 < 2 OPT(I ) which implies OPT(I ) + OPT(I ) = (1 + ) OPT(I ) V ≤ (1 + 0 ) OPT(I ) + c 0 < 1 + 2 2 and V

1 OPT(I ) − OPT(I ) 0 (1 + ) 2 2 2++ = OPT(I ) − OPT(I ) 2 + 2 2 1 = + OPT(I ) − OPT(I ) 1+ 2 2 1 OPT(I ). = 1+ ≥

So the definition of an asymptotic approximation scheme is meaningful only for problems (such as bin-packing or colouring problems) whose restriction to a constant optimum value is still difficult. For many problems this restriction can be solved in polynomial time by some kind of complete enumeration.

16.5 Maximum Satisfiability The S ATISFIABILITY Problem was our first NP-complete problem. In this section we analyse the corresponding optimization problem:

416

16 Approximation Algorithms

M AXIMUM S ATISFIABILITY (M AX -S AT ) Instance: Task:

A set X of variables, a family Z of clauses over X , and a weight function c : Z → R+ . Find a truth assignment T of X such that the total weight of the clauses in Z that are satisfied by T is maximum.

As we shall see, approximating M AX -S AT is a nice example (and historically one of the first) for the algorithmic use of the probabilistic method. Let us first consider the following trivial randomized algorithm: set each variable independently true with probability 12 . Obviously this algorithm satisfies each clause Z with probability 1 − 2−|Z | . Let us write r for random variables which are true with probability 12 and false otherwise, and let R = (r, r, . . . , r ) be the random variable uniformly distributed over all truth assignments (several copies of r are independent of each other). If we write c(T ) for the total weight of the clauses satisfied by the truth assignment T , the expected total weight of the clauses satisfied by R is X c(Z ) Prob(R satisfies Z ) Exp (c(R)) = Z ∈Z

=

X

c(Z ) 1 − 2−|Z |

(16.5)

Z ∈Z

≥

1 − 2− p

X

c(Z ),

Z ∈Z

where p := min Z ∈Z |Z |; Exp and Prob P denote expectation and probability. Since the optimum cannot exceed Z ∈Z c(Z ), R is expected to yield a solution within a factor 1−21 − p of the optimum. But what we would really like to have is a deterministic approximation algorithm. In fact, we can turn our (trivial) randomized algorithm into a deterministic algorithm while preserving the performance guarantee. This step is often called derandomization. Let us fix the truth assignment step by step. Suppose X = {x1 , . . . , xn }, and we have already fixed a truth assignment T for x1 , . . . , xk (0 ≤ k < n). If we now set xk+1 , . . . , xn randomly, setting each variable independently true with probability 12 , we will satisfy clauses of expected total weight e0 = Exp(c(T (x1 ), . . . , T (xk ), r, . . . , r )). If we set xk+1 true (false), and then set xk+2 , . . . , xn randomly, the satisfied clauses will have some expected total weight e1 (e2 , respectively). e1 and e2 can be thought of as conditional expectations. 2 Trivially e0 = e1 +e 2 , so at least one of e1 , e2 must be at least e0 . We set x k+1 to be true if e1 ≥ e2 and false otherwise. This is sometimes called the method of conditional probabilities.

16.5 Maximum Satisfiability

417

J OHNSON ’ S A LGORITHM F OR M AX -S AT Input: Output:

A set X = {x1 , . . . , xn } of variables, a family Z of clauses over X , and a weight function c : Z → R+ . A truth assignment T : X → {true, false}.

For k := 1 to n do: If Exp(c(T (x1 ), . . . , T (xk−1 ), true, r, . . . , r )) ≥ Exp(c(T (x1 ), . . . , T (xk−1 ), false, r, . . . , r )) then set T (xk ) := true else set T (xk ) := false.

1

The expectations can be easily computed with (16.5). Theorem 16.28. (Johnson [1974]) J OHNSON ’ S A LGORITHM F OR M AX -S AT is a 1−21 − p -factor approximation algorithm for M AX -S AT, where p is the minimum cardinality of a clause. Proof: Let us define the conditional expectation sk := Exp(c(T (x1 ), . . . , T (xk ), r, . . . , r )) for k = 0, . . . , n. Observe that sn = c(T ) is the total weight P of the clauses satisfied by our algorithm, while s0 = Exp(c(R)) ≥ 1 − 2− p Z ∈Z c(Z ) by (16.5). = 1, . . . , n). So Furthermore, si ≥Psi−1 by the choice of T (xi ) in 1 (for i P c(Z ). Since the optimum is at most s n ≥ s0 ≥ 1 − 2 − p Z ∈Z Z ∈Z c(Z ), the proof is complete. Since p ≥ 1, we have a 2-factor approximation algorithm. However, this is not too interesting as there is a much simpler 2-factor approximation algorithm: either set all variables true or all false, whichever is better. However, Chen, Friesen and Zheng [1999] showed that J OHNSON ’ S A LGORITHM F OR M AX -S AT is indeed a 3 2 -factor approximation algorithm. If there are no one-element clauses ( p ≥ 2), it is a 43 -factor approximation algorithm (by Theorem 16.28), for p ≥ 3 it is an 78 -factor approximation algorithm. Yannakakis [1994] found a 43 -factor approximation algorithm for the general case using network flow techniques. We shall describe a simpler 43 -factor approximation algorithm due to Goemans and Williamson [1994]. It is straightforward to translate M AX -S AT into an integer linear program: If we have variables X = {x1 , . . . , xn }, clauses Z = {Z 1 , . . . , Z m }, and weights c1 , . . . , cm , we can write max

m X

cj z j

j=1

s.t.

zj

≤

X i:xi ∈Z j

yi , z j

∈ {0, 1}

yi +

X

(1 − yi )

( j = 1, . . . , m)

i:xi ∈Z j

(i = 1, . . . , n, j = 1, . . . , m).

418

16 Approximation Algorithms

Here yi = 1 means that variable xi is true, and z j = 1 means that clause Z j is satisfied. Now consider the LP relaxation: max

m X

cj z j

j=1

s.t.

zj

X

≤

i:xi ∈Z j

yi yi zj zj

≤ ≥ ≤ ≥

yi +

X

(1 − yi )

( j = 1, . . . , m)

i:xi ∈Z j

(i = 1, . . . , n) (i = 1, . . . , n) ( j = 1, . . . , m) ( j = 1, . . . , m).

1 0 1 0

(16.6)

Let (y ∗ , z ∗ ) be an optimum solution of (16.6). Now independently set each variable xi true with probability yi∗ . This step is known as randomized rounding, a technique which has been introduced by Raghavan and Thompson [1987]. The above method constitutes another randomized algorithm for M AX -S AT, which can be derandomized as above. Let r p be the random variable which is true with probability p and false otherwise.

G OEMANS -W ILLIAMSON A LGORITHM F OR M AX -S AT Input: Output:

A set X = {x1 , . . . , xn } of variables, a family Z of clauses over X , and a weight function c : Z → R+ . A truth assignment T : X → {true, false}.

1

Solve the linear program (16.6); let (y ∗ , z ∗ ) be an optimum solution.

2

For k := 1 to n do: ∗ , . . . , r y∗ ) If Exp(c(T (x1 ), . . . , T (xk−1 ), true, r yk+1 n ∗ , . . . , r y∗ ) ≥ Exp(c(T (x1 ), . . . , T (xk−1 ), false, r yk+1 n then set T (xk ) := true else set T (xk ) := false.

Theorem 16.29. (Goemans and Williamson [1994]) The G OEMANS -W ILLIAM 1 q -factor approximation algorithm, SON A LGORITHM F OR M AX -S AT is a 1 1− 1− q

where q is the maximum cardinality of a clause. Proof: Let us write ∗ , . . . , r y ∗ )) sk := Exp(c(T (x1 ), . . . , T (xk ), r yk+1 n

for k = 0, . . . , n. We again have si ≥ si−1 for i = 1, . . . , n and sn = c(T ) is the total weight of clauses satisfied by our algorithm. So it remains to estimate s0 = Exp(c(R y ∗ )), where R y ∗ = (r y1∗ , . . . , r yn∗ ). For j = 1, . . . , m, the probability that the clause Z j is satisfied by R y ∗ is

16.5 Maximum Satisfiability

1−

Y

(1 − yi∗ ) ·

i:xi ∈Z j

419

Y

yi∗ .

i:xi ∈Z j

Since the geometric mean is always less than or equal to the arithmetic mean, this probability is at least |Z j | X X 1 1− (1 − yi∗ ) + yi∗ |Z j | i:xi ∈Z j

i:xi ∈Z j

|Z j |

= 1 − 1 −

1 |Z j |

X

yi∗ +

i:xi ∈Z j

≥ 1− 1− ≥

z ∗j

X

(1 − yi∗ )

i:xi ∈Z j

!|Z j |

|Z j | ! 1 |Z j | ∗ 1− 1− zj . |Z j |

To prove the last inequality, observe that for any 0 ≤ a ≤ 1 and any k ∈ N ! a k 1 k 1− 1− ≥ a 1− 1− k k holds: both sides of the inequality are equal for a ∈ {0, 1}, and the left-hand side (as a function of a) is concave, while the right-hand side is linear. So we have s0 = Exp(c(R y ∗ )) =

m X

c j Prob(R y ∗ satisfies Z j )

j=1

≥

m X

cj

j=1

1 1− 1− |Z j |

|Z j | !

z ∗j

m 1 q X ∗ ≥ 1− 1− cj z j q j=1

(observe that the sequence

1−

1 k

k

is monotonously increasing and conP ∗ verges to 1e ). Since the optimum is less than or equal to m j=1 z j c j , the optimum value of the LP relaxation, the proof is complete. q e e -factor approximation algorithm ( e−1 is Since 1 − q1 < 1e , we have an e−1 about 1.582). k∈N

420

16 Approximation Algorithms

We now have two similar algorithms that behave differently: the first one is better for long clauses, while the second is better for short clauses. Hence it is natural to combine them: Theorem 16.30. (Goemans and Williamson [1994]) The following is a 43 -factor approximation algorithm for M AX -S AT: run both J OHNSON ’ S A LGORITHM F OR M AX -S AT and the G OEMANS -W ILLIAMSON A LGORITHM F OR M AX -S AT and choose the better of the two solutions. Proof: We use the notation of the above proofs. The algorithm returns a truth assignment satisfying clauses of total weight at least max{Exp(c(R)), Exp(c(R y ∗ ))} 1 Exp(c(R)) + Exp(c(R y ∗ )) ≥ 2 ! |Z j | ! m 1X 1 ≥ 1 − 2−|Z j | c j + 1 − 1 − z ∗j c j 2 |Z j | j=1 |Z j | ! m X 1 1 2 − 2−|Z j | − 1 − ≥ z ∗j c j 2 |Z j | j=1

≥

m 3X

4

z ∗j c j .

j=1

k For the last inequality observe that 2 − 2−k − 1 − k1 ≥ 32 for all k ∈ N: for k k ∈ {1, 2} we have equality; for k ≥ 3 we have 2−2−k − 1 − k1 ≥ 2− 18 − 1e > 32 . P ∗ Since the optimum is at least m j=1 z j c j , the theorem is proved. Slightly better approximation algorithms for M AX -S AT (using semidefinite programming) have been found; see Goemans and Williamson [1995], Mahajan and Ramesh [1999], and Feige and Goemans [1995]. The currently best known algorithm achieves a performance ratio of 1.270 (Asano [2006]). Indeed, Bellare and Sudan [1994] showed that approximating M AX -S AT to 74 within a factor of 73 is NP-hard. Even for M AX -3S AT (which is M AX -S AT restricted to instances where each clause has exactly three literals) no approximation scheme exists (unless P = NP), as we shall show in the next section.

16.6 The PCP Theorem Many non-approximability results are based on a deep theorem which gives a new characterization of the class NP. Recall that a decision problem belongs to NP if and only if there is a polynomial-time certificate-checking algorithm. Now

16.6 The PCP Theorem

421

we consider randomized certificate-checking algorithms that read the complete instance but only a small part of the certificate to be checked. They always accept yes-instances with correct certificates but sometimes also accept no-instances. Which bits of the certificate are read is decided randomly in advance; more precisely this decision depends on the instance x and on O(log(size(x))) random bits. We now formalize this concept. If s is a string and t ∈ Nk , then st denotes the string of length k whose i-th component is the ti -th component of s (i = 1, . . . , k). Definition 16.31. A decision problem (X, Y ) belongs to the class PCP(log n,1) if there is a polynomial p and a constant k ∈ N, a function n o f : (x, r ) : x ∈ X, r ∈ {0, 1}blog( p(size(x)))c → Nk computable in polynomial time, with f (x, r ) ∈ {1, . . . , b p(size(x))c}k for all x and r , and a decision problem (X 0 , Y 0 ) in P, where X 0 := {(x, π, γ ) : x ∈ X, π ∈ {1, . . . , b p(size(x))c}k , γ ∈ {0, 1}k }, such that for any instance x ∈ X : If x ∈ Y then there exists a c ∈ {0, 1}b p(size(x))c with Prob (x, f (x, r ), c f (x,r ) ) ∈ Y 0 = 1. If x ∈ / Y then Prob (x, f (x, r ), c f (x,r ) ) ∈ Y 0 < 12 for all c ∈ {0, 1}b p(size(x))c . Here the probability is taken over the uniform distribution of random strings r ∈ {0, 1}blog( p(size(x)))c . The letters “PCP” stand for “probabilistically checkable proof ”. The parameters log n and 1 reflect that, for an instance of size n, O(log n) random bits are used and O(1) bits of the certificate are read. For any yes-instance there is a certificate which is always accepted; while for no-instances no string is accepted as a certificate with probability 12 or more. Note that this error probability 12 can be replaced equivalently by any number between zero and one (Exercise 19). Proposition 16.32. PCP(log n, 1) ⊆ NP. Proof: Let (X, Y ) ∈ PCP(log n, 1), and let p, k, f, (X 0 , Y 0 ) be given as in Def 00 inition 16.31. Let X := (x, c) : x ∈ X, c ∈ {0, 1}b p(size(x))c , and let Y 00 := (x, c) ∈ X 00 : Prob (x, f (x, r ), c f (x,r ) ) ∈ Y 0 = 1 . To show that (X, Y ) ∈ NP it suffices to show that (X 00 , Y 00 ) ∈ P. But since there are only 2blog( p(size(x)))c , i.e. at most p(size(x)) many strings r ∈ {0, 1}blog( p(size(x)))c , we can try them all. For each one we compute f (x, r ) and test whether (x, f (x, r ), c f (x,r ) ) ∈ Y 0 (we use that (X 0 , Y 0 ) ∈ P). The overall running time is polynomial in size(x). Now the surprising result is that these randomized verifiers, which read only a constant number of bits of the certificate, are as powerful as the standard (deterministic) certificate-checking algorithms which have the full information. This is the so-called PCP Theorem:

422

16 Approximation Algorithms

Theorem 16.33. (Arora et al. [1998]) NP = PCP(log n, 1).

The proof of NP ⊆ PCP(log n, 1) is very difficult and beyond the scope of this book. It is based on earlier (and weaker) results of Feige et al. [1996] and Arora and Safra [1998]. For a self-contained proof of the PCP Theorem 16.33, see also Arora [1994], Hougardy, Pr¨omel and Steger [1994], or Ausiello et al. [1999]. Stronger results were found subsequently by Bellare, Goldreich and Sudan [1998] and H˚astad [2001]. For example, the number k in Definition 16.31 can be chosen to be 9. A new proof of the PCP Theorem was proposed by Dinur [2007]. We show some of its consequences for the non-approximability of combinatorial optimization problems. We start with the M AXIMUM C LIQUE P ROBLEM and the M AXIMUM S TABLE S ET P ROBLEM: given an undirected graph G, find a clique, or a stable set, of maximum cardinality in G. Recall Proposition 2.2 (and Corollary 15.24): The problems of finding a maximum clique, a maximum stable set, or a minimum vertex cover are all equivalent. However, the 2-factor approximation algorithm for the M INIMUM V ERTEX C OVER P ROBLEM (Section 16.1) does not imply an approximation algorithm for the M AX IMUM S TABLE S ET P ROBLEM or the M AXIMUM C LIQUE P ROBLEM . Namely, it can happen that the algorithm returns a vertex cover C of size n − 2, while the optimum is n2 −1 (where n = |V (G)|). The complement V (G)\C is then a stable set of cardinality 2, but the maximum stable set has cardinality n2 + 1. This example shows that transferring an algorithm to another problem via a polynomial transformation does not in general preserve its performance guarantee. We shall consider a restricted type of transformation in the next section. Here we deduce a non-approximability result for the M AXIMUM C LIQUE P ROBLEM from the PCP Theorem: Theorem 16.34. (Arora and Safra [1998]) Unless P = NP there is no 2-factor approximation algorithm for the M AXIMUM C LIQUE P ROBLEM. Proof: Let P = (X, Y ) be some NP-complete problem. By the PCP Theorem 16.33, P ∈ PCP(log n, 1), so let p, k, f , P 0 := (X 0 , Y 0 ) be as in Definition 16.31. For any given x ∈ X we construct a graph G x as follows. Let n o V (G x ) := (r, a) : r ∈ {0, 1}blog( p(size(x)))c , a ∈ {0, 1}k , (x, f (x, r ), a) ∈ Y 0 (representing all “accepting runs” of the randomized certificate checking algorithm). Two vertices (r, a) and (r 0 , a 0 ) are joined by an edge if ai = a j0 whenever the i-th component of f (x, r ) equals the j-th component of f (x, r 0 ). Since P 0 ∈ P and there are only a polynomial number of random strings, G x can be computed in polynomial time (and has polynomial size).

16.6 The PCP Theorem

423

If x ∈ Y then by definition there exists a certificate c ∈ {0, 1}b p(size(x))c such that (x, f (x, r ), c f (x,r ) ) ∈ Y 0 for all r ∈ {0, 1}blog( p(size(x)))c . Hence there is a clique of size 2blog( p(size(x)))c in G x . On the other hand, if x ∈ / Y then there is no clique of size 12 2blog( p(size(x)))c in G x : Suppose (r (1) , a (1) ), . . . , (r (t) , a (t) ) are the vertices of a clique. Then r (1) , . . . , r (t) ( j) are pairwise different. We set ci := ak whenever the k-th component of f (x, r ( j) ) equals i, and set the remaining components of c (if any) arbitrarily. This way we obtain a certificate c with (x, f (x, r (i) ), c f (x,r (i) ) ) ∈ Y 0 for all i = 1, . . . , t. If x∈ / Y we have t < 12 2blog( p(size(x)))c . So any 2-factor approximation algorithm for the M AXIMUM C LIQUE P ROBLEM is able to decide if x ∈ Y , i.e. to solve P. Since P is NP-complete, this is possible only if P = NP. The reduction in the above proof is due to Feige et al. [1996]. Since the error probability 12 in Definition 16.31 can be replaced by any number between 0 and 1 (Exercise 19), we get that there is no ρ-factor approximation algorithm for the M AXIMUM C LIQUE P ROBLEM for any ρ ≥ 1 (unless P = NP). With more effort Zuckerman [2006] showed that, unless P = NP, no polynomialtime algorithm computes the maximum size of a clique in any graph with n vertices up to a factor of n 1− , for any fixed > 0. The best known algorithm guarantees to k log3 n in this case (Feige [2004]). Of course, all this also find a clique of size n(log log n)2 holds for the M AXIMUM S TABLE S ET P ROBLEM (by considering the complement of the given graph). Now we turn to the following restriction of M AX -S AT:

M AX -3S AT Instance: Task:

A set X of variables and a family Z of clauses over X , each with exactly three literals. Find a truth assignment T of X such that the number of clauses in Z that are satisfied by T is maximum.

In Section 16.5 we had a simple 87 -factor approximation algorithm for M AX 3S AT, even for the weighted form (Theorem 16.28). H˚astad [2001] showed that this is best possible: no ρ-factor approximation algorithm for M AX -3S AT can exist for any ρ < 78 unless P = NP. Here we prove the following weaker result: Theorem 16.35. (Arora et al. [1998]) Unless P = NP there is no approximation scheme for M AX -3S AT. Proof: Let P = (X, Y ) be some NP-complete problem. By the PCP Theorem 16.33, P ∈ PCP(log n, 1), so let p, k, f , P 0 := (X 0 , Y 0 ) be as in Definition 16.31. For any given x ∈ X we construct a 3S AT-instance Jx . Namely, for each random string r ∈ {0, 1}blog( p(size(x)))c we define a family Zr of 3S AT-clauses (the union of all these families will be Jx ). We first construct a family Zr0 of clauses with an arbitrary number of literals and then apply Proposition 15.21.

424

16 Approximation Algorithms

So let r ∈ {0, 1}blog( p(size(x)))c and f (x, r ) = (t1 , . . . , tk ). Let {a (1) , . . . , a (sr ) } be the set of strings a ∈ {0, 1}k for which (x, f (x, r ), a) ∈ Y 0 . If sr = 0 then we simply set Z 0 := {{y}, { y¯ }}, where y is some variable not used anywhere else. Otherwise let c ∈ {0, 1}b p(size(x))c . We have that (x, f (x, r ), c f (x,r ) ) ∈ Y 0 if and only if ! sr k _ ^ ( j) cti = ai . j=1

i=1

This is equivalent to ^

( j)

cti j = ai j

(i 1 ,...,i sr )∈{1,...,k}sr

sr _

.

j=1

This conjunction of clauses can be constructed in polynomial time because P 0 ∈ P and k is a constant. By introducing Boolean variables π1 , . . . , πb p(size(x))c representing the bits c1 , . . . , cb p(size(x))c we obtain a family Zr0 of k sr clauses (each with sr literals) such that Zr0 is satisfied if and only if (x, f (x, r ), c f (x,r ) ) ∈ Y 0 . By Proposition 15.21, we can rewrite each Zr0 equivalently as a conjunction of 3S AT-clauses, where the number of clauses increases by at most a factor of max{sr − 2, 4}. Let this family of clauses be Zr . Since sr ≤ 2k , each Zr consists k of at most l := k 2 max{2k − 2, 4} 3S AT-clauses. Our 3S AT-instance Jx is the union of all the families Zr for all r . Note that Jx can be computed in polynomial time. Now if x is a yes-instance, then there exists a certificate c as in Definition 16.31. This c immediately defines a truth assignment satisfying Jx . On the other hand, if x is a no-instance, then only 21 of the formulas Zr are simultaneously satisfiable. So in this case any truth assignment leaves at least a fraction of 2l1 of the clauses unsatisfied. 2l So any k-factor approximation algorithm for M AX -3S AT with k < 2l−1 satisfies 2l−1 1 more than a fraction of 2l = 1 − 2l of the clauses of any satisfiable instance. Hence such an algorithm can decide whether x ∈ Y or not. Since P is NP-complete, such an algorithm cannot exist unless P = NP.

16.7 L-Reductions Our goal is to show, for other problems than M AX -3S AT, that they have no approximation scheme unless P = NP. As with the NP-completeness proofs (Section 15.5), it is not necessary to have a direct proof using the definition of PCP(log n, 1) for each problem. Rather we use a certain type of reduction which preserves approximability (general polynomial transformations do not): Definition 16.36. Let P = (X, (Sx )x∈X , c, goal) and P 0 = (X 0 , (Sx0 )x∈X 0 , c0 , goal0 ) be two optimization problems with nonnegative weights. An L-reduction from P

16.7 L-Reductions

425

to P 0 is a pair of functions f and g, both computable in polynomial time, and two constants α, β > 0 such that for any instance x of P: (a) f (x) is an instance of P 0 with OPT( f (x)) ≤ α OPT(x); (b) For any feasible solution y 0 of f (x), g(x, y 0 ) is a feasible solution of x such that |c(x, g(x, y 0 )) − OPT(x)| ≤ β|c0 ( f (x), y 0 ) − OPT( f (x))|. We say that P is L-reducible to P 0 if there is an L-reduction from P to P 0 . The letter “L” in the term L-reduction stands for “linear”. L-reductions were introduced by Papadimitriou and Yannakakis [1991]. The definition immediately implies that L-reductions can be composed: Proposition 16.37. Let P, P 0 , P 00 be optimization problems with nonnegative weights. If ( f, g, α, β) is an L-reduction from P to P 0 and ( f 0 , g 0 , α 0 , β 0 ) is an Lreduction from P 0 to P 00 , then their composition ( f 00 , g 00 , αα 0 , ββ 0 ) is an L-reduction from P to P 00 , where f 00 (x) = f 0 ( f (x)) and g 00 (x, y 00 ) = g(x, g 0 (x 0 , y 00 )). The decisive property of L-reductions is that they preserve approximability: Theorem 16.38. (Papadimitriou and Yannakakis [1991]) Let P and P 0 be two optimization problems with nonnegative weights. Let ( f, g, α, β) be an L-reduction from P to P 0 . If there is an approximation scheme for P 0 , then there is an approximation scheme for P. Proof: Given an instance x of P and a number 0 < < 1, we apply the approximation scheme for P 0 to f (x) and 0 := 2αβ . We obtain a feasible solution 0 0 y of f (x) and finally return y := g(x, y ), a feasible solution of x. Since |c(x, y) − OPT(x)| ≤ β|c0 ( f (x), y 0 ) − OPT( f (x))| n ≤ β max (1 + 0 ) OPT( f (x)) − OPT( f (x)), o 1 OPT( f (x)) OPT( f (x)) − 0 1+ ≤ β 0 OPT( f (x)) ≤ αβ 0 OPT(x) OPT(x) = 2 we get c(x, y) ≤ OPT(x) + |c(x, y) − OPT(x)| ≤

1+ OPT(x) 2

and c(x, y) ≥ OPT(x)−| OPT(x)−c(x, y)| ≥

1−

1 OPT(x) > OPT(x), 2 1+

so this constitutes an approximation scheme for P.

This theorem together with Theorem 16.35 motivates the following definition:

426

16 Approximation Algorithms

Definition 16.39. An optimization problem P with nonnegative weights is called MAXSNP-hard if M AX -3S AT is L-reducible to P. The name MAXSNP refers to a class of optimization problems introduced by Papadimitriou and Yannakakis [1991]. Here we do not need this class, so we omit its (nontrivial) definition. Corollary 16.40. Unless P = NP there is no approximation scheme for any MAXSNP-hard problem. Proof:

Directly from Theorems 16.35 and 16.38.

We shall show MAXSNP-hardness for several problems by describing Lreductions. We start with a restricted version of M AX -3S AT:

3-O CCURRENCE M AX -S AT P ROBLEM Instance:

Task:

A set X of variables and a family Z of clauses over X , each with at most three literals, such that no variable occurs in more than three clauses. Find a truth assignment T of X such that the number of clauses in Z that are satisfied by T is maximum.

That this problem is NP-hard can be proved by a simple transformation from 3S AT (or M AX -3S AT), cf. Exercise 9 of Chapter 15. Since this transformation is not an L-reduction, it does not imply MAXSNP-hardness. We need a more complicated construction, using so-called expander graphs: Definition 16.41. Let G be an undirected graph and γ > 0 a constant. G is a we have |0(A)| ≥ γ |A|. γ -expander if for each A ⊆ V (G) with |A| ≤ |V (G)| 2 For example, a complete graph is a 1-expander. However, one is interested in expanders with a small number of edges. We cite the following theorem without its quite complicated proof: Theorem 16.42. (Ajtai [1994]) There exists a positive constant γ such that for any given even integer n ≥ 4, a 3-regular γ -expander with n vertices can be constructed in O(n 3 log3 n) time. The following corollary was mentioned (and used) by Papadimitriou [1994], and a correct proof was given by Fern´andez-Baca and Lagergren [1998]: Corollary 16.43. For any given integer n ≥ 3, a digraph G with O(n) vertices and a set S ⊆ V (G) of cardinality n with the following properties can be constructed in O(n 3 log3 n) time: |δ − (v)| + |δ + (v)| ≤ 3 for each v ∈ V (G); |δ − (v)| + |δ + (v)| = 2 for each v ∈ S; and |δ + (A)| ≥ min{|S ∩ A|, |S \ A|} for each A ⊆ V (G).

16.7 L-Reductions

427

l m

Proof: Let γ > 0 be the constant of Theorem 16.42, and let k := γ1 . We first construct a 3-regular γ -expander H with n or n + 1 vertices, using Theorem 16.42. We replace each edge {v, w} by k parallel edges (v, w) and k parallel edges (w, v). Let the resulting digraph be H 0 . Note that for any A ⊆ V (H 0 ) with 0 )| |A| ≤ |V (H we have 2 |δ + H 0 (A)| = k|δ H (A)| ≥ k|0 H (A)| ≥ kγ |A| ≥ |A|. Similarly we have for any A ⊆ V (H 0 ) with |A| >

|V (H 0 )| : 2

0 0 |δ + H 0 (A)| = k|δ H (V (H ) \ A)| ≥ k|0 H (V (H ) \ A)|

≥ kγ |V (H 0 ) \ A| ≥ |V (H 0 ) \ A|. 0 So in both cases we have |δ + H 0 (A)| ≥ min{|A|, |V (H ) \ A|}. 0 Now we split up each vertex v ∈ V (H ) into 6k +1 vertices xv,i (i = 0, . . . , 6k) such that each vertex except xv,0 has degree 1. For each vertex xv,i we now add vertices wv,i, j and yv,i, j ( j = 0, . . . , 6k) connected by a path of length 12k +2 with vertices wv,i,0 , wv,i,1 , . . . , wv,i,6k , xv,i , yv,i,0 , . . . , yv,i,6k in this order. Finally we add edges (yv,i, j , wv, j,i ) for all v ∈ V (H 0 ), all i ∈ {0, . . . , 6k} and all j ∈ {0, . . . , 6k} \ {i}. Altogether we have a vertex set Z v of cardinality (6k + 1)(12k + 3) for each v ∈ V (H 0 ). The overall resulting graph G has |V (H 0 )|(6k + 1)(12k + 3) = O(n) vertices, each of degree two or three. By the construction, G[Z v ] contains min{|X 1 |, |X 2 |} vertex-disjoint paths from X 1 to X 2 for any pair of disjoint subsets X 1 , X 2 of {xv,i : i = 0, . . . , 6k}. We choose S to be an n-element subset of {xv,0 : v ∈ V (H 0 )}; note that each of these vertices has one entering and one leaving edge. It remains to prove that |δ + (A)| ≥ min{|S ∩ A|, |S \ A|} for each A ⊆ V (G). We prove this byS induction on |{v ∈ V (H 0 ) : ∅ 6= A ∩ Z v 6= Z v }|. If this number is zero, i.e. A = v∈B Z v for some B ⊆ V (H 0 ), then we have + 0 |δG (A)| = |δ + H 0 (B)| ≥ min{|B|, |V (H ) \ B|} ≥ min{|S ∩ A|, |S \ A|}.

Otherwise let v ∈ V (H 0 ) with ∅ 6= A ∩ Z v 6= Z v . Let P := {xv,i : i = 0, . . . , 6k} ∩ A and Q := {xv,i : i = 0, . . . , 6k} \ A. If |P| ≤ 3k, then by the property of G[Z v ] we have + |E G (Z v ∩ A, Z v \ A)| ≥ |P| = |P \ S| + |P ∩ S| + ≥ |E G (A \ Z v , A ∩ Z v )| + |P ∩ S|.

By applying the induction hypothesis to A \ Z v we therefore get + + |δG (A)| ≥ |δG (A \ Z v )| + |P ∩ S|

≥ min{|S ∩ (A \ Z v )|, |S \ (A \ Z v )|} + |P ∩ S| ≥ min{|S ∩ A|, |S \ A|}.

428

16 Approximation Algorithms

Similarly, if |P| ≥ 3k + 1, then |Q| ≤ 3k and by the property of G[Z v ] we have + |E G (Z v ∩ A, Z v \ A)| ≥ |Q| = |Q \ S| + |Q ∩ S| + (Z v \ A, V (G) \ (A ∪ Z v ))| + |Q ∩ S|. ≥ |E G

By applying the induction hypothesis to A ∪ Z v we therefore get + + (A)| ≥ |δG (A ∪ Z v )| + |Q ∩ S| |δG

≥ min{|S ∩ (A ∪ Z v )|, |S \ (A ∪ Z v )|} + |Q ∩ S| ≥ min{|S ∩ A|, |S \ A|}. Now we can prove: Theorem 16.44. (Papadimitriou and Yannakakis [1991], Papadimitriou [1994], Fern´andez-Baca and Lagergren [1998]) The 3-O CCURRENCE M AX -S AT P ROB LEM is MAXSNP-hard. Proof: We describe an L-reduction ( f, g, α, β) from M AX -3S AT. To define f , let (X, Z) be an instance of M AX -3S AT. For each variable x ∈ X which occurs in more than three, say in k clauses, we modify the instance as follows. We replace x by a new and distinct variable in each clause. This way we introduce new variables x1 , . . . , xk . We introduce additional constraints (and further variables) which ensure, roughly speaking, that it is favourable to assign the same truth value to all the variables x1 , . . . , xk . We construct G and S as in Corollary 16.43 and rename the vertices such that S = {1, . . . , k}. Now for each vertex v ∈ V (G) \ S we introduce a new variable xv , and for each edge (v, w) ∈ E(G) we introduce a clause {xv , xw }. In total we have added at most 2 1 1 1 3 (k + 1) 6 k +1 12 + 3 ≤ 315 2 γ γ γ new clauses, where γ is again the constant of Theorem 16.42. Applying the above substitution for each variable we obtain an instance (X 0 , Z 0 ) = f (X, Z) of the 3-O CCURRENCE M AX -S AT P ROBLEM with |Z 0 | ≤ |Z| + 315

2 2 1 1 3|Z| ≤ 946 |Z|. γ γ

Hence 2 2 1 1 |Z| ≤ 1892 OPT(X, Z), OPT(X , Z ) ≤ |Z | ≤ 946 γ γ 0

0

0

16.7 L-Reductions

429

because at least half of the clauses of a M AX -S AT-instance can be satisfied (either l m2 by setting all variables true or all false). So we can set α := 1892 γ1 . To describe g, let T 0 be a truth assignment of X 0 . We first construct a truth assignment T 00 of X 0 satisfying at least as many clauses of Z 0 as T 0 , and satisfying all new clauses (corresponding to edges of the graphs G above). Namely, for any variable x occurring more than three times in (X, Z), let G be the graph constructed above, and let A := {v ∈ V (G) : T 0 (xv ) = true}. If |S ∩ A| ≥ |S \ A| then we set T 00 (xv ) := true for all v ∈ V (G), otherwise we set T 00 (xv ) := false for all v ∈ V (G). It is clear that all new clauses (corresponding to edges) are satisfied. There are at most min{|S ∩ A|, |S \ A|} old clauses satisfied by T 0 but not by T 00 . On the other hand, T 0 does not satisfy any of the clauses {xv , xw } for + (A). By the properties of G, the number of these clauses is at least (v, w) ∈ δG min{|S ∩ A|, |S \ A|}. Now T 00 yields a truth assignment T = g(X, Z, T 0 ) of X in the obvious way: Set T (x) := T 00 (x) = T 0 (x) for x ∈ X ∩ X 0 and T (x) := T 00 (xi ) if xi is any variable replacing x in the construction from (X, Z) to (X 0 , Z 0 ). T violates as many clauses as T 00 . So if c(X, Z, T ) denotes the number of clauses in instance (X, Z) that are satisfied by T , we conclude |Z| − c(X, Z, T ) = |Z 0 | − c(X 0 , Z 0 , T 00 ) ≤ |Z 0 | − c(X 0 , Z 0 , T 0 ).

(16.7)

On the other hand, any truth assignment T of X leads to a truth assignment T 0 of X 0 violating the same number of clauses (by setting the variables xv (v ∈ V (G)) uniformly to T (x) for each variable x and the corresponding graph G in the above construction). Hence |Z| − OPT(X, Z) ≥ |Z 0 | − OPT(X 0 , Z 0 ).

(16.8)

Combining (16.7) and (16.8) we get | OPT(X, Z) − c(X, Z, T )| = (|Z| − c(X, Z, T )) − (|Z| − OPT(X, Z)) ≤ OPT(X 0 , Z 0 ) − c(X 0 , Z 0 , T 0 ) = | OPT(X 0 , Z 0 ) − c(X 0 , Z 0 , T 0 )|, where T = g(X, Z, T 0 ). So ( f, g, α, 1) is indeed an L-reduction.

This result is the starting point of several MAXSNP-hardness proofs. For example: Corollary 16.45. (Papadimitriou and Yannakakis [1991]) The M AXIMUM S TA BLE S ET P ROBLEM restricted to graphs with maximum degree 4 is MAXSNP-hard. Proof: The construction of the proof of Theorem 15.23 defines an L-reduction from the 3-O CCURRENCE M AX -S AT P ROBLEM to the M AXIMUM S TABLE S ET P ROBLEM restricted to graphs with maximum degree 4: for each instance (X, Z)

430

16 Approximation Algorithms

a graph G is constructed such that from each truth assignment satisfying k clauses one easily obtains a stable set of cardinality k, and vice versa. Indeed, the M AXIMUM S TABLE S ET P ROBLEM is MAXSNP-hard even when restricted to 3-regular graphs (Berman and Fujito [1999]). On the other hand, a simple greedy algorithm, which in each step chooses a vertex v of minimum degree and deletes v and all its neighbours, is a (k+2) 3 -factor approximation algorithm for the M AXIMUM S TABLE S ET P ROBLEM in graphs with maximum degree k (Halld´orsson and Radhakrishnan [1997]). For k = 4 this gives a performance ratio of 2 which is better than the ratio 8 we get from the following proof (using the 2-factor approximation algorithm for the M INIMUM V ERTEX C OVER P ROBLEM). Theorem 16.46. (Papadimitriou and Yannakakis [1991]) The M INIMUM V ER TEX C OVER P ROBLEM restricted to graphs with maximum degree 4 is MAXSNPhard. Proof: Consider the trivial transformation from the M AXIMUM S TABLE S ET P ROBLEM (Proposition 2.2) with f (G) := G and g(G, X ) := V (G) \ X for all graphs G and all X ⊆ V (G). Although this is not an L-reduction in general, it is an L-reduction if restricted to graphs with maximum degree 4, as we shall show. If G has maximum degree 4, there exists a stable set of cardinality at least |V (G)| 5 . So if we denote by α(G) the maximum cardinality of a stable set and by τ (G) the minimum cardinality of a vertex cover we have 1 1 (|V (G)| − α(G)) = τ (G) 4 4 and α(G) − |X | = |V (G) \ X | − τ (G) for any stable set X ⊆ V (G). Hence ( f, g, 4, 1) is an L-reduction. α(G) ≥

See Clementi and Trevisan [1999] and Chleb´ık and Chleb´ıkov´a [2006] for stronger statements. In particular, there is no approximation scheme for the M IN IMUM V ERTEX C OVER P ROBLEM (unless P = NP). We shall prove MAXSNPhardness of other problems in later chapters; see also Exercise 22.

Exercises 1. Formulate a 2-factor approximation algorithm for the following problem. Given a digraph with edge weights, find an acyclic subgraph of maximum weight. Note: No k-factor approximation algorithm for this problem is known for k < 2. 2. The k-C ENTER P ROBLEM is defined as follows: given an undirected graph G, weights c : E(G) → R+ , and a number k ∈ N with k ≤ |V (G)|, find a set X ⊆ V (G) of cardinality k such that max min dist(v, x)

v∈V (G) x∈X

is minimum. As usual we denote the optimum value by OPT(G, c, k).

Exercises

∗

3. 4. 5.

6.

∗

7.

431

(a) Let S be a maximal stable set in (V (G), {{v, w} : dist(v, w) ≤ 2R}). Show that then OPT(G, c, |S| − 1) > R. (b) Use (a) to describe a 2-factor approximation algorithm for the k-C ENTER P ROBLEM. (Hochbaum and Shmoys [1985]) (c) Show that there is no r -factor approximation algorithm for the k-C ENTER P ROBLEM for any r < 2. Hint: Use Exercise 12 of Chapter 15. (Hsu and Nemhauser [1979]) Can one find a minimum vertex cover (or a maximum stable set) in a bipartite graph in polynomial time? Show that the performance guarantee in Theorem 16.5 is tight. Show that the following is a 2-factor approximation algorithm for the M INIMUM V ERTEX C OVER P ROBLEM: compute a DFS-tree and output all its vertices with nonzero out-degree. (Bar-Yehuda [unpublished]) Show that the LP relaxation min{cx : M > x ≥ 1l, x ≥ 0} of the M INIMUM W EIGHT V ERTEX C OVER P ROBLEM, where M is the incidence matrix of an V (G) undirected graph and c ∈ R+ , always has a half-integral optimum solution 1 (i.e. one with entries 0, 2 , 1 only). Derive another 2-factor approximation algorithm from this fact. Consider the M INIMUM W EIGHT F EEDBACK V ERTEX S ET P ROBLEM: given an undirected graph G and weights c : V (G) → R+ , find a vertex set X ⊆ V (G) of minimum weight such that G − X is a forest. Consider the following recursive algorithm A: If E(G) = ∅, then return A(G, c) := ∅. If |δG (x)| ≤ 1 for some x ∈ V (G), then return A(G, c) := A(G − x, c). If c(x) = 0 for some x ∈ V (G), then return A(G, c) := {x} ∪ A(G − x, c). Otherwise let :=

c(v) x∈V (G) |δ(v)| min

and c0 (v) := c(v) − |δ(v)| (v ∈ V (G)). Let X := A(G, c0 ). For each x ∈ X do: If G − (X \ {x}) is a forest, then set X := X \ {x}. Return A(G, c) := x. Prove that this is a 2-factor approximation algorithm for the M INIMUM W EIGHT F EEDBACK V ERTEX S ET P ROBLEM. (Becker and Geiger [1996]) 8. Show that the M AXIMUM C UT P ROBLEM is NP-hard even for simple graphs. 9. Prove that the simple greedy algorithm for M AX -C UT described at the beginning of Section 16.2 is a 2-factor approximation algorithm. 10. Consider the following local search algorithm for the M AXIMUM C UT P ROB LEM . Start with any nonempty proper subset S of V (G). Now check iteratively if some vertex can be added to S or deleted from S such that |δ(S)| increases. Stop if no such improvement is possible. (a) Prove that the above is a 2-factor approximation algorithm. (Recall Exercise 10 of Chapter 2.)

432

16 Approximation Algorithms

(b) Can the algorithm be extended to the M AXIMUM W EIGHT C UT P ROBLEM, where we have nonnegative edge weights? (c) Does the algorithm always find an optimum solution for planar graphs, or for bipartite graphs? For both classes there is a polynomial-time algorithm (Exercise 7 of Chapter 12 and Proposition 2.27). 11. In the D IRECTED M AXIMUM W EIGHT C UT P ROBLEM we are given a digraph G with P weights c : E(G) → R+ , and we look for a set X ⊆ V (G) such that e∈δ + (X ) c(e) is maximum. Show that there is a 4-factor approximation algorithm for this problem. Hint: Use Exercise 10. Note: There is a 1.165-factor but no 1.09-factor approximation algorithm unless P = NP (Feige and Goemans [1995], H˚astad [2001]). 12. Show that ( π1 arccos(yi> y j ))1≤i, j≤n is a convex combination of cut semimetrics δ R , R ⊆ {1, . . . , n}, where δi,R j = 1 if |R ∩ {i, j}| = 1 and δi,R j = 0 otherwise. Hint: Write ! X [ \ µ H (yi ) \ H (yi ) δ R . (µ(H (yi )4H (y j )))1≤i, j≤n = R⊆{1,...,n}

i∈R

i ∈R /

Note: See Deza and Laurent [1997] for a lot of related information. 13. Show that for each n ∈ N there is a bipartite graph on 2n vertices for which the G REEDY C OLOURING A LGORITHM needs n colours. So the algorithm may give arbitrarily bad results. However, show that there always exists an order of the vertices for which the algorithm finds an optimum colouring. √ 14. Show that one can colour any 3-colourable graph G with at most 2 2n colours in polynomial time, where n := |V (G)|. √ Hint: As long as there is a vertex v of degree at least 2n, colour 0(v) optimally with at most two colours (not to be used again), and delete these vertices. Finally use the G REEDY C OLOURING A LGORITHM. (Wigderson [1983]) 15. Show that the following graphs are perfect: (a) bipartite graphs; (b) interval graphs: ({v1 , . . . , vn }, {{vi , v j } : i 6= j, [ai , bi ] ∩ [a j , b j ] 6= ∅}), where [a1 , b1 ], . . . , [an , bn ] is a set of closed intervals; (c) chordal graphs (see Exercise 28 of Chapter 8). ∗ 16. Let G be an undirected graph. Prove that the following statements are equivalent: (a) G is perfect. (b) For any weight function c : V (G) → Z+ the maximum weight of a clique in G equals the minimum number of stable sets such that each vertex v is contained in c(v) of them. (c) For any weight function c : V (G) → Z+ the maximum weight of a stable set in G equals the minimum number of cliques such that each vertex v is contained in c(v) of them. (d) The inequality system defining (16.3) is TDI.

Exercises

433

(e) The clique polytope of G, i.e. the convex hull of the incidence vectors of all cliques in G, is given by ( ) X V (G) : xv ≤ 1 for all stable sets S in G . (16.9) x ∈ R+ v∈S

(f) The inequality system defining (16.9) is TDI. Note: The polytope (16.9) is called the antiblocker of the polytope (16.3). 17. An instance of M AX -S AT is called k-satisfiable if any k of its clauses can be simultaneously satisfied. Let rk be the infimum of the fraction of clauses that one can satisfy in any k-satisfiable instance. (a) Prove that r1 = 21 . √

18.

19.

20.

21. 22.

(b) Prove that r2 = 5−1 2 . (Hint: Some variables occur in one-element clauses (w.l.o.g. all oneelement clauses are positive), set them true with probability a (for some 1 1 2 < a < 1), and set the other variables true with probability 2 . Apply the derandomization technique and choose a appropriately.) (c) Prove that r3 ≥ 23 . (Lieberherr and Specker [1981]) Erd˝os [1967] showed the following: for each constant k ∈ N, the (asymptotically) best fraction of the edges that we can guarantee to be in a maximum cut is 12 , even if we restrict attention to graphs without odd circuits of length k or less. (Compare Exercise 10(a).) (a) What about k = ∞? (b) Show how the M AXIMUM C UT P ROBLEM can be reduced to M AX -S AT. Hint: Use a variable for each vertex and two clauses {x, y}, {x, ¯ y¯ } for each edge {x, y}. (c) Use (b) and Erd˝os’ Theorem in order to prove that rk ≤ 34 for all k. (For a definition of rk , see Exercise 17.) Note: Trevisan [2004] proved that limk→∞ rk = 34 . Prove that the error probability 12 in Definition 16.31 can be replaced equivalently by any number between 0 and 1. Deduce from this (and the proof of Theorem 16.34) that there is no ρ-factor approximation algorithm for the M AXIMUM C LIQUE P ROBLEM for any ρ ≥ 1 (unless P = NP). Prove that the M AXIMUM C LIQUE P ROBLEM is L-reducible to the S ET PACK ING P ROBLEM : given a set system (U, S), find a maximum cardinality subfamily R ⊆ S whose elements are pairwise disjoint. Prove that the M INIMUM V ERTEX C OVER P ROBLEM has no absolute approximation algorithm (unless P = NP). Prove that M AX -2S AT is MAXSNP-hard. Hint: Use Corollary 16.45. (Papadimitriou and Yannakakis [1991])

434

16 Approximation Algorithms

References General Literature: Asano, T., Iwama, K., Takada, H., and Yamashita, Y. [2000]: Designing high-quality approximation algorithms for combinatorial optimization problems. IEICE Transactions on Communications/Electronics/Information and Systems E83-D (2000), 462–478 Ausiello, G., Crescenzi, P., Gambosi, G., Kann, V., Marchetti-Spaccamela, A., and Protasi, M. [1999]: Complexity and Approximation: Combinatorial Optimization Problems and Their Approximability Properties. Springer, Berlin 1999 Garey, M.R., and Johnson, D.S. [1979]: Computers and Intractability; A Guide to the Theory of NP-Completeness. Freeman, San Francisco 1979, Chapter 4 Hochbaum, D.S. [1996]: Approximation Algorithms for NP-Hard Problems. PWS, Boston, 1996 Horowitz, E., and Sahni, S. [1978]: Fundamentals of Computer Algorithms. Computer Science Press, Potomac 1978, Chapter 12 Shmoys, D.B. [1995]: Computing near-optimal solutions to combinatorial optimization problems. In: Combinatorial Optimization; DIMACS Series in Discrete Mathematics and Theoretical Computer Science 20 (W. Cook, L. Lov´asz, P. Seymour, eds.), AMS, Providence 1995 Papadimitriou, C.H. [1994]: Computational Complexity, Addison-Wesley, Reading 1994, Chapter 13 Vazirani, V.V. [2001]: Approximation Algorithms. Springer, Berlin, 2001

Cited References: Ajtai, M. [1994]: Recursive construction for 3-regular expanders. Combinatorica 14 (1994), 379–416 Alizadeh, F. [1995]: Interior point methods in semidefinite programming with applications to combinatorial optimization. SIAM Journal on Optimization 5 (1995), 13–51 Appel, K., and Haken, W. [1977]: Every planar map is four colorable; Part I; Discharging. Illinois Journal of Mathematics 21 (1977), 429–490 Appel, K., Haken, W., and Koch, J. [1977]: Every planar map is four colorable; Part II; Reducibility. Illinois Journal of Mathematics 21 (1977), 491–567 Arora, S. [1994]: Probabilistic checking of proofs and the hardness of approximation problems, Ph.D. thesis, U.C. Berkeley, 1994 Arora, S., Lund, C., Motwani, R., Sudan, M., and Szegedy, M. [1998]: Proof verification and hardness of approximation problems. Journal of the ACM 45 (1998), 501–555 Arora, S., and Safra, S. [1998]: Probabilistic checking of proofs. Journal of the ACM 45 (1998), 70–122 Asano, T. [2006]: An improved analysis of Goemans and Williamson’s LP-relaxation for MAX SAT. Theoretical Computer Science 354 (2006), 339–353 Bar-Yehuda, R., and Even, S. [1981]: A linear-time approximation algorithm for the weighted vertex cover problem. Journal of Algorithms 2 (1981), 198–203 Becker, A., and Geiger, D. [1996]: Optimization of Pearl’s method of conditioning and greedy-like approximation algorithms for the vertex feedback set problem. Artificial Intelligence Journal 83 (1996), 1–22 Bellare, M., and Sudan, M. [1994]: Improved non-approximability results. Proceedings of the 26th Annual ACM Symposium on the Theory of Computing (1994), 184–193

References

435

Bellare, M., Goldreich, O., and Sudan, M. [1998]: Free bits, PCPs and nonapproximability – towards tight results. SIAM Journal on Computing 27 (1998), 804–915 Berge, C. [1961]: F¨arbung von Graphen, deren s¨amtliche bzw. deren ungerade Kreise starr sind. Wissenschaftliche Zeitschrift, Martin Luther Universit¨at Halle-Wittenberg, Mathematisch-Naturwissenschaftliche Reihe (1961), 114–115 Berge, C. [1962]: Sur une conjecture relative au probl`eme des codes optimaux. Communication, 13`eme assembl´ee g´en´erale de l’URSI, Tokyo 1962 Berman, P., and Fujito, T. [1999]: On approximation properties of the independent set problem for low degree graphs. Theory of Computing Systems 32 (1999), 115–132 Brooks, R.L. [1941]: On colouring the nodes of a network. Proceedings of the Cambridge Philosophical Society 37 (1941), 194–197 Chen, J., Friesen, D.K., and Zheng, H. [1999]: Tight bound on Johnson’s algorithm for maximum satisfiability. Journal of Computer and System Sciences 58 (1999), 622–640 Chleb´ık, M. and Chleb´ıkov´a, J. [2006]: Complexity of approximating bounded variants of optimization problems. Theoretical Computer Science 354 (2006), 320-338 Chudnovsky, M., Cornu´ejols, G., Liu, X., Seymour, P., and Vu˘skovi´c, K. [2005]: Recognizing Berge graphs. Combinatorica 25 (2005), 143–186 Chudnovsky, M., Robertson, N., Seymour, P., and Thomas, R. [2006]: The strong perfect graph theorem. Annals of Mathematics 164 (2006), 51–229 Chv´atal, V. [1975]: On certain polytopes associated with graphs. Journal of Combinatorial Theory B 18 (1975), 138–154 Chv´atal, V. [1979]: A greedy heuristic for the set cover problem. Mathematics of Operations Research 4 (1979), 233–235 Clementi, A.E.F., and Trevisan, L. [1999]: Improved non-approximability results for minimum vertex cover with density constraints. Theoretical Computer Science 225 (1999), 113–128 Deza, M.M., and Laurent, M. [1997]: Geometry of Cuts and Metrics. Springer, Berlin 1997 Dinur, I. [2007]: The PCP theorem by gap amplification. Journal of the ACM 54 (2007), Article 12 Dinur, I., and Safra, S. [2002]: On the hardness of approximating minimum vertex cover. Annals of Mathematics 162 (2005), 439–485 Erd˝os, P. [1967]: On bipartite subgraphs of graphs. Mat. Lapok. 18 (1967), 283–288 Feige, U. [1998]: A threshold of ln n for the approximating set cover. Journal of the ACM 45 (1998), 634–652 Feige, U. [2004]: Approximating maximum clique by removing subgraphs. SIAM Journal on Discrete Mathematics 18 (2004), 219–225 Feige, U., and Goemans, M.X. [1995]: Approximating the value of two prover proof systems, with applications to MAX 2SAT and MAX DICUT. Proceedings of the 3rd Israel Symposium on Theory of Computing and Systems (1995), 182–189 Feige, U., Goldwasser, S., Lov´asz, L., Safra, S., and Szegedy, M. [1996]: Interactive proofs and the hardness of approximating cliques. Journal of the ACM 43 (1996), 268–292 Fern´andez-Baca, D., and Lagergren, J. [1998]: On the approximability of the Steiner tree problem in phylogeny. Discrete Applied Mathematics 88 (1998), 129–145 Fulkerson, D.R. [1972]: Anti-blocking polyhedra. Journal of Combinatorial Theory B 12 (1972), 50–71 F¨urer, M., and Raghavachari, B. [1994]: Approximating the minimum-degree Steiner tree to within one of optimal. Journal of Algorithms 17 (1994), 409–423 Garey, M.R., and Johnson, D.S. [1976]: The complexity of near-optimal graph coloring. Journal of the ACM 23 (1976), 43–49

436

16 Approximation Algorithms

Garey, M.R., Johnson, D.S., and Stockmeyer, L. [1976]: Some simplified NP-complete graph problems. Theoretical Computer Science 1 (1976), 237–267 Goemans, M.X., and Williamson, D.P. [1994]: New 3/4-approximation algorithms for the maximum satisfiability problem. SIAM Journal on Discrete Mathematics 7 (1994), 656–666 Goemans, M.X., and Williamson, D.P. [1995]: Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming. Journal of the ACM 42 (1995), 1115–1145 Gr¨otschel, M., Lov´asz, L., and Schrijver, A. [1988]: Geometric Algorithms and Combinatorial Optimization. Springer, Berlin 1988 Halld´orsson, M.M., and Radhakrishnan, J. [1997]: Greed is good: approximating independent sets in sparse and bounded degree graphs. Algorithmica 18 (1997), 145–163 H˚astad, J. [2001]: Some optimal inapproximability results. Journal of the ACM 48 (2001), 798–859 Heawood, P.J. [1890]: Map colour theorem. Quarterly Journal of Pure Mathematics 24 (1890), 332–338 Hochbaum, D.S. [1982]: Approximation algorithms for the set covering and vertex cover problems. SIAM Journal on Computing 11 (1982), 555–556 Hochbaum, D.S., and Shmoys, D.B. [1985]: A best possible heuristic for the k-center problem. Mathematics of Operations Research 10 (1985), 180–184 Holyer, I. [1981]: The NP-completeness of edge-coloring. SIAM Journal on Computing 10 (1981), 718–720 Hougardy, S., Pr¨omel, H.J., and Steger, A. [1994]: Probabilistically checkable proofs and their consequences for approximation algorithms. Discrete Mathematics 136 (1994), 175–223 Hsu, W.L., and Nemhauser, G.L. [1979]: Easy and hard bottleneck location problems. Discrete Applied Mathematics 1 (1979), 209–216 Johnson, D.S. [1974]: Approximation algorithms for combinatorial problems. Journal of Computer and System Sciences 9 (1974), 256–278 Khanna, S., Linial, N., and Safra, S. [2000]: On the hardness of approximating the chromatic number. Combinatorica 20 (2000), 393–415 Knuth, D.E. [1969]: The Art of Computer Programming; Vol. 2. Seminumerical Algorithms. Addison-Wesley, Reading 1969 (third edition: 1997) ¨ K¨onig, D. [1916]: Uber Graphen und ihre Anwendung auf Determinantentheorie und Mengenlehre. Mathematische Annalen 77 (1916), 453–465 Lieberherr, K., and Specker, E. [1981]: Complexity of partial satisfaction. Journal of the ACM 28 (1981), 411–421 Lov´asz, L. [1972]: Normal hypergraphs and the perfect graph conjecture. Discrete Mathematics 2 (1972), 253–267 Lov´asz, L. [1975]: On the ratio of optimal integral and fractional covers. Discrete Mathematics 13 (1975), 383–390 Lov´asz, L. [1979a]: On the Shannon capacity of a graph. IEEE Transactions on Information Theory 25 (1979), 1–7 Lov´asz, L. [1979b]: Graph theory and integer programming. In: Discrete Optimization I; Annals of Discrete Mathematics 4 (P.L. Hammer, E.L. Johnson, B.H. Korte, eds.), North-Holland, Amsterdam 1979, pp. 141–158 Lov´asz, L. [2003]: Semidefinite programs and combinatorial optimization. In: Recent Advances in Algorithms and Combinatorics (B.A. Reed, C.L. Sales, eds.), Springer, New York (2003), pp. 137–194

References

437

Mahajan, S., and Ramesh, H. [1999]: Derandomizing approximation algorithms based on semidefinite programming. SIAM Journal on Computing 28 (1999), 1641–1663 Papadimitriou, C.H., and Steiglitz, K. [1982]: Combinatorial Optimization; Algorithms and Complexity. Prentice-Hall, Englewood Cliffs 1982, pp. 406–408 Papadimitriou, C.H., and Yannakakis, M. [1991]: Optimization, approximation, and complexity classes. Journal of Computer and System Sciences 43 (1991), 425–440 Papadimitriou, C.H., and Yannakakis, M. [1993]: The traveling salesman problem with distances one and two. Mathematics of Operations Research 18 (1993), 1–12 Raghavan, P., and Thompson, C.D. [1987]: Randomized rounding: a technique for provably good algorithms and algorithmic proofs. Combinatorica 7 (1987), 365–374 Raz, R., and Safra, S. [1997]: A sub constant error probability low degree test, and a sub constant error probability PCP characterization of NP. Proceedings of the 29th Annual ACM Symposium on Theory of Computing (1997), 475–484 Robertson, N., Sanders, D.P., Seymour, P., and Thomas, R. [1996]: Efficiently four-coloring planar graphs. Proceedings of the 28th Annual ACM Symposium on the Theory of Computing (1996), 571–575 Robertson, N., Sanders, D.P., Seymour, P., and Thomas, R. [1997]: The four colour theorem. Journal of Combinatorial Theory B 70 (1997), 2–44 Sanders, P., and Steurer, D. [2005]: An asymptotic approximation scheme for multigraph edge coloring. Proceedings of the 16th Annual ACM-SIAM Symposium on Discrete Algorithms (2005), 897–906 Singh, M. and Lau, L.C. [2007]: Approximating minimum bounded degree spanning trees to within one of optimal. Proceedings of the 39th Annual ACM Symposium on Theory of Computing (2007), 661–670 Slav´ık, P. [1997]: A tight analysis of the greedy algorithm for set cover. Journal of Algorithms 25 (1997), 237–254 Stockmeyer, L.J. [1973]: Planar 3-colorability is polynomial complete. ACM SIGACT News 5 (1973), 19–25 Trevisan, L. [2004]: On local versus global satisfiability. SIAM Journal on Discrete Mathematics 17 (2004), 541–547 Vizing, V.G. [1964]: On an estimate of the chromatic class of a p-graph. Diskret. Analiz 3 (1964), 23–30 [in Russian] Wigderson, A. [1983]: Improving the performance guarantee for approximate graph coloring. Journal of the ACM 30 (1983), 729–735 Yannakakis, M. [1994]: On the approximation of maximum satisfiability. Journal of Algorithms 17 (1994), 475–502 Zuckerman, D. [2006]: Linear degree extractors and the inapproximability of Max Clique and Chromatic Number. Proceedings of the 38th Annual ACM Symposium on Theory of Computing (2006), 681–690

17 The Knapsack Problem

The M INIMUM W EIGHT P ERFECT M ATCHING P ROBLEM and the W EIGHTED M ATROID I NTERSECTION P ROBLEM discussed in earlier chapters are among the “hardest” problems for which a polynomial-time algorithm is known. In this chapter we deal with the following problem which turns out to be, in a sense, the “easiest” NP-hard problem:

K NAPSACK P ROBLEM Instance: Task:

Nonnegative integers n, c1 , . . . , cn , w1 , . . . , wn and W . P P Find a subset S ⊆ {1, . . . , n} such that j∈S w j ≤ W and j∈S c j is maximum.

Applications arise whenever we want to select an optimum subset of bounded weight from a set of elements each of which has a weight and a profit. We start by considering the fractional version in Section 17.1, which turns out to be solvable in linear time. The integral knapsack problem is NP-hard as shown in Section 17.2, but a pseudopolynomial algorithm solves it optimally. Combined with a rounding technique this can be used to design a fully polynomial approximation scheme, which is the subject of Section 17.3.

17.1 Fractional Knapsack and Weighted Median Problem We consider the following problem:

F RACTIONAL K NAPSACK P ROBLEM Instance: Task:

Nonnegative integers n, c1 , . . . , cn , w1 , . . . , wn and W . P Find numbers x1 , . . . , xn ∈ [0, 1] such that nj=1 x j w j ≤ W and Pn j=1 x j c j is maximum.

The following observation suggests a simple algorithm which requires sorting the elements appropriately: Proposition 17.1. (Dantzig Pn [1957]) Let c1 , . . . , cn , w1 , . . . , wn and W be nonwi > W and negative integers with i=1

440

17 The Knapsack Problem

c2 cn c1 ≥ ≥ ··· ≥ , w1 w2 wn and let k := min

j ∈ {1, . . . , n} :

j X

wi > W

i=1

.

Then an optimum solution of the given instance of the F RACTIONAL K NAPSACK P ROBLEM is defined by xj

xk := xj

for j = 1, . . . , k − 1,

:= 1 W−

Pk−1 j=1

wj

wk

:= 0

, for j = k + 1, . . . , n.

Sorting the elements takes O(n log n) time (Theorem 1.5), and computing k can be done in O(n) time by simple linear scanning. Although this algorithm is quite fast, one can do even better. Observe that the problem reduces to a weighted median search: Definition P 17.2. Let n ∈ N, z 1 , . . . , z n ∈ R, w1 , . . . , wn ∈ R+ and W ∈ R with n 0 < W ≤ i=1 wi . Then the (w1 , . . . , wn ; W )-weighted median with respect to (z 1 , . . . , z n ) is defined to be the unique number z ∗ for which X X wi < W ≤ wi . i:z i z m for i = l + 1, . . . , n.

4

If

k X i=1

If

l X

wi < W ≤

l X

wi then stop (z ∗ := z m ).

i=1 l X wi < W then find recursively the wl+1 , . . . , wn ; W − wi -

i=1

i=1

weighted median with respect to (zl+1 , . . . , z n ). Stop. k X wi ≥ W then find recursively the w1 , . . . , wk ; W -weighted If i=1

median with respect to (z 1 , . . . , z k ). Stop. Theorem 17.3. The W EIGHTED M EDIAN A LGORITHM works correctly and takes O(n) time only. Proof: The correctness is easily checked. Let us denote the worst-case running time for n elements by f (n). We obtain l m l n m 1 n 1 ln m f (n) = O(n) + f + O(n) + f 5+ 2 , 5 2 5 2 5 because the recursive call in 4 omits at least three elements out of at least half of the five-element blocks. The above recursion formula yields as f (n) = O(n): n 9 9 7 9 5 ≤ 41 n for all n ≥ 37, one obtains f (n) ≤ cn + f 41 n + f 2 41 n for a suitable c and n ≥ 37. Given this, f (n) ≤ (82c + f (36))n can be verified easily by induction. So indeed the overall running time is linear. We immediately obtain the following corollaries: Corollary 17.4. (Blum et al. [1973]) The S ELECTION P ROBLEM can be solved in O(n) time. Proof: Set wi := 1 for i = 1, . . . , n and W := k and apply Theorem 17.3.

Corollary 17.5. The F RACTIONAL K NAPSACK P ROBLEM can be solved in linear time.

442

17 The Knapsack Problem

Proof: Setting z i := − wcii (i = 1, . . . , n) reduces the F RACTIONAL K NAPSACK P ROBLEM to the W